Sample data:
Audit:[id=, timestamp=07-26-2013 10:45:09.664, user=admin, action=search, info=failed, search_id='1374853508.52', total_run_time=0.08, event_count=0, result_count=0, available_count=0, scan_count=0, drop_count=0, exec_time=1374853508, api_et=N/A, api_lt=N/A, search_et=N/A, search_lt=N/A, is_realtime=0, savedsearch_name=""][NoZeQKz4PV7hFsxbpYbHGu8Uj2E1mvQFIRjoxNsOwRHddn58f3WF/VAPpTxBZzSX4f9BVPn7l0niLbyxPiUKReuy1pfYOZ/iXcMu1GbnypYL5GdJAKV9/gTJWZd4JxapTH2BRqUIIu4asdfewaR1dJXvm+dXNIekM2uKd7utX6t29liScOiDvVn1HN+wHlQX2EoqPJz7NZUrxYa4dpwL4ugooFS8HzVQ/h5MRsLbQl5DU73quBXsabrhafE/aRpRou1TrUbYceqIQ60GA42QtzqNAlovgr6/ni8fTsjIuCOdxRHDhemobvMwpNMZbpM5glXcN+sckLt4MxgDIbBQ==]
Why is the 'id' sub-field blank or null?
Search is: index=_audit
I have many occurrences of this on a non-active (no external to splunk data feeds) instance
I enabled audit signing, shutdown splunk, flushed all the indexes in my development area, restarted splunk.
Same issue (query was: index=_audit | audit):
? Can't validate!
Audit:[id=, timestamp=07-31-2013 09:59:02.407, user=xxxxxxx, action=search, info=granted REST: /search/jobs/1375282742.2887/control]
The signed _audit event is (query: index=_audit):
Audit:[id=, timestamp=07-31-2013 09:59:02.407, user=xxxxxxxx, action=search, info=granted REST: /search/jobs/1375282742.2887/control][Z/Mk8qOQK9oUp9hqksAStp2rTvhqlU6nY7GKi9bVHI7gRtfYlOIRqcm6feGX9kAT0+/T4fREJAzD52aekPlus+mQBYwnOHXPl6Rfft/GWjQcZ53HKoJzeC3Svc/atuAyNxOc67gLt3Bn4E7cg37QssElCWyx+3CZUUP6WNYL7fcoyHzyIdHtO8SAySQNIoxHZ84FUpE1CP/GS35D+hjp7PDQjiQlzOoB/zLOj347Gc6QxESZ6GDPlsaIgS49JDsaPxDS7GlXhmYacPzd4uKuok9Fz3NClKVP532qDdyv7u3RFdhdAyy5fYTOsSfP9ozEoGosaaEVCuISrXpH0EiIDw==]
So this still doesn't explain what is happening. About 1130 events have this problem.
Above is not an answer, just didn't have enough space in the comment field!
Version 5.0.3, Build 163460
What version Splunk did this come from?