I'm seeing many action=restart_splunkd messages from my "_audit" index. I can tell from my processor status that splunkd is not restarting, yet I'm receiving these messages in my _audit index. Can someone help me understand what produces these messages? Also, how can I tell when splunkd actually did restart?
Thanks.
Hi ,I see this noise in Splunk 8.0.1 also.
Splunk 8.0.5 too.
This is some unfortunate noise from the audit handler. In the future, we hope to improve the audit logging. Genti's answer is correct regarding detecting actual shut downs.
Occurring in 5.0.4, too. Always nice to see the official answer from Genti! (He was here last week helping us)
It is still occurring in version 5.0.3.
Yeap, 2 more bugs submitted regarding the above
Actually, if you notice audit.log will have this message logged every minute, and sometimes more then once per minute. (ie. it sends the action twice - or at least logs it twice)
For real splunkd restart check your splunkd.log (located at /spluhome/var/log/splunk/) for messages like:
10-21-2010 14:40:17.044 INFO loader - Splunkd starting (build 82143).
and
10-21-2010 14:40:13.029 INFO ShutdownHandler - Shutdown complete in 2125.5 milliseconds
It looks like it's still occuring in newer versions, we are currently in 6.4 and still the same problem.
My question is, with your solution above, it's not possible to track which user did launch the restart?
Fast forward to 2019, Splunk 7, the bug is still happening.
One dashboard queries and evals action="restart_splunkd" which causes an Audit:[timestamp=XXX, user=XXX, action=restart_splunkd, info=granted][n/a] log to appear in the _audit index with an audittrail sourcetype (everytime the dashboad is reloaded).
Almost 2023 in Splunk 9.x and it's still an issue...