Getting Data In

Query that filters out servers that reboot at the same time every day, but still reports results if reboot happens outside window

rwflowers
New Member

We currently have an alert that shows any time a server is rebooted. We have some servers that reboot at the same time every day. Is there an easy way to filter out those servers for only the time frame that they reboot in, but still report if that same server is rebooted outside that window of time? Here is our current query:
index=winevents (EventCode=4609 OR EventCode=6008 OR EventCode=513 OR EventCode=4608 OR EventCode=1074) | table ComputerName, _time, EventCode, name, user, Message | rename ComputerName AS "Host Name" _time AS "Time" EventCode AS "Event Code" name AS "Event" user AS "Origin Login" Message AS "Reason" | convert timeformat="%Y/%m/%d %H:%M:%S" ctime("Time")

If I insert (host=[servername] earliest!=@d-3 latest!=@d-2) after my index statement and before the event codes, this successfully filters out that servers reboot, but won't give me any results for any other server for any time frame. I have multiple servers I would like to do this for. I know I could create separate alerts for the servers that reboot every night and then just NOT them out of the all inclusive alert, but I would like to just integrate all of them into one alert. Any ideas?

Tags (1)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If your reboot windows are static you could build a lookup like this:

host,reboot_start,reboot_end
rebooting_1,2330,2345
rebooting_2,0100,0115

Only add hosts that have a rebooting window scheduled. Add the lookup to your data to allow searches like this:

index=winevents (blah blah) | eval reboot_time = tonumber(strftime(_time, "%H%M")) | where isnull(reboot_start) OR reboot_time < reboot_start OR reboot_time > reboot_end | table blah

Expressing hours and minutes of the day as numbers and comparing them numerically feels a little hacked, but should work.

If your reboot times are dynamic you could build a search that regularly checks the past week for a reboot time window that happened at least six out of seven days and assume that as "okay" for the next day... that's entirely possible, but a bit more work to build, test, and verify.

0 Karma

linu1988
Champion

splunk wont do it automatically. I am not able to understand the requirement here. Your search is quite correct. If any reboot happens you could only check for 6005 code, if anything is happening outside the time window make an filter with the date_hour <5 AND date_hour >10 with a case statement. Do clarify more.

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...