I have set up an alert using a "Saved search" in Splunk Enterprise Security. I am throttling alerts for an hour when src, dest, and threat identifier are same. For the most part, this alert is working fine. However, every now and then I see anomalies like the ones below:
Some time, I see that same alerts are fired twice in quick succession.
Other times, I see the same alert fired a second time before Hour limit of throttling is over (e.g. second alert after 30-40 mins of first alert).
The event ID and event hash are different for different alerts. It seems 'notable' macro is assigning a different event_id to the same event. Now, I am wondering how is that possible and how do I fix this?
... View more