Alerting

How to stop alerts from being generated during maintenance?

Rakzskull
Path Finder

I've seen a few posts on the subject, but I'd like to know how we can disable the multiple alerts throughout the maintenance window.

For example, I'd like to disable alerts 1, 2, and 3 from Saturday 11:30 p.m. until Sunday 6:00 a.m.

Thank you in advance.

------------------------------------

reference alert query

index=ABC sourcetype=XYZ ("Internal System Error")
|stats count
|where count >=30

Labels (4)
Tags (1)
0 Karma

Rakzskull
Path Finder

@gcusello 
I'm a rookie, so I don't know much about creating lookup csv. If you could explain the detailed technique with steps, I'd appreciate it. 🙂 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Rakzskull,

if you don't know how to create a lookup I hint to follow the Splunk Search Tutorial (https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchTutorial/WelcometotheSearchTutorial)

Anyway, you have to:

  • go in [Settings -- Lookup -- Lookup table files -- Lookup table files] and create the lookup with one column and one row
  • go in [Settings -- Lookup -- Lookup table files -- Lookup definitions] and create a definition for the lookup

Ciao.

Giuseppe

 

0 Karma

chaker
Contributor

It could also be done via the REST API:
https://community.splunk.com/t5/Alerting/How-do-you-disable-enable-alerts-via-the-REST-API/m-p/44155...

There is also a good suggestion here to group the alerts by app, then disable the app:

https://community.splunk.com/t5/Alerting/How-can-we-suppress-a-set-of-alerts/m-p/480144

 

Need to add more points to this idea:  +4 from me 😁

https://ideas.splunk.com/ideas/PLECID-I-297

 

gcusello
SplunkTrust
SplunkTrust

Hi @Rakzskull,

if they are few, the easiest way it to manually disable them during maintenence period.

If you want to disable all the alert and you haven't scheduled reports or dashboards, you could disable the eMail configuration, so the alerts are triggered but the emails aren't sent.

There's a more elegant way, but it requires a little bit of work:

  • create a lookup (called e.g. maintenance.csv) containing only one columns (e.g. maintenance) and only two values (yes/not),
  • in each alert add the condition maintenance=not.
  • In this way, modifying the value in the lookup you stop all the alerts.

This surely is an interesting new feature, I hint to add it to Splunk Ideas.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...