Alerting

How to stop alerts from being generated during maintenance?

Rakzskull
Path Finder

I've seen a few posts on the subject, but I'd like to know how we can disable the multiple alerts throughout the maintenance window.

For example, I'd like to disable alerts 1, 2, and 3 from Saturday 11:30 p.m. until Sunday 6:00 a.m.

Thank you in advance.

------------------------------------

reference alert query

index=ABC sourcetype=XYZ ("Internal System Error")
|stats count
|where count >=30

Labels (4)
Tags (1)
0 Karma

Rakzskull
Path Finder

@gcusello 
I'm a rookie, so I don't know much about creating lookup csv. If you could explain the detailed technique with steps, I'd appreciate it. 🙂 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Rakzskull,

if you don't know how to create a lookup I hint to follow the Splunk Search Tutorial (https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchTutorial/WelcometotheSearchTutorial)

Anyway, you have to:

  • go in [Settings -- Lookup -- Lookup table files -- Lookup table files] and create the lookup with one column and one row
  • go in [Settings -- Lookup -- Lookup table files -- Lookup definitions] and create a definition for the lookup

Ciao.

Giuseppe

 

0 Karma

chaker
Contributor

It could also be done via the REST API:
https://community.splunk.com/t5/Alerting/How-do-you-disable-enable-alerts-via-the-REST-API/m-p/44155...

There is also a good suggestion here to group the alerts by app, then disable the app:

https://community.splunk.com/t5/Alerting/How-can-we-suppress-a-set-of-alerts/m-p/480144

 

Need to add more points to this idea:  +4 from me 😁

https://ideas.splunk.com/ideas/PLECID-I-297

 

gcusello
SplunkTrust
SplunkTrust

Hi @Rakzskull,

if they are few, the easiest way it to manually disable them during maintenence period.

If you want to disable all the alert and you haven't scheduled reports or dashboards, you could disable the eMail configuration, so the alerts are triggered but the emails aren't sent.

There's a more elegant way, but it requires a little bit of work:

  • create a lookup (called e.g. maintenance.csv) containing only one columns (e.g. maintenance) and only two values (yes/not),
  • in each alert add the condition maintenance=not.
  • In this way, modifying the value in the lookup you stop all the alerts.

This surely is an interesting new feature, I hint to add it to Splunk Ideas.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...