Eliminating false positive while monitoring Hot/Cold deplolyment?

[bigll]

Hi.

I am monitoring service status on number of paired servers.

While service is running on server1 report on service stopped on server 2 is a false positive

But if it stopped on server 1 and did not start on server 2 it's a case when I need to be alerted.

Any example of alert logic you can share?

Thank you

[gcusello]

Hi @bigll,

the main problem is to identify the paired hosts:

if you have few paired hosts, you can create a search like this:

<your_search>
| eval paired_host=case(host=host1,"Paired_host_1",host=host2,"Paired_host_1",host=host3,"Paired_host_2",host=host4,"Paired_host_2")
| stats dc(host) AS host_count values(host) AS host BY paired_host
| eval status=if(host_count=2,"Both active","Only ".paired_host." active")

If instead you have many paired host and it's too difficoult to create one search, you have to create a lookup  (called e.g. paired_hosts.csv) containing two fields:

• host
• paired_host

and then run a search like this:

<your_search>
| lookup paired_host.csv host OUTPUT paired_host
| stats dc(host) AS host_count values(host) AS host BY paired_host
| eval status=if(host_count=2,"Both active","Only ".paired_host." active")

In both cases, you can create your alerts or your dashboard.

Ciao.

Giuseppe