What options are there to find if a forwarder has stopped sending data to our Splunk server?

[sunnyparmar]

Hi,

Is there any way or any work around or any app through which I can know if Splunk stop receiving data from the forwarders side that have been installed on the machines? Kindly suggest for this please.

Thanks

[ChrisG]

Use the Distributed Management Console. It has a dashboard about deployed forwarders that will tell you whether they are active or missing. See About the Distributed Management Console in the Distributed Management Console Manual.

[hexx]

Note that the DMC also comes with a built in alert to proactively detect missing forwarders - see http://docs.splunk.com/Documentation/Splunk/6.3.0/DMC/Platformalerts

[bevant · Accepted solution]

Something like this:

| metadata type=hosts | eval secs_since_last_saw=now()-lastTime

Will give you a field that is the number of seconds since an event was seen from the host, so a "| where secs_since_last_saw > N" where N is the number of seconds you're willing to wait for them, might be a starting point.

Obviously you don't even need the eval, I just put it there to help explain the point...

[sunnyparmar]

thanks for replying but as i am executing the query it is giving me the data something like below with eval command. With this if i am running with where it is not giving the result so could you please elaborate more here.

firstTime host lastTime recentTime secs_since_last_saw totalCount type
1442244251 10.0.28.1 1442321750 1442321750 3194718 58509793 hosts

thanks

[esix_splunk]

You now need to convert those times from epoch to human readable

.. | convert ctime(firstTime) ctime(lastTime) ctime(recentTime) 

[sunnyparmar]

thanks a ton buddy..