Splunk Search

Summary stats from distinct (overflowing) counters.

PickleRick
SplunkTrust
SplunkTrust

Hello.

I have a set of hosts which send some stats. In my case these are rsyslog impstats statistics but it can be anything - for example SNMP interface counters.

The point is that I have a counter which increases with time and I want to compute incremental statistics.

Yes, I know you'll point me towards delta command but it can only count difference from one even to another and I have several different sources for which I need separate stat. (let's say something like | delta <parameter> by host - unfortunately there's no such command ;-)).

After some poking around it seems that range() statistical function seems to fit nicely - it calculates - as the name implies - a range  between lowest and highest  value of the given field so if I pair it with timechart it works beautifuly. Almost.

The problem is that the counters have finite length and after some time overflow back to 0.  And if this happens... of course the range() command returns some ridiculous values.

If it was a simple delta calculation, I'd probably just do some modulo operation or some other conditional eval to account for it but I don't see a reasonable way to do it with already summed up  values since even the field names of the summary table are variable and depend on host names and I can't know the list of hosts beforehand.

Is there any reasonable way to filter out the "overflowed" values? Just using "outliers" removes also "bottom" values which is not what I need.

Labels (1)
Tags (3)
0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...