Splunk Search

Best approach for speeding up Splunk Searches

Dark_Ichigo
Builder

Searches with the "Transaction" command for example can be really slow, What would be the best approach for speeding up searches in Splunk in terms of Hardware and Search Optimization.

Note: Please do not advise that I use Sub-searches in any way, I have given up on using them long ago!

dwaddle
SplunkTrust
SplunkTrust

As Kristian demonstrated, sometimes the best way to make transaction operations faster is not to use transaction at all. If your transaction is simple enough, you can express it using stats with a by clause. This is much much faster.

If your transaction is too complex to manage with stats, then you should go out of your way to minimize the number of events dealt with. For example, if you're looking for 3 or 4 events out of 20 that make up a transaction, search only on those 3 or 4, like:

sourcetype=foo  ( event 1 stuff ) OR ( event 2 stuff ) OR ( event 3 stuff ) | transaction

Also, limit the size/scope of your transactions by using startswith / endswith so that transaction can "complete" them as soon as possible.

kristian_kolb
Ultra Champion

In general, try to limit the time span first. And be as specific as possible before the first pipe. The more you can limit the search before the first pipe, the better.

The following searches for unique visits (not visitors) and bytecount should result in a table like:

clientip      visits     sum(bytes) 
10.11.12.13     42         89231
11.12.13.14     32         12212
12.13.14.15     11         12341

BAD:

sourcetype=access_combined | transaction JSESSIONID | search host=www1.company.com | stats count AS visits sum(bytes) by clientip

BETTER:

sourcetype=access_combined host=www1.company.com | transaction JSESSIONID | stats count AS visits sum(bytes) by clientip

BEST:

sourcetype=access_combined host=www1.company.com | stats distinct_count(JSESSIONID) AS visits sum(bytes) by clientip

Hope this helps,

Kristian

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...