How can I determine how much an index is being searched?

[jamesvz84]

I am trying to determine how many searches are searching on a particular index per day.

I know how much data the index has, but I need to know if people are actually searching on that data or not.

Is there any way to get this information? The searches I have tried only have gotten search name or previously run search strings, but I cannot break it down by index using that information alone (search string/saved search may rely on a default index for the user, which will not be in the search string).

[skillfulobj]

i got the results with below query

index=_audit action="search" search="*" NOT user="splunk-system-user" savedsearch_name="" NOT search="\'|history*" NOT search="\'typeahead*"
| rex "index=(?P<myIndex>\w+)\s+\w+="
| stats count by myIndex

[a212830]

No, I want to know if the index was searched, so we can identify if data is actually being used.

[DalJeanis]

You don't really want to know whether the index was searched, I believe. I think you need to know whether any data was actually returned from the index. When someone runs a "search Index=*" that happens to include an inappropriate index is not something you want to count as "using" that index.

Seems like some of the data returned in the job inspector should be useful here.

Hmmm.

[MuS]

Hi jamesvz84,

Maybe this one is not 100% accurate, but it is a starting point.
Search in the index=_audit for action=search and filter out the saved searches and typeahead or history ones and you can get a pretty nice count on the index= values used in the searches:

index=_audit action="search" search="*" NOT user="splunk-system-user" savedsearch_name="" NOT search="\'|history*" NOT search="\'typeahead*" | rex "index\=(?<myIndex>[^\s,|']*)" | stats count by myIndex

There is one problem with this search, it can return index=* and then you would also need to check the user and its default search index... but, haven't found another way to achieve it yet.

Hope this helps ...

cheers, MuS

[martin_mueller]

This should be achievable with a sprinkling of | rest services/authorization/roles and | rest services/authentication/users - join user to the audit event, join his roles to that, get default and allowed indexes, augment audit events that don't have explicit indexes. Should at least fill all the gaps where the index is not calculated in a subsearch etc.... but I'm going to bed 😛 so this is left as an exercise for the reader.

[a212830]

Anyway to reverse this - get a listing of all indexes, and then run a search for indexes that were not queried?

[sloshburch]

Still gonna get screwed by use of tags/eventtypes/macros which I think abstract that detail from those data points 😞

[martin_mueller]

For an individual job you can determine this from its search.log, look for these lines:

07-14-2015 22:39:25.385 INFO  IndexScopedSearch - 0000000003D1CA60 LISPY for index=main is lispy='[ AND ]' ct=2147483647 et=0 lt=2147483647 dbsize=6

Trouble is, these logs aren't indexed by default so you can't easily run searches against them. You could of course index them yourself, but keep an eye on extra volume - there can be lots and lots of search.log files!