Knowledge Management

summary indexing created multi value in orig_host entries

ualbanytech
Path Finder

I'm using Splunk 4.1.6 and getting started with creating summary data.

Edit: What I'm trying to do is eliminate fields I don't need when writing to a summary index. I originally tried using the fields command but, I continued to see fields I did not specify in my summary index.

I created a scheduled search which runs daily for "yesterday" and writes to a summary index.

The search (I have replaced my real host names with <hostA_3>, <hostA_4>, <hostB_5>, <hostB_7>😞

splunk_server=splunk-uad* index=uad-* host=<hostB>* OR host=<hostA>* sourcetype=access_combined_rsptime NOT netid="-" | dedup netid, clientip, host |sort - _time|  stats values(host) AS host by _time, req_time, clientip, netid

This went fine.

However when I started playing with some reports against this summary data I noticed that I seemed to get 2 sporadic events out of 9,998 whose "orig_host" value is mangled.

orig_host="<hostA_3>.itsli.albany.edu <hostB_5>.itsli.albany.edu"
orig_host="<hostA_4>.itsli.albany.edu <hostB_7>.itsli.albany.edu"

It seems splunk concatenated two of my host names together for two arbitrary events.

I checked the events the summary events were created from and there is no "host" fields with those bad values in the original events.

Am I doing something subtle/ignorant in my summary search that caused this?

Tags (1)
0 Karma

ualbanytech
Path Finder

I went back and adjusted my search as follows:
index=uad-* host=* OR host=* sourcetype=access_combined_rsptime NOT netid="-" | dedup netid, clientip, host | stats values(netid) AS netid by clientip, host

I decided I don't care about the time since req time is irrelevant for what I'm trying to answer with my sum. data. Uniq users on our hosts per day.

Still not sure why I couldn't control the fields being written to summary index with the fields command.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

your stats is split by _time, req_time, clientip, and netid. If you happen to have two entries with identical values for those four, which is possible (and is in fact what probably happened, for two sets of those values), then all values of host that have the same value set will be made into a multi-valued field with a list of the values. That's what the values() function does and is supposed to do. If that's not what you want, then you might want to use first() or last() instead.

0 Karma

ualbanytech
Path Finder

Thanks for the clarification. I modified my question to be a little clearer. I had never noticed that multi value behavior in any results I've examined previously so it threw me.

Really just want to eliminate unwanted fields when creating summary data.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...