Lookups are only indexed over a certain size:
http://splunk-base.splunk.com/answers/8326/are-lookup-tables-indexed
http://splunk-base.splunk.com/answers/10160/at-what-point-do-very-large-lookup-files-csv-get-indexed
I tested using a sorted and then randomized lookup table. The results are below.
Sorted lookup table:
Duration (seconds) Component Invocations Input count Output count
147.425 command.lookup 302 104 104
Randomized lookup table:
Duration (seconds) Component Invocations Input count Output count
199.059 command.lookup 301 104 104
Caveats:
- The original lookup table was created using outputlookup
- The random version was created using sort -R
- For some reason, my Splunk instance has NOT indexed these lookup tables, even though they are 15 MB (over the limit)
- Both files are gzipped
- I ran the test several times, results were always close this output
answered
15 Nov '11, 08:21
supersleepwa...
63●3
accept rate:
40%