conditional index and sourcetype name inputs.conf by file-naming-convention

Question

So my goal is to be able to pass a file to a splunk-monitored directory.. and have splunk apply it to the appropriate index and sourcetype...by a sort of naming convention.

file would come in as "indexname_sourcetype_filename.txt" or whatever... and my inputs.conf would do the appropriate thing.

is this possible?

Accepted Answer

2

Yes, this should be possible. It would be something along these lines:

In transforms.conf:

# transforms.conf
[override-sourcetype]
SOURCE_KEY = source
DEST_KEY = MetaData:Sourcetype
REGEX = \w+_(\w+)_\w+\.txt$
FORMAT = sourcetype::$1

[override-index]
SOURCE_KEY = source
DEST_KEY = _MetaData:Index
REGEX = (\w+)_\w+_\w+\.txt$
FORMAT = index::$1

And then in props.conf:

# props.conf
[source::/var/log/inputdir/*]
TRANSFORMS-sourcetype = override-sourcetype
TRANSFORMS-index = override-index

link

answered 09 Sep '10, 20:43

southeringtonp ♦
4.9k●2●5●24
accept rate: 35%

That is cool. What happens if the index doesnt exist yet?

(10 Sep '10, 13:36) hiddenkirby

and no inputs.conf at all?

(10 Sep '10, 13:37) hiddenkirby

No, configure inputs.conf as normal, and create each destination index ahead of time via indexes.conf or the Manager. The settings above will override the sourcetype and destination index as the file is indexed. Not sure what happens if the index doesn't exist - it will probably throw an indexing error but it might revert back to the default index.

(10 Sep '10, 14:17) southeringtonp ♦

ok so no way to create the index dynamically. thanks for this. :)

(10 Sep '10, 14:48) hiddenkirby

BTW - I was doing this for just the sourcetype override, and needed to change SOURCE_KEY = MetaData:Source in the transforms.conf [override-sourcetype] stanza to get this to work.

(23 Aug '11, 17:47) platform_pie

conditional index and sourcetype name inputs.conf by file-naming-convention

Follow this question

Splunk Resources

Related questions