Splunk Search

Field Extraction do not work when using the UPLOAD method

Genti
Splunk Employee
Splunk Employee

Customer's issue was actually that for csv files, when setting the CHECK_FOR_HEADER=TRUE in props.conf and when uploading the file using the one time upload button through splunkweb, no automatic field extraction would happen.

I was able to reproduce this in my environment but the issue seems to go even further. When using props.conf to extract fields (at index time, this is no longer a csv-header issue) and then uploading a file, no field extractions happen at all.

Is this the default behavior? Is there any documentation about it?
Is it a bug?

0 Karma
1 Solution

Genti
Splunk Employee
Splunk Employee

Asking the dev's we understand that this is not the default behavior and that something is clearly broken in the code.
The workaround, till this gets fixed, would be not to use file uploading as a means to bring data to splunk if you care for field extractions. If you use regular monitoring stanza, both index-time field extractions as well as header-checking field extractions happen without any issues.

Cheers,
.gz

View solution in original post

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

Another workaround here is to continue to use file uploads, but manually configure the delimiter based extraction for the source or sourcetype. It should be noted that CHECK_FOR_HEADER doesn't perform any magic beyond setting a per-sourcetype search-time field extraction rule. This is easy to achieve for a person after indexing the data. The documentation at http://www.splunk.com/base/Documentation/latest/Admin/Extractfieldsfromfileheadersatindextime shows the configuration that CHECK_FOR_HEADER makes when a new input comes in.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

In many live environments, this is necessary anyway, as CHECK_FOR_HEADER doesn't work if files are collected by a forwarder and sent to an indexer, or if you have a distributed search head separate from your indexer or forwarder.

0 Karma

Genti
Splunk Employee
Splunk Employee

Asking the dev's we understand that this is not the default behavior and that something is clearly broken in the code.
The workaround, till this gets fixed, would be not to use file uploading as a means to bring data to splunk if you care for field extractions. If you use regular monitoring stanza, both index-time field extractions as well as header-checking field extractions happen without any issues.

Cheers,
.gz

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...