Hi,
I have a CSV file with header that is monitored by Splunk. Rows are correctly read, but the headers are also included as an event row. I just want to have the header extracted as the field names (which already works at the same time).
I tried several ideas using props.conf without any success. I also had a look to the similar questions already asked by other users.
My last props.conf looks like:
[mysourcetype]
INDEXED_EXTRACTIONS = csv
HEADER_FIELD_LINE_NUMBER = 1
HEADER_FIELD_DELIMITER = ","
FIELD_DELIMITER = ","
FIELD_HEADER_REGEX = hostname,SCSI logical unit,DeviceID,SCSIBus,SCSIPort,SCSITargetId
I hope someone can help sort this out.
Thanks,
SirHill
I have had no success with the PREAMBLE_REGEX and HEADER_FIELD_LINE_NUMBER clauses... But this solution works: https://answers.splunk.com/answers/206718/how-to-pull-out-a-header-before-indexing.html ---It's a workaround, sadly, but until PREAMBLE_REGEX and HEADER_FIELD_LINE_NUMBER are fixed, that's all we have.
YOU MUST DEPLOY THIS ON YOUR FORWARDER. That is the problem.
Did you tried to insert props.conf in your Forwarder?
Bye.
Giuseppe
Did you deploy this file to your FORWARDER (not your indexers) and did you restart splunkd there?
Hi SirHill17,
to exclude header from indexing you have to insert in your props.conf the following line
PREAMBLE_REGEX = <regex>
This attribute specifies a regular expression which allows Splunk to ignore these preamble lines, based on the pattern specified.
for other information see https://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Propsconf
Bye.
Giuseppe
Try
FIELD_HEADER_REGEX=your_regex
Bye.
Giuseppe
As per my inital question, I already tried that and everything here:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Extractfieldsfromfileswithstructureddata
If none of the previous options correctly runs, you could filter your header in this way:
props.conf
[your_sourcetype]
TRANSFORMS-set-remove_headers=set_OK,set_nullqueue
transforms.conf
[set_nullqueue]
REGEX=your_header_regex
DEST_KEY=queue
FORMAT=nullQueue
[set_OK]
REGEX=.
DEST_KEY = queue
FORMAT = indexQueue
Bye.
Giuseppe
Just tried adding that to my current props.conf but now it index the entire csv as one event (including the header).
props.conf is defined at the indexers level (master-node), not at the forwarder level but I don't think it changes anything.