Can't load my custom fields

Question

I try to index my logs with a custom Source type. Here is my files :

Log file exemple :

#HeaderKey1: header value 1
#HeaderKey2: header value 2
id|severity|date|time|duration|description|uri|operation|collection|lang|format|size|source|errorcode|links|badlinks|collectionid|sourceid|printableuri
...

transforms.conf :

[my_log]
DELIMS="|"
FIELDS = "id", "severity", "date", "time", "duration", "description", "uri", "operation", "collection", "lang", "format", "size", "source", "errorcode", "links", "badlinks", "collectionid", "sourceid", "printableuri"

[my_log_comment]
REGEX = ^#
DEST_KEY = queue
FORMAT = nullQueue

props.conf :

[my_log]
EXTRACT-my_log = my_log_comment, my_log

The result is that all the line are indexed (header include) and the only field i got is _raw... I do not care about the information in the header; I don't want to index it or to take information from it. I only want to have all my log with all the fields.

What is wrong in my configuration? What i should do to index my custom log file correctly

thank you

Accepted Answer

2

TRANSFORMS is for index-time creation of fields and doesn't support extracting fields using delimiters like you want to. bojanz is right however in that you shouldn't be using EXTRACT, as it is used for defining directly in props.conf how fields should be extracted using regexes. For extractions that use definitions in transforms.conf, you should use REPORT.

link

answered 05 Feb, 05:32

Ayn
9.6k●1●3●15
accept rate: 39%

edited 05 Feb, 05:34

Thank you there is my new props.conf [ces_index_log] EXTRACT-ces_index_log = (?<id>d)|(?<severity>w)|(?<date>d{4}-d{2}-d{2})|(?<time>d{2}:d{2}:d{2})|(?<duration>d)|(?<description>[^|])|(?<uri>[^|])|(?<operation>[^|])|(?<collection>[^|])|(?<lang>[^|])|(?<format>[^|])|(?<size>d)|(?<source>[^|])|(?<errorcode>[^|])|(?<links>[^|])|(?<badlinks>[^|])|(?<collectionid>d)|(?<sourceid>d)|(?<printableuri>[^|]*)

(06 Feb, 07:23) apare

Answer 2

-1	Your props.conf file is wrong. It's not EXTRACT but TRANSFORMS. The rest looks ok. link answered 05 Feb, 05:13 bojanz 806●1●10 accept rate: 38%

Can't load my custom fields

Follow this question

Splunk Resources

Related questions