Getting Data In

How to customise value of _time from event data at index-time field extraction?

imahadevia_splu
Splunk Employee
Splunk Employee

I am trying to extract following data, and I want the date which is in EVENT tab as default TIME field which is extracted by _time.

Sample data:

2012-02-03 20:11:56 SampleClass3 [INFO] everything normal for id 530537821
2012-02-03 20:11:56 SampleClass3 [TRACE] verbose detail for id 1718828806
2012-02-03 20:11:56 SampleClass8 [DEBUG] detail for id 2083681507

Current Output:

alt text

I have tried using different time formats in my prpos.conf but it didn't work for me. My current props.conf is as follows :

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE

There has been a lot of Q&As about _time but I have not found any definitive answers. Any help is appreciated!

Thank You

1 Solution

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

View solution in original post

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

thomasroulet
Path Finder

Hello,

because the date 2012-02-03 20:11:56
is too far in the past (more than 7 years) you have to add a parameter in your props.conf

MAX_DAYS_AGO = 3650

you have to adjust the value of the parameter.
Default: 2000 (5.48 years).

edit:
you may have to edit the frozenTimePeriodInSecs parameter in your indexes.conf
The default value is 188697600 (6 years).

Get Updates on the Splunk Community!

Join Us at the Builder Bar at .conf24 – Empowering Innovation and Collaboration

What is the Builder Bar? The Builder Bar is more than just a place; it's a hub of creativity, collaboration, ...

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...