Getting Data In

How to customise value of _time from event data at index-time field extraction?

imahadevia_splu
Splunk Employee
Splunk Employee

I am trying to extract following data, and I want the date which is in EVENT tab as default TIME field which is extracted by _time.

Sample data:

2012-02-03 20:11:56 SampleClass3 [INFO] everything normal for id 530537821
2012-02-03 20:11:56 SampleClass3 [TRACE] verbose detail for id 1718828806
2012-02-03 20:11:56 SampleClass8 [DEBUG] detail for id 2083681507

Current Output:

alt text

I have tried using different time formats in my prpos.conf but it didn't work for me. My current props.conf is as follows :

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE

There has been a lot of Q&As about _time but I have not found any definitive answers. Any help is appreciated!

Thank You

1 Solution

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

View solution in original post

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

thomasroulet
Path Finder

Hello,

because the date 2012-02-03 20:11:56
is too far in the past (more than 7 years) you have to add a parameter in your props.conf

MAX_DAYS_AGO = 3650

you have to adjust the value of the parameter.
Default: 2000 (5.48 years).

edit:
you may have to edit the frozenTimePeriodInSecs parameter in your indexes.conf
The default value is 188697600 (6 years).

Get Updates on the Splunk Community!

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...