I have log events in the following format:
2016-10-29 13:24:43.310_394 [145-xxxxxxxxxxxxxxxxxx:49] [XXXX_XXXX_MASTER] (DEBUG) Mapping Instrument Code Type
2016-10-29 13:24:43.310_805 [145-xxxxxxxxxxxxxxxxxx:49] [EnrichmentManager] (INFO) &CC_CURRENCY.process(): table lookup failed - no enrichment performed
where '_394' and '_805' are the microsecond values.
So to parse the input, I've configured my props.conf as follows:
[efx_gfixappia_ulbridgelog_pfseq]
SEDCMD-timefix=s/_//
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%6N
SHOULD_LINEMERGE = false
TZ_ALIAS = HKT=GMT+08:00
Now when I try sorting the events accoring to _time, it gets sorted by the millisecond but the events remain unsorted beyond that. On trying to striptime the microsecond value, it gets displayed as 13:24:43.310000.
I tried removing the sed command in props.conf, and removed the underscores in the logfile itself (in case the timestamp was being extracted before sed), to no avail.
Any suggestions on what I need to fix/should look at to fix this issue?
... View more