Hi guys,
I need to extract headers from a log file, so that when it is pushed to the Indexer, those headers will be displayed.
The log file looks like this.
#Software: IIS Advanced Logging Module
#Version: 1.0
#Start-Date: 2014-11-11 00:00:00.210
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes TimeTakenMS
2014-11-11 00:00:03.283 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:03.736 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:08.291 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:08.728 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:13.299 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:13.751 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:18.306 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:18.759 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 62
2014-11-11 00:00:23.064 172.18.10.88 GET /announce/6mBill-result.html - 80 - "69.191.211.202" "BLP_bbot/0.1" - 301 0 0 257 270 0
2014-11-11 00:00:23.314 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:23.579 172.18.10.88 GET /News/T-Bill-Announcements.aspx - 80 - "69.191.211.202" "BLP_bbot/0.1" "http://www.sgs.gov.sg/announce/6mBill-result.html" 200 0 0 24830 493 62
2014-11-11 00:00:23.766 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 62
2014-11-11 00:00:28.337 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 62
2014-11-11 00:00:28.665 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:33.329 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:33.673 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 46
2014-11-11 00:00:38.384 172.18.10.88 GET / - 80 - - - - 200 0 0 0 7 62
As you see, the header line starts at line 4 and line 1-3 contains garbage with some time stamp. I tried putting the props.conf file in my universal forwarder's app, but it does not seem to be extracting the header. My props.conf looks like this.
[demozxc]
FIELD_DELIMITER = \s
FIELD_HEADER_REGEX = #Fields:\s+(.*)
May I know if there is anything I have done incorrectly? I tried putting "HEADER_FIELD_LINE_NUMBER = 4" in the props.conf, but it did not work as well.
... View more