Does SEDCMD work line-by-line or on the entire event?

Question

Out of the box, the unix sed command operates on a line-by-line basis. Is this the same for the SEDCMD setting in props.conf? Or does it operate on the entire event?

Just for the sake of an example. Say I'm using the following unix command to filter out comment lines (lines starting with a ";"). (Technically, this example only clears the comment lines rather than remove them, "grep" would be more obvious choice of unix commands, but splunk doesn't have a GREPCMD, so it's a mute point.)

cat myfile |  sed -re 's/^\s*;.*$//'

Will that work the same way as the following props.conf entry:

SEDCMD-drop_comments = s/^\*;.*$//

Accepted Answer

3

splunk's SEDCMD works with one event at a time

link

answered 02 Jul '10, 17:10

Ledion Bitincka ♦
1.5k●3●6
accept rate: 35%

That's what I was suspecting. I found you can make it work line-by-line by using multi-line regex mode. (I added an answer of my own with an example.)

(02 Jul '10, 19:19) Lowell ♦

Answer 2

4

To get the SEDCMD to work on a line-by-line basis, you can use the following:

SEDCMD-drop_comments = s/(?m)^\*;.*$//g

The difference is:

The (?m) enables multi-line mode in regex engine. This causes ^ and $ to now mean start-of-line and end-of-line, rather than start-of-event and end-of-event.
The g on the end tells the sed command to be applied as many times as possible; therefore the replacement will occur more than once. So without the g, only the first matching line would be replaced instead of all the lines.

link

answered 02 Jul '10, 19:15

Lowell ♦
9.6k●6●37
accept rate: 40%

Note that Splunk's SEDCMD uses PCRE regex, not standard sed regex.

(02 Jul '10, 20:41) gkanapathy ♦

@gkanapathy, very good point. (Yeah, strait sed-style regex are a pain once you've gotten use to a powerful regex engine. I normally end up using the sed -r which enables extended regexs, which are still pretty weak when compared to PCRE.) Splunk is definitely more flexible in that regard ;-)

(02 Jul '10, 21:16) Lowell ♦

Does SEDCMD work line-by-line or on the entire event?

Follow this question

Splunk Resources

Related questions