testing new regex

Question

Before really putting my custom regex in transforms.conf, is there a quick way to test and debug it?

Answer 1

2

I usually use $SPLUNK_HOME/bin/pcregextest on the Splunk command line because it uses the exact regex engine as comes with Splunk. The command line is somewhat inconvenient and you do have to make sure that you quote a few characters correctly according to shell quoting rules. (Please note that you need to quote characters when using rex as well, and rex quoting rules are an even worse headache. Furthermore, bash shell quoting is well-defined and completely documented.)

link

answered 10 Sep '10, 16:38

gkanapathy ♦
26.4k●1●6●22
accept rate: 42%

Good point about rex and the quoting considerations. Generally I've only hit issues when I have to match a literal " (double-quote) character, but it's important to be aware that you can't always just copy and paste between a rex command and a config file.

(10 Sep '10, 17:31) Lowell ♦

Answer 2

1

Use the rex command: ..... | rex "" | .....,

For example use the following regex to extract indexes.conf into the conf_file variable in the following event:

05-21-2010 17:41:51.166 INFO  IndexProcessor - indexes.conf - memPoolMB param autotuned to 512MB
... | rex " - (?<conf_file>\w+\.conf) - "| .....

link

answered 30 Jul '10, 14:54

Ledion Bitincka ♦
1.5k●3●6
accept rate: 35%

i assume you are talking about using the splunk command line, are you? how do i invoke the splunk command line? (sorry for sounding so n00by! installed splunk only yesterday :p )

(30 Jul '10, 15:00) njathan

Searching the internal index for regex extraction processor errors may help too. Like so...

index=_internal regexExtractionProcessor ERROR

7:59:13.931 AM
07-30-2010 07:59:13.931 ERROR regexExtractionProcessor - Cannot compile RE "((.AcctCode=d{5}-)." for trade: missing ) host=class4.splunk.com Options| sourcetype=splunkd Options| source=/home/rick4uc/opt/splunk/var/log/splunk/splunkd.log Options

(30 Jul '10, 15:15) rroberts

@njathan - I was talking about testing the regex using the rex search command - either from splunk's search UI or CLI

(24 Aug '10, 23:24) Ledion Bitincka ♦

Answer 3

0

You may also want to look at Kodos (free) or RegexBuddy (commercial).

link

answered 10 Sep '10, 16:30

southeringtonp ♦
4.5k●1●2●15
accept rate: 35%

I normally end up using Kodos for testing. The only down side is that it uses Python regular expression syntax instead of pcre-style regular expressions. The only difference I've see are: (1) You have to add a P in a capture group name, like: (?P<group>..) Of course, PCRE accepts both, so I often let the extra "P" in the final regex. And (2), and python (and therefore Kodos) doesn't support atomic grouping or possessive quantifiers (like \s++). But this generally hasn't been too much of a problem since I don't often use these advanced regex these features.

(10 Sep '10, 17:27) Lowell ♦

testing new regex

Follow this question

Splunk Resources

Related questions