props.conf and transforms.conf not extracting fields

[tkwaller]

Hello

I set up custom field extractions for a facter app I created but it seems that it is not extracting the fields.
facter.sh outputs data almost in a field:value but uses "=>" instead of "=" so I had to create a REGEX to extract the data. Heres a sample of the output:

architecture => x86_64
augeasversion => 1.2.0
bios_release_date => 06/22/2012
bios_vendor => Phoenix Technologies LTD
bios_version => 6.00
blockdevice_fd0_size => 4096
blockdevice_hdc_size => 4294965248
blockdevice_sda_model => Virtual disk
blockdevice_sda_size => 85899345920
blockdevice_sda_vendor => VMware
boardmanufacturer => Intel Corporation
boardproductname => 440BX Desktop Reference Platform
boardserialnumber => None
facterversion => 1.7.4

My props.conf in SPLUNKHOME/etc/apps/myapp/local/:

[facter]
REPORT-facter=extract-puppet-fields

My transforms.conf in SPLUNKHOME/etc/apps/myapp/local/:

[extract-puppet-fields]
REGEX  = [\t (),./0-9A-Z_a-z]{1,22}
FORMAT = $1::$2

My inputs.conf in SPLUNKHOME/etc/apps/myapp/default:

[script://./bin/facter.sh]
disabled = false
index = os
interval= 3600
sourcetype=facter
source=facter

Any ideas what I am missing? I can get the output of the script in inputs.conf when searching in splunkweb but none of the fields are being extracted.

As a side note: I had the extraction set to REGEX = (S+)s=>s(S+) as I saw it in an answer on another post but it didn't work either. I tested the one above in a regex tester before trying.

Thanks for the help!

[woodcock]

Try this alternate syntax:

[extract-puppet-fields]
REGEX = (?<_KEY_1>[^\s\n\r]+)\s+=>\s+(?<_VAL_1>[^\s\n\r]+)
MV_ADD = true

[woodcock · Accepted solution]

Keep everything you originally posted but try this:

 REGEX  = ([^\s\n\r]+)\s+=>\s+([^\s\n\r]+)
 MV_ADD = true

[tkwaller]

Ok so with this in my transforms.conf in SPLUNKHOME/etc/apps/myapp/local:

[extract-puppet-fields]
REGEX = ([^\s\n\r]+)\s+=>\s+([^\s\n\r]+)
MV_ADD = true
FORMAT = $1::$2

Still extracts nothing but defaults

[woodcock]

Did you deploy this to all of your Search Heads? That is the only thing I can think that could be wrong because the configurations are all correct.

[tkwaller]

Sorry for the delay, been working on other priorities, yes I did put this on my search heads. I just tested again searching on search head and it is extracting properly.
Thanks so much for the assistance, you are awesome!

[tkwaller]

I will test and post back an update once deployed. Thanks so much for your help sir!

[woodcock]

In order for this to work, you need to have 2 RegEx capture groups; the first will be used for $1 and the second for $2. You have 0 capture groups set in you RegEx which is done by putting something inside parentheses: "(a captured group is here)".

[tkwaller]

So something like this for transforms.conf?
[extract-puppet-fields]
REGEX = (?<$1>[t (),./0-9A-Z_a-z]{1,22})
REGEX = (?<$2>[t (),./0-9A-Z_a-z]{1,22})
FORMAT = $1::$2

or just one line
[extract-puppet-fields]
REGEX = (?<$1>[t (),./0-9A-Z_a-z]{1,22}) (?<$2>[t (),./0-9A-Z_a-z]{1,22})
FORMAT = $1::$2

[woodcock]

No, let's take the example you copied, which is REGEX = (\S+)\s=>\s(\S+). This says take a group of non-whitespace characters (\S+) and capture it (but don't name the capture). Skip over 1 whitespace character followed by an equals sign followed by another whitespace character \s=>\s and then take the next group of non-whitespace characters (\S+) and capture it, which becomes your second captured group (even though you have not explicitly named or numbered them). These 2 captured groups are used on the FORMAT line as $1 and $2. You need to do it like that (if that is even what you are trying to do, which I am not sure).

[riqbal]

I have one xml file
I want to extract (at search time) the fields/values IN BETWEEN and and throw away any of the lines before the very first and after the very last .
(In XML, the fields/values are located on each line in the form value)
4. Use the date in the ActionDate field and the time in the ActionTime field as the timestamp.

' <Interceptor>
            <AttackCoords>-423423445345345.10742916222947</AttackCoords>
            <Outcome>Inteccccn</Outcome>
            <Infiltrators>20</Infiltrators>
            <Enforcer>Iwildwood</Enforcer>
            <ActionDate>2013-04-24</ActionDate>
            <ActionTime>00:07:00</ActionTime>
            <RecordNotes></RecordNotes>
            <NumEscaped>0</NumEscaped>
            <LaunchCoords>-80.23429525620114,24.08680387475695</LaunchCoords>
            <AttackVessel>local</AttackVessel>
        </Interceptor>'

below is my props.conf and transforms.conf
props.conf
[dreamcrusher]
BREAK_ONLY_BEFORE =
DATETIME_CONFIG =
NO_BINARY_CHECK = true
TIME_FORMAT =
TIME_PREFIX =
category = Custom
disabled = false
pulldown_type = true
PREAMBLE_REGEX = ^<\S+.*
REPORT-dream = dream

transforms.conf
[dream]
REGEX = ^<(.*?)>(\S+)<

FORMAT = $1::$2

when i check the events there are no search time extraction