Splunk Search

Filtering on HF via regex not working

twhisnant
New Member

The overview: a syslog server acting as a UF receives data via syslog to various local files. Inputs.conf is specified on the server and tags all files with index and sourcetype. This works.

We have a new source that can only send over udp 514, meaning that we have multiple sourcetypes in a single log (we'll say sourceA and sourceB).

Inputs.conf on the UF labels the file with sourcetype and index.

Inputs.conf:
[monitor:///var/log/net-default.log]
disabled = false
sourcetype=sourceA
index=A

All data from UFs are sent to balanced HFs. The HFs have an app with a props.conf and transforms.conf file that are trying to identify the traffic and sourcetype/index as desired.

On the HF:

props.conf
[source::....net-default.log]
TRANSFORMS-force_sourcetype_for_B = force_sourcetype_for_B
TRANSFORMS-force_index_for_B = force_index_for_B

[source::....log]
TRANSFORMS-set_host = set_host

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = sourcetype::sourceB

[force_index_for_B]
DEST_KEY = _MetaData:Index
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = index::B

The regex correctly picks up the data via "regex _raw". Unfortunately everything is going into indexA.

What we want:
Syslog/HF(file(A/B) -> HF(parse file - tag events as B) -> Indexers (file:indexB)

Now that everyone is thoroughly confused...

Any help is appreciated.
- Tim
@heywiz

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

Where are you defining sourcetype=sourceB?

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

In your props.conf, you reference the transform set_host, but it's not in your transforms.conf.

0 Karma

twhisnant
New Member

That is a great question. I was thinking I was defining it in the transforms.conf on the HF.

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = .blah.
FORMAT = sourcetype::sourceB

Incorrect syntax?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...