Getting Data In

Filter windows events on indexer from a universal forwarder

jstockamp
Communicator

I've been beating my head against a wall with this for a few hours. My setup is I have a linux indexer with a few windows universal forwarders sending data to it. I'm trying to filter out some event codes from the index. I'm assuming that since these are universal forwarders, I'll need to forward all events and filter out the ones I don't want at index time, correct?

Based on that assumption on the indexer I've got the following in $SPLUNK_HOME/etc/system/local/props.conf

[WinEventLog:Security]

TRANSFORMS-Filter_Events = FilterSecurityEvents

and in $SPLUNK_HOME/etc/system/local/transforms.conf

[FilterSecurityEvents]

REGEX=(?msi)^EventCode=(4634|4624|4769)

DEST_KEY=queue

FORMAT=nullQueue

I'm still getting events with these 3 codes in the index. Any idea(s) why this isn't working?

1 Solution

jstockamp
Communicator

I believe I've solved this. The issue is that the forwarder was running version 4.2.1 of the universal forwarder and my indexer was still running splunk version 4.2. I upgraded the indexer to 4.2 and it now seems to be working and filtering correctly.

Kind of odd that there were no errors in splunkd.log and that the forwarder clearly had no problems talking to the indexer since events were still making it into the index (just not being filtered).

View solution in original post

gwege
New Member

Any update to this? I'm having a similar issue.

0 Karma

jstockamp
Communicator

I believe I've solved this. The issue is that the forwarder was running version 4.2.1 of the universal forwarder and my indexer was still running splunk version 4.2. I upgraded the indexer to 4.2 and it now seems to be working and filtering correctly.

Kind of odd that there were no errors in splunkd.log and that the forwarder clearly had no problems talking to the indexer since events were still making it into the index (just not being filtered).

jstockamp
Communicator

Thanks. I've opened a case with support. I've tried specifying a specific host and source in props.conf instead of using a sourcetype and it still didn't work.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

I tried your stanzas in my lab, and the events are being filtered properly. I tested with your original RegEx (which is more efficient than using no anchor, but doesn't account for possible odd multi-line behavior). You might contact support to see if there is an issue when using a forwarder. One other option might be to try specifying the transform using host or source, to see if it's limited to sourcetype.

One other consideration: My lab is eating the security eventlog as a file, rather than using WMI. The structure is exactly the same; however, the issue may have something to do with the input.

Cheers

0 Karma

jstockamp
Communicator

This is still not working. Here's my props.conf from /opt/splunk/etc/system/local:

[host::web002.xxx]
TRANSFORMS-no_lb = no_lb_traffic_transform
[WinEventLog:Security]
TRANSFORMS-FilterEvents = filtersecurityevents

and here's the transforms.conf file from the same directory:

[no_lb_traffic_transform]
REGEX=(10\.100\.0\.3|10\.100\.0\.2)(.*) 
DEST_KEY=queue
FORMAT=nullQueue
[filtersecurityevents]
REGEX=EventCode=(4634|4624|4769)
DEST_KEY=queue
FORMAT=nullQueue

As you can see, I have an additional stanza in each from some filtering I'm doing on some other web logs. That filtering is working fine, it's just the windows events that aren't being filtered. I have tested my regex in the search app and it is matching all the events that I want to filter out.

Any help would be greatly appreciated.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

You can test your RegEx in Search, before you hit the drawing board with props/transforms. Use: ... | regex "EventCode=(4634|4624|4769)" to make sure your events show.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

If you remove your ^ anchor, it should work:

REGEX=EventCode=(4634|4624|4769)

EDIT: Don't forget to restart Splunk after making/changing an index-time transform.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...