How to filter Windows Event 4663 at indexing time?

[kiran331]

Hello,

How to filter out wineventlog with "EventCode 4663" and "Accesses: ReadData (or ListDirectory)", using props.conf and transforms.conf below is sample event.

08/17/2017 01:35:55 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4663
EventType=0
Type=Information
ComputerName=abc.cde
TaskCategory=Removable Storage
OpCode=Info
RecordNumber=2326
Keywords=Audit Success
Message=An attempt was made to access an object.

Subject:
Security ID: S-1-5-80-1390545455-656-4545454545
Account Name: AAAADDDDDD
Account Domain: NT SERVICE
Logon ID: 0xC8184

Object:
Object Server: Security
Object Type: File
Object Name: D:\Program Files\dir\test\tt.exe
Handle ID: 0x11a5c
Resource Attributes:
Process Information:
Process ID: 0x224
Process Name: D:\Program Files\dir\test\tt.exe

Access Request Information:
Accesses: ReadData (or ListDirectory)

Access Mask:        0x1

[gcusello · Accepted solution]

Hi kiran331,
following Splunk documentation https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad you have to create props.conf and transforms.conf:

props.conf

TRANSFORMS-set-null=set_parsing,set_null

transforms.conf

[set_parsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

[set_null]
REGEX=(?ms)EventCode\=4663.*Accesses:\sReadData\s\(or\sListDirectory\)
DEST_KEY=queue
FORMAT=nullQueue

Beware to the order of stanzas in TRANSFORMS command in props.conf, instead order in transforms.conf it's not important.

Bye.
Giuseppe

[kiran331]

Hi Cusello,

Its still indexing Read Events. Below are my config files, Did I miss anthing?

props.conf

[WinEventLog:Security]

Returns most of the space savings XML would provide

SEDCMD-clean0 = s/(?m)(^\s+[^:]+:)\s+-?$/\1/g s/(?m)(^\s+[^:]+:)\s+-?$/\1/g s/(?m)(:)(\s+NULL SID)$/\1/g s/(?m)(ID:)(\s+0x0)$/\1/g

Returns most of the space savings XML would provide

SEDCMD-clean1 = s/This event is generated[\S\s\r\n]+$//g
SEDCMD-clean2 = s/Certificate information is only[\S\s\r\n]+$//g

addresses most of the Ipv6 log event issues

SEDCMD-clean3 = s/::ffff://g

FIX #1

SEDCMD-clean4 = s/Token Elevation Type indicates[\S\s\r\n]+$//g

TRANSFORMS-set-null=set_parsing,set_null

TRANSFORMS-set-exclude=set_nullqueue

transforms.conf:

[Target_Server_Name_as_dest_nt_host]
SOURCE_KEY = Target_Server_Name
REGEX = ^(?!localhost)([\]+)?([^-].*)
FORMAT = dest_nt_host::"$2"

[Target_Server_Name_as_dest]
SOURCE_KEY = Target_Server_Name
REGEX = ^(?!localhost)([\]+)?([^-].*)
FORMAT = dest::"$2"

[set_parsing]
REGEX= .
DEST_KEY=queue
FORMAT=indexQueue

[set_null]
REGEX=(?ms)EventCode=4663.*Accesses:\sReadData\s(or\sListDirectory)
DEST_KEY=queue
FORMAT=nullQueue

[set_nullqueue]
REGEX=C:\Program Files\SplunkUniversalForwarder\bin\*
DEST_KEY=queue
FORMAT=nullQueue

[gcusello]

Hi kiran331,
Parentheses are special characters for regex so you must put a backslash before them where they are in the search string.
So the regex of set_null stanza is

(?ms)EventCode\=4663.*Accesses:\sReadData\s\(or\sListDirectory\)

Bye.
Giuseppe

[tmarlette]

Check out this doc:
Router and Filter Data

Do a ctrl+f on your browser and search for setnull

[kiran331]

I tried below, its not working, did I miss anything?

props.conf

TRANSFORMS-set-null=set_null

transforms.conf

[set_null]
REGEX="(?msi)EventCode=4663.*readdata|EventCode=4663.*listdirectory"
DEST_KEY=queue
FORMAT=nullQueue

[tmarlette]

Try this regex instead.

(EventCode=4663*.readdata|EventCode=4663.*listdirectory)

[kiran331]

I tried, its not working.

[tmarlette]

[set_null_1]
REGEX=EventCode=4663.*readdata
DEST_KEY=queue
FORMAT=nullQueue

[set_null_2]
REGEX=EventCode=4663.*listdirectory
DEST_KEY=queue
FORMAT=nullQueue

try two seperate stanza's and call them out in props. Sometimes regex is funky with the ".*" and "|" matches in preindexing.

[kiran331]

I tried, its not filtering out the events, I'm also using one more regex to filter out splunk events, May be this is caousing the issue.

props.conf:

TRANSFORMS-set-exclude=set_exclude,set_nullqueue

TRANSFORMS-set-null1=set_null_1

TRANSFORMS-set-null2=set_null_2

transforms.conf:

[set_exclude]
REGEX=.
DEST_KEY = queue
FORMAT = indexQueue

[set_nullqueue]
REGEX=C:\Program Files\SplunkUniversalForwarder\bin\*
DEST_KEY=queue
FORMAT=nullQueue

[set_null_1]
REGEX=EventCode=4663.*readdata
DEST_KEY=queue
FORMAT=nullQueue

[set_null_2]
REGEX=EventCode=4663.*listdirectory
DEST_KEY=queue
FORMAT=nullQueue

[skoelpin]

Did you restart splunkd after making these changes?

[kiran331]

yes, I restarted all Indexers

[tmarlette]

As a test, remove the ".* " and everything after it from your regexes and see if that works.

[kiran331]

I removed everything after EventCode, it filtered out all Event with 4663.

[set_remove]
REGEX = EventCode=4663
DEST_KEY=queue
FORMAT=nullQueue

[tmarlette]

is it filtering out all of those events?

[kiran331]

yes, it filtering all events with 4663 Eventcodes, but I have filter out with Accesses: ReadData (or ListDirectory)

[tmarlette]

yeah, so you'll need to find the right RegEx match to filter out what you need. I use .* or .+ in my extractions, but for some reason, during pre-indexing Splunk doesn't like those big wildcards.