Disabling or removing extra description text in Windows 2008 event logs?

Question

I just recently started using Windows 2008 and when I got splunk setup and forwarding thge Windows event logs and I noticed that a large number of the events now have a sentence or two about the event itself at the end of the event. This is nice when you first run across a new type of event, but it clutters up splunk with unnecessary fluff (If I wanted this info within splunk, I would prefer to create a splunk lookup table. It seems that the same message is always used for the same "EventCode"). Sometimes the extra text is quite long.

Does anybody know if this is a "feature" that can be turned off? Or, has anyone come up with a reasonable way of stripping this kind of boiler-plate text?

I'm thinking that this could greatly impact splunk's overall licensing volume as we deploy and start more heavily using Windows 2008 servers.

Here are some (truncated) samples:

07/16/10 08:11:34 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4648
EventType=0
Type=Information
ComputerName=WIN64.domain.com
TaskCategory=Logon
OpCode=Info
RecordNumber=5973
Keywords=Audit Success
Message=A logon was attempted using explicit credentials.
....
This event is generated when a process attempts to log on an account by explicitly specifying that account’s credentials.  This most commonly occurs in batch-type configurations such as scheduled tasks, or when using the RUNAS command.

For an even more absurd example, take a look at the following event:

07/16/10 11:18:30 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4624
EventType=0
Type=Information
ComputerName=HIDDEN
TaskCategory=Logon
OpCode=Info
RecordNumber=6039
Keywords=Audit Success
Message=An account was successfully logged on.
....
This event is generated when a logon session is created. It is generated on the computer that was accessed.
The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).
The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.
The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
The authentication information fields provide detailed information about this specific logon request.
    - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

In the last event there is more than 1k of static text, and many many terms I don't necessarily want indexed. That can add up quickly.

I know that a transformer could be used to clean this up, but I don't want to strip off a legitimate message by accident. In the two examples shown, they both start with "This event is generated when", but that's not true for all the the messages I've seen so far. So without a very large sample it seems difficult to safely come up with a regex that only removes the extra verbose message.

Answer 1

We're testing this SEDCMD configuration and it seems to work well for truncating the message text for event IDs 4624, 4634, and a few others which are the high-volume events in an Active Directory environment:

# Install on Splunk indexer in $SPLUNK_HOME/etc/system/local/props.conf
[source::WinEventLog:Security]
SEDCMD-win = s/(?mis)(Token Elevation Type indicates|This event is generated).*$//g

Strictly speaking the trailing "g" probably isn't necessary. We did run into one problem which is that agents must be configured as light forwarders for the parsing to happen at the indexer. This would need to be deployed to non-lightweight forwarders as well, I believe. Since we deployed this only two days ago and had to upgrade a bunch of agents in the process, I'm still in waiting mode to see what volume reduction this generates.

-- James

Answer 2

My research and testing suggests that this feature cannot be turned off. Fortunately it appears only a limited number of events have this verbose text added to them, at least at this time. 13 events in total:

4612: Internal resources allocated for the queuing of audit messages have been exhausted, leading to the loss of some audits.
4616: The system time was changed.
4618: A monitored security event pattern has occurred
4624: An account was successfully logged on
4625: An account failed to log on
4634: An account was logged off
4647: User initiated logoff
4648: A logon was attempted using explicit credentials
4769: A Kerberos service ticket was requested
4778: A session was reconnected to a Window Station
4779: A session was disconnected from a Window Station
4780: The ACL was set on accounts which are members of administrators groups
4908: Special Groups Logon table modified

Out of these, only 4624, 4625, 4634, 4647, 4648, 4769 and possibly 4778 and 4779 are common enough to worry about stripping the description text. Based on this limited number of event IDs it might be best to just use transforms or sedcmd...

Answer 3

1

Simple fix for removing the repeating detailed description from the message field but leave details:

in props.conf

# message shortener for windows event security
# removes text from message field starting with: This event is generated

[WinEventLog:Security]
TRANSFORM-windows_events = win_event_shortener

in transforms.conf

[win_event_shortener]
DEST_KEY = _raw
REGEX = ((.*+[\v])+)(?=This event is generated)
FORMAT = $1

link

answered 21 Jan, 12:37

sjohnson_splunk
31●1
accept rate: 0%

Is this more efficient than the simple replace in the props.conf shown above it? I also wonder if it would be possible limiting application of the substitute or regex to events with EventCode of 4624 or 4634. This as it is would be tested against all incoming events Security events in this index.

(05 Mar, 16:20) wrangler2x

Answer 4

We haven't migrated to 2k8 yet, so I haven't had the pleasure of indexing 2k8 logs, but looking at group policy documentation for 2k8 R2 the following sticks out:

Machine, wdi.admx, Diagnostics: Configure scenario execution level.

Determines the execution level for Diagnostic Policy Service (DPS) scenarios.\n\n\nIf you enable this policy setting, you must select an execution level from the dropdown menu. If you select problem detection and troubleshooting only, the DPS will detect problems and attempt to determine their root causes. These root causes will be logged to the event log when detected, but no corrective action will be taken. [...] The DPS can be configured with the Services snap-in to the Microsoft Management Console.

What execution level do you have the Diagnostic Policy Service (DPS) set for? Could it be that it's configured for detection and troubleshooting, and causes it to log the extra data?

[edit] FYI 2k8 r2 group policies: http://www.microsoft.com/downloads/details.aspx?FamilyID=18c90c80-8b0a-4906-a4f5-ff24cc2030fb&displaylang=en

Answer 5

0

Came across this today: http://technet.microsoft.com/en-us/library/ee126097(WS.10).aspx#BKMK_Controlling

Event Viewer and Resulting Internet Communication in Windows 7 and Windows Server 2008 R2 - Procedures for preventing the flow of information to and from the Internet through Event Viewer

Sounds like an interesting lead.

link

answered 28 Jul '10, 20:00

ftk ♦
6.8k●1●7●27
accept rate: 38%

I don't think this helps. This seems to be talking about the Event Viewer looking up event information online, which doesn't seem to be related to the extra verbose messages at the bottom of the windows events that I'm trying to filter out.

(02 Aug '10, 16:22) Lowell ♦

Answer 6

0	For the record, I am noting the obvious Splunk-based solution would be to use `SEDCMD` or a `TRANSFORM` to just delete it from the incoming data. link answered 28 Jul '10, 20:36 gkanapathy ♦ 32.4k●4●8●27 accept rate: 41%

Disabling or removing extra description text in Windows 2008 event logs?

Follow this question

Splunk Resources

Related questions