Getting Data In

Log file is importing: How to parse the event?

richtate
Path Finder

I am getting the log file imported to Splunk, but each line is an event with no field name.  Can I break up the line into columns?  If not, how do I parse the line to extract a number?

Index is:

index=test_7d sourcetype=kafka:producer:bigfix

Events are:

2023-06-22 09:15:44,270 root - INFO - 114510 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:37,204 root - INFO - Executing getDatafromDB
2023-06-22 09:15:35,704 root - INFO - 35205 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:33,286 root - INFO - Executing getDatafromDB
2023-06-22 09:15:32,703 root - INFO - 167996 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:22,479 root - INFO - Executing getDatafromDB
2023-06-22 09:15:19,031 root - INFO - 181 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka

Each line/event starts with the date, the wordwrap is making it look incorrect.  I need to parse the bold number of each line after '- INFO -' and add a zero if no number.  I can do this with a eval, but how do I parse if there is no field name to add to the 'regex' command?

Thank you for looking at this problem!

Labels (5)
Tags (2)
0 Karma
1 Solution

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount

View solution in original post

Tags (1)
0 Karma

richtate
Path Finder

Any help is appreciated, even if it means this is in the wrong category..

0 Karma

richtate
Path Finder

I found the erex command that works,

| erex ImportCount examples="0,18729,49377"

But you have to enter a sample of the text you are looking for.  So it only works for one day and it has to be changed.  Can regex be used in place of the examples?

0 Karma

richtate
Path Finder

For example, here I'm using 'regex' to remove Operating Systems from dataset on a fieldname 'operating_system' which is one column of an sourcetype:

| regex operating_system!="(Linux|AIX|CENTOS|WINDOWS|Digital UNIX|FreeBSD|HP-UX|Hyper-V|Juniper|Mac|Windows|NetBSD|OpenBSD|OpenVMS|Server 2012|Server Core 2012|Server 2016|Server 2019|Ubuntu|Solaris|Unix|ESX|vCenter Server|rbash|[\*\*\*\*\*\*]|\A[\-\-\-\-\-\-\-\-\-\-]|[\=\=\=\=\=\=\=\=\=\=])"

0 Karma

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...