Splunk Search

How to extract key-value pairs from log and visualize them

arnavkumarsaxen
Explorer

My logs are in the format:

 

My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121

 

I currently use query:

index="myindex" "My-Application Log:" | extract pairdelim=",  " kvdelim="= " | table Some-Key  SomeOtherKey SomeOtherKey2 "Some Different Key" "Another Key"

 

It is able to extract events however the table is filled with blank/null values.

 

How can i visualise the data if i have this format of logs.

I have to group by Some-key.

Example visualization should be grouped basis Some-key

Thanks in advance.

0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

arnavkumarsaxen
Explorer

Works for me. However, i am new to splunk and i might have a small question.

The value from the last field contains \n" and sometimes \n"}

How to remove this as well.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try it like this

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\n\}]+)"
0 Karma

arnavkumarsaxen
Explorer

arnavkumarsaxen_0-1650366380136.png

Values still coming in this way

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Assuming you don't want anything after (and including) a backslash, try this:

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\\\\]+)(?=\\\\)?"

arnavkumarsaxen
Explorer

Thanks a lot. It worked.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...