Splunk Search

How to extract different sourcetypes from the same file

splunkuzleuven
Loves-to-Learn Lots

I'm wondering if/how I can do the following:

I have a JSON structured file that is being parsed perfectly as JSON, so I get all my fields.
However, there is a certain field, let's call it "message" that is a standard web Access log.

So currently, I'm reading the log file and parse it as a json, and that works perfectly, except that the value of "message" is just a textblob of course. How can I parse the "message" part as well, so that all fields are being extraced at indextime?
Can I somehow combine sourcetypes?

I want to have all fields related to the same event though so that searching is easier.

Is this possible? Or will I need to JSON my "message" part as well before writing it to a log file?

0 Karma
1 Solution

martynoconnor
Communicator

You can't parse part of an event as one sourcetype and another part as a different sourcetype. If it's necessary to read it as that particular sourcetype you should split the content off before it reaches Splunk and read it in separately under a different sourcetype. If that's not an option you could (though I really don't recommend it) extend the JSON sourcetype definition to include additional field extractions in props.conf

View solution in original post

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@splunkuzleuven - If you already have control over writing log file then I would suggest that you write message part as JSON only. As doing props and transforms will be a performance downside in this case as it will require to go through the _raw event with regex.

0 Karma

martynoconnor
Communicator

You can't parse part of an event as one sourcetype and another part as a different sourcetype. If it's necessary to read it as that particular sourcetype you should split the content off before it reaches Splunk and read it in separately under a different sourcetype. If that's not an option you could (though I really don't recommend it) extend the JSON sourcetype definition to include additional field extractions in props.conf

0 Karma

splunkuzleuven
Loves-to-Learn Lots

Okay, it's what I was afraid of already. Thanks for the feedback.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...