Splunk Search

How to do a field extraction of a field that already exists?

neerajs_81
Builder

Gentlemen,
We are on Splunk Cloud.

In my raw events coming from AWS , splunk by default shows a field called "category" under "Interesting fields" . However, it's value ( as in it's extraction)  isn't what we are expecting it to be. It only manages to extract a part of the complete string.  

For example:  
The raw events have category as follows (In JSON format) 
"Policy:IAMUser/RootCredentialUsage"   (without quotes)

But Splunk is instead showing the value of category as:  Policy

neerajs_81_0-1650304556324.png

 



.Now,  whats happening is if i use the IFX or rex command to create a field extraction  keeping the same name for my field i.e.  category and value: Policy:IAMUser/RootCredentialUsage   ,  my newly extracted field keeps getting overwritten with the default old values again . I am assuming this is because  the names of the fields are same  ( category) , so splunk takes its own precedence. 

IS this the case of Index time vs Search Time field extraction conflict ?

How to make Splunk use whatever value my field extraction ( as in rex or IFX) is extracting for category and at the same time also retain its name as is ?   Dont want the category field to display its old indexed value.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

The question is where does this field come from. You're saying it's auto-extracted but if the value in raw event is completely different then the field must be either a calculated field or provided as indexed field by the source (are you pushing the events with HEC?).

It's kinda strange and you should check all your ingestion process and source/sourcetype definitions.

But if you do a | rex command to extract your own value of category field, it should overwrite whatever is included in the event. Are you sure you're not mixing upper/lowercase in field names?

neerajs_81
Builder

Thanks for responding. Yes, the "category" field is provided as an indexed field.  The data is getting ingested via HEC (  Splunk TA for Amazon Kinesis Firehose).  Does anything change if its coming via HEC ?
Positive there is no issue of upper case/lower case mismatch

What do you recommend then,  if the indexed field is not getting overwritten by whatever value my rex / IFX is extracting for it ?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

To be fully honest, I've never investigated what happens if you have the indexed field and try to extract it search-time as well. But as I said, any field should be overwritable with explicit field manipulation in search-time.

Anyway, is there no possibility to change the source behaviour? Because it's very strange that it defines a field which is already present (with another value) within the event itself.

Of course you can simply define an extraction to another field name (like eventcategory or something like that) but that's counterintuitive and can be confusing.

neerajs_81
Builder

Thanks, yeah maybe a different field name is the way to go.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...