Splunk Search

How to do a field extraction of a field that already exists?

neerajs_81
Builder

Gentlemen,
We are on Splunk Cloud.

In my raw events coming from AWS , splunk by default shows a field called "category" under "Interesting fields" . However, it's value ( as in it's extraction)  isn't what we are expecting it to be. It only manages to extract a part of the complete string.  

For example:  
The raw events have category as follows (In JSON format) 
"Policy:IAMUser/RootCredentialUsage"   (without quotes)

But Splunk is instead showing the value of category as:  Policy

neerajs_81_0-1650304556324.png

 



.Now,  whats happening is if i use the IFX or rex command to create a field extraction  keeping the same name for my field i.e.  category and value: Policy:IAMUser/RootCredentialUsage   ,  my newly extracted field keeps getting overwritten with the default old values again . I am assuming this is because  the names of the fields are same  ( category) , so splunk takes its own precedence. 

IS this the case of Index time vs Search Time field extraction conflict ?

How to make Splunk use whatever value my field extraction ( as in rex or IFX) is extracting for category and at the same time also retain its name as is ?   Dont want the category field to display its old indexed value.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

The question is where does this field come from. You're saying it's auto-extracted but if the value in raw event is completely different then the field must be either a calculated field or provided as indexed field by the source (are you pushing the events with HEC?).

It's kinda strange and you should check all your ingestion process and source/sourcetype definitions.

But if you do a | rex command to extract your own value of category field, it should overwrite whatever is included in the event. Are you sure you're not mixing upper/lowercase in field names?

neerajs_81
Builder

Thanks for responding. Yes, the "category" field is provided as an indexed field.  The data is getting ingested via HEC (  Splunk TA for Amazon Kinesis Firehose).  Does anything change if its coming via HEC ?
Positive there is no issue of upper case/lower case mismatch

What do you recommend then,  if the indexed field is not getting overwritten by whatever value my rex / IFX is extracting for it ?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

To be fully honest, I've never investigated what happens if you have the indexed field and try to extract it search-time as well. But as I said, any field should be overwritable with explicit field manipulation in search-time.

Anyway, is there no possibility to change the source behaviour? Because it's very strange that it defines a field which is already present (with another value) within the event itself.

Of course you can simply define an extraction to another field name (like eventcategory or something like that) but that's counterintuitive and can be confusing.

neerajs_81
Builder

Thanks, yeah maybe a different field name is the way to go.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...