Hi All;
I have an interesting issue. Currently, I have data free flowing into a port on in Splunk, and one of the fields in this data has become corrupt and is not allowing me to search my data correctly. What I want to do is remove this field from the data before it is indexed. Is there any way I can do this in Splunk itself?
Note: I really want to avoid sending the data else where for this change to be made and then sending it to Splunk, and I would want to not be limited the option of changing the field in the source.
Any suggestions are greatly appreciated as always.
Thanks,
Tyler
I would advise against deleting the time information after Last_Action. What if you wanted to use it in a search?
You could simply tell splunk where to break events and where to look for the timestamp of the event itself explicitly, like so:
[mobile]
LINE_BREAKER=([\r\n]+)\d{4}-
SHOULD_LINEMERGE=false
TIME_PREFIX=^
This should break your events properly and still retain all data.
These are good pointers, and I'm still trying to see if I can make this work. To be more specific of the field that is corrupting my data is this field has a timestamp component to it.
Scenario: A single event is sent to splunk that looks like this
2015-12-02T15:34:45-0800
User: Jim
Event_Name: "Click_Event"
Action_Type: "Lower_Menu_Item"
Last_Action: "click_2015-12-01T12:00:00-0800"
Last_Action_Type: "Upper Right Button"
Splunk then indexes this single event as 2 events:
One like this:
_time: 2015-12-02T15:34:45-0800
2015-12-02T15:34:45-0800
User: Jim
Event_Name: "Click_Event"
Action_Type: "Lower_Menu_Item"
The other like this:
_time: 2015-12-01T12:00:00-0800"
Last_Action: "click_2015-12-01T12:00:00-0800"
Last_Action_Type: "Upper Right Button"
End Goal: Stop splunk from splitting up my events.
Would settle for removing the "Last_Action" field if I can do it before splunk splits the event.
Then you asked the wrong question. See what @jeffland said.
Tried this in our props.conf file just to remove the field entirely but still no success. Is there something I'm doing wrong?
[mobile]
SEDCMD-nonrequiredtimestamps = s/[Last_Action =].*/Last_Action =/g
I came across this document see if it's of any help.
http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Routeandfilterdatad