Getting Data In

Changing the index on forwarded data

brianm1002
New Member

I have one Splunk indexer that both indexes and forwards the data to a second Splunk indexer. The name of the index is not the same on both indexers. When the data is received by the second indexer, I get the error that it received an event for an unrecognized index.

How can I change the index when it's sent to the second indexer? I've tried adding an index=newindex to the inputs.conf on the second indexer, but it's still trying to use the name of the index from the first indexer.

Brian

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Why not just name the indexes the same? It's not really a good idea to configure Splunk to be able to make this change. Basically, you would need to put:

[splunktcp]
route=has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

into inputs.conf on the second indexer, then send all the received data through a TRANSFORM to set the index key. This can have non-trivial consequences, so I strongly discourage it.

0 Karma

jrodman
Splunk Employee
Splunk Employee

Note that reparsing already parsed events means that they will be timestamped and possibly merged a second time, with possibly worse results than the first time.

There are also some performance costs with processing all data again even if it is already processed.

0 Karma

brianm1002
New Member

The first indexer is managed by a different department and I have no control over it. The second indexer is one that I maintain, and I'm currently using multiple indexes to segregate different sets of data based on disk space and retention requirements.

What are some of the consequences of doing as you suggest?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...