As title suggest, i want to index internal logs only and forwards all other logs to forwarders or idxs.
Here is the setup :
Following is default output.conf:
[tcpout]
maxQueueSize = auto
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup)
forwardedindex.filter.disable = false
indexAndForward = false
Here is what I have done outputs.conf
[tcpout]
defaultGroup=noforward
disabled=false
[indexAndForward]
index=true
selectiveIndexing=true
[tcpout:forwarders]
server:<forwarders>:9997
Below is my props.conf
[default]
TRANSFORMS-forwardit = forwardit
[host::*.foo.splunk.com]
TRANSFORMS-routing = indexing
Below is transforms.conf
[forwardit]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = forwarders
[indexing]
REGEX = .
DEST_KEY = _INDEX_AND_FORWARD_ROUTING
FORMAT = local
Essentially all internal indexes should stay within cluster indexes but rest of index or logs forwarded to external indexes.
I believe you are looking for below: Note: you can only index _internal logs using this method.
Well. This tells me i have to use inputs.conf to ensure routing. By default I want to forward logs. But if i see internal logs i will index it and not forward it. This basically is telling me i have to put _INDEX_AND_FORWARD_ROUTING on all internal inputs.conf this can cause the issue.
For me by default i want to forward new indexes created and internal indexes has to be indexed locally. My thoughts is , setup tcpgroup for forwarders and in outputs.conf and inputs.conf i should modify but not sure how.
Hi, if the intention is to index all internal indexes, i have set _INDEX_AND_FORWARD_ROUTING and
_TCP_ROUTING which can cause the issue.
The Splunk Doc is very much detailed on the question you have asked. check it out using below link.