I am encountering the following error when distributing search between a 4.2 search head and a 4.1.5 indexer:
Asynchronous bundle replication might cause (pre 4.2) search peers to run searches with different bundle/config versions. Results might not be correct
Questions:
Is a 4.2 search head designed to be compatible with a pre 4.2 indexer via distributed search?
Is there any way to resolve this error besides upgrading the indexers to 4.2?
I'm actually getting this same alert with two 4.2 indexers running distributed search. Same advice to set sync_bundle_replication = true
?
I still see errors in the logs such as:
Most recent bundles for search peer went missing: /opt/splunk/var/run/searchpeers/splunk.domain.com-1300878227
Failed to create a bundles setup with server name 'splunk.domain.com'. Using peer's local bundles to execute the search, results might not be correct
Could it be a naming problem? Splunk host is splunk1.city and splunk URL is splunk.domain.com.
does the message go away? how long has it persisted?
Yes, a 4.2 search head is compatible with pre-4.2 indexers. However, by default, 4.2 tries to use asynchronous bundle replication, which yields better performance. The warning is that the bundles may not be perfectly synchronized. To disable this behavior (and not get the usually innocuous error), add to etc/system/local/limits.conf:
[search]
sync_bundle_replication = true
More practically, this means that changes to searchtime configuration may remain stale for a relatively short window of time. For example, if the way a field is extracted is modified, the 4.1 search node (indexer) may not perform searches with this new field extraction until a small number of minutes later.
Well, the error isn't an error. It's a warning that, because configs are not synchronously replicated to remote indexers, different indexers could, for the same search, use different configurations, which would make your search results look wrong.
You could make it go away if you really want by completely disabling bundle replication and mounting the search head config onto each indexer. That particular trick is better-supported in 4.2 anyway, and really it's probably easier to upgrade the indexers than to do that.
The mount method of config availability is probably not worth considering until you have fairly large amounts of searchtime data, such as larger lookups. (Our replication algorithm is currently -- 4.1/4.2 -- a bit naive, so a frequently changing small file will cause resending of a static large file.) This may be the common case for large, many indexer environments, I don't personally know.