Maximum number of historical concurrent system-wide searches has been reached

[sureshsala]

What does this message means
The maximum number of historical concurrent system-wide searches has been reached. current=8 maximum=8
Why I am getting this message.
How to stop this message.

[harsmarvania57]

Hi @sureshsala

I guess you are running with default splunk configuration, then as per limits.conf

base_max_searches = <int>
* A constant to add to the maximum number of searches, computed as a multiplier
  of the CPUs.
* Defaults to 6

max_searches_per_cpu = <int>
* The maximum number of concurrent historical searches per CPU. The system-wide
  limit of historical searches is computed as:
  max_hist_searches =  max_searches_per_cpu x number_of_cpus + base_max_searches
* Note: the maximum number of real-time searches is computed as:
  max_rt_searches = max_rt_search_multiplier x max_hist_searches
* Defaults to 1

So in your case you have 2 CPU then max_hist_searches = ( 1 * 2) + 6 = 8 and this error showing that you are running 8 searches at same time and splunk ca run maximum 8 searches with default configuration as per your server specification.

Now you have 2 options to remove this error

1. Increase max_searches_per_cpu but I'll not recommend this because it will reduce your search performance.
2. Add more CPU to your splunk servers.

If above options are not feasible then you need to run small number of searches concurrently.

Thanks,
Harshil