|
A query to count tag=pci entries by eventtype (and happens to be part of the application): ...results in this error message:
I have about 156m global indexed events on a single indexer and every event is tag=pci. I'm hoping to understand where an upper limit is defined and what I may be doing wrong. Thanks! ~Nate |
|
Your tag=pci is expanding to more than the number of search terms allowed, which is currently 420. You need to reduce the number of terms with a more restricted tag, perhaps filtering out those results with another search. Future versions of Splunk should allow a massive increase of this size, but for now it's fixed at 420. |
|
The problem is not the number of search results with the tag, but the size of the search query. I'm assuming that you have items tagged |
|
For this case, Johnvey's suggestion will be vastly more efficient. If you can come up with a variety of situations where you run into this ceiling currently, that will help us design the product to better serve those types of needs. The problem you're bumping into first is that our query parser handles disjunctions with a recursive approach. Infinite recursion on the C stack leads to crashes, so we cut it off at some number before things will break. While our query parser could be redesigned to recurse via iteration or to have successive simplifications or other patterns, it's a lot of work, and it won't necessarily make these searches perform. I sort of think the best channel for this is support, where you provide the set of cases where you bump into this ceiling, which can be broken into alternative query responses, or possibly Enhancement Requests that lead to a better design. |
What is the reason for tagging every event in the system with 'pci'? A simpler way to achieve this is to run: * | top limit=100 eventtype
Didn't see johnvey's comment. This is a query in the Splunk PCI app. We happen to be using Splunk exclusively for PCI. Based on the comment, perhaps a fix is in order for the upcoming release of the latest PCI app.