Bug? splunk advanced searching/views does not display correctly

Question

Hi... Its been a while I have problems with searching in Google maps or geoip which the thread was going on here: geoip search results not correct

if u follow up the thread, u see it came to a point that we all realized there exists some sort of limit that does not let geoip or Google maps to display more than ten thousand... today suddenly, I realized its not only geoip/Google maps, but it actually is any kind of advanced searches. for instance if you search for all the data in normal search using "*", and if you have huge number of indexed data, (i.e. billions of data), u'll probably see all in the search but if you change the view to "Advanced Charting View" then you'll only see partially few thousands of those results... Im not sure whether this is a bug or if there is some sort of limitation in any file... but that definitely causes major problems. Does anyone have any idea how to solve this issue..?

Answer 1

1

Looking at the module references the is a limit within the XML that you can define although you want to be careful how you adjust this as it can have an impact on the performance. Hopefully with 4.3 this won't be as large an impact thanks to the HTML5 visualisations but I expect the calculations in the background also have an equally large impact;

            <module name="FlashChart">
                <param name="width">100%</param>
                <param name="height">400px</param>
                <param name="maxResultCount">10000</param>
            </module>

Also here, http://splunk-base.splunk.com/answers/10349/chart-only-showing-1000-events , Nick has some ideas on how to expand the number of results in other ways (Nick is pretty much the expert on all things XML related)

link

answered 12 Jan, 01:05

Drainy
3.2k●2●9
accept rate: 24%

1

thanks for your response...

ok, to access the module I have to go to Manage Views, right..?? there was no such thing as maxResultCount for charting view.. i saw the width, height, even maxpages, but no max result count...

also, if that was the case, how come when I search in normal search, it gives me 5 billions of data but when I just add geoip commands to the same exact search windows, suddenly only shows 19,000 events!!?!!

to be more specific, when I search for SourceIP="" I get billions of event results, but when I search for SourceIP="" | geoip SourceIP I only get 19,000..

so for the case of advanced charting also when I search for SourceIP="*" I only get 15000 while the normal search as I said were few billions...

(12 Jan, 18:50) nina15

from what I understand, that post is only about charting, time ranges and XML... my problem I think resides in lower level of splunk, is not the matter of how it is represented, but the problem is that the results are not fetched at all... the poster of that thread had a problem that the results were shown in the table, but not on the chart, only. mine is not displayed anywhere when I search in Advanced charting or when I use geoip in the normal search...

(12 Jan, 20:17) nina15

still no answers/opinions...???

(15 Jan, 21:24) nina15

1

Nina,

Can you open a support ticket? Let me know the ticket is and I will escalate for you.

(17 Jan, 18:21) dmaislin_splunk

thanks so much dmaislin for responding... i really was feeling im loosing it... anyways, i've already opened a support ticket. the number is CASE [73624]. thanks alot for ur support :)

(18 Jan, 17:33) nina15

Answer 2

The display of result rows will be limited on a per-command basis and is typically configurable in limits.conf with maxresultsrows for stanzas such as [searchresults] or [stats]. Now, even if the display of results rows is truncated, reporting commands such as stats will still show accurate aggregates, which take all input events/results into account.

For more information and details, I would recommend to read this Splunk Answer as well as this one.

Update: Adding the information below to clarify the purpose of the advanced charting view.

The important thing to understand is that the advanced charting view is designed to process the results (not the raw events!) produced by a reporting command. This is why you will see the exact same search which would show millions of events in the flashtimeline return an approximate maximum of 10,000 results in the advanced charting view. Now, if you take that same search and pipe it to | stats count, you will see that the search powering the advanced charting view will indeed process all expected events into results before letting you decide how to render those into a chart.

To sum it up : Do not use the advanced charting view to render and view events, it is not its purpose. Instead, use it to experiment with different visualization methods to apply on the results of your reporting search.

Answer 3

Alright... since this problem behaves exactly the same with any kind of searching I do, I start from a very simple search first...

I am applying field extraction using DELIMS, hence I have a field called SIP which stands for sourceip...

so now, what I want to show u is results for SIP= for a normal search, then results for same search on advanced charting view, then SIP= | geoip SIP to also have geoip behaviour... as you can see, both geoip and advanced chart only retrieve 10000 results !

these are snapshots for the normal search where you can see is up to 7 millions and I had to actually stop it since there was already enough results and it was taking much time... but the point is, the data that exists is way more than 10000...

alt text

then here it is on advanced charting:

alt text

and last but not least, geoip!

alt text

as you can see, both advanced charting and geoip have only 10000 results! the matching event is different. which indicates 2 problems actually:

both of them stop only on counting 10000 results
accuracy of these 2 as well is now a question mark! because if they both only fetch 10000 results, given that my search in both was the same, their matching events as well should be exactly the same!!

Answer 4

thanks hexx for detailed information and references you provide here.. but this is getting even weirder... :D

although the command is used by all the users without the lookup command, I tried your way and received error:

[EventsViewer module] Error in 'lookup' command: The lookup table 'geoip' does not exist.

and trying the same command in Google Maps gives this error:

Rendering...
Error : Traceback: 
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/maps/appserver/modules/GoogleMaps/GoogleMaps.py", line 53, in generateResults
    for result in getattr(job, entity_name)[offset:end]:
  File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 1280, in __getitem__
    self.job.pushValidation()
  File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 610, in pushValidation
    raise splunk.SearchException, fatality
SearchException: Error in 'lookup' command: The lookup table 'geoip' does not exist.

besides, the SPP about page, located at .../app/maps/about is a help document with this search as example:

Perform a geolocation lookup for values of the clientip field in access_combined events: sourcetype=access_combined | geoip clientip

Same as the previous example, but also perform DNS lookups in case when the value of the clientip field is a hostname and not an IP: sourcetype=access_combined | geoip clientip resolve_hostnames=true

Same as the first example, but using the geo lookup instead of the command sourcetype=access_combined | lookup geo ip as clientip

etc... etc... etc...

I even tried this:

SIP="*" | lookup geo SIP

and got the same error...

Bug? splunk advanced searching/views does not display correctly

Follow this question

Splunk Resources

Related questions