Refine your search:

10
4

It looks like there is a hard cap (10000 lines) when exporting via SplunkWeb. How, then, do I export more than 10000 lines? I really need this.

asked 26 Mar '10, 17:00

the_wolverine's gravatar image

the_wolverine ♦
5.2k172085
accept rate: 52%

edited 17 Jan '12, 12:58

hexx's gravatar image

hexx ♦
13.6k91568


5 Answers:

Here's a blog post about 10K+ events you should take a look at.

link

answered 26 Mar '10, 17:00

the_wolverine's gravatar image

the_wolverine ♦
5.2k172085
accept rate: 52%

edited 26 Mar '10, 20:51

Justin%20Grant's gravatar image

Justin Grant
1.7k181860

The 10k lines limit affects version 4.0x. This is resolved in version 4.1x and is not an issue in version 3.x

(17 May '10, 19:55) the_wolverine ♦

Just curious what's the upper limit in 4.1 now?

(17 May '10, 21:57) Chris R.
1

When "exporting results" to CSV after a search is run, from SplunkWeb, the limit is still 10,000 lines. Tested in 4.1.5.

(24 Feb '11, 16:50) Jason

As of Splunk 4.3, you can now export an unlimited number of events from the UI. Do note, however, that exporting too many events in that manner (typically, several millions) could cause Splunkweb to misbehave and possibly to become temporarily unresponsive.

If you really need to often export large number of events, we would still recommend the use of the outputcsv command and/or to run the search from the CLI.

link

answered 17 Jan '12, 12:57

hexx's gravatar image

hexx ♦
13.6k91568
accept rate: 56%

Hexx, Pease can you confirm this is fixed in 4.3? i have a scheduled saved search which emails results with CSV of results as its alert action. it seems to be truncating at 10000 rows.

This one comment by you is the only mention that this has been changed in 4.3, however i am running 4.3.1 and am still having the issue!

Could this be the reason?

limits.conf [scheduler] max_action_results = <integer> The maximum number of results to load when triggering >an alert action. Defaults to 10000

?

(04 Jul '12, 06:08) r999

@bob999 : The csv row limit for the email alert action is indeed completely unrelated to the csv export row limit in the flashtimeline which is discussed here. I believe that the limits.conf setting that you found is pertinent to your problem, although action.email.maxresults in savedsearches.conf is probably more so.

(04 Jul '12, 08:45) hexx ♦

Splunk for Excel Export will allow you to export more than 10K results:

Splunk for Excel Export on Splunkbase

link

answered 13 Sep '11, 07:40

araitz's gravatar image

araitz ♦
7.9k3925
accept rate: 46%

This is still an issue ("safety feature") as of 4.2.

I find this especially frustrating when you have a search that you have spent a long time (hours, days...) running and find it has a few events over the 10,000 limit. And you need those events exported to CSV badly.

Here is what I had to do to get them all. Important notes: this requires filesystem-level access to the searchhead that ran the search, may take a lot of CPU (unzipping), *and* if you have different data types in the search, with differing numbers of fields, perhaps you shouldn't strip out the field names as instructed below (replace awk FNR-1 with cat.)

  • First of all, save the search results via Actions->Save Results, so they don't disappear on you.
  • Next, open the Jobs window and locate your search in the listing. It will likely be listed as "Done" or "Finalized" status. Click the search text to open the search results, and copy the sid value from the end of the URL.
  • On the search head's filesystem, go to $SPLUNK_HOME/var/run/splunk/dispatch/sid
  • Copy the events directory to a separate folder (cp -r events/ events_temp)
  • Un-gzip all the files in the temp folder (gunzip events_temp/*)
  • Concatenate all the files together, ignoring the headers (this can get big. Sorry, I don't know how to do this on Windows. *nix is awk FNR-1 events_temp/* >> fullresults.csv)
  • Fullresults.csv is now the full listing of all fields in all events. As I mentioned before, I hope your number of fields are the same for all events returned, otherwise you'll have a mess to clean up.
link

answered 18 Oct '11, 22:40

Jason's gravatar image

Jason
3.6k71074
accept rate: 43%

edited 18 Oct '11, 23:11

Anyone else with a little more unix foo is welcome to let me know how to pull just a certain column out of the results based on where that field is in the header line...

(18 Oct '11, 23:01) Jason

Or just download the Splunk for Excel Add-on...

(19 Oct '11, 10:26) araitz ♦

All the other suggestions, including Excel, are only useful if you want to run the long search again... The above was for dealing with exporting the saved results of a search that took a long time to run. It took so long to get results that I didn't want to waste a lot more time running it again with outputcsv or some add-on.

(20 Oct '11, 00:09) Jason

Of course if there is a magical command such as | searchresults 13186745524.423 that uses the results of a previous saved search without running again... then I'll happily use outputcsv or Excel.

(20 Oct '11, 00:12) Jason

Have you looked at the '| savedsearch "mysearchname" ' command. If it is a scheduled search, it just resurrects the last scheduled results.

(06 Apr '12, 04:02) BobM

@Jason : there is indeed such a thing : the loadjob command.

(04 Jul '12, 08:37) hexx ♦

try exporting with the outputlookup , I know it has its own limitation of size but it bypass the 10000 lines limit.

link

answered 13 Sep '11, 04:25

gooza's gravatar image

gooza
28527
accept rate: 22%

Post your answer
toggle preview

Follow this question

Log In to enable email subscriptions

RSS:

Answers

Answers + Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×111
×110

Asked: 26 Mar '10, 17:00

Seen: 3,271 times

Last updated: 24 Dec '12, 10:41

Copyright © 2005-2012 Splunk Inc. All rights reserved.