|
I'd like to index a directory of 50,000 gzip files. The files range in size from 1 KB to 5 MB. Can Splunk monitor these files without first unpacking the gzips? |
2 Answers:
|
The good news is "YES, Splunk can index gzip files as is!" The bad news is, Splunk will monitor these files one at a time, instead of in parallel. Because it is not possible to predict the uncompressed size of a gzip file, Splunk processes these files in sequence for better control of disk allocation. With respect to performance, this is not ideal for handling 50k files so please consider uncompresing them before having Splunk monitor them to take advantage of Splunk's multi-threading file monitoring capabilities. |
