Refine your search:

What filesystem is recommended for maximum performance on centos/redhat 5.x? (64 bit)

We were thinking either EXT3 or XFS as they are what we have used the most but wanted to get the official splunk recommendation. This will be for our indexing servers that will be doing high volume indexing and searching and storing data for long periods of time.

Overall hardware config will be:

  • 2x Quad Core Intel CPU (L5410)
  • 16GB Ram
  • 16x1TB SATA drives (7200 rpm) in an eSAS chassis.
  • LSI RAID 5 or 10 depending on performance

asked 01 Jul '10, 22:34

zscgeek's gravatar image

zscgeek
3481216
accept rate: 42%


One Answer:

Raid 10 is supposed to have the best performance, and I've heard that recommended from several sources. I'm not sure the filesystem makes as significant a difference as long as its supported (someone please correct me if I'm wrong.).

http://www.splunk.com/wiki/Community:HardwareTuningFactors

link

answered 01 Jul '10, 23:04

Jeremiah's gravatar image

Jeremiah
2501210
accept rate: 25%

My concern with filesystems is that some handle large number of files better then others. Splunk can end up creating insane numbers of files when you get to holding several TB worth of logs in the raw dirs. EXT3 is the OS default of course but I would not mind knowing if people have hit limitations with it.

(02 Jul '10, 14:34) zscgeek

I use XFS for my primary splunk partition ($SPLUNK_HOME/), and then I use ext3 for the $SPLUNK_HOME/var/run partition (so that all the search jobs and temporary files that splunk creates doesn't cause fragmentation issues with the actual indexing process). (There is one minor bug with outputlookup because of my separate partitions, which has been reported and should be fixed soon.) I've also got all my partitions on top of LVM for easy partition reallocation.

(02 Jul '10, 15:11) Lowell ♦
Post your answer
toggle preview

Follow this question

Log In to enable email subscriptions

RSS:

Answers

Answers + Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×238
×134

Asked: 01 Jul '10, 22:34

Seen: 2,190 times

Last updated: 01 Jul '10, 23:04

Copyright © 2005-2012 Splunk Inc. All rights reserved.