We plan to use Splunk to keep log for several java application including web server like Tomcat. Those application are using log4j with org.apache.log4j.RollingFileAppender. The partial config will be like below:
That is when the server.log reaches 10MB, it will be renamed/rollover to server.log.1 and server.log.1 will be server.log.2 and so forth...
asked 19 Mar '10, 19:12
You should monitor the whole set of
The cost is that the monitoring processor has to track more files, but this is not a very big problem in version 4.1+, and with only 20 files, this is fine in older versions too (though you might want to raise the max_fd in those versions).
answered 08 Sep '10, 18:51
Let me throw in one more scenario, say
In general, Splunk does a really good job a keeping up with log files but there are a few scenarios that just aren't covered. And since splunk automatically recognizes rotated log files (and therefore will only indexed the previously unread portions of the log file), to me it can make sense to monitor all of the log files at once. Especially, in scenarios where you can't afford to drop and events.
Also, I don't like seeing multiple sources in this kind of scenario (e.g.
Here is an example set of configs:
[monitor:///opt/java_app/server.log*] sourcetype = java_app_server
[java_app_server] TRANSFORMS-rename_source = drop_trailing_digit
[drop_trailing_digit] DEST_KEY = MetaData:Source SOURCE_KEY = MetaData:Source REGEX = source::(.*)[-._]\d+$ FORMAT = source::$1
I've found this kind of setup to work well. Anyone know of a reason why not to monitor all the rotated log files too? I know there are some advantages to monitoring a single file vs a directory like this, but I've never observed any performance impact from it.
For anyone interested, there are some additional source renaming examples here: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/4420#4420
I may be able to answer question 1 but question 2 is definitely worth investigating.
IMHO, you only need to monitor server.log
This is because when log4j detects that server.log reaches 10MB it will rollover and write its content to a new file, namely server.log.n, and then clears itself to append new output stream (which the other log file would become server.log.n+1).
answered 21 Mar '10, 07:48
Nicholas Key ♦♦