|
I have a group of about 60 servers saturated among Windows, Linux, and AIX operating systems. Each system is configured to forward their process information to my primary index server at a frequency of about 1 minute. For AIX and Linux, we use the ps command/sh; and on Windows we use the wmi-input script. At the moment, we have a scheduled savedsearch running at about the same interval (1 min) which takes all the individual events from separate hosts and aggregates them into single events using mvcombine. This result is then written into a separate (custom) index using the collect command. For example - an event in our "process" index might look something like this:
Now, I want to do two things with this data:
Any ideas or comments? Thanks in advance --edit-- Furthermore, we have already tried many things in regards to question 1 above. The most promising solution is to use a subsearch (for newly created processes):
With this search command, we are able to identify those process which are there now, but wasn't there in the previous event. However when we tried to save this result back into the "process" index, by appending the following to the above command:
.. there are some issues we were not able to solve. First problem is that the result is written into the index with search_name=KelProcessCollect, and not with the actual name of our saved search. Fine. So to prevent duplicated data in our "process" index, we tried | collect index=main marker=test1=test2. And when we look in the main index for this result, "test1=test2" is written in the _raw, but cannot be located as a field; which is not so bad, since we can still search for search_name=KelProcessCollect in the main index and find what we need. The real problem is the Name field now reverts back to the original Name field from the process-index - not the single process-name, but the long string of text, ie. Name="/sbin/init [kthreadd] [ksoftirqd/0] [kworker/u:0] [migration/0] [cpuset] [khelpe..." So this doesn't help one bit, since we can't identify which of those Names was the actual new process. We have tried to rename Name AS Name2, but the results in main-index only shows the _raw data; whats wrong? Is there any way to manually format the output of what gets written with collect? or perhaps a way to remove old metadata and assign new ones? Many many thanks! |
