Filter events from forwards

This is quite thoroughly discussed in the docs, please see: http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Keep_specific_events_and_discard_the_rest

If you have a heavy forwarder, the settings should be placed there. In all other cases, the settings should be on the indexer.

In short, what you need to do is

1. In props.conf:

   [your_applog_sourcetype] TRANSFORMS-keep_only_bidprice= setnull,setparsing

2. In transforms.conf:

[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue

[setparsing] REGEX = BIDPRICE DEST_KEY = queue FORMAT = indexQueue

UPDATE:

What this does is to apply a set of rules to your incoming data. First, in props.conf you tell splunk that all events of a certain sourcetype should pass through a transform (or in this case, two transforms - setnull and setparsing).

Then, in transforms.conf, you state how data should be treated. setnull routes data to the nullQueue, i.e. throws it away. setparsing will match all events containing the string BIDPRICE and send these on for parsing and indexing.

If this does not work for you, then please tell us more about your setup, and post the relevant parts of props.conf and transforms.conf, along with a few sample events from your application log file.

You should know though that this will only work for new data coming in, and not alter any existing events already in your index.

UPDATE AGAIN:

DEST_KEY = queue is where you state what parameter should be altered by the transform.

FORMAT = nullQueue is the value that will be set for the parameter. (nullQueue is something like /dev/null)

Think of it as saying queue = nullQueue.

So what happens when use TRANSFORMS-set = setnull, setparsing in props.conf is;

• set the destination of all events to nullQueue (since all events will match the dot (.) in the regex, i.e. throw them away.

• then, for those matching the regex in setparsing, i.e. BidPrice, the destination should be re-written to be the indexQueue, which is where events normally for indexing.

Hope this helps,

Kristian