search-time vs index-time field extraction

We do not recommend that you add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time, such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see "Index time versus search time" in the Admin manual.

With those caveats, there are times when you may find a need to change or add to your indexed fields. For example, you may have situations where certain search-time field extractions are noticeably impacting search performance. This can happen, for example, if you commonly search a large event set with expressions like foo!=bar or NOT foo=bar, and the field foo nearly always takes on the value bar.

Conversely, you may want to add an indexed field if the value of a search-time extracted field exists outside of the field more often than not. For example, if you commonly search only for foo=1, but 1 occurs in many events that do not have foo=1, you may want to add foo to the list of fields extracted by Splunk at index time.

In general, you should try to extract your fields at search time. For more information see "Create search-time field extractions" in the Knowledge Manager manual.

[ from http://www.splunk.com/base/Documentation/latest/Admin/Configureindex-timefieldextraction ]