Refine your search:

For a particular sourcetype I need to have two fields extracted at index time and also 10+ fields extracted at search time. what is the syntax to do this? should I have multiple sourcetype stanzas in props.conf for the same sourcetype or can i combine index and search time extraction into the same stanza?

asked 01 Jul '11, 08:40

tpsplunk's gravatar image

tpsplunk
383215
accept rate: 28%


One Answer:

You can have different extractions in the same stanza, that isn't going to be a problem for you. Here is an example of something you might do

[sourcetype]

EXTRACT-searchtime = (?msi)search\s+time:\s+(?<searchtime>[^\r\n]+)[\r\n]
TRANSFORMS-indextime = indextimeextraction
link

answered 01 Jul '11, 08:57

jbsplunk's gravatar image

jbsplunk ♦
10.6k1625
accept rate: 48%

edited 01 Jul '11, 08:57

ok excellent, that makes sense. currently i'm using a delimited based search time extraction. this will probably cause an overlap where the field i want to change to index time extraction will also be search time extracted. will that cause any weirdness?

(01 Jul '11, 09:52) tpsplunk

I am not sure why you'd need to do both a search and index time field extraction at the same time, but this could definitely cause some wierdness. Most of the time search time field extraction is the way to go. I'd say a good 80% of the time, index time field extraction isn't the right solution. It can be quite expensive, and usually isn't worth the cost.

(01 Jul '11, 10:19) jbsplunk ♦

i don't really need to do both- its just that the delimiter based search time extraction is also going to pick up the field that i'm adding to the index time extraction.

(01 Jul '11, 11:03) tpsplunk

Then, I guess my question becomes why is search time field extraction us a delimiter not sufficient to meet your requirements, and how is the index time extraction going to meet that requirement?

(01 Jul '11, 11:19) jbsplunk ♦

i don't want to get into the "should i be using index time extraction" discussion. let's just assume that i need to and focus on how/if i can use delimiter based search time field extraction and index time field extraction where the index time field extracted field will also be picked up by the delimiter based search time extraction. from the docs it looks like i need to set a fields.conf stanza for that field with INDEXED=FALSE, but that seems counter-intuitive (http://www.splunk.com/base/Documentation/4.2.2/Data/Configureindex-timefieldextraction ).

(01 Jul '11, 11:34) tpsplunk

oh and i should say i'd like to keep the delimiter based search time extraction because its very simple for me to maintain (i.e. i don't have to do anything when devs add new logging fields as long as they follow the delimiter format)

(01 Jul '11, 11:40) tpsplunk

That is correct.

(01 Jul '11, 12:10) jbsplunk ♦

note that if you have a distributed environment you will end up with the index time props and transforms.conf on your indexers and the search time props and transforms.conf + fields.conf on your searchhead(s)

(15 Jul '11, 10:41) tpsplunk
Post your answer
toggle preview

Follow this question

Log In to enable email subscriptions

RSS:

Answers

Answers + Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×1,640
×449
×341
×72

Asked: 01 Jul '11, 08:40

Seen: 872 times

Last updated: 15 Jul '11, 10:41

Copyright © 2005-2012 Splunk Inc. All rights reserved.