We have a large number of logs deserve a different sourcetype, but are effectively from the same application, and have the same format. I use a prefix on the sourcetype to indicate what app they came from, so they can be linked in searches using sourcetype=TOM* for example
eg. TOMEnvoyPayments, TOMMarketPermissions, TOMpaymentsbpmadmin
I want to extract the same field from the same part of the log entries, using the same regex. Obviously I could do this by specifying a stanza for each sourcetype specifically (three stanzas, in this example), but is it possible to use a wildcard to condense this into a single stanza, eg.
[TOM*]
REPORT-TOMgeneral = misc_MsgType_startofline
I know wildcards are possible in [source::blah] stanzas, I am interested in sourcetype stanzas specifically.
Cheers,
Glenn
This is still working in 8.1x
But also please vote for an official method for wildcarding sourcertypes :
Splunk, please add official support for wildcards in sourcetype stanzas in props.conf
: for example, [TOM*]
.
I don't think we should be recommending this hack very much. If this is a common need, then make Splunk add proper support for wildcarding sourcetypes. This above thing relies on a very internal interaction between the regex expression and the way the matching is accomplished and it could easily change in the future.
It does work currently, but I would definitely recommend against leaning on it heavily -- leaving it installed for other parties at customer sites, putting it in apps, leaving an employer as a splunk admin where critical functionality relies on this without specifically passing on that information, etc, are all things that may do others a disservice in the longer run.
Agree with Jrodman. This "hack" is not supported. Actually this caused unexpected behavior in an example user tied.
Splunk doc should mention this hack is not supported.
Masa, your view contrasts strongly with that of Jason Conger, who in July 2014 published the Splunk blog post "Quick Tip: Wildcard Sourcetypes in Props.conf".
Jason writes "Here is a quick one [tip] I use often".
Different Splunk staff are sending (or have sent) different signals about this technique.
Splunk, could you please clarify the support for this technique?
I think folks have been trying to do this for 5+ years. Much easier to use a simple pattern then add 100 rules. Why not add it officially?
Hey I'm curious about the use of this "hack", because it still seems to work (in 6.1.4). Was it implemented as a feature? Is it still just a hack that may or may not go away at some point? Thanks.
Tested with Splunk 7.1.3., still works.
Hesitant to use it though, looking for supported alternative.
It doesn't seem that a supported alternative exists.
Yes, you should be able to use:
[(?:::){0}TOM*]
Does this work for prefix wild card as well ? We have source types like UIT1_XXX_APPLICATION, UIT2_XXX_APPLICATION etc. The above command does'nt seem to be working.
Interesting little hack there. It seems to be working, but I'm a 'tad uncomfortable using it without fully understanding what it is doing.
Could someone shed some more light as to why that stanza header works?
It seems that official support for wildcarding sourcetypes in this context is something that comes up pretty often. Enhancement coming soon? 🙂
Please send your use cases into support to be passed along to product management.
Also tried this, [(?:::){0}httperr-*] and it worked perfectly for my http error files which had 10 different source types
Thanks gkanapathy for the solution. Glenn please mark this post as answered...
My application has sourcetypes coming in from (ie. resp101, resp201, resp202, resp203, etc..)
and in my props.conf, all I needed to define for the
[(?:::){0}resp*]
Works wonderfully. Someone from Splunk should put this somewhere in the props.conf documentation.
klee
My mistake. The regex stanza need to be [(::){0}TOM*]
. Updated original
Hi, I tested this, but it doesn't seem to work?
I add this to props.conf:
[TOMConfirms]
REPORT-TOMall = misc_MsgType_startofline
... and "run |extract reload=t" and I get the "MsgType" field extracted correctly, it shows in the "pick fields" list.
I add this to props.conf:
[::TOM*]
REPORT-TOMall = misc_MsgType_startofline
And run "|extract reload=t" and the MsgType field is not being extracted for ANY TOM* sourcetypes.