Getting Data In

Sourcetype would increment?

hochit
Path Finder

Hi,

I found the sourcetype that I've been set was growing with incremental digit. e.g. sourcetype=abc-1, sourcetype=abc-2, sourcetype=abc-3 ...

Do you know why this happens?

Thanks! Philip

Tags (1)
1 Solution

rotten
Communicator

It isn't so obvious how to force a sourcetype using props.conf. Here is how we do it:

In props.conf on the indexer you need to identify the files you want to map to a sourcetype using some sort of expression that looks similar to the inputs.conf expressions:

[source::/my/appserver/path/.../logs/access*]
TRANSFORMS-appserver_access = fix_appserver_access_sourcetype

Then you need to put this in transforms.conf:

[fix_appserver_access_sourcetype]
REGEX = .
FORMAT = sourcetype::appserver_access
DEST_KEY = MetaData:Sourcetype

Then you need to restart the indexer. All new files that are identified will get the new sourcetype. The files it has already seen will still get mapped with the old sourcetype.

--

Another note: If you want to force the sourcetype on the inputs.conf you do that on the forwarder.

--

You can also tag sources that have already been indexed to give them an sourcetype alias. This can get really tedious if you have hundreds and hundreds of files that were indexed with the wrong sourcetype.

View solution in original post

rotten
Communicator

It isn't so obvious how to force a sourcetype using props.conf. Here is how we do it:

In props.conf on the indexer you need to identify the files you want to map to a sourcetype using some sort of expression that looks similar to the inputs.conf expressions:

[source::/my/appserver/path/.../logs/access*]
TRANSFORMS-appserver_access = fix_appserver_access_sourcetype

Then you need to put this in transforms.conf:

[fix_appserver_access_sourcetype]
REGEX = .
FORMAT = sourcetype::appserver_access
DEST_KEY = MetaData:Sourcetype

Then you need to restart the indexer. All new files that are identified will get the new sourcetype. The files it has already seen will still get mapped with the old sourcetype.

--

Another note: If you want to force the sourcetype on the inputs.conf you do that on the forwarder.

--

You can also tag sources that have already been indexed to give them an sourcetype alias. This can get really tedious if you have hundreds and hundreds of files that were indexed with the wrong sourcetype.

Simeon
Splunk Employee
Splunk Employee

Splunk will try to learn your sourcetype automatically. As a result, similar source files with varying event punctuation may show up with the same name with incremental digits.

The correct way to fix this is to explicitly set the sourcetype for your data source. This can be done within inputs.conf or via the gui. In some scenarios, you may not be able to explicitly set this as the input source has many sourcetypes. In that case, you must use a props.conf setting to identify individual sourcetypes (somewhat of an advanced topic).

If you want to get rid of the automatic sourcetyping, you can disable the learned sourcetypes although that is not recommended. To do this, you can simply disable the learned app via the UI. Unfortunately, you cannot go back and change the indexed sourcetype value without re-indexing.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...