Getting Data In

Input data from directory sourcetype changing

zitacheung
Engager

Hello every one,

I am a new user of the splunk.

I have facing a problem that input the log file from directory.

In that directoy every 15 mins a new log file will be generate.

I use the Manager -> Data Input -> Files & directories to input file.

suppose I define a new sourcetype "ABC" in the first of input.

I found that every 15 mins the new files is generate and import in the splunk. A new sourcetype is create ABC-1 ...ABC-12....ABC-256.

Is there anything I can do to prevent the sourcetype keep increase?

Tags (1)
0 Karma
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

There is a good solution for this documented on answers here:

http://splunk-base.splunk.com/answers/10154/sourcetype-would-increment

I would start with trying to implement a similar solution and see if it works for your situation. If it doesn't, let us know and we can try to offer additional information.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

There is a good solution for this documented on answers here:

http://splunk-base.splunk.com/answers/10154/sourcetype-would-increment

I would start with trying to implement a similar solution and see if it works for your situation. If it doesn't, let us know and we can try to offer additional information.

zitacheung
Engager

Thanks jbsplunk,
I have one more question, is it normal that the number of wrong sorcetype keep increasing as more file be recognised

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

You can't surgically delete data from Splunk. The data will be aged out via your retention policy, or you can clean all the data from the index in which these sourcetypes exist. But, after data has been indexed, you can't alter the metadata or delete specific records yourself.

zitacheung
Engager

Thanks I can make all the new input log file become the same sourcetype.
But how can I change the sourcetype of log that already imported.
And how can I remove the wrong sourcetype(ABC-1...ABC-300)?

0 Karma

zitacheung
Engager

Thanks for helping me. I have try to modify the props.conf and the transforms.conf but I cannot fix it,
the file i change locate at "C:\Program Files\Splunk\etc\system\local"
the code I use is

[source::C:\logfile*]
TRANSFORMS-Bandwidth=fix_Bandwidth


[fix_Bandwidth]
REGEX = .
FORMAT = sourcetype::BandWidth
DEST_KEY = MetaData:Sourcetype

But I still cannot get it.And in the link it mention I can tag the wrong sourcetype to the correct one. May I know where I can do it??
Many Thanks.

Zita

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...