Getting Data In

Log file with CRLF not producing multiple indexed events

shocko
Contributor

I'm using Splunk Enterprise 9 on Windows Server 2019 and monitoring a simple log file that has CRLF lines endings and is encoded as UTF8. My inputs stanza is as follows:

 

[monitor://c:\windows\debug\test.log]
disabled = 0
sourcetype = my_sourcetype
index=test

 

Consider two consectuive lines in the log file

 

Some data 1
Some data 2

 

When indexed this creates a single event rather than my expectation of 2 events.

Where am I going wrong?

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the props.conf settings for [mysourcetype]?

---
If this reply helps you, Karma would be appreciated.
0 Karma

shocko
Contributor

I dont have one as I didn't think I needed one for something this simple. I have tried just now though adding this to no avail

 

[my_sourcetype]
SHOULD_LINEMERGE = FALSE
LINE_BREAKER = ([\r\n]+)

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Hi
This should work. If you are looking spec file for props.conf you see that SHOULD_LINEMERGE = true for unknown reason? It should be false for almost 100% of cases.
r. Ismo
0 Karma

shocko
Contributor

Is there any tooling (btool perhaps) that would tell me what props/transfroms are being applied to my sourcetype? Even if I drop the sourcetype form my inputs.conf the issue perists 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You could use btool to look what is applied to your sourcetype, BUT if there is also apply to source or host something those will override sourcetype definitions. Unfortunately I don't know if there is any tool which can show to you which of those are applied ;-(

0 Karma

shocko
Contributor

So I spun up a new Splunk instance in Podman (completely clean) and ingested the same file and the behaviourt is the same with no line breaking! This is with UTF8 encoding and CRLF or LF endings. So I went into the UI and created a sourcetype for it:

 

[netlogon]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = 1

 


Now working on 9.2.0.1 but not on 9.1.2 😞

0 Karma
Get Updates on the Splunk Community!

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...

Archived Metrics Now Available for APAC and EMEA realms

We’re excited to announce the launch of Archived Metrics in Splunk Infrastructure Monitoring for our customers ...