Splunk integartion with Flat file

[mukhan1]

Hello Team,

I have a .log flat file this file give us the data whenever we open and run command it give us some logs, now i am integrating this .log file with Splunk but it is not integrating. I ran following command to integrate it,
"/splunk/bin ---> ./splunk add monitor [file name]" it give me message that file has been added to monitor list. 

However i don't see this file on my Splunk, further if i have this file on Splunk how it will takes data from it whenever we run any command, also this .log file doesn't store data in any other directory whenever we close the file data disappears. Please note the OS im using is Sun Solaris 

[PickleRick]

OK. First things first.

1) Do you have _anything_ ingested from this forwarder? Check your _internal index for any logs coming from this UF

2) If you didn't specify a destination index, the forwarder will be trying to send the data to the default "main" index - it's not the best idea.

3) Check the output of

splunk list inputstatus

and

splunk list monitor

And verify if that file is being read by your forwarder

[mukhan1]

@gcusello thanks for your reply, i have checked the connection by telnet the Splunk it is successfully connected, also cross checked it by adding other path of log files. It is adding successfully. 

I have added the file path manually but still file is not showing on splunk GUI. Further going through the doc you provided hope it will help.

[gcusello]

Hi @mukhan1,

ok, perform also the check I hinted to verify connection because telnet is important but it isn't the only check to perform: you could have an open connectin but you could not correctly configure outputs.conf in your Forwarder!

let me know if you solved or if I can help you more.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

[mukhan1]

Hey @gcusello,

Already checked the outputs.conf file it is working fine, i don't think this issue is related to outputs.conf because if the issue is with outputs.conf then other path of logs also failed to send logs however im receiving logs from same Host but unable to fetch .logs file into Splunk.

This file is actually have code in it, whenever i open this file and run command then it will give me some logs against the command i run. I want to ingest those logs into Splunk. Please remember once the file is close the data will wiped also. No other records of these logs.

I think Splunk doesn't support this type of file ingestion with Splunk.  

[gcusello]

Hi @mukhan1,

I'm confident that you can read this file with Splunk: Splunk can read every kind of text file!

Check if the path and filename i the stanza header is correct and then check if the user you're using to run Splunk can read that file.

Ciao.

Giuseppe

[mukhan1]

@gcusello just for ur understanding ,
No, this not .txt file this is the flat file

[gcusello]

Hi @mukhan1,

a flat file, for my knowledge is a text file continously updated.

If your flat file is a text file continously updated Splunk can read it.

Ciao.

Giuseppe

[mukhan1]

@gcusello  yes you're correct splunk can integrate any text file but my issue is that i have .log file namely as "F.JBASE.JED.AUDIT.LOG" this is the file name this file is not a text file

[gcusello]

Hi @mukhan1,

have you an API to extract the content of this file?

If yes, you could develop a script that periodically extract the logs and writes them in a text file readable from Splunk or directly in Splunk.

Splunk developed a connector (e.g. for wineventlog) to extract not text files.

Ciao.

Giuseppe

Ciao.

Giuseppe

[gcusello]

Hi @mukhan1,

at first read:

https://docs.splunk.com/Documentation/Splunk/latest/Data/Monitorfilesanddirectories 

https://lantern.splunk.com/Splunk_Platform/Getting_Started/Getting_data_into_Enterprise 

then check if the connection between the Forwarder and Splunk is open running a simple search on Splunk:

index=_internal host=<your_forwarder_host>

if you have events the connectin is established, if not you have primarly to configure the connection.

If the connectin is ok, then, you should have in $SPLUNK_HOME/etc/system/local and inputs.conf file.

In this file you should have a stanza that starts with [monitor://yourfile]

take the path you have after monitor:// and run ls -la your path to see if your monitor stanza really reache the file to monitor.

The issue could be have that the path isn't correct or that the user you're using to run Splunk hasn't the grants on that folder.

Manually modify the inputs.conf stanza and restart Splunk on the Forwarder.

Ciao.

Giuseppe