Getting Data In

On a Universal Forwarder I did a "splunk clean all", changed some things and started the forwarder, but one of my monitor stanzas is not forwarding...

neiljpeterson
Communicator

The forwarding from this directory was working previous to the clean. My understanding was this was supposed to clean out all indexes including the fishbucket, causing splunk to forget was had already been indexed and reindex it all. There have even been new events generated since clean, which I would certainly expect to be forwarded and indexed, but I am not seeing anything.

Other stanzas, from the same inputs.conf, are working, like performance data.

What am I doing wrong here?

For completeness sake, this is the stanza I am expecting to see data from.

[monitor://C:\Websites\logs\...\*]
disabled = false
sourcetype = app_logs
index = app_logs
0 Karma
1 Solution

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

View solution in original post

0 Karma

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

0 Karma

sunrise
Contributor

I don't know whether "splunk clean all"command delete fishbuckets or not.
But if you delete fishbucket manually and start UF instance, it will retransfer that monitoring data to Indexer.

cd $SPLUNK_HOME/var/lib/splunk/fishbucket
rm -fR *
0 Karma

neiljpeterson
Communicator

Unfortunately I am on windows... but I did another clean all and it did empty out the fishbucket dir.

0 Karma

MuS
Legend

Hi neiljpeterson,

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.
On an indexer splunk clean eventdata -index _fishbucket will do the magic.

cheers, MuS

0 Karma

neiljpeterson
Communicator

This is what I found

 04-01-2014 09:18:42.197 -0500 INFO  TailingProcessor - Parsing configuration stanza: monitor://C:\Websites\logs\...\*.
0 Karma

MuS
Legend

check your forwarders splunkd.log for anything related to tailingprocess regarding this input

0 Karma

neiljpeterson
Communicator

splunk clean all does this. I just did it again as a test.

0 Karma

MuS
Legend

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.

0 Karma

neiljpeterson
Communicator

one or the other or both?

0 Karma
Get Updates on the Splunk Community!

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...