About jessieb_83

jessieb_83

I'm seeing this same error on a new build. Did you ever find an answer?

jessieb_83

Running 9.2 and getting the same error

jessieb_83

Finally figured out it was a permission issue. I didn't give splunk ownership over the index locations.

jessieb_83 · ‎04-11-2024

I left the Frozen drive to point to $SPLUNK_DB on the indexer's drive, but I'm not trying to employ frozen buckets at all. I'm trying to use the volumes on external drives for hot and cold, that's how our current instance is set up. The difference being the current is on Windows, and this new one is going to be on RHEL8.

jessieb_83 · ‎04-10-2024

I'm setting up a lab instance of Splunk Ent in prep to replace our legacy instance in a live environment and getting this error message: "homePath='/mnt/splunk_hot/abc/db' of index=abc on unusable filesystem" I'm running RHEL 8 VM's, running Splunk 9.1, 2 indexers clustered together and a cluster manager. I've attached external drives for hot and cold to each indexer. The external drives have been formatted in ext4 and set in fdisk to mount at boot every time as /mnt/splunk_hot and /mnt/splunk_cold and pointed indexes.conf by volume to them. They come up at boot, I can navigate to them and write to them. They're currently owned by root. I couldn't find who should have permission over them so I left them as is to start. I tried to enable OPTIMISTIC_ABOUT_FILE_LOCKING=1 but that didn't do anything. That being said, i suspect I've missed a step in the actions taken mounting the external drives. I wasn't able to find specifics about the way I'm doing this, so I pose the question: Am I doing something wrong, or missing a step on mounting these external drives? Is that now a bad practice? I'm stumped. my indexes.conf: [volume:hot] path=/mnt/splunk_hot [volume:cold] path=/mnt/splunk_cold [abc] repFactor = auto homePath = volume:hot/abc/db coldPath = volume:cold/abc/db thawedPath = $SPLUNK_DB/abc/thaweddb ##We're not utilizing frozen storage at all so I left it default Any advice here would be greatly appreciated!

jessieb_83 · ‎08-15-2023

Reviving an older post, I'm in this exact same position and am curious; How did you go about switching the file structure over? I've seen some people create scripts. Is that how this is getting done?

jessieb_83 · ‎12-09-2022

I don't remember at this point exactly what I did, but this is the article I used to get through it. Resync the KV store - Splunk Documentation

jessieb_83 · ‎06-17-2022

Did you find the solution? Different version, but I'm having the exact same problem.

jessieb_83 · ‎05-20-2022

The metadata is gone, but the indexed data is still there. I'm contractually obligated to maintain the last 5 years worth. I was just wondering if there was a way to look at the old data and see if I can figure out how big it was back then. Apologies being confusing. I was having a hard time putting that request to words.

jessieb_83 · ‎05-20-2022

I had not seen this before. It will be very helpful in our future planning. Thank you!

jessieb_83 · ‎05-20-2022

For the normal indexes ( not including _internal or the Drive monitor index) our retention policy is contractually specified to 5 years. I just want to see if we can extrapolate from what exists, how big it was back then.

jessieb_83 · ‎05-19-2022

I've been asked to find historical index volume information, going back 5 years, to make projections for future infrastructure and license needs. _internal is of no use, because it's cleared after 30 days. We track disk space, and I can find the disk space info for the Cold bucket on the indexer, but it's set to roll off after 60 days so that's out as well. I understand that anything like that would be slightly lower than the actual, as there are several indexes that whose index info would have rolled off, but I'm just trying to find a rough base line to track program growth. This is pretty far beyond my SPL abilities, so I would be grateful for any help!

jessieb_83 · ‎05-18-2022

@prakash007Did you ever find the solution?

jessieb_83 · ‎04-06-2022

Thanks for the reply Mayurr! I thought the same thing. I did find an app being pushed to all the UF's [and verified it's getting to the workstations] to override default with the following entry from %splunkHome\etc\apps\Splunk_UF\default\outputs.conf: [tcpout] forwardedindex.0.whitelist = .* forwardedindex.1.blacklist = .* forwardedindex.2.whitelist = (_audit | _introspection | _internal | _telemetry) forwardedindex.fileter.disable = false It seems like the 0.whitelist entry is unnecessary but I wonder if that is actually conflicting with gathering audit info.

jessieb_83 · ‎04-06-2022

With little to no Splunk experience, I inherited a 7.2.3 windows deployment (We're closed network and I'm not cleared to upgrade yet) I've been finding little things here and there. One of the bigger ones being I'm ONLY getting _Audit logs from the Splunk servers; I'm not getting any audit input from any work stations, or other production servers. I've been dredging the boards for 3 days now and haven't found anything that seems along this line. I've checked the %Splunk\var\log\audit.log on several and the host's audit logs are getting input, but they're not getting ingested. I've gone through the deployment_app input.conf and output.conf files and don't see any glaring indications. So, I'm asking for ideas on other things to check.

jessieb_83 · ‎11-23-2021

https://wiki.splunk.com/Community:DeployHardenedSplunk This walk through covers binding to LDAP.

jessieb_83 · ‎08-28-2020

Trying to figure out what gets added upon running OUTPUTCSV vs manually exporting to CSV and how to remove it. We run a weekly audit I'd like to automate and then run a script to zip the resulting file. I did some experimenting and found that using outputcsv or outputlookup instead of exporting manually is almost tripling the file size of the output, from 300 Mb to 1.11 Gb. Exact same search criteria, the exact same time range, returning the exact same number of results. I also checked https://docs.splunk.com/Documentation/Splunk/8.0.5/SearchReference/Outputcsv and removed the internal fields it listed as being added. Any insight here would be great!

Posts	17
Solutions	1
Karma Given	6
Karma Received	1
Member Since	‎11-15-2019

Online Status	Offline
Date Last Visited	Thursday

Unusable Filesystem

How to find historical index volumes?

Why is _audit index not capturing all hosts?

What does outputcsv add to

Re: Receiving "Client is not authenticated" Error

Re: Has anyone else run into this Product Version ...

Re: Unusable Filesystem

Re: Unusable Filesystem

Unusable Filesystem

Re: Migrate Splunk from Windows to Linux question

Re: KV Store status failed

Re: KV Store status failed

Re: How to find historical index volumes?

Re: How to find historical index volumes?

Re: How to find historical index volumes?

How to find historical index volumes?

Re: WHat does it mean this error regarding bundle ...

Re: Why is _audit index not capturing all hosts?

Why is _audit index not capturing all hosts?

Re: Update BindDNpassword on deployer in authentic...

What does outputcsv add to