Refine your search:

I have noticed that every 20-30 minutes we see a hugh CPU spike on two of our indexers. And looking at the splunkd.log file, the CPU spike co-incides with a large number of errors as follows:

02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/monitor%3A%2F%2F%2Fvar%2Flog] modtime = 1321526706.498255000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/monitor%3A%2F%2F~%2FLibrary%2FLogs] modtime = 1321526706.628675000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fcpu.sh] modtime = 1321526706.896315000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fdf.sh] modtime = 1321526706.990800000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fhardware.sh] modtime = 1321526707.087146000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Finterfaces.sh] modtime = 1321526707.179597000

perhaps 500 or so lines...

I am assuming this is related to the deployment manager or bundle replication, but a quick google hasn't given me much to go on...

Does anyone have any suggestions where to start looking?

asked 27 Feb '12, 13:19

nickhillscpl's gravatar image

nickhillscpl
13816
accept rate: 50%


3 Answers:

After a bit of a wait, Splunk suggested that this could be a bundle issue between the 4.3 searchhead, and 4.2.x indexers.

Whilst this was not listed as a possible problem on the compatibility matrix, (here: http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Whatisdistributedsearch#Cross-version_compatibility) since upgrading everything to 4.3.1 the problem has been resolved.

link

answered 10 Apr '12, 01:52

nickhillscpl's gravatar image

nickhillscpl
13816
accept rate: 50%

Do you have any update on this error? We are experiencing the same problems here.

link

answered 09 Mar '12, 00:24

ttrolf's gravatar image

ttrolf
412
accept rate: 0%

No. I also have a ticket open with spunk support since the 27th, and have not heard anything back yet.

Can you post an overview of your environment, incase there are any similarities i can update the ticket with.

We have 1 search head, with a pool of two indexers. We see the errors on BOTH indexers at the same time.

Indexer1 is a deployment server, which deploys to about 12 universal forwarders, and we have a few manually configured forwarders too.

Our entire deployment is hosted in AWS.

i'll update this ticket when i get an answer from support, but in the meantime.... any ideas????

(09 Mar '12, 00:44) nickhillscpl

We have a similar setup, running on 4.2.1, and 4.2.0 on our two indexer servers (Linux 64bit on search heads and indexers). I see that both servers threw this message but I cannot see any CPU problems on any of the servers. We use HW based servers. We around 80 forwarders connected to our system and it lost connections to many of them at the same time.

  1. Unable to distribute to peer named indexer02 at uri https://IP:8089 because replication was unsuccessful. replicationStatus Failed
link

answered 09 Mar '12, 02:26

ttrolf's gravatar image

ttrolf
412
accept rate: 0%

Post your answer
toggle preview

Follow this question

Log In to enable email subscriptions

RSS:

Answers

Answers + Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×356
×152
×86

Asked: 27 Feb '12, 13:19

Seen: 880 times

Last updated: 10 Apr '12, 01:52

Copyright © 2005-2012 Splunk Inc. All rights reserved.