splunkweb takes ages to start. it even runs into an internal timeout and only with some tweaking (touch /appl/splunk/var/run/splunk/splunkweb.pid) it finally starts up after 2-10 minutes. We experience the same on 6 different Splunk instances. Find below an example which represents the second restart of a splunk instance. This one is faster as the first one but it still takes too long.
Without tweaking (service log): Starting splunk server daemon (splunkd)... Done. Starting splunkweb... Timed out waiting for splunkweb to start
With tweaking (web_service.log) : 2012-01-05 14:08:26,875 INFO [4f05a0c7da160f4d0] root:243 - Enabling SSL 2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Serving on 0.0.0.0:443 2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Bus STARTED
Any idea what migth cause this behaviour?
OS: Solaris 10 HW: x86 Splunk: 4.2.5
asked 05 Jan '12, 05:39
I spotted slow startup once after I was cleaning up my buckets (splunk clean all). After that Splunk was checking/re-config all buckets (and I have/had a load of buckets) which took some 40mins for startup! But this happened only once after the cleanup!
answered 05 Jan '12, 06:45
This could indicate issues with DNS, so I would suggest that you check if the DNS server(s) Splunk server is using (/etc/resolv.conf) work correctly.
answered 05 Jan '12, 07:36