Hi, recently we had an issue with the LUN drive where data is stored and after fixing it, a new problem came up. splunk services starts normally but the web access does not work anymore. the output of the splunk start command is the following     \bin>splunk.exe start Splunk> Map. Reduce. Recycle. Checking prerequisites... Checking mgmt port [8089]: open Checking configuration... Done. Checking critical directories... Done Checking indexes... (skipping validation of index paths because not running as LocalSystem) Validated: _configtracker _introspection _metrics _metrics_rollup _thefishbucket anomaly_detection autek azure cim_modactions cisco citrix email eusc_apps firedalerts ftp hyper-v infraops itsi_grouped_alerts itsi_im_meta itsi_im_metrics itsi_import_objects itsi_notable_archive itsi_notable_audit itsi_summary itsi_summary_metrics itsi_tracked_alerts kubernetes metrics_sc4s msad msexchange netauth netfw netops netproxy os osnix pan_logs perfmon rancher_k8sca rancher_k8scc rancher_k8scs rancherprod sample snmptrapd sns symantec sysmon test thor windefender windows wineventlog winevents Done Bypassing local license checks since this instance is configured with a remote license master. Checking filesystem compatibility... Done Checking conf files for problems... Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features. One or more regexes in your configuration are not valid. For details, please see btool.log or directly above. Done Checking default conf files for edits... Validating installed files against hashes from 'C:\Program Files\Splunk\splunk-9.0.8-4fb5067d40d2-windows-64-manifest' All installed files intact. Done All preliminary checks passed. Starting splunk server daemon (splunkd)... Splunkd: Starting (pid 38432) Done     extract of btool.log      05-06-2024 11:07:35.039 WARN ConfMetrics - single_action=BASE_INITIALIZE took wallclock_ms=1014 05-06-2024 11:17:25.445 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 11:17:25.445 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 13:00:58.310 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 13:00:58.310 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 13:00:58.373 WARN btool-support - Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features. 05-06-2024 13:19:36.176 WARN ConfMetrics - single_action=BASE_INITIALIZE took wallclock_ms=1234 05-06-2024 14:44:42.912 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 14:44:42.912 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 14:44:42.975 WARN btool-support - Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features. 05-06-2024 14:44:51.022 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 14:44:51.022 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 14:44:51.084 WARN btool-support - Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features. 05-06-2024 16:36:21.051 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 16:36:21.051 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 16:36:21.114 WARN btool-support - Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features. 05-06-2024 16:36:29.661 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v14 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 16:36:29.661 WARN IConfCache - Stanza has an expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-ClientAccess\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1], ignoring alternate expansion [script://C:\Program Files\Splunk\etc\apps\TA-Exchange-Mailbox\bin\exchangepowershell.cmd v15 read-audit-logs_2010_2013.ps1] in inputs.conf 05-06-2024 16:36:29.723 WARN btool-support - Bad regex value: '(::)?...', of param: props.conf / [(::)?...]; why: this regex is likely to apply to all data and may break summary indexing, among other Splunk features.       I already checked the /etc/system/local/web.conf and everything seems fine.     [settings] enableSplunkWebSSL = 1 httpport = 443     system/default/web.conf     [default] [settings] # enable/disable the appserver startwebserver = 1 # First party apps: splunk_dashboard_app_name = splunk-dashboard-studio # enable/disable splunk dashboard app feature enable_splunk_dashboard_app_feature = true # port number tag is missing or 0 the server will NOT start an http listener # this is the port used for both SSL and non-SSL (we only have 1 port now). httpport = 8000 # this determines whether to start SplunkWeb in http or https. enableSplunkWebSSL = false # location of splunkd; don't include http[s]:// in this anymore. mgmtHostPort = 127.0.0.1:8089 # list of ports to start python application servers on (although usually # one port is enough) # # In the past a special value of "0" could be passed here to disable # the modern UI appserver infrastructure, but that is no longer supported. appServerPorts = 8065     any suggestion? many thanks. jose ... View more

Hi, I run splunk 9.0.8 and after an issue with our storage (LUN full). I had to full scan the disk and successfully repaired the filesystem. splunk starts now but there is a persistent error with the KV store. the status is the following: show kvstore-status This member: backupRestoreStatus : Ready disabled : 0 guid : E59FAAA8-D66A-498E-8DDF-0F5C29866F95 port : 8191 standalone : 1 status : failed storageEngine : wiredTiger Any suggestion on how could get an operational status? many thanks. Jose ... View more

Hi, By chance, I discovered that a power user with admin rights disabled sysmon agent and splunk forwarder on his computer to gain some extra CPU for his daily tasks. To react quicker to this type of "issue" in the future, I would like to have some alerts in splunk informing me whenever any host, initially domain joined workstations and servers, might have stopped reporting events. Did someone implement something like this already? any good article to follow? I plan to create a lookup table using ldapsearch and then, an alert detecting which hosts from that table are not present in a basic listing host search on sysmon index for example. Any other better or simpler approach?   many thanks ... View more

Hi, I am using Splunk 9.0.6, and I configured HEC + Syslog Connector for Splunk for the data ingestion. At the moment, I receive events from our two different firewall (PaloAlto and Stormshield). My problem arises with the fact that Stormshield is not directly supported by SC4S, so the extracted fields are not CIM compliant. More precisely, the field action should contain blocked or allowed as possible values, but it contains pass and block instead. My question is how it would be the best way to implement this transformation. I tried creating the following files in the path  C:\Program Files\Splunk\etc\apps\splunk_httpinput\local props.conf [StormShield:StormShield] TRANSFORMS = rewriteaction transform.conf [rewriteaction] EVAL-action = case(action="pass", "allowed", action="block", "blocked" , 1=1, "UNKNOWN") I restarted Splunk, but nothing really happened. Any idea of what I am doing wrong?  Many thanks.   ... View more

Hi, I am deploying sysmon all acrros our company but for some reason the sysmon events are not getting indexed Our deployment is the following: Splunk 9.0.5 running on Windows server sysmon index created manually in Splunk. inbound firewall rules created allowing traffic TCP in port 9997 Sysmon TA installed in the server in C:\Program Files\Splunk\etc\deployment-apps\Splunk_TA_microsoft_sysmon default/input.cont enabled (by default)   [WinEventLog://Microsoft-Windows-Sysmon/Operational] disabled = false renderXml = 1 source = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational ​   local/input.conf containing    [WinEventLog://Microsoft-Windows-Sysmon/Operational] index=sysmon​   Splunk Universal forwarder 9.1.0 deployed in all hosts All UF are reporting correctly to Splunk confirmed that sysmon TA is present in all hosts , deployed via forwarder management using a server class /etc/system/default/ouputs.conf is pointing to the right splunk server   [tcpout] defaultGroup = default-autolb-group [tcpout:default-autolb-group] server = xxxxxx:9997 [tcpout-server://xxxxxx:9997]​   Sysmon 15 deployed in all hosts confirmed that the events are being created locally in the hosts in Microsoft-->Windows-->Sysmon tree But no single event appears in the sysmon index 😞 Does anyone have any idea or suggestion of what might be missing? many thanks ... View more