I just stumbled upon this post while looking for something semi-unrelated. FWIW: There are some instances where it must be set to "true" in the .conf files. I had an issue back in Feb where queries were not displaying length of execution in Splunk 9.0.8.  Found a KB article in Splunk support that suggested it might be caused by a setting** in limits.conf that was set to "1" instead of "true". We changed it to "true" and that fixed it. We did a little digging with the rest API and found that it would return 1/0 for the configs, but when looking at the .confs, they were written as true/false. **I won't reference the setting so as to not upset the Splunk Gods who may hold support contracts sacred. ... View more

This is an area where I feel Splunk is severely lacking. It would be great if there was a mechanism for sharing one password with a group of users without having to give out list_storage_passwords, then retroactively update all of our apps to limit access. We have custom SPL commands that require passwords to be entered on a setup screen. We cannot give everyone list_storage_passwords, so the ability to use these commands is limited to our admins. Edit: Just saw the ideas links. Added my Vote. ... View more

I ended up removing the renderXml=true option. It does not look like evt_resolve_ad_obj works with XML or is broken. ... View more

Old post but following up in case anyone else has this same issue. Mine was caused by setting this to True in my get_scheme:   scheme.use_single_instance = False   I also had 2 stanza in my inputs.conf for the same script and the first one had an interval of "-1" I think the combination of the two ensured my stanzas would run once, then that was it until I restarted Splunk or enabled/disabled either of the stanzas.     ... View more

Old post but following up anyway. Leave the "interval" parameter off of the inputs.conf.spec file. You can get to it from the Data Inputs page by clicking "More settings" at the bottom of the input page. ... View more

Old post but this is how I fixed it in a dashboard. Convert to epoch, rename what you want the final field to be, then apply fieldformat to the result. | eval admin_appcreatedon=strptime(admin_appcreatedon,"%m/%d/%Y") | rename admin_appcreatedon as "Created On" | fieldformat "Created On" = strftime('Created On', "%m/%d/%Y")   ... View more

Sure enough, downgrading to version 2.2.4 fixed my issue.  There must be something fundamentally different with the latest 2.3.1 version that is not playing well with an externally-based application calling Splunk. npm install @splunk/splunk-utils@"2.2.4"  The 2.3.1 version appears to be reliant on the Config package, but running outside of a Splunk environment, I do not see how it is possible to gather those facts and the documentation on this endpoint is lacking. https://splunkui.splunk.com/Packages/splunk-utils/Config Does the 2.3.1 version of getData require a cookie instead of a bearer token in a header? What am I missing?  I might need to move this to a case with Splunk. ... View more

Adding additional information. Looking further at the search.js's getData implementation, I might have a versioning issue happening with my splunk-utils package. The notes for getData suggest it is doing a GET: https://splunkui.splunk.com/Packages/splunk-utils/Search  Makes a GET request to the search/${endpointVersion}/jobs/${search_id}/${endpoint} REST API endpoint to get data from the specified search job. My version of the package is doing a POST. These changes appear to have been pushed in the last few weeks according to the change log.   if (parameters.search && endpointVersion === '/v2') { return fetch(url, _objectSpread(_objectSpread({}, baseFetchInit), {}, { method: 'POST', body: qs })).then((0, _fetch.handleResponse)([200, 204]))["catch"]((0, _fetch.handleError)("Failed to get ".concat(endpoint || 'status', " for job ").concat(sid))); }     ... View more

I was seeing the same thing on the Event Types page. Fixed it by granting the admin role inheritance of the following: itoa_admin itoa_analyst itoa_user ... View more

Hello @justinhaynes , This sounds like it is an Alert Action?  Unfortunately I do not have experience with using that component of the AOB. If you are familiar with python and not set on an Alert Action, you could possibly accomplish all that you need to do adding the splunklib.client library within a Data Collection python input to read your alerting events. I use AOB python inputs somewhat frequently to perform various API calls and write to Splunk events as well as KV stores.  You could send the alerts to your ticketing system one event at a time, capturing the HTTP response. From here, you have several options for writing the error response and structuring it to also log the event details. 1. Writing it to a field in cim_modaction (although I won't be much help here. I rarely use the CIM) 2. Writing to the inputs log file using the AOB's included helper.log_error functions 3. Writing to an index using the ew.log function From here, you can write a saved search to look for any errors and the specific event the process failed on. I hope this helps. Please let me know if I missed the forest for the trees. ... View more

FWIW, I had this same problem and fixed it by changing my OS command from subprocess.run() to subprocess.Popen.  Not sure why, but it seem's Splunk's implementation of Python 3.7 has issues with subprocess.run() and streaming commands. This allowed me to use  chunked = true without any problems.  ... View more

Thank you for the information, Rich.  I will reach out to Support. ... View more

That is an interesting development. Do you happen to know the maintenance release or Issue Number where it was fixed? We are running 8.2.6 This is using the Splunk _json sourcetype which has INDEXED_EXTRACTIONS=true. We are going to make our own sourcetype with INDEXED_EXTRACTIONS=false as a workaround. Either way, it seems there should be a way to prevent the queue from blocking even when INDEXED_EXTRACTIONS=true and the indexer field limit is hit. ... View more

Hello Rich, Thank you for your reply. This is the Too Many Fields  message that begins our issue:  07-18-2022 07:19:48.370 -0700 ERROR TcpInputProc [2721 FwdDataReceiverThread] - Encountered Streaming S2S error=Too many fields (274382) for data received from src=myhf.myco.com:62421.  This is when the queue to the indexer shows as paused:  07-18-2022 07:48:56.793 -0700 WARN TcpOutputProc [376 indexerPipe] - The TCP output processor has paused the data flow. Forwarding to host_dest=myindexer.myco.com inside output group my_indexers from host_src=myhf has been blocked for blocked_seconds=1740. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.  Here are the results of running btool on this particular machine: C:\Program Files\Splunk\bin>splunk.exe btool outputs list --debug C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf [indexAndForward] C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf index = false C:\Program Files\Splunk\etc\system\default\outputs.conf [syslog] C:\Program Files\Splunk\etc\system\default\outputs.conf maxEventSize = 1024 C:\Program Files\Splunk\etc\system\default\outputs.conf priority = <13> C:\Program Files\Splunk\etc\system\default\outputs.conf type = udp C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf [tcpout] C:\Program Files\Splunk\etc\system\default\outputs.conf ackTimeoutOnShutdown = 30 C:\Program Files\Splunk\etc\system\default\outputs.conf autoLBFrequency = 30 C:\Program Files\Splunk\etc\system\default\outputs.conf autoLBVolume = 0 C:\Program Files\Splunk\etc\system\default\outputs.conf blockOnCloning = true C:\Program Files\Splunk\etc\system\default\outputs.conf blockWarnThreshold = 100 C:\Program Files\Splunk\etc\system\default\outputs.conf cipherSuite = <removed by poster> C:\Program Files\Splunk\etc\system\default\outputs.conf compressed = false C:\Program Files\Splunk\etc\system\default\outputs.conf connectionTTL = 0 C:\Program Files\Splunk\etc\system\default\outputs.conf connectionTimeout = 20 C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf defaultGroup = my_indexers C:\Program Files\Splunk\etc\system\default\outputs.conf disabled = false C:\Program Files\Splunk\etc\system\default\outputs.conf dropClonedEventsOnQueueFull = 5 C:\Program Files\Splunk\etc\system\default\outputs.conf dropEventsOnQueueFull = -1 C:\Program Files\Splunk\etc\system\default\outputs.conf ecdhCurves =<removed by poster> C:\Program Files\Splunk\etc\system\default\outputs.conf forceTimebasedAutoLB = false C:\Program Files\Splunk\etc\system\default\outputs.conf forwardedindex.0.whitelist = .* C:\Program Files\Splunk\etc\system\default\outputs.conf forwardedindex.1.blacklist = _.* C:\Program Files\Splunk\etc\system\default\outputs.conf forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup) C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf forwardedindex.filter.disable = true C:\Program Files\Splunk\etc\system\default\outputs.conf heartbeatFrequency = 30 C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf indexAndForward = false C:\Program Files\Splunk\etc\system\default\outputs.conf maxConnectionsPerIndexer = 2 C:\Program Files\Splunk\etc\system\default\outputs.conf maxFailuresPerInterval = 2 C:\Program Files\Splunk\etc\system\default\outputs.conf maxQueueSize = auto C:\Program Files\Splunk\etc\system\default\outputs.conf readTimeout = 300 C:\Program Files\Splunk\etc\system\default\outputs.conf secsInFailureInterval = 1 C:\Program Files\Splunk\etc\system\default\outputs.conf sendCookedData = true C:\Program Files\Splunk\etc\system\default\outputs.conf sslQuietShutdown = false C:\Program Files\Splunk\etc\system\default\outputs.conf sslVersions = tls1.2 C:\Program Files\Splunk\etc\system\default\outputs.conf tcpSendBufSz = 0 C:\Program Files\Splunk\etc\system\default\outputs.conf useACK = false C:\Program Files\Splunk\etc\system\default\outputs.conf useClientSSLCompression = true C:\Program Files\Splunk\etc\system\default\outputs.conf writeTimeout = 300 C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf [tcpout:my_indexers] C:\Program Files\Splunk\etc\apps\MY_outputs\default\outputs.conf server = myindexer.myco.com:9997 ... View more

Thank you.  I will accept the solution, but might open an SR with Splunk to inquire.  I will follow up when I have more information. ... View more