Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About vcarbona
vcarbona
Path Finder
Member since:
08-06-2010
07-29-2022
Community Statistics
| Posts | 30 |
| Solutions | 4 |
| Karma Given | 27 |
| Karma Received | 24 |
| Member Since | 08-06-2010 |
Activity Feed
- Karma Will Splunk's timeline viz support drilldown and other standard Splunk viz features at some point? for andrewtrobec. 12-01-2021 03:50 PM
- Posted Installed IT Essentials for Work: Not seeing all entities on All Apps and Add-ons. 09-23-2021 01:06 PM
- Tagged Installed IT Essentials for Work: Not seeing all entities on All Apps and Add-ons. 09-23-2021 01:06 PM
- Tagged Installed IT Essentials for Work: Not seeing all entities on All Apps and Add-ons. 09-23-2021 01:06 PM
- Karma AWS Failed logins and coalesce command for samadmemon. 06-05-2020 12:50 AM
- Karma What does this error mean regarding bundle conflicting timestamp? for luis290311. 06-05-2020 12:49 AM
- Karma DUO Log Add-on for Splunk: Why am I getting error "ModularInputs - Validation for scheme=duo failed: The script returned with exit status 1."? for michaelstanton. 06-05-2020 12:48 AM
- Karma REST endpoint for modifying $app/local/macros.conf for jpvlsmv. 06-05-2020 12:47 AM
- Karma Does anyone know what the metric "active_searches" in remote_searches.log represents? for chris. 06-05-2020 12:47 AM
- Got Karma for Re: REST endpoint for modifying $app/local/macros.conf. 06-05-2020 12:47 AM
- Got Karma for Re: REST endpoint for modifying $app/local/macros.conf. 06-05-2020 12:47 AM
- Got Karma for Re: REST endpoint for modifying $app/local/macros.conf. 06-05-2020 12:47 AM
- Got Karma for Re: REST endpoint for modifying $app/local/macros.conf. 06-05-2020 12:47 AM
- Got Karma for Re: Indexer recv-q high and certain sources lagging. How do I fix that?. 06-05-2020 12:47 AM
- Got Karma for Indexer recv-q high and certain sources lagging. How do I fix that?. 06-05-2020 12:47 AM
- Got Karma for Re: Still seeing connection_host typos on universal forwarder version 6.0.5 in my custom app. 06-05-2020 12:47 AM
- Got Karma for Re: Still seeing connection_host typos on universal forwarder version 6.0.5 in my custom app. 06-05-2020 12:47 AM
- Karma Re: What's the meaning of aggregration queue? for _d_. 06-05-2020 12:46 AM
- Karma What is the impact of increasing thruput in Universal Forwarders? for adityapavan18. 06-05-2020 12:46 AM
- Karma File growth rate must be higher than indexing or forwarding rate. for splunkranger. 06-05-2020 12:46 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 2 |
09-23-2021
01:06 PM
I have SAI 2.0.2 installed on my SHC and the SAI Addon on the indexer cluster. Went ahead and installed the full IT Essentials on the SHC and installed the SA_IndexCreation on the indexer cluster per the instructions. So far I only see 2 entities in IT Essentials. The ironic thing is that SAI app is showing much more. Anything I missed? Perhaps SAI and IT Essentials for Work apps are in conflict with one another?
... View more
Labels
- Labels:
-
troubleshooting
12-02-2019
01:21 PM
It was working fine initially then I was getting these errors in some of the panels after a few hours. It is on a 4-member search head cluster.
... View more
11-14-2019
08:32 AM
I'm thinking this field should not be overwritten rather a new field should be created indicating the status whether it is success or failure. Not sure if doing so will break anything else.
... View more
10-14-2019
09:23 AM
I'd like to leverage collectd to gather not just metrics for SAI but also statsd data from custom apps. I'm using the write_splunk plugin to write SAI metrics using a UF UDP port. The problem comes in where I would also like to accept UDP data from the statsd collectd plugin and also send that data into the UF via UDP but route to another index. I basically am looking for a good way to not populate both SAI data and custom app metrics into the same em_metrics index.
... View more
04-30-2019
08:59 AM
Wouldn't this only work if the user manually specified the index name in the search query? Users would have to specify it if the index were only allowed and not the default.
... View more
09-14-2018
03:19 PM
When I tried to install the Oracle JDK, I started to get this error:
08-27-2015 20:31:26.244 +0000 ERROR ModularInputs - Unable to initialize modular input "kafka" defined inside the app "kafka_ta": Introspecting scheme=kafka: script running failed (exited with code 1).
I was able to fix this by adding JAVA_HOME= in my /opt/splunk/etc/splunk-launch.conf and restarting Splunk. In my case, I used the JRE.
... View more
01-21-2016
12:11 PM
I don't believe this is a correct answer. In our testing, we shutoff an indexer which was reporting to our license master. It still showed a current updated time field for the shutoff indexer! Unless of course this is being read incorrectly or this rest call is not for this purpose.
From my observation, it seems the only thing to rely on is the warning_count when it comes to answering this question which may be of some value but doesn't completely answer the poster's question.
... View more
07-21-2015
11:46 AM
Yeah...it got rid of the error but the forwarder gives me these messages when I try to send something to it.
07-21-2015 13:43:42.154 -0500 INFO TcpInputConfig - No matching config for xx.xx.xx.xx
07-21-2015 13:43:42.154 -0500 WARN TcpInputProc - Could not find matching host.
... View more
07-20-2015
03:00 PM
1 Karma
Addendum: Apparently btool checks these spec files (one of them being inputs.conf.spec) to verify if the actual conf files are in good order.
... View more
07-20-2015
02:48 PM
1 Karma
I filed the bug along with a suggested fix:
Simply adding the following entry in the $SPLUNK_HOME/etc/system/README/inputs.conf.spec removed the error message:
[tcp:<port>]
connection_host = <ip | dns | none>
queueSize = <integer>[KB|MB|GB]
persistentQueueSize = <integer>[KB|MB|GB|TB]
requireHeader = <bool>
listenOnIPv6 = <no | yes | only>
acceptFrom = <network_acl> ...
rawTcpDoneTimeout = <seconds>
####The udp spec below is included in version 6.1.4 thru 6.2.4 so it shouldn't report a typo on udp entries.
####Add this entry also if you're running a forwarder 6.1.3 or lower.
[udp: ]
* This input stanza is same as [udp:// : ] but without any remote server restriction
* Please see the documentation for [udp:// : ] to follow supported settings:
connection_host = [ip|dns|none]
_rcvbuf =
no_priority_stripping = [true|false]
no_appending_timestamp = [true|false]
queueSize = [KB|MB|GB]
persistentQueueSize = [KB|MB|GB|TB]
listenOnIPv6 =
acceptFrom = ...
... View more
07-20-2015
12:40 PM
The host parameter is already specified in $SPLUNK_HOME/etc/system/local/inputs.conf and it is a global attribute that takes into effect across all apps. To give benefit of the doubt, I added this in but got the same typo error. The TCP actually works fine. I'm trying to figure out why I'm getting this typo.
... View more
07-16-2015
01:45 PM
I saw an answer that said this was corrected in 4.2 but I'm seeing this on 6.0.5 universal forwarder. Noticed the spec file has the correct config also so I'm not sure why btool complains about this. What gives?:
Possible typo in stanza [tcp:9999] in /opt/splunkforwarder/6.0.5-214064/etc/apps/mytcp/default/inputs.conf, line 5: connection_host = none
/opt/splunkforwarder/6.0.5-214064/etc/apps/mytcp/default/inputs.conf:
[tcp:9999]
sourcetype = foo
index = foo
queueSize = 1GB
connection_host = none
/opt/splunkforwarder/6.0.5-214064/etc/system/README/inputs.conf.spec:
[tcp://<remote server>:<port>]
* Configure Splunk to listen on a specific port.
* If a connection is made from <remote server>, this stanza is used to configure the input.
* If <remote server> is empty, this stanza matches all connections on the specified port.
* Will generate events with source set to tcp:portnumber, for example: tcp:514
* If sourcetype is unspecified, will generate events with set sourcetype to tcp-raw.
# Additional attributes:
connection_host = [ip|dns|none]
* "ip" sets the host to the IP address of the system sending the data.
* "dns" sets the host to the reverse DNS entry for IP address of the system sending the data.
* "none" leaves the host as specified in inputs.conf, typically the splunk system hostname.
* Defaults to "dns".
queueSize = <integer>[KB|MB|GB]
* Maximum size of the in-memory input queue.
* Defaults to 500KB.
... View more
07-03-2015
10:48 AM
it is possible to reschedule using one of the SDKs. I'm doing this currently for any skipped searches using the Python SDK. The SDK uses REST to accomplish this. It is also possible to do POST requests via REST when using the "curl" command without necessarily using the SDK.
... View more
05-04-2015
09:46 AM
1 Karma
We've done a few things since then to control this issue:
1) Set our ulimit to 10240 on all indexers (But this didn't in itself resolve the problem yet helped out with the load)
2) We've also added the following line to the offending forwarder's outputs.conf under the [tcpout] stanza and restarted the forwarder:
forceTimebasedAutoLB = true
This latter solution has helped out tremendously so far in forcing data to be load balanced. It appears the default behavior is to stream it to one indexer until the "batch" of data is done. The setting appears to force it to another indexer.
Thanks for asking by the way! Almost forgot I posted the question awhile back 🙂
... View more
03-30-2015
02:23 PM
4 Karma
The link below will have the answer in the question.
With the Python SDK, you can access it even easier:
from splunklib.client import connect
...
service = connect(username="admin", password="changeme", host="myhost", app="search")
# To update a macro named "test" in the search app
service.post('properties/macros/test', definition="test123")
# To read a macro named "test" in the search app
print service.get('properties/macros/test/definition')["body"]
... View more
12-16-2014
11:57 AM
1 Karma
Noticing from netstat there are high recv-q numbers on the indexer. We also notice some sources lagging in the indexer. The ports with the high recv-q are from the forwarders which contain these particular sources.
Our setting in limits.conf on the indexer is the following:
[thruput]
throughput limiting at index time
maxKBps = 0
We don't have excessive cpu, memory, or io on the indexer. Indexers will typically have about 35 connections from forwarders open. We also have a ulimit set to 4096.
Our current workaround is to restart the indexer but it reappears on other indexers of the same server group. Obviously this is starting to become an annoyance.
Anything else we should be checking?
... View more
07-08-2014
10:50 AM
Actually "ignored file (crc conflict, needs crcSalt)" does appear, it just has a different error. I run it as "index=_internal component=TailingProcessor ERROR salt". It shows up as an ERROR alert level. The "File did not match whitelist ..." appears when I set TailingProcessor to DEBUG level.
... View more
07-07-2014
11:48 AM
I noticed the btprobe command shows some interesting data about the file status. It appears it is able to retrieve the modtime and seek pointer. Is it correct to assume that sptr (or seek pointer) is where the forwarder left off reading the file?
splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/messages
Using logging configuration at /opt/splunkforwarder/6.0.3-204106/etc/log-cmdline.cfg.
key=0xf4e82f9f021c429d scrc=0xc6e25d94afc02135 sptr=871 fcrc=0x452905a167cf4509 flen=0 mdtm=1404740503 wrtm=1404740504
... View more
03-27-2014
11:45 AM
Thanks! This actually helped us to identify some problem areas. But it appears conditions like "ignored file (crc conflict, needs crcSalt)" or "File did not match whitelist ..." do not show up.
... View more
03-27-2014
09:43 AM
That would have been cool, but I get this:
ps -ef|grep splunk
splunk 5604 1 53 11:39 ? 00:00:13 splunkd -p 8089 restart
splunk 5605 5604 0 11:39 ? 00:00:00 [splunkd pid=5604] splunkd -p 8089 restart [process-runner]
./splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus
QUERYING: 'https://127.0.0.1:8089/services/admin/inputstatus/TailingProcessor:FileStatus'
This command [GET /services/admin/inputstatus/TailingProcessor:FileStatus] needs splunkd to be up, and splunkd is down.
... View more
03-27-2014
07:16 AM
We would like to have forwarders run as root in order to overcome file permissions. However, we also will be security hardening it as much as possible. One of these measures is to stop port 8089 on the forwarder. I assume this will not give us the ability to read the REST endpoint https://hostname:8089/services/admin/inputstatus/TailingProcessor%3AFileStatus. Are there any other ways to gather this data without the REST endpoint being available?
... View more
- Tags:
- cli
- tailingprocessor
09-26-2013
12:12 PM
I only monitor for ERRORLOG and still got the same issue. I tested the props.conf ONLY on the universal forwarder it works fine. So no need to update the indexer's props.conf.
... View more
05-22-2013
10:12 AM
1 Karma
After trial and error, it seems to work when I specify it without any spaces and then restart the indexer. Using $SPLUNK_HOME/etc/system/local/props.conf in 4.3.6 version:
[source::*\\mydata\\Log*]
TZ=UTC
... View more
12-20-2012
02:04 PM
3 Karma
I've been able to debug this using $SPLUNK_HOME\bin\splunk-perfmon.exe -showzero
You must first set your SPLUNK_HOME environment variable before executing the above command.
A GUI will popup and you simply select which counter you suspect is not working. Within 10 seconds you start seeing the counter values (CTRL-C to break out of it). If you omit the -showzero parameter and use that same counter nothing will print to the screen and you'll have your answer.
I noticed that the scripted input (in inputs.conf) shows it uses the $SPLUNK_HOME\bin\scripts\splunk_perfmon.path. I guess zero (0) values are suppressed because it saves disk. If you absolutely want those to NOT be suppressed, just update the splunk-perfmon.path file and add the -showzero parameter to it and restart. ie:
$SPLUNK_HOME\bin\splunk-perfmon.exe -noui -showzero
One more thing: When I say zero (0) values are suppressed, I mean absolute 0 NOT 0.125673 . The latter will be captured by splunk-perfmon.exe and forwarded out.
... View more
05-16-2012
03:03 PM
1 Karma
If you're just trying to substitute the SOH character I was FINALLY able to do it after spending a ton of time and it's a very simple solution. I may be reiterating what Lowell said but hopefully this example saves a ton of time for someone else. Additionally the solution handles it at index time and not at search time. So it makes it easier to read for users who don't realize there's a SOH delimiter to deal with:
edit $SPLUNK_HOME/etc/system/local/props.conf (on the indexer box if your search head and indexer are 2 different boxes) and add the following:
[myfixsourcetype]
SEDCMD-stripsoh = s/\x01/ /g
Then restart Splunk. Now any NEW FIX data will have the SOH character replaced with a space character. This will NOT affect existing, indexed FIX data in Splunk already.
Note: Of course, the "myfixsourcetype" needs to be replaced with the actual sourcetype name that your FIX data is coming in as otherwise it has no way of identifying your data in order to apply the sed command to. See props.conf spec for other data identifiers you can use (ie. host or source).
FYI - I'm running Splunk on a RedHat Linux box.
REFERENCES:
FIX protocol field delimiter
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
07-29-2022
11:52 AM
|
Karma given to
