Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Splunk Answers
- :
- About kjonesdba_lm
kjonesdba_lm
Explorer
Member since:
08-09-2019
01-27-2021
Community Statistics
| Posts | 17 |
| Solutions | 0 |
| Karma Given | 1 |
| Karma Received | 1 |
| Member Since | 08-09-2019 |
Activity Feed
- Posted datadog and splunk question on Alerting. 01-26-2021 12:17 PM
- Karma Re: How to extract fields between backslashes and quotes with rex command? for grittonc. 06-05-2020 12:51 AM
- Got Karma for How to extract fields between backslashes and quotes with rex command?. 06-05-2020 12:51 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-04-2020 11:02 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-04-2020 06:09 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 01:42 PM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 12:32 PM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 12:30 PM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 10:23 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 10:22 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 10:21 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 09:22 AM
- Posted Re: How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-03-2020 05:41 AM
- Posted How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-02-2020 02:14 PM
- Tagged How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-02-2020 02:14 PM
- Tagged How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-02-2020 02:14 PM
- Tagged How to extract fields between backslashes and quotes with rex command? on Splunk Search. 06-02-2020 02:14 PM
- Posted Re: cant figure this out on Splunk Search. 08-12-2019 09:05 AM
- Posted Re: cant figure this out on Splunk Search. 08-12-2019 06:17 AM
- Posted Re: cant figure this out on Splunk Search. 08-12-2019 06:16 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 1 | |||
| 0 |
01-26-2021
12:17 PM
Good Afternoon We are going to use datadog to monitor Atlas databases. Can splunk ingest datadog data and allow us to continue to use splunk for alerting as that is our enterprise solution. Apologies if this is not question is not framed correctly Thanks
... View more
Labels
- Labels:
-
alert action
06-04-2020
11:02 AM
Thank you so much grittonc and to4kawa.. The answer is perfect. and the explanations.! This will help me down the road.
... View more
06-04-2020
06:09 AM
Awesome still testing this out and working so far!... I made these changes .. made server_name upper case and stripped out the .lm.lmig.com.. I will get back this afternoon.
| rex field=_raw "from '((?<AOAG_Name>[^\\\]+)\\\)?(?<SERVER_NAME>[^\\\]+)\\\(?<Instance_Name>[^\\\]+)'. "
| rex field=SERVER_NAME "(?<SERVER_NAME>\w+-\w+)"
... View more
06-03-2020
01:42 PM
I think the issue is that AOAG_Name sometimes does not exist in _raw - or - Server_Name is not always the first value in the string.. When AOAG_Name exists then Server_Name is the second value in the string.. When AOAG_Name doesn't exist then Server_Name is the first value in the string
... View more
06-03-2020
12:32 PM
for this _raw data
2020-06-02T22:15:50+00:00 10.177.121.152 1 2020-06-02T22:15:50.797Z RVMHM188S014111 Rubrik-JobFetcherLoop - EVENT [mdc@18060 instanceId="-1" jobId="" jobType="" ndc="MSSQL_DB_BATCH_BACKUP_84c7801c-91b9-4a21-ae97-16a09b68b47f_01d2ffcd-c800-41df-8d61-854d0c5d3171:::2" pid="21544" profile="false" taskId=""] [eventdetail@49929 eventType=Backup eventName=Snapshot.BackupFromLocationFailed objectType=Mssql objectName=MSP_T1_11_EXP eventId=1591136150660-60fb3bcb-b292-4914-a105-dd294abf5a47 eventSeriesId=b2f70082-c262-417c-bacf-ad3e04cb5ae5 objectId=28bad07f-97f7-4e5b-b2c5-110d25ad3380 status=Failure eventSeverity=Critical locationName=- clusterName=rubrik_pdc_dev nodeId=RVMHM188S014111 nodeIpAddress=10.176.50.81] Failed backup of Microsoft SQL Server Database 'MSP_T1_11_EXP' from 'VMPIT-H4RBRK13.lm.lmig.com\MSSQLSERVER'. Reason: Internal server error 'Protecting the 'MSP_T1_11_EXP' database requires 'NT AUTHORITY\SYSTEM' to be a sysadmin or have the db_backupoperator role for that database.'
... View more
06-03-2020
12:30 PM
It is not quite working it gives this for SERVER_NAME (lm_ci)
lm_ci
MSSQLSERVER'. Reason: Internal server error 'Protecting the 'MSP_T1_11_EXP' database requires 'NT AUTHORITY
... View more
06-03-2020
10:23 AM
index="storage" sourcetype="rubrik:dev" "status=Failure" ndc="MSSQL_DB_*BACKUP_*" NOT "is not online." AND "eventSeverity=Critical"
| rex field=_raw "from [\'](?<SERVER_NAME>[^\']\w+-\w+)"
| search NOT [ | inputlookup Servers_Pending_Deletion.csv | fields SERVER_NAME ]
| dedup SERVER_NAME sortby SERVER_NAME
| table SERVER_NAME _time _raw
| eval lm_action="ticket"
| eval lm_assigned_group="HS-AE-DATABASE-ALERTS"
| eval lm_summary=(SERVER_NAME." Rubrik status=MSSQL PR DB Failure")
| eval lm_ci=SERVER_NAME
| eval lm_severity="CRITICAL"
| eval lm_status="OPEN"
| eval lm_market="Hosting"
| eval lm_notes=(SERVER_NAME." "._raw)
| eval lm_env="PR"
| table lm*
... View more
06-03-2020
10:22 AM
the bold should look like this | rex field=_raw "from \'"
... View more
06-03-2020
10:21 AM
I will add the 'correct' old(current) splunk query here...
index="storage" sourcetype="rubrik:prod" "status=Failure" ndc="MSSQL_DB_BACKUP_" NOT "is not online." AND "eventSeverity=Critical"
| rex field=_raw "from \'"
| search NOT [ | inputlookup Servers_Pending_Deletion.csv | fields SERVER_NAME ]
| dedup SERVER_NAME sortby SERVER_NAME
| table SERVER_NAME _time _raw
| eval lm_action="ticket"
| eval lm_assigned_group="HS-AE-DATABASE-ALERTS"
| eval lm_summary=(SERVER_NAME." Rubrik status=MSSQL PR DB Failure")
| eval lm_ci=SERVER_NAME
| eval lm_severity="CRITICAL"
| eval lm_status="OPEN"
| eval lm_market="Hosting"
| eval lm_notes=(SERVER_NAME." "._raw)
| eval lm_env="PR"
| table lm*
... View more
06-03-2020
09:22 AM
I am really sorry but I dont see the "code sample" button. I have only used answers.splunk.com once before
... View more
06-03-2020
05:41 AM
This is actually the _raw data in splunk and original splunk query.. hopefully this will help with the changes to the query
1st and 2nd rows format
2020-06-02T22:15:58+00:00 10.177.121.152 1 2020-06-02T22:15:58.707Z RVMHM188S014111 Rubrik-JobFetcherLoop - EVENT [mdc@18060 instanceId="-1" jobId="" jobType="" ndc="MSSQL_DB_BATCH_BACKUP_84c7801c-91b9-4a21-ae97-16a09b68b47f_01d2ffcd-c800-41df-8d61-854d0c5d3171:::2" pid="21544" profile="false" taskId=""] [eventdetail@49929 eventType=Backup eventName=Snapshot.BackupFromLocationFailed objectType=Mssql objectName=TEST1 eventId=1591136158613-30269267-ed66-4fd2-be78-f43db513975f eventSeriesId=fa25e08b-7149-4846-b852-1d94b0beebdd objectId=6d16756a-196f-45d7-a34e-bb80ede64268 status=Failure eventSeverity=Critical locationName=- clusterName=rubrik_pdc_dev nodeId=RVMHM188S014111 nodeIpAddress=10.176.50.81] Failed backup of Microsoft SQL Server Database 'TEST1' from 'VMPIT-H4RBRK13.lm.lmig.com\MSSQLSERVER'. Reason: Internal server error 'Protecting the 'TEST1' database requires 'NT AUTHORITY\SYSTEM' to be a sysadmin or have the db_backupoperator role for that database.'
2020-06-02T22:00:09+00:00 10.177.121.152 1 2020-06-02T22:00:09.775Z RVMHM188S014111 Rubrik-JobFetcherLoop - EVENT [mdc@18060 instanceId="-1" jobId="" jobType="" ndc="MSSQL_DB_BATCH_BACKUP_2140df0f-dd16-4d37-8828-ed017862369b_3ea7bcf7-9528-48ad-bd89-a593c109ce74:::0" pid="21544" profile="false" taskId=""] [eventdetail@49929 eventType=Backup eventName=Snapshot.BackupFromLocationStarted objectType=Mssql objectName=master eventId=1591135209716-d5f5301a-99d7-4cdc-bc51-688b3c8c9895 eventSeriesId=21978596-fd3b-46d9-8820-a8b417076c9f objectId=70fb4b7b-37c4-486a-84a2-cf86d5a64b4a status=Running eventSeverity=Informational locationName=- clusterName=rubrik_pdc_dev nodeId=RVMHM188S014111 nodeIpAddress=10.176.50.81] Creating backup of Microsoft SQL Server Database 'master' from 'vmpit-h4rbrk04\MSSQLSERVER'
3rd and 4th rows
2020-06-01T22:09:32+00:00 10.177.121.152 1 2020-06-01T22:09:32.620Z RVMHM188S014111 Rubrik-JobFetcherLoop - EVENT [mdc@18060 instanceId="-1" jobId="" jobType="" ndc="MSSQL_DB_BATCH_BACKUP_2140df0f-dd16-4d37-8828-ed017862369b_f9977671-210b-4483-9422-2d8ccb34723f:::0" pid="21544" profile="false" taskId=""] [eventdetail@49929 eventType=Backup eventName=Snapshot.BackupFromLocationSucceeded objectType=Mssql objectName=random eventId=1591049372335-0534736f-1a57-40c6-8030-28ef1cf3b2aa eventSeriesId=b62d38d2-cc31-4eb4-baac-b68a5f04187c objectId=f68cf817-5e94-498d-acd0-1715839e7d52 status=Success eventSeverity=Informational locationName=- clusterName=rubrik_pdc_dev nodeId=RVMHM188S014111 nodeIpAddress=10.176.50.81] Completed backup of Microsoft SQL Server Database 'random' from 'RBRK_AG1\vmpit-h4rbrk04\MSSQLSERVER'
original splunk query
index="storage" sourcetype="rubrik:prod" "status=Failure" ndc="MSSQL_DB_BACKUP_" NOT "is not online." AND "eventSeverity=Critical"
| rex field=_raw "from \'"
| search NOT [ | inputlookup Servers_Pending_Deletion.csv | fields SERVER_NAME ]
| dedup SERVER_NAME sortby SERVER_NAME
| table SERVER_NAME _time _raw
| eval lm_action="ticket"
| eval lm_assigned_group="HS-AE-DATABASE-ALERTS"
| eval lm_summary=(SERVER_NAME." Rubrik status=MSSQL PR DB Failure")
| eval lm_ci=SERVER_NAME
| eval lm_severity="CRITICAL"
| eval lm_status="OPEN"
| eval lm_market="Hosting"
| eval lm_notes=(SERVER_NAME." "._raw)
| eval lm_env="PR"
| table lm*
... View more
06-02-2020
02:14 PM
1 Karma
These rows have a field that begins and ends with a quote, but have different meanings between the backslashes.
1st and 2nd rows are: 'Server_Name\Instance_Name' from 'vmpit-ugzcg8xk\MSSQLSERVER' from 'vmpit-ugzcg8xk.lm.lmig.com\MSSQLSERVER'
3rd and 4th rows are: 'AOAG_Name\Server_Name\Instance_Name' from 'rbrk_ag1\vmpit-ugzcg8xk\MSSQLSERVER' from 'rbrk_ag1\vmpit-ugzcg8xk.lm.lmig.com\MSSQLSERVER'
I need to be able to have a rex command that finds Server_Name , Instance_Name , and AOAG_Name from these 4 rows ( AOAG_Name would not have a value in the rows where it is not applicable).
My 'old' rex command before the data changed was:
| rex field=_raw "from [\'](?[^\']\w+-\w+)"
This is probably pretty easy for someone who is good with rex , but I am not and have not yet figured out how to do it.
Would anyone be able to help with this?
... View more
08-12-2019
09:05 AM
How would I do this 'without' passing in server_name and database_name ... to do this for all nodes and servers we have.
... View more
08-12-2019
06:17 AM
That was very helpful. Thanks for taking some time with this.. it is doing what I want and I understand what you did.
... View more
08-11-2019
07:56 AM
how do you alert on that "if the condition didn't return any events"? that's the issue.. we use splunk to create problem tickets.. one wont be created for no results which is what occurs if backups fail on a database for 30days straight.
I am stumped
... View more
08-09-2019
01:33 PM
I have this query below .. I need to report on the last successful backup 'over' 24 hours.. which this does... however what I cant figure out how to do is report when all backups have failed for 30 days. (no successful backups ever). _time is the time the backup ran and the log information written to splunk.
For instance if I remove the 'where' statement on the query it brings back 2 rows (1 success, 1 failed) on those backups that had both. it will bring 1 row back where there has been no successful backup (all failed)
I cannot figure out how to code the where to handle both conditions
index="storage" sourcetype="rubrik:prod" ndc="MSSQL_DB*" status=Failure OR status=Success
| rex field=_raw "from [\'](?[^\']\w+-\w+)"
| rex field=_raw "backup for (?\w+-\w+)\."
| rex field=_raw "eventSeriesId=(?.*?)\ objectId="
| rex field=_raw "objectName=(?.*?)\ eventId"
| rex field=_raw "Microsoft SQL Server Database \'(?.+)\' from"
| search SERVER_NAME="VMPIT-G4FDB003"
| search DATABASE_NAME="FBI"
| search NOT [ | inputlookup Servers_Pending_Deletion.csv | fields SERVER_NAME ]
| table SERVER_NAME _time _raw status DATABASE_NAME| stats max(_time) as TopTime by SERVER_NAME, DATABASE_NAME,status | sort by SERVER_NAME,DATABASE_NAME, _time desc
| where ((TopTime <= relative_time(now(),"-24h") and TopTime > relative_time(now(),"-30d") and status="Success" ))
| eval lm_24_ago=strftime(relative_time(now()-14400,"-24h"),"%m-%d-%y %H:%M:%S")
| eval lm_report_date=strftime(now()-14400,"%m-%d-%y %H:%M:%S")
| eval lm_7d_ago=strftime(relative_time(now()-14400,"-7d"),"%m-%d-%y %H:%M:%S")
| eval lm_last_backup=strftime(TopTime-14400, "%m-%d-%y %H:%M:%S") | sort by lm_last_backup desc
| eval lm_ci=SERVER_NAME
| eval lm_database=DATABASE_NAME
| eval lm_status=status
| eval lm_rows=rct
| table lm*
... View more
- Tags:
- searching
