About msrama5

msrama5 · ‎04-03-2023

Made some changes environment=test earliest=@d+5h latest=@d+9h (index=iis_openapi OR index=iis_securehostopenapi OR index=iis_securepayment) cs_method=POST | regex cs_uri_stem=(?i)"/account/v1/login/forgot-password"| bin _time span=1m | stats count as RPM by _time | eval TPS=RPM/60 | stats max(TPS) as MaxTPS it is still taking 56 secs for this query to run , here is the results from inspect job Duration(sec) Component Invocations Input count Output count 2.45 command.prestats 3,258 15,495 8,954 34.23 command.regex 3,258 32,047,620 15,495 3.73 command.remotetl 3,258 15,495 15,495 2,383.81 command.search 6,516 32,047,620 64,095,240 57.63 command.search.calcfields 9,202 32,196,524 32,196,524 2.74 command.search.expand_search 56 - - 77.43 command.search.fieldalias 9,202 32,196,524 32,196,524 26.30 command.search.filter 9,202 32,196,524 32,047,620 22.42 command.search.index 9,539 - - 312.24 command.search.kv 9,202 - - 246.08 command.search.lookups 9,202 32,196,524 32,196,524 155.64 command.search.rawdata 9,202 - - 1,358.04 command.search.typer 9,202 32,047,620 32,047,620 61.37 command.search.tags 9,202 32,047,620 32,047,620 7.24 command.search.track_sourcetypes 3,258 - - 2,424.81 dispatch.stream.remote 3,257 - 69,230,943 This is phase0 search query from inspect job litsearch (cs_method=POST (__f!=v OR environment=prod) (index=iis_openapi OR index=iis_securehostopenapi OR index=iis_securepayment) (index!=dp_sec_log OR index!=log-vdi-prod OR index!=idm-win-other OR index!=www_app OR index!=www_app) _time>=1680523200.000 _time<1680537600.000) | litsearch (cs_method=POST environment=prod (index=iis_openapi OR index=iis_securehostopenapi OR index=iis_securepayment) (index!=dp_sec_log OR index!=log-vdi-prod OR index!=idm-win-other OR index!=www_app OR index!=www_app) _time>=1680523200.000 _time<1680537600.000) | regex cs_uri_stem=(?i)"/account/v1/login/forgot-password" | bin _time span=1m | addinfo type=count label=prereport_events track_fieldmeta_events=true | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "_time" "host" "index" "linecount" "prestats_reserved_*" "psrsvd_*" "source" "sourcetype" "splunk_server" | remotetl nb=300 et=1680523200.000000 lt=1680537600.000000 max_count=1000 max_prefetch=100 | prestats count by _time

msrama5 · ‎04-03-2023

@bowesmana @ITWhisperer @inventsekar This is where it it taking more time from inspect job Duration (seconds) Component Invocations Input count Output count 2,133.38 command.search 6,598 32,047,620 64,095,240 52.30 command.search.calcfields 9,307 32,196,524 32,196,524 0.00 command.search.evalfilter 9,307 32,196,524 32,196,524 69.98 command.search.fieldalias 9,307 32,196,524 32,196,524 23.20 command.search.filter 9,307 32,196,524 32,047,620 213.23 command.search.lookups 9,307 32,196,524 32,196,524 1,219.75 command.search.typer 9,307 32,047,620 32,047,620 56.40 command.search.tags 9,307 32,047,620 32,047,620 6.95 command.search.track_sourcetypes 3,299 - - 0.01 dispatch.preview.write_results_to_disk 30 - - 7.64 dispatch.process_remote_timeline 1,860 19,383,546 - 3.95 dispatch.remote_timeline_fullevents 1,034 11,207,755 5,400 0.00 dispatch.stream.local 1 - - 2,171.22 dispatch.stream.remote 3,298 - 70,330,684 31.46 command.regex 3,299 32,047,620 15495 Following is phase0 search value from job inspector litsearch (cs_method=POST (__f!=v OR environment=prod) (index=iis_openapi OR index=iis_securehostopenapi OR index=iis_securepayment) (index!=dp_sec_log OR index!=log-vdi-prod OR index!=idm-win-other OR index!=www_app OR index!=www_app) _time>=1680523200.000 _time<1680537600.000) | litsearch (cs_method=POST environment=prod (index=iis_openapi OR index=iis_securehostopenapi OR index=iis_securepayment) (index!=dp_sec_log OR index!=log-vdi-prod OR index!=idm-win-other OR index!=www_app OR index!=www_app) _time>=1680523200.000 _time<1680537600.000) | regex cs_uri_stem=(?i)"/account/v1/login/forgot-password" | eval Hour=strftime('_time',"%H") | bin _time span=1m | addinfo type=count label=prereport_events track_fieldmeta_events=true | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "_time" "host" "index" "linecount" "prestats_reserved_*" "psrsvd_*" "source" "sourcetype" "splunk_server" | remotetl nb=300 et=1680523200.000000 lt=1680537600.000000 max_count=1000 max_prefetch=100 | prestats count by _time

msrama5 · ‎03-30-2023

Hello, following query is slow and processing a lot of data environment=tesxt earliest=-0d@d (index=iis_openapi OR index=iis OR index=iis1 ) cs_method=POST | regex cs_uri_stem=(?i)"/account/v1/login/forgot-password" |eval Hour=strftime(_time,"%H")|search Hour>=5 AND Hour<9| bin _time span=60s | stats count as RPM by _time | eval TPS=RPM/60 | stats max(TPS) as MaxTPS Are there way to optimize this query ?

msrama5 · ‎03-24-2023

Hello, I am trying to measure the downtime or slo breaches of certain customer endpoints over period of time, for example the metrics are success rate, latency are something we measure for the endpoints, currently we capture and query splunk every 5 mins and get these values, the values when below < 97% for success rate are breaches , one issue we have with in that 5 mins the slo breaches could have lasted few secs and mins and not the entire 5 minutes, if we capture the data from splunk for every min data for success rate that will be too many queries hits to the splunk and storing 1440 values/day instead of 288 values/day when queried every 1 min + storage cost for storing data and parsing to compute slo breaches 1440 mins/5 mins = 288 values 1440 mins/ 1 mins = 1440 values Any ideas how we can query splunk and get the threshold breaches accurately to secs so we can report downtime for prod incidents accurately to what is the amount of time the customer impact lasted with less hits to splunk and also more real time data provided to business on impact ?

msrama5 · ‎02-25-2021

Hello, I have 3 queries as below and all 3 return starid, I need to check if starid from query 1 exists on starid from (Query2 + Query3) and return those star ids that does not exist for Query 2 + Query 3, I wrote the join query but returning incorrect results , How can this query be written to return correct results for starid exists on Query1 only that does exist in Query2+ Query3 Query 1 - [search earliest=-3d latest=-2d index=_* OR index=* sourcetype=OPENAPI_STARS_LOGS "Processing AwardStarsRequest complete" *prmen* | rename Star_TranID as starid] Query 2 - [ search earliest =-4d environment=test8 sourcetype=FEI_Utility Level=Information SessionM | search messagetype=transaction | eval MessageTemplate=replace(MessageTemplate,"\\\\\"","\"") | spath MessageTemplate | mvexpand MessageTemplate | rex field=MessageTemplate "message\":\"(?<msg>.*)" | search msg="*" | eval _raw=replace(msg,"\\\\\"","\"") | spath | table transaction.sourceTransactionId | rename transaction.sourceTransactionId as starid] Query 3- [ search earliest =-4d environment=test8 sourcetype=FEI_Utility Level=Information *prmen* | search messagetype=accrual | spath MessageTemplate | mvexpand MessageTemplate | rex field=field3 "message\":\"(?<msg>.*)" | search msg="*" | eval _raw=replace(msg,"\\\\\"","\"") | spath | table "accrual.sourceTransactionId" | rename "accrual.sourceTransactionId" as starid ]

msrama5 · ‎10-29-2020

Hello, I wrote the following query which calculates success percentage based on expected 1 value every 5 mins which is 288 per day, but when it is less than a day or roll over multiple days , I need to compute the success percentage dynamically computing the expected values based on the duration divided by 5 mins and compute success percentages, how can this be done , below is the query for 24 hrs values. Need to change dynamically based on duration so number divided will change every time earliest = -24h index=error_log | eventstats count as Success_Count by "Properties.QueryName" | eval Success_Percentage=round(Success_Count/288*100,2) | table Success_Percentage

msrama5 · ‎08-21-2020

Hello, I have this query that needs dynamically adjusted for time duration, the results are written every 5 mins so , in 24 hrs period there will be 288 results written, I am dividing by 288 in query below to calculate percentage rate. earliest = -24h index=error_log | eventstats count as fcount by "Properties.QueryName" | eval percent = round((fcount/288)*100,2) | stats values(percent) as Failure_Percentage by "Properties.QueryName" If I change this query to pass start time and end time , the query needs to calculate duration and divide by 5 mins to get number of data counts and calculate the percentage rate, how can this query be modified to calculate time duration and find the right count to divide assuming data is expected every 5 mins.

msrama5 · ‎08-19-2020

Hello, from splunk data analysis I see some duplicate events being logged few days back and can't search the same data now, I need to write splunk query that will look for same events logged multiple times and report the stats, what is the query that can be used to do this for duplicate events analysis and summary of such events and counts

msrama5 · ‎06-15-2020

@to4kawa any ideas on this ?

msrama5 · ‎06-14-2020

Hi All, I have query below that needs to modified for sub string matching condition - splunk query: sourcetype=source1 id1="*" OR sourcetype=source2 id2="*" OR sourcetype=source3 id2="*" Id=coalesce(id1,id2,id3) | stats count by Id sourcetype | xyseries Id sourcetype count | fillnull source1 source2 source3 value="Not exists" | table source1 source2 source3 when id1=F80C05F3-19AF-40D3-AC73-19544E928D21 id2=XOP-F80C05F3-19AF-40D3-AC73-19544E928D21 id3=ABC-F80C05F3-19AF-40D3-AC73-19544E928D21 The query above needs to be modified for substring matching based on id1 existing in id2 or id3 and it needs to return the results, how can this query below be modified?

msrama5 · ‎06-09-2020

Hi All, I have query below which joins 3 sources 1,2,3 on id field, this works when id values matches across 3 sources, I have 1 additional condition for the ids could also match with substring matching for example- id1=F80C05F3-19AF-40D3-AC73-19544E928D21 id2=XOP-F80C05F3-19AF-40D3-AC73-19544E928D21 id3=ABC-F80C05F3-19AF-40D3-AC73-19544E928D21 the query below needs modified for substring matching based on id1 existing in id2 or id3 and it needs to return the results , how can this query below be modified ? splunk query- sourcetype=source1 id1="*" OR sourcetype=source2 id2="*" OR sourcetype=source3 id2="*" Id=coalesce(id1,id2,id3) | stats count by Id sourcetype | xyseries Id sourcetype count | fillnull source1 source2 source3 value="Not exists" | table source1 source2 source3

msrama5 · ‎06-01-2020

Hi All, I have the following query with 5 source types and 2 evals in one query, common field between source types is correlationid and elapsed time which may or may not exist and using coalesce since name formats can be different, I want to return unique correlation id in different sources and elapsedtime and return null if it does not exist, when I run the query below it is not returning any results,, what is wrong with the query below, is using 2 evals an issue ? (sourcetype=source1) OR (sourcetype=source2) OR (sourcetype=source3) OR (sourcetype=source4) OR (sourcetype=source5) | eval CorrelationId=coalesce('Properties.CorrelationId',CorrelationId,x-correlation-id,x_correlation_id ) | eval ElapsedTime = coalesce('Properties.elapsedMs','Properties.ElapsedMs','Properties.ElapsedTime',elapsedMs,elapsed) | stats values(ElapsedTime) as ElapsedTime by CorrelationId sourcetype | xyseries CorrelationId sourcetype ElapsedTime | fillnull source1 source2 source3 source4 source5 value="Not exists" | table CorrelationId source1 source2 source3 source4 source5 | sort CorrelationId

msrama5 · ‎06-01-2020

Hi All, does splunk log the elapsed time automatically ? I am trying to join few different source types in splunk that are joined by unique correlationid id logged and flowing between them , I don't see elapsed times on all calls. I want to compute the latency between different calls for unique ids flowing between them, splunk time stamp is not accurate, what is expected here ? do I need to ask my feature teams to log elapsed time in splunk ? is this not automatically logged ? I don't see this logged for many source types in splunk as is.

msrama5 · ‎05-18-2020

Hello, I have created the following search to show fieldsummary on 4 fields: devicename , ip , platform , and market as below. I want to filter the fieldsummary results further by market which shows devicename , ip , platform with individual market combinations, and show null counts and percentages computed in search. Can the search be modified to further group the devicename , ip , platform by market for null values? environment=test sourcetype=API1 | fields + devicename,Ip,platform,market | fieldsummary maxvals=10 | where (values like "%null%" OR isnull(values) OR values like "%NULL%") | fields - is_exact, max , mean, min, numeric_count, stdev, distinct_count | rex field=values "null\",\"count\":(?\d+)"| eval Percentage_null=if(isnull((null_count/count)*100),0,round((null_count/count*100),2)) | fieldformat count=tostring(count,"commas") | fieldformat null_count=tostring(null_count,"commas")| sort Percentage_null,values desc Current results: field Percentage_null count null_count values devicename 1.60 4,388 130 [{"value":"null","count":701},cf28ng==","count":24}] IP 1.33 4,388 212 [{"value":"null","count":512},{"value":"44.55.5.55":206}] platform 0.45 4,388 852 [{"value":"null","count":273},{"value":"Android":4000}] market 0.14 4,388 100 [{"value":"null","count":2000},{"value":"CA","count":2000},{"value":"JP","count":6}

msrama5 · ‎05-11-2020

correct, want to exclude source1 count values <=0

msrama5 · ‎05-11-2020

for example in below case source1 =0 values also shown Actual Id source1 source2 source3 source4 X 1 1 1 1 Y 0 1 1 1 Z 1 0 0 1 Expected- exclude <=0 values for source1 count, so output will be Id source1 source2 source3 source4 X 1 1 1 1 Z 1 0 0 1

msrama5 · ‎05-11-2020

hi @606866581 thanks, it works, I need to add one more filter condition for values count in source1 sourcetype > 0 and only show those values , where to apply filter for source1 values count > 0 ?

msrama5 · ‎05-11-2020

thanks, it works, I need to add one more filter condition for values count in source1 sourcetype > 0 and only show those values , where to apply filter for source1 values count > 0 ?

msrama5 · ‎05-11-2020

Hello, I have 4 sources (source 1-4) , common field for source 1 to 3 is Properties.Id, source4 common field is Id, does not have properties., I need to join these 4 sources on field Properties.Id, since source 4 just has Id, I renamed this field to Properties.Id using eval, with eval change, the query below is not working, it is not doing the join correctly using field Properties.Id, any ideas what is the issue here ? Splunk query- sourcetype=source1 OR sourcetype=source2 OR sourcetype=source3 OR sourcetype=source4 | eval Properties.Id=if(sourcetype="source4",Id,null()) | stats values(Properties.Id) as Id by sourcetype |append [|makeresults |eval sourcetype=split("source1 ,source2 ,source3 ,source4" ,",") | mvexpand sourcetype | fields sourcetype] | fillnull value="Not exists" Id| chart count over Id by sourcetype |sort (Id )

msrama5 · ‎04-23-2020

Getting close answer like one below after couple changes, need 2 more changes 1)Want to break down results correlation id by correlation id and display all 4 sourcetypes for each corr-id 2)display null values for source types where corr-id don't exists Actual - 1129213a-6d34-46e3-a7d7-2e2fcc552045 API_HUB 1 1129213a-6d34-46e3-a7d7-2e2fcc552045 API_RISK 1 116f755f-2c67-4604-b98a-90ed0bb43f74 API_RISK 1 11f7a987-e7a8-4ba3-9445-8ad92f4ecc05 API_PAY 1 11f7a987-e7a8-4ba3-9445-8ad92f4ecc05 API_RISK 1 Expected- 1129213a-6d34-46e3-a7d7-2e2fcc552045 API_HUB 1 1129213a-6d34-46e3-a7d7-2e2fcc552045 API_RISK 1 NULL API_PAY NULL API_PAY1 116f755f-2c67-4604-b98a-90ed0bb43f74 API_RISK 1 NULL API_RISK 1 NULL API_PAY NULL API_PAY1 Modified new query is below, can this be modified to get expected results above ? sourcetype=API_HUB OR sourcetype=API_RISK OR sourcetype=API_PAY OR sourcetype=API_PAY2 | stats values(CorrelationId) as CorrelationId by sourcetype |append [|makeresults |eval sourcetype=split("API_HUB,API_RISK,API_PAY ,API_PAY2" ,",") | mvexpand sourcetype | fields sourcetype] | stats count by CorrelationId,sourcetype | fillnull value="Not exists" CorrelationId | sort (CorrelationId)

msrama5 · ‎04-22-2020

I wrote the query using map for find correlation id in first query and substitute one by one in second query to return results for each id as below, this query is not working, wanting to get corrid and substitute in second query as is, any ideas what is wrong with query below ? no results are returned for query below sourcetype=OPENAPI_HUB | stats count by CorrelationId | map search="search "$CorrelationId$" sourcetype=OPENAPI_HUB OR sourcetype=OPEN_ASSESSMENT OR sourcetype=OPEN_TEST OR sourcetype=OPENAPI_DEV | eval CorrelationId = coalesce(CorrelationId,"Not exists") | stats count by CorrelationId,sourcetype | sort(CorrelationId,sourcetype)"

msrama5 · ‎04-22-2020

Hello, I have query which joins across 4 sources and correlationid may or may not exists across all sources, I want to print the id and exists or not exists on each source, wrote the query below and actual results only shows one source where value is present and does not show other 3 sources, any ideas how can this query be modified to show expected results ? Query - sourcetype=API_HUB OR sourcetype=API_RISK OR sourcetype=API_SECURE OR sourcetype=API_PAYMENT | eval CorrelationId = coalesce(CorrelationId,"Not exists") | table sourcetype,CorrelationId | dedup CorrelationId,sourcetype | sort(sourcetype,CorrelationId) Actual results sourcetype CorrelationId API_RISK 123456-34344-55555 Expected results sourcetype CorrelationId API_RISK 123456-34344-55555 API_PAYMENT Not exists API_SECURE Not exists API_HUB Not exists

msrama5 · ‎04-13-2020

Hello, I have the splunk query below which has multiple sourcetype rows and if the row has x-correlation-id keywpord matching it needs to return Status x-correlation-id (or) return missing otherwise, there are couple of rows which has x-correlation-id in header and all rows are currently returning as "missing" , any ideas why the match is not working ? environment=prod sourcetype=* "x-correlation-id" | fields sourcetype, Properties.Headers{} , Properties.CorrelationId, CorrelationId, Category | eval Status=if(match("Properties.Headers{}", "X-Correlation-id"),"x-correlation-id", "missing") | dedup sourcetype | table sourcetype,Status, Properties.CorrelationId, CorrelationId, Category | sort str(Properties.Headers{}) sourcetype status Properties.CorrelationId o1 missing c1 o2 missing c2 o3 missing c3 o4 missing c4 o5 missing c5 o6 missing c6 o7 missing c7 08 missing c8

msrama5 · ‎03-17-2020

Hello, I am trying to convert the working search url below for curl commands to app specific Working url - https://test.splunkcloud.com:8089/services/search/jobs Convert to app specific https://test.splunkcloud.com:8089/en-US/app/Sim/services/search/jobs Getting error 405 method not allowed when running app specific query above for app Sim, what is the way to change the working url for Sim app specific so I can use the curl command to run the query in sim app

msrama5 · ‎03-04-2020

Adding bracket is returning the same results , what I need is card id present in first query should not return based on the second sub search query, why would this query fail and return cardid present in first query ?

Posts	50
Solutions	0
Karma Given	0
Karma Received	1
Member Since	‎04-15-2019

Online Status	Offline
Date Last Visited	‎04-12-2023 03:29 PM

Are there way to optimize this query?

How do I get slo breaches over period of time?

sub search join not working , returning incorrect ...

Dynamically compute success percentage based on v...

Dynamically adjust percentage based on time for 5 ...

Splunk duplicate events analysis

Join fields with partly matching

Join 3 source types with common field which matche...

Two evals in one query , query not returning resul...

Elapsed time logging in splunk

Re: Search optimization

Re: Search optimization

Are there way to optimize this query?

How do I get slo breaches over period of time?

sub search join not working , returning incorrect ...

Dynamically compute success percentage based on v...

Dynamically adjust percentage based on time for 5 ...

Splunk duplicate events analysis

Re: SPLUNK JOIN FIELDS WITH PARTLY MATCHING

Join fields with partly matching

Join 3 source types with common field which matche...

Two evals in one query , query not returning resul...

Elapsed time logging in splunk

How to filter search results to group by one of th...

Re: Join 4 source types with common field after ev...

Re: Join 4 source types with common field after ev...

Re: Join 4 source types with common field after ev...

Re: Join 4 source types with common field after ev...

Join 4 source types with common field after eval

Re: Join across multiple sources display all sourc...

Re: Join across multiple sources display all sourc...

Join across multiple sources display all sources w...

Eval check condition not working

Getting error converting search url to app specifi...

Re: Splunk where not query returning incorrect res...