Appreciate if you can share some example . ... View more

is there way to add AM OR PM according to time. ... View more

My apologies i was using "eventTimestamp" instead of  "@timestamp" in my rex command  i just realized and its working now , However i do not need date in last column need only time. Please help how to do that. please find below details  ================================================================================ Query index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*" |fields message |rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})" |rex field=_raw "sourceSystem=(?<Source>[^,]*)" |rex field=_raw "entityType=(?<Entity>\w+)" |rex field=_raw "\"@timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please help Here  |sort Time desc |dedup Entity |table Source, BusDate, Entity, Time ================================================================================ Screenshot   ------------------------------------------------------------------------------------------- raw data {"@timestamp":"2024-04-04T02:25:59.366Z","level":"INFO","message":"Snapshot event published: SnapshotEvent(version=SnapshotVersion(sourceSystem=dbI-LDN, entityType=ACCOUNT, subType=, date=2024-04-03, version=1, snapshotSize=326718, uuid=8739e273-cedc-482b-b696-48357efc8704, eventTimestamp=2024-04-04T02:24:52.762129638), status=CREATED)","thread":"snapshot-checker-3","loggerName":"com.db.sdda.dc.kafka.snapshot.writer.InternalEventSender"} Show syntax highlighted ... View more

My apologies i was using "eventTimestamp" instead of  "@timestamp" in my rex command  i just realized and its working now , However i do not need date in last column need only time. Please help how to do that. please find below details  ================================================================================ Query index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*" |fields message |rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})" |rex field=_raw "sourceSystem=(?<Source>[^,]*)" |rex field=_raw "entityType=(?<Entity>\w+)" |rex field=_raw "\"@timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please help Here  |sort Time desc |dedup Entity |table Source, BusDate, Entity, Time ================================================================================ Screenshot ------------------------------------------------------------------------------------------- raw data {"@timestamp":"2024-04-04T02:25:59.366Z","level":"INFO","message":"Snapshot event published: SnapshotEvent(version=SnapshotVersion(sourceSystem=dbI-LDN, entityType=ACCOUNT, subType=, date=2024-04-03, version=1, snapshotSize=326718, uuid=8739e273-cedc-482b-b696-48357efc8704, eventTimestamp=2024-04-04T02:24:52.762129638), status=CREATED)","thread":"snapshot-checker-3","loggerName":"com.db.sdda.dc.kafka.snapshot.writer.InternalEventSender"} Show syntax highlighted ... View more

Still not working i replaced semicolon with "=" sign Please check screenshot. ============================================================================= Sample raw data   ... View more

Yes, i released that its not "timestamp " and its changes to "eventTimestamp" in raw data  However  modified query but still its not working. ====================================================================== index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*" |fields message |rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})" |rex field=_raw "sourceSystem=(?<Source>[^,]*)" |rex field=_raw "entityType=(?<Entity>\w+)" |rex field=_raw "\"eventTimestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please suggest here  |sort Time desc |dedup Entity |table Source, BusDate, Entity, Time ====================================================================== attaching sample raw screenshot for your reference     ... View more

please check the sample raw data , where i need time only ... View more

Also please check below query which is working , however it does not giving me required output , i need only time. in Last column =============================================================== index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*" |fields message |rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})" |rex field=_raw "sourceSystem=(?<Source>[^,]*)" |rex field=_raw "entityType=(?<Entity>\w+)" |rex field=_raw "eventTimestamp=(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"   --> Need only time  |sort Time desc |dedup Entity |table Source, BusDate, Entity, Time ================================================================   please check screenshot for more clear understanding     ... View more

Unable to understand solution , could you please elaborate more    I see in raw data as below eventTimestamp=2024-04-04T02:24:52.762129638)   i would like extract time from above like = 02:24  ... View more

some of report i observed records shows somehing in 3d format  ... View more

Many Thanks below query working as expected  index=* ("Exception occurred during ORC" OR ("Response received, system status NOK") AND NOT "n. a." sourcetype="kube:container:*slimits*-service") namespace IN ("dk1692-e","dk1399-c","dk1371-c","dk1398-c","dk1400-d") | eval Date=strftime(_time, "%F") | eval Time=strftime(_time, "%T") | regex _raw="system status NOK for (EMS|ORC|SDS|FCS|ROUTER)|Exception occurred during (EMS|ORC|SDS|FCS|ROUTER)" | rex "(?<Status>system status NOK for (EMS|ORC|SDS|FCS|ROUTER)|Exception occurred during (EMS|ORC|SDS|FCS|ROUTER))" |table Date, Time, Status   is there any option to make more formatting and presentable      ... View more

Please note that its not only for EMS and ORC , there are multiple keywords  i wanted to find below and convert status column Exception occurred during [ORC,SDS,FCS,ROUTER] Response received, system status NOK for  [ORC,SDS,FCS,ROUTER] Note - your provided query only search for EMS and ORC  can you assist something like with | rex field=_raw <Date> | rex field=_raw <Time> | rex field=_raw <Status>   Sample record    2023-05-18 05:16:48,083 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-2) - [e2eCallReference: ] Response received, system status NOK for FCS     2023-05-14 22:32:18,020 ERROR [com.db.gtb.bankingapi.slimits.orc.scheduled.OrcScheduler] (task-pool-3) - [e2eCallReference: ] Exception occurred during ORC, due to Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: ems-pool - Connection is not available, request timed out after 5000ms..     2023-05-14 22:32:11,204 ERROR [com.db.gtb.bankingapi.slimits.orc.scheduled.OrcScheduler] (task-pool-2) - [e2eCallReference: ] Exception occurred during ORC, due to Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: tds-pool - Connection is not available, request timed out after 5000ms..     2023-05-14 22:31:53,882 ERROR [com.db.gtb.bankingapi.slimits.orc.scheduled.OrcScheduler] (task-pool-4) - [e2eCallReference: ] Exception occurred during ORC, due to Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: tds-pool - Connection is not available, request timed out after 5000ms..     2023-05-14 22:31:48,996 ERROR [com.db.gtb.bankingapi.slimits.orc.scheduled.OrcScheduler] (task-pool-4) - [e2eCallReference: ] Exception occurred during ORC, due to Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: tds-pool - Connection is not available, request timed out after 5000ms..     2023-05-14 22:31:48,897 ERROR [com.db.gtb.bankingapi.slimits.orc.scheduled.OrcScheduler] (task-pool-9) - [e2eCallReference: ] Exception occurred during ORC, due to Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: sds-pool - Connection is not available, request timed out after 5001ms..     2023-05-13 05:07:03,306 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-9) - [e2eCallReference: ] Response received, system status NOK for SDS     2023-05-13 05:06:53.020 [INFO] [scheduling-1] orcCheck(OrcServiceImpl.java:60) - Response received, system status NOK for ROUTER     2023-05-13 05:06:38,190 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-3) - [e2eCallReference: ] Response received, system status NOK for FCS     2023-05-13 05:06:19,503 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-10) - [e2eCallReference: ] Response received, system status NOK for FCS     2023-05-13 05:06:13,067 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-2) - [e2eCallReference: ] Response received, system status NOK for EMS     2023-05-13 05:06:05.808 [INFO] [scheduling-1] orcCheck(OrcServiceImpl.java:60) - Response received, system status NOK for ROUTER     2023-05-13 05:05:43,982 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-9) - [e2eCallReference: ] Response received, system status NOK for SDS     2023-05-13 05:05:35,650 INFO [com.db.gtb.bankingapi.slimits.orc.service.internal.OrcServiceImpl] (task-pool-5) - [e2eCallReference: ] Response received, system status NOK for EMS ... View more

Could you please help to segregate date and time in separate column  ... View more

Thank you so much ... View more

you mean i need to add value namespace = $result.namespace$ into Suppress results containing field value ??   please confirm    ... View more

I would like to use throttle option  But when am using each result option for trigger its asking field name suppressing.   how to manage that ?  ... View more

index=* ("ORC from FCS completed" OR "ORC from SDS completed." OR "ORC from ROUTER completed" OR "ORC from EMS completed" OR "ORC from TDS completed") namespace IN ("dk1692-b","dk1399-b","dk1371-b","dk1398-b","dk1400-c") | eval dk1692=if(searchmatch("\"ORC from ROUTER completed\" namespace=dk1692-b"),1,0) | eval dk1399=if(searchmatch("\"ORC from SDS completed\" namespace=dk1399-b"),1,0) | eval dk1371=if(searchmatch("\"ORC from FCS completed\" namespace=dk1371-b"),1,0) | eval dk1398=if(searchmatch("\"ORC from EMS completed\" namespace=dk1398-b"),1,0) | eval dk1400=if(searchmatch("\"ORC from TDS completed\" namespace=dk1400-c"),1,0) | stats sum(dk*) as dk* | search dk1692<90 OR dk1399<60 OR dk1371<60 OR dk1398<60 OR dk1400<60 | eval _time=now() | untable _time namespace count | where (namespace="dk1692" and count<90) OR (namespace="dk1399" and count<60) OR (namespace="dk1371" and count<60) OR (namespace="dk1398" and count<60) OR (namespace="dk1400" and count<60)   ---------------------------------------------------------------------------------------------------------------- Thanks in above query , i would like to add one more column which says which namespace is below threshold  example if namespace dk1399-b below 60 it column is should show name SDS    ... View more

index=* ("ORC from FCS completed" OR "ORC from SDS completed." OR "ORC from ROUTER completed") namespace IN ("dk1692-b","dk1399-b","dk1371-b") | eval dk1692=if(searchmatch("\"ORC from ROUTER completed\" namespace=dk1692-b"),1,0) | eval dk1399=if(searchmatch("\"ORC from SDS completed\" namespace=dk1399-b"),1,0) | eval dk1371=if(searchmatch("\"ORC from FCS completed\" namespace=dk1371-b"),1,0) | stats sum(dk*) as dk* | search dk1692<90 OR dk1399<60 OR dk1371<60 | eval _time=now() | untable _time namespace count | where (namespace="dk1692" and count<90) OR (namespace="dk1399" and count<60) OR (namespace="dk1371" and count<60) ------------------------------------------------------------------------------------------------------------- this query good as per my requirement , however how can i setup alert with trigger  Because its giving one row and count. for setting up condition can i use  number of results - is greater than = 0 *** So whenever there is record in above query it will send alert.            ... View more

Yes but there are three namespaces with different threshould  Do i have to setup 3 different alerts by using less than trigger ??     ... View more

Sorry , please check latest screen shot , in this wherever  namespace value less than threashhod it shoud throw email alert  For example - dk1399 is 45 which is less than 60   ... View more