I am looking at logs for asynchronous calls ( sending msg & receiving ack from kafka ) . So we have 2 event , first one is when we receive the message and start processing then send it to Kafka , second one is when we receive response back from kafka. I have unique message ID to track both event.
I want to capture average processing time for all unique ID. In below query I have not added condition for unique ID. in below query I am not getting "Diffrence" value.
Can you please help !!
index=web* "Message sent to Kafka" OR "Response received from Kafka"
| stats earlies(_time) as Msg_received, latest(_time) as Response_Kafka
| eval difference=Response_Kafka-Msg_received
| eval difference=strftime(difference,"%d-%m-%Y %H:%M:%S")
| eval Msg_received=strftime(Msg_received,"%d-%m-%Y %H:%M:%S")
| eval Response_Kafka=strftime(Response_Kafka,"%d-%m-%Y %H:%M:%S")
Hi @yuvrajsharma_13,
for the difference you have to use the tostring option (https://docs.splunk.com/Documentation/SCS/current/SearchReference/ConversionFunctions#tostring.28.26...) and not strftime that is used to convert date format, so please try this:
index=web* "Message sent to Kafka" OR "Response received from Kafka"
| stats earlies(_time) as Msg_received, latest(_time) as Response_Kafka by Unique_ID
| eval difference=tostring(Response_Kafka-Msg_received,"duration")
| eval Msg_received=strftime(Msg_received,"%d-%m-%Y %H:%M:%S")
| eval Response_Kafka=strftime(Response_Kafka,"%d-%m-%Y %H:%M:%S")
Ciao.
Giuseppe
Updated Query :
Time difference is coming as "12/31/23 19:00:30:295 "
index=web* "Message sent to Kafka" OR "Response received from Kafka"
| stats earlies(_time) as Msg_received, latest(_time) as Response_Kafka by Unique_ID
| eval difference=Response_Kafka-Msg_received
| eval difference=strftime(difference,"%d-%m-%Y %H:%M:%S")
| eval Msg_received=strftime(Msg_received,"%d-%m-%Y %H:%M:%S")
| eval Response_Kafka=strftime(Response_Kafka,"%d-%m-%Y %H:%M:%S")
Hi @yuvrajsharma_13,
for the difference you have to use the tostring option (https://docs.splunk.com/Documentation/SCS/current/SearchReference/ConversionFunctions#tostring.28.26...) and not strftime that is used to convert date format, so please try this:
index=web* "Message sent to Kafka" OR "Response received from Kafka"
| stats earlies(_time) as Msg_received, latest(_time) as Response_Kafka by Unique_ID
| eval difference=tostring(Response_Kafka-Msg_received,"duration")
| eval Msg_received=strftime(Msg_received,"%d-%m-%Y %H:%M:%S")
| eval Response_Kafka=strftime(Response_Kafka,"%d-%m-%Y %H:%M:%S")
Ciao.
Giuseppe
@gcusello , that helped, however now I have to change my query as we are not receiving response for few Unique_ID so difference is showing as 0 seconds.
I am using subsearch for this, so it should capture events for which we received response . Subsearch itself is not returning any results.
After this I need to work on time difference.
index=web* "Message sent to Kafka" | where UNIQUE_ID IN ( [ search index=web* "Response received from Kafka" | fields UNIQUE_ID ]) | table UNIQUE_ID, _time
Hi @yuvrajsharma_13,
as I said, if the issue is that the difference is showed as a date, you can use the tostring option to show in hours, minutes and seconds.
for the missing UNIQUE_ID, you found the solution.
let me know if I can help you more, or, please, accept one answer for the other people of Community.
Ciao and happy splunking
Giuseppe
P.S.: Karma Points are appreciated 😉