Yes, i need to extract errorcode and errormessage from log field which is in json format. The log looks like this: {"log":"21/Mar/2023:20:06:29 +1100 [defaultJmsListenerContainer-1] [correlationId=] ERROR au.com.commbank.pso.payments.pj.listener.util.LoggingUtil - Severity=ERROR, DateTimestamp=21/Mar/2023 20:06:29, ErrorCode=PJ_LISTENER_ERR_0003, ErrorMessage=PJ Listener connection to MQ has failed, MicroserviceName=PJ_LISTENER, ExceptionStackTrace=com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ0018: Failed to connect to queue manager 'NPAT01' with connection mode 'Client' and host name '10.31.39.168(1417)'.\\nCheck the queue manager is started and if running in client mode, check there is a listener running. Please see the linked exception for more information."}   I used this query to extract the data from this: index="a0_payservutil_generic_app_audit_npd" sourcetype="cba:appinfra:hec:json" "PJ_LISTENER_ERR_000*" | rename _raw as temp, log as _raw | kv | rename temp as _raw |table ErrorCode ErrorMessage     ... View more

[1=javax.net.ssl.SSLHandshakeException[Remote host terminated the handshake],3=ip-10-31-39-168.ap-southeast-2.compute.internal/10.31.39.168:1417 (ip-10-31-39-168.ap-southeast-2.compute.internal),4=SSLSocket.startHandshake,5=default]\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1325)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnection.connect(RemoteConnection.java:863)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSessionFromNewConnection(RemoteConnectionSpecification.java:409)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSession(RemoteConnectionSpecification.java:305)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionPool.getSession(RemoteConnectionPool.java:146)\\n\tat com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1730)\\n\t... 21 more\\nCaused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake\\n\tat sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1575)\\n\tat sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1405)\\n\tat sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305)\\n\tat sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1294)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1286)\\n\tat java.security.AccessController.doPrivileged(Native Method)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1286)\\n\t... 26 more\\nCaused by: java.io.EOFException: SSL peer shut down incorrectly\\n\tat sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:167)\\n\tat sun.security.ssl.SSLTransport.decode(SSLTransport.java:109)\\n\tat sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397)\\n\t... 32 more\\n java.io.EOFException: SSL peer shut down incorrectly\\n\tat sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:167)\\n\tat sun.security.ssl.SSLTransport.decode(SSLTransport.java:109)\\n\tat sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397)\\n\t... 32 common frames omitted\\nWrapped by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake\\n\tat sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1575)\\n\tat sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1405)\\n\tat sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305)\\n\tat sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1294)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1286)\\n\tat java.security.AccessController.doPrivileged(Native Method)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1286)\\n\t... 26 common frames omitted\\nWrapped by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host terminated the handshake],3=ip-10-31-39-168.ap-southeast-2.compute.internal/10.31.39.168:1417 (ip-10-31-39-168.ap-southeast-2.compute.internal),4=SSLSocket.startHandshake,5=default]\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1325)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnection.connect(RemoteConnection.java:863)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSessionFromNewConnection(RemoteConnectionSpecification.java:409)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSession(RemoteConnectionSpecification.java:305)\\n\tat com.ibm.mq.jmqi.remote.impl.RemoteConnectionPool.getSession(RemoteConnectionPool.java:146)\\n\tat com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1730)\\n\t... 21 common frames omitted\\nWrapped by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9204: Connection to host '10.31.39.168(1417)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host terminated the handshake],3=ip-10-31-39-168.ap-southeast-2.compute.internal/10.31.39.168:1417 (ip-10-31-39-168.ap-southeast-2.compute.internal),4=SSLSocket.startHandshake,5=default]],3=10.31.39.168(1417),5=RemoteTCPConnection.protocolConnect]\\n\tat com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:2282)\\n\tat com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1294)\\n\tat  '10.31.39.168(1417)'.\\n\tat  com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:299)\\n\tat com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)\\n\tat com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6024)\\n\tat com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6049)\\n\tat org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:410)\\n\tat org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:350)\\n\tat org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:328)\\n\tat org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:243)\\n\tat org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196)\\n\tat org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:412)\\n\tat {"container_id":"167559efc3f8cc23c1a2be1aa697978f1ec5092efc60b7f87b91f28a6f6dea76"},"kubernetes":{"container_name":"pj-npp-event-listener-psu-api","namespace_name":"msaas-badev","pod_name":"pj-npp-event-listener-psu-api-3.3.4.4-f57686594-q4shr","container_image":"pso.docker.internal.cba/pj-npp-jms-listener:3.3.4.4","container_image_id":"docker-pullable://pso.docker.internal.cba/pj-npp-jms-listener@sha256:83b20688216eb0ab4737123a03566c064bc565de4a89858978abdf4333b48ea0","pod_id":"8d7f77a2-0111-474d-82e2-c66103a4d807","pod_ip":"100.64.186.130","host":"ip-10-3-198-164.ap-southeast-2.compute.internal","labels":{"app":"pj-npp-event-listener","app.kubernetes.io/instance":"pj-npp-event-listener","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"pj-npp-event-listener","helm.sh/chart":"psu-api-1.7.0","heritage":"Helm","pod-template-hash":"f57686594","project":"pjpds","release":"pj-npp-event-listener"},"master_url":"https://172.20.0.1:443/api","namespace_id":"25c93690-5c3b-4f2b-a967-8d0355ea90f2","namespace_labels":{"argocd.argoproj.io/instance":"appspaces","ci":"CM0953076","kubernetes.io/metadata.name":"msaas-badev","name":"msaas-badev","platform":"PSU","service_owner":"somersd","spg":"CBA_PAYMENTS_TEST_COORDINATION"}},"hostname":"ip-10-3-198-164.ap-southeast-2.compute.internal","host_ip":"10.3.198.164","cluster":"nonprod/pmn02"} ... View more

Thanks Splunk champion..That was very clear...I Saw many videos for learning spath command and your solution for my question was crisp and clear...Easily understandable.Very helpful.Thanks a lot again. ... View more

Thanks for your reply gcusello...That works great splunk champ ... View more

Yes, i am able to see results now. But again one more issue i am facing i need to extract "ServiceCalloutEventData" field - which is highlighted under,it is not extracted as key value pair from log. here is the sample log: log\":\"[18:15:21.888] [INFO ] [] [c.c.n.t.e.i.T.ServiceCalloutEventData] [akka://MmsAuCluster/user/$b/workMonitorActor/$M+c] - channel=\\\"AutoNotification\\\", productVersion=\\\"2.3.3-0-1-eb5b8cadd\\\", apiVersion=\\\"V1\\\", uuid=\\\"0b8549ff-1f14-4fd5-99c5-b3f2240d7da8\\\", eventDateTime=\\\"2023-01-06T07:15:21.888Z\\\", severity=\\\"INFO ... View more

Yes i tried but not able to fetch results .i shared screenshot as well. ... View more

Hi Yuanliu,     Thanks for your reply. May be ia m using in wrong way or not i am not sure..i am not able to fetch results using your command.   And i tried using makeresults but my requirement is to fetch details from log events...i mean key value pairs which are present under field log.Can you please help me on the same.     ... View more

Hi Sir,         Not able to get perfect result. the alert is triggering for every 15 minutes as i gave the timespan of 15 minutes. Below is the query: index=a0_payservutil_generic_app_audit_prd MANDATE_NOTIFICATION_RETRIEVAL.CALLOUT* | rex field=log "eventName=\"*(?<eventName>[^,\"\s]+)"| rex field=log "serviceName=\"*(?<serviceName>[^\"]+)" | search eventName="MANDATE_NOTIFICATION_RETRIEVAL.CALLOUT*" AND serviceName="Consume Notification" | stats count by eventName|where count=0|eval message="No events"|fields eventName count message I tried with |where count<0 also...Same is happening..Can you help me in triggering an alert only when results are zero for such eventname ... View more

i need to trigger an alert only when the count is "0", I mean no events for that eventName ..I am stuck there ... View more

Hi Sir,     It is not working sir,,Eventhough there are results in last 15 minutes for that particular event i am able to get message from query.   I should get an alert when there are no results for particular eventName,I the above screenshot if you observe there are more than 100 events for that eventName still i am getting this no events message.It should not happen I should get an alert only when there are no events for that particular eventName in last 15 minutes   ... View more

Hi Gcusello,          Thanks a lot for your reply. But i am not able to view results  Below is the screenhot:   There is no events in last 40 seconds but i didnt get any message ,If i get any message an alert will trigger.That si the main motto of me.alert should trigger if there are no events in particular time.   ... View more

i have used same approach but no luck.. index=a0_payservutil_generic_app_audit_prd MANDATE_NOTIFICATION_RETRIEVAL.CALLOUT*| rex field=log "eventName=\"*(?<eventName>[^,\"\s]+)"| rex field=log "serviceName=\"*(?<serviceName>[^\"]+)"|search eventName="MANDATE_NOTIFICATION_RETRIEVAL.CALLOUT*" AND serviceName="Consume Notification" | stats count BY eventName | append [ | inputlookup mandatecount.csv | eval count=0 | fields eventName count ] | stats sum(count) AS total BY eventName |fillnull total | where total=0 Above is the query is used ... View more

Hi Whiser,        Thanks a lot...It is working for me ... View more

Hi gcusello,                  It is working and thanks for introducing new command as well to me..I will try to check documentation for spath command. Appreciate you for quick response ... View more