Splunk Search

Dedup is not working with mstats

sabari80
Explorer

We are streaming Dynatrace metric data into Splunk, for some reason we are seeing duplicate 'MessageDeduplicationId'. So trying to avoid the duplicate entries using dedup command. But not retrieving any results after using dedup command. Here is my initial query and getting results for this with duplicates-

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") After adding dedup to avoid duplicate 'MessageDeduplicationId' , no results | mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id 
| eval Response=round((Response/1000000),2), Count=round(Count,0) 
| search Dimension.id IN ("*Process.aspx") 
| dedup MessageDeduplicationId sample payload: Dimension.id: xxxProcess.aspx Dimension.name: Literal Not Found MessageDeduplicationId: a901b712889217fc194cd0446a70325e aggregation: avg entity.service.id: xxx entity.service.name:xxxx metric_name:calc:service.thaa_stress_requests_lr_tags: 1613759 resolution: 1m source.name: xxxx unit: MicroSecond
Labels (2)
Tags (1)
0 Karma

sabari80
Explorer

sample Payload

=========

 

sample payload:

Dimension.id: xxxProcess.aspx

Dimension.name: Literal Not Found

MessageDeduplicationId: a901b712889217fc194cd0446a70325e

aggregation: avg

entity.service.id: xxx

entity.service.name:xxxx

metric_name:calc:xxxx_

lr_tags: 1613759

resolution: 1m s

ource.name: xxxx

unit: MicroSecond
0 Karma

sabari80
Explorer

Modified Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") | dedup MessageDeduplicationId 
0 Karma

sabari80
Explorer

Initial Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx")
0 Karma
Get Updates on the Splunk Community!

Archived Metrics Now Available for APAC and EMEA realms

We’re excited to announce the launch of Archived Metrics in Splunk Infrastructure Monitoring for our customers ...

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Dashboard Challenge and Watch the .conf24 Global Broadcast!

The Splunk Community Dashboard Challenge is still happening, and it's not too late to enter for the week of ...