Hi, I am trying to create a timechart using mstats command but I have some questions as follows, I would appreciate it if I am able to get some answers or clarifications on them: What is the difference between the aggregations which are rate_avg() and rate_sum() when using mstats command? We observed that no matter which aggregations we are using, the graphs are returning the same result. Example are as follows: Using rate_avg Using rate_sum Thank you very much.   Best Regards, Kelvin.   @ericaooi  ... View more

Hi, I had tried to recreate Prometheus metrics graphs from Grafana in Splunk. However, I am getting offsets for the value of certain queries as shown in the case below: Case 1: Queries that are using irate in PromQL Eg: PromQL Queries: (sum by(instance) (irate(node_cpu_seconds_total{instance="$node",job="$job", mode!="idle"}[$__rate_interval])) / on(instance) group_left sum by (instance)((irate(node_cpu_seconds_total{instance="$node",job="$job"}[$__rate_interval])))) * 100 Result: Splunk Queries: | mstats rate_sum(node_cpu_seconds_total) as seconds_total where index=<index_name> by job instance mode span=15s | sort - _time | dedup mode | stats sum(seconds_total) as seconds_total sum(eval(if(mode!="idle",seconds_total,0))) as cpu_busy | eval "CPU Busy" = round((cpu_busy / seconds_total) * 100,2) | fields "CPU Busy" Result:   2. Queries that are not using irate in PromQL: PromQL Queries: avg(node_load5{instance="$node",job="$job"}) /  count(count(node_cpu_seconds_total{instance="$node",job="$job"}) by (cpu)) * 100 Result:   Splunk Queries: | mstats avg("node_load5") prestats=true WHERE index=<index_name> span=15s | table _time psrsvd_sm_node_load5 | sort - _time | stats first(psrsvd_sm_node_load5) as latest_psrsvd_sm_node_load5 | join type=inner[| mstats count(node_cpu_seconds_total) prestats=true WHERE index=<index_name> by cpu span=15s | table cpu psrsvd_ct_node_cpu_seconds_total | dedup cpu | stats count(cpu) as cpu_count | table cpu_count] | eval sys_load=latest_psrsvd_sm_node_load5 / cpu_count * 100 | sort - sys_load | table sys_load Result:     Can I check is there answers or solutions on the following questions please? What is the irate equivalent aggregate functions that we should use in Splunk. For example: rate, rate_sum, rate_avg or another that is not listed? What might be the cause of the values in Grafana and Splunk having an offset? Thank you very much. ... View more

I have go through a few videos in youtube and documentation provided but I am unable to find a source with the steps to integrate the data into Splunk Observability Cloud. Most of the tutorials available have the integration done pre-recording- and the videos are mainly to explain on the functionality of Splunk APM.  May I have a reference on how to setup the Splunk APM in Splunk Observability Cloud? ... View more