Monitoring Splunk

IOPS reported by bonnie++ and Splunk Monitoring console

koshyk
Super Champion

One of our client have 10K HDD in RAID10 and as per Bonnie++ Random Seeks (IOPS) comes to approx 1500 IOPS and wanted to build a dashboard for IOPS and disk usage. I was thinking to re-use the Monitoring console searches

But when I look into the Monitoring Console or DMC, the results show some Indexers of 6000 IOPS !! which is Not possible. Is this a problem with the Splunk api or does this involve RAM assistance?

the query used in DMC is:

| rest splunk_server_group=* splunk_server_group="*" /services/server/status/resource-usage/iostats   | eval iops = round(reads_ps + writes_ps)
0 Karma

nnmiller
Contributor

You can't sum these as you have in your query, since they are IOPS per disk. From the /services/server/status/resource-usage/iostats docs page:

Access the most recent disk I/O statistics for each disk. This endpoint is currently supported for Linux, Windows, and Solaris. By default this endpoint is updated every 60s seconds.

Running:

splunk cmd splunkd instrument-resource-usage --debug > some_log_file.log 2>&1

shows that these stats come from /proc/diskstats

(HT: blachance_splunk)

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...