Hi @gmemcc. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Hi @gmemcc , thanks for proposing this!

Just to confirm, which metric(s) are initialized to 0 and causing the distortion on first time collection? Could you share a Prometheus metrics output (curl localhost:20257/metrics) or a screenshot with me? Thanks!

(From my understanding, only the queue length should have this problem. But I want to confirm.

/ok-to-test

@xueweiz Thanks for your review and comments, you are right, it's my misunderstanding. only disk_avg_queue_len is distorted because its metric type is 'gauge'

here's my metrics report with the config file config/system-stats-monitor.json, sdb is idle during the test:

on startup:

disk_avg_queue_len{device_name="sdb"} 69.453342040168
disk_io_time{device_name="sdb"} 4.1983876e+07
disk_weighted_io{device_name="sdb"} 2.9159205e+09

1 minutes later:

disk_avg_queue_len{device_name="sdb"} 0
disk_io_time{device_name="sdb"} 4.1983876e+07
disk_weighted_io{device_name="sdb"} 2.9159205e+09

It doesn't matter although the io time is very large.

/lgtm
/approve

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gmemcc, xueweiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here