Add cgroup cpu/mem/disk usage metrics #16472

adithyachakilam · 2024-05-20T03:18:20Z

Description

This PRs expands the existing metrics reported by the Cgroup monitors. Specifically adds the usage for cpu, memory and disk.

Release note

Introduces new usage metrics for cpu and memory cgroups

Key changed/added classes in this PR

CgroupCpuMonitor.java
CgroupMemoryMonitor.java
CgroupDiskMonitor.java

This PR has:

processing/src/main/java/org/apache/druid/java/util/metrics/CgroupCpuMonitor.java

processing/src/main/java/org/apache/druid/java/util/metrics/cgroups/CgroupUtil.java

processing/src/main/java/org/apache/druid/java/util/metrics/cgroups/Cpu.java

docs/operations/metrics.md

arunramani · 2024-05-20T22:12:02Z

Added a few comments but overall, the change looks good to me. Were you able to validate that you were getting the right values for these? One way would be to test your assumptions in Docker and see if your numbers match with what Docker sees.

docs/operations/metrics.md

adithyachakilam · 2024-05-20T23:52:31Z

Were you able to validate that you were getting the right values for these? One way would be to test your assumptions in Docker and see if your numbers match with what Docker sees.

Yeah tested in docker to see the formats match. But for cpu usage percentage, where do you want me to confirm from ?

arunramani · 2024-05-20T23:54:03Z

Were you able to validate that you were getting the right values for these? One way would be to test your assumptions in Docker and see if your numbers match with what Docker sees.

Yeah tested in docker to see the formats match. But for cpu usage percentage, where do you want me to confirm from ?

The Docker desktop UI displays CPU usage for a container, but I'm not 100% sure if even Docker does it right. But if it does, it'll give you additional confidence.

adithyachakilam · 2024-05-21T16:46:20Z

The Docker desktop UI displays CPU usage for a container, but I'm not 100% sure if even Docker does it right. But if it does, it'll give you additional confidence.

I have tested with Datadog, even though the %s aren't exactly matching up, the spikes are matching. So I think it is safe to assume working fine.

docs/operations/metrics.md

…rics

processing/src/test/java/org/apache/druid/java/util/metrics/CgroupDiskMonitorTest.java

processing/src/test/java/org/apache/druid/java/util/metrics/cgroups/DiskTest.java

processing/src/main/java/org/apache/druid/java/util/metrics/CgroupCpuMonitor.java


  public CgroupCpuMonitor(CgroupDiscoverer cgroupDiscoverer, final Map<String, String[]> dimensions, String feed)
  {
    super(feed);
    this.cgroupDiscoverer = cgroupDiscoverer;
    this.dimensions = dimensions;
+    try {
+      Process p = new ProcessBuilder("getconf", "CLK_TCK").start();


suneet-s

Thanks @adithyachakilam !

Add cgroup cpu/mem usage metrics

fe9d836

github-actions bot added Area - Documentation Area - Metrics/Event Emitting labels May 20, 2024

checks

38fccc0