Build CloudWatch Dashboards For EKS Monitoring Using Prometheus Data

In this post, we’ll see how we can build CloudWatch dashboards for EKS monitoring using telemetry data stored in Prometheus. So far we have used Grafana to visualise Prometheus data but we can also leverage CloudWatch for this.

Step 1: Follow Step 1–7 given at https://aws.plainenglish.io/monitoring-eks-fargate-cluster-using-adot-and-amazon-managed-prometheus-grafana-c8b99289d661 to create an EKS cluster along with an Amazon managed Prometheus workspace to store telemetry data.

Step 2: Go to CloudWatch->All metrics->Multi source query and create a data source.

For data source type, select Amazon Prometheus

and select your AMP workspace which we created earlier

Step 3: Now go back to Multi source query and choose Prometheus data source and use PromQL queries to fetch data and build a dashboard.

Sample queries that you can use

sum (container_memory_working_set_bytes{id="/"}) Number
sum (machine_memory_bytes) Number 
sum (container_memory_working_set_bytes{id="/"}) / sum (machine_memory_bytes) 
sum (rate (container_cpu_usage_seconds_total{id="/"}[5m])) / sum (machine_cpu_cores) * 100
sum by (container, pod, namespace) (container_memory_working_set_bytes{container!=""})  / 1024 / 1024

to build a dashboard like this

Some other queries

label 
[min: ${MIN}, max: ${MAX}, avg: ${AVG}] 

(sum(rate(container_cpu_usage_seconds_total{name!="", namespace ="default" , pod=~".*test-api.*"}[5m])) BY (pod) / sum(kube_pod_container_resource_requests{resource="cpu", namespace ="default", pod=~".*test-api.*"} > 0 ) BY (pod) * 100)

label
[min: ${MIN}, max: ${MAX}, avg: ${AVG}] 


(sum(container_memory_working_set_bytes{name!="", namespace="default", pod=~".*test-api.*"}) BY (pod) / sum(kube_pod_container_resource_requests{resource="memory", namespace="default", pod=~".*test-api.*"} > 0 ) BY (pod)) * 100


label
[min: ${MIN}, max: ${MAX}] 
kube_deployment_status_replicas_available{namespace="app",deployment=~".*booking-api.*"}