Note: For pod resource requests and limits, kube-scheduler exposes kube_pod_resource_request and kube_pod_resource_limit on its /metrics/resources endpoint. These are computed from the pod's effective resource requirements and already account for pod-level resources (pod.spec.resources, beta and enabled by default since Kubernetes 1.34), so they are recommended over the container-level metrics below. kube-state-metrics does not expose a separate metric for pod-level resources. These metrics are defined in KEP-1748.

It is not straightforward to get the Pod states for certain cases like "Terminating" and "Unknown" since it is not stored behind a field in the Pod.Status.

So to mimic the logic used by the kubectl command line, you will need to compose multiple metrics.

For example:

• To get the list of pods that are in the Unknown state, you can run the following PromQL query: sum(kube_pod_status_phase{phase="Unknown"}) by (namespace, pod) or (count(kube_pod_deletion_timestamp) by (namespace, pod) * sum(kube_pod_status_reason{reason="NodeLost"}) by(namespace, pod))

• For Pods in Terminating state: count(kube_pod_deletion_timestamp) by (namespace, pod) * count(kube_pod_status_reason{reason="NodeLost"} == 0) by (namespace, pod)

Here is an example of a Prometheus rule that can be used to alert on a Pod that has been in the Terminating state for more than 5m.

groups:
- name: Pod state
  rules:
  - alert: PodsBlockedInTerminatingState
    expr: count(kube_pod_deletion_timestamp) by (namespace, pod) * count(kube_pod_status_reason{reason="NodeLost"} == 0) by (namespace, pod) > 0
    for: 5m
    labels:
      severity: page
    annotations:
      summary: Pod {{$labels.namespace}}/{{$labels.pod}} blocked in Terminating state.