k8stools 4.0.2__tar.gz → 4.2.0__tar.gz

{k8stools-4.0.2 → k8stools-4.2.0}/CHANGELOG.md

@@ -3,6 +3,85 @@
 All notable changes to k8stools. Versions follow [semantic versioning](https://semver.org/);
 dates are release-tag dates.
 
+## 4.2.0 — 2026-10-24
+
+### Issues fixed
+- [#28](https://github.com/BenedatLLC/k8stools/issues/28): get_pvc_summaries:
+  an unbound claim's events are not in its summary.
+
+### Added
+- `PVCSummary.events`: for a claim that is not Bound, its events, newest
+  first, at most five (issue #28). Why a claim waits (`ProvisioningFailed:
+  storageclass ... not found`, `WaitForFirstConsumer`) was in `get_events`,
+  and the pods that mount it in `get_pvc_summaries`, so no one result tied the
+  reason to the pod. The tool makes one extra events call when any claim is
+  unbound, and none when all are bound. Replays fill it from the capture's
+  events, so older captures replay it too; captures don't store it.
+- The `--mock` fixture has an unbound claim, `quote-cache`, whose storage
+  class doesn't exist, and a new `quote` Deployment whose only pod waits on
+  it. The scheduler's, PV controller's and Deployment controller's messages
+  for it were recorded from a real cluster with the same objects.
+- `WorkloadHealth.claims` and `WorkloadReport.claims`: the unbound claims an
+  unscheduled pod mounts, each with its storage class and newest event, as
+  'PersistentVolumeClaim/cache: Pending, storage class fast-ssd;
+  ProvisioningFailed: storageclass.storage.k8s.io "fast-ssd" not found'. The
+  scheduler's reason, "pod has unbound immediate PersistentVolumeClaims", said
+  the pod waited on a claim but not why; the claim's events do. The composites
+  read claims only when some pod is unscheduled.
+
+### Changed
+- `get_namespace_health` on the `--mock` fixture lists three unhealthy
+  workloads, not two: `Deployment/quote` (0/1) is new. Tests that pin the
+  mock's output will see it.
+
+### Fixed
+- The live test of replica set ordering failed on a cluster where a
+  Deployment had been rolled back: a rollback reuses an older replica set
+  under the newest revision number. It now accepts that when
+  `get_workload_history` marks the revision `reused`.
+
+### Tests
+- `tests/test_live_writes.py` creates the failures #27 and #28 describe (a
+  rollout whose new pod can't be scheduled, a claim on a missing storage
+  class) in a namespace of its own on a real cluster, and checks the tools
+  against the scheduler's and controllers' own messages. It writes to the
+  cluster, so it runs only with `K8STOOLS_LIVE_WRITE=1`. A third test has a
+  Deployment's pod mount a claim on a missing storage class, and checks that
+  the composites name the claim and the PV controller's event.
+
+## 4.1.0 — 2026-10-21
+
+### Issues fixed
+- [#27](https://github.com/BenedatLLC/k8stools/issues/27): get_namespace_health
+  counts a workload with a stuck rollout as healthy.
+
+### Changed
+- **A rollout that can't progress is not healthy** (issue #27). A Deployment
+  whose new pod couldn't be scheduled (an invalid nodeSelector on the ITBench
+  otel-demo) was listed healthy: its old pod was still ready, and a pod with no
+  node has no containers to be waiting. `get_namespace_health` now counts a
+  workload with a pod that has no node as unhealthy, and so is a Deployment
+  whose `Progressing` condition is `False` (e.g. `ProgressDeadlineExceeded`)
+  or whose `ReplicaFailure` is `True` (e.g. a ResourceQuota rejected its pods,
+  so there's no pod to find). `ready` still counts the pods serving.
+  Workloads not scheduled for the same reason are grouped in `common_failures`.
+
+### Added
+- `PodSummary.unscheduled`: why a pod has no node, from its `PodScheduled`
+  condition.
+- `DeploymentSummary.conditions`: `Available`, `Progressing` and
+  `ReplicaFailure`, with reason, message and how long ago each changed.
+- `WorkloadHealth` and `WorkloadReport` gain `unscheduled` (one line per pod:
+  how long, and the scheduler's reason) and `conditions` (a Deployment's
+  failing conditions, in its controller's words). `get_workload_report`
+  previously showed an unscheduled pod nowhere, since it has no instances.
+- The `--mock` fixture has a `checkout` Deployment whose new pod can't be
+  scheduled, past its progress deadline, and every Deployment has conditions.
+
+Captures made before 4.1.0 have neither field: an unscheduled pod is still
+found (no node and no container statuses), with its reason "not recorded", and
+`DeploymentSummary.conditions` is None.
+
 ## 4.0.2 — 2026-10-17
 
 ### Issues fixed

{k8stools-4.0.2 → k8stools-4.2.0}/CLAUDE.md

@@ -24,6 +24,7 @@ tests/
   test_mock_state.py         - Unit tests for MockState (filters, replicaset ordering, logs, replay clock)
   test_capture.py            - Capture serialization round-trip and capture-time redaction
   test_k8s_tools_realk8s.py  - Integration tests (real cluster, auto-skipped if unreachable)
+  test_live_writes.py        - Integration tests that create the failure they check, in their own namespace (opt-in: K8STOOLS_LIVE_WRITE=1)
   test_mock_tools.py         - Tests for mock_tools module (incl. parity with k8s_tools.TOOLS)
   test_events.py             - Event aggregation (count/first_seen), incl. the events.k8s.io fallback
   test_cluster_binding.py    - Cluster selection: configure(), the shared binding, get_cluster_info, server flags
@@ -35,6 +36,7 @@ tests/
   test_notes.py              - Description budgets, notes on results and logs, notes in replay
   test_composites.py         - get_namespace_health / get_workload_report: semantics, grouping, bounds
   test_init_containers.py    - Init containers and native sidecars: the pod model, composites, capture, older captures
+  test_stuck_rollouts.py     - Unscheduled pods and failing Deployment conditions make a workload unhealthy (#27)
   test_hpa.py                - get_hpa_summaries: metrics, conditions, at-max note, replay, composites
   test_endpoints.py          - get_endpoint_summaries and ServiceSummary endpoint counts
   test_metrics.py            - get_container_metrics / get_node_metrics: units, notes, the three replay cases
@@ -79,6 +81,10 @@ uv run pytest tests/test_k8s_tools.py tests/test_mock_tools.py -v
 # Integration tests against a real cluster (skipped automatically if unreachable)
 KUBECONFIG=~/.kube/your-config.yaml uv run pytest tests/test_k8s_tools_realk8s.py -v
 
+# Integration tests that create objects (an unbound claim, a stuck rollout) in a
+# k8stools-test-* namespace and delete it afterwards; skipped unless asked
+K8STOOLS_LIVE_WRITE=1 uv run pytest tests/test_live_writes.py -v
+
 # All tests
 uv run pytest -v
 ```
@@ -183,6 +189,17 @@ otherwise assembles from five or six. Rules that keep them trustworthy:
   running again after a termination) within `_RECOVERING_WINDOW` (24h) goes in
   `recovering`, in full, most recent first, capped at 10 (`recovering_more` lines
   after). A fixed window, stated in `NOTE_RECOVERING`; no "abnormal" threshold.
+- **Not scheduled is not healthy** (#27): a pod with no node has no container
+  statuses, so nothing in it waits, and mid-rollout the old pods keep `ready >=
+  desired`. A pod with no node makes its workload unhealthy, with the
+  scheduler's reason (`PodSummary.unscheduled`), and so does a Deployment's
+  `Progressing=False` or `ReplicaFailure=True` (the latter with no pod at all).
+  `ready` stays the count of pods serving. Captures before 4.1.0 have no reason:
+  there a pod with no node and no container status counts, with "reason not
+  recorded" (not node alone: test captures leave running pods' node unset).
+  The unbound claims an unscheduled pod mounts go in `claims`, with the claim's
+  newest event: the scheduler's "unbound immediate PersistentVolumeClaims" says
+  nothing of why. Claims are read only when some pod is unscheduled.
 - **Init containers apart** (#24): a completed init container (exit 0) is never
   a termination; one that hasn't completed makes the workload unhealthy and is
   named. `WorkloadReport.init_containers` is always serialized (`_keep_empty`), so
@@ -368,6 +385,10 @@ Invariants worth preserving when touching this code:
 - **A pod not in the capture does not exist** for any tool. `mock_tools` no longer
   synthesizes answers for unknown pod names, and `get_pod_container_statuses`
   raises for one, as live.
+- **An unbound claim's events are derived, not stored** (#28). Replay fills
+  `PVCSummary.events` from the capture's event records, so captures from before
+  the field replay it, and the capture drops it rather than store a copy that
+  could disagree with the event list.
 - **Init container statuses: absent means unknown, `[]` means none** (#24). A pod
   record's `init_container_statuses` key is written only when they were read;
   replay turns a missing key into `init_containers=None` with

{k8stools-4.0.2 → k8stools-4.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: k8stools
-Version: 4.0.2
+Version: 4.2.0
 Summary: Tools and an MCP server for building Kubernetes agents
 Project-URL: Homepage, https://github.com/BenedatLLC/k8stools
 Project-URL: Changelog, https://github.com/BenedatLLC/k8stools/blob/main/CHANGELOG.md
@@ -367,7 +367,7 @@ it is in `investigate`. Every tool is in `all`.
 
 | Tool | What it answers | T | I |
 |---|---|:-:|:-:|
-| `get_namespace_health` | What is wrong in a namespace, and where. A full entry for each unhealthy workload (last termination with the exit code's meaning, how long instances live, the restart back-off, memory limit vs request, when the template last changed), the same for each *recovering* one (ready, but restarted within the last 24 hours), and one line for each healthy one. Workloads failing the same way are grouped. | ✓ | ✓ |
+| `get_namespace_health` | What is wrong in a namespace, and where. A full entry for each unhealthy workload (last termination with the exit code's meaning, how long instances live, the restart back-off, memory limit vs request, when the template last changed), pods that can't be scheduled and a Deployment's failing rollout conditions, the same for each *recovering* one (ready, but restarted within the last 24 hours), and one line for each healthy one. Workloads failing the same way are grouped. | ✓ | ✓ |
 | `get_workload_report` | Everything about one workload in one call: images, resources and probes; each instance's state and last termination; deduplicated events; current and previous log tails; the last template change; the ConfigMaps and Secrets it uses; its Services, Ingress rules, HPA and resource usage. | ✓ | ✓ |
 | `get_cluster_info` | Which cluster the tools are answering from: kubeconfig context, API server URL and version, or the capture being replayed. | ✓ | ✓ |
 
@@ -424,7 +424,7 @@ out-of-memory kill".
 |---|---|:-:|:-:|
 | `get_configmap_summaries` | ConfigMaps, like `kubectl get configmaps` | | ✓ |
 | `get_configmap` | The full contents of one ConfigMap (redacted by the server) | | ✓ |
-| `get_pvc_summaries` | PersistentVolumeClaims, like `kubectl get pvc`, with the pods that mount each one | | ✓ |
+| `get_pvc_summaries` | PersistentVolumeClaims, like `kubectl get pvc`, with the pods that mount each one, and an unbound claim's events | | ✓ |
 
 ### Custom resources
 
@@ -766,7 +766,11 @@ application, with problems for an agent to find:
 * a crash-looping `ad` service, mid-upgrade, whose Service has no ready backends;
 * an `accounting` service that is ready but was OOM-killed 40 minutes ago, after
   578 restarts, and whose previous log file is gone;
+* a `checkout` rollout that can't finish: its new pod can't be scheduled (a node
+  selector no node matches), while the old pod still serves;
 * a Deployment whose HPA is at its maximum;
+* a new `quote` Deployment whose pod can't be scheduled, because the
+  PersistentVolumeClaim it mounts can't bind: its storage class doesn't exist;
 * an Ingress with one rule to a missing Service;
 * a cert-manager `Certificate` that isn't Ready;
 * a DaemonSet, a CronJob and its Job, a StatefulSet with PVCs and a completed
@@ -1136,7 +1140,21 @@ uv run pytest -v
 
 # Integration tests against a real cluster (skipped automatically if unreachable)
 KUBECONFIG=~/.kube/your-config.yaml uv run pytest tests/test_k8s_tools_realk8s.py -v
+
+# Integration tests that write to the cluster (skipped unless asked)
+K8STOOLS_LIVE_WRITE=1 KUBECONFIG=~/.kube/your-config.yaml uv run pytest tests/test_live_writes.py -v
 ```
 
+The integration tests above only read what the cluster has. The ones in
+`tests/test_live_writes.py` create the failures they check, such as a claim on a
+storage class that doesn't exist or a rollout whose new pod can't be scheduled,
+and compare the tools' output with the cluster's own messages. Each test creates
+a `k8stools-test-<random>` namespace and deletes it afterwards, so it needs
+permission to create and delete namespaces, Deployments and
+PersistentVolumeClaims. The tests pull `registry.k8s.io/pause`. Use a test
+cluster such as Minikube or kind. They take about 30 seconds. If a run is
+interrupted, remove what it left with
+`kubectl get ns -o name | grep k8stools-test- | xargs -r kubectl delete`.
+
 Planned and deferred work is in
 [docs/ROADMAP.md](https://github.com/BenedatLLC/k8stools/blob/main/docs/ROADMAP.md).

{k8stools-4.0.2 → k8stools-4.2.0}/README.md

@@ -146,7 +146,7 @@ it is in `investigate`. Every tool is in `all`.
 
 | Tool | What it answers | T | I |
 |---|---|:-:|:-:|
-| `get_namespace_health` | What is wrong in a namespace, and where. A full entry for each unhealthy workload (last termination with the exit code's meaning, how long instances live, the restart back-off, memory limit vs request, when the template last changed), the same for each *recovering* one (ready, but restarted within the last 24 hours), and one line for each healthy one. Workloads failing the same way are grouped. | ✓ | ✓ |
+| `get_namespace_health` | What is wrong in a namespace, and where. A full entry for each unhealthy workload (last termination with the exit code's meaning, how long instances live, the restart back-off, memory limit vs request, when the template last changed), pods that can't be scheduled and a Deployment's failing rollout conditions, the same for each *recovering* one (ready, but restarted within the last 24 hours), and one line for each healthy one. Workloads failing the same way are grouped. | ✓ | ✓ |
 | `get_workload_report` | Everything about one workload in one call: images, resources and probes; each instance's state and last termination; deduplicated events; current and previous log tails; the last template change; the ConfigMaps and Secrets it uses; its Services, Ingress rules, HPA and resource usage. | ✓ | ✓ |
 | `get_cluster_info` | Which cluster the tools are answering from: kubeconfig context, API server URL and version, or the capture being replayed. | ✓ | ✓ |
 
@@ -203,7 +203,7 @@ out-of-memory kill".
 |---|---|:-:|:-:|
 | `get_configmap_summaries` | ConfigMaps, like `kubectl get configmaps` | | ✓ |
 | `get_configmap` | The full contents of one ConfigMap (redacted by the server) | | ✓ |
-| `get_pvc_summaries` | PersistentVolumeClaims, like `kubectl get pvc`, with the pods that mount each one | | ✓ |
+| `get_pvc_summaries` | PersistentVolumeClaims, like `kubectl get pvc`, with the pods that mount each one, and an unbound claim's events | | ✓ |
 
 ### Custom resources
 
@@ -545,7 +545,11 @@ application, with problems for an agent to find:
 * a crash-looping `ad` service, mid-upgrade, whose Service has no ready backends;
 * an `accounting` service that is ready but was OOM-killed 40 minutes ago, after
   578 restarts, and whose previous log file is gone;
+* a `checkout` rollout that can't finish: its new pod can't be scheduled (a node
+  selector no node matches), while the old pod still serves;
 * a Deployment whose HPA is at its maximum;
+* a new `quote` Deployment whose pod can't be scheduled, because the
+  PersistentVolumeClaim it mounts can't bind: its storage class doesn't exist;
 * an Ingress with one rule to a missing Service;
 * a cert-manager `Certificate` that isn't Ready;
 * a DaemonSet, a CronJob and its Job, a StatefulSet with PVCs and a completed
@@ -915,7 +919,21 @@ uv run pytest -v
 
 # Integration tests against a real cluster (skipped automatically if unreachable)
 KUBECONFIG=~/.kube/your-config.yaml uv run pytest tests/test_k8s_tools_realk8s.py -v
+
+# Integration tests that write to the cluster (skipped unless asked)
+K8STOOLS_LIVE_WRITE=1 KUBECONFIG=~/.kube/your-config.yaml uv run pytest tests/test_live_writes.py -v
 ```
 
+The integration tests above only read what the cluster has. The ones in
+`tests/test_live_writes.py` create the failures they check, such as a claim on a
+storage class that doesn't exist or a rollout whose new pod can't be scheduled,
+and compare the tools' output with the cluster's own messages. Each test creates
+a `k8stools-test-<random>` namespace and deletes it afterwards, so it needs
+permission to create and delete namespaces, Deployments and
+PersistentVolumeClaims. The tests pull `registry.k8s.io/pause`. Use a test
+cluster such as Minikube or kind. They take about 30 seconds. If a run is
+interrupted, remove what it left with
+`kubectl get ns -o name | grep k8stools-test- | xargs -r kubectl delete`.
+
 Planned and deferred work is in
 [docs/ROADMAP.md](https://github.com/BenedatLLC/k8stools/blob/main/docs/ROADMAP.md).

{k8stools-4.0.2 → k8stools-4.2.0}/docs/TOOL_REFERENCE.md

@@ -134,9 +134,12 @@ over the pod's age is not a rate, and pointing to metrics
 #### Returns
 
 - **`namespace`** (`str`)
-- **`workloads`** (`list[WorkloadHealth]`): the unhealthy workloads, by name. A workload is healthy when all desired pods are ready, no container is waiting, every init container has completed, and (for a Job) no exit failed. A completed init container's exit 0 is not a termination; one that hasn't completed is, and the entry's `notes` name it (e.g. "Init container 'migrate': Waiting (CrashLoopBackOff)"). Native sidecars count as app containers. Each has:
+- **`workloads`** (`list[WorkloadHealth]`): the unhealthy workloads, by name. A workload is healthy when all desired pods are ready, no container is waiting, every init container has completed, every pod has a node, (for a Deployment) no condition is failing, and (for a Job) no exit failed. A completed init container's exit 0 is not a termination; one that hasn't completed is, and the entry's `notes` name it (e.g. "Init container 'migrate': Waiting (CrashLoopBackOff)"). Native sidecars count as app containers. Each has:
   - **`workload`** (`str`): "Kind/name", e.g. "Deployment/ad", or "Pod/name" for a pod nothing owns.
-  - **`ready`**, **`desired`** (`int`): ready and desired pods; for a Job, 1/1 once it succeeded.
+  - **`ready`**, **`desired`** (`int`): ready and desired pods; for a Job, 1/1 once it succeeded. Mid-rollout this counts the pods serving, which may be the old revision's.
+  - **`unscheduled`** (`list[str]`): its pods that have no node, with how long and the scheduler's reason: "ad-7d9f-x2k: not scheduled for 3h12m: Unschedulable: 0/3 nodes are available: 3 node(s) didn't match Pod's node affinity/selector." Such a pod has no containers, so it is in no other field. From a capture made before 4.1.0 the reason reads "reason not recorded"; a pod there counts as unscheduled when it has neither a node nor any container status.
+  - **`claims`** (`list[str]`): the claims those pods mount that aren't Bound, each with its status, storage class and newest event: 'PersistentVolumeClaim/cache: Pending, storage class fast-ssd; ProvisioningFailed: storageclass.storage.k8s.io "fast-ssd" not found'. The scheduler says only "pod has unbound immediate PersistentVolumeClaims"; why a claim is unbound is in the claim's events. A claim with none reads "no events recorded" (events expire, by default after an hour). Read from `get_pvc_summaries`, only when some pod is unscheduled; empty when claims can't be read or the capture has none.
+  - **`conditions`** (`list[str]`): a Deployment's failing conditions in its controller's words: `Progressing=False` (e.g. `ProgressDeadlineExceeded`: the rollout made no progress within `progressDeadlineSeconds`, 10 minutes by default) or `ReplicaFailure=True` (e.g. `FailedCreate`: a ResourceQuota or an admission webhook rejected its pods, so there may be no pod to show).
   - **`restarts`** (`int`): container restarts across the workload's pods; absent when none of its pods could be found.
   - **`last_termination`** (`Termination`): the most recent termination across its pods and containers:
     - **`pod`**, **`container`**
@@ -156,7 +159,7 @@ over the pod's age is not a rate, and pointing to metrics
 - **`ingress_problems`** (`list[str]`): Ingress rules that can't reach a pod: a backend that doesn't resolve, or one with no ready endpoints, e.g. "Ingress/shop: shop.example.com/old -> legacy:80: Service 'legacy' not found".
 - **`recovering`** (`list[WorkloadHealth]`): workloads that pass the healthy rule (all ready, nothing waiting) but have a container that restarted within the last 24 hours: its last termination finished then, and it is running again. Same fields as `workloads`, with `last_termination` the most recent restart. Most recent first, at most 10. A Job's finished container and a completed init container are not restarts. "Ready" is not "healthy": a workload that restarts and recovers would otherwise be listed as healthy, and answers quote that word.
 - **`recovering_more`** (`list[str]`): recovering workloads past the first 10, one line each, e.g. "Deployment/w10 1/1, 5 restarts, last terminated 11h0m ago: exit 137 (OOMKilled)".
-- **`healthy`** (`list[str]`): one line per healthy workload (ready, and no restart within 24 hours), e.g. "Deployment/cart 3/3", with what's worth knowing about it added: restarts, "HPA at max 3", or "memory at 99.8% of limit" when a container is at 90% or more of its memory limit (with the working-set note in `notes`).
+- **`healthy`** (`list[str]`): one line per healthy workload (as defined under `workloads`, and no restart within 24 hours), e.g. "Deployment/cart 3/3", with what's worth knowing about it added: restarts, "HPA at max 3", or "memory at 99.8% of limit" when a container is at 90% or more of its memory limit (with the working-set note in `notes`).
 - **`common_failures`** (`list[FailureGroup]`): two or more unhealthy workloads whose last terminations share an exit code, reason and memory shape. The signature also gives their instance lifetimes ("lifetimes 2s-20s"), which are shown rather than required to match: one workload's instances can live 7s, 20s and 2m on successive restarts. A shared signature is a fact worth checking for a common cause, not a conclusion.
 - **`notes`** (`list[str]`)
 
@@ -182,6 +185,7 @@ those tools, so it answers the same way from a replayed capture.
 #### Returns
 
 - **`workload`**, **`ready`**, **`desired`**
+- **`unscheduled`**, **`claims`**, **`conditions`** (`list[str]`): as in `get_namespace_health`.
 - **`containers`** (`list[ContainerEssentials]`): per container of the focus pod: `image`, `requests`, `limits`, and `probes` (e.g. "liveness: httpGet :8080/healthz every 10s, fails after 3", or "none configured"). Native sidecars come after the app containers, with `sidecar: true`.
 - **`init_containers`** (`list[ContainerEssentials]`): the focus pod's init containers, from its spec. Always present: `[]` means it has none.
 - **`instances`** (`list[Instance]`): every pod's containers now: `pod_age` (time since the pod was created; when its restarts happened isn't recorded), `state`, `started` (time since the current instance started), `ready`, `restarts`, and `last_termination` as in `get_namespace_health`.
@@ -276,6 +280,7 @@ A list of PodSummary objects, each providing a summary of a pod's status with th
 - **`age`** (`datetime.timedelta`): Age of the pod (current time minus creation timestamp).
 - **`ip`** (`Optional[str]`): Pod IP address (None if not assigned).
 - **`node`** (`Optional[str]`): Name of the node where the pod is running (None if not scheduled).
+- **`unscheduled`** (`Optional[str]`): For a pod with no node, why: its `PodScheduled` condition's reason and message, e.g. "Unschedulable: 0/3 nodes are available: 3 node(s) didn't match Pod's node affinity/selector.", or "no scheduling decision recorded yet" (`NOT_YET_SCHEDULED`). None once it has a node, and in captures made before 4.1.0.
 - **`owner`** (`Optional[str]`): The pod's controlling owner, as "Kind/name" - the object that created and manages it, e.g. "DaemonSet/otel-collector-agent", "StatefulSet/valkey-cart", "Job/backup-29318400". None for a bare pod nobody manages. This is the direct owner only: a Deployment's pods are owned by one of its replica sets ("ReplicaSet/ad-7d9f8c6b5"), whose `owner_deployment` (`get_replicaset_summaries`) names the Deployment; likewise a Job's `owner` names its CronJob. Static (mirror) pods such as the control plane's report "Node/<node name>". Group pods into workloads by this field rather than by stripping name suffixes.
 - **`notes`** (`list[str]`): Warnings that apply to this item; empty when none do. See [Notes in results](#notes-in-results).
 
@@ -439,6 +444,7 @@ A list of DeploymentSummary objects, each providing a summary of a deployment's
 - **`up_to_date_replicas`** (`int`): Number of replicas that are up to date.
 - **`available_replicas`** (`int`): Number of replicas that are available.
 - **`age`** (`datetime.timedelta`): Age of the deployment (current time minus creation timestamp).
+- **`conditions`** (`Optional[list[ResourceCondition]]`): Its status conditions, each with `type`, `status`, `reason`, `message` and `since` (time since its status last changed): `Available`; `Progressing` (`True` while a rollout progresses or once it completed, `False` with reason `ProgressDeadlineExceeded` when it stopped progressing); `ReplicaFailure` (`True` when pods couldn't be created, e.g. `FailedCreate` from a ResourceQuota). Mid-rollout `ready_replicas` counts the old revision's pods too, so these say whether the rollout itself is moving. None when a replayed capture didn't record them (made before 4.1.0).
 
 ### `get_replicaset_summaries`
 
@@ -1028,7 +1034,9 @@ namespace or all namespaces, similar to `kubectl get pvc`.
 
 For each PVC, this also resolves which pods currently mount it (by scanning pod
 volumes in the same scope), which is useful for spotting orphaned PVCs — claims
-with an empty `mounted_by` and no owning pod.
+with an empty `mounted_by` and no owning pod. A claim that is not Bound also
+carries its recent events, which say why it waits: they and the pods that
+mount it are in one result.
 
 #### Parameters
 
@@ -1048,6 +1056,7 @@ A list of PVCSummary objects, each with the following fields:
 - **`storage_class`** (`Optional[str]`): StorageClass backing the claim.
 - **`mounted_by`** (`list[str]`): Names of pods (in the same scope) currently mounting this PVC. Empty for orphaned/unmounted claims.
 - **`age`** (`datetime.timedelta`): Age of the PVC (current time minus creation timestamp).
+- **`events`** (`list[EventSummary]`): For a claim that is not Bound, its events, newest first, at most five (e.g. `ProvisioningFailed: storageclass.storage.k8s.io "fast-ssd" not found`, or `WaitForFirstConsumer`). Empty for a bound claim. In a replay, they come from the capture's event records.
 
 ### `get_events`
 

{k8stools-4.0.2 → k8stools-4.2.0}/pyproject.toml

@@ -1,7 +1,7 @@
 
 [project]
 name = "k8stools"
-version = "4.0.2"
+version = "4.2.0"
 description = "Tools and an MCP server for building Kubernetes agents"
 readme = "README.md"
 requires-python = ">=3.12"

{k8stools-4.0.2 → k8stools-4.2.0}/src/k8stools/__init__.py

@@ -98,4 +98,4 @@ the built-in OTel Demo snapshot by default, or your own via load_mock_state() or
 the server's --state-file option.
 """
 
-__version__ = "4.0.2"
+__version__ = "4.2.0"

{k8stools-4.0.2 → k8stools-4.2.0}/src/k8stools/capture.py

@@ -217,7 +217,10 @@ def capture_state(namespaces: Optional[list[str]] = None,
                     redactor(k8s_tools.get_workload_history(w.name, ns, kind)), captured_at))
         state["cronjobs"] += _encode_all(redactor(k8s_tools.get_cronjob_summaries(ns)), captured_at)
         state["jobs"] += _encode_all(redactor(k8s_tools.get_job_summaries(ns)), captured_at)
-        state["pvcs"] += _encode_all(redactor(k8s_tools.get_pvc_summaries(ns)), captured_at)
+        # Without their events: replay derives them from the capture's event
+        # records, so a stored copy would only duplicate those, unread.
+        state["pvcs"] += [{k: v for k, v in record.items() if k != "events"} for record in
+                          _encode_all(redactor(k8s_tools.get_pvc_summaries(ns)), captured_at)]
         state["configmaps"] += _capture_configmaps(ns, captured_at, redactor)
         state["events"] += _capture_events(ns, captured_at, redactor)
         state["pods"] += _capture_pods(ns, captured_at, include_logs, max_log_lines,

{k8stools-4.0.2 → k8stools-4.2.0}/src/k8stools/composites.py

@@ -26,6 +26,7 @@ over the record, not the current rhythm.
 """
 
 import datetime
+import functools
 import re
 import signal
 from typing import Any, Optional
@@ -36,7 +37,7 @@ from .k8s_tools import (ContainerEssentials, ContainerUsage, EndpointSummary, Hp
                         ContainerStateWaiting, EventLine, FailureGroup, Instance, K8sApiError,
                         K8sLogReclaimed, LastChange, LOG_NOTE_PREFIX, LOG_RECLAIMED_REASON,
                         LogTail, NamespaceHealth, NOTE_INIT_NOT_RECORDED, NOTE_INIT_PENDING,
-                        NOTE_RECOVERING, NOTE_RESTART_HISTORY,
+                        NOTE_RECOVERING, NOTE_RESTART_HISTORY, PVCSummary,
                         Termination, WorkloadHealth, WorkloadReport, _init_completed,
                         _to_whole_seconds)
 
@@ -88,21 +89,89 @@ def _now() -> datetime.datetime:
     return datetime.datetime.now(datetime.timezone.utc)
 
 
-def _workloads(api, namespace: str) -> list[tuple[str, str, int, int]]:
-    """(kind, name, ready, desired) for each workload in the namespace."""
+def _workloads(api, namespace: str) -> list[tuple[str, str, int, int, list[str]]]:
+    """(kind, name, ready, desired, failing conditions) for each workload in the
+    namespace."""
     out = []
     for d in api.get_deployment_summaries(namespace):
-        out.append(("Deployment", d.name, d.ready_replicas, d.total_replicas))
+        out.append(("Deployment", d.name, d.ready_replicas, d.total_replicas,
+                     _failing_conditions(d.conditions or [])))
     for s in api.get_statefulset_summaries(namespace):
-        out.append(("StatefulSet", s.name, s.ready_replicas, s.total_replicas))
+        out.append(("StatefulSet", s.name, s.ready_replicas, s.total_replicas, []))
     for d in api.get_daemonset_summaries(namespace):
-        out.append(("DaemonSet", d.name, d.number_ready, d.desired_number_scheduled))
+        out.append(("DaemonSet", d.name, d.number_ready, d.desired_number_scheduled, []))
     for j in api.get_job_summaries(namespace):
         # A Job is "ready" when it has succeeded; desired is 1 run.
-        out.append(("Job", j.name, 1 if j.succeeded else 0, 1))
+        out.append(("Job", j.name, 1 if j.succeeded else 0, 1, []))
     return out
 
 
+#: Deployment conditions in the state that means a rollout can't make progress:
+#: past its progress deadline, or unable to create pods (quota, admission).
+_FAILING_CONDITIONS = {("Progressing", "False"), ("ReplicaFailure", "True")}
+
+
+def _failing_conditions(conditions) -> list[str]:
+    """A Deployment's failing conditions in its controller's words, e.g.
+    'Progressing=False (ProgressDeadlineExceeded): ReplicaSet "x" has timed out
+    progressing.'"""
+    return [f"{c.type}={c.status}" + (f" ({c.reason})" if c.reason else "")
+            + (f": {c.message[:200]}" if c.message else "")
+            for c in conditions if (c.type, c.status) in _FAILING_CONDITIONS]
+
+
+#: WorkloadHealth.unscheduled when a capture recorded no reason (before 4.1.0).
+_REASON_NOT_RECORDED = "reason not recorded (the capture predates k8stools 4.1.0)"
+
+
+def _unscheduled(pod, statuses) -> Optional[str]:
+    """Why a pod isn't scheduled, or None if it is.
+
+    A pod is unscheduled when it has no node. Captures before 4.1.0 recorded no
+    reason; there, a pod with no node and no container statuses at all hasn't
+    been scheduled either (a scheduled pod's kubelet reports its containers).
+    """
+    if pod.node is not None or (pod.unscheduled is None
+                                and (statuses.containers or statuses.init_containers)):
+        return None
+    return pod.unscheduled or _REASON_NOT_RECORDED
+
+
+def _unscheduled_line(pod, reason: str) -> str:
+    return f"{pod.name}: not scheduled for {_human(pod.age)}: {reason}"
+
+
+def _claim_line(c: PVCSummary) -> str:
+    """An unbound claim and why it waits, in its newest event's words, e.g.
+    'PersistentVolumeClaim/cache: Pending, storage class fast-ssd; ProvisioningFailed:
+    storageclass.storage.k8s.io "fast-ssd" not found'."""
+    line = f"PersistentVolumeClaim/{c.name}: {c.status}" + (
+        f", storage class {c.storage_class}" if c.storage_class else "")
+    if not c.events:
+        return line + "; no events recorded (events expire, by default after an hour)"
+    e = c.events[0]
+    return line + f"; {e.reason}: {e.message[:200]}"
+
+
+def _unbound_claims(api, namespace: str) -> dict[str, list[str]]:
+    """The unbound claims each pod mounts, as _claim_line()s, by pod name.
+
+    The scheduler says only that a pod has unbound claims; why a claim is
+    unbound is in the claim's events. Best-effort: the claims are context for
+    a pod already reported unscheduled.
+    """
+    try:
+        claims = api.get_pvc_summaries(namespace)
+    except K8sApiError:
+        return {}
+    by_pod: dict[str, list[str]] = {}
+    for c in claims:
+        if c.status != "Bound":
+            for pod in c.mounted_by:
+                by_pod.setdefault(pod, []).append(_claim_line(c))
+    return by_pod
+
+
 #: Generated pod-name suffixes: a ReplicaSet's, Job's or DaemonSet's pods are
 #: "<name>-<5 chars>"; a StatefulSet's are "<name>-<ordinal>".
 _RANDOM_SUFFIX = r"-[a-z0-9]{5}"
@@ -115,20 +184,20 @@ NOTE_MATCHED_BY_NAME = (
 
 
 def _match_by_name(pod_name: str, rs_owner: dict[str, str],
-                   workloads: list[tuple[str, str, int, int]]) -> Optional[tuple[str, str]]:
+                   workloads: list[tuple]) -> Optional[tuple[str, str]]:
     """The workload a pod's generated name points to, for a pod with no owner."""
     for rs, deployment in rs_owner.items():
         if deployment and re.fullmatch(re.escape(rs) + _RANDOM_SUFFIX, pod_name):
             return ("Deployment", deployment)
     for kind, suffix in (("StatefulSet", _ORDINAL_SUFFIX), ("Job", _RANDOM_SUFFIX),
                          ("DaemonSet", _RANDOM_SUFFIX)):
-        for k, name, _, _ in workloads:
+        for k, name, *_ in workloads:
             if k == kind and re.fullmatch(re.escape(name) + suffix, pod_name):
                 return (kind, name)
     return None
 
 
-def _pods_by_workload(api, namespace: str, workloads: list[tuple[str, str, int, int]]
+def _pods_by_workload(api, namespace: str, workloads: list[tuple]
                       ) -> tuple[dict[tuple[str, str], list], list, bool]:
     """Pods grouped by their workload; pods no workload owns; and whether any
     pod was matched by name.
@@ -309,11 +378,20 @@ def namespace_health(api, namespace: str = "default") -> NamespaceHealth:
     entries: list[WorkloadHealth] = []
     recovering: list[WorkloadHealth] = []
     healthy_lines: list[str] = []
+    unschedulable: dict[str, list[str]] = {}  # workloads by why their pods aren't scheduled
+    # Read only when some pod is unscheduled, and then once.
+    unbound_claims = functools.cache(lambda: _unbound_claims(api, namespace))
 
-    def assess(label, kind, name, ready, desired, pods):
-        terminations, first_pod, waiting, init_pending = [], [], False, []
+    def assess(label, kind, name, ready, desired, pods, conditions):
+        terminations, first_pod, waiting, init_pending, unscheduled = [], [], False, [], {}
+        claims: dict[str, None] = {}
         for i, pod in enumerate(pods):
             statuses = api.get_pod_container_statuses(pod.name, namespace)
+            # A pod with no node has no containers to wait on: without this, the
+            # old pods of a rollout that can't schedule its new one read healthy.
+            if reason := _unscheduled(pod, statuses):
+                unscheduled[_unscheduled_line(pod, reason)] = reason
+                claims.update(dict.fromkeys(unbound_claims().get(pod.name, [])))
             for s in statuses.containers:
                 if i == 0:
                     first_pod.append(s)
@@ -337,7 +415,7 @@ def namespace_health(api, namespace: str = "default") -> NamespaceHealth:
         hpa = autoscalers.get(label)
         pod_names = {p.name for p in pods}
         readings = [u for u in usage or [] if u.pod in pod_names]
-        if ready >= desired and not waiting and not failed_job:
+        if ready >= desired and not (waiting or failed_job or unscheduled or conditions):
             # A restart: a container running again after a termination (not a
             # Job's container that finished, nor a completed init container).
             restart = _latest([t for t in terminations if t.restart_gap is not None])
@@ -354,8 +432,12 @@ def namespace_health(api, namespace: str = "default") -> NamespaceHealth:
                                  + (f", memory at {peak}% of limit"
                                     if peak is not None and peak >= _NEAR_LIMIT_PERCENT else ""))
             return
-        entries.append(entry(label, kind, name, ready, desired, pods, restarts, last,
-                             first_pod, hpa, readings, init_pending))
+        w = entry(label, kind, name, ready, desired, pods, restarts, last, first_pod, hpa,
+                  readings, init_pending)
+        w.unscheduled, w.claims, w.conditions = list(unscheduled), list(claims), conditions
+        for reason in set(unscheduled.values()):
+            unschedulable.setdefault(reason, []).append(label)
+        entries.append(w)
 
     def entry(label, kind, name, ready, desired, pods, restarts, last, first_pod, hpa,
               readings, init_pending) -> WorkloadHealth:
@@ -380,11 +462,12 @@ def namespace_health(api, namespace: str = "default") -> NamespaceHealth:
             last.notes = []
         return w
 
-    for kind, name, ready, desired in workloads:
-        assess(f"{kind}/{name}", kind, name, ready, desired, grouped.get((kind, name), []))
+    for kind, name, ready, desired, conditions in workloads:
+        assess(f"{kind}/{name}", kind, name, ready, desired, grouped.get((kind, name), []),
+               conditions)
     for pod in standalone:
         assess(f"Pod/{pod.name}", None, pod.name, pod.ready_containers,
-               pod.total_containers, [pod])
+               pod.total_containers, [pod], [])
 
     entries.sort(key=lambda w: w.workload)
     recovering.sort(key=lambda w: w.last_termination.finished)
@@ -408,6 +491,11 @@ def namespace_health(api, namespace: str = "default") -> NamespaceHealth:
                            f"{memory or 'unknown'}, "
                            f"{_lifetimes([m.last_termination for m in members])}"),
                 workloads=[m.workload for m in members]))
+    # Pods that can't be scheduled for the same reason (a node selector no node
+    # matches, set on several workloads at once) are one failure too.
+    failures += [FailureGroup(signature=f"not scheduled: {reason}", workloads=sorted(members))
+                 for reason, members in unschedulable.items()
+                 if len(members) > 1 and reason != _REASON_NOT_RECORDED]
     result = NamespaceHealth(
         namespace=namespace, workloads=entries, recovering=recovering[:_MAX_RECOVERING],
         recovering_more=recovering_more, healthy=healthy_lines,
@@ -446,7 +534,7 @@ _MAX_EVENTS = 20
 
 
 def _find_kind(api, name: str, namespace: str) -> str:
-    found = [kind for kind, n, _, _ in _workloads(api, namespace) if n == name]
+    found = [kind for kind, n, *_ in _workloads(api, namespace) if n == name]
     if not found:
         raise K8sApiError(f"No Deployment, StatefulSet, DaemonSet or Job named '{name}' "
                           f"in namespace '{namespace}'.")
@@ -589,13 +677,14 @@ def workload_report(api, name: str, namespace: str = "default", kind: Optional[s
         raise K8sApiError(f"Unsupported kind '{kind}': expected one of {', '.join(_KINDS)}.")
     kind = kind or _find_kind(api, name, namespace)
     workloads = _workloads(api, namespace)
-    matches = [(r, d) for k, n, r, d in workloads if (k, n) == (kind, name)]
+    matches = [(r, d, c) for k, n, r, d, c in workloads if (k, n) == (kind, name)]
     if not matches:
         raise K8sApiError(f"{kind} '{name}' not found in namespace '{namespace}'.")
-    ready, desired = matches[0]
+    ready, desired, conditions = matches[0]
     grouped, _, by_name = _pods_by_workload(api, namespace, workloads)
     pods = grouped.get((kind, name), [])
-    report = WorkloadReport(workload=f"{kind}/{name}", ready=ready, desired=desired)
+    report = WorkloadReport(workload=f"{kind}/{name}", ready=ready, desired=desired,
+                            conditions=conditions)
     if by_name and pods and all(p.owner is None for p in pods):
         report.notes.append(NOTE_MATCHED_BY_NAME)
 
@@ -615,7 +704,12 @@ def workload_report(api, name: str, namespace: str = "default", kind: Optional[s
         return [s for s in statuses_by_pod[pod.name].init_containers or []
                 if not _init_completed(s)]
 
+    unbound_claims = functools.cache(lambda: _unbound_claims(api, namespace))
     for pod in pods:
+        if reason := _unscheduled(pod, statuses_by_pod[pod.name]):
+            report.unscheduled.append(_unscheduled_line(pod, reason))
+            report.claims.extend(c for c in unbound_claims().get(pod.name, [])
+                                 if c not in report.claims)
         report.instances.extend(instance(pod, s) for s in statuses_by_pod[pod.name].containers)
         report.init_instances.extend(instance(pod, s) for s in pending_init(pod))
     if any(st.init_containers is None for st in statuses_by_pod.values()):