k8stools 1.2.0__tar.gz → 2.0.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. {k8stools-1.2.0 → k8stools-2.0.2}/CLAUDE.md +62 -4
  2. {k8stools-1.2.0 → k8stools-2.0.2}/PKG-INFO +216 -8
  3. {k8stools-1.2.0 → k8stools-2.0.2}/README.md +213 -7
  4. k8stools-2.0.2/designs/mock-state-capture.md +754 -0
  5. {k8stools-1.2.0 → k8stools-2.0.2}/docs/ROADMAP.md +11 -9
  6. {k8stools-1.2.0 → k8stools-2.0.2}/pyproject.toml +4 -5
  7. {k8stools-1.2.0 → k8stools-2.0.2}/src/k8stools/__init__.py +9 -1
  8. k8stools-2.0.2/src/k8stools/capture.py +398 -0
  9. k8stools-2.0.2/src/k8stools/fixtures/otel-demo.json +570 -0
  10. {k8stools-1.2.0 → k8stools-2.0.2}/src/k8stools/k8s_tools.py +17 -1
  11. {k8stools-1.2.0 → k8stools-2.0.2}/src/k8stools/mcp_client.py +14 -16
  12. {k8stools-1.2.0 → k8stools-2.0.2}/src/k8stools/mcp_server.py +36 -5
  13. k8stools-2.0.2/src/k8stools/mock_state.py +649 -0
  14. k8stools-2.0.2/src/k8stools/mock_tools.py +211 -0
  15. {k8stools-1.2.0 → k8stools-2.0.2}/src/k8stools/redaction.py +80 -11
  16. k8stools-2.0.2/tests/test_capture.py +367 -0
  17. k8stools-2.0.2/tests/test_mock_state.py +331 -0
  18. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_mock_tools.py +43 -22
  19. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_new_tools.py +4 -1
  20. k8stools-2.0.2/tests/test_redaction.py +248 -0
  21. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_replicaset_tool.py +16 -0
  22. {k8stools-1.2.0 → k8stools-2.0.2}/uv.lock +6 -234
  23. k8stools-1.2.0/designs/mock-state-capture.md +0 -395
  24. k8stools-1.2.0/src/k8stools/mock_tools.py +0 -603
  25. k8stools-1.2.0/tests/test_redaction.py +0 -133
  26. {k8stools-1.2.0 → k8stools-2.0.2}/.claude/settings.local.json +0 -0
  27. {k8stools-1.2.0 → k8stools-2.0.2}/.github/workflows/unit-tests.yml +0 -0
  28. {k8stools-1.2.0 → k8stools-2.0.2}/.gitignore +0 -0
  29. {k8stools-1.2.0 → k8stools-2.0.2}/.python-version +0 -0
  30. {k8stools-1.2.0 → k8stools-2.0.2}/.vscode/mcp.json +0 -0
  31. {k8stools-1.2.0 → k8stools-2.0.2}/.vscode/settings.json +0 -0
  32. {k8stools-1.2.0 → k8stools-2.0.2}/LICENSE +0 -0
  33. {k8stools-1.2.0 → k8stools-2.0.2}/chat-with-cluster.png +0 -0
  34. {k8stools-1.2.0 → k8stools-2.0.2}/envrc.template +0 -0
  35. {k8stools-1.2.0 → k8stools-2.0.2}/tests/conftest.py +0 -0
  36. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_k8s_tools.py +0 -0
  37. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_k8s_tools_realk8s.py +0 -0
  38. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_mcp_client.py +0 -0
  39. {k8stools-1.2.0 → k8stools-2.0.2}/tests/test_version.py +0 -0
@@ -7,15 +7,21 @@ Kubernetes monitoring tools for AI agents, exposed as direct Python functions or
7
7
  ```
8
8
  src/k8stools/
9
9
  k8s_tools.py - Core tools: Kubernetes API wrappers with Pydantic return types
10
- mock_tools.py - Static mock versions of all tools (captured from real Minikube + OTel Demo)
10
+ mock_state.py - MockState: loads a capture file and serves it to the tool queries
11
+ capture.py - `k8s-capture-state` CLI: snapshots a live cluster to a capture file
12
+ mock_tools.py - Mock versions of all tools; thin delegates to MockState
11
13
  redaction.py - Secret-redaction pass applied at the MCP server output boundary
12
14
  mcp_server.py - MCP server (stdio or streamable-http transport)
13
15
  mcp_client.py - MCP client used for manual testing
16
+ fixtures/
17
+ otel-demo.json - Built-in capture served by `--mock` (real Minikube + OTel Demo)
14
18
  tests/
15
19
  test_k8s_tools.py - Unit tests for the original tools (mocked K8s API)
16
20
  test_new_tools.py - Unit tests for the 1.1.0 tools (ConfigMaps, CronJobs/Jobs, PVCs, StatefulSets, events, log enhancements)
17
21
  test_replicaset_tool.py - Unit tests for get_replicaset_summaries (1.2.0), incl. mock-data consistency
18
22
  test_redaction.py - Unit tests for the secret-redaction pass
23
+ test_mock_state.py - Unit tests for MockState (filters, replicaset ordering, logs, replay clock)
24
+ test_capture.py - Capture serialization round-trip and capture-time redaction
19
25
  test_k8s_tools_realk8s.py - Integration tests (real cluster, auto-skipped if unreachable)
20
26
  test_mock_tools.py - Tests for mock_tools module (incl. parity with k8s_tools.TOOLS)
21
27
  test_mcp_client.py - MCP client tests
@@ -96,13 +102,56 @@ k8s-mcp-server
96
102
  # HTTP — for remote access
97
103
  k8s-mcp-server --transport=streamable-http --port=8000
98
104
 
99
- # Mock mode (no real cluster needed)
105
+ # Mock mode (no real cluster needed) — serves fixtures/otel-demo.json
100
106
  k8s-mcp-server --mock
101
107
 
108
+ # Replay a capture of your own; `frozen` pins ages for a repeatable test suite
109
+ k8s-mcp-server --state-file incident-1234.json
110
+ k8s-mcp-server --state-file incident-1234.json --state-time frozen
111
+
102
112
  # Disable secret redaction (on by default)
103
113
  k8s-mcp-server --no-redact
104
114
  ```
105
115
 
116
+ ## Mock state capture & replay
117
+
118
+ `k8s-capture-state` snapshots a live cluster into a single JSON file that
119
+ `MockState` replays through the whole tool surface. Design and its deviations:
120
+ `designs/mock-state-capture.md`.
121
+
122
+ ```bash
123
+ k8s-capture-state -o incident-1234.json # whole cluster
124
+ k8s-capture-state --namespace otel-demo -o incident.json # one namespace
125
+ ```
126
+
127
+ Invariants worth preserving when touching this code:
128
+
129
+ - **All time is stored relative to `captured_at`** (`<field>_seconds` for
130
+ `timedelta`, `<field>_offset_seconds` for `datetime`) and reconstructed against
131
+ the server's start time. This is what makes *intervals between* resources survive
132
+ replay — "deployment 8d old, current replica set 7h34m old, so it was upgraded
133
+ 7h34m ago". The replay clock quantizes to whole seconds and samples once per
134
+ query to keep that exact; don't reintroduce per-field `datetime.now()`.
135
+ - **`MockState` must match the real tool's documented behavior**, not just its
136
+ types — filter semantics, result ordering (`get_replicaset_summaries` re-sorts
137
+ at query time), and which failures raise vs. return empty.
138
+ - **The capture applies redaction itself.** It is a library consumer of the `get_*`
139
+ functions, so it bypasses the MCP server's redaction boundary; without an
140
+ explicit `redact_object` pass it would write raw secrets to disk while the server
141
+ in front of the same cluster redacted them. `tests/test_capture.py` guards this.
142
+ - **A pod not in the capture does not exist** for any tool. `mock_tools` no longer
143
+ synthesizes answers for unknown pod names.
144
+
145
+ The built-in fixture lives under `src/k8stools/` (not `tests/`) because
146
+ `--mock` serves it, so it has to ship in the wheel.
147
+
148
+ Captures are written **indented, and uncompressed by default**, on purpose: these
149
+ files get re-captured and re-committed, and git deltas successive versions of
150
+ indented JSON well while it cannot delta a gzip blob at all (measured: +61 KB per
151
+ update as `.json`, +358 KB as `.gz`). An output name ending in `.gz` writes
152
+ gzipped for files that travel outside a repo; reading detects gzip by magic bytes,
153
+ not by extension.
154
+
106
155
  ## Secret redaction
107
156
 
108
157
  `redaction.py` applies a single redaction pass to every tool's return value at the
@@ -110,8 +159,17 @@ MCP server output boundary (`mcp_server.py` wraps each tool with
110
159
  `wrap_with_redaction`). It is **on by default** and opt-out via `--no-redact` or
111
160
  `K8STOOLS_REDACT=0`. It matches secret-shaped values (AWS keys, JWTs, PEM private
112
161
  keys) and values under keys/env-var names matching
113
- `key|secret|token|password|credential`, replacing them with a visible `[REDACTED]`
114
- marker. Direct Python callers of the tool functions get raw values (no redaction) and
162
+ `key|secret|token|password|credential` **as a whole word** (names split on
163
+ separators and camelCase, so `apiKey` matches and `VALKEY_ADDR` does not),
164
+ replacing them with a visible `[REDACTED]` marker.
165
+
166
+ A field named *exactly* `key` is exempt — in the Kubernetes API that is always
167
+ structural (taint, map entry, label selector, projected-volume item), never a
168
+ credential. Before that exemption a real 38-pod capture produced 139 redactions of
169
+ which 3 were secrets; after, 5. If you widen the name rule, re-measure against a
170
+ real capture rather than reasoning about it — the false positives are not where
171
+ you would guess (toleration keys, `ca.crt`, and `secretKeyRef.key`, which is a
172
+ pointer to a secret, not a secret). Direct Python callers of the tool functions get raw values (no redaction) and
115
173
  can call `redaction.redact_object` themselves. We never add a reader for `Secret`
116
174
  objects.
117
175
 
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: k8stools
3
- Version: 1.2.0
3
+ Version: 2.0.2
4
4
  Summary: Tools and an MCP server for building Kubernetes agents
5
5
  Project-URL: Homepage, https://github.com/BenedatLLC/k8stools
6
6
  Author: Jeff Fischer
@@ -212,7 +212,9 @@ Classifier: Programming Language :: Python :: 3
212
212
  Requires-Python: >=3.12
213
213
  Requires-Dist: kubernetes>=36.0.3
214
214
  Requires-Dist: mcp>=2.1.1
215
+ Requires-Dist: pydantic>=2.0
215
216
  Requires-Dist: pyyaml>=6.0
217
+ Requires-Dist: rich>=14.0.0
216
218
  Description-Content-Type: text/markdown
217
219
 
218
220
  # Kubernetes Tools
@@ -225,6 +227,8 @@ directly to an agent as tools or placed behind an MCP server (included). Some us
225
227
  * Build [agents](https://github.com/BenedatLLC/orca-agent/) to monitor your cluster or perform root cause analysis.
226
228
  * Vibe-code a custom chat UI.
227
229
  * Use in non-agentic automations.
230
+ * Snapshot a live cluster to a file and replay it as a test fixture, with no cluster
231
+ needed — see [Capturing and replaying a cluster](#capturing-and-replaying-a-cluster).
228
232
 
229
233
  <a href="https://glama.ai/mcp/servers/@BenedatLLC/k8stools">
230
234
  <img width="380" height="200" src="https://glama.ai/mcp/servers/@BenedatLLC/k8stools/badge" alt="Kubernetes Tools Server MCP server" />
@@ -265,6 +269,10 @@ Via `uv`:
265
269
  uv add k8stools
266
270
  ```
267
271
 
272
+ This installs three commands: `k8s-mcp-server` (the MCP server), `k8s-mcp-client`
273
+ (a client for manual testing), and `k8s-capture-state` (snapshot a cluster to a
274
+ replayable file).
275
+
268
276
  ## Current tools
269
277
 
270
278
  These are the tools we define:
@@ -331,7 +339,7 @@ Here are the command line arguments for the server:
331
339
  ```
332
340
  usage: k8s-mcp-server [-h] [--transport {streamable-http,stdio}] [--host HOST] [--port PORT]
333
341
  [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--debug] [--mock]
334
- [--no-redact]
342
+ [--state-file FILE] [--state-time {advancing,frozen}] [--no-redact]
335
343
 
336
344
  Run the MCP server.
337
345
 
@@ -345,6 +353,9 @@ options:
345
353
  Log level [default: INFO]
346
354
  --debug Enable debug mode [default: False]
347
355
  --mock Run mock versions of the tools that don't need a cluster
356
+ --state-file FILE Serve a captured cluster snapshot from FILE (implies --mock)
357
+ --state-time {advancing,frozen}
358
+ Replay clock for a captured snapshot [default: advancing]
348
359
  --no-redact Disable secret redaction of tool output (on by default)
349
360
  ```
350
361
 
@@ -424,13 +435,195 @@ data: {"jsonrpc":"2.0","id":1,"result":{"tools":[.... long text elided ...]}}
424
435
 
425
436
  ## Mock tools
426
437
  When building agents, it can be helpful to test them against *mock* versions that do
427
- not go against a real cluster, but return static (but realistic) values. The module
438
+ not go against a real cluster, but return realistic values. The module
428
439
  `k8stools.mock_tools` does just that. The data values were captured when running
429
440
  against a real Minikube instance running the
430
441
  [Open Telemetry Demo](https://github.com/open-telemetry/opentelemetry-demo)
431
442
  application. When running the MCP server, this may be enabled by using the
432
443
  `--mock` command line option.
433
444
 
445
+ The mock serves a *capture* — a JSON snapshot of one cluster — rather than a set of
446
+ per-tool canned answers, so the tools agree with each other: a pod that is not in the
447
+ capture does not exist for any tool, and one that is has consistent statuses, events,
448
+ spec and logs. You can snapshot your own cluster the same way — see
449
+ [Capturing and replaying a cluster](#capturing-and-replaying-a-cluster).
450
+
451
+ ## Capturing and replaying a cluster
452
+
453
+ `k8s-capture-state` snapshots a live cluster into a single JSON file, and the MCP
454
+ server replays that file as if it were the cluster. A whole scenario — a crash
455
+ loop, a full volume, a bad rollout — becomes a fixture you can re-run, commit, or
456
+ attach to an issue, with no cluster needed to reproduce it.
457
+
458
+ ```bash
459
+ # Snapshot the current cluster (respects KUBECONFIG)
460
+ k8s-capture-state -o incident-1234.json
461
+
462
+ # Replay it
463
+ k8s-mcp-server --state-file incident-1234.json
464
+ ```
465
+
466
+ ### `k8s-capture-state` options
467
+
468
+ ```
469
+ usage: k8s-capture-state [-h] [--namespace NS [NS ...]] [-o FILE] [--no-logs]
470
+ [--max-log-lines MAX_LOG_LINES] [--no-previous-logs]
471
+ [--no-redact]
472
+ [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
473
+
474
+ options:
475
+ -h, --help show this help message and exit
476
+ --namespace NS [NS ...]
477
+ Namespace(s) to capture namespaced resources from
478
+ [default: all]. Namespaces and nodes are always
479
+ captured in full.
480
+ -o FILE, --output FILE
481
+ Output file [default: k8s-state-<timestamp>.json]
482
+ --no-logs Skip container logs
483
+ --max-log-lines MAX_LOG_LINES
484
+ Log lines to capture per container [default: 1000]
485
+ --no-previous-logs Skip the previous-instance logs of restarted containers
486
+ --no-redact Disable secret redaction of captured values (redaction
487
+ is on by default; can also be disabled with
488
+ K8STOOLS_REDACT=0)
489
+ --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
490
+ Log level [default: INFO]
491
+ ```
492
+
493
+ It finishes with a summary of what it got:
494
+
495
+ ```
496
+ $ k8s-capture-state --namespace default -o incident-1234.json
497
+ Captured 27 pod(s), 31 container(s).
498
+ logs captured: 31
499
+ previous logs captured: 31
500
+ values redacted: 4
501
+ Wrote incident-1234.json (1,435,335 bytes, redacted).
502
+ ```
503
+
504
+ ### What gets captured
505
+
506
+ Everything the tools can read, so that every tool answers on replay:
507
+
508
+ | | |
509
+ |---|---|
510
+ | Cluster-wide | namespaces, nodes |
511
+ | Per namespace | deployments, replica sets, services, statefulsets, cronjobs, jobs, PVCs, events |
512
+ | ConfigMaps | summary **and** full contents, so `get_configmap` works too |
513
+ | Per pod | summary, labels, container statuses, spec, and per-container logs |
514
+
515
+ `--namespace` restricts only the namespaced resources; namespaces and nodes are
516
+ always captured in full so the cluster still makes sense.
517
+
518
+ Logs dominate the file size — expect a few MB for a few dozen pods (the 27-pod
519
+ capture above was 1.4 MB with only 50 lines per container; a 38-pod one at 200
520
+ lines was 3.3 MB). Use `--max-log-lines` to trade size against history, or
521
+ `--no-logs` for a structure-only snapshot. Captures compress about 10x, and can be
522
+ written and read gzipped — see [Compression, and what to check
523
+ in](#compression-and-what-to-check-in).
524
+
525
+ ### Compression, and what to check in
526
+
527
+ A name ending in `.gz` is written gzipped, and captures are read back compressed
528
+ or not either way — detection is by content, so a capture still loads after being
529
+ renamed:
530
+
531
+ ```bash
532
+ k8s-capture-state -o incident-1234.json.gz # 3.3 MB -> 364 KB
533
+ k8s-mcp-server --state-file incident-1234.json.gz
534
+ ```
535
+
536
+ **For a fixture you check into git, use plain `.json` anyway.** Git already stores
537
+ blobs compressed, so you do not pay the uncompressed size — and it *deltas*
538
+ successive versions of a text file against each other, which it cannot do for a
539
+ gzip blob. Measured on two captures of the same 38-pod cluster:
540
+
541
+ | | first commit | updating the fixture later |
542
+ |---|---|---|
543
+ | `.json` (3.3 MB) | 404 KB | **+61 KB** |
544
+ | `.json.gz` (364 KB) | 388 KB | **+358 KB** |
545
+
546
+ The two cost about the same once, but every re-capture of a `.gz` adds a whole
547
+ fresh blob, so the repo grows roughly 6x faster. Captures are also indented rather
548
+ than compact for the same reason — it gives git something to delta. This is also
549
+ why you probably do *not* need git-lfs here: LFS stores every version whole, which
550
+ gives up exactly the delta compression that makes these files cheap.
551
+
552
+ Reach for `.gz` when the file travels on its own — attached to an issue, dropped
553
+ in object storage, or simply too large to keep expanded in a working tree.
554
+
555
+ ### Previous-instance logs
556
+
557
+ The capture includes the **previous-instance logs** of every restarted container —
558
+ what `get_logs_for_pod_and_container(..., previous=True)` returns. For a
559
+ crash-looping container the current instance's logs are usually empty or
560
+ post-restart, and the stack trace or OOM message that explains the crash lives in
561
+ the previous instance, so a capture without them would be missing the evidence
562
+ that matters most.
563
+
564
+ They roughly double the log payload for a cluster where a lot is restarting —
565
+ which is exactly the cluster worth capturing. `--no-previous-logs` skips them. The
566
+ summary reports how many could not be read (the previous instance may have been
567
+ garbage-collected), because a crash-loop capture that lost them looks identical to
568
+ one that never had any.
569
+
570
+ ### Replay clock
571
+
572
+ Every age in a capture is stored relative to the moment of capture, so the
573
+ *intervals* between resources survive replay exactly: a deployment aged 8 days
574
+ whose newest replica set is aged 7h34m still says "upgraded 7h34m ago" whenever it
575
+ is replayed.
576
+
577
+ By default those ages advance in real time from when the server starts, which is
578
+ what makes an interactive session feel like a live cluster. For an automated test
579
+ suite pass `--state-time frozen`:
580
+
581
+ ```bash
582
+ k8s-mcp-server --state-file incident-1234.json --state-time frozen
583
+ ```
584
+
585
+ which pins every age at its captured value, so repeated runs are identical and a
586
+ long session cannot see ages drift between its first tool call and its last.
587
+ (`--state-time` requires `--state-file`; it is rejected rather than ignored, since
588
+ a suite that believed it had frozen the clock and had not would be flaky with no
589
+ visible cause.)
590
+
591
+ ### Replaying from Python
592
+
593
+ For a test suite, skip the server and load the capture directly. `MockState` has
594
+ the same query methods as the tools:
595
+
596
+ ```python
597
+ from pathlib import Path
598
+ from k8stools.mock_state import MockState
599
+
600
+ state = MockState.from_file(Path("incident-1234.json"), frozen=True)
601
+
602
+ pods = state.get_pod_summaries("default")
603
+ logs = state.get_logs_for_pod_and_container("ad-647b4947cc-s5mpm", "default",
604
+ previous=True)
605
+ ```
606
+
607
+ Or point the `mock_tools` module at it, so anything already calling
608
+ `k8stools.mock_tools` serves your capture instead of the built-in one:
609
+
610
+ ```python
611
+ from k8stools import mock_tools
612
+
613
+ mock_tools.load_mock_state(Path("incident-1234.json"), frozen=True)
614
+ agent = Agent(model="openai:gpt-4.1", tools=mock_tools.TOOLS)
615
+ ```
616
+
617
+ ### Captures and secrets
618
+
619
+ `k8s-capture-state` applies the same redaction pass as the MCP server, **on by
620
+ default**, opt out with `--no-redact` or `K8STOOLS_REDACT=0`. This matters because
621
+ a capture file is far more portable than cluster access — it gets committed to
622
+ git, attached to issues, and passed around long after anyone remembers which
623
+ cluster it came from. Redaction is one-way: replaying a redacted capture with
624
+ `--no-redact` restores nothing, so a scenario that genuinely needs real values has
625
+ to be re-captured.
626
+
434
627
  ## Secret redaction
435
628
  Some read-only resources can carry secret-shaped values even though they are not
436
629
  Kubernetes `Secret` objects — `ConfigMap` data and the `env` blocks in a pod spec
@@ -440,11 +633,26 @@ straight into the model's context.
440
633
 
441
634
  To prevent that, the MCP server applies a redaction pass to every tool's output.
442
635
  It is **on by default** and can be disabled with `--no-redact` or by setting
443
- `K8STOOLS_REDACT=0`. Redaction matches both by value shape (AWS access keys, JWT /
444
- bearer tokens, PEM private-key blocks) and by key / env-var name
445
- (`key|secret|token|password|credential`), replacing each match with a visible
446
- `[REDACTED]` marker so the agent can tell "hidden" from "absent". We never provide
447
- a reader for Kubernetes `Secret` objects.
636
+ `K8STOOLS_REDACT=0`. Redaction matches two ways, replacing each match with a
637
+ visible `[REDACTED]` marker so the agent can tell "hidden" from "absent". We never
638
+ provide a reader for Kubernetes `Secret` objects.
639
+
640
+ 1. **By value shape** — AWS access keys, JWT / bearer tokens, PEM private-key blocks.
641
+ 2. **By key / env-var name** — the name contains `key`, `secret`, `token`,
642
+ `password`, `passwd`, `credential` or `cred` **as a whole word**. Names are split
643
+ on separators and camelCase, so `AWS_SECRET_ACCESS_KEY`, `apiKey` and
644
+ `x-auth-token` all match while `VALKEY_ADDR` does not.
645
+
646
+ A field named *exactly* `key` is exempt: in the Kubernetes API that is always a
647
+ map entry, a taint, a label selector or a projected-volume item, never a
648
+ credential. This matters more than it sounds — on a real 38-pod cluster it was
649
+ 130 of 139 redactions, none of them secrets: toleration keys such as
650
+ `node.kubernetes.io/not-ready`, the filename `ca.crt`, and `secretKeyRef.key`.
651
+ That last is worth stating plainly: **a reference to a secret is not a secret.**
652
+ The value lives in the `Secret` object and is resolved by the kubelet, never
653
+ appearing in tool output, so redacting the pointer hides which key feeds an env
654
+ var and protects nothing. The exemption does not apply to env-var names, where
655
+ a variable someone named `KEY` plausibly does hold one.
448
656
 
449
657
  If you call the tool functions directly in Python (rather than through the MCP
450
658
  server), you get raw, un-redacted values; you can apply the same pass yourself via
@@ -8,6 +8,8 @@ directly to an agent as tools or placed behind an MCP server (included). Some us
8
8
  * Build [agents](https://github.com/BenedatLLC/orca-agent/) to monitor your cluster or perform root cause analysis.
9
9
  * Vibe-code a custom chat UI.
10
10
  * Use in non-agentic automations.
11
+ * Snapshot a live cluster to a file and replay it as a test fixture, with no cluster
12
+ needed — see [Capturing and replaying a cluster](#capturing-and-replaying-a-cluster).
11
13
 
12
14
  <a href="https://glama.ai/mcp/servers/@BenedatLLC/k8stools">
13
15
  <img width="380" height="200" src="https://glama.ai/mcp/servers/@BenedatLLC/k8stools/badge" alt="Kubernetes Tools Server MCP server" />
@@ -48,6 +50,10 @@ Via `uv`:
48
50
  uv add k8stools
49
51
  ```
50
52
 
53
+ This installs three commands: `k8s-mcp-server` (the MCP server), `k8s-mcp-client`
54
+ (a client for manual testing), and `k8s-capture-state` (snapshot a cluster to a
55
+ replayable file).
56
+
51
57
  ## Current tools
52
58
 
53
59
  These are the tools we define:
@@ -114,7 +120,7 @@ Here are the command line arguments for the server:
114
120
  ```
115
121
  usage: k8s-mcp-server [-h] [--transport {streamable-http,stdio}] [--host HOST] [--port PORT]
116
122
  [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--debug] [--mock]
117
- [--no-redact]
123
+ [--state-file FILE] [--state-time {advancing,frozen}] [--no-redact]
118
124
 
119
125
  Run the MCP server.
120
126
 
@@ -128,6 +134,9 @@ options:
128
134
  Log level [default: INFO]
129
135
  --debug Enable debug mode [default: False]
130
136
  --mock Run mock versions of the tools that don't need a cluster
137
+ --state-file FILE Serve a captured cluster snapshot from FILE (implies --mock)
138
+ --state-time {advancing,frozen}
139
+ Replay clock for a captured snapshot [default: advancing]
131
140
  --no-redact Disable secret redaction of tool output (on by default)
132
141
  ```
133
142
 
@@ -207,13 +216,195 @@ data: {"jsonrpc":"2.0","id":1,"result":{"tools":[.... long text elided ...]}}
207
216
 
208
217
  ## Mock tools
209
218
  When building agents, it can be helpful to test them against *mock* versions that do
210
- not go against a real cluster, but return static (but realistic) values. The module
219
+ not go against a real cluster, but return realistic values. The module
211
220
  `k8stools.mock_tools` does just that. The data values were captured when running
212
221
  against a real Minikube instance running the
213
222
  [Open Telemetry Demo](https://github.com/open-telemetry/opentelemetry-demo)
214
223
  application. When running the MCP server, this may be enabled by using the
215
224
  `--mock` command line option.
216
225
 
226
+ The mock serves a *capture* — a JSON snapshot of one cluster — rather than a set of
227
+ per-tool canned answers, so the tools agree with each other: a pod that is not in the
228
+ capture does not exist for any tool, and one that is has consistent statuses, events,
229
+ spec and logs. You can snapshot your own cluster the same way — see
230
+ [Capturing and replaying a cluster](#capturing-and-replaying-a-cluster).
231
+
232
+ ## Capturing and replaying a cluster
233
+
234
+ `k8s-capture-state` snapshots a live cluster into a single JSON file, and the MCP
235
+ server replays that file as if it were the cluster. A whole scenario — a crash
236
+ loop, a full volume, a bad rollout — becomes a fixture you can re-run, commit, or
237
+ attach to an issue, with no cluster needed to reproduce it.
238
+
239
+ ```bash
240
+ # Snapshot the current cluster (respects KUBECONFIG)
241
+ k8s-capture-state -o incident-1234.json
242
+
243
+ # Replay it
244
+ k8s-mcp-server --state-file incident-1234.json
245
+ ```
246
+
247
+ ### `k8s-capture-state` options
248
+
249
+ ```
250
+ usage: k8s-capture-state [-h] [--namespace NS [NS ...]] [-o FILE] [--no-logs]
251
+ [--max-log-lines MAX_LOG_LINES] [--no-previous-logs]
252
+ [--no-redact]
253
+ [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
254
+
255
+ options:
256
+ -h, --help show this help message and exit
257
+ --namespace NS [NS ...]
258
+ Namespace(s) to capture namespaced resources from
259
+ [default: all]. Namespaces and nodes are always
260
+ captured in full.
261
+ -o FILE, --output FILE
262
+ Output file [default: k8s-state-<timestamp>.json]
263
+ --no-logs Skip container logs
264
+ --max-log-lines MAX_LOG_LINES
265
+ Log lines to capture per container [default: 1000]
266
+ --no-previous-logs Skip the previous-instance logs of restarted containers
267
+ --no-redact Disable secret redaction of captured values (redaction
268
+ is on by default; can also be disabled with
269
+ K8STOOLS_REDACT=0)
270
+ --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
271
+ Log level [default: INFO]
272
+ ```
273
+
274
+ It finishes with a summary of what it got:
275
+
276
+ ```
277
+ $ k8s-capture-state --namespace default -o incident-1234.json
278
+ Captured 27 pod(s), 31 container(s).
279
+ logs captured: 31
280
+ previous logs captured: 31
281
+ values redacted: 4
282
+ Wrote incident-1234.json (1,435,335 bytes, redacted).
283
+ ```
284
+
285
+ ### What gets captured
286
+
287
+ Everything the tools can read, so that every tool answers on replay:
288
+
289
+ | | |
290
+ |---|---|
291
+ | Cluster-wide | namespaces, nodes |
292
+ | Per namespace | deployments, replica sets, services, statefulsets, cronjobs, jobs, PVCs, events |
293
+ | ConfigMaps | summary **and** full contents, so `get_configmap` works too |
294
+ | Per pod | summary, labels, container statuses, spec, and per-container logs |
295
+
296
+ `--namespace` restricts only the namespaced resources; namespaces and nodes are
297
+ always captured in full so the cluster still makes sense.
298
+
299
+ Logs dominate the file size — expect a few MB for a few dozen pods (the 27-pod
300
+ capture above was 1.4 MB with only 50 lines per container; a 38-pod one at 200
301
+ lines was 3.3 MB). Use `--max-log-lines` to trade size against history, or
302
+ `--no-logs` for a structure-only snapshot. Captures compress about 10x, and can be
303
+ written and read gzipped — see [Compression, and what to check
304
+ in](#compression-and-what-to-check-in).
305
+
306
+ ### Compression, and what to check in
307
+
308
+ A name ending in `.gz` is written gzipped, and captures are read back compressed
309
+ or not either way — detection is by content, so a capture still loads after being
310
+ renamed:
311
+
312
+ ```bash
313
+ k8s-capture-state -o incident-1234.json.gz # 3.3 MB -> 364 KB
314
+ k8s-mcp-server --state-file incident-1234.json.gz
315
+ ```
316
+
317
+ **For a fixture you check into git, use plain `.json` anyway.** Git already stores
318
+ blobs compressed, so you do not pay the uncompressed size — and it *deltas*
319
+ successive versions of a text file against each other, which it cannot do for a
320
+ gzip blob. Measured on two captures of the same 38-pod cluster:
321
+
322
+ | | first commit | updating the fixture later |
323
+ |---|---|---|
324
+ | `.json` (3.3 MB) | 404 KB | **+61 KB** |
325
+ | `.json.gz` (364 KB) | 388 KB | **+358 KB** |
326
+
327
+ The two cost about the same once, but every re-capture of a `.gz` adds a whole
328
+ fresh blob, so the repo grows roughly 6x faster. Captures are also indented rather
329
+ than compact for the same reason — it gives git something to delta. This is also
330
+ why you probably do *not* need git-lfs here: LFS stores every version whole, which
331
+ gives up exactly the delta compression that makes these files cheap.
332
+
333
+ Reach for `.gz` when the file travels on its own — attached to an issue, dropped
334
+ in object storage, or simply too large to keep expanded in a working tree.
335
+
336
+ ### Previous-instance logs
337
+
338
+ The capture includes the **previous-instance logs** of every restarted container —
339
+ what `get_logs_for_pod_and_container(..., previous=True)` returns. For a
340
+ crash-looping container the current instance's logs are usually empty or
341
+ post-restart, and the stack trace or OOM message that explains the crash lives in
342
+ the previous instance, so a capture without them would be missing the evidence
343
+ that matters most.
344
+
345
+ They roughly double the log payload for a cluster where a lot is restarting —
346
+ which is exactly the cluster worth capturing. `--no-previous-logs` skips them. The
347
+ summary reports how many could not be read (the previous instance may have been
348
+ garbage-collected), because a crash-loop capture that lost them looks identical to
349
+ one that never had any.
350
+
351
+ ### Replay clock
352
+
353
+ Every age in a capture is stored relative to the moment of capture, so the
354
+ *intervals* between resources survive replay exactly: a deployment aged 8 days
355
+ whose newest replica set is aged 7h34m still says "upgraded 7h34m ago" whenever it
356
+ is replayed.
357
+
358
+ By default those ages advance in real time from when the server starts, which is
359
+ what makes an interactive session feel like a live cluster. For an automated test
360
+ suite pass `--state-time frozen`:
361
+
362
+ ```bash
363
+ k8s-mcp-server --state-file incident-1234.json --state-time frozen
364
+ ```
365
+
366
+ which pins every age at its captured value, so repeated runs are identical and a
367
+ long session cannot see ages drift between its first tool call and its last.
368
+ (`--state-time` requires `--state-file`; it is rejected rather than ignored, since
369
+ a suite that believed it had frozen the clock and had not would be flaky with no
370
+ visible cause.)
371
+
372
+ ### Replaying from Python
373
+
374
+ For a test suite, skip the server and load the capture directly. `MockState` has
375
+ the same query methods as the tools:
376
+
377
+ ```python
378
+ from pathlib import Path
379
+ from k8stools.mock_state import MockState
380
+
381
+ state = MockState.from_file(Path("incident-1234.json"), frozen=True)
382
+
383
+ pods = state.get_pod_summaries("default")
384
+ logs = state.get_logs_for_pod_and_container("ad-647b4947cc-s5mpm", "default",
385
+ previous=True)
386
+ ```
387
+
388
+ Or point the `mock_tools` module at it, so anything already calling
389
+ `k8stools.mock_tools` serves your capture instead of the built-in one:
390
+
391
+ ```python
392
+ from k8stools import mock_tools
393
+
394
+ mock_tools.load_mock_state(Path("incident-1234.json"), frozen=True)
395
+ agent = Agent(model="openai:gpt-4.1", tools=mock_tools.TOOLS)
396
+ ```
397
+
398
+ ### Captures and secrets
399
+
400
+ `k8s-capture-state` applies the same redaction pass as the MCP server, **on by
401
+ default**, opt out with `--no-redact` or `K8STOOLS_REDACT=0`. This matters because
402
+ a capture file is far more portable than cluster access — it gets committed to
403
+ git, attached to issues, and passed around long after anyone remembers which
404
+ cluster it came from. Redaction is one-way: replaying a redacted capture with
405
+ `--no-redact` restores nothing, so a scenario that genuinely needs real values has
406
+ to be re-captured.
407
+
217
408
  ## Secret redaction
218
409
  Some read-only resources can carry secret-shaped values even though they are not
219
410
  Kubernetes `Secret` objects — `ConfigMap` data and the `env` blocks in a pod spec
@@ -223,11 +414,26 @@ straight into the model's context.
223
414
 
224
415
  To prevent that, the MCP server applies a redaction pass to every tool's output.
225
416
  It is **on by default** and can be disabled with `--no-redact` or by setting
226
- `K8STOOLS_REDACT=0`. Redaction matches both by value shape (AWS access keys, JWT /
227
- bearer tokens, PEM private-key blocks) and by key / env-var name
228
- (`key|secret|token|password|credential`), replacing each match with a visible
229
- `[REDACTED]` marker so the agent can tell "hidden" from "absent". We never provide
230
- a reader for Kubernetes `Secret` objects.
417
+ `K8STOOLS_REDACT=0`. Redaction matches two ways, replacing each match with a
418
+ visible `[REDACTED]` marker so the agent can tell "hidden" from "absent". We never
419
+ provide a reader for Kubernetes `Secret` objects.
420
+
421
+ 1. **By value shape** — AWS access keys, JWT / bearer tokens, PEM private-key blocks.
422
+ 2. **By key / env-var name** — the name contains `key`, `secret`, `token`,
423
+ `password`, `passwd`, `credential` or `cred` **as a whole word**. Names are split
424
+ on separators and camelCase, so `AWS_SECRET_ACCESS_KEY`, `apiKey` and
425
+ `x-auth-token` all match while `VALKEY_ADDR` does not.
426
+
427
+ A field named *exactly* `key` is exempt: in the Kubernetes API that is always a
428
+ map entry, a taint, a label selector or a projected-volume item, never a
429
+ credential. This matters more than it sounds — on a real 38-pod cluster it was
430
+ 130 of 139 redactions, none of them secrets: toleration keys such as
431
+ `node.kubernetes.io/not-ready`, the filename `ca.crt`, and `secretKeyRef.key`.
432
+ That last is worth stating plainly: **a reference to a secret is not a secret.**
433
+ The value lives in the `Secret` object and is resolved by the kubelet, never
434
+ appearing in tool output, so redacting the pointer hides which key feeds an env
435
+ var and protects nothing. The exemption does not apply to env-var names, where
436
+ a variable someone named `KEY` plausibly does hold one.
231
437
 
232
438
  If you call the tool functions directly in Python (rather than through the MCP
233
439
  server), you get raw, un-redacted values; you can apply the same pass yourself via