@mokuyoaxis/agent-guard 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/CONTRIBUTING.md +101 -0
  2. package/LICENSE +21 -0
  3. package/README.md +558 -0
  4. package/README.zh-CN.md +488 -0
  5. package/adapters/INTEGRATION.md +94 -0
  6. package/adapters/claude/README.md +141 -0
  7. package/adapters/claude/compatibility.json +17 -0
  8. package/adapters/claude/harness/README.md +59 -0
  9. package/adapters/claude/harness/mock_anthropic_api.mjs +47 -0
  10. package/adapters/claude/harness/run_scenario.sh +79 -0
  11. package/adapters/claude/harness/run_scenario_no_python3.sh +49 -0
  12. package/adapters/claude/hook_bridge.sh +33 -0
  13. package/adapters/claude/pre_tool_use.py +264 -0
  14. package/adapters/claude/settings.example.json +16 -0
  15. package/adapters/codex/harness/README.md +53 -0
  16. package/adapters/codex/harness/acceptance.py +226 -0
  17. package/adapters/dsh/README.md +62 -0
  18. package/adapters/dsh/cordis.patch.yml +30 -0
  19. package/adapters/dsh/lib/index.js +500 -0
  20. package/adapters/kimi-code/README.md +169 -0
  21. package/adapters/kimi-code/compatibility.json +17 -0
  22. package/adapters/kimi-code/config.example.toml +26 -0
  23. package/adapters/kimi-code/hook_bridge.sh +32 -0
  24. package/adapters/kimi-code/pre_tool_use.py +72 -0
  25. package/core/__init__.py +47 -0
  26. package/core/audit.py +86 -0
  27. package/core/classifier.py +1175 -0
  28. package/core/dialects.py +898 -0
  29. package/core/policy.py +817 -0
  30. package/core/recovery.py +700 -0
  31. package/core/redaction.py +1096 -0
  32. package/core/safe_view.py +212 -0
  33. package/docs/architecture.md +223 -0
  34. package/docs/compatibility.md +142 -0
  35. package/docs/development-note-unguarded-deletion.md +187 -0
  36. package/docs/friction.md +266 -0
  37. package/docs/harness-capabilities.md +72 -0
  38. package/docs/host-drift.md +142 -0
  39. package/docs/publishing-from-ephemeral-environments.md +104 -0
  40. package/docs/release-notes-0.2.0-rc1.md +46 -0
  41. package/docs/release-notes-0.2.0.md +72 -0
  42. package/docs/release-notes-0.2.1.md +78 -0
  43. package/docs/release-notes-0.2.2.md +120 -0
  44. package/docs/secret-guard-analysis.md +416 -0
  45. package/docs/secret-guard-architecture.md +93 -0
  46. package/docs/secret-guard-design.md +649 -0
  47. package/docs/test-report-claude-code-harness.md +462 -0
  48. package/docs/test-report-codex-gpt-5.6-sol.md +335 -0
  49. package/docs/test-report-codex-gpt-6-astra-high.md +129 -0
  50. package/docs/test-report-dsh-0.1.5-rc.1.md +84 -0
  51. package/docs/test-report-dsh-v0.1.1.md +126 -0
  52. package/docs/test-report-kimi-code-block.md +198 -0
  53. package/docs/test-report-zcode-glm-flash.md +241 -0
  54. package/docs/threat-model.md +44 -0
  55. package/doctor.py +756 -0
  56. package/live_sentinel.py +339 -0
  57. package/package.json +69 -0
  58. package/skills/delete-guard/SKILL.md +109 -0
  59. package/skills/delete-guard/agents/openai.yaml +4 -0
  60. package/skills/delete-guard/references/policy.md +161 -0
  61. package/skills/delete-guard/scripts/_bootstrap.py +8 -0
  62. package/skills/delete-guard/scripts/check.py +370 -0
  63. package/skills/delete-guard/scripts/gc.py +103 -0
  64. package/skills/delete-guard/scripts/restore.py +98 -0
  65. package/skills/delete-guard/scripts/safe_delete.py +256 -0
  66. package/skills/delete-guard/scripts/status.py +130 -0
  67. package/skills/exfil-guard/SKILL.md +143 -0
  68. package/skills/exfil-guard/agents/openai.yaml +4 -0
  69. package/skills/exfil-guard/references/channels.md +58 -0
  70. package/skills/exfil-guard/references/rules.md +194 -0
  71. package/skills/exfil-guard/scripts/_bootstrap.py +8 -0
  72. package/skills/exfil-guard/scripts/check_span.py +250 -0
  73. package/skills/exfil-guard/scripts/sanitize.py +155 -0
  74. package/skills/exfil-guard/scripts/view.py +44 -0
  75. package/skills/recovery-audit/SKILL.md +283 -0
  76. package/skills/recovery-audit/agents/openai.yaml +4 -0
  77. package/skills/recovery-audit/references/evidence-records.md +85 -0
@@ -0,0 +1,187 @@
1
+ # Development Note: Learning from an Unguarded Deletion Incident
2
+
3
+ - **Incident window:** September 2026
4
+ - **Scope:** destructive-command interception, recovery evidence, and adapter coverage
5
+ - **Document status:** public, de-identified engineering note
6
+
7
+ > This is not the primary incident record. The byte-exact report and its local
8
+ > evidence index are retained outside the published source tree. This note
9
+ > removes host identity, session identifiers, unrelated project names, private
10
+ > filesystem details, and executable destructive commands. Its purpose is to
11
+ > preserve the engineering lessons without publishing operator-specific data.
12
+
13
+ ## 1. Why this note exists
14
+
15
+ An agent session issued a recursively destructive command whose path looked
16
+ local but normalized to a much broader ancestor directory. The active harness
17
+ did not have an agent-guard adapter, so the command never reached the guard.
18
+ The host was also running in an automatic approval mode. The operation began
19
+ removing user files and stopped only after external interruption.
20
+
21
+ The event exposed a useful distinction:
22
+
23
+ - a policy can be correct and still provide no protection when the harness has
24
+ no interception path;
25
+ - an adapter can be present and still be unsafe if path normalization happens
26
+ after an early shape decision;
27
+ - a passing test suite does not prove that every supported harness actually
28
+ invokes the guard;
29
+ - recovery claims need evidence provenance, not only a rebuilt tree that looks
30
+ plausible.
31
+
32
+ This note turns those observations into product requirements.
33
+
34
+ ## 2. The failure class
35
+
36
+ The command combined two properties:
37
+
38
+ 1. it changed the working directory before a destructive operation; and
39
+ 2. its target contained parent traversal that resolved outside the intended
40
+ workspace.
41
+
42
+ In abstract form:
43
+
44
+ ```text
45
+ change-directory <temporary-location>
46
+ then recursively-delete <temporary-location>/../<protected-ancestor>
47
+ ```
48
+
49
+ The important fact is the normalized target, not the spelling supplied to the
50
+ shell. No destructive reproduction is included here. A safe diagnostic should
51
+ resolve and print the target only; it must never execute the deletion.
52
+
53
+ ## 3. What failed
54
+
55
+ ### 3.1 No adapter meant no enforcement
56
+
57
+ agent-guard has two different integration levels:
58
+
59
+ - the Skill and CLI path, where an agent deliberately calls the guard; and
60
+ - a harness adapter, which intercepts destructive-looking commands before the
61
+ shell executes them.
62
+
63
+ The incident happened on a harness with neither a native adapter nor an
64
+ equivalent pre-execution hook. Prompt guidance alone could not enforce the
65
+ policy. This is an integration gap, not a classifier false negative.
66
+
67
+ ### 3.2 Automatic approval removed the human boundary
68
+
69
+ The session could execute shell operations without a contemporaneous human
70
+ decision. In such a host, `ASK` cannot honestly mean “ask once.” It must degrade
71
+ to `BLOCK`, or the host must provide a real approval callback.
72
+
73
+ ### 3.3 Shape classification could precede hard boundaries
74
+
75
+ The policy recognized “change directory, then delete” as an ambiguous compound
76
+ shape. That shape historically returned `ASK` before the resolved target was
77
+ checked against the workspace boundary.
78
+
79
+ Hard boundaries need priority over interaction shape:
80
+
81
+ ```text
82
+ resolve target
83
+ -> protected root or outside workspace: BLOCK
84
+ -> undeterminable effect: BLOCK
85
+ -> otherwise evaluate compound shape: ASK or compensate
86
+ ```
87
+
88
+ An operation does not become merely ambiguous because it is written as a
89
+ compound command.
90
+
91
+ ## 4. What recovery taught us
92
+
93
+ The most valuable surviving source was not a backup of the deleted projects.
94
+ It was the collection of agent-session records that preserved successful patch
95
+ payloads and command results. That evidence was sufficient to reconstruct some
96
+ work, but not enough to call every result an exact recovery.
97
+
98
+ Three labels are necessary:
99
+
100
+ - **recovered** — exact bytes or an exact patch are evidenced;
101
+ - **reconstructed** — behavior was rebuilt from plans, tests, or prose;
102
+ - **missing** — prior existence is evidenced, but the content is unavailable.
103
+
104
+ Collapsing these labels makes a green build look stronger than the evidence
105
+ supports. A trustworthy recovery therefore needs an evidence ledger, replay
106
+ reports, deterministic reruns, and explicit gap markers.
107
+
108
+ These requirements became the [`recovery-audit`](../skills/recovery-audit/SKILL.md)
109
+ Skill. It complements `delete-guard` and `exfil-guard`:
110
+
111
+ ```text
112
+ before destruction -> delete-guard
113
+ before disclosure -> exfil-guard
114
+ after an incident -> recovery-audit
115
+ ```
116
+
117
+ ## 5. Product changes suggested by the incident
118
+
119
+ ### Immediate
120
+
121
+ 1. Treat harness coverage as a first-class compatibility claim. “The CLI
122
+ works” and “the harness intercepts every destructive shell call” are
123
+ different statements.
124
+ 2. Evaluate protected roots, workspace escape, and undeterminable targets
125
+ before compound-command interaction rules.
126
+ 3. Make `ASK` fail closed when the host cannot prove that a human approval
127
+ channel exists.
128
+ 4. Keep original recovery evidence outside publishable package paths.
129
+ 5. Scan package contents—not only Git-tracked files—for private paths,
130
+ temporary artifacts, and internal reports.
131
+
132
+ ### Next experiments
133
+
134
+ - a harness capability handshake that reports interception, approval, sandbox,
135
+ and shell-dialect support;
136
+ - an adapter conformance probe that deliberately submits safe, non-executed
137
+ destructive samples and verifies they reach the shared policy core;
138
+ - protected ancestor roots independent of workspace configuration;
139
+ - first-class detection of parent traversal in destructive targets;
140
+ - a local incident bundle containing hashes, verdicts, and redacted metadata,
141
+ but never credential material;
142
+ - deterministic recovery replay with machine-readable coverage reports.
143
+
144
+ ## 6. What this incident does not prove
145
+
146
+ - agent-guard is not an operating-system sandbox and cannot intercept a host
147
+ that never calls it;
148
+ - a local reconstruction is not automatically an exact recovery;
149
+ - a mocked adapter test is not equivalent to a live harness run;
150
+ - one interrupted deletion does not establish how or why the process stopped;
151
+ - the event does not justify publishing host-specific logs or attributing
152
+ intent to a model or harness.
153
+
154
+ The project should continue to describe itself as reliability infrastructure
155
+ with human escalation, not as a boundary against a malicious or fully
156
+ privileged process. See [`docs/threat-model.md`](threat-model.md).
157
+
158
+ ## 7. Public evidence policy
159
+
160
+ Public reports may include:
161
+
162
+ - the failure class and normalized policy facts;
163
+ - de-identified timelines at a useful engineering granularity;
164
+ - minimal non-destructive probes;
165
+ - code-level findings, tests, and remediation status;
166
+ - clearly stated uncertainty.
167
+
168
+ They should exclude:
169
+
170
+ - usernames, home directories, session identifiers, and local project names;
171
+ - inventories of authentication files or local configuration;
172
+ - absolute paths into caches, logs, or recovery archives;
173
+ - copy-pasteable destructive reproductions;
174
+ - claims that cannot be separated from private evidence.
175
+
176
+ The private incident record remains the source of truth for the event. This
177
+ document is a reconstructed public narrative derived from that record and is
178
+ not byte-equivalent to it.
179
+
180
+ ## 8. Direction
181
+
182
+ The incident changes the roadmap in a small but important way: adapter coverage
183
+ and recovery provenance are now product features, not only operational notes.
184
+ The near-term goal is not to add more command patterns. It is to prove that the
185
+ same policy reaches every advertised harness, fails closed when human approval
186
+ is unavailable, and leaves enough evidence to recover honestly when prevention
187
+ fails.
@@ -0,0 +1,266 @@
1
+ # Friction log — real-agent validation (DSH adapter, first live session)
2
+
3
+ The adapter plugin (`tools/pre-execute` interception + three model tools +
4
+ prompt section) was pointed at a live coding agent performing real
5
+ destructive operations. Findings, in the order they hurt.
6
+
7
+ ## F1 · Compound `cd X && rm y` resolves targets against the wrong base
8
+
9
+ The interceptor classifies against the tool call's declared workdir. A
10
+ command like `cd subdir && rm -rf build` runs `rm` inside `subdir`, but the
11
+ guard resolved `build` against the parent — wrong tree entirely.
12
+
13
+ **Shipped, then superseded by the Decision Protocol:** shape rule F1 now
14
+ yields decision=ASK (COMPOUND_CWD_DELETE) - single-execution authorization
15
+ instead of a dead end. Verified live: the ask surfaced to the human, who
16
+ declined; splitting the command avoids the prompt entirely.
17
+
18
+ **Corrected by the 2026-09-20 incident (P1).** That live validation assumed
19
+ an interactive human on the other end. Under an auto-approving host the ASK
20
+ is granted without anyone seeing it, so F1 converted a guaranteed refusal
21
+ into a silent execution - in the incident, `cd /tmp && rm -rf "$PWD/../home"`
22
+ resolved to `/home`. A shape rule describes *compensation* difficulty, never
23
+ effect scope: F1 now evaluates the target set first and lets any hard
24
+ boundary (out-of-workspace, protected path, undeterminable effect) win. The
25
+ legitimate single-execution ASK survives for in-workspace, resolvable
26
+ targets. Regression: `tests/test_incident_regression.py`.
27
+
28
+ ## F2 · Create-then-delete in one line is a timing blind spot
29
+
30
+ Observed live: `touch junk_a.tmp junk_b.tmp && mkdir -p empty_dir && git clean -fd`.
31
+ Pre-execution interception enumerates *before* the command runs — the junk
32
+ files did not exist yet, so compensation could not cover them and git truly
33
+ deleted them.
34
+
35
+ The same event proved the value: an untracked but valuable `keep.txt` lying
36
+ in the repo WAS enumerated and relocated before `git clean -fd` could
37
+ silently destroy it, then restored via `restore.py <txid>`. The guard saved
38
+ exactly the class of file it exists for.
39
+
40
+ **Shipped, then superseded by the Decision Protocol:** shape rule F2 now
41
+ yields decision=ASK (COMPOUND_CREATE_DELETE); position-independent
42
+ compensations (`reset --hard` stash; force-push, blocked anyway) remain
43
+ exempt. Verified live end-to-end, including the user declining the ask.
44
+
45
+ ## F3 · Unmatched globs produced a self-contradicting BLOCK
46
+
47
+ `safe_delete '*.log'` with no *.log files answered BLOCK_WILDCARD — while its
48
+ own docs say safe_delete expands globs itself. **Fixed:** unmatched patterns
49
+ are now reported as `no_match` / ALLOW_NOOP instead of reaching the classifier
50
+ as opaque wildcards.
51
+
52
+ ## F4 · Harness integration notes (DSH)
53
+
54
+ For future adapter authors embedding `check.py` in another harness:
55
+
56
+ 1. Always call `shell.run(shell.resolve(request))` — raw specs crash host
57
+ integration.
58
+ 2. `ShellRunResult.stdout/stderr` are `CollectedOutput` objects; payload is
59
+ `.text`.
60
+ 3. Resolve the caller's sandbox policy from `exec.agent.session` and pass it
61
+ in the request; otherwise you get the deployment default
62
+ (workspace-write), which fails on hosts without a sandbox backend even
63
+ when the calling session runs unconfined.
64
+ 4. Dynamic tool registration must go through `harness.defineTool`;
65
+ parameters are property maps (`required` is a per-property annotation);
66
+ object output schemas must declare `additionalProperties`.
67
+ 5. Per-destructive-command overhead is one python3 startup (~0.3 s); the
68
+ keyword prefilter keeps non-destructive traffic at regex cost only.
69
+
70
+ ## F5 · Heredoc bodies were scanned as command syntax (fixed live)
71
+
72
+ While writing THIS very file through a shell heredoc, the guard denied the
73
+ write twice: the documentation text quotes destructive commands, and the
74
+ classifier treated quoted examples as part of the command line. A file-WRITE
75
+ was blocked over its textual content. Even deploying the fix required a
76
+ maintenance window, because the old classifier denied the patch command too —
77
+ the chicken-and-egg is inherent to self-hosting guards.
78
+
79
+ **Fixed in V1:** `classifier.strip_heredocs()` removes heredoc payloads before
80
+ parsing; a regression test pins the behavior
81
+ (`test_heredoc_body_is_payload_not_syntax`). Redirection-into-file is write
82
+ territory, not delete territory.
83
+
84
+ ## F6 · Heredoc stripping re-matched its own operator and truncated
85
+
86
+ While writing the bilingual README through shell heredocs, the guard denied
87
+ the write a second time - even after F5 was fixed. Post-mortem: F5 kept the
88
+ `<<'TAG'` operator token in the command string, so the stripping loop
89
+ re-matched it on every iteration; and when no terminator line was found,
90
+ the fallback truncated everything after the operator's line - physically
91
+ cutting a trailing `sed -i 's/.../'` command mid-quotation and manufacturing
92
+ the exact unbalanced-quote danger the classifier exists to refuse.
93
+
94
+ **Fixed:** the operator is replaced (never re-matches), the truncating
95
+ fallback is gone (later command lines always survive), and three regression
96
+ tests pin the shapes: operator+trailing-quoted-line, double heredoc,
97
+ unterminated heredoc. This is also the cleanest example of the
98
+ fail-closed trade: the bug was annoying and visible, never dangerous.
99
+
100
+ ## F7 · Heredoc terminator cut double-offset into following commands
101
+
102
+ The F5 fix introduced its own bug: the cut after a found terminator was
103
+ `out[nl + 1 + end_m.end():]` — but `end_m` was already searched from
104
+ `nl + 1`, so its end index is absolute and adding `nl + 1` again pushed
105
+ the cut deep into whatever command line followed the heredoc. The longer
106
+ the body, the deeper the blade went, slicing trailing `sed`/quote-laden
107
+ lines into unbalanced fragments — manufacturing the very parse errors the
108
+ classifier refuses.
109
+
110
+ **Fixed:** cut is now `out[end_m.end():]`. Regression test uses a
111
+ deliberately LONG body (the short-body version passed even while broken -
112
+ regression tests must span the length dimension too).
113
+
114
+ ## F8 · Unresolved workspace root broke boundary checks on macOS
115
+
116
+ CI was green on all five Ubuntu jobs and red on all five macOS jobs. Root
117
+ cause: on macOS `/var` is a symlink to `/private/var`. `tempfile.mkdtemp()`
118
+ returned an unresolved `/var/...` path (test's idea of the workspace) while
119
+ child processes reported their physical cwd `/private/var/...` — so every
120
+ target compared against the wrong root string and came back
121
+ BLOCK_OUT_OF_WORKSPACE.
122
+
123
+ **Fixed:** `discover_workspace()` resolves its result through realpath.
124
+ Deliberate asymmetry preserved: the *boundary root* is physicalized, but
125
+ *target* analysis stays lexical (deleting a symlink still deletes the link,
126
+ never its target).
127
+
128
+ ## F9 · Workdir context is load-bearing
129
+
130
+ Three separate incidents shared one root cause: a command executed under a
131
+ different working directory than assumed.
132
+
133
+ - `safe_delete.py` chained after a `cd` into another directory resolved its
134
+ target against the wrong workspace and self-blocked (KeyError on the
135
+ blocked-result shape).
136
+ - Cloning a helper repository to `/tmp` was refused as
137
+ OUT_OF_WORKSPACE - correctly; the fix was moving the clone inside the
138
+ boundary, not weakening it.
139
+ - An SSH smoke test flapped between failure and success until IPv4 was
140
+ forced; dual-stack hosts answer from different vantage points.
141
+
142
+ Lesson for adapters: always pass an explicit workdir; lesson for agents:
143
+ deletion commands deserve their own process, their own directory, and
144
+ nothing else on the line.
145
+
146
+ ## F10 · Restored transactions still looked live
147
+
148
+ Independent medium- and high-reasoning Codex runs both restored quarantined
149
+ content successfully, then reported the same ambiguity: `restore.py list`
150
+ still showed the transaction with its original item count, without saying
151
+ whether those items remained recoverable or had already returned to origin.
152
+
153
+ **Fixed in v0.1.1:** manifest grouping now exposes explicit `RESTORABLE`,
154
+ `RESTORED`, `FAILED`, and `PURGED` state plus a live-item count. `status.py`
155
+ separates historical transaction count from currently restorable count.
156
+
157
+ ## F11 · Auditing a refusal could dirty a fresh repository
158
+
159
+ An enforced hard block correctly left every target untouched but appended
160
+ `audit.jsonl` before ensuring `.agent-trash/` was ignored. The refusal itself
161
+ therefore created an untracked path. Under read-only Git metadata, even a
162
+ best-effort audit could not repair that status pollution afterward.
163
+
164
+ **Fixed in v0.1.1:** establish an existing `.gitignore` or local exclude rule
165
+ before creating audit storage. If protected Git metadata prevents that, retain
166
+ the safer verdict, return an `audit unavailable` warning, and leave no new
167
+ quarantine directory.
168
+
169
+ ## F12 · exfil-guard: a rule that fires on documentation is a rule that gets disabled
170
+
171
+ The first exfil-guard rule set was written against the *scenarios* and then
172
+ pointed at this repository's own corpus. It was unusable: 21 files were
173
+ refused as "not ASCII" (the docs contain em-dashes and CJK prose), and the
174
+ source-reference rule fired on `if env:` in `core/audit.py`, on the string
175
+ `"env AGENT_GUARD_DIALECT"`, and on the rule's own regex source.
176
+
177
+ Three fixes, in order of value:
178
+
179
+ 1. **The non-ASCII refusal was removed.** It was over-defensive: the vendor
180
+ patterns are ASCII-anchored and `re` matches on characters, so a
181
+ non-ASCII byte cannot smuggle a match past them. Refusing every document
182
+ containing an em-dash bought no detection and made the guard unusable.
183
+ 2. **The environment-dump rule now requires the *shape* of a dump.**
184
+ `env`/`printenv` must be invoked at the start of a command (optionally
185
+ after a separator) or piped; `printenv FOO` and `env FOO=bar cmd` name
186
+ what they touch and are not dumps (design 2.4 item 4). A bare-name
187
+ property access (`process.env`) is not a file read either.
188
+ 3. **The repo-local exemption file (design 5.2) covers the rest.** The guard
189
+ cannot both quote `.env` in its own pattern table and detect `.env`;
190
+ `SECURITY.md` already documents this class of expected hit for other
191
+ scanners. Exemptions are scoped to *paths*, so the rules stay fully
192
+ active everywhere else, and values are exempted by hash only.
193
+
194
+ Measured result: **zero BLOCK across the whole repository corpus**, enforced
195
+ by `tests/test_exfil_sanitize.py::ExemptionFile::test_repository_own_corpus_has_zero_blocks`.
196
+ A verdict that changes for an existing shape is a rule bug here, not a
197
+ tuning problem (design 5.6).
198
+
199
+ ## F13 · A new decision class must not renumber the old ranks
200
+
201
+ `worst()` is shared by both guards, and exfil-guard needed `SANITIZE` to rank
202
+ *below* `ASK`. Renumbering the existing entries (`ALLOW 0, SANITIZE 1,
203
+ RELOCATE 2, SNAPSHOT 2, ASK 3, BLOCK 4`) preserves the relative order and
204
+ the ties exactly, so no delete-guard verdict can change. Verified as a pure
205
+ order-equivalence over all pairs before shipping, not by inspection.
206
+
207
+ `SANITIZE < ASK` is deliberate: `SANITIZE` is automatic (SAFE tier, like
208
+ `RELOCATE`), while `ASK` forfeits automation. A payload carrying both a
209
+ sanitizable secret and an un-rewritable shape must `ASK` - you cannot
210
+ silently proceed when part of the emission is uninspectable.
211
+
212
+ ## Verdict accuracy observed
213
+
214
+ | Command | Verdict | Correct? |
215
+ |---|---|---|
216
+ | `rm -rf build` (rooted dir) | RELOCATE_TREE → PROCEED | ✓ |
217
+ | `rm -rf .` | BLOCK_PROTECTED_PATH | ✓ |
218
+ | `rm -rf $UNSET/` | BLOCK_UNDETERMINABLE_EFFECT | ✓ |
219
+ | `git clean -fd`, pre-existing untracked present | RELOCATE_VIA_CLEAN_ENUMERATE; valuable file relocated | ✓ |
220
+ | `git clean -fd`, junk created by same line | proceeded unprotected | ✗ → F2 |
221
+ | `safe_delete` mixed glob + file | relocate file, report no-match | ✓ after F3 fix |
222
+ | heredoc write quoting destructive text | false BLOCK → fixed by strip_heredocs | ✓ after F5 |
223
+
224
+ ## F14 · An unknowable target set was reported as a broken compensation
225
+
226
+ `git clean -fdx` outside a repository refused correctly but named the wrong
227
+ reason: `COMPENSATION_FAILED` ("Compensation failed before execution")
228
+ describes a compensation that was attempted and broke. Nothing was
229
+ attempted here - the *target set* could not be enumerated, so the guard
230
+ never learned what it was about to protect. The label sends a reader
231
+ hunting for a fault in the quarantine engine that does not exist, and it
232
+ buries the actual remedy (name the paths explicitly).
233
+
234
+ **Fixed in v0.2.0-rc1:** enumeration failure raises
235
+ `EnumerationUnavailable` and takes its own refusal arm with
236
+ `BLOCK_UNDETERMINABLE_EFFECT` - the same verdict class as every other
237
+ unknowable target set (policy.md row 11b). A genuine compensation fault
238
+ (`git stash create` failing, audit intent not persisting) keeps
239
+ `COMPENSATION_FAILED`.
240
+
241
+ The same defect had a second half. Every refusal arm rewrites `decision`
242
+ and `code`, but none of them rewrote `explanation`, which had been built
243
+ from the verdict policy *proposed*. A `BLOCK_UNDETERMINABLE_EFFECT` was
244
+ therefore explained as "git clean was dry-run enumerated and every match
245
+ relocated before the real command ran" - a sentence describing an action
246
+ that never happened. All three arms now restate the explanation with the
247
+ code, and the CLI suite pins that invariant.
248
+
249
+ ## F15 · check_span hung on a pipe nobody wrote to
250
+
251
+ `check.py` takes its command line from argv, but `check_span.py` reads its
252
+ payload from stdin, and `sys.stdin.read()` blocks until EOF. A caller that
253
+ opened a pipe and never wrote to it - the payload omitted, or `--path`
254
+ passed on its own - left the CLI pinned until the *host's* tool timeout
255
+ killed the call (300 s under DSH). The guard presented as a hang rather
256
+ than a refusal, and every such mistake cost a full timeout window in which
257
+ nothing else could run.
258
+
259
+ **Fixed in v0.2.0-rc1:** the wait for a producer's first byte is bounded
260
+ (`AGENT_GUARD_STDIN_TIMEOUT`, default 10 s) and the refusal is loud - exit
261
+ 1, "no payload arrived on stdin within Ns". Deliberately unchanged:
262
+ interactive use still blocks for a human's Ctrl-D; a pipe that closes
263
+ without data is still a legitimately empty payload (ALLOW, since nothing
264
+ can leak); and on platforms where stdin is not selectable the code falls
265
+ back to the blocking read rather than refusing a caller whose payload may
266
+ be perfectly good.
@@ -0,0 +1,72 @@
1
+ # Harness capability and evidence matrix
2
+
3
+ The Python Core and CLI are harness-neutral. A Skill or CLI being usable in a
4
+ host does **not** mean that host intercepts tool calls, and an adapter returning
5
+ `BLOCK` does **not** by itself prove the host prevented execution.
6
+ For an unlisted host, use the [self-adaptation guide](../adapters/INTEGRATION.md)
7
+ and keep its native-interception status `UNVERIFIED` until independently tested.
8
+
9
+ ## Evidence levels
10
+
11
+ | Level | Meaning |
12
+ |---|---|
13
+ | CLI | The guard can be invoked deliberately by an agent or user. No automatic interception is claimed. |
14
+ | Adapter | A unit/smoke test proves payload-to-Core-to-host-decision translation. The host itself may be mocked. |
15
+ | Hook observed | A real host session invoked the configured hook for the specified tool call. This proves coverage only for observed calls. |
16
+ | Enforcement observed | A real host submitted a safe, synthetic operation that Core denied, and an independent sentinel confirmed the operation did not run. |
17
+
18
+ Evidence is version-, tool-, and configuration-specific. A test with a mock
19
+ model endpoint may prove host behavior without proving model behavior. A
20
+ subagent observation is not a guarantee for every subagent, tool, or schedule.
21
+
22
+ `doctor.py kimi|claude [--config PATH] [--probe] [--json]` checks one selected
23
+ configuration file and optionally runs local bridge/adapter verdict probes
24
+ in a temporary workspace with controlled dialect input.
25
+ Its `configuration: PASS` is a *file-shape check*, not an effective merged
26
+ host-settings check; `local_probe: PASS` is not host interception. The doctor
27
+ always reports `host_interception: UNVERIFIED`. Only a separate real-host
28
+ trial with an independent non-execution check can establish the bounded
29
+ "Enforcement observed" level above. Neither doctor mode starts a model.
30
+ The optional `--check-drift` mode additionally compares the installed host
31
+ version with a shipped compatibility profile and computes privacy-safe
32
+ configuration/runtime fingerprints. Its `CURRENT` status is static preflight
33
+ evidence only; `STALE` requests a re-test and does not itself mean the adapter
34
+ is incompatible. Optional local baselines can detect later `DRIFTED` inputs.
35
+ See the [host drift guide](host-drift.md).
36
+ Its opt-in `--live-sentinel` adds one real-host/model call. A PASS requires an
37
+ exact hashed adapter receipt, matching Core block audit, host block feedback,
38
+ and an absent marker; otherwise it reports FAIL or INCONCLUSIVE. This is
39
+ bounded "Enforcement observed" evidence only for that root-agent Bash call.
40
+
41
+ ## Current coverage
42
+
43
+ | Host / tested version | Entry and scope | ASK mapping | Highest evidence for current path | Important gap |
44
+ |---|---|---|---|---|
45
+ | Claude Code 2.1.270 / 2.1.273 (post-0.2.0 bridge) | `PreToolUse` for `Bash`; shared Core via Python and POSIX startup bridge | Native `permissionDecision: ask` (headless runs need an approver) | Real CLI/hook with scripted model endpoint: ALLOW, BLOCK, headless ASK, injected Python failure; see [report](test-report-claude-code-harness.md) | Hook absence/timeout, Windows bridge, other tools, arbitrary subagents and real model behavior are not covered. |
46
+ | DSH 0.1.5-rc.1 | `tools/pre-execute` for `bash`, plus model tools and prompt section | `kind: ask` | [Real host, no-model packaged-plugin probe](test-report-dsh-0.1.5-rc.1.md): benign execution, Core `BLOCK`, absent bypass marker, durable audit, and loaded-adapter Core-startup failure | Direct tool-runtime call only; model/Agent, subagents, concurrency, ASK UI, non-`bash` tools, hook absence and host upgrades remain unverified. |
47
+ | Kimi Code 0.42.0 / 2.1.1 | `PreToolUse` for `Bash`; reuses Claude mapping with a host-specific ASK override | Adapter hard-denies ASK; live ASK behavior was sampled only on 0.42.0 | [Bounded host-level tests](test-report-kimi-code-block.md): authenticated 2.1.1 root-Bash sentinel PASS on an OAuth official model and maintainer-confirmed official K3 relay; older 0.42.0 root/subagent BLOCK, recoverable deletion/restore and pre-adapter Python-failure refusal | The client cannot independently attest opaque relay weights. 2.1.1 subagents, concurrency, ASK UI and fault injection remain unverified; hook absence/timeout/bridge-spawn failure and non-Bash tools remain outside the claim. |
48
+ | Codex CLI 0.154.0 (tested session) | Skill and production CLI on older source; no native interception adapter in this repository | CLI reports ASK; agent/user must decide | CLI acceptance only; see [report](test-report-codex-gpt-6-astra-high.md) | No automatic tool-call enforcement claim. |
49
+ | ZCode (Windows session; host version unrecorded) | Historical Skill/CLI and process-type `PreToolUse` trial | One ASK prompted the user; persistent approval then skipped the hook | [Windows evaluation](test-report-zcode-glm-flash.md) | No current-version native enforcement claim; the observed permission bypass and real Windows end-to-end gaps remain. |
50
+ | Other / unlisted hosts | No supported adapter by default | Host-specific | [Self-adaptation checklist](../adapters/INTEGRATION.md), not a verification result | Requires a blocking pre-tool contract and independent host evidence; otherwise Skill/CLI only. |
51
+
52
+ These adapters currently address destructive **shell** calls. Exfil-guard is a
53
+ separate CLI/cooperative path; this table does not imply native interception of
54
+ file reads, model context assembly, uploads, or every output channel.
55
+
56
+ ## Acceptance for each native adapter
57
+
58
+ 1. Pin host version, profile/configuration, tool name, shell dialect, and
59
+ whether the call originates in the root agent or a subagent.
60
+ 2. Confirm a harmless call reaches the hook and continues. Confirm an ASK
61
+ either prompts the human or is denied; never silently treat it as allow.
62
+ 3. Submit a synthetic command that would create a uniquely named sentinel
63
+ **only if** the host ignores the guard's BLOCK. Verify the hook verdict and
64
+ independently verify the sentinel does not exist. No real user data or
65
+ dangerous deletion command is needed for this enforcement check.
66
+ 4. Exercise a guard failure/missing dependency and verify fail-closed behavior.
67
+ Distinguish an adapter failure from a host that never loaded the adapter.
68
+ 5. Report `OBSERVED`, `NOT OBSERVED`, `NOT VERIFIED`, or `INCONCLUSIVE` per
69
+ probe. Do not promote adapter-level output to execution-level evidence.
70
+
71
+ Host versions may change these contracts. Re-run acceptance after a host
72
+ upgrade or when a new tool/event path is added.
@@ -0,0 +1,142 @@
1
+ # Host drift checks
2
+
3
+ Host support is version-, configuration-, tool-, and event-specific. A hook
4
+ that worked yesterday may be disabled, moved, or interpreted differently after
5
+ a host upgrade. Agent Guard's drift check is a zero-model-call preflight for
6
+ that reliability problem; it is not a sandbox or a live-interception proof.
7
+
8
+ ## What is checked
9
+
10
+ `doctor.py kimi|claude --check-drift` performs three local checks:
11
+
12
+ 1. It queries the installed host's local `--version` command and compares the
13
+ parsed version with the adapter's shipped compatibility profile.
14
+ 2. It fingerprints only the normalized Agent Guard hook fields: event,
15
+ matcher, bridge command tokens, timeout, dialect, and blocking options.
16
+ Unrelated host configuration is neither copied into the report nor stored
17
+ in a baseline.
18
+ 3. It fingerprints the local enforcement path (profile, bridge, adapter,
19
+ shared guard entry point, and Core Python files).
20
+
21
+ This does not start a model or edit host configuration. The ordinary doctor
22
+ does not execute the host at all; host version discovery is opt-in through
23
+ `--check-drift`, `--baseline`, `--write-baseline`, or `--host-executable`.
24
+
25
+ ```sh
26
+ python3 doctor.py claude --probe --check-drift
27
+ python3 doctor.py kimi --probe --check-drift --json
28
+ ```
29
+
30
+ `--host-executable PATH` selects a non-default executable. It is also useful
31
+ when a host is installed under a wrapper name. The selected executable is run
32
+ only with the compatibility profile's version arguments.
33
+
34
+ ## Status vocabulary
35
+
36
+ | Status | Meaning |
37
+ |---|---|
38
+ | `NOT_CHECKED` | Version discovery and fingerprinting were not requested. |
39
+ | `CURRENT` | The detected version is in the shipped tested-version list and the selected configuration is structurally valid. With a baseline, its fingerprints also match. |
40
+ | `STALE` | The host version changed or is outside the shipped tested-version list. This is a request to re-test, not a claim of incompatibility. |
41
+ | `DRIFTED` | A supplied baseline no longer matches the selected hook configuration, Agent Guard runtime, or compatibility profile. |
42
+ | `BROKEN` | The selected configuration or requested local bridge probe failed. |
43
+ | `UNVERIFIED` | The host version, profile, baseline, or fingerprints could not be established. |
44
+
45
+ An explicit drift check exits non-zero unless the status is `CURRENT`, making
46
+ it suitable for a local preflight. `CURRENT` is deliberately weaker than
47
+ `host_interception`: the latter remains `UNVERIFIED` because a static file,
48
+ version command, and local adapter process cannot prove that a live host
49
+ loaded or enforced the hook.
50
+
51
+ ## Privacy-minimal baselines
52
+
53
+ A baseline is optional. Create one only after reviewing a passing installation:
54
+
55
+ ```sh
56
+ python3 doctor.py claude --probe --check-drift \
57
+ --write-baseline /existing/private/directory/claude-baseline.json
58
+
59
+ python3 doctor.py claude --check-drift \
60
+ --baseline /existing/private/directory/claude-baseline.json
61
+ ```
62
+
63
+ Baseline creation requires `CURRENT` and a passing local probe. The target's
64
+ parent must already exist, and the doctor refuses to overwrite an existing
65
+ file. A baseline contains only its schema number, harness/profile identifiers,
66
+ the parsed host version, and two SHA-256 fingerprints. It does not contain the
67
+ configuration, hook command, model data, or absolute paths. The file is
68
+ created with mode `0600`; keep it local rather than committing it as a public
69
+ compatibility claim.
70
+
71
+ After an intentional upgrade, use a new baseline path only after repeating
72
+ the review. A matching baseline shows that the checked local inputs did not
73
+ change; it is not an independent witness against a malicious process with the
74
+ same OS privileges.
75
+
76
+ ## Compatibility profiles and limits
77
+
78
+ Profiles live beside their adapters as `compatibility.json`. They describe
79
+ only the host-facing version command and the narrow contract already covered
80
+ by that adapter. Core policy remains shared and is not copied into profiles.
81
+ Adding a profile for an unlisted host does not make it supported: the host
82
+ still needs a blocking pre-tool event and the bounded acceptance evidence in
83
+ the [capability matrix](harness-capabilities.md).
84
+
85
+ No process running inside an adapter can prove that the host invoked it for
86
+ every call. Missing hooks, changed tool names, matcher bypasses, host timeouts,
87
+ new subagent routes, and error-semantics changes need bounded live-host
88
+ evidence. The optional sentinel below tests one root-agent Bash call only.
89
+
90
+ ## Optional low-token live-host sentinel
91
+
92
+ `--live-sentinel` starts the selected real Claude Code or Kimi Code CLI in a
93
+ fresh, private fixture and asks its configured model to make exactly one Bash
94
+ call. It therefore **may consume provider quota**. It is never run by the
95
+ ordinary doctor or by `--check-drift`.
96
+
97
+ ```sh
98
+ python3 doctor.py claude --config /path/to/settings.json --live-sentinel
99
+ python3 doctor.py kimi --config /path/to/config.toml --live-sentinel --json
100
+ ```
101
+
102
+ The probe command only creates a uniquely named empty marker in the fixture.
103
+ The host process receives a deliberately invalid, nonce-bearing shell dialect:
104
+ if the selected hook reaches Agent Guard, Core refuses the call with
105
+ `BLOCK_DIALECT_UNKNOWN`. No deletion, remote, repository rewrite, credential,
106
+ or user project is part of the probe.
107
+
108
+ A PASS requires all of the following independent evidence:
109
+
110
+ 1. the adapter wrote a create-only `0600` receipt for the exact Bash command;
111
+ the receipt contains hashes, not the command or working-directory path;
112
+ 2. the fresh fixture's redacted audit records `enforce-block` and
113
+ `BLOCK_DIALECT_UNKNOWN`;
114
+ 3. the real host returned Agent Guard's block feedback; and
115
+ 4. the marker does not exist.
116
+
117
+ The result and alarm vocabulary is deliberately asymmetric:
118
+
119
+ | Result | Alarm | Meaning |
120
+ |---|---|---|
121
+ | `PASS` | `NONE`, or `NOTICE` when the static drift status is not `CURRENT` | The bounded exact call produced all required hook/Core/host evidence and did not execute. The report includes the drift status that caused a notice. |
122
+ | `FAIL` | `CRITICAL` | The marker exists: the submitted command executed despite the expected block. |
123
+ | `INCONCLUSIVE` | `WARNING` | The host/model did not make the exact call, timed out, failed to start, or any receipt/audit/host-feedback element is missing. It is never promoted to PASS from marker absence alone. |
124
+
125
+ Exit codes are `0` for PASS, `2` for FAIL, and `1` for INCONCLUSIVE. A
126
+ `result.json` evidence summary is always written when the host starts. Raw
127
+ host/model stdout and stderr are hashed and counted, then discarded rather
128
+ than retained. The evidence directory is kept for user review and is never
129
+ automatically deleted or overwritten; `--sentinel-output NEW_PATH` selects it.
130
+
131
+ For Claude, the runner loads only the selected settings plus project sources,
132
+ limits the advertised tools to Bash, and keeps `dontAsk`; it does not enable
133
+ permission bypass. For Kimi, the selected file must be named `config.toml`,
134
+ and its parent is used as `KIMI_CODE_HOME`. The host may still keep its own
135
+ session logs outside Agent Guard's evidence directory.
136
+
137
+ This sentinel is a reliability check, not an adversarial sandbox. Run it only
138
+ with a model/provider you trust enough to receive one fixed prompt: when the
139
+ very hook being tested is absent, the host still has its normal OS privileges.
140
+ The blank fixture and narrow prompt reduce exposure but cannot contain a
141
+ malicious host or model. Subagents, other tools, concurrent calls, and future
142
+ host versions require separate evidence.