harness-everything 0.40.0 → 0.41.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/harness-everything/project-docs.json +9 -0
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +51 -0
- package/benchmarks/results/f1-doc-path-baseline/1-source-clean.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/1-source-clean.stdout.json +20 -0
- package/benchmarks/results/f1-doc-path-baseline/2-injected-architecture.md +229 -0
- package/benchmarks/results/f1-doc-path-baseline/2-source-injected.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/2-source-injected.stdout.json +33 -0
- package/benchmarks/results/f1-doc-path-baseline/3-source-reverted.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/3-source-reverted.stdout.json +20 -0
- package/benchmarks/results/f1-doc-path-baseline/4a-packaged-clean.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/4a-packaged-clean.stdout.json +20 -0
- package/benchmarks/results/f1-doc-path-baseline/4b-packaged-injected.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/4b-packaged-injected.stdout.json +33 -0
- package/benchmarks/results/f1-doc-path-baseline/5-source-missing-config.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/5-source-missing-config.stdout.json +15 -0
- package/benchmarks/results/f1-doc-path-baseline/5b-packaged-missing-config.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/5b-packaged-missing-config.stdout.json +15 -0
- package/benchmarks/results/f1-doc-path-baseline/README.md +54 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/1-source-clean.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/1-source-clean.stdout.json +115 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/2-injected-document.md +328 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/2-source-injected.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/2-source-injected.stdout.json +127 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/3-source-reverted.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/3-source-reverted.stdout.json +115 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/4a-packaged-clean.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/4a-packaged-clean.stdout.json +115 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/4b-packaged-injected.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/4b-packaged-injected.stdout.json +127 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/5-source-missing-config.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/5-source-missing-config.stdout.json +15 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/5b-packaged-missing-config.stderr.txt +0 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/5b-packaged-missing-config.stdout.json +15 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/6-supplementary-fixture-ok.input.md +8 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/6-supplementary-fixture-ok.stdout.json +16 -0
- package/benchmarks/results/f1-doc-path-baseline/before-fix/README.md +34 -0
- package/benchmarks/results/f1-doc-path-baseline/probe-31-docs/after.json +248 -0
- package/benchmarks/results/f1-doc-path-baseline/probe-31-docs/before.json +254 -0
- package/benchmarks/results/f1-doc-path-baseline/probe-31-docs/probe.js +26 -0
- package/benchmarks/results/f2-doc-craft-baseline/README.md +32 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f1-report.md +84 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f1-task.md +21 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f1.diff +19 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f2-report.md +110 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f2-task.md +21 -0
- package/benchmarks/results/f2-doc-craft-baseline/arm-f2.diff +19 -0
- package/benchmarks/results/f3-derived-views-baseline/README.md +34 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f2-report.md +43 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f2-task.md +21 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f2.diff +21 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f3-report.md +57 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f3-task.md +21 -0
- package/benchmarks/results/f3-derived-views-baseline/arm-f3.diff +21 -0
- package/benchmarks/results/live-host/claude-325-3b-s2b/README.md +99 -0
- package/benchmarks/results/live-host/claude-325-3b-s2b/capture.sanitized.jsonl +26 -0
- package/ci/archive/325-prep/README.md +71 -0
- package/ci/{prep → archive/325-prep}/doc-reconciliation-red.js +9 -7
- package/ci/archive/325-prep/f4-expected-red.json +34 -0
- package/ci/archive/325-prep/historical-qualification.test.js +143 -0
- package/ci/fixtures/doc-obligations/reconciliation-prep/cases.json +17 -4
- package/ci/fixtures/doc-obligations/reconciliation-prep/pending-red.json +8 -56
- package/ci/fixtures/doc-obligations/reconciliation-prep/trust-contract.json +2 -2
- package/ci/fixtures/doc-obligations/reconciliation-s1/cases.json +17 -5
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/harness.js +415 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/off-handoff-golden.json +16 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-post-tool-use-failure.json +11 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-post-tool-use-success-minimal.json +9 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-post-tool-use-success.json +13 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-subagent-start.json +7 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-subagent-stop-freetext-reject.json +11 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-subagent-stop-reject.json +11 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-subagent-stop.json +11 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/capture/payload-task-result.json +19 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/off-capture-only.json +99 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/off-legacy.json +67 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-concurrent-writers.json +62 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-lookalikes-runtime-mutations.json +67 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-malformed.json +85 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-normal-session.json +74 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-runtime-drift-later-npm-test.json +58 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/corpus/on-valid-direct-quoted-review.json +85 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/f4/cases.json +2267 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/wiring/cases.json +1110 -0
- package/ci/fixtures/doc-obligations/reconciliation-s2b/wiring/legacy-off-golden.json +674 -0
- package/ci/lib/doc-reconciliation-file-spy.js +23 -0
- package/ci/lib/doc-reconciliation-prep-fixtures.js +136 -10
- package/ci/lib/doc-reconciliation-qualification-preload.js +37 -0
- package/ci/lib/doc-reconciliation-replay.js +104 -0
- package/ci/lib/doc-reconciliation-staged-tree.js +102 -0
- package/ci/mechanism-2p-opencode-v2-entrypoint.test.js +179 -0
- package/ci/mechanism-325-3b-preparation.test.js +27 -101
- package/ci/mechanism-325-3b-s2b-capture.test.js +561 -0
- package/ci/mechanism-325-3b-s2b-wiring.test.js +668 -0
- package/ci/mechanism-325-f1-doc-check.test.js +674 -0
- package/ci/mechanism-325-f2-doc-craft.test.js +597 -0
- package/ci/mechanism-325-f3-derived-views.test.js +798 -0
- package/ci/mechanism-325-f4-invariants.test.js +606 -0
- package/ci/mechanism-325-f4-projections.test.js +382 -0
- package/ci/mechanism-325-f4-qualification.test.js +228 -0
- package/ci/mechanism-325-f4-trust-boundary.test.js +880 -0
- package/ci/mechanism-365-step-edit-accounting.test.js +395 -0
- package/ci/mutation-check.js +26 -3
- package/docs/architecture.md +5 -0
- package/docs/doc-workflow-reconciliation.md +331 -4
- package/docs/document-lifecycle.md +11 -0
- package/docs/governed-index.md +20 -0
- package/docs/platform-capabilities.md +22 -2
- package/grill-with-docs/SKILL.md +1 -1
- package/grill-with-docs/references/session-playbook.md +4 -0
- package/harness-everything/SKILL.md +1 -1
- package/harness-everything/references/doc-check.md +186 -0
- package/harness-everything/references/documentation/_shared/maintenance.md +41 -0
- package/harness-everything/references/documentation/_shared/writing-craft.md +59 -0
- package/harness-everything/references/triage-and-tiers.md +4 -0
- package/harness-everything/scripts/doc-check.js +442 -0
- package/harness-everything/scripts/doc-lifecycle.js +3 -0
- package/hooks/hooks.json +46 -0
- package/hooks/scripts/lib/doc-reconciliation-capture.sanitized.jsonl +26 -0
- package/hooks/scripts/lib/doc-reconciliation-pilot.js +171 -2
- package/hooks/scripts/lib/doc-reconciliation-provider.js +1059 -0
- package/hooks/scripts/lib/doc-reconciliation-qualification.json +34 -0
- package/hooks/scripts/lib/workflow-obligations.js +7 -0
- package/hooks/scripts/state-persist.js +105 -3
- package/hooks/scripts/workflow-stop-gate.js +7 -2
- package/opencode-plugin/README.md +15 -0
- package/opencode-plugin/index.mjs +77 -0
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/hooks/scripts/lib/doc-reconciliation-capture.sanitized.jsonl +26 -0
- package/plugins/harness-everything/hooks/scripts/lib/doc-reconciliation-pilot.js +171 -2
- package/plugins/harness-everything/hooks/scripts/lib/doc-reconciliation-provider.js +1059 -0
- package/plugins/harness-everything/hooks/scripts/lib/doc-reconciliation-qualification.json +34 -0
- package/plugins/harness-everything/hooks/scripts/lib/workflow-obligations.js +7 -0
- package/plugins/harness-everything/hooks/scripts/state-persist.js +105 -3
- package/plugins/harness-everything/hooks/scripts/workflow-stop-gate.js +7 -2
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +1 -1
- package/plugins/harness-everything/skills/grill-with-docs/references/session-playbook.md +4 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/references/doc-check.md +186 -0
- package/plugins/harness-everything/skills/harness-everything/references/documentation/_shared/maintenance.md +41 -0
- package/plugins/harness-everything/skills/harness-everything/references/documentation/_shared/writing-craft.md +59 -0
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +4 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/doc-check.js +442 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/doc-lifecycle.js +3 -0
- package/plugins/harness-everything/skills/repo-docs/SKILL.md +1 -1
- package/plugins/harness-everything/skills/repo-docs/references/process-guide.md +4 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +1 -1
- package/plugins/harness-everything/skills/to-spec/references/process.md +4 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +1 -1
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +4 -0
- package/repo-docs/SKILL.md +1 -1
- package/repo-docs/references/process-guide.md +4 -0
- package/sbom.cdx.json +7 -7
- package/to-spec/SKILL.md +1 -1
- package/to-spec/references/process.md +4 -0
- package/to-tickets/SKILL.md +1 -1
- package/to-tickets/references/publishing-and-templates.md +4 -0
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
---
|
|
2
|
+
role: reference
|
|
3
|
+
status: active
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# Document workflow reconciliation — Phase 3b S1 runtime
|
|
2
7
|
|
|
3
8
|
**Status: S1 runtime ships default OFF; opt in with `HARNESS_DOC_RECONCILIATION_PILOT=1`.
|
|
@@ -286,12 +291,12 @@ path and file content with NUL bytes. The current list is
|
|
|
286
291
|
`hooks/scripts/workflow-stop-gate.js`; `doc-evidence.js` is excluded because the P01–P08 replays
|
|
287
292
|
do not load it. A changed fingerprint fails qualification. To refresh deliberately, the runner
|
|
288
293
|
prints an explicit command such as
|
|
289
|
-
`node ci/prep/doc-reconciliation-red.js --recapture-candidate ci/fixtures/doc-obligations/reconciliation-prep/pending-red.candidate.json --baseline-sha <40-hex-commit
|
|
294
|
+
`node ci/archive/325-prep/doc-reconciliation-red.js --recapture-candidate ci/fixtures/doc-obligations/reconciliation-prep/pending-red.candidate.json --baseline-sha <40-hex-commit>` (historical).
|
|
290
295
|
It writes a separate candidate; review its actual/desired values and reason codes before replacing
|
|
291
296
|
the committed registry, then remove the candidate. Nothing is auto-accepted.
|
|
292
297
|
|
|
293
|
-
The [preparation runner](../ci/prep/doc-reconciliation-red.js) has two replay modes:
|
|
294
|
-
`--strict` exits 1 when a desired-contract gap remains; `--expect-red` exits 0 only when every
|
|
298
|
+
The [archived preparation runner](../ci/archive/325-prep/doc-reconciliation-red.js) is historical T2 evidence, not a current gate. It has two replay modes:
|
|
299
|
+
`--strict` exits 1 when a desired-contract gap remains; historical `--expect-red` exits 0 only when every
|
|
295
300
|
current mismatch matches an exact case/assertion/distribution/actual/desired registry entry and
|
|
296
301
|
all controls and parity checks pass. A stale fingerprint, unexpected PASS, unqualified mismatch,
|
|
297
302
|
wrong assertion, crash, missing fixture, import error, or other infrastructure failure fails
|
|
@@ -504,7 +509,7 @@ registry retains these five S2 assertion IDs × two distributions:
|
|
|
504
509
|
withholding case for S1. Merely changing exit policy introduces an unregistered initial-state gap.
|
|
505
510
|
- Deliberately review/recapture the changed baseline and registry actual/desired qualification;
|
|
506
511
|
include the newly imported pilot module in the ordered runtime fingerprint. Then exactly the
|
|
507
|
-
targeted entries retire and the remaining `--expect-red` run must qualify on that reviewed S1
|
|
512
|
+
targeted entries retire and the remaining historical `--expect-red` run must qualify on that reviewed S1
|
|
508
513
|
baseline. Deleting registry rows alone proves nothing.
|
|
509
514
|
- Additional strict S1 cases: governed diff without authored `doc-change`, direct aggregate PASS
|
|
510
515
|
after authored completion, correlated Fable stage binding, persisted untyped workflow, legacy
|
|
@@ -874,6 +879,204 @@ Outside this contract: attestation service/PKI, a new command runner/authorizati
|
|
|
874
879
|
review provider, PRD-root/schema generalization, automatic verifier/generator, code→doc graph
|
|
875
880
|
(#350) and new locks. S2a does not change the public derive/check APIs or weaken reviewer independence.
|
|
876
881
|
|
|
882
|
+
## S2b — host capture, wiring and freshness
|
|
883
|
+
|
|
884
|
+
**Docs-stage contract; host wiring and positive runtime credit are not claimed as shipped.**
|
|
885
|
+
This section implements the existing [S2 row](https://github.com/dyphn1/Harness-everything/issues/325#issuecomment-6308606462)
|
|
886
|
+
using the [owner-approved host-origin decisions](https://github.com/dyphn1/Harness-everything/issues/325#issuecomment-6312323110).
|
|
887
|
+
S2a supplies the producer, accepted-event adapter and expected-set core; S2b supplies the actual
|
|
888
|
+
Claude Code origin provider and workflow boundaries. Default OFF, unsupported-host withholding,
|
|
889
|
+
unchanged checker APIs and the separate E1/3c/promotion gates remain binding.
|
|
890
|
+
|
|
891
|
+
### Activation and pilot-only scope
|
|
892
|
+
|
|
893
|
+
The pilot MUST establish an actual ON + supported decision before any pilot capture or changed
|
|
894
|
+
workflow disposition: exact switch value `1`, a loaded non-Fable typed contract, and correlated
|
|
895
|
+
Claude Code host context. A process environment hint, authored doc step or caller host label alone
|
|
896
|
+
cannot establish support. OFF MUST return to the existing legacy path before pilot collection,
|
|
897
|
+
journal writes, report mutation or principal checks. This restriction includes helper functions
|
|
898
|
+
and review handling, not only the final credit branch.
|
|
899
|
+
|
|
900
|
+
OpenCode and Codex MUST report positive capability `unavailable` when ON: OpenCode has no supported
|
|
901
|
+
shell-result/subagent capture here; Codex tool-level hooks are not live-verified. Fable and loaded
|
|
902
|
+
untyped workflows likewise remain unsupported. Their ON unavailable report is a diagnostic surface,
|
|
903
|
+
not permission to capture or change their lifecycle; OFF emits no new pilot report. Unsupported
|
|
904
|
+
hosts MUST NOT borrow Claude's exit mapping. On supported Claude paths, absence of any required
|
|
905
|
+
origin/input or the live qualification below retains S1 withholding. The live gate controls credit,
|
|
906
|
+
not the ON + supported capture needed to gather the qualifying evidence.
|
|
907
|
+
|
|
908
|
+
### Claude capture and origin-provider contract
|
|
909
|
+
|
|
910
|
+
Capture MUST be pilot-only and fail-open to the host: missing fields, storage failure and collector
|
|
911
|
+
exceptions produce no accepted evidence and never block a tool or Stop. Existing handoff behavior
|
|
912
|
+
in `state-persist.js` remains intact. Capture does not execute a command, verifier or generator.
|
|
913
|
+
|
|
914
|
+
| Capture seam | Retained fields and required binding |
|
|
915
|
+
| --- | --- |
|
|
916
|
+
| Existing PreToolUse identity seam | Correlate the authorized invocation's `tool_use_id`, session/turn/task/workflow and start snapshot before execution. Retain the S2a identity tuple and reference; a Post-only current snapshot cannot be relabeled as the invocation's start identity. Missing start correlation is unavailable |
|
|
917
|
+
| `state-persist.js` on PostToolUse / PostToolUseFailure | Retain event name, exact command, cwd, `tool_use_id`, `session_id`, `prompt_id` when supplied, `agent_id`/`agent_type` when supplied, `duration_ms`, success `interrupted` or failure `is_interrupt`, failure disposition and response digest/reference. Bind end snapshot, actual capability/scope and event/result refs to the same invocation; absent optional fields remain absent, never invented |
|
|
918
|
+
| Agent/Task result capture | Add Claude matcher coverage for the spawning tool's actual name. Retain parent invocation/session correlation, host result principal fields and review-result digest/reference. A background launch is not a completed review; a caller-supplied name or review prose is not an agent identity |
|
|
919
|
+
| Claude SubagentStart / SubagentStop manifest entries | Record host-issued `agent_id`/`agent_type`, parent session/task correlation and start/stop event refs. SubagentStop or completed Agent/Task result supplies the reviewer principal and bound final review artifact; inconsistent identities between surfaces are rejected |
|
|
920
|
+
|
|
921
|
+
For shell-form input, adaptation MAY use `argv = [command]` to satisfy the existing command/argv
|
|
922
|
+
consistency check; this records the shell form, not an invented parsed argv. Preserve cwd, invocation
|
|
923
|
+
identity and immutable response reference. A digest binds captured bytes; it does not authenticate
|
|
924
|
+
their origin. Failure errors/interruption are retained even without a successful tool_response.
|
|
925
|
+
An absent numeric exit is handled only by the exact approved mapping below; absent or ambiguous
|
|
926
|
+
interruption/success/failure correlation cannot be treated as success. A numeric failure or other
|
|
927
|
+
contradictory captured outcome MUST NOT be overridden by the success mapping.
|
|
928
|
+
|
|
929
|
+
The following trust-contract rows reproduce the owner decision verbatim. They specialize the S2a
|
|
930
|
+
origin adapter for Claude; they do not change the checker schema or introduce generic null→0
|
|
931
|
+
conversion. The reviewer row defines the bounded claim for this host, including its forgery limit.
|
|
932
|
+
|
|
933
|
+
| # | Decision | Bound |
|
|
934
|
+
|---|---|---|
|
|
935
|
+
| 1 | **Exit mapping.** Claude Code's `PostToolUse` success payload has no numeric exit code. For a given `tool_use_id`, the S2b origin provider treats "`PostToolUse` received, `interrupted` is false, and no `PostToolUseFailure` arrived" as exit `0`. This is a named mapping of the host's own success/failure classification, not a null→0 conversion. | A `PostToolUseFailure` is never credited. Its `Exit code N` text is used for diagnostics only. The mapping is written verbatim as a row in the trust contract. |
|
|
936
|
+
| 2 | **Reviewer independence uses the host-issued subagent principal.** The reviewer is identified by the subagent's `agent_id`/`agent_type`, taken from `SubagentStop` or the Agent/Task result payload. It must differ from the author's `session_id` and from every executor principal. S2b adds `SubagentStart`/`SubagentStop` and Task-result capture to the Claude manifest. | The claim is bounded. The agent can run arbitrary code, so it can forge local journals. The guarantee is that host payload fields are re-validated at assessment time, and copied-record replay tests are required negatives. **Cryptographic unforgeability is not claimed.** Human approval is not available as a reviewer principal. |
|
|
937
|
+
|
|
938
|
+
The origin provider MUST implement the existing `resolveContext` / `resolveEvent` seam and
|
|
939
|
+
re-validate captured host payload fields against current workspace/session/task/workflow, pinned
|
|
940
|
+
baseline, requested event reference, invocation and principal relationships on every assessment.
|
|
941
|
+
It MUST reject copied records from another session, task, invocation, workflow revision or snapshot,
|
|
942
|
+
changed payload/result digests, conflicting success/failure records and fabricated principal labels.
|
|
943
|
+
An exact duplicate or altered record cannot become a new accepted event by changing its filename.
|
|
944
|
+
A failure arriving for a previously mapped invocation invalidates that invocation's current credit
|
|
945
|
+
on reassessment. Missing origins remain unavailable; writable local journals, permissions and
|
|
946
|
+
hash agreement alone cannot upgrade them. These replay checks do not promise protection from an
|
|
947
|
+
agent replacing all local state or invoking hook code with fabricated input.
|
|
948
|
+
|
|
949
|
+
Review approval still needs approve verdict, nonempty summary, current identity, reviewed artifact
|
|
950
|
+
digests and scope. The host principal is derived from capture, never `independentOf` prose.
|
|
951
|
+
Author and executor relationships MUST also be correlated; unknown relationships cannot prove
|
|
952
|
+
independence. Host result aliases such as `agentId` require a demonstrated capture mapping to the
|
|
953
|
+
same subagent identity; never guess it from text. Human approval is unavailable for this S2b path.
|
|
954
|
+
The provider MUST validate what a successful command actually checked: an arbitrary exit-0 command
|
|
955
|
+
cannot claim document lint, link, conflict or requirement-sync capability.
|
|
956
|
+
|
|
957
|
+
### Retained live-capture gate
|
|
958
|
+
|
|
959
|
+
Positive credit MUST stay disabled until one retained live Claude Code capture contains populated
|
|
960
|
+
`tool_use_id` for an actual tool outcome and `agent_id` for its correlated reviewer-subagent path.
|
|
961
|
+
Use `benchmarks/results/live-host/claude-325-3b-s2b/` as the retained evidence location, with
|
|
962
|
+
`capture.sanitized.jsonl` for hook payloads and `README.md` for host/version, plugin/revision,
|
|
963
|
+
capture procedure, event references, digests and bounded conclusions. These are required future
|
|
964
|
+
artifacts, not claims that files or a live capture already exist. Preserve the required correlation
|
|
965
|
+
fields through sanitization; a transcript alone, synthetic payload or screenshot of a name is not
|
|
966
|
+
this hook-capture evidence.
|
|
967
|
+
|
|
968
|
+
The origin provider's qualification check MUST resolve that retained evidence reference (or its
|
|
969
|
+
verified export), validate captured host/event kinds, nonempty tool/subagent IDs, parent/session
|
|
970
|
+
correlation and recorded digests, and match the supported host/capture implementation being used.
|
|
971
|
+
Missing, malformed, mismatched or unresolvable qualification evidence returns unavailable and no
|
|
972
|
+
credit. Do not enable qualification from a caller `trusted` flag, environment toggle, populated
|
|
973
|
+
fields in the current journal or a fake provider. Tests may inject an explicitly fake qualification
|
|
974
|
+
through dependency injection; production has no fixture trust switch. Qualification proves this
|
|
975
|
+
host capture path exposed the fields; every later event still needs current origin/identity checks.
|
|
976
|
+
A retained capture does not authenticate arbitrary local journals or imply E1 GO. Record the live
|
|
977
|
+
gate as unmet until this check actually passes; do not label the docs or deterministic tests as live.
|
|
978
|
+
|
|
979
|
+
### Workflow credit, review progression and freshness
|
|
980
|
+
|
|
981
|
+
Every ON + supported credit attempt MUST invoke the following current chain, with the same
|
|
982
|
+
correlated origin provider: `produceDocSnapshot` → `acceptDocEvents` →
|
|
983
|
+
`checkAcceptedDocEvidence`. Only the current adapter output may feed the core; neither CLI JSON,
|
|
984
|
+
a caller-fabricated `accepted` object nor a previously persisted report may bypass that chain.
|
|
985
|
+
S2a core `eligible` is mode-specific, never by itself full workflow verification.
|
|
986
|
+
|
|
987
|
+
| Boundary | Required decision and persisted behavior |
|
|
988
|
+
| --- | --- |
|
|
989
|
+
| Authored doc-change | Existing authorized/binding-checked advance is allowed; it grants no document-MUST verification credit |
|
|
990
|
+
| Governed normative review PASS | After existing active-step/binding validation, use review mode. Current independent quality-review evidence can pass before later verifier records exist. Missing or self-reviewed approval leaves review pending, refuses the PASS with exit 2, and Stop reports the review step. Review eligibility never sets full pilotCredit |
|
|
991
|
+
| Verification-step PASS | Live qualification plus verification mode must credit every expected applicable MUST, with unknown MUSTs unresolved, N/A reasoned and SHOULD diagnostic-only. Otherwise retain S1 withholding, no verification-step PASS and exit 2 |
|
|
992
|
+
| Direct aggregate execute / verify PASS | Apply the same current full evidence guard even when authored steps are already complete. Text is not a bypass. Internal derived authored execute may still reflect completed authored work; document verification cannot be derived from loaded skills |
|
|
993
|
+
| Completion, including persisted satisfied / all-PASS | Recollect all ten identity fields and re-resolve accepted origins/review bindings immediately before completion credit. Any stale/unavailable/unresolved MUST revokes current aggregate completion credit, retains authored history, and uses S1 running/pending diagnostics. Both Stop surfaces return success to the host |
|
|
994
|
+
|
|
995
|
+
At verification and completion, bracket collection/checking and the credit write with identity
|
|
996
|
+
rechecks. A mutation of candidate/config/task/plan/baseline or accepted bindings between collection,
|
|
997
|
+
checker evaluation and persistence MUST withhold rather than persist a stale PASS. The pinned
|
|
998
|
+
baseline cannot advance after a commit; passing a step or changing a timestamp cannot alter stable
|
|
999
|
+
task semantics. ON→OFF→ON rechecks current evidence instead of trusting legacy PASS. No host lock
|
|
1000
|
+
or command execution is introduced at these boundaries. A complete trusted non-document decision
|
|
1001
|
+
retains P06 legacy behavior and never fabricates verified document credit.
|
|
1002
|
+
|
|
1003
|
+
A positive runtime report MUST identify the current tuple, full expected MUST IDs, unchanged
|
|
1004
|
+
checker results, accepted record refs and diagnostics; verified pilotCredit requires the current
|
|
1005
|
+
full verification decision and live gate. Unknown/unavailable/failed evidence stays visible.
|
|
1006
|
+
Recollection or report-persistence failure cannot earn credit. The report and workflow state must
|
|
1007
|
+
agree about withholding; a stale positive report cannot remain the authority after revocation.
|
|
1008
|
+
|
|
1009
|
+
### S2b GREEN implementation decisions (coordinator, HOW only)
|
|
1010
|
+
|
|
1011
|
+
| Question raised at RED | Decision |
|
|
1012
|
+
| --- | --- |
|
|
1013
|
+
| How do tests hand origin evidence to the wired CLI/Stop? | Only through the session capture journal that ON `state-persist` writes (content discovery, as in the capture contract) plus a simulated live-capture artifact in the documented form. No other input channel reaches the wired runtime. |
|
|
1014
|
+
| Where does `taskId` come from? | A workflow-local task identity: the workflow id, recorded at ON `plan`/`start` next to `docReconciliationBaseline` and validated against the current workflow at every assessment. A changed or missing task identity is `identity-mismatch`, never a new task. |
|
|
1015
|
+
| What is the credit seam? | `assess(context, set, options)` / `updateStep(..., options)` accept `options.originProvider` for in-process tests only. Production always builds `createClaudeOriginProvider` from the session journal. No CLI argument, environment value or host payload can inject a provider or an `accepted` object (closes carry-over N1 at the wiring boundary). |
|
|
1016
|
+
|
|
1017
|
+
### Exact S2b RED handoff and registry retirement
|
|
1018
|
+
|
|
1019
|
+
The following five IDs each have `source` and `packaged` entries: exactly ten pending gaps.
|
|
1020
|
+
All GREEN claims MUST exercise the actual host-provider/wiring path with declared test-origin
|
|
1021
|
+
fixtures, not a hardcoded eligible result. Deterministic fake-host evidence remains mechanism
|
|
1022
|
+
proof; authentic host claims additionally require the retained live gate.
|
|
1023
|
+
|
|
1024
|
+
| Assertion ID (both legs) | Required GREEN condition |
|
|
1025
|
+
| --- | --- |
|
|
1026
|
+
| `P02e.on.review-pass-exit` | Current self/unknown reviewer provenance refuses review PASS with exit 2; an otherwise-valid distinct host subagent approval passes |
|
|
1027
|
+
| `P02e.on.stop-reports` | Rejected review remains active/pending and Stop includes `step:req-review:pending`, exit 0 |
|
|
1028
|
+
| `P03.on.verify-retry-exit` | Valid independent review remains reachable; first verification is refused without verifier events; later fresh accepted events for the same tuple make retry exit 0. Set retry ON expectExit to exactly 0 and restore final satisfied completion from S1's staged running assertion |
|
|
1029
|
+
| `P05.on.stop-on-2-withheld` | Despite the historical suffix, this ID currently asserts initial `verify-pass.exitCode === 0`; provide complete positive producer/events before mutation |
|
|
1030
|
+
| `P05.on.stop-on-3-withheld` | This ID currently asserts initial `stop-on-1.workflow.state === satisfied`; establish positive completion, then retain the separate candidate/config stale-credit and ON→OFF→ON negatives |
|
|
1031
|
+
|
|
1032
|
+
Materialize real Git baseline/candidate/config fixtures for the positive P02e/P03/P05 paths and
|
|
1033
|
+
correlate their host-origin test records. Empty/non-Git maps and arbitrary `prep-host-events` JSON
|
|
1034
|
+
cannot become positive runtime authority. Keep all S1 refusal/compatibility coverage with its
|
|
1035
|
+
intended evidence absence. An empty registry is the strict-prep mechanism gate, not live-host
|
|
1036
|
+
proof, E1 GO or permission for wider rollout.
|
|
1037
|
+
|
|
1038
|
+
S2a's pure A01–A14 vectors remain required. S2b MUST additionally execute these wired strict cases:
|
|
1039
|
+
|
|
1040
|
+
| Case | Required runtime assertion |
|
|
1041
|
+
| --- | --- |
|
|
1042
|
+
| Task/host substitution | Reuse otherwise-valid evidence after task/scope/requirements/bindings/revision or host changes; current origin/tuple mismatch rejects credit, while unchanged context succeeds |
|
|
1043
|
+
| Config/root/inventory | Parse/hash substitution, equal parsed config with different bytes/host selection, unsupported custom PRD root, baseline/candidate disagreement, omitted new/deleted/rename paths and PRD-only whole-identity substitution remain no-credit through actual disposition/Stop |
|
|
1044
|
+
| Omitted authored step/result | Same governed diff derives the same MUSTs without doc-change; missing/empty/subset results cannot pass merely because every returned result passes |
|
|
1045
|
+
| Old typed PASS gains governed scope | Start genuinely non-document and satisfied, then change governed content without changing authored steps; Stop revokes current credit, retains history and exits 0 |
|
|
1046
|
+
| Commit/rename freshness | Commit mid-workflow and move content across rename endpoints; current HEAD never replaces the pinned baseline and all current stored-byte bindings are checked |
|
|
1047
|
+
| Capture/check/persist race | Mutate candidate/config/task or accepted binding at each boundary after a previously valid positive evaluation; no stale report/PASS survives |
|
|
1048
|
+
| Live qualification and replay | Missing/invalid qualification, copied or altered journal/result, same ID with failure/interruption, self reviewer or principal alias substitution all remove the prior positive path; fresh valid captures pass |
|
|
1049
|
+
| Ignored-path producer coverage | Exercise S2a's closed F4 case through disposition/completion: ignored governed/derived/config/relevant code content is included or unavailable, never silently non-document |
|
|
1050
|
+
| Actual unsupported topology | Exercise a genuinely active correlated Fable branch and persisted untyped workflow separately from fallback-generic fixtures; report unavailable independently of generic completion without Fable credit or a new lock |
|
|
1051
|
+
|
|
1052
|
+
### Carry-over dispositions and non-vacuity
|
|
1053
|
+
|
|
1054
|
+
| Review item | S2b disposition |
|
|
1055
|
+
| --- | --- |
|
|
1056
|
+
| V2a-N1: fabricated accepted structure | Close in wiring: only current `acceptDocEvents` output, after actual provider resolution, enters the core. Injecting an internally consistent forged accepted/checker object at a public boundary MUST fail; test that bypassing acceptance changes a successful-origin negative |
|
|
1057
|
+
| V2a-N2: final origin outage labeled snapshot-changed | Preserve existing S2a frozen diagnostic and no-credit behavior. Add a final-only provider-outage case at the wired boundary and retain phase/origin context in runtime diagnostics; reason-code refinement is optional and must not weaken A14 or change the checker API |
|
|
1058
|
+
| V2a-N3: candidate Git failure labeled baseline-unavailable | Preserve the conservative existing code; assert candidate check-attr failure is no-credit, with operation context identifying the candidate phase. Do not relabel it successful or change evidence semantics to improve wording |
|
|
1059
|
+
| V2a-N4: rejected capability marked seen | Preserve conservative rejection for a batch containing invalid/duplicate/conflicting evidence; a new assessment with fresh valid refs can recover. Do not filter relevant failures out of a batch to manufacture PASS; add mixed-invalid/valid and fresh-retry cases |
|
|
1060
|
+
| S1 F4: ignored paths | Closed by S2a producer, not re-accepted as an S2b exemption; verify through wired positive-then-ignored-change tests |
|
|
1061
|
+
| S1 F6: unsupported-path coupling | Execute the actual Fable branch rather than counting generic fallback reporting as its coverage; assert unsupported diagnostics and Stop success independently |
|
|
1062
|
+
|
|
1063
|
+
Non-vacuity MUST begin with a successful full-credit wired vector and a successful review-only
|
|
1064
|
+
progression. Then require targeted mutations to fail the intended assertions: bypass current
|
|
1065
|
+
provider/acceptDocEvents resolution; skip reviewer independence; apply Claude success mapping to
|
|
1066
|
+
failure/interruption/missing ID; trust copied result/journal or old pilot report; bypass the live
|
|
1067
|
+
gate; iterate only returned results; re-anchor baseline to HEAD; hash only PRDs; drop config-byte
|
|
1068
|
+
binding; exclude ignored relevant paths; skip verification/completion recollection; and skip the
|
|
1069
|
+
last recheck before persistence. A mutant is not killed merely because S1 still refuses everything,
|
|
1070
|
+
a fixture cannot start, a dependency is missing or a test crashes. Assert exact reason/state and
|
|
1071
|
+
loss of a previously valid credit path; keep fake-origin and retained-host evidence layers separate.
|
|
1072
|
+
|
|
1073
|
+
OFF/unset/`0`/other values, genuine non-document P06, P07u, P02d diagnostic ordering, existing
|
|
1074
|
+
required bindings/authorization and unsupported-host lifecycle MUST remain compatible. Intentional
|
|
1075
|
+
PASS refusal has exact exit 2; both workflow Stop and generic Stop have exit 0 under missing origins,
|
|
1076
|
+
collector/report failure, stale evidence and unsupported topology. Capture failure itself always
|
|
1077
|
+
fails open to the host while withholding evidence credit. No tool/Stop lock, auto-commit, forced
|
|
1078
|
+
provider, discovery execution, public checker change or new attestation service/PKI is introduced.
|
|
1079
|
+
|
|
877
1080
|
## Implementation and acceptance after E0
|
|
878
1081
|
|
|
879
1082
|
S1 ships the default-OFF switch, narrow disposition/completion no-credit integration and runtime
|
|
@@ -884,3 +1087,127 @@ relevant green CI including Windows. The S1 suites establish their tested mechan
|
|
|
884
1087
|
contracts; they do not establish independent host provenance, live-host loading or architectural
|
|
885
1088
|
benefit. E1 has not started and begins only after a creditable S2; E1 GO and H1–H4/paired evidence
|
|
886
1089
|
remain separate gates.
|
|
1090
|
+
|
|
1091
|
+
## F4: credited reconciliation (Claude-only, pilot ON)
|
|
1092
|
+
|
|
1093
|
+
Status: **implemented behind the pilot switch; mechanism and package evidence, plus one live field-availability capture.** Credit works for `doc-lint` and `link-check` (a registered `doc-check` run) and for an independent `quality-review` (a host reviewer with bound path digests). One live capture (Claude Code 2.1.308, macOS, one session) proves field availability, not live-host credit. There is no live-host credit run yet. Deterministic mechanism tests prove the chain. Design source: the A3 advisor report (`.worktrees/packets/feat-path/A3-advisor-report.md`, a local coordination file, not a shipped doc).
|
|
1094
|
+
|
|
1095
|
+
### Whole path
|
|
1096
|
+
|
|
1097
|
+
```mermaid
|
|
1098
|
+
flowchart LR
|
|
1099
|
+
P["Plan/start: pin baseline + task/workflow semantics"] --> B["PreToolUse / review start: capture identity + command + principal"]
|
|
1100
|
+
B --> X["Claude runs registered doc check or verifier"]
|
|
1101
|
+
X --> H["PostToolUse/Failure or SubagentStop: capture raw result + IDs"]
|
|
1102
|
+
H --> J["Session-owned origin records outside candidate tree"]
|
|
1103
|
+
J --> A["Current assess: live qualification + origin provider"]
|
|
1104
|
+
A --> S["produceDocSnapshot"]
|
|
1105
|
+
S --> E["acceptDocEvents: current refs, identity, scope, principal"]
|
|
1106
|
+
E --> C["checkAcceptedDocEvidence: expected MUST set"]
|
|
1107
|
+
C --> R["Recheck identity/origins; persist eligible mode only"]
|
|
1108
|
+
R --> T["Completion/Stop: recollect; stale evidence loses credit"]
|
|
1109
|
+
```
|
|
1110
|
+
|
|
1111
|
+
### Modes
|
|
1112
|
+
|
|
1113
|
+
| Mode | Condition | Result |
|
|
1114
|
+
|---|---|---|
|
|
1115
|
+
| OFF | `HARNESS_DOC_RECONCILIATION_PILOT` unset (default) | Legacy behaviour. No pilot capture, assessment or report writes |
|
|
1116
|
+
| ON without qualifying evidence | Pilot on, but a host field, capture or live qualification is missing, or an identity or origin recheck fails | Withheld: no credit. Both Stop entry points exit 0 |
|
|
1117
|
+
| ON with current accepted evidence and live qualification | Every identity current; capture validated | Credit only for the capabilities in the result contract below |
|
|
1118
|
+
|
|
1119
|
+
### Projection and trust limits
|
|
1120
|
+
|
|
1121
|
+
- The narrow candidate scanner only selects doc references. It does not prove that a command contains no checker execution.
|
|
1122
|
+
- Trust uses a separate, conservative rule for generic verification and reviewer independence. A command that is compound (newline, heredoc, `$()`, shell operators, malformed quoting) or names the checker is not generic verification, and its shell executor counts for independence. Accepted tradeoff (owner decision 3): benign compounds such as `npm test && echo ok` lose generic credit and must run separately.
|
|
1123
|
+
- Accepted known limit (owner decision 4): in unreleased pilot-ON dev sessions created before WB2, aggregate-only verification may be cleared on checker drift and must be re-verified. Current source-tagged state keeps R3 preservation.
|
|
1124
|
+
|
|
1125
|
+
### Test transport
|
|
1126
|
+
|
|
1127
|
+
- Raw fixture hook payloads go through the real hook subprocess entry points, which write their own session records. The disposition and Stop subprocesses then run against that same isolated task and session.
|
|
1128
|
+
- Qualification comes only from a CI-only preloader passed as `node --require <ci-preloader> <entry>`. It supplies explicit fake qualification through the origin-provider factory. It may change qualification only. It never changes the snapshot, the `resolveEvent` outcome, principals, capability, accepted records or core eligibility.
|
|
1129
|
+
- No production environment or argv switch exists. The preloader stays under `ci/` and is not packaged.
|
|
1130
|
+
- This proves **deterministic simulated-host behaviour**. It does not prove live-host behaviour.
|
|
1131
|
+
|
|
1132
|
+
Replay negatives that must reject:
|
|
1133
|
+
|
|
1134
|
+
- Cross-session, cross-task, workflow-revision, baseline or candidate replay
|
|
1135
|
+
- The same record under a new name
|
|
1136
|
+
- A duplicate tool ID
|
|
1137
|
+
- Conflicting success and failure for one tool
|
|
1138
|
+
- An edited raw payload or result
|
|
1139
|
+
- A false `independentOf` claim in prose
|
|
1140
|
+
- A stale review
|
|
1141
|
+
- Forged accepted or checker structures
|
|
1142
|
+
|
|
1143
|
+
### Owner's bounded claim
|
|
1144
|
+
|
|
1145
|
+
Quoted from the #325 owner decisions of 2026-10-21 (09:10 comment), verbatim.
|
|
1146
|
+
|
|
1147
|
+
Trust contract row, exit mapping (decision 1):
|
|
1148
|
+
|
|
1149
|
+
> For a given `tool_use_id`, the S2b origin provider treats "`PostToolUse` received, `interrupted` is false, and no `PostToolUseFailure` arrived" as exit `0`. This is a named mapping of the host's own success/failure classification, not a null→0 conversion.
|
|
1150
|
+
|
|
1151
|
+
Bound: "A `PostToolUseFailure` is never credited. Its `Exit code N` text is used for diagnostics only."
|
|
1152
|
+
|
|
1153
|
+
Reviewer principal and claim boundary (decision 2, bound column):
|
|
1154
|
+
|
|
1155
|
+
> The claim is bounded. The agent can run arbitrary code, so it can forge local journals. The guarantee is that host payload fields are re-validated at assessment time, and copied-record replay tests are required negatives. **Cryptographic unforgeability is not claimed.** Human approval is not available as a reviewer principal.
|
|
1156
|
+
|
|
1157
|
+
Supported host for positive credit: **Claude Code only.** OpenCode and Codex stay unavailable.
|
|
1158
|
+
|
|
1159
|
+
### Result contract
|
|
1160
|
+
|
|
1161
|
+
Logical command IDs: `harness.doc-check.check.v1` and `harness.doc-check.compare-derived.v1`. An ID is bound to the observed executable and argv, the resolved installed runtime path and an implementation digest. Stdout that repeats the ID is not trusted.
|
|
1162
|
+
|
|
1163
|
+
Only direct, single, registered Node invocations are credited. Compound shell commands, aliases and wrappers stay uncredited unless their expansion is validated independently. The script must resolve (realpath, relative to the captured cwd) to this install's own `harness-everything/scripts/doc-check.js` (source) or `skills/harness-everything/scripts/doc-check.js` (packaged). Its digest must equal the digest observed at capture; any other path or digest earns no credit, no capability event and no verification evidence.
|
|
1164
|
+
|
|
1165
|
+
Candidate classification follows command roles, not arbitrary text matches. A direct `doc-check`/`doc-check.js` target, a checker script operand to a recognized launcher, or a checker command in a recognized shell command-string remains a candidate when its launcher, subcommand, flags, quoting or composition is unsupported; it is refused as `unregistered-command` and cannot fall through to generic verification. An unrelated command whose data argument mentions `doc-check`—including workflow-plan JSON, `echo` text or a supplied filename—is not a candidate. Candidate detection never authorizes a wrapper, and opaque aliases or dynamically constructed targets remain unsupported.
|
|
1166
|
+
|
|
1167
|
+
| Capability | Credit needs |
|
|
1168
|
+
|---|---|
|
|
1169
|
+
| `doc-lint` | One projection from an observed successful `check`: lint exit 0, valid schema, the exact expected governed and baseline paths, and complete relevant coverage |
|
|
1170
|
+
| `link-check` | A separate projection of the same invocation, scoped to the expected paths: no broken, skipped, unreadable or unsupported link syntax |
|
|
1171
|
+
| `derived-regenerate` | An observed successful generation **and** an observed successful pure comparison, bound to the same source and config identity and to the same generated bytes. `--generation-status ok` is caller-reported and cannot establish generation alone |
|
|
1172
|
+
| `quality-review` | A completed review from a correlated host `agent_id` distinct from the author session and every executor. Executors are the author session and the principals of registered doc-check runs in the task, which are the execution evidence a doc obligation uses (see the rationale below). An Agent/Task result must name the same `agent_id`. The verdict needs an explicit approve line, a non-empty summary and one `Reviewed artifact: <path> sha256:<hex>` line per reviewed path, which must match the candidate. SubagentStop alone is never approval |
|
|
1173
|
+
| `conflict-check`, `requirement-sync` | Stay unresolved. F4 gives them no credit |
|
|
1174
|
+
| SHOULD reminders | Never earn credit, and their absence never withholds it |
|
|
1175
|
+
|
|
1176
|
+
### Live capture gate
|
|
1177
|
+
|
|
1178
|
+
Positive production credit stays **disabled** until a live capture validates. Procedure (A3 §4):
|
|
1179
|
+
|
|
1180
|
+
1. Pin the capture implementation. Stage the Claude plugin outside the checkout with `scripts/stage-claude-plugin.js`. Use a small scratch Git project with one governed reference and a committed baseline. Keep Harness state and output outside that project.
|
|
1181
|
+
2. Install a capture-only logging wrapper through a session `--settings` file. For PreToolUse, PostToolUse, PostToolUseFailure, SubagentStart and SubagentStop, write the raw stdin payload to owned JSONL before running the staged hook. Record the settings and wrapper hashes.
|
|
1182
|
+
3. Set `HARNESS_DOC_RECONCILIATION_PILOT=1` and a scratch `HARNESS_STATE_HOME`. Use a fresh session UUID, a foreground read-only `doc-reviewer` agent and normal scoped permissions. Do not add a bypass flag. If the required tools are denied or authentication fails, record the gate as unmet.
|
|
1183
|
+
4. Run the command below. After the edit and before delegation, the parent uses one read-only `shasum -a 256 docs/a.md` command and passes the relative path and actual post-edit digest to the independent reviewer. Require a plain `Review verdict:` line followed by `approve` or `reject`, a `Reviewed artifact: docs/a.md sha256:<digest>` line, and a non-empty summary. Do not pre-fill the verdict or edit between hashing and review. The prompt asks Claude to read the reference, run the registered `doc-check` command, invoke `doc-reviewer`, wait for its completion and stop.
|
|
1184
|
+
5. Validate a populated `tool_use_id`, an explicit interruption value, and the reviewer's `agent_id` and `agent_type`, with parent-session, start, stop and Agent-result correlation. A missing field keeps the gate unmet.
|
|
1185
|
+
|
|
1186
|
+
```text
|
|
1187
|
+
HARNESS_DOC_RECONCILIATION_PILOT=1 HARNESS_STATE_HOME="<scratch-state>" \
|
|
1188
|
+
claude -p "<fixed capture prompt, containing the exact check command>" \
|
|
1189
|
+
--plugin-dir "<staged-claude-plugin>" \
|
|
1190
|
+
--settings "<scratch-capture-settings.json>" \
|
|
1191
|
+
--agents '{"doc-reviewer":{"description":"Read-only documentation reviewer","prompt":"Read only. Return explicit verdict, summary and reviewed artifact digests; never edit or execute the implementation."}}' \
|
|
1192
|
+
--session-id "<fresh-UUID>" --setting-sources project,local \
|
|
1193
|
+
--output-format stream-json --verbose
|
|
1194
|
+
```
|
|
1195
|
+
|
|
1196
|
+
**Retained artifacts** go to `benchmarks/results/live-host/claude-325-3b-s2b/`: `capture.sanitized.jsonl`, `README.md`, the relevant stream-json transcript, settings, manifest and wrapper digests, the host version, the runtime and tree fingerprint, and the command and exit summary. Raw payloads stay private, outside the repository. Transcript text alone is not hook evidence.
|
|
1197
|
+
|
|
1198
|
+
**Qualification export** is a sanitized file at the canonical path `hooks/scripts/lib/doc-reconciliation-qualification.json`, copied by plugin sync, with a byte-identical copy of its sanitized capture beside it (`doc-reconciliation-capture.sanitized.jsonl`). The runtime validates schema, host version, payload kinds, correlation and the capture digest against that sibling file. It does not search ancestor directories and does not accept a file by size.
|
|
1199
|
+
|
|
1200
|
+
### Scope and limits
|
|
1201
|
+
|
|
1202
|
+
- The first positive scope is a body-only edit to an existing governed reference, with no derived declaration, code, test, PRD or ADR normative change. Its applicable MUSTs are lint and links.
|
|
1203
|
+
- Simulated replay and one live field capture are not E1 GO, and they do not permit a wider rollout.
|
|
1204
|
+
- Not claimed: cryptographic attestation, a sandbox, a forced review provider, default-ON, live-host behaviour beyond the capture, or support for OpenCode or Codex.
|
|
1205
|
+
|
|
1206
|
+
**Executor rationale.** The owner's text (#325, 2026-10-21) says the reviewer "must differ from the author's `session_id` and from every executor principal". An executor principal is the principal of execution evidence used in this assessment. A reviewer that read or computed digests with plain tool calls is not an executor. A reviewer whose registered doc-check run is refused still counts, which keeps the rule fail-closed.
|
|
1207
|
+
|
|
1208
|
+
### Known limits (documented, no code change)
|
|
1209
|
+
|
|
1210
|
+
- **OFF is not the F3 wiring (F8).** Pilot-off writes are unchanged, but the capture hooks mount on Bash, PowerShell, Task and Agent PreToolUse, so each of those runs one extra Node process (about 25 ms, measured). "OFF = legacy behaviour" holds for writes, not for the hook set.
|
|
1211
|
+
- **The live gate is a repository-pinned field-availability check, not a running-host compatibility check (F10).** The qualification export records Claude Code `2.1.308`, but the retained Claude hook payloads expose no host-version field for the runtime to compare. The export and sibling sanitized capture validate the recorded payload kinds and correlations for that host version; they do not establish compatibility with the current session. Editing both repository artifacts and updating the capture digest can reopen the gate without a new live capture.
|
|
1212
|
+
- **The unsupported-host check is vacuous in production (F11).** The only production caller asserts the Claude host id. OpenCode and Codex stay unavailable because their plugins do not mount `state-persist.js`, not because the host is detected.
|
|
1213
|
+
- **Event files are not tamper-proof (F13).** Event files are created exclusively, and the journal is checked against them. The local agent can still rewrite event files and rebuild the journal. This is the owner's bounded claim, not a guarantee of tamper resistance.
|
|
@@ -129,6 +129,7 @@ Globs are repo-relative POSIX paths supporting `*` (one segment) and `**` (any d
|
|
|
129
129
|
- Frontmatter fields: `id`, `role`, `status`, `supersedes`, `aliases`. `superseded_by` and `amends` are not stored; `superseded_by` is derived in the index.
|
|
130
130
|
- **Ratchet:** schema checks (role/status/ID) apply to governed files named in `changedPaths`. Full mode is explicit (`--all` / `{ all: true }`); the CLI refuses to run without `--changed`, `--changed-path` or `--all`. Cross-file integrity diagnostics are errors when any involved file is changed (deleted and renamed paths count as changed), warnings otherwise. Warnings that affect resolution still appear in the index's unresolved list. CLI `--changed` entries resolve against `--workspace`, normalize to repo-relative POSIX paths, and fail as usage errors when outside the workspace; `--changed` takes one comma-separated argument and cannot represent filenames containing commas, so repeat `--changed-path <path>` (one occurrence per path, never comma-split, normalizes and resolves like the CSV form, and mutually exclusive with `--all`; may be combined with `--changed`) when a filename contains a comma. Occurrences of either changed selector accumulate: repeated `--changed` or `--changed-path` flags add to the changed set instead of overwriting earlier declarations. The CSV form trims whitespace around its comma-separated items; `--changed-path` preserves commas and outer whitespace while resolving paths and normalizing separators. Both `/` and `\` are separators on every host; literal backslashes in POSIX filenames cannot be selected through these CLI flags — use `--all` or the API `changedPaths` for those files. Relative `--config` paths also resolve from `--workspace`. Without a trusted base, escalation for deleted or renamed targets covers explicit `legacy:ID@path` selectors only; ID-only references escalate when a trusted base (`--base` or a supplied inventory, see D6.9) maps the ID to a changed path.
|
|
131
131
|
- Parsing conventions: role comes from the first matching `governed` entry; frontmatter is the leading `---` block in a YAML subset (scalars, inline `[a, b]` lists, `- a` block lists); references inside fenced code blocks are ignored.
|
|
132
|
+
- Bare `<X>-NNN` tokens (no `REQ-`, `PRD-` or `legacy:` prefix) are references only when `X` is in `domains` or the exact token is an active canonical ID, alias or legacy ID. Other bare tokens such as `SHA-256` are not references.
|
|
132
133
|
|
|
133
134
|
## IDs, aliases and references (D6.2, D6.8)
|
|
134
135
|
|
|
@@ -310,10 +311,20 @@ Decision extraction recognizes ATX boundaries with 0–3 leading spaces and igno
|
|
|
310
311
|
- Checks are pure and generators deterministic, so reruns are idempotent.
|
|
311
312
|
- `requirement-sync` is the only genuinely new rule; the rest reuse D5 and D6.3 checks.
|
|
312
313
|
|
|
314
|
+
## Derived-view lane (F3)
|
|
315
|
+
|
|
316
|
+
- Applicability is conservative (`conservative-all-governed`): a change to any governed or legacy source path marks every declared view as affected; a changed derived output marks every declared view (not only that output); a config change marks all candidate declarations. This is not exact dependency inference.
|
|
317
|
+
- The `sources` seam in `doc-obligations.js` is an implementation detail, not a public schema. F3 adds no source mapping; a view with no mapping is covered by the conservative rule.
|
|
318
|
+
- `check` never runs a generator. A view it lists as affected needs an authorized generation step and a `compare-derived` record; see `harness-everything/references/doc-check.md` § Derived views.
|
|
319
|
+
|
|
320
|
+
F4 is the creditable bounded consumer; its first positive scope is lint and links on a body-only reference edit (see `docs/doc-workflow-reconciliation.md`, F4 section).
|
|
321
|
+
|
|
313
322
|
## Packaging
|
|
314
323
|
|
|
315
324
|
Source checkout, Codex plugin and Claude plugin copy whole skill trees. Callers locate `doc-lifecycle.js` and the resolver relative to their own skill directory, never the source-tree root. A to-spec-only install must use the same resolver implementation (copied byte-identically and checked), and a broken shared module is an error, never a silent fallback to older host scanning.
|
|
316
325
|
|
|
326
|
+
The shared writing craft and maintenance references live in `harness-everything/references/documentation/_shared/` and ship with the plugin package, so every doc-producing skill links to one copy rather than its own.
|
|
327
|
+
|
|
317
328
|
## Generated index
|
|
318
329
|
|
|
319
330
|
`buildIndex` returns Markdown with no timestamps or environment data, so two runs over the same tree are byte-identical. Sections: active documents (sorted by ID then path, with derived `superseded by`), legacy documents (sorted by ID then path, with projected status), and the unresolved list (resolution-affecting error diagnostics).
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# Document lifecycle index
|
|
2
|
+
|
|
3
|
+
## Active documents
|
|
4
|
+
|
|
5
|
+
- `(no id)` — `docs/architecture.md` (role: reference; status: active)
|
|
6
|
+
- `(no id)` — `docs/doc-workflow-reconciliation.md` (role: reference; status: active)
|
|
7
|
+
|
|
8
|
+
## PRD revision chains
|
|
9
|
+
|
|
10
|
+
_None._
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
## Legacy documents
|
|
14
|
+
|
|
15
|
+
_None._
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
## Unresolved
|
|
19
|
+
|
|
20
|
+
_None._
|
|
@@ -32,7 +32,7 @@ This is a **repository/agent contract**, not an automatic upgrade to any platfor
|
|
|
32
32
|
| Codex | `Supported` | `Mechanism verified` | `Partial`: user-hook compatibility capture is live-verified for `UserPromptSubmit` and `Stop` on one Windows Codex CLI host; native plugin hooks remain mechanism-verified | `.agents/skills/`, `AGENTS.md` | `~/.agents/skills/` | [Live evidence](../benchmarks/results/live-host/codex-2026-10-13/README.md) verifies two consecutive turns on Codex CLI 0.162.1; native plugin mounting, other lifecycle events, and other versions remain unverified. |
|
|
33
33
|
| ChatGPT / OpenAI Plugin | `Unknown` for a repository-local standalone path | `Mechanism verified` | `Partial`: local Work/Codex hooks are separate from ordinary Chat | Managed workspace marketplace | Not established | The public Skills-only bundle is mechanically reproducible; import, approval, and live behavior remain `Unknown`. |
|
|
34
34
|
| Claude Code | `Supported` | `Supported` | `Mechanism verified` for Harness configuration | `.claude/skills/` | `~/.claude/skills/` | Native Claude surfaces and the state-specific plugin sync command are mechanically checked; fresh-host execution is not preserved here. |
|
|
35
|
-
| OpenCode | `Supported` | `Partial` live-host loading for the project-scope, `.js`-filename install path on OpenCode 1.18.31 (macOS); other install paths remain `Mechanism verified` | `Partial` live-host evidence for project-scope `.js` loading and edit/verification state; hard lock reported only, with no retained blocked-tool trace and a post-reset final snapshot | `.opencode/skills/`, `.agents/skills/`, `.opencode/plugins/` | `~/.config/opencode/skills/`, `~/.agents/skills/`, `~/.config/opencode/plugins/` | [Retained evidence](../benchmarks/results/live-host/opencode-2026-09-16/README.md): loading and state effects verified in one host session; final snapshot post-reset; hard lock only an interactive observation; reflection operator-seeded, then agent-rewritten; no raw transcript or behavioral-effectiveness claim. `.mjs` auto-discovery broken (issue #127); global scope, npm-package installation, and other host versions remain unverified. |
|
|
35
|
+
| OpenCode | `Supported` | `Partial` live-host loading for the project-scope, `.js`-filename install path on OpenCode 1.18.31 (macOS); other install paths remain `Mechanism verified` | `Partial` live-host evidence for project-scope `.js` loading and edit/verification state; hard lock reported only, with no retained blocked-tool trace and a post-reset final snapshot | `.opencode/skills/`, `.agents/skills/`, `.opencode/plugins/` | `~/.config/opencode/skills/`, `~/.agents/skills/`, `~/.config/opencode/plugins/` | [Retained evidence](../benchmarks/results/live-host/opencode-2026-09-16/README.md): loading and state effects verified in one host session; final snapshot post-reset; hard lock only an interactive observation; reflection operator-seeded, then agent-rewritten; no raw transcript or behavioral-effectiveness claim. `.mjs` auto-discovery broken (issue #127); global scope, npm-package installation, and other host versions remain unverified; V2 default-export load verified live on OpenCode 2.1.3 (macOS, 2026-10-23, server-log evidence only). |
|
|
36
36
|
| GitHub Copilot agent surfaces | `Supported` | `Unknown` for a Harness-specific plugin install | `Unknown` | `.github/skills/`, `.agents/skills/`, repository instructions | `~/.copilot/skills/`, `~/.agents/skills/` | GitHub Agent Skills paths and installer behavior are checked; no live Copilot session or plugin install is preserved. |
|
|
37
37
|
| Cursor | `Supported` | `Mechanism verified` for the portable package shape | `Unknown` for Harness | `.cursor/skills/`, `.agents/skills/` | `~/.cursor/skills/`, `~/.agents/skills/` | Official paths and package shape are checked; Cursor plugin loading and hooks remain unverified. |
|
|
38
38
|
| Continue.dev | `Unknown` for standalone `SKILL.md` discovery | `Unknown` | `Unknown` | `.continue/rules/` is documented; `.continue/skills/` is only an installer candidate | `~/.continue/skills/` is only an installer candidate | The reviewed official docs establish rules, not an Agent Skills host contract. |
|
|
@@ -132,7 +132,21 @@ The general installer merges shared files only through Harness-owned markers and
|
|
|
132
132
|
|
|
133
133
|
## OpenCode boundary
|
|
134
134
|
|
|
135
|
-
The OpenCode adapter
|
|
135
|
+
The OpenCode adapter targets the V2 plugin contract: it default-exports
|
|
136
|
+
`{ id: "harness.enforcement", setup(ctx) }` that registers
|
|
137
|
+
`ctx.tool.hook("execute.before|after")` and `ctx.event.subscribe()` through the
|
|
138
|
+
OpenCode 2 plugin context, and keeps the V1-shaped `HarnessEnforcement` factory
|
|
139
|
+
for OpenCode 1.x hosts. OpenCode 2 rejects named-export-only plugins with
|
|
140
|
+
`Plugin must export a default definition with an id and an effect or setup
|
|
141
|
+
function`; that failure (observed on OpenCode 2.1.3, macOS) and the recovery
|
|
142
|
+
after the V2 entrypoint were verified live on one host, with retained evidence
|
|
143
|
+
in the OpenCode server log only (loading succeeded after a plugin-file
|
|
144
|
+
reload; no hook-fire artifact). The adapter itself has deterministic mechanism
|
|
145
|
+
coverage (`ci/mechanism-2n-opencode-plugin.test.js`, V1 shapes;
|
|
146
|
+
`ci/mechanism-2p-opencode-v2-entrypoint.test.js`, V2 shapes). Retained
|
|
147
|
+
evidence supports project-scope `.js` loading and edit/verification state on
|
|
148
|
+
OpenCode 1.18.31 (macOS): [live-host evidence](../benchmarks/results/live-host/opencode-2026-09-16/README.md). The
|
|
149
|
+
retained snapshot predates #190 and remains historical evidence only. Current OpenCode removes the permanent post-reflection hard lock; only the third-failure reflection boundary pauses edits. Reflection was operator-seeded, then agent-rewritten, so do not promote this into a behavioral-effectiveness claim. `.mjs` auto-discovery is broken on this host version (issue #127). npm-package installation and behavioral V2-hook fire remain unverified.
|
|
136
150
|
|
|
137
151
|
## Official sources
|
|
138
152
|
|
|
@@ -158,3 +172,9 @@ ordinary terminal sees it only when set at user/OS level. Keep the mode `off`
|
|
|
158
172
|
until a Harness-trained checkpoint passes the rollout gates.
|
|
159
173
|
Public Skills-only artifacts do not acquire lifecycle hooks from this feature.
|
|
160
174
|
See [System One routing](system-one-routing.md) for limits, rollback and evidence.
|
|
175
|
+
|
|
176
|
+
The editorial doc check (`harness-everything/scripts/doc-check.js`, F1) is reached by instruction only: the `SKILL.md` deep-dive line leads to `references/triage-and-tiers.md`, whose governed-doc section names `references/doc-check.md`, which holds the command. No `doc-change` step binding exists. Evidence is package and mechanism only. It adds no host enforcement, no Stop lock and no trusted document credit, and a non-empty `coverage.unsupported` marks the check as partial.
|
|
177
|
+
|
|
178
|
+
`compare-derived` (`harness-everything/scripts/doc-check.js`, F3) compares a caller-captured generator output with a declared derived view, byte for byte. It runs no generator and writes no files. Its evidence is package and mechanism only: the generation status is caller-reported, and the output is ordinary command evidence. It is not trusted credit, and it makes no live-host claim.
|
|
179
|
+
|
|
180
|
+
Credited reconciliation for F4 (Claude Code only, pilot ON) is **implemented, with mechanism and package evidence and one live field-availability capture**. Credit works for `doc-lint` and `link-check` through a registered `doc-check` run, and for an independent `quality-review` with bound path digests. The capture under `benchmarks/results/live-host/claude-325-3b-s2b/` proves field availability on Claude Code 2.1.308 for one session. It is not live-host credit, and no live-host credit run exists yet. Deterministic mechanism tests (replay through the real hook entry points, with a CI-only qualification preloader) prove the chain.
|
package/grill-with-docs/SKILL.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# Grill With Docs — Full Session Playbook
|
|
2
2
|
|
|
3
|
+
Before drafting, read `<skills-repo-root>/harness-everything/references/documentation/_shared/writing-craft.md`; for governed-doc events, follow `<skills-repo-root>/harness-everything/references/documentation/_shared/maintenance.md`.
|
|
4
|
+
`<skills-repo-root>` is the directory that holds the sibling skill directories: the source checkout root, or `<plugin-root>/skills` when packaged.
|
|
5
|
+
If the harness-everything skill is not installed, say the shared reference is unavailable.
|
|
6
|
+
|
|
3
7
|
## Decision Flow
|
|
4
8
|
|
|
5
9
|
```mermaid
|