skill-harness 0.14.0 → 0.16.0

package/docs/PRODUCT-GUIDE.md

@@ -0,0 +1,133 @@
+# Factory: work, review, learn
+
+**Product guide for the 0.16.0 release candidate**, paired with pi-daddy **0.27.0**;
+Principal **3.2.0** is unchanged. The prior audited baseline was harness 0.15.0 /
+producer 0.26.1. Harness PR80 is merged and independently approved with green CI;
+source/built cross-package checks passed with synthetic fixtures, not human evidence.
+Canonical release packaging, publication and final installed-flow verification remain
+separate pending gates. After installation, expect `skill-harness --version` **0.16.0**
+and producer package **0.27.0**; check `skill-harness learning help` and loaded Pi resources.
+Do not assume an existing session reloaded because a package was updated. Use a **fresh
+Pi session** when upgrading an already-published immutable dashboard bridge: `/reload`
+cannot replace the old global API. No other running session needs to be stopped.
+
+## Who does what?
+
+**pi-daddy** coordinates permitted work and owns execution, steering and the future-order
+registry. **Herdr** displays its sessions and side panel. **skill-harness** retains
+measurement and learning evidence. The panel is not an autonomous worker. Finished
+execution is not human acceptance.
+
+Use the producer's `/grants work` guided setup for multiple obligations, dependencies,
+priority and explicit agent/model/thinking selections, then its supported run/host controls.
+For existing single-task use, `pi-daddy work add --id report --outcome "Review the report"`
+remains supported. Only listed controls are available. Pausing new dispatch does not
+cancel running work; changing a next-order policy does not mutate active sessions.
+Principal's ordinary current/stale/superseded reports remain unchanged.
+
+## Open learning in Pi
+
+With both matching extensions loaded, use **`/grants learning`** from the current host.
+The producer binds the exact current snapshot, archive, population and local review author.
+A changed scope must report a mismatch, not silently reuse an old learning workspace.
+
+The harness also exposes **`/skill-harness learning --state "WORKSPACE"`** (use the
+workspace path displayed by the host), or `skill-harness learning --state "WORKSPACE"`
+in a terminal. These open the **same guided flow and durable records**, not another app.
+
+1. **Readiness** explains what's retained, what's missing, and why automatic questions
+   are silent. Opening a review is deliberate: it neither reserves attention nor
+   represents an earned automatic question. It starts no model or delegation.
+2. **Connect retained input** lists compatible case batches and qualified comparisons
+   already in the selected archive. Pick one and give it a short name and title.
+   No hand-written manifests or ledger JSON. No compatible evidence is an honest empty state.
+3. **Review cases** shows nominations and lets you open their actual retained evidence
+   sources by selection (CLI: `learning evidence BATCH --item N --evidence N`). Redacted
+   or unavailable sources are marked, never called complete evidence. Record confirmed defect, expected
+   behavior, exemplar, uncertain or skip, with a reason. Corrections are append-only.
+   Confirming a case is not promoting a test or establishing a cause.
+4. **Review comparisons** opens complete retained artifacts under A/B labels. Acknowledge
+   each complete artifact only after reviewing it. Then choose one, tie, none acceptable,
+   or insufficient evidence. **Reveal model/cost** is a separate action, enabled only after
+   the durable quality receipt. Artifact contents themselves may disclose identity.
+5. **Record adopt / reject / defer** records a separate scope-bound decision. Reject and defer
+   are normal outcomes; neither forces a preference or reveal. Missing evidence can be
+   durably deferred. Adopt is intent, not activation.
+6. **Propose hypothesis** records prediction, disproof, downside, rollback and a competing
+   explanation from an actual case. **Link original hypothesis** only accepts the comparison's
+   actual frozen origin, never a retrospective replacement.
+
+Previously retained excerpt feedback concerned
+**excerpts**. Keep that receipt unchanged and linked as prior feedback. It is not full-artifact
+acceptance, detector calibration or a fresh blind experiment. An old unscoped quality choice
+also does not automatically become a full-artifact acknowledgement.
+
+## Configure trust without writing manifests
+
+Open **Trust / independent labels** in the same guided flow:
+
+- Before freezing, explicitly add any real unflagged incident evidence and select its split.
+  Only selected evidence files are retained; do not include private sessions or secrets.
+- Select a retained case batch, detector and evaluation split. The product derives the
+  cohort and predictions from actual nominations and groups repeated target observations
+  into one incident. Detector version and population remain exact.
+- Automatic questions default to **silent**. To configure them, explicitly select minimum
+  resolved independent incidents, the confidence lower bound, attention budget and expiry.
+  The form previews the exact frozen policy. It creates **no independent labels**.
+- Review cases independently, then choose **Link current case label**. Unflagged sample
+  labels have their own explicit evidence/author step. Preference, model self-report,
+  panel agreement and exit zero are not independent correctness labels.
+
+Missing calibration, missing unflagged samples, held-out absence, conflicting labels,
+expired policy and exhausted attention stay visible. Reopening neither resets choices nor
+refills attention. Correcting a linked case withdraws its old label from confidence.
+The original producer closing path still requires presence, quiescence and its exposure gate.
+
+## Scoped adoption and later work
+
+Use **`/grants learning` in interactive Pi** for activation/rollback. The matching producer
+implements two separate profiles: unchanged **`fixed-policy-v1`**, and
+**`ordinary-work-policy-v1`** for model/effort-only changes to the same selected task IDs and
+agent definitions. The latter does not change instructions, skills, grants, capabilities,
+topology or assessment policy; do not disguise those changes as model/effort or fixed policy.
+The producer prepares policy bytes and owns independent authority and the original registry.
+Source/built integration has been checked with synthetic fixtures; final installed-release
+verification remains pending. Bare legacy fixed-policy views lacking applied activation
+proof defer; they must not be treated as activation receipts.
+
+Activation needs the exact selected candidate, original confirmed case/hypothesis/comparison,
+current independently verified eligible facts, explicit adoption authority, and registry
+revision consent. The original registry revalidates these again. Only its actual activation
+receipt advances the learning record to activated. New ordinary-work orders pin the policy
+under fresh eligibility confirmation; active or history-bearing runs never migrate.
+Rollback similarly needs its own authorized request and original applied registry receipt.
+A stored lifecycle link, human preference or prepared receipt cannot substitute for either.
+
+**Adoption / outcomes** can inspect linked later observations or record one using complete
+artifact, original/current requirement and independent evidence files. Unknown is valid.
+Acceptance requires both the exact accepted artifact and independent acceptance evidence.
+Changed requirements, changed artifact, out-of-scope, success, caught defect and escape remain
+distinct. No outcome triggers automatic rollback, and no single outcome proves improvement.
+
+## CLI and local archive use
+
+```sh
+skill-harness learning status --state "WORKSPACE"       # human readiness
+skill-harness learning status --state "WORKSPACE" --json # integration/details
+skill-harness learning review reports --state "WORKSPACE"
+skill-harness learning help                             # all explicit operations
+skill-harness learning guide                            # this guide, packaged offline
+skill-harness learning current                          # packaged requirement register
+```
+
+Without a producer host, `skill-harness learning` guides connection to an explicitly selected
+archive and selecting an actual retained case scope. Or use
+`learning init --archive DIR --author NAME --confirm`; multiple retained scopes ask you to
+select `--scope-item N`. Snapshot digest and population are derived from actual cases, not
+typed hashes. For a comparison-only archive with no cases, `--scope TEXT --population TEXT`
+can explicitly declare a local review scope; it cannot bind mismatching case snapshots.
+Use the producer flow to connect a host. Noninteractive writes require explicit `--confirm`.
+All learning commands are offline/no-model. Command completion is never a quality verdict.
+Local author attribution and content hashes are not authentication or hostile-owner attestation.
+
+Current requirements and remaining evidence: [STATUS.md](STATUS.md).

package/docs/STATUS.md

@@ -0,0 +1,45 @@
+# Factory requirement register — 0.16.0 release preparation
+
+Current entry point: [PRODUCT-GUIDE.md](PRODUCT-GUIDE.md). The dated
+[CURRENT.md](CURRENT.md), build reports and transport decisions are retained history,
+not current instructions to supply manifests or judge engineering repairs.
+
+## Baseline and candidate
+
+Release pair: **skill-harness 0.16.0 / pi-daddy 0.27.0**; Principal 3.2.0 is unchanged.
+The prior audited baseline was harness 0.15.0 (`a725579`) / producer 0.26.1 (`93a9b01`).
+Harness PR80 merged at `c6cd55d4db9ccf9a1ec790459c3cc5e6b1eac69f` with green CI and
+Sol approval. Actual source/built cross-package learning/registry checks passed with
+synthetic fixtures. This is **release preparation**, not publication, final installed
+proof, human approval, calibration or measured improvement. Loaded resources, source
+HEAD, canonical archives, published bytes and active sessions remain separate identities.
+Canonical packing, publication and final installed verification are pending.
+
+| Requirement | Current implementation / boundary | Remaining evidence |
+|---|---|---|
+| Understandable side panel | Producer owns selected-outcome-first status, observed agents, available controls and Details. | Matching producer's installed narrow-pane/host checks. No invented progress/ETA/acceptance. |
+| Ordinary multi-obligation work | Producer owns work setup, dependencies, bounded parallel/sequential execution, model/effort choice and revision consent. | Installed checks of the matching producer's supported profiles. Existing Principal workflows remain unchanged. |
+| Retained cases/comparisons | Harness learning workspace composes existing archive, case-review, blind-choice and CAS journal primitives. Named bindings and guided CLI/Pi entrypoints; missing/stale/mismatched inputs defer. | Final installed producer connection; source review/CI and synthetic source/built linkage passed. No new model evidence claimed. |
+| Complete quality review | Complete-output acknowledgement, durable choice before reveal, immutable replay, separate adopt/reject/defer. Prior excerpt feedback stays separate. | A real full-artifact product choice remains absent where not actually recorded. Excerpt preference is not acceptance. |
+| Scope-bound trust | Guided frozen cohort/policy, actual nomination predictions, incident grouping, explicit independent case/unflagged labels and pure exposure preview. Existing automatic gates unchanged. | Real independent labels, unflagged coverage and held-out evidence where absent. Implementation cannot manufacture them. |
+| Hypothesis and comparison linkage | Human-authored falsifiable hypothesis form, confirmed-case linkage, monotonic optional context and retained lifecycle navigation. | Ongoing weekly model-backed retro is not demonstrated or automatically scheduled. Existing bounded weekly/model primitives remain separate authorized paths. |
+| Adoption and rollback | Harness validates scope/choice/current independent authority and eligible facts. Producer source supports unchanged `fixed-policy-v1` plus separate `ordinary-work-policy-v1` (model/effort only, same task IDs/agent definitions and scope/assessment). Producer owns activation/rollback and fresh-eligibility next-order pins; profile-agnostic harness linkage retains original receipts. | Synthetic source/built original activation/rollback linkage passed. Configured production adoption/rollback and final installed integration remain unverified; bare legacy views without applied activation proof defer. No active/history-bearing run migration or instruction/skill/grant/capability/topology/assessment expansion. |
+| Later outcomes | Explicit file-based CLI/Pi observation forms plus producer API linkage; independent reference attribution; changed requirements/artifacts/out-of-scope/unknown remain distinct. | Genuine later work, exact acceptance and held-out efficacy where absent. No automatic rollback or universal quality score. |
+| Packaging / portability | Public adapter exports, additive frozen dashboard bridge, CLI export and bundled Pi entrypoint; guide/register shipped offline via `learning guide` / `learning current`. A pre-existing immutable bridge needs a fresh Pi session. | Release-metadata review/CI, exact merged canonical pack with retained archives, empty-prefix installed Pi/registry verification, then authorized publication. |
+
+## Historical claims remain bounded
+
+- The failure-derived abort-race case and the public-weather comparison are different chains.
+- Previously retained feedback covered excerpts only. It does not advance full comparison
+  acceptance or detector trust.
+- Earlier no-tool measured SDK observations do not establish universal ordinary-child Usage,
+  writable-effect containment or actual provider-internal reasoning effort.
+- No OS sandbox or malicious-local-owner protection has been added.
+- No historical session/evidence, auth or unrelated worktree was modified in this candidate.
+
+Feature source review/CI and source/built cross-package checks are completed evidence,
+not final released-install proof. Metadata preparation runs consistency/diff validation only:
+no builds, installs, suites or model calls. Canonical pack and actual installed two-extension
+verification remain release gates under `PUBLISHING.md`. Prior live route smokes remain
+historical and are not replayed merely for a version bump. Human acceptance, calibration and
+later efficacy still require their own real evidence.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "skill-harness",
-  "version": "0.14.0",
+  "version": "0.16.0",
   "description": "Test/optimize loop for agent skills — run spec'd scenarios on pi, LLM-judge, score, review, re-run",
   "type": "module",
   "keywords": [
@@ -18,7 +18,9 @@
     "skill-harness": "bin.js"
   },
   "pi": {
-    "extensions": ["./dist/index.js"]
+    "extensions": [
+      "./dist/index.js"
+    ]
   },
   "files": [
     "bin.js",
@@ -26,11 +28,13 @@
     "dist/prompt-capture-extension.js",
     "assets/report.template.html",
     "assets/report.grade.js",
+    "docs/PRODUCT-GUIDE.md",
+    "docs/STATUS.md",
     "LICENSE",
     "README.md"
   ],
   "dependencies": {
-    "@skill-harness/cli": "0.14.0",
+    "@skill-harness/cli": "0.16.0",
     "typebox": "^1.1.38"
   },
   "repository": {