uni-harness 0.1.0

package/.claude/skills/ratchet/SKILL.md

@@ -0,0 +1,72 @@
+---
+name: ratchet
+description: Convert failures into permanent structure (rules, sensors, guards, permissions) instead of one-off corrections. With an argument, handles that one mistake end-to-end (/ratchet [what happened]); without arguments, diagnoses recurring failure patterns from the harness logs. Proposes only — applying always requires user approval.
+---
+
+# Ratchet — Turning Failures into Structure
+
+Goal: the same failure never happens twice. Always put the fix in the
+**strongest layer** (weak → strong: in-conversation correction < CLAUDE.md
+rule < automated sensor < guard/permissions block). **Never apply any
+change without user approval.**
+
+## Mode A — Single Incident (`/ratchet [what happened]`)
+
+Handle one mistake end-to-end:
+
+1. **Reproduce** — reproduce the failure with the same input/conditions.
+   If it doesn't reproduce, report that; never create a rule from a guess.
+2. **Classify** — use the failure-class table below.
+3. **Pick the layer** — choose the single strongest fix layer for that class.
+4. **Draft the fix** — produce a concrete diff/pattern per "Proposal format".
+5. **Verify** — (after user approval and applying) re-run the original
+   failing input from step 1 and confirm it is now blocked or caught.
+6. **Regression check** — run the full TEST command to confirm nothing
+   else broke.
+
+## Mode B — Log Diagnosis (`/ratchet` with no arguments)
+
+1. **Observe** — collect this (or the most recent) session's records from
+   `.harness/logs/tool_calls.jsonl`. Focus on `PostToolUseFailure` entries,
+   repeated edits to the same file, and repeated runs of the same command.
+   If there are no logs, tell the user and stop.
+2. **Diagnose** — classify failures by **failure class**, not symptom.
+   Each diagnosis gets "one-sentence root cause + the log entries behind it".
+3. **Write** — draft one fix per class, in its strongest layer.
+
+## Failure Class → Fix Layer
+
+| Failure class | Signal | Strongest fix layer |
+|---|---|---|
+| Known bad pattern recurring | same type of code error again | linter rule (sensor) > guide rule |
+| Missing context | worked without knowing project conventions | CLAUDE.md guide entry |
+| Wrong tool use | dangerous/inappropriate command attempted | guard-pre-bash.sh pattern or permissions |
+| Quality drift | declared done without verification | strengthen sensor (TEST_CMD scope) |
+| Lost state | repeated already-completed steps | strengthen checkpoint protocol |
+| Runaway/cost blowup | tool-call surge, same-error loop | adjust tripwire thresholds |
+
+## Proposal Format
+
+- CLAUDE.md rule → a diff in the form `- [today's date] rule (failure summary)`
+- Guard pattern → the regex to add to DENY_PATTERNS in guard-pre-bash.sh
+- Permission → the allow/ask/deny change for settings.json permissions
+- Sensor → commands.env change or file-extension additions in sensor-post-edit.sh
+
+With each proposal, include: (1) the failure class it prevents,
+(2) possible side effects (over-blocking etc.), (3) overlap/conflict with
+existing rules.
+
+## Approval and Cleanup
+
+- Apply only what the user approves.
+- After applying, if an existing guide rule is now enforced automatically
+  by a sensor, propose deleting it (guide hygiene — see `/guide-audit`).
+
+## Forbidden
+
+- No auto-applying without approval.
+- No proposals that weaken verification: disabling tests, ignoring
+  failures, or loosening blocks is regression, not ratcheting. If the logs
+  show such a request, report that itself as a failure class.
+- No more than 5 proposals at once (start with the most frequent class).
+- No rules for failures that didn't reproduce (don't grow rules from guesses).

package/.harness/commands.env

@@ -0,0 +1,13 @@
+# ════════════════════════════════════════════════════════════════
+# Verification sensor commands — fill in for your project
+# (/harness-init fills these for you). Empty = that sensor is off.
+# Keep in sync with the PROJECT section of CLAUDE.md.
+# ════════════════════════════════════════════════════════════════
+
+# e.g. LINT_CMD="ruff check ."        (Python)
+#      LINT_CMD="npm run lint"        (Node)
+LINT_CMD=""
+
+# e.g. TEST_CMD="pytest -q"           (Python)
+#      TEST_CMD="npm test"            (Node)
+TEST_CMD=""

package/CLAUDE.md

@@ -0,0 +1,68 @@
+# PROJECT
+
+- PROJECT: [project name]
+- LANGUAGE: [primary language, e.g. Python 3.12 / TypeScript 5]
+- BUILD: [exact build command, e.g. `npm run build`]
+- TEST: [exact test command, e.g. `npm test` / `pytest -q`]
+- LINT: [exact lint command, e.g. `npm run lint` / `ruff check .`]
+
+> Setting up for the first time? Run `/harness-init` — it scans the repository
+> and fills in this section plus `.harness/commands.env`. The verification
+> sensors only work once commands.env is filled. Keep both places in sync.
+
+## RULES
+
+- After modifying code, always run the TEST command, read the full failure
+  output, and fix it. Never declare work "done" without passing tests.
+- Never modify config / environment / CI / harness files (.env, *.config.*,
+  .github/**, .claude/**, .harness/**) without asking the user. In particular,
+  any change that weakens tests, lint, or guards requires confirming the
+  reason with the user — even if the user asked for it.
+- Announce dependency additions (package installs) to the user with the
+  reason before running them.
+- If the same error occurs 3 times, stop repeating the same approach and
+  write an escalation packet (see Work Loop below).
+- [YYYY-MM-DD] [add new rules here with a date — /ratchet proposes them]
+
+## ANTI-PATTERNS
+
+- [YYYY-MM-DD] [observed failure pattern, e.g. "ran only a -k subset because
+  tests were slow, then reported the full suite as passing"]
+
+## Work Loop
+
+- For any task with 3+ steps, write the plan to `plan.md` before starting.
+- Retry ceiling: at most 3 fix attempts per goal. Beyond that, stop and
+  write an escalation packet.
+- Escalation packet format (report to the user):
+  1. What decision is needed
+  2. Alternatives already tried and why each failed
+  3. The cost of waiting
+  4. The safest default action if there is no response
+- When the budget is exhausted (retries/time), report the best output so
+  far + completed work + unresolved issues + the reason for stopping.
+  Never hide partial failure behind a fluent summary.
+
+## Checkpoints
+
+- At session start, if `progress.json` (checkpoint) exists, read it first
+  and do not repeat completed steps. (The SessionStart hook will remind you.)
+- After completing each meaningful step, run `/checkpoint` to update
+  plan.md / decisions.jsonl / progress.json.
+- Keep transient facts and exploratory reasoning out of checkpoints. Record
+  only settled decisions in decisions.jsonl.
+
+## When a Mistake Is Found
+
+- Don't just fix it in conversation — make it structural: handle the mistake
+  with `/ratchet [what happened]` — reproduce → classify root cause →
+  propose a rule/sensor/permission → verify. Apply only after user approval.
+
+## Guide Maintenance (for humans)
+
+- Once a month, audit this file with `/guide-audit`: delete rules the
+  sensors now enforce automatically, and merge contradictory rules.
+- Every rule must be traceable to a date and an originating failure.
+  200 undated rules is not a harness — it's tech debt.
+- The same review comment 3 times → promote to a rule; the same rule
+  violated 3 times → promote to a gate (hook block).

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 GAFFWC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,212 @@
+# uni-harness
+
+![license](https://img.shields.io/badge/license-MIT-blue)
+![node](https://img.shields.io/badge/node-%E2%89%A516-brightgreen)
+![runtime](https://img.shields.io/badge/runtime-bash%20%2B%20python3%20stdlib-lightgrey)
+![for](https://img.shields.io/badge/for-Claude%20Code-d97757)
+
+An agent harness kit for Claude Code. It wraps the coding agent with
+automatic verification (sensors), destructive-command blocking (guards),
+checkpoint-based session recovery, full tool-call logging with tripwires,
+and a ratchet workflow that turns every failure into permanent structure.
+
+> Formula: **Agent = Model + Harness.** The model brings the reasoning;
+> this kit brings everything else — rules, sensors, loop limits, memory,
+> and observability.
+
+## Install
+
+```bash
+npx uni-harness init            # in your project root (or: init <path>)
+```
+
+Then open Claude Code in the project — Claude will notice the harness is
+unconfigured and **offer to run `/harness-init`** (or run it yourself). It
+scans the repository, detects your build/test/lint commands, **verifies
+them by actually running them**, and fills in `CLAUDE.md` and
+`.harness/commands.env` with your approval. The verification sensors stay
+inactive until this step is done.
+
+Other installer commands:
+
+```bash
+npx uni-harness update          # refresh kit machinery (never touches your files)
+npx uni-harness doctor          # diagnose the installation
+npx uni-harness uninstall --yes # remove kit machinery, keep your files
+```
+
+Requirements: bash, python3 (stdlib only — no packages), node ≥16 for the
+installer itself.
+
+**Existing projects are safe.** `init` never overwrites anything you own:
+an existing `CLAUDE.md` is kept (run `/harness-init` to have the harness
+sections proposed as an append), existing `settings.json` hooks and
+permissions are preserved (kit hooks are merged in), and your `.gitignore`
+is appended, not replaced. `update` refreshes only unmodified kit files —
+anything you've customized is skipped (listed, with `--force` to override).
+
+## What's Inside
+
+| File | Role |
+|---|---|
+| `CLAUDE.md` | Project commands, rules, anti-patterns, work-loop and checkpoint protocols |
+| `.claude/settings.json` | Hook registration |
+| `.claude/hooks/sensor-post-edit.sh` | Runs lint immediately on code edits, feeds failures back |
+| `.claude/hooks/stop-gate.sh` | Runs tests in batch at turn end; blocks stopping on failure (3-per-session cap, then demands escalation) |
+| `.claude/hooks/guard-pre-bash.sh` | Blocks destructive commands and verification bypasses (`--no-verify`) before execution |
+| `.claude/hooks/session-start.sh` | Re-injects in-progress checkpoints at session start/resume/compaction; nudges `/ratchet` when failures pile up |
+| `.claude/hooks/observe-log.sh` | Logs every tool call as JSONL + tripwires (same failure 3x, call surge) |
+| `.claude/agents/TEMPLATE.md` | Custom agent template (inactive until you uncomment `name`; example: fresh-context code reviewer) |
+| `harness/harness_report.py` | Health scorecard from the logs |
+| `harness/tests/` | Self-tests for the hooks and the installer |
+| `bin/cli.js` | The installer (init / update / doctor / uninstall) |
+
+Skills:
+
+| Skill | Role |
+|---|---|
+| `/harness-init` | Scan the repo → fill PROJECT section & commands.env (once, at install) |
+| `/checkpoint` | Save state to plan.md / decisions.jsonl / progress.json |
+| `/ratchet [mistake]` | Reproduce → classify → propose rule/sensor/permission → verify (no args: diagnose the logs) |
+| `/guide-audit` | Audit CLAUDE.md rules — keep / delete / convert-to-sensor (monthly) |
+
+## Recommended Permissions (optional)
+
+The kit does not impose a permission policy. For unattended or
+high-autonomy use, consider adding something like this to your project's
+`.claude/settings.json` — in particular the entries that stop the agent
+from editing its own harness:
+
+```json
+{
+  "permissions": {
+    "ask": [
+      "Edit(.claude/**)", "Write(.claude/**)",
+      "Edit(.harness/**)", "Write(.harness/**)",
+      "Edit(.env*)", "Edit(*.config.*)", "Edit(.github/**)"
+    ],
+    "deny": [
+      "Read(.env)", "Read(.env.*)", "Read(**/secrets/**)",
+      "Read(**/*.pem)", "Bash(sudo*)"
+    ]
+  }
+}
+```
+
+## Operating Routine
+
+- **Daily / per task:** just work. Sensors and guards are automatic. For
+  long tasks, save state with `/checkpoint`.
+- **When you spot a mistake:** don't just fix it in conversation — run
+  `/ratchet [what happened]` to make it structural. Same review comment
+  3x → rule; same rule violated 3x → promote to a guard/permission.
+- **Weekly:** check the scorecard with `python3 harness/harness_report.py`;
+  if repeated failures show up, run `/ratchet`.
+- **Monthly:** audit CLAUDE.md with `/guide-audit` — delete rules the
+  sensors now enforce, merge contradictions.
+
+## Customizing
+
+- Dangerous command patterns: `DENY_PATTERNS` in `guard-pre-bash.sh`
+- Tripwire thresholds: `observe-log.sh` (defaults: same failure 3x, 300 calls)
+- Sensor file extensions: the case statement in `sensor-post-edit.sh`
+- Stop-gate block cap: `stop-gate.sh` (default: 3 per session)
+- If you modify a hook, add a case to `harness/tests/test_hooks.sh` and run it
+
+## When Not to Use This
+
+It's overkill for one-off questions, exploratory brainstorming, or
+unverifiable creative work. It earns its keep on work that runs
+repeatedly, has real consequences on failure, runs unsupervised, or must
+preserve state across sessions.
+
+## Design Background
+
+This kit is a working implementation of two sources. Full analyses live in
+[`docs/`](docs/) — they are not shipped with the npm package (the running
+agent doesn't need theory), but they explain every design decision here.
+
+### 📄 EnvHarness: Awakening Static Worlds for Agent Learning
+
+> **[arXiv:2608.19880](https://arxiv.org/abs/2608.19880)** · Chengsong Huang, Zifeng Wang, Rujun Han, Chen-Yu Lee et al. (2026)
+> Deep-dive: [docs/analysis-01-envharness.md](docs/analysis-01-envharness.md)
+
+**What the paper says.** Just as attaching a harness (tools, memory,
+skills) to an *agent* extends it without touching its weights, you can
+attach a harness to an *environment* and customize its learning signal
+without touching its code: `Static Env + EnvHarness = Customized Env`.
+Environments are transformed only at the interface layer through three
+composable components — **Stage** (reshape the initial state), **Contract**
+(filter actions, transform observations, wrap transitions), **Chain**
+(compose environments) — so the original ground-truth verifier is always
+preserved, which is the method's core advantage over LLM-generated
+environments. An automated loop called **EnvRigger** drives the design:
+*Observe* failure trajectories → *Diagnose* root causes → *Write* a
+transformation → *Validate* it against the original failure. Across five
+benchmarks the wrapped environments lift agent performance (e.g. ALFWorld
+out-of-distribution +9.0 points, SWE-bench Verified +2.7 with 9.8% fewer
+steps) — with the biggest gains out-of-distribution, evidence of transfer
+rather than memorization.
+
+**What this kit takes from it:**
+
+| Paper concept | Implementation here |
+|---|---|
+| EnvRigger loop (Observe → Diagnose → Write → Validate) | the `/ratchet` skill — with automated acceptance replaced by **user approval** |
+| Contract: action filtering | `guard-pre-bash.sh` (destructive commands blocked pre-execution) |
+| Contract: structured feedback | `sensor-post-edit.sh` / `stop-gate.sh` (verification results fed back) |
+| Stage: prepared initial state | checkpoint recovery at session start |
+| **Preserve the original verifier** | invariant: the harness wraps your test suite and may never modify or bypass it |
+
+### 📘 Harness Engineering: The 6-Layer Production Playbook
+
+> A synthesis of public practitioner material — [Mitchell Hashimoto](https://mitchellh.com/)'s
+> ratchet methodology, the [OpenAI Codex field report](https://openai.com/index/harness-engineering),
+> [Martin Fowler](https://martinfowler.com/)'s guides-and-sensors taxonomy, and
+> Anthropic / LangChain / Cursor sources.
+> Deep-dive: [docs/analysis-02-harness-engineering.md](docs/analysis-02-harness-engineering.md)
+
+**What the playbook says.** Prompt engineering (what the model *says*) and
+context engineering (what the model *sees*) are subsumed by harness
+engineering: what the model *can do*, what survives failure, what is
+permitted, and what counts as done. The claim **Agent = Model + Harness**
+is backed by self-reported but consistent evidence — the same model
+jumping 30.91% → 74.55% on GAIA from a harness swap alone, a fixed model
+climbing rank 30 → 5 on Terminal Bench through harness optimization. The
+architecture is six layers, and they map one-to-one onto this kit:
+
+| # | Layer | Principle | In this kit |
+|---|---|---|---|
+| 1 | **Guides** | each line = permanent prevention of one past failure | `CLAUDE.md` |
+| 2 | **Sensors** | external deterministic checks, never self-judgment | `sensor-post-edit.sh`, `stop-gate.sh` |
+| 3 | **Agentic Loop** | ceilings on every budget, escalate on exhaustion | Work Loop protocol + tripwires + 3-block stop cap |
+| 4 | **Memory** | the filesystem is the memory; must pass the recovery test | `/checkpoint` + `session-start.sh` |
+| 5 | **Permissions** | the model cannot limit itself | `guard-pre-bash.sh` + recommended permissions |
+| 6 | **Observability** | record everything, alert on drift | `observe-log.sh` + `harness_report.py` |
+
+The operating rules come from it too — Hashimoto's **ratchet principle**
+("every time the agent makes a mistake, engineer a solution that makes
+that mistake impossible to repeat") and Cursor's promotion ladder:
+
+```mermaid
+flowchart LR
+    F[failure observed] --> R["/ratchet: reproduce + diagnose"]
+    R --> P{strongest layer}
+    P -->|context was missing| G["guide rule (CLAUDE.md)"]
+    P -->|detectable by check| S[sensor / lint rule]
+    P -->|should be impossible| H[guard / permission]
+    G & S & H --> V[verified against the original failure]
+    V --> N[the same failure cannot recur]
+```
+
+Same review comment **3×** → becomes a rule. Same rule violated **3×** →
+becomes a gate. And the maturity signal is a *declining* growth rate of
+rules — a harness that only accumulates is debt, which is what
+`/guide-audit` exists to prevent.
+
+---
+
+Extending the kit itself? Start with
+[docs/AGENT_BRIEFING.md](docs/AGENT_BRIEFING.md) — it carries the
+invariants (never weaken verification, no auto-apply without approval,
+minimal infrastructure) and the vetted backlog.