@miller-tech/uap 1.221.0 → 2.0.0

package/docs/plans/deliver-hardening-plan-2026-07-13.md

@@ -0,0 +1,173 @@
+# Deliver/enforcement hardening plan — 2026-07-13
+
+Motivating incident (pay2u, 2026-07-13): the `delivery-enforcement` policy blocked
+direct source edits and routed a pre-reviewed 6-file C++ change into `uap deliver`.
+The run declared **"Delivered — all required gates pass" after writing ZERO files**:
+the only detected project gates were npm build/test for `apps/web`, which were
+already green, so the loop had nothing that could fail. Meanwhile every blocked
+edit auto-spawned a **blind** background deliver run whose mission was the literal
+string "implement the intended change to <file>" — no content, no spec.
+Related, earlier incident (same repo, board finding): `expert-review-required`
+policy demanded artifacts under paths that Enforcement-Self-Protect locks agents
+out of, while the skill it named did not exist — a compliance catch-22.
+
+Defect sites (all paths relative to this repo):
+
+| # | Defect | Where |
+|---|--------|-------|
+| 1 | "Delivered" = required gates green; when gates are green at (or near) baseline the loop succeeds without a diff. Two paths: the `baselineCheck` short-circuit (`alreadyDelivered`) and the normal ladder pass — the incident run went green via the latter in 2 turns with zero writes | `src/delivery/convergence-loop.ts` ~742 (`baselineCheck`) and ladder success path |
+| 2 | Gate relevance to the MISSION is never established — npm web gates "verified" a C++ handler change | `src/cli/deliver.ts` gate selection, `src/delivery/execution-gate.ts` |
+| 3 | Self-gate (anti-vacuous floor) only engages when NO project gates are detected | `src/cli/deliver.ts` ~43 (`needsSelfGate = … noRealGates \|\| forceSelfGate`) |
+| 4 | `--no-auto` silently disables the acceptance judge (a verification rail) along with exploration aids | `src/cli/deliver.ts` auto-plan wiring |
+| 5 | Acceptance judge can accept with an empty diff (diff-blind) | `src/delivery/acceptance-judge.ts` |
+| 6 | Gate detection is npm-centric; polyglot repos (C++/docker, SQL migrations) get irrelevant gates | `src/delivery/execution-gate.ts` (`detectArtifactType`, package.json only) |
+| 7 | Autoroute spawns deliver with a vacuous hint; the blocked edit's actual content (old/new strings) is discarded | `src/policies/enforcers/delivery_enforcement.py` ~158 (`deliverHint=f'implement the intended change to {rel}'`), `templates/hooks/deliver_autoroute.py` |
+| 8 | Agentic executor has whole-file `write_file` only — no anchored edit tool; weak models re-read in loops and whole-file rewrites of 1,200-line files are high-risk | `src/delivery/agentic-executor.ts` |
+| 9 | keep-best / protect-tests snapshots cover the whole tree — a deliver run in a SHARED worktree reverts other agents' concurrent (legitimate) edits, including their newly written tests (observed: frontend agent's new test file reverted) | `src/delivery/…` snapshot/rollback + protect-tests paths |
+| 10 | Policy liveness: a policy can demand a compliant path that does not work (missing skill, agent-locked artifact dir, deliver that no-ops) with no detection | `src/policies/*` (no liveness concept) |
+| 11 | Inverted self-protect: compliance artifact paths locked, but the policies DB the gate reads is not protected | pay2u operator patch `uap-policy-fix-APPLY.sh` exists; not upstreamed |
+| 12 | Operator escape hatches are env-var-only (`UAP_DELIVER_BYPASS`, `UAP_WORKDIR_ALLOW`); env does not reliably reach hook processes in every harness, so the sanctioned override paths are effectively unreachable mid-session | `templates/hooks/uap-policy-gate.sh`, enforcers |
+| 13 | Default executor model `qwen35-a3b` regardless of repo criticality | `src/cli/deliver.ts` defaults |
+| 14 | Hook resolves the governing repo from the shell CWD / `cd` target, not from the operation's TARGET path — cross-repo writes are gated by one repo's policy but read the other repo's compliance state (observed: `uap plan validate` recorded in this repo was invisible to the gate anchored in pay2u) | `templates/hooks/uap-policy-gate.sh` MAIN_ROOT resolution; enforcers using relative `.uap/` state |
+
+---
+
+## Options per area
+
+### A. Kill the false-green (defects 1–3) — pick one, A2 recommended
+
+- **A1 — No-op guard (minimum).** For coding missions, `success` requires a
+  non-empty applied-files set across the run (`filesApplied` union), unless the
+  caller passes `--allow-noop`. The `alreadyDelivered` baseline short-circuit is
+  only taken when the acceptance judge affirmatively confirms the mission is
+  already satisfied, with cited evidence. Small, surgical; does not fix gate
+  relevance (a run that writes one irrelevant file still "delivers").
+- **A2 — Universal anti-vacuous floor (recommended).** Generalize the self-gate
+  rule that already exists for gate AUTHORING to the whole run: every deliver run
+  must have ≥1 required gate that FAILS at baseline. If all detected project
+  gates are green at turn 0, auto-engage self-gate authoring (today that happens
+  only when `noRealGates`). Change is essentially
+  `needsSelfGate = selfGateAllowed && (noRealGates || allRequiredGatesGreenAtBaseline || forceSelfGate)`
+  plus threading the baseline result into gate selection. HARD-FAIL rule: if the
+  required self-gate is still vacuous after its authoring retries (passes on the
+  unsolved repo), the RUN fails — today a vacuous gate is only a warning, which
+  re-opens the false-green door. This is the smallest change that makes
+  "delivered" always mean "something that was red is now green".
+- **A3 — Mission-scoped gate relevance (deep).** Classify the mission → affected
+  paths → require gates whose declared scope covers those paths (needs B1's gate
+  manifests with `scope:` globs). Most correct; biggest lift; do after A2.
+
+### B. Polyglot gates (defect 6) — B1 recommended, B2 additive
+
+- **B1 — Project-declared gates.** `.uap.json → delivery.gates[]`:
+  `{id, name, cmd, cwd, scope: [globs], required, tier, timeoutSec}`. Declared
+  gates merge with (and outrank) detected ones. pay2u would declare:
+  docker-buildx `--target builder` for `apps/api/**`, `gen_openapi.py --check`
+  for handler files, vitest workspace for `apps/web/**`. Also gives A3 its
+  scope metadata later.
+- **B2 — Broader detection.** Recognize Dockerfile targets, CMakeLists, cargo,
+  pyproject, migrations dirs. Heuristic; useful defaults, never as trustworthy
+  as B1.
+
+### C. Verification rails (defects 4–5) — do both, small
+
+- **C1 — Diff-aware acceptance.** The judge prompt always includes
+  `git diff --stat` + changed-file list since run start; hard rule before the
+  LLM call: empty diff ⇒ acceptance FAIL (for coding missions). An LLM judge
+  must never be the only thing standing between a no-op and success — make the
+  empty-diff case deterministic.
+- **C2 — Decouple rails from `--no-auto`.** `--no-auto` disables exploration
+  aids only (candidates, ideation, practices, escalation). Acceptance judging
+  and the anti-vacuous floor are verification rails and stay on unless
+  explicitly disabled (`--no-acceptance`).
+
+### D. Autoroute with content (defects 7–8) — D1+D3 recommended, D2 policy choice
+
+- **D1 — Capture the blocked edit as an intent.** The PreToolUse payload already
+  contains the exact `old_string`/`new_string`/`content`. The enforcer writes the
+  FULL intent into `.uap/pending-deliver.jsonl` (file, op, old, new, ts, agent).
+  The spawned deliver mission becomes "apply the recorded edit intents for
+  <file>" and the executor gets a deterministic `apply_pending_edits` fast path
+  (anchored exact-match application, no model improvisation; model only engages
+  when an anchor no longer matches). This converts "block and hope" into a
+  queue of exact diffs applied through the sanctioned pipeline with gates —
+  today's entire incident would have been a non-event.
+- **D2 — Spawn policy.** Default OFF for auto-spawn (log + surface the pending
+  intent to the agent in the block message: "recorded; run `uap deliver
+  --pending <file>` to apply"), batch multiple intents per file per run. Avoids
+  background fan-out entirely; the agent stays in control of when to flush.
+- **D3 — `edit_file` tool in the agentic executor.** Anchored `{path, old, new,
+  occurrence}` replacement alongside `write_file`. Needed by D1's fallback; also
+  removes the whole-file-rewrite hazard for large files generally.
+
+### E. Shared-workspace safety (defect 9)
+
+- **E1 — Exclusive lease.** `deliver.lock` becomes a real mutex: a second run
+  against the same worktree refuses to start (override flag for operators).
+  Concurrent agents queue rather than interleave.
+- **E2 — Scoped snapshots (recommended with E1).** keep-best rollback and
+  protect-tests restoration operate ONLY on files the run itself wrote (tracked
+  per-run), never tree-wide — a concurrent agent's test files can't be reverted
+  by someone else's deliver run.
+
+### F. Policy liveness (defects 10–12, 14) — the catch-22 class
+
+- **F1 — Compliance-path liveness check (recommended, with a sabotage guard).**
+  Every enforcer declares its compliant path as machine-checkable requirements
+  (`requires:` — commands that must exist, dirs that must be agent-writable,
+  skills that must resolve). `uap policy enable`/install runs the liveness
+  check and re-runs it on session-start; a policy whose compliant path is dead
+  raises a loud operator alert and is marked "unhealthy" in `uap policy status`.
+  IMPORTANT — auto-degrade to advisory is NOT the default: an agent could
+  deliberately break a compliant path (delete a skill, wedge deliver) to force
+  a blocking policy down to advisory. Degrade-on-dead-path is opt-in per policy
+  (`degradeOnDeadPath: true`) and only when the breakage is outside
+  agent-writable surfaces; otherwise the policy stays blocking and the operator
+  decides. Liveness probe for deliver itself: a canary mission in a temp repo
+  that must produce a 1-line diff and pass its gate.
+- **F2 — Upstream the self-protect inversion fix.** Port pay2u's operator patch
+  (`uap-policy-fix-APPLY.sh`): protect `agents/data/memory/policies.db` (the
+  enforcement DB) while allowing agent WRITES (not edits/deletes) to compliance
+  artifact paths the gates demand (`.uap/reviews/…`).
+- **F3 — Operator overrides via signed file.** Honor
+  `.uap/operator-overrides.json` (owner-writable only; hook verifies uid/mode)
+  carrying `{deliverBypass: true, workdirAllow: [...], expiresAt}` in addition
+  to env vars — env doesn't reach hook subprocesses in all harnesses, which
+  today makes the documented escape hatches unreachable mid-session. Time-boxed
+  by `expiresAt` so overrides can't rot open.
+- **F4 — Anchor policy state to the TARGET repo (defect 14).** The gate hook
+  resolves MAIN_ROOT from the operation's target path (file_path) when present,
+  falling back to cwd only for path-less ops; enforcers read state via
+  `UAP_STATE_DIR` derived from that root, never a bare relative `.uap/`.
+  Cross-repo work then meets the target repo's policies with the target repo's
+  compliance state.
+
+### G. Model routing (defect 13)
+
+- **G1 —** `.uap.json → delivery.model` / `delivery.routing` as the
+  config-authoritative default (same authoritative-over-env pattern as
+  `delivery.enforcement`); ship a `criticality: money|normal|sandbox` shorthand
+  that maps to routing presets. Local qwen stays the default only for `sandbox`.
+
+---
+
+## Recommended phasing
+
+- **P0 (stops the bleeding; small diffs):** A2 + C1 + C2, and D2's log-only
+  default for autoroute (one-line default flip). After P0, a false-green no-op
+  is structurally impossible and blind agents stop spawning.
+- **P1 (makes the sanctioned path actually good):** D1 + D3 + B1 + E1 + E2.
+  After P1, a blocked edit flows through deliver as an exact diff with real,
+  project-declared gates — the enforcement stops costing precision.
+- **P2 (kills the catch-22 class):** F1 + F2 + F3 + F4 + G1.
+- **P3 (correctness ceiling):** A3 mission-scoped gate relevance on top of B1
+  scopes.
+
+Rollout: fix in this repo → version-bump (1.145.0 for P0) → reinstall global
+(`npm i -g …`) → pay2u picks it up next session; add regression tests:
+(1) deliver on a green-gated repo with a real mission must NOT return
+`alreadyDelivered` or ladder-pass without a failing-at-baseline required gate;
+(2) empty-diff acceptance must fail; (3) autoroute intent must round-trip
+`old/new` verbatim; (4) policy liveness canary on the two historical
+catch-22s; (5) vacuous required self-gate ⇒ run failure, not warning;
+(6) cross-repo write meets the TARGET repo's policy state.

package/docs/plans/deliver-hardening-review-response-2026-07-13.md

@@ -0,0 +1,180 @@
+# Deliver Hardening — Parallel-Review Response (2026-07-13)
+
+Three review angles ran concurrently against the uncommitted hardening diff
+(code-quality-reviewer, security-code-reviewer, architect-reviewer). This
+document records every finding and its disposition. The implementation plan
+itself is `deliver-hardening-plan-2026-07-13.md`.
+
+## Blockers — fixed
+
+### 1. E2 scoped restore was inert on the default executor (code-quality + architect)
+
+`runFiles` came from `history[].filesApplied`, but the agentic executor (the
+DEFAULT whenever the repo has inspectable content) writes through its own
+tools behind a no-op applier, so the write-set was empty by construction: a
+regressed run logged "nothing to do" and disposed the only good snapshot —
+worse than the blast-radius defect E2 fixed.
+
+**Fix:**
+
+- `AgenticExecutorOptions.writeLedger` — a caller-owned per-run `Set` of
+  project-relative paths, recorded at the `write_file`/`edit_file`/`edit_range`
+  `writeFileSync` sites (NOT via the bash sweep's `authorised` map, which is a
+  no-op when bash is disabled — the common case).
+- `SweepOutcome.changed` — the turn-end bash sweep now reports every
+  stat-moved path, judged or not, so shell writes reach the same ledger.
+- deliver.ts drains the ledger in the applier seam, so
+  `history[].filesApplied` is truthful again. The isolated-task executor is
+  deliberately NOT wired in: its paths are relative to its own tree and would
+  mis-target the projectRoot-scoped restore.
+- Empty write-set + warranted rollback now fails LOUD: the snapshot is
+  preserved with an operator-actionable message instead of being silently
+  disposed. (A whole-tree fallback restore was considered and rejected — in a
+  shared worktree it reverts other agents' work, the incident E2 exists to
+  fix.)
+- The convergence loop's idle-breaker comment was updated: `filesApplied` is
+  no longer empty-by-construction on the agentic path.
+
+Tests: `test/delivery/agentic-write-ledger.test.ts` (8: tool writes recorded,
+refusals not, end-to-end through a mocked model, shell write via the sweep),
+plus a `changed`-field test in `bash-sweep.test.ts`.
+
+### 2. operator-overrides.json was agent-mintable (security BLOCK)
+
+Every check on the file was satisfiable by a same-uid agent (uid==euid is
+vacuous when the agent IS the user), and an interpreter-mediated write
+(`python3 -c 'open(...)'`) walked past the self-protect text scan — one Bash
+call minted `deliverBypass` + `selfProtectOff`, neutralizing all enforcement.
+
+**Fix:**
+
+- The hook now requires the override file (and the liveness cache, below) to
+  be **ROOT-owned** (`st_uid == 0`), regular, not group/world-writable,
+  symlink-free (O_NOFOLLOW + fstat-on-fd), and unexpired. Root ownership is
+  the only check with teeth under a same-uid agent: the operator plants the
+  file with `sudo`, the agent cannot. (When the agent itself runs as root, no
+  local check can help — enforcement is already theater there.)
+- Defense-in-depth: `enforcement_self_protect.py` gained an interpreter-write
+  rule — a bash command naming either trust anchor
+  (`.uap/operator-overrides.json`, `.uap/policy-liveness.json`) with an
+  interpreter (python/perl/ruby/node) and a write primitive
+  (`open(…"w"/"a"/"x"/"+")`, `.write*/.dump/.rename/.unlink/…`) is blocked at
+  any wrapper depth. Reads (`open` without a mode, `json.load`, `cat`) stay
+  allowed; ordinary `.uap` runtime state stays writable.
+- Newline/colon injection through `workdirAllow` values is rejected (a value
+  carrying `\nUAP_SELF_PROTECT_OFF=1` would have smuggled an extra export into
+  the hook's eval loop).
+
+Tests: the override suite was restructured around the new trust model — the
+user-owned-refusal case runs everywhere (it IS the security property); the
+trusted-file cases are `runIf(getuid()===0)` since a non-root process cannot
+mint a root-owned fixture by construction. New injection test. Six new
+enforcer tests (`test_enforcement_self_protect.py`).
+
+## High — fixed
+
+### 3. Liveness cache trusted blindly (security)
+
+The degrade consult now applies the same trust terms as the override file:
+root-owned, regular, not group/world-writable, no symlink, and FRESH
+(`checkedAt` < 24h — a fixed path silently re-tightens; staleness bound was
+the architect's should-fix). The file was already added to PROTECTED_TARGETS
+pre-review. Consequence, documented in the hook and tests: `uap policy
+liveness` run as the user writes the cache for STATUS display; the gate never
+degrades on it. No seeded policy sets `degradeOnDeadPath` today, so no live
+behavior changes; when one does, the operator refreshes the cache elevated —
+a deliberate act, matching the override story.
+
+### 4. Declared-gates cwd containment (security #3, partial)
+
+`runRung` refuses a declared `cwd` that resolves outside the project root
+(one-line containment check, fail-closed with a clear message; test added).
+The deeper finding — `.uap.json` is writable via interpreter-mediated bash,
+and declared gates make that RCE — is PRE-EXISTING and honestly documented in
+the enforcer (`HONEST SCOPE`); the same interpreter-write class now blocked
+for the trust anchors could be extended to `.uap.json` in a follow-up. The
+id-shadowing of detected rungs by declared ones is an intended feature
+(documented outrank semantics), kept.
+
+## Should-fix — fixed
+
+- **Liveness worktree alignment (code-quality + architect):** `liveness.ts`
+  gained `mainCheckoutRoot()` (git toplevel + `.worktrees` strip, same rule
+  as the hook's `${CHECKOUT_ROOT%%/.worktrees/*}`), applied in
+  `checkPolicyLiveness`/`writeLivenessCache`/`readLivenessCache`/
+  `enabledPolicySlugs`. Session-start runs the refresh from `COORD_ROOT`.
+  Previously a worktree session (the mandated workflow) wrote the cache and
+  read the DB where the gate never looks — F1 was inert there.
+- **Cache contract:** `version: 1` field added; the slug rule is documented
+  as one rule in three places (TS writer, `policyNameSlug`, hook re.sub).
+- **`classifySurface`** now compares separator-terminated (`/repo-evil` no
+  longer classifies as inside `/repo`), and an unwritable dir INSIDE the repo
+  is `agent-writable` (agent could chmod it — the hardcoded `external` was a
+  sabotage hole once any policy opts into degrade).
+- **Python execution gate** prefers `.venv/bin/python` over ambient
+  `python3` (deps live in the venv; ambient import failed healthy repos).
+  Test with a stub venv interpreter added.
+- **`runNativeBin`** no longer counts a binary that failed to exec at all
+  (ENOENT/bad shebang/wrong arch) as pass evidence; all-unexecutable reads as
+  an honest skip-pass ("not measurable here"), not a pass.
+- **Hook hot path:** the override parse is guarded by an existence early-out
+  — zero python spawns on the common path. (Merging the two payload parses
+  was considered and skipped: churn on a string the tests pin byte-exactly,
+  for ~one spawn on path-carrying ops only.)
+- **F4 anchor:** walks to the nearest EXISTING ancestor directory (a Write
+  into a not-yet-created dir in another repo no longer falls back to cwd),
+  and the extractor accepts the same key set as the self-protect extractor
+  (`file_path`, `path`, `filePath`, `target`, `filename`, `file`).
+- **deliver.ts** uses `Object.hasOwn` for the criticality lookup (prototype-
+  chain values like `"constructor"` no longer pass).
+- **plan.ts** `stateDir` honors an ABSOLUTE `UAP_STATE_DIR` (the hook exports
+  it as `$MAIN_ROOT/.uap` since F4; `join` would have concatenated it).
+- **acceptance-judge** labels the git section "UNCOMMITTED CHANGES IN THIS
+  TREE" with a shared-worktree caveat instead of "CHANGES IN THIS RUN" — the
+  diff is against HEAD, so in a shared worktree it lists other agents' work
+  and the old label actively misled the judge.
+- **acceptance-judge test fixture** sets local git identity (flakes on
+  machines without global user.email).
+- **`uap policy status` help** documents the cache-write side effect.
+
+## Considered, not done
+
+- **Whole-tree restore fallback for an empty write-set** — rejected: it is
+  the shared-worktree incident E2 fixes. Loud preserve instead.
+- **Signing overrides with an HMAC/keyfile instead of root ownership** —
+  root ownership is simpler, needs no key management, and is auditable with
+  `ls -l`. Revisit if multi-user operator workflows appear.
+- **Re-deriving `degradable` in the hook from the registry** — the registry
+  is TS; the hook is bash. Root-ownership + freshness + the opt-in bit in the
+  cache achieves the same guard without a TS eval in the hot path.
+- **ADR adoption** — the repo convention is `docs/plans/`; the plan doc plus
+  this response satisfy the decision-record intent.
+
+## Behavior changes operators should know (in the commit message)
+
+1. **`UAP_STATE_DIR` is now exported by the gate hook as `$MAIN_ROOT/.uap`.**
+   In-flight per-worktree state (`plan_state.json`, `read_log.state`, IaC
+   acks) is orphaned once — a one-time fail-closed reset: sessions may be
+   re-prompted for plan review. State then lives consistently in the main
+   checkout instead of fragmenting per worktree.
+2. **Config beats env for deliver model routing** (G1): a declared
+   `delivery.model`/`routing`/`criticality` in `.uap.json` now outranks
+   ambient `UAP_DELIVER_MODEL`/`UAP_DELIVER_ROUTING`.
+3. **The operator-override file must be root-owned** (sudo) to be honored —
+   the original uid/mode-only check was vacuous against a same-uid agent.
+4. **The liveness degrade path** consults only root-owned, fresh (<24h)
+   caches.
+
+## Verification
+
+- `tsc --noEmit` clean.
+- All touched vitest suites green (ledger 8, bash-sweep 39, snapshot-scoped
+  10, declared-gates 9, execution-gate-polyglot 13, acceptance-judge 26,
+  policy-liveness 12, gate hook suites 22, routing 13, run-state 15).
+- Python enforcer suite: 1212 tests; 3F+9E all in `test_uap_compliance`
+  (live-DB schema checks), byte-identical failure set on unmodified master —
+  pre-existing environmental.
+- Hook mirror parity enforced by `test_gate_failclosed_indirection` +
+  `test_gate_failclosed_schema_diff` (these caught an unsynced mirror during
+  remediation — the parity net works).
+- Full vitest suite: see PR body for the final run.

package/docs/plans/k3-uap-uplift-validation.md

@@ -0,0 +1,216 @@
+# Kimi K3 × UAP Uplift Validation — Factory Droid Harness Plan
+
+**Status:** Proposed (awaiting go-ahead before any paid API runs)
+**Date:** 2026-09-02
+**Supersedes:** the earlier opencode-based variant of this plan (the harness is now Factory Droid, the harness this repo's own sessions run in)
+
+## Goal
+
+Measure UAP's uplift on a frontier open-weight model by re-running an existing,
+publicly-anchored benchmark with UAP enabled vs disabled, holding **model,
+harness, task set, and attempt counts fixed**. The model is **Kimi K3**
+(Moonshot AI, 2.8T-param MoE, open-weight, July 2026); the harness is
+**Factory Droid** in headless mode (`droid exec`); the benchmark is
+**Terminal-Bench 2.1**.
+
+The only thing that differs between the two arms is the UAP treatment. That
+paired design is the entire experiment: any resolution-rate delta is
+attributable to UAP and nothing else.
+
+## Why Terminal-Bench 2.1
+
+K3's published scores with harness attribution (from the MoonshotAI/Kimi-K3
+tech report and repo):
+
+| Benchmark | K3 score | Harness | Re-runnable? |
+|---|---|---|---|
+| **Terminal-Bench 2.1** | **88.3** | Kimi Code | Yes — public tasks, deterministic verifiers, API-feasible |
+| SWE-bench Verified | 93.4 | Vals AI | Partially — harness not public |
+| SWE-Marathon | 42.0 | Claude Code / Harbor | Yes, but long-horizon = expensive |
+| PostTrainBench | 36.6 | Claude Code | Yes, but GPU-calibrated |
+| DeepSWE / FrontierSWE / Kimi-Code-Bench | various | internal | No — harness-locked |
+
+Terminal-Bench 2.1 (89 Dockerized terminal tasks, time-boxed, resolved only
+when post-run verifier tests pass) is the cleanest anchor:
+
+- Public dataset with deterministic, non-LLM verifiers — no judge drift.
+- Already supported by our in-repo paired-harness pattern
+  (`benchmarks/terminal_bench/uap_opencode_agent.py`), which we adapt to Droid.
+- Factory has published its own Droid-on-Terminal-Bench methodology
+  (factory.ai/news/terminal-bench, Sept 2025, TB Core v0.1.1), so our Droid
+  setup mirrors a canonical, publicly-described configuration rather than an
+  ad-hoc one.
+
+### Comparability caveat (read before quoting numbers)
+
+K3's 88.3 was achieved on the **Kimi Code** harness, not Droid. Factory's own
+TB analysis found harness design moves scores as much as model choice (Droid +
+Sonnet beat Claude Code + Opus). So:
+
+- **Primary claim (internally valid):** paired Δ = UAP-on minus UAP-off, same
+  Droid+K3 stack. This is the number the experiment is designed to produce.
+- **Secondary claim (cross-harness, caveated):** Droid+K3+UAP vs the published
+  88.3 Kimi-Code anchor. Report it, always with the harness difference named.
+
+## Configuration
+
+### Kimi K3 via BYOK (`~/.factory/settings.json` inside the task container)
+
+Primary endpoint — Moonshot's official API (OpenAI-compatible dialect):
+
+```json
+{
+  "customModels": [
+    {
+      "model": "kimi-k3",
+      "displayName": "Kimi K3",
+      "baseUrl": "https://api.platform.kimi.ai/v1",
+      "apiKey": "${KIMI_API_KEY}",
+      "provider": "generic-chat-completion-api",
+      "maxOutputTokens": 65536,
+      "extraArgs": { "temperature": 1.0, "top_p": 1.0 }
+    }
+  ]
+}
+```
+
+Fallback endpoint — OpenRouter (`baseUrl: https://openrouter.ai/api/v1`,
+`model: moonshotai/kimi-k3`, `apiKey: ${OPENROUTER_API_KEY}`). Identical
+pricing at time of writing: **$2.55 / Mtok input, $12.75 / Mtok output**.
+
+Sampling parameters follow the K3 tech report's agentic-eval settings:
+`temperature=1.0`, `top_p=1.0`, maximum reasoning effort. Droid's
+`-r/--reasoning-effort` flag does **not** apply to custom models; if the
+endpoint exposes a reasoning-effort knob (e.g. `reasoning_effort`), pass it
+through `extraArgs` and confirm acceptance during Phase 0.
+
+Model reference on the command line: `custom:Kimi-K3-0` (display name with
+spaces dasherized + 0-based index into `customModels`).
+
+### Arms
+
+| Arm | Model route | UAP treatment |
+|---|---|---|
+| **A — Baseline** | Droid → K3 direct | none |
+| **B — UAP (protocol)** | Droid → K3 direct | UAP operating protocol dropped as `AGENTS.md` in the task workdir (Droid reads `AGENTS.md` natively) — same honest treatment surface as the existing opencode paired shim: gates discipline, verify-don't-assume, stop-when-green |
+| **C — UAP (enforced, optional)** | Droid → **UAP proxy :4100** → K3 | Arm B **plus** the UAP proxy in the request path, so deterministic enforcers (MANDATE-DELIVER, guardrails) are in the loop. Mirrors the `UAP_TB_UAP_BASE_URL` pattern of the opencode shim. |
+
+A vs B is the primary paired comparison (cheapest, cleanest attribution).
+C runs only if A/B shows a signal worth deepening, since it exercises the full
+enforcement stack rather than prompt-level discipline alone.
+
+### Droid invocation (per TB task, inside the task container)
+
+Mirrors Factory's published TB methodology: non-interactive task mode, all
+permissions skipped (containers are disposable), task specification passed as
+the prompt.
+
+```bash
+droid exec \
+  --skip-permissions-unsafe \
+  --model "custom:Kimi-K3-0" \
+  --output-format json \
+  "$(cat /task/instruction.md)"
+```
+
+The UAP arm additionally writes `AGENTS.md` into the task working directory
+before the run (and, for arm C, points the model `baseUrl` at the proxy).
+
+### New adapter: `benchmarks/terminal_bench/uap_droid_agent.py`
+
+A custom `AbstractInstalledAgent` — the same integration point Factory used
+for its own TB runs — with two exported classes, `DroidBaseline` and
+`DroidUAP`, differing only in the AGENTS.md drop (and proxy base URL for arm
+C). A companion `uap-droid-setup.sh.j2` runs inside the container:
+
+1. Install the Droid CLI: `curl -fsSL https://app.factory.ai/cli | sh`
+2. Write `~/.factory/settings.json` with the customModels block above
+3. Export `FACTORY_API_KEY` (headless auth) and the K3 provider key from
+   host-passed env (`UAP_TB_KIMI_API_KEY` / `UAP_TB_OPENROUTER_API_KEY`,
+   `UAP_TB_FACTORY_API_KEY`)
+
+Run shape (from the bench venv):
+
+```bash
+UAP_TB_FACTORY_API_KEY=fk-... UAP_TB_KIMI_API_KEY=sk-... \
+.tbvenv/bin/tb run \
+  --agent-import-path uap_droid_agent:DroidBaseline \
+  --dataset terminal-bench==2.1 -t <task-id> --n-attempts 5
+# then again with :DroidUAP, and diff the two results.json
+```
+
+(Exact TB 2.1 dataset/registry name to be confirmed during Phase 0; the
+harness moved to the Harbor registry since the opencode shim was written.)
+
+## Protocol
+
+Three phases, each gated on the previous one. **No paid run starts without an
+explicit go from the user.**
+
+### Phase 0 — Smoke (no benchmark scoring; ~$5)
+
+Single cheap TB task (e.g. a hello-world-tier task), a few attempts, both
+arms. Validates:
+
+1. K3 responds correctly through Droid's `generic-chat-completion-api`
+   provider — tool calls parse, multi-turn works.
+2. **Preserved thinking history:** K3 requires prior `reasoning_content` to be
+   passed back verbatim across turns. Whether Droid round-trips this field for
+   generic providers is unknown — inspect request logs (or the arm-C proxy
+   log) to confirm. If it does not round-trip, note the degradation, keep both
+   arms identical (the paired comparison remains valid), and consider the
+   Anthropic-compatible Kimi endpoint with `provider: "anthropic"` as a
+   variant.
+3. Container networking: container → provider egress; container → host proxy
+   for arm C.
+4. Headless auth: `FACTORY_API_KEY` works non-interactively in the container.
+
+### Phase 1 — Pilot (~10–12 tasks × 3 attempts × 2 arms; est. $40–90)
+
+Stratified sample across TB 2.1 categories (coding, build/deps, data/ML,
+systems, security). Purpose: shake out adapter bugs, measure per-run token
+burn to firm up the Phase 2 budget, and check variance. No conclusions drawn.
+
+### Phase 2 — Full sweep (89 tasks × 5 attempts × 2 arms)
+
+Five attempts per task matches both Factory's published TB convention and the
+existing paired-harness format. Seeds fixed where the endpoint allows;
+temperature/top_p held at the tech-report values in both arms.
+
+### Analysis (format per `docs/benchmarks/PAIRED_FINDINGS.md`)
+
+- Per-task paired resolution rates; Δ with bootstrap 95% CI.
+- McNemar 2×2 on discordant task-attempt pairs.
+- Secondary: tokens/cost per resolved task, wall-clock, gate-loop behavior
+  (arm B should show more build/test iterations and fewer one-shot stops).
+- Report both the paired Δ (primary) and the Droid+K3+UAP absolute score vs
+  the 88.3 Kimi-Code anchor (secondary, harness caveat attached).
+
+## Cost estimate
+
+Rough per-run burn for TB-scale tasks under Droid (environment bootstrap +
+multi-turn tool loop): assume 0.3–1.5M input tokens and 20–80k output tokens
+per attempt. At $2.55/$12.75 per Mtok that is ~**$1.00–4.85 per attempt**.
+Full sweep = 89 × 5 × 2 = 890 attempts → **~$900–4,300**. Pilot firms this up
+before any full-sweep commitment. Prompt caching, if the endpoint honors it
+for generic providers, cuts the input side substantially.
+
+## Risks and open questions
+
+| Risk | Impact | Mitigation |
+|---|---|---|
+| `reasoning_content` not round-tripped by Droid's generic provider | K3 degrades below published behavior | Phase 0 log inspection; both arms affected equally (paired Δ still valid); try Anthropic-compatible endpoint variant |
+| Custom models get Droid's generic tool scaffolding, not per-model tuning | Absolute score likely below Kimi Code's 88.3 even before UAP | Expected and acceptable — the paired Δ is the claim; report absolute with caveat |
+| Headless `FACTORY_API_KEY` auth inside containers | Adapter blocked | Phase 0 smoke |
+| K3 rate limits / TTFT under parallel attempts | Wall-clock blowup on TB's aggressive timeouts | Bounded concurrency (xargs -P), retry-with-backoff in the adapter |
+| Exact TB 2.1 registry/dataset name changed under Harbor | Run command fails | Confirm in Phase 0 against the current terminal-bench docs |
+| Endpoint pricing moves | Budget | Re-quote at Phase 2 kickoff |
+
+## Deliverables when executed
+
+1. `benchmarks/terminal_bench/uap_droid_agent.py` + `uap-droid-setup.sh.j2`
+   (the adapter; not part of this plan PR)
+2. Phase 0/1/2 run artifacts under `benchmark-results/` in the established
+   paired format
+3. A findings doc modeled on `docs/benchmarks/PAIRED_FINDINGS.md` with the
+   paired Δ, CI, McNemar table, and the caveated cross-harness comparison