@miller-tech/uap 2.16.7 → 2.20.0

package/docs/guides/MISSIONS.md

@@ -0,0 +1,170 @@
+# Missions & the Orchestrator Poll Daemon
+
+> **🏭 Where this fits:** BUILD — the durable layer *above* `uap deliver`.
+> **What it delivers:** a prompt — small or large — becomes a mission the
+> system owns until its acceptance criteria are verified met. It survives
+> reboots, crashes, session teardown, and context exhaustion. No babysitting.
+
+**Visual explainer:** [docs/visual/mission-orchestrator.html](../visual/mission-orchestrator.html) —
+the architecture, poll cycle, decision table, backoff, and salvage flow as a
+self-contained page (open in a browser; no build, no dependencies).
+
+**Architecture record:** [ADR-0003](../architecture/adr/0003-mission-orchestrator.md)
+— why the goal layer lives above deliver, the frozen-hash policy,
+provenance-checked linkage, and the claim/backoff semantics.
+
+[The task orchestrator](./ORCHESTRATOR.md) makes a single *run* converge, and
+the epic controller loops epics *within a run*. What neither owns is the goal
+itself across process lifetimes: a detached deliver run survives its tool
+call, but nothing resumes it after a reboot, relaunches it after a crash, or
+notices it quietly died. That is the mission layer's job — the same insight
+as Choir (Harvard/Weber's multi-agent autoformalization protocol): **the loop
+clock and the state must be durable and external to any agent process.**
+
+## The model
+
+```
+uap mission create ──▶ missions.db (frozen acceptance, attempt ledger)
+        │                        ▲
+        │ --launch (or wait)     │ discovery (mission:#N marker)
+        ▼                        │
+  uap deliver run ──run state──▶ uap mission poll (every 300s, systemd)
+                                 │ decide per mission:
+                                 │   live run        → one supervisor cycle
+                                 │   pid dead        → record interrupted, salvage, relaunch
+                                 │   interrupted     → salvage, resume (backoff after repeats)
+                                 │   failed         → salvage, relaunch (backoff)
+                                 │   delivered      → close the mission
+                                 └ board notes on delivery + struggle
+```
+
+## Creating a mission
+
+```bash
+uap mission create "Harden the policy gate for admin-protected routes" \
+  --acceptance "npm test and tsc --noEmit pass; no new quality-metric violations" \
+  --launch
+```
+
+- The acceptance criteria are **frozen** (sha256) at creation — the
+  statement-immutability analog. Changing them is an explicit replan that
+  records the prior text, hash, and reason:
+  `uap mission replan <id> --acceptance "<new>" --reason "<why>"`.
+- `--launch` starts the first run immediately; without it the daemon launches
+  it on the next poll. The criteria travel inside the run instruction and
+  `--acceptance` turns on the behavioral acceptance judge.
+- `uap mission list|show|pause|resume` manage the ledger. Paused missions are
+  invisible to the daemon.
+
+## The poll daemon
+
+`uap mission poll` is the loop clock (Choir's `orch poll`; hidden alias
+`uap orch poll` — NOT the same as `uap orchestrator on|off`, which toggles
+the in-deliver blackboard orchestrator). One cycle:
+
+1. **Discover** — link a fresh launch's run to its mission. Linkage is
+   provenance-checked: the run's instruction must carry the mission's
+   `[mission:#N:<hash8>]` marker (the hash slice makes it replan-scoped, so
+   an old-criteria run can never be relinked) AND its pid must match the pid
+   the orchestrator itself spawned. A planted run-state file cannot know a
+   future child pid, so it can never steer the daemon. Any status links: a
+   short mission can deliver inside one interval and must still close.
+2. **Sync** — terminal run states become attempt rows; a `running` state whose
+   pid is gone is recorded as interrupted (a crash that never marked itself).
+   Rows dedupe per (run, outcome, observedAt): identical re-observations
+   collapse, genuinely new failures of the same run accumulate.
+3. **Decide** — the pure decision table in `src/mission/poll-cycle.ts`:
+   live → supervise (one `runSupervisor` cycle: it watches and acts on
+   reviewed policy — STOP/RETRY/ESCALATE — never signals); terminal →
+   salvage + relaunch; delivered → close.
+4. **Act + backoff** — launches and relaunches go through an ATOMIC CLAIM
+   (a 60-second window), so the daemon poll and `uap mission create --launch`
+   can never both spawn the same mission. The first relaunch is immediate (a
+   clean interruption is the healthy case, not a crash loop); each further
+   relaunch doubles the wait (15, 30, 60 min … capped at a day).
+
+A cycle never throws — a broken ledger produces a report note, not a dead
+daemon — and systemd `Restart=always` covers the rest.
+
+### The service
+
+`uap-orchestrator.service` (installed by `installSystemdUserServices`) runs
+`uap mission poll --loop --interval 300`. Node and the CLI entrypoint are
+captured absolute at install time — a systemd user unit's PATH has no nvm.
+`uap doctor` probes it against its declared RSS budget in
+`config/capacity-policy.json` (512 MiB; restart budget 3).
+
+```bash
+systemctl --user enable --now uap-orchestrator.service
+journalctl --user -u uap-orchestrator.service -f
+```
+
+## Struggle & salvage (approval-gated)
+
+Every failed/interrupted attempt seeds a **salvage proposal**: what happened,
+what partial work exists, and — once a task has struggled past 3 failures — a
+recommendation to escalate the preset or decompose before the next relaunch.
+Proposals never auto-schedule:
+
+```bash
+uap mission salvage list
+uap mission salvage approve <id>   # or: reject <id>
+```
+
+Struggling missions also post a board note (`uap coord board`), so an
+operator — or an overseer agent — sees the escalation ask without watching
+the daemon. This is the Choir split: the orchestrator proposes; the overseer
+rules.
+
+## Merge gate (evidence, not testimony)
+
+A delivered run's DONE report is a claim. The merge gate is the last check
+between that claim and master — it recomputes the verdict from recorded
+evidence, deterministically, before work becomes irreversible:
+
+| Check | What it verifies |
+| --- | --- |
+| **provenance** | The PR's `[mission:#N:<hash8>]` marker matches the mission's CURRENT frozen acceptance (a replan after the PR was cut supersedes its criteria) — and the ledger itself still hashes intact |
+| **green-recompute** | The gate-evidence artifact for the exact sha being landed (all gates exited 0, no env hatches) — or, failing that, a fresh run-state read showing `delivered` with a passing final recorded iteration — and the run pid matches the launch; no salvage proposal awaits a ruling |
+| **sorry-delta** | No stub markers (`todo!()`, `NotImplementedError`, TODO/FIXME) in ADDED lines of production files, no apology phrasing in the run's summaries |
+| **axiom-honesty** | The diff does not touch gate infrastructure (hooks, CI, enforcers, baselines, this gate itself) without acknowledgment |
+
+```bash
+uap merge gate 7                 # run the gate for mission #7 (exit 1 on fail)
+uap merge gate 7 --json          # machine-readable findings
+uap merge gate 7 --allow-gate-infra   # only when the PR deliberately IS gate work
+```
+
+The merge queue runs this automatically before landing any PR whose title or
+body carries the mission marker; a gate failure skips the PR and lists every
+failed check. `--force` skips stale CI checks but never the gate, and there is
+no environment escape hatch. Deliberate gate-infrastructure changes are
+acknowledged with a `gate-infra` PR label, in plain sight of reviewers.
+
+Details and rationale: [ADR-0004](../architecture/adr/0004-deterministic-merge-gate.md).
+
+## Where state lives
+
+| Store | Path |
+| --- | --- |
+| Mission ledger (SQLite, WAL) | `.uap/missions.db` |
+| Durable run state (per run) | `.uap/deliver-runs/<runId>/state.json` |
+| Supervisor state/events | `.uap/supervise/<runId>/` |
+| Orch-spawned run logs | `.uap/orch-logs/orch-<stamp>.log` |
+| Stop latch | `.uap/deliver-runs/STOP` (cooperative; the supervisor uses it too) |
+
+## Design notes
+
+- **Frozen acceptance** is what makes "complete" checkable — the merge gate
+  (see above) verifies a PR's criteria against the frozen hash before any
+  merge.
+- **Attempt dedupe** is per (mission, run, outcome, taskRef, observedAt), so
+  re-observing the same terminal state cannot inflate the struggle metric —
+  while a run that keeps failing after each resume accumulates and reaches
+  the escalation threshold.
+- **Salvage dedupe**: one open proposal per failed attempt — a mission
+  waiting out a backoff window never duplicates proposals or board notes.
+- The daemon never *drives* a live run — it supervises. Launching and
+  resuming are the only writes it makes to a mission's fate, and both go
+  through deliver's own detached-run discipline (own session, stdio to a
+  file, checkpointed state).

package/docs/guides/PROXY.md

@@ -41,6 +41,54 @@ the "keep a small model from wedging" logic lives.
 | `GET /health` | Liveness (probes the upstream; stays unauthenticated) |
 | `GET /v1/context` | Per-session token-usage / utilization / guardrail telemetry (dashboards & debug) |
 
+### Ollama-compatible surface (`PROXY_OLLAMA_PORT`, default 11434)
+
+The same proxy also serves the Ollama wire protocol on a **second port**
+(default `11434`, Ollama's well-known port — the one Ollama-native tooling
+auto-discovers and cannot be re-pointed). Set `PROXY_OLLAMA_PORT=0` to disable
+it (do that on a box running a real Ollama, which owns 11434); if the port is
+busy at startup the surface disables itself and the Anthropic/OpenAI surface
+carries on unaffected.
+
+| Route | Purpose |
+|---|---|
+| `GET /api/version` | Ollama discovery handshake |
+| `GET /api/tags` | Model list — backend-served ids only (the Anthropic-protocol Claude ids are not leaked here) |
+| `POST /api/show` | Model card (404s unknown ids like real Ollama) |
+| `GET /api/ps` | Running-model list (the backend stays loaded; capacity policy owns that) |
+| `POST /api/chat` | Chat — messages, options, tools (OpenAI tool schema) |
+| `POST /api/generate` | Text generation — prompt + optional system |
+
+Every request runs the **full guardrail path** (admission control, rail
+budgeting, loop detection, tool narrowing, malformed-tool retry, context
+pruning) — an Ollama client gets the same treatment an Anthropic client does.
+The tradeoff matches the OpenAI-compatible surface: streaming is buffered
+through the guardrails, then emitted as Ollama NDJSON (one content line plus a
+final `done: true` line), not token-by-token.
+
+Options map only where they survive both translation hops — `temperature`,
+`top_p`, `top_k`, `stop`, and `num_predict` → `max_tokens`. Everything else is
+dropped honestly rather than accepted and silently ignored: Ollama's
+`format` (structured output) is not carried by the underlying pipeline yet
+(wiring `response_format` through is a tracked contract change), `num_ctx`
+would fight the rail budget, and `min_p`/`repeat_penalty`/`seed` are not
+forwarded by the second hop. `keep_alive`/`raw`/`template` are lifecycle
+features with no local equivalent.
+
+Two auth behaviors to know about when `PROXY_AUTH_TOKEN` is set: the
+discovery routes above stay unauthenticated (Ollama clients probe them before
+any credential could be presented), but `POST /api/chat`, `POST /api/generate`
+and `POST /api/show` gate exactly like `/v1/messages` — and since most
+Ollama-native tooling has no way to configure a bearer token, auth-on
+realistically means Ollama clients stop at a 401. On the default loopback
+deployment (no token) everything works.
+
+If the port is busy at startup (typically a real Ollama), the surface
+disables itself and the Anthropic/OpenAI surface carries on unaffected — the
+error lands in this proxy's journal. The reverse ordering has the opposite
+signature: if this proxy binds 11434 first, a later-started real Ollama is the
+one that fails to bind, and its error shows up in Ollama's logs, not ours.
+
 ---
 
 ## Lifecycle: `uap proxy`
@@ -190,6 +238,8 @@ set is below.
 | `PROXY_CONTEXT_WINDOW` (0 = auto) | Explicit local context window; 0 derives it from the live rail |
 | `PROXY_COUNT_TOKENS_SCALE` (auto) | Scales the reported token count so the client auto-compacts before the local rail overflows |
 | `PROXY_CONTEXT_PRUNE_THRESHOLD` (0.85) / `PROXY_CONTEXT_PRUNE_TARGET_FRACTION` (0.50) | Backstop pruning: prune at 85% of the window, land the session at ~50% |
+| `PROXY_PRUNE_SPIRAL_STREAK` (3) / `PROXY_PRUNE_SPIRAL_EPSILON` (0.01) | Prune circuit breaker fires only on a RISING post-prune floor (3 consecutive prunes each >1 point higher). A flat floor above the threshold is fixed system+tools overhead, not a death spiral — the breaker is held |
+| `PROXY_FINAL_ANSWER_CHARS` (800; deprecated env alias `PROXY_END_TURN_FINAL_CHARS`) | Content reaching this many characters is a legitimate final answer: the unexpected-end-turn retry is skipped and the client's loop ends naturally (thinking-runaway stalls have empty content). Also exempts a `required` tool turn returning such content from the `required_tool_miss` streak, retries, and forced-tool dampener; keeps the stream-path `malformed_tool_streak` from counting it; and stops the recon-convergence no-write streak from advancing on it — a finished model's summary is a completion, not a miss. Content matching the deferral phrases ("I need more cycles…") stays retried even when long — a capitulation is not a final answer. 0 = always retry |
 
 ### Connection health & backpressure
 

package/docs/reports/2026-10-02-plane-admin-fail-closed.md

@@ -0,0 +1,70 @@
+# Plane-admin enforcer now fails closed on shell operations
+
+Date: 2026-10-02
+Deliver run: `run-20261002T030209-1af47d` (status: delivered)
+
+## Problem
+
+`uap-policy-gate.sh` maps an errored or missing enforcer to ALLOW for everything
+except the two enforcers named in `must_fail_closed()` (`enforcement_self_protect`
+on `SEC_SENSITIVE`, `schema_diff_gate` on `COMMIT_OP`). `plane_admin_protect` —
+the guard over Plane admin credential resets — was not in that selector, so a
+crashed or absent plane enforcer was a silent bypass of the admin controls on
+exactly the surface it exists to protect.
+
+## Change
+
+New `SHELL_OP` arm, keyed on the tool name (the enforcer's own activation test is
+tool-based — a shell tool is the only one it is ever asked about):
+
+```bash
+SHELL_OP=0
+case " $TOOL " in
+  *" Bash "*|*" bash "*|*" BashTool "*|*" Shell "*|*" shell "*|*" Terminal "*|*" terminal "*|*" execute_command "*|*" run_command "*|*" exec "*|*" local_shell "*|*" command "*)
+    SHELL_OP=1
+    ;;
+esac
+[[ "${UAP_SELF_PROTECT_OFF:-}" == "1" ]] && SHELL_OP=0
+```
+
+and the selector case:
+
+```bash
+must_fail_closed() {
+  case "$1" in
+    enforcement_self_protect) [[ "$SEC_SENSITIVE" == "1" ]] ;;
+    schema_diff_gate)         [[ "$COMMIT_OP" == "1" ]] ;;
+    plane_admin_protect)      [[ "$SHELL_OP" == "1" ]] ;;
+    *)                        false ;;
+  esac
+}
+```
+
+Scoping rationale (mirrors why `schema_diff_gate` is gated on `COMMIT_OP`): the
+plane-admin surface is mutable through the shell and nothing else. Arming the
+fail-closed net on every edit would turn a broken plane enforcer into a blanket
+block on all edits in the session. Non-shell tools keep failing open.
+`UAP_SELF_PROTECT_OFF=1` clears this arm too — otherwise the override named in
+the refusal is a no-op for the case that prints it.
+
+## Files changed
+
+- `templates/hooks/uap-policy-gate.sh` — +33 lines (SHELL_OP arm, selector case, rationale comments)
+- `.claude/hooks/uap-policy-gate.sh`, `.factory/hooks/uap-policy-gate.sh`, `.omp/hooks/uap-policy-gate.sh` — identical mirrors (drift test requirement; `uap worktree create` seeds from `templates/hooks/`)
+- `tools/agents/tests/test_gate_failclosed_plane_admin.py` — new, 15 tests
+- `package.json` — `test:enforcers` now registers `test_gate_failclosed_plane_admin` (additive; nothing removed)
+
+## Verification
+
+- `UAP_PROXY_ENV_AUTOLOAD=0 python3 -m unittest tools.agents.tests.test_gate_failclosed_plane_admin` → 15 tests OK
+- `tools.agents.tests.test_gate_failclosed_schema_diff` → 23 tests OK (drift sweep green)
+- All four tracked hook copies byte-identical to `templates/hooks/`; `bash -n` clean on each
+- No other fail-open behavior changed
+
+## Note on the pending-intent log
+
+The earlier recorded intent `ts=1790903849` (a `_pt_for` path-resolution change to
+the gate) was **not** applied. It was archived to `.uap/pending-deliver.applied.jsonl`
+with an operator note recording that it was deferred deliberately: the delivered
+hardening is the `must_fail_closed` arm, not the path-resolution rewrite. 38 other
+intents remain pending and visible.

package/docs/reports/2026-10-02-plane-admin-protect-session.md

@@ -0,0 +1,116 @@
+# Session summary — Plane Admin Protect enforcer (hardened + shipped to force)
+
+Date: 2026-10-02. Repo: `universal-agent-protocol` (UAP). Consuming project: `pay2u`.
+
+## Why
+
+The Plane admin account `admin@pay2u.com.au` (droplet `134.199.173.65`,
+`/opt/plane/plane-app`) had its credentials mutated by an agent command
+(`docker exec plane-app-api-1 python manage.py changepassword …`). A guard
+policy — `plane-admin-protect` — existed but was weak: no expiry on its waiver,
+no fail-closed behaviour, no tests, and the copy the gate actually executes had
+drifted from the source.
+
+## What was built
+
+### 1. Hardened enforcer — `src/policies/enforcers/plane_admin_protect.py` (132 lines)
+
+- Shell ops only (`bash`/`run_bash`/`shell`/`execute_command`/`terminal`); everything
+  else allows.
+- Blocks only when a command **both** targets the Plane stack **and** carries a
+  credential-mutation verb:
+  - targets: `plane-app`, `134.199.173.65`, `/opt/plane`, `plane-db`
+  - verbs: `reset_password`, `set_password`, `changepassword`, `createsuperuser`, `passwd`
+- Scans via `scannable_command()` so quoted payloads are visible.
+- Owner waiver `policies/waivers/plane-admin-reset` must carry `expires: YYYY-MM-DD`:
+  - valid → allow, with `waiver valid until <date>`
+  - expired / missing expiry / **unreadable** → **fails CLOSED** (`allowed:false`,
+    exit 2, `route:"owner-waiver"`, `waiverHint`) — never a crash (exit 1), never a
+    silent allow.
+- Documents its own limit: text scan raises cost, it is not a boundary (see below).
+
+### 2. Policy schema doc — `src/policies/schemas/policies/plane-admin-protect.md`
+
+Category `safety`, level `REQUIRED`, stage `pre-exec`, default `on`, tags
+`plane, credentials, admin, safety, droplet, owner-waiver`. States the rule, the
+target/verb lists, and the allowed read-only cases.
+
+### 3. Tests — `tools/agents/tests/test_plane_admin_protect.py` (10 tests, all green)
+
+Observed incident spelling blocked · `bash -c` wrapper blocked · non-shell op
+allowed · read-only Plane command allowed · **prose** naming a verb NOT blocked
+(`git commit -m "refused to passwd /opt/plane admin"`, `uap memory store …`) ·
+heredoc body naming a verb NOT blocked · valid waiver allows · expired waiver
+fails closed · missing expiry fails closed · unreadable waiver fails closed
+without crashing.
+
+Registered in `package.json` → `test:enforcers` (the suite-coverage guard
+`test_enforcer_suite_coverage` passes, so the module is genuinely in the gate).
+
+## Verification
+
+- `python3 -m unittest …test_plane_admin_protect` → 10/10 OK.
+- `npm run test:enforcers` → **Ran 1348 tests, OK (skipped=1)**.
+- `npm run build` (tsc) → clean.
+- Indirection matrix (15 real commands, temp repo root): blocked — heredoc that
+  writes the mutation script, `T=…; V=…` variable indirection, `printf … | sh`,
+  `docker compose exec`, `kubectl exec`, `ansible plane-app`. Allowed —
+  `bash r.sh` (script contents invisible), base64 payload, `ssh host '…'` with no
+  shell-exec token, `ansible all -m shell -a …` (no target token), read-only
+  Plane commands, prose in commit messages / memory stores, target-only, verb-only.
+  Net: the guard catches the realistic accidental paths; deliberate evasion via
+  an indirect payload is out of scope and is stated in the enforcer docstring.
+- End-to-end through the real gate in `pay2u`
+  (`.claude/hooks/uap-policy-gate.sh`, `tool_name`/`tool_input` payload):
+  mutation command → **blocked** with the owner-waiver hint; read-only Plane
+  commands → allowed (only the unrelated `expert-review-required` gate fires).
+
+## Deployed to force in pay2u
+
+`uap policy add-tool -p 7c26d0f7-3adc-4b36-9ebc-5ea554b6a687 -t plane_admin_protect -c <src>`
+run from the `pay2u` root. Result:
+
+- `.policy-tools/7c26d0f7-…_plane_admin_protect.py` sha `63841c556f3ba71c` == source sha.
+- `policies.db` `executable_tools.code` byte-identical to the materialized copy.
+- `.integrity.sha256` manifest rewritten to match (the gate verifies and
+  self-repairs against it, so the fix is now actually in force, not just merged).
+
+Re-verified end-to-end through the live gate in `pay2u`:
+
+| payload | exit | verdict |
+| --- | --- | --- |
+| `docker exec plane-app-api-1 python manage.py createsuperuser` | 2 | blocked (owner-waiver hint) |
+| `kubectl logs deploy/plane-app-worker -n plane --tail=50` | 0 | allowed |
+
+Canonical row is now `709d51b3-…` / `plane-admin-protect` (active, matches the
+built-in matrix slug, hardened code in force); `7c26d0f7-…` (`Plane Admin
+Protect`) is inactive. Both materialized copies are identical to source.
+
+## Open items (not done)
+
+1. **No waiver exists**: neither `pay2u/policies/waivers/` nor the UAP root has
+   `plane-admin-reset`. That is correct default posture (owner approval required);
+   create it only with explicit owner instruction and a real `expires:` date.
+2. **UAP changes are uncommitted** (`package.json` modified; enforcer, schema doc
+   and test untracked). Commit + push not performed — needs an explicit go-ahead.
+3. **Gate fail-open for this enforcer**: `must_fail_closed()` in
+   `.claude/hooks/uap-policy-gate.sh` fail-closes only for
+   `enforcement_self_protect` (when sensitive) and `schema_diff_gate` (on
+   commit/push). A plane enforcer that errors or is missing is read as ALLOW.
+   The integrity self-repair covers the missing/stale-copy path; an import-time
+   crash of this enforcer specifically would still pass the command.
+4. **Known, documented evasion surface** (accepted, not a boundary): `bash r.sh`
+   where `r.sh` holds the mutation, base64 payloads, and `ssh host '<verb>'`
+   with no shell-exec token. Text scan is a cost-raiser, stated in the enforcer
+   docstring.
+5. `pay2u` gate blocks inline `UAP_NO_REVIEW=1`; a review artifact or waiver is
+   needed before shipping there.
+
+## Resolved since first draft
+
+- Duplicate policy rows: `709d51b3-…` (`plane-admin-protect`, the built-in slug)
+  is the single active row; `7c26d0f7-…` is inactive. No double verdict.
+- Stale `rtk_wrap` manifest entry: no longer present in
+  `pay2u/.policy-tools/.integrity.sha256`, no `rtk_wrap.py` in `.policy-tools/`,
+  and the last `restored: …rtk_wrap.py` log line is ~6 days old — the per-call
+  restore churn has stopped.