@hasna/loops 0.4.13 → 0.4.22

package/docs/USAGE.md

@@ -15,12 +15,13 @@ It supports deterministic command loops, JSON-defined workflows, and guarded CLI
 
 OpenLoops defaults to `local`, where SQLite in `LOOPS_DATA_DIR` is
 authoritative and `loops-daemon` executes scheduled work. The package also
-defines `self_hosted` and `cloud` contracts for future non-local control
-planes:
+defines `self_hosted` and `cloud` contracts for non-local control planes:
 
-- `self_hosted`: user-operated `loops-api` control-plane contract; this
-  release exposes storage-backed API/runner foundations plus local migration
-  previews.
+- `self_hosted`: the Hasna-owned AWS/RDS control-plane deployment, served by
+  `loops-serve` and backed by Postgres, with the embeddable `loops-api` contract
+  shared by serve, SDK, and tests. This release exposes storage-backed `/v1`
+  loop CRUD and run listing, runner registration/claim/heartbeat/finalize
+  foundations, and local migration previews.
 - `cloud`: hosted control-plane contract; this release exposes client/runner
   status only, and requires `LOOPS_CLOUD_API_URL` plus `LOOPS_CLOUD_TOKEN` or
   `HASNA_LOOPS_CLOUD_TOKEN` before status can report ready.
@@ -29,13 +30,16 @@ Scheduler state is explicit in status JSON. `schedulerState.localStore` is
 SQLite plus local run artifact files: authoritative in `local`, cache/spool in
 non-local modes. `schedulerState.remoteStore` names the non-local contract
 (`api_control_plane_contract`, `postgres_contract`, or
-`hosted_control_plane_contract`) and reports `applySupported=false` because this
-public package does not directly mutate remote Postgres, S3/object storage, AWS
-resources, or hosted credentials. Route admission remains bounded by
-`max_dispatch`, `max_active`, `max_active_per_project`,
-`max_active_per_project_group`, `max_active_scope`, and `max_per_profile`.
+`hosted_control_plane_contract`). The standalone `loops` CLI reports
+`applySupported=false` for non-local apply because it does not perform
+id-preserving remote migration, S3/object storage mutation, AWS resource
+mutation, or hosted credential mutation. `loops-serve` mutates self-hosted
+Postgres for normal control-plane CRUD and runner protocol routes when it is
+explicitly configured. Route admission remains bounded by `max_dispatch`,
+`max_active`, `max_active_per_project`, `max_active_per_project_group`,
+`max_active_scope`, and `max_per_profile`.
 
-Useful status commands:
+Useful status and setup commands:
 
 ```bash
 loops mode
@@ -48,6 +52,8 @@ loops self-hosted runner-register --runner-id <id> --machine-id <machine>
 loops self-hosted runner-register --runner-id <id> --machine-id <machine> --apply
 loops cloud status
 loops-api status
+loops-serve version
+HASNA_LOOPS_DATABASE_URL=... loops-serve migrate --dry-run
 loops-runner status
 ```
 
@@ -93,14 +99,16 @@ The preview may inspect `LOOPS_API_URL`/`HASNA_LOOPS_API_URL`, but it refuses
 remote apply because normal loop CRUD would generate new ids. Use
 `loops self-hosted runner-register` to verify runner registration against an
 API, then use `loops-runner run-once` for the current bounded non-workflow
-claim/execute/finalize protocol.
+claim/execute/finalize protocol. `loops-serve migrate` applies the Postgres
+schema and `api_keys` table for a self-hosted control-plane host; the standalone
+CLI migration previews still do not perform id-preserving remote apply.
 Runner registration is preview-only unless `--apply` is present.
 
 ## Install
 
 **OpenLoops requires the [Bun](https://bun.sh) runtime (`bun >= 1.0`).** The
-installed binaries are `loops`, `loops-daemon`, `loops-api`, `loops-runner`, and
-`loops-mcp`; each uses a `#!/usr/bin/env bun` shebang.
+installed binaries are `loops`, `loops-daemon`, `loops-api`, `loops-serve`,
+`loops-runner`, and `loops-mcp`; each uses a `#!/usr/bin/env bun` shebang.
 
 From npm:
 
@@ -208,6 +216,23 @@ failures are recorded as failed loop runs with a `runtime preflight failed`
 error, so health/routing checks can create follow-up tasks without spawning the
 worker.
 
+For controlled launches, scheduled todos-task drains can be guarded by blocker
+tasks:
+
+```bash
+loops routes schedule todos-task platform-drain \
+  --every 5m \
+  --todos-project /path/to/source/todos-project \
+  --tags auto:route \
+  --launch-gate "pa19-controlled-launch" \
+  --launch-gate-blocker "/path/to/open-codewith::2d9d931b" \
+  --launch-gate-blocker "/path/to/open-loops::816e99db" \
+  --worktree-mode required
+```
+
+The drain creates zero worker loops while any blocker task is not completed.
+Each blocked tick writes launch-gate evidence and leaves source tasks untouched.
+
 Run a Claude loop every morning:
 
 ```bash
@@ -418,6 +443,13 @@ for those. Set `"blockedExitCodes": []` on a gate-named step to opt out.
 
 ## Templates And Task Events
 
+OpenLoops is the runtime/scheduler/workflow engine for these flows — not the
+automation domain model. Todos-task routes are OpenLoops-native admission; they
+do not replace the OpenAutomations product queue. See
+[Runtime Boundary](./RUNTIME_BOUNDARY.md) for ownership split and external
+compiler handoff paths (`@hasna/automations` claim-queue,
+`@hasna/actions` planned upsert-one-shot).
+
 Built-in templates turn common orchestration flows into reusable workflow JSON.
 `todos-task-worker-verifier` performs one todos task and then verifies it.
 `event-worker-verifier` handles any Hasna event envelope and then verifies the
@@ -425,7 +457,11 @@ handling. `bounded-agent-worker-verifier` is for recurring bounded agent work:
 one worker runs a narrow objective, then a fresh verifier audits the result.
 The catalog also includes `task-lifecycle`, `pr-review`, `scheduled-audit`,
 `knowledge-refresh`, `report-only`, `incident-response`, and
-`deterministic-check-create-task` for common operator workflows.
+`deterministic-check-create-task` for common operator workflows. Use
+`routing-remediation` for bounded routing-doctor repair runs: it dry-runs by
+default, gates `safe_auto` capacity, applies only supported
+`todos doctor routing --apply` repairs when explicitly enabled, and files
+blocker tasks for human/cross-repo/unsupported findings.
 
 ```bash
 loops templates list
@@ -459,6 +495,12 @@ loops templates render pr-review \
 loops templates render deterministic-check-create-task \
   --var projectPath=/path/to/repo \
   --var checkCommand='your deterministic check and todos upsert command'
+loops templates render routing-remediation \
+  --var projectPath=/path/to/repo \
+  --var todosProjectPath=/path/to/todos-project \
+  --var shard=0/6 \
+  --var maxRepairs=25 \
+  --var idempotencyKey=routing-health:repo:shard0
 ```
 
 Custom reusable workflow templates live under the OpenLoops app data directory:
@@ -478,21 +520,39 @@ worker finishes; pass `--verifier-idle-timeout none` or template variable
 `verifierIdleTimeoutMs=none` only when another heartbeat is guaranteed. Use a
 positive numeric `timeoutMs` only when an agentic step is intentionally bounded.
 
-To migrate existing workflow loops, do not edit `workflow_specs.steps_json`
-directly because historical workflow runs must keep pointing at their original
-spec. Use the append-only migrator:
+To migrate existing agentic loops, use the timeout migrator instead of editing
+the database directly. Workflow loops are migrated append-only because
+historical workflow runs must keep pointing at their original spec; direct
+agent loops selected with `--loop` update their stored target in place for
+future executions:
 
 ```bash
 loops workflows migrate-agent-timeouts --loop <loop-id-or-name>
 loops workflows migrate-agent-timeouts --loop <loop-id-or-name> --apply
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name>
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name> --apply
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name> --apply --archive-old
 ```
 
-The command dry-runs by default. With `--apply`, it creates a new workflow spec
-with the requested agent timeout policy, retargets only future executions of
-eligible non-running workflow loops, and leaves terminal timed-out workflow runs
-as audit history. Use `loops workflows recover` only for interrupted `running`
-workflow runs whose recorded child process is gone; terminal `timed_out` runs
-must be requeued with `loops routes requeue <work-item-id> --reason "<cause fixed>"` before
+Both migrators dry-run by default. For eligible non-running workflow loops,
+`--apply` creates a new workflow spec and retargets only future executions;
+historical workflow runs keep pointing at their original spec. For direct agent
+loops selected with `--loop`, `migrate-agent-timeouts --apply` updates the
+stored target in place for future executions. Use `--archive-old` to archive
+superseded workflow specs when no active loops still reference them.
+
+`migrate-agent-timeouts` clones the workflow with the requested agent timeout
+policy (`--timeout none` by default). `migrate-goal-wrappers` targets loops that
+define both a loop-level goal and a redundant workflow-level top-level goal: it
+clones a goal-free workflow spec, retargets the loop, and leaves the loop-level
+goal as the sole orchestration wrapper. Loops with only a workflow-level goal or
+only a loop-level goal are skipped. New workflow loops cannot combine both
+wrappers; use loop-level `--goal` on `loops create workflow` instead of nesting
+a top-level `"goal"` in the workflow JSON.
+
+Use `loops workflows recover` only for interrupted `running` workflow runs whose
+recorded child process is gone; terminal `timed_out` runs must be requeued with
+`loops routes requeue <work-item-id> --reason "<cause fixed>"` before
 re-delivering or draining the original task/event route.
 
 ```json
@@ -561,12 +621,14 @@ on the remote machine. The generated agent target includes worktree metadata
 inspection expose the exact checkout.
 
 Before a worker starts, worktree preparation verifies that any existing
-managed path is a real git worktree with the same top-level checkout, the same
-git common directory as the source repo, and the expected generated branch. A
-detached HEAD or unexpected branch fails closed with evidence
-(`worktreeMode=required`) instead of silently running a mutating workflow in
-the wrong state; `worktreeMode=auto` falls back to the original checkout and
-records the fallback.
+managed path is a real git worktree with the same top-level checkout and the
+same git common directory as the source repo. If the checkout is on a detached
+HEAD or unexpected branch, OpenLoops only reattaches it to the expected
+generated branch when the worktree is clean. Dirty worktrees, symlinked paths,
+non-worktree paths, different common dirs, and unrecoverable branch switches
+fail closed with cleanup evidence (`worktreeMode=required`) instead of
+silently running a mutating workflow in the wrong state; `worktreeMode=auto`
+falls back to the original checkout and records the fallback.
 
 Use explicit main/default checkout mode only when the task truly requires it:
 
@@ -610,6 +672,9 @@ blocked, completed/done, cancelled/canceled, failed, archived, manual,
 approval-required, or `no-auto` tasks. This guard exists even when the upstream
 `@hasna/events` webhook filter is misconfigured, so task existence alone is not
 permission to execute agent work.
+Route work items are the durable reservation ledger: they include the stable
+idempotency key, task/event references, project/group keys, admitting machine
+ID, route scope, workflow/loop IDs, and the current terminal or active status.
 
 Task route drains can select providers from task metadata instead of running one
 fixed provider/account pool for the whole drain. Add one or more
@@ -619,13 +684,20 @@ matching rule wins. Rule profiles become a Codewith auth-profile pool for
 can also carry `provider_hint`/`route_provider`, `auth_profile_pool`, or
 `account_pool` metadata. Dry-run, drain evidence, and route invocation scope
 include `providerRouting` so operators can see why a provider/account was
-selected.
+selected. Selector values may contain `:`; the parser treats the colon before a
+supported provider id as the route delimiter, so exact tag selectors such as
+`tags=area:frontend:claude:account003,account015` and
+`tags=provider:claude-code:claude:account003,account015` match literal task
+tags `area:frontend` and `provider:claude-code`.
 
 ```bash
 loops routes drain todos-task \
   --dry-run \
+  --provider-rule tags=area:frontend:claude:account003,account015 \
+  --provider-rule tags=provider:claude-code:claude:account003,account015 \
   --provider-rule area=frontend:claude:claude-ui-a,claude-ui-b \
   --provider-rule area=backend:codewith:account001,account002 \
+  --provider-rule tags=task-lifecycle:codewith:account001,account002 \
   --worktree-mode required
 ```
 
@@ -703,6 +775,10 @@ equivalent directory-scoped write controls.
 Inspect route state with:
 
 ```bash
+loops routes policies list
+loops routes policies show oss
+loops routes policies render oss
+loops routes policies validate
 cat task-created-event.json | loops routes preview todos-task --sandbox workspace-write
 cat task-created-event.json | loops routes create todos-task --sandbox workspace-write
 loops routes drain todos-task --task-list oss --max-dispatch 2 --compact
@@ -713,6 +789,29 @@ loops routes requeue <work-item-id> --reason "fixed upstream blocker"
 loops routes invocations
 ```
 
+Compact drain output includes route reservation IDs, work-item status, machine
+ID, workflow ID, loop ID, and route scope for each considered task.
+
+Named route policies expand the long recurring drain commands used by the live
+task routers into explicit, replayable options. `repoops-pr-queue`, `oss`,
+`pilot`, and `machine-sync` are built in. Operators can inspect the policy,
+render the exact `loops --json routes drain todos-task ...` command, and validate
+that the rendered args no longer depend on `--policy`/`--preset`:
+
+```bash
+loops routes policies render oss
+loops routes schedule todos-task machine-oss-task-lifecycle-router --policy oss
+```
+
+Scheduled policy routes store explicit drain args plus
+`--route-policy-evidence <id>`, so future runs remain auditable even if a policy
+definition changes later. Policy drains and dry-runs include `routePolicy`
+evidence with the source, safety class, guards, expanded options, and rendered
+args. Passing a conflicting explicit option fails before the route is created.
+The `pilot` policy uses `sandbox=danger-full-access` and is treated as a paused
+manual break-glass lane; applying it requires the operator to pass
+`--manual-break-glass` explicitly.
+
 When a workflow run starts from an admitted work item, OpenLoops writes a
 manifest under:
 
@@ -744,6 +843,8 @@ loops routes drain todos-task \
   --max-active-per-project 1 \
   --max-active-per-project-group 4 \
   --max-active 12 \
+  --provider-active-cap 6 \
+  --provider-admission-check \
   --worktree-mode required \
   --evidence-dir "$HOME/.hasna/loops/reports/task-drain"
 ```
@@ -761,6 +862,16 @@ when requested, and leaves excess ready tasks in todos for a later drain pass.
 Use `--dry-run` to preview candidates and rendered workflows without mutating
 OpenLoops state.
 
+The route throttle flags count OpenLoops routed workflow work items, not the
+live background-agent slots inside a provider. For Codewith drains, add
+`--provider-active-cap <n>` (or `--codewith-active-cap <n>`) so each candidate
+checks `codewith agent diagnostics --json` before workflow-loop creation and
+defers when `activeRunCount >= n`. Use `--provider-admission-check` when the
+drain should also fail closed on diagnostics errors, unsupported providers, or
+Codewith reports with no available admission slots. Backlog prioritizer and
+drain loops should use these first-class flags rather than a shell guard around
+`codewith agent diagnostics`.
+
 For an OSS task-created route, keep the drain deterministic and narrow:
 
 ```bash
@@ -779,6 +890,8 @@ loops routes schedule todos-task oss-task-route-drain \
   --max-active-per-project 1 \
   --max-active-per-project-group 4 \
   --max-active 12 \
+  --provider-active-cap 6 \
+  --provider-admission-check \
   --worktree-mode required \
   --evidence-dir "$HOME/.hasna/loops/reports/oss-task-route-drain" \
   --compact
@@ -789,11 +902,13 @@ in with the `auto:route` tag, `route_enabled=true`, or
 `automation.allowed=true` should be routed. Keep repo-mutating worker/verifier
 runs on a Codewith account pool with `--worktree-mode required`. Do not dispatch
 or paste task prompts into tmux panes. Use max-active throttles and
-`--max-dispatch` to bound agent fan-out, and write compact evidence into a
-bounded reports directory so operators can audit each drain without unbounded
-stdout or loop history growth. Keep `--scan-limit` large enough for the current
-ready-task backlog; if the scan is exhausted before matching tasks appear, the
-drain will correctly do no work.
+`--max-dispatch` to bound OpenLoops agent fan-out, and use
+`--provider-active-cap` plus `--provider-admission-check` to bound live Codewith
+background-agent admission. Write compact evidence into a bounded reports
+directory so operators can audit each drain without unbounded stdout or loop
+history growth. Keep `--scan-limit` large enough for the current ready-task
+backlog; if the scan is exhausted before matching tasks appear, the drain will
+correctly do no work.
 
 Generated task/event route workflow specs are lifecycle-managed. After a
 generated one-shot route workflow run reaches `succeeded`, `failed`,
@@ -829,6 +944,27 @@ is requested without `manualBreakGlass=true`. Use `workspace-write` for
 unattended task/event routes. Full access is an explicit manual emergency path,
 not a default automation mode.
 
+## Run Receipts
+
+Use run receipts when an agent, scheduler, route, or external workflow needs a
+stable run outcome without parsing raw stdout or wrapper-script text. Receipts
+are scheduler-neutral JSON records with these public snake_case fields:
+`loop_id`, `run_id`, `machine`, `repo`, `task_ids`, `knowledge_ids`,
+`digest_id`, `started_at`, `finished_at`, `status`, `exit_code`, `summary`, and
+`evidence_paths`. The summary contains bounded, scrubbed excerpts and byte
+counts; raw unbounded stdout/stderr stays out of the receipt contract.
+
+```bash
+cat receipt.json | loops --json receipts write --file -
+loops --json receipts read run_123
+loops --json receipts list --loop-id loop_123 --task-id task_123
+```
+
+MCP clients can use `loops_receipt_read`, `loops_receipts_list`, and the
+mutation-gated `loops_receipt_write`. The SDK exposes `writeReceipt`,
+`receipt`, and `receipts`; the HTTP API exposes `POST /v1/receipts`,
+`GET /v1/receipts/{runId}`, and `GET /v1/receipts`.
+
 ## Transcript-Driven Loops
 
 OpenLoops can turn long-form media or meeting transcripts into recurring workflow work when paired with `iapp-transcriber`. The template at `docs/workflows/transcript-feedback-to-loops.json` transcribes an authorized media URL, asks an agent to extract recurring loop candidates, authors workflow specs, and validates generated workflows before scheduling. Copy it into the target repo, replace `/path/to/repo` with that repo's absolute path, and provide `TRANSCRIBER_SOURCE_URL` through the runner environment or a private, uncommitted workflow copy before storing or scheduling it. Do not commit private or signed media URLs.
@@ -865,25 +1001,46 @@ agent-run failures for default-loop SLOs:
 
 ```bash
 loops health --json
+loops health scan --include active,paused --latest-run --doctor --daemon --json
 loops expectations <loop-id-or-name> --json
 ```
 
 The JSON contains the expectation result, bounded error/stdout/stderr evidence,
 a stable failure fingerprint, route metadata, and recommended task fields.
-OpenLoops does not mutate Todos from `health` or `expectations`. To turn failed
-expectations into deduped tasks, use the explicit routing command:
+OpenLoops does not mutate Todos from `health`, `expectations`, or read-only
+`health scan`. To turn failed expectations or scan findings into deduped tasks,
+use an explicit mutating command:
 
 ```bash
 loops health route-tasks \
   --project ~/.hasna/loops \
   --task-list loop-error-self-heal \
   --max-actions 5
+
+loops health scan \
+  --include active,paused \
+  --latest-run \
+  --doctor \
+  --daemon \
+  --upsert-todos \
+  --dry-run \
+  --max-actions 5 \
+  --evidence-dir ~/.hasna/loops/reports/health-scan
 ```
 
 Use `--dry-run --json` first when testing a new automation path. Routed tasks
 include the stable failure fingerprint, classification, loop id/name, and
 `no_tmux_dispatch=true` metadata.
 
+`health scan` replaces local loop-error self-heal scripts with package-owned
+CLI/SDK/MCP primitives. It inventories included loop statuses, detects daemon,
+doctor, preflight, latest-run, and stale-running issues, writes bounded
+`summary.json` and `report.md` files under
+`$LOOPS_DATA_DIR/reports/health-scan` or `--report-dir`/`--evidence-dir`, and
+keeps output compact. It is read-only by default. The only safe self-heal is
+`--start-daemon`, which starts the daemon only when status proves it is not
+running; it does not stop, resume, archive, delete, or reap loops.
+
 Use `--evidence-dir <dir>` when a deterministic loop needs a compact JSON
 heartbeat/report on disk. Use `--auto-route` only on task lists that should feed
 the task-created headless worker/verifier workflow; it adds the `auto:route`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hasna/loops",
-  "version": "0.4.13",
+  "version": "0.4.22",
   "description": "Persistent local loop and workflow runner for deterministic commands and headless AI coding agents",
   "type": "module",
   "main": "dist/index.js",
@@ -9,6 +9,7 @@
     "loops": "dist/cli/index.js",
     "loops-daemon": "dist/daemon/index.js",
     "loops-api": "dist/api/index.js",
+    "loops-serve": "dist/serve/index.js",
     "loops-runner": "dist/runner/index.js",
     "loops-mcp": "dist/mcp/index.js"
   },
@@ -21,6 +22,14 @@
       "types": "./dist/sdk/index.d.ts",
       "import": "./dist/sdk/index.js"
     },
+    "./sdk/http": {
+      "types": "./dist/sdk/http.d.ts",
+      "import": "./dist/sdk/http.js"
+    },
+    "./serve": {
+      "types": "./dist/serve/index.d.ts",
+      "import": "./dist/serve/index.js"
+    },
     "./mcp": {
       "types": "./dist/mcp/index.d.ts",
       "import": "./dist/mcp/index.js"
@@ -66,7 +75,7 @@
     "LICENSE"
   ],
   "scripts": {
-    "build": "rm -rf dist && bun build src/cli/index.ts src/daemon/index.ts src/api/index.ts src/runner/index.ts src/mcp/index.ts src/index.ts src/sdk/index.ts src/lib/store.ts src/lib/mode.ts src/lib/storage/index.ts src/lib/storage/contract.ts src/lib/storage/sqlite.ts src/lib/storage/postgres.ts src/lib/storage/postgres-schema.ts --root src --outdir dist --target bun --packages external && chmod +x dist/cli/index.js dist/daemon/index.js dist/api/index.js dist/runner/index.js dist/mcp/index.js && tsc -p tsconfig.build.json --emitDeclarationOnly --outDir dist",
+    "build": "rm -rf dist && bun build src/cli/index.ts src/daemon/index.ts src/api/index.ts src/serve/index.ts src/runner/index.ts src/mcp/index.ts src/index.ts src/sdk/index.ts src/sdk/http.ts src/lib/store.ts src/lib/mode.ts src/lib/storage/index.ts src/lib/storage/contract.ts src/lib/storage/sqlite.ts src/lib/storage/postgres.ts src/lib/storage/postgres-schema.ts --root src --outdir dist --target bun --packages external && chmod +x dist/cli/index.js dist/daemon/index.js dist/api/index.js dist/serve/index.js dist/runner/index.js dist/mcp/index.js && tsc -p tsconfig.build.json --emitDeclarationOnly --outDir dist",
     "build:bin": "bun build src/cli/index.ts --compile --outfile dist/loops",
     "typecheck": "tsc --noEmit",
     "test": "bun test",
@@ -106,13 +115,16 @@
     "@openrouter/ai-sdk-provider": "2.9.1",
     "ai": "6.0.204",
     "commander": "^13.1.0",
+    "pg": "^8.13.1",
     "zod": "4.4.3"
   },
   "optionalDependencies": {
-    "@hasna/machines": "0.0.49"
+    "@hasna/machines": "0.0.49",
+    "@hasna/contracts": "^0.4.2"
   },
   "devDependencies": {
     "@types/bun": "latest",
+    "@types/pg": "^8.11.10",
     "typescript": "^5.7.3"
   },
   "publishConfig": {