@hasna/loops 0.4.12 → 0.4.14

package/dist/types.d.ts

@@ -365,6 +365,9 @@ export interface Loop {
     retryDelayMs: number;
     leaseMs: number;
     expiresAt?: string;
+    latestRunId?: string;
+    latestRunStatus?: RunStatus;
+    lastRunAt?: string;
     createdAt: string;
     updatedAt: string;
 }

package/docs/CUTOVER-RUNBOOK.md

@@ -0,0 +1,61 @@
+# Loops Postgres Cutover Runbook
+
+Status: **backend foundation landed; daemon still runs on local sqlite.** Do NOT
+flip the running daemon to Postgres until every step below is green.
+
+PURE REMOTE (Amendment A1): in cloud mode, reads AND writes hit cloud Postgres
+directly. There is no hybrid/sync/cache mode. After a verified migration the
+local sqlite file is renamed to `<name>.db.pre-cloud-2026-07-06.bak`.
+
+## What shipped in this PR
+
+- Vendored `@hasna/contracts` storage kit → `src/generated/storage-kit/`
+  (pool/query/tls/mode/migrations/health). Regenerate: `bunx @hasna/contracts vendor-kit`.
+- `src/lib/storage/pg-executor.ts` — `PgPoolExecutor` adapting the vendored
+  `pg.Pool` client to `PostgresQueryExecutor` (drives `PostgresStorage.migrate`).
+- `src/lib/storage/pg-runner-claim.ts` — `claimNextRun` / `heartbeatRunLease`
+  using `SELECT ... FOR UPDATE SKIP LOCKED` for concurrent runners.
+- `src/lib/storage/postgres-concurrency.test.ts` — two-connection race test
+  (skips unless `LOOPS_TEST_DATABASE_URL` is set; verified green against a
+  throwaway local Postgres 16).
+- `pg` promoted to a direct dependency; `@types/pg` added as devDependency.
+
+## Remaining before daemon cutover (tracked, NOT done here)
+
+1. **Full `PostgresLoopStorage` implementing `LoopStorageContract`** (60+ methods
+   in `src/lib/storage/contract.ts`). Only the migration ledger + claim/lease
+   primitives exist today. The claim primitive here is the correctness core to
+   build the rest on.
+2. **Async consumer wiring.** All call sites construct `new Store()` synchronously
+   (`src/cli`, `src/daemon`, `src/sdk`, `src/mcp`, `src/lib/route`). PURE REMOTE
+   requires routing them through the async `LoopStorageContract` when
+   `LOOPS_DATABASE_URL` is set (mode resolved by `src/lib/mode.ts`, which stays
+   authoritative). `SqliteLoopStorage` already wraps the sync store for the
+   local path.
+3. **`loops migrate-local` command** (data migration TOOLING only): copy
+   definitions + schedules + last 30 days of runs from local sqlite into cloud
+   Postgres, then rename the sqlite file to the `.pre-cloud-2026-07-06.bak`
+   backup. Does NOT enable cloud mode.
+4. **Apply schema to shared RDS** via SSM tunnel on local port 15438 using the
+   `hasna/oss/loops/database-url-owner` secret, then run
+   `PostgresStorage(new PgPoolExecutor(...)).migrate()` (dry-run first). Kill the
+   tunnel and remove temp files afterward. Never run the test suite against RDS.
+
+## Enable steps (run only after 1–4 are green)
+
+1. Confirm `PostgresStorage.migrate({ dryRun: true })` reports all four
+   migrations already applied on the loops RDS database.
+2. Stop the loops daemon: `loops daemon stop`.
+3. `loops migrate-local --database-url "$LOOPS_DATABASE_URL"` (moves defs +
+   schedules + 30d of runs; verify counts; sqlite renamed to backup).
+4. Export `LOOPS_DATABASE_URL` (from `hasna/oss/loops/database-url`) and the
+   cloud storage-mode env for the daemon unit; restart the daemon.
+5. Verify `loops status` reports `deploymentMode: cloud`, `sourceOfTruth:
+   cloud_control_plane`, and that a test loop run claims + finalizes against
+   Postgres.
+
+## Rollback
+
+Cloud mode is off until `LOOPS_DATABASE_URL` is present in the daemon env.
+To roll back: unset it, restore the sqlite backup
+(`mv <name>.db.pre-cloud-2026-07-06.bak <name>.db`), restart the daemon.

package/docs/DEPLOYMENT_MODES.md

@@ -68,7 +68,7 @@ loops import ./loops-export.json --apply
 Human status output is intentionally compact:
 
 ```text
-deploymentMode=local active source=default truth=local_sqlite local=authoritative control_plane=none
+deploymentMode=local active source=default truth=local_sqlite local=authoritative scheduler=local_sqlite control_plane=none
 ```
 
 JSON uses these field names:
@@ -85,6 +85,22 @@ JSON uses these field names:
   presence signal.
 - `controlPlane.apiUrl`: a display-safe URL without credentials, query string,
   or fragment.
+- `schedulerState.localStore`: always names the local SQLite store and local
+  run artifact files. In `local` mode this store is authoritative; in
+  non-local modes it is a cache, offline spool, and audit copy.
+- `schedulerState.remoteStore`: names the remote scheduler contract:
+  `api_control_plane_contract`, `postgres_contract`,
+  `hosted_control_plane_contract`, `unconfigured`, or `none`. Remote apply is
+  `false` in this public package until a control-plane host wires a full
+  storage adapter.
+- `schedulerState.remoteStore.objectArtifacts`: `object_store_contract` means
+  remote artifact/object storage is a control-plane contract. The public package
+  does not create or mutate S3 buckets, AWS resources, or hosted credentials.
+- `schedulerState.routeAdmission`: names the active route-state store and the
+  bounded gates (`max_dispatch`, `max_active`, `max_active_per_project`,
+  `max_active_per_project_group`, `max_active_scope`, and `max_per_profile`).
+  Live active counts use admitted/running work items; dry-runs do not open or
+  migrate the live store to compute counts.
 
 `loops-api` is a separate process in the same public package. It is not a
 separate package at this stage because self-hosted users and the hosted service
@@ -150,6 +166,12 @@ would generate new ids, so it is not a no-loss migration. Local SQLite remains
 authoritative until a safe import is applied; in non-local modes it may remain
 a cache, offline spool, and audit copy.
 
+`LOOPS_DATABASE_URL` selects the self-hosted Postgres scheduler-state contract,
+but it does not make the standalone CLI mutate a remote database by itself.
+Remote execution still flows through a configured control-plane API and runner
+protocol. `loops-runner` needs `LOOPS_API_URL` or `HASNA_LOOPS_API_URL` to claim
+work; a database URL alone is migration/readiness configuration.
+
 `loops self-hosted runner-register` is also preview-only unless `--apply` is
 present. The dry run prints the runner id, machine id, labels, and
 capabilities that would be posted, without exposing tokens.

package/docs/USAGE.md

@@ -25,6 +25,16 @@ planes:
   status only, and requires `LOOPS_CLOUD_API_URL` plus `LOOPS_CLOUD_TOKEN` or
   `HASNA_LOOPS_CLOUD_TOKEN` before status can report ready.
 
+Scheduler state is explicit in status JSON. `schedulerState.localStore` is
+SQLite plus local run artifact files: authoritative in `local`, cache/spool in
+non-local modes. `schedulerState.remoteStore` names the non-local contract
+(`api_control_plane_contract`, `postgres_contract`, or
+`hosted_control_plane_contract`) and reports `applySupported=false` because this
+public package does not directly mutate remote Postgres, S3/object storage, AWS
+resources, or hosted credentials. Route admission remains bounded by
+`max_dispatch`, `max_active`, `max_active_per_project`,
+`max_active_per_project_group`, `max_active_scope`, and `max_per_profile`.
+
 Useful status commands:
 
 ```bash
@@ -468,21 +478,39 @@ worker finishes; pass `--verifier-idle-timeout none` or template variable
 `verifierIdleTimeoutMs=none` only when another heartbeat is guaranteed. Use a
 positive numeric `timeoutMs` only when an agentic step is intentionally bounded.
 
-To migrate existing workflow loops, do not edit `workflow_specs.steps_json`
-directly because historical workflow runs must keep pointing at their original
-spec. Use the append-only migrator:
+To migrate existing agentic loops, use the timeout migrator instead of editing
+the database directly. Workflow loops are migrated append-only because
+historical workflow runs must keep pointing at their original spec; direct
+agent loops selected with `--loop` update their stored target in place for
+future executions:
 
 ```bash
 loops workflows migrate-agent-timeouts --loop <loop-id-or-name>
 loops workflows migrate-agent-timeouts --loop <loop-id-or-name> --apply
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name>
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name> --apply
+loops workflows migrate-goal-wrappers --loop <loop-id-or-name> --apply --archive-old
 ```
 
-The command dry-runs by default. With `--apply`, it creates a new workflow spec
-with the requested agent timeout policy, retargets only future executions of
-eligible non-running workflow loops, and leaves terminal timed-out workflow runs
-as audit history. Use `loops workflows recover` only for interrupted `running`
-workflow runs whose recorded child process is gone; terminal `timed_out` runs
-must be requeued with `loops routes requeue <work-item-id> --reason "<cause fixed>"` before
+Both migrators dry-run by default. For eligible non-running workflow loops,
+`--apply` creates a new workflow spec and retargets only future executions;
+historical workflow runs keep pointing at their original spec. For direct agent
+loops selected with `--loop`, `migrate-agent-timeouts --apply` updates the
+stored target in place for future executions. Use `--archive-old` to archive
+superseded workflow specs when no active loops still reference them.
+
+`migrate-agent-timeouts` clones the workflow with the requested agent timeout
+policy (`--timeout none` by default). `migrate-goal-wrappers` targets loops that
+define both a loop-level goal and a redundant workflow-level top-level goal: it
+clones a goal-free workflow spec, retargets the loop, and leaves the loop-level
+goal as the sole orchestration wrapper. Loops with only a workflow-level goal or
+only a loop-level goal are skipped. New workflow loops cannot combine both
+wrappers; use loop-level `--goal` on `loops create workflow` instead of nesting
+a top-level `"goal"` in the workflow JSON.
+
+Use `loops workflows recover` only for interrupted `running` workflow runs whose
+recorded child process is gone; terminal `timed_out` runs must be requeued with
+`loops routes requeue <work-item-id> --reason "<cause fixed>"` before
 re-delivering or draining the original task/event route.
 
 ```json
@@ -674,11 +702,12 @@ idempotency key before rendering worktree plans or checking route limits. In
 dry-run mode, throttle counts are not evaluated because opening the live loop
 store can create or migrate the local database.
 Terminal routed work items such as failed, dead-letter, cancelled, or succeeded
-history remain deduped until an operator runs `loops routes requeue
-<work-item-id> --reason "<cause fixed>"`; a later drain should not create
-another worker just because the previous workflow ended. The next route-created
-output records `requeue` evidence with the previous work item id, previous
-attempts, operator reason, new attempt, workflow id, and loop id.
+history are re-admitted only when the todos task is still actionable, the
+per-attempt backoff has elapsed, and the redispatch cap has not been reached.
+Operators can still force a retry with `loops routes requeue <work-item-id>
+--reason "<cause fixed>"`. The next route-created output records `requeue`
+evidence with the previous work item id, previous attempts, reason, new attempt,
+workflow id, and loop id.
 
 When a sandboxed Codewith/Codex worker must update app stores outside the repo
 worktree, pass those stores explicitly with `--add-dir` or template `addDirs`.
@@ -854,25 +883,46 @@ agent-run failures for default-loop SLOs:
 
 ```bash
 loops health --json
+loops health scan --include active,paused --latest-run --doctor --daemon --json
 loops expectations <loop-id-or-name> --json
 ```
 
 The JSON contains the expectation result, bounded error/stdout/stderr evidence,
 a stable failure fingerprint, route metadata, and recommended task fields.
-OpenLoops does not mutate Todos from `health` or `expectations`. To turn failed
-expectations into deduped tasks, use the explicit routing command:
+OpenLoops does not mutate Todos from `health`, `expectations`, or read-only
+`health scan`. To turn failed expectations or scan findings into deduped tasks,
+use an explicit mutating command:
 
 ```bash
 loops health route-tasks \
   --project ~/.hasna/loops \
   --task-list loop-error-self-heal \
   --max-actions 5
+
+loops health scan \
+  --include active,paused \
+  --latest-run \
+  --doctor \
+  --daemon \
+  --upsert-todos \
+  --dry-run \
+  --max-actions 5 \
+  --evidence-dir ~/.hasna/loops/reports/health-scan
 ```
 
 Use `--dry-run --json` first when testing a new automation path. Routed tasks
 include the stable failure fingerprint, classification, loop id/name, and
 `no_tmux_dispatch=true` metadata.
 
+`health scan` replaces local loop-error self-heal scripts with package-owned
+CLI/SDK/MCP primitives. It inventories included loop statuses, detects daemon,
+doctor, preflight, latest-run, and stale-running issues, writes bounded
+`summary.json` and `report.md` files under
+`$LOOPS_DATA_DIR/reports/health-scan` or `--report-dir`/`--evidence-dir`, and
+keeps output compact. It is read-only by default. The only safe self-heal is
+`--start-daemon`, which starts the daemon only when status proves it is not
+running; it does not stop, resume, archive, delete, or reap loops.
+
 Use `--evidence-dir <dir>` when a deterministic loop needs a compact JSON
 heartbeat/report on disk. Use `--auto-route` only on task lists that should feed
 the task-created headless worker/verifier workflow; it adds the `auto:route`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hasna/loops",
-  "version": "0.4.12",
+  "version": "0.4.14",
   "description": "Persistent local loop and workflow runner for deterministic commands and headless AI coding agents",
   "type": "module",
   "main": "dist/index.js",
@@ -9,6 +9,7 @@
     "loops": "dist/cli/index.js",
     "loops-daemon": "dist/daemon/index.js",
     "loops-api": "dist/api/index.js",
+    "loops-serve": "dist/serve/index.js",
     "loops-runner": "dist/runner/index.js",
     "loops-mcp": "dist/mcp/index.js"
   },
@@ -21,6 +22,14 @@
       "types": "./dist/sdk/index.d.ts",
       "import": "./dist/sdk/index.js"
     },
+    "./sdk/http": {
+      "types": "./dist/sdk/http.d.ts",
+      "import": "./dist/sdk/http.js"
+    },
+    "./serve": {
+      "types": "./dist/serve/index.d.ts",
+      "import": "./dist/serve/index.js"
+    },
     "./mcp": {
       "types": "./dist/mcp/index.d.ts",
       "import": "./dist/mcp/index.js"
@@ -66,7 +75,7 @@
     "LICENSE"
   ],
   "scripts": {
-    "build": "rm -rf dist && bun build src/cli/index.ts src/daemon/index.ts src/api/index.ts src/runner/index.ts src/mcp/index.ts src/index.ts src/sdk/index.ts src/lib/store.ts src/lib/mode.ts src/lib/storage/index.ts src/lib/storage/contract.ts src/lib/storage/sqlite.ts src/lib/storage/postgres.ts src/lib/storage/postgres-schema.ts --root src --outdir dist --target bun --packages external && chmod +x dist/cli/index.js dist/daemon/index.js dist/api/index.js dist/runner/index.js dist/mcp/index.js && tsc -p tsconfig.build.json --emitDeclarationOnly --outDir dist",
+    "build": "rm -rf dist && bun build src/cli/index.ts src/daemon/index.ts src/api/index.ts src/serve/index.ts src/runner/index.ts src/mcp/index.ts src/index.ts src/sdk/index.ts src/sdk/http.ts src/lib/store.ts src/lib/mode.ts src/lib/storage/index.ts src/lib/storage/contract.ts src/lib/storage/sqlite.ts src/lib/storage/postgres.ts src/lib/storage/postgres-schema.ts --root src --outdir dist --target bun --packages external && chmod +x dist/cli/index.js dist/daemon/index.js dist/api/index.js dist/serve/index.js dist/runner/index.js dist/mcp/index.js && tsc -p tsconfig.build.json --emitDeclarationOnly --outDir dist",
     "build:bin": "bun build src/cli/index.ts --compile --outfile dist/loops",
     "typecheck": "tsc --noEmit",
     "test": "bun test",
@@ -101,11 +110,13 @@
     "bun": ">=1.0.0"
   },
   "dependencies": {
+    "@hasna/contracts": "^0.4.1",
     "@hasna/events": "^0.1.9",
     "@modelcontextprotocol/sdk": "^1.29.0",
     "@openrouter/ai-sdk-provider": "2.9.1",
     "ai": "6.0.204",
     "commander": "^13.1.0",
+    "pg": "^8.13.1",
     "zod": "4.4.3"
   },
   "optionalDependencies": {
@@ -113,6 +124,7 @@
   },
   "devDependencies": {
     "@types/bun": "latest",
+    "@types/pg": "^8.11.10",
     "typescript": "^5.7.3"
   },
   "publishConfig": {