@theaileverage/marionette 0.2.2 → 0.3.0

package/documentation/effect-compatibility.md

@@ -0,0 +1,56 @@
+# Effect v4 compatibility acceptance guide
+
+This is baseline acceptance analysis for the integrated reviewer, not a verdict on the ongoing rewrite. Source and test references below refer to committed baseline `88c1034d0d65d77b8a1fdf5f6a2d76bf38f15fd6`, inspected using `git show HEAD:<path>` and `git grep ... HEAD`. Existing tests were read, not executed in this review. Proposed checks below are additional acceptance work, not passing evidence.
+
+The reviewed local skill is `.agents/skills/effect/SKILL.md`, with `SERVICES_LAYERS`, `TESTING`, `SCHEDULING`, `STREAMS`, `SCHEMA`, and `CONFIG` references. The installed and working-package Effect version was `4.0.0-rc.112`. Acceptance requires genuine Effect composition plus preserved durable, transport, SDK, and package behavior.
+
+## SQLite and authoritative state
+
+`src/store.ts:Store.transaction` executes `BEGIN IMMEDIATE`, nested savepoints, commit/release, and rollback synchronously. `Store` uses WAL, `synchronous=FULL`, a 5000 ms busy timeout, schema version 2, and refuses newer databases. Preserve persisted records, events, ordering, and restart compatibility.
+
+Keep each complete transaction inside one synchronous effect boundary. A callback returning a Promise or lazy Effect lets this baseline transaction commit before that work executes; do not yield, await, fork, or perform provider I/O inside it. `src/service.ts:submitAssignment`, `controlTask`, `report`, and `src/orchestration.ts:invoke` couple records, idempotency results, and events. Those writes must commit together or leave no partial state. Fibers and in-memory locks cannot replace persisted authority.
+
+Existing evidence: `tests/core.test.ts`, “submission is nonblocking, idempotent and rejects reused keys with different intent” and “SQLite restart recovers running tasks and credentials without redispatch”; `tests/orchestration.test.ts`, “cancellation is atomic and cascades to all descendants”. These do not directly inject mid-transaction failure. **Proposed:** throw after a record write but before its event/idempotency write; reopen SQLite and verify rollback. Exercise nested rollback caught by the outer callback, uncaught nested failure, and rejection/prevention of asynchronous transaction callbacks.
+
+## Lease, task, and revision fencing
+
+`src/service.ts:guard` compares the persisted lead owner, epoch, and token hash. `workerGuard`/`report` bind credentials to the current task run and reject obsolete revisions; `updateTask` merges onto the current row so stale I/O snapshots cannot erase newer controls or receipts. Preserve rechecks after external awaits, including reconciliation in `Service.invoke`, profile validation in `src/orchestration.ts:invoke`, and `src/cleanup.ts:release`. `src/supervisor.ts:verify` rereads the revision and pending operations after command execution before accepting results.
+
+Existing tests in `tests/core.test.ts`: “handover persists a briefing, fences the old lead, and rejects simultaneous takeover”, “redirect rejects stale reports and waits for the previous turn to settle”, and “late output snapshots cannot roll back a concurrent redirect or receipt”. Retain path/symlink and artifact-ownership tests: changing schemas must not broaden task authority or expose tokens.
+
+## Ambiguous external effects must not replay
+
+`src/supervisor.ts:recover` treats persisted `creating`, `prompting`, and control `sending` phases as ambiguous; scheduling reservations before side effects are recoverable separately. Dispatch persists run/creation intent before mutation. `src/continuation.ts:process` and `recover` preserve lead-delivery uncertainty; `src/cleanup.ts:release` persists `closing` before pane/tab closure, and `recover` converts interrupted closure to `uncertain`. A timeout, cancellation, or disconnected acknowledgement cannot prove the external action did not happen. Preserve identity-pinned explicit reconciliation and prohibit automatic mutation retry.
+
+Existing evidence: `tests/core.test.ts`, “ambiguous prompt delivery is never retried automatically, including restart” and “lost split acknowledgement persists intent and reconciles without replaying or prompting”; `tests/orchestration.test.ts`, “ambiguous lead delivery survives restart without replay and stale ownership cannot resume”; `tests/cleanup.test.ts`, “lost pane-close acknowledgement remains uncertain and reconciles with sibling still present”. Schedule-based polling may retry safe observations; it must not wrap an entire dispatch/close/send operation in generic retries.
+
+## Completion includes descendants and current evidence
+
+`Service.report` records a completion receipt; `Supervisor.verify` independently checks artifacts/commands and `Orchestration.unmetTask`. `src/orchestration.ts:unmet`, `taskEvidenceCurrent`, and `refreshEvidence` require current required descendants and criterion assessments, and reopen stale completed work. Preserve scope/depth/budget limits, yielding ownership/capacity, parent revision changes, and independent parent integration checks.
+
+Existing `tests/orchestration.test.ts` cases include “recursive completion rejects failed, blocked, cancelled and unresolved descendants”, “completed parents require their own integration checks after all children pass”, “a new required child fences an in-flight parent verification result”, and “changed verified artifacts reopen the task, ancestors and outcome without invalidating unrelated work”. A successful child fiber or receipt alone cannot complete the parent outcome.
+
+## Resource acquisition, drain, and interruption
+
+`src/server.ts:serve/shutdown` rejects new action calls while stopping, drains `Supervisor.stop`, closes HTTP connections, closes SQLite, then removes its owned lock. `Supervisor.stop` stops scheduling and waits for continuation, cleanup, and busy tasks. `src/continuation.ts:stop` and `src/cleanup.ts:stop` drain active work. External Herdr workers remain alive. Preserve this graceful-drain contract; scope interruption must not silently replace it with task cancellation or terminal closure.
+
+Critical **proposed checks**, using readiness gates rather than arbitrary sleeps:
+
+- Inject failure after lock acquisition, during Store/Service construction, during listener binding, and after background startup. Assert acquired resources finalize exactly once, SQLite closes, listeners/timers disappear, and retry succeeds. Baseline `serve` constructs Store/Service before its listen-error cleanup; this acquisition window is not covered by the integration happy path.
+- Hold a Herdr send/close or verification command active, request shutdown, and assert no new scheduling, no database close before draining writes, and no duplicate external call. Separately interrupt the owning operation and assert durable uncertainty where delivery is ambiguous, with no writes after database disposal.
+- Test simultaneous startup, stale PID recovery, malformed lock data, and lock replacement between acquisition and failure/shutdown. Remove only the lock this acquisition owns. Baseline normal shutdown checks `lockId`, but listen-error cleanup unlinks unconditionally. `tests/integration.test.ts` covers duplicate startup and normal lock removal; `tests/setup.test.ts` covers a concurrent AGY lock, not this full matrix. Also fault `src/agy-trust.ts:trustAgyWorkspace` cleanup without deleting another owner's replacement lock.
+
+## Process and SDK cancellation
+
+`src/files.ts:command` uses argument-array spawning, strips `MARIONETTE_`/`HERDR_` variables, bounds output, and kills the POSIX process group on timeout. Preserve these properties. The existing “verification timeouts stop their process group and capture failure” in `tests/core.test.ts` only launches one process and checks timeout/nonzero exit. **Proposed:** spawn a descendant, interrupt the owning fiber, verify both processes exit and pipes/timers settle; test spawn failure and platform-specific behavior.
+
+`src/herdr-transport.ts:socketRequest`/`JsonConnection` close sockets and detach abort listeners; preserve SDK Promise/async-iterator contracts, explicit targeting, deadlines, correlated replies, overflow errors, and no reconnect/replay. `tests/herdr-streams.test.ts` already covers cancellation, reader wakeup, overflow/EOF, and lost graphics ACKs. Add cancellation through the application Effect adapter, not merely direct `AbortController` calls.
+
+Audit every `Effect.runPromise` inside an Effect program. In installed `node_modules/effect/src/internal/effect.ts`, `runPromiseWith` reaches `runForkWith`, which creates a new fiber; interruption is connected only when explicitly supplied a signal. Wrapping a nested runner in `tryPromise` without forwarding cancellation detaches work. Prefer `yield*` composition and scoped fibers internally; allow runners at deliberate Promise-facing ingress boundaries with owned lifetime. Prove parent interruption reaches the actual SDK socket/command and waits for finalization. Source search alone cannot establish propagation.
+
+## Integrated review steps
+
+1. Review the final diff against the pinned baseline and trace acquisition/finalization, transaction callbacks, retries, runners, and authority rechecks. Run the proposed fault/cancellation checks alongside retained tests.
+2. Run final `npm run check`, `npm test`, `npm run build`, configured lint/Effect diagnostics, and package smoke checks. Record exact revision, commands, results, and skipped platform cases.
+3. Retain `tests/integration.test.ts` HTTP/MCP/CLI parity: loopback/Host/Origin/auth enforcement, worker-token separation, error envelopes/statuses, 47 baseline MCP tools, durable shared state, and CLI JSON. Schema errors must still map to `invalid_input`/400 rather than internal failures.
+4. Verify `package.json`'s Node `>=22.13`, executable CLI/MCP bundles, public dashboard assets, `./herdr-sdk` export and declarations, plus isolated packed-package imports without workspace dependencies. Retain `tests/herdr-api.test.ts` protocol-22 routing and `tests/setup.test.ts` durable runtime/MCP argument tests. Passing source tests alone does not establish packed-artifact compatibility.

package/documentation/effect-runtime.md

@@ -0,0 +1,73 @@
+# Effect runtime
+
+Marionette runs on Bun and uses Effect v4 for application workflows and resource lifetimes. The
+installed version is pinned because the v4 API is still a release candidate.
+See [CONTRIBUTING.md](../CONTRIBUTING.md) for compiler diagnostics, lint rules,
+and the installed Effect skill.
+
+## Application lifetime
+
+`src/application.ts` composes configuration, the supervisor lock, SQLite storage,
+and the application service through `Context.Service` and `Layer`. Resources use
+`Effect.acquireRelease`, so failure during later acquisition unwinds earlier
+resources. A failed database constructor closes its database handle before
+throwing. Lock cleanup checks its acquisition ID before removing the file.
+
+`src/server.ts` owns this graph through `ManagedRuntime`. HTTP listening and
+signal handlers belong to the same lifetime. The Promise returned by `shutdown`
+is shared between callers, so every caller waits for disposal to finish.
+
+The supervisor stops its scheduled poll before draining finite background jobs.
+Continuation and cleanup work also drain before SQLite is closed. Shutdown
+preserves external Herdr workers. Explicit fiber interruption is a separate
+operation; it must not silently substitute for graceful shutdown.
+
+## Workflow and transport boundaries
+
+Application operations use named `Effect.fn` functions and compose through
+`yield*`. `AppError` and `BoundaryError` are tagged schema errors: application
+errors retain their HTTP status and code, while infrastructure errors retain
+their operation and original cause. Effect Schema decodes application inputs;
+Zod remains at the MCP SDK boundary to satisfy that SDK's registration contract.
+
+Promise runners belong at ingress points such as HTTP, MCP, CLI, or compatibility
+methods used by existing callers. New internal code should call the Effect
+operation directly. A nested `Effect.runPromise` starts another fiber and needs
+explicit cancellation wiring; wrapping it in `tryPromise` alone does not provide
+that wiring.
+
+The published Herdr SDK keeps its Promise and async-iterator interfaces and has
+no Effect dependency. The application's Herdr adapter passes Effect's abort
+signal into the SDK socket request. `src/process.ts` scopes subprocess groups,
+bounds captured output, and waits for stdio closure after termination. Loopback
+HTTP uses Effect's HTTP client. Retry schedules are for safe readiness
+observations, not potentially delivered mutations.
+
+## Durable authority
+
+SQLite records remain authoritative for leases, revisions, ownership,
+idempotency, and uncertain delivery. A fiber is an execution mechanism, not a
+replacement for persisted state. Persist intent before an external mutation and
+recheck current authority after external I/O. A lost acknowledgement must not
+cause automatic send or close replay.
+
+Persistence currently uses `bun:sqlite` directly. Effect owns its lifetime and wraps application operations; queries do not use `@effect/sql-sqlite-bun`.
+
+Store transactions remain synchronous. Keep all writes in a transaction inside
+one synchronous boundary; never return a Promise or lazy Effect from a
+transaction callback. This preserves atomic records, events, and idempotency
+updates without committing before deferred work executes.
+
+## Verification
+
+`bun run check` includes patched TypeScript diagnostics and anti-slop lint.
+`bun run tooling:check` proves the rules reject deliberate failures after
+installation. `bun run runtime:check` exercises failed resource acquisition,
+duplicate startup, concurrent shutdown, and signal cleanup. `bun run test` covers
+durable orchestration and protocol behavior. `bun pm pack` runs checks and builds;
+`bun scripts/release.mjs smoke PATH_TO_TARBALL` checks the isolated CLI and SDK,
+including declaration use without a workspace Effect dependency.
+
+The [compatibility acceptance guide](effect-compatibility.md) records the
+pre-migration invariants and additional review cases. It is a review checklist,
+not a test result or a claim of integrated acceptance.

package/package.json

@@ -1,41 +1,51 @@
 {
   "name": "@theaileverage/marionette",
-  "version": "0.2.2",
+  "version": "0.3.0",
   "type": "module",
   "engines": {
-    "node": ">=22.13"
+    "bun": ">=1.3.14"
   },
   "scripts": {
-    "dev": "node --import tsx src/cli.ts serve",
-    "start": "node dist/cli.js serve",
-    "build": "tsc && vite build && node scripts/bundle.mjs",
-    "check": "tsc --noEmit && tsc -p tsconfig.web.json && tsc -p tsconfig.tests.json",
-    "test": "node --import tsx --test tests/*.test.ts",
-    "cli": "node --import tsx src/cli.ts",
-    "mcp": "node --import tsx src/mcp.ts",
-    "format": "prettier --write src web skills .github tests scripts *.json *.ts *.md",
-    "format:check": "prettier --check src web skills .github tests scripts *.json *.ts *.md",
-    "prepack": "npm run check && npm test && npm run build",
-    "release:prepare": "node scripts/release.mjs prepare",
-    "release:check": "node scripts/release.mjs check",
-    "sdk:generate": "node scripts/generate-herdr-sdk.mjs"
+    "dev": "bun src/cli.ts serve",
+    "start": "bun dist/cli.js serve",
+    "build": "bun --bun tsc && bun --bun vite build && bun scripts/bundle.mjs",
+    "check": "bun --bun tsc --noEmit && bun --bun tsc -p tsconfig.web.json && bun --bun tsc -p tsconfig.tests.json && bun run lint",
+    "test": "bun test ./tests --timeout 120000",
+    "cli": "bun src/cli.ts",
+    "mcp": "bun src/mcp.ts",
+    "format": "bun --bun prettier --write src web documentation skills .github tests scripts .oxlintrc.json .vscode *.json *.ts *.md",
+    "format:check": "bun --bun prettier --check src web documentation skills .github tests scripts .oxlintrc.json .vscode *.json *.ts *.md",
+    "prepack": "bun run check && bun run test && bun run runtime:check && bun run build",
+    "release:prepare": "bun scripts/release.mjs prepare",
+    "release:check": "bun scripts/release.mjs check",
+    "sdk:generate": "bun scripts/generate-herdr-sdk.mjs",
+    "prepare": "bun --bun effect-tsgo patch --typescript --oxlint",
+    "lint": "bun --bun oxlint src web tests scripts vite.config.ts",
+    "effect:diagnostics": "bun --bun effect-tsgo diagnostics --project tsconfig.json",
+    "tooling:check": "bun scripts/check-quality-tooling.mjs",
+    "runtime:check": "bun scripts/check-runtime-lifecycle.mjs"
   },
   "devDependencies": {
+    "@effect/tsgo": "0.43.0",
+    "@modelcontextprotocol/sdk": "^1.27.1",
+    "@oxlint/plugins": "1.82.0",
     "@types/express": "^5.0.0",
     "@types/node": "^22.15.0",
     "@types/react": "^19.1.0",
     "@types/react-dom": "^19.1.0",
+    "effect": "4.0.0-rc.112",
     "esbuild": "^0.28.2",
-    "prettier": "^3.9.6",
-    "tsx": "^4.20.0",
-    "typescript": "^5.8.3",
-    "vite": "^6.3.5",
-    "@modelcontextprotocol/sdk": "^1.27.1",
     "express": "^5.1.0",
     "lucide-react": "^0.468.0",
+    "oxlint": "1.82.0",
+    "prettier": "^3.9.6",
     "react": "^19.1.0",
     "react-dom": "^19.1.0",
-    "zod": "^3.25.0"
+    "typescript": "7.0.2",
+    "vite": "^6.3.5",
+    "zod": "^3.25.0",
+    "oxlint-tsgolint": "7.0.2001",
+    "@types/bun": "1.3.14"
   },
   "description": "Persistent multi-agent orchestration for Herdr, with a named lead, MCP tools, and a local dashboard.",
   "bin": {
@@ -63,7 +73,9 @@
     "dist/herdr-streams.d.ts",
     "dist/herdr-transport.js",
     "dist/herdr-transport.d.ts",
-    "vendor/herdr-0.9.0/LICENSE"
+    "vendor/herdr-0.9.0/LICENSE",
+    "documentation/effect-runtime.md",
+    "documentation/effect-compatibility.md"
   ],
   "publishConfig": {
     "access": "public"
@@ -82,5 +94,6 @@
       "types": "./dist/herdr-sdk.d.ts",
       "import": "./dist/herdr-sdk.js"
     }
-  }
+  },
+  "packageManager": "bun@1.3.14"
 }