@defrex/autobuild 0.11.0 → 0.13.0

package/docs/hosted-service.md

@@ -0,0 +1,240 @@
+# Deploy a hosted service
+
+A service hosts the BuildStore, tickets, dashboard, sign-in, and MCP endpoint.
+The default deployment stores state while dispatch runs on another machine.
+You need Bun 1.4.x, a Git repository, a Vercel project, PostgreSQL, a blob
+backend, and a GitHub OAuth application.
+
+Create a new directory containing these two files (also shipped in the hosted
+package's `deployment/` directory and in `templates/hosted-service/`).
+
+`package.json`:
+
+```json
+{
+  "private": true,
+  "scripts": {
+    "build": "ab-hosted-service-build"
+  },
+  "dependencies": {
+    "@defrex/autobuild-hosted-store-service": "0.13.0",
+    "@defrex/autobuild": "0.13.0",
+    "next": "16.1.6",
+    "react": "19.2.4",
+    "react-dom": "19.2.4"
+  }
+}
+```
+
+`vercel.json`:
+
+```json
+{
+  "$schema": "https://openapi.vercel.sh/vercel.json",
+  "framework": "nextjs",
+  "installCommand": "bun install --frozen-lockfile",
+  "buildCommand": "bun run build",
+  "bunVersion": "1.4.x"
+}
+```
+
+Run `bun install`, then commit both files and `bun.lock`. Add `.gitignore`
+entries for `node_modules/`, `.next/`, `.vercel/`, `app/`, `next.config.ts`,
+`next-env.d.ts`, `tsconfig.json`, and `.autobuild-service-assets.json`.
+These application assets are generated during each build. Keep your own files
+outside those names; the builder refuses to overwrite consumer-owned files.
+Do not commit generated application sources or edit them to customize deployment.
+
+Push the directory to your Git host. Import it in Vercel with the repository
+root as Root Directory and Next.js as the framework. The checked-in platform
+configuration installs the lockfile and runs `bun run build`. Each build stages
+the installed package, runs its idempotent database migrations against the
+configured database, then builds Next into the root `.next/`. Missing database
+configuration, incompatible schema, or a failed migration stops the build.
+Provide database access from the build environment as well as runtime.
+
+## Connect a project
+
+From a checkout with an origin remote, run `ab connect https://YOUR_ORIGIN`.
+The CLI prints a verification URL and short code and attempts to open a browser.
+Sign in as an allowed owner, check the full repository identity and code, and
+approve or deny. Another device can open the printed URL, or visit `/connect`
+and enter the code. Approval requires the web-enabled service origin; a
+standalone machine-only listener cannot present browser consent. Requests
+expire after ten minutes. Denial and timeout leave local configuration unchanged.
+
+On approval the service registers the project and issues a project credential.
+The CLI keeps it in `$XDG_CONFIG_HOME/autobuild/credentials` (or
+`~/.config/autobuild/credentials`), keyed by normalized service and origin,
+with directory mode 0700 and file mode 0600. Credentials never belong in the
+repository. It adds the non-secret `[store] url` to `autobuild.toml` when
+absent; review and commit that change yourself. An existing different URL
+must be changed explicitly before connecting. Reconnecting replaces the saved
+credential only after approval and revokes the previous credential.
+
+All worktrees share the main checkout's configuration and the user's saved
+credential. A service manager running as that user uses the same HOME and
+XDG_CONFIG_HOME, with no AB_STORE or AB_TOKEN exports. A fresh clone with
+committed store configuration but no saved credential fails with `ab connect`
+guidance; it never silently creates local state. Run connect on each host/user.
+
+`ab connect` without a URL reports service, project, and credential status:
+accepted, rejected, missing, or unreachable. `ab disconnect` revokes the saved
+credential remotely before removing it locally, keeping the project registered,
+its data, and store configuration. A network/server failure retains the local
+entry for retry; an already revoked credential can be removed. If saving a new
+connection fails, the CLI cancels/revokes it and retains the previous entry.
+If remote cleanup cannot be confirmed, check the printed credential id with
+the owner's administrative tools before retrying.
+
+Store selection is `--store` > nonblank `AB_STORE` > `[store] url` > the
+main checkout's `.autobuild/`. `AB_TOKEN` overrides the exact service/project's
+saved credential for CI and other non-interactive hosts. A service override
+never borrows another service's credential. Build and harvest sessions keep
+their explicit scoped authority.
+
+## Environment
+
+Set these server-side variables in every Vercel environment you deploy:
+
+- `AB_POSTGRES_URL` or `DATABASE_URL`: PostgreSQL connection URL. The explicit
+  `AB_POSTGRES_URL` takes precedence; use the same database at build and runtime.
+- `AB_STORE_SECRET`: a high-entropy secret for machine token signing.
+- `AB_BLOB_BACKEND=vercel` with `BLOB_READ_WRITE_TOKEN`, or `AB_BLOB_BACKEND=s3`
+  with `AB_S3_BUCKET`, `AB_S3_REGION`, `AB_S3_ACCESS_KEY_ID`, and
+  `AB_S3_SECRET_ACCESS_KEY` (and optional endpoint for compatible storage).
+- `BETTER_AUTH_SECRET`: an independent high-entropy secret of at least 32 characters.
+- `BETTER_AUTH_URL`: the exact production HTTPS origin, with no path.
+- `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`: GitHub OAuth application credentials.
+- `AB_WEB_AUTH_PROVIDERS=github`.
+- `AB_WEB_ALLOWED_EMAILS`: comma-separated operator email allowlist.
+- `AB_WEB_REPOSITORIES`: comma-separated repository origins visible to operators.
+
+Register `https://YOUR_ORIGIN/api/auth/callback/github` as the GitHub callback
+and allow read-only email access (`user:email`). Sign-in must use the configured
+production origin. Do not expose secrets through `NEXT_PUBLIC_` variables.
+See the [complete environment reference](configuration.md) for storage,
+project ticket lifecycle, credential administration, and optional MCP resource configuration.
+
+## Check the deployment
+
+`GET /health` returns service and protocol versions. Open `/sign-in` and sign
+in; `/` redirects unauthenticated browsers to sign-in. Machine routes
+require a machine token; `/mcp` challenges an unauthenticated client with OAuth
+resource metadata. Check both discovery documents:
+
+```sh
+curl https://YOUR_ORIGIN/.well-known/oauth-protected-resource
+curl https://YOUR_ORIGIN/.well-known/oauth-authorization-server
+```
+
+The authorization-server document advertises a reachable `/api/auth/jwks`
+URI and `/api/auth/mcp/register` dynamic registration endpoint. MCP clients
+register with `redirect_uris` as an array of strings and
+`token_endpoint_auth_method: "none"`; treat the returned `client_id` as opaque.
+See [MCP](mcp.md) for authorization and client setup.
+
+Without the optional hosted dispatcher installed, `GET /api/dispatch` returns
+503 JSON: `{"error":"dispatch runs elsewhere for this deployment"}`. The
+template has no cron and installs no dispatcher/provider package. Run dispatch
+from a connected project or follow the separate, opt-in
+[hosted dispatcher procedure](hosted-dispatcher.md), which retains its
+source-checkout packaging and cron setup.
+
+## Upgrade
+
+Bump the exact hosted-service and core package pins in `package.json` to the
+same new release, run `bun install` to refresh `bun.lock`, commit, and push.
+Vercel's Git integration migrates and builds the pinned package automatically.
+Upgrade the service **before connected projects**: that is the order expected
+by the protocol compatibility window. No source extraction, patch application,
+or manual production deployment is required.
+
+## Hosted ticket projects and v1 migration
+
+Hosted queues are partitioned by registered repository. Credentials select the
+project. Independent ticket policy selects lifecycle; `ab connect --tickets URL`
+publishes it before dispatch. Older clients retain uploaded dispatcher configuration
+as a fallback. See the [ticket protocol](remote-ticket-protocol.md). Projects using file,
+Linear or another external source continue to access it directly through their
+dispatcher. Service ticket surfaces refuse those projects with source feedback.
+The service never connects to Linear.
+
+The removed `AB_TICKET_BACKEND` and `LINEAR_API_KEY` settings are ignored for
+all values, including blanks, with one startup message asking for removal.
+The removed `AB_TICKET_TRIAGE_STATE`, `AB_TICKET_READY_STATE`,
+`AB_TICKET_DOING_STATE` and `AB_TICKET_DONE_STATE` settings are ignored even if
+malformed. Lifecycle now comes only from project configuration.
+
+The environment-aware migration first imports `AB_WEB_REPOSITORIES` into the
+registry once, including upgrades from released main schema v9 where registry
+tables do not yet exist. This import and the main-schema promotion share the
+ticket transaction: a stopped ticket migration rolls them back too. The owner
+`projects add` and `tickets assign-team` commands can independently bootstrap
+the registry while that old main marker remains, so recovery does not require
+a successful ticket migration. If the retired environment is unavailable,
+register the intended project with `ab-hosted-store projects add REPOSITORY
+--user IDENTITY` before assigning its legacy team. Once the import succeeds,
+later changes to the retired environment are ignored.
+
+Fresh databases install ticket schema v2 directly. Existing team-keyed v1
+queues migrate in one transaction. For each registered project, migration reads
+the effective configuration selected by its latest dispatcher run and reloads,
+including the actual immutable artifact bytes in blob storage. A legacy team
+must have one claimant, or an explicit owner assignment. A registered project
+with no dispatcher run or no selected effective-config reference makes no team
+claim and does not block other projects. Unclaimed teams still stop unchanged.
+When ownership must be inferred, a selected but missing, inaccessible or invalid
+configuration fails closed with the project and remaining teams in the message.
+Restore those bytes or explicitly assign every remaining legacy team, then
+rerun; complete assignments to registered projects settle ownership without
+reading unrelated configurations. Conflicting claimants and asset ownership
+conflicts also stop without changing schema markers or tickets.
+Historical ids, states, timestamps, blockers, comments and asset rows remain
+unchanged. Review the named projects and resolve a shared legacy team locally:
+
+```sh
+ab-hosted-store tickets assign-team ENG https://github.com/acme/app --user owner@example.com
+ab-postgres-store migrate
+```
+
+The assignment command works before the v2 migration succeeds and records the
+actor in an append-only journal. The latest assignment moves the whole legacy
+team once; it grants no future cross-project access. It must name a registered
+project. Rerun migration after resolving every named team or configuration error.
+
+For populated-v1 upgrades, both the owner migration CLI and `deploy:build` need
+the same blob backend, access, credentials and `AB_BLOB_PREFIX` as the running
+service: S3 bucket/region/credentials (and endpoint/path style when configured),
+or Vercel access mode and token/OIDC store identity. The deployment build inherits
+these variables and runs the environment-aware migration before Next build.
+Missing blob configuration or denied reads stop deployment before Next builds;
+diagnostics never print credentials. SQL-only fresh installs, empty-v1 upgrades
+and already-v2 reruns do not open blob storage and keep their existing database
+requirements. Migration reruns are idempotent.
+
+## Local builds with hosted tickets
+
+Hosted tickets do not require hosted builds. Set `[tickets] source = "hosted"`
+and `teamKey = "ENG"`, then run `ab connect --tickets https://your-service.example`.
+Approve the project in the browser. This saves a service-scoped credential and
+`[tickets] url` while leaving `[store]` unchanged. Ticket commands work immediately,
+before any dispatcher runs. `ab connect` reports every service and whether it serves
+builds, tickets, or both. Use `ab disconnect --tickets` to revoke ticket access;
+when services differ use `--builds` to select build access. Shared uses share one
+credential, so disconnecting either shared use affects both.
+
+Ticket URL selection is `[tickets] url`, then the selected build service (including
+AB_STORE or a supported --store override). With local builds, `[tickets] url` is
+required. Builds, events, artifacts, and harvest journals remain in the local store.
+The service stores ticket records, project ticket policy, and restricted ticket
+grants. Normal project clients synchronize current `[tickets]` policy before use;
+failed synchronization fails the operation. Harvest grants can read and create;
+phase grants can only read. Neither can change policy or access BuildStore routes.
+
+New hosted policy defaults to `triage`, `ready`, `doing`, `done`, with creation and
+harvest proposals in `triage`. Explicit state names preserve their spelling.
+To retain earlier names configure `triageState = "Backlog"`,
+`claimedState = "Doing"`, `doneState = "Done"`, and the desired `readyState`.
+Existing ticket rows are never renamed. Older combined-service clients continue
+using the uploaded dispatcher configuration until independent policy is published.

package/docs/mcp.md

@@ -110,8 +110,14 @@ the standard surface the endpoint serves.
   the default MCP resource's base.
 - `AB_WEB_ALLOWED_EMAILS`: the operator allowlist. Sign-in admission and every
   MCP tool call are checked against it.
-- `AB_WEB_REPOSITORIES`: the served repository identities (the
-  `repositories.list` payload and the per-call `repo` scope).
+- `AB_WEB_REPOSITORIES` (deprecated): optional once-per-database registry
+  bootstrap, never an ongoing allowlist. `repositories.list` queries current
+  active registrations on every call, and every tool checks current membership.
+
+Every allowlisted user sees every registered project. Database-authorized
+owners use the [project commands](setup.md#connected-projects) to add or remove
+projects without redeploying. Removal hides retained data until re-registration;
+registry failures never authorize a call.
 
 The endpoint requires no additional variables beyond the hosted service's
 existing web/auth set. Note that consent renders when the client sends

package/docs/operator-api.md

@@ -6,17 +6,48 @@ Every request must send `X-Autobuild-Version`,
 `X-Autobuild-Protocol-Version: 3`, and `Authorization: Bearer …`. Identity is
 checked before authentication or resource lookup.
 
-Mint a token offline; the signed user is used as the human actor on every write:
+Project credentials are opaque `abp_1_<id>_<secret>` bearer values bound to one
+registered normalized repository. The service stores only a SHA-256 digest of
+the random secret, plus project, creator, creation time, last use, expiry and
+revocation metadata. It checks credential status and project registration on
+every request. Revocation takes effect on the next request without affecting
+other credentials. Unknown, expired or revoked credentials receive `401 auth`;
+removed projects receive `403 auth` with `project not registered`.
+
+The deployment owner runs these commands locally with database access:
 
 ```sh
-AB_STORE_SECRET='…' bun packages/hosted-store-service/src/bin.ts mint operator \
-  --user 'Ada Lovelace' --ttl-seconds 3600
+ab-hosted-store credentials issue https://github.com/acme/app --user owner@example.com
+ab-hosted-store credentials list https://github.com/acme/app
+ab-hosted-store credentials revoke https://github.com/acme/app --id ID --user owner@example.com
 ```
 
-Admin, build, repository, and deployment (`{ "operator": true, … }`) tokens
-have no attributed operator authority. Human-operator tokens have no raw
-`/builds`, `/repos`, `/sessions`, or `/tickets` authority. Encode repository,
-build, session, and artifact kind as separate URL path segments.
+Issue prints the secret once; list contains metadata only. Issue optionally
+accepts `--ttl-seconds N` or `--expires-at ISO-8601`. For administrative/non-interactive issuance, set the returned token as
+`AB_TOKEN`. Interactive project onboarding uses `ab connect <service-url>`;
+it saves the credential outside the repository and commits no secret. Previous `AB_TOKEN` values and `mint operator` / `mint build` are
+retired: the service responds that the credential format changed and points
+to `credentials issue`. Supported old protocol shapes still work with a new
+credential; protocol compatibility does not restore retired authentication.
+
+`mint admin --ttl-seconds N` creates an explicit versioned, attributed owner
+break-glass credential. It can cross project boundaries. Keep it owner-only:
+never give it or `AB_STORE_SECRET` to a dispatcher or build agent.
+
+Owner-purpose project credentials attribute every write to their recorded creator.
+Dispatcher-purpose credentials are refused. Signed-in owners in the dashboard
+and MCP receive short-lived internal signatures with an explicit version,
+selected repository, signed user, expiry and MCP `via`. Only the service holds
+the signing key. These signatures authorize operator routes only and require a
+registered project; replay against another project is refused.
+
+The path project is checked before registry lookup, body parsing and generic
+tool execution. Body repositories and secondary build/session identifiers are
+also checked inside that project. Explicit project mismatches return `403 auth:
+wrong project`; removal returns `403 auth: project not registered`. Encode each
+repository, build, session and artifact kind as a separate URL path segment.
+Registration changes use the database-local `projects list|add|remove` owner
+commands; see [owner workflow](setup.md#connected-projects).
 
 ## Operator sessions
 
@@ -56,6 +87,7 @@ turn runner (a later ticket) is the only producer of turn facts.
 | `GET …/harvest/status` | Harvest gate, runs, steps, recovery, and attention projection. |
 | `GET …/builds/{slug}/artifacts/{kind}?rev=N` | Raw bytes (`application/octet-stream`) plus content-disposition and `X-Autobuild-Artifact-*` metadata headers. Omit `rev` for latest. |
 | `GET …/tickets?state=S&label=L` | `{states,tickets,diagnostics,criteria,triageState,readyState}`. The two lifecycle names come from effective repository configuration (including provider fallback). With no `state`, `criteria.state` is `triageState`; repeated `label` parameters narrow that state with AND filters. An explicit `state` selects that exact backend workflow state. |
+| `GET …/tickets/states` | Full state metadata `{name,tickets,about?}[]`, including empty states (`tickets.states` in MCP). |
 | `GET …/tickets/{id}` | `{ticket,blockers,build}`. Blockers include native `exists`/`resolved` status; `build` prefers an active matching repository build, otherwise the most recently updated one. |
 
 For artifact reads, an absent `rev` selects the latest revision. A supplied value
@@ -88,6 +120,7 @@ JSON bodies are strict: unknown fields are rejected.
 | `POST …/settings/auto-merge-default/toggle` | Empty body. |
 | `POST …/bulk-control` | `{"action":"pause"\|"resume"}`. The hold fact, intake fact, then eligible build events are written in that order. |
 | `POST …/harvest/control` | `{"action":"toggle-gate"}` or `{"action":"run","run":"…"}`. A run action is bound to that concrete projected run. |
+| `POST …/tickets/states` | `{name,about?}` → `{state,created}`; requires attributed operator identity (`tickets.state.create` in MCP). Hosted duplicate names are refused case-insensitively. |
 | `POST …/tickets` | `{"title":"…","body":"…","labels"?:[],"state"?:"…","blockedBy"?:[]}`; omitted state preserves the backend default. |
 | `PATCH …/tickets/{id}` | Any nonempty subset of `title`, `body`, and complete-replacement `labels`. Body bytes are not normalized. |
 | `POST …/tickets/{id}/move` | `{"state":"backend state name"}`. |
@@ -201,3 +234,60 @@ identity, plus the usual repo/build/state/cursor fields. It invents no durable
 sequence or actor. Retain the cursor to avoid repeat reports across invocations.
 No lease alone is not evidence of death. SIGINT, SIGHUP and SIGTERM drain dispatch,
 record the first signal in the stop fact, and release its lease before exit.
+
+### CLI attention monitoring
+
+The default watch reports actionable conditions only:
+
+- Build `escalation.raised`, or `build.merge-decision`: a finalized open PR
+  with auto-merge off or a current actionable merge-gate deferral. Armed consent
+  awaiting application is silent. One merge decision is reported per build/PR.
+- Repository `harvest.escalated`, `harvest.recovery-exhausted`,
+  `dispatcher.config-rejected`, `dispatcher.config-reloaded` with nonempty
+  `restartRequired`, abnormal `dispatcher.run-stopped`, and `dispatcher.lost`.
+- `dispatcher.ticket-source-persistent`: degradation past its advertised
+  reset time, once a tick started at or after reset has completed or failed
+  with the episode still open; crossing reset alone is silent. Without a reset,
+  the deadline is 15 minutes after onset. Repeated evidence never postpones
+  the deadline; recovery ends the episode.
+- `dispatcher.tick-failure-persistent`: continuously failing completed ticks
+  for five minutes in one run. A successful tick, new run, or matching stop
+  ends the episode. Source grace and cooldown-driven failures do not start
+  this clock; after grace, the first qualifying failure starts a new five-minute
+  interval. Each persistence episode is reported once.
+
+Repository conditions are included with no slugs or with `--repository`.
+Routine outcomes and self-healing failures are silent. Repeatable
+`--event <glob>` replaces the default and selects exactly matching catalog
+events (plus the existing `dispatcher.lost` observation); use explicit globs
+to opt into failures, finalize, merge, or completion events.
+
+Derived records use `observation` instead of `event`, without inventing a
+durable sequence or actor. Merge observations carry `type`, `pr`, `reason`,
+and `observedAt`, plus the build slug and normal reduced state. Persistence
+observations carry source/cause/message/resetAt when advertised (or
+run/firstFailure/error), `episodeSeq`, and `observedAt`; repository records
+have `repo` and null build/state. Source observations also report
+`dependencyLimited`.
+
+Every record carries a cursor, including timer-only records. Resuming suppresses
+delivered merge decisions and persistence episodes even when stream positions
+did not advance. Old v1 cursors remain valid: historical eligible merge
+decisions are treated as consumed; active overdue repository episodes may
+report because old watches had no persistence notification. Fresh attachment
+silently baselines historical direct events and merge decisions, but retains
+active repository timers. Reconcile existing conditions through status after
+arming the watch. Named watches still end when every named build is terminal;
+learn their outcomes from status reconciliation at exit.
+
+`ab wait --for attention` is build-only and uses the same build conditions:
+an open escalation or a merge decision. Existing conditions return immediately,
+even with `--since`; live and backlog matches ignore routine outcomes and
+self-healing failures. A merge match uses the watch observation with
+`condition: "attention"`. Explicit `--event` globs remain event-only.
+Watch and wait preserve each other's cursor notification metadata.
+
+Hosted ticket tools resolve independent project ticket policy before legacy uploaded
+configuration. Queue, detail, state discovery and mutations work before any build
+or dispatcher run; detail reports a null matching build. Ticket policy publication
+never creates synthetic build or repository history.