@wascartech/devbot 0.0.0-stage → 2.1.0

package/README.md

@@ -1,3 +1,639 @@
-# Temporary Holding Version
+# DevBot
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+NodeJS orchestrator that runs [OpenCode](https://opencode.ai) agents in Podman containers, triggered by forge webhooks when the bot is mentioned in a work item (issue, task, etc.).
+
+DevBot is **forge-agnostic**: a platform abstraction layer supports **GitLab**, **ForgeJo**, **Gitea**, and **GitHub**. One bot instance talks to exactly one forge instance, selected with `PLATFORM` (`gitlab|forgejo|gitea|github`). GitLab is the default. ForgeJo and Gitea share a common `/api/v1` driver; GitHub targets github.com (not Enterprise Server).
+
+## Workflow
+
+1. The forge sends a webhook when the bot account is `@mentioned` in a work item (comment on the work item, or work item creation with a mention in the description).
+2. The mention is parsed: `@devbot /build <prompt>` triggers **build mode**; anything else triggers plan mode (the default).
+3. DevBot starts a Podman container with a per-work-item named volume (`devbot-wi-<id>`), bind-mounting the OpenCode config read-only.
+4. The container clones the repo (first run) or fetches (subsequent runs). On first run it clones with a fallback chain: `wi-<number>` (prior WIP branch if it exists remotely) → the `/base` target branch if given, else the first existing branch from the global base-branches list tried **top-to-bottom** (default `dev,master` → tries `dev` then `master`, set in the WebUI). A `/branch <name>` mention overrides that work-branch name (stored per work item, first naming only; a name that already exists remotely is rejected). In build mode it then force-checks-out the work branch off whatever HEAD landed on; plan mode stays on the successful clone's branch. If neither the work branch nor any configured branch exists remotely (e.g. a brand-new repo with no branches), the run aborts with an actionable `⚠️ No base branch found` note — no bare-clone fall-through. On subsequent runs it fetches and checks out the work branch (build mode pulls --rebase; plan mode tolerates a missing branch). With `GIT_SUBMODULES_ENABLED` (default `true`), clone and each fetch also run `git submodule update --init --recursive` (best-effort; see [Git submodules](#git-submodules)).
+5. OpenCode runs with `--continue` (one session per work item, resumed across mentions) and `--format json`. The agent is `plan` or `build` per the mode.
+6. **Build mode only**: the container commits and pushes to the branch. Plan mode makes no git changes.
+7. DevBot parses the JSON output and extracts assistant `text` parts only. In build mode it also extracts the agent's ` ```mr ` content block (title + description) from the reply, stripping it from the comment text.
+8. DevBot posts the result as a top-level comment on the work item.
+9. **Build mode only**: DevBot creates a merge/pull request targeting the base branch if none exists for the branch, using the agent-authored title + structured-Markdown description (ending with `Resolves #<number>` so the forge auto-closes the work item on merge); if a request already exists, it updates the title + description with the agent's latest content.
+10. **CI failure auto report (opt-in)**: when enabled, a failed pipeline on an MR DevBot itself created automatically starts a **plan run** on the linked work item with the failing CI job logs (tails) as context. See [CI failure auto report](#ci-failure-auto-report).
+
+## Modes
+
+- **Plan mode** (default): the agent analyzes and replies with a plan/note. No branch, commit, push, or MR. Bash is allowed for read-only inspection (`git log`, `grep`, `find`) but destructive/mutating commands (`rm`, `git push/commit/checkout/reset/rebase/merge/cherry-pick`) are denied. File edits are denied.
+- **Build mode** (`/build`): the agent implements changes on branch `wi-<iid>` and commits. Publishing is opt-in: the branch is pushed and the MR opened only when **Settings → Runtime → Build output** enables auto-push / auto-MR (both off by default), or per run with the `/push` and `/mr` tokens. Add `/force-push` to make the host-side push use `--force-with-lease` (see [Safety](#safety)). On its own, `/force-push` is a **push-only command** — no agent run; it force-pushes the work item's existing local branch. `/push` and `/mr` also work standalone as push-only / publish-and-open tasks (see below). When **auto-resolve conflicts** is enabled, DevBot checks the branch against its MR target before opening the MR and, on conflicts, runs a direct build run to rebase/merge the target in; if it cannot resolve them, the branch is pushed but the MR is skipped with an actionable note. `/fix-conflicts` re-runs that conflict fix on demand (e.g. after an MR's target branch moved) without publishing or touching the MR. The host-side publish is rewrite-aware: it force-pushes (SHA-pinned lease) only when the remote still equals the run's starting tip, so a rebase never errors with git's raw fast-forward hint.
+
+## Subagents
+
+Subagents are supplied by admin-registered git repositories rather than a vendored bundle (work item #63): register one or more repositories of opencode agent markdown under **Settings → Skills & Agents**, and every `/agent <name>` mention routes to an installed agent. The bot injects `@<name>` into the prompt and OpenCode routes to that subagent. Multiple `/agent` calls can appear in a single mention, on any lines, anywhere in the message — each gets its own prompt segment and all run in one OpenCode session. An unknown name is reported in-thread after the repositories sync. OpenCode's built-in `general`, `explore`, and `scout` subagents are always available.
+
+The full command reference — every token, its parsing rules, combinations, and precedence — lives in [docs/commands.md](./docs/commands.md). The table below is the quick summary.
+
+| Token | Effect |
+|---|---|
+| `/build` | build mode (existing) |
+| `/base <branch>` | set the base/target branch for this work item's first run. Overrides the global base-branch walk entirely (hard per-work-item override) and retargets the MR (build mode). First-run only: on subsequent runs it's stripped and a `⚠️ /base '<branch>' ignored — work item already initialized.` note is posted. Rejects the whole mention if the branch is missing on the remote or the name has unsafe characters. Persists per work item (first-write-wins); cleared by `/reset` and work-item close. May appear anywhere, like `/build`; duplicate `/base` in one mention → last wins. |
+| `/branch <name>` | name this work item's work branch for every run (default `wi-<number>`). Honored on the **first naming only**: the name is stored per work item (first-write-wins) and reused on later runs, including after `/reset` and work-item close. Rejects the whole mention if the name has unsafe characters, or if the branch already exists on the remote (fail-closed — never renames or clobbers). A later, different `/branch` on an already-named work item is ignored with a `⚠️` note. May appear anywhere, like `/build`; duplicate `/branch` in one mention → last wins. |
+| `/agent <name> <segment>` | route `<segment>` to subagent `<name>` (case-insensitive, `.md` suffix optional). May appear multiple times in one mention, on any lines. `<segment>` is the rest of the line after the `/agent <name>` token. |
+| `/force-push` | standalone push-only command (no agent run) **or** a build modifier. Alone, it dispatches a push task that force-pushes the work item's existing local `wi-<iid>` branch with `--force-with-lease` — no opencode run, no clone, no commit; it fails with an actionable note if the workspace or branch doesn't exist. Combined with `/build`, it keeps the original meaning: run the agent, then force-push its result. Stripped from the prompt wherever it appears, like `/build`. |
+| `/push` | standalone push-only command (no agent run) **or** a build modifier. Alone, it publishes the work item's existing local branch with a plain fast-forward push (no force); if nothing changed it posts `ℹ️ Nothing to push`. Combined with `/build`, it forces the run's host-side push even when the global/project auto-push setting is off. Stripped from the prompt wherever it appears. |
+| `/mr` | standalone publish-and-open command (no agent run) **or** a build modifier. Alone, it publishes the existing local branch (skipping the push when already up to date) then creates/updates its MR with the fallback title/description. Combined with `/build`, it forces both the push and the MR even when auto-push/auto-MR are off. `/mr` implies a push. Stripped from the prompt wherever it appears. |
+| `/fix-conflicts` | standalone conflict-fix command (no agent run) **or** a build modifier. Alone, it re-runs the conflict resolver on the work item's existing branch against the current target of its open MR (falling back to the `/base` target), then re-pushes the resolved branch — no publish-first and no MR create/update. Combined with `/build`, it forces the post-build conflict check for that run. Bypasses the auto-resolve-conflicts setting. Stripped from the prompt wherever it appears. |
+| `/reset` | destroy this work item's volumes (`devbot-wi-<id>` and its `-podman` / `-xdg` siblings) and clear any `/base` choice. Force-stops any in-flight session container first, then removes the volumes. Volume-only: the `wi-<iid>` branch and any open MR are untouched. Posts a confirmation note back. Short-circuits over `/build`, `/base`, `/force-push`, and `/agent`. |
+| `/stop` | cancel this work item's running task and drop its queued backlog. Kills the phase containers (`-clone`/`-sync`/`-run`/`-push`/…); the run unwinds cleanly — no `⚠️` error reply, no error-log row, no push/MR. The volume is untouched (`/reset` discards it). Short-circuits over `/build`, `/base`, `/force-push`, and `/agent`; `/reset` wins if both appear. |
+| `/help` | reply-only command: posts the list of command tokens with one-line descriptions and dispatches no run. Short-circuits over every token except `/reset` and `/stop` (both still win). Sits behind the usual trigger policy and needs no provider/model config. |
+| `/status` | reply-only command: posts a local snapshot of this work item's running/queued tasks, global gate depth, branch/MR, retries, and effective auto-* settings, and dispatches no run. Insensitive to `QUEUE_FEEDBACK`; allowed for plan-only subjects. `/help` wins if both appear. |
+| `/auto-push`, `/auto-mr`, `/auto-resolve-conflicts`, `/auto-ci-report` | contextual auto-* commands (work item #58). Bare token queries the feature's effective state for this work item; `… off` disables it here (wins over a project-on setting) and `… on` re-enables it. A non-admin cannot enable a feature the project resolves **off** — that requires an allowlisted DevBot admin (`ONBOARDING_USERS`), and the enable is for this work item only. Applied and acknowledged before any run in the same mention is dispatched; a mention with only these tokens dispatches nothing. `/reset` and `/stop` still win. |
+| `/auto` | query all four auto-* features at once; `/auto off` disables all four for this work item; `/auto on` enables all four (per-feature admin check on project-off features). |
+| `/ask` | alias of `/auto off` — disable all four auto-* features for this work item. |
+
+A token is a command only when it stands as its own word: preceded by the start of a line or whitespace, and not followed by a word character or `/`. A token embedded in a path (`(some/foo/bar)`, `/build/output`, `https://host/x/build/y`) is literal prose and preserved verbatim in the prompt; so is a token inside inline code, a fenced block, `"…"` / `'…'` quotes, `(…)` parentheses, or `**…**` bold (parentheses nesting-aware). Sentence-final punctuation is fine (`/build.` still builds).
+
+`/build` is a global flag — order-independent, stripped wherever it appears. `/agent <name>` is line-scoped: one `/agent` per line, segment is the rest of that line. `/base <branch>` and `/branch <name>` are global flags like `/build`, stripped wherever they appear (`/branch` names the work branch, `/base` its MR target). `/force-push`, `/push`, and `/mr` are global commands/modifiers like `/build`; standalone they dispatch their own task, and with `/build` they force the corresponding build-output step. Standalone they short-circuit over `/base` and `/agent` (they neither clone nor run the agent) but, unlike `/reset`, are still queued behind any in-flight run for the same work item. `/reset` short-circuits entirely: it wins over every other token and no run is dispatched. `/stop` short-circuits over `/build`/`/agent` too; `/reset` still wins if both appear. Non-`/agent` prose (before, between, after `/agent` lines) becomes a shared preamble prepended once to the combined OpenCode prompt; the main `plan`/`build` agent reads it as background, then dispatches each `@agent` block in one session.
+
+### Single-agent examples
+
+All equivalent (segment folds the pre-`/agent` text into the agent's prompt):
+
+```
+@devbot /build /agent python-pro review types here
+@devbot /agent python-pro /build review types here
+@devbot /build review types here /agent python-pro
+```
+
+Result: `mode=build`, prompt sent to opencode = `@python-pro review types here`.
+
+### Multi-agent examples
+
+Each `/agent` line carries its own segment; other lines are shared context:
+
+```
+@devbot looking at this bug.
+
+/agent python-pro review the models
+
+Also check the FE here.
+
+/agent typescript-pro fix the typing
+```
+
+Result: `mode=plan`, prompt sent to opencode:
+
+```
+looking at this bug.
+
+Also check the FE here.
+
+@python-pro review the models
+
+@typescript-pro fix the typing
+```
+
+One OpenCode run, one combined note posted back, build mode (if set) commits once at the end.
+
+### Errors (no run dispatched)
+
+- `@devbot /agent nope …` → `⚠️ Unknown agent: nope.` (validated after the configured agent repositories sync)
+- `@devbot /agent python-pro` (no segment on the line) → `⚠️ No prompt given for /agent python-pro.`
+- `@devbot /agent python-pro x\n/agent python-pro y` → `⚠️ Duplicate agent: python-pro.`
+
+Step-0 error replies (unknown agent, authorization denial, `/reset` failure) are posted **in the same discussion thread** as the triggering mention when one exists (Note Hook), or as a **top-level note** on the work-item-creation path (Issue Hook, where there's no triggering note to reply to).
+
+## Prompt construction
+
+- **First run** on a work item (volume `devbot-wi-<id>` doesn't exist yet): the prompt is `Work item #<iid>: <title>\n\n<description>\n\n---\n\n<mention text>`, seeding the OpenCode session with full work item context. If the description is empty, it's omitted. If the mention text is empty, the separator is omitted.
+- **Subsequent runs** (volume exists → session has prior context): the prompt is the cleaned mention text only (work item title + description are already in the session history).
+
+## Requirements
+
+- Node.js 22+
+- Podman on the host PATH
+- A forge bot account (numeric user ID + access token) — GitLab, ForgeJo, Gitea, or GitHub
+- An LLM provider API key (Anthropic, OpenAI, etc.)
+- **Attachments (optional)**: DevBot does not read or OCR attachment content itself. The work item's attachment links are passed to the agent, which downloads the whole files on demand (`download_work_item_attachment`) and handles them with its own tools. OCR/PDF text extraction runs inside the builder image (`docker/Dockerfile.builder` installs `tesseract`, `tesseract-langpack-all`, and `poppler-utils`), so the agent can OCR an image or extract PDF text itself. The only attachment knob is `ATTACHMENT_MAX_BYTES` (the download/upload size cap).
+
+## Global settings
+
+Provider credentials, global plan/build models (with optional variants), OpenCode plugins, base branches, the post-build auto-push / auto-MR / conflict auto-fix switches, the CI-failure auto-report switch, and the global LSP server selection are configured from the web panel's **Settings** page (sidebar → Settings), with global MCP servers on the **Settings → MCP** tab; all are stored in `devbot.db` — not `.env`. The AI provider list is pulled by the browser from [models.dev](https://models.dev/api.json) when the page opens; a custom OpenAI-compatible provider can be added manually. Legacy `LLM_*`/`OPENCODE_PLUGINS`/`BASE_BRANCHES`/`ATTACHMENT_OCR_LANG` env vars (and `GITLAB_BOT_URL`) are removed and ignored — there is no auto-import, so configure everything in the WebUI. See [docs/global-settings.md](./docs/global-settings.md). Until a provider with a key and both global models are set, mentions are blocked with an actionable note.
+
+## CI failure auto report
+
+Opt-in (**Settings → Runtime → CI failure reports**, plus a per-project override, both off by default). When on, a failed CI pipeline on a merge/pull request that DevBot **created** starts an automatic plan run on the linked work item, with the failing job logs (tails — the head is dropped on long logs) injected as read-only context. Detection uses forge webhooks (GitLab Pipeline Hook, GitHub/ForgeJo/Gitea `workflow_run`), so it is near-realtime.
+
+"Created by DevBot" is matched against a durable registry of MRs the runner opened — not a title or author heuristic. The plan run is skipped when the work item is closed, the bot is unconfigured, or the same pipeline was already reported (dedup by pipeline id/sha). Enabling the toggle attempts to subscribe the project's forge webhook to CI events using the signed-in user's OAuth token (the bot token cannot manage hooks); if that fails, re-onboard the project. Turning the toggle off leaves the subscription in place — the server ignores CI events while it is off. See [docs/global-settings.md](./docs/global-settings.md#ci-failure-reports) and [docs/platforms.md](./docs/platforms.md#ci-failure-webhooks).
+
+## Runs
+
+The sidebar's **Runs** page is one table over every run, with a **status filter** (`All runs` · `Active` · `Errored` · `Cancelled` · `Settled`, stored in the URL as `?status=` so it is deep-linkable). **Active** covers queued and running tasks; **Errored** shows failed runs; **Cancelled** shows stopped runs; **Settled** covers every finished run (ok, failed, or stopped). Alongside the status filter the header toolbar offers the same filters as the Usage page — period (7/14/30/90 days, all time, or a custom range), project, model, and mode — plus a debounced **work item title search**. Rows show the work item (`#iid title`, linked to the forge when the project is managed), project, mode, the resolved **model** (`provider/model (variant)`, variant omitted when unset), state, phase, queued and finished times, duration, and retry attempts; the table never exposes the agent prompt or work item description. The concurrency quota (`active / max` running, `waiting` on the gate) is shown for the **All runs** and **Active** filters. The old `/history` URL redirects to `/runs?status=settled`.
+
+The listing is served by `GET /api/runs?status=all|active|errored|cancelled|settled&days=&projectId=&model=&mode=&q=&limit=&offset=` (a custom range uses `from=YYYY-MM-DD&to=YYYY-MM-DD` instead of `days`), which merges the in-flight queue and the settled run history on their shared task `seq` (newest first); the response also carries the `filterOptions` and `projectPaths` the dropdowns need. The queue and history sources are also still available directly at `GET /api/tasks` and `GET /api/run-history/:id`.
+
+Each row has a **Logs** button. For an active task it opens a live view of the **currently-active phase** container output — clone, sync, auth, agent run, push, and so on — not just the agent's `-run` container, polled every couple of seconds while the slideover is open (log bodies are redacted). The viewer names the active phase (e.g. "Cloning repository") and shows its output as it arrives; when the active phase has produced no output yet it says so explicitly. Every line is prefixed with a server-local `HH:MM:SS` clock derived from `podman logs --timestamps` (opencode event timestamps are used as a fallback; shell-only lines with neither source stay unprefixed). Because phase containers are short-lived (`--rm`), live logs are only available while a phase is in flight; phase containers pin the `k8s-file` log driver so `podman logs` can serve a running container on hosts whose default driver (e.g. `journald`) would otherwise expose nothing until exit. For a settled run the same button reads the captured run-log file instead; a run that captured nothing has it disabled. If a task finishes while you are watching it, the live poll switches to the archived file automatically.
+
+Each active row also has a **Stop** button (two-step inline confirm): it cancels that one task — a queued row is removed, a running row is flagged and its phase containers are killed, and the stop is posted to the work item thread. Stopping via the button is task-scoped; `/stop` in a work-item thread is work-item-scoped and also drops the queued backlog.
+
+Log rendering is verbose: every opencode part type is shown. Tool calls print their title, compact input, and indented output (or `error:` on failure); `step-finish` lines carry token counts and cost; `file`, `subtask`, and `retry` parts carry their detail; unknown part types dump their JSON body. Tool input/output and unknown-part bodies are capped per event (`…[truncated]`) so one huge result can't crowd out the rest.
+
+### Run-log capture and retention
+
+Capture is in-process: every phase container (clone, sync, auth, detect, agent run, push, conflict check, tip, and MCP-triggered peer/attachment operations) has its stdout/stderr tee'd into one file per run at `RUN_LOG_DIR` (default `run-logs/`, or `${INSTANCE_ID}-run-<seq>.log` when `INSTANCE_ID` is set). Output is redacted before it touches disk (line-buffered so a secret split across stream chunks is redacted whole) and capped at 2 MB per file (`log_truncated` is then set). `run_history` in `devbot.db` stores only metadata plus the filename — never the body. Reads use the same formatter as the live and error-log views, so NDJSON becomes readable lines with a server-local `HH:MM:SS` clock.
+
+Log files are retained for `RUN_LOG_TTL_HOURS` (default **48**; `0` keeps them forever), configurable in **Settings → Runtime → Maintenance** or via the env var as the fallback. Expiry unlinks the **files** only: the history row survives and the detail view says "logs expired". This is separate from `ERROR_LOG_RETENTION_DAYS`, which governs error-**row** retention. In a container deployment, mount a volume at `RUN_LOG_DIR` so the files survive container recreation. This is why no `journalctl` integration exists: phase containers pin the `k8s-file` log driver and are `--rm`, so host journald has nothing to read.
+
+Below the Runs page, an **Error logs** card lists persisted failures from every phase (attachment, prepare/clone/sync, run, push, MR) with the final message, the exact model the run used, and redacted output. Only the final failure is stored — retries collapse to one row. The header toolbar filters by the same period/project/model/mode as Usage and searches the work item title (`GET /api/error-logs?days=&projectId=&model=&mode=&q=&limit=&offset=`). Rows older than `ERROR_LOG_RETENTION_DAYS` (default 30; `0` keeps them forever) are purged at startup and by a daily sweep; see the configuration table.
+
+## Usage
+
+The sidebar's **Usage** page aggregates token consumption and cost across settled runs. Every run's opencode output already carries per-step token counts and the USD cost opencode computed; DevBot sums those `step-finish` parts while the container streams, accumulates them per work item (the main agent run and any conflict-resolver runs), and stores the total on the run's history row when the task settles. Retries reuse the same task row, so all attempts sum into one entry. In-flight runs are not counted — the page covers **settled** runs only, and rows recorded before this feature read as zero.
+
+The page is filterable by period (7/14/30/90 days, or all time), project, model, and mode, and shows stat cards (runs, tokens in/out, cache, total, cost) with period-over-period deltas, a stacked token or line-cost chart, full-width by-model/by-project/by-mode breakdown rows, and the ten most recent matching runs. Every breakdown and the recent-runs table splits tokens into **In / Out / Cache** columns (Out counts reasoning, Cache counts read + write). The token chart has a color legend (input, output, cache read, cache write). Per-run tokens and cost are also shown in the Runs table and the run detail slideover.
+
+The period select also offers **Custom**, which reveals a date-range picker (two months side by side, bounded to the last 365 days). Selecting a range filters every card, the chart, and the tables to those calendar days (inclusive at both ends); the deltas compare against the immediately preceding, equally long window.
+
+The data is served by `GET /api/stats/usage?days=&projectId=&model=&mode=` (session-gated), which reads `run_history` and aggregates in a pure module (local-calendar-day buckets). A custom range is requested with `from=YYYY-MM-DD&to=YYYY-MM-DD` instead of `days`.
+
+### Custom-model prices
+
+OpenCode computes `cost` from a model's price table, and a price in the generated `opencode.json` overrides the models.dev catalog. Custom providers added in **Settings → Providers** therefore have optional per-model price fields (**input**, **output**, **cache read**, **cache write**) in **USD per 1M tokens** — the same unit models.dev and opencode use. When any field is non-zero the provider's model is emitted with a `cost` block in the per-run config; leave all fields empty to use the provider default (or report no cost). Registry providers take their prices from models.dev and are not editable here.
+
+## Volumes
+
+The sidebar's **Volumes** page lists the per-work-item volumes and their GC state, and lets an admin delete one. A header toolbar filters by GC state and project and searches the work item title. It merges the volume ledger (`work_item_volumes`) with `podman volume ls`, so a volume that exists in podman but has no ledger row (e.g. volume GC never reconciled because `VOLUME_GC_ENABLED=false`) still shows up as **Untracked** and is deletable. Each row shows the work item (`#iid` + title, linked when the project is managed), the sibling volumes present (`workspace`, `podman` for the nested-Podman store, `xdg`), created/last-used times, and a GC-state badge: **Purges in ~Xd** / **Purge eligible** (TTL-based, computed from `last_run_at` + `VOLUME_GC_TTL_MS`), **Missing** (the ledger volume is gone from podman), **Untracked** (podman-only), and **Unmapped** (a ledger orphan with no resolvable work item). GC state is derived from the ledger and the live runtime TTL only, so loading the page makes **no per-row forge call**; the sweep's liveness pass (closed/deleted work items purged before TTL) is shown as a note, not evaluated per row.
+
+Deleting a volume force-stops any in-flight phase container, removes the workspace volume and its `-podman`/`-xdg` siblings, and drops the ledger row — the same teardown as `/reset`. When the ledger row maps to a work item that is still **open**, a `🧹 Volume … removed from the WebUI by <admin>. Mention @devbot to resume.` note is posted to it; closed items, deleted items (forge 404), and untracked volumes are removed silently. The work branch (`wi-<iid>`) and any open merge request are untouched, and the stored `/base` choice is **not** cleared (still `/reset` semantics). Deleting a volume mid-run behaves like `/reset`: the killed run posts its own error reply. The page degrades to a ledger-only list with a warning when `podman volume ls` fails.
+
+The listing is served by `GET /api/volumes?state=&projectId=&q=&limit=&offset=` (session-gated, `{ volumes, total, gcEnabled, ttlMs, podmanAvailable, filterOptions, projectPaths }`); deletion is `DELETE /api/volumes/:workItemId`.
+
+## Installation
+
+```bash
+# Configure the scoped registry once
+npm config set @wascartech:registry https://git.wascar.tech/api/v4/projects/37/packages/npm/
+
+# Install globally
+npm install -g @wascartech/devbot
+
+# Run
+devbot
+```
+
+### Dev channel
+
+Every push to `dev` publishes a prerelease under the `dev` dist-tag, so
+`latest` keeps pointing at the last stable release. Install the newest dev
+build with:
+
+```bash
+npm install -g @wascartech/devbot@dev
+```
+
+Dev versions are stamped `2.1.0-dev.<UTC timestamp>.<short sha>` (the base
+version tracks the upcoming release line).
+
+For private projects, authenticate with a GitLab deploy token or personal access token in `~/.npmrc`:
+```
+//git.wascar.tech/api/v4/projects/37/packages/npm/:_authToken=<your-token>
+```
+
+## Builder image
+
+The builder image is based on Fedora 44 and bundles:
+
+- **Node.js** 24 (Fedora `nodejs24` package)
+- **PHP** 8.5 + composer (extensions: mbstring, xml, curl, gd, pdo, pdo_mysql, zip, pgsql, pdo_pgsql, redis)
+- **Python** 3.14
+- **Chromium**: headless browser for the bundled `chrome-devtools-mcp` MCP server (browser automation, performance tracing, screenshots). Available on both amd64 and arm64.
+- **chrome-devtools-mcp**: globally installed npm package providing Chrome DevTools tools to agents via MCP
+- **Android SDK**: API 34 + build-tools 34.0.0, platform-tools (no NDK). **amd64 only** — Google ships no native arm64 Linux Android SDK, so the arm64 variant of the image omits it.
+- **JDK**: Temurin (Adoptium) OpenJDK 17 (default `JAVA_HOME`)
+- **OpenCode** (latest)
+
+Published to the GitLab Container Registry on every push to `main` (`:main` tag) and on version tags (`:vX.Y.Z` + `:latest`) as a **multi-arch manifest** (amd64 + arm64):
+```bash
+podman pull registry.wascar.tech/opensource/devbot:main
+```
+
+For a custom image, clone this repo and build from `docker/Dockerfile.builder` (native arch, or `--platform linux/amd64,linux/arm64` for a multi-arch manifest):
+```bash
+podman build -t devbot-builder -f docker/Dockerfile.builder .
+```
+
+The image used for a run is resolved per project: **project setting → global setting → `BUILDER_IMAGE` env var**. Set a global builder image in **Global settings → Runtime → Builder image**, and override it per project in **Project settings → Runtime → Builder image** (a per-project value applies to every container spawned for that project: clone/sync, the agent run, peer sync, push, and the nested-podman store init). An empty field inherits the next level, so unset deployments keep using `BUILDER_IMAGE`. A custom image must bundle the same toolchain the default does (`git`, `sh`, `opencode`).
+
+To run the DevBot process itself in a container, `npm run build` then build `docker/Dockerfile.devbot`:
+```bash
+podman build -t devbot -f docker/Dockerfile.devbot .
+```
+
+## Setup
+
+### 1. Configure environment
+
+```bash
+cp .env.example .env
+# edit .env with your GitLab URL, token, bot user ID, model, API keys
+```
+
+### 2. Install and run
+
+```bash
+npm install
+npm run build
+npm start
+```
+
+Dev mode with auto-reload:
+
+```bash
+npm run dev
+```
+
+### 4. Configure the forge webhook
+
+DevBot configures webhooks per project via a web onboarding flow — no manual webhook setup. See [docs/gitlab-setup.md](./docs/gitlab-setup.md) (GitLab) and [docs/platforms.md](./docs/platforms.md) (ForgeJo/Gitea/GitHub). In short:
+
+1. Create an OAuth app with redirect URI `${PUBLIC_URL}/oauth/callback`. GitLab: User Settings → Applications (scope `api`). ForgeJo/Gitea: an app at `/admin/applications`. GitHub: Settings → Developer settings → OAuth Apps (the `repo` scope is requested).
+2. Start DevBot, visit `${PUBLIC_URL}/onboarding`, and sign in with an account whose username is listed in `ONBOARDING_USERS` (comma-separated, case-insensitive, **fails closed** when empty). No forge admin status is required.
+3. Select the projects to onboard. DevBot adds the bot, creates/updates the webhook with a per-project HMAC signing token, and stores the token locally in `devbot.db`.
+4. Mention the bot in a work item: `@devbot /build <prompt>` for build mode, `@devbot <prompt>` for plan mode.
+
+To revoke the bot from a project later, visit `${PUBLIC_URL}/onboarding/managed` (the **Projects** page lists the onboarded projects with a path search; `GET /api/managed?q=&limit=&offset=`). If you rename or transfer an onboarded project on the forge, DevBot reconciles the stored path automatically (on the managed-list load plus a startup/daily sweep), so the displayed name, work-item links, MCP path selectors and clone URLs stay correct.
+
+## MCP servers
+
+DevBot lets an admin attach external MCP servers (Sentry, Context7, Grep, custom local commands, etc.) to each project; OpenCode connects to them during runs. The bot ships two internal, always-active MCP servers that are never user-configurable: `devbot-forge` (forge tools — issues, merge/pull requests, work items, repository files/commits/branches/tags, CI/CD runs/jobs, labels, milestones, members, releases, wikis, scoped search, plus `download_work_item_attachment` (pull a work-item attachment into the workspace) and `upload_work_item_attachment` for sending files back) and `chrome-devtools` (headless Chrome DevTools automation via `chrome-devtools-mcp`). The browser only launches when an agent actually invokes a browser tool, so the idle cost is just a Node stdio process. By default `devbot-forge` is scoped to the run's own project; an admin can grant a project access to another enrolled project from the project row's **Access** button, letting runs read a sibling project and clone its code into a git-excluded `.devbot-peers/` folder. See [docs/mcp-config.md](./docs/mcp-config.md) for the server shapes, the web-panel workflow, the OAuth token paste flow, the REST API, and the storage model, and [docs/project-access.md](./docs/project-access.md) for cross-project grants.
+
+### Reading attachments and sending files back
+
+Work-item attachments are not pre-read; their Markdown links appear inline in the prompt. To inspect one, the agent calls `download_work_item_attachment` with the link exactly as it appears; the tool fetches the whole file into `.devbot-out/incoming/` (git-excluded) and returns its local path. Image/PDF OCR is available on the downloaded file (`tesseract`, `pdftotext`, `pdftoppm` are installed in the builder image).
+
+The agent can share screenshots, test logs, or generated artifacts with `upload_work_item_attachment`: it uploads a file from the run workspace and posts it as a work-item comment (with an optional Markdown caption). Agent scratch output belongs in `.devbot-out/` at the repo root — git-excluded like `.devbot-peers/`, so it can never be committed. Files must live under `/workspace` (the tool rejects traversal and symlink escapes). Upload is native on GitLab (`POST /projects/:id/uploads`) and ForgeJo/Gitea (`POST /repos/:o/:r/issues/:n/assets`); GitHub has no public REST API to attach a file, so the tool reports "not supported on this platform" there.
+
+## Outbound webhooks
+
+Besides receiving forge webhooks, DevBot can **send** signed JSON events to URLs an admin configures — a CI runner, chat bridge, or deployment hook that should react when a run finishes. Configure endpoints per project (**project → Webhooks** tab) or bot-wide (**Settings → Webhooks** tab); a global endpoint fires for every project. Events: `run.started`, `run.succeeded`, `run.failed`, `run.cancelled`, `mr.created`, `mr.updated`.
+
+Deliveries are POSTed with `X-DevBot-Event`, a unique `X-DevBot-Delivery` id, and `X-DevBot-Signature-256: sha256=HMAC-SHA256(body)` keyed by a per-endpoint secret (revealable/rotatable in the panel). They are persisted and retried with a linear backoff (never blocking a run), with a per-endpoint delivery log and manual replay. Custom static headers are supported for receivers that don't implement HMAC. Loopback/private targets are **blocked by default** (`WEBHOOK_ALLOW_PRIVATE_HOSTS=true` to allow internal endpoints). See [docs/outbound-webhooks.md](./docs/outbound-webhooks.md).
+
+## Dev containers
+
+When `DEVCONTAINER_ENABLED=true`, **build-mode** runs get a private rootless
+Podman runtime *inside* the builder container, so an agent can `podman build`,
+run service images, and create private networks for integration tests. Each work
+item has its own runtime store (`devbot-wi-<id>-podman`), so task A cannot see or
+affect task B's containers, images, or networks. Plan mode never gets a runtime,
+and the feature is off by default.
+
+- The runtime is **nested**, not shared: no host Podman socket is mounted.
+- The store lives in a separate volume, never `/workspace`, so container
+  artifacts cannot enter the git diff or be committed.
+- The store is created on the first dev-container run and removed with the work
+  item (`/reset`, work-item close, or the volume GC), exactly like the workspace
+  volume.
+- **Enabling this broadens the builder container's privileges** (nested user
+  namespaces + mount + mknod) versus the default unprivileged launch; hence the
+  off-by-default global toggle.
+
+See [docs/dev-containers.md](./docs/dev-containers.md) for the workflow, the
+storage-driver choice, the isolation guarantees, and the host smoke-test
+runbook.
+
+## Git submodules
+
+When `GIT_SUBMODULES_ENABLED=true` (default), DevBot initializes and updates
+submodules on **first-run clone** and on **every subsequent fetch** by running
+`git submodule update --init --recursive`. This keeps the agent's view of the
+repo complete and picks up new submodules or updated gitlinks between runs.
+
+- **Best-effort.** A submodule that cannot be fetched does **not** abort the run;
+  the affected paths are listed in an informational `ℹ️ Some git submodules could
+  not be fetched: …` note and the run continues without them. This makes the
+  default safe for repos with a private/inaccessible submodule.
+- **Token scoping.** The bot token is attached only to the configured
+  `PLATFORM_URL` host (`http.<PLATFORM_URL>/.extraHeader`), so a submodule on an
+  external host (e.g. `github.com`) is fetched anonymously instead of leaking the
+  token. Public external submodules work; private external ones don't (no
+  per-submodule credentials).
+- **SSH → HTTPS rewrite.** Same-host submodule URLs written as
+  `git@<PLATFORM_URL-host>:…` or `ssh://git@<PLATFORM_URL-host>/…` are rewritten
+  to `PLATFORM_URL` over HTTPS, so they inherit the scoped token and work
+  without an SSH key in the container. If your forge's SSH listens on a
+  non-default port (e.g. `ssh://git@git.example.com:2222/…`), set
+  `PLATFORM_SSH_PORT=2222` so the port-bearing URL is rewritten too. Submodules
+  on other SSH hosts are not rewritten.
+- The submodule step runs after the clone/checkout and before `.devbot/init.sh`,
+  so `init.sh` sees populated submodules.
+
+Set `GIT_SUBMODULES_ENABLED=false` (or `0`) to skip submodules entirely.
+
+## ENV vars & `.devbot/init.sh`
+
+DevBot lets an admin configure per-project environment variables via the web panel. These are injected into every builder container run (so the agent, MCP servers, and `.devbot/init.sh` all see them). A project may include an optional `.devbot/init.sh` script that runs once after the initial `git clone` (e.g. to install deps, run migrations). Non-zero exit aborts the run. See [docs/env-config.md](./docs/env-config.md) for the web-panel workflow, the REST API, the `.devbot/init.sh` contract, and the storage model.
+
+Per-project model, model variant, plugin, and base-branch overrides live in the same web panel (open a project row → **Settings** tab) and default to the global values set in **Settings**. See [docs/project-settings.md](./docs/project-settings.md) and [docs/global-settings.md](./docs/global-settings.md).
+
+## Configuration
+
+Infrastructure and identity variables only — provider, model, plugin, base-branch, and runtime (timeout/queue/attachment/maintenance/dev-container) config moved to the WebUI ([docs/global-settings.md](./docs/global-settings.md)). The runtime vars below remain as the **fallback** an empty WebUI field inherits.
+
+| Variable | Description | Default |
+|---|---|---|
+| `PLATFORM` | Forge backend: `gitlab`, `forgejo`, `gitea`, or `github`. One instance ↔ one forge. | `gitlab` |
+| `PLATFORM_URL` | Forge base URL | — |
+| `PLATFORM_TOKEN` | Bot access token (api scope; Gitea/ForgeJo: write:repository + write:issue; GitHub: repo scope) | — |
+| `PLATFORM_SSH_PORT` | Forge SSH port for the same-host submodule SSH→HTTPS rewrite (set when SSH is not on 22) | `22` |
+| `PUBLIC_URL` | Public base URL of the bot (webhook URL + OAuth redirect) | — |
+| `PLATFORM_OAUTH_CLIENT_ID` | OAuth app client id (for the onboarding flow) | — |
+| `PLATFORM_OAUTH_CLIENT_SECRET` | OAuth app secret | — |
+| `ONBOARDING_USERS` | Comma-separated forge usernames allowed to sign in to the WebUI onboarding flow (all platforms). **Fails closed** when empty. | `""` |
+| `GITHUB_API_URL` | GitHub only: API base (defaults to github.com) | `https://api.github.com` |
+| `SESSION_KEY` | 32+ char key for encrypting the session cookie (AES-GCM) | — |
+| `BOT_USERNAME` | Bot account username (for mention regex) | — |
+| `BOT_USER_ID` | Bot account numeric ID (for self-loop guard + project invite) | — |
+| `BOT_GIT_NAME` | Git commit author name | `DevBot` |
+| `BOT_GIT_EMAIL` | Git commit author email (use the bot's verified email for profile linking) | — |
+| `BUILDER_IMAGE` | Podman builder image tag (fallback when unset in the WebUI; global + per-project overrides live in settings) | `devbot-builder:latest` |
+| `PLAN_TIMEOUT_MS` | Per-run timeout for plan mode (ms) (WebUI Runtime override) | `600000` |
+| `BUILD_TIMEOUT_MS` | Per-run timeout for build mode (ms) (WebUI Runtime override) | `1800000` |
+| `RUN_STALL_TIMEOUT_MS` | Kill an agent run that produces no output for this long (ms); bounds a wedged or looping agent before the total phase timeout. 0 disables (env-only) | `900000` |
+| `MAX_QUEUE_PER_WORK_ITEM` | Max pending runs per work item (overflow dropped) (WebUI Runtime override) | `3` |
+| `MAX_CONCURRENT_RUNS` | Max simultaneous podman builder runs across all work items (WebUI Runtime override) | `4` |
+| `VOLUME_GC_TTL_MS` | Inactivity period after which a per-work-item volume is auto-purged (WebUI Runtime override) | `604800000` (1 week) |
+| `VOLUME_GC_INTERVAL_MS` | How often the GC sweep runs (WebUI Runtime override; live re-arm) | `86400000` (daily) |
+| `VOLUME_GC_ENABLED` | Enable the volume GC (startup + periodic sweep) (WebUI Runtime override) | `true` |
+| `ERROR_LOG_RETENTION_DAYS` | Days to keep persisted container/run error logs (startup + daily purge; 0 = keep forever) (WebUI Runtime override) | `30` |
+| `RUN_LOG_TTL_HOURS` | Hours to keep per-run log files (startup + hourly purge; 0 = keep forever). Governs files only — history rows persist (WebUI Runtime override) | `48` |
+| `RUN_LOG_DIR` | Directory for the per-run log files (relative to the process cwd unless absolute). Mount a volume here in a container deployment (env-only) | `run-logs` |
+| `PROJECT_SYNC_ENABLED` | Reconcile enrolled project paths from the forge after a rename/transfer (startup + periodic sweep; WebUI list always refreshes) (WebUI Runtime override) | `true` |
+| `PROJECT_SYNC_INTERVAL_MS` | How often the project-path sweep runs (WebUI Runtime override; live re-arm) | `86400000` (daily) |
+| `MAX_GLOBAL_QUEUE` | Max pending tasks in the global FIFO (0 = unbounded; overflow dropped with ⚠️) (WebUI Runtime override) | `50` |
+| `QUEUE_POLL_INTERVAL_MS` | Worker poll cadence (ms) for the DB-backed queue | `250` |
+| `MAX_RETRIES` | App-level retry on transient failures (podman spawn, git network). 0 = disabled (WebUI Runtime override) | `2` |
+| `RETRY_BACKOFF_MS` | Linear backoff base for retry: `RETRY_BACKOFF_MS * attempts` (WebUI Runtime override) | `5000` |
+| `QUEUE_FEEDBACK` | Post `⏳`/`⚠️`/`📨`/`▶️`/`🔁` queue notes to the work item (WebUI Runtime override) | `true` |
+| `PORT` | HTTP listen port | `3000` |
+| `DRY_RUN` | If true, skip podman/opencode; log prompt only | `0` |
+| `ATTACHMENT_MAX_BYTES` | Attachment size cap for agent downloads and outbound uploads (larger files are rejected) (WebUI Runtime override) | `10000000` |
+| `DEVCONTAINER_ENABLED` | Enable nested rootless Podman for build-mode runs (private runtime per work item). Broadens the builder container's privileges — see [Dev containers](#dev-containers). (WebUI Runtime override) | `false` |
+| `DEVCONTAINER_STORAGE_DRIVER` | Inner runtime storage driver: `auto` (probe `/dev/fuse` → `fuse-overlayfs`, else `vfs`), `fuse-overlayfs`, or `vfs`. (WebUI Runtime override) | `auto` |
+| `GIT_SUBMODULES_ENABLED` | On first-run clone and every fetch, run `git submodule update --init --recursive` (best-effort: failures are named in an `ℹ️` note and don't abort the run). Bot token is scoped to `PLATFORM_URL`; same-host `git@host:`/`ssh://git@host/` submodule URLs are rewritten to HTTPS (see `PLATFORM_SSH_PORT`). (WebUI Runtime override) | `true` |
+| `WEBHOOK_ALLOW_PRIVATE_HOSTS` | Allow outbound webhook endpoints on loopback/private/CGNAT/ULA addresses (blocked by default, SSRF hardening). See [Outbound webhooks](#outbound-webhooks). | `false` |
+| `WEBHOOK_DELIVERY_TIMEOUT_MS` | Per-attempt HTTP timeout for an outbound webhook delivery | `10000` |
+| `WEBHOOK_RETRY_MAX` | Total attempts per delivery before it is marked terminally failed | `5` |
+| `WEBHOOK_RETRY_BACKOFF_MS` | Linear backoff base for retries (`base * attempts`) | `5000` |
+| `WEBHOOK_POLL_INTERVAL_MS` | Outbound delivery worker poll cadence | `1000` |
+| `WEBHOOK_RETENTION_DAYS` | Days to keep outbound delivery rows (startup + daily purge; 0 = keep forever) | `30` |
+
+> **Migration (global settings)**: `LLM_MODEL`, `LLM_API_KEY`, `LLM_BASE_URL`, `PLAN_LLM_MODEL`, `BUILD_LLM_MODEL`, `OPENCODE_PLUGINS`, `BASE_BRANCHES`, `ATTACHMENT_OCR_LANG`, and `GITLAB_BOT_URL` are removed from the schema and ignored — there is no auto-import. Configure everything in **Settings**.
+>
+> **Runtime settings**: timeouts, queue caps/retries, attachment limits, volume GC, project sync, dev containers, and git submodules are now editable in **Settings → Runtime**. Their env vars remain as the fallback: an empty WebUI field inherits the env value (shown as the placeholder), which itself falls back to the built-in default. `DRY_RUN` and `LOG_LEVEL` stay env-only.
+
+> **Migration (from `MR_TARGET_BRANCH`)**: the former single-value `MR_TARGET_BRANCH` env var is removed. Set the base-branch list in the WebUI global settings (default `dev,master`, top entry tried first).
+
+## Safety
+
+- **Self-loop guard**: notes authored by the bot never re-trigger.
+- **Trigger authorization**: a mention triggers the bot only if it passes the configured trigger policy. The baseline has two gates, each a global default with a per-project override: **require author/assignee** (default on) — the mentioning user must be the work item's author or assignee; and **minimum project role** (default Developer, i.e. access_level ≥ 30). The denial note names the resolved policy. Mention failures to resolve the actor's role or the work item fail closed (deny + note). See [docs/global-settings.md](./docs/global-settings.md#trigger-policy) and [docs/project-settings.md](./docs/project-settings.md#permissions).
+- **Per-project access rules** (optional): an admin can refine the role gate per project by adding a rule for a **role** (normalized permission: none/read/write/admin/owner) or a specific **user** (resolved by username to a stable numeric id). Each rule sets a level — **Full** (plan + build), **Plan only**, or **Denied**. A rule replaces the role gate for the matched subject (it can broaden or narrow), but cannot bypass the author/assignee gate. An exact user rule beats a role rule. Denied users are ignored silently (204, no reply, no run); plan-only users are ignored on build/force-push/push/mr mentions. Rules are read live, so changes take effect immediately.
+- **Access mode + per-project whitelist**: the baseline authorization above is always enforced. The global **Access mode** switch (Global settings) selects **Role based** (baseline only) or **Whitelist mode**; each project can override that mode from its own Settings panel. In whitelist mode an additional per-project list (forge usernames, resolved to stable numeric ids) must contain the mentioning user. A user who passes the baseline rule but is not listed is **silently ignored** (204, no reply, no run); a user who fails the baseline rule is still denied with the usual `🚫` note — the whitelist narrows access, never grants it. Entries are read live, so adding/removing one takes effect immediately. See [docs/project-settings.md](./docs/project-settings.md#access-mode--whitelist).
+- **Per-work-item FIFO queue**: concurrent webhooks for the same work item are queued (max `MAX_QUEUE_PER_WORK_ITEM` pending). The queue is DB-backed (`pending_tasks` table); a restart resets orphaned running rows to queued and replays them. Position > 0 gets a `⏳ Queued (position N)` note; overflow gets `⚠️ Queue full, retry later.`. Different work items run in parallel up to `MAX_CONCURRENT_RUNS` simultaneous runs; beyond that, tasks wait in a global FIFO (also DB-backed). The thread is notified at three milestones: `⏳ Queued globally (position N).` at enqueue, `⏳ You're next.` when it reaches the head of the queue, and one wait-aware start note. The start note is posted whenever the task waited in its own work-item queue or for a global slot, measures the wait from the row's `enqueued_at`, and names how many tasks are waiting plus which one is next. Prompt excerpts are rendered as inline code so user text cannot mention users or break the note layout. Order is preserved so each mention resumes the same OpenCode session and sees prior context. A global overflow cap (`MAX_GLOBAL_QUEUE`) drops excess with `⚠️ Global queue full, retry later.`
+- **App-level retry**: transient failures (podman spawn, git clone/fetch network blip) are retried up to `MAX_RETRIES` times with linear backoff (`RETRY_BACKOFF_MS * attempts`). The same row is requeued (preserving its `seq` so FIFO order is unchanged). Terminal failures (timeout, branch divergence, agent errors) are not retried. A `🔁 Retrying (attempt N/M).` note is posted on each retry; `⚠️ Gave up after M attempts.` on exhaustion.
+- **Run timeout**: runaway agents are killed after `PLAN_TIMEOUT_MS` (plan mode) or `BUILD_TIMEOUT_MS` (build mode). The podman container is `rm -f`'d on timeout, not just the client process. A run that goes completely silent for `RUN_STALL_TIMEOUT_MS` is killed earlier with a distinct "stalled" note (env-only; `0` disables) — this bounds a wedged opencode process or a model looping on a sandbox-blocked tool call.
+- **In-flight stop**: `/stop` (work-item scope) or the WebUI row **Stop** button (task scope) sets a durable `cancel_requested` flag on the queued/running row and `rm -f`s the work item's phase containers. The runner checks the flag at each phase boundary (post-attachment, post-clone/sync, post-run, pre-push, pre-MR) and unwinds as a clean stop: no `⚠️` error reply, no error-log row, and no push/MR. Authorization is the same as any mention (Developer+ AND author/assignee); a stop works even when the bot is unconfigured. The flag is durable so a crash mid-stop does not replay the task on restart (`purgeCancelled` runs before replay). The per-work-item volume is untouched — use `/reset` to discard it.
+- **No silent rebase loss**: if `git pull --rebase` fails on a diverged branch, the run aborts with a note rather than discarding local commits.
+- **Force push is opt-in and leased**: `/force-push` overwrites the `wi-<iid>` remote branch using `--force-with-lease` (never plain `--force`). The lease refuses the push if the remote moved since the run's fetch, so a concurrent/upstream change is not clobbered. It only ever touches the work item branch — the MR target branch is untouched. With `/build` the host-side HEAD guard and fallback-commit rescue still apply. Standalone (no `/build`), `/force-push` runs no agent: it just fetches and force-pushes the existing local `wi-<iid>` branch (a repair action after a normal push was refused); if the workspace or the local branch doesn't exist it posts an actionable note and pushes nothing.
+- **Branch naming is fail-closed**: `/branch <name>` names the work branch on the first naming only, after verifying the name does not already exist on the remote. A collision, an unsafe name, or a forge lookup failure rejects the mention and runs nothing. The chosen name is stored per work item and reused on every later run (including after `/reset` and close); it is never auto-renamed.
+- **Rewrite-aware publish**: the post-build host-side push detects a rewritten branch history (e.g. after a rebase or a conflict auto-fix) and, when the remote still equals the run's **starting** tip, publishes with a SHA-pinned `--force-with-lease` instead of failing with git's raw fast-forward hint. If the remote moved during the run, the push is refused with a clean "re-run" note — it never clobbers commits it did not produce. The fallback-commit rescue also refuses to commit a working tree that still contains conflict markers.
+- **Conflict auto-fix is opt-in and fails closed**: when enabled, DevBot checks the branch against its MR target and delegates resolution to a direct build run (rebase-first, merge fallback; the resolver never pushes). It only opens/updates the MR after a clean re-check; an unresolved conflict leaves the branch pushed but opens no MR and posts the conflicting paths. With the switch off there is no check and no behavior change.
+- **Per-run auth**: the provider `auth.json`, the per-run `opencode.json` (carrying the scoped MCP bearer token), and pasted `mcp-auth.json` are written at mode 0600 into the per-work-item **xdg volume** (`devbot-wi-<id>-xdg`, mounted at `/home/devbot/.xdg`) — outside the cloned repository, so the agent's `git add -A` can never commit them. No keys are baked into the image.
+
+## Known ceilings (documented, not built)
+
+- **In-process idempotency**: a restart mid-run + a webhook retry can double-run. The dedup table uses `INSERT OR IGNORE` for atomicity, but it's still single-host (no shared store). Acceptable at this scale; upgrade path is a shared Redis SETNX if HA lands.
+- **Volumes auto-purged after inactivity or on close/deletion**: per-work-item volumes (`devbot-wi-<id>` and its `-podman` / `-xdg` siblings, tracked as one ledger row) are deleted after `VOLUME_GC_TTL_MS` (default 1 week) of no run completing, even if the work item stays open. The sweep additionally runs a **liveness pass**: a volume whose work item is closed or deleted (forge 404) is purged immediately rather than waiting out the TTL — a deleted item can never run again, and a closed one matches the close-hook behavior. A `🧹 Volume purged …` note is posted to the work item on purge (skipped for deleted items and for un-mappable orphans, which have no thread); mention `@devbot` to resume (the next run re-clones into a fresh volume). Volumes created before the tracking table existed are reconciled at startup, stamped with their **real** `podman` creation time (so old orphans are TTL-eligible at once), and purged silently. The sweep runs on startup and every `VOLUME_GC_INTERVAL_MS` (default daily); set `VOLUME_GC_ENABLED=false` to disable. Close and `/reset` still remove the volumes immediately and clean the ledger row.
+- **Attachment content is agent-handled**: DevBot does not extract or OCR attachments host-side. The agent must download each one it needs with `download_work_item_attachment`; a file the agent never downloads is never read. Content is treated as untrusted data.
+- **No attachment cache across runs**: a downloaded attachment lives in the disposable workspace volume and is re-downloaded after a close/reopen cycle; the tool does not dedupe across runs. Within a run, a repeat download of the same reference is suffixed (`-1`, `-2`) rather than returned from cache.
+- **GitHub attachment upload unsupported**: GitHub has no public REST API to attach a file to an issue or comment, so `upload_work_item_attachment` reports "not supported on this platform" on `PLATFORM=github`. GitLab and ForgeJo/Gitea upload natively.
+- **Private external submodules unsupported**: submodules are fetched anonymously on external hosts (the bot token is scoped to `PLATFORM_URL` only). A private submodule on another host needs a per-project credential store, not built. Same-host submodules (HTTPS, relative, or SSH-to-HTTPS-rewritten) work with the bot token.
+- **Submodule state not updated remotely**: `git submodule update` (no `--remote`) checks out the gitlink commit recorded by the superproject, matching git's default. Advancing submodules to their remote branch tip is not done.
+- **Stop mid-clone can leave a partial volume**: stopping during the first-run clone or `.devbot/init.sh` kills the container mid-setup, so the per-work-item volume can be partially initialized. The next mention re-syncs (and re-runs `init.sh` only on a detected empty workspace), but if the volume is wedged use `/reset` to discard and rebuild it. Stop never removes the volume itself.
+- **ForgeJo/Gitea webhook secret is create-only**: both forges honor `config.secret` when the hook is created but silently ignore it on `PATCH` (`editHook`), and the API never returns the stored secret. DevBot sends the secret in the create payload, so fresh onboarding works; an already-onboarded hook with an empty secret must be fixed in the forge UI (paste `project_tokens.signing_token`) or by revoke + re-onboard. See [docs/platforms.md](./docs/platforms.md).
+
+## Development
+
+```bash
+npm run typecheck   # strict tsc --noEmit
+npm run build       # tsc → dist/ + tailwindcss → public/app.css
+npm run test:signature   # HMAC webhook-signature self-check
+npm run test:mention     # mention parser self-check (multi-agent, error cases)
+npm run test:global-queue # global concurrency gate self-check
+npm run test:queue       # DB-backed per-work-item queue self-check
+npm run test:queue-store # queue store (claim/requeue/markDone) self-check
+npm run test:idempotency # webhook-id dedup self-check
+npm run test:migrate     # migration runner self-check
+npm run test:dispatch    # dispatch integration self-check (retry, overflow, replay)
+npm run test:mr-content  # agent-authored MR block parser self-check
+npm run test:attachments # attachment discovery + download self-check
+npm run test:base-branch # /base branch store self-check
+npm run test:work-item-branch # /branch work-branch store self-check
+npm run test:branch-naming # /branch Step-0 gate self-check
+npm run test:volume-manage # WebUI volume list + delete self-check
+npm run test:global-settings # global settings + provider store self-check
+npm run test:runtime-settings # ENV -> WebUI runtime resolver + live apply self-check
+npm run test:provider-registry # models.dev normalization self-check
+npm run test:mcp-download-tool # download_work_item_attachment handler self-check
+npm run test:config-generation # auth.json + provider block self-check
+npm run test:ui-e2e   # browser E2E for the whole WebUI (Chrome DevTools MCP; skips if no browser)
+npm run test:path-sandbox # path policy + guard plugin parity self-check
+npm run test:shell-sandbox # managed config + bwrap shell wrapper self-check
+npm run test:podman-volume # real-Podman volume + readFileFromVolume deny/symlink self-check
+npm run test:podman-run    # real-Podman runPodmanCapture + logs self-check
+npm run test:nested-podman # nested rootless Podman smoke (nightly; needs builder image)
+npm run test:devcontainer # nested-podman flag/driver builder self-check
+npm run test:devcontainer-wiring # store volume lifecycle self-check
+npm run test:platform-webhooks # platform webhook parse + signature self-check
+npm run test:platform-drivers  # Gitea/ForgeJo driver contract self-check
+npm run test:ci-adapter  # GitLab + Actions CI normalization self-check
+npm run test:github-webhooks   # GitHub webhook signature + event parsing self-check
+npm run test:github-driver     # GitHub driver contract + ZIP log self-check
+npm run test:onboarding-allowlist # ONBOARDING_USERS gate self-check (fail closed)
+npm run test:outbound-webhooks # outbound endpoint store + delivery worker + signing/SSRF self-check
+npm test            # (none — see acceptance checks below)
+npm run test:custom-repos # custom skills/agents repo store + sync self-check
+```
+
+### Browser E2E (`test:ui-e2e`)
+
+`scripts/check-ui-e2e.ts` drives the built WebUI through the bundled
+`chrome-devtools-mcp` server (the same one runs use) against a real server +
+temp SQLite DB and a mock GitLab. It covers the sign-in gate, managed list,
+Settings (registry + custom providers, models), ENV, MCP,
+per-project Settings, and Remove project — asserting both rendered DOM and
+server persistence.
+
+It **skips** (exit 0) when `chrome-devtools-mcp` or a Chromium binary is absent,
+so `npm test` stays green without a browser. Override with
+`CHROME_DEVTOOLS_MCP_BIN`, `CHROMIUM_BIN`, or force-skip with
+`DEVBOT_E2E_SKIP=1`. The harness starts the server with
+`DEVBOT_TEST_AUTH=1` to sign a session without GitLab OAuth — that flag
+must never be set in production. It always rebuilds `public/` first, so a
+stale SPA build from an earlier run cannot silently serve old code.
+
+### Test profiles (`DEVBOT_TEST_PROFILE`)
+
+`scripts/run-all-tests.mjs` auto-discovers every `test:*` script. CI splits them
+by runtime requirement:
+
+| Profile | Suites | Where |
+| --- | --- | --- |
+| `hermetic` (default for the `test` job) | everything except the tooling suites | slim `node:22-slim` |
+| `integration` | `test:podman-*`, `test:ui-e2e`, `test:shell-sandbox` | privileged runner (`test:integration`) |
+| `nested` | `test:nested-podman` | nightly (`test:nested-podman`) |
+| `all` | every suite (default with no env var) | local `npm test` |
+
+The integration suites self-skip when Podman/Chromium/bubblewrap is absent, but
+the CI integration and nightly jobs set `DEVBOT_REQUIRE_PODMAN=1`, which turns a
+failed Podman probe (or a non-working nested run) into a hard failure with the
+captured reason rather than a silent skip — so missing coverage cannot hide. On
+a dev host without the tooling the suites skip gracefully.
+
+For a manual interactive pass, `node --experimental-sqlite --import tsx
+scripts/e2e-serve.ts` seeds a temp DB and serves the built app with the same
+test-login route, printing the URL to open in a browser.
+
+### Acceptance checks (DRY_RUN)
+
+```bash
+DRY_RUN=1 npm start
+# missing token → 401
+# wrong token → 401
+# bot-authored note → 204 (self-loop guard)
+# non-issue note → 204
+# no @bot mention → 204
+# valid note hook → 202 + prompt logged
+# valid work item hook (open + mention in description) → 202 + prompt logged
+```
+
+### Skills & agents
+
+Custom skills and subagents come from git repositories an admin registers in **Settings → Skills & Agents** (globally, with optional per-project overrides). DevBot shallow-clones each repo once per work item and fetches thereafter; skill dirs are added to OpenCode's `skills.paths`, and subagent markdown is flattened into the config dir OpenCode scans. See [docs/custom-repos.md](./docs/custom-repos.md).
+
+## Layout
+
+```
+src/
+  config.ts        env validation (zod)
+  logger.ts        tiny console logger
+  migrate.ts       versioned migration runner (schema_migrations + migrations/*.sql)
+  platform/        forge abstraction: Platform port + drivers
+    types.ts       neutral domain types + NormalizedWebhookEvent + errors + attachment types
+    port.ts        aggregate Platform interface (capability sub-interfaces) + CAPABILITY_METHODS
+    index.ts       getPlatform(): driver selected by PLATFORM
+    git-auth.ts    Basic-auth git extraHeader helper
+    gitlab/        GitLab driver (all GitLab code lives here)
+      http.ts      schema-validated fetch, token auth, pagination, raw bytes
+      schemas.ts   GitLab wire schemas (API + webhooks)
+      schemas-read.ts  GitLab read-endpoint schemas
+      api.ts       typed GitLab REST client (notes, MRs, members, …)
+      read.ts      read-only cross-project resources (MCP)
+      admin.ts     OAuth-token REST calls (onboarding)
+      oauth.ts     OAuth authorize/token/user helpers
+      signature.ts Standard Webhooks HMAC verification
+      attachments.ts  GitLab upload discovery + fetch
+      index.ts     capability assembly
+    gitealike/     shared Gitea/ForgeJo /api/v1 driver
+      http.ts      schema-validated fetch, token auth, pagination, raw bytes
+      schemas.ts   Gitea/ForgeJo wire schemas
+      base.ts      capability implementations
+      forgejo.ts   ForgeJo overrides (X-Forgejo-* headers, OAuth paths)
+      gitea.ts     Gitea overrides (X-Gitea-* headers)
+      attachments.ts  Gitea/ForgeJo attachment discovery + fetch
+    github/        GitHub (github.com) driver
+      http.ts      schema-validated fetch, bearer auth, pagination, raw bytes
+      schemas.ts   GitHub REST wire schemas
+      zip.ts       minimal ZIP reader for Actions log archives
+      attachments.ts  GitHub attachment discovery + fetch
+      index.ts     capability implementations
+  onboarding-allowlist.ts  ONBOARDING_USERS gate for all drivers (fails closed)
+  queue.ts         per-issue DB-backed FIFO queue (worker-per-key, claim/markDone/retry)
+  global-queue.ts  global concurrency gate (DB backlog + in-memory active counter)
+  queue-store.ts   SQLite store for pending_tasks (claim/requeue/markDone/rank/housekeeping)
+  mention.ts       mention parser + subagent resolver (pure, testable)
+  mr-content.ts    agent-authored MR/PR block parser (pure, testable)
+  types/           zod schemas: env, opencode, outbound-webhook, mcp-config, access
+  cookie.ts        AES-GCM session cookie encrypt/decrypt
+  token-store.ts   SQLite store for per-project whsec_ signing tokens (node:sqlite)
+  access-grant-store.ts  SQLite store for directed cross-project MCP access grants
+  mcp-auth-store.ts   SQLite store for per-project/global MCP OAuth entries (encrypted)
+  env-config-store.ts SQLite store for per-project env vars (encrypted values)
+  global-settings-store.ts  SQLite store for global settings + provider registry
+  types/provider-registry.ts  models.dev registry types + normalizer (pure, shared with the browser)
+  base-branches.ts          base-branch list parser (pure)
+  devbot-scripts.ts   .devbot/ script phase builders (init.sh post-clone today)
+  base-branch-store.ts  SQLite store for per-work-item /base branch choice (node:sqlite)
+  work-item-branch-store.ts  SQLite store for per-work-item /branch work branch (node:sqlite)
+  change-request-store.ts  SQLite registry of MRs DevBot created (CI-failure correlation + dedup)
+  ci-failure.ts    CI-failure context: failing job log tails for the auto-report plan run
+  outbound-webhook.ts        outbound event emitter + signed delivery worker
+  outbound-webhook-store.ts  SQLite store for outbound endpoints + delivery queue
+  idempotency.ts   atomic webhook-id dedup (INSERT OR IGNORE, TTL window)
+  views.ts         server-rendered HTML for the onboarding flow (Tailwind)
+  styles/          tailwind entry (compiled to public/app.css at build time)
+  opencode.ts      spawn podman, parse NDJSON, extract text parts
+  devcontainer.ts  nested rootless Podman flag/driver/storage helpers (pure)
+  attachments.ts   forge-free attachment discovery/filename helpers
+  runner.ts        volume lifecycle, clone/pull, run, commit/push (returns RunOutcome)
+  server.ts        Express webhook receiver + onboarding routes + filter chain
+  mcp/
+    peer-sync.ts   on-demand peer clone/fetch into the git-excluded .devbot-peers/ dir
+    volume-files.ts  read agent files from the run volume + git-excluded .devbot-out/ staging
+    tools/         internal devbot-forge MCP tool handlers (incl. cross-project reads + attachment upload)
+AGENTS.md          agent workflow + platform-parity rules
+migrations/
+  0001_initial.sql       baseline: all current tables (idempotent)
+  0002_queue_status.sql  pending_tasks status/attempts/claimed_at/enqueued_at/last_error
+  0005_global_settings.sql  global_settings + global_providers; drops project_settings.base_model
+  0006_project_access_grants.sql  directed cross-project access grants
+  0022_auto_resolve_conflicts.sql  conflict auto-fix column on both settings tables
+  0023_ci_fail_auto_report.sql  ci_fail_auto_report columns + change_requests registry
+scripts/
+  check-platform-webhooks.ts  platform webhook parse + signature self-check
+  check-platform-drivers.ts   Gitea/ForgeJo driver contract self-check
+  check-platform-parity.ts    port coverage + no-leak parity self-check
+  check-ci-adapter.ts         GitLab + Actions CI normalization self-check
+  check-ci-fail-report.ts     CI-failure registry + log context + resolver self-check
+  check-github-webhooks.ts    GitHub webhook signature + event parsing self-check
+  check-github-driver.ts      GitHub driver contract + ZIP log self-check
+  check-onboarding-allowlist.ts  ONBOARDING_USERS gate self-check (fail closed)
+  check-signature.ts  HMAC webhook-signature self-check (npm run test:signature)
+  check-mention.ts    mention parser self-check (npm run test:mention)
+  check-attachments.ts attachment discovery + download self-check (npm run test:attachments)
+  check-upload-attachments.ts  outbound upload (multipart, path validation, volume read) self-check
+  check-mcp-upload-tool.ts     upload_work_item_attachment handler self-check
+  check-mcp-download-tool.ts   download_work_item_attachment handler self-check
+  check-base-branch-store.ts  /base store self-check (npm run test:base-branch)
+  check-work-item-branch-store.ts  /branch work-branch store self-check (npm run test:work-item-branch)
+  check-branch-naming.ts      /branch Step-0 gate self-check (npm run test:branch-naming)
+  check-devcontainer.ts  nested-podman helpers self-check (npm run test:devcontainer)
+  check-devcontainer-wiring.ts  store volume lifecycle self-check (npm run test:devcontainer-wiring)
+  check-queue.ts      DB-backed queue self-check (npm run test:queue)
+  check-global-queue.ts  global gate self-check (npm run test:global-queue)
+  check-queue-store.ts   queue store self-check (npm run test:queue-store)
+  check-idempotency.ts   idempotency self-check (npm run test:idempotency)
+  check-migrate.ts      migration runner self-check (npm run test:migrate)
+  check-dispatch.ts     dispatch integration self-check (npm run test:dispatch)
+  check-access-grants.ts  cross-project grant store self-check (npm run test:access-grants)
+  check-mcp-scope.ts    MCP cross-project scope resolver self-check (npm run test:mcp-scope)
+  check-peer-sync.ts    peer clone/fetch command builder self-check (npm run test:peer-sync)
+  check-outbound-webhooks.ts  outbound endpoint store + delivery worker + signing/SSRF self-check
+tailwind.config.cjs  Tailwind CLI config (v3, CommonJS)
+opencode-config/   written/mounted into the run container
+  opencode.json    base config (per-run config is generated into the xdg volume)
+  plugins/         path-guard plugin source
+  managed/         root-owned managed config baked into the builder image
+Dockerfile.builder node + git + build tools + opencode-ai + tesseract/poppler (attachment OCR)
+Dockerfile.devbot   DevBot process image (node)
+```
+
+## License
+
+Apache License 2.0 — see [LICENSE](./LICENSE).