@miroshnik/elves 0.0.0 → 2026.1018.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (68) hide show
  1. package/AGENTS.md +310 -0
  2. package/README.md +344 -0
  3. package/bin/ai-dev.ts +7 -0
  4. package/bin/elves.ts +2847 -0
  5. package/bin/shims/timeout +113 -0
  6. package/claude/CLAUDE.md +71 -0
  7. package/dist/github/scripts/github.js +3428 -0
  8. package/dist/spec/scripts/architecture.js +628 -0
  9. package/dist/spec/scripts/eslint-config.js +54 -0
  10. package/dist/spec/scripts/harness-qzb932c8.js +959 -0
  11. package/dist/spec/scripts/harness-v9s1v9w3.js +16045 -0
  12. package/dist/spec/scripts/harness.js +53 -0
  13. package/dist/spec/scripts/knip-hints.cjs +34 -0
  14. package/docs/ci-concurrency.md +82 -0
  15. package/docs/cloud-sessions.md +203 -0
  16. package/docs/judgment.md +81 -0
  17. package/docs/parallel-checkouts.md +220 -0
  18. package/docs/pr-checks.md +117 -0
  19. package/docs/testing.md +304 -0
  20. package/package.json +70 -1
  21. package/skills/ci-wait/SKILL.md +180 -0
  22. package/skills/ci-wait/scripts/wait-ci.sh +377 -0
  23. package/skills/dashboard/SKILL.md +212 -0
  24. package/skills/dashboard/scripts/dashboard.ts +912 -0
  25. package/skills/dashboard/scripts/sessions.ts +772 -0
  26. package/skills/est/SKILL.md +221 -0
  27. package/skills/est/reference.md +381 -0
  28. package/skills/est/scripts/est.ts +4854 -0
  29. package/skills/github/SKILL.md +206 -0
  30. package/skills/github/locales/en.json +449 -0
  31. package/skills/github/locales/ru.json +449 -0
  32. package/skills/github/reference.md +1073 -0
  33. package/skills/github/scripts/github.ts +4390 -0
  34. package/skills/github/scripts/locale.ts +495 -0
  35. package/skills/github/scripts/package.json +3 -0
  36. package/skills/slot/SKILL.md +145 -0
  37. package/skills/slot/scripts/package.json +3 -0
  38. package/skills/slot/scripts/slot.ts +279 -0
  39. package/skills/spec/SKILL.md +247 -0
  40. package/skills/spec/canon.md +219 -0
  41. package/skills/spec/checks.md +709 -0
  42. package/skills/spec/ci.md +378 -0
  43. package/skills/spec/migration.md +125 -0
  44. package/skills/spec/reference.md +889 -0
  45. package/skills/spec/scripts/architecture.ts +839 -0
  46. package/skills/spec/scripts/c4.ts +184 -0
  47. package/skills/spec/scripts/eslint-config.ts +82 -0
  48. package/skills/spec/scripts/harness.ts +1483 -0
  49. package/skills/spec/scripts/knip-hints.cjs +34 -0
  50. package/skills/spec/scripts/knip-hints.d.cts +17 -0
  51. package/skills/spec/scripts/package-run.ts +22 -0
  52. package/skills/spec/scripts/package.json +3 -0
  53. package/skills/spec/scripts/spec-break.ts +232 -0
  54. package/skills/spec/scripts/spec-claims.ts +198 -0
  55. package/skills/spec/scripts/spec-diff.ts +1111 -0
  56. package/skills/spec/scripts/spec-doc.ts +744 -0
  57. package/skills/spec/scripts/spec-exceptions.ts +233 -0
  58. package/skills/spec/scripts/spec-publish.ts +214 -0
  59. package/skills/spec/scripts/spec-run.ts +235 -0
  60. package/skills/spec/scripts/spec-shard.ts +267 -0
  61. package/skills/spec/scripts/speclib.ts +826 -0
  62. package/skills/spec/scripts/vendor/babel-parser.LICENSE +19 -0
  63. package/skills/spec/scripts/vendor/babel-parser.cjs +14615 -0
  64. package/skills/spec/scripts/vendor/babel-parser.d.cts +3 -0
  65. package/skills/task-env/SKILL.md +231 -0
  66. package/skills/task-env/proxy/compose.yaml +51 -0
  67. package/skills/task-env/reference.md +254 -0
  68. package/skills/task-env/scripts/task-env.ts +1017 -0
@@ -0,0 +1,220 @@
1
+ # Parallel checkouts: the task's Compose stack, addresses through the proxy, shared `.git/config` — for "Subagents and worktrees"
2
+
3
+ Reference for the "Subagents and worktrees" section of `AGENTS.md`: the full
4
+ rule, what the project's compose file holds, what moves with the address,
5
+ callbacks of external services, the runner config and the sandbox. The stack
6
+ itself — its names, the e2e stack and its variables, profiles, taking it
7
+ down — is the `task-env` skill; here are links to it, not a copy. Shared git
8
+ config — the "Shared `.git/config`" section at the end.
9
+
10
+ ## Rule
11
+
12
+ **Parallel checkouts share ports, databases and cookies.** Two sessions of
13
+ one project on ports of the host fight over a port, migrate one database to
14
+ different branches and share Redis, queues and mail; on `localhost` they share
15
+ cookies too — cookies don't tell ports apart, and the agent's browser and the
16
+ owner's log each other out. So the task's apps, databases, queues, mail and
17
+ e2e run in its own Docker Compose stack — a copy of the project's compose
18
+ file, the `task-env` skill:
19
+
20
+ ```bash
21
+ bun --no-env-file <skill dir>/scripts/task-env.ts up # the task's stack by the branch, its addresses
22
+ bun --no-env-file <skill dir>/scripts/task-env.ts e2e -- npx playwright test # the e2e stack anew, the run with its addresses
23
+ ```
24
+
25
+ - **The stack** — Compose project `<project>-<N>` by the number from the
26
+ branch, e2e — `<project>-<N>-e2e` on clean volumes, the main checkout —
27
+ `<project>`; the e2e variables `TASK_ENV_*`, heavy services in `profiles`,
28
+ `env.http` in `.agents/elves.json`, taking it down (`task-env down`;
29
+ stacks of closed tasks left on the machine — `task-env prune`) and the
30
+ limit of Docker's subnets — the `task-env` skill,
31
+ section "The task's stack".
32
+ - **The addresses** — `http://<service>.<N>.<project>.localhost`, e2e —
33
+ `http://<service>.e2e.<N>.<project>.localhost`: one proxy per machine
34
+ (Traefik) routes them to containers by their labels, nothing is published
35
+ on the host, a different host has its own cookies — the `task-env` skill,
36
+ section "The proxy". The addresses of the tasks in progress are on the
37
+ dashboard's Sessions tab (the `dashboard` skill). A database or a queue has
38
+ no address: inside the stack it is reached by the service name
39
+ (`db:5432`), from the host — by a published `127.0.0.1::5432` (`docker
40
+ compose -p <stack> port db 5432`, in e2e — `TASK_ENV_DB_5432`).
41
+ - **Other stacks aren't mine.** A stack is taken down only by `task-env down`
42
+ of my own task (`task close` of the `github` skill calls it); `docker
43
+ compose down`, `stop` or `rm` of another stack — never: the stack's name
44
+ carries the task number (`docker compose ls`). A bare `docker compose down
45
+ -v` without `-p` goes to the project Compose picks by itself — the
46
+ directory or the file's `name:` — and may wipe the main checkout's stack.
47
+ - **A busy port of the host** (the proxy's 80, an app outside the stack) —
48
+ first its owner: `lsof -nP -iTCP:<port> -sTCP:LISTEN` and the process's cwd
49
+ (`lsof -a -p <PID> -d cwd -Fn`). Someone else's — I don't kill it: `kill`
50
+ by port breaks someone else's run; the proxy takes another port
51
+ (`ELVES_PROXY_PORT`, the `task-env` skill, section "Settings").
52
+
53
+ ## Everything in the compose file
54
+
55
+ The task's environment is the project's compose file and nothing else: what
56
+ runs outside it — an app started on the host, a shared database container,
57
+ the machine's Redis — is shared by all tasks again, with its port and data.
58
+
59
+ - **Every app and everything it reaches** — databases, queues, cache, mail,
60
+ storage, emulators of external services — is a service of the compose
61
+ file. Apps reach them by the service name, as in the project
62
+ (`postgres://db:5432/app`): no host variables, no names per task.
63
+ - **HTTP services** and their ports in the container — `env.http` in
64
+ `.agents/elves.json` (`{"env": {"http": {"web": 3000}}}`): only they get an
65
+ address; in a compose file a database's port doesn't differ from an app's.
66
+ - **The code** — the worktree's: a build (`build: .`) or a bind mount by a
67
+ relative path (`./src:/app/src`); the dev server's HMR in a container works
68
+ (the `task-env` skill, section "Checked on the machine").
69
+ - **A heavy service** (ClickHouse, Elasticsearch, Kafka) — in `profiles`:
70
+ every task gets its own copy, hundreds of MB, so only the tasks that need it
71
+ start it (`up --profile <name>`).
72
+ - **The isolation rules** — no `container_name`, no fixed host port, volumes
73
+ and networks without `name:` and `external`, code by relative paths: each
74
+ of the four breaks isolation silently. The project's `check` (and the
75
+ session start hook) names each violation; after editing the compose file —
76
+ `task-env check` before the commit. The table with the fixes — the
77
+ `task-env` skill, section "Compose file isolation rules".
78
+
79
+ ## What moves with the address
80
+
81
+ The task's app lives at `http://web.<N>.<project>.localhost`, not at
82
+ `localhost:3000`: everything that builds absolute links follows the address.
83
+
84
+ - **The app's base URL** (`APP_URL`, `NEXTAUTH_URL` / `AUTH_URL`, `ORIGIN`) —
85
+ from a variable, not only from the request: `task-env up` and `e2e` give
86
+ Compose the stack's addresses in `TASK_ENV_<SERVICE>_URL`, and the compose
87
+ file passes them on with a default for CI —
88
+ `APP_URL: ${TASK_ENV_WEB_URL:-http://localhost:3000}` (the `task-env`
89
+ skill, section "The task's stack"); a public variable baked in at build
90
+ (`NEXT_PUBLIC_*_URL`, `VITE_*_URL`) — into `build.args` the same way. A
91
+ framework with the address in its settings doesn't read the request; one
92
+ that does may take it from there: the proxy passes the client's `Host` on
93
+ (Traefik's `passHostHeader` is on by default) — Auth.js behind a proxy with
94
+ `AUTH_TRUST_HOST=true` and without `AUTH_URL`. A value written in the
95
+ compose file as is — the same for every task.
96
+ - **Links in emails** — the stack's own mail catcher (Mailpit as a service of
97
+ the compose file) shows them with the task's address.
98
+ - **Callbacks of external services** — the next section.
99
+ - **Check:** no host port of the app is left —
100
+ `grep -rn "localhost:3000" *compose*.y*ml .env* playwright.config.*`.
101
+
102
+ ## Callbacks of external services
103
+
104
+ - **OAuth redirect URI** — the task's address:
105
+ `http://web.<N>.<project>.localhost/api/auth/callback/<provider>`. The
106
+ redirect goes through the user's browser, and the browser resolves
107
+ `*.localhost` itself. Providers match the URI exactly: Google doesn't take
108
+ wildcards, takes plain `http` only for localhost and wants a host on the
109
+ public suffix list — whether it takes a `*.localhost` subdomain, its docs
110
+ don't say: check in the console; GitHub matches exactly, with wildcard
111
+ matching on — by the host without subdomains and the port, so the callback
112
+ `http://<project>.localhost/…` covers every task's address. A dev OAuth app
113
+ of its own, not the production one; what a provider doesn't take — a stub in
114
+ e2e (`docs/testing.md`, "e2e").
115
+ - **Webhooks** come from the internet and don't reach `*.localhost`: the
116
+ forwarder is a service of the task's stack and delivers by the service
117
+ name. Stripe — its CLI:
118
+
119
+ ```yaml
120
+ services:
121
+ stripe:
122
+ image: stripe/stripe-cli
123
+ profiles: [stripe]
124
+ command: listen --api-key ${STRIPE_SECRET_KEY:-} --forward-to http://web:3000/api/webhooks/stripe
125
+ ```
126
+
127
+ The signing secret doesn't change between runs of `listen` (the Stripe CLI
128
+ docs): `stripe listen --print-secret` once — and it goes into the app's
129
+ `STRIPE_WEBHOOK_SECRET`. The key is a test-mode one, from the environment or
130
+ the project's `.env`, not from the compose file. Every running `listen`
131
+ receives every event of the account: one task's app also gets the other
132
+ tasks' events and answers `2xx` to an object it doesn't have rather than
133
+ failing. A tunnel (ngrok, cloudflared) is the same — one per stack, its
134
+ address in the provider's settings.
135
+
136
+ ## Runner config: the e2e stack's address
137
+
138
+ ```ts
139
+ // playwright.config.ts — `task-env e2e -- npx playwright test` gives the e2e stack's address;
140
+ // without it (CI, a run not through task-env) everything stays as before
141
+ import { defineConfig } from '@playwright/test';
142
+
143
+ const stack = process.env.TASK_ENV_WEB_URL; // http://web.e2e.42.shop.localhost
144
+ const baseURL = stack ?? 'http://localhost:3000';
145
+
146
+ export default defineConfig({
147
+ use: { baseURL },
148
+ webServer: stack ? undefined : { // the stack serves the app: the runner starts no server
149
+ command: 'npm run build && npm run start',
150
+ url: baseURL,
151
+ reuseExistingServer: false, // a busy port is an error, not someone else's server
152
+ },
153
+ });
154
+ ```
155
+
156
+ - The address is `TASK_ENV_<SERVICE>_URL` of the HTTP service: `web` →
157
+ `TASK_ENV_WEB_URL`. The stack's database for factories and seeds —
158
+ `TASK_ENV_DB_5432` (`127.0.0.1:<port>`) when its `db` publishes
159
+ `127.0.0.1::5432`: the run builds **all** its connection variables from it
160
+ (`DATABASE_URL`, `DIRECT_URL`, `PG*`) — one forgotten variable sends seeds
161
+ into another database.
162
+ - The e2e stack is created anew with clean volumes on every run and stays
163
+ until the next one: a failed run is examined on it (`docker compose -p
164
+ <project>-<N>-e2e logs`).
165
+ - Without the variable — the runner's own server on its port, as in CI.
166
+ `reuseExistingServer: !process.env.CI` from the Playwright template, outside
167
+ CI, silently picks up the neighbor's server on the same port: the tests run
168
+ against someone else's code and database. The behavior of `webServer` is
169
+ checked against the Playwright docs ("Web server", `TestConfig.webServer`):
170
+ `reuseExistingServer: false` fails if something is already listening on
171
+ `url`.
172
+
173
+ ## The sandbox
174
+
175
+ Docker doesn't work inside the Claude Code sandbox, and the addresses
176
+ `*.localhost` aren't reachable from it by default — the `task-env` skill,
177
+ sections "Docker and the sandbox" and "Checked on the machine".
178
+
179
+ - **The `task-env` script** — a separate Bash call outside the sandbox: no
180
+ `excludedCommands` entry takes out a script run by `bun`; inside the sandbox
181
+ it exits with code 2 and says so.
182
+ - **The e2e run** is the script's child and runs outside the sandbox with it.
183
+ - **Docker itself** (`docker compose -p <stack> logs`, `exec`, `ps`) — the
184
+ owner's step: `sandbox.excludedCommands` in `~/.claude/settings.json`
185
+ (`docker compose *`, `docker ps *`), a call without pipes and redirections.
186
+ Access to Docker is access to the whole machine.
187
+ - **The task's addresses from sandboxed commands** (curl, a test runner) —
188
+ with `sandbox.network.allowLocalBinding: true`, the owner's setting: it
189
+ opens the loopback only, Docker stays outside — the `task-env` skill,
190
+ section "Local ports and task addresses". Without it — only an unsandboxed
191
+ command.
192
+ - **The agent's browser pane** is outside the sandbox and opens the task's
193
+ addresses itself.
194
+
195
+ ## Shared `.git/config`
196
+
197
+ All worktrees of a repository — and the main checkout — read one
198
+ `.git/config`: each copy has its own `HEAD`, index and working files, but the
199
+ config, hooks and branches are shared. `git config user.name` or `user.email`
200
+ in any copy changes the commit author for all sessions of the repository, not
201
+ just its own. The host may not deploy a commit whose author has no access to
202
+ the project: the deploy doesn't run, and the site quietly falls behind
203
+ `main`.
204
+
205
+ - **No `git config` in copies or in subagent assignments:** not `--local`,
206
+ not `--global` (that one changes every repository on the machine), not
207
+ `--worktree`: without `extensions.worktreeConfig` it writes to the same
208
+ place, and the extension itself is turned on in the shared config.
209
+ - **A test commit with a different author** — a setting for one command:
210
+ `git -c user.name=… -c user.email=… commit`.
211
+ - **Git experiments** (hooks, settings, history) — `git clone` into a temp
212
+ folder, not in the shared repository.
213
+ - **A branch rename writes the config too** — the branch's section
214
+ (upstream): another checkout's git holds the lock, `git branch -m` fails
215
+ with `could not lock config file`, the ref renamed, the section not. The
216
+ task branch — `github task status <N> "In progress" --branch …`: it retries
217
+ on the lock.
218
+ - **After a commit** — `git log -1 --format=%ae`: the author is who it should
219
+ be. Someone else — first `git config --show-origin user.email`: where it
220
+ came from, rather than a fix by feel.
@@ -0,0 +1,117 @@
1
+ # Waiting for PR checks and merging — for "Git, PRs and merging"
2
+
3
+ The principle is in `AGENTS.md` ("Git, PRs and merging": merge only with all
4
+ checks green); the `ci-wait` skill does the waiting with a script. Here are the
5
+ waiting and merging rules in full, the app's Auto-fix, then the polling
6
+ mechanics. Rebase, the check before a push and cleaning up the merged branch —
7
+ the `github` skill, section "Git, PRs and merging — mechanics".
8
+
9
+ ## Rules
10
+
11
+ - **The full SHA:** I don't dig up the SHA of my merge — `wait-ci.sh merged
12
+ <N>` takes it from the PR itself (`git rev-parse origin/main` returns
13
+ someone else's, later commit during parallel merges); of my commit — `git
14
+ log -1 --format=%H`; I don't complete a short one or take a "similar" one
15
+ from memory: for a nonexistent SHA the API silently returns an empty status.
16
+ - **Merge only with all checks green**, including the preview deploy; "a
17
+ known red first build" is no exception: I re-run my failed run (`gh run
18
+ rerun <id> --failed`) and wait for green; red for another reason — I fix it
19
+ before merging. I wait with a polling loop over the API: a PR — by its head
20
+ SHA, a commit — by the full SHA (mechanics and traps below; the `ci-wait`
21
+ skill). Green — then also `github pr premerge <N>`: if `main` moved after
22
+ CI — fast checks on the merge with a fresh `main` (`typecheck`,
23
+ `test:spec` — the scripts CI runs; standards and architecture go there
24
+ once, in a job of their own, not inside `test` and e2e — `ci.md` of the
25
+ `spec` skill); `main` is red by itself — exit code 3, I don't merge: one bug
26
+ for it, and I wait for it to close (the `github` skill).
27
+ An empty list or `no checks reported` means "not registered yet", not
28
+ "passed"; a failed check means "red" at once, without waiting for the rest.
29
+ - **Actions jobs didn't start** with the annotation "The job was not started
30
+ because recent account payments have failed or your spending limit needs
31
+ to be increased" — this is the account owner's billing: neither a rerun nor
32
+ a code change will help. I don't re-run or fix — I tell the owner; the merge
33
+ waits.
34
+ - **A merge that deploys** (deploy from `main`) isn't the end of the task yet:
35
+ before reporting "done" I wait for the statuses on the merge SHA — the
36
+ deploy and the checks after it (the `ci-wait` skill, `merged <N>`). A
37
+ production smoke test is a check in the project's CI, not a manual agent
38
+ step. Failed — logs and a fix at once: it blocks the report and closing the
39
+ task.
40
+ - **Deploy — by the commit status, not by polling the site.** Whether the
41
+ merge rolled out is told by the host's status on the merge SHA (`ci-wait
42
+ merged <N>`), not by the page: frequent requests to the live site turn on
43
+ the host's bot protection, and from then on the site answers with a
44
+ challenge instead of the page — to the agent, or even to everyone from the
45
+ same address. Need to see the page — one request after a green status, not
46
+ a loop.
47
+
48
+ ## Auto-fix — off
49
+
50
+ Auto-fix of the Claude Code app's CI monitor wakes the session with a
51
+ `<ci-monitor-event>` on failed checks, a conflict and review comments, and on
52
+ a conflict tells it to merge the base branch in, never rebase or force-push.
53
+ The flow merges by rebase only (`github project fix` turns the other merge
54
+ methods off): a merge commit in the PR's branch brings the conflict back at
55
+ the rebase merge. In pilot #471 the event about an executor's PR came to the
56
+ dispatcher session while the executor was in the middle of a rebase in its
57
+ worktree: `main` merged in there would have broken it (#529).
58
+
59
+ - **Off for the flow's PRs:** the app binds each new PR of the session with
60
+ Auto-fix on, so right after `gh pr create` — `get_status` (the newest PR is
61
+ the bound one, earlier ones go to `otherBoundPrs`); Auto-fix on for my PR —
62
+ `set_monitor` with `auto_fix: false` and the PR's url. Nothing is lost: red checks and a conflict come from
63
+ `ci-wait` (FAIL; ERROR "rebase onto origin/main"), and its completion
64
+ wakes the session.
65
+ - **An event came anyway** (Auto-fix was on): a conflict — `git fetch
66
+ origin`, `git rebase origin/main`, `git push --force-with-lease`, then
67
+ `ci-wait` again; red checks — the root cause, as on FAIL; the base is never
68
+ merged in. Then Auto-fix off, as above.
69
+ - **A dispatcher session:** the app binds the executors' PRs to it, and their
70
+ events come to it. An executor's PR isn't the dispatcher's step: the
71
+ executor drives it, the dispatcher doesn't touch its branch or worktree; an
72
+ executor that stopped (a limit, a dropped session) is continued with
73
+ `SendMessage`.
74
+
75
+ ## Polling mechanics
76
+
77
+ - Decide only by `bucket` (`pending`, `fail`, `cancel`, `pass`, `skipping`).
78
+ The list is empty or has `pending` → wait. Merge — when the list is
79
+ non-empty, there is no `pending` and the snapshot repeated two polls in a
80
+ row (checks register with a delay). `fail` or `cancel` — the outcome is
81
+ known at once: a long check alongside (a preview deploy) won't change it.
82
+ - By a commit's full SHA (deploy, an external check, a run on `main`) checks
83
+ live in two places: statuses of hosts and external services —
84
+ `gh api repos/{owner}/{repo}/commits/<sha>/status` (the `statuses` list by
85
+ context), GitHub Actions — `commits/<sha>/check-runs` (`status`,
86
+ `conclusion`). Wait for both: on a commit where all CI is Actions there are
87
+ no statuses at all. `gh pr checks` doesn't accept a SHA.
88
+ - By PR number — on every poll `gh api repos/{owner}/{repo}/pulls/<N>`
89
+ (`head.sha`, `mergeable`, `base.ref`), checks — by the head SHA, as for a
90
+ commit: a new push changes the head. `gh pr checks` goes to GraphQL, whose
91
+ quota (5000 points per hour) is shared by all the user's sessions — with
92
+ parallel sessions it runs out before REST does.
93
+ - `mergeable: false` — a conflict with the base: there is no merge ref,
94
+ workflows on `pull_request` don't start, and a non-empty snapshot of host
95
+ statuses alone gives a false outcome. Nothing to wait for — rebase. `null` —
96
+ GitHub is still computing: wait. Right after a push the API still returns
97
+ the previous head's `false`: a conflict — only when the next poll of the
98
+ same head repeats it.
99
+ - The base branch's required checks — `rules/branches/<base>` (ruleset, rule
100
+ `required_status_checks`) and `branches/<base>`
101
+ (`protection.required_status_checks`, if `enforcement_level` isn't `off`):
102
+ while any of them is missing from the snapshot — wait.
103
+ - Actions runs on a SHA — `actions/runs?head_sha=<sha>`: a run that failed
104
+ without a single job gives no check run — the workflow wasn't parsed (the
105
+ run's name is the file path, event `push`) or didn't start. That is "red",
106
+ not an empty spot: without it a snapshot of host statuses alone gives a
107
+ false "green". A workflow whose `on:` fires on the PR at the head SHA but
108
+ which produced no run — wait.
109
+ - A response with `rate limit` isn't an error: wait until `reset` from `gh api
110
+ rate_limit` (that request doesn't spend quota), a secondary limit — a
111
+ minute.
112
+ - Right after a push `gh pr checks` answers `no checks reported` with exit 1 —
113
+ that means "not registered yet", not "failed" and not "passed". Exit codes
114
+ 0 / 1 (failed) / 8 (`pending`) are set only **without** `--json`; with
115
+ `--json` it exits 0 on both `pending` and `fail`.
116
+ - Without branch protection `gh pr merge --auto` merges without waiting for
117
+ CI.
@@ -0,0 +1,304 @@
1
+ # How to test: runs, environment, e2e, UI in a browser — for "Tests and code"
2
+
3
+ The rule — which test is needed and how it can fail — is `AGENTS.md`,
4
+ "Specification — decisions" (in full — `skills/spec/canon.md`), and
5
+ `AGENTS.md`, "Tests and code"; the task's Compose stack, its addresses and
6
+ the runner config — `docs/parallel-checkouts.md`.
7
+ Here — how to run tests, read the result, write e2e tests that don't flake,
8
+ and check UI in the agent's browser. The rules hold for any stack, the
9
+ examples are Playwright. Three things depend on the framework: which env files
10
+ each command reads, what is baked into the build, and how to serve a
11
+ production build locally — they are in "Framework: three questions". Checked
12
+ against the Playwright, Next.js, Vite, SvelteKit and Nuxt documentation.
13
+
14
+ ## Run and result
15
+
16
+ - **A heavy run goes through the machine queue.** The flow install wraps the
17
+ project's checks (`lint`, `typecheck`, `test`, `test:*`) in `slot`: two runs
18
+ go at once, together on half the cores; the rest wait in order of arrival
19
+ and print whom they wait for (the `slot` skill). Without the queue,
20
+ neighboring sessions, the linter and another worktree's e2e share the cores —
21
+ timeouts and flakes that aren't about the code. Runner workers come from
22
+ `ELVES_SLOT_CPUS` (a quarter of the cores); CI has its own:
23
+
24
+ ```ts
25
+ maxWorkers: Number(process.env.ELVES_SLOT_CPUS) || undefined, // vitest.config.ts
26
+ workers: Number(process.env.ELVES_SLOT_CPUS) || (process.env.CI ? 2 : undefined), // playwright.config.ts
27
+ ```
28
+
29
+ A monorepo on turbo 2 — `"globalPassThroughEnv": ["ELVES_SLOT",
30
+ "ELVES_SLOT_CPUS"]` in `turbo.json`: otherwise strict env mode won't pass
31
+ them to tasks, and `env`/`globalEnv` would add them to the cache hash.
32
+
33
+ Compare time by CI or by CPU (`user` from `time`), not by wall clock: that
34
+ includes the queue wait. Check a flake by repeating it (`--repeat-each`) —
35
+ via a project script, in a slot, otherwise the repeat measures the
36
+ neighbors.
37
+ - **A long run — under watch.** A full run, all the more with a queue, does
38
+ not go right up against the agent tool's timeout: on timeout the process is
39
+ killed mid-tests (`Killed: 9`), and the wrapper (`| tail`, `&& echo ok`, a
40
+ script without `set -o pipefail`) may return 0. Run it as a background
41
+ command with a generous timeout (in Claude Code — up to 2 h, completion
42
+ wakes the session) or in a terminal; wait for the process to finish, not
43
+ for a deadline.
44
+ - **Before a commit — what the change affects; the full run is CI's.** A full
45
+ local run repeats what CI does anyway — in parallel jobs and shards, faster
46
+ and reproducibly; locally the agent waits twice — the queue and the run,
47
+ then CI — and the run holds the shared machine. An error before the push
48
+ is almost always caught by the tests the change itself affects. Before a
49
+ commit:
50
+ - `lint` and `typecheck` — in full: they are fast and incremental;
51
+ - `test:spec` — in full: standards read the code from disk and the
52
+ architecture model takes the whole tree, the import graph never selects
53
+ them;
54
+ - `test` — the affected: the runner's `--changed=origin/main` (Vitest,
55
+ `bun test`; Jest — `--changedSince=origin/main`) through the project's
56
+ script (`npm run test -- --changed=origin/main`) runs the test files that
57
+ import a file changed against `origin/main`, committed or not. Nothing
58
+ affected isn't an error: Vitest needs `--passWithNoTests` for it
59
+ (otherwise exit code 1), `bun test` passes by itself;
60
+ - tests that reach the code not by an import — a script started as a
61
+ process, a file read from disk, an app behind its server, a dynamic
62
+ `import()` — the graph doesn't see: a separate run by path — the tests
63
+ that name the changed file (`grep -rl <file> tests`) or its decision's
64
+ folder; a path together with `--changed` is their intersection, not a
65
+ union;
66
+ - `test` in full — when the change touches what every test depends on and
67
+ no import shows: the runner config, its global setup (`setupFiles`,
68
+ `globalSetup`, `preload` in `bunfig.toml`), the lock file;
69
+ - e2e — the changed specs (Playwright `--only-changed=origin/main
70
+ --pass-with-no-tests`): it follows the specs' own imports (fixtures, page
71
+ objects), not the app code the browser opens; the rest — CI;
72
+ - `test:slow` — when the change touches what it checks (below).
73
+
74
+ The full run is CI's; `github pr premerge` stays as it is: `typecheck` and
75
+ `test:spec` on the merge with a fresh `main` (`docs/pr-checks.md`).
76
+ - **A slow level — a script of its own.** Tests that need Docker or a live
77
+ server go into a separate script (`test:slow`, by file name — the runner
78
+ selects files, not test blocks), so `test` stays fast before every commit.
79
+ CI runs every level; locally a slow level runs when the change touches what
80
+ it checks.
81
+ - **The result — by the runner's summary, not the exit code.** Playwright's
82
+ summary: `passed`, `failed`, `flaky`, `skipped`, `did not run`,
83
+ `interrupted`; Vitest — the `Test Files` and `Tests` lines; `bun test` —
84
+ `pass` and `fail`. A run is good when the summary is there, there are no
85
+ failed, not-run or interrupted tests, and flakes are investigated
86
+ (`AGENTS.md`, "Tests and code": a flake is a defect). No summary — the run didn't finish,
87
+ whatever the exit code. Filtering the output by `Error:` lies: the app
88
+ server writes harmless errors (an expected 404, a cancelled request) to the
89
+ same stream.
90
+ - **A failed test's artifacts — set aside at once.** The runner clears the
91
+ results folder at the start of a run (Playwright — `outputDir`, by default
92
+ `test-results/`): copy the trace, screenshots and video before rerunning.
93
+
94
+ ```bash
95
+ cp -R test-results "$TMPDIR/e2e-<task>-$(date +%H%M%S)"
96
+ npx playwright show-trace "$TMPDIR"/e2e-<task>-*/<test>/trace.zip
97
+ ```
98
+ - **A build and a run against it — not in parallel.** The build directory
99
+ (section "Framework: three questions") is shared: rebuilding under a running
100
+ server swaps the chunks, and open pages fail loading them. Build first, then
101
+ run; don't build in another terminal of the same checkout.
102
+
103
+ ## Environment
104
+
105
+ - **Each command reads its own env files, and the production server
106
+ sometimes none.** The dev server, the build and the production server of
107
+ one app take their environment from different places (table below). An env
108
+ pulled from the host (`vercel env pull`, `netlify env:import` and the like)
109
+ may carry production secrets and production addresses: a local production
110
+ build will take it and go to the production database.
111
+ - **Exports win over files.** `.env*` doesn't override a variable already set
112
+ in the environment (Next.js, Vite and everything built on it, dotenv
113
+ without `override`), and a production server that reads no files sees only
114
+ the exports. Local build, production server and e2e — only with an
115
+ explicitly exported local environment:
116
+
117
+ ```bash
118
+ set -a; . ./.env.e2e; set +a # own file, local addresses only
119
+ npm run build && npm run test:e2e
120
+ ```
121
+ - **What is baked into the build can't be changed by an export.** Public
122
+ variables (and in some frameworks private ones — see the table) are
123
+ substituted into the code at build time: the build for e2e uses the same
124
+ local environment as the run; changed an address or a port — rebuild.
125
+ - **Where the env points — check with a command, don't rely on memory:**
126
+
127
+ ```bash
128
+ grep -H -E '^[A-Z_]*_URL=' .env* | sed -E 's#(://[^:]+:)[^@]+@#\1***@#'
129
+ ```
130
+
131
+ A guard "a laptop doesn't connect to a remote database without an explicit
132
+ flag" is a project standard (`tests/standards/<name>`), not attentiveness.
133
+ - **A fresh database per run.** CI starts from an empty database and catches
134
+ fixture races; a populated local database hides them — or the other way
135
+ round, its old rows show up in others' lists. The task's e2e stack starts
136
+ on clean volumes every run (`task-env e2e`, `docs/parallel-checkouts.md`).
137
+ A test that is green on CI's empty database and red on a populated local
138
+ one is a defect of the test (it depends on others' data), not of the
139
+ database.
140
+
141
+ ## e2e
142
+
143
+ - **Against a production build, not the dev server.** In the task's e2e
144
+ stack the app is a service of the compose file, and for e2e it serves the
145
+ production build (table below) — a profile or a build target by a variable
146
+ set on the `task-env e2e` command: the script passes its environment to
147
+ Compose. Without the stack (CI) — `webServer.command` is the build and the
148
+ production server, not `dev`. The dev server changes code under a run:
149
+ Next.js compiles a route on its first request, Vite (and SvelteKit, Nuxt,
150
+ React Router built on it) re-bundles dependencies and reloads the page when
151
+ it meets a new dependency mid-run — timeouts and warnings that production
152
+ doesn't have.
153
+ - **External services — stubs or emulators** via environment flags of the
154
+ e2e server (the service's `environment` in the stack, otherwise
155
+ `webServer.env`): email, payments, storage, LLM. A side effect ("email
156
+ sent", "payment created") — by polling your own database or the emulator,
157
+ not via a UI that itself depends on the external service.
158
+ - **Shared data — a setup project, once per run.** What all parallel files
159
+ see (reference data, catalogs, users to log in with) is created by the
160
+ runner's setup project, not by a file's `beforeAll`: otherwise two files at
161
+ once see "missing — creating" (TOCTOU) and fail on a unique key or share one
162
+ row.
163
+
164
+ ```ts
165
+ // playwright.config.ts
166
+ projects: [
167
+ { name: 'setup', testMatch: /global\.setup\.ts/ },
168
+ { name: 'e2e', dependencies: ['setup'], use: { storageState: 'tests/.auth/user.json' } },
169
+ ],
170
+ ```
171
+ - **Your own data — under a unique prefix.** Test data gets a prefix from the
172
+ worker, the time and a random suffix
173
+ (`` `e2e-${testInfo.workerIndex}-${Date.now()}-${rnd}` ``); creating and
174
+ cleaning up — directly in the database (factories in `tests/lib`), not via
175
+ the UI: UI setup steps are slow and fail in the wrong place.
176
+ - **A fixture doesn't surface in others' lists.** Its creation date is in the
177
+ past (lists sort by it, a new row would push the expected one out of first
178
+ place); a test looks for a known entity by name, not for the table's first
179
+ row.
180
+ - **Wait for the response — before the action and by its own URL.**
181
+
182
+ ```ts
183
+ const saved = page.waitForResponse((r) => r.url().includes(`/api/orders/${id}`) && r.request().method() === 'PATCH');
184
+ await page.getByRole('button', { name: 'Save' }).click();
185
+ expect((await saved).ok()).toBe(true);
186
+ ```
187
+
188
+ A wait registered after the click misses a fast response; without a URL
189
+ filter it catches someone else's request (analytics, prefetch).
190
+ - **The first action after an SSR navigation may be lost** before hydration
191
+ (Next.js, Nuxt, SvelteKit, React Router, Astro islands): the button is
192
+ already in the HTML, the handler isn't there yet. Put the action together
193
+ with the check of its result into a retried block, not a pause:
194
+
195
+ ```ts
196
+ await expect(async () => {
197
+ await page.getByRole('button', { name: 'Filter' }).click();
198
+ await expect(page.getByRole('dialog')).toBeVisible({ timeout: 1_000 });
199
+ }).toPass({ timeout: 10_000 });
200
+ ```
201
+
202
+ `toPass` has a default timeout of 0 — it retries until the test timeout:
203
+ your own `timeout` is required. Only an idempotent action (open, select)
204
+ may be retried, not "create". A SPA without SSR doesn't have this race: the
205
+ button doesn't exist before render, and the runner waits for it. Navigate
206
+ straight to the canonical URL: a path redirect means an extra hydration and
207
+ a race. Assertions are web-first (`toHaveText`, `toBeVisible`), not
208
+ `textContent()` and a snapshot comparison.
209
+ - **Log in with a saved session.** The setup project logs in once and writes
210
+ `storageState`; tests start with it instead of filling in the login form. A
211
+ visual check by the agent (the session's browser) uses the same
212
+ `storageState`, without typing a password; page state — from the DOM
213
+ (section "UI in the agent's browser"). The login form is checked by one test
214
+ of the login itself.
215
+ - **A guard for a framework's dev warning** (React, Vue: hydration, list
216
+ keys) — a unit test in dev mode (jsdom, happy-dom): the production build
217
+ strips the warnings, so in e2e against it the guard can't fail (`AGENTS.md`,
218
+ "Tests and code").
219
+
220
+ ## UI in the agent's browser
221
+
222
+ The browser of the agent's session (Claude Code's built-in browser, Chrome
223
+ with the extension) is for looking at a page through a human's eyes: layout,
224
+ look, the path through screens.
225
+
226
+ - **State — from the DOM, a screenshot — only for the look.** Whether a popup
227
+ (popover, menu, calendar, dialog) is open and what a click did — read it
228
+ from the page: the accessibility tree (`read_page`, `find`) — is there an
229
+ element with role `dialog`, `menu`, `listbox`; JS on the page
230
+ (`javascript_tool`) — the state attribute and the animations, before the
231
+ action and after it:
232
+
233
+ ```js
234
+ ((el) => el && {
235
+ state: el.dataset.state ?? (el.hasAttribute('data-closed') ? 'closed' : 'open'),
236
+ anims: el.getAnimations({ subtree: true }).map((a) => a.playState),
237
+ })(document.querySelector('[role="dialog"], [role="menu"], [role="listbox"]'));
238
+ ```
239
+
240
+ The attribute depends on the library: `data-state="open|closed"` (Radix,
241
+ shadcn/ui), `data-open`/`data-closed` (Base UI, Headless UI); on the trigger
242
+ button — `aria-expanded`. No element — the popup is closed and unmounted.
243
+ - **A screenshot lags behind the state.** A hidden browser pane or a
244
+ background tab is throttled: CSS animations and `requestAnimationFrame`
245
+ almost stand still. A popup that is closed by state (`closed`, exit
246
+ animation `running`) stays mounted until the animation ends and looks open
247
+ in a screenshot for seconds more; a screenshot may also show a frame from
248
+ before the last click.
249
+ - **A "doesn't close", "doesn't open", "the click didn't work" bug — only
250
+ after checking the DOM.** `closed` with an exit animation running is not a
251
+ bug but throttling. A bug is when the state didn't change after the action,
252
+ or the animations have ended and the closed element is still visible. A
253
+ screenshot of the result — with the pane visible and after the animations
254
+ end, not after a pause.
255
+
256
+ ## Unit tests
257
+
258
+ - **A mock goes at the external edge and behaves like the real thing.** A
259
+ mock of your own module hides its behavior (the framework's request queue,
260
+ form parsing, a transaction) and is forbidden by lint ("Specification —
261
+ decisions", `skills/spec/canon.md`). What is replaced is the boundary with
262
+ the outside world — a third-party API's HTTP, email, the clock — and the
263
+ way it really responds (codes, delays, error format per its docs).
264
+ - **An entry point — through the same input as in production.** A form
265
+ handler gets a `Request` or `FormData` (a Next.js server action, a React
266
+ Router `action`, a SvelteKit form action), a route gets an HTTP request,
267
+ not a ready-made object passed straight into the validation schema:
268
+ otherwise the test doesn't see form parsing, type coercion and missing
269
+ fields.
270
+
271
+ ## Framework: three questions
272
+
273
+ Before a project's first e2e run — answer from its framework's documentation
274
+ and write the answer into the project's `docs/` (an environment recipe):
275
+
276
+ 1. Which env files the dev server, the build and the production server read,
277
+ and whether an already exported environment wins over them.
278
+ 2. What is baked into the build and doesn't change without a rebuild.
279
+ 3. How to serve a production build locally and where its directory is.
280
+
281
+ | Framework | Who reads `.env*` | Baked in at build | Production build locally | Directory |
282
+ |---|---|---|---|---|
283
+ | Next.js | `dev` — `.env.development*`, `build` and `start` — `.env.production*`, `.env.local` and `.env` — both; under `NODE_ENV=test` `.env.local` isn't read | `NEXT_PUBLIC_*` | `next build && next start` | `.next` |
284
+ | Vite (SPA) | `dev` — `.env.development*`, `build` — `.env.production*` (`--mode` changes it), `.env[.local]` — both; production is static files, no environment | `VITE_*` (`import.meta.env`) | `vite build && vite preview` — for checking only, not a production server | `dist` |
285
+ | SvelteKit (adapter-node) | `dev` and build — as in Vite; `node build` doesn't read `.env` — `node --env-file=<file> build` | all of `$env/static/*`, private too; `$env/dynamic/*` — at runtime | `vite build && node --env-file=.env.e2e build` | `.svelte-kit`, `build` |
286
+ | Nuxt | `dev`, `build`, `generate`, `preview` — `.env` (another file — `--dotenv`); `node .output/server/index.mjs` — reads none | `runtimeConfig` set via `process.env.<another name>`; at runtime only `NUXT_*` overrides it | `nuxt build && nuxt preview` (reads `.env`) | `.nuxt`, `.output` |
287
+
288
+ A framework not in the table (React Router, Astro, a backend without a
289
+ frontend) — the same three questions to its documentation; the answer goes
290
+ into the project's `docs/`, not into memory.
291
+
292
+ ## Test knowledge in the project
293
+
294
+ - The project's `docs/` — only recipes for its own environment: which env
295
+ files, containers, emulators, hosting, how to bring them up, the answers to
296
+ the three questions above.
297
+ - A checkable rule about tests ("e2e tests don't mock their own API",
298
+ "fixtures get a prefix") is a standard `tests/standards/<name>`, not a
299
+ paragraph in a document.
300
+ - Knowledge about a helper (what a factory creates, what it cleans up, what
301
+ permissions a user has) — in its JSDoc in `tests/lib`: the editor's hint
302
+ shows it, a document doesn't.
303
+ - A lesson repeated in two projects is a task in Elves for this reference
304
+ (`AGENTS.md`, "This file and skills"), not the agent's memory.