@catladder/cli 5.5.0 → 5.7.0

package/dist/skills/catladder-builds/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: catladder-builds
-description: Configuring how a component is built in a catladder project — the `build` config in catladder.ts (build types node / rails / meteor / custom, build & start commands, artifacts, caching, lint/test/audit jobs, the Docker image, project-declared job images, and shared workspace builds). Use when adding or changing a component's build, choosing a build type, wiring a Dockerfile/nginx image, declaring a custom CI job image (`images` config), tuning the build/test/lint jobs, or setting up a monorepo workspace build. Triggers on "build config", "buildCommand", "build type", "Dockerfile", "docker image", "job image", "jobImage", "custom image", "project image", "rails build", "monorepo build", "artifacts".
+description: Configuring how a component is built in a catladder project — the `build` config in catladder.ts (build types node / rails / meteor / custom, build & start commands, artifacts, caching, lint/test/audit jobs, the Docker image, project-declared job images, and shared workspace builds). Use when adding or changing a component's build, choosing a build type, wiring a Dockerfile/nginx image, declaring a custom CI job image (`images` config), tuning the build/test/lint jobs, adding e2e tests against the build (`postBuildTests`), or setting up a monorepo workspace build. Triggers on "build config", "postBuildTests", "post-build test", "e2e", "playwright", "buildCommand", "build type", "Dockerfile", "docker image", "job image", "jobImage", "custom image", "project image", "rails build", "monorepo build", "artifacts".
 ---
 
 # Component builds with catladder
@@ -66,6 +66,10 @@ type: `build: { from: "web" }` (see workspace builds below).
   won't run).
 - `lint` / `test` / `audit` — customize (`{ command, jobImage, … }`) or
   set to `false` to disable that job.
+- `postBuildTests` — named tests against the **build output** (e.g. an
+  e2e suite against the production build with a Postgres service). Each
+  entry becomes a job in the `post-build` stage between build and deploy
+  and blocks the deploy when it fails (see below).
 - `artifactsPaths` / `artifactsExcludePaths` — extra build artifacts
   (`dist` and `.next` are always included).
 - `cache` — build caching (see the `catladder-pipelines` skill for the
@@ -82,11 +86,55 @@ type: `build: { from: "web" }` (see workspace builds below).
   (`{ type: "nginx" | "node" | "meteor" }`) or
   `{ type: "custom" }` (expects a `Dockerfile`).
 
+## Post-build tests (e2e against the build)
+
+`lint`/`test` run against the sources, in parallel with the build. Tests
+that need the built app go into `postBuildTests`:
+
+```ts
+build: {
+  type: "node",
+  postBuildTests: {
+    e2e: {
+      command: "pnpm test:e2e",                  // runs in the build dir
+      jobImage: "mcr.microsoft.com/playwright:v1.56.0-noble",
+      services: [{ name: "postgres:17", alias: "postgres",
+                   variables: { POSTGRES_PASSWORD: "postgres" } }],
+      vars: { DATABASE_URL: "postgres://postgres:postgres@postgres:5432/postgres" },
+      artifacts: { paths: ["apps/web/playwright-report"] }, // repo-root relative
+    },
+  },
+}
+```
+
+- The job (`🔬 e2e`, stage `post-build <env>`) downloads the build
+  artifacts (`.next`, `dist`, `artifactsPaths`), installs dependencies
+  for node builds, then runs `command`. The command starts the app
+  itself, e.g. playwright `webServer: { command: "pnpm start" }`.
+  Builds without artifacts (`rails`, `buildCommand: false`) just run
+  it after the build.
+- It is a quality gate: the env's deploy waits for it; on github it is
+  part of the `catladder ✅` required check. `allowFailure: true` makes
+  it report-only.
+- It gets the build vars plus `vars`, **not** the deployed env's
+  runtime vars/secrets — use services instead of the real database.
+- Artifacts are uploaded also when the job fails.
+- Not run in tagged releases. Disable one per env:
+  `env: { dev: { build: { postBuildTests: { e2e: false } } } }`.
+- Names must not clash with other jobs of the component (e.g. `test`,
+  `lint`) — generation fails with a clear error.
+- Workspace builds: on `builds.<ws>` they block the deploys of all
+  components in the workspace; on `build: { from: "<ws>", postBuildTests }`
+  only that component's deploy.
+- To test the **deployed** env instead (smoke tests on dev / review
+  apps), use the component's `verify` (post-deploy, does not block the
+  deploy). The same suite can serve both, with `BASE_URL` as the switch.
+
 ## Project images (custom CI job images)
 
 When a job needs a toolchain catladder doesn't ship (Java, Playwright,
 …), declare a Docker image at the top level under `images` and reference
-it in any `jobImage` field (build, `test.jobImage`, custom/pages deploy,
+it in any `jobImage` field (build, `test.jobImage`, `postBuildTests`, custom/pages deploy,
 verify):
 
 ```ts

package/dist/skills/catladder-builds/references/build-types.md

@@ -12,6 +12,7 @@ generated pipeline files.
 | `startCommand` | `string` | runtime start command |
 | `postInstall` | `string \| string[]` | runs after the package-manager install (node family) |
 | `lint` / `test` / `audit` | `false \| TestJobCustom` | customize or disable the job |
+| `postBuildTests` | `Record<string, PostBuildTestConfig \| false>` | named tests against the build output in the `post-build` stage; block the deploy |
 | `artifactsPaths` | `string[]` | extra artifacts (`dist`, `.next` always included) |
 | `artifactsExcludePaths` | `string[]` | artifact excludes |
 | `artifactsReports` | `{ junit?: string[] }` | CI test report paths |
@@ -27,6 +28,13 @@ generated pipeline files.
 `artifactsReports`, `artifacts`, `runnerVariables`, `allowFailure`
 (`allowFailure` defaults to `true` for `audit`).
 
+`PostBuildTestConfig` (for `postBuildTests` entries): `TestJobCustom` with
+a required `command`, plus `services?: Services` (gitlab shape, reachable
+by alias) and `vars?: Record<string,string>`. The job gets the build
+artifacts, the build vars and `vars` (not the deployed env's runtime
+vars), installs dependencies for node builds, and uploads `artifacts`
+also on failure.
+
 `audit` additionally takes `level`: `"info" | "low" | "moderate" | "high" |
 "critical"` (default `"critical"`) — the lowest severity that fails the job,
 translated to the package manager's own flag (`pnpm audit --audit-level`,
@@ -81,5 +89,5 @@ defaulted `buildCommand: "yarn build-storybook --quiet -o ./dist"`.
 Top-level `builds: Record<string, WorkspaceBuildConfig>`. Only
 `type: "node"` is supported.
 
-- Node workspace build: `dir?`, `buildCommand?` (default `<pm> build`), `dockerDefaults?: { yarnRebuildEnabled }`, plus shared `lint?` (default `{ command: "<pm> lint" }`), `test?` (default `{ command: "<pm> test" }`), `audit?`, `runnerVariables?`, `artifactsReports?`, `jobImage?`, `jobTags?`, `cache?` (`<pm>` = `yarn`/`pnpm` per detection).
-- A component references it via `build: { from: "<name>", docker?, startCommand?, artifactsPaths?, artifactsExcludePaths?, cache? }` — the per-component overrides layer on top of the shared build.
+- Node workspace build: `dir?`, `buildCommand?` (default `<pm> build`), `dockerDefaults?: { yarnRebuildEnabled }`, plus shared `lint?` (default `{ command: "<pm> lint" }`), `test?` (default `{ command: "<pm> test" }`), `audit?`, `postBuildTests?` (block the deploys of all its components), `runnerVariables?`, `artifactsReports?`, `jobImage?`, `jobTags?`, `cache?` (`<pm>` = `yarn`/`pnpm` per detection).
+- A component references it via `build: { from: "<name>", docker?, startCommand?, artifactsPaths?, artifactsExcludePaths?, cache?, postBuildTests? }` — the per-component overrides layer on top of the shared build.

package/dist/skills/catladder-cli/references/commands.md

@@ -86,6 +86,19 @@ pin the review apps of a merge request (disable their auto-stop) by adding the p
 - `mr` (positional): merge request IID (defaults to the open MR of the current branch)
 - `--pipeline`: trigger a merge-request pipeline so the pin takes effect immediately
 
+## `catladder mr review-app-off [mr]`
+
+stop deploying the review apps of a merge request: switches the reviewApps label off (opt-in: removes it, opt-out: adds the skip label). Running review apps stay until the MR closes or they auto-stop
+
+- `mr` (positional): merge request IID (defaults to the open MR of the current branch)
+
+## `catladder mr review-app-on [mr]`
+
+deploy the review apps of a merge request: switches the reviewApps label on (opt-in: adds it, opt-out: removes the skip label) and triggers a pipeline, since gitlab doesn't start one for label changes
+
+- `mr` (positional): merge request IID (defaults to the open MR of the current branch)
+- `--pipeline`: trigger a merge-request pipeline so the review apps deploy now
+
 ## `catladder mr unpin [mr]`
 
 unpin the review apps of a merge request — the auto-stop timer re-arms with the next deploy

package/dist/skills/catladder-config/SKILL.md

@@ -104,6 +104,10 @@ Key concepts:
     affects both). The object form separates the axes:
     `inherit: { config: "dev" }` inherits config only,
     `inherit: { config: "dev", secrets: "dev" }` both.
+- **Review apps**: top-level `reviewApps: { deploy, label, skipLabel,
+  drafts }` decides whether merge/pull requests deploy their review
+  apps — every MR/PR (default), opt-in or opt-out by label, or on
+  request; and what drafts run. See the `catladder-pipelines` skill.
 - **Pipelines**: `pipelines: { gitlab: true, github: true }` selects
   which CI systems get generated files. Options objects instead of
   `true` allow per-pipeline settings (e.g. `runnerVariables`,

package/dist/skills/catladder-deploys/SKILL.md

@@ -46,6 +46,12 @@ is `auto`.** Set per component or per environment (`env.<name>.deploy`).
 `waitFor: ["otherComponent"]` (experimental) makes a deploy wait for
 another component to deploy first.
 
+Whether merge/pull requests deploy review apps at all (opt-in/opt-out
+by label, drafts, deploy on request for all components at once) is the
+project-wide top-level `reviewApps` config — see the
+`catladder-pipelines` skill. With it set, per-component `when` of the
+review env is ignored.
+
 ## Environment lifetimes
 
 Review apps auto-stop after 1 week, dev environments after 4 weeks

package/dist/skills/catladder-pipelines/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: catladder-pipelines
-description: Understanding and debugging the CI/CD pipelines catladder generates (GitLab CI and GitHub Actions) — pipeline triggers, stages, environments, caching, job images, releases, and infrastructure drift. Use when investigating failing CI jobs, when asked why/when a job runs, when changing caching or job behavior, when deployments misbehave, or when a review app stopped/expired or should be kept alive (pinning). Triggers on "pipeline", "CI job", "gitlab-ci", "workflow", "deploy failed", "cache", "release", "auto-stop", "review app stopped", "pin review app", "keep review app".
+description: Understanding and debugging the CI/CD pipelines catladder generates (GitLab CI and GitHub Actions) — pipeline triggers, stages, environments, caching, job images, releases, and infrastructure drift. Use when investigating failing CI jobs, when asked why/when a job runs, when changing caching or job behavior, when deployments misbehave, when a review app should (not) be deployed for a merge/pull request (opt-in/opt-out labels, drafts, deploy on request), or when a review app stopped/expired or should be kept alive (pinning). Triggers on "pipeline", "CI job", "gitlab-ci", "workflow", "deploy failed", "cache", "release", "auto-stop", "review app stopped", "pin review app", "keep review app", "deploy review app", "review app label", "skip review app", "draft PR", "reviewApps".
 ---
 
 # How catladder pipelines work
@@ -46,8 +46,9 @@ review app`); the same workflow can be dispatched with a PR number to
   tear one down while the pull request is still open (gitlab runs these as manual jobs inside the pipeline
   instead).
 
-Stages: setup → test → build → deploy → verify (post-deploy checks),
-plus stop jobs for review-app teardown.
+Stages: setup → test → build → post-build (`postBuildTests` against
+the build output, block the deploy) → deploy → verify (post-deploy
+checks), plus stop jobs for review-app teardown.
 
 ## Superseded pipelines are cancelled
 
@@ -81,6 +82,84 @@ started keeps _itself_ alive — under gitlab's own default
 (`conservative`, which catladder overrides) it would shield the whole
 pipeline from cancellation.
 
+## When review apps deploy (`reviewApps`)
+
+By default every merge/pull request deploys its review apps. Projects
+with many MRs/PRs (e.g. opened by agents) can put the review delivery
+chains — docker image → deploy → verify, for all components — behind
+ONE switch per MR/PR with the top-level `reviewApps` config. Tests,
+lint and audit always run; only the delivery waits.
+
+```ts
+reviewApps: {
+  deploy: "optIn",   // auto (default) | manual | optIn | optOut
+  // label: "catladder:review-app",       // optIn label (default)
+  // skipLabel: "catladder:no-review-app", // optOut label (default)
+  drafts: "checksOnly", // likeReady (default) | checksOnly (no review apps) | skip (no pipeline)
+},
+```
+
+- `optIn`: deploys only while the MR/PR carries `label`.
+- `optOut`: deploys unless it carries `skipLabel`.
+- `manual`: never on its own — only on request.
+- `drafts: "checksOnly"`: drafts run tests, lint and audit only, the review apps deploy once the
+  MR/PR is marked ready; `"skip"`: drafts run nothing.
+- Per-component `deploy.when` of the review env is ignored then.
+
+**GitHub** — like gitlab, the review workflow decides at its start
+from the PR's labels and draft state: when the policy says deploy, the
+chains run in the same run (one app build). Whatever changes later goes
+through the `▶️ catladder deploy review` workflow
+(`catladder-deploy-review.yml`, which rebuilds the app):
+
+- when the review workflow is green but didn't deploy, its last job
+  (`▶️ trigger review deploy`) reads the PR's labels/draft state again
+  and dispatches the deploy if the policy now says so — a label set
+  while CI ran counts;
+- switching the label on (opt-in: add `label`, opt-out: remove
+  `skipLabel`) deploys right away if the PR's `catladder ✅` is green;
+  switching it off stops the review apps (`🛑 catladder stop review
+  app`), and so does closing the PR — the stop first waits for a
+  review run of the PR that may still be deploying (on close it
+  cancels it), so a late deploy can't outlive the teardown;
+- on request, any time (agents included):
+  `gh workflow run catladder-deploy-review.yml --ref <pr-branch> -f pr=<number>`
+  — explicit requests always deploy, but need the PR open and its CI
+  green (the run reuses the green CI instead of re-running tests);
+- merge gates wait for **`catladder review app ✅`** (stable name,
+  independent of the components), the last job of whichever run
+  deploys. It also sets the commit status `catladder review app` on
+  the PR head (`success`/`failure`; the deploy workflow marks it
+  `pending` first) — dispatched runs are attached to the commit but not
+  listed on the pull request, the commit status is. A run that doesn't
+  deploy, or is cancelled (superseded, stopped), reports nothing — a
+  gate must treat a missing or `pending` status on the current head as
+  "wait". In the review run a skipped chain doesn't
+  fail `catladder ✅`, a failed one does;
+- `gh pr edit <number> --add-label catladder:review-app` is the
+  agent-friendly opt-in. Labels set with the workflow's own
+  `GITHUB_TOKEN` (from inside a workflow) trigger nothing — use a user
+  or app token;
+- the dispatch only works once the workflow exists on the default
+  branch (github rule) — the PR introducing `reviewApps` can't dispatch
+  yet.
+
+A single component with `env.review.deploy.when: "manual"` (and no
+`reviewApps`) uses the same workflow for just that component, without
+the automatic trigger.
+
+**GitLab** — a `🚀 deploy review` job in the MR pipeline is the
+switch: the docker and deploy jobs of all components `need` it. The
+policy decides whether it starts on its own (label set / opt-out not
+set) or waits as a manual job; playing it any time deploys (early
+clicks wait for builds and tests). It has `allow_failure: true`, so an
+unplayed switch leaves the pipeline green. Label changes don't start
+a pipeline on gitlab — use `yarn catladder mr review-app-on` (sets the
+label and triggers a pipeline) / `mr review-app-off`. Drafts are
+detected by their title prefix (`Draft:`, `[Draft]`, `(Draft)`); with
+`drafts: "skip"` marking an MR ready starts no pipeline, the next push
+does.
+
 ## Review-app auto-stop and pinning (GitLab)
 
 GitLab review environments stop automatically after 1 week, dev
@@ -151,6 +230,21 @@ pipelines: { github: { repository: "AcmeCorp/Nautilus" } },
 `yarn catladder project doctor` reports a mixed-case repository whose
 generated workflows still carry the expression.
 
+### Artifacts between jobs (GitHub)
+
+A job's `artifacts.paths` are handed to the jobs that need them as one
+tar archive (`catladder-artifacts-<job-id>.tar`): an "Archive
+artifacts" step tars them after the job, `upload-artifact` uploads the
+archive under the job id, and the consumers get a matching "Extract
+artifacts" step after `download-artifact`. The tar keeps symlinks, file
+modes and repo-relative paths, as GitLab artifacts do. Plain
+`upload-artifact` loses them, which breaks Next.js/Turbopack builds in
+pnpm workspaces: `.next/node_modules/<pkg>-<hash>` are symlinks into
+the pnpm store, and as copied directories they fail at runtime with
+`ERR_MODULE_NOT_FOUND`. If a deployed app shows that error, check that
+the committed workflows contain the archive steps and regenerate if
+they don't.
+
 ## Caching
 
 Cache configuration is generated per build type. **yarn** node builds