@coldtea/qa 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Coldtea
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,274 @@
+# @coldtea/qa — the `coldtea-qa` CLI
+
+Run Coldtea QA from a terminal or CI: manage projects, tests and groups,
+start runs and follow them to a verdict, upload mobile builds, merge
+projects. Every command talks to the public `/v1` API and nothing else.
+
+## Install
+
+The CLI is TypeScript with **zero runtime dependencies**, compiled to
+`dist/` before it runs. The build is not optional: Node refuses to strip
+types from anything under `node_modules`, so an installed package has to
+contain JavaScript. `npm install` builds it for you via the package's
+`prepare` script; `npm run build -w @coldtea/qa` rebuilds it by hand.
+Compiled output needs Node >= 20.6.
+
+One consequence worth knowing: `prepare` runs on every root `npm install` and
+`npm ci`, so a type error in this package fails INSTALL — before the dedicated
+typecheck job can report it with a useful message. That is the price of the
+binary always being built, and it is the right trade, but a confusing `npm ci`
+failure in an unrelated CI job is usually this.
+
+Inside this monorepo:
+
+```sh
+node packages/qa-cli/bin/coldtea-qa.mjs --help
+```
+
+## The API reference
+
+The CLI talks to `/v1` and nothing else, so the API reference IS the CLI's
+reference for what any command can do:
+
+    https://www.coldtea.ai/openapi/coldtea-v1.json
+
+and the same thing as a page you can read:
+
+    https://www.coldtea.ai/a/art_lsopxy23iqrvtojwpdiaole7w7x1325skjbjpoms19sygt6yc9
+
+OpenAPI 3.1, generated from the routes themselves — every endpoint, the scope
+it needs, its request body and what it returns. Import it into Postman,
+Insomnia, Apidog, or hand it to an agent. It is regenerated on every change and
+CI fails when it drifts, so it cannot describe an API we no longer have.
+
+## Auth
+
+Create an API key in the Coldtea dashboard, then:
+
+```sh
+export COLDTEA_API_KEY=coldtea_sk_…
+coldtea-qa whoami
+```
+
+The key identifies your organization; there is no organization flag. Most
+commands work on resource ids alone. Commands that list or create inside a
+project take `--project <id>` or `COLDTEA_PROJECT_ID`.
+
+`COLDTEA_BASE_URL` chooses the server. **Unset it points at production**
+(`https://www.coldtea.ai`); set it to a full origin to point somewhere else.
+Setting it to an EMPTY value is refused rather than falling back to
+production, because that fallback is silent and the key in your hand may not
+be one production should see.
+
+**The origin must be https** — every request sends your API key there, and
+plain http would put it on the wire in plaintext. Loopback is the exception
+(`localhost`, `127.0.0.0/8`, `[::1]`), because that is where dev servers
+live. For an http-only private host (`host.docker.internal`, a VPC staging
+mirror), set `COLDTEA_ALLOW_INSECURE_HTTP=1` — an explicit, greppable line
+in your CI file saying the key travels unencrypted.
+
+`--api-key` overrides the environment, but prefer the variable: a flag is
+visible in `ps` while the process runs, in shell history, and in any CI log
+that echoes the step it is about to run.
+
+## Output, exit codes, CI
+
+- `--json` prints one machine-readable JSON document on stdout, in the
+  API's own envelope: **`{ data, meta }`, and that is the contract** — the
+  same shape `/v1` sends and the same shape this CLI's errors use, so one
+  reader handles all three. It is deliberately not a CLI-shaped envelope;
+  changing it breaks every consumer rather than tidying anything. `meta` carries `requestId` on every
+  call, `nextCursor` on a list, and anything else the endpoint sends — so
+  `credentials list --json` can see `meta.unlistedCredentials`, and a
+  SUCCESSFUL call has a request id to quote, not just a failing one.
+- Errors go to stderr, as `{ "error": …, "meta": { "requestId": … } }` —
+  the same envelope the API sends, so one reader handles both. Quote the
+  request id in support. **On failure stdout is empty**: check the exit code
+  before parsing, because empty is not valid JSON.
+- Color only on a TTY, and never when `NO_COLOR` is set. Nothing prompts
+  without a TTY.
+- Exit codes are a contract, and the line between `2` and `4` is the one
+  worth knowing: **`2` is worth retrying unchanged, `4` never is.**
+  `0` pass (warnings pass unless `--fail-on-warning`) · `1` fail — the app
+  is broken; the only exit that means this · `2` couldn't run or verify:
+  a timeout, a batch with skipped tests, a 5xx, rate limiting, no credits,
+  a dropped connection · `3` conflict needing a human (the 409 family) ·
+  `4` auth or configuration error, **including a request the API refused as
+  malformed or as naming something that is not there** — a typo'd test id
+  is `4`, not `2`, because retrying it can only fail again.
+- A group run that queued nothing exits `2`, never `0`. Nothing was
+  verified, so nothing passed — and a batch replayed from an earlier run of
+  the same group on the same commit says so rather than looking fresh.
+
+## Projects
+
+```sh
+coldtea-qa projects list [--include-archived]
+coldtea-qa projects create "Storefront"
+coldtea-qa projects get prj_…
+coldtea-qa projects link-repo prj_… --provider github --repository-id 812345
+```
+
+`--repository-id` is the provider's stable id, not a URL or `owner/name`.
+A project holds at most one repository: linking over an existing link
+exits 3 and names the current repo. If the repo already holds report-name
+reservations that clash with this project's rule names, the CLI lists
+every conflict.
+
+## Tests and groups
+
+```sh
+coldtea-qa tests list --project prj_…             # or COLDTEA_PROJECT_ID
+coldtea-qa tests create "Checkout with a saved card" --url https://shop.example
+coldtea-qa tests create "Smoke the preview" --deployment
+coldtea-qa tests create "Login on device" --mobile ios
+coldtea-qa tests get mqt_…
+coldtea-qa tests update mqt_… --group mqg_…        # move between groups
+coldtea-qa tests delete mqt_…                      # archives; cancels runs
+
+coldtea-qa groups list
+coldtea-qa groups create "Smoke" --description "Fast checks"
+coldtea-qa groups update mqg_… --name "Nightly"
+coldtea-qa groups delete mqg_…                     # archives its tests too
+```
+
+A test targets exactly one of: a fixed URL (`--url`), the linked
+repository's deployments (`--deployment` — link a repo first), or a mobile
+app (`--mobile android|ios`). Deletes are soft (archive), and deployment
+tests cannot be moved between groups.
+
+## Environments, credentials and rules
+
+Enough to set a project up from nothing, which is what an agent or a new
+teammate actually needs.
+
+```sh
+coldtea-qa repositories list                       # ids for link-repo
+coldtea-qa environments create "Staging" --url https://staging.example
+coldtea-qa rules create "PRs" --deployment --branch main --groups mqg_… --checks
+coldtea-qa skips list                              # why a rule did not run
+coldtea-qa apps list                               # mobile apps and their builds
+```
+
+An **environment** is the place a run opens in. `run test --environment env_…`
+has always named one; this is how you make one. Only web environments are
+creatable here — a mobile one needs a device profile from the provider.
+
+A **credential** is a secret an environment signs in with, and the secret is
+read from stdin, never a flag: a command line is visible in shell history, in
+`ps`, and in CI logs.
+
+```sh
+echo -n "$VERCEL_BYPASS" | coldtea-qa credentials create "Preview wall" \
+  --method vercel_bypass
+coldtea-qa environments create "Preview" --url https://preview.example \
+  --credential qcr_… --vercel-project prj_…
+```
+
+Attaching a stored credential needs a **scope**, and it is not a formality:
+the address on the environment does not bound the secret, because a run brings
+its own URL. `--vercel-project <id>` spends it only on that Vercel project's
+deployments; `--any-url` spends it on every URL a run brings, which is
+unbounded and has to be said out loud rather than by omission.
+
+Nothing reads a secret back. `credentials list` is metadata only, and it may
+say that credentials exist it cannot name — those were stored before
+credentials recorded a TeaHouse.
+
+A **rule** is what runs QA when nobody asks, and it is the only thing that
+reports a check onto a pull request — starting a run directly stamps the commit
+and posts nothing, because check names are reserved per repository by the rule
+that owns them.
+
+```sh
+coldtea-qa rules create "CI" --webhook --groups mqg_… --checks
+```
+
+A webhook rule's hook URL is printed **once**, on create, because it contains
+the token. Put it in your CI's secrets then; there is no way to read it back.
+
+## Runs
+
+```sh
+coldtea-qa run test mqt_… --wait --timeout 10m
+coldtea-qa run group mqg_… --wait
+coldtea-qa runs list --project prj_… --status completed --outcome failed --since 2026-08-01
+
+coldtea-qa run test mqt_… --wait --environment env_…      # run in one place
+coldtea-qa run test mqt_… --wait --environment none       # run signed out
+coldtea-qa run group mqg_… --wait \
+  --vcs-provider github --vcs-repository-id 812345 \
+  --vcs-head-sha "$BITRISE_GIT_COMMIT" --vcs-pr-number "$BITRISE_PULL_REQUEST"
+```
+
+Without `--wait`, a run start exits 0 as soon as the queue accepts and
+prints the run id(s) and report URL(s). With `--wait`, the CLI polls
+until a verdict and the exit code reflects it; on timeout it exits 2 and
+the run keeps going server-side. A link is printed on every path — pass,
+fail, and timeout. It is the run's report page, which outlives the run;
+the live view URL is only a fallback, because a finished run has no live
+browser session left to look at.
+
+`--environment` names the place a run opens in, and replaces the group's
+environment whole rather than merging with it. `--environment none` runs
+signed out. Omitting it inherits the group's.
+
+The `--vcs-*` group attaches a run to a pull request: it stamps the commit
+and PR onto the run so the dashboard and report page show what was tested.
+It does **not** post a check back onto the PR — check names are reserved
+per repository by rules, so a check from CI comes from a webhook rule
+(`POST /v1/hooks/<token>`), not from a run start. All four flags travel
+together, and an empty value counts as absent, so passing an unset CI
+variable is safe.
+
+Deployment tests need `--target-url` (which deployment to test); mobile
+tests without a default build need `--build <artifactId>`.
+
+A group start is a batch. Tests skipped at admission (not runnable, or not
+enough credits) never become runs, so the CLI prints `skippedCount` /
+`insufficientCredits` from the 202 and a batch with skips exits 2 at best
+— even when every admitted run passes. A real failure still exits 1.
+
+Retries are safe: the CLI derives the idempotency key from the test id (or
+group id, for batches) plus the commit sha in CI, so re-running the same
+CI job replays the original run instead of billing twice. Off CI without a
+git sha, each invocation is a fresh run.
+
+Two things that sentence does not say, and both bite. The sha comes from CI
+variables _or_ from `git rev-parse HEAD` in the working directory, so the
+replay happens in any git checkout, not only in CI — re-running a test twice
+from your laptop gives you the first run back, and the CLI now says
+"Replayed run …" rather than "Queued". And the key is scoped **per API
+key**: rotating the key starts a fresh idempotency namespace, so the same
+commit can run, and be billed, twice.
+
+## Builds
+
+```sh
+coldtea-qa builds upload app-release.apk --platform android \
+  --vcs-provider github --vcs-repository-id 812345 \
+  --vcs-head-sha <sha> --vcs-pr-number 42
+```
+
+Three steps under the hood: initiate (a signed upload URL, 10-minute TTL),
+PUT the file to it (streamed, plain fetch), confirm with the file's
+sha256. The CLI then waits for registration by default — `ready` exits 0
+and prints the artifact id to run against; `failed` exits 2 with the
+failure code (e.g. `sha256_mismatch`). `--no-wait` stops after confirm and
+hands back a build id — read it later with `coldtea-qa builds get <id>`.
+Android takes `.apk`; iOS takes a Simulator `.app` in `.zip`/`.tar.gz`;
+max 1 GB.
+
+## Merge
+
+```sh
+coldtea-qa merge prj_old --into prj_new [--yes]
+```
+
+Merging is irreversible, so the survivor is a named flag on purpose — no
+positional argument order to get backwards. The CLI always fetches and
+prints the dry-run plan (what moves, what happens to the repo link) before
+asking. Interactively you must type `merge`; in CI pass `--yes`; with
+neither, it exits 4 without calling the merge endpoint. The merge is then
+polled to `completed`/`failed`; a failed merge keeps what already moved
+and re-running the same command resumes it.

package/bin/coldtea-qa.mjs

@@ -0,0 +1,114 @@
+#!/usr/bin/env node
+// THIS FILE IS A TWIN. packages/qa-cli/bin and packages/tasks-cli/bin are
+// the same bootstrap with the product names swapped, and a test
+// (tasks-cli's bins.test.ts) asserts they stay identical after
+// normalizing those names — a bin-level fix that lands in one and not the
+// other ships broken with nothing red anywhere.
+//
+// Runs the compiled `dist/`, not `src/`. Node refuses to strip types for
+// anything under node_modules (ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING),
+// which is where this package lives after any install — so pointing at the
+// TypeScript worked from a repo checkout and nowhere else. `npm install`
+// builds dist via the package's prepare script.
+//
+// STDOUT AND STDERR FAIL DIFFERENTLY, AND ONLY STDOUT CAN CHANGE THE EXIT
+// CODE.
+//
+// `coldtea-qa runs list | head` closes the pipe while we are still
+// writing. Node turns that into an unhandled EPIPE and a stack trace in the
+// build log. Swallow it on both streams: the reader has stopped caring what
+// we say, and the exit code — which is what CI actually reads — stays
+// whatever the command decided.
+//
+// Any OTHER stdout failure (ENOSPC on a redirected file, EIO) means the
+// caller did not get the output the exit code vouches for, so the exit code
+// becomes 2: we could not deliver. A stderr failure changes NOTHING: stderr
+// carries diagnostics, and flipping a delivered result to 2 — or worse,
+// overwriting a real verdict — because a log volume filled up would make
+// the exit code lie in the other direction.
+//
+// A RETHROW HERE WOULD BE AN UNCAUGHT EXCEPTION, AND NODE THEN PICKS THE
+// EXIT CODE — deterministically 1 in an adversarial full-disk pass, and 1
+// is the one code the contract reserves for a product verdict. A full disk
+// must never counterfeit one.
+//
+// AND THE POST-runCli CHECK CANNOT BE THE ONLY ONE, because `error` events
+// are asynchronous: a large buffered `--json` payload fails on a later tick
+// than that check. So the last word is an exit handler — `process.exitCode`
+// is still settable there — and it also SAYS what happened with a
+// synchronous fd write, because a silent 2 with healthy stderr explains
+// nothing.
+import { writeSync } from "node:fs";
+
+let stdoutFailure = null;
+let failureReported = false;
+process.stdout.on("error", (error) => {
+  if (error?.code === "EPIPE") {
+    return;
+  }
+  stdoutFailure ??= error;
+});
+process.stderr.on("error", (error) => {
+  // Diagnostics stream: EPIPE or not, a stderr failure never edits the
+  // exit code. Swallowing keeps it from becoming an uncaught exception.
+});
+
+function reportStreamFailure() {
+  if (failureReported) {
+    return;
+  }
+  failureReported = true;
+  try {
+    writeSync(
+      2,
+      `coldtea-qa could not write its output: ${stdoutFailure.message}\n`,
+    );
+  } catch {
+    // The error stream is gone too. Nothing left to say it on.
+  }
+}
+
+process.on("exit", () => {
+  if (stdoutFailure) {
+    reportStreamFailure();
+    process.exitCode = 2;
+  }
+});
+
+// Checked before the import, not in its catch: on an old Node the import
+// fails with whatever syntax or missing-export error it hits first, and that
+// stack tells the reader nothing about the actual problem.
+const [major, minor] = process.versions.node.split(".").map(Number);
+if (major < 20 || (major === 20 && minor < 6)) {
+  console.error(
+    `coldtea-qa needs Node >= 20.6 (found ${process.versions.node}).`,
+  );
+  process.exitCode = 4;
+} else {
+  try {
+    const { runCli } = await import("../dist/main.js");
+    process.exitCode = await runCli(process.argv.slice(2));
+    // A write that already failed means the caller did not get what the
+    // exit code claims. Say so now while stderr is definitely writable;
+    // the exit handler covers a failure that fires later than this line.
+    if (stdoutFailure) {
+      reportStreamFailure();
+      process.exitCode = 2;
+    }
+  } catch (error) {
+    // Only a missing dist means "not built". The same code thrown from
+    // inside a command is a real bug and must keep its stack.
+    const missingDist =
+      error?.code === "ERR_MODULE_NOT_FOUND" &&
+      String(error?.url ?? error?.message ?? "").includes("dist/main.js");
+    if (missingDist) {
+      console.error(
+        "coldtea-qa is not built. Run `npm run build -w @coldtea/qa`.",
+      );
+    } else {
+      console.error(error);
+    }
+    // Exit 4: the CLI could not even configure itself.
+    process.exitCode = 4;
+  }
+}