@outerlayer/cli 0.12.0 → 0.14.0

package/skill-pack/template/writing-tests/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: writing-tests
+description: Writes and reviews tests that catch a specific bug - assertion strength, the seam each test binds at, binding a test to an acceptance-criterion id, and checking by mutation that a test would notice a break. Use when writing or changing any test, when binding a test to a criterion id, when reviewing whether a test proves anything, or when a mutation tool reports survivors.
+---
+
+# Writing tests
+
+A test earns its place by catching a specific bug. Before keeping one, ask:
+what is the smallest change to production code that would still pass this
+test? If the answer is "almost anything", the test checks wiring, not
+behavior. Strengthen it or delete it.
+
+The rules here hold in any language and any test framework. The examples
+use TypeScript with a Jest-style `expect`. Read each one as the equivalent
+assertion in the repository's own framework.
+
+## Assertion strength
+
+These pass for almost any implementation, so they are never the only proof
+in a test:
+
+- An "is defined" or "is not null" check passes for any value at all.
+- A truthy or falsy check. Assert the exact value instead.
+- "Was called" proves it ran, not what it ran with.
+- "Does not throw" on its own. Assert the return value too.
+
+Negative forms state a real expectation and are fine: "was never called",
+"is absent". Prefer:
+
+- Exact equality on the actual return value or the whole object shape.
+- "Called with" and the full argument shape.
+- Positional equality on lists. It catches reordering and off-by-one.
+- A negative assertion that pins an injection path shut, such as
+  `expect(query).not.toContain(rawUserValue)`.
+- On an error path, the message or status a user or operator sees, not only
+  that something failed.
+
+Do not pin brittle internals: exact log strings, incidental call counts, the
+order of unrelated calls. Those break on refactors without catching bugs.
+
+## Choose the seam
+
+Name the seam before writing the test: the boundary the test drives the code
+through.
+
+- Business behavior is proven below the user interface, in the service or
+  the pure function.
+- A user interface contract is proven at the component level, through what
+  the user sees and does.
+- Behavior that lives in the data store, such as an access rule, a trigger
+  or the real shape of a query, is proven against the real store.
+- A criterion that spans two seams gets two tests, one at each.
+
+Check two blind spots every time:
+
+- **Fake-seam blindness.** A fake that returns every field whatever the query
+  asked for proves nothing about the query. Pin the contract: assert the
+  request the code sent, make the fake honor what was asked for, or test
+  against the real dependency.
+- **Structural pinning.** "X is never called for Y" needs a spy that is
+  present and asserts zero calls. A missing fake skips silently, exactly like
+  the real skip.
+
+A field that crosses a module or package boundary needs an assertion at every
+hop: writer, relay, reader. Pinning only the ends leaves the middle free to
+drop it.
+
+## Bind a test to its criterion
+
+When a spec has acceptance criteria with ids, the test that proves a
+criterion carries the id in a comment directly above the test, never in its
+title:
+
+```ts
+// proves LOGIN-03
+it("refuses a password reset for an unknown address without saying so", () => {
+  // ...
+});
+```
+
+The title says what the test proves to someone reading a failure. The
+comment lets a reader, or a script, find the test from the criterion. Never
+put a criterion id in a comment that is not that binding, because a reader
+will take it as one.
+
+Write the test from the criterion's words, at the seam it names, before the
+implementation exists. Run it and read the failure. It must fail because the
+behavior is absent, not because of a typo or a missing import.
+
+A criterion with a proof form such as `(proof: screenshot)` is proven by the
+evidence a reviewer sees, not by a test. It may still have one.
+
+## Mutation: prove the test would notice
+
+A green suite proves the tests run, not that they would catch a bug. Check
+the code the change adds or alters.
+
+**By hand.** For every condition, guard, limit and assignment the change
+adds, break it: invert it, widen it, drop half of an `and`, ignore the field.
+Run the one test file that should catch it. If nothing fails, write the
+assertion that would. Then take each criterion and break the production code
+it describes. Its test must fail. A test bound to a criterion that survives
+the break is not coverage.
+
+**With a tool.** Mutation testing is optional. Where the repository has a
+mutation tool, run it over the changed lines only. Each mutant that survives
+is a test still to write, not a score to accept. Two mutants are equivalent
+only when no observable output can differ; say why when you claim it. Hand
+breaks still matter for files the tool does not mutate, such as SQL, shell
+scripts, markup and configuration.
+
+## The repository's own rules
+
+The instructions file may name test commands, required checks, mocking rules
+or coverage thresholds. Those come first where they are stricter than this
+skill. Add them here when they are the team's standard, because this file is
+the team's to edit.

package/skill-pack/validators/acceptance-criteria.yaml

@@ -0,0 +1,35 @@
+# Yours to edit. Written once by outerlayer init; never overwritten.
+#
+# How to satisfy this row: once the pull request is open, have an agent that
+# did not write the change judge each recorded acceptance criterion against
+# the test that cites it. Write its verdicts as an .html file and emit it as
+# a report artifact bound to this check:
+#   outerlayer emit artifact criteria.html --for acceptance-criteria --caption "Each criterion judged against the test that cites it"
+# Then run `outerlayer sync`, and copy the report's deep link from the pull
+# request's evidence comment. The link can take up to a minute to appear
+# after the sync. If it is not there yet, wait a minute and read the comment
+# again; do not give up after one look. Never build the link by hand, and
+# never link a pull request comment: a comment can be edited or deleted later.
+#
+# Then record the verdict once, with that link as its proof:
+#   outerlayer emit acceptance-criteria --item "$OUTERLAYER_WORK" --result pass --link <the report's deep link>
+# Use --result pass only when the judge found every criterion it judged met.
+# Use --result fail when it found one unmet. A criterion whose proof is a
+# screenshot, video or other artifact is not judged here: its own criterion
+# row asks for that artifact.
+# When there is no criterion to judge, nothing is judged and nothing is
+# recorded, so the row never reads as passed.
+# Until a result is recorded, the row reads "not proven". While a host still
+# holds the work item's claim, it reads as waiting.
+#
+# `where: ci` means the result comes from `outerlayer emit`: OuterLayer runs
+# nothing for this check, and a result from a session, CI or a person counts.
+# The row claims only what that proves: a criteria judge's result is
+# recorded, not that the criteria hold.
+id: acceptance-criteria
+kind: validation
+row: "A criteria judge result is recorded for this change"
+level: warn
+run:
+  where: ci
+  emit: acceptance-criteria

package/skill-pack/validators/code-review.yaml

@@ -5,8 +5,10 @@
 # clean bill) as an .html file. Emit it as a report artifact with no --for:
 #   outerlayer emit artifact review.html --caption "Review of the change and what it found"
 # Then run `outerlayer sync`, and copy the report's deep link from the pull
-# request's evidence comment. Never build the link by hand, and never link a
-# pull request comment: a comment can be edited or deleted later.
+# request's evidence comment. The link can take up to a minute to appear
+# after the sync. If it is not there yet, wait a minute and read the comment
+# again; do not give up after one look. Never build the link by hand, and
+# never link a pull request comment: a comment can be edited or deleted later.
 #
 # Then record the verdict once, with that link as its proof:
 #   outerlayer emit code-review --item "$OUTERLAYER_WORK" --result pass --link <the report's deep link>

package/dist/chunk-T6OIJQRT.js

@@ -1,31 +0,0 @@
-#!/usr/bin/env node
-import { createRequire } from 'node:module';
-import { RUNNER_DIR_MODE, RUNNER_FILE_MODE } from './chunk-ZM2IMMYN.js';
-import { existsSync, readFileSync, mkdirSync, writeFileSync } from 'fs';
-import { dirname } from 'path';
-
-createRequire(import.meta.url);
-function readJobFile(path) {
-  if (!existsSync(path)) return null;
-  try {
-    return JSON.parse(readFileSync(path, "utf8"));
-  } catch {
-    return null;
-  }
-}
-function readJobFileChecked(path) {
-  if (!existsSync(path)) return { kind: "missing" };
-  try {
-    const parsed = JSON.parse(readFileSync(path, "utf8"));
-    if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) return { kind: "unreadable" };
-    return { kind: "ok", job: parsed };
-  } catch {
-    return { kind: "unreadable" };
-  }
-}
-function writeJobFile(path, job) {
-  mkdirSync(dirname(path), { recursive: true, mode: RUNNER_DIR_MODE });
-  writeFileSync(path, JSON.stringify(job, null, 2), { mode: RUNNER_FILE_MODE });
-}
-
-export { readJobFile, readJobFileChecked, writeJobFile };

package/dist/emit-cmd-LXEIDKLT.js

@@ -1,10 +0,0 @@
-#!/usr/bin/env node
-import { createRequire } from 'node:module';
-export { EmitError, runEmit } from './chunk-G5XY6CVG.js';
-import './chunk-FO7KKPYG.js';
-import './chunk-AHZQBGD2.js';
-import './chunk-325272HM.js';
-import './chunk-RA3O54FC.js';
-import './chunk-Z5GEFMRW.js';
-
-createRequire(import.meta.url);

package/dist/mcp-install-cmd-XZARXNYT.js

@@ -1,9 +0,0 @@
-#!/usr/bin/env node
-import { createRequire } from 'node:module';
-export { DEFAULT_MCP_COMMAND, DOCS_MCP_SERVER_NAME, McpInstallError, addDocsMcpEntryIfAbsent, addStdioMcpEntryIfAbsent, buildServerEntry, buildStdioServerEntry, defaultMcpUrl, handleMcpInstallError, runMcpInstall } from './chunk-A56PYPDF.js';
-import './chunk-YRYMNBGQ.js';
-import './chunk-EABW6AJQ.js';
-import './chunk-VNVZDWO3.js';
-import './chunk-Z5GEFMRW.js';
-
-createRequire(import.meta.url);

package/dist/starter-pack-Z22VRFPF.js

@@ -1,11 +0,0 @@
-#!/usr/bin/env node
-import { createRequire } from 'node:module';
-export { StarterPackError, assertKnownTemplate, formatStarterPackOutput, installStarterPack } from './chunk-ZQUCTFXC.js';
-import './chunk-YRYMNBGQ.js';
-import './chunk-FO7KKPYG.js';
-import './chunk-AHZQBGD2.js';
-import './chunk-325272HM.js';
-import './chunk-RA3O54FC.js';
-import './chunk-Z5GEFMRW.js';
-
-createRequire(import.meta.url);

package/dist/{chunk-3FMPZPJQ.js → chunk-R7PF5OOU.js}

@@ -1,7 +1,7 @@
 #!/usr/bin/env node
 import { createRequire } from 'node:module';
-import { startTimeOf, claimPidFile, RunnerAlreadyRunningError, releasePidFile } from './chunk-N5FOE5PS.js';
 import { RecipeError, locateRecipe, readRecipe, assertBuildable, writeSanitizedRecipe, classifyProperties, SANITIZED_NAME } from './chunk-QOZKPLVJ.js';
+import { startTimeOf, claimPidFile, RunnerAlreadyRunningError, releasePidFile } from './chunk-N5FOE5PS.js';
 import { CLAUDE_CODE_VERSION, NODE_IMAGE, DEFAULT_BASE_IMAGE, TINI_VERSION, GH_VERSION, GH_SHA256, TINI_SHA256 } from './chunk-CX22S5BA.js';
 import { withGitConfig, tokenHeader } from './chunk-BYT5O5RV.js';
 import { RUNNER_DIR_MODE, repoCacheDir, imagesDir, attemptStamp, runnerDir, rootfsPathFor, RUNNER_FILE_MODE } from './chunk-ZM2IMMYN.js';