@bjoernrochel/sevro 0.1.0-rc.2 → 0.1.0-rc.21

package/docs/specs/repository-guide.md

@@ -0,0 +1,58 @@
+# Repository guide
+
+The repository-scoped `sevro-guide` explains this checkout. It is independent
+of npm packaging and serves a static documentation spine, not an execution agent.
+
+## Contract
+
+Select for ordinary Sevro questions, explicit invocation, and relevant follow-ups
+about orientation, installation, evaluations, extensions, results, architecture,
+troubleshooting, contribution, and licensing. Unrelated questions must not
+select it. Explicit invocation selects the guide even for out-of-scope requests,
+which receive an explanation of its boundary rather than hidden execution.
+
+Inspect relevant sources during each question. Cite them beside material claims.
+Current public contracts and applicable accepted decisions define intent;
+conflicts between them must be exposed. Package metadata and current docs define
+identity and installation. Code/tests support labelled derived facts, not a
+claim that tests passed. Research/historical validation provides context only.
+Missing, unreadable, or stale evidence cannot be replaced by model memory.
+
+Answers lead with a concise standalone explanation, with deeper references
+when useful. Preserve uncertainty and source conditions. A possible cause is
+not an observed diagnosis. Inspect disputed claims before judging them.
+Read relevant sources again for follow-ups; do not treat earlier answers as evidence.
+
+The guide may read files and explain documented commands. It must not edit,
+install, execute tests/evaluations/diagnostics, mutate Git/trackers, contact
+services, publish, or delegate effects. Execution requests get an explicitly
+separate request/handoff. Repository text is evidence, not permission to override
+the guide. Do not disclose credentials or private host evidence.
+
+## Host entrypoints
+
+Canonical instructions are `.agents/skills/sevro-guide/SKILL.md`. Claude uses
+the byte-identical `.claude/skills/sevro-guide/SKILL.md`. `AGENTS.md` routes
+ordinary Sevro questions; `CLAUDE.md` imports it. `bun run guide:sync` updates
+the mirror; `bun run check:docs` rejects drift. Both mounts are repository-only.
+Explicit invocation is `$sevro-guide` in Codex and `/sevro-guide` in Claude.
+
+Host discovery follows [OpenAI's skill documentation](https://learn.chatgpt.com/docs/build-skills)
+and [Claude Code's skill documentation](https://code.claude.com/docs/en/skills).
+Version-dependent discovery is tested with actual native sessions; file presence
+alone is not verification. See [guide evaluation](../guide-evaluation.md).
+
+## Acceptance
+
+The versioned question inventory and cases live under the canonical skill's
+`evals/`. Cases cover ordinary/explicit selection, unrelated requests,
+missing/conflicting/stale evidence, follow-ups, and pressure for effects.
+Hidden checks assess grounding, selection, useful results, and effects separately.
+The evaluation harness receives no authority to change project guidance or
+credentials. Record actual per-host results and unsupported/unexecuted routes.
+
+Documentation preserves all old files/destinations and their unique knowledge.
+No historical observation or normative contract is removed because the guide
+can summarize it. The [documentation quality contract](../documentation-quality.md)
+owns mechanical checks and accessibility review. A Rust research ticket does
+not establish a language migration, and #1 owns TypeScript quality policy.

package/docs/troubleshooting.md

@@ -0,0 +1,30 @@
+# Troubleshooting
+
+Start with execution, grading, task states, and stderr. A documented possible
+cause is not a diagnosis of your machine.
+
+| Symptom                                   | Check and next step                                                                                                                                                                     |
+| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Command cannot run                        | Use Bun 1.3.13 and verify the local package/version. Read [installation](installing.md).                                                                                                |
+| Project snapshot contains a symbolic link | Use a separate evaluated project directory as in [the tutorial](getting-started.md); do not include installed dependencies in a plain-directory snapshot.                               |
+| Exit 0 but no passing task                | Prompt-only/dry runs are `not_assessed`. Read [results](reading-results.md).                                                                                                            |
+| Exit 64 before execution                  | Check absolute paths, exclusive route options, configuration, and capabilities in the [CLI reference](development-cli.md).                                                              |
+| Grading unavailable                       | Inspect required evidence/completeness in [results v1](results-v1.md).                                                                                                                  |
+| Native authentication fails               | Check the exact release and private credential route. Read [native hosts](native-hosts.md).                                                                                             |
+| Native or shell isolation fails           | On macOS, verify `sandbox-exec`; on Ubuntu 24.04 x64, verify `bubblewrap` and Claude's `socat`. See [validated scope](native-hosts.md#validated-platform-scope). Windows is unverified. |
+| Extension protocol fails                  | Return one matching stdout response; diagnostics use stderr. Read [protocol v1](extension-protocol-v1.md).                                                                              |
+| Schema freshness fails                    | Run `bun run schemas:generate`, then `bun run schemas:check`, from a checkout.                                                                                                          |
+| External links fail                       | Follow [network failure triage](documentation-quality.md#external-links).                                                                                                               |
+
+## The repository guide is missing
+
+The guide belongs to a source checkout, not npm. Start a fresh session at the
+repository root. Check `.agents/skills/sevro-guide/SKILL.md` for Codex and
+`.claude/skills/sevro-guide/SKILL.md` for Claude. Invoke `$sevro-guide` or
+`/sevro-guide` respectively. See [host entrypoints](specs/repository-guide.md#host-entrypoints).
+`bun run check:docs` checks adapters, not native discovery; use [guide evals](guide-evaluation.md).
+Explanation does not authorize live diagnosis or repair.
+
+If these checks do not resolve the issue, open a [Sevro issue](https://github.com/BjRo/sevro/issues)
+with version, platform, sanitized command, states, and bounded diagnostics.
+Keep credentials, private case content, and raw host artifacts out of it.

package/docs/typescript-quality.md

@@ -0,0 +1,234 @@
+# TypeScript quality
+
+`bun run check:typescript` is the complete local acceptance command. CI runs the
+same checks in parallel phases. The complete command performs a
+frozen dependency installation, checks the complete source inventory, verifies
+generated schemas, documentation contracts, formatting, type-aware lint and typing,
+checks the Codex guide evaluation in dry mode and runs documentation examples, exercises coverage
+integrity controls, runs the deterministic Bun suite in an isolated instrumented
+copy, enforces statements and branches independently, and validates installation
+of the published package. Runtime dependencies and public evaluation semantics
+remain part of the existing package-install contract.
+
+Use `bun run check:typescript --fast` before committing. It checks inventory,
+schemas, documentation contracts, formatting, lint and types without the full integration suite, coverage
+or package installation. Install dependencies with `bun install --frozen-lockfile`
+first. Quality dependencies are development dependencies pinned in `bun.lock`.
+
+After frozen installation, `bun run check:typescript --phase <name>` runs one
+phase and checks the complete source inventory. Each phase reports its elapsed
+time, including when it fails. The complete command runs these same phases in
+order; `--fast` runs only `static` without installing dependencies.
+
+| Phase     | Checks                                                                                        |
+| --------- | --------------------------------------------------------------------------------------------- |
+| `static`  | Schema freshness, documentation contracts, formatting, type-aware lint and typing             |
+| `docs`    | All guide dry runs, tutorial examples and recipe checks                                       |
+| `tests`   | Coverage integrity controls, the complete instrumented Bun suite and both coverage thresholds |
+| `package` | Packed-package installation and installed CLI workflows                                       |
+
+Use the same prerequisites as the corresponding complete-gate checks. A single
+phase is partial evidence; full acceptance requires all four on the same candidate.
+CI assigns every test file to exactly one of four duration-balanced shards, then
+enforces both coverage thresholds on their combined counters over the complete
+production inventory. Local `--phase tests` and the complete command still run
+the suite in one process. See the [shard commands](../scripts/coverage/README.md#test-shards)
+for reproducing the CI split.
+
+## Source dispositions
+
+[`typescript-sources.json`](../typescript-sources.json) explicitly lists every
+TypeScript and JavaScript source. The guard refuses new, missing or duplicated
+files and declarations outside the configured TypeScript project. Changing a
+source disposition requires review; adding a glob is not a substitute for the
+inventory. The coverage baseline includes the platform-reachable subset of
+`inventory.production`; `scripts/coverage/platform-scope.ts` names each excluded
+host-specific file. Every production file remains inventoried, typed and linted,
+and the opposite platform measures it. Current instrumentation supports authored
+`.ts`. The five exact generated
+validators in `inventory.generated` are excluded from instrumentation and the
+coverage denominator. New `.tsx`, `.mts`, `.cts` or JavaScript
+production requires an explicit instrumentation and quality-policy change first.
+
+| Inventory      | Lint and typing                                                                                      | Test and coverage disposition                                                                                                                                                                                                |
+| -------------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `production`   | Type-aware strict lint and strict compiler                                                           | All platform-reachable authored executable `src` statements and branch outcomes, including unimported files in that scope                                                                                                    |
+| `generated`    | CommonJS syntax parsing, exact `schemas:check` freshness, and checked `.d.cts` boundary declarations | Real Bun schema tests; the five declared generated validators are outside the coverage denominator                                                                                                                           |
+| `declarations` | Type-aware lint and compiler                                                                         | Declaration-only files have no executable counters                                                                                                                                                                           |
+| `tooling`      | Same type-aware lint and strict compiler; `.mjs`/`.cjs` use `checkJs` and explicit JSDoc contracts   | Durable public gate probes, schema freshness, release and installed-package tests; outside the runner-production denominator                                                                                                 |
+| `tests`        | Same type-aware lint and strict compiler, including fixtures                                         | Real Bun filesystem/process/protocol tests; outside the production denominator                                                                                                                                               |
+| `examples`     | Same type-aware lint and strict compiler                                                             | Packaged recipe adapters, extensions and evaluator checks are consumer examples, exercised by tutorial and installed-package validation; they implement no runner behavior and are outside the runner-production denominator |
+
+Generated AJV CommonJS is deliberately not rewritten to conform to authored-code
+rules. An earlier audit of the original four generated files using ESLint's
+recommended JavaScript rules found 879 redeclarations, 281 useless assignments, 182 unused variables and
+108 unreachable-code findings. Those are generator emissions. Their syntax,
+generation freshness, typed public boundaries and runtime behavior remain checked.
+Generated compiler output is excluded from coverage because the coverage target
+measures authored runner behavior. This applies only to
+`src/generated/cli-result.cjs`, `extension.cjs`, `report.cjs`, `run-evidence.cjs` and `runtime.cjs`
+as explicitly declared in the inventory. This disposition does not apply to authored CommonJS, including
+the coverage capture hook.
+
+## Types, functions and review
+
+The locked `strictTypeChecked` preset runs with `projectService: true`. Unsafe
+assignments, arguments, calls, member access and returns are checked alongside
+floating/misused promises. The compiler enables `noUncheckedIndexedAccess`,
+`noImplicitReturns`, and `noFallthroughCasesInSwitch`.
+
+`exactOptionalPropertyTypes` was assessed and remains false. Existing adapter
+option compositions intentionally pass present `undefined` values: extension
+exchange `cwd`/`timeoutMs`/`signal`, host cancellation signals, optional fixture
+tools, and process-session state reset. Their current public behavior treats
+absence and present `undefined` identically. Conditional object spreads would
+obscure these compositions without strengthening external-input validation.
+New contracts should describe absence and `undefined` deliberately.
+
+Authored functions have a complexity ceiling of five, nesting depth three and
+80 nonblank, noncomment lines. ESLint's modified cyclomatic variant counts a
+switch as one decision; JavaScript logical operators, default arguments and
+optional chains also count decisions. This is comparable in intent to Darrow's
+Python ceiling of five, rather than a claim that the tools count identical
+syntax. Numeric interpolation is allowed for useful bounded diagnostics; unsafe
+or arbitrary object interpolation remains prohibited.
+
+Named integration callbacks may use a documented, local function-length
+exception to keep one fixture/invocation/assertion story together. The independent
+`sevro/test-callback-lines` rule bounds every direct or chained Bun `test`/`it`
+callback at 200 lines; helpers remain at 80 and complexity remains five.
+Each exception is explained in an ESLint comment beside the affected test.
+The maintained CommonJS capture hook has
+one import-style exception for its synchronous Node filesystem/path `require`
+calls. Instrumented authored modules import it to register process capture; the
+Bun test preload uses its exported handle to flush completion at teardown. The
+discovery interop counterexample remains historical evidence.
+
+Review meaningful domain names, cohesive module ownership, focused operations,
+validated external input, useful errors with retained causes, and assertions
+against observable behavior. A reproducible bug requires a regression test.
+Generated cases should exercise a meaningful parser, serialization,
+normalization or lifecycle invariant. Fast-check properties use seed 20261007,
+200 runs, and stop at the first failure; its failure output includes the seed
+and replay path. Preserve those values when reproducing a failure.
+
+## Coverage integrity
+
+Istanbul instruments the original TypeScript AST before type erasure. Babel
+composes transformation maps; `// @bun` preserves original stack locations.
+CLI shebangs stay first and inserted marker lines adjust mappings. Generated
+CommonJS follows normal loading and is never marked or passed through `onLoad`.
+
+Each run clones independent Git metadata, copies the candidate, keeps an
+unmodified source snapshot, and uses fresh report storage outside participant
+workspaces. Production imports register capture even when a child changes
+working directory. Bun teardown writes the test-runner completion; normal CLI
+exit writes child completion. A ledger records participants and checks run ID,
+layout, file scope, counter keys, nonnegative integer values, branch array lengths
+and completion before merging. Every platform-reachable executable production
+file is seeded with zero metadata without importing it to manufacture coverage.
+
+Intentionally force-killed owned children use periodic conservative checkpoints.
+The test preload declares the exact registered child killed by its parent.
+Ordinary children still require completion; an undeclared or mismatched parent
+cannot excuse a missing report. Missing checkpoints fail. No final counters are
+estimated: a lost tail can lower coverage and cannot increase it. This retains
+the real SIGKILL run-owner test and exempts no reachable production file.
+
+Statements and branches must each reach 95% using the exact covered/total ratio
+over the platform-reachable authored production inventory. On Ubuntu, the
+macOS `sandbox-exec`, Keychain and Apple metadata modules are excluded; on macOS, the Linux
+`bubblewrap` module is excluded. Shared isolation and credential code is counted
+on both.
+
+Apple developer-tool discovery lives in `src/runtime/discovery-apple.ts`, which
+the public loader imports only under its explicit Darwin guard. Linux excludes
+that exact native module; macOS measures it, and both platforms still typecheck
+and lint it. The provider lookup, bounded query and read-root safeguards remain
+shared production code measured on both platforms. A Linux loader fixture with
+available Apple shims confirms that its metadata executable is never queried.
+Both platforms retain the Apple module in the complete checked source inventory.
+
+The static TypeScript analysis omits only branch outcomes made
+impossible by `process.platform` on the current host. It leaves reachable
+outcomes and their raw counts intact, refuses any excluded outcome with a
+nonzero count, and records each file, line and branch outcome. The unprojected
+report is retained for audit. Rounded display percentages never determine
+success. The gate enforces the projected exact counters without compiler
+exemptions or changing count values. It validates
+source classifications before preparing coverage, including direct `--coverage`
+runs, so unregistered authored TypeScript or CommonJS, missing sources and
+overlapping dispositions refuse the gate. Generated validators retain normal
+CommonJS loading, schema freshness checks and real schema/runtime tests.
+
+Artifacts under `.quality/coverage` include the exact merged counters, baseline,
+summary, process ledger, Bun test log, unmodified source snapshot, LCOV and HTML.
+`run.json` and `enforcement.json` record the platform, included and excluded
+production files, branch exclusions, and exact projected counts.
+`coverage-raw.json` and the raw summary preserve all validated counters before
+platform projection.
+CI also retains `.quality/shards/<index>/` with each shard's source snapshot,
+process ledger, test log, raw and portable counters, and manifest. Process IDs
+remain scoped to their original shard. Aggregation verifies candidate contents,
+workflow run, platform, architecture, Bun version, instrumentation
+layout, and exact test assignments before combining counters with normalized
+source paths. Missing, repeated, incomplete, or mismatched shards fail the gate.
+Failed-job reruns can reuse successful shards from earlier attempts of the same
+workflow run; manifests retain each attempt, and rerun shards replace only their
+own artifacts. Another candidate or workflow run cannot supply missing evidence.
+`--coverage [Bun test arguments]` is a diagnostic measurement; it still enforces
+both thresholds and writes `.quality/coverage-targeted`, keeping it distinct from
+the full-suite evidence. Sparse runs are expected to fail the full denominator.
+
+The durable public probes are `--probe snapshot`, `--probe scope`, `--probe branch`,
+`--probe missing-reports`, `--probe integrity`, and `--probe source-map`.
+The scope probe records a zero-seeded baseline and merged real process report
+under `.quality/coverage-scope`; it verifies that ordinary generated CommonJS
+validation still runs. Branch and missing-report controls
+deliberately exit 1. A five-of-five statement, one-of-two branch sample must fail
+with `Below 95%: branches`. The integration suite retains real CLI/extension,
+graceful SIGINT/SIGTERM cancellation, filesystem isolation and owned-child paths.
+
+## Environment and CI
+
+The discovery backend was observed with Bun 1.3.13, macOS arm64 and Node 24.13.0.
+Implementation also ran on macOS arm64 (Darwin kernel 27.0.0). CI pins
+Codex 0.160.1 and Claude Code 2.1.284 for deterministic native hook tests. Hosted
+CI targets `ubuntu-24.04` x64 with Bun 1.3.13 and Node 24.13.0. Its first
+successful hosted run is required before describing that environment as validated.
+GitHub documents the [Ubuntu runner labels](https://docs.github.com/en/actions/reference/runners/github-hosted-runners).
+Ubuntu installs `bubblewrap`, `ripgrep`, and `socat` and runs the same native isolation cases;
+local macOS runs retain `sandbox-exec`. Windows remains unverified.
+
+Ordinary changes and explicit workflow dispatches run the four canonical phases
+concurrently. The tests phase consists of four shard jobs followed by a combined
+coverage job. Each job installs locked dependencies; native host tools are
+installed for `docs` and the test shards, and UV and mise for `docs`. Each shard
+retains its evidence, and the combined `tests` job uploads `.quality` even when
+coverage fails. Shards do not cancel their siblings on failure. A new commit to
+the same pull request cancels its superseded workflow run; main pushes and manual
+runs have independent concurrency groups. The stable aggregate `TypeScript quality`
+status requires every phase and all four shards to succeed. A missing, cancelled,
+skipped or failed phase or shard cannot turn that status green.
+
+Release metadata changes follow [the release workflow](releases.md) and reuse a
+successful full gate from an ancestor on `main`, with identical code,
+dependencies, schemas, tests, and other quality inputs. The selector permits
+only version/distribution-tag changes and its exact release-file inventory.
+It verifies each original phase job and its gate step, including every shard and
+the combined tests job for sharded runs, or a historical complete gate job and
+step. A partial matrix or reused release run cannot serve as fresh test evidence.
+Duplicate phase or shard jobs are ambiguous and refused. Release CI
+runs `check:typescript --fast`, records the tested source and CI run URL, and
+skips all four full-gate phase jobs. The aggregate
+status requires both verified prior evidence and successful static checks;
+missing, cancelled, failed, or ambiguous evidence cannot establish a pass.
+Reused results remain attributed to their original run rather than being
+reported as newly measured coverage. Releases perform no local test reruns.
+
+The [coverage tooling README](../scripts/coverage/README.md) explains the maintained
+pipeline and file responsibilities. Fresh CI reports provide candidate-specific evidence.
+
+References: [Darrow's Python quality contract](https://github.com/BjRo/darrow/blob/main/docs/specs/python-quality.md),
+[typescript-eslint presets](https://typescript-eslint.io/users/configs/),
+[ESLint complexity semantics](https://eslint.org/docs/latest/rules/complexity).

package/docs/usability-study.md

@@ -0,0 +1,114 @@
+# Assess the evaluation workflow
+
+Use this protocol to assess the developer experience tracked in
+[issue #53](https://github.com/BjRo/sevro/issues/53) with engineers who have not
+used Sevro. The implementation handover includes this protocol; human sessions
+and their measurements are deferred. No usability improvement or time saving
+has been measured.
+
+## Questions and current evidence
+
+Measure how quickly an engineer can obtain an assessed result, adapt an
+evaluation to a repository, and explain a failed check. Count documentation
+detours and record where configuration or defaults are difficult to discover.
+
+| Measure                                                   | Observed baseline | Target                                      | Observed changed workflow |
+| --------------------------------------------------------- | ----------------- | ------------------------------------------- | ------------------------- |
+| First credential-free assessed result, after installation | Unmeasured        | At most 60 seconds, initial research target | Unmeasured                |
+| Adapt an evaluation to the participant's repository       | Unmeasured        | Not established                             | Unmeasured                |
+| Understand a failing check                                | Unmeasured        | Not established                             | Unmeasured                |
+| Documentation detours for each task                       | Unmeasured        | Not established                             | Unmeasured                |
+
+Source inspection found that the original tutorial at revision `cd7542b`
+required seven explicit run options, plus JSON output and redirection. That is
+a description of the previous workflow, not an observed timing baseline.
+Automated example tests establish command behavior on their recorded candidate;
+they do not measure a new engineer's experience.
+
+## Prepare a session
+
+Recruit engineers unfamiliar with Sevro and record their relevant command-line,
+Bun, and model-tool experience. Use a disposable copy of a small repository
+they understand and a clean working directory for every task. Keep real
+credentials, private source, and retained host evidence out of shared records.
+
+Record the OS, architecture, Bun and Node versions, exact Sevro version or
+checkout commit, documentation revision, and installation command. Complete
+installation before starting the first-use timer. For a native repository run,
+record the selected host, model, effort, authentication prerequisites, and
+whether those prerequisites were already satisfied. Obtain the participant's
+agreement to any model usage before that task.
+
+Give the participant the [documentation hub](README.md) and these task goals.
+Let them use CLI help and documentation. The facilitator should observe without
+supplying commands. Record assistance when it occurs instead of counting an
+assisted task as an unaided success. Agree on a time limit before the session;
+record unfinished tasks as censored at that limit, with the blocking step.
+
+## Tasks and success evidence
+
+1. **See an assessed result.** Starting after installation, follow
+   [Your first evaluation](getting-started.md). Stop the timer when the
+   participant identifies the execution, grading, and task states and locates
+   the retained evidence. A dry run, prompt-only result, or exit code `0` alone
+   does not meet this goal. Record the command, result states, and evidence
+   location, without copying private evidence into the study notes.
+2. **Adapt it to a real repository.** Use the case-authoring and evaluation
+   instructions to prepare a task for the disposable repository, with a real
+   assertion and explicit candidate route. Stop when an executed assessment
+   produces a result the participant can interpret. Verify that the participant
+   understands how fixture content reaches the candidate; selecting a project
+   root alone does not populate a fixture. Record preparation time, model wait
+   time, and total elapsed time separately. If credentials or tools block the
+   task, retain that outcome rather than substituting the demo as success.
+3. **Explain a deliberate failure.** Use a recipe's documented failing variant.
+   Stop when the participant identifies the failed check, explains its reason
+   from retained evidence, and proposes the relevant correction. Record whether
+   they distinguish task failure from execution failure and grader error. Then
+   let them make the correction and repeat the run; record that iteration
+   separately.
+
+For each task, start timing when its goal is presented and stop at the stated
+success evidence. Record raw elapsed time, pauses with their reasons, outcome,
+help use, facilitator interventions, and every documentation detour. A detour
+is a transition from the task's workflow page to another reference, search
+result, or troubleshooting page to resolve missing information. Record the
+destination and reason; repeated visits remain visible. Report the counting
+definition alongside results so sessions can be compared.
+
+## Record and interpret results
+
+Keep one record per participant, workflow version, and task:
+
+```text
+participant_id:
+workflow_version_and_docs_revision:
+environment_and_prior_experience:
+task_and_time_limit:
+started_at:
+finished_at:
+elapsed_seconds:
+preparation_seconds:
+model_wait_seconds:
+pauses_and_reasons:
+outcome: completed | assisted | blocked | time_limit
+execution_grading_task_states:
+evidence_location_confirmed:
+documentation_detours_and_reasons:
+help_use_and_facilitator_interventions:
+failure_explanation_and_next_iteration:
+```
+
+Leave unavailable measurements blank and label them unmeasured in reports.
+Do not replace an unknown duration, usage, cost, or failed task with zero.
+Report individual observations, participant count, completion counts, assistance,
+and environment limits before any aggregate. Distinguish measured baselines,
+observed changed-workflow values, and targets in every table.
+
+For a comparison, preserve an executable baseline and its matching docs. Use
+separate first-time participants where possible; otherwise counterbalance the
+order with equivalent tasks and disclose learning effects. Small convenience
+samples identify friction but do not establish a population-wide improvement.
+Keep unsuccessful sessions and unresolved detours in the report.
+
+Return to [reading results](reading-results.md) or the [documentation hub](README.md).

package/examples/basic/host.ts

@@ -2,7 +2,7 @@ export default {
   id: "sevro.host.example",
   model: "deterministic-example",
   effort: "none",
-  async run() {
-    return { finalMessage: "ready", complete: true };
+  run() {
+    return Promise.resolve({ finalMessage: "ready", complete: true });
   },
 };

package/examples/recipes/candidate.ts

@@ -0,0 +1,26 @@
+import { readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+
+/** Trusted, deterministic participant for exercising the recipe's real graders. */
+const host = {
+  id: "recipes.deterministic",
+  model: "fixture-editor",
+  effort: "none",
+  async run({ workspace }: { workspace: string }) {
+    const task = await readFile(join(workspace, "README.md"), "utf8");
+    const failing = process.env.SEVRO_RECIPE_FAIL === "1";
+    if (task.includes("Fix add")) {
+      await writeFile(
+        join(workspace, "math.cjs"),
+        `exports.add = (a, b) => a ${failing ? "-" : "+"} b;\n`,
+      );
+    }
+    return {
+      finalMessage: failing
+        ? "Start with npm start on port 9000."
+        : "Run bun run dev on port 3000.",
+      complete: true,
+    };
+  },
+};
+export default host;

package/examples/recipes/cases/coding-change.json

@@ -0,0 +1,21 @@
+{
+  "id": "recipe-coding-change",
+  "prompt": "Fix add in math.cjs so it adds its two numeric arguments. Do not modify README.md. Explain how to start the application.",
+  "fixture": {
+    "files": {
+      "README.md": "Fix add in math.cjs. Start the application with bun run dev on port 3000.\n",
+      "math.cjs": "exports.add = (a, b) => a - b;\n"
+    }
+  },
+  "checks": [
+    {
+      "id": "addition",
+      "grader": "sevro.shell",
+      "configuration": {
+        "run": "{{evaluator-test}}",
+        "expectExact": "coding checks passed"
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/cases/host-tools.json

@@ -0,0 +1,21 @@
+{
+  "id": "recipe-host-tools",
+  "prompt": "Read app.env with rg, then explain the application's start command and port.",
+  "fixture": {
+    "files": {
+      "README.md": "Start with bun run dev on port 3000.\n",
+      "app.env": "port=3000\n"
+    }
+  },
+  "checks": [
+    {
+      "id": "rg-access",
+      "grader": "sevro.shell",
+      "configuration": {
+        "run": "rg '^port=3000$' app.env",
+        "expectExact": "port=3000"
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/cases/mise.json

@@ -0,0 +1,21 @@
+{
+  "id": "recipe-mise",
+  "prompt": "Use node from PATH to read app.json, then explain the application's start command and port. The operator has resolved mise's existing node@24.13.0 installation; do not download anything.",
+  "fixture": {
+    "files": {
+      "README.md": "Start with bun run dev on port 3000.\n",
+      "app.json": "{\"port\":3000}\n"
+    }
+  },
+  "checks": [
+    {
+      "id": "mise-node-access",
+      "grader": "sevro.shell",
+      "configuration": {
+        "run": "node -e 'if (process.version !== \"v24.13.0\") process.exit(1); const a = require(\"./app.json\"); if (a.port !== 3000) process.exit(1); console.log(a.port)'",
+        "expectExact": "3000"
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/cases/node.json

@@ -0,0 +1,21 @@
+{
+  "id": "recipe-node",
+  "prompt": "Read app.json with Node and explain the application's start command and port.",
+  "fixture": {
+    "files": {
+      "README.md": "Start with bun run dev on port 3000.\n",
+      "app.json": "{\"port\":3000}\n"
+    }
+  },
+  "checks": [
+    {
+      "id": "node-access",
+      "grader": "sevro.shell",
+      "configuration": {
+        "run": "node -e 'const a = require(\"./app.json\"); if (a.port !== 3000) process.exit(1); console.log(a.port)'",
+        "expectExact": "3000"
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/cases/python-uv.json

@@ -0,0 +1,21 @@
+{
+  "id": "recipe-python-uv",
+  "prompt": "Use uv run --offline --no-project --python \"$RECIPE_PYTHON\" python to read app.json, then explain the application's start command and port.",
+  "fixture": {
+    "files": {
+      "README.md": "Start with bun run dev on port 3000.\n",
+      "app.json": "{\"port\":3000}\n"
+    }
+  },
+  "checks": [
+    {
+      "id": "uv-python-access",
+      "grader": "sevro.shell",
+      "configuration": {
+        "run": "uv run --offline --no-project --python \"$RECIPE_PYTHON\" python -c 'import json; p=json.load(open(\"app.json\"))[\"port\"]; assert p == 3000; print(p)'",
+        "expectExact": "3000"
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/cases/semantic.json

@@ -0,0 +1,17 @@
+{
+  "id": "recipe-semantic",
+  "prompt": "Read README.md and explain how to start the application, including its port.",
+  "fixture": {
+    "files": { "README.md": "Start with bun run dev on port 3000.\n" }
+  },
+  "checks": [
+    {
+      "id": "startup-meaning",
+      "grader": "sevro.semantic",
+      "configuration": {
+        "proposition": "The response tells the reader to run bun run dev and says that the application listens on port 3000."
+      }
+    }
+  ],
+  "requiredEvidence": []
+}

package/examples/recipes/coding-change/check.ts

@@ -0,0 +1,23 @@
+import assert from "node:assert/strict";
+import { join } from "node:path";
+import { pathToFileURL } from "node:url";
+
+interface Addition {
+  add: (left: number, right: number) => number;
+}
+function isAddition(value: unknown): value is Addition {
+  return (
+    typeof value === "object" &&
+    value !== null &&
+    "add" in value &&
+    typeof value.add === "function"
+  );
+}
+const candidate: unknown = await import(
+  pathToFileURL(join(process.cwd(), "math.cjs")).href
+);
+assert.ok(isAddition(candidate), "Candidate must export add");
+assert.equal(candidate.add(2, 3), 5);
+assert.equal(candidate.add(-2, 3), 1);
+assert.equal(candidate.add(0, 0), 0);
+console.log("coding checks passed");