@stsepelin/checktrail 0.1.0-alpha.1

package/docs/EXECUTION.md

@@ -0,0 +1,250 @@
+# Autonomous execution ledger
+
+This ledger tracks the full plan without treating a prepared scaffold as a
+completed release. Work proceeds through independently testable implementations;
+an unavailable runtime or external service does not count as verification.
+
+## M0 and M1
+
+- Implemented: public contracts, synthetic fixtures, CLI/library/MCP engine,
+  trust controls, bounded execution, native Node/Python/Go checks, schemas and CI
+  definition. Integration evidence is recorded in `STATUS.md`.
+- Verified separately: PHP syntax against PHP 8.4.23 in an isolated official Linux
+  container, using the actual engine integration test.
+- Verified: fresh offline package installation with Claude Code health/discovery
+  and Codex app-server direct tool calls; see `CLIENTS.md` for the precise profiles.
+- Verified: strict standard compilation of exported schemas, equivalent URL/path
+  boundary cases, and warning-free Claude Code discovery after correcting the
+  nonstandard prefix format. Original frozen evaluation artifacts remain separate.
+- Remaining: hosted CI runs and broader client profiles. PHP syntax
+  planning still reports unavailable when PHP is absent from the consumer runtime.
+
+## M2
+
+- Implemented: explicit local TypeScript and ESLint adapters with file accounting.
+- Implemented: TypeScript project-reference solution validation using fresh
+  in-memory declarations, source accounting and normalized compiler diagnostics.
+- Implemented: Vitest execution, strict evidence parsing, and native regression
+  cases for skipped/excluded tests, focused tests and snapshot/error bypasses.
+- Implemented: Jest native execution and evidence, with conservative pending-test
+  handling and collection-only/snapshot/result-processor bypass tests.
+- Implemented: vue-tsc with native script/template error cases, file inclusion
+  accounting and inherited template-check disabling rejection.
+- Implemented: pytest lifecycle evidence, collection accounting, fixture and
+  unexpected-pass failures; verified with prepared Python 3.12 Linux tooling.
+- Implemented: Ruff file/settings/JSON evidence, disabled fixes, native diagnostics,
+  exclusion and no-active-rule cases in prepared Linux tooling.
+- Implemented: mypy source-count evidence, per-module broad suppression guard,
+  no stub installation/cache writes, and native Linux regression fixtures.
+- Implemented: opt-in Go race validation, with native racing and atomic-update
+  fixtures on macOS. Native package/file and package-test participation accounting
+  now reports excluded source and untested packages as incomplete.
+- Implemented: PHPStan per-file/debug and native JSON accounting, with return-type,
+  excluded-file and bad-configuration fixtures in PHP 8.5 Linux tooling.
+- Implemented: PHPUnit fresh JUnit streaming, exact file/assertion accounting and
+  native failures/skips/empty cases in PHP 8.5 Linux tooling.
+- Implemented: standalone JUnit import through CLI/library, labeled as imported
+  evidence, with bounded XML and aggregate-counter validation.
+- Implemented: Pest fresh evidence, focused-sibling and dataset checks, explicit
+  TIA disabling and missing-snapshot rejection in prepared PHP Linux tooling.
+- Implemented: Pint native file/fixer accounting, dry-run formatting, empty-rule
+  and unverified Blade integration guards, with native Linux fixtures.
+- Implemented: Playwright native lifecycle/project/file evidence, focused-test and
+  snapshot controls, flaky failure and explicit browser prerequisites, including
+  real Chromium DOM assertions.
+- Implemented: Staticcheck all-rule JSON analysis with surfaced suppressions,
+  normalized findings and native regression cases.
+- Implemented: a constrained configured golangci-lint profile, explicit linter
+  selection, hidden-filter/fix prevention and native suppression accounting.
+- Implemented: bounded native version probes and labeled package-metadata/runtime
+  identities in detailed reports, with incomplete results for unknown identities.
+- Implemented: normalized ESLint/Ruff/PHPStan findings and SARIF 2.1.0 export
+  through CLI/library, with official-schema and native CLI regression evidence.
+- Implemented: per-project environment requirements with operator CLI/MCP startup
+  permissions, value-free command metadata and environment-scoped tool identity.
+- Implemented: explicit workspace dependencies and Git base selection, transitive
+  consumer expansion, full-plan fallback and post-run Git identity validation.
+- Implemented: exact finding baselines and exceptions, owner/reason/expiry,
+  occurrence-count reconciliation, stale entries, explicit limits and optional
+  comparison against a previous baseline. CLI/library/MCP keep comparison status
+  separate from native validation. Partial analysis cannot be baselined.
+- Remaining: broader framework scope and hosted toolchain CI evidence.
+
+## M3
+
+- Implemented: schema-validated runtime inventory comparison through CLI/library/
+  MCP, exact registration multiplicity and ordering, explicit incomplete collection
+  accounting, and synthetic tests for every supported assembly category. Imported
+  comparison remains distinct from native framework capture.
+
+- Implemented: reusable public JSON check profiles, pinned private/local pack
+  loading and additive operator-selected overlays, with post-execution policy
+  verification and conservative Git selection. Native framework semantics remain
+  separate from these check profiles.
+- Implemented: captured producer/consumer JSON Schema contracts with strict
+  non-mutating validation, bounded workers, CLI/library/MCP and synthetic native
+  producer serialization. Live service integration remains separate.
+- Implemented: an opt-in FastAPI flat native route collector and exact duplicate
+  registration rule, after lifespan startup, with Linux native broken/fixed/near-
+  miss fixtures and explicit unsupported-shape handling. Broader framework
+  semantics and other assembly categories remain separate.
+- Implemented: a Django native nested URL resolver collector and exact duplicate
+  pattern-chain rule, with setup-time wiring, regex matching-mode preservation,
+  protected test settings and Linux native regression evidence.
+- Implemented: Laravel testing assembly collection for routes, middleware, exact
+  and wildcard listeners, schedules and a defined container binding projection.
+  Native PHP/Linux fixtures exercise actual Artisan initialization, framework-added
+  routes, mutation comparisons, cold caches, dotenv exclusion and incomplete assembly.
+- Verified: public synthetic FastAPI, Django and Laravel examples through their
+  native collectors; no private application code is needed for these regressions.
+- Implemented: explicit language-agnostic dependency graph policies, exact layer
+  allowlists, iterative cycle detection, incomplete capture accounting and
+  CLI/library/MCP surfaces. Native source import graph collection remains separate.
+- Verified: a real TypeScript producer tarball installed offline into a synthetic
+  consumer, with type and runtime checks for changed and misleading declarations;
+  a declared package-graph boundary check accompanies the integration fixture.
+- Implemented: a native Vue Router testing assembly profile with awaited registration,
+  exact URL probe chains, alias/parent participation, bounded metadata projection
+  and incomplete-record accounting. macOS/Linux native tests and installed-package
+  library/CLI/MCP checks passed; see `VUE-ROUTER.md`.
+- Implemented: a Nuxt SSR testing assembly using a fresh native build and in-process
+  requests, final runtime route capture, stable-assembly checks and complete record
+  participation. Native macOS/Linux fixtures and an offline read-only installed
+  library/CLI/MCP example passed; see `NUXT.md`.
+- Pending: broader framework semantics, native assembly/import graph collectors
+  and live service integrations.
+- Private repository results and proprietary narratives must stay out of public
+  source, examples, reports and history.
+
+## M4
+
+- Implemented: a Rust/Cargo 1.98.1 single-package compilation profile, locked
+  offline dependencies, fresh build directories, native target and source
+  accounting, structured compiler diagnostics and Linux regression evidence.
+  Rust test execution, formatting, Clippy and workspace matrices remain separate.
+- Implemented: MRI Ruby syntax checking for source and Ruby DSL manifests,
+  per-file native success accounting, disabled gem/preload behavior, synthetic
+  broken/fixed cases and Linux/macOS native verification. Ruby tests and framework
+  semantics remain separate capabilities.
+- Implemented: Swift source/manifest grammar checking without SwiftPM evaluation,
+  native macOS compiler identity, generated .build exclusion and explicit regression
+  evidence distinguishing syntax from type checking. Swift build/test remain work.
+- Implemented: prepared C/C++ Clang compilation-database checks with closed flag
+  handling, native SARIF diagnostics, fresh dependency/source accounting and native
+  macOS/Linux broken/fixed cases. Linking, CTest, clang-tidy and cross-target
+  build matrices remain separate.
+- Implemented: explicit Java release/classpath compilation with checksummed JARs,
+  disabled annotation processing, native source/type accounting, Linux regression
+  evidence and fresh packaged library/CLI/MCP verification. JVM build systems,
+  modules, Kotlin/Scala and test runners remain separate. See `JAVA.md`.
+- Implemented: explicit C# compilation through the prepared .NET SDK/Roslyn API,
+  pinned DLL references, native syntax/semantic accounting, source-suppression
+  handling and Linux regression evidence. MSBuild, restore, source generators,
+  analyzers, F#/VB, framework-specific builds and test runners remain separate.
+  See `DOTNET.md` for the precise profile and reproduction steps.
+- Implemented: GitHub Actions workflow validation through actionlint with explicit
+  labels/variables, copied local dependencies, repository-ignore bypass and native
+  per-file completion evidence. Terraform, Helm and Kustomize remain unsupported.
+  See `ACTIONLINT.md` for profile limits and reproduction steps.
+- Verified: fresh offline tarball install; public library, installed CLI and
+  packaged MCP stdio checks against eight synthetic JavaScript toolchain projects.
+  Reproduce with `node scripts/smoke-package.mjs`. This now also exercises the
+  Clang profile when a verified local compiler is present and records unavailable
+  otherwise; a verified macOS compiler passed the installed profile.
+- Implemented: bounded local private pack distribution with SHA-256 pinning.
+- Implemented: production npm dependency metadata/notice audit reconciled with
+  the installed tree and lockfile, including a pinned upstream notice for one
+  distribution that omitted it. Development/native/container provenance remains
+  separate; this is not a complete supply-chain or legal-compliance attestation.
+- Implemented: bounded synthetic planning/execution performance measurement with
+  raw samples, source/tool/harness identities and engine-only resource accounting.
+  Recorded macOS/Linux snapshots and limitations are in `PERFORMANCE.md`.
+- Implemented: operator-registered executable adapters with pinned manifests/files,
+  exact scoped-file accounting, Node/Python/PHP/native runtimes and shared cancellation
+  cleanup. Linux native fixtures and offline installed library/CLI/MCP checks passed;
+  delegated tool metadata remains adapter-reported. See `EXTERNAL-ADAPTERS.md`.
+- Implemented: explicit HTTPS data-only pack distribution with independently pinned
+  digests, bounded TLS transfers, strict content validation and exclusive atomic
+  file publication. Planning and MCP never download automatically. See
+  `PACK-DISTRIBUTION.md`. Executable bundle distribution remains separate.
+- Implemented: unpublished registry metadata with matching npm identity, a required
+  local root, execution disabled by default, official-schema validation and a
+  metadata-derived MCP startup regression. See `RELEASE.md` for concrete release
+  gates.
+- Pending: broader runtime/OS/performance matrices and external release gates.
+- Windows execution needs actual process-tree cancellation evidence before it
+  can be advertised. Hosted CI and a public release have not happened.
+- Commits, remote creation and publication retain their existing explicit-action
+  requirements; local implementation and release preparation can continue.
+
+## M5
+
+- Implemented: versioned public advisory guidance selected by exact check/topic
+  triggers, with CLI/library/MCP surfaces, bounded schemas and summary projections.
+  Retrieval is separate from automated findings and coverage claims.
+- Implemented: targeted literal mutations for dependency-free flat Node tests,
+  temporary source copies, assertion-specific kill evidence, survivor/incomplete
+  accounting, baseline gating, cancellation and original-source preservation.
+  Broader language/runner profiles and automatic mutation selection remain pending.
+- Implemented: paired full/Git-selected native impact measurement with original
+  transitive-consumer fixtures, baseline passes, exact assertion identities,
+  conservative fallbacks, misdeclared-graph misses and raw cost observations.
+  macOS/Linux evidence and limits are in `IMPACT-MEASUREMENT.md`.
+- Implemented: optional local/model review exchange with bounded selected source,
+  operator-gated disclosure, context/source digests, exact citation checks, explicit
+  unreviewed-file accounting and declared model/version/token/cost metadata. Imported
+  claims remain advisory and never alter deterministic results; see `REVIEW-EXCHANGE.md`.
+- Pending: broader impact models. Review exchange does not itself establish model
+  inference quality or independent evaluation.
+- Implemented: a frozen post-implementation development corpus with per-family
+  diagnostic-specific detection, false-positive and incomplete-result accounting,
+  direct native-command comparisons, raw timing/tool/artifact identities and
+  macOS/Linux observations. See `EVALUATION.md` for interpretation limits.
+- Implemented: external ESLint integration evaluation using independently authored
+  synthetic upstream cases and a verifier frozen before case inspection. Native
+  and verifier diagnostics agree on the stated macOS/Linux cohort; raw labels,
+  observations, artifact identities and exclusion accounting are retained in
+  `EXTERNAL-EVALUATION.md`. It is not independent native-rule effectiveness or
+  general review-quality evidence.
+- Implemented: external Ruff diagnostic integration cohort with declared source
+  selection and a frozen verifier, paired native/macOS/Linux observations and
+  primary-location reconciliation. The mixed upstream files do not supply an
+  independent clean-case denominator; see `EXTERNAL-RUFF-EVALUATION.md`.
+- Pending independent measurement: broader end-to-end review cases and rule families,
+  prior-workflow comparisons and representative cost/latency estimates. The
+  development corpus is not an independent held-out sample.
+- Implemented: library-only bounded local validation task storage with atomic
+  transitions, exclusive process ownership, retained projected results, two-stage
+  cancellation and explicit interruption errors after reopening. Native macOS/Linux
+  crash and contention evidence is documented in `TASK-STORAGE.md`.
+- Implemented: a library worker connecting durable storage to the shared engine,
+  with startup-only permissions, responsive polling, overlap rejection, confirmed
+  cancellation, parent-disconnect cleanup and retained native outcomes. See
+  `VALIDATION-TASKS.md` for native macOS/Linux evidence and the explicit limitation
+  on forcibly killing the worker itself.
+- Durable MCP Tasks remains pending. The current SDK routing failure is reproduced
+  by `npm run probe:mcp-tasks`; storage and worker lifecycle are implemented
+  independently, but Tasks support must not be advertised before wire tests pass.
+
+The dedicated Ruby and Swift CI paths now require their exact native regression
+names, rather than accepting a successful suite that skipped native work. Local
+Ruby/Linux/Node 22 and Swift/macOS execution pass. Runner regressions reject
+missing names, wrong-file names, skips, TODOs, duplicate names, suite-only matches
+and unrelated failures. Removing the missing-test rejection fails the named
+regression. These profiles subsequently passed hosted CI at `52ba415`; see
+`NATIVE-CI.md` for the complete workflow evidence.
+
+The same exact-name gate now covers the prepared CI core, JavaScript, Python,
+Go, PHP, framework and Rust profiles, plus the Clang, Java, C#, actionlint,
+Vue Router, Nuxt, review and durable-task container suites. Local required-profile
+runs passed and their accounting is retained in `NATIVE-CI.md`. Both dynamically
+named cancel/close lifecycle cases are explicit requirements. Existing external
+adapter per-runtime gates remain, and the PHP syntax shim now requires exactly
+one successful native TAP case. Hosted CI remains a separate acceptance gate.
+
+## External dependencies
+
+The installed MCP server SDK is still 2.0.0. Its standard Tasks routing failure is
+documented in `MCP-COMPATIBILITY.md`. Hosted CI, publication and independent target
+OS/client evidence remain separate acceptance gates. A gate is completed only
+when its implementation and corresponding evidence exist.

package/docs/EXTERNAL-ADAPTERS.md

@@ -0,0 +1,170 @@
+# Trusted external adapters
+
+An operator can register a local executable bundle without modifying the engine.
+The bundle declares discovery markers, scoped checks and SHA-256 hashes for each
+file it needs. Registration reads and verifies metadata without executing the
+adapter. Running it requires the same trust permission as built-in checks.
+
+This is a local extension protocol, not a plugin marketplace or sandbox. A digest
+identifies bytes; it does not authenticate an author, prove an analysis correct or
+pin system libraries and tools. A trusted adapter can fabricate protocol evidence.
+Review its implementation, licenses and native regression cases before using it.
+
+## Try the original example
+
+From a built checkout, compute the digest of the exact manifest bytes:
+
+```sh
+adapter_manifest="$PWD/examples/external-adapter/bundle/adapter.json"
+adapter_digest=$(shasum -a 256 "$adapter_manifest" | cut -d ' ' -f 1)
+node dist/src/cli.js plan --root examples/external-adapter/project \
+  --adapter "$adapter_manifest#sha256=$adapter_digest" --detailed
+node dist/src/cli.js run --root examples/external-adapter/project \
+  --adapter "$adapter_manifest#sha256=$adapter_digest" --trust-project --detailed
+node dist/src/cli.js serve --root examples/external-adapter/project \
+  --adapter "$adapter_manifest#sha256=$adapter_digest" --allow-execution
+```
+
+The example checks trailing spaces and tabs in text files. Its manifest includes
+the original implementation and its MIT license. These examples are checkout
+assets, not files installed by the npm package. For an independently obtained
+bundle, compare its digest against a separately trusted value; computing a hash
+of an unreviewed download does not establish trust.
+
+`--adapter` can be repeated. Library callers pass
+`externalAdapters: [{ path: absoluteManifestPath, sha256: expectedDigest }]` to
+`createPlan` or `validate`; validation also needs `trusted: true`. MCP registration
+is startup-only. A tool argument or repository configuration cannot add adapters
+or grant execution permission. Project policy may select already registered check
+IDs such as `external.example-lines.whitespace`. The registered manifest identities
+contribute to the policy fingerprint. External registration conservatively disables
+Git narrowing because the engine cannot establish an external tool's impact model.
+
+## Bundle contract
+
+The generated schemas are `schemas/external-manifest.schema.json`,
+`external-reference.schema.json`, `external-request.schema.json` and
+`external-result.schema.json`. Runtime validation also enforces cross-field
+constraints such as uniqueness, test accounting and exact scope matching.
+
+Manifest version 1 declares:
+
+- `id`: `external.` followed by a lowercase name; `version`: a version string.
+- `runtime`: `node`, `python3`, `php` or `native`; `entry`: a listed relative file.
+- `files`: normalized relative paths and lowercase SHA-256 digests.
+- `markers`: exact file basenames identifying project roots.
+- `checks`: unique IDs, descriptions, kind (`analysis`, `syntax`, `format`, `test`),
+  failure level (`error` or `warning`), and filename/suffix scope selectors.
+
+Check IDs are the adapter ID plus `.` plus the declared check ID. Scope selectors
+form a union over inventoried files assigned to the closest detected project root.
+They are case-sensitive exact basenames or literal suffixes, not glob patterns.
+The shared inventory exclusions apply. An empty selected scope is unavailable.
+
+Manifest and declared files must be regular files. Bundle file paths cannot escape
+the canonical bundle directory or traverse symbolic links. The manifest cannot
+include itself. Only declared, verified bytes are copied into a temporary bundle;
+unlisted files are neither hashed nor copied. Relative imports must refer to files
+included in the bundle. Python isolated mode does not automatically put the bundle
+directory on the import path; adapters needing bundled modules must load them
+explicitly. Nothing prevents trusted code from reading additional external files.
+
+## Invocation and evidence
+
+The adapter runs with the discovered project as its working directory. Its sole
+application argument is the path to a JSON request. Node uses the current Node
+executable; Python uses `python3 -I -B`; PHP uses `php -n`; native execution uses
+the copied entry with executable permissions. Python's
+[isolated mode](https://docs.python.org/3/using/cmdline.html#cmdoption-I) controls
+startup/import behavior and is not operating-system isolation.
+
+The request contains `protocolVersion: 1`, canonical `root`, relative `project`,
+adapter `identity`, `runtime`, `checkId`, `kind`, `failOn`, project-relative `scope`
+and `sourceFingerprint`. stdout must contain exactly one result JSON object:
+
+```json
+{
+  "protocolVersion": 1,
+  "identity": {
+    "id": "external.example",
+    "version": "1.0.0",
+    "sha256": "<manifest digest>"
+  },
+  "checkId": "external.example.whitespace",
+  "sourceFingerprint": "<request fingerprint>",
+  "files": [{ "path": "value.txt", "status": "checked" }],
+  "findingsComplete": true,
+  "findings": [],
+  "tools": [{ "name": "example-checker", "version": "1.0.0" }]
+}
+```
+
+The placeholders above must be replaced with the request's actual digests. Results
+must echo the identity, check ID and source fingerprint and account for every
+planned file exactly once. File statuses are `checked`, `skipped` and `unavailable`.
+Diagnostics have `ruleId`, `level`, `message` and optional scoped `file`/`line`.
+The engine prefixes rule IDs with the full check ID and converts finding paths to
+root-relative paths. Unknown fields, unplanned files, omissions, duplicates and
+contradictory evidence cannot pass.
+
+Test checks require per-file `tests: { total, passed, failed, skipped }`. Counts
+must reconcile; empty, all-skipped or partly skipped files are incomplete. Other
+check kinds must not include test counts. Exit 1 must agree with failed tests or
+diagnostics at the declared failure level. Exit 0 must agree with their absence.
+Other exits are errors. A known failure can coexist with incomplete coverage;
+`findingsComplete` preserves that distinction. stderr is retained as bounded
+detailed process evidence and is not itself a diagnostic protocol.
+
+Delegated tool identities have `source: "adapter-reported"` in detailed reports.
+They are not independently attested. The engine separately probes supported
+interpreter versions and reads the pinned manifest version without executing the
+adapter for its version probe. Default summaries omit bundle paths, delegated
+metadata, raw output and source paths; check IDs and outcomes remain visible.
+
+## Execution limits and cleanup
+
+Registration allows at most eight bundles. Each manifest is at most 256 KiB;
+each bundle lists at most 512 files, 32 MiB per file and 128 MiB combined. A manifest
+has at most 32 markers and 32 checks. A check has at most 20,000 scoped files and a
+100 KiB serialized invocation. Results have at most 2,000 findings and 32 tools;
+combined child stdout/stderr is bounded to 1 MiB. Shared runner limits can make a
+large encoded result incomplete before these individual maxima are reached.
+
+The shared runner applies its timeout, output bound and POSIX process-group
+cancellation. It owns the temporary directory and removes it after the child closes,
+including after cancellation kills the wrapper. This does not guarantee cleanup
+after an engine crash or prevent malicious processes from detaching. The engine
+checks inventoried source and pinned bundle contents before and after execution.
+Ignored dependencies, transient restored modifications and external services remain
+outside that fingerprint. Concurrent adversarial changes require OS isolation.
+
+External code inherits the narrow runner environment plus operator-authorized
+variables. `PATH`, `NODE_OPTIONS` and the owned temporary-directory setting are
+protected from project environment overrides. No dependency installation, download
+or network isolation is supplied by this protocol.
+
+## Reproduce verification
+
+```sh
+npm run check
+docker build --file scripts/external-tools.Dockerfile \
+  --tag checktrail-external-test:1 scripts
+node scripts/verify-external-container.mjs
+```
+
+The helper requires Docker and an npm cache prepared by dependency installation.
+Image preparation may access the network; verification containers use
+`--network none` and read-only checkout/installed-package mounts. The separate
+PHP image and the build stages are pinned by digest in the helper/Dockerfile.
+
+Verified profiles are macOS Node 26.8.1, Python 3.9.6 and compiled Go 1.27.1, and
+Linux arm64 Node 22.23.2, Python 3.12.13, Go 1.26.5 and PHP 8.5.6. The native fixture
+is compiled from the original `examples/external-adapter/native/adapter.go`.
+Tests exercise actual broken/fixed whitespace, a CRLF near miss, bundle tampering,
+scope/result contradictions, trust boundaries, startup-only MCP registration and
+child cancellation with cleanup. Removing the file-count reconciliation guards
+was manually verified to produce a false pass caught by the accounting regression;
+the guards were restored before the final passing runs. The helper also verifies a fresh offline package
+through the library, CLI and MCP. These checks establish this protocol's behavior;
+they do not establish arbitrary adapter correctness or Windows support. The
+separate hosted Linux job passed at `52ba415`; see `NATIVE-CI.md`.

package/docs/EXTERNAL-EVALUATION.md

@@ -0,0 +1,155 @@
+# External ESLint integration evaluation
+
+This evaluation uses independently authored synthetic ESLint rule fixtures that
+were not used to build Checktrail's adapter. The verifier runtime and lockfile
+were hashed before the selected upstream case files were downloaded or inspected.
+They remained unchanged during measurement. This supplies an external integration
+holdout for that frozen verifier, not independent evidence about ESLint's own
+rule algorithms or general code-review quality.
+
+## Provenance and selection
+
+The locally declared plan is `scripts/external-evaluation-plan.json`. It selected
+all literal valid/invalid cases from `eqeqeq`, `no-dupe-args` and `no-unreachable`
+at ESLint tag `v10.10.0`, resolved to commit
+`3f20a57c6293371b6193d3fb6746c2b7b2ac2689`. The source-file sizes and SHA-256 values,
+including the upstream MIT license, are pinned in
+`scripts/external-evaluation-inputs.json`. The plan digest uses its parsed JSON
+serialized with two-space indentation and a trailing newline, so formatting alone
+does not change its identity. This is a local declaration, not an externally
+witnessed preregistration or proof of an author's prior knowledge.
+
+The source cases and labels come from ESLint's
+[rule tests](https://github.com/eslint/eslint/tree/3f20a57c6293371b6193d3fb6746c2b7b2ac2689/tests/lib/rules)
+and their documented [RuleTester contract](https://eslint.org/docs/latest/integrate/nodejs-api#ruletester).
+Copyright belongs to the OpenJS Foundation and other contributors under the
+[pinned MIT license](https://github.com/eslint/eslint/blob/3f20a57c6293371b6193d3fb6746c2b7b2ac2689/LICENSE).
+Prepared copies stay in a local input directory, include that license and are not
+bundled in the npm package. Reports retain IDs, hashes, labels, diagnostic metadata
+and measurements; they contain no copied case source or raw diagnostic prose.
+The new extraction, measurement and regression-test code is original.
+
+The extractor parses JavaScript syntax with the prepared Espree 11.2.0. It does
+not execute the upstream test modules, invoke their hooks or evaluate fixture
+expressions. It preserves literal RuleTester constructor settings, case language
+settings and rule options. One recognized array transformation supplies a default
+`output: null` without changing source or diagnostic labels; no arbitrary map or
+filter is evaluated. Unsupported cases retain an ID, upstream label and exclusion
+reason. A dynamic list whose cardinality cannot be established stops measurement.
+Excluded cases remain visible in each system's classification denominator.
+
+The baseline and verifier see the same generated files and settings. The baseline
+runs the prepared ESLint 10.10.0 CLI with fixes/cache disabled, explicit configuration
+and both candidate files selected. The verifier uses `javascript.eslint` through
+its shared engine. The configuration file itself has a separate active rule and
+is included in file accounting. Copied dependency trees and source fingerprints
+are checked for changes; inputs, runtime, lockfile and harness identities are
+checked before and after measurement. Nothing is installed during measurement.
+
+## What is checked
+
+Upstream `valid` cases must produce no diagnostics. For an `invalid` case, detection
+requires the intended rule ID, each declared message ID, the declared diagnostic
+locations and the exact diagnostic count. An unrelated failure is `wrong-signal`,
+not a successful detection. A passing invalid case is `missed`; a failing valid
+case is `false-positive`. Incomplete results and excluded inputs are separate
+columns and are not dropped. Every executable case requires one observation for
+each system; missing, duplicate or malformed observations fail the harness.
+
+This does not rerun every RuleTester assertion. Autofixes, suggestions, interpolated
+message wording/data and locations not specified by the upstream case are not
+assessed. It measures diagnostic preservation for the stated profile. Upstream
+unit fixtures are deliberately selected and native ESLint already uses them as
+regressions, so they cannot establish independent native-rule effectiveness.
+
+## Recorded results
+
+The [macOS/Node 26 snapshot](measurements/external-eslint-darwin-arm64-node26.json)
+and [Linux/Node 22 snapshot](measurements/external-eslint-linux-arm64-node22.json)
+have the same corpus, frozen runtime, harness identity and classification matrix.
+
+| Rule             | Cases | Valid cases clean, each system | Invalid cases detected, each system |
+| ---------------- | ----: | -----------------------------: | ----------------------------------: |
+| `eqeqeq`         |    77 |                             31 |                                  46 |
+| `no-dupe-args`   |    13 |                              5 |                                   8 |
+| `no-unreachable` |    70 |                             32 |                                  38 |
+
+Across this fixed cohort there are no false positives, misses, wrong-signal
+failures, incomplete results or exclusions. Both systems agree; this is evidence
+of integration equivalence on these cases, not an improvement over native ESLint.
+No comparison with a previous human/model/private review workflow was performed.
+
+Each case was measured once per system, alternating order between cases. Wall
+time includes process startup and native execution; engine time excludes the outer
+worker startup. Fixture/dependency preparation and cleanup are outside the timed
+operation. Raw observations are retained. These purposive, related unit fixtures
+are not a random population sample, so population confidence intervals, production
+false-positive estimates and representative cost/overhead claims are not justified.
+No model was called; model tokens and provider cost are zero. Human and machine
+costs are not measured.
+
+Linux measurement used Node 22.23.2 in
+`node:22-alpine@sha256:c610fcdfb1d5b4740dd70c284ed3cb16bb857e0f7166196e36a5501df7a3aa32`,
+with networking disabled and prepared inputs/repository mounted read-only.
+Temporary fixtures were writable. macOS used Node 26.8.1. Hosted CI, Windows,
+other native-tool versions and additional rule/language families are separate gates.
+
+## Preserve and reproduce the frozen checkout
+
+The current engine has changed since this measurement: exported guidance and
+route schemas were corrected after an actual client exposed a nonstandard format.
+Those changes do not retroactively update the observations above. The original
+runtime, source, lockfile and original measurement harness were preserved locally
+before that edit. The [preservation record](measurements/external-eslint-preservation.json)
+identifies the archive and its member manifest. The archive contains original
+project code and the MIT license, not upstream cases or installed dependencies.
+It is a local prepared artifact, not a published download.
+
+When the compiled checkout still matches the plan's frozen identities, this helper
+creates a fresh snapshot, archives the selected files, and prepares copies of the
+installed locked dependencies for local replay:
+
+```sh
+node scripts/preserve-external-evaluation.mjs .checktrail > /tmp/frozen-evaluation.json
+```
+
+It intentionally fails on a later runtime before creating the output snapshot.
+Do not edit the freeze pins to make it accept the current engine. Reuse the
+preserved artifact or obtain its original exact bytes when reproducing this
+particular evaluation. A restored archive needs the original locked dependencies
+prepared separately; dependency installation is not part of measurement.
+
+From that frozen checkout, prepare the pinned upstream inputs and run:
+
+```sh
+node scripts/prepare-external-evaluation.mjs /path/to/input-parent > /tmp/external-inputs.json
+external_inputs_path="$(node -p 'JSON.parse(require("node:fs").readFileSync("/tmp/external-inputs.json", "utf8")).directory')"
+node scripts/measure-external-evaluation.mjs "$external_inputs_path" > external-evaluation.json
+node scripts/verify-external-evaluation-container.mjs "$external_inputs_path" > external-evaluation-linux.json
+```
+
+Preparation explicitly downloads only the pinned public files, verifies every
+byte hash before creating a fresh private directory, and retains their license.
+It requires `curl` and does not modify an existing input directory. Measurement
+requires the original locked development dependencies and compiled runtime; it
+never fetches inputs or tools. The container helper additionally requires Docker
+and the specified image already available for offline execution.
+
+Archive extraction was checked against every recorded member digest. Rebuilding
+the preserved source with the locked TypeScript compiler reproduced the original
+runtime digest. The preserved harness was replayed on macOS and Linux: corpus identities, classifications,
+diagnostic metadata and source fingerprints matched the original observations.
+Replays are reproducibility checks, not new independent holdouts or fresh latency
+comparisons. The latest engine's ordinary native regression suite is separate.
+
+The measurement command intentionally rejects a changed verifier runtime or
+lockfile. Reproducing the original observation requires the recorded frozen
+artifacts. After tuning on these cases, evaluate them as known regressions rather
+than claiming a new holdout; declare a new independent cohort for further claims.
+The labels here do not prove semantic equivalence to a previous review process.
+
+Synthetic harness tests cover non-execution, configuration inheritance, exact
+identifiers and locations, incomplete/excluded denominators, malformed evidence,
+privacy projection and independent recomputation of the recorded matrices.
+Removing excluded rows from aggregation made the named denominator regression
+fail; restoring the guard restored the suite.