@stsepelin/checktrail 0.1.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +42 -0
- package/LICENSE +21 -0
- package/README.md +355 -0
- package/SECURITY.md +46 -0
- package/dist/src/actionlint-evidence.d.ts +2 -0
- package/dist/src/actionlint-evidence.js +166 -0
- package/dist/src/actionlint-inputs.d.ts +5 -0
- package/dist/src/actionlint-inputs.js +123 -0
- package/dist/src/actionlint-runner.d.ts +1 -0
- package/dist/src/actionlint-runner.js +122 -0
- package/dist/src/actionlint.d.ts +19 -0
- package/dist/src/actionlint.js +80 -0
- package/dist/src/adapters.d.ts +49 -0
- package/dist/src/adapters.js +430 -0
- package/dist/src/architecture.d.ts +107 -0
- package/dist/src/architecture.js +230 -0
- package/dist/src/clang-evidence.d.ts +2 -0
- package/dist/src/clang-evidence.js +194 -0
- package/dist/src/clang-protocol.d.ts +3 -0
- package/dist/src/clang-protocol.js +25 -0
- package/dist/src/clang-runner.d.ts +1 -0
- package/dist/src/clang-runner.js +159 -0
- package/dist/src/clang.d.ts +44 -0
- package/dist/src/clang.js +197 -0
- package/dist/src/cli.d.ts +2 -0
- package/dist/src/cli.js +338 -0
- package/dist/src/config.d.ts +33 -0
- package/dist/src/config.js +140 -0
- package/dist/src/contract-schema.d.ts +85 -0
- package/dist/src/contract-schema.js +56 -0
- package/dist/src/contract-worker.d.ts +1 -0
- package/dist/src/contract-worker.js +123 -0
- package/dist/src/contracts.d.ts +22 -0
- package/dist/src/contracts.js +123 -0
- package/dist/src/django-evidence.d.ts +2 -0
- package/dist/src/django-evidence.js +100 -0
- package/dist/src/django-runner.d.ts +1 -0
- package/dist/src/django-runner.js +80 -0
- package/dist/src/django.d.ts +9 -0
- package/dist/src/django.js +50 -0
- package/dist/src/dotnet-compiler.d.ts +1 -0
- package/dist/src/dotnet-compiler.js +87 -0
- package/dist/src/dotnet-evidence.d.ts +2 -0
- package/dist/src/dotnet-evidence.js +127 -0
- package/dist/src/dotnet-runner.d.ts +1 -0
- package/dist/src/dotnet-runner.js +167 -0
- package/dist/src/dotnet.d.ts +65 -0
- package/dist/src/dotnet.js +120 -0
- package/dist/src/engine.d.ts +19 -0
- package/dist/src/engine.js +239 -0
- package/dist/src/environment.d.ts +6 -0
- package/dist/src/environment.js +36 -0
- package/dist/src/eslint-evidence.d.ts +2 -0
- package/dist/src/eslint-evidence.js +98 -0
- package/dist/src/eslint-runner.d.ts +1 -0
- package/dist/src/eslint-runner.js +61 -0
- package/dist/src/eslint.d.ts +2 -0
- package/dist/src/eslint.js +40 -0
- package/dist/src/evidence.d.ts +2 -0
- package/dist/src/evidence.js +313 -0
- package/dist/src/external-adapter.d.ts +164 -0
- package/dist/src/external-adapter.js +265 -0
- package/dist/src/external-evidence.d.ts +2 -0
- package/dist/src/external-evidence.js +127 -0
- package/dist/src/external-runner.d.ts +1 -0
- package/dist/src/external-runner.js +122 -0
- package/dist/src/fastapi-evidence.d.ts +2 -0
- package/dist/src/fastapi-evidence.js +126 -0
- package/dist/src/fastapi-runner.d.ts +1 -0
- package/dist/src/fastapi-runner.js +84 -0
- package/dist/src/fastapi.d.ts +10 -0
- package/dist/src/fastapi.js +52 -0
- package/dist/src/fetch-pack.d.ts +16 -0
- package/dist/src/fetch-pack.js +154 -0
- package/dist/src/finding-policy-schema.d.ts +146 -0
- package/dist/src/finding-policy-schema.js +70 -0
- package/dist/src/finding-policy.d.ts +62 -0
- package/dist/src/finding-policy.js +206 -0
- package/dist/src/git-selection.d.ts +3 -0
- package/dist/src/git-selection.js +229 -0
- package/dist/src/go-directives.d.ts +1 -0
- package/dist/src/go-directives.js +32 -0
- package/dist/src/go-scope.d.ts +11 -0
- package/dist/src/go-scope.js +122 -0
- package/dist/src/golangci-evidence.d.ts +2 -0
- package/dist/src/golangci-evidence.js +92 -0
- package/dist/src/golangci-runner.d.ts +1 -0
- package/dist/src/golangci-runner.js +124 -0
- package/dist/src/golangci.d.ts +2 -0
- package/dist/src/golangci.js +35 -0
- package/dist/src/guidance.d.ts +98 -0
- package/dist/src/guidance.js +247 -0
- package/dist/src/index.d.ts +31 -0
- package/dist/src/index.js +16 -0
- package/dist/src/inventory.d.ts +4 -0
- package/dist/src/inventory.js +94 -0
- package/dist/src/java-compiler.d.ts +1 -0
- package/dist/src/java-compiler.js +110 -0
- package/dist/src/java-evidence.d.ts +2 -0
- package/dist/src/java-evidence.js +114 -0
- package/dist/src/java-runner.d.ts +1 -0
- package/dist/src/java-runner.js +79 -0
- package/dist/src/java.d.ts +25 -0
- package/dist/src/java.js +102 -0
- package/dist/src/jest-evidence.d.ts +2 -0
- package/dist/src/jest-evidence.js +117 -0
- package/dist/src/jest-runner.d.ts +1 -0
- package/dist/src/jest-runner.js +46 -0
- package/dist/src/jest.d.ts +2 -0
- package/dist/src/jest.js +36 -0
- package/dist/src/junit.d.ts +18 -0
- package/dist/src/junit.js +157 -0
- package/dist/src/laravel-evidence.d.ts +2 -0
- package/dist/src/laravel-evidence.js +175 -0
- package/dist/src/laravel-runner.d.ts +1 -0
- package/dist/src/laravel-runner.js +168 -0
- package/dist/src/laravel.d.ts +8 -0
- package/dist/src/laravel.js +50 -0
- package/dist/src/local-tool.d.ts +1 -0
- package/dist/src/local-tool.js +19 -0
- package/dist/src/mcp.d.ts +14 -0
- package/dist/src/mcp.js +362 -0
- package/dist/src/mutation.d.ts +118 -0
- package/dist/src/mutation.js +370 -0
- package/dist/src/mypy-evidence.d.ts +2 -0
- package/dist/src/mypy-evidence.js +63 -0
- package/dist/src/mypy.d.ts +2 -0
- package/dist/src/mypy.js +59 -0
- package/dist/src/node-reporter.d.ts +1 -0
- package/dist/src/node-reporter.js +28 -0
- package/dist/src/nuxt-evidence.d.ts +2 -0
- package/dist/src/nuxt-evidence.js +160 -0
- package/dist/src/nuxt-protocol.d.ts +52 -0
- package/dist/src/nuxt-protocol.js +29 -0
- package/dist/src/nuxt-runner.d.ts +1 -0
- package/dist/src/nuxt-runner.js +165 -0
- package/dist/src/nuxt.d.ts +24 -0
- package/dist/src/nuxt.js +105 -0
- package/dist/src/output.d.ts +3 -0
- package/dist/src/output.js +48 -0
- package/dist/src/phpstan-evidence.d.ts +2 -0
- package/dist/src/phpstan-evidence.js +81 -0
- package/dist/src/phpstan.d.ts +2 -0
- package/dist/src/phpstan.js +51 -0
- package/dist/src/phpunit-evidence.d.ts +2 -0
- package/dist/src/phpunit-evidence.js +53 -0
- package/dist/src/phpunit.d.ts +2 -0
- package/dist/src/phpunit.js +58 -0
- package/dist/src/pint-evidence.d.ts +2 -0
- package/dist/src/pint-evidence.js +79 -0
- package/dist/src/pint-runner.d.ts +1 -0
- package/dist/src/pint-runner.js +53 -0
- package/dist/src/pint.d.ts +2 -0
- package/dist/src/pint.js +37 -0
- package/dist/src/playwright-evidence.d.ts +2 -0
- package/dist/src/playwright-evidence.js +127 -0
- package/dist/src/playwright-reporter.d.ts +10 -0
- package/dist/src/playwright-reporter.js +37 -0
- package/dist/src/playwright-runner.d.ts +1 -0
- package/dist/src/playwright-runner.js +61 -0
- package/dist/src/playwright.d.ts +2 -0
- package/dist/src/playwright.js +35 -0
- package/dist/src/policy-pack.d.ts +21 -0
- package/dist/src/policy-pack.js +41 -0
- package/dist/src/pytest-evidence.d.ts +2 -0
- package/dist/src/pytest-evidence.js +116 -0
- package/dist/src/pytest-runner.d.ts +1 -0
- package/dist/src/pytest-runner.js +48 -0
- package/dist/src/pytest.d.ts +2 -0
- package/dist/src/pytest.js +25 -0
- package/dist/src/report-validation.d.ts +129 -0
- package/dist/src/report-validation.js +27 -0
- package/dist/src/review.d.ts +270 -0
- package/dist/src/review.js +349 -0
- package/dist/src/ruby.d.ts +2 -0
- package/dist/src/ruby.js +25 -0
- package/dist/src/ruff-evidence.d.ts +2 -0
- package/dist/src/ruff-evidence.js +99 -0
- package/dist/src/ruff.d.ts +2 -0
- package/dist/src/ruff.js +40 -0
- package/dist/src/runner.d.ts +8 -0
- package/dist/src/runner.js +117 -0
- package/dist/src/runtime-inventory.d.ts +112 -0
- package/dist/src/runtime-inventory.js +200 -0
- package/dist/src/rust-dep-info.d.ts +1 -0
- package/dist/src/rust-dep-info.js +46 -0
- package/dist/src/rust-evidence.d.ts +2 -0
- package/dist/src/rust-evidence.js +157 -0
- package/dist/src/rust-runner.d.ts +1 -0
- package/dist/src/rust-runner.js +161 -0
- package/dist/src/rust.d.ts +7 -0
- package/dist/src/rust.js +39 -0
- package/dist/src/sarif.d.ts +86 -0
- package/dist/src/sarif.js +113 -0
- package/dist/src/schemas.d.ts +393 -0
- package/dist/src/schemas.js +222 -0
- package/dist/src/staticcheck-evidence.d.ts +2 -0
- package/dist/src/staticcheck-evidence.js +75 -0
- package/dist/src/swift.d.ts +2 -0
- package/dist/src/swift.js +20 -0
- package/dist/src/task-store.d.ts +37 -0
- package/dist/src/task-store.js +349 -0
- package/dist/src/tool-versions.d.ts +3 -0
- package/dist/src/tool-versions.js +260 -0
- package/dist/src/types.d.ts +157 -0
- package/dist/src/types.js +37 -0
- package/dist/src/typescript-build-evidence.d.ts +2 -0
- package/dist/src/typescript-build-evidence.js +86 -0
- package/dist/src/typescript-build-runner.d.ts +1 -0
- package/dist/src/typescript-build-runner.js +132 -0
- package/dist/src/typescript-build.d.ts +2 -0
- package/dist/src/typescript-build.js +41 -0
- package/dist/src/typescript.d.ts +2 -0
- package/dist/src/typescript.js +54 -0
- package/dist/src/validation-task-worker.d.ts +1 -0
- package/dist/src/validation-task-worker.js +205 -0
- package/dist/src/validation-tasks-protocol.d.ts +69 -0
- package/dist/src/validation-tasks-protocol.js +57 -0
- package/dist/src/validation-tasks.d.ts +11 -0
- package/dist/src/validation-tasks.js +144 -0
- package/dist/src/vitest-evidence.d.ts +2 -0
- package/dist/src/vitest-evidence.js +112 -0
- package/dist/src/vitest-runner.d.ts +1 -0
- package/dist/src/vitest-runner.js +48 -0
- package/dist/src/vitest.d.ts +2 -0
- package/dist/src/vitest.js +36 -0
- package/dist/src/vue-router-capture.d.ts +42 -0
- package/dist/src/vue-router-capture.js +87 -0
- package/dist/src/vue-router-evidence.d.ts +2 -0
- package/dist/src/vue-router-evidence.js +126 -0
- package/dist/src/vue-router-protocol.d.ts +63 -0
- package/dist/src/vue-router-protocol.js +44 -0
- package/dist/src/vue-router-runner.d.ts +1 -0
- package/dist/src/vue-router-runner.js +88 -0
- package/dist/src/vue-router.d.ts +23 -0
- package/dist/src/vue-router.js +108 -0
- package/dist/src/vue-tsc-runner.d.ts +1 -0
- package/dist/src/vue-tsc-runner.js +24 -0
- package/dist/src/workspace.d.ts +7 -0
- package/dist/src/workspace.js +53 -0
- package/docs/ACCEPTANCE.md +56 -0
- package/docs/ACTIONLINT.md +112 -0
- package/docs/ARCHITECTURE-POLICY.md +82 -0
- package/docs/ARCHITECTURE.md +122 -0
- package/docs/CLANG.md +113 -0
- package/docs/CLIENTS.md +109 -0
- package/docs/CONTRACTS.md +73 -0
- package/docs/DEPENDENCIES.md +42 -0
- package/docs/DJANGO.md +70 -0
- package/docs/DOTNET.md +114 -0
- package/docs/ENVIRONMENTS.md +52 -0
- package/docs/ESLINT.md +68 -0
- package/docs/EVALUATION.md +124 -0
- package/docs/EXECUTION.md +250 -0
- package/docs/EXTERNAL-ADAPTERS.md +170 -0
- package/docs/EXTERNAL-EVALUATION.md +155 -0
- package/docs/EXTERNAL-RUFF-EVALUATION.md +85 -0
- package/docs/FASTAPI.md +73 -0
- package/docs/FINDING-POLICY.md +67 -0
- package/docs/GO-RACE.md +19 -0
- package/docs/GO-SCOPE.md +63 -0
- package/docs/GOLANGCI-LINT.md +64 -0
- package/docs/GUIDANCE.md +46 -0
- package/docs/IMPACT-MEASUREMENT.md +97 -0
- package/docs/INSTALLATION.md +102 -0
- package/docs/JAVA.md +79 -0
- package/docs/JEST.md +36 -0
- package/docs/JUNIT.md +31 -0
- package/docs/LANGUAGES.md +113 -0
- package/docs/LARAVEL.md +111 -0
- package/docs/MCP-COMPATIBILITY.md +68 -0
- package/docs/MUTATIONS.md +93 -0
- package/docs/MYPY.md +37 -0
- package/docs/NATIVE-CI.md +96 -0
- package/docs/NUXT.md +123 -0
- package/docs/PACK-DISTRIBUTION.md +98 -0
- package/docs/PERFORMANCE.md +71 -0
- package/docs/PEST.md +36 -0
- package/docs/PHPSTAN.md +48 -0
- package/docs/PHPUNIT.md +32 -0
- package/docs/PINT.md +41 -0
- package/docs/PLAN.md +265 -0
- package/docs/PLAYWRIGHT.md +73 -0
- package/docs/POLICY-PACKS.md +106 -0
- package/docs/PRIOR-WORKFLOW-EVALUATION.md +116 -0
- package/docs/PUBLICATION.md +28 -0
- package/docs/PYTEST.md +51 -0
- package/docs/RELEASE.md +115 -0
- package/docs/RENAMING.md +55 -0
- package/docs/REVIEW-EXCHANGE.md +136 -0
- package/docs/RUBY.md +50 -0
- package/docs/RUFF.md +39 -0
- package/docs/RUNTIME-INVENTORY.md +56 -0
- package/docs/RUST.md +74 -0
- package/docs/SARIF.md +64 -0
- package/docs/STATUS.md +530 -0
- package/docs/SWIFT.md +48 -0
- package/docs/TASK-STORAGE.md +141 -0
- package/docs/TOOL-IDENTITY.md +30 -0
- package/docs/TYPESCRIPT-BUILD.md +55 -0
- package/docs/VALIDATION-TASKS.md +119 -0
- package/docs/VITEST.md +36 -0
- package/docs/VUE-ROUTER.md +130 -0
- package/docs/VUE-TSC.md +26 -0
- package/docs/WORKSPACES.md +81 -0
- package/docs/licenses/nodable-entities-3.0.0.txt +21 -0
- package/docs/measurements/client-claude-before-schema-fix-darwin-arm64-node26.json +60 -0
- package/docs/measurements/client-claude-darwin-arm64-node26.json +55 -0
- package/docs/measurements/client-codex-darwin-arm64-node26.json +175 -0
- package/docs/measurements/evaluation-corpus-repo-verifier.json +332 -0
- package/docs/measurements/evaluation-darwin-arm64-node26.json +1236 -0
- package/docs/measurements/evaluation-linux-arm64-node22.json +1236 -0
- package/docs/measurements/external-eslint-darwin-arm64-node26.json +11866 -0
- package/docs/measurements/external-eslint-linux-arm64-node22.json +11866 -0
- package/docs/measurements/external-eslint-preservation.json +1527 -0
- package/docs/measurements/external-ruff-darwin-arm64-node26.json +1031 -0
- package/docs/measurements/external-ruff-linux-arm64-node22.json +1031 -0
- package/docs/measurements/hosted-ci-52ba415.json +1005 -0
- package/docs/measurements/impact-darwin-arm64-node26.json +6685 -0
- package/docs/measurements/impact-linux-arm64-node22.json +6690 -0
- package/docs/measurements/performance-darwin-arm64-node26.json +845 -0
- package/docs/measurements/performance-linux-arm64-node22.json +845 -0
- package/docs/measurements/required-native-profiles.json +500 -0
- package/package.json +72 -0
- package/packs/actionlint.json +8 -0
- package/packs/cpp.json +8 -0
- package/packs/dotnet.json +8 -0
- package/packs/go.json +7 -0
- package/packs/java.json +8 -0
- package/packs/javascript-node.json +7 -0
- package/packs/nuxt.json +7 -0
- package/packs/php-pest.json +7 -0
- package/packs/php-phpunit.json +7 -0
- package/packs/python-pytest.json +7 -0
- package/packs/ruby.json +7 -0
- package/packs/rust.json +7 -0
- package/packs/swift.json +7 -0
- package/packs/vue-router.json +7 -0
- package/packs/vue.json +11 -0
- package/schemas/actionlint-config.schema.json +19 -0
- package/schemas/architecture-policy.schema.json +51 -0
- package/schemas/architecture-report.schema.json +123 -0
- package/schemas/architecture-summary.schema.json +82 -0
- package/schemas/clang-database.schema.json +23 -0
- package/schemas/config.schema.json +69 -0
- package/schemas/contract-bundle.schema.json +78 -0
- package/schemas/contract-report.schema.json +114 -0
- package/schemas/contract-summary.schema.json +72 -0
- package/schemas/dependency-graph.schema.json +65 -0
- package/schemas/django-config.schema.json +16 -0
- package/schemas/dotnet-config.schema.json +60 -0
- package/schemas/external-manifest.schema.json +97 -0
- package/schemas/external-reference.schema.json +10 -0
- package/schemas/external-request.schema.json +56 -0
- package/schemas/external-result.schema.json +111 -0
- package/schemas/fastapi-config.schema.json +27 -0
- package/schemas/finding-baseline.schema.json +92 -0
- package/schemas/finding-comparison-summary.schema.json +95 -0
- package/schemas/finding-comparison.schema.json +121 -0
- package/schemas/guidance-context.schema.json +34 -0
- package/schemas/guidance-report.schema.json +176 -0
- package/schemas/guidance-summary.schema.json +68 -0
- package/schemas/java-config.schema.json +24 -0
- package/schemas/junit.schema.json +69 -0
- package/schemas/laravel-config.schema.json +11 -0
- package/schemas/mutation-recipe.schema.json +31 -0
- package/schemas/mutation-report.schema.json +238 -0
- package/schemas/mutation-summary.schema.json +98 -0
- package/schemas/nuxt-config.schema.json +51 -0
- package/schemas/plan-summary.schema.json +63 -0
- package/schemas/plan.schema.json +244 -0
- package/schemas/policy-pack.schema.json +41 -0
- package/schemas/report-summary.schema.json +100 -0
- package/schemas/report.schema.json +463 -0
- package/schemas/review-assessment.schema.json +133 -0
- package/schemas/review-context-summary.schema.json +31 -0
- package/schemas/review-context.schema.json +254 -0
- package/schemas/review-receipt-summary.schema.json +104 -0
- package/schemas/review-receipt.schema.json +271 -0
- package/schemas/review-selection.schema.json +30 -0
- package/schemas/runtime-comparison-summary.schema.json +68 -0
- package/schemas/runtime-comparison.schema.json +127 -0
- package/schemas/runtime-inventory.schema.json +99 -0
- package/schemas/vue-router-config.schema.json +67 -0
- package/server.json +47 -0
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Required native CI evidence
|
|
2
|
+
|
|
3
|
+
The ordinary test suite permits explicit skips for unavailable optional tools.
|
|
4
|
+
That makes it useful on a developer machine, but its aggregate exit code cannot
|
|
5
|
+
establish that a native adapter ran. CI's prepared profiles additionally use
|
|
6
|
+
`node scripts/verify-required-native-tests.mjs PROFILE`.
|
|
7
|
+
|
|
8
|
+
The [profile manifest](../scripts/required-native-tests.json) lists exact test
|
|
9
|
+
names and files. The runner requires each listed test to pass exactly once,
|
|
10
|
+
rejects any failure, skip or TODO in the selected files, and distinguishes a
|
|
11
|
+
passing suite from a passing test. A similarly named test, a test in another
|
|
12
|
+
file, or a passing unit test cannot stand in for a missing native regression.
|
|
13
|
+
Expected names are fixed inputs; the runner does not discover its requirements
|
|
14
|
+
from whichever tests happen to remain in the source tree.
|
|
15
|
+
|
|
16
|
+
The main CI matrix prepares and requires `core`, `javascript`, `python`,
|
|
17
|
+
`frameworks`, `go`, `php-tools`, `laravel` and `rust`. Dedicated container jobs
|
|
18
|
+
require `clang`, `java`, `dotnet`, `actionlint`, `vue-router` and `nuxt`.
|
|
19
|
+
Ruby and Swift have dedicated required-native jobs. The packaged review and
|
|
20
|
+
durable-task helpers require `review` and `tasks` before their installed-package
|
|
21
|
+
checks. The separate PHP syntax helper checks its exact successful TAP test name;
|
|
22
|
+
the external-adapter helper already checks exact required native names for each
|
|
23
|
+
of its different runtime containers.
|
|
24
|
+
|
|
25
|
+
A tool version preflight remains useful but is not the acceptance condition.
|
|
26
|
+
Installing the expected binary cannot compensate for a skipped, renamed, removed
|
|
27
|
+
or failing required test. When intentionally renaming or replacing a regression,
|
|
28
|
+
review its behavior and update the manifest with that change. Do not remove a
|
|
29
|
+
requirement merely to make an unavailable toolchain green.
|
|
30
|
+
|
|
31
|
+
The native profiles supplement the full suite; they do not replace linting,
|
|
32
|
+
type checking, parser tests, privacy tests or package verification. Some selected
|
|
33
|
+
files contain both native and unit cases, so `passed` counts in their summaries
|
|
34
|
+
are test counts, not native-tool invocations. Profiles can overlap with the full
|
|
35
|
+
suite. Do not add their pass counts together as independent coverage.
|
|
36
|
+
|
|
37
|
+
Container helpers retain their existing prepared dependencies, network-disabled
|
|
38
|
+
execution and read-only source mounts. They do not install tools while measuring.
|
|
39
|
+
A main hosted job prepares dependencies before validation. Local container runs
|
|
40
|
+
are evidence for their actual versions and platforms only; the GitHub Actions
|
|
41
|
+
matrix requires its own successful run before being claimed as verified.
|
|
42
|
+
|
|
43
|
+
The runner's regression tests exercise required-test absence, exact name/file
|
|
44
|
+
identity, duplicates, skips, TODOs, suite-only matches and unrelated failures.
|
|
45
|
+
Deleting its missing-test rejection makes the absent-test case fail. To diagnose
|
|
46
|
+
a native assertion failure, rerun the named file directly with `node --test` in
|
|
47
|
+
the same prepared runtime for the full assertion diagnostics.
|
|
48
|
+
|
|
49
|
+
The [recorded local profile runs](measurements/required-native-profiles.json) retain
|
|
50
|
+
requirement/helper digests, result accounting and available container/package
|
|
51
|
+
identities for the expanded profiles. Each listed requirement passed in those
|
|
52
|
+
runs. The unchanged Ruby/Swift paths have their separate local evidence in
|
|
53
|
+
`RUBY.md` and `SWIFT.md`. These records do not claim that the hosted matrix ran,
|
|
54
|
+
that all profiles ran in a single environment, or that overlapping counts are
|
|
55
|
+
independent tests.
|
|
56
|
+
|
|
57
|
+
## Fresh-runner package preparation
|
|
58
|
+
|
|
59
|
+
`npm ci` installs from the repository lockfile but does not necessarily cache
|
|
60
|
+
the registry metadata needed to resolve a new consumer's dependencies. Before
|
|
61
|
+
packaged checks, CI runs `node scripts/prepare-package-cache.mjs` with network
|
|
62
|
+
access. It packs the current build and installs it in a disposable consumer with
|
|
63
|
+
lifecycle scripts disabled, using the same npm cache as the later smoke helpers.
|
|
64
|
+
The disposable installation is removed; only the npm cache is reused.
|
|
65
|
+
|
|
66
|
+
The smoke helpers still create fresh consumers and install with `--offline`.
|
|
67
|
+
Native container checks still use `--network none`. Cache preparation is setup,
|
|
68
|
+
not evidence that a package can be installed without previously cached dependencies.
|
|
69
|
+
For local reproduction, run `npm run build`, the cache preparation script, then
|
|
70
|
+
`node scripts/smoke-package.mjs`, keeping the same npm cache configuration.
|
|
71
|
+
|
|
72
|
+
The [first hosted run](https://github.com/stsepelin/checktrail/actions/runs/35570317427)
|
|
73
|
+
exposed this missing setup: Java, C#, actionlint, external adapters, Vue Router and
|
|
74
|
+
Nuxt reached packaged installation after their native checks, then failed with
|
|
75
|
+
`ENOTCACHED`. All three Linux main-matrix jobs reached the same failure after
|
|
76
|
+
passing their full suites and required native profiles. Clang, Ruby and Swift
|
|
77
|
+
jobs passed. The macOS main job failed separately because Pint attempted to seek
|
|
78
|
+
within `/dev/null`; the adapter now uses a fresh regular cache file in a
|
|
79
|
+
runner-owned directory, with cleanup asserted after successful and failed checks.
|
|
80
|
+
See `PINT.md`. A later macOS assertion also needed canonical rather than aliased
|
|
81
|
+
temporary paths; its fixture now exercises a symlinked root on every platform.
|
|
82
|
+
All three fixes are included in the successful run below.
|
|
83
|
+
|
|
84
|
+
## Successful hosted baseline
|
|
85
|
+
|
|
86
|
+
[Run 35573066804](https://github.com/stsepelin/checktrail/actions/runs/35573066804)
|
|
87
|
+
passed all 13 jobs at commit `52ba415dfc5cfcaab459dc648baaa864627f1642`.
|
|
88
|
+
The [job and step record](measurements/hosted-ci-52ba415.json) retains the source
|
|
89
|
+
identity, run URLs, outcomes and timestamps. The main matrix covered Linux Node
|
|
90
|
+
22/24/26 and macOS Node 24; dedicated jobs covered Java, C#, Clang, actionlint,
|
|
91
|
+
external adapters, Vue Router, Nuxt, Ruby and Swift.
|
|
92
|
+
|
|
93
|
+
This establishes the configured profiles for that revision and those hosted
|
|
94
|
+
environments. Optional skips in the general suite still do not count as native
|
|
95
|
+
coverage. The `0.1.0-alpha.1` release metadata and documentation follow this
|
|
96
|
+
baseline; the release commit needs its own CI run before publication.
|
package/docs/NUXT.md
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# Nuxt SSR testing assembly
|
|
2
|
+
|
|
3
|
+
`javascript.nuxt-runtime` builds a fresh Nuxt SSR testing assembly and sends
|
|
4
|
+
explicitly declared page requests through Nitro's in-process `localFetch`. An
|
|
5
|
+
injected server-only plugin observes the actual Nuxt router at `app:rendered`,
|
|
6
|
+
after plugins, initial navigation and rendering. The hook retains a reader; the
|
|
7
|
+
collector invokes it only after the in-process request completes, including later
|
|
8
|
+
awaited render hooks. This includes generated page
|
|
9
|
+
routes and runtime registration; it does not reconstruct routes from filenames.
|
|
10
|
+
No listening socket or external infrastructure is started by the collector.
|
|
11
|
+
|
|
12
|
+
This experimental profile is version-gated to Nuxt/Kit/Vite-builder/Nitro-server
|
|
13
|
+
4.5.2, Nitro 2.13.4, Vue 3.5.43 and Vue Router 5.3.1. Dependencies must be prepared
|
|
14
|
+
inside the operator root. Fixture tooling has a separate lockfile in
|
|
15
|
+
`scripts/nuxt-tools`; the production package does not install or redistribute that
|
|
16
|
+
framework toolchain. Package metadata identifies tools, not an attestation of
|
|
17
|
+
unaltered installed bytes or bundler aliases.
|
|
18
|
+
|
|
19
|
+
## Request contract
|
|
20
|
+
|
|
21
|
+
Select `javascript.nuxt-runtime` explicitly and add `checktrail.nuxt.json`:
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{
|
|
25
|
+
"schemaVersion": 1,
|
|
26
|
+
"assembly": "example.catalog",
|
|
27
|
+
"environment": "test",
|
|
28
|
+
"probes": [{ "path": "/", "matched": [{ "path": "/", "name": "index" }] }]
|
|
29
|
+
}
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
The project must contain exactly one inventoried `nuxt.config.js`, `.ts`, `.mjs`,
|
|
33
|
+
`.mts`, `.cjs` or `.cts`. Planning never loads it. Execution requires operator
|
|
34
|
+
trust. Configuration, modules, Vite/Nitro plugins, server handlers, middleware,
|
|
35
|
+
SSR plugins and page rendering execute project code with the user's privileges.
|
|
36
|
+
They may perform their own I/O; the execution allowlist is not a sandbox.
|
|
37
|
+
|
|
38
|
+
Each probe expects HTTP 200 and an exact ordered matched chain. Names may be null.
|
|
39
|
+
At least one probe and one matched record per probe are required; at most 16
|
|
40
|
+
unique request paths and 32 records per chain are allowed. Every final native
|
|
41
|
+
route record, including aliases, must participate in a declared SSR request.
|
|
42
|
+
Negative HTTP status contracts, redirects, client-only navigation and browser
|
|
43
|
+
hydration are outside this profile. An unexpected HTTP status is a failed finding;
|
|
44
|
+
missing capture remains incomplete even when HTTP 200 was returned.
|
|
45
|
+
|
|
46
|
+
Every request gets a new Nuxt application through the built SSR renderer. The
|
|
47
|
+
ordered route projection must remain identical across requests. Request-dependent
|
|
48
|
+
registration is reported as incomplete rather than merged into an invented
|
|
49
|
+
single assembly. Record indices, signatures, counts, matched identities and
|
|
50
|
+
source identity are reconciled by the evidence parser. Empty, partial or malformed
|
|
51
|
+
evidence cannot pass. Known request/route failures remain failed even when capture
|
|
52
|
+
is also incomplete, with `findingsComplete: false`.
|
|
53
|
+
|
|
54
|
+
## Assembly and projection boundaries
|
|
55
|
+
|
|
56
|
+
The collector sets `NODE_ENV=test`, clears `NODE_OPTIONS`, disables Nuxt telemetry
|
|
57
|
+
and devtools, and selects non-development SSR with the native Vite builder and
|
|
58
|
+
Nitro's `nitro-prerender` preset. Nuxt dotenv loading is disabled. It uses a fresh
|
|
59
|
+
canonical temporary build/output/cache directory owned by the shared runner;
|
|
60
|
+
normal completion, timeout and cancellation remove that directory. Nuxt build-cache
|
|
61
|
+
reuse, Chrome DevTools project metadata and the experimental TypeScript plugin
|
|
62
|
+
are disabled. Project code can still
|
|
63
|
+
read local files or change settings; this is a selected testing assembly, not a
|
|
64
|
+
proof of production equivalence. The internal Nitro preset is pinned and tested.
|
|
65
|
+
|
|
66
|
+
Probes use the synthetic host `checktrail.invalid`, without supplied credentials
|
|
67
|
+
or cookies. Server route rules or handlers may intercept requests; a response
|
|
68
|
+
without the instrumented SSR capture cannot establish route coverage. The check
|
|
69
|
+
executes the selected SSR behavior but does not prove that middleware permissions,
|
|
70
|
+
rendered content, data access or monetary calculations are correct. It does not
|
|
71
|
+
validate all possible route parameters, browser state or deployment presets.
|
|
72
|
+
|
|
73
|
+
Route entries use the [Vue Router projection](VUE-ROUTER.md): path/name, alias
|
|
74
|
+
identity, view keys, bounded metadata/static redirect data, guard function names,
|
|
75
|
+
child counts and global matcher flags. Function bodies and component behavior are
|
|
76
|
+
not serialized. Unsupported data makes the collection incomplete. Captures are
|
|
77
|
+
bounded to 2,048 native records, with shared process time/output limits. The planned
|
|
78
|
+
scope identifies the Nuxt configuration entry; this is not per-source type checking,
|
|
79
|
+
linting or an assertion that every source file was included by the build.
|
|
80
|
+
|
|
81
|
+
Detailed reports retain the runtime inventory and request paths. Summary output
|
|
82
|
+
omits them. Source fingerprints cover the engine inventory and retain its usual
|
|
83
|
+
limits for dependencies, external services and ignored files. Runtime comparison
|
|
84
|
+
can review changes in the captured projection under equivalent settings.
|
|
85
|
+
|
|
86
|
+
## Reproduction
|
|
87
|
+
|
|
88
|
+
Prepare the separate development fixture installation explicitly:
|
|
89
|
+
|
|
90
|
+
```sh
|
|
91
|
+
mkdir -p .checktrail/nuxt-tools
|
|
92
|
+
cp scripts/nuxt-tools/package.json scripts/nuxt-tools/package-lock.json .checktrail/nuxt-tools/
|
|
93
|
+
npm ci --prefix .checktrail/nuxt-tools --ignore-scripts --no-audit --no-fund
|
|
94
|
+
npm run build
|
|
95
|
+
node --test dist/test/nuxt.test.js
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
`node scripts/verify-nuxt-container.mjs` runs the native tests and a fresh offline
|
|
99
|
+
package installation through library, CLI and MCP in a pinned Node 22 Alpine
|
|
100
|
+
container with networking disabled and read-only source mounts. On macOS it needs
|
|
101
|
+
a second prepared installation in `.checktrail/nuxt-linux-tools`, installed by
|
|
102
|
+
that Linux image from the same lockfile, because native compiler bindings differ.
|
|
103
|
+
On Linux it uses `.checktrail/nuxt-tools`. The helper requires Docker and the
|
|
104
|
+
pinned image. `examples/nuxt` contains only original synthetic fixtures.
|
|
105
|
+
|
|
106
|
+
Native regressions cover generated pages, asynchronous runtime registration,
|
|
107
|
+
broken/fixed route contracts, uncovered records, HTTP failure, request-dependent
|
|
108
|
+
assemblies, unsupported metadata, parser accounting and cancellation observed
|
|
109
|
+
while startup is actually running. A lifecycle regression first reproduced a false
|
|
110
|
+
pass when a later awaited `app:rendered` hook added an uncovered route; deferring
|
|
111
|
+
the native read until request completion makes that case incomplete. The read-only
|
|
112
|
+
package fixture also verifies that Chrome DevTools metadata is not written into
|
|
113
|
+
the project dependency directory. Verification status belongs in `STATUS.md`;
|
|
114
|
+
a defined helper or CI job is not evidence that it ran.
|
|
115
|
+
|
|
116
|
+
A manual mutation removed the complete-record participation guard. The native
|
|
117
|
+
uncovered-route regression failed because the result became passed instead of
|
|
118
|
+
incomplete. Restoring the original evaluator restored the passing regression;
|
|
119
|
+
subsequent container/package checks used the restored code.
|
|
120
|
+
|
|
121
|
+
References: [Nuxt hooks](https://nuxt.com/docs/4.x/api/advanced/hooks),
|
|
122
|
+
[Nuxt programmatic APIs](https://nuxt.com/docs/4.x/api/kit/programmatic).
|
|
123
|
+
The pinned installed framework and native tests determine this profile's behavior.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Pinned policy pack downloads
|
|
2
|
+
|
|
3
|
+
`fetch-pack` explicitly downloads one data-only JSON policy pack from an
|
|
4
|
+
operator-supplied HTTPS endpoint. It verifies the exact bytes against a separately
|
|
5
|
+
trusted SHA-256 digest and validates the policy before making a local file
|
|
6
|
+
available. Planning, validation and MCP do not download packs automatically.
|
|
7
|
+
|
|
8
|
+
```sh
|
|
9
|
+
checktrail fetch-pack --root /path/to/project \
|
|
10
|
+
--url https://policies.example.org/checks/core-1.0.0.json \
|
|
11
|
+
--sha256 REPLACE_WITH_TRUSTED_64_CHARACTER_SHA256 \
|
|
12
|
+
--output policies/core-1.0.0.json
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
This fictional URL and digest placeholder must be replaced. The parent directory
|
|
16
|
+
must already exist under the selected root. The destination must be a normalized
|
|
17
|
+
relative `.json` path; symbolic-link directories and existing destinations are
|
|
18
|
+
rejected. The command returns pack identity, byte count and a `reference` object
|
|
19
|
+
that can be added to a project's `packs` array in `checktrail.json`. It does not
|
|
20
|
+
edit project policy or activate the downloaded checks.
|
|
21
|
+
|
|
22
|
+
Library consumers can call:
|
|
23
|
+
|
|
24
|
+
```js
|
|
25
|
+
import { fetchPolicyPack } from "@stsepelin/checktrail";
|
|
26
|
+
|
|
27
|
+
const downloaded = await fetchPolicyPack(projectRoot, {
|
|
28
|
+
url: endpoint,
|
|
29
|
+
sha256: expectedDigest,
|
|
30
|
+
output: "policies/core-1.0.0.json",
|
|
31
|
+
timeoutMs: 30_000,
|
|
32
|
+
signal: controller.signal,
|
|
33
|
+
});
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The digest establishes content integrity, not authorship. Obtain it through a
|
|
37
|
+
trusted release or review process independent of the endpoint being checked.
|
|
38
|
+
Updates require a new download destination and an explicit policy reference
|
|
39
|
+
change. Unknown or inapplicable check IDs are rejected when the project resolves
|
|
40
|
+
the downloaded pack, since downloading alone has no execution context.
|
|
41
|
+
|
|
42
|
+
## Transport and storage
|
|
43
|
+
|
|
44
|
+
The downloader sends a GET to the exact configured HTTPS endpoint, without a
|
|
45
|
+
request body, authorization header, cookie, source file or project metadata.
|
|
46
|
+
It uses Node's certificate/hostname verification and supports the operator's
|
|
47
|
+
Node CA configuration. Certificate verification cannot be disabled by its options.
|
|
48
|
+
Only HTTP 200 is accepted. Redirects are not followed, so the endpoint must serve
|
|
49
|
+
the bytes directly. Encoded/compressed responses are rejected; the digest covers
|
|
50
|
+
the received representation. Bodies are bounded to 64 KiB and response headers
|
|
51
|
+
to 16 KiB. The total network deadline defaults to 30 seconds and permits at most
|
|
52
|
+
120 seconds. SIGINT/SIGTERM or a library abort signal cancels an active transfer.
|
|
53
|
+
|
|
54
|
+
URLs with user information or fragments are rejected. Signed query URLs can be
|
|
55
|
+
used for private distribution, but command arguments may be visible to other
|
|
56
|
+
local processes and URLs may appear in endpoint logs. The library avoids putting
|
|
57
|
+
such a URL in CLI arguments. Returned metadata and normal download errors omit
|
|
58
|
+
the endpoint and its response body. There is no credential store, token refresh,
|
|
59
|
+
registry lookup or publisher-signature system.
|
|
60
|
+
|
|
61
|
+
After transfer, the downloader checks the raw-byte digest, strict UTF-8, JSON
|
|
62
|
+
schema and duplicate requirements. A leading byte-order mark is rejected to match
|
|
63
|
+
the offline pack loader. It writes a private temporary file, syncs it and uses an
|
|
64
|
+
exclusive hard link to publish complete bytes. Existing files—including a file
|
|
65
|
+
created during the download—are never replaced. Temporary files are removed in
|
|
66
|
+
normal completion and handled failure/cancellation. The destination has mode 0600.
|
|
67
|
+
|
|
68
|
+
This requires a filesystem supporting hard links. Power-loss durability, cleanup
|
|
69
|
+
after a process crash and containment of hostile concurrent filesystem changes
|
|
70
|
+
are not guaranteed. The operator must trust the local destination environment.
|
|
71
|
+
No downloaded code is executed by this operation. Executable external adapter
|
|
72
|
+
distribution is separate; its local registration contract is documented in
|
|
73
|
+
`EXTERNAL-ADAPTERS.md`.
|
|
74
|
+
|
|
75
|
+
## Reproduce verification
|
|
76
|
+
|
|
77
|
+
```sh
|
|
78
|
+
npm run build
|
|
79
|
+
node --test dist/test/fetch-pack.test.js
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
The tests require an installed `openssl` command to generate an ephemeral original
|
|
83
|
+
certificate. They use an in-process loopback HTTPS server and explicitly trust
|
|
84
|
+
that certificate in child clients through `NODE_EXTRA_CA_CERTS`. The untrusted
|
|
85
|
+
certificate case must fail; production verification is never disabled. Tests
|
|
86
|
+
exercise exact bytes, offline composition, malformed UTF-8/JSON, duplicate
|
|
87
|
+
requirements, redirects, non-success status, body limits, truncated responses,
|
|
88
|
+
timeouts, cancellation, output boundaries and simultaneous publication. Manual
|
|
89
|
+
guard mutations confirmed that removing digest verification is caught, and replacing
|
|
90
|
+
exclusive publication with an overwriting rename causes both competing writers to
|
|
91
|
+
succeed and fails the race regression. The original guards were restored and the
|
|
92
|
+
tests passed again.
|
|
93
|
+
|
|
94
|
+
`scripts/verify-external-container.mjs` also runs these cases on Node 22 in the
|
|
95
|
+
pinned Linux PHP/OpenSSL container, with external networking disabled. It repeats
|
|
96
|
+
the CLI/library downloads against a fresh offline package installation. The local
|
|
97
|
+
server remains reachable over container loopback. Hosted CI and a production
|
|
98
|
+
private endpoint are separate, unverified deployment environments.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Performance measurements
|
|
2
|
+
|
|
3
|
+
The benchmark creates original synthetic JavaScript projects, then measures the
|
|
4
|
+
shared planning and validation engine in a fresh Node process for each sample.
|
|
5
|
+
It makes no comparison with human review quality, another product or a private
|
|
6
|
+
workflow. These are observations on particular machines and runtime artifacts,
|
|
7
|
+
not a performance promise or CI speed budget.
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
npm run build
|
|
11
|
+
node scripts/measure-performance.mjs > performance.json
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
The default is ten samples per case/operation; `--samples 5` through
|
|
15
|
+
`--samples 30` selects a bounded sample count. Progress goes to stderr and the
|
|
16
|
+
complete JSON goes to stdout only after all samples succeed. Temporary fixture
|
|
17
|
+
creation and final cleanup are outside measured operations. No dependencies,
|
|
18
|
+
services, accounts or network access are needed for these fixtures.
|
|
19
|
+
|
|
20
|
+
## Workloads and accounting
|
|
21
|
+
|
|
22
|
+
The harness defines a small single package, a single package with many small
|
|
23
|
+
source modules, and a multi-package workspace. The exact project, module, file
|
|
24
|
+
and byte counts are derived into each report. Every generated test imports all
|
|
25
|
+
of its package's modules and asserts their combined value. Planning must report
|
|
26
|
+
the expected files, projects and checks. Validation must report one passing,
|
|
27
|
+
non-skipped test per project with unchanged source and no incomplete checks.
|
|
28
|
+
A fast failure, empty plan or omitted test is rejected as a measurement.
|
|
29
|
+
|
|
30
|
+
Cases rotate order between rounds. There is no warm-up exclusion and the
|
|
31
|
+
filesystem cache is uncontrolled; a fresh process does not imply cold storage.
|
|
32
|
+
Each worker has a 60-second bound and 64 KiB output limit. Validation also uses
|
|
33
|
+
the engine's normal run limits. Each sample preserves raw timings, counters and
|
|
34
|
+
the fixture fingerprint. Runtime artifacts, lockfile and harness digests must
|
|
35
|
+
stay unchanged during measurement.
|
|
36
|
+
|
|
37
|
+
- `wallMs`: elapsed time for launching and completing the fresh worker, including
|
|
38
|
+
module loading, the engine operation and waiting for native child processes.
|
|
39
|
+
- `engineMs`: elapsed time around the requested engine operation, excluding
|
|
40
|
+
worker/module startup but including waits for child processes.
|
|
41
|
+
- `engineCpuMs`: user plus system CPU consumed by the worker during the operation;
|
|
42
|
+
child-process CPU is excluded.
|
|
43
|
+
- `enginePeakRssKiB`: the worker's lifetime peak resident memory, including startup;
|
|
44
|
+
child-process memory is excluded. This uses Node's documented
|
|
45
|
+
[resource usage API](https://nodejs.org/api/process.html#processresourceusage).
|
|
46
|
+
|
|
47
|
+
Summaries include minimum, median, nearest-rank p95 and maximum. The median
|
|
48
|
+
averages the middle two observations for an even sample count. With the default
|
|
49
|
+
small sample count, p95 is the maximum observation; it is not a reliable estimate
|
|
50
|
+
of production tail latency. Raw samples are retained for independent analysis.
|
|
51
|
+
No confidence interval or statistical regression threshold is claimed.
|
|
52
|
+
|
|
53
|
+
## Recorded evidence
|
|
54
|
+
|
|
55
|
+
The snapshots in [measurements/](measurements/) identify their runtime, platform,
|
|
56
|
+
architecture, available parallelism, timestamps and measured artifact hashes:
|
|
57
|
+
|
|
58
|
+
- [macOS arm64, Node 26](measurements/performance-darwin-arm64-node26.json).
|
|
59
|
+
- [Linux arm64, Node 22](measurements/performance-linux-arm64-node22.json).
|
|
60
|
+
|
|
61
|
+
The Linux sample was run in the pinned Node container with networking disabled
|
|
62
|
+
and the repository mounted read-only; fixtures lived in the container's temporary
|
|
63
|
+
filesystem. These environments differ in OS, Node version and filesystem, so
|
|
64
|
+
their timings should not be interpreted as an isolated comparison of any one
|
|
65
|
+
variable. Neither snapshot is hosted CI or a Windows measurement.
|
|
66
|
+
|
|
67
|
+
Native compiler/test matrices, large individual files, dependency-heavy builds,
|
|
68
|
+
Git impact selection, long-running jobs, concurrent users and peak memory across
|
|
69
|
+
whole process trees need separate performance evidence. Measurements do not
|
|
70
|
+
establish detection quality or false-positive rates. The separate development
|
|
71
|
+
corpus and its limited native comparisons are described in [EVALUATION.md](EVALUATION.md).
|
package/docs/PEST.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Pest validation
|
|
2
|
+
|
|
3
|
+
Select `php.pest` explicitly with Pest installed in `vendor` and a local
|
|
4
|
+
`phpunit.xml` or `phpunit.xml.dist`. Planning reads paths only. Configuration is
|
|
5
|
+
required because Pest otherwise creates temporary configuration, including during
|
|
6
|
+
its version probe. Execution loads trusted project bootstrap code and plugins.
|
|
7
|
+
|
|
8
|
+
The adapter uses `--ci` to run focused siblings, `--no-tia` to disable cached
|
|
9
|
+
impact-result replay, and PHPUnit's full selection, no history, warning/risky
|
|
10
|
+
failure and fresh JUnit flags. Pest writes progress to stdout; JUnit is streamed
|
|
11
|
+
separately through stderr and parsed with bounded XML validation. Malformed or
|
|
12
|
+
mixed output is incomplete. Every planned `*Test.php` file must have native case
|
|
13
|
+
evidence, and passed cases need positive assertions. Native method suffixes in
|
|
14
|
+
Pest's file attributes are removed before exact file comparison; filenames
|
|
15
|
+
containing `::` are unavailable because that representation is ambiguous.
|
|
16
|
+
|
|
17
|
+
Pest 5.2.1 with PHP 8.5.6 has been exercised in the installed official Composer
|
|
18
|
+
Linux image with no network and synthetic sources created inside the container. Regression cases
|
|
19
|
+
cover assertions, focused siblings, datasets, fixture failures, skipped/todo and
|
|
20
|
+
empty suites, assertionless cases, missing snapshot rejection, multiple files,
|
|
21
|
+
missing plugin metadata and configured always-on TIA. No source changes occurred.
|
|
22
|
+
This does not establish Laravel integration, browser tests, parallel execution,
|
|
23
|
+
ancestor-vendor workspace bootstrapping or compatibility with older Pest versions.
|
|
24
|
+
The verifier does not select Pest automatically from PHP filenames.
|
|
25
|
+
|
|
26
|
+
Prepare tools as documented in [PHPStan](PHPSTAN.md), then run
|
|
27
|
+
`node scripts/verify-php-tools-container.mjs` after building. The locked development
|
|
28
|
+
manifest allows only `pestphp/pest-plugin`; Composer uses that plugin to generate
|
|
29
|
+
Pest's plugin inventory. Project scripts remain disabled. The native fixture
|
|
30
|
+
prepares the mutation plugin's empty vendor cache directory before its
|
|
31
|
+
run. This preparation is separate from consumer validation and never runs
|
|
32
|
+
implicitly during planning or validation.
|
|
33
|
+
|
|
34
|
+
Pest documents its [CI behavior](https://pestphp.com/docs/continuous-integration)
|
|
35
|
+
and [CLI flags](https://pestphp.com/docs/cli-api-reference). Native tests, rather
|
|
36
|
+
than those descriptions alone, establish the behavior advertised here.
|
package/docs/PHPSTAN.md
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# PHPStan validation
|
|
2
|
+
|
|
3
|
+
Select `php.phpstan` explicitly. PHPStan must be installed under `vendor` in the
|
|
4
|
+
project or an ancestor within the configured root. A local `phpstan.neon`,
|
|
5
|
+
`phpstan.neon.dist` or `phpstan.dist.neon` is required, in that priority order.
|
|
6
|
+
Planning only resolves paths; PHP, Composer autoloaders, configured bootstrap
|
|
7
|
+
files and extensions execute only after operator trust.
|
|
8
|
+
|
|
9
|
+
The adapter passes each inventoried PHP file explicitly and runs native debug
|
|
10
|
+
mode with JSON formatting. Debug mode prints each analysed file and disables
|
|
11
|
+
parallel analysis and result-cache reuse. A passing result needs every planned
|
|
12
|
+
path exactly once and consistent native diagnostic totals. An excluded file with
|
|
13
|
+
exit zero is incomplete. File/global diagnostics fail; unparseable execution
|
|
14
|
+
errors are errors. Filenames containing newlines are unavailable because native
|
|
15
|
+
debug paths are line-delimited. Configuration selects rule levels, extensions,
|
|
16
|
+
baselines and suppressions; rule sufficiency and baseline reconciliation are
|
|
17
|
+
separate capabilities.
|
|
18
|
+
|
|
19
|
+
Verified native versions: PHPStan 2.2.14 and PHP 8.5.6, using the installed official
|
|
20
|
+
Composer image. Synthetic fixtures pass valid return types, reject incompatible
|
|
21
|
+
returns and bad configuration, and prevent an excluded file from yielding a
|
|
22
|
+
passing result. Verification uses no network and creates synthetic fixtures inside Linux. The host has no PHP and explicitly skips this native test.
|
|
23
|
+
|
|
24
|
+
Prepare pinned development dependencies separately, then run verification:
|
|
25
|
+
|
|
26
|
+
```sh
|
|
27
|
+
mkdir -p .checktrail/php-tools
|
|
28
|
+
cp scripts/php-tools/composer.json scripts/php-tools/composer.lock .checktrail/php-tools/
|
|
29
|
+
docker run --rm \
|
|
30
|
+
--mount "type=bind,src=$PWD/.checktrail/php-tools,target=/app" \
|
|
31
|
+
--workdir /app composer:2 install \
|
|
32
|
+
--no-interaction --no-scripts --prefer-dist --no-progress
|
|
33
|
+
npm run build
|
|
34
|
+
node scripts/verify-php-tools-container.mjs
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The preparation command downloads locked development dependencies, enables only
|
|
38
|
+
the pinned Pest plugin manager to generate its plugin inventory, and requires
|
|
39
|
+
installed official Composer and Node 22 Alpine images. The verification script inspects that
|
|
40
|
+
local images' digests and never pulls or downloads. It copies the Node executable
|
|
41
|
+
from the installed Node image and runs the engine tests in the PHP container,
|
|
42
|
+
where fixture creation and native execution share one filesystem. The repository
|
|
43
|
+
is mounted read-only; temporary synthetic fixtures are writable inside the
|
|
44
|
+
discarded container. This is development tooling, not container support in the verifier. Ordinary PHP syntax has a separate PHP
|
|
45
|
+
8.4 verification script. Framework/Larastan integration is still unverified.
|
|
46
|
+
|
|
47
|
+
See the official [PHPStan CLI reference](https://phpstan.org/user-guide/command-line-usage#--debug)
|
|
48
|
+
for the native debug and result-cache behavior.
|
package/docs/PHPUNIT.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# PHPUnit validation
|
|
2
|
+
|
|
3
|
+
Select `php.phpunit` explicitly. PHPUnit must be installed under `vendor` within
|
|
4
|
+
the configured root. Candidate files end in `Test.php`; PHPT and other filename
|
|
5
|
+
conventions need separate adapters. Planning does not execute PHP or autoloaders.
|
|
6
|
+
|
|
7
|
+
The adapter invokes the installed PHPUnit and streams fresh JUnit XML to stdout.
|
|
8
|
+
Normal console output is disabled to keep that stream unambiguous. Configured
|
|
9
|
+
logging and test-run history recording are disabled; the adapter does not create
|
|
10
|
+
or reuse a JUnit file. Native configuration, bootstrap, extensions, environment
|
|
11
|
+
requirements and coverage configuration still apply. Trusted application code
|
|
12
|
+
can have side effects, and missing prerequisites are not installed automatically.
|
|
13
|
+
|
|
14
|
+
Native `--all` disables configuration-based test selection, while the explicit
|
|
15
|
+
file list fixes this adapter's scope. Empty suites, risky tests and warnings
|
|
16
|
+
cannot silently pass. The parser reconciles native suite/case counts, requires
|
|
17
|
+
every planned file, and requires positive assertion counts for passing cases.
|
|
18
|
+
This last check also rejects assertionless tests when native strictness is turned
|
|
19
|
+
off. Skipped cases do not supply passing evidence. A zero-exit report omitting a
|
|
20
|
+
planned file is incomplete. Native failures remain failures.
|
|
21
|
+
|
|
22
|
+
Verified native versions are PHPUnit 13.3.4 and PHP 8.5.6. Fixtures exercise
|
|
23
|
+
passing/failing assertions, exceptions, skips, empty files, assertionless tests
|
|
24
|
+
with strictness disabled and multiple test files. Parser cases exercise omitted
|
|
25
|
+
files, missing assertions, truncation and process failure. Project frameworks,
|
|
26
|
+
inherited tests declared outside the planned files, alternate runners and older
|
|
27
|
+
PHPUnit CLI versions remain unverified.
|
|
28
|
+
|
|
29
|
+
The [PHP tooling preparation and container script](PHPSTAN.md) also run these
|
|
30
|
+
fixtures. Verification uses an inspected local official Composer image, no
|
|
31
|
+
network and synthetic fixtures created inside Linux. Host tests explicitly skip the native
|
|
32
|
+
case when PHP or the prepared tools are absent.
|
package/docs/PINT.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Pint validation
|
|
2
|
+
|
|
3
|
+
Select `php.pint` explicitly with Pint installed under `vendor` within the
|
|
4
|
+
configured root. This experimental adapter supports Pint 1.32.1. Other versions
|
|
5
|
+
are incomplete until their internal integration and native fixtures are verified.
|
|
6
|
+
Planning only resolves installed paths and inventoried non-Blade PHP files.
|
|
7
|
+
`*.blade.php` templates are outside this check's scope.
|
|
8
|
+
|
|
9
|
+
After operator trust, a small PHP integration boots the installed Pint PHAR and
|
|
10
|
+
uses its native console command. A command-start listener records the native
|
|
11
|
+
finder's exact files and resolved fixers. The normal native command then runs in
|
|
12
|
+
`--test` mode with JSON agent reporting, no interaction, and a fresh regular cache
|
|
13
|
+
file in the process runner's temporary directory. The runner removes that directory
|
|
14
|
+
after completion, failure, timeout or cancellation; caches are never reused.
|
|
15
|
+
Every planned file must be accounted for, and there must be active
|
|
16
|
+
rules before a clean run can pass. Source syntax and style diagnostics fail.
|
|
17
|
+
Empty rules, unexpected/missing/duplicate files, unsupported versions, malformed
|
|
18
|
+
output and execution failures cannot pass. Explicit file arguments override
|
|
19
|
+
Pint's finder exclusions; the native regression proves that an explicitly passed
|
|
20
|
+
`notName` file is still checked. This follows native CLI path-override behavior.
|
|
21
|
+
|
|
22
|
+
Blade/Prettier fixers are rejected before their dependency preparation runs.
|
|
23
|
+
This adapter does not install dependencies, update code, run parallel fixing, or
|
|
24
|
+
claim Blade support. Configured PHP rules, presets and local configuration
|
|
25
|
+
inheritance remain native Pint behavior. The internal PHAR integration is
|
|
26
|
+
version-gated because the public CLI JSON output lacks clean-file scope evidence.
|
|
27
|
+
|
|
28
|
+
Native checks use Pint 1.32.1, PHP 8.5.6 and Node 22.23.2 in Linux. They cover
|
|
29
|
+
valid formatting, style and syntax failures, unchanged source/cache files,
|
|
30
|
+
explicitly selected excluded files, empty rules, invalid presets, blocked Blade
|
|
31
|
+
integration and multiple files. Run `node scripts/verify-php-tools-container.mjs`
|
|
32
|
+
after the [development preparation](PHPSTAN.md) and build. Synthetic fixtures
|
|
33
|
+
are created in the container filesystem with networking disabled.
|
|
34
|
+
|
|
35
|
+
The first macOS hosted run exposed that using `/dev/null` as a cache file fails
|
|
36
|
+
when Pint seeks within it. The regular temporary file fixes that mechanism;
|
|
37
|
+
the corrected hosted macOS suite and required profile passed at `52ba415`.
|
|
38
|
+
The fixture now exercises a symlinked project root and compares canonical native
|
|
39
|
+
paths on every platform. See `NATIVE-CI.md`.
|
|
40
|
+
|
|
41
|
+
See [Pint's official documentation](https://laravel.com/framework/docs/pint).
|