@stsepelin/checktrail 0.1.0-alpha.2 → 0.1.0-alpha.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -6
- package/dist/src/adapters.js +3 -1
- package/dist/src/architecture.d.ts +1 -1
- package/dist/src/contract-schema.d.ts +1 -1
- package/dist/src/contracts.d.ts +1 -1
- package/dist/src/evidence.js +2 -1
- package/dist/src/finding-policy-schema.d.ts +1 -1
- package/dist/src/go-scope-policy.d.ts +11 -0
- package/dist/src/go-scope-policy.js +50 -0
- package/dist/src/go-scope.d.ts +1 -1
- package/dist/src/go-scope.js +42 -5
- package/dist/src/output.js +6 -0
- package/dist/src/report-validation.d.ts +7 -0
- package/dist/src/runtime-inventory.d.ts +2 -2
- package/dist/src/schemas.d.ts +17 -0
- package/dist/src/schemas.js +16 -2
- package/dist/src/types.d.ts +4 -1
- package/dist/src/types.js +1 -1
- package/dist/src/typescript-arguments.d.ts +4 -0
- package/dist/src/typescript-arguments.js +10 -0
- package/dist/src/typescript-runner.d.ts +1 -0
- package/dist/src/typescript-runner.js +19 -0
- package/dist/src/typescript.js +3 -9
- package/dist/src/vue-tsc-runner.js +2 -1
- package/docs/ACCEPTANCE.md +24 -13
- package/docs/CLIENTS.md +6 -4
- package/docs/GO-RACE.md +4 -2
- package/docs/GO-SCOPE.md +70 -5
- package/docs/INSTALLATION.md +6 -6
- package/docs/LANGUAGES.md +16 -1
- package/docs/MCP-COMPATIBILITY.md +6 -0
- package/docs/NATIVE-CI.md +4 -0
- package/docs/ONBOARDING.md +8 -2
- package/docs/PUBLIC-ADOPTION.md +118 -0
- package/docs/RELEASE.md +77 -20
- package/docs/SETUP-SCOPES.md +107 -0
- package/docs/STATUS.md +33 -11
- package/docs/TYPESCRIPT.md +78 -0
- package/docs/measurements/go-scope-policy-replay.json +91 -0
- package/docs/measurements/public-adoption-alpha2.json +2034 -0
- package/docs/measurements/release-alpha3.json +362 -0
- package/docs/measurements/typescript-legacy-replay.json +39 -0
- package/package.json +1 -1
- package/schemas/go-scope-policy.schema.json +31 -0
- package/schemas/plan-summary.schema.json +6 -1
- package/schemas/plan.schema.json +30 -0
- package/schemas/report-summary.schema.json +5 -0
- package/schemas/report.schema.json +30 -0
- package/server.json +2 -2
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Choose and understand setup coverage
|
|
2
|
+
|
|
3
|
+
A repository can contain a runnable library, a documentation project and CI
|
|
4
|
+
workflows. Each can need different tools. `init` proposes checks; `doctor` reports
|
|
5
|
+
static setup gaps; only a trusted `run` produces validation evidence.
|
|
6
|
+
|
|
7
|
+
Use the [installation guide](INSTALLATION.md) and inspect before writing:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
checktrail init --root "$PWD"
|
|
11
|
+
checktrail plan --root "$PWD" --detailed
|
|
12
|
+
checktrail doctor --root "$PWD" --detailed
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
A `needs-selection` or `attention-required` result exits `2`. Read its JSON before
|
|
16
|
+
choosing checks. These inspection commands do not run project scripts or install
|
|
17
|
+
dependencies. Detailed output includes paths and may include absolute diagnostic paths.
|
|
18
|
+
|
|
19
|
+
## A library with documentation and workflows
|
|
20
|
+
|
|
21
|
+
Consider this original, generic repository:
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
package.json scripts.test is "node --test"
|
|
25
|
+
sum.test.js one passing Node test
|
|
26
|
+
.github/workflows/verify.yml a GitHub Actions workflow
|
|
27
|
+
docs/requirements.txt documentation dependencies
|
|
28
|
+
docs/conf.py documentation configuration, no tests
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Discovery identifies JavaScript and infrastructure at `.`, and Python at `docs`.
|
|
32
|
+
The directory name does not prove that it is documentation, nor that it can be
|
|
33
|
+
ignored: the operator knows its purpose. Checktrail does not infer a documentation
|
|
34
|
+
build command from a Python manifest.
|
|
35
|
+
|
|
36
|
+
`init` returns `needs-selection` for the Python project and writes nothing, even
|
|
37
|
+
though the Node test command is recognized. Supplying a check for `.` does not
|
|
38
|
+
select it for nested projects. For example, this explicit preview covers all three
|
|
39
|
+
discovered ecosystems:
|
|
40
|
+
|
|
41
|
+
```sh
|
|
42
|
+
checktrail init --root "$PWD" \
|
|
43
|
+
--check '.#javascript.node-test' \
|
|
44
|
+
--check '.#infrastructure.actionlint' \
|
|
45
|
+
--check 'docs#python.unittest'
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
It produces a policy preview, but the choice of `unittest` is not evidence of tests
|
|
49
|
+
in `docs`. If that policy is created with `--write`, `doctor --detailed` reports
|
|
50
|
+
`unavailable-check` for `python.unittest`: no candidate test files were discovered.
|
|
51
|
+
Choosing a runner merely to resolve setup selection does not validate documentation.
|
|
52
|
+
Use a separately supported check or the project's own documentation build; keep
|
|
53
|
+
that work outside Checktrail's coverage claim when no applicable adapter exists.
|
|
54
|
+
|
|
55
|
+
## Prepare workflow validation separately
|
|
56
|
+
|
|
57
|
+
`infrastructure.actionlint` is static workflow analysis. It does not execute CI
|
|
58
|
+
jobs or validate their shell bodies. It needs both the prepared native executable
|
|
59
|
+
and `checktrail.actionlint.json` beside the workflow repository root:
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{
|
|
63
|
+
"schemaVersion": 1,
|
|
64
|
+
"runnerLabels": [],
|
|
65
|
+
"variables": []
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The empty arrays declare no extra runner labels or configuration variables. Review
|
|
70
|
+
actual workflow requirements before adopting those values. See [ACTIONLINT.md](ACTIONLINT.md)
|
|
71
|
+
for the supported tool version, configuration, dependency checks and limitations.
|
|
72
|
+
Without this configuration, diagnosis reports `unavailable-check`; without an
|
|
73
|
+
executable on the effective PATH it also reports `missing-executable`. Prepare the
|
|
74
|
+
tool explicitly, then repeat diagnosis. A successful Node test cannot supply this
|
|
75
|
+
workflow evidence.
|
|
76
|
+
|
|
77
|
+
## Interpret an intentionally narrowed policy
|
|
78
|
+
|
|
79
|
+
After reviewing the desired scope, an operator may explicitly maintain a
|
|
80
|
+
language-only policy. For the example above, it is:
|
|
81
|
+
|
|
82
|
+
```json
|
|
83
|
+
{
|
|
84
|
+
"schemaVersion": 1,
|
|
85
|
+
"projects": [{ "path": ".", "checks": ["javascript.node-test"] }]
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Edit an existing policy deliberately; `init --write` preserves it and cannot
|
|
90
|
+
replace it or narrow its checks through `--check` arguments. Do not discard
|
|
91
|
+
existing overlays, packs, environment requirements or workspace dependencies.
|
|
92
|
+
|
|
93
|
+
With the language-only policy, the commands have different meanings:
|
|
94
|
+
|
|
95
|
+
| Command | Result for this example | Coverage |
|
|
96
|
+
| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
|
|
97
|
+
| `checktrail doctor --root "$PWD" --detailed` | `attention-required`, exit `2`; `unselected-project` for infrastructure at `.` and Python at `docs` | Workflow and documentation-project checks are omitted. |
|
|
98
|
+
| `checktrail run --root "$PWD" --trust-project --detailed` | `passed`, exit `0`, if the Node test passes | The selected Node test only. |
|
|
99
|
+
|
|
100
|
+
These results are consistent: validation passed its selected check while diagnosis
|
|
101
|
+
reported broader discovered scope left unselected. `doctor` does not assess every
|
|
102
|
+
possible check within a selected ecosystem either; selecting Node tests does not
|
|
103
|
+
claim linting, typing, browser behavior or documentation coverage.
|
|
104
|
+
|
|
105
|
+
Keep omitted work visible in your review and CI requirements. The
|
|
106
|
+
[public adoption report](PUBLIC-ADOPTION.md) records these distinctions on pinned
|
|
107
|
+
public projects; its historical results are not a claim about your repository.
|
package/docs/STATUS.md
CHANGED
|
@@ -3,15 +3,38 @@
|
|
|
3
3
|
This is an experimental foundation with public source at
|
|
4
4
|
[stsepelin/checktrail](https://github.com/stsepelin/checktrail).
|
|
5
5
|
Check the [installation guide](INSTALLATION.md) for package availability and setup.
|
|
6
|
-
The `0.1.0-alpha.
|
|
7
|
-
[hosted run at
|
|
8
|
-
passed all jobs
|
|
9
|
-
|
|
10
|
-
|
|
6
|
+
The `0.1.0-alpha.3` preview is published on npm and active in the MCP Registry.
|
|
7
|
+
The [hosted run at b0b447d](https://github.com/stsepelin/checktrail/actions/runs/35597877167)
|
|
8
|
+
passed all jobs. The published tarball matched the reviewed artifact; fresh
|
|
9
|
+
registry installation, CLI/library validation, generated npx startup and MCP
|
|
10
|
+
execution/trust behavior were verified. Earlier toolchain evidence remains in
|
|
11
|
+
`NATIVE-CI.md`; release details and tag limitations are in `RELEASE.md`.
|
|
12
|
+
The GitHub alpha.3 prerelease includes the verified artifact and checksum.
|
|
13
|
+
[Public adoption](PUBLIC-ADOPTION.md) records alpha.2 on five pinned libraries, including incomplete
|
|
14
|
+
coverage and the TypeScript 4.9.5 incompatibility fixed in alpha.3.
|
|
15
|
+
|
|
16
|
+
## Alpha.4 candidate (unpublished)
|
|
17
|
+
|
|
18
|
+
The checkout version is `0.1.0-alpha.4`. Release verification is in progress;
|
|
19
|
+
alpha.3 remains the published preview. No alpha.4 publication or hosted release
|
|
20
|
+
verification is claimed.
|
|
21
|
+
|
|
22
|
+
- [Go scope policy](GO-SCOPE.md): exact per-file declarations can acknowledge
|
|
23
|
+
native build-constraint exclusions. Go must confirm every exemption; stale
|
|
24
|
+
declarations and other omissions remain incomplete. Detailed reports retain
|
|
25
|
+
reasons and summaries show counts. Formatting still covers all source. Race
|
|
26
|
+
package listing now uses the same `-race` constraints as execution.
|
|
27
|
+
|
|
28
|
+
## Alpha.3 release
|
|
29
|
+
|
|
30
|
+
- [TypeScript compatibility](TYPESCRIPT.md): plain typechecking now passes the
|
|
31
|
+
native 4.9.5 regression while retaining the 6.0.3 `noCheck` override and native
|
|
32
|
+
file accounting. A packed CLI/library/MCP replay fixes the recorded mitt case.
|
|
33
|
+
Published alpha.2 and its immutable adoption record retain the original failure.
|
|
11
34
|
|
|
12
35
|
## Implemented
|
|
13
36
|
|
|
14
|
-
- Alpha.2
|
|
37
|
+
- Alpha.2: conservative multi-language `init`, execution-free `doctor`,
|
|
15
38
|
and version-pinned MCP configuration output for Codex, Claude Code/Desktop,
|
|
16
39
|
Cursor and VS Code. Existing files are preserved. See [ONBOARDING.md](ONBOARDING.md)
|
|
17
40
|
for scope, ambiguity handling and package upgrade/rollback verification.
|
|
@@ -153,11 +176,10 @@ a general speedup/graph-completeness claim.
|
|
|
153
176
|
|
|
154
177
|
## Release preparation
|
|
155
178
|
|
|
156
|
-
`server.json
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
separately in `NATIVE-CI.md`. See `RELEASE.md`.
|
|
179
|
+
`server.json`, the npm package and the engine identify the published
|
|
180
|
+
`0.1.0-alpha.3` preview. Its source commit passed hosted CI, the published tarball
|
|
181
|
+
matched the reviewed artifact, and fresh CLI/library/MCP installation checks passed.
|
|
182
|
+
Publication evidence and the sequence for future releases are in `RELEASE.md`.
|
|
161
183
|
|
|
162
184
|
## Pinned pack distribution
|
|
163
185
|
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# TypeScript compiler compatibility
|
|
2
|
+
|
|
3
|
+
The source checkout's `javascript.typescript` profile now supports the exercised
|
|
4
|
+
TypeScript 4.9.5 and 6.0.3 configurations. This fix is published in
|
|
5
|
+
**0.1.0-alpha.3**. Published alpha.2 still passes an option that
|
|
6
|
+
TypeScript 4.9.5 rejects. The original
|
|
7
|
+
[public adoption record](PUBLIC-ADOPTION.md) remains unchanged as release evidence.
|
|
8
|
+
Other compiler versions require their own native verification.
|
|
9
|
+
|
|
10
|
+
## Invocation and boundaries
|
|
11
|
+
|
|
12
|
+
Planning locates the project-local or root-hoisted compiler without importing it.
|
|
13
|
+
During explicitly trusted execution, a runner loads the selected compiler API
|
|
14
|
+
and uses its public command-line parser to check support for `--noCheck false`.
|
|
15
|
+
Supported compilers receive that override, so `noCheck: true` cannot disable the
|
|
16
|
+
selected type check. Older compilers that reject this option receive the remaining
|
|
17
|
+
arguments; an unexpected capability response stops before the compiler CLI starts.
|
|
18
|
+
The probe parses fixed arguments and does not parse project configuration.
|
|
19
|
+
|
|
20
|
+
The compiler entry and API must resolve inside the configured root. They are
|
|
21
|
+
trusted executable dependencies; this check is not a sandbox. No compiler is
|
|
22
|
+
installed or downloaded during planning or execution. `doctor` remains static
|
|
23
|
+
and does not perform this runtime capability probe.
|
|
24
|
+
|
|
25
|
+
Checks still disable emitted output and incremental state and require every
|
|
26
|
+
inventoried TypeScript source in the native file list. A successful compiler exit
|
|
27
|
+
with omitted source is incomplete. A legacy compiler's unsupported project options
|
|
28
|
+
remain compiler errors; the runner does not rewrite `tsconfig.json`.
|
|
29
|
+
|
|
30
|
+
The Vue runner uses the same capability helper. Its separately tested Vue 3.5.43,
|
|
31
|
+
vue-tsc 3.3.11 and TypeScript 6.0.3 profile retains type-error detection and the
|
|
32
|
+
`skipTemplateCodegen` rejection. This does not establish legacy Vue compatibility.
|
|
33
|
+
The solution-build adapter remains gated to TypeScript 6.0.3; see
|
|
34
|
+
[TYPESCRIPT-BUILD.md](TYPESCRIPT-BUILD.md).
|
|
35
|
+
|
|
36
|
+
## Verification
|
|
37
|
+
|
|
38
|
+
The new 4.9.5 regression failed on the previous adapter's TS5023 error. With this
|
|
39
|
+
change it accepts valid code, rejects a real TS2322 error, accounts for excluded
|
|
40
|
+
source, identifies the actual tool version, supports a root-hoisted compiler and
|
|
41
|
+
paths with spaces, and leaves emitted/incremental files absent. Existing native
|
|
42
|
+
6.0.3 and Vue regressions still reject type errors even with `noCheck: true`.
|
|
43
|
+
Separate regressions reject an escaping compiler API and an ambiguous capability
|
|
44
|
+
response before the compiler CLI executes. Planning is checked against executable
|
|
45
|
+
traps in both the CLI and API files.
|
|
46
|
+
|
|
47
|
+
The [known-case replay](measurements/typescript-legacy-replay.json) uses mitt at
|
|
48
|
+
the same pinned revision and the same TypeScript 4.9.5 tool profile as the alpha.2
|
|
49
|
+
adoption exercise. After generating its required root declaration, native checking
|
|
50
|
+
and the packed fix pass; an original injected TS2322 error fails through the CLI,
|
|
51
|
+
library and MCP. The policy and tracked upstream files stay unchanged, and the
|
|
52
|
+
injected source and generated declaration are removed afterwards. This is a
|
|
53
|
+
regression replay after observing the failure, not a new independent holdout.
|
|
54
|
+
The record identifies the tested local tarball and adapter source hashes; it is
|
|
55
|
+
not the published alpha.2 artifact.
|
|
56
|
+
|
|
57
|
+
## Prepare the legacy regression
|
|
58
|
+
|
|
59
|
+
The old compiler has a separate locked installation so its `tsc` binary cannot
|
|
60
|
+
replace the compiler used to build Checktrail:
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
mkdir -p .checktrail/typescript-legacy-tools
|
|
64
|
+
cp scripts/typescript-legacy-tools/package.json scripts/typescript-legacy-tools/package-lock.json .checktrail/typescript-legacy-tools/
|
|
65
|
+
npm ci --prefix .checktrail/typescript-legacy-tools --ignore-scripts --no-audit --no-fund
|
|
66
|
+
npm run build
|
|
67
|
+
node --test dist/test/typescript.test.js dist/test/vue-tsc.test.js dist/test/typescript-build.test.js
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
The general suite explicitly skips the legacy native case when this installation
|
|
71
|
+
is absent. The hosted main matrix prepares it and the `javascript` required-native
|
|
72
|
+
profile rejects a skipped or missing legacy regression. A local pass does not
|
|
73
|
+
establish hosted CI results.
|
|
74
|
+
|
|
75
|
+
TypeScript introduced the public `noCheck` option in
|
|
76
|
+
[TypeScript 5.6](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-6.html#the--nocheck-option).
|
|
77
|
+
The runner checks actual parser behavior instead of assuming capabilities from a
|
|
78
|
+
package version string.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
{
|
|
2
|
+
"scope": "Known-case replay of an unreleased local package; excludes the JavaScript target without validating it; not an independent holdout",
|
|
3
|
+
"upstreamCommit": "2d3c2a9cc518326daf99a383f07c4d3c44317e4d",
|
|
4
|
+
"tarballSha256": "08069a85dc7863394a88218a1670a2e00381c7e176a1ae4c987ce8496a4c3e19",
|
|
5
|
+
"platform": "darwin",
|
|
6
|
+
"arch": "arm64",
|
|
7
|
+
"node": "v26.8.1",
|
|
8
|
+
"observations": [
|
|
9
|
+
{
|
|
10
|
+
"phase": "strict",
|
|
11
|
+
"outcome": "failed",
|
|
12
|
+
"checks": [
|
|
13
|
+
{
|
|
14
|
+
"id": "go.format",
|
|
15
|
+
"status": "failed"
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
"id": "go.vet",
|
|
19
|
+
"status": "inconclusive"
|
|
20
|
+
},
|
|
21
|
+
{
|
|
22
|
+
"id": "go.test",
|
|
23
|
+
"status": "inconclusive"
|
|
24
|
+
}
|
|
25
|
+
],
|
|
26
|
+
"mcpCounts": [0, 0, 0]
|
|
27
|
+
},
|
|
28
|
+
{
|
|
29
|
+
"phase": "declared",
|
|
30
|
+
"outcome": "failed",
|
|
31
|
+
"checks": [
|
|
32
|
+
{
|
|
33
|
+
"id": "go.format",
|
|
34
|
+
"status": "failed"
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"id": "go.vet",
|
|
38
|
+
"status": "passed"
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"id": "go.test",
|
|
42
|
+
"status": "passed",
|
|
43
|
+
"tests": {
|
|
44
|
+
"total": 213,
|
|
45
|
+
"passed": 212,
|
|
46
|
+
"failed": 0,
|
|
47
|
+
"skipped": 1
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
],
|
|
51
|
+
"mcpCounts": [0, 1, 1]
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
"phase": "failing-test",
|
|
55
|
+
"outcome": "failed",
|
|
56
|
+
"checks": [
|
|
57
|
+
{
|
|
58
|
+
"id": "go.format",
|
|
59
|
+
"status": "failed"
|
|
60
|
+
},
|
|
61
|
+
{
|
|
62
|
+
"id": "go.vet",
|
|
63
|
+
"status": "passed"
|
|
64
|
+
},
|
|
65
|
+
{
|
|
66
|
+
"id": "go.test",
|
|
67
|
+
"status": "failed",
|
|
68
|
+
"tests": {
|
|
69
|
+
"total": 214,
|
|
70
|
+
"passed": 212,
|
|
71
|
+
"failed": 1,
|
|
72
|
+
"skipped": 1
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
],
|
|
76
|
+
"mcpCounts": [0, 1, 1]
|
|
77
|
+
}
|
|
78
|
+
],
|
|
79
|
+
"baseCommit": "bc0b95a511706475df97d1472b4724b36dbc2848",
|
|
80
|
+
"goVersion": "go version go1.27.1 darwin/arm64",
|
|
81
|
+
"sources": {
|
|
82
|
+
"src/go-scope-policy.ts": "be40a2fa681fd5c320e36ea4608a972cb369e045227e1e3fd539f915444287f3",
|
|
83
|
+
"src/go-scope.ts": "cc85c48962b91ab4b88977a78f8bddc62e866f8d863c265ffe1b1cfd523a4895",
|
|
84
|
+
"src/adapters.ts": "ba2dab60ac9ecf20d9843285b614d62b4853f06972e2e27946fe0f8de01c4a61",
|
|
85
|
+
"src/evidence.ts": "477a82d4b40cb3480fa9d5b6ab2a0d41c06685f275fda0abb061219eb2a7a086",
|
|
86
|
+
"src/output.ts": "6befd79c5cc095137edd5b11c6f29622ffa22f9d924ed64152a085abef7da891",
|
|
87
|
+
"src/schemas.ts": "e5add92fee2727e50222757703c8dd786b72c6ebf2b40d1fcad4864961805ee2",
|
|
88
|
+
"src/types.ts": "c30a5c65903342148a7a07c210d6c0b5231295bbf7b1d94944a29c070289587c"
|
|
89
|
+
},
|
|
90
|
+
"harnessSha256": "acd5dcad4cc8fce8818963f48714cc8af8139720f64779a2dbc79d6fb4866c3f"
|
|
91
|
+
}
|