@shipispec/tsfix 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +106 -1
- package/README.md +88 -26
- package/dist/cli.js +869 -24
- package/dist/index.d.ts +26 -6
- package/dist/index.js +747 -8
- package/dist/types/applyEditBlock.d.ts +68 -0
- package/dist/types/index.d.ts +26 -6
- package/dist/types/mendAgent.d.ts +53 -0
- package/dist/types/runMendLoop.d.ts +79 -0
- package/dist/types/stubAndContinue.d.ts +68 -0
- package/dist/types/typeContext.d.ts +48 -0
- package/package.json +12 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,110 @@ All notable changes to `@shipispec/tsfix` are documented here. Format follows [K
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
+
### Added (Layer 4 — stub-and-continue escape hatch)
|
|
8
|
+
- **`stubAndContinue(opts)`** — new public API. Inserts `// @ts-expect-error - tsfix: <codes> — <message>` immediately above each unresolved error site so `tsc --noEmit` exits 0. Closes the "tsfix never leaves the workspace worse than it found it" property. Uses `@ts-expect-error` (not `@ts-ignore`) so directives self-destruct once the underlying issue is fixed by other means.
|
|
9
|
+
- **`runMendLoop` opt-in flag** — new `stubOnFailure?: boolean` option (default `false`). When the LLM loop terminates with leftover errors and the flag is set, Layer 4 runs automatically. New `"stubbed"` stop reason; new `stubs?: AppliedStub[]` result field with what was applied.
|
|
10
|
+
- **Idempotency** — re-running `stubAndContinue` on an already-stubbed workspace is a no-op. Detects existing `@ts-expect-error` / `@ts-ignore` directives on the line above and skips.
|
|
11
|
+
- **Safe skips** — `node_modules/`, `.d.ts` files, missing files, and lines beyond file length are recorded as `skipped` (with reason) rather than crashing.
|
|
12
|
+
- **Multi-error coalescing** — multiple diagnostics on the same line collapse into one stub comment listing all TS codes and joined messages.
|
|
13
|
+
- **Indent + CRLF preservation** — comment matches the indentation of the line it's stubbing; CRLF line endings on Windows-authored files survive the rewrite.
|
|
14
|
+
- **`dryRun`** support — same semantics as Layer 2: reports `stubsApplied` without writing.
|
|
15
|
+
|
|
16
|
+
### Added (fixture engine — Day 2/3 mutators)
|
|
17
|
+
- **5 new ts-morph mutators** covering codes the original 3-mutator set didn't reach:
|
|
18
|
+
- `ts2322-incompatible-return.mjs` — replaces a return expression with a wrong-typed primitive literal in a function with a primitive return type
|
|
19
|
+
- `ts2304-cannot-find-name.mjs` — renames a value-position identifier (variable, call, parameter usage) to a no-near-match string; Layer 0's auto-import abstains because there's no candidate
|
|
20
|
+
- `ts2345-arg-type-mismatch.mjs` — replaces a function-call argument with a wrong-typed primitive when the parameter's declared type is `string` / `number` / `boolean`
|
|
21
|
+
- `ts2554-arg-count-mismatch.mjs` — drops the trailing argument from a call that currently satisfies its signature
|
|
22
|
+
- `ts2365-operator-mismatch.mjs` — replaces one operand of a numeric binary expression (`<`, `>`, `<=`, `>=`, `-`, `*`, `/`, `%`) with a string literal
|
|
23
|
+
- **50 new generated fixtures** (10 per new code × 8 codes total). Total Layer-2 fixture corpus: **85** (was 35) — 3 minimal + 2 realistic + 80 generated across 8 codes. Total fixture count across all layers: **99** (14 Layer-0 + 85 Layer-2).
|
|
24
|
+
|
|
25
|
+
### Added (tests)
|
|
26
|
+
- **19 new Layer-4 unit tests** — 16 in `stubAndContinue.test.ts` + 3 in `runMendLoop.test.ts` covering single error, multi-error-same-line, multi-code, indent preservation, descending-order processing, idempotency, node_modules skip, .d.ts skip, missing-file skip, dry-run, message truncation, CRLF preservation, first-line edge case, no-eligible case, warning/suggestion filtering, and the runMendLoop integration (stopReason flip, default-off behavior, dryRun interaction).
|
|
27
|
+
|
|
28
|
+
### Changed
|
|
29
|
+
- **Public surface** at `src/index.ts` extended with `stubAndContinue`, `StubAndContinueOptions`, `StubAndContinueResult`, `AppliedStub`, `SkippedStub`. Layer 0/1/2 surface unchanged.
|
|
30
|
+
- **`RunMendLoopOptions`** gains `stubOnFailure?: boolean`. **`RunMendLoopResult`** gains optional `stubs?: AppliedStub[]`. **`StopReason`** union gains `"stubbed"`. All additive — old callers unaffected.
|
|
31
|
+
- **`scripts/generate-fixtures.mjs`** now runs via `tsx` and imports from `src/index.ts` directly instead of `dist/index.js`. Reason: the v0.4.0 dist bundle inlines `@vercel/oidc` (transitive of `ai`), which uses dynamic `require()` patterns that fail under esbuild's ESM output at module-init time. The generator only needs Layer 0/1 entry points, so importing from source bypasses the AI SDK entirely. Side benefit: no `npm run build` prerequisite — `npm run generate-fixtures` works from a fresh clone.
|
|
32
|
+
- Removed `pregenerate-fixtures: npm run build` hook from `package.json`.
|
|
33
|
+
|
|
34
|
+
### Fixed
|
|
35
|
+
- **`stubAndContinue` resolves relative paths** against `workspaceRoot`. Diagnostics from `runInProcessTsc` use relative paths; consumers may pass absolute. Both work.
|
|
36
|
+
|
|
37
|
+
### Added (Layer-2 benchmark — Day 4)
|
|
38
|
+
- **Parallelism** — `npm run benchmark:llm` now runs fixtures concurrently via an inline `pLimit(N)` semaphore (no new dep). Default concurrency is 8; configurable via `--concurrency=N`. 100 fixtures at ~1.5s/each: sequential ~2 min → parallel ~20s. Per-fixture workspaces are isolated (snapshot/restore is local) so parallelism is safe; tsfix's program cache thrashes harmlessly between fixtures.
|
|
39
|
+
- **File-based response cache** — every LLM call is keyed by `sha256(systemBlock + userBlock + model)` and stored under `.benchmark-cache/<hash>.json`. Re-runs against unchanged fixtures replay cached responses for free. Any change to the system prompt template, fixture content, or model invalidates automatically (it's all in the hash). `--no-cache` bypasses; `--clear-cache` wipes and exits. `.benchmark-cache/` added to `.gitignore`.
|
|
40
|
+
- **Cache module** — extracted to `benchmark/cache.ts` so the logic is unit-testable independent of the full benchmark. 16 new unit tests covering: deterministic keying, key sensitivity per input, hex format, separator-confusion resistance, round-trip read/write, corrupted-entry handling, miss → store → hit cycle, parameter discrimination, bypass behavior, apiKey NOT in the cache key (rotating keys doesn't invalidate), error propagation without poisoning the cache.
|
|
41
|
+
- **Failure reporting** — when fixtures fail, the per-iteration LLM raw response dump is collected and printed in a single block at the end of the run (instead of inline during the loop, which would interleave under concurrency).
|
|
42
|
+
- **Layer-2 fixture filter (LLM benchmark)** — the LLM benchmark now filters fixtures by `expected.json` shape (`costUsdMax` or `expectedErrorCode`), mirroring the Layer-0 benchmark's filter. Prevents accidentally running Layer-0 fixtures through the LLM.
|
|
43
|
+
- **`benchmark/run-llm-benchmark.ts` rewritten** around the parallel + cached worker model. Per-fixture output gets a `[n/m]` progress prefix and prints in completion order; final summary sorted by name for deterministic output. Total wall time, total cost (cache misses only — hits are free), and cache hit rate are reported at the end.
|
|
44
|
+
|
|
45
|
+
### Added (CLI — Layer 2 exposure)
|
|
46
|
+
- **`--llm` flag** — opt-in escalation to Layer 2 (single-file LLM mend) for errors that survive Layer 0/1. Off by default; CLI default path remains zero-network.
|
|
47
|
+
- **`--llm-model <name>`** — Anthropic model (default `claude-haiku-4-5`). Known-priced models hardcoded for cost estimation: `claude-haiku-4-5`, `claude-sonnet-4-5`, `claude-opus-4-7`. Unknown models warn and report cost as 0.
|
|
48
|
+
- **`--llm-max-iterations <N>`** — cap on LLM retries (default 3).
|
|
49
|
+
- **`--llm-budget-usd <amount>`** — soft cost cap. If exceeded, exits with code 3 (Layer 2 still ran; partial work persisted to disk).
|
|
50
|
+
- **Exit code 3** added — Layer 2 budget exceeded.
|
|
51
|
+
- **Validation:** `--llm` without `ANTHROPIC_API_KEY` → exit 2 with helpful error. `--llm + --dry-run` is rejected as mutually exclusive (Layer 2 writes patches to disk).
|
|
52
|
+
- **JSON report extension** — `layer2: { ran, stopReason, errorsBefore, errorsAfter, iterations, totalInputTokens, totalOutputTokens, totalCostUsd, budgetExceeded, model } | null`. Layer 0/1 surface unchanged.
|
|
53
|
+
- **Human report extension** — new "Layer 2 (LLM)" line in the per-run summary when `--llm` was used. Shows error count delta, iteration count, tokens, cost, and stopReason.
|
|
54
|
+
- **13 new CLI integration tests** in `cli/run-stack.test.ts` covering argument validation, exit codes, no-key errors, mutual-exclusion checks, JSON report shape, and the no-Layer-2-when-Layer-0-clean path. Tests spawn the actual `tsx cli/run-stack.ts` process — catches integration issues that pure unit tests of `parseArgs` would miss.
|
|
55
|
+
|
|
56
|
+
### Fixed (latent v0.4.0 bundle bug)
|
|
57
|
+
- **Externalized `ai` and `@ai-sdk/anthropic`** from the esbuild bundle in `scripts/build.mjs`. v0.4.0 inlined them, which (1) bloated `dist/index.js` from ~25 KB to 1.3 MB, and (2) crashed under plain `node` execution at module-init time because `@vercel/oidc` (transitive) uses dynamic `require()` patterns that fail in ESM bundles. Both packages are declared in `dependencies` so npm install pulls them in automatically for consumers using Layer 2. **Bundle sizes after fix:** `dist/index.js` 1.3 MB → 36 KB; `dist/cli.js` ~22 KB → 45 KB. Library import via plain `node` now works (verified with `node -e 'import("./dist/index.js")'`).
|
|
58
|
+
|
|
59
|
+
### Deferred (fixture engine)
|
|
60
|
+
- **TS2532** (Object is possibly undefined) — seeds don't currently contain optional chains or `Map.get()`-style calls that would produce TS2532 deterministically. Mutator deferred until seeds expand or a real-failure capture provides better candidates.
|
|
61
|
+
- **TS2551-negative** (LSP returns multiple equally-close fix candidates → abstains) — engineering a deterministic TS2551 case where Layer 0's `fixesAreEquivalent` check abstains is contrived. Defer until we see a real-world example.
|
|
62
|
+
|
|
63
|
+
### Also changed (CLI)
|
|
64
|
+
- **CLI public surface** at `cli/run-stack.ts` extended with the Layer 2 flags listed above. The bin entry (`dist/cli.js`) now imports `runMendLoop`, `runInProcessTsc` and the contract types from `src/index.ts` in addition to the previous `runValidationLoop` + `discoverTsFiles`. Tree-shaking keeps Layer 2 out of the bundle's code path unless `--llm` is set.
|
|
65
|
+
|
|
66
|
+
## [0.4.0] - 2026-05-14
|
|
67
|
+
|
|
68
|
+
**Layer 2 LLM mend is now in-package.** The previously-planned sister package `@shipispec/tsmend` has been folded into `tsfix` so the deterministic Layer 0/1 stack and the LLM-driven Layer 2 stack ship as one. This reverses the v0.3.0 sister-package decision (D3) — see roadmap update.
|
|
69
|
+
|
|
70
|
+
### Added (Layer 2 — single-file LLM mend)
|
|
71
|
+
- **`getTypeContext(opts)`** — TS Language Service helper. Resolves an error site to its declaring type via `getTypeAtLocation()` + `getDeclarations()`, slices ±3 lines around the error site and ±20 lines around the declaration. Bounded walk-up (4 hops) plus a special case for `PropertyAccessExpression` so TS2339 errors resolve to the *receiver's* type, not the non-existent property's. The architectural moat — no other OSS tool does this for TypeScript specifically.
|
|
72
|
+
- **`mendSingleFile(opts)`** — single-LLM repair via Vercel AI SDK + `@ai-sdk/anthropic`. Uses top-level `system:` parameter (v6 pattern), markdown-headered file delimiters in the prompt (XML wrappers caused Claude to mirror them in output and break the parser). Returns `rawResponse`, parsed `blocks`, `apply` result, token counts, latency.
|
|
73
|
+
- **`applySingleBlock(content, search, replace)`** + **`applyEditBlocks(opts)`** + **`parseEditBlocks(text)`** — Aider-style `editblock` parser and 3-tier fuzzy applier (exact → rstrip → strip). Defensive parser handles `<file path="…">` wrappers Claude emits when the system prompt uses XML markers. Abstains on ambiguous matches (multiple hits) rather than guess.
|
|
74
|
+
- **`runMendLoop(opts)`** — bounded retry (default 3 iterations) with no-progress / regression detection via error-signature-set comparison. Streams per-iteration data: `patchesApplied`, `patchesFailed`, `inputTokens`, `outputTokens`, `latencyMs`, `rawResponse`. Stop reasons: `noErrors`, `fixed`, `noProgress`, `regressed`, `maxIterations`.
|
|
75
|
+
|
|
76
|
+
### Added (fixtures + harness)
|
|
77
|
+
- **3 hand-authored minimal Layer-2 fixtures** — `mend-ts2339-property-typo`, `mend-ts7006-implicit-any`, `mend-ts2741-missing-prop`.
|
|
78
|
+
- **2 realistic Layer-2 fixtures** — `realistic-multi-error-user-helpers` (3 errors, 1 file, `taskDescription` populated), `realistic-rename-ripple` (2 errors, 2 files).
|
|
79
|
+
- **30 auto-generated fixtures** via `scripts/generate-fixtures.mjs` (ts-morph AST mutators × 3 codes × 3 seeds × 10 each). Total Layer-2 fixture corpus: **35**.
|
|
80
|
+
- **`benchmark/run-llm-benchmark.ts`** (`npm run benchmark:llm`) — Layer 2 live LLM benchmark against Anthropic. Skips silently with exit 0 when `ANTHROPIC_API_KEY` is unset.
|
|
81
|
+
- **`scripts/generate-fixtures.mjs`** (`npm run generate-fixtures`) — ts-morph AST mutators that introduce one targeted error per fixture into a valid seed file. Validation gate: every mutation runs through `runInProcessTsc` to confirm the expected error code, then through `runValidationLoop` to confirm Layer 0 abstains. Memory-bounded shared `Project` + tempDir + cache resets to prevent OOMs.
|
|
82
|
+
|
|
83
|
+
### Added (tests)
|
|
84
|
+
- **33 unit tests** across `typeContext`, `applyEditBlock`, `mendAgent`, `runMendLoop`. Mocked LLM call via injectable `_callLLM` — tests never hit the real API.
|
|
85
|
+
|
|
86
|
+
### Added (CI)
|
|
87
|
+
- Workflow gains a Layer-2 benchmark step gated on `ANTHROPIC_API_KEY` (skips cleanly when unset). Existing Layer-0 benchmark + matrix steps unchanged.
|
|
88
|
+
- Bumped `actions/checkout` + `actions/setup-node` v4 → v5.
|
|
89
|
+
|
|
90
|
+
### Changed
|
|
91
|
+
- **Dependencies added (runtime):** `@ai-sdk/anthropic@^3.0.44`, `ai@^6.0.86`. Previous "near-zero deps" north star (Layer 0/1 only) is superseded — package now spans Layer 0/1/2.
|
|
92
|
+
- **Dependencies added (dev):** `ts-morph@^28.0.0` (fixture generation).
|
|
93
|
+
- **Public-API surface** at `src/index.ts` extended with the Layer-2 exports listed above. Layer 0/1 surface unchanged — `runValidationLoop`, `runInProcessTsc`, `runLSPFixerPass`, `discoverTsFiles`, and the contract types stay byte-identical.
|
|
94
|
+
- **Roadmap decision D3 reversed** — previous decision was "mend in sister package"; current decision is "mend in-package." Updated in `tsc-defense-roadmap.md`.
|
|
95
|
+
|
|
96
|
+
### Engines
|
|
97
|
+
- Node `>=20.9.0` (unchanged)
|
|
98
|
+
- TypeScript `>=5.0.0` peer (unchanged)
|
|
99
|
+
|
|
100
|
+
### Performance signals (Layer 2, 35-fixture run, claude-haiku-4-5)
|
|
101
|
+
|
|
102
|
+
| Metric | Target | Observed |
|
|
103
|
+
|---|---|---|
|
|
104
|
+
| Pass rate | ≥70% (Haiku floor) | **100%** (35/35) |
|
|
105
|
+
| Iter-1 success | ≥40% | **97%** (34/35) |
|
|
106
|
+
| Cost / fixture | ≤$0.005 | **$0.001 avg** |
|
|
107
|
+
| Latency / fixture | P95 ≤25s | ~1.5s |
|
|
108
|
+
|
|
109
|
+
Caveat: 30 of 35 fixtures are single-error mutations of 3 seeds. Real-world diversity will dent these numbers.
|
|
110
|
+
|
|
7
111
|
## [0.3.0] - 2026-05-07
|
|
8
112
|
|
|
9
113
|
Phase 2 contract release. **Establishes the public types `MendContext`, `LayerEvent`, and `Diagnostic` so a downstream LLM-mend package (e.g. `@shipispec/tsmend`) can consume tsfix's output without redefining the shape.** No behavior changes; purely additive types. Also collapses several dev-only improvements that landed since v0.2.0 into a single release.
|
|
@@ -97,7 +201,8 @@ Initial public release. **Layers 0–1 only** (deterministic detection + auto-fi
|
|
|
97
201
|
- Node `>=20.9.0` (matches VS Code Extension Host runtime)
|
|
98
202
|
- TypeScript `>=5.0.0` (peer dep, must be installed in the consuming workspace)
|
|
99
203
|
|
|
100
|
-
[Unreleased]: https://github.com/owgreen-dev/tsfix/compare/v0.
|
|
204
|
+
[Unreleased]: https://github.com/owgreen-dev/tsfix/compare/v0.4.0...HEAD
|
|
205
|
+
[0.4.0]: https://github.com/owgreen-dev/tsfix/compare/v0.3.0...v0.4.0
|
|
101
206
|
[0.3.0]: https://github.com/owgreen-dev/tsfix/compare/v0.2.0...v0.3.0
|
|
102
207
|
[0.2.0]: https://github.com/owgreen-dev/tsfix/compare/v0.1.1...v0.2.0
|
|
103
208
|
[0.1.1]: https://github.com/owgreen-dev/tsfix/compare/v0.1.0...v0.1.1
|
package/README.md
CHANGED
|
@@ -1,12 +1,15 @@
|
|
|
1
1
|
# tsfix
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Two-layer TypeScript error recovery for LLM-generated code — fix `TS2304`, `TS2305`, `TS2551`, `TS2552`, `TS2724` deterministically with the same engine that powers VS Code's Quick Fix, and escalate the rest to a single-file LLM mend.
|
|
4
4
|
|
|
5
|
-
`@shipispec/tsfix`
|
|
5
|
+
`@shipispec/tsfix` is what you reach for when you've just generated a few hundred files of TypeScript with an LLM and `tsc --noEmit` is screaming at you. It runs in two layers:
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
- **Layer 0/1** — Deterministic. Borrows the same TypeScript Language Service that powers VS Code's "Quick Fix" lightbulb and runs it as a CLI. Fixes typos, missing imports, and did-you-mean errors with no LLM, no network, no config.
|
|
8
|
+
- **Layer 2** — Opt-in. A single-file LLM mend agent (Vercel AI SDK + Anthropic) that picks up what Layer 0 abstains on: TS2339 (property doesn't exist), TS7006 (implicit `any`), TS2741 (missing required prop), and other cases where the LSP can't statically derive the fix. Driven by **type-context injection** — when tsc says "Property 'foo' doesn't exist on type 'Bar'", tsfix resolves the `Bar` declaration via the TypeChecker and feeds its source to the model.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
Layer 2 only runs if you explicitly call its API or set `ANTHROPIC_API_KEY` and use the `runMendLoop` entry point. The default `tsfix --workspace ...` CLI is still **Layer 0/1 only**.
|
|
11
|
+
|
|
12
|
+
## Before / after (Layer 0)
|
|
10
13
|
|
|
11
14
|
```
|
|
12
15
|
$ tsc --noEmit
|
|
@@ -62,7 +65,9 @@ npx @shipispec/tsfix --workspace . --json
|
|
|
62
65
|
| `--verbose` | Per-fix logging. |
|
|
63
66
|
| `--help` | Print usage. |
|
|
64
67
|
|
|
65
|
-
|
|
68
|
+
The CLI does not run Layer 2 — call the library API for that (below).
|
|
69
|
+
|
|
70
|
+
## What Layer 0 fixes
|
|
66
71
|
|
|
67
72
|
| TS code | Meaning | What tsfix does |
|
|
68
73
|
|---|---|---|
|
|
@@ -72,11 +77,11 @@ npx @shipispec/tsfix --workspace . --json
|
|
|
72
77
|
| `TS2552` | Cannot find name, did you mean Y | Spelling fix |
|
|
73
78
|
| `TS2724` | Module member did-you-mean | Import rename |
|
|
74
79
|
|
|
75
|
-
Against a 14-fixture benchmark spanning typos, did-you-mean cases, multi-file ripples, and 4 API-drift scenarios: **14/14 fixtures pass and 14/25 errors are auto-fixed (56%).** The remaining errors are intentionally outside Layer 0's scope
|
|
80
|
+
Against a 14-fixture benchmark spanning typos, did-you-mean cases, multi-file ripples, and 4 API-drift scenarios: **14/14 fixtures pass and 14/25 errors are auto-fixed (56%).** The remaining errors are intentionally outside Layer 0's scope and escape to Layer 2.
|
|
76
81
|
|
|
77
|
-
## What
|
|
82
|
+
## What Layer 0 does *not* fix (Layer 2 picks these up)
|
|
78
83
|
|
|
79
|
-
By design,
|
|
84
|
+
By design, Layer 0 only applies fixes that are **deterministic** and **non-structural**. It refuses to:
|
|
80
85
|
|
|
81
86
|
- Add or remove function declarations
|
|
82
87
|
- Insert type annotations or change types
|
|
@@ -84,25 +89,33 @@ By design, tsfix only applies fixes that are **deterministic** and **non-structu
|
|
|
84
89
|
- Rewrite JSX trees
|
|
85
90
|
- Add object-literal stub properties
|
|
86
91
|
|
|
87
|
-
The internal allowlist is two-layered: error codes (`SAFE_FIXABLE_CODES`) and Quick Fix names (`SAFE_FIX_NAMES = ['import', 'fixImport', 'spelling', 'fixSpelling']`). When the language service offers anything outside that allowlist,
|
|
92
|
+
The internal allowlist is two-layered: error codes (`SAFE_FIXABLE_CODES`) and Quick Fix names (`SAFE_FIX_NAMES = ['import', 'fixImport', 'spelling', 'fixSpelling']`). When the language service offers anything outside that allowlist, Layer 0 abstains and surfaces the error so Layer 2 (or a human) can pick it up.
|
|
88
93
|
|
|
89
|
-
|
|
94
|
+
Layer 2 is built for the cases the LSP can't statically resolve:
|
|
90
95
|
|
|
91
|
-
|
|
96
|
+
- `TS2339` — Property doesn't exist on type. The LLM needs to see *the type's declaration* to decide whether the receiver should grow a field, the call site has a typo with no near-match, or the receiver is the wrong type entirely.
|
|
97
|
+
- `TS7006` — Implicit `any`. The LLM picks the right annotation from surrounding context.
|
|
98
|
+
- `TS2741` — Missing required property. The LLM sees the contextual type and supplies a real value, not a placeholder.
|
|
99
|
+
|
|
100
|
+
Against a 35-fixture Layer-2 benchmark (3 hand-authored minimal + 2 realistic + 30 ts-morph-generated mutations across TS2339/TS7006/TS2741), **35/35 pass at $0.001/fixture avg, P95 latency ~1.5s on `claude-haiku-4-5`.** Caveat: the 30 generated fixtures are mutations of 3 seeds — real-world diversity will move these numbers.
|
|
101
|
+
|
|
102
|
+
## The four-layer model
|
|
92
103
|
|
|
93
104
|
```
|
|
94
105
|
Layer 0 — Prevention (prompt rules, exported-API injection — your problem)
|
|
95
|
-
Layer 1 —
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
Layer 3 — Multi-file LLM
|
|
99
|
-
Layer 4 — Stub-and-continue
|
|
106
|
+
Layer 1 — Deterministic (this package: LSP auto-fix, CLI default)
|
|
107
|
+
Layer 2 — Single-file LLM (this package: opt-in via library API)
|
|
108
|
+
─────────────────────────────────────────────────────────────────
|
|
109
|
+
Layer 3 — Multi-file LLM (planned: blast-radius search/replace via findReferences)
|
|
110
|
+
Layer 4 — Stub-and-continue (planned: escape hatch)
|
|
100
111
|
```
|
|
101
112
|
|
|
102
|
-
The bet: roughly half of TypeScript errors in LLM output are deterministically fixable. By catching them in Layer 1
|
|
113
|
+
The bet: roughly half of TypeScript errors in LLM output are deterministically fixable. By catching them in Layer 1 you dodge the LLM tax (latency, cost, nondeterminism) on the easy half. Layer 2 takes the other half — but only when you explicitly invoke it.
|
|
103
114
|
|
|
104
115
|
## Library API
|
|
105
116
|
|
|
117
|
+
### Layer 0/1 — deterministic loop
|
|
118
|
+
|
|
106
119
|
```typescript
|
|
107
120
|
import { runValidationLoop } from '@shipispec/tsfix';
|
|
108
121
|
|
|
@@ -121,24 +134,62 @@ result.lspFixer.filesEdited; // string[]
|
|
|
121
134
|
result.passed; // boolean — true if errorsAfter === 0
|
|
122
135
|
```
|
|
123
136
|
|
|
124
|
-
Other exports:
|
|
137
|
+
Other Layer 0/1 exports:
|
|
125
138
|
|
|
126
139
|
- `runInProcessTsc(opts)` — validation only, no fixer. Returns structured diagnostics.
|
|
127
140
|
- `runLSPFixerPass(opts)` — Layer 0 fixer alone, no validation loop wrapper.
|
|
128
141
|
- `discoverTsFiles(workspaceRoot)` — file-walking helper. Skips `node_modules`, `.next`, `dist`, `build`, `out`, `coverage`, `.git`.
|
|
129
142
|
|
|
143
|
+
### Layer 2 — LLM mend (opt-in)
|
|
144
|
+
|
|
145
|
+
```typescript
|
|
146
|
+
import { runValidationLoop, runMendLoop } from '@shipispec/tsfix';
|
|
147
|
+
|
|
148
|
+
// Layer 0/1 first.
|
|
149
|
+
const layer1 = runValidationLoop({ workspaceRoot });
|
|
150
|
+
|
|
151
|
+
if (!layer1.passed) {
|
|
152
|
+
// Layer 2 escalation.
|
|
153
|
+
const layer2 = await runMendLoop({
|
|
154
|
+
context: {
|
|
155
|
+
workspaceRoot,
|
|
156
|
+
diagnostics: layer1.remainingDiagnostics,
|
|
157
|
+
erroredFiles: layer1.lspFixer.filesWithErrors,
|
|
158
|
+
// Optional fields that improve mend quality:
|
|
159
|
+
// taskDescription: 'Build a user CRUD module',
|
|
160
|
+
// featureSpecText: '...the markdown spec...',
|
|
161
|
+
// acceptanceCriteria: '...',
|
|
162
|
+
// installedTypes: '...', // compact API surface from npm deps
|
|
163
|
+
},
|
|
164
|
+
llm: {
|
|
165
|
+
provider: 'anthropic',
|
|
166
|
+
model: 'claude-haiku-4-5',
|
|
167
|
+
apiKey: process.env.ANTHROPIC_API_KEY,
|
|
168
|
+
},
|
|
169
|
+
maxIterations: 3,
|
|
170
|
+
});
|
|
171
|
+
|
|
172
|
+
console.log(layer2.stopReason); // 'fixed' | 'noProgress' | 'regressed' | 'maxIterations'
|
|
173
|
+
console.log(layer2.totalCostUsd);
|
|
174
|
+
}
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Other Layer 2 exports:
|
|
178
|
+
|
|
179
|
+
- `mendSingleFile(opts)` — one LLM call for one file. The building block under `runMendLoop`.
|
|
180
|
+
- `getTypeContext(opts)` — resolve a `Diagnostic` to its declaring type via the TS Language Service and return ±N lines around the declaration. The architectural moat — every other LLM-driven repair tool uses generic grep or repo-maps.
|
|
181
|
+
- `parseEditBlocks(text)` / `applyEditBlocks(opts)` — Aider-style SEARCH/REPLACE patch parser + 3-tier fuzzy applier.
|
|
182
|
+
- Types: `MendContext`, `LayerEvent`, `Diagnostic`, plus the per-function option/result types.
|
|
183
|
+
|
|
130
184
|
## Trust model
|
|
131
185
|
|
|
132
|
-
|
|
186
|
+
Layer 0/1 loads `typescript` from your workspace's `node_modules` — it does **not** bundle its own. This ensures the fixer behaves identically to the `tsc` your project actually compiles with.
|
|
133
187
|
|
|
134
188
|
> **Run tsfix only on workspaces you trust.** Loading `typescript` from an attacker-controlled `node_modules` is equivalent to running `node_modules/.bin/tsc` against it.
|
|
135
189
|
|
|
136
|
-
|
|
190
|
+
**Network surface (Layer 0/1):** none. No telemetry, no calls home, no background processes, no config files written outside `--workspace`.
|
|
137
191
|
|
|
138
|
-
-
|
|
139
|
-
- No telemetry.
|
|
140
|
-
- No background processes.
|
|
141
|
-
- No config files written or modified outside `--workspace`.
|
|
192
|
+
**Network surface (Layer 2):** every `mendSingleFile` call hits Anthropic's API via the Vercel AI SDK. The source files in `MendContext.erroredFiles` and the resolved type-context slices are sent in the prompt. If your code is sensitive, do not call Layer 2 — the CLI never does, and the library exports are explicit.
|
|
142
193
|
|
|
143
194
|
## Engines
|
|
144
195
|
|
|
@@ -154,7 +205,7 @@ run: npm install --save-dev typescript
|
|
|
154
205
|
|
|
155
206
|
## Contributing
|
|
156
207
|
|
|
157
|
-
|
|
208
|
+
### Adding a Layer-0 fix
|
|
158
209
|
|
|
159
210
|
1. **Probe** — write a tiny test workspace with the exact error you want fixable. Drop it under `fixtures/<descriptive-name>/` with an `expected.json` declaring `errorsBefore`, `errorsAfterMax`, `lspFixesAppliedMin/Max`, and `mustPass`.
|
|
160
211
|
2. **Verify** — run `npm run benchmark -- --fixture <name>` and inspect what the language service offers (the `fix.fixName` field).
|
|
@@ -163,7 +214,18 @@ The contract for adding a fix:
|
|
|
163
214
|
|
|
164
215
|
Each new code/fix-name pair gets its own fixture. We don't trust the language service blindly — we trust it under specific, pinned conditions.
|
|
165
216
|
|
|
166
|
-
|
|
217
|
+
### Adding a Layer-2 fixture
|
|
218
|
+
|
|
219
|
+
Layer-2 fixtures live under `fixtures/` alongside Layer-0 ones, identified by `expectedErrorCode` (singular) or `costUsdMax` in their `expected.json`. The Layer-0 benchmark skips them; `npm run benchmark:llm` runs them against Anthropic.
|
|
220
|
+
|
|
221
|
+
- Hand-author one under `fixtures/mend-<descriptive-name>/` for new error classes.
|
|
222
|
+
- Or generate one via `npm run generate-fixtures -- --code=TS2339 --seed=apiRouter.ts --count=10 --rng-seed=42`. The generator validates every mutation through Layer 0 first to confirm Layer 0 abstains (otherwise it's not Layer 2 territory).
|
|
223
|
+
|
|
224
|
+
### Pre-publish gates
|
|
225
|
+
|
|
226
|
+
- `npm run benchmark` — Layer 0, 14 fixtures, no network.
|
|
227
|
+
- `npm run benchmark:llm` — Layer 2, 35 fixtures, requires `ANTHROPIC_API_KEY`. Total cost ~$0.04 per run.
|
|
228
|
+
- `npm run matrix` — runs the local tarball against 6 distinct project shapes (Next.js, Vite + React, plain `nodenext`, plain `bundler`, plain CommonJS, monorepo with project references). Adds ~3 min; run manually before tagging.
|
|
167
229
|
|
|
168
230
|
## License
|
|
169
231
|
|