@autotelic/joggle 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/JOGGLE.md +634 -0
- package/LICENSE +21 -0
- package/README.md +88 -2
- package/bin/joggle.js +6 -0
- package/dist/answer-store.js +305 -0
- package/dist/architecture.js +190 -0
- package/dist/ask.js +184 -0
- package/dist/astro.js +75 -0
- package/dist/atoms.js +76 -0
- package/dist/bundles.js +156 -0
- package/dist/calibration.js +60 -0
- package/dist/candidates.js +561 -0
- package/dist/canonical.js +24 -0
- package/dist/cascade.js +98 -0
- package/dist/check.js +854 -0
- package/dist/cluster.js +69 -0
- package/dist/config.js +195 -0
- package/dist/content-mappers.js +83 -0
- package/dist/decision.js +203 -0
- package/dist/fingerprint.js +61 -0
- package/dist/git.js +127 -0
- package/dist/gitignore.js +65 -0
- package/dist/glob.js +51 -0
- package/dist/imports.js +153 -0
- package/dist/jsonc.js +95 -0
- package/dist/main.js +630 -0
- package/dist/manifest.js +42 -0
- package/dist/meta.js +170 -0
- package/dist/moves.js +47 -0
- package/dist/operation.js +133 -0
- package/dist/parsecache.js +264 -0
- package/dist/plans.js +425 -0
- package/dist/plugin.js +71 -0
- package/dist/plugins.js +136 -0
- package/dist/policy.js +676 -0
- package/dist/presets/composition.js +32 -0
- package/dist/report.js +474 -0
- package/dist/reporting.js +132 -0
- package/dist/result.js +28 -0
- package/dist/roles.js +303 -0
- package/dist/rule.js +194 -0
- package/dist/rules/append-only-honoured.js +190 -0
- package/dist/rules/bundle-conformance.js +245 -0
- package/dist/rules/call-pattern.js +209 -0
- package/dist/rules/cluster-verdict.js +700 -0
- package/dist/rules/compose-types.js +250 -0
- package/dist/rules/data-error-as-outage.js +482 -0
- package/dist/rules/declared-staleness.js +242 -0
- package/dist/rules/dependency-fit.js +297 -0
- package/dist/rules/deviation-has-a-shape.js +225 -0
- package/dist/rules/doc-matches-code.js +191 -0
- package/dist/rules/duplicate-call-run.js +240 -0
- package/dist/rules/duplicate-implementation.js +89 -0
- package/dist/rules/duplicate-meaning.js +130 -0
- package/dist/rules/fact-or-derivation.js +202 -0
- package/dist/rules/field-type-drift.js +373 -0
- package/dist/rules/generic-carries-a-caller.js +198 -0
- package/dist/rules/hoist-to-domain.js +288 -0
- package/dist/rules/import-architecture.js +393 -0
- package/dist/rules/index.js +105 -0
- package/dist/rules/inferred-over-recorded.js +214 -0
- package/dist/rules/intent-drift.js +286 -0
- package/dist/rules/invariant-has-a-reconciler.js +190 -0
- package/dist/rules/language-drift.js +234 -0
- package/dist/rules/meaning-switched-by-flag.js +189 -0
- package/dist/rules/module-direction.js +231 -0
- package/dist/rules/name-as-address.js +216 -0
- package/dist/rules/name-the-primitive.js +203 -0
- package/dist/rules/naming-drift.js +218 -0
- package/dist/rules/nullability-drift.js +220 -0
- package/dist/rules/object-shape.js +280 -0
- package/dist/rules/one-concept-one-type.js +243 -0
- package/dist/rules/page-needs-composition.js +227 -0
- package/dist/rules/reimplemented-primitive.js +283 -0
- package/dist/rules/reversal.js +264 -0
- package/dist/rules/rule-judgment.js +252 -0
- package/dist/rules/schema-excludes-domain-value.js +190 -0
- package/dist/rules/shallow-module.js +204 -0
- package/dist/rules/single-path.js +237 -0
- package/dist/rules/structural-collapse.js +198 -0
- package/dist/rules/temporal-coupling.js +205 -0
- package/dist/rules/types-over-logic.js +198 -0
- package/dist/rules/unaccounted-drop.js +190 -0
- package/dist/rules/unbounded-default-read.js +227 -0
- package/dist/run-cache.js +132 -0
- package/dist/schema.js +285 -0
- package/dist/similarity.js +119 -0
- package/dist/source.js +30 -0
- package/dist/sql.js +34 -0
- package/dist/stages.js +39 -0
- package/dist/state.js +65 -0
- package/dist/testing.js +131 -0
- package/dist/tsconfig-scope.js +32 -0
- package/dist/tsgo.js +127 -0
- package/dist/typefacts.js +455 -0
- package/dist/typetrace.js +283 -0
- package/dist/verdict.js +65 -0
- package/dist/vocabulary.js +365 -0
- package/dist/workspace.js +1963 -0
- package/extensions/joggle.ts +454 -0
- package/package.json +85 -3
package/JOGGLE.md
ADDED
|
@@ -0,0 +1,634 @@
|
|
|
1
|
+
# joggle
|
|
2
|
+
|
|
3
|
+
> Cross-file patterns and idioms for TypeScript, enforced like a linter.
|
|
4
|
+
|
|
5
|
+
`tsc` checks one program. `oxlint` checks one file. Neither can tell you that
|
|
6
|
+
two functions in different modules are the same concept, that a name means
|
|
7
|
+
something different here than it does there, or that a module is not shaped
|
|
8
|
+
like the other twelve modules of its kind.
|
|
9
|
+
|
|
10
|
+
joggle is the layer above both. It indexes the codebase into facts, generates
|
|
11
|
+
candidates cheaply and deterministically, and then asks narrow typed questions
|
|
12
|
+
about the candidates it cannot decide by looking. Answers come back as
|
|
13
|
+
probabilities, code keeps control of the thresholds, and the output is an
|
|
14
|
+
ordinary diagnostic -- so the same tool serves an editor, a pre-commit hook and
|
|
15
|
+
CI.
|
|
16
|
+
|
|
17
|
+
The design rule is one sentence: **be deterministic where you can prove, and
|
|
18
|
+
judge only where you must.** What the tool is for -- the entropy reverser, and
|
|
19
|
+
the three moves a finding can propose -- is [docs/thesis.md](./docs/thesis.md).
|
|
20
|
+
|
|
21
|
+
## Where it sits
|
|
22
|
+
|
|
23
|
+
| Tool | Unit | Question | Output |
|
|
24
|
+
| --- | --- | --- | --- |
|
|
25
|
+
| `tsc` / `tsgo` | the program | is this well-typed? | diagnostics |
|
|
26
|
+
| `oxlint` | the file and its AST | is this well-formed? | diagnostics |
|
|
27
|
+
| **joggle** | the codebase and its facts | is this the same thing as that? | diagnostics |
|
|
28
|
+
|
|
29
|
+
Because the output shape is identical, a host does not have to care which kind
|
|
30
|
+
of rule produced a finding. There is no separate report, no separate format,
|
|
31
|
+
and no separate gate.
|
|
32
|
+
|
|
33
|
+
## The shape of a rule
|
|
34
|
+
|
|
35
|
+
Every rule is a file with the same five moves.
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
find deterministic, high recall, noise tolerated
|
|
39
|
+
|
|
|
40
|
+
v
|
|
41
|
+
evidence the panel a reviewer would need in order to decide
|
|
42
|
+
|
|
|
43
|
+
v
|
|
44
|
+
questions atomic, typed (Noul / Choice), all sent in one request
|
|
45
|
+
|
|
|
46
|
+
v
|
|
47
|
+
policy weights and thresholds, imported from policy.ts
|
|
48
|
+
|
|
|
49
|
+
v
|
|
50
|
+
diagnose span, message, help, confidence
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
A rule's deterministic half is its **candidate generation**: a predicate over the
|
|
54
|
+
facts, free and high-recall. The verdict is a question. Every rule in the set is
|
|
55
|
+
judged, because a check that is not a question belongs in the linter or the
|
|
56
|
+
typechecker, and a meta-rule rejects one.
|
|
57
|
+
|
|
58
|
+

|
|
61
|
+
|
|
62
|
+
*The diagram is generated by `scripts/diagram-rules.mjs`.*
|
|
63
|
+
|
|
64
|
+
## Running it
|
|
65
|
+
|
|
66
|
+
```sh
|
|
67
|
+
pnpm install
|
|
68
|
+
|
|
69
|
+
pnpm joggle rules # what is enforced
|
|
70
|
+
pnpm joggle check src # analyse a directory
|
|
71
|
+
pnpm joggle check --typecheck # carry tsgo diagnostics too
|
|
72
|
+
pnpm joggle check --format github # GitHub Actions annotations
|
|
73
|
+
pnpm joggle check --offline # replay from cache, no network
|
|
74
|
+
pnpm joggle check --rule joggle/naming-drift # one rule
|
|
75
|
+
pnpm joggle check --max-warnings 0 # warnings fail the build
|
|
76
|
+
pnpm joggle check --since origin/main # only what this branch changed
|
|
77
|
+
pnpm joggle check --pr # only what this branch's PR changed
|
|
78
|
+
pnpm joggle check --pr-number 123 # only what PR 123 changed
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
With no paths, joggle asks `tsgo --listFilesOnly` what the project is, so it
|
|
82
|
+
analyses exactly what the compiler sees rather than whatever is on disk. With
|
|
83
|
+
paths, it walks them itself.
|
|
84
|
+
|
|
85
|
+
### Installing the command
|
|
86
|
+
|
|
87
|
+
The simplest way to use joggle from another workspace is to put it on `PATH`
|
|
88
|
+
once:
|
|
89
|
+
|
|
90
|
+
```sh
|
|
91
|
+
cd path/to/joggle && pnpm install && pnpm build
|
|
92
|
+
ln -sf "$PWD/scripts/joggle.sh" ~/.local/bin/joggle
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Then, from any checkout:
|
|
96
|
+
|
|
97
|
+
```sh
|
|
98
|
+
joggle check --since origin/main
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The wrapper resolves its own checkout through the symlink and runs the build
|
|
102
|
+
(falling back to the source). It looks for the TypeSafe key in three places, in
|
|
103
|
+
order: the environment, a gitignored `.env.local` beside the wrapper, and
|
|
104
|
+
doppler. The doppler fetch runs from the joggle root, because doppler resolves a
|
|
105
|
+
project from the **working directory's** `doppler.yaml` and a sibling
|
|
106
|
+
repository's setup wins over explicit `--project`/`--config` (and over
|
|
107
|
+
`DOPPLER_PROJECT` / `DOPPLER_CONFIG`); it also restores `HOME` from the passwd
|
|
108
|
+
database, because a host such as pi can spawn the command with no `HOME` and
|
|
109
|
+
doppler needs it to find its own auth. `JOGGLE_DOPPLER_PROJECT` and
|
|
110
|
+
`JOGGLE_DOPPLER_CONFIG` override the defaults `joggle` and `dev`. Commands that
|
|
111
|
+
need no key (`rules`, help, `--offline`) skip the lookup entirely, and a key
|
|
112
|
+
that cannot be found degrades to unverified findings -- with a warning on
|
|
113
|
+
stderr, which the pi tools surface in their result -- rather than taking the
|
|
114
|
+
tool down.
|
|
115
|
+
|
|
116
|
+
### From another repository
|
|
117
|
+
|
|
118
|
+
joggle runs its source in development, and its build when installed. Build once,
|
|
119
|
+
then link it into the repository you are working in:
|
|
120
|
+
|
|
121
|
+
```sh
|
|
122
|
+
cd path/to/joggle && pnpm install && pnpm build
|
|
123
|
+
|
|
124
|
+
cd path/to/other-repo
|
|
125
|
+
pnpm add -D file:/absolute/path/to/joggle # or: pnpm link /absolute/path/to/joggle
|
|
126
|
+
|
|
127
|
+
pnpm exec joggle check --pr # after cutting a PR
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
`file:` installs a copy of `dist/`, so nothing in the other repository has to
|
|
131
|
+
understand TypeScript. `pnpm link` symlinks the source, which works because the
|
|
132
|
+
symlink's real path sits outside `node_modules` -- but a `file:` install needs
|
|
133
|
+
`dist/`, and `pnpm build` is what makes it.
|
|
134
|
+
|
|
135
|
+
### Inspecting a checkout you do not want to touch
|
|
136
|
+
|
|
137
|
+
A repository is only modified if it is being onboarded. To analyse a sibling
|
|
138
|
+
checkout without writing anything into it:
|
|
139
|
+
|
|
140
|
+
```sh
|
|
141
|
+
pnpm joggle:elsewhere ../shakti-v2 --since origin/develop --offline
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`scripts/joggle-elsewhere.sh` runs `check --cwd <path>` and sends the answer
|
|
145
|
+
cache to the machine cache **unless the target already has a committed
|
|
146
|
+
`.joggle/`**, in which case the repository's own cache is used so a replay still
|
|
147
|
+
works. Nothing else is written: the run cache, the parse cache and any type
|
|
148
|
+
trace already live under the machine cache, and the default run makes no network
|
|
149
|
+
call when there is no key.
|
|
150
|
+
|
|
151
|
+
### As a pi extension
|
|
152
|
+
|
|
153
|
+
The repository is also a [pi](https://pi.dev) package. Install it once and pi
|
|
154
|
+
gains two tools and a command in every repository it works in:
|
|
155
|
+
|
|
156
|
+
```sh
|
|
157
|
+
pi install /absolute/path/to/joggle # or: pi -e /absolute/path/to/joggle
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
| Name | What it does |
|
|
161
|
+
| --- | --- |
|
|
162
|
+
| `joggle_check` | Run a scoped check and return the findings. Defaults to the changed scope, so it answers what the current batch of work introduced; `scope: "pr"` answers what a pull request introduced. Returns at most 50 findings; pass `limit` to change that, or `0` for all of them. |
|
|
163
|
+
| `joggle_rules` | List the rules a repository enforces. |
|
|
164
|
+
| `/joggle` | Run a check from the prompt line; arguments pass through, e.g. `/joggle --pr`. |
|
|
165
|
+
|
|
166
|
+
Both tools take a `cwd`, so a session in one repository can check a sibling:
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
joggle_check { cwd: "../shakti-v2", scope: "since", since: "origin/develop" }
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
`joggle_check` follows the same non-invasive cache rule as the script above: it
|
|
173
|
+
passes `--cache-dir` to the machine cache when the target has no `.joggle/`, and
|
|
174
|
+
uses the committed cache when it does. Pass an explicit `cacheDir` to override.
|
|
175
|
+
The extension is a shell over the CLI in this checkout (`dist/main.js`, falling
|
|
176
|
+
back to `src/main.ts`), and `JOGGLE_BIN` points it at a different build or a
|
|
177
|
+
wrapper.
|
|
178
|
+
|
|
179
|
+
Output formats follow oxlint, because oxlint already decided what a linter's
|
|
180
|
+
output should be and its decisions are worth copying:
|
|
181
|
+
|
|
182
|
+
| Format | Shape | For |
|
|
183
|
+
| --- | --- | --- |
|
|
184
|
+
| `text` (default) | `path:line:col: severity rule: message help: ...`, one line per problem | terminals and CI logs |
|
|
185
|
+
| `stylish` | grouped per file, aligned columns, colour on a TTY | reading by hand |
|
|
186
|
+
| `unix` | `path:line:col: message [Severity/rule]` plus a count | editors and scripts |
|
|
187
|
+
| `json` | everything, including per-answer confidence | another tool |
|
|
188
|
+
| `github` | Actions workflow commands | annotations |
|
|
189
|
+
|
|
190
|
+
A hand-rolled code frame was removed in favour of the compact default. On a
|
|
191
|
+
1,870-file codebase the frames made the report 8,083 lines; oxlint's shape
|
|
192
|
+
prints 343. A linter's output is read on a terminal and scrolled in CI, and
|
|
193
|
+
those two want different things.
|
|
194
|
+
|
|
195
|
+
| Flag | Meaning |
|
|
196
|
+
| --- | --- |
|
|
197
|
+
| `--rule <ids>` | comma-separated rule ids |
|
|
198
|
+
| `--format text\|stylish\|unix\|json\|github` | output format (see below) |
|
|
199
|
+
| `--max-warnings <n>` | fail when warnings exceed `n` (`-1` disables) |
|
|
200
|
+
| `--typecheck` | include tsgo diagnostics as `joggle/typecheck` |
|
|
201
|
+
| `--no-tsgo` | never invoke tsgo |
|
|
202
|
+
| `--run-external-code` | let `tsc` execute content-mapper packages declared in tsconfig (TypeScript 7.1+); off by default |
|
|
203
|
+
| `--offline` | answer only from the judgement cache |
|
|
204
|
+
| `--changed` | scope to what changed since the stored run |
|
|
205
|
+
| `--since <rev>` | scope to what changed since a git revision, working tree included |
|
|
206
|
+
| `--pr` | scope to the pull request for the current branch (resolved with `gh`) |
|
|
207
|
+
| `--pr-number <n\|url>` | scope to a specific pull request |
|
|
208
|
+
| `--cache-dir <path>` | where the answer cache lives: `<cwd>/.joggle` when it exists, else the machine cache |
|
|
209
|
+
| `--cwd <path>` | project root |
|
|
210
|
+
|
|
211
|
+
Content mappers are TypeScript 7.1's plugin point for `.vue`, `.svelte`, `.mdx`
|
|
212
|
+
and similar files. joggle does not run them unless `--run-external-code` (or
|
|
213
|
+
`"runExternalCode": true` in `joggle.config.json`) says so: a mapper is arbitrary
|
|
214
|
+
code named by the repository. Even with it on, joggle's own parser indexes only
|
|
215
|
+
TypeScript and Astro, so `check` reports a note naming every declared extension
|
|
216
|
+
it did not index. [docs/content-mappers.md](./docs/content-mappers.md) has the
|
|
217
|
+
full contract.
|
|
218
|
+
|
|
219
|
+
## CI and replay
|
|
220
|
+
|
|
221
|
+
A judgement is a network call, so the cache key is the contract:
|
|
222
|
+
|
|
223
|
+
```
|
|
224
|
+
question version + model + evidence + questions
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
It is serialised with sorted keys, so the same candidate produces the same key
|
|
228
|
+
on every machine — which means **run from the repository root**. Evidence
|
|
229
|
+
carries root-relative paths; running from a subdirectory changes the key and
|
|
230
|
+
misses every cached verdict. Successful judgements are written to
|
|
231
|
+
`.joggle/answers/`, one sharded, sorted, line-oriented file per key prefix;
|
|
232
|
+
commit that directory and CI replays them with `--offline` and **no API key at
|
|
233
|
+
all**. A new answer is a new line in a shard it shares with few others, so two
|
|
234
|
+
branches that judge different candidates merge as a directory merge, and
|
|
235
|
+
`joggle cache prune` keeps the store from growing without a ceiling. A legacy
|
|
236
|
+
single `answers.json` is still read, and the next write migrates it. The layout,
|
|
237
|
+
the merge driver and the deployment modes are in
|
|
238
|
+
[docs/artifacts.md](./docs/artifacts.md). Bump `policy.decisionVersion` when you
|
|
239
|
+
change a question's wording, and every cached answer for it is invalidated at
|
|
240
|
+
once instead of silently replayed against newer questions.
|
|
241
|
+
|
|
242
|
+
When the judge is unavailable, the rule's own `onUnavailable` decides, and a
|
|
243
|
+
repository may override it for the whole run with `unavailable`, or per rule with
|
|
244
|
+
`onUnavailable`:
|
|
245
|
+
|
|
246
|
+
| Answer | What happens |
|
|
247
|
+
| --- | --- |
|
|
248
|
+
| `report` (the default) | the rule still reports; each candidate is an unverified finding |
|
|
249
|
+
| `propagate` | the rule steps aside and the engine reports it as skipped |
|
|
250
|
+
| `count` | the candidates are counted in one note and not printed |
|
|
251
|
+
|
|
252
|
+
A repository that runs without a key in CI sets `"unavailable": "count"` once and
|
|
253
|
+
the whole judged layer becomes a count; a developer's machine with a key is
|
|
254
|
+
unaffected. Answers already cached and replayed with `--offline` are judgements,
|
|
255
|
+
so none of this applies to them. A missing key degrades the gate; it does not
|
|
256
|
+
disable it.
|
|
257
|
+
|
|
258
|
+
### Where the key goes
|
|
259
|
+
|
|
260
|
+
The key is read from the process environment only, so it never appears in the
|
|
261
|
+
repository or in any config file joggle owns.
|
|
262
|
+
|
|
263
|
+
Locally, use the secret manager the team already has:
|
|
264
|
+
|
|
265
|
+
```sh
|
|
266
|
+
doppler run -- pnpm joggle check src # recommended
|
|
267
|
+
direnv allow # or a gitignored .envrc.local
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
In CI, add `TYPESAFE_API_KEY` as a repository secret and map it onto the step
|
|
271
|
+
that judges new candidates. The replay job needs no secret at all.
|
|
272
|
+
|
|
273
|
+
Two properties make this safe to live with:
|
|
274
|
+
|
|
275
|
+
* `.joggle/judgements.json` holds questions, evidence and answers. It never
|
|
276
|
+
holds the key, so it is safe to commit and review.
|
|
277
|
+
* Because the cache is the replay path, **the judged run can be done by a
|
|
278
|
+
person and the result handed to CI or to an agent**. Neither of them ever
|
|
279
|
+
needs the credential; they work from the committed verdicts.
|
|
280
|
+
|
|
281
|
+
If the key is ever pasted into a file, into chat, or into a shell that gets
|
|
282
|
+
logged, rotate it in the TypeSafe console. This repository is private, so
|
|
283
|
+
GitHub's secret scanning will not catch it for you.
|
|
284
|
+
|
|
285
|
+
### What leaves the machine
|
|
286
|
+
|
|
287
|
+
The judged rules send the evidence panel -- including source excerpts of the
|
|
288
|
+
declarations being compared -- to `api.typesafe.ai`. Deterministic rules and
|
|
289
|
+
`--offline` send nothing.
|
|
290
|
+
|
|
291
|
+
Decide this per repository. For a client codebase, either point
|
|
292
|
+
`TYPESAFE_BASE_URL` at a deployment you control, or run the judged pass over
|
|
293
|
+
public and fixture code only and let the committed replay cache carry the
|
|
294
|
+
verdicts into the client's CI.
|
|
295
|
+
|
|
296
|
+
```sh
|
|
297
|
+
TYPESAFE_BASE_URL=https://typesafe.internal.example doppler run -- pnpm joggle check src
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
## The judgement
|
|
301
|
+
|
|
302
|
+
The evidence panel is the System One `state`; the questions are the typed
|
|
303
|
+
primitives. Everything the model returns is constrained to the options the rule
|
|
304
|
+
supplied, so nothing has to be recovered from generated prose.
|
|
305
|
+
|
|
306
|
+
- `Noul` -- the probability that a yes/no statement holds.
|
|
307
|
+
- `Choice` -- one option out of a set, with the full distribution and a
|
|
308
|
+
confidence.
|
|
309
|
+
|
|
310
|
+
Thresholds live in `src/policy.ts` and nowhere else. When a verdict stops
|
|
311
|
+
matching what the team would decide, you change a weight, re-run, and see the
|
|
312
|
+
difference -- you do not rewrite a prompt.
|
|
313
|
+
|
|
314
|
+
### What the judge is actually for
|
|
315
|
+
|
|
316
|
+
Code finds; the model verifies. This is the opposite of how joggle started, and
|
|
317
|
+
the first real measurements are what corrected it.
|
|
318
|
+
|
|
319
|
+
Findings were being produced two ways: exact shape equality (a fact, but
|
|
320
|
+
reported unverified) and a hand-tuned near-duplicate search that asked the model
|
|
321
|
+
to *discover*. Over 55 API calls that produced 5 findings, and the questions
|
|
322
|
+
could not have done better, because two of the four were constants:
|
|
323
|
+
|
|
324
|
+
* `same_behavior` was asked about pairs selected *because their text differs*.
|
|
325
|
+
Of 49 real judgements, 40 came back below 0.25. The answer was decided by the
|
|
326
|
+
candidate filter before the question was asked.
|
|
327
|
+
* `merge_changes_behavior` -- "would replacing one with the other change
|
|
328
|
+
behaviour for some caller?" -- was made a hard veto at 0.4. Across 49
|
|
329
|
+
judgements it never once fell below 0.4 (mean 0.64). Asked without any call
|
|
330
|
+
sites in the state, the model always answers "probably yes". A question whose
|
|
331
|
+
answer is fixed in advance is not a judgement.
|
|
332
|
+
|
|
333
|
+
Both are gone. Behaviour is now asked about only where it is genuinely open,
|
|
334
|
+
and only in the band where the text actually differs.
|
|
335
|
+
|
|
336
|
+
The judge's job is intent, not behaviour. When two declarations are
|
|
337
|
+
syntactically identical, equivalence is already an AST fact; the only open
|
|
338
|
+
question is whether the repetition is redundancy or two things that merely look
|
|
339
|
+
alike. That is a perceptual question about names and purpose, which is what a
|
|
340
|
+
calibrated model is good at, and code decides what to do with the answer.
|
|
341
|
+
|
|
342
|
+
One policy rule has since been deleted for the same reason. It suppressed a
|
|
343
|
+
finding when the two declarations sat in different areas and neither was a
|
|
344
|
+
general-purpose utility. Over 500 verdicts that removed 284 candidates --
|
|
345
|
+
including a type named `GhostProjectSummary` declared once in the API and once
|
|
346
|
+
in the UI, where the model itself answered `sameName 0.97` and "keep left". The
|
|
347
|
+
gate inverted a correct judgement. **Locality is not intent**, and
|
|
348
|
+
`docs/reference/names.md` is the argument: one concept should have one name and one
|
|
349
|
+
declaration. The model may now override the deterministic fact only when it
|
|
350
|
+
judges the two names to mean different things, or says confidently that both
|
|
351
|
+
should stay.
|
|
352
|
+
|
|
353
|
+
Below the judgement budget, candidates are still reported, marked unverified. A
|
|
354
|
+
budget must never silently delete a finding.
|
|
355
|
+
|
|
356
|
+
## Adding a rule
|
|
357
|
+
|
|
358
|
+
1. Add a `src/rules/<name>.ts` with the five moves above.
|
|
359
|
+
2. Register it in `src/rules/index.ts`.
|
|
360
|
+
3. Put every new threshold in `src/policy.ts`.
|
|
361
|
+
4. Test it twice: once with answers that should activate the policy, once with
|
|
362
|
+
answers that should not.
|
|
363
|
+
5. Run `pnpm joggle check src` -- joggle is expected to pass its own rules.
|
|
364
|
+
|
|
365
|
+
## Why the next layer is types, not more AST
|
|
366
|
+
|
|
367
|
+
joggle reasons about syntax today: it parses with oxc, normalises a
|
|
368
|
+
declaration's text, and hashes it. That catches copy-pasted code -- on a
|
|
369
|
+
1,870-file codebase it found a `delay` helper copied into six files and one
|
|
370
|
+
type declared in 48 -- but 79% of what it reported were *type* declarations,
|
|
371
|
+
judged by text similarity. That is the wrong instrument, and the material in
|
|
372
|
+
`docs/` says why:
|
|
373
|
+
|
|
374
|
+
* `docs/reference/names.md` -- types are the one contract the compiler enforces, and a
|
|
375
|
+
type name is a search term exactly like a function name. Every `any` leaves
|
|
376
|
+
the compiler with nothing to say and the agent with nothing to search.
|
|
377
|
+
* `docs/reference/parse.md` -- the entire argument is that the type system is where a
|
|
378
|
+
program makes illegal states unrepresentable, and that validation which
|
|
379
|
+
discards what it learned is the anti-pattern.
|
|
380
|
+
* `docs/reference/something.md` -- the goal is a compiler on top of the codebase that can
|
|
381
|
+
refuse a change because "this file is in the wrong relationship to that".
|
|
382
|
+
* `docs/new-passes.md` names four pillars: AST structure, dependency graph,
|
|
383
|
+
**type resolution data**, and text analysis. joggle has the first, part of
|
|
384
|
+
the second, and none of the third.
|
|
385
|
+
|
|
386
|
+
Types are also the only layer where the answer is *provable* rather than
|
|
387
|
+
similar. Measured on that same codebase with `tsgo --generateTrace` -- 11.5s
|
|
388
|
+
and 301MB of type facts for 1,870 files:
|
|
389
|
+
|
|
390
|
+
```
|
|
391
|
+
User: 46 resolved entries across 24 declaring files, 3 distinct resolved shapes
|
|
392
|
+
Object (x38), Any (x4), TypeParameter (x4)
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
One name, three meanings, four of them `any`. Text hashing cannot see that. The
|
|
396
|
+
checker knows it exactly.
|
|
397
|
+
|
|
398
|
+
So the direction is: keep the AST as the index and the source of spans, and
|
|
399
|
+
move the *judgements* to the type graph.
|
|
400
|
+
|
|
401
|
+
The first type-aware rule should be **`joggle/one-concept-one-type`**: the same
|
|
402
|
+
declared name resolving to different types in different places. Divergence is a
|
|
403
|
+
correctness problem; provable identity is a collapse candidate. Neither is
|
|
404
|
+
reachable from text.
|
|
405
|
+
|
|
406
|
+
Two implementation notes, because a trace is a hack that works and not the
|
|
407
|
+
final shape:
|
|
408
|
+
|
|
409
|
+
* `--generateTrace` needs no fork and is read-only, but the join from a source
|
|
410
|
+
declaration to its resolved type is not a plain name lookup -- aliases expand,
|
|
411
|
+
and one name can appear as several entries. `CompanyType` did not appear at
|
|
412
|
+
all under its own name. Solving that join is the first task.
|
|
413
|
+
* The eventual host is the checker itself, the way `Effect-TS/language-service`
|
|
414
|
+
and `effect-ts/tsgo` do it: run inside the compiler and ask
|
|
415
|
+
`getTypeAtLocation`. The trace route buys the same facts today without
|
|
416
|
+
maintaining a fork.
|
|
417
|
+
|
|
418
|
+
Deliberately *not* next: widening AST coverage to class and object-literal
|
|
419
|
+
methods. That is more surface on a weaker signal. It comes after types.
|
|
420
|
+
|
|
421
|
+
|
|
422
|
+
## Two committed artifacts, two machine artifacts
|
|
423
|
+
|
|
424
|
+
joggle's state is four files with four lifetimes. Two belong to the repository
|
|
425
|
+
and two belong to the machine.
|
|
426
|
+
|
|
427
|
+
| artifact | lives in | committed | what it is |
|
|
428
|
+
| --- | --- | --- | --- |
|
|
429
|
+
| `answers/` | `<root>/.joggle/` | **yes** | one shard per key prefix, one line per answer, keyed by the decision and the state it read. **This is the replay path**: CI replays it with no API key and no tokens. |
|
|
430
|
+
| `baseline/` | `<root>/.joggle/` | **yes** | the findings already accepted, one fragment per finding: the ratchet. |
|
|
431
|
+
| `wire.json` | `<machine cache>/joggle/<root>/` | no | every wire request's response, keyed by the request. A performance artifact, and far too large to commit: 11.5 megabytes on a 2,493-file repository. |
|
|
432
|
+
| `last-run.json` | `<machine cache>/joggle/<root>/` | no | a manifest and the last report. A performance artifact. |
|
|
433
|
+
|
|
434
|
+
The first two are decisions a person should be able to read in a diff, and they
|
|
435
|
+
are per-repository because the evidence is keyed on root-relative paths. The
|
|
436
|
+
others are the largest, churn on every edit, and are worthless to anyone else — so
|
|
437
|
+
it lives in `$XDG_CACHE_HOME` (or `~/Library/Caches` on macOS), keyed by the
|
|
438
|
+
analysed root. One machine cache serves every repository, and pointing joggle at
|
|
439
|
+
somebody else's checkout writes nothing into it.
|
|
440
|
+
|
|
441
|
+
### Running against a diff
|
|
442
|
+
|
|
443
|
+
```sh
|
|
444
|
+
joggle check --update-baseline .joggle/baseline # on main: record what is accepted
|
|
445
|
+
joggle check --baseline .joggle/baseline # on a branch: only what is new
|
|
446
|
+
joggle check # nothing changed: the previous run
|
|
447
|
+
joggle cache prune --max-age 90 # the cache is regenerable; keep it small
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
The baseline is by identity, not by line: rule, kept symbol and cluster
|
|
451
|
+
membership. Code moves constantly, and a baseline that reports every edit as a
|
|
452
|
+
new finding is a baseline nobody reads. A rename or a new member does change the
|
|
453
|
+
identity, which is correct — the finding is a different finding then.
|
|
454
|
+
|
|
455
|
+
It is stored as **fragments**: one file per accepted finding, named by the
|
|
456
|
+
identity's hash, under the target directory. A new acceptance is a new file, so
|
|
457
|
+
two branches that accept different findings cannot conflict. A target ending in
|
|
458
|
+
`.json` is still written as the old single file.
|
|
459
|
+
|
|
460
|
+
### Why an unchanged run costs nothing
|
|
461
|
+
|
|
462
|
+
`last-run.json` carries a manifest: a hash of `policy.analysisVersion`, the
|
|
463
|
+
question version, the model, the rule set, the analysed root, and the content of
|
|
464
|
+
every file. If it matches, the run ends after reading the files and before
|
|
465
|
+
parsing them. On a 1,870-file codebase:
|
|
466
|
+
|
|
467
|
+
```
|
|
468
|
+
cold, judgements cached 0 calls, 0 tokens, 3,718ms
|
|
469
|
+
again 0 calls, 0 tokens, 325ms (replayed)
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
The 325ms is file reads and hashing, which is what makes the check honest: it is
|
|
473
|
+
content, not mtimes. Add one file that duplicates nothing and the run is a full
|
|
474
|
+
analysis with zero calls; add the second copy and a new cluster exists, so
|
|
475
|
+
exactly that candidate is judged:
|
|
476
|
+
|
|
477
|
+
```
|
|
478
|
+
first copy of a new duplicate 0 calls, 0 tokens
|
|
479
|
+
second copy 1 call, 846 tokens
|
|
480
|
+
nothing changed 0 calls, 0 tokens, replayed
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
**`policy.analysisVersion` is the one thing that must be kept honest by hand.**
|
|
484
|
+
A candidate filter or a clustering rule can change every finding while leaving
|
|
485
|
+
every question byte-identical, and the manifest is what decides whether a stale
|
|
486
|
+
report gets replayed. Bump it when the rules move.
|
|
487
|
+
|
|
488
|
+
|
|
489
|
+
### Answering "what did this change introduce?"
|
|
490
|
+
|
|
491
|
+
Two scopes answer this. `--changed` compares against the last stored run, which
|
|
492
|
+
is machine-local and only exists once the repository has been analysed before:
|
|
493
|
+
|
|
494
|
+
```sh
|
|
495
|
+
joggle check --changed # scope candidate generation to the changed declarations
|
|
496
|
+
```
|
|
497
|
+
|
|
498
|
+
`--since` and `--pr` answer the same question from git, so a fresh clone is
|
|
499
|
+
scoped on its first run. They include the working tree and untracked files, which
|
|
500
|
+
is what makes a second pass see the edit made after the first:
|
|
501
|
+
|
|
502
|
+
```sh
|
|
503
|
+
joggle check --since origin/main # this branch against where it forked
|
|
504
|
+
joggle check --pr # the current branch's pull request
|
|
505
|
+
joggle check --pr-number 123 # a specific pull request
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
Both are sound because a change elsewhere cannot create a duplicate between two
|
|
509
|
+
declarations it did not touch: every *new* finding has at least one changed
|
|
510
|
+
member. Dependents come into scope only when a file's **exports** moved, because
|
|
511
|
+
only then can their type names resolve to something else — a refinement the
|
|
512
|
+
stored-run scope makes and the git scope does not.
|
|
513
|
+
|
|
514
|
+
A **pure rename** is not a content change, and a PR that moves files is mostly
|
|
515
|
+
renames. Both scopes split it out: `changed` is added, modified and
|
|
516
|
+
renamed-with-edits, and `moved` is a rename git is certain kept every byte
|
|
517
|
+
(`--changed` finds the same thing from matching content hashes). Content rules
|
|
518
|
+
read `changed` alone — a moved file's bytes are not new — while the graph rules
|
|
519
|
+
read the union, because the move itself can put a module on the wrong side of a
|
|
520
|
+
boundary it used to respect. The scope note says how many were set aside.
|
|
521
|
+
|
|
522
|
+
What neither does is report removals, and neither is a full report — so a scoped
|
|
523
|
+
run leaves the stored run alone rather than becoming the next comparison base,
|
|
524
|
+
and a git scope never replays the stored full report.
|
|
525
|
+
|
|
526
|
+
Measured on 1,870 files:
|
|
527
|
+
|
|
528
|
+
| | calls | tokens | wall clock |
|
|
529
|
+
| --- | --- | --- | --- |
|
|
530
|
+
| nothing changed | 0 | 0 | **488ms** (replayed) |
|
|
531
|
+
| body edit to a widely-imported file | 0 | 0 | 4,737ms |
|
|
532
|
+
| comment inside a duplicated declaration | 1 | 1,571 | 5,284ms |
|
|
533
|
+
| same edit seen again | 0 | 0 | ~0 (cached verdict) |
|
|
534
|
+
| a full analysis | 0 | 0 | ~15s |
|
|
535
|
+
|
|
536
|
+
The 4.7s is file reads and hashing — correctness needs content, not mtimes. What
|
|
537
|
+
remains after that is parsing every file; an incremental fact cache keyed on
|
|
538
|
+
content hash would remove it. The candidate generation is already scoped.
|
|
539
|
+
|
|
540
|
+
## One column, two nullabilities
|
|
541
|
+
|
|
542
|
+
A type whose nullability comes from outside TypeScript: the database.
|
|
543
|
+
|
|
544
|
+
`joggle/nullability-drift` reads a knex migration statically — `table.uuid('x')`
|
|
545
|
+
is nullable, `.notNullable()` is what makes it required — and compares it with a
|
|
546
|
+
schema field that names the column it reads:
|
|
547
|
+
|
|
548
|
+
```ts
|
|
549
|
+
export const PayrollCrewRowSchema = Schema.Struct({
|
|
550
|
+
personId: Schema.NullOr(uuid).annotate({ sourceColumn: 'payroll_crew.shakti_user_id' }),
|
|
551
|
+
})
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
The mapping is an **annotation**, because it is not derivable: the field is
|
|
555
|
+
`personId`, the column is `shakti_user_id`, and that alias exists only in the SQL.
|
|
556
|
+
A repository opts in by annotating; a field with no `sourceColumn` is not
|
|
557
|
+
compared at all, and the rule's note says so rather than printing a clean zero.
|
|
558
|
+
|
|
559
|
+
The comparison is deterministic: a migration is a specification and
|
|
560
|
+
`Schema.NullOr` is a fact. What is a judgement is what the disagreement *means* --
|
|
561
|
+
a schema that should agree with the migration, a null that is data the adapter
|
|
562
|
+
handles, or an annotation that reads the wrong column -- and that is the question.
|
|
563
|
+
What it catches is the decode that fails the first time real data arrives: the
|
|
564
|
+
column can hold a null and the schema refuses it.
|
|
565
|
+
|
|
566
|
+
## A 5xx for a row that is not there
|
|
567
|
+
|
|
568
|
+
The amplifier the nullability work cannot see: the data is legitimately absent,
|
|
569
|
+
and the endpoint reports a server outage for it. One bad request becomes an
|
|
570
|
+
outage signal — pagers, error budgets, retries against a request that will never
|
|
571
|
+
succeed.
|
|
572
|
+
|
|
573
|
+
`joggle/data-error-as-outage` is judged, and the split is deliberate:
|
|
574
|
+
|
|
575
|
+
- **Candidate (deterministic, high recall):** any branch that could be about a
|
|
576
|
+
row and answers a 5xx — a nullish guard (`!row`, `row == null`) with a 5xx
|
|
577
|
+
inside, or a `try` whose `catch` answers a 5xx (`reply.code(500)`,
|
|
578
|
+
`res.statusCode = 503`).
|
|
579
|
+
- **Questions (Jev), atomic, over the same handler source:**
|
|
580
|
+
- `about_a_row` (Noul) — does the branch concern a row from a data store?
|
|
581
|
+
- `verdict` (Choice) — is the absent row normal, or a broken invariant?
|
|
582
|
+
- **Composed in code:** report only when the branch IS about a row and the
|
|
583
|
+
absence is normal. `row_absence_is_normal` reports; `row_absence_is_an_error`
|
|
584
|
+
drops; a deliberate 5xx is pinned in `.joggle/answers/`.
|
|
585
|
+
|
|
586
|
+
A nullish guard with a 5xx inside is specific enough to report unverified when no
|
|
587
|
+
model is available; a bare `catch` is not — every ordinary error handler has one —
|
|
588
|
+
so a `catch` is reported only once the model confirms it is about a row.
|
|
589
|
+
|
|
590
|
+
The candidate does **not** decide what a row read is. An earlier version named it
|
|
591
|
+
in code (`find*`, `get*`, `load*`, `parse*`), and against a real repository it
|
|
592
|
+
missed a read inside a callback, a bare call, and a verb that was not on the list
|
|
593
|
+
— three false negatives no answer could recover, because a candidate that is
|
|
594
|
+
never sent cannot be judged. So the code finds the *shape* and asks the question
|
|
595
|
+
the names were guessing at: state, not a pre-decided classifier. Nothing here
|
|
596
|
+
knows Fastify, Express or a schema library.
|
|
597
|
+
|
|
598
|
+
## Not here yet
|
|
599
|
+
|
|
600
|
+
The honest list: **no type awareness** (see above -- it is the next layer), no
|
|
601
|
+
LSP, no `--fix`, no emission as an oxlint rule, and no
|
|
602
|
+
calibration harness. The calibration harness is the piece that has to exist
|
|
603
|
+
before any judged rule can move from `warn` to `error`.
|
|
604
|
+
|
|
605
|
+
## Layout
|
|
606
|
+
|
|
607
|
+
```
|
|
608
|
+
src/
|
|
609
|
+
policy.ts every threshold and question version, one file
|
|
610
|
+
schema.ts diagnostics, the System One contract, typed errors
|
|
611
|
+
workspace.ts oxc facts, normalisation, similarity
|
|
612
|
+
typetrace.ts tsgo --generateTrace reader, and the type facts cache
|
|
613
|
+
tsgo.ts tsgo subprocess adapter
|
|
614
|
+
decision.ts System One client, with the wire and answer caches
|
|
615
|
+
atoms.ts the run's shared facts, content-addressed
|
|
616
|
+
plans.ts the plan engine: one request per run, cache per question
|
|
617
|
+
rule.ts the Rule and PlannedRule vocabulary
|
|
618
|
+
rules/ one file per rule
|
|
619
|
+
check.ts the two-phase analysis pass
|
|
620
|
+
report.ts text / json / github
|
|
621
|
+
main.ts CLI
|
|
622
|
+
extensions/
|
|
623
|
+
joggle.ts the pi extension: tools and a command over the CLI
|
|
624
|
+
scripts/
|
|
625
|
+
joggle.sh the global command: resolves the checkout, supplies the key
|
|
626
|
+
joggle-elsewhere.sh check another checkout without writing into it
|
|
627
|
+
tests/
|
|
628
|
+
fixtures/corpus/ intentionally duplicated fixture project
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
`docs/` is the thinking that motivated this layer -- how agents navigate code,
|
|
632
|
+
why names are addresses, parse don't validate, incremental computation, the
|
|
633
|
+
deterministic harness. The previous implementation -- a Rust CLI of hand-written
|
|
634
|
+
heuristics -- has been removed; the ideas worth keeping are the rules here.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 joggle contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|