@autotelic/joggle 0.4.4 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/JOGGLE.md +39 -14
- package/dist/answer-store.js +24 -0
- package/dist/check.js +37 -5
- package/dist/main.js +2 -0
- package/package.json +1 -1
package/JOGGLE.md
CHANGED
|
@@ -421,6 +421,22 @@ Deliberately *not* next: widening AST coverage to class and object-literal
|
|
|
421
421
|
methods. That is more surface on a weaker signal. It comes after types.
|
|
422
422
|
|
|
423
423
|
|
|
424
|
+
### The committed cache has a size, and it is not free
|
|
425
|
+
|
|
426
|
+
Committing the answer cache buys keyless replay, and it costs bytes in every clone
|
|
427
|
+
and every CI checkout. GitHub warns above **50 MB per file** and refuses above
|
|
428
|
+
100 MB; a cache is worth nowhere near that.
|
|
429
|
+
|
|
430
|
+
- `joggle cache prune --max-age 90` drops answers nobody has used in a season;
|
|
431
|
+
`joggle cache prune --max-bytes 4000000` caps the file, oldest answers first. Both
|
|
432
|
+
are safe: an answer is regenerable by judging again.
|
|
433
|
+
- A run says so when the store passes 8 MB, because a cache nobody measured is how
|
|
434
|
+
a 0-byte ledger on the base branch becomes 55 MB at the top of a stack.
|
|
435
|
+
- In a stack the growth is per branch: each scoped accept adds this branch's answers
|
|
436
|
+
and the same growing file rides in every commit above it. Cap it before the last
|
|
437
|
+
push, or stop committing it and let a keyed CI judge instead — but only with the
|
|
438
|
+
key, because without one the identities change (see the stack section above).
|
|
439
|
+
|
|
424
440
|
## Two committed artifacts, two machine artifacts
|
|
425
441
|
|
|
426
442
|
joggle's state is four files with four lifetimes. Two belong to the repository
|
|
@@ -472,23 +488,32 @@ full-repository budget: at ~2.5k candidates a keyed run, a repository with 12k
|
|
|
472
488
|
unjudged candidates is a 25-minute run that still refuses to write.
|
|
473
489
|
|
|
474
490
|
```sh
|
|
475
|
-
|
|
491
|
+
# the gate: what this branch adds, judged
|
|
492
|
+
joggle check --types --no-replay --since origin/<parent> --baseline .joggle/baseline
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
**A scoped run is for the gate. Acceptance is a full-run act**, and the command
|
|
496
|
+
enforces it: `--update-baseline` refuses a scoped run unless
|
|
497
|
+
`--allow-scoped-baseline` says otherwise.
|
|
498
|
+
|
|
499
|
+
The reason is the fold. Two findings at the same site with the same move are folded
|
|
500
|
+
into the richer one, and which survivor wins depends on what ran: a scoped run sees
|
|
501
|
+
fewer candidates at that site than a full run does, so it can name the problem with
|
|
502
|
+
one rule where a full run names it with another. Identities are therefore not
|
|
503
|
+
scope-invariant, a scoped accept can miss exactly the identity CI fails on, and no
|
|
504
|
+
amount of `--merge-baseline` repairs a ledger that never saw it. (The fix is aliases
|
|
505
|
+
on a folded survivor, so the ledger can match any of them; until that lands, the
|
|
506
|
+
refusal is the honest state.) Accept with one full run at the head of a stack:
|
|
507
|
+
|
|
508
|
+
```sh
|
|
509
|
+
joggle check --types --no-replay \
|
|
476
510
|
--baseline .joggle/baseline --update-baseline .joggle/baseline --merge-baseline
|
|
477
511
|
```
|
|
478
512
|
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
ledger. `--allow-unjudged-baseline` is not the shortcut here: it ratifies the
|
|
484
|
-
thousands nobody looked at, which is a commit nobody can defend.
|
|
485
|
-
|
|
486
|
-
A scoped accept is necessary, not sufficient. Cross-file rules only fire over the
|
|
487
|
-
whole repository, and a scoped run folds some findings into another rule at the same
|
|
488
|
-
site (the note counts them), so an identity CI fails on can be one this run never had.
|
|
489
|
-
Use the scoped accept to keep a stack moving and treat the full run as the authority:
|
|
490
|
-
when CI names an identity the scoped ledger lacks, accept that one and move on. Budget
|
|
491
|
-
one full judged run for the stack's head, not one per branch.
|
|
513
|
+
`--merge-baseline` keeps the identities already accepted and adds this run's, so
|
|
514
|
+
accepting at the head does not discard the stack's accepted set. `--allow-unjudged-baseline`
|
|
515
|
+
is not the shortcut here: it ratifies thousands nobody looked at, which is a commit
|
|
516
|
+
nobody can defend.
|
|
492
517
|
|
|
493
518
|
### The cascade, not the conflicts
|
|
494
519
|
|
package/dist/answer-store.js
CHANGED
|
@@ -343,6 +343,30 @@ export const make = (fs, path, dir, options = {}) => Effect.gen(function* () {
|
|
|
343
343
|
};
|
|
344
344
|
});
|
|
345
345
|
const sizeOf = (entry) => renderEntry(entry).length + 1;
|
|
346
|
+
/**
|
|
347
|
+
* How many bytes the ledger occupies on disk, without parsing it.
|
|
348
|
+
*
|
|
349
|
+
* A committed answer cache is a cache, and a cache that only grows is a repository-health
|
|
350
|
+
* problem: GitHub warns above 50 MB per file and refuses above 100 MB, and every clone pays
|
|
351
|
+
* for it. Reading the bytes (rather than the census) keeps the check cheap enough to run on
|
|
352
|
+
* every run, including on a ledger measured in tens of megabytes.
|
|
353
|
+
*/
|
|
354
|
+
export const answerStoreBytes = (fs, path, dir) => Effect.gen(function* () {
|
|
355
|
+
const sizeOfFile = (file) => fs.stat(file).pipe(Effect.map((info) => Number(info.size)), Effect.orElseSucceed(() => 0));
|
|
356
|
+
const single = answersFile(path, dir);
|
|
357
|
+
let total = yield* sizeOfFile(single);
|
|
358
|
+
total += yield* sizeOfFile(answersObject(path, dir));
|
|
359
|
+
const shards = answersDir(path, dir);
|
|
360
|
+
const names = yield* fs
|
|
361
|
+
.readDirectory(shards)
|
|
362
|
+
.pipe(Effect.orElseSucceed(() => []));
|
|
363
|
+
for (const name of names) {
|
|
364
|
+
if (!name.endsWith(".jsonl"))
|
|
365
|
+
continue;
|
|
366
|
+
total += yield* sizeOfFile(path.join(shards, name));
|
|
367
|
+
}
|
|
368
|
+
return total;
|
|
369
|
+
});
|
|
346
370
|
/**
|
|
347
371
|
* Read the ledger and count what is in it.
|
|
348
372
|
*
|
package/dist/check.js
CHANGED
|
@@ -9,6 +9,7 @@ import { shortHash } from "./state.js";
|
|
|
9
9
|
import { canonical } from "./canonical.js";
|
|
10
10
|
import { sourceFingerprint } from "./fingerprint.js";
|
|
11
11
|
import { loadParses } from "./parsecache.js";
|
|
12
|
+
import { answerStoreBytes } from "./answer-store.js";
|
|
12
13
|
import { baselinePath, manifestOf, readBaseline, readStored, writeBaseline, writeStored, } from "./run-cache.js";
|
|
13
14
|
import { policy } from "./policy.js";
|
|
14
15
|
import { Rules } from "./rules/index.js";
|
|
@@ -28,6 +29,14 @@ import { looksLikeSource } from "./source.js";
|
|
|
28
29
|
* consequences is a record of something that never happened.
|
|
29
30
|
*/
|
|
30
31
|
const UNJUDGED_STAGES = new Set(["unreadable", "no_evidence", "budget"]);
|
|
32
|
+
/**
|
|
33
|
+
* The size at which a growing ledger stops being a cache and starts being repository debt.
|
|
34
|
+
*
|
|
35
|
+
* GitHub warns above 50 MB per file and refuses above 100 MB, and every clone and CI
|
|
36
|
+
* checkout pays. Well below that is already larger than a diff anyone reviews, so the note
|
|
37
|
+
* fires early — at 8 MB — and names the cap rather than waiting for the warning.
|
|
38
|
+
*/
|
|
39
|
+
const CACHE_WARN_BYTES = 8 * 1024 * 1024;
|
|
31
40
|
const reasonOf = (error) => error.message;
|
|
32
41
|
const typecheckFindings = (output) => output.map((diagnostic) => finding({
|
|
33
42
|
ruleId: "joggle/typecheck",
|
|
@@ -218,6 +227,18 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
|
|
|
218
227
|
});
|
|
219
228
|
}
|
|
220
229
|
if (options.updateBaselinePath !== undefined) {
|
|
230
|
+
const target = baselinePath(path, options.updateBaselinePath);
|
|
231
|
+
// A scoped run is for the gate. Acceptance is a full-run act until folded findings
|
|
232
|
+
// carry the identities they absorbed, because which survivor wins a fold depends on
|
|
233
|
+
// what ran -- so a scoped accept can miss the identity a full run reports, silently.
|
|
234
|
+
const scoped = options.changed === true || options.changedPaths !== undefined;
|
|
235
|
+
if (scoped && options.allowScopedBaseline !== true) {
|
|
236
|
+
return yield* WorkspaceError.make({
|
|
237
|
+
path: options.updateBaselinePath,
|
|
238
|
+
operation: "baseline",
|
|
239
|
+
cause: new Error(`refusing to write ${target} for a scoped run: a scoped run's identities are not a subset of a full run's — the same site can fold to a different survivor when candidates outside the scope are missing — so this ledger could be missing the identity a full run reports. Accept with one full run (at the head of a stack), or pass --allow-scoped-baseline to accept the risk. Unscoped runs are unaffected.`),
|
|
240
|
+
});
|
|
241
|
+
}
|
|
221
242
|
if (unjudged > 0 && options.allowUnjudgedBaseline !== true) {
|
|
222
243
|
return yield* WorkspaceError.make({
|
|
223
244
|
path: options.updateBaselinePath,
|
|
@@ -226,13 +247,11 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
|
|
|
226
247
|
"Judging needs a model key in this run's environment (TYPESAFE_API_KEY); without one, " +
|
|
227
248
|
"the same findings are named from what the rule could see alone, so a ledger written " +
|
|
228
249
|
"now would not match a judged run. Judge first (raise --max-tokens if the budget was " +
|
|
229
|
-
"the limit), or
|
|
230
|
-
"
|
|
231
|
-
"
|
|
232
|
-
"Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
|
|
250
|
+
"the limit), or run scoped (--since <base>) to see the change judged with a pull " +
|
|
251
|
+
"request's budget — a scoped run reports the gate's answer but cannot write the " +
|
|
252
|
+
"ledger. Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
|
|
233
253
|
});
|
|
234
254
|
}
|
|
235
|
-
const target = baselinePath(path, options.updateBaselinePath);
|
|
236
255
|
const fresh = report.diagnostics
|
|
237
256
|
.map((entry) => entry.identity)
|
|
238
257
|
.filter((identity) => identity !== undefined);
|
|
@@ -847,6 +866,19 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
|
|
|
847
866
|
: "; a folded finding's identity is not in this run, so an accept taken from a scoped run can miss one a full run reports"),
|
|
848
867
|
});
|
|
849
868
|
}
|
|
869
|
+
// A committed answer cache is a cache, and a cache that only grows is a repository-health
|
|
870
|
+
// problem the run can see: GitHub warns above 50 MB per file and refuses above 100 MB, and
|
|
871
|
+
// every clone and CI checkout pays for the bytes. Scoped accepts on a deep stack grow it
|
|
872
|
+
// fastest, because each branch judges its own delta and keeps every answer.
|
|
873
|
+
const cacheBytes = yield* answerStoreBytes(fs, path, options.cacheDir);
|
|
874
|
+
if (cacheBytes > CACHE_WARN_BYTES) {
|
|
875
|
+
const megabytes = Math.round((cacheBytes / (1024 * 1024)) * 10) / 10;
|
|
876
|
+
notes.push({
|
|
877
|
+
ruleId: "joggle",
|
|
878
|
+
reason: `the judgement cache at ${options.cacheDir} is ${megabytes} MB; keep it machine-local or cap what is committed (` +
|
|
879
|
+
"joggle cache prune --max-bytes 4000000) — GitHub warns above 50 MB per file and every clone pays for it",
|
|
880
|
+
});
|
|
881
|
+
}
|
|
850
882
|
// The workspace's own phases, alongside the per-rule timings. A run that says
|
|
851
883
|
// "workspace 4.4s" and nothing else is a run nobody can make faster.
|
|
852
884
|
timings.push(...workspace.phases);
|
package/dist/main.js
CHANGED
|
@@ -88,6 +88,7 @@ const checkCommand = Command.make("check", {
|
|
|
88
88
|
baseline: Flag.String("baseline").pipe(Flag.withDescription("Report only findings not already accepted in this baseline file."), Flag.optional),
|
|
89
89
|
updateBaseline: Flag.String("update-baseline").pipe(Flag.withDescription("Write the current findings to this file as the accepted baseline."), Flag.optional),
|
|
90
90
|
mergeBaseline: Flag.Boolean("merge-baseline").pipe(Flag.withDescription("With --update-baseline, keep the identities already in that file and add this run's. A scoped run (--since) judges all of its own files, so this grows a ledger by one change instead of replacing it."), Flag.withDefault(false)),
|
|
91
|
+
allowScopedBaseline: Flag.Boolean("allow-scoped-baseline").pipe(Flag.withDescription("Write a ledger from a scoped run. Refused by default: a scoped run's identities are not a subset of a full run's, so the ledger can miss the identity CI reports. Accept from a full run instead."), Flag.withDefault(false)),
|
|
91
92
|
allowUnjudgedBaseline: Flag.Boolean("allow-unjudged-baseline").pipe(Flag.withDescription("Write the baseline even though some candidates were not judged. A baseline is what was reviewed; this is the explicit way to accept that some of it was not."), Flag.withDefault(false)),
|
|
92
93
|
cacheDir: Flag.String("cache-dir").pipe(Flag.withDescription("Where the answer cache lives: <cwd>/.joggle when it exists (an onboarded repository), otherwise the machine cache."), Flag.optional),
|
|
93
94
|
cwd: Flag.String("cwd").pipe(Flag.withDescription("Project root to analyse (default: the current directory)."), Flag.optional),
|
|
@@ -165,6 +166,7 @@ const checkCommand = Command.make("check", {
|
|
|
165
166
|
updateBaselinePath: Option.getOrUndefined(config.updateBaseline),
|
|
166
167
|
allowUnjudgedBaseline: config.allowUnjudgedBaseline,
|
|
167
168
|
mergeBaseline: config.mergeBaseline,
|
|
169
|
+
allowScopedBaseline: config.allowScopedBaseline,
|
|
168
170
|
}).pipe(Effect.provide(Layer.mergeAll(rulesLayer, decisionLayer({
|
|
169
171
|
cacheDir,
|
|
170
172
|
wireCacheDir: machineCache,
|