@autotelic/joggle 0.4.4 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/JOGGLE.md CHANGED
@@ -421,6 +421,22 @@ Deliberately *not* next: widening AST coverage to class and object-literal
421
421
  methods. That is more surface on a weaker signal. It comes after types.
422
422
 
423
423
 
424
+ ### The committed cache has a size, and it is not free
425
+
426
+ Committing the answer cache buys keyless replay, and it costs bytes in every clone
427
+ and every CI checkout. GitHub warns above **50 MB per file** and refuses above
428
+ 100 MB; a cache is worth nowhere near that.
429
+
430
+ - `joggle cache prune --max-age 90` drops answers nobody has used in a season;
431
+ `joggle cache prune --max-bytes 4000000` caps the file, oldest answers first. Both
432
+ are safe: an answer is regenerable by judging again.
433
+ - A run says so when the store passes 8 MB, because a cache nobody measured is how
434
+ a 0-byte ledger on the base branch becomes 55 MB at the top of a stack.
435
+ - In a stack the growth is per branch: each scoped accept adds this branch's answers
436
+ and the same growing file rides in every commit above it. Cap it before the last
437
+ push, or stop committing it and let a keyed CI judge instead — but only with the
438
+ key, because without one the identities change (see the stack section above).
439
+
424
440
  ## Two committed artifacts, two machine artifacts
425
441
 
426
442
  joggle's state is four files with four lifetimes. Two belong to the repository
@@ -472,23 +488,32 @@ full-repository budget: at ~2.5k candidates a keyed run, a repository with 12k
472
488
  unjudged candidates is a 25-minute run that still refuses to write.
473
489
 
474
490
  ```sh
475
- joggle check --types --no-replay --since origin/<parent> \
491
+ # the gate: what this branch adds, judged
492
+ joggle check --types --no-replay --since origin/<parent> --baseline .joggle/baseline
493
+ ```
494
+
495
+ **A scoped run is for the gate. Acceptance is a full-run act**, and the command
496
+ enforces it: `--update-baseline` refuses a scoped run unless
497
+ `--allow-scoped-baseline` says otherwise.
498
+
499
+ The reason is the fold. Two findings at the same site with the same move are folded
500
+ into the richer one, and which survivor wins depends on what ran: a scoped run sees
501
+ fewer candidates at that site than a full run does, so it can name the problem with
502
+ one rule where a full run names it with another. Identities are therefore not
503
+ scope-invariant, a scoped accept can miss exactly the identity CI fails on, and no
504
+ amount of `--merge-baseline` repairs a ledger that never saw it. (The fix is aliases
505
+ on a folded survivor, so the ledger can match any of them; until that lands, the
506
+ refusal is the honest state.) Accept with one full run at the head of a stack:
507
+
508
+ ```sh
509
+ joggle check --types --no-replay \
476
510
  --baseline .joggle/baseline --update-baseline .joggle/baseline --merge-baseline
477
511
  ```
478
512
 
479
- A scoped run judges every candidate in the files it read, so its identities are
480
- a pull request's worth and not the repository's. `--merge-baseline` keeps the
481
- identities already accepted and adds those; without it, `--update-baseline`
482
- writes the file as this run alone, which for a scoped run would truncate the
483
- ledger. `--allow-unjudged-baseline` is not the shortcut here: it ratifies the
484
- thousands nobody looked at, which is a commit nobody can defend.
485
-
486
- A scoped accept is necessary, not sufficient. Cross-file rules only fire over the
487
- whole repository, and a scoped run folds some findings into another rule at the same
488
- site (the note counts them), so an identity CI fails on can be one this run never had.
489
- Use the scoped accept to keep a stack moving and treat the full run as the authority:
490
- when CI names an identity the scoped ledger lacks, accept that one and move on. Budget
491
- one full judged run for the stack's head, not one per branch.
513
+ `--merge-baseline` keeps the identities already accepted and adds this run's, so
514
+ accepting at the head does not discard the stack's accepted set. `--allow-unjudged-baseline`
515
+ is not the shortcut here: it ratifies thousands nobody looked at, which is a commit
516
+ nobody can defend.
492
517
 
493
518
  ### The cascade, not the conflicts
494
519
 
@@ -343,6 +343,30 @@ export const make = (fs, path, dir, options = {}) => Effect.gen(function* () {
343
343
  };
344
344
  });
345
345
  const sizeOf = (entry) => renderEntry(entry).length + 1;
346
+ /**
347
+ * How many bytes the ledger occupies on disk, without parsing it.
348
+ *
349
+ * A committed answer cache is a cache, and a cache that only grows is a repository-health
350
+ * problem: GitHub warns above 50 MB per file and refuses above 100 MB, and every clone pays
351
+ * for it. Reading the bytes (rather than the census) keeps the check cheap enough to run on
352
+ * every run, including on a ledger measured in tens of megabytes.
353
+ */
354
+ export const answerStoreBytes = (fs, path, dir) => Effect.gen(function* () {
355
+ const sizeOfFile = (file) => fs.stat(file).pipe(Effect.map((info) => Number(info.size)), Effect.orElseSucceed(() => 0));
356
+ const single = answersFile(path, dir);
357
+ let total = yield* sizeOfFile(single);
358
+ total += yield* sizeOfFile(answersObject(path, dir));
359
+ const shards = answersDir(path, dir);
360
+ const names = yield* fs
361
+ .readDirectory(shards)
362
+ .pipe(Effect.orElseSucceed(() => []));
363
+ for (const name of names) {
364
+ if (!name.endsWith(".jsonl"))
365
+ continue;
366
+ total += yield* sizeOfFile(path.join(shards, name));
367
+ }
368
+ return total;
369
+ });
346
370
  /**
347
371
  * Read the ledger and count what is in it.
348
372
  *
package/dist/check.js CHANGED
@@ -9,6 +9,7 @@ import { shortHash } from "./state.js";
9
9
  import { canonical } from "./canonical.js";
10
10
  import { sourceFingerprint } from "./fingerprint.js";
11
11
  import { loadParses } from "./parsecache.js";
12
+ import { answerStoreBytes } from "./answer-store.js";
12
13
  import { baselinePath, manifestOf, readBaseline, readStored, writeBaseline, writeStored, } from "./run-cache.js";
13
14
  import { policy } from "./policy.js";
14
15
  import { Rules } from "./rules/index.js";
@@ -28,6 +29,14 @@ import { looksLikeSource } from "./source.js";
28
29
  * consequences is a record of something that never happened.
29
30
  */
30
31
  const UNJUDGED_STAGES = new Set(["unreadable", "no_evidence", "budget"]);
32
+ /**
33
+ * The size at which a growing ledger stops being a cache and starts being repository debt.
34
+ *
35
+ * GitHub warns above 50 MB per file and refuses above 100 MB, and every clone and CI
36
+ * checkout pays. Well below that is already larger than a diff anyone reviews, so the note
37
+ * fires early — at 8 MB — and names the cap rather than waiting for the warning.
38
+ */
39
+ const CACHE_WARN_BYTES = 8 * 1024 * 1024;
31
40
  const reasonOf = (error) => error.message;
32
41
  const typecheckFindings = (output) => output.map((diagnostic) => finding({
33
42
  ruleId: "joggle/typecheck",
@@ -218,6 +227,18 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
218
227
  });
219
228
  }
220
229
  if (options.updateBaselinePath !== undefined) {
230
+ const target = baselinePath(path, options.updateBaselinePath);
231
+ // A scoped run is for the gate. Acceptance is a full-run act until folded findings
232
+ // carry the identities they absorbed, because which survivor wins a fold depends on
233
+ // what ran -- so a scoped accept can miss the identity a full run reports, silently.
234
+ const scoped = options.changed === true || options.changedPaths !== undefined;
235
+ if (scoped && options.allowScopedBaseline !== true) {
236
+ return yield* WorkspaceError.make({
237
+ path: options.updateBaselinePath,
238
+ operation: "baseline",
239
+ cause: new Error(`refusing to write ${target} for a scoped run: a scoped run's identities are not a subset of a full run's — the same site can fold to a different survivor when candidates outside the scope are missing — so this ledger could be missing the identity a full run reports. Accept with one full run (at the head of a stack), or pass --allow-scoped-baseline to accept the risk. Unscoped runs are unaffected.`),
240
+ });
241
+ }
221
242
  if (unjudged > 0 && options.allowUnjudgedBaseline !== true) {
222
243
  return yield* WorkspaceError.make({
223
244
  path: options.updateBaselinePath,
@@ -226,13 +247,11 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
226
247
  "Judging needs a model key in this run's environment (TYPESAFE_API_KEY); without one, " +
227
248
  "the same findings are named from what the rule could see alone, so a ledger written " +
228
249
  "now would not match a judged run. Judge first (raise --max-tokens if the budget was " +
229
- "the limit), or scope the run to the change: --since origin/<base> judges every " +
230
- "candidate in the files it read, so --update-baseline <file> --merge-baseline grows " +
231
- "the ledger by one pull request's identities instead of re-judging the repository. " +
232
- "Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
250
+ "the limit), or run scoped (--since <base>) to see the change judged with a pull " +
251
+ "request's budget — a scoped run reports the gate's answer but cannot write the " +
252
+ "ledger. Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
233
253
  });
234
254
  }
235
- const target = baselinePath(path, options.updateBaselinePath);
236
255
  const fresh = report.diagnostics
237
256
  .map((entry) => entry.identity)
238
257
  .filter((identity) => identity !== undefined);
@@ -847,6 +866,19 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
847
866
  : "; a folded finding's identity is not in this run, so an accept taken from a scoped run can miss one a full run reports"),
848
867
  });
849
868
  }
869
+ // A committed answer cache is a cache, and a cache that only grows is a repository-health
870
+ // problem the run can see: GitHub warns above 50 MB per file and refuses above 100 MB, and
871
+ // every clone and CI checkout pays for the bytes. Scoped accepts on a deep stack grow it
872
+ // fastest, because each branch judges its own delta and keeps every answer.
873
+ const cacheBytes = yield* answerStoreBytes(fs, path, options.cacheDir);
874
+ if (cacheBytes > CACHE_WARN_BYTES) {
875
+ const megabytes = Math.round((cacheBytes / (1024 * 1024)) * 10) / 10;
876
+ notes.push({
877
+ ruleId: "joggle",
878
+ reason: `the judgement cache at ${options.cacheDir} is ${megabytes} MB; keep it machine-local or cap what is committed (` +
879
+ "joggle cache prune --max-bytes 4000000) — GitHub warns above 50 MB per file and every clone pays for it",
880
+ });
881
+ }
850
882
  // The workspace's own phases, alongside the per-rule timings. A run that says
851
883
  // "workspace 4.4s" and nothing else is a run nobody can make faster.
852
884
  timings.push(...workspace.phases);
package/dist/main.js CHANGED
@@ -88,6 +88,7 @@ const checkCommand = Command.make("check", {
88
88
  baseline: Flag.String("baseline").pipe(Flag.withDescription("Report only findings not already accepted in this baseline file."), Flag.optional),
89
89
  updateBaseline: Flag.String("update-baseline").pipe(Flag.withDescription("Write the current findings to this file as the accepted baseline."), Flag.optional),
90
90
  mergeBaseline: Flag.Boolean("merge-baseline").pipe(Flag.withDescription("With --update-baseline, keep the identities already in that file and add this run's. A scoped run (--since) judges all of its own files, so this grows a ledger by one change instead of replacing it."), Flag.withDefault(false)),
91
+ allowScopedBaseline: Flag.Boolean("allow-scoped-baseline").pipe(Flag.withDescription("Write a ledger from a scoped run. Refused by default: a scoped run's identities are not a subset of a full run's, so the ledger can miss the identity CI reports. Accept from a full run instead."), Flag.withDefault(false)),
91
92
  allowUnjudgedBaseline: Flag.Boolean("allow-unjudged-baseline").pipe(Flag.withDescription("Write the baseline even though some candidates were not judged. A baseline is what was reviewed; this is the explicit way to accept that some of it was not."), Flag.withDefault(false)),
92
93
  cacheDir: Flag.String("cache-dir").pipe(Flag.withDescription("Where the answer cache lives: <cwd>/.joggle when it exists (an onboarded repository), otherwise the machine cache."), Flag.optional),
93
94
  cwd: Flag.String("cwd").pipe(Flag.withDescription("Project root to analyse (default: the current directory)."), Flag.optional),
@@ -165,6 +166,7 @@ const checkCommand = Command.make("check", {
165
166
  updateBaselinePath: Option.getOrUndefined(config.updateBaseline),
166
167
  allowUnjudgedBaseline: config.allowUnjudgedBaseline,
167
168
  mergeBaseline: config.mergeBaseline,
169
+ allowScopedBaseline: config.allowScopedBaseline,
168
170
  }).pipe(Effect.provide(Layer.mergeAll(rulesLayer, decisionLayer({
169
171
  cacheDir,
170
172
  wireCacheDir: machineCache,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@autotelic/joggle",
3
- "version": "0.4.4",
3
+ "version": "0.5.0",
4
4
  "type": "module",
5
5
  "description": "Cross-file patterns and idioms for TypeScript — the layer above the linter and the typechecker.",
6
6
  "license": "MIT",