@autotelic/joggle 0.4.5 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/JOGGLE.md CHANGED
@@ -229,10 +229,11 @@ on every machine — which means **run from the repository root**. Evidence
229
229
  carries root-relative paths; running from a subdirectory changes the key and
230
230
  misses every cached verdict. Successful judgements are written to
231
231
  `.joggle/answers.jsonl`, one sorted, line-oriented answer per line; commit that
232
- file and CI replays them with `--offline` and **no API key at all**. A new
233
- answer is a new line, so two branches that judge different candidates merge with
234
- git's built-in `merge=union`, and `joggle cache prune` keeps the store from
235
- growing without a ceiling. A store large enough to make a whole-file rewrite or a
232
+ file and CI replays them with `--offline` and **no API key at all**. A line is the
233
+ key and the answer and nothing else, so two branches that judge different
234
+ candidates merge with git's built-in `merge=union` and the same answer merges to
235
+ one line rather than a copy; `joggle cache compact` and `cache prune --max-bytes`
236
+ keep the store from growing without a ceiling. A store large enough to make a whole-file rewrite or a
236
237
  merge expensive can be sharded instead (`cache.layout: "sharded"`, or
237
238
  `joggle cache migrate --layout sharded`); both layouts are read, and
238
239
  `cache migrate` moves between them. The older `answers.json` object is read and
@@ -427,15 +428,19 @@ Committing the answer cache buys keyless replay, and it costs bytes in every clo
427
428
  and every CI checkout. GitHub warns above **50 MB per file** and refuses above
428
429
  100 MB; a cache is worth nowhere near that.
429
430
 
430
- - `joggle cache prune --max-age 90` drops answers nobody has used in a season;
431
- `joggle cache prune --max-bytes 4000000` caps the file, oldest answers first. Both
432
- are safe: an answer is regenerable by judging again.
431
+ - `joggle cache prune --max-bytes 4000000` caps the file, oldest first, on any store.
432
+ `--max-age 90` needs ages, which only the machine cache records.
433
433
  - A run says so when the store passes 8 MB, because a cache nobody measured is how
434
434
  a 0-byte ledger on the base branch becomes 55 MB at the top of a stack.
435
- - In a stack the growth is per branch: each scoped accept adds this branch's answers
436
- and the same growing file rides in every commit above it. Cap it before the last
437
- push, or stop committing it and let a keyed CI judge instead — but only with the
438
- key, because without one the identities change (see the stack section above).
435
+ - **A scoped run does not write the committed ledger.** `--since`, `--pr` and
436
+ `--changed` answer into the machine cache unless `--cache-dir` names a store: a
437
+ branch's verdicts are a cache, and committing them is what put 131,440 lines in
438
+ every pull request of a stack instead of the 5,890 answers they held.
439
+ - **`joggle cache compact` repairs a duplicated ledger in place.** A line used to
440
+ carry `writtenAt`, so the same answer serialised differently on every branch and
441
+ `merge=union` kept both sides on every rebase. The line is now the key and the
442
+ answer only, so merges collapse to one line; for a history that already grew,
443
+ compact reads each key once and rewrites the set. No re-judging, no key needed.
439
444
 
440
445
  ## Two committed artifacts, two machine artifacts
441
446
 
@@ -488,23 +493,32 @@ full-repository budget: at ~2.5k candidates a keyed run, a repository with 12k
488
493
  unjudged candidates is a 25-minute run that still refuses to write.
489
494
 
490
495
  ```sh
491
- joggle check --types --no-replay --since origin/<parent> \
496
+ # the gate: what this branch adds, judged
497
+ joggle check --types --no-replay --since origin/<parent> --baseline .joggle/baseline
498
+ ```
499
+
500
+ **A scoped run is for the gate. Acceptance is a full-run act**, and the command
501
+ enforces it: `--update-baseline` refuses a scoped run unless
502
+ `--allow-scoped-baseline` says otherwise.
503
+
504
+ The reason is the fold. Two findings at the same site with the same move are folded
505
+ into the richer one, and which survivor wins depends on what ran: a scoped run sees
506
+ fewer candidates at that site than a full run does, so it can name the problem with
507
+ one rule where a full run names it with another. Identities are therefore not
508
+ scope-invariant, a scoped accept can miss exactly the identity CI fails on, and no
509
+ amount of `--merge-baseline` repairs a ledger that never saw it. (The fix is aliases
510
+ on a folded survivor, so the ledger can match any of them; until that lands, the
511
+ refusal is the honest state.) Accept with one full run at the head of a stack:
512
+
513
+ ```sh
514
+ joggle check --types --no-replay \
492
515
  --baseline .joggle/baseline --update-baseline .joggle/baseline --merge-baseline
493
516
  ```
494
517
 
495
- A scoped run judges every candidate in the files it read, so its identities are
496
- a pull request's worth and not the repository's. `--merge-baseline` keeps the
497
- identities already accepted and adds those; without it, `--update-baseline`
498
- writes the file as this run alone, which for a scoped run would truncate the
499
- ledger. `--allow-unjudged-baseline` is not the shortcut here: it ratifies the
500
- thousands nobody looked at, which is a commit nobody can defend.
501
-
502
- A scoped accept is necessary, not sufficient. Cross-file rules only fire over the
503
- whole repository, and a scoped run folds some findings into another rule at the same
504
- site (the note counts them), so an identity CI fails on can be one this run never had.
505
- Use the scoped accept to keep a stack moving and treat the full run as the authority:
506
- when CI names an identity the scoped ledger lacks, accept that one and move on. Budget
507
- one full judged run for the stack's head, not one per branch.
518
+ `--merge-baseline` keeps the identities already accepted and adds this run's, so
519
+ accepting at the head does not discard the stack's accepted set. `--allow-unjudged-baseline`
520
+ is not the shortcut here: it ratifies thousands nobody looked at, which is a commit
521
+ nobody can defend.
508
522
 
509
523
  ### The cascade, not the conflicts
510
524
 
@@ -4,9 +4,17 @@ import { StoredAnswer } from "./plans.js";
4
4
  /**
5
5
  * The answer cache: one logical ledger, two on-disk layouts.
6
6
  *
7
- * An entry is a line of `<key>\t<unix-seconds>\t<json>`, sorted by key. The
8
- * key (`answerKeyFor`) already content-addresses the decision, the atoms it
9
- * read, the model and the decision version, so the line is the identity.
7
+ * An entry is a line of `<key>\t<json>`, sorted by key. The key
8
+ * (`answerKeyFor`) already content-addresses the decision, the atoms it read,
9
+ * the model and the decision version, so the line IS the identity — and a line
10
+ * that is a function of the key is a line two branches can merge.
11
+ *
12
+ * The timestamp is deliberately NOT in that line. A committed ledger is merged
13
+ * with git's built-in `merge=union`, which keeps both sides of a differing line;
14
+ * stamping every copy with the wall clock made the same answer a different line in
15
+ * every clone, so each rebase of a 21-branch stack duplicated it again — 5,890
16
+ * answers became 131,440 lines, 20.5 MB for 0.9 MB of content. Age is a machine
17
+ * fact: the machine cache records it, the committed ledger does not.
10
18
  *
11
19
  * **Single** (the default) is one file, `answers.jsonl`. It is the shape a
12
20
  * reviewer can read: a verdict change is one changed line in one diff. Sharding
@@ -69,10 +77,14 @@ const AnswersObject = Schema.Struct({
69
77
  });
70
78
  const decodeAnswer = Schema.fromJsonString(StoredAnswer);
71
79
  /**
72
- * One line: `<key>\t<unix-seconds>\t<json>`. A tab cannot appear unescaped in
73
- * any of the three fields, so the split is unambiguous.
80
+ * One line: `<key>\t<json>`, or `<key>\t<unix-seconds>\t<json>` when the store
81
+ * records ages. A tab cannot appear unescaped in any field, and a payload always
82
+ * starts with `{`, so the split is unambiguous.
74
83
  */
75
- export const renderEntry = (entry) => entry.key + "\t" + entry.writtenAt + "\t" + canonical(entry.answer);
84
+ export const renderEntry = (entry) => entry.key +
85
+ "\t" +
86
+ (entry.writtenAt === undefined ? "" : entry.writtenAt + "\t") +
87
+ canonical(entry.answer);
76
88
  /** Parse a ledger file. A line that does not decode is skipped and COUNTED, never fatal. */
77
89
  export const parseLines = (text) => {
78
90
  const entries = [];
@@ -81,18 +93,24 @@ export const parseLines = (text) => {
81
93
  if (line === "")
82
94
  continue;
83
95
  const first = line.indexOf("\t");
84
- const second = line.indexOf("\t", first + 1);
85
- if (first < 0 || second < 0) {
96
+ if (first < 0) {
86
97
  skippedLines += 1;
87
98
  continue;
88
99
  }
89
- const writtenAt = Number(line.slice(first + 1, second));
90
- const answer = Result.getOrUndefined(SchemaParser.decodeResult(decodeAnswer)(line.slice(second + 1)));
91
- if (answer === undefined || !Number.isFinite(writtenAt)) {
100
+ const rest = line.slice(first + 1);
101
+ // A payload is JSON and starts with `{`; anything else is a legacy age field.
102
+ const stamped = /^(\d+)\t/.exec(rest);
103
+ const writtenAt = stamped === null ? undefined : Number(stamped[1]);
104
+ const payload = stamped === null ? rest : rest.slice(stamped[0].length);
105
+ const answer = Result.getOrUndefined(SchemaParser.decodeResult(decodeAnswer)(payload));
106
+ if (answer === undefined || (writtenAt !== undefined && !Number.isFinite(writtenAt))) {
92
107
  skippedLines += 1;
93
108
  continue;
94
109
  }
95
- entries.push({ key: line.slice(0, first), writtenAt, answer });
110
+ // Two shapes rather than a spread or a widened binding: an entry either has an age
111
+ // (machine store) or it does not (committed ledger).
112
+ const key = line.slice(0, first);
113
+ entries.push(writtenAt === undefined ? { key, answer } : { key, writtenAt, answer });
96
114
  }
97
115
  return { entries, skippedLines };
98
116
  };
@@ -103,12 +121,32 @@ export const renderShard = (entries) => {
103
121
  return sorted.length === 0 ? "" : sorted.map(renderEntry).join("\n") + "\n";
104
122
  };
105
123
  /**
106
- * Union two sides of one file by key.
124
+ * Which of two entries for one key survives.
125
+ *
126
+ * An age is evidence of what the model produced last, so a stamped entry wins over an
127
+ * unstamped one. Two unstamped entries with one key are the same answer by construction.
128
+ *
129
+ * @param pair.left - The existing entry.
130
+ * @param pair.right - The challenger.
131
+ * @returns The entry that stands.
132
+ */
133
+ const newer = ({ left, right }) => {
134
+ if (left.writtenAt === undefined)
135
+ return right;
136
+ if (right.writtenAt === undefined)
137
+ return left;
138
+ return right.writtenAt > left.writtenAt ? right : left;
139
+ };
140
+ /**
141
+ * Union two sides of one ledger by key.
107
142
  *
108
- * A key that only one side carries is kept. A key both sides carry with
109
- * different content keeps the newer timestamp: for a content-addressed key the
110
- * two answers should be identical, and the timestamp is the only signal that
111
- * says which the model produced last. Ties keep ours.
143
+ * A key only one side carries is kept. A key both sides carry keeps the newer answer when
144
+ * ages exist, and otherwise ours — for a content-addressed key the payloads are identical,
145
+ * which is what makes a merge collapse to one line instead of copying it.
146
+ *
147
+ * @param input.ours - Our side.
148
+ * @param input.theirs - Their side.
149
+ * @returns One entry per key.
112
150
  */
113
151
  export const mergeEntries = ({ ours, theirs, }) => {
114
152
  const merged = new Map();
@@ -116,8 +154,10 @@ export const mergeEntries = ({ ours, theirs, }) => {
116
154
  merged.set(entry.key, entry);
117
155
  for (const entry of theirs) {
118
156
  const existing = merged.get(entry.key);
119
- if (existing === undefined || entry.writtenAt > existing.writtenAt)
157
+ if (existing === undefined)
120
158
  merged.set(entry.key, entry);
159
+ else
160
+ merged.set(entry.key, newer({ left: existing, right: entry }));
121
161
  }
122
162
  return [...merged.values()];
123
163
  };
@@ -168,10 +208,15 @@ const loadAll = (fs, path, dir) => Effect.gen(function* () {
168
208
  const merged = new Map();
169
209
  const absorb = (entry) => {
170
210
  const existing = merged.get(entry.key);
171
- if (existing === undefined || entry.writtenAt > existing.writtenAt)
211
+ if (existing === undefined)
172
212
  merged.set(entry.key, entry);
213
+ else
214
+ merged.set(entry.key, newer({ left: existing, right: entry }));
173
215
  };
174
216
  let files = 0;
217
+ // Lines and skipped lines together are what the file held; the difference from
218
+ // `merged.size` is duplication a union merge left behind.
219
+ let lines = 0;
175
220
  let bytes = 0;
176
221
  let skippedLines = 0;
177
222
  const older = answersObject(path, dir);
@@ -193,6 +238,7 @@ const loadAll = (fs, path, dir) => Effect.gen(function* () {
193
238
  files += 1;
194
239
  bytes += bytesOf(text);
195
240
  skippedLines += parsed.skippedLines;
241
+ lines += parsed.entries.length + parsed.skippedLines;
196
242
  for (const entry of parsed.entries)
197
243
  absorb(entry);
198
244
  }
@@ -210,6 +256,7 @@ const loadAll = (fs, path, dir) => Effect.gen(function* () {
210
256
  files += 1;
211
257
  bytes += bytesOf(text);
212
258
  skippedLines += parsed.skippedLines;
259
+ lines += parsed.entries.length + parsed.skippedLines;
213
260
  for (const entry of parsed.entries)
214
261
  absorb(entry);
215
262
  }
@@ -219,6 +266,7 @@ const loadAll = (fs, path, dir) => Effect.gen(function* () {
219
266
  entries: merged,
220
267
  layout: shards > 0 ? "sharded" : "single",
221
268
  files,
269
+ lines,
222
270
  bytes,
223
271
  skippedLines,
224
272
  };
@@ -299,11 +347,18 @@ export const make = (fs, path, dir, options = {}) => Effect.gen(function* () {
299
347
  const entries = yield* Ref.make(loaded.entries);
300
348
  const dirty = yield* Ref.make(new Set());
301
349
  const layoutRef = yield* Ref.make(chosen);
302
- const fullWrite = yield* Ref.make(loaded.layout !== chosen);
350
+ const aged = [...loaded.entries.values()].some((entry) => entry.writtenAt !== undefined);
351
+ const fullWrite = yield* Ref.make(loaded.layout !== chosen || (options.timestamps !== true && aged));
303
352
  const writer = yield* Semaphore.make(1);
353
+ // A ledger written before the line was stable carries ages; a store that does not record
354
+ // ages rewrites them away on its first write, so a merge stops seeing two shapes of one key.
355
+ const strip = options.timestamps !== true;
356
+ const sanitise = (current) => strip
357
+ ? new Map([...current].map(([key, entry]) => [key, { key, answer: entry.answer }]))
358
+ : current;
304
359
  const flush = writer.withPermits(1)(Effect.gen(function* () {
305
360
  const changed = yield* Ref.getAndSet(dirty, new Set());
306
- const current = yield* Ref.get(entries);
361
+ const current = sanitise(yield* Ref.get(entries));
307
362
  const layout = yield* Ref.get(layoutRef);
308
363
  const rewrite = yield* Ref.getAndSet(fullWrite, false);
309
364
  if (rewrite || layout === "single") {
@@ -326,7 +381,9 @@ export const make = (fs, path, dir, options = {}) => Effect.gen(function* () {
326
381
  return {
327
382
  get: (key) => Effect.map(Ref.get(entries), (current) => Option.fromUndefinedOr(current.get(key)?.answer)),
328
383
  put: (key, answer) => Effect.gen(function* () {
329
- const entry = { key, writtenAt: seconds(yield* Clock.currentTimeMillis), answer };
384
+ const entry = options.timestamps === true
385
+ ? { key, writtenAt: seconds(yield* Clock.currentTimeMillis), answer }
386
+ : { key, answer };
330
387
  yield* Ref.update(entries, (current) => new Map(current).set(key, entry));
331
388
  yield* Ref.update(dirty, (current) => new Set(current).add(key));
332
389
  yield* flush;
@@ -384,8 +441,10 @@ export const census = (fs, path, dir) => Effect.gen(function* () {
384
441
  let oldest;
385
442
  let newest;
386
443
  for (const entry of loaded.entries.values()) {
387
- oldest = oldest === undefined ? entry.writtenAt : Math.min(oldest, entry.writtenAt);
388
- newest = newest === undefined ? entry.writtenAt : Math.max(newest, entry.writtenAt);
444
+ if (entry.writtenAt !== undefined) {
445
+ oldest = oldest === undefined ? entry.writtenAt : Math.min(oldest, entry.writtenAt);
446
+ newest = newest === undefined ? entry.writtenAt : Math.max(newest, entry.writtenAt);
447
+ }
389
448
  kinds.set(entry.answer.kind, (kinds.get(entry.answer.kind) ?? 0) + 1);
390
449
  if (entry.answer.kind === "Probability") {
391
450
  if (entry.answer.probability >= 0.5)
@@ -399,6 +458,7 @@ export const census = (fs, path, dir) => Effect.gen(function* () {
399
458
  }
400
459
  return {
401
460
  entries: loaded.entries.size,
461
+ lines: loaded.lines,
402
462
  files: loaded.files,
403
463
  layout: loaded.layout,
404
464
  bytes: loaded.bytes,
@@ -417,9 +477,11 @@ export const census = (fs, path, dir) => Effect.gen(function* () {
417
477
  /**
418
478
  * Drop cache entries and rewrite only the file or shards that changed.
419
479
  *
420
- * The age cut is per entry, from the timestamp each line carries. The size cut
421
- * is oldest-first across the whole ledger. An emptied file or shard is deleted
422
- * rather than written empty.
480
+ * The age cut is per entry, from the timestamp each line carries — a committed
481
+ * ledger carries none, so age pruning leaves it alone and only `--max-bytes`
482
+ * retires what it holds. The size cut is oldest-first across the whole ledger,
483
+ * with ageless entries last. An emptied file or shard is deleted rather than
484
+ * written empty.
423
485
  */
424
486
  export const prune = (fs, path, dir, options = {}) => Effect.gen(function* () {
425
487
  const now = options.now ?? seconds(yield* Clock.currentTimeMillis);
@@ -428,7 +490,8 @@ export const prune = (fs, path, dir, options = {}) => Effect.gen(function* () {
428
490
  const kept = new Map();
429
491
  let removed = 0;
430
492
  for (const entry of loaded.entries.values()) {
431
- if (cutoff !== undefined && entry.writtenAt < cutoff) {
493
+ // A store that does not record ages cannot be pruned by age: absent is not "old".
494
+ if (cutoff !== undefined && entry.writtenAt !== undefined && entry.writtenAt < cutoff) {
432
495
  removed += 1;
433
496
  continue;
434
497
  }
@@ -441,7 +504,8 @@ export const prune = (fs, path, dir, options = {}) => Effect.gen(function* () {
441
504
  if (total > options.maxBytes) {
442
505
  // A total order: oldest first, ties by key, so the result does not
443
506
  // depend on directory iteration.
444
- const ordered = [...kept.values()].sort((left, right) => left.writtenAt - right.writtenAt || byKey(left, right));
507
+ const age = (entry) => entry.writtenAt ?? Number.POSITIVE_INFINITY;
508
+ const ordered = [...kept.values()].sort((left, right) => age(left) - age(right) || byKey(left, right));
445
509
  for (const entry of ordered) {
446
510
  if (total <= options.maxBytes)
447
511
  break;
@@ -460,3 +524,26 @@ export const prune = (fs, path, dir, options = {}) => Effect.gen(function* () {
460
524
  bytes += sizeOf(entry);
461
525
  return { removed, kept: kept.size, bytes };
462
526
  });
527
+ /**
528
+ * Rewrite the ledger as the set it is: one sorted line per key.
529
+ *
530
+ * `merge=union` keeps both sides of a *differing* line, so before the line was a function of
531
+ * the key alone, every rebase of a stack copied the same answers again with a new stamp. The
532
+ * repair is not a prune (the copies are not stale) and not a migration (the layout is right):
533
+ * read every key once, newest answer winning, and write the file a fresh run would write.
534
+ */
535
+ export const compact = (fs, path, dir) => Effect.gen(function* () {
536
+ const bytesBefore = yield* answerStoreBytes(fs, path, dir);
537
+ const loaded = yield* loadAll(fs, path, dir);
538
+ // Ages are a machine fact: the repaired file is the set, in the one committed shape.
539
+ const set = new Map([...loaded.entries].map(([key, entry]) => [key, { key, answer: entry.answer }]));
540
+ yield* writeAll(fs, path, dir, loaded.layout, set);
541
+ const bytesAfter = yield* answerStoreBytes(fs, path, dir);
542
+ return {
543
+ lines: loaded.lines,
544
+ entries: loaded.entries.size,
545
+ duplicates: Math.max(0, loaded.lines - loaded.entries.size),
546
+ bytesBefore,
547
+ bytesAfter,
548
+ };
549
+ });
package/dist/check.js CHANGED
@@ -227,6 +227,18 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
227
227
  });
228
228
  }
229
229
  if (options.updateBaselinePath !== undefined) {
230
+ const target = baselinePath(path, options.updateBaselinePath);
231
+ // A scoped run is for the gate. Acceptance is a full-run act until folded findings
232
+ // carry the identities they absorbed, because which survivor wins a fold depends on
233
+ // what ran -- so a scoped accept can miss the identity a full run reports, silently.
234
+ const scoped = options.changed === true || options.changedPaths !== undefined;
235
+ if (scoped && options.allowScopedBaseline !== true) {
236
+ return yield* WorkspaceError.make({
237
+ path: options.updateBaselinePath,
238
+ operation: "baseline",
239
+ cause: new Error(`refusing to write ${target} for a scoped run: a scoped run's identities are not a subset of a full run's — the same site can fold to a different survivor when candidates outside the scope are missing — so this ledger could be missing the identity a full run reports. Accept with one full run (at the head of a stack), or pass --allow-scoped-baseline to accept the risk. Unscoped runs are unaffected.`),
240
+ });
241
+ }
230
242
  if (unjudged > 0 && options.allowUnjudgedBaseline !== true) {
231
243
  return yield* WorkspaceError.make({
232
244
  path: options.updateBaselinePath,
@@ -235,13 +247,11 @@ export const checkRepository = Effect.fn("joggle.check")(function* (options) {
235
247
  "Judging needs a model key in this run's environment (TYPESAFE_API_KEY); without one, " +
236
248
  "the same findings are named from what the rule could see alone, so a ledger written " +
237
249
  "now would not match a judged run. Judge first (raise --max-tokens if the budget was " +
238
- "the limit), or scope the run to the change: --since origin/<base> judges every " +
239
- "candidate in the files it read, so --update-baseline <file> --merge-baseline grows " +
240
- "the ledger by one pull request's identities instead of re-judging the repository. " +
241
- "Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
250
+ "the limit), or run scoped (--since <base>) to see the change judged with a pull " +
251
+ "request's budget — a scoped run reports the gate's answer but cannot write the " +
252
+ "ledger. Pass --allow-unjudged-baseline only to accept the unjudged debt itself."),
242
253
  });
243
254
  }
244
- const target = baselinePath(path, options.updateBaselinePath);
245
255
  const fresh = report.diagnostics
246
256
  .map((entry) => entry.identity)
247
257
  .filter((identity) => identity !== undefined);
package/dist/decision.js CHANGED
@@ -185,6 +185,7 @@ export const layer = (options) => Layer.effectContext(Effect.gen(function* () {
185
185
  // `docs/artifacts.md`.
186
186
  const answers = yield* makeAnswerStore(fs, path, options.cacheDir, {
187
187
  layout: options.layout,
188
+ timestamps: options.timestamps,
188
189
  });
189
190
  const planAnswers = {
190
191
  get: (key) => Effect.map(answers.get(key), (stored) => Option.map(stored, answerOf)),
package/dist/main.js CHANGED
@@ -5,9 +5,9 @@ import * as FetchHttpClient from "effect/http/FetchHttpClient";
5
5
  import { NodeRuntime, NodeServices } from "@effect/platform-node";
6
6
  import { checkRepository } from "./check.js";
7
7
  import { layer as decisionLayer } from "./decision.js";
8
- import { census as censusAnswers, make as makeAnswerStore, mergeShardText, prune as pruneAnswers, } from "./answer-store.js";
8
+ import { census as censusAnswers, compact as compactAnswers, make as makeAnswerStore, mergeShardText, prune as pruneAnswers, } from "./answer-store.js";
9
9
  import { ask } from "./ask.js";
10
- import { cacheDirFor, runCacheDirFor, shortHash } from "./state.js";
10
+ import { cacheDirFor, runCacheDirFor, scopedAnswerCacheDir, shortHash } from "./state.js";
11
11
  import { canonical } from "./canonical.js";
12
12
  import { loadConfig, isEnabled, severityFor } from "./config.js";
13
13
  import { layer as gitLayer, Service as Git } from "./git.js";
@@ -88,6 +88,7 @@ const checkCommand = Command.make("check", {
88
88
  baseline: Flag.String("baseline").pipe(Flag.withDescription("Report only findings not already accepted in this baseline file."), Flag.optional),
89
89
  updateBaseline: Flag.String("update-baseline").pipe(Flag.withDescription("Write the current findings to this file as the accepted baseline."), Flag.optional),
90
90
  mergeBaseline: Flag.Boolean("merge-baseline").pipe(Flag.withDescription("With --update-baseline, keep the identities already in that file and add this run's. A scoped run (--since) judges all of its own files, so this grows a ledger by one change instead of replacing it."), Flag.withDefault(false)),
91
+ allowScopedBaseline: Flag.Boolean("allow-scoped-baseline").pipe(Flag.withDescription("Write a ledger from a scoped run. Refused by default: a scoped run's identities are not a subset of a full run's, so the ledger can miss the identity CI reports. Accept from a full run instead."), Flag.withDefault(false)),
91
92
  allowUnjudgedBaseline: Flag.Boolean("allow-unjudged-baseline").pipe(Flag.withDescription("Write the baseline even though some candidates were not judged. A baseline is what was reviewed; this is the explicit way to accept that some of it was not."), Flag.withDefault(false)),
92
93
  cacheDir: Flag.String("cache-dir").pipe(Flag.withDescription("Where the answer cache lives: <cwd>/.joggle when it exists (an onboarded repository), otherwise the machine cache."), Flag.optional),
93
94
  cwd: Flag.String("cwd").pipe(Flag.withDescription("Project root to analyse (default: the current directory)."), Flag.optional),
@@ -98,7 +99,8 @@ const checkCommand = Command.make("check", {
98
99
  // file list against the root, git's changed files against the workspace --
99
100
  // and a relative root makes those comparisons fail quietly.
100
101
  const cwd = path.resolve(Option.getOrUndefined(config.cwd) ?? process.cwd());
101
- const cacheDir = yield* cacheDirFor(cwd, Option.getOrUndefined(config.cacheDir));
102
+ const declaredCacheDir = Option.getOrUndefined(config.cacheDir);
103
+ const repositoryCache = yield* cacheDirFor(cwd, declaredCacheDir);
102
104
  const apiKey = yield* Config.option(Config.String("TYPESAFE_API_KEY"));
103
105
  const ruleFlag = Option.getOrUndefined(config.rule);
104
106
  const rules = ruleFlag === undefined
@@ -137,6 +139,16 @@ const checkCommand = Command.make("check", {
137
139
  // The machine cache holds the run cache and the wire cache. The
138
140
  // per-decision answer cache in `cacheDir` is the one CI replays.
139
141
  const machineCache = yield* runCacheDirFor(cwd);
142
+ // A branch's answers are a cache, not a commit: a scoped run warms this machine and
143
+ // leaves the committed ledger alone. Otherwise every pull request in a stack carries
144
+ // thousands of lines of verdicts nobody can review (see docs/artifacts.md).
145
+ const scopedRun = config.changed || changedPaths !== undefined;
146
+ const cacheDir = scopedAnswerCacheDir({
147
+ declared: declaredCacheDir,
148
+ repository: repositoryCache,
149
+ machine: machineCache,
150
+ scoped: scopedRun,
151
+ });
140
152
  const report = yield* checkRepository({
141
153
  config: effective,
142
154
  plugins: loaded,
@@ -165,18 +177,35 @@ const checkCommand = Command.make("check", {
165
177
  updateBaselinePath: Option.getOrUndefined(config.updateBaseline),
166
178
  allowUnjudgedBaseline: config.allowUnjudgedBaseline,
167
179
  mergeBaseline: config.mergeBaseline,
180
+ allowScopedBaseline: config.allowScopedBaseline,
168
181
  }).pipe(Effect.provide(Layer.mergeAll(rulesLayer, decisionLayer({
169
182
  cacheDir,
170
183
  wireCacheDir: machineCache,
171
184
  offline: config.offline,
172
185
  apiKey,
173
186
  layout: effective.cache?.layout,
187
+ timestamps: cacheDir === machineCache,
174
188
  }), tsgoLayer(cwd, {
175
189
  runExternalCode: config.runExternalCode || effective.runExternalCode === true,
176
190
  }))));
177
- yield* write(render(report, config.format, { color: process.stdout.isTTY === true }));
191
+ // Say where the answers went when the run left the committed ledger alone: a reader
192
+ // who expected the cache to grow would otherwise think judging did nothing.
193
+ const answered = scopedRun && declaredCacheDir === undefined && repositoryCache !== machineCache;
194
+ const reported = answered
195
+ ? {
196
+ ...report,
197
+ notes: [
198
+ {
199
+ ruleId: "joggle",
200
+ reason: "answers for this scoped run went to the machine cache, not the committed ledger: a branch's verdicts are a cache, and committing them puts thousands of lines in every pull request",
201
+ },
202
+ ...report.notes,
203
+ ],
204
+ }
205
+ : report;
206
+ yield* write(render(reported, config.format, { color: process.stdout.isTTY === true }));
178
207
  yield* Effect.sync(() => {
179
- process.exitCode = exitCodeFor(report, config.maxWarnings);
208
+ process.exitCode = exitCodeFor(reported, config.maxWarnings);
180
209
  });
181
210
  }).pipe(Effect.provide(gitLayer))).pipe(Command.withDescription("Report cross-file duplication and naming drift. With no paths, analyse the project tsgo sees. --since, --pr and --pr-number answer what a change introduced; --changed does the same against the last stored run."), Command.withShortDescription("Report cross-file duplication and naming drift."), Command.withExamples([
182
211
  { command: "joggle check", description: "Analyse the project tsgo sees." },
@@ -590,11 +619,50 @@ const cacheReport = Command.make("report", {
590
619
  " verdicts Probability " + found.verdicts.yes + " yes / " + found.verdicts.no + " no",
591
620
  " labels " + (labels === "" ? "none" : labels),
592
621
  " written " + date(found.oldest) + " .. " + date(found.newest),
622
+ ...(found.lines > found.entries
623
+ ? [
624
+ " duplicates " +
625
+ (found.lines - found.entries) +
626
+ " line(s) over " +
627
+ found.entries +
628
+ " answer(s); `joggle cache compact` rewrites the set",
629
+ ]
630
+ : []),
593
631
  ...(found.skippedLines === 0 ? [] : [" skipped " + found.skippedLines + " unreadable line(s)"]),
594
632
  ].join("\n"));
595
633
  })).pipe(Command.withDescription("Render the committed answer ledger for a reader: entry and file counts, layout, verdict distribution, labels, and the date range. A summary cannot be replayed; this is what to read when reviewing the diff."), Command.withShortDescription("Report on the answer cache."), Command.withExamples([
596
634
  { command: "joggle cache report", description: "Count what the committed ledger holds." },
597
635
  ]));
636
+ const cacheCompact = Command.make("compact", {
637
+ cwd: Flag.String("cwd").pipe(Flag.withDescription("Project root whose cache to compact (default: the current directory)."), Flag.optional),
638
+ cacheDir: Flag.String("cache-dir").pipe(Flag.withDescription("Where the answer cache lives: <cwd>/.joggle when it exists, otherwise the machine cache."), Flag.optional),
639
+ }, (config) => Effect.gen(function* () {
640
+ const fs = yield* FileSystem.FileSystem;
641
+ const path = yield* Path.Path;
642
+ const cwd = path.resolve(Option.getOrUndefined(config.cwd) ?? process.cwd());
643
+ const dir = yield* cacheDirFor(cwd, Option.getOrUndefined(config.cacheDir));
644
+ const result = yield* compactAnswers(fs, path, dir);
645
+ if (result.duplicates === 0) {
646
+ yield* write("joggle: the ledger is already a set: " + result.entries + " answer(s), " + result.bytesAfter + " bytes");
647
+ return;
648
+ }
649
+ yield* write("joggle: " +
650
+ result.lines +
651
+ " line(s) held " +
652
+ result.entries +
653
+ " answer(s); compacted " +
654
+ result.duplicates +
655
+ " duplicate(s) away, " +
656
+ result.bytesBefore +
657
+ " -> " +
658
+ result.bytesAfter +
659
+ " bytes");
660
+ })).pipe(Command.withDescription("Rewrite the answer ledger as the set it is: one sorted line per key, newest answer winning. Repairs the duplication a union merge leaves behind (a differing line is kept from both sides), without re-judging anything."), Command.withShortDescription("Deduplicate the answer cache."), Command.withExamples([
661
+ {
662
+ command: "joggle cache compact",
663
+ description: "Collapse duplicate lines left by a stack's merges.",
664
+ },
665
+ ]));
598
666
  const cacheMigrate = Command.make("migrate", {
599
667
  cwd: Flag.String("cwd").pipe(Flag.withDescription("Project root whose cache to migrate (default: the current directory)."), Flag.optional),
600
668
  cacheDir: Flag.String("cache-dir").pipe(Flag.withDescription("Where the answer cache lives: <cwd>/.joggle when it exists, otherwise the machine cache."), Flag.optional),
@@ -609,7 +677,7 @@ const cacheMigrate = Command.make("migrate", {
609
677
  const target = config.layout === "single" ? path.join(dir, "answers.jsonl") : path.join(dir, "answers");
610
678
  yield* write("joggle: wrote " + written + " answer(s) to " + target);
611
679
  })).pipe(Command.withDescription("Rewrite the cache in one layout and remove the other: a single sorted answers.jsonl by default, or shards with --layout sharded. Idempotent."), Command.withShortDescription("Consolidate or shard the answer cache."));
612
- const cacheCommand = Command.make("cache", {}, () => write("joggle cache: run `joggle cache report`, `prune`, or `migrate --layout single|sharded`; docs/artifacts.md has the reasoning.")).pipe(Command.withDescription("Work on the answer cache itself: prune it, migrate it, and see how it is laid out."), Command.withShortDescription("Prune the answer cache."), Command.withSubcommands([cachePrune, cacheMigrate, cacheReport]));
680
+ const cacheCommand = Command.make("cache", {}, () => write("joggle cache: run `joggle cache compact`, `report`, `prune`, or `migrate --layout single|sharded`; docs/artifacts.md has the reasoning.")).pipe(Command.withDescription("Work on the answer cache itself: compact it, prune it, migrate it, and see how it is laid out."), Command.withShortDescription("Work on the answer cache."), Command.withSubcommands([cacheCompact, cachePrune, cacheMigrate, cacheReport]));
613
681
  const cli = Command.make("joggle").pipe(Command.withDescription("Cross-file patterns and idioms for TypeScript, enforced like a linter: deterministic where it can prove, System One where it has to judge."), Command.withExamples([
614
682
  {
615
683
  command: "joggle check --pr",
package/dist/state.js CHANGED
@@ -40,6 +40,29 @@ export const machineCacheRoot = Effect.gen(function* () {
40
40
  }).pipe(Effect.orElseSucceed(() => homedir()));
41
41
  /** A machine-local directory for one analysed root. */
42
42
  export const runCacheDirFor = (root) => Effect.map(machineCacheRoot, (base) => `${base}/joggle/${shortHash(root)}`);
43
+ /**
44
+ * Where a SCOPED run's answers go.
45
+ *
46
+ * A scoped run answers "what did this branch introduce", and a branch's answers are a cache,
47
+ * not a commit. Writing them into the repository's `.joggle/` puts thousands of lines into
48
+ * every pull request of a stack — unreviewable, and a blob in every clone and every CI
49
+ * checkout — which is what a 22-branch migration produced: a 55 MB file, 777 commits touching
50
+ * it, and a diff nobody could read.
51
+ *
52
+ * So a scoped run writes the machine cache unless the caller names a store with
53
+ * `--cache-dir`. The committed ledger stays the trunk's: one full run on the base branch
54
+ * decides what is accepted, and a branch's answers warm that machine and nobody else's diff.
55
+ *
56
+ * @param input.declared - `--cache-dir`, when the caller gave one. It always wins.
57
+ * @param input.repository - What {@link cacheDirFor} resolved: `.joggle/` or the machine.
58
+ * @param input.machine - The machine-local cache for this root.
59
+ * @param input.scoped - Whether the run was scoped to a change (`--since`, `--pr`, `--changed`).
60
+ */
61
+ export const scopedAnswerCacheDir = (input) => {
62
+ if (input.declared !== undefined)
63
+ return input.declared;
64
+ return input.scoped ? input.machine : input.repository;
65
+ };
43
66
  /**
44
67
  * Where a run's answer cache goes.
45
68
  *
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@autotelic/joggle",
3
- "version": "0.4.5",
3
+ "version": "0.6.0",
4
4
  "type": "module",
5
5
  "description": "Cross-file patterns and idioms for TypeScript — the layer above the linter and the typechecker.",
6
6
  "license": "MIT",