@biffo/cli 0.296.14 → 0.296.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,485 @@
1
+ """No module in this plugin's own package may re-derive an expression one of
2
+ its helpers already holds — for **every** such helper, enumerated from the
3
+ code itself rather than named here one at a time.
4
+
5
+ ## The class this closes (biffo-template#1587)
6
+
7
+ A shared helper is introduced to fix a defect, adopted at some call sites and
8
+ not others, and the helper's existence makes everyone believe it is fixed. The
9
+ unconverted sites keep the bug, behind a closed issue, looking done. Measured
10
+ twice in `biffo-plugin-marketing`: `public_base_url_for` was adopted at
11
+ **1 call site of 3** (marketing#72 — the two missed sites 500'd both packs),
12
+ and an artefact-body parser was filed against **2** sites when the tree held
13
+ **7 across 6 files** (marketing#49, swept in marketing#111).
14
+
15
+ `biffo-plugin-marketing` and `biffo-plugin-ideation` each grew a copy of this
16
+ guard by hand, after the fact, once their own instance of the class had
17
+ already shipped. #1587 is that the skeleton — the thing every plugin repo is
18
+ actually born from — carried no such test, so a **third** plugin starts life
19
+ non-adopting by construction and repeats the same class a third time before
20
+ anyone notices. Shipping the sweep here, rather than as a pattern to remember
21
+ to copy, is the fix: it travels with the code it watches from commit one.
22
+
23
+ ## Why this is parameterised rather than a copy of the reference
24
+
25
+ `biffo-plugin-marketing`'s copy of this file hardcodes `src / "marketing"` as
26
+ the package it sweeps. That literal cannot be copied into this skeleton: every
27
+ plugin scaffolded from it gets a *different* package name (`biffo plugin
28
+ create` rewrites `example_plugin` throughout the repo — see
29
+ `.scaffold-tokens.json` and `cli/src/lib/plugin-scaffold.ts`'s
30
+ `substitutions()`), so a hardcoded name here would be correct for zero
31
+ scaffolded repos. `_package_dir()` below finds the package by shape — the sole
32
+ directory under `src/` with an `__init__.py` — so it resolves correctly both
33
+ in this template (`example_plugin`) and in every plugin it produces, with
34
+ nothing to rename when the scaffold runs.
35
+
36
+ ## What counts as a consolidation helper
37
+
38
+ A function whose entire body is ``return <expression>`` (a docstring is
39
+ allowed). That is the shape of every helper this class has produced in the
40
+ sibling plugins: one expression, held once, called from many places. The
41
+ expression becomes a **template** whose parameters are wildcards; any
42
+ expression elsewhere in the package that structurally unifies with it is a
43
+ call site that bypassed the helper.
44
+
45
+ Three shapes are excluded, each for a reason rather than to quieten the output
46
+ (mirrored from the marketing reference copy, where each was needed for real):
47
+
48
+ - **FastAPI dependency providers** (a parameter defaulting to ``Depends(...)``).
49
+ A plugin adding an `api_ingress` app (ADR-0021) will typically declare these
50
+ per-router on purpose, so `dependency_overrides` can replace them per app —
51
+ consolidating them would break that.
52
+ - **Pure delegations** — a lone call whose every argument is a bare parameter
53
+ (``return await self._load_owned(owner_sub=owner_sub, ...)``). A façade like
54
+ that holds no expression, so every ordinary call to the underlying function
55
+ would be reported as bypassing it.
56
+ - **Expressions below `_MIN_TEMPLATE_NODES`.** ``return value or {}`` matches
57
+ half of any package and means nothing.
58
+
59
+ ## What this plugin's package sweeps to today, and why that is correct
60
+
61
+ At the time this file was added, this skeleton's own `example_plugin` package
62
+ holds **zero** consolidation-helper-shaped functions — `on_install()` and
63
+ `on_uninstall()` (**not invoked** by anything, biffo-template#709) are no-ops
64
+ returning a bare `None`, which is far below `_MIN_TEMPLATE_NODES`. That is the
65
+ right starting state, not a gap: a fresh
66
+ plugin has not yet had the chance to duplicate anything. The guard exists so
67
+ that when the first real helper — and later, the first hand-copy of its
68
+ expression — is written, this file catches it without anyone having to add a
69
+ new test for it. `test_the_swept_package_points_at_real_source` and
70
+ `test_the_guard_can_actually_fail` below exist precisely so a zero here is
71
+ never mistaken for "the collector is broken" (an empty denominator passing
72
+ for the wrong reason is the failure this whole class is about).
73
+
74
+ ## Waivers are a ledger, not a mute button
75
+
76
+ A real duplicate that should *not* be consolidated goes in `_ACCEPTED_DUPLICATES`
77
+ with the reason, so it is one review-visible line rather than a silently
78
+ tolerated copy — and `test_no_accepted_duplicate_has_gone_stale` deletes the
79
+ excuse when the code moves on.
80
+ """
81
+
82
+ from __future__ import annotations
83
+
84
+ import ast
85
+ from pathlib import Path
86
+
87
+ # --------------------------------------------------------------------------
88
+ # Subject: this plugin's own package, located by shape rather than by name —
89
+ # see the module docstring's "Why this is parameterised" section.
90
+ # --------------------------------------------------------------------------
91
+
92
+ _PLUGIN_ROOT = Path(__file__).resolve().parents[1]
93
+ _SRC_ROOT = _PLUGIN_ROOT / "src"
94
+
95
+
96
+ def _package_dir() -> Path:
97
+ """The plugin's own package directory, found by shape rather than by name.
98
+
99
+ `src/` holds exactly one package directory in this template and in every
100
+ plugin scaffolded from it (`packages = ["src/<name>"]` in `pyproject.toml`
101
+ is single-valued too), so locating the sole directory with an
102
+ `__init__.py` works everywhere a literal package name would not.
103
+ """
104
+ candidates = sorted(
105
+ p for p in _SRC_ROOT.iterdir() if p.is_dir() and (p / "__init__.py").is_file()
106
+ )
107
+ assert len(candidates) == 1, (
108
+ f"expected exactly one package under {_SRC_ROOT}, found "
109
+ f"{[c.name for c in candidates]} — the sweep cannot tell which one is "
110
+ "this plugin's own source."
111
+ )
112
+ return candidates[0]
113
+
114
+
115
+ _SRC = _package_dir()
116
+
117
+ #: Below this many AST nodes an expression is too common to mean anything.
118
+ #: Matches the marketing reference copy's threshold: it admits real duplicated
119
+ #: expressions (a ternary, an `HTTPException(...)` call) while still rejecting
120
+ #: `raw or {}` (4 nodes) — see `test_a_near_miss_is_not_reported` below for a
121
+ #: fixture at exactly that shape.
122
+ _MIN_TEMPLATE_NODES = 8
123
+
124
+ #: (helper name, file that re-derives it) -> why that copy is correct.
125
+ #: Empty today: this skeleton ships with no known-correct duplicate. A real
126
+ #: plugin built from it adds entries here as it earns them, same as the
127
+ #: marketing and ideation copies did.
128
+ _ACCEPTED_DUPLICATES: dict[tuple[str, str], str] = {}
129
+
130
+
131
+ # --------------------------------------------------------------------------
132
+ # Finding the helpers
133
+ # --------------------------------------------------------------------------
134
+
135
+
136
+ def _parameter_names(fn: ast.FunctionDef | ast.AsyncFunctionDef) -> set[str]:
137
+ """Every name bound by the signature — the template's wildcards."""
138
+ args = fn.args
139
+ names = {a.arg for a in [*args.posonlyargs, *args.args, *args.kwonlyargs]}
140
+ if args.vararg:
141
+ names.add(args.vararg.arg)
142
+ if args.kwarg:
143
+ names.add(args.kwarg.arg)
144
+ return names
145
+
146
+
147
+ def _sole_returned_expression(fn: ast.FunctionDef | ast.AsyncFunctionDef) -> ast.expr | None:
148
+ """The expression of a body that is exactly ``return <expr>``, or `None`.
149
+
150
+ A leading docstring is skipped, because helpers in this plugin's style
151
+ tend to have one and requiring a bare body would exclude them.
152
+ """
153
+ body = list(fn.body)
154
+ if (
155
+ body
156
+ and isinstance(body[0], ast.Expr)
157
+ and isinstance(body[0].value, ast.Constant)
158
+ and isinstance(body[0].value.value, str)
159
+ ):
160
+ body = body[1:]
161
+ if len(body) != 1:
162
+ return None
163
+ only = body[0]
164
+ return only.value if isinstance(only, ast.Return) and only.value is not None else None
165
+
166
+
167
+ def _node_count(node: ast.AST) -> int:
168
+ return sum(1 for _ in ast.walk(node))
169
+
170
+
171
+ def _is_dependency_provider(fn: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
172
+ """True for a FastAPI dependency (a parameter defaulting to `Depends(...)`).
173
+
174
+ Declared per-router on purpose where they exist: `dependency_overrides` is
175
+ keyed by the function object, so a shared one could not be overridden for
176
+ a single app under test.
177
+ """
178
+ defaults = [d for d in [*fn.args.defaults, *fn.args.kw_defaults] if d is not None]
179
+ return any(
180
+ isinstance(d, ast.Call)
181
+ and (getattr(d.func, "id", None) == "Depends" or getattr(d.func, "attr", None) == "Depends")
182
+ for d in defaults
183
+ )
184
+
185
+
186
+ def _is_pure_delegation(expr: ast.expr, params: set[str]) -> bool:
187
+ """True for ``return f(a, b=b)`` — a call forwarding parameters unchanged.
188
+
189
+ Such a function is a façade over another function, not a held expression:
190
+ treating it as a template would report every ordinary call to the callee as
191
+ a bypass of the façade, which is the opposite of what this guard means.
192
+ """
193
+ call = expr.value if isinstance(expr, ast.Await) else expr
194
+ if not isinstance(call, ast.Call):
195
+ return False
196
+ arguments = [*call.args, *[kw.value for kw in call.keywords]]
197
+ return all(isinstance(a, ast.Name) and a.id in params for a in arguments)
198
+
199
+
200
+ #: Node types that make an expression a *composition* rather than a lookup.
201
+ #: `JoinedStr`/`BinOp` are included because URL and key building is the
202
+ #: sibling plugins' most expensive drift shape (marketing#72 was a base URL),
203
+ #: and an f-string helper contains no call at all.
204
+ _COMPOSING_NODES = (
205
+ ast.Call,
206
+ ast.IfExp,
207
+ ast.BoolOp,
208
+ ast.Compare,
209
+ ast.JoinedStr,
210
+ ast.BinOp,
211
+ ast.ListComp,
212
+ ast.DictComp,
213
+ ast.SetComp,
214
+ ast.GeneratorExp,
215
+ )
216
+
217
+
218
+ def _has_structure(expr: ast.expr) -> bool:
219
+ """True if the expression actually composes something. Pure attribute
220
+ chains, subscripts and literals are shared by too much code to be evidence
221
+ of anything."""
222
+ return any(isinstance(n, _COMPOSING_NODES) for n in ast.walk(expr))
223
+
224
+
225
+ class _Helper:
226
+ """A consolidation helper and the expression it holds."""
227
+
228
+ def __init__(
229
+ self, module: str, fn: ast.FunctionDef | ast.AsyncFunctionDef, expr: ast.expr
230
+ ) -> None:
231
+ self.module = module
232
+ self.name = fn.name
233
+ self.lineno = fn.lineno
234
+ self.template = expr
235
+ self.params = _parameter_names(fn)
236
+ #: Node identities of the helper's own definition — the one place its
237
+ #: expression is allowed to appear. Identity, not line numbers, so
238
+ #: moving the function neither breaks the guard nor widens it.
239
+ self.own_nodes = {id(n) for n in ast.walk(fn)}
240
+
241
+ def __repr__(self) -> str: # pragma: no cover - diagnostics only
242
+ return f"{self.module}:{self.lineno} {self.name}()"
243
+
244
+
245
+ def consolidation_helpers(trees: dict[str, ast.AST]) -> list[_Helper]:
246
+ """Every single-expression helper in the swept package, discovered from
247
+ the tree.
248
+
249
+ This is the sweep's **denominator**, and the reason the guard does not
250
+ need a hand-maintained list of helper names: a helper added tomorrow lands
251
+ here on its own.
252
+ """
253
+ found: list[_Helper] = []
254
+ for module, tree in trees.items():
255
+ for node in ast.walk(tree):
256
+ if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
257
+ continue
258
+ expr = _sole_returned_expression(node)
259
+ if expr is None or _is_dependency_provider(node):
260
+ continue
261
+ if _node_count(expr) < _MIN_TEMPLATE_NODES or not _has_structure(expr):
262
+ continue
263
+ if _is_pure_delegation(expr, _parameter_names(node)):
264
+ continue
265
+ found.append(_Helper(module, node, expr))
266
+ return found
267
+
268
+
269
+ # --------------------------------------------------------------------------
270
+ # Matching a call site against a helper's expression
271
+ # --------------------------------------------------------------------------
272
+
273
+ #: Fields that carry no meaning for this comparison. `ctx` is Load/Store/Del,
274
+ #: which differs between an expression that is read and one that is assigned to
275
+ #: without the expressions themselves differing at all.
276
+ _IGNORED_FIELDS = {"ctx"}
277
+
278
+
279
+ def _unifies(
280
+ template: ast.AST, candidate: ast.AST, params: set[str], binding: dict[str, str]
281
+ ) -> bool:
282
+ """True if `candidate` is `template` with its parameters filled in.
283
+
284
+ A parameter is a wildcard that matches any sub-expression, but must match
285
+ the *same* sub-expression everywhere it appears — so
286
+ ``json.loads(raw) if isinstance(raw, str) else raw`` matches a call site
287
+ using one variable throughout and not one that mixes two.
288
+ """
289
+ if isinstance(template, ast.Name) and template.id in params:
290
+ dumped = ast.dump(candidate)
291
+ already = binding.get(template.id)
292
+ if already is None:
293
+ binding[template.id] = dumped
294
+ return True
295
+ return already == dumped
296
+
297
+ if type(template) is not type(candidate):
298
+ return False
299
+ if isinstance(template, ast.Name):
300
+ return template.id == candidate.id # type: ignore[attr-defined]
301
+ if isinstance(template, ast.Constant):
302
+ other = candidate.value # type: ignore[attr-defined]
303
+ return type(template.value) is type(other) and template.value == other
304
+
305
+ candidate_fields = dict(ast.iter_fields(candidate))
306
+ for field, expected in ast.iter_fields(template):
307
+ if field in _IGNORED_FIELDS:
308
+ continue
309
+ actual = candidate_fields.get(field)
310
+ if isinstance(expected, list):
311
+ if not isinstance(actual, list) or len(expected) != len(actual):
312
+ return False
313
+ for want, got in zip(expected, actual, strict=True):
314
+ if isinstance(want, ast.AST):
315
+ if not isinstance(got, ast.AST) or not _unifies(want, got, params, binding):
316
+ return False
317
+ elif want != got:
318
+ return False
319
+ elif isinstance(expected, ast.AST):
320
+ if not isinstance(actual, ast.AST) or not _unifies(expected, actual, params, binding):
321
+ return False
322
+ elif expected != actual:
323
+ return False
324
+ return True
325
+
326
+
327
+ def bypassing_call_sites(trees: dict[str, ast.AST]) -> list[tuple[str, str, int]]:
328
+ """`(helper name, module, line)` for every expression that re-derives a
329
+ helper's body outside that helper."""
330
+ helpers = consolidation_helpers(trees)
331
+ hits: set[tuple[str, str, int]] = set()
332
+ for helper in helpers:
333
+ for module, tree in trees.items():
334
+ for node in ast.walk(tree):
335
+ if not isinstance(node, ast.expr) or id(node) in helper.own_nodes:
336
+ continue
337
+ if type(node) is not type(helper.template):
338
+ continue
339
+ if _unifies(helper.template, node, helper.params, {}):
340
+ hits.add((helper.name, module, node.lineno))
341
+ return sorted(hits)
342
+
343
+
344
+ def _parse_package() -> dict[str, ast.AST]:
345
+ return {p.name: ast.parse(p.read_text(), filename=str(p)) for p in sorted(_SRC.glob("*.py"))}
346
+
347
+
348
+ # --------------------------------------------------------------------------
349
+ # The guard
350
+ # --------------------------------------------------------------------------
351
+
352
+
353
+ def test_the_swept_package_points_at_real_source() -> None:
354
+ """A collector that silently resolves to nothing passes for the wrong
355
+ reason. #1587 exists because the previous state of this skeleton had no
356
+ sweep at all; this pins that `_package_dir()` actually finds this plugin's
357
+ real, non-empty source tree rather than an empty or missing directory."""
358
+ assert _SRC.is_dir(), f"{_SRC} does not exist — the sweep is watching nothing"
359
+ py_files = list(_SRC.glob("*.py"))
360
+ assert py_files, f"{_SRC} has no source files — the sweep is watching nothing"
361
+
362
+
363
+ def test_no_module_re_derives_a_shared_helpers_expression() -> None:
364
+ offenders = [
365
+ f"{module}:{line} re-derives {helper}()"
366
+ for helper, module, line in bypassing_call_sites(_parse_package())
367
+ if (helper, module) not in _ACCEPTED_DUPLICATES
368
+ ]
369
+ assert not offenders, (
370
+ "These write out an expression a helper in this plugin already holds, "
371
+ "instead of calling it (biffo-template#1587). A hand-written copy "
372
+ "stops tracking the helper the moment the helper is corrected, and "
373
+ "review cannot see the difference — an unconverted call site looks "
374
+ "exactly like code that was never meant to use the helper. Call the "
375
+ "helper, or record why this copy is correct in "
376
+ "`_ACCEPTED_DUPLICATES`:\n " + "\n ".join(offenders)
377
+ )
378
+
379
+
380
+ def test_the_guard_can_actually_fail() -> None:
381
+ """The detector must detect, on source that is never imported.
382
+
383
+ This plugin's real package holds zero consolidation-helper-shaped
384
+ functions today (see the module docstring), so proving the guard against
385
+ it would only prove it stays green over an empty denominator. This plants
386
+ a fresh helper and a site that bypasses it instead — the same synthetic
387
+ check the marketing reference copy uses, for the same reason.
388
+ """
389
+ tree = ast.parse(
390
+ "def _normalise(raw):\n"
391
+ ' """Hold this once."""\n'
392
+ " return json.loads(raw) if isinstance(raw, str) else (raw or {})\n"
393
+ "\n"
394
+ "def converted(row):\n"
395
+ " return _normalise(row.get('body'))\n"
396
+ "\n"
397
+ "def bypassing(row):\n"
398
+ " raw = row.get('body')\n"
399
+ " return json.loads(raw) if isinstance(raw, str) else (raw or {})\n"
400
+ )
401
+ assert bypassing_call_sites({"planted.py": tree}) == [("_normalise", "planted.py", 10)]
402
+
403
+
404
+ def test_a_near_miss_is_not_reported() -> None:
405
+ """Different code must not be called drift.
406
+
407
+ A guard that fires on anything adjacent gets waived wholesale, so the
408
+ near-misses are pinned: a different test, a different call, and the same
409
+ shape over two different variables are all legitimate code.
410
+
411
+ (Each near-miss is written with a second statement so it is a call site
412
+ rather than a helper of its own. A one-line function IS a template, and a
413
+ template whose parameters are wildcards legitimately matches another
414
+ helper's body — that is a duplicated *helper*, which this guard reports on
415
+ purpose.)
416
+ """
417
+ tree = ast.parse(
418
+ "def _normalise(raw):\n"
419
+ " return json.loads(raw) if isinstance(raw, str) else (raw or {})\n"
420
+ "\n"
421
+ "def different_test(row):\n"
422
+ " raw = row.get('body')\n"
423
+ " return json.loads(raw) if raw.startswith('{') else (raw or {})\n"
424
+ "\n"
425
+ "def different_call(row):\n"
426
+ " raw = row.get('body')\n"
427
+ " return int(raw) if isinstance(raw, str) else (raw or {})\n"
428
+ "\n"
429
+ "def two_variables(row, other):\n"
430
+ " raw = row.get('body')\n"
431
+ " return json.loads(raw) if isinstance(other, str) else (other or {})\n"
432
+ )
433
+ assert bypassing_call_sites({"planted.py": tree}) == []
434
+
435
+
436
+ def test_a_dependency_provider_is_not_treated_as_a_helper() -> None:
437
+ """A router-style `Depends(...)` default must not be swept as a helper.
438
+
439
+ If this exclusion were dropped, a plugin declaring the same FastAPI
440
+ dependency provider in several routers on purpose (so
441
+ `dependency_overrides` can replace it per app) would report every
442
+ duplicate as an offender, and the only correct resolution would be a
443
+ waiver — a guard that starts life with a wall of waivers is one nobody
444
+ reads.
445
+ """
446
+ tree = ast.parse(
447
+ "def get_campaign_client(admin = Depends(require_admin)):\n"
448
+ " return principal_client.PrincipalCoreClient(admin.token, verify=True)\n"
449
+ "\n"
450
+ "def other_router_provider(admin = Depends(require_admin)):\n"
451
+ " return principal_client.PrincipalCoreClient(admin.token, verify=True)\n"
452
+ )
453
+ assert consolidation_helpers({"planted.py": tree}) == []
454
+ assert bypassing_call_sites({"planted.py": tree}) == []
455
+
456
+
457
+ def test_a_pure_delegation_is_not_treated_as_a_helper() -> None:
458
+ """`return await self._load_owned(owner_sub=..., session_id=...)` holds no
459
+ expression — every ordinary call to `_load_owned` would otherwise be
460
+ reported as bypassing the façade in front of it."""
461
+ tree = ast.parse(
462
+ "class S:\n"
463
+ " async def get_session(self, *, owner_sub, session_id):\n"
464
+ " return await self._load_owned(owner_sub=owner_sub, session_id=session_id)\n"
465
+ "\n"
466
+ " async def chat_turn(self, *, owner_sub, session_id):\n"
467
+ " session = await self._load_owned(owner_sub=owner_sub, session_id=session_id)\n"
468
+ " return session\n"
469
+ )
470
+ assert consolidation_helpers({"planted.py": tree}) == []
471
+
472
+
473
+ def test_no_accepted_duplicate_has_gone_stale() -> None:
474
+ """A waiver outlives the code it excuses unless something deletes it.
475
+
476
+ Each entry must still name a duplicate the sweep really finds. When the
477
+ copy is consolidated or the file is renamed, this fails and the excuse goes
478
+ with it, rather than sitting there granting a permission nobody re-examined.
479
+ """
480
+ live = {(helper, module) for helper, module, _ in bypassing_call_sites(_parse_package())}
481
+ stale = sorted(pair for pair in _ACCEPTED_DUPLICATES if pair not in live)
482
+ assert not stale, (
483
+ "These waivers no longer excuse anything — the duplicate is gone or has "
484
+ f"moved. Delete them: {stale}"
485
+ )
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@biffo/cli",
3
- "version": "0.296.14",
3
+ "version": "0.296.16",
4
4
  "description": "Biffo project scaffolding CLI",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -43,6 +43,7 @@
43
43
  # eval "$(sh scripts/pg-test-db.sh --export)" # export BIFFO_TEST_PG_DSN and TABSII_TEST_PG_DSN
44
44
  # sh scripts/pg-test-db.sh # print the DSN on stdout
45
45
  # sh scripts/pg-test-db.sh --recreate # force a rebuild
46
+ # sh scripts/pg-test-db.sh --reap # housekeeping only: reap and exit
46
47
  #
47
48
  # Only the DSN reaches stdout, so it is safe to capture; progress goes to stderr.
48
49
  #
@@ -164,10 +165,12 @@ CONTAINER="${BIFFO_PG_CONTAINER:-biffo-pg-test-$_checkout_suffix}"
164
165
 
165
166
  RECREATE=0
166
167
  EXPORT=0
168
+ REAP_ONLY=0
167
169
  for arg in "$@"; do
168
170
  case "$arg" in
169
171
  --recreate) RECREATE=1 ;;
170
172
  --export) EXPORT=1 ;;
173
+ --reap) REAP_ONLY=1 ;;
171
174
  -h | --help)
172
175
  sed -n '2,72p' "$0" | sed 's/^#\{1,2\} \{0,1\}//'
173
176
  exit 0
@@ -250,6 +253,7 @@ if [ "$BIFFO_PG_REAP_HOURS" -gt 0 ] 2>/dev/null && command -v docker >/dev/null
250
253
  say "cannot compute a reap cutoff on this date(1); skipping container reaping"
251
254
  else
252
255
  _reaped=0
256
+ _reaped_gone=0
253
257
  _considered=0
254
258
  # TWO filters, not one literal name (#1383). The label is what containers
255
259
  # created from here now carry; the name prefix keeps covering every one
@@ -272,14 +276,65 @@ if [ "$BIFFO_PG_REAP_HOURS" -gt 0 ] 2>/dev/null && command -v docker >/dev/null
272
276
  _made=$(docker inspect -f '{{.Created}}' "$_c" 2>/dev/null | cut -c1-19)
273
277
  [ -z "$_made" ] && continue
274
278
  _considered=$((_considered + 1))
279
+ # ── Ownership beats age, where ownership is knowable ─────────────────
280
+ #
281
+ # Age was only ever a PROXY. The container is keyed to a checkout
282
+ # (see `CONTAINER` above), so the honest question is not "is this old?"
283
+ # but "does the checkout that owns it still exist?" -- and once the
284
+ # worktree is deleted the answer is a fact, not an estimate. A container
285
+ # whose checkout is gone can never be reused by anything, so there is no
286
+ # 4-second-rebuild trade to weigh: it is pure garbage the moment the
287
+ # directory disappears.
288
+ #
289
+ # Measured 2026-08-22 on one workstation: 42 live containers, of which 19
290
+ # belonged to checkouts that no longer existed. Under the age rule alone
291
+ # those 19 each held a running Postgres and ~500MB for up to 24 more
292
+ # hours -- and #703's real complaint was never disk, it was that these
293
+ # compete for the same page cache and I/O as the lane being timed.
294
+ #
295
+ # The path is read from a LABEL SET AT CREATION, never derived from the
296
+ # container's name. Deriving it would mean hashing candidate paths to see
297
+ # which produces this suffix, and the `biffo-pg-test-` prefix is shared by
298
+ # every repo in the estate -- so a run in one repo, finding no matching
299
+ # worktree of its OWN, would confidently reap a container another repo's
300
+ # test lane was mid-run against. The label makes the claim self-describing
301
+ # and repo-independent.
302
+ #
303
+ # Containers created before this label existed report an empty value and
304
+ # fall through to the age rule below, exactly as `biffo.ephemeral=1`
305
+ # migrated in (#1383). Nothing is stranded; they simply age out once.
306
+ _owner=$(docker inspect -f '{{index .Config.Labels "biffo.checkout"}}' "$_c" 2>/dev/null)
307
+ if [ -n "$_owner" ] && [ "$_owner" != "<no value>" ] && [ ! -d "$_owner" ]; then
308
+ if docker rm -f -v "$_c" >/dev/null 2>&1; then
309
+ _reaped=$((_reaped + 1))
310
+ _reaped_gone=$((_reaped_gone + 1))
311
+ fi
312
+ continue
313
+ fi
275
314
  # Both are UTC ISO-8601 to the second, so a string compare IS a time
276
315
  # compare -- no epoch conversion, and portable across date(1) flavours.
277
316
  if awk -v a="$_made" -v b="$_reap_cutoff" 'BEGIN { exit !(a < b) }'; then
278
- docker rm -f "$_c" >/dev/null 2>&1 && _reaped=$((_reaped + 1))
317
+ # `-v` REMOVES THE CONTAINER'S ANONYMOUS VOLUME WITH IT.
318
+ #
319
+ # Without it every reap orphans a full Postgres data directory. Measured on one
320
+ # workstation 2026-08-20: 413 dangling volumes holding 104.8GB -- 95% of all local
321
+ # volume space -- against 11 live containers totalling 2.5MB. The containers were
322
+ # tidied and their data was not, so the leak grew by roughly a database per reap
323
+ # and nothing pointed at it.
324
+ #
325
+ # It is invisible by construction: `docker ps` looks clean, the reaper reports how
326
+ # many it removed, and the space is only findable with `docker volume ls -qf
327
+ # dangling=true`. The first symptom is a full disk somewhere unrelated.
328
+ docker rm -f -v "$_c" >/dev/null 2>&1 && _reaped=$((_reaped + 1))
279
329
  fi
280
330
  done
281
- [ "$_reaped" -gt 0 ] &&
282
- say "reaped $_reaped of $_considered container(s) unused for over ${BIFFO_PG_REAP_HOURS}h (set BIFFO_PG_REAP_HOURS=0 to disable)"
331
+ # Two reasons, counted apart. A single total would let the cheap, certain
332
+ # rule and the age guess read as one number, and the whole point of the
333
+ # ownership rule is that it is NOT a guess -- if it ever reaps something
334
+ # still wanted, that total must say so on its own.
335
+ if [ "$_reaped" -gt 0 ]; then
336
+ say "reaped $_reaped of $_considered container(s): $_reaped_gone whose checkout no longer exists, $((_reaped - _reaped_gone)) unused for over ${BIFFO_PG_REAP_HOURS}h (set BIFFO_PG_REAP_HOURS=0 to disable)"
337
+ fi
283
338
 
284
339
  # ── What the reaper can SEE but must not touch ──────────────────────────
285
340
  #
@@ -330,10 +385,31 @@ if [ "$BIFFO_PG_REAP_HOURS" -gt 0 ] 2>/dev/null && command -v docker >/dev/null
330
385
  say "NOT reaped -- Postgres containers over ${BIFFO_PG_REAP_HOURS}h old that this script did not create:"
331
386
  for _u in $_unclaimed; do say " $_u"; done
332
387
  say " A stale one costs test failures that belong to nobody (#1383). Remove by hand"
333
- say " (docker rm -f <name>), or start it with --label biffo.ephemeral=1 to have it reaped."
388
+ say " (docker rm -f -v <name> -- the -v matters, or its data volume is orphaned),"
389
+ say " or start it with --label biffo.ephemeral=1 to have it reaped."
334
390
  fi
335
391
  }
336
392
  fi
393
+
394
+ # `--reap` is housekeeping ONLY: reap, report, and stop before starting or
395
+ # touching a server.
396
+ #
397
+ # The reaper is otherwise LAZY -- it runs only when something else runs the
398
+ # lane, so the moment the fleet goes quiet nothing reclaims anything and the
399
+ # mess sits until the next test. That is the opposite of what is wanted: idle
400
+ # is exactly when reclaiming is free. This flag is the callable form, so a
401
+ # worktree teardown or a periodic sweep can collect without standing up a
402
+ # Postgres nobody asked for.
403
+ if [ "$REAP_ONLY" -eq 1 ]; then
404
+ exit 0
405
+ fi
406
+ fi
407
+
408
+ # Guard the case above: with reaping disabled there is nothing for `--reap` to
409
+ # do, and it must still not fall through into starting a server.
410
+ if [ "$REAP_ONLY" -eq 1 ]; then
411
+ say "reaping is disabled (BIFFO_PG_REAP_HOURS=0); nothing to do"
412
+ exit 0
337
413
  fi
338
414
 
339
415
  if ! psql_admin -c 'SELECT 1' >/dev/null 2>&1; then
@@ -352,7 +428,11 @@ if ! psql_admin -c 'SELECT 1' >/dev/null 2>&1; then
352
428
  # is still scanned, but it only ever described containers this script named;
353
429
  # anything started under another name was outside the reaper's denominator
354
430
  # entirely. A label travels with the container whatever it is called.
431
+ # `biffo.checkout` is what makes the container's owner knowable after the
432
+ # fact. The reaper above uses it to remove a container the moment its
433
+ # checkout is deleted, rather than waiting out a 24-hour proxy.
355
434
  docker run -d --name "$CONTAINER" --label biffo.ephemeral=1 \
435
+ --label "biffo.checkout=$REPO_ROOT" \
356
436
  -e POSTGRES_PASSWORD="$PASS" -p "$PORT:5432" "$IMAGE" >/dev/null
357
437
  fi
358
438
  # Polled, not slept: a cold image pull and a warm restart differ by an order of