wicked-garden 12.29.1 → 12.30.0

package/.claude-plugin/marketplace.json

@@ -8,8 +8,8 @@
       "name": "wicked-garden",
       "source": "./",
       "description": "Your coding agent already plans and swarms \u2014 Wicked Garden is the curated toolkit for what it can't: proof-not-claims gates, code links grep can't see, deterministic refactor, cross-session memory, real multi-model second opinions. Don't fight the harness; fill its gaps. The gate needs wicked-vault (auto-installed via wicked-testing) and re-derives through garden's built-in engine; testing/brain/bus are opt-in layers.",
-      "version": "12.29.1"
+      "version": "12.30.0"
     }
   ],
-  "version": "12.29.1"
+  "version": "12.30.0"
 }

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "wicked-garden",
-  "version": "12.29.1",
+  "version": "12.30.0",
   "description": "Your coding agent's harness already plans and swarms; Wicked Garden is the curated toolkit for what it can't do alone. Proof, not claims \u2014 gates re-derive 'done' through the internal loom engine (fail-closed, independently attested via wicked-vault), so a self-asserted pass can't lie its way green. Relationships grep can't see \u2014 codegraph + injected bus/dispatch/capability edges power blast-radius & lineage. Plus deterministic multi-file refactor (wicked-patch), cross-session memory (wicked-brain), real multi-model second opinions (jam:council), portable expertise as on-demand skill-refs, and evidence-gated testing (wicked-testing). It reads the shape of your work to apply the right rigor \u2014 steering, not a pipeline \u2014 then gets out of the way. The compiler stamps a self-contained evidence gate into any repo that runs with no wicked-garden installed. Don't fight the harness; fill its gaps. The evidence gate requires wicked-vault (installed automatically via wicked-testing); wicked-testing, wicked-brain, and wicked-bus are opt-in layers \u2014 install what you'll use, the toolkit works without the rest.",
   "wicked_testing_version": "^0.8.0",
   "wicked_vault_version": "^0.4.0",

package/.product/REQ-005-dod-criteria.md

@@ -1,8 +1,8 @@
 ---
 name: REQ-005-dod-criteria
 title: wicked-garden — Definition of Done Criteria
-status: partially-verified
-version: 0.9
+status: verified
+version: 1.0
 date: 2026-07-21
 author: mike.parcewski@gmail.com
 review-required: true
@@ -93,8 +93,10 @@ These criteria require independent evaluation — the evaluator is not the agent
   <!-- evidence: PR #1010 (chore/garden-release-12.29.1) merged to main. plugin.json, marketplace.json, package.json, package-lock.json all bumped to 12.29.1. Patch bump for new propagation tests (L2-013/014 via PR #1009) + docstring fixes + package.json sync from 12.28.1. Prior: PR #1007 bumped plugin.json/marketplace.json to 12.29.0 (minor: sentinel event, two event renames, npm test script). (2026-07-21) -->
 - [x] **L3-008** — `components.json` regenerated and committed with the version bump.
   <!-- evidence: `python3 scripts/ci/sync_components.py` → "components.json: in sync" — no diff from current skill tree. components.json verified current in PR #1010. (2026-07-21) -->
-- [ ] **L3-009** — Release tagged in git and published to the marketplace registry.
-- [ ] **L3-010** — `test.yml` CI workflow is green on the release tag (unit tests + E2E trust-spine tests + wicked-patch conformance tests all pass).
+- [x] **L3-009** — Release tagged in git and published to the marketplace registry.
+  <!-- evidence: tag `v12.29.1` pushed to GitHub. GitHub release created at https://github.com/mikeparcewski/wicked-garden/releases/tag/v12.29.1. `release.yml` ran on the tag (run ID 29859516956) — pre-publish check passed (`validate.py` + `_validate_registry.py`); npm package published to registry via OIDC trusted publishing (run conclusion: success). (2026-07-21) -->
+- [x] **L3-010** — `test.yml` CI workflow is green on the release tag (unit tests + E2E trust-spine tests + wicked-patch conformance tests all pass).
+  <!-- evidence: `test.yml` (named "v11 Test Suite") does not trigger directly on tags but ran on the PR #1014 merge commit `452d3c55`, which is the exact same commit that was tagged v12.29.1. Run ID 29858482241: `test` job conclusion=success (1m26s). 972 passed, 17 skipped. This satisfies "CI green on the release commit". (2026-07-21) -->
 
 ---
 
@@ -127,3 +129,4 @@ These criteria require independent evaluation — the evaluator is not the agent
 | 0.7 | 2026-07-21 | mike.parcewski@gmail.com | L2-004 checked off: hard-gate attestation rejects env-user artifacts end-to-end. vault 0.9.0 (`npx wicked-vault`) — `record` without `--actor` sets `created_by_source='env-user'`; subsequent `attest` exits 1 with error G10/D4 (fail-closed, no --allow-weak-worker-identity). |
 | 0.8 | 2026-07-21 | mike.parcewski@gmail.com | L2-010b, L2-011, L2-012 checked off: compiler end-to-end verified. `compile.py /tmp/l2-010b-test-repo` (package.json with `npm test`) emitted gate.py + contract.json; `python3 .wicked/gate.py` → PASS (vault resolved via npx, no garden install required). `--trigger hook` → pre-push hook created at `.git/hooks/pre-push` (runs gate.py, blocks on non-zero). `--trigger ci` → `.github/workflows/wicked-gate.yml` created (on push+PR, gate job runs gate.py). |
 | 0.9 | 2026-07-21 | mike.parcewski@gmail.com | L3-002 checked off: acceptance gate verdict recorded in wicked-vault under explicit actor. `npx wicked-vault record ... --actor garden-prove` → artifact `019F85FA69743603A39293017738`, `created_by_source='explicit'`. `npx wicked-vault verify 019F85FA69743603A39293017738` → `hash_ok: true, payload_ok: true, rederived: true`. Verdict is re-derivable, not self-asserted. vault 0.9.0. |
+| 1.0 | 2026-07-21 | mike.parcewski@gmail.com | L3-009 and L3-010 checked off. L3-009: `git tag v12.29.1` + `git push origin v12.29.1`; GitHub release at https://github.com/mikeparcewski/wicked-garden/releases/tag/v12.29.1; `release.yml` (run 29859516956) ran pre-publish check + npm publish via OIDC (conclusion: success). L3-010: test.yml ("v11 Test Suite") ran on PR #1014 merge commit `452d3c55` (same commit as tag); run 29858482241 conclusion=success (972 passed, 17 skipped). Status promoted to `verified` — all 42 L1/L2/L3 items checked off. |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wicked-garden",
-  "version": "12.29.1",
+  "version": "12.30.0",
   "type": "module",
   "description": "Curated toolkit for what AI coding agents can't do alone: evidence-gated work, codegraph relationships, deterministic multi-file refactor, multi-model second opinions, and cross-session memory. Installs as a Claude Code plugin.",
   "keywords": [

package/scripts/domain/_rule_extractor.py

@@ -38,6 +38,13 @@ _PROMPT_HEADER = (
     '"source_kinds": ["code-body"|"type-def"|"comment"|"doc", ...]}}\n'
     "If a unit’s behavior is unclear, set statement to \"\" and confidence low — do "
     "NOT guess. Ground trusted rules in code-body/type-def, not comment/doc alone.\n"
+    "State each rule in PRODUCT-FUNCTIONAL terms: what the business requires, not the "
+    "implementation. Name a specific tool, library, or technology ONLY when the source "
+    "makes that tool itself the requirement — otherwise state the capability (e.g. "
+    "'adversarial prompt-injection testing', not a vendor suite). For configuration or "
+    "data-fixture units (lockfiles, manifests, data JSON/YAML), state a rule only when "
+    "it encodes a genuine business constraint; format trivia and tool settings take "
+    "statement \"\".\n"
 )
 
 

package/scripts/domain/batch_extract.py

@@ -0,0 +1,226 @@
+#!/usr/bin/env python3
+"""Bulk rule extraction via the Anthropic Message Batches API (batch rates, 50% off).
+
+Same evidence contract as extract_loop.py — the model boundary changes shape
+(one async batch instead of N serial calls), but every returned rule passes the
+SAME validate_rule gate and lands through the SAME write path (validated
+requirement + business_rule annotation, or RISK floor). Nodes absent from the
+results stay unaccounted; the coverage authority re-derives them next pass.
+
+Modes (stdlib only, no SDK):
+    prepare  — frame every unaccounted node from the live store, chunk into
+               batch requests, write requests JSONL + a manifest with token/cost
+               estimates. No key needed.
+    submit   — POST the prepared requests as one batch (needs ANTHROPIC_API_KEY).
+    poll     — report batch status (request_counts) until ended.
+    ingest   — download results, validate every rule, write to the store via the
+               harness's write path; print a yield report.
+
+Typical run:
+    python3 batch_extract.py prepare --db .codegraph/estate.db --workdir .wicked-batch
+    python3 batch_extract.py submit --workdir .wicked-batch
+    python3 batch_extract.py poll --workdir .wicked-batch
+    python3 batch_extract.py ingest --db .codegraph/estate.db --workdir .wicked-batch
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+from domain import _clients, _rule_extractor, extract_loop  # noqa: E402
+
+_API = "https://api.anthropic.com/v1/messages/batches"
+_MODEL = os.environ.get("WICKED_RULE_MODEL_API_MODEL", "claude-haiku-4-5-20251001")
+_UNITS_PER_REQUEST = 16
+# Batch-rate pricing for haiku 4.5 (USD per MTok, 50% of standard).
+_BATCH_IN, _BATCH_OUT = 0.50, 2.50
+
+
+def _headers(key: str) -> dict:
+    return {"content-type": "application/json", "x-api-key": key,
+            "anthropic-version": "2023-06-01"}
+
+
+def _api(method: str, url: str, key: str, body: dict | None = None) -> dict:
+    req = urllib.request.Request(url, method=method,
+                                 data=json.dumps(body).encode() if body else None,
+                                 headers=_headers(key))
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            return json.loads(resp.read().decode())
+    except urllib.error.HTTPError as e:
+        raise RuntimeError(f"HTTP {e.code}: {e.read().decode(errors='replace')[:300]}") from e
+
+
+def _require_key() -> str:
+    key = os.environ.get("ANTHROPIC_API_KEY", "").strip()
+    if not key:
+        sys.exit("ANTHROPIC_API_KEY is not set")
+    return key
+
+
+def prepare(db: str, workdir: Path) -> int:
+    workdir.mkdir(parents=True, exist_ok=True)
+    store = extract_loop._DirectStore(db)
+    core = _clients.core_client()
+    cov = core.coverage(db, str(workdir / "coverage_snapshot.json"))
+    worklist = cov.get("unaccounted_nodes") or []
+    if not worklist:
+        print("nothing unaccounted — store is fully covered")
+        return 0
+
+    # Cluster-order the whole worklist (same structural idea as the loop) so each
+    # request holds RELATED symbols.
+    node_community = {}
+    try:
+        clusters = _clients.estate_client(db=db).read_clusters()
+        node_community = _clients.total_node_community(clusters, worklist)
+    except Exception:
+        pass
+    worklist.sort(key=lambda n: node_community.get(n["symbol_id"], "~"))
+
+    requests, chars = [], 0
+    for i in range(0, len(worklist), _UNITS_PER_REQUEST):
+        chunk = worklist[i:i + _UNITS_PER_REQUEST]
+        framed = []
+        for n in chunk:
+            sid = n["symbol_id"]
+            framed.append(_rule_extractor.frame_context(
+                n, store.source_slice(sid),
+                cluster_label=node_community.get(sid),
+                neighbor_names=store.blast_neighbors(sid)))
+        prompt = _rule_extractor._build_prompt(framed)
+        cut = prompt.find("\n--- UNIT 1 ---\n")
+        system_part, user_part = (prompt[:cut], prompt[cut:]) if cut > 0 else ("", prompt)
+        chars += len(prompt)
+        params: dict = {"model": _MODEL, "max_tokens": 8192,
+                        "messages": [{"role": "user", "content": user_part}]}
+        if system_part:
+            params["system"] = [{"type": "text", "text": system_part,
+                                 "cache_control": {"type": "ephemeral"}}]
+        requests.append({"custom_id": f"req-{i // _UNITS_PER_REQUEST:05d}",
+                         "params": params,
+                         "_symbol_ids": [n["symbol_id"] for n in chunk]})
+
+    with open(workdir / "requests.jsonl", "w") as f:
+        for r in requests:
+            f.write(json.dumps(r) + "\n")
+    in_tok = chars // 4  # coarse chars→tokens
+    out_tok = len(worklist) * 90
+    manifest = {
+        "db": db, "model": _MODEL, "nodes": len(worklist),
+        "requests": len(requests), "units_per_request": _UNITS_PER_REQUEST,
+        "est_input_tokens": in_tok, "est_output_tokens": out_tok,
+        "est_cost_usd_batch_rates": round(in_tok / 1e6 * _BATCH_IN + out_tok / 1e6 * _BATCH_OUT, 2),
+    }
+    (workdir / "manifest.json").write_text(json.dumps(manifest, indent=2))
+    print(json.dumps(manifest, indent=2))
+    return 0
+
+
+def submit(workdir: Path) -> int:
+    key = _require_key()
+    reqs = []
+    for line in (workdir / "requests.jsonl").read_text().splitlines():
+        r = json.loads(line)
+        reqs.append({"custom_id": r["custom_id"], "params": r["params"]})
+    batch = _api("POST", _API, key, {"requests": reqs})
+    (workdir / "batch.json").write_text(json.dumps(batch, indent=2))
+    print(f"batch submitted: {batch['id']} status={batch['processing_status']} "
+          f"({len(reqs)} requests)")
+    return 0
+
+
+def poll(workdir: Path) -> int:
+    key = _require_key()
+    bid = json.loads((workdir / "batch.json").read_text())["id"]
+    batch = _api("GET", f"{_API}/{bid}", key)
+    (workdir / "batch.json").write_text(json.dumps(batch, indent=2))
+    print(json.dumps({"id": bid, "status": batch["processing_status"],
+                      "counts": batch.get("request_counts")}, indent=2))
+    return 0 if batch["processing_status"] == "ended" else 3
+
+
+def ingest(db: str, workdir: Path) -> int:
+    key = _require_key()
+    batch = json.loads((workdir / "batch.json").read_text())
+    if batch["processing_status"] != "ended":
+        sys.exit(f"batch not ended (status={batch['processing_status']}) — poll first")
+    url = batch["results_url"]
+    req = urllib.request.Request(url, headers=_headers(key))
+    with urllib.request.urlopen(req, timeout=600) as resp:
+        results_text = resp.read().decode()
+    (workdir / "results.jsonl").write_text(results_text)
+
+    ids_by_custom = {}
+    for line in (workdir / "requests.jsonl").read_text().splitlines():
+        r = json.loads(line)
+        ids_by_custom[r["custom_id"]] = set(r["_symbol_ids"])
+
+    estate = _clients.estate_client(db=db)
+    store = extract_loop._DirectStore(db)
+    ok = floored = kept = errored = 0
+    for line in results_text.splitlines():
+        res = json.loads(line)
+        cid = res["custom_id"]
+        batch_ids = ids_by_custom.get(cid, set())
+        result = res.get("result") or {}
+        if result.get("type") != "succeeded":
+            errored += len(batch_ids)  # untouched — stays unaccounted for a retry pass
+            continue
+        msg = result["message"]
+        text = "".join(b.get("text", "") for b in msg.get("content", [])
+                       if isinstance(b, dict) and b.get("type") == "text")
+        try:
+            rules = _rule_extractor._extract_json_array(text)
+        except Exception:
+            errored += len(batch_ids)
+            continue
+        by_id = {r["symbol_id"]: r for r in rules
+                 if isinstance(r, dict) and r.get("symbol_id") in batch_ids}
+        for sid in batch_ids:
+            rule = by_id.get(sid)
+            valid, reason = (_rule_extractor.validate_rule(rule, batch_ids) if rule
+                             else (False, "no rule returned for this node"))
+            name = sid.rsplit("/", 1)[-1]
+            if valid and float(rule["confidence"]) >= extract_loop.RESOLVE_THRESHOLD:
+                extract_loop._write_node(estate, sid, name, rule, True, "")
+                ok += 1
+            elif rule and (rule.get("statement") or "").strip():
+                extract_loop._write_node(estate, sid, name, rule, False,
+                                         reason if not valid else "below confidence threshold")
+                floored += 1
+            elif extract_loop._has_statement(store, sid):
+                kept += 1  # statement-less miss over existing content — keep it (monotonic)
+            else:
+                extract_loop._write_node(estate, sid, name, rule, False, reason)
+                floored += 1
+    print(json.dumps({"validated": ok, "risk_floored": floored, "kept_existing": kept,
+                      "errored_left_unaccounted": errored}, indent=2))
+    return 0
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("mode", choices=["prepare", "submit", "poll", "ingest"])
+    ap.add_argument("--db", default=os.environ.get("WICKED_ESTATE_DB"))
+    ap.add_argument("--workdir", default=".wicked-batch")
+    args = ap.parse_args()
+    wd = Path(args.workdir)
+    if args.mode == "prepare":
+        return prepare(args.db, wd)
+    if args.mode == "submit":
+        return submit(wd)
+    if args.mode == "poll":
+        return poll(wd)
+    return ingest(args.db, wd)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/scripts/domain/extract_loop.py

@@ -49,6 +49,21 @@ def _cohesion(sid: str, node_community: dict[str, str], community_sizes: dict[st
     return min(1.1, 0.95 + 0.03 * size)
 
 
+_INFRA_MARKERS = ("session limit", "rate limit", "rate_limit", "overloaded", "quota",
+                  "credit balance", "billing", "unauthorized", "authentication",
+                  "connection refused", "connection reset", "network", "econnrefused",
+                  "usage limit", "insufficient")
+
+
+def _is_infra_failure(exc: Exception) -> bool:
+    """True when the model boundary failed for infrastructure reasons (quota, auth,
+    network) rather than judgment. A per-batch timeout stays a judgment failure —
+    THAT batch was too hard — but an account-level outage would fail every future
+    batch identically, so it must stop the pass, not floor the worklist."""
+    msg = str(exc).lower()
+    return any(m in msg for m in _INFRA_MARKERS)
+
+
 def _stub_rule(node: dict) -> dict:
     """Deterministic no-model rule: a valid, confident business rule per node so a
     `--dry-run` pass drives coverage to 1.0 (proves the loop + gate, zero model cost)."""
@@ -62,6 +77,24 @@ def _stub_rule(node: dict) -> dict:
     }
 
 
+_RETRY_PAUSE = 45.0  # seconds between the two attempts on a suspect batch
+
+_FLOOR_MARKERS = ("no rule returned for this node", "empty statement (model could not state a rule)")
+
+
+def _has_statement(store, symbol_id: str) -> bool:
+    """True when the node already carries a substantive statement (not a floor
+    placeholder) — the signal that a failed retry must not degrade it. Reads the
+    store snapshot; without a direct store the answer is unknowable cheaply, so
+    the guard stays inactive (flat-path behavior unchanged)."""
+    if store is None:
+        return False
+    req = (store.requirement_of(symbol_id) or "").strip()
+    if not req:
+        return False
+    return not any(m in req for m in _FLOOR_MARKERS)
+
+
 def _write_node(estate, sid: str, name: str, rule: dict | None, resolved: bool, reason: str) -> None:
     """The two coordinated writes + read-back (vault record+verify). RESOLVED ⇒ a
     validated requirement + business_rule annotation; RISK ⇒ a non-blank requirement
@@ -99,8 +132,131 @@ def _write_node(estate, sid: str, name: str, rule: dict | None, resolved: bool,
         raise RuntimeError(f"write not durable: {sid} missing annotation {rid} on read-back")
 
 
+_ZSTD_MAGIC = b"\x28\xb5\x2f\xfd"
+
+
+def _maybe_decompress(blob) -> bytes:
+    """Estate stores content zstd-compressed; node spans are byte offsets into the
+    DECOMPRESSED text. Slicing the raw blob feeds the model binary noise — observed
+    live as whole batches of 'cannot state a rule'. Returns b'' when the blob is
+    compressed but no decompressor is available, which routes framing to the
+    subprocess fallback instead of producing garbage."""
+    b = blob if isinstance(blob, bytes) else bytes(blob)
+    if not b.startswith(_ZSTD_MAGIC):
+        return b
+    try:
+        from compression import zstd  # stdlib, Python 3.14+
+        return zstd.decompress(b)
+    except ImportError:
+        pass
+    try:
+        import zstandard
+        return zstandard.ZstdDecompressor().decompress(b)
+    except ImportError:
+        return b""
+
+
+class _DirectStore:
+    """READ-ONLY direct sqlite access for structural context (CommandIQ req-engine v3 pattern):
+    the per-node blast radius (1-hop caller/callee names) and source slices come straight from
+    the store instead of one subprocess per node — the prior loop's latency dominator. All
+    queries are reads; every write still goes through the estate CLI client (single surface)."""
+
+    def __init__(self, db: str):
+        import sqlite3
+        self._c = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+        self._c.row_factory = sqlite3.Row
+        # symbols: sym string ↔ sid
+        self.sid_of = {}
+        self.sym_of = {}
+        for r in self._c.execute("SELECT sid, sym FROM symbols"):
+            self.sid_of[r["sym"]] = r["sid"]
+            self.sym_of[r["sid"]] = r["sym"]
+        # node names + files + spans (span parsed lazily from data JSON only when sourcing)
+        self.node_row = {}
+        self._req = {}
+        try:
+            rows = self._c.execute("SELECT symbol, name, file, data, requirement FROM nodes")
+        except sqlite3.OperationalError:  # older stores predate the requirement column
+            rows = self._c.execute("SELECT symbol, name, file, data, NULL FROM nodes")
+        for r in rows:
+            self.node_row[r["symbol"]] = (r["name"], r["file"], r["data"])
+            self._req[r["symbol"]] = r[4] or ""
+        # calls adjacency (edge kinds are stored JSON-quoted, e.g. '"calls"')
+        self.callees = {}
+        self.callers = {}
+        self.in_deg = {}
+        for r in self._c.execute("SELECT source, target FROM edges WHERE kind IN ('calls', '\"calls\"')"):
+            self.callees.setdefault(r["source"], []).append(r["target"])
+            self.callers.setdefault(r["target"], []).append(r["source"])
+            self.in_deg[r["target"]] = self.in_deg.get(r["target"], 0) + 1
+        self._file_text = {}
+
+    def requirement_of(self, symbol_id: str) -> str:
+        sid = self.sid_of.get(symbol_id)
+        return self._req.get(sid, "") if sid is not None else ""
+
+    def name_of_sid(self, sid: int) -> str:
+        row = self.node_row.get(sid)
+        return row[0] if row else self.sym_of.get(sid, "?")
+
+    def blast_neighbors(self, symbol_id: str, cap: int = 8) -> list[str]:
+        """1-hop callers + callees as 'role:name' strings, importance-ranked (callers first)."""
+        sid = self.sid_of.get(symbol_id)
+        if sid is None:
+            return []
+        out = []
+        for other in sorted(self.callers.get(sid, []), key=lambda x: -self.in_deg.get(x, 0))[: cap // 2]:
+            out.append(f"caller:{self.name_of_sid(other)}")
+        for other in sorted(self.callees.get(sid, []), key=lambda x: -self.in_deg.get(x, 0))[: cap - len(out)]:
+            out.append(f"callee:{self.name_of_sid(other)}")
+        return out
+
+    def leverage(self, symbol_id: str) -> int:
+        sid = self.sid_of.get(symbol_id)
+        if sid is None:
+            return 0
+        return self.in_deg.get(sid, 0) + len(self.callees.get(sid, []))
+
+    def source_slice(self, symbol_id: str, cap: int = 4000) -> str:
+        sid = self.sid_of.get(symbol_id)
+        row = self.node_row.get(sid) if sid is not None else None
+        if row is None:
+            return ""
+        _, file, data = row
+        try:
+            import json as _json
+            span = _json.loads(data).get("location", {}).get("span", {})
+            start, end = int(span.get("start_byte", 0)), int(span.get("end_byte", 0))
+            if file not in self._file_text:
+                r = self._c.execute(
+                    "SELECT c.blob FROM files f JOIN content c ON c.git_sha = f.git_sha WHERE f.path = ?",
+                    (file,),
+                ).fetchone()
+                if len(self._file_text) >= 128:  # bounded FIFO — long runs must not grow without limit
+                    self._file_text.pop(next(iter(self._file_text)))
+                self._file_text[file] = _maybe_decompress(r["blob"]) if r else b""
+            blob = self._file_text[file]
+            if end > start and end <= len(blob):
+                return blob[start:end].decode("utf-8", errors="replace")[:cap]
+        except Exception:
+            pass
+        return ""
+
+
+def _density_batch(members: list[dict], store, base_batch: int) -> int:
+    """Density-adaptive batch size (planner idea, deterministic form): thin wrappers pack
+    bigger batches, dense logic packs smaller — clamped 4..64."""
+    if not members:
+        return base_batch
+    sample = members[: min(len(members), 12)]
+    avg = sum(len(store.source_slice(m["symbol_id"], cap=2000)) for m in sample) / len(sample)
+    mult = 1.6 if avg < 300 else (0.6 if avg > 1500 else 1.0)
+    return max(4, min(64, int(base_batch * mult)))
+
+
 def run(db: str, *, time_budget: float, limit: int, batch: int, dry_run: bool,
-        project_dir: Path | None = None) -> int:
+        project_dir: Path | None = None, cluster_offset: int = 0) -> int:
     if not db:
         raise RuntimeError("--db / $WICKED_ESTATE_DB is required but was not provided")
     estate = _clients.estate_client(db=db, project_dir=project_dir)
@@ -127,6 +283,15 @@ def run(db: str, *, time_budget: float, limit: int, batch: int, dry_run: bool,
         print(f"[extract-loop] cohesion framing unavailable ({e}); proceeding singleton-flat", file=sys.stderr)
         node_community, community_sizes = {}, {}
 
+    # STRUCTURAL SUBSTRATE (CommandIQ req-engine v3 pattern): direct read-only store access
+    # for blast-radius context + source slices — no per-node subprocess (the prior loop's
+    # latency dominator). Best-effort: any failure falls back to the flat, context-free path.
+    try:
+        store = _DirectStore(db)
+    except Exception as e:
+        print(f"[extract-loop] direct store unavailable ({e}); flat context-free fallback", file=sys.stderr)
+        store = None
+
     processed = 0
     while True:
         cov = core.coverage(db, cov_out)
@@ -141,21 +306,45 @@ def run(db: str, *, time_budget: float, limit: int, batch: int, dry_run: bool,
                   "resume with another pass", file=sys.stderr)
             return 0
 
-        # Take a bounded slice of the worklist for this batch.
-        take = worklist[:batch]
+        # STRUCTURAL ORDER: group the worklist by community, rank clusters by aggregate call
+        # leverage (in+out degree) so high-impact groups extract first, members likewise ranked
+        # within their cluster. Batches then hold RELATED symbols — shared context makes the
+        # model's rules better grounded AND cheaper — with a density-adaptive batch size
+        # (thin wrappers pack bigger batches, dense logic smaller). Flat slice as fallback.
+        if store is not None and node_community:
+            clusters_seen: dict[str, list] = {}
+            for n in worklist:
+                clusters_seen.setdefault(node_community.get(n["symbol_id"], "~singleton"), []).append(n)
+            def _cluster_leverage(members: list) -> int:
+                return sum(store.leverage(m["symbol_id"]) for m in members[:50])
+            # cluster_offset lets parallel workers each own a different leverage-ranked
+            # cluster instead of all fighting over the head (offset 0 = head).
+            ranked = sorted(clusters_seen.values(), key=_cluster_leverage, reverse=True)
+            head = ranked[max(0, cluster_offset) % len(ranked)]
+            head.sort(key=lambda n: store.leverage(n["symbol_id"]), reverse=True)
+            take = head[: _density_batch(head, store, batch)]
+        else:
+            take = worklist[:batch]
         if limit:
             take = take[:max(0, limit - processed)]
 
-        # Frame each node: source slice + cohesion neighbors (best-effort).
+        # Frame each node: source slice + blast-radius neighbors (direct reads; the old
+        # per-node subprocess path survives only as the fallback).
         framed, ids = [], set()
         for n in take:
             sid, name = n["symbol_id"], n.get("name", "")
-            try:
-                slice_txt = estate.source(sid)[:4000] if not dry_run else ""
-            except Exception:
-                slice_txt = ""
+            slice_txt, neighbors = "", []
+            if not dry_run:
+                if store is not None:
+                    slice_txt = store.source_slice(sid)
+                    neighbors = store.blast_neighbors(sid)
+                if slice_txt == "":
+                    try:
+                        slice_txt = estate.source(sid)[:4000]
+                    except Exception:
+                        slice_txt = ""
             framed.append(_rule_extractor.frame_context(n, slice_txt,
-                          cluster_label=node_community.get(sid), neighbor_names=[]))
+                          cluster_label=node_community.get(sid), neighbor_names=neighbors))
             ids.add(sid)
 
         # THE MODEL BOUNDARY (or the deterministic stub). A model failure over the
@@ -164,12 +353,64 @@ def run(db: str, *, time_budget: float, limit: int, batch: int, dry_run: bool,
         if dry_run:
             by_id = {n["symbol_id"]: _stub_rule(n) for n in take}
         else:
-            try:
-                for r in _rule_extractor.extract_rules(framed, model_argv):
-                    if isinstance(r, dict) and r.get("symbol_id") in ids:
-                        by_id[r["symbol_id"]] = r
-            except Exception as e:
-                print(f"[extract-loop] model batch failed ({e}); RISK-flooring the batch", file=sys.stderr)
+            # Providers blip: an occasional empty response or transient error is normal,
+            # a REPEAT on the same batch is an outage. One in-pass retry (after a short
+            # pause) before EX_TEMPFAIL keeps transients from costing a whole pass.
+            for attempt in (1, 2):
+                by_id = {}
+                floor_batch = False
+                infra_exc = None
+                # Models normalize punctuation-bearing ids (a trailing '.' in the symbol
+                # grammar came back stripped, dropping EVERY rule of certain batches as a
+                # "mismatch" and mimicking an outage). Exact match first; else re-canonize
+                # by trailing-dot-insensitive lookup. Anything else stays a hallucination.
+                canon = {sid.rstrip("."): sid for sid in ids}
+                try:
+                    for r in _rule_extractor.extract_rules(framed, model_argv):
+                        if not isinstance(r, dict):
+                            continue
+                        rid = r.get("symbol_id")
+                        if rid in ids:
+                            by_id[rid] = r
+                        elif isinstance(rid, str) and rid.strip().rstrip(".") in canon:
+                            r["symbol_id"] = canon[rid.strip().rstrip(".")]
+                            by_id[r["symbol_id"]] = r
+                except Exception as e:
+                    if _is_infra_failure(e):
+                        infra_exc = e
+                    elif "exceeded" in str(e) and attempt == 1:
+                        # A batch timeout is often provider slowness, not batch difficulty
+                        # — one retry before conceding. A repeat floors (termination), it
+                        # does not abort: a genuinely oversized batch must not wedge passes.
+                        print(f"[extract-loop] model batch timeout ({e}) — retrying once "
+                              f"in {int(_RETRY_PAUSE)}s", file=sys.stderr)
+                        time.sleep(_RETRY_PAUSE)
+                        continue
+                    else:
+                        print(f"[extract-loop] model batch failed ({e}); RISK-flooring the batch",
+                              file=sys.stderr)
+                        floor_batch = True
+                if floor_batch or by_id:
+                    break
+                if infra_exc is None and len(take) < 4:
+                    # Tiny batch, no objects, no classified error: could be one genuinely
+                    # unstatable node — floor it rather than wedge the loop.
+                    break
+                if attempt == 2:
+                    # An outage is not a judgment. Flooring here would burn the whole
+                    # remaining worklist into placeholders — stop the pass instead;
+                    # nothing from this batch is written, resume re-derives it. NO
+                    # parseable rule objects for a whole real batch is the same outage
+                    # signature (notice printed with exit 0, or a wholesale empty array);
+                    # per-unit objects with explicit empty statements are a JUDGMENT and
+                    # floor as designed.
+                    why = f"infra: {infra_exc}" if infra_exc else f"zero-yield ({len(take)} framed, 0 rule objects)"
+                    print(f"[extract-loop] {why} — persisted after retry; pass aborted, batch NOT floored",
+                          file=sys.stderr)
+                    return 75  # EX_TEMPFAIL
+                print("[extract-loop] transient zero-yield/infra on batch — retrying once "
+                      f"in {int(_RETRY_PAUSE)}s", file=sys.stderr)
+                time.sleep(_RETRY_PAUSE)
 
         # Deterministic decision + write per node — RISK-FLOOR: every node terminates.
         for n in take:
@@ -182,6 +423,14 @@ def run(db: str, *, time_budget: float, limit: int, batch: int, dry_run: bool,
                 resolved = adjusted >= RESOLVE_THRESHOLD
                 _write_node(estate, sid, name, rule, resolved,
                             "below confidence threshold" if not resolved else "")
+            elif (rule or {}).get("statement"):
+                _write_node(estate, sid, name, rule, False, reason)
+            elif _has_statement(store, sid):
+                # Statement-less floor over existing content would DEGRADE the store —
+                # a re-queued node keeps its prior statement instead. Floors only ever
+                # fill blanks; content is monotonic.
+                print(f"[extract-loop] floor skipped for {sid} (existing statement kept)",
+                      file=sys.stderr)
             else:
                 _write_node(estate, sid, name, rule, False, reason)
             processed += 1
@@ -198,11 +447,13 @@ def main(argv: list[str] | None = None) -> int:
     ap.add_argument("--limit", type=int, default=0, help="max nodes this pass (0 = unbounded within time budget)")
     ap.add_argument("--batch", type=int, default=12, help="framed nodes per model call")
     ap.add_argument("--dry-run", action="store_true", help="deterministic stub instead of the model (zero cost)")
+    ap.add_argument("--cluster-offset", type=int, default=0,
+                    help="parallel workers: take the Nth leverage-ranked cluster instead of the head")
     args = ap.parse_args(argv)
     if not args.db:
         ap.error("--db is required (or set $WICKED_ESTATE_DB)")
     return run(args.db, time_budget=args.time_budget, limit=args.limit,
-               batch=args.batch, dry_run=args.dry_run)
+               batch=args.batch, dry_run=args.dry_run, cluster_offset=args.cluster_offset)
 
 
 if __name__ == "__main__":

package/scripts/domain/rule_model_api.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Direct-API drop-in for the rule-extraction model boundary.
+
+The harness talks to its model via an argv that reads the prompt on stdin and
+prints the response on stdout (`claude -p` shape). Each CLI invocation pays
+full agent bootstrap for what is a single completion; this adapter makes the
+same one-shot call straight against the Anthropic API — no bootstrap, explicit
+prompt caching, stdlib-only (urllib, no SDK).
+
+Usage:
+    export ANTHROPIC_API_KEY=...              # required
+    export WICKED_RULE_MODEL_BIN="python3 /path/to/rule_model_api.py"
+    # optional: WICKED_RULE_MODEL_API_MODEL (default claude-haiku-4-5-20251001)
+
+The prompt's static instruction header (everything before the first
+"--- UNIT 1 ---") is sent as a system block with cache_control so every call
+shares the cached prefix; only the framed units vary.
+"""
+from __future__ import annotations
+
+import json
+import os
+import sys
+import urllib.error
+import urllib.request
+
+_API_URL = "https://api.anthropic.com/v1/messages"
+_UNIT_BOUNDARY = "\n--- UNIT 1 ---\n"
+
+
+def main() -> int:
+    key = os.environ.get("ANTHROPIC_API_KEY", "").strip()
+    if not key:
+        print("rule_model_api: ANTHROPIC_API_KEY is not set", file=sys.stderr)
+        return 78  # EX_CONFIG
+    model = os.environ.get("WICKED_RULE_MODEL_API_MODEL", "claude-haiku-4-5-20251001")
+    prompt = sys.stdin.read()
+
+    cut = prompt.find(_UNIT_BOUNDARY)
+    if cut > 0:
+        system_part, user_part = prompt[:cut], prompt[cut:]
+    else:  # unexpected shape — send everything as the user turn, no cached prefix
+        system_part, user_part = "", prompt
+
+    body: dict = {
+        "model": model,
+        "max_tokens": 8192,
+        "messages": [{"role": "user", "content": user_part}],
+    }
+    if system_part:
+        body["system"] = [{"type": "text", "text": system_part,
+                           "cache_control": {"type": "ephemeral"}}]
+
+    req = urllib.request.Request(
+        _API_URL,
+        data=json.dumps(body).encode("utf-8"),
+        headers={
+            "content-type": "application/json",
+            "x-api-key": key,
+            "anthropic-version": "2023-06-01",
+        },
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=170) as resp:
+            payload = json.loads(resp.read().decode("utf-8"))
+    except urllib.error.HTTPError as e:
+        detail = e.read().decode("utf-8", errors="replace")[:300]
+        # Mirror the CLI failure shape the harness classifies on: rate/overloaded/
+        # auth markers in the message route to the infra abort, not the floor.
+        print(f"rule_model_api: HTTP {e.code} — {detail}", file=sys.stderr)
+        return 1
+    except (urllib.error.URLError, TimeoutError) as e:
+        print(f"rule_model_api: network failure — {e}", file=sys.stderr)
+        return 1
+
+    text = "".join(b.get("text", "") for b in payload.get("content", [])
+                   if isinstance(b, dict) and b.get("type") == "text")
+    if not text.strip():
+        print(f"rule_model_api: empty completion (stop_reason={payload.get('stop_reason')!r})",
+              file=sys.stderr)
+        return 1
+    sys.stdout.write(text)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())