@tyroneross/build-loop 0.30.3

package/docs/memory-setup.md

@@ -0,0 +1,226 @@
+# Memory Setup
+
+Build-loop's advisory judges and the Phase 1 Assess memory-load step read from
+one consolidated tree under `~/dev/git-folder/build-loop-memory/` by default:
+
+| Tier | Location | Owner | Versioning |
+|---|---|---|---|
+| Global | `~/dev/git-folder/build-loop-memory/` plus top-level lanes such as `lessons/` | This user | **Should be in a private git repo** (your lessons live here) |
+| Project | `~/dev/git-folder/build-loop-memory/projects/<slug>/` (slug derived via `scripts/_paths.derive_slug_from_cwd` — basename of the git repo root, lowercased + normalized; `workers/` sub-component becomes `<slug>/workers`) | This user | Same private repo as global |
+
+> **History** — until PR 3 of the memory-consolidation series (merged 2026-05-13), the legacy per-repo location was also read by `memory_facade._resolve_memory_dirs` as a transitional shim. As of PR 3, only the consolidated tree is read; any pre-migration content still at the legacy path is invisible. Operators with such content should run `scripts/migrate_project_memory.py --apply` (idempotent), then `scripts/cleanup_legacy_memory_stubs.py --apply` to remove the now-inert `.MOVED.md` stubs.
+
+The build-loop public repo ships only the **scaffolding** — templates, a public
+seed manifest, and the setup script. Your actual lessons, constitution rules,
+patterns, project plans, raw artifacts, generated indexes, and decisions belong
+in a private repo because they contain references to specific projects and
+operator preferences that are not appropriate for public distribution.
+
+## Quick start
+
+```bash
+# Guided terminal install. This validates the packaged public seed first, then
+# creates ~/dev/git-folder/build-loop-memory/ with scaffold-only files.
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/install_memory.py --guided
+
+# Bootstrap with templates (creates ~/dev/git-folder/build-loop-memory/ if missing,
+# seeds constitution.md + MEMORY.md from templates/memory/, plus indexes/ and projects/)
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/install_memory.py
+
+# Check status anytime
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/install_memory.py --check
+
+# Validate the shipped public seed without writing
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/install_memory.py --validate-seed
+```
+
+This creates:
+
+```
+~/dev/git-folder/build-loop-memory/
+├── constitution.md     # template — replace with your invariants
+├── MEMORY.md           # template — index for entries you add
+├── indexes/            # rebuildable local indexes, including semantic_facts.sqlite
+│   └── updates.jsonl   # created on first memory mutation; global update ledger
+└── projects/           # project-scoped memory trees
+```
+
+## Packaged public seed
+
+The install package includes `templates/memory/manifest.json`, which is the
+contract for the public `build-loop-memory` seed. It allowlists only generic
+template files and generated empty directories:
+
+- `constitution.md.template` -> `constitution.md`
+- `MEMORY.md.template` -> `MEMORY.md`
+- `indexes/`
+- `projects/README.md`
+- empty `projects/<slug>/raw/.../.gitkeep` lanes when `--ensure-project` is used
+
+The seed validator refuses unlisted template files and scans the shipped
+templates for common secret, personal-path, private-repo, and operator-identity
+patterns. This protects the install package from accidentally shipping a
+developer's private `build-loop-memory` contents.
+
+## Linking to a private repo
+
+You have a few options for versioning the global memory. The recommended one is **clone a private repo into the memory dir**:
+
+### Option A — Clone a private repo at install time
+
+```bash
+# Empty / non-existent build-loop-memory directory required
+python3 scripts/install_memory.py --link-repo git@github.com:<you>/<your-memory-repo>.git
+```
+
+The repo should contain `constitution.md` and `MEMORY.md` plus your existing lessons. The install script will clone it directly; no templates are seeded (the repo provides content).
+
+### Option B — Bootstrap with templates, then init a repo
+
+```bash
+python3 scripts/install_memory.py
+cd ~/dev/git-folder/build-loop-memory
+git init
+git add constitution.md MEMORY.md
+git commit -m "init: build-loop memory scaffolding"
+git remote add origin git@github.com:<you>/<your-memory-repo>.git
+git push -u origin main
+```
+
+### Option C — Override the canonical root
+
+If your private memory repo lives somewhere else, set
+`BUILD_LOOP_MEMORY_STORE_ROOT` or pass `--dest <path>` to the setup script. The
+scripts resolve the root through `scripts/_paths.py`.
+
+### Option D — Use a repo-local ignored store
+
+For experiments or repos that should carry their own local memory without a
+separate private repo, set the root to an ignored path such as:
+
+```bash
+export BUILD_LOOP_MEMORY_STORE_ROOT="$PWD/.build-loop/memory"
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/install_memory.py --dest "$BUILD_LOOP_MEMORY_STORE_ROOT"
+```
+
+This is supported because build-loop owns the writer scripts and schema, while
+the memory root is just data. Keep `.build-loop/` ignored if the contents should
+not be committed.
+
+## What goes in the global store
+
+Add entries over time as you accumulate lessons:
+
+| Type | Filename pattern | When to add |
+|---|---|---|
+| `feedback` | `feedback_<slug>.md` | After a user correction OR a validated success worth remembering |
+| `pattern` | `pattern_<slug>.md` | Recurring solution worth naming |
+| `reference` | `reference_<slug>.md` | Durable facts about external systems / topologies |
+| `decision` | `decision_<slug>.md` | One-shot architectural decisions |
+| `user` | `user_<slug>.md` | User-profile facts that shape collaboration style |
+
+Each file uses YAML frontmatter:
+
+```markdown
+---
+name: <kebab-case-slug>
+description: <one-sentence summary for relevance scoring>
+metadata:
+  type: feedback | pattern | reference | decision | user
+---
+
+# Title
+
+Body content. Link related entries with [[other-slug]].
+
+For feedback / pattern entries, structure as:
+- The rule or fact
+- **Why:** the rationale
+- **How to apply:** when/where the rule fires
+```
+
+`MEMORY.md` is the index. Each entry there is one line:
+
+```markdown
+- [Title](filename.md) — one-line hook for relevance matching
+```
+
+Keep `MEMORY.md` under ~200 lines; entries past that get truncated when loaded into orchestrator context.
+
+## What about decisions?
+
+The **canonical decision store** is
+`~/dev/git-folder/build-loop-memory/projects/<project>/decisions/`. It is
+project-tagged and written by `scripts/write_decision.py` — it captures the
+discrete "we decided X" events of any project.
+
+The top-level `lessons/` lane is for **cross-project lessons** — patterns and
+feedback that apply broadly, not project-specific decisions.
+
+```
+~/dev/git-folder/build-loop-memory/                  # canonical root
+~/dev/git-folder/build-loop-memory/lessons/          # cross-project lessons
+~/dev/git-folder/build-loop-memory/indexes/updates.jsonl  # global update ledger
+~/dev/git-folder/build-loop-memory/projects/<slug>/  # project-local memory
+~/dev/git-folder/build-loop-memory/projects/_archive/<slug>/  # retired projects, still queryable
+<repo>/.build-loop/memory/                           # legacy project location — no longer read (PR 3 removed the shim); migrate via scripts/migrate_project_memory.py
+<repo>/.episodic/decisions/                          # legacy local decision store (migration/archive)
+```
+
+## Global update ledger
+
+Build-loop records every canonical memory mutation in
+`<memory-root>/indexes/updates.jsonl`. The ledger is append-only JSONL and is
+written by the plugin scripts:
+
+- `scripts/memory_writer.py` for lessons and provenance updates
+- `scripts/write_decision.py` for project decisions and supersession
+- `scripts/append_milestone.py` for run-level milestones
+
+Minimum useful fields are `ts`, `project`, `lane`, `action`, `path`, `writer`,
+and `source_commit` when the update is tied to a repo HEAD. Freshness checks use
+the latest project ledger row with `source_commit`; if no row exists, they fall
+back to `projects/<slug>/milestones.jsonl`.
+
+Read recent updates:
+
+```bash
+python3 scripts/memory_update_ledger.py tail --project build-loop --limit 20
+```
+
+## Constitution rules
+
+`constitution.md` is the highest-priority store — loaded at every Phase 1 Assess and cited by advisory judges. Keep it concise:
+
+- **Stable rule IDs** (`C-SEC/no_secret_in_repo`, `C-AUTH/owner_only_writes`) — never renumber
+- **One sentence per rule** — detail goes in linked memory entries
+- **Under 200 lines total** — judges re-read this on every Phase 3 checkpoint
+
+Edit the template that the install script seeded — the prefilled rules are reasonable defaults but your invariants may differ.
+
+## Verifying setup
+
+```bash
+# Check status
+python3 scripts/install_memory.py --check
+
+# Test memory recall (will exit 0 even if backends are down)
+python3 scripts/memory_facade.py recall --query "test" --limit 3
+
+# Backend health probe
+python3 scripts/backend_health.py
+```
+
+If any of these fail, see `references/memory-systems.md` for the full backend topology and graceful-degradation contract.
+
+## Privacy
+
+Treat `~/dev/git-folder/build-loop-memory/` as containing potentially-sensitive context:
+
+- Operator preferences, project-specific decisions, client names if you write them in
+- Constitution rules may reveal architectural patterns from past projects
+- Feedback entries often quote real exchanges
+
+The recommendation is a **private git repo** (`Option A` or `B` above), not the
+public build-loop repo. Build-loop ships only the manifest-backed public seed
+and empty scaffolding.

package/hooks/_arch_scan_bg.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com>
+# SPDX-License-Identifier: Apache-2.0
+"""Background scan worker — invoked by both hook scripts.
+
+Acquires a non-blocking ``fcntl.flock`` on
+``.build-loop/architecture/.scan.lock`` and runs scan + acp + mark-fresh.
+
+Exits silently if the lock is held; never raises.
+
+Intended to be called as::
+
+    nohup python3 hooks/_arch_scan_bg.py --workdir <path> </dev/null \\
+        >/dev/null 2>&1 &
+
+Why a separate file: putting the Python payload in a heredoc inside a shell
+hook conflicts with the `</dev/null` redirection used to fully detach the
+backgrounded process — the second stdin redirection wins, and the heredoc is
+never delivered to the interpreter.
+"""
+
+from __future__ import annotations
+
+import argparse
+import fcntl
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+
+def main(argv: list | None = None) -> int:
+    p = argparse.ArgumentParser()
+    p.add_argument("--workdir", required=True)
+    args = p.parse_args(argv)
+
+    workdir = Path(args.workdir).resolve()
+    lockpath = workdir / ".build-loop" / "architecture" / ".scan.lock"
+    script = workdir / "scripts" / "architecture_freshness.py"
+
+    try:
+        lockpath.parent.mkdir(parents=True, exist_ok=True)
+        lf = open(lockpath, "a+")
+        fcntl.flock(lf.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
+    except (BlockingIOError, OSError):
+        return 0  # another scan is in flight; bail silently
+
+    try:
+        env = os.environ.copy()
+        for cmd in (
+            ["uv", "run", "python", "-m", "build_loop.architecture", "scan", "--incremental"],
+            ["uv", "run", "python", "-m", "build_loop.architecture", "acp"],
+        ):
+            try:
+                subprocess.run(
+                    cmd,
+                    cwd=str(workdir),
+                    stdout=subprocess.DEVNULL,
+                    stderr=subprocess.DEVNULL,
+                    env=env,
+                    check=False,
+                    timeout=120,
+                )
+            except (FileNotFoundError, subprocess.TimeoutExpired):
+                pass
+
+        if script.exists():
+            try:
+                subprocess.run(
+                    [sys.executable, str(script), "--mark-fresh", "--workdir", str(workdir)],
+                    stdout=subprocess.DEVNULL,
+                    stderr=subprocess.DEVNULL,
+                    check=False,
+                    timeout=10,
+                )
+            except (FileNotFoundError, subprocess.TimeoutExpired):
+                pass
+    finally:
+        try:
+            fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
+            lf.close()
+        except Exception:
+            pass
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/hooks/_plugin_drift_check_bg.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com>
+# SPDX-License-Identifier: Apache-2.0
+"""Background plugin-cache drift detector. Compares cached plugin install
+SHA against the source repo's main HEAD; writes markers under
+~/.claude/plugins/.drift-markers/ for Phase 1 Assess to surface.
+Fire-and-forget; fcntl single-flight; per-plugin 24h freshness gate."""
+from __future__ import annotations
+import argparse, fcntl, json, subprocess, time
+from pathlib import Path
+
+
+def _j(p):
+    try: return json.loads(Path(p).read_text())
+    except Exception: return None
+
+def _url(s):
+    if not isinstance(s, dict): return None
+    if s.get("source") == "github" and s.get("repo"): return f"https://github.com/{s['repo']}"
+    if s.get("source") == "git" and s.get("url"): return s["url"]
+    return None
+
+def _sha(url, t=8):
+    try:
+        r = subprocess.run(["git","ls-remote",url,"refs/heads/main"],
+                           capture_output=True, text=True, timeout=t, check=False)
+        if r.returncode == 0 and r.stdout.strip(): return r.stdout.split()[0]
+    except (FileNotFoundError, subprocess.TimeoutExpired): pass
+    return None
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--workdir", default=str(Path.home() / ".claude" / "plugins"))
+    ap.add_argument("--force", action="store_true")
+    a = ap.parse_args(argv)
+    root = Path(a.workdir).resolve()
+    plugins = (_j(root/"installed_plugins.json") or {}).get("plugins", {}) or {}
+    markets = _j(root/"known_marketplaces.json") or {}
+    if not plugins: return 0
+
+    md = root/".drift-markers"; md.mkdir(parents=True, exist_ok=True)
+    try:
+        lf = open(md/".check.lock", "a+"); fcntl.flock(lf.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
+    except (BlockingIOError, OSError): return 0
+    try:
+        now = time.time()
+        for key, entries in plugins.items():
+            if not entries or "@" not in key: continue
+            name, mkt = key.split("@", 1)
+            loc = (markets.get(mkt) or {}).get("installLocation")
+            src = None
+            if loc:
+                for pl in (_j(Path(loc)/".claude-plugin"/"marketplace.json") or {}).get("plugins", []) or []:
+                    if pl.get("name") == name: src = _url(pl.get("source") or {}); break
+            if not src: continue
+            ip = Path(entries[0].get("installPath") or "")
+            if not ip.exists(): continue
+            pj = ip/".claude-plugin"/"plugin.json"
+            if not pj.exists(): pj = ip/"plugin.json"
+            cv = (_j(pj) or {}).get("version")
+            csha = entries[0].get("version")
+            mp = md/f"{name}__{mkt}.json"
+            if not a.force and mp.exists():
+                try:
+                    if now - mp.stat().st_mtime < 86400: continue
+                except OSError: pass
+            msha = _sha(src)
+            if not msha: continue
+            drifted = bool(csha) and not msha.startswith(str(csha))
+            if drifted:
+                mp.write_text(json.dumps({"plugin": name, "marketplace": mkt, "source": src,
+                    "cached_version": cv, "cached_sha": csha, "main_sha": msha,
+                    "commits_behind": None, "drifted": True, "last_checked": int(now)}, indent=2))
+            elif mp.exists():
+                try: mp.unlink()
+                except OSError: pass
+    finally:
+        try: fcntl.flock(lf.fileno(), fcntl.LOCK_UN); lf.close()
+        except Exception: pass
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/hooks/_session_start_lib.sh

@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: Apache-2.0
+# Shared SessionStart helpers. Source, do not exec.
+# Contract for all SessionStart hooks (see feedback_hook_design.md):
+# command-type only, exit 0 always, silent on success, fire-and-forget,
+# <100ms return via nohup + disown.
+
+# Background-fire a python worker with stdin/stdout/stderr fully detached.
+bl_fire_bg() {
+  nohup python3 "$@" </dev/null >/dev/null 2>&1 &
+  disown 2>/dev/null || true
+}
+
+# Echo "yes" if $1 is missing or older than $2 seconds; "no" otherwise.
+bl_stale() {
+  F="$1" MAX="$2" python3 - <<'PY' 2>/dev/null
+import os, time
+from pathlib import Path
+f = Path(os.environ["F"])
+m = float(os.environ["MAX"])
+if not f.exists():
+    print("yes")
+else:
+    try:
+        print("yes" if time.time() - f.stat().st_mtime > m else "no")
+    except OSError:
+        print("yes")
+PY
+}
+
+# Spawn a long-running daemon once; PID-file + /health probe single-flight.
+# Args: script_path port pidfile_basename logfile_basename log_dir
+bl_spawn_daemon() {
+  local script="$1" port="$2" pidfile="$5/$3.pid" log="$5/$4.log"
+  [ -f "$script" ] || return 0
+  if [ -f "$pidfile" ]; then
+    local pid; pid=$(cat "$pidfile" 2>/dev/null)
+    [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null && return 0
+  fi
+  curl -fsS --max-time 0.2 "http://127.0.0.1:${port}/health" >/dev/null 2>&1 && return 0
+  nohup python3 "$script" </dev/null >>"$log" 2>&1 &
+  disown 2>/dev/null || true
+}

package/hooks/git/pre-push

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com>
+# SPDX-License-Identifier: Apache-2.0
+"""build-loop pre-push hook — path-agnostic push-HOLD gate.
+
+INSTALLED VIA: ``scripts/install_git_hooks.py --install`` writes this file
+verbatim to ``.git/hooks/pre-push`` (chmod +x).  The committed source lives at
+``hooks/git/pre-push`` so it's distributable / reviewable.
+
+WHY THIS LIVES AT THE GIT LAYER
+-------------------------------
+``scripts/deployment_policy.py`` is an *app-layer* policy — anything that
+doesn't consult it (a launchd job, a Codex shell, a stray ``git push`` from a
+crashed automation) bypasses it.  The git pre-push hook is the only place that
+fires for EVERY push regardless of who initiated it.
+
+CONTRACT
+--------
+Per ``man githooks``: pre-push receives ``<remote> <url>`` as argv, and four
+fields per ref on stdin: ``<local_ref> <local_sha> <remote_ref> <remote_sha>``.
+Exit non-zero → push is blocked.
+
+This hook:
+
+1. Locates the repo (git rev-parse) and adds ``<repo>/scripts`` to sys.path.
+2. Imports ``push_hold.evaluate_push``.
+3. Calls it with the stdin lines.
+4. On ``allow``/``bypass``: exit 0 (push proceeds).
+5. On ``block``: print the reason + release instructions to stderr, exit 1.
+6. On ANY internal exception (broken import, syntax error, OSError): log to
+   stderr and exit 0.  A broken hook MUST NOT permanently wedge the user's
+   ability to push — fail OPEN on internal errors, fail CLOSED on holds.
+
+The block-vs-fail-open distinction is the whole reason for the try/except
+wrapper below.  Do NOT collapse it into a bare ``raise``.
+"""
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+
+def _repo_root() -> Path:
+    """Resolve the repo root via ``git rev-parse --show-toplevel``."""
+    try:
+        out = subprocess.check_output(
+            ["git", "rev-parse", "--show-toplevel"],
+            stderr=subprocess.DEVNULL,
+            text=True,
+        ).strip()
+        if out:
+            return Path(out)
+    except (subprocess.CalledProcessError, FileNotFoundError, OSError):
+        pass
+    # Fallback: walk up from the hook's location.  In a normal install the
+    # hook lives at ``<repo>/.git/hooks/pre-push``; the hook's __file__ is
+    # often a real path but may be a symlink target.  Walk up looking for
+    # ``.git``.
+    here = Path(__file__).resolve()
+    for parent in [here] + list(here.parents):
+        if (parent / ".git").exists():
+            return parent
+    return Path.cwd()
+
+
+def _fail_open(reason: str) -> int:
+    """Internal-error path: print to stderr, return 0 (allow push)."""
+    sys.stderr.write(
+        f"[build-loop pre-push] internal error — allowing push: {reason}\n"
+    )
+    return 0
+
+
+def _format_block_message(verdict: dict) -> str:
+    targets = ",".join(verdict.get("protected_targets") or []) or "(unknown)"
+    reason = verdict.get("reason") or "hold active"
+    source = verdict.get("source") or "unknown"
+
+    # Corrupt-marker note: help the operator self-rescue without digging into docs.
+    corrupt_note = ""
+    if source == "marker" and ("unparseable" in reason or "malformed" in reason or "not a JSON" in reason):
+        corrupt_note = (
+            "  NOTE: the hold marker is corrupt (not valid JSON).\n"
+            "        Run `push_hold.py --release` to clear it (works even\n"
+            "        on a corrupt marker, no prior --set needed).\n"
+            "  ---------------------------------------------------------------\n"
+        )
+
+    return (
+        "\n"
+        "===============================================================\n"
+        "  BUILD-LOOP PUSH HOLD — push BLOCKED\n"
+        "===============================================================\n"
+        f"  protected target(s): {targets}\n"
+        f"  hold source        : {source}\n"
+        f"  reason             : {reason}\n"
+        "  ---------------------------------------------------------------\n"
+        + corrupt_note
+        + "  To release this hold:\n"
+        "    1. Resolve the underlying issue (auditor findings / brief).\n"
+        "    2. Run:   python3 scripts/push_hold.py --release \\\n"
+        '                       --reason "<one-line why>"\n'
+        "    3. Re-push.\n"
+        "\n"
+        "  EMERGENCY override (logged):\n"
+        "    BUILDLOOP_PUSH_HOLD_BYPASS=1 git push <remote> <branch>\n"
+        "===============================================================\n"
+    )
+
+
+def main() -> int:
+    try:
+        repo = _repo_root()
+        scripts_dir = repo / "scripts"
+        if str(scripts_dir) not in sys.path:
+            sys.path.insert(0, str(scripts_dir))
+        try:
+            import push_hold  # type: ignore
+        except Exception as exc:  # pragma: no cover — exercised by installer test
+            return _fail_open(f"could not import push_hold: {exc!r}")
+
+        try:
+            stdin_lines = sys.stdin.readlines()
+        except OSError as exc:
+            return _fail_open(f"could not read stdin: {exc!r}")
+
+        verdict = push_hold.evaluate_push(repo, stdin_lines, env=os.environ)
+        action = verdict.get("action", "allow")
+        if action == "block":
+            sys.stderr.write(_format_block_message(verdict))
+            return int(verdict.get("exit_code") or 1)
+        # allow / bypass / anything else → exit 0
+        return 0
+    except Exception as exc:  # noqa: BLE001 — broad on purpose; we MUST fail-open
+        return _fail_open(f"unhandled exception: {exc!r}")
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())