@jenga-ai/agent 3.2.0 → 3.5.0

package/skills/j-uncharted/assets/NODE_QUESTION_TEMPLATE.md

@@ -0,0 +1,69 @@
+<!--
+  NODE QUESTION TEMPLATE — /uncharted Convergence Loop.
+
+  Added by E40_S06_T03. Consumed by the Convergence Loop's Step 5
+  ("Confirm/correct", skills/j-uncharted/SKILL.md) whenever risk-weighted
+  gating (Step 4) decides a finding needs a confirm prompt under
+  `verification_depth: shallow` or `moderate`. Never reached under `strict`
+  — Step 4's strict branch converges the node directly, without ever
+  reaching Step 5, so this template is simply not invoked in that case.
+
+  Two fixed variants, selected by node kind:
+
+    INTERNAL — the node represents the candidate/service itself (the thing
+      `/uncharted` is investigating).
+    EXTERNAL — the node represents something outside the candidate that the
+      traces surfaced: a dependency it calls, or a consumer that calls it.
+
+  This replaces a fully open-ended "propose understanding, ask the user to
+  confirm or correct it" prompt for any node that fits one of these two
+  kinds — which, per the Convergence Loop, is every node this flow drafts.
+  Wording is fixed and matches the brainstorm's agreed phrasing verbatim
+  (project/documentation/examples/uncharted-conversational-elicitation-procedure.md);
+  do not paraphrase it when presenting the prompt.
+
+  Usage: present the drafted node/edge summary first (per Step 3's draft),
+  then ask the questions below for the selected variant, then offer the
+  same confirm / correct-with-detail / defer-as-unconfirmed / other choice
+  Step 5 already documents. This file supplies the fixed *questions*; the
+  response mechanics (turn cap, converge call) are unchanged and live in
+  SKILL.md, not here.
+-->
+
+# Node Question Template
+
+## Internal Node
+
+Use when the node represents the candidate/service itself — the thing this investigation is about,
+not something external to it.
+
+1. What is this?
+2. What does it do?
+3. Who consumes it?
+4. Other (describe below)
+
+## External Node
+
+Use when the node represents a dependency or consumer the traces surfaced outside the candidate —
+something the service calls, or something that calls the service.
+
+1. What is this?
+2. What does it do?
+3. Is the service producing to it, or consuming from it?
+4. Who is the producer, and who is the consumer?
+5. Other (describe below)
+
+---
+
+After the selected question set, present the standard confirm/correct choice (per the Interaction
+Pattern in `CLAUDE.md` and Convergence Loop Step 5):
+
+```
+1. Confirm the draft as accurate
+2. Correct it — describe what's wrong
+3. Defer — mark this node "unconfirmed" for now
+4. Other (describe below)
+```
+
+Silence, a counter-question, or an ambiguous reply is not consent — re-ask, the same convention used
+at every other confirmation gate in this skill.

package/skills/j-uncharted/scripts/elicitation-state.sh

@@ -53,7 +53,23 @@
 #     "nodes": {
 #       "<node-id>": { "turns": <int>, "status": "pending"|"converged"|"flagged", "note": "<text>" }
 #     },
-#     "checkpoint": { ...arbitrary, agent-defined fields, e.g. directory-triage results... }
+#     "checkpoint": {
+#       ...arbitrary, agent-defined fields, e.g. directory-triage results...
+#       "verification_depth": {
+#         // Recognized field (E40_S06_T01). Per-candidate Familiarity Check
+#         // answer from skills/j-uncharted/SKILL.md's Convergence Loop Step 1,
+#         // keyed by candidate/node id so a resumed elicitation can tell which
+#         // candidates already answered. Values are "shallow" | "moderate" |
+#         // "strict". Written and read by the agent driving /uncharted, same
+#         // as every other checkpoint field — this script assigns it no
+#         // special handling beyond the dict-merge behavior documented under
+#         // `checkpoint` below (which exists so that checking in one
+#         // candidate's depth never clobbers another's already recorded
+#         // here). Consumed by E40_S06_T02's risk-weighted gating; unread by
+#         // anything in this script or E40_S06_T01's own scope.
+#         "<candidate-id>": "shallow" | "moderate" | "strict"
+#       }
+#     }
 #   }
 #
 # ---------------------------------------------------------------------------
@@ -79,11 +95,22 @@
 #       the node's note (e.g. a one-line summary of what was confirmed).
 #
 #   checkpoint --id ID --json FILE
-#       Shallow-merge the JSON object in FILE (or stdin when FILE is "-")
-#       into the state's top-level "checkpoint" field. New keys are added;
-#       existing keys are overwritten by the new value. This is the generic
-#       "save progress" primitive — directory-triage results, draft node
-#       content, anything else the flow wants durable before it might pause.
+#       Merge the JSON object in FILE (or stdin when FILE is "-") into the
+#       state's top-level "checkpoint" field. New keys are added; existing
+#       keys are overwritten by the new value — EXCEPT when both the existing
+#       value and the new value for a given key are themselves JSON objects,
+#       in which case they are merged one level deep instead of one replacing
+#       the other (existing sub-keys are kept, new sub-keys are added,
+#       conflicting sub-keys take the new value). This one-level dict merge
+#       is what lets a map-shaped field addressed by its own sub-keys — e.g.
+#       "verification_depth", keyed per candidate id (E40_S06_T01) — accumulate
+#       entries across separate checkpoint calls instead of each call
+#       clobbering every entry a previous call wrote. Plain (non-dict)
+#       values — strings, numbers, lists, directory-triage's own arrays —
+#       still simply overwrite, exactly as before this addition. This is the
+#       generic "save progress" primitive — directory-triage results, draft
+#       node content, anything else the flow wants durable before it might
+#       pause.
 #
 #   pause --id ID
 #       Set status "paused" and update "updated_at". The caller (the agent
@@ -422,7 +449,18 @@ elif subcommand == "checkpoint":
     state = load()
     payload = json.loads(checkpoint_json)
     cp = state.setdefault("checkpoint", {})
-    cp.update(payload)
+    # One-level-deep merge when both sides are dicts (E40_S06_T01) — lets a
+    # map-shaped field keyed by its own sub-keys (e.g. verification_depth,
+    # keyed per candidate id) accumulate entries across separate checkpoint
+    # calls instead of each call replacing the whole field. Anything else
+    # (strings, numbers, lists, or a dict landing on a non-dict/absent key)
+    # keeps the prior plain overwrite behavior.
+    for key, value in payload.items():
+        existing = cp.get(key)
+        if isinstance(existing, dict) and isinstance(value, dict):
+            existing.update(value)
+        else:
+            cp[key] = value
     state["updated_at"] = now
     atomic_write(state)
     result = {"checkpoint_keys": list(payload.keys())}

package/skills/jenga/scripts/load-nl-catalog.js

@@ -11,10 +11,16 @@
  * inventory (`readSkillAllowList()`, which reads the committed `lib/skill-allow-list.json`
  * artifact) — this script does not independently re-scan `skills/` for a name list of its own,
  * per E53_S01_T02's acceptance criteria and the drift lesson E41_S04 already documented for that
- * generator. For each name in that inventory, this script reads exactly one file —
- * `skills/<name>/SKILL.md` — to populate the remaining catalog fields: `description`, `keywords`,
- * `examples`, and `metadata.prefered_agent`. These are the same fields `/route`'s Step 1
- * ("Discover Available Skills") collects.
+ * generator. That inventory holds bare identifiers (e.g. "brainstorm"), stripped of the `j.`
+ * frontmatter prefix — but per docs/skill-authoring.md's Canonical Naming Contract, the actual
+ * on-disk directory is `skills/j-<name>/`, except the three permanent exceptions (`jenga`,
+ * `jenga-permission-level`, `index`) which keep their bare directory name. For each name in the
+ * inventory, this script re-derives that canonical directory name, reads exactly one file —
+ * `skills/<dirName>/SKILL.md` — to populate the remaining catalog fields: `description`,
+ * `keywords`, `examples`, and `metadata.prefered_agent`, and emits `dirName` (not the bare
+ * identifier) as the catalog entry's `name` — callers like `/jenga`'s Skill invocation and
+ * `playbook-new.sh`'s `validate-skill` need the real, invokable directory name. These are the
+ * same fields `/route`'s Step 1 ("Discover Available Skills") collects.
  *
  * ---------------------------------------------------------------------------
  * USAGE
@@ -61,6 +67,15 @@ import { readFileSync, existsSync } from "fs";
 import { join } from "path";
 import { pathToFileURL } from "url";
 
+// The three permanent exceptions to the `j-<name>` canonical directory convention — see
+// docs/skill-authoring.md's Canonical Naming Contract and scripts/audit-twin-divergence.sh's
+// NEVER_TWINNED list, which this mirrors.
+const NEVER_TWINNED = new Set(["jenga", "jenga-permission-level", "index"]);
+
+function canonicalSkillDir(name) {
+  return NEVER_TWINNED.has(name) ? name : `j-${name}`;
+}
+
 /**
  * Parses YAML frontmatter from a SKILL.md's content into a plain object. This is a hand-rolled,
  * intentionally minimal parser scoped to the small set of shapes SKILL.md frontmatter actually
@@ -161,7 +176,8 @@ async function main() {
 
   const catalog = [];
   for (const name of names) {
-    const skillMdPath = join(pkgRoot, "skills", name, "SKILL.md");
+    const dirName = canonicalSkillDir(name);
+    const skillMdPath = join(pkgRoot, "skills", dirName, "SKILL.md");
     if (!existsSync(skillMdPath)) {
       process.stderr.write(
         `Warning: ${skillMdPath} not found for allow-listed skill '${name}' — skipped\n`
@@ -186,7 +202,7 @@ async function main() {
     }
 
     catalog.push({
-      name,
+      name: dirName,
       description: fm.description,
       keywords: Array.isArray(fm.keywords) ? fm.keywords : [],
       examples: Array.isArray(fm.examples) ? fm.examples : [],

package/skills/jenga/scripts/load-playbooks.sh

@@ -256,10 +256,30 @@
 # ---------------------------------------------------------------------------
 # DATA SOURCE
 # ---------------------------------------------------------------------------
-# Every `*.json` file directly under `skills/jenga/playbooks/`, EXCLUDING `schema.json` (which
-# documents the required shape — see that file's own header — but is never itself a playbook
-# entry). See `schema.json` for the authoritative field list; this script's validation below is
-# a runtime mirror of that schema, not a substitute for it.
+# TWO directories are scanned and MERGED into one catalog (E53_S09_T01):
+#
+#   1. `skills/jenga/playbooks/` (BUILTIN, framework-owned) — every `*.json` file directly under
+#      it, EXCLUDING `schema.json` (which documents the required shape — see that file's own
+#      header — but is never itself a playbook entry). See `schema.json` for the authoritative
+#      field list; this script's validation below is a runtime mirror of that schema, not a
+#      substitute for it.
+#
+#   2. `project/.playbooks/` (PROJECT, project-owned, resolved relative to this script's own
+#      `PROJECT_DIR` — see its resolution below, including the `JENGA_PLAYBOOKS_TEST_ROOT`
+#      override) — every `*.json` file directly under it, same `schema.json`-exclusion rule.
+#      A MISSING `project/.playbooks/` directory is a SILENT NO-OP: no warning, no error — a
+#      project with no custom playbooks is the common case, not an exceptional one.
+#
+# The builtin directory is scanned and locally validated FIRST, in full, before the project
+# directory is touched at all. Every project-directory file is then checked for an `id` collision
+# against the already-loaded builtin set: if a project playbook's basename/id matches a
+# SUCCESSFULLY LOADED builtin playbook's id, the project playbook is SKIPPED — a stderr warning
+# names BOTH the project file's path and the builtin file's path — and the builtin entry is the
+# one that survives into the catalog. This is never a silent override in either direction. A
+# project playbook whose id does not collide goes through the exact same local validation,
+# composition resolution, and `forward_from`/`conditional` validation as a builtin one — no
+# separate or weaker path. See "VALIDATION / SKIP CONDITIONS" below for the exact collision rule,
+# and "OUTPUT SCHEMA" for the `source` field every catalog entry gains as a result of this merge.
 #
 # ---------------------------------------------------------------------------
 # USAGE
@@ -273,8 +293,10 @@
 #
 # Additive sibling mode (E53_S06_T02), for direct-by-id lookup (`j.playbook <id>`,
 # `skills/j-playbook/SKILL.md`, E53_S06_T03) — runs the SAME PASS 1-3 pipeline as the no-argument
-# mode above (never a separate implementation), then emits exactly ONE JSON object to stdout
-# (never the full catalog, never warnings about OTHER playbooks) and exits 0:
+# mode above (never a separate implementation), against the SAME MERGED builtin+project catalog
+# (E53_S09_T01) — an id may resolve from either source, and the returned `playbook` object carries
+# the same `source` field a full-catalog entry would — then emits exactly ONE JSON object to
+# stdout (never the full catalog, never warnings about OTHER playbooks) and exits 0:
 #
 #   {"status": "valid", "playbook": {...}}    -- <id> resolved to a real file and passed every
 #                                                 validation pass; `playbook` has the same field
@@ -306,6 +328,11 @@
 # asserting on candidate sets never targets the repository root. Never set this variable in a real
 # invocation.
 #
+# Because the SAME override also becomes the PROJECT root (see "DATA SOURCE" above), a fixture
+# wanting project-local playbook coverage places its files at
+# `<value>/project/.playbooks/<id>.json` — no second, project-specific test variable is
+# introduced (E53_S09_T01, `tests/load-playbooks-project-source.bats`).
+#
 # ---------------------------------------------------------------------------
 # OUTPUT SCHEMA
 # ---------------------------------------------------------------------------
@@ -318,11 +345,16 @@
 #       "description": "...",
 #       "keywords":    ["..."],
 #       "examples":    ["..."],
-#       "steps":       ["j-brainstorm", "j-todo", "j-do", "j-dev-done", "j-mirror-public"]
+#       "steps":       ["j-brainstorm", "j-todo", "j-do", "j-dev-done", "j-mirror-public"],
+#       "source":      "builtin"
 #     },
 #     ...
 #   ]
 #
+# `source` (E53_S09_T01) is `"builtin"` for anything loaded from `skills/jenga/playbooks/`, or
+# `"project"` for anything loaded from `project/.playbooks/` — present on EVERY catalog entry,
+# regardless of source, additive to the pre-existing field shape.
+#
 # `steps` entries are emitted exactly as validated/resolved: a depth-1 bare string stays a bare
 # string; a depth-1 StepObject is emitted as an object carrying only its recognized fields (`skill`
 # XOR `playbook` — though by the time this is emitted, a `playbook`-type step has already been
@@ -347,6 +379,11 @@
 #   - `id` does not equal the filename's basename without `.json`          -> skipped
 #     (prevents a playbook's identity from silently drifting from its
 #     file location)
+#   - A PROJECT playbook's id/basename matches a SUCCESSFULLY LOADED
+#     BUILTIN playbook's id                                    (E53_S09_T01) -> project playbook
+#     skipped; stderr warning names BOTH the project file's path and the
+#     builtin file's path; the builtin entry is the one that survives into
+#     the catalog (never a silent override in either direction)
 #   - A `steps` entry is neither a string nor an object, or is an empty
 #     string                                                               -> skipped
 #   - A StepObject step carries BOTH `skill` and `playbook`, or NEITHER    -> skipped
@@ -424,18 +461,30 @@ if [ -n "${JENGA_PLAYBOOKS_TEST_ROOT:-}" ]; then
   # E53_S05_T01 to also cover playbook-config.json resolution). Never set in a real invocation.
   PKG_ROOT="$JENGA_PLAYBOOKS_TEST_ROOT"
   PROJECT_DIR="$JENGA_PLAYBOOKS_TEST_ROOT"
-# Resolve the jenga-agent PACKAGE root (where the canonical skills/ tree actually lives) — same
-# monorepo-checkout vs. installed-npm-package detection used by
-# skills/jenga/scripts/load-nl-catalog.sh's PKG_ROOT resolution and skills/init/scripts/init.sh.
-elif [ -d "$SCRIPT_DIR/../../../templates" ]; then
-  PKG_ROOT="$SCRIPT_DIR/../../.."
-  PROJECT_DIR="${JENGA_PROJECT_DIR:-${CLAUDE_PROJECT_DIR:-$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel 2>/dev/null || pwd)}}"
-elif [ -n "${CLAUDE_PROJECT_DIR:-}" ] && [ -d "${CLAUDE_PROJECT_DIR}/node_modules/@jenga-ai/agent/templates" ]; then
-  PKG_ROOT="${CLAUDE_PROJECT_DIR}/node_modules/@jenga-ai/agent"
-  PROJECT_DIR="${JENGA_PROJECT_DIR:-$CLAUDE_PROJECT_DIR}"
 else
-  echo "Error: could not locate the jenga-agent package root (templates/ not found via monorepo checkout or node_modules/@jenga-ai/agent)." >&2
-  exit 2
+  # Resolve JENGA_PROJECT_DIR the same way every other script in skills/jenga/scripts/ does.
+  if [ -f "$SCRIPT_DIR/../../../lib/resolve-project-dir.sh" ]; then
+    # shellcheck source=/dev/null
+    source "$SCRIPT_DIR/../../../lib/resolve-project-dir.sh"
+  elif [ -n "${CLAUDE_PROJECT_DIR:-}" ]; then
+    JENGA_PROJECT_DIR="$CLAUDE_PROJECT_DIR"
+  else
+    JENGA_PROJECT_DIR="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel 2>/dev/null || pwd)"
+  fi
+
+  # Resolve the jenga-agent PACKAGE root (where the canonical skills/ tree actually lives) — same
+  # monorepo-checkout vs. installed-npm-package detection used by
+  # skills/jenga/scripts/load-nl-catalog.sh's PKG_ROOT resolution and skills/init/scripts/init.sh.
+  if [ -d "$SCRIPT_DIR/../../../templates" ]; then
+    PKG_ROOT="$SCRIPT_DIR/../../.."
+    PROJECT_DIR="$JENGA_PROJECT_DIR"
+  elif [ -d "$JENGA_PROJECT_DIR/node_modules/@jenga-ai/agent/templates" ]; then
+    PKG_ROOT="$JENGA_PROJECT_DIR/node_modules/@jenga-ai/agent"
+    PROJECT_DIR="$JENGA_PROJECT_DIR"
+  else
+    echo "Error: could not locate the jenga-agent package root (templates/ not found via monorepo checkout or node_modules/@jenga-ai/agent)." >&2
+    exit 2
+  fi
 fi
 
 PLAYBOOKS_DIR="$PKG_ROOT/skills/jenga/playbooks"
@@ -467,6 +516,12 @@ project_dir = sys.argv[3] if len(sys.argv) > 3 and sys.argv[3] else None
 mode = sys.argv[4] if len(sys.argv) > 4 and sys.argv[4] else "catalog"
 lookup_id = sys.argv[5] if len(sys.argv) > 5 else ""
 
+# --- E53_S09_T01: project-local playbook source directory ------------------------------------
+# `project/.playbooks/`, resolved relative to the SAME project_dir already threaded above (which
+# already honors JENGA_PLAYBOOKS_TEST_ROOT -- see header "TESTING OVERRIDE"). A missing directory
+# is a silent no-op, not an error -- see header "DATA SOURCE".
+project_playbooks_dir = os.path.join(project_dir, "project", ".playbooks") if project_dir else None
+
 REQUIRED_FIELDS = ["id", "name", "description", "keywords", "examples", "steps"]
 LIST_FIELDS = ["keywords", "examples", "steps"]
 
@@ -673,7 +728,7 @@ def extract_output_types(skill_md_path):
 
 
 try:
-    filenames = sorted(
+    builtin_filenames = sorted(
         f for f in os.listdir(playbooks_dir)
         if f.endswith(".json") and f != "schema.json"
     )
@@ -681,17 +736,41 @@ except OSError as e:
     print(f"Error: could not list {playbooks_dir}: {e}", file=sys.stderr)
     sys.exit(2)
 
+# --- E53_S09_T01: project-local playbook directory listing --------------------------------------
+# Non-fatal, unlike the builtin listing above: a missing directory (the common case -- most
+# projects have no custom playbooks) or an OSError while listing it both yield an empty list,
+# never a setup-error exit. See header "DATA SOURCE".
+project_filenames = []
+if project_playbooks_dir and os.path.isdir(project_playbooks_dir):
+    try:
+        project_filenames = sorted(
+            f for f in os.listdir(project_playbooks_dir)
+            if f.endswith(".json") and f != "schema.json"
+        )
+    except OSError as e:
+        print(f"Warning: could not list {project_playbooks_dir}: {e}", file=sys.stderr)
+        project_filenames = []
+
 # --- PASS 1: local (non-composition) validation -------------------------------------------------
 # Builds `raw[pid]` for every playbook that passes purely local validation (parse/shape/id-match/
 # step-shape/skill-existence) -- independent of any OTHER playbook. Composition resolution (PASS
 # 2, below) needs this map fully populated before it can recurse into a referenced playbook.
+#
+# As of E53_S09_T01, `raw`/`order` are populated from BOTH the builtin and project directories
+# (builtin first, in full, then project -- see the id-collision check below). Every playbook run
+# through `process_playbook_file()` carries a `source` tag ("builtin" or "project") in its `raw`
+# entry, which flows through unchanged into the final catalog entry (PASS 3, below).
 raw = {}
-order = []  # preserves the original sorted-filename order for stable catalog/warning output
+order = []  # preserves the original builtin-then-project, sorted-within-source filename order
 
-for filename in filenames:
-    path = os.path.join(playbooks_dir, filename)
-    basename = filename[: -len(".json")]
 
+def process_playbook_file(path, basename, source):
+    """Runs PASS 1's local validation for one playbook file (identical logic regardless of which
+    source directory it came from -- see header 'DATA SOURCE': project playbooks go through the
+    exact same pipeline as builtin ones, never a separate or weaker path). On success, populates
+    `raw[basename]` (tagged with `source`) and appends to `order`. On failure, prints the same
+    stderr warning this script has always printed for that condition and records the reason in
+    `skip_reasons[basename]`."""
     try:
         with open(path, encoding="utf-8") as fh:
             data = json.load(fh)
@@ -699,20 +778,20 @@ for filename in filenames:
         reason = f"is not valid JSON ({e})"
         print(f"Warning: {path} {reason} — skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     if not isinstance(data, dict):
         reason = "is not a JSON object"
         print(f"Warning: {path} {reason} — skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     missing = [f for f in REQUIRED_FIELDS if f not in data]
     if missing:
         reason = f"missing required field(s) {missing}"
         print(f"Warning: {path} {reason} — skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     bad_list = [
         f for f in LIST_FIELDS
@@ -722,7 +801,7 @@ for filename in filenames:
         reason = f"field(s) {bad_list} must be non-empty lists"
         print(f"Warning: {path} {reason} — skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     if data["id"] != basename:
         reason = (
@@ -730,7 +809,7 @@ for filename in filenames:
         )
         print(f"Warning: {path} {reason} — skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     # --- E53_S03_T01: StepObject shape acceptance + bare-string back-compat ---
     normalized_steps = []
@@ -745,7 +824,7 @@ for filename in filenames:
     if step_error:
         print(f"Warning: {path} {step_error} — skipped", file=sys.stderr)
         skip_reasons[basename] = step_error
-        continue
+        return
 
     # Existence check for skill-type steps only (bare string, or StepObject with `skill`).
     # `playbook`-type steps have no single skill name (step_skill_name returns None for them) and
@@ -766,10 +845,11 @@ for filename in filenames:
         )
         print(f"Warning: {path} {reason} — playbook skipped", file=sys.stderr)
         skip_reasons[basename] = reason
-        continue
+        return
 
     raw[basename] = {
         "path": path,
+        "source": source,
         "name": data["name"],
         "description": data["description"],
         "keywords": data["keywords"],
@@ -778,6 +858,36 @@ for filename in filenames:
     }
     order.append(basename)
 
+
+for filename in builtin_filenames:
+    path = os.path.join(playbooks_dir, filename)
+    basename = filename[: -len(".json")]
+    process_playbook_file(path, basename, "builtin")
+
+# --- E53_S09_T01: project playbooks, id-collision check against the already-loaded builtin set --
+# Runs AFTER the builtin loop above has fully populated `raw`, so a collision check here is
+# checking against every SUCCESSFULLY LOADED builtin playbook -- never a builtin file that itself
+# failed validation (that basename never made it into `raw`, so a project playbook may legitimately
+# claim that id instead). See header 'VALIDATION / SKIP CONDITIONS'.
+for filename in project_filenames:
+    path = os.path.join(project_playbooks_dir, filename)
+    basename = filename[: -len(".json")]
+
+    if basename in raw:
+        builtin_path = raw[basename]["path"]
+        reason = (
+            f"id '{basename}' collides with a built-in playbook already loaded from "
+            f"{builtin_path} — the built-in entry is retained, never silently overridden"
+        )
+        print(
+            f"Warning: {path} {reason} (project file skipped)",
+            file=sys.stderr,
+        )
+        skip_reasons[basename] = reason
+        continue
+
+    process_playbook_file(path, basename, "project")
+
 # --- PASS 2: composition resolution (E53_S05_T01) ------------------------------------------------
 # Recursively resolves `{"playbook": "<id>"}` steps into a flat, fully-spliced step list. See
 # header 'COMPOSITION RESOLUTION' for the full algorithm description; this is that algorithm.
@@ -987,6 +1097,7 @@ for pid in order:
         "keywords": entry["keywords"],
         "examples": entry["examples"],
         "steps": flattened_steps,
+        "source": entry["source"],
     })
 
 # --- E53_S06_T02: `lookup <id>` mode output ------------------------------------------------------
@@ -1001,11 +1112,11 @@ if mode == "lookup":
     if result is None:
         if lookup_id in skip_reasons:
             result = {"status": "invalid", "reason": skip_reasons[lookup_id]}
-        elif f"{lookup_id}.json" in filenames:
-            # Defensive fallback -- should not normally happen, since every basename scanned
-            # into `filenames` either lands in `catalog` (valid) or `skip_reasons` (invalid) by
-            # this point. Guards against ever silently reporting `not_found` for a file that
-            # does exist on disk.
+        elif f"{lookup_id}.json" in builtin_filenames or f"{lookup_id}.json" in project_filenames:
+            # Defensive fallback -- should not normally happen, since every basename scanned from
+            # EITHER source directory (E53_S09_T01) either lands in `catalog` (valid) or
+            # `skip_reasons` (invalid) by this point. Guards against ever silently reporting
+            # `not_found` for a file that does exist on disk.
             result = {
                 "status": "invalid",
                 "reason": "failed validation (no specific reason captured)",

package/templates/SCRUM_BOARD_SCHEMA.md

@@ -105,7 +105,9 @@ date_started:
 date_completed:
 dates_previously_completed:  # comma-separated list, e.g. 2026-01-15, 2026-03-22
 reopened_on:                 # comma-separated list, e.g. 2026-02-01, 2026-04-10
-reopened_reason:             # comma-separated list, e.g. "Scope expanded", "Bug found post-release"
+reopened_reason:             # YAML list, one entry per reopen cycle; see Reopen Tracking Fields
+  - "Scope expanded"
+  - "Bug found post-release"
 docs: []                     # optional list of repo-relative documentation paths, e.g. ["README.md", "docs/API.md"]
 epic_scope_approval: false     # set to true by the human operator only when any task in this epic has execution_scope: epic
 provenance:                  # optional; only valid value is `backfilled` (epic reverse-engineered from pre-existing code by `/uncharted onboard`). Omit for normally-authored epics.
@@ -139,7 +141,9 @@ date_started:
 date_completed:
 dates_previously_completed:  # comma-separated list, e.g. 2026-01-15, 2026-03-22
 reopened_on:                 # comma-separated list, e.g. 2026-02-01, 2026-04-10
-reopened_reason:             # comma-separated list, e.g. "Scope expanded", "Bug found post-release"
+reopened_reason:             # YAML list, one entry per reopen cycle; see Reopen Tracking Fields
+  - "Scope expanded"
+  - "Bug found post-release"
 docs: []                     # optional list of repo-relative documentation paths, e.g. ["README.md", "docs/API.md"]
 crucial_level:                # optional; advisory | gated | locked; absence means no elevated caution
 crucial_set_by:                # required when crucial_level is set; user | scrum-master | <agent>-escalation
@@ -177,7 +181,9 @@ date_started:
 date_completed:
 dates_previously_completed:  # comma-separated list, e.g. 2026-01-15, 2026-03-22
 reopened_on:                 # comma-separated list, e.g. 2026-02-01, 2026-04-10
-reopened_reason:             # comma-separated list, e.g. "Scope expanded", "Bug found post-release"
+reopened_reason:             # YAML list, one entry per reopen cycle; see Reopen Tracking Fields
+  - "Scope expanded"
+  - "Bug found post-release"
 assigned_to: developer | tester | scrum-master
 docs: []                     # optional list of repo-relative documentation paths, e.g. ["README.md", "docs/API.md"]
 execution_scope: task          # task | story | epic | inline | light; omit for legacy tasks (defaults to task)
@@ -209,7 +215,7 @@ crucial_declined_note:           # required when crucial_declined: true; free-te
 
 ### Runtime-written task fields
 
-These four fields are **not authored by hand**. They are appended to a task's frontmatter after execution and are absent from any task that has not yet run. They are listed here so that tooling — in particular `scripts/validate-board.sh` — recognises them as valid rather than unknown.
+These five fields are **not authored by hand**. They are appended to a task's frontmatter after execution and are absent from any task that has not yet run. They are listed here so that tooling — in particular `scripts/validate-board.sh` — recognises them as valid rather than unknown.
 
 | Field | Written by | Meaning |
 |---|---|---|
@@ -217,8 +223,9 @@ These four fields are **not authored by hand**. They are appended to a task's fr
 | `actual_lines_delta` | `/close-story` | Net line delta for the task, from the same extraction |
 | `scope_divergence_flag` | `/close-story` | Set when actual diff stats exceed the thresholds that justified the assigned `execution_scope` |
 | `divergence_flag` | `/do` | Set to `true` by the intent-vs-diff check when a `needs_docs: false` task touched unregistered files |
+| `date_deployed_prod` | `scripts/mark-deployed.sh` | ISO 8601 date (e.g. `2026-09-12`) written in the same locked write window a ticket is set to `status: Deployed to Prod`. Never written on a `Deployed to Stage`-only write. |
 
-All four are advisory and non-blocking — they record evidence for later review and never change a task's Passed/Failed outcome.
+All five are advisory and non-blocking — they record evidence for later review and never change a task's Passed/Failed outcome. `date_deployed_prod` is allow-listed in `scripts/validate-board.sh`'s `ALLOWED_KEYS` for `epic`, `story`, and `task` (a new `DEPLOY_KEYS` group), even though in practice only tasks (and occasionally stories) are expected to ever carry it — `mark-deployed.sh` only ever writes it to a task/story board file, never to an epic.
 
 > `task_changed_files` is **not** a frontmatter field despite the similar name. It lives in the bundle manifest at `project/queue/bundle-<E##_S##>.json`, keyed by task ID.
 
@@ -228,7 +235,12 @@ All four are advisory and non-blocking — they record evidence for later review
 
 ## Reopen Tracking Fields
 
-**`dates_previously_completed`, `reopened_on`, `reopened_reason`** — These fields are **only populated when a previously completed item is being reopened and modified**. Leave them blank on first-run items. Each value is a comma-separated list to support multiple reopen cycles.
+**`dates_previously_completed`, `reopened_on`, `reopened_reason`** — These fields are **only populated when a previously completed item is being reopened and modified**. Leave them blank on first-run items. All three support multiple reopen cycles, but they are **not written the same way**:
+
+- `dates_previously_completed` and `reopened_on` hold a plain comma-separated string, e.g. `reopened_on: 2026-02-01, 2026-04-10`. Bare dates need no quoting, so this parses fine.
+- `reopened_reason` **must be a YAML list** (a block sequence, one `- "reason"` per line). It cannot be comma-separated, because reopen reasons are free text that routinely contains colons and commas and therefore has to be quoted — and a run of comma-separated quoted strings (`reopened_reason: "A", "B"`) is not valid YAML. This schema previously documented exactly that invalid form, and every board file that followed it became unparseable: the board parser skipped those files silently, so the board under-reported itself with no error surfaced anywhere. 14 files were repaired on 2026-09-13; do not reintroduce the comma-separated form here.
+
+Note the resulting asymmetry is deliberate and load-bearing, not an oversight: the first two fields stay strings because migrating them would churn every existing file for no parsing benefit.
 
 ## Planning Fields