@zenera/cli 1.1.22 → 1.1.23

package/templates/editor/.github/skills/zen-cli/references/rag.md

@@ -75,12 +75,18 @@ In a setup step, in this order:
 
 ```sh
 if [ "${FORCE:-0}" = 0 ] && zen rag docs ready --dir "$OUT" --quiet; then exit 3; fi
+if [ "${CHECK:-0}" = 1 ]; then zen rag docs ready --dir "$OUT" || true; exit 4; fi
 if [ "${FORCE:-0}" = 0 ] && [ -f "$OUT/manifest.json" ]; then
     zen rag docs restore --dir "$OUT"; exit 0
 fi
 # … otherwise index it from the documents
 ```
 
+Exit 4 is what `scripts/_setup.sh --check` reports as `needs setup`. Dropping
+`--quiet` there is deliberate: `ready` prints what is missing and the command
+that fixes it, and repeating that in the step would be a second opinion to keep
+in step with this one.
+
 ## Which index gets read
 
 Every reading command takes `-d`, `--dir`. Without one:

package/templates/editor/.github/skills/zen-memory/SKILL.md

@@ -610,7 +610,7 @@ can act on.
 ## Inspecting and repairing it
 
 ```
-zen memory [stats|ls|show|export|forget] [args] [options]
+zen memory [stats|ls|show|export|merge|forget] [args] [options]
 ```
 
 `zen memory export --open` is the one to reach for: one self-contained HTML
@@ -627,6 +627,7 @@ so inspection is free and offline.
 | What has been corrected?                 | `zen memory ls --stale`          |
 | What files are being kept?               | `zen memory ls --files`          |
 | Why was that recalled?                   | `zen memory show <id>`           |
+| Fold parallel warmup graphs into one     | `zen memory merge <dir...>`      |
 | That should never have been written down | `zen memory forget <id>`         |
 
 The directory is not fixed. `zen run --memory <dir>` sends one run's memory

package/templates/editor/.github/skills/zen-memory-warmup/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: zen-memory-warmup
-description: Pre-populating an agent project's memory graph through warm-up queries before production or deployment — isolating scratch memory and workspaces in timestamped .tmp directories, using stronger reasoning models, validating machine-readable JSON output, categorizing queries with limits, shuffling, and dry-run execution, inspecting the graph with zen memory stats/ls, atomic promotion into the project, and generating automated population scripts in scripts/memory_warmup.sh.
+description: Pre-populating an agent project's memory graph through warm-up queries before production or deployment — isolating scratch memory and workspaces in timestamped .tmp directories, using stronger reasoning models, validating machine-readable JSON output, categorizing queries with limits, shuffling, dry-run execution, fanning queries out across parallel runs and folding the results back with zen memory merge, inspecting the graph with zen memory stats/ls, atomic promotion into the project, and generating automated population scripts in scripts/memory_warmup.sh.
 ---
 
 # Memory Warmup
@@ -33,7 +33,7 @@ synthesize, connect, and commit high-value knowledge nodes.
 
 ## Core Architecture & Safety Rules
 
-Warmup is an offline staging process. It must adhere to five safety rules:
+Warmup is an offline staging process. It must adhere to six safety rules:
 
 ### 1. Never warm up directly into the project's active `memory/`
 
@@ -101,6 +101,56 @@ warmup queries, and do not overwrite the project's `memory/` until the final
 graph in `.tmp/warmup-<timestamp>/memory` has been verified to contain expected
 nodes and vector embeddings.
 
+### 6. Give every parallel run its own memory directory
+
+The memory lock is **per directory**. Two runs pointed at the same `--memory`
+do not interleave — the second one refuses to start. So a warmup that wants to
+use the wall clock has to fan out into one memory per run:
+
+```sh
+.tmp/warmup-$STAMP/memory-1
+.tmp/warmup-$STAMP/memory-2
+...
+```
+
+and fold them back together afterwards with `zen memory merge`, which is
+offline, contacts no model, and preserves node ids:
+
+```sh
+zen memory merge .tmp/warmup-$STAMP/memory-* --dir .tmp/warmup-$STAMP/memory
+```
+
+Merging is what makes `--incremental` safe under fan-out: every shard starts
+from a copy of the same project memory, so most of what comes back is a shared
+ancestor rather than a collision. Shared ancestors reconcile silently; two runs
+that genuinely changed the same memory in different ways stop the merge and
+name the ids, because picking a winner silently is how a merge loses an answer.
+
+---
+
+## Fanning Out
+
+```mermaid
+flowchart LR
+    A["Selected queries"] --> B1["zen run → memory-1"]
+    A --> B2["zen run → memory-2"]
+    A --> B3["zen run → memory-N"]
+    B1 --> M["zen memory merge memory-* --dir memory"]
+    B2 --> M
+    B3 --> M
+    M --> V["zen memory stats --dir memory"]
+    V --> P["Backup & promote to memory/"]
+```
+
+Bound the fan-out with `--jobs <n>`. Each run is a model call **and** a Podman
+container, so the useful ceiling is set by the machine and the provider's rate
+limit rather than by the number of queries; 4 is a sane default.
+
+Sequential warmup is still the better choice when the queries build on each
+other, because a later query can recall what an earlier one committed. Fan-out
+trades that away for wall-clock time, and `merge` gives back everything except
+the cross-query recall.
+
 ---
 
 ## The Warmup Cycle
@@ -181,6 +231,7 @@ The script should automate the entire pipeline:
     - `--shuffle`: Randomize query order before applying limits.
     - `--category <name>`: Filter queries to a specific category.
     - `--model <provider:model>`: Override model with a reasoning flagship.
+    - `--jobs <n>`: Run up to $n$ queries at once, each into its own memory.
     - `--incremental`: Copy existing project memory into staging before warming up.
     - `-y`, `--yes`: Promote memory without interactive confirmation.
 2. Create timestamped staging paths under `.tmp/` (e.g.
@@ -189,8 +240,9 @@ The script should automate the entire pipeline:
 4. If `--dry-run` is active, print the execution plan and exit 0 before touching disk.
 5. Invoke `zen run` with `--memory`, `--workspace`, `--model`, and `--json`.
 6. Parse each outcome with `jq`, halt if a turn fails, and report per-query execution time and total elapsed time since start.
-7. Print graph statistics via `zen memory stats --dir ...` between queries to track graph growth, followed by a final validation.
-8. Safely backup and replace the project's `memory/` directory with the warmed graph.
+7. Under `--jobs <n>`, give each run its own `memory-N` directory and fold them back with `zen memory merge memory-* --dir memory`.
+8. Print graph statistics via `zen memory stats --dir ...` between queries to track graph growth, followed by a final validation.
+9. Safely backup and replace the project's `memory/` directory with the warmed graph.
 
 ### Recommended Script Template (`scripts/memory_warmup.sh`)
 
@@ -211,6 +263,7 @@ pattern:
 #   --shuffle               Randomize query order before applying limits
 #   --category <name>       Filter queries to a specific category
 #   --model <ref>           Override model (e.g. anthropic:claude-sonnet-4-5)
+#   --jobs <n>              Run up to n queries at once, one memory each (default 1)
 #   --incremental           Seed staging memory with current memory/ before runs
 #   -y, --yes               Assume yes for memory promotion
 #   -h, --help              Show this help
@@ -219,6 +272,7 @@ pattern:
 #   ./scripts/memory_warmup.sh --dry-run
 #   ./scripts/memory_warmup.sh --dry-run --category workflows --limit 2
 #   ./scripts/memory_warmup.sh --shuffle --limit 5 --model anthropic:claude-sonnet-4-5
+#   ./scripts/memory_warmup.sh --jobs 4
 #   ./scripts/memory_warmup.sh --incremental
 #
 
@@ -234,6 +288,7 @@ INCREMENTAL=false
 ASSUME_YES=false
 DRY_RUN=false
 LIMIT=0
+JOBS=1
 SHUFFLE=false
 CATEGORY_FILTER=""
 
@@ -275,6 +330,10 @@ while [[ $# -gt 0 ]]; do
             MODEL="$2"
             shift 2
             ;;
+        --jobs)
+            JOBS="$2"
+            shift 2
+            ;;
         --incremental)
             INCREMENTAL=true
             shift
@@ -292,6 +351,7 @@ while [[ $# -gt 0 ]]; do
             echo "  --shuffle          Randomize query order before applying limits"
             echo "  --category <name>  Filter queries to a specific category"
             echo "  --model <ref>      Override model (e.g. anthropic:claude-sonnet-4-5)"
+            echo "  --jobs <n>         Run up to n queries at once, one memory each (default 1)"
             echo "  --incremental      Seed staging memory with current memory/ before runs"
             echo "  -y, --yes          Assume yes for memory promotion"
             echo "  -h, --help         Show this help"
@@ -378,6 +438,7 @@ if [ "$DRY_RUN" = true ]; then
     echo "Category filter:   ${CATEGORY_FILTER:-<all>}"
     echo "Shuffled:          $SHUFFLE"
     echo "Limit:             ${LIMIT:-none}"
+    echo "Jobs:              $JOBS"
     echo "Selected queries:  ${#SELECTED[@]} of ${#ALL_QUERIES[@]}"
     echo ""
     echo "Planned queries:"
@@ -389,7 +450,15 @@ if [ "$DRY_RUN" = true ]; then
     echo ""
     echo "Sample command that would execute:"
     SAMPLE_Q="${SELECTED[0]#*|}"
-    echo "  zen run --memory \"$STAGING_MEM\" --workspace \"$WORKSPACES_DIR/run-1-<timestamp>\" ${MODEL:+--model \"$MODEL\" }--json \"$SAMPLE_Q\""
+    if [ "$JOBS" -le 1 ]; then
+        SAMPLE_MEM="$STAGING_MEM"
+    else
+        SAMPLE_MEM="$SCRATCH_DIR/memory-1"
+    fi
+    echo "  zen run --memory \"$SAMPLE_MEM\" --workspace \"$WORKSPACES_DIR/run-1-<timestamp>\" ${MODEL:+--model \"$MODEL\" }--json \"$SAMPLE_Q\""
+    if [ "$JOBS" -gt 1 ]; then
+        echo "  zen memory merge \"$SCRATCH_DIR\"/memory-* --dir \"$STAGING_MEM\" --yes"
+    fi
     echo "  zen memory stats --dir \"$STAGING_MEM\""
     echo ""
     echo "Post-run validation that would run:"
@@ -421,78 +490,139 @@ cleanup() {
 }
 trap cleanup EXIT
 
-# Prepare staging memory
-mkdir -p "$STAGING_MEM" "$WORKSPACES_DIR"
+# Prepare staging paths
+mkdir -p "$WORKSPACES_DIR"
 
-if [ "$INCREMENTAL" = true ] && [ -d "$PROJECT_MEM" ]; then
-    echo "Copying existing memory for incremental warmup..."
-    cp -R "$PROJECT_MEM/." "$STAGING_MEM/"
-fi
-
-echo "Running ${#SELECTED[@]} warmup queries..."
-echo "----------------------------------------"
-
-COUNT=0
-for item in "${SELECTED[@]}"; do
-    COUNT=$((COUNT + 1))
-    CATEGORY="${item%%|*}"
-    QUERY="${item#*|}"
-
-    RUN_STAMP="$(date +%s)"
-    RUN_WORKSPACE="$WORKSPACES_DIR/run-$COUNT-$RUN_STAMP"
-    mkdir -p "$RUN_WORKSPACE"
+seed_memory() {
+    mkdir -p "$1"
+    if [ "$INCREMENTAL" = true ] && [ -d "$PROJECT_MEM" ]; then
+        cp -R "$PROJECT_MEM/." "$1/"
+    fi
+}
 
-    echo "[$COUNT/${#SELECTED[@]}] [$CATEGORY] \"$QUERY\""
-    echo "  -> Workspace: $RUN_WORKSPACE"
+# One query, into the memory directory it is handed. The lock is per directory,
+# so this is the unit that more than one of can be in flight at a time.
+run_query() {
+    local index="$1" category="$2" query="$3" mem="$4"
+    local workspace="$WORKSPACES_DIR/run-$index-$(date +%s)"
+    mkdir -p "$workspace"
 
-    # Assemble arguments
-    CMD=(zen run --memory "$STAGING_MEM" --workspace "$RUN_WORKSPACE" --json)
+    local cmd=(zen run --memory "$mem" --workspace "$workspace" --json)
     if [ -n "$MODEL" ]; then
-        CMD+=(--model "$MODEL")
+        cmd+=(--model "$MODEL")
     fi
-    CMD+=("$QUERY")
-
-    QUERY_START=$(date +%s)
+    cmd+=("$query")
 
-    # Run query and capture JSON output
-    JSON_OUT="$("${CMD[@]}")"
-    EXIT_CODE=$?
+    echo "[$index/${#SELECTED[@]}] [$category] \"$query\""
+    echo "  -> Workspace: $workspace"
 
-    QUERY_END=$(date +%s)
-    QUERY_DURATION=$((QUERY_END - QUERY_START))
-    TOTAL_ELAPSED=$((QUERY_END - WARMUP_START))
-
-    if [ $EXIT_CODE -ne 0 ]; then
-        echo "  -> Command exited with code $EXIT_CODE (query: $(format_duration $QUERY_DURATION), total elapsed: $(format_duration $TOTAL_ELAPSED))" >&2
-        exit $EXIT_CODE
+    local start out
+    start=$(date +%s)
+    if ! out="$("${cmd[@]}")"; then
+        echo "  -> Command failed" >&2
+        return 1
     fi
 
-    # Parse outcome via jq if available
     if command -v jq >/dev/null 2>&1; then
-        STOP_REASON=$(echo "$JSON_OUT" | jq -r '.stopReason // empty')
-        AGENT=$(echo "$JSON_OUT" | jq -r '.agent // empty')
-        echo "  -> Completed: agent=$AGENT stopReason=$STOP_REASON"
-
-        if [ "$STOP_REASON" != "final" ]; then
-            echo "  -> Unexpected stop reason: $STOP_REASON" >&2
-            exit 1
+        local stop agent
+        stop=$(echo "$out" | jq -r '.stopReason // empty')
+        agent=$(echo "$out" | jq -r '.agent // empty')
+        echo "  -> Completed: agent=$agent stopReason=$stop"
+        if [ "$stop" != "final" ]; then
+            echo "  -> Unexpected stop reason: $stop" >&2
+            return 1
         fi
     else
         echo "  -> Turn completed successfully."
     fi
 
-    echo "  -> Timing: query took $(format_duration $QUERY_DURATION) | elapsed since start: $(format_duration $TOTAL_ELAPSED)"
+    local now
+    now=$(date +%s)
+    echo "  -> Timing: query took $(format_duration $((now - start))) | elapsed since start: $(format_duration $((now - WARMUP_START)))"
+}
+
+# `wait` with no arguments always reports success, so every child is waited for
+# by pid. Batches of $JOBS rather than a rolling queue: `wait -n` wants bash
+# 4.3 and macOS still ships 3.2.
+PIDS=()
+FAILED=false
+drain() {
+    if [ ${#PIDS[@]} -eq 0 ]; then
+        return 0
+    fi
+    local pid
+    for pid in "${PIDS[@]}"; do
+        wait "$pid" || FAILED=true
+    done
+    PIDS=()
+}
+
+echo "Running ${#SELECTED[@]} warmup queries (jobs: $JOBS)..."
+echo "----------------------------------------"
+
+if [ "$JOBS" -le 1 ]; then
+    seed_memory "$STAGING_MEM"
+    COUNT=0
+    for item in "${SELECTED[@]}"; do
+        COUNT=$((COUNT + 1))
+        run_query "$COUNT" "${item%%|*}" "${item#*|}" "$STAGING_MEM"
 
-    # Print memory stats between queries to track graph growth
-    if [ -f "$STAGING_MEM/manifest.json" ]; then
+        # Print memory stats between queries to track graph growth
+        if [ -f "$STAGING_MEM/manifest.json" ]; then
+            echo ""
+            echo "Memory stats after query $COUNT:"
+            zen memory stats --dir "$STAGING_MEM"
+        else
+            echo "  -> Memory stats: No memories committed yet."
+        fi
         echo ""
-        echo "Memory stats after query $COUNT:"
-        zen memory stats --dir "$STAGING_MEM"
-    else
-        echo "  -> Memory stats: No memories committed yet."
+    done
+else
+    SHARDS=()
+    COUNT=0
+    for item in "${SELECTED[@]}"; do
+        COUNT=$((COUNT + 1))
+        SHARD="$SCRATCH_DIR/memory-$COUNT"
+        seed_memory "$SHARD"
+        SHARDS+=("$SHARD")
+        run_query "$COUNT" "${item%%|*}" "${item#*|}" "$SHARD" \
+            >"$SCRATCH_DIR/run-$COUNT.log" 2>&1 &
+        PIDS+=($!)
+        if [ ${#PIDS[@]} -ge "$JOBS" ]; then
+            drain
+        fi
+    done
+    drain
+
+    for log in "$SCRATCH_DIR"/run-*.log; do
+        if [ -f "$log" ]; then
+            cat "$log"
+            echo ""
+        fi
+    done
+
+    if [ "$FAILED" = true ]; then
+        echo "One or more warmup runs failed; nothing was promoted." >&2
+        exit 1
     fi
+
+    # Only a memory something was committed to can be merged; a run that found
+    # nothing worth keeping never writes a manifest.
+    MERGE=()
+    for shard in "${SHARDS[@]}"; do
+        if [ -f "$shard/manifest.json" ]; then
+            MERGE+=("$shard")
+        fi
+    done
+    if [ ${#MERGE[@]} -eq 0 ]; then
+        echo "No warmup run committed anything." >&2
+        exit 1
+    fi
+
+    echo "Merging ${#MERGE[@]} memories into $STAGING_MEM..."
+    zen memory merge "${MERGE[@]}" --dir "$STAGING_MEM" --yes
     echo ""
-done
+fi
 
 echo "----------------------------------------"
 TOTAL_RUN_TIME=$(( $(date +%s) - WARMUP_START ))

package/templates/editor/.github/skills/zen-spec-sync/SKILL.md

@@ -111,11 +111,12 @@ However small the diff:
    last.
 2. **`scripts/_setup.sh` has run to completion in this session, and re-runs
    clean.** A one-line specification change can still break a setup step, and a
-   pass that did not run it does not know. `skipped` is only a pass if the
-   artefact really is usable: for every index the project builds, check
-   `zen rag <subject> ready --dir <dir>` agrees. A step that tests for a
-   committed `manifest.json` skips on a fresh clone whose git-ignored `lance/`
-   vectors are missing, and the project cannot search.
+   pass that did not run it does not know. Then `scripts/_setup.sh --check` must
+   report every step `skipped`: it makes each step's own readiness test without
+   doing the work, which is what tells a real `skipped` from a step that only
+   looked done. A step that tests for a committed `manifest.json` skips on a
+   fresh clone whose git-ignored `lance/` vectors are missing, and the project
+   cannot search.
 3. **§9 of the copilot instructions, worked as a checklist**, group by group.
 4. **The capability obligations in §0.1.** For every capability the project has
    turned on, load that capability's skill and re-check what it requires - in

package/templates/project/SPECIFICATION.md.tmpl

@@ -28,6 +28,8 @@ something in it — and report what was done in enough detail to be checked.
   are the steps it runs in order. A step is safe to run twice and reports
   `skipped` the second time — testing for something that survives a clone, not
   for a file `.gitignore` excludes.
+- `scripts/_setup.sh --check` asks the same questions and does none of the work,
+  so what is missing can be read without paying to find out.
 - Currently: no steps. Nothing has to be prepared before a run.
 
 ## Agents

package/templates/project/scripts/_setup.sh

@@ -4,6 +4,7 @@
 #
 #   scripts/_setup.sh            do whatever is not done yet
 #   scripts/_setup.sh --force    do all of it again
+#   scripts/_setup.sh --check    say what is not done, and do none of it
 #
 # Anything that has to be in place before a run — an index built, a document
 # fetched, a file generated — is a script in this directory rather than a
@@ -17,6 +18,11 @@
 #   - takes no arguments, and reads $FORCE to know whether to redo finished work
 #   - exits 0 when it did the work, 3 when there was nothing to do, non-zero
 #     when it failed
+#   - under $CHECK, does none of the work: it runs the same "already done" test,
+#     exits 3 when the answer is yes, and otherwise says what is missing and
+#     exits 4. A check must not assume an earlier step has run — a missing
+#     prerequisite is a 4 of its own, not a hard failure, or one unbuilt thing
+#     hides the rest of the list
 #   - writes under .tmp/ and moves the result into place, so an interrupted run
 #     never leaves half an artefact behind
 #
@@ -27,6 +33,10 @@
 # first search fails for a reason nobody connects to setup. Ask the tool instead
 # of guessing at files — `zen rag docs ready` answers exactly that question.
 #
+# That test is the whole of `--check`, which is why this is one script and not
+# two: the steps already know what "done" means, and a second script asking the
+# same question would be a second answer to keep in step with this one.
+#
 # Running this twice has to be safe, and the second run is what proves it: every
 # step should report `skipped`.
 
@@ -34,10 +44,17 @@ set -eu
 cd "$(dirname "$0")/.."
 
 FORCE=0
-if [ "${1:-}" = "--force" ]; then
-    FORCE=1
-fi
-export FORCE
+CHECK=0
+case "${1:-}" in
+    '') ;;
+    --force) FORCE=1 ;;
+    --check) CHECK=1 ;;
+    *)
+        echo "usage: scripts/_setup.sh [--force|--check]" >&2
+        exit 2
+        ;;
+esac
+export FORCE CHECK
 
 LOGS=.tmp/logs
 
@@ -58,6 +75,12 @@ LOGS=.tmp/logs
 #       echo "$OUT is already built"
 #       exit 3
 #   fi
+#   if [ "${CHECK:-0}" = 1 ]; then
+#       # Without --quiet it names what is missing and the command that fixes
+#       # it; `|| true` because not-ready is the exit code we came for.
+#       zen rag docs ready --dir "$OUT" || true
+#       exit 4
+#   fi
 #   # Committed index, git-ignored vectors: re-embed from the copies the index
 #   # already holds rather than indexing the documents over again.
 #   if [ "${FORCE:-0}" = 0 ] && [ -f "$OUT/manifest.json" ]; then
@@ -80,9 +103,16 @@ fi
 mkdir -p "$LOGS"
 summary=""
 failed=0
+pending=0
 
 for name in $STEPS; do
-    log="$LOGS/setup-$name.log"
+    # A dry run keeps its own log: overwriting the build's would lose the
+    # record of the build to a question about it.
+    if [ "$CHECK" = 1 ]; then
+        log="$LOGS/check-$name.log"
+    else
+        log="$LOGS/setup-$name.log"
+    fi
     echo
     echo "-- $name  $(date '+%H:%M:%S')  ($log)"
 
@@ -96,6 +126,10 @@ for name in $STEPS; do
     case "$code" in
         0) word=ok ;;
         3) word=skipped ;;
+        4)
+            word="needs setup"
+            pending=1
+            ;;
         *)
             word="failed ($code)"
             failed=1
@@ -105,12 +139,20 @@ for name in $STEPS; do
 "
     # Later steps depend on earlier ones, so a failure stops rather than
     # producing a second failure that is only a consequence of the first.
+    # `needs setup` does not stop it: the point of asking is the whole list.
     if [ "$failed" = 1 ]; then
         break
     fi
 done
 
 echo
-echo "setup:"
+if [ "$CHECK" = 1 ]; then
+    echo "setup check:"
+else
+    echo "setup:"
+fi
 printf '%s' "$summary"
-exit "$failed"
+if [ "$failed" = 1 ] || [ "$pending" = 1 ]; then
+    exit 1
+fi
+exit 0