aether-colony 5.2.1 → 5.3.1

package/.aether/commands/opencode/watch.md

@@ -0,0 +1,227 @@
+<!-- Generated from .aether/commands/watch.yaml - DO NOT EDIT DIRECTLY -->
+---
+name: ant:watch
+description: "👁️🔄🐜🏠🔄👁️ Set up tmux session to watch ants working in real-time"
+---
+
+### Step -1: Normalize Arguments
+
+Run: `normalized_args=$(bash .aether/aether-utils.sh normalize-args "$@")`
+
+This ensures arguments work correctly in both Claude Code and OpenCode. Use `$normalized_args` throughout this command.
+
+You are the **Queen**. Set up live visibility into colony activity.
+
+## Instructions
+
+### Step 1: Check Prerequisites
+
+Use Bash to check if tmux is available:
+```bash
+command -v tmux >/dev/null 2>&1 && echo "tmux_available" || echo "tmux_missing"
+```
+
+If tmux is missing:
+```
+tmux is required for live colony viewing.
+
+Install with:
+  macOS:  brew install tmux
+  Ubuntu: sudo apt install tmux
+  Fedora: sudo dnf install tmux
+```
+Stop here.
+
+### Step 2: Initialize Activity Log
+
+Ensure activity log exists:
+```bash
+mkdir -p .aether/data
+touch .aether/data/activity.log
+```
+
+### Step 3: Create Status File
+
+Write initial status to `.aether/data/watch-status.txt`:
+
+```
+       .-.
+      (o o)  AETHER COLONY
+      | O |  Live Status
+       `-`
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+State: IDLE
+Phase: -/-
+
+Active Workers:
+  (none)
+
+Last Activity:
+  (waiting for colony activity)
+```
+
+### Step 4: Create or Attach to tmux Session
+
+Check if session exists:
+```bash
+tmux has-session -t aether-colony 2>/dev/null && echo "exists" || echo "new"
+```
+
+**If session exists:** Attach to it
+```bash
+tmux attach-session -t aether-colony
+```
+Output: `Attached to existing aether-colony session.`
+Stop here.
+
+**If session is new:** Create the layout.
+
+### Step 5: Create tmux Layout (4-Pane)
+
+Use Bash to create the session with 4 panes in a 2x2 grid:
+
+```bash
+# Create session with first pane
+tmux new-session -d -s aether-colony -n colony
+
+# Split into 4 panes (2x2 grid)
+# First split horizontally (left|right)
+tmux split-window -h -t aether-colony:colony
+
+# Split left side vertically (top-left, bottom-left)
+tmux split-window -v -t aether-colony:colony.0
+
+# Split right side vertically (top-right, bottom-right)
+tmux split-window -v -t aether-colony:colony.2
+
+# Set pane contents:
+# Pane 0 (top-left): Status display
+tmux send-keys -t aether-colony:colony.0 'watch -n 1 cat .aether/data/watch-status.txt' C-m
+
+# Pane 1 (bottom-left): Progress bar
+tmux send-keys -t aether-colony:colony.1 'watch -n 1 cat .aether/data/watch-progress.txt' C-m
+
+# Pane 2 (top-right): Spawn tree visualization
+tmux send-keys -t aether-colony:colony.2 'bash .aether/utils/watch-spawn-tree.sh .aether/data' C-m
+
+# Pane 3 (bottom-right): Colorized activity log stream
+tmux send-keys -t aether-colony:colony.3 'bash .aether/utils/colorize-log.sh .aether/data/activity.log' C-m
+
+# Set pane titles (if supported)
+tmux select-pane -t aether-colony:colony.0 -T "Status"
+tmux select-pane -t aether-colony:colony.1 -T "Progress"
+tmux select-pane -t aether-colony:colony.2 -T "Spawn Tree"
+tmux select-pane -t aether-colony:colony.3 -T "Activity Log"
+
+# Balance panes for even 2x2 grid
+tmux select-layout -t aether-colony:colony tiled
+
+echo "Session created"
+```
+
+### Step 6: Create Progress File
+
+Write initial progress to `.aether/data/watch-progress.txt`:
+
+```
+       .-.
+      (o o)  AETHER COLONY
+      | O |  Progress
+       `-`
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Phase: -/-
+
+[░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 0%
+
+⏳ Waiting for build...
+
+Target: 95% confidence
+```
+
+### Step 7: Attach and Display
+
+```bash
+tmux attach-session -t aether-colony
+```
+
+Before attaching, output:
+
+```
+       .-.
+      (o o)  AETHER COLONY :: WATCH
+      | O |
+       `-`
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+tmux session 'aether-colony' created.
+
+Layout (4-pane 2x2 grid):
+  +------------------+------------------+
+  | Status           | Spawn Tree       |
+  | Colony state     | Worker hierarchy |
+  +------------------+------------------+
+  | Progress         | Activity Log     |
+  | Phase progress   | Live stream      |
+  +------------------+------------------+
+
+Commands:
+  Ctrl+B D          Detach from session
+  Ctrl+B [          Scroll mode (q to exit)
+  Ctrl+B Arrow      Navigate between panes
+  tmux kill-session -t aether-colony   Stop watching
+
+The session will update in real-time as colony works.
+Attaching now...
+```
+
+---
+
+## Status Update Protocol
+
+Workers and commands update watch files as they work:
+
+### Activity Log
+Workers write via: `bash .aether/aether-utils.sh activity-log "ACTION" "caste" "description"`
+
+For named ants (recommended):
+```bash
+# Generate a name first
+ant_name=$(bash .aether/aether-utils.sh generate-ant-name "builder" | jq -r '.result')
+# Log with ant name
+bash .aether/aether-utils.sh activity-log "CREATED" "$ant_name (Builder)" "Implemented auth module"
+```
+
+### Spawn Tracking
+Log spawns for tree visualization:
+```bash
+bash .aether/aether-utils.sh spawn-log "Prime" "builder" "Hammer-42" "implementing auth"
+bash .aether/aether-utils.sh spawn-complete "Hammer-42" "completed" "auth module done"
+```
+
+### Status File
+Commands update `.aether/data/watch-status.txt` with current state:
+- State: PLANNING, EXECUTING, READY
+- Phase: current/total
+- Active Workers: list of named ants
+- Last Activity: most recent log entry
+
+### Progress File
+Update via: `bash .aether/aether-utils.sh update-progress <percent> "<message>" <phase> <total>`
+
+Example:
+```bash
+bash .aether/aether-utils.sh update-progress 45 "Building auth module..." 2 5
+```
+
+---
+
+## Cleanup
+
+To stop watching:
+```bash
+tmux kill-session -t aether-colony
+```
+
+This stops the session but preserves all log files.

package/.aether/docs/command-playbooks/build-full.md

@@ -254,7 +254,7 @@ git rev-parse --git-dir 2>/dev/null
 
 - **If succeeds** (is a git repo):
   1. Check for changes in Aether-managed directories only: `.aether .claude/commands/ant .claude/commands/st .opencode bin`
-  2. **If changes exist**: Run using the Bash tool with description "Creating git checkpoint...": `git stash push -m "aether-checkpoint: pre-phase-$PHASE_NUMBER" -- .aether .claude/commands/ant .claude/commands/st .opencode bin`
+  2. **If changes exist**: Run using the Bash tool with description "Creating git checkpoint...": `git stash push -m "aether-checkpoint: pre-phase-$PHASE_NUMBER" -- .aether .claude/commands/ant .claude/commands/st .opencode bin ":(exclude).aether/data/"`
      - IMPORTANT: Never use `--include-untracked` — it stashes ALL files including user work!
      - Run using the Bash tool with description "Verifying checkpoint...": `git stash list | head -1 | grep "aether-checkpoint"` — warn if empty
      - Store checkpoint as `{type: "stash", ref: "aether-checkpoint: pre-phase-$PHASE_NUMBER"}`

package/.aether/docs/command-playbooks/build-prep.md

@@ -226,7 +226,7 @@ Parse the JSON result (`.result.blockers`):
 
 ### Step 2: Update State
 
-Read then update `.aether/data/COLONY_STATE.json`:
+Update `.aether/data/COLONY_STATE.json` using state-mutate (atomic, locked, validated):
 - Set `state` to `"EXECUTING"`
 - Set `current_phase` to the phase number
 - Set the phase's `status` to `"in_progress"` in `plan.phases[N]`
@@ -235,7 +235,14 @@ Read then update `.aether/data/COLONY_STATE.json`:
 
 If `events` exceeds 100 entries, keep only the last 100.
 
-Write COLONY_STATE.json.
+Run using the Bash tool with description "Updating colony state for build...":
+```bash
+bash .aether/aether-utils.sh state-mutate \
+  --argjson phase "$PHASE_NUMBER" \
+  --arg phase_name "$PHASE_NAME" \
+  --arg timestamp "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
+  '.state = "EXECUTING" | .current_phase = $phase | .plan.phases |= map(if .id == $phase then .status = "in_progress" else . end) | .build_started_at = $timestamp | .events = ((.events[-99:]) + [$timestamp + "|phase_started|build|Phase " + ($phase|tostring) + ": " + $phase_name + " started"])'
+```
 
 Validate the state file:
 Run using the Bash tool with description "Validating colony state...":
@@ -254,7 +261,7 @@ git rev-parse --git-dir 2>/dev/null
 
 - **If succeeds** (is a git repo):
   1. Check for changes in Aether-managed directories only: `.aether .claude/commands/ant .claude/commands/st .opencode bin`
-  2. **If changes exist**: Run using the Bash tool with description "Creating git checkpoint...": `git stash push -m "aether-checkpoint: pre-phase-$PHASE_NUMBER" -- .aether .claude/commands/ant .claude/commands/st .opencode bin`
+  2. **If changes exist**: Run using the Bash tool with description "Creating git checkpoint...": `git stash push -m "aether-checkpoint: pre-phase-$PHASE_NUMBER" -- .aether .claude/commands/ant .claude/commands/st .opencode bin ":(exclude).aether/data/"`
      - IMPORTANT: Never use `--include-untracked` — it stashes ALL files including user work!
      - Run using the Bash tool with description "Verifying checkpoint...": `git stash list | head -1 | grep "aether-checkpoint"` — warn if empty
      - Store checkpoint as `{type: "stash", ref: "aether-checkpoint: pre-phase-$PHASE_NUMBER"}`

package/.aether/docs/command-playbooks/build-verify.md

@@ -1,3 +1,16 @@
+### Step 5.3.5: Generate Compact Review Context for Review Cycles
+
+Before spawning review agents, generate compact colony context for CI/autopilot agent review cycles.
+
+Run using the Bash tool with description "Generating compact review context...":
+```bash
+REVIEW_CONTEXT=$(bash "$AETHER_UTILS" pr-context --compact 2>/dev/null) || true
+```
+
+- If empty, continue without review context (non-blocking)
+- Provides compact colony context for CI/autopilot agent review cycles
+- Output is available for Watcher and subsequent review agents but does not gate them
+
 ### Step 5.4: Spawn Watcher for Verification
 
 **MANDATORY: Always spawn a Watcher — testing must be independent.**
@@ -403,3 +416,41 @@ bash .aether/aether-utils.sh memory-capture \
 ```
 
 This ensures verification failures are persisted as blockers that survive context resets. Chaos Ant findings are flagged in Step 5.7.
+
+### Step 5.9: Midden Collection for Merged Branches (NON-BLOCKING)
+
+**Per D-04: Wire midden-collect into /ant:run flow (build-verify phase).**
+
+If this build is running on main after a merge (detected via COLONY_STATE or git log), attempt midden collection:
+
+Run using the Bash tool with description "Collecting midden from merged branch...":
+```bash
+# Only runs if merge context is available
+if [[ -n "${last_merged_branch:-}" && -n "${last_merge_sha:-}" ]]; then
+  collect_result=$(bash .aether/aether-utils.sh midden-collect \
+    --branch "$last_merged_branch" --merge-sha "$last_merge_sha" \
+    2>/dev/null || echo '{"ok":false}')
+  collect_ok=$(echo "$collect_result" | jq -r '.ok // false' 2>/dev/null)
+  if [[ "$collect_ok" == "true" ]]; then
+    collect_status=$(echo "$collect_result" | jq -r '.result.status // "unknown"' 2>/dev/null)
+    new_entries=$(echo "$collect_result" | jq -r '.result.entries_collected // 0' 2>/dev/null)
+    if [[ "$collect_status" == "collected" && "$new_entries" -gt 0 ]]; then
+      echo "Midden: collected $new_entries entries from $last_merged_branch"
+    fi
+  fi
+
+  # Run cross-PR analysis after collection (per D-05)
+  analysis_result=$(bash .aether/aether-utils.sh midden-cross-pr-analysis --window 14 \
+    2>/dev/null || echo '{"ok":false}')
+  analysis_ok=$(echo "$analysis_result" | jq -r '.ok // false' 2>/dev/null)
+  if [[ "$analysis_ok" == "true" ]]; then
+    systemic=$(echo "$analysis_result" | jq -r '.result.systemic_categories // [] | length' 2>/dev/null || echo "0")
+    if [[ "$systemic" -gt 0 ]]; then
+      categories=$(echo "$analysis_result" | jq -r '.result.systemic_categories | join(", ")' 2>/dev/null)
+      echo "Midden: cross-PR systemic: $categories"
+    fi
+  fi
+fi
+```
+
+This step is NON-BLOCKING -- build verification proceeds regardless of collection or analysis outcome.

package/.aether/docs/command-playbooks/continue-advance.md

@@ -1,3 +1,16 @@
+### Step 2.0.7: Generate Review Context (pr-context)
+
+After verification passes, generate structured colony context for the review and learning steps that follow.
+
+Run using the Bash tool with description "Generating review context...":
+```bash
+REVIEW_CONTEXT=$(bash "$AETHER_UTILS" pr-context 2>/dev/null) || true
+```
+
+- If REVIEW_CONTEXT is empty or fails, continue without it (non-blocking)
+- This provides structured colony context for review/learning steps
+- Output is available for subsequent steps but does not gate them
+
 ### Step 2: Update State
 
 Find current phase in `plan.phases`.
@@ -285,20 +298,116 @@ Update COLONY_STATE.json:
    - Keep max 30 instincts (remove lowest confidence)
    - Keep max 100 events
 
-Write the updated state through the locked subcommand. Construct the full updated COLONY_STATE.json content as a variable, then pipe it to state-write:
+Write the updated state through targeted `state-mutate` calls. Each call acquires a lock, creates a backup, applies the jq expression, validates, and writes atomically. Do NOT use the Write tool or `state-write` with full JSON content — always use `state-mutate` to avoid stale-context corruption.
 
-Run using the Bash tool with description "Writing colony state...":
+Run using the Bash tool with description "Advancing colony state...":
 ```bash
-cat << 'STATEOF' | bash .aether/aether-utils.sh state-write
-<the full JSON content>
-STATEOF
+# Mark current phase completed
+bash .aether/aether-utils.sh state-mutate --argjson pid "$current_phase" \
+  '.plan.phases |= map(if .id == $pid then .status = "completed" else . end)'
+
+# Append learning (if any — skip if no learnings were extracted in Step 2)
+# bash .aether/aether-utils.sh state-mutate --argjson learning "$learning_json" \
+#   '.memory.phase_learnings += [$learning]'
+
+# Advance to next phase
+bash .aether/aether-utils.sh state-mutate \
+  --argjson pid "$current_phase" \
+  --argjson next "$next_phase" \
+  --arg timestamp "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
+  '.current_phase = $next | .state = "READY" | .build_started_at = null | .events += [$timestamp + "|phase_advanced|continue|Completed Phase " + ($pid|tostring) + ", advancing to Phase " + ($next|tostring)]'
 ```
 
-This acquires a lock, creates a rolling backup, validates JSON, and writes atomically. Do NOT use the Write tool to write COLONY_STATE.json directly — always go through state-write.
+Run using the Bash tool with description "Enforcing memory caps...":
+```bash
+# Cap enforcement — keep arrays bounded
+bash .aether/aether-utils.sh state-mutate \
+  '.memory.phase_learnings = (.memory.phase_learnings[-20:]) | .memory.decisions = (.memory.decisions[-30:]) | .memory.instincts = (.memory.instincts | sort_by(.confidence) | .[-30:]) | .events = (.events[-100:])'
+```
 
 Validate the state file:
 Run using the Bash tool with description "Validating colony state...": `bash .aether/aether-utils.sh validate-state colony`
 
+### Step 2.0.5: Pheromone Merge-Back (SILENT, NON-BLOCKING)
+
+If a `pheromone-branch-export.json` exists in `.aether/exchange/` (written by seal ceremony on a PR branch and merged to main), run merge-back to collect branch-discovered signals into main's pheromone store. This entire step is silent and non-blocking -- continue proceeds even if merge-back fails.
+
+Run using the Bash tool with description "Checking for pheromone merge-back file...":
+```bash
+# Check if a branch pheromone export was merged into main
+export_file=".aether/exchange/pheromone-branch-export.json"
+if [[ -f "$export_file" ]]; then
+  merge_result=$(bash .aether/aether-utils.sh pheromone-merge-back --export-file "$export_file" 2>/dev/null || echo '{"ok":false}')
+  merge_ok=$(echo "$merge_result" | jq -r '.ok // false' 2>/dev/null)
+  if [[ "$merge_ok" == "true" ]]; then
+    new_count=$(echo "$merge_result" | jq -r '.result.new_signals_written // 0' 2>/dev/null)
+    skipped=$(echo "$merge_result" | jq -r '.result.skipped_count // 0' 2>/dev/null)
+    conflicts=$(echo "$merge_result" | jq -r '.result.conflicts_resolved // [] | length' 2>/dev/null)
+    if [[ "$new_count" -gt 0 || "$conflicts" -gt 0 ]]; then
+      echo "Pheromone merge-back: $new_count new, $conflicts conflicts resolved, $skipped skipped (from merged branch)"
+    fi
+    # Clean up export file after successful merge
+    rm -f "$export_file" 2>/dev/null || true
+  else
+    echo "Pheromone merge-back: failed (non-blocking)"
+  fi
+fi
+```
+
+### Step 2.0.6: Midden Collection (NON-BLOCKING)
+
+After pheromone merge-back, collect failure records from any recently merged branch worktrees. This step is silent and non-blocking -- continue proceeds even if collection fails.
+
+**Per D-04: Wire midden-collect into /ant:continue flow.**
+
+If the colony uses a PR-based workflow and a merge just happened, attempt to collect the branch's midden entries:
+
+Run using the Bash tool with description "Collecting branch midden entries...":
+```bash
+# Check if there's a recently merged branch to collect from
+# The merge info comes from git log or COLONY_STATE context
+last_merge_branch="${last_merged_branch:-}"
+last_merge_sha="${last_merge_sha:-}"
+
+if [[ -n "$last_merge_branch" && -n "$last_merge_sha" ]]; then
+  collect_result=$(bash .aether/aether-utils.sh midden-collect \
+    --branch "$last_merge_branch" --merge-sha "$last_merge_sha" \
+    2>/dev/null || echo '{"ok":false}')
+  collect_ok=$(echo "$collect_result" | jq -r '.ok // false' 2>/dev/null)
+  if [[ "$collect_ok" == "true" ]]; then
+    collect_status=$(echo "$collect_result" | jq -r '.result.status // "unknown"' 2>/dev/null)
+    if [[ "$collect_status" == "collected" ]]; then
+      new_entries=$(echo "$collect_result" | jq -r '.result.entries_collected // 0' 2>/dev/null)
+      echo "Midden: collected $new_entries failure entries from branch $last_merge_branch"
+    fi
+  fi
+fi
+```
+
+This step is NON-BLOCKING -- continue proceeds regardless of collection outcome. If `last_merge_branch` and `last_merge_sha` are not set (e.g., no recent merge), this step is silently skipped.
+
+### Step 2.0.7: Cross-PR Midden Analysis (NON-BLOCKING)
+
+After midden collection (Step 2.0.6), run cross-PR analysis to detect systemic failure patterns across multiple merged branches. Auto-emits REDIRECT pheromones to hub if systemic patterns found.
+
+**Per D-05: Wire midden-cross-pr-analysis into /ant:continue flow.**
+
+Run using the Bash tool with description "Running cross-PR midden analysis...":
+```bash
+analysis_result=$(bash .aether/aether-utils.sh midden-cross-pr-analysis --window 14 \
+  2>/dev/null || echo '{"ok":false}')
+analysis_ok=$(echo "$analysis_result" | jq -r '.ok // false' 2>/dev/null)
+if [[ "$analysis_ok" == "true" ]]; then
+  systemic=$(echo "$analysis_result" | jq -r '.result.systemic_categories // [] | length' 2>/dev/null || echo "0")
+  if [[ "$systemic" -gt 0 ]]; then
+    categories=$(echo "$analysis_result" | jq -r '.result.systemic_categories | join(", ")' 2>/dev/null)
+    echo "Midden: cross-PR systemic pattern detected in: $categories (REDIRECT emitted)"
+  fi
+fi
+```
+
+This step is NON-BLOCKING -- advance proceeds regardless of analysis outcome.
+
 ### Step 2.1: Auto-Emit Phase Pheromones (SILENT)
 
 **This entire step produces NO user-visible output.** All pheromone operations run silently — learnings are deposited in the background. If any pheromone call fails, log the error and continue. Phase advancement must never fail due to pheromone errors.

package/.aether/docs/command-playbooks/continue-verify.md

@@ -405,3 +405,35 @@ Cross-reference worker claims against reality. This step catches fabricated succ
    **CRITICAL:** Do NOT proceed to Step 1.6. Do NOT advance the phase. The verification failure is a hard block just like a test failure.
 
 Continue to Step 1.6.
+
+### Step 2.0.6: Midden Collection (NON-BLOCKING)
+
+After verification passes, collect failure records from any recently merged branch worktrees. This step is silent and non-blocking -- continue proceeds even if collection fails.
+
+**Per D-04: Wire midden-collect into /ant:continue flow.**
+
+If the colony uses a PR-based workflow and a merge just happened, attempt to collect the branch's midden entries:
+
+Run using the Bash tool with description "Collecting branch midden entries...":
+```bash
+# Check if there's a recently merged branch to collect from
+# The merge info comes from git log or COLONY_STATE context
+last_merge_branch="${last_merged_branch:-}"
+last_merge_sha="${last_merge_sha:-}"
+
+if [[ -n "$last_merge_branch" && -n "$last_merge_sha" ]]; then
+  collect_result=$(bash .aether/aether-utils.sh midden-collect \
+    --branch "$last_merge_branch" --merge-sha "$last_merge_sha" \
+    2>/dev/null || echo '{"ok":false}')
+  collect_ok=$(echo "$collect_result" | jq -r '.ok // false' 2>/dev/null)
+  if [[ "$collect_ok" == "true" ]]; then
+    collect_status=$(echo "$collect_result" | jq -r '.result.status // "unknown"' 2>/dev/null)
+    if [[ "$collect_status" == "collected" ]]; then
+      new_entries=$(echo "$collect_result" | jq -r '.result.entries_collected // 0' 2>/dev/null)
+      echo "Midden: collected $new_entries failure entries from branch $last_merge_branch"
+    fi
+  fi
+fi
+```
+
+This step is NON-BLOCKING -- continue proceeds regardless of collection outcome. If `last_merge_branch` and `last_merge_sha` are not set (e.g., no recent merge), this step is silently skipped.