ultimate-pi 0.11.0 → 0.13.0

package/.pi/prompts/harness-setup.md

@@ -156,7 +156,7 @@ bash "$UP_PKG/.pi/scripts/harness-cli-verify.sh"
 
 **Required (script must exit 0):** scrapling + harness-web smoke, ctx7, biome, ast-grep (`sg`), sentrux (when harness manifest present).
 
-**Warnings allowed:** gh (if not authenticated), agent-browser (if OS libs need manual `sudo apt-get install`), ck (empty corpus on tiny repos).
+**Warnings allowed:** gh (if not authenticated), agent-browser (if OS libs need manual `sudo apt-get install`), cocoindex-code (empty corpus on tiny repos; first `[full]` install downloads local embedding model).
 
 If the script reports **agent-browser shared library errors** on Linux/WSL, run the fix it prints, then re-verify:
 
@@ -232,20 +232,24 @@ if [ ! -f .pi/harness/browser.json ]; then
 fi
 ```
 
-### 2.4 — ck-search (Semantic Code Search)
+### 2.4 — CocoIndex Code (semantic code search)
+
+Offline semantic search via [CocoIndex Code](https://cocoindex.io/cocoindex-code/) (`ccc`). **CLI only** — do not register `ccc mcp` in harness setup (MCP defaults to refresh-before-search and is slower).
 
 ```bash
-if ! command -v ck &>/dev/null || [ "$FORCE" = "true" ]; then
-	npm install -g @beaconbay/ck-search
-fi
+command -v uv &>/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh
+export PATH="$HOME/.local/bin:$PATH"
+uv tool install 'cocoindex-code[full]'   # or: pipx install 'cocoindex-code[full]'
+bash "$UP_PKG/.pi/scripts/harness-cocoindex-bootstrap.sh"
+# Rebuild index: bash "$UP_PKG/.pi/scripts/harness-cocoindex-bootstrap.sh" --force
+ccc status
 ```
 
-Verify: `ck --version`
+**Indexing before scouts is automatic:** `harness-subagents` runs incremental `ccc index` before plan/execute subagent batches. Agents use `ccc search` only.
 
-Register as MCP server (if Claude MCP available):
-```bash
-claude mcp list 2>/dev/null && claude mcp add ck-search -s user -- ck --serve || echo "MCP not available — ck will be used as CLI only"
-```
+**First install:** `[full]` pulls sentence-transformers + torch (~hundreds of MB–1GB+). Run bootstrap once; do not run `ccc index` inside scouts.
+
+Verify: `ccc doctor` and `ccc search --limit 3 "export function"` (no `--refresh`).
 
 ### 2.5 — biome (Lint + Format Gate)
 
@@ -345,7 +349,7 @@ Verify each package:
 |---------|---------|-------|
 | `@posthog/pi` | Analytics event capture | F0 |
 | `pi-lean-ctx` | Context runtime (read/bash/find/grep/MCP bridge) | F0 |
-| `harness-subagents` (bundled extension) | L4 sub-agent spawn, blackboard, package agents | P16 |
+| `harness-subagents` (bundled extension) | L4 `subagent` tool, subprocess spawns, package agents | P16 |
 | Vendored `pi-vcc` (`vendor/pi-vcc`, `.pi/extensions/ultimate-pi-vcc.ts`) | VCC compaction / `vcc_recall` — env-only: `HARNESS_VCC_COMPACTION` (default on), `HARNESS_VCC_DEBUG` | Shipped |
 | `pi-model-router` | Vendored (`vendor/`); activates after `.pi/model-router.json` exists | F0 |
 
@@ -383,11 +387,11 @@ Manual override: **`/router profile auto`** anytime after reload if they changed
 
 ## Step 3.6 — Harness agents (package-resolved)
 
-`harness-subagents` loads agents from the installed **`ultimate-pi`** package (`$UP_PKG/.pi/agents/**`) with namespaced ids (`harness/planner`, `pi-pi/agent-expert`). **Do not copy** agents into the project unless you want a deliberate override.
+`harness-subagents` loads agents from the installed **`ultimate-pi`** package (`$UP_PKG/.pi/agents/**`) with namespaced ids (`harness/executor`, `harness/planning/scout-graphify`, `pi-pi/agent-expert`). **Do not copy** agents into the project unless you want a deliberate override.
 
 **Slash commands are orchestrators:** `/harness-plan`, `/harness-run`, etc. spawn `harness/*` agents via the `Agent` tool — bootstrap stays **script-first**; only optionally spawn `harness/sentrux-bootstrap` for Sentrux (see Step 4.2).
 
-Optional per-repo overrides: place `.md` files at the **same relative path** (e.g. `.pi/agents/harness/planning/scout-graphify.md` overrides the package scout). Deprecated: `harness/planner.md` — use `harness/planning/` agents instead.
+Optional per-repo overrides: place `.md` files at the **same relative path** (e.g. `.pi/agents/harness/planning/scout-graphify.md` overrides the package scout).
 
 Verify manifest drift after `pi update ultimate-pi`:
 
@@ -478,16 +482,25 @@ Template keys (placeholders — user fills secrets): `HARNESS_TELEMETRY_ENABLED`
 
 ### 4.1 — .gitignore Entries
 
-Ensure `.gitignore` contains:
+Ensure `.gitignore` contains harness runtime entries (see repo root `.gitignore` — **do not** ignore `.pi/harness/specs/`; JSON schemas are shared contracts):
+
 ```
 .env
 .web/
 .searxng/
 .raw/
 .vault-meta/
-.pi/harness/critics/
+.pi/harness/active-run.json
+.pi/harness/release-readiness-report.md
 .pi/harness/plans/
-.pi/harness/specs/
+.pi/harness/critics/
+.pi/harness/runs/**
+!.pi/harness/runs/README.md
+.pi/harness/incidents/*
+!.pi/harness/incidents/README.md
+.pi/harness/debates/*
+!.pi/harness/debates/README.md
+.pi/harness/router/proposals/*
 
 # Model router config (user-specific — generated from env)
 .pi/model-router.json
@@ -657,7 +670,7 @@ Output summary table:
 | scrapling / harness-web | ✓/✗ | Auth: yes/no |
 | ctx7 | ✓/✗ | Login: yes/no |
 | agent-browser | ✓/✗ | Config: .pi/harness/browser.json |
-| ck-search | ✓/✗ | MCP: registered/CLI-only |
+| cocoindex-code | ✓/✗ | `ccc status`; index auto-refreshed before harness scouts |
 | biome | ✓/✗ | Project config: found/default |
 | ast-grep | ✓/✗ | AST-aware code search (`sg`)
 | gh CLI | ✓/✗ | Auth: yes/no |

package/.pi/prompts/harness-trace.md

@@ -20,10 +20,10 @@ Happy path: omit `--run`.
 2. Spawn:
 
 ```
-Agent({ subagent_type: "harness/trace-librarian", prompt: "…" })
+subagent({ agentScope: "both", agent: "harness/trace-librarian", task: "…" })
 ```
 
-3. `get_subagent_result` — present timeline and artifact index to user.
+3. Present timeline and artifact index from tool result to user.
 
 ## Completion
 

package/.pi/scripts/README.md

@@ -21,6 +21,7 @@ From **Typescript extensions**, use `resolveHarnessScript()` / `getHarnessPackag
 | Action | Command |
 |--------|---------|
 | Graphify bootstrap | `bash "$UP_PKG/.pi/scripts/harness-graphify-bootstrap.sh"` |
+| CocoIndex Code bootstrap | `bash "$UP_PKG/.pi/scripts/harness-cocoindex-bootstrap.sh"` |
 | CLI tool install + smoke tests | `bash "$UP_PKG/.pi/scripts/harness-cli-verify.sh"` |
 | Deterministic harness checks | `node "$UP_PKG/.pi/scripts/harness-verify.mjs"` |
 | Sentrux rules bootstrap (harness-setup) | `node "$UP_PKG/.pi/scripts/harness-sentrux-bootstrap.mjs"` |

package/.pi/scripts/harness-agents-manifest.mjs

@@ -14,7 +14,7 @@ import {
 	isSafeAgentId,
 	sha256Content,
 	walkAgentsDir,
-} from "../../test/harness-subagents-loader.core.mjs";
+} from "../lib/harness-agent-discovery.mjs";
 
 const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", "..");
 const MANIFEST_PATH = join(ROOT, ".pi", "harness", "agents.manifest.json");

package/.pi/scripts/harness-cli-verify.sh

@@ -231,22 +231,32 @@ verify_ctx7() {
 	fi
 }
 
-verify_ck() {
-	log "[ck-search]"
-	npm_global_install "@beaconbay/ck-search" "ck" || { fail "ck-search npm install"; return; }
-	if ! ck --version &>/dev/null; then
-		fail "ck --version failed"
+verify_cocoindex() {
+	log "[cocoindex-code]"
+	local _bootstrap
+	_bootstrap="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)/harness-cocoindex-bootstrap.sh"
+	if [ ! -f "$_bootstrap" ]; then
+		fail "harness-cocoindex-bootstrap.sh missing"
 		return
 	fi
-	# Fast grep-mode smoke (no embedding model download)
-	local ck_target="."
-	[ -d .pi ] && ck_target=".pi"
-	if ck -l 1 "export" "$ck_target" 2>/dev/null | head -1 | grep -q .; then
-		pass "ck $(ck --version 2>/dev/null | head -1)"
-	elif ck --status "$ck_target" 2>/dev/null | head -1 | grep -q .; then
-		pass "ck $(ck --version 2>/dev/null | head -1) (index status ok)"
+	if [ "$FORCE" = true ]; then
+		bash "$_bootstrap" --force || { fail "cocoindex bootstrap --force"; return; }
+	else
+		bash "$_bootstrap" || { fail "cocoindex bootstrap"; return; }
+	fi
+	if ! command -v ccc &>/dev/null; then
+		fail "ccc not on PATH after bootstrap"
+		return
+	fi
+	if ! ccc doctor &>/dev/null; then
+		warn "ccc doctor reported issues (daemon/model/SQLite)"
+	fi
+	if ccc search --limit 3 "export function" 2>/dev/null | head -1 | grep -qE '\.(ts|js|py|md)|\[|score|path'; then
+		pass "cocoindex-code (ccc search smoke)"
+	elif ccc status 2>/dev/null | head -1 | grep -q .; then
+		pass "cocoindex-code (index present; search empty on tiny corpus)"
 	else
-		warn "ck installed but smoke search empty"
+		warn "cocoindex installed but smoke search empty — first [full] install downloads local embedding model"
 	fi
 }
 
@@ -335,7 +345,7 @@ log ""
 verify_scrapling
 verify_ctx7
 verify_agent_browser
-verify_ck
+verify_cocoindex
 verify_biome
 verify_sg
 verify_gh

package/.pi/scripts/harness-cocoindex-bootstrap.sh

@@ -0,0 +1,182 @@
+#!/usr/bin/env bash
+# harness-cocoindex-bootstrap — install CocoIndex Code (ccc), seed settings, build index.
+# Non-interactive (no `ccc init` prompts). Used by harness-cli-verify and /harness-setup.
+
+set -euo pipefail
+
+FORCE=false
+for arg in "$@"; do
+	case "$arg" in
+	--force) FORCE=true ;;
+	-h | --help)
+		echo "Usage: $0 [--force]"
+		echo "  --force   rebuild index even when cocoindex.db exists"
+		exit 0
+		;;
+	*)
+		echo "Unknown argument: $arg" >&2
+		exit 2
+		;;
+	esac
+done
+
+export PATH="${HOME}/.local/bin:${PATH}"
+
+log() { printf '%s\n' "$*"; }
+die() { printf 'error: %s\n' "$*" >&2; exit 1; }
+
+ROOT="$(pwd)"
+
+if ! python3 --version 2>/dev/null | grep -qE 'Python 3\.(1[0-9]|[2-9][0-9])'; then
+	die "Python 3.10+ required (got: $(python3 --version 2>/dev/null || echo missing))"
+fi
+log "✓ Python 3.10+"
+
+cocoindex_installed() {
+	command -v ccc &>/dev/null && return 0
+	command -v uv &>/dev/null && uv tool list 2>/dev/null | grep -qE '(^|[[:space:]])cocoindex-code([[:space:]]|$)' && return 0
+	return 1
+}
+
+install_cocoindex() {
+	if command -v uv &>/dev/null; then
+		log "Installing cocoindex-code[full] via uv tool..."
+		uv tool install 'cocoindex-code[full]'
+	elif command -v pipx &>/dev/null; then
+		log "Installing cocoindex-code[full] via pipx..."
+		pipx install 'cocoindex-code[full]'
+	else
+		die "Need uv or pipx to install cocoindex-code[full]"
+	fi
+	export PATH="${HOME}/.local/bin:${PATH}"
+	command -v ccc &>/dev/null || die "ccc not on PATH after install"
+}
+
+if ! cocoindex_installed; then
+	install_cocoindex
+fi
+command -v ccc &>/dev/null || die "ccc CLI not found"
+log "✓ ccc ($(command -v ccc))"
+
+CCC_BIN="$(command -v ccc)"
+CCC_PYTHON=""
+if [ -f "$CCC_BIN" ]; then
+	CCC_PYTHON="$(sed -n '1s/^#!//p' "$CCC_BIN" | tr -d '\r')"
+fi
+if [ -z "$CCC_PYTHON" ] || [ ! -x "$CCC_PYTHON" ]; then
+	CCC_VENV="$(cd "$(dirname "$CCC_BIN")/.." && pwd)"
+	CCC_PYTHON="$CCC_VENV/bin/python"
+fi
+if [ ! -x "$CCC_PYTHON" ]; then
+	CCC_PYTHON="$(command -v python3)"
+fi
+
+# Seed global + project settings and merge harness exclude_patterns (non-interactive).
+"$CCC_PYTHON" - <<'PY'
+from __future__ import annotations
+
+import os
+import sys
+from pathlib import Path
+
+try:
+    import yaml
+except ImportError:
+    print("error: PyYAML required for harness cocoindex bootstrap", file=sys.stderr)
+    sys.exit(1)
+
+ROOT = Path(os.environ.get("HARNESS_COCOINDEX_ROOT", ".")).resolve()
+SETTINGS_DIR = ROOT / ".cocoindex_code"
+PROJECT_SETTINGS = SETTINGS_DIR / "settings.yml"
+USER_DIR = Path.home() / ".cocoindex_code"
+USER_SETTINGS = USER_DIR / "global_settings.yml"
+
+HARNESS_EXCLUDES = [
+    "graphify-out/**",
+    "raw/**",
+    ".raw/**",
+    ".pi/wiki-search/**",
+    "vendor/**",
+    "node_modules/**",
+    "**/graph.html",
+    "**/*.jsonl",
+    ".cursor/**",
+]
+
+DEFAULT_GLOBAL = {
+    "embedding": {
+        "provider": "sentence-transformers",
+        "model": "Snowflake/snowflake-arctic-embed-xs",
+    }
+}
+
+
+def load_yaml(path: Path) -> dict:
+    if not path.is_file():
+        return {}
+    data = yaml.safe_load(path.read_text(encoding="utf-8"))
+    return data if isinstance(data, dict) else {}
+
+
+def save_yaml(path: Path, data: dict) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(yaml.safe_dump(data, default_flow_style=False), encoding="utf-8")
+
+
+# Global settings (offline embeddings — no LiteLLM prompt).
+if not USER_SETTINGS.is_file():
+    save_yaml(USER_SETTINGS, DEFAULT_GLOBAL)
+    print(f"Created {USER_SETTINGS}")
+
+# Project settings — minimal file with merged excludes only.
+project = load_yaml(PROJECT_SETTINGS)
+excludes = list(project.get("exclude_patterns") or [])
+seen = set(excludes)
+for pat in HARNESS_EXCLUDES:
+    if pat not in seen:
+        excludes.append(pat)
+        seen.add(pat)
+project["exclude_patterns"] = excludes
+save_yaml(PROJECT_SETTINGS, project)
+print(f"Ensured project settings: {PROJECT_SETTINGS}")
+
+# .gitignore entry (mirror ccc init behavior).
+gitignore = ROOT / ".gitignore"
+entry = "/.cocoindex_code/"
+comment = "# CocoIndex Code (ccc)"
+if (ROOT / ".git").is_dir() and gitignore.is_file():
+    lines = gitignore.read_text(encoding="utf-8").splitlines()
+    if entry not in lines:
+        content = gitignore.read_text(encoding="utf-8")
+        if content and not content.endswith("\n"):
+            content += "\n"
+        content += f"{comment}\n{entry}\n"
+        gitignore.write_text(content, encoding="utf-8")
+elif (ROOT / ".git").is_dir():
+    gitignore.write_text(f"{comment}\n{entry}\n", encoding="utf-8")
+PY
+
+NEED_INDEX=true
+if [ "$FORCE" = false ] && [ -f ".cocoindex_code/cocoindex.db" ]; then
+	NEED_INDEX=false
+	log "✓ Existing cocoindex.db (use --force to rebuild)"
+fi
+
+if [ "$NEED_INDEX" = true ] || [ "$FORCE" = true ]; then
+	log "Building semantic code index (ccc index)..."
+	ccc index
+fi
+
+if ccc doctor 2>/dev/null; then
+	log "✓ ccc doctor"
+else
+	log "! ccc doctor reported issues (see output above)"
+fi
+
+if ccc status 2>/dev/null | head -5; then
+	log "✓ ccc status"
+else
+	log "! ccc status unavailable"
+fi
+
+log "CocoIndex output: .cocoindex_code/"

package/.pi/scripts/harness-verify.mjs

@@ -202,32 +202,51 @@ async function main() {
 	if (!(await fileExists(runCtxLib))) fail("missing lib/harness-run-context.ts");
 	ok("lib/harness-run-context.ts");
 
-	const vendoredIndex = join(
+	const pkgJson = JSON.parse(
+		await readFile(join(ROOT, "package.json"), "utf-8"),
+	);
+	if (!pkgJson.files?.includes("vendor/pi-subagents")) {
+		fail(
+			'package.json "files" must include vendor/pi-subagents (npm publish ships subagents vendor)',
+		);
+	}
+	ok('package.json files includes vendor/pi-subagents');
+
+	const subagentsVendor = join(
+		ROOT,
+		"vendor",
+		"pi-subagents",
+		"src",
+		"subagents.ts",
+	);
+	if (!(await fileExists(subagentsVendor))) {
+		fail("missing vendor/pi-subagents/src/subagents.ts");
+	}
+	const bridgePath = join(
 		ROOT,
 		".pi",
 		"extensions",
 		"lib",
-		"harness-subagents",
-		"vendored",
-		"index.ts",
+		"harness-subagents-bridge.ts",
 	);
-	const vendoredSrc = await readFile(vendoredIndex, "utf-8");
-	const runCtxImport = vendoredSrc.match(
-		/from ["']([^"']*harness-run-context\.js)["']/,
-	);
-	if (!runCtxImport) {
-		fail("vendored/index.ts must import harness-run-context.js");
+	if (!(await fileExists(bridgePath))) {
+		fail("missing harness-subagents-bridge.ts");
 	}
-	const runCtxImportPath = resolve(
-		dirname(vendoredIndex),
-		runCtxImport[1].replace(/\.js$/, ".ts"),
-	);
-	if (runCtxImportPath !== runCtxLib) {
-		fail(
-			`vendored/index.ts harness-run-context import resolves to ${runCtxImportPath}, expected ${runCtxLib}`,
-		);
+	const bridgeSrc = await readFile(bridgePath, "utf-8");
+	if (!bridgeSrc.includes("precheckHarnessSubagentSpawn")) {
+		fail("harness-subagents-bridge must run precheckHarnessSubagentSpawn");
+	}
+	if (!bridgeSrc.includes("packageRoot")) {
+		fail("harness-subagents-bridge must pass packageRoot for agent discovery");
+	}
+	const subagentsSrc = await readFile(subagentsVendor, "utf-8");
+	if (!subagentsSrc.includes("discoverAgents")) {
+		fail("vendor subagents.ts must implement discoverAgents");
+	}
+	if (!subagentsSrc.includes("packageRoot")) {
+		fail("vendor subagents.ts must pass packageRoot into discovery");
 	}
-	ok("vendored/index.ts harness-run-context import path");
+	ok("vendor pi-subagents + harness bridge");
 
 	const policyGateSrc = await readFile(
 		join(ROOT, ".pi", "extensions", "policy-gate.ts"),