@pi-unipi/unipi 3.0.0-alpha.2 → 3.0.0-alpha.21

package/packages/compactor/skills/compactor-detail/SKILL.md

@@ -1,116 +0,0 @@
----
-name: compactor-detail
-description: Full compactor reference — tool parameters, anti-patterns, sandbox languages, context budget, workflows.
----
-
-# Compactor — Full Reference
-
-## Tool Parameter Reference
-
-### compact
-```
-compact(dryRun?: boolean)
-```
-Agent-facing compaction request. Use `dryRun: true` to preview estimated message reduction without compacting.
-
-For user-triggered immediate compaction, prefer the slash command `/unipi:lossless-compact`.
-
-### session_recall
-```
-session_recall(query: string, mode?: "bm25" | "regex", limit?: number, offset?: number, expand?: boolean)
-```
-Search current session history. BM25 is default; regex is fallback. The recall path prefers Pi's append-only session branch so it can find messages that are no longer present in live LLM context after compaction.
-
-Deprecated alias: `vcc_recall`.
-
-### sandbox
-```
-sandbox(language: string, code: string, timeout?: number)
-```
-Run code in a sandboxed environment. Only stdout enters context. 100MB output cap. 30s default timeout.
-
-Deprecated alias: `ctx_execute`.
-
-### sandbox_file
-```
-sandbox_file(language: string, path: string, timeout?: number)
-```
-Execute a file. File content is injected as `FILE_CONTENT`.
-
-Deprecated alias: `ctx_execute_file`.
-
-### sandbox_batch
-```
-sandbox_batch(items: Array<{ type: "execute", language: string, code: string, timeout?: number }>)
-```
-Run multiple sandbox executions and return a single combined result.
-
-Deprecated alias: `ctx_batch_execute`.
-
-### compactor_stats
-```
-compactor_stats()
-```
-Dashboard: session events, compactions, tokens saved, compression ratio, sandbox runs, and search queries.
-
-Deprecated alias: `ctx_stats`.
-
-### compactor_doctor
-```
-compactor_doctor()
-```
-Diagnostics: config file, session DB, and available runtimes (node, python3, bash).
-
-Deprecated alias: `ctx_doctor`.
-
-### context_budget
-```
-context_budget()
-```
-Estimate remaining context tokens and percent full. Uses Pi's live context usage when available and includes percentage auto-compaction guidance when enabled.
-
-## Sandbox Language Reference
-
-| Language | Binaries | Timeout | Notes |
-|----------|----------|---------|-------|
-| javascript | Bun/node | 30s | Default: Bun if available |
-| typescript | Bun/node | 30s | Compiled via Bun transform |
-| python | python3 | 30s | - |
-| shell | bash | 30s | Pipelines supported |
-| ruby | ruby | 30s | - |
-| go | go | 30s | go run |
-| rust | rustc+cargo | 30s | cargo script or rustc |
-| php | php | 30s | - |
-| perl | perl | 30s | - |
-| r | Rscript | 30s | - |
-| elixir | elixir | 30s | - |
-
-## Anti-Patterns
-
-1. **Don't compact in a tight loop.** Use `context_budget` first and compact at natural break points.
-2. **Don't use `session_recall` for project-wide code search.** It searches the conversation/session only — use `read`/`bash`(rg) for files.
-3. **Don't use `sandbox` for file operations.** Use normal file tools for reads/writes; sandbox is for computation or quick scripts.
-4. **Don't use empty or vague recall queries.** Search for specific filenames, issue numbers, commands, or decisions.
-5. **Don't compact mid-thought if avoidable.** Finish the current step, then compact.
-
-## Workflow Patterns
-
-### Before Complex Work
-1. `context_budget` — check pressure.
-2. `session_recall("goal files decisions")` — recover relevant prior context.
-3. `compact(dryRun: true)` — preview if context is tight.
-4. `/unipi:lossless-compact` or `compact` at a clean boundary.
-
-### Diagnose Compactor Health
-1. `compactor_doctor` — check config, DB, runtimes.
-2. Fix reported issues.
-3. `compactor_stats` — verify counters and savings.
-
-### Long Session Recovery
-1. `session_recall(query)` — find older details hidden by compaction.
-2. `context_budget` — decide whether another compaction is needed.
-3. `compactor_stats` — confirm compactions/savings are recorded.
-
-### Project Search
-Compactor does not own project content indexing. For code search use `bash` with rg.
-1. `bash` with `rg` for code search

package/packages/compactor/skills/compactor-doctor/SKILL.md

@@ -1,71 +0,0 @@
----
-name: compactor-doctor
-description: Diagnostics — validate config, session DB, runtimes, and troubleshoot compactor issues.
----
-
-# Compactor Doctor
-
-Run diagnostics and troubleshoot compactor issues.
-
-## Commands and Tools
-
-- `/unipi:compact-doctor` — user-facing diagnostics
-- `compactor_doctor` tool — agent-callable diagnostics
-- `ctx_doctor` tool — deprecated alias
-
-## Checks Performed
-
-| Check | What It Validates |
-|-------|-------------------|
-| **Config file** | `~/.unipi/config/compactor/config.json` exists or defaults can be used |
-| **Session DB** | SQLite connection works and session schema is usable |
-| **Runtime: node** | Node.js available for sandbox/runtime helpers |
-| **Runtime: python3** | Python 3 available for Python sandbox execution |
-| **Runtime: bash** | Bash available for shell sandbox execution |
-
-## Status Icons
-
-- ✅ **pass** — check succeeded
-- ⚠️ **warn** — non-critical issue, such as an optional runtime missing
-- ❌ **fail** — critical issue; a feature may not work
-
-## Common Issues
-
-### "Config file: Using defaults"
-
-- Normal on first run.
-- Config is created by the extension or on settings save.
-- Fix: open `/unipi:settings (Compactor)`, adjust if desired, then save.
-
-### "Session DB: Connection failed"
-
-- SQLite or `better-sqlite3` may be unavailable.
-- Compaction can still degrade gracefully, but stats/recall may be limited.
-- Fix: verify dependencies and run package install/build steps.
-
-### "Runtime: python3 Not found"
-
-- Python is not installed or not in `PATH`.
-- Only needed for Python sandbox execution.
-- Fix: install Python 3 or ignore if not needed.
-
-## Manual Diagnostics
-
-### Check config
-```bash
-cat ~/.unipi/config/compactor/config.json
-```
-
-### Check DB files
-```bash
-ls -la ~/.unipi/db/compactor/
-```
-
-### Check runtimes
-```bash
-node --version
-python3 --version
-bash --version
-```
-
-Project content indexing is not part of compactor.

package/packages/compactor/skills/compactor-stats/SKILL.md

@@ -1,51 +0,0 @@
----
-name: compactor-stats
-description: Stats display — context savings, session metrics, compactions, sandbox and recall/search counters.
----
-
-# Compactor Stats
-
-Display and interpret compactor statistics.
-
-## Commands and Tools
-
-- `/unipi:compact-stats` — user-facing dashboard
-- `compactor_stats` tool — agent-callable stats
-- `ctx_stats` tool — deprecated alias
-
-## Metrics Explained
-
-| Metric | Description |
-|--------|-------------|
-| **Session events** | Events tracked for the current session in the compactor session DB |
-| **Compactions** | Number of compaction events recorded |
-| **Tokens saved** | Estimated tokens freed by compaction, using Pi token counts when available and DB fallbacks otherwise |
-| **Compression ratio** | Approximate before/kept ratio from stored compaction stats |
-| **Sandbox runs** | Agent sandbox executions recorded in runtime counters/DB |
-| **Search queries** | Session recall/search queries recorded in runtime counters/DB |
-
-## Health Indicators
-
-- **Compactions > 0** on long sessions = compaction is happening.
-- **Tokens saved > 0** after compaction = savings were recorded.
-- **Sandbox runs** increasing = sandbox tool usage is tracked.
-- **Search queries** increasing = recall/search activity is tracked.
-
-## Reading Stats
-
-```text
-📊 Compactor Stats
-Session events: 42
-Compactions: 3
-Tokens saved: 15000
-Sandbox runs: 7
-Search queries: 15
-```
-
-This means:
-
-- 42 events are tracked for the session.
-- 3 compactions saved roughly 15K tokens.
-- 7 sandbox executions and 15 recall/search queries were recorded.
-
-Project content indexing stats are not owned by compactor; use `read`/`bash` (rg) for file search.

package/packages/compactor/src/compaction/build-sections.ts

@@ -1,76 +0,0 @@
-/**
- * Stage 3: Build Sections — Goals, Files, Commits, Blockers, Preferences
- */
-
-import type { NormalizedBlock } from "../types.js";
-import { clip, clipSentence, firstLine, nonEmptyLines } from "./content.js";
-import { extractGoals } from "./extract/goals.js";
-import { extractFiles } from "./extract/files.js";
-import { extractPreferences, dedupPreferencesAgainstGoals } from "./extract/preferences.js";
-import { extractCommits, formatCommits } from "./extract/commits.js";
-import { buildBriefSections, stringifyBrief } from "./brief.js";
-import type { SectionData } from "../types.js";
-export type { SectionData } from "../types.js";
-
-const BLOCKER_RE =
-  /\b(fail(ed|s|ure|ing)?|broken|cannot|can't|won't work|does not work|doesn't work|still (broken|failing|wrong)|blocked|blocker|not (fixed|resolved|working)|crash(es|ed|ing)?)\b/i;
-
-const extractOutstandingContext = (blocks: NormalizedBlock[]): string[] => {
-  const items: string[] = [];
-  const tail = blocks.slice(-20);
-
-  for (const b of tail) {
-    if (b.kind === "tool_result" && b.isError) {
-      items.push(`[${b.name}] ${firstLine(b.text, 150)}`);
-      continue;
-    }
-
-    if (b.kind === "assistant" || b.kind === "user") {
-      for (const line of nonEmptyLines(b.text)) {
-        if (!BLOCKER_RE.test(line)) continue;
-        if (line.length < 15) continue;
-        if (/^\s*[-*+>]\s/.test(line)) continue;
-        if (/^\s*\(/.test(line)) continue;
-        if (!/^\s*["'`*_]?[A-Z`]/.test(line)) continue;
-        const clipped = b.kind === "user" ? `[user] ${clipSentence(line, 150)}` : clipSentence(line, 150);
-        if (!items.includes(clipped)) items.push(clipped);
-        break;
-      }
-    }
-  }
-
-  return items.slice(0, 5);
-};
-
-const formatFileActivity = (blocks: NormalizedBlock[]): string[] => {
-  const act = extractFiles(blocks);
-  for (const p of act.modified) act.created.delete(p);
-  const lines: string[] = [];
-  const cap = (set: Set<string>, limit: number) => {
-    const arr = [...set];
-    if (arr.length <= limit) return arr.join(", ");
-    return arr.slice(0, limit).join(", ") + ` (+${arr.length - limit} more)`;
-  };
-  if (act.modified.size > 0) lines.push(`Modified: ${cap(act.modified, 10)}`);
-  if (act.created.size > 0) lines.push(`Created: ${cap(act.created, 10)}`);
-  if (act.read.size > 0) lines.push(`Read: ${cap(act.read, 10)}`);
-  return lines;
-};
-
-export const buildSections = (input: { blocks: NormalizedBlock[]; briefBlocks?: NormalizedBlock[] }): SectionData => {
-  const { blocks } = input;
-  const briefSections = buildBriefSections(input.briefBlocks ?? blocks);
-  const sessionGoal = extractGoals(blocks);
-  const userPreferences = dedupPreferencesAgainstGoals(
-    extractPreferences(blocks),
-    sessionGoal,
-  );
-  return {
-    sessionGoal,
-    outstandingContext: extractOutstandingContext(blocks),
-    filesAndChanges: formatFileActivity(blocks),
-    commits: formatCommits(extractCommits(blocks)),
-    userPreferences,
-    briefTranscript: stringifyBrief(briefSections),
-  };
-};

package/packages/compactor/src/compaction/extract/goals.ts

@@ -1,36 +0,0 @@
-/**
- * Goal extraction — regex-based, scope changes, task verbs
- */
-
-import type { NormalizedBlock } from "../../types.js";
-import { nonEmptyLines } from "../content.js";
-
-const SCOPE_CHANGE_RE = /\b(scope|focus|switch|move|now|instead|rather than|change|shift|pivot|let's|let us)\b/i;
-const TASK_VERB_RE = /\b(add|create|build|implement|fix|refactor|update|remove|delete|rename|extract|merge|split|convert|migrate|optimize|test|write|generate|setup|configure|install|deploy|release|publish|document|review|audit|analyze|debug|resolve|handle|support|enable|disable|integrate|connect|sync|import|export|validate|verify|check|ensure|make|set|get|put|post|patch|delete)\b/i;
-const GOAL_PREFIX_RE = /^(?:Goal|Objective|Task|Plan|Target|Aim|Intent|Purpose|Mission|Action item|To-do|TODO|FIXME|NOTE|IDEA|HACK|BUG|FEATURE|STORY|EPIC):?\s*/i;
-
-export function extractGoals(blocks: NormalizedBlock[]): string[] {
-  const goals: string[] = [];
-  const seen = new Set<string>();
-
-  for (const b of blocks) {
-    if (b.kind !== "user" && b.kind !== "assistant") continue;
-    for (const line of nonEmptyLines(b.text)) {
-      let candidate = line;
-      // Strip goal prefix
-      candidate = candidate.replace(GOAL_PREFIX_RE, "").trim();
-      if (candidate.length < 10) continue;
-      // Must contain a task verb or scope change
-      if (!TASK_VERB_RE.test(candidate) && !SCOPE_CHANGE_RE.test(candidate)) continue;
-      // Deduplicate
-      const key = candidate.toLowerCase().slice(0, 80);
-      if (seen.has(key)) continue;
-      seen.add(key);
-      goals.push(candidate);
-      if (goals.length >= 6) break;
-    }
-    if (goals.length >= 6) break;
-  }
-
-  return goals;
-}

package/packages/compactor/src/compaction/extract/preferences.ts

@@ -1,40 +0,0 @@
-/**
- * User preference extraction — track explicit preferences and constraints
- */
-
-import type { NormalizedBlock } from "../../types.js";
-import { nonEmptyLines } from "../content.js";
-
-const PREF_RE = /\b(prefer|preference|want|would like|should|must|need to|important|critical|avoid|don't|do not|never|always|only|make sure|ensure|remember|keep in mind|note that)\b/i;
-const PREF_PREFIX_RE = /^(?:Preference|Note|Remember|Keep in mind|Important|Critical|Constraint|Rule|Guideline|Style|Format):?\s*/i;
-
-export function extractPreferences(blocks: NormalizedBlock[]): string[] {
-  const prefs: string[] = [];
-  const seen = new Set<string>();
-
-  for (const b of blocks) {
-    if (b.kind !== "user") continue;
-    for (const line of nonEmptyLines(b.text)) {
-      let candidate = line.replace(PREF_PREFIX_RE, "").trim();
-      if (candidate.length < 10) continue;
-      if (!PREF_RE.test(candidate)) continue;
-      const key = candidate.toLowerCase().slice(0, 80);
-      if (seen.has(key)) continue;
-      seen.add(key);
-      prefs.push(candidate);
-      if (prefs.length >= 10) break;
-    }
-    if (prefs.length >= 10) break;
-  }
-
-  return prefs;
-}
-
-/** Remove preferences that overlap with goals */
-export function dedupPreferencesAgainstGoals(prefs: string[], goals: string[]): string[] {
-  const goalSet = new Set(goals.map((g) => g.toLowerCase().slice(0, 60)));
-  return prefs.filter((p) => {
-    const key = p.toLowerCase().slice(0, 60);
-    return !goalSet.has(key);
-  });
-}

package/packages/compactor/src/compaction/format.ts

@@ -1,53 +0,0 @@
-/**
- * Stage 5: Format — Render sections with separators
- */
-
-import type { SectionData } from "../types.js";
-
-const section = (title: string, items: string[]): string => {
-  if (items.length === 0) return "";
-  const body = items.map((i) => `- ${i}`).join("\n");
-  return `[${title}]\n${body}`;
-};
-
-export const BRIEF_MAX_LINES = 120;
-
-export const capBrief = (text: string): string => {
-  const lines = text.split("\n");
-  if (lines.length <= BRIEF_MAX_LINES) return text;
-  const omitted = lines.length - BRIEF_MAX_LINES;
-  const kept = lines.slice(-BRIEF_MAX_LINES);
-  const firstHeader = kept.findIndex((l) => /^\[.+\]/.test(l));
-  const clean = firstHeader > 0 ? kept.slice(firstHeader) : kept;
-  return `...(${omitted} earlier lines omitted)\n\n${clean.join("\n")}`;
-};
-
-export const RECALL_NOTE =
-  "Use `vcc_recall` to search for prior work, decisions, and context from before this summary. " +
-  "Do not redo work already completed.";
-
-export interface FormatSummaryOptions {
-  capBriefTranscript?: boolean;
-}
-
-export const formatSummary = (data: SectionData, options: FormatSummaryOptions = {}): string => {
-  const capBriefTranscript = options.capBriefTranscript ?? true;
-  const headerParts = [
-    section("Session Goal", data.sessionGoal),
-    section("Files And Changes", data.filesAndChanges),
-    section("Commits", data.commits),
-    section("Outstanding Context", data.outstandingContext),
-    section("User Preferences", data.userPreferences),
-  ].filter(Boolean);
-
-  const parts: string[] = [];
-  if (headerParts.length > 0) {
-    parts.push(headerParts.join("\n\n"));
-  }
-  if (data.briefTranscript) {
-    parts.push(capBriefTranscript ? capBrief(data.briefTranscript) : data.briefTranscript);
-  }
-
-  if (parts.length === 0) return "";
-  return parts.join("\n\n---\n\n");
-};

package/packages/compactor/src/compaction/merge.ts

@@ -1,143 +0,0 @@
-/**
- * Stage 6: Merge — Merge with previous summary, dedup, rolling window
- */
-
-import { capBrief, BRIEF_MAX_LINES } from "./format.js";
-
-const HEADER_NAMES = ["Session Goal", "Files And Changes", "Commits", "Outstanding Context", "User Preferences"];
-const SEPARATOR = "\n\n---\n\n";
-
-const sectionOf = (text: string, header: string): string => {
-  const tag = `[${header}]`;
-  const start = text.indexOf(tag);
-  if (start < 0) return "";
-  const after = text.slice(start);
-  const nextSection = HEADER_NAMES
-    .filter((h) => h !== header)
-    .map((h) => after.indexOf(`[${h}]`))
-    .filter((n) => n > 0);
-  const nextSep = after.indexOf("\n\n---\n\n");
-  const candidates = [...nextSection, ...(nextSep > 0 ? [nextSep] : [])].sort((a, b) => a - b);
-  const end = candidates[0];
-  return (end ? after.slice(0, end) : after).trim();
-};
-
-const briefOf = (text: string): string => {
-  const idx = text.indexOf(SEPARATOR);
-  if (idx < 0) return "";
-  return text.slice(idx + SEPARATOR.length).trim();
-};
-
-const mergeHeaderSection = (header: string, prev: string, fresh: string): string => {
-  if (header === "Outstanding Context") return fresh;
-  if (!prev) return fresh;
-  if (!fresh) return prev;
-
-  if (header === "Files And Changes") {
-    return mergeFileLines(prev, fresh);
-  }
-
-  const isClean = (l: string) => l.startsWith("- ") && !l.includes("<skill") && !l.includes("</skill");
-  const prevLines = prev.split("\n").filter(isClean);
-  const freshLines = fresh.split("\n").filter(isClean);
-  const combined = [...new Set([...prevLines, ...freshLines])];
-  const CAP = header === "Session Goal" ? 8 : header === "Commits" ? 8 : 15;
-  const capped = combined.length > CAP ? combined.slice(-CAP) : combined;
-  if (capped.length === 0) return "";
-  return `[${header}]\n${capped.join("\n")}`;
-};
-
-const mergeFileLines = (prev: string, fresh: string): string => {
-  const categories = ["Modified", "Created", "Read"] as const;
-  const merged: Record<string, Set<string>> = {};
-  for (const cat of categories) merged[cat] = new Set();
-
-  for (const text of [prev, fresh]) {
-    for (const line of text.split("\n")) {
-      for (const cat of categories) {
-        const prefix = `- ${cat}: `;
-        if (!line.startsWith(prefix)) continue;
-        let rest = line.slice(prefix.length).replace(/\s*\(\+\d+ more\)\s*$/, "");
-        for (const p of rest.split(",")) {
-          const trimmed = p.trim();
-          if (trimmed) merged[cat].add(trimmed);
-        }
-      }
-    }
-  }
-
-  for (const p of merged.Modified) merged.Created.delete(p);
-
-  const cap = (set: Set<string>, limit: number) => {
-    const arr = [...set];
-    if (arr.length <= limit) return arr.join(", ");
-    return arr.slice(0, limit).join(", ") + ` (+${arr.length - limit} more)`;
-  };
-
-  const lines: string[] = [];
-  if (merged.Modified.size > 0) lines.push(`- Modified: ${cap(merged.Modified, 10)}`);
-  if (merged.Created.size > 0) lines.push(`- Created: ${cap(merged.Created, 10)}`);
-  if (merged.Read.size > 0) lines.push(`- Read: ${cap(merged.Read, 10)}`);
-  if (lines.length === 0) return "";
-  return `[Files And Changes]\n${lines.join("\n")}`;
-};
-
-const mergeBriefTranscript = (prev: string, fresh: string): string => {
-  if (!prev) return fresh;
-  if (!fresh) return prev;
-  return prev + "\n\n" + fresh;
-};
-
-const briefLineCount = (text: string): number =>
-  text ? text.split("\n").length : 0;
-
-const capBriefToLineBudget = (text: string, maxLines: number): string => {
-  if (!text || maxLines <= 0) return "";
-  const lines = text.split("\n");
-  if (lines.length <= maxLines) return text;
-  const kept = lines.slice(-maxLines);
-  const firstHeader = kept.findIndex((l) => /^\[.+\]/.test(l));
-  const clean = firstHeader > 0 ? kept.slice(firstHeader) : kept;
-  const omitted = lines.length - clean.length;
-  return `...(${omitted} earlier lines omitted)\n\n${clean.join("\n")}`;
-};
-
-/** Fresh brief gets the line budget; previous brief fills the remainder (pi-vcc parity) */
-export const mergeBriefTranscriptWithFreshBudget = (prev: string, fresh: string): string => {
-  if (!prev) return fresh;
-  if (!fresh) return capBrief(prev);
-  const freshLines = briefLineCount(fresh);
-  const remainingPrevLines = Math.max(0, BRIEF_MAX_LINES - freshLines);
-  const prevTail = capBriefToLineBudget(prev, remainingPrevLines);
-  return prevTail ? `${prevTail}\n\n${fresh}` : fresh;
-};
-
-export const mergePrevious = (
-  prev: string,
-  fresh: string,
-  options: { preserveFreshBrief?: boolean } = {},
-): string => {
-  const headers = HEADER_NAMES
-    .map((header) => {
-      const freshSec = sectionOf(fresh, header);
-      const prevSec = sectionOf(prev, header);
-      return mergeHeaderSection(header, prevSec, freshSec);
-    })
-    .filter(Boolean);
-
-  const prevBrief = briefOf(prev);
-  const freshBrief = briefOf(fresh);
-  const mergedBrief = options.preserveFreshBrief
-    ? mergeBriefTranscriptWithFreshBudget(prevBrief, freshBrief)
-    : mergeBriefTranscript(prevBrief, freshBrief);
-
-  const parts: string[] = [];
-  if (headers.length > 0) {
-    parts.push(headers.join("\n\n"));
-  }
-  if (mergedBrief) {
-    parts.push(options.preserveFreshBrief ? mergedBrief : capBrief(mergedBrief));
-  }
-
-  return parts.join(SEPARATOR);
-};