@pi-unipi/utility 0.1.1 → 0.2.1

package/README.md

@@ -1,51 +1,151 @@
 # @pi-unipi/utility
 
-Utility commands and tools for the Pi coding agent — part of the Unipi suite.
+Comprehensive utility suite for the Pi coding agent — part of the Unipi extension suite.
 
 ## Features
 
-### `/unipi:continue` Command
+### Commands
+
+| Command | Description |
+|---------|-------------|
+| `/unipi:continue` | Continue agent without polluting context |
+| `/unipi:reload` | Explain how to reload extensions |
+| `/unipi:status` | Show status of all unipi modules |
+| `/unipi:cleanup` | Clean stale DBs, temp files, old sessions |
+| `/unipi:env` | Show environment info (Node, Pi, OS, paths) |
+| `/unipi:doctor` | Run diagnostics across all modules |
+
+### Tools
+
+| Tool | Description |
+|------|-------------|
+| `ctx_batch` | Atomic batch execution with rollback support |
+| `ctx_env` | Environment inspection for debugging |
+
+### Modules (Programmatic API)
+
+| Module | Path | Description |
+|--------|------|-------------|
+| **ProcessLifecycle** | `lifecycle/process` | Parent PID polling, orphan detection, signal handlers, cleanup registry |
+| **cleanupStale** | `lifecycle/cleanup` | Stale DB/temp/session/cache cleanup with dry-run support |
+| **TTLCache** | `cache/ttl-cache` | Memory or SQLite-backed TTL cache with auto-expiration |
+| **AnalyticsCollector** | `analytics/collector` | Privacy-respecting event collection with daily rollup |
+| **runDiagnostics** | `diagnostics/engine` | Cross-module health checks with plugin architecture |
+| **detectCapabilities** | `display/capabilities` | Terminal feature detection (color, Nerd Font, unicode) |
+| **Width Utilities** | `display/width` | ANSI-aware clamp, wrap, collapse, pad, center |
+| **SettingsInspector** | `tui/settings-inspector` | Reusable settings overlay data model |
 
-Continue the agent from where it left off **without adding a user message** to the conversation transcript.
+## Installation
+
+```bash
+pi install npm:@pi-unipi/utility
+```
+
+Or install the full Unipi suite:
 
+```bash
+pi install npm:@pi-unipi/unipi
 ```
-/unipi:continue
+
+## Usage
+
+### Commands
+
+```
+/unipi:continue           # Resume agent cleanly
+/unipi:cleanup             # Clean stale files
+/unipi:cleanup --dry-run   # Preview what would be cleaned
+/unipi:env                 # Show environment
+/unipi:doctor              # Run diagnostics
 ```
 
-This sends a "steer" message that tells the agent to proceed to the next step. Unlike typing "continue" yourself, this doesn't pollute the context with an extra user message.
+### Batch Execution (Code)
 
-### `continue_task` Tool
+```typescript
+import { BatchBuilder } from "@pi-unipi/utility/tools/batch";
 
-The agent can call this tool programmatically when it finishes a step and needs to proceed to the next without waiting for user input.
+const report = await new BatchBuilder()
+  .addCommand("search", { query: "refactor" })
+  .addTool("memory_search", { query: "patterns" })
+  .withOptions({ failFast: true, commandTimeoutMs: 30000 })
+  .execute(myExecutor);
 
+if (!report.success) {
+  console.log("Failed:", report.results.find(r => !r.success)?.error);
+}
 ```
-continue_task()
+
+### TTL Cache (Code)
+
+```typescript
+import { TTLCache } from "@pi-unipi/utility/cache/ttl-cache";
+
+const cache = new TTLCache({ defaultTtlMs: 60000 });
+await cache.set("key", { data: "value" });
+const value = await cache.get("key");
 ```
 
-**When to use:** The agent has completed one step and should automatically proceed to the next.
+### Diagnostics (Code)
 
-## Installation
+```typescript
+import { runDiagnostics, formatDiagnosticsReport } from "@pi-unipi/utility/diagnostics/engine";
 
-```bash
-pi install npm:@pi-unipi/utility
+const report = await runDiagnostics();
+console.log(formatDiagnosticsReport(report));
 ```
 
-Or install the full Unipi suite:
+### Terminal Capabilities (Code)
 
-```bash
-pi install npm:@pi-unipi/unipi
+```typescript
+import { detectCapabilities, getIcon } from "@pi-unipi/utility/display/capabilities";
+
+const caps = detectCapabilities();
+console.log("Nerd Font:", caps.nerdFont);
+console.log(getIcon("󰘳", "[OK]")); // Uses Nerd Font if available
 ```
 
-## How It Works
+## Architecture
 
-Both the command and tool use Pi's `sendUserMessage` API with `deliverAs: "steer"` to inject a continuation prompt without creating a user message in the transcript. This keeps the conversation clean while allowing the agent to continue working.
+```
+packages/utility/src/
+├── index.ts              # Extension entry point
+├── commands.ts           # Command registration
+├── types.ts              # Shared types
+├── info-screen.ts        # Info-screen integration
+├── lifecycle/
+│   ├── process.ts        # Process lifecycle manager
+│   └── cleanup.ts        # Stale cleanup utility
+├── cache/
+│   └── ttl-cache.ts      # TTL cache (memory + SQLite)
+├── analytics/
+│   └── collector.ts      # Event collection + rollup
+├── diagnostics/
+│   └── engine.ts         # Health check engine
+├── display/
+│   ├── capabilities.ts   # Terminal detection
+│   └── width.ts          # Width utilities
+├── tui/
+│   └── settings-inspector.ts  # Settings overlay model
+└── tools/
+    ├── batch.ts          # Batch execution
+    └── env.ts            # Environment info
+```
 
-The default continue prompt is:
+## Privacy
 
-> Continue from where you left off. Proceed with the next step.
+The analytics collector is **privacy-respecting** by design:
+- No file contents are recorded
+- No sensitive data (API keys, tokens, passwords) — redacted automatically
+- Strings truncated to 500 characters
+- All data stays local (in-memory by default, optional SQLite)
 
 ## Dependencies
 
-- `@pi-unipi/core` — Shared constants and utilities
+- `@pi-unipi/core` — Shared constants, events, utilities
 - `@mariozechner/pi-coding-agent` — Pi extension API
-- `@sinclair/typebox` — Schema validation
+- `@sinclair/typebox` — Schema validation (peer dependency)
+- `sqlite3` — Optional, for persistent cache/analytics
+
+## License
+
+MIT

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@pi-unipi/utility",
-  "version": "0.1.1",
-  "description": "Utility commands and tools for Pi coding agent — continue, status helpers",
+  "version": "0.2.1",
+  "description": "Utility commands and tools for Pi coding agent — lifecycle, diagnostics, cache, analytics, display, batch execution",
   "type": "module",
   "license": "MIT",
   "author": "Neuron Mr White",
@@ -16,16 +16,22 @@
     "pi-coding-agent",
     "unipi",
     "utility",
-    "continue"
+    "continue",
+    "diagnostics",
+    "cache",
+    "analytics"
   ],
   "files": [
-    "index.ts",
-    "commands.ts",
-    "README.md"
+    "src/index.ts",
+    "src/commands.ts",
+    "src/types.ts",
+    "src/**/*.ts",
+    "README.md",
+    "skills/**/*.md"
   ],
   "pi": {
     "extensions": [
-      "index.ts"
+      "src/index.ts"
     ]
   },
   "publishConfig": {
@@ -37,5 +43,8 @@
   "peerDependencies": {
     "@mariozechner/pi-coding-agent": "*",
     "@sinclair/typebox": "*"
+  },
+  "scripts": {
+    "test": "npx tsx --test tests/**/*.test.ts"
   }
 }

package/skills/utility/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: utility
+scope: agent
+description: |
+  Guidance for using the @pi-unipi/utility extension commands and tools.
+  Provides lifecycle management, diagnostics, cleanup, batch execution,
+  environment inspection, and terminal utilities.
+---
+
+# @pi-unipi/utility — Agent Guidance
+
+## Overview
+
+The `@pi-unipi/utility` extension provides general-purpose utilities for the Pi coding agent. It is **not** related to compaction — it provides helpers that any module or agent can use.
+
+## Commands
+
+| Command | Purpose | When to Use |
+|---------|---------|-------------|
+| `/unipi:continue` | Continue agent without polluting context | After interruption, to resume cleanly |
+| `/unipi:reload` | Explain how to reload extensions | When user asks about hot-reloading |
+| `/unipi:status` | Request status from all modules | To check which modules are loaded |
+| `/unipi:cleanup` | Clean stale files, DBs, sessions | When disk space is low or after crashes |
+| `/unipi:env` | Show environment info | For debugging version/path issues |
+| `/unipi:doctor` | Run diagnostics | When something seems broken |
+
+### Cleanup Options
+
+- `--dry-run` — Report what would be cleaned without removing anything
+- Default max ages: DBs 14 days, temp 7 days, sessions 30 days
+
+## Tools
+
+| Tool | Purpose |
+|------|---------|
+| `ctx_batch` | Atomic batch execution of commands with rollback |
+| `ctx_env` | Environment inspection |
+
+### Batch Execution Pattern
+
+```typescript
+import { BatchBuilder } from "@pi-unipi/utility/tools/batch";
+
+const report = await new BatchBuilder()
+  .addCommand("search", { query: "foo" })
+  .addTool("memory_search", { query: "bar" })
+  .withOptions({ failFast: true, commandTimeoutMs: 30000 })
+  .execute(myExecutor);
+```
+
+## Modules (for code use)
+
+| Module | Import Path | Purpose |
+|--------|-------------|---------|
+| ProcessLifecycle | `@pi-unipi/utility/lifecycle/process` | Parent PID polling, orphan detection, cleanup registry |
+| cleanupStale | `@pi-unipi/utility/lifecycle/cleanup` | Stale file/DB/session cleanup |
+| TTLCache | `@pi-unipi/utility/cache/ttl-cache` | In-memory or SQLite-backed TTL cache |
+| AnalyticsCollector | `@pi-unipi/utility/analytics/collector` | Privacy-respecting event collection |
+| runDiagnostics | `@pi-unipi/utility/diagnostics/engine` | Cross-module health checks |
+| detectCapabilities | `@pi-unipi/utility/display/capabilities` | Terminal feature detection |
+| clampWidth, wrapLines | `@pi-unipi/utility/display/width` | ANSI-aware text formatting |
+| SettingsInspector | `@pi-unipi/utility/tui/settings-inspector` | Reusable settings overlay model |
+
+## Integration Notes
+
+- Lifecycle manager is a singleton — use `getLifecycle()` to access
+- Analytics collector is a singleton — use `getAnalyticsCollector()`
+- Diagnostics engine supports custom plugins via `registerDiagnosticPlugin()`
+- TTLCache defaults to memory backend; set `persistent: true` for SQLite
+- All modules emit unipi events for cross-module integration

package/src/analytics/collector.ts

@@ -0,0 +1,293 @@
+/**
+ * @pi-unipi/utility — Analytics Collector
+ *
+ * Lightweight metrics collection for all unipi modules.
+ * Privacy-respecting: no file contents, no sensitive data.
+ */
+
+import { randomUUID } from "node:crypto";
+import type {
+  AnalyticsEvent,
+  AnalyticsEventType,
+  AnalyticsRollup,
+  AnalyticsOptions,
+} from "../types.js";
+
+/** Default options */
+const DEFAULTS: Required<AnalyticsOptions> = {
+  dbPath: "~/.unipi/analytics/events.db",
+  maxEvents: 10000,
+  rollupEnabled: true,
+};
+
+/** In-memory event buffer */
+const eventBuffer: AnalyticsEvent[] = [];
+let bufferFlushTimer: ReturnType<typeof setInterval> | null = null;
+
+/** Generate a unique event ID */
+function generateId(): string {
+  return randomUUID();
+}
+
+/** Get today's date as YYYY-MM-DD */
+function today(): string {
+  return new Date().toISOString().slice(0, 10);
+}
+
+/**
+ * AnalyticsCollector records events and provides rollup aggregation.
+ * Uses in-memory buffering with periodic flush to avoid I/O overhead.
+ */
+export class AnalyticsCollector {
+  private opts: Required<AnalyticsOptions>;
+  private enabled: boolean;
+
+  constructor(options: AnalyticsOptions = {}) {
+    this.opts = { ...DEFAULTS, ...options };
+    this.enabled = true;
+    this.startFlushTimer();
+  }
+
+  /** Disable collection */
+  disable(): void {
+    this.enabled = false;
+    this.stopFlushTimer();
+  }
+
+  /** Enable collection */
+  enable(): void {
+    this.enabled = true;
+    this.startFlushTimer();
+  }
+
+  /** Record an analytics event */
+  record(
+    type: AnalyticsEventType,
+    metadata?: Record<string, unknown>,
+  ): AnalyticsEvent {
+    if (!this.enabled) {
+      return {
+        id: generateId(),
+        type,
+        timestamp: Date.now(),
+        metadata,
+      };
+    }
+
+    const event: AnalyticsEvent = {
+      id: generateId(),
+      type,
+      timestamp: Date.now(),
+      metadata: this.sanitizeMetadata(metadata),
+    };
+
+    eventBuffer.push(event);
+
+    // Auto-flush if buffer is full
+    if (eventBuffer.length >= this.opts.maxEvents) {
+      this.flush().catch(() => {
+        // Best-effort flush
+      });
+    }
+
+    return event;
+  }
+
+  /** Record a command execution */
+  recordCommand(
+    command: string,
+    module: string,
+    durationMs: number,
+    success: boolean,
+  ): AnalyticsEvent {
+    return this.record("command_run", {
+      command,
+      module,
+      durationMs,
+      success,
+    });
+  }
+
+  /** Record a tool call */
+  recordTool(
+    tool: string,
+    module: string,
+    durationMs: number,
+    success: boolean,
+  ): AnalyticsEvent {
+    return this.record("tool_call", {
+      tool,
+      module,
+      durationMs,
+      success,
+    });
+  }
+
+  /** Record an error */
+  recordError(
+    module: string,
+    errorType: string,
+    message?: string,
+  ): AnalyticsEvent {
+    return this.record("error", {
+      module,
+      errorType,
+      message: message ? this.sanitizeString(message) : undefined,
+    });
+  }
+
+  /** Record a module load */
+  recordModuleLoad(module: string, version: string): AnalyticsEvent {
+    return this.record("module_load", { module, version });
+  }
+
+  /** Record a search operation */
+  recordSearch(
+    module: string,
+    query: string,
+    resultCount: number,
+  ): AnalyticsEvent {
+    return this.record("search", {
+      module,
+      query: this.sanitizeString(query),
+      resultCount,
+    });
+  }
+
+  /** Get all buffered events */
+  getEvents(): readonly AnalyticsEvent[] {
+    return eventBuffer;
+  }
+
+  /** Get events filtered by type */
+  getEventsByType(type: AnalyticsEventType): AnalyticsEvent[] {
+    return eventBuffer.filter((e) => e.type === type);
+  }
+
+  /** Compute daily rollup from buffered events */
+  getRollup(date?: string): AnalyticsRollup {
+    const targetDate = date ?? today();
+    const dayStart = new Date(targetDate).getTime();
+    const dayEnd = dayStart + 24 * 60 * 60 * 1000;
+
+    const dayEvents = eventBuffer.filter(
+      (e) => e.timestamp >= dayStart && e.timestamp < dayEnd,
+    );
+
+    const events: Record<AnalyticsEventType, number> = {
+      module_load: 0,
+      command_run: 0,
+      tool_call: 0,
+      error: 0,
+      compaction: 0,
+      search: 0,
+    };
+
+    let totalDurationMs = 0;
+    let errorCount = 0;
+
+    for (const e of dayEvents) {
+      events[e.type]++;
+      if (e.metadata?.durationMs) {
+        totalDurationMs += Number(e.metadata.durationMs);
+      }
+      if (e.type === "error" || e.metadata?.success === false) {
+        errorCount++;
+      }
+    }
+
+    return {
+      date: targetDate,
+      events,
+      totalDurationMs,
+      errorCount,
+    };
+  }
+
+  /** Export all events to JSON */
+  exportToJSON(): string {
+    return JSON.stringify(eventBuffer, null, 2);
+  }
+
+  /** Flush buffered events to storage (placeholder for future SQLite persistence) */
+  async flush(): Promise<number> {
+    const count = eventBuffer.length;
+    // TODO: persist to SQLite when sqlite3 is available
+    // For now, keep in memory but trim if over max
+    if (count > this.opts.maxEvents) {
+      eventBuffer.splice(0, count - this.opts.maxEvents);
+    }
+    return count;
+  }
+
+  /** Clear all buffered events */
+  clear(): void {
+    eventBuffer.length = 0;
+  }
+
+  // ─── Private ───────────────────────────────────────────────────────────────
+
+  private startFlushTimer(): void {
+    if (bufferFlushTimer) return;
+    bufferFlushTimer = setInterval(() => {
+      this.flush().catch(() => {
+        // Best-effort
+      });
+    }, 60000); // Flush every minute
+    if (bufferFlushTimer.unref) {
+      bufferFlushTimer.unref();
+    }
+  }
+
+  private stopFlushTimer(): void {
+    if (bufferFlushTimer) {
+      clearInterval(bufferFlushTimer);
+      bufferFlushTimer = null;
+    }
+  }
+
+  /** Sensitive key patterns to redact */
+  private static SENSITIVE_KEYS =
+    /api[_-]?key|apiKey|token|password|secret|auth|credential|private[_-]?key/i;
+
+  /** Remove sensitive data from metadata */
+  private sanitizeMetadata(
+    metadata?: Record<string, unknown>,
+  ): Record<string, unknown> | undefined {
+    if (!metadata) return undefined;
+    const sanitized: Record<string, unknown> = {};
+    for (const [key, value] of Object.entries(metadata)) {
+      if (AnalyticsCollector.SENSITIVE_KEYS.test(key)) {
+        sanitized[key] = "[REDACTED]";
+      } else if (typeof value === "string") {
+        sanitized[key] = this.sanitizeString(value);
+      } else {
+        sanitized[key] = value;
+      }
+    }
+    return sanitized;
+  }
+
+  /** Truncate long strings, remove potential secrets */
+  private sanitizeString(str: string): string {
+    // Truncate to 500 chars
+    let result = str.length > 500 ? str.slice(0, 500) + "…" : str;
+    // Redact common secret patterns in strings
+    result = result.replace(
+      /(api[_-]?key|apiKey|token|password|secret|auth)\s*[:=]\s*\S+/gi,
+      "$1: [REDACTED]",
+    );
+    return result;
+  }
+}
+
+/** Global singleton */
+let globalCollector: AnalyticsCollector | null = null;
+
+/** Get or create the global analytics collector */
+export function getAnalyticsCollector(options?: AnalyticsOptions): AnalyticsCollector {
+  if (!globalCollector) {
+    globalCollector = new AnalyticsCollector(options);
+  }
+  return globalCollector;
+}