triage-trip 0.1.0

package/README.md

@@ -0,0 +1,146 @@
+# `ai-blackbox`
+
+> Autonomous AI Regression Triage & Provenance CLI. Pinpoint breaking contract changes introduced by AI coding tools in seconds.
+
+<p align="center">
+  <img src="./assets/demo.gif" alt="AI-Blackbox Demo" width="700">
+</p>
+---
+
+## The Problem
+
+AI coding assistants (Cursor, GitHub Copilot, Windsurf) dramatically accelerate drafting and refactoring code. However, they frequently cause **silent contract drift**:
+
+* An AI renames an object key or subtle schema attribute (e.g., `userId` ➔ `user_id`).
+* Standard linters and compilers pass because the syntax is completely valid.
+* Downstream modules crash 2–3 commits later with cryptic runtime errors (`TypeError: Cannot read properties of undefined`).
+* Developers spend 20–30 minutes manually inspecting multi-file diffs, running `git bisect`, and searching for what changed.
+
+---
+
+## What `ai-blackbox` Does
+
+`ai-blackbox` connects the dots across your development environment to trace the exact lineage of a regression:
+
+```text
+[ User Prompt in IDE ] ➔ [ AI Generated Diff ] ➔ [ Git Commit ] ➔ [ Runtime Error ]
+                                                                          │
+                                                                   ai-blackbox triage
+                                                                          │
+                                                                          ▼
+                                                       • Exact Root Cause & Commit
+                                                       • Attributed AI Prompt
+                                                       • Semantic Contract Shift
+                                                       • Clean Unified Diff Patch
+
+```
+
+1. **Inspects Git History:** Automatically reads uncommitted working changes and recent commits using `simple-git`.
+2. **Extracts AI Session Provenance:** Scans local IDE storage (e.g., Cursor's local SQLite database `state.vscdb`) to retrieve recent prompts and user instructions.
+3. **Triages Semantic Drift:** Correlates the runtime stack trace with recent AST modifications and prompts via the OpenAI API to deliver an instant diagnostic report.
+
+---
+
+## Installation
+
+### Prerequisites
+
+* **Node.js** (v18 or higher, ESM native)
+* An active **OpenAI API Key**
+
+### Local Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/your-username/ai-blackbox.git
+cd ai-blackbox
+
+# Install dependencies
+npm install
+
+# Link executable globally (optional for direct CLI access)
+npm link
+
+```
+
+Set your OpenAI API key in your terminal environment or add it to a `.env` file:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+
+```
+
+---
+
+## Quickstart & Usage
+
+Run the triage engine directly against any runtime error message or stack trace:
+
+```bash
+# Via npm script
+node bin/cli.js triage "TypeError: Cannot read properties of undefined (reading 'toUpperCase')"
+
+# Or globally (if linked)
+ai-blackbox triage "TypeError: Cannot read properties of undefined (reading 'toUpperCase')"
+
+```
+
+### Options
+
+* `<query_or_error>` *(required)*: The error message or failing test description.
+* `-c, --commits <number>`: Number of recent commits to analyze (default: `5`).
+
+---
+
+## Example Output
+
+```markdown
+■ AI-Blackbox Triage Engine
+
+✔ Root cause isolated.
+
+1. **Root Cause**: The error occurs in `app.js` at line 6, where `profile.userId` is accessed. 
+   The breaking change was introduced in commit `b83c577` ("refactor: update user schema"), 
+   which modified the `getUserProfile` function in `user.js`.
+
+2. **Attributed Prompt**: "Rename userId to user_id across the returned profile object."
+
+3. **The Shift**: The `getUserProfile` function now returns `user_id` instead of `userId`, 
+   causing `profile.userId` to evaluate to `undefined`.
+
+4. **Patch**:
+--- a/app.js
++++ b/app.js
+@@ -6,2 +6,2 @@
+-  console.log(`Loading dashboard for user ID: ${profile.userId.toUpperCase()}`);
++  console.log(`Loading dashboard for user ID: ${profile.user_id.toUpperCase()}`);
+
+```
+
+---
+
+## Project Structure
+
+```text
+ai-blackbox/
+├── bin/
+│   └── cli.js            # CLI commands & terminal output formatting (Commander, Ora, Picocolors)
+├── src/
+│   ├── git.js            # Working tree and commit diff extraction (simple-git)
+│   ├── cursor.js         # Cursor SQLite database parser (better-sqlite3)
+│   └── triage.js         # OpenAI triage diagnostic engine (GPT-4o)
+├── package.json
+└── README.md
+
+```
+
+---
+
+## Roadmap
+
+* [x] Git diff & commit stream extraction
+* [x] Local SQLite prompt extraction for Cursor Composer
+* [x] OpenAI regression diagnostic engine
+* [ ] Auto-patch application flag (`--fix` via `git apply`)
+* [ ] Multi-IDE adapters (GitHub Copilot Chat, Windsurf Cascade, Claude Code)
+* [ ] GitHub Action runner for automated PR failure triage

package/bin/cli.js

@@ -0,0 +1,75 @@
+#!/usr/bin/env node
+
+import { Command } from 'commander';
+import pc from 'picocolors';
+import ora from 'ora';
+import { confirm } from '@inquirer/prompts';
+import 'dotenv/config';
+import { getRecentChanges } from '../src/git.js';
+import { triageRegression } from '../src/triage.js';
+import { getRecentCursorPrompts } from '../src/cursor.js';
+import { extractDiff, applyDiff } from '../src/patch.js';
+
+const program = new Command();
+
+program
+  .name('triage-trip')
+  .description('Autonomous triage for regressions introduced by AI edits')
+  .version('0.1.0');
+
+program
+  .command('triage')
+  .description('Diagnose why your build or code is failing')
+  .argument('<query_or_error>', 'The error message or description of failure')
+  .option('-c, --commits <number>', 'Number of recent commits to analyze', '5')
+  .option('-y, --yes', 'Automatically apply the patch without asking', false)
+  .action(async (errorInput, options) => {
+    console.log(pc.bold(pc.cyan('\n■ AI-Blackbox Triage Engine\n')));
+
+    const spinner = ora('Reading recent git changes & Cursor prompts...').start();
+    let diagnosis = '';
+    
+    try {
+      const commitCount = parseInt(options.commits, 10);
+      const [changes, prompts] = await Promise.all([
+        getRecentChanges(commitCount),
+        Promise.resolve().then(() => getRecentCursorPrompts(10))
+      ]);
+
+      spinner.text = 'Isolating structural contract shifts and AST regressions...';
+      diagnosis = await triageRegression(errorInput, changes, prompts);
+
+      spinner.succeed(pc.green('Root cause isolated.'));
+      console.log('\n' + diagnosis + '\n');
+    } catch (err) {
+      spinner.fail(pc.red('Triage failed: ' + err.message));
+      process.exit(1);
+    }
+
+    // Attempt to extract patch from LLM output
+    const patch = extractDiff(diagnosis);
+
+    if (patch) {
+      let shouldApply = options.yes;
+
+      if (!shouldApply) {
+        shouldApply = await confirm({
+          message: 'Apply this patch to your working tree now?',
+          default: true,
+        });
+      }
+
+      if (shouldApply) {
+        const patchSpinner = ora('Applying patch via git apply...').start();
+        try {
+          await applyDiff(patch);
+          patchSpinner.succeed(pc.green('Patch applied successfully to working tree.'));
+          console.log(pc.dim('Run `git diff` to review or `git commit` to save the fix.\n'));
+        } catch (err) {
+          patchSpinner.fail(pc.red(err.message));
+        }
+      }
+    }
+  });
+
+program.parse(process.argv);

package/package.json

@@ -0,0 +1,43 @@
+{
+  "name": "triage-trip",
+  "version": "0.1.0",
+  "description": "Autonomous CLI to pinpoint and patch regressions introduced by AI code assistants",
+  "main": "index.js",
+  "scripts": {
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "keywords": [
+    "cursor",
+    "ai",
+    "triage",
+    "git",
+    "debugging"
+  ],
+  "author": "Anuj Prakash",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/arules3/ai-blackbox.git"
+  },
+  "files": [
+    "bin",
+    "src"
+  ],
+  "type": "module",
+  "bin": {
+    "triage-trip": "bin/cli.js"
+  },
+  "dependencies": {
+    "@anthropic-ai/sdk": "^0.131.0",
+    "@babel/parser": "^7.29.9",
+    "@babel/traverse": "^7.29.8",
+    "@inquirer/prompts": "^8.7.2",
+    "better-sqlite3": "^13.0.3",
+    "commander": "^15.0.0",
+    "dotenv": "^18.0.5",
+    "openai": "^7.27.0",
+    "ora": "^9.4.1",
+    "picocolors": "^1.1.1",
+    "simple-git": "^4.0.2"
+  }
+}

package/src/cursor.js

@@ -0,0 +1,176 @@
+// src/cursor.js
+import fs from 'fs';
+import path from 'path';
+import os from 'os';
+import Database from 'better-sqlite3';
+
+function getCursorUserDir() {
+    const home = os.homedir();
+    if (process.platform === 'darwin') {
+        return path.join(home, 'Library', 'Application Support', 'Cursor', 'User');
+    }
+    if (process.platform === 'win32') {
+        return path.join(process.env.APPDATA || path.join(home, 'AppData', 'Roaming'), 'Cursor', 'User');
+    }
+    return path.join(home, '.config', 'Cursor', 'User');
+}
+
+export function findCurrentWorkspaceDb(cwd = process.cwd()) {
+    const userDir = getCursorUserDir();
+    const workspaceStorageDir = path.join(userDir, 'workspaceStorage');
+    if (!fs.existsSync(workspaceStorageDir)) return null;
+
+    try {
+        const entries = fs.readdirSync(workspaceStorageDir);
+        let bestMatch = null;
+        const normalizedCwd = path.resolve(cwd).toLowerCase();
+
+        for (const entry of entries) {
+            const fullPath = path.join(workspaceStorageDir, entry);
+            const jsonPath = path.join(fullPath, 'workspace.json');
+            const dbPath = path.join(fullPath, 'state.vscdb');
+
+            if (fs.existsSync(jsonPath) && fs.existsSync(dbPath)) {
+                try {
+                    const metadata = JSON.parse(fs.readFileSync(jsonPath, 'utf8'));
+                    if (metadata.folder) {
+                        let folderPath = decodeURIComponent(metadata.folder);
+                        if (folderPath.startsWith('file://')) {
+                            folderPath = folderPath.replace('file://', '');
+                        }
+                        const normalizedFolder = path.resolve(folderPath).toLowerCase();
+
+                        // Exact folder match takes highest priority
+                        if (normalizedFolder === normalizedCwd) {
+                            return dbPath;
+                        }
+                    }
+                } catch { }
+            }
+        }
+    } catch { }
+    return null;
+}
+
+// Inside src/cursor.js - update getRecentCursorPrompts:
+
+export function getRecentCursorPrompts(limit = 5) {
+    const userDir = getCursorUserDir();
+    const workspaceDbPath = findCurrentWorkspaceDb();
+    const globalDbPath = path.join(userDir, 'globalStorage', 'state.vscdb');
+
+    if (!fs.existsSync(globalDbPath)) return [];
+
+    let targetComposerIds = [];
+    if (workspaceDbPath && fs.existsSync(workspaceDbPath)) {
+        try {
+            const wsDb = new Database(workspaceDbPath, { readonly: true, fileMustExist: true });
+            const row = wsDb.prepare("SELECT value FROM ItemTable WHERE key = 'composer.composerData'").get();
+            if (row?.value) {
+                const parsed = JSON.parse(row.value);
+                if (Array.isArray(parsed.selectedComposerIds)) {
+                    targetComposerIds.push(...parsed.selectedComposerIds);
+                }
+                if (Array.isArray(parsed.lastFocusedComposerIds)) {
+                    targetComposerIds.push(...parsed.lastFocusedComposerIds);
+                }
+            }
+            wsDb.close();
+        } catch { }
+    }
+
+    targetComposerIds = [...new Set(targetComposerIds)];
+
+    try {
+        const gDb = new Database(globalDbPath, { readonly: true, fileMustExist: true });
+        let rows = [];
+
+        if (targetComposerIds.length > 0) {
+            const placeholders = targetComposerIds.map(() => 'key LIKE ?').join(' OR ');
+            const params = targetComposerIds.map((id) => `bubbleId:${id}:%`);
+            rows = gDb
+                .prepare(`SELECT key, value FROM cursorDiskKV WHERE ${placeholders}`)
+                .all(...params);
+        }
+
+        if (rows.length === 0) {
+            rows = gDb
+                .prepare("SELECT key, value FROM cursorDiskKV WHERE key LIKE 'bubbleId:%' ORDER BY rowid DESC LIMIT 200")
+                .all();
+        }
+
+        const prompts = [];
+
+        for (const row of rows) {
+            try {
+                const data = JSON.parse(row.value);
+                let text = null;
+
+                if (typeof data.text === 'string') {
+                    text = data.text;
+                } else if (Array.isArray(data.richText)) {
+                    text = data.richText.map((n) => n.text || '').join(' ');
+                }
+
+                if (text) {
+                    text = text.trim();
+
+                    const isAssistant =
+                        data.type === 2 ||
+                        data.type === 'ai' ||
+                        data.type === 'assistant' ||
+                        data.bubbleType === 'ai' ||
+                        data.bubbleType === 'assistant' ||
+                        text.startsWith('#') ||
+                        text.startsWith('```');
+
+                    const isUser =
+                        data.type === 1 ||
+                        data.type === 'user' ||
+                        data.bubbleType === 'user';
+
+                    // Only keep user prompts, skip assistant replies & system notifications
+                    if (isUser && !text.startsWith('<timestamp>') && !text.startsWith('<system_') && text.length > 2) {
+                        // Normalize timestamp to integer epoch
+                        const ts = data.createdAt || data.timestamp || data.clientTimestamp || 0;
+                        const timeNum = typeof ts === 'string' ? new Date(ts).getTime() : Number(ts);
+
+                        prompts.push({
+                            prompt: text,
+                            timestamp: timeNum || 0,
+                        });
+                    } else if (!isAssistant && !isUser && text.length > 2 && text.length < 500 && !text.startsWith('<timestamp>') && !text.startsWith('<system_')) {
+                        const ts = data.createdAt || data.timestamp || data.clientTimestamp || 0;
+                        const timeNum = typeof ts === 'string' ? new Date(ts).getTime() : Number(ts);
+
+                        prompts.push({
+                            prompt: text,
+                            timestamp: timeNum || 0,
+                        });
+                    }
+                }
+            } catch { }
+        }
+
+        gDb.close();
+
+        // 1. Sort strictly descending by timestamp (newest first)
+        prompts.sort((a, b) => b.timestamp - a.timestamp);
+
+        // 2. Deduplicate keeping only the latest occurrence
+        const uniquePrompts = [];
+        for (const item of prompts) {
+            if (!uniquePrompts.some((p) => p.prompt === item.prompt)) {
+                uniquePrompts.push({
+                    prompt: item.prompt,
+                    timestamp: item.timestamp ? new Date(item.timestamp).toISOString() : null,
+                });
+            }
+        }
+
+        return uniquePrompts.slice(0, limit);
+    } catch (err) {
+        return [];
+    }
+}
+

package/src/git.js

@@ -0,0 +1,67 @@
+import { simpleGit } from 'simple-git';
+
+const git = simpleGit();
+
+/**
+ * Truncates diff text if it exceeds max character limit to prevent LLM token blowup.
+ */
+function sanitizeDiff(diffStr, maxChars = 4000) {
+  if (!diffStr) return null;
+  if (diffStr.length > maxChars) {
+    return diffStr.slice(0, maxChars) + '\n\n... [diff truncated for length] ...';
+  }
+  return diffStr;
+}
+
+export async function getRecentChanges(maxCommits = 3) {
+  const isRepo = await git.checkIsRepo();
+  if (!isRepo) {
+    throw new Error('Not inside a valid Git repository.');
+  }
+
+  const status = await git.status();
+
+  // 1. Capture both unstaged and staged uncommitted diffs (excluding noisy lockfiles)
+  const lockfileIgnores = [
+    ':!package-lock.json',
+    ':!yarn.lock',
+    ':!pnpm-lock.yaml',
+    ':!bun.lockb',
+  ];
+
+  const unstagedDiff = await git.diff(['--', '.', ...lockfileIgnores]);
+  const stagedDiff = await git.diff(['--cached', '--', '.', ...lockfileIgnores]);
+
+  let combinedWorkingDiff = '';
+  if (stagedDiff) combinedWorkingDiff += `--- STAGED CHANGES ---\n${stagedDiff}\n`;
+  if (unstagedDiff) combinedWorkingDiff += `--- UNSTAGED CHANGES ---\n${unstagedDiff}\n`;
+
+  // 2. Fetch commit history with bounded diff size
+  const log = await git.log({ maxCount: maxCommits });
+  const commitDiffs = await Promise.all(
+    log.all.map(async (c) => {
+      const rawDiff = await git.show([
+        c.hash,
+        '-p',
+        '--',
+        '.',
+        ...lockfileIgnores,
+      ]);
+
+      return {
+        hash: c.hash.slice(0, 7),
+        author: c.author_name,
+        date: c.date,
+        message: c.message,
+        diff: sanitizeDiff(rawDiff),
+      };
+    })
+  );
+
+  return {
+    branch: status.current,
+    untrackedFiles: status.not_added, // Tells the LLM if newly created files exist
+    workingDiff: sanitizeDiff(combinedWorkingDiff) || null,
+    recentCommits: commitDiffs,
+  };
+}

package/src/patch.js

@@ -0,0 +1,120 @@
+// src/patch.js
+import fs from 'fs';
+import path from 'path';
+import { spawn } from 'child_process';
+
+export function extractDiff(llmOutput) {
+  const diffMatch =
+    llmOutput.match(/```diff\s*([\s\S]*?)```/) ||
+    llmOutput.match(/(--- [^\n]+\n\+\+\+ [^\n]+[\s\S]*)/);
+
+  if (!diffMatch) return null;
+  let rawDiff = diffMatch[1] || diffMatch[0];
+
+  rawDiff = rawDiff.replace(/\u00A0/g, ' ').replace(/\r\n/g, '\n');
+  rawDiff = rawDiff.replace(/^index [0-9a-f]+\.\.[0-9a-f]+.*$\n?/m, '');
+
+  if (!rawDiff.endsWith('\n')) {
+    rawDiff += '\n';
+  }
+
+  return rawDiff.trim() + '\n';
+}
+
+export async function applyDiff(diffString) {
+  const flags = [
+    '--recount',
+    '--ignore-space-change',
+    '--ignore-whitespace',
+    '--whitespace=nowarn',
+    '-C1'
+  ];
+
+  // 1. Try git apply
+  const attempt = await runGitApply(diffString, flags);
+  if (attempt.success) return true;
+
+  // 2. Try git apply with 3way
+  const attempt3Way = await runGitApply(diffString, [...flags, '--3way']);
+  if (attempt3Way.success) return true;
+
+  // 3. Fallback: Direct line search & replace on disk
+  const applied = fallbackManualPatch(diffString);
+  if (applied) return true;
+
+  throw new Error(`Patch cannot apply cleanly:\n${attempt.stderr}`);
+}
+
+function runGitApply(diffContent, flags = []) {
+  return new Promise((resolve) => {
+    const git = spawn('git', ['apply', ...flags, '-'], {
+      stdio: ['pipe', 'pipe', 'pipe']
+    });
+
+    let stderr = '';
+    git.stderr.on('data', (chunk) => {
+      stderr += chunk.toString();
+    });
+
+    git.on('close', (code) => {
+      resolve({
+        success: code === 0,
+        stderr: stderr.trim()
+      });
+    });
+
+    git.stdin.write(diffContent);
+    git.stdin.end();
+  });
+}
+
+function fallbackManualPatch(diffString) {
+  try {
+    const fileMatch = diffString.match(/\+\+\+ b\/(.+)/);
+    if (!fileMatch) return false;
+
+    const relativePath = fileMatch[1].trim();
+    const filePath = path.resolve(process.cwd(), relativePath);
+    if (!fs.existsSync(filePath)) return false;
+
+    const fileContent = fs.readFileSync(filePath, 'utf8');
+
+    // Extract removed (-) and added (+) lines from the hunk
+    const lines = diffString.split('\n');
+    let oldLine = null;
+    let newLine = null;
+
+    for (const line of lines) {
+      if (line.startsWith('-') && !line.startsWith('---')) {
+        oldLine = line.slice(1).trim();
+      } else if (line.startsWith('+') && !line.startsWith('+++')) {
+        newLine = line.slice(1).trim();
+      }
+    }
+
+    if (!oldLine || !newLine) return false;
+
+    // Scan file lines and replace the matching content while preserving original indentation
+    const fileLines = fileContent.split('\n');
+    let replaced = false;
+
+    for (let i = 0; i < fileLines.length; i++) {
+      if (fileLines[i].includes(oldLine)) {
+        const indentMatch = fileLines[i].match(/^\s*/);
+        const indent = indentMatch ? indentMatch[0] : '';
+        fileLines[i] = indent + newLine;
+        replaced = true;
+        break;
+      }
+    }
+
+    if (replaced) {
+      fs.writeFileSync(filePath, fileLines.join('\n'), 'utf8');
+      return true;
+    }
+
+    return false;
+  } catch {
+    return false;
+  }
+}

package/src/triage.js

@@ -0,0 +1,69 @@
+// src/triage.js
+import OpenAI from 'openai';
+import Anthropic from '@anthropic-ai/sdk';
+
+export async function triageRegression(errorDescription, gitContext, cursorPrompts = []) {
+  const anthropicKey = process.env.ANTHROPIC_API_KEY;
+  const openaiKey = process.env.OPENAI_API_KEY;
+
+  if (!anthropicKey && !openaiKey) {
+    throw new Error('Please set either ANTHROPIC_API_KEY or OPENAI_API_KEY in your environment.');
+  }
+
+  const systemInstructions = `
+You are an expert developer tool agent specializing in Git diff analysis, AST regression diagnosis, and code patch generation.
+
+When analyzing the regression:
+- Focus on contract/schema shifts between callers and callees (e.g. key renames in returned objects, changed parameter signatures).
+- Prefer aligning property access to match the actual returned schema (e.g. renaming \`profile.userId\` to \`profile.user_id\`) rather than applying defensive null checks or fallbacks like ternary operators or 'UNKNOWN', unless explicitly intended.
+
+When generating the "Patch" section:
+- Always use a valid, standard unified diff format enclosed in \`\`\`diff.
+- Include proper headers (--- a/path/to/file.js, +++ b/path/to/file.js).
+- Ensure @@ -line,count +line,count @@ markers accurately match the context.
+- Include 1-2 lines of unchanged surrounding context above and below the change.
+- Only touch lines that actually exist in the file.
+- Do not invent comments in context lines.
+`.trim();
+
+  const userPrompt = `
+User problem / error trace:
+"${errorDescription}"
+
+Recent Cursor Composer Prompts:
+${JSON.stringify(cursorPrompts, null, 2)}
+
+Recent Repository Git Stream (Working diff + Commits):
+${JSON.stringify(gitContext, null, 2)}
+
+Provide a strict, concise triage output formatted with terminal-friendly Markdown:
+1. **Root Cause**: Pinpoint the file, line, and commit/working diff where the breaking change occurred.
+2. **Attributed Prompt**: Name the specific Cursor prompt that likely caused this diff (if found in the list; otherwise state none).
+3. **The Shift**: Explain what broke (e.g., schema key rename, dropped parameter, async/await omission).
+4. **Patch**: Provide the exact unified diff adhering strictly to the system diff formatting instructions.
+`;
+
+  // 1. Prefer Anthropic if key exists
+  if (anthropicKey) {
+    const anthropic = new Anthropic({ apiKey: anthropicKey });
+    const response = await anthropic.messages.create({
+      model: 'claude-3-5-sonnet-20241022',
+      max_tokens: 1500,
+      system: systemInstructions,
+      messages: [{ role: 'user', content: userPrompt }]
+    });
+    return response.content[0].text;
+  }
+
+  // 2. Fallback to OpenAI
+  const openai = new OpenAI({ apiKey: openaiKey });
+  const response = await openai.chat.completions.create({
+    model: 'gpt-4o-mini',
+    temperature: 0.1,
+    messages: [
+      { role: 'system', content: systemInstructions },
+      { role: 'user', content: userPrompt }
+    ]
+  });
+  return response.choices[0].message.content;
+}