@haystackeditor/cli 0.8.0 → 0.9.0

package/README.md

@@ -4,19 +4,61 @@ Set up Haystack verification for your project. When PRs are opened, an AI agent
 
 ## Quick Start
 
+### Fastest: AI-Assisted Setup
+
+```bash
+npx @haystackeditor/cli skills install
+```
+
+This auto-detects your coding CLI (Claude Code, Codex, Cursor) and installs the Haystack MCP server.
+
+Then invoke the setup in your coding CLI:
+
+| CLI | How to invoke |
+|-----|---------------|
+| **Claude Code** | `/setup-haystack` |
+| **Codex CLI** | `/setup-haystack` or ask "set up haystack verification" |
+| **Cursor** | `/setup-haystack` (in Composer) |
+
+The AI will analyze your codebase, create `.haystack.json`, and configure verification flows.
+
+### Alternative: Direct CLI Setup
+
+For quick setup without AI assistance:
+
 ```bash
 npx @haystackeditor/cli init
 ```
 
-This auto-detects your framework, package manager, and ports, then creates:
-- `.haystack.json` - Configuration for the verification agent
-- `.agents/skills/haystack.md` - Skill file for AI agent discovery
+This runs an interactive wizard that auto-detects your project and creates `.haystack.json`.
 
-## Commands
+### Additional Skills
 
-### `haystack init`
+| Skill | Purpose |
+|-------|---------|
+| `/setup-haystack` | **Start here** - diagnoses project, creates config |
+| `/prepare-haystack` | Add aria-labels and data-testid for browser automation |
+| `/setup-haystack-secrets` | Configure API keys, LLM credentials, secrets |
 
-Interactive setup wizard:
+---
+
+## CLI Commands
+
+All commands can be run directly:
+
+### `haystack skills`
+
+Install AI skills into your coding CLI (auto-detects Claude Code, Codex, Cursor):
+
+```bash
+npx @haystackeditor/cli skills install              # Auto-detect CLI
+npx @haystackeditor/cli skills install --cli codex  # Install for Codex
+npx @haystackeditor/cli skills install --cli cursor # Install for Cursor
+npx @haystackeditor/cli skills install --cli manual # Show manual setup
+npx @haystackeditor/cli skills list                 # List available skills
+```
+
+### `haystack init`
 
 ```bash
 npx @haystackeditor/cli init       # Interactive wizard
@@ -73,6 +115,54 @@ npx @haystackeditor/cli secrets set API_KEY xxx --scope org --scope-id myorg
 npx @haystackeditor/cli secrets set API_KEY xxx --scope repo --scope-id owner/repo
 ```
 
+### `haystack submit`
+
+Create a PR from current changes. Runs pre-PR triage (code review, rules validation, intent drift), pushes your branch, and opens the PR.
+
+```bash
+haystack submit                      # Triage → create PR → wait for analysis
+haystack submit --title "Fix auth"   # Custom PR title
+haystack submit --draft              # Create as draft PR
+haystack submit --force              # Skip triage checks
+haystack submit --no-wait            # Don't wait for analysis results
+```
+
+**Review routing**: By default, PRs go to the auto-merge queue — if analysis passes, the PR is merged automatically. Use `--review` to route it for human review instead:
+
+```bash
+haystack submit --review             # Needs review (goes to assignment queue)
+haystack submit --review octocat     # Request review from a specific teammate
+```
+
+When `--review` is used without a username, the PR is labeled `haystack:needs-review` and appears in your team's assignment queue. When a username is provided, that person is also requested as a reviewer on GitHub.
+
+### `haystack triage`
+
+View Haystack analysis results for any PR. Shows the same data as the Haystack web feed: rating, verdict, structured findings with details, verified bugs, human review reasons, and agent fix prompts.
+
+```bash
+haystack triage 42                          # Current repo, PR #42
+haystack triage owner/repo#99               # Fully qualified
+haystack triage https://github.com/o/r/pull/1  # From GitHub URL
+haystack triage 42 --json                   # Machine-readable JSON output
+haystack triage 42 --no-wait                # Don't wait if analysis is pending
+```
+
+The `--json` output includes `agentFixPrompt` fields — ready-to-paste instructions for coding agents to fix each finding.
+
+### `haystack check-pending`
+
+Check the status of a previously submitted PR's analysis (from the current repo):
+
+```bash
+haystack check-pending              # Rich terminal output
+haystack check-pending --hook       # Minimal output (for session-start hooks)
+haystack check-pending --json       # Machine-readable JSON
+haystack check-pending --clear      # Clear pending state
+```
+
+---
+
 ## Configuration
 
 The `init` command creates `.haystack.json`:
@@ -104,15 +194,18 @@ verification:
 | Key user journeys | Flows describing what to verify |
 | API calls needing auth | Fixtures to mock responses |
 
-See the generated `.agents/skills/haystack.md` for full documentation on flows, fixtures, and monorepo configuration.
+Use `/setup-haystack` in Claude Code for AI-assisted configuration of flows and fixtures.
+
+---
 
 ## How It Works
 
-1. You run `npx @haystackeditor/cli init` and commit the config
-2. When a PR is opened, Haystack's AI agent:
-   - Spins up your app in a Modal sandbox
-   - Reads the flows to understand what to verify
-   - Navigates the app autonomously
+1. Run `npx @haystackeditor/cli init` or use `/setup-haystack` in Claude Code
+2. Commit the generated `.haystack.json`
+3. Install the [Haystack GitHub App](https://haystackeditor.com/github-app)
+4. When PRs are opened, Haystack's AI agent:
+   - Spins up your app in a cloud sandbox
+   - Runs verification commands
    - Captures screenshots and evidence
    - Reports results on the PR
 

package/dist/assets/hooks/agent-context/detect.ts

@@ -0,0 +1,136 @@
+import * as fs from 'fs';
+import * as path from 'path';
+import * as os from 'os';
+import * as crypto from 'crypto';
+import type { DetectionResult, AgentType } from './types.js';
+
+const STALENESS_MINUTES = parseInt(
+  process.env.AGENT_CONTEXT_STALENESS_MINUTES ?? '5',
+  10,
+);
+
+export function sanitizePath(absPath: string): string {
+  return absPath.replace(/[^a-zA-Z0-9]/g, '-');
+}
+
+function sha256(input: string): string {
+  return crypto.createHash('sha256').update(input).digest('hex');
+}
+
+function isRecent(filePath: string): boolean {
+  try {
+    const stat = fs.statSync(filePath);
+    const ageMs = Date.now() - stat.mtimeMs;
+    return ageMs < STALENESS_MINUTES * 60 * 1000;
+  } catch {
+    return false;
+  }
+}
+
+function findMostRecentFile(dir: string, pattern: RegExp): string | null {
+  if (!fs.existsSync(dir)) return null;
+
+  let entries: fs.Dirent[];
+  try {
+    entries = fs.readdirSync(dir, { withFileTypes: true });
+  } catch {
+    return null;
+  }
+
+  let bestPath: string | null = null;
+  let bestMtime = 0;
+
+  for (const entry of entries) {
+    if (!entry.isFile() || !pattern.test(entry.name)) continue;
+    const fullPath = path.join(dir, entry.name);
+    try {
+      const stat = fs.statSync(fullPath);
+      if (stat.mtimeMs > bestMtime) {
+        bestMtime = stat.mtimeMs;
+        bestPath = fullPath;
+      }
+    } catch {
+      continue;
+    }
+  }
+
+  return bestPath;
+}
+
+function extractSessionId(filePath: string, ext: string): string {
+  return path.basename(filePath, ext);
+}
+
+function detectClaude(repoPath: string): DetectionResult {
+  const overrideDir = process.env.ENTIRE_TEST_CLAUDE_PROJECT_DIR;
+  const sessionDir =
+    overrideDir ||
+    path.join(os.homedir(), '.claude', 'projects', sanitizePath(repoPath));
+
+  const sessionFile = findMostRecentFile(sessionDir, /\.jsonl$/);
+  if (!sessionFile || !isRecent(sessionFile)) {
+    return { detected: false };
+  }
+
+  return {
+    detected: true,
+    agent: 'claude-code',
+    sessionFilePath: sessionFile,
+    sessionId: extractSessionId(sessionFile, '.jsonl'),
+  };
+}
+
+function detectGemini(repoPath: string): DetectionResult {
+  const overrideDir = process.env.ENTIRE_TEST_GEMINI_PROJECT_DIR;
+  const projectHash = sha256(repoPath);
+  const sessionDir =
+    overrideDir ||
+    path.join(os.homedir(), '.gemini', 'tmp', projectHash, 'chats');
+
+  const sessionFile = findMostRecentFile(sessionDir, /^session-.*\.json$/);
+  if (!sessionFile || !isRecent(sessionFile)) {
+    return { detected: false };
+  }
+
+  return {
+    detected: true,
+    agent: 'gemini-cli',
+    sessionFilePath: sessionFile,
+    sessionId: extractSessionId(sessionFile, '.json'),
+  };
+}
+
+function detectOpenCode(repoPath: string): DetectionResult {
+  const overrideDir = process.env.ENTIRE_TEST_OPENCODE_PROJECT_DIR;
+  const sessionDir =
+    overrideDir ||
+    path.join(os.tmpdir(), 'entire-opencode', sanitizePath(repoPath));
+
+  const sessionFile = findMostRecentFile(sessionDir, /\.json$/);
+  if (!sessionFile || !isRecent(sessionFile)) {
+    return { detected: false };
+  }
+
+  return {
+    detected: true,
+    agent: 'opencode',
+    sessionFilePath: sessionFile,
+    sessionId: extractSessionId(sessionFile, '.json'),
+  };
+}
+
+const detectors: Array<(repoPath: string) => DetectionResult> = [
+  detectClaude,
+  detectGemini,
+  detectOpenCode,
+];
+
+export async function detectAgent(
+  repoPath: string,
+): Promise<DetectionResult> {
+  for (const detect of detectors) {
+    const result = detect(repoPath);
+    if (result.detected) return result;
+  }
+  return { detected: false };
+}

package/dist/assets/hooks/agent-context/format.ts

@@ -0,0 +1,99 @@
+import type { AgentContext, InteractionTurn } from './types.js';
+
+const AGENT_DISPLAY_NAMES: Record<string, string> = {
+  'claude-code': 'Claude Code',
+  'gemini-cli': 'Gemini CLI',
+  opencode: 'OpenCode',
+};
+
+// intentional: truncation for display formatting in hook output, not LLM context
+function truncate(text: string, maxLen: number): string {
+  if (text.length <= maxLen) return text;
+  return text.slice(0, maxLen) + '...';
+}
+
+function formatNumber(n: number): string {
+  return n.toLocaleString('en-US');
+}
+
+function formatTurnSummary(turn: InteractionTurn): string {
+  const role = turn.role.toUpperCase();
+  const content = truncate(turn.content.replace(/\n/g, ' '), 200);
+  const toolNames = turn.toolUses.map((t) => t.toolName);
+  const toolSuffix = toolNames.length > 0 ? ` [tools: ${toolNames.join(', ')}]` : '';
+  return `  [${turn.turnIndex + 1}] ${role}: ${content}${toolSuffix}`;
+}
+
+function formatHuman(ctx: AgentContext): string {
+  const lines: string[] = [];
+
+  lines.push('========================================');
+  lines.push('  AGENT CONTEXT DETECTED');
+  lines.push('========================================');
+  lines.push(`Agent:       ${AGENT_DISPLAY_NAMES[ctx.agent] || ctx.agent}`);
+  lines.push(`Session ID:  ${ctx.sessionId}`);
+
+  if (ctx.metadata.model) {
+    lines.push(`Model:       ${ctx.metadata.model}`);
+  }
+  if (ctx.metadata.version) {
+    lines.push(`Version:     ${ctx.metadata.version}`);
+  }
+  if (ctx.metadata.gitBranch) {
+    lines.push(`Git Branch:  ${ctx.metadata.gitBranch}`);
+  }
+  if (ctx.metadata.startedAt) {
+    lines.push(`Started At:  ${ctx.metadata.startedAt}`);
+  }
+
+  lines.push('');
+  lines.push('Task Prompt:');
+  lines.push(`  ${truncate(ctx.taskPrompt.replace(/\n/g, ' '), 500)}`);
+
+  lines.push('');
+  lines.push(`Modified Files (${ctx.modifiedFiles.length}):`);
+  if (ctx.modifiedFiles.length === 0) {
+    lines.push('  (none)');
+  } else {
+    for (const f of ctx.modifiedFiles) {
+      // Find which tools modified this file
+      const tools = new Set<string>();
+      for (const turn of ctx.transcript) {
+        for (const mf of turn.modifiedFiles) {
+          if (mf.filePath === f) tools.add(mf.toolName);
+        }
+      }
+      lines.push(`  - ${f} (${Array.from(tools).join(', ')})`);
+    }
+  }
+
+  lines.push('');
+  lines.push('Token Usage:');
+  lines.push(
+    `  Input:  ${formatNumber(ctx.tokenUsage.inputTokens)}  Output: ${formatNumber(ctx.tokenUsage.outputTokens)}`,
+  );
+  if (ctx.tokenUsage.cacheCreationTokens || ctx.tokenUsage.cacheReadTokens) {
+    lines.push(
+      `  Cache Creation: ${formatNumber(ctx.tokenUsage.cacheCreationTokens || 0)}  Cache Read: ${formatNumber(ctx.tokenUsage.cacheReadTokens || 0)}`,
+    );
+  }
+
+  lines.push('');
+  lines.push(`Transcript (${ctx.transcript.length} turns):`);
+  for (const turn of ctx.transcript) {
+    lines.push(formatTurnSummary(turn));
+  }
+
+  lines.push('========================================');
+  return lines.join('\n');
+}
+
+function formatJson(ctx: AgentContext): string {
+  return JSON.stringify(ctx, null, 2);
+}
+
+export function formatContext(ctx: AgentContext): string {
+  const mode = process.env.AGENT_CONTEXT_FORMAT || 'human';
+  if (mode === 'json') return formatJson(ctx);
+  return formatHuman(ctx);
+}

package/dist/assets/hooks/agent-context/index.ts

@@ -0,0 +1,39 @@
+import { detectAgent } from './detect.js';
+import { ClaudeParser } from './parsers/claude.js';
+import { GeminiParser } from './parsers/gemini.js';
+import { OpenCodeParser } from './parsers/opencode.js';
+import { formatContext } from './format.js';
+import type { AgentParser } from './types.js';
+
+const PARSERS: Record<string, AgentParser> = {
+  'claude-code': new ClaudeParser(),
+  'gemini-cli': new GeminiParser(),
+  opencode: new OpenCodeParser(),
+};
+
+async function main() {
+  const repoRoot = process.argv[2] || process.cwd();
+
+  const detection = await detectAgent(repoRoot);
+
+  if (!detection.detected || !detection.agent || !detection.sessionFilePath) {
+    if (process.env.AGENT_CONTEXT_VERBOSE) {
+      console.error('[agent-context] No AI agent session detected.');
+    }
+    return;
+  }
+
+  const parser = PARSERS[detection.agent];
+  if (!parser) {
+    console.error(`[agent-context] No parser for agent: ${detection.agent}`);
+    return;
+  }
+
+  const context = await parser.parse(detection.sessionFilePath, repoRoot);
+  const output = formatContext(context);
+  console.error(output);
+}
+
+main().catch((err) => {
+  console.error(`[agent-context] Error: ${err.message}`);
+});

package/dist/assets/hooks/agent-context/parsers/claude.ts

@@ -0,0 +1,253 @@
+import * as fs from 'fs';
+import type {
+  AgentContext,
+  AgentParser,
+  DetectionResult,
+  InteractionTurn,
+  ModifiedFile,
+  TokenUsage,
+  ToolUseInfo,
+} from '../types.js';
+import { detectAgent } from '../detect.js';
+
+const FILE_MOD_TOOLS = new Set([
+  'Write',
+  'Edit',
+  'NotebookEdit',
+  'mcp__acp__Write',
+  'mcp__acp__Edit',
+]);
+
+interface ClaudeLine {
+  type?: string;
+  sessionId?: string;
+  version?: string;
+  gitBranch?: string;
+  uuid?: string;
+  timestamp?: string;
+  message?: {
+    id?: string;
+    model?: string;
+    role?: string;
+    content?: unknown;
+    usage?: {
+      input_tokens?: number;
+      output_tokens?: number;
+      cache_creation_input_tokens?: number;
+      cache_read_input_tokens?: number;
+    };
+  };
+}
+
+interface ContentBlock {
+  type: string;
+  text?: string;
+  thinking?: string;
+  name?: string;
+  id?: string;
+  input?: Record<string, unknown>;
+}
+
+function parseLines(raw: string): ClaudeLine[] {
+  const lines: ClaudeLine[] = [];
+  for (const line of raw.split('\n')) {
+    const trimmed = line.trim();
+    if (!trimmed) continue;
+    try {
+      lines.push(JSON.parse(trimmed));
+    } catch {
+      // Skip unparseable lines (e.g. partial write from concurrent agent)
+    }
+  }
+  return lines;
+}
+
+function extractTextContent(content: unknown): string {
+  if (typeof content === 'string') return content;
+  if (!Array.isArray(content)) return '';
+
+  const parts: string[] = [];
+  for (const block of content as ContentBlock[]) {
+    if (block.type === 'text' && block.text) {
+      parts.push(block.text);
+    }
+  }
+  return parts.join('\n');
+}
+
+function extractModifiedFiles(content: unknown): ModifiedFile[] {
+  if (!Array.isArray(content)) return [];
+  const files: ModifiedFile[] = [];
+
+  for (const block of content as ContentBlock[]) {
+    if (block.type !== 'tool_use' || !block.name) continue;
+    if (!FILE_MOD_TOOLS.has(block.name)) continue;
+
+    const filePath =
+      (block.input?.file_path as string) ||
+      (block.input?.path as string) ||
+      (block.input?.notebook_path as string);
+    if (filePath) {
+      files.push({
+        filePath,
+        toolName: block.name,
+        toolUseId: block.id,
+      });
+    }
+  }
+  return files;
+}
+
+function extractToolUses(content: unknown): ToolUseInfo[] {
+  if (!Array.isArray(content)) return [];
+  const tools: ToolUseInfo[] = [];
+
+  for (const block of content as ContentBlock[]) {
+    if (block.type !== 'tool_use' || !block.name) continue;
+    tools.push({
+      toolName: block.name,
+      toolUseId: block.id || '',
+      input: sanitizeInput(block.input || {}),
+    });
+  }
+  return tools;
+}
+
+// intentional: sanitize tool input for display in hook output, not LLM context
+function sanitizeInput(input: Record<string, unknown>): Record<string, unknown> {
+  const sanitized: Record<string, unknown> = {};
+  for (const [key, value] of Object.entries(input)) {
+    if (typeof value === 'string' && value.length > 200) {
+      sanitized[key] = value.slice(0, 200) + '... (truncated)';
+    } else {
+      sanitized[key] = value;
+    }
+  }
+  return sanitized;
+}
+
+export class ClaudeParser implements AgentParser {
+  async detect(repoPath: string): Promise<DetectionResult> {
+    return detectAgent(repoPath);
+  }
+
+  async parse(
+    sessionFilePath: string,
+    _repoPath: string,
+  ): Promise<AgentContext> {
+    const raw = fs.readFileSync(sessionFilePath, 'utf-8');
+    const allLines = parseLines(raw);
+
+    // Filter to user and assistant lines only
+    const userLines = allLines.filter((l) => l.type === 'user');
+    const assistantLines = allLines.filter((l) => l.type === 'assistant');
+
+    // Deduplicate assistant messages by message.id (streaming produces duplicates)
+    const dedupedAssistant = new Map<string, ClaudeLine>();
+    for (const line of assistantLines) {
+      const msgId = line.message?.id;
+      if (msgId) {
+        dedupedAssistant.set(msgId, line);
+      }
+    }
+    const uniqueAssistant = Array.from(dedupedAssistant.values());
+
+    // Build transcript turns
+    const transcript: InteractionTurn[] = [];
+    let turnIndex = 0;
+
+    // Interleave user and assistant messages by position
+    // User messages appear at even indices, assistant at odd
+    const maxTurns = Math.max(userLines.length, uniqueAssistant.length);
+    for (let i = 0; i < maxTurns; i++) {
+      if (i < userLines.length) {
+        const user = userLines[i];
+        transcript.push({
+          turnIndex: turnIndex++,
+          role: 'user',
+          content: extractTextContent(user.message?.content),
+          modifiedFiles: [],
+          toolUses: [],
+        });
+      }
+      if (i < uniqueAssistant.length) {
+        const asst = uniqueAssistant[i];
+        const content = asst.message?.content;
+        const usage = asst.message?.usage;
+        transcript.push({
+          turnIndex: turnIndex++,
+          role: 'assistant',
+          content: extractTextContent(content),
+          modifiedFiles: extractModifiedFiles(content),
+          tokenUsage: usage
+            ? {
+                inputTokens: usage.input_tokens || 0,
+                outputTokens: usage.output_tokens || 0,
+                cacheCreationTokens: usage.cache_creation_input_tokens,
+                cacheReadTokens: usage.cache_read_input_tokens,
+              }
+            : undefined,
+          toolUses: extractToolUses(content),
+        });
+      }
+    }
+
+    // Task prompt: first user message where content is a plain string
+    // (not a tool_result array) and not a system instruction injected by the host
+    let taskPrompt = '';
+    for (const user of userLines) {
+      const content = user.message?.content;
+      if (typeof content !== 'string') continue;
+      if (content.startsWith('<system_instruction>') || content.startsWith('<system-')) continue;
+      taskPrompt = content;
+      break;
+    }
+
+    // Aggregate token usage
+    const totalUsage: TokenUsage = {
+      inputTokens: 0,
+      outputTokens: 0,
+      cacheCreationTokens: 0,
+      cacheReadTokens: 0,
+    };
+    for (const asst of uniqueAssistant) {
+      const usage = asst.message?.usage;
+      if (usage) {
+        totalUsage.inputTokens += usage.input_tokens || 0;
+        totalUsage.outputTokens += usage.output_tokens || 0;
+        totalUsage.cacheCreationTokens! += usage.cache_creation_input_tokens || 0;
+        totalUsage.cacheReadTokens! += usage.cache_read_input_tokens || 0;
+      }
+    }
+
+    // Collect all modified files (deduplicated)
+    const allModifiedFiles = new Set<string>();
+    for (const turn of transcript) {
+      for (const f of turn.modifiedFiles) {
+        allModifiedFiles.add(f.filePath);
+      }
+    }
+
+    // Metadata from first lines
+    const firstUser = userLines[0];
+    const firstAssistant = uniqueAssistant[0];
+    const stat = fs.statSync(sessionFilePath);
+
+    return {
+      agent: 'claude-code',
+      sessionId: firstUser?.sessionId || '',
+      sessionFilePath,
+      taskPrompt,
+      modifiedFiles: Array.from(allModifiedFiles),
+      tokenUsage: totalUsage,
+      transcript,
+      metadata: {
+        model: firstAssistant?.message?.model,
+        version: firstUser?.version,
+        gitBranch: firstUser?.gitBranch,
+        startedAt: firstUser?.timestamp,
+        sessionFileModifiedAt: stat.mtime.toISOString(),
+      },
+    };
+  }
+}