@haystackeditor/cli 0.8.1 → 0.10.0

package/README.md

@@ -1,68 +1,43 @@
 # @haystackeditor/cli
 
-Set up Haystack verification for your project. When PRs are opened, an AI agent spins up your app in a sandbox and verifies changes work correctly.
+Set up Haystack for your project. When PRs are opened, Haystack automatically analyzes them for bugs, intent drift, and rule violations.
 
 ## Quick Start
 
-### Fastest: AI-Assisted Setup
-
-```bash
-npx @haystackeditor/cli skills install
-```
-
-This auto-detects your coding CLI (Claude Code, Codex, Cursor) and installs the Haystack MCP server.
-
-Then invoke the setup in your coding CLI:
-
-| CLI | How to invoke |
-|-----|---------------|
-| **Claude Code** | `/setup-haystack` |
-| **Codex CLI** | `/setup-haystack` or ask "set up haystack verification" |
-| **Cursor** | `/setup-haystack` (in Composer) |
-
-The AI will analyze your codebase, create `.haystack.json`, and configure verification flows.
-
-### Alternative: Direct CLI Setup
-
-For quick setup without AI assistance:
-
 ```bash
-npx @haystackeditor/cli init
+npm install -g @haystackeditor/cli
+haystack login
+haystack setup
 ```
 
-This runs an interactive wizard that auto-detects your project and creates `.haystack.json`.
-
-### Additional Skills
+The `setup` command walks you through an interactive wizard:
 
-| Skill | Purpose |
-|-------|---------|
-| `/setup-haystack` | **Start here** - diagnoses project, creates config |
-| `/prepare-haystack` | Add aria-labels and data-testid for browser automation |
-| `/setup-haystack-secrets` | Configure API keys, LLM credentials, secrets |
+1. **Select repositories** to configure
+2. **Scan for coding rules** (conventions your team follows)
+3. **Scan for CI/bot signals** (checks to wait for before merging)
+4. **Scan for review policies** (who should review what)
+5. **Review and toggle** discovered items
+6. **Write `.haystack.json`** to your repos
 
 ---
 
 ## CLI Commands
 
-All commands can be run directly:
-
-### `haystack skills`
+### `haystack setup`
 
-Install AI skills into your coding CLI (auto-detects Claude Code, Codex, Cursor):
+Interactive onboarding wizard — scan your repos and generate `.haystack.json`:
 
 ```bash
-npx @haystackeditor/cli skills install              # Auto-detect CLI
-npx @haystackeditor/cli skills install --cli codex  # Install for Codex
-npx @haystackeditor/cli skills install --cli cursor # Install for Cursor
-npx @haystackeditor/cli skills install --cli manual # Show manual setup
-npx @haystackeditor/cli skills list                 # List available skills
+haystack setup
 ```
 
 ### `haystack init`
 
+Quick local setup — auto-detects your project and creates `.haystack.json` without scanning:
+
 ```bash
-npx @haystackeditor/cli init       # Interactive wizard
-npx @haystackeditor/cli init -y    # Accept all defaults
+haystack init              # Auto-detect and create config
+haystack init --force      # Overwrite existing .haystack.json
 ```
 
 ### `haystack status`
@@ -70,22 +45,22 @@ npx @haystackeditor/cli init -y    # Accept all defaults
 Check if your project is configured:
 
 ```bash
-npx @haystackeditor/cli status
+haystack status
 ```
 
 ### `haystack login`
 
-Authenticate with GitHub (required for secrets management):
+Authenticate with GitHub (required for setup and secrets):
 
 ```bash
-npx @haystackeditor/cli login
+haystack login
 ```
 
 This uses GitHub's device flow - you'll get a code to enter at github.com/login/device.
 
 ```bash
 # Log out (removes stored credentials)
-npx @haystackeditor/cli logout
+haystack logout
 ```
 
 ### `haystack secrets`
@@ -94,13 +69,13 @@ Manage secrets that will be injected into your sandbox environment:
 
 ```bash
 # List all secrets (keys only, values are never shown)
-npx @haystackeditor/cli secrets list
+haystack secrets list
 
 # Set a secret
-npx @haystackeditor/cli secrets set OPENAI_API_KEY sk-xxx
+haystack secrets set OPENAI_API_KEY sk-xxx
 
 # Delete a secret
-npx @haystackeditor/cli secrets delete OPENAI_API_KEY
+haystack secrets delete OPENAI_API_KEY
 ```
 
 Secrets are encrypted and stored securely. They're automatically injected as environment variables when the sandbox runs your app.
@@ -109,59 +84,90 @@ Secrets are encrypted and stored securely. They're automatically injected as env
 
 ```bash
 # Org-scoped (available to all repos in the org)
-npx @haystackeditor/cli secrets set API_KEY xxx --scope org --scope-id myorg
+haystack secrets set API_KEY xxx --scope org --scope-id myorg
 
 # Repo-scoped (available only to this repo)
-npx @haystackeditor/cli secrets set API_KEY xxx --scope repo --scope-id owner/repo
+haystack secrets set API_KEY xxx --scope repo --scope-id owner/repo
 ```
 
----
+### `haystack submit`
 
-## Configuration
+Create a PR from current changes. Runs pre-PR triage (code review, rules validation, intent drift), pushes your branch, and opens the PR.
 
-The `init` command creates `.haystack.json`:
-
-```yaml
-version: "1"
-name: my-app
-
-dev_server:
-  command: pnpm dev
-  port: 3000
-  ready_pattern: "Local:"
-  env:
-    SKIP_AUTH: "true"
-
-verification:
-  commands:
-    - name: build
-      run: pnpm build
-    - name: lint
-      run: pnpm lint
+```bash
+haystack submit                      # Triage -> create PR -> wait for analysis
+haystack submit --title "Fix auth"   # Custom PR title
+haystack submit --draft              # Create as draft PR
+haystack submit --force              # Skip triage checks
+haystack submit --no-wait            # Don't wait for analysis results
 ```
 
-### Customizing After Init
+**Review routing**: By default, PRs go to the auto-merge queue -- if analysis passes, the PR is merged automatically. Use `--review` to route it for human review instead:
 
-| If your app has... | Add this |
-|-------------------|----------|
-| Login/authentication | Auth bypass env var in `dev_server.env` |
-| Key user journeys | Flows describing what to verify |
-| API calls needing auth | Fixtures to mock responses |
+```bash
+haystack submit --review             # Needs review (goes to assignment queue)
+haystack submit --review octocat     # Request review from a specific teammate
+```
 
-Use `/setup-haystack` in Claude Code for AI-assisted configuration of flows and fixtures.
+When `--review` is used without a username, the PR is labeled `haystack:needs-review` and appears in your team's assignment queue. When a username is provided, that person is also requested as a reviewer on GitHub.
+
+### `haystack triage`
+
+View Haystack analysis results for any PR. Shows the same data as the Haystack web feed: rating, verdict, structured findings with details, verified bugs, human review reasons, and agent fix prompts.
+
+```bash
+haystack triage 42                          # Current repo, PR #42
+haystack triage owner/repo#99               # Fully qualified
+haystack triage https://github.com/o/r/pull/1  # From GitHub URL
+haystack triage 42 --json                   # Machine-readable JSON output
+haystack triage 42 --no-wait                # Don't wait if analysis is pending
+```
+
+The `--json` output includes `agentFixPrompt` fields -- ready-to-paste instructions for coding agents to fix each finding.
+
+### `haystack check-pending`
+
+Check the status of a previously submitted PR's analysis (from the current repo):
+
+```bash
+haystack check-pending              # Rich terminal output
+haystack check-pending --hook       # Minimal output (for session-start hooks)
+haystack check-pending --json       # Machine-readable JSON
+haystack check-pending --clear      # Clear pending state
+```
+
+---
+
+## Configuration
+
+The `setup` wizard writes `.haystack.json` to your repos with discovered rules, signals, and policies. You can also create a base config locally with `haystack init`:
+
+```json
+{
+  "version": "1",
+  "name": "my-app",
+  "dev_server": {
+    "command": "pnpm dev",
+    "port": 3000,
+    "ready_pattern": "Local:",
+    "env": {
+      "SKIP_AUTH": "true"
+    }
+  }
+}
+```
 
 ---
 
 ## How It Works
 
-1. Run `npx @haystackeditor/cli init` or use `/setup-haystack` in Claude Code
-2. Commit the generated `.haystack.json`
-3. Install the [Haystack GitHub App](https://haystackeditor.com/github-app)
-4. When PRs are opened, Haystack's AI agent:
-   - Spins up your app in a cloud sandbox
-   - Runs verification commands
-   - Captures screenshots and evidence
+1. Run `haystack setup` to configure your repos (or `haystack init` for local-only config)
+2. Install the [Haystack GitHub App](https://haystackeditor.com/github-app)
+3. When PRs are opened, Haystack automatically:
+   - Analyzes the code for bugs, intent drift, and rule violations
    - Reports results on the PR
+   - Routes the PR to the right inbox tab (Good to Merge, Issues Found, etc.)
+   - If auto-merge is enabled, clean PRs merge automatically
 
 ## License
 

package/dist/assets/hooks/llm-rules-template.md

@@ -2,6 +2,27 @@
 
 These rules govern how LLM agents should interact with this codebase. They are enforced by a pre-commit hook.
 
+## Creating Pull Requests
+
+**Always use `haystack submit` instead of `gh pr create` or GitHub API.**
+
+```bash
+# Auto-merge queue (default) - PR merges automatically if Haystack approves
+haystack submit
+
+# Request Haystack review - for complex/uncertain changes
+haystack submit --review
+```
+
+This routes your PR through the Haystack auto-merge queue:
+1. Haystack analyzes your changes
+2. If approved: auto-merged
+3. If issues found: routed for review with feedback
+
+Do NOT use `gh pr create` - it bypasses the auto-merge queue.
+
+After submitting, use `haystack triage <pr-number>` to check analysis results (rating, findings, agent fix prompts). Use `--json` for machine-readable output.
+
 - When you write or edit LLM prompts in response to a failure running against some test data, be certain that your edit is not specific to the test data, and is actually addressing the more generic problem that caused the failure.
 - If the user specifies a model, follow it. Your understanding of current LLM models is outdated.
 - Do not add silent fallbacks.

package/dist/assets/hooks/package.json

@@ -5,7 +5,7 @@
   "type": "module",
   "description": "Git hooks for AI agent quality checks (installed by @haystackeditor/cli)",
   "dependencies": {
-    "tree-sitter": "^0.22.4",
-    "tree-sitter-typescript": "^0.23.2"
+    "tree-sitter": "0.22.4",
+    "tree-sitter-typescript": "0.23.2"
   }
 }

package/dist/assets/hooks/scripts/pre-push.sh

@@ -2,4 +2,24 @@
 # Haystack hooks (powered by Entire)
 # Pre-push hook: push session logs alongside user's push
 # $1 is the remote name (e.g., "origin")
+
+# Check if pushing a non-haystack branch and remind about haystack submit
+current_branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null)
+if [ -n "$current_branch" ] && [ "$current_branch" != "main" ] && [ "$current_branch" != "master" ]; then
+  case "$current_branch" in
+    haystack/*)
+      # Already a haystack branch, no reminder needed
+      ;;
+    *)
+      # Check if this is an AI agent session (set by agent-context detector)
+      if [ -n "$HAYSTACK_AGENT_SESSION" ] || [ -n "$CLAUDE_CODE" ] || [ -n "$CODEX_CLI" ]; then
+        echo ""
+        echo "💡 Tip: Use 'haystack submit' to create PRs through the auto-merge queue."
+        echo "   This enables automatic analysis and merge when approved."
+        echo ""
+      fi
+      ;;
+  esac
+fi
+
 "$HOME/.haystack/bin/entire" hooks git pre-push "$1" || true

package/dist/assets/skills/prepare-haystack.md

@@ -0,0 +1,323 @@
+# Prepare Codebase for Verification
+
+> **PRO Feature**: Verification Flow requires a Haystack PRO subscription to run cloud sandboxes. Free users can still prepare their codebase — it will be ready when they upgrade. See https://haystackeditor.com/pricing for details.
+
+**Your job**: Make this codebase easy to verify by adding semantic identifiers that the verification system can target.
+
+The verification Planner needs to find UI elements by selectors. Generic selectors like `div` or `.flex` are useless. Your job is to add meaningful identifiers throughout the codebase.
+
+---
+
+## What to Add
+
+### 1. `aria-label` on Interactive Elements
+
+Every clickable/interactive element should have an aria-label describing what it does:
+
+```tsx
+// Before
+<button onClick={onSave}>💾</button>
+<button onClick={() => setOpen(true)}>
+  <MenuIcon />
+</button>
+
+// After
+<button onClick={onSave} aria-label="Save document">💾</button>
+<button onClick={() => setOpen(true)} aria-label="Open menu">
+  <MenuIcon />
+</button>
+```
+
+**Target elements:**
+- Buttons (especially icon-only buttons)
+- Links without descriptive text
+- Toggle switches
+- Dropdown triggers
+- Modal open/close buttons
+- Form submit buttons
+
+### 2. `data-testid` on Key Sections
+
+Major UI sections should have data-testid for easy targeting:
+
+```tsx
+// Before
+<div className="flex flex-col p-4">
+  <h1>Dashboard</h1>
+  {/* content */}
+</div>
+
+// After
+<div className="flex flex-col p-4" data-testid="dashboard-container">
+  <h1>Dashboard</h1>
+  {/* content */}
+</div>
+```
+
+**Target sections:**
+- Page containers (dashboard, settings, profile)
+- Navigation bars/sidebars
+- Modal/dialog content
+- Form containers
+- Data tables/lists
+- Card components
+- Loading states
+- Error states
+- Empty states
+
+### 3. `role` Attributes for Semantic Structure
+
+Add ARIA roles where HTML semantics aren't clear:
+
+```tsx
+// Before
+<div className="modal-overlay">
+  <div className="modal-content">
+
+// After
+<div className="modal-overlay" role="presentation">
+  <div className="modal-content" role="dialog" aria-modal="true">
+```
+
+**Common roles:**
+- `role="dialog"` - Modals/dialogs
+- `role="navigation"` - Nav sections
+- `role="main"` - Main content area
+- `role="alert"` - Error/success messages
+- `role="status"` - Loading indicators
+- `role="tablist"`, `role="tab"`, `role="tabpanel"` - Tab interfaces
+
+### 4. State Indicators
+
+Add attributes that indicate UI state:
+
+```tsx
+// Before
+<button onClick={toggle}>
+  {isOpen ? 'Close' : 'Open'}
+</button>
+
+// After
+<button
+  onClick={toggle}
+  aria-expanded={isOpen}
+  aria-label={isOpen ? 'Close panel' : 'Open panel'}
+>
+  {isOpen ? 'Close' : 'Open'}
+</button>
+```
+
+**State attributes:**
+- `aria-expanded` - Collapsible sections
+- `aria-selected` - Selected items in lists
+- `aria-checked` - Checkboxes/toggles
+- `aria-disabled` - Disabled elements
+- `aria-busy` - Loading states
+- `data-state="loading|error|success"` - Custom states
+
+### 5. Form Accessibility
+
+Forms should have proper labels and descriptions:
+
+```tsx
+// Before
+<input type="email" placeholder="Email" />
+<span className="text-red-500">{error}</span>
+
+// After
+<input
+  type="email"
+  placeholder="Email"
+  aria-label="Email address"
+  aria-describedby={error ? "email-error" : undefined}
+  aria-invalid={!!error}
+/>
+{error && <span id="email-error" role="alert" className="text-red-500">{error}</span>}
+```
+
+---
+
+## Step 1: Scan for Missing Identifiers
+
+```bash
+# Find buttons without aria-label
+grep -rn "<button" src/ --include="*.tsx" | grep -v "aria-label" | head -20
+
+# Find icon-only buttons (likely missing labels)
+grep -rn "<button.*Icon\|<button.*>.*</.*Icon>" src/ --include="*.tsx" | head -20
+
+# Find modals/dialogs without role
+grep -rn "modal\|dialog\|Modal\|Dialog" src/ --include="*.tsx" | grep -v "role=" | head -20
+
+# Find forms without proper labeling
+grep -rn "<input\|<select\|<textarea" src/ --include="*.tsx" | grep -v "aria-label\|id=" | head -20
+
+# Find major components (likely need data-testid)
+ls src/components/ src/pages/ 2>/dev/null
+```
+
+---
+
+## Step 2: Prioritize by Impact
+
+Focus on elements the verification system is most likely to need:
+
+**High Priority (do first):**
+1. Navigation elements (header, sidebar, menu buttons)
+2. Primary actions (submit buttons, save buttons, CTAs)
+3. Modal triggers and dialogs
+4. Form inputs and submit buttons
+5. Page-level containers
+
+**Medium Priority:**
+1. Toggle switches and checkboxes
+2. Dropdown menus
+3. Tab interfaces
+4. Cards and list items
+5. Loading/error states
+
+**Lower Priority:**
+1. Decorative elements
+2. Static content sections
+3. Footer links
+
+---
+
+## Step 3: Add Identifiers Systematically
+
+Go component by component. For each component:
+
+1. **Check the component's purpose** - What does it DO?
+2. **Add aria-label** to interactive elements describing the ACTION
+3. **Add data-testid** to the container if it's a major section
+4. **Add role** if the semantic HTML isn't clear
+5. **Add state attributes** if the component has dynamic states
+
+### Naming Conventions
+
+**aria-label**: Describe the action, not the element
+- ✅ `aria-label="Close modal"`
+- ✅ `aria-label="Submit contact form"`
+- ❌ `aria-label="Button"`
+- ❌ `aria-label="Click here"`
+
+**data-testid**: Use kebab-case, describe the section
+- ✅ `data-testid="user-profile-card"`
+- ✅ `data-testid="search-results-list"`
+- ❌ `data-testid="div1"`
+- ❌ `data-testid="container"`
+
+---
+
+## Step 4: Verify Coverage
+
+After adding identifiers, check coverage:
+
+```bash
+# Count aria-labels added
+grep -r "aria-label" src/ --include="*.tsx" | wc -l
+
+# Count data-testid added
+grep -r "data-testid" src/ --include="*.tsx" | wc -l
+
+# Count role attributes
+grep -r "role=" src/ --include="*.tsx" | wc -l
+
+# List all data-testid values (check for meaningful names)
+grep -oh 'data-testid="[^"]*"' src/ -r --include="*.tsx" | sort -u
+```
+
+---
+
+## Step 5: Commit
+
+```bash
+git add src/
+git commit -m "Add accessibility attributes for verification
+
+- Added aria-labels to interactive elements
+- Added data-testid to major sections
+- Added ARIA roles for semantic structure
+- Added state indicators (aria-expanded, etc.)"
+```
+
+---
+
+## Quick Reference: Common Patterns
+
+### Icon Button
+```tsx
+<button onClick={onAction} aria-label="Descriptive action name">
+  <Icon />
+</button>
+```
+
+### Modal
+```tsx
+<div role="dialog" aria-modal="true" aria-labelledby="modal-title" data-testid="settings-modal">
+  <h2 id="modal-title">Settings</h2>
+  <button onClick={onClose} aria-label="Close settings">×</button>
+</div>
+```
+
+### Navigation
+```tsx
+<nav aria-label="Main navigation" data-testid="main-nav">
+  <a href="/dashboard" aria-current={isActive ? "page" : undefined}>Dashboard</a>
+</nav>
+```
+
+### Toggle
+```tsx
+<button
+  onClick={toggle}
+  aria-pressed={isOn}
+  aria-label={`${isOn ? 'Disable' : 'Enable'} notifications`}
+>
+  {isOn ? 'On' : 'Off'}
+</button>
+```
+
+### Loading State
+```tsx
+<div data-testid="content-area" aria-busy={isLoading}>
+  {isLoading ? (
+    <div role="status" aria-label="Loading content">
+      <Spinner />
+    </div>
+  ) : (
+    content
+  )}
+</div>
+```
+
+### Form Field
+```tsx
+<div data-testid="email-field">
+  <label htmlFor="email">Email</label>
+  <input
+    id="email"
+    type="email"
+    aria-describedby={error ? "email-error" : "email-hint"}
+    aria-invalid={!!error}
+  />
+  <span id="email-hint">We'll never share your email</span>
+  {error && <span id="email-error" role="alert">{error}</span>}
+</div>
+```
+
+### Expandable Section
+```tsx
+<div data-testid="faq-section">
+  <button
+    onClick={() => setExpanded(!expanded)}
+    aria-expanded={expanded}
+    aria-controls="faq-content"
+  >
+    FAQ
+  </button>
+  <div id="faq-content" hidden={!expanded}>
+    {content}
+  </div>
+</div>
+```