@haystackeditor/cli 0.10.1 → 0.10.3

package/dist/assets/skills/setup-haystack.md

@@ -1,639 +0,0 @@
-# Haystack Verification Setup
-
-> **PRO Feature**: Verification Flow requires a Haystack PRO subscription to run cloud sandboxes. Free users can still set up the `.haystack.json` configuration — it will be ready when they upgrade. See https://haystackeditor.com/pricing for details.
-
-**Your job**: Help the verification system understand this app so it can visually verify PRs work correctly.
-
-The verification system has two parts:
-1. **Planner** - An AI that explores the codebase to figure out what to test
-2. **Executor** - Takes screenshots based on the plan
-
-Your `.haystack.json` flows feed into the **corpus** - they're hints for the Planner about what routes exist, what selectors to use, and what the core user journey looks like.
-
----
-
-## Step 0: Configure Dev Server for Cloud Sandboxes
-
-Cloud sandboxes access your dev server via tunnel URLs (`*.modal.host`). Most modern dev servers block non-localhost hosts by default. **You must configure your dev server to allow external hosts.**
-
-### Vite / SvelteKit / Astro / Nuxt 3
-
-```typescript
-// vite.config.ts (or framework equivalent)
-export default defineConfig({
-  server: {
-    allowedHosts: true, // Required for Haystack cloud sandboxes
-  },
-});
-```
-
-### Next.js
-
-Next.js allows all hosts by default in development. No configuration needed.
-
-If you've restricted hosts, add to `next.config.js`:
-```javascript
-module.exports = {
-  allowedDevHosts: ['localhost', '.modal.host'],
-};
-```
-
-### Webpack Dev Server (CRA, Vue CLI, custom)
-
-```javascript
-// webpack.config.js or vue.config.js
-module.exports = {
-  devServer: {
-    allowedHosts: 'all', // or ['.modal.host']
-  },
-};
-```
-
-For Create React App, set environment variable:
-```bash
-DANGEROUSLY_DISABLE_HOST_CHECK=true npm start
-```
-
-### Angular
-
-```json
-// angular.json → serve → options
-{
-  "disableHostCheck": true
-}
-```
-
-Or run with: `ng serve --disable-host-check`
-
-### Remix
-
-Remix dev server allows all hosts by default. No configuration needed.
-
-### Other Frameworks
-
-Look for `allowedHosts`, `disableHostCheck`, or `host` options in your dev server config. The goal is to allow requests from `*.modal.host` domains.
-
-**Skip this step if**: Your dev server already allows external hosts or doesn't have host checking.
-
----
-
-## Step 1: Generate Base Config
-
-```bash
-npx @haystackeditor/cli init --yes
-```
-
-This detects your dev server command, port, and service dependencies.
-
----
-
-## Step 2: Understand the App's Core Feature
-
-Before writing any flows, answer this:
-
-**What is the ONE main thing users do in this app?**
-
-Read the codebase to understand the core user journey:
-- E-commerce: browse products → add to cart → checkout
-- Dashboard: view metrics → filter data → export
-- Editor: create document → edit → save
-- Social: view feed → create post → interact
-- SaaS: sign up → configure → use feature
-
-Your flows should describe THIS journey, not just "pages load".
-
----
-
-## Step 3: Verify ALL Services Have Flows (CRITICAL)
-
-**Every service in `.haystack.json` must have at least one verification flow.** This is the most common mistake - adding services but forgetting to add flows that exercise them.
-
-### Check for Uncovered Services
-
-```bash
-# List all services
-grep -A1 '"services"' .haystack.json | grep -E '^\s+"[^"]+":' | sed 's/.*"\([^"]*\)".*/\1/'
-
-# List all flows
-grep '"name":' .haystack.json
-
-# MANUAL CHECK: Does every service have a flow?
-```
-
-### Types of Services
-
-| Service Type | How to Verify |
-|--------------|---------------|
-| **Web frontend** | UI flows (navigate, wait_for, screenshot) |
-| **API/Worker** | UI flows that hit API endpoints |
-| **CLI/Batch** | Backend flows with golden inputs |
-| **Analysis pipeline** | Backend flows with known PR input |
-
-### Backend Verification (CLI tools, analysis pipelines, batch jobs)
-
-**Not everything is a web page.** For CLI tools and batch processes, you need:
-
-1. **Golden input** - A known-good input that produces predictable output
-2. **Run command** - How to execute the service
-3. **Output assertion** - What to check in the output
-
-```json
-{
-  "name": "Backend pipeline - golden input",
-  "description": "Run pipeline on known input to verify it works",
-  "trigger": "on_change",
-  "watch_patterns": ["packages/my-pipeline/src/**"],
-  "service": "my-pipeline",
-  "type": "backend",
-  "steps": [
-    {
-      "action": "run",
-      "command": "pnpm start",
-      "env": { "INPUT_ID": "known-good-input-123" },
-      "timeout": 120
-    },
-    { "action": "assert_exit_code", "code": 0 },
-    { "action": "assert_output_contains", "pattern": "Processing complete" }
-  ]
-}
-```
-
----
-
-## STOP - Confirm Golden URLs and Golden Data
-
-**If you identified backend services that need verification, you MUST ask the user with a proposed example:**
-
-First, explore the codebase to find:
-- Example inputs used in tests, scripts, or documentation
-- Default/demo values in config files or environment variables
-- IDs or URLs referenced in comments or READMEs
-
-Then propose a specific example:
-
-> I found these backend services that need verification:
-> - **[service name]**: [what it does]
->
-> For backend verification, I need **golden test data** - stable inputs that produce predictable output.
->
-> **Based on what I found in the codebase, here's my suggestion:**
-> - **Golden URL/Input**: `[specific example you found, e.g., a test ID, demo endpoint, or sample file]`
-> - **Command**: `[the command to run with this input]`
-> - **Expected Output**: `[what success looks like based on the code]`
->
-> Does this look right? Or should I use different test data?
-
-**Wait for the user to confirm or provide alternatives before adding backend flows.**
-
----
-
-## Step 4: Assess Data Needs
-
-```bash
-# Find API calls
-grep -r "fetch(\|useQuery\|useSWR\|axios" src/ --include="*.tsx" --include="*.ts" | wc -l
-
-# Find dynamic routes
-grep -r "useParams\|router.query\|\[.*\]" src/ --include="*.tsx" | head -10
-
-# Find external domains
-grep -r "https://" src/ --include="*.ts" --include="*.tsx" | grep -v node_modules | head -10
-```
-
----
-
-## STOP - Ask About Auth Requirements
-
-**You MUST ask the user before proceeding with fixtures or flows:**
-
-> Does this app require signed-in users to access the main features?
->
-> Looking at the codebase, I see [authentication code, protected routes, user state, etc.].
->
-> **How should verification handle authentication?**
->
-> | Strategy | Best For | Complexity |
-> |----------|----------|------------|
-> | **Bypass** | Dev mode skips auth entirely | ⭐ Simplest |
-> | **Fixture** | Mock auth endpoints return fake user | ⭐⭐ Easy |
-> | **Token** | Inject real session token (you provide) | ⭐⭐⭐ Medium |
-> | **Test Account** | Full login flow with test credentials | ⭐⭐⭐⭐ Complex |
->
-> 1. **Bypass** - Set env var like `SKIP_AUTH=true` (does your app support this?)
-> 2. **Fixture** - I'll create a mock `/auth/status` that returns a fake user
-> 3. **Token** - You provide a session cookie/token value to inject
-> 4. **Test Account** - Automated login with username/password (most realistic)
-
-**Wait for the user's response, then configure auth accordingly.**
-
-### Configuring Auth Based on User Response
-
-**Bypass (simplest):**
-```json
-{
-  "auth": {
-    "strategy": "bypass",
-    "env": { "VITE_SKIP_AUTH": "true" }
-  }
-}
-```
-
-**Fixture (mock auth endpoint):**
-```json
-{
-  "auth": {
-    "strategy": "fixture",
-    "env": { "VITE_SKIP_AUTH": "true" },
-    "fixture": {
-      "pattern": "/auth/status",
-      "source": "file://fixtures/auth-status.json"
-    }
-  }
-}
-```
-
-Create `fixtures/auth-status.json`:
-```json
-{
-  "authenticated": true,
-  "user": {
-    "id": "test-user-123",
-    "email": "test@example.com",
-    "name": "Test User"
-  }
-}
-```
-
-**Token (inject session):**
-```json
-{
-  "auth": {
-    "strategy": "token",
-    "token": {
-      "type": "cookie",
-      "name": "session",
-      "value": "$SESSION_TOKEN"
-    }
-  }
-}
-```
-
-The `$SESSION_TOKEN` is resolved from Haystack secrets. Tell the user:
-> To use token auth, you'll need to add your session value as a secret:
-> ```bash
-> npx @haystackeditor/cli secrets set SESSION_TOKEN "your-session-value"
-> ```
-
-**Test Account (full login flow):**
-```json
-{
-  "auth": {
-    "strategy": "test_account",
-    "test_account": {
-      "username": "$TEST_USERNAME",
-      "password": "$TEST_PASSWORD",
-      "login_url": "/login",
-      "steps": [
-        { "action": "type", "selector": "input[name=email]", "value": "$TEST_USERNAME" },
-        { "action": "type", "selector": "input[name=password]", "value": "$TEST_PASSWORD" },
-        { "action": "click", "selector": "button[type=submit]" }
-      ]
-    }
-  }
-}
-```
-
----
-
-## STOP - Ask About Database Requirements
-
-**You MUST ask the user if the app needs database access:**
-
-> Does this app need a real database for verification?
->
-> Looking at the codebase, I see [Prisma/Drizzle/Knex/raw SQL queries/etc.].
->
-> **How should verification handle database access?**
->
-> | Strategy | Best For | Complexity |
-> |----------|----------|------------|
-> | **Fixture** | Mock API responses (no real DB needed) | ⭐ Simplest |
-> | **Memory** | In-memory SQLite for read-heavy apps | ⭐⭐ Easy |
-> | **Seed** | File-based DB with seed data | ⭐⭐⭐ Medium |
-> | **Docker** | Spin up Postgres/MySQL/Redis containers | ⭐⭐⭐⭐ Complex |
-> | **Remote** | Connect to read-only staging database | ⭐⭐⭐⭐⭐ Advanced |
->
-> For most apps, **fixtures are sufficient** — they mock the API layer so no real DB is needed.
-
-**Wait for the user's response, then configure database accordingly.**
-
-### Configuring Database Based on User Response
-
-**Fixture (simplest - mock API responses):**
-```json
-{
-  "fixtures": [
-    { "pattern": "/api/users", "source": "file://fixtures/users.json" },
-    { "pattern": "/api/posts/*", "source": "file://fixtures/posts.json" }
-  ]
-}
-```
-
-**Memory (in-memory SQLite):**
-```json
-{
-  "database": {
-    "strategy": "memory",
-    "env": { "DATABASE_URL": "sqlite::memory:" }
-  }
-}
-```
-
-**Seed (file-based with seed data):**
-```json
-{
-  "database": {
-    "strategy": "seed",
-    "env": { "DATABASE_URL": "file:./test.db" },
-    "seed_command": "pnpm db:seed"
-  }
-}
-```
-
-**Docker (real database containers):**
-```json
-{
-  "database": {
-    "strategy": "docker",
-    "services": [
-      {
-        "image": "postgres:15-alpine",
-        "port": 5432,
-        "env": {
-          "POSTGRES_USER": "test",
-          "POSTGRES_PASSWORD": "test",
-          "POSTGRES_DB": "testdb"
-        },
-        "health_check": "pg_isready -U test",
-        "health_timeout": 30
-      }
-    ],
-    "env": { "DATABASE_URL": "postgresql://test:test@localhost:5432/testdb" },
-    "migrate_command": "pnpm prisma migrate deploy",
-    "seed_command": "pnpm db:seed"
-  }
-}
-```
-
-**Remote (read-only staging database):**
-```json
-{
-  "database": {
-    "strategy": "remote",
-    "env": { "DATABASE_URL": "$STAGING_DATABASE_URL" },
-    "read_only": true
-  }
-}
-```
-
-The `$STAGING_DATABASE_URL` is resolved from Haystack secrets. Tell the user:
-> To use a remote database, add the connection string as a secret:
-> ```bash
-> npx @haystackeditor/cli secrets set STAGING_DATABASE_URL "postgresql://..."
-> ```
-
-**Fallback strategies** (try multiple approaches):
-```json
-{
-  "database": {
-    "strategy": "memory",
-    "env": { "DATABASE_URL": "sqlite::memory:" },
-    "fallback": {
-      "strategy": "seed",
-      "env": { "DATABASE_URL": "file:./test.db" },
-      "seed_command": "pnpm db:seed"
-    }
-  }
-}
-```
-
----
-
-## STOP - Ask About Test Data Strategy
-
-**You MUST ask the user before proceeding:**
-
-> I analyzed the codebase and found:
-> - X API fetch calls
-> - Y dynamic routes with parameters
-> - These external domains: [list them]
->
-> **How should I handle test data for verification?**
->
-> 1. **Passthrough** - Let API calls through to real servers (add domains to `network.allow`)
-> 2. **Staging URL** - Point flows at your staging/demo environment
-> 3. **Local fixtures** - JSON files in `fixtures/` directory
-> 4. **Skip data pages** - Only test static pages that don't need API data
-
-**Wait for the user's response before continuing.**
-
----
-
-## STOP - Confirm Frontend Golden URLs
-
-**Goldens are URLs with real staging data** - they let verification test pages that need actual content to render properly. You need MULTIPLE goldens to cover different scenarios.
-
-**Before writing flows for dynamic routes, explore the codebase to find example data:**
-
-1. Check test files (`*.test.ts`, `cypress/`, `playwright/`) for example IDs/URLs
-2. Check README/docs for demo links or example usage
-3. Check seed data, fixtures, or mock files
-4. Check `.env.example` or config for staging URLs
-5. Check existing routes to understand the URL format
-
-**Then propose MULTIPLE golden URLs (aim for 3-5) covering different scenarios:**
-
-> I found these dynamic routes that need staging data to render:
->
-> **[Route Name]** (`/path/:param1/:param2`)
->
-> | Golden | URL | Why |
-> |--------|-----|-----|
-> | Large dataset | `/path/example/large-123` | Tests performance with many items |
-> | Typical case | `/path/example/medium-456` | Common user scenario |
-> | Edge case | `/path/example/empty-789` | Tests empty/minimal state |
-> | Error state | `/path/example/error-000` | Tests error handling UI |
->
-> **Evidence:**
-> - Found `large-123` in `cypress/fixtures/test-data.json`
-> - Found `medium-456` in README as demo example
-> - Found `empty-789` in `src/tests/edge-cases.test.ts`
->
-> Does this look right?
-> 1. ✅ Yes, use these goldens
-> 2. 🔄 Use different URLs (I'll provide them)
-
-**Add goldens to `verification.goldens` in `.haystack.json` using this EXACT format:**
-
-```json
-{
-  "verification": {
-    "goldens": {
-      "frontend": [
-        {
-          "id": "large-dataset",
-          "description": "100+ items - tests rendering performance",
-          "route": "/the/actual/url/with/data"
-        },
-        {
-          "id": "typical-case",
-          "description": "Standard 10-20 items",
-          "route": "/another/url/with/typical/data"
-        },
-        {
-          "id": "empty-state",
-          "description": "No items - tests empty state UI",
-          "route": "/url/that/shows/empty/state"
-        }
-      ],
-      "backend": [
-        {
-          "id": "golden-input-1",
-          "description": "Known-good input for pipeline testing",
-          "input": { "type": "github_pr", "value": "owner/repo#123" }
-        }
-      ]
-    },
-    "commands": [...]
-  }
-}
-```
-
-**DO NOT:**
-- Ask the user what format to use - USE THIS FORMAT
-- Put goldens at the top level - they go inside `verification.goldens`
-- Use a flat object like `{ "goldens": { "name": "url" } }` - use the array format above
-- Embed golden URLs only in flow steps - also add them to the goldens section for discoverability
-
-**CRITICAL**:
-- Propose MULTIPLE goldens (3-5), not just one
-- Cover different scenarios: large, typical, empty, error states
-- Always propose SPECIFIC URLs, never ask the user to freestyle
-- Show WHERE you found the example data (file path, line if possible)
-- Make sure URLs match the app's actual route format (check `<Route path=`)
-- Wait for confirmation before writing flows with these URLs
-
----
-
-## Step 5: Write Flows
-
-Flows tell the Planner about your app's routes, UI elements, and user journeys.
-
-### Structure
-
-```yaml
-flows:
-  - name: "Descriptive name of what this tests"
-    trigger: always  # or on_change with watch_patterns
-    steps:
-      - action: navigate
-        url: "/"
-      - action: wait_for
-        selector: "[data-testid='specific-element']"
-      - action: click
-        selector: "[data-testid='button']"
-      - action: screenshot
-        name: "result"
-```
-
-### What to Include
-
-**Core journey flow** (trigger: always):
-- The main thing users do in your app
-- Multiple steps with interactions (click, type)
-- Waits for meaningful state changes
-
-**Route coverage flows**:
-- One flow per major route
-- Uses specific selectors the Planner can learn from
-- `watch_patterns` to only run when relevant files change
-
-### Finding Good Selectors
-
-```bash
-# Find data-testid attributes
-grep -r "data-testid" src/ --include="*.tsx" | head -20
-
-# Find aria-labels
-grep -r "aria-label" src/ --include="*.tsx" | head -20
-
-# Find component class names
-grep -r "className=" src/components/ --include="*.tsx" | head -20
-```
-
-Use specific selectors like:
-- `[data-testid='dashboard-chart']`
-- `[aria-label='Submit form']`
-- `.pricing-table`
-- `button[type='submit']`
-
-If good selectors don't exist, add `data-testid` to key components.
-
-### watch_patterns
-
-For flows that only matter when certain files change:
-
-```yaml
-- name: "Settings page works"
-  trigger: on_change
-  watch_patterns:
-    - "src/pages/settings/**"
-    - "src/components/settings/**"
-  steps:
-    - action: navigate
-      url: "/settings"
-    # ...
-```
-
----
-
-## Step 6: Configure Fixtures (if needed)
-
-Based on user's answer in Step 3:
-
-**Passthrough**:
-```yaml
-network:
-  allow:
-    - "api.example.com"
-    - "cdn.example.com"
-```
-
-**Staging URL**:
-```yaml
-flows:
-  - name: "Dashboard with real data"
-    steps:
-      - action: navigate
-        url: "https://staging.example.com/dashboard"
-```
-
-**Local fixtures**:
-```yaml
-fixtures:
-  - pattern: "/api/user"
-    source: "file://fixtures/user.json"
-  - pattern: "/api/data/*"
-    source: "file://fixtures/data.json"
-```
-
----
-
-## Step 7: Commit
-
-```bash
-git add .haystack.json fixtures/
-git commit -m "Add Haystack verification"
-```
-
-Done! The Planner will use your flows to understand the app and create verification plans for PRs.

package/dist/assets/templates/CLAUDE.md.snippet

@@ -1,42 +0,0 @@
-# Haystack Verification
-
-This project uses Haystack for PR verification. Configuration is in `.haystack.yml`.
-
-## For the AI Agent
-
-When making changes to this codebase:
-
-1. **After completing changes**, run verification:
-   - Use the `haystack_get_verify_targets` tool to see what URLs to check
-   - Use `haystack_get_auth_config` to get auth bypass instructions
-   - Start the dev server with auth bypass env vars
-   - Navigate to each verification URL and confirm it loads
-
-2. **If `.haystack.yml` doesn't exist**, help the user create one:
-   - Use `haystack_setup_start` to begin the wizard
-   - Ask each question and call `haystack_setup_answer` with responses
-
-3. **Fixture data** is available via `haystack_load_fixtures`:
-   - This loads test data from staging/local files
-   - Use this instead of real API calls when verifying
-
-## Quick Reference
-
-```yaml
-# .haystack.yml structure
-dev:
-  command: pnpm dev      # How to start dev server
-  port: 3000             # What port it runs on
-
-auth:
-  strategy: bypass       # How to skip auth
-  env: "SKIP_AUTH=true"  # Env var to set
-
-fixtures:                # Test data sources
-  - pattern: "/api/*"
-    source: "https://staging.example.com/api"
-
-verify:                  # Pages to check
-  - url: "/"
-    wait_for: "#app"
-```