@haystackeditor/cli 0.25.1 → 0.27.0

package/dist/assets/skills/map-your-system.md

@@ -1,143 +0,0 @@
-# Map Your System for Haystack QA
-
-Create `.haystack/system.yml` — a short file of facts about how this
-repository's system works, so Haystack QA can run it.
-
-## Why this file exists
-
-When Haystack reviews a risky PR, it doesn't just read the diff — it runs the
-affected code in a sandbox and captures evidence. Haystack decides **what** to
-test from each PR on its own. What it can't always infer is **how your system
-works**: how to install and build it, how services boot, how to know they're
-ready, how to seed data, how to act as a logged-in test user. That is what
-this file provides. Every fact you add converts risks Haystack identified but
-couldn't execute into checks that actually run.
-
-## Hard rules
-
-1. **Facts only — never test selection.** The file describes how the system
-   works. Keys like `scenarios`, `tests`, `skip`, or `focus` are rejected;
-   Haystack alone decides what to test.
-2. **Only verified facts.** Run every command yourself before declaring it.
-   A wrong fact is worse than a missing one — Haystack tracks which declared
-   facts fail and will flag them on PRs.
-3. **No secrets.** Reference environment variable *names* if needed, never
-   values. Anything that looks like a credential is rejected.
-4. **Personas are seeded FAKE users** reachable through dev/test-only login
-   paths. Never real accounts, never customer data. If a dev login helper
-   exists, it must be disabled in production builds.
-5. **Small and true beats big and aspirational.** Ten verified lines are more
-   useful than sixty guesses. Omit anything you could not verify.
-
-## Steps
-
-### 1. Inspect the repository
-
-Determine, with evidence from the tree (lockfiles, manifests, scripts,
-docker-compose, CI config, READMEs):
-
-- package manager, install command, and any required build/codegen steps
-- each runnable service: start command, port, and how "ready" is observable
-  (health endpoint, log line, or a command that exits 0)
-- how to seed and reset local/test data
-- how existing tests authenticate — is there a seeded test user or dev login?
-- the test runner and any test helpers/factories generated tests should reuse
-- external services the code calls, and how development runs without hitting
-  them for real (mock fixture, log driver, sandbox mode)
-- where runtime logs are written
-
-### 2. Verify each fact by running it
-
-From a clean state where possible: run the install, run the build, boot each
-service and confirm its readiness signal, run the seed command, run the login
-helper. Record the exact commands that worked. Drop anything you could not
-make work — do not declare it with a caveat.
-
-### 3. Ask the developer only what you cannot determine
-
-Typical questions worth asking a human: "Is there a dev-only way to log in as
-a test user?", "Which services must be running for the API to work?", "Is
-there a canonical seed script?". Keep it to the few facts you genuinely
-cannot infer or verify.
-
-### 4. Write `.haystack/system.yml`
-
-```yaml
-version: 1
-
-environment:
-  setup:
-    - run: "pnpm install --frozen-lockfile"
-    - run: "pnpm codegen"
-      rationale: "generated types are imported throughout"
-  toolchain: { node: "20", pnpm: "10" }
-
-services:
-  api:
-    root: "apps/api"                                # optional
-    boot: "pnpm dev"
-    ready: { http: "http://localhost:8080/health" } # or { log_pattern: "listening" } or { command: "pg_isready" }
-    env: { APP_ENV: "test" }
-    depends_on: [postgres]
-
-data:
-  seed: "pnpm db:seed --profile=qa"
-  reset: "pnpm db:reset"
-  personas:
-    admin:
-      login: "pnpm test:login admin"
-      yields: "prints an Authorization header on stdout"
-
-observability:
-  logs: ["apps/api/log/dev.log"]
-  inspect_db: "pnpm db:console --readonly"
-
-external_services:
-  stripe:
-    offline: "repo ships a stripe-mock fixture used by tests"
-    must_not_touch_live: true
-
-testing:
-  runner: "pnpm vitest"
-  probe_helpers: "tests/helpers/ has request builders and DB factories"
-
-constraints:
-  - "full build takes ~6 min; apps/api boots standalone without the web build"
-```
-
-Every section is optional — include only what you verified. All commands run
-from the repo root unless a `cwd`/`root` says otherwise.
-
-### 5. Validate before opening the PR
-
-Run the validator — it applies the same rules Haystack QA applies:
-
-```bash
-npx @haystackeditor/cli system-map validate
-```
-
-Fix every reported error, then confirm the things the validator cannot check:
-
-- [ ] every `run`/`boot`/`login`/`seed` command was executed and worked
-- [ ] every service has a `ready` signal you observed
-- [ ] personas are seeded fake users via dev/test-only paths
-
-Haystack re-validates the file on its next QA run and reports precise errors
-if something is wrong.
-
-### 6. Open a small, reviewable PR
-
-Add only `.haystack/system.yml` (plus a dev login helper if one was genuinely
-needed — dev/test-gated, with a test proving it's off in production). In the
-PR description, list which facts you verified by running them and which came
-from the developer's answers.
-
-## What happens next
-
-Haystack treats each entry as a claim. When a QA run executes a declared
-command successfully, the fact is marked **verified** with the run's receipt;
-if it fails at a newer commit it is marked **broken**, and Haystack will ask
-about it on a PR instead of trusting it. When QA hits something the map
-doesn't cover, it asks at most a couple of targeted questions on the affected
-PR, each with a ready-to-paste snippet — answering them grows this file over
-time.

package/dist/assets/skills/submit.md

@@ -1,200 +0,0 @@
-# Submit PR via Haystack
-
-**Your job**: Submit your changes as a PR using `haystack submit` instead of `gh pr create` or GitHub API.
-
----
-
-## Why Use Haystack Submit?
-
-When you create PRs through Haystack, the CLI keeps PR creation inside Haystack's analysis workflow:
-
-1. **Runs pre-PR triage** on your current branch
-2. **Pushes that branch** to origin
-3. **Creates the PR** on GitHub
-4. **Waits for Haystack analysis** by default
-5. **Applies auto-merge** only if the repo has it enabled in `.haystack.json`
-
----
-
-## Command Reference
-
-```bash
-# Default: Submit current branch and create the PR
-haystack submit
-
-# Optional alpha auto-fix for straightforward mechanical issues
-haystack submit --auto-fix
-
-# Request Haystack review (for complex/risky changes)
-haystack submit --review
-
-# With custom title
-haystack submit --title "Fix authentication bug"
-
-# Target different base branch
-haystack submit --base develop
-
-# With markdown PR body (write to file first to preserve formatting)
-cat > /tmp/pr-body.md << 'EOF'
-## Summary
-- Fixed authentication bug in session handling
-- Added integration tests for token refresh
-
-## Test plan
-- [x] Auth flow works end-to-end
-EOF
-haystack submit --body-file /tmp/pr-body.md
-```
-
-> **Tip**: Always use `--body-file` instead of `--body` when the description contains markdown.
-> The `--body` flag is a single CLI argument that mangles newlines and special characters (`#`, `*`, etc.).
-
----
-
-## When to Use Each Mode
-
-### Default Submit
-
-Use for straightforward changes that don't need explicit human review:
-
-- Bug fixes with clear scope
-- Small features with tests
-- Refactoring with no behavior change
-- Documentation updates
-- Dependency updates
-
-```bash
-haystack submit
-```
-
-If the repo has `preferences.auto_merge` enabled in `.haystack.json`, the PR will also enter the auto-merge queue.
-
-### Auto-Fix (Alpha, Discouraged)
-
-> **Do NOT default to `--auto-fix`.** It is an alpha feature.
-> Use it only when the user explicitly wants to opt in, or when they are
-> comfortable using the sandbox fixer for clearly straightforward mechanical issues.
-
-Use `--auto-fix` only for issues like:
-
-- Missing null checks
-- Obvious typos or lint fixes
-- Simple deterministic bugs with one clear fix
-
-```bash
-haystack submit --auto-fix
-```
-
-Repos that want auto-fix on by default for every `haystack submit` can opt
-in via `.haystack.json`:
-
-```json
-{
-  "preferences": {
-    "auto_fix": true
-  }
-}
-```
-
-When set, `haystack submit` applies the `haystack:auto-fix` label without
-needing the explicit flag. Auto-fix is still alpha — use the repo-level
-opt-in only when the team is comfortable with the sandbox fixer running on
-mechanical findings. `haystack submit --no-auto-fix` overrides the repo
-default for a single submission.
-
-### Review Mode
-
-> **Do NOT use `--review` unless the user explicitly asks for human review.**
-> Plain `haystack submit` already runs full Haystack analysis. Using `--review`
-> blocks auto-merge when auto-merge is enabled, which delays merging unnecessarily.
-
-Use `--review` only when the user specifically requests human review:
-
-- User says "get someone to review this" or "needs human eyes"
-- User explicitly passes `--review` in their instructions
-
-```bash
-haystack submit --review
-```
-
----
-
-## Workflow
-
-1. **Make your changes** on a feature branch
-2. **Commit** your changes: `git add . && git commit -m "..."`
-3. **Submit** via Haystack: `haystack submit`
-
-Haystack will:
-- Push your current branch to origin
-- Create the PR with appropriate labels
-- Trigger analysis automatically
-
----
-
-## Prerequisites
-
-Before using `haystack submit`, ensure you're logged in:
-
-```bash
-haystack login
-```
-
-This uses GitHub device flow - no tokens to copy/paste.
-
----
-
-## Important Notes
-
-- **Always use `haystack submit`** instead of `gh pr create` when working in Haystack-enabled repos
-- Prefer plain `haystack submit` by default; use `--auto-fix` only as an explicit alpha opt-in for straightforward mechanical issues
-- The command uses your current branch as the PR head
-- Labels are applied automatically based on your chosen mode
-- You don't need to push manually - the command handles it
-
-## Close the Loop After Submitting
-
-After `haystack submit` creates the PR, **run `haystack triage <pr>` before ending your turn.**
-It blocks until Haystack's full analysis completes (waking instantly on the completion event),
-then prints the rating, findings, and a per-finding `agentFixPrompt`.
-
-- **Findings marked auto-fixing**: leave them alone — Haystack's fixer is already on them
-- **Actionable findings**: address them (the `agentFixPrompt` is written for you), commit, and
-  push — analysis re-runs on the new commit
-- **Good to merge**: report the rating to your user and stop
-
-Don't end the session right after submit without checking — the verdict usually lands within a
-few minutes, and acting on it immediately is much cheaper than a human bouncing the PR back.
-
----
-
-## Example Session
-
-```
-Agent: I've fixed the authentication bug. Let me submit this as a PR.
-
-$ haystack submit --title "Fix session expiry handling"
-
-Haystack Submit
-
-✓ Authenticated
-✓ Repository: acme/webapp
-  Current branch: fix-auth
-  Base branch: main
-✓ Branch pushed
-✓ Pull request created: #142
-✓ Auto-merge enabled for this PR
-
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-
-Pull Request Created!
-
-  URL:    https://github.com/acme/webapp/pull/142
-  Title:  Fix session expiry handling
-  Branch: fix-auth → main
-  Merge:  Auto-merge if safe
-
-Haystack will analyze this PR and:
-  • Auto-merge if approved
-  • Route for review if issues found
-```

package/dist/commands/ask.js

@@ -1,20 +0,0 @@
-import chalk from 'chalk';
-import { loadToken } from '../utils/auth.js';
-import { postAskHaystack } from '../utils/haystack-api.js';
-import { parsePrRef } from '../utils/pr-ref.js';
-export async function fetchAskPayload(ref, question, options) {
-    const pr = parsePrRef(ref, 'ask');
-    const token = await loadToken({ owner: pr.owner, repo: pr.repo });
-    if (!token)
-        throw new Error('Not authenticated. Run `haystack login` first.');
-    return postAskHaystack(pr.owner, pr.repo, pr.prNumber, question, options.session, token);
-}
-export async function askHaystackCommand(ref, question, options) {
-    const response = await fetchAskPayload(ref, question, options);
-    if (options.json) {
-        console.log(JSON.stringify(response, null, 2));
-        return;
-    }
-    console.log(`\n${response.answer}\n`);
-    console.log(chalk.dim(`${response.evidence.length} source${response.evidence.length === 1 ? '' : 's'} consulted · session ${response.session_id}\n`));
-}

package/dist/commands/cloud-verifier-behaviors.js

@@ -1,218 +0,0 @@
-import { createHash } from 'node:crypto';
-import { lstatSync, readFileSync } from 'node:fs';
-import { join } from 'node:path';
-import chalk from 'chalk';
-import { parseDocument } from 'yaml';
-import { findGitRoot } from '../utils/hooks.js';
-// Keep behavior validation on the same bounded artifact envelope as universe
-// validation. Large maps must not need a lossy reference projection merely to
-// validate their exact interface and evidence joins.
-export const MAX_BEHAVIOR_ARTIFACT_BYTES = 32 * 1024 * 1024;
-const GIT_COMMIT = /^[0-9a-f]{40}$/;
-const STABLE_UNIVERSE_ID = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
-const REPOSITORY = /^[A-Za-z0-9._-]+\/[A-Za-z0-9._-]+$/;
-const RUNTIME_ID = /^(?:action|symbol|boundary)-[0-9a-f]{24}$/;
-const BOUNDARY_KINDS = new Set(['response', 'state', 'event', 'external', 'ui', 'cli', 'job']);
-const TRIGGER_KINDS = new Set(['http', 'event', 'job', 'cli', 'ui']);
-function stableIdentity(prefix, parts) {
-    const framed = parts.map(part => `${Buffer.byteLength(part, 'utf8')}:${part}`).join('|');
-    return `${prefix}-${createHash('sha256').update(framed).digest('hex').slice(0, 24)}`;
-}
-function normalizeRepository(repository) {
-    if (!REPOSITORY.test(repository))
-        throw new Error('must be a canonical owner/repository identity');
-    return repository.toLowerCase();
-}
-function normalizePath(sourcePath) {
-    const normalized = sourcePath.replace(/\\/g, '/').replace(/^\.\//, '').replace(/^\/+/, '');
-    if (!normalized || normalized.split('/').some(part => !part || part === '..')) {
-        throw new Error('must be a normalized repository-relative source path');
-    }
-    return normalized;
-}
-export function derivedBehaviorActionId(repository, trigger) {
-    let identity;
-    if (trigger?.kind === 'http') {
-        if (typeof trigger.method !== 'string' || typeof trigger.route_template !== 'string') {
-            throw new Error('HTTP trigger requires method and route_template');
-        }
-        identity = `${trigger.method.toUpperCase()} ${trigger.route_template}`;
-    }
-    else if (trigger?.kind === 'event') {
-        if (typeof trigger.channel_identity !== 'string' || typeof trigger.event_type !== 'string') {
-            throw new Error('event trigger requires channel_identity and event_type');
-        }
-        identity = `${trigger.channel_identity}\0${trigger.event_type}`;
-    }
-    else if (trigger?.kind === 'job')
-        identity = trigger.job_identity;
-    else if (trigger?.kind === 'cli')
-        identity = trigger.command_identity;
-    else if (trigger?.kind === 'ui')
-        identity = trigger.action_identity;
-    else
-        throw new Error('trigger kind must be http, event, job, cli, or ui');
-    if (typeof identity !== 'string' || !identity.trim())
-        throw new Error('trigger identity must be non-empty');
-    return stableIdentity('action', [normalizeRepository(repository), trigger.kind, identity]);
-}
-export function derivedBehaviorSymbolId(repository, sourcePath, qualifiedName) {
-    if (typeof qualifiedName !== 'string' || !qualifiedName.trim())
-        throw new Error('qualified_name must be non-empty');
-    return stableIdentity('symbol', [normalizeRepository(repository), normalizePath(sourcePath), qualifiedName.trim()]);
-}
-export function derivedBehaviorBoundaryId(repository, kind, identityKey) {
-    if (!BOUNDARY_KINDS.has(kind))
-        throw new Error('boundary kind is unsupported');
-    if (typeof identityKey !== 'string' || !identityKey.trim())
-        throw new Error('identity_key must be non-empty');
-    return stableIdentity('boundary', [normalizeRepository(repository), kind, identityKey.trim()]);
-}
-function parseYaml(text, name) {
-    const document = parseDocument(text, { prettyErrors: false });
-    if (document.errors.length)
-        throw new Error(`${name} is malformed YAML: ${document.errors[0].message}`);
-    const value = document.toJS({ maxAliasCount: 50 });
-    if (!value || typeof value !== 'object' || Array.isArray(value))
-        throw new Error(`${name} must contain a mapping`);
-    return value;
-}
-export function validateBehaviorArtifact(value, references) {
-    const issues = [];
-    const fail = (path, message) => issues.push({ path, message });
-    if (value?.version !== 1)
-        fail('version', 'must equal 1');
-    if (value?.schema_version !== 'haystack-behavior-map-v1') {
-        fail('schema_version', 'must equal haystack-behavior-map-v1');
-    }
-    let repository = '';
-    try {
-        repository = normalizeRepository(value?.repository);
-    }
-    catch (error) {
-        fail('repository', error.message);
-    }
-    if (!GIT_COMMIT.test(value?.source_commit || ''))
-        fail('source_commit', 'must be an exact Git commit');
-    const universeCommit = references.universe?.map_revision?.source_commit;
-    if (value?.source_commit !== universeCommit)
-        fail('source_commit', 'must exactly match universe.yml map_revision.source_commit');
-    const interfaceIds = new Set((Array.isArray(references.universe?.interfaces)
-        ? references.universe.interfaces : []).map((item) => item?.id));
-    const evidenceIds = new Set((Array.isArray(references.evidence?.evidence)
-        ? references.evidence.evidence : []).map((item) => item?.id));
-    if (!Array.isArray(value?.actions) || value.actions.length === 0) {
-        fail('actions', 'must contain at least one meaningful customer action');
-        return { version: 'haystack-behavior-validation-v1', status: 'invalid', issues, summary: { actions: 0, boundaries: 0 } };
-    }
-    const actionIds = new Set();
-    const triggerIds = new Set();
-    let boundaryCount = 0;
-    for (const [index, action] of value.actions.entries()) {
-        const path = `actions[${index}]`;
-        if (!action || typeof action !== 'object' || Array.isArray(action)) {
-            fail(path, 'must be a mapping');
-            continue;
-        }
-        let expectedAction = '';
-        try {
-            expectedAction = derivedBehaviorActionId(repository || value.repository, action.trigger);
-        }
-        catch (error) {
-            fail(`${path}.trigger`, error.message);
-        }
-        if (!RUNTIME_ID.test(action.id || '') || action.id !== expectedAction)
-            fail(`${path}.id`, `must equal derived exact id ${expectedAction}`);
-        if (actionIds.has(action.id))
-            fail(`${path}.id`, 'must be unique');
-        actionIds.add(action.id);
-        if (!TRIGGER_KINDS.has(action.trigger?.kind))
-            fail(`${path}.trigger.kind`, 'is unsupported');
-        if (triggerIds.has(expectedAction))
-            fail(`${path}.trigger`, 'duplicates another structured trigger');
-        triggerIds.add(expectedAction);
-        if (!STABLE_UNIVERSE_ID.test(action.interface_id || '') || !interfaceIds.has(action.interface_id)) {
-            fail(`${path}.interface_id`, 'must reference an exact universe.yml interface id');
-        }
-        let expectedSymbol = '';
-        try {
-            expectedSymbol = derivedBehaviorSymbolId(repository || value.repository, action.handler?.source_path, action.handler?.qualified_name);
-        }
-        catch (error) {
-            fail(`${path}.handler`, error.message);
-        }
-        if (action.handler?.symbol_id !== expectedSymbol)
-            fail(`${path}.handler.symbol_id`, `must equal derived exact id ${expectedSymbol}`);
-        if (!Array.isArray(action.observable_boundaries) || action.observable_boundaries.length === 0) {
-            fail(`${path}.observable_boundaries`, 'must contain a customer-relevant boundary');
-        }
-        else {
-            const boundaryIds = new Set();
-            for (const [boundaryIndex, boundary] of action.observable_boundaries.entries()) {
-                boundaryCount += 1;
-                const boundaryPath = `${path}.observable_boundaries[${boundaryIndex}]`;
-                let expectedBoundary = '';
-                try {
-                    expectedBoundary = derivedBehaviorBoundaryId(repository || value.repository, boundary?.kind, boundary?.identity_key);
-                }
-                catch (error) {
-                    fail(boundaryPath, error.message);
-                }
-                if (boundary?.id !== expectedBoundary)
-                    fail(`${boundaryPath}.id`, `must equal derived exact id ${expectedBoundary}`);
-                if (boundaryIds.has(boundary?.id))
-                    fail(`${boundaryPath}.id`, 'must be unique in the action');
-                boundaryIds.add(boundary?.id);
-            }
-        }
-        if (!Array.isArray(action.evidence_ids) || action.evidence_ids.length === 0) {
-            fail(`${path}.evidence_ids`, 'must contain exact repository evidence');
-        }
-        else {
-            for (const evidenceId of action.evidence_ids) {
-                if (!evidenceIds.has(evidenceId))
-                    fail(`${path}.evidence_ids`, `references unknown evidence id ${String(evidenceId)}`);
-            }
-        }
-    }
-    return {
-        version: 'haystack-behavior-validation-v1',
-        status: issues.length ? 'invalid' : 'valid',
-        issues,
-        summary: { actions: value.actions.length, boundaries: boundaryCount },
-    };
-}
-function readArtifact(root, name) {
-    const path = join(root, '.haystack', 'cloud-verifier', name);
-    const metadata = lstatSync(path, { throwIfNoEntry: false });
-    if (!metadata || !metadata.isFile() || metadata.isSymbolicLink())
-        throw new Error(`${name} must be a regular non-symlink file`);
-    if (metadata.size > MAX_BEHAVIOR_ARTIFACT_BYTES) {
-        throw new Error(`${name} exceeds ${MAX_BEHAVIOR_ARTIFACT_BYTES} bytes`);
-    }
-    return readFileSync(path, 'utf8');
-}
-export function resolveBehaviorArtifactRoot(gitRoot, currentDirectory = process.cwd()) {
-    return gitRoot ?? currentDirectory;
-}
-export async function cloudVerifierBehaviorsValidateCommand(options) {
-    // Source-only evaluation archives deliberately omit Git metadata. The
-    // behavior validator only reads verifier artifacts, so current-directory
-    // fallback is both sufficient and consistent with validate-universe.
-    const root = resolveBehaviorArtifactRoot(findGitRoot());
-    const value = parseYaml(readArtifact(root, 'behaviors.yml'), 'behaviors.yml');
-    const universe = parseYaml(readArtifact(root, 'universe.yml'), 'universe.yml');
-    const evidence = parseYaml(readArtifact(root, 'evidence.yml'), 'evidence.yml');
-    const result = validateBehaviorArtifact(value, { universe, evidence });
-    if (options.json)
-        console.log(JSON.stringify(result, null, 2));
-    else if (result.status === 'valid') {
-        console.log(chalk.green(`Behavior map valid: ${result.summary.actions} actions, ${result.summary.boundaries} boundaries`));
-    }
-    else {
-        for (const issue of result.issues)
-            console.error(chalk.red(`behaviors.yml:${issue.path}: ${issue.message}`));
-    }
-    if (result.status !== 'valid')
-        process.exitCode = 1;
-}