create-qa-architect 5.13.6 → 5.14.0

package/docs/plans/PLAN-vibe-code-auditor.md

@@ -0,0 +1,130 @@
+# Plan: Vibe-Code Auditor
+
+**Created:** 2026-05-27
+**Status:** Active
+**Branch:** feat/vibe-code-auditor (new from feat/polar-migration)
+
+---
+
+## Problem
+
+QA Architect currently covers 2/7 security audit categories (secrets via Gitleaks Pro, and CVEs via npm audit free). The product is positioned as a quality bootstrap tool, but the 2026 market opportunity is "vibe-code security auditor" — a self-serve CLI that catches OWASP Top-10 patterns, injection vectors, auth gaps, production misconfigs, and hallucinated packages in AI-generated code. No CLI tool in the market does this for solo/indie developers. New web-based competitors (VibeDoctor, Vibe App Scanner) are filling the web-SaaS gap but not the CLI gap. The current product needs a new `audit` subcommand powered by semgrep, covering 5/7 categories in free tier, repositioned to match the buyer persona: "worried vibe coder about to ship."
+
+Additionally the pricing needs a drop from $49/mo to $29/mo to match market expectations, and the README/help text needs a full repositioning.
+
+---
+
+## Options Considered
+
+### Option A: Add `audit` subcommand via semgrep (chosen)
+
+Add `lib/commands/audit.js` that runs semgrep with an extended security ruleset covering SQL injection, XSS, command injection, auth bypass, production misconfigs, hardcoded secrets, and hallucinated package checks. Integrate as free-tier feature (basic 5 categories) with Pro extension (hallucination check + full OWASP pack + `--fix` prompt generation).
+
+**Pros:**
+
+- Semgrep rules already exist in `.semgrep/defensive-patterns.yaml` — extend rather than build from scratch
+- Consistent with existing spawnSync/arg-array security pattern
+- CLI-native = key differentiator vs web-based competitors
+- Adds credible "auditor" positioning without changing existing ship-check/pr-check Pro features
+
+**Cons:**
+
+- Semgrep must be installed on user machine (document in README, add detection+guidance)
+- Semgrep rules won't catch everything — must be honest about coverage
+
+### Option B: Use eslint-plugin-security only
+
+Run eslint with security plugin, no new dependency.
+
+**Pros:** No semgrep required
+
+**Cons:** Much weaker coverage — misses SQL injection, auth bypass, command injection patterns. eslint-plugin-security already part of existing setup flow so no net new value for "audit" use case.
+
+### Option C: Bundle semgrep binary (like gitleaks)
+
+Pin and cache semgrep binary like gitleaks v8.28.0.
+
+**Cons:** Semgrep binary is 50-100MB — not viable for npm package. Document as prerequisite instead.
+
+---
+
+## Decision
+
+**Approach:** Option A — semgrep-powered `audit` subcommand + pricing update + README repositioning
+
+**Rationale:** Semgrep is the de-facto OSS SAST engine with a large rule library and a simple CLI. The existing `.semgrep/defensive-patterns.yaml` gives a head start. The CLI gap in the market is real — web SaaS competitors all require uploading code to their service; this runs entirely locally, which is a trust advantage for security-conscious builders. Pricing drop from $49 to $29 reduces conversion friction without undermining value.
+
+---
+
+## Implementation Plan
+
+### New Files to Create
+
+- `lib/commands/audit.js` — main audit command handler
+- `.semgrep/vibe-audit-rules.yaml` — extended security ruleset (SQL, XSS, command injection, auth, CORS, debug mode, hardcoded secrets, missing validation, hallucinated packages)
+- `tests/audit.test.js` — unit tests for audit command
+
+### Files to Modify
+
+- `setup.js` — add `--audit` flag parsing + routing to `handleAudit()`, add to help text
+- `lib/licensing.js` — add `auditBasic` (free) and `auditPro` (Pro) feature flags; update pricing from $49→$29
+- `README.md` — full repositioning: vibe-code security auditor, lead with `audit` command, update pricing table
+- `package.json` — update description, add `audit` to scripts if helpful
+
+### Execution Order
+
+1. **Create branch** `feat/vibe-code-auditor` from `feat/polar-migration`
+2. **Create `.semgrep/vibe-audit-rules.yaml`** — the audit categories (SQL/XSS/cmd injection, auth bypass, CORS/debug misconfigs, hardcoded creds, missing rate limits). Build on existing defensive-patterns.yaml, add new categories.
+3. **Create `lib/commands/audit.js`** — the audit command:
+   - Detect if semgrep is installed, provide install guidance if not
+   - Run semgrep with both defensive-patterns.yaml and vibe-audit-rules.yaml
+   - Run npm audit (CVEs, free)
+   - Check for hallucinated packages (Pro: verify package names against npm registry)
+   - Produce structured output: Critical/High/Medium/Low findings with file:line, what's wrong, why it matters, suggested fix
+   - Pro: `--fix` flag generates Claude Code prompts for each finding
+   - Support `--json` output flag
+   - Support `--out <path>` to write markdown report to file
+4. **Update `setup.js`** — add `--audit` and `--audit-fix` flags, routing, and help text
+5. **Update `lib/licensing.js`** — add feature flags, change Pro price from 4900 to 2900 (cents)
+6. **Create `tests/audit.test.js`** — tests for: semgrep not installed handling, result parsing, severity mapping, json output, markdown output
+7. **Update `README.md`** — full rewrite of positioning sections: new tagline, lead with `audit`, update pricing table ($29/mo), update command reference
+8. **Run full test suite** — `npm test`, `npm run lint`
+9. **Version bump** to 5.14.0 (new feature, not patch)
+10. **Commit + PR** with `/bs:quality --merge`
+
+### Out of Scope
+
+- Supabase RLS gap detection (requires DB schema access — defer to v2)
+- GDPR/soft-delete/audit-log checks (high false-positive risk — defer)
+- Python audit rules (defer — focus on JS/TS/Next.js stack for v1)
+- IDE plugin (CLI only)
+- Web dashboard
+- Auto-fix (only generates Claude Code prompts, doesn't apply them)
+- Agentic workflow scanning (LangGraph/CrewAI — different product)
+- Renaming the product (not worth it at this stage)
+
+---
+
+## Verification Steps
+
+1. `node setup.js --audit` on this repo — should detect 0 critical findings (or real ones in test fixtures)
+2. `node setup.js --audit --json` — valid JSON output
+3. `node setup.js --audit --out /tmp/report.md` — file written
+4. Test with semgrep not installed — should show clear install guidance, not crash
+5. `node tests/audit.test.js` — all pass
+6. `npm test` — all 50+ tests still pass
+7. `npm run lint` — clean
+8. `node setup.js --license-status` — confirms Pro price shows $29/mo
+
+---
+
+## Notes / Gotchas
+
+- Semgrep detection: use `which semgrep` or `semgrep --version` via spawnSync, handle ENOENT gracefully
+- Semgrep output is JSON (`semgrep --json`), parse `results[].path`, `.start.line`, `.check_id`, `.message`, `.severity`, `.extra.message`
+- Semgrep severity levels: `ERROR` → Critical, `WARNING` → High/Medium depending on rule metadata
+- The existing `.semgrep/defensive-patterns.yaml` rules are already good for SQL injection, command injection, auth bypass, CORS, hardcoded secrets — extend rather than duplicate
+- Hallucinated package check (Pro): hit `https://registry.npmjs.org/<package>` for each dep in package.json, flag 404s. Cache results to avoid rate limits.
+- Pricing: `lib/licensing.js` stores price in cents. Change from `4900` to `2900`. Also update any string references to "$49" → "$29".
+- The `quality.yml` template-as-product invariant: if audit is added to the project's own CI, ensure it uses `npx @latest` pattern and not `node_modules/` references.
+- `--fix` flag for Pro: format as "Copy this prompt into Claude Code:" followed by a structured prompt that includes the file path, line number, issue, and the recommended fix pattern.

package/docs/plans/POLAR-MIGRATION.md

@@ -0,0 +1,111 @@
+# Polar.sh Migration Plan
+
+**Status:** in progress
+**Date:** 2026-05-17
+**Replaces:** Stripe-direct billing
+
+## Why
+
+We're migrating billing from **Stripe-direct** to **Polar.sh** (Merchant-of-Record).
+
+|                                          | Stripe direct                                                             | Polar.sh (MoR)          |
+| ---------------------------------------- | ------------------------------------------------------------------------- | ----------------------- |
+| Effective fee on $49/mo                  | ~3.5%                                                                     | ~4.8% (~1.3% premium)   |
+| Global sales tax / VAT / GST             | **We owe it** (~$200-400/mo for Anrok at scale, plus state registrations) | Polar collects + remits |
+| Customer portal (cancel/update/invoices) | Build it                                                                  | Built-in                |
+| Dunning / failed payment retries         | Build it                                                                  | Built-in                |
+| Effort to ship v1                        | High (tax, portal, dunning)                                               | Low (webhook swap only) |
+
+Crossover where Stripe-direct wins: ~$50K MRR. Until then, Polar is **cheaper in total cost** and ships faster.
+
+## Architecture: what stays, what changes
+
+### Stays (the moat)
+
+- `lib/license-signing.js` — Ed25519 signing primitives (re-exports `@buildproven/license-core`)
+- `lib/license-validator.js` — offline signature verification
+- `lib/licensing.js` — tier definitions, feature gates, activation flow
+- `public-key.pem` bundled in the npm package — CLI verifies signed payloads offline
+- `lib/blob-storage.js` — Vercel Blob layout for private DB + public signed registry
+- `--activate-license` CLI flow — fetches public registry, falls back to cached offline data
+- License key format `QAA-XXXX-XXXX-XXXX-XXXX`, deterministic from customer ID
+
+### Changes
+
+- `webhook-handler.js` — event source swap: Stripe `checkout.session.completed` → Polar `subscription.created/active/canceled/revoked`. Same Vercel Blob writes, same signing.
+- `docs/STRIPE-LIVE-MODE-DEPLOYMENT.md` → archived. Replaced by `docs/POLAR-DEPLOYMENT.md`.
+- `lib/billing-dashboard.html` → archived. Polar's hosted customer portal replaces it.
+- Buy URLs (`buildproven.ai/qa-architect`) → Polar checkout URL.
+- `admin-license.js` → kept, comments updated to clarify it's a manual fallback only.
+
+### New
+
+- `subscription.canceled` / `subscription.revoked` → revocation list. Signed JSON at known URL, cached by CLI with grace period so offline CI doesn't break. Closes the "cancel but keep Pro forever" loophole.
+- `COMMERCIAL.md` — paid-tier terms gated by the license-key check at runtime.
+- `LICENSE` swap: custom EULA → standard **Apache-2.0**.
+
+## Why we don't use Polar's built-in license keys
+
+Polar's built-in license-key benefit requires online validation against `/v1/customer-portal/license-keys/validate`. Our CLI is designed for **offline verification** — `npx create-qa-architect` must work in CI sandboxes with no outbound HTTP. We use Polar only for billing/MoR/checkout/portal/dunning. Our existing Ed25519 signing + Vercel Blob + offline-verifying CLI stays as-is.
+
+## Polar webhook events we handle
+
+| Event                   | Action                                                                                                                  |
+| ----------------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| `subscription.created`  | Generate license key, sign payload, write to private DB + public signed registry                                        |
+| `subscription.active`   | Idempotent re-issue (covers renewal + late activations)                                                                 |
+| `subscription.updated`  | If `product_id` changed (plan switch), update tier in DB                                                                |
+| `subscription.canceled` | Mark as `pending_cancel` in private DB. Subscription is still active until `current_period_end`. **Do not** revoke yet. |
+| `subscription.revoked`  | Move license key to revocation list. CLI will refuse it after next registry pull (with cache + grace period).           |
+
+## Polar product setup (manual, user does this)
+
+1. Create Polar org at https://polar.sh
+2. Create product **"QA Architect Pro"** with two prices:
+   - $49/mo (recurring monthly)
+   - $490/yr (recurring yearly)
+3. Save the `product_id` — single product, two prices.
+4. Configure webhook endpoint: `https://<your-vercel-domain>/webhook`
+5. Webhook secret → env var `POLAR_WEBHOOK_SECRET`
+6. API token (for any server-side Polar API calls) → env var `POLAR_ACCESS_TOKEN`
+
+## Env vars (replaces Stripe vars)
+
+| Old (Stripe)                   | New (Polar)                               |
+| ------------------------------ | ----------------------------------------- |
+| `STRIPE_SECRET_KEY`            | `POLAR_ACCESS_TOKEN`                      |
+| `STRIPE_WEBHOOK_SECRET`        | `POLAR_WEBHOOK_SECRET`                    |
+| `LICENSE_REGISTRY_PRIVATE_KEY` | (unchanged)                               |
+| `LICENSE_REGISTRY_KEY_ID`      | (unchanged)                               |
+| `BLOB_READ_WRITE_TOKEN`        | (unchanged)                               |
+| (none)                         | `POLAR_PRO_PRODUCT_ID` — maps to PRO tier |
+
+## Verification
+
+Before declaring done:
+
+1. `npm test` — full suite passes
+2. `npm run lint` — clean
+3. Manual smoke test: simulate a `subscription.created` event with Polar's CLI / Postman → verify license appears in Blob → run `npx . --activate-license` with the issued key → confirm Pro feature unlocks.
+4. Manual revocation test: simulate `subscription.revoked` → confirm license appears in revocation list → confirm CLI rejects key after registry pull.
+
+## Replicate to claude-kit-pro
+
+Same migration, same architecture. Differences:
+
+- License key prefix differs (`CKP-` instead of `QAA-`)
+- Webhook URL differs (deploy under claude-kit-pro's Vercel project)
+- `POLAR_PRO_PRODUCT_ID` env var points at claude-kit-pro's Polar product
+
+Follow this doc step-by-step in `../claude-kit-pro/`.
+
+## Rollback
+
+If Polar fails for any reason, the migration is reversible in ~half a day:
+
+1. Restore `webhook-handler.js` from git (it was just an event-source swap)
+2. Restore `docs/STRIPE-LIVE-MODE-DEPLOYMENT.md`
+3. Repoint Vercel webhook to Stripe events
+4. Existing signed licenses in Vercel Blob keep working — they're product-agnostic.
+
+The signing/verification layer never depended on the billing provider. That's why this swap is cheap.

package/docs/plans/pro-features-2026-05.md

@@ -0,0 +1,159 @@
+# Pro Feature Expansion — May 2026
+
+Spec for 4 new Pro-tier commands. Positioning: "quality gate for AI-assisted small teams."
+
+## 1. `--ship-check` (unified release readiness)
+
+**CLI**: `npx create-qa-architect --ship-check [--json] [--out <path>]`
+
+**Module**: `lib/commands/ship-check.js`
+
+**Gating**: requires Pro tier (proxy: `hasFeature('coverageThresholds')`).
+
+**Behavior**: orchestrates existing Pro checks, never re-implements them. For each check:
+
+- run the existing command/validator in a child process or via direct import
+- capture pass/fail + summary
+- never fail-fast; collect all results
+
+Checks (in order):
+
+1. Lint (`npm run lint` if script exists)
+2. Tests (`npm test` if script exists — short timeout, allow opt-out via `--skip-tests`)
+3. Security scan (gitleaks current-files, plus `npm audit --omit=dev` if package.json)
+4. Coverage thresholds (read `coverage/coverage-summary.json` if present, compare to `.qualityrc.json` thresholds)
+5. Bundle size (run `size-limit` if configured)
+6. Lighthouse thresholds (skip if not configured — info only)
+7. Env validation (check `.env.example` vs documented env vars)
+8. CI cost summary (call existing `analyze-ci` module functions in `--summary` mode)
+9. Docs validation (existing docs validator)
+
+**Output**:
+
+- Default: human-readable terminal report with section headers + final verdict.
+- `--json`: machine-readable JSON (for CI consumption).
+- `--out report.md`: write markdown suitable for PR comments.
+
+**Verdict logic**:
+
+- `SHIP`: zero failures, zero critical warnings.
+- `REVIEW`: warnings but no failures.
+- `BLOCK`: any failure.
+
+**Exit code**: 0 on SHIP/REVIEW, 1 on BLOCK.
+
+---
+
+## 2. `--pr-check` (diff-aware risk classifier)
+
+**CLI**: `npx create-qa-architect --pr-check [--base <branch>] [--json] [--out <path>]`
+
+**Module**: `lib/commands/pr-check.js`
+
+**Gating**: requires Pro tier.
+
+**Behavior**:
+
+1. Determine base branch (default: `main`, fallback `master`).
+2. Get diff: `git diff --name-status <base>...HEAD`.
+3. Classify each changed file by path patterns + content sniff:
+   - **HIGH risk**: auth, crypto, payments, env files, db migrations, GitHub workflows, security headers, license logic, anything matching `/auth|crypto|payment|stripe|webhook|migration|secret|token|key/i`.
+   - **MEDIUM risk**: config (package.json, tsconfig, eslint), public API surface (`index.ts`, `lib/**` exports), dependency changes (`package*.json`, `requirements.txt`).
+   - **LOW risk**: docs (`*.md`), tests (`*.test.*`, `tests/**`), comments-only.
+4. For each non-test source file changed, check if a matching test file changed too. Flag missing tests.
+5. For HIGH-risk files: check if covered by CODEOWNERS (if file exists).
+6. Emit risk summary + per-file table.
+
+**Output**: markdown report (PR-comment-ready) with:
+
+- Risk summary (counts per tier)
+- Missing tests warning
+- High-risk file list with reasons
+- Verdict: SHIP / REVIEW / BLOCK (BLOCK only if HIGH + no tests + no codeowner)
+
+**Exit code**: 0 on SHIP/REVIEW, 1 on BLOCK (configurable via `--no-fail`).
+
+---
+
+## 3. CI Doctor (expand `--analyze-ci`)
+
+**CLI**: existing `--analyze-ci` adds new `--doctor` flag for the extra checks. Default still shows cost analysis.
+
+**Module**: extend `lib/commands/analyze-ci.js` — add a `runDoctor(workflows)` function.
+
+**New findings** (each with concrete fix suggestion):
+
+1. **Duplicated jobs**: detect jobs with identical `runs-on` + steps signature → suggest reusable workflow.
+2. **Missing path filters**: workflows triggered on every push without `paths:` or `paths-ignore:` → suggest path filters for monorepos / docs-only changes.
+3. **Expensive matrix**: matrix with >10 combinations → suggest pruning or `include`/`exclude`.
+4. **Cache mistakes**: `actions/setup-node` without `cache:` parameter → suggest enabling.
+5. **Unnecessary scheduled runs**: cron more frequent than weekly with no obvious need → suggest reducing.
+6. **Flaky test detection**: parse `gh run list --json` if `gh` CLI available, look for jobs with success rate <90% over last 30 runs. Skip gracefully if `gh` not authenticated.
+
+**Output**: appended section to existing report. Each finding shows: title, affected workflow/job, fix suggestion (1-2 lines), estimated savings if applicable.
+
+---
+
+## 4. `--history-scan` (historical secrets)
+
+**CLI**: `npx create-qa-architect --history-scan [--depth <N>] [--json]`
+
+**Module**: `lib/commands/history-scan.js`
+
+**Gating**: requires `hasFeature('securityScanning')`.
+
+**Behavior**:
+
+1. Reuse `resolveGitleaksBinary()` from `lib/validation/config-security.js`.
+2. Run: `<gitleaks> detect --no-banner --redact --report-format=json --report-path=<tmp> --log-opts="--all"` (or `HEAD~<depth>` if `--depth` given).
+3. Parse JSON output, deduplicate by `{secret, file, commit}`.
+4. Group findings by commit SHA, list files, secret types.
+5. Report counts per secret type + top 10 oldest exposures.
+
+**Output**: terminal report + optional JSON. Markdown export for PR.
+
+**Exit code**: 0 if zero findings, 1 if any.
+
+**Safety**: pass `--all` only when explicitly requested; default `HEAD~1000` to bound cost on huge repos.
+
+---
+
+## Licensing changes
+
+Add flags to `lib/licensing.js` FEATURES map:
+
+- `shipCheck`: PRO only
+- `prCheck`: PRO only
+- `ciDoctor`: PRO only (expansion of existing `ciCostAnalysis`)
+- `historicalSecretsScan`: PRO only (under existing `securityScanning` umbrella)
+
+Update PRO roadmap array. Add FREE roadmap line: "❌ No release readiness gate (--ship-check), risk-aware PR review, CI doctor, or historical secrets scan".
+
+---
+
+## Testing
+
+One test file per command:
+
+- `tests/ship-check.test.js`
+- `tests/pr-check.test.js`
+- `tests/ci-doctor.test.js`
+- `tests/history-scan.test.js`
+
+Each covers:
+
+- Free tier blocked (proper upgrade message)
+- Pro tier runs (via `QAA_DEVELOPER=true` or stub license)
+- Output format validates (markdown structure / JSON shape)
+- Edge cases: empty diff (pr-check), no workflows (ci-doctor), no .git history (history-scan), no coverage report (ship-check)
+
+Goal: keep coverage ≥75% lines / 70% functions per project standard.
+
+---
+
+## Out of scope (deferred)
+
+- LLM-powered fix suggestions (depends on API key, adds cost dimension)
+- Monorepo selective CI (deeper architecture change)
+- Team dashboard (needs hosted component)
+- PR inline annotations via Checks API (depends on GH App / token model; document for v6)

package/lib/commands/analyze-ci.js

@@ -12,7 +12,11 @@ const path = require('path')
 const { execSync } = require('child_process')
 const yaml = require('js-yaml')
 const { showProgress } = require('../ui-helpers')
-const { hasFeature, showUpgradeMessage } = require('../licensing')
+const {
+  hasFeature,
+  showUpgradeMessage,
+  ensureLicenseFresh,
+} = require('../licensing')
 
 const DAYS_PER_MONTH = 30
 const DEFAULT_PULL_REQUEST_FACTOR = 0.8
@@ -771,20 +775,26 @@ function generateReport(analysis) {
 }
 
 /**
- * Main handler for --analyze-ci command
+ * Main handler for --analyze-ci command. When `options.doctor` is true,
+ * appends the CI Doctor diagnostics (flaky tests, duplicated jobs, etc.)
  */
-async function handleAnalyzeCi() {
+async function handleAnalyzeCi(options = {}) {
   const projectPath = process.cwd()
 
+  await ensureLicenseFresh()
   if (!hasFeature('ciCostAnalysis')) {
     showUpgradeMessage('GitHub Actions cost analysis')
     process.exit(1)
   }
 
+  if (options.doctor && !hasFeature('ciDoctor')) {
+    showUpgradeMessage('CI Doctor (workflow health + flaky test detection)')
+    process.exit(1)
+  }
+
   const spinner = showProgress('Analyzing GitHub Actions workflows...')
 
   try {
-    // Step 1: Discover workflows
     const workflowFiles = discoverWorkflows(projectPath)
 
     if (workflowFiles.length === 0) {
@@ -796,7 +806,6 @@ async function handleAnalyzeCi() {
       process.exit(1)
     }
 
-    // Step 2: Parse and analyze workflows
     const workflows = []
     for (const wf of workflowFiles) {
       try {
@@ -815,13 +824,8 @@ async function handleAnalyzeCi() {
       }
     }
 
-    // Step 3: Get commit frequency
     const commitStats = getCommitFrequency(projectPath)
-
-    // Step 4: Calculate costs
     const costs = calculateMonthlyCosts(workflows, commitStats.commitsPerDay)
-
-    // Step 5: Analyze optimization opportunities
     const optimizations = analyzeOptimizations(
       workflows,
       commitStats.commitsPerDay
@@ -829,7 +833,6 @@ async function handleAnalyzeCi() {
 
     spinner.succeed('Analysis complete')
 
-    // Step 6: Generate report
     generateReport({
       workflows,
       costs,
@@ -837,6 +840,12 @@ async function handleAnalyzeCi() {
       optimizations,
     })
 
+    if (options.doctor) {
+      const { runDoctorChecks, buildDoctorReport } = require('./ci-doctor')
+      const findings = runDoctorChecks(workflows, projectPath, options)
+      process.stdout.write(buildDoctorReport(findings))
+    }
+
     process.exit(0)
   } catch (error) {
     spinner.fail('Analysis failed')