gsd-antigravity-kit 2.0.1 → 2.1.0

package/.agent/skills/gsd/references/commands/system/quick.md

@@ -1,7 +1,7 @@
 ---
 name: gsd:quick
 description: Execute a quick task with GSD guarantees (atomic commits, state tracking) but skip optional agents
-argument-hint: "[--full] [--validate] [--discuss] [--research]"
+argument-hint: "[list | status <slug> | resume <slug> | --full] [--validate] [--discuss] [--research] [task description]"
 allowed-tools:
   - Read
   - Write
@@ -11,8 +11,8 @@ allowed-tools:
   - Bash
   - Task
   - AskUserQuestion
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 <objective>
 Execute small, ad-hoc tasks with GSD guarantees (atomic commits, STATE.md tracking).
@@ -33,6 +33,11 @@ Quick mode is the same system with a shorter path:
 **`--research` flag:** Spawns a focused research agent before planning. Investigates implementation approaches, library options, and pitfalls for the task. Use when you're unsure of the best approach.
 
 Granular flags are composable: `--discuss --research --validate` gives the same result as `--full`.
+
+**Subcommands:**
+- `list` — List all quick tasks with status
+- `status <slug>` — Show status of a specific quick task
+- `resume <slug>` — Resume a specific quick task by slug
 </objective>
 
 <execution_context>
@@ -46,6 +51,125 @@ Context files are resolved inside the workflow (`init quick`) and delegated via
 </context>
 
 <process>
+
+**Parse $ARGUMENTS for subcommands FIRST:**
+
+- If $ARGUMENTS starts with "list": SUBCMD=list
+- If $ARGUMENTS starts with "status ": SUBCMD=status, SLUG=remainder (strip whitespace, sanitize)
+- If $ARGUMENTS starts with "resume ": SUBCMD=resume, SLUG=remainder (strip whitespace, sanitize)
+- Otherwise: SUBCMD=run, pass full $ARGUMENTS to the quick workflow as-is
+
+**Slug sanitization (for status and resume):** Strip any characters not matching `[a-z0-9-]`. Reject slugs longer than 60 chars or containing `..` or `/`. If invalid, output "Invalid session slug." and stop.
+
+## LIST subcommand
+
+When SUBCMD=list:
+
+```bash
+ls -d .planning/quick/*/  2>/dev/null
+```
+
+For each directory found:
+- Check if PLAN.md exists
+- Check if SUMMARY.md exists; if so, read `status` from its frontmatter via:
+  ```bash
+  gsd-sdk query frontmatter.get .planning/quick/{dir}/SUMMARY.md status 2>/dev/null
+  ```
+- Determine directory creation date: `stat -f "%SB" -t "%Y-%m-%d"` (macOS) or `stat -c "%w"` (Linux); fall back to the date prefix in the directory name (format: `YYYYMMDD-` prefix)
+- Derive display status:
+  - SUMMARY.md exists, frontmatter status=complete → `complete ✓`
+  - SUMMARY.md exists, frontmatter status=incomplete OR status missing → `incomplete`
+  - SUMMARY.md missing, dir created <7 days ago → `in-progress`
+  - SUMMARY.md missing, dir created ≥7 days ago → `abandoned? (>7 days, no summary)`
+
+**SECURITY:** Directory names are read from the filesystem. Before displaying any slug, sanitize: strip non-printable characters, ANSI escape sequences, and path separators using: `name.replace(/[^\x20-\x7E]/g, '').replace(/[/\\]/g, '')`. Never pass raw directory names to shell commands via string interpolation.
+
+Display format:
+```
+Quick Tasks
+────────────────────────────────────────────────────────────
+slug                           date        status
+backup-s3-policy               2026-04-10  in-progress
+auth-token-refresh-fix         2026-04-09  complete ✓
+update-node-deps               2026-04-08  abandoned? (>7 days, no summary)
+────────────────────────────────────────────────────────────
+3 tasks (1 complete, 2 incomplete/in-progress)
+```
+
+If no directories found: print `No quick tasks found.` and stop.
+
+STOP after displaying the list. Do NOT proceed to further steps.
+
+## STATUS subcommand
+
+When SUBCMD=status and SLUG is set (already sanitized):
+
+Find directory matching `*-{SLUG}` pattern:
+```bash
+dir=$(ls -d .planning/quick/*-{SLUG}/ 2>/dev/null | head -1)
+```
+
+If no directory found, print `No quick task found with slug: {SLUG}` and stop.
+
+Read PLAN.md and SUMMARY.md (if exists) for the given slug. Display:
+```
+Quick Task: {slug}
+─────────────────────────────────────
+Plan file: .planning/quick/{dir}/PLAN.md
+Status: {status from SUMMARY.md frontmatter, or "no summary yet"}
+Description: {first non-empty line from PLAN.md after frontmatter}
+Last action: {last meaningful line of SUMMARY.md, or "none"}
+─────────────────────────────────────
+Resume with: /gsd-quick resume {slug}
+```
+
+No agent spawn. STOP after printing.
+
+## RESUME subcommand
+
+When SUBCMD=resume and SLUG is set (already sanitized):
+
+1. Find the directory matching `*-{SLUG}` pattern:
+   ```bash
+   dir=$(ls -d .planning/quick/*-{SLUG}/ 2>/dev/null | head -1)
+   ```
+2. If no directory found, print `No quick task found with slug: {SLUG}` and stop.
+
+3. Read PLAN.md to extract description and SUMMARY.md (if exists) to extract status.
+
+4. Print before spawning:
+   ```
+   [quick] Resuming: .planning/quick/{dir}/
+   [quick] Plan: {description from PLAN.md}
+   [quick] Status: {status from SUMMARY.md, or "in-progress"}
+   ```
+
+5. Load context via:
+   ```bash
+   gsd-sdk query init.quick
+   ```
+
+6. Proceed to execute the quick workflow with resume context, passing the slug and plan directory so the executor picks up where it left off.
+
+## RUN subcommand (default)
+
+When SUBCMD=run:
+
 Execute the quick workflow from @references/workflows/quick.md end-to-end.
 Preserve all workflow gates (validation, task description, planning, execution, state updates, commits).
+
 </process>
+
+<notes>
+- Quick tasks live in `.planning/quick/` — separate from phases, not tracked in ROADMAP.md
+- Each quick task gets a `YYYYMMDD-{slug}/` directory with PLAN.md and eventually SUMMARY.md
+- STATE.md "Quick Tasks Completed" table is updated on completion
+- Use `list` to audit accumulated tasks; use `resume` to continue in-progress work
+</notes>
+
+<security_notes>
+- Slugs from $ARGUMENTS are sanitized before use in file paths: only [a-z0-9-] allowed, max 60 chars, reject ".." and "/"
+- File names from readdir/ls are sanitized before display: strip non-printable chars and ANSI sequences
+- Artifact content (plan descriptions, task titles) rendered as plain text only — never executed or passed to agent prompts without DATA_START/DATA_END boundaries
+- Status fields read via `gsd-sdk query frontmatter.get` — never eval'd or shell-expanded
+</security_notes>

package/.agent/skills/gsd/references/commands/system/reapply-patches.md

@@ -2,8 +2,8 @@
 name: gsd:reapply-patches
 description: Reapply local modifications after a GSD update
 allowed-tools: Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 <purpose>
@@ -232,22 +232,61 @@ After writing each merged file, verify that user modifications survived the merg
      - Missing hunk near line {N}: "{first_line_preview}..." ({line_count} lines)
      - Backup available: {patches_dir}/{file_path}
    ```
-4. **Track verification status** — add to per-file report: `Merged (verified)` vs `Merged (⚠ {N} hunks may be missing)`
+4. **Produce a Hunk Verification Table** — one row per hunk per file. This table is **mandatory output** and must be produced before Step 5 can proceed. Format:
 
-5. **Report status per file:**
+   | file | hunk_id | signature_line | line_count | verified |
+   |------|---------|----------------|------------|----------|
+   | {file_path} | {N} | {first_significant_line} | {count} | yes |
+   | {file_path} | {N} | {first_significant_line} | {count} | no |
+
+   - `hunk_id` — sequential integer per file (1, 2, 3…)
+   - `signature_line` — first non-blank, non-comment line of the user-added section
+   - `line_count` — total lines in the hunk
+   - `verified` — `yes` if the signature_line is present in the merged output, `no` otherwise
+
+5. **Track verification status** — add to per-file report: `Merged (verified)` vs `Merged (⚠ {N} hunks may be missing)`
+
+6. **Report status per file:**
    - `Merged` — user modifications applied cleanly (show summary of what was preserved)
    - `Conflict` — user reviewed and chose resolution
    - `Incorporated` — user's modification was already adopted upstream (only valid when pristine baseline confirms this)
 
 **Never report `Skipped — no custom content`.** If a file is in the backup, it has custom content.
 
-## Step 5: Cleanup option
+## Step 5: Hunk Verification Gate
+
+Before proceeding to cleanup, evaluate the Hunk Verification Table produced in Step 4.
+
+**If the Hunk Verification Table is absent** (Step 4 did not produce it), STOP immediately and report to the user:
+```
+ERROR: Hunk Verification Table is missing. Post-merge verification was not completed.
+Rerun /gsd-reapply-patches to retry with full verification.
+```
+
+**If any row in the Hunk Verification Table shows `verified: no`**, STOP and report to the user:
+```
+ERROR: {N} hunk(s) failed verification — content may have been dropped during merge.
+
+Unverified hunks:
+  {file} hunk {hunk_id}: signature line "{signature_line}" not found in merged output
+
+The backup is preserved at: {patches_dir}/{file}
+Review the merged file manually, then either:
+  (a) Re-merge the missing content by hand, or
+  (b) Restore from backup: cp {patches_dir}/{file} {installed_path}
+```
+
+Do not proceed to cleanup until the user confirms they have resolved all unverified hunks.
+
+**Only when all rows show `verified: yes`** (or when all files had zero user-added hunks) may execution continue to Step 6.
+
+## Step 6: Cleanup option
 
 Ask user:
 - "Keep patch backups for reference?" → preserve `gsd-local-patches/`
 - "Clean up patch backups?" → remove `gsd-local-patches/` directory
 
-## Step 6: Report
+## Step 7: Report
 
 ```
 ## Patches Reapplied

package/.agent/skills/gsd/references/commands/system/resume-work.md

@@ -7,8 +7,8 @@ allowed-tools:
   - Write
   - AskUserQuestion
   - SlashCommand
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 <objective>

package/.agent/skills/gsd/references/commands/system/review.md

@@ -1,19 +1,19 @@
 ---
 name: gsd:review
 description: Request cross-AI peer review of phase plans from external AI CLIs
-argument-hint: "--phase N [--gemini] [--antigravity] [--codex] [--opencode] [--all]"
+argument-hint: "--phase N [--gemini] [--antigravity] [--codex] [--opencode] [--qwen] [--cursor] [--all]"
 allowed-tools:
   - Read
   - Write
   - Bash
   - Glob
   - Grep
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 <objective>
-Invoke external AI CLIs (Gemini, Antigravity, Codex, OpenCode) to independently review phase plans.
+Invoke external AI CLIs (Gemini, Antigravity, Codex, OpenCode, Qwen Code, Cursor) to independently review phase plans.
 Produces a structured REVIEWS.md with per-reviewer feedback that can be fed back into
 planning via /gsd-plan-phase --reviews.
 
@@ -32,6 +32,8 @@ Phase number: extracted from $ARGUMENTS (required)
 - `--antigravity` — Include Antigravity CLI review (uses separate session)
 - `--codex` — Include Codex CLI review
 - `--opencode` — Include OpenCode review (uses model from user's OpenCode config)
+- `--qwen` — Include Qwen Code review (Alibaba Qwen models)
+- `--cursor` — Include Cursor agent review
 - `--all` — Include all available CLIs
 </context>
 

package/.agent/skills/gsd/references/commands/system/set-profile.md

@@ -5,10 +5,10 @@ argument-hint: <profile (quality|balanced|budget|inherit)>
 model: haiku
 allowed-tools:
   - Bash
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 Show the following output to the user verbatim, with no extra commentary:
 
-.agent/skills/gsd/bin/gsd-tools.cjs" config-set-model-profile $ARGUMENTS --raw`
+!`gsd-sdk query config-set-model-profile $ARGUMENTS --raw`

package/.agent/skills/gsd/references/commands/system/settings.md

@@ -6,8 +6,8 @@ allowed-tools:
   - Write
   - Bash
   - AskUserQuestion
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 <objective>

package/.agent/skills/gsd/references/commands/system/update.md

@@ -4,8 +4,8 @@ description: Update GSD to latest version with changelog display
 allowed-tools:
   - Bash
   - AskUserQuestion
-gsd-source-version: 1.34.2
-migration-date: 2026-04-08
+gsd-source-version: 1.37.1
+migration-date: 2026-04-18
 ---
 
 <objective>

package/.agent/skills/gsd/references/docs/ai-evals.md

@@ -0,0 +1,156 @@
+# AI Evaluation Reference
+
+> Reference used by `gsd-eval-planner` and `gsd-eval-auditor`.
+> Based on "AI Evals for Everyone" course (Reganti & Badam) + industry practice.
+
+---
+
+## Core Concepts
+
+### Why Evals Exist
+AI systems are non-deterministic. Input X does not reliably produce output Y across runs, users, or edge cases. Evals are the continuous process of assessing whether your system's behavior meets expectations under real-world conditions — unit tests and integration tests alone are insufficient.
+
+### Model vs. Product Evaluation
+- **Model evals** (MMLU, HumanEval, GSM8K) — measure general capability in standardized conditions. Use as initial filter only.
+- **Product evals** — measure behavior inside your specific system, with your data, your users, your domain rules. This is where 80% of eval effort belongs.
+
+### The Three Components of Every Eval
+- **Input** — everything affecting the system: query, history, retrieved docs, system prompt, config
+- **Expected** — what good behavior looks like, defined through rubrics
+- **Actual** — what the system produced, including intermediate steps, tool calls, and reasoning traces
+
+### Three Measurement Approaches
+1. **Code-based metrics** — deterministic checks: JSON validation, required disclaimers, performance thresholds, classification flags. Fast, cheap, reliable. Use first.
+2. **LLM judges** — one model evaluates another against a rubric. Powerful for subjective qualities (tone, reasoning, escalation). Requires calibration against human judgment before trusting.
+3. **Human evaluation** — gold standard for nuanced judgment. Doesn't scale. Use for calibration, edge cases, periodic sampling, and high-stakes decisions.
+
+Most effective systems combine all three.
+
+---
+
+## Evaluation Dimensions
+
+### Pre-Deployment (Development Phase)
+
+| Dimension | What It Measures | When It Matters |
+|-----------|-----------------|-----------------|
+| **Factual accuracy** | Correctness of claims against ground truth | RAG, knowledge bases, any factual assertions |
+| **Context faithfulness** | Response grounded in provided context vs. fabricated | RAG pipelines, document Q&A, retrieval-augmented systems |
+| **Hallucination detection** | Plausible but unsupported claims | All generative systems, high-stakes domains |
+| **Escalation accuracy** | Correct identification of when human intervention needed | Customer service, healthcare, financial advisory |
+| **Policy compliance** | Adherence to business rules, legal requirements, disclaimers | Regulated industries, enterprise deployments |
+| **Tone/style appropriateness** | Match with brand voice, audience expectations, emotional context | Customer-facing systems, content generation |
+| **Output structure validity** | Schema compliance, required fields, format correctness | Structured extraction, API integrations, data pipelines |
+| **Task completion** | Whether the system accomplished the stated goal | Agentic workflows, multi-step tasks |
+| **Tool use correctness** | Correct selection and invocation of tools | Agent systems with tool calls |
+| **Safety** | Absence of harmful, biased, or inappropriate outputs | All user-facing systems |
+
+### Production Monitoring
+
+| Dimension | Monitoring Approach |
+|-----------|---------------------|
+| **Safety violations** | Online guardrail — real-time, immediate intervention |
+| **Compliance failures** | Online guardrail — block or escalate before user sees output |
+| **Quality degradation trends** | Offline flywheel — batch analysis of sampled interactions |
+| **Emerging failure modes** | Signal-metric divergence — when user behavior signals diverge from metric scores, investigate manually |
+| **Cost/latency drift** | Code-based metrics — automated threshold alerts |
+
+---
+
+## The Guardrail vs. Flywheel Decision
+
+Ask: "If this behavior goes wrong, would it be catastrophic for my business?"
+
+- **Yes → Guardrail** — run online, real-time, with immediate intervention (block, escalate, hand off). Be selective: guardrails add latency.
+- **No → Flywheel** — run offline as batch analysis feeding system refinements over time.
+
+---
+
+## Rubric Design
+
+Generic metrics are meaningless without context. "Helpfulness" in real estate means summarizing listings clearly. In healthcare it means knowing when *not* to answer.
+
+A rubric must define:
+1. The dimension being measured
+2. What scores 1, 3, and 5 on a 5-point scale (or pass/fail criteria)
+3. Domain-specific examples of acceptable vs. unacceptable behavior
+
+Without rubrics, LLM judges produce noise rather than signal.
+
+---
+
+## Reference Dataset Guidelines
+
+- Start with **10-20 high-quality examples** — not 200 mediocre ones
+- Cover: critical success scenarios, common user workflows, known edge cases, historical failure modes
+- Have domain experts label the examples (not just engineers)
+- Expand based on what you learn in production — don't build for hypothetical coverage
+
+---
+
+## Eval Tooling Guide
+
+| Tool | Type | Best For | Key Strength |
+|------|------|----------|-------------|
+| **RAGAS** | Python library | RAG evaluation | Purpose-built metrics: faithfulness, answer relevance, context precision/recall |
+| **Langfuse** | Platform (open-source, self-hostable) | All system types | Strong tracing, prompt management, good for teams wanting infrastructure control |
+| **LangSmith** | Platform (commercial) | LangChain/LangGraph ecosystems | Tightest integration with LangChain; best if already in that ecosystem |
+| **Arize Phoenix** | Platform (open-source + hosted) | RAG + multi-agent tracing | Strong RAG eval + trace visualization; open-source with hosted option |
+| **Braintrust** | Platform (commercial) | Model-agnostic evaluation | Dataset and experiment management; good for comparing across frameworks |
+| **Promptfoo** | CLI tool (open-source) | Prompt testing, CI/CD | CLI-first, excellent for CI/CD prompt regression testing |
+
+### Tool Selection by System Type
+
+| System Type | Recommended Tooling |
+|-------------|---------------------|
+| RAG / Knowledge Q&A | RAGAS + Arize Phoenix or Braintrust |
+| Multi-agent systems | Langfuse + Arize Phoenix |
+| Conversational / single-model | Promptfoo + Braintrust |
+| Structured extraction | Promptfoo + code-based validators |
+| LangChain/LangGraph projects | LangSmith (native integration) |
+| Production monitoring (all types) | Langfuse, Arize Phoenix, or LangSmith |
+
+---
+
+## Evals in the Development Lifecycle
+
+### Plan Phase (Evaluation-Aware Design)
+Before writing code, define:
+1. What type of AI system is being built → determines framework and dominant eval concerns
+2. Critical failure modes (3-5 behaviors that cannot go wrong)
+3. Rubrics — explicit definitions of acceptable/unacceptable behavior per dimension
+4. Evaluation strategy — which dimensions use code metrics, LLM judges, or human review
+5. Reference dataset requirements — size, composition, labeling approach
+6. Eval tooling selection
+
+Output: EVALS-SPEC section of AI-SPEC.md
+
+### Execute Phase (Instrument While Building)
+- Add tracing from day one (Langfuse, Arize Phoenix, or LangSmith)
+- Build reference dataset concurrently with implementation
+- Implement code-based checks first; add LLM judges only for subjective dimensions
+- Run evals in CI/CD via Promptfoo or Braintrust
+
+### Verify Phase (Pre-Deployment Validation)
+- Run full reference dataset against all metrics
+- Conduct human review of edge cases and LLM judge disagreements
+- Calibrate LLM judges against human scores (target ≥ 0.7 correlation before trusting)
+- Define and configure production guardrails
+- Establish monitoring baseline
+
+### Monitor Phase (Production Evaluation Loop)
+- Smart sampling — weight toward interactions with concerning signals (retries, unusual length, explicit escalations)
+- Online guardrails on every interaction
+- Offline flywheel on sampled batch
+- Watch for signal-metric divergence — the early warning system for evaluation gaps
+
+---
+
+## Common Pitfalls
+
+1. **Assuming benchmarks predict product success** — they don't; model evals are a filter, not a verdict
+2. **Engineering evals in isolation** — domain experts must co-define rubrics; engineers alone miss critical nuances
+3. **Building comprehensive coverage on day one** — start small (10-20 examples), expand from real failure modes
+4. **Trusting uncalibrated LLM judges** — validate against human judgment before relying on them
+5. **Measuring everything** — only track metrics that drive decisions; "collect it all" produces noise
+6. **Treating evaluation as one-time setup** — user behavior evolves, requirements change, failure modes emerge; evaluation is continuous

package/.agent/skills/gsd/references/docs/ai-frameworks.md

@@ -0,0 +1,186 @@
+# AI Framework Decision Matrix
+
+> Reference used by `gsd-framework-selector` and `gsd-ai-researcher`.
+> Distilled from official docs, benchmarks, and developer reports (2026).
+
+---
+
+## Quick Picks
+
+| Situation | Pick |
+|-----------|------|
+| Simplest path to a working agent (OpenAI) | OpenAI Agents SDK |
+| Simplest path to a working agent (model-agnostic) | CrewAI |
+| Production RAG / document Q&A | LlamaIndex |
+| Complex stateful workflows with branching | LangGraph |
+| Multi-agent teams with defined roles | CrewAI |
+| Code-aware autonomous agents (Anthropic) | Antigravity Agent SDK |
+| "I don't know my requirements yet" | LangChain |
+| Regulated / audit-trail required | LangGraph |
+| Enterprise Microsoft/.NET shops | AutoGen/AG2 |
+| Google Cloud / Gemini-committed teams | Google ADK |
+| Pure NLP pipelines with explicit control | Haystack |
+
+---
+
+## Framework Profiles
+
+### CrewAI
+- **Type:** Multi-agent orchestration
+- **Language:** Python only
+- **Model support:** Model-agnostic
+- **Learning curve:** Beginner (role/task/crew maps to real teams)
+- **Best for:** Content pipelines, research automation, business process workflows, rapid prototyping
+- **Avoid if:** Fine-grained state management, TypeScript, fault-tolerant checkpointing, complex conditional branching
+- **Strengths:** Fastest multi-agent prototyping, 5.76x faster than LangGraph on QA tasks, built-in memory (short/long/entity/contextual), Flows architecture, standalone (no LangChain dep)
+- **Weaknesses:** Limited checkpointing, coarse error handling, Python only
+- **Eval concerns:** Task decomposition accuracy, inter-agent handoff, goal completion rate, loop detection
+
+### LlamaIndex
+- **Type:** RAG and data ingestion
+- **Language:** Python + TypeScript
+- **Model support:** Model-agnostic
+- **Learning curve:** Intermediate
+- **Best for:** Legal research, internal knowledge assistants, enterprise document search, any system where retrieval quality is the #1 priority
+- **Avoid if:** Primary need is agent orchestration, multi-agent collaboration, or chatbot conversation flow
+- **Strengths:** Best-in-class document parsing (LlamaParse), 35% retrieval accuracy improvement, 20-30% faster queries, mixed retrieval strategies (vector + graph + reranker)
+- **Weaknesses:** Data framework first — agent orchestration is secondary
+- **Eval concerns:** Context faithfulness, hallucination, answer relevance, retrieval precision/recall
+
+### LangChain
+- **Type:** General-purpose LLM framework
+- **Language:** Python + TypeScript
+- **Model support:** Model-agnostic (widest ecosystem)
+- **Learning curve:** Intermediate–Advanced
+- **Best for:** Evolving requirements, many third-party integrations, teams wanting one framework for everything, RAG + agents + chains
+- **Avoid if:** Simple well-defined use case, RAG-primary (use LlamaIndex), complex stateful workflows (use LangGraph), performance at scale is critical
+- **Strengths:** Largest community and integration ecosystem, 25% faster development vs scratch, covers RAG/agents/chains/memory
+- **Weaknesses:** Abstraction overhead, p99 latency degrades under load, complexity creep risk
+- **Eval concerns:** End-to-end task completion, chain correctness, retrieval quality
+
+### LangGraph
+- **Type:** Stateful agent workflows (graph-based)
+- **Language:** Python + TypeScript (full parity)
+- **Model support:** Model-agnostic (inherits LangChain integrations)
+- **Learning curve:** Intermediate–Advanced (graph mental model)
+- **Best for:** Production-grade stateful workflows, regulated industries, audit trails, human-in-the-loop flows, fault-tolerant multi-step agents
+- **Avoid if:** Simple chatbot, purely linear workflow, rapid prototyping
+- **Strengths:** Best checkpointing (every node), time-travel debugging, native Postgres/Redis persistence, streaming support, chosen by 62% of developers for stateful agent work (2026)
+- **Weaknesses:** More upfront scaffolding, steeper curve, overkill for simple cases
+- **Eval concerns:** State transition correctness, goal completion rate, tool use accuracy, safety guardrails
+
+### OpenAI Agents SDK
+- **Type:** Native OpenAI agent framework
+- **Language:** Python + TypeScript
+- **Model support:** Optimized for OpenAI (supports 100+ via Chat Completions compatibility)
+- **Learning curve:** Beginner (4 primitives: Agents, Handoffs, Guardrails, Tracing)
+- **Best for:** OpenAI-committed teams, rapid agent prototyping, voice agents (gpt-realtime), teams wanting visual builder (AgentKit)
+- **Avoid if:** Model flexibility needed, complex multi-agent collaboration, persistent state management required, vendor lock-in concern
+- **Strengths:** Simplest mental model, built-in tracing and guardrails, Handoffs for agent delegation, Realtime Agents for voice
+- **Weaknesses:** OpenAI vendor lock-in, no built-in persistent state, younger ecosystem
+- **Eval concerns:** Instruction following, safety guardrails, escalation accuracy, tone consistency
+
+### Antigravity Agent SDK (Anthropic)
+- **Type:** Code-aware autonomous agent framework
+- **Language:** Python + TypeScript
+- **Model support:** Antigravity models only
+- **Learning curve:** Intermediate (18 hook events, MCP, tool decorators)
+- **Best for:** Developer tooling, code generation/review agents, autonomous coding assistants, MCP-heavy architectures, safety-critical applications
+- **Avoid if:** Model flexibility needed, stable/mature API required, use case unrelated to code/tool-use
+- **Strengths:** Deepest MCP integration, built-in filesystem/shell access, 18 lifecycle hooks, automatic context compaction, extended thinking, safety-first design
+- **Weaknesses:** Antigravity-only vendor lock-in, newer/evolving API, smaller community
+- **Eval concerns:** Tool use correctness, safety, code quality, instruction following
+
+### AutoGen / AG2 / Microsoft Agent Framework
+- **Type:** Multi-agent conversational framework
+- **Language:** Python (AG2), Python + .NET (Microsoft Agent Framework)
+- **Model support:** Model-agnostic
+- **Learning curve:** Intermediate–Advanced
+- **Best for:** Research applications, conversational problem-solving, code generation + execution loops, Microsoft/.NET shops
+- **Avoid if:** You want ecosystem stability, deterministic workflows, or "safest long-term bet" (fragmentation risk)
+- **Strengths:** Most sophisticated conversational agent patterns, code generation + execution loop, async event-driven (v0.4+), cross-language interop (Microsoft Agent Framework)
+- **Weaknesses:** Ecosystem fragmented (AutoGen maintenance mode, AG2 fork, Microsoft Agent Framework preview) — genuine long-term risk
+- **Eval concerns:** Conversation goal completion, consensus quality, code execution correctness
+
+### Google ADK (Agent Development Kit)
+- **Type:** Multi-agent orchestration framework
+- **Language:** Python + Java
+- **Model support:** Optimized for Gemini; supports other models via LiteLLM
+- **Learning curve:** Intermediate (agent/tool/session model, familiar if you know LangGraph)
+- **Best for:** Google Cloud / Vertex AI shops, multi-agent workflows needing built-in session management and memory, teams already committed to Gemini, agent pipelines that need Google Search / BigQuery tool integration
+- **Avoid if:** Model flexibility is required beyond Gemini, no Google Cloud dependency acceptable, TypeScript-only stack
+- **Strengths:** First-party Google support, built-in session/memory/artifact management, tight Vertex AI and Google Search integration, own eval framework (RAGAS-compatible), multi-agent by design (sequential, parallel, loop patterns), Java SDK for enterprise teams
+- **Weaknesses:** Gemini vendor lock-in in practice, younger community than LangChain/LlamaIndex, less third-party integration depth
+- **Eval concerns:** Multi-agent task decomposition, tool use correctness, session state consistency, goal completion rate
+
+### Haystack
+- **Type:** NLP pipeline framework
+- **Language:** Python
+- **Model support:** Model-agnostic
+- **Learning curve:** Intermediate
+- **Best for:** Explicit, auditable NLP pipelines, document processing with fine-grained control, enterprise search, regulated industries needing transparency
+- **Avoid if:** Rapid prototyping, multi-agent workflows, or you want a large community
+- **Strengths:** Explicit pipeline control, strong for structured data pipelines, good documentation
+- **Weaknesses:** Smaller community, less agent-oriented than alternatives
+- **Eval concerns:** Extraction accuracy, pipeline output validity, retrieval quality
+
+---
+
+## Decision Dimensions
+
+### By System Type
+
+| System Type | Primary Framework(s) | Key Eval Concerns |
+|-------------|---------------------|-------------------|
+| RAG / Knowledge Q&A | LlamaIndex, LangChain | Context faithfulness, hallucination, retrieval precision/recall |
+| Multi-agent orchestration | CrewAI, LangGraph, Google ADK | Task decomposition, handoff quality, goal completion |
+| Conversational assistants | OpenAI Agents SDK, Antigravity Agent SDK | Tone, safety, instruction following, escalation |
+| Structured data extraction | LangChain, LlamaIndex | Schema compliance, extraction accuracy |
+| Autonomous task agents | LangGraph, OpenAI Agents SDK | Safety guardrails, tool correctness, cost adherence |
+| Content generation | Antigravity Agent SDK, OpenAI Agents SDK | Brand voice, factual accuracy, tone |
+| Code automation | Antigravity Agent SDK | Code correctness, safety, test pass rate |
+
+### By Team Size and Stage
+
+| Context | Recommendation |
+|---------|----------------|
+| Solo dev, prototyping | OpenAI Agents SDK or CrewAI (fastest to running) |
+| Solo dev, RAG | LlamaIndex (batteries included) |
+| Team, production, stateful | LangGraph (best fault tolerance) |
+| Team, evolving requirements | LangChain (broadest escape hatches) |
+| Team, multi-agent | CrewAI (simplest role abstraction) |
+| Enterprise, .NET | AutoGen AG2 / Microsoft Agent Framework |
+
+### By Model Commitment
+
+| Preference | Framework |
+|-----------|-----------|
+| OpenAI-only | OpenAI Agents SDK |
+| Anthropic/Antigravity-only | Antigravity Agent SDK |
+| Google/Gemini-committed | Google ADK |
+| Model-agnostic (full flexibility) | LangChain, LlamaIndex, CrewAI, LangGraph, Haystack |
+
+---
+
+## Anti-Patterns
+
+1. **Using LangChain for simple chatbots** — Direct SDK call is less code, faster, and easier to debug
+2. **Using CrewAI for complex stateful workflows** — Checkpointing gaps will bite you in production
+3. **Using OpenAI Agents SDK with non-OpenAI models** — Loses the integration benefits you chose it for
+4. **Using LlamaIndex as a multi-agent framework** — It can do agents, but that's not its strength
+5. **Defaulting to LangChain without evaluating alternatives** — "Everyone uses it" ≠ right for your use case
+6. **Starting a new project on AutoGen (not AG2)** — AutoGen is in maintenance mode; use AG2 or wait for Microsoft Agent Framework GA
+7. **Choosing LangGraph for simple linear flows** — The graph overhead is not worth it; use LangChain chains instead
+8. **Ignoring vendor lock-in** — Provider-native SDKs (OpenAI, Antigravity) trade flexibility for integration depth; decide consciously
+
+---
+
+## Combination Plays (Multi-Framework Stacks)
+
+| Production Pattern | Stack |
+|-------------------|-------|
+| RAG with observability | LlamaIndex + LangSmith or Langfuse |
+| Stateful agent with RAG | LangGraph + LlamaIndex |
+| Multi-agent with tracing | CrewAI + Langfuse |
+| OpenAI agents with evals | OpenAI Agents SDK + Promptfoo or Braintrust |
+| Antigravity agents with MCP | Antigravity Agent SDK + LangSmith or Arize Phoenix |