@amg555/frameforge-mcp 2.7.6 → 2.7.8

package/.agent/skills/gstack-benchmark-models/SKILL.md.tmpl

@@ -1,155 +0,0 @@
----
-name: benchmark-models
-preamble-tier: 1
-version: 1.0.0
-description: |
-  Cross-model benchmark for gstack skills. Runs the same prompt through Claude,
-  GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
-  and optionally quality via LLM judge. Answers "which model is actually best
-  for this skill?" with data instead of vibes. Separate from /benchmark, which
-  measures web page performance. Use when: "benchmark models", "compare models",
-  "which model is best for X", "cross-model comparison", "model shootout". (gstack)
-voice-triggers:
-  - "compare models"
-  - "model shootout"
-  - "which model is best"
-triggers:
-  - cross model benchmark
-  - compare claude gpt gemini
-  - benchmark skill across models
-  - which model should I use
-allowed-tools:
-  - Bash
-  - Read
-  - AskUserQuestion
----
-
-{{PREAMBLE}}
-
-# /benchmark-models — Cross-Model Skill Benchmark
-
-You are running the `/benchmark-models` workflow. Wraps the `gstack-model-benchmark` binary with an interactive flow that picks a prompt, confirms providers, previews auth, and runs the benchmark.
-
-Different from `/benchmark` — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.
-
----
-
-## Step 0: Locate the binary
-
-```bash
-BIN="{{RUNTIME_ROOT}}/bin/gstack-model-benchmark"
-[ -x "$BIN" ] || BIN=".claude/skills/gstack/bin/gstack-model-benchmark"
-[ -x "$BIN" ] || { echo "ERROR: gstack-model-benchmark not found. Run ./setup in the gstack install dir." >&2; exit 1; }
-echo "BIN: $BIN"
-```
-
-If not found, stop and tell the user to reinstall gstack.
-
----
-
-## Step 1: Choose a prompt
-
-Use AskUserQuestion with the preamble format:
-- **Re-ground:** current project + branch.
-- **Simplify:** "A cross-model benchmark runs the same prompt through 2-3 AI models and shows you how they compare on speed, cost, and output quality. What prompt should we use?"
-- **RECOMMENDATION:** A because benchmarking against a real skill exposes tool-use differences, not just raw generation.
-- **Options:**
-  - A) Benchmark one of my gstack skills (we'll pick which skill next). Completeness: 10/10.
-  - B) Use an inline prompt — type it on the next turn. Completeness: 8/10.
-  - C) Point at a prompt file on disk — specify path on the next turn. Completeness: 8/10.
-
-If A: list top-level gstack skills that have SKILL.md files (from `find . -maxdepth 2 -name SKILL.md -not -path './.*'`), ask the user to pick one via a second AskUserQuestion. Use the picked SKILL.md path as the prompt file.
-
-If B: ask the user for the inline prompt. It never goes into a shell command: create a prompt file and write the prompt into it verbatim, then use the printed path as the prompt file.
-
-{{FREE_TEXT_FILE:PROMPT_FILE=benchmark-prompt}}
-
-If C: ask for the path. Verify it exists. Use as positional argument.
-
----
-
-## Step 2: Choose providers
-
-```bash
-"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini --dry-run
-```
-
-Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).
-
-If ALL three show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, or `gemini login` / `export GOOGLE_API_KEY`.
-
-If at least one is OK: AskUserQuestion:
-- **Simplify:** "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
-- **RECOMMENDATION:** A (all authed providers) because running as many as possible gives the richest comparison.
-- **Options:**
-  - A) All authed providers. Completeness: 10/10.
-  - B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
-  - C) Pick two — specify on next turn. Completeness: 8/10.
-
----
-
-## Step 3: Decide on judge
-
-```bash
-[ -n "$ANTHROPIC_API_KEY" ] || grep -q 'ANTHROPIC' "$HOME/.claude/.credentials.json" 2>/dev/null && echo "JUDGE_AVAILABLE" || echo "JUDGE_UNAVAILABLE"
-```
-
-If judge is available, AskUserQuestion:
-- **Simplify:** "The quality judge scores each model's output on a 0-10 scale using Anthropic's Claude as a tiebreaker. Adds about USD 0.05/run. Recommended if you care about output quality, not just latency and cost."
-- **RECOMMENDATION:** A — the whole point is comparing quality, not just speed.
-- **Options:**
-  - A) Enable judge (adds about USD 0.05). Completeness: 10/10.
-  - B) Skip judge — speed/cost/tokens only. Completeness: 7/10.
-
-If judge is NOT available, skip this question and omit the `--judge` flag.
-
----
-
-## Step 4: Run the benchmark
-
-Construct the command from Step 1, 2, 3 decisions:
-
-```bash
-PROMPT_PATH="<prompt-path>"
-[ -s "$PROMPT_PATH" ] || { echo "ERROR: $PROMPT_PATH is missing or empty; the benchmark needs a prompt file." >&2; exit 1; }
-"$BIN" --models <picked-models> [--judge] --output table -- "$PROMPT_PATH"
-```
-
-`<prompt-path>` is the prompt file from Step 1 (the SKILL.md path, the printed prompt file, or the user's path) and `<picked-models>` is the comma-separated list from Step 2. Use a path only if it has no `'`, `"`, backtick, `$` or `\`; otherwise copy the file into a new Step 1B prompt file and use that.
-
-Stream the output as it arrives. This is slow — each provider runs the prompt fully. Expect 30s-5min depending on prompt complexity and whether `--judge` is on.
-
----
-
-## Step 5: Interpret results
-
-After the table prints, summarize for the user:
-- **Fastest** — provider with lowest latency.
-- **Cheapest** — provider with lowest cost.
-- **Highest quality** (if `--judge` ran) — provider with highest score.
-- **Best overall** — use judgment. If judge ran: quality-weighted. Otherwise: note the tradeoff the user needs to make.
-
-If any provider hit an error (auth/timeout/rate_limit), call it out with the remediation path.
-
----
-
-## Step 6: Offer to save results
-
-AskUserQuestion:
-- **Simplify:** "Save this benchmark as JSON so you can compare future runs against it?"
-- **RECOMMENDATION:** A — skill performance drifts as providers update their models; a saved baseline catches quality regressions.
-- **Options:**
-  - A) Save to `~/.gstack/benchmarks/<date>-<skill-or-prompt-slug>.json`. Completeness: 10/10.
-  - B) Just print, don't save. Completeness: 5/10 (loses trend data).
-
-If A: re-run with `--output json` and tee to the dated file. Print the path so the user can diff future runs against it.
-
----
-
-## Important Rules
-
-- **Never run a real benchmark without Step 2's dry-run first.** Users need to see auth status before spending API calls.
-- **Never hardcode model names.** Always pass providers from user's Step 2 choice — the binary handles the rest.
-- **Never auto-include `--judge`.** It adds real cost; user must opt in.
-- **If zero providers are authed, STOP.** Don't attempt the benchmark — it produces no useful output.
-- **Cost is visible.** Every run shows per-provider cost in the table. Users should see it before the next run.

package/.agent/skills/gstack-browse/PLAN-snapshot-dropdown-interactive.md

@@ -1,102 +0,0 @@
-# Plan: Snapshot Dropdown/Autocomplete Interactive Element Detection
-
-## Problem
-
-`snapshot -i` misses dropdown/autocomplete items on modern web apps. These elements:
-1. Are often `<div>`/`<li>` with click handlers but no semantic ARIA roles
-2. Live inside dynamically-created portals/popovers (floating containers)
-3. Don't appear in Playwright's accessibility tree (`ariaSnapshot()`)
-
-The `-C` flag (cursor-interactive scan) was designed for this but:
-- Requires separate flag — agents using `-i` don't get it automatically
-- Skips elements that HAVE an ARIA role (even if the ARIA tree missed them)
-- Doesn't prioritize popover/portal containers where dropdown items live
-
-## Root Cause
-
-Playwright's `ariaSnapshot()` builds from the browser's accessibility tree. Dynamically-rendered popovers (React portals, Radix Popover, etc.) may not be in the accessibility tree if:
-- The component doesn't set ARIA roles
-- The portal renders outside the scoped `body` locator's subtree timing
-- The browser hasn't updated the accessibility tree yet after DOM mutation
-
-## Changes
-
-### 1. Auto-enable cursor-interactive scan with `-i` flag
-
-**File:** `browse/src/snapshot.ts`
-
-When `-i` (interactive) is passed, automatically include the cursor-interactive scan. This means agents always see clickable non-ARIA elements when they ask for interactive elements.
-
-The `-C` flag remains as a standalone option for non-interactive snapshots.
-
-```
-if (opts.interactive) {
-  opts.cursorInteractive = true;
-}
-```
-
-### 2. Add popover/portal priority scanning
-
-**File:** `browse/src/snapshot.ts` (inside cursor-interactive evaluate block)
-
-Before the general cursor:pointer scan, specifically scan for visible floating containers (popovers, dropdowns, menus) and include ALL their direct children as interactive:
-
-Detection heuristics for floating containers:
-- `position: fixed` or `position: absolute` with `z-index >= 10`
-- Has `role="listbox"`, `role="menu"`, `role="dialog"`, `role="tooltip"`, `[data-radix-popper-content-wrapper]`, `[data-floating-ui-portal]`, etc.
-- Appeared recently in the DOM (not in initial page load)
-- Is visible (`offsetParent !== null` or `position: fixed`)
-
-For each floating container, include child elements that:
-- Have text content
-- Are visible
-- Have cursor:pointer OR onclick OR role="option" OR role="menuitem"
-- Tag with reason `popover-child` for clarity
-
-### 3. Remove the `hasRole` skip in cursor-interactive scan
-
-**File:** `browse/src/snapshot.ts`
-
-Currently: `if (hasRole) continue;` — skips any element with an ARIA role, assuming the ARIA tree already captured it.
-
-Problem: if the ARIA tree MISSED the element (timing, portal, bad DOM structure), it falls through both systems.
-
-Fix: Only skip if the element's role is in `INTERACTIVE_ROLES` AND it was actually captured in the main refMap. Otherwise include it.
-
-Since we can't easily check the refMap from inside `page.evaluate()`, the simpler fix: remove the `hasRole` skip entirely for elements inside detected floating containers. For elements outside floating containers, keep the `hasRole` skip as-is (to avoid duplicates in normal page content).
-
-### 4. Add dropdown test fixture and tests
-
-**File:** `browse/test/fixtures/dropdown.html`
-
-HTML page with:
-- A combobox input that shows a dropdown on focus/type
-- Dropdown items as `<div>` with click handlers (no ARIA roles)
-- Dropdown items as `<li>` with `role="option"`
-- A React-portal-style container (`position: fixed`, high z-index)
-
-**File:** `browse/test/snapshot.test.ts`
-
-New test cases:
-- `snapshot -i` on dropdown page finds dropdown items via cursor scan
-- `snapshot -i` on dropdown page includes popover-child elements
-- `@c` refs from dropdown scan are clickable
-- Elements inside floating containers with ARIA roles are captured even when ARIA tree misses them
-
-## Rollout Risk
-
-**Low.** The `-C` scan is additive — it only adds `@c` refs, never removes `@e` refs. The change to auto-enable it with `-i` increases output size but agents already handle mixed ref types.
-
-**One concern:** The `-C` scan queries ALL elements (`document.querySelectorAll('*')`) which can be slow on heavy pages. For the popover-specific scan, we limit to elements inside detected floating containers, which is fast (small subtree).
-
-## Testing
-
-```bash
-cd /data/gstack/browse && bun test snapshot
-```
-
-## Files Changed
-
-1. `browse/src/snapshot.ts` — auto-enable -C with -i, popover scanning, remove hasRole skip in floating containers
-2. `browse/test/fixtures/dropdown.html` — new test fixture
-3. `browse/test/snapshot.test.ts` — new dropdown/popover test cases