@design-ai/cli 4.59.0 → 4.61.0

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "design-ai",
-  "version": "4.59.0",
+  "version": "4.61.0",
   "description": "Senior product designer for any AI coding agent. 20 skills, 17 commands, 4 review agents covering UI/UX, website improvement, design systems, motion, illustration, print, video, game UI, conversational, and spatial design. Korean market depth.",
   "author": {
     "name": "Sungjin Park",

package/CHANGELOG.md

@@ -2,6 +2,57 @@
 
 User-facing release notes for design-ai. Versions follow semver.
 
+## v4.61.0 — Ranked-search eval checkpoint (2026-07)
+
+Closes the last open follow-up from the AI-learning Phase A review (FU-3 in `docs/AI-LEARNING-PHASE2.md`): `design-ai search` gains eval-checkpoint modes mirroring the route/prompt/pack/learn pattern, run through the shipped ranked (BM25) search path. This is the standing evidence artifact for any future decision to promote `--ranked` to the default search mode. Additive and backward-compatible; default `search` behavior is unchanged.
+
+### Added
+- `search --eval-template [--json]` — emit a versioned ranked-search checkpoint template. The default template ships six cases that all pass against the shipped corpus, including the Korean particle-form regression queries (`버튼`, `버튼을`, `접근성이`) from the Hangul-bigram work, so the eval doubles as the retrieval-level Korean tokenization regression.
+- `search --eval [--strict] [--json]` with `--from-file file` / `--stdin` — run a checkpoint file's cases through the ranked search path and report pass/warn/fail per case; `--strict` exits nonzero on any failure. Case assertions: `expectRelPathIn` (top hit), `minHits`, and `matchedTokenIncludes`.
+
+### Changed
+- `search` help/usage lines (CLI `help` topic and command help) document the new eval modes; the shared smoke assertion pins were updated in lockstep.
+
+### Verified
+- All 8 audits passed.
+- `npm run release:check`.
+- `npm run release:metadata`.
+- `git diff --check`.
+- Main-branch GitHub Actions (`Design-AI audit`, `Deploy doc site`) passed for the constituent commits.
+
+### Versions
+- `package.json` + `.claude-plugin/plugin.json`: 4.60.0 → 4.61.0.
+- `vscode-extension/package.json`: remains 0.4.1.
+
+### What this enables
+- Any future `--ranked` default promotion is now evidence-backed and announced: the checkpoint demonstrates ranked-search quality (including Korean agglutinative queries) reproducibly, per the FU-3 gate set in the Phase A implementation review.
+
+## v4.60.0 — Agent SDK types and local-write namespace (2026-07)
+
+Extends the Agent SDK (`@design-ai/cli/sdk`) two ways, both additive and backward-compatible. First, hand-written TypeScript declarations now ship with the package, so TypeScript and editors resolve `@design-ai/cli/sdk` with full types — no `@types` install and no build step. Second, an opt-in `learn.*` namespace adds the SDK's first local-write verbs, keeping the eight Phase A verbs read-only and unchanged.
+
+### Added
+- `cli/sdk/index.d.ts` — hand-written type declarations for all eight read-only verbs (precise signatures and return interfaces, including `search` ranked/plain overloads, the `RouteConfidence` union, and optional `learningContext`/`recall`/`relatedKnowledge` fields) plus the `learn` namespace. Wired through an `exports` `types` condition, so `moduleResolution: node16`/`nodenext`/`bundler` consumers get types automatically. A `node --test` guard (`cli/sdk/types.test.mjs`) keeps the declarations in exact sync with the runtime exports; the declarations are verified to compile under `tsc --strict`.
+- `learn` — a frozen namespace grouping the SDK's only writing verbs (Phase B): `learn.remember(text, { category })`, `learn.feedback(text, { outcome, category })`, and `learn.captureFromCheck(artifact, { routeId })`. Each writes only the local learning profile (`DESIGN_AI_LEARNING_FILE`), never the network, mirroring the CLI's `learn remember` / `learn feedback` / `check --learn` write paths. There is no `filePath` or timestamp option — consumers target a profile via the env var, exactly like the CLI.
+
+### Changed
+- The eight Phase A verbs (`route`, `prompt`, `pack`, `search`, `recall`, `check`, `routes`, `version`) remain read-only and unchanged. `check` in particular stays read-only: capture lives in `learn.captureFromCheck`, never as a `check` option. The SDK contract test (`cli/sdk/index.test.mjs`) now pins both the eight read-only function verbs and the frozen `learn` namespace with its three verbs and their return-shape keys.
+- `tools/audit/package-smoke.py` now also exercises `learn.remember` from the installed tarball against a temp `DESIGN_AI_LEARNING_FILE`, asserting the write.
+
+### Verified
+- All 8 audits passed.
+- `npm run release:check`.
+- `npm run release:metadata`.
+- `git diff --check`.
+- Main-branch GitHub Actions (`Design-AI audit`, `Deploy doc site`) passed for the constituent commits.
+
+### Versions
+- `package.json` + `.claude-plugin/plugin.json`: 4.59.0 → 4.60.0.
+- `vscode-extension/package.json`: remains 0.4.1.
+
+### What this enables
+- TypeScript consumers get first-class types for the whole SDK with zero toolchain cost on either side, and agents can now record design preferences, feedback, and check-derived learnings through a single explicit, opt-in write namespace — while every read-only verb keeps its no-write guarantee.
+
 ## v4.59.0 — Agent SDK Phase A (2026-07)
 
 Adds a curated, semver-stable programmatic surface — `@design-ai/cli/sdk` — so an external Node.js program can call design-ai's deterministic design verbs as importable functions, without shelling out to the CLI or spawning the MCP server. The SDK is a thin adapter over the same `cli/lib` functions the CLI and MCP already call, so no design behavior changes. Phase A is read-only: no file writes, no network, no runtime dependencies. All additions are additive and backward-compatible.

package/README.ko.md

@@ -12,7 +12,7 @@
 
 > **모델이 아니에요. 파인튜닝도 아니에요.** 디자인 전문 지식을 구조화한 코퍼스 + 에이전트가 바로 실행할 수 있는 지시문이에요. 범용 LLM을 이번 세션에서만큼은 시니어 디자이너로 바꿔주는 셈이에요.
 
-> **배포 상태, 2026-07-04 확인:** 로컬 `npm run release:check`는 통과했고, GitHub Pages 문서는 live 상태이며, GitHub Release `v4.58.0`과 npm `@design-ai/cli@4.58.0`(`latest`) publish가 provenance와 함께 확인됐어요. 수정 후 live `npm run registry:smoke`가 published `@design-ai/cli@4.58.0`에 대해 깨끗이 통과했어요(retrieval 표면과 route enrichment 포함). Homebrew formula는 `v4.58.0`에 pinning되어 있고, VS Code Marketplace에는 `sungjin.design-ai-vscode@0.4.1`이 공개되어 있어요. 자세한 내용은 [`docs/external-status.md`](docs/external-status.md)를 확인하세요.
+> **배포 상태, 2026-07-05 확인:** 로컬 `npm run release:check`는 통과했고, GitHub Pages 문서는 live 상태이며, GitHub Release `v4.60.0`과 npm `@design-ai/cli@4.60.0`(`latest`) publish가 provenance와 함께 확인됐어요. 이번 릴리스는 Agent SDK(`@design-ai/cli/sdk`)에 TypeScript 선언과 opt-in `learn.*` 로컬 쓰기 네임스페이스를 추가했고, 읽기 전용 8개 verb는 그대로라 CLI 출력도 변함없어요. live `npm run registry:smoke`가 published `@design-ai/cli@4.60.0`에 대해 깨끗이 통과했어요(retrieval 표면과 route enrichment 포함). Homebrew formula는 `v4.60.0`에 pinning되어 있고, VS Code Marketplace에는 `sungjin.design-ai-vscode@0.4.1`이 공개되어 있어요. 자세한 내용은 [`docs/external-status.md`](docs/external-status.md)를 확인하세요.
 
 ## 한눈에 보는 커버리지
 
@@ -75,7 +75,7 @@ design-ai install
 /design-from-brief 프리랜서를 위한 한국 핀테크
 ```
 
-CLI 명령어: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
+CLI 명령어: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--strict] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
 
 자세한 내용은 [`docs/DISTRIBUTION.ko.md`](docs/DISTRIBUTION.ko.md)를 확인하세요.
 
@@ -172,7 +172,7 @@ design-ai는 한국 시장을 1순위로 만들어졌고, 글로벌 시장 패
 
 ## 상태
 
-전체 단계 로그는 [`docs/ROADMAP.md`](docs/ROADMAP.md), 현재 완료 범위는 [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md)에서 확인하세요. 현재 **v4.58.0**: public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, 90%+ component coverage가 완료됐어요.
+전체 단계 로그는 [`docs/ROADMAP.md`](docs/ROADMAP.md), 현재 완료 범위는 [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md)에서 확인하세요. 현재 **v4.60.0**: Agent SDK(`@design-ai/cli/sdk`) — TypeScript 선언 + opt-in `learn.*` 로컬 쓰기 네임스페이스, public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, 90%+ component coverage가 완료됐어요.
 
 핵심 디자인 컨설팅 워크플로우는 로컬 릴리스 기준으로 준비되어 있어요. 웹사이트 개선 컨트롤 타워는 zero-dependency static Web App과 `website-improvement` route/skill/command로 제공되고, Site Profile, audit checklist, MCP readiness, refactor prompt, handoff evidence tracking, bundle export/verify/repair를 한 번에 다뤄요. 로컬 학습 선호도는 `design-ai learn`으로 관리해요 — profile bootstrap, feedback 캡처, 읽기 전용 signal registry, 반복 QA 신호에서 만드는 skill 제안, 그리고 backup/restore/curate/audit까지 전부 로컬에서만 동작하는 opt-in 기능이에요. AI 모델 학습이나 fine-tuning은 여전히 현재 배포 범위 밖이에요.
 

package/README.md

@@ -12,7 +12,7 @@ A model-agnostic design knowledge base + skill system. Drop it in front of any A
 
 > **Not a model. Not a fine-tune.** A structured corpus of design expertise + agent-ready instructions that turn a general-purpose LLM into an expert.
 
-> **Distribution status, checked 2026-07-04:** local `npm run release:check` passes, GitHub Pages docs are live, GitHub Release `v4.58.0` is published, `@design-ai/cli@4.58.0` is public on npm (`latest`) with provenance, and the live `npm run registry:smoke` passes cleanly against `@design-ai/cli@4.58.0`. The Homebrew formula is pinned to `v4.58.0`, and `sungjin.design-ai-vscode@0.4.1` is public on the VS Code Marketplace. See [`docs/external-status.md`](docs/external-status.md).
+> **Distribution status, checked 2026-07-05:** local `npm run release:check` passes, GitHub Pages docs are live, GitHub Release `v4.60.0` is published, `@design-ai/cli@4.60.0` is public on npm (`latest`) with provenance, and the live `npm run registry:smoke` passes cleanly against `@design-ai/cli@4.60.0`. The Homebrew formula is pinned to `v4.60.0`, and `sungjin.design-ai-vscode@0.4.1` is public on the VS Code Marketplace. See [`docs/external-status.md`](docs/external-status.md).
 
 ## Coverage at a glance
 
@@ -75,7 +75,7 @@ Any available path installs all 20 skills, 17 commands, and 4 agents under `~/.c
 /design-from-brief Korean fintech for freelancers
 ```
 
-CLI commands: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
+CLI commands: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--strict] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
 
 See [`docs/DISTRIBUTION.md`](docs/DISTRIBUTION.md) for the full distribution guide.
 
@@ -214,7 +214,7 @@ Refresh refs/ on demand: `./tools/extractors/run-all.sh`.
 
 ## Status
 
-See [`docs/ROADMAP.md`](docs/ROADMAP.md) for the full phase log and [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md) for the current completion boundary. Currently at **v4.58.0**: public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, and 90%+ component coverage are complete.
+See [`docs/ROADMAP.md`](docs/ROADMAP.md) for the full phase log and [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md) for the current completion boundary. Currently at **v4.60.0**: the Agent SDK (`@design-ai/cli/sdk`) with shipped TypeScript types and the opt-in `learn.*` local-write namespace, public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, and 90%+ component coverage are complete.
 
 Core design consulting workflows are locally release-ready. The website improvement control tower ships as a zero-dependency static Web App plus a `website-improvement` route/skill/command, covering Site Profiles, audit checklists, MCP readiness, refactor prompts, handoff evidence tracking, and bundle export/verify/repair. Local learning preferences are available through `design-ai learn` — profile bootstrap, feedback capture, a read-only signals registry, skill-proposal generation from repeated QA signals, and full backup/restore/curate/audit tooling, all local-only and opt-in. AI model training or fine-tuning remains outside the shipped scope.
 

package/cli/commands/help.mjs

@@ -33,7 +33,7 @@ export const HELP_COMMANDS = [
   { topic: "uninstall", usage: "uninstall [--json]", description: "Remove symlinks (keeps source files)" },
   { topic: "status", usage: "status [--json]", description: "Show what's installed (use VERBOSE=1 for full list)" },
   { topic: "list", usage: "list [skills|commands|agents] [--json]", description: "List catalog from the plugin manifest" },
-  { topic: "search", usage: "search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]", description: "Search the local markdown corpus" },
+  { topic: "search", usage: "search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--strict] [--json]", description: "Search the local markdown corpus and ranked-search eval checkpoints" },
   { topic: "index", usage: "index [--build|--status|--verify] [--json] [--embeddings [--provider \"cmd args\"]]", description: "Build, inspect, and verify the local retrieval index" },
   { topic: "show", usage: "show <file[:line]> [--lines N:M] [--context N] [--json]", description: "Print a corpus file or line range" },
   { topic: "route", usage: "route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N]", description: "Recommend commands, skills, knowledge, and route eval checkpoints" },

package/cli/commands/search.mjs

@@ -2,13 +2,19 @@
 
 import { DESIGN_AI_HOME } from "../lib/paths.mjs";
 import { dim, header, info, warn } from "../lib/log.mjs";
+import { resolveBriefInput } from "../lib/brief.mjs";
 import { configuredEmbeddingProvider, defaultConfigFile } from "../lib/local-config.mjs";
+import { buildSearchEvalTemplate, searchEvalReport } from "../lib/search-eval.mjs";
 import { embeddingRerankSearch, rankedSearchCorpus } from "../lib/search-ranked.mjs";
 import { formatSearchJson, parseSearchArgs, searchCorpus } from "../lib/search.mjs";
 
 function printHelp() {
-  console.log("Usage:  design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]\n");
-  console.log("Searches markdown files across knowledge, examples, skills, docs, agents, and commands.\n");
+  console.log("Usage:  design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]");
+  console.log("        design-ai search --eval-template [--json]");
+  console.log("        design-ai search --eval --from-file search-eval.json [--strict] [--json]");
+  console.log("        cat search-eval.json | design-ai search --eval --stdin [--strict] [--json]\n");
+  console.log("Searches markdown files across knowledge, examples, skills, docs, agents, and commands.");
+  console.log("Search eval is read-only and checks deterministic ranked-search fixtures.\n");
   console.log("Options:");
   console.log("  --limit N   Maximum hits to return, 1-500. Default: 20");
   console.log("  --dir kind  Restrict to one corpus directory. Repeatable.");
@@ -18,7 +24,18 @@ function printHelp() {
   console.log("                        Requires --ranked and a configured provider; falls back to the");
   console.log("                        lexical ranking (with a notice) if no provider or sidecar is available");
   console.log("  --provider \"cmd args\" Embedding provider command for this invocation (overrides config)");
+  console.log("  --from-file file  Read ranked-search eval JSON from a file (with --eval)");
+  console.log("  --stdin           Read ranked-search eval JSON from standard input (with --eval)");
+  console.log("  --eval-template   Generate a runnable ranked-search eval checkpoint JSON template");
+  console.log("  --eval            Run deterministic ranked-search checkpoint cases");
+  console.log("  --strict          With --eval, exit non-zero on warning or failure");
   console.log("  --json      Emit machine-readable results");
+  console.log("");
+  console.log("Examples:");
+  console.log("  design-ai search Pretendard");
+  console.log("  design-ai search \"button component\" --ranked");
+  console.log("  design-ai search --eval-template --json > search-eval.json");
+  console.log("  design-ai search --eval --from-file search-eval.json --strict --json");
 }
 
 function parseProviderFlag(value) {
@@ -45,6 +62,37 @@ function printRankedHits(query, hits, backend) {
   info(`Hits: ${hits.length}`);
 }
 
+function printSearchEvalTemplate(template) {
+  header("design-ai search", "Ranked-search eval checkpoint template");
+  info(`Source: ${DESIGN_AI_HOME}`);
+  info(`Cases: ${template.cases.length}`);
+  console.log();
+  console.log("Write the JSON below to a file, edit cases if needed, then run:");
+  console.log("design-ai search --eval --from-file search-eval.json --strict");
+  console.log();
+  console.log(formatSearchJson(template));
+}
+
+function printSearchEvalReport(report) {
+  header("design-ai search", "Ranked-search eval report");
+  info(`Source: ${report.source}`);
+  info(`Status: ${report.status}`);
+  info(`Cases: ${report.summary.total} (${report.summary.pass} pass, ${report.summary.warn} warn, ${report.summary.fail} fail)`);
+  console.log();
+
+  for (const result of report.cases) {
+    const top = result.topRelPath || "none";
+    console.log(`${result.status.toUpperCase()} ${result.id}`);
+    console.log(`   query:    ${result.query}`);
+    console.log(`   top:      ${top} ${dim(`(hits ${result.hitCount})`)}`);
+    if (result.matchedTokens.length > 0) {
+      console.log(`   matched:  ${result.matchedTokens.join(", ")}`);
+    }
+    console.log(`   result:   ${result.message}`);
+    console.log();
+  }
+}
+
 export async function runSearch(args) {
   const parsed = parseSearchArgs(args);
   if (parsed.help) {
@@ -52,6 +100,37 @@ export async function runSearch(args) {
     return;
   }
 
+  if (parsed.evalTemplate) {
+    const template = buildSearchEvalTemplate({ sourceRoot: DESIGN_AI_HOME });
+    if (parsed.json) {
+      console.log(formatSearchJson(template));
+      return;
+    }
+    printSearchEvalTemplate(template);
+    return;
+  }
+
+  if (parsed.eval) {
+    const evalText = resolveBriefInput(parsed);
+    const report = searchEvalReport({
+      evalText,
+      source: parsed.fromFile || "stdin",
+      sourceRoot: DESIGN_AI_HOME,
+      limit: parsed.limit,
+    });
+
+    if (parsed.json) {
+      console.log(formatSearchJson(report));
+    } else {
+      printSearchEvalReport(report);
+    }
+
+    if (parsed.strict && report.status !== "pass") {
+      process.exitCode = 1;
+    }
+    return;
+  }
+
   if (parsed.embeddings && !parsed.ranked) {
     throw new Error("--embeddings requires --ranked");
   }

package/cli/lib/search-eval.mjs

@@ -0,0 +1,241 @@
+// Ranked-search eval checkpoint generation and reports for `design-ai search`.
+// Mirrors cli/lib/route.mjs's eval-checkpoint pattern (docs/AI-LEARNING-PHASE2.md
+// FU-3): runs hand-authored / generated cases through the shipped deterministic
+// ranked search path (rankedSearchCorpus, BM25-style lexical scorer) and reports
+// pass/warn/fail. This is the standing evidence artifact for any future decision
+// to promote `--ranked` to the default `search` behavior.
+
+import { rankedSearchCorpus } from "./search-ranked.mjs";
+import { DEFAULT_SEARCH_DIRS } from "./search.mjs";
+
+export const SEARCH_EVAL_VERSION = 1;
+// Default per-case result limit when a case does not specify its own `limit`.
+// Kept small and eval-appropriate (route's default is 3; search corpus hits are
+// noisier, so 10 gives enough headroom for `minHits` assertions without needing
+// a large recall window).
+export const SEARCH_EVAL_DEFAULT_LIMIT = 10;
+
+function isoTimestamp(now = new Date()) {
+  return (now instanceof Date ? now : new Date(now)).toISOString();
+}
+
+export function buildSearchEvalTemplate({ sourceRoot, generatedAt = new Date() } = {}) {
+  return {
+    version: SEARCH_EVAL_VERSION,
+    generatedAt: isoTimestamp(generatedAt),
+    description: "Deterministic ranked-search checkpoints for design-ai's BM25-style lexical retrieval.",
+    cases: [
+      {
+        id: "korean-button-stem",
+        query: "버튼",
+        minHits: 1,
+      },
+      {
+        id: "korean-button-particle",
+        query: "버튼을",
+        minHits: 1,
+      },
+      {
+        id: "korean-accessibility-particle",
+        query: "접근성이",
+        minHits: 1,
+      },
+      {
+        id: "accessibility-english",
+        query: "accessibility",
+        minHits: 5,
+        matchedTokenIncludes: ["accessibility"],
+      },
+      {
+        id: "button-component-spec-english",
+        query: "button component spec",
+        minHits: 5,
+        matchedTokenIncludes: ["button", "component", "spec"],
+      },
+      {
+        id: "color-palette-english",
+        query: "color palette",
+        expectRelPathIn: ["commands/palette-from-brand.md"],
+        minHits: 5,
+        matchedTokenIncludes: ["color", "palette"],
+      },
+    ],
+  };
+}
+
+function searchEvalStatus(counts) {
+  if (counts.fail > 0) return "fail";
+  if (counts.warn > 0) return "warn";
+  return "pass";
+}
+
+export function normalizeSearchEvalPayload(evalText, source = "search-eval.json") {
+  let payload;
+  try {
+    payload = JSON.parse(evalText);
+  } catch (err) {
+    throw new Error(`Could not parse search eval JSON from ${source}: ${err.message}`);
+  }
+
+  if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
+    throw new Error("Search eval payload must be a JSON object");
+  }
+  if (payload.version !== SEARCH_EVAL_VERSION) {
+    throw new Error(`Search eval payload version must be ${SEARCH_EVAL_VERSION}`);
+  }
+  if (!Array.isArray(payload.cases)) {
+    throw new Error("Search eval payload must include a cases array");
+  }
+
+  return payload;
+}
+
+function normalizeStringList(value, { field, id }) {
+  if (value === undefined || value === null) return [];
+  if (!Array.isArray(value)) {
+    throw new Error(`Search eval case ${id} field ${field} must be an array`);
+  }
+  return value.map((item) => String(item || "").trim()).filter(Boolean);
+}
+
+export function normalizeSearchEvalCase(rawCase, index) {
+  if (!rawCase || typeof rawCase !== "object" || Array.isArray(rawCase)) {
+    throw new Error(`Search eval case ${index + 1} must be a JSON object`);
+  }
+
+  const id = String(rawCase.id || `case-${index + 1}`).trim();
+  if (!id) throw new Error(`Search eval case ${index + 1} is missing id`);
+
+  const query = String(rawCase.query || "").trim();
+  if (!query) throw new Error(`Search eval case ${id} is missing query`);
+
+  const dirs = normalizeStringList(rawCase.dirs, { field: "dirs", id });
+  for (const dir of dirs) {
+    if (!DEFAULT_SEARCH_DIRS.includes(dir)) {
+      throw new Error(`Search eval case ${id} field dirs has invalid value: ${dir}`);
+    }
+  }
+
+  const limit = rawCase.limit === undefined || rawCase.limit === null
+    ? SEARCH_EVAL_DEFAULT_LIMIT
+    : Number(rawCase.limit);
+  if (!Number.isInteger(limit) || limit < 1 || limit > 500) {
+    throw new Error(`Search eval case ${id} limit must be an integer from 1 to 500`);
+  }
+
+  const expectRelPathIn = normalizeStringList(rawCase.expectRelPathIn, { field: "expectRelPathIn", id });
+  const matchedTokenIncludes = normalizeStringList(rawCase.matchedTokenIncludes, { field: "matchedTokenIncludes", id })
+    .map((token) => token.toLowerCase());
+
+  let minHits;
+  if (rawCase.minHits === undefined || rawCase.minHits === null) {
+    minHits = undefined;
+  } else {
+    minHits = Number(rawCase.minHits);
+    if (!Number.isInteger(minHits) || minHits < 0) {
+      throw new Error(`Search eval case ${id} field minHits must be a non-negative integer`);
+    }
+  }
+
+  return {
+    id,
+    query,
+    dirs: dirs.length > 0 ? dirs : DEFAULT_SEARCH_DIRS,
+    limit,
+    expectRelPathIn,
+    minHits,
+    matchedTokenIncludes,
+  };
+}
+
+export function evaluateSearchEvalCase(testCase, sourceRoot, defaultLimit) {
+  const limit = testCase.limit || defaultLimit || SEARCH_EVAL_DEFAULT_LIMIT;
+  const { hits } = rankedSearchCorpus({
+    query: testCase.query,
+    designAiPath: sourceRoot,
+    dirs: testCase.dirs,
+    limit,
+  });
+  const topHit = hits[0] || null;
+  const topRelPath = topHit?.relPath || "";
+  const topMatchedTokens = (topHit?.matchedTokens || []).map((token) => token.toLowerCase());
+
+  const hasAssertions = testCase.expectRelPathIn.length > 0
+    || testCase.minHits !== undefined
+    || testCase.matchedTokenIncludes.length > 0;
+
+  let status = "pass";
+  let message = "All checkpoint expectations were satisfied.";
+
+  if (!hasAssertions) {
+    status = "warn";
+    message = "Case has no expectRelPathIn, minHits, or matchedTokenIncludes assertion.";
+  } else if (testCase.expectRelPathIn.length > 0 && !testCase.expectRelPathIn.includes(topRelPath)) {
+    status = "fail";
+    message = topRelPath
+      ? `Expected top hit in [${testCase.expectRelPathIn.join(", ")}], but top hit was ${topRelPath}.`
+      : `Expected top hit in [${testCase.expectRelPathIn.join(", ")}], but there were no hits.`;
+  } else if (testCase.minHits !== undefined && hits.length < testCase.minHits) {
+    status = "fail";
+    message = `Expected at least ${testCase.minHits} hit(s), got ${hits.length}.`;
+  } else if (
+    testCase.matchedTokenIncludes.length > 0
+    && !testCase.matchedTokenIncludes.every((token) => topMatchedTokens.includes(token))
+  ) {
+    status = "fail";
+    const missing = testCase.matchedTokenIncludes.filter((token) => !topMatchedTokens.includes(token));
+    message = topHit
+      ? `Top hit ${topRelPath} did not match expected token(s): ${missing.join(", ")}.`
+      : `Expected matched tokens [${missing.join(", ")}], but there were no hits.`;
+  }
+
+  return {
+    id: testCase.id,
+    status,
+    message,
+    query: testCase.query,
+    dirs: testCase.dirs,
+    limit,
+    topRelPath,
+    hitCount: hits.length,
+    matchedTokens: topHit?.matchedTokens || [],
+    expectRelPathIn: testCase.expectRelPathIn,
+    minHits: testCase.minHits ?? null,
+    matchedTokenIncludes: testCase.matchedTokenIncludes,
+    hits,
+  };
+}
+
+export function searchEvalReport({
+  evalText,
+  source = "search-eval.json",
+  sourceRoot,
+  limit = SEARCH_EVAL_DEFAULT_LIMIT,
+  generatedAt = new Date(),
+}) {
+  const payload = normalizeSearchEvalPayload(evalText, source);
+  const normalizedCases = payload.cases.map((testCase, index) => normalizeSearchEvalCase(testCase, index));
+  const results = normalizedCases.map((testCase) => evaluateSearchEvalCase(testCase, sourceRoot, limit));
+  const counts = results.reduce(
+    (acc, result) => ({
+      ...acc,
+      [result.status]: acc[result.status] + 1,
+    }),
+    { pass: 0, warn: 0, fail: 0 },
+  );
+  const summary = {
+    total: results.length,
+    pass: counts.pass,
+    warn: counts.warn,
+    fail: counts.fail,
+  };
+
+  return {
+    evalVersion: payload.version,
+    source,
+    generatedAt: isoTimestamp(generatedAt),
+    status: searchEvalStatus(counts),
+    summary,
+    cases: results,
+  };
+}

package/cli/lib/search.mjs

@@ -7,6 +7,7 @@ import {
 } from "node:fs";
 import path from "node:path";
 
+import { parseBriefSourceFlag } from "./brief.mjs";
 import { expectedValueMessage, unknownOptionMessage } from "./suggest.mjs";
 
 export const DEFAULT_SEARCH_DIRS = [
@@ -20,7 +21,21 @@ export const DEFAULT_SEARCH_DIRS = [
 
 const PREVIEW_LEN = 120;
 const PREVIEW_BEFORE = 50;
-const SEARCH_OPTIONS = ["-h", "--help", "--json", "--limit", "--dir", "--ranked", "--embeddings", "--provider"];
+const SEARCH_OPTIONS = [
+  "-h",
+  "--help",
+  "--json",
+  "--limit",
+  "--dir",
+  "--ranked",
+  "--embeddings",
+  "--provider",
+  "--eval-template",
+  "--eval",
+  "--strict",
+  "--from-file",
+  "--stdin",
+];
 
 function exists(p) {
   try {
@@ -118,11 +133,17 @@ export function parseSearchArgs(args) {
     ranked: false,
     embeddings: false,
     provider: "",
+    evalTemplate: false,
+    eval: false,
+    strict: false,
+    fromFile: "",
+    stdin: false,
     help: false,
   };
 
   for (let i = 0; i < args.length; i += 1) {
     const arg = args[i];
+    out.index = i;
     if (arg === "-h" || arg === "--help") {
       out.help = true;
     } else if (arg === "--json") {
@@ -131,9 +152,17 @@ export function parseSearchArgs(args) {
       out.ranked = true;
     } else if (arg === "--embeddings") {
       out.embeddings = true;
+    } else if (arg === "--eval-template") {
+      out.evalTemplate = true;
+    } else if (arg === "--eval") {
+      out.eval = true;
+    } else if (arg === "--strict") {
+      out.strict = true;
     } else if (arg === "--provider") {
       out.provider = args[i + 1] || "";
       i += 1;
+    } else if (parseBriefSourceFlag(args, out)) {
+      i = out.index;
     } else if (arg === "--limit") {
       const next = args[i + 1];
       const limit = Number(next);
@@ -156,8 +185,34 @@ export function parseSearchArgs(args) {
     }
   }
 
+  if (out.eval && out.evalTemplate) {
+    throw new Error("Choose either --eval-template or --eval, not both");
+  }
+  if (out.strict && !out.eval) {
+    throw new Error("--strict can only be used with --eval");
+  }
+  if (
+    out.evalTemplate
+    && (out.fromFile || out.stdin || out.queryParts.length > 0 || out.ranked || out.embeddings || out.provider || out.dirs.length > 0)
+  ) {
+    throw new Error("--eval-template cannot be combined with a query, --from-file, --stdin, --ranked, --embeddings, --provider, or --dir");
+  }
+  if (out.eval && (!out.fromFile && !out.stdin)) {
+    throw new Error("--eval requires --from-file or --stdin");
+  }
+  if ((out.fromFile || out.stdin) && !out.eval) {
+    throw new Error("--from-file and --stdin require --eval");
+  }
+  if (
+    out.eval
+    && (out.queryParts.length > 0 || out.ranked || out.embeddings || out.provider || out.dirs.length > 0)
+  ) {
+    throw new Error("--eval cannot be combined with an inline query, --ranked, --embeddings, --provider, or --dir");
+  }
+
   return {
     ...out,
+    index: undefined,
     query: out.queryParts.join(" ").trim(),
     dirs: out.dirs.length > 0 ? out.dirs : DEFAULT_SEARCH_DIRS,
   };