@design-ai/cli 4.60.0 → 4.61.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "design-ai",
3
- "version": "4.60.0",
3
+ "version": "4.61.0",
4
4
  "description": "Senior product designer for any AI coding agent. 20 skills, 17 commands, 4 review agents covering UI/UX, website improvement, design systems, motion, illustration, print, video, game UI, conversational, and spatial design. Korean market depth.",
5
5
  "author": {
6
6
  "name": "Sungjin Park",
package/CHANGELOG.md CHANGED
@@ -2,6 +2,31 @@
2
2
 
3
3
  User-facing release notes for design-ai. Versions follow semver.
4
4
 
5
+ ## v4.61.0 — Ranked-search eval checkpoint (2026-07)
6
+
7
+ Closes the last open follow-up from the AI-learning Phase A review (FU-3 in `docs/AI-LEARNING-PHASE2.md`): `design-ai search` gains eval-checkpoint modes mirroring the route/prompt/pack/learn pattern, run through the shipped ranked (BM25) search path. This is the standing evidence artifact for any future decision to promote `--ranked` to the default search mode. Additive and backward-compatible; default `search` behavior is unchanged.
8
+
9
+ ### Added
10
+ - `search --eval-template [--json]` — emit a versioned ranked-search checkpoint template. The default template ships six cases that all pass against the shipped corpus, including the Korean particle-form regression queries (`버튼`, `버튼을`, `접근성이`) from the Hangul-bigram work, so the eval doubles as the retrieval-level Korean tokenization regression.
11
+ - `search --eval [--strict] [--json]` with `--from-file file` / `--stdin` — run a checkpoint file's cases through the ranked search path and report pass/warn/fail per case; `--strict` exits nonzero on any failure. Case assertions: `expectRelPathIn` (top hit), `minHits`, and `matchedTokenIncludes`.
12
+
13
+ ### Changed
14
+ - `search` help/usage lines (CLI `help` topic and command help) document the new eval modes; the shared smoke assertion pins were updated in lockstep.
15
+
16
+ ### Verified
17
+ - All 8 audits passed.
18
+ - `npm run release:check`.
19
+ - `npm run release:metadata`.
20
+ - `git diff --check`.
21
+ - Main-branch GitHub Actions (`Design-AI audit`, `Deploy doc site`) passed for the constituent commits.
22
+
23
+ ### Versions
24
+ - `package.json` + `.claude-plugin/plugin.json`: 4.60.0 → 4.61.0.
25
+ - `vscode-extension/package.json`: remains 0.4.1.
26
+
27
+ ### What this enables
28
+ - Any future `--ranked` default promotion is now evidence-backed and announced: the checkpoint demonstrates ranked-search quality (including Korean agglutinative queries) reproducibly, per the FU-3 gate set in the Phase A implementation review.
29
+
5
30
  ## v4.60.0 — Agent SDK types and local-write namespace (2026-07)
6
31
 
7
32
  Extends the Agent SDK (`@design-ai/cli/sdk`) two ways, both additive and backward-compatible. First, hand-written TypeScript declarations now ship with the package, so TypeScript and editors resolve `@design-ai/cli/sdk` with full types — no `@types` install and no build step. Second, an opt-in `learn.*` namespace adds the SDK's first local-write verbs, keeping the eight Phase A verbs read-only and unchanged.
package/README.ko.md CHANGED
@@ -12,7 +12,7 @@
12
12
 
13
13
  > **모델이 아니에요. 파인튜닝도 아니에요.** 디자인 전문 지식을 구조화한 코퍼스 + 에이전트가 바로 실행할 수 있는 지시문이에요. 범용 LLM을 이번 세션에서만큼은 시니어 디자이너로 바꿔주는 셈이에요.
14
14
 
15
- > **배포 상태, 2026-07-04 확인:** 로컬 `npm run release:check`는 통과했고, GitHub Pages 문서는 live 상태이며, GitHub Release `v4.59.0`과 npm `@design-ai/cli@4.59.0`(`latest`) publish가 provenance와 함께 확인됐어요. 이번 릴리스는 읽기 전용 Agent SDK 표면(`@design-ai/cli/sdk`, 8개 어댑터 verb)을 추가했고 additive라 CLI 출력은 그대로예요. live `npm run registry:smoke`가 published `@design-ai/cli@4.59.0`에 대해 깨끗이 통과했어요(retrieval 표면과 route enrichment 포함). Homebrew formula는 `v4.59.0`에 pinning되어 있고, VS Code Marketplace에는 `sungjin.design-ai-vscode@0.4.1`이 공개되어 있어요. 자세한 내용은 [`docs/external-status.md`](docs/external-status.md)를 확인하세요.
15
+ > **배포 상태, 2026-07-05 확인:** 로컬 `npm run release:check`는 통과했고, GitHub Pages 문서는 live 상태이며, GitHub Release `v4.60.0`과 npm `@design-ai/cli@4.60.0`(`latest`) publish가 provenance와 함께 확인됐어요. 이번 릴리스는 Agent SDK(`@design-ai/cli/sdk`)에 TypeScript 선언과 opt-in `learn.*` 로컬 쓰기 네임스페이스를 추가했고, 읽기 전용 8개 verb 그대로라 CLI 출력도 변함없어요. live `npm run registry:smoke`가 published `@design-ai/cli@4.60.0`에 대해 깨끗이 통과했어요(retrieval 표면과 route enrichment 포함). Homebrew formula는 `v4.60.0`에 pinning되어 있고, VS Code Marketplace에는 `sungjin.design-ai-vscode@0.4.1`이 공개되어 있어요. 자세한 내용은 [`docs/external-status.md`](docs/external-status.md)를 확인하세요.
16
16
 
17
17
  ## 한눈에 보는 커버리지
18
18
 
@@ -75,7 +75,7 @@ design-ai install
75
75
  /design-from-brief 프리랜서를 위한 한국 핀테크
76
76
  ```
77
77
 
78
- CLI 명령어: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
78
+ CLI 명령어: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--strict] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
79
79
 
80
80
  자세한 내용은 [`docs/DISTRIBUTION.ko.md`](docs/DISTRIBUTION.ko.md)를 확인하세요.
81
81
 
@@ -172,7 +172,7 @@ design-ai는 한국 시장을 1순위로 만들어졌고, 글로벌 시장 패
172
172
 
173
173
  ## 상태
174
174
 
175
- 전체 단계 로그는 [`docs/ROADMAP.md`](docs/ROADMAP.md), 현재 완료 범위는 [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md)에서 확인하세요. 현재 **v4.59.0**: 읽기 전용 Agent SDK(`@design-ai/cli/sdk`), public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, 90%+ component coverage가 완료됐어요.
175
+ 전체 단계 로그는 [`docs/ROADMAP.md`](docs/ROADMAP.md), 현재 완료 범위는 [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md)에서 확인하세요. 현재 **v4.60.0**: Agent SDK(`@design-ai/cli/sdk`) — TypeScript 선언 + opt-in `learn.*` 로컬 쓰기 네임스페이스, public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, 90%+ component coverage가 완료됐어요.
176
176
 
177
177
  핵심 디자인 컨설팅 워크플로우는 로컬 릴리스 기준으로 준비되어 있어요. 웹사이트 개선 컨트롤 타워는 zero-dependency static Web App과 `website-improvement` route/skill/command로 제공되고, Site Profile, audit checklist, MCP readiness, refactor prompt, handoff evidence tracking, bundle export/verify/repair를 한 번에 다뤄요. 로컬 학습 선호도는 `design-ai learn`으로 관리해요 — profile bootstrap, feedback 캡처, 읽기 전용 signal registry, 반복 QA 신호에서 만드는 skill 제안, 그리고 backup/restore/curate/audit까지 전부 로컬에서만 동작하는 opt-in 기능이에요. AI 모델 학습이나 fine-tuning은 여전히 현재 배포 범위 밖이에요.
178
178
 
package/README.md CHANGED
@@ -12,7 +12,7 @@ A model-agnostic design knowledge base + skill system. Drop it in front of any A
12
12
 
13
13
  > **Not a model. Not a fine-tune.** A structured corpus of design expertise + agent-ready instructions that turn a general-purpose LLM into an expert.
14
14
 
15
- > **Distribution status, checked 2026-07-04:** local `npm run release:check` passes, GitHub Pages docs are live, GitHub Release `v4.59.0` is published, `@design-ai/cli@4.59.0` is public on npm (`latest`) with provenance, and the live `npm run registry:smoke` passes cleanly against `@design-ai/cli@4.59.0`. The Homebrew formula is pinned to `v4.59.0`, and `sungjin.design-ai-vscode@0.4.1` is public on the VS Code Marketplace. See [`docs/external-status.md`](docs/external-status.md).
15
+ > **Distribution status, checked 2026-07-05:** local `npm run release:check` passes, GitHub Pages docs are live, GitHub Release `v4.60.0` is published, `@design-ai/cli@4.60.0` is public on npm (`latest`) with provenance, and the live `npm run registry:smoke` passes cleanly against `@design-ai/cli@4.60.0`. The Homebrew formula is pinned to `v4.60.0`, and `sungjin.design-ai-vscode@0.4.1` is public on the VS Code Marketplace. See [`docs/external-status.md`](docs/external-status.md).
16
16
 
17
17
  ## Coverage at a glance
18
18
 
@@ -75,7 +75,7 @@ Any available path installs all 20 skills, 17 commands, and 4 agents under `~/.c
75
75
  /design-from-brief Korean fintech for freelancers
76
76
  ```
77
77
 
78
- CLI commands: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
78
+ CLI commands: `design-ai install [--json]`, `update [--dry-run] [--json]`, `uninstall [--json]`, `status [--json]`, `list [skills|commands|agents] [--json]`, `route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N] [--explain] [--strict] [--json]`, `routes [--json]`, `prompt <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--strict] [--json]`, `pack <brief|--from-file file|--stdin|--eval-template|--eval> [--out file] [--route id] [--with-learning] [--learning-category kind] [--learning-limit N] [--with-recall] [--recall-limit N] [--max-bytes N] [--strict] [--json]`, `learn [--init|--remember text|--feedback text|--list|--export|--query text|--explain|--recall query|--backup|--redact|--verify|--diff|--restore|--restore-backups [--prune]|--import|--audit [--fix]|--curate|--stats|--usage|--signals [--strict]|--agent-backlog [--strict]|--propose-skills [--min-evidence N] [--review-file path] [--review-check|--apply-plan] [--strict]|--eval-template|--eval [--strict]|--forget id|--clear] [--json|--report|--patch|--review-template] [--out file]`, `check <artifact.md|--stdin|--examples> [--route id|--all-routes] [--issues-only] [--strict] [--learn [--yes] [--learning-file path]] [--json]`, `workspace [--root path] [--learning-file path] [--learning-usage path] [--learning-eval path] [--strict] [--json]`, `site <workspace.json|--stdin> [--strict] [--json|--mcp-check [--probes]|--mcp-plan [--probes] [--json]|--next-actions [--json]|--graph|--tasks|--bundle|--report|--prompts|--prompt id [--task id]] [--out file] | site <bundle-dir> --bundle-check [--json] | site <bundle-dir> --bundle-compare other-bundle-dir [--json] | site <bundle-dir> --bundle-handoff [--task id] [--json] | site <bundle-dir> --bundle-repair [--yes] [--json] [--out file] | site --init --name name --live-url url [--next-actions] [--out file] | site --init --name name --live-url url --bundle --out dir | site --from-intake file.md|--stdin [--json|--next-actions [--json]|--tasks|--bundle [--tasks] --out dir] [--out file] | site --intake-template [--language en|ko] [--json] [--out file] | site --sample [--out file] | site --prompt-list [--json]`, `examples [query|--route id] [--limit N] [--json]`, `search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider "cmd args"]] [--strict] [--json]`, `index [--build|--status|--verify] [--json] [--embeddings [--provider "cmd args"]]`, `show <file[:line]> [--lines N:M] [--context N] [--json]`, `audit [--strict] [--quiet] [--json]`, `doctor [--strict] [--json] [--fix]`, `mcp`, `version [--json]`, `help [command|--json]`.
79
79
 
80
80
  See [`docs/DISTRIBUTION.md`](docs/DISTRIBUTION.md) for the full distribution guide.
81
81
 
@@ -214,7 +214,7 @@ Refresh refs/ on demand: `./tools/extractors/run-all.sh`.
214
214
 
215
215
  ## Status
216
216
 
217
- See [`docs/ROADMAP.md`](docs/ROADMAP.md) for the full phase log and [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md) for the current completion boundary. Currently at **v4.59.0**: the read-only Agent SDK (`@design-ai/cli/sdk`), public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, and 90%+ component coverage are complete.
217
+ See [`docs/ROADMAP.md`](docs/ROADMAP.md) for the full phase log and [`docs/PRODUCT-READINESS.md`](docs/PRODUCT-READINESS.md) for the current completion boundary. Currently at **v4.60.0**: the Agent SDK (`@design-ai/cli/sdk`) with shipped TypeScript types and the opt-in `learn.*` local-write namespace, public npm publish, provenance-backed GitHub Actions release, public registry smoke, Website Console MCP readiness, workspace learning restore/eval coverage, handoff bundle verification, and 90%+ component coverage are complete.
218
218
 
219
219
  Core design consulting workflows are locally release-ready. The website improvement control tower ships as a zero-dependency static Web App plus a `website-improvement` route/skill/command, covering Site Profiles, audit checklists, MCP readiness, refactor prompts, handoff evidence tracking, and bundle export/verify/repair. Local learning preferences are available through `design-ai learn` — profile bootstrap, feedback capture, a read-only signals registry, skill-proposal generation from repeated QA signals, and full backup/restore/curate/audit tooling, all local-only and opt-in. AI model training or fine-tuning remains outside the shipped scope.
220
220
 
@@ -33,7 +33,7 @@ export const HELP_COMMANDS = [
33
33
  { topic: "uninstall", usage: "uninstall [--json]", description: "Remove symlinks (keeps source files)" },
34
34
  { topic: "status", usage: "status [--json]", description: "Show what's installed (use VERBOSE=1 for full list)" },
35
35
  { topic: "list", usage: "list [skills|commands|agents] [--json]", description: "List catalog from the plugin manifest" },
36
- { topic: "search", usage: "search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]", description: "Search the local markdown corpus" },
36
+ { topic: "search", usage: "search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--strict] [--json]", description: "Search the local markdown corpus and ranked-search eval checkpoints" },
37
37
  { topic: "index", usage: "index [--build|--status|--verify] [--json] [--embeddings [--provider \"cmd args\"]]", description: "Build, inspect, and verify the local retrieval index" },
38
38
  { topic: "show", usage: "show <file[:line]> [--lines N:M] [--context N] [--json]", description: "Print a corpus file or line range" },
39
39
  { topic: "route", usage: "route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N]", description: "Recommend commands, skills, knowledge, and route eval checkpoints" },
@@ -2,13 +2,19 @@
2
2
 
3
3
  import { DESIGN_AI_HOME } from "../lib/paths.mjs";
4
4
  import { dim, header, info, warn } from "../lib/log.mjs";
5
+ import { resolveBriefInput } from "../lib/brief.mjs";
5
6
  import { configuredEmbeddingProvider, defaultConfigFile } from "../lib/local-config.mjs";
7
+ import { buildSearchEvalTemplate, searchEvalReport } from "../lib/search-eval.mjs";
6
8
  import { embeddingRerankSearch, rankedSearchCorpus } from "../lib/search-ranked.mjs";
7
9
  import { formatSearchJson, parseSearchArgs, searchCorpus } from "../lib/search.mjs";
8
10
 
9
11
  function printHelp() {
10
- console.log("Usage: design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]\n");
11
- console.log("Searches markdown files across knowledge, examples, skills, docs, agents, and commands.\n");
12
+ console.log("Usage: design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]");
13
+ console.log(" design-ai search --eval-template [--json]");
14
+ console.log(" design-ai search --eval --from-file search-eval.json [--strict] [--json]");
15
+ console.log(" cat search-eval.json | design-ai search --eval --stdin [--strict] [--json]\n");
16
+ console.log("Searches markdown files across knowledge, examples, skills, docs, agents, and commands.");
17
+ console.log("Search eval is read-only and checks deterministic ranked-search fixtures.\n");
12
18
  console.log("Options:");
13
19
  console.log(" --limit N Maximum hits to return, 1-500. Default: 20");
14
20
  console.log(" --dir kind Restrict to one corpus directory. Repeatable.");
@@ -18,7 +24,18 @@ function printHelp() {
18
24
  console.log(" Requires --ranked and a configured provider; falls back to the");
19
25
  console.log(" lexical ranking (with a notice) if no provider or sidecar is available");
20
26
  console.log(" --provider \"cmd args\" Embedding provider command for this invocation (overrides config)");
27
+ console.log(" --from-file file Read ranked-search eval JSON from a file (with --eval)");
28
+ console.log(" --stdin Read ranked-search eval JSON from standard input (with --eval)");
29
+ console.log(" --eval-template Generate a runnable ranked-search eval checkpoint JSON template");
30
+ console.log(" --eval Run deterministic ranked-search checkpoint cases");
31
+ console.log(" --strict With --eval, exit non-zero on warning or failure");
21
32
  console.log(" --json Emit machine-readable results");
33
+ console.log("");
34
+ console.log("Examples:");
35
+ console.log(" design-ai search Pretendard");
36
+ console.log(" design-ai search \"button component\" --ranked");
37
+ console.log(" design-ai search --eval-template --json > search-eval.json");
38
+ console.log(" design-ai search --eval --from-file search-eval.json --strict --json");
22
39
  }
23
40
 
24
41
  function parseProviderFlag(value) {
@@ -45,6 +62,37 @@ function printRankedHits(query, hits, backend) {
45
62
  info(`Hits: ${hits.length}`);
46
63
  }
47
64
 
65
+ function printSearchEvalTemplate(template) {
66
+ header("design-ai search", "Ranked-search eval checkpoint template");
67
+ info(`Source: ${DESIGN_AI_HOME}`);
68
+ info(`Cases: ${template.cases.length}`);
69
+ console.log();
70
+ console.log("Write the JSON below to a file, edit cases if needed, then run:");
71
+ console.log("design-ai search --eval --from-file search-eval.json --strict");
72
+ console.log();
73
+ console.log(formatSearchJson(template));
74
+ }
75
+
76
+ function printSearchEvalReport(report) {
77
+ header("design-ai search", "Ranked-search eval report");
78
+ info(`Source: ${report.source}`);
79
+ info(`Status: ${report.status}`);
80
+ info(`Cases: ${report.summary.total} (${report.summary.pass} pass, ${report.summary.warn} warn, ${report.summary.fail} fail)`);
81
+ console.log();
82
+
83
+ for (const result of report.cases) {
84
+ const top = result.topRelPath || "none";
85
+ console.log(`${result.status.toUpperCase()} ${result.id}`);
86
+ console.log(` query: ${result.query}`);
87
+ console.log(` top: ${top} ${dim(`(hits ${result.hitCount})`)}`);
88
+ if (result.matchedTokens.length > 0) {
89
+ console.log(` matched: ${result.matchedTokens.join(", ")}`);
90
+ }
91
+ console.log(` result: ${result.message}`);
92
+ console.log();
93
+ }
94
+ }
95
+
48
96
  export async function runSearch(args) {
49
97
  const parsed = parseSearchArgs(args);
50
98
  if (parsed.help) {
@@ -52,6 +100,37 @@ export async function runSearch(args) {
52
100
  return;
53
101
  }
54
102
 
103
+ if (parsed.evalTemplate) {
104
+ const template = buildSearchEvalTemplate({ sourceRoot: DESIGN_AI_HOME });
105
+ if (parsed.json) {
106
+ console.log(formatSearchJson(template));
107
+ return;
108
+ }
109
+ printSearchEvalTemplate(template);
110
+ return;
111
+ }
112
+
113
+ if (parsed.eval) {
114
+ const evalText = resolveBriefInput(parsed);
115
+ const report = searchEvalReport({
116
+ evalText,
117
+ source: parsed.fromFile || "stdin",
118
+ sourceRoot: DESIGN_AI_HOME,
119
+ limit: parsed.limit,
120
+ });
121
+
122
+ if (parsed.json) {
123
+ console.log(formatSearchJson(report));
124
+ } else {
125
+ printSearchEvalReport(report);
126
+ }
127
+
128
+ if (parsed.strict && report.status !== "pass") {
129
+ process.exitCode = 1;
130
+ }
131
+ return;
132
+ }
133
+
55
134
  if (parsed.embeddings && !parsed.ranked) {
56
135
  throw new Error("--embeddings requires --ranked");
57
136
  }
@@ -0,0 +1,241 @@
1
+ // Ranked-search eval checkpoint generation and reports for `design-ai search`.
2
+ // Mirrors cli/lib/route.mjs's eval-checkpoint pattern (docs/AI-LEARNING-PHASE2.md
3
+ // FU-3): runs hand-authored / generated cases through the shipped deterministic
4
+ // ranked search path (rankedSearchCorpus, BM25-style lexical scorer) and reports
5
+ // pass/warn/fail. This is the standing evidence artifact for any future decision
6
+ // to promote `--ranked` to the default `search` behavior.
7
+
8
+ import { rankedSearchCorpus } from "./search-ranked.mjs";
9
+ import { DEFAULT_SEARCH_DIRS } from "./search.mjs";
10
+
11
+ export const SEARCH_EVAL_VERSION = 1;
12
+ // Default per-case result limit when a case does not specify its own `limit`.
13
+ // Kept small and eval-appropriate (route's default is 3; search corpus hits are
14
+ // noisier, so 10 gives enough headroom for `minHits` assertions without needing
15
+ // a large recall window).
16
+ export const SEARCH_EVAL_DEFAULT_LIMIT = 10;
17
+
18
+ function isoTimestamp(now = new Date()) {
19
+ return (now instanceof Date ? now : new Date(now)).toISOString();
20
+ }
21
+
22
+ export function buildSearchEvalTemplate({ sourceRoot, generatedAt = new Date() } = {}) {
23
+ return {
24
+ version: SEARCH_EVAL_VERSION,
25
+ generatedAt: isoTimestamp(generatedAt),
26
+ description: "Deterministic ranked-search checkpoints for design-ai's BM25-style lexical retrieval.",
27
+ cases: [
28
+ {
29
+ id: "korean-button-stem",
30
+ query: "버튼",
31
+ minHits: 1,
32
+ },
33
+ {
34
+ id: "korean-button-particle",
35
+ query: "버튼을",
36
+ minHits: 1,
37
+ },
38
+ {
39
+ id: "korean-accessibility-particle",
40
+ query: "접근성이",
41
+ minHits: 1,
42
+ },
43
+ {
44
+ id: "accessibility-english",
45
+ query: "accessibility",
46
+ minHits: 5,
47
+ matchedTokenIncludes: ["accessibility"],
48
+ },
49
+ {
50
+ id: "button-component-spec-english",
51
+ query: "button component spec",
52
+ minHits: 5,
53
+ matchedTokenIncludes: ["button", "component", "spec"],
54
+ },
55
+ {
56
+ id: "color-palette-english",
57
+ query: "color palette",
58
+ expectRelPathIn: ["commands/palette-from-brand.md"],
59
+ minHits: 5,
60
+ matchedTokenIncludes: ["color", "palette"],
61
+ },
62
+ ],
63
+ };
64
+ }
65
+
66
+ function searchEvalStatus(counts) {
67
+ if (counts.fail > 0) return "fail";
68
+ if (counts.warn > 0) return "warn";
69
+ return "pass";
70
+ }
71
+
72
+ export function normalizeSearchEvalPayload(evalText, source = "search-eval.json") {
73
+ let payload;
74
+ try {
75
+ payload = JSON.parse(evalText);
76
+ } catch (err) {
77
+ throw new Error(`Could not parse search eval JSON from ${source}: ${err.message}`);
78
+ }
79
+
80
+ if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
81
+ throw new Error("Search eval payload must be a JSON object");
82
+ }
83
+ if (payload.version !== SEARCH_EVAL_VERSION) {
84
+ throw new Error(`Search eval payload version must be ${SEARCH_EVAL_VERSION}`);
85
+ }
86
+ if (!Array.isArray(payload.cases)) {
87
+ throw new Error("Search eval payload must include a cases array");
88
+ }
89
+
90
+ return payload;
91
+ }
92
+
93
+ function normalizeStringList(value, { field, id }) {
94
+ if (value === undefined || value === null) return [];
95
+ if (!Array.isArray(value)) {
96
+ throw new Error(`Search eval case ${id} field ${field} must be an array`);
97
+ }
98
+ return value.map((item) => String(item || "").trim()).filter(Boolean);
99
+ }
100
+
101
+ export function normalizeSearchEvalCase(rawCase, index) {
102
+ if (!rawCase || typeof rawCase !== "object" || Array.isArray(rawCase)) {
103
+ throw new Error(`Search eval case ${index + 1} must be a JSON object`);
104
+ }
105
+
106
+ const id = String(rawCase.id || `case-${index + 1}`).trim();
107
+ if (!id) throw new Error(`Search eval case ${index + 1} is missing id`);
108
+
109
+ const query = String(rawCase.query || "").trim();
110
+ if (!query) throw new Error(`Search eval case ${id} is missing query`);
111
+
112
+ const dirs = normalizeStringList(rawCase.dirs, { field: "dirs", id });
113
+ for (const dir of dirs) {
114
+ if (!DEFAULT_SEARCH_DIRS.includes(dir)) {
115
+ throw new Error(`Search eval case ${id} field dirs has invalid value: ${dir}`);
116
+ }
117
+ }
118
+
119
+ const limit = rawCase.limit === undefined || rawCase.limit === null
120
+ ? SEARCH_EVAL_DEFAULT_LIMIT
121
+ : Number(rawCase.limit);
122
+ if (!Number.isInteger(limit) || limit < 1 || limit > 500) {
123
+ throw new Error(`Search eval case ${id} limit must be an integer from 1 to 500`);
124
+ }
125
+
126
+ const expectRelPathIn = normalizeStringList(rawCase.expectRelPathIn, { field: "expectRelPathIn", id });
127
+ const matchedTokenIncludes = normalizeStringList(rawCase.matchedTokenIncludes, { field: "matchedTokenIncludes", id })
128
+ .map((token) => token.toLowerCase());
129
+
130
+ let minHits;
131
+ if (rawCase.minHits === undefined || rawCase.minHits === null) {
132
+ minHits = undefined;
133
+ } else {
134
+ minHits = Number(rawCase.minHits);
135
+ if (!Number.isInteger(minHits) || minHits < 0) {
136
+ throw new Error(`Search eval case ${id} field minHits must be a non-negative integer`);
137
+ }
138
+ }
139
+
140
+ return {
141
+ id,
142
+ query,
143
+ dirs: dirs.length > 0 ? dirs : DEFAULT_SEARCH_DIRS,
144
+ limit,
145
+ expectRelPathIn,
146
+ minHits,
147
+ matchedTokenIncludes,
148
+ };
149
+ }
150
+
151
+ export function evaluateSearchEvalCase(testCase, sourceRoot, defaultLimit) {
152
+ const limit = testCase.limit || defaultLimit || SEARCH_EVAL_DEFAULT_LIMIT;
153
+ const { hits } = rankedSearchCorpus({
154
+ query: testCase.query,
155
+ designAiPath: sourceRoot,
156
+ dirs: testCase.dirs,
157
+ limit,
158
+ });
159
+ const topHit = hits[0] || null;
160
+ const topRelPath = topHit?.relPath || "";
161
+ const topMatchedTokens = (topHit?.matchedTokens || []).map((token) => token.toLowerCase());
162
+
163
+ const hasAssertions = testCase.expectRelPathIn.length > 0
164
+ || testCase.minHits !== undefined
165
+ || testCase.matchedTokenIncludes.length > 0;
166
+
167
+ let status = "pass";
168
+ let message = "All checkpoint expectations were satisfied.";
169
+
170
+ if (!hasAssertions) {
171
+ status = "warn";
172
+ message = "Case has no expectRelPathIn, minHits, or matchedTokenIncludes assertion.";
173
+ } else if (testCase.expectRelPathIn.length > 0 && !testCase.expectRelPathIn.includes(topRelPath)) {
174
+ status = "fail";
175
+ message = topRelPath
176
+ ? `Expected top hit in [${testCase.expectRelPathIn.join(", ")}], but top hit was ${topRelPath}.`
177
+ : `Expected top hit in [${testCase.expectRelPathIn.join(", ")}], but there were no hits.`;
178
+ } else if (testCase.minHits !== undefined && hits.length < testCase.minHits) {
179
+ status = "fail";
180
+ message = `Expected at least ${testCase.minHits} hit(s), got ${hits.length}.`;
181
+ } else if (
182
+ testCase.matchedTokenIncludes.length > 0
183
+ && !testCase.matchedTokenIncludes.every((token) => topMatchedTokens.includes(token))
184
+ ) {
185
+ status = "fail";
186
+ const missing = testCase.matchedTokenIncludes.filter((token) => !topMatchedTokens.includes(token));
187
+ message = topHit
188
+ ? `Top hit ${topRelPath} did not match expected token(s): ${missing.join(", ")}.`
189
+ : `Expected matched tokens [${missing.join(", ")}], but there were no hits.`;
190
+ }
191
+
192
+ return {
193
+ id: testCase.id,
194
+ status,
195
+ message,
196
+ query: testCase.query,
197
+ dirs: testCase.dirs,
198
+ limit,
199
+ topRelPath,
200
+ hitCount: hits.length,
201
+ matchedTokens: topHit?.matchedTokens || [],
202
+ expectRelPathIn: testCase.expectRelPathIn,
203
+ minHits: testCase.minHits ?? null,
204
+ matchedTokenIncludes: testCase.matchedTokenIncludes,
205
+ hits,
206
+ };
207
+ }
208
+
209
+ export function searchEvalReport({
210
+ evalText,
211
+ source = "search-eval.json",
212
+ sourceRoot,
213
+ limit = SEARCH_EVAL_DEFAULT_LIMIT,
214
+ generatedAt = new Date(),
215
+ }) {
216
+ const payload = normalizeSearchEvalPayload(evalText, source);
217
+ const normalizedCases = payload.cases.map((testCase, index) => normalizeSearchEvalCase(testCase, index));
218
+ const results = normalizedCases.map((testCase) => evaluateSearchEvalCase(testCase, sourceRoot, limit));
219
+ const counts = results.reduce(
220
+ (acc, result) => ({
221
+ ...acc,
222
+ [result.status]: acc[result.status] + 1,
223
+ }),
224
+ { pass: 0, warn: 0, fail: 0 },
225
+ );
226
+ const summary = {
227
+ total: results.length,
228
+ pass: counts.pass,
229
+ warn: counts.warn,
230
+ fail: counts.fail,
231
+ };
232
+
233
+ return {
234
+ evalVersion: payload.version,
235
+ source,
236
+ generatedAt: isoTimestamp(generatedAt),
237
+ status: searchEvalStatus(counts),
238
+ summary,
239
+ cases: results,
240
+ };
241
+ }
@@ -7,6 +7,7 @@ import {
7
7
  } from "node:fs";
8
8
  import path from "node:path";
9
9
 
10
+ import { parseBriefSourceFlag } from "./brief.mjs";
10
11
  import { expectedValueMessage, unknownOptionMessage } from "./suggest.mjs";
11
12
 
12
13
  export const DEFAULT_SEARCH_DIRS = [
@@ -20,7 +21,21 @@ export const DEFAULT_SEARCH_DIRS = [
20
21
 
21
22
  const PREVIEW_LEN = 120;
22
23
  const PREVIEW_BEFORE = 50;
23
- const SEARCH_OPTIONS = ["-h", "--help", "--json", "--limit", "--dir", "--ranked", "--embeddings", "--provider"];
24
+ const SEARCH_OPTIONS = [
25
+ "-h",
26
+ "--help",
27
+ "--json",
28
+ "--limit",
29
+ "--dir",
30
+ "--ranked",
31
+ "--embeddings",
32
+ "--provider",
33
+ "--eval-template",
34
+ "--eval",
35
+ "--strict",
36
+ "--from-file",
37
+ "--stdin",
38
+ ];
24
39
 
25
40
  function exists(p) {
26
41
  try {
@@ -118,11 +133,17 @@ export function parseSearchArgs(args) {
118
133
  ranked: false,
119
134
  embeddings: false,
120
135
  provider: "",
136
+ evalTemplate: false,
137
+ eval: false,
138
+ strict: false,
139
+ fromFile: "",
140
+ stdin: false,
121
141
  help: false,
122
142
  };
123
143
 
124
144
  for (let i = 0; i < args.length; i += 1) {
125
145
  const arg = args[i];
146
+ out.index = i;
126
147
  if (arg === "-h" || arg === "--help") {
127
148
  out.help = true;
128
149
  } else if (arg === "--json") {
@@ -131,9 +152,17 @@ export function parseSearchArgs(args) {
131
152
  out.ranked = true;
132
153
  } else if (arg === "--embeddings") {
133
154
  out.embeddings = true;
155
+ } else if (arg === "--eval-template") {
156
+ out.evalTemplate = true;
157
+ } else if (arg === "--eval") {
158
+ out.eval = true;
159
+ } else if (arg === "--strict") {
160
+ out.strict = true;
134
161
  } else if (arg === "--provider") {
135
162
  out.provider = args[i + 1] || "";
136
163
  i += 1;
164
+ } else if (parseBriefSourceFlag(args, out)) {
165
+ i = out.index;
137
166
  } else if (arg === "--limit") {
138
167
  const next = args[i + 1];
139
168
  const limit = Number(next);
@@ -156,8 +185,34 @@ export function parseSearchArgs(args) {
156
185
  }
157
186
  }
158
187
 
188
+ if (out.eval && out.evalTemplate) {
189
+ throw new Error("Choose either --eval-template or --eval, not both");
190
+ }
191
+ if (out.strict && !out.eval) {
192
+ throw new Error("--strict can only be used with --eval");
193
+ }
194
+ if (
195
+ out.evalTemplate
196
+ && (out.fromFile || out.stdin || out.queryParts.length > 0 || out.ranked || out.embeddings || out.provider || out.dirs.length > 0)
197
+ ) {
198
+ throw new Error("--eval-template cannot be combined with a query, --from-file, --stdin, --ranked, --embeddings, --provider, or --dir");
199
+ }
200
+ if (out.eval && (!out.fromFile && !out.stdin)) {
201
+ throw new Error("--eval requires --from-file or --stdin");
202
+ }
203
+ if ((out.fromFile || out.stdin) && !out.eval) {
204
+ throw new Error("--from-file and --stdin require --eval");
205
+ }
206
+ if (
207
+ out.eval
208
+ && (out.queryParts.length > 0 || out.ranked || out.embeddings || out.provider || out.dirs.length > 0)
209
+ ) {
210
+ throw new Error("--eval cannot be combined with an inline query, --ranked, --embeddings, --provider, or --dir");
211
+ }
212
+
159
213
  return {
160
214
  ...out,
215
+ index: undefined,
161
216
  query: out.queryParts.join(" ").trim(),
162
217
  dirs: out.dirs.length > 0 ? out.dirs : DEFAULT_SEARCH_DIRS,
163
218
  };
@@ -202,8 +202,8 @@ This review answers the five open questions against the shipped Phase A implemen
202
202
 
203
203
  - **FU-1 (Q4, before Phase B):** Add Hangul-aware tokenization (CJK bigramming or particle stripping) in `cli/lib/lexical.mjs` behind Korean `learn --eval` checkpoints; regression-test that `버튼`, `버튼을`, `버튼이` converge on the same button docs. _Done (2026-07-03): Hangul runs >= 2 chars now emit overlapping character bigrams alongside the surface form; the review's zero-hit queries recover (`버튼` 0 → 3 ranked hits, `접근성이` 0 → 3), with unit regression coverage in `lexical.test.mjs`._
204
204
  - **FU-2 (Q2, before Phase B index-as-source-of-truth):** Key the corpus index by corpus digest / record `designAiPath` in the payload so multiple checkouts do not overwrite each other once the index is read for content rather than staleness. _Done (2026-07-03): sidecar format bumped to version 2 (auto-invalidating v1 files); the corpus payload records resolved `designAiPath` and the learning payload the resolved `learningFile`, and `index --status` reports `sourceMatch` and treats identity mismatch as not fresh._
205
- - **FU-3 (Q1, gates default promotion):** Land a ranked-search eval checkpoint so any future `--ranked` default promotion is evidence-backed and announced.
206
- - **FU-4 (Q5, Phase B):** Specify and implement `~/.design-ai/config.json` as the Phase B provider config home with per-invocation `--embeddings` still required.
205
+ - **FU-3 (Q1, gates default promotion):** Land a ranked-search eval checkpoint so any future `--ranked` default promotion is evidence-backed and announced. _Done (2026-07-05): `design-ai search --eval-template`/`--eval [--strict]` landed in `cli/lib/search-eval.mjs` and `cli/commands/search.mjs`, mirroring the route/prompt/pack/learn eval-checkpoint pattern; cases run through `rankedSearchCorpus` (the shipped ranked path), and the default template includes the three Korean particle-form regression cases from FU-1 (버튼/버튼을/접근성이) plus English cases, doubling as a retrieval-level Hangul-bigram regression check._
206
+ - **FU-4 (Q5, Phase B):** Specify and implement `~/.design-ai/config.json` as the Phase B provider config home with per-invocation `--embeddings` still required. _Done (2026-07-03, Phase B): `cli/lib/local-config.mjs` reads `~/.design-ai/config.json` (or `DESIGN_AI_CONFIG_FILE`); the config only supplies the provider — every invocation still requires the explicit `--embeddings` flag to arm it._
207
207
 
208
208
  ### Phase B gate: cleared-with-conditions
209
209
 
@@ -118,8 +118,17 @@ Cross-check any surface work against the release gate: `npm run audit` (8/8, lin
118
118
 
119
119
  ## Decision record
120
120
 
121
- - **Status:** proposedawaiting maintainer sign-off
122
- - **Date:** 2026-07-03
121
+ - **Status:** accepted and executed both recommended surfaces shipped
122
+ - **Date:** 2026-07-03 (proposed) / 2026-07-05 (executed)
123
123
  - **Deciders:** maintainer
124
124
  - **Supersedes:** none
125
125
  - **Related:** [`PRODUCT-READINESS.md`](PRODUCT-READINESS.md), [`external-status.md`](external-status.md), `AI-LEARNING-PHASE2.md`
126
+
127
+ ### Outcome (2026-07-05)
128
+
129
+ The recommendation was executed in order and both surfaces are live:
130
+
131
+ - **CLI deepening** shipped as v4.57.0 (local retrieval memory: `index`, `search --ranked`, `--with-recall`, `learn --recall`, opt-in embedding rerank) and v4.58.0 (`route --explain` related knowledge, recall quality, corpus depth). The dogfood friction this document identified — missing corpus and missing flags — is retired; retrieval spans every CLI/MCP surface.
132
+ - **Agent SDK** followed exactly on the stated revisit trigger (recall interface stable across two releases): design doc [`AGENT-SDK.md`](AGENT-SDK.md), then v4.59.0 (Phase A — read-only `@design-ai/cli/sdk`, 8 verbs) and v4.60.0 (TypeScript declarations + the opt-in `learn.*` local-write namespace).
133
+
134
+ The deferred surfaces (VS Code deepening, Web UI, Figma plugin) remain deferred with the same revisit triggers. The next-surface question is open again; re-evaluate against the criteria above when the next planning window opens, factoring in that the SDK now also serves programmatic adopters.
package/docs/ROADMAP.md CHANGED
@@ -1,5 +1,42 @@
1
1
  # Roadmap
2
2
 
3
+ ## Phase 763 — Ranked-search Eval Checkpoint Release (v4.61.0) ✓ ready
4
+
5
+ Ships the FU-3 ranked-search eval checkpoint (Phase 762) as an npm version: `search --eval-template` / `search --eval [--strict]` over the shipped BM25 path, with the Korean particle-form regression cases in the default template. Additive and backward-compatible; default `search` behavior is unchanged. With this release the AI-learning Phase A follow-ups (FU-1–FU-4) are all closed and published.
6
+
7
+ ### Verified
8
+ - All 8 audits passed.
9
+ - `npm run release:check` (unit tests incl. search-eval and search-command suites, strict audits, whitespace, package contents, release metadata, release self-tests, packed-tarball smoke).
10
+ - `npm run release:metadata`.
11
+ - `git diff --check`.
12
+ - Main-branch GitHub Actions (`Design-AI audit`, `Deploy doc site`) passed for the constituent commits.
13
+
14
+ ### Versions
15
+ - `package.json` + `.claude-plugin/plugin.json`: 4.60.0 → 4.61.0.
16
+ - `vscode-extension/package.json`: remains 0.4.1.
17
+
18
+ ### What this enables
19
+ - The `--ranked` default-promotion decision becomes evidence-backed for npm users too: the published CLI carries the checkpoint tooling, so the eval can be re-run against any installed version — including Korean agglutinative-query regressions — before and after such a change.
20
+
21
+ ### What's still ahead
22
+ - The `--ranked` default-promotion decision itself (separate, evidence-backed call).
23
+ - Next-surface re-evaluation per the updated decision record in [NEXT-SURFACE-DECISION.md](NEXT-SURFACE-DECISION.md).
24
+
25
+ ## Phase 762 — Ranked-search eval checkpoint (FU-3, implemented, unreleased)
26
+
27
+ Closes the last open follow-up from the AI-learning Phase A review ([AI-LEARNING-PHASE2.md](AI-LEARNING-PHASE2.md), FU-3): `design-ai search` gains `--eval-template` / `--eval [--strict]` checkpoint modes mirroring the route/prompt/pack/learn eval pattern, run through the shipped ranked (BM25) search path. This is the standing evidence artifact for any future decision to promote `--ranked` to the default; with FU-1/FU-2/FU-4 already annotated done, all four Phase A follow-ups are now closed.
28
+
29
+ ### Delivered
30
+ - [x] `cli/lib/search-eval.mjs` — `SEARCH_EVAL_VERSION = 1`, `buildSearchEvalTemplate()`, `searchEvalReport()` over `rankedSearchCorpus`; route-style payload normalization and error messages. Case schema: `id`, `query`, optional `dirs`/`limit`, assertions `expectRelPathIn` / `minHits` / `matchedTokenIncludes`.
31
+ - [x] Default template includes the FU-1 Korean particle-form regression cases (`버튼`, `버튼을`, `접근성이`) plus three English cases — so the eval doubles as the retrieval-level Hangul-bigram regression. All six verified passing against the shipped corpus (`--strict` exit 0).
32
+ - [x] CLI wiring in `cli/lib/search.mjs` / `cli/commands/search.mjs` (`--eval-template`, `--eval`, `--strict`, `--from-file`, `--stdin`, mutual-exclusivity validation), help topic usage line, and the shared `smoke_assertions.py` pins (4 spots — shared constants, so both package and registry smokes stay aligned).
33
+ - [x] Tests: `search-eval.test.mjs` (template, pass/warn/fail reports, malformed payloads) and `search-command.test.mjs` (CLI-level, strict exit semantics). 578/578 total.
34
+ - [x] `docs/AI-LEARNING-PHASE2.md` FU-3 and FU-4 done annotations.
35
+
36
+ ### Remaining
37
+ - [ ] Release: bundle into the next npm version.
38
+ - [ ] The `--ranked` default-promotion decision itself — a future, separate call that this checkpoint now makes evidence-backed.
39
+
3
40
  ## Phase 761 — Agent SDK types + Phase B Release (v4.60.0) ✓ ready
4
41
 
5
42
  Ships the Agent SDK TypeScript declarations (Phase 759) and the `learn.*` local-write namespace (Phase 760) together as one npm version. Both are additive and backward-compatible: the eight Phase A verbs stay read-only and unchanged, and TypeScript types resolve automatically for `@design-ai/cli/sdk` with no build step on either side.
@@ -1,22 +1,22 @@
1
1
  # External Publication Status
2
2
 
3
- > Checked: 2026-07-04
3
+ > Checked: 2026-07-05
4
4
  > Scope: npm registry, GitHub Pages, Homebrew tap, VS Code Marketplace, Claude/Codex MCP
5
5
 
6
6
  ## Summary
7
7
 
8
- npm is publicly published at `@design-ai/cli@4.59.0` (npm dist-tag `latest` = `4.59.0`); publish run from tag `v4.59.0` succeeded with provenance and all pre-publish gates green. This release adds the read-only Agent SDK surface (`@design-ai/cli/sdk`, eight adapter verbs) additive only, so CLI output is unchanged. The live `npm run registry:smoke` passes cleanly against published `@design-ai/cli@4.59.0` ("Registry smoke passed"), covering the retrieval surfaces (index/ranked/embeddings/recall) and route enrichment. GitHub Release `v4.59.0` is published, and the Homebrew tap formula points at the `v4.59.0` release source tarball with a verified SHA-256. The VS Code extension `sungjin.design-ai-vscode` remains published at `0.4.1`. GitHub Pages docs are publicly reachable.
8
+ npm is publicly published at `@design-ai/cli@4.60.0` (npm dist-tag `latest` = `4.60.0`); publish run from tag `v4.60.0` succeeded with provenance and all pre-publish gates green. This release extends the Agent SDK (`@design-ai/cli/sdk`) with shipped TypeScript declarations (via an `exports` `types` condition) and the opt-in `learn.*` local-write namespace; the eight read-only verbs are unchanged, so CLI output is unchanged. The live `npm run registry:smoke` passes cleanly against published `@design-ai/cli@4.60.0` ("Registry smoke passed"), covering the retrieval surfaces (index/ranked/embeddings/recall) and route enrichment. GitHub Release `v4.60.0` is published, and the Homebrew tap formula points at the `v4.60.0` release source tarball with a verified SHA-256. The VS Code extension `sungjin.design-ai-vscode` remains published at `0.4.1`. GitHub Pages docs are publicly reachable.
9
9
 
10
10
  ## Results
11
11
 
12
12
  | Surface | Checked target | Result | Evidence |
13
13
  |---|---|---|---|
14
- | npm registry | `@design-ai/cli` | Published latest is `4.59.0` (tag `v4.59.0`, provenance). Pre-publish packed-tarball smoke (incl. SDK import) and live `npm run registry:smoke` both pass for `@design-ai/cli@4.59.0`. | `npm view @design-ai/cli version` → `4.59.0`; live registry smoke "Registry smoke passed" |
14
+ | npm registry | `@design-ai/cli` | Published latest is `4.60.0` (tag `v4.60.0`, provenance). Pre-publish packed-tarball smoke (incl. SDK import + `learn.remember` write) and live `npm run registry:smoke` both pass for `@design-ai/cli@4.60.0`. | `npm view @design-ai/cli version` → `4.60.0`; live registry smoke "Registry smoke passed" |
15
15
  | GitHub Pages | `https://sungjin9288.github.io/design-ai/` | Published and reachable: HTTP `200`, design-ai MkDocs page rendered | `evidence/cli-logs/github-pages-status.log` |
16
- | Homebrew tap | `Formula/design-ai.rb` | Formula pinned to `v4.59.0` release source tarball with SHA-256 `e711203fb96fee5f6e7f25d809bc42108553bc8108ddf3e8cd181f0a76f6e0f5` (recomputed from the published tag tarball) | `Formula/design-ai.rb` |
16
+ | Homebrew tap | `Formula/design-ai.rb` | Formula pinned to `v4.60.0` release source tarball with SHA-256 `fa34af51c32d8e59dba4c1d293746688820e85ae03efa3b0bae4f3b3b2a34583` (recomputed from the published tag tarball) | `Formula/design-ai.rb` |
17
17
  | VS Code Marketplace | `sungjin.design-ai-vscode` | Published: run `28431571256` published `v0.4.1`, and the Marketplace Gallery API returned visible version `0.4.1` on 2026-07-02. | `evidence/cli-logs/vscode-marketplace-status.log`, `evidence/cli-logs/vscode-marketplace-secret-status.log`, `evidence/cli-logs/vscode-extension-vsce-package.log`, `evidence/cli-logs/vscode-publish-workflow-status.log` |
18
- | GitHub Release | `v4.59.0` | Published for tag `v4.59.0` at commit `9f53345` | `gh release view v4.59.0` |
19
- | MCP server | `@design-ai/cli@4.59.0` / local clone | Public npm `design-ai-mcp` responds to initialize and tools/list with 10 tools; local Codex and Claude Code both report `design-ai` MCP as configured and connected. | `evidence/cli-logs/npm-registry-smoke.log`, `evidence/cli-logs/design-ai-mcp-client-status.log` |
18
+ | GitHub Release | `v4.60.0` | Published for tag `v4.60.0` at commit `2d73c42` | `gh release view v4.60.0` |
19
+ | MCP server | `@design-ai/cli@4.60.0` / local clone | Public npm `design-ai-mcp` responds to initialize and tools/list with 10 tools; local Codex and Claude Code both report `design-ai` MCP as configured and connected. | `evidence/cli-logs/npm-registry-smoke.log`, `evidence/cli-logs/design-ai-mcp-client-status.log` |
20
20
 
21
21
  ## Interpretation
22
22
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@design-ai/cli",
3
- "version": "4.60.0",
3
+ "version": "4.61.0",
4
4
  "description": "Senior product designer for any AI coding agent. Installs design-ai (20 skills, 17 commands, 4 agents) into Claude Code globally. Korean market depth plus website improvement control tower.",
5
5
  "bin": {
6
6
  "design-ai": "cli/bin/design-ai.mjs",
@@ -160,7 +160,7 @@ EXPECTED_HELP_TOPIC_USAGES = {
160
160
  "uninstall": "design-ai uninstall [--json]",
161
161
  "status": "design-ai status [--json]",
162
162
  "list": "design-ai list [skills|commands|agents] [--json]",
163
- "search": "design-ai search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]",
163
+ "search": "design-ai search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--strict] [--json]",
164
164
  "index": "design-ai index [--build|--status|--verify] [--json] [--embeddings [--provider \"cmd args\"]]",
165
165
  "show": "design-ai show <file[:line]> [--lines N:M] [--context N] [--json]",
166
166
  "route": "design-ai route <brief|--from-file file|--stdin|--list|--eval-template|--eval> [--limit N]",
@@ -184,7 +184,12 @@ EXPECTED_HELP_TOPIC_FRAGMENTS = {
184
184
  "uninstall": ("Usage:", "design-ai uninstall [--json]"),
185
185
  "status": ("Usage:", "design-ai status [--json]"),
186
186
  "list": ("Usage:", "design-ai list [skills|commands|agents]"),
187
- "search": ("Usage:", "design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]"),
187
+ "search": (
188
+ "Usage:",
189
+ "design-ai search <query> [--limit N] [--dir kind] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]",
190
+ "design-ai search --eval-template [--json]",
191
+ "design-ai search --eval --from-file search-eval.json [--strict] [--json]",
192
+ ),
188
193
  "index": ("Usage:", "design-ai index [--build|--status|--verify] [--json] [--embeddings [--provider \"cmd args\"]]"),
189
194
  "show": ("Usage:", "design-ai show <file[:line|start-end]> [--lines N:M] [--context N] [--json]"),
190
195
  "route": ("Usage:", "design-ai route <brief>", "design-ai route --list [--json]", "design-ai route --eval-template [--json]"),
@@ -364,7 +369,7 @@ EXPECTED_MAIN_HELP_FRAGMENTS = (
364
369
  "Usage: design-ai <command> [args]",
365
370
  "design-ai help [command|--json]",
366
371
  "install",
367
- "search <query>",
372
+ "search <query|--eval-template|--eval>",
368
373
  "show <file[:line]>",
369
374
  "route <brief|--from-file file|--stdin|--list|--eval-template|--eval>",
370
375
  "improve homepage conversion and SEO",
@@ -4669,8 +4674,8 @@ def passing_main_help_output() -> str:
4669
4674
  " design-ai help [command|--json]",
4670
4675
  "",
4671
4676
  " install Symlink design-ai into Claude Code (~/.claude)",
4672
- " search <query> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--json]",
4673
- " Search the local markdown corpus",
4677
+ " search <query|--eval-template|--eval> [--dir kind] [--limit N] [--ranked] [--embeddings [--provider \"cmd args\"]] [--strict] [--json]",
4678
+ " Search the local markdown corpus and ranked-search eval checkpoints",
4674
4679
  " index [--build|--status|--verify] [--json] [--embeddings [--provider \"cmd args\"]]",
4675
4680
  " Build, inspect, and verify the local retrieval index",
4676
4681
  " show <file[:line]> [--lines N:M] [--context N] [--json] Print a corpus file or line range",