@ait-co/devtools 0.1.39 → 0.1.41

package/README.en.md

@@ -777,6 +777,30 @@ Please file an issue: https://github.com/apps-in-toss-community/devtools/issues
 2. Updates to the latest version and runs the type check
 3. On detecting a new version, automatically opens a GitHub Issue (including whether there are type errors)
 
+## Fidelity QA
+
+`scripts/fidelity-qa/` automatically measures SDK API fidelity between the mock and a real-device relay session.
+
+```bash
+pnpm qa:fidelity --runner=mock           # mock-only (CI default, regression detection)
+pnpm qa:fidelity --runner=relay          # requires attached device (devtools MCP)
+pnpm qa:fidelity --runner=both --diff    # run both + print diff
+pnpm qa:fidelity --include-writes        # include Storage write cycle (off by default)
+pnpm qa:fidelity --output=results.json  # write JSON results to file
+```
+
+CI runs `pnpm qa:fidelity --runner=mock` automatically (exits 0 on a clean state).
+
+**Diff labels**:
+
+- `MATCH` — mock and relay values are equal
+- `EXPECTED_MISMATCH` — known difference registered in `scripts/fidelity-qa/whitelist.json` (e.g. jsdom UA vs real WebView UA)
+- `UNEXPECTED` — mismatch not in whitelist → exits 1 (potential regression)
+
+**Updating the whitelist**: when an intentional difference is found during a relay session, add `{ "id": "<probe-id>", "reason": "<explanation>" }` to `scripts/fidelity-qa/whitelist.json`.
+
+The relay runner is currently a stub (CDP Runtime.evaluate implementation is a follow-up in devtools#261).
+
 ## Contributing
 
 ### Adding a new API mock
@@ -847,6 +871,8 @@ A local browser (env 1) and a phone Toss WebView (env 2/3) both speak CDP, so ev
 
 ### Debug mode (CDP via Chii)
 
+For a step-by-step walkthrough of the on-device relay debug loop (dogfood build → QR scan → relay attach) including common failure recovery, see **[`docs/dogfood-relay-loop.md`](./docs/dogfood-relay-loop.md)** (Korean). For crash triage — `list_pages.crashDetectedAt`, iOS Console.app `.ips` analysis, and the redact procedure — see **[`docs/crash-triage.md`](./docs/crash-triage.md)** (Korean).
+
 Read-only tools only. Tools are registered in two tiers based on attach state — before attach, only the bootstrap tools (`build_attach_url`, `list_pages`) are visible; once a relay/local page attaches, the attach-dependent tools are registered dynamically in the same session via `notifications/tools/list_changed` (no session restart needed). The phone attach roundtrip is fully wired; all that remains is a single on-device acceptance run. The tool layer is CI-verified via a mockable injectable CDP connection / AIT source.
 
 Running `devtools-mcp` as a stdio server starts a local Chii relay on an OS-assigned port and opens a cloudflared quick tunnel, printing a public `wss://*.trycloudflare.com` URL and a QR code in the terminal (secrets/auth codes are never printed). When the phone enters the dogfood entry point, the in-app attach UI connects to the relay with that URL, and the agent reads console/network/page state via `chrome-devtools-mcp`-compatible tools — diagnosing regressions without anyone watching the phone.

package/README.md

@@ -788,6 +788,30 @@ Please file an issue: https://github.com/apps-in-toss-community/devtools/issues
 2. 최신 버전으로 업데이트 후 타입 체크 실행
 3. 새 버전 감지 시 자동으로 GitHub Issue 생성 (타입 에러 여부 포함)
 
+## Fidelity QA
+
+`scripts/fidelity-qa/`는 mock SDK와 실기기 relay 사이의 **SDK API 정합성을 자동으로 측정**하는 도구다.
+
+```bash
+pnpm qa:fidelity --runner=mock           # mock-only (CI 기본값, 회귀 감지)
+pnpm qa:fidelity --runner=relay          # 실기기 relay 필요 (devtools MCP 연결)
+pnpm qa:fidelity --runner=both --diff    # mock+relay 동시 실행 + diff 출력
+pnpm qa:fidelity --include-writes        # Storage write 사이클 포함 (기본 OFF)
+pnpm qa:fidelity --output=results.json  # JSON 결과 파일 저장
+```
+
+CI는 `pnpm qa:fidelity --runner=mock`을 자동 실행한다 (mock-only, exit 0이면 통과).
+
+**Diff 레이블**:
+
+- `MATCH` — mock과 relay 값이 동일
+- `EXPECTED_MISMATCH` — `scripts/fidelity-qa/whitelist.json`에 등록된 알려진 차이 (예: jsdom UA vs 실 WebView UA)
+- `UNEXPECTED` — whitelist에 없는 불일치 → exit 1 (회귀 의심)
+
+**whitelist 갱신 절차**: relay 세션에서 의도적 차이가 발견되면 `scripts/fidelity-qa/whitelist.json`에 `{ "id": "<probe-id>", "reason": "<설명>" }`을 추가한다.
+
+relay runner는 현재 stub (devtools#261 follow-up에서 CDP Runtime.evaluate 구현 예정).
+
 ## Contributing
 
 ### 새 API mock 추가 절차
@@ -859,6 +883,8 @@ AI 코딩 에이전트(Claude Code, Cursor 등)가 [MCP(Model Context Protocol)]
 
 ### Debug 모드 (CDP via Chii)
 
+실기기 relay 디버깅 루프(dogfood 빌드 → QR 스캔 → relay attach)의 단계별 절차와 복구 방법은 **[`docs/dogfood-relay-loop.md`](./docs/dogfood-relay-loop.md)** 를 참고하세요. crash가 발생한 경우 — `list_pages.crashDetectedAt`, iOS Console.app `.ips` 분석, redact 절차를 포함한 원인 추적 절차는 **[`docs/crash-triage.md`](./docs/crash-triage.md)** 를 참고하세요.
+
 read-only tool만 노출합니다. 도구는 attach 상태에 따라 2단계로 등록됩니다 — attach 전에는 bootstrap
 도구(`build_attach_url`·`list_pages`)만 보이고, 릴레이/로컬 페이지가 attach되면 `notifications/tools/list_changed`로
 attach 의존 도구가 같은 세션에서 동적 등록됩니다(세션 재시작 불필요). 폰 attach 라운드트립은 fully wired