kenaz 0.0.0-stage → 0.3.2

package/core/dev/jev-eval/CONTEXT_REVIEW.md

@@ -0,0 +1,31 @@
+# Historical context review
+
+This stage restores observed conversational evidence for the local pending corpus. It does not assign correction labels or infer a task goal. No source text is sent to Jev or another external model.
+
+## Review order
+
+1. Confirm the record is an actual user interaction rather than a smoke test, generated prompt, pasted transcript or unrelated task. A session cwd proves workspace association, not authorship or topic.
+2. Read the target message and only the observed preceding context. Check source lineage, missing/truncated/redacted context and compaction boundaries. When context is insufficient, keep the decision unresolved.
+3. Identify the exact prior assertion or plan that the user changes. Quote the local evidence in the review, distinguishing an observed statement from an annotator inference.
+4. A correction must change the previous understanding or direction; a question, status request, new requirement, agreement or quoted correction is not automatically positive. See ANNOTATION_GUIDE.md.
+5. Keep decisions blank until reviewed. Model/agent proposals must remain agent_proposed. A context restoration status is never human_verified and does not make a case scoring-ready.
+
+## Provenance limits
+
+Original pending manifests contain source-relative paths and line numbers, but no historic source snapshot hash. Target identity/text checks and newly computed prefix hashes bind this review to the source as read now; they cannot prove all preceding text is unchanged since the earlier export. Context reconstruction is bounded and intentionally omits tool output, internal reasoning, secrets and unsupported branches. Session and family grouping need review before any held-out split.
+
+Private review artifacts stay in the ignored `.kenaz/jev-corpus/` directory. Public reports contain aggregate counts only. The viewer is a read-only aid; opening it does not submit or save a review. Labels, reviewer identity and approval remain separate, explicit review records.
+
+## Verified result ? 2026-09-20
+
+All 458 candidate IDs retained: 419 partial context packets, 23 missing context, 16 rejected for ambiguous source identity. No case is marked fully restored or human-reviewed. The 16 rejections are in one Claude file with repeated attachment UUIDs whose parent links differ; duplicate display metadata alone is tolerated.
+
+54/54 tests passed. Parent independently verified 442 source-prefix hashes and 2,352 exported context items against original visible text, source line order and filtering rules. All 458 HTML articles are escaped, without scripts, iframes, images or external stylesheets. Source pending file remains unchanged. Aggregate evidence: [context report](reports/context-review-2026-09-20.json).
+
+Current private artifact: `.kenaz/jev-corpus/context-2026-09-20-v2/review.html`; matching `review-context.jsonl` holds blank decisions. First export `context-2026-09-20` is REJECTED: a JSON-escaped quoted credential escaped a precheck; raw-text, normalized-text and final-output checks now prevent that known pattern, with a regression test. This does not guarantee removal of every possible secret or personal detail.
+
+```powershell
+node dev/jev-eval/context.mjs --pending .kenaz/jev-corpus/history-2026-09-20-v2/pending.jsonl --manifest .kenaz/jev-corpus/history-2026-09-20-v2/manifest.json --out-dir .kenaz/jev-corpus/context-new
+```
+
+The next stage is evidence-based candidate triage and annotation. Cases lacking the relevant prior assumption remain unresolved. No accuracy claim, held-out split, real-data external upload or production activation has occurred.

package/core/dev/jev-eval/EVALUATION_STATUS.md

@@ -0,0 +1,39 @@
+# Jev evaluation status ? 2026-09-20
+
+**Conclusion: promising synthetic latency and engineering feasibility; real-workflow quality and replacement value remain unproven.** A separate, default-off production correction adapter and Dashboard switch are now implemented locally at the user's request. The research harness itself is not imported into production. See [active correction experiment](ACTIVE_CORRECTION.md) for behavior, verification, and release limits.
+
+| Evidence | Observed result | Interpretation limit |
+| --- | --- | --- |
+| Live Jev smoke | Two retained runs, 11/11 successful each; p50 272.66ms and 271.89ms | Same 11 synthetic cases; middle run excluded after output truncation; cache state unknown |
+| Synthetic correction | 8/8 matching agent-proposed labels per run | Repeated cases, not 16 independent samples; not human-reviewed accuracy |
+| Synthetic reranking | NDCG@5 = 1 on two evaluable queries | Third zero-relevance query excluded; far below 100-query target |
+| Synthetic length probe | 9/9 succeeded at 800ms deadline; maximum observed 715ms | Repetitive filler, three calls per length; no robust p99 or concurrency evidence |
+| Engineering checks | Latest 54/54 tests passed | Research harness and context tests, not installed-application acceptance |
+| Historical corpus | 458 unique candidates, 68 session IDs; 42 short of 500 | User-role history is not proof of human authorship or representativeness |
+| Restored context | 419 partial, 23 missing, 16 rejected | Context availability is not annotation or scoring readiness |
+| Annotation pilot | 12 cases: 9 matching non-null agent proposals, 1 shared null, 2 disagreements | Same model family, no independent human review; not a Jev result |
+| Haiku baseline | No successful batch in recorded run; CLI weekly quota blocked it | No fair quality or speedup comparison available |
+
+## Thalamus local integration
+
+A separate default-off Thalamus delivery-ranking experiment is implemented locally. It affects native Claude Active Sensing order and MCP calls that provide taskContext; it does not change neural dynamics or claim automatic Bifrost integration. See [Thalamus ranking](THALAMUS_RANKING.md) for tests and release limits. These engineering checks do not establish real Jev relevance improvement.
+
+## New local annotation findings
+
+Two separately produced agent reviews covered 12 short-context cases from distinct inferred groups. Selection was deterministic by ID, with four cases in each of three target-length bins and a maximum of 1,800 characters of target plus observed context. This convenience pilot is neither representative nor held out.
+
+Nine non-null proposals matched: two correction and seven non-correction. One case was unresolved by both reviewers; two had null versus false disagreements. All three remain unresolved in the comparison artifact. Parent verified all 12 IDs in each review and all 49 exact evidence substrings. Both reviewers are agents from the same model family: agreement does not establish truth. Human-verified labels remain zero, and no Jev predictions were made on these real cases.
+
+The difficult boundaries were: whether a session restart addresses a suggested MCP-server restart; whether a question about retiring an older task changes a prior plan absent from context; and whether a new downstream error contradicts a narrowly scoped successful repair. These require precise premise and scope definitions. A question mark or a negative word in an error log cannot decide the label by itself.
+
+Private pilot artifacts remain under `.kenaz/jev-corpus/pilot-2026-09-20/`. Public aggregate: [annotation pilot](reports/annotation-pilot-2026-09-20.json). The nine agreements are still `agent_proposed`, and the comparison explicitly sets scoring eligibility false. No original corpus labels or splits were changed.
+
+## Product implication
+
+Continue bounded research on correction detection and recall ranking. The user explicitly authorized an optional active experiment: OFF keeps existing behavior; ON lets validated Jev responses add/remove correction and prioritize positive correction for existing primary-type consumers. This opt-in decision does not establish quality. The actual Muninn/Haiku routine still translates and emits seven signal labels; Jev has a smaller output contract. A fair comparison must account for those downstream consumers rather than comparing latency alone.
+
+The separate active adapter now has 16 passing runtime tests, including downstream user-profile effects and OFF/failure behavior; the unchanged research suite passes 54 tests. Dashboard controller/SSR tests pass 13 cases and Rust settings tests pass 8 cases. These are engineering checks, not quality scores or installed-build acceptance. Users without their own Jev key keep the existing workflow. No shared developer key is stored or shipped.
+
+Next evidence needed: recover the material premises for unresolved cases and freeze the annotation rule; expand context-grounded agent proposals without treating them as gold; independent human adjudication; fixed model/question/threshold and lineage split; equivalent Haiku/Synapse baselines; installed-build no-key/offline/failure acceptance. Current real cases have not been submitted to Jev or Haiku.
+
+Sources: [live smoke](reports/live-smoke-2026-09-20.json), [validation progress](VALIDATION_PROGRESS.md), [baseline](BASELINE_COMPARISON.md), [historical corpus](HISTORICAL_CORPUS.md), [context review](CONTEXT_REVIEW.md).

package/core/dev/jev-eval/HISTORICAL_CORPUS.md

@@ -0,0 +1,33 @@
+# Local historical corpus preparation ? 2026-09-20
+
+User authorized local Kenaz-related historical sessions on this machine. No transcript was sent to Jev, Claude, or another external model. Source history was read only.
+
+## Result
+
+638 files discovered in the four Kenaz-named Claude project directories and Codex session tree. Exact cwd checks restrict candidate records to the KenazAI tree and installed Kenaz core. This is workspace-based relevance, not proof that every message discusses Kenaz product code. Nested subagents are excluded. No archived Codex files were found by the parent inventory.
+
+580 eligible candidate records became **458 unique pending messages across 68 session IDs / 68 inferred groups** after 122 duplicate removals. Sources: Claude 160, Codex 298. Target 500 has a **42-message shortfall**; no synthetic padding was added. These are source-history candidates, not independently verified human-authored gold. Shared lineage cannot be completely inferred from logs; session IDs are not proof of statistical independence.
+
+Private artifacts: `.kenaz/jev-corpus/history-2026-09-20-v2/{pending.jsonl,manifest.json,aggregate.json}`. The private root is excluded using local `.git/info/exclude`. Do not commit or upload it. The manifest maps hashed IDs to local relative source paths and line numbers. Public aggregate: [report](reports/history-corpus-2026-09-20.json).
+
+The first export (`history-2026-09-20`) is marked REJECTED: parent spot checks caught dispatch and summarization prompts incorrectly recorded as user-role events. Source confirmation in `.formation/tools/prompt.js:HEADLESS_PREFIX` and `.formation/tools/muninn.js:translateChunk` led to whole-session exclusion, including matching session IDs across files. Its 500-row count is invalid and superseded.
+
+## Contract and remaining review
+
+Every row is `pending_historical_review`, `unreviewed`, with no fabricated goal, assumptions or labels. All splits remain unassigned. Context contains at most the preceding eligible user message; 57 rows have none, and all rows explicitly flag incomplete correction context because assistant assumptions are absent. This file intentionally cannot be scored as a schema-v1 labelled evaluation dataset or applied directly with review.mjs.
+
+Next review must verify human authorship and topical suitability, retrieve preceding assistant context where necessary from the local manifest, establish the actual goal/assumptions, and annotate independently. Ambiguous cases remain excluded from scoring. Assign splits by lineage only after this review. Local manual review is also needed for privacy: pattern filters are not a guarantee against every credential or personal detail.
+
+44/44 offline tests passed, including 11 corpus-specific tests. Parent separately checked exact row/ID/normalized-message counts, manifest coverage, pending provenance and obvious credential/automation markers, then spot-checked message prefixes. Neither these checks nor 458 pending rows complete the planned human-reviewed 500-message dataset. Recall and handoff targets also remain separate work. Production Jev remains disabled; the Haiku baseline remains blocked by CLI weekly quota.
+
+## Reproduce
+
+```powershell
+# Inventory only, aggregate stdout; no files written
+node dev/jev-eval/corpus.mjs --target 500
+# Explicit local export: requires ignored root and a NEW direct-child directory
+node dev/jev-eval/corpus.mjs --target 500 --out-dir .kenaz/jev-corpus/history-new
+node --test dev/jev-eval/test/*.test.mjs
+```
+
+The extractor streams with line/file/candidate budgets and reports truncations. Current scan discarded 10 oversized lines; it did not exhaust file or candidate budgets. Counts of exclusion events may overlap and are not distinct-person or distinct-message counts. Credentials trigger whole-message omission, followed by context reset; known automated session seeds reject the whole source/session. Generic repeated short messages are globally deduplicated without asserting common lineage; copied messages of at least 160 characters conservatively join groups. No split has been assigned.

package/core/dev/jev-eval/PACKAGING_VERIFICATION.md

@@ -0,0 +1,42 @@
+# Jev packaging verification — TASK_42
+
+Verified on 2026-09-20 using Node v24.14.1. This is an isolated packaging candidate check, not an installer build or installed application acceptance.
+
+**Current blocker:** `.formation/tools/jev-correction.js` is untracked in the real repository index. The actual `dashboard/scripts/kenaz-core-staging.js` dry run omits it. Muninn is tracked and included, so releasing this working tree without including the helper in the release index would leave the optional adapter unavailable and preserve the baseline silently.
+
+Reproduce from the repository root:
+
+```powershell
+node .kenaz/jev-package-verification/verify.cjs
+```
+
+The script expects the present untracked-helper condition. It creates a fresh `run-*` directory beneath `.kenaz/jev-package-verification/` and never deletes an existing stage. Its real-repository staging call uses `dryRun: true`; its physical staging calls use a tiny independent Git fixture, not the active runtime stage. Git writes are restricted to that fixture's private index and object store; no commit is created. The fixture baseline tracks only the three already-tracked runtime files below; the candidate adds only the Jev helper. This models the relevant whitelist transition without copying the full repository/dependency tree or changing the real index.
+
+The successful evidence is `.kenaz/jev-package-verification/run-gHsj58/report.json`. It records the real staging implementation hash, original index hashes, runtime file manifest, actual worker result and assertions. Source, fixture and staged SHA256 values match for all four files:
+
+| Runtime file | SHA256 |
+| --- | --- |
+| `.formation/tools/muninn.js` | `4446cbd2e0c8b676dc702bc5286555813b4a1b754e2277140de8de88868f074b` |
+| `.formation/tools/log-reader.js` | `7064552a30695a55fd60c9bcbbba465b80c661b890400ad72ee4ee5a8c97a9ab` |
+| `.formation/tools/utils.js` | `f3e6504b63e188fb50866674e58d088bbfbf4323da0c0e49942c94df2f75bb7a` |
+| `.formation/tools/jev-correction.js` | `864f0bded96862657cd138b4f0ab17b6374c9416680b873e36fbd6e3c77d0674` |
+
+The real index SHA256 remained `f0492789c69e2f678c055f5d87ba7a70c46e491ef85b543cb600b6e7ac0f03a2` before and after. The four original runtime source hashes were rechecked after execution and remained unchanged.
+
+Verified behaviors:
+
+- Original index and fixture baseline exclude the helper; isolated candidate includes it using the unchanged real staging function.
+- Staged Muninn and its direct local dependencies load successfully. The helper uses Node built-ins; this test does not execute every dynamic Muninn dependency.
+- Actual staged worker is launched using the current Node executable with synthetic stdin and an explicitly empty `TYPESAFE_API_KEY`. It exits successfully with `{"status":"missing_key"}`. A preload guard instruments fetch/HTTP entry points and observes zero network attempts.
+- Staged adapter with mocked child results preserves exact baseline for OFF, missing key and failure. OFF/missing key produce zero subprocess calls. Positive correction becomes the primary label; negative removes correction; translation is preserved.
+- Synthetic settings, corpus and telemetry sentinels are absent from the stage. No real settings, corpus, credentials or user profile are copied. Settings have an explicit exclusion; Jev corpus/telemetry are excluded here because they are untracked. Do not infer an explicit Jev-specific privacy exclusion if those paths were accidentally tracked.
+
+Build prerequisites and next verification steps, inspected but not executed:
+
+1. Include the new runtime helper and reviewed changes through the normal authorized release tracking workflow. The current blocker remains until that happens; this task does not stage or commit real changes.
+2. Repeat actual staging in an isolated release workspace and verify the helper/Muninn hashes there. Do not invoke the staging function on an existing live stage for this check: it wipes its destination, with a Windows shell fallback on removal failure.
+3. Prepare the existing frontend, k-cli and Rust toolchains and resources in that isolated workspace. `dashboard/src-tauri/Cargo.toml` specifies Rust 1.94.1. `build-static.js` installs/builds/prunes k-cli dependencies, stages kenaz-core, exports the cortex seed, mirrors the debug stage, builds the PTY server, builds standalone Next.js and obfuscates server output. Those operations have broader side effects and were not part of this check.
+4. Review version/release handling before invoking build commands. `npm run tauri:build` runs `sync-version.js --bump`; Tauri's `beforeBuildCommand` runs `npm run build:static`. The configured target is NSIS and resources include kenaz-core, nextjs-server, PTY, ONNX and documentation. No version bump, full build, installer execution or release occurred here.
+5. In a separately authorized installer test, verify the packaged Node runtime, settings persistence, worker environment inheritance, restart behavior and actual installed correction flow. Run no-key/OFF tests before any opt-in live model test. Turning OFF restores future baseline decisions; it does not undo earlier profile updates.
+
+No real provider request was made. The candidate uses the current Node executable rather than validating a bundled portable Node. Native binaries, UI assets, installer signing/updater artifacts and installed startup behavior remain unverified by TASK_42.

package/core/dev/jev-eval/README.md

@@ -0,0 +1,101 @@
+# Kenaz Jev 離線評測工具
+
+> **LEGACY (TASK_1987, 2026-09-21)**: the Muninn observer this integration overlays (translateAndClassifyMessages via muninn-watcher) is retired from all default paths; the Dashboard *Jev-assisted judgments* switch stays OFF and reaches no live consumer. Evidence and decision: `dev/MUNINN_RETIREMENT.md`. The research harness below remains runnable offline.
+
+這是 J0/J1 研究原型。用途是驗證資料格式、API adapter、回放與評分流程；尚未接入 Dashboard、Synapse 或 Muninn。`promotion_eligible` 固定為 `false`，不會因合成資料拿到高分就自動上線。
+
+需要 Node.js 22 以上。使用內建模組，不需 npm install。以下命令從 KenazAI 根目錄執行。
+
+## 離線操作
+
+```powershell
+node --test dev/jev-eval/test/*.test.mjs
+node dev/jev-eval/cli.mjs validate --dataset dev/jev-eval/fixtures/synthetic.jsonl
+node dev/jev-eval/cli.mjs evaluate --dataset dev/jev-eval/fixtures/synthetic.jsonl --provider rules
+node dev/jev-eval/cli.mjs template
+node dev/jev-eval/cli.mjs preflight
+```
+
+`rules` 是簡單、可重複的本地基準，不是 Jev 模型或其品質模擬。報告可能顯示錯誤，這是比較基準的觀察結果；工程測試綠燈只代表評測工具符合測試契約。
+
+CLI 的 JSON 輸出可供其他工具讀取。先檢查 exit code，再解析 stdout；錯誤不代表資料集得零分。不要把 live provider 的不可用/逾時樣本當作正確回答。
+
+## 資料與回放
+
+故障研究矩陣可離線執行：`node dev/jev-eval/resilience.mjs`。它用本機 HTTP fixture 驗證 off、shadow、evaluation-active，結果分開記錄 provider 失敗與 fallback 交付。shadow 透過 deliver callback 先交付基準，但函式回傳的 Promise 仍等待觀察結果，不能將 await 直接當作產品非阻塞介面。這不是 production provider，沒有安裝版驗收宣告。
+
+人工盲審工具：
+
+```powershell
+node dev/jev-eval/review.mjs export --dataset dev/jev-eval/fixtures/synthetic.jsonl --out review-new.json
+node dev/jev-eval/review.mjs apply --dataset dev/jev-eval/fixtures/synthetic.jsonl --reviews review-new.json --out reviewed-new.jsonl
+```
+
+export 隱藏原始 gold 與模型結果；人工填寫 decision、reviewed、reviewer、reviewed_at（UTC ISO 時間）後才能 apply。apply 要求每筆完整、身分和 hash 相符，且輸出必須是新檔，不覆寫來源。先寫同目錄暫存檔再以 hard link 發布，檔案系統若不支援 hard link 會失敗，不降級成覆寫。審閱旗標是聲明，不是真人身分驗證。
+
+本輪已準備 [6 筆真實片段待審檔](reports/conversation-review.pending.json) 與 [11 筆合成案例待審檔](reports/synthetic-review.pending.json)，皆未填答案。來源、比較設計、剩餘缺口見 [VALIDATION_PROGRESS.md](VALIDATION_PROGRESS.md)。
+
+資料使用 JSONL，一行一筆，欄位包括 schema_version、id、project_id、session_id、group_id、task_id、event_id、event_seq、task_revision、purpose、language、split、provenance、state。
+
+- correction：`labels.correction` 是 gold，`state.message` 是待判斷原文，`state.assumptions` 是必要前提。
+- relevance：`candidates` 含穩定 id、text 和 grade 0–3。grade 只用於評測，不送到 Jev。
+- provenance 明確區分 synthetic/real 與 agent_proposed/human_verified。審閱流程見 [ANNOTATION_GUIDE.md](ANNOTATION_GUIDE.md)。
+- 同 session/group 不能跨 development、calibration、held_out，避免資料洩漏。
+
+回放 prediction 必須與 dataset 的 id/project/session/task/event/sequence/revision 及 state_hash 對應，並包含 model、question_set_version、status、failure_reason、http_status、usage、latency_ms，以及 correction_probability 或 scores。改了原文或候選後不能沿用舊結果。每筆資料必須有一筆 prediction；未回答使用明確失敗狀態及 null，不把缺失或非法分數當成 0。完整契約以 schema.mjs 的 validatePrediction 為準。
+
+```powershell
+node dev/jev-eval/cli.mjs evaluate --dataset <dataset.jsonl> --provider replay --predictions <predictions.jsonl>
+```
+
+回放延遲是錄製資料，rules 延遲是本地計算時間，兩者都不是當下 Jev 網路延遲。`--split held_out` 用於凍結問法與門檻後的驗收，不得用它反覆調參。
+
+修正分類以 0.5 為研究門檻；recall_including_abstentions 將未回答的正例計入漏失，Brier 僅涵蓋已回答案例，須搭配 coverage 判讀。重排只排序已回答的候選，但 NDCG 的理想排序包含所有候選；另列完整查詢限定的 NDCG 與候選覆蓋率。全部 grade=0 的查詢不納入 NDCG/Recall 平均，排除數會列出。這些數字不含信賴區間，不能直接當上線驗收。
+
+## 真實 API 測試
+
+live 模式需要本機 `TYPESAFE_API_KEY`。請透過本機安全設定配置，不把 key 貼到聊天、JSONL、報告、git 或命令列參數。
+
+`preflight` 只回報金鑰是否存在與固定模型版本，不連網、不輸出金鑰。CLI 不提供 endpoint 或模型覆寫選項。
+
+```powershell
+node dev/jev-eval/cli.mjs evaluate --dataset dev/jev-eval/fixtures/synthetic.jsonl --provider live --timeout-ms 800
+```
+
+這個命令會把指定資料集的必要 state/候選文字送到 TypeSafe API，因此僅使用已準備可外送的資料。預設固定模型版本 `jev-1.13.0`。工具不掃描環境檔案、不擷取 live session、不自動上傳 Synapse，也不在離線模式使用金鑰。
+
+本原型沒有同步重試；總期限應涵蓋 response body。401/422、429/529、畸形回答、缺少題目、越界分数都需要在結果中辨認，不以原始 HTTP body 作錯誤訊息，避免供應商回顯的敏感資料流入 log。
+
+## 評估邊界
+
+### 2026-09-20 API 冒煙實測
+
+已安裝並使用專案的 [TypeSafe skill](../../.agents/skills/typesafe-ai/SKILL.md)，以官方即時 API/model 文件核對既有 adapter，無需更改請求格式。金鑰僅透過測試子程序環境變數使用，未保存於專案。
+
+[實測 JSON](reports/live-smoke-2026-09-20.json) 保留兩輪各 11 筆合成案例：10 秒期限輪全部成功，p50 273ms、最大 736ms；800ms 期限輪全部成功，p50 272ms、最大 653ms。更正辨識各為 8/8，兩筆可計算的排序查詢 NDCG@5 為 1；全無相關候選的第三筆不納入此平均。同期 rules 更正辨識為 4/8，NDCG@5 約 0.822。
+
+共執行三輪 API 請求，中間一輪工具輸出被截斷，不計入保留的統計。兩輪使用相同小型合成資料；重複輸入的服務端快取狀態未知，p95/p99 因每輪僅 11 筆均等於最大值。這不是獨立真人品質評估、穩態尾延遲或 production SLO。下一步仍是人工覆核的真實評測集與現行分類／排序基準比較，production 未啟用 Jev。
+
+目前附帶的是小型合成回歸集合，標籤由代理提出。它不能取代計畫中的 500 則訊息、100 組 recall 與 30 段交接情境，也不構成真人驗證的 held-out 效果。
+
+正式報告需另補：繁中/英文分開標註、現行 Haiku 與 Synapse 的公平基準、延遲分布、人工複核，以及壓縮/交接後任務是否真的更少漂移。沒有供應商實測就不能宣稱 Jev 比現有流程快或準。
+
+## 來源
+
+現有 Muninn 分類基準的隔離執行與本機額度阻塞紀錄，見 [BASELINE_COMPARISON.md](BASELINE_COMPARISON.md)。`node dev/jev-eval/muninn-baseline.mjs` 預設只做離線契約檢查；live 須明確指定 CLI，且只接受合成資料。回傳報告的 status 也必須成功，不能只用 exit code 判定模型回答有效。
+
+- [完整導入規劃](../JEV_INTEGRATION_PLAN.md)
+- [TypeSafe API](https://docs.typesafe.ai/api)
+- [模型版本與語言限制](https://docs.typesafe.ai/models)
+- [機率與 confidence](https://docs.typesafe.ai/confidence)
+- [Jev 已知限制](https://docs.typesafe.ai/model-jaggedness/jev-1.13)
+
+所有 production 功能開關保持既有狀態；本目錄的工具不會啟用模型、修改任務狀態、寫入記憶權重或啟動額外常駐服務。
+
+## Local historical corpus preparation
+
+The separate opt-in `corpus.mjs` tool inventories authorized local history without network calls. See [HISTORICAL_CORPUS.md](HISTORICAL_CORPUS.md): 458 unreviewed candidates, 42 short of target, not a labelled evaluation set. Existing evaluation CLI does not automatically collect or upload history.
+
+Local observed-context restoration and read-only review packets: [CONTEXT_REVIEW.md](CONTEXT_REVIEW.md). 419 partial, 23 missing, 16 rejected; 0 human-reviewed.
+
+Current consolidated findings and local annotation pilot: [EVALUATION_STATUS.md](EVALUATION_STATUS.md).

package/core/dev/jev-eval/THALAMUS_RANKING.md

@@ -0,0 +1,58 @@
+# Jev Thalamus context ranking
+
+Status: local implementation verified, 2026-09-20. No installed release or real relevance improvement claimed.
+
+## Delivery contract
+
+The single Dashboard **Jev-assisted judgments** switch (TASK_1970) controls both existing project-local fields, `jevThalamusEnabled` and `jevCorrectionEnabled`, in one atomic settings request. Both default to false. Runtime fields remain separate for compatibility, while the Dashboard presents one control. The server reads `TYPESAFE_API_KEY` from its environment; no developer key is embedded or returned.
+
+The shared Rust service receives the current task and a bounded shortlist, asks one TypeSafe Score question per candidate, and returns a stable descending permutation. Same-score ties retain input order. It preserves all candidates; existing delivery budgets decide what is shown. It does not alter Thalamus potential, firing thresholds, decay, propagation, or cascade chain order.
+
+POST `/api/synapse/jev-thalamus-rank` accepts `{projectPath, taskContext, candidates:[{id,text}]}`. The project must be an explicit existing absolute directory with its own opt-in flag. The response is `{orderedIds,status,reason}`; only `status: ranked` may change delivery. Unknown or duplicated returned IDs must never be applied by consumers.
+
+Limits: at most eight candidates; task 4096 UTF-8 bytes; candidate text 2048 bytes and ID 512 bytes; request 32 KiB; provider response 64 KiB. The fixed provider is `https://api.typesafe.ai/v1/systemone`, model `jev-1.13.0`. One 800 ms deadline covers provider transport and body, with no retry or redirect. Config reads have separate bounds, so 800 ms is not an end-to-end workflow latency promise.
+
+Missing/false/malformed config, missing key, unusable input, sensitive-text patterns, timeout, HTTP failure or invalid Score envelope preserve baseline order. Disabling the flag while a request runs discards its result. Sensitive-text checks are conservative heuristics, not a guarantee that arbitrary text contains no secrets. No raw-text decision telemetry is written by this service.
+
+## Consumers
+
+- Native Claude UserPromptSubmit: TASK_44 wiring orders recalled Active Sensing entries before the existing 800-character rendering budget. It sends the current bounded prompt and existing candidate summaries, not the full transcript. The hook retains original context, DDx/reflex messages, and Alaya output.
+- MCP `thalamus_context`: optional `taskContext` enables ranking of delivered near-threshold/recent-fire/cluster-symbol lists. Names, paths, and docstrings come from the project's existing read-only search index. Missing context or candidates retains baseline. A 200 ms caller deadline bounds enrichment waiting; the blocking SQLite worker may finish later. This is not hard cancellation of database computation.
+- Bifrost does not invoke native `.claude/hooks/` automatically. This implementation does not claim automatic Bifrost injection; its callers need the explicit MCP context argument.
+
+TASK_44 corrected the existing native hook recall mismatch (`query` instead of required `q`) and added explicit `projectPath` to recall and turn-context requests. Its separate existing seconds/milliseconds cooldown mismatch is outside this change.
+
+## Settings and feedback
+
+TASK_1970 supersedes the separate UI introduced in TASK_1967 with one Default Models switch and one overall feedback area. GET status uses `feature=all`, derives both flags from the same config snapshot, and reports mixed legacy settings explicitly. A toggle sends both flags with the same value in one request and requires both acknowledgements. New overall feedback is tagged `all`; previous correction/Thalamus records retain their classifications, and legacy records without a discriminator belong to correction. Feedback is overall experience, not labeled ground truth.
+
+## Verification and rollout
+
+Parent verified `node --test .claude/hooks/jev-thalamus-context.test.js`: 7/7 passed. This executes the real hook in a fixture VM and checks rendered selection before truncation, exact OFF/failure fallback, disabled-in-flight results, request bounds, and preserved safety/Alaya content. `node dev/scripts/run-rust-tests.js jev_thalamus` passed 12/12 tests (parent inspected the canonical runner log). Coverage includes shared scoring, read-only enrichment, actual route OFF/body limit, and MCP no-context/OFF behavior. The positive MCP path is assembled from those tested components; no live provider-to-MCP run was performed. Parent reran the two focused Jest files: 17/17 passed. Full Dashboard TypeScript checking passed with --noEmit --incremental false. Final combined `node dev/scripts/run-rust-tests.js jev` passed 23/23 (12 ranking/route/MCP + 11 settings). Together with 77 Node and 17 Jest tests, 117 engineering checks passed. Existing correction/research regression: `node --test .formation/tools/jev-correction.test.js dev/jev-eval/test/*.test.mjs` passed 70 tests during this work.
+
+Real Jev quality, Chinese-task relevance, representative end-to-end latency and comparison against baseline remain unmeasured for this feature. Offline mocked ranking validates delivery and fallback, not model accuracy. No historical session data was uploaded during this implementation.
+
+The core stager includes tracked `.claude` files only. A new untracked hook helper is omitted from an actual package until it is included in the tracked release candidate. No installer build, merge, or release has been performed.
+
+References: [TypeSafe API](https://docs.typesafe.ai/api), [Score](https://docs.typesafe.ai/primitives/score), [reranking cookbook](https://docs.typesafe.ai/cookbooks/rerank_typesafe). Official examples motivate the mechanism; they are not Kenaz measurements.
+
+## Task tracking migration
+
+The user explicitly restored KenazAI as the tracking project. Existing implementations were not redispatched. Old quotation cards retain their history and forward references, with duplicate tracking closed as superseded; this does not cancel the implementation.
+
+| quotation history | KenazAI authoritative task | Scope |
+| --- | --- | --- |
+| TASK_43 | TASK_1965 | Shared Rust ranker and MCP/HTTP delivery |
+| TASK_44 | TASK_1966 | Native hook delivery |
+| TASK_45 | TASK_1967 | Independent settings and feedback |
+| TASK_46 | TASK_1968 | Verification documentation |
+
+TASK_1970 UI update: parent verified focused Jest 23/23, including actual panel SSR with exactly one checkbox, mixed legacy state, one-request dual-flag persistence, and partial-acknowledgement rejection. Canonical Rust filter `jev` passed 26/26; full Dashboard TypeScript and diff checks passed. Live combined status returned feature=all, enabled=true, mixed=false, both flags=true; runtime keyConfigured remained false, so no provider probe was sent.
+
+TASK_1971 follow-up: Windows startup falls back to HKCU Environment TYPESAFE_API_KEY
+only when absent from the inherited process environment. Six startup tests and
+12 Jev regression tests passed. After natural dev reload, combined status reported
+keyConfigured=true; one synthetic live request returned ranked, orderedIds
+[startup, docx], reason=null. Client elapsed time was 2726ms including localhost
+request overhead; this is not a provider-only latency measurement or quality eval.
+No real transcript was sent, and installed release packaging remains unverified.

package/core/dev/jev-eval/VALIDATION_PROGRESS.md

@@ -0,0 +1,54 @@
+# Jev 驗證進度 — 2026-09-20
+
+本輪授權：自主執行先前提出的品質比較與故障測試計畫。TASK_33 追蹤研究工具；TASK_27 延續資料與驗證文件。production 尚未接入，不把研究工具的回退視為安裝版已驗收。
+
+## 資料與人工審閱
+
+- 原有 11 筆 synthetic/agent_proposed 案例繼續用作工程回歸。
+- 新增 `fixtures/conversation-review.seed.jsonl`：本對話 6 則真實用戶片段，全部 real/agent_proposed、同 session/group、僅 development，不是獨立盲測。沒有呼叫 API。
+- `state.message` 是用戶原文片段；waitlist 案例只保留 BUG 回饋連結前的文字。金鑰及含金鑰整則訊息均不納入。goal 與 assumptions 是代理依鄰近對話整理的摘要，不是用戶原話或已驗證隱含意圖。
+- session/event identifiers 是本地摘錄識別碼，event_seq 是此摘錄集合的編號，不代表 Dashboard 原始事件序列。原始來源是本 Codex 對話，未掃描私有 transcript 或其他專案。
+- installed-only 與 waitlist-priority 的暫定正例尤其需要人工裁決：補充環境、改變優先序是否算更正，取決於已定義用途。上下文不足時應排除或補足，不為了類別平衡強行確定 gold。
+- 真人覆核數目前為 0。500 則訊息、100 組搜尋候選、30 段交接仍是目標，未用大量改寫湊數。
+
+人工先只看 `review.mjs export` 產生的 review packet，不看 seed 的代理標籤、模型分數或排名。填入判斷、逐項 reviewed、reviewer 與 reviewed_at 後，由 apply 驗證身分、內容 hash、候選與完整性，產生另一份資料集。此操作記錄人的審閱宣告，不能技術上證明填寫者是人。保留原始 seed；不自動把 generated 改成 human_verified。
+
+## 比較設計
+
+1. 先收集獨立 session 的足量材料，去識別與人工審閱，保留繁中、英文、混合語言及難例。來源不足先列缺口。
+2. 同一原始案例的翻譯、改寫與候選排列共用 group；按 session/group 分割 development、calibration、held_out。此單一對話不可拆散到三個 split。
+3. 在 held_out 前凍結 model、question/rubric、threshold、資料 hash 與比較版本。測試結果看過後調參，需新盲測集。
+4. 同批資料比較現行 Kenaz 分類／排序、rules、Jev。rules 只是詞彙基準，不能冒充現有 Haiku 或 Synapse。先保存現行流程的 prediction，驗證身分與 state_hash 後才回放比較。
+5. 分語言、正負例報告更正誤報／漏報與 coverage；排序列 NDCG、候選覆蓋及零相關候選排除數。失敗不能用 fallback 的正確結果冒充 Jev 回答正確。
+6. 性能按短／中／接近容量輸入、首次／後續連線及併發分組，單獨記錄失敗與請求量；不把 11 筆的最大值當可靠 p99。現有 API smoke 只證明可接通，這些性能組尚未執行。
+
+## 故障研究與產品驗收邊界
+
+`resilience.mjs` 只驗證研究協調器：off 不請求，shadow 保留基準結果，evaluation-active 在失敗時回基準；使用本地 HTTP fixture。基準注入以觀察是否重複執行，預設研究 rules 並非產品回退。
+
+產品驗收仍需真正的 provider 接線與安裝包：無金鑰啟動、離線啟動、401/429/529、請求中斷、頁面可用、任務執行與記憶寫入不受影響，並確認無重複副作用。應在隔離測試專案與測試安裝環境執行，不對現有用戶圖做破壞性重建。這些安裝版驗收目前未完成。
+
+驗收方向：無金鑰與 off 必須零 Jev 請求；shadow 不更動交付結果；provider 失敗保留原有行為；800ms 是 provider 等待預算，不是整個應用的完成時限。品質提升門檻需依代表性人工資料及誤判代價預先決定，不用本輪合成分數反推合格線。
+
+## 本輪執行結果
+
+主代理已獨立執行以下驗證：
+
+- `node --test dev/jev-eval/test/*.test.mjs`：26/26 通過，0 skipped。涵蓋既有 adapter、審閱流程、provider 身分／版本、非 Error 例外、缺金鑰、shadow 提早交付、基準失敗不重試、逾時後晚到回應不重複交付。
+- `node dev/jev-eval/resilience.mjs`：27 組離線觀察、4 次預期逾時，每組只交付一次；off／缺金鑰零 transport calls。原始結果在 [resilience-2026-09-20.json](reports/resilience-2026-09-20.json)。本地矩陣以 Connection: close 隔離短命伺服器，避免連線重用干擾 fault 類型；不是正式 provider 設定。
+- 真實 seed 6 筆通過既有 schema validate；兩份盲審檔共 17 筆，皆無原始 gold、模型結果與 reviewer。獨立嘗試 apply 未審的真實片段檔，得到 unreviewed_item，且沒有建立部分輸出檔。
+- 針對 production 的 dashboard/src、dashboard/src-tauri/src 與 Muninn 搜尋，沒有新增研究工具/API/key 引用。目前執行中的 Dashboard 路徑是 target/debug，不是安裝版；未宣稱安裝版驗收。
+
+合成長度探測使用現有 liveProvider、固定 jev-1.13.0、800ms deadline，逐筆送出，每個長度 3 次，9/9 收到合法回答。只送合成填充內容，不送 6 則真實片段。原始結果在 [length-probe-2026-09-20.json](reports/length-probe-2026-09-20.json)。
+
+| 實際 JSON 請求 bytes | 次數 | 成功 | 中位數 ms | 最大 ms |
+| --- | --- | --- | --- | --- |
+| 959 | 3 | 3 | 568 | 715 |
+| 8639 | 3 | 3 | 278 | 299 |
+| 48456 | 3 | 3 | 441 | 457 |
+
+重複填充文字不代表真實文件分布，供應商快取狀態未知，沒有併發或冷／熱連線分組。不能從 9 次請求推論可靠 p99、輸入長度因果關係或正式 SLO。
+
+另查證 `.formation/tools/muninn.js:translateAndClassifyMessages` 同時產生英文翻譯與七類訊號；Jev correction Noul 不提供翻譯，也不是七類多標籤分類。正式比較須分別驗證更正訊號、其他六類與翻譯下游依賴，不以單一 Noul 時間宣稱完全替換 Haiku。
+
+限制：shadow callback 先交付，但 evaluateMode 的 Promise 仍等待 provider observation；它是研究介面，未接應用 critical path。原子輸出需要 hard link 支援。500/100/30 目標、人工覆核、獨立 held_out、Haiku/Synapse 等價比較、安裝版驗收均未完成。production 不啟用，沒有打包／發布／commit／merge。

package/core/dev/jev-eval/cli.mjs

@@ -0,0 +1,50 @@
+import { fileURLToPath } from 'node:url';
+import { resolve } from 'node:path';
+import { readJsonl, validateDataset, SPLITS, check } from './schema.mjs';
+import { MODEL, rules, replayProvider, liveProvider } from './providers.mjs';
+import { report } from './metrics.mjs';
+
+export function template() {
+  return { schema_version: 1, id: 'example-1', project_id: 'example-project', session_id: 'example-session', group_id: 'example-group', task_id: 'TASK_EXAMPLE', event_id: 'event-1', event_seq: 1, task_revision: 1,
+    purpose: 'correction', language: 'en', split: 'development', provenance: { source: 'synthetic', annotation: 'agent_proposed', reviewer: null, reviewed_at: null },
+    state: { goal: 'Diagnose installed Dashboard startup', assumptions: ['Development startup also fails'], message: 'Actually, only the installed application fails.' }, labels: { correction: true } };
+}
+export async function main(argv = process.argv.slice(2)) {
+  const [command, ...args] = argv;
+  check(['validate', 'evaluate', 'template', 'preflight'].includes(command), 'invalid_command');
+  const options = {};
+  for (let i = 0; i < args.length; i += 2) {
+    check(/^--[a-z-]+$/.test(args[i]) && args[i + 1] && !args[i + 1].startsWith('--') && !Object.hasOwn(options, args[i]), 'invalid_arguments');
+    options[args[i]] = args[i + 1];
+  }
+  const allowed = command === 'evaluate' ? ['--dataset', '--provider', '--predictions', '--split', '--k', '--timeout-ms'] : command === 'validate' ? ['--dataset'] : [];
+  check(Object.keys(options).every(k => allowed.includes(k)), 'unknown_option');
+  if (command === 'template') return template();
+  if (command === 'preflight') return { schema_version: 1, model: MODEL, api_key_available: typeof process.env.TYPESAFE_API_KEY === 'string' && process.env.TYPESAFE_API_KEY.trim().length > 0 && !/[\r\n]/.test(process.env.TYPESAFE_API_KEY), network_attempted: false };
+  check(options['--dataset'], 'dataset_required');
+  const all = validateDataset(await readJsonl(options['--dataset']));
+  if (command === 'validate') return { schema_version: 1, valid: true, count: all.length, promotion_eligible: false };
+  const name = options['--provider'] ?? 'rules';
+  check(['rules', 'replay', 'live'].includes(name), 'invalid_provider');
+  check(!options['--predictions'] || name === 'replay', 'predictions_require_replay');
+  check(!options['--timeout-ms'] || name === 'live', 'timeout_requires_live');
+  check(!options['--split'] || SPLITS.includes(options['--split']), 'invalid_split');
+  const k = Number(options['--k'] ?? 5), timeoutMs = Number(options['--timeout-ms'] ?? 800);
+  check(Number.isInteger(k) && k >= 1 && k <= 12, 'invalid_k');
+  let provider;
+  if (name === 'replay') { check(options['--predictions'], 'predictions_required'); provider = replayProvider(all, await readJsonl(options['--predictions'])); }
+  else provider = name === 'live' ? liveProvider({ timeoutMs }) : rules;
+  const rows = all.filter(r => !options['--split'] || r.split === options['--split']);
+  check(rows.length > 0, 'empty_split');
+  const predictions = [];
+  // Sequential offline research execution avoids rate-limit bursts; each row batches its questions.
+  for (const row of rows) predictions.push(await provider(row));
+  return report(rows, predictions, { provider: name, k });
+}
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+  main().then(result => console.log(JSON.stringify(result, null, 2))).catch(error => {
+    // Only expose fixed error codes, never paths, provider bodies, keys, or dataset contents.
+    const code = /^[a-z_]{1,80}$/.test(error.message) ? error.message : 'operation_failed';
+    console.error(JSON.stringify({ error: code })); process.exitCode = 1;
+  });
+}