ltcai 9.9.4 → 9.9.5

package/README.md

@@ -11,7 +11,7 @@
 [![CI Status](https://github.com/TaeSooPark-PTS/LatticeAI/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/TaeSooPark-PTS/LatticeAI/actions/workflows/ci.yml)
 [![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
-![v9.9.4 Living Brain walkthrough](output/release/v9.9.4/gifs/v9.9.4-living-brain-walkthrough.gif)
+![v9.9.5 Living Brain walkthrough](output/release/v9.9.5/gifs/v9.9.5-living-brain-walkthrough.gif)
 
 Chat, files, folders, notes, and web pages all flow into one durable knowledge
 graph on your computer. Any model — local MLX or cloud — can speak with that
@@ -24,9 +24,9 @@ memory. Nothing leaves your machine without explicit consent.
 
 | | |
 | --- | --- |
-| **Chat with a Brain that remembers** — every conversation grows durable, source-linked memory ![Brain Chat](output/release/v9.9.4/screenshots/04-brain-chat-home.png) | **See how knowledge connects** — a real relationship graph, not a file list ![Memory Graph](output/release/v9.9.4/screenshots/05-memory-graph.png) |
-| **Capture anything** — files, whole folders, notes, screenshots, web pages ![Capture](output/release/v9.9.4/screenshots/06-capture.png) | **Automate with review** — agent changes become proposals you approve first ![Review Center](output/release/v9.9.4/screenshots/12-review-center.png) |
-| **Pick a model in one click** — recommended local models for your hardware ![Recommended Models](output/release/v9.9.4/screenshots/02-recommended-models.png) | **Stay in control** — audit, roles, retention in a separate admin surface ![Admin Console](output/release/v9.9.4/screenshots/10-admin-console.png) |
+| **Chat with a Brain that remembers** — every conversation grows durable, source-linked memory ![Brain Chat](output/release/v9.9.5/screenshots/04-brain-chat-home.png) | **See how knowledge connects** — a real relationship graph, not a file list ![Memory Graph](output/release/v9.9.5/screenshots/05-memory-graph.png) |
+| **Capture anything** — files, whole folders, notes, screenshots, web pages ![Capture](output/release/v9.9.5/screenshots/06-capture.png) | **Automate with review** — agent changes become proposals you approve first ![Review Center](output/release/v9.9.5/screenshots/12-review-center.png) |
+| **Pick a model in one click** — recommended local models for your hardware ![Recommended Models](output/release/v9.9.5/screenshots/02-recommended-models.png) | **Stay in control** — audit, roles, retention in a separate admin surface ![Admin Console](output/release/v9.9.5/screenshots/10-admin-console.png) |
 
 ## Why Lattice AI
 
@@ -57,52 +57,47 @@ First-run flow — wake the Brain, pick the owner, load a recommended model:
 
 | | | |
 | --- | --- | --- |
-| ![Login](output/release/v9.9.4/screenshots/01-login.png) | ![Model install](output/release/v9.9.4/screenshots/03-install-load-progress.png) | ![Model library](output/release/v9.9.4/screenshots/07-model-library.png) |
+| ![Login](output/release/v9.9.5/screenshots/01-login.png) | ![Model install](output/release/v9.9.5/screenshots/03-install-load-progress.png) | ![Model library](output/release/v9.9.5/screenshots/07-model-library.png) |
 
 Screenshot index and capture notes:
-[output/release/v9.9.4/SCREENSHOT_INDEX.md](output/release/v9.9.4/SCREENSHOT_INDEX.md)
+[output/release/v9.9.5/SCREENSHOT_INDEX.md](output/release/v9.9.5/SCREENSHOT_INDEX.md)
 
 ## Current Release
 
-The current release is **9.9.4 — Durable Loops**:
-
-- **Loops that survive restarts.** Paused `awaiting_approval` runs are
-  persisted to disk (hashed tokens, wall-clock expiry) and resume after a
-  server restart; expired approvals answer with a one-click replan hint, and
-  `GET /agent/approvals` re-surfaces pending runs. Approval cards show a live
-  TTL countdown.
-- **You can watch the loop work.** Streamed agent runs emit named
-  `agent_step` SSE events (plan → tools → verify → terminal) rendered as a
-  live step timeline; weak-model parse repairs are counted and surfaced
-  honestly ("모델 응답을 N회 보정했어요").
-- **One retrieval policy.** A single `RetrievalPolicy` module (rule-based
-  query rewrite, query-class weights, 14-day recency age-decay in a
-  [0.5, 1.0] band) feeds both the service and graph fusion layers, gated by
-  the benchmark CI test; embedder swaps now report `stale_embedder` instead
-  of silently emptying the vector channel.
-- **Retrieval quality at the chunk level.** Markdown chunks at heading
-  boundaries (with `heading_path` provenance), code at function boundaries,
-  PDF chunks carry page numbers, plain text stays byte-identical — and
-  cited source cards click through to the stored chunk text.
-- **Grounded answers, cited sources.** When retrieved context exists the
-  answer prompt instructs [1]-style citations; a synthetic grounding bench
-  gates the verdict heuristic in CI; briefings suggest periodic graph noise
-  curation, and an opt-in review mode stages curator promotions for human
-  approval.
-- **A wider harness.** Manifest-aware plan rewriting plus React/Vite and
-  Python-package manifests, a bounded executor context window, five
-  deterministic workflow scenarios, a funnel soft gate, and a weekly
-  fail-open real-model agent smoke with a report artifact.
+The current release is **9.9.5 — Closed Gaps**:
+
+- **One approval path.** Legacy `human_in_loop` now rides the same durable
+  approval store as `awaiting_approval` (hashed tokens, restart-safe). The
+  old wire contract (`waiting_approval` + `context_id`) still works; the
+  separate in-memory `_pending` map is gone.
+- **Rollback that is honest.** File recovery is `git` → pre-write `snapshot`
+  → `none`, so non-git workspaces and newly created files can still be
+  restored. Each rollback entry reports its mode.
+- **Critic sees artifact truth.** Before judging file work, the critic gets a
+  deterministic checklist of written paths with sanitize/repair flags — an
+  auto-repaired scaffold cannot pass as fulfillment unchecked.
+- **Mid-run workspace awareness.** Later executor steps see files this run
+  already wrote, so multi-step "create then explain" work is not blind.
+- **Optional cross-encoder rerank.** Off by default; set
+  `LATTICEAI_CROSS_ENCODER_RERANK=1` to reorder hybrid matches (identity
+  fallback when the model is missing). Hybrid responses carry a `rerank`
+  meta block.
+- **Surface parity for approvals.** VS Code gains List/Approve/Reject
+  commands with token cache; Telegram handles both `waiting_approval` and
+  `awaiting_approval` with token-first resume.
+- **Live sidecar E2E.** Nightly Playwright first-value tests hit a real
+  FastAPI sidecar (`npm run test:e2e:sidecar`), not only the visual mock
+  server.
 
 Release notes: [RELEASE.md](RELEASE.md) · Full history: [docs/CHANGELOG.md](docs/CHANGELOG.md)
 
-Expected artifacts for 9.9.4 release must use exact filenames:
+Expected artifacts for 9.9.5 release must use exact filenames:
 
-- `dist/ltcai-9.9.4-py3-none-any.whl`
-- `dist/ltcai-9.9.4.tar.gz`
-- `ltcai-9.9.4.tgz`
-- `dist/ltcai-9.9.4.vsix`
-- `src-tauri/target/release/bundle/dmg/Lattice AI_9.9.4_aarch64.dmg`
+- `dist/ltcai-9.9.5-py3-none-any.whl`
+- `dist/ltcai-9.9.5.tar.gz`
+- `ltcai-9.9.5.tgz`
+- `dist/ltcai-9.9.5.vsix`
+- `src-tauri/target/release/bundle/dmg/Lattice AI_9.9.5_aarch64.dmg`
 
 Do not use wildcard artifact uploads. Package registry publishing remains owner-run.
 
@@ -122,7 +117,7 @@ See [ARCHITECTURE.md](ARCHITECTURE.md) for details and
 
 - External package registries are owner-published and can lag behind GitHub.
 - PostgreSQL/pgvector is optional scale/migration tooling. SQLite remains the
-  live local Brain store in 9.9.4.
+  live local Brain store in 9.9.5.
 - Docker, model downloads, cloud model calls, Telegram, Brain Network, and
   update checks require explicit user action.
 - Conversation does not fabricate answers when no model is loaded. Agent and
@@ -134,6 +129,7 @@ See [ARCHITECTURE.md](ARCHITECTURE.md) for details and
 
 | Version | Theme |
 | --- | --- |
+| 9.9.5 | Closed Gaps |
 | 9.9.4 | Durable Loops |
 | 9.9.3 | Closed Loops |
 | 9.9.2 | Artifact Trust |

package/docs/CHANGELOG.md

@@ -4,6 +4,41 @@ The top entry is either the current unreleased main-branch work or the current
 release line. Older entries are historical and may describe behavior as it
 existed at that release.
 
+## [9.9.5] - 2026-07-26
+
+### Added
+- Optional cross-encoder rerank (`lattice_brain/graph/rerank.py`): env
+  `LATTICEAI_CROSS_ENCODER_RERANK=1` (model via
+  `LATTICEAI_CROSS_ENCODER_MODEL`); hybrid_search returns additive `rerank`
+  meta; default path is identity with no model download.
+- Sidecar-backed Playwright first-value E2E: `tests/e2e/`,
+  `scripts/run_sidecar_e2e.mjs`, `npm run test:e2e:sidecar`, nightly
+  `.github/workflows/e2e-sidecar.yml`.
+- VS Code approval commands: `ltcai.listApprovals`, `ltcai.approveAgent`,
+  `ltcai.rejectAgent` with pause-token session cache.
+- Telegram approval handles both `waiting_approval` and `awaiting_approval`
+  with run_id+token resume preferred.
+- Agent loop L4/L5/L7 helpers: `artifact_checklist`, `files_written`,
+  snapshot rollback ports (`snapshot_file` / `restore_snapshot`) with
+  mode-tagged results (`git` | `snapshot` | `none`).
+- Knowledge-graph read surface decomposition:
+  `lattice_brain/graph/retrieval_reads.py` (`KnowledgeGraphReadsMixin`).
+
+### Changed
+- Legacy `human_in_loop` pauses now use the durable `AgentRunStore` path
+  (`legacy_context=True`); the separate in-memory `_pending` map is removed.
+  Wire contract (`status=waiting_approval`, `context_id`) is preserved.
+- Rollback recovers file-create actions via pre-write snapshots when git is
+  unavailable or not governed.
+- Critic verify prompts include a deterministic artifact checklist derived
+  from transcript sanitize/repair flags.
+- Executor prompts list files already written in the current run.
+- `docs/SURFACE_PARITY.md` marks VS Code/Telegram approval as provided.
+
+### Tests
+- `test_agent_loop_l4_l5_l7.py`, `test_cross_encoder_rerank.py`,
+  `test_snapshot_rollback_ports.py`, L1 approval flow extensions.
+
 ## [9.9.4] - 2026-07-26
 
 ### Added

package/docs/COMMUNITY_AND_PLUGINS.md

@@ -1,6 +1,6 @@
 # Community And Plugins
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 LatticeAI defines the path from a strong local-first framework (8.4.0
 action-aware baseline, 8.5.0 registry+DI hardening, 8.6.0 capture/navigation

package/docs/DEVELOPMENT.md

@@ -3,7 +3,7 @@
 > **Status: canonical** — current contributor guidance, kept in sync with the
 > current release.
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 This document is for contributors working on the local-first Digital Brain
 codebase. Product positioning and quick start stay in `README.md`; release

package/docs/ONBOARDING.md

@@ -1,6 +1,6 @@
 # Lattice AI Onboarding
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 The first-run goal is a five-minute path from "I opened the app" to "my Brain
 has a source, a question, and proof." This page is the product contract behind

package/docs/OPERATIONS.md

@@ -1,4 +1,4 @@
-# Lattice AI — Operations Guide (v9.9.4)
+# Lattice AI — Operations Guide (v9.9.5)
 
 > **Status: canonical** — kept in sync with the current release. Storage layout
 > below reflects the SQLite live Brain store and workspace scoping, not the

package/docs/SURFACE_PARITY.md

@@ -2,6 +2,7 @@
 
 > **Status: reference**
 > 도입: v9.9.4 (2026-07-25, 리뷰 Wave 1.4 "표면 패리티 체크리스트").
+> 갱신: v9.9.5 — VS Code/Telegram 승인 흐름 갭 해소.
 > 목적: 각 표면(서피스)이 Brain 계약의 어디까지를 제공하는지 **정직하게** 기록한다.
 > "앱마다 다른 Lattice"를 방지하는 기준표이며, 기능을 과장하지 않는다.
 
@@ -24,15 +25,15 @@
 | 첫 저장 (Capture) | ✅ DnD·폴더·웹 수집, 품질 경고 | ✅ (웹과 동일) | ◐ `sendToLattice` 선택 영역 전송 | ✅ 페이지 캡처 (단일 목적) | ◐ 메시지 수집 |
 | 첫 회상 (Recall) | ✅ 하이브리드 검색 + grounding 배지 + 출처→청크 | ✅ (동일) | ◐ `askCurrentFile` — 배지 없음 ✖ | — | ◐ 답변만, 배지 없음 |
 | 첫 산출물 (Artifact) | ✅ artifacts[] 카드·미리보기·Brain 기억 칩 | ✅ (동일) | ◐ `createFile` — 검증 파이프라인 경유, 카드 UI 없음 | — | ◐ 파일 전송만 |
-| 첫 보호 (Approval) | ✅ 승인 카드 + TTL 카운트다운 + 재시작 생존 | ✅ (동일) | ✖ 승인 UI 없음 — 웹으로 유도 | — | ✖ 승인 불가 — 웹으로 유도 |
+| 첫 보호 (Approval) | ✅ 승인 카드 + TTL 카운트다운 + 재시작 생존 | ✅ (동일) | ✅ List/Approve/Reject 커맨드 + 토큰 캐시 | — | ✅ 인라인 Done/Cancel + run_id/token 재개 |
 
 ## 기능 상세 × 표면
 
 | 기능 | Web `/app` | Desktop | VS Code | Browser | Telegram |
 | --- | --- | --- | --- | --- | --- |
 | 에이전트 스텝 타임라인 (v9.9.4) | ✅ 라이브 SSE | ✅ | — | — | — |
-| 승인 재개 (`/agent/resume`) | ✅ | ✅ | ✖ | — | ✖ |
-| 대기 중 승인 목록 (`GET /agent/approvals`, v9.9.4) | ✅ | ✅ | ✖ | — | — |
+| 승인 재개 (`/agent/resume`) | ✅ | ✅ | ✅ `ltcai.approveAgent` / `rejectAgent` | — | ✅ callback → resume |
+| 대기 중 승인 목록 (`GET /agent/approvals`, v9.9.4) | ✅ | ✅ | ✅ `ltcai.listApprovals` | — | — (봇 로컬 pending map) |
 | Watch 상태 신호 (v9.9.4) | ✅ 홈 카드 | ✅ | — | — | — |
 | Review Center (제안 승인/거절) | ✅ | ✅ | ✖ | — | — |
 | 데모 코퍼스 First Value Loop | ✅ | ✅ | — | — | — |
@@ -43,5 +44,10 @@
 
 1. **API 계약이 기준이다.** 모든 표면은 동일한 sidecar API를 소비한다. 표면별 전용 API를 만들지 않는다.
 2. **◐/✖ 는 문서화된 상태로만 존재한다.** 새 기능을 Web에 추가할 때 이 표를 갱신하고, 다른 표면에 제공하지 않을 경우 "—(의도)" 인지 "✖(갭)" 인지 명시한다.
-3. **✖ 갭은 릴리스 노트의 Honest Limitations 후보다.** 현재 갭: VS Code/Telegram 승인 흐름(웹으로 유도하는 안내만 존재), VS Code 회상 배지.
+3. **✖ 갭은 릴리스 노트의 Honest Limitations 후보다.** 현재 갭: VS Code 회상 배지, VS Code Review Center.
 4. 릴리스마다 이 표를 검토한다 (release checklist의 docs 단계).
+
+## VS Code / Telegram 승인 메모 (v9.9.5)
+
+- **토큰은 GET `/agent/approvals`에 다시 내려가지 않는다** (보안). Web은 일시 중지 응답의 토큰을 클라이언트에 보관하고, VS Code도 동일하게 pause 응답의 토큰을 확장 세션 캐시에 둔다. 캐시에 없으면 붙여넣기 또는 웹 UI로 유도한다.
+- **Telegram**은 `waiting_approval`(레거시 `human_in_loop`)과 `awaiting_approval`(토큰) 둘 다 인라인 키보드로 처리하며, resume 시 run_id+token을 우선한다.

package/docs/TRUST_MODEL.md

@@ -1,6 +1,6 @@
 # Lattice AI Trust Model
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 Lattice AI is local-first, explicit about external communication, and honest
 when a capability is unavailable.

package/docs/WHY_LATTICE.md

@@ -1,6 +1,6 @@
 # Why Lattice AI Exists
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 **Lattice AI is a local-first Digital Brain that keeps your knowledge durable
 across any AI model.**

package/docs/kg-schema.md

@@ -1,6 +1,6 @@
 # Knowledge Graph Schema
 
-Current release: **9.9.4 — Durable Loops**.
+Current release: **9.9.5 — Closed Gaps**.
 
 명세 출처: `lattice_ai_full_spec.pptx` 슬라이드 20·21·22
 구현: `lattice_brain/graph/schema.py`

package/lattice_brain/__init__.py

@@ -26,7 +26,7 @@ from .storage import (
     storage_from_env,
 )
 
-__version__ = "9.9.4"
+__version__ = "9.9.5"
 
 __all__ = [
     "AgentRuntime",

package/lattice_brain/graph/rerank.py

@@ -0,0 +1,160 @@
+"""Optional cross-encoder rerank for hybrid retrieval (v9.9.5).
+
+Default path is identity (fused score preserved) — no model download, no
+latency tax, no unearned claims. Opt in with::
+
+    LATTICEAI_CROSS_ENCODER_RERANK=1
+    # optional model id (sentence-transformers CrossEncoder):
+    LATTICEAI_CROSS_ENCODER_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
+
+When the env kill-switch is off, or ``sentence_transformers`` / the model is
+unavailable, :func:`rerank_matches` returns the candidates unchanged and
+reports ``mode="identity"``. Failures never raise into the search path.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+CROSS_ENCODER_RERANK_ENV = "LATTICEAI_CROSS_ENCODER_RERANK"
+CROSS_ENCODER_MODEL_ENV = "LATTICEAI_CROSS_ENCODER_MODEL"
+DEFAULT_CROSS_ENCODER_MODEL = "cross-encoder/ms-marco-MiniLM-L-6-v2"
+
+_model_cache: Dict[str, Any] = {}
+
+
+def _rerank_enabled() -> bool:
+    raw = os.getenv(CROSS_ENCODER_RERANK_ENV, "").strip().lower()
+    return raw in {"1", "true", "yes", "on"}
+
+
+def _model_id() -> str:
+    raw = os.getenv(CROSS_ENCODER_MODEL_ENV, "").strip()
+    return raw or DEFAULT_CROSS_ENCODER_MODEL
+
+
+def _candidate_text(match: Dict[str, Any]) -> str:
+    parts = [
+        str(match.get("title") or ""),
+        str(match.get("summary") or ""),
+        str((match.get("metadata") or {}).get("snippet") or ""),
+    ]
+    return " ".join(p for p in parts if p).strip() or str(match.get("node_id") or "")
+
+
+def _load_cross_encoder(model_id: str) -> Any:
+    if model_id in _model_cache:
+        return _model_cache[model_id]
+    from sentence_transformers import CrossEncoder  # type: ignore
+
+    model = CrossEncoder(model_id)
+    _model_cache[model_id] = model
+    return model
+
+
+def identity_rerank(
+    query: str,
+    candidates: List[Dict[str, Any]],
+    *,
+    top_k: Optional[int] = None,
+) -> Dict[str, Any]:
+    """Preserve fused ordering; stamp identity scores for an honest contract."""
+    del query  # identity path does not use the query text
+    ranked = list(candidates)
+    for item in ranked:
+        item["rerank_score"] = float(item.get("score") or item.get("fused_score") or 0.0)
+    if top_k is not None:
+        ranked = ranked[: max(1, int(top_k))]
+    return {
+        "matches": ranked,
+        "mode": "identity",
+        "model": None,
+        "detail": None,
+    }
+
+
+def cross_encoder_rerank(
+    query: str,
+    candidates: List[Dict[str, Any]],
+    *,
+    top_k: Optional[int] = None,
+    model_id: Optional[str] = None,
+) -> Dict[str, Any]:
+    """Score (query, candidate) pairs with a CrossEncoder when available."""
+    if not candidates:
+        return {
+            "matches": [],
+            "mode": "cross_encoder",
+            "model": model_id or _model_id(),
+            "detail": None,
+        }
+    mid = model_id or _model_id()
+    try:
+        model = _load_cross_encoder(mid)
+    except Exception as exc:  # noqa: BLE001 — never break search
+        logger.info("cross-encoder unavailable (%s); falling back to identity", exc)
+        result = identity_rerank(query, candidates, top_k=top_k)
+        result["detail"] = f"cross_encoder_unavailable: {exc}"
+        return result
+
+    pairs = [[str(query or ""), _candidate_text(c)] for c in candidates]
+    try:
+        scores = model.predict(pairs)
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("cross-encoder predict failed: %s", exc)
+        result = identity_rerank(query, candidates, top_k=top_k)
+        result["detail"] = f"cross_encoder_predict_failed: {exc}"
+        return result
+
+    ranked = list(candidates)
+    for item, score in zip(ranked, scores):
+        item["rerank_score"] = float(score)
+        # Surface the rerank score as the primary ranking key while keeping
+        # the pre-rerank fused score under scores.fused for audit.
+        scores_map = item.setdefault("scores", {})
+        if isinstance(scores_map, dict):
+            scores_map.setdefault("fused", float(item.get("score") or 0.0))
+            scores_map["rerank"] = float(score)
+        item["score"] = float(score)
+    ranked.sort(key=lambda m: (-float(m.get("rerank_score") or 0.0), str(m.get("node_id") or "")))
+    if top_k is not None:
+        ranked = ranked[: max(1, int(top_k))]
+    for rank, match in enumerate(ranked, start=1):
+        match["rank"] = rank
+    return {
+        "matches": ranked,
+        "mode": "cross_encoder",
+        "model": mid,
+        "detail": None,
+    }
+
+
+def rerank_matches(
+    query: str,
+    candidates: List[Dict[str, Any]],
+    *,
+    top_k: Optional[int] = None,
+    force: Optional[bool] = None,
+) -> Dict[str, Any]:
+    """Public entry: cross-encoder when enabled, else identity.
+
+    ``force=True/False`` overrides the env kill-switch (tests only).
+    """
+    enabled = _rerank_enabled() if force is None else bool(force)
+    if not enabled:
+        return identity_rerank(query, candidates, top_k=top_k)
+    return cross_encoder_rerank(query, candidates, top_k=top_k)
+
+
+__all__ = [
+    "CROSS_ENCODER_MODEL_ENV",
+    "CROSS_ENCODER_RERANK_ENV",
+    "DEFAULT_CROSS_ENCODER_MODEL",
+    "cross_encoder_rerank",
+    "identity_rerank",
+    "rerank_matches",
+]