ltcai 8.8.0 → 9.0.0

package/README.md

@@ -1,6 +1,6 @@
 # Lattice AI
 
-**Lattice AI 8.8.0 is the local-first Digital Brain platform. This release removes internal-only Brain compatibility layers, strengthens recall proof quality, and keeps package, desktop, extension, static app, and documentation metadata synchronized to the 8.8.0 line.**
+**Lattice AI 9.0.0 is the local-first Digital Brain platform. This release closes the July 8 code-review follow-up, hardens chat/runtime reliability, pays down cleanup debt across runtime utilities and setup detection, and keeps package, desktop, extension, static app, and documentation metadata synchronized to the 9.0.0 line.**
 
 **Lattice AI는 모델이 바뀌어도 내 지식과 맥락을 보존하는 로컬 우선 AI 브레인입니다.**
 
@@ -67,7 +67,9 @@ You need Lattice AI when:
 The screenshots below are the latest checked-in visual evidence captures. They
 keep the first-run Brain flow, memory graph, source capture, model library,
 system view, admin console, and review center visible as release gates while
-8.8.0 focuses on Brain Core extraction readiness and recall proof hardening.
+9.0.0 focuses on code-review closure, chat/runtime reliability,
+AgentRuntime and ToolRegistry maintainability, release artifact correctness,
+and frontend/runtime cleanup.
 
 ### 1. Wake Brain
 
@@ -169,7 +171,9 @@ pip install "ltcai[local]"
 - **Backend**: FastAPI on localhost is the UI source of truth.
 - **Brain Core**: independent `lattice_brain` package for graph, memory,
   context, conversations, ingestion, runtime, workflow, storage, and portability.
-- **Storage**: SQLite default; PostgreSQL/pgvector is optional scale mode.
+- **Storage**: SQLite is the live local Brain store; PostgreSQL/pgvector tooling
+  is optional scale-mode planning/migration support, not the default live graph
+  backend.
 - **Portability**: encrypted `.latticebrain` archives plus backup, restore,
   inspect, verify, import dry-run, and confirmed restore/import flows.
 - **Trust boundary**: local-first by default; cloud calls, downloads, Telegram,
@@ -206,38 +210,38 @@ See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for developer workflow details.
 
 ## Current Release
 
-The current release is **8.8.0 — Brain Core Extraction & Recall Proof Hardening**:
-
-- Internal-only Brain shim layers are removed: pre-graph flat modules,
-  the deprecated `latticeai.brain` namespace, and the service-layer
-  AgentRuntime alias are tracked as intentional 8.8.0 removals.
-- Root compatibility shims remain managed for external entrypoints while Brain
-  Core now has one physical import surface guarded by isolation tests.
-- AgentRuntime rejects unknown roles at the boundary, exposes contract envelopes
-  for legacy run rows, and persists clamped retry budgets.
-- Brain Chat adds resumable conversation history, stop/regenerate/copy actions,
-  richer ingestion panels, and answer proof explainability with matched terms
-  and confidence labels.
-- Memory recall drops zero-evidence noise when lexical evidence exists, so
-  citations explain why each source was selected.
-
-Expected artifacts for 8.8.0 release must use exact filenames:
-
-- `dist/ltcai-8.8.0-py3-none-any.whl`
-- `dist/ltcai-8.8.0.tar.gz`
-- `ltcai-8.8.0.tgz`
-- `dist/ltcai-8.8.0.vsix`
-- `src-tauri/target/release/bundle/dmg/Lattice AI_8.8.0_aarch64.dmg`
+The current release is **9.0.0 — Code Review Closure & Runtime Cleanup**:
+
+- July 8 code-review follow-ups are fixed across chat, agent runs, permissions,
+  Telegram integration, runtime audit timestamps, embeddings, and Brain UI copy.
+- Chat/document streaming now preserves terminal SSE events and history/trace
+  persistence when generation fails mid-stream.
+- Runtime audit appends use JSONL while legacy JSON reads remain compatible.
+- Shared JSON/ISO/hash, setup detection, and frontend helper utilities replace
+  duplicated implementations across runtime, Brain Core, setup, and React code.
+- The legacy `server_app` namespace is allowlist-based instead of exposing all
+  app-factory locals.
+- The main `/chat` handler now routes through focused intent handlers and shared
+  response epilogues for lower future regression risk.
+
+Expected artifacts for 9.0.0 release must use exact filenames:
+
+- `dist/ltcai-9.0.0-py3-none-any.whl`
+- `dist/ltcai-9.0.0.tar.gz`
+- `ltcai-9.0.0.tgz`
+- `dist/ltcai-9.0.0.vsix`
+- `src-tauri/target/release/bundle/dmg/Lattice AI_9.0.0_aarch64.dmg`
 
 Do not use wildcard artifact uploads. Package registry publishing remains owner-run.
 
 See [docs/ROADMAP_RECOMMENDATIONS.md](docs/ROADMAP_RECOMMENDATIONS.md) for the
-strategic roadmap slices applied through 8.8.0 and the follow-up tracks.
+strategic roadmap slices applied through 9.0.0 and the follow-up tracks.
 
 ## Known Limitations
 
 - External package registries are owner-published and can lag behind GitHub.
-- PostgreSQL/pgvector is optional scale mode. SQLite is the default local Brain.
+- PostgreSQL/pgvector is optional scale/migration tooling. SQLite remains the
+  live local Brain store in 9.0.0.
 - Docker, model downloads, cloud model calls, Telegram, Brain Network, and update
   checks require explicit user action.
 - Conversation does not fabricate answers when no model is loaded.
@@ -249,6 +253,8 @@ strategic roadmap slices applied through 8.8.0 and the follow-up tracks.
 
 | Version | Theme |
 | --- | --- |
+| 9.0.0 | Code Review Closure & Runtime Cleanup: July 8 code-review follow-ups fixed, chat/runtime reliability improved, duplicated utility surfaces consolidated, runtime audit append paths moved to JSONL, and release metadata/artifacts synchronized |
+| 8.9.0 | Scoped Memory & Tool Policy Hardening: authenticated history/KG reads are workspace-scoped, direct Tool API paths enforce registry policy, local approvals hash tokens at rest, AgentRuntime approval semantics are explicit, and frontend/runtime seams are split |
 | 8.8.0 | Brain Core Extraction & Recall Proof Hardening: internal-only Brain shim layers are removed, AgentRuntime run contracts/retry budgets are tighter, Brain Chat gains conversation controls, and citation recall exposes matched evidence |
 | 8.7.0 | Runtime State Hygiene & Release Evidence Refresh: model-runtime internals prefer typed state over legacy globals, compatibility sync is deprecated, 8.7.0 visual evidence is refreshed, and all release metadata/docs are synchronized |
 | 8.6.0 | Desktop Capture & Navigation Reliability: native folder selection works from the Tauri localhost app, picker failures surface in Capture, web saving remains one-action, and the Brain shell sidebar/admin flow is CI-covered |
@@ -258,16 +264,6 @@ strategic roadmap slices applied through 8.8.0 and the follow-up tracks.
 | 8.2.0 | Brain Brief: evidence-backed home briefing, honest empty-state guidance, recall/graph/model-proof next actions, and continued model/workspace runtime extraction |
 | 8.1.0 | Intuitive Brain Home: living Brain, recent memory, connected topic, next action, and composer are visible in one product-first screen with refreshed 8.1.0 evidence and artifacts |
 | 8.0.0 | Runtime Architecture Contract: AgentRuntime, ToolRegistry, central Config, server decomposition, and KG hardening are captured as machine-checkable release boundaries with exact 8.0.0 artifacts |
-| 7.9.0 | Agent Runtime Boundary Hardening: explicit `SingleAgentRuntime`, compatibility alias preservation, injected rollback port, and release/readiness docs aligned to the product AgentRuntime facade |
-| 7.8.0 | Brain Chat Home UX Simplification: chat-first first viewport, visible workspace navigation, collapsed source/status utilities, hidden default depth controls, and removal of obsolete Brain UX components |
-| 7.7.0 | Complete Product Polish: command-center Brain Home, repaired Review Center evidence, exact 7.7 docs/artifacts, product readiness gate, and stronger CI/release checks |
-| 7.6.0 | Brain-Centered UX & Architecture Closure: Wake Brain first-run surface, concentric memory rings with direct depth controls, and machine-checkable closure of the two architecture/UX review files |
-| 7.5.0 | Runtime Debt Burn-down & Release Risk Cleanup: API consumers get normalized contract views, retrieval quality uses a 250+ record corpus fixture, stale artifact risk is removed, npm audit is clean, and Tauri is updated past the old block warning |
-| 7.4.0 | Runtime Contract Convergence & Corpus Retrieval: agent/workflow/audit/realtime records share the agent-run-contract/v1 family, and retrieval quality gates run against a real corpus-scale fixture |
-| 7.3.0 | Runtime Contract & Retrieval Quality: shared agent-run contract across runtimes plus deterministic hybrid recall/ranking regression gates |
-| 7.2.0 | Runtime Trust Baseline: agent run preview/readiness, simulation-mode guardrails, live ToolRegistry manifest/diagnostics, and tests for dispatch/governance/catalog drift |
-| 7.1.0 | Brain Usability Completion: clearer first-run onboarding, ingestion progress/emergence, richer graph controls, inline answer proof, workspace/profile/admin discovery, empty/error/consent feedback, and VS Code sync status |
-| 7.0.0 | Brain Productization Loop: first-screen ingestion for files/folders/notes/web, answer-level memory proof and source citations, model-continuity demo flow, five-minute first-run loop, and recall/KG quality eval in CI |
 
 ## Documentation
 

package/auto_setup.py

@@ -25,7 +25,7 @@ Lattice AI — Zero-Config Auto Setup
 python3 auto_setup.py probe                  # 1단계만
 python3 auto_setup.py recommend              # 1+2단계
 python3 auto_setup.py plan                   # 1+2+3 (설치 계획 출력)
-python3 auto_setup.py plan --apply           # 실제 설치 실행 (위험)
+python3 auto_setup.py plan --apply --confirm-token <token>
 python3 auto_setup.py verify                 # 4단계 단독
 python3 auto_setup.py preset                 # 5단계
 python3 auto_setup.py all                    # 전체 흐름
@@ -47,6 +47,20 @@ from dataclasses import asdict, dataclass, field
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Tuple
 
+from latticeai.services.process_audit import (
+    CommandConfirmationError,
+    append_process_audit_event,
+    command_plan,
+    command_plan_for_commands,
+    require_command_confirmation,
+)
+from latticeai.services.setup_detection import (
+    detect_cuda,
+    detect_tools,
+    detect_wsl_from_text,
+    parse_windows_video_controllers as _parse_windows_video_controllers,
+)
+
 __all__ = [
     "SystemProfile", "Recommendation", "InstallPlan",
     "probe", "recommend", "plan", "verify", "preset", "run_all",
@@ -145,44 +159,6 @@ def _which(binary: str) -> Optional[str]:
     return None
 
 
-def _parse_windows_video_controllers(raw: str) -> List[Dict[str, Any]]:
-    controllers: List[Dict[str, Any]] = []
-    if not raw:
-        return controllers
-    try:
-        data = json.loads(raw)
-        if isinstance(data, dict):
-            data = [data]
-        if isinstance(data, list):
-            for item in data:
-                name = str(item.get("Name") or "").strip()
-                if not name:
-                    continue
-                try:
-                    ram_mb = int(item.get("AdapterRAM") or 0) // (1024 * 1024)
-                except Exception:
-                    ram_mb = 0
-                controllers.append({"name": name, "vram_mb": ram_mb})
-        if controllers:
-            return controllers
-    except Exception:
-        pass
-    current: Dict[str, Any] = {}
-    for line in raw.splitlines():
-        if line.startswith("Name="):
-            if current:
-                controllers.append(current)
-            current = {"name": line.split("=", 1)[-1].strip(), "vram_mb": 0}
-        elif line.startswith("AdapterRAM=") and current:
-            try:
-                current["vram_mb"] = int(line.split("=", 1)[-1].strip()) // (1024 * 1024)
-            except ValueError:
-                current["vram_mb"] = 0
-    if current:
-        controllers.append(current)
-    return controllers
-
-
 def _detect_gpu(prof_os: str, arch: str) -> GPUInfo:
     """OS별 휴리스틱으로 GPU 감지. 외부 라이브러리 없이 가능한 만큼만."""
     gpu = GPUInfo()
@@ -268,36 +244,17 @@ def _detect_package_manager(prof_os: str) -> Optional[str]:
 
 
 def _detect_tools() -> Dict[str, str]:
-    tools: Dict[str, str] = {}
-    for binary in ("ollama", "lms", "nvidia-smi", "nvcc", "winget", "brew", "apt", "git", "node", "python", "python3"):
-        found = _which(binary)
-        if found:
-            tools[binary] = found
-    return tools
+    detected = detect_tools(_which, ("ollama", "lms", "nvidia-smi", "nvcc", "winget", "brew", "apt", "git", "node", "python", "python3"))
+    return {binary: path for binary, path in detected.items() if path}
 
 
 def _detect_wsl(prof_os: str) -> Tuple[bool, str]:
-    if prof_os != "linux":
-        return False, ""
-    raw = _read_text("/proc/version")
-    is_wsl = "microsoft" in raw.lower() or "wsl" in raw.lower()
-    version = "2" if "microsoft-standard" in raw.lower() or "wsl2" in raw.lower() else ("1" if is_wsl else "")
-    return is_wsl, version
+    return detect_wsl_from_text(prof_os, _read_text("/proc/version"))
 
 
 def _detect_cuda() -> Tuple[bool, str]:
-    nvidia_smi = _which("nvidia-smi")
-    nvcc = _which("nvcc")
-    version = ""
-    if nvidia_smi:
-        raw = _run([nvidia_smi, "--query-gpu=driver_version", "--format=csv,noheader"], timeout=4.0)
-        version = raw.splitlines()[0].strip() if raw.splitlines() else ""
-    if nvcc:
-        raw = _run([nvcc, "--version"], timeout=4.0)
-        m = re.search(r"release\s+([\d.]+)", raw)
-        if m:
-            version = m.group(1)
-    return bool(nvidia_smi or nvcc), version
+    available, version, _nvidia_smi, _nvcc = detect_cuda(_which, lambda args: _run(args, timeout=4.0))
+    return available, version
 
 
 def _detect_cpu_details(prof_os: str) -> Tuple[str, int, int, List[str]]:
@@ -547,6 +504,21 @@ class InstallStep:
     command: List[str]
     requires_admin: bool = False
 
+    def to_json(self) -> Dict[str, Any]:
+        return {
+            "name": self.name,
+            "why": self.why,
+            "command": self.command,
+            "requires_admin": self.requires_admin,
+            "command_plan": command_plan(
+                self.command,
+                name=self.name,
+                purpose="auto_setup_install",
+                requires_admin=self.requires_admin,
+                metadata={"why": self.why},
+            ),
+        }
+
 
 @dataclass
 class InstallPlan:
@@ -555,10 +527,18 @@ class InstallPlan:
     notes: List[str] = field(default_factory=list)
 
     def to_json(self) -> Dict[str, Any]:
+        plan_summary = command_plan_for_commands(
+            [step.command for step in self.steps],
+            name="auto_setup_plan",
+            purpose="auto_setup_install",
+            metadata={"package_manager": self.package_manager},
+        )
         return {
             "package_manager": self.package_manager,
-            "steps": [asdict(s) for s in self.steps],
+            "steps": [s.to_json() for s in self.steps],
             "notes": self.notes,
+            "command_plan": plan_summary,
+            "confirmation_token": plan_summary["confirmation_token"],
         }
 
 
@@ -661,23 +641,55 @@ def plan(profile: SystemProfile, rec: Recommendation) -> InstallPlan:
     return InstallPlan(package_manager=pm, steps=steps, notes=notes)
 
 
-def apply_plan(plan_obj: InstallPlan, *, confirm: bool = False) -> List[Dict[str, Any]]:
-    """위험: 실제로 설치 명령을 실행한다. ``confirm=True`` 필수."""
+def apply_plan(
+    plan_obj: InstallPlan,
+    *,
+    confirm: bool = False,
+    confirmation_token: Optional[str] = None,
+) -> List[Dict[str, Any]]:
+    """위험: 실제로 설치 명령을 실행한다. ``confirm`` and token are required."""
     if not confirm:
         raise RuntimeError("refuse to apply: pass confirm=True")
+    expected = plan_obj.to_json().get("confirmation_token")
+    if str(confirmation_token or "").strip() != expected:
+        raise CommandConfirmationError("refuse to apply: confirmation token mismatch")
     results: List[Dict[str, Any]] = []
     for step in plan_obj.steps:
+        step_plan = command_plan(
+            step.command,
+            name=step.name,
+            purpose="auto_setup_install",
+            requires_admin=step.requires_admin,
+            metadata={"why": step.why},
+        )
         try:
+            require_command_confirmation(
+                step.command,
+                step_plan["confirmation_token"],
+                purpose="auto_setup_install",
+            )
+            append_process_audit_event("installer_command", plan=step_plan, status="started")
             r = subprocess.run(step.command, capture_output=True, text=True,
                                timeout=300, check=False)
+            append_process_audit_event(
+                "installer_command",
+                plan=step_plan,
+                status="finished",
+                returncode=r.returncode,
+                stdout=r.stdout,
+                stderr=r.stderr,
+            )
             results.append({
                 "name": step.name,
+                "command_hash": step_plan["command_hash"],
+                "command_preview": step_plan["command_preview"],
                 "returncode": r.returncode,
                 "stdout_tail": (r.stdout or "")[-2000:],
                 "stderr_tail": (r.stderr or "")[-2000:],
             })
         except Exception as exc:
-            results.append({"name": step.name, "error": str(exc)})
+            append_process_audit_event("installer_command", plan=step_plan, status="error", error=str(exc))
+            results.append({"name": step.name, "command_hash": step_plan["command_hash"], "error": str(exc)})
     return results
 
 
@@ -780,13 +792,13 @@ def preset(profile: SystemProfile, rec: Recommendation) -> Dict[str, Any]:
 
 
 # ── orchestrator ────────────────────────────────────────────────────────────
-def run_all(*, apply_install: bool = False) -> Dict[str, Any]:
+def run_all(*, apply_install: bool = False, confirmation_token: Optional[str] = None) -> Dict[str, Any]:
     p = probe()
     r = recommend(p)
     pl = plan(p, r)
     install_results = None
     if apply_install:
-        install_results = apply_plan(pl, confirm=True)
+        install_results = apply_plan(pl, confirm=True, confirmation_token=confirmation_token)
     v = verify(p, r)
     ps = preset(p, r)
     return {
@@ -809,6 +821,8 @@ def _main() -> int:
     sp_plan = sub.add_parser("plan")
     sp_plan.add_argument("--apply", action="store_true",
                          help="actually run the install commands (DANGER)")
+    sp_plan.add_argument("--confirm-token", default=None,
+                         help="confirmation token from the dry-run plan output")
     sub.add_parser("verify")
     sub.add_parser("preset")
     sub.add_parser("all")
@@ -825,7 +839,7 @@ def _main() -> int:
         p = probe(); r = recommend(p); pl = plan(p, r)
         out: Dict[str, Any] = {"plan": pl.to_json()}
         if args.apply:
-            out["install"] = apply_plan(pl, confirm=True)
+            out["install"] = apply_plan(pl, confirm=True, confirmation_token=args.confirm_token)
         print(json.dumps(out, indent=2, ensure_ascii=False)); return 0
     if args.cmd == "verify":
         p = probe(); r = recommend(p)