ltcai 0.3.0 → 0.3.2

package/README.md

@@ -21,15 +21,55 @@
 
 ---
 
+## What's new in 0.3.2
+
+- **Consistent "current model"** — model-card click → prepare/load → smoke test →
+  `current` update → chat-readiness flows through one path in the web UI, so the
+  model you see is always the model chat uses.
+- **3-way smoke status** — load-time chat probe now reports `ok` / `degraded` /
+  `failed` (special-token leakage, runaway repetition, over-long output), and the
+  UI shows a compatibility warning for `degraded` while still allowing chat.
+- **Timezone-correct security dashboard** — "events today" now uses the same
+  timezone as audit timestamps (configurable via `LATTICE_TZ`, e.g. `Asia/Seoul`),
+  fixing off-by-one day counts.
+- **Cleaner auto-graph** — Korean particle stripping, generic-word / file-extension
+  blacklists, and a single-source penalty so only concepts repeated across multiple
+  sources get promoted to nodes.
+- **Honest docs** — toned down unverifiable claims (telemetry, skill/plugin counts).
+
+## What's new in 0.3.1
+
+- **Reliable model selection** — `ModelResolution` unifies recommended card ID,
+  download ID, load ID, router cache key, and the front-end `current` so
+  "downloaded but not loaded" / "loaded but UI shows a different model"
+  classes of bugs are gone.
+- **Smoke test on load** — every local model load runs a one-shot Korean
+  chat probe and surfaces `ready_to_chat` / `compatibility_status` to the UI.
+- **Model Compatibility Layer** — per-family profiles (GPT-OSS, Gemma, Qwen,
+  Llama, Mistral, Phi, Deepseek …) with cached stop tokens, postprocess
+  rules, and Fast / Slow / Recovery paths so chat speed stays the same.
+- **Auto graph curator** — topic extraction → alias clustering → promotion
+  with secret/PII firewall, so the graph builds itself without the user
+  managing nodes.
+- **AI Security & Audit Command Center** — admin dashboard now shows
+  per-user risk matrix (compliant chats vs risky chats vs compliant files
+  vs risky files), sensitive-type donut, drill-down, raw explorer, and
+  JSON / CSV / XLSX / PDF exports — with hard-secret redaction
+  enforced on every response.
+
+See [docs/CHANGELOG.md](./docs/CHANGELOG.md) for the full list.
+
+---
+
 ## Why Lattice AI?
 
 Most AI tools forget everything after each conversation. Your files sit in folders, your chats vanish, and nothing connects.
 
-**Lattice AI remembers.** It reads your local files, indexes your conversations, and builds a knowledge graph that links people, projects, concepts, and documents — all on your machine, with zero data leaving your PC.
+**Lattice AI remembers.** It reads your local files, indexes your conversations, and builds a knowledge graph that links people, projects, concepts, and documents — all on your machine. With local models, nothing leaves your PC; cloud models are opt-in and clearly labeled.
 
 - **Your data stays local** — everything lives in `~/.ltcai/`, never sent to external servers
 - **Your AI gets smarter over time** — every chat and file builds your personal knowledge graph
-- **One install, works everywhere** — web UI, VS Code, Telegram, MCP clients, all connected to the same brain
+- **One local server, many surfaces** — a single local server powers the web UI, VS Code extension, and optional integrations (Telegram, MCP)
 
 ---
 
@@ -140,7 +180,7 @@ Based on public product behavior as of 2026-05.
 | Telegram bot | **Yes** | No | No | No |
 | MCP registry (one-click install) | **Yes** | Partial | Yes | No |
 | Admin + audit log | **Yes** | Yes | No | No |
-| Zero telemetry, self-hosted | **Yes** | Yes | Yes | No |
+| No built-in telemetry, self-hosted | **Yes** | Yes | Yes | No |
 | One-command public tunnel | **Yes** | No | No | No |
 | Free | **Yes** | Yes | Yes | No |
 
@@ -178,7 +218,7 @@ The setup wizard auto-detects your hardware and recommends the best model for yo
 | | |
 |---|---|
 | **Storage** | All data in `~/.ltcai/` on your machine |
-| **Telemetry** | None — no analytics, no tracking, no phoning home |
+| **Telemetry** | No built-in analytics or product telemetry by default |
 | **File access** | Approval-token gated — explicit consent per folder |
 | **Cloud models** | When using cloud APIs, prompts are sent to the provider. Local models keep everything offline. |
 | **Sensitive files** | `.env`, credentials, keys, certificates auto-excluded from indexing |
@@ -205,8 +245,8 @@ The setup wizard auto-detects your hardware and recommends the best model for yo
 | **Multi-LLM pipeline** | Plan → Execute → Review with different models |
 | **MCP server** | Use Lattice tools in Claude Desktop / Cursor |
 | **MCP registry** | One-click install from registry.modelcontextprotocol.io |
-| **Skills marketplace** | 77 official skills (Anthropic + verified third-party) |
-| **Plugin directory** | Browse 149 open-source plugins |
+| **Skills browser** | Optional browser for Anthropic + third-party skills |
+| **Plugin browser** | Browse open-source plugins from the registry |
 
 ### Access & Communication
 | Feature | Description |
@@ -363,7 +403,7 @@ Full reference: [docs/mcp-tools.md](docs/mcp-tools.md)
 | VS Code Marketplace | [marketplace.visualstudio.com](https://marketplace.visualstudio.com/items?itemName=parktaesoo.ltcai) |
 | Open VSX | [open-vsx.org](https://open-vsx.org/extension/parktaesoo/ltcai) |
 
-Current version: **0.2.2** — [Changelog](docs/CHANGELOG.md)
+Current version: **0.3.2** — [Changelog](docs/CHANGELOG.md)
 
 ---
 

package/docs/CHANGELOG.md

@@ -1,5 +1,122 @@
 # Changelog
 
+## [0.3.2] - 2026-05-29
+
+> 안정화 릴리스 — 모델 current 일관성, smoke test 3분류, 보안 대시보드 timezone
+> 버그 수정, 자동 그래프 한국어 노이즈 개선, README 과장 표현 정리.
+
+### Model loading & UI
+
+- 웹 UI 모델 선택을 단일 흐름으로 통일(`selectModelByCard` → `prepareAndLoadModel`
+  → smoke test → `current` 반영 → 채팅 가능 여부 표시). cloud(`loadSelectedModel`)
+  경로도 백엔드 `current`를 단일 진실원으로 사용. "보이는 모델 ≠ 채팅에 쓰이는
+  모델" 문제 제거.
+- Smoke test 결과를 **ok / degraded / failed** 3분류로 확장
+  (`model_compat.classify_smoke_response()`). 특수/role 토큰 누출, 폭주 반복,
+  과도한 길이를 감지. `degraded`는 채팅은 가능하되 UI에 호환성 경고 표시.
+  `/models/load`·`/engines/prepare-model/stream` 응답의 `compatibility_status`가
+  3분류 값을 그대로 노출.
+
+### Security dashboard
+
+- **Timezone 버그 수정** — audit timestamp는 로컬 시간으로 기록되는데
+  "events_today"는 UTC로 계산해 한국 사용자에게 날짜가 어긋나던 문제 수정.
+  새 모듈 `latticeai/core/timezones.py`로 기준 시간대를 통일(`LATTICE_TZ` /
+  `LTCAI_TZ` 환경변수, 기본 시스템 로컬). overview 응답에 `timezone` 필드 추가.
+
+### Auto graph curator
+
+- 한국어 노이즈 감소 — 조사 제거, 일반어/파일확장자 blacklist, 단일 출처
+  후보 score 감점(여러 출처에서 반복된 개념만 승격).
+
+### Docs & tests
+
+- README/확장 설명의 과장 표현 완화(telemetry, skills/plugins 수치 등).
+- 단위 테스트 추가: timezone, smoke 3분류, graph 노이즈, export secret redaction.
+  (tests/unit 149 passed)
+
+## [0.3.1] - 2026-05-29
+
+> Model loading reliability + auto-graph curation + AI Security & Audit Command Center.
+>
+> 외부 리뷰 5건(모델 추천/다운로드, 사용자 직접 모델 선택, 모델 호환성 계층,
+> 자동 그래프 방향, 관리자 보안/감사 대시보드) 피드백을 모두 반영했다.
+
+### Model loading & inference
+
+- 새 모듈 `latticeai/core/model_resolution.py` — `ModelResolution`이
+  `input_id / engine / resolved_model / download_id / load_id / expected_current`을
+  하나로 묶어 추천 카드, 다운로드, 로드, router cache, 프론트 current 표시가
+  단계마다 어긋나는 문제를 제거.
+- `prepare_and_load_model()` 와 `/engines/prepare-model/stream`이 동일한
+  `ModelResolution`을 공유하도록 통합. LM Studio처럼 `instance_id`가 부여되는
+  엔진은 `resolution.update_after_load()`로 후처리.
+- 로드 직후 `_smoke_test_loaded_model()`가 한국어 짧은 채팅 테스트를 실행 →
+  응답에 `ready_to_chat`, `compatibility_status`, `smoke_test` 필드 추가.
+  Cloud 모델은 사용자 비용 발생을 피하기 위해 자동 skip.
+- `/models` 응답에 `engine_options`(local_mlx / ollama / lmstudio / llamacpp /
+  vllm 별 실제 model_id)와 `compat_profiles` 추가.
+- 새 엔드포인트 `GET /models/compat-profiles`.
+
+### Model compatibility layer
+
+- 새 모듈 `latticeai/core/model_compat.py` — Family detection
+  (gpt-oss / gemma / qwen / llama / mistral / phi / deepseek …),
+  family 프로파일(stop tokens, disable_draft, postprocess, generation params),
+  `fast_postprocess`, `validate_smoke_response`, `record_smoke_result`,
+  `compat_cache`. 무거운 검사는 모델 로드 시 1회(Slow Path), 채팅 중에는
+  캐시된 profile만 사용하는 Fast Path. 답변이 깨졌을 때만 1회 retry하는
+  Recovery Path 구조.
+
+### Auto knowledge graph curation
+
+- 새 모듈 `latticeai/core/graph_curator.py` — 대화/파일/작업 로그에서
+  Topic candidate 추출 → alias clustering(자동 병합) → promotion 결정
+  (secret 차단, 중복 차단, 출처 최소치) → 파생 이야기 엣지 → 행동 시그널
+  기반 큐레이션. Secret/API key/private key는 그래프 후보에서 자동 제거.
+
+### Frontend — user-trusted current model
+
+- `static/scripts/chat.js`의 `prepareAndLoadModel` 결과에서 백엔드
+  `response.current`를 신뢰하고, `ready_to_chat=false` 또는
+  `compatibility_status=degraded`일 때 사용자에게 호환성 경고 표시.
+- 모델 카드를 직접 클릭할 때도 같은 표준 흐름을 타는
+  `window.selectModelByCard()` 헬퍼 추가.
+
+### Admin — AI Security & Audit Command Center
+
+- 새 라우터 `latticeai/api/security_dashboard.py`가 11개 엔드포인트 추가:
+  `/admin/security/{overview,users,events,events/{id},conversations/{id},`
+  `conversations/{id}/raw,files,files/{id},files/{id}/content,raw,export}`.
+- 모든 응답에서 hard secret(`sk-…`, `ghp_…`, `xoxb-…`, `AKIA…`,
+  private key block 등)을 자동 redact. 원문/raw 조회는 별도
+  `admin_view_sensitive_raw` 감사 이벤트로 기록.
+- 관리자 UI: Security Overview 카드(오늘 이벤트, High Risk, 위험 채팅/파일,
+  Secret/외부 전송 차단, 관리자 원문 조회 수, 검토 필요), User Risk Matrix
+  (stacked bar), 민감정보 유형 donut chart, 민감 채팅/위험 파일 모니터,
+  감사 타임라인, Raw Data Explorer.
+- 사용자별 막대 클릭 → drill-down. JSON / CSV / XLSX / PDF / TXT
+  추출 지원.
+
+### Tests / CI
+
+- 새 단위 테스트 28개 — `tests/unit/test_model_compat.py`,
+  `tests/unit/test_model_resolution.py`, `tests/unit/test_graph_curator.py`,
+  `tests/unit/test_security_dashboard.py`.
+- `.github/workflows/ci.yml` syntax-check 단계에 4개 새 모듈 추가.
+- 새 `.github/workflows/release.yml` — tag `v*` 푸시 시 PyPI / npm /
+  VS Code Marketplace / Open VSX 자동 배포(필요 secrets: `PYPI_TOKEN`,
+  `NPM_TOKEN`, `VSCE_PAT`, `OVSX_TOKEN`). 해당 secret이 비어 있는 job은
+  자동 skip.
+
+### Fixed
+
+- FastAPI에서 `Request` 인자에 `= None` 디폴트 사용 시 발생하던 잠재 문제 수정
+  (`security_dashboard.py` `/admin/security/raw`).
+- `gpt-oss` family postprocess 순서를
+  `trim_after_user_marker → strip_role_tokens`로 보정 — `<|user|>` 마커가
+  먼저 제거돼 trim이 동작하지 않던 버그.
+
 ## [0.3.0] - 2026-05-27
 
 ### Knowledge Graph — LLM Structured Output Extraction

package/knowledge_graph_api.py

@@ -38,14 +38,22 @@ def create_knowledge_graph_router(
         """Serve the interactive knowledge graph canvas UI."""
         graph()
         require_user(request)
-        return FileResponse(static_dir / "graph.html")
+        response = FileResponse(static_dir / "graph.html")
+        response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
+        response.headers["Pragma"] = "no-cache"
+        response.headers["Expires"] = "0"
+        return response
 
     @router.get("/knowledge-graph")
     async def knowledge_graph_legacy_page(request: Request):
         """Backward-compatible route for the graph page."""
         graph()
         require_user(request)
-        return FileResponse(static_dir / "graph.html")
+        response = FileResponse(static_dir / "graph.html")
+        response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
+        response.headers["Pragma"] = "no-cache"
+        response.headers["Expires"] = "0"
+        return response
 
     @router.get("/knowledge-graph/stats")
     async def knowledge_graph_stats(request: Request):