ltcai 2.2.7 → 3.0.1

package/README.md

@@ -3,10 +3,11 @@
 
   # Lattice AI
 
-  **Local-first AI workspace for knowledge graphs, AI pipelines, and multi-agent coding workflows.**
+  **Lattice AI v3 — Local-First AI Workspace Platform.**
 
-  Plan, execute, review, and remember work across local models, cloud models,
-  files, and team workflows.
+  Work across Personal and Organization workspaces with Knowledge Graph,
+  Vector Index, Hybrid Search, Native Chat, agents, files, models, and
+  Basic / Advanced / Admin modes.
 </div>
 
 <div align="center">
@@ -15,7 +16,7 @@
 [![npm version](https://img.shields.io/npm/v/ltcai?label=npm)](https://www.npmjs.com/package/ltcai)
 [![VS Code Marketplace](https://vsmarketplacebadges.dev/version-short/parktaesoo.ltcai.svg)](https://marketplace.visualstudio.com/items?itemName=parktaesoo.ltcai)
 [![Open VSX](https://img.shields.io/open-vsx/v/parktaesoo/ltcai?label=Open%20VSX)](https://open-vsx.org/extension/parktaesoo/ltcai)
-[![GitHub release](https://img.shields.io/github/v/release/TaeSooPark-PTS/LatticeAI?label=GitHub%20release)](https://github.com/TaeSooPark-PTS/LatticeAI/releases/tag/v2.2.7)
+[![GitHub release](https://img.shields.io/github/v/release/TaeSooPark-PTS/LatticeAI?label=GitHub%20release)](https://github.com/TaeSooPark-PTS/LatticeAI/releases/tag/v3.0.0)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
 [![VS Code extension](https://img.shields.io/badge/VS%20Code-extension-blue?logo=visualstudiocode)](https://marketplace.visualstudio.com/items?itemName=parktaesoo.ltcai)
@@ -48,7 +49,7 @@ Install the coding extension:
 
 - [VS Code Marketplace: parktaesoo.ltcai](https://marketplace.visualstudio.com/items?itemName=parktaesoo.ltcai)
 - [Open VSX: parktaesoo.ltcai](https://open-vsx.org/extension/parktaesoo/ltcai)
-- [GitHub Release v2.2.7](https://github.com/TaeSooPark-PTS/LatticeAI/releases/tag/v2.2.7)
+- [GitHub Release v3.0.0](https://github.com/TaeSooPark-PTS/LatticeAI/releases/tag/v3.0.0)
 
 ## Quick Start
 
@@ -61,7 +62,7 @@ LTCAI
 Then open:
 
 ```text
-http://127.0.0.1:4825
+http://127.0.0.1:4825/app
 ```
 
 Development checkout:
@@ -81,9 +82,14 @@ npm run build
 
 ## What Is Lattice AI?
 
-Lattice AI is a local-first AI workspace for people and teams who want their
-files, models, graph context, and agent workflows in one place.
+Lattice AI v3 is a local-first AI workspace platform for people and teams who
+want their files, models, graph context, retrieval, and agent workflows in one
+place.
 
+- **Primary app shell**: `/app` is the default product experience with Native
+  Chat, Knowledge Graph, Hybrid Search, Files, Pipeline, Agents, Models, My
+  Computer, Settings, and Admin areas. Legacy `/chat` remains available as a
+  rollback/debug path.
 - **Local-first AI Workspace**: work starts on your machine, with local data and
   workspace state by default.
 - **AI Pipeline Platform**: plan, execute, review, retry, and replay work across
@@ -94,7 +100,9 @@ files, models, graph context, and agent workflows in one place.
 - **Multi-Agent Workflow Platform**: agents hand off structured context, review
   work, retry with reasons, and keep timelines inspectable.
 - **Personal / Organization Workspace**: move between personal work and team
-  workspaces with role-aware views.
+  workspaces with role-aware views and Basic / Advanced / Admin modes.
+- **Vector Index and Hybrid Search**: local vector rows are derived from the
+  Knowledge Graph and fused with keyword and graph signals.
 - **Local Model Management**: choose current multimodal local models with source
   disclosure, hardware-aware recommendations, and cloud fallback options.
 - **SSO for teams**: organization workspaces can be paired with Okta or
@@ -114,28 +122,23 @@ and disconnected automations. Lattice AI keeps those parts together:
 - multi-agent workflows leave behind replayable plans, reviews, retries, and
   outcomes.
 
-## v2.2.7 Visual Stabilization Highlights
-
-Lattice AI v2.2.7 is a visual stabilization release driven by rendered browser
-screens, not stylesheet cleanup alone. It keeps the existing design-token
-structure while making the app feel like one cohesive product.
-
-- The dark chat composer is now an intentional dark surface with a crisp outer
-  focus state, no white haze, and no legacy inner textarea box.
-- Knowledge Graph canvas and Workspace OS panels no longer fall back to
-  washed-out light surfaces in dark mode.
-- Workspace select, onboarding, recommendation, auto setup, mode select,
-  pipeline, My Computer, profile, settings, VPC, and model-state surfaces use a
-  shared modal/panel language.
-- Static frontend asset cache-busting is normalized to `?v=2.2.7`, including
-  `/static/scripts/chat.js?v=2.2.7`.
-- Playwright visual coverage now locks the v2.2.7 composer, graph canvas,
-  Workspace OS dark inputs/cards, and mobile composer behavior.
-
-Carried forward from v2.2.x: mobile-first responsive layout, design-token
-light/dark themes, keyboard-safe chat composer, Knowledge Graph responsive UX,
-Admin mobile card layout, drag-and-drop file attachment, and model-card source
-disclosure.
+## v3.0.0 Highlights
+
+Lattice AI v3.0.0 makes `/app` the primary workspace shell and ships the v3
+backend retrieval stack together with the native frontend.
+
+- Native v3 Chat lives inside `/app#/chat` and streams through the real
+  `POST /chat` backend while showing friendly setup guidance when no model is
+  loaded.
+- Knowledge Graph, Vector Index, and Hybrid Search are first-class retrieval
+  surfaces. Hybrid results show keyword, local vector, and graph scores.
+- Personal and Organization workspaces, plus Basic / Advanced / Admin modes,
+  are built into the shell.
+- Legacy `/chat` remains reachable for rollback and debugging.
+- The default embedding signal is `lattice-local-hash-v1`, a deterministic
+  local fallback. It is not described as a production semantic embedding model;
+  future providers may include Ollama, MLX, OpenAI-compatible endpoints, and
+  other local embedding runtimes.
 
 ## Screenshots
 
@@ -175,6 +178,32 @@ files / documents / images / screenshots / conversations / decisions
 
 The graph keeps useful workspace context available even when you change models.
 
+## v3 Backend Retrieval
+
+The v3 backend adds a local-first retrieval stack that combines the Knowledge
+Graph, a SQLite vector index, and hybrid result fusion. It preserves existing
+graph data while adding derived vector rows that can be rebuilt at any time.
+
+Embedding status: the current default is `lattice-local-hash-v1`, a
+deterministic local fallback embedder for indexing and tests. It provides a
+stable vector signal without downloads or cloud calls; it is not a production
+semantic embedding model. Future provider support may include Ollama, MLX,
+OpenAI-compatible providers, and other local embedding runtimes.
+
+Core API contracts:
+
+- `POST /api/search/hybrid`
+- `GET /api/search/keyword?q=...`
+- `GET /api/search/vector?q=...`
+- `GET /api/graph`
+- `GET /api/graph/node?node_id=...`
+- `GET /api/graph/relationship`
+- `GET /api/index/status`
+- `POST /api/index/rebuild`
+
+See [docs/V3_BACKEND_ARCHITECTURE.md](docs/V3_BACKEND_ARCHITECTURE.md) for the
+storage model, search model, migration behavior, and API response shape.
+
 ## Local Model Policy
 
 Lattice AI recommends current-generation multimodal models for local use and
@@ -230,7 +259,9 @@ Core areas:
 
 | Version | Theme |
 | --- | --- |
-| **2.2.7** | Visual system stabilization — cohesive dark/light screens, crisp chat composer, dark graph canvas, Workspace OS polish |
+| **3.0.1** | Release-blocker remediation — provider-backed embeddings (Hash/MLX/Ollama/OpenAI/Custom), unified AgentRuntime boundary, every v3 surface connected or clearly unavailable |
+| 3.0.0 | v3 local-first AI workspace platform — `/app`, Native Chat, Knowledge Graph, Vector Index, Hybrid Search, workspace modes |
+| 2.2.7 | Visual system stabilization — cohesive dark/light screens, crisp chat composer, dark graph canvas, Workspace OS polish |
 | 2.2.6 | Token-native CSS foundation |
 | 2.2.5 | Release hygiene hotfix — dark overlays, modal stack, cache-busting, favicon, and Telegram log masking |
 | 2.2.4 | Chat dark-mode completion |

package/docs/CHANGELOG.md

@@ -1,5 +1,87 @@
 # Changelog
 
+## [3.0.0] - 2026-06-07
+
+> v3 — Local-first AI Workspace Platform. The hybrid-search
+> backend and the token-native `/app` workspace shell now ship together: the
+> shell's adapters call the real v3 retrieval APIs, and Chat is a first-class
+> native view (no link-out to the legacy page). Legacy `/chat` remains available
+> as a rollback/debug path.
+
+### Added — Backend retrieval
+
+- **Hybrid search API** — added `/api/search/hybrid`, `/api/search/keyword`,
+  `/api/search/vector`, `/api/graph`, `/api/graph/node`,
+  `/api/graph/relationship`, `/api/index/status`, and `/api/index/rebuild`.
+- **SQLite vector index** — added local deterministic embeddings,
+  `vector_embeddings`, and `vector_index_operations` for incremental indexing,
+  rebuilds, and status monitoring.
+- **Embedding status** — the default `lattice-local-hash-v1` embedder is a
+  deterministic local fallback, not a production semantic embedding model.
+  Future providers may include Ollama, MLX, OpenAI-compatible providers, and
+  other local embedding runtimes.
+- **Graph retrieval helpers** — added node lookup, relationship search, bounded
+  traversal, neighbor expansion, and service-level result fusion.
+- **Backend architecture doc** — added `docs/V3_BACKEND_ARCHITECTURE.md` with
+  storage, search, API, and migration details.
+
+### Added — Native app shell (`/app`)
+
+- **Unified app shell** (`static/v3/`) — nav rail, command palette (⌘K),
+  workspace switcher (Personal/Organization), and mode switcher
+  (Basic/Advanced/Admin); hash-routed views for every primary and admin area
+  (Home, Chat, Knowledge Graph, Hybrid Search, Files, Pipeline, Agents, Models,
+  My Computer, Settings, and Admin · Users/Permissions/Audit/Security/Policies/
+  Private VPC).
+- **Native Chat view** — a first-class 3-pane chat (conversations · thread ·
+  retrieval context) wired to the real backend (`POST /chat` SSE + `/history/*`)
+  with streaming, empty/error/loading states; surfaces Knowledge Graph, Vector,
+  Hybrid Search, and indexed-file context per answer. The legacy `/chat` page
+  stays reachable but is no longer the primary chat experience.
+- **Primary entry behavior** — `/app` is the product entry after login and SSO;
+  the PWA manifest starts at `/app`.
+- **Retrieval identity** — Knowledge Graph + Vector Index + Hybrid Search are
+  surfaced as a first-class "retrieval lattice" on Home and a live index chip.
+- **Token-native design system** — `static/v3/css/lattice.*.css` built on top of
+  `tokens.css` with no dependency on the legacy override layers; full light/dark
+  and desktop/tablet/mobile support.
+- **Integration adapters** — `static/v3/js/core/api.js` calls the real v3
+  endpoints and degrades to clearly-badged sample data; no backend logic in the UI.
+
+### Validation
+
+- Backend coverage includes v3 indexing, migration status, vector retrieval,
+  graph relationship traversal, hybrid result fusion, and API contract tests.
+- Frontend coverage: `tests/visual/v3.spec.js` and `scripts/lint_v3.mjs` (wired
+  into `npm run lint`); see `docs/V3_FRONTEND.md` for IA + design decisions.
+- Release preparation builds exact `3.0.0` Python, npm, and VSIX artifacts.
+  Package-store publication remains manual and is not triggered by pushing the
+  release tag.
+
+> Frontend Product Shell Redesign — workspace navigation, auth entry, and shared
+> product surfaces were realigned around the local-first AI workspace model
+> without changing backend contracts.
+
+### Changed
+
+- **Workspace IA** — the workspace shell now separates primary user workflows,
+  admin controls, and runtime tooling, with Basic, Advanced, and Admin modes.
+- **Navigation** — Chat and Workspace navigation now use consistent labels for
+  Home, Chat, Knowledge Graph, Files, Pipeline, My Computer, Search, and
+  organization administration.
+- **Design tokens** — shared product surfaces moved away from the prior
+  lavender-heavy treatment toward neutral work surfaces with blue, teal, and
+  amber accents.
+- **Auth surface** — account screens use the same token-native product shell as
+  the workspace experience and hide decorative background elements.
+
+### Validation
+
+- Frontend validation includes lint, Python checks, browser-rendered workspace
+  smoke checks, and Playwright visual regression coverage.
+- Production build output was intentionally not generated for this frontend-only
+  redesign pass.
+
 ## [2.2.7] - 2026-06-05
 
 > Visual Stabilization Release — browser-rendered screens were reviewed and

package/docs/V3_BACKEND_ARCHITECTURE.md

@@ -0,0 +1,138 @@
+# Lattice AI v3 Backend Architecture
+
+Lattice AI v3 keeps the product local-first and knowledge-centric by combining
+three retrieval layers inside the backend:
+
+```text
+User Data
+  -> Chunking
+  -> Local Embeddings
+  -> SQLite Vector Index
+
+User Data
+  -> Entity Extraction
+  -> Relationship Extraction
+  -> SQLite Knowledge Graph
+
+Search
+  -> Hybrid Retrieval
+     -> Keyword
+     -> Vector
+     -> Graph
+  -> Unified Results
+```
+
+## Storage Model
+
+The authoritative knowledge store remains `knowledge_graph.py` on SQLite.
+Existing legacy graph rows are preserved, with the normalized v2 projection kept
+as a derived read surface.
+
+- `nodes`: entities, documents, folders, conversations, tasks, decisions, and
+  chunk nodes.
+- `edges`: directed relationships with type, weight, metadata, and timestamps.
+- `chunks`: extracted text chunks linked back to the source node.
+- `knowledge_sources` and `local_file_index`: approved local folder indexing
+  state, source metadata, and incremental scan bookkeeping.
+- `vector_embeddings`: derived local vector index rows for non-chunk nodes and
+  full chunk text.
+- `vector_index_operations`: rebuild/incremental indexing history and status.
+
+Vector rows are derived data. A full rebuild may delete and recreate vector
+rows, but it does not mutate source documents, nodes, relationships, chunks, or
+local file records.
+
+## Embeddings
+
+The default embedding model is `lattice-local-hash-v1`, a deterministic local
+feature-hashing embedder in `latticeai/core/local_embeddings.py`.
+
+It provides:
+
+- no cloud dependency,
+- no model download requirement,
+- deterministic output for tests and migrations,
+- normalized vectors for cosine similarity by dot product,
+- a stable interface for replacing the implementation with a local model
+  runtime later.
+
+It is a fallback vector signal, not a production semantic embedding model.
+Future provider support may include Ollama, MLX, OpenAI-compatible providers,
+and other local embedding providers behind the same vector-index interface.
+
+## Search Model
+
+`latticeai/services/search_service.py` composes the retrieval layers.
+
+- Keyword search uses the existing graph text search over title, summary, and
+  metadata.
+- Vector search embeds the query locally and ranks SQLite vector rows by
+  similarity.
+- Graph search performs direct graph matching, relationship search, and bounded
+  neighbor expansion.
+- Hybrid search fuses the three result streams with configurable weights and
+  returns one UI-ready result shape with per-source scores.
+
+Default hybrid weights:
+
+```json
+{
+  "keyword": 0.35,
+  "vector": 0.40,
+  "graph": 0.25
+}
+```
+
+## API Contracts
+
+The v3 backend exposes frontend-ready routes under `/api` while preserving all
+existing `/knowledge-graph/...` compatibility routes.
+
+```text
+POST /api/search/hybrid
+GET  /api/search/hybrid?q=...
+POST /api/search/keyword
+GET  /api/search/keyword?q=...
+POST /api/search/vector
+GET  /api/search/vector?q=...
+GET  /api/graph
+GET  /api/graph/node?node_id=...
+POST /api/graph/node
+GET  /api/graph/relationship
+POST /api/graph/relationship
+GET  /api/index/status
+POST /api/index/rebuild
+```
+
+Results include stable fields for UI consumption:
+
+- `id`
+- `node_id`
+- `item_type`
+- `type`
+- `title`
+- `summary`
+- `score`
+- `rank`
+- `sources`
+- `source_scores`
+- `metadata`
+- `updated_at`
+- optional `graph_context`
+
+## Migration Impact
+
+The migration is additive and non-destructive:
+
+- new SQLite tables are created with `CREATE TABLE IF NOT EXISTS`,
+- existing graph and local file data is preserved,
+- normalized v2 graph projection remains a derived compatibility layer,
+- vector rows can be rebuilt from existing graph/chunk rows,
+- `GET /api/index/status` reports missing or stale vector items,
+- `POST /api/index/rebuild` repairs the derived index.
+
+## Backend Ownership
+
+This architecture deliberately avoids frontend shell redesign. The backend owns
+storage, indexing, retrieval, ranking, and API contracts; the UI can consume the
+new `/api` routes without requiring visual system changes.

package/docs/V3_FRONTEND.md

@@ -0,0 +1,136 @@
+# Lattice AI v3 — Frontend Product Shell
+
+> A token-native single-page workspace shell for Lattice AI v3. It ships as the
+> primary product experience and calls the real v3 retrieval/chat APIs while
+> keeping graceful, clearly-badged sample states for unavailable local services.
+
+Entry point: **`/app`** (served by `latticeai/api/static_routes.py` →
+`static/v3/index.html`). Login, auto-login after registration, SSO callback, and
+the PWA manifest land on `/app`. The legacy multi-page screens (`/workspace`,
+`/chat`, `/graph`, `/admin`, …) remain reachable; `/chat` is the rollback/debug
+path for the native v3 Chat view.
+
+---
+
+## Product model
+
+| Concept | Surface |
+| --- | --- |
+| **Personal / Organization workspace** | Scope switcher in the rail (`X-Workspace-Id` header is sent on every adapter call). |
+| **Basic / Advanced / Admin mode** | Mode switcher in the topbar. Mode gates which rail items appear and how dense each view is. |
+| **Retrieval lattice** | The signature identity: **Knowledge Graph + Vector Index + Hybrid Search**, surfaced on Home and as a live 3-dot index chip in the topbar. |
+
+Mode gating (`MODE_RANK` in `core/routes.js`): Basic ⊂ Advanced ⊂ Admin.
+Deep-linking into an `admin/*` route auto-promotes the shell to Admin mode.
+
+---
+
+## Information architecture
+
+One declarative table (`static/v3/js/core/routes.js`) drives the nav rail, the
+command palette, the router, breadcrumbs, and lazy view loading.
+
+```
+Workspace      Home · Chat
+Retrieval      Knowledge Graph · Hybrid Search
+Data           Files · Pipeline*
+Compute        Agents* · Models · My Computer*
+System         Settings
+Administration Users · Permissions · Audit Logs · Security · Policies · Private VPC   (admin mode)
+```
+
+`*` = revealed in Advanced mode and above. Routes are hash-based
+(`#/knowledge-graph`, `#/admin/users`) so the SPA needs no server rewrites.
+
+---
+
+## Design system
+
+Token-native, layered on the existing **single color source**
+(`static/css/tokens.css`, `data-lt-theme` light/dark). No legacy override layers
+(`responsive.css` / `workspace.css` / `platform.css`) are loaded by v3.
+
+| File | Responsibility |
+| --- | --- |
+| `css/lattice.tokens.css` | Structural tokens — type scale, spacing (4pt), radii, elevation, motion, z-index, layout rails, pillar accents, lattice mesh. |
+| `css/lattice.base.css` | Reset, element defaults, the signature lattice backdrop, a11y utilities. |
+| `css/lattice.components.css` | Primitive vocabulary — cards, panels, buttons, inputs, pills, stats, tables, tabs, switches, meters, empty/loading/error states, toasts. |
+| `css/lattice.shell.css` | App chrome — rail, topbar, command palette, mobile drawer, retrieval pillars. |
+| `css/lattice.views.css` | Bespoke view layouts — hero, graph canvas, search results, chat, files, pipeline, admin. |
+
+**Rule:** component CSS uses only `var(--token)`; no per-component hardcoded
+themed colors and no `!important`. Both themes flow from the token values.
+
+**Identity:** a crystalline *lattice* mark, a faint node-and-edge mesh backdrop,
+and a three-pillar retrieval motif (graph = blue, vector = teal, hybrid =
+magenta). Deliberately **not** a clone of ChatGPT / Claude / Cursor / VS Code /
+Notion / Obsidian.
+
+---
+
+## Shell + view contract
+
+`core/shell.js` builds the chrome and renders views. Each view is a lazy ES
+module under `js/views/` exporting:
+
+```js
+export async function render(ctx) { /* … */ return singleDomNode; }
+// optional: export const layout = "flush";  // full-bleed (chat)
+```
+
+`ctx = { h, icon, api, store, c, route, params, navigate, toast }`
+
+- `h()` — dependency-free hyperscript (`core/dom.js`); child strings are
+  auto-escaped (no injection surface).
+- `c` — component factories (`core/components.js`).
+- `store` — observable app state (theme / mode / workspace), persisted to
+  `localStorage` (`core/store.js`).
+
+---
+
+## Integration contract
+
+`core/api.js` is the only transport layer. Every call hits the **real** endpoint
+first and degrades to a clearly-badged sample payload (`core/fixtures.js`) when
+the endpoint is absent — returning `{ ok, status, data, source }` where `source`
+is `"live"` or `"placeholder"`. The UI always renders a **Sample data** badge for
+placeholder responses, so nothing fake is presented as backend output.
+
+Live surfaces wired in v3.0.0:
+
+| Adapter | Endpoint | Fallback |
+| --- | --- | --- |
+| `api.indexStatus()` | `GET /api/index/status` | sample KG/vector/hybrid pipeline state |
+| `api.graph(params)` | `GET /api/graph` → `GET /knowledge-graph/graph` | sample mesh |
+| `api.hybridSearch(q, opts)` | `POST /api/search/hybrid` | sample fused results |
+| `api.streamChat(body)` | `POST /chat` SSE | sample stream only when endpoint is unavailable |
+
+No backend retrieval logic is implemented in the frontend — only transport,
+normalization, clear provenance badges, and graceful fallback. When the live
+chat backend reports `no_model_loaded`, v3 Chat shows a setup message instead of
+falling back to sample generation.
+
+Embedding disclosure: the current default vector signal is
+`lattice-local-hash-v1`, a deterministic local fallback. The UI avoids calling
+it a production semantic embedding model.
+
+---
+
+## Responsive strategy
+
+- **Desktop (≥1101px):** full 268px rail + content.
+- **Tablet (761–1100px):** rail collapses to a 76px icon rail.
+- **Mobile (≤760px):** rail becomes an off-canvas drawer (hamburger in the
+  topbar); grids collapse to single column; the chat context rail hides.
+
+Plus: `prefers-color-scheme` following, `prefers-reduced-motion` honored, visible
+focus rings, skip link, keyboard command palette (⌘K / Ctrl-K).
+
+---
+
+## Validation
+
+- `npm run lint` (extended to cover `static/v3/**` via `scripts/lint_v3.mjs`).
+- `npm run test:visual` (`tests/visual/v3.spec.js` against the mock server, which
+  serves `/app` and mocks the future API surfaces).
+- Browser-rendered smoke checks of every route in light and dark themes.