ltcai 1.4.0 → 1.5.0

package/README.md

@@ -15,7 +15,7 @@
 
   <br/>
 
-  <img src="https://raw.githubusercontent.com/TaeSooPark-PTS/LatticeAI/main/docs/images/lattice-ai-demo.gif" alt="Lattice AI demo showing chat, knowledge graph, and admin dashboard" width="100%"/>
+  <img src="docs/images/hero.gif" alt="Lattice AI — AI Workspace OS for local-first graph, memory, and agents" width="100%"/>
 </div>
 
 ---
@@ -36,80 +36,28 @@ Automatic knowledge graph
 Graph-aware chat, snapshots, memory, agents, workflows, skills, and timeline
 ```
 
-### New in 1.4.0: Server App Final Decomposition
-
-- **server_app.py final decomposition** — the app shell is now FastAPI assembly,
-  lifespan, middleware, static mounting, and router wiring only
-  (~5,381 → 1,303 lines)
-- **Chat / model / tools extraction** — chat/history/agent, model runtime and
-  provider helpers, tools, local files, computer-use, permissions, upload,
-  garden/setup/static UI, MCP, and KG glue now live in API routers and services
-- **AppContext and dependency cleanup** — routers receive explicit dependencies
-  and do not import the FastAPI app; service modules own runtime and dispatch
-  business logic
-- **Safety validation suite** — route compatibility, import/startup, streaming,
-  model endpoint, tools/local/CU, release-artifact, and documentation stale
-  checks guard the split
-- **Compatibility preserved** — all public API paths, request/response schemas,
-  `server:app`, CLI, UI, Knowledge Graph, Admin/Security, Workspace OS, and
-  VS Code expectations remain unchanged
-
-### New in 1.2.0: Server App Modularization
-
-- **server_app.py modularized** — Workspace/Organization and health/engine
-  endpoints extracted into dedicated routers (`latticeai/api/*`) backed by a
-  service layer (`latticeai/services/*`); `server_app` is now app assembly +
-  router include (~6,585 → ~5,948 lines)
-- **Routers / services split** — `create_workspace_router`,
-  `create_health_router`, `WorkspaceService`, `ModelService`, `ChatService`
-- **Workspace API service layer** — scope resolution and role/permission checks
-  centralized in `WorkspaceService`
-- **Workspace / org guardrails** — non-members can't read/write org data,
-  viewers can't write, owners/admins manage members; no-auth local owner
-  fallback preserved
-- **Health / model / chat modularization** — `/health`, `/mode`,
-  `/runtime_features`, `/engines` via the health router; chat trace recording
-  via the chat service (streaming behavior unchanged)
-- **Compatibility preserved** — `server:app` import path, all API routes, CLI,
-  Knowledge Graph / Admin / Security routers, and VS Code integration unchanged
-
-### New in 1.1.0: Organization Workspace Foundation
-
-- **Organization Workspace** alongside Personal Workspace — create shared org
-  workspaces, list/switch between them, and archive (non-destructively)
-- **Workspace roles & permissions** — `owner`, `admin`, `member`, `viewer`
-  mapped to read / write / manage-members / manage-workspace
-- **Workspace-scoped data** — snapshots, memory, agent runs, workflows, traces,
-  and timeline carry a `workspace_id`; reads scope via the `X-Workspace-Id` header
-- **CI / release hardening** — Node.js 24 ready workflow, version-scoped
-  artifact upload (never `dist/*`), and a release artifact validator
-- **Enterprise extension foundation (open-core)** — a stable seam for a future
-  Enterprise plugin; Community ships everything it has today, unrestricted
-  (see [docs/ENTERPRISE.md](docs/ENTERPRISE.md) and
-  [docs/EDITION_STRATEGY.md](docs/EDITION_STRATEGY.md))
-
-### New in 1.0.0: AI Workspace OS
-
-- Workspace OS command center at `/workspace`
-- First-run onboarding state API and UI
-- Graph RAG answer traces with sources, nodes, edges, confidence, and jump links
-- Local indexing dashboard with watcher state, success/failure counts, pause/resume/remove
-- Workspace snapshots, Time Machine views, export, and Knowledge Diff
-- Personal memory CRUD/search linked back to the graph
-- Multi-agent graph entities and agent run history
-- Relationship Explorer for inbound, outbound, related entities, and shortest path
-- Local Computer Memory remains OFF by default and requires explicit approval
-- Skill Marketplace registry with install, uninstall, update, enable, disable, and version state
-- Workflow Graph for upload -> summarize -> generate -> export style work histories
-- VS Code commands for Explain Selection, Refactor Selection, Generate Tests, Send To Lattice, and Ask About Current File
-
-### Built for people who want
-
-- a private AI workspace that runs from their own machine
-- local model setup without hunting through many tools
-- folder indexing that becomes useful AI memory
-- a visual knowledge graph instead of disconnected files and chats
-- optional team/admin controls for audit, permissions, and sensitive-data monitoring
+## Why Lattice AI?
+
+- **Local-first by default** — models, data, and your knowledge graph stay on your machine (`~/.ltcai/`); cloud is strictly opt-in.
+- **Memory that compounds** — every chat, file, and folder you approve becomes durable, searchable context instead of being forgotten.
+- **A graph, not a pile of files** — people, projects, documents, decisions, and tasks are linked automatically and explored visually.
+- **One workspace, everywhere** — the same local knowledge powers the web UI, VS Code / Cursor, Telegram, and MCP clients.
+- **Built-in governance** — Personal and Organization workspaces, roles, an audit timeline, and sensitive-data monitoring for teams.
+
+## Core Capabilities
+
+| Capability | What it does |
+|---|---|
+| 🧠 Automatic knowledge graph | Turns chats, files, and folders into linked nodes and edges, curated automatically |
+| 💬 Graph-aware chat & agents | Answers and multi-step agents grounded in your indexed local memory |
+| 🖥️ Local model recommendation | Scans your hardware and rates each model **Recommended / Compatible / Not Recommended** |
+| 🗂️ Workspaces & roles | Personal and Organization workspaces with owner / admin / member / viewer permissions |
+| 🧩 Skills & MCP | Install skills and connect MCP tools from the in-product marketplace |
+| 🔒 Admin & security | Audit timeline, permission approvals, sensitive-data detection, exportable reports |
+
+<div align="center">
+  <img src="docs/images/onboarding.png" alt="Onboarding flow: install, system scan, model recommendation, workspace, indexing, knowledge graph, first chat" width="100%"/>
+</div>
 
 ---
 
@@ -171,6 +119,20 @@ LTCAI
 
 ---
 
+## Architecture
+
+`server:app` stays a thin compatibility entrypoint; the FastAPI app is assembled in
+`latticeai/server_app.py`, and the work lives in focused API routers, a service
+layer, and core modules — so the app shell never grows monolithic again.
+
+<div align="center">
+  <img src="docs/images/architecture.png" alt="Lattice AI architecture — entrypoint, API routers, services, core, local engines and knowledge graph" width="100%"/>
+</div>
+
+See [docs/architecture.md](docs/architecture.md) for request and data-flow detail.
+
+---
+
 ## Product Preview
 
 <table>
@@ -193,6 +155,44 @@ LTCAI
 </tr>
 </table>
 
+> Screenshots above are the live web UI. The diagrams below map the product
+> experience to the current v1.5.0 structure.
+
+---
+
+## Product Experience
+
+### Local model recommendation
+
+Lattice AI detects your OS, CPU, GPU, RAM, and disk, then rates every local model
+**Recommended**, **Compatible**, or **Not Recommended** for your machine — grouped
+by family (Gemma, Qwen, Llama, Phi, DeepSeek, and more).
+
+<div align="center">
+  <img src="docs/images/model-recommendation.png" alt="Tri-state local model recommendation grouped by family" width="100%"/>
+</div>
+
+### Workspaces & organization
+
+Switch instantly between a **Personal** workspace and shared **Organization**
+workspaces. Org data is scoped by `workspace_id`, and `owner / admin / member /
+viewer` roles map to a transparent permission matrix.
+
+<div align="center">
+  <img src="docs/images/workspace.png" alt="Personal and Organization workspace model" width="49%"/>
+  <img src="docs/images/organization.png" alt="Organization roles and permission matrix" width="49%"/>
+</div>
+
+### Knowledge graph & skills
+
+Your work becomes a typed knowledge graph (built automatically), and skills extend
+the workspace through an in-product marketplace.
+
+<div align="center">
+  <img src="docs/images/graph.png" alt="Knowledge graph node and edge taxonomy" width="49%"/>
+  <img src="docs/images/skills.png" alt="Skill marketplace: recommended, popular, installed, updates" width="49%"/>
+</div>
+
 ---
 
 ## Why it is different
@@ -333,20 +333,24 @@ Supported routes include OpenAI-compatible APIs, OpenRouter, Groq, Together, xAI
 
 ## Current release
 
-**1.4.0** completes the Server App Final Decomposition release:
-
-- `server.py` remains the thin compatibility entrypoint and
-  `latticeai/server_app.py` is now a compact app assembly shell
-- chat/history/agent, model runtime/provider helpers, tools/local/CU/
-  permissions/upload, garden/setup/static pages, MCP, and KG router wiring are
-  extracted into `latticeai/api/*` and `latticeai/services/*`
-- route compatibility, streaming, model endpoint, tools/local/CU,
-  import/startup, build, packaging, and documentation stale-reference checks
-  are part of the release validation
+**1.5.0 — Unified Product Release.** Onboarding, model recommendation, and CI
+stabilization in one release:
+
+- **CI / VSIX recovery** — the stale `@azure/core-tracing` lockfile pin that
+  broke `npm ci` (ETARGET) is regenerated, so the VSIX build is green again
+- **Local model recommendation** — a hardware-aware engine
+  (`latticeai/services/model_recommendation.py`) classifies the model catalog as
+  Recommended / Compatible / Not Recommended, exposed at `/models/recommendations`
+- **Catalog extraction** — the static model catalog moved to
+  `latticeai/services/model_catalog.py`, simplifying `model_runtime.py`
+- **Enterprise PoC seam** — admin policy / audit-export / SIEM-stub / org-settings
+  surfaces consult the capability registry (Community keeps everything ungated)
+- **Documentation & visuals** — README rewritten as a product page with an
+  up-to-date architecture diagram and structural visuals
 - Python package, npm package, VS Code extension, FastAPI app, and `/health`
-  version metadata are aligned at `1.4.0`
+  version metadata are aligned at `1.5.0`
 
-See the full [changelog](docs/CHANGELOG.md).
+See the full [changelog](docs/CHANGELOG.md) and [RELEASE.md](RELEASE.md).
 
 ---
 
@@ -535,6 +539,20 @@ Full reference: [docs/mcp-tools.md](docs/mcp-tools.md)
 
 ---
 
+## Documentation
+
+| Doc | What's inside |
+|---|---|
+| [docs/architecture.md](docs/architecture.md) | App structure, request and data flow |
+| [docs/CHANGELOG.md](docs/CHANGELOG.md) | Full version history |
+| [RELEASE.md](RELEASE.md) | Release notes and the build/publish checklist |
+| [SECURITY.md](SECURITY.md) | Security model and vulnerability reporting |
+| [docs/ENTERPRISE.md](docs/ENTERPRISE.md) · [docs/EDITION_STRATEGY.md](docs/EDITION_STRATEGY.md) | Open-core boundary and edition strategy |
+| [docs/kg-schema.md](docs/kg-schema.md) · [docs/mcp-tools.md](docs/mcp-tools.md) | Knowledge graph schema and MCP tool catalog |
+| [docs/privacy.md](docs/privacy.md) · [docs/public-deploy.md](docs/public-deploy.md) · [docs/OPERATIONS.md](docs/OPERATIONS.md) | Privacy, public deployment, operations |
+
+---
+
 ## Contributing
 
 See [CONTRIBUTING.md](CONTRIBUTING.md). Issues and pull requests are welcome.

package/docs/CHANGELOG.md

@@ -1,5 +1,59 @@
 # Changelog
 
+## [1.5.0] - 2026-06-01
+
+> Unified Product Release — CI/VSIX recovery, hardware-aware local model
+> recommendation, model-catalog extraction, an Enterprise PoC seam, and a
+> product-page README with an up-to-date architecture diagram. The public route
+> contract, schemas, `server:app`, CLI, UI, and VS Code integration are
+> unchanged.
+
+### Fixed
+
+- **VSIX / `npm ci` (ETARGET)** — `vscode-extension/package-lock.json` pinned a
+  non-existent `@azure/core-tracing@^1.4.0` (the registry's latest is `1.3.1`),
+  breaking `npm ci` and the GitHub Actions VSIX build. The lockfile is
+  regenerated so the published `^1.3.0` ranges resolve; `npm ci` → `npm run
+  compile` → `vsce package` is green again.
+
+### Added
+
+- **Local model recommendation** — `latticeai/services/model_recommendation.py`
+  classifies the model catalog into **recommended / compatible / not_recommended**
+  from a detected system profile (OS/RAM/CPU/GPU/disk), grouped by family
+  (Gemma, Qwen, Llama, Phi, DeepSeek, …). Exposed at `GET /models/recommendations`
+  and folded into `/workspace/onboarding/model-recommendations` as a `catalog`
+  field. Covered by `tests/unit/test_model_recommendation.py`.
+- **Enterprise PoC surfaces** — `latticeai/core/enterprise_admin.py` plus
+  `GET /admin/enterprise` and `GET /admin/enterprise/siem-export` provide admin
+  policy, audit-export, SIEM-export-stub, and organization-settings views built
+  on the existing capability seam. Community reports every Enterprise capability
+  as disabled and never gates a Community feature
+  (`tests/unit/test_enterprise_admin.py`).
+- **DeepSeek family** — added to the Ollama and llama.cpp catalogs with
+  identifiers chosen so the version-dedup filter is unaffected.
+
+### Changed
+
+- **Model catalog extraction** — the static catalog (`ENGINE_MODEL_CATALOG`,
+  `ENGINE_INSTALLERS`, `MODEL_ENGINE_ALIASES`) and the pure version-dedup helpers
+  moved to `latticeai/services/model_catalog.py`, re-exported by `model_runtime`
+  for backward compatibility. `model_runtime.py` shrank from 1,973 to 1,721 lines
+  (`tests/unit/test_model_catalog.py` pins the re-export identity).
+- **README rewritten as a product page** — Why / Core Capabilities / Quick Start
+  / Architecture / Current Release / Documentation, with structural diagrams
+  (`docs/images/*`) and a current architecture diagram. Historical "New in 1.x"
+  marketing blocks were removed from the README top (this changelog remains the
+  version history).
+- Python package, npm package, VS Code extension, FastAPI app, and `/health`
+  version metadata aligned at `1.5.0`.
+
+### Validation
+
+- 266 unit tests pass; route-compatibility, import/startup, streaming, model
+  endpoint, MCP/KG contract tests preserved; `npm run check:python` green; VSIX
+  build verified. Test/build/packaging artifacts only — no package-store publish.
+
 ## [1.4.0] - 2026-05-31
 
 > Server App Final Decomposition — chat, model runtime, tools/local/CU,

package/latticeai/__init__.py

@@ -1,3 +1,3 @@
 """Lattice AI - modular server package."""
 
-__version__ = "1.4.0"
+__version__ = "1.5.0"

package/latticeai/api/admin.py

@@ -184,4 +184,21 @@ def create_admin_router(
         )
         return public_sso_config(saved)
 
+    @router.get("/admin/enterprise")
+    async def admin_enterprise_overview(request: Request):
+        """Enterprise PoC surface: edition matrix, admin policies, audit export,
+        SIEM stub, and org-governance capabilities. Community reports every
+        Enterprise capability as disabled and never gates Community features."""
+        require_admin(request)
+        from latticeai.core.enterprise_admin import poc_overview
+        return poc_overview()
+
+    @router.get("/admin/enterprise/siem-export")
+    async def admin_enterprise_siem_export(request: Request):
+        """Preview the SIEM export envelope. In Community this is a stub
+        (``streamed=false``) — no events are pushed to an external SIEM."""
+        require_admin(request)
+        from latticeai.core.enterprise_admin import siem_export_stub
+        return siem_export_stub()
+
     return router

package/latticeai/api/models.py

@@ -304,4 +304,20 @@ def create_models_router(
         _router.unload_all()
         return {"status": "ok", "unloaded": unloaded}
 
+    @router.get("/models/recommendations")
+    async def model_recommendations(request: Request, engine: str = "local_mlx"):
+        """Hardware-aware tri-state model recommendation for this machine.
+
+        Detects the system profile (OS/RAM/CPU/GPU/disk) and classifies the
+        ``engine`` catalog into recommended / compatible / not_recommended,
+        grouped by family. Used by the onboarding and model-picker UIs.
+        """
+        require_user(request)
+        from auto_setup import probe as auto_setup_probe
+        from latticeai.services.model_recommendation import recommend_catalog
+
+        profile = await asyncio.to_thread(lambda: auto_setup_probe().to_json())
+        catalog = recommend_catalog(profile, engine=engine)
+        return {"profile": profile, "recommendations": catalog}
+
     return router

package/latticeai/api/workspace.py

@@ -262,9 +262,20 @@ def create_workspace_router(
         require_user(request)
         env = await asyncio.to_thread(scan_environment)
         recommendations = get_recommendations(env)
+        # Tri-state, family-grouped catalog (recommended / compatible /
+        # not_recommended) for this machine, used by the onboarding model step.
+        catalog = None
+        try:
+            from auto_setup import probe as auto_setup_probe
+            from latticeai.services.model_recommendation import recommend_catalog
+            profile = await asyncio.to_thread(lambda: auto_setup_probe().to_json())
+            catalog = recommend_catalog(profile, engine="local_mlx")
+        except Exception as exc:  # pragma: no cover - recommendation is best-effort
+            logging.warning("model recommendation catalog failed: %s", exc)
         payload = {
             "environment": env,
             "recommendations": recommendations,
+            "catalog": catalog,
             "default_local_model": LOCAL_MODEL,
             "default_public_model": PUBLIC_MODEL,
         }

package/latticeai/core/enterprise_admin.py

@@ -0,0 +1,158 @@
+"""Enterprise PoC surfaces (admin policies, audit export, SIEM stub, org settings).
+
+This module is **structure only** — it prepares concrete, discoverable shapes for
+Enterprise governance features while keeping the open-source Community edition
+fully functional and ungated. Every capability here is consulted through
+:data:`latticeai.core.enterprise.capability_registry`; in the Community build
+each is reported ``enabled=False`` and the Community behaviour (local audit
+export, the four base roles, single-tenant local storage) is always available.
+
+Nothing in this module restricts a Community feature. It answers "what *would*
+an Enterprise provider light up, and is it active?" so the admin UI can show an
+honest edition/capability matrix and a SIEM export *preview envelope* without
+shipping any Enterprise implementation.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from latticeai.core.enterprise import (
+    EnterpriseCapability,
+    capability_registry,
+)
+
+COMMUNITY_NOTICE = (
+    "Community edition: this is an Enterprise extension point and is not "
+    "enforced. Local-first behaviour is always available. See "
+    "docs/ENTERPRISE.md and docs/EDITION_STRATEGY.md."
+)
+
+
+def _cap(capability: EnterpriseCapability) -> bool:
+    return capability_registry.is_capability_enabled(capability)
+
+
+def admin_policies() -> Dict[str, Any]:
+    """Admin policy-pack status + the effective (open) Community policy."""
+    enabled = _cap(EnterpriseCapability.ADMIN_POLICY_PACKS)
+    return {
+        "capability": EnterpriseCapability.ADMIN_POLICY_PACKS.value,
+        "enabled": enabled,
+        "enforced": enabled,
+        "effective_policy": {
+            # Community defaults — descriptive, not enforced by a policy engine.
+            "base_roles": ["owner", "admin", "member", "viewer"],
+            "local_file_access": "approval-token gated (per path/user/action)",
+            "package_install": "admin-only with audit trail",
+            "network_binding": "127.0.0.1 by default",
+            "managed_policy_packs": [] if not enabled else "provided-by-enterprise",
+        },
+        "note": COMMUNITY_NOTICE,
+    }
+
+
+def audit_export_descriptor() -> Dict[str, Any]:
+    """What audit export is available locally vs. via Enterprise SIEM streaming."""
+    siem_enabled = _cap(EnterpriseCapability.SIEM_EXPORT)
+    retention_enabled = _cap(EnterpriseCapability.COMPLIANCE_RETENTION)
+    return {
+        "local_export": {
+            "available": True,
+            "endpoint": "/admin/security/export",
+            "formats": ["json", "csv", "xlsx", "txt", "pdf"],
+            "note": "Community local audit export is always available to admins.",
+        },
+        "siem_streaming": {
+            "capability": EnterpriseCapability.SIEM_EXPORT.value,
+            "enabled": siem_enabled,
+            "note": COMMUNITY_NOTICE,
+        },
+        "compliance_retention": {
+            "capability": EnterpriseCapability.COMPLIANCE_RETENTION.value,
+            "enabled": retention_enabled,
+            "note": COMMUNITY_NOTICE,
+        },
+    }
+
+
+def siem_export_stub(events: Optional[List[Dict[str, Any]]] = None) -> Dict[str, Any]:
+    """A preview of the envelope an Enterprise SIEM exporter would emit.
+
+    In the Community build this is a *stub*: it returns the envelope *shape*
+    (so integrators can see the contract) but ``streamed=False`` and no events
+    are actually pushed to an external SIEM.
+    """
+    enabled = _cap(EnterpriseCapability.SIEM_EXPORT)
+    sample = events or [
+        {
+            "id": "evt_sample",
+            "type": "audit_event",
+            "timestamp": "1970-01-01T00:00:00Z",
+            "actor": "admin@example.com",
+            "action": "model_load",
+            "severity": "informational",
+        }
+    ]
+    envelope = {
+        "format": "ltcai.siem.v1",
+        "encoding": "ndjson",
+        "vendor": "LatticeAI",
+        "product": "Workspace OS",
+        "records": [
+            {
+                "ts": e.get("timestamp"),
+                "actor": e.get("actor"),
+                "act": e.get("action"),
+                "sev": e.get("severity", "informational"),
+                "kind": e.get("type"),
+                "id": e.get("id"),
+            }
+            for e in sample
+        ],
+    }
+    return {
+        "capability": EnterpriseCapability.SIEM_EXPORT.value,
+        "enabled": enabled,
+        "streamed": False if not enabled else True,
+        "destination": None if not enabled else "configured-by-enterprise",
+        "preview_envelope": envelope,
+        "note": COMMUNITY_NOTICE,
+    }
+
+
+def organization_settings() -> Dict[str, Any]:
+    """Org-scale governance capabilities and their (Community=off) state."""
+    governance_caps = [
+        EnterpriseCapability.TENANT_ISOLATION,
+        EnterpriseCapability.RBAC_ABAC_ADVANCED,
+        EnterpriseCapability.SCIM,
+        EnterpriseCapability.IDP_PROVISIONING,
+        EnterpriseCapability.SSO_ADVANCED,
+        EnterpriseCapability.DLP_POLICY,
+        EnterpriseCapability.EDISCOVERY,
+        EnterpriseCapability.PRIVATE_VPC,
+        EnterpriseCapability.AIR_GAPPED_DEPLOYMENT,
+    ]
+    return {
+        "community_baseline": {
+            "workspaces": ["personal", "organization"],
+            "roles": ["owner", "admin", "member", "viewer"],
+            "data_isolation": "single-tenant local storage (~/.ltcai)",
+        },
+        "governance_capabilities": {
+            cap.value: _cap(cap) for cap in governance_caps
+        },
+        "note": COMMUNITY_NOTICE,
+    }
+
+
+def poc_overview() -> Dict[str, Any]:
+    """Combined Enterprise PoC surface for the admin dashboard."""
+    return {
+        "edition": capability_registry.describe(),
+        "admin_policies": admin_policies(),
+        "audit_export": audit_export_descriptor(),
+        "siem_export": siem_export_stub(),
+        "organization_settings": organization_settings(),
+    }

package/latticeai/core/workspace_os.py

@@ -18,7 +18,7 @@ from pathlib import Path
 from typing import Any, Callable, Dict, Iterable, List, Optional
 
 
-WORKSPACE_OS_VERSION = "1.4.0"
+WORKSPACE_OS_VERSION = "1.5.0"
 
 # Workspace types separate single-user Personal workspaces from shared
 # Organization workspaces. Both keep the same local-first JSON store; the type