ltcai 3.5.0 → 3.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -3,10 +3,11 @@
3
3
 
4
4
  # Lattice AI
5
5
 
6
- **Local-first AI workspace for your files, chats, knowledge, models, and agents.**
6
+ **A local-first Digital Brain Platform. Your Knowledge Graph is the durable asset; models just read it.**
7
7
 
8
- Keep your work context on your own machine. Connect documents, conversations,
9
- local models, graph memory, and agent workflows in one self-hosted workspace.
8
+ Every source files, folders, web pages, browser tabs converges into one
9
+ Knowledge Graph on your own machine. Connect models, agents, and search to that
10
+ graph instead of placing your work inside any single model.
10
11
  </div>
11
12
 
12
13
  <div align="center">
@@ -24,10 +25,21 @@
24
25
 
25
26
  ![Lattice AI — local-first AI workspace home](docs/assets/v3.4.0/home.png)
26
27
 
27
- > **Lattice AI is a self-hosted AI workspace that keeps your files, chats, knowledge, local models, and agents together on your own machine.**
28
+ > **Lattice AI is not a model-personalization system. It is a Digital Brain Platform.**
29
+ > The Knowledge Graph is your durable asset. **Models are replaceable. Knowledge is durable.**
28
30
 
29
- It isn't another chat window. It's a workspace built around your work local-first
30
- by default, cloud only when you choose.
31
+ It isn't another chat window, and it isn't a way to "fine-tune a model on you." The
32
+ purpose of Lattice AI is to **connect models to your Knowledge Graph** — your digital
33
+ brain — not to place you inside a model. AI reads your knowledge; you own it.
34
+
35
+ - **Models are replaceable.** Swap MLX, Ollama, LM Studio, or a cloud LLM at will.
36
+ - **Agents, RAG, and the UI are replaceable.** They are implementations, not the asset.
37
+ - **Your Knowledge Graph is durable.** It outlives every model and is yours to export,
38
+ import, and back up locally — no cloud required.
39
+
40
+ Local-first by default; cloud only when you choose. (The Vercel site is a
41
+ landing/download/demo surface only — never the runtime. Lattice AI runs on your
42
+ machine over local SQLite.)
31
43
 
32
44
  ## Why install Lattice AI?
33
45
 
@@ -187,32 +199,51 @@ npm run dev
187
199
 
188
200
  ## Latest Release
189
201
 
190
- ### v3.4.1Runtime Completion
191
-
192
- - Full hooks lifecycle across HTTP, agent, workflow, upload, and indexing paths.
193
- - Real Local Agent probes instead of hardcoded readiness.
194
- - Connect Folder verified end-to-end.
195
- - Folder Watch verified, including restore after restart.
196
-
197
- See [RELEASE_NOTES_v3.4.1.md](RELEASE_NOTES_v3.4.1.md) and
202
+ ### v3.6.0Knowledge Graph First
203
+
204
+ - **Unified ingestion pipeline** one entrypoint normalizes every source
205
+ (file, folder, web URL, browser tab, text/markdown/code) into the graph,
206
+ idempotent by content hash and bracketed by the `pre_tool`/`post_tool` hook
207
+ lifecycle.
208
+ - **Formalized entity/relationship model** — first-class `Source`, `Repository`,
209
+ `Meeting`, `Organization`, `Workflow`, `Agent` entities and `indexed_from`,
210
+ `modified_by`, `belongs_to_project`, `part_of`, `discussed_in`, `decided_by`,
211
+ `generated_by`, `used_by_agent` relationships.
212
+ - **Browser & web ingestion** — `POST /api/browser/read-url` and
213
+ `/ingest-current-tab`, plus a local Manifest V3 extension that posts only to
214
+ `127.0.0.1`.
215
+ - **Portability** — local Knowledge Graph export/import (versioned JSON) and
216
+ binary backup/restore (DB + blobs, integrity-checked). No cloud required.
217
+ - **Provenance** — every node records where it came from; a queryable audit trail
218
+ makes the graph explainable.
219
+ - **Knowledge Graph as the primary surface** — the view becomes your digital brain
220
+ with Status, Sources, Capture, and Backup tabs.
221
+
222
+ See [RELEASE_NOTES_v3.6.0.md](RELEASE_NOTES_v3.6.0.md),
223
+ [docs/kg-schema.md](docs/kg-schema.md),
224
+ [docs/RUNTIME_HOOK_COVERAGE_v3.6.0.md](docs/RUNTIME_HOOK_COVERAGE_v3.6.0.md), and
198
225
  [FEATURE_STATUS.md](FEATURE_STATUS.md).
199
226
 
200
- ## How it works
227
+ ## How it works — every source converges into the graph
228
+
229
+ As of v3.6.0, all data sources flow through **one unified ingestion pipeline** into
230
+ the Knowledge Graph — no source bypasses it, none becomes an isolated silo:
201
231
 
202
232
  ```text
203
- files / chats / notes / images / decisions
204
- -> workspace memory
205
- -> knowledge graph
206
- -> hybrid search
207
- -> chat / agents / workflows
208
- -> reusable outputs
233
+ source (file · folder · PDF · web URL · browser tab · text)
234
+ -> extraction -> normalization -> content hash (idempotent)
235
+ -> chunking -> entity detection -> relationship detection -> embedding
236
+ -> Knowledge Graph (Source -[indexed_from]- content -[contains]- chunks)
237
+ -> RAG / agents / memory / hybrid search
209
238
  ```
210
239
 
211
- - Your content stays on your machine and becomes durable workspace memory.
212
- - Memory is organized into a knowledge graph of entities and relationships.
213
- - Hybrid search fuses keyword, vector, and graph signals over that context.
214
- - Chat, agents, and workflows draw on the same grounded context.
215
- - Outputs documents, analysis, and decisions feed back into the workspace.
240
+ - **Every node is explainable.** Each ingested item carries provenance where it
241
+ came from, when, how it was processed, whether it was embedded or linked.
242
+ - **The graph is the asset.** Memory, search, and agents are views over it; models
243
+ read it. Swap a model and your knowledge is unchanged.
244
+ - **Portable, no cloud.** Export/import the graph as JSON, or take a full local
245
+ binary backup (DB + blobs) and restore it.
246
+ - **Local-first protects the graph.** It lives in local SQLite on your machine.
216
247
 
217
248
  For the deeper design, see [ARCHITECTURE.md](ARCHITECTURE.md) and
218
249
  [docs/architecture.md](docs/architecture.md).
@@ -247,16 +278,19 @@ For the deeper design, see [ARCHITECTURE.md](ARCHITECTURE.md) and
247
278
  ### Releases
248
279
 
249
280
  - [RELEASE_NOTES.md](RELEASE_NOTES.md) — current release notes
281
+ - [RELEASE_NOTES_v3.6.0.md](RELEASE_NOTES_v3.6.0.md)
282
+ - [RELEASE_NOTES_v3.5.0.md](RELEASE_NOTES_v3.5.0.md)
250
283
  - [RELEASE_NOTES_v3.4.1.md](RELEASE_NOTES_v3.4.1.md)
251
284
  - [RELEASE_NOTES_v3.4.0.md](RELEASE_NOTES_v3.4.0.md)
252
- - [RELEASE_NOTES_v3.3.0.md](RELEASE_NOTES_v3.3.0.md)
253
285
  - [CHANGELOG.md](CHANGELOG.md) and [docs/CHANGELOG.md](docs/CHANGELOG.md)
254
286
 
255
287
  ## Release History
256
288
 
257
289
  | Version | Theme |
258
290
  | --- | --- |
259
- | **3.4.1** | Runtime completionfull hooks lifecycle, real Local Agent probes, Connect Folder and Folder Watch verified end-to-end |
291
+ | **3.6.0** | Knowledge Graph First unified ingestion pipeline, formalized entity/relationship model, browser/web ingestion, local export/import/backup, provenance, KG as the primary surface |
292
+ | 3.5.0 | Foundation stabilization & verification — OIDC verifier, trusted-proxy gating, runtime hook coverage, `tools/` package, reproducible artifacts |
293
+ | 3.4.1 | Runtime completion — full hooks lifecycle, real Local Agent probes, Connect Folder and Folder Watch verified end-to-end |
260
294
  | 3.4.0 | Platform completion — hooks execution, uploads in Files, vision image input, agent run trigger, on-device Local Agent / Connect Folder / Folder Watch |
261
295
  | 3.3.1 | Visual product rebuild — rebuilt `/app` shell, Basic/Advanced/Admin navigation, refreshed design system |
262
296
  | **3.3.0** | Product quality & honesty release — evidence-based feature audit, single-source version truth, working document upload, documented design system |
@@ -0,0 +1,61 @@
1
+ # v3.5.0 → v3.6.0 Carry-Over Audit
2
+
3
+ **Date:** 2026-06-09
4
+ **Baseline:** v3.5.0 (tag published, GitHub Release live, CI + Visual Smoke green on `main`)
5
+ **Purpose:** Classify every open v3.5.0 carry-over risk as **blocking**, **non-blocking**, or
6
+ **obsolete** before starting v3.6.0 Knowledge Graph First work.
7
+
8
+ ## Headline
9
+
10
+ **No carry-over item blocks v3.6.0.** v3.5.0 was a stabilization/verification release that added no
11
+ product surface. Every documented limitation is either an intentional, honestly-labeled scope
12
+ boundary or a closed issue. KG work can start immediately on the existing local SQLite store.
13
+
14
+ ## Verified baseline state
15
+
16
+ | Check | Result | Evidence |
17
+ |---|---|---|
18
+ | `v3.5.0` git tag | exists | `git tag` |
19
+ | GitHub Release `v3.5.0` | published, not draft | `gh release view v3.5.0` |
20
+ | CI on `main` | success | `gh run list --branch main` (run 27155690240) |
21
+ | Visual Smoke on `main` | success | run 27155690270 |
22
+ | VSIX reproducibility fix | merged | commit `78deb95` |
23
+
24
+ ## Classified carry-over items
25
+
26
+ ### Blocking
27
+ *(none)*
28
+
29
+ ### Non-blocking
30
+
31
+ | Item | Why it does not block | Evidence |
32
+ |---|---|---|
33
+ | OIDC is RSA-only (RS256/384/512); ES*/HS*/`alg:none` rejected fail-closed | v3.6.0 KG work does not touch the SSO callback. EC support is additive when a provider needs it. | `latticeai/core/oidc.py:36` |
34
+ | Memory/KG maintenance endpoints (`/api/memory/{prune,compact,rebuild,clear}`) sit outside `pre_tool`/`post_tool` | Intentional, documented decision; these ops carry their own audit events. v3.6.0 follows the same convention and adds a coverage row rather than routing maintenance through `dispatch_tool`. | `docs/RUNTIME_HOOK_COVERAGE_v3.5.0.md:47` |
35
+ | Knowledge Graph is config-dependent on `LATTICEAI_ENABLE_GRAPH`, backed by a large `knowledge_graph.py` | This is the surface v3.6.0 builds on. Size/coupling is refactor context, addressed additively (new service modules, not a rewrite). | `FEATURE_STATUS.md:257-263` |
36
+ | Memory project/graph/vector tiers PARTIAL; prune/clear API-only (no UI) | KG-adjacent tiers v3.6.0 completes; honestly labeled, not broken. v3.6.0 adds the ingestion/export/provenance UI. | `FEATURE_STATUS.md:280-286` |
37
+ | Hybrid-search "fusion" explainer renders illustrative bars | Cosmetic placeholder; orthogonal to KG-first ingestion work. | `FEATURE_STATUS.md:249-251` |
38
+ | Chat grounding chips set state but `ChatRequest` drops them | Wiring grounding into generation is a natural KG-first feature; additive, not a blocker. | `FEATURE_STATUS.md:131-138` |
39
+ | CI action versions inconsistent (`ci.yml` `checkout@v4` vs `release.yml` `@v5`) | None are currently deprecated. Aligning is hygiene; addressed opportunistically. | `.github/workflows/ci.yml` |
40
+ | Live MCP tool calls + VLM inference PARTIAL (env/model dependent) | Honestly badged; orthogonal to KG-first work. | `FEATURE_STATUS.md:169-178,392` |
41
+
42
+ ### Obsolete
43
+
44
+ | Item | Why it is closed |
45
+ |---|---|
46
+ | "Hooks registered but not executing" (v3.3.0 issue) | v3.4.1 added real runners for all built-ins; v3.5.0 closed the last tool-path bypasses at 100% coverage. |
47
+ | Legacy `/account` `/admin` glassmorphism | Blur removed in v3.5.0 (`account.css:120`, `admin.css:48`). Remaining "not restyled" note is a deliberate scope boundary — these pages are outside the v3 SPA view set. |
48
+ | Vercel deployment returns HTTP 500 | Settled posture: Vercel is **landing/download/demo only, never runtime**. Lattice AI is local-first; the KG runs on local SQLite. Do not present any Vercel URL as a product surface. |
49
+ | CI syntax-gate staleness | Fixed in v3.5.0 by `scripts/check_python.py` (discover-based via `rglob`); new v3.6.0 modules are covered automatically with zero maintenance. |
50
+
51
+ ## v3.6.0 posture decisions taken from this audit
52
+
53
+ 1. **Build additively on the existing store.** Mirror the proven `tools.py → tools/` decomposition
54
+ pattern: preserve the import surface, add focused modules (`services/ingestion.py`,
55
+ `services/kg_portability.py`, `api/browser.py`, `api/portability.py`) rather than rewriting
56
+ `knowledge_graph.py`.
57
+ 2. **Route new ingestion paths through `dispatch_tool`** so `pre_tool`/`post_tool` fire — this is the
58
+ one v3.5.0 gap (no ingest path fired hooks). Maintenance ops stay audit-only by the documented
59
+ convention.
60
+ 3. **Keep Vercel landing-only and OIDC RSA-only** — settled, out of v3.6.0 scope.
61
+ 4. **Leave legacy `/account` `/admin` pages alone** — out of the local-first KG scope.
package/docs/CHANGELOG.md CHANGED
@@ -1,5 +1,37 @@
1
1
  # Changelog
2
2
 
3
+ ## [3.6.0] - 2026-06-10
4
+
5
+ > Knowledge Graph First. The Knowledge Graph becomes the primary architecture:
6
+ > every data source converges into it through one unified ingestion pipeline, with
7
+ > formalized entities/relationships, browser/web inputs, local portability, and
8
+ > per-node provenance. Lattice AI is a Digital Brain Platform — the graph is the
9
+ > durable asset; models read it and are replaceable.
10
+
11
+ ### Added
12
+
13
+ - Unified ingestion pipeline (`latticeai/services/ingestion.py`): one entrypoint
14
+ for files, folders, web URLs, browser tabs, and text — idempotent by content
15
+ hash, bracketed by `pre_tool`/`post_tool`.
16
+ - Knowledge Graph entities `Source`/`Repository`/`Meeting`/`Organization`/
17
+ `Workflow`/`Agent` and relationships `indexed_from`/`modified_by`/
18
+ `belongs_to_project`/`part_of`/`discussed_in`/`decided_by`/`generated_by`/
19
+ `used_by_agent` (additive, lossless `from_legacy`).
20
+ - Browser & web ingestion routes (`/api/browser/read-url`, `/ingest-current-tab`)
21
+ and a Manifest V3 extension scaffold that posts only to `127.0.0.1`.
22
+ - Knowledge Graph export/import (versioned JSON) and binary backup/restore
23
+ (`latticeai/services/kg_portability.py`,
24
+ `/api/knowledge-graph/{export,import,backup,restore,portability,provenance}`).
25
+ - Provenance trail (`ingestion_provenance` table + query API) — every node is
26
+ explainable.
27
+ - Knowledge Graph UI tabs: Status, Sources, Capture, Backup.
28
+
29
+ ### Changed
30
+
31
+ - KG ingestion now fires the tool hook lifecycle (closes the v3.5.0 gap);
32
+ coverage documented in `docs/RUNTIME_HOOK_COVERAGE_v3.6.0.md`.
33
+ - README repositioned as a Digital Brain Platform; Vercel remains landing-only.
34
+
3
35
  ## [3.3.1] - 2026-06-08
4
36
 
5
37
  > v3.3.1 — Visual Product Rebuild. The `/app` frontend keeps the same runtime
@@ -0,0 +1,46 @@
1
+ # Lattice AI v3.6.0 — Completion Record (Knowledge Graph First)
2
+
3
+ **Status: ✅ RELEASED — 2026-06-10.** All planned scopes implemented, tested,
4
+ committed, pushed, CI green, tagged, and published. No external publish/deploy.
5
+
6
+ - **Tag:** `v3.6.0` → commit `3c85675`
7
+ - **GitHub Release:** https://github.com/TaeSooPark-PTS/LatticeAI/releases/tag/v3.6.0
8
+ (published, not draft; assets: `ltcai-3.6.0-py3-none-any.whl`,
9
+ `ltcai-3.6.0.tar.gz`, `ltcai-3.6.0.tgz`, `ltcai-3.6.0.vsix`)
10
+ - **CI:** main CI ✓, Visual Smoke ✓, release.yml (tag) ✓
11
+ - **Tests:** unit 455 passing · lint 64/64 · check:python 153 (3.11/3.12/3.14) ·
12
+ release artifacts built + validated
13
+
14
+ ## Commits (v3.5.0 → v3.6.0)
15
+
16
+ | Commit | Scope |
17
+ |---|---|
18
+ | `baa2bf6` | chore(audit) — v3.5.0 carry-over (0 blocking) |
19
+ | `5a6a7d4` | feat(kg) — entities/relationships schema |
20
+ | `135e81a` | feat(kg) — unified ingestion pipeline + provenance |
21
+ | `b548885` | feat(browser) — browser/web ingestion + MV3 extension |
22
+ | `39a7a0c` | feat(kg) — export/import/backup/restore |
23
+ | `21cfb97` | fix(runtime) — hook coverage for ingestion paths |
24
+ | `7009e39` | fix(ui) — Knowledge Graph as primary surface |
25
+ | `fa89a84` | docs(philosophy) — Digital Brain Platform rewrite |
26
+ | `aa011a5` | release: v3.6.0 (version bump) |
27
+ | `3c85675` | fix(ci) — 3.11-compatible f-string (PEP 701 quote reuse) |
28
+
29
+ ## Carry-over audit result
30
+
31
+ Zero blocking items. Settled postures preserved: Vercel landing-only, OIDC
32
+ RSA-only, legacy `/account` `/admin` out of scope. The one honest v3.5.0 gap (KG
33
+ ingestion not firing tool hooks) is **closed**. Full detail:
34
+ `docs/CARRYOVER_AUDIT_v3.6.0.md`.
35
+
36
+ ## Key facts for future work
37
+
38
+ - New seams: `latticeai/services/ingestion.py` (single write-side entrypoint),
39
+ `latticeai/services/kg_portability.py`, `latticeai/api/browser.py`,
40
+ `latticeai/api/portability.py`, provenance in `knowledge_graph.py`.
41
+ - **Gotcha:** PEP 701 f-string quote reuse (`f'{x or ''}'`) compiles on 3.12+ but
42
+ is a SyntaxError on 3.11 — always run `python3.11 scripts/check_python.py`
43
+ before pushing (CI tests on 3.11 + 3.12).
44
+ - Version canonical: `WORKSPACE_OS_VERSION`; mirrors enforced by
45
+ `test_version_consistency.py`.
46
+ - Local tests need `.venv/bin/python` (system `python3` lacks fastapi).
@@ -0,0 +1,49 @@
1
+ # Runtime Hook Coverage — v3.6.0
2
+
3
+ v3.6.0 makes the Knowledge Graph the primary architecture and adds new
4
+ **ingestion** paths (web URL, browser tab, unified text/file pipeline) plus
5
+ **portability** ops (export/import/backup/restore). This doc extends
6
+ [`RUNTIME_HOOK_COVERAGE_v3.5.0.md`](./RUNTIME_HOOK_COVERAGE_v3.5.0.md) and records
7
+ that the new data-mutating paths run through the unified lifecycle.
8
+
9
+ The single tool path is `dispatch_tool(hooks, name, args, run_fn)` in
10
+ `latticeai/core/hooks.py` (`pre_tool → execute → post_tool`). v3.5.0's one honest
11
+ gap was that **KG ingestion did not fire hooks**. v3.6.0 closes it: every source
12
+ now flows through `IngestionPipeline.ingest` (`latticeai/services/ingestion.py`),
13
+ which wraps the store write in `dispatch_tool(..., source="ingestion")`.
14
+
15
+ **Result.** All v3.5.0 coverage is preserved (no regression), and every new
16
+ v3.6.0 ingestion path is covered. Portability admin/maintenance ops follow the
17
+ v3.5.0 convention for service maintenance (own audit events), documented below.
18
+
19
+ ## New v3.6.0 execution paths
20
+
21
+ | Entrypoint | Execution | Lifecycle path | pre fired | post fired | Test |
22
+ |---|---|---|---|---|---|
23
+ | `IngestionPipeline.ingest` (any source) | `ingest_source` / `ingest_document` | `dispatch_tool(name="kg_ingest.<type>", source="ingestion")` | yes (`pre_tool`) | yes (`post_tool`) | `test_ingestion_pipeline` |
24
+ | `POST /api/browser/read-url` | fetch URL → `pipeline.ingest` (web_url) | pipeline → `dispatch_tool` | yes | yes | `test_browser_ingestion`, `test_runtime_coverage_v36` |
25
+ | `POST /api/browser/ingest-current-tab` | sanitize → `pipeline.ingest` (browser_tab) | pipeline → `dispatch_tool` | yes | yes | `test_browser_ingestion`, `test_runtime_coverage_v36` |
26
+ | Local file / upload via pipeline | `ingest_document` | pipeline → `dispatch_tool` | yes | yes | `test_ingestion_pipeline` |
27
+ | Provenance write per ingestion | `record_provenance` | inside the bracketed `dispatch_tool` run_fn | (bracketed) | (bracketed) | `test_ingestion_pipeline` |
28
+
29
+ A blocking `pre_tool` hook makes ingestion return `status="blocked"` (the
30
+ `PermissionError` from `dispatch_tool` is caught and surfaced honestly), exactly
31
+ mirroring how a blocked tool call is handled — verified in
32
+ `test_runtime_coverage_v36` and `test_ingestion_pipeline`.
33
+
34
+ ## Intentionally outside the tool lifecycle (documented, not gaps)
35
+
36
+ | Entrypoint | Why not `pre_tool`/`post_tool` |
37
+ |---|---|
38
+ | `POST /api/knowledge-graph/{export,export-file,backup,restore,import}` | Admin **portability/maintenance** operations over the whole machine-global graph, not agent-vocabulary tools. They are admin-gated (`require_admin`) and recorded via the platform audit trail — same convention as the v3.5.0 memory maintenance ops (`prune/compact/rebuild/clear`). Wrapping a whole-store backup in `pre_tool` would misrepresent it as a per-action agent tool. |
39
+ | `read_document` inside upload (`upload_service.py`) | Already inside the `pre_upload`→`post_upload` lifecycle (unchanged from v3.5.0). |
40
+ | `POST /api/memory/{prune,compact,rebuild,clear}` | Unchanged from v3.5.0 — service maintenance with own audit events. |
41
+ | Read-only KG reads (`/knowledge-graph/{stats,graph,search,...}`, `/api/knowledge-graph/portability` status) | Execute no mutation — nothing to gate. |
42
+
43
+ ## Summary
44
+
45
+ - v3.5.0 coverage of tool/agent execution paths: **100%, preserved** (no regression).
46
+ - v3.6.0 gap closed: **KG ingestion now fires `pre_tool`/`post_tool`** via the unified pipeline (the one honest carry-over note from v3.5.0).
47
+ - New ingestion paths (unified pipeline, `read-url`, `ingest-current-tab`): **covered**.
48
+ - Portability admin ops: **documented as audit-gated maintenance**, consistent with the v3.5.0 convention — not bypasses.
49
+ - Coverage of discovered mutating ingestion paths: **100%**.
@@ -1,17 +1,18 @@
1
1
  # Lattice AI Architecture
2
2
 
3
- > v3.3.1feature-complete for non-enterprise use cases with a rebuilt `/app`
4
- > visual shell. The agent ecosystem
5
- > (registry, marketplace + templates, workflow agents, autonomous planning),
6
- > the long-term memory platform + manager, and the skills/hooks/tool/MCP
7
- > registries are all operable from `/app`. Enterprise controls remain future
8
- > work.
9
-
10
- Lattice AI is a local-first **AI workspace, AI pipeline platform, Knowledge
11
- Graph platform, and multi-agent workflow platform**. The architecture is
12
- organized around one durable center: the Knowledge Graph. Models, tools,
13
- agents, workflows, and UI modes are replaceable layers that operate on top of
14
- workspace and graph context.
3
+ > v3.6.0**Knowledge Graph First.** Every data source converges into the graph
4
+ > through one unified ingestion pipeline (`latticeai/services/ingestion.py`), with
5
+ > formalized entities/relationships (`docs/kg-schema.md`), browser/web inputs,
6
+ > per-node provenance, and local export/import/backup
7
+ > (`latticeai/services/kg_portability.py`). The agent ecosystem, long-term memory,
8
+ > and skills/hooks/tool/MCP registries are all operable from `/app`. Enterprise
9
+ > controls remain future work.
10
+
11
+ Lattice AI is a local-first **Digital Brain Platform**. The architecture is
12
+ organized around one durable center and the user's asset: the **Knowledge
13
+ Graph**. Models, tools, agents, RAG, workflows, and UI modes are replaceable
14
+ layers that operate as views over graph context. Models are replaceable;
15
+ knowledge is durable.
15
16
 
16
17
  ## Architecture Goals
17
18
 
package/docs/kg-schema.md CHANGED
@@ -83,6 +83,61 @@ Edge {
83
83
 
84
84
  ---
85
85
 
86
+ ## v3.6.0 — Knowledge Graph First 엔티티/관계
87
+
88
+ v3.6.0 은 "모든 데이터 소스가 Knowledge Graph 로 수렴한다"는 원칙을 1급 스키마로
89
+ 승격한다. 아래 타입은 **추가형(additive)**이다 — 기존 enum/legacy 매핑을 깨지 않고
90
+ `from_legacy` 가 무손실로 정규화하며, 알 수 없는 타입은 여전히 `CONCEPT`/`MENTIONS` 로
91
+ 폴백한다. 스키마는 **확장 가능**하게 유지한다: 새 도메인 엔티티는 enum 멤버 1개 +
92
+ `_LEGACY_NODE_MAP`/`_LEGACY_EDGE_MAP` 별칭만 추가하면 된다.
93
+
94
+ ### 추가 노드 타입
95
+
96
+ | 타입 | 의미 | 대표 `attrs` / 출처 |
97
+ |------|------|--------------------|
98
+ | `SOURCE` | 수집 출처(파일/URL/브라우저 탭/git 등)의 **출처 노드** | `source_type`, `source_uri`, `content_hash`, `captured_at` |
99
+ | `REPOSITORY` | git 저장소 | `remote`, `branch`, `head` |
100
+ | `MEETING` | 회의 / 미팅 | `started_at`, `attendees[]` |
101
+ | `ORGANIZATION` | 조직 / 회사 / 팀 | `domain`, `members[]` |
102
+ | `WORKFLOW` | 워크플로우 정의/실행 | `workflow_id`, `status` |
103
+ | `AGENT` | 에이전트(역할/실행 주체) | `role`, `model_id` |
104
+
105
+ ### 추가 엣지 타입
106
+
107
+ | 타입 | 허용 source → target | 의미 |
108
+ |------|---------------------|------|
109
+ | `INDEXED_FROM` | ANY → `SOURCE` | 이 노드가 **어떤 출처에서 색인**됐는가 (provenance) |
110
+ | `MODIFIED_BY` | ANY → `PERSON` | 마지막 수정자 |
111
+ | `BELONGS_TO_PROJECT` | ANY → `PROJECT` | 프로젝트 귀속 |
112
+ | `PART_OF` | ANY → ANY | 구성요소 관계 |
113
+ | `DISCUSSED_IN` | `CONCEPT`·`DECISION` → `MEETING`·`CHAT` | 어디에서 논의됨 |
114
+ | `DECIDED_BY` | `DECISION` → `PERSON` | 결정 주체 |
115
+ | `GENERATED_BY` | ANY → `AGENT`·`MODEL`·`WORKFLOW` | 생성 주체 |
116
+ | `USED_BY_AGENT` | ANY → `AGENT` | 에이전트가 사용함 |
117
+
118
+ ### 통합 수집 형태 (Unified Ingestion)
119
+
120
+ 모든 출처는 동일한 형태로 그래프에 들어온다:
121
+
122
+ ```
123
+ SOURCE ──INDEXED_FROM◄── Document/File ──CONTAINS──► Chunk[]
124
+ ▲ │
125
+ │ └──(언급/포함)──► Concept / Task / Decision …
126
+ provenance(source_type, source_uri, content_hash, captured_at, modified_at,
127
+ owner, workspace_id, permissions, pipeline, embedded, linked)
128
+ ```
129
+
130
+ - **콘텐츠 노드**(Document/File/web 노드)는 `content_hash` 로 멱등(idempotent) 처리된다 —
131
+ 같은 콘텐츠를 다시 수집하면 새 노드를 만들지 않고 갱신/링크한다.
132
+ - 모든 콘텐츠 노드는 `SOURCE` 노드로 `INDEXED_FROM` 엣지를 가져 **출처를 항상 설명 가능**하다.
133
+ - provenance 는 노드 `metadata.provenance` 에 임베드되며, 동시에 감사 가능한
134
+ `ingestion_provenance` 테이블에 기록된다 (`KnowledgeGraphStore.get_provenance(node_id)`).
135
+
136
+ 구현: `latticeai/services/ingestion.py` (`IngestionPipeline`) 가 단일 진입점이며,
137
+ 파일/로컬폴더/URL/브라우저 탭/텍스트를 모두 이 형태로 정규화한다.
138
+
139
+ ---
140
+
86
141
  ## 예시 (PPT 슬라이드 22 와 동일)
87
142
 
88
143
  ```json
package/docs/privacy.md CHANGED
@@ -13,8 +13,10 @@
13
13
  | 사용자 계정 | `~/.ltcai/users.json` | 이름, scrypt 해시 비밀번호, 역할 |
14
14
  | 세션 토큰 | `~/.ltcai/sessions.json` | UUID 토큰, 만료시간 |
15
15
  | 채팅 히스토리 | `~/.ltcai/chat_history.json` | 사용자-AI 대화 내용 |
16
- | 지식 그래프 | `~/.ltcai/knowledge_graph.sqlite` | 채팅/문서 노드/엣지 |
17
- | 업로드 파일 | `~/.ltcai/knowledge_graph_blobs/` | 원본 PDF/DOCX 등 |
16
+ | 지식 그래프 | `~/.ltcai/knowledge_graph.sqlite` | 채팅/문서/웹/탭 노드·엣지, 프로비넌스 |
17
+ | 업로드/수집 파일 | `~/.ltcai/knowledge_graph_blobs/` | 원본 PDF/DOCX/웹 텍스트 등 |
18
+ | 수집 출처 기록(프로비넌스) | `~/.ltcai/knowledge_graph.sqlite` (`ingestion_provenance`) | 각 노드의 출처/시각/처리 방식 — 외부 전송 없음 |
19
+ | 그래프 내보내기/백업 | `~/.ltcai/workspace_exports/` | 사용자가 직접 만든 로컬 export/backup 파일 (클라우드 미사용) |
18
20
  | 지식 정원 | `~/.ltcai-brain/` | P-Reinforce 분류 저장 |
19
21
  | 설정 | `~/.ltcai/config.json` | 모델 설정, API 키 (keyring) |
20
22
 
@@ -36,6 +38,20 @@
36
38
 
37
39
  Apple Silicon MLX 로컬 모델 사용 시에는 프롬프트가 외부로 전송되지 않습니다.
38
40
 
41
+ ## 웹/브라우저 수집 (v3.6.0)
42
+
43
+ - **URL 읽기**(`/api/browser/read-url`): 사용자가 명시적으로 요청한 URL을 **로컬
44
+ 런타임이 직접** 가져와 텍스트만 추출해 그래프에 색인합니다. Lattice AI가 임의로
45
+ 크롤링하지 않으며, 가져온 페이지는 외부로 다시 전송되지 않습니다.
46
+ - **브라우저 탭 수집**(`/api/browser/ingest-current-tab`) 및 Manifest V3 확장:
47
+ 확장 프로그램은 **오직 `127.0.0.1`(로컬)** 로만 전송합니다. 클라우드 엔드포인트가
48
+ 존재하지 않습니다(`browser-extension/` 소스에서 단일 `fetch` 대상 확인 가능).
49
+
50
+ ## 그래프 내보내기/백업 (v3.6.0)
51
+
52
+ 지식 그래프 export/import 및 백업/복원은 **전적으로 로컬**에서 동작하며 클라우드
53
+ 서비스를 요구하지 않습니다. 내보낸 파일의 이동·공유는 사용자 책임입니다.
54
+
39
55
  ## API 키 보안
40
56
 
41
57
  - API 키는 OS keyring(macOS Keychain, Windows Credential Manager, Linux Secret Service)에 저장됩니다
@@ -78,6 +78,23 @@ MAGIC_NUMBERS = {
78
78
  - 업로드 시 파일 첫 바이트와 확장자 매핑 검증
79
79
  - 불일치 시 400 에러
80
80
 
81
+ ## 수집 & 그래프 포터빌리티 보안 (v3.6.0)
82
+
83
+ - **수집 라이프사이클**: 모든 수집은 `IngestionPipeline.ingest` → `dispatch_tool`
84
+ 를 거쳐 `pre_tool`/`post_tool` 훅이 발화됩니다. `pre_tool`이 차단하면 수집은
85
+ 정직하게 `status="blocked"`로 거부됩니다(권한 게이트·민감정보 가드 적용).
86
+ - **웹 URL 읽기**(`/api/browser/read-url`): `http(s)` 스킴만 허용(파일/기타 스킴
87
+ 거부), 12초 타임아웃, 4MB 응답 상한, HTML/text 컨텐츠 타입만 처리. 차단/로그인
88
+ 필요 페이지는 5xx가 아닌 **422로 우아하게 실패**합니다. 로컬 런타임이 사용자가
89
+ 명시한 URL만 가져옵니다(자동 크롤링 없음).
90
+ - **브라우저 탭 수집**(`/api/browser/ingest-current-tab`): payload 정화(스크립트/
91
+ 스타일 제거) + 페이로드 크기 상한(413). Manifest V3 확장은 **`127.0.0.1`로만**
92
+ 전송하며 클라우드 엔드포인트가 없습니다.
93
+ - **포터빌리티 권한**: 그래프 export/status 읽기는 로그인 사용자, **import /
94
+ backup / restore 는 admin 전용**(`require_admin`). 그래프는 머신-전역 자원입니다.
95
+ - **복원 무결성**: 백업 아카이브는 `manifest.json`의 sha256과 대조 검증 후에만
96
+ 복원되며, 불일치 시 거부됩니다.
97
+
81
98
  ## 에이전트 도구 샌드박스
82
99
 
83
100
  ### `run_command()` 위험 플래그 차단
package/kg_schema.py CHANGED
@@ -99,6 +99,15 @@ class NodeType(str, Enum):
99
99
  DECISION = "DECISION" # 결정 사항
100
100
  ERROR = "ERROR" # 오류 / 버그
101
101
  EVENT = "EVENT" # 분석/시스템 이벤트(동적 타입 폴백)
102
+ # v3.6.0 Knowledge Graph First — 모든 데이터 소스가 그래프로 수렴하기 위한
103
+ # 1급 엔티티. 추가형(additive)·확장 가능(extensible): 새 도메인 엔티티는
104
+ # 여기에 enum 멤버를 추가하고 _LEGACY_NODE_MAP 에 별칭만 등록하면 된다.
105
+ SOURCE = "SOURCE" # 수집 출처(파일/URL/브라우저 탭/git 등)의 출처 노드
106
+ REPOSITORY = "REPOSITORY" # git 저장소
107
+ MEETING = "MEETING" # 회의 / 미팅
108
+ ORGANIZATION = "ORGANIZATION" # 조직 / 회사 / 팀
109
+ WORKFLOW = "WORKFLOW" # 워크플로우 정의/실행
110
+ AGENT = "AGENT" # 에이전트(역할/실행 주체)
102
111
 
103
112
  @classmethod
104
113
  def from_legacy(cls, label: str) -> "NodeType":
@@ -143,6 +152,16 @@ class EdgeType(str, Enum):
143
152
  DISCUSSES = "DISCUSSES" # SLIDE/PAGE → TOPIC (discusses)
144
153
  IMPLIES = "IMPLIES" # NODE → NODE (implies)
145
154
  RELATED_TO = "RELATED_TO" # ANY ↔ ANY (related_to)
155
+ # v3.6.0 Knowledge Graph First — 출처/소유/구성/결정 관계를 1급 엣지로 승격.
156
+ # 추가형: 새 관계는 enum 멤버 추가 + _LEGACY_EDGE_MAP 별칭 등록만으로 확장된다.
157
+ INDEXED_FROM = "INDEXED_FROM" # NODE → SOURCE (어떤 출처에서 색인됐는가)
158
+ MODIFIED_BY = "MODIFIED_BY" # NODE → PERSON (마지막 수정자)
159
+ BELONGS_TO_PROJECT = "BELONGS_TO_PROJECT" # NODE → PROJECT
160
+ PART_OF = "PART_OF" # NODE → NODE (구성요소 관계)
161
+ DISCUSSED_IN = "DISCUSSED_IN" # CONCEPT/DECISION → MEETING/CHAT
162
+ DECIDED_BY = "DECIDED_BY" # DECISION → PERSON
163
+ GENERATED_BY = "GENERATED_BY" # NODE → AGENT/MODEL/WORKFLOW
164
+ USED_BY_AGENT = "USED_BY_AGENT" # NODE → AGENT (에이전트가 사용함)
146
165
 
147
166
  @classmethod
148
167
  def from_legacy(cls, label: str) -> "EdgeType":
@@ -199,6 +218,19 @@ _LEGACY_NODE_MAP: Dict[str, NodeType] = {
199
218
  "보고서": NodeType.DOCUMENT,
200
219
  "계획서": NodeType.DOCUMENT,
201
220
  "기획서": NodeType.DOCUMENT,
221
+ # v3.6.0 Knowledge Graph First 엔티티
222
+ "source": NodeType.SOURCE,
223
+ "ingestionsource": NodeType.SOURCE,
224
+ "repository": NodeType.REPOSITORY,
225
+ "repo": NodeType.REPOSITORY,
226
+ "gitrepo": NodeType.REPOSITORY,
227
+ "meeting": NodeType.MEETING,
228
+ "organization": NodeType.ORGANIZATION,
229
+ "org": NodeType.ORGANIZATION,
230
+ "company": NodeType.ORGANIZATION,
231
+ "team": NodeType.ORGANIZATION,
232
+ "workflow": NodeType.WORKFLOW,
233
+ "agent": NodeType.AGENT,
202
234
  }
203
235
 
204
236
  _LEGACY_EDGE_MAP: Dict[str, EdgeType] = {
@@ -254,6 +286,20 @@ _LEGACY_EDGE_MAP: Dict[str, EdgeType] = {
254
286
  "영감받음": EdgeType.INSPIRED_BY,
255
287
  "상충함": EdgeType.CONTRADICTS,
256
288
  "발전함": EdgeType.EVOLVES_FROM,
289
+ # v3.6.0 Knowledge Graph First 관계
290
+ "indexed_from": EdgeType.INDEXED_FROM,
291
+ "modified_by": EdgeType.MODIFIED_BY,
292
+ "belongs_to_project": EdgeType.BELONGS_TO_PROJECT,
293
+ "belongs_to": EdgeType.BELONGS_TO_PROJECT,
294
+ "part_of": EdgeType.PART_OF,
295
+ "discussed_in": EdgeType.DISCUSSED_IN,
296
+ "decided_by": EdgeType.DECIDED_BY,
297
+ "generated_by": EdgeType.GENERATED_BY,
298
+ "used_by_agent": EdgeType.USED_BY_AGENT,
299
+ "색인됨": EdgeType.INDEXED_FROM,
300
+ "수정함": EdgeType.MODIFIED_BY,
301
+ "결정함": EdgeType.DECIDED_BY,
302
+ "구성요소": EdgeType.PART_OF,
257
303
  }
258
304
 
259
305
  # ── SQLite v2 store ─────────────────────────────────────────────────────────