ltcai 10.10.0 → 11.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -42
- package/docs/CHANGELOG.md +68 -0
- package/docs/COMMUNITY_AND_PLUGINS.md +1 -1
- package/docs/DEVELOPMENT.md +1 -1
- package/docs/ONBOARDING.md +1 -1
- package/docs/OPERATIONS.md +1 -1
- package/docs/PERFORMANCE.md +135 -11
- package/docs/TRUST_MODEL.md +1 -1
- package/docs/WHY_LATTICE.md +1 -1
- package/docs/kg-schema.md +1 -1
- package/docs/v11.1.0_PRODUCT_INTELLIGENCE_PLAN.md +313 -0
- package/lattice_brain/__init__.py +1 -1
- package/lattice_brain/embeddings.py +12 -37
- package/lattice_brain/graph/_kg_contract.py +11 -0
- package/lattice_brain/graph/curator.py +1 -1
- package/lattice_brain/graph/discovery_index.py +30 -32
- package/lattice_brain/graph/fusion.py +184 -3
- package/lattice_brain/graph/image_vectors.py +230 -0
- package/lattice_brain/graph/ingest.py +11 -5
- package/lattice_brain/graph/proactive.py +138 -1
- package/lattice_brain/graph/projection.py +10 -2
- package/lattice_brain/graph/provenance.py +27 -2
- package/lattice_brain/graph/retrieval.py +249 -8
- package/lattice_brain/graph/retrieval_docgen.py +23 -23
- package/lattice_brain/graph/retrieval_policy.py +6 -0
- package/lattice_brain/graph/retrieval_reads.py +188 -1
- package/lattice_brain/graph/retrieval_vector.py +474 -110
- package/lattice_brain/graph/schema.py +134 -2
- package/lattice_brain/graph/vector_index/__init__.py +85 -0
- package/lattice_brain/graph/vector_index/base.py +170 -0
- package/lattice_brain/graph/vector_index/brute_force.py +114 -0
- package/lattice_brain/graph/vector_index/hnsw.py +293 -0
- package/lattice_brain/graph/vector_index/jobs.py +287 -0
- package/lattice_brain/graph/vector_index/quantized.py +151 -0
- package/lattice_brain/graph/vector_index/selector.py +131 -0
- package/lattice_brain/ingestion.py +259 -7
- package/lattice_brain/multimodal.py +738 -0
- package/lattice_brain/portability.py +654 -2
- package/lattice_brain/runtime/agent_runtime.py +1 -1
- package/lattice_brain/runtime/contracts.py +1 -1
- package/lattice_brain/runtime/multi_agent.py +1 -1
- package/lattice_brain/self_model.py +675 -0
- package/lattice_brain/synthesis.py +801 -0
- package/latticeai/__init__.py +1 -1
- package/latticeai/api/brain_intelligence.py +83 -1
- package/latticeai/api/chat_stream.py +4 -1
- package/latticeai/api/local_files.py +62 -0
- package/latticeai/api/memory.py +128 -1
- package/latticeai/api/models.py +1 -1
- package/latticeai/api/portability.py +130 -1
- package/latticeai/api/security_dashboard.py +48 -13
- package/latticeai/api/voice_capture.py +4 -1
- package/latticeai/api/workspace.py +11 -5
- package/latticeai/core/agent.py +5 -1
- package/latticeai/core/agent_prompts.py +66 -0
- package/latticeai/core/context_builder.py +92 -8
- package/latticeai/core/embedding_providers.py +548 -1
- package/latticeai/core/legacy_compatibility.py +1 -1
- package/latticeai/core/marketplace.py +1 -1
- package/latticeai/core/messages.py +51 -0
- package/latticeai/core/model_compat.py +2 -2
- package/latticeai/core/tool_registry.py +0 -7
- package/latticeai/core/workspace_os.py +43 -0
- package/latticeai/core/workspace_os_constants.py +1 -1
- package/latticeai/core/workspace_os_utils.py +4 -50
- package/latticeai/core/workspace_reorganization.py +335 -0
- package/latticeai/core/workspace_review_items.py +12 -1
- package/latticeai/integrations/telegram_bot.py +28 -10
- package/latticeai/models/router.py +1 -1
- package/latticeai/runtime/access_runtime.py +1 -1
- package/latticeai/runtime/build_phases.py +5 -2
- package/latticeai/runtime/network_boundary_wiring.py +9 -5
- package/latticeai/runtime/permission_mode_wiring.py +9 -6
- package/latticeai/runtime/persistence_runtime.py +41 -4
- package/latticeai/runtime/router_registration.py +3 -0
- package/latticeai/runtime/runtime_context.py +1 -0
- package/latticeai/services/architecture_readiness.py +1 -1
- package/latticeai/services/brain_intelligence.py +253 -0
- package/latticeai/services/change_proposals.py +50 -10
- package/latticeai/services/memory_service.py +35 -1
- package/latticeai/services/model_catalog.py +4 -3
- package/latticeai/services/model_engines.py +28 -14
- package/latticeai/services/multimodal_ports.py +87 -0
- package/latticeai/services/obsidian_bridge.py +618 -0
- package/latticeai/services/product_readiness.py +1 -1
- package/latticeai/services/self_model_service.py +171 -0
- package/latticeai/services/voice_capture.py +27 -1
- package/latticeai/tools/filesystem.py +4 -1
- package/package.json +1 -1
- package/scripts/bench_vector_index.py +295 -0
- package/scripts/check_current_release_docs.mjs +1 -1
- package/scripts/release_screen_claims.json +26 -0
- package/src-tauri/Cargo.lock +1 -1
- package/src-tauri/Cargo.toml +1 -1
- package/src-tauri/tauri.conf.json +1 -1
- package/static/app/asset-manifest.json +37 -37
- package/static/app/assets/{Act-CC8SpNRm.js → Act-D0HWqtn0.js} +1 -1
- package/static/app/assets/{AdminConsole-DGIY_Tb_.js → AdminConsole-D-QDW-A4.js} +1 -1
- package/static/app/assets/{Brain-BDSC2Mol.js → Brain-CzCsI1mi.js} +1 -1
- package/static/app/assets/BrainHome-Btns-_TA.js +2 -0
- package/static/app/assets/BrainSignals-2dHQNkns.js +1 -0
- package/static/app/assets/{Capture-ClcKLsmI.js → Capture-CT8v1StE.js} +1 -1
- package/static/app/assets/{CommandPalette-DpwbyF1S.js → CommandPalette-DoLXC2KH.js} +1 -1
- package/static/app/assets/{Library-D8VNtffM.js → Library-DDoxFE5c.js} +1 -1
- package/static/app/assets/{LivingBrain-CeWwc0NN.js → LivingBrain-BXMWIK_2.js} +1 -1
- package/static/app/assets/{ProductFlow-CS5CCM4t.js → ProductFlow-DOYf7JIs.js} +1 -1
- package/static/app/assets/{ReviewCard-Bap4yPfI.js → ReviewCard-COQsqidK.js} +1 -1
- package/static/app/assets/{System-odPxU1YL.js → System-BRllvYXd.js} +1 -1
- package/static/app/assets/arrow-left-DnyMzss-.js +1 -0
- package/static/app/assets/{bot-CmvhqTeF.js → bot-4BvN07ux.js} +1 -1
- package/static/app/assets/brain-uMb_5hnO.js +1 -0
- package/static/app/assets/{button-DyppYhFH.js → button-CDjtnAoU.js} +1 -1
- package/static/app/assets/{circle-pause-Bm5z7EBI.js → circle-pause-D_RMn7tp.js} +1 -1
- package/static/app/assets/{circle-play-GjKdH0Fg.js → circle-play-B5OpB8ae.js} +1 -1
- package/static/app/assets/{cpu-CuVMv-F5.js → cpu-BIlWInHf.js} +1 -1
- package/static/app/assets/{download-C9_946tf.js → download-BtjXfL3z.js} +1 -1
- package/static/app/assets/{folder-open-B3QKdxib.js → folder-open-DefMpxI2.js} +1 -1
- package/static/app/assets/{hard-drive-BwW9Hx4r.js → hard-drive-BQ8NZVkw.js} +1 -1
- package/static/app/assets/{index-BrUQ-bF8.js → index-0AvoEBzJ.js} +3 -3
- package/static/app/assets/index-vtEfYvQY.css +2 -0
- package/static/app/assets/{input-Db2Hw2VH.js → input-B_5ZJ9oy.js} +1 -1
- package/static/app/assets/{permissionCopy-BpbIHRio.js → permissionCopy-BqZ5tsgL.js} +1 -1
- package/static/app/assets/{primitives-4B_Ynf7r.js → primitives-CVwew78r.js} +1 -1
- package/static/app/assets/search-DkhnOKZt.js +1 -0
- package/static/app/assets/{share-2-BgvRTDPJ.js → share-2-D5zg_0fY.js} +1 -1
- package/static/app/assets/{shield-alert-DWFLyykG.js → shield-alert-B5pZzkUb.js} +1 -1
- package/static/app/assets/{textarea-Bk4vEqzT.js → textarea-nEVIweKY.js} +1 -1
- package/static/app/assets/{useFocusTrap-CGRXEg_p.js → useFocusTrap-Cm99AHlz.js} +1 -1
- package/static/app/assets/{useQuery-CdPzp3_U.js → useQuery-Dm__N6bL.js} +1 -1
- package/static/app/assets/{utils-orU8igel.js → utils-DcDMoZIe.js} +1 -1
- package/static/app/assets/{workspace-Brj2Ylb2.js → workspace-LtRRSKTf.js} +1 -1
- package/static/app/index.html +4 -4
- package/static/sw.js +1 -1
- package/static/app/assets/BrainHome-BYgYp3ON.js +0 -2
- package/static/app/assets/BrainSignals-DYwVFmcO.js +0 -1
- package/static/app/assets/arrow-left-J47Yde1U.js +0 -1
- package/static/app/assets/brain-DLT9uuGq.js +0 -1
- package/static/app/assets/index-CYb18LtP.css +0 -2
- package/static/app/assets/search-YH-7-x5Y.js +0 -1
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
[](https://github.com/TaeSooPark-PTS/LatticeAI/actions/workflows/ci.yml)
|
|
12
12
|
[](LICENSE)
|
|
13
13
|
|
|
14
|
-

|
|
15
15
|
|
|
16
16
|
Chat, files, folders, notes, and web pages all flow into one durable knowledge
|
|
17
17
|
graph on your computer. Any model — local MLX or cloud — can speak with that
|
|
@@ -24,10 +24,10 @@ memory. Nothing leaves your machine without explicit consent.
|
|
|
24
24
|
|
|
25
25
|
| | |
|
|
26
26
|
| --- | --- |
|
|
27
|
-
| **Chat with a Brain that remembers** — every conversation grows durable, source-linked memory  | **See how knowledge connects** — a real relationship graph, not a file list  |
|
|
28
|
+
| **Capture anything** — files, whole folders, notes, screenshots, web pages  | **Automate with review** — agent changes become proposals you approve first  |
|
|
29
|
+
| **Pick a model in one click** — recommended local models for your hardware  | **Stay in control** — audit, roles, retention in a separate admin surface  |
|
|
30
|
+
| **Watch a file become memory** — three named steps, not a pipeline diagram  | **Say how much it may do alone** — one dial in plain words; dangerous actions stay blocked either way  |
|
|
31
31
|
|
|
32
32
|
## Why Lattice AI
|
|
33
33
|
|
|
@@ -58,52 +58,64 @@ First-run flow — wake the Brain, pick the owner, load a recommended model:
|
|
|
58
58
|
|
|
59
59
|
| | | |
|
|
60
60
|
| --- | --- | --- |
|
|
61
|
-
|  |  |  |
|
|
62
62
|
|
|
63
63
|
Screenshot index and capture notes:
|
|
64
|
-
[output/release/
|
|
64
|
+
[output/release/v11.1.0/SCREENSHOT_INDEX.md](output/release/v11.1.0/SCREENSHOT_INDEX.md)
|
|
65
65
|
|
|
66
66
|
## Current Release
|
|
67
67
|
|
|
68
|
-
The current release is **
|
|
69
|
-
|
|
70
|
-
The
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
-
|
|
95
|
-
|
|
96
|
-
|
|
68
|
+
The current release is **11.1.0 — Product Intelligence**:
|
|
69
|
+
|
|
70
|
+
The v9–v11.0 line hardened the foundation — proposal-first trust, honest
|
|
71
|
+
signals, a 100% line-and-branch test floor. 11.1.0 builds the intelligence
|
|
72
|
+
layer on top of it: **the Brain gets fast at scale, notices things on its
|
|
73
|
+
own, remembers pictures and recordings, learns who you are, and connects to
|
|
74
|
+
the tools you already use.**
|
|
75
|
+
|
|
76
|
+
- **Fast at scale.** A pluggable vector-index layer (brute-force default,
|
|
77
|
+
int8 quantized and HNSW opt-in via the `hnsw` extra) plus a durable
|
|
78
|
+
background embed queue. Measured on Apple Silicon: hybrid search p50 at
|
|
79
|
+
10k vectors went from 299 ms to **10.1 ms**, and stays at 43.9 ms at 50k
|
|
80
|
+
(recall@10 0.987) — the plan's <50 ms target met at 5× the target corpus.
|
|
81
|
+
Approximate results say `approx: true`; quantized's honest verdict (no RAM
|
|
82
|
+
win here, ~2.2× slower) is printed, not hidden.
|
|
83
|
+
- **Alive, not just searchable.** Contradiction detection now files
|
|
84
|
+
review-queue proposals with plain-language resolutions; approving one
|
|
85
|
+
stamps the temporal model (`valid_from`/`valid_to`/`superseded_by`, with
|
|
86
|
+
`as_of(timestamp)` slicing). Event-driven synthesis proposes parent
|
|
87
|
+
concepts, missing links and a proactive Brain Brief after every 25th
|
|
88
|
+
ingest — every write goes through the proposal path, asserted by tests.
|
|
89
|
+
- **Pictures and recordings are memories.** Behind `allow_multimodal`
|
|
90
|
+
(default off, off ⇒ byte-identical): images become first-class `Image`
|
|
91
|
+
nodes with OCR text, real captions only when a vision model produced one
|
|
92
|
+
(the caption-fabricating stub was deleted), separate image vectors with
|
|
93
|
+
late fusion, and inline thumbnails in the Evidence panel that never bypass
|
|
94
|
+
the local-file approval gate. Recordings are first-class `Audio` nodes
|
|
95
|
+
with honest transcription degradation.
|
|
96
|
+
- **It knows you — transparently.** A Self-Model subgraph (Self /
|
|
97
|
+
Preference / Decision / Habit / Relationship) built only from proposals
|
|
98
|
+
you approve, injected into answer context under a strict token budget,
|
|
99
|
+
fully listable and deletable. Agents can propose whole-folder
|
|
100
|
+
reorganizations — structurally incapable of proposing deletions.
|
|
101
|
+
- **Connected, selectively.** An approval-gated Obsidian vault bridge
|
|
102
|
+
(wikilinks become edges, idempotent re-runs) and a signed, encrypted
|
|
103
|
+
subgraph-share prototype where received knowledge arrives as proposals —
|
|
104
|
+
off by default behind `LATTICEAI_BRAIN_NETWORK`.
|
|
105
|
+
|
|
106
|
+
All of it lands with the floor intact: **6,261 tests, 100.00% of 37,590
|
|
107
|
+
statements and 10,658 branches**, verified on macOS 3.14, a fresh-resolve
|
|
108
|
+
python 3.11 environment, and a clean linux python:3.14 container.
|
|
97
109
|
|
|
98
110
|
Release notes: [RELEASE.md](RELEASE.md) · Full history: [docs/CHANGELOG.md](docs/CHANGELOG.md)
|
|
99
111
|
|
|
100
|
-
Expected artifacts for
|
|
112
|
+
Expected artifacts for 11.1.0 release must use exact filenames:
|
|
101
113
|
|
|
102
|
-
- `dist/ltcai-
|
|
103
|
-
- `dist/ltcai-
|
|
104
|
-
- `ltcai-
|
|
105
|
-
- `dist/ltcai-
|
|
106
|
-
- `src-tauri/target/release/bundle/dmg/Lattice
|
|
114
|
+
- `dist/ltcai-11.1.0-py3-none-any.whl`
|
|
115
|
+
- `dist/ltcai-11.1.0.tar.gz`
|
|
116
|
+
- `ltcai-11.1.0.tgz`
|
|
117
|
+
- `dist/ltcai-11.1.0.vsix`
|
|
118
|
+
- `src-tauri/target/release/bundle/dmg/Lattice AI_11.1.0_aarch64.dmg`
|
|
107
119
|
|
|
108
120
|
Do not use wildcard artifact uploads. Package registry publishing remains owner-run.
|
|
109
121
|
|
|
@@ -138,6 +150,9 @@ See [ARCHITECTURE.md](ARCHITECTURE.md) for details and
|
|
|
138
150
|
|
|
139
151
|
| Version | Theme |
|
|
140
152
|
| --- | --- |
|
|
153
|
+
| 11.1.0 | Product Intelligence |
|
|
154
|
+
| 11.0.1 | Both Branches |
|
|
155
|
+
| 11.0.0 | Full Measure |
|
|
141
156
|
| 10.10.0 | Quiet Station |
|
|
142
157
|
| 10.9.0 | Never Blocks |
|
|
143
158
|
| 10.8.0 | Within Reach |
|
package/docs/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,74 @@ The top entry is either the current unreleased main-branch work or the current
|
|
|
4
4
|
release line. Older entries are historical and may describe behavior as it
|
|
5
5
|
existed at that release.
|
|
6
6
|
|
|
7
|
+
## [11.1.0] - 2026-08-10 — Product Intelligence Layer
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- 플러그형 벡터 인덱스 레이어(`lattice_brain/graph/vector_index/`):
|
|
11
|
+
BruteForce 기본, int8 Quantized·HNSW(`ltcai[hnsw]`) 옵트인, 영속 배경
|
|
12
|
+
임베딩 큐(`vector_jobs`), `vector_freshness_breakdown()`, RRF 융합·이웃
|
|
13
|
+
후보 확장 옵션. 하이브리드 p50 10k 299ms → 10.1ms, 50k 43.9ms
|
|
14
|
+
(recall@10 0.987, docs/PERFORMANCE.md).
|
|
15
|
+
- Temporal 지식 모델: `valid_from`/`valid_to`/`superseded_by`(멱등 제자리
|
|
16
|
+
승급, NULL 규약) + `as_of(timestamp)` 슬라이스. 모순 감지 → 리뷰 제안
|
|
17
|
+
→ 승인 시 temporal 스탬프. 이벤트 기반 합성(상위 개념·누락 엣지·
|
|
18
|
+
proactive Brief 제안, 25개 인제스트마다, 격리 배선) + 중요도/정리 제안.
|
|
19
|
+
새 API: `/api/brain/proactive-brief|importance|synthesize|contradictions/*`.
|
|
20
|
+
- 멀티모달 1등 시민(`allow_multimodal`, 기본 꺼짐 — 꺼짐=바이트 동일):
|
|
21
|
+
Image/Audio 1등 노드, OCR·실캡션만(`caption_status`), 별도 이미지 벡터
|
|
22
|
+
공간+late fusion, Evidence 인라인 썸네일(승인 게이트 비우회), 비디오는
|
|
23
|
+
정직 거부.
|
|
24
|
+
- Self-Model 서브그래프(제안-우선 생성, 사용자 직접 소유), 컨텍스트
|
|
25
|
+
예산 주입, 삭제 불가능 구조의 폴더 재구성 제안,
|
|
26
|
+
`/api/memory/self-model*` 5종.
|
|
27
|
+
- Obsidian vault 브릿지(`POST /api/ingestion/obsidian`, 승인 게이트,
|
|
28
|
+
위키링크→엣지, 멱등) + 서명·암호화 선택적 서브그래프 공유 프로토타입
|
|
29
|
+
(`LATTICEAI_BRAIN_NETWORK` 기본 꺼짐, 수신=리뷰 제안).
|
|
30
|
+
|
|
31
|
+
### Removed
|
|
32
|
+
- `VisionStub`/`get_vision_embedder` — 파일명으로 캡션을 합성하고 그
|
|
33
|
+
문자열의 해시를 이미지 임베딩으로 저장하던 경로(정직성 위반) 삭제.
|
|
34
|
+
|
|
35
|
+
## [11.0.1] - 2026-08-10
|
|
36
|
+
|
|
37
|
+
### Fixed
|
|
38
|
+
- 11.0.0 릴리스 노트가 기록한 결함 11건 전부: Telegram `send_web_link`의
|
|
39
|
+
서버 토큰 유출과 전체 언로드 성공 조작, `inspect_html` 스타일시트 데드
|
|
40
|
+
수집, 리뷰 아이템 id 초 단위 충돌, vLLM 좀비 재확인(침묵 성공 → 409),
|
|
41
|
+
보안 대시보드 목록/상세 마스킹 갭과 invalid-JSON 내보내기, 임베딩
|
|
42
|
+
`model_id` 차원 동결, fast-path `X-Model`/SSE 본문 불일치, 워크스페이스
|
|
43
|
+
응답의 죽은 `"ok"` 리터럴과 미도달 404 arm. 각 수정은 결함을 고정하던
|
|
44
|
+
테스트를 반전하고 회귀 테스트를 추가했습니다.
|
|
45
|
+
|
|
46
|
+
### Changed
|
|
47
|
+
- 커버리지 게이트가 분기까지 잡습니다: `[tool.coverage.run] branch = true`,
|
|
48
|
+
9,828아크 전부 실행, `fail_under = 100`은 이제 라인+분기 합산.
|
|
49
|
+
테스트 5,426 → 5,798개. `pragma: no branch`는 사유 명시 2줄.
|
|
50
|
+
|
|
51
|
+
### Removed
|
|
52
|
+
- 참조-0 증명 후 죽은 코드 삭제: `tool_registry._wa()`,
|
|
53
|
+
`workspace_os_utils._snapshot_graph_import_payload`, 워크스페이스 404
|
|
54
|
+
arm 2개, vLLM 침묵 성공 분기, 증명된 죽은 조건 3곳
|
|
55
|
+
(model_catalog 버전 가드 · model_compat MLX 조건 · retrieval_docgen
|
|
56
|
+
blank-query 가드).
|
|
57
|
+
|
|
58
|
+
## [11.0.0] - 2026-08-10
|
|
59
|
+
|
|
60
|
+
### Changed
|
|
61
|
+
- Python 테스트 커버리지 72.80% → **100.00%**, CI 게이트 `fail_under = 100`.
|
|
62
|
+
테스트 2,269 → 5,426개(+3,157, 신규 파일 145개). 제외는 도달 불가 사유가
|
|
63
|
+
명시된 `pragma: no cover` 8줄뿐이며, MLX/Windows/watchdog 등 플랫폼 잠금
|
|
64
|
+
분기는 페이크 모듈·시임 패치로 ubuntu CI에서도 실행됩니다.
|
|
65
|
+
- `build_phases` 후반 4개 페이즈가 `test_security.py`의 server-import
|
|
66
|
+
부수효과 없이 전용 테스트(RuntimeContext 직접 구동)로 커버됩니다.
|
|
67
|
+
- ARCHITECTURE.md 검증 다이어그램이 실측값으로 갱신되고 "보고만 되고 강제
|
|
68
|
+
안 되는" 점선 상자가 사라졌습니다(양쪽 커버리지 모두 100% 플로어).
|
|
69
|
+
|
|
70
|
+
### Internal
|
|
71
|
+
- 커버리지 작업이 드러낸 실제 결함·죽은 분기 11건은 동작 변경 없이
|
|
72
|
+
RELEASE_NOTES_v11.0.0.md에 기록되고, 현재 동작을 그대로 단언하는
|
|
73
|
+
테스트로 고정됐습니다(수정은 다음 릴리스 후보).
|
|
74
|
+
|
|
7
75
|
## [10.10.0] - 2026-08-06
|
|
8
76
|
|
|
9
77
|
### Changed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Community And Plugins
|
|
2
2
|
|
|
3
|
-
Current release: **
|
|
3
|
+
Current release: **11.1.0 — Product Intelligence**.
|
|
4
4
|
|
|
5
5
|
LatticeAI defines the path from a strong local-first framework (8.4.0
|
|
6
6
|
action-aware baseline, 8.5.0 registry+DI hardening, 8.6.0 capture/navigation
|
package/docs/DEVELOPMENT.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
> **Status: canonical** — current contributor guidance, kept in sync with the
|
|
4
4
|
> current release.
|
|
5
5
|
|
|
6
|
-
Current release: **
|
|
6
|
+
Current release: **11.1.0 — Product Intelligence**.
|
|
7
7
|
|
|
8
8
|
This document is for contributors working on the local-first Digital Brain
|
|
9
9
|
codebase. Product positioning and quick start stay in `README.md`; release
|
package/docs/ONBOARDING.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Lattice AI Onboarding
|
|
2
2
|
|
|
3
|
-
Current release: **
|
|
3
|
+
Current release: **11.1.0 — Product Intelligence**.
|
|
4
4
|
|
|
5
5
|
The first-run goal is a five-minute path from "I opened the app" to "my Brain
|
|
6
6
|
has a source, a question, and proof." This page is the product contract behind
|
package/docs/OPERATIONS.md
CHANGED
package/docs/PERFORMANCE.md
CHANGED
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
# Knowledge Graph Performance Baseline
|
|
2
2
|
|
|
3
|
-
> **Status:
|
|
4
|
-
>
|
|
5
|
-
>
|
|
6
|
-
>
|
|
7
|
-
>
|
|
8
|
-
>
|
|
9
|
-
> these as 9.8.0 numbers.
|
|
3
|
+
> **Status: mixed provenance — read the date on each table.** The
|
|
4
|
+
> `scripts/profile_kg.py` tables were last captured on the v9.6.0 working tree
|
|
5
|
+
> and are carried forward as the regression reference; regenerate them before
|
|
6
|
+
> treating them as current. The **vector index backend** table is a fresh
|
|
7
|
+
> 11.1.0 measurement (2026-08-10) from `scripts/bench_vector_index.py`, and
|
|
8
|
+
> says so inline.
|
|
10
9
|
|
|
11
10
|
Synthetic performance and memory baseline for `KnowledgeGraphStore`
|
|
12
11
|
(`lattice_brain/graph/`). Measured with `scripts/profile_kg.py`.
|
|
@@ -92,16 +91,141 @@ Keyword search p50 grew ~3.6x but stays under 5 ms.
|
|
|
92
91
|
Regenerate with `.venv/bin/python scripts/profile_kg.py` after graph-layer
|
|
93
92
|
changes and compare against these tables.
|
|
94
93
|
|
|
94
|
+
## Vector index backends (11.1.0)
|
|
95
|
+
|
|
96
|
+
`scripts/bench_vector_index.py` measures the three selectable backends
|
|
97
|
+
(`LATTICEAI_VECTOR_INDEX`) against each other. It answers two questions
|
|
98
|
+
together, because either one alone is misleading: **how fast** each backend
|
|
99
|
+
answers, and **how much of the exact answer it returns** (recall@10 against
|
|
100
|
+
the brute-force scan, which is ground truth by construction).
|
|
101
|
+
|
|
102
|
+
Methodology, and what it does not cover:
|
|
103
|
+
|
|
104
|
+
- The corpus is synthetic — deterministic pseudo-text from a fixed vocabulary
|
|
105
|
+
(English + Korean) embedded with the built-in hash embedder. Real embeddings
|
|
106
|
+
cluster differently, so the recall figures are an upper bound for a
|
|
107
|
+
well-separated corpus, not a promise about your notes.
|
|
108
|
+
- Rows are written straight into `vector_embeddings`. This measures the
|
|
109
|
+
**search** path only; ingestion throughput stays in the tables above.
|
|
110
|
+
- Latency and memory are measured in separate passes. tracemalloc roughly
|
|
111
|
+
triples Python allocation cost, so timing under it would measure the
|
|
112
|
+
profiler — the `peak MB` column comes from one extra traced query.
|
|
113
|
+
- `peak MB` counts **Python** allocations only. hnswlib keeps its graph in C++
|
|
114
|
+
memory, which tracemalloc cannot see, so that column understates HNSW by the
|
|
115
|
+
size of the graph itself. Read it as "what the query costs the interpreter",
|
|
116
|
+
not as the process's resident size.
|
|
117
|
+
- `first ms` is the *first* query after the corpus changed. For `hnsw` that is
|
|
118
|
+
where the graph gets built and the `.hnsw` sidecar written; for the other two
|
|
119
|
+
it is an ordinary query.
|
|
120
|
+
- `hybrid p50` is `hybrid_search` (lexical + vector fusion) on that backend —
|
|
121
|
+
the number a user's question actually pays.
|
|
122
|
+
- The candidate cap is **lifted** (`--max-candidates 0`). This matters more
|
|
123
|
+
than it sounds: with the product default of 10 000, the exact scan on a
|
|
124
|
+
larger index silently becomes "the newest 10 000 rows", and recall@10 would
|
|
125
|
+
then be measured against a baseline that never looked at most of the corpus.
|
|
126
|
+
The first 50k run made exactly that mistake and scored HNSW at 0.18 — not
|
|
127
|
+
because it missed anything, but because the two backends had searched
|
|
128
|
+
different candidate sets. The `trunc` column now reports whether any query
|
|
129
|
+
hit a cap.
|
|
130
|
+
|
|
131
|
+
Run it:
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
.venv/bin/python scripts/bench_vector_index.py # 10k vectors
|
|
135
|
+
.venv/bin/python scripts/bench_vector_index.py --nodes 1000 # quick
|
|
136
|
+
.venv/bin/python scripts/bench_vector_index.py --json # machine-readable
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### Measured, 2026-08-10 — 10 000 vectors, 30 queries, top_k 10
|
|
140
|
+
|
|
141
|
+
macOS 27 (arm64, Apple Silicon), Python 3.14.5, SQLite 3.53.1,
|
|
142
|
+
hnswlib 0.8.0, uncapped candidates, corpus build 2.19 s.
|
|
143
|
+
|
|
144
|
+
| backend | p50 ms | p95 ms | first ms | peak MB | recall@10 | hybrid p50 ms |
|
|
145
|
+
|-----------|-------:|-------:|---------:|--------:|----------:|--------------:|
|
|
146
|
+
| brute | 293.25 | 299.31 | 293.91 | 39.72 | 1.000 | 299.17 |
|
|
147
|
+
| quantized | 640.58 | 650.15 | 636.87 | 38.38 | 0.987 | 653.33 |
|
|
148
|
+
| hnsw | 7.01 | 7.50 | 799.95 | 0.05 | 0.953 | 10.07 |
|
|
149
|
+
|
|
150
|
+
What the table says, plainly:
|
|
151
|
+
|
|
152
|
+
- **The default is exact and slow.** 293 ms per query at 10k, and hybrid
|
|
153
|
+
inherits nearly all of it (299 ms) — the vector channel *is* the cost.
|
|
154
|
+
- **HNSW meets the 11.1.0 target and prices it.** Hybrid p50 **10.1 ms** at 10k
|
|
155
|
+
(target: < 50 ms), a 30x improvement, in exchange for **4.7% of the exact
|
|
156
|
+
top-10 going missing**. That is the trade, stated as a number rather than as
|
|
157
|
+
the word "approximate".
|
|
158
|
+
- **The 800 ms first query is the graph build**, paid once per index generation
|
|
159
|
+
and then persisted to the `.hnsw` sidecar (and held in memory for the rest of
|
|
160
|
+
the process). Any write to `vector_embeddings` invalidates the fingerprint
|
|
161
|
+
and buys that cost again, which is why `brute` stays the default for a
|
|
162
|
+
continuously-ingesting brain.
|
|
163
|
+
- **Quantized is currently the wrong choice on every axis.** ~2.2x the latency
|
|
164
|
+
for 0.987 recall, and its RAM advantage does not materialise (38.4 vs
|
|
165
|
+
39.7 MB): the exact scan already feeds the index in bounded batches, so
|
|
166
|
+
resident vectors were never the dominant term — the fetched SQLite rows are.
|
|
167
|
+
It ships as an honest, exhaustive backend and as the representation a held
|
|
168
|
+
cross-query index would need. It is not a recommendation.
|
|
169
|
+
|
|
170
|
+
### Measured, 2026-08-10 — 50 000 vectors, 15 queries, top_k 10
|
|
171
|
+
|
|
172
|
+
Same machine and settings; corpus build 11.19 s.
|
|
173
|
+
|
|
174
|
+
| backend | p50 ms | p95 ms | first ms | peak MB | recall@10 | hybrid p50 ms |
|
|
175
|
+
|-----------|--------:|--------:|---------:|--------:|----------:|--------------:|
|
|
176
|
+
| brute | 1515.31 | 1522.94 | 1520.89 | 194.92 | 1.000 | 1514.75 |
|
|
177
|
+
| quantized | 3254.76 | 3275.69 | 3253.54 | 195.05 | 0.967 | 3441.13 |
|
|
178
|
+
| hnsw | 35.58 | 38.19 | 6114.60 | 0.04 | 0.987 | 43.90 |
|
|
179
|
+
|
|
180
|
+
- **The exact scan is linear and unusable at this size**: 1.5 s per query, and
|
|
181
|
+
~195 MB of Python allocation to score one question.
|
|
182
|
+
- **HNSW still clears the 50 ms budget at 5x the target corpus** — hybrid p50
|
|
183
|
+
43.9 ms — and its recall here is 0.987, higher than the 10k run's 0.953
|
|
184
|
+
(15 queries is a small sample; treat the two as "around 0.95–0.99", not as a
|
|
185
|
+
trend).
|
|
186
|
+
- **The remaining HNSW cost is not the search.** 7 ms at 10k → 36 ms at 50k is
|
|
187
|
+
suspiciously linear for a graph index, and it is: every query first runs the
|
|
188
|
+
freshness check (`SELECT COUNT(*), MAX(indexed_at) FROM vector_embeddings
|
|
189
|
+
WHERE embedding_model=? AND embedding_dim=?`), which has no covering index
|
|
190
|
+
and walks the table. A named follow-up, not a mystery: an index on
|
|
191
|
+
`(embedding_model, embedding_dim)` would remove it. The budget is met either
|
|
192
|
+
way, so it was not worth a schema change in this release.
|
|
193
|
+
- **The 6.1 s first query is the 50k graph build.** It is paid once per index
|
|
194
|
+
generation, and the sidecar means a restart does not pay it again.
|
|
195
|
+
|
|
196
|
+
### A measurement mistake worth keeping
|
|
197
|
+
|
|
198
|
+
The first 50k run reported HNSW recall of **0.18** and was wrong. The default
|
|
199
|
+
candidate cap (10 000) was still in force, so the "exact" baseline had scored
|
|
200
|
+
only the newest 10 000 of 50 000 rows while HNSW searched all of them — the
|
|
201
|
+
disagreement was the baseline's blind spot, not the ANN's error. The bench now
|
|
202
|
+
lifts the cap by default and prints a `trunc` column. Recorded here because
|
|
203
|
+
the failure mode is generic: *any* recall number measured against a truncated
|
|
204
|
+
baseline is measuring the truncation.
|
|
205
|
+
|
|
206
|
+
### Not measured
|
|
207
|
+
|
|
208
|
+
- **100k+ vectors.** Nothing is claimed beyond the 50k run above.
|
|
209
|
+
- **sqlite-vec ANN.** The optional `ann` extra was not installed, so
|
|
210
|
+
`vector_search_backend` reported `bruteforce-cosine` throughout. Its numbers
|
|
211
|
+
are unknown, not zero.
|
|
212
|
+
- **A real embedding provider.** Everything here uses the deterministic hash
|
|
213
|
+
embedder. With a model- or network-backed embedder, query-embedding cost
|
|
214
|
+
moves into the foreground and these ratios change.
|
|
215
|
+
- **Resident process memory.** See the tracemalloc caveat above: the HNSW graph
|
|
216
|
+
lives outside Python's allocator and is not in the `peak MB` column.
|
|
217
|
+
|
|
95
218
|
## Observations
|
|
96
219
|
|
|
97
220
|
- Keyword `search()` / `context_for_query()` stay low-millisecond thanks to
|
|
98
221
|
the FTS5 trigram index; they are not the scaling bottleneck.
|
|
99
222
|
- `traverse(depth=2)` cost grows with edge fan-out (synthetic corpus creates
|
|
100
223
|
dense shared-concept hubs); p95 is ~5 ms at 500 sources and ~16 ms at 5000.
|
|
101
|
-
- `vector_search()` is a brute-force scan over every
|
|
102
|
-
(O(index size) per query). It is the dominant cost at scale
|
|
103
|
-
|
|
104
|
-
|
|
224
|
+
- `vector_search()` on the default backend is a brute-force scan over every
|
|
225
|
+
stored embedding (O(index size) per query). It is the dominant cost at scale
|
|
226
|
+
— which is what the 11.1.0 backend table above measures, and what
|
|
227
|
+
`LATTICEAI_VECTOR_INDEX=hnsw` addresses. `profile_kg.py` still caps vector
|
|
228
|
+
queries at 10 for this reason.
|
|
105
229
|
- `rebuild_vector_index(full=True)` embeds documents and chunks with the hash
|
|
106
230
|
embedder; with a real embedding provider expect this phase to be slower by
|
|
107
231
|
the provider's per-call latency times the item count.
|
package/docs/TRUST_MODEL.md
CHANGED
package/docs/WHY_LATTICE.md
CHANGED