smart_brain 0.1.2 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (74) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +15 -0
  3. data/MEMPAL_GUIDE.md +1074 -0
  4. data/README.en.md +173 -173
  5. data/README.md +467 -173
  6. data/config/brain.yml +69 -1
  7. data/conversation_demo.rb +438 -438
  8. data/db/migrate/002_turn_events_payload.sql +9 -0
  9. data/db/migrate/003_tiers_and_lifecycle.sql +28 -0
  10. data/db/migrate/004_kg_edges.sql +30 -0
  11. data/db/migrate/005_domains_and_memory_scopes.sql +163 -0
  12. data/docs/coding_todo.md +139 -0
  13. data/docs/context_package.md +220 -0
  14. data/docs/evidence_pack.md +190 -0
  15. data/docs/gap_vs_mempal.md +161 -0
  16. data/docs/mcp.md +93 -0
  17. data/docs/memory_types.md +278 -0
  18. data/docs/multi_scope_memory_refactor_plan.md +483 -0
  19. data/docs/multi_scope_migration.md +65 -0
  20. data/docs/policies.md +308 -0
  21. data/docs/retrieval_plan.md +231 -0
  22. data/docs/smartbrain_design.md +299 -0
  23. data/docs/user_guide.md +546 -0
  24. data/example.rb +91 -91
  25. data/examples/01_memory_basic.rb +57 -0
  26. data/examples/02_governance.rb +63 -0
  27. data/examples/03_postgres_persistence.rb +63 -0
  28. data/examples/04_ollama_llm.rb +69 -0
  29. data/examples/05_smart_rag_integration.rb +79 -0
  30. data/examples/06_multi_scope_memory.rb +50 -0
  31. data/examples/README.md +49 -0
  32. data/exe/smart_brain +168 -0
  33. data/lib/smart_brain/adapters/smart_rag/direct_client.rb +16 -5
  34. data/lib/smart_brain/adapters/smart_rag/http_client.rb +16 -5
  35. data/lib/smart_brain/adapters/smart_rag/null_client.rb +7 -2
  36. data/lib/smart_brain/adapters/smart_rag/scope_filter.rb +60 -0
  37. data/lib/smart_brain/configuration.rb +57 -0
  38. data/lib/smart_brain/consolidator/working_summary.rb +80 -12
  39. data/lib/smart_brain/context_composer/composer.rb +40 -3
  40. data/lib/smart_brain/contracts/retrieval_plan.rb +10 -0
  41. data/lib/smart_brain/contracts/scope_context.rb +46 -0
  42. data/lib/smart_brain/contracts/scope_ref.rb +25 -0
  43. data/lib/smart_brain/db.rb +109 -0
  44. data/lib/smart_brain/event_store/in_memory.rb +6 -2
  45. data/lib/smart_brain/event_store/postgres.rb +199 -0
  46. data/lib/smart_brain/fusion/merger.rb +31 -2
  47. data/lib/smart_brain/governance/briefing.rb +146 -0
  48. data/lib/smart_brain/governance/fact_check.rb +110 -0
  49. data/lib/smart_brain/governance/knowledge_graph.rb +60 -0
  50. data/lib/smart_brain/governance/lifecycle.rb +225 -0
  51. data/lib/smart_brain/governance/tiers.rb +60 -0
  52. data/lib/smart_brain/memory_extractor/extractor.rb +25 -7
  53. data/lib/smart_brain/memory_store/in_memory.rb +202 -17
  54. data/lib/smart_brain/memory_store/postgres.rb +500 -0
  55. data/lib/smart_brain/model_provider/base.rb +87 -0
  56. data/lib/smart_brain/model_provider/factory.rb +49 -0
  57. data/lib/smart_brain/model_provider/ollama.rb +60 -0
  58. data/lib/smart_brain/model_provider/openai.rb +60 -0
  59. data/lib/smart_brain/model_provider/stub.rb +26 -0
  60. data/lib/smart_brain/model_provider.rb +7 -0
  61. data/lib/smart_brain/observability/tracker.rb +39 -1
  62. data/lib/smart_brain/retrievers/exact_retriever.rb +6 -0
  63. data/lib/smart_brain/retrievers/memory_retriever.rb +59 -5
  64. data/lib/smart_brain/runtime.rb +288 -16
  65. data/lib/smart_brain/scopes/conflict_resolver.rb +67 -0
  66. data/lib/smart_brain/scopes/registry.rb +133 -0
  67. data/lib/smart_brain/scopes/resolver.rb +32 -0
  68. data/lib/smart_brain/server/http_app.rb +143 -0
  69. data/lib/smart_brain/server/mcp_server.rb +385 -0
  70. data/lib/smart_brain/server/service.rb +129 -0
  71. data/lib/smart_brain/support/levenshtein.rb +35 -0
  72. data/lib/smart_brain/version.rb +5 -5
  73. data/lib/smart_brain.rb +80 -35
  74. metadata +88 -36
data/README.md CHANGED
@@ -1,173 +1,467 @@
1
- # SmartBrain
2
-
3
- SmartBrain 是一个面向 Agent 的记忆运行时(Memory Runtime)与上下文编排器(Context Composer)。
4
-
5
- 它的核心职责:
6
- - `commit_turn`:记录事件真相并沉淀结构化记忆
7
- - `compose_context`:在每轮请求前组装最小充分上下文
8
- - 联动 SmartRAG:对话记忆由 SmartBrain 管理,资源检索由 SmartRAG 提供
9
-
10
- ## 当前进展
11
-
12
- 当前仓库已实现并打通:
13
- - `commit_turn` / `compose_context` 主链路
14
- - Retention / Consolidation / Retrieval / Composition 策略
15
- - 检索器:exact + relational
16
- - 融合层:去重、规则重排、多样性、预算截断
17
- - SmartRAG 适配器:`NullClient` / `HttpClient` / `DirectClient`
18
- - 契约与可观测:`request_id` / `plan_id` / `context_id` 全链路追踪
19
- - RSpec 测试(单元 + 集成 + 回归)
20
-
21
- ## 项目结构
22
-
23
- - `lib/smart_brain.rb`:入口 API
24
- - `lib/smart_brain/runtime.rb`:主运行时编排
25
- - `lib/smart_brain/contracts/`:RetrievalPlan / EvidencePack / ContextPackage 校验
26
- - `lib/smart_brain/observability/`:日志与指标
27
- - `lib/smart_brain/event_store/`:事件存储(当前内存实现)
28
- - `lib/smart_brain/memory_store/`:记忆存储(当前内存实现)
29
- - `lib/smart_brain/retrievers/`:exact/relational 检索
30
- - `lib/smart_brain/fusion/`:多源融合
31
- - `lib/smart_brain/context_composer/`:上下文装配
32
- - `lib/smart_brain/adapters/smart_rag/`:SmartRAG 适配层
33
- - `config/brain.yml`:策略配置
34
- - `example.rb`:SmartBrain + SmartAgent + SmartPrompt + SmartRAG 联动示例
35
- - `docs/`:设计与协议文档
36
-
37
- ## 安装
38
-
39
- ```bash
40
- bundle install
41
- ```
42
-
43
- 如遇本地权限或 shared gem 污染,建议:
44
-
45
- ```bash
46
- bundle config set --local path 'vendor/bundle'
47
- bundle config set --local disable_shared_gems 'true'
48
- ```
49
-
50
- ## 快速开始(仅 SmartBrain)
51
-
52
- ```ruby
53
- require_relative 'lib/smart_brain'
54
-
55
- SmartBrain.configure
56
-
57
- SmartBrain.commit_turn(
58
- session_id: 'demo',
59
- turn_events: {
60
- messages: [
61
- { role: 'user', content: '请记住:默认数据库是 Postgres。' },
62
- { role: 'assistant', content: '已记录。' }
63
- ],
64
- decisions: [
65
- { key: 'decision:smartbrain:storage', decision: 'Use Postgres by default' }
66
- ]
67
- }
68
- )
69
-
70
- context = SmartBrain.compose_context(
71
- session_id: 'demo',
72
- user_message: '继续并总结关键结论'
73
- )
74
-
75
- puts context[:context_id]
76
- puts context.dig(:debug, :trace, :request_id)
77
- puts context.dig(:debug, :trace, :plan_id)
78
- ```
79
-
80
- ## SmartRAG 集成方式
81
-
82
- ### 1) NullClient(默认)
83
-
84
- 不配置 `smart_rag_client` 时,资源证据为空,仅使用记忆侧证据。
85
-
86
- ### 2) HttpClient
87
-
88
- ```ruby
89
- transport = lambda do |plan, timeout_seconds:|
90
- {
91
- plan_id: 'p1',
92
- supports_language_filter: true,
93
- evidences: []
94
- }
95
- end
96
-
97
- client = SmartBrain::Adapters::SmartRag::HttpClient.new(transport: transport, timeout_seconds: 2)
98
- SmartBrain.configure(smart_rag_client: client)
99
- ```
100
-
101
- ### 3) DirectClient(当前示例使用)
102
-
103
- ```ruby
104
- require 'smart_rag'
105
- require_relative 'lib/smart_brain/adapters/smart_rag/direct_client'
106
-
107
- rag_config = SmartRAG::Config.load(ENV.fetch('SMARTRAG_CONFIG_PATH'))
108
- rag = SmartRAG::SmartRAG.new(rag_config)
109
- client = SmartBrain::Adapters::SmartRag::DirectClient.new(rag: rag)
110
-
111
- SmartBrain.configure(smart_rag_client: client)
112
- ```
113
-
114
- ## example.rb 说明(已更新)
115
-
116
- `example.rb` 演示完整链路:
117
- 1. SmartBrain `compose_context`
118
- 2. SmartAgent 通过 `call_worker` 调用 SmartPrompt worker
119
- 3. SmartBrain `commit_turn`
120
- 4. 打印 `evidence(memory/resource)` 验证 SmartRAG 是否参与
121
-
122
- 示例依赖的本地文件:
123
- - `config/example_agent.yml`
124
- - `config/example_llm.yml`
125
- - `agents/brain_assistant.rb`
126
- - `workers/brain_assistant.rb`
127
- - `templates/brain_assistant.erb`
128
-
129
- 运行:
130
-
131
- ```bash
132
- bundle exec ruby example.rb
133
- ```
134
-
135
- ## 核心 API
136
-
137
- ### `SmartBrain.configure(config_path: nil, smart_rag_client: nil, clock: -> { Time.now.utc })`
138
- 初始化运行时并注入 SmartRAG 客户端(可选)。
139
-
140
- ### `SmartBrain.commit_turn(session_id:, turn_events:)`
141
- 写入事件、抽取记忆、冲突处理、摘要更新。
142
-
143
- ### `SmartBrain.compose_context(session_id:, user_message:, agent_state: {})`
144
- 生成 `ContextPackage`,内部包含检索计划与证据融合结果。
145
-
146
- ### `SmartBrain.diagnostics`
147
- 返回 compose/commit 观测日志与指标快照。
148
-
149
- ## 测试
150
-
151
- ```bash
152
- rspec
153
- ```
154
-
155
- ## 常见问题
156
-
157
- ### 1) `cannot load such file -- sequel/extensions/pgvector`
158
- 当前示例已在 `example.rb` 做兼容处理(`Sequel.extension 'pgvector'` + 去除 `database.extensions` 连接参数)。
159
-
160
- ### 2) `Config file not found: config/llm_config.yml`
161
- `example.rb` 已将 SmartRAG 里 EmbeddingService 的 `config_path` 注入为 `./config/example_llm.yml`。
162
-
163
- ### 3) `ruby-lsp: not found`
164
- ```bash
165
- gem install --user-install ruby-lsp debug
166
- ```
167
- 并将用户 gem bin 加入 `PATH`。
168
-
169
- ## 路线图
170
-
171
- - 将 EventStore/MemoryStore 从内存实现切换到 Postgres 实现
172
- - 接入真实 reranker / embedding 模型
173
- - 完善 SmartRAG ingest 与跨会话评估工具
1
+ # SmartBrain
2
+
3
+ SmartBrain(v0.2.0)是一个面向 Agent 的**记忆运行时(Memory Runtime)与上下文编排器(Context Composer)**。
4
+
5
+ 在多轮 Agent 对话中,SmartBrain 解决的核心问题是:如何高效地记录、检索、融合记忆,并装配出"最小充分"的上下文发给 LLM。
6
+
7
+ ## 核心职责
8
+
9
+ - **`commit_turn`**:记录事件真相并沉淀结构化长期记忆(写链路)
10
+ - **`compose_context`**:在每轮请求前生成最小充分上下文(读 + 装配链路)
11
+ - **联动 SmartRAG**:对话记忆由 SmartBrain 管理(EventStore / MemoryStore),资源检索由 SmartRAG 提供(适配层)
12
+
13
+ ## 架构边界
14
+
15
+ SmartBrain Agent 的可用信息分为两类,分工明确:
16
+
17
+ | 类别 | 管理者 | 内容 | 特点 |
18
+ |------|--------|------|------|
19
+ | **对话记忆** (Conversation Memory) | SmartBrain | 原始事件(turn/messages/tool_calls/refs)、结构化记忆(profile/preferences/tasks/decisions/entities/events)、滚动摘要(working_summary) | 强时序、更新频繁、有写入门控与冲突管理 |
20
+ | **资源记忆** (Resource Memory) | SmartRAG | 文档/网页/代码仓库等长期资源 | 相对静态、内容大、需要高质量检索 |
21
+
22
+ > SmartRAG 是 SmartBrain 的"资源知识库后端"。SmartBrain 不复制 SmartRAG 的索引能力,只通过契约适配层调用它。
23
+
24
+ ## 当前进展
25
+
26
+ 当前仓库已实现并打通:
27
+
28
+ - `commit_turn` / `compose_context` 主链路
29
+ - Retention / Consolidation / Retrieval / Composition 四类策略
30
+ - 检索器:exact(关键词匹配) + relational(实体/引用关联)
31
+ - 融合层:去重、词重叠加分重排、多样性约束、预算截断、memory/resource 比例控制
32
+ - SmartRAG 适配器:`NullClient`(默认空)/ `HttpClient`(HTTP 远程)/ `DirectClient`(进程内直接调用)
33
+ - 契约与可观测:`request_id` → `plan_id` → `context_id` 全链路追踪
34
+ - RSpec 测试(单元 + 集成 + PostgreSQL 回归)
35
+
36
+ ## 安装
37
+
38
+ ```bash
39
+ bundle install
40
+ ```
41
+
42
+ 如遇本地权限或 shared gem 污染:
43
+
44
+ ```bash
45
+ bundle config set --local path 'vendor/bundle'
46
+ bundle config set --local disable_shared_gems 'true'
47
+ ```
48
+
49
+ ## 核心 API
50
+
51
+ SmartBrain 提供以下核心工作流 API:
52
+
53
+ ### `SmartBrain.configure(config_path: nil, smart_rag_client: nil, clock: -> { Time.now.utc })`
54
+
55
+ 初始化运行时并注入 SmartRAG 客户端(可选)。
56
+
57
+ - `config_path`:策略配置文件路径,默认 `config/brain.yml`
58
+ - `smart_rag_client`:SmartRAG 适配器实例,默认 `NullClient`(资源证据为空)
59
+ - `clock`:时间源函数,默认 `Time.now.utc`
60
+
61
+ ### `SmartBrain.commit_turn(session_id:, turn_events:, domain_id: nil, scope_context: nil)`
62
+
63
+ 写入事件、抽取记忆、冲突处理、摘要更新。返回 `CommitResult`。
64
+
65
+ **`turn_events` 支持的事件类型:**
66
+
67
+ | 字段 | 类型 | 说明 | 示例 |
68
+ |------|------|------|------|
69
+ | `messages` | Array | 消息列表,每条含 `role` 和 `content` | `[{ role: 'user', content: '...' }]` |
70
+ | `decisions` | Array | 决策/选择/承诺 | `[{ key: 'decision:pg:pool_size', decision: '...' }]` |
71
+ | `tasks` | Array | 任务,含状态流转 | `[{ key: 'task:brain:mvp', status: 'doing' }]` |
72
+ | `goals` | Array | 长期目标 | `[{ key: 'goal:learn:ruby', goal: '...' }]` |
73
+ | `entities` | Array | 重要实体 | `[{ key: 'entity:lang:ruby', name: 'Ruby', canonical: 'ruby', kind: 'language', remember: true }]` |
74
+ | `preferences` | Array | 偏好,需 `confirmed: true` 才写入长期记忆 | `[{ key: 'pref:writing:tone', value: '...', confirmed: true }]` |
75
+ | `events` | Array | 里程碑/异常等高价值事件 | — |
76
+ | `refs` | Array | 文件/URL/产物引用 | — |
77
+ | `retractions` | Array | 撤回旧记忆 | `[{ type: 'decisions', key: 'decision:old:topic' }]` |
78
+
79
+ **写入门控策略:**
80
+ - **必写**:`tasks`、`decisions`(confidence: 0.9)
81
+ - **条件写**:`goals`(需显示声明)、`preferences`(需 `confirmed: true`)、`entities`(需出现频率 ≥ 2 或含 URL/路径等结构信号)
82
+ - **不写入**:闲聊、未确认推测
83
+
84
+ **冲突处理:**
85
+ - 覆盖型(`preferences/goals/tasks`):新值写入,旧值 → `status=superseded`
86
+ - 多版本并存(`decisions/events`):保留历史
87
+ - 撤回(`retracted`):旧条目 → `status=retracted`
88
+
89
+ **返回值:**
90
+
91
+ ```ruby
92
+ {
93
+ ok: true,
94
+ commit_id: "uuid",
95
+ session_id: "...",
96
+ turn_id: "uuid",
97
+ memory_written: { count: 3, items: [...], conflicts: [...] },
98
+ summary: { triggered: true, trigger_reason: "turn_threshold", text: "..." },
99
+ explain: { retention: [...], conflicts: [...], summary: {...} }
100
+ }
101
+ ```
102
+
103
+ ### `SmartBrain.compose_context(session_id:, user_message:, agent_state: {}, domain_id: nil, scope_context: nil)`
104
+
105
+ 生成 `ContextPackage`,内部执行 5 阶段流水线:
106
+
107
+ ```
108
+ Plan Retrieve(双路并行)→ Fuse → Compose → ContextPackage
109
+ ```
110
+
111
+ 1. **Plan**:`RetrievalPlanner` 分析意图,生成 `RetrievalPlan`(主查询 + 扩展查询),自动判断是否启用资源检索(`enable_resource_retrieval: 'auto'` 时,关键词"查资料/引用/论文/文档/compare/对比/来源"触发)
112
+ 2. **Retrieve**:双路并行 —
113
+ - 记忆侧:`ExactRetriever`(词重叠匹配,支持中英文分词)+ `RelationalRetriever`(实体/引用关联)
114
+ - 资源侧:通过 SmartRAG 适配器检索文档资源(仅当 planner 启用时)
115
+ 3. **Fuse**:`Merger` 归一化 → 去重 → 词重叠加分重排 → 多样性约束(同文档 ≤ 3 条,同来源 ≤ 2 条)→ 预算裁剪(总条数 ≤ `evidence_max_items`,snippet ≤ `max_snippet_chars`)→ memory/resource 比例控制(默认 40/60)
116
+ 4. **Compose**:`ContextComposer` 按固定槽位装配:
117
+ ```
118
+ system_blocks developer_blocks working_summary
119
+ recent_turns evidence → user_message
120
+ ```
121
+ 5. **输出**:`ContextPackage`(经过契约校验)
122
+
123
+ ### 多 Scope 记忆
124
+
125
+ 0.2.0 支持在同一 domain 内联合读取 `global/project/expert/task` 记忆,并将长期记忆写入明确授权的 scope:
126
+
127
+ ```ruby
128
+ scope_context = {
129
+ read: [
130
+ { type: 'global', id: 'default' },
131
+ { type: 'project', id: 'project-001' },
132
+ { type: 'task', id: 'task-030' }
133
+ ],
134
+ write: [
135
+ { type: 'project', id: 'project-001' },
136
+ { type: 'task', id: 'task-030' }
137
+ ],
138
+ default_write: { type: 'task', id: 'task-030' }
139
+ }
140
+
141
+ SmartBrain.commit_turn(
142
+ domain_id: 'tenant-a', session_id: 'conversation-1', scope_context: scope_context,
143
+ turn_events: { decisions: [{ key: 'decision:db', decision: 'use PostgreSQL' }] }
144
+ )
145
+ ```
146
+
147
+ `write` 必须是 `read` 的子集,`default_write` 必须包含在 `write` 中。单条结构化记忆可以用 `scope_ref` 覆盖默认写 scope,但目标仍须在 `write` 中。提供 `scope_context` 时必须同时提供 `domain_id`;两者都省略时继续使用隔离的 `legacy/session:<session_id>`,兼容 0.1.x 调用。
148
+
149
+ **返回值(ContextPackage):**
150
+
151
+ ```ruby
152
+ {
153
+ version: '0.1',
154
+ context_id: "uuid", # 本次装配唯一 ID
155
+ session_id: "...",
156
+ working_summary: "...", # 滚动摘要文本
157
+ recent_turns: [...], # 最近 N 轮对话
158
+ evidence: [ # 融合后的证据列表
159
+ {
160
+ id: "...",
161
+ source: 'memory' | 'resource',
162
+ source_uri: "...",
163
+ title: "...",
164
+ snippet: "...",
165
+ mode: 'exact' | 'relational' | 'hybrid',
166
+ score: 0.85,
167
+ ref: { memory_item_id: "..." }
168
+ }
169
+ ],
170
+ user_message: { role: 'user', content: "..." },
171
+ constraints: {
172
+ token_budget: { limit: 8000, used_estimate: 1200 },
173
+ diversity: { by_document: 3, by_source: 2 },
174
+ truncation: { snippets_max_chars: 800, recent_turns_max: 8 }
175
+ },
176
+ debug: {
177
+ trace: { request_id: "uuid", plan_id: "uuid", context_id: "uuid" },
178
+ planner: { purpose: "qa", queries: ["..."] },
179
+ why_selected: ["item-1 score=0.85 source=memory"],
180
+ ignored: [],
181
+ dropped: [{ id: "...", reason: "diversity" }]
182
+ }
183
+ }
184
+ ```
185
+
186
+ ### `SmartBrain.diagnostics`
187
+
188
+ 返回 compose/commit 观测日志与指标快照:
189
+
190
+ ```ruby
191
+ {
192
+ compose_logs: [...],
193
+ commit_logs: [...],
194
+ metrics: {
195
+ compose_p95_ms: 45.2,
196
+ memory_resource_ratio: "3/2",
197
+ token_over_budget_rate: 0.0
198
+ }
199
+ }
200
+ ```
201
+
202
+ 全链路追踪:每个请求有 `request_id` → `plan_id` → `context_id` 三连 ID,贯穿 Plan → Retrieve → Fuse → Compose 全流程。
203
+
204
+ ## 策略配置
205
+
206
+ SmartBrain 的四类策略通过 `config/brain.yml` 配置:
207
+
208
+ | 策略类别 | 生效点 | 关键参数 |
209
+ |---------|--------|---------|
210
+ | **Retention**(写入门控) | `commit_turn` | `entity_gate: { window_turns: 20, freq_threshold: 2 }`、`confidence: { user_asserted: 0.8, tool_derived: 0.9, inferred: 0.6 }`、`summarize_after_turns: 12` |
211
+ | **Retrieval**(检索) | `compose_context` | `top_k: 30`、`candidate_k: 200`、`enable_resource_retrieval: auto`、`query_expansion: { enabled: true, max_queries: 8 }` |
212
+ | **Composition**(装配) | `compose_context` | `recent_turns_max: 8`、`evidence_max_items: 12`、`max_snippet_chars: 800`、`diversity: { by_document: 3, by_source_uri: 2 }`、`memory_resource_ratio: "40/60"` |
213
+ | **Observability**(可观测) | 全局 | `trace: true` |
214
+
215
+ ## 项目结构
216
+
217
+ ```
218
+ lib/smart_brain.rb # Ruby 入口 API
219
+ lib/smart_brain/version.rb # 版本号
220
+ lib/smart_brain/configuration.rb # YAML 配置加载
221
+ lib/smart_brain/runtime.rb # 主运行时编排(commit_turn / compose_context)
222
+ lib/smart_brain/contracts/ # 契约校验
223
+ retrieval_plan.rb # RetrievalPlan 校验
224
+ evidence_pack.rb # EvidencePack 校验
225
+ context_package.rb # ContextPackage 校验
226
+ lib/smart_brain/event_store/
227
+ in_memory.rb # 事件存储(当前内存实现,含 entity_frequencies 统计)
228
+ lib/smart_brain/memory_store/
229
+ in_memory.rb # 记忆存储(当前内存实现,含 upsert 冲突管理)
230
+ lib/smart_brain/memory_extractor/
231
+ extractor.rb # 从事件中抽取结构化记忆(写入门控)
232
+ lib/smart_brain/consolidator/
233
+ working_summary.rb # 滚动摘要维护(turn_threshold / token_pressure / stage_event 触发)
234
+ lib/smart_brain/retrieval_planner/
235
+ planner.rb # 生成 RetrievalPlan(query expansion + 资源检索启停判断 + filter hints)
236
+ lib/smart_brain/retrievers/
237
+ exact_retriever.rb # 词重叠关键词匹配(支持中英文)
238
+ relational_retriever.rb # 实体/引用关联检索
239
+ memory_retriever.rb # 记忆检索门面(exact + relational 合并)
240
+ lib/smart_brain/fusion/
241
+ merger.rb # 多源融合(归一化 → 去重 → 词重叠加分重排 → 多样性 → 预算截断)
242
+ lib/smart_brain/context_composer/
243
+ composer.rb # 固定槽位上下文装配 + token 估算
244
+ lib/smart_brain/adapters/smart_rag/
245
+ null_client.rb # 默认空适配器(资源证据为空)
246
+ http_client.rb # HTTP 远程调用适配器(支持超时降级)
247
+ direct_client.rb # 进程内直接调用适配器
248
+ lib/smart_brain/observability/
249
+ tracker.rb # 日志与指标(P95 / memory_resource_ratio / token_over_budget_rate)
250
+ config/
251
+ example_llm.yml # LLM 配置示例(ollama + silicon_flow)
252
+ docs/
253
+ smartbrain_design.md # 总体设计文档
254
+ policies.md # 策略规范文档
255
+ memory_types.md # 记忆类型规范(9 种 type + key 规则)
256
+ context_package.md # ContextPackage 协议
257
+ retrieval_plan.md # RetrievalPlan 协议
258
+ evidence_pack.md # EvidencePack 协议
259
+ spec/
260
+ spec_helper.rb
261
+ commit_turn_spec.rb # commit_turn 单元测试
262
+ compose_context_spec.rb # compose_context 单元测试
263
+ fusion_merger_spec.rb # 融合层测试
264
+ integration_smart_rag_adapter_spec.rb # SmartRAG 适配器集成测试
265
+ observability_metrics_spec.rb # 可观测性测试
266
+ planner_policy_spec.rb # 检索计划策略测试
267
+ regression_context_spec.rb # 回归测试
268
+ example.rb # 简单示例
269
+ conversation_demo.rb # 完整多轮对话演示(SmartBrain + SmartRAG + SmartAgent)
270
+ ```
271
+
272
+ ## 快速开始
273
+
274
+ ### 仅 SmartBrain(不依赖外部服务)
275
+
276
+ ```ruby
277
+ require_relative 'lib/smart_brain'
278
+
279
+ SmartBrain.configure
280
+
281
+ SmartBrain.commit_turn(
282
+ session_id: 'demo',
283
+ turn_events: {
284
+ messages: [
285
+ { role: 'user', content: '请记住:默认数据库是 Postgres。' },
286
+ { role: 'assistant', content: '已记录。' }
287
+ ],
288
+ decisions: [
289
+ { key: 'decision:smartbrain:storage', decision: 'Use Postgres by default' }
290
+ ]
291
+ }
292
+ )
293
+
294
+ context = SmartBrain.compose_context(
295
+ session_id: 'demo',
296
+ user_message: '继续并总结关键结论'
297
+ )
298
+
299
+ puts context[:context_id]
300
+ puts context.dig(:debug, :trace, :request_id)
301
+ puts context.dig(:debug, :trace, :plan_id)
302
+ ```
303
+
304
+ ### 多轮对话循环(与 SmartAgent + SmartRAG 联动)
305
+
306
+ ```ruby
307
+ require 'smart_brain'
308
+ require 'smart_agent'
309
+ require 'smart_rag'
310
+ require_relative 'lib/smart_brain/adapters/smart_rag/direct_client'
311
+
312
+ # 1. 初始化 SmartRAG
313
+ rag = SmartRAG::SmartRAG.new(database: {...}, llm: {...})
314
+ client = SmartBrain::Adapters::SmartRag::DirectClient.new(rag: rag)
315
+
316
+ # 2. 初始化 SmartBrain
317
+ SmartBrain.configure(smart_rag_client: client)
318
+
319
+ # 3. 初始化 SmartAgent
320
+ engine = SmartAgent::Engine.new('./config/example_agent.yml')
321
+ agent = engine.build_agent(:brain_assistant)
322
+
323
+ # 4. 多轮对话循环
324
+ session_id = "demo-session"
325
+ user_message = "Ruby 类名应该用什么命名风格?"
326
+
327
+ # 4a. 获取上下文(内部可能调用 SmartRAG 检索文档资源)
328
+ context = SmartBrain.compose_context(
329
+ session_id: session_id,
330
+ user_message: user_message,
331
+ agent_state: { turn: 1 }
332
+ )
333
+
334
+ # 4b. 调用 Agent(LLM)生成回复
335
+ response = agent.please(context.to_json)
336
+
337
+ # 4c. 提交本轮(沉淀记忆)
338
+ commit = SmartBrain.commit_turn(
339
+ session_id: session_id,
340
+ turn_events: {
341
+ messages: [
342
+ { role: 'user', content: user_message },
343
+ { role: 'assistant', content: response.to_s }
344
+ ],
345
+ decisions: [
346
+ { key: 'decision:ruby:class_naming', decision: '使用 CamelCase' }
347
+ ],
348
+ entities: [
349
+ { key: 'entity:lang:ruby', name: 'Ruby', canonical: 'ruby', kind: 'language', remember: true }
350
+ ]
351
+ }
352
+ )
353
+
354
+ # 5. 查看诊断信息
355
+ pp SmartBrain.diagnostics
356
+ ```
357
+
358
+ ## SmartRAG 集成方式
359
+
360
+ ### 1) NullClient(默认)
361
+
362
+ 不配置 `smart_rag_client` 时,资源证据为空,仅使用记忆侧证据。适合纯对话记忆场景。
363
+
364
+ ```ruby
365
+ SmartBrain.configure # 无需额外配置
366
+ ```
367
+
368
+ ### 2) HttpClient(HTTP 远程)
369
+
370
+ 通过自定义 transport lambda 调用远端 SmartRAG 服务,支持超时降级。
371
+
372
+ ```ruby
373
+ transport = lambda do |plan, timeout_seconds:|
374
+ {
375
+ plan_id: 'p1',
376
+ supports_language_filter: true,
377
+ evidences: []
378
+ }
379
+ end
380
+
381
+ scope_mapper = lambda do |domain_id:, scopes:|
382
+ { filters: { topic_ids: scopes.map { |scope| "#{domain_id}:#{scope[:type]}:#{scope[:id]}" } } }
383
+ end
384
+ client = SmartBrain::Adapters::SmartRag::HttpClient.new(
385
+ transport: transport, timeout_seconds: 2, scope_mapper: scope_mapper
386
+ )
387
+ SmartBrain.configure(smart_rag_client: client)
388
+ ```
389
+
390
+ ### 3) DirectClient(进程内直接调用)
391
+
392
+ 直接依赖 `smart_rag` gem,进程内调用。需要配置 PostgreSQL 连接和 LLM。
393
+
394
+ ```ruby
395
+ require 'smart_rag'
396
+ require_relative 'lib/smart_brain/adapters/smart_rag/direct_client'
397
+
398
+ rag_config = SmartRAG::Config.load(ENV.fetch('SMARTRAG_CONFIG_PATH'))
399
+ rag = SmartRAG::SmartRAG.new(rag_config)
400
+ scope_mapper = lambda do |domain_id:, scopes:|
401
+ { filters: { topic_ids: scopes.map { |scope| "#{domain_id}:#{scope[:type]}:#{scope[:id]}" } } }
402
+ end
403
+ client = SmartBrain::Adapters::SmartRag::DirectClient.new(rag: rag, scope_mapper: scope_mapper)
404
+
405
+ SmartBrain.configure(smart_rag_client: client)
406
+ ```
407
+
408
+ Mapper 返回的 `filters` 会作为 `scope_filters` 发送给 SmartRAG。业务 scope 存在时,后端必须返回 `scope_filter_applied: true`;mapper 缺失、映射失败或后端未确认时,adapter 默认 fail closed,丢弃资源证据并在 `warnings` 和 `explain.ignored_fields` 中说明原因。
409
+
410
+ ## 运行示例
411
+
412
+ ### example.rb
413
+
414
+ SmartBrain + SmartAgent + SmartPrompt + SmartRAG 联动示例(2 轮对话)。
415
+
416
+ ```bash
417
+ bundle exec ruby example.rb
418
+ ```
419
+
420
+ 依赖的本地文件:
421
+ - `config/example_agent.yml`
422
+ - `config/example_llm.yml`
423
+ - `agents/brain_assistant.rb`
424
+ - `workers/brain_assistant.rb`
425
+ - `templates/brain_assistant.erb`
426
+
427
+ ### conversation_demo.rb
428
+
429
+ 完整多轮对话演示(5 轮),包含 SmartRAG 文档存储、检索、SmartAgent LLM 调用和 SmartBrain 记忆沉淀全流程。
430
+
431
+ ```bash
432
+ bundle exec ruby conversation_demo.rb
433
+ ```
434
+
435
+ ## 测试
436
+
437
+ ```bash
438
+ bundle exec rspec
439
+ ```
440
+
441
+ 测试覆盖 commit/compose、存储、治理、KG、协议适配、多 scope 隔离以及 PostgreSQL 迁移和持久化。
442
+
443
+ ## 常见问题
444
+
445
+ ### 1) `cannot load such file -- sequel/extensions/pgvector`
446
+
447
+ 当前示例已在 `example.rb` 做兼容处理(`Sequel.extension 'pgvector'` + 去除 `database.extensions` 连接参数)。
448
+
449
+ ### 2) `Config file not found: config/llm_config.yml`
450
+
451
+ `example.rb` 已将 SmartRAG 里 EmbeddingService 的 `config_path` 注入为 `./config/example_llm.yml`。
452
+
453
+ ### 3) `ruby-lsp: not found`
454
+
455
+ ```bash
456
+ gem install --user-install ruby-lsp debug
457
+ ```
458
+
459
+ 并将用户 gem bin 加入 `PATH`。
460
+
461
+ ## 路线图
462
+
463
+ - 将 EventStore/MemoryStore 从内存实现切换到 Postgres 持久化实现
464
+ - 接入真实 reranker / embedding 模型(当前为规则分 + 词重叠加分)
465
+ - 完善 SmartRAG ingest 与跨会话评估工具
466
+ - 扩展 `SemanticRetriever`(依赖 pgvector/embedding)
467
+ - 支持 `memory_chunks` 表(FTS 索引 + 可选 embedding 字段)