smart_brain 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 034abbb06e5d99b9bda6b84d3934270e47f8406ff78f78b6bc8c269941bd0504
+  data.tar.gz: a48aec950053b94adb32798c47f0ed2c0a81be794fd74911547c3f68a924b477
+SHA512:
+  metadata.gz: 88c1ccdfdd8a6bd7e3fd0350127e1f4f040a5cd3103601210705960f7db7d6e9a08e595be4725efb4f7b532cbb3a42263f0aaa2b085dee1f5a7e07a2c38df865
+  data.tar.gz: e6059ec7ca9c966781b0e19db2068916f8c8c9aa09ab344b87be23f902df6d7e889d90a2ef7d1e655ed4ae97f9ae4ebce846fa136c3dc20e2f3ca21d8bf62c59

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 zhuang biaowei
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.en.md

@@ -0,0 +1,173 @@
+# SmartBrain
+
+SmartBrain is an Agent Memory Runtime and Context Composer.
+
+Core responsibilities:
+- `commit_turn`: persist event truth and structured memory
+- `compose_context`: build minimal, sufficient context per turn
+- integrate with SmartRAG: SmartBrain handles conversation memory, SmartRAG handles resource retrieval
+
+## Current Progress
+
+The repository now includes a runnable v0.1 flow with:
+- `commit_turn` / `compose_context` end-to-end pipeline
+- retention / consolidation / retrieval / composition policies
+- retrievers: exact + relational
+- fusion: dedupe, rule-based rerank, diversity, budget truncation
+- SmartRAG adapters: `NullClient`, `HttpClient`, `DirectClient`
+- traceability via `request_id` / `plan_id` / `context_id`
+- RSpec coverage (unit + integration + regression)
+
+## Project Layout
+
+- `lib/smart_brain.rb`: public API
+- `lib/smart_brain/runtime.rb`: runtime orchestration
+- `lib/smart_brain/contracts/`: RetrievalPlan / EvidencePack / ContextPackage validation
+- `lib/smart_brain/observability/`: logs and metrics
+- `lib/smart_brain/event_store/`: event storage (in-memory currently)
+- `lib/smart_brain/memory_store/`: memory storage (in-memory currently)
+- `lib/smart_brain/retrievers/`: exact/relational retrievers
+- `lib/smart_brain/fusion/`: multi-source fusion
+- `lib/smart_brain/context_composer/`: context assembly
+- `lib/smart_brain/adapters/smart_rag/`: SmartRAG adapters
+- `config/brain.yml`: policy config
+- `example.rb`: SmartBrain + SmartAgent + SmartPrompt + SmartRAG demo
+- `docs/`: design and protocol documents
+
+## Installation
+
+```bash
+bundle install
+```
+
+If you hit permission/shared-gem issues:
+
+```bash
+bundle config set --local path 'vendor/bundle'
+bundle config set --local disable_shared_gems 'true'
+```
+
+## Quick Start (SmartBrain Only)
+
+```ruby
+require_relative 'lib/smart_brain'
+
+SmartBrain.configure
+
+SmartBrain.commit_turn(
+  session_id: 'demo',
+  turn_events: {
+    messages: [
+      { role: 'user', content: 'Remember this: default DB is Postgres.' },
+      { role: 'assistant', content: 'Saved.' }
+    ],
+    decisions: [
+      { key: 'decision:smartbrain:storage', decision: 'Use Postgres by default' }
+    ]
+  }
+)
+
+context = SmartBrain.compose_context(
+  session_id: 'demo',
+  user_message: 'Continue and summarize key points'
+)
+
+puts context[:context_id]
+puts context.dig(:debug, :trace, :request_id)
+puts context.dig(:debug, :trace, :plan_id)
+```
+
+## SmartRAG Integration Options
+
+### 1) NullClient (default)
+
+If no SmartRAG client is injected, resource evidence is empty.
+
+### 2) HttpClient
+
+```ruby
+transport = lambda do |plan, timeout_seconds:|
+  {
+    plan_id: 'p1',
+    supports_language_filter: true,
+    evidences: []
+  }
+end
+
+client = SmartBrain::Adapters::SmartRag::HttpClient.new(transport: transport, timeout_seconds: 2)
+SmartBrain.configure(smart_rag_client: client)
+```
+
+### 3) DirectClient (used in `example.rb`)
+
+```ruby
+require '/home/mlf/smart_ai/smart_rag/lib/smart_rag'
+require_relative 'lib/smart_brain/adapters/smart_rag/direct_client'
+
+rag_config = SmartRAG::Config.load('/home/mlf/smart_ai/smart_rag/config/smart_rag.yml')
+rag = SmartRAG::SmartRAG.new(rag_config)
+client = SmartBrain::Adapters::SmartRag::DirectClient.new(rag: rag)
+
+SmartBrain.configure(smart_rag_client: client)
+```
+
+## `example.rb` (Updated)
+
+The example demonstrates the real loop:
+1. SmartBrain `compose_context`
+2. SmartAgent calls SmartPrompt worker via `call_worker`
+3. SmartBrain `commit_turn`
+4. prints `evidence(memory/resource)` so you can verify SmartRAG participation
+
+Files used by the demo:
+- `config/example_agent.yml`
+- `config/example_llm.yml`
+- `agents/brain_assistant.rb`
+- `workers/brain_assistant.rb`
+- `templates/brain_assistant.erb`
+
+Run:
+
+```bash
+bundle exec ruby example.rb
+```
+
+## Core API
+
+### `SmartBrain.configure(config_path: nil, smart_rag_client: nil, clock: -> { Time.now.utc })`
+Initialize runtime and optionally inject a SmartRAG client.
+
+### `SmartBrain.commit_turn(session_id:, turn_events:)`
+Persist events, extract memory, resolve conflicts, update summary.
+
+### `SmartBrain.compose_context(session_id:, user_message:, agent_state: {})`
+Build a `ContextPackage` with planning and fused evidence.
+
+### `SmartBrain.diagnostics`
+Return observability snapshot for compose/commit logs and metrics.
+
+## Test
+
+```bash
+rspec
+```
+
+## Troubleshooting
+
+### 1) `cannot load such file -- sequel/extensions/pgvector`
+`example.rb` already applies compatibility handling (`Sequel.extension 'pgvector'` and strips `database.extensions` from DB connect config).
+
+### 2) `Config file not found: config/llm_config.yml`
+`example.rb` injects `config_path: ./config/example_llm.yml` for SmartRAG EmbeddingService startup.
+
+### 3) `ruby-lsp: not found`
+```bash
+gem install --user-install ruby-lsp debug
+```
+and ensure user gem `bin` is in `PATH`.
+
+## Roadmap
+
+- migrate EventStore/MemoryStore from in-memory to Postgres-backed implementations
+- integrate real reranker/embedding models
+- improve SmartRAG ingest pipeline and cross-session evaluation tooling

data/README.md

@@ -0,0 +1,173 @@
+# SmartBrain
+
+SmartBrain 是一个面向 Agent 的记忆运行时（Memory Runtime）与上下文编排器（Context Composer）。
+
+它的核心职责：
+- `commit_turn`：记录事件真相并沉淀结构化记忆
+- `compose_context`：在每轮请求前组装最小充分上下文
+- 联动 SmartRAG：对话记忆由 SmartBrain 管理，资源检索由 SmartRAG 提供
+
+## 当前进展
+
+当前仓库已实现并打通：
+- `commit_turn` / `compose_context` 主链路
+- Retention / Consolidation / Retrieval / Composition 策略
+- 检索器：exact + relational
+- 融合层：去重、规则重排、多样性、预算截断
+- SmartRAG 适配器：`NullClient` / `HttpClient` / `DirectClient`
+- 契约与可观测：`request_id` / `plan_id` / `context_id` 全链路追踪
+- RSpec 测试（单元 + 集成 + 回归）
+
+## 项目结构
+
+- `lib/smart_brain.rb`：入口 API
+- `lib/smart_brain/runtime.rb`：主运行时编排
+- `lib/smart_brain/contracts/`：RetrievalPlan / EvidencePack / ContextPackage 校验
+- `lib/smart_brain/observability/`：日志与指标
+- `lib/smart_brain/event_store/`：事件存储（当前内存实现）
+- `lib/smart_brain/memory_store/`：记忆存储（当前内存实现）
+- `lib/smart_brain/retrievers/`：exact/relational 检索
+- `lib/smart_brain/fusion/`：多源融合
+- `lib/smart_brain/context_composer/`：上下文装配
+- `lib/smart_brain/adapters/smart_rag/`：SmartRAG 适配层
+- `config/brain.yml`：策略配置
+- `example.rb`：SmartBrain + SmartAgent + SmartPrompt + SmartRAG 联动示例
+- `docs/`：设计与协议文档
+
+## 安装
+
+```bash
+bundle install
+```
+
+如遇本地权限或 shared gem 污染，建议：
+
+```bash
+bundle config set --local path 'vendor/bundle'
+bundle config set --local disable_shared_gems 'true'
+```
+
+## 快速开始（仅 SmartBrain）
+
+```ruby
+require_relative 'lib/smart_brain'
+
+SmartBrain.configure
+
+SmartBrain.commit_turn(
+  session_id: 'demo',
+  turn_events: {
+    messages: [
+      { role: 'user', content: '请记住：默认数据库是 Postgres。' },
+      { role: 'assistant', content: '已记录。' }
+    ],
+    decisions: [
+      { key: 'decision:smartbrain:storage', decision: 'Use Postgres by default' }
+    ]
+  }
+)
+
+context = SmartBrain.compose_context(
+  session_id: 'demo',
+  user_message: '继续并总结关键结论'
+)
+
+puts context[:context_id]
+puts context.dig(:debug, :trace, :request_id)
+puts context.dig(:debug, :trace, :plan_id)
+```
+
+## SmartRAG 集成方式
+
+### 1) NullClient（默认）
+
+不配置 `smart_rag_client` 时，资源证据为空，仅使用记忆侧证据。
+
+### 2) HttpClient
+
+```ruby
+transport = lambda do |plan, timeout_seconds:|
+  {
+    plan_id: 'p1',
+    supports_language_filter: true,
+    evidences: []
+  }
+end
+
+client = SmartBrain::Adapters::SmartRag::HttpClient.new(transport: transport, timeout_seconds: 2)
+SmartBrain.configure(smart_rag_client: client)
+```
+
+### 3) DirectClient（当前示例使用）
+
+```ruby
+require '/home/mlf/smart_ai/smart_rag/lib/smart_rag'
+require_relative 'lib/smart_brain/adapters/smart_rag/direct_client'
+
+rag_config = SmartRAG::Config.load('/home/mlf/smart_ai/smart_rag/config/smart_rag.yml')
+rag = SmartRAG::SmartRAG.new(rag_config)
+client = SmartBrain::Adapters::SmartRag::DirectClient.new(rag: rag)
+
+SmartBrain.configure(smart_rag_client: client)
+```
+
+## example.rb 说明（已更新）
+
+`example.rb` 演示完整链路：
+1. SmartBrain `compose_context`
+2. SmartAgent 通过 `call_worker` 调用 SmartPrompt worker
+3. SmartBrain `commit_turn`
+4. 打印 `evidence(memory/resource)` 验证 SmartRAG 是否参与
+
+示例依赖的本地文件：
+- `config/example_agent.yml`
+- `config/example_llm.yml`
+- `agents/brain_assistant.rb`
+- `workers/brain_assistant.rb`
+- `templates/brain_assistant.erb`
+
+运行：
+
+```bash
+bundle exec ruby example.rb
+```
+
+## 核心 API
+
+### `SmartBrain.configure(config_path: nil, smart_rag_client: nil, clock: -> { Time.now.utc })`
+初始化运行时并注入 SmartRAG 客户端（可选）。
+
+### `SmartBrain.commit_turn(session_id:, turn_events:)`
+写入事件、抽取记忆、冲突处理、摘要更新。
+
+### `SmartBrain.compose_context(session_id:, user_message:, agent_state: {})`
+生成 `ContextPackage`，内部包含检索计划与证据融合结果。
+
+### `SmartBrain.diagnostics`
+返回 compose/commit 观测日志与指标快照。
+
+## 测试
+
+```bash
+rspec
+```
+
+## 常见问题
+
+### 1) `cannot load such file -- sequel/extensions/pgvector`
+当前示例已在 `example.rb` 做兼容处理（`Sequel.extension 'pgvector'` + 去除 `database.extensions` 连接参数）。
+
+### 2) `Config file not found: config/llm_config.yml`
+`example.rb` 已将 SmartRAG 里 EmbeddingService 的 `config_path` 注入为 `./config/example_llm.yml`。
+
+### 3) `ruby-lsp: not found`
+```bash
+gem install --user-install ruby-lsp debug
+```
+并将用户 gem bin 加入 `PATH`。
+
+## 路线图
+
+- 将 EventStore/MemoryStore 从内存实现切换到 Postgres 实现
+- 接入真实 reranker / embedding 模型
+- 完善 SmartRAG ingest 与跨会话评估工具

data/agents/brain_assistant.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+SmartAgent.define :brain_assistant do
+  result = call_worker(:brain_assistant_worker, params, with_tools: false, with_history: true)
+  if result.call_tools
+    call_tools(result)
+    params[:text] = 'Please continue.'
+    result = call_worker(:brain_assistant_worker, params, with_tools: false, with_history: true)
+  end
+  result.content
+end

data/config/brain.yml

@@ -0,0 +1,32 @@
+policies:
+  retention:
+    summarize_after_turns: 12
+    entity_gate:
+      window_turns: 20
+      freq_threshold: 2
+    confidence:
+      user_asserted: 0.8
+      tool_derived: 0.9
+      inferred: 0.6
+  retrieval:
+    enable_resource_retrieval: auto
+    top_k: 30
+    candidate_k: 200
+    query_expansion:
+      enabled: true
+      max_queries: 8
+  composition:
+    token_limit: 8192
+    system_blocks_max_tokens: 800
+    summary_max_tokens: 600
+    recent_turns_max: 8
+    evidence_max_items: 12
+    max_snippet_chars: 800
+    diversity:
+      by_document: 3
+      by_source_uri: 2
+      memory_resource_ratio: '40/60'
+  observability:
+    trace: true
+    store_plans: true
+    store_contexts: true