rcrewai 0.5.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +32 -1
- data/README.md +30 -4
- data/ROADMAP.md +1 -2
- data/docs/superpowers/specs/2026-07-06-cognitive-memory-design.md +118 -0
- data/docs/upgrading-to-0.6.md +101 -0
- data/examples/cognitive_memory_example.rb +66 -0
- data/lib/rcrewai/agent.rb +13 -1
- data/lib/rcrewai/knowledge/embedder.rb +78 -14
- data/lib/rcrewai/knowledge/store.rb +4 -14
- data/lib/rcrewai/memory/base_memory.rb +107 -0
- data/lib/rcrewai/memory/entity_memory.rb +62 -0
- data/lib/rcrewai/memory/in_memory_store.rb +40 -0
- data/lib/rcrewai/memory/llm_entity_extractor.rb +34 -0
- data/lib/rcrewai/memory/long_term_memory.rb +39 -0
- data/lib/rcrewai/memory/short_term_memory.rb +20 -0
- data/lib/rcrewai/memory/sqlite_store.rb +114 -0
- data/lib/rcrewai/memory/tool_memory.rb +38 -0
- data/lib/rcrewai/memory.rb +57 -160
- data/lib/rcrewai/similarity.rb +43 -0
- data/lib/rcrewai/version.rb +1 -1
- data/lib/rcrewai.rb +9 -0
- data/rcrewai.gemspec +1 -0
- metadata +27 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 32edcb190b0d85db67d404b260fbb86d9b2821d6a89adf350d792b60a0e21a06
|
|
4
|
+
data.tar.gz: 46d061ecebca5a67714a6a8139d41bf9d20ce60d4eb5f0a1505d965defffaeb8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a77a41b7fa6dad5de896c7b635777ee5f2429eb826e65516b2d3a5b2d9e9446ca8de0bf6fd0fd94f9efd4159c5080a94d534aa6c65447abd4d781b8d05a63ff4
|
|
7
|
+
data.tar.gz: a65c248d53e9f346cd88c4f9f8e04280e89d67d5bf9695ff252ca14ee270864f23ee1f64395831ce7f2be95d682fc54ecb655d64a8c2b09b99800dcd561d3524
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.6.1] - 2026-07-06
|
|
11
|
+
|
|
12
|
+
Follow-ups to the 0.6.0 Cognitive Memory release, deepening the areas that
|
|
13
|
+
shipped as focused first cuts. All additive and backward compatible.
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
- Multi-provider embeddings: `Knowledge::Embedder.new(provider:)` now supports `:openai` (default), `:azure`, `:google`, and `:ollama` embedding endpoints, removing the hard OpenAI dependency for RAG and memory. `:anthropic` raises a clear error (no first-party embeddings API). Per-provider default models via `DEFAULT_MODELS`; existing OpenAI usage is unchanged.
|
|
17
|
+
- LLM-backed entity extraction: `EntityMemory` accepts a pluggable `extractor:` (anything responding to `call(text) -> [names]`); `RCrewAI::Memory::LlmEntityExtractor` prompts an LLM for entities (handling multi-word names the capitalized-token heuristic misses). Falls back to the heuristic when the extractor is absent, returns nothing, or raises. Wire via `Agent.new(memory: { entity_extractor: ... })`.
|
|
18
|
+
- Bounded SQLite recall: `Memory::SqliteStore.new(max_candidates:)` (default 1000) caps how many most-recent rows a search cosines, keeping recall cost constant as total memory grows instead of brute-forcing every row. `nil` considers all rows.
|
|
19
|
+
|
|
20
|
+
## [0.6.0] - 2026-07-06
|
|
21
|
+
|
|
22
|
+
Replaces the placeholder agent memory with a **cognitive memory** system —
|
|
23
|
+
semantic recall (embeddings + cosine), optional SQLite persistence, and four
|
|
24
|
+
memory types. This is the one area where CrewAI advanced past what the gem
|
|
25
|
+
originally ported. Backward compatible: the `Memory` public API is unchanged and
|
|
26
|
+
the zero-config default behaves as before.
|
|
27
|
+
|
|
28
|
+
### Added
|
|
29
|
+
- Cognitive memory: `RCrewAI::Memory` is now a semantic, optionally-persistent, multi-type memory system, replacing the previous word-overlap placeholder. It composes four memory types — `ShortTermMemory` (recent executions, capped), `LongTermMemory` (durable, deduped insights), `EntityMemory` (facts about entities seen in work), and `ToolMemory` (tool-call history) — behind the original `Memory` public API.
|
|
30
|
+
- Semantic recall: pass `Agent.new(memory: { embedder: RCrewAI::Knowledge::Embedder.new })` to recall conceptually related past work via embeddings + cosine similarity. Without an embedder, recall falls back to lexical (word-overlap) similarity — and embedding failures fall back gracefully, so memory never breaks execution.
|
|
31
|
+
- Persistent memory: `RCrewAI::Memory::SqliteStore` persists memories to SQLite (vectors packed as floats, metadata as JSON) so recall survives restarts. Default store is `InMemoryStore` (volatile). The store interface is pluggable.
|
|
32
|
+
- Memory scoping: each agent's memory is scoped to its name by default, so agents sharing a persistent store don't cross-read; override via `memory: { scope: ... }`.
|
|
33
|
+
- `RCrewAI::Similarity` — shared cosine + lexical similarity helpers (extracted from `Knowledge::Store`, now used by both Knowledge and Memory).
|
|
34
|
+
- `sqlite3` added as a runtime dependency (required lazily; only needed for `SqliteStore`).
|
|
35
|
+
|
|
36
|
+
### Changed
|
|
37
|
+
- `Agent.new(memory:)` accepts a pre-built `Memory`, an options hash (`{ embedder:, store:, scope:, short_term_limit: }`), or nothing (zero-config default, unchanged behavior).
|
|
38
|
+
|
|
10
39
|
## [0.5.0] - 2026-07-03
|
|
11
40
|
|
|
12
41
|
Polish release completing the roadmap backlog: crew lifecycle hooks, batch
|
|
@@ -179,7 +208,9 @@ output, guardrails, planning, and training/testing. See `ROADMAP.md`.
|
|
|
179
208
|
- CLI usage documentation
|
|
180
209
|
- Real-world use cases and examples
|
|
181
210
|
|
|
182
|
-
[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.
|
|
211
|
+
[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.6.1...HEAD
|
|
212
|
+
[0.6.1]: https://github.com/gkosmo/rcrewAI/compare/v0.6.0...v0.6.1
|
|
213
|
+
[0.6.0]: https://github.com/gkosmo/rcrewAI/compare/v0.5.0...v0.6.0
|
|
183
214
|
[0.5.0]: https://github.com/gkosmo/rcrewAI/compare/v0.4.0...v0.5.0
|
|
184
215
|
[0.4.0]: https://github.com/gkosmo/rcrewAI/compare/v0.3.0...v0.4.0
|
|
185
216
|
[0.3.0]: https://github.com/gkosmo/rcrewAI/compare/v0.1.0...v0.3.0
|
data/README.md
CHANGED
|
@@ -13,7 +13,7 @@ RCrewAI is a Ruby implementation of the CrewAI framework, allowing you to create
|
|
|
13
13
|
- **🤖 Intelligent Agents**: AI agents with reasoning loops, memory, and tool usage capabilities
|
|
14
14
|
- **🔗 Multi-LLM Support**: OpenAI, Anthropic (Claude), Google (Gemini), Azure OpenAI, and Ollama
|
|
15
15
|
- **🛠️ Rich Tool Ecosystem**: Web search, file operations, SQL, email, code execution, PDF processing, and custom tools
|
|
16
|
-
- **🧠
|
|
16
|
+
- **🧠 Cognitive Memory**: Semantic recall (embeddings + cosine) with optional SQLite persistence and short-term/long-term/entity/tool memory types
|
|
17
17
|
- **🤝 Human-in-the-Loop**: Interactive approval workflows, human guidance, and collaborative decision making
|
|
18
18
|
- **⚡ Advanced Task System**: Dependencies, retries, async/concurrent execution, and context sharing
|
|
19
19
|
- **🏗️ Hierarchical Teams**: Manager agents that coordinate and delegate tasks to specialist agents
|
|
@@ -370,9 +370,35 @@ support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
|
|
|
370
370
|
crew = RCrewAI::Crew.new('support_crew', knowledge: kb)
|
|
371
371
|
```
|
|
372
372
|
|
|
373
|
-
Embeddings default to OpenAI's `text-embedding-3-small
|
|
374
|
-
`
|
|
375
|
-
|
|
373
|
+
Embeddings default to OpenAI's `text-embedding-3-small`. Use another provider
|
|
374
|
+
with `RCrewAI::Knowledge::Embedder.new(provider: :ollama)` (also `:azure`,
|
|
375
|
+
`:google`; `:anthropic` has no embeddings API), or pass any custom `embedder:`
|
|
376
|
+
(anything responding to `embed(texts)`) / vector store to swap the backend.
|
|
377
|
+
|
|
378
|
+
## 🧠 Cognitive Memory
|
|
379
|
+
|
|
380
|
+
Agents remember what they've done and recall it semantically on future tasks.
|
|
381
|
+
Memory is zero-config by default (in-memory, lexical recall); add an embedder
|
|
382
|
+
for semantic recall and a SQLite store for persistence:
|
|
383
|
+
|
|
384
|
+
```ruby
|
|
385
|
+
embedder = RCrewAI::Knowledge::Embedder.new
|
|
386
|
+
store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
|
|
387
|
+
|
|
388
|
+
agent = RCrewAI::Agent.new(
|
|
389
|
+
name: 'engineer', role: '...', goal: '...',
|
|
390
|
+
memory: { embedder: embedder, store: store } # both optional
|
|
391
|
+
)
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
- **Semantic recall** — with an embedder, an agent recalls conceptually related
|
|
395
|
+
past work even when the wording differs (falls back to word-overlap without one).
|
|
396
|
+
- **Persistence** — a `SqliteStore` makes memory survive restarts.
|
|
397
|
+
- **Memory types** — `agent.memory.short_term` / `long_term` / `entity` / `tool`.
|
|
398
|
+
- **Scoping** — memory is scoped per agent so a shared store doesn't cross-read.
|
|
399
|
+
|
|
400
|
+
Memory is best-effort: embedding failures fall back to lexical similarity, so it
|
|
401
|
+
never breaks agent execution.
|
|
376
402
|
|
|
377
403
|
## 🌊 Flows
|
|
378
404
|
|
data/ROADMAP.md
CHANGED
|
@@ -19,8 +19,7 @@ Since CrewAI's `1.0`, the framework grew a second pillar (**Flows**) plus
|
|
|
19
19
|
**Training/Testing**. RCrewAI now implements all of these — see the matrix below.
|
|
20
20
|
|
|
21
21
|
**Status: complete.** All milestone issues (#5–#12) shipped in v0.4.0, and all
|
|
22
|
-
backlog items (#15–#20)
|
|
23
|
-
release). There is no outstanding roadmap work.
|
|
22
|
+
backlog items (#15–#20) shipped in v0.5.0. There is no outstanding roadmap work.
|
|
24
23
|
|
|
25
24
|
## Parity matrix
|
|
26
25
|
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# Cognitive Memory: Semantic, Persistent, Multi-Type Agent Memory
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-07-06
|
|
4
|
+
**Status:** Approved design (pending implementation)
|
|
5
|
+
**Target version:** `rcrewai` 0.6.0 (current is 0.5.0)
|
|
6
|
+
**Scope:** Replace the placeholder `RCrewAI::Memory` with a cognitive memory system modeled on CrewAI's memory taxonomy — short-term (semantic recent recall), long-term (persistent insights), entity memory, and tool-usage memory — backed by embeddings and a persistent SQLite vector store. No new external dependencies; reuses the in-repo `Knowledge::Embedder` and cosine-search infrastructure.
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Motivation
|
|
11
|
+
|
|
12
|
+
The current `RCrewAI::Memory` is a placeholder. It matches "relevant" past executions with word-overlap similarity (`common_words / total_words`) plus a hardcoded keyword/stopword list and a `classify_task_type` that greps the description for words like "research" or "write". Nothing is embedded; nothing is persisted; everything is lost when the process exits. It is the weakest module in the codebase and it undercuts the framework's core promise — agents that learn from what they've done.
|
|
13
|
+
|
|
14
|
+
Meanwhile CrewAI has advanced *past* the version this project ported: its 2026 "cognitive memory" work adds semantic recall, distinct memory types (short-term, long-term, entity), and persistence. This is genuine parity work, not gold-plating — and the infrastructure to do it well already exists in-repo (`Knowledge::Embedder`, cosine `Knowledge::Store`), so we compose rather than build from scratch.
|
|
15
|
+
|
|
16
|
+
## Goals
|
|
17
|
+
|
|
18
|
+
1. **Semantic recall.** Replace word-overlap with embedding-based similarity so an agent recalls conceptually related past work, not just keyword-overlapping work.
|
|
19
|
+
2. **Persistence.** Memory survives process restarts via a SQLite-backed vector store (no external service).
|
|
20
|
+
3. **Memory types.** Model CrewAI's taxonomy: short-term, long-term, entity, and tool-usage memory, each with clear write/read semantics.
|
|
21
|
+
4. **Backward compatibility.** The existing `Memory` public API (`add_execution`, `add_tool_usage`, `relevant_executions`, `tool_usage_for`, `clear_*!`, `stats`) keeps working, so `Agent` and the runners need no changes to function.
|
|
22
|
+
5. **No new hard dependencies.** SQLite via Ruby's stdlib-adjacent `sqlite3` gem (already common in Ruby projects) — but the store is pluggable, and the default remains in-memory so the gem works with zero setup.
|
|
23
|
+
|
|
24
|
+
## Non-goals
|
|
25
|
+
|
|
26
|
+
- External vector databases (Chroma/Qdrant/pgvector). The pluggable store interface leaves room for them later; we ship SQLite + in-memory.
|
|
27
|
+
- Reranking, query rewriting, or summarization of memories (CrewAI has some of this; it's a follow-up).
|
|
28
|
+
- Multi-tenant / shared cross-agent memory servers.
|
|
29
|
+
|
|
30
|
+
## Decisions captured during brainstorming
|
|
31
|
+
|
|
32
|
+
| # | Decision | Choice |
|
|
33
|
+
|---|----------|--------|
|
|
34
|
+
| 1 | Similarity mechanism | Embeddings + cosine, reusing `Knowledge::Embedder` and the cosine math already in `Knowledge::Store`. |
|
|
35
|
+
| 2 | Persistence | Pluggable vector store; ship `InMemoryStore` (default) and `SqliteStore`. |
|
|
36
|
+
| 3 | Memory taxonomy | Four types: `ShortTermMemory`, `LongTermMemory`, `EntityMemory`, `ToolMemory`. |
|
|
37
|
+
| 4 | Backward compat | Keep the `Memory` facade and its current method signatures; delegate internally to the new types. |
|
|
38
|
+
| 5 | Default behavior | Zero-config: in-memory store, embeddings only if an embedder is available/keyed; graceful fallback to the current lexical similarity when no embedder. |
|
|
39
|
+
| 6 | Embedding failures | Never fatal. If embedding fails (no key, network), fall back to lexical similarity so agents still run. |
|
|
40
|
+
| 7 | Namespacing | Memories are scoped by agent (role/name) so one agent's memory doesn't leak into another, matching CrewAI's per-role collections. |
|
|
41
|
+
|
|
42
|
+
## Architecture
|
|
43
|
+
|
|
44
|
+
```
|
|
45
|
+
RCrewAI::Memory (facade — preserves today's public API)
|
|
46
|
+
├── ShortTermMemory recent executions, semantic recall, capped, volatile-by-default
|
|
47
|
+
├── LongTermMemory durable insights from successful executions, persistent
|
|
48
|
+
├── EntityMemory facts about entities (people, systems, concepts) extracted from work
|
|
49
|
+
└── ToolMemory tool-call history + outcomes (replaces @tool_usage)
|
|
50
|
+
|
|
51
|
+
Each memory type holds:
|
|
52
|
+
- a VectorStore (InMemoryStore | SqliteStore) ← persistence + search
|
|
53
|
+
- an Embedder (Knowledge::Embedder | nil) ← semantic vectors
|
|
54
|
+
- a lexical fallback (current word-overlap) ← when no embedder
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Storage: `Memory::VectorStore`
|
|
58
|
+
|
|
59
|
+
A small interface (mirrors `Knowledge::Store` so they can converge later):
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
store.add(id:, text:, vector:, metadata:) # upsert a record
|
|
63
|
+
store.search(vector, k:, scope:) # cosine top-k, filtered by scope
|
|
64
|
+
store.all(scope:) # enumerate (for stats / lexical fallback)
|
|
65
|
+
store.delete(scope:) # clear a scope
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- **`InMemoryStore`** — array of records; cosine in Ruby (lift the existing private cosine out of `Knowledge::Store` into a shared `Similarity` helper). Default.
|
|
69
|
+
- **`SqliteStore`** — one row per memory: `id, scope, type, text, vector (blob), metadata (json), created_at`. Vectors stored as packed floats (`Array#pack('e*')`). Cosine computed in Ruby over candidate rows filtered by `scope`/`type` (fine for the thousands-of-memories scale agents produce; a real ANN index is a later optimization). Opened at a configurable path (default `~/.rcrewai/memory.db` or `:memory:`).
|
|
70
|
+
|
|
71
|
+
### Embeddings
|
|
72
|
+
|
|
73
|
+
Reuse `Knowledge::Embedder`. The memory system takes an optional `embedder:`; when present, `add_*` embeds the text and stores the vector, and recall embeds the query. When absent (no key, or explicitly disabled), the store keeps `vector: nil` and recall falls back to the current lexical similarity over `store.all`. This keeps the zero-config path working.
|
|
74
|
+
|
|
75
|
+
### The four memory types
|
|
76
|
+
|
|
77
|
+
- **ShortTermMemory** — every execution is written here; recall returns the top-k semantically similar recent items. Capped (default 100) and, by default, uses the in-memory store (volatile). This is the direct upgrade of today's `@short_term` + `relevant_executions`.
|
|
78
|
+
- **LongTermMemory** — successful executions promote a distilled "insight" record (task type, what worked, result summary) into a persistent store, deduped by similarity so we don't store near-identical insights repeatedly. Upgrade of today's `@long_term`.
|
|
79
|
+
- **EntityMemory** — optional lightweight entity extraction (heuristic first: capitalized noun phrases / quoted terms; pluggable LLM extractor later) so agents accumulate facts about recurring entities. New capability.
|
|
80
|
+
- **ToolMemory** — replaces `@tool_usage`; same API (`add_tool_usage`, `tool_usage_for`) but persistable and searchable.
|
|
81
|
+
|
|
82
|
+
### Facade: `RCrewAI::Memory`
|
|
83
|
+
|
|
84
|
+
Keeps today's signatures, delegating to the types:
|
|
85
|
+
|
|
86
|
+
```ruby
|
|
87
|
+
Memory.new(embedder: nil, store: nil, scope: nil, short_term_limit: 100)
|
|
88
|
+
|
|
89
|
+
add_execution(task, result, execution_time) # -> ShortTerm + (if success) LongTerm + Entity
|
|
90
|
+
add_tool_usage(tool_name, params, result) # -> ToolMemory
|
|
91
|
+
relevant_executions(task, limit = 3) # -> semantic recall across Short+Long, formatted string|nil
|
|
92
|
+
tool_usage_for(tool_name, limit = 5) # -> ToolMemory
|
|
93
|
+
clear_short_term! / clear_all! / stats # -> delegate
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
`Agent#initialize` gains optional `memory:` config (embedder/store/persistence) but defaults to the zero-config in-memory, lexical-fallback behavior — so existing code is unchanged.
|
|
97
|
+
|
|
98
|
+
## Backward compatibility & migration
|
|
99
|
+
|
|
100
|
+
- The `Memory` public API is preserved exactly; `agent.rb` and the runners are untouched functionally.
|
|
101
|
+
- Default construction (`Memory.new` with no args) behaves like today from the caller's perspective — just with better recall when an embedder is configured, and identical lexical recall when not.
|
|
102
|
+
- `relevant_executions` still returns the same formatted-string-or-nil shape consumed at `agent.rb:241`.
|
|
103
|
+
|
|
104
|
+
## Testing strategy
|
|
105
|
+
|
|
106
|
+
- **Unit**: each memory type with a fake deterministic embedder (as `knowledge_spec.rb` already does) — assert semantic recall ranks the conceptually-closest item first, not the keyword-overlapping one.
|
|
107
|
+
- **Persistence**: `SqliteStore` round-trip — write memories, reopen the DB, recall survives.
|
|
108
|
+
- **Fallback**: with no embedder, recall still returns results via lexical similarity (proves the graceful path).
|
|
109
|
+
- **Facade compat**: the existing `memory_spec.rb` expectations continue to pass (or are updated only where the placeholder behavior was clearly a bug).
|
|
110
|
+
- **Agent integration**: an agent with `reasoning`/knowledge off recalls a semantically-related prior execution injected into its context.
|
|
111
|
+
|
|
112
|
+
## Dependencies
|
|
113
|
+
|
|
114
|
+
- `sqlite3` gem — added as a runtime dependency, but only required lazily inside `SqliteStore` so the gem loads and the in-memory path works even if it's absent. (If we'd rather avoid the dependency entirely, the fallback is a JSON-file-backed store; SQLite is preferred for query/scale.)
|
|
115
|
+
|
|
116
|
+
## Rollout
|
|
117
|
+
|
|
118
|
+
Ship in `0.6.0`. `docs/upgrading-to-0.6.md` documents opt-in persistence and embeddings; the default path is unchanged, so it's a no-action upgrade for existing users.
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# Upgrading to RCrewAI 0.6
|
|
2
|
+
|
|
3
|
+
RCrewAI 0.6 replaces the placeholder memory with a **cognitive memory** system:
|
|
4
|
+
semantic recall (embeddings + cosine), optional SQLite persistence, and four
|
|
5
|
+
memory types (short-term, long-term, entity, tool).
|
|
6
|
+
|
|
7
|
+
**0.6 is backward compatible.** The `RCrewAI::Memory` public API is unchanged,
|
|
8
|
+
and the default (`Memory.new` with no config) behaves like before from the
|
|
9
|
+
caller's perspective — just with better recall once an embedder is configured.
|
|
10
|
+
Everything below is opt-in.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 1. What you must do
|
|
15
|
+
|
|
16
|
+
Nothing. Upgrade the gem; existing code keeps working:
|
|
17
|
+
|
|
18
|
+
```ruby
|
|
19
|
+
gem 'rcrewai', '~> 0.6'
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Note: `0.6` adds `sqlite3` as a runtime dependency (used only if you opt into
|
|
23
|
+
persistence). It is required lazily, so the in-memory default works even if the
|
|
24
|
+
native extension isn't present.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 2. What you can do (new capabilities)
|
|
29
|
+
|
|
30
|
+
### 2a. Semantic recall via embeddings
|
|
31
|
+
|
|
32
|
+
By default, memory recall uses lexical (word-overlap) similarity — same as
|
|
33
|
+
before. Pass an embedder to get semantic recall, so an agent remembers
|
|
34
|
+
conceptually related work even when the wording differs:
|
|
35
|
+
|
|
36
|
+
```ruby
|
|
37
|
+
embedder = RCrewAI::Knowledge::Embedder.new # OpenAI text-embedding-3-small
|
|
38
|
+
agent = RCrewAI::Agent.new(
|
|
39
|
+
name: 'engineer', role: '...', goal: '...',
|
|
40
|
+
memory: { embedder: embedder }
|
|
41
|
+
)
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
If embedding fails (no API key, network error), recall automatically falls back
|
|
45
|
+
to lexical similarity — memory never breaks agent execution.
|
|
46
|
+
|
|
47
|
+
### 2b. Persistent memory (SQLite)
|
|
48
|
+
|
|
49
|
+
Give memory a SQLite store and it survives restarts:
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
|
|
53
|
+
agent = RCrewAI::Agent.new(
|
|
54
|
+
name: 'engineer', role: '...', goal: '...',
|
|
55
|
+
memory: { embedder: embedder, store: store }
|
|
56
|
+
)
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The default store is in-memory (volatile). Any object implementing the store
|
|
60
|
+
interface (`add` / `search` / `all` / `delete` / `delete_record`) can be used —
|
|
61
|
+
e.g. a future Chroma/pgvector adapter.
|
|
62
|
+
|
|
63
|
+
### 2c. Memory scoping
|
|
64
|
+
|
|
65
|
+
Each agent's memory is scoped to its name by default, so agents sharing a
|
|
66
|
+
persistent store don't read each other's memories. Override with
|
|
67
|
+
`memory: { scope: 'custom' }` if you want deliberate sharing.
|
|
68
|
+
|
|
69
|
+
### 2d. Memory types (direct access)
|
|
70
|
+
|
|
71
|
+
The `Memory` facade exposes the underlying types for advanced use:
|
|
72
|
+
|
|
73
|
+
```ruby
|
|
74
|
+
agent.memory.short_term # recent executions, semantic recall, capped
|
|
75
|
+
agent.memory.long_term # durable insights from successful executions, deduped
|
|
76
|
+
agent.memory.entity # facts about entities (people, systems) seen in work
|
|
77
|
+
agent.memory.tool # tool-call history + outcomes
|
|
78
|
+
|
|
79
|
+
agent.memory.entity.entities # => ["Alice", "AWS", ...]
|
|
80
|
+
agent.memory.long_term.recall('...', limit: 3)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### 2e. Pre-built Memory
|
|
84
|
+
|
|
85
|
+
You can also construct and pass a `Memory` directly:
|
|
86
|
+
|
|
87
|
+
```ruby
|
|
88
|
+
memory = RCrewAI::Memory.new(scope: 'shared', embedder: embedder, store: store)
|
|
89
|
+
agent = RCrewAI::Agent.new(name: 'a', role: '...', goal: '...', memory: memory)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Behavior notes
|
|
95
|
+
|
|
96
|
+
- `add_execution` writes to short-term always, and (on success) promotes a
|
|
97
|
+
deduped insight to long-term and extracts entities.
|
|
98
|
+
- `relevant_executions(task, limit)` recalls across short- and long-term and
|
|
99
|
+
returns the same formatted-string-or-nil shape as before.
|
|
100
|
+
- `clear_short_term!` leaves long-term insights intact; `clear_all!` wipes
|
|
101
|
+
everything.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# Cognitive Memory — semantic recall + persistence.
|
|
5
|
+
#
|
|
6
|
+
# Agents remember past executions and recall them by *meaning*, not just shared
|
|
7
|
+
# words. With a SQLite store, memory survives restarts.
|
|
8
|
+
#
|
|
9
|
+
# This example uses a fake, deterministic embedder so it runs WITHOUT an API
|
|
10
|
+
# key. In real use you'd pass `RCrewAI::Knowledge::Embedder.new` (OpenAI).
|
|
11
|
+
#
|
|
12
|
+
# Run:
|
|
13
|
+
# ruby examples/cognitive_memory_example.rb
|
|
14
|
+
|
|
15
|
+
require_relative '../lib/rcrewai'
|
|
16
|
+
require 'tmpdir'
|
|
17
|
+
|
|
18
|
+
RCrewAI.configure(validate: false) do |c|
|
|
19
|
+
c.llm_provider = :openai
|
|
20
|
+
c.api_key = 'demo-key'
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# Fake embedder: maps text to a concept vector by keyword. Any object
|
|
24
|
+
# responding to embed(texts) -> [[float, ...], ...] works.
|
|
25
|
+
class ConceptEmbedder
|
|
26
|
+
def embed(texts)
|
|
27
|
+
texts.map do |t|
|
|
28
|
+
l = t.downcase
|
|
29
|
+
[
|
|
30
|
+
l.match?(/payment|billing|invoice|charge/) ? 1.0 : 0.0,
|
|
31
|
+
l.match?(/auth|login|session|token/) ? 1.0 : 0.0,
|
|
32
|
+
l.match?(/deploy|release|ci|pipeline/) ? 1.0 : 0.0
|
|
33
|
+
]
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
Task = Struct.new(:name, :description)
|
|
39
|
+
|
|
40
|
+
Dir.mktmpdir do |dir|
|
|
41
|
+
store = RCrewAI::Memory::SqliteStore.new(path: File.join(dir, 'memory.db'))
|
|
42
|
+
agent = RCrewAI::Agent.new(
|
|
43
|
+
name: 'engineer', role: 'Senior engineer', goal: 'Ship reliable software',
|
|
44
|
+
memory: { embedder: ConceptEmbedder.new, store: store }
|
|
45
|
+
)
|
|
46
|
+
|
|
47
|
+
# Record a few past executions.
|
|
48
|
+
agent.memory.add_execution(Task.new('t1', 'fixed the billing invoice bug'),
|
|
49
|
+
'adjusted the payment retry logic', 1.2)
|
|
50
|
+
agent.memory.add_execution(Task.new('t2', 'patched the login session flow'),
|
|
51
|
+
'rotated the auth tokens', 0.8)
|
|
52
|
+
agent.memory.add_execution(Task.new('t3', 'sped up the release pipeline'),
|
|
53
|
+
'parallelized the CI stages', 2.0)
|
|
54
|
+
|
|
55
|
+
puts '== Semantic recall =='
|
|
56
|
+
# Query shares NO words with the billing execution, but is conceptually close.
|
|
57
|
+
query = Task.new('q', 'a customer got double-charged on checkout')
|
|
58
|
+
puts agent.memory.relevant_executions(query, 1)
|
|
59
|
+
|
|
60
|
+
puts "\n== Persistence (reopen the DB in a fresh agent) =="
|
|
61
|
+
store2 = RCrewAI::Memory::SqliteStore.new(path: File.join(dir, 'memory.db'))
|
|
62
|
+
agent2 = RCrewAI::Agent.new(name: 'engineer', role: 'r', goal: 'g',
|
|
63
|
+
memory: { embedder: ConceptEmbedder.new, store: store2 })
|
|
64
|
+
puts agent2.memory.relevant_executions(query, 1)
|
|
65
|
+
puts "stats: #{agent2.memory.stats.inspect}"
|
|
66
|
+
end
|
data/lib/rcrewai/agent.rb
CHANGED
|
@@ -39,7 +39,7 @@ module RCrewAI
|
|
|
39
39
|
@reasoning = options.fetch(:reasoning, false)
|
|
40
40
|
@max_reasoning_attempts = options.fetch(:max_reasoning_attempts, 3)
|
|
41
41
|
@respect_context_window = options.fetch(:respect_context_window, false)
|
|
42
|
-
@memory =
|
|
42
|
+
@memory = build_memory(options[:memory])
|
|
43
43
|
@rate_limiter = options[:max_rpm] ? RateLimiter.new(max_rpm: options[:max_rpm]) : nil
|
|
44
44
|
@llm_client = wrap_with_rate_limiter(build_llm_client(options[:llm]))
|
|
45
45
|
@knowledge = build_knowledge(options[:knowledge], options[:knowledge_sources])
|
|
@@ -220,6 +220,18 @@ module RCrewAI
|
|
|
220
220
|
RateLimiter::ThrottledClient.new(client, @rate_limiter)
|
|
221
221
|
end
|
|
222
222
|
|
|
223
|
+
# Builds the agent's memory. Accepts a pre-built Memory, an options hash
|
|
224
|
+
# (`{ embedder:, store:, scope:, short_term_limit: }`), or nil for the
|
|
225
|
+
# zero-config default. Memory is scoped to the agent's name so agents don't
|
|
226
|
+
# share recall (matters when a persistent store is shared).
|
|
227
|
+
def build_memory(memory)
|
|
228
|
+
return memory if memory.is_a?(Memory)
|
|
229
|
+
|
|
230
|
+
opts = memory.is_a?(Hash) ? memory : {}
|
|
231
|
+
Memory.new(scope: opts.fetch(:scope, name),
|
|
232
|
+
**opts.slice(:embedder, :store, :short_term_limit, :entity_extractor))
|
|
233
|
+
end
|
|
234
|
+
|
|
223
235
|
# Accepts a pre-built Knowledge::Base via +knowledge:+ or an array of
|
|
224
236
|
# sources via +knowledge_sources:+ (wrapped in a Base). Returns nil if
|
|
225
237
|
# neither is given.
|
|
@@ -5,36 +5,100 @@ require 'faraday'
|
|
|
5
5
|
|
|
6
6
|
module RCrewAI
|
|
7
7
|
module Knowledge
|
|
8
|
-
# Turns text into embedding vectors.
|
|
9
|
-
#
|
|
10
|
-
#
|
|
8
|
+
# Turns text into embedding vectors. Supports multiple providers
|
|
9
|
+
# (OpenAI [default], Azure OpenAI, Google Gemini, Ollama); Anthropic has no
|
|
10
|
+
# first-party embeddings API and raises a clear error. `#embed` takes an
|
|
11
|
+
# array of strings and returns an array of vectors. Any object responding to
|
|
12
|
+
# `#embed` can be substituted.
|
|
11
13
|
class Embedder
|
|
12
|
-
|
|
14
|
+
DEFAULT_MODELS = {
|
|
15
|
+
openai: 'text-embedding-3-small',
|
|
16
|
+
azure: 'text-embedding-3-small',
|
|
17
|
+
google: 'text-embedding-004',
|
|
18
|
+
ollama: 'nomic-embed-text'
|
|
19
|
+
}.freeze
|
|
20
|
+
|
|
13
21
|
OPENAI_URL = 'https://api.openai.com/v1/embeddings'
|
|
22
|
+
GOOGLE_BASE = 'https://generativelanguage.googleapis.com/v1beta'
|
|
23
|
+
OLLAMA_DEFAULT_URL = 'http://localhost:11434'
|
|
24
|
+
|
|
25
|
+
attr_reader :provider, :model
|
|
26
|
+
|
|
27
|
+
def initialize(provider: :openai, model: nil, api_key: nil, config: RCrewAI.configuration)
|
|
28
|
+
@provider = provider.to_sym
|
|
29
|
+
if @provider == :anthropic
|
|
30
|
+
raise EmbeddingError,
|
|
31
|
+
'anthropic does not provide an embeddings API; use :openai, :azure, :google, or :ollama'
|
|
32
|
+
end
|
|
14
33
|
|
|
15
|
-
|
|
16
|
-
@model = model
|
|
17
|
-
@api_key = api_key
|
|
34
|
+
@config = config
|
|
35
|
+
@model = model || DEFAULT_MODELS[@provider] || DEFAULT_MODELS[:openai]
|
|
36
|
+
@api_key = api_key
|
|
18
37
|
end
|
|
19
38
|
|
|
20
39
|
def embed(texts)
|
|
21
40
|
texts = Array(texts)
|
|
22
41
|
return [] if texts.empty?
|
|
23
42
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
43
|
+
send("embed_#{@provider}", texts)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
private
|
|
47
|
+
|
|
48
|
+
def embed_openai(texts)
|
|
49
|
+
body = post_json(OPENAI_URL, { model: @model, input: texts },
|
|
50
|
+
'Authorization' => "Bearer #{api_key_for(:openai)}")
|
|
51
|
+
body['data'].map { |d| d['embedding'] }
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def embed_azure(texts)
|
|
55
|
+
base = @config.base_url
|
|
56
|
+
version = @config.api_version || '2024-02-01'
|
|
57
|
+
deployment = @config.deployment_name || @model
|
|
58
|
+
url = "#{base}/openai/deployments/#{deployment}/embeddings?api-version=#{version}"
|
|
59
|
+
body = post_json(url, { input: texts }, 'api-key' => api_key_for(:azure))
|
|
60
|
+
body['data'].map { |d| d['embedding'] }
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def embed_google(texts)
|
|
64
|
+
key = api_key_for(:google)
|
|
65
|
+
texts.map do |text|
|
|
66
|
+
url = "#{GOOGLE_BASE}/models/#{@model}:embedContent?key=#{key}"
|
|
67
|
+
payload = { model: "models/#{@model}", content: { parts: [{ text: text }] } }
|
|
68
|
+
body = post_json(url, payload)
|
|
69
|
+
body.dig('embedding', 'values')
|
|
28
70
|
end
|
|
71
|
+
end
|
|
29
72
|
|
|
73
|
+
def embed_ollama(texts)
|
|
74
|
+
base = @config.base_url || OLLAMA_DEFAULT_URL
|
|
75
|
+
texts.map do |text|
|
|
76
|
+
body = post_json("#{base}/api/embeddings", { model: @model, prompt: text })
|
|
77
|
+
body['embedding']
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def post_json(url, payload, headers = {})
|
|
82
|
+
response = connection.post(url) do |req|
|
|
83
|
+
req.headers['Content-Type'] = 'application/json'
|
|
84
|
+
headers.each { |k, v| req.headers[k] = v }
|
|
85
|
+
req.body = JSON.generate(payload)
|
|
86
|
+
end
|
|
30
87
|
raise EmbeddingError, "embedding request failed: #{response.status}" unless response.success?
|
|
31
88
|
|
|
32
89
|
body = response.body
|
|
33
|
-
body
|
|
34
|
-
body['data'].map { |d| d['embedding'] }
|
|
90
|
+
body.is_a?(String) ? JSON.parse(body) : body
|
|
35
91
|
end
|
|
36
92
|
|
|
37
|
-
|
|
93
|
+
def api_key_for(provider)
|
|
94
|
+
return @api_key if @api_key
|
|
95
|
+
|
|
96
|
+
case provider
|
|
97
|
+
when :openai then @config.openai_api_key || @config.api_key
|
|
98
|
+
when :azure then @config.azure_api_key || @config.api_key
|
|
99
|
+
when :google then @config.google_api_key || @config.api_key
|
|
100
|
+
end
|
|
101
|
+
end
|
|
38
102
|
|
|
39
103
|
def connection
|
|
40
104
|
@connection ||= Faraday.new do |f|
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative '../similarity'
|
|
4
|
+
|
|
3
5
|
module RCrewAI
|
|
4
6
|
module Knowledge
|
|
5
7
|
# In-memory vector store with cosine-similarity search. The default backing
|
|
@@ -38,20 +40,8 @@ module RCrewAI
|
|
|
38
40
|
|
|
39
41
|
private
|
|
40
42
|
|
|
41
|
-
def cosine_similarity(
|
|
42
|
-
|
|
43
|
-
norm_a = 0.0
|
|
44
|
-
norm_b = 0.0
|
|
45
|
-
a.each_index do |i|
|
|
46
|
-
ai = a[i].to_f
|
|
47
|
-
bi = (b[i] || 0).to_f
|
|
48
|
-
dot += ai * bi
|
|
49
|
-
norm_a += ai * ai
|
|
50
|
-
norm_b += bi * bi
|
|
51
|
-
end
|
|
52
|
-
return 0.0 if norm_a.zero? || norm_b.zero?
|
|
53
|
-
|
|
54
|
-
dot / (Math.sqrt(norm_a) * Math.sqrt(norm_b))
|
|
43
|
+
def cosine_similarity(vec_a, vec_b)
|
|
44
|
+
Similarity.cosine(vec_a, vec_b)
|
|
55
45
|
end
|
|
56
46
|
end
|
|
57
47
|
end
|