rcrewai 0.5.0 → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 68a381d21e047e37972fda09fd7e9faa93da77e6fe01a040a4c77b9ee0066aca
4
- data.tar.gz: 3818acc6b9b6eb71d27d4099cb1c1fc574adbdd0a3a8128389803c051c364774
3
+ metadata.gz: 32edcb190b0d85db67d404b260fbb86d9b2821d6a89adf350d792b60a0e21a06
4
+ data.tar.gz: 46d061ecebca5a67714a6a8139d41bf9d20ce60d4eb5f0a1505d965defffaeb8
5
5
  SHA512:
6
- metadata.gz: 65d796bde314db55466f7f4f043db9eeb772dee3e72914d2bc27947eb6dfe84ce246b89a6181d473c3f7d175d353fbc9531914c8baa5b7b010e02229324bac27
7
- data.tar.gz: b2fffaab35dc672b12d2a5aea5fd0b02e6519b2f4a5ca98daaf66bef740551e2e674f4cc39967f3d987ed9dfe1c13e7dec4c8a30e5b86e2140242e347dd96a11
6
+ metadata.gz: a77a41b7fa6dad5de896c7b635777ee5f2429eb826e65516b2d3a5b2d9e9446ca8de0bf6fd0fd94f9efd4159c5080a94d534aa6c65447abd4d781b8d05a63ff4
7
+ data.tar.gz: a65c248d53e9f346cd88c4f9f8e04280e89d67d5bf9695ff252ca14ee270864f23ee1f64395831ce7f2be95d682fc54ecb655d64a8c2b09b99800dcd561d3524
data/CHANGELOG.md CHANGED
@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.6.1] - 2026-07-06
11
+
12
+ Follow-ups to the 0.6.0 Cognitive Memory release, deepening the areas that
13
+ shipped as focused first cuts. All additive and backward compatible.
14
+
15
+ ### Added
16
+ - Multi-provider embeddings: `Knowledge::Embedder.new(provider:)` now supports `:openai` (default), `:azure`, `:google`, and `:ollama` embedding endpoints, removing the hard OpenAI dependency for RAG and memory. `:anthropic` raises a clear error (no first-party embeddings API). Per-provider default models via `DEFAULT_MODELS`; existing OpenAI usage is unchanged.
17
+ - LLM-backed entity extraction: `EntityMemory` accepts a pluggable `extractor:` (anything responding to `call(text) -> [names]`); `RCrewAI::Memory::LlmEntityExtractor` prompts an LLM for entities (handling multi-word names the capitalized-token heuristic misses). Falls back to the heuristic when the extractor is absent, returns nothing, or raises. Wire via `Agent.new(memory: { entity_extractor: ... })`.
18
+ - Bounded SQLite recall: `Memory::SqliteStore.new(max_candidates:)` (default 1000) caps how many most-recent rows a search cosines, keeping recall cost constant as total memory grows instead of brute-forcing every row. `nil` considers all rows.
19
+
20
+ ## [0.6.0] - 2026-07-06
21
+
22
+ Replaces the placeholder agent memory with a **cognitive memory** system —
23
+ semantic recall (embeddings + cosine), optional SQLite persistence, and four
24
+ memory types. This is the one area where CrewAI advanced past what the gem
25
+ originally ported. Backward compatible: the `Memory` public API is unchanged and
26
+ the zero-config default behaves as before.
27
+
28
+ ### Added
29
+ - Cognitive memory: `RCrewAI::Memory` is now a semantic, optionally-persistent, multi-type memory system, replacing the previous word-overlap placeholder. It composes four memory types — `ShortTermMemory` (recent executions, capped), `LongTermMemory` (durable, deduped insights), `EntityMemory` (facts about entities seen in work), and `ToolMemory` (tool-call history) — behind the original `Memory` public API.
30
+ - Semantic recall: pass `Agent.new(memory: { embedder: RCrewAI::Knowledge::Embedder.new })` to recall conceptually related past work via embeddings + cosine similarity. Without an embedder, recall falls back to lexical (word-overlap) similarity — and embedding failures fall back gracefully, so memory never breaks execution.
31
+ - Persistent memory: `RCrewAI::Memory::SqliteStore` persists memories to SQLite (vectors packed as floats, metadata as JSON) so recall survives restarts. Default store is `InMemoryStore` (volatile). The store interface is pluggable.
32
+ - Memory scoping: each agent's memory is scoped to its name by default, so agents sharing a persistent store don't cross-read; override via `memory: { scope: ... }`.
33
+ - `RCrewAI::Similarity` — shared cosine + lexical similarity helpers (extracted from `Knowledge::Store`, now used by both Knowledge and Memory).
34
+ - `sqlite3` added as a runtime dependency (required lazily; only needed for `SqliteStore`).
35
+
36
+ ### Changed
37
+ - `Agent.new(memory:)` accepts a pre-built `Memory`, an options hash (`{ embedder:, store:, scope:, short_term_limit: }`), or nothing (zero-config default, unchanged behavior).
38
+
10
39
  ## [0.5.0] - 2026-07-03
11
40
 
12
41
  Polish release completing the roadmap backlog: crew lifecycle hooks, batch
@@ -179,7 +208,9 @@ output, guardrails, planning, and training/testing. See `ROADMAP.md`.
179
208
  - CLI usage documentation
180
209
  - Real-world use cases and examples
181
210
 
182
- [Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.5.0...HEAD
211
+ [Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.6.1...HEAD
212
+ [0.6.1]: https://github.com/gkosmo/rcrewAI/compare/v0.6.0...v0.6.1
213
+ [0.6.0]: https://github.com/gkosmo/rcrewAI/compare/v0.5.0...v0.6.0
183
214
  [0.5.0]: https://github.com/gkosmo/rcrewAI/compare/v0.4.0...v0.5.0
184
215
  [0.4.0]: https://github.com/gkosmo/rcrewAI/compare/v0.3.0...v0.4.0
185
216
  [0.3.0]: https://github.com/gkosmo/rcrewAI/compare/v0.1.0...v0.3.0
data/README.md CHANGED
@@ -13,7 +13,7 @@ RCrewAI is a Ruby implementation of the CrewAI framework, allowing you to create
13
13
  - **🤖 Intelligent Agents**: AI agents with reasoning loops, memory, and tool usage capabilities
14
14
  - **🔗 Multi-LLM Support**: OpenAI, Anthropic (Claude), Google (Gemini), Azure OpenAI, and Ollama
15
15
  - **🛠️ Rich Tool Ecosystem**: Web search, file operations, SQL, email, code execution, PDF processing, and custom tools
16
- - **🧠 Agent Memory**: Short-term and long-term memory for learning from past executions
16
+ - **🧠 Cognitive Memory**: Semantic recall (embeddings + cosine) with optional SQLite persistence and short-term/long-term/entity/tool memory types
17
17
  - **🤝 Human-in-the-Loop**: Interactive approval workflows, human guidance, and collaborative decision making
18
18
  - **⚡ Advanced Task System**: Dependencies, retries, async/concurrent execution, and context sharing
19
19
  - **🏗️ Hierarchical Teams**: Manager agents that coordinate and delegate tasks to specialist agents
@@ -370,9 +370,35 @@ support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
370
370
  crew = RCrewAI::Crew.new('support_crew', knowledge: kb)
371
371
  ```
372
372
 
373
- Embeddings default to OpenAI's `text-embedding-3-small`; pass a custom
374
- `embedder:` (anything responding to `embed(texts)`) or vector store to swap the
375
- backend.
373
+ Embeddings default to OpenAI's `text-embedding-3-small`. Use another provider
374
+ with `RCrewAI::Knowledge::Embedder.new(provider: :ollama)` (also `:azure`,
375
+ `:google`; `:anthropic` has no embeddings API), or pass any custom `embedder:`
376
+ (anything responding to `embed(texts)`) / vector store to swap the backend.
377
+
378
+ ## 🧠 Cognitive Memory
379
+
380
+ Agents remember what they've done and recall it semantically on future tasks.
381
+ Memory is zero-config by default (in-memory, lexical recall); add an embedder
382
+ for semantic recall and a SQLite store for persistence:
383
+
384
+ ```ruby
385
+ embedder = RCrewAI::Knowledge::Embedder.new
386
+ store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
387
+
388
+ agent = RCrewAI::Agent.new(
389
+ name: 'engineer', role: '...', goal: '...',
390
+ memory: { embedder: embedder, store: store } # both optional
391
+ )
392
+ ```
393
+
394
+ - **Semantic recall** — with an embedder, an agent recalls conceptually related
395
+ past work even when the wording differs (falls back to word-overlap without one).
396
+ - **Persistence** — a `SqliteStore` makes memory survive restarts.
397
+ - **Memory types** — `agent.memory.short_term` / `long_term` / `entity` / `tool`.
398
+ - **Scoping** — memory is scoped per agent so a shared store doesn't cross-read.
399
+
400
+ Memory is best-effort: embedding failures fall back to lexical similarity, so it
401
+ never breaks agent execution.
376
402
 
377
403
  ## 🌊 Flows
378
404
 
data/ROADMAP.md CHANGED
@@ -19,8 +19,7 @@ Since CrewAI's `1.0`, the framework grew a second pillar (**Flows**) plus
19
19
  **Training/Testing**. RCrewAI now implements all of these — see the matrix below.
20
20
 
21
21
  **Status: complete.** All milestone issues (#5–#12) shipped in v0.4.0, and all
22
- backlog items (#15–#20) are done and merged to `main` (awaiting the next
23
- release). There is no outstanding roadmap work.
22
+ backlog items (#15–#20) shipped in v0.5.0. There is no outstanding roadmap work.
24
23
 
25
24
  ## Parity matrix
26
25
 
@@ -0,0 +1,118 @@
1
+ # Cognitive Memory: Semantic, Persistent, Multi-Type Agent Memory
2
+
3
+ **Date:** 2026-07-06
4
+ **Status:** Approved design (pending implementation)
5
+ **Target version:** `rcrewai` 0.6.0 (current is 0.5.0)
6
+ **Scope:** Replace the placeholder `RCrewAI::Memory` with a cognitive memory system modeled on CrewAI's memory taxonomy — short-term (semantic recent recall), long-term (persistent insights), entity memory, and tool-usage memory — backed by embeddings and a persistent SQLite vector store. No new external dependencies; reuses the in-repo `Knowledge::Embedder` and cosine-search infrastructure.
7
+
8
+ ---
9
+
10
+ ## Motivation
11
+
12
+ The current `RCrewAI::Memory` is a placeholder. It matches "relevant" past executions with word-overlap similarity (`common_words / total_words`) plus a hardcoded keyword/stopword list and a `classify_task_type` that greps the description for words like "research" or "write". Nothing is embedded; nothing is persisted; everything is lost when the process exits. It is the weakest module in the codebase and it undercuts the framework's core promise — agents that learn from what they've done.
13
+
14
+ Meanwhile CrewAI has advanced *past* the version this project ported: its 2026 "cognitive memory" work adds semantic recall, distinct memory types (short-term, long-term, entity), and persistence. This is genuine parity work, not gold-plating — and the infrastructure to do it well already exists in-repo (`Knowledge::Embedder`, cosine `Knowledge::Store`), so we compose rather than build from scratch.
15
+
16
+ ## Goals
17
+
18
+ 1. **Semantic recall.** Replace word-overlap with embedding-based similarity so an agent recalls conceptually related past work, not just keyword-overlapping work.
19
+ 2. **Persistence.** Memory survives process restarts via a SQLite-backed vector store (no external service).
20
+ 3. **Memory types.** Model CrewAI's taxonomy: short-term, long-term, entity, and tool-usage memory, each with clear write/read semantics.
21
+ 4. **Backward compatibility.** The existing `Memory` public API (`add_execution`, `add_tool_usage`, `relevant_executions`, `tool_usage_for`, `clear_*!`, `stats`) keeps working, so `Agent` and the runners need no changes to function.
22
+ 5. **No new hard dependencies.** SQLite via Ruby's stdlib-adjacent `sqlite3` gem (already common in Ruby projects) — but the store is pluggable, and the default remains in-memory so the gem works with zero setup.
23
+
24
+ ## Non-goals
25
+
26
+ - External vector databases (Chroma/Qdrant/pgvector). The pluggable store interface leaves room for them later; we ship SQLite + in-memory.
27
+ - Reranking, query rewriting, or summarization of memories (CrewAI has some of this; it's a follow-up).
28
+ - Multi-tenant / shared cross-agent memory servers.
29
+
30
+ ## Decisions captured during brainstorming
31
+
32
+ | # | Decision | Choice |
33
+ |---|----------|--------|
34
+ | 1 | Similarity mechanism | Embeddings + cosine, reusing `Knowledge::Embedder` and the cosine math already in `Knowledge::Store`. |
35
+ | 2 | Persistence | Pluggable vector store; ship `InMemoryStore` (default) and `SqliteStore`. |
36
+ | 3 | Memory taxonomy | Four types: `ShortTermMemory`, `LongTermMemory`, `EntityMemory`, `ToolMemory`. |
37
+ | 4 | Backward compat | Keep the `Memory` facade and its current method signatures; delegate internally to the new types. |
38
+ | 5 | Default behavior | Zero-config: in-memory store, embeddings only if an embedder is available/keyed; graceful fallback to the current lexical similarity when no embedder. |
39
+ | 6 | Embedding failures | Never fatal. If embedding fails (no key, network), fall back to lexical similarity so agents still run. |
40
+ | 7 | Namespacing | Memories are scoped by agent (role/name) so one agent's memory doesn't leak into another, matching CrewAI's per-role collections. |
41
+
42
+ ## Architecture
43
+
44
+ ```
45
+ RCrewAI::Memory (facade — preserves today's public API)
46
+ ├── ShortTermMemory recent executions, semantic recall, capped, volatile-by-default
47
+ ├── LongTermMemory durable insights from successful executions, persistent
48
+ ├── EntityMemory facts about entities (people, systems, concepts) extracted from work
49
+ └── ToolMemory tool-call history + outcomes (replaces @tool_usage)
50
+
51
+ Each memory type holds:
52
+ - a VectorStore (InMemoryStore | SqliteStore) ← persistence + search
53
+ - an Embedder (Knowledge::Embedder | nil) ← semantic vectors
54
+ - a lexical fallback (current word-overlap) ← when no embedder
55
+ ```
56
+
57
+ ### Storage: `Memory::VectorStore`
58
+
59
+ A small interface (mirrors `Knowledge::Store` so they can converge later):
60
+
61
+ ```ruby
62
+ store.add(id:, text:, vector:, metadata:) # upsert a record
63
+ store.search(vector, k:, scope:) # cosine top-k, filtered by scope
64
+ store.all(scope:) # enumerate (for stats / lexical fallback)
65
+ store.delete(scope:) # clear a scope
66
+ ```
67
+
68
+ - **`InMemoryStore`** — array of records; cosine in Ruby (lift the existing private cosine out of `Knowledge::Store` into a shared `Similarity` helper). Default.
69
+ - **`SqliteStore`** — one row per memory: `id, scope, type, text, vector (blob), metadata (json), created_at`. Vectors stored as packed floats (`Array#pack('e*')`). Cosine computed in Ruby over candidate rows filtered by `scope`/`type` (fine for the thousands-of-memories scale agents produce; a real ANN index is a later optimization). Opened at a configurable path (default `~/.rcrewai/memory.db` or `:memory:`).
70
+
71
+ ### Embeddings
72
+
73
+ Reuse `Knowledge::Embedder`. The memory system takes an optional `embedder:`; when present, `add_*` embeds the text and stores the vector, and recall embeds the query. When absent (no key, or explicitly disabled), the store keeps `vector: nil` and recall falls back to the current lexical similarity over `store.all`. This keeps the zero-config path working.
74
+
75
+ ### The four memory types
76
+
77
+ - **ShortTermMemory** — every execution is written here; recall returns the top-k semantically similar recent items. Capped (default 100) and, by default, uses the in-memory store (volatile). This is the direct upgrade of today's `@short_term` + `relevant_executions`.
78
+ - **LongTermMemory** — successful executions promote a distilled "insight" record (task type, what worked, result summary) into a persistent store, deduped by similarity so we don't store near-identical insights repeatedly. Upgrade of today's `@long_term`.
79
+ - **EntityMemory** — optional lightweight entity extraction (heuristic first: capitalized noun phrases / quoted terms; pluggable LLM extractor later) so agents accumulate facts about recurring entities. New capability.
80
+ - **ToolMemory** — replaces `@tool_usage`; same API (`add_tool_usage`, `tool_usage_for`) but persistable and searchable.
81
+
82
+ ### Facade: `RCrewAI::Memory`
83
+
84
+ Keeps today's signatures, delegating to the types:
85
+
86
+ ```ruby
87
+ Memory.new(embedder: nil, store: nil, scope: nil, short_term_limit: 100)
88
+
89
+ add_execution(task, result, execution_time) # -> ShortTerm + (if success) LongTerm + Entity
90
+ add_tool_usage(tool_name, params, result) # -> ToolMemory
91
+ relevant_executions(task, limit = 3) # -> semantic recall across Short+Long, formatted string|nil
92
+ tool_usage_for(tool_name, limit = 5) # -> ToolMemory
93
+ clear_short_term! / clear_all! / stats # -> delegate
94
+ ```
95
+
96
+ `Agent#initialize` gains optional `memory:` config (embedder/store/persistence) but defaults to the zero-config in-memory, lexical-fallback behavior — so existing code is unchanged.
97
+
98
+ ## Backward compatibility & migration
99
+
100
+ - The `Memory` public API is preserved exactly; `agent.rb` and the runners are untouched functionally.
101
+ - Default construction (`Memory.new` with no args) behaves like today from the caller's perspective — just with better recall when an embedder is configured, and identical lexical recall when not.
102
+ - `relevant_executions` still returns the same formatted-string-or-nil shape consumed at `agent.rb:241`.
103
+
104
+ ## Testing strategy
105
+
106
+ - **Unit**: each memory type with a fake deterministic embedder (as `knowledge_spec.rb` already does) — assert semantic recall ranks the conceptually-closest item first, not the keyword-overlapping one.
107
+ - **Persistence**: `SqliteStore` round-trip — write memories, reopen the DB, recall survives.
108
+ - **Fallback**: with no embedder, recall still returns results via lexical similarity (proves the graceful path).
109
+ - **Facade compat**: the existing `memory_spec.rb` expectations continue to pass (or are updated only where the placeholder behavior was clearly a bug).
110
+ - **Agent integration**: an agent with `reasoning`/knowledge off recalls a semantically-related prior execution injected into its context.
111
+
112
+ ## Dependencies
113
+
114
+ - `sqlite3` gem — added as a runtime dependency, but only required lazily inside `SqliteStore` so the gem loads and the in-memory path works even if it's absent. (If we'd rather avoid the dependency entirely, the fallback is a JSON-file-backed store; SQLite is preferred for query/scale.)
115
+
116
+ ## Rollout
117
+
118
+ Ship in `0.6.0`. `docs/upgrading-to-0.6.md` documents opt-in persistence and embeddings; the default path is unchanged, so it's a no-action upgrade for existing users.
@@ -0,0 +1,101 @@
1
+ # Upgrading to RCrewAI 0.6
2
+
3
+ RCrewAI 0.6 replaces the placeholder memory with a **cognitive memory** system:
4
+ semantic recall (embeddings + cosine), optional SQLite persistence, and four
5
+ memory types (short-term, long-term, entity, tool).
6
+
7
+ **0.6 is backward compatible.** The `RCrewAI::Memory` public API is unchanged,
8
+ and the default (`Memory.new` with no config) behaves like before from the
9
+ caller's perspective — just with better recall once an embedder is configured.
10
+ Everything below is opt-in.
11
+
12
+ ---
13
+
14
+ ## 1. What you must do
15
+
16
+ Nothing. Upgrade the gem; existing code keeps working:
17
+
18
+ ```ruby
19
+ gem 'rcrewai', '~> 0.6'
20
+ ```
21
+
22
+ Note: `0.6` adds `sqlite3` as a runtime dependency (used only if you opt into
23
+ persistence). It is required lazily, so the in-memory default works even if the
24
+ native extension isn't present.
25
+
26
+ ---
27
+
28
+ ## 2. What you can do (new capabilities)
29
+
30
+ ### 2a. Semantic recall via embeddings
31
+
32
+ By default, memory recall uses lexical (word-overlap) similarity — same as
33
+ before. Pass an embedder to get semantic recall, so an agent remembers
34
+ conceptually related work even when the wording differs:
35
+
36
+ ```ruby
37
+ embedder = RCrewAI::Knowledge::Embedder.new # OpenAI text-embedding-3-small
38
+ agent = RCrewAI::Agent.new(
39
+ name: 'engineer', role: '...', goal: '...',
40
+ memory: { embedder: embedder }
41
+ )
42
+ ```
43
+
44
+ If embedding fails (no API key, network error), recall automatically falls back
45
+ to lexical similarity — memory never breaks agent execution.
46
+
47
+ ### 2b. Persistent memory (SQLite)
48
+
49
+ Give memory a SQLite store and it survives restarts:
50
+
51
+ ```ruby
52
+ store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
53
+ agent = RCrewAI::Agent.new(
54
+ name: 'engineer', role: '...', goal: '...',
55
+ memory: { embedder: embedder, store: store }
56
+ )
57
+ ```
58
+
59
+ The default store is in-memory (volatile). Any object implementing the store
60
+ interface (`add` / `search` / `all` / `delete` / `delete_record`) can be used —
61
+ e.g. a future Chroma/pgvector adapter.
62
+
63
+ ### 2c. Memory scoping
64
+
65
+ Each agent's memory is scoped to its name by default, so agents sharing a
66
+ persistent store don't read each other's memories. Override with
67
+ `memory: { scope: 'custom' }` if you want deliberate sharing.
68
+
69
+ ### 2d. Memory types (direct access)
70
+
71
+ The `Memory` facade exposes the underlying types for advanced use:
72
+
73
+ ```ruby
74
+ agent.memory.short_term # recent executions, semantic recall, capped
75
+ agent.memory.long_term # durable insights from successful executions, deduped
76
+ agent.memory.entity # facts about entities (people, systems) seen in work
77
+ agent.memory.tool # tool-call history + outcomes
78
+
79
+ agent.memory.entity.entities # => ["Alice", "AWS", ...]
80
+ agent.memory.long_term.recall('...', limit: 3)
81
+ ```
82
+
83
+ ### 2e. Pre-built Memory
84
+
85
+ You can also construct and pass a `Memory` directly:
86
+
87
+ ```ruby
88
+ memory = RCrewAI::Memory.new(scope: 'shared', embedder: embedder, store: store)
89
+ agent = RCrewAI::Agent.new(name: 'a', role: '...', goal: '...', memory: memory)
90
+ ```
91
+
92
+ ---
93
+
94
+ ## Behavior notes
95
+
96
+ - `add_execution` writes to short-term always, and (on success) promotes a
97
+ deduped insight to long-term and extracts entities.
98
+ - `relevant_executions(task, limit)` recalls across short- and long-term and
99
+ returns the same formatted-string-or-nil shape as before.
100
+ - `clear_short_term!` leaves long-term insights intact; `clear_all!` wipes
101
+ everything.
@@ -0,0 +1,66 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # Cognitive Memory — semantic recall + persistence.
5
+ #
6
+ # Agents remember past executions and recall them by *meaning*, not just shared
7
+ # words. With a SQLite store, memory survives restarts.
8
+ #
9
+ # This example uses a fake, deterministic embedder so it runs WITHOUT an API
10
+ # key. In real use you'd pass `RCrewAI::Knowledge::Embedder.new` (OpenAI).
11
+ #
12
+ # Run:
13
+ # ruby examples/cognitive_memory_example.rb
14
+
15
+ require_relative '../lib/rcrewai'
16
+ require 'tmpdir'
17
+
18
+ RCrewAI.configure(validate: false) do |c|
19
+ c.llm_provider = :openai
20
+ c.api_key = 'demo-key'
21
+ end
22
+
23
+ # Fake embedder: maps text to a concept vector by keyword. Any object
24
+ # responding to embed(texts) -> [[float, ...], ...] works.
25
+ class ConceptEmbedder
26
+ def embed(texts)
27
+ texts.map do |t|
28
+ l = t.downcase
29
+ [
30
+ l.match?(/payment|billing|invoice|charge/) ? 1.0 : 0.0,
31
+ l.match?(/auth|login|session|token/) ? 1.0 : 0.0,
32
+ l.match?(/deploy|release|ci|pipeline/) ? 1.0 : 0.0
33
+ ]
34
+ end
35
+ end
36
+ end
37
+
38
+ Task = Struct.new(:name, :description)
39
+
40
+ Dir.mktmpdir do |dir|
41
+ store = RCrewAI::Memory::SqliteStore.new(path: File.join(dir, 'memory.db'))
42
+ agent = RCrewAI::Agent.new(
43
+ name: 'engineer', role: 'Senior engineer', goal: 'Ship reliable software',
44
+ memory: { embedder: ConceptEmbedder.new, store: store }
45
+ )
46
+
47
+ # Record a few past executions.
48
+ agent.memory.add_execution(Task.new('t1', 'fixed the billing invoice bug'),
49
+ 'adjusted the payment retry logic', 1.2)
50
+ agent.memory.add_execution(Task.new('t2', 'patched the login session flow'),
51
+ 'rotated the auth tokens', 0.8)
52
+ agent.memory.add_execution(Task.new('t3', 'sped up the release pipeline'),
53
+ 'parallelized the CI stages', 2.0)
54
+
55
+ puts '== Semantic recall =='
56
+ # Query shares NO words with the billing execution, but is conceptually close.
57
+ query = Task.new('q', 'a customer got double-charged on checkout')
58
+ puts agent.memory.relevant_executions(query, 1)
59
+
60
+ puts "\n== Persistence (reopen the DB in a fresh agent) =="
61
+ store2 = RCrewAI::Memory::SqliteStore.new(path: File.join(dir, 'memory.db'))
62
+ agent2 = RCrewAI::Agent.new(name: 'engineer', role: 'r', goal: 'g',
63
+ memory: { embedder: ConceptEmbedder.new, store: store2 })
64
+ puts agent2.memory.relevant_executions(query, 1)
65
+ puts "stats: #{agent2.memory.stats.inspect}"
66
+ end
data/lib/rcrewai/agent.rb CHANGED
@@ -39,7 +39,7 @@ module RCrewAI
39
39
  @reasoning = options.fetch(:reasoning, false)
40
40
  @max_reasoning_attempts = options.fetch(:max_reasoning_attempts, 3)
41
41
  @respect_context_window = options.fetch(:respect_context_window, false)
42
- @memory = Memory.new
42
+ @memory = build_memory(options[:memory])
43
43
  @rate_limiter = options[:max_rpm] ? RateLimiter.new(max_rpm: options[:max_rpm]) : nil
44
44
  @llm_client = wrap_with_rate_limiter(build_llm_client(options[:llm]))
45
45
  @knowledge = build_knowledge(options[:knowledge], options[:knowledge_sources])
@@ -220,6 +220,18 @@ module RCrewAI
220
220
  RateLimiter::ThrottledClient.new(client, @rate_limiter)
221
221
  end
222
222
 
223
+ # Builds the agent's memory. Accepts a pre-built Memory, an options hash
224
+ # (`{ embedder:, store:, scope:, short_term_limit: }`), or nil for the
225
+ # zero-config default. Memory is scoped to the agent's name so agents don't
226
+ # share recall (matters when a persistent store is shared).
227
+ def build_memory(memory)
228
+ return memory if memory.is_a?(Memory)
229
+
230
+ opts = memory.is_a?(Hash) ? memory : {}
231
+ Memory.new(scope: opts.fetch(:scope, name),
232
+ **opts.slice(:embedder, :store, :short_term_limit, :entity_extractor))
233
+ end
234
+
223
235
  # Accepts a pre-built Knowledge::Base via +knowledge:+ or an array of
224
236
  # sources via +knowledge_sources:+ (wrapped in a Base). Returns nil if
225
237
  # neither is given.
@@ -5,36 +5,100 @@ require 'faraday'
5
5
 
6
6
  module RCrewAI
7
7
  module Knowledge
8
- # Turns text into embedding vectors. Defaults to OpenAI's embeddings API;
9
- # #embed takes an array of strings and returns an array of vectors. Any
10
- # object responding to #embed can be substituted (see specs).
8
+ # Turns text into embedding vectors. Supports multiple providers
9
+ # (OpenAI [default], Azure OpenAI, Google Gemini, Ollama); Anthropic has no
10
+ # first-party embeddings API and raises a clear error. `#embed` takes an
11
+ # array of strings and returns an array of vectors. Any object responding to
12
+ # `#embed` can be substituted.
11
13
  class Embedder
12
- DEFAULT_MODEL = 'text-embedding-3-small'
14
+ DEFAULT_MODELS = {
15
+ openai: 'text-embedding-3-small',
16
+ azure: 'text-embedding-3-small',
17
+ google: 'text-embedding-004',
18
+ ollama: 'nomic-embed-text'
19
+ }.freeze
20
+
13
21
  OPENAI_URL = 'https://api.openai.com/v1/embeddings'
22
+ GOOGLE_BASE = 'https://generativelanguage.googleapis.com/v1beta'
23
+ OLLAMA_DEFAULT_URL = 'http://localhost:11434'
24
+
25
+ attr_reader :provider, :model
26
+
27
+ def initialize(provider: :openai, model: nil, api_key: nil, config: RCrewAI.configuration)
28
+ @provider = provider.to_sym
29
+ if @provider == :anthropic
30
+ raise EmbeddingError,
31
+ 'anthropic does not provide an embeddings API; use :openai, :azure, :google, or :ollama'
32
+ end
14
33
 
15
- def initialize(model: DEFAULT_MODEL, api_key: nil, config: RCrewAI.configuration)
16
- @model = model
17
- @api_key = api_key || config.openai_api_key || config.api_key
34
+ @config = config
35
+ @model = model || DEFAULT_MODELS[@provider] || DEFAULT_MODELS[:openai]
36
+ @api_key = api_key
18
37
  end
19
38
 
20
39
  def embed(texts)
21
40
  texts = Array(texts)
22
41
  return [] if texts.empty?
23
42
 
24
- response = connection.post(OPENAI_URL) do |req|
25
- req.headers['Authorization'] = "Bearer #{@api_key}"
26
- req.headers['Content-Type'] = 'application/json'
27
- req.body = JSON.generate(model: @model, input: texts)
43
+ send("embed_#{@provider}", texts)
44
+ end
45
+
46
+ private
47
+
48
+ def embed_openai(texts)
49
+ body = post_json(OPENAI_URL, { model: @model, input: texts },
50
+ 'Authorization' => "Bearer #{api_key_for(:openai)}")
51
+ body['data'].map { |d| d['embedding'] }
52
+ end
53
+
54
+ def embed_azure(texts)
55
+ base = @config.base_url
56
+ version = @config.api_version || '2024-02-01'
57
+ deployment = @config.deployment_name || @model
58
+ url = "#{base}/openai/deployments/#{deployment}/embeddings?api-version=#{version}"
59
+ body = post_json(url, { input: texts }, 'api-key' => api_key_for(:azure))
60
+ body['data'].map { |d| d['embedding'] }
61
+ end
62
+
63
+ def embed_google(texts)
64
+ key = api_key_for(:google)
65
+ texts.map do |text|
66
+ url = "#{GOOGLE_BASE}/models/#{@model}:embedContent?key=#{key}"
67
+ payload = { model: "models/#{@model}", content: { parts: [{ text: text }] } }
68
+ body = post_json(url, payload)
69
+ body.dig('embedding', 'values')
28
70
  end
71
+ end
29
72
 
73
+ def embed_ollama(texts)
74
+ base = @config.base_url || OLLAMA_DEFAULT_URL
75
+ texts.map do |text|
76
+ body = post_json("#{base}/api/embeddings", { model: @model, prompt: text })
77
+ body['embedding']
78
+ end
79
+ end
80
+
81
+ def post_json(url, payload, headers = {})
82
+ response = connection.post(url) do |req|
83
+ req.headers['Content-Type'] = 'application/json'
84
+ headers.each { |k, v| req.headers[k] = v }
85
+ req.body = JSON.generate(payload)
86
+ end
30
87
  raise EmbeddingError, "embedding request failed: #{response.status}" unless response.success?
31
88
 
32
89
  body = response.body
33
- body = JSON.parse(body) if body.is_a?(String)
34
- body['data'].map { |d| d['embedding'] }
90
+ body.is_a?(String) ? JSON.parse(body) : body
35
91
  end
36
92
 
37
- private
93
+ def api_key_for(provider)
94
+ return @api_key if @api_key
95
+
96
+ case provider
97
+ when :openai then @config.openai_api_key || @config.api_key
98
+ when :azure then @config.azure_api_key || @config.api_key
99
+ when :google then @config.google_api_key || @config.api_key
100
+ end
101
+ end
38
102
 
39
103
  def connection
40
104
  @connection ||= Faraday.new do |f|
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative '../similarity'
4
+
3
5
  module RCrewAI
4
6
  module Knowledge
5
7
  # In-memory vector store with cosine-similarity search. The default backing
@@ -38,20 +40,8 @@ module RCrewAI
38
40
 
39
41
  private
40
42
 
41
- def cosine_similarity(a, b)
42
- dot = 0.0
43
- norm_a = 0.0
44
- norm_b = 0.0
45
- a.each_index do |i|
46
- ai = a[i].to_f
47
- bi = (b[i] || 0).to_f
48
- dot += ai * bi
49
- norm_a += ai * ai
50
- norm_b += bi * bi
51
- end
52
- return 0.0 if norm_a.zero? || norm_b.zero?
53
-
54
- dot / (Math.sqrt(norm_a) * Math.sqrt(norm_b))
43
+ def cosine_similarity(vec_a, vec_b)
44
+ Similarity.cosine(vec_a, vec_b)
55
45
  end
56
46
  end
57
47
  end