engram 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c623f8198a9f71905e6e0c904b6cf60d5d6a0734b1e8fdaf877611a815f2a919
4
- data.tar.gz: aca21e3da50239a5d02822306524adc6647883ef607af57747b05ce7ecb78827
3
+ metadata.gz: f2d96222b1c43e68da7da579a6b9d1d39f85f5cbe2480be6cd39a19dcf36adc6
4
+ data.tar.gz: 53bc4261b347e498d20ea4171164b9478c55134067c33715641017b7572c78a7
5
5
  SHA512:
6
- metadata.gz: 55f5e8335a86ec28cd28b17af162883fcc7c58dc56b903fc18b924c4278b593d9c8945cbf19cb28649a4c66de57ff38e6af46f3c50b15a6a96a8d729b90fc6ba
7
- data.tar.gz: 519081acdc6f1b284df51bc664d274ca31befbc2d7a002ef9d62fee2f31c165f64c2896ab5f2dddca771b02c2052dcb630ef785e440776931d53d7f994de6e47
6
+ metadata.gz: a8001a49d7c280ded764b75e2f62695f3eeaf229e114e5455ee9bfa32f58d0c2a94fd9d70a1b563e246da0efcb570fe5f95ca6ecd61e69ca3cad758a0491abae
7
+ data.tar.gz: 29d1e339241f3edb54485925f20b78f81740770fc966c8e838cefaef839e191597080249f578f33c8f79c5eb2cc89563b0187cf897b296ab660ff4510a3b0d4d
data/CHANGELOG.md CHANGED
@@ -5,6 +5,31 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.7.0] - 2026-09-13
9
+
10
+ ### Added
11
+ - `Memory#forget(id:)` deletes one memory in the current scope, returning `1` when deleted
12
+ or `0` when missing. Explicit deletion bypasses persistence hooks and emits `forget.engram`
13
+ with the deleted count. Arrays, ranges, nil, and blank IDs are rejected.
14
+ - Optional `min_similarity:` on recall, injection, and `Engram.with_memory`, with
15
+ `config.recall_min_similarity` as the facade default. Filters cosine similarity before
16
+ importance/recency ranking and touching; nil preserves existing top-k behavior.
17
+ - Optional `max_bytes:` on injection and `Engram.with_memory`, with
18
+ `config.injection_max_bytes` as the default. Includes the complete appended memory block,
19
+ skips whole memories that do not fit, and preserves the original prompt.
20
+ - Metrics for filtered candidates, injected/skipped memories, and appended bytes.
21
+ - `MIN_SIMILARITY` eval control and negative-query false-positive/abstention measurements.
22
+
23
+ ### Fixed
24
+ - Load `ObserveJob` on first use so `observe_later` works before another job has loaded
25
+ `ActiveJob::Base` in a Rails application.
26
+ - Forward RubyLLM streaming blocks and retain the memory wrapper when fluent chat methods
27
+ return the underlying chat, so chained configuration does not bypass memory injection.
28
+
29
+ ### Changed
30
+ - Recall rejects negative, fractional, nil, and string limits with `ArgumentError`; use an
31
+ integer. A zero limit skips embedding and search. A zero injection byte budget skips recall.
32
+
8
33
  ## [0.6.0] - 2026-07-24
9
34
 
10
35
  ### Added
data/README.md CHANGED
@@ -58,6 +58,8 @@ chat.ask("Why am I being rate limited?")
58
58
  - RubyLLM embedder and completion adapters for provider-backed embeddings and extraction.
59
59
  - Canonical memory kinds: `fact`, `preference`, `instruction`, and `episodic`.
60
60
  - Typed recall filters and typed, escaped memory injection.
61
+ - Optional cosine relevance thresholds and byte budgets for injected context.
62
+ - RubyLLM streaming and fluent chat configuration through the memory wrapper.
61
63
  - Persistence policy that rejects obvious secrets and transient task-progress updates before storage.
62
64
  - Idempotent observation, recency/importance-aware ranking, recall touching, and stale-memory pruning.
63
65
 
@@ -74,7 +76,7 @@ explicitly version-gated:
74
76
  - Core types and facade: `Engram::Memory`, `Engram::Record`, `Engram::Decision`, `Engram::PersistencePolicy`, and `Engram.with_memory`.
75
77
  - Store and adapter ports: `Engram::Ports::MemoryStore`, `Engram::Ports::Embedder`, `Engram::Ports::Completion`.
76
78
  - Rails integration points: `has_memory`, `Memory#observe_later`, and generator outputs under `engram:` rake tasks.
77
- - Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget_stale`, `rebuild_embeddings`, and `memories_from_source`.
79
+ - Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_stale`, `rebuild_embeddings`, and `memories_from_source`.
78
80
  - RubyLLM adapter contract points and evaluator entrypoints (`rake eval`, `rake eval:real`).
79
81
 
80
82
  ### Backward-compatibility commitments (pre-1.0)
@@ -223,6 +225,25 @@ class AddEngramMemoryEmbeddingIndex < ActiveRecord::Migration[8.0]
223
225
  end
224
226
  ```
225
227
 
228
+ ### Filtered vector search
229
+
230
+ Keep the generated B-tree index on `scope`. With an approximate vector index, pgvector
231
+ applies scope and kind filters after the index scan. This can return fewer than `limit`
232
+ memories even when enough matching rows exist.
233
+
234
+ On pgvector 0.8.0+, HNSW iterative scans can search further:
235
+
236
+ ```ruby
237
+ Engram::MemoryRecord.transaction do
238
+ Engram::MemoryRecord.connection.execute("SET LOCAL hnsw.iterative_scan = strict_order")
239
+ current_user.memory.recall("billing preferences", limit: 5)
240
+ end
241
+ ```
242
+
243
+ The setting lasts for the transaction. Iterative scans add latency and still have scan
244
+ limits. Compare them with exact search on your scoped data before enabling them.
245
+ See [pgvector's filtering docs](https://github.com/pgvector/pgvector#filtering).
246
+
226
247
  ## Model/provider configuration
227
248
 
228
249
  Engram is model-provider agnostic. The core only depends on two ports:
@@ -272,6 +293,25 @@ chat.ask("why am I being rate limited?")
272
293
  # recall + inject happen automatically before the model sees the message
273
294
  ```
274
295
 
296
+ Streaming and chained chat configuration work through the wrapper:
297
+
298
+ ```ruby
299
+ chat = Engram.with_memory(
300
+ RubyLLM.chat,
301
+ memory: current_user.memory,
302
+ kinds: [:fact, :preference],
303
+ min_similarity: 0.5, # tune for your embedding model
304
+ max_bytes: 8_000
305
+ )
306
+
307
+ response = chat.with_instructions("Keep answers concise.").ask("Which plan am I on?") do |chunk|
308
+ print chunk.content
309
+ end
310
+ ```
311
+
312
+ `ask` forwards request options and the streaming block to RubyLLM and returns its final
313
+ response. `with_memory` captures memory defaults when the wrapper is created.
314
+
275
315
  ## Automatic memory
276
316
 
277
317
  Instead of adding facts by hand, let engram derive them from a conversation turn. It
@@ -348,8 +388,8 @@ Engram.configure do |config|
348
388
  end
349
389
  ```
350
390
 
351
- Write-content filtering and transformation are not deletion authorization. `forget` still
352
- validates scope, target existence, candidate integrity, and provenance, but it does not run
391
+ Write-content filtering and transformation are not deletion authorization. A consolidation
392
+ `forget` decision still validates scope, target existence, candidate integrity, and provenance, but it does not run
353
393
  `before_persist` or the policy's `call` method; this allows secret, transient, and redacted
354
394
  memories to be removed. A custom policy can additionally control destructive decisions by
355
395
  implementing `allow_destructive?(record)` and returning exactly `true` or `false`. Policies
@@ -482,6 +522,39 @@ with the old `semantic` kind value.
482
522
 
483
523
  ## Tuning and maintenance
484
524
 
525
+ ### Recall thresholds and injection limits
526
+
527
+ Recall returns the nearest matches even for unrelated queries. Set a minimum cosine
528
+ similarity to exclude weak matches and a byte limit to keep injected context small:
529
+
530
+ ```ruby
531
+ Engram.configure do |config|
532
+ config.recall_min_similarity = 0.5 # tune for your embedding model
533
+ config.injection_max_bytes = 8_000
534
+ end
535
+
536
+ memory.recall("billing preferences", min_similarity: 0.6)
537
+ memory.inject_into(prompt, query: "billing preferences", max_bytes: 4_000)
538
+ ```
539
+
540
+ `min_similarity` accepts a finite number in `[-1, 1]`. It filters candidates before
541
+ importance/recency ranking and touching, and can return no matches. Filtering happens in
542
+ Ruby; custom stores keep their existing search signature. It does not refill the candidate
543
+ pool or suppress embedding compatibility errors.
544
+
545
+ `max_bytes` limits the appended memory section, including escaped text, headers, and tags.
546
+ It excludes the original prompt and chat history and counts bytes, not tokens. Memories
547
+ are considered in recall order; those that do not fit are skipped without truncation.
548
+ If none fit, the prompt is unchanged. With `touch_on_recall`, retrieved memories are touched
549
+ even if the byte limit later excludes them.
550
+
551
+ Both settings default to nil. Pass nil explicitly to override a configured value.
552
+ `limit` and `max_bytes` require non-negative integers, except that `max_bytes: nil` removes
553
+ the cap. `limit: 0` skips embedding and search; `max_bytes: 0` skips recall during injection.
554
+ Invalid values raise `ArgumentError`.
555
+
556
+ ### Observation and maintenance
557
+
485
558
  Observation uses a scope-and-turn claim before extraction. While a claim lease is live and
486
559
  the turn has not completed, calls for the same scope and turn raise
487
560
  `Engram::ObservationInProgressError` instead of reporting success without doing the work; a
@@ -515,7 +588,24 @@ Engram.configure do |config|
515
588
  end
516
589
  ```
517
590
 
518
- Prune memories you no longer need:
591
+ Delete a specific memory:
592
+
593
+ ```ruby
594
+ record = current_user.memory.add("Prefers tea")
595
+ current_user.memory.forget(id: record.id) # => 1
596
+ current_user.memory.forget(id: record.id) # => 0 (already deleted)
597
+ ```
598
+
599
+ `forget` deletes one record within the memory's scope and returns the affected row count.
600
+ Missing records and IDs belonging to another scope return `0`. Pass the ID returned by
601
+ the store; integers and non-blank strings are accepted. Arrays, ranges, and nil raise
602
+ `ArgumentError`.
603
+
604
+ This is an explicit application operation: authorize the request before calling it.
605
+ It does not run extraction, consolidation, or persistence hooks, including `allow_destructive?`.
606
+ Source messages and processed-turn markers are kept; a later observation can recreate the memory.
607
+
608
+ Prune stale memories:
519
609
 
520
610
  ```ruby
521
611
  # Forget memories untouched for 90 days, but keep anything important
@@ -549,6 +639,7 @@ When ActiveSupport is loaded, Engram emits `ActiveSupport::Notifications` events
549
639
  main memory pipeline:
550
640
 
551
641
  - `add.engram`
642
+ - `forget.engram`
552
643
  - `recall.engram`
553
644
  - `inject.engram`
554
645
  - `observe.engram`
@@ -580,6 +671,11 @@ ActiveSupport::Notifications.subscribe(/\.engram\z/) do |name, _started, _finish
580
671
  end
581
672
  ```
582
673
 
674
+ Recall events include `candidate_count`, `filtered_count`, `result_count`, and
675
+ `min_similarity` when set. Injection events include the input `memory_count`,
676
+ `injected_count`, `skipped_count`, `injected_bytes`, and `max_bytes` when set.
677
+ `forget.engram` reports `deleted_count` without the record ID or content.
678
+
583
679
  Avoid adding memory content or raw prompts to subscriber logs; recalled content is
584
680
  user-derived and should be treated as sensitive application data.
585
681
 
@@ -592,7 +688,8 @@ user-derived and should be treated as sensitive application data.
592
688
  - Configure ActiveJob for `observe_later`; keep automatic observation off the request path.
593
689
  - Configure `Engram::Rails::CacheProcessedTurns` or another persistent processed-turns adapter for retries.
594
690
  - Review persistence policy settings and add app-specific redaction/denylist patterns.
595
- - Set recall limits and `kinds:` filters appropriate for your prompt budget and threat model.
691
+ - Calibrate `recall_min_similarity` and set `injection_max_bytes`, recall limits, and `kinds:` filters.
692
+ - Check filtered pgvector recall coverage and latency on representative tenant sizes.
596
693
  - Run the deterministic test/eval suite plus pgvector integration tests before release.
597
694
 
598
695
  ## How it works
@@ -667,9 +764,10 @@ safe to run in CI as a smoke test.
667
764
  The harness reports recall@k over labelled relevant memories, a labelled precision
668
765
  proxy@k, near-distractor retrieval rate, contradiction-pair full recall, extraction
669
766
  structured-output parsing cases, consolidation decision cases, and a heuristic duplicate-add
670
- baseline. Negative queries are printed for inspection, but top-k recall currently has no
671
- similarity threshold, so the harness does not report a hallucination rate. Treat the default
672
- NullEmbedder recall numbers as a mechanics check, not as a semantic retrieval benchmark.
767
+ baseline. Set `MIN_SIMILARITY=0.5` on either eval task to test a threshold. With a semantic
768
+ embedder, it also reports how often unrelated queries retrieve memories and how often
769
+ recall returns nothing. Compare these against positive-query recall when tuning the
770
+ threshold. NullEmbedder results only check mechanics; they do not measure semantic quality.
673
771
 
674
772
  Before opening a release PR, also verify the gem package:
675
773
 
@@ -684,7 +782,9 @@ gem unpack engram-*.gem --target /tmp/engram-package-check
684
782
  - v0.2 (done): extract and consolidate (ADD / UPDATE / FORGET), background jobs.
685
783
  - v0.3 (done): idempotent observation, importance/recency recall, forgetting and decay.
686
784
  - v0.4 (done): memory kinds, persistence policy, typed recall filters, safer injection, and observability hooks.
687
- - v0.5 (in progress): embedding provenance and scoped embedding rebuild operations.
785
+ - v0.5 (done): embedding provenance and scoped embedding rebuild operations.
786
+ - v0.6 (done): structured provenance, source impact lookup, and grounding reports.
787
+ - v0.7 (done): recall thresholds, injection limits, scoped deletion, and RubyLLM streaming fixes.
688
788
  - later: additional storage backends and larger real-provider eval benchmarks.
689
789
 
690
790
  ## License
@@ -8,13 +8,16 @@ module Engram
8
8
  attr_accessor :store, :embedder, :completion, :default_limit,
9
9
  :consolidator, :extraction_min_confidence, :processed_turns,
10
10
  :importance_weight, :recency_weight, :recency_halflife, :touch_on_recall,
11
- :persistence_policy, :before_persist, :instrumentation_scope_identifier
11
+ :persistence_policy, :before_persist, :instrumentation_scope_identifier,
12
+ :recall_min_similarity, :injection_max_bytes
12
13
 
13
14
  def initialize
14
15
  @store = Adapters::InMemoryStore.new
15
16
  @embedder = Adapters::NullEmbedder.new
16
17
  @completion = nil # required for observe (extract/consolidate); nil until configured
17
18
  @default_limit = 5
19
+ @recall_min_similarity = nil # opt-in cosine floor, before importance/recency ranking
20
+ @injection_max_bytes = nil # cap the complete appended memory block, not the host prompt
18
21
  @consolidator = :heuristic # :heuristic (deterministic) or :llm (LLM-as-judge)
19
22
  @extraction_min_confidence = 0.5
20
23
  @processed_turns = Adapters::InMemoryProcessedTurns.new # idempotency for observe
@@ -9,21 +9,27 @@ module Engram
9
9
  # chat = Engram.with_memory(RubyLLM.chat, memory: current_user.memory)
10
10
  # chat.ask("why am I rate limited?") # recall + inject happen automatically
11
11
  class MemoryChat
12
- def initialize(chat, memory:, limit: Engram.config.default_limit)
12
+ def initialize(chat, memory:, limit: Engram.config.default_limit, kinds: nil,
13
+ min_similarity: Engram.config.recall_min_similarity, max_bytes: Engram.config.injection_max_bytes)
13
14
  @chat = chat
14
15
  @memory = memory
15
16
  @limit = limit
17
+ @kinds = kinds
18
+ @min_similarity = min_similarity
19
+ @max_bytes = max_bytes
16
20
  end
17
21
 
18
- def ask(message, **opts)
19
- augmented = @memory.inject_into(message.to_s, query: message.to_s, limit: @limit)
20
- @chat.ask(augmented, **opts)
22
+ def ask(message, **opts, &block)
23
+ augmented = @memory.inject_into(message.to_s, query: message.to_s, limit: @limit,
24
+ kinds: @kinds, min_similarity: @min_similarity, max_bytes: @max_bytes)
25
+ @chat.ask(augmented, **opts, &block)
21
26
  end
22
27
 
23
28
  def method_missing(name, *args, **kwargs, &block)
24
29
  return super unless @chat.respond_to?(name)
25
30
 
26
- @chat.public_send(name, *args, **kwargs, &block)
31
+ result = @chat.public_send(name, *args, **kwargs, &block)
32
+ result.equal?(@chat) ? self : result
27
33
  end
28
34
 
29
35
  def respond_to_missing?(name, include_private = false)
@@ -34,7 +40,9 @@ module Engram
34
40
  end
35
41
 
36
42
  # Convenience entrypoint.
37
- def self.with_memory(chat, memory:, limit: config.default_limit)
38
- Integrations::RubyLLM::MemoryChat.new(chat, memory: memory, limit: limit)
43
+ def self.with_memory(chat, memory:, limit: config.default_limit, kinds: nil,
44
+ min_similarity: config.recall_min_similarity, max_bytes: config.injection_max_bytes)
45
+ Integrations::RubyLLM::MemoryChat.new(chat, memory: memory, limit: limit, kinds: kinds,
46
+ min_similarity: min_similarity, max_bytes: max_bytes)
39
47
  end
40
48
  end
data/lib/engram/memory.rb CHANGED
@@ -30,7 +30,8 @@ module Engram
30
30
  end
31
31
 
32
32
  # Return the most relevant memories for a query.
33
- def recall(query, limit: Engram.config.default_limit, kinds: nil)
33
+ def recall(query, limit: Engram.config.default_limit, kinds: nil,
34
+ min_similarity: Engram.config.recall_min_similarity)
34
35
  UseCases::Recall.new(
35
36
  store: @store,
36
37
  embedder: @embedder,
@@ -38,13 +39,15 @@ module Engram
38
39
  recency_weight: Engram.config.recency_weight,
39
40
  recency_halflife: Engram.config.recency_halflife,
40
41
  touch: Engram.config.touch_on_recall
41
- ).call(query, scope: scope, limit: limit, kinds: kinds)
42
+ ).call(query, scope: scope, limit: limit, kinds: kinds, min_similarity: min_similarity)
42
43
  end
43
44
 
44
45
  # Recall, then inject into a prompt string.
45
- def inject_into(prompt, query:, limit: Engram.config.default_limit, kinds: nil)
46
- memories = recall(query, limit: limit, kinds: kinds)
47
- UseCases::Inject.new.call(prompt: prompt, memories: memories)
46
+ def inject_into(prompt, query:, limit: Engram.config.default_limit, kinds: nil,
47
+ min_similarity: Engram.config.recall_min_similarity, max_bytes: Engram.config.injection_max_bytes)
48
+ UseCases::Inject.validate_max_bytes!(max_bytes)
49
+ memories = (max_bytes == 0) ? [] : recall(query, limit: limit, kinds: kinds, min_similarity: min_similarity)
50
+ UseCases::Inject.new.call(prompt: prompt, memories: memories, max_bytes: max_bytes)
48
51
  end
49
52
 
50
53
  # Derive memories from a conversation turn and consolidate them (v0.2).
@@ -86,6 +89,18 @@ module Engram
86
89
  @store.all(scope: scope)
87
90
  end
88
91
 
92
+ # Delete one memory in this scope. Returns 1 when deleted, or 0 when missing.
93
+ # Explicit deletion bypasses extraction, consolidation, and persistence hooks.
94
+ def forget(id:)
95
+ valid_id = id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
96
+ raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id
97
+
98
+ payload = Engram::Instrumentation.payload(scope: scope, store: @store)
99
+ Engram::Instrumentation.instrument("forget", payload) do
100
+ payload[:deleted_count] = @store.delete(scope: scope, id: id)
101
+ end
102
+ end
103
+
89
104
  # Recompute embeddings (and embedding metadata) for memories in the scope.
90
105
  # When `stale_only` is true, only records whose current metadata does not match the
91
106
  # active embedder are rebuilt.
@@ -15,9 +15,7 @@ module Engram
15
15
  end
16
16
 
17
17
  initializer "engram.active_job" do
18
- ActiveSupport.on_load(:active_job) do
19
- require "engram/rails/observe_job"
20
- end
18
+ Engram.autoload :ObserveJob, "engram/rails/observe_job" if defined?(::ActiveJob::Base)
21
19
  end
22
20
 
23
21
  rake_tasks do
@@ -12,14 +12,39 @@ module Engram
12
12
  @header = header
13
13
  end
14
14
 
15
- # Returns a new prompt string. If there are no memories, the prompt is unchanged.
16
- def call(prompt:, memories:)
17
- payload = {memory_count: memories&.size.to_i}
18
- Engram::Instrumentation.instrument("inject", payload) do
19
- next prompt if memories.nil? || memories.empty?
15
+ def self.validate_max_bytes!(value)
16
+ return if value.nil? || (value.is_a?(Integer) && value >= 0)
17
+
18
+ raise ArgumentError, "max_bytes must be a non-negative integer, or nil"
19
+ end
20
20
 
21
- block = memories.map { |memory| render_memory(memory) }.join("\n")
22
- "#{prompt}\n\n#{@header}:\n<engram-memories>\n#{block}\n</engram-memories>"
21
+ # The budget includes the header, delimiters, escaping, and separators, but
22
+ # excludes the original prompt. Skip whole memories that do not fit.
23
+ def call(prompt:, memories:, max_bytes: nil)
24
+ self.class.validate_max_bytes!(max_bytes)
25
+ payload = {memory_count: memories&.size.to_i, injected_count: 0,
26
+ skipped_count: memories&.size.to_i, injected_bytes: 0, max_bytes: max_bytes}.compact
27
+ Engram::Instrumentation.instrument("inject", payload) do
28
+ next prompt if memories.nil? || memories.empty? || max_bytes == 0
29
+
30
+ prefix = "\n\n#{@header}:\n<engram-memories>\n"
31
+ suffix = "\n</engram-memories>"
32
+ bytes = prefix.bytesize + suffix.bytesize
33
+ lines = []
34
+ memories.each do |memory|
35
+ line = render_memory(memory)
36
+ added_bytes = line.bytesize + (lines.empty? ? 0 : 1)
37
+ next if max_bytes && bytes + added_bytes > max_bytes
38
+
39
+ lines << line
40
+ bytes += added_bytes
41
+ end
42
+ next prompt if lines.empty?
43
+
44
+ payload[:injected_count] = lines.size
45
+ payload[:skipped_count] -= lines.size
46
+ payload[:injected_bytes] = bytes
47
+ "#{prompt}#{prefix}#{lines.join("\n")}#{suffix}"
23
48
  end
24
49
  end
25
50
 
@@ -24,17 +24,27 @@ module Engram
24
24
  end
25
25
 
26
26
  # Returns Array<Record>, most relevant first.
27
- def call(query, scope:, limit: Engram.config.default_limit, kinds: nil)
27
+ def call(query, scope:, limit: Engram.config.default_limit, kinds: nil, min_similarity: nil)
28
28
  raise ArgumentError, "query must be a non-empty string" if query.to_s.strip.empty?
29
+ unless limit.is_a?(Integer) && limit >= 0
30
+ raise ArgumentError, "limit must be a non-negative integer"
31
+ end
32
+ validate_min_similarity!(min_similarity)
29
33
 
30
34
  payload = Engram::Instrumentation.payload(
31
35
  scope: scope,
32
36
  store: @store,
33
37
  limit: limit,
34
38
  kinds: Array(kinds).map(&:to_s),
35
- reranking: reranking?
39
+ reranking: reranking?,
40
+ min_similarity: min_similarity,
41
+ candidate_count: 0,
42
+ filtered_count: 0,
43
+ result_count: 0
36
44
  )
37
45
  Engram::Instrumentation.instrument("recall", payload) do
46
+ next [] if limit.zero?
47
+
38
48
  embedding = @embedder.embed(query)
39
49
  embedding_metadata = Engram::EmbeddingMetadata.for_embedder(@embedder, embedding: embedding)
40
50
  pool_limit = reranking? ? limit * @pool_factor : limit
@@ -47,16 +57,43 @@ module Engram
47
57
  kinds: kinds
48
58
  )
49
59
 
50
- results = (reranking? ? rerank(pool, embedding) : pool).first(limit)
60
+ candidates = if min_similarity.nil?
61
+ pool
62
+ else
63
+ pool.select { |record| meets_similarity?(record.embedding, embedding, min_similarity) }
64
+ end
65
+ results = (reranking? ? rerank(candidates, embedding) : candidates).first(limit)
51
66
  touch(results, scope) if @touch
52
67
  payload[:result_count] = results.size
53
68
  payload[:candidate_count] = pool.size
69
+ payload[:filtered_count] = pool.size - candidates.size
54
70
  results
55
71
  end
56
72
  end
57
73
 
58
74
  private
59
75
 
76
+ def validate_min_similarity!(value)
77
+ return if value.nil?
78
+ return if value.is_a?(Numeric) && value.real? && value.finite? && value.between?(-1, 1)
79
+
80
+ raise ArgumentError, "min_similarity must be a finite number between -1 and 1, or nil"
81
+ end
82
+
83
+ def meets_similarity?(embedding, query_embedding, minimum)
84
+ return false unless comparable_vector?(embedding) && comparable_vector?(query_embedding)
85
+ return false unless embedding.length == query_embedding.length
86
+
87
+ similarity = Engram::Math.cosine_similarity(query_embedding, embedding)
88
+ similarity.finite? && similarity.clamp(-1.0, 1.0) >= minimum
89
+ end
90
+
91
+ def comparable_vector?(vector)
92
+ vector.is_a?(Array) && !vector.empty? &&
93
+ vector.all? { |value| value.is_a?(Numeric) && value.real? && value.finite? } &&
94
+ vector.any? { |value| !value.zero? }
95
+ end
96
+
60
97
  def reranking?
61
98
  !@importance_weight.zero? || !@recency_weight.zero?
62
99
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Engram
4
- VERSION = "0.6.0"
4
+ VERSION = "0.7.0"
5
5
  end
@@ -9,4 +9,9 @@ Engram.configure do |config|
9
9
 
10
10
  # How many memories to recall by default.
11
11
  config.default_limit = 5
12
+
13
+ # Minimum cosine similarity. Tune for your embedding model.
14
+ # config.recall_min_similarity = 0.5
15
+ # Cap the appended memory block in bytes, including escaped text and delimiters.
16
+ # config.injection_max_bytes = 8_000
12
17
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: engram
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.0
4
+ version: 0.7.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexandr Kholodniak
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-07-28 00:00:00.000000000 Z
11
+ date: 2026-09-13 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: |
14
14
  Engram gives AI agents durable, long-term memory. It recalls relevant facts about a