engram 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b3d370df000e9c12a40df932919f0859d9eb7623270ed5aeaa781747f239d865
4
- data.tar.gz: 2c49fd3e8126a510f49b711a9ccf9d159ac01b6e540ef2ab168e7e51fa6509f3
3
+ metadata.gz: 50765b71eaace1c3abc16581e67c2466bf992fa023b082fe29d4b2c889cfb9d9
4
+ data.tar.gz: c439b5c3f34b9c91b8e1b7099b015c9b05eda5db93edc66de2bd913370c19303
5
5
  SHA512:
6
- metadata.gz: 161c8e9c199352d663e0ce670f3a6dc76ac98aba0b429566a1c3c96fa8ee5c7db260a449355f449b11dfdb25de4b85f207ca263fcffa6a76248d1269632ecddc
7
- data.tar.gz: 1c3867577e6a86128fa4a4036b64cc2790754336c4cc462495284f25f5a7ac63ac56be24694ffb9578d2b50999bde325ca4ced68b6d8e0aed9143234108b0888
6
+ metadata.gz: e862092cd85d8501d437d18a9c31dfe039809196c927c685235ee42bc32be629df3fda5b3f3934fdc92c6f2a64e110eb30aecd1b2e8889178a1c97a2295e5564
7
+ data.tar.gz: fddf7838f9396078287d58d67f25cb1b182c355c21b32b2937b4715e8ed63541305927c8b48b070dee426e3f3196330572a11c02cf7f17a94ed76b2d1f7985dd
data/CHANGELOG.md CHANGED
@@ -5,6 +5,42 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.9.0] - 2026-10-12
9
+
10
+ ### Added
11
+
12
+ - `Memory#update(id:, ...)` changes the given attributes of a memory in the current scope,
13
+ keeping its id and timestamps. It returns the stored record, `nil` when `before_persist` or
14
+ the policy rejects the edit, and raises `Engram::MemoryNotFoundError` for an unknown id.
15
+ Content changes re-embed the memory and remove its provenance. Emits `update.engram`.
16
+ - `Memory#all(limit:, after_id:)` reads one page of memories in id order, and
17
+ `Memory#each_memory(batch_size:)` iterates a scope in batches. `all` without arguments is
18
+ unchanged.
19
+ - `Memory#forget_all(batch_size:, dry_run:)` and the `engram:forget_all[scope]` task delete
20
+ every memory in a scope in batches and return matched/deleted counts. Emits
21
+ `forget_all.engram`. The README describes the full erasure steps and an export example.
22
+ - Optional `MemoryStore#delete_many(scope:, ids:)`, implemented by the built-in stores; custom
23
+ stores without it fall back to `delete`.
24
+ - Optional `MemoryStore#find(scope:, id:)`. The in-memory and pgvector stores implement it;
25
+ custom stores without it fall back to scanning `all`.
26
+
27
+ ### Fixed
28
+
29
+ - `InMemoryStore#update` keeps the stored `created_at` and `last_accessed_at`, matching the
30
+ pgvector store.
31
+ - `Memory#add` no longer sends content to the embedder before `before_persist` and the
32
+ persistence policy run. Rejected text is not embedded, and redacted text is embedded once,
33
+ after redaction.
34
+ - `has_memory` raises `Engram::Error` for an owner that is not persisted (new or destroyed)
35
+ instead of deriving its scope from an unsaved or reused id.
36
+ - The install generator configures the embedder with the requested `--dimensions` so the
37
+ initializer matches the migration.
38
+
39
+ ### Changed
40
+
41
+ - On `Memory#add`, `before_persist` hooks now receive a record without an embedding. Hooks
42
+ that read or transform `record.embedding` on this path must handle `nil`.
43
+
8
44
  ## [0.8.0] - 2026-09-15
9
45
 
10
46
  ### Added
data/README.md CHANGED
@@ -76,7 +76,7 @@ explicitly version-gated:
76
76
  - Core types and facade: `Engram::Memory`, `Engram::Record`, `Engram::Decision`, `Engram::PersistencePolicy`, and `Engram.with_memory`.
77
77
  - Store and adapter ports: `Engram::Ports::MemoryStore`, `Engram::Ports::Embedder`, `Engram::Ports::Completion`.
78
78
  - Rails integration points: `has_memory`, `Memory#observe_later`, and generator outputs under `engram:` rake tasks.
79
- - Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_stale`, `forget_expired`, `rebuild_embeddings`, and `memories_from_source`.
79
+ - Lifecycle methods in `Engram::Memory`: `add`, `update`, `all`, `each_memory`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_all`, `forget_stale`, `forget_expired`, `rebuild_embeddings`, and `memories_from_source`.
80
80
  - RubyLLM adapter contract points and evaluator entrypoints (`rake eval`, `rake eval:real`).
81
81
 
82
82
  ### Backward-compatibility commitments (pre-1.0)
@@ -140,6 +140,9 @@ current_user.memory.add("Works at Acme Corp", kind: :fact)
140
140
  current_user.memory.recall("where does the user work?")
141
141
  ```
142
142
 
143
+ The default scope needs a persisted record; calling `memory` on a new or destroyed owner
144
+ raises `Engram::Error`. Pass `has_memory scope: -> { ... }` to use a different key.
145
+
143
146
  Run automatic observation off the request path:
144
147
 
145
148
  ```ruby
@@ -388,6 +391,11 @@ Engram.configure do |config|
388
391
  end
389
392
  ```
390
393
 
394
+ `Memory#add` runs `before_persist` and the policy before computing the embedding, so rejected
395
+ or redacted text is never sent to the embedder, and hooks receive a record without an
396
+ embedding. `observe` is different: the conversation goes to the completion provider, and
397
+ extracted facts are embedded for consolidation before hooks and policy run.
398
+
391
399
  Write-content filtering and transformation are not deletion authorization. A consolidation
392
400
  `forget` decision still validates scope, target existence, candidate integrity, and provenance, but it does not run
393
401
  `before_persist` or the policy's `call` method; this allows secret, transient, and redacted
@@ -396,6 +404,47 @@ implementing `allow_destructive?(record)` and returning exactly `true` or `false
396
404
  that only implement `call` affect writes but do not prevent deletion. The built-in policy uses
397
405
  this separate hook to retain its ungrounded-provenance protection.
398
406
 
407
+ ### Listing memories
408
+
409
+ `memory.all` returns every memory in the scope, including expired ones. For large scopes, read
410
+ one page at a time or iterate in batches:
411
+
412
+ ```ruby
413
+ page = memory.all(limit: 50)
414
+ next_page = memory.all(limit: 50, after_id: page.last.id)
415
+
416
+ memory.each_memory(batch_size: 500) { |record| export(record) }
417
+ ```
418
+
419
+ Both return memories in id order. A custom store whose `all` does not accept `limit:` and
420
+ `after_id:` is read in a single call.
421
+
422
+ ### Correcting memories
423
+
424
+ ```ruby
425
+ memory.update(id: record.id, content: "Lives in Berlin")
426
+ memory.update(id: record.id, importance: 0.2, expires_at: nil)
427
+ ```
428
+
429
+ `update` changes only the attributes you pass (`content`, `kind`, `importance`, `metadata`,
430
+ `expires_at`) and keeps the id, `created_at`, and `last_accessed_at`. `metadata:` replaces the
431
+ application metadata; `{}` clears it, and the reserved `_engram` key is not accepted.
432
+ `expires_at: nil` removes the expiry. Other attributes do not accept `nil`.
433
+
434
+ The edit goes through `before_persist` and the persistence policy like a new memory, and a
435
+ policy that implements `allow_destructive?` must also allow replacing the original record.
436
+ Expired memories can be updated too. `update` returns the stored record, `nil` if the hook or policy rejects it (the stored memory is
437
+ left unchanged), and raises `Engram::MemoryNotFoundError` if the id is not in this scope.
438
+ The embedding is recomputed when the content changes, or generated if the stored memory has
439
+ no embedding.
440
+
441
+ Changing the content removes the record's provenance, because the old sources no longer
442
+ support the new text. Other edits keep it.
443
+
444
+ `update` reads the record, applies the changes, and writes it back without conflict
445
+ detection. If `observe`, an embedding rebuild, or another `update` writes the same memory at
446
+ the same time, one of the changes can be lost.
447
+
399
448
  ### Extraction provenance
400
449
 
401
450
  Custom extractors remain compatible when they return an array of plain `Engram::Record`
@@ -614,6 +663,47 @@ Errors stop the run; completed batches stay deleted and a retry can process the
614
663
  Custom stores need the optional `expired_ids` and `delete_expired` methods for cleanup.
615
664
  The `forget_expired.engram` event reports counts without memory content or IDs.
616
665
 
666
+ ### Erasing or exporting a user's memory
667
+
668
+ To delete everything Engram stores for one owner, preview the count, then run it:
669
+
670
+ ```ruby
671
+ user.memory.forget_all(dry_run: true)
672
+ # => {matched: 87, deleted: 0, dry_run: true}
673
+ user.memory.forget_all(batch_size: 500)
674
+ # => {matched: 87, deleted: 87, dry_run: false}
675
+ ```
676
+
677
+ ```sh
678
+ DRY_RUN=true bin/rails 'engram:forget_all[user:42]'
679
+ bin/rails 'engram:forget_all[user:42]'
680
+ ```
681
+
682
+ `forget_all` deletes expired memories too and, like `forget(id:)`, bypasses hooks and policies.
683
+ It only removes rows in the memory store. Before running it for an erasure request:
684
+
685
+ 1. Stop new `observe` and `observe_later` calls for the owner, and let queued `ObserveJob`s
686
+ finish or remove them. Otherwise they can write new memories during or after the erasure.
687
+ 2. Run `forget_all`. Memories added while it runs may or may not be deleted, so check
688
+ `memory.all` afterwards.
689
+ 3. Delete conversation transcripts and other source data your app keeps; Engram does not
690
+ store them. Processed-turn markers in the cache expire on their own and hold no content.
691
+
692
+ To export, iterate in batches and write the fields you need:
693
+
694
+ ```ruby
695
+ File.open("memories.jsonl", "w") do |file|
696
+ user.memory.each_memory(batch_size: 500) do |record|
697
+ file.puts({id: record.id, kind: record.kind, content: record.content,
698
+ created_at: record.created_at, expires_at: record.expires_at}.to_json)
699
+ end
700
+ end
701
+ ```
702
+
703
+ Custom stores can implement the optional `delete_many(scope:, ids:)` to delete a batch in one
704
+ call; without it, `forget_all` deletes one id at a time. The `forget_all.engram` event reports
705
+ counts without memory content or IDs.
706
+
617
707
  ### Observation and maintenance
618
708
 
619
709
  Observation uses a scope-and-turn claim before extraction. While a claim lease is live and
@@ -700,8 +790,10 @@ When ActiveSupport is loaded, Engram emits `ActiveSupport::Notifications` events
700
790
  main memory pipeline:
701
791
 
702
792
  - `add.engram`
793
+ - `update.engram`
703
794
  - `forget.engram`
704
795
  - `forget_expired.engram`
796
+ - `forget_all.engram`
705
797
  - `recall.engram`
706
798
  - `inject.engram`
707
799
  - `observe.engram`
@@ -848,6 +940,7 @@ gem unpack engram-*.gem --target /tmp/engram-package-check
848
940
  - v0.6 (done): structured provenance, source impact lookup, and grounding reports.
849
941
  - v0.7 (done): recall thresholds, injection limits, scoped deletion, and RubyLLM streaming fixes.
850
942
  - v0.8 (done): explicit memory expiry, Rails upgrade migration, and scoped cleanup with dry runs.
943
+ - v0.9 (done): memory correction, paged listing and batched iteration, and scope erasure.
851
944
  - later: additional storage backends and larger real-provider eval benchmarks.
852
945
 
853
946
  ## License
@@ -51,14 +51,20 @@ module Engram
51
51
  @mutex.synchronize { ids.uniq.select { |id| @records[id]&.scope == scope } }
52
52
  end
53
53
 
54
+ def find(scope:, id:)
55
+ @mutex.synchronize do
56
+ record = @records[id]
57
+ record if record&.scope == scope
58
+ end
59
+ end
60
+
54
61
  def update(scope:, id:, record:)
55
62
  @mutex.synchronize do
56
63
  existing = @records[id]
57
64
  raise Engram::Error, "no memory with id #{id.inspect} in scope #{scope.inspect}" unless existing&.scope == scope
58
65
  raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
59
66
 
60
- record.id = id
61
- @records[id] = record
67
+ @records[id] = record.with(id: id, created_at: existing.created_at, last_accessed_at: existing.last_accessed_at)
62
68
  end
63
69
  end
64
70
 
@@ -90,6 +96,17 @@ module Engram
90
96
  end
91
97
  end
92
98
 
99
+ def delete_many(scope:, ids:)
100
+ @mutex.synchronize do
101
+ ids.uniq.count do |id|
102
+ next false unless @records[id]&.scope == scope
103
+
104
+ @records.delete(id)
105
+ true
106
+ end
107
+ end
108
+ end
109
+
93
110
  def delete_expired(scope:, ids:, at:)
94
111
  @mutex.synchronize do
95
112
  ids.uniq.count do |id|
@@ -69,6 +69,11 @@ module Engram
69
69
  end
70
70
  end
71
71
 
72
+ def find(scope:, id:)
73
+ row = model.find_by(scope: scope, id: id)
74
+ to_record(row) if row
75
+ end
76
+
72
77
  def update(scope:, id:, record:)
73
78
  raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
74
79
 
@@ -104,6 +109,12 @@ module Engram
104
109
  query.order(:id).limit(limit).pluck(:id)
105
110
  end
106
111
 
112
+ def delete_many(scope:, ids:)
113
+ return 0 if ids.empty?
114
+
115
+ model.where(scope: scope, id: ids).delete_all
116
+ end
117
+
107
118
  def delete_expired(scope:, ids:, at:)
108
119
  return 0 if ids.empty? || !expiry_column?
109
120
 
data/lib/engram/memory.rb CHANGED
@@ -4,6 +4,9 @@ module Engram
4
4
  # The friendly facade. Bound to one `scope` (an owner), it wires the configured store
5
5
  # and embedder into the use cases. This is what `user.memory` returns in Rails.
6
6
  class Memory
7
+ UNSET = Object.new.freeze
8
+ private_constant :UNSET
9
+
7
10
  attr_reader :scope
8
11
 
9
12
  def initialize(scope:, store: Engram.config.store, embedder: Engram.config.embedder)
@@ -16,18 +19,24 @@ module Engram
16
19
  # persistence policy rejects the record.
17
20
  def add(content, kind: :fact, importance: 1.0, metadata: {}, expires_at: nil)
18
21
  Engram::Instrumentation.instrument("add", Engram::Instrumentation.payload(scope: scope, store: @store, kind: kind)) do
19
- record = Record.new(
22
+ persist(Record.new(
20
23
  content: content,
21
24
  scope: scope,
22
25
  kind: kind,
23
26
  importance: importance,
24
27
  metadata: metadata,
25
28
  expires_at: expires_at
26
- )
27
- persist(record.with(embedding: @embedder.embed(content)))
29
+ ))
28
30
  end
29
31
  end
30
32
 
33
+ # Correct supplied attributes of a memory in this scope. Omitted fields are kept.
34
+ def update(id:, content: UNSET, kind: UNSET, importance: UNSET, metadata: UNSET, expires_at: UNSET)
35
+ attributes = {content: content, kind: kind, importance: importance, metadata: metadata, expires_at: expires_at}
36
+ .reject { |_, value| value.equal?(UNSET) }
37
+ UseCases::Update.new(store: @store, embedder: @embedder).call(scope: scope, id: id, attributes: attributes)
38
+ end
39
+
31
40
  # Return the most relevant memories for a query.
32
41
  def recall(query, limit: Engram.config.default_limit, kinds: nil,
33
42
  min_similarity: Engram.config.recall_min_similarity)
@@ -84,15 +93,35 @@ module Engram
84
93
  end
85
94
  end
86
95
 
87
- def all
88
- @store.all(scope: scope)
96
+ # Memories in this scope, including expired ones. Pass limit and after_id (the last id of
97
+ # the previous page) to read one page in id order.
98
+ def all(limit: nil, after_id: nil)
99
+ return @store.all(scope: scope) if limit.nil? && after_id.nil?
100
+ if !limit.nil? && !(limit.is_a?(Integer) && limit.positive?)
101
+ raise ArgumentError, "limit must be a positive integer"
102
+ end
103
+ if !after_id.nil? && !valid_id?(after_id)
104
+ raise ArgumentError, "after_id must be an Integer or a non-empty String"
105
+ end
106
+
107
+ @store.all(scope: scope, limit: limit, after_id: after_id)
108
+ end
109
+
110
+ # Yield every memory in this scope in id order, loading batch_size records at a time.
111
+ # Returns an Enumerator without a block.
112
+ def each_memory(batch_size: 100, &block)
113
+ unless batch_size.is_a?(Integer) && batch_size.positive?
114
+ raise ArgumentError, "batch_size must be a positive integer"
115
+ end
116
+ return enum_for(:each_memory, batch_size: batch_size) unless block
117
+
118
+ UseCases::EachMemory.new(store: @store).call(scope: scope, batch_size: batch_size, &block)
89
119
  end
90
120
 
91
121
  # Delete one memory in this scope. Returns 1 when deleted, or 0 when missing.
92
122
  # Explicit deletion bypasses extraction, consolidation, and persistence hooks.
93
123
  def forget(id:)
94
- valid_id = id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
95
- raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id
124
+ raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id?(id)
96
125
 
97
126
  payload = Engram::Instrumentation.payload(scope: scope, store: @store)
98
127
  Engram::Instrumentation.instrument("forget", payload) do
@@ -100,6 +129,12 @@ module Engram
100
129
  end
101
130
  end
102
131
 
132
+ # Delete every memory in this scope in batches. Returns matched and deleted counts.
133
+ # Bypasses extraction, consolidation, and persistence hooks, like #forget.
134
+ def forget_all(batch_size: 100, dry_run: false)
135
+ UseCases::ForgetAll.new(store: @store).call(scope: scope, batch_size: batch_size, dry_run: dry_run)
136
+ end
137
+
103
138
  # Recompute embeddings (and embedding metadata) for memories in the scope.
104
139
  # When `stale_only` is true, only records whose current metadata does not match the
105
140
  # active embedder are rebuilt.
@@ -142,6 +177,10 @@ module Engram
142
177
 
143
178
  private
144
179
 
180
+ def valid_id?(id)
181
+ id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
182
+ end
183
+
145
184
  def persist(record)
146
185
  Persistence.new(store: @store, embedder: @embedder).add(record)
147
186
  end
@@ -48,20 +48,30 @@ module Engram
48
48
  # Applies all write transformations without mutating the store. Input is validated
49
49
  # before callbacks can remove or replace provenance, and final output is validated too.
50
50
  def prepare(record)
51
- original_provenance = validate_provenance!(record)
52
51
  original_content = record.content
52
+ record = transform(record)
53
+ # Embed only after hooks and policy so rejected or redacted text never reaches the embedder.
54
+ if record && (record.embedding.nil? || record.content != original_content)
55
+ record = record.with(embedding: @embedder.embed(record.content))
56
+ end
57
+ record = Engram::EmbeddingMetadata.attach(record, embedder: @embedder) if record
58
+ validate_provenance!(record) if record
59
+ record
60
+ end
61
+
62
+ # Runs before_persist and the policy without embedding. Memory#update passes a block
63
+ # that runs after each step to check the result and drop invalidated provenance.
64
+ def transform(record)
65
+ original_provenance = validate_provenance!(record)
53
66
  record = @before_persist.call(record) if @before_persist
54
67
  if record
55
68
  transformed_provenance = validate_provenance!(record)
56
69
  validate_provenance_trust!(original_provenance, transformed_provenance) if @before_persist
57
70
  end
71
+ record = yield(record) if record && block_given?
58
72
  record = @persistence_policy.call(record) if record && @persistence_policy
59
73
  validate_provenance!(record) if record
60
- if record && record.content != original_content
61
- record = record.with(embedding: @embedder.embed(record.content))
62
- end
63
- record = Engram::EmbeddingMetadata.attach(record, embedder: @embedder) if record
64
- validate_provenance!(record) if record
74
+ record = yield(record) if record && block_given?
65
75
  record
66
76
  end
67
77
 
@@ -2,13 +2,12 @@
2
2
 
3
3
  module Engram
4
4
  module Ports
5
- # PLACEHOLDER (v0.2). Contract for deriving candidate facts from a conversation turn.
6
- # Declared now so the differentiator (extract -> consolidate) slots in without
7
- # reworking the core. Not implemented in v0.1.
5
+ # Contract for deriving candidate facts from a conversation turn.
6
+ # Implementation: Extractors::LLMExtractor.
8
7
  module Extractor
9
8
  # Given conversation messages, return an Array whose members are Record or Extraction values.
10
9
  def extract(messages:, scope:)
11
- raise NotImplementedError, "Extractor arrives in v0.2"
10
+ raise NotImplementedError, "#{self.class} must implement #extract"
12
11
  end
13
12
  end
14
13
  end
@@ -26,6 +26,12 @@ module Engram
26
26
  raise NotImplementedError, "#{self.class} must implement #all"
27
27
  end
28
28
 
29
+ # Optional scoped lookup, including expired records, without touching access time.
30
+ # Returns a Record or nil. Legacy adapters can fall back to #all.
31
+ def find(scope:, id:)
32
+ raise NotImplementedError, "#{self.class} does not implement #find"
33
+ end
34
+
29
35
  # Optional performance capability: return the subset of requested ids that
30
36
  # exists in `scope`, preserving each requested value's representation rather
31
37
  # than returning a store-native cast of it. Return at most one requested
@@ -50,6 +56,12 @@ module Engram
50
56
  raise NotImplementedError, "#{self.class} must implement #delete"
51
57
  end
52
58
 
59
+ # Optional bulk deletion: delete the requested IDs that exist in scope and return the
60
+ # deleted count. Callers fall back to #delete for adapters without it.
61
+ def delete_many(scope:, ids:)
62
+ raise NotImplementedError, "#{self.class} does not implement #delete_many"
63
+ end
64
+
53
65
  # Optional expiry maintenance: up to `limit` IDs with expires_at <= at,
54
66
  # ordered by ID, strictly after after_id. Never includes nil expiry.
55
67
  def expired_ids(scope:, at:, limit:, after_id: nil)
@@ -22,6 +22,11 @@ module Engram
22
22
  if scope_proc
23
23
  instance_exec(&scope_proc)
24
24
  else
25
+ unless persisted?
26
+ raise Engram::Error,
27
+ "#{self.class.name} must be saved before using memory, or pass a scope to has_memory"
28
+ end
29
+
25
30
  "#{self.class.name.underscore}:#{id}"
26
31
  end
27
32
  Engram::Memory.new(scope: key)
@@ -18,6 +18,21 @@ namespace :engram do
18
18
  puts "matched=#{result[:matched]} deleted=#{result[:deleted]} dry_run=#{result[:dry_run]}"
19
19
  end
20
20
 
21
+ desc "Delete every memory in a scope. Usage: bundle exec rake 'engram:forget_all[user:1]'"
22
+ task :forget_all, [:scope] => :environment do |_, args|
23
+ scope = args[:scope]
24
+ raise ArgumentError, "forget_all requires a scope argument" if scope.to_s.empty?
25
+
26
+ dry_run = case ENV.fetch("DRY_RUN", "false").downcase
27
+ when "true", "1", "yes" then true
28
+ when "false", "0", "no" then false
29
+ else raise ArgumentError, "DRY_RUN must be true or false"
30
+ end
31
+ batch_size = Integer(ENV.fetch("BATCH_SIZE", "100"))
32
+ result = Engram::Memory.new(scope: scope).forget_all(batch_size: batch_size, dry_run: dry_run)
33
+ puts "matched=#{result[:matched]} deleted=#{result[:deleted]} dry_run=#{result[:dry_run]}"
34
+ end
35
+
21
36
  desc "Rebuild embeddings in a memory scope. Usage: bundle exec rake 'engram:rebuild_embeddings[user:1]'"
22
37
  task :rebuild_embeddings, [:scope] => :environment do |_, args|
23
38
  scope = args[:scope]
@@ -0,0 +1,56 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Engram
4
+ module UseCases
5
+ # Yields every memory in a scope in id order, loading at most batch_size records at a time.
6
+ # Stores whose #all does not accept limit/after_id are read in one call.
7
+ class EachMemory
8
+ def initialize(store:)
9
+ @store = store
10
+ end
11
+
12
+ def call(scope:, batch_size: 100, &block)
13
+ unless batch_size.is_a?(Integer) && batch_size.positive?
14
+ raise ArgumentError, "batch_size must be a positive integer"
15
+ end
16
+ unless keyset_supported?
17
+ sort_by_id(@store.all(scope: scope)).each(&block)
18
+ return nil
19
+ end
20
+
21
+ after_id = nil
22
+ loop do
23
+ batch = @store.all(scope: scope, limit: batch_size, after_id: after_id)
24
+ break if batch.empty?
25
+ unless batch.size <= batch_size && advances?(batch.map(&:id), after_id)
26
+ raise Engram::Error, "store #all must return pages that advance by id"
27
+ end
28
+
29
+ batch.each(&block)
30
+ after_id = batch.last.id
31
+ end
32
+ nil
33
+ end
34
+
35
+ private
36
+
37
+ def advances?(ids, after_id)
38
+ ids = [after_id, *ids] unless after_id.nil?
39
+ ids.none?(&:nil?) && ids.each_cons(2).all? { |previous, current| previous < current }
40
+ rescue ArgumentError, NoMethodError
41
+ false
42
+ end
43
+
44
+ def sort_by_id(records)
45
+ records.sort_by(&:id)
46
+ rescue ArgumentError, NoMethodError
47
+ raise Engram::Error, "store #all returned ids that cannot be ordered"
48
+ end
49
+
50
+ def keyset_supported?
51
+ keywords = @store.method(:all).parameters.filter_map { |kind, name| name if %i[key keyreq].include?(kind) }
52
+ keywords.include?(:limit) && keywords.include?(:after_id)
53
+ end
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,59 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Engram
4
+ module UseCases
5
+ # Deletes every memory in a scope in id-ordered batches. Like Memory#forget, it bypasses
6
+ # extraction, consolidation, and persistence hooks.
7
+ class ForgetAll
8
+ def initialize(store:)
9
+ @store = store
10
+ end
11
+
12
+ def call(scope:, batch_size: 100, dry_run: false)
13
+ unless batch_size.is_a?(Integer) && batch_size.positive?
14
+ raise ArgumentError, "batch_size must be a positive integer"
15
+ end
16
+ unless dry_run.equal?(true) || dry_run.equal?(false)
17
+ raise ArgumentError, "dry_run must be true or false"
18
+ end
19
+
20
+ counts = {matched: 0, deleted: 0, dry_run: dry_run}
21
+ payload = Engram::Instrumentation.payload(scope: scope, store: @store, **counts)
22
+ Engram::Instrumentation.instrument("forget_all", payload) do
23
+ ids = []
24
+ flush = lambda do
25
+ counts[:matched] += ids.size
26
+ payload.merge!(counts)
27
+ unless dry_run
28
+ delete(scope, ids) do |deleted|
29
+ counts[:deleted] += deleted
30
+ payload.merge!(counts)
31
+ end
32
+ end
33
+ ids = []
34
+ end
35
+ EachMemory.new(store: @store).call(scope: scope, batch_size: batch_size) do |record|
36
+ ids << record.id
37
+ flush.call if ids.size == batch_size
38
+ end
39
+ flush.call unless ids.empty?
40
+ counts
41
+ end
42
+ end
43
+
44
+ private
45
+
46
+ def delete(scope, ids)
47
+ if @store.respond_to?(:delete_many)
48
+ begin
49
+ return yield @store.delete_many(scope: scope, ids: ids)
50
+ rescue NotImplementedError
51
+ # Optional capability: legacy adapters may inherit the default stub.
52
+ end
53
+ end
54
+
55
+ ids.each { |id| yield @store.delete(scope: scope, id: id).to_i }
56
+ end
57
+ end
58
+ end
59
+ end
@@ -0,0 +1,185 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Engram
4
+ module UseCases
5
+ # Scoped read/merge/write correction. Concurrent writers require host coordination.
6
+ class Update
7
+ def initialize(store:, embedder:)
8
+ @store = store
9
+ @embedder = embedder
10
+ end
11
+
12
+ def call(scope:, id:, attributes:)
13
+ attributes = validate_attributes!(id, attributes)
14
+ payload = Engram::Instrumentation.payload(scope: scope, store: @store,
15
+ fields: attributes.keys.map(&:to_s), content_changed: false, reembedded: false)
16
+ Engram::Instrumentation.instrument("update", payload) do
17
+ original = find(scope, id)
18
+ unless original && Engram::Internal::Scope.record_matches?(original, scope)
19
+ payload[:outcome] = "missing"
20
+ raise Engram::MemoryNotFoundError, "no memory in this scope"
21
+ end
22
+
23
+ # Validate before any edit can remove inherited evidence.
24
+ Engram::Provenance.canonical_payload_for_persistence(original.metadata)
25
+ original = detach(plain_times(original))
26
+ original = original.with(metadata: Engram::Provenance.canonical_metadata_for_persistence(original.metadata))
27
+ persistence = Engram::Persistence.new(store: @store, embedder: @embedder)
28
+ unless persistence.allowed?(detach(original))
29
+ payload[:outcome] = "rejected"
30
+ next nil
31
+ end
32
+
33
+ candidate = merge(original, attributes)
34
+ candidate = invalidate_provenance(candidate, original)
35
+ payload[:content_changed] = candidate.content != original.content
36
+ evidence = provenance(candidate)
37
+ candidate = persistence.transform(candidate) do |transformed|
38
+ validate_transformation!(transformed, original, scope)
39
+ unless provenance(transformed) == evidence
40
+ raise Engram::Error, "update transformations cannot change provenance trust"
41
+ end
42
+ transformed = invalidate_provenance(transformed, original)
43
+ evidence = provenance(transformed)
44
+ payload[:content_changed] = transformed.content != original.content
45
+ transformed
46
+ end
47
+ unless candidate
48
+ payload[:outcome] = "rejected"
49
+ next nil
50
+ end
51
+
52
+ payload[:content_changed] = candidate.content != original.content
53
+ candidate = replace_reserved_field(candidate, "embedding")
54
+ if payload[:content_changed] || original.embedding.nil?
55
+ candidate = candidate.with(embedding: @embedder.embed(candidate.content))
56
+ payload[:reembedded] = true
57
+ candidate = Engram::EmbeddingMetadata.attach(candidate, embedder: @embedder)
58
+ else
59
+ candidate = replace_reserved_field(candidate, "embedding", source: original)
60
+ .with(embedding: original.embedding)
61
+ end
62
+ candidate = candidate.with(created_at: original.created_at, last_accessed_at: original.last_accessed_at)
63
+ stored = persistence.update_prepared(scope: scope, id: id, record: candidate)
64
+ payload[:outcome] = "updated"
65
+ stored
66
+ end
67
+ end
68
+
69
+ private
70
+
71
+ def validate_attributes!(id, attributes)
72
+ valid_id = id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
73
+ raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id
74
+ raise ArgumentError, "at least one attribute must be supplied" if attributes.empty?
75
+
76
+ attributes = attributes.dup
77
+ attributes.each do |field, value|
78
+ valid = case field
79
+ when :content
80
+ value.is_a?(String) && value.valid_encoding? && !value.strip.empty?
81
+ when :kind
82
+ attributes[field] = Engram::MemoryKind.normalize(value)
83
+ true
84
+ when :importance
85
+ (value.is_a?(Integer) || value.is_a?(Float)) && value.finite?
86
+ when :metadata
87
+ value.is_a?(Hash) && !reserved_metadata?(value)
88
+ when :expires_at
89
+ value.nil? || value.is_a?(Time)
90
+ else
91
+ false
92
+ end
93
+ raise ArgumentError, "invalid #{field}" unless valid
94
+ end
95
+ attributes
96
+ end
97
+
98
+ def reserved_metadata?(metadata)
99
+ Engram::Internal::CoreHash.each_pair(metadata) do |key, _|
100
+ return true if key.equal?(:_engram)
101
+ return true if key.is_a?(String) && String.instance_method(:==).bind_call(key, "_engram")
102
+ end
103
+ false
104
+ end
105
+
106
+ def find(scope, id)
107
+ if @store.respond_to?(:find)
108
+ begin
109
+ return @store.find(scope: scope, id: id)
110
+ rescue NotImplementedError
111
+ # Optional capability: legacy adapters may inherit the default stub.
112
+ end
113
+ end
114
+ @store.all(scope: scope).find { |record| record.id == id }
115
+ end
116
+
117
+ def detach(record)
118
+ Engram::Internal::CandidateIntegrity.new.detach(record)
119
+ rescue Engram::Internal::CandidateIntegrity::Error => error
120
+ raise Engram::Error, error.message
121
+ end
122
+
123
+ # Rails stores return ActiveSupport::TimeWithZone; candidates accept only plain Time.
124
+ def plain_times(record)
125
+ record.with(
126
+ created_at: plain_time(record.created_at),
127
+ last_accessed_at: plain_time(record.last_accessed_at),
128
+ expires_at: plain_time(record.expires_at)
129
+ )
130
+ end
131
+
132
+ def plain_time(value)
133
+ (value.nil? || value.instance_of?(Time)) ? value : Time.at(value.to_r).utc
134
+ end
135
+
136
+ def merge(original, attributes)
137
+ if attributes.key?(:metadata)
138
+ metadata = Hash.instance_method(:transform_values).bind_call(attributes[:metadata]) { |value| value }
139
+ ["_engram", :_engram].each do |key|
140
+ metadata[key] = original.metadata[key] if original.metadata.key?(key)
141
+ end
142
+ attributes = attributes.merge(metadata: metadata)
143
+ end
144
+ detach(original.with(**attributes))
145
+ end
146
+
147
+ def provenance(record)
148
+ Engram::Provenance.canonical_integrity_representation_for_persistence(record.metadata)
149
+ end
150
+
151
+ def invalidate_provenance(record, original)
152
+ return record if record.content == original.content
153
+
154
+ replace_reserved_field(record, "provenance")
155
+ end
156
+
157
+ # Keep unrelated reserved siblings and preserve the stored embedding schema
158
+ # exactly when reusing its vector, including legacy symbol-key aliases.
159
+ def replace_reserved_field(record, field, source: nil)
160
+ metadata = record.metadata.dup
161
+ ["_engram", :_engram].each do |key|
162
+ reserved = metadata[key]&.dup || {}
163
+ reserved.delete(field)
164
+ reserved.delete(field.to_sym)
165
+ inherited = source&.metadata&.fetch(key, {}) || {}
166
+ [field, field.to_sym].each do |alias_key|
167
+ reserved[alias_key] = inherited[alias_key] if inherited.key?(alias_key)
168
+ end
169
+ metadata[key] = reserved if metadata.key?(key) || !reserved.empty?
170
+ end
171
+ record.with(metadata: metadata)
172
+ end
173
+
174
+ def validate_transformation!(record, original, scope)
175
+ unless Engram::Internal::Scope.record_matches?(record, scope) && record.id == original.id
176
+ raise Engram::Error, "update transformations cannot change memory identity or scope"
177
+ end
178
+ validate_attributes!(record.id,
179
+ content: record.content, kind: record.kind, importance: record.importance, expires_at: record.expires_at)
180
+ rescue ArgumentError => error
181
+ raise Engram::Error, error.message
182
+ end
183
+ end
184
+ end
185
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Engram
4
- VERSION = "0.8.0"
4
+ VERSION = "0.9.0"
5
5
  end
data/lib/engram.rb CHANGED
@@ -31,6 +31,9 @@ require_relative "engram/use_cases/recall"
31
31
  require_relative "engram/use_cases/inject"
32
32
  require_relative "engram/use_cases/observe"
33
33
  require_relative "engram/use_cases/forget"
34
+ require_relative "engram/use_cases/update"
35
+ require_relative "engram/use_cases/each_memory"
36
+ require_relative "engram/use_cases/forget_all"
34
37
  require_relative "engram/use_cases/forget_expired"
35
38
  require_relative "engram/use_cases/rebuild_embeddings"
36
39
  require_relative "engram/use_cases/source_impact"
@@ -65,6 +68,9 @@ module Engram
65
68
  # Raised when a live claim suppresses an observation that has not completed.
66
69
  class ObservationInProgressError < Error; end
67
70
 
71
+ # Raised when a correction targets a memory absent from the caller's scope.
72
+ class MemoryNotFoundError < Error; end
73
+
68
74
  class << self
69
75
  def config
70
76
  @config ||= Configuration.new
@@ -44,6 +44,12 @@ module Engram
44
44
  options[:dimensions]
45
45
  end
46
46
 
47
+ def embedder_arguments
48
+ return "" if dimensions == DEFAULT_DIMENSIONS
49
+
50
+ "(dimensions: #{dimensions})"
51
+ end
52
+
47
53
  def validate_dimensions!
48
54
  return if dimensions.is_a?(Integer) && dimensions.positive?
49
55
 
@@ -5,7 +5,8 @@ Engram.configure do |config|
5
5
  config.store = Engram::Adapters::PgvectorStore.new
6
6
 
7
7
  # Generate embeddings via RubyLLM. Configure RubyLLM separately with your provider keys.
8
- config.embedder = Engram::Adapters::RubyLLMEmbedder.new
8
+ # The embedder must return <%= dimensions %>-dimensional vectors to match the migration.
9
+ config.embedder = Engram::Adapters::RubyLLMEmbedder.new<%= embedder_arguments %>
9
10
 
10
11
  # How many memories to recall by default.
11
12
  config.default_limit = 5
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: engram
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.0
4
+ version: 0.9.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexandr Kholodniak
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-09-15 00:00:00.000000000 Z
11
+ date: 2026-10-11 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: |
14
14
  Engram gives AI agents durable, long-term memory. It recalls relevant facts about a
@@ -64,7 +64,9 @@ files:
64
64
  - lib/engram/record.rb
65
65
  - lib/engram/reserved_metadata.rb
66
66
  - lib/engram/turn_digest.rb
67
+ - lib/engram/use_cases/each_memory.rb
67
68
  - lib/engram/use_cases/forget.rb
69
+ - lib/engram/use_cases/forget_all.rb
68
70
  - lib/engram/use_cases/forget_expired.rb
69
71
  - lib/engram/use_cases/grounding_report.rb
70
72
  - lib/engram/use_cases/inject.rb
@@ -72,6 +74,7 @@ files:
72
74
  - lib/engram/use_cases/rebuild_embeddings.rb
73
75
  - lib/engram/use_cases/recall.rb
74
76
  - lib/engram/use_cases/source_impact.rb
77
+ - lib/engram/use_cases/update.rb
75
78
  - lib/engram/version.rb
76
79
  - lib/generators/engram/expiry_generator.rb
77
80
  - lib/generators/engram/install_generator.rb