engram 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: f2d96222b1c43e68da7da579a6b9d1d39f85f5cbe2480be6cd39a19dcf36adc6
4
- data.tar.gz: 53bc4261b347e498d20ea4171164b9478c55134067c33715641017b7572c78a7
3
+ metadata.gz: b3d370df000e9c12a40df932919f0859d9eb7623270ed5aeaa781747f239d865
4
+ data.tar.gz: 2c49fd3e8126a510f49b711a9ccf9d159ac01b6e540ef2ab168e7e51fa6509f3
5
5
  SHA512:
6
- metadata.gz: a8001a49d7c280ded764b75e2f62695f3eeaf229e114e5455ee9bfa32f58d0c2a94fd9d70a1b563e246da0efcb570fe5f95ca6ecd61e69ca3cad758a0491abae
7
- data.tar.gz: 29d1e339241f3edb54485925f20b78f81740770fc966c8e838cefaef839e191597080249f578f33c8f79c5eb2cc89563b0187cf897b296ab660ff4510a3b0d4d
6
+ metadata.gz: 161c8e9c199352d663e0ce670f3a6dc76ac98aba0b429566a1c3c96fa8ee5c7db260a449355f449b11dfdb25de4b85f207ca263fcffa6a76248d1269632ecddc
7
+ data.tar.gz: 1c3867577e6a86128fa4a4036b64cc2790754336c4cc462495284f25f5a7ac63ac56be24694ffb9578d2b50999bde325ca4ced68b6d8e0aed9143234108b0888
data/CHANGELOG.md CHANGED
@@ -5,6 +5,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.8.0] - 2026-09-15
9
+
10
+ ### Added
11
+
12
+ - Optional `expires_at` on memories. Expired records are excluded from search and injection while remaining available for inspection and deletion.
13
+ - `engram:expiry` generator for existing Rails apps, with a concurrent index for scoped expiry queries.
14
+ - Scoped `forget_expired` cleanup with batching, dry runs, and a Rails task. Deletion rechecks the deadline so concurrent expiry extensions are preserved.
15
+
8
16
  ## [0.7.0] - 2026-09-13
9
17
 
10
18
  ### Added
data/README.md CHANGED
@@ -76,7 +76,7 @@ explicitly version-gated:
76
76
  - Core types and facade: `Engram::Memory`, `Engram::Record`, `Engram::Decision`, `Engram::PersistencePolicy`, and `Engram.with_memory`.
77
77
  - Store and adapter ports: `Engram::Ports::MemoryStore`, `Engram::Ports::Embedder`, `Engram::Ports::Completion`.
78
78
  - Rails integration points: `has_memory`, `Memory#observe_later`, and generator outputs under `engram:` rake tasks.
79
- - Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_stale`, `rebuild_embeddings`, and `memories_from_source`.
79
+ - Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_stale`, `forget_expired`, `rebuild_embeddings`, and `memories_from_source`.
80
80
  - RubyLLM adapter contract points and evaluator entrypoints (`rake eval`, `rake eval:real`).
81
81
 
82
82
  ### Backward-compatibility commitments (pre-1.0)
@@ -228,7 +228,7 @@ end
228
228
  ### Filtered vector search
229
229
 
230
230
  Keep the generated B-tree index on `scope`. With an approximate vector index, pgvector
231
- applies scope and kind filters after the index scan. This can return fewer than `limit`
231
+ applies scope, kind, and expiry filters after the index scan. This can return fewer than `limit`
232
232
  memories even when enough matching rows exist.
233
233
 
234
234
  On pgvector 0.8.0+, HNSW iterative scans can search further:
@@ -553,6 +553,67 @@ Both settings default to nil. Pass nil explicitly to override a configured value
553
553
  the cap. `limit: 0` skips embedding and search; `max_bytes: 0` skips recall during injection.
554
554
  Invalid values raise `ArgumentError`.
555
555
 
556
+ ### Memory expiry
557
+
558
+ Set a deadline for temporary memories:
559
+
560
+ ```ruby
561
+ user.memory.add("Trial access is active", expires_at: 7.days.from_now)
562
+ ```
563
+
564
+ `expires_at` accepts a Time (including Rails time-zone values) or nil. Nil is the default
565
+ and never expires. A memory is expired at `expires_at <= Time.now`. Both built-in stores
566
+ exclude expired records before the search limit, so they no longer participate in recall
567
+ or consolidation. Injection checks expiry again when rendering.
568
+
569
+ Expiry does not delete rows. `memory.all` still returns them, and `memory.forget(id:)` can
570
+ delete them. A new observation may recreate the same fact from retained source data.
571
+ Custom extractors or `before_persist` can set expiry with `record.with(expires_at: deadline)`;
572
+ Engram does not infer deadlines from conversation text. Store updates replace expiry with
573
+ the supplied record's value, including nil.
574
+
575
+ New Rails installs include the expiry column and index. For a table created by 0.7.0 or
576
+ earlier, generate and run the upgrade migration:
577
+
578
+ ```sh
579
+ bin/rails generate engram:expiry
580
+ bin/rails db:migrate
581
+ ```
582
+
583
+ The migration adds a nullable datetime column and builds a partial index on scope and
584
+ expiry concurrently. Existing rows keep their data and do not expire. Reads and
585
+ non-expiring writes continue to work before migration; expiring writes raise an error
586
+ instead of losing the deadline. Restart app processes after migrating so ActiveRecord
587
+ refreshes its cached schema. If you use a custom table, adjust the generated migration.
588
+
589
+ Custom stores must persist `expires_at` and exclude expired records before limiting search
590
+ results. Recall also discards any expired records a custom store returns, but cannot fill
591
+ the gaps left by a store that filters after its limit.
592
+
593
+ To remove expired rows, preview the count, then run cleanup:
594
+
595
+ ```ruby
596
+ user.memory.forget_expired(dry_run: true)
597
+ # => {matched: 12, deleted: 0, dry_run: true}
598
+ user.memory.forget_expired(batch_size: 500)
599
+ # => {matched: 12, deleted: 12, dry_run: false}
600
+ ```
601
+
602
+ The Rails task requires a scope:
603
+
604
+ ```sh
605
+ DRY_RUN=true BATCH_SIZE=500 bin/rails 'engram:forget_expired[user:42]'
606
+ BATCH_SIZE=500 bin/rails 'engram:forget_expired[user:42]'
607
+ ```
608
+
609
+ Cleanup uses one cutoff for the run and reads only IDs in batches. It rechecks scope and
610
+ expiry when deleting, so a concurrent extension or removal of the deadline is preserved.
611
+ `matched` can exceed `deleted` if records change during the run; dry-run counts are advisory.
612
+ Like `forget(id:)`, this is explicit deletion and bypasses persistence hooks and policies.
613
+ Errors stop the run; completed batches stay deleted and a retry can process the remainder.
614
+ Custom stores need the optional `expired_ids` and `delete_expired` methods for cleanup.
615
+ The `forget_expired.engram` event reports counts without memory content or IDs.
616
+
556
617
  ### Observation and maintenance
557
618
 
558
619
  Observation uses a scope-and-turn claim before extraction. While a claim lease is live and
@@ -640,6 +701,7 @@ main memory pipeline:
640
701
 
641
702
  - `add.engram`
642
703
  - `forget.engram`
704
+ - `forget_expired.engram`
643
705
  - `recall.engram`
644
706
  - `inject.engram`
645
707
  - `observe.engram`
@@ -785,6 +847,7 @@ gem unpack engram-*.gem --target /tmp/engram-package-check
785
847
  - v0.5 (done): embedding provenance and scoped embedding rebuild operations.
786
848
  - v0.6 (done): structured provenance, source impact lookup, and grounding reports.
787
849
  - v0.7 (done): recall thresholds, injection limits, scoped deletion, and RubyLLM streaming fixes.
850
+ - v0.8 (done): explicit memory expiry, Rails upgrade migration, and scoped cleanup with dry runs.
788
851
  - later: additional storage backends and larger real-provider eval benchmarks.
789
852
 
790
853
  ## License
@@ -10,23 +10,26 @@ module Engram
10
10
  def initialize
11
11
  @records = {}
12
12
  @sequence = 0
13
+ @mutex = Mutex.new
13
14
  end
14
15
 
15
16
  def add(record)
16
17
  validate_scope!(record.scope)
17
18
 
18
- record.id = (@sequence += 1)
19
- @records[record.id] = record
20
- record
19
+ @mutex.synchronize do
20
+ record.id = (@sequence += 1)
21
+ @records[record.id] = record
22
+ record
23
+ end
21
24
  end
22
25
 
23
26
  def search(embedding:, scope:, limit:, kinds: nil, embedding_metadata: nil)
24
27
  Engram::EmbeddingMetadata.validate_query!(embedding, embedding_metadata)
25
28
  allowed_kinds = normalize_kinds(kinds)
29
+ now = Time.now
26
30
 
27
- results = @records
28
- .values
29
- .select { |r| searchable?(r, scope, allowed_kinds) }
31
+ results = @mutex.synchronize { @records.values }
32
+ .select { |r| searchable?(r, scope, allowed_kinds) && !r.expired?(at: now) }
30
33
  .map { |r| [r, Engram::Math.cosine_similarity(embedding, r.embedding)] }
31
34
  .sort_by { |(_, score)| -score }
32
35
  .first(limit)
@@ -37,7 +40,7 @@ module Engram
37
40
  end
38
41
 
39
42
  def all(scope:, limit: nil, offset: 0, after_id: nil)
40
- records = @records.values.select { |r| r.scope == scope }.sort_by { |record| record.id }
43
+ records = @mutex.synchronize { @records.values }.select { |r| r.scope == scope }.sort_by { |record| record.id }
41
44
  records = records.drop_while { |record| !after_id.nil? && record.id && record.id <= after_id }
42
45
  records = records.drop(offset) if offset > 0
43
46
  records = records.take(limit) if limit
@@ -45,36 +48,65 @@ module Engram
45
48
  end
46
49
 
47
50
  def existing_ids(scope:, ids:)
48
- ids.uniq.select { |id| @records[id]&.scope == scope }
51
+ @mutex.synchronize { ids.uniq.select { |id| @records[id]&.scope == scope } }
49
52
  end
50
53
 
51
54
  def update(scope:, id:, record:)
52
- existing = @records[id]
53
- raise Engram::Error, "no memory with id #{id.inspect} in scope #{scope.inspect}" unless existing&.scope == scope
54
- raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
55
-
56
- record.id = id
57
- @records[id] = record
55
+ @mutex.synchronize do
56
+ existing = @records[id]
57
+ raise Engram::Error, "no memory with id #{id.inspect} in scope #{scope.inspect}" unless existing&.scope == scope
58
+ raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
59
+
60
+ record.id = id
61
+ @records[id] = record
62
+ end
58
63
  end
59
64
 
60
65
  def delete(scope:, id:)
61
- return 0 unless @records[id]&.scope == scope
66
+ @mutex.synchronize do
67
+ return 0 unless @records[id]&.scope == scope
62
68
 
63
- @records.delete(id)
64
- 1
69
+ @records.delete(id)
70
+ 1
71
+ end
65
72
  end
66
73
 
67
74
  def touch(scope:, id:, at: Time.now)
68
- record = @records[id]
69
- return 0 unless record&.scope == scope
75
+ @mutex.synchronize do
76
+ record = @records[id]
77
+ return 0 unless record&.scope == scope
78
+
79
+ record.last_accessed_at = at
80
+ 1
81
+ end
82
+ end
70
83
 
71
- record.last_accessed_at = at
72
- 1
84
+ def expired_ids(scope:, at:, limit:, after_id: nil)
85
+ # IDs follow insertion order; the hash retains it across updates/deletes.
86
+ @mutex.synchronize do
87
+ @records.each_value.lazy
88
+ .select { |record| record.scope == scope && (after_id.nil? || record.id > after_id) && record.expired?(at: at) }
89
+ .map(&:id).take(limit).force
90
+ end
91
+ end
92
+
93
+ def delete_expired(scope:, ids:, at:)
94
+ @mutex.synchronize do
95
+ ids.uniq.count do |id|
96
+ record = @records[id]
97
+ next false unless record&.scope == scope && record.expired?(at: at)
98
+
99
+ @records.delete(id)
100
+ true
101
+ end
102
+ end
73
103
  end
74
104
 
75
105
  def clear
76
- @records.clear
77
- @sequence = 0
106
+ @mutex.synchronize do
107
+ @records.clear
108
+ @sequence = 0
109
+ end
78
110
  end
79
111
 
80
112
  private
@@ -18,6 +18,7 @@ module Engram
18
18
  validate_scope!(record.scope)
19
19
 
20
20
  row = model.create!(
21
+ **expiry_attributes(record),
21
22
  content: record.content,
22
23
  scope: record.scope,
23
24
  kind: record.kind.to_s,
@@ -31,6 +32,10 @@ module Engram
31
32
  def search(embedding:, scope:, limit:, kinds: nil, embedding_metadata: nil)
32
33
  Engram::EmbeddingMetadata.validate_query!(embedding, embedding_metadata)
33
34
  query = model.where(scope: scope)
35
+ if expiry_column?
36
+ expiry = model.arel_table[:expires_at]
37
+ query = query.where(expiry.eq(nil).or(expiry.gt(Time.now)))
38
+ end
34
39
  normalized_kinds = normalize_kinds(kinds)
35
40
  query = query.where(kind: normalized_kinds) if normalized_kinds
36
41
 
@@ -70,6 +75,7 @@ module Engram
70
75
  model.transaction do
71
76
  row = model.lock.find_by!(id: id, scope: scope)
72
77
  row.update!(
78
+ **expiry_attributes(record),
73
79
  content: record.content,
74
80
  kind: record.kind.to_s,
75
81
  importance: record.importance,
@@ -90,8 +96,37 @@ module Engram
90
96
  model.where(id: id, scope: scope).update_all(last_accessed_at: at)
91
97
  end
92
98
 
99
+ def expired_ids(scope:, at:, limit:, after_id: nil)
100
+ return [] unless expiry_column?
101
+
102
+ query = expired_scope(scope, at)
103
+ query = query.where(model.arel_table[:id].gt(after_id)) unless after_id.nil?
104
+ query.order(:id).limit(limit).pluck(:id)
105
+ end
106
+
107
+ def delete_expired(scope:, ids:, at:)
108
+ return 0 if ids.empty? || !expiry_column?
109
+
110
+ expired_scope(scope, at).where(id: ids).delete_all
111
+ end
112
+
93
113
  private
94
114
 
115
+ def expired_scope(scope, at)
116
+ model.where(scope: scope).where(model.arel_table[:expires_at].lteq(at))
117
+ end
118
+
119
+ def expiry_column?
120
+ model.column_names.include?("expires_at")
121
+ end
122
+
123
+ def expiry_attributes(record)
124
+ return {expires_at: record.expires_at} if expiry_column?
125
+ return {} if record.expires_at.nil?
126
+
127
+ raise Engram::Error, "PgvectorStore needs a nullable expires_at datetime column to store expiring memories"
128
+ end
129
+
95
130
  def validate_scope!(scope)
96
131
  raise Engram::Error, "memory scope cannot be nil" if scope.nil?
97
132
  end
@@ -124,7 +159,8 @@ module Engram
124
159
  importance: row.importance || 1.0,
125
160
  metadata: row.metadata || {},
126
161
  created_at: row.created_at,
127
- last_accessed_at: row.try(:last_accessed_at)
162
+ last_accessed_at: row.try(:last_accessed_at),
163
+ expires_at: row.try(:expires_at)
128
164
  )
129
165
  end
130
166
 
@@ -29,7 +29,8 @@ module Engram
29
29
  kind: [Symbol],
30
30
  importance: [Integer, Float],
31
31
  created_at: [Time],
32
- last_accessed_at: [NilClass, Time]
32
+ last_accessed_at: [NilClass, Time],
33
+ expires_at: [NilClass, Time]
33
34
  }.freeze
34
35
 
35
36
  Snapshot = Struct.new(:collection, :collection_state, :records)
data/lib/engram/memory.rb CHANGED
@@ -14,18 +14,17 @@ module Engram
14
14
 
15
15
  # Persist a memory record of the given kind. Returns nil when the configured
16
16
  # persistence policy rejects the record.
17
- def add(content, kind: :fact, importance: 1.0, metadata: {})
17
+ def add(content, kind: :fact, importance: 1.0, metadata: {}, expires_at: nil)
18
18
  Engram::Instrumentation.instrument("add", Engram::Instrumentation.payload(scope: scope, store: @store, kind: kind)) do
19
- embedding = @embedder.embed(content)
20
19
  record = Record.new(
21
20
  content: content,
22
21
  scope: scope,
23
- embedding: embedding,
24
22
  kind: kind,
25
23
  importance: importance,
26
- metadata: metadata
24
+ metadata: metadata,
25
+ expires_at: expires_at
27
26
  )
28
- persist(record)
27
+ persist(record.with(embedding: @embedder.embed(content)))
29
28
  end
30
29
  end
31
30
 
@@ -119,6 +118,13 @@ module Engram
119
118
  .call(scope: scope, older_than: older_than, min_importance: min_importance)
120
119
  end
121
120
 
121
+ # Delete expired records without loading their content or embeddings.
122
+ # Returns matched/deleted counts; dry runs only count eligible records.
123
+ def forget_expired(batch_size: 100, dry_run: false)
124
+ UseCases::ForgetExpired.new(store: @store)
125
+ .call(scope: scope, batch_size: batch_size, dry_run: dry_run)
126
+ end
127
+
122
128
  # Return the memories in this scope whose provenance references the exact host
123
129
  # source. `source_id` and `source_type` must each be non-blank Strings and are
124
130
  # matched exactly, without normalization. Source IDs are references, not an
@@ -12,12 +12,13 @@ module Engram
12
12
 
13
13
  # Return up to `limit` Records in `scope` nearest to `embedding`,
14
14
  # ordered most-relevant first. When `kinds` is provided, only records with
15
- # those canonical memory kinds are eligible.
15
+ # those canonical memory kinds are eligible. Exclude expired records before
16
+ # ordering and limiting; expires_at <= Time.now is expired, nil never expires.
16
17
  def search(embedding:, scope:, limit:, kinds: nil, embedding_metadata: nil)
17
18
  raise NotImplementedError, "#{self.class} must implement #search"
18
19
  end
19
20
 
20
- # All Records for a scope (mostly for inspection/tests).
21
+ # All Records for a scope, including expired records (inspection/maintenance).
21
22
  # Supports optional `limit` and `offset` for batching large sweeps.
22
23
  # Returned records are sorted in stable `id` order when batching is used.
23
24
  # Use `after_id` for keyset pagination.
@@ -49,6 +50,19 @@ module Engram
49
50
  raise NotImplementedError, "#{self.class} must implement #delete"
50
51
  end
51
52
 
53
+ # Optional expiry maintenance: up to `limit` IDs with expires_at <= at,
54
+ # ordered by ID, strictly after after_id. Never includes nil expiry.
55
+ def expired_ids(scope:, at:, limit:, after_id: nil)
56
+ raise NotImplementedError, "#{self.class} does not implement #expired_ids"
57
+ end
58
+
59
+ # Optional expiry maintenance: delete only the requested IDs in scope that
60
+ # are still expired at the same cutoff. Recheck expiry atomically at deletion
61
+ # so a concurrent deadline extension is preserved. Returns the deleted count.
62
+ def delete_expired(scope:, ids:, at:)
63
+ raise NotImplementedError, "#{self.class} does not implement #delete_expired"
64
+ end
65
+
52
66
  # Update the last-accessed timestamp of a memory. Used by recency-aware recall.
53
67
  # Returns the number of affected rows: 1 when touched, 0 when the scoped record
54
68
  # does not exist.
@@ -3,6 +3,21 @@
3
3
  # Loaded by the Railtie in Rails apps (where :environment boots the app and its
4
4
  # initializers) and by the gem's own Rakefile (which defines a no-op :environment).
5
5
  namespace :engram do
6
+ desc "Delete expired memories in a scope. Usage: bundle exec rake 'engram:forget_expired[user:1]'"
7
+ task :forget_expired, [:scope] => :environment do |_, args|
8
+ scope = args[:scope]
9
+ raise ArgumentError, "forget_expired requires a scope argument" if scope.to_s.empty?
10
+
11
+ dry_run = case ENV.fetch("DRY_RUN", "false").downcase
12
+ when "true", "1", "yes" then true
13
+ when "false", "0", "no" then false
14
+ else raise ArgumentError, "DRY_RUN must be true or false"
15
+ end
16
+ batch_size = Integer(ENV.fetch("BATCH_SIZE", "100"))
17
+ result = Engram::Memory.new(scope: scope).forget_expired(batch_size: batch_size, dry_run: dry_run)
18
+ puts "matched=#{result[:matched]} deleted=#{result[:deleted]} dry_run=#{result[:dry_run]}"
19
+ end
20
+
6
21
  desc "Rebuild embeddings in a memory scope. Usage: bundle exec rake 'engram:rebuild_embeddings[user:1]'"
7
22
  task :rebuild_embeddings, [:scope] => :environment do |_, args|
8
23
  scope = args[:scope]
data/lib/engram/record.rb CHANGED
@@ -9,15 +9,18 @@ module Engram
9
9
  # `semantic` kind is normalized to `fact` for compatibility with pre-1.0 records.
10
10
  class Record
11
11
  STATE_READERS = %i[
12
- id content scope embedding kind importance metadata created_at last_accessed_at
12
+ id content scope embedding kind importance metadata created_at last_accessed_at expires_at
13
13
  ].freeze
14
14
 
15
15
  attr_accessor :id, :last_accessed_at
16
16
  attr_reader :content, :embedding, :scope, :kind, :importance, :metadata,
17
- :created_at
17
+ :created_at, :expires_at
18
18
 
19
19
  def initialize(content:, scope:, id: nil, embedding: nil, kind: :fact,
20
- importance: 1.0, metadata: {}, created_at: nil, last_accessed_at: nil)
20
+ importance: 1.0, metadata: {}, created_at: nil, last_accessed_at: nil, expires_at: nil)
21
+ unless expires_at.nil? || expires_at.is_a?(Time)
22
+ raise ArgumentError, "expires_at must be a Time or nil"
23
+ end
21
24
  @id = id
22
25
  @content = content
23
26
  @scope = scope
@@ -27,12 +30,17 @@ module Engram
27
30
  @metadata = metadata
28
31
  @created_at = created_at || Time.now
29
32
  @last_accessed_at = last_accessed_at
33
+ @expires_at = expires_at&.getutc
30
34
  end
31
35
 
32
36
  def with(**attributes)
33
37
  self.class.new(**to_h.merge(attributes))
34
38
  end
35
39
 
40
+ def expired?(at: Time.now)
41
+ !expires_at.nil? && expires_at <= at
42
+ end
43
+
36
44
  # Return structured supporting-source metadata when this record carries a
37
45
  # provenance schema understood by the installed Engram version.
38
46
  # Legacy, malformed, and future-schema records remain readable and return nil.
@@ -0,0 +1,43 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Engram
4
+ module UseCases
5
+ # Deletes expired memories in bounded batches using one cutoff for the whole run.
6
+ class ForgetExpired
7
+ def initialize(store:)
8
+ @store = store
9
+ end
10
+
11
+ def call(scope:, batch_size: 100, dry_run: false, now: Time.now)
12
+ unless batch_size.is_a?(Integer) && batch_size.positive?
13
+ raise ArgumentError, "batch_size must be a positive integer"
14
+ end
15
+ unless dry_run.equal?(true) || dry_run.equal?(false)
16
+ raise ArgumentError, "dry_run must be true or false"
17
+ end
18
+ unless @store.respond_to?(:expired_ids) && @store.respond_to?(:delete_expired)
19
+ raise Engram::Error, "forget_expired requires a store implementing expired_ids and delete_expired"
20
+ end
21
+
22
+ counts = {matched: 0, deleted: 0, dry_run: dry_run}
23
+ payload = Engram::Instrumentation.payload(scope: scope, store: @store, **counts)
24
+ Engram::Instrumentation.instrument("forget_expired", payload) do
25
+ after_id = nil
26
+ loop do
27
+ ids = @store.expired_ids(scope: scope, at: now, limit: batch_size, after_id: after_id)
28
+ break if ids.empty?
29
+ raise Engram::Error, "expired_ids must advance its cursor" if ids.last == after_id
30
+
31
+ counts[:matched] += ids.length
32
+ counts[:deleted] += @store.delete_expired(scope: scope, ids: ids, at: now) unless dry_run
33
+ payload.merge!(counts)
34
+ after_id = ids.last
35
+ end
36
+ counts
37
+ end
38
+ rescue NotImplementedError
39
+ raise Engram::Error, "forget_expired requires a store implementing expired_ids and delete_expired"
40
+ end
41
+ end
42
+ end
43
+ end
@@ -31,7 +31,10 @@ module Engram
31
31
  suffix = "\n</engram-memories>"
32
32
  bytes = prefix.bytesize + suffix.bytesize
33
33
  lines = []
34
+ now = Time.now
34
35
  memories.each do |memory|
36
+ next if memory.expired?(at: now)
37
+
35
38
  line = render_memory(memory)
36
39
  added_bytes = line.bytesize + (lines.empty? ? 0 : 1)
37
40
  next if max_bytes && bytes + added_bytes > max_bytes
@@ -57,10 +57,10 @@ module Engram
57
57
  kinds: kinds
58
58
  )
59
59
 
60
- candidates = if min_similarity.nil?
61
- pool
62
- else
63
- pool.select { |record| meets_similarity?(record.embedding, embedding, min_similarity) }
60
+ now = Time.now
61
+ candidates = pool.select do |record|
62
+ !record.expired?(at: now) &&
63
+ (min_similarity.nil? || meets_similarity?(record.embedding, embedding, min_similarity))
64
64
  end
65
65
  results = (reranking? ? rerank(candidates, embedding) : candidates).first(limit)
66
66
  touch(results, scope) if @touch
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Engram
4
- VERSION = "0.7.0"
4
+ VERSION = "0.8.0"
5
5
  end
data/lib/engram.rb CHANGED
@@ -31,6 +31,7 @@ require_relative "engram/use_cases/recall"
31
31
  require_relative "engram/use_cases/inject"
32
32
  require_relative "engram/use_cases/observe"
33
33
  require_relative "engram/use_cases/forget"
34
+ require_relative "engram/use_cases/forget_expired"
34
35
  require_relative "engram/use_cases/rebuild_embeddings"
35
36
  require_relative "engram/use_cases/source_impact"
36
37
  require_relative "engram/use_cases/grounding_report"
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "rails/generators"
4
+ require "rails/generators/active_record"
5
+
6
+ module Engram
7
+ module Generators
8
+ # Adds expiry to a table created by Engram 0.7 or earlier.
9
+ class ExpiryGenerator < ::Rails::Generators::Base
10
+ include ::Rails::Generators::Migration
11
+
12
+ source_root File.expand_path("templates", __dir__)
13
+
14
+ def create_migration_file
15
+ migration_template "add_expiry_to_engram_memories.rb.tt",
16
+ "db/migrate/add_expiry_to_engram_memories.rb"
17
+ end
18
+
19
+ def self.next_migration_number(dir)
20
+ ::ActiveRecord::Generators::Base.next_migration_number(dir)
21
+ end
22
+
23
+ private
24
+
25
+ def migration_version
26
+ "#{::ActiveRecord::VERSION::MAJOR}.#{::ActiveRecord::VERSION::MINOR}"
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+
3
+ class AddExpiryToEngramMemories < ActiveRecord::Migration[<%= migration_version %>]
4
+ disable_ddl_transaction!
5
+
6
+ def change
7
+ add_column :engram_memories, :expires_at, :datetime
8
+ add_index :engram_memories, [:scope, :expires_at],
9
+ name: "index_engram_memories_on_scope_and_expiry",
10
+ where: "expires_at IS NOT NULL", algorithm: :concurrently
11
+ end
12
+ end
@@ -12,11 +12,15 @@ class CreateEngramMemories < ActiveRecord::Migration[<%= migration_version %>]
12
12
  t.jsonb :metadata, null: false, default: {}
13
13
  t.vector :embedding, limit: <%= dimensions %>
14
14
  t.datetime :last_accessed_at
15
+ t.datetime :expires_at
15
16
 
16
17
  t.timestamps
17
18
  end
18
19
 
19
20
  add_index :engram_memories, :scope
21
+ add_index :engram_memories, [:scope, :expires_at],
22
+ name: "index_engram_memories_on_scope_and_expiry",
23
+ where: "expires_at IS NOT NULL"
20
24
 
21
25
  # For larger datasets, choose one approximate vector index after backfilling data.
22
26
  # HNSW is a strong default for read-heavy recall workloads with frequent inserts.
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: engram
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.7.0
4
+ version: 0.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexandr Kholodniak
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-09-13 00:00:00.000000000 Z
11
+ date: 2026-09-15 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: |
14
14
  Engram gives AI agents durable, long-term memory. It recalls relevant facts about a
@@ -65,6 +65,7 @@ files:
65
65
  - lib/engram/reserved_metadata.rb
66
66
  - lib/engram/turn_digest.rb
67
67
  - lib/engram/use_cases/forget.rb
68
+ - lib/engram/use_cases/forget_expired.rb
68
69
  - lib/engram/use_cases/grounding_report.rb
69
70
  - lib/engram/use_cases/inject.rb
70
71
  - lib/engram/use_cases/observe.rb
@@ -72,7 +73,9 @@ files:
72
73
  - lib/engram/use_cases/recall.rb
73
74
  - lib/engram/use_cases/source_impact.rb
74
75
  - lib/engram/version.rb
76
+ - lib/generators/engram/expiry_generator.rb
75
77
  - lib/generators/engram/install_generator.rb
78
+ - lib/generators/engram/templates/add_expiry_to_engram_memories.rb.tt
76
79
  - lib/generators/engram/templates/create_engram_memories.rb.tt
77
80
  - lib/generators/engram/templates/initializer.rb.tt
78
81
  - lib/generators/engram/templates/memory_record.rb.tt