engram 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f2d96222b1c43e68da7da579a6b9d1d39f85f5cbe2480be6cd39a19dcf36adc6
-  data.tar.gz: 53bc4261b347e498d20ea4171164b9478c55134067c33715641017b7572c78a7
+  metadata.gz: 50765b71eaace1c3abc16581e67c2466bf992fa023b082fe29d4b2c889cfb9d9
+  data.tar.gz: c439b5c3f34b9c91b8e1b7099b015c9b05eda5db93edc66de2bd913370c19303
 SHA512:
-  metadata.gz: a8001a49d7c280ded764b75e2f62695f3eeaf229e114e5455ee9bfa32f58d0c2a94fd9d70a1b563e246da0efcb570fe5f95ca6ecd61e69ca3cad758a0491abae
-  data.tar.gz: 29d1e339241f3edb54485925f20b78f81740770fc966c8e838cefaef839e191597080249f578f33c8f79c5eb2cc89563b0187cf897b296ab660ff4510a3b0d4d
+  metadata.gz: e862092cd85d8501d437d18a9c31dfe039809196c927c685235ee42bc32be629df3fda5b3f3934fdc92c6f2a64e110eb30aecd1b2e8889178a1c97a2295e5564
+  data.tar.gz: fddf7838f9396078287d58d67f25cb1b182c355c21b32b2937b4715e8ed63541305927c8b48b070dee426e3f3196330572a11c02cf7f17a94ed76b2d1f7985dd

data/CHANGELOG.md

@@ -5,6 +5,50 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-10-12
+
+### Added
+
+- `Memory#update(id:, ...)` changes the given attributes of a memory in the current scope,
+  keeping its id and timestamps. It returns the stored record, `nil` when `before_persist` or
+  the policy rejects the edit, and raises `Engram::MemoryNotFoundError` for an unknown id.
+  Content changes re-embed the memory and remove its provenance. Emits `update.engram`.
+- `Memory#all(limit:, after_id:)` reads one page of memories in id order, and
+  `Memory#each_memory(batch_size:)` iterates a scope in batches. `all` without arguments is
+  unchanged.
+- `Memory#forget_all(batch_size:, dry_run:)` and the `engram:forget_all[scope]` task delete
+  every memory in a scope in batches and return matched/deleted counts. Emits
+  `forget_all.engram`. The README describes the full erasure steps and an export example.
+- Optional `MemoryStore#delete_many(scope:, ids:)`, implemented by the built-in stores; custom
+  stores without it fall back to `delete`.
+- Optional `MemoryStore#find(scope:, id:)`. The in-memory and pgvector stores implement it;
+  custom stores without it fall back to scanning `all`.
+
+### Fixed
+
+- `InMemoryStore#update` keeps the stored `created_at` and `last_accessed_at`, matching the
+  pgvector store.
+- `Memory#add` no longer sends content to the embedder before `before_persist` and the
+  persistence policy run. Rejected text is not embedded, and redacted text is embedded once,
+  after redaction.
+- `has_memory` raises `Engram::Error` for an owner that is not persisted (new or destroyed)
+  instead of deriving its scope from an unsaved or reused id.
+- The install generator configures the embedder with the requested `--dimensions` so the
+  initializer matches the migration.
+
+### Changed
+
+- On `Memory#add`, `before_persist` hooks now receive a record without an embedding. Hooks
+  that read or transform `record.embedding` on this path must handle `nil`.
+
+## [0.8.0] - 2026-09-15
+
+### Added
+
+- Optional `expires_at` on memories. Expired records are excluded from search and injection while remaining available for inspection and deletion.
+- `engram:expiry` generator for existing Rails apps, with a concurrent index for scoped expiry queries.
+- Scoped `forget_expired` cleanup with batching, dry runs, and a Rails task. Deletion rechecks the deadline so concurrent expiry extensions are preserved.
+
 ## [0.7.0] - 2026-09-13
 
 ### Added

data/README.md

@@ -76,7 +76,7 @@ explicitly version-gated:
 - Core types and facade: `Engram::Memory`, `Engram::Record`, `Engram::Decision`, `Engram::PersistencePolicy`, and `Engram.with_memory`.
 - Store and adapter ports: `Engram::Ports::MemoryStore`, `Engram::Ports::Embedder`, `Engram::Ports::Completion`.
 - Rails integration points: `has_memory`, `Memory#observe_later`, and generator outputs under `engram:` rake tasks.
-- Lifecycle methods in `Engram::Memory`: `add`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_stale`, `rebuild_embeddings`, and `memories_from_source`.
+- Lifecycle methods in `Engram::Memory`: `add`, `update`, `all`, `each_memory`, `recall`, `inject_into`, `observe`, `observe_later`, `forget`, `forget_all`, `forget_stale`, `forget_expired`, `rebuild_embeddings`, and `memories_from_source`.
 - RubyLLM adapter contract points and evaluator entrypoints (`rake eval`, `rake eval:real`).
 
 ### Backward-compatibility commitments (pre-1.0)
@@ -140,6 +140,9 @@ current_user.memory.add("Works at Acme Corp", kind: :fact)
 current_user.memory.recall("where does the user work?")
 ```
 
+The default scope needs a persisted record; calling `memory` on a new or destroyed owner
+raises `Engram::Error`. Pass `has_memory scope: -> { ... }` to use a different key.
+
 Run automatic observation off the request path:
 
 ```ruby
@@ -228,7 +231,7 @@ end
 ### Filtered vector search
 
 Keep the generated B-tree index on `scope`. With an approximate vector index, pgvector
-applies scope and kind filters after the index scan. This can return fewer than `limit`
+applies scope, kind, and expiry filters after the index scan. This can return fewer than `limit`
 memories even when enough matching rows exist.
 
 On pgvector 0.8.0+, HNSW iterative scans can search further:
@@ -388,6 +391,11 @@ Engram.configure do |config|
 end
 ```
 
+`Memory#add` runs `before_persist` and the policy before computing the embedding, so rejected
+or redacted text is never sent to the embedder, and hooks receive a record without an
+embedding. `observe` is different: the conversation goes to the completion provider, and
+extracted facts are embedded for consolidation before hooks and policy run.
+
 Write-content filtering and transformation are not deletion authorization. A consolidation
 `forget` decision still validates scope, target existence, candidate integrity, and provenance, but it does not run
 `before_persist` or the policy's `call` method; this allows secret, transient, and redacted
@@ -396,6 +404,47 @@ implementing `allow_destructive?(record)` and returning exactly `true` or `false
 that only implement `call` affect writes but do not prevent deletion. The built-in policy uses
 this separate hook to retain its ungrounded-provenance protection.
 
+### Listing memories
+
+`memory.all` returns every memory in the scope, including expired ones. For large scopes, read
+one page at a time or iterate in batches:
+
+```ruby
+page = memory.all(limit: 50)
+next_page = memory.all(limit: 50, after_id: page.last.id)
+
+memory.each_memory(batch_size: 500) { |record| export(record) }
+```
+
+Both return memories in id order. A custom store whose `all` does not accept `limit:` and
+`after_id:` is read in a single call.
+
+### Correcting memories
+
+```ruby
+memory.update(id: record.id, content: "Lives in Berlin")
+memory.update(id: record.id, importance: 0.2, expires_at: nil)
+```
+
+`update` changes only the attributes you pass (`content`, `kind`, `importance`, `metadata`,
+`expires_at`) and keeps the id, `created_at`, and `last_accessed_at`. `metadata:` replaces the
+application metadata; `{}` clears it, and the reserved `_engram` key is not accepted.
+`expires_at: nil` removes the expiry. Other attributes do not accept `nil`.
+
+The edit goes through `before_persist` and the persistence policy like a new memory, and a
+policy that implements `allow_destructive?` must also allow replacing the original record.
+Expired memories can be updated too. `update` returns the stored record, `nil` if the hook or policy rejects it (the stored memory is
+left unchanged), and raises `Engram::MemoryNotFoundError` if the id is not in this scope.
+The embedding is recomputed when the content changes, or generated if the stored memory has
+no embedding.
+
+Changing the content removes the record's provenance, because the old sources no longer
+support the new text. Other edits keep it.
+
+`update` reads the record, applies the changes, and writes it back without conflict
+detection. If `observe`, an embedding rebuild, or another `update` writes the same memory at
+the same time, one of the changes can be lost.
+
 ### Extraction provenance
 
 Custom extractors remain compatible when they return an array of plain `Engram::Record`
@@ -553,6 +602,108 @@ Both settings default to nil. Pass nil explicitly to override a configured value
 the cap. `limit: 0` skips embedding and search; `max_bytes: 0` skips recall during injection.
 Invalid values raise `ArgumentError`.
 
+### Memory expiry
+
+Set a deadline for temporary memories:
+
+```ruby
+user.memory.add("Trial access is active", expires_at: 7.days.from_now)
+```
+
+`expires_at` accepts a Time (including Rails time-zone values) or nil. Nil is the default
+and never expires. A memory is expired at `expires_at <= Time.now`. Both built-in stores
+exclude expired records before the search limit, so they no longer participate in recall
+or consolidation. Injection checks expiry again when rendering.
+
+Expiry does not delete rows. `memory.all` still returns them, and `memory.forget(id:)` can
+delete them. A new observation may recreate the same fact from retained source data.
+Custom extractors or `before_persist` can set expiry with `record.with(expires_at: deadline)`;
+Engram does not infer deadlines from conversation text. Store updates replace expiry with
+the supplied record's value, including nil.
+
+New Rails installs include the expiry column and index. For a table created by 0.7.0 or
+earlier, generate and run the upgrade migration:
+
+```sh
+bin/rails generate engram:expiry
+bin/rails db:migrate
+```
+
+The migration adds a nullable datetime column and builds a partial index on scope and
+expiry concurrently. Existing rows keep their data and do not expire. Reads and
+non-expiring writes continue to work before migration; expiring writes raise an error
+instead of losing the deadline. Restart app processes after migrating so ActiveRecord
+refreshes its cached schema. If you use a custom table, adjust the generated migration.
+
+Custom stores must persist `expires_at` and exclude expired records before limiting search
+results. Recall also discards any expired records a custom store returns, but cannot fill
+the gaps left by a store that filters after its limit.
+
+To remove expired rows, preview the count, then run cleanup:
+
+```ruby
+user.memory.forget_expired(dry_run: true)
+# => {matched: 12, deleted: 0, dry_run: true}
+user.memory.forget_expired(batch_size: 500)
+# => {matched: 12, deleted: 12, dry_run: false}
+```
+
+The Rails task requires a scope:
+
+```sh
+DRY_RUN=true BATCH_SIZE=500 bin/rails 'engram:forget_expired[user:42]'
+BATCH_SIZE=500 bin/rails 'engram:forget_expired[user:42]'
+```
+
+Cleanup uses one cutoff for the run and reads only IDs in batches. It rechecks scope and
+expiry when deleting, so a concurrent extension or removal of the deadline is preserved.
+`matched` can exceed `deleted` if records change during the run; dry-run counts are advisory.
+Like `forget(id:)`, this is explicit deletion and bypasses persistence hooks and policies.
+Errors stop the run; completed batches stay deleted and a retry can process the remainder.
+Custom stores need the optional `expired_ids` and `delete_expired` methods for cleanup.
+The `forget_expired.engram` event reports counts without memory content or IDs.
+
+### Erasing or exporting a user's memory
+
+To delete everything Engram stores for one owner, preview the count, then run it:
+
+```ruby
+user.memory.forget_all(dry_run: true)
+# => {matched: 87, deleted: 0, dry_run: true}
+user.memory.forget_all(batch_size: 500)
+# => {matched: 87, deleted: 87, dry_run: false}
+```
+
+```sh
+DRY_RUN=true bin/rails 'engram:forget_all[user:42]'
+bin/rails 'engram:forget_all[user:42]'
+```
+
+`forget_all` deletes expired memories too and, like `forget(id:)`, bypasses hooks and policies.
+It only removes rows in the memory store. Before running it for an erasure request:
+
+1. Stop new `observe` and `observe_later` calls for the owner, and let queued `ObserveJob`s
+   finish or remove them. Otherwise they can write new memories during or after the erasure.
+2. Run `forget_all`. Memories added while it runs may or may not be deleted, so check
+   `memory.all` afterwards.
+3. Delete conversation transcripts and other source data your app keeps; Engram does not
+   store them. Processed-turn markers in the cache expire on their own and hold no content.
+
+To export, iterate in batches and write the fields you need:
+
+```ruby
+File.open("memories.jsonl", "w") do |file|
+  user.memory.each_memory(batch_size: 500) do |record|
+    file.puts({id: record.id, kind: record.kind, content: record.content,
+      created_at: record.created_at, expires_at: record.expires_at}.to_json)
+  end
+end
+```
+
+Custom stores can implement the optional `delete_many(scope:, ids:)` to delete a batch in one
+call; without it, `forget_all` deletes one id at a time. The `forget_all.engram` event reports
+counts without memory content or IDs.
+
 ### Observation and maintenance
 
 Observation uses a scope-and-turn claim before extraction. While a claim lease is live and
@@ -639,7 +790,10 @@ When ActiveSupport is loaded, Engram emits `ActiveSupport::Notifications` events
 main memory pipeline:
 
 - `add.engram`
+- `update.engram`
 - `forget.engram`
+- `forget_expired.engram`
+- `forget_all.engram`
 - `recall.engram`
 - `inject.engram`
 - `observe.engram`
@@ -785,6 +939,8 @@ gem unpack engram-*.gem --target /tmp/engram-package-check
 - v0.5 (done): embedding provenance and scoped embedding rebuild operations.
 - v0.6 (done): structured provenance, source impact lookup, and grounding reports.
 - v0.7 (done): recall thresholds, injection limits, scoped deletion, and RubyLLM streaming fixes.
+- v0.8 (done): explicit memory expiry, Rails upgrade migration, and scoped cleanup with dry runs.
+- v0.9 (done): memory correction, paged listing and batched iteration, and scope erasure.
 - later: additional storage backends and larger real-provider eval benchmarks.
 
 ## License

data/lib/engram/adapters/in_memory_store.rb

@@ -10,23 +10,26 @@ module Engram
       def initialize
         @records = {}
         @sequence = 0
+        @mutex = Mutex.new
       end
 
       def add(record)
         validate_scope!(record.scope)
 
-        record.id = (@sequence += 1)
-        @records[record.id] = record
-        record
+        @mutex.synchronize do
+          record.id = (@sequence += 1)
+          @records[record.id] = record
+          record
+        end
       end
 
       def search(embedding:, scope:, limit:, kinds: nil, embedding_metadata: nil)
         Engram::EmbeddingMetadata.validate_query!(embedding, embedding_metadata)
         allowed_kinds = normalize_kinds(kinds)
+        now = Time.now
 
-        results = @records
-          .values
-          .select { |r| searchable?(r, scope, allowed_kinds) }
+        results = @mutex.synchronize { @records.values }
+          .select { |r| searchable?(r, scope, allowed_kinds) && !r.expired?(at: now) }
           .map { |r| [r, Engram::Math.cosine_similarity(embedding, r.embedding)] }
           .sort_by { |(_, score)| -score }
           .first(limit)
@@ -37,7 +40,7 @@ module Engram
       end
 
       def all(scope:, limit: nil, offset: 0, after_id: nil)
-        records = @records.values.select { |r| r.scope == scope }.sort_by { |record| record.id }
+        records = @mutex.synchronize { @records.values }.select { |r| r.scope == scope }.sort_by { |record| record.id }
         records = records.drop_while { |record| !after_id.nil? && record.id && record.id <= after_id }
         records = records.drop(offset) if offset > 0
         records = records.take(limit) if limit
@@ -45,36 +48,82 @@ module Engram
       end
 
       def existing_ids(scope:, ids:)
-        ids.uniq.select { |id| @records[id]&.scope == scope }
+        @mutex.synchronize { ids.uniq.select { |id| @records[id]&.scope == scope } }
+      end
+
+      def find(scope:, id:)
+        @mutex.synchronize do
+          record = @records[id]
+          record if record&.scope == scope
+        end
       end
 
       def update(scope:, id:, record:)
-        existing = @records[id]
-        raise Engram::Error, "no memory with id #{id.inspect} in scope #{scope.inspect}" unless existing&.scope == scope
-        raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
+        @mutex.synchronize do
+          existing = @records[id]
+          raise Engram::Error, "no memory with id #{id.inspect} in scope #{scope.inspect}" unless existing&.scope == scope
+          raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
 
-        record.id = id
-        @records[id] = record
+          @records[id] = record.with(id: id, created_at: existing.created_at, last_accessed_at: existing.last_accessed_at)
+        end
       end
 
       def delete(scope:, id:)
-        return 0 unless @records[id]&.scope == scope
+        @mutex.synchronize do
+          return 0 unless @records[id]&.scope == scope
 
-        @records.delete(id)
-        1
+          @records.delete(id)
+          1
+        end
       end
 
       def touch(scope:, id:, at: Time.now)
-        record = @records[id]
-        return 0 unless record&.scope == scope
+        @mutex.synchronize do
+          record = @records[id]
+          return 0 unless record&.scope == scope
+
+          record.last_accessed_at = at
+          1
+        end
+      end
 
-        record.last_accessed_at = at
-        1
+      def expired_ids(scope:, at:, limit:, after_id: nil)
+        # IDs follow insertion order; the hash retains it across updates/deletes.
+        @mutex.synchronize do
+          @records.each_value.lazy
+            .select { |record| record.scope == scope && (after_id.nil? || record.id > after_id) && record.expired?(at: at) }
+            .map(&:id).take(limit).force
+        end
+      end
+
+      def delete_many(scope:, ids:)
+        @mutex.synchronize do
+          ids.uniq.count do |id|
+            next false unless @records[id]&.scope == scope
+
+            @records.delete(id)
+            true
+          end
+        end
+      end
+
+      def delete_expired(scope:, ids:, at:)
+        @mutex.synchronize do
+          ids.uniq.count do |id|
+            record = @records[id]
+            next false unless record&.scope == scope && record.expired?(at: at)
+
+            @records.delete(id)
+            true
+          end
+        end
       end
 
       def clear
-        @records.clear
-        @sequence = 0
+        @mutex.synchronize do
+          @records.clear
+          @sequence = 0
+        end
       end
 
       private

data/lib/engram/adapters/pgvector_store.rb

@@ -18,6 +18,7 @@ module Engram
         validate_scope!(record.scope)
 
         row = model.create!(
+          **expiry_attributes(record),
           content: record.content,
           scope: record.scope,
           kind: record.kind.to_s,
@@ -31,6 +32,10 @@ module Engram
       def search(embedding:, scope:, limit:, kinds: nil, embedding_metadata: nil)
         Engram::EmbeddingMetadata.validate_query!(embedding, embedding_metadata)
         query = model.where(scope: scope)
+        if expiry_column?
+          expiry = model.arel_table[:expires_at]
+          query = query.where(expiry.eq(nil).or(expiry.gt(Time.now)))
+        end
         normalized_kinds = normalize_kinds(kinds)
         query = query.where(kind: normalized_kinds) if normalized_kinds
 
@@ -64,12 +69,18 @@ module Engram
         end
       end
 
+      def find(scope:, id:)
+        row = model.find_by(scope: scope, id: id)
+        to_record(row) if row
+      end
+
       def update(scope:, id:, record:)
         raise Engram::Error, "cannot move memory across scopes" unless record.scope == scope
 
         model.transaction do
           row = model.lock.find_by!(id: id, scope: scope)
           row.update!(
+            **expiry_attributes(record),
             content: record.content,
             kind: record.kind.to_s,
             importance: record.importance,
@@ -90,8 +101,43 @@ module Engram
         model.where(id: id, scope: scope).update_all(last_accessed_at: at)
       end
 
+      def expired_ids(scope:, at:, limit:, after_id: nil)
+        return [] unless expiry_column?
+
+        query = expired_scope(scope, at)
+        query = query.where(model.arel_table[:id].gt(after_id)) unless after_id.nil?
+        query.order(:id).limit(limit).pluck(:id)
+      end
+
+      def delete_many(scope:, ids:)
+        return 0 if ids.empty?
+
+        model.where(scope: scope, id: ids).delete_all
+      end
+
+      def delete_expired(scope:, ids:, at:)
+        return 0 if ids.empty? || !expiry_column?
+
+        expired_scope(scope, at).where(id: ids).delete_all
+      end
+
       private
 
+      def expired_scope(scope, at)
+        model.where(scope: scope).where(model.arel_table[:expires_at].lteq(at))
+      end
+
+      def expiry_column?
+        model.column_names.include?("expires_at")
+      end
+
+      def expiry_attributes(record)
+        return {expires_at: record.expires_at} if expiry_column?
+        return {} if record.expires_at.nil?
+
+        raise Engram::Error, "PgvectorStore needs a nullable expires_at datetime column to store expiring memories"
+      end
+
       def validate_scope!(scope)
         raise Engram::Error, "memory scope cannot be nil" if scope.nil?
       end
@@ -124,7 +170,8 @@ module Engram
           importance: row.importance || 1.0,
           metadata: row.metadata || {},
           created_at: row.created_at,
-          last_accessed_at: row.try(:last_accessed_at)
+          last_accessed_at: row.try(:last_accessed_at),
+          expires_at: row.try(:expires_at)
         )
       end
 

data/lib/engram/internal/candidate_integrity.rb

@@ -29,7 +29,8 @@ module Engram
         kind: [Symbol],
         importance: [Integer, Float],
         created_at: [Time],
-        last_accessed_at: [NilClass, Time]
+        last_accessed_at: [NilClass, Time],
+        expires_at: [NilClass, Time]
       }.freeze
 
       Snapshot = Struct.new(:collection, :collection_state, :records)

data/lib/engram/memory.rb

@@ -4,6 +4,9 @@ module Engram
   # The friendly facade. Bound to one `scope` (an owner), it wires the configured store
   # and embedder into the use cases. This is what `user.memory` returns in Rails.
   class Memory
+    UNSET = Object.new.freeze
+    private_constant :UNSET
+
     attr_reader :scope
 
     def initialize(scope:, store: Engram.config.store, embedder: Engram.config.embedder)
@@ -14,21 +17,26 @@ module Engram
 
     # Persist a memory record of the given kind. Returns nil when the configured
     # persistence policy rejects the record.
-    def add(content, kind: :fact, importance: 1.0, metadata: {})
+    def add(content, kind: :fact, importance: 1.0, metadata: {}, expires_at: nil)
       Engram::Instrumentation.instrument("add", Engram::Instrumentation.payload(scope: scope, store: @store, kind: kind)) do
-        embedding = @embedder.embed(content)
-        record = Record.new(
+        persist(Record.new(
           content: content,
           scope: scope,
-          embedding: embedding,
           kind: kind,
           importance: importance,
-          metadata: metadata
-        )
-        persist(record)
+          metadata: metadata,
+          expires_at: expires_at
+        ))
       end
     end
 
+    # Correct supplied attributes of a memory in this scope. Omitted fields are kept.
+    def update(id:, content: UNSET, kind: UNSET, importance: UNSET, metadata: UNSET, expires_at: UNSET)
+      attributes = {content: content, kind: kind, importance: importance, metadata: metadata, expires_at: expires_at}
+        .reject { |_, value| value.equal?(UNSET) }
+      UseCases::Update.new(store: @store, embedder: @embedder).call(scope: scope, id: id, attributes: attributes)
+    end
+
     # Return the most relevant memories for a query.
     def recall(query, limit: Engram.config.default_limit, kinds: nil,
       min_similarity: Engram.config.recall_min_similarity)
@@ -85,15 +93,35 @@ module Engram
       end
     end
 
-    def all
-      @store.all(scope: scope)
+    # Memories in this scope, including expired ones. Pass limit and after_id (the last id of
+    # the previous page) to read one page in id order.
+    def all(limit: nil, after_id: nil)
+      return @store.all(scope: scope) if limit.nil? && after_id.nil?
+      if !limit.nil? && !(limit.is_a?(Integer) && limit.positive?)
+        raise ArgumentError, "limit must be a positive integer"
+      end
+      if !after_id.nil? && !valid_id?(after_id)
+        raise ArgumentError, "after_id must be an Integer or a non-empty String"
+      end
+
+      @store.all(scope: scope, limit: limit, after_id: after_id)
+    end
+
+    # Yield every memory in this scope in id order, loading batch_size records at a time.
+    # Returns an Enumerator without a block.
+    def each_memory(batch_size: 100, &block)
+      unless batch_size.is_a?(Integer) && batch_size.positive?
+        raise ArgumentError, "batch_size must be a positive integer"
+      end
+      return enum_for(:each_memory, batch_size: batch_size) unless block
+
+      UseCases::EachMemory.new(store: @store).call(scope: scope, batch_size: batch_size, &block)
     end
 
     # Delete one memory in this scope. Returns 1 when deleted, or 0 when missing.
     # Explicit deletion bypasses extraction, consolidation, and persistence hooks.
     def forget(id:)
-      valid_id = id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
-      raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id
+      raise ArgumentError, "id must be an Integer or a non-empty String" unless valid_id?(id)
 
       payload = Engram::Instrumentation.payload(scope: scope, store: @store)
       Engram::Instrumentation.instrument("forget", payload) do
@@ -101,6 +129,12 @@ module Engram
       end
     end
 
+    # Delete every memory in this scope in batches. Returns matched and deleted counts.
+    # Bypasses extraction, consolidation, and persistence hooks, like #forget.
+    def forget_all(batch_size: 100, dry_run: false)
+      UseCases::ForgetAll.new(store: @store).call(scope: scope, batch_size: batch_size, dry_run: dry_run)
+    end
+
     # Recompute embeddings (and embedding metadata) for memories in the scope.
     # When `stale_only` is true, only records whose current metadata does not match the
     # active embedder are rebuilt.
@@ -119,6 +153,13 @@ module Engram
         .call(scope: scope, older_than: older_than, min_importance: min_importance)
     end
 
+    # Delete expired records without loading their content or embeddings.
+    # Returns matched/deleted counts; dry runs only count eligible records.
+    def forget_expired(batch_size: 100, dry_run: false)
+      UseCases::ForgetExpired.new(store: @store)
+        .call(scope: scope, batch_size: batch_size, dry_run: dry_run)
+    end
+
     # Return the memories in this scope whose provenance references the exact host
     # source. `source_id` and `source_type` must each be non-blank Strings and are
     # matched exactly, without normalization. Source IDs are references, not an
@@ -136,6 +177,10 @@ module Engram
 
     private
 
+    def valid_id?(id)
+      id.instance_of?(Integer) || (id.instance_of?(String) && id.valid_encoding? && !id.strip.empty?)
+    end
+
     def persist(record)
       Persistence.new(store: @store, embedder: @embedder).add(record)
     end

data/lib/engram/persistence.rb

@@ -48,20 +48,30 @@ module Engram
     # Applies all write transformations without mutating the store. Input is validated
     # before callbacks can remove or replace provenance, and final output is validated too.
     def prepare(record)
-      original_provenance = validate_provenance!(record)
       original_content = record.content
+      record = transform(record)
+      # Embed only after hooks and policy so rejected or redacted text never reaches the embedder.
+      if record && (record.embedding.nil? || record.content != original_content)
+        record = record.with(embedding: @embedder.embed(record.content))
+      end
+      record = Engram::EmbeddingMetadata.attach(record, embedder: @embedder) if record
+      validate_provenance!(record) if record
+      record
+    end
+
+    # Runs before_persist and the policy without embedding. Memory#update passes a block
+    # that runs after each step to check the result and drop invalidated provenance.
+    def transform(record)
+      original_provenance = validate_provenance!(record)
       record = @before_persist.call(record) if @before_persist
       if record
         transformed_provenance = validate_provenance!(record)
         validate_provenance_trust!(original_provenance, transformed_provenance) if @before_persist
       end
+      record = yield(record) if record && block_given?
       record = @persistence_policy.call(record) if record && @persistence_policy
       validate_provenance!(record) if record
-      if record && record.content != original_content
-        record = record.with(embedding: @embedder.embed(record.content))
-      end
-      record = Engram::EmbeddingMetadata.attach(record, embedder: @embedder) if record
-      validate_provenance!(record) if record
+      record = yield(record) if record && block_given?
       record
     end