event_engine-store 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0888fb8cb32a3a92676f641ce96b85d56753a890bdecf9027c9521d3b642ec9c'
-  data.tar.gz: b488a04eba91857bcc65ae37bdb4cf0608d94a8f8c4bc6d3e42798810fd7a264
+  metadata.gz: 11e8057d3dfa1adb99b82eb9e6add21d9f5bfc3fd5a06895ba7e391d136e11ae
+  data.tar.gz: 60c1108aef5cae04c14b1b9f77baae03c98a0a8628bba364da51ad267359ee02
 SHA512:
-  metadata.gz: fa6fd5beff74cfcea49f0917bbda4f82448592a2c1893d093d5074c97dc41f4cd506c4536398e4b6fe0061e5397414f44e48ca9cf6c586d14fb95e26a5624bb7
-  data.tar.gz: 964a48580c13ddb2f1be6d0803e8d0ce26b229fe62d6521c0fdd17f82e228ea3485898a2b9b48e2b23cb505e3009e91feac87873f05530e7e2a7e8e249378c3e
+  metadata.gz: 5356e1efded2136e940c183b4f7dc84d02c7fe74929d4a60c81d3efeb961992b7de81818ee7c09576c0cec265e6a3a99c088f36d39e8587a819f0c3ac55789a6
+  data.tar.gz: 7823ee9ae1e1d3f2ded90f5c664a2e6996bded30620e8e4d52273814b211702b31f4fad868e1f5816b182601656fac4cbd0e59f9efdc4112bb4995cce50fa7e2

data/CHANGELOG.md

@@ -6,6 +6,16 @@ All notable changes to this project are documented here, following
 
 ## [Unreleased]
 
+## [0.1.1] - 2026-11-07
+
+### Added
+
+- The gem ships its expert agents, so an app that installs it can ask an event_engine-store expert how events are stored, replayed and built into projections.
+
+### Fixed
+
+- The stored events migration runs on Rails 7.1.
+
 ## [0.1.0] - 2026-07-29
 
 First published release of `event_engine-store`, the durable event record for the

data/README.md

@@ -67,14 +67,14 @@ itself up at boot. Once migrated, every dispatched event is recorded automatical
 
 ## How it hooks into the core
 
-At Rails boot the engine registers **two** handlers with the core, for all levels:
+At Rails boot the engine registers **two** handlers with the core, for every process type:
 
 ```ruby
 # lib/event_engine/store/engine.rb
 initializer "event_engine.store.register_recorder" do
   config.after_initialize do
-    EventEngine.register_handler(Recorder.new, levels: :all)
-    EventEngine.register_handler(ProjectionDispatcher.new, levels: :all)
+    EventEngine.register_handler(Recorder.new, process_types: :all)
+    EventEngine.register_handler(ProjectionDispatcher.new, process_types: :all)
   end
 end
 ```
@@ -85,9 +85,10 @@ So on every `EventEngine.<event>` call:
 2. `ProjectionDispatcher#call(event)` calls `apply(event)` on each registered
    projection.
 
-> Because both register at `levels: :all`, the store records **every** level —
-> including non-durable levels 1 and 2. That's by design: the store is your record of
-> what happened, independent of how it was delivered. If you only want to record some
+> Because both register at `process_types: :all`, the store records events of
+> **every** process type, whichever processor `config/event_rules.yml` routes them to.
+> That's by design: the store is your record of what happened, independent of how it
+> was processed. If you only want to record some
 > events, see [Recording only some events](#recording-only-some-events).
 
 ---
@@ -103,7 +104,7 @@ persisted records), so an attempt to update raises `ActiveRecord::ReadOnlyRecord
 | `event_name` | string | ✓ | NOT NULL |
 | `event_type` | string | | classification |
 | `event_version` | integer | | schema version |
-| `event_level` | integer | | dispatched level |
+| `process_type` | string | | processor the event was routed to |
 | `payload` | json | | event data |
 | `metadata` | json | | context (actor, request id, …) |
 | `occurred_at` | datetime | ✓ | logical event time |
@@ -120,7 +121,7 @@ create_table :event_engine_store_stored_events do |t|
   t.string   :event_name, null: false
   t.string   :event_type
   t.integer  :event_version
-  t.integer  :event_level
+  t.string   :process_type
   t.json     :payload
   t.json     :metadata
   t.datetime :occurred_at
@@ -212,7 +213,7 @@ module EventEngine
           event_name:        event.event_name,
           event_type:        event.event_type,
           event_version:     event.event_version,
-          event_level:       event.event_level,
+          process_type:      event.process_type,
           payload:           event.payload,
           metadata:          event.metadata,
           occurred_at:       event.occurred_at,
@@ -293,7 +294,7 @@ EventEngine::Store::Recorder.prepend(RecordDomainOnly)
 ```
 
 **Why:** keep the log focused on domain/audit-worthy events and avoid recording noisy
-level-1 system pings.
+system events.
 
 ### Replacing the recorder entirely
 
@@ -308,7 +309,7 @@ class MyAuditRecorder
 end
 
 Rails.application.config.after_initialize do
-  EventEngine.register_handler(MyAuditRecorder.new, levels: :all)
+  EventEngine.register_handler(MyAuditRecorder.new, process_types: :all)
 end
 ```
 
@@ -362,7 +363,7 @@ log in append order (ordered by `id`, batched via `find_each`):
 
 ```ruby
 EventEngine::Store::Replay.each do |event|
-  # event is a fully-rehydrated EventEngine::Event (symbol-keyed payload)
+  # event is an EventEngine::Event rebuilt from its row
   puts "#{event.occurred_at} #{event.event_name}"
 end
 
@@ -371,6 +372,13 @@ enum = EventEngine::Store::Replay.each
 enum.count
 ```
 
+A replayed event differs from the one that was dispatched:
+
+- `payload` and `metadata` have **string keys**, because they are read back from JSON
+  columns (`event.payload["weight"]`, not `event.payload[:weight]`).
+- `event_name` and `process_type` are **strings**, read back from string columns.
+- `subject` and `domain` are **nil**, because the table has no columns for them.
+
 Use replay to rebuild read models, backfill a new projection, or audit history.
 
 ---
@@ -386,7 +394,7 @@ class OrdersByDay
   attr_reader :counts
 
   def apply(event)
-    @counts[event.occurred_at.to_date] += 1 if event.event_name == :order_placed
+    @counts[event.occurred_at.to_date] += 1 if event.event_name.to_s == "order_placed"
   end
 end
 

data/Rakefile

@@ -7,3 +7,5 @@ require "bundler/gem_tasks"
 
 task test: "app:test"
 task default: :test
+
+require "the_local/rake"

data/db/migrate/20260605000001_create_event_engine_store_stored_events.rb

@@ -1,4 +1,4 @@
-class CreateEventEngineStoreStoredEvents < ActiveRecord::Migration[8.1]
+class CreateEventEngineStoreStoredEvents < ActiveRecord::Migration[7.1]
   def change
     create_table :event_engine_store_stored_events do |t|
       t.string :event_name, null: false

data/lib/event_engine/store/version.rb

@@ -1,5 +1,5 @@
 module EventEngine
   module Store
-    VERSION = "0.1.0"
+    VERSION = "0.1.1"
   end
 end

data/the_local/agents/event_engine-store-develop.md

@@ -0,0 +1,165 @@
+---
+name: event_engine-store-develop
+description: Use PROACTIVELY for reading an app's recorded event history on event_engine-store — querying the stored-events log, replaying it in order, writing and registering projections, rebuilding a read model from the log, and changing which events get recorded — MUST BE USED instead of hand-rolling an audit table, an event-history query, or a script that recomputes a read model from scratch.
+tools: Read, Write, Edit, Grep
+scope: event store — recording every dispatched event to an append-only table, replaying the log in order, and rebuilding projections from it
+---
+
+This local builds on a host app's event record with event_engine-store, following
+the steps below in order. Where a step names a decision, it asks the developer
+instead of choosing.
+
+## What event_engine-store is
+
+A Rails engine that writes every event event_engine dispatches to an append-only
+table, and reads that table back as events. Fire this local when an app needs its
+event history queried, a read model kept up to date from events, a read model
+rebuilt or backfilled from past events, or a change to what the log records.
+
+## Interface
+
+- `EventEngine::Store::StoredEvent` — the ActiveRecord model for the log, one row
+  per dispatched event. Query it like any model; a saved row cannot be changed.
+- `EventEngine::Store::Replay.each` — yields every recorded event, oldest first,
+  as an event object. Without a block it returns an Enumerator.
+- `EventEngine::Store.register_projection` — adds a projection that receives
+  every event as it is dispatched.
+- `EventEngine::Store.rebuild` — replays the whole log into one projection.
+- `EventEngine::Store.projections` — the list of registered projections.
+- `EventEngine::Store.reset_projections!` — removes every registered projection.
+- `EventEngine::Store::Recorder` — the handler that writes each dispatched event
+  to the log. Prepend a module to it to change what is recorded.
+
+## How to use it
+
+The `event_engine_store_stored_events` table must exist in the host app. If it
+does not, hand off to `event_engine-store-install` first and come back.
+
+1. Query the log with plain ActiveRecord on `EventEngine::Store::StoredEvent`. Its
+   columns are `event_name`, `event_type`, `event_version`, `process_type`,
+   `payload`, `metadata`, `occurred_at`, `idempotency_key`, `aggregate_type`,
+   `aggregate_id`, `aggregate_version`, and `created_at`. `event_name`,
+   `occurred_at`, and `idempotency_key` are indexed.
+
+   ```ruby
+   EventEngine::Store::StoredEvent.where(aggregate_type: "Order", aggregate_id: order.id.to_s).order(:id)
+   ```
+
+   The model has no scopes. To add some, decorate it in an initializer inside
+   `Rails.application.config.to_prepare`, so the decoration is reapplied when
+   Rails reloads the model:
+
+   ```ruby
+   Rails.application.config.to_prepare do
+     EventEngine::Store::StoredEvent.class_eval do
+       scope :named, ->(name) { where(event_name: name.to_s) }
+     end
+   end
+   ```
+
+2. Replay the log when you need events back as event objects rather than rows:
+
+   ```ruby
+   EventEngine::Store::Replay.each { |event| puts "#{event.occurred_at} #{event.event_name}" }
+   ```
+
+   Events come back in insertion order, loaded in batches. The Enumerator returned
+   without a block runs its query each time it is iterated, so it sees rows
+   written after it was created.
+
+3. Write a projection. A projection is any object that responds to
+   `apply(event)`. Ask the developer what read model it maintains and where that
+   state lives (memory, a table, a cache) — there is no default.
+
+   ```ruby
+   class OrdersPerDay
+     attr_reader :counts
+
+     def initialize
+       @counts = Hash.new(0)
+     end
+
+     def apply(event)
+       @counts[event.occurred_at.to_date] += 1 if event.event_name.to_s == "order_placed"
+     end
+   end
+   ```
+
+4. Register it for live updates, once per process, from an initializer:
+
+   ```ruby
+   Rails.application.config.after_initialize do
+     EventEngine::Store.register_projection(OrdersPerDay.new)
+   end
+   ```
+
+   From then on it receives every dispatched event, of every process type, after
+   the event has been written to the log. Registered projections run in
+   registration order.
+
+5. Rebuild when the projection's state must reflect history — a new projection, a
+   changed `apply`, or lost state:
+
+   ```ruby
+   EventEngine::Store.rebuild(projection)
+   ```
+
+   It calls `apply` once per recorded event, oldest first. It does not clear the
+   projection first, and the projection does not need to be registered. Ask the
+   developer whether rebuilding should start from a fresh projection or cleared
+   state, and whether it can run while events are still being emitted — an event
+   emitted during a rebuild of a registered projection can reach it twice.
+
+6. In tests, call `EventEngine::Store.reset_projections!` in teardown after
+   registering a projection, and read `EventEngine::Store.projections` to assert
+   on what is registered.
+
+7. Change what is recorded only if the developer asks for it, by prepending a
+   module to `EventEngine::Store::Recorder` from an initializer. Its `call(event)`
+   returns the created row. Ask which change they want:
+
+   - Record a subset — return early for events to skip, otherwise call `super`.
+   - Never let recording break an emit — rescue around `super`, log, return
+     `nil`. A failed write is then lost unless it is retried.
+   - Record extra columns — add them with a host migration, then have `call`
+     create the row itself with the default attributes plus the new ones. The
+     default recorder writes only its fixed attributes, so new columns stay
+     `NULL` otherwise.
+   - Stop recording entirely — define `call(_event)` with an empty body.
+     Registered projections still run.
+
+   ```ruby
+   module RecordOrdersOnly
+     def call(event)
+       return unless event.event_name.to_s.start_with?("order_")
+
+       super
+     end
+   end
+
+   EventEngine::Store::Recorder.prepend(RecordOrdersOnly)
+   ```
+
+## Conventions
+
+- Never update or destroy a saved `StoredEvent`. `update` and `destroy` on a saved
+  row raise `ActiveRecord::ReadOnlyRecord`. Correct history by emitting a new
+  event.
+- Recording and projections run synchronously inside the emit call. A database
+  error while recording, or an exception raised by `apply`, propagates to the
+  code that emitted the event and stops later handlers and projections. Keep
+  `apply` fast; enqueue a job for heavy work.
+- A replayed event differs from a live one. `event_name` comes back as a string,
+  `payload` and `metadata` come back with string keys, and `subject` and `domain`
+  are `nil` because they are not stored. Write `apply` to accept both: compare
+  `event_name.to_s`, and read payload keys as strings or with indifferent access.
+- `register_projection` keeps a plain in-memory list. Registering the same
+  projection twice applies every event to it twice, and each process registers
+  its own.
+- The idempotency key is not unique, so the same event emitted twice is recorded
+  twice. A projection that must count each event once has to deduplicate itself.
+- Do not call `EventEngine.reset_handlers!` to stop recording. It removes every
+  handler, including the one that feeds registered projections.
+- Out of scope for this local: adding the gem and creating the table
+  (`event_engine-store-install`), and defining, routing, or emitting events
+  (`event_engine-develop`).

data/the_local/agents/event_engine-store-info.md

@@ -0,0 +1,62 @@
+---
+name: event_engine-store-info
+description: Use to learn what event_engine-store offers — the permanent append-only record of every dispatched event, replay of that record in order, and projections kept live and rebuilt from it.
+tools: Read
+scope: event store — recording every dispatched event to an append-only table, replaying the log in order, and rebuilding projections from it
+---
+
+This local explains event_engine-store and the vocabulary it uses. It changes
+nothing and gives no steps — read it to orient, then go to the local that owns the
+work.
+
+## What event_engine-store is
+
+event_engine-store is a Rails engine that adds a permanent record to an app built
+on event_engine. Once installed, every event event_engine dispatches is written as
+one row to a table the host app owns, whatever its process type. Rows are never
+updated: the table is the history of what happened, in the order it happened.
+
+Reach for it when an app needs to answer "what happened, and when" from a durable
+log, rebuild a read model (counters, dashboards, search indexes) from history, or
+backfill a new read model from events recorded before it existed. It does not
+aggregate or compute metrics, and it does not deliver events anywhere — it records
+them and reads them back.
+
+## Interface
+
+This local documents no commands — it is background only.
+
+The `event_engine-store-install` local owns adding the gem to a host app and
+creating its table. The `event_engine-store-develop` local owns everything an app
+does with the record: querying it, replaying it, registering and rebuilding
+projections, and changing what gets recorded.
+
+## How to use it
+
+One decision: is the store already in the app?
+
+- Adding the gem to an app for the first time, or the table is missing →
+  `event_engine-store-install`.
+- Querying the log, replaying it, writing a projection, or changing what is
+  recorded → `event_engine-store-develop`.
+
+If you only needed the vocabulary, you have it — stop here.
+
+## Conventions
+
+- **Stored event** — one row in the log, holding one dispatched event's name,
+  type, version, process type, payload, metadata, occurred-at time, idempotency
+  key, and aggregate type / id / version. The event's subject and domain are not
+  stored.
+- **Append-only** — a row is written once and never changed. Order is the row's
+  insertion order.
+- **Recording** — happens automatically for every dispatched event, inside the
+  emit call, with no configuration.
+- **Replay** — reading the log back from the first row to the last, as event
+  objects.
+- **Projection** — an object with an `apply(event)` method that builds a read
+  model from events. A registered projection receives each event as it is
+  dispatched; any projection can be rebuilt by replaying the whole log into it.
+- **Companion gems** — event_engine (required) defines and dispatches events.
+  event_engine-delivery keeps a temporary outbox for delivery; the two can run
+  side by side.

data/the_local/agents/event_engine-store-install.md

@@ -0,0 +1,94 @@
+---
+name: event_engine-store-install
+description: Use to hook event_engine-store into a Rails app — adding the gem, copying and running its stored-events migration, and confirming dispatched events are being recorded.
+tools: Bash, Read, Edit
+scope: event store — recording every dispatched event to an append-only table, replaying the log in order, and rebuilding projections from it
+---
+
+This local follows the steps below exactly, in order, and invents none. Where a
+step names a decision, it asks the developer instead of choosing.
+
+## What event_engine-store is
+
+A Rails engine that records every event event_engine dispatches into an
+append-only table in the host app — hook it in when an app needs a permanent,
+replayable log of its events.
+
+## Interface
+
+- `bin/rails event_engine_store:install:migrations` — copies the migration that creates the
+  `event_engine_store_stored_events` table into the host app's `db/migrate/`.
+
+## How to use it
+
+1. Confirm event_engine is already installed in the host app — its initializer
+   exists and the app boots with its schema catalog. If not, hand off to the
+   `event_engine-install` local first and come back. This gem requires
+   `event_engine` 0.2.1 or later.
+
+2. Add the gem to the host app's `Gemfile`:
+
+   ```ruby
+   gem "event_engine-store"
+   ```
+
+   Ask the developer whether to take the released gem, pin a version, or track the
+   source repository (`github: "DYB-Development/event_engine-store"`) — do not
+   pick for them. The host must be a Rails app; this gem is a Rails engine.
+
+3. Install it:
+
+   ```bash
+   bundle install
+   ```
+
+4. Copy the migration into the host app:
+
+   ```bash
+   bin/rails event_engine_store:install:migrations
+   ```
+
+   It writes one new file under `db/migrate/` that creates
+   `event_engine_store_stored_events`. The command copies pending migrations from
+   every installed engine, not only this one — show the developer the list of new
+   files before continuing.
+
+5. Check the host's Rails version. The copied migration is declared as
+   `ActiveRecord::Migration[8.1]`, which Rails versions before 8.1 reject. If the
+   host runs an older Rails, ask the developer whether to change the bracketed
+   version in the copied file to the host's own Rails version — do not change it
+   without asking.
+
+6. Run the migration and commit both the migration and the updated schema file:
+
+   ```bash
+   bin/rails db:migrate
+   ```
+
+7. Verify the table exists:
+
+   ```bash
+   bin/rails runner 'p ActiveRecord::Base.connection.table_exists?("event_engine_store_stored_events")'
+   ```
+
+   `true` means the install is done. `false` means step 6 did not run against
+   this environment's database.
+
+8. Verify recording. In a Rails console, note
+   `ActiveRecord::Base.connection.select_value("SELECT COUNT(*) FROM event_engine_store_stored_events")`,
+   emit any catalogued event the app already emits, and read the count again. It
+   grows by one per dispatched event.
+
+## Conventions
+
+- There is no generator, no initializer, and no configuration. Once the gem is in
+  the bundle and the table exists, every dispatched event is recorded at boot
+  without any host code.
+- Recording is synchronous and inside the emit call, so an emit fails if the
+  table is missing. Do not deploy the gem to an environment before its migration
+  has run there.
+- The table has no foreign keys and no unique index. The idempotency key is
+  indexed but not unique, so a duplicate emit is recorded twice.
+- Re-running step 4 copies nothing that is already in `db/migrate/`.
+- Out of scope for this local: querying or replaying the log, projections, and
+  changing what gets recorded. All of that belongs to `event_engine-store-develop`.

data/the_local/interface.yml

@@ -0,0 +1,22 @@
+scope: event store — recording every dispatched event to an append-only table, replaying the log in order, and rebuilding projections from it
+
+install:
+  - bin/rails event_engine_store:install:migrations
+
+develop:
+  - EventEngine::Store::StoredEvent
+  - EventEngine::Store::Replay.each
+  - EventEngine::Store.register_projection
+  - EventEngine::Store.rebuild
+  - EventEngine::Store.projections
+  - EventEngine::Store.reset_projections!
+  - EventEngine::Store::Recorder
+
+sources:
+  - lib/event_engine/store.rb
+  - lib/event_engine/store/engine.rb
+  - lib/event_engine/store/recorder.rb
+  - lib/event_engine/store/replay.rb
+  - lib/event_engine/store/projection_dispatcher.rb
+  - app/models/event_engine/store/stored_event.rb
+  - db/migrate/20260605000001_create_event_engine_store_stored_events.rb

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: event_engine-store
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - tylercschneider
@@ -103,6 +103,10 @@ files:
 - lib/event_engine/store/replay.rb
 - lib/event_engine/store/version.rb
 - lib/tasks/event_engine/store_tasks.rake
+- the_local/agents/event_engine-store-develop.md
+- the_local/agents/event_engine-store-info.md
+- the_local/agents/event_engine-store-install.md
+- the_local/interface.yml
 homepage: https://github.com/DYB-Development/event_engine-store
 licenses:
 - MIT