solid_objects 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +23 -0
  3. data/README.md +223 -15
  4. data/benchmark/adoption_latency.rb +5 -0
  5. data/benchmark/support.rb +35 -0
  6. data/docs/architecture.md +62 -15
  7. data/docs/authorization.md +98 -0
  8. data/docs/benchmarks.md +75 -1
  9. data/docs/correctness.md +19 -1
  10. data/docs/database-schema.md +5 -0
  11. data/docs/development.md +45 -4
  12. data/docs/fit.md +98 -0
  13. data/docs/migrating-existing-state.md +140 -0
  14. data/docs/operations.md +94 -4
  15. data/docs/roadmap.md +12 -3
  16. data/docs/security.md +28 -3
  17. data/docs/state-migrations.md +6 -0
  18. data/lib/generators/solid_objects/templates/solid_objects.rb +45 -0
  19. data/lib/solid_objects/activation.rb +33 -4
  20. data/lib/solid_objects/actor.rb +47 -6
  21. data/lib/solid_objects/actor_definition.rb +2 -0
  22. data/lib/solid_objects/actor_snapshot.rb +10 -4
  23. data/lib/solid_objects/application_write_guard.rb +24 -0
  24. data/lib/solid_objects/caller_process.rb +28 -0
  25. data/lib/solid_objects/cli.rb +44 -5
  26. data/lib/solid_objects/client.rb +98 -5
  27. data/lib/solid_objects/commit_action_registry.rb +42 -0
  28. data/lib/solid_objects/configuration.rb +27 -1
  29. data/lib/solid_objects/database_adapter.rb +28 -1
  30. data/lib/solid_objects/database_adapters/mysql.rb +46 -0
  31. data/lib/solid_objects/database_adapters/postgresql.rb +29 -0
  32. data/lib/solid_objects/database_adapters/sqlite.rb +28 -0
  33. data/lib/solid_objects/doctor.rb +311 -0
  34. data/lib/solid_objects/errors.rb +144 -0
  35. data/lib/solid_objects/executor.rb +64 -4
  36. data/lib/solid_objects/instance_pruner.rb +97 -0
  37. data/lib/solid_objects/message_pruner.rb +97 -0
  38. data/lib/solid_objects/message_reference.rb +9 -0
  39. data/lib/solid_objects/process_pruner.rb +49 -0
  40. data/lib/solid_objects/reference.rb +5 -0
  41. data/lib/solid_objects/state_snapshot.rb +41 -0
  42. data/lib/solid_objects/sync_deadline.rb +57 -0
  43. data/lib/solid_objects/sync_diagnostics.rb +133 -0
  44. data/lib/solid_objects/synchronous_invocation.rb +26 -7
  45. data/lib/solid_objects/test_helper.rb +78 -0
  46. data/lib/solid_objects/version.rb +1 -1
  47. data/lib/solid_objects/worker.rb +1 -1
  48. data/lib/solid_objects.rb +34 -0
  49. data/lib/tasks/solid_objects_tasks.rake +10 -0
  50. data/sig/generated/lib/solid_objects/activation.rbs +3 -0
  51. data/sig/generated/lib/solid_objects/actor.rbs +26 -0
  52. data/sig/generated/lib/solid_objects/application_write_guard.rbs +8 -0
  53. data/sig/generated/lib/solid_objects/caller_process.rbs +11 -0
  54. data/sig/generated/lib/solid_objects/cli.rbs +11 -2
  55. data/sig/generated/lib/solid_objects/client.rbs +15 -0
  56. data/sig/generated/lib/solid_objects/commit_action_registry.rbs +43 -0
  57. data/sig/generated/lib/solid_objects/configuration.rbs +27 -7
  58. data/sig/generated/lib/solid_objects/database_adapter.rbs +9 -0
  59. data/sig/generated/lib/solid_objects/database_adapters/mysql.rbs +8 -0
  60. data/sig/generated/lib/solid_objects/database_adapters/postgresql.rbs +8 -0
  61. data/sig/generated/lib/solid_objects/database_adapters/sqlite.rbs +8 -0
  62. data/sig/generated/lib/solid_objects/doctor.rbs +111 -0
  63. data/sig/generated/lib/solid_objects/errors.rbs +118 -0
  64. data/sig/generated/lib/solid_objects/executor.rbs +12 -0
  65. data/sig/generated/lib/solid_objects/instance_pruner.rbs +36 -0
  66. data/sig/generated/lib/solid_objects/message_pruner.rbs +42 -0
  67. data/sig/generated/lib/solid_objects/message_reference.rbs +3 -0
  68. data/sig/generated/lib/solid_objects/process_pruner.rbs +27 -0
  69. data/sig/generated/lib/solid_objects/reference.rbs +3 -0
  70. data/sig/generated/lib/solid_objects/state_snapshot.rbs +30 -0
  71. data/sig/generated/lib/solid_objects/sync_deadline.rbs +31 -0
  72. data/sig/generated/lib/solid_objects/sync_diagnostics.rbs +34 -0
  73. data/sig/generated/lib/solid_objects/synchronous_invocation.rbs +3 -0
  74. data/sig/generated/lib/solid_objects/test_helper.rbs +25 -0
  75. data/sig/generated/lib/solid_objects.rbs +12 -0
  76. metadata +26 -1
@@ -0,0 +1,98 @@
1
+ # Authorization policies
2
+
3
+ Solid Objects treats actor identities as identifiers, never capabilities.
4
+ Knowing an actor ID, message ID, or signed stream token grants no permission.
5
+ All five policies deny by default, so a generated installation is
6
+ intentionally inert until the host application defines its trust boundary.
7
+
8
+ ## Policy reference
9
+
10
+ | Policy | Gates | Caller context | Risk if opened globally |
11
+ | --- | --- | --- | --- |
12
+ | `authorize_message` | Direct actor methods, explicit `sync` messages, and public `async` enqueue | Value passed as `authorization_context:`; often a user, service principal, or trusted internal marker | Anyone reaching the call site can mutate any known actor identity |
13
+ | `authorize_query` | Attribute reads, declared queries, committed snapshots, observable reads, and component reads | Explicit call context or the Rails view context supplied by `solid_object` | Actor state can leak across users or tenants |
14
+ | `authorize_destroy` | `reference.destroy` | Value passed as `authorization_context:` | Complete actor state, mailbox, reminders, and pending outboxes can be deleted |
15
+ | `authorize_subscription` | Action Cable subscription to one actor stream | The `ActionCable::Connection` object | Clients can receive future observable updates for other actors |
16
+ | `authorize_administration` | Engine administration controllers, process inspection/cleanup/pruning, message pruning, and dead-letter inspection/retry | Rails controller or `{ source: "cli" }` | Operational metadata, arguments, errors, deletion, and retries become exposed or mutable |
17
+
18
+ Waiting again through `MessageReference#wait` reauthorizes the stored
19
+ invocation as a message or query. Internal reminder, effect-callback, and
20
+ actor-to-actor deliveries come from
21
+ already committed runtime rows and do not re-enter the public client policy.
22
+
23
+ ## A tenant-aware policy
24
+
25
+ Pass the authenticated user as the call context:
26
+
27
+ ```ruby
28
+ cart = ShoppingCart.ref(Current.user.id)
29
+ cart.add_item(
30
+ product_id: "shirt-123",
31
+ authorization_context: Current.user
32
+ )
33
+ ```
34
+
35
+ Authorize only the matching user and actor type:
36
+
37
+ ```ruby
38
+ SolidObjects.configure do |configuration|
39
+ owns_actor = lambda do |actor_type:, actor_id:, authorization_context:, **|
40
+ user = authorization_context
41
+
42
+ actor_type == "ShoppingCart" &&
43
+ user.present? &&
44
+ actor_id == user.id.to_s
45
+ end
46
+
47
+ configuration.authorize_message = owns_actor
48
+ configuration.authorize_query = owns_actor
49
+ configuration.authorize_destroy = owns_actor
50
+
51
+ configuration.authorize_subscription = lambda do |actor_type:, actor_id:, authorization_context:|
52
+ connection = authorization_context
53
+
54
+ actor_type == "ShoppingCart" &&
55
+ connection.current_user.present? &&
56
+ actor_id == connection.current_user.id.to_s
57
+ end
58
+
59
+ configuration.authorize_administration = lambda do |authorization_context:, **|
60
+ context = authorization_context
61
+ user = context.respond_to?(:current_user) ? context.current_user : nil
62
+
63
+ user&.administrator?
64
+ end
65
+ end
66
+ ```
67
+
68
+ The policy receives normalized actor type and ID strings, message name and
69
+ arguments where relevant, and the context supplied by the caller. Avoid
70
+ authorizing from arguments alone; bind the actor identity to the authenticated
71
+ principal and tenant.
72
+
73
+ ## Server-side-only pilots
74
+
75
+ Allowing `authorize_message` and `authorize_query` unconditionally can be a
76
+ reasonable short-lived pilot only when every call site is trusted server code,
77
+ actor IDs cannot come from an unauthorized request, and the feature is not
78
+ exposed through Action Cable or administration routes.
79
+
80
+ Keep `authorize_destroy`, `authorize_subscription`, and
81
+ `authorize_administration` denied until each feature has an explicit policy.
82
+ Replace unconditional policies before exposing actor IDs to controllers, API
83
+ clients, MCP tools, jobs carrying user input, or browser subscriptions.
84
+
85
+ For commands executed only on hosts where shell access is already the
86
+ authenticated administration boundary, the generated initializer shows an
87
+ optional CLI-scoped policy:
88
+
89
+ ```ruby
90
+ configuration.authorize_administration = lambda do |authorization_context:, **|
91
+ authorization_context.is_a?(Hash) &&
92
+ authorization_context[:source] == "cli"
93
+ end
94
+ ```
95
+
96
+ Run `bin/rails solid_objects:doctor` after configuration. Its neutral policy
97
+ probe is deliberately conservative: a context-aware policy may correctly warn
98
+ because it denies a `nil` context.
data/docs/benchmarks.md CHANGED
@@ -1,10 +1,31 @@
1
- # Local benchmarks
1
+ # Performance and storage costs
2
2
 
3
3
  These numbers are development measurements, not universal capacity guarantees.
4
4
  They include the runtime's Active Record and database query overhead and will
5
5
  vary with hardware, schema size, connection pools, durability settings, and
6
6
  contention.
7
7
 
8
+ ## Production-shaped adoption measurement
9
+
10
+ An adoption evaluation measured Solid Objects 0.2.0 from a macOS Rails process
11
+ against Docker MySQL 8 over a published TCP port. The host used Rails 8.1, Ruby
12
+ 4.0.5, roughly 165 gems, and an approximately 2,200-line schema.
13
+
14
+ | Operation | Existing key-value row | Solid Objects |
15
+ | --- | ---: | ---: |
16
+ | Write | median 4.7 ms | median 60 ms, minimum 33 ms, maximum 163 ms |
17
+ | Read | median 0.2 ms | median 28 ms |
18
+ | First call for a cold identity | approximately 5 ms | 315 ms |
19
+
20
+ This is not a controlled cross-database benchmark and no sample count was
21
+ recorded. It is still useful adoption evidence: a synchronous actor call is not
22
+ a substitute for a direct indexed row read when single-digit-millisecond
23
+ latency is the requirement. The first call includes actor-instance creation,
24
+ caller-process registration, message enqueue, activation claim, handler
25
+ execution, fenced commit, and activation release.
26
+
27
+ ## Project development benchmark
28
+
8
29
  Measured 2026-08-06 on an Apple M5 with 24 GB RAM, Ruby 4.0.5, Rails 8.1.3.1,
9
30
  and SQLite 3.51.0. Each throughput scenario used 200 operations; the concurrent
10
31
  scenario used four worker threads.
@@ -21,6 +42,59 @@ scenario used four worker threads.
21
42
  | Activation reuse | 98.0%, four activations for 200 messages |
22
43
  | Queries for one message turn | 29 |
23
44
 
45
+ The difference between the SQLite development result and the MySQL adoption
46
+ result is why Solid Objects does not publish one latency promise. Network
47
+ topology, adapter behavior, host schema, logging, callbacks, and contention all
48
+ matter.
49
+
50
+ ## Durable row growth
51
+
52
+ The storage cost is deterministic even when latency is not:
53
+
54
+ - the first call for one actor identity inserts one
55
+ `solid_objects_instances` row;
56
+ - every direct, `sync`, query, attribute read, or `async` call inserts one
57
+ permanent `solid_objects_messages` row;
58
+ - ready and claimed membership rows exist only while the call is pending or
59
+ executing;
60
+ - one caller process row is registered per application process that performs
61
+ synchronous calls;
62
+ - effects and observable changes add outbox rows; and
63
+ - reminders add one row per named actor reminder.
64
+
65
+ Attribute reads are therefore not free snapshots from the instance row. They
66
+ are ordered durable query messages and grow message history exactly like
67
+ writes.
68
+
69
+ Built-in pruning previews by default and must be scheduled explicitly. Budget
70
+ message growth before retention as:
71
+
72
+ ```text
73
+ daily durable messages = daily actor writes + daily actor reads + daily callbacks
74
+ ```
75
+
76
+ Review the [retention requirements](operations.md#retention-and-backups) before
77
+ adopting a high-volume surface.
78
+
79
+ Use `reference.snapshot` for an authorized current-state read when mailbox
80
+ ordering is unnecessary. It avoids a message row but can observe state before
81
+ an in-flight turn commits.
82
+
83
+ ## Measure the host application
84
+
85
+ Run the adoption benchmark against a dedicated empty database with the same
86
+ adapter and topology as production:
87
+
88
+ ```bash
89
+ COUNT=25 \
90
+ SOLID_OBJECTS_DATABASE_URL=mysql2://localhost/solid_objects_benchmark \
91
+ bundle exec ruby -Ilib benchmark/adoption_latency.rb
92
+ ```
93
+
94
+ It reports the first cold call, warm synchronous writes, ordered reads, and
95
+ durable instance/message growth. Run it near the application process, with
96
+ production-like TLS and network boundaries where applicable.
97
+
24
98
  The scripts and invocation examples are in the
25
99
  [development guide](development.md#benchmarks). PostgreSQL and MySQL should be
26
100
  benchmarked independently before selecting production capacity.
data/docs/correctness.md CHANGED
@@ -101,13 +101,20 @@ The following are atomic:
101
101
  - activation owner, expiration, and generation acquisition;
102
102
  - ready-to-claimed membership move and attempt increment;
103
103
  - state, state version, message result/completion, claimed deletion, effects,
104
- reminders, outbound actor messages, and observable broadcasts;
104
+ same-database commit actions, reminders, outbound actor messages, and
105
+ observable broadcasts;
105
106
  - failed-attempt record plus ready reinsertion or dead letter;
106
107
  - effect completion plus its optional actor outcome message;
107
108
  - reminder occurrence enqueue plus reminder advancement; and
108
109
  - actor destruction plus cascading removal of all actor-owned rows.
109
110
 
110
111
  Actor Ruby code and external I/O are never inside the actor-state transaction.
112
+ Rails write prevention rejects direct Active Record writes while handlers,
113
+ observables, lifecycle hooks, or state migrations run. A registered commit
114
+ action is the only application-record write path inside the fenced commit, and
115
+ it is available only when Solid Objects and `ActiveRecord::Base` share one
116
+ connection pool. Commit actions must contain only bounded database work.
117
+ External I/O belongs in the effect outbox.
111
118
 
112
119
  ## Synchronous invocation
113
120
 
@@ -119,6 +126,17 @@ result. A worker may win the activation instead; the caller then observes the
119
126
  durable result through wake-up hints with bounded polling as fallback.
120
127
 
121
128
  Timeout raises `SolidObjects::SyncTimeout` but does not cancel the message.
129
+ The exception reports actor identity, message ID and sequence, durable status,
130
+ an earlier mailbox blocker, and activation-owner metadata without exposing
131
+ arguments. Its `message_reference` can reauthorize and wait for the eventual
132
+ result. Adapter lock/query deadlines cover the durable enqueue and coordination
133
+ transactions. If enqueue cannot commit, `SyncEnqueueTimeout` is raised and no
134
+ message reference exists. MySQL lock waits have one-second InnoDB granularity.
135
+ Ruby handlers that already started are not preempted.
136
+
137
+ A synchronous call made while the Solid Objects connection already has an open
138
+ transaction raises `SolidObjects::SyncInsideTransaction` before the message is
139
+ enqueued.
122
140
  Destroying the actor while a synchronous caller waits removes its message,
123
141
  wakes the caller, and raises `SolidObjects::ActorDestroyed`.
124
142
 
@@ -24,6 +24,11 @@ Indexes:
24
24
  - owner: dead-process cleanup;
25
25
  - last used/ID: retention and reconciliation.
26
26
 
27
+ Instance expiration is disabled unless its actor type has an explicit
28
+ retention policy. The pruner uses the last-used index for candidate discovery,
29
+ then locks and rechecks that the actor has no owner, mailbox membership,
30
+ scheduled reminder, unresolved outbox, or dead letter before deletion.
31
+
27
32
  ### `messages`
28
33
 
29
34
  Durable immutable invocation identity and arguments plus sequence, attempt
data/docs/development.md CHANGED
@@ -28,6 +28,46 @@ SOLID_OBJECTS_DATABASE_URL=mysql2://... bundle exec rake test
28
28
  Each database run must start from an empty dedicated test database because the
29
29
  test helper applies the engine migration.
30
30
 
31
+ ## Host application tests
32
+
33
+ Rails transactional tests keep the application connection inside an outer
34
+ transaction. Synchronous actor invocation deliberately rejects that condition
35
+ because nested savepoints retain actor locks until the test transaction ends
36
+ and make durable behavior unlike production.
37
+
38
+ Use an actor-specific base class:
39
+
40
+ ```ruby
41
+ require "solid_objects/test_helper"
42
+
43
+ class SolidObjectsTestCase < ActiveSupport::TestCase
44
+ include SolidObjects::TestHelper
45
+ end
46
+ ```
47
+
48
+ The helper disables transactional tests for that class and removes Solid
49
+ Objects instances and process registrations before and after each test. It
50
+ preserves application configuration, actor registration, and effect/commit
51
+ action registration. If actor commit actions create application records, clean
52
+ those records with fixtures or explicit teardown because they are no longer
53
+ covered by Rails' transaction rollback.
54
+
55
+ Use `drain_solid_objects` to process actor, reminder, effect, callback, and
56
+ broadcast work to a deterministic fixed point without arbitrary sleeps:
57
+
58
+ ```ruby
59
+ message = Counter.ref("test").async(:increment)
60
+
61
+ assert_equal 1, drain_solid_objects
62
+ assert_equal "completed", message.status
63
+ ```
64
+
65
+ Pass `roles: [:actors]` when a test intentionally wants to leave outboxes or
66
+ reminders pending.
67
+
68
+ `SolidObjects::TestHelper.reset_actors!` is also available for explicit suite
69
+ boundaries.
70
+
31
71
  ## Inline RBS
32
72
 
33
73
  Ruby source starts with:
@@ -66,12 +106,13 @@ correct change, rerun the focused test, then the complete database matrix.
66
106
 
67
107
  ## Benchmarks
68
108
 
69
- Scripts in `benchmark/` cover enqueue, claim, processing, cold actors, a hot
70
- actor, concurrent actors, synchronous latency, cache reuse, and query counts.
71
- Results describe one machine and database configuration; they are not universal
72
- capacity guarantees.
109
+ Scripts in `benchmark/` cover adoption latency and durable row growth, enqueue,
110
+ claim, processing, cold actors, a hot actor, concurrent actors, synchronous
111
+ latency, cache reuse, and query counts. Results describe one machine and
112
+ database configuration; they are not universal capacity guarantees.
73
113
 
74
114
  ```bash
115
+ COUNT=25 bundle exec ruby -Ilib benchmark/adoption_latency.rb
75
116
  COUNT=500 bundle exec ruby -Ilib benchmark/enqueue.rb
76
117
  COUNT=500 bundle exec ruby -Ilib benchmark/claim.rb
77
118
  COUNT=500 bundle exec ruby -Ilib benchmark/processing.rb
data/docs/fit.md ADDED
@@ -0,0 +1,98 @@
1
+ # Is Solid Objects a good fit?
2
+
3
+ Solid Objects trades database work and retained message history for one strong
4
+ property: all committed turns for one durable identity execute in order behind
5
+ a fenced activation. Adopt it when that coordination property removes
6
+ application-level locking, recovery, and scheduling code that would otherwise
7
+ be difficult to make correct.
8
+
9
+ ## Strong fit signals
10
+
11
+ Solid Objects is a good candidate when most of these are true:
12
+
13
+ - State belongs to one durable identity such as a cart, room, device, session,
14
+ workflow, or user-specific schedule.
15
+ - Writes for that identity must be serialized.
16
+ - The state is naturally a bounded JSON document.
17
+ - The object needs per-identity reminders, transactional external effects, or
18
+ reactive Rails views.
19
+ - Different identities should run concurrently while one hot identity remains
20
+ deliberately sequential.
21
+ - A durable mailbox and at-least-once retry are more valuable than minimum
22
+ request latency.
23
+ - The application can operate and monitor additional database tables and, for
24
+ asynchronous features, a Solid Objects runtime process.
25
+
26
+ Typical fits include checkout state machines, collaborative rooms, device
27
+ twins, durable assessments, approval workflows, and user-specific scheduling.
28
+
29
+ ## Poor fit and anti-patterns
30
+
31
+ Prefer ordinary Active Record, cache storage, Active Job, or an event pipeline
32
+ when any of these dominate:
33
+
34
+ - High-QPS request-path reads. Every actor attribute read is an ordered durable
35
+ message, not a direct `SELECT`, and retains a message-history row.
36
+ - Hot counters such as abuse limits, impressions, page views, or metrics. One
37
+ identity is a serialization point and cannot gain throughput by adding
38
+ workers.
39
+ - High-volume append workloads. Actor state rewrites a JSON document and the
40
+ mailbox retains one durable message per call.
41
+ - Latency budgets where tens of milliseconds are already unacceptable.
42
+ - Large, relational, or query-heavy state. Keep that data normalized in
43
+ application tables.
44
+ - CPU-heavy work or slow network I/O inside a handler.
45
+ - Cross-actor transactions or synchronous actor-to-actor call graphs.
46
+ - State that is clearer as a normal record with database constraints and direct
47
+ service methods.
48
+
49
+ A rate limiter is usually a poor actor: it is hot, request-critical, and often
50
+ expires rather than requiring permanent message history. An impressions
51
+ pipeline is also a poor actor: its value is high-throughput append and
52
+ aggregation, not serialized mutable state.
53
+
54
+ ## Cost model
55
+
56
+ Every synchronous or asynchronous invocation:
57
+
58
+ - inserts one permanent `solid_objects_messages` row;
59
+ - briefly occupies one ready or claimed membership row;
60
+ - performs several short coordination transactions; and
61
+ - may add effect, broadcast, or reminder records.
62
+
63
+ The first call for an identity also inserts one `solid_objects_instances` row.
64
+ Each application process that performs synchronous calls registers one caller
65
+ process row. Actor state is rewritten as a JSON value on each successful
66
+ mutation.
67
+
68
+ Built-in bounded pruning is explicit and dry-run by default. Configure global
69
+ and per-actor-type message retention, then schedule the reviewed execute
70
+ commands. Actor instances expire only for types explicitly listed in
71
+ `instance_retention_by_actor_type`. See
72
+ [performance measurements](benchmarks.md) and
73
+ [retention guidance](operations.md#retention-and-backups).
74
+
75
+ Actor handlers may read application records but may not write them directly.
76
+ Use a same-database commit action for a short atomic database change or an
77
+ idempotent effect for external work. If the domain needs broad relational
78
+ updates throughout arbitrary handler code, an ordinary Active Record service
79
+ is likely a clearer fit.
80
+
81
+ ## Decision checklist
82
+
83
+ Before adopting an actor, answer:
84
+
85
+ 1. What exact race or lifecycle problem requires serialized per-identity turns?
86
+ 2. What is the canonical actor identity?
87
+ 3. How hot can one identity become?
88
+ 4. Can the request path tolerate the measured cold and warm latency?
89
+ 5. How many calls and durable rows will this surface create per day?
90
+ 6. Which calls can be asynchronous?
91
+ 7. Which effects need downstream idempotency?
92
+ 8. Which runtime roles and operational alerts will the feature require?
93
+ 9. How will completed messages and outbox history be retained?
94
+ 10. How will existing state be cut over and rolled back?
95
+
96
+ Benchmark the actual host database and deployment topology before committing a
97
+ latency-sensitive surface. Local benchmark results are evidence about query
98
+ shape, not universal capacity guarantees.
@@ -0,0 +1,140 @@
1
+ # Migrating existing state
2
+
3
+ Moving an existing Redis, cache, or key-value state machine into Solid Objects
4
+ is a data migration and a coordination cutover. Treat it as a staged production
5
+ change, not a rewrite that switches storage in one deploy.
6
+
7
+ ## 1. Write down the existing contract
8
+
9
+ Inventory:
10
+
11
+ - every read and write path;
12
+ - the current canonical and secondary keys;
13
+ - expiration and cleanup behavior;
14
+ - concurrency guards and idempotency keys;
15
+ - external effects;
16
+ - expected request latency and volume; and
17
+ - rollback requirements.
18
+
19
+ Run the [fit checklist](fit.md#decision-checklist) before migrating. A hot
20
+ counter or append pipeline may be better left in its existing store.
21
+
22
+ ## 2. Choose one canonical identity
23
+
24
+ Solid Objects addresses an actor with one `(actor_type, actor_id)` pair. Do not
25
+ hide two competing identities inside actor code or reintroduce a scan.
26
+
27
+ When existing state is written by `(user_id, assessment_short)` but read by
28
+ `session_id`, create a normalized lookup record:
29
+
30
+ ```ruby
31
+ class AssessmentSession < ApplicationRecord
32
+ validates :session_id, uniqueness: true
33
+ validates :assessment_short, uniqueness: { scope: :user_id }
34
+
35
+ def actor
36
+ Assessment.ref(id)
37
+ end
38
+ end
39
+ ```
40
+
41
+ The lookup row gives both old keys one stable primary key. The actor ID is the
42
+ lookup record ID, and ordinary indexed Active Record queries resolve either
43
+ external key. This is clearer and safer than delimiter-joining composite values
44
+ or preserving a `LIKE` scan.
45
+
46
+ Create and backfill the lookup table before actor traffic begins. Enforce every
47
+ identity invariant with unique database indexes.
48
+
49
+ ## 3. Add an idempotent bootstrap message
50
+
51
+ Never bulk-update `solid_objects_instances.state`. Direct writes bypass actor
52
+ ordering, state migrations, observables, activation ownership, and fencing.
53
+
54
+ Import through a normal actor message:
55
+
56
+ ```ruby
57
+ class Assessment < SolidObjects::Actor
58
+ attribute :imported, default: false
59
+ attribute :answers, default: -> { [] }
60
+
61
+ def bootstrap(answers:)
62
+ return if imported
63
+
64
+ self.answers = answers
65
+ self.imported = true
66
+ end
67
+ end
68
+ ```
69
+
70
+ Give every bootstrap call an idempotency key derived from the legacy record:
71
+
72
+ ```ruby
73
+ session.actor.async(
74
+ :bootstrap,
75
+ answers: legacy.answers,
76
+ idempotency_key: "legacy-assessment:#{legacy.id}",
77
+ available_at: jittered_time
78
+ )
79
+ ```
80
+
81
+ Spread large backfills over a dispatch window and monitor mailbox age,
82
+ failures, and dead letters. Asynchronous backfill requires the worker runtime.
83
+
84
+ ## 4. Prefer shadow comparison over blind dual writes
85
+
86
+ Two independent stores cannot be updated atomically without a shared
87
+ transaction or outbox. A controller that writes Redis and an actor in sequence
88
+ can leave them divergent after a timeout or crash.
89
+
90
+ A safer rollout:
91
+
92
+ 1. Keep the legacy store authoritative.
93
+ 2. Bootstrap the actor from a consistent legacy snapshot.
94
+ 3. Mirror new changes to the actor with stable idempotency keys.
95
+ 4. Read both stores in a background comparison path.
96
+ 5. Record divergence counts without changing the user response.
97
+ 6. Repair through actor messages, never direct actor-state SQL.
98
+ 7. Cut reads over only after divergence remains acceptably low.
99
+
100
+ If the actor becomes authoritative before the legacy system is retired, emit a
101
+ transactional effect that updates the legacy store. The effect is at least once,
102
+ so the legacy write still needs idempotency.
103
+
104
+ When the legacy state is an application table in the same connection pool as
105
+ Solid Objects, a registered `commit_action` can update it in the fenced actor
106
+ transaction instead. Keep that action database-only and bounded. A separate
107
+ actor database cannot make this atomic; use the outbox effect and reconciliation
108
+ path there.
109
+
110
+ ## 5. Cut over in reversible stages
111
+
112
+ A typical zero-downtime sequence is:
113
+
114
+ 1. Deploy the lookup table and dual-key resolution.
115
+ 2. Deploy actor code and policies with reads still on the legacy store.
116
+ 3. Start the required runtime roles.
117
+ 4. Backfill actors in bounded batches.
118
+ 5. Enable shadow comparison and reconcile drift.
119
+ 6. Move a small cohort of reads to actors.
120
+ 7. Expand the cohort while watching latency, database growth, retries, and
121
+ divergence.
122
+ 8. Move writes to the actor.
123
+ 9. Retain the legacy state through an explicit rollback window.
124
+ 10. Remove dual writes and legacy data only after the rollback window closes.
125
+
126
+ Use a feature flag whose rollback restores legacy reads and writes without
127
+ requiring actor deletion. Do not assume a timed-out synchronous actor call did
128
+ not commit; use `error.message_reference.wait` to reauthorize and recover the
129
+ durable result, or use an idempotency key before retrying.
130
+
131
+ ## 6. Plan for dormant state and future changes
132
+
133
+ Actor state migrations and legacy-store migration solve different problems:
134
+
135
+ - this cookbook moves ownership from another store into an actor;
136
+ - `state_version` evolves actor JSON after that ownership exists.
137
+
138
+ Keep every published actor migration step. A dormant actor can reactivate years
139
+ later with an old state representation. See the
140
+ [state migration guide](state-migrations.md) for rolling-deployment rules.
data/docs/operations.md CHANGED
@@ -1,5 +1,20 @@
1
1
  # Operations guide
2
2
 
3
+ ## Installation verification
4
+
5
+ Run the installation doctor after generating the initializer and migrating:
6
+
7
+ ```bash
8
+ bin/rails solid_objects:doctor
9
+ ```
10
+
11
+ It validates runtime configuration, required tables and columns, neutral policy
12
+ posture, live runtime roles, and a real workerless synchronous actor round-trip.
13
+ Engine migration timestamps are rewritten when copied into a host application,
14
+ so the schema check compares the required shape instead of a fixed timestamp.
15
+ Warnings such as an all-deny neutral policy do not fail the command because a
16
+ context-aware production policy may correctly deny the probe.
17
+
3
18
  ## Runtime
4
19
 
5
20
  Start all configured roles:
@@ -19,6 +34,9 @@ Process inspection, cleanup, dead-letter inspection, and retry all require an
19
34
  administration policy that authorizes the CLI context:
20
35
 
21
36
  ```bash
37
+ bundle exec solid_objects prune_messages
38
+ bundle exec solid_objects prune_instances
39
+ bundle exec solid_objects prune_processes
22
40
  bundle exec solid_objects dead_letters
23
41
  bundle exec solid_objects retry_dead_letter 123
24
42
  ```
@@ -41,6 +59,9 @@ Important controls include:
41
59
  - payload, state, and result byte limits
42
60
  - retry attempts and delay
43
61
  - heartbeat interval and alive threshold
62
+ - message retention and per-actor-type overrides
63
+ - opt-in actor-instance retention by actor type
64
+ - stopped-process retention and prune batch size
44
65
 
45
66
  Keep lease duration comfortably above renewal interval and expected database
46
67
  pause time. A handler can exceed the pass-duration budget because Ruby code is
@@ -111,12 +132,81 @@ Alert on:
111
132
  - reconciliation drift;
112
133
  - database lock waits, deadlocks, and SQLite busy errors.
113
134
 
135
+ ## Instrumentation and logging
136
+
137
+ Active Support notifications use the `solid_objects.` prefix. Core events
138
+ include message enqueue/start/completion/failure/rejection, activation
139
+ claim/start/renew/release/deactivation failure, sync timeout/enqueue timeout/
140
+ transaction rejection, commit-action start/completion/failure, effect and
141
+ broadcast enqueue/completion, reminder enqueue, actor destruction/expiration,
142
+ retention pruning, process cleanup, and supervisor lifecycle.
143
+
144
+ Payloads contain stable runtime identifiers, actor identity, sequence,
145
+ attempts, ownership generations, and safe exception summaries where relevant.
146
+ Arguments, actor state, results, and outbox payloads are excluded. The bundled
147
+ log subscriber turns the same notifications into structured logger hashes.
148
+
114
149
  ## Retention and backups
115
150
 
116
- The schema has cleanup indexes, but automatic pruning commands are still
117
- roadmap work. Until implemented, define application-owned bounded deletes that
118
- preserve unfinished messages, dead letters under investigation, and synchronous
119
- results for the promised lookup period.
151
+ Every actor call creates a durable message-history row, including queries and
152
+ attribute reads. The default retention policy keeps terminal message history
153
+ for 30 days and stopped process records for 7 days:
154
+
155
+ `reference.snapshot` is the explicit exception: it performs an authorized
156
+ current-state read without mailbox ordering or a message row.
157
+
158
+ ```ruby
159
+ SolidObjects.configure do |configuration|
160
+ configuration.message_retention = 30.days
161
+ configuration.message_retention_by_actor_type = {
162
+ "AuditActor" => 365.days,
163
+ "EphemeralCounter" => 1.day
164
+ }
165
+ configuration.instance_retention_by_actor_type = {
166
+ "EphemeralCounter" => 30.days
167
+ }
168
+ configuration.process_retention = 7.days
169
+ configuration.prune_batch_size = 1_000
170
+ end
171
+ ```
172
+
173
+ Both pruning commands are dry-run previews by default:
174
+
175
+ ```bash
176
+ bundle exec solid_objects prune_messages
177
+ bundle exec solid_objects prune_instances
178
+ bundle exec solid_objects prune_processes
179
+ ```
180
+
181
+ After reviewing the counts, execute bounded deletion:
182
+
183
+ ```bash
184
+ bundle exec solid_objects prune_messages --execute
185
+ bundle exec solid_objects prune_instances --execute
186
+ bundle exec solid_objects prune_processes --execute
187
+ ```
188
+
189
+ Message pruning keeps ready and claimed work, dead letters and their retry
190
+ links, messages with unfinished effects, and messages with undelivered
191
+ broadcasts. Deleting eligible history cascades to completed effects, delivered
192
+ broadcasts, and other message-owned rows. Choose a cutoff longer than every
193
+ `sync` timeout because a caller whose result row disappears can no longer
194
+ observe it.
195
+
196
+ Actor expiration is disabled by default. `prune_instances` considers only
197
+ actor types listed in `instance_retention_by_actor_type`, excludes active or
198
+ paused actors, and preserves ready/claimed mailbox work, scheduled reminders,
199
+ unfinished or dead outboxes, and dead letters. It locks and rechecks every
200
+ candidate before cascading deletion. Preview counts first, then schedule
201
+ `--execute` only after the application has accepted the loss of dormant state
202
+ and completed history.
203
+
204
+ Use authorized `reference.destroy` when deletion is an explicit application
205
+ operation rather than a retention policy.
206
+
207
+ Run stale-process `cleanup` before `prune_processes`. Normal caller processes
208
+ mark their registrations stopped at exit; hard kills remain recoverable through
209
+ heartbeat cleanup.
120
210
 
121
211
  Back up actor tables with the same consistency guarantees as application data.
122
212
  Restoring only instances without their mailboxes/outboxes, or vice versa, can