typed_eav 0.7.1 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +87 -2
  3. data/README.md +79 -1505
  4. data/RELEASING.md +81 -0
  5. data/docs/adr/0001-collapse-column-mapping-stack.md +28 -0
  6. data/docs/adr/0002-entity-query-orchestration.md +33 -0
  7. data/docs/adr/0003-keep-event-dispatcher-broker.md +41 -0
  8. data/docs/adr/0004-field-family-intermediate-bases.md +49 -0
  9. data/docs/adr/0005-keep-phase-six-modules-independent.md +48 -0
  10. data/docs/adr/0006-include-missing-via-set-complement.md +77 -0
  11. data/docs/adr/0007-visibility-versus-mutation-relations.md +33 -0
  12. data/docs/adr/0008-partial-covering-scalar-indexes.md +83 -0
  13. data/docs/adr/0009-string-search-indexing.md +110 -0
  14. data/docs/adr/0010-planner-statistics-policy.md +109 -0
  15. data/docs/adr/0011-multi-filter-query-strategy.md +111 -0
  16. data/docs/adr/0012-cross-scope-administrative-query-policy.md +76 -0
  17. data/docs/adr/0013-durable-versioning-and-field-deletion.md +125 -0
  18. data/docs/adr/index.md +101 -0
  19. data/docs/getting-started.md +79 -0
  20. data/docs/guides/architecture.md +125 -0
  21. data/docs/guides/bulk-operations.md +205 -0
  22. data/docs/guides/csv-import.md +88 -0
  23. data/docs/guides/development.md +73 -0
  24. data/docs/guides/events-and-versioning.md +360 -0
  25. data/docs/guides/fields.md +342 -0
  26. data/docs/guides/performance.md +107 -0
  27. data/docs/guides/queries.md +188 -0
  28. data/docs/guides/schema.md +134 -0
  29. data/docs/guides/scoping.md +254 -0
  30. data/docs/guides/upgrading.md +26 -0
  31. data/docs/guides/usage.md +259 -0
  32. data/docs/index.md +44 -0
  33. data/docs/maintaining.md +82 -0
  34. data/docs/reference/api.md +133 -0
  35. data/docs/reference/configuration.md +64 -0
  36. data/docs/reference/index.md +16 -0
  37. data/lib/typed_eav/bulk_read.rb +143 -22
  38. data/lib/typed_eav/entity_query.rb +154 -2
  39. data/lib/typed_eav/has_typed_eav/dirty_tracking.rb +207 -0
  40. data/lib/typed_eav/has_typed_eav.rb +6 -2
  41. data/lib/typed_eav/scalar_query.rb +228 -0
  42. data/lib/typed_eav/schema_portability/preview.rb +379 -0
  43. data/lib/typed_eav/schema_portability.rb +20 -0
  44. data/lib/typed_eav/version.rb +1 -1
  45. data/lib/typed_eav.rb +1 -0
  46. metadata +38 -1
data/README.md CHANGED
@@ -1,31 +1,22 @@
1
1
  # TypedEAV
2
2
 
3
- Add dynamic custom fields to ActiveRecord models at runtime, backed by **native database typed columns** instead of jsonb blobs.
3
+ Add runtime-defined custom fields to Active Record models, with typed values,
4
+ validation, and SQL filtering, sorting, and summaries.
4
5
 
5
- TypedEAV uses a hybrid EAV (Entity-Attribute-Value) pattern where each value type gets its own column (`integer_value`, `date_value`, `string_value`, etc.) in the values table. This lets PostgreSQL index, sort, and enforce constraints on custom field data while the Field owns input normalization and validation.
6
+ Scalar values use native PostgreSQL columns such as `integer_value` and
7
+ `date_value`; collection and JSON fields use JSONB. Field definitions control
8
+ casting and validation, and can be shared globally or scoped to tenants.
6
9
 
7
- ## Why Typed Columns?
8
-
9
- JSONB is a useful fit when an application owns stable paths and wants expression B-tree indexes or GIN containment indexes. A single JSONB document is not inherently faster or slower than TypedEAV; the right choice depends on access patterns, selectivity, update shape, and operational constraints. For example, an application-owned expression index may support a stable path:
10
-
11
- ```sql
12
- CAST(value_meta->>'const' AS bigint) = 42
13
- ```
14
-
15
- This can work well for stable, known paths. It does not provide the same column-level schema and typed-value contract as TypedEAV, and arbitrary paths still require application-owned index and validation decisions. GIN is useful for containment workloads; expression B-trees are useful for selected stable scalar paths.
16
-
17
- TypedEAV stores values in native columns, so queries become:
18
-
19
- ```sql
20
- WHERE integer_value = 42
21
- ```
22
-
23
- TypedEAV supplies stable typed columns and ordinary per-type indexes. Range scans and sorts can use those indexes, while each Field casts and validates the operand according to its own semantics before the query reaches the typed column. Neither design is a universal storage winner; choose from measured workload fit.
10
+ TypedEAV is useful when users define fields at runtime and your application
11
+ needs to query them through a consistent typed interface. JSONB also works well
12
+ for many workloads; choose based on your query patterns and operational needs.
13
+ See [Storage and performance](docs/guides/performance.md) for the tradeoffs,
14
+ indexing guidance, and benchmark evidence.
24
15
 
25
16
  ## Compatibility
26
17
 
27
18
  The canonical support contract lives in
28
- [`.github/compatibility.json`](.github/compatibility.json). Typed EAV supports:
19
+ [`.github/compatibility.json`](https://github.com/dchuk/typed_eav/blob/main/.github/compatibility.json). Typed EAV supports:
29
20
 
30
21
  | Runtime | Supported versions |
31
22
  |---|---|
@@ -53,552 +44,46 @@ bin/rails typed_eav:install:migrations
53
44
  bin/rails db:migrate
54
45
  ```
55
46
 
47
+ PostgreSQL is required; MySQL and SQLite are not supported. For existing
48
+ installations, see [Upgrading](docs/guides/upgrading.md) and the
49
+ [Changelog](CHANGELOG.md).
50
+
56
51
  ## Quick Start
57
52
 
58
- ### 1. Include the concern
53
+ Assuming your application already has a `Contact` model and table:
59
54
 
60
55
  ```ruby
61
56
  class Contact < ApplicationRecord
62
57
  has_typed_eav
63
58
  end
64
59
 
65
- # With multi-tenant scoping:
66
- class Contact < ApplicationRecord
67
- has_typed_eav scope_method: :tenant_id
68
- end
69
-
70
- # With restricted field types:
71
- class Contact < ApplicationRecord
72
- has_typed_eav types: [:text, :integer, :boolean, :select]
73
- end
74
- ```
75
-
76
- ### 2. Create field definitions
77
-
78
- ```ruby
79
- # Simple fields
80
- TypedEAV::Field::Text.create!(
81
- name: "nickname",
82
- entity_type: "Contact"
83
- )
84
-
85
60
  TypedEAV::Field::Integer.create!(
86
61
  name: "age",
87
- entity_type: "Contact",
88
- required: true,
62
+ entity_type: Contact.polymorphic_name,
89
63
  options: { min: 0, max: 150 }
90
64
  )
91
65
 
92
- TypedEAV::Field::Date.create!(
93
- name: "birthday",
94
- entity_type: "Contact",
95
- options: { max_date: Date.today.to_s }
96
- )
97
-
98
- # Select field with options
99
- status = TypedEAV::Field::Select.create!(
100
- name: "status",
101
- entity_type: "Contact",
102
- required: true
103
- )
104
- status.field_options.create!([
105
- { label: "Active", value: "active", sort_order: 1 },
106
- { label: "Inactive", value: "inactive", sort_order: 2 },
107
- { label: "Lead", value: "lead", sort_order: 3 },
108
- ])
109
-
110
- # Multi-select (stored as json array)
111
- tags = TypedEAV::Field::MultiSelect.create!(
112
- name: "tags",
113
- entity_type: "Contact"
114
- )
115
- tags.field_options.create!([
116
- { label: "VIP", value: "vip" },
117
- { label: "Partner", value: "partner" },
118
- { label: "Prospect", value: "prospect" },
119
- ])
120
- ```
66
+ contact = Contact.new
67
+ contact.set_typed_eav_value("age", "40")
68
+ contact.save! # Supply any other attributes your Contact model requires.
121
69
 
122
- When deriving `entity_type` from a model class, use
123
- `Contact.polymorphic_name`. Rails stores polymorphic associations under that
124
- canonical name, which is the base-class type for STI hosts and respects the
125
- application's namespaced-polymorphism setting.
126
-
127
- ### 3. Set values on records
128
-
129
- ```ruby
130
- contact = Contact.new(name: "Darrin")
70
+ contact.typed_eav_value("age") # => 40 (Integer)
71
+ contact.typed_eav_hash # => { "age" => 40 }
131
72
 
132
- # Individual assignment
133
- contact.set_typed_eav_value("age", 40)
134
- contact.set_typed_eav_value("status", "active")
135
-
136
- # Bulk assignment by field NAME (ergonomic for scripting / seeds)
137
- contact.typed_eav_attributes = [
138
- { name: "age", value: 40 },
139
- { name: "status", value: "active" },
140
- { name: "tags", value: ["vip", "partner"] },
141
- ]
142
-
143
- # Bulk assignment by field ID (standard Rails form contract).
144
- # Your form templates emit this shape when you use fields_for :typed_values.
145
- contact.typed_values_attributes = [
146
- { id: 12, field_id: 4, value: "40" },
147
- { field_id: 7, value: "active" },
148
- ]
149
-
150
- contact.save!
151
-
152
- # Reading
153
- contact.typed_eav_value("age") # => 40 (Ruby Integer)
154
- contact.typed_eav_value("status") # => "active"
155
- contact.typed_eav_hash # => { "age" => 40, "status" => "active", ... }
156
- ```
157
-
158
- ### 4. Query with the DSL
159
-
160
- Queries use native typed columns and the indexes shipped for each type. The Field remains the owner of operand casting and validation; the query builder receives a field-normalized value rather than applying a generic Active Record cast.
161
-
162
- ```ruby
163
- # Short form - single field filter
164
- Contact.with_field("age", :gt, 21)
165
- Contact.with_field("status", "active") # :eq is the default operator
166
- Contact.with_field("nickname", :contains, "smith")
167
-
168
- # Chain them
169
- Contact.with_field("age", :gteq, 18)
170
- .with_field("status", "active")
171
- .with_field("tags", :any_eq, "vip")
172
-
173
- # Multi-filter form (good for search UIs)
174
- Contact.where_typed_eav(
175
- { name: "age", op: :gt, value: 21 },
176
- { name: "status", op: :eq, value: "active" },
177
- { name: "city", op: :contains, value: "port" },
178
- )
179
-
180
- # Compact keys (for URL params / form submissions)
181
- Contact.where_typed_eav(
182
- { n: "age", op: :gt, v: 21 },
183
- { n: "status", v: "active" },
184
- )
185
-
186
- # With scoping
187
- Contact.where_typed_eav(
188
- { name: "priority", op: :eq, value: "high" },
189
- scope: current_tenant.id
190
- )
191
-
192
- # Combine with standard ActiveRecord
193
- Contact.where(company_id: 42)
194
- .with_field("status", "active")
195
- .with_field("age", :gteq, 21)
196
- .order(:name)
73
+ Contact.with_field("age", :gteq, 21)
74
+ .order_typed_eav("age", direction: :desc)
197
75
  .limit(25)
198
76
  ```
199
77
 
200
- ### Available Operators
201
-
202
- | Operator | Works On | Description |
203
- |----------|----------|-------------|
204
- | `:eq` | all | Equal (default) |
205
- | `:not_eq` | all | Not equal (NULL-safe) |
206
- | `:gt` | numeric, date, datetime | Greater than |
207
- | `:gteq` | numeric, date, datetime | Greater than or equal |
208
- | `:lt` | numeric, date, datetime | Less than |
209
- | `:lteq` | numeric, date, datetime | Less than or equal |
210
- | `:between` | numeric, date, datetime | Between (pass Range or Array) |
211
- | `:contains` | text, long_text | ILIKE %value% |
212
- | `:not_contains` | text, long_text | NOT ILIKE %value% |
213
- | `:starts_with` | text, long_text | ILIKE value% |
214
- | `:ends_with` | text, long_text | ILIKE %value |
215
- | `:any_eq` | json arrays | Array contains element |
216
- | `:all_eq` | json arrays | Array contains all elements |
217
- | `:is_null` | all | Value is NULL |
218
- | `:is_not_null` | all | Value is not NULL |
219
-
220
- ### Optional trigram indexing for string search
221
-
222
- TypedEAV keeps its partial-covering `text_pattern_ops` B-tree as the default
223
- string index. Equality uses that B-tree, while `:starts_with`, `:contains`, and
224
- `:ends_with` use `ILIKE`; `:not_contains` uses `NOT ILIKE`. The gem does not
225
- require or install `pg_trgm` and does not create a trigram index automatically.
226
-
227
- An application with frequent positive `ILIKE` searches containing at least
228
- three useful characters may evaluate its own partial GIN index. This is a
229
- workload decision: the representative benchmark used GIN for measured prefix,
230
- contains, suffix, and escaped-literal patterns, but not for `NOT ILIKE` or
231
- one/two-character probes. It does not prove that every positive pattern or
232
- selectivity will benefit. A `lower(string_value) LIKE ...` expression index is
233
- not equivalent to TypedEAV's public `ILIKE`, and the benchmark did not justify
234
- GiST.
235
-
236
- Application owners should check extension availability and deploy-role
237
- privileges in preproduction, then create the extension and index in their own
238
- migrations. Use nontransactional `CREATE INDEX CONCURRENTLY`, a stable
239
- application-specific name, and workload-specific `EXPLAIN (ANALYZE, BUFFERS,
240
- WAL, SETTINGS)` plus storage and write-WAL measurements. Rollback should drop
241
- only the application-owned index concurrently; do not drop the database-wide
242
- extension because other objects may share it. See
243
- [ADR 0009](docs/adr/0009-string-search-indexing.md) and the
244
- [benchmark guide](bench/README.md#phase-3-string-search-benchmark) for the
245
- operator matrix, measured costs, SQL, and evidence limits.
246
-
247
- ### Optional planner statistics for correlated field/value predicates
248
-
249
- TypedEAV does not install PostgreSQL extended-statistics objects. An application
250
- whose own plans persistently misestimate `field_id = ... AND typed_value = ...`
251
- may evaluate application-owned `dependencies` statistics for that exact typed
252
- column. Dependency statistics apply to compatible equality and `IN` clauses,
253
- not range predicates. `mcv` describes common value combinations, while
254
- `ndistinct` primarily informs distinct-group estimates; neither should be added
255
- without workload evidence.
256
-
257
- The representative PostgreSQL 17 benchmark found better aggregate equality
258
- estimates from dependencies, but no plan-shape or demonstrated runtime benefit.
259
- Its combined object mirrored MCV on the four changed probes because matching MCV
260
- groups supplied those estimates. The experiment's target of 100 was a controlled
261
- input, not a universal recommendation. One probe labeled common-date equality
262
- actually queried an absent date and returned zero rows; it is not evidence about
263
- common-date estimates.
264
-
265
- Applications should own stable names and DDL, select targets from representative
266
- data, run `ANALYZE`, and compare estimated/actual rows, plans, runtime, planning
267
- cost, maintenance cost, and data churn before retaining an object. Coordinate
268
- ownership in shared databases, inspect catalog definitions before changing
269
- objects, and drop only application-owned statistics during rollback. See
270
- [ADR 0010](docs/adr/0010-planner-statistics-policy.md) and the
271
- [benchmark guide](bench/README.md#phase-4a-planner-extended-statistics) for safe
272
- evaluation SQL and evidence limits.
273
-
274
- ### Multi-filter query strategy
275
-
276
- TypedEAV retains its current multi-filter query shape: it resolves each field,
277
- builds the corresponding typed value subquery, and chains those results onto
278
- the host relation with `id IN (...)`. There is no adaptive strategy or alternate
279
- production query API.
280
-
281
- A PostgreSQL 17 benchmark compared the shipped shape with `INTERSECT`,
282
- correlated `EXISTS`, and direct grouped `HAVING` under resource-capped
283
- co-tenancy. The run retained 2,940 attempts, including 622 right-censored
284
- timeouts, and 294 representative identity oracles. Twelve oracles timed out, so
285
- representative equivalence is unproved even though all 282 completed oracles
286
- matched and the smaller 98-oracle smoke matched. Alternatives remain
287
- research-only. Grouped `HAVING` is additionally ineligible for missing-value,
288
- host-universe complement, and empty-filter semantics.
289
-
290
- The result also does not establish valid buffer comparisons or 20-distinct-
291
- field scaling. A parser defect made every derived buffer total a false zero;
292
- nonzero counters remain recoverable from the retained raw plans. The
293
- 20-predicate workloads repeat ten fields, and the skewed 10/20 workloads repeat
294
- five. Future research must repair and validate buffer extraction, exercise
295
- actual 10/20 distinct fields, complete every representative equivalence oracle,
296
- cover the full scope/NULL/missing/polymorphic/error contract, and show the
297
- pre-registered p95, planning-time, buffer, and plan-shape gates before any
298
- adaptive or replacement proposal. See
299
- [ADR 0011](docs/adr/0011-multi-filter-query-strategy.md) and the
300
- [benchmark guide](bench/README.md#phase-4b-multi-filter-query-shapes).
301
-
302
- ### How Type Inference Works
303
-
304
- The owning Field casts and validates query operands before SQL generation;
305
- Active Record supplies the SQL bind plumbing:
306
-
307
- ```ruby
308
- # The Integer Field casts and validates the operand before SQL generation
309
- Contact.with_field("age", :gt, "21")
310
- # SQL: WHERE integer_value > 21 (not '21')
311
-
312
- # The Date Field owns date parsing and validation
313
- Contact.with_field("birthday", :lt, "2000-01-01")
314
- # SQL: WHERE date_value < '2000-01-01'::date
315
-
316
- # The Boolean Field owns truthy/falsy casting
317
- Contact.with_field("active", "true")
318
- # SQL: WHERE boolean_value = TRUE
319
- ```
320
-
321
- Field-owned casting keeps query operands aligned with write semantics, including strict range/array shapes and specialized fields such as Currency and Reference. The resulting normalized operand is bound against the Field's typed column; Active Record supplies SQL bind plumbing, not the field's domain semantics.
322
-
323
- ## Forms
324
-
325
- Wire typed fields into Rails forms via nested attributes:
326
-
327
- ```erb
328
- <%= form_with model: @contact do |f| %>
329
- <%= f.text_field :name %>
330
-
331
- <%= render_typed_value_inputs(form: f, record: @contact) %>
332
-
333
- <%= f.submit %>
334
- <% end %>
335
- ```
336
-
337
- The helper emits one input per available field, including the hidden `id` / `field_id` markers required by `accepts_nested_attributes_for`. Permit the nested shape in your controller — the `value: []` form is required for array/multi-select types:
338
-
339
- ```ruby
340
- def contact_params
341
- params.require(:contact).permit(
342
- :name,
343
- typed_values_attributes: [
344
- :id, :field_id, :_destroy, :value, { value: [] }
345
- ]
346
- )
347
- end
348
- ```
349
-
350
- For list pages, preload the field association to avoid N+1:
351
-
352
- ```ruby
353
- @contacts = Contact.includes(typed_values: :field).all
354
- ```
355
-
356
- ## Admin Scaffold
357
-
358
- To manage field definitions through a UI, run the scaffold generator:
359
-
360
- ```bash
361
- bin/rails g typed_eav:scaffold
362
- bin/rails db:migrate
363
- ```
364
-
365
- This copies a controller, views, helper, Stimulus controllers, and an initializer into your app, and adds routes mounted at `/typed_eav_fields`.
366
-
367
- **Security**: the generated controller ships with `authorize_typed_eav_admin!` returning `head :not_found` by default — fail-closed. Edit the method directly in `app/controllers/typed_eav_controller.rb` to wire it to your auth system:
368
-
369
- ```ruby
370
- def authorize_typed_eav_admin!
371
- return if current_user&.admin?
372
- head :not_found
373
- end
374
- ```
375
-
376
- Defining `authorize_typed_eav_admin!` in `ApplicationController` does **not** override it — the scaffold sets it on its own controller.
377
-
378
- ## Multi-Tenant Scoping
379
-
380
- Field definitions are partitioned by a `scope` column so multiple tenants (or accounts, workspaces, orgs — any partition key your app uses) can each define their own fields without collisions. Fields with `scope = NULL` are global, visible to every partition.
381
-
382
- ### Declaring a scoped model
383
-
384
- ```ruby
385
- class Contact < ApplicationRecord
386
- has_typed_eav scope_method: :tenant_id
387
- end
388
- ```
389
-
390
- `scope_method:` names an instance method on your model. When the record reads its own field definitions (e.g., in a form), that method tells TypedEAV which partition the record belongs to.
391
-
392
- ### Class-level queries resolve scope automatically
393
-
394
- Queries like `Contact.where_typed_eav(...)` consult an **ambient scope resolver** — no need to pass `scope:` on every call:
395
-
396
- ```ruby
397
- # The resolver tells TypedEAV which partition is active.
398
- Contact.where_typed_eav({ name: "age", op: :gt, value: 21 })
399
- ```
400
-
401
- The resolver chain (highest priority first):
402
-
403
- 1. Explicit `scope:` keyword argument on the query
404
- 2. Active `TypedEAV.with_scope(value) { ... }` block
405
- 3. Configured `TypedEAV.config.scope_resolver` callable
406
- 4. `nil`
407
-
408
- If every step returns `nil` and the model declared `scope_method:`, queries raise `TypedEAV::ScopeRequired` — the **fail-closed default**. This is the whole point: forgetting to set scope can't silently leak other partitions' data.
409
-
410
- ### Wiring the resolver
78
+ Fields cast and validate both assigned values and query operands. Queries
79
+ return Active Record relations, so you can combine them with ordinary host
80
+ filters. Use `Contact.polymorphic_name` when creating definitions to respect
81
+ Rails' STI and namespaced-polymorphism settings.
411
82
 
412
- Pick the pattern that matches your app and set it once in `config/initializers/typed_eav.rb`:
413
-
414
- ```ruby
415
- TypedEAV.configure do |c|
416
- # acts_as_tenant (auto-detected — no config needed if loaded)
417
- # c.scope_resolver = -> { ActsAsTenant.current_tenant&.id }
418
-
419
- # Rails CurrentAttributes
420
- # c.scope_resolver = -> { Current.account&.id }
421
-
422
- # Custom class
423
- # c.scope_resolver = -> { MyApp::Tenancy.current_workspace_id }
424
-
425
- # Subdomain / session / thread-local
426
- # c.scope_resolver = -> { Thread.current[:org_id] }
427
-
428
- # Disable ambient resolution entirely
429
- # c.scope_resolver = nil
430
-
431
- c.require_scope = true # fail-closed (default). Set false for gradual adoption.
432
- end
433
- ```
434
-
435
- The resolver MUST return a 2-element Array `[scope, parent_scope]`. Each slot
436
- accepts a raw value (`"t1"`, `42`), an AR record (TypedEAV calls `.id.to_s`
437
- on anything that responds to `#id`), or `nil`. If you don't use parent_scope,
438
- return `[scope, nil]`. A bare scalar return raises `ArgumentError` at the
439
- next ambient query — see [Migrating from v0.1.x](#migrating-from-v01x) for
440
- the upgrade path.
441
-
442
- ### Block APIs
443
-
444
- ```ruby
445
- # Run a block with a specific ambient scope (background jobs, console, rake tasks):
446
- TypedEAV.with_scope(tenant_id) do
447
- Contact.where_typed_eav({ name: "status", op: :eq, value: "active" })
448
- end
449
-
450
- # Escape hatch for admin tools, migrations, or cross-tenant audits:
451
- TypedEAV.unscoped do
452
- Contact.where_typed_eav({ name: "status", op: :eq, value: "active" })
453
- # returns matches across ALL partitions
454
- end
455
- ```
456
-
457
- Both are exception-safe via `ensure` and nest cleanly.
458
-
459
- `unscoped` is an explicit administrative/analytics escape hatch, not the
460
- ordinary tenant request path. It keeps every same-name definition across the
461
- visible partitions and unions their matches for each filter. For broad audits
462
- or migrations, bound the definition universe to the work you actually need and
463
- batch the job at an application-owned boundary. TypedEAV does not prescribe a
464
- universal limit or batch size; measure generated SQL, planning/execution,
465
- memory, and workload interference in your application. Keep normal request
466
- traffic on scoped resolution so global, scope-only, and full-tuple definitions
467
- collapse to the most-specific match.
468
-
469
- ### Explicit `scope:` override
470
-
471
- Any query method accepts `scope:` as an override for admin tools and tests:
472
-
473
- ```ruby
474
- Contact.where_typed_eav({ name: "status", value: "active" }, scope: "t1")
475
- Contact.with_field("age", :gt, 21, scope: "t1")
476
- ```
477
-
478
- Explicit wins over ambient. Passing `scope: nil` explicitly (as opposed to omitting the kwarg) means "filter to global fields only" — useful for admin UIs that want to see unscoped field definitions without activating `unscoped` mode.
479
-
480
- ### Background jobs
481
-
482
- ActiveJob (including Sidekiq via the ActiveJob adapter) wraps every `perform` in Rails' executor, which already clears `ActiveSupport::CurrentAttributes` between jobs — so if your resolver reads from `Current.account`, each job starts clean. For raw `Sidekiq::Job` (no ActiveJob), wrap the job body manually:
483
-
484
- ```ruby
485
- class ExportJob
486
- include Sidekiq::Job
487
-
488
- def perform(tenant_id, ...)
489
- TypedEAV.with_scope(tenant_id) do
490
- Contact.where_typed_eav(...)
491
- end
492
- end
493
- end
494
- ```
495
-
496
- ### Disabling enforcement for gradual adoption
497
-
498
- If your app has existing typed-eav queries that don't yet pass scope, flip `require_scope` to `false` in the initializer. When no scope resolves, queries fall back to **global fields only** (definitions stored with `scope: nil`) instead of raising — they do **not** return all partitions' fields. Audit and fix callers, then flip back to `true`.
499
-
500
- To intentionally query across every partition (admin tools, migrations, cross-tenant audits), use the explicit escape hatch `TypedEAV.unscoped { ... }` rather than relying on `require_scope = false`.
501
-
502
- ### Two-level scoping (`parent_scope`)
503
-
504
- When a single tenant axis isn't enough — say, `tenant_id` for the customer AND
505
- `workspace_id` for an in-tenant partition — declare both:
506
-
507
- ```ruby
508
- class Project < ApplicationRecord
509
- has_typed_eav scope_method: :tenant_id, parent_scope_method: :workspace_id
510
- end
511
- ```
512
-
513
- Field (and section) definitions partition on the tuple `(entity_type, scope,
514
- parent_scope)`. A `Project` record reads field definitions in three precedence
515
- layers: a full-triple `(scope, parent_scope)` match wins, then `(scope, nil)`
516
- (tenant-wide), then `(nil, nil)` (truly global). The same precedence applies
517
- to the class-level query path.
518
-
519
- `parent_scope_method:` requires `scope_method:` — declaring it without a scope
520
- method raises at macro-expansion time (no host can have a parent partition
521
- without a scope partition).
522
-
523
- Both `with_scope` and the configured `scope_resolver` carry the tuple now:
524
-
525
- ```ruby
526
- TypedEAV.with_scope(["t1", "w1"]) do
527
- Project.where_typed_eav({ name: "status", value: "active" })
528
- end
529
-
530
- # Single-axis call still works (parent_scope = nil):
531
- TypedEAV.with_scope("t1") do
532
- Contact.where_typed_eav({ name: "age", op: :gt, value: 21 })
533
- end
534
-
535
- # Custom resolver — MUST return [scope, parent_scope]:
536
- TypedEAV.configure do |c|
537
- c.scope_resolver = -> { [Current.tenant&.id, Current.workspace&.id] }
538
- end
539
- ```
540
-
541
- Per-query overrides accept `parent_scope:` alongside `scope:` on
542
- `where_typed_eav`, `with_field`, and `typed_eav_definitions`:
543
-
544
- ```ruby
545
- Project.where_typed_eav(
546
- { name: "priority", value: "high" },
547
- scope: "t1",
548
- parent_scope: "w1",
549
- )
550
- ```
551
-
552
- When `acts_as_tenant` is loaded, the auto-detected `DEFAULT_SCOPE_RESOLVER`
553
- returns `[ActsAsTenant.current_tenant, nil]` — the parent_scope slot is `nil`
554
- because the tenant gem has no parent-scope analog. Configure your own resolver
555
- when you need both axes.
556
-
557
- ### Migrating from v0.1.x
558
-
559
- The resolver-callable contract is a **breaking change**: any custom
560
- `Config.scope_resolver` lambda must now return `[scope, parent_scope]` (a
561
- 2-element Array) instead of a bare scalar. A scalar return raises
562
- `ArgumentError` at the next ambient query so the failure is loud, not silent.
563
- If you don't use parent_scope, return `[scope, nil]`.
564
-
565
- Run `bin/rails typed_eav:install:migrations` to copy the new
566
- `AddParentScopeToTypedEavPartitions` migration into your app, then
567
- `bin/rails db:migrate`. The migration is safe on production: it adds a
568
- nullable `parent_scope` column (catalog-only, instantaneous) and uses
569
- `CREATE INDEX CONCURRENTLY` for all index changes, so existing rows aren't
570
- rewritten. Existing fields end up with `parent_scope = NULL` (the
571
- global-parent shape) and continue to work for every single-scope caller.
572
-
573
- See the [CHANGELOG](CHANGELOG.md) for the full upgrade checklist.
574
-
575
- ### Orphan-parent invariant
576
-
577
- A `Field` or `Section` row with `parent_scope` set and `scope` blank is
578
- invalid — model-level validation rejects it on save. Reason: a "global field
579
- within one workspace" has no semantic resolution path; the row would never
580
- match any record's resolver. The paired partial unique indexes rely on this
581
- invariant.
582
-
583
- The shipped migration chain also includes
584
- `EnforceParentScopeInvariant`, which declares the database check constraints
585
- nontransactionally and validates them after its preflight, and
586
- `UsePartialCoveringScalarIndexes`, which creates the six `*_present` indexes
587
- before removing their legacy counterparts. Both migrations use
588
- `disable_ddl_transaction!`; run them through the normal migration command and
589
- do not wrap them in an application transaction.
590
-
591
- ### Name collisions across scopes
592
-
593
- When both a global field (`scope: nil`) and a scoped field share a name, the **scoped definition wins** for the partition that owns it: forms render exactly one input (the scoped one), reads return the scoped value, and writes target the scoped row.
594
-
595
- `TypedEAV.unscoped { Contact.where_typed_eav(...) }` OR-across every partition's matching `field_id` per filter (still AND-ing across filters), so cross-tenant audit queries see every partition's matches — they don't collapse to a single tenant.
596
-
597
- Because that administrative path constructs work for every matching
598
- definition, applications should narrow and batch high-cardinality audits rather
599
- than treating `unscoped` as tenant-request routing. No built-in numeric
600
- threshold is implied; choose operational bounds from measurements of the
601
- consuming workload. See [ADR 0012](docs/adr/0012-cross-scope-administrative-query-policy.md).
83
+ See [Reading, writing, and forms](docs/guides/usage.md) for bulk assignment,
84
+ nested attributes, form helpers, and the admin scaffold, or
85
+ [Querying typed fields](docs/guides/queries.md) for operators, multi-field
86
+ filters, sorting, distinct values, counts, and numeric aggregates.
602
87
 
603
88
  ## Field Types
604
89
 
@@ -627,965 +112,54 @@ consuming workload. See [ADR 0012](docs/adr/0012-cross-scope-administrative-quer
627
112
  | `File` | `string_value` (signed_id) + `:attachment` has_one_attached | String (Active Storage signed_id) | `allowed_content_types`, `max_size_bytes` |
628
113
  | `Reference` | `integer_value` (FK) | Integer (target record ID) | `target_entity_type`, `target_scope` |
629
114
 
630
- ## Sections (Optional UI Grouping)
631
-
632
- ```ruby
633
- general = TypedEAV::Section.create!(
634
- name: "General Info",
635
- code: "general",
636
- entity_type: "Contact",
637
- sort_order: 1
638
- )
639
-
640
- social = TypedEAV::Section.create!(
641
- name: "Social Media",
642
- code: "social",
643
- entity_type: "Contact",
644
- sort_order: 2
645
- )
646
-
647
- TypedEAV::Field::Text.create!(
648
- name: "twitter_handle",
649
- entity_type: "Contact",
650
- section: social
651
- )
652
- ```
653
-
654
- ## Custom Field Types
655
-
656
- Override `cast(raw)` to return a `[casted_value, invalid?]` tuple.
657
- `invalid?` tells `Value#validate_value` whether to surface `:invalid`
658
- (vs `:blank`) when raw input can't be coerced. For types that never
659
- fail to coerce, always return `[value, false]`.
660
-
661
- ```ruby
662
- # app/models/fields/phone.rb
663
- module Fields
664
- class Phone < TypedEAV::Field::Base
665
- value_column :string_value
666
- operators :eq, :contains, :starts_with, :is_null, :is_not_null
667
-
668
- def cast(raw)
669
- # Strip everything but digits and +; never rejects as invalid
670
- [raw&.to_s&.gsub(/[^\d+]/, ""), false]
671
- end
672
- end
673
- end
674
-
675
- # Register it
676
- TypedEAV.configure do |c|
677
- c.register_field_type :phone, "Fields::Phone"
678
- end
679
- ```
680
-
681
- ### Family intermediate bases (extension points)
682
-
683
- `Field::Base` is the universal parent, but three intermediate family
684
- bases collapse the most common per-leaf duplication. Pick the right
685
- parent and you inherit the family's validation surface for free.
686
-
687
- - **`TypedEAV::Field::ValidatedString`** — subclass when your custom
688
- type stores in `string_value` and wants a min/max-length + regex-pattern
689
- validation surface. Inherits `value_column :string_value`,
690
- `store_accessor :options, :min_length, :max_length, :pattern`,
691
- numericality validators on `min_length` / `max_length`, a
692
- `max_gte_min_length` guard that rejects inverted bounds at field-save,
693
- and a `validate_pattern_syntax` guard that rejects bad regexes at
694
- field-save. The default `validate_typed_value(record, val)` runs
695
- `validate_length` plus `validate_pattern if pattern.present?`. Override
696
- it and call `super` to layer on a format-specific check (the built-in
697
- `Field::Email` / `Field::Url` are the canonical pattern).
698
-
699
- ```ruby
700
- class Fields::Slug < TypedEAV::Field::ValidatedString
701
- SLUG_FORMAT = /\A[a-z0-9-]+\z/
702
-
703
- def cast(raw)
704
- [raw&.to_s&.strip&.downcase, false]
705
- end
706
-
707
- def validate_typed_value(record, val)
708
- super # length + pattern from the family base
709
- record.errors.add(:value, "is not a valid slug") unless SLUG_FORMAT.match?(val.to_s)
710
- end
711
- end
712
- ```
713
-
714
- - **`TypedEAV::Field::RangeBounded`** — subclass when your custom type
715
- stores a single comparable value (numeric or temporal) constrained by
716
- a min/max bound. Each leaf still declares its own `value_column` and
717
- its own `store_accessor` (key names vary by family member: `:min`/`:max`
718
- for numeric; `:min_date`/`:max_date` for date;
719
- `:min_datetime`/`:max_datetime` for datetime). The family base
720
- provides protected `validate_range` / `validate_date_range` /
721
- `validate_datetime_range` helpers. Each leaf should pair its
722
- `store_accessor` with the macro
723
- `validates :max, comparison: { greater_than_or_equal_to: :min }, allow_nil: true, if: :min`
724
- (or the analogous form for the leaf's key names) so inverted bounds
725
- fail at field-save.
726
-
727
- ```ruby
728
- class Fields::Score < TypedEAV::Field::RangeBounded
729
- value_column :integer_value
730
-
731
- store_accessor :options, :min, :max
732
- validates :max, comparison: { greater_than_or_equal_to: :min }, allow_nil: true, if: :min
733
-
734
- def cast(raw)
735
- raw.nil? ? [nil, false] : [Integer(raw.to_s, exception: false), raw.to_s.empty? ? false : true]
736
- end
737
-
738
- def validate_typed_value(record, val)
739
- validate_range(record, val)
740
- end
741
- end
742
- ```
743
-
744
- - **`TypedEAV::Field::Optionable`** — `include` this concern when your
745
- custom type's valid values are drawn from a `Field::Option` set.
746
- Provides `optionable? = true`, a public-facing sorted
747
- `allowed_values` helper, and protected
748
- `validate_option_inclusion` / `validate_multi_option_inclusion`
749
- helpers. Mixin (not inheritance) because option-set field types may
750
- use different `value_column`s — the built-in `Field::Select` stores in
751
- `string_value` while `Field::MultiSelect` stores in `json_value`, and
752
- both stay as direct children of `Field::Base`.
753
-
754
- ```ruby
755
- class Fields::Tag < TypedEAV::Field::Base
756
- include TypedEAV::Field::Optionable
757
-
758
- value_column :string_value
759
- operators :eq, :not_eq, :is_null, :is_not_null
760
-
761
- def cast(raw)
762
- [raw&.to_s, false]
763
- end
764
-
765
- def validate_typed_value(record, val)
766
- validate_option_inclusion(record, val)
767
- end
768
- end
769
- ```
770
-
771
- The rule of thumb: subclass an intermediate family base when the new
772
- field type shares its storage and validation surface with the family;
773
- include `Optionable` when it draws values from an option set; subclass
774
- `Field::Base` directly (as the `Phone` example above does) when none of
775
- the family surfaces fit. `validate_array_size` lives on `Field::Base`
776
- itself — its callers span unrelated families.
777
-
778
- ### Multi-cell field types
779
-
780
- External field types may store their logical value across multiple typed
781
- columns. The entire storage surface lives directly on `Field::Base` via
782
- the `Field::TypedStorage` concern, so a custom multi-cell type is just a
783
- `Field::Base` subclass that overrides three instance methods.
784
-
785
- **Class-level DSL** (declared at class load time):
786
-
787
- - `value_column :col` – single-cell sugar; declares the primary cell.
788
- - `value_columns :a, :b, ...` – plural form for multi-cell types. The
789
- primary cell is `value_columns.first`. Both forms share storage;
790
- `value_column` and `value_columns` are interchangeable getters/setters.
791
- - `operators :eq, :gt, ...` – restrict the supported operator set.
792
- - `self.operator_column(op)` – override to route different operators to
793
- different cells. Defaults to `value_columns.first`.
794
-
795
- **Override-point instance methods** (the entire extension surface for
796
- multi-cell types):
797
-
798
- - `read_value(record)` – compose the logical value from the cells.
799
- - `write_value(record, casted)` – unpack the casted value across cells.
800
- - `apply_default(record)` – populate cells from `default_value`.
801
-
802
- The defaults target `value_columns.first`, so single-cell field types
803
- keep working without overrides. The three methods are paired – override
804
- all three or your reads will see a multi-cell shape that writes / defaults
805
- cannot produce.
806
-
807
- **Concrete snapshot helpers** (NOT overridable; derived from
808
- `value_columns`):
809
-
810
- - `value_changed?(record)` – true iff any cell saw a saved change.
811
- - `before_snapshot(record, change_type)` / `after_snapshot(record, change_type)`
812
- – per-cell hashes keyed by string column names; powers the versioning
813
- jsonb shape.
814
-
815
- Custom multi-cell type example (matches the built-in `Field::Currency`):
816
-
817
- ```ruby
818
- class Fields::Money < TypedEAV::Field::Base
819
- AMOUNT_COLUMN = :decimal_value
820
- CURRENCY_COLUMN = :string_value
821
-
822
- value_columns AMOUNT_COLUMN, CURRENCY_COLUMN
823
- operators :eq, :gt, :lt, :gteq, :lteq, :between, :currency_eq, :is_null, :is_not_null
824
-
825
- def self.operator_column(operator)
826
- operator == :currency_eq ? CURRENCY_COLUMN : AMOUNT_COLUMN
827
- end
828
-
829
- def read_value(value_record)
830
- amount = value_record[AMOUNT_COLUMN]
831
- currency = value_record[CURRENCY_COLUMN]
832
- return nil if amount.nil? && currency.nil?
833
-
834
- { amount: amount, currency: currency }
835
- end
836
-
837
- def write_value(value_record, casted)
838
- if casted.nil?
839
- value_record[AMOUNT_COLUMN] = nil
840
- value_record[CURRENCY_COLUMN] = nil
841
- else
842
- value_record[AMOUNT_COLUMN] = casted[:amount]
843
- value_record[CURRENCY_COLUMN] = casted[:currency]
844
- end
845
- end
846
-
847
- def apply_default(value_record)
848
- default = default_value
849
- return unless default.is_a?(Hash)
850
-
851
- value_record[AMOUNT_COLUMN] = default[:amount] || default["amount"]
852
- value_record[CURRENCY_COLUMN] = default[:currency] || default["currency"]
853
- end
854
- end
855
- ```
856
-
857
- The built-in `Field::Currency` is the canonical multi-cell consumer of
858
- these extension points and reads as a normal `Field::Base` subclass with
859
- exactly three method overrides.
860
-
861
- ### Built-in field types
862
-
863
- - **`Currency`:** Stores `{amount: BigDecimal, currency: String}` across two typed columns (`decimal_value` for the amount; `string_value` for the ISO 4217 currency code). Multi-cell storage is declared via `value_columns :decimal_value, :string_value`; reads, writes, and default application override `read_value`, `write_value`, and `apply_default` directly on `Field::Currency`. Operators: `:eq`, `:gt`, `:lt`, `:gteq`, `:lteq`, `:between` target the amount; `:currency_eq` targets the currency code; `:is_null` / `:is_not_null` target the amount column (a Currency value is null when its amount is null). Cast input MUST be a hash with `:amount` and/or `:currency` keys — bare numeric/string values are rejected with `:invalid` to enforce explicit currency dimension at write time. Options: `default_currency` (String ISO code, applied as fallback only when an amount is given without an explicit currency), `allowed_currencies` (Array of ISO codes; `validate_typed_value` enforces inclusion). Versioning snapshots automatically capture both columns because the snapshot helpers iterate `value_columns`. The `:currency_eq` operator is registered ONLY on `Field::Currency`; the QueryBuilder operator-validation gate rejects it with a clear `ArgumentError` if invoked on any other field type.
864
-
865
- ```ruby
866
- Contact.where_typed_eav(name: "price", op: :currency_eq, value: "USD")
867
- Contact.where_typed_eav(name: "price", op: :between, value: [50, 150])
868
- ```
869
-
870
- - **`Percentage`:** A `Field::Decimal` subclass storing the underlying fraction in 0..1 (inclusive). The `:percent` representation is a format-time concern — call `field.format(value)` with `display_as: :percent` to render `0.75` as `"75.0%"`. Options: `decimal_places` (Integer >= 0, default 2; format-time precision only — does NOT alter what's stored in `decimal_value`), `display_as` (`:fraction` default, or `:percent`). Validation: out-of-range values (e.g., `1.5`) fail with the message `"must be between 0.0 and 1.0"`. Storage and operator semantics inherit from `Field::Decimal`.
871
-
872
- ```ruby
873
- pf = TypedEAV::Field::Percentage.create!(
874
- name: "discount", entity_type: "Order", scope: tenant_id,
875
- options: { display_as: :percent, decimal_places: 1 },
876
- )
877
- pf.format(BigDecimal("0.755")) # => "75.5%"
878
- ```
879
-
880
- - **`Image`:** Active Storage-backed field type. Stores the attached blob's `signed_id` (a String) in `string_value`. Operators: `:eq`, `:is_null`, `:is_not_null`. Options: `allowed_content_types` (Array of strings; supports exact matches like `"image/png"` and `image/*` family wildcards), `max_size_bytes` (Integer; nil disables the cap). The single `:attachment` has_one_attached association is declared on `TypedEAV::Value` at engine boot when Active Storage is loaded; otherwise `Field::Image#cast` raises `NotImplementedError` with an actionable install message. The `:attachment` association is shared with `Field::File` — Image vs File is a class-identity distinction (used by the `on_image_attached` hook), not a separate association.
881
-
882
- ```ruby
883
- field = TypedEAV::Field::Image.create!(
884
- name: "avatar", entity_type: "Contact",
885
- options: { allowed_content_types: %w[image/png image/jpeg image/webp], max_size_bytes: 5_000_000 },
886
- )
887
- value = TypedEAV::Value.create!(entity: contact, field: field)
888
- value.attachment.attach(io: file_io, filename: "avatar.png", content_type: "image/png")
889
- value.update!(string_value: value.attachment.blob.signed_id)
890
- value.value # => the signed_id String
891
- ```
892
-
893
- - **`File`:** Same shape as `Field::Image` but without image-specific semantics. Stores `signed_id` in `string_value`; same operator set; same options (`allowed_content_types`, `max_size_bytes`). The Image vs File distinction is by `value.field.class` at runtime — apps that want strict image-only validation set `allowed_content_types: ["image/*"]` on `Field::Image`; `Field::File` is a general-purpose attachment slot.
894
-
895
- - **Active Storage dependency:** Lazy soft-detect via `defined?(::ActiveStorage::Blob)`. The gem does NOT add Active Storage as a hard dependency — apps that never use Image/File never need to install it. To use Image or File fields, add `gem "activestorage"` to your Gemfile (included in supported Rails versions via the `rails` meta-gem) and run `bin/rails active_storage:install` to create the `active_storage_blobs` / `active_storage_attachments` / `active_storage_variant_records` tables. The mirror precedent is `acts_as_tenant`, which is also soft-detected (see `Config::DEFAULT_SCOPE_RESOLVER`).
896
-
897
- - **`on_image_attached` hook:** Fires from `after_commit` on `TypedEAV::Value` when a `Field::Image`-typed Value's attachment is added or replaced. Receives `(value, blob)`. Configure via `TypedEAV.configure { |c| c.on_image_attached = ->(v, b) { ... } }`. Hook ordering: runs AFTER versioning (Phase 4) and AFTER `on_value_change` (Phase 3) so it sees the persisted version row and the user-callback context. File attachments do NOT fire this hook — the name is image-specific by design. Use `on_value_change` for a generic value-mutation signal that covers File-typed Values too.
898
-
899
- ```ruby
900
- TypedEAV.configure do |c|
901
- c.on_image_attached = ->(value, blob) {
902
- ProcessImageJob.perform_later(value.id, blob.id)
903
- }
904
- end
905
- ```
906
-
907
- - **`Reference`:** Foreign-key field type. Stores the target record's integer ID in `integer_value`. Operators: `:eq`, `:is_null`, `:is_not_null`, `:references` (explicit narrowing — does NOT inherit `:integer_value`'s `:gt`/`:lt`/`:between` defaults; arithmetic comparisons on FKs don't carry useful semantics). The `:references` operator accepts AR record instances OR Integer IDs at query time, normalizing via `field.cast` (a class-mismatched record routes to `base.none` rather than `:is_null`). Options: `target_entity_type` (REQUIRED — String class name of the target model, validated to constantize at field save), `target_scope` (OPTIONAL — when set, the field is REJECTED at save time if `target_entity_type` is not registered with `has_typed_eav scope_method:` (Gating Decision 2); when set with a scoped target, value-time validation rejects writes whose target's `typed_eav_scope` does not match `target_scope` via a `target_partition_matches?` helper structurally parallel to Phase 1's `entity_partition_axis_matches?` but on the target axis). Cross-scope safety mirrors the existing `Value#validate_field_scope_matches_entity` guard pattern applied to the target rather than the source.
908
-
909
- ```ruby
910
- rf = TypedEAV::Field::Reference.create!(
911
- name: "manager", entity_type: "Contact", scope: tenant_id,
912
- options: { target_entity_type: "Contact", target_scope: tenant_id },
913
- )
914
- TypedEAV::Value.create!(entity: alice, field: rf, value: bob) # accepts AR record
915
- TypedEAV::Value.create!(entity: alice, field: rf, value: bob.id) # accepts Integer FK
916
- Contact.where_typed_eav(name: "manager", op: :references, value: bob) # filter by record
917
- Contact.where_typed_eav(name: "manager", op: :references, value: 42) # filter by FK
918
- ```
919
-
920
- - **Summary:** The built-in field types **Image, File, Reference, Currency, Percentage** all preserve the cast-tuple contract (`[casted, invalid?]`), the operator-dispatch model (`supported_operators` + `operator_column` for multi-cell types), and the no-hardcoded-attribute-references foundational principle. The multi-cell extension surface (`read_value`, `write_value`, `apply_default`, and `operator_column`) is the canonical way to build any future external multi-cell field type.
921
-
922
- ## Validation Behavior
923
-
924
- A few non-obvious contracts worth knowing about up front:
925
-
926
- - **Required + blank**: `required: true` fields reject empty strings, whitespace-only strings, and arrays whose every element is nil/blank/whitespace.
927
- - **Array all-or-nothing cast**: integer/decimal/date arrays mark the **whole** value invalid (stored as `nil`) when any element fails to cast. There is no silent partial — a failed form re-renders with the original input intact so the user can correct the bad element.
928
- - **`Integer` array rejects fractional input**: `"1.9"` is rejected rather than truncated to `1`. Same rules as the scalar `Integer` field.
929
- - **`Json` parses string input**: a JSON string posted from a form is parsed; parse failures surface as `:invalid` rather than being stored as the literal string.
930
- - **`TextArray` does not support `:contains`**: it backs a jsonb column where SQL `LIKE` doesn't apply. Use `:any_eq` for "array contains element".
931
- - **Orphaned values are skipped**: if a field row is deleted while values remain, `typed_eav_value` and `typed_eav_hash` silently skip the orphans rather than raising.
932
- - **Cross-scope writes are rejected**: assigning a `Value` to a record whose `typed_eav_scope` doesn't match the field's `scope` adds a validation error on `:field`. The same guard covers the `parent_scope` axis.
933
- - **Orphan-parent rows rejected**: a `Field` or `Section` row with `parent_scope` set but `scope` blank is invalid. The `Value`-side guard rejects cross-`(scope, parent_scope)` writes too.
934
- - **Event hooks fire from `after_commit`**: the `on_value_change` and `on_field_change` callbacks fire after the database write is durable; their exceptions never break a save. See §"Event hooks" for the full contract.
935
- - **Versioning is opt-in**: When enabled (`TypedEAV.config.versioning = true` on the gem; `versioned: true` per host), every `:create` / `:update` / `:destroy` event on a Value writes an append-only audit row in `typed_eav_value_versions`. See §"Versioning" for the full contract.
936
- - **`label` is cosmetic, `name` is the machine key**: A field's optional `label` is free-text human display, independent of the slug `name`. Render via `display_name`, which returns `label` when present else `name.humanize`. `label` has no uniqueness or format constraints (only a 255-char max) and never affects ordering, lookup, partitioning, or rename detection — editing only `label` fires `on_field_change` with `:update`, never `:rename`. Existing rows (`label` NULL) render unchanged. Schema export round-trips the raw `label` (legacy payloads without a `label` key import as NULL); snapshot export carries the resolved `display_name`.
937
-
938
- ## Event hooks
939
-
940
- `typed_eav` fires `after_commit` events for value and field changes. Use them
941
- for audit logs, search-index synchronization, cache invalidation, or any
942
- out-of-band reaction that must wait until the database write is durable.
943
-
944
- ### Public callback slots
945
-
946
- ```ruby
947
- TypedEAV.configure do |c|
948
- c.on_value_change = ->(value, change_type, context) {
949
- # change_type ∈ [:create, :update, :destroy]
950
- # context is a frozen Hash (see `with_context` below) — read-only
951
- }
952
-
953
- c.on_field_change = ->(field, change_type) {
954
- # change_type ∈ [:create, :update, :destroy, :rename]
955
- # NOTE: no context arg — field changes are CRUD-on-config, not
956
- # per-entity user actions
957
- }
958
- end
959
- ```
960
-
961
- The `:rename` change_type fires whenever the field's `name` column changed
962
- in the just-committed save, even when bundled with other attribute changes
963
- (options, sort_order, default_value, etc.). The detection is intentionally
964
- escalating so any registered consumer receives a rename event whenever the
965
- persisted name changes.
966
-
967
- `:update` on Value fires only when the typed value column changed. Saving
968
- a Value record without modifying its typed column (e.g., touching only
969
- bookkeeping columns) is a no-op for event dispatch.
970
-
971
- `field_dependent: :nullify` cascades produce **no** Value `:destroy`
972
- events. The FK `ON DELETE SET NULL` runs at the database level and
973
- bypasses AR callbacks. Only the Field `:destroy` event fires. Use
974
- `field_dependent: :destroy` if your consumer needs per-Value events on
975
- field deletion.
976
-
977
- For a persisted `field_dependent: :destroy` field with a large population,
978
- call `field.destroy_with_values_in_batches!(batch_size: 1_000)` outside an
979
- open transaction. The opt-in API selects only that exact `field_id` in ordered
980
- primary-key batches, calls `Value#destroy!` for callback/version behavior, and
981
- commits each batch independently. A retry resumes from the remaining rows. The
982
- Field is retained until a locked, bounded residual drain proves zero rows, then
983
- its ordinary callback-preserving `destroy!` runs. The API rejects unsaved or
984
- non-destroy fields, open transactions, invalid batch sizes, and mismatched
985
- connection pools. Existing `destroy`/`destroy!`, `:nullify`, and `:restrict`
986
- behavior is unchanged.
987
-
988
- ### Thread-local context with `with_context`
989
-
990
- ```ruby
991
- TypedEAV.with_context(request_id: request.uuid, actor_id: current_user.id) do
992
- contact.update!(typed_eav: { phone: "555-1234" })
993
- # on_value_change receives { request_id: "...", actor_id: 42 } as context
994
- end
995
- ```
996
-
997
- `with_context` is a thread-local stack with shallow per-key merge:
998
-
999
- ```ruby
1000
- TypedEAV.with_context(request_id: "abc") do
1001
- TypedEAV.with_context(source: :bulk) do
1002
- # current context: { request_id: "abc", source: :bulk }
1003
- end
1004
- # current context: { request_id: "abc" }
1005
- end
1006
- # current context: {}
1007
- ```
1008
-
1009
- The current-context hash is frozen — callbacks cannot mutate it. Outer
1010
- context is restored on exit even if the inner block raises.
1011
-
1012
- `TypedEAV.current_context` returns the current frozen Hash (or a shared
1013
- frozen `{}` when no `with_context` block is active). It's safe to call
1014
- from any code path; it never returns nil.
1015
-
1016
- ### Error policy
1017
-
1018
- User callbacks (`Config.on_value_change`, `Config.on_field_change`) are
1019
- rescued — exceptions are logged via `Rails.logger.error` and **do not
1020
- propagate** to the user's save call. The save row is already committed
1021
- when `after_commit` fires; re-raising would surface a misleading
1022
- "save failed" error.
1023
-
1024
- This is the deliberate split with first-party features. Internal
1025
- observers used by `typed_eav` itself follow a different rule: their exceptions
1026
- **propagate**. Transactional version-writing errors are separate: they
1027
- propagate inside and roll back the source transaction.
1028
-
1029
- ### Ordering guarantee
1030
-
1031
- When multiple subscribers are registered, they fire in this order:
1032
-
1033
- 1. First-party generic observers, in registration order. Errors propagate.
1034
- 2. The user proc on `Config.on_value_change` / `Config.on_field_change`,
1035
- last. Errors are rescued and logged.
1036
-
1037
- Reassigning `Config.on_value_change` after gem initialization does **not**
1038
- disable internal subscribers — they live on a separate dispatcher list
1039
- and survive `Config.reset!`.
1040
-
1041
- ### Test isolation
1042
-
1043
- Test files that exercise event hooks should opt in to the `:event_callbacks`
1044
- metadata:
1045
-
1046
- ```ruby
1047
- RSpec.describe "my feature", :event_callbacks do
1048
- it "fires the hook" do
1049
- captured = []
1050
- TypedEAV::Config.on_value_change = ->(v, t, _ctx) { captured << [v.id, t] }
1051
- contact.update!(typed_eav: { phone: "555-1234" })
1052
- expect(captured).to include([be_a(Integer), :update])
1053
- end
1054
- end
1055
- ```
1056
-
1057
- The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshots and
1058
- restores Config user procs and the internal-subscriber lists around each
1059
- example, so test mutations don't leak across examples and engine-load
1060
- registrations from later phases stay intact.
1061
-
1062
- Integration specs that create real AR records and need `after_commit` to
1063
- fire durably should additionally opt in to `:real_commits`:
1064
-
1065
- ```ruby
1066
- RSpec.describe "my model", :event_callbacks, :real_commits do
1067
- # ...
1068
- end
1069
- ```
1070
-
1071
- `:real_commits` disables transactional fixtures for the example and
1072
- manually deletes typed_eav rows in FK order after.
1073
-
1074
- ### Reset semantics
1075
-
1076
- | Method | What it resets |
1077
- |---|---|
1078
- | `TypedEAV::Config.reset!` | User procs (`on_value_change`, `on_field_change`) plus `field_types`, `scope_resolver`, `require_scope`. Does **not** clear internal subscribers. |
1079
- | `TypedEAV::EventDispatcher.reset!` | Internal subscribers only. Does **not** touch Config. |
1080
-
1081
- Production code rarely calls either — they exist for test isolation and
1082
- for the rare case where a host app wants to fully unwire the gem in a
1083
- specific request lifecycle.
1084
-
1085
- ## Versioning
1086
-
1087
- `typed_eav` ships an opt-in append-only audit log for changes to typed
1088
- values. When enabled, each `:create` / `:update` / `:destroy` event on
1089
- a Value writes a row to `typed_eav_value_versions` capturing the
1090
- before-state, after-state, actor, context, and timestamp.
1091
-
1092
- Default off. Apps that don't enable it pay zero overhead — transactional
1093
- Value callbacks are not installed at boot
1094
- at all when `Config.versioning = false`. Zero callable in the dispatcher
1095
- chain, zero per-write method dispatch, zero per-write config read.
1096
-
1097
- ### Enabling versioning
1098
-
1099
- Two steps:
1100
-
1101
- ```ruby
1102
- # 1. Set the gem-level master switch in an initializer.
1103
- # config/initializers/typed_eav.rb
1104
- TypedEAV.configure do |c|
1105
- c.versioning = true
1106
- c.actor_resolver = -> { Current.user } # optional; nil is permissive
1107
- end
1108
-
1109
- # 2. Opt the host model in. Either via the kwarg form:
1110
- class Contact < ApplicationRecord
1111
- has_typed_eav scope_method: :tenant_id, versioned: true
1112
- end
1113
-
1114
- # Or via the concern (equivalent — pick whichever fits your conventions):
1115
- class Contact < ApplicationRecord
1116
- has_typed_eav scope_method: :tenant_id
1117
- include TypedEAV::Versioned
1118
- end
1119
- ```
1120
-
1121
- The two opt-in forms produce identical Registry state. The kwarg form is
1122
- preferred for new code; the concern form fits codebases with established
1123
- mixin-based feature wiring.
1124
-
1125
- ### Querying history
1126
-
1127
- ```ruby
1128
- contact.typed_eav_attributes = [{ name: "age", value: 41 }]
1129
- contact.save!
1130
- contact.typed_eav_attributes = [{ name: "age", value: 42 }]
1131
- contact.save!
1132
-
1133
- value = contact.typed_values.find_by(field: age_field)
1134
- value.history # most-recent-first relation
1135
- # => [<ValueVersion change_type: "update" before: {"integer_value" => 41} after: {"integer_value" => 42}>,
1136
- # <ValueVersion change_type: "create" before: {} after: {"integer_value" => 41}>]
1137
-
1138
- value.history.first.changed_by # => "42" (User#42 — coerced to id.to_s)
1139
- value.history.first.context # => { "request_id" => "abc-123" } if with_context was active
1140
- ```
1141
-
1142
- `value.history` is a chainable relation. Filter, paginate, pluck:
1143
-
1144
- ```ruby
1145
- value.history.where(change_type: "update").pluck(:changed_at, :changed_by)
1146
- value.history.limit(5).each { |v| ... }
1147
- ```
1148
-
1149
- ### Querying full audit history (including destroy events)
1150
-
1151
- `Value#history` returns versions where `value_id` matches the live Value
1152
- record. After the live Value is destroyed, the FK `ON DELETE SET NULL`
1153
- nullifies `value_id` on the existing version rows, and the new `:destroy`
1154
- version is written by the transactional destroy callback with `value_id: nil`
1155
- before the parent row is removed. So `Value#history`
1156
- cannot surface destroy versions, and after Value destruction it can no
1157
- longer be called at all.
1158
-
1159
- To query the FULL audit history for a given (entity, field), including
1160
- destroy events and post-destruction lookup, use the entity-scoped query
1161
- directly:
1162
-
1163
- ```ruby
1164
- TypedEAV::ValueVersion
1165
- .where(entity_type: contact.class.polymorphic_name, entity_id: contact.id, field_id: age_field.id)
1166
- .order(changed_at: :desc, id: :desc)
1167
- # => [<ValueVersion change_type: "destroy" before: {"integer_value" => 42} after: {} value_id: nil>,
1168
- # <ValueVersion change_type: "update" before: {"integer_value" => 41} after: {"integer_value" => 42} value_id: nil>,
1169
- # <ValueVersion change_type: "create" before: {} after: {"integer_value" => 41} value_id: nil>]
1170
- ```
1171
-
1172
- This pattern is the canonical way to surface "what happened to this
1173
- field on this entity" across the full lifecycle, including post-destroy.
1174
- The `entity_type` + `entity_id` columns remain the durable identity even
1175
- after the parent Value row is gone, and `field_id` survives because
1176
- destroying a Value does not destroy its Field.
1177
-
1178
- For broader audit views — "show all version history across all fields
1179
- for a given entity" (e.g., admin entity-history pages, compliance
1180
- exports) — drop the `field_id` filter:
1181
-
1182
- ```ruby
1183
- TypedEAV::ValueVersion
1184
- .where(entity_type: contact.class.polymorphic_name, entity_id: contact.id)
1185
- .order(changed_at: :desc, id: :desc)
1186
- # => all version rows for every typed field on this contact, most-recent-first.
1187
- # Includes :create, :update, and :destroy events across every field the
1188
- # entity has ever had a typed value for.
1189
- ```
1190
-
1191
- The field-scoped query (with `field_id:`) is the common case for
1192
- "history of a single field"; the entity-scoped query (without `field_id:`)
1193
- is the broad-audit case for "all version history across all fields for
1194
- this entity".
1195
-
1196
- ### Version row jsonb shape
1197
-
1198
- `before_value` and `after_value` are jsonb hashes keyed by typed-column
1199
- name:
1200
-
1201
- | Field type | Snapshot shape (single key) |
115
+ See [Field types and validation](docs/guides/fields.md) for options, casting
116
+ rules, attachments, sections, and custom single- or multi-column field types.
117
+
118
+ ## Essential Behavior
119
+
120
+ - **Scoping selects definitions, not host records.** Apply tenant and
121
+ authorization filters to the host relation. Scoped models require scope by
122
+ default; global definitions can be overridden by more specific partitions.
123
+ See [Multi-tenant scoping](docs/guides/scoping.md).
124
+ - **Missing rows and explicit NULLs differ.** Filtering, sorting, summaries,
125
+ and reads have documented rules for each. See
126
+ [Queries](docs/guides/queries.md) and [Bulk reads](docs/guides/bulk-operations.md#bulk-reads-bulkread).
127
+ - **The admin scaffold requires authorization.** Its generated controller
128
+ returns `404` until you implement `authorize_typed_eav_admin!` directly on
129
+ that controller. See [Admin scaffold](docs/guides/usage.md#admin-scaffold).
130
+ - **Versioning is opt-in.** Audit rows share the value transaction; public
131
+ change hooks run after commit. In-memory dirty tracking is separate from
132
+ audit history. See [Events and versioning](docs/guides/events-and-versioning.md)
133
+ and [Typed-value changes](docs/guides/usage.md#in-memory-typed-value-changes).
134
+ - **Bulk write APIs have different guarantees.** The regular writer saves
135
+ hosts with callbacks and validations. Fast upsert requires explicit
136
+ acknowledgment that host saves, persistence callbacks, and versioning are
137
+ skipped. See [Bulk operations](docs/guides/bulk-operations.md).
138
+
139
+ ## Documentation
140
+
141
+ Start at the [documentation home](docs/index.md), or browse the
142
+ [public API and configuration reference](docs/reference/index.md).
143
+
144
+ | Guide | Covers |
1202
145
  |---|---|
1203
- | `text`, `email`, `url`, `color` | `{"string_value": "..."}` |
1204
- | `long_text` | `{"text_value": "..."}` |
1205
- | `integer` | `{"integer_value": 42}` |
1206
- | `decimal` | `{"decimal_value": "10.5"}` |
1207
- | `boolean` | `{"boolean_value": true}` |
1208
- | `date` | `{"date_value": "2026-05-05"}` |
1209
- | `date_time` | `{"datetime_value": "2026-05-05T12:00:00Z"}` |
1210
- | `select` | `{"string_value": "..."}` |
1211
- | `multi_select`, `*_array`, `json` | `{"json_value": [...]}` |
1212
-
1213
- Multi-cell field types (e.g., `Currency`) produce two-key snapshots:
1214
- `{"decimal_value": "99.99", "string_value": "USD"}`. The version row's
1215
- snapshot asks the field's storage contract for its cells, so new field
1216
- types get the right shape automatically.
1217
-
1218
- `{}` (empty hash) and `{"<col>": null}` are distinct semantics:
1219
-
1220
- - `{}` means **no recorded value** — typical of `before_value` on a
1221
- `:create` event, or `after_value` on a `:destroy` event.
1222
- - `{"<col>": null}` means **recorded nil** — the user explicitly
1223
- cleared the cell.
1224
-
1225
- ### Reverting
1226
-
1227
- ```ruby
1228
- target = value.history.find_by(change_type: "update")
1229
- value.revert_to(target)
1230
- # value's typed columns now match target.before_value.
1231
- # A NEW version row is written capturing the revert (append-only).
1232
- ```
1233
-
1234
- `revert_to` writes the targeted version's `before_value` columns back
1235
- via `self[col] = …` and `save!`. The transactional version callback writes a
1236
- NEW version row whose
1237
- `after_value` reflects the targeted version's `before_value`. The
1238
- audit log is append-only — every revert is itself versioned.
1239
-
1240
- To record the intent of the revert, wrap the call in `with_context`:
1241
-
1242
- ```ruby
1243
- TypedEAV.with_context(reverted_from_version_id: target.id, actor: current_user) do
1244
- value.revert_to(target)
1245
- end
1246
- # The new version row's `context` column captures both keys.
1247
- ```
1248
-
1249
- `revert_to` raises `ArgumentError` in three documented conditions, checked in order:
1250
-
1251
- - when `version.value_id` is nil (the source Value was destroyed — destroy
1252
- versions have `value_id: nil` per the locked subscriber contract; you
1253
- can't restore a destroyed AR record by `save!`);
1254
- - when the version's `before_value` is empty (the version represents a
1255
- `:create` event with no before-state to revert to);
1256
- - when the version belongs to a different Value (`value_id` mismatch).
1257
-
1258
- In practice only `:update` versions are revertable. To restore a
1259
- destroyed entity's typed values, create a new `TypedEAV::Value` record
1260
- manually using `version.before_value` as the seed state.
1261
-
1262
- ### Hook ordering guarantee
1263
-
1264
- Versioning is installed as boot-latched transactional callbacks on `Value`,
1265
- and the public callback remains an after-commit observer. The version row is
1266
- written in the source transaction.
1267
- ```
1268
- Value#save! → transactional Value callback → ValueVersion.create!
1269
- → after_commit → EventDispatcher.dispatch_value_change:
1270
- 1. ... any other generic internal observers ...
1271
- 2. Config.on_value_change user proc # sees the persisted version
1272
- ```
1273
-
1274
- Internal observer errors propagate. Transactional version-writing errors also
1275
- propagate inside and roll back the source transaction.
1276
- User proc errors are rescued and logged via `Rails.logger.error` —
1277
- the save itself already committed.
1278
-
1279
- ### Actor resolution
1280
-
1281
- `Config.actor_resolver` mirrors `Config.scope_resolver`'s callable shape
1282
- but returns whatever the app chooses (an AR record, a string, an integer,
1283
- nil). The subscriber coerces non-nil returns via `id.to_s` (for AR
1284
- records) or `to_s` (for scalars) before storing in the `changed_by`
1285
- column (string, nullable).
1286
-
1287
- `nil` is the documented permissive sentinel: system writes, migrations,
1288
- console-without-actor, and background jobs without a `with_context(actor:
1289
- ...)` wrap all flow through with `changed_by: nil`. This is intentional —
1290
- forcing every Versioned write to have an actor would reject every console
1291
- save and every migration backfill, which is hostile-by-default for a gem.
1292
-
1293
- Apps that need stricter enforcement do it inside the resolver:
1294
-
1295
- ```ruby
1296
- c.actor_resolver = -> { Current.user || raise(MyApp::ActorRequired) }
1297
- ```
1298
-
1299
- `Config.reset!` (documented in §"Event hooks") also resets `Config.versioning`
1300
- to `false` and `Config.actor_resolver` to `nil`.
1301
-
1302
- ### What versioning does not do
1303
-
1304
- - **No branching/merging across version chains.** Phase 4 ships event-log
1305
- shape only. Roadmap explicitly defers branching to a future design.
1306
- - **No snapshot storage by default.** `typed_eav_value_versions` is an
1307
- event log — one row per change, not a full-row snapshot. For
1308
- high-volume apps that want snapshot storage, extend `ValueVersion` in
1309
- your own code (the gem keeps the event-log shape canonical so future
1310
- upgrades don't break your extension).
1311
- - **No automatic `reverted_from_version_id` injection.** Use
1312
- `with_context` to record revert intent; the gem captures whatever
1313
- context the caller set.
1314
- - **No per-Field versioning toggle.** Opt-in is per-entity (host model)
1315
- in Phase 4. Per-field granularity may land later if a real need
1316
- surfaces.
1317
- - **No GIN indexes on `before_value` / `after_value` content.** Apps
1318
- that need to query inside the snapshot jsonb add their own indexes.
1319
- Phase 4 ships only the temporal indexes (`changed_at DESC` keyed on
1320
- `value_id`, `(entity_type, entity_id)`, and `field_id`).
1321
-
1322
- ### Test isolation
1323
-
1324
- Specs that exercise versioning should opt into the `:event_callbacks`
1325
- and `:real_commits` metadata flags (see §"Event hooks" — same pattern):
1326
-
1327
- ```ruby
1328
- RSpec.describe "my versioning behavior", :event_callbacks, :real_commits do
1329
- before do
1330
- TypedEAV.registry.register("Contact", versioned: true)
1331
- TypedEAV::Config.versioning = true
1332
- # Transactional Value callbacks are boot-latched and remain installed;
1333
- # the hook isolates only public and generic EventDispatcher observers.
1334
- end
1335
- after { TypedEAV.registry.register("Contact", versioned: false) }
1336
-
1337
- it "writes a version row" do
1338
- # ...
1339
- end
1340
- end
1341
- ```
1342
-
1343
- The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshots and
1344
- restores `Config.versioning`, `Config.actor_resolver`, and generic
1345
- EventDispatcher observer lists around each example. Transactional Value
1346
- callback installation is tested independently through callback-chain and
1347
- boot-latch specs. The
1348
- `:real_commits` hook disables transactional fixtures (so `after_commit`
1349
- fires durably) and cleans up `TypedEAV::ValueVersion` rows in
1350
- FK-respecting order between examples.
1351
-
1352
- ## Database Support
1353
-
1354
- Requires PostgreSQL. The `text_pattern_ops` index on `string_value` and the jsonb `@>` containment operator are Postgres-specific. MySQL/SQLite support would require removing those index types and changing the array query operators.
1355
-
1356
- As of v0.2.0, the paired partial unique indexes cover the three-key partition tuple `(entity_type, scope, parent_scope)`. The orphan-parent invariant means the `WHERE scope IS NULL` partials don't include `parent_scope` — a global row always has `parent_scope` NULL too.
1357
-
1358
- ## Schema
1359
-
1360
- The gem creates five tables:
1361
-
1362
- - `typed_eav_fields` - field definitions (STI, one row per field per entity type)
1363
- - `typed_eav_values` - values (one row per entity per field, with typed columns)
1364
- - `typed_eav_options` - allowed values for select/multi-select fields
1365
- - `typed_eav_sections` - optional UI grouping
1366
- - `typed_eav_value_versions` - opt-in, append-only audit history for Value
1367
- create, update, and destroy events; it retains durable entity identity even
1368
- when the live Value row is later removed
1369
-
1370
- ## Architecture
1371
-
1372
- Internal module layout as of 0.7.0. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by ADR-0001 through ADR-0013.
1373
-
1374
- ### Macro entry: `HasTypedEav`
1375
-
1376
- `lib/typed_eav/has_typed_eav.rb` (~120 LOC) is the macro shell. When you call `has_typed_eav` on an AR model, it:
1377
-
1378
- 1. `extend`s `TypedEAV::EntityQuery` onto the class (class-level query methods).
1379
- 2. `include`s `TypedEAV::HasTypedEav::InstanceMethods` (per-record accessors).
1380
- 3. Wires scope/parent-scope kwargs into the model's class-level configuration.
1381
- 4. Registers the model with `TypedEAV::Registry`.
1382
-
1383
- The macro is intentionally thin. All real behavior lives in the modules it pulls in.
1384
-
1385
- ### Class-level reads: two-altitude query pattern
1386
-
1387
- ```
1388
- Contact.where_typed_eav(...) ← public class method
1389
- │
1390
- ▼
1391
- TypedEAV::EntityQuery ← high altitude: orchestrator
1392
- • resolves scope/parent_scope from ambient context or explicit kwargs
1393
- • owns the UNSET_SCOPE / ALL_SCOPES sentinels
1394
- • delegates to FilterQuery
1395
- │
1396
- ▼
1397
- TypedEAV::FilterQuery ← multi-filter composition
1398
- • normalizes filter input shapes (positional, hash, hash-of-hashes)
1399
- • looks up field definitions via TypedEAV::Partition
1400
- • per filter, asks QueryBuilder for the SQL fragment
1401
- • unions/intersects per-field entity-id sets
1402
- • returns an ActiveRecord::Relation scoped to the host model
1403
- │
1404
- ▼
1405
- TypedEAV::QueryBuilder ← low altitude: per-field SQL primitive
1406
- • turns a single (field, op, value) into a WHERE clause against typed_eav_values
1407
- • knows about typed-column projections (integer_value, string_value, etc.)
1408
- • knows about operator-specific column choice (currency-cents vs currency-code)
1409
- ```
1410
-
1411
- `QueryBuilder` is the single place that decides "given this field and this operator, which column and which SQL fragment?" `FilterQuery` never builds SQL fragments directly; `EntityQuery` never touches columns. Splitting the two altitudes keeps custom field types extending only the column-mapping surface (`value_column`, `operators`, `operator_column`) without ever subclassing `FilterQuery`.
1412
-
1413
- ### Bulk reads: `BulkRead`
1414
-
1415
- `typed_eav_hash_for(records)` (the plural read) routes through `TypedEAV::BulkRead`. Given a record collection and an effective `(scope, parent_scope)`, it:
1416
-
1417
- 1. Resolves visible definitions and groups the requested field IDs.
1418
- 2. Loads definitions, values, and field associations through one batched
1419
- definition query, one values query, and one field-association preload
1420
- (three SQL queries total; no host-table query).
1421
- 3. Returns a `{record_id => {field_name => value}}` map while skipping orphaned
1422
- values and preserving logical missingness.
1423
-
1424
- Definitions, filters, reads, registry entries, and writes all use the host's
1425
- Rails `polymorphic_name`, so an STI leaf class reads and queries the same rows
1426
- written under its base-class polymorphic type.
1427
-
1428
- The final production characterization reduced the 1,002 SQL statements observed
1429
- across 1,000 scopes to three for the same BulkRead shape. This is a statement-
1430
- count result, not a representative throughput claim; applications should still
1431
- measure their own scope cardinality, selected fields, hydration, and contention.
1432
-
1433
- Single-record reads (`typed_eav_value`, `typed_eav_hash`) live on `InstanceMethods` and use the same partition helpers but without batching.
1434
-
1435
- ### Bulk writes: `BulkWrite`
1436
-
1437
- `bulk_set_typed_eav_values(records, attrs)` routes through `TypedEAV::BulkWrite`,
1438
- and `bulk_set_typed_eav_values_per_record(values_by_record)` is its sibling for
1439
- record-varying hashes. Both are semantic writers that:
1440
-
1441
- 1. Memoizes field definitions for the call via `Thread.current[:typed_eav_bulk_defs_memo]`.
1442
- 2. Validates each attribute against its field type's cast contract.
1443
- 3. Saves each host through the normal callback/validation path inside an outer
1444
- transaction with per-record savepoints.
1445
-
1446
- `bulk_set_typed_eav_values_per_record` uses records as Hash keys, so two AR
1447
- instances of the same persisted row collapse to one entry; sequence separate
1448
- calls for two ordered updates, while the uniform Array API preserves duplicate
1449
- instances and caller order.
1450
-
1451
- Under `transaction: :all`, per-record validation failures are captured at their
1452
- savepoints while other successes can commit, but an uncaught exception rolls
1453
- the entire outer transaction back. The default `transaction: :all` commits the
1454
- whole successful batch or rolls it back;
1455
- `transaction: :chunks, chunk_size: N` commits completed chunks while isolating
1456
- later failures, preserving earlier committed chunks. Both forms require the
1457
- host, Field, and Value pools to match.
1458
- `bulk_upsert_typed_eav_values` is a separate reduced-semantics fast path: it
1459
- casts and validates typed values, then performs one PostgreSQL upsert while
1460
- omitting host saves, persistence callbacks, delete shorthand, and versioning.
1461
-
1462
- Callers must pass `acknowledge_reduced_semantics: true`. The same values hash
1463
- applies to every record; records must be persisted and unique, and string or
1464
- symbol field keys that normalize to the same name are rejected. The return
1465
- value is the integer number of value rows upserted, not a semantic
1466
- `successes`/`errors_by_record` result. Value casting, domain/entity/partition
1467
- checks, and Value validation callbacks remain; host callbacks and validations,
1468
- Value persistence callbacks, versioning, delete shorthand, and per-record
1469
- savepoint isolation are skipped.
1470
-
1471
- Within each `transaction: :all` unit—or each requested chunk—the upsert path
1472
- resolves every record partition through one batched field-definition SELECT.
1473
- It shares BulkRead's internal tuple resolver, retaining global, scope-only, and
1474
- full-tuple precedence independently for each record without broadening tenant
1475
- visibility.
1476
-
1477
- `BulkWrite` and `BulkRead` are siblings — one read path, one write path — but they don't share a base class. Per [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md), keeping them independent preserves the option to evolve each on its own schedule.
1478
-
1479
- ### Per-record reads/writes: `InstanceMethods`
1480
-
1481
- `lib/typed_eav/has_typed_eav/instance_methods.rb` (~250 LOC) holds the per-record API:
1482
-
1483
- - `typed_eav_value(name)` / `typed_eav_hash` — reads
1484
- - `set_typed_eav_value(name, value)` / `typed_eav_attributes=` (aliased as `typed_eav=`) — writes
1485
- - `typed_eav_definitions` — resolved field-definitions map for the host record
1486
- - `typed_eav_scope` / `typed_eav_parent_scope` — scope resolution per record
1487
-
1488
- Every method uses `TypedEAV::Partition.definitions_by_name` so the collision-precedence rules for ambient/explicit/parent scopes are computed in one place.
1489
-
1490
- ### Partition visibility: `Partition`
1491
-
1492
- Host applications that need to inspect effective schema should use the
1493
- documented-public `TypedEAV::Partition` seam rather than rebuilding tuple
1494
- predicates. It exposes `visible_fields`, `effective_fields_by_name`,
1495
- `definitions_by_name`, `definitions_multimap_by_name`, `visible_sections`,
1496
- and `find_visible_section!`. These methods preserve global, scope-only, and
1497
- full-tuple precedence; ADR-0006 additionally fixes include-missing set
1498
- composition at the `FilterQuery` altitude.
1499
-
1500
- ### Field types and storage: `Field::TypedStorage`
1501
-
1502
- `TypedEAV::Field::Base` is the STI parent of every field type. The shared storage surface lives in the `TypedEAV::Field::TypedStorage` concern (`lib/typed_eav/field/typed_storage.rb`, ~200 LOC), auto-included on `Field::Base`. Per [ADR-0001](docs/adr/0001-collapse-column-mapping-stack.md), it provides:
1503
-
1504
- - **Class DSL**: `value_column`, `value_columns`, `operators`, `operator_column`, `supported_operators` — describe where typed values live and which operators they support.
1505
- - **Instance override points**: `read_value(record)`, `write_value(record, casted)`, `apply_default(record)` — the three methods a multi-cell field type overrides.
1506
- - **Concrete snapshot helpers**: `value_changed?`, `before_snapshot`, `after_snapshot` — derived automatically from `value_columns`; not overridable.
1507
-
1508
- Custom multi-cell field types subclass `Field::Base` directly and override only the three instance methods. See §[Multi-cell field types](#multi-cell-field-types) for `Currency` as the canonical worked example.
1509
-
1510
- ### Field families: intermediate STI bases
1511
-
1512
- Per [ADR-0004](docs/adr/0004-field-family-intermediate-bases.md), three intermediate STI parents factor shared validation behavior out of `Field::Base`:
1513
-
1514
- - **`TypedEAV::Field::ValidatedString`** — parent of `Text`, `Email`, `Url`. Owns string-length and pattern-validation helpers including `max_gte_min_length` (which now covers Email/Url, not just Text).
1515
- - **`TypedEAV::Field::RangeBounded`** — parent of `Integer`, `Decimal`, `Date`, `DateTime` (and `Percentage < Decimal`). Owns range-validation helpers including `validates :max, comparison: { greater_than_or_equal_to: :min }` (which now covers Date/DateTime, not just Integer/Decimal).
1516
- - **`TypedEAV::Field::Optionable`** — a Rails concern included by `Select` and `MultiSelect`. Owns the public-facing sorted `allowed_values` reader and the option-inclusion validators.
1517
-
1518
- `Color`, `Boolean`, `Json`, and the array field types (`TextArray`, `IntegerArray`, `DecimalArray`, `DateArray`) remain direct children of `Field::Base`. See §[Family intermediate bases](#family-intermediate-bases-extension-points) for extension examples.
1519
-
1520
- ### Scope tuple normalization: `ScopeTuple`
1521
-
1522
- `TypedEAV::ScopeTuple` (`lib/typed_eav/scope_tuple.rb`, ~120 LOC) is the canonical source of truth for the `(scope, parent_scope)` partition tuple. It provides:
1523
-
1524
- - `normalize_permissive(scope)` — coerces input to a tuple; tolerates bare scalars (used by `with_scope`, `normalize_scope`, `Field#validate_parent_scope_invariant`).
1525
- - `normalize_strict(scope)` — same shape, but raises on bare-scalar input (used by `current_scope`; preserves Phase-1's asymmetric contract that `Config.scope_resolver` must return a tuple).
1526
- - `invariant_satisfied?(scope, parent_scope)` — Boolean check for the orphan-parent invariant (`parent_scope` set without `scope` = invalid).
1527
-
1528
- Each calling site keeps its own response policy (raise / AR error / silent narrow) using the Boolean return — `ScopeTuple` is a predicate, not an enforcer.
1529
-
1530
- ### Partition tuple helpers: `Partition`
1531
-
1532
- `TypedEAV::Partition` (`lib/typed_eav/partition.rb`, ~100 LOC) owns the `(entity_type, scope, parent_scope)` precedence rules:
1533
-
1534
- - `definitions_by_name(model, scope, parent_scope)` — returns the field-definitions map for a single resolved partition.
1535
- - `definitions_multimap_by_name(model)` — returns the cross-partition multimap used by `unscoped { }` blocks.
1536
- - `visible_fields(model, scope, parent_scope)` / `visible_sections(...)` — scope-respecting field/section iteration with the orphan-parent invariant inlined via `ScopeTuple.invariant_satisfied?`.
1537
-
1538
- The definitions helpers used to live as class methods on `HasTypedEav` before 0.3.0. They moved to `Partition` per [ADR-0002](docs/adr/0002-entity-query-orchestration.md) because they describe the partition domain, not the macro.
1539
-
1540
- ### Events: `EventDispatcher`
1541
-
1542
- `TypedEAV::EventDispatcher` (`lib/typed_eav/event_dispatcher.rb`, ~150 LOC) is the broker for `on_value_change` and `on_field_change` callbacks. Per [ADR-0003](docs/adr/0003-keep-event-dispatcher-broker.md), it intentionally stays a broker rather than getting absorbed into either `Value` or `Field` — its multi-publisher / multi-subscriber shape doesn't belong on either model. See §[Event hooks](#event-hooks) for the public callback contract.
1543
-
1544
- ### Schema portability and CSV: independent modules
1545
-
1546
- `TypedEAV::SchemaPortability` and `TypedEAV::CSVMapper` (Phase-6 modules) are deliberately decoupled from the core read/write path per [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md). They depend on the public `has_typed_eav` macro surface, never on internal modules.
1547
-
1548
- ### Bulk operation guarantees
1549
-
1550
- `bulk_upsert_typed_eav_values` is an explicit reduced-semantics API: it
1551
- prevalidates/casts values and performs a PostgreSQL upsert, while intentionally
1552
- omitting host callbacks and versioning. Use the regular bulk writer when those
1553
- semantics are required; chunked semantic transactions are opt-in.
1554
-
1555
- The fast path still casts and runs domain, entity, partition, and validation
1556
- callbacks before its single upsert against the exact entity/field conflict
1557
- target; it omits host saves/host callbacks, Value persistence callbacks,
1558
- delete shorthand, and versioning. It requires one shared connection pool and
1559
- returns validation errors before SQL. `:all` is one unit; `:chunks` commits
1560
- completed chunks before a later failure. Semantic writes retain host saves,
1561
- per-record savepoint/error isolation, and one outer `:all` transaction.
1562
-
1563
- BulkWrite evidence is intentionally bounded to the exercised 100- and 1,000-host
1564
- lanes. It does not establish 10,000- or 100,000-host throughput, nor does it
1565
- justify a universal batch size or storage choice.
1566
-
1567
- ### Operational guarantees
1568
-
1569
- The semantic writer preserves the caller's transaction and callback/versioning
1570
- contract. Version rows are written in the source transaction, so a rollback
1571
- rolls back the Value mutation and its audit row together. The reduced-semantics
1572
- upsert is intentionally separate and does not claim those callbacks or audit
1573
- guarantees.
1574
-
1575
- Field deletion has a callback-preserving, keyset-batched path that locks and
1576
- destroys only the exact field's Values before bounded finalization. It scales by
1577
- bounded primary-key batches and preserves the Field if a batch fails; it is not
1578
- a claim of unbounded deletion throughput.
1579
-
1580
- ### Default backfill narrowing
1581
-
1582
- `Field::Base#backfill_default!` optionally accepts an exact-host
1583
- `ActiveRecord::Relation` to SQL-narrow eligible entities before batching. The default
1584
- all-host behavior remains unchanged; partition checks, batch transactions,
1585
- callbacks, validations, idempotence, versions, and errors remain in force.
1586
- Typed storage defines logical missingness across all declared cells, so a
1587
- partially populated multi-cell value is present while a fully empty Currency
1588
- value is missing.
146
+ | [Reading, writing, and forms](docs/guides/usage.md) | Assignment by name or ID, nested forms, admin scaffold, pending and saved changes |
147
+ | [Querying typed fields](docs/guides/queries.md) | Operators, filter composition, sorting, distinct values, counts, aggregates |
148
+ | [Multi-tenant scoping](docs/guides/scoping.md) | Resolvers, two-level partitions, precedence, background jobs, administrative queries |
149
+ | [Field types and validation](docs/guides/fields.md) | Built-in types, options, sections, attachments, custom types, validation contracts |
150
+ | [Bulk operations](docs/guides/bulk-operations.md) | Selective and preloaded reads, semantic writes, fast upsert, transactions, default backfills |
151
+ | [Events and versioning](docs/guides/events-and-versioning.md) | Callbacks, context, actor resolution, audit history, reverting |
152
+ | [CSV mapping](docs/guides/csv-import.md) | Header/index mapping, typed casting, row errors, saving mapped values |
153
+ | [Database schema and portability](docs/guides/schema.md) | Tables, PostgreSQL requirements, read-only schema previews |
154
+ | [Storage and performance](docs/guides/performance.md) | JSONB tradeoffs, optional indexes and statistics, query strategy, evidence limits |
155
+
156
+ ## Development and Upgrades
157
+
158
+ - [Architecture](docs/guides/architecture.md) and [design decisions](docs/adr/)
159
+ - [Development and test isolation](docs/guides/development.md)
160
+ - [Benchmark guide](bench/README.md)
161
+ - [Upgrading](docs/guides/upgrading.md) and [Changelog](CHANGELOG.md)
162
+ - [Release process](RELEASING.md)
1589
163
 
1590
164
  ## License
1591
165