typed_eav 0.8.0 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +25 -2
- data/README.md +79 -1710
- data/RELEASING.md +81 -0
- data/docs/adr/0001-collapse-column-mapping-stack.md +28 -0
- data/docs/adr/0002-entity-query-orchestration.md +33 -0
- data/docs/adr/0003-keep-event-dispatcher-broker.md +41 -0
- data/docs/adr/0004-field-family-intermediate-bases.md +49 -0
- data/docs/adr/0005-keep-phase-six-modules-independent.md +48 -0
- data/docs/adr/0006-include-missing-via-set-complement.md +77 -0
- data/docs/adr/0007-visibility-versus-mutation-relations.md +33 -0
- data/docs/adr/0008-partial-covering-scalar-indexes.md +83 -0
- data/docs/adr/0009-string-search-indexing.md +110 -0
- data/docs/adr/0010-planner-statistics-policy.md +109 -0
- data/docs/adr/0011-multi-filter-query-strategy.md +111 -0
- data/docs/adr/0012-cross-scope-administrative-query-policy.md +76 -0
- data/docs/adr/0013-durable-versioning-and-field-deletion.md +125 -0
- data/docs/adr/index.md +101 -0
- data/docs/getting-started.md +79 -0
- data/docs/guides/architecture.md +125 -0
- data/docs/guides/bulk-operations.md +205 -0
- data/docs/guides/csv-import.md +88 -0
- data/docs/guides/development.md +73 -0
- data/docs/guides/events-and-versioning.md +360 -0
- data/docs/guides/fields.md +342 -0
- data/docs/guides/performance.md +107 -0
- data/docs/guides/queries.md +188 -0
- data/docs/guides/schema.md +134 -0
- data/docs/guides/scoping.md +254 -0
- data/docs/guides/upgrading.md +26 -0
- data/docs/guides/usage.md +259 -0
- data/docs/index.md +44 -0
- data/docs/maintaining.md +82 -0
- data/docs/reference/api.md +133 -0
- data/docs/reference/configuration.md +64 -0
- data/docs/reference/index.md +16 -0
- data/lib/typed_eav/version.rb +1 -1
- metadata +35 -1
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Architecture"
|
|
3
|
+
nav_group: Project
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Architecture
|
|
7
|
+
|
|
8
|
+
[Documentation home](../index.md)
|
|
9
|
+
|
|
10
|
+
Current internal module layout. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by ADR-0001 through ADR-0013.
|
|
11
|
+
|
|
12
|
+
## Macro entry: `HasTypedEav`
|
|
13
|
+
|
|
14
|
+
`lib/typed_eav/has_typed_eav.rb` (~120 LOC) is the macro shell. When you call `has_typed_eav` on an AR model, it:
|
|
15
|
+
|
|
16
|
+
1. `extend`s `TypedEAV::EntityQuery` onto the class (class-level query methods).
|
|
17
|
+
2. `include`s `TypedEAV::HasTypedEav::InstanceMethods` (per-record accessors).
|
|
18
|
+
3. Wires scope/parent-scope kwargs into the model's class-level configuration.
|
|
19
|
+
4. Registers the model with `TypedEAV::Registry`.
|
|
20
|
+
|
|
21
|
+
The macro is intentionally thin. All real behavior lives in the modules it pulls in.
|
|
22
|
+
|
|
23
|
+
## Class-level reads: two-altitude query pattern
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
Contact.where_typed_eav(...) ← public class method
|
|
27
|
+
│
|
|
28
|
+
▼
|
|
29
|
+
TypedEAV::EntityQuery ← high altitude: orchestrator
|
|
30
|
+
• resolves scope/parent_scope from ambient context or explicit kwargs
|
|
31
|
+
• owns the UNSET_SCOPE / ALL_SCOPES sentinels
|
|
32
|
+
• delegates to FilterQuery
|
|
33
|
+
│
|
|
34
|
+
▼
|
|
35
|
+
TypedEAV::FilterQuery ← multi-filter composition
|
|
36
|
+
• normalizes filter input shapes (positional, hash, hash-of-hashes)
|
|
37
|
+
• looks up field definitions via TypedEAV::Partition
|
|
38
|
+
• per filter, asks QueryBuilder for the SQL fragment
|
|
39
|
+
• unions/intersects per-field entity-id sets
|
|
40
|
+
• returns an ActiveRecord::Relation scoped to the host model
|
|
41
|
+
│
|
|
42
|
+
▼
|
|
43
|
+
TypedEAV::QueryBuilder ← low altitude: per-field SQL primitive
|
|
44
|
+
• turns a single (field, op, value) into a WHERE clause against typed_eav_values
|
|
45
|
+
• knows about typed-column projections (integer_value, string_value, etc.)
|
|
46
|
+
• knows about operator-specific column choice (currency-cents vs currency-code)
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
`QueryBuilder` is the single place that decides "given this field and this operator, which column and which SQL fragment?" `FilterQuery` never builds SQL fragments directly; `EntityQuery` never touches columns. Splitting the two altitudes keeps custom field types extending only the column-mapping surface (`value_column`, `operators`, `operator_column`) without ever subclassing `FilterQuery`.
|
|
50
|
+
|
|
51
|
+
Scalar ordering and summaries are a separate `EntityQuery` delegation to
|
|
52
|
+
`ScalarQuery`: it resolves one winning definition, checks scalar support, and
|
|
53
|
+
builds SQL over the field's declared native column. It does not add operators
|
|
54
|
+
to the filter DSL or load the host/Value graph to calculate summaries.
|
|
55
|
+
|
|
56
|
+
## Per-record reads/writes: `InstanceMethods`
|
|
57
|
+
|
|
58
|
+
`lib/typed_eav/has_typed_eav/instance_methods.rb` (~250 LOC) holds the per-record API:
|
|
59
|
+
|
|
60
|
+
- `typed_eav_value(name)` / `typed_eav_hash` — reads
|
|
61
|
+
- `set_typed_eav_value(name, value)` / `typed_eav_attributes=` (aliased as `typed_eav=`) — writes
|
|
62
|
+
- `typed_eav_definitions` — resolved field-definitions map for the host record
|
|
63
|
+
- `typed_eav_scope` / `typed_eav_parent_scope` — scope resolution per record
|
|
64
|
+
|
|
65
|
+
Every method uses `TypedEAV::Partition.definitions_by_name` so the collision-precedence rules for ambient/explicit/parent scopes are computed in one place.
|
|
66
|
+
|
|
67
|
+
## Partition visibility: `Partition`
|
|
68
|
+
|
|
69
|
+
Host applications that need to inspect effective schema should use the
|
|
70
|
+
documented-public `TypedEAV::Partition` seam rather than rebuilding tuple
|
|
71
|
+
predicates. It exposes `visible_fields`, `effective_fields_by_name`,
|
|
72
|
+
`definitions_by_name`, `definitions_multimap_by_name`, `visible_sections`,
|
|
73
|
+
and `find_visible_section!`. These methods preserve global, scope-only, and
|
|
74
|
+
full-tuple precedence; ADR-0006 additionally fixes include-missing set
|
|
75
|
+
composition at the `FilterQuery` altitude.
|
|
76
|
+
|
|
77
|
+
## Field types and storage: `Field::TypedStorage`
|
|
78
|
+
|
|
79
|
+
`TypedEAV::Field::Base` is the STI parent of every field type. The shared storage surface lives in the `TypedEAV::Field::TypedStorage` concern (`lib/typed_eav/field/typed_storage.rb`, ~200 LOC), auto-included on `Field::Base`. Per [ADR-0001](../adr/0001-collapse-column-mapping-stack.md), it provides:
|
|
80
|
+
|
|
81
|
+
- **Class DSL**: `value_column`, `value_columns`, `operators`, `operator_column`, `supported_operators` — describe where typed values live and which operators they support.
|
|
82
|
+
- **Instance override points**: `read_value(record)`, `write_value(record, casted)`, `apply_default(record)` — the three methods a multi-cell field type overrides.
|
|
83
|
+
- **Concrete snapshot helpers**: `value_changed?`, `before_snapshot`, `after_snapshot` — derived automatically from `value_columns`; not overridable.
|
|
84
|
+
|
|
85
|
+
Custom multi-cell field types subclass `Field::Base` directly and override only the three instance methods. See [Multi-cell field types](fields.md#multi-cell-field-types) for `Currency` as the canonical worked example.
|
|
86
|
+
|
|
87
|
+
## Field families: intermediate STI bases
|
|
88
|
+
|
|
89
|
+
Per [ADR-0004](../adr/0004-field-family-intermediate-bases.md), three intermediate STI parents factor shared validation behavior out of `Field::Base`:
|
|
90
|
+
|
|
91
|
+
- **`TypedEAV::Field::ValidatedString`** — parent of `Text`, `Email`, `Url`. Owns string-length and pattern-validation helpers including `max_gte_min_length` (which now covers Email/Url, not just Text).
|
|
92
|
+
- **`TypedEAV::Field::RangeBounded`** — parent of `Integer`, `Decimal`, `Date`, `DateTime` (and `Percentage < Decimal`). Owns range-validation helpers including `validates :max, comparison: { greater_than_or_equal_to: :min }` (which now covers Date/DateTime, not just Integer/Decimal).
|
|
93
|
+
- **`TypedEAV::Field::Optionable`** — a Rails concern included by `Select` and `MultiSelect`. Owns the public-facing sorted `allowed_values` reader and the option-inclusion validators.
|
|
94
|
+
|
|
95
|
+
`Color`, `Boolean`, `Json`, and the array field types (`TextArray`, `IntegerArray`, `DecimalArray`, `DateArray`) remain direct children of `Field::Base`. See [Family intermediate bases](fields.md#family-intermediate-bases-extension-points) for extension examples.
|
|
96
|
+
|
|
97
|
+
## Scope tuple normalization: `ScopeTuple`
|
|
98
|
+
|
|
99
|
+
`TypedEAV::ScopeTuple` (`lib/typed_eav/scope_tuple.rb`, ~120 LOC) is the canonical source of truth for the `(scope, parent_scope)` partition tuple. It provides:
|
|
100
|
+
|
|
101
|
+
- `normalize_permissive(scope)` — coerces input to a tuple; tolerates bare scalars (used by `with_scope`, `normalize_scope`, `Field#validate_parent_scope_invariant`).
|
|
102
|
+
- `normalize_strict(scope)` — same shape, but raises on bare-scalar input (used by `current_scope`; preserves Phase-1's asymmetric contract that `Config.scope_resolver` must return a tuple).
|
|
103
|
+
- `invariant_satisfied?(scope, parent_scope)` — Boolean check for the orphan-parent invariant (`parent_scope` set without `scope` = invalid).
|
|
104
|
+
|
|
105
|
+
Each calling site keeps its own response policy (raise / AR error / silent narrow) using the Boolean return — `ScopeTuple` is a predicate, not an enforcer.
|
|
106
|
+
|
|
107
|
+
## Partition tuple helpers: `Partition`
|
|
108
|
+
|
|
109
|
+
`TypedEAV::Partition` (`lib/typed_eav/partition.rb`, ~100 LOC) owns the `(entity_type, scope, parent_scope)` precedence rules:
|
|
110
|
+
|
|
111
|
+
- `definitions_by_name(model, scope, parent_scope)` — returns the field-definitions map for a single resolved partition.
|
|
112
|
+
- `definitions_multimap_by_name(model)` — returns the cross-partition multimap used by `unscoped { }` blocks.
|
|
113
|
+
- `visible_fields(model, scope, parent_scope)` / `visible_sections(...)` — scope-respecting field/section iteration with the orphan-parent invariant inlined via `ScopeTuple.invariant_satisfied?`.
|
|
114
|
+
|
|
115
|
+
The definitions helpers used to live as class methods on `HasTypedEav` before 0.3.0. They moved to `Partition` per [ADR-0002](../adr/0002-entity-query-orchestration.md) because they describe the partition domain, not the macro.
|
|
116
|
+
|
|
117
|
+
## Events: `EventDispatcher`
|
|
118
|
+
|
|
119
|
+
`TypedEAV::EventDispatcher` (`lib/typed_eav/event_dispatcher.rb`, ~150 LOC) is the broker for `on_value_change` and `on_field_change` callbacks. Per [ADR-0003](../adr/0003-keep-event-dispatcher-broker.md), it intentionally stays a broker rather than getting absorbed into either `Value` or `Field` — its multi-publisher / multi-subscriber shape doesn't belong on either model. See [Event hooks](events-and-versioning.md#event-hooks) for the public callback contract.
|
|
120
|
+
|
|
121
|
+
## Schema portability and CSV: independent modules
|
|
122
|
+
|
|
123
|
+
`TypedEAV::SchemaPortability` and `TypedEAV::CSVMapper` (Phase-6 modules) are deliberately decoupled from the core read/write path per [ADR-0005](../adr/0005-keep-phase-six-modules-independent.md). They depend on the public `has_typed_eav` macro surface, never on internal modules.
|
|
124
|
+
|
|
125
|
+
See [Bulk operations](bulk-operations.md) for bulk reads, writes, and their guarantees.
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Bulk operations"
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Bulk operations
|
|
6
|
+
|
|
7
|
+
[Documentation home](../index.md)
|
|
8
|
+
|
|
9
|
+
## Bulk reads: `BulkRead`
|
|
10
|
+
|
|
11
|
+
`typed_eav_hash_for(records)` (the plural read) routes through `TypedEAV::BulkRead`. Given a record collection and an effective `(scope, parent_scope)`, it:
|
|
12
|
+
|
|
13
|
+
1. Resolves visible definitions and groups the requested field IDs.
|
|
14
|
+
2. Loads definitions, values, and field associations through one batched
|
|
15
|
+
definition query, one values query, and one field-association preload
|
|
16
|
+
(three SQL queries total; no host-table query).
|
|
17
|
+
3. Returns a `{record_id => {field_name => value}}` map while skipping orphaned
|
|
18
|
+
values and preserving logical missingness.
|
|
19
|
+
|
|
20
|
+
Definitions, filters, reads, registry entries, and writes all use the host's
|
|
21
|
+
Rails `polymorphic_name`, so an STI leaf class reads and queries the same rows
|
|
22
|
+
written under its base-class polymorphic type.
|
|
23
|
+
|
|
24
|
+
The final production characterization reduced the 1,002 SQL statements observed
|
|
25
|
+
across 1,000 scopes to three for the same BulkRead shape. This is a statement-
|
|
26
|
+
count result, not a representative throughput claim; applications should still
|
|
27
|
+
measure their own scope cardinality, selected fields, hydration, and contention.
|
|
28
|
+
|
|
29
|
+
Single-record reads (`typed_eav_value`, `typed_eav_hash`) live on `InstanceMethods` and use the same partition helpers but without batching.
|
|
30
|
+
|
|
31
|
+
Use `fields:` to load only the values needed by a view or export:
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
Contact.typed_eav_hash_for(contacts, fields: [:name, :score])
|
|
35
|
+
# => {123 => {"name" => "Ada", "score" => 42}, ...}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Omitting `fields:` (or passing `nil`) retains the all-fields behavior. A single
|
|
39
|
+
String/Symbol or an enumerable of names is accepted; duplicates are removed.
|
|
40
|
+
Unknown names and names unavailable in an individual record's partition are
|
|
41
|
+
omitted, as are missing value rows. An explicitly stored NULL remains `nil`.
|
|
42
|
+
`fields: []` returns an empty inner hash for each record without definition or
|
|
43
|
+
value queries (the supplied collection itself may still need loading). Selected
|
|
44
|
+
winning field IDs constrain the value query before hydration, so unrequested
|
|
45
|
+
values and their field readers are not loaded or evaluated. Definition lookup
|
|
46
|
+
remains batched across the records' partitions.
|
|
47
|
+
|
|
48
|
+
To explicitly reuse already-loaded values:
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
contacts = Contact.where(tenant_id: "t1").includes(typed_values: :field).to_a
|
|
52
|
+
Contact.typed_eav_hash_for(contacts, fields: [:name], source: :preloaded)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The default `source: :database` still fetches persisted values afresh, even
|
|
56
|
+
when associations are loaded or edited in memory. `:preloaded` uses the caller's
|
|
57
|
+
association targets, including unsaved Value builds/assignments, without saving
|
|
58
|
+
or mutating them. It performs one fresh batched definition query to choose
|
|
59
|
+
current winners, but no Value or field-association preload queries. This is a
|
|
60
|
+
value snapshot, not a guarantee of current database contents or frozen schema.
|
|
61
|
+
|
|
62
|
+
Every host's `typed_values` association must be loaded, as must each retained
|
|
63
|
+
Value's `field` association; incomplete preloads raise `ArgumentError` instead
|
|
64
|
+
of silently issuing N+1 queries. With `fields:`, unselected Values need no field
|
|
65
|
+
preload. With all fields selected, all field associations must be loaded
|
|
66
|
+
(including loaded `nil` for orphans). `fields: []` needs neither associations
|
|
67
|
+
nor definition/value queries. Only `:database` and `:preloaded` are valid sources.
|
|
68
|
+
|
|
69
|
+
## Bulk writes: `BulkWrite`
|
|
70
|
+
|
|
71
|
+
`bulk_set_typed_eav_values(records, attrs)` routes through `TypedEAV::BulkWrite`,
|
|
72
|
+
and `bulk_set_typed_eav_values_per_record(values_by_record)` is its sibling for
|
|
73
|
+
record-varying hashes. Both are semantic writers that:
|
|
74
|
+
|
|
75
|
+
1. Memoize field definitions for the call via `Thread.current[:typed_eav_bulk_defs_memo]`.
|
|
76
|
+
2. Validate each attribute against its field type's cast contract.
|
|
77
|
+
3. Save each host through the normal callback/validation path inside an outer
|
|
78
|
+
transaction with per-record savepoints.
|
|
79
|
+
|
|
80
|
+
`bulk_set_typed_eav_values_per_record` uses records as Hash keys, so two AR
|
|
81
|
+
instances of the same persisted row collapse to one entry; sequence separate
|
|
82
|
+
calls for two ordered updates, while the uniform Array API preserves duplicate
|
|
83
|
+
instances and caller order.
|
|
84
|
+
|
|
85
|
+
Under the default `transaction: :all`, per-record validation failures are captured at their
|
|
86
|
+
savepoints while other successes can commit, but an uncaught exception rolls
|
|
87
|
+
the entire outer transaction back.
|
|
88
|
+
`transaction: :chunks, chunk_size: N` commits completed chunks while isolating
|
|
89
|
+
later failures, preserving earlier committed chunks. Both forms require the
|
|
90
|
+
host, Field, and Value pools to match.
|
|
91
|
+
`bulk_upsert_typed_eav_values` is a separate reduced-semantics fast path: it
|
|
92
|
+
casts and validates typed values, then performs one PostgreSQL upsert while
|
|
93
|
+
omitting host saves, persistence callbacks, delete shorthand, and versioning.
|
|
94
|
+
|
|
95
|
+
Callers must pass `acknowledge_reduced_semantics: true`. The same values hash
|
|
96
|
+
applies to every record; records must be persisted and unique, and string or
|
|
97
|
+
symbol field keys that normalize to the same name are rejected. The return
|
|
98
|
+
value is the integer number of value rows upserted, not a semantic
|
|
99
|
+
`successes`/`errors_by_record` result. Value casting, domain/entity/partition
|
|
100
|
+
checks, and Value validation callbacks remain; host callbacks and validations,
|
|
101
|
+
Value persistence callbacks, versioning, delete shorthand, and per-record
|
|
102
|
+
savepoint isolation are skipped.
|
|
103
|
+
|
|
104
|
+
Within each `transaction: :all` unit—or each requested chunk—the upsert path
|
|
105
|
+
resolves every record partition through one batched field-definition SELECT.
|
|
106
|
+
It shares BulkRead's internal tuple resolver, retaining global, scope-only, and
|
|
107
|
+
full-tuple precedence independently for each record without broadening tenant
|
|
108
|
+
visibility.
|
|
109
|
+
|
|
110
|
+
`BulkWrite` and `BulkRead` are siblings — one read path, one write path — but they don't share a base class. Per [ADR-0005](../adr/0005-keep-phase-six-modules-independent.md), keeping them independent preserves the option to evolve each on its own schedule.
|
|
111
|
+
|
|
112
|
+
## Writing a batch and handling results
|
|
113
|
+
|
|
114
|
+
```ruby
|
|
115
|
+
contacts = Contact.where(tenant_id: "t1").to_a
|
|
116
|
+
result = Contact.bulk_set_typed_eav_values(contacts, { score: 10 })
|
|
117
|
+
result[:successes].map(&:id)
|
|
118
|
+
result[:errors_by_record].each do |record, errors|
|
|
119
|
+
Rails.logger.info(contact_id: record.id, errors: errors)
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
# Different sparse updates for each record:
|
|
123
|
+
result = Contact.bulk_set_typed_eav_values_per_record(
|
|
124
|
+
{ alice => { score: 12 }, bob => { nickname: { _destroy: true } } },
|
|
125
|
+
transaction: :chunks, chunk_size: 100
|
|
126
|
+
)
|
|
127
|
+
|
|
128
|
+
# Explicit fast path; returns a count of Value rows:
|
|
129
|
+
rows_written = Contact.bulk_upsert_typed_eav_values(
|
|
130
|
+
contacts, { score: 10 }, acknowledge_reduced_semantics: true
|
|
131
|
+
)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Semantic results are a symbol-keyed Hash with `successes` (host instances) and
|
|
135
|
+
`errors_by_record` (host-instance keys, string-keyed validation-message hashes).
|
|
136
|
+
An empty batch returns empty collections. Unlisted fields are untouched; the
|
|
137
|
+
`{ _destroy: true }` value removes the named Value through normal destruction.
|
|
138
|
+
Input errors and uncaught save exceptions propagate instead of becoming result
|
|
139
|
+
entries. Check `errors_by_record` even when `transaction: :all` is used: ordinary
|
|
140
|
+
validation failures do not make this an all-or-nothing batch.
|
|
141
|
+
|
|
142
|
+
Records can span partitions; each scoped record resolves its own definitions.
|
|
143
|
+
Filter and authorize the records before handing them to the writer. `chunk_size:`
|
|
144
|
+
must be a positive Integer with `transaction: :chunks`; it is ignored with
|
|
145
|
+
`:all`. An enclosing application transaction can still roll back work described
|
|
146
|
+
as committed chunks. Host saves also run validations/callbacks for other pending
|
|
147
|
+
host changes, so use records whose pending state you intend to save.
|
|
148
|
+
|
|
149
|
+
### Version grouping
|
|
150
|
+
|
|
151
|
+
Both semantic APIs accept `version_grouping:`:
|
|
152
|
+
|
|
153
|
+
| Value | Behavior |
|
|
154
|
+
| --- | --- |
|
|
155
|
+
| `:default` | Groups changes per record when versioning callbacks are installed; otherwise does no grouping. |
|
|
156
|
+
| `:per_record` | Assigns a group UUID for each record's pending typed-value changes. |
|
|
157
|
+
| `:per_field` | Shares a UUID for each written field name across the batch, including across chunks. Per-record hashes use the union of their field names. |
|
|
158
|
+
| `:none` | Adds no bulk grouping; normal enabled versioning still runs. |
|
|
159
|
+
|
|
160
|
+
Explicit `:per_record` or `:per_field` requires versioning installed at boot and
|
|
161
|
+
raises `ArgumentError` otherwise. Unknown grouping values also raise. Grouping
|
|
162
|
+
controls audit identity, not transaction atomicity. See
|
|
163
|
+
[events and versioning](events-and-versioning.md) for enabling history.
|
|
164
|
+
|
|
165
|
+
## Bulk operation guarantees
|
|
166
|
+
|
|
167
|
+
`bulk_upsert_typed_eav_values` is an explicit reduced-semantics API: it
|
|
168
|
+
prevalidates/casts values and performs a PostgreSQL upsert, while intentionally
|
|
169
|
+
omitting host callbacks and versioning. Use the regular bulk writer when those
|
|
170
|
+
semantics are required; chunked semantic transactions are opt-in.
|
|
171
|
+
|
|
172
|
+
The fast path still casts and runs domain, entity, partition, and validation
|
|
173
|
+
callbacks before its single upsert against the exact entity/field conflict
|
|
174
|
+
target; it omits host saves/host callbacks, Value persistence callbacks,
|
|
175
|
+
delete shorthand, and versioning. It requires one shared connection pool and
|
|
176
|
+
returns validation errors before SQL. `:all` is one unit; `:chunks` commits
|
|
177
|
+
completed chunks before a later failure. Semantic writes retain host saves,
|
|
178
|
+
per-record savepoint/error isolation, and one outer `:all` transaction.
|
|
179
|
+
|
|
180
|
+
BulkWrite evidence is intentionally bounded to the exercised 100- and 1,000-host
|
|
181
|
+
lanes. It does not establish 10,000- or 100,000-host throughput, nor does it
|
|
182
|
+
justify a universal batch size or storage choice.
|
|
183
|
+
|
|
184
|
+
## Operational guarantees
|
|
185
|
+
|
|
186
|
+
The semantic writer preserves the caller's transaction and callback/versioning
|
|
187
|
+
contract. Version rows are written in the source transaction, so a rollback
|
|
188
|
+
rolls back the Value mutation and its audit row together. The reduced-semantics
|
|
189
|
+
upsert is intentionally separate and does not claim those callbacks or audit
|
|
190
|
+
guarantees.
|
|
191
|
+
|
|
192
|
+
Field deletion has a callback-preserving, keyset-batched path that locks and
|
|
193
|
+
destroys only the exact field's Values before bounded finalization. It scales by
|
|
194
|
+
bounded primary-key batches and preserves the Field if a batch fails; it is not
|
|
195
|
+
a claim of unbounded deletion throughput.
|
|
196
|
+
|
|
197
|
+
## Default backfill narrowing
|
|
198
|
+
|
|
199
|
+
`Field::Base#backfill_default!` optionally accepts an exact-host
|
|
200
|
+
`ActiveRecord::Relation` to SQL-narrow eligible entities before batching. The default
|
|
201
|
+
all-host behavior remains unchanged; partition checks, batch transactions,
|
|
202
|
+
callbacks, validations, idempotence, versions, and errors remain in force.
|
|
203
|
+
Typed storage defines logical missingness across all declared cells, so a
|
|
204
|
+
partially populated multi-cell value is present while a fully empty Currency
|
|
205
|
+
value is missing.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Mapping and importing CSV rows"
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Mapping and importing CSV rows
|
|
6
|
+
|
|
7
|
+
[Documentation home](../index.md)
|
|
8
|
+
|
|
9
|
+
`TypedEAV::CSVMapper` transforms one CSV row into field-name attributes. It does
|
|
10
|
+
not read files, create records, resolve tenants, or save values. Your application
|
|
11
|
+
owns those steps and can use the normal bulk writer for persistence.
|
|
12
|
+
|
|
13
|
+
## Map headers and validate casts
|
|
14
|
+
|
|
15
|
+
```ruby
|
|
16
|
+
require "csv"
|
|
17
|
+
|
|
18
|
+
mapping = { "Nickname" => :nickname, "Age" => :age }
|
|
19
|
+
fields = TypedEAV::Partition.effective_fields_by_name(
|
|
20
|
+
entity_type: Contact.polymorphic_name, scope: "t1", parent_scope: nil
|
|
21
|
+
)
|
|
22
|
+
unknown = mapping.values.map(&:to_s) - fields.keys
|
|
23
|
+
raise ArgumentError, "Unknown mapped fields: #{unknown.join(', ')}" if unknown.any?
|
|
24
|
+
|
|
25
|
+
row = CSV::Row.new(["Nickname", "Age"], ["Ada", "37"])
|
|
26
|
+
result = TypedEAV::CSVMapper.row_to_attributes(row, mapping, fields_by_name: fields)
|
|
27
|
+
result.attributes # => {"nickname" => "Ada", "age" => 37}
|
|
28
|
+
result.errors # => {}
|
|
29
|
+
result.success? # => true
|
|
30
|
+
|
|
31
|
+
# Once your application has selected an authorized destination record:
|
|
32
|
+
if result.success?
|
|
33
|
+
saved = Contact.bulk_set_typed_eav_values_per_record({ contact => result.attributes })
|
|
34
|
+
saved[:errors_by_record] # Check full model/value validation failures here too.
|
|
35
|
+
end
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Resolve definitions for the destination record's partition. If a file spans
|
|
39
|
+
partitions, resolve the appropriate map for each destination. Use the effective
|
|
40
|
+
map so inherited same-name fields follow normal precedence.
|
|
41
|
+
|
|
42
|
+
Typed mode calls each field's `cast`; it does not run the complete persistence
|
|
43
|
+
validation pipeline. For example, an integer may cast successfully but violate
|
|
44
|
+
its field's minimum or the host's validations when saved. A successful mapping
|
|
45
|
+
therefore does not guarantee a successful save.
|
|
46
|
+
|
|
47
|
+
## Headerless files and raw previews
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
TypedEAV::CSVMapper.row_to_attributes(
|
|
51
|
+
["Ada", "37"], { 0 => :nickname, 1 => :age }, fields_by_name: fields
|
|
52
|
+
).attributes
|
|
53
|
+
# => {"nickname" => "Ada", "age" => 37}
|
|
54
|
+
|
|
55
|
+
# Omit fields_by_name for a raw mapping preview without casts:
|
|
56
|
+
TypedEAV::CSVMapper.row_to_attributes(row, mapping).attributes
|
|
57
|
+
# => {"nickname" => "Ada", "age" => "37"}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Mapping keys must be all String header names (for `CSV::Row`) or all Integer
|
|
61
|
+
column indexes (for Arrays). Mixed or unsupported key types raise `ArgumentError`
|
|
62
|
+
before processing. An empty mapping is valid. Field-name values may be Strings
|
|
63
|
+
or Symbols; result keys are Strings. Match the key style to the row shape.
|
|
64
|
+
|
|
65
|
+
## Errors and empty cells
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
bad_row = CSV::Row.new(["Nickname", "Age"], ["Ada", "not-a-number"])
|
|
69
|
+
result = TypedEAV::CSVMapper.row_to_attributes(bad_row, mapping, fields_by_name: fields)
|
|
70
|
+
result.attributes # => {"nickname" => "Ada"}
|
|
71
|
+
result.errors # => {"age" => ["is invalid"]}
|
|
72
|
+
result.failure? # => true
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Cast failures are collected per field and the invalid cell is omitted from
|
|
76
|
+
`attributes`. Do not persist the partial attributes unless your import policy
|
|
77
|
+
explicitly allows partial rows. Empty cells (`nil` or an empty String) cast to
|
|
78
|
+
`nil` without a cast error; required-field checks still happen when saving.
|
|
79
|
+
Missing headers/indexes that return `nil` follow that same path, so validate the
|
|
80
|
+
file's expected columns before importing if omission must be an error.
|
|
81
|
+
|
|
82
|
+
In typed mode, unknown field names are silently skipped without an error; the
|
|
83
|
+
upfront mapping check above prevents accidental omissions. In raw mode, every
|
|
84
|
+
mapped cell passes through unchanged. `attributes` and `errors` are frozen
|
|
85
|
+
Hashes; `success?` means only that `errors` is empty. Row numbering, logging,
|
|
86
|
+
file parsing errors, transactions, and retry policy remain application concerns.
|
|
87
|
+
See [bulk operations](bulk-operations.md#writing-a-batch-and-handling-results)
|
|
88
|
+
for save results and transaction options.
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Development and test isolation"
|
|
3
|
+
nav_group: Project
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Development and test isolation
|
|
7
|
+
|
|
8
|
+
[Documentation home](../index.md)
|
|
9
|
+
|
|
10
|
+
These examples use metadata and hooks defined in this gem’s `spec/spec_helper.rb`. Host applications need equivalent test setup.
|
|
11
|
+
|
|
12
|
+
## Event-hook tests
|
|
13
|
+
|
|
14
|
+
Test files that exercise event hooks should opt in to the `:event_callbacks`
|
|
15
|
+
metadata:
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
RSpec.describe "my feature", :event_callbacks do
|
|
19
|
+
it "fires the hook" do
|
|
20
|
+
captured = []
|
|
21
|
+
TypedEAV::Config.on_value_change = ->(v, t, _ctx) { captured << [v.id, t] }
|
|
22
|
+
contact.update!(typed_eav: { phone: "555-1234" })
|
|
23
|
+
expect(captured).to include([be_a(Integer), :update])
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshots and
|
|
29
|
+
restores Config user procs and the internal-subscriber lists around each
|
|
30
|
+
example, so test mutations don't leak across examples and engine-load
|
|
31
|
+
registrations from later phases stay intact.
|
|
32
|
+
|
|
33
|
+
Integration specs that create real AR records and need `after_commit` to
|
|
34
|
+
fire durably should additionally opt in to `:real_commits`:
|
|
35
|
+
|
|
36
|
+
```ruby
|
|
37
|
+
RSpec.describe "my model", :event_callbacks, :real_commits do
|
|
38
|
+
# ...
|
|
39
|
+
end
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
`:real_commits` disables transactional fixtures for the example and
|
|
43
|
+
manually deletes typed_eav rows in FK order after.
|
|
44
|
+
|
|
45
|
+
## Versioning tests
|
|
46
|
+
|
|
47
|
+
Specs that exercise versioning should opt into the `:event_callbacks`
|
|
48
|
+
and `:real_commits` metadata flags (see [Event hooks](events-and-versioning.md#event-hooks) — same pattern):
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
RSpec.describe "my versioning behavior", :event_callbacks, :real_commits do
|
|
52
|
+
before do
|
|
53
|
+
TypedEAV.registry.register("Contact", versioned: true)
|
|
54
|
+
TypedEAV::Config.versioning = true
|
|
55
|
+
# Transactional Value callbacks are boot-latched and remain installed;
|
|
56
|
+
# the hook isolates only public and generic EventDispatcher observers.
|
|
57
|
+
end
|
|
58
|
+
after { TypedEAV.registry.register("Contact", versioned: false) }
|
|
59
|
+
|
|
60
|
+
it "writes a version row" do
|
|
61
|
+
# ...
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshots and
|
|
67
|
+
restores `Config.versioning`, `Config.actor_resolver`, and generic
|
|
68
|
+
EventDispatcher observer lists around each example. Transactional Value
|
|
69
|
+
callback installation is tested independently through callback-chain and
|
|
70
|
+
boot-latch specs. The
|
|
71
|
+
`:real_commits` hook disables transactional fixtures (so `after_commit`
|
|
72
|
+
fires durably) and cleans up `TypedEAV::ValueVersion` rows in
|
|
73
|
+
FK-respecting order between examples.
|