typed_eav 0.5.0 → 0.7.0

data/README.md

@@ -2,22 +2,17 @@
 
 Add dynamic custom fields to ActiveRecord models at runtime, backed by **native database typed columns** instead of jsonb blobs.
 
-TypedEAV uses a hybrid EAV (Entity-Attribute-Value) pattern where each value type gets its own column (`integer_value`, `date_value`, `string_value`, etc.) in the values table. This means the database can natively index, sort, and enforce constraints on your custom field data with zero runtime type casting.
+TypedEAV uses a hybrid EAV (Entity-Attribute-Value) pattern where each value type gets its own column (`integer_value`, `date_value`, `string_value`, etc.) in the values table. This lets PostgreSQL index, sort, and enforce constraints on custom field data while the Field owns input normalization and validation.
 
 ## Why Typed Columns?
 
-Most Rails custom field gems serialize everything into a single `jsonb` column. When you query, they generate SQL like:
+JSONB is a useful fit when an application owns stable paths and wants expression B-tree indexes or GIN containment indexes. A single JSONB document is not inherently faster or slower than TypedEAV; the right choice depends on access patterns, selectivity, update shape, and operational constraints. For example, an application-owned expression index may support a stable path:
 
 ```sql
 CAST(value_meta->>'const' AS bigint) = 42
 ```
 
-This works, but:
-
-- **No B-tree indexes** on the actual values (only GIN for jsonb containment)
-- **Runtime CAST overhead** on every query
-- **No database-level type enforcement** (a "number" could be stored as a string)
-- **The query planner can't optimize** range scans, sorts, or joins
+This can work well for stable, known paths. It does not provide the same column-level schema and typed-value contract as TypedEAV, and arbitrary paths still require application-owned index and validation decisions. GIN is useful for containment workloads; expression B-trees are useful for selected stable scalar paths.
 
 TypedEAV stores values in native columns, so queries become:
 
@@ -25,7 +20,23 @@ TypedEAV stores values in native columns, so queries become:
 WHERE integer_value = 42
 ```
 
-Standard B-tree indexes work. Range scans work. The query planner is happy. ActiveRecord handles all type casting automatically through the column's registered type.
+TypedEAV supplies stable typed columns and ordinary per-type indexes. Range scans and sorts can use those indexes, while each Field casts and validates the operand according to its own semantics before the query reaches the typed column. Neither design is a universal storage winner; choose from measured workload fit.
+
+## Compatibility
+
+The canonical support contract lives in
+[`.github/compatibility.json`](.github/compatibility.json). Typed EAV supports:
+
+| Runtime | Supported versions |
+|---|---|
+| Ruby | 3.3 through 4.0 (`>= 3.3`, `< 4.1`) |
+| Rails | 7.2 through 8.1 (`>= 7.2`, `< 8.2`) |
+| PostgreSQL | 15 through 18 |
+
+CI proves representative floor, middle, and ceiling combinations rather than
+every Cartesian product. Versions outside these ranges and prerelease versions
+are outside the support guarantee. PostgreSQL compatibility claims assume the
+current minor release for each supported major version.
 
 ## Installation
 
@@ -141,7 +152,7 @@ contact.typed_eav_hash              # => { "age" => 40, "status" => "active", ..
 
 ### 4. Query with the DSL
 
-This is where typed columns pay off. All queries go through native columns with proper indexes.
+Queries use native typed columns and the indexes shipped for each type. The Field remains the owner of operand casting and validation; the query builder receives a field-normalized value rather than applying a generic Active Record cast.
 
 ```ruby
 # Short form - single field filter
@@ -201,25 +212,108 @@ Contact.where(company_id: 42)
 | `:is_null` | all | Value is NULL |
 | `:is_not_null` | all | Value is not NULL |
 
+### Optional trigram indexing for string search
+
+TypedEAV keeps its partial-covering `text_pattern_ops` B-tree as the default
+string index. Equality uses that B-tree, while `:starts_with`, `:contains`, and
+`:ends_with` use `ILIKE`; `:not_contains` uses `NOT ILIKE`. The gem does not
+require or install `pg_trgm` and does not create a trigram index automatically.
+
+An application with frequent positive `ILIKE` searches containing at least
+three useful characters may evaluate its own partial GIN index. This is a
+workload decision: the representative benchmark used GIN for measured prefix,
+contains, suffix, and escaped-literal patterns, but not for `NOT ILIKE` or
+one/two-character probes. It does not prove that every positive pattern or
+selectivity will benefit. A `lower(string_value) LIKE ...` expression index is
+not equivalent to TypedEAV's public `ILIKE`, and the benchmark did not justify
+GiST.
+
+Application owners should check extension availability and deploy-role
+privileges in preproduction, then create the extension and index in their own
+migrations. Use nontransactional `CREATE INDEX CONCURRENTLY`, a stable
+application-specific name, and workload-specific `EXPLAIN (ANALYZE, BUFFERS,
+WAL, SETTINGS)` plus storage and write-WAL measurements. Rollback should drop
+only the application-owned index concurrently; do not drop the database-wide
+extension because other objects may share it. See
+[ADR 0009](docs/adr/0009-string-search-indexing.md) and the
+[benchmark guide](bench/README.md#phase-3-string-search-benchmark) for the
+operator matrix, measured costs, SQL, and evidence limits.
+
+### Optional planner statistics for correlated field/value predicates
+
+TypedEAV does not install PostgreSQL extended-statistics objects. An application
+whose own plans persistently misestimate `field_id = ... AND typed_value = ...`
+may evaluate application-owned `dependencies` statistics for that exact typed
+column. Dependency statistics apply to compatible equality and `IN` clauses,
+not range predicates. `mcv` describes common value combinations, while
+`ndistinct` primarily informs distinct-group estimates; neither should be added
+without workload evidence.
+
+The representative PostgreSQL 17 benchmark found better aggregate equality
+estimates from dependencies, but no plan-shape or demonstrated runtime benefit.
+Its combined object mirrored MCV on the four changed probes because matching MCV
+groups supplied those estimates. The experiment's target of 100 was a controlled
+input, not a universal recommendation. One probe labeled common-date equality
+actually queried an absent date and returned zero rows; it is not evidence about
+common-date estimates.
+
+Applications should own stable names and DDL, select targets from representative
+data, run `ANALYZE`, and compare estimated/actual rows, plans, runtime, planning
+cost, maintenance cost, and data churn before retaining an object. Coordinate
+ownership in shared databases, inspect catalog definitions before changing
+objects, and drop only application-owned statistics during rollback. See
+[ADR 0010](docs/adr/0010-planner-statistics-policy.md) and the
+[benchmark guide](bench/README.md#phase-4a-planner-extended-statistics) for safe
+evaluation SQL and evidence limits.
+
+### Multi-filter query strategy
+
+TypedEAV retains its current multi-filter query shape: it resolves each field,
+builds the corresponding typed value subquery, and chains those results onto
+the host relation with `id IN (...)`. There is no adaptive strategy or alternate
+production query API.
+
+A PostgreSQL 17 benchmark compared the shipped shape with `INTERSECT`,
+correlated `EXISTS`, and direct grouped `HAVING` under resource-capped
+co-tenancy. The run retained 2,940 attempts, including 622 right-censored
+timeouts, and 294 representative identity oracles. Twelve oracles timed out, so
+representative equivalence is unproved even though all 282 completed oracles
+matched and the smaller 98-oracle smoke matched. Alternatives remain
+research-only. Grouped `HAVING` is additionally ineligible for missing-value,
+host-universe complement, and empty-filter semantics.
+
+The result also does not establish valid buffer comparisons or 20-distinct-
+field scaling. A parser defect made every derived buffer total a false zero;
+nonzero counters remain recoverable from the retained raw plans. The
+20-predicate workloads repeat ten fields, and the skewed 10/20 workloads repeat
+five. Future research must repair and validate buffer extraction, exercise
+actual 10/20 distinct fields, complete every representative equivalence oracle,
+cover the full scope/NULL/missing/polymorphic/error contract, and show the
+pre-registered p95, planning-time, buffer, and plan-shape gates before any
+adaptive or replacement proposal. See
+[ADR 0011](docs/adr/0011-multi-filter-query-strategy.md) and the
+[benchmark guide](bench/README.md#phase-4b-multi-filter-query-shapes).
+
 ### How Type Inference Works
 
-You don't need to think about types when querying. Rails handles it:
+The owning Field casts and validates query operands before SQL generation;
+Active Record supplies the SQL bind plumbing:
 
 ```ruby
-# You pass a string, Rails casts to integer via the column type
+# The Integer Field casts and validates the operand before SQL generation
 Contact.with_field("age", :gt, "21")
 # SQL: WHERE integer_value > 21  (not '21')
 
-# You pass a string, Rails casts to date
+# The Date Field owns date parsing and validation
 Contact.with_field("birthday", :lt, "2000-01-01")
 # SQL: WHERE date_value < '2000-01-01'::date
 
-# Boolean columns handle truthy/falsy casting
+# The Boolean Field owns truthy/falsy casting
 Contact.with_field("active", "true")
 # SQL: WHERE boolean_value = TRUE
 ```
 
-This works because `ActiveRecord::Base.columns_hash` knows every column's type from the schema, and `where()` / Arel predicates automatically cast values through the column's registered `ActiveRecord::Type`.
+Field-owned casting keeps query operands aligned with write semantics, including strict range/array shapes and specialized fields such as Currency and Reference. The resulting normalized operand is bound against the Field's typed column; Active Record supplies SQL bind plumbing, not the field's domain semantics.
 
 ## Forms
 
@@ -357,6 +451,16 @@ end
 
 Both are exception-safe via `ensure` and nest cleanly.
 
+`unscoped` is an explicit administrative/analytics escape hatch, not the
+ordinary tenant request path. It keeps every same-name definition across the
+visible partitions and unions their matches for each filter. For broad audits
+or migrations, bound the definition universe to the work you actually need and
+batch the job at an application-owned boundary. TypedEAV does not prescribe a
+universal limit or batch size; measure generated SQL, planning/execution,
+memory, and workload interference in your application. Keep normal request
+traffic on scoped resolution so global, scope-only, and full-tuple definitions
+collapse to the most-specific match.
+
 ### Explicit `scope:` override
 
 Any query method accepts `scope:` as an override for admin tools and tests:
@@ -471,12 +575,26 @@ within one workspace" has no semantic resolution path; the row would never
 match any record's resolver. The paired partial unique indexes rely on this
 invariant.
 
+The shipped migration chain also includes
+`EnforceParentScopeInvariant`, which declares the database check constraints
+nontransactionally and validates them after its preflight, and
+`UsePartialCoveringScalarIndexes`, which creates the six `*_present` indexes
+before removing their legacy counterparts. Both migrations use
+`disable_ddl_transaction!`; run them through the normal migration command and
+do not wrap them in an application transaction.
+
 ### Name collisions across scopes
 
 When both a global field (`scope: nil`) and a scoped field share a name, the **scoped definition wins** for the partition that owns it: forms render exactly one input (the scoped one), reads return the scoped value, and writes target the scoped row.
 
 `TypedEAV.unscoped { Contact.where_typed_eav(...) }` OR-across every partition's matching `field_id` per filter (still AND-ing across filters), so cross-tenant audit queries see every partition's matches — they don't collapse to a single tenant.
 
+Because that administrative path constructs work for every matching
+definition, applications should narrow and batch high-cardinality audits rather
+than treating `unscoped` as tenant-request routing. No built-in numeric
+threshold is implied; choose operational bounds from measurements of the
+consuming workload. See [ADR 0012](docs/adr/0012-cross-scope-administrative-query-policy.md).
+
 ## Field Types
 
 | Type | Column | Ruby Type | Options |
@@ -769,7 +887,7 @@ exactly three method overrides.
 
 - **`File`:** Same shape as `Field::Image` but without image-specific semantics. Stores `signed_id` in `string_value`; same operator set; same options (`allowed_content_types`, `max_size_bytes`). The Image vs File distinction is by `value.field.class` at runtime — apps that want strict image-only validation set `allowed_content_types: ["image/*"]` on `Field::Image`; `Field::File` is a general-purpose attachment slot.
 
-- **Active Storage dependency:** Lazy soft-detect via `defined?(::ActiveStorage::Blob)`. The gem does NOT add Active Storage as a hard dependency — apps that never use Image/File never need to install it. To use Image or File fields, add `gem "activestorage"` to your Gemfile (already included in Rails 7.1+ via the `rails` meta-gem) and run `bin/rails active_storage:install` to create the `active_storage_blobs` / `active_storage_attachments` / `active_storage_variant_records` tables. The mirror precedent is `acts_as_tenant`, which is also soft-detected (see `Config::DEFAULT_SCOPE_RESOLVER`).
+- **Active Storage dependency:** Lazy soft-detect via `defined?(::ActiveStorage::Blob)`. The gem does NOT add Active Storage as a hard dependency — apps that never use Image/File never need to install it. To use Image or File fields, add `gem "activestorage"` to your Gemfile (included in supported Rails versions via the `rails` meta-gem) and run `bin/rails active_storage:install` to create the `active_storage_blobs` / `active_storage_attachments` / `active_storage_variant_records` tables. The mirror precedent is `acts_as_tenant`, which is also soft-detected (see `Config::DEFAULT_SCOPE_RESOLVER`).
 
 - **`on_image_attached` hook:** Fires from `after_commit` on `TypedEAV::Value` when a `Field::Image`-typed Value's attachment is added or replaced. Receives `(value, blob)`. Configure via `TypedEAV.configure { |c| c.on_image_attached = ->(v, b) { ... } }`. Hook ordering: runs AFTER versioning (Phase 4) and AFTER `on_value_change` (Phase 3) so it sees the persisted version row and the user-callback context. File attachments do NOT fire this hook — the name is image-specific by design. Use `on_value_change` for a generic value-mutation signal that covers File-typed Values too.
 
@@ -838,8 +956,8 @@ end
 The `:rename` change_type fires whenever the field's `name` column changed
 in the just-committed save, even when bundled with other attribute changes
 (options, sort_order, default_value, etc.). The detection is intentionally
-escalating — Phase 7's materialized index needs to regenerate column DDL on
-every rename.
+escalating so any registered consumer receives a rename event whenever the
+persisted name changes.
 
 `:update` on Value fires only when the typed value column changed. Saving
 a Value record without modifying its typed column (e.g., touching only
@@ -851,6 +969,17 @@ bypasses AR callbacks. Only the Field `:destroy` event fires. Use
 `field_dependent: :destroy` if your consumer needs per-Value events on
 field deletion.
 
+For a persisted `field_dependent: :destroy` field with a large population,
+call `field.destroy_with_values_in_batches!(batch_size: 1_000)` outside an
+open transaction. The opt-in API selects only that exact `field_id` in ordered
+primary-key batches, calls `Value#destroy!` for callback/version behavior, and
+commits each batch independently. A retry resumes from the remaining rows. The
+Field is retained until a locked, bounded residual drain proves zero rows, then
+its ordinary callback-preserving `destroy!` runs. The API rejects unsaved or
+non-destroy fields, open transactions, invalid batch sizes, and mismatched
+connection pools. Existing `destroy`/`destroy!`, `:nullify`, and `:restrict`
+behavior is unchanged.
+
 ### Thread-local context with `with_context`
 
 ```ruby
@@ -888,16 +1017,15 @@ when `after_commit` fires; re-raising would surface a misleading
 "save failed" error.
 
 This is the deliberate split with first-party features. Internal
-subscribers used by `typed_eav` itself (Phase 4 versioning, Phase 7
-materialized index) follow a different rule: their exceptions
-**propagate**. Versioning corruption must be loud.
+observers used by `typed_eav` itself follow a different rule: their exceptions
+**propagate**. Transactional version-writing errors are separate: they
+propagate inside and roll back the source transaction.
 
 ### Ordering guarantee
 
 When multiple subscribers are registered, they fire in this order:
 
-1. First-party internal subscribers (versioning, matview, etc.), in
-   registration order. Errors propagate.
+1. First-party generic observers, in registration order. Errors propagate.
 2. The user proc on `Config.on_value_change` / `Config.on_field_change`,
    last. Errors are rescued and logged.
 
@@ -956,8 +1084,8 @@ values. When enabled, each `:create` / `:update` / `:destroy` event on
 a Value writes a row to `typed_eav_value_versions` capturing the
 before-state, after-state, actor, context, and timestamp.
 
-Default off. Apps that don't enable it pay zero overhead — the Phase 04
-internal subscriber is not registered with `EventDispatcher.value_change_internals`
+Default off. Apps that don't enable it pay zero overhead — transactional
+Value callbacks are not installed at boot
 at all when `Config.versioning = false`. Zero callable in the dispatcher
 chain, zero per-write method dispatch, zero per-write config read.
 
@@ -1018,9 +1146,8 @@ value.history.limit(5).each { |v| ... }
 `Value#history` returns versions where `value_id` matches the live Value
 record. After the live Value is destroyed, the FK `ON DELETE SET NULL`
 nullifies `value_id` on the existing version rows, and the new `:destroy`
-version is also written with `value_id: nil` (the parent
-`typed_eav_values` row is gone by `after_commit on: :destroy` time —
-writing a non-nil `value_id` would FK-fail at INSERT). So `Value#history`
+version is written by the transactional destroy callback with `value_id: nil`
+before the parent row is removed. So `Value#history`
 cannot surface destroy versions, and after Value destruction it can no
 longer be called at all.
 
@@ -1100,8 +1227,8 @@ value.revert_to(target)
 ```
 
 `revert_to` writes the targeted version's `before_value` columns back
-via `self[col] = …` and `save!`. The existing `after_commit` chain
-fires; the versioning subscriber writes a NEW version row whose
+via `self[col] = …` and `save!`. The transactional version callback writes a
+NEW version row whose
 `after_value` reflects the targeted version's `before_value`. The
 audit log is append-only — every revert is itself versioned.
 
@@ -1129,19 +1256,18 @@ manually using `version.before_value` as the seed state.
 
 ### Hook ordering guarantee
 
-Versioning is registered as an internal subscriber on
-`TypedEAV::EventDispatcher`. It runs **first** (slot 0) for every Value
-event. Your `Config.on_value_change` user proc fires **last**, after
-the version row is persisted:
-
+Versioning is installed as boot-latched transactional callbacks on `Value`,
+and the public callback remains an after-commit observer. The version row is
+written in the source transaction.
 ```
-Value#save! → after_commit → EventDispatcher.dispatch_value_change:
-  1. TypedEAV::Versioning::Subscriber.call  # writes version row
-  2. ... any other internal subscribers (Phase 7 matview, etc.) ...
-  3. Config.on_value_change user proc        # sees the persisted version
+Value#save! → transactional Value callback → ValueVersion.create!
+          → after_commit → EventDispatcher.dispatch_value_change:
+  1. ... any other generic internal observers ...
+  2. Config.on_value_change user proc        # sees the persisted version
 ```
 
-Internal subscriber errors propagate (versioning corruption is loud).
+Internal observer errors propagate. Transactional version-writing errors also
+propagate inside and roll back the source transaction.
 User proc errors are rescued and logged via `Rails.logger.error` —
 the save itself already committed.
 
@@ -1198,14 +1324,8 @@ RSpec.describe "my versioning behavior", :event_callbacks, :real_commits do
   before do
     TypedEAV.registry.register("Contact", versioned: true)
     TypedEAV::Config.versioning = true
-    # CRITICAL: the :event_callbacks hook clears
-    # EventDispatcher.value_change_internals at example entry, so the
-    # engine-boot-registered subscriber is gone for the duration of
-    # the example. Re-register explicitly inside the before block.
-    # The hook's ensure block restores the snapshot — no leak.
-    TypedEAV::EventDispatcher.register_internal_value_change(
-      TypedEAV::Versioning::Subscriber.method(:call),
-    )
+    # Transactional Value callbacks are boot-latched and remain installed;
+    # the hook isolates only public and generic EventDispatcher observers.
   end
   after { TypedEAV.registry.register("Contact", versioned: false) }
 
@@ -1215,12 +1335,11 @@ RSpec.describe "my versioning behavior", :event_callbacks, :real_commits do
 end
 ```
 
-The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshot/
-restores `Config.versioning`, `Config.actor_resolver`, and the
-EventDispatcher subscriber lists around each example, so your changes
-don't leak to subsequent tests. The snapshot/restore CLEARS the
-internals list at example entry — that's why the re-registration
-above is required for any spec that needs the subscriber to fire. The
+The `:event_callbacks` around hook in `spec/spec_helper.rb` snapshots and
+restores `Config.versioning`, `Config.actor_resolver`, and generic
+EventDispatcher observer lists around each example. Transactional Value
+callback installation is tested independently through callback-chain and
+boot-latch specs. The
 `:real_commits` hook disables transactional fixtures (so `after_commit`
 fires durably) and cleans up `TypedEAV::ValueVersion` rows in
 FK-respecting order between examples.
@@ -1233,16 +1352,19 @@ As of v0.2.0, the paired partial unique indexes cover the three-key partition tu
 
 ## Schema
 
-The gem creates four tables:
+The gem creates five tables:
 
 - `typed_eav_fields` - field definitions (STI, one row per field per entity type)
 - `typed_eav_values` - values (one row per entity per field, with typed columns)
 - `typed_eav_options` - allowed values for select/multi-select fields
 - `typed_eav_sections` - optional UI grouping
+- `typed_eav_value_versions` - opt-in, append-only audit history for Value
+  create, update, and destroy events; it retains durable entity identity even
+  when the live Value row is later removed
 
 ## Architecture
 
-Internal module layout as of 0.3.0. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by [ADR-0001](docs/adr/0001-collapse-column-mapping-stack.md) through [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md).
+Internal module layout as of 0.7.0. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by ADR-0001 through ADR-0013.
 
 ### Macro entry: `HasTypedEav`
 
@@ -1287,19 +1409,55 @@ TypedEAV::QueryBuilder                ← low altitude: per-field SQL primitive
 
 `typed_eav_hash_for(records)` (the plural read) routes through `TypedEAV::BulkRead`. Given a record collection and an effective `(scope, parent_scope)`, it:
 
-1. Groups records by partition tuple via `TypedEAV::Partition.definitions_by_name`.
-2. Issues one batched `WHERE entity_id IN (...) AND field_id IN (...)` query per partition.
-3. Returns a `{record_id => {field_name => value}}` map.
+1. Resolves visible definitions and groups the requested field IDs.
+2. Loads definitions, values, and field associations through one batched
+   definition query, one values query, and one field-association preload
+   (three SQL queries total; no host-table query).
+3. Returns a `{record_id => {field_name => value}}` map while skipping orphaned
+   values and preserving logical missingness.
+
+The final production characterization reduced the 1,002 SQL statements observed
+across 1,000 scopes to three for the same BulkRead shape. This is a statement-
+count result, not a representative throughput claim; applications should still
+measure their own scope cardinality, selected fields, hydration, and contention.
 
 Single-record reads (`typed_eav_value`, `typed_eav_hash`) live on `InstanceMethods` and use the same partition helpers but without batching.
 
 ### Bulk writes: `BulkWrite`
 
-`bulk_set_typed_eav_values(records, attrs)` routes through `TypedEAV::BulkWrite`, an executor that:
+`bulk_set_typed_eav_values(records, attrs)` routes through `TypedEAV::BulkWrite`,
+and `bulk_set_typed_eav_values_per_record(values_by_record)` is its sibling for
+record-varying hashes. Both are semantic writers that:
 
 1. Memoizes field definitions for the call via `Thread.current[:typed_eav_bulk_defs_memo]`.
 2. Validates each attribute against its field type's cast contract.
-3. Upserts in a single SQL round trip per typed column.
+3. Saves each host through the normal callback/validation path inside an outer
+   transaction with per-record savepoints.
+
+`bulk_set_typed_eav_values_per_record` uses records as Hash keys, so two AR
+instances of the same persisted row collapse to one entry; sequence separate
+calls for two ordered updates, while the uniform Array API preserves duplicate
+instances and caller order.
+
+Under `transaction: :all`, per-record validation failures are captured at their
+savepoints while other successes can commit, but an uncaught exception rolls
+the entire outer transaction back. The default `transaction: :all` commits the
+whole successful batch or rolls it back;
+`transaction: :chunks, chunk_size: N` commits completed chunks while isolating
+later failures, preserving earlier committed chunks. Both forms require the
+host, Field, and Value pools to match.
+`bulk_upsert_typed_eav_values` is a separate reduced-semantics fast path: it
+casts and validates typed values, then performs one PostgreSQL upsert while
+omitting host saves, persistence callbacks, delete shorthand, and versioning.
+
+Callers must pass `acknowledge_reduced_semantics: true`. The same values hash
+applies to every record; records must be persisted and unique, and string or
+symbol field keys that normalize to the same name are rejected. The return
+value is the integer number of value rows upserted, not a semantic
+`successes`/`errors_by_record` result. Value casting, domain/entity/partition
+checks, and Value validation callbacks remain; host callbacks and validations,
+Value persistence callbacks, versioning, delete shorthand, and per-record
+savepoint isolation are skipped.
 
 `BulkWrite` and `BulkRead` are siblings — one read path, one write path — but they don't share a base class. Per [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md), keeping them independent preserves the option to evolve each on its own schedule.
 
@@ -1314,6 +1472,16 @@ Single-record reads (`typed_eav_value`, `typed_eav_hash`) live on `InstanceMetho
 
 Every method uses `TypedEAV::Partition.definitions_by_name` so the collision-precedence rules for ambient/explicit/parent scopes are computed in one place.
 
+### Partition visibility: `Partition`
+
+Host applications that need to inspect effective schema should use the
+documented-public `TypedEAV::Partition` seam rather than rebuilding tuple
+predicates. It exposes `visible_fields`, `effective_fields_by_name`,
+`definitions_by_name`, `definitions_multimap_by_name`, `visible_sections`,
+and `find_visible_section!`. These methods preserve global, scope-only, and
+full-tuple precedence; ADR-0006 additionally fixes include-missing set
+composition at the `FilterQuery` altitude.
+
 ### Field types and storage: `Field::TypedStorage`
 
 `TypedEAV::Field::Base` is the STI parent of every field type. The shared storage surface lives in the `TypedEAV::Field::TypedStorage` concern (`lib/typed_eav/field/typed_storage.rb`, ~200 LOC), auto-included on `Field::Base`. Per [ADR-0001](docs/adr/0001-collapse-column-mapping-stack.md), it provides:
@@ -1362,6 +1530,48 @@ The definitions helpers used to live as class methods on `HasTypedEav` before 0.
 
 `TypedEAV::SchemaPortability` and `TypedEAV::CSVMapper` (Phase-6 modules) are deliberately decoupled from the core read/write path per [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md). They depend on the public `has_typed_eav` macro surface, never on internal modules.
 
+### Bulk operation guarantees
+
+`bulk_upsert_typed_eav_values` is an explicit reduced-semantics API: it
+prevalidates/casts values and performs a PostgreSQL upsert, while intentionally
+omitting host callbacks and versioning. Use the regular bulk writer when those
+semantics are required; chunked semantic transactions are opt-in.
+
+The fast path still casts and runs domain, entity, partition, and validation
+callbacks before its single upsert against the exact entity/field conflict
+target; it omits host saves/host callbacks, Value persistence callbacks,
+delete shorthand, and versioning. It requires one shared connection pool and
+returns validation errors before SQL. `:all` is one unit; `:chunks` commits
+completed chunks before a later failure. Semantic writes retain host saves,
+per-record savepoint/error isolation, and one outer `:all` transaction.
+
+BulkWrite evidence is intentionally bounded to the exercised 100- and 1,000-host
+lanes. It does not establish 10,000- or 100,000-host throughput, nor does it
+justify a universal batch size or storage choice.
+
+### Operational guarantees
+
+The semantic writer preserves the caller's transaction and callback/versioning
+contract. Version rows are written in the source transaction, so a rollback
+rolls back the Value mutation and its audit row together. The reduced-semantics
+upsert is intentionally separate and does not claim those callbacks or audit
+guarantees.
+
+Field deletion has a callback-preserving, keyset-batched path that locks and
+destroys only the exact field's Values before bounded finalization. It scales by
+bounded primary-key batches and preserves the Field if a batch fails; it is not
+a claim of unbounded deletion throughput.
+
+### Default backfill narrowing
+
+`Field::Base#backfill_default!` optionally accepts an exact-host
+`ActiveRecord::Relation` to SQL-narrow eligible entities before batching. The default
+all-host behavior remains unchanged; partition checks, batch transactions,
+callbacks, validations, idempotence, versions, and errors remain in force.
+Typed storage defines logical missingness across all declared cells, so a
+partially populated multi-cell value is present while a fully empty Currency
+value is missing.
+
 ## License
 
 MIT