typed_eav 0.8.0 → 0.8.1

data/RELEASING.md

@@ -0,0 +1,81 @@
+# Releasing typed_eav
+
+A release is complete only when the same stable version exists in all three
+places:
+
+1. `lib/typed_eav/version.rb` and `CHANGELOG.md`
+2. RubyGems
+3. a published GitHub Release for the exact `vVERSION` tag
+
+The newest stable GitHub Release must also be marked **Latest**. A successful
+gem push by itself is not a completed release.
+
+## Prepare the release
+
+1. Confirm `main` is clean and up to date.
+2. Move the relevant notes from `[Unreleased]` into a dated
+   `## [VERSION] - YYYY-MM-DD` section in `CHANGELOG.md`.
+3. Add the matching link definition at the bottom of `CHANGELOG.md`:
+
+   ```markdown
+   [VERSION]: https://github.com/dchuk/typed_eav/releases/tag/vVERSION
+   ```
+
+4. Set `TypedEAV::VERSION` in `lib/typed_eav/version.rb`.
+5. Run the full release verification locally:
+
+   ```sh
+   bundle exec rspec
+   bundle exec rubocop
+   gem build typed_eav.gemspec
+   ```
+
+6. Commit the release preparation using the repository commit convention.
+
+## Publish
+
+Create and push an annotated `vVERSION` tag that points at the release commit.
+The `Release` GitHub Actions workflow then:
+
+1. verifies the exact tagged source and compatibility matrix;
+2. builds and tests one checksummed gem artifact;
+3. publishes that artifact to RubyGems using trusted publishing; and
+4. creates or verifies the stable GitHub Release and marks it **Latest**.
+
+Do not manually publish the gem ahead of this workflow. Do not declare the
+release complete until both the `Push verified gem to RubyGems` and
+`Publish GitHub release` jobs succeed.
+
+## Verify completion
+
+Replace `VERSION` below, then verify all release surfaces:
+
+```sh
+test "$(ruby -r ./lib/typed_eav/version -e 'print TypedEAV::VERSION')" = "VERSION"
+test "$(curl -fsSL https://rubygems.org/api/v1/gems/typed_eav.json | jq -r .version)" = "VERSION"
+gh release view "vVERSION" --json tagName,isDraft,isPrerelease,url
+gh release list --limit 10
+```
+
+The GitHub release must use tag `vVERSION`, with `isDraft: false` and
+`isPrerelease: false`, and the release list must label it `Latest`.
+
+## Recover a missing GitHub Release
+
+If RubyGems publishing succeeds but the final GitHub job fails, rerun the
+failed `Publish GitHub release` job. The job is idempotent: it creates a
+missing release or verifies an existing stable release and restores its
+**Latest** designation.
+
+If automation is unavailable, use:
+
+```sh
+gh release create "vVERSION" \
+  --verify-tag \
+  --title "typed_eav VERSION" \
+  --generate-notes \
+  --latest
+```
+
+Then repeat the completion checks above. Never create, move, or replace a tag
+just to repair missing GitHub release metadata.

data/docs/adr/0001-collapse-column-mapping-stack.md

@@ -0,0 +1,28 @@
+---
+title: "Collapse column-mapping stack; break `FieldStorageContract` extension API pre-1.0"
+---
+
+# Collapse column-mapping stack; break `FieldStorageContract` extension API pre-1.0
+
+**Status:** accepted
+
+The Phase-5 storage stack had five participants for one concept ("where does a typed value live in the row?"): `ColumnMapping`, `FieldStorageContract`, `CurrencyStorageContract`, Field instance methods (`read_value` / `write_value` / `apply_default_to`), and the `storage_contract_class` macro. `FieldStorageContract` was a pass-through wrapper for 19 of 20 field types — one real adapter (Currency) on a seam designed for many. Currency carried double-declarations across the Field class and its dedicated contract subclass, where the same column list lived in three places.
+
+We collapsed the stack into a single `Field::TypedStorage` concern that lives on `Field::Base`. The contract classes, the `storage_contract_class` macro, and `field.storage_contract` are removed. Multi-cell field types override three instance methods on their Field subclass (`read_value`, `write_value`, `apply_default`); snapshot/change-detection methods become concrete (derived from `value_columns`). Currency becomes a normal Field subclass with overrides — the same shape external custom multi-cell types now use.
+
+## Considered alternatives
+
+- **α (strict BC):** keep `FieldStorageContract` and `storage_contract_class` as the public extension surface; rewire internal callers to use Field methods directly; delete only `CurrencyStorageContract`. Achieves the internal cleanup without breaking external authors. Rejected because the wrapper class would survive as legacy plumbing with no internal justification, leaving the documented extension API and the actual internal pattern divergent forever.
+- **β (deprecate with warnings):** same as α, plus deprecation warnings on `storage_contract_class` / `field.storage_contract`. Rejected because CONTEXT.md's "BC is binding" rule provides no on-ramp to removal — the deprecation would point at nothing and become permanent noise.
+- **γ (chosen):** break BC. Remove `FieldStorageContract` entirely. Re-ship as 0.3.0.
+
+## BC reconciliation
+
+CONTEXT.md (`.vbw-planning/CONTEXT.md`) lists "Backwards compatibility is binding" as a key decision. That rule was scoped to the Phase 1–7 enhancement arc — every *phase* preserves current API surface. This refactor is not a phase; it's an out-of-band architectural cleanup happening pre-1.0 where SemVer allows breaking changes at the minor version. Future architectural refactors with the same character should follow the same pattern: an explicit ADR + a minor-version bump, not a silent break.
+
+## Consequences
+
+- External authors who subclass `FieldStorageContract` must migrate to override-on-Field (subclass `Field::Base`, override `value_columns` / `operator_column` / `read_value` / `write_value` / `apply_default`). The Currency source serves as the canonical example.
+- The README's §"Multi-cell field types" is rewritten around the new pattern.
+- Three spec files (`field_storage_contract_spec`, `column_mapping_spec`, `column_mapping_value_columns_spec`) consolidate into one `field/typed_storage_spec`.
+- Override surface shrinks from 7 methods to 3. Snapshot shape becomes a versioning-coupled invariant locked in by `value_columns` rather than an extension point external authors could silently break.

data/docs/adr/0002-entity-query-orchestration.md

@@ -0,0 +1,33 @@
+---
+title: "Split `HasTypedEav` into mixin + `EntityQuery` + query objects"
+---
+
+# Split `HasTypedEav` into mixin + `EntityQuery` + query objects
+
+**Status:** accepted
+
+`HasTypedEav` had grown to 881 lines holding three responsibilities (the macro, class-level queries, instance accessors) with two heavy methods (`where_typed_eav`, `typed_eav_hash_for`) carrying blanket `Metrics/CyclomaticComplexity` rubocop disables. The file size was one friction; the methods' depth was another — and the file split alone wouldn't have addressed the second.
+
+We split `HasTypedEav` structurally and extracted the heavy method bodies into focused query classes. The macro stays in `has_typed_eav.rb` (~150 lines). Per-record API lives in `has_typed_eav/instance_methods.rb`. Class-level orchestration lives in a new top-level `TypedEAV::EntityQuery` module. The `where_typed_eav` body extracts into `TypedEAV::FilterQuery`; `typed_eav_hash_for` extracts into `TypedEAV::BulkRead`. `bulk_set_typed_eav_values` already delegated to the existing `TypedEAV::BulkWrite`; no change. Field-collision helpers move from `HasTypedEav` module-statics to `TypedEAV::Partition` (where the partition-tuple precedence rule belongs).
+
+## Two altitudes of query module
+
+The project now has two layers of query module on purpose:
+
+- **`QueryBuilder`** — low-level SQL primitives. Given `(field, operator, value)`, returns a relation or predicate. Knows nothing about scope, collision, or multiple filters.
+- **`FilterQuery` / `BulkRead`** — high-level orchestration. Given filters + a resolved scope tuple + a model, calls down into `QueryBuilder` per filter and composes the result.
+
+This split is intentional — keeping `QueryBuilder` narrow lets per-field SQL details (Arel predicates, ILIKE escaping, type casting) stay testable in isolation, while orchestration concerns (input normalization, scope resolution, multi-tuple collision) live one level up. Future query-shape additions should pick the matching altitude: per-field predicate work belongs in `QueryBuilder`; cross-filter or cross-tuple orchestration belongs in a new top-level query class.
+
+## Considered alternatives
+
+- **(a) Two-way relocation only**, no method extraction. Solves file-size friction; leaves the `Metrics/CyclomaticComplexity` disables in place. Rejected because the depth friction was the larger of the two.
+- **(c) Depth-only**, no structural split. Extracts query classes but leaves the 881-line file behind (would shrink to ~500). Rejected because the shape friction was real on its own.
+- **(d) Three-way split** with the macro in its own tiny module. Rejected as over-decomposition — the macro is naturally co-located with its module entry.
+
+## Consequences
+
+- Zero BC impact on public surfaces. `has_typed_eav` macro and all class/instance method signatures unchanged.
+- `HasTypedEav.definitions_by_name` was technically reachable externally; treated as internal (not documented in README). Callers should use `Partition.definitions_by_name`.
+- New unit-test surfaces (`filter_query_spec`, `bulk_read_spec`, `entity_query_spec`) can exercise query construction with stub models, decoupling tests from full AR setup where possible.
+- The "extract heavy class methods into focused query classes" pattern is established as the project's stance for similar future refactors.

data/docs/adr/0003-keep-event-dispatcher-broker.md

@@ -0,0 +1,41 @@
+---
+title: "Keep the EventDispatcher broker; do not inline"
+---
+
+# Keep the EventDispatcher broker; do not inline
+
+**Status:** accepted
+
+An architecture review surfaced `EventDispatcher` as a possible "one adapter = hypothetical seam" — the internal-subscriber list (`value_change_internals`) is intentionally generic, and the broker pattern looked over-built for a 1:1 relationship. On closer inspection the broker is correctly designed for its public/internal callback split, and inlining would cost more than it saved.
+
+We're keeping `EventDispatcher` as-is. The internal-vs-user-proc split is load-bearing on three counts that don't survive an inline.
+
+## Why the broker stays
+
+The broker remains useful for the shipped split between generic internal
+observers and public callbacks. The earlier Phase 7 materialized-view plan is
+retired and is not a current subscriber, ordering promise, or implementation
+milestone; future consumers must register and document their own contract.
+
+**The error policy split is two different contracts, not stylistic.**
+
+- Internal subscribers fail-closed: exceptions PROPAGATE. Transactional
+  versioning is outside this broker; its failures propagate inside and roll
+  back the source transaction.
+- User procs fail-soft: `rescue StandardError`, log via `Rails.logger.error`, swallow. The Value/Field row is already committed when `after_commit` fires; re-raising would surface a misleading "save failed" error to the caller, when the save actually succeeded.
+
+The broker is what enforces this split. Inlining would either duplicate the rescue logic across every subscriber site (bug surface) or collapse the contracts (silently demotes internal errors to logged-and-swallowed, breaking the fail-closed invariant).
+
+**The user-proc seam is public API and stays.** `Config.on_value_change` / `Config.on_field_change` are documented in README. External callers register here. Any refactor would have to preserve them — at which point the question becomes "do you keep the broker for the user procs and inline only the internals?" That partial inline loses the shared registration and error-policy seam while making Value/Field know each subscriber.
+
+## Considered alternatives
+
+- **(c) Collapse only the value-change internals path.** Direct calls from `Value#after_commit` to subscribers. Rejected because the broker preserves a generic future-consumer seam and the public/internal error-policy split.
+- **(d) Original "inline the broker" recommendation.** Rejected because the broker is the documented callback boundary, not because of an imminent materialized projection.
+- **(b) Defer the question until a future consumer exists.** Rejected because the current seam already has a stable public contract and does not need a speculative redesign.
+
+## Where the friction came from
+
+The friction my original review identified was cognitive (tracing through Value → EventDispatcher → Subscriber → Registry takes four files) rather than architectural. Each file is doing one job well. The trace looks long because audit-trail plumbing genuinely involves four concerns (emit, route, write, gate per-entity opt-in); collapsing them would lose the seams, not the work.
+
+Future contributors who hit the same "shouldn't this be inlined?" reaction should land here first.

data/docs/adr/0004-field-family-intermediate-bases.md

@@ -0,0 +1,49 @@
+---
+title: "Introduce family intermediate bases on `Field` (ValidatedString, RangeBounded, Optionable)"
+---
+
+# Introduce family intermediate bases on `Field` (ValidatedString, RangeBounded, Optionable)
+
+**Status:** accepted
+
+The Field STI hierarchy was nearly flat — 20+ leaf subclasses all directly under `Field::Base` (the only exception being Percentage < Decimal). Three real duplication families had grown across the leaves: Text/Email/Url duplicated min/max_length + pattern validation (~30 lines × 3); Integer/Decimal/Date/DateTime duplicated min/max range patterns with inconsistencies (Integer/Decimal had a macro-level `validates :max, comparison:` check; Date/DateTime didn't); Select/MultiSelect duplicated `optionable?` + `allowed_values` + option-inclusion validators. Each family had three or more leaves — well above the "two adapters = real seam" threshold.
+
+We introduced three intermediate types — one inheritance-vs-mixin choice per family — and documented them as public extension API. ValidatedString and RangeBounded are intermediate classes (`< Field::Base`); Optionable is a concern. Leaf subclasses move to the appropriate family. The protected validation helpers move from Field::Base down to the family that owns them (validate_length/validate_pattern → ValidatedString; validate_range/_date_range/_datetime_range → RangeBounded; validate_option_inclusion/validate_multi_option_inclusion → Optionable). `validate_array_size` stays on Field::Base because its callers (MultiSelect via Optionable and IntegerArray directly) don't share a family.
+
+True stubs (Color, Boolean, Json) stay as direct children of Field::Base — no family fits them; the stub form is the correct shape when there's nothing to absorb.
+
+## The inheritance-vs-concern rule
+
+The choice per family was driven by storage shape:
+
+- **Inheritance class when children share storage.** ValidatedString's leaves (Text, Email, Url) all use `string_value`. RangeBounded's leaves use different columns, but the family is identified by "has min/max bounds" — no storage declaration on the parent; each leaf still declares its own value_column.
+- **Concern (mixin) when children don't share storage.** Optionable's leaves (Select, MultiSelect) use different columns and one is array-typed. Inheritance can't fix storage for both; a concern adds the shared behavior without claiming a parent slot.
+
+This rule should govern future family extractions in the same codebase. Mixing the two patterns is intentional, not accidental.
+
+## Grooming fixes folded in
+
+Extracting the families surfaced inconsistencies that were latent bugs:
+
+- ValidatedString's `max_gte_min_length` validator (previously only on Text) now covers Email and Url. Email configured with `max_length: 5, min_length: 10` now raises at field-save time instead of saving silently.
+- RangeBounded's `validates :max, comparison: { greater_than_or_equal_to: :min }`-style macros now cover Date and DateTime (previously only Integer and Decimal). Inverted date bounds raise at field-save time.
+
+CHANGELOG entry notes the behavior change for users who may have invalid configurations they never noticed.
+
+## Public extension API
+
+ValidatedString, RangeBounded, and Optionable are documented in README §"Custom field types" as recommended extension bases. External authors building (e.g.) a Phone or Slug field type subclass ValidatedString instead of duplicating min/max_length plumbing.
+
+## Considered alternatives
+
+- **(a) Status quo + ADR.** Rejected because three subclasses per family is real duplication, and the inconsistencies (max_gte_min_length only on Text; comparison macro only on Integer/Decimal) prove the duplication was already drifting.
+- **(d) Registry-driven declarations for true stubs.** Rejected because it breaks STI for affected types — Field would need two parallel type-identification schemes (class names vs registry keys). Cost outweighs the gain of removing three small files.
+- **(a3) All families as concerns** (no intermediate classes). Rejected because it abandons the existing Percentage < Decimal precedent and forces an unnatural "include for these but inherit for those" rule with no clear principle.
+- **(b2) Internal-only family bases.** Rejected because external authors face the same duplication; keeping the bases gem-private solves the gem maintainer's problem and leaves external authors with the original friction.
+
+## Consequences
+
+- Test surface shrinks: per-leaf specs cover only leaf-specific behavior. Shared family behavior tested once on the family's spec.
+- Future custom-field-type authors get the family's validation surface for free by picking the right parent.
+- The "inheritance when children share storage; concern otherwise" rule is now an established pattern. Future families (e.g., if "array-typed" grows beyond MultiSelect + IntegerArray + others) should apply the same test.
+- One outlier remains: `validate_array_size` lives on Field::Base because its callers span unrelated families. This is acknowledged technical debt; if a third array-family caller emerges, extract to an ArraySupport concern.

data/docs/adr/0005-keep-phase-six-modules-independent.md

@@ -0,0 +1,48 @@
+---
+title: "Keep Phase-6 modules independent; do not introduce an import-pipeline orchestrator"
+---
+
+# Keep Phase-6 modules independent; do not introduce an import-pipeline orchestrator
+
+**Status:** accepted
+
+An architecture review surfaced `BulkWrite`, `CSVMapper`, and `SchemaPortability` as a possible missing-orchestrator situation: three Phase-6 modules that *could* form a 4-step import pipeline (export schema → import schema → parse CSV → bulk-write) but don't share an interface, return three different result shapes, and force users assembling the full flow to write per-module glue. On closer inspection the three are correctly scoped as independent toolbox utilities, not a fragmented pipeline.
+
+We're keeping them as-is. Each module owns a focused responsibility and is independently useful:
+
+- `SchemaPortability` moves Field+Section definitions between environments. Used standalone for environment-to-environment schema sync — no CSV or bulk-write involved.
+- `CSVMapper.row_to_attributes` is a pure stateless transform with no record context. Used standalone for preview UIs, custom validation flows, anything that needs "header → field name" mapping without committing to a write.
+- `BulkWrite.execute` runs batch typed-value writes with savepoint isolation and version-group stamping. Used standalone for bulk imports from non-CSV sources (JSON APIs, admin forms, migrations).
+
+The "import pipeline" is a usage pattern when users compose all three — not an architectural concept the gem needs to name.
+
+## Why the asymmetric return shapes are correct
+
+The three return shapes match each module's natural axis:
+
+- `BulkWrite` returns `{ successes: [...], errors_by_record: {...} }` because per-record error attribution is what a bulk-write caller needs.
+- `SchemaPortability.import_schema` returns aggregate counts (`created`, `updated`, `skipped`, `unchanged`, `errors`) because schema import is a category-of-action operation, not per-row.
+- `CSVMapper` returns a `Result` value object (`attributes`, `errors`, `success?`) because it transforms one row at a time.
+
+Forcing a unified `Result` shape across all three would either lose the per-record axis (BulkWrite) or shoehorn aggregate counts into a `success?`/`failure?` boolean that doesn't fit (SchemaPortability). The shapes diverge because the use cases diverge.
+
+## Why an orchestrator class would hurt
+
+An `ImportPipeline` class taking a schema Hash + CSV source + host class would either:
+- Hardcode a single pipeline shape (export-then-import-then-CSV-then-bulk), inflexible for users with custom flows, OR
+- Become a fluent builder that duplicates each module's existing API in a wrapper, growing surface area without addressing the original modules.
+
+Users assembling the full flow today write straightforward composition code. That's correct — the composition is theirs, not the gem's.
+
+## Considered alternatives
+
+- **(b) Unify result shapes only.** Rejected because each return shape's axis is correct for its module's domain.
+- **(c) Add `ImportPipeline` orchestrator.** Rejected because it imposes a pipeline shape on users who compose modules differently.
+- **(d) b + c.** Rejected for the union of both objections.
+- **(e) Minor error-message standardization.** No concrete inconsistencies worth a separate change — each module's error vocabulary fits its domain.
+
+## Where the friction came from
+
+The original review identified cognitive friction (three modules with similar names; no obvious "pipeline" home) and proposed an orchestrator. The friction doesn't survive contact with the implementations: CSVMapper's docstring explicitly documents its scope as "a pure stateless transform with no record context," BulkWrite's docstring states its purpose as "internal executor for host-class bulk typed-value writes," and SchemaPortability's docstring states its purpose as "export and import field + section definitions for an exact partition tuple." Each module is doing one thing well and saying so out loud.
+
+Future contributors who hit the same "shouldn't these compose under one orchestrator?" reaction should land here first.

data/docs/adr/0006-include-missing-via-set-complement.md

@@ -0,0 +1,77 @@
+---
+title: "Compose `include_missing:` via set-complement at the FilterQuery altitude"
+---
+
+# Compose `include_missing:` via set-complement at the FilterQuery altitude
+
+**Status:** accepted
+
+`Entity.with_field("status", :is_null)` today matches only hosts that have a `typed_eav_values` row whose value-column is NULL. Hosts that have no row at all are invisible — a common usability gap when admins build "is empty" filters. The G3 ticket (#19) adds an opt-in `include_missing:` keyword that broadens the `:is_null` semantic to "no non-NULL value, including no-row hosts" (Reading A).
+
+The PRD's first sketch proposed a LEFT JOIN inside `QueryBuilder` for the `:is_null` branch. We're rejecting that framing and composing the wider predicate at the `FilterQuery` altitude as a **set complement** instead. `QueryBuilder` is not modified.
+
+## Decision
+
+In `FilterQuery`:
+
+- **Single-scope branch** when `include_missing: true && operator == :is_null`:
+  ```ruby
+  non_missing_ids = QueryBuilder.entity_ids(field, :is_not_null, nil)
+  query.where.not(id: non_missing_ids)
+  ```
+- **Multimap (`ALL_SCOPES`) branch** when `include_missing: true && operator == :is_null`:
+  ```ruby
+  non_missing_ids = fields.flat_map { |f|
+    QueryBuilder.entity_ids(f, :is_not_null, nil).pluck(:entity_id)
+  }.uniq
+  query.where.not(id: non_missing_ids)
+  ```
+- `:is_not_null` + `include_missing: true` → no-op. The natural complement already covers the intent.
+- Any other operator + `include_missing: true` → silently ignored.
+
+The wrapper methods `Entity.with_field` and `Entity.where_typed_eav` accept `include_missing: false` by default and thread it through to `FilterQuery#initialize`.
+
+## Why set-complement, not LEFT JOIN
+
+The LEFT JOIN framing would push the "no row" branch into `QueryBuilder`, which today is a per-field SQL primitive that knows nothing about multi-filter composition, partition collision, or the multimap-vs-single-scope split (ADR-0002). Adding a LEFT JOIN there:
+
+- Forks the `:is_null` branch's return-type contract (the existing `filter` returns an `ActiveRecord::Relation` of `TypedEAV::Value` records suitable for `select(:entity_id)`; a LEFT JOIN against the host table breaks that shape).
+- Introduces a host-table dependency at the per-field altitude, which is exactly the multi-filter composition concern `FilterQuery` was extracted to own.
+- Doesn't generalise cleanly to the multimap branch — "no non-NULL value across any matching field def" is set-complement at the host level, not a per-field LEFT JOIN.
+
+Set-complement at `FilterQuery` reuses the existing `:is_not_null` primitive verbatim. `QueryBuilder.entity_ids(field, :is_not_null, nil)` returns the hosts that DO have a non-NULL value; `where.not(id: ...)` is the host-level complement. The math is "all hosts minus hosts with a value," which is precisely Reading A.
+
+## Reading A vs Reading B on the multimap branch
+
+The multimap branch unions field definitions across tenants (e.g. `name` defined separately for `ws-1`, `ws-2`, `ws-3`). When a user asks for "is empty," two readings are possible:
+
+- **Reading A — "no non-NULL value across ANY matching field def."** A host matches iff none of the per-tenant field defs have a non-NULL value for it. A host with a NULL row in ws-1 and a populated row in ws-2 does NOT match (it has a non-NULL value in ws-2).
+- **Reading B — "no row for any field def."** A host matches iff it has zero rows across all the matching field defs. A host with a NULL row anywhere does not match.
+
+We're pinning **Reading A**. Rationale:
+
+1. Reading A is the single-scope semantic, generalised. Users who escalate from a single-scope query to an `unscoped { }` block don't expect the meaning of `:is_null` to flip on them.
+2. Reading B is operationally indistinguishable from "do any rows exist," which is a different question with its own clear phrasing.
+3. The set-complement implementation falls out naturally for Reading A — union the non-missing entity_ids across all matching field defs, then complement. Reading B would need a separate row-existence query that doesn't reuse `:is_not_null`.
+
+The `FilterQuery` RDoc pins Reading A explicitly so future contributors don't second-guess it.
+
+## Why `:is_not_null` is a no-op
+
+`:is_not_null` already returns the natural complement of `:is_null`'s NULL-row-only semantic. Layering `include_missing: true` on top would either (a) flip the meaning of `:is_not_null` to "has a non-NULL value OR has no row" (incoherent — a no-row host has neither a value nor a NULL), or (b) silently leave it alone. We pick (b) so filter UIs can pass `include_missing: true` uniformly without branching per operator.
+
+## Why other operators silently ignore
+
+Same UI ergonomics. A filter UI that exposes "Include records with no value" as a checkbox should be able to pass `include_missing: true` regardless of the current operator selection. `:eq`, `:gt`, `:contains`, `:references`, `:between`, `:starts_with`, etc. all have their own well-defined semantics that don't compose with "or has no row" in a useful way. Silent-ignore is the least-surprise behavior for a filter UI.
+
+## Considered alternatives
+
+- **LEFT JOIN inside `QueryBuilder`.** Rejected — forks the return-type contract and pushes multi-filter composition concerns into the per-field primitive (see "Why set-complement, not LEFT JOIN").
+- **A new operator symbol (`:is_empty`).** Rejected — proliferates the operator vocabulary and forces filter UIs to branch on operator selection. The opt-in kwarg keeps the operator surface stable.
+- **Reading B on the multimap branch.** Rejected — see "Reading A vs Reading B."
+- **Make `include_missing:` default `true`.** Rejected — would silently change the meaning of `:is_null` for existing callers. The kwarg is opt-in by design.
+
+## References
+
+- Issue #19 — G3 PRD.
+- ADR-0002 — `EntityQuery` / `FilterQuery` / `QueryBuilder` altitude split.

data/docs/adr/0007-visibility-versus-mutation-relations.md

@@ -0,0 +1,33 @@
+---
+title: "ADR 0007: Visibility versus mutation relations"
+---
+
+# ADR 0007: Visibility versus mutation relations
+
+## Status
+
+Accepted
+
+## Decision
+
+Typed EAV keeps two relations for partitioned definitions:
+
+- `for_entity` is a visibility relation. It includes the requested
+  `(entity_type, scope, parent_scope)` row plus the global and partial-scope
+  fallback rows used for collision precedence.
+- `for_partition` is a mutation relation. It matches `entity_type`, `scope`,
+  and `parent_scope` exactly, including SQL `NULL` axes, and is used by
+  ordering mutations and their `FOR UPDATE` locks.
+
+Field ordering queries `TypedEAV::Field::Base.for_partition` so STI field
+subclasses remain one ordering partition. Section ordering queries the Section
+base relation directly for the same reason of keeping the mutation boundary
+independent from the receiver's relation scope.
+
+## Consequences
+
+Moving a definition cannot renumber a global fallback, a scope-only fallback,
+or a different full `(scope, parent_scope)` tuple. Visibility lookup and its
+collision precedence remain unchanged. Mutation locks still run in one
+transaction, acquire every exact-partition row in deterministic `id` order,
+and normalize only those rows.

data/docs/adr/0008-partial-covering-scalar-indexes.md

@@ -0,0 +1,83 @@
+---
+title: "ADR 0008: Partial-covering scalar indexes"
+---
+
+# ADR 0008: Partial-covering scalar indexes
+
+## Status
+
+Accepted
+
+## Context
+
+Each `typed_eav_values` row has one applicable typed cell and normally leaves
+the other typed cells NULL. The shipped scalar indexes are full B-trees on
+`(field_id, typed_value)` with `INCLUDE (entity_id, entity_type)`, so every
+scalar index maintains many rows that cannot satisfy a non-NULL typed query.
+
+The representative scalar comparison selected a partial-covering layout as
+the balanced default. The follow-up NULL comparison tested whether dropping
+NULL entries required an automatic replacement index. It used 100,000 hosts,
+six typed field families, 5,000 missing integer rows, and both 1% and 50%
+explicit NULL among present integer rows. Three same-seed trials ran under
+resource-capped co-tenancy; decisions use relative plans, buffers, storage,
+and WAL rather than absolute latency.
+
+## Decision
+
+Replace the six scalar indexes with:
+
+```sql
+(field_id, typed_value) INCLUDE (entity_id)
+WHERE typed_value IS NOT NULL
+```
+
+The string index retains `text_pattern_ops`. The migration uses these stable
+names:
+
+- `idx_te_values_field_int_present`
+- `idx_te_values_field_dec_present`
+- `idx_te_values_field_date_present`
+- `idx_te_values_field_dt_present`
+- `idx_te_values_field_bool_present`
+- `idx_te_values_field_str_present`
+
+Do not ship automatic typed NULL indexes. Applications with a measured,
+low-NULL workload dominated by explicit-NULL or `eq nil` probes may evaluate a
+targeted concurrent index such as:
+
+```sql
+CREATE INDEX CONCURRENTLY app_te_values_integer_null
+ON typed_eav_values (field_id, entity_id)
+WHERE integer_value IS NULL;
+```
+
+That guidance is opt-in and requires application-specific `EXPLAIN (ANALYZE,
+BUFFERS)` and write/storage measurements. Under current query semantics, the
+predicate also indexes every logical value stored in another typed column.
+
+## Evidence
+
+The partial-covering scalar layout reduced total relation bytes 54.6%, all
+index bytes 67.2%, insert WAL 61.3%, and improved insert/update throughput
+76.9%/131.1% relative to the shipped covering layout while preserving the
+core non-NULL equality/range plan shape.
+
+In the NULL follow-up, an additional integer NULL index contained 500,947 of
+595,000 rows at low logical NULL and 547,737 at high logical NULL. At low NULL,
+it reduced explicit-NULL and `eq nil` median root-plan buffers 83.4%, but added
+37.5% index bytes and 28.7% insert WAL. At high NULL, it increased those query
+buffers 84.4%, index bytes 43.4%, and insert WAL 32.4%. It increased
+NULL-inclusive `not_eq` buffers in both distributions and did not change
+`is_not_null` or `include_missing` plans. The raw evidence is in
+`bench/results/phase-2-scalar-representative.json` and
+`bench/results/phase-2-null-distributions.json`.
+
+## Migration consequences
+
+The upgrade migration must be nontransactional and use concurrent index DDL.
+It creates all six new indexes before dropping any legacy scalar index, so an
+upgrade does not open a core-query coverage gap. The down path recreates every
+legacy `(field_id, typed_value) INCLUDE (entity_id, entity_type)` index before
+removing its partial-covering replacement. Shipped migrations remain
+unchanged, rollback is explicit, and no NULL index is created or dropped.

data/docs/adr/0009-string-search-indexing.md

@@ -0,0 +1,110 @@
+---
+title: "ADR 0009: Application-owned trigram indexes for measured string workloads"
+---
+
+# ADR 0009: Application-owned trigram indexes for measured string workloads
+
+## Status
+
+Accepted
+
+## Context
+
+TypedEAV's public string operators use three distinct SQL contracts: equality
+uses `=`, positive pattern operators use `ILIKE`, and `not_contains` uses
+`NOT ILIKE`. The shipped partial-covering `text_pattern_ops` B-tree preserves
+efficient equality. It does not make every `ILIKE` shape efficient.
+
+The Phase 3 comparison kept those predicates unchanged while adding either a
+`lower(string_value) text_pattern_ops` B-tree or a partial `pg_trgm` GIN index.
+The deterministic dataset contained 250,000 target-field rows and 250,000
+noise-field rows. Three candidate-order rotations ran on PostgreSQL 17.11
+under resource-capped co-tenancy. Disposable PostgreSQL 15.19, 16.15, and 18.6
+lanes tested extension lifecycle only; they are not planner evidence.
+
+## Decision
+
+Keep the shipped partial-covering B-tree and public SQL unchanged. TypedEAV
+does not require `pg_trgm`, install it, create a trigram index, or provide an
+installer or generator for one.
+
+Applications whose measured workload is dominated by positive `ILIKE`
+patterns with extractable trigrams may evaluate an application-owned partial
+GIN index:
+
+```sql
+CREATE INDEX CONCURRENTLY app_te_values_string_trgm
+ON typed_eav_values USING gin (string_value gin_trgm_ops)
+WHERE string_value IS NOT NULL;
+```
+
+The decision is operator- and workload-specific:
+
+| Public query | Shipped B-tree | Observed use of additive GIN | Policy |
+| --- | --- | --- | --- |
+| `eq` (`=`) | Index-only in the representative plan | Not needed | Retain the B-tree |
+| `starts_with` (`ILIKE 'value%'`) | Not reliably pattern-indexed by this B-tree for public `ILIKE` | Used for the measured extractable pattern | Evaluate only with application plans/selectivity |
+| `contains` (`ILIKE '%value%'`) | May scan/filter B-tree entries or the relation | Used for the measured extractable pattern | Primary candidate for workload-specific evaluation |
+| `ends_with` (`ILIKE '%value'`) | Not suffix-indexed | Used for the measured extractable pattern | Evaluate only with application plans/selectivity |
+| Escaped `%` and `_` literals | Semantics remain escaped `ILIKE` | Used when the remaining literal supplied trigrams | Do not infer support for every escaped pattern |
+| `not_contains` (`NOT ILIKE`) | No selective negative-search guarantee | Not used | Do not recommend GIN as acceleration |
+| One- or two-character probes | No useful trigram extraction | Not used | Do not recommend GIN as acceleration |
+
+The separate `lower(string_value) LIKE ...` prototype used its expression
+B-tree, but that predicate is not the public `ILIKE` contract. It does not
+justify changing public semantics or assuming collation equivalence. GiST was
+smoke-only and has no representative justification, so it is not recommended.
+
+## Application ownership and deployment
+
+Before deployment, the application owner must confirm `pg_trgm` is available
+and that the deploy role may create it in the target database:
+
+```sql
+SELECT name, default_version, installed_version
+FROM pg_available_extensions
+WHERE name = 'pg_trgm';
+
+CREATE EXTENSION IF NOT EXISTS pg_trgm;
+```
+
+The benchmark's non-superuser database owner could manage the extension on
+stock PostgreSQL 15, 16, and 18. Hosted services may impose different
+privileges, so this is a preproduction check, not a portability promise.
+
+The consuming application should own a nontransactional migration with a
+stable application-specific index name. It must create and drop the index with
+`CONCURRENTLY`, monitor invalid indexes after interruption, and verify the
+catalog definition rather than accepting a same-named but different index.
+Rollback drops only the application-owned index:
+
+```sql
+DROP INDEX CONCURRENTLY IF EXISTS app_te_values_string_trgm;
+```
+
+Do not drop `pg_trgm` during index rollback. Extensions are database-wide and
+may be shared by unrelated application objects. Extension removal belongs in
+a separate, explicitly owned operation only after catalog checks prove no
+remaining dependants.
+
+Before keeping the index, compare representative before/after plans with
+`EXPLAIN (ANALYZE, BUFFERS, WAL, SETTINGS)`, including real pattern lengths,
+selectivity, field distribution, and concurrency. Also measure index and total
+relation bytes, build and write WAL, insert/update throughput, and ongoing
+maintenance. A per-field partial predicate may be worth testing for a known
+hot field, but it was not the Phase 3 candidate and requires its own evidence.
+
+## Consequences and evidence limits
+
+The additive GIN candidate increased median candidate index bytes by 61.417%
+and build WAL by 50.293% versus the current B-tree candidate. Median insert and
+update throughput fell 56.995% and 58.060%, while their WAL rose 156.499% and
+195.441%. These costs make automatic installation inappropriate.
+
+The retained artifact contains the full plans, checksums, sizes, and build
+measurements for trial 1 plus cross-trial metric arrays and rotation metadata.
+It does not retain the raw trial 2/3 plans, checksums, sizes, or build times, so
+those items are not independently auditable from the artifact. The result is
+relative evidence under active co-tenants; absolute timings are diagnostic,
+and PostgreSQL 17 plan choices must not be generalized to every version or
+workload. Applications should rerun the comparison in their own environment.