typed_eav 0.3.2 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '09c84a25e8888fd675c1225a2a20cf0dfcfcbc26f6ccc5fe025d70fe8ba8269b'
-  data.tar.gz: 2f5308382740430050acbefc7e857f6b2f4d17ea906991def3c43316ee017820
+  metadata.gz: 392d8c372c1b70b4b710cef0b9f67d79531fb237bbca37419906ed468ced365f
+  data.tar.gz: e8d9fd82f5d0d3a2cd894f89090a31522882c0606f28e691be373aacccf73cce
 SHA512:
-  metadata.gz: efe552ea5914d737eb82b2d6c33a6198c5d0c161d1e0c08d7037c4e52f85b8b199a0ea608b15b910004dc7e645c3b3a1a3099b07f847cefac14380103f5cbab4
-  data.tar.gz: 283c08abec00ff9604d56ac7d5cacec43b1596e9c9ba44710a5087edf326218bd075b6b9ac7d2098b9934ea4c86dd40588cb8a7bc5e72c4b7818c0ea532f76b2
+  metadata.gz: 0fa26b021b7d5223f1af10e016dad89c546102f8f807d00fa6d3ea8d07a95a3e91948fa67e09137b2d93bb4cba38bf0ed2a91887812a7777fef4821bd87db190
+  data.tar.gz: 195ee963d09d43cb358ff2f39ee4f62bd131cb75fbb110bbe6a00e569c5b693fbd21313e27f899e492edc9ebb921a1021bd432aa754059df4f609d6fa50cd30a

data/CHANGELOG.md

@@ -5,6 +5,103 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.0] - 2026-05-26
+
+Closes four follow-up gaps (PRD #15) surfaced when a downstream Rails app
+consolidated onto 0.3.2 and found four places where the gem's public
+surface forced workarounds: a missing per-record-varying bulk-write entry
+point, a dedup defect on unsaved entities with in-memory `typed_values`
+builds, an `:is_null` operator that couldn't honor the user-intuitive "is
+empty" semantic, and a portable-schema shape that was wrong for in-app
+snapshot stores. A fifth gap (G5) was scoped down to a documentation
+promotion on the existing `Partition` module rather than a new wrapper.
+
+All five changes are additive — no public-API breakage. New entry points
+default to current shapes; existing callers of `bulk_set_typed_eav_values`,
+`with_field`/`where_typed_eav` (without the new kwarg), `export_schema`,
+and `initialize_typed_values` (on persisted records with no in-memory
+builds) keep their behavior byte-for-byte. New ADR: ADR-0006 pins the G3
+`include_missing` strategy as set-complement at the `FilterQuery` altitude
+(rejects the LEFT JOIN framing the PRD originally sketched).
+
+### Added
+
+- `Entity.bulk_set_typed_eav_values_per_record(values_by_record,
+  version_grouping: :default)` — per-record-varying sibling to
+  `bulk_set_typed_eav_values`. Takes a `Hash<host_record,
+  Hash<field_name, value>>` and routes each record's value-set through
+  the same outer-transaction-plus-savepoint-per-record envelope,
+  returning the same `{ successes: [...], errors_by_record: { record
+  => errors_hash } }` shape. Supports sparse-update semantics
+  (unlisted fields untouched), `{ _destroy: true }` value-removal
+  shorthand, mixed-scope records (each record honors its own
+  `[scope, parent_scope]` even inside `TypedEAV.unscoped { ... }`),
+  and `:per_field` UUID allocation across the union of field names.
+  Empty input short-circuits without opening a transaction. Internally,
+  both public executors (`BulkWrite.execute` and
+  `BulkWrite.execute_per_record`) now share a single
+  `execute_pairs(pairs, effective_grouping, field_uuids)` helper that
+  takes ordered `[record, vbn]` pairs — preserving `execute`'s byte-
+  for-byte behavior on duplicate in-memory instances of the same
+  persisted row (Hash-key collision is documented as a gotcha only
+  on the new API). G1 (issue #18).
+
+- `Entity.with_field` and `Entity.where_typed_eav` accept an opt-in
+  `include_missing:` keyword (default `false`). Threaded through to
+  `FilterQuery#initialize`. When paired with `:is_null`, the operator
+  matches hosts with **no non-NULL value** for the field — including
+  hosts that have no `typed_eav_values` row at all (Reading A: the
+  user-intuitive "is empty" semantic). Implemented as a set-complement
+  against `:is_not_null` at the `FilterQuery` altitude; `QueryBuilder`
+  is not modified. With `:is_not_null` the kwarg is a no-op; with any
+  other operator (`:eq`, `:gt`, `:contains`, `:references`, `:between`,
+  `:starts_with`, etc.) it is silently ignored — filter UIs can pass
+  the kwarg uniformly without branching per operator. On the multimap
+  (`ALL_SCOPES`) branch, "no non-NULL value" reads across all matching
+  field definitions for the name: a host matches iff none of the
+  per-tenant field defs have a non-NULL value for it. G3 (issue #19).
+  See ADR-0006.
+
+- `TypedEAV::SchemaPortability.export_snapshot_schema(entity_type:,
+  scope: nil, parent_scope: nil)` — sibling to `export_schema` that
+  returns a lean, restore-oriented projection in a versioned envelope:
+  `{ "snapshot_schema_version" => 1, "fields" => [...] }`. Per-field
+  entries carry only `name`, `field_type_name`, `required`,
+  `sort_order`, `options`, and (for optionable types) `options_data`
+  — `entity_type`, `scope`, `parent_scope`, `type` (AR STI class name),
+  `field_dependent`, and `default_value_meta` are omitted. Non-optionable
+  fields omit the `options_data` key entirely (absent, not nil). The
+  `snapshot_schema_version` integer will be bumped explicitly when the
+  inner shape evolves — it is not frozen forever. Fields are ordered by
+  `sort_order` and `options_data` mirrors the loaded/unloaded ordering
+  rule used by `export_schema`. G4 (PRD #15).
+
+### Documentation
+
+- `TypedEAV::Partition.find_visible_section!` is **documented-public**
+  going forward. Apps building admin UIs that need to authorize a
+  section lookup before editing, rendering, or destroying it should
+  call this rather than `Section.find(id)`. Method shape and behavior
+  do not change — this is a documentation clarification that promotes
+  an existing, already-shipping method into the documented surface
+  area, alongside the sibling `Partition` methods (`visible_fields`,
+  `effective_fields_by_name`, `definitions_by_name`,
+  `definitions_multimap_by_name`, `visible_sections`). G5 (issue #20).
+
+### Fixed
+
+- `InstanceMethods#initialize_typed_values` no longer builds duplicate
+  rows on entities that already have in-memory `typed_values` builds
+  (form path with `field_id`, scripting path via `typed_eav_attributes=`,
+  or direct `typed_values.build(...)` on a persisted record). Covers
+  three cases: (1) new record + nested attributes, (2) new record +
+  scripting setter, (3) persisted record + unloaded association + a
+  build that lives in `target` without flipping `@loaded`. The
+  persisted-no-builds fast path still uses `pluck` only — no extra
+  association load. Dedup also tolerates an in-memory build whose
+  `field_id` is nil but whose `field` association is set
+  (`field_id || field&.id` fallback). G2 (PRD #15).
+
 ## [0.3.2] - 2026-05-25
 
 Documentation-only release. No code or behavior changes.

data/lib/typed_eav/bulk_write.rb

@@ -3,9 +3,25 @@
 module TypedEAV
   # Internal executor for host-class bulk typed-value writes.
   #
-  # Host models keep the public `bulk_set_typed_eav_values` API; this module
-  # owns the transaction shape, savepoint isolation, error aggregation, field
-  # name resolution delegation, and version-group stamping.
+  # Host models keep the public `bulk_set_typed_eav_values` (uniform
+  # values-per-record) and `bulk_set_typed_eav_values_per_record`
+  # (per-record-varying values) APIs; this module owns the transaction
+  # shape, savepoint isolation, error aggregation, field name resolution
+  # delegation, and version-group stamping.
+  #
+  # ## Internal shape (G1, issue #18)
+  #
+  # Both public executors (`execute` and `execute_per_record`) are thin
+  # adapters: they validate their inputs, resolve the version grouping,
+  # allocate field UUIDs, and then hand off to `execute_pairs(pairs,
+  # effective_grouping, field_uuids)` — a single shared loop that takes
+  # ordered `[record, vbn]` pairs and runs the outer-transaction-plus-
+  # savepoint-per-record envelope.
+  #
+  # Pair-shaped (not Hash-shaped) so `execute`'s `[record, vbn]` list can
+  # carry duplicate in-memory instances of the same persisted row without
+  # silently collapsing them via Hash-key collision — preserving
+  # `execute`'s byte-for-byte behavior contract.
   module BulkWrite
     class << self
       def execute(host_class:, records:, values_by_field_name:, version_grouping: :default)
@@ -20,7 +36,29 @@ module TypedEAV
         vbn = values_by_field_name.transform_keys(&:to_s)
         field_uuids = effective_grouping == :per_field ? vbn.keys.index_with { SecureRandom.uuid } : nil
 
-        execute_records(records, vbn, effective_grouping, field_uuids)
+        execute_pairs(records.map { |r| [r, vbn] }, effective_grouping, field_uuids)
+      end
+
+      # Per-record-varying sibling to `execute`. Accepts a `Hash<host_record,
+      # Hash<field_name, value>>` and routes each record's value-set through
+      # the same shared `execute_pairs` envelope.
+      #
+      # Empty `values_by_record` short-circuits to the empty result without
+      # opening a transaction (matches `execute`'s empty-records contract).
+      def execute_per_record(host_class:, values_by_record:, version_grouping: :default)
+        validate_per_record_inputs!(values_by_record, version_grouping)
+
+        return { successes: [], errors_by_record: {} } if values_by_record.empty?
+
+        validate_record_classes!(host_class, values_by_record.keys, method: :bulk_set_typed_eav_values_per_record)
+
+        effective_grouping = resolve_grouping(version_grouping)
+        pairs = values_by_record.map { |record, vbn| [record, vbn.transform_keys(&:to_s)] }
+        field_uuids = if effective_grouping == :per_field
+                        pairs.flat_map { |(_record, vbn)| vbn.keys }.uniq.index_with { SecureRandom.uuid }
+                      end
+
+        execute_pairs(pairs, effective_grouping, field_uuids)
       end
 
       def apply_record_save(record:, vbn:, effective_grouping:, uuids:, accumulator:)
@@ -30,7 +68,7 @@ module TypedEAV
                     end
 
         do_save = lambda do
-          record.typed_eav_attributes = vbn.map { |name, value| { name: name, value: value } }
+          record.typed_eav_attributes = vbn.map { |name, value| typed_eav_entry_for(name, value) }
           stamp_pending_version_group_ids(record, effective_grouping, uuids)
 
           if record.save
@@ -50,24 +88,36 @@ module TypedEAV
 
       private
 
-      def execute_records(records, vbn, effective_grouping, field_uuids)
+      # Shared loop over ordered `[record, vbn]` pairs. Holds the outer
+      # transaction, the `ActiveRecord::Base.cache` block, the bulk-
+      # definitions memo envelope, and the per-record `requires_new: true`
+      # savepoint loop. Calls `apply_record_save` per iteration.
+      #
+      # Pair-shaped (not Hash-shaped) so duplicate in-memory instances of
+      # the same persisted row iterate each instance separately — matters
+      # for `execute`'s byte-for-byte behavior contract on
+      # `[Entity.find(1), Entity.find(1)]`-shaped input.
+      def execute_pairs(pairs, effective_grouping, field_uuids)
         successes = []
         errors_by_record = {}
 
         with_bulk_definitions_memo do
           ActiveRecord::Base.cache do
             ActiveRecord::Base.transaction do
-              records.each do |record|
+              pairs.each do |(record, vbn)|
                 record_uuid = effective_grouping == :per_record ? SecureRandom.uuid : nil
+                record_field_uuids = record_scoped_field_uuids(field_uuids, vbn)
 
-                ActiveRecord::Base.transaction(requires_new: true) do
-                  apply_record_save(
-                    record: record,
-                    vbn: vbn,
-                    effective_grouping: effective_grouping,
-                    uuids: { record: record_uuid, field: field_uuids },
-                    accumulator: { successes: successes, errors_by_record: errors_by_record },
-                  )
+                with_record_scope(record) do
+                  ActiveRecord::Base.transaction(requires_new: true) do
+                    apply_record_save(
+                      record: record,
+                      vbn: vbn,
+                      effective_grouping: effective_grouping,
+                      uuids: { record: record_uuid, field: record_field_uuids },
+                      accumulator: { successes: successes, errors_by_record: errors_by_record },
+                    )
+                  end
                 end
               end
             end
@@ -77,6 +127,63 @@ module TypedEAV
         { successes: successes, errors_by_record: errors_by_record }
       end
 
+      # For `:per_field`, the global `field_uuids` map covers the union of
+      # field names across all records' value hashes. The per-record save
+      # only needs the slice for the names actually being written on THIS
+      # record so `apply_record_save`'s push_uuid lookup
+      # (`uuids[:field].values.first`) stays well-defined.
+      def record_scoped_field_uuids(field_uuids, vbn)
+        return nil unless field_uuids
+
+        vbn.keys.index_with { |name| field_uuids[name] }
+      end
+
+      # Honor the record's own `[scope, parent_scope]` when iterating a
+      # potentially mixed-scope batch. Inside `TypedEAV.unscoped { ... }`,
+      # `EntityQuery#resolve_scope` short-circuits to `ALL_SCOPES` and
+      # ignores the explicit `scope:` kwarg — which would surface the
+      # wrong field definitions for a record whose scope/parent_scope
+      # differs from its siblings. We restore strict scoping per record by
+      # temporarily clearing the `unscoped?` flag and pushing the record's
+      # own tuple onto the `with_scope` stack. Records on hosts without
+      # `scope_method:` (no `typed_eav_scope` to read) pass through
+      # unchanged.
+      def with_record_scope(record, &)
+        scope_method = record.class.respond_to?(:typed_eav_scope_method) ? record.class.typed_eav_scope_method : nil
+        return yield unless scope_method
+
+        s  = record.typed_eav_scope
+        ps = record.typed_eav_parent_scope
+
+        # `:typed_eav_unscoped` is the same thread-local key used by
+        # `TypedEAV.unscoped` / `TypedEAV.unscoped?` (the constant is
+        # `private_constant` on `TypedEAV`; we use the literal symbol to
+        # avoid reaching into private internals).
+        prev_unscoped = Thread.current[:typed_eav_unscoped]
+        Thread.current[:typed_eav_unscoped] = nil
+        TypedEAV.with_scope([s, ps], &)
+      ensure
+        Thread.current[:typed_eav_unscoped] = prev_unscoped if scope_method
+      end
+
+      # Translates one `(name, value)` from a vbn hash into the nested-
+      # attributes entry shape that `typed_eav_attributes=` expects.
+      # `_destroy: true` shorthand (`{ "field" => { _destroy: true } }`)
+      # is detected here and emitted as `{ name:, _destroy: true }` so
+      # the value is removed rather than written as a literal Hash payload.
+      def typed_eav_entry_for(name, value)
+        return { name: name, _destroy: true } if destroy_marker?(value)
+
+        { name: name, value: value }
+      end
+
+      def destroy_marker?(value)
+        return false unless value.is_a?(Hash)
+
+        flag = value[:_destroy] || value["_destroy"]
+        ActiveRecord::Type::Boolean.new.cast(flag)
+      end
+
       def validate_inputs!(records, values_by_field_name, version_grouping)
         if records.nil?
           raise ArgumentError,
@@ -89,6 +196,28 @@ module TypedEAV
                 "got #{values_by_field_name.class}"
         end
 
+        validate_grouping!(version_grouping)
+      end
+
+      def validate_per_record_inputs!(values_by_record, version_grouping)
+        unless values_by_record.is_a?(Hash)
+          raise ArgumentError,
+                "bulk_set_typed_eav_values_per_record requires a Hash of values_by_record, " \
+                "got #{values_by_record.class}"
+        end
+
+        values_by_record.each do |record, vbn|
+          next if vbn.is_a?(Hash)
+
+          raise ArgumentError,
+                "bulk_set_typed_eav_values_per_record: per-record value for #{record.inspect} " \
+                "must be a Hash of field-name => value, got #{vbn.class}"
+        end
+
+        validate_grouping!(version_grouping)
+      end
+
+      def validate_grouping!(version_grouping)
         valid_grouping = %i[default per_record per_field none]
         unless valid_grouping.include?(version_grouping)
           raise ArgumentError,
@@ -104,12 +233,12 @@ module TypedEAV
               "version_grouping: :none to opt out explicitly, or omit the kwarg to silently no-op."
       end
 
-      def validate_record_classes!(host_class, records)
+      def validate_record_classes!(host_class, records, method: :bulk_set_typed_eav_values)
         return if records.all?(host_class)
 
         classes = records.map { |record| record.class.name }.uniq
         raise ArgumentError,
-              "bulk_set_typed_eav_values expects records of class #{host_class.name} (or its subclasses); " \
+              "#{method} expects records of class #{host_class.name} (or its subclasses); " \
               "got mixed classes: #{classes.join(", ")}"
       end
 

data/lib/typed_eav/entity_query.rb

@@ -35,7 +35,19 @@ module TypedEAV
     #   - omitted        -> resolve from ambient (`with_scope` -> resolver -> raise/nil)
     #   - passed a value -> use verbatim (explicit override; admin/test path)
     #   - passed nil     -> filter to global-only on that axis (prior behavior)
-    def where_typed_eav(*filters, scope: UNSET_SCOPE, parent_scope: UNSET_SCOPE)
+    #
+    # `include_missing:` behavior (opt-in, default `false`):
+    #   - Only meaningful when paired with `:is_null`. When `true`, the
+    #     `:is_null` predicate broadens to the user-intuitive "is empty"
+    #     semantic: matches hosts with **no non-NULL value** for the field —
+    #     including hosts that have no `typed_eav_values` row at all
+    #     (Reading A from ADR-0006).
+    #   - With `:is_not_null`, the kwarg is a no-op (lets filter UIs pass
+    #     it uniformly without branching per operator).
+    #   - With any other operator (`:eq`, `:gt`, `:contains`, `:between`,
+    #     `:starts_with`, `:references`, etc.), the kwarg is silently
+    #     ignored.
+    def where_typed_eav(*filters, scope: UNSET_SCOPE, parent_scope: UNSET_SCOPE, include_missing: false)
       resolved = resolve_scope(scope, parent_scope)
       effective_scope, effective_parent = scope_pair(resolved)
 
@@ -44,6 +56,7 @@ module TypedEAV
         filters: filters,
         scope: effective_scope,
         parent_scope: effective_parent,
+        include_missing: include_missing,
       ).to_relation
     end
 
@@ -56,14 +69,20 @@ module TypedEAV
     # Accepts both `scope:` and `parent_scope:` kwargs with the same
     # ambient/explicit/nil semantics as `where_typed_eav`. Single-scope
     # callers (no `parent_scope:`) are unaffected.
-    def with_field(name, operator_or_value = nil, value = nil, scope: UNSET_SCOPE, parent_scope: UNSET_SCOPE)
+    #
+    # `include_missing:` (opt-in, default `false`) is forwarded to
+    # `where_typed_eav` unchanged. See its RDoc for full semantics — in
+    # short: meaningful only with `:is_null` (Reading A "no non-NULL
+    # value," includes no-row hosts), no-op with `:is_not_null`, silently
+    # ignored otherwise.
+    def with_field(name, operator_or_value = nil, value = nil, scope: UNSET_SCOPE, parent_scope: UNSET_SCOPE, include_missing: false)
       filter = if value.nil? && !operator_or_value.is_a?(Symbol)
                  # Two-arg form: with_field("name", "value") implies :eq
                  { name: name, op: :eq, value: operator_or_value }
                else
                  { name: name, op: operator_or_value, value: value }
                end
-      where_typed_eav(filter, scope: scope, parent_scope: parent_scope)
+      where_typed_eav(filter, scope: scope, parent_scope: parent_scope, include_missing: include_missing)
     end
 
     # Returns field definitions for this entity type.
@@ -105,6 +124,76 @@ module TypedEAV
       )
     end
 
+    # Per-record-varying bulk write API. Sibling to `bulk_set_typed_eav_values`
+    # for callers (sync importers, per-row updaters) where each record carries
+    # its own values hash. Routes through the same outer-transaction-plus-
+    # savepoint envelope and returns the same
+    # `{ successes: [...], errors_by_record: { record => errors_hash } }`
+    # shape. See `TypedEAV::BulkWrite` for the transaction shape and the
+    # `version_grouping:` semantics.
+    #
+    # ## Input shape
+    #
+    #   Contact.bulk_set_typed_eav_values_per_record(
+    #     alice => { "name" => "Alice", "age" => 31 },
+    #     bob   => { "name" => "Bob",   "city" => "Portland" },
+    #   )
+    #
+    # `values_by_record` is `Hash<host_record, Hash<field_name, value>>`.
+    # Field-name keys may be strings or symbols (normalized to strings).
+    # Ruby's insertion-ordered Hash invariant determines record iteration
+    # order — callers can rely on it.
+    #
+    # ## AR persisted-record hash-key collision gotcha
+    #
+    # Two distinct in-memory instances of the **same persisted row**
+    # (e.g. `Contact.find(1)` and `Contact.find(1)`) collide as Hash keys
+    # because AR's `eql?`/`hash` is defined by `class + id`. In a Hash,
+    # the second instance silently overwrites the first's value entry
+    # and only ONE save runs. If you need to apply two updates to the
+    # same row in caller order, sequence the calls outside the Hash —
+    # this API iterates whatever the Hash holds.
+    #
+    # The sibling `bulk_set_typed_eav_values(records, vbn)` API takes an
+    # Array of records and is unaffected — duplicate in-memory instances
+    # iterate each instance separately. The internal `execute_pairs`
+    # helper preserves that contract for both surfaces.
+    #
+    # ## Sparse-update semantic
+    #
+    # Unlisted fields on a record are **not** touched. To delete a value,
+    # pass the destroy-marker hash:
+    #
+    #   Contact.bulk_set_typed_eav_values_per_record(
+    #     alice => { "old_field" => { _destroy: true } },
+    #   )
+    #
+    # ## Mixed-scope records
+    #
+    # Records in one call may span multiple partitions: r1 in workspace 1
+    # and r2 in workspace 2 in the same call resolve their own scopes
+    # independently. Each record's `typed_eav_attributes=` consults the
+    # field definitions for its own `[scope, parent_scope]`. The thread-
+    # local definition memo keys `[host_class, scope, parent_scope]`
+    # collect one entry per distinct partition touched — no collisions.
+    # Callers spanning multiple partitions should wrap the call in
+    # `TypedEAV.unscoped { ... }` so each record can apply its own scope.
+    #
+    # ## `:per_field` union semantic
+    #
+    # When `version_grouping: :per_field`, the per-field UUIDs span the
+    # **union** of field names across all records. Records that both
+    # write `"name"` share one UUID for that cell; a record writing
+    # `"city"` (that no other record writes) gets its own UUID for
+    # `"city"`. Overlapping fields share a version group across records.
+    def bulk_set_typed_eav_values_per_record(values_by_record, version_grouping: :default)
+      TypedEAV::BulkWrite.execute_per_record(
+        host_class: self,
+        values_by_record: values_by_record,
+        version_grouping: version_grouping,
+      )
+    end
+
     private
 
     # Translates a resolved scope into the `(scope, parent_scope)` pair

data/lib/typed_eav/filter_query.rb

@@ -27,15 +27,31 @@ module TypedEAV
   # already-resolved scope value. `parent_scope:` is `String | nil`. Scope
   # resolution and sentinel handling live in `EntityQuery#resolve_scope`;
   # this class works on resolved tuples only.
+  #
+  # ## `include_missing:` (Reading A)
+  #
+  # Opt-in kwarg (default `false`). Only meaningful when
+  # `operator == :is_null`; silently ignored otherwise (`:is_not_null` becomes
+  # a no-op; other operators are unaffected). When `true`, the `:is_null`
+  # predicate is composed as a **set complement** against `:is_not_null`:
+  # matches hosts that have **no non-NULL value** for the field — including
+  # hosts with no `typed_eav_values` row at all.
+  #
+  # On the multimap (`ALL_SCOPES`) branch, "no non-NULL value" reads across
+  # ALL matching field definitions for the name: a host matches iff none of
+  # the per-tenant field defs have a non-NULL value for it (Reading A —
+  # NOT "no row anywhere"). See ADR-0006 for why composition happens at this
+  # altitude rather than via a LEFT JOIN in `QueryBuilder`.
   class FilterQuery
     FILTER_KEYS = %w[name n op operator value v].freeze
     private_constant :FILTER_KEYS
 
-    def initialize(model:, filters:, scope:, parent_scope:)
-      @model        = model
-      @raw_filters  = filters
-      @scope        = scope
-      @parent_scope = parent_scope
+    def initialize(model:, filters:, scope:, parent_scope:, include_missing: false)
+      @model           = model
+      @raw_filters     = filters
+      @scope           = scope
+      @parent_scope    = parent_scope
+      @include_missing = include_missing
     end
 
     def to_relation
@@ -106,8 +122,17 @@ module TypedEAV
       filters.inject(model.all) do |query, filter|
         spec  = parse_filter(filter)
         field = fields_by_name[spec[:name]] || raise_unknown_field(spec[:name], fields_by_name.keys)
-        matching_ids = TypedEAV::QueryBuilder.entity_ids(field, spec[:operator], spec[:value])
-        query.where(id: matching_ids)
+
+        if invert_is_null?(spec[:operator])
+          # Reading A: "no non-NULL value." Set-complement against the
+          # scope-winning field's `:is_not_null` subquery — includes both
+          # NULL-column rows and entities with no row at all. ADR-0006.
+          non_missing_ids = TypedEAV::QueryBuilder.entity_ids(field, :is_not_null, nil)
+          query.where.not(id: non_missing_ids)
+        else
+          matching_ids = TypedEAV::QueryBuilder.entity_ids(field, spec[:operator], spec[:value])
+          query.where(id: matching_ids)
+        end
       end
     end
 
@@ -117,11 +142,31 @@ module TypedEAV
         fields  = fields_multimap[spec[:name]]
         raise_unknown_field(spec[:name], fields_multimap.keys) unless fields&.any?
 
-        union_ids = union_entity_ids(fields, spec[:operator], spec[:value])
-        query.where(id: union_ids)
+        if invert_is_null?(spec[:operator])
+          # Reading A across all matching field defs: a host matches iff NO
+          # field def has a non-NULL value for it. Union the non-missing
+          # entity_ids across all per-tenant field defs, then complement at
+          # the host level. ADR-0006.
+          non_missing_ids = fields.flat_map do |f|
+            TypedEAV::QueryBuilder.entity_ids(f, :is_not_null, nil).pluck(:entity_id)
+          end.uniq
+          query.where.not(id: non_missing_ids)
+        else
+          union_ids = union_entity_ids(fields, spec[:operator], spec[:value])
+          query.where(id: union_ids)
+        end
       end
     end
 
+    # `include_missing: true` only modifies the `:is_null` branch (Reading A:
+    # "no non-NULL value"). For every other operator — including
+    # `:is_not_null`, which already returns the natural complement — the
+    # kwarg is silently ignored, so filter UIs can pass it uniformly without
+    # branching per operator.
+    def invert_is_null?(operator)
+      @include_missing && operator == :is_null
+    end
+
     # OR-across all field_ids that share the same name (across tenants),
     # while preserving AND between filters via the chained `.where`. Use the
     # underlying Value scope (`.filter`) and `pluck(:entity_id)` to collapse

data/lib/typed_eav/has_typed_eav/instance_methods.rb

@@ -46,7 +46,7 @@ module TypedEAV
       # render two inputs for the same name — but only the scoped one
       # round-trips on save (it wins in `typed_eav_defs_by_name`).
       def initialize_typed_values
-        existing_field_ids = typed_values.loaded? ? typed_values.map(&:field_id) : typed_values.pluck(:field_id)
+        existing_field_ids = existing_typed_value_field_ids
 
         typed_eav_defs_by_name.each_value do |field|
           next if existing_field_ids.include?(field.id)
@@ -141,6 +141,31 @@ module TypedEAV
 
       private
 
+      # Field ids already represented in `typed_values`, accounting for both
+      # persisted rows and in-memory builds. Three branches:
+      #
+      # - **new_record? or loaded?** — walk the in-memory collection, with a
+      #   `field_id || field&.id` fallback so callers who bypass the
+      #   belongs_to FK setter (e.g. assigning via `association(:field).target=`)
+      #   still get dedup-correct results.
+      # - **persisted + unloaded** — combine a cheap `pluck` of persisted rows
+      #   with any in-memory builds in `typed_values.target`. AR's
+      #   `add_to_target` (called by `build`) does not flip `@loaded`, so
+      #   target-resident builds are otherwise invisible to `pluck`. The
+      #   persisted-no-builds happy path is unaffected: `target` is empty,
+      #   `pluck` runs once, no extra association load.
+      def existing_typed_value_field_ids
+        return walk_in_memory_typed_value_field_ids if new_record? || typed_values.loaded?
+
+        persisted = typed_values.pluck(:field_id)
+        in_memory = typed_values.target.reject(&:persisted?).filter_map { |tv| tv.field_id || tv.field&.id }
+        (persisted + in_memory).uniq
+      end
+
+      def walk_in_memory_typed_value_field_ids
+        typed_values.filter_map { |tv| tv.field_id || tv.field&.id }
+      end
+
       # Selects the candidate value for `typed_eav_value`. On a collision,
       # prefer the row attached to the winning field_id; otherwise fall back
       # to the first orphan/non-collision candidate.

data/lib/typed_eav/partition.rb

@@ -88,6 +88,49 @@ module TypedEAV
         TypedEAV::Section.for_entity(entity_type, scope: scope, parent_scope: parent_scope)
       end
 
+      # Looks up a single {TypedEAV::Section} by `id` constrained to the
+      # caller's partition tuple. Documented-public surface: apps building
+      # admin UIs that need to authorize a section lookup before editing,
+      # rendering, or destroying it should call this rather than
+      # `Section.find(id)`, which would happily return a section belonging
+      # to another tenant's partition.
+      #
+      # Visibility merge matches {visible_sections}: rows whose
+      # `(entity_type, scope, parent_scope)` is either the requested tuple
+      # or the global `(scope: nil, parent_scope: nil)` partition are
+      # eligible. The most-specific-wins precedence used for field
+      # collision resolution does not apply here — section lookup is by
+      # primary key, not by name.
+      #
+      # Sibling documented-public methods on this module that share the
+      # same partition-visibility surface: {visible_fields},
+      # {effective_fields_by_name}, {definitions_by_name},
+      # {definitions_multimap_by_name}, {visible_sections}.
+      #
+      # @param id [Integer, String] the section's primary key. Blank input
+      #   is the caller's responsibility to guard upstream; this method
+      #   does not silently swallow `nil` / `""` — it forwards to
+      #   `ActiveRecord::Relation#find`, which raises
+      #   `ActiveRecord::RecordNotFound` on blank.
+      # @param entity_type [String] the host AR class name the section
+      #   belongs to (matches `Section#entity_type`).
+      # @param scope [Object, nil] the resolved scope value from the
+      #   caller's partition. `nil` means the global partition only.
+      # @param parent_scope [Object, nil] the resolved parent_scope value.
+      #   Must be `nil` when `scope` is blank (the orphan-parent invariant
+      #   shared with {visible_sections}).
+      # @param mode [Symbol] `:partition` (default) restricts to the
+      #   caller's tuple plus the global tuple. `:all_partitions` is the
+      #   deliberate admin bypass that ignores the tuple entirely —
+      #   distinct from `scope: nil`, which means "global partition only."
+      # @return [TypedEAV::Section] the section record when it belongs to
+      #   the caller's partition or the global tuple.
+      # @raise [ActiveRecord::RecordNotFound] when the section's
+      #   `(scope, parent_scope)` falls outside the visibility merge for
+      #   the requested tuple, or when no section with `id` exists.
+      # @raise [ArgumentError] when `parent_scope` is present and `scope`
+      #   is blank (orphan-parent), or when `mode` is neither
+      #   `:partition` nor `:all_partitions`.
       def find_visible_section!(id, entity_type:, scope: nil, parent_scope: nil, mode: :partition)
         visible_sections(entity_type: entity_type, scope: scope, parent_scope: parent_scope, mode: mode).find(id)
       end

data/lib/typed_eav/schema_portability.rb

@@ -27,6 +27,63 @@ module TypedEAV
         }
       end
 
+      # Lean, restore-oriented projection of the field schema for a partition
+      # tuple. Sibling to {.export_schema} — same partition filter, narrower
+      # per-field surface, no sections, no partition-identity keys.
+      #
+      # The envelope is:
+      #
+      #   {
+      #     "snapshot_schema_version" => 1,
+      #     "fields" => [ <snapshot_field_entry>, ... ]   # ordered by sort_order
+      #   }
+      #
+      # The `snapshot_schema_version` integer will be bumped explicitly when
+      # the inner per-field shape evolves in a non-additive way — it is NOT
+      # frozen forever. Consumers should branch on the version to handle
+      # cross-version snapshots.
+      #
+      # Each per-field entry is a strict subset of the full
+      # {.export_field_entry} shape:
+      #
+      #   {
+      #     "name" => field.name,
+      #     "field_type_name" => field.field_type_name,
+      #     "required" => field.required,
+      #     "sort_order" => field.sort_order,
+      #     "options" => field.options,
+      #     "options_data" => [...]   # ONLY present when field.optionable?
+      #   }
+      #
+      # Omitted vs the full schema export: `entity_type`, `scope`,
+      # `parent_scope`, `type` (the AR STI class name), `field_dependent`,
+      # and `default_value_meta`. Non-optionable fields omit `options_data`
+      # entirely (absent, not nil, not an empty array).
+      #
+      # The `field_type_name` value is the documented field-type dispatch
+      # identifier — robust to namespace relocations of the field class
+      # because it strips the namespace via `demodulize` before
+      # `underscore`-ing. It is NOT robust to renames of the leaf class
+      # itself: `Field::Select` → `"select"`, but renaming the class to
+      # `Field::Status` would change the dispatch identifier to `"status"`.
+      #
+      # @param entity_type [String] host AR model class name (e.g. "Contact")
+      # @param scope [String, nil] first partition axis
+      # @param parent_scope [String, nil] second partition axis
+      # @return [Hash] versioned snapshot envelope
+      def export_snapshot_schema(entity_type:, scope: nil, parent_scope: nil)
+        fields = TypedEAV::Field::Base
+                 .where(entity_type: entity_type, scope: scope, parent_scope: parent_scope)
+                 .includes(:field_options)
+                 .order(:sort_order)
+                 .map { |field| export_snapshot_field_entry(field) }
+
+        {
+          "snapshot_schema_version" => 1,
+          "fields" => fields,
+        }
+      end
+
       def import_schema(hash, on_conflict: :error)
         validate_schema_version!(hash)
         validate_conflict_policy!(on_conflict)
@@ -80,6 +137,35 @@ module TypedEAV
       end
       # rubocop:enable Metrics/AbcSize
 
+      # Lean per-field projection used by {.export_snapshot_schema}. Mirrors
+      # the option-row ordering rule from {.export_field_entry} — sort
+      # loaded-association rows by `[sort_order || 0, label, id]`, and
+      # delegate to the `field_options.sorted` scope on the unloaded path.
+      def export_snapshot_field_entry(field)
+        entry = {
+          "name" => field.name,
+          "field_type_name" => field.field_type_name,
+          "required" => field.required,
+          "sort_order" => field.sort_order,
+          "options" => field.options,
+        }
+
+        if field.optionable?
+          options_rows = if field.field_options.loaded?
+                           field.field_options.sort_by do |option|
+                             [option.sort_order || 0, option.label.to_s, option.id]
+                           end
+                         else
+                           field.field_options.sorted
+                         end
+          entry["options_data"] = options_rows.map do |option|
+            { "label" => option.label, "value" => option.value, "sort_order" => option.sort_order }
+          end
+        end
+
+        entry
+      end
+
       def export_section_entry(section)
         {
           "name" => section.name,

data/lib/typed_eav/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module TypedEAV
-  VERSION = "0.3.2"
+  VERSION = "0.4.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: typed_eav
 version: !ruby/object:Gem::Version
-  version: 0.3.2
+  version: 0.4.0
 platform: ruby
 authors:
 - dchuk