typed_eav 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 37b913418c998e4620c90efb72a0be8c93b838a2977db0347c9198b743ced2f3
-  data.tar.gz: 0a8a7f3a622712cc29b5738b8b0f4a54e3df3ffc56657277a69ebb956d3c46bf
+  metadata.gz: 026326e328ecf153d7ba72eeac001470ba67307ed17bf5cef68f52548b6abd47
+  data.tar.gz: 6a76dce02983e81312d67662de6837804b51e691ab0d79dbfb13cacc2b25c19c
 SHA512:
-  metadata.gz: c5bdc06300d8bb92538c93cfe23ac3afb6bed9719a29f6f7590740e6d4f56ac14998e21b4230c58fa2eed10e89dfe053af86144f3cbcf9637049c3d5b811d90b
-  data.tar.gz: 4760251616a62f91b7ed931dafc8b92a25653047cab8fcbd21dd027fe6353fdd2e7209a0629233306c69f77ec8eec08459854e8cc81443595d0b68e0503f4e8b
+  metadata.gz: 1677489ecff776e1914305db8d9681a6cb52891282c6602b3d6b12acb99b89d2263bca877606e686e64d260ecff1f5170ac0f6a733b8f7f5592447d5f8852b90
+  data.tar.gz: 528cbe73ee76daa4e2bb50806b1e148483a68ac9563e2b3c49a3110ee153fba00bd7a6f79612824dc018fc7d13bfcde7e81ed1b0daa67190b401320ee9ea1e5f

data/CHANGELOG.md

@@ -7,7 +7,97 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-No unreleased changes.
+## [0.8.0] - 2026-09-04
+
+New SQL-backed query APIs, selective/preloaded bulk reads, logical dirty
+tracking, and read-only schema previews. Existing read defaults are preserved;
+no database migrations or new runtime dependencies are required.
+
+### Added
+
+- Read-only `SchemaPortability.preview_schema` compares exact-partition
+  portable schemas with current definitions, exposing field/section/option
+  differences, conditional import actions, and risk flags. Type swaps remain
+  blocked and omitted definitions are not treated as deletions. Previewing
+  never applies changes or guarantees a later import's success. (#51)
+- `saved_typed_eav_changes` exposes the last successful host save's logical
+  changes inside `after_save` callbacks. Failed saves preserve the prior
+  snapshot; no-op saves, reload, and rollback have explicit lifecycle
+  semantics independent of audit/versioning. (#48)
+- Database-backed `aggregate_typed_eav` min/max/sum for Integer and Decimal
+  field families, including Percentage. Decimal precision, caller host sets,
+  and explicit empty/NULL semantics are preserved; references and multi-cell
+  Currency are rejected instead of silently aggregated. (#50)
+- SQL-backed distinct typed values, exact distinct counts, and bounded
+  per-value host counts. Caller host filters/pagination and field-definition
+  precedence are retained; explicit NULL, false, and empty strings have
+  documented semantics without host or Value hydration. (#49)
+- Explicit `source: :preloaded` bulk reads reuse caller-loaded Value graphs,
+  including unsaved edits, with one batched definition lookup and no duplicate
+  Value hydration. Incomplete required preloads fail clearly; fresh database
+  reads remain the default. (#46)
+- Pending logical typed-value changes via `typed_eav_changes`, covering
+  host-associated named/nested edits and marked removals without loading
+  untouched Value collections. Failed saves retain validation feedback and
+  pending changes; this API is separate from persisted audit history. (#47)
+- Database-backed `order_typed_eav` for scalar fields, with explicit NULL
+  placement and stable primary-key tie-breaking. Caller host filters and
+  partition-definition precedence are preserved without Value hydration. (#44)
+- Field-selective bulk reads with `typed_eav_hash_for(records, fields: names)`.
+  Selected winning field IDs narrow value loading before hydration; omitted
+  `fields:` preserves all-fields reads. Empty, duplicate, unknown, and
+  partition-specific names have explicit projection semantics. (#45)
+
+### Compatibility and usage notes
+
+- Typed sorting and summaries select one effective field definition. Scope
+  arguments do not authorize or filter host records: retain tenant and access
+  filters on the caller's Active Record relation. All-partitions mode and
+  unsupported collection/multi-cell operations fail explicitly.
+- Distinct lists and grouped counts default to 100 values (maximum 1,000),
+  ordered by value rather than frequency. Exact distinct counts include the
+  explicit NULL category; missing rows are omitted. Numeric min/max/sum retain
+  Integer/BigDecimal types and do not aggregate reference IDs or currencies.
+- Preloaded reads are explicit in-memory value snapshots with a current
+  definition lookup; database reads remain the default. Dirty/saved changes
+  describe host-associated editing and save state, not durable audit history
+  or independently executed bulk SQL writes.
+- Schema previews are advisory, exact-partition comparisons. They neither
+  apply changes nor reserve the schema or guarantee a later import's success.
+- Local verification: 1,404 examples, 0 failures; 160 Ruby files lint-clean.
+  The release workflow additionally verifies the supported compatibility
+  matrix and the exact packaged artifact before publication.
+
+## [0.7.1] - 2026-09-04
+
+A focused patch release for inherited Active Record hosts and multi-partition
+bulk upserts. No new database migrations or public API changes are required.
+
+### Fixed
+
+- Resolve inherited and namespaced Active Record host definitions, filters,
+  bulk reads, writes, and registry/versioning opt-ins through Rails'
+  canonical polymorphic name. STI subclasses now consistently share their
+  base class's EAV schema and stored values while queries retain their host
+  class restriction. Partition isolation and most-specific field precedence
+  are preserved. (#42)
+
+### Performance
+
+- Batch BulkUpsert field-definition resolution into one SELECT per transaction
+  unit while preserving exact tuple isolation and definition precedence. A
+  20-partition regression case now issues one definition SELECT instead of 20;
+  chunked transactions intentionally issue one per chunk. BulkRead and
+  BulkUpsert share the internal batched resolver. This is a query-count
+  improvement, not a universal throughput claim. (#43)
+
+### Compatibility and verification
+
+- BulkUpsert's reduced-semantics acknowledgement, validation, transaction
+  boundaries, and callback/versioning behavior are unchanged.
+- Regression coverage includes true STI and namespaced hosts, public
+  read/filter/write paths, partition isolation, definition precedence, and
+  all/chunk transaction behavior.
 
 ## [0.7.0] - 2026-08-18
 
@@ -580,7 +670,9 @@ worked examples.
 
 Initial release.
 
-[Unreleased]: https://github.com/dchuk/typed_eav/compare/v0.7.0...HEAD
+[Unreleased]: https://github.com/dchuk/typed_eav/compare/v0.8.0...HEAD
+[0.8.0]: https://github.com/dchuk/typed_eav/releases/tag/v0.8.0
+[0.7.1]: https://github.com/dchuk/typed_eav/releases/tag/v0.7.1
 [0.7.0]: https://github.com/dchuk/typed_eav/releases/tag/v0.7.0
 [0.6.0]: https://github.com/dchuk/typed_eav/releases/tag/v0.6.0
 [0.5.0]: https://github.com/dchuk/typed_eav/releases/tag/v0.5.0

data/README.md

@@ -119,6 +119,11 @@ tags.field_options.create!([
 ])
 ```
 
+When deriving `entity_type` from a model class, use
+`Contact.polymorphic_name`. Rails stores polymorphic associations under that
+canonical name, which is the base-class type for STI hosts and respects the
+application's namespaced-polymorphism setting.
+
 ### 3. Set values on records
 
 ```ruby
@@ -192,6 +197,78 @@ Contact.where(company_id: 42)
        .limit(25)
 ```
 
+### Sorting by a typed field
+
+```ruby
+Contact.where(tenant_id: "t1")
+       .order_typed_eav("age", direction: :desc, nulls: :last, scope: "t1")
+       .limit(25)
+```
+
+`order_typed_eav` returns an Active Record relation and orders in PostgreSQL,
+without loading typed values into Ruby. It replaces prior ordering while
+preserving host filters, STI restrictions, limits, and offsets. `direction:`
+accepts `:asc` (default) or `:desc`; `nulls:` accepts `:first` or `:last`
+(default in either direction). Missing rows and explicit NULLs share that
+placement. Equal values use the host primary key ascending as a stable tie-break.
+
+Scope arguments (or the ambient scope) select the winning field definition;
+**they do not filter host records by tenant**. Keep authorization/tenant filters
+on the caller relation. All-partitions `TypedEAV.unscoped` is rejected for this
+API: choose one effective definition instead. Single native scalar cells are
+supported, using their stored values (for example, reference IDs and attachment
+signed IDs, not display labels). JSON/array and multi-cell fields such as
+Currency are rejected rather than assigned an implicit ordering.
+
+### Distinct values and grouped counts
+
+```ruby
+contacts = Contact.where(tenant_id: "t1")
+contacts.distinct_typed_eav_values("status", scope: "t1", limit: 100)
+# => ["active", "paused", nil]
+contacts.typed_eav_value_counts("status", scope: "t1", limit: 100)
+# => {"active" => 24, "paused" => 3, nil => 2}
+contacts.count_distinct_typed_eav_values("status", scope: "t1")
+# => 3
+```
+
+These scalar queries run in SQL without hydrating hosts or Values. Results
+use native ascending value order, with explicit NULL (`nil`) last. Missing
+value rows contribute nothing; `false` and empty strings remain real values.
+Grouped counts count host identities, not duplicate rows introduced by joins.
+The caller's filters, STI restrictions, distinctness, and pagination determine
+the host set before summarization. Scope arguments choose field-definition
+visibility, not host authorization, just as with typed sorting.
+
+Lists and grouped-count hashes default to 100 values and accept a positive
+Integer `limit:` up to 1,000. They are truncated in value order, not ranked by
+frequency. Compare their length with `count_distinct_typed_eav_values` to
+detect truncation; that exact SQL count includes one NULL category and returns
+only an Integer, regardless of cardinality. It still requires database work
+over the matching set. Collection/multi-cell fields and all-partitions mode
+are unsupported, matching scalar sorting.
+
+### Numeric aggregates
+
+```ruby
+Contact.where(tenant_id: "t1").aggregate_typed_eav(
+  "score", operation: :sum, scope: "t1"
+)
+```
+
+`aggregate_typed_eav` requires `operation: :min`, `:max`, or `:sum` and returns
+one SQL-calculated scalar over the caller's host set. Integer fields return
+Integers; Decimal and Percentage fields preserve `BigDecimal` precision, with
+no Float conversion. Percentage values remain stored fractions, not formatted
+percent strings. Missing rows and explicit NULLs are ignored. With no non-NULL
+values, min/max return `nil` and sum returns the field's typed zero.
+
+The same host filtering, partition visibility, STI, and pagination rules as
+distinct queries apply. Only Integer/Decimal families (including Percentage)
+with a single numeric cell are supported. Reference IDs, text, collections,
+and multi-cell Currency are rejected; the gem does not silently sum identifiers
+or combine currencies. No values or host records are loaded to compute the result.
+
 ### Available Operators
 
 | Operator | Works On | Description |
@@ -1077,6 +1154,55 @@ Production code rarely calls either — they exist for test isolation and
 for the rare case where a host app wants to fully unwire the gem in a
 specific request lifecycle.
 
+## In-memory typed-value changes
+
+```ruby
+contact.set_typed_eav_value("age", 41)
+contact.typed_eav_changes # => {"age" => [40, 41]}
+contact.save!
+contact.typed_eav_changes # => {}
+```
+
+`typed_eav_changes` reports logical `[before, after]` pairs for pending changes
+on this host's in-memory `typed_values` target. It covers named setters,
+`typed_eav_attributes=`, nested `typed_values_attributes=`, association builds,
+edits to target Values, and `mark_for_destruction`/nested `_destroy`. Multi-cell
+fields such as Currency retain their logical shape; returned hashes, pairs,
+and mutable values are copies. Same-value assignments, reversions, and logical
+`nil`-to-`nil` changes are omitted, including creating/removing a NULL value row.
+Invalid input reports the cast logical result without discarding validation errors.
+
+Failed saves retain pending state; successful saves and reload clear it. An
+outer rollback follows Active Record's restored child dirty state. The API
+resolves effective field names using this record's partition precedence, and
+does not load all persisted Values merely to inspect an untouched host.
+
+After saving, `saved_typed_eav_changes` exposes the most recent successful
+host save's logical pairs:
+
+```ruby
+contact.set_typed_eav_value("age", 42)
+contact.save!
+contact.saved_typed_eav_changes # => {"age" => [41, 42]}
+contact.typed_eav_changes       # => {}
+```
+
+Saved changes are available in normal host `after_save` callbacks, including
+values assigned by `before_save`. Each successful save replaces the snapshot;
+a no-op save replaces it with `{}`. Failed validation or a save-callback
+exception preserves the previous successful snapshot. Reload and an outer
+transaction rollback clear saved changes. These are successful-save semantics,
+not proof of a durable commit: use `after_commit` when external effects must
+wait for commit. Exceptions after a transaction has already committed cannot
+undo persisted data. Neither dirty API requires versioning or adds audit rows.
+
+This is in-memory editing state, not audit history. Independently loaded/saved
+Values, reassignment of an existing Value's field identity, SQL/`delete_all`,
+and collection operations that immediately remove rows from the host target
+are not tracked. Use nested destruction or `mark_for_destruction` for tracked
+removal. Reduced `BulkUpsert` does not update unrelated in-memory host objects;
+reload them after external writes. Bulk reads do not create dirty state.
+
 ## Versioning
 
 `typed_eav` ships an opt-in append-only audit log for changes to typed
@@ -1157,7 +1283,7 @@ directly:
 
 ```ruby
 TypedEAV::ValueVersion
-  .where(entity_type: contact.class.name, entity_id: contact.id, field_id: age_field.id)
+  .where(entity_type: contact.class.polymorphic_name, entity_id: contact.id, field_id: age_field.id)
   .order(changed_at: :desc, id: :desc)
 # => [<ValueVersion change_type: "destroy" before: {"integer_value" => 42} after: {} value_id: nil>,
 #     <ValueVersion change_type: "update"  before: {"integer_value" => 41} after: {"integer_value" => 42} value_id: nil>,
@@ -1176,7 +1302,7 @@ exports) — drop the `field_id` filter:
 
 ```ruby
 TypedEAV::ValueVersion
-  .where(entity_type: contact.class.name, entity_id: contact.id)
+  .where(entity_type: contact.class.polymorphic_name, entity_id: contact.id)
   .order(changed_at: :desc, id: :desc)
 # => all version rows for every typed field on this contact, most-recent-first.
 # Includes :create, :update, and :destroy events across every field the
@@ -1362,9 +1488,50 @@ The gem creates five tables:
   create, update, and destroy events; it retains durable entity identity even
   when the live Value row is later removed
 
+## Read-only schema previews
+
+```ruby
+schema = TypedEAV::SchemaPortability.export_schema(
+  entity_type: "Contact", scope: "t1"
+)
+schema["fields"].first["required"] = true
+preview = TypedEAV::SchemaPortability.preview_schema(schema, on_conflict: :overwrite)
+preview["fields"].first["changes"]
+# => {"required" => {"from" => false, "to" => true}}
+preview["risks"] # => ["required_false_to_true"]
+```
+
+The preview compares a version-1 portable export with the database's exact
+target partition. It requires the envelope's `entity_type`, `scope`, and
+`parent_scope` to match every entry; mixed-target payloads and duplicate
+identities are rejected. This intentionally stricter preview input does not
+change `import_schema` or silently retarget definitions.
+
+The plain Hash result contains envelope metadata, `summary`, `fields`,
+`sections`, `risks`, and `importable`. Each entry includes its exact `identity`,
+`status` (`unchanged`, `added`, `changed`, or `conflict`), conditional `action`,
+and attribute `changes` with `from`/`to` values. Field entries also contain
+option-row `added`, `removed`, and `changed` lists, matched by option value.
+Raw option ordering and key-presence differences remain visible because the
+importer compares the complete exported payload, not just equivalent settings.
+
+`on_conflict: :error` blocks divergent definitions; `:skip` leaves them alone;
+`:overwrite` predicts an update. Type swaps always produce an error action,
+even under skip/overwrite. Risks flag type changes, removed options, newly
+required fields, and changes to options, defaults, or field dependencies.
+Omitted target definitions are **not deletions** and are not listed as such.
+
+`importable: true` means no known conflict-policy/type-swap blocker was found,
+not that validation or a later import is guaranteed to succeed. Actions are
+conditional predictions: a blocking error aborts the existing transactional
+import, including otherwise acceptable additions. Previewing does not save
+definitions, run mutation/validation callbacks, enqueue jobs, execute DDL, or
+convert values. It is an advisory snapshot, not a lock or reservation; model
+validations and concurrent changes still apply to the actual import.
+
 ## Architecture
 
-Internal module layout as of 0.7.0. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by ADR-0001 through ADR-0013.
+Current internal module layout. Most consumers never reach for these directly — the public surface is the `has_typed_eav` macro and the instance/class methods it installs — but the split matters if you're extending the gem, debugging an integration, or evaluating it for production. Decisions are anchored by ADR-0001 through ADR-0013.
 
 ### Macro entry: `HasTypedEav`
 
@@ -1405,6 +1572,11 @@ TypedEAV::QueryBuilder                ← low altitude: per-field SQL primitive
 
 `QueryBuilder` is the single place that decides "given this field and this operator, which column and which SQL fragment?" `FilterQuery` never builds SQL fragments directly; `EntityQuery` never touches columns. Splitting the two altitudes keeps custom field types extending only the column-mapping surface (`value_column`, `operators`, `operator_column`) without ever subclassing `FilterQuery`.
 
+Scalar ordering and summaries are a separate `EntityQuery` delegation to
+`ScalarQuery`: it resolves one winning definition, checks scalar support, and
+builds SQL over the field's declared native column. It does not add operators
+to the filter DSL or load the host/Value graph to calculate summaries.
+
 ### Bulk reads: `BulkRead`
 
 `typed_eav_hash_for(records)` (the plural read) routes through `TypedEAV::BulkRead`. Given a record collection and an effective `(scope, parent_scope)`, it:
@@ -1416,6 +1588,10 @@ TypedEAV::QueryBuilder                ← low altitude: per-field SQL primitive
 3. Returns a `{record_id => {field_name => value}}` map while skipping orphaned
    values and preserving logical missingness.
 
+Definitions, filters, reads, registry entries, and writes all use the host's
+Rails `polymorphic_name`, so an STI leaf class reads and queries the same rows
+written under its base-class polymorphic type.
+
 The final production characterization reduced the 1,002 SQL statements observed
 across 1,000 scopes to three for the same BulkRead shape. This is a statement-
 count result, not a representative throughput claim; applications should still
@@ -1423,6 +1599,44 @@ measure their own scope cardinality, selected fields, hydration, and contention.
 
 Single-record reads (`typed_eav_value`, `typed_eav_hash`) live on `InstanceMethods` and use the same partition helpers but without batching.
 
+Use `fields:` to load only the values needed by a view or export:
+
+```ruby
+Contact.typed_eav_hash_for(contacts, fields: [:name, :score])
+# => {123 => {"name" => "Ada", "score" => 42}, ...}
+```
+
+Omitting `fields:` (or passing `nil`) retains the all-fields behavior. A single
+String/Symbol or an enumerable of names is accepted; duplicates are removed.
+Unknown names and names unavailable in an individual record's partition are
+omitted, as are missing value rows. An explicitly stored NULL remains `nil`.
+`fields: []` returns an empty inner hash for each record without definition or
+value queries (the supplied collection itself may still need loading). Selected
+winning field IDs constrain the value query before hydration, so unrequested
+values and their field readers are not loaded or evaluated. Definition lookup
+remains batched across the records' partitions.
+
+To explicitly reuse already-loaded values:
+
+```ruby
+contacts = Contact.where(tenant_id: "t1").includes(typed_values: :field).to_a
+Contact.typed_eav_hash_for(contacts, fields: [:name], source: :preloaded)
+```
+
+The default `source: :database` still fetches persisted values afresh, even
+when associations are loaded or edited in memory. `:preloaded` uses the caller's
+association targets, including unsaved Value builds/assignments, without saving
+or mutating them. It performs one fresh batched definition query to choose
+current winners, but no Value or field-association preload queries. This is a
+value snapshot, not a guarantee of current database contents or frozen schema.
+
+Every host's `typed_values` association must be loaded, as must each retained
+Value's `field` association; incomplete preloads raise `ArgumentError` instead
+of silently issuing N+1 queries. With `fields:`, unselected Values need no field
+preload. With all fields selected, all field associations must be loaded
+(including loaded `nil` for orphans). `fields: []` needs neither associations
+nor definition/value queries. Only `:database` and `:preloaded` are valid sources.
+
 ### Bulk writes: `BulkWrite`
 
 `bulk_set_typed_eav_values(records, attrs)` routes through `TypedEAV::BulkWrite`,
@@ -1459,6 +1673,12 @@ checks, and Value validation callbacks remain; host callbacks and validations,
 Value persistence callbacks, versioning, delete shorthand, and per-record
 savepoint isolation are skipped.
 
+Within each `transaction: :all` unit—or each requested chunk—the upsert path
+resolves every record partition through one batched field-definition SELECT.
+It shares BulkRead's internal tuple resolver, retaining global, scope-only, and
+full-tuple precedence independently for each record without broadening tenant
+visibility.
+
 `BulkWrite` and `BulkRead` are siblings — one read path, one write path — but they don't share a base class. Per [ADR-0005](docs/adr/0005-keep-phase-six-modules-independent.md), keeping them independent preserves the option to evolve each on its own schedule.
 
 ### Per-record reads/writes: `InstanceMethods`