typed_eav 0.8.0 → 0.8.1

data/docs/guides/fields.md

@@ -0,0 +1,342 @@
+---
+title: "Field types and validation"
+---
+
+# Field types and validation
+
+[Documentation home](../index.md)
+
+## Field Types
+
+| Type | Column | Ruby Type | Options |
+|------|--------|-----------|---------|
+| `Text` | `string_value` | String | `min_length`, `max_length`, `pattern` |
+| `LongText` | `text_value` | String | `min_length`, `max_length` |
+| `Integer` | `integer_value` | Integer | `min`, `max` |
+| `Decimal` | `decimal_value` | BigDecimal | `min`, `max`, `precision_scale` |
+| `Boolean` | `boolean_value` | Boolean | |
+| `Date` | `date_value` | Date | `min_date`, `max_date` |
+| `DateTime` | `datetime_value` | Time | `min_datetime`, `max_datetime` |
+| `Select` | `string_value` | String | options via `TypedEAV::Option` |
+| `MultiSelect` | `json_value` | Array | options via `TypedEAV::Option` |
+| `IntegerArray` | `json_value` | Array | `min_size`, `max_size`, `min`, `max` |
+| `DecimalArray` | `json_value` | Array | `min_size`, `max_size` |
+| `TextArray` | `json_value` | Array | `min_size`, `max_size` |
+| `DateArray` | `json_value` | Array | `min_size`, `max_size` |
+| `Email` | `string_value` | String | auto-downcases, strips whitespace |
+| `Url` | `string_value` | String | strips whitespace |
+| `Color` | `string_value` | String | hex color values |
+| `Json` | `json_value` | Hash/Array | arbitrary JSON |
+| `Currency` | `decimal_value` + `string_value` | `{amount: BigDecimal, currency: String}` | `default_currency`, `allowed_currencies` |
+| `Percentage` | `decimal_value` | BigDecimal (0..1 range) | `decimal_places`, `display_as: :fraction \| :percent` |
+| `Image` | `string_value` (signed_id) + `:attachment` has_one_attached | String (Active Storage signed_id) | `allowed_content_types`, `max_size_bytes` |
+| `File` | `string_value` (signed_id) + `:attachment` has_one_attached | String (Active Storage signed_id) | `allowed_content_types`, `max_size_bytes` |
+| `Reference` | `integer_value` (FK) | Integer (target record ID) | `target_entity_type`, `target_scope` |
+
+## Sections (Optional UI Grouping)
+
+```ruby
+general = TypedEAV::Section.create!(
+  name: "General Info",
+  code: "general",
+  entity_type: "Contact",
+  sort_order: 1
+)
+
+social = TypedEAV::Section.create!(
+  name: "Social Media",
+  code: "social",
+  entity_type: "Contact",
+  sort_order: 2
+)
+
+TypedEAV::Field::Text.create!(
+  name: "twitter_handle",
+  entity_type: "Contact",
+  section: social
+)
+```
+
+## Custom Field Types
+
+Override `cast(raw)` to return a `[casted_value, invalid?]` tuple.
+`invalid?` tells `Value#validate_value` whether to surface `:invalid`
+(vs `:blank`) when raw input can't be coerced. For types that never
+fail to coerce, always return `[value, false]`.
+
+```ruby
+# app/models/fields/phone.rb
+module Fields
+  class Phone < TypedEAV::Field::Base
+    value_column :string_value
+    operators :eq, :contains, :starts_with, :is_null, :is_not_null
+
+    def cast(raw)
+      # Strip everything but digits and +; never rejects as invalid
+      [raw&.to_s&.gsub(/[^\d+]/, ""), false]
+    end
+  end
+end
+
+# Register it
+TypedEAV.configure do |c|
+  c.register_field_type :phone, "Fields::Phone"
+end
+```
+
+### Family intermediate bases (extension points)
+
+`Field::Base` is the universal parent, but three intermediate family
+bases collapse the most common per-leaf duplication. Pick the right
+parent and you inherit the family's validation surface for free.
+
+- **`TypedEAV::Field::ValidatedString`** — subclass when your custom
+  type stores in `string_value` and wants a min/max-length + regex-pattern
+  validation surface. Inherits `value_column :string_value`,
+  `store_accessor :options, :min_length, :max_length, :pattern`,
+  numericality validators on `min_length` / `max_length`, a
+  `max_gte_min_length` guard that rejects inverted bounds at field-save,
+  and a `validate_pattern_syntax` guard that rejects bad regexes at
+  field-save. The default `validate_typed_value(record, val)` runs
+  `validate_length` plus `validate_pattern if pattern.present?`. Override
+  it and call `super` to layer on a format-specific check (the built-in
+  `Field::Email` / `Field::Url` are the canonical pattern).
+
+  ```ruby
+  class Fields::Slug < TypedEAV::Field::ValidatedString
+    SLUG_FORMAT = /\A[a-z0-9-]+\z/
+
+    def cast(raw)
+      [raw&.to_s&.strip&.downcase, false]
+    end
+
+    def validate_typed_value(record, val)
+      super  # length + pattern from the family base
+      record.errors.add(:value, "is not a valid slug") unless SLUG_FORMAT.match?(val.to_s)
+    end
+  end
+  ```
+
+- **`TypedEAV::Field::RangeBounded`** — subclass when your custom type
+  stores a single comparable value (numeric or temporal) constrained by
+  a min/max bound. Each leaf still declares its own `value_column` and
+  its own `store_accessor` (key names vary by family member: `:min`/`:max`
+  for numeric; `:min_date`/`:max_date` for date;
+  `:min_datetime`/`:max_datetime` for datetime). The family base
+  provides protected `validate_range` / `validate_date_range` /
+  `validate_datetime_range` helpers. Each leaf should pair its
+  `store_accessor` with the macro
+  `validates :max, comparison: { greater_than_or_equal_to: :min }, allow_nil: true, if: :min`
+  (or the analogous form for the leaf's key names) so inverted bounds
+  fail at field-save.
+
+  ```ruby
+  class Fields::Score < TypedEAV::Field::RangeBounded
+    value_column :integer_value
+
+    store_accessor :options, :min, :max
+    validates :max, comparison: { greater_than_or_equal_to: :min }, allow_nil: true, if: :min
+
+    def cast(raw)
+      raw.nil? ? [nil, false] : [Integer(raw.to_s, exception: false), raw.to_s.empty? ? false : true]
+    end
+
+    def validate_typed_value(record, val)
+      validate_range(record, val)
+    end
+  end
+  ```
+
+- **`TypedEAV::Field::Optionable`** — `include` this concern when your
+  custom type's valid values are drawn from a `Field::Option` set.
+  Provides `optionable? = true`, a public-facing sorted
+  `allowed_values` helper, and protected
+  `validate_option_inclusion` / `validate_multi_option_inclusion`
+  helpers. Mixin (not inheritance) because option-set field types may
+  use different `value_column`s — the built-in `Field::Select` stores in
+  `string_value` while `Field::MultiSelect` stores in `json_value`, and
+  both stay as direct children of `Field::Base`.
+
+  ```ruby
+  class Fields::Tag < TypedEAV::Field::Base
+    include TypedEAV::Field::Optionable
+
+    value_column :string_value
+    operators :eq, :not_eq, :is_null, :is_not_null
+
+    def cast(raw)
+      [raw&.to_s, false]
+    end
+
+    def validate_typed_value(record, val)
+      validate_option_inclusion(record, val)
+    end
+  end
+  ```
+
+The rule of thumb: subclass an intermediate family base when the new
+field type shares its storage and validation surface with the family;
+include `Optionable` when it draws values from an option set; subclass
+`Field::Base` directly (as the `Phone` example above does) when none of
+the family surfaces fit. `validate_array_size` lives on `Field::Base`
+itself — its callers span unrelated families.
+
+### Multi-cell field types
+
+External field types may store their logical value across multiple typed
+columns. The entire storage surface lives directly on `Field::Base` via
+the `Field::TypedStorage` concern, so a custom multi-cell type is just a
+`Field::Base` subclass that overrides three instance methods.
+
+**Class-level DSL** (declared at class load time):
+
+- `value_column :col` – single-cell sugar; declares the primary cell.
+- `value_columns :a, :b, ...` – plural form for multi-cell types. The
+  primary cell is `value_columns.first`. Both forms share storage;
+  `value_column` and `value_columns` are interchangeable getters/setters.
+- `operators :eq, :gt, ...` – restrict the supported operator set.
+- `self.operator_column(op)` – override to route different operators to
+  different cells. Defaults to `value_columns.first`.
+
+**Override-point instance methods** (the entire extension surface for
+multi-cell types):
+
+- `read_value(record)` – compose the logical value from the cells.
+- `write_value(record, casted)` – unpack the casted value across cells.
+- `apply_default(record)` – populate cells from `default_value`.
+
+The defaults target `value_columns.first`, so single-cell field types
+keep working without overrides. The three methods are paired – override
+all three or your reads will see a multi-cell shape that writes / defaults
+cannot produce.
+
+**Concrete snapshot helpers** (NOT overridable; derived from
+`value_columns`):
+
+- `value_changed?(record)` – true iff any cell saw a saved change.
+- `before_snapshot(record, change_type)` / `after_snapshot(record, change_type)`
+  – per-cell hashes keyed by string column names; powers the versioning
+  jsonb shape.
+
+Custom multi-cell type example (matches the built-in `Field::Currency`):
+
+```ruby
+class Fields::Money < TypedEAV::Field::Base
+  AMOUNT_COLUMN = :decimal_value
+  CURRENCY_COLUMN = :string_value
+
+  value_columns AMOUNT_COLUMN, CURRENCY_COLUMN
+  operators :eq, :gt, :lt, :gteq, :lteq, :between, :currency_eq, :is_null, :is_not_null
+
+  def self.operator_column(operator)
+    operator == :currency_eq ? CURRENCY_COLUMN : AMOUNT_COLUMN
+  end
+
+  def read_value(value_record)
+    amount = value_record[AMOUNT_COLUMN]
+    currency = value_record[CURRENCY_COLUMN]
+    return nil if amount.nil? && currency.nil?
+
+    { amount: amount, currency: currency }
+  end
+
+  def write_value(value_record, casted)
+    if casted.nil?
+      value_record[AMOUNT_COLUMN] = nil
+      value_record[CURRENCY_COLUMN] = nil
+    else
+      value_record[AMOUNT_COLUMN] = casted[:amount]
+      value_record[CURRENCY_COLUMN] = casted[:currency]
+    end
+  end
+
+  def apply_default(value_record)
+    default = default_value
+    return unless default.is_a?(Hash)
+
+    value_record[AMOUNT_COLUMN] = default[:amount] || default["amount"]
+    value_record[CURRENCY_COLUMN] = default[:currency] || default["currency"]
+  end
+end
+```
+
+The built-in `Field::Currency` is the canonical multi-cell consumer of
+these extension points and reads as a normal `Field::Base` subclass with
+exactly three method overrides.
+
+### Built-in field types
+
+- **`Currency`:** Stores `{amount: BigDecimal, currency: String}` across two typed columns (`decimal_value` for the amount; `string_value` for the ISO 4217 currency code). Multi-cell storage is declared via `value_columns :decimal_value, :string_value`; reads, writes, and default application override `read_value`, `write_value`, and `apply_default` directly on `Field::Currency`. Operators: `:eq`, `:gt`, `:lt`, `:gteq`, `:lteq`, `:between` target the amount; `:currency_eq` targets the currency code; `:is_null` / `:is_not_null` target the amount column (a Currency value is null when its amount is null). Cast input MUST be a hash with `:amount` and/or `:currency` keys — bare numeric/string values are rejected with `:invalid` to enforce explicit currency dimension at write time. Options: `default_currency` (String ISO code, applied as fallback only when an amount is given without an explicit currency), `allowed_currencies` (Array of ISO codes; `validate_typed_value` enforces inclusion). Versioning snapshots automatically capture both columns because the snapshot helpers iterate `value_columns`. The `:currency_eq` operator is registered ONLY on `Field::Currency`; the QueryBuilder operator-validation gate rejects it with a clear `ArgumentError` if invoked on any other field type.
+
+  ```ruby
+  Contact.where_typed_eav(name: "price", op: :currency_eq, value: "USD")
+  Contact.where_typed_eav(name: "price", op: :between,     value: [50, 150])
+  ```
+
+- **`Percentage`:** A `Field::Decimal` subclass storing the underlying fraction in 0..1 (inclusive). The `:percent` representation is a format-time concern — call `field.format(value)` with `display_as: :percent` to render `0.75` as `"75.0%"`. Options: `decimal_places` (Integer >= 0, default 2; format-time precision only — does NOT alter what's stored in `decimal_value`), `display_as` (`:fraction` default, or `:percent`). Validation: out-of-range values (e.g., `1.5`) fail with the message `"must be between 0.0 and 1.0"`. Storage and operator semantics inherit from `Field::Decimal`.
+
+  ```ruby
+  pf = TypedEAV::Field::Percentage.create!(
+    name: "discount", entity_type: "Order", scope: tenant_id,
+    options: { display_as: :percent, decimal_places: 1 },
+  )
+  pf.format(BigDecimal("0.755")) # => "75.5%"
+  ```
+
+- **`Image`:** Active Storage-backed field type. Stores the attached blob's `signed_id` (a String) in `string_value`. Operators: `:eq`, `:is_null`, `:is_not_null`. Options: `allowed_content_types` (Array of strings; supports exact matches like `"image/png"` and `image/*` family wildcards), `max_size_bytes` (Integer; nil disables the cap). The single `:attachment` has_one_attached association is declared on `TypedEAV::Value` at engine boot when Active Storage is loaded; otherwise `Field::Image#cast` raises `NotImplementedError` with an actionable install message. The `:attachment` association is shared with `Field::File` — Image vs File is a class-identity distinction (used by the `on_image_attached` hook), not a separate association.
+
+  ```ruby
+  field = TypedEAV::Field::Image.create!(
+    name: "avatar", entity_type: "Contact",
+    options: { allowed_content_types: %w[image/png image/jpeg image/webp], max_size_bytes: 5_000_000 },
+  )
+  value = TypedEAV::Value.create!(entity: contact, field: field)
+  value.attachment.attach(io: file_io, filename: "avatar.png", content_type: "image/png")
+  value.update!(string_value: value.attachment.blob.signed_id)
+  value.value # => the signed_id String
+  ```
+
+- **`File`:** Same shape as `Field::Image` but without image-specific semantics. Stores `signed_id` in `string_value`; same operator set; same options (`allowed_content_types`, `max_size_bytes`). The Image vs File distinction is by `value.field.class` at runtime — apps that want strict image-only validation set `allowed_content_types: ["image/*"]` on `Field::Image`; `Field::File` is a general-purpose attachment slot.
+
+- **Active Storage dependency:** Lazy soft-detect via `defined?(::ActiveStorage::Blob)`. The gem does NOT add Active Storage as a hard dependency — apps that never use Image/File never need to install it. To use Image or File fields, add `gem "activestorage"` to your Gemfile (included in supported Rails versions via the `rails` meta-gem) and run `bin/rails active_storage:install` to create the `active_storage_blobs` / `active_storage_attachments` / `active_storage_variant_records` tables. The mirror precedent is `acts_as_tenant`, which is also soft-detected (see `Config::DEFAULT_SCOPE_RESOLVER`).
+
+- **`on_image_attached` hook:** Fires from `after_commit` on `TypedEAV::Value` when a `Field::Image`-typed Value's attachment is added or replaced. Receives `(value, blob)`. Configure via `TypedEAV.configure { |c| c.on_image_attached = ->(v, b) { ... } }`. Hook ordering: runs AFTER versioning (Phase 4) and AFTER `on_value_change` (Phase 3) so it sees the persisted version row and the user-callback context. File attachments do NOT fire this hook — the name is image-specific by design. Use `on_value_change` for a generic value-mutation signal that covers File-typed Values too.
+
+  ```ruby
+  TypedEAV.configure do |c|
+    c.on_image_attached = ->(value, blob) {
+      ProcessImageJob.perform_later(value.id, blob.id)
+    }
+  end
+  ```
+
+- **`Reference`:** Foreign-key field type. Stores the target record's integer ID in `integer_value`. Operators: `:eq`, `:is_null`, `:is_not_null`, `:references` (explicit narrowing — does NOT inherit `:integer_value`'s `:gt`/`:lt`/`:between` defaults; arithmetic comparisons on FKs don't carry useful semantics). The `:references` operator accepts AR record instances OR Integer IDs at query time, normalizing via `field.cast` (a class-mismatched record routes to `base.none` rather than `:is_null`). Options: `target_entity_type` (REQUIRED — String class name of the target model, validated to constantize at field save), `target_scope` (OPTIONAL — when set, the field is REJECTED at save time if `target_entity_type` is not registered with `has_typed_eav scope_method:` (Gating Decision 2); when set with a scoped target, value-time validation rejects writes whose target's `typed_eav_scope` does not match `target_scope` via a `target_partition_matches?` helper structurally parallel to Phase 1's `entity_partition_axis_matches?` but on the target axis). Cross-scope safety mirrors the existing `Value#validate_field_scope_matches_entity` guard pattern applied to the target rather than the source.
+
+  ```ruby
+  rf = TypedEAV::Field::Reference.create!(
+    name: "manager", entity_type: "Contact", scope: tenant_id,
+    options: { target_entity_type: "Contact", target_scope: tenant_id },
+  )
+  TypedEAV::Value.create!(entity: alice, field: rf, value: bob)         # accepts AR record
+  TypedEAV::Value.create!(entity: alice, field: rf, value: bob.id)      # accepts Integer FK
+  Contact.where_typed_eav(name: "manager", op: :references, value: bob) # filter by record
+  Contact.where_typed_eav(name: "manager", op: :references, value: 42)  # filter by FK
+  ```
+
+- **Summary:** The built-in field types **Image, File, Reference, Currency, Percentage** all preserve the cast-tuple contract (`[casted, invalid?]`), the operator-dispatch model (`supported_operators` + `operator_column` for multi-cell types), and the no-hardcoded-attribute-references foundational principle. The multi-cell extension surface (`read_value`, `write_value`, `apply_default`, and `operator_column`) is the canonical way to build any future external multi-cell field type.
+
+## Validation Behavior
+
+A few non-obvious contracts worth knowing about up front:
+
+- **Required + blank**: `required: true` fields reject empty strings, whitespace-only strings, and arrays whose every element is nil/blank/whitespace.
+- **Array all-or-nothing cast**: integer/decimal/date arrays mark the **whole** value invalid (stored as `nil`) when any element fails to cast. There is no silent partial — a failed form re-renders with the original input intact so the user can correct the bad element.
+- **`Integer` array rejects fractional input**: `"1.9"` is rejected rather than truncated to `1`. Same rules as the scalar `Integer` field.
+- **`Json` parses string input**: a JSON string posted from a form is parsed; parse failures surface as `:invalid` rather than being stored as the literal string.
+- **`TextArray` does not support `:contains`**: it backs a jsonb column where SQL `LIKE` doesn't apply. Use `:any_eq` for "array contains element".
+- **Orphaned values are skipped**: if a field row is deleted while values remain, `typed_eav_value` and `typed_eav_hash` silently skip the orphans rather than raising.
+- **Cross-scope writes are rejected**: assigning a `Value` to a record whose `typed_eav_scope` doesn't match the field's `scope` adds a validation error on `:field`. The same guard covers the `parent_scope` axis.
+- **Orphan-parent rows rejected**: a `Field` or `Section` row with `parent_scope` set but `scope` blank is invalid. The `Value`-side guard rejects cross-`(scope, parent_scope)` writes too.
+- **Event hooks fire from `after_commit`**: the `on_value_change` and `on_field_change` callbacks fire after the database write is durable; their exceptions never break a save. See [Event hooks](events-and-versioning.md#event-hooks) for the full contract.
+- **Versioning is opt-in**: When enabled (`TypedEAV.config.versioning = true` on the gem; `versioned: true` per host), every `:create` / `:update` / `:destroy` event on a Value writes an append-only audit row in `typed_eav_value_versions`. See [Versioning](events-and-versioning.md#versioning) for the full contract.
+- **`label` is cosmetic, `name` is the machine key**: A field's optional `label` is free-text human display, independent of the slug `name`. Render via `display_name`, which returns `label` when present else `name.humanize`. `label` has no uniqueness or format constraints (only a 255-char max) and never affects ordering, lookup, partitioning, or rename detection — editing only `label` fires `on_field_change` with `:update`, never `:rename`. Existing rows (`label` NULL) render unchanged. Schema export round-trips the raw `label` (legacy payloads without a `label` key import as NULL); snapshot export carries the resolved `display_name`.

data/docs/guides/performance.md

@@ -0,0 +1,107 @@
+---
+title: "Storage and performance"
+---
+
+# Storage and performance
+
+[Documentation home](../index.md)
+
+## Why Typed Columns?
+
+JSONB is a useful fit when an application owns stable paths and wants expression B-tree indexes or GIN containment indexes. A single JSONB document is not inherently faster or slower than TypedEAV; the right choice depends on access patterns, selectivity, update shape, and operational constraints. For example, an application-owned expression index may support a stable path:
+
+```sql
+CAST(value_meta->>'const' AS bigint) = 42
+```
+
+This can work well for stable, known paths. It does not provide the same column-level schema and typed-value contract as TypedEAV, and arbitrary paths still require application-owned index and validation decisions. GIN is useful for containment workloads; expression B-trees are useful for selected stable scalar paths.
+
+TypedEAV stores values in native columns, so queries become:
+
+```sql
+WHERE integer_value = 42
+```
+
+TypedEAV supplies stable typed columns and ordinary per-type indexes. Range scans and sorts can use those indexes, while each Field casts and validates the operand according to its own semantics before the query reaches the typed column. Neither design is a universal storage winner; choose from measured workload fit.
+
+### Optional trigram indexing for string search
+
+TypedEAV keeps its partial-covering `text_pattern_ops` B-tree as the default
+string index. Equality uses that B-tree, while `:starts_with`, `:contains`, and
+`:ends_with` use `ILIKE`; `:not_contains` uses `NOT ILIKE`. The gem does not
+require or install `pg_trgm` and does not create a trigram index automatically.
+
+An application with frequent positive `ILIKE` searches containing at least
+three useful characters may evaluate its own partial GIN index. This is a
+workload decision: the representative benchmark used GIN for measured prefix,
+contains, suffix, and escaped-literal patterns, but not for `NOT ILIKE` or
+one/two-character probes. It does not prove that every positive pattern or
+selectivity will benefit. A `lower(string_value) LIKE ...` expression index is
+not equivalent to TypedEAV's public `ILIKE`, and the benchmark did not justify
+GiST.
+
+Application owners should check extension availability and deploy-role
+privileges in preproduction, then create the extension and index in their own
+migrations. Use nontransactional `CREATE INDEX CONCURRENTLY`, a stable
+application-specific name, and workload-specific `EXPLAIN (ANALYZE, BUFFERS,
+WAL, SETTINGS)` plus storage and write-WAL measurements. Rollback should drop
+only the application-owned index concurrently; do not drop the database-wide
+extension because other objects may share it. See
+[ADR 0009](../adr/0009-string-search-indexing.md) and the
+[benchmark guide](https://github.com/dchuk/typed_eav/blob/main/bench/README.md#phase-3-string-search-benchmark) for the
+operator matrix, measured costs, SQL, and evidence limits.
+
+### Optional planner statistics for correlated field/value predicates
+
+TypedEAV does not install PostgreSQL extended-statistics objects. An application
+whose own plans persistently misestimate `field_id = ... AND typed_value = ...`
+may evaluate application-owned `dependencies` statistics for that exact typed
+column. Dependency statistics apply to compatible equality and `IN` clauses,
+not range predicates. `mcv` describes common value combinations, while
+`ndistinct` primarily informs distinct-group estimates; neither should be added
+without workload evidence.
+
+The representative PostgreSQL 17 benchmark found better aggregate equality
+estimates from dependencies, but no plan-shape or demonstrated runtime benefit.
+Its combined object mirrored MCV on the four changed probes because matching MCV
+groups supplied those estimates. The experiment's target of 100 was a controlled
+input, not a universal recommendation. One probe labeled common-date equality
+actually queried an absent date and returned zero rows; it is not evidence about
+common-date estimates.
+
+Applications should own stable names and DDL, select targets from representative
+data, run `ANALYZE`, and compare estimated/actual rows, plans, runtime, planning
+cost, maintenance cost, and data churn before retaining an object. Coordinate
+ownership in shared databases, inspect catalog definitions before changing
+objects, and drop only application-owned statistics during rollback. See
+[ADR 0010](../adr/0010-planner-statistics-policy.md) and the
+[benchmark guide](https://github.com/dchuk/typed_eav/blob/main/bench/README.md#phase-4a-planner-extended-statistics) for safe
+evaluation SQL and evidence limits.
+
+### Multi-filter query strategy
+
+TypedEAV retains its current multi-filter query shape: it resolves each field,
+builds the corresponding typed value subquery, and chains those results onto
+the host relation with `id IN (...)`. There is no adaptive strategy or alternate
+production query API.
+
+A PostgreSQL 17 benchmark compared the shipped shape with `INTERSECT`,
+correlated `EXISTS`, and direct grouped `HAVING` under resource-capped
+co-tenancy. The run retained 2,940 attempts, including 622 right-censored
+timeouts, and 294 representative identity oracles. Twelve oracles timed out, so
+representative equivalence is unproved even though all 282 completed oracles
+matched and the smaller 98-oracle smoke matched. Alternatives remain
+research-only. Grouped `HAVING` is additionally ineligible for missing-value,
+host-universe complement, and empty-filter semantics.
+
+The result also does not establish valid buffer comparisons or 20-distinct-
+field scaling. A parser defect made every derived buffer total a false zero;
+nonzero counters remain recoverable from the retained raw plans. The
+20-predicate workloads repeat ten fields, and the skewed 10/20 workloads repeat
+five. Future research must repair and validate buffer extraction, exercise
+actual 10/20 distinct fields, complete every representative equivalence oracle,
+cover the full scope/NULL/missing/polymorphic/error contract, and show the
+pre-registered p95, planning-time, buffer, and plan-shape gates before any
+adaptive or replacement proposal. See
+[ADR 0011](../adr/0011-multi-filter-query-strategy.md) and the
+[benchmark guide](https://github.com/dchuk/typed_eav/blob/main/bench/README.md#phase-4b-multi-filter-query-shapes).

data/docs/guides/queries.md

@@ -0,0 +1,188 @@
+---
+title: "Querying typed fields"
+---
+
+# Querying typed fields
+
+[Documentation home](../index.md)
+
+## Filtering
+
+Queries use native typed columns and the indexes shipped for each type. The Field remains the owner of operand casting and validation; the query builder receives a field-normalized value rather than applying a generic Active Record cast.
+
+```ruby
+# Short form - single field filter
+Contact.with_field("age", :gt, 21)
+Contact.with_field("status", "active")           # :eq is the default operator
+Contact.with_field("nickname", :contains, "smith")
+
+# Chain them
+Contact.with_field("age", :gteq, 18)
+       .with_field("status", "active")
+       .with_field("tags", :any_eq, "vip")
+
+# Multi-filter form (good for search UIs)
+Contact.where_typed_eav(
+  { name: "age",    op: :gt,       value: 21 },
+  { name: "status", op: :eq,       value: "active" },
+  { name: "city",   op: :contains, value: "port" },
+)
+
+# Compact keys (for URL params / form submissions)
+Contact.where_typed_eav(
+  { n: "age", op: :gt, v: 21 },
+  { n: "status", v: "active" },
+)
+
+# With scoping
+Contact.where_typed_eav(
+  { name: "priority", op: :eq, value: "high" },
+  scope: current_tenant.id
+)
+
+# Combine with standard ActiveRecord
+Contact.where(company_id: 42)
+       .with_field("status", "active")
+       .with_field("age", :gteq, 21)
+       .order(:name)
+       .limit(25)
+```
+
+## Missing values and explicit NULL
+
+By default, `:is_null` finds existing Value rows whose typed value is NULL.
+It does not find hosts with no Value row. For an "is empty" search that covers
+both cases, opt into `include_missing:`:
+
+```ruby
+contacts = Contact.where(tenant_id: "t1")
+contacts.with_field("age", :is_null, scope: "t1")
+contacts.with_field("age", :is_null, scope: "t1", include_missing: true)
+
+# The same option applies to every :is_null filter in a multi-filter query:
+contacts.where_typed_eav(
+  { name: "age", op: :is_null },
+  { name: "status", value: "active" },
+  scope: "t1", include_missing: true
+)
+```
+
+`include_missing: true` means the host has **no non-NULL value** for the
+selected definition. It has no effect on `:is_not_null` or any other operator;
+it does not make an unknown field name valid. In administrative
+`TypedEAV.unscoped` queries, no matching same-name definition may hold a
+non-NULL value for that host. Other filters remain ANDed, and host authorization
+still belongs on the caller relation.
+
+## Sorting by a typed field
+
+```ruby
+Contact.where(tenant_id: "t1")
+       .order_typed_eav("age", direction: :desc, nulls: :last, scope: "t1")
+       .limit(25)
+```
+
+`order_typed_eav` returns an Active Record relation and orders in PostgreSQL,
+without loading typed values into Ruby. It replaces prior ordering while
+preserving host filters, STI restrictions, limits, and offsets. `direction:`
+accepts `:asc` (default) or `:desc`; `nulls:` accepts `:first` or `:last`
+(default in either direction). Missing rows and explicit NULLs share that
+placement. Equal values use the host primary key ascending as a stable tie-break.
+
+Scope arguments (or the ambient scope) select the winning field definition;
+**they do not filter host records by tenant**. Keep authorization/tenant filters
+on the caller relation. All-partitions `TypedEAV.unscoped` is rejected for this
+API: choose one effective definition instead. Single native scalar cells are
+supported, using their stored values (for example, reference IDs and attachment
+signed IDs, not display labels). JSON/array and multi-cell fields such as
+Currency are rejected rather than assigned an implicit ordering.
+
+## Distinct values and grouped counts
+
+```ruby
+contacts = Contact.where(tenant_id: "t1")
+contacts.distinct_typed_eav_values("status", scope: "t1", limit: 100)
+# => ["active", "paused", nil]
+contacts.typed_eav_value_counts("status", scope: "t1", limit: 100)
+# => {"active" => 24, "paused" => 3, nil => 2}
+contacts.count_distinct_typed_eav_values("status", scope: "t1")
+# => 3
+```
+
+These scalar queries run in SQL without hydrating hosts or Values. Results
+use native ascending value order, with explicit NULL (`nil`) last. Missing
+value rows contribute nothing; `false` and empty strings remain real values.
+Grouped counts count host identities, not duplicate rows introduced by joins.
+The caller's filters, STI restrictions, distinctness, and pagination determine
+the host set before summarization. Scope arguments choose field-definition
+visibility, not host authorization, just as with typed sorting.
+
+Lists and grouped-count hashes default to 100 values and accept a positive
+Integer `limit:` up to 1,000. They are truncated in value order, not ranked by
+frequency. Compare their length with `count_distinct_typed_eav_values` to
+detect truncation; that exact SQL count includes one NULL category and returns
+only an Integer, regardless of cardinality. It still requires database work
+over the matching set. Collection/multi-cell fields and all-partitions mode
+are unsupported, matching scalar sorting.
+
+## Numeric aggregates
+
+```ruby
+Contact.where(tenant_id: "t1").aggregate_typed_eav(
+  "score", operation: :sum, scope: "t1"
+)
+```
+
+`aggregate_typed_eav` requires `operation: :min`, `:max`, or `:sum` and returns
+one SQL-calculated scalar over the caller's host set. Integer fields return
+Integers; Decimal and Percentage fields preserve `BigDecimal` precision, with
+no Float conversion. Percentage values remain stored fractions, not formatted
+percent strings. Missing rows and explicit NULLs are ignored. With no non-NULL
+values, min/max return `nil` and sum returns the field's typed zero.
+
+The same host filtering, partition visibility, STI, and pagination rules as
+distinct queries apply. Only Integer/Decimal families (including Percentage)
+with a single numeric cell are supported. Reference IDs, text, collections,
+and multi-cell Currency are rejected; the gem does not silently sum identifiers
+or combine currencies. No values or host records are loaded to compute the result.
+
+## Available Operators
+
+| Operator | Works On | Description |
+|----------|----------|-------------|
+| `:eq` | all | Equal (default) |
+| `:not_eq` | all | Not equal (NULL-safe) |
+| `:gt` | numeric, date, datetime | Greater than |
+| `:gteq` | numeric, date, datetime | Greater than or equal |
+| `:lt` | numeric, date, datetime | Less than |
+| `:lteq` | numeric, date, datetime | Less than or equal |
+| `:between` | numeric, date, datetime | Between (pass Range or Array) |
+| `:contains` | text, long_text | ILIKE %value% |
+| `:not_contains` | text, long_text | NOT ILIKE %value% |
+| `:starts_with` | text, long_text | ILIKE value% |
+| `:ends_with` | text, long_text | ILIKE %value |
+| `:any_eq` | json arrays | Array contains element |
+| `:all_eq` | json arrays | Array contains all elements |
+| `:is_null` | all | Value is NULL |
+| `:is_not_null` | all | Value is not NULL |
+
+## How Type Inference Works
+
+The owning Field casts and validates query operands before SQL generation;
+Active Record supplies the SQL bind plumbing:
+
+```ruby
+# The Integer Field casts and validates the operand before SQL generation
+Contact.with_field("age", :gt, "21")
+# SQL: WHERE integer_value > 21  (not '21')
+
+# The Date Field owns date parsing and validation
+Contact.with_field("birthday", :lt, "2000-01-01")
+# SQL: WHERE date_value < '2000-01-01'::date
+
+# The Boolean Field owns truthy/falsy casting
+Contact.with_field("active", "true")
+# SQL: WHERE boolean_value = TRUE
+```
+
+Field-owned casting keeps query operands aligned with write semantics, including strict range/array shapes and specialized fields such as Currency and Reference. The resulting normalized operand is bound against the Field's typed column; Active Record supplies SQL bind plumbing, not the field's domain semantics.