huginn_datatable 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 05de880382d0477bad2668f9f64ae7cc9d3bbbc78f71c6e359dfba0ccf9cc43b
-  data.tar.gz: 7d8053d071e2bc52b09463af4a8b6ed7c5b024afd5f9fa5b3f6e39f30cbb5f04
+  metadata.gz: '05719914312baa1225e26059bd5154312ee5df8bc369df6f5f9676d423ed9d0e'
+  data.tar.gz: b876f81b7add7781a4bd6b7424e3674f251c09ecfe38e0a70e8f891bb2286c76
 SHA512:
-  metadata.gz: 20676dc384afff5cb64a1b6289f13207fdc1cde8bc549659ce876408eb435efb9668d2aff5064968bf7cc3d21e71c8e801603ba8737d5c0cc034c9f368788b9e
-  data.tar.gz: 36b11ac27abd80641faa22db4e44feb865881d27f5dd3d8c2b6cb9df1b8dcef10a24b22271021c1e12603481da18a7e17eef4201983586c0c2d2851cf9a86bd6
+  metadata.gz: bed2c0bc55d70a6ceaa57a6a84fb264360df75cbab4f42a2a82031073fdd0f15778b1681afcdc35aba31a478e4e8c18a6acfc960b725a1018306211845e72c72
+  data.tar.gz: 8f58d9e3fec2d2a38350dc010f9b68d6ab60449e27338291dfae8ae7bcaba2b96cbb64f945067487c1ed97889c6642bbb451dfde21644b0170e708ae109aa30a

data/CHANGELOG.md

@@ -2,6 +2,16 @@
 
 ### Unreleased
 
+#### 0.2.1
+
+- Association-scoped `search` now matches through **primary-key semi-join subqueries** (`pk IN (SELECT DISTINCT pk FROM <base> JOIN <chain> WHERE <fuzzy>)`) instead of `left_joins` — one subquery per distinct association chain, combined with OR. The base relation never joins, so the count is a plain `COUNT(*)`.
+
+#### 0.2.0
+
+- **Breaking:** `:pg_trgm` search now uses the PostgreSQL `%` operator (`UNACCENT(col) % UNACCENT('term')`), driven by the **indexable** operator instead of a bare `similarity()` comparison. A `gin_trgm_ops` GIN index on `UNACCENT(col)` is used when present (BitmapOr over the same index with the ILIKE branch); without it, results stay correct via sequential scan.
+- **Breaking:** removed the `fuzzy_threshold` configuration. The similarity cutoff is now the PostgreSQL GUC `pg_trgm.similarity_threshold` (default `0.3`, same as the old gem default) — tune with `SELECT set_limit(...)` / `SET pg_trgm.similarity_threshold`.
+- New `unaccent_function` configuration (default `"unaccent"`): the unaccent function used in search SQL. Point it at an `IMMUTABLE` wrapper to make GIN trigram indexes usable (the pg builtin `unaccent()` is `STABLE` on PG 13+ and cannot be indexed directly).
+- New generator `rails g huginn:trigram_indexes [Model...]` scaffolds a migration that creates an `IMMUTABLE public.f_unaccent` wrapper and `CREATE INDEX ... USING gin (public.f_unaccent(col) gin_trgm_ops)` for Searchable models (all `:string`/`:text` columns, including association-scoped ones).
 - Runtime dependency on Pagy declared in the gemspec (`pagy >= 6`) — consumers install it automatically (datatable pagination is not test-only anymore).
 - Supported Rails floor bumped to **7.1** (`>= 7.1, < 9`); Ruby `>= 3.0` without an upper bound (Rails 8 + Ruby 4 supported).
 - Appraisals now cover **Rails 7.1 / 7.2 / 8.0** (was 6.1/7.0/7.1) with generated `gemfiles/*.gemfile`; CI matrix runs every Rails over supported Rubies (3.0 up to 4.0, including Rails 8 on Ruby 4.0).

data/README.md

@@ -1,3 +1,7 @@
+<p align="center">
+  <img src="./logo.png" width="140" alt="Huginn logo">
+</p>
+
 <h1 align="center">Huginn</h1>
 
 <p align="center">
@@ -9,14 +13,14 @@
   🇺🇸 English · <a href="./README.pt-BR.md">🇧🇷 Português</a>
 </p>
 
-Huginn is a lightweight query layer for Rails that turns a raw datatable request into a **lean count, a paginated subset and one preload** — instead of a massive JOIN materialized in memory. It also ships a PostgreSQL fuzzy-search builder (`pg_trgm` similarity with `unaccent` and `ILIKE` fallback) that is tolerant to typos and accents.
+Huginn is a lightweight query layer for Rails that turns a raw datatable request into a **lean count, a paginated subset and one preload** — instead of a massive JOIN materialized in memory. It also ships a PostgreSQL fuzzy-search builder (`pg_trgm` similarity via the `%` operator with `unaccent` and `ILIKE` fallback) that is tolerant to typos and accents — and that can lean on a GIN index when present.
 
 ## Highlights
 
 - **Two-phase execution** — association filters/orders/range become reflection-secured subqueries, then a lean count and `preload` **only on the paginated subset**.
-- **Lean counts** — `COUNT(DISTINCT pk)` through a stripped relation; no JOIN materialization.
+- **Lean counts** — the base relation never joins (`COUNT(*)` over a stripped relation); association matches run as pk subqueries instead.
 - **SQL injection safe ordering/filtering** — every column reference is resolved through Arel reflection, never string-interpolated.
-- **Accent/typo tolerant search** — `pg_trgm` similarity OR `unaccent+ILIKE`, with a configurable fallback chain.
+- **Accent/typo tolerant search** — `pg_trgm` similarity (`%` operator) OR `unaccent+ILIKE`, with a configurable fallback chain; uses a `gin_trgm_ops` GIN index when one exists.
 - **Rails conventions** — works with `ActionController::Parameters`, Railtie auto-includes both concerns (toggleable), zero boilerplate.
 
 ## Development
@@ -62,17 +66,24 @@ gem "huginn"
 ```ruby
 # config/initializers/huginn.rb
 Huginn.configure do |config|
-  # :pg_trgm  (recommended) — trigram similarity OR unaccent+ILIKE
+  # :pg_trgm  (recommended) — trigram similarity (%) OR unaccent+ILIKE
   # :unaccent               — unaccent + ILIKE only
   # :simple                 — plain LIKE
   config.search_strategy = :pg_trgm
 
-  config.fuzzy_threshold = 0.3   # similarity() cutoff used by :pg_trgm
+  # Unaccent function used around search terms/columns. Defaults to the pg
+  # builtin UNACCENT(). Point it at the IMMUTABLE wrapper created by
+  # `rails g huginn:trigram_indexes` ("public.f_unaccent") so GIN trigram
+  # indexes can actually be used.
+  config.unaccent_function = "unaccent"
+
   config.pagy_items = 10         # default page size
   config.pagy_max_items = 500    # hard cap for per_page
 end
 ```
 
+> **Threshold:** under `:pg_trgm` the similarity cutoff is the PostgreSQL GUC `pg_trgm.similarity_threshold` (default `0.3`), **not** a gem-level setting. Tune it with `SET pg_trgm.similarity_threshold = 0.4` / `SELECT set_limit(0.4)` on the database.
+
 ## Railtie (automatic include)
 
 By default the Railtie includes `Huginn::Datatable` and `Huginn::Searchable` into every `ActiveRecord::Base` model. You do **not** need `include` statements unless you opt in selectively:
@@ -118,7 +129,7 @@ Any `column` **or** `association.column` reference is validated and mapped to it
 Plano.datatable({ orders: [{ "operadora.pessoa.nome" => "asc" }] }, allowed_paths: [{ operadora: :pessoa }])
 ```
 
-> Association filters/range and ordering use reflection-resolved subqueries (see the "allowlist of associations" section below). The main relation stays singular and the count is `COUNT(DISTINCT pk)`.
+> Association filters/range, ordering and search use reflection-resolved subqueries (see the "allowlist of associations" section below). The base relation stays singular and join-free, so the count is a plain `COUNT(*)`.
 
 ## Association allowlist (`allowed_paths`)
 
@@ -174,17 +185,57 @@ class Person < ApplicationRecord
   searchable_columns :name, company: [:name, :cnpj]
 end
 
-Person.search("globex")                            # matches company.name via a left_join
+Person.search("globex")                            # matches company.name via a pk subquery
 Person.search("kayky", distinct: false)            # disable the implicit DISTINCT
 ```
 
 `Huginn::Datatable` reuses `Huginn::Searchable.search` automatically when the model responds to `search`.
 
+## Trigram indexes (performance)
+
+Under `:pg_trgm`, each searchable column produces an indexable predicate:
+
+```sql
+(UNACCENT(col) % UNACCENT('term')) OR (UNACCENT(col) ILIKE UNACCENT('%term%'))
+```
+
+Both branches are supported by a **GIN trigram index**, so the planner can run a `BitmapOr` on it. Because the pg builtin `unaccent()` is `STABLE` (not `IMMUTABLE`) on PostgreSQL 13+, it cannot be used directly in an index expression — you need an `IMMUTABLE` wrapper plus an expression index on it:
+
+```sql
+CREATE OR REPLACE FUNCTION public.f_unaccent(text)
+RETURNS text AS $$
+  SELECT public.unaccent('public.unaccent', $1);
+$$ LANGUAGE sql IMMUTABLE PARALLEL SAFE;
+
+CREATE INDEX index_people_name_trgm ON people USING gin (public.f_unaccent(name) gin_trgm_ops);
+```
+
+Then point the gem at the wrapper:
+
+```ruby
+Huginn.configure { |c| c.unaccent_function = "public.f_unaccent" }
+```
+
+Or scaffold the migration for every Searchable model (wrapper + indexes included):
+
+```sh
+rails g huginn:trigram_indexes                     # all Searchable models
+rails g huginn:trigram_indexes Person Product      # specific models
+```
+
+The index is **optional** — without it the search still returns correct results (via a sequential scan), and the same indexes also accelerate the `:unaccent` (ILIKE) and `:simple` (LIKE) strategies.
+
+Notes:
+
+- Requires the `pg_trgm` and `unaccent` extensions.
+- Only helpful for search terms of **3 or more characters** — trigrams need that many to match. Shorter terms always scan.
+- To see it working, run a `Person.search("term").explain` with the wrapper configured and a `gin_trgm_ops` index present.
+
 ## Query efficiency
 
 ```
 phase 1  build the relation          subqueries (pk IN … / ORDER BY (SELECT …)) + search + filters + order   (no data in memory)
-phase 2  count                       SELECT COUNT(DISTINCT "<pk column>") ... (subquery, pk-indexed)
+phase 2  count                       SELECT COUNT(*) ... (base relation is join-free)
           paginate                   offset / limit
           preload                    SELECT ... WHERE id IN (subset)        (2nd lightweight query)
 ```
@@ -195,7 +246,7 @@ For a `Plano` datatable with deep `includes:`, this is exactly **2 extra queries
 
 ```
 lib/huginn.rb                       entry, Huginn.configure, Huginn.instrument
-lib/huginn/configuration.rb         search_strategy, fuzzy_threshold, pagy_*
+lib/huginn/configuration.rb         search_strategy, unaccent_function, pagy_*
 lib/huginn/railtie.rb               auto-includes concerns into ActiveRecord
 lib/huginn/datatable.rb             Huginn::Datatable (aggregator)
 lib/huginn/datatable/datatable.rb   the datatable Concern
@@ -206,8 +257,9 @@ lib/huginn/datatable/filter_normalizer.rb  functional param normalization
 lib/huginn/datatable/paginator.rb    lean count, pagination, isolated preload
 lib/huginn/searchable.rb            Huginn::Searchable (aggregator)
 lib/huginn/searchable/searchable.rb the search Concern + DSL
-lib/huginn/searchable/query.rb      tolerant search builder (joins + OR)
+lib/huginn/searchable/query.rb      tolerant search builder (subqueries + OR)
 lib/huginn/searchable/fuzzy.rb      pg_trgm / unaccent / simple predicates
+lib/generators/...                  `huginn:trigram_indexes` (GIN index migrations)
 ```
 
 ## Instrumentation

data/README.pt-BR.md

@@ -1,3 +1,7 @@
+<p align="center">
+  <img src="./logo.png" width="140" alt="Logo do Huginn">
+</p>
+
 <h1 align="center">Huginn</h1>
 
 <p align="center">
@@ -9,14 +13,14 @@
   <a href="./README.md">🇺🇸 English</a> · 🇧🇷 Português
 </p>
 
-O Huginn é uma camada de consulta leve para Rails que transforma uma requisição de datatable bruta em um **count enxuto, um subconjunto paginado e um único preload** — em vez de um JOIN enorme materializado em memória. Também traz um construtor de busca *fuzzy* para PostgreSQL (similaridade `pg_trgm` com `unaccent` e fallback `ILIKE`) tolerante a erros de digitação e acentuação.
+O Huginn é uma camada de consulta leve para Rails que transforma uma requisição de datatable bruta em um **count enxuto, um subconjunto paginado e um único preload** — em vez de um JOIN enorme materializado em memória. Também traz um construtor de busca *fuzzy* para PostgreSQL (similaridade `pg_trgm` via operador `%` com `unaccent` e fallback `ILIKE`) tolerante a erros de digitação e acentuação — e que pode usar um índice GIN quando presente.
 
 ## Destaques
 
 - **Execução em duas fases** — filtros/orders/ranges de associação se tornam subqueries resolvidas por reflexão, e o `preload` é feito **somente no subconjunto paginado**.
-- **Counts enxutos** — `COUNT(DISTINCT pk)` através de uma relation restrita, sem materialização de JOINs.
+- **Counts enxutos** — a relation base nunca faz join (`COUNT(*)` sobre uma relation restrita); as buscas de associação acontecem via subqueries no pk.
 - **Ordenação/filtro seguros contra SQL injection** — toda referência de coluna é resolvida via reflexão do Arel, nunca interpolada como string.
-- **Busca tolerante a acentos/typos** — similaridade `pg_trgm` OU `unaccent+ILIKE`, com cadeia de fallback configurável.
+- **Busca tolerante a acentos/typos** — similaridade `pg_trgm` (operador `%`) OU `unaccent+ILIKE`, com cadeia de fallback configurável; usa índice GIN `gin_trgm_ops` quando existente.
 - **Convenções do Rails** — funciona com `ActionController::Parameters`, Railtie inclui ambos os concerns automaticamente (desligável), zero boilerplate.
 
 ## Desenvolvimento
@@ -62,17 +66,24 @@ gem "huginn"
 ```ruby
 # config/initializers/huginn.rb
 Huginn.configure do |config|
-  # :pg_trgm  (recomendado) — similaridade trigram OU unaccent+ILIKE
+  # :pg_trgm  (recomendado) — similaridade trigram (%) OU unaccent+ILIKE
   # :unaccent               — somente unaccent + ILIKE
   # :simple                 — LIKE simples
   config.search_strategy = :pg_trgm
 
-  config.fuzzy_threshold = 0.3   # cutoff de similarity() usado por :pg_trgm
+  # Função unaccent usada em torno de termos/colunas. O default é o UNACCENT()
+  # built-in do PG. Aponte para o wrapper IMMUTABLE criado por
+  # `rails g huginn:trigram_indexes` ("public.f_unaccent") para que índices
+  # GIN trigram possam ser usados de fato.
+  config.unaccent_function = "unaccent"
+
   config.pagy_items = 10         # tamanho de página padrão
   config.pagy_max_items = 500    # teto máximo de per_page
 end
 ```
 
+> **Limiar:** sob `:pg_trgm`, o corte de similaridade é o GUC do PostgreSQL `pg_trgm.similarity_threshold` (default `0.3`), **não** uma configuração da gem. Ajuste com `SET pg_trgm.similarity_threshold = 0.4` / `SELECT set_limit(0.4)` no banco.
+
 ## Railtie (include automático)
 
 Por padrão, o Railtie inclui `Huginn::Datatable` e `Huginn::Searchable` em toda `ActiveRecord::Base`. Você **não** precisa de `include`, a menos que opte seletivamente:
@@ -118,7 +129,7 @@ Qualquer referência `column` **ou** `associacao.column` é validada e mapeada p
 Plano.datatable({ orders: [{ "operadora.pessoa.nome" => "asc" }] }, allowed_paths: [{ operadora: :pessoa }])
 ```
 
-> Filtros/ranges e order de associação usam subqueries resolvidas por reflexão (veja a seção "Allowlist de associações" abaixo). A relation principal permanece única e o count é `COUNT(DISTINCT pk)`.
+> Filtros/ranges, order e busca de associação usam subqueries resolvidas por reflexão (veja a seção "Allowlist de associações" abaixo). A relation base permanece única e sem joins, então o count é um `COUNT(*)` simples.
 
 ## Allowlist de associações (`allowed_paths`)
 
@@ -174,17 +185,57 @@ class Person < ApplicationRecord
   searchable_columns :name, company: [:name, :cnpj]
 end
 
-Person.search("globex")                            # encontra company.name via left_join
+Person.search("globex")                            # encontra company.name via subquery no pk
 Person.search("kayky", distinct: false)            # desativa o DISTINCT implícito
 ```
 
 `Huginn::Datatable` reutiliza `Huginn::Searchable.search` automaticamente quando o modelo responde a `search`.
 
+## Índices trigram (performance)
+
+Sob `:pg_trgm`, cada coluna pesquisável gera um predicado indexável:
+
+```sql
+(UNACCENT(col) % UNACCENT('termo')) OR (UNACCENT(col) ILIKE UNACCENT('%termo%'))
+```
+
+Ambos os ramos são suportados por um **índice GIN trigram**, então o planner pode executar um `BitmapOr` sobre ele. Como o `unaccent()` built-in do PG é `STABLE` (não `IMMUTABLE`) desde o PostgreSQL 13+, ele não pode ser usado diretamente numa expressão de índice — você precisa de um wrapper `IMMUTABLE` + índice de expressão:
+
+```sql
+CREATE OR REPLACE FUNCTION public.f_unaccent(text)
+RETURNS text AS $$
+  SELECT public.unaccent('public.unaccent', $1);
+$$ LANGUAGE sql IMMUTABLE PARALLEL SAFE;
+
+CREATE INDEX index_people_name_trgm ON people USING gin (public.f_unaccent(name) gin_trgm_ops);
+```
+
+E aponte a gem para o wrapper:
+
+```ruby
+Huginn.configure { |c| c.unaccent_function = "public.f_unaccent" }
+```
+
+Ou gere a migration para todos os modelos Searchable (wrapper + índices inclusos):
+
+```sh
+rails g huginn:trigram_indexes                     # todos os modelos Searchable
+rails g huginn:trigram_indexes Person Product      # modelos específicos
+```
+
+O índice é **opcional** — sem ele a busca ainda retorna resultados corretos (via sequential scan), e os mesmos índices também aceleram as estratégias `:unaccent` (ILIKE) e `:simple` (LIKE).
+
+Observações:
+
+- Requer as extensões `pg_trgm` e `unaccent`.
+- Só ajuda em termos de busca de **3 caracteres ou mais** — trigramas precisam disso para casar. Termos menores sempre fazem scan.
+- Para conferir o uso, rode `Person.search("termo").explain` com o wrapper configurado e um índice `gin_trgm_ops` presente.
+
 ## Eficiência da query
 
 ```
 fase 1  construir a relation      subqueries (pk IN … / ORDER BY (SELECT …)) + search + filters + order   (sem dados em memória)
-fase 2  count                     SELECT COUNT(DISTINCT "<pk>") ... (subquery, indexada por pk)
+fase 2  count                     SELECT COUNT(*) ... (relation base sem joins)
         paginate                  offset / limit
         preload                   SELECT ... WHERE id IN (subset)        (2ª query leve)
 ```
@@ -195,7 +246,7 @@ Para um datatable de `Plano` com `includes:` profundos, isso são exatamente **2
 
 ```
 lib/huginn.rb                       entry, Huginn.configure, Huginn.instrument
-lib/huginn/configuration.rb         search_strategy, fuzzy_threshold, pagy_*
+lib/huginn/configuration.rb         search_strategy, unaccent_function, pagy_*
 lib/huginn/railtie.rb               auto-inclui os concerns no ActiveRecord
 lib/huginn/datatable.rb             Huginn::Datatable (agregador)
 lib/huginn/datatable/datatable.rb   o Concern do datatable
@@ -206,8 +257,9 @@ lib/huginn/datatable/filter_normalizer.rb  normalização funcional de params
 lib/huginn/datatable/paginator.rb  count enxuto, paginação, preload isolado
 lib/huginn/searchable.rb            Huginn::Searchable (agregador)
 lib/huginn/searchable/searchable.rb o Concern do search + DSL
-lib/huginn/searchable/query.rb      construtor de busca tolerante (joins + OR)
+lib/huginn/searchable/query.rb      construtor de busca tolerante (subqueries + OR)
 lib/huginn/searchable/fuzzy.rb      predicados pg_trgm / unaccent / simple
+lib/generators/...                  gerador `huginn:trigram_indexes` (migrations de índices GIN)
 ```
 
 ## Instrumentação

data/lib/generators/huginn/trigram_indexes/templates/add_huginn_trigram_indexes.rb.tt

@@ -0,0 +1,26 @@
+class AddHuginnTrigramIndexes < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def up
+    enable_extension "unaccent" unless connection.extension_enabled?("unaccent")
+
+    # The pg builtin unaccent() is STABLE (not IMMUTABLE) since PG 13+, so it
+    # cannot be used in an index expression. Wrap it in an IMMUTABLE function
+    # that always pins the default dictionary.
+    execute <<~SQL
+      CREATE OR REPLACE FUNCTION public.f_unaccent(text)
+      RETURNS text AS $$
+        SELECT public.unaccent('public.unaccent', $1);
+      $$
+      LANGUAGE sql IMMUTABLE PARALLEL SAFE;
+    SQL
+<% @entries.each do |entry| %>
+    add_index :<%= entry[:table] %>, "public.f_unaccent(<%= entry[:column] %>) gin_trgm_ops", using: :gin, name: "index_<%= entry[:table] %>_<%= entry[:column] %>_trgm"
+<% end %>
+  end
+
+  def down
+<% @entries.each do |entry| %>
+    remove_index :<%= entry[:table] %>, name: "index_<%= entry[:table] %>_<%= entry[:column] %>_trgm"
+<% end %>
+    execute "DROP FUNCTION IF EXISTS public.f_unaccent(text)"
+  end
+end

data/lib/generators/huginn/trigram_indexes/trigram_indexes_generator.rb

@@ -0,0 +1,90 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/migration"
+
+module Huginn
+  module Generators
+    # Scaffolds a migration that creates the PostgreSQL expression GIN indexes
+    # (gin_trgm_ops on an IMMUTABLE unaccent wrapper of the column) backing
+    # Huginn::Searchable's :pg_trgm strategy for every model that includes
+    # Huginn::Searchable.
+    #
+    #   rails g huginn:trigram_indexes                 # all Searchable models
+    #   rails g huginn:trigram_indexes Person Product  # specific models
+    #
+    # The migration creates an IMMUTABLE public.f_unaccent(text) wrapper and
+    # the GIN indexes. To make the search actually use them, point the gem at
+    # the wrapper (the pg builtin unaccent() is STABLE on PG 13+ and cannot be
+    # indexed directly):
+    #
+    #   Huginn.configure { |c| c.unaccent_function = "public.f_unaccent" }
+    #
+    # Indexes are only used when the :pg_trgm strategy is configured and the
+    # pg_trgm/unaccent extensions are installed.
+    class TrigramIndexesGenerator < Rails::Generators::Base
+      include Rails::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      argument :model_names, type: :array, default: [], desc: "Models to index (default: all Searchable models)"
+
+      def self.next_migration_number(dirname)
+        next_migration_number = current_migration_number(dirname) + 1
+        ActiveRecord::Migration.next_migration_number(next_migration_number)
+      end
+
+      def copy_migration
+        @entries = resolve_entries
+        return if @entries.empty?
+
+        migration_template(
+          "add_huginn_trigram_indexes.rb.tt",
+          "db/migrate/add_huginn_trigram_indexes.rb"
+        )
+      end
+
+      private
+
+      def resolve_entries
+        models = if model_names.any?
+                   model_names.map { |name| constantize(name) }
+                 else
+                   ActiveRecord::Base.descendants
+                 end
+
+        models.filter_map do |model|
+          next unless model.respond_to?(:searchable_columns_config_resolved)
+
+          columns_for(model)
+        end.reject(&:empty?).flatten.uniq
+      end
+
+      def columns_for(model)
+        scope = []
+        resolved = model.searchable_columns_config_resolved
+
+        Array(resolved[:columns]).map(&:to_s).reject(&:blank?).each do |field|
+          scope << { table: model.table_name, column: field } if model.columns_hash.key?(field)
+        end
+
+        resolved[:associations].each do |assoc, cols|
+          reflection = model.reflect_on_association(assoc.to_sym)
+          next unless reflection
+
+          Array(cols).map(&:to_s).each do |col|
+            scope << { table: reflection.klass.table_name, column: col } if reflection.klass.columns_hash.key?(col)
+          end
+        end
+
+        scope
+      end
+
+      def constantize(klass_name)
+        klass_name.constantize
+      rescue NameError
+        raise Thor::Error, "Could not find model #{klass_name.inspect}"
+      end
+    end
+  end
+end

data/lib/huginn/configuration.rb

@@ -2,19 +2,26 @@
 
 module Huginn
   class Configuration
-    attr_accessor :fuzzy_threshold, :pagy_items, :pagy_max_items
+    attr_accessor :pagy_items, :pagy_max_items, :unaccent_function
     attr_writer :search_strategy, :auto_include_datatable, :auto_include_searchable
 
     def initialize
-      @fuzzy_threshold = 0.3
       @pagy_items = 10
       @pagy_max_items = 500
+      # The unaccent function SQL emits around searchable columns/terms. Defaults
+      # to the pg builtin UNACCENT(). To let GIN trigram indexes be used, point
+      # this at an IMMUTABLE wrapper (see `rails g huginn:trigram_indexes`),
+      # e.g. "public.f_unaccent".
+      @unaccent_function = "unaccent"
       @search_strategy = :pg_trgm
       @auto_include_datatable = true
       @auto_include_searchable = true
     end
 
-    # :pg_trgm  -> similarity(trgm) OR unaccent+ILIKE (recommended)
+    # :pg_trgm  -> trigram similarity (%) OR unaccent+ILIKE. The similarity
+    #              cutoff is the PostgreSQL GUC pg_trgm.similarity_threshold
+    #              (tune with SELECT set_limit(...)); a gin_trgm_ops GIN index
+    #              on UNACCENT(col) is used when present.
     # :unaccent -> unaccent + ILIKE only
     # :simple   -> plain LIKE
     def search_strategy

data/lib/huginn/datatable/paginator.rb

@@ -10,10 +10,9 @@ module Huginn
     #   2. count lean, paginate with Pagy, then preload associations only
     #      on the small paginated subset.
     #
-    # The count runs against a stripped relation (no select/includes/order/
-    # offset/limit) selecting only the primary key with DISTINCT, so that
-    # PostgreSQL answers it through the PK index as a subquery instead of a
-    # massive COUNT(DISTINCT ...) over the joins.
+    # The base relation never carries joins (association filters, ranges,
+    # orders and the search are all resolved through primary-key subqueries),
+    # so the count stays a plain COUNT(*) over the stripped relation.
     class Paginator
       attr_reader :relation, :params, :includes
 
@@ -77,10 +76,7 @@ module Huginn
       end
 
       def total_count
-        stripped = relation.except(:select, :includes, :order, :offset, :limit)
-        return stripped.count unless stripped.left_outer_joins_values.any? || stripped.joins_values.any?
-
-        stripped.select(stripped.arel_table[stripped.primary_key]).distinct.count
+        relation.except(:select, :includes, :order, :offset, :limit).count
       end
 
       def config

data/lib/huginn/searchable/fuzzy.rb

@@ -6,16 +6,24 @@ module Huginn
     #
     # Strategy chain (driven by Huginn.configuration.search_strategy):
     #
-    #   :pg_trgm   -> trigram similarity OR unaccent+ILIKE (best for typos)
+    #   :pg_trgm   -> trigram similarity (%) OR unaccent+ILIKE (best for typos).
+    #                 Both branches are supported by a gin_trgm_ops GIN index
+    #                 on UNACCENT(col); the index is used only when it exists.
     #   :unaccent  -> unaccent + ILIKE (case/accents insensitive)
     #   :simple    -> plain LIKE
     module Fuzzy
-      # Gentle helpers to wrap a node in UNACCENT(...).
+      # Gentle helpers to wrap a node in UNACCENT(...). The function name comes
+      # from Huginn.configuration.unaccent_function so an IMMUTABLE wrapper can
+      # be swapped in to make GIN trigram indexes usable.
       module Unaccentable
         private
 
         def unaccent(node)
-          Arel::Nodes::NamedFunction.new("UNACCENT", [node])
+          Arel::Nodes::NamedFunction.new(unaccent_function, [node])
+        end
+
+        def unaccent_function
+          Huginn.configuration.unaccent_function
         end
       end
 
@@ -62,23 +70,22 @@ module Huginn
       class Trigram
         include Unaccentable
 
-        def initialize(column, value, threshold)
+        def initialize(column, value)
           @column = column
           @value = value
-          @threshold = threshold
         end
 
         def predicate
-          similarity = Arel::Nodes::NamedFunction.new("similarity", [
-            unaccent(@column),
-            unaccent(Arel::Nodes.build_quoted(@value, @column))
-          ])
-          threshold = Arel::Nodes.build_quoted(@threshold)
-          Arel::Nodes::InfixOperation.new(">", similarity, threshold).or(fallback)
+          similarity = Arel::Nodes::InfixOperation.new("%", unaccent(@column), unaccent(value))
+          similarity.or(fallback)
         end
 
         private
 
+        def value
+          Arel::Nodes.build_quoted(@value, @column)
+        end
+
         def fallback
           Unaccent.new(@column, @value).predicate
         end

data/lib/huginn/searchable/query.rb

@@ -1,13 +1,23 @@
 # frozen_string_literal: true
 
+require_relative "../datatable/association_path"
+
 module Huginn
   module Searchable
+    AssociationPath = Datatable::AssociationPath unless const_defined?(:AssociationPath)
     # Builds a tolerant full-text search relation over a model's searchable
-    # columns, optionally crossing associations through left_joins.
+    # columns.
+    #
+    # Columns are split in two groups following the datatable pattern:
     #
-    # All matched columns are fused into a single Arel OR predicate, so the
-    # whole search stays one query regardless of how many columns (or
-    # associations) are involved.
+    #   * the model's own columns  -> direct fuzzy predicates on the base WHERE
+    #   * association columns      -> semi-join subqueries on the primary key
+    #                                  pk IN (SELECT DISTINCT pk FROM base
+    #                                       JOIN <chain> ... WHERE <fuzzy>)
+    #
+    # The base relation never carries joins, so every match group stays one
+    # query and the count remains a plain COUNT(*). Multiple association
+    # chains produce one subquery each, combined with OR.
     class Query
       def self.call(model, value, options = {})
         new(model, value, options).call
@@ -25,52 +35,72 @@ module Huginn
         return apply_distinct(relation) if @value.strip.empty?
 
         resolution = resolve_columns
-        return apply_distinct(relation) if resolution.fetch(:attrs).empty?
+        predicates = fuzzy_predicates(resolution.fetch(:own))
+        resolution.fetch(:associations).each do |chain, attrs|
+          subquery = association_subquery(chain, attrs)
+          predicates << arel_table[primary_key].in(subquery.arel) if subquery
+        end
 
-        joins = resolution.fetch(:joins)
-        relation = relation.left_joins(*joins) if joins.any?
+        return apply_distinct(relation) if predicates.empty?
 
-        predicate = resolution.fetch(:attrs).map { |attr| fuzzy_predicate(attr) }.reduce(&:or)
-        apply_distinct(relation.where(predicate))
+        apply_distinct(relation.where(predicates.reduce(&:or)))
       end
 
       private
 
+      def fuzzy_predicates(attrs)
+        attrs.map { |attr| fuzzy_predicate(attr) }
+      end
+
       def resolve_columns
         columns = @model.searchable_columns_config_resolved
-        joins = []
-        attrs = []
+        own = []
+        associations = Hash.new { |hash, key| hash[key] = [] }
 
         Array(columns[:columns]).map(&:to_s).reject(&:blank?).each do |field|
           if field.include?(".")
-            name, col = field.split(".", 2)
-            push_association(joins, attrs, name, col)
+            chain, column = field.split(".", 2)
+            push_association(associations, chain, column)
           else
-            attrs << @model.arel_table[field] if @model.columns_hash.key?(field)
+            own << @model.arel_table[field] if @model.columns_hash.key?(field)
           end
         end
 
         columns[:associations].each do |name, cols|
-          Array(cols).each { |col| push_association(joins, attrs, name.to_s, col.to_s) }
+          Array(cols).each { |column| push_association(associations, name.to_s, column.to_s) }
         end
 
-        { joins: joins.uniq, attrs: attrs }
+        { own: own, associations: associations }
+      end
+
+      def push_association(groups, chain, column)
+        path = AssociationPath.call(@model, "#{chain}.#{column}")
+        return unless path.valid?
+        return unless path.target_klass.columns_hash.key?(path.column)
+
+        groups[path.association_names] << path.target_table[path.column]
       end
 
-      def push_association(joins, attrs, name, col)
-        association = @model.reflect_on_association(name.to_sym)
-        return unless association
+      # Semi-join subquery over the base table: DISTINCT pk while joining the
+      # association chain of the group and applying the fuzzy predicates on the
+      # target columns. Mirrors datatable's association_ids_subquery.
+      def association_subquery(chain, attrs)
+        return nil if chain.empty?
 
-        table = association.klass.arel_table
-        attrs << table[col] if association.klass.columns_hash.key?(col)
-        joins << name.to_sym unless joins.include?(name.to_sym)
+        spec = AssociationPath.join_spec_for(chain.map(&:to_sym))
+        base = @model.all.except(:select, :order, :offset, :limit)
+                        .select(arel_table[primary_key])
+                        .distinct
+
+        predicate = fuzzy_predicates(attrs).reduce(&:or)
+        base.joins(spec).where(predicate)
       end
 
       def fuzzy_predicate(attr)
         case strategy
         when :pg_trgm
           if pg_trgm_available?
-            Fuzzy::Trigram.new(attr, @value, Huginn.configuration.fuzzy_threshold).predicate
+            Fuzzy::Trigram.new(attr, @value).predicate
           else
             fallback_fuzzy(attr)
           end
@@ -121,6 +151,14 @@ module Huginn
         @model.connection.adapter_name.downcase.include?("postgres")
       end
 
+      def arel_table
+        @model.arel_table
+      end
+
+      def primary_key
+        @model.primary_key
+      end
+
       def apply_distinct(relation)
         @distinct ? relation.distinct : relation
       end

data/lib/huginn/searchable/searchable.rb

@@ -17,8 +17,8 @@ module Huginn
       #
       # When nothing is declared, every :string/:text column of the model is
       # searched automatically. Columns may be association-scoped ("person.name")
-      # or declared via keyword arguments; scoped columns are reached through a
-      # left_join.
+      # or declared via keyword arguments; scoped columns are resolved through
+      # semi-join subqueries on the primary key (no joins on the base relation).
       def searchable_columns(*columns, **associations)
         self.searchable_columns_config = {
           columns: columns.flatten.compact.map(&:to_s),

data/lib/huginn/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Huginn
-  VERSION = "0.1.0"
+  VERSION = "0.2.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: huginn_datatable
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Kayky Marcelo
@@ -139,6 +139,8 @@ files:
 - LICENSE.txt
 - README.md
 - README.pt-BR.md
+- lib/generators/huginn/trigram_indexes/templates/add_huginn_trigram_indexes.rb.tt
+- lib/generators/huginn/trigram_indexes/trigram_indexes_generator.rb
 - lib/huginn.rb
 - lib/huginn/configuration.rb
 - lib/huginn/datatable.rb