pgi 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 00a4f48d4d9337019f30a191e60474cb75ff45a19c1653d0bf8a1897dd6551bf
-  data.tar.gz: 2187c091a690e52b9d66efdbe7038932d6cc783aab8d95516246fe0ca564d48d
+  metadata.gz: 0e861fe364065922d9e21beb03770004e3645a31b2cd9fe827b0a517aa35739e
+  data.tar.gz: 45e6948a019f2194f4d1f2cc49ab81d575c55d16014baf3585f00a85b957374e
 SHA512:
-  metadata.gz: 2cd4e11c2341f5151ba6b09549221e6345f0f9d1442c36d9631b14242815b95b470096524229e94296bdc35811f966eb5ff4a7250c1de442678d73cd498f81b6
-  data.tar.gz: 267ba7b3dc84f507b6d47407cc77a42e6bc3a42fad1797c36494a90d672a2cf5a1dffed85ddf3953dc55666e00e37cdf8e08c9883fd1a8bce761a3b078897d6a
+  metadata.gz: 22c23e207efec5a443f93c316dad62e556ad63bffe3d18b618d26dc211d56676a51789f94ccdeb9acb791433b3c94b76d6145e455358435ed8dff193090d7519
+  data.tar.gz: 2a6f49983336cedaf4997d625923964611336f309c686f8b548d484324dcf110c1a71da20e8e0c7618a8c1909d043be341437dea3cef3293ec71a8e0b0ffb1c4

data/CHANGELOG.md

@@ -1,5 +1,64 @@
 # CHANGELOG
 
+## 1.1.0 (2026-08-28)
+
+The query release: joins, search, projected columns — and server notices that
+respect your logger.
+
+- **Joins** — `#join(table, on:)` / `#page(joins:)` add INNER JOINs for
+  filtering and sorting on combined rows; result rows stay the base table's,
+  so model mapping is unchanged.
+
+  ```ruby
+  Repository.join(:memberships, on: { id: :member_id })
+            .where(memberships: { accepted: true }).all
+  ```
+
+  `#where` and `#order` take qualified columns (`{ users: :name }`), keyset
+  pagination works over a joined sort column (FK -> PK cardinality), and an
+  `on:` key may itself be qualified to chain a second hop off an earlier join
+  (declare joins in dependency order).
+
+- **Search** — `#search(columns, terms)` / `#page(search:)`: case-insensitive
+  substring search. Every term must hit in some column — the behaviour of a
+  search box that narrows as words are added.
+
+  ```ruby
+  Repository.search([:name, :email], %w[john smith]).all
+  ```
+
+  LIKE metacharacters are escaped, columns may be join-qualified, and it only
+  adds a predicate, so it composes with keyset pagination.
+
+- **Projecting joined columns** — `#select({ table => column })` appends a
+  joined table's column to the result alongside `"base".*` (`#join` alone
+  stays filter/sort-only). Alias form `{ table => { column => alias } }`;
+  rows come back as raw hashes. A projected column colliding with a base
+  column raises when the rows come back — alias to disambiguate.
+
+- **Projections** — a declared catalog of computed columns, opted into per
+  read; never evaluated unless asked, so cost stays a visible per-read
+  decision.
+
+  ```ruby
+  extend PGI::Dataset[DB, :teams,
+                      projections: { mates_count: "SELECT COUNT(*) FROM teammates WHERE team_id = teams.id" }]
+
+  TeamRepository.page(nil, 25, project: [:mates_count])
+  ```
+
+  Unknown names raise at `#project`; `#count` ignores projections; writes shed
+  projected attribute keys so model round-trips just work.
+
+- **Collation** — `#order(:name, :asc, collate: "da-x-icu")` and
+  `#page(..., collate:)` sort text in a named (e.g. ICU) collation, so Danish
+  `æ ø å` file after `z` regardless of the database default.
+
+- **Server notices route to the configured logger** instead of libpq's stderr,
+  on both constructor doors (`conn_uri:` and `conn:`). Severity is preserved:
+  `RAISE WARNING` logs at `warn`, everything else at `debug` — a logger at
+  INFO stays quiet through chatter but still surfaces warnings.
+
 ## 1.0.0
 
 First public release.

data/README.md

@@ -1,15 +1,18 @@
 # PGI
 
 PGI is a simple and convenient interface for PostgreSQL with a few enhancements.
+It gives you pooled, self-healing connections (`PGI::DB`), a super lightweight
+repository toolkit (`PGI::Dataset`), and plain SQL migrations
+(`PGI::SchemaMigrator`) — and nothing else. No ActiveRecord, no DSL to learn on
+top of SQL you already know.
 
 ## PGI::DB
 
-The `PGI::DB` handles connections to a PostgreSQL databases. It features...
+`PGI::DB` handles connections to a PostgreSQL database. It features:
 
-* Connection Pool
-* Connection auto-healing capabilities
-
-Usage:
+* a connection pool
+* connection auto-healing: lost connections and pool checkout timeouts share a
+  retry budget, so a database restart is a pause, not a crash
 
 ```ruby
 DB = PGI::DB.configure do |options|
@@ -25,35 +28,57 @@ end
 DB.exec_stmt("my_stmt", "SELECT 1+1")
 ```
 
-## PGI::Dataset
+### Server notices go to your logger
 
-The `PGI::Dataset` is a super light weight ActiveRecord::Relation replacement. It delivers a clean and simple querying interface:
+Anything the server says on the side — a `RAISE NOTICE`, a `DROP CASCADE`'s
+chatter — goes to the configured logger instead of libpq's stderr default,
+and keeps its severity: a `RAISE WARNING` logs at `warn`, everything else at
+`debug`. A logger running at INFO stays quiet through routine chatter but
+still shows you warnings.
 
-* `#select(column1, ...)` allows you to limit the result set to only contain specified columns
-* `#where(...)` - can only be called once per query, so combine all conditions in a single call. Two forms:
-  * `#where("name = ? AND age > ?", ['joe', 21])` - a string clause with placeholders (`?` or `$1`)
-  * `#where(name: 'joe')` - as a Hash (multiple keys are concatenated with an ' AND ')
-* `#order(:column, <:asc|:desc>)` - sort result set by column and direction, can be invoked multiple times
-* `#limit(<num>)` - limits the result set to the specified number of records
-* `#first` - get the first record in a set
-* `#all`- get an array of records
-* `#count`- get the number of rows in a table
-* `#page(cursor, size, sort_by, sort_dir)` - keyset pagination; pass `nil` for the first page, then the **id of the last row** as the cursor for each subsequent page
+```ruby
+DB.exec_stmt("noisy", "DO $$ BEGIN RAISE WARNING 'heads up'; END $$")
+# => logger.warn("heads up")
+```
+
+## PGI::Dataset
+
+`PGI::Dataset` is a super lightweight `ActiveRecord::Relation` replacement.
+Extend a repository class with it and you get a clean querying interface:
 
 ```ruby
 class Repository
   extend PGI::Dataset[DB, :members, scope: "deleted_at IS NULL"]
 end
-```
 
-```sql
--- Each column used as sort_by in page() needs a composite index on (column, id).
--- Two separate single-column indexes are not sufficient — Postgres needs
--- the combined (column, id) ordering to seek directly to the cursor position.
-CREATE INDEX ON members (name ASC, id ASC);
-CREATE INDEX ON members (created_at ASC, id ASC);
+Repository.find(id)                     # one row by id
+Repository.where(name: "joe").all       # rows matching a condition
+Repository.page(nil, 20, :name, :asc)   # first page of 20, sorted by name
 ```
 
+The pieces:
+
+* `#select(column1, ...)` — limit the result set to the specified columns
+  (also appends **joined** columns, see [Projecting joined columns](#projecting-joined-columns))
+* `#where(...)` — can only be called once per query, so combine all conditions
+  in a single call. Two forms:
+  * `#where("name = ? AND age > ?", ['joe', 21])` — a string clause with placeholders (`?` or `$1`)
+  * `#where(name: 'joe')` — as a Hash (multiple keys are AND'ed together)
+* `#order(:column, <:asc|:desc>)` — sort by column and direction, can be invoked multiple times
+* `#limit(<num>)` — cap the number of rows
+* `#first` / `#all` — one row / all rows
+* `#count` — number of rows
+* `#page(cursor, size, sort_by, sort_dir)` — keyset pagination (see below)
+* `#join(table, on:)` — INNER JOIN for filtering and sorting ([Joins](#joins))
+* `#search(columns, terms)` — substring search across columns ([Search](#search))
+* `#project(*names)` — opt into declared computed columns ([Projections](#projections--computed-columns-you-opt-into))
+
+### Keyset pagination
+
+`#page` fetches rows at constant cost no matter how deep you page. The cursor
+is always the **id of the last row** from the previous page; pass `nil` for the
+first page.
+
 ```ruby
 # First page — sorted by name
 page1 = Repository.page(nil, 20, :name, :asc)
@@ -62,6 +87,15 @@ page1 = Repository.page(nil, 20, :name, :asc)
 page2 = Repository.page(page1.last["id"], 20, :name, :asc)
 ```
 
+Each column used as `sort_by` needs a **composite** index on `(column, id)` —
+two separate single-column indexes are not sufficient, because Postgres needs
+the combined ordering to seek directly to the cursor position:
+
+```sql
+CREATE INDEX ON members (name ASC, id ASC);
+CREATE INDEX ON members (created_at ASC, id ASC);
+```
+
 Generated SQL for page 2 (`sort_by != :id`):
 ```sql
 SELECT * FROM members
@@ -80,34 +114,208 @@ ORDER BY id ASC
 LIMIT 20
 ```
 
-### How keyset pagination works
+How it works:
+
+- **Sorting by id** — `WHERE id > $cursor ORDER BY id`. Simple seek on the
+  primary key index.
+- **Sorting by another column** — the composite subquery cursor above keeps
+  pages globally sorted: Postgres resolves the subquery via the primary-key
+  index (one fast lookup), then uses the composite `(sort_col, id)` index to
+  seek to that exact position and scan forward.
+- `LIMIT/OFFSET` scans and discards all prior rows on every page — cost grows
+  with depth. Keyset pagination does not.
+
+### Collation — locale-aware text ordering
+
+Text sorts in whatever collation you name, per read. Danish files
+`æ ø å` after `z`; the database's default (often `en_US`) files them wrong.
+
+```ruby
+Repository.order(:name, :asc, collate: "da-x-icu").all
+Repository.page(cursor, 20, :name, :asc, collate: "da-x-icu")
+```
+
+The named collation must exist in the database (Postgres ships ICU collations
+like `da-x-icu`; `und-x-icu` is a sane universal order). For `#page`, the
+composite index should be built with the same collation or Postgres falls back
+to sorting.
+
+### Joins
+
+`#join(table, on:)` adds an `INNER JOIN` so `#where` and `#order` (and keyset
+pagination) can reference the joined table's columns. Joins are for **filtering
+and sorting only, not projection** — the select list stays the base table's
+columns, so result rows still map to the base model unchanged.
+
+```ruby
+# Members that have an accepted membership in some account.
+# `on:` maps base-table column => joined-table column.
+Repository
+  .join(:memberships, on: { id: :member_id })
+  .where(memberships: { accepted: true })
+  .all
+```
+
+```sql
+SELECT "members".* FROM members
+INNER JOIN "memberships" ON "members"."id" = "memberships"."member_id"
+WHERE "memberships"."accepted" = true
+```
+
+A Hash value under a table-name key (`memberships: { accepted: true }`)
+qualifies its columns with that table — the key must be the base table or an
+already-joined table, never guessed. `#order` takes the same
+`{ table => column }` form to sort by a joined column.
+
+`#join` returns a `Query` yielding **raw row hashes** (like `#where`); model
+mapping is reserved for `Dataset` methods. For a paginated, model-mapped joined
+read, pass the `joins:` keyword to `#page`:
+
+```ruby
+# Page members sorted by their user's name.
+Repository.page(cursor, 20, { users: :name }, :asc,
+                joins: { memberships: { id: :member_id },
+                         users:       { { memberships: :user_id } => :id } })
+```
+
+`joins:` maps *joined table => on-mapping*. An on-key may itself be qualified
+(`{ { memberships: :user_id } => :id }`) to chain a **second hop** off an
+earlier join — join `memberships` to the base, then join `users` to
+`memberships`. The referenced table must already be joined, so **declare joins
+in dependency order** (an ordered Hash preserves it).
+
+Notes:
+
+- Keyset pagination over a joined sort column requires **at most one joined row
+  per base row** (e.g. `FK -> PK`); with 1:N joins the page boundaries are
+  ill-defined and the cursor lookup fails.
+- A `scope:` with unqualified columns becomes ambiguous once a join shares a
+  column name — qualify the scope's columns (e.g. `members.deleted_at IS NULL`).
+
+### Projecting joined columns
+
+`#join` is filter/sort-only — the projection stays `"base".*`. To also
+**return** a joined table's column, append it with `#select`:
+
+```ruby
+# Teams the base row belongs to, plus the roster row's `owner` flag.
+Repository.join(:teammates, on: { id: :team_id })
+          .join(:memberships, on: { { teammates: :membership_id } => :id })
+          .select({ teammates: :owner })
+          .where(memberships: { user_id: user_id })
+          .to_a
+# SELECT "teams".*, "teammates"."owner" FROM teams INNER JOIN ...
+```
+
+`#select` takes the same grammar as `#join`/`#where`: a bare column (qualified
+with the base table) or a `{ table => column }` pair. An alias form
+`{ table => { column => alias } }` renders `AS "alias"`.
+
+Because a projected joined column is by definition absent from the base model's
+schema, a `Query` carrying `#select` yields **raw row hashes only** — it never
+threads through `#page`/`#all` model mapping. `"base".*` is opaque (pgi has no
+schema introspection), so a joined column that shares a base column's name
+cannot be caught up front; it surfaces as a duplicate result field and `#to_a`/
+`#first`/`#each` **raise**. Pre-empt it with the alias form.
 
-`#page` fetches rows at constant cost regardless of page depth. The cursor is always the scalar `id` of the last row from the previous page.
+### Projections — computed columns you opt into
 
-- **Sorting by id** — `WHERE id > $cursor ORDER BY id`. Simple seek on the primary key index.
-- **Sorting by another column** — generates a composite subquery cursor so pages are globally sorted:
-  ```sql
-  WHERE (sort_col, id) > (SELECT sort_col, id FROM table WHERE id = $cursor)
-  ORDER BY sort_col, id
-  ```
-  Postgres resolves the subquery via the primary-key index (a single fast lookup), then uses the composite `(sort_col, id)` index to seek directly to that position and scan forward. The two separate single-column indexes are not equivalent — a composite B-tree index is required so that Postgres can seek to an exact `(sort_col, id)` position rather than scanning and filtering.
+`projections:` declares named computed columns when the dataset is extended —
+raw SQL you author, like `scope:`, never request data. Declaring costs
+nothing: a projection is only evaluated when a read **asks for it**, so the
+cost of a computed column stays a visible, per-read decision.
 
-`LIMIT/OFFSET` scans and discards all prior rows on every page request — cost grows linearly with depth. Keyset pagination does not.
+```ruby
+class TeamRepository
+  extend PGI::Dataset[DB, :teams,
+                      scope: "deleted_at IS NULL",
+                      projections: { mates_count: "SELECT COUNT(*) FROM teammates WHERE team_id = teams.id" }]
+end
+
+TeamRepository.page(nil, 25, project: [:mates_count])     # the list pays for what it shows
+TeamRepository.project(:mates_count).where(id: id).first  # chain form (raw rows)
+TeamRepository.find(id)                                   # pays nothing
+```
+
+Notes:
+
+- **Opt-in, never ambient** — an unprojected read carries no projection keys
+  and pays no projection cost. Opting into an undeclared name raises.
+  `#count` ignores projections (an aggregate has no row to enrich).
+- **Writes shed projection keys** — a model round-trip (find → to_h → update)
+  may carry projected attributes; INSERT/UPDATE drop them silently. Write
+  RETURNING never projects: presence of a projected key means "this read chose
+  to know".
+- **Cost when opted in**: evaluated per *output* row — after WHERE/LIMIT — so
+  a paginated read pays `page_size × subquery`. With an indexed correlate
+  (e.g. `teammates(team_id)`) that is an index-only probe per row. Sorting or
+  filtering **on** a projection promotes evaluation to the whole scope; that
+  EXPLAIN is the caller's to own. If a projection ever measures hot, the
+  escalation is a trigger-maintained column, not a cleverer query.
+- A projection name colliding with a base column will overwrite it in the row
+  hash — pick names that cannot collide (`mates_count`, not `count`).
+
+### Search
+
+`#search(columns, terms)` adds a case-insensitive substring search and AND's it
+into the WHERE clause. Each term becomes an OR-group of `ILIKE` matches across
+every column — a term hits when **any** column contains it, and a row matches
+only when **every** term hits somewhere. That is the behaviour of a search box
+that narrows as words are added: "john smith" finds the row named
+"Smith, John". Tokenising the query string is the caller's job; pass the terms
+as an array.
+
+```ruby
+# Chain form:
+Repository.search([:name, :email], %w[john smith]).all
+
+# Paginated, across a join:
+Repository.page(cursor, 20, :name, :asc,
+                search: { columns: [{ users: :name }, { users: :email }],
+                          terms:   %w[john smith] },
+                joins:  { users: { user_id: :id } })
+```
+
+LIKE metacharacters (`% _ \`) in a term are escaped so they match literally;
+blank terms are dropped. Columns take the same `{ table => column }` grammar as
+`#where`, so a search spans the same joins. It only adds a predicate, so it is
+**keyset-compatible** — the cursor stays on the sort column.
 
 ### Constraints
 
-- **`sort_by` columns must be `NOT NULL`.** SQL row comparison with NULL yields NULL, so rows with a NULL sort value are silently excluded from every cursor page — and if the anchor row itself has a NULL sort value, the next page comes back empty mid-stream.
-- **Hard-deleting an anchor row ends that pagination sequence.** The anchor lookup is by primary key; if the row is gone, the next page is empty and indistinguishable from the end of the result set. Soft deletion via a `scope:` (e.g. `deleted_at IS NULL`) is safe — the anchor lookup deliberately bypasses the scope, so a row that left the scope between pages still anchors correctly.
-- **Every table is expected to have a unique, totally ordered `id`** (SERIAL, UUIDv7, ...) — it is the tie-breaker that makes pages deterministic, and the whole `Dataset` interface assumes it.
+- **`sort_by` columns must be `NOT NULL`.** SQL row comparison with NULL yields
+  NULL, so rows with a NULL sort value are silently excluded from every cursor
+  page — and if the anchor row itself has a NULL sort value, the next page
+  comes back empty mid-stream.
+- **Hard-deleting an anchor row ends that pagination sequence.** The anchor
+  lookup is by primary key; if the row is gone, the next page is empty and
+  indistinguishable from the end of the result set. Soft deletion via a
+  `scope:` (e.g. `deleted_at IS NULL`) is safe — the anchor lookup deliberately
+  bypasses the scope, so a row that left the scope between pages still anchors
+  correctly.
+- **Every table is expected to have a unique, totally ordered `id`** (SERIAL,
+  UUIDv7, ...) — it is the tie-breaker that makes pages deterministic, and the
+  whole `Dataset` interface assumes it.
+
+## PGI::SchemaMigrator
+
+Plain up/down SQL migrations tracked in a `schema_migrations` table — no DSL,
+the migration *is* the SQL.
 
-## Documentation
+## Development
 
 Dependencies:
 
 * https://github.com/ged/ruby-pg
 * https://github.com/mperham/connection_pool
 
-Create developer/test DB:
+Run the test suite (rubocop + specs, against a containerized Postgres):
+
+```
+podman compose run --rm ruby
+```
+
+Or create a developer/test DB by hand:
 
 ```
 sudo su - postgres

data/VERSION

@@ -1 +1 @@
-1.0.0
+1.1.0

data/lib/pgi/connection.rb

@@ -25,6 +25,18 @@ module PGI
         new_conn.type_map_for_results = PG::BasicTypeMapForResults.new(new_conn, registry: regi)
         new_conn.type_map_for_queries = PG::BasicTypeMapForQueries.new(new_conn, registry: regi)
       end || raise("no connection provided")
+
+      # Server notices (a DROP CASCADE's chatter, a RAISE NOTICE) route to
+      # the configured logger instead of libpq's stderr default: a library
+      # that accepts a logger must not print around it - whichever door the
+      # connection came through. Severity survives the trip, so a RAISE
+      # WARNING still reaches a consumer whose logger runs at INFO.
+      @conn.set_notice_receiver do |result|
+        severity = result.result_error_field(PG::PG_DIAG_SEVERITY_NONLOCALIZED) ||
+                   result.result_error_field(PG::PG_DIAG_SEVERITY)
+        message  = result.result_error_field(PG::PG_DIAG_MESSAGE_PRIMARY) || result.error_message.strip
+        severity == "WARNING" ? @logger&.warn(message) : @logger&.debug(message)
+      end
     end
 
     # Execute a prepared statement. Statements are auto-created with fallback to exec_params

data/lib/pgi/dataset/query.rb

@@ -11,6 +11,9 @@ module PGI
       # @param command [String] the command part of the query (default: `SELECT * FROM <table>`)
       # @param options [Hash] hash of options: scope, where, params, limit, order, returning
       # @return [Query] new instance of Query
+      TABLE_NAME     = /\A[a-z_][a-z0-9_]*\z/
+      COLLATION_NAME = /\A[A-Za-z0-9_-]+\z/
+
       def initialize(database, table, command, **options)
         @database  = database
         @table     = table
@@ -21,6 +24,94 @@ module PGI
         @order     = options.fetch(:order, {})
         @limit     = options.fetch(:limit, 10)
         @returning = options.fetch(:returning, nil)
+        # Declared computed columns (see Dataset projections:) - the catalog
+        # a read may opt into via #project. Never evaluated unless asked:
+        # cost is a visible, per-read decision.
+        @projections = options.fetch(:projections, {})
+        @projected   = []
+        @joins       = []
+        @join_tables = []
+        @select      = []
+      end
+
+      # Opt into declared projections for THIS read: the named computed
+      # columns join the select list as (expr) AS name. Names must be
+      # declared on the dataset (the trust boundary stays at extension
+      # time); unknown names raise.
+      #
+      # @param names [Array<Symbol>] declared projection names
+      # @return [Query] the Query instance (for method chaining)
+      def project(*names)
+        unknown = names.map(&:to_sym) - @projections.keys.map(&:to_sym)
+        raise "Unknown projection(s): #{unknown.inspect}" unless unknown.empty?
+
+        @projected |= names.map(&:to_sym)
+        self
+      end
+
+      # Adds an INNER JOIN so WHERE/ORDER BY/keyset can reference the joined
+      # table's columns. Joins are for filtering and sorting, not projection:
+      # the select list stays the base table's columns, so result rows map to
+      # the base model unchanged. Identifiers only - never parameterized.
+      #
+      # An on-mapping key is a bare column (qualified with the base table) or a
+      # { table => column } pair that qualifies it with a previously joined
+      # table - the same grammar #order/#where accept. That is what enables a
+      # second hop: join A to the base, then join B to A. The referenced table
+      # must already be joined, so declare joins in dependency order.
+      #
+      # Notes:
+      # - a Dataset :scope with unqualified columns becomes ambiguous once a
+      #   join shares a column name - qualify scope columns to combine them
+      # - keyset pagination over a joined sort column requires at most one
+      #   joined row per base row (e.g. FK -> PK); with 1:N joins page
+      #   boundaries are ill-defined and the cursor lookup will fail
+      #
+      # @param table [Symbol] the table to join
+      # @param on [Hash] key column(s) => joined-table column(s); a key may be a
+      #   bare base-table column or a { table => column } pair naming the base
+      #   table or an already-joined table
+      # @raise [RuntimeError] if the table name or on-mapping is invalid
+      # @return [Query] return the Query instance (for method chaining)
+      def join(table, on:, type: :inner)
+        raise "Invalid JOIN table: #{table.inspect}" unless table.to_s.match?(TABLE_NAME)
+        raise "JOIN on: must map base column(s) to joined column(s)" unless on.is_a?(Hash) && !on.empty?
+        raise "Invalid JOIN type: #{type.inspect}" unless %i[inner left].include?(type)
+
+        conditions = on.map do |base_col, joined_col|
+          "#{qualified_column(base_col)} = #{Utils.sanitize_column(joined_col, table)}"
+        end.join(" AND ")
+
+        @join_tables << table.to_sym
+        @joins << %(#{type == :left ? "LEFT" : "INNER"} JOIN "#{table}" ON #{conditions})
+        self
+      end
+
+      # Appends columns to the projection so a joined read can return a joined
+      # table's columns alongside the base table's `"base".*` - the one thing
+      # #join deliberately does not do (it is filter/sort only). Because the
+      # projected column is by definition absent from the base model's schema,
+      # a Query with #select yields RAW row hashes only; it never threads
+      # through #page/#all model mapping. Identifiers only - never parameterized.
+      #
+      # Columns take the same grammar as #join/#where/#order plus an alias form:
+      # - a bare column               -> qualified with the base table
+      # - a { table => column } pair  -> qualified with the base or a joined
+      #                                  table (which must already be joined)
+      # - a { table => { column => alias } } pair -> the above, AS "alias"
+      #
+      # "base".* is opaque (pgi has no schema introspection), so a joined column
+      # sharing a base column's name cannot be caught here - it surfaces as a
+      # duplicate field name when the rows come back, where #first/#to_a/#each
+      # RAISE rather than silently clobber. Pre-empt it with the alias form.
+      #
+      # @param columns [Array<Symbol, Hash>] columns to append to the projection
+      # @raise [RuntimeError] if a column reference or alias mapping is malformed
+      #   or names an unknown table
+      # @return [Query] return the Query instance (for method chaining)
+      def select(*columns)
+        columns.each { |column| @select << projected_column(column) }
+        self
       end
 
       # Adds a WHERE clause to the query - can only be called once per query,
@@ -38,12 +129,28 @@ module PGI
 
         case clause
         when Hash
-          clause = clause.map do |k, v|
-            @params << v
-            "#{Utils.sanitize_columns(k, @table).first} = $#{@params.size}"
+          # A Hash value under a table-name key qualifies its columns with that
+          # table: where(account_id: 1, users: { name: "x" }). The key must be
+          # the base table or a joined table - never guessed - which fences the
+          # namespace for possible future non-table Hash semantics (e.g. JSONB).
+          clause = clause.flat_map do |k, v|
+            if v.is_a?(Hash)
+              assert_known_table!(k)
+              v.map do |col, val|
+                @params << val
+                "#{Utils.sanitize_column(col, k)} = $#{@params.size}"
+              end
+            else
+              @params << v
+              ["#{Utils.sanitize_column(k, @table)} = $#{@params.size}"]
+            end
           end.join(" AND ")
         when String
-          raise "Use placeholders in WHERE clause" if clause =~ /=(?!\s*[?$])/
+          # The guard lints against inlined VALUES - quoted strings and bare
+          # numbers, the injection surface. Identifiers (join conditions,
+          # subqueries), keywords (true/NULL) and constructs like = ANY($n)
+          # are legitimate parameterized SQL and pass (issue #19).
+          raise "Use placeholders in WHERE clause" if clause =~ /[=<>]\s*-?\s*['0-9]/
 
           offset = @params.size
           @params += params
@@ -57,16 +164,53 @@ module PGI
         self
       end
 
+      # Adds a case-insensitive substring search and AND's it into the WHERE
+      # clause. Each term becomes an OR-group of ILIKE matches across every
+      # given column; the groups are AND'ed together. So a term hits when ANY
+      # column contains it, and a row matches only when EVERY term hits
+      # somewhere - the shape of a search box that narrows as words are added.
+      # Each term binds a single %term% parameter, shared by its OR-group.
+      #
+      # LIKE metacharacters (% _ \) in a term are escaped so they match
+      # literally. Blank terms are dropped; empty columns or terms are a no-op.
+      # Like #keyset, this combines with an existing WHERE (call it after
+      # #where), so it does not raise the "already set" guard.
+      #
+      # Columns take the same grammar as #where/#order - a bare column
+      # (qualified with the base table) or a { table => column } pair naming the
+      # base table or an already-joined table - so a search can span joins.
+      #
+      # @param columns [Array] the text columns to match against
+      # @param terms [Array<String>] search terms, already tokenised by the caller
+      # @return [Query] return the Query instance (for method chaining)
+      def search(columns, terms)
+        columns = Array(columns)
+        terms   = Array(terms).reject { |t| t.to_s.strip.empty? }
+        return self if columns.empty? || terms.empty?
+
+        groups = terms.map do |term|
+          @params << "%#{escape_like(term)}%"
+          placeholder = "$#{@params.size}"
+          "(#{columns.map { |col| "#{qualified_column(col)} ILIKE #{placeholder}" }.join(" OR ")})"
+        end.join(" AND ")
+
+        @where = @where ? "#{groups} AND (#{@where})" : groups
+        self
+      end
+
       # Adds a ORDER BY clause to the query - suports multiple calls to the method
       #
-      # @param column [Symbol] the columns
+      # @param column [Symbol, Hash] the column - a single-pair Hash qualifies
+      #   it with a joined table: { users: :name }
       # @param direction [Symbol] the direction the sort should take - can be either `:desc` or `:asc`
+      # @param collate [String, nil] collation for text ordering, e.g. "da-x-icu"
+      #   (policy - which locale maps to which collation - belongs to the caller)
       # @raise [RuntimeError] if the direction param is invalid
       # @return [Query] return the Query instance (for method chaining)
-      def order(column, direction = :asc)
+      def order(column, direction = :asc, collate: nil)
         raise "Invalid ORDER BY direction: #{direction.inspect}" unless %i[asc desc].include?(direction)
 
-        @order[Utils.sanitize_columns(column, @table)] = direction.to_s.upcase
+        @order[[collated_column(column, collate)]] = direction.to_s.upcase
         self
       end
 
@@ -93,22 +237,30 @@ module PGI
       # @param sort_by [Symbol] the sort column
       # @param cursor_id [*, nil] id of the last row from the previous page, or nil for the first page
       # @param sort_dir [Symbol] :asc or :desc
+      # @param collate [String, nil] collation for the sort column. Must be the
+      #   same everywhere the column orders or compares - ORDER BY, the cursor
+      #   tuple and the cursor subselect all carry it, or page boundaries drift
       # @return [Query] return the Query instance (for method chaining)
-      def keyset(sort_by, cursor_id, sort_dir)
-        order(sort_by, sort_dir)
-        order(:id, sort_dir) unless sort_by.to_sym == :id
+      def keyset(sort_by, cursor_id, sort_dir, collate: nil)
+        sort_on_id = !sort_by.is_a?(Hash) && sort_by.to_sym == :id
+        order(sort_by, sort_dir, collate: (collate unless sort_on_id))
+        order(:id, sort_dir) unless sort_on_id
         return self unless cursor_id
 
         op     = sort_dir == :asc ? ">" : "<"
-        id_col = Utils.sanitize_columns(:id, @table).first
+        id_col = Utils.sanitize_column(:id, @table)
         @params << cursor_id
 
         clause =
-          if sort_by.to_sym == :id
+          if sort_on_id
             "#{id_col} #{op} $#{@params.size}"
           else
-            sort_col = Utils.sanitize_columns(sort_by, @table).first
-            "(#{sort_col}, #{id_col}) #{op} (SELECT #{sort_col}, #{id_col} FROM #{@table} WHERE #{id_col} = $#{@params.size})"
+            # The cursor row's sort value must be resolved through the same
+            # FROM (incl. joins) as the outer query, or a joined sort column
+            # would not exist in the subselect.
+            sort_col = collated_column(sort_by, collate)
+            from     = ["FROM #{@table}", *@joins].join(" ")
+            "(#{sort_col}, #{id_col}) #{op} (SELECT #{sort_col}, #{id_col} #{from} WHERE #{id_col} = $#{@params.size})"
           end
 
         @where = @where ? "#{clause} AND (#{@where})" : clause
@@ -124,6 +276,23 @@ module PGI
         scope << " AND " if scope && @where
 
         command = @command.dup
+        if command.start_with?("SELECT") && (@joins.any? || @select.any?)
+          # Joins are filter/sort-only: qualify the default star so joined
+          # columns never leak into result rows (and never collide). #select
+          # then appends its explicitly-projected joined columns onto it.
+          extra = @select.empty? ? "" : ", #{@select.join(", ")}"
+          if command == "SELECT * FROM #{@table}"
+            command = %(SELECT "#{@table}".*#{extra} FROM #{@table})
+          elsif @select.any?
+            command = command.sub(" FROM #{@table}", "#{extra} FROM #{@table}")
+          end
+          command << " #{@joins.join(" ")}" if @joins.any?
+        end
+        if @projected.any? && command.start_with?("SELECT") && command.include?(" FROM #{@table}")
+          # Additive: base columns plus the OPTED-IN computed columns
+          fragments = Utils.projection_fragments(@projections.slice(*@projected)).join(", ")
+          command = command.sub(" FROM #{@table}", ", #{fragments} FROM #{@table}")
+        end
         command << " WHERE #{scope}#{@where}" if @where || scope
         command << " ORDER BY #{Array(@order).map { |x| x.join(" ") }.join(", ")}" unless @order.empty?
         command << " LIMIT #{@limit}" if @limit
@@ -141,25 +310,19 @@ module PGI
       # @return [Hash]
       def first
         limit(1)
-        @database
-          .exec_stmt(Utils.stmt_name(@table, sql), sql, params)
-          .first
+        result.first
       end
 
       # Get all the records in a result set
       #
       # @return [Array] Array of records as Hashes
       def to_a
-        @database
-          .exec_stmt(Utils.stmt_name(@table, sql), sql, params)
-          .to_a
+        result.to_a
       end
 
       # Loop through records in a result set
       def each(&)
-        @database
-          .exec_stmt(Utils.stmt_name(@table, sql), sql, params)
-          .each(&)
+        result.each(&)
       end
 
       # Explain some query
@@ -189,6 +352,8 @@ module PGI
       def count
         @command = "SELECT COUNT(*) FROM #{@table}"
         @order   = {}
+        @select    = [] # projection is irrelevant to a COUNT (and would corrupt it)
+        @projected = [] # likewise: an aggregate has no row to enrich
         first&.fetch("count", 0)
       end
 
@@ -199,6 +364,96 @@ module PGI
         "#<PGI::Dataset::Query:#{object_id} @sql=#{sql} @params=#{params}>"
       end
       alias inspect to_s
+
+      private
+
+      # Run the query and guard against a projected-column name collision
+      # before the rows are handed back (see #select).
+      #
+      # @return [PG::Result]
+      def result
+        res = @database.exec_stmt(Utils.stmt_name(@table, sql), sql, params)
+        assert_projection_unambiguous!(res)
+        res
+      end
+
+      # Raise if #select projected two columns onto the same result field name
+      # (a joined column clashing with a base column, or two joined columns) -
+      # the row hash would silently keep only the last, losing data. Only
+      # relevant when #select added columns; a plain base.* read cannot clash.
+      #
+      # @param result [PG::Result]
+      # @raise [RuntimeError] listing the duplicated field name(s)
+      def assert_projection_unambiguous!(result)
+        return if @select.empty?
+
+        dups = result.fields.tally.select { |_, n| n > 1 }.keys
+        return if dups.empty?
+
+        raise "Ambiguous projected column(s): #{dups.join(", ")} - alias with " \
+              "select(table => { column: :alias }) to disambiguate"
+      end
+
+      # Render a #select column: a bare column (base table), a { table => column }
+      # pair (qualified), or a { table => { column => alias } } pair (qualified,
+      # AS "alias").
+      #
+      # @param column [Symbol, Hash] the column reference
+      # @raise [RuntimeError] if the Hash form is malformed or names an unknown table
+      # @return [String] the sanitized projection fragment
+      def projected_column(column)
+        return qualified_column(column) unless column.is_a?(Hash) && column.values.first.is_a?(Hash)
+
+        raise "Aliased column must be a single { table => { column => alias } } pair" unless column.size == 1
+
+        table, mapping = column.first
+        raise "Aliased column must map a single column to a single alias" unless mapping.size == 1
+
+        col, as = mapping.first
+        assert_known_table!(table)
+        "#{Utils.sanitize_column(col, table)} AS #{Utils.sanitize_column(as)}"
+      end
+
+      # Sanitize a column reference: a bare column qualifies with the base
+      # table; a single-pair Hash ({ users: :name }) qualifies with that table,
+      # which must be the base table or a joined table.
+      #
+      # @param column [Symbol, Hash] column or { table => column }
+      # @raise [RuntimeError] if the Hash form is malformed or names an unknown table
+      # @return [String] sanitized, qualified column
+      def qualified_column(column)
+        return Utils.sanitize_column(column, @table) unless column.is_a?(Hash)
+
+        raise "Qualified column must be a single { table => column } pair" unless column.size == 1
+
+        table, col = column.first
+        assert_known_table!(table)
+        Utils.sanitize_column(col, table)
+      end
+
+      # Escape LIKE/ILIKE metacharacters (\ % _) so a search term matches
+      # literally. Backslash is Postgres' default LIKE escape character, so no
+      # ESCAPE clause is needed - the escaped value binds straight as a param.
+      def escape_like(term)
+        term.to_s.gsub(/[\\%_]/) { |c| "\\#{c}" }
+      end
+
+      def assert_known_table!(table)
+        return if table.to_sym == @table.to_sym || @join_tables.include?(table.to_sym)
+
+        raise "Unknown table #{table.inspect} - qualify only the base table or joined tables"
+      end
+
+      # A column reference with an optional COLLATE. Collation names are
+      # identifiers (e.g. "da-x-icu"), validated and quoted - never params.
+      def collated_column(column, collate)
+        col = qualified_column(column)
+        return col unless collate
+
+        raise "Invalid collation: #{collate.inspect}" unless collate.to_s.match?(COLLATION_NAME)
+
+        %(#{col} COLLATE "#{collate}")
+      end
     end
   end
 end

data/lib/pgi/dataset/utils.rb

@@ -29,6 +29,16 @@ module PGI
           "#{table}_#{Digest::MD5.hexdigest(sql)}"
         end
 
+        # Build "(expr) AS name" fragments from a projections declaration.
+        # Names pass the column sanitizer; expressions are TRUSTED raw SQL -
+        # authored at dataset-extension time like :scope, never request data.
+        #
+        # @param projections [Hash] name => SQL expression
+        # @return [Array<String>] fragments ready for a select/RETURNING list
+        def projection_fragments(projections)
+          projections.map { |name, expr| "(#{expr}) AS #{sanitize_column(name)}" }
+        end
+
         # Get a sanitized column name(s)
         #
         # @param columns [String|Array] the column name(s) to sanitize

data/lib/pgi/dataset.rb

@@ -21,6 +21,41 @@ module PGI
       Query.new(@database, @table, nil, **@options).where(*)
     end
 
+    # Start a query with an INNER JOIN so WHERE/ORDER BY can reference the
+    # joined table's columns (filtering and sorting only - result rows stay
+    # base-table rows). Like #where, the returned Query yields raw row hashes;
+    # model mapping is the privilege of Dataset methods - for a paginated,
+    # model-mapped joined read use #page with the joins: keyword.
+    #
+    # @param table [Symbol] the table to join
+    # @param on [Hash] base-table column(s) => joined-table column(s)
+    # @return [Query]
+    def join(table, on:, type: :inner)
+      Query.new(@database, @table, nil, **@options).join(table, on: on, type: type)
+    end
+
+    # Start a query with a case-insensitive substring search across the given
+    # columns (see Query#search). Like #where/#join the returned Query yields
+    # raw row hashes; for a paginated, model-mapped search use #page with the
+    # search: keyword.
+    #
+    # @param columns [Array] the text columns to match against
+    # @param terms [Array<String>] search terms, already tokenised by the caller
+    # @return [Query]
+    def search(columns, terms)
+      Query.new(@database, @table, nil, **@options).search(columns, terms)
+    end
+
+    # Start a query with declared projections opted in (see Query#project).
+    # Like #where, yields raw row hashes; for a paginated, model-mapped
+    # projected read use #page with the project: keyword.
+    #
+    # @param names [Array<Symbol>] declared projection names
+    # @return [Query]
+    def project(*names)
+      Query.new(@database, @table, nil, **@options).project(*names)
+    end
+
     # Insert new row
     #
     # @param args [Hash|Object] row data
@@ -121,10 +156,33 @@ module PGI
     # @param size [Integer] number of rows per page
     # @param sort_by [Symbol] column to sort by
     # @param sort_dir [Symbol] :asc or :desc
+    # Joins (filter/sort only - result rows stay base-table rows) are passed
+    # as data: joins: { users: { user_id: :id } } maps joined table => on
+    # mapping, enabling qualified where ({ users: { name: "x" } }) and a
+    # qualified sort_by ({ users: :name }). Keyset over a joined sort column
+    # requires at most one joined row per base row (e.g. FK -> PK). An on-key
+    # may itself be qualified ({ { memberships: :user_id } => :id }) to chain a
+    # second hop off an earlier join; declare the joins in dependency order (an
+    # ordered Hash preserves it).
+    #
     # @param where [Array] optional WHERE clause forwarded to Query#where
+    # @param joins [Hash] joined table => on-mapping, forwarded to Query#join
+    # @param search [Hash, nil] { columns:, terms: } forwarded to Query#search -
+    #   a substring search AND'ed into the WHERE, keyset-compatible (it only
+    #   adds a predicate; the cursor stays on the sort column). Columns may be
+    #   qualified with a joined table, so a search can span the same joins.
+    # @param collate [String, nil] collation for the sort column (e.g.
+    #   "da-x-icu"), forwarded to Query#keyset
     # @return [Array] list of Models or Hashes
-    def page(cursor = nil, size = 10, sort_by = :id, sort_dir = :asc, *where)
-      _to_models Query.new(@database, @table, nil, **@options).where(*where).limit(size).keyset(sort_by, cursor, sort_dir).to_a
+    def page(cursor = nil, size = 10, sort_by = :id, sort_dir = :asc, *where, joins: {}, search: nil, collate: nil,
+             project: [])
+      query = Query.new(@database, @table, nil, **@options)
+      query.project(*project) if project.any?
+      joins.each { |table, on| query.join(table, on: on) }
+      query.where(*where)
+      query.search(search[:columns], search[:terms]) if search
+
+      _to_models query.limit(size).keyset(sort_by, cursor, sort_dir, collate: collate).to_a
     end
 
     private
@@ -136,7 +194,11 @@ module PGI
     # @param attributes [Hash] column => value
     # @return [Array(Array, Array, Array)] sanitized columns, placeholders, values
     def sql_params(attributes)
-      attrs = attributes.sort.to_h
+      # Projections are READ-ONLY facts computed by the dataset - they ride
+      # every read and RETURNING, so a model round-trip (find -> to_h ->
+      # update) naturally carries them; writes must shed them silently.
+      projections = @options.fetch(:projections, {})
+      attrs = attributes.reject { |k, _| projections.key?(k.to_sym) }.sort.to_h
       [Utils.sanitize_columns(attrs.keys), (1..attrs.size).map { |i| "$#{i}" }, attrs.values]
     end
 
@@ -160,6 +222,11 @@ module PGI
       def [](database, table, **options)
         raise "Invalid table name: #{table}" unless table.to_s =~ /\A[a-z_][a-z0-9_]*\z/
 
+        options.fetch(:projections, {}).each do |name, expr|
+          raise "Invalid projection name: #{name.inspect}" unless Utils.valid_column?(name) && name.to_s != "*"
+          raise "Invalid projection expression for #{name.inspect}" unless expr.is_a?(String) && !expr.strip.empty?
+        end
+
         mod = clone
         mod.instance_variable_set("@database", database)
         mod.instance_variable_set("@table", table)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgi
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Coherify