truffler 0.1.4 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3be74707295fff06a13ad0b12bbfb2d876f851893f0fff5f3f49b14c368bdb9b
-  data.tar.gz: 59a2e7d7e57669e8ffa35eadf656cc4e52fbae6cc47798f9ae4d01a43f870109
+  metadata.gz: 41cc5bfdb1f4f8a5b18ab9388518140bb5b2c927c87f115094c43ffa1c37659a
+  data.tar.gz: 11ee26029552cfaaa18d68fed182a2ede4641f9491c942eecbf6aa86326baef8
 SHA512:
-  metadata.gz: e71fd94ea16d17b9ca2e2f219591b51dfc94ab82f749bd21d5d277389039e6fa30f3c29130dd07c2b8aecd588abe0e1d91e8c6e70b66d199b485a77a90cc0a59
-  data.tar.gz: b3ea709b2d26f377b3c7731e7cc020d5249780bfbc756ad1f01dde3506a542f80b34bc7e8846387527d9311cbbe0015c826ea90916fdd251fdaa596efef123c6
+  metadata.gz: b1bc305d0c1745424ce0c55e905a6f592a423f37274edb420def42073119de9f089ac52fded8f0af0319327e9e7e3738b41e6d62fe77ec1d398d1f7c7bddd170
+  data.tar.gz: 8dd567f12f014c36ac80e900a87965390cbaae8f3c3e63e588cd6668510e68afc020f8d0039198fea2c13a98aabff2425b6a5eef839c5b3be64864ca8be1e868

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## [0.1.5]
+
+Fixes from the Cora integration at Postgres scale, and two from happyhappy's 0.1.4 upgrade.
+
+- Tenant-scoped indexing: `index_if ->(record) { ... }` and `index_scope ->(relation) { ... }` in `truffler do`, plus `config.tenant_enabled = ->(model, tenant_key) { ... }`. Records outside them, or in a disabled tenant, are never labeled, embedded or backfilled. The after-commit hooks, `Labeling::Queue`, the labeler, both backfills, lens backfills, `LabelFlushJob`, `ResumeJob` and `EmbedJob` all honor them. `BackfillJob` takes `tenant_key:`, over-cap demotion and `ResumeJob` enqueue one job per tenant, and `truffler:backfill` and `truffler:status` accept `TENANT=`.
+- `Embeddings::Backfill` pages per tenant with a limited `NOT EXISTS` anti-join instead of a whole-table `NOT IN`.
+- The backfill spend ledger is per tenant for tenant-scoped models (`truffler_backfill_spends.tenant_key`; `config.backfill_spend_cap_scope = :tenant`, the default, or `:app`), so `backfill_spend_cap` applies to each tenant. Existing installs: `rails g truffler:upgrade && rails db:migrate`.
+- `rails g truffler:upgrade` is safe to rerun. It skips migrations already present or applied and writes only the missing ones, with no conflicts.
+- Choice labels store a row only for options at or above `config.choice_min_probability` (default 0.05; `nil` stores all), plus the most likely option. Missing options read as 0.0 everywhere, and no migration is needed.
+- An out-of-shape `from:` answer settles: nothing is stored for that label, the record is marked labeled, and `truffler.supplied_label_failed` fires with `permanent: true`. A raising `from:` still retries and fails after `max_attempts`.
+- Postgres: upserts no longer assign `updated_at` twice (Rails 8.1 raised `PG::SyntaxError` on every live enqueue and embedding write). The suite now also runs on Postgres with pgvector in CI (`TRUFFLER_DATABASE_URL`). As a side effect, re-enqueuing an already pending row no longer refreshes its `updated_at`.
+- Keystroke SQL at scale on Postgres. A `keyword` or `exact` callable that returns a relation runs once as `id = ANY(ARRAY(subquery))` instead of a hashed `IN` over every tenant row; returning an Array of ids is the documented fast path. Label-only and filtered searches add up label scores in one grouped subquery joined on `record_id` instead of a subquery per row: 120 ms p50 before, 51 ms after, for 50k records. New installs get `INCLUDE (record_id)` on `index_truffler_labels_for_search`, and `truffler:upgrade` adds it on existing Postgres installs. On Postgres, `NeighborStore` scores text from the tenant's top-K neighbors (`ORDER BY embedding <=> q LIMIT k`, default 200); `NeighborStore.new(k:, top_k:)` is new, and `config.vector_store` accepts a store instance. Records outside the top `k` get no text score. The README covers `SET LOCAL jit = off` for the keystroke transaction.
+- New `invite_on_pending_encoding` declaration (default true): a model with a `keyword` source, such as a blind index, still shows the `:encoding_pending` Smart search row while a first-time query's encoding is in flight. `Result#local_weak?` keeps the backup provider off when the keyword list is strong.
+- New `Truffler::Clients::Evaluator` for clients that read `schema.questions` and return an evaluation object without `to_h`, such as Cora's `TypeSafeClient`. The Cora checklist in `docs/host-integration.md` now shows it instead of the broken `Callable` line.
+- A query word that names any declared label key, option key or option search-text word, applied or not, is never dropped as filler. `email` and `emails` are no longer default `config.filler_words`.
+- Removing the last label chip no longer makes filler nouns required keywords while a time phrase still applies.
+- A lens backfill never pages disabled tenants (filtered in SQL, so a capped `LensBackfillJob` cannot refetch them forever), and the labeler never deletes a disabled tenant's state rows; it returns them to pending at backfill priority, so re-enabling relabels only what is stale.
+- Removing chips keeps declared label and option words (option search text included) as keywords, the same as the encoder and the cold-cache path.
+- `truffler:status` and `Backfill.status(model, tenant_key:)` count only records inside `index_scope` and enabled tenants, and only their state rows, so a finished partial rollout reports nothing missing or pending.
+
 ## [0.1.4]
 
 Fixes from happyhappy production.

data/README.md

@@ -51,6 +51,10 @@ bin/rails db:migrate
 
 The migration creates only that table and skips it if it already exists. Until you run it, backfills log one warning and cap spend per run, as 0.1.1 did.
 
+0.1.5 adds `truffler_backfill_spends.tenant_key` for per-tenant spend ledgers. The same `truffler:upgrade` command writes that migration. Until you run it, every tenant shares the model's app-wide ledger and a warning is logged once.
+
+`truffler:upgrade` is safe to rerun. It skips any migration already in `db/migrate` or already applied to the database, and writes only the missing ones.
+
 Truffler digests user keys and query misses with `secret_key_base`. Rails supplies it automatically; outside Rails, set `config.secret_key_base`.
 
 ## Declaring a model
@@ -84,6 +88,7 @@ class Email < ApplicationRecord
     surface :inbox, explicit_action: :enter     # :enter, :key, or :row
     ranking label: 1.0, text: 1.0, keyword: 0.5, exact: 1.0
     weak_below 3                                # fewer keystroke results than this counts as weak
+    invite_on_pending_encoding true             # Smart search row while a first-time query's encoding is in flight
   end
 end
 ```
@@ -94,9 +99,10 @@ Here is what each option does:
 - Choice `options:` may be a callable of the tenant key, which gives each tenant its own vocabulary. A tenant it gives no options (`{}` or nil) simply lacks the label: it is neither asked nor encoded there. The option name `truffler:none` is reserved (see `Truffler::NO_OPTION` below).
 - An option's value may be `{ description: "...", search: "..." }` instead of a plain description, e.g. `options: { "prod-a1" => { description: "Cora, the AI email assistant that drafts replies...", search: "Cora email assistant" } }`. Jev labels records with the description; query words are matched against the short search text (else the description), and the request state's `option_names` carries it. A per-tenant callable may return the same shape. Editing a search text never stales labels or triggers a backfill; it only changes the query-encoding cache key, so queries are re-encoded.
 - `watch :column, ...` relabels every label when one of those columns changes. Saving a record only relabels on columns that changed, so a `reads` field backed by a method (a conversation built from messages, say) needs the columns it is built from in `watch`.
-- `keyword` also accepts a single callable, `->(scope, tokens) { relation }`.
+- `keyword` also accepts a single callable, `->(scope, tokens) { relation }`. It, and each `exact` callable, may return an Array of ids instead of a relation, which is far faster on a large tenant (see [Keystroke search at scale](#keystroke-search-at-scale)).
 - `embeddings column: :my_vector` searches a vector column you maintain yourself. Truffler never writes it.
 - `embeddings` refuses to send encrypted fields to the embedding provider unless you pass `allow_encrypted: true`.
+- `invite_on_pending_encoding` (default true) shows the Smart search row with reason `:encoding_pending` while a query's first encoding is in flight, even when a `keyword` source matched. A model with no `keyword` source always does. Pass `false` to invite only on weak or empty results.
 
 `label key, type, **options` takes:
 
@@ -149,7 +155,10 @@ end
 | `:choice` | one option string (meaning 1.0 for it), or `{ option => probability }` | one row per option, `label:option`; options left out store 0.0 |
 | `:score` | a level index into `legend:` | `index / (levels - 1)`, as for Jev scores |
 
-`nil` stores nothing: the record reads as missing that label, not as 0. An answer out of shape (an undeclared option, a probability outside 0..1, a level past the legend) or a `from:` that raises stores nothing for that label and emits `truffler.supplied_label_failed` with the label key and error class. Rows already stored keep serving until the next good write.
+`nil` stores nothing: the record reads as missing that label, not as 0. Both kinds of failure emit `truffler.supplied_label_failed` with the label key, the error class, and `permanent`:
+
+- An answer out of shape (an undeclared option, a probability outside 0..1, a level past the legend) is permanent (`permanent: true`). It is treated like `nil`: nothing is stored for that label, earlier rows are cleared, and the record is marked labeled, so backfills do not claim it again. A later valid answer, through `watch:` or `truffler_refresh_labels!`, is written as usual.
+- A `from:` that raises is transient (`permanent: false`). The record stays pending at backfill priority and is retried for free, then marked failed after `max_attempts`. Rows already stored keep serving until the next good write.
 
 Supplied labels never reach Jev. They are never in a labeling request, take no budget slot, cost nothing, and do not count toward a lens or backfill spend cap. A flush or backfill where only supplied labels are stale makes no Jev call. The labeling job writes them before it asks Jev anything, so a Jev outage or a budget denial never holds them back. Otherwise they behave like asked labels: same `truffler_labels` rows, the same filters, boosts, chips, and label vectors, and query encoding asks Jev how a query uses them (using `description:`), within the one encoding call per query.
 
@@ -168,11 +177,17 @@ The default client is `Truffler::Clients::RubyLLMTypeSafe`. It needs ruby_llm 2
 If you already have a TypeSafe client, wrap it in `Truffler::Clients::Callable`. The wrapped object must respond to `evaluate(state:, schema:)`, and may also accept `model:`. It returns either the answers hash or `{"answers" => ..., "model" => ..., "usage" => {"input_tokens" => ...}}`:
 
 ```ruby
-config.client = Truffler::Clients::Callable.new(TypeSafeClient.new)
+config.client = Truffler::Clients::Callable.new(MyJevClient.new)
 ```
 
 `schema:` holds the questions in TypeSafe wire shape. When a response carries no token count, Truffler estimates it from the request size and flags the estimate. Errors reach you as `Truffler::ClientError`, carrying only the HTTP status and the error class name.
 
+If your client takes a schema object (it calls `schema.questions`) and returns an evaluation object with `answers`, `model`, and `input_tokens` readers rather than a hash, use `Truffler::Clients::Evaluator` instead. For any other shape, subclass `Truffler::Clients::Base` and implement `perform` (see [docs/host-integration.md](docs/host-integration.md)).
+
+```ruby
+config.client = Truffler::Clients::Evaluator.new(TypeSafeClient.new)
+```
+
 ## Keystroke search
 
 ```ruby
@@ -184,7 +199,7 @@ result = Email.truffler(params[:q], tenant: Current.account.id, scope: Current.a
 
 A keystroke search makes no network call. It reads the query encoding and query vector from the cache. When the cache misses, it enqueues `EncodeQueryJob`, and the next keystroke or reload picks up the result.
 
-Query encoding sends Jev the label vocabulary (each label's description and a choice label's option names) next to the query, and asks for each label whether the query filters on it, prefers it, or ignores it, plus the role of each word. A choice label's option question also offers `Truffler::NO_OPTION` (`"truffler:none"`), meaning the query names none of its options, so a host option literally called `none` stays filterable. Word roles are then checked locally: a word that names a label the query applies (its key, a word of its key, or the chosen option key, ignoring case and plurals or sharing their first three letters when both words have four letters or more, so "angry" names `anger`; or, matched exactly, a word of that option's display name or description) counts as naming the label, and common stopwords and `config.filler_words` (generic nouns such as "customers", "users", "emails") are dropped. They are kept only when dropping them would leave no keyword, no applied label, and no time phrase, so "customers in the last 3 hours" lists the window's records while a lone "customers" still searches text; the same rule applies before the encoding is cached. When the encoding applies a label filter, the filter decides which records match and keyword hits only rank them; without one, the remaining keywords must match.
+Query encoding sends Jev the label vocabulary (each label's description and a choice label's option names) next to the query, and asks for each label whether the query filters on it, prefers it, or ignores it, plus the role of each word. A choice label's option question also offers `Truffler::NO_OPTION` (`"truffler:none"`), meaning the query names none of its options, so a host option literally called `none` stays filterable. Word roles are then checked locally: a word that names a label the query applies (its key, a word of its key, or the chosen option key, ignoring case and plurals or sharing their first three letters when both words have four letters or more, so "angry" names `anger`; or, matched exactly, a word of that option's display name or description) counts as naming the label, and common stopwords and `config.filler_words` (generic nouns such as "customers", "users", "items") are dropped. A word that names any declared label (a word of its key, of an option key, or of an option's search text), applied or not, is never dropped as filler. Filler is kept only when dropping them would leave no keyword, no applied label, and no time phrase, so "customers in the last 3 hours" lists the window's records while a lone "customers" still searches text; the same rule applies before the encoding is cached. When the encoding applies a label filter, the filter decides which records match and keyword hits only rank them; without one, the remaining keywords must match.
 
 When the searcher removes a chip, the words that named only that label (or only labels that are now all removed) become keywords again, so removing the `urgent` chip from "urgent refunds" searches for both words. A word that named a label or option key only by shared prefix ("urgently" for `urgent`) also adds a small keyword score (a quarter of the keyword weight) while its label applies, ranking records whose text contains it higher without requiring it.
 
@@ -195,6 +210,7 @@ The returned `Truffler::Search::Result` exposes:
 - `records` and `ids`.
 - `chips`: `[{key:, label:, kind: :filter | :boost | :time, name:}]`.
 - `invite_row`: `{query:, reason: :weak | :empty | :encoding_pending}` or nil.
+- `local_weak?`: whether the list is too weak to stand alone (starts the backup provider). A pending encoding on a model with a `keyword` source invites Smart search without making the list weak.
 - `encoding_status`: `:cached`, `:pending`, or `:none`.
 - `watermark` and `new_matches_count`.
 - `explicit_action`: the surface's declared action.
@@ -207,6 +223,24 @@ For the "N new matches" row on a later request, pass the watermark back in:
 Email.jev_new_matches_count(params[:q], tenant: account.id, scope: account.emails, user: current_user, since: Time.iso8601(params[:since]))
 ```
 
+### Keystroke search at scale
+
+The keystroke query is shaped so a large tenant (tens of thousands of records) stays fast on Postgres:
+
+- Return ids from `keyword` and `exact` callables when you can, e.g. `.limit(2_000).pluck(:id)` from a blind index. Ids become a literal `IN (...)` list. A returned relation is rendered as `id = ANY(ARRAY(subquery))` on Postgres (plain `IN (subquery)` elsewhere), so it runs once instead of as a hashed filter over every tenant row, but the id list is still faster.
+- Label-only and filtered searches, where every record past the tenant and filters is a candidate, read label scores from one grouped aggregate over `truffler_labels` joined on `record_id`, never a subquery per row. New installs get `index_truffler_labels_for_search` with `INCLUDE (record_id)`, which makes that aggregate an index-only scan. On an existing Postgres install, `bin/rails generate truffler:upgrade && bin/rails db:migrate` adds it (built concurrently, then the old index is dropped). It is optional: the grouped aggregate already replaces the per-row subquery, and the index only saves heap reads.
+- With embeddings on Postgres, text similarity comes from the tenant's top `k` neighbors (`ORDER BY embedding <=> q LIMIT k`), joined on `record_id`. An HNSW index on `truffler_embeddings.embedding` serves it. Records outside the top `k` score no text similarity; see `vector_store` under [Configuration](#configuration).
+- The plans these queries produce are costed high enough that Postgres JIT often compiles them, which can add 0.3 to 0.9 s to a query that runs in tens of milliseconds. Turn JIT off for the keystroke only:
+
+  ```ruby
+  result = Email.transaction do
+    Email.connection.execute("SET LOCAL jit = off")
+    Email.truffler(params[:q], tenant: account.id, scope: account.emails, user: current_user)
+  end
+  ```
+
+  `SET LOCAL` lasts until the transaction ends, so other queries keep JIT. `result.records` is loaded inside the block.
+
 ## Smart search and streaming
 
 The searcher's explicit action (the `explicit_action` their surface declares) starts a Smart run:
@@ -341,12 +375,12 @@ Set these in `Truffler.configure do |config| ... end`.
 | `max_field_chars`, `request_token_budget`, `max_questions_per_request` | 4,000, 48,000, 200 | Request packing limits. |
 | `queue_name` | `:default` | Queue for every Truffler job. |
 | `cost_per_million_tokens` | 0.042 | Jev input price, used in usage events and estimates. |
-| `backfill_spend_cap` | 5.0 | Dollar cap on backfill spend per model and vocabulary version, shared by `BackfillJob` chains, `ResumeJob` backfills, and `truffler:backfill` runs (see Backfill). `nil` disables it; for the rake task, `SPEND_CAP=none` does. Supplied labels cost nothing and are still written once it is reached. |
+| `backfill_spend_cap` | 5.0 | Dollar cap on backfill spend per model (per tenant for tenant-scoped models; see `backfill_spend_cap_scope`) and vocabulary version, shared by `BackfillJob` chains, `ResumeJob` backfills, and `truffler:backfill` runs (see Backfill). `nil` disables it; for the rake task, `SPEND_CAP=none` does. Supplied labels cost nothing and are still written once it is reached. |
 | `resume_pending_after` | 5 minutes | How long before `ResumeJob` treats work as stuck. |
 | `embedder` | `Embeddings::RubyLLMEmbedder.new` | Any `Embeddings::Embedder` subclass. The default calls `RubyLLM.embed`, which works on ruby_llm 1.x and 2. |
 | `embedding_cost_per_million_tokens` | 0.02 | Embedding price. |
-| `vector_store` | `:auto` | `:neighbor` (pgvector `<=>`, or sqlite-vec `vec_distance_cosine` when you load the extension), `:ruby` (exact cosine in Ruby), or `:auto` (neighbor when available, otherwise Ruby). A model declaring `embeddings column:` always reads its own column. |
-| `filler_words` | `customer(s) people person user(s) message(s) email(s) item(s) stuff thing(s)` | Generic nouns never required as keywords on their own (matched ignoring plurals). Replace the list or extend it (`config.filler_words += %w[ticket]`). |
+| `vector_store` | `:auto` | `:neighbor` (pgvector `<=>`, or sqlite-vec `vec_distance_cosine` when you load the extension), `:ruby` (exact cosine in Ruby), or `:auto` (neighbor when available, otherwise Ruby). Or a store instance, e.g. `Truffler::Embeddings::NeighborStore.new(k: 500)` or your own `VectorStore` subclass. On Postgres the neighbor store scores text from the tenant's top `k` (default 200) neighbors; `NeighborStore.new(top_k: false)` scores every row exactly. A model declaring `embeddings column:` always reads its own column. |
+| `filler_words` | `customer(s) people person user(s) message(s) item(s) stuff thing(s)` | Generic nouns never required as keywords on their own (matched ignoring plurals). Replace the list or extend it (`config.filler_words += %w[ticket]`). |
 | `encoding_prefetch` | `QueryEncoding::Prefetch.new` | Cache-miss hook, called as `call(model, query, cache_key:, tenant_key:, user_key:)`. |
 | `encoding_deadline` | 1.0 | Seconds a Smart run waits for an in-flight query encoding. |
 | `rerank_depth`, `rerank_chunk_size`, `rerank_max_field_chars` | 30, 10, 1,200 | Candidates reranked, candidates per Jev request, characters per field sent. |
@@ -358,6 +392,9 @@ Set these in `Truffler.configure do |config| ... end`.
 | `lenses.creators`, `lenses.authorize_lens`, `lenses.proposals` | `:developers`, nil, false | Lens policy (see Lenses). |
 | `lenses.spend_cap_usd`, `lenses.sample_size`, `lenses.max_questions`, `lenses.expire_after` | 1.0, 20, 8, 30 days | Lens limits. |
 | `lenses.generator`, `lenses.drafter_model`, `lenses.user_key` | `RubyLLMGenerator`, nil, `"User:42"` style | Drafting model seam and user key mapping. |
+| `tenant_enabled` | nil (every tenant) | `->(model, tenant_key) { ... }`, asked for tenant-scoped models only. A disabled tenant is never labeled, embedded, or backfilled (see Indexing only some tenants). |
+| `backfill_spend_cap_scope` | `:tenant` | `:tenant` keeps a spend ledger per tenant for tenant-scoped models, so `backfill_spend_cap` applies to each tenant. `:app` keeps one ledger per model. Unscoped models always use one. |
+| `choice_min_probability` | 0.05 | Choice labels store a row only for options at or above this probability, plus the most likely option. A missing option reads as 0.0 in filters, boosts, label vectors, and contributions. `nil` stores every option. No migration is needed: rows written earlier stay until their record is relabeled. |
 
 ## Jobs to schedule
 
@@ -396,10 +433,36 @@ truffler_expire_lenses:
 |---|---|
 | `SPEND_CAP=20` or `none` | Overrides `backfill_spend_cap` for this run. |
 | `MAX_DURATION=600` | Stops after that many seconds with `paused` and the cursor. Rerun to continue. |
-| `RESET_SPEND=1` | Zeroes the spend recorded for the current vocabulary version before starting. |
+| `RESET_SPEND=1` | Zeroes the spend recorded for the current vocabulary version before starting (every tenant's ledger when no `TENANT` is given). |
+| `TENANT=42` | Backfills only that tenant, against that tenant's ledger. `truffler:status` takes it too, to print that tenant's spend. |
 
 The spend cap holds across runs. Truffler records backfill spend per model and app-wide vocabulary version in `truffler_backfill_spends`, and reserves each request's estimate against the cap in SQL. A rerun, a `ResumeJob` backfill, and overlapping `BackfillJob` chains all draw on the same total, so together they stop at `backfill_spend_cap`. Changing the vocabulary (a reworded question, a new label, or an activated lens) starts a new total. Lens backfills are capped separately by each lens's `spend_cap_usd`. `BackfillJob` reschedules itself after a denial with the same backoff. In code, `Truffler::Labeling::Backfill.new(Email).run(wait: true, max_duration: 600)` does what the task does.
 
+For tenant-scoped models the ledger is per tenant (`backfill_spend_cap_scope :tenant`, the default), so the cap applies to each tenant. A whole-model run skips a tenant once its ledger reaches the cap, keeps labeling the others, and ends with `spend_cap_reached`. A tenant run (`TENANT=`, or `BackfillJob.perform_later("Email", tenant_key: "42")`) stops at that tenant's cap. `ResumeJob` and the flush job's over-cap demotion enqueue one `BackfillJob` per tenant.
+
+### Indexing only some tenants
+
+For a per-account rollout, or to leave some records out, declare which records Truffler indexes:
+
+```ruby
+truffler do
+  tenant :account_id
+  # ...
+  index_if ->(email) { !email.spam? }                  # single records: after-commit hooks, Queue, EmbedJob
+  index_scope ->(relation) { relation.where(spam: false) } # batch paths: backfills and sweeps
+end
+
+Truffler.configure do |config|
+  config.tenant_enabled = ->(model, tenant_key) { Account.search_enabled?(tenant_key) }
+end
+```
+
+- The after-commit label and embed hooks skip records that are not indexable: `index_if` rejects them or their tenant is disabled. No state row or job is created.
+- `Labeling::Backfill`, `Embeddings::Backfill`, and lens backfills page only over `index_scope` and skip disabled tenants. The labeler drops claimed records that are no longer indexable instead of labeling them.
+- `index_if` and `index_scope` should select the same records. Keep `index_scope` index-friendly, because backfills page over it.
+- The `ResumeJob` embedding sweep runs tenant by tenant over the enabled tenants that have records in `index_scope`. Each tenant pass is a `NOT EXISTS` anti-join limited to the sweep size, not a `NOT IN` over the table.
+- Stored labels of a tenant you disable stay in place and keep serving search. When you re-enable the tenant, run `truffler:backfill` with `TENANT=` for it.
+
 ## Privacy
 
 These guarantees hold for every model, and matter most for models that use Active Record encryption:

data/lib/generators/truffler/install/templates/migration.rb.tt

@@ -11,7 +11,7 @@ class CreateTrufflerTables < ActiveRecord::Migration<%= migration_version %>
     end
     add_index :truffler_labels, [:record_type, :record_id, :label_key], unique: true,
       name: "index_truffler_labels_on_record_and_label"
-    add_index :truffler_labels, [:record_type, :tenant_key, :label_key, :value],
+    add_index :truffler_labels, [:record_type, :tenant_key, :label_key, :value], include: [:record_id],
       name: "index_truffler_labels_for_search"
 
     create_table :truffler_record_states do |t|
@@ -108,12 +108,15 @@ class CreateTrufflerTables < ActiveRecord::Migration<%= migration_version %>
 
     create_table :truffler_backfill_spends do |t|
       t.string :record_type, null: false
+      t.string :tenant_key
       t.string :vocabulary_version, null: false
       t.float :spent_usd, null: false, default: 0.0
       t.integer :requests, null: false, default: 0
       t.timestamps
     end
-    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true,
-      name: "index_truffler_backfill_spends_on_ledger"
+    add_index :truffler_backfill_spends, [:record_type, :tenant_key, :vocabulary_version], unique: true,
+      name: "index_truffler_backfill_spends_on_tenant_ledger"
+    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true, where: "tenant_key IS NULL",
+      name: "index_truffler_backfill_spends_on_app_ledger"
   end
 end

data/lib/generators/truffler/upgrade/templates/backfill_spends_tenant_key_migration.rb.tt

@@ -0,0 +1,25 @@
+class AddTenantKeyToTrufflerBackfillSpends < ActiveRecord::Migration<%= migration_version %>
+  def up
+    add_column :truffler_backfill_spends, :tenant_key, :string unless column_exists?(:truffler_backfill_spends, :tenant_key)
+    if index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_ledger")
+      remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_ledger"
+    end
+    unless index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_tenant_ledger")
+      add_index :truffler_backfill_spends, [:record_type, :tenant_key, :vocabulary_version], unique: true,
+        name: "index_truffler_backfill_spends_on_tenant_ledger"
+    end
+    unless index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_app_ledger")
+      add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true, where: "tenant_key IS NULL",
+        name: "index_truffler_backfill_spends_on_app_ledger"
+    end
+  end
+
+  def down
+    execute "DELETE FROM truffler_backfill_spends WHERE tenant_key IS NOT NULL" if column_exists?(:truffler_backfill_spends, :tenant_key)
+    remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_tenant_ledger", if_exists: true
+    remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_app_ledger", if_exists: true
+    remove_column :truffler_backfill_spends, :tenant_key, if_exists: true
+    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true,
+      name: "index_truffler_backfill_spends_on_ledger", if_not_exists: true
+  end
+end

data/lib/generators/truffler/upgrade/templates/labels_search_covering_migration.rb.tt

@@ -0,0 +1,32 @@
+class CoverTrufflerLabelsForSearch < ActiveRecord::Migration<%= migration_version %>
+  disable_ddl_transaction!
+
+  COVERING = "index_truffler_labels_for_search_covering".freeze
+  COLUMNS = [ :record_type, :tenant_key, :label_key, :value ].freeze
+
+  def up
+    return unless postgres? && !covered?
+
+    add_index :truffler_labels, COLUMNS, include: [ :record_id ], name: COVERING, algorithm: :concurrently
+    remove_index :truffler_labels, name: "index_truffler_labels_for_search", algorithm: :concurrently, if_exists: true
+  end
+
+  def down
+    return unless postgres? && index_name_exists?(:truffler_labels, COVERING)
+
+    add_index :truffler_labels, COLUMNS, name: "index_truffler_labels_for_search", algorithm: :concurrently,
+      if_not_exists: true
+    remove_index :truffler_labels, name: COVERING, algorithm: :concurrently
+  end
+
+  private
+
+  def postgres?
+    connection.adapter_name.match?(/postg/i)
+  end
+
+  def covered?
+    index_name_exists?(:truffler_labels, COVERING) ||
+      select_value("SELECT indexdef FROM pg_indexes WHERE indexname = 'index_truffler_labels_for_search'").to_s.include?("INCLUDE")
+  end
+end

data/lib/generators/truffler/upgrade/upgrade_generator.rb

@@ -3,19 +3,65 @@ require "rails/generators/active_record"
 
 module Truffler
   module Generators
-    # Adds the tables a newer Truffler needs to an app installed with an older
-    # one. Each migration skips a table that already exists.
+    # Adds the tables and columns a newer Truffler needs to an app installed
+    # with an older one. Safe to rerun: a step is skipped when db/migrate
+    # already holds a migration of that name or the database already has
+    # what it adds, so only missing migrations are written. Each migration
+    # also skips a table, column, or index that already exists.
     class UpgradeGenerator < Rails::Generators::Base
       include ActiveRecord::Generators::Migration
 
       source_root File.expand_path("templates", __dir__)
 
+      class_attribute :schema_connection, default: -> { ActiveRecord::Base.connection }
+
       def create_backfill_spends_migration
-        migration_template "backfill_spends_migration.rb.tt", File.join(db_migrate_path, "create_truffler_backfill_spends.rb")
+        add_migration "backfill_spends_migration.rb.tt", "create_truffler_backfill_spends" do |connection|
+          connection.table_exists?(:truffler_backfill_spends)
+        end
+      end
+
+      def create_backfill_spends_tenant_key_migration
+        add_migration "backfill_spends_tenant_key_migration.rb.tt", "add_tenant_key_to_truffler_backfill_spends" do |connection|
+          connection.column_exists?(:truffler_backfill_spends, :tenant_key)
+        end
+      end
+
+      # Postgres only: the 0.1.5 covering index that makes label-only ranking
+      # an index-only scan. Other databases and installs that already have it
+      # skip the step.
+      def create_labels_search_covering_migration
+        add_migration "labels_search_covering_migration.rb.tt", "cover_truffler_labels_for_search" do |connection|
+          !connection.adapter_name.match?(/postg/i) || labels_search_covered?(connection)
+        end
       end
 
       private
 
+      def labels_search_covered?(connection)
+        connection.index_name_exists?(:truffler_labels, "index_truffler_labels_for_search_covering") ||
+          connection.select_value("SELECT indexdef FROM pg_indexes WHERE indexname = 'index_truffler_labels_for_search'").to_s.include?("INCLUDE")
+      end
+
+      def add_migration(template, name, &applied)
+        existing = self.class.migration_exists?(File.join(destination_root, db_migrate_path), name)
+        if existing
+          say_status :skip, "#{name} (#{File.basename(existing)} exists)", :yellow
+        elsif schema_applied?(&applied)
+          say_status :skip, "#{name} (already in the database)", :yellow
+        else
+          migration_template template, File.join(db_migrate_path, "#{name}.rb")
+        end
+      end
+
+      # Without a database (not configured, not created yet) every step counts
+      # as missing; the migrations themselves are no-ops where already applied.
+      def schema_applied?
+        yield schema_connection.call
+      rescue ActiveRecord::ActiveRecordError
+        false
+      end
+
       def migration_version
         "[#{ActiveRecord::Migration.current_version}]"
       end

data/lib/tasks/truffler.rake

@@ -27,40 +27,55 @@ namespace :truffler do
 
   describe_cursor = ->(cursor) { cursor.nil? ? "none" : cursor }
 
+  resolve_tenant = ->(value) { value.to_s.strip.presence }
+
+  describe_model = ->(model, tenant_key) { tenant_key ? "#{model.name} (tenant #{tenant_key})" : model.name }
+
   desc "Backfill stale, missing, and failed labels for a model, waiting out budget denials " \
-    "(SPEND_CAP=dollars or none; default config.backfill_spend_cap; MAX_DURATION=seconds; RESET_SPEND=1 for a fresh spend ledger)"
+    "(TENANT=key for one tenant; SPEND_CAP=dollars or none; default config.backfill_spend_cap; MAX_DURATION=seconds; " \
+    "RESET_SPEND=1 for a fresh spend ledger)"
   task :backfill, [ :model ] => :setup do |_, args|
     model = resolve_model.call(args[:model])
+    tenant_key = resolve_tenant.call(ENV.fetch("TENANT", nil))
+    name = describe_model.call(model, tenant_key)
     spend_cap = resolve_spend_cap.call(ENV.fetch("SPEND_CAP", nil))
     max_duration = resolve_max_duration.call(ENV.fetch("MAX_DURATION", nil))
     if ENV.fetch("RESET_SPEND", nil) == "1"
-      Truffler::Labeling::Backfill.reset_spend!(model)
-      puts "#{model.name}: fresh spend ledger for the current vocabulary version"
+      ledger_tenant = model.truffler_definition.ledger_tenant(tenant_key)
+      all_tenants = ledger_tenant.nil? && !model.truffler_definition.ledger_tenant("").nil?
+      Truffler::Labeling::Backfill.reset_spend!(model, tenant_key: ledger_tenant, all_tenants: all_tenants)
+      puts all_tenants ? "#{name}: fresh spend ledgers for every tenant" : "#{name}: fresh spend ledger for the current vocabulary version"
     end
     progress = lambda do |so_far, delay|
-      puts "#{model.name}: waiting #{format('%.1f', delay)}s for backfill budget (#{so_far.labeled} labeled, " \
+      puts "#{name}: waiting #{format('%.1f', delay)}s for backfill budget (#{so_far.labeled} labeled, " \
         "$#{format('%.6f', so_far.cost)} spent, cursor #{describe_cursor.call(so_far.cursor)})"
     end
-    result = Truffler::Labeling::Backfill.new(model, spend_cap: spend_cap).run(wait: true, max_duration: max_duration, progress: progress)
-    summary = "#{model.name}: #{result.status}, #{result.labeled} labeled in #{result.requests} requests, $#{format('%.6f', result.cost)}"
+    result = Truffler::Labeling::Backfill.new(model, tenant_key: tenant_key, spend_cap: spend_cap)
+      .run(wait: true, max_duration: max_duration, progress: progress)
+    summary = "#{name}: #{result.status}, #{result.labeled} labeled in #{result.requests} requests, $#{format('%.6f', result.cost)}"
     summary += ", cursor #{describe_cursor.call(result.cursor)}" if result.status == :paused
     puts summary
   end
 
-  print_status = lambda do |model|
-    puts model.name
+  print_status = lambda do |model, tenant_key = nil|
+    puts describe_model.call(model, tenant_key)
     Truffler::Labeling::Backfill.status(model).each { |key, count| puts format("  %-9s %d", key, count) }
-    if Truffler::Records::BackfillSpend.available?
-      ledger = Truffler::Labeling::Backfill.spend(model)
+    ledger_tenant = model.truffler_definition.ledger_tenant(tenant_key)
+    if Truffler::Records::BackfillSpend.available? && Truffler::Records::BackfillSpend.tenant_ledgers? &&
+        model.truffler_definition.ledger_tenant("") && ledger_tenant.nil?
+      puts format("  %-9s %s", "spent", "per tenant; pass TENANT=key")
+    elsif Truffler::Records::BackfillSpend.available?
+      ledger = Truffler::Labeling::Backfill.spend(model, tenant_key: ledger_tenant)
       cap = Truffler.config.backfill_spend_cap
       puts format("  %-9s $%.6f in %d requests (vocabulary %s, cap %s)", "spent", ledger&.spent_usd.to_f, ledger&.requests.to_i,
-        Truffler::Labeling::Backfill.ledger_version(model).first(12), cap ? format("$%.2f", cap) : "none")
+        Truffler::Labeling::Backfill.ledger_version(model, ledger_tenant).first(12), cap ? format("$%.2f", cap) : "none")
     else
       puts format("  %-9s %s", "spent", "not tracked across runs; run bin/rails g truffler:upgrade && bin/rails db:migrate")
     end
   end
 
-  desc "Print a model's labeling counts by status and staleness (every registered Truffler model when none is named)"
+  desc "Print a model's labeling counts by status and staleness (every registered Truffler model when none is named; " \
+    "TENANT=key for a tenant's spend ledger)"
   task :status, [ :model ] => :setup do |_, args|
     if args[:model].to_s.strip.empty?
       Rails.application.eager_load! if defined?(Rails.application) && Rails.application
@@ -68,7 +83,7 @@ namespace :truffler do
       abort "No Truffler models are registered" if models.empty?
       models.each(&print_status)
     else
-      print_status.call(resolve_model.call(args[:model]))
+      print_status.call(resolve_model.call(args[:model]), resolve_tenant.call(ENV.fetch("TENANT", nil)))
     end
   end
 end

data/lib/truffler/benchmark/runner.rb

@@ -223,7 +223,7 @@ module Truffler
         failures = @dataset.injections.filter_map do |injection|
           clean = stored[injection.clean_id]
           twin = stored[injection.record.id]
-          changed = (clean.keys | twin.keys).reject { |key| Metrics.agree?(clean[key], twin[key], tolerance) }
+          changed = (clean.keys | twin.keys).reject { |key| Metrics.agree?(clean.fetch(key, 0.0), twin.fetch(key, 0.0), tolerance) }
           next if changed.empty?
 
           { "fixture" => injection.id, "clean_id" => injection.clean_id, "twin_id" => injection.record.id,

data/lib/truffler/clients/evaluator.rb

@@ -0,0 +1,44 @@
+module Truffler
+  module Clients
+    # Wraps a host client whose `evaluate(state:, schema:)` takes a schema
+    # object (it reads `schema.questions`) and returns an evaluation object
+    # with `answers`, `model`, and `input_tokens` (or a `usage` hash) readers
+    # instead of a hash, such as Cora's TypeSafeClient and its
+    # TypeSafe::Evaluation. The schema also reads like the question hash, so
+    # a client that treats it as a Hash works too. The pinned model is passed
+    # when the host's method accepts `model:`.
+    #
+    #   config.client = Truffler::Clients::Evaluator.new(TypeSafeClient.new)
+    class Evaluator < Callable
+      Schema = Data.define(:questions) do
+        delegate :[], :each, :keys, :size, :empty?, :as_json, :to_json, to: :questions
+
+        def ids
+          questions.keys
+        end
+
+        def to_h
+          questions
+        end
+      end
+
+      def perform(state:, questions:, model:)
+        response(super(state: state, questions: Schema.new(questions: questions), model: model))
+      end
+
+      private
+
+      def response(evaluation)
+        return evaluation if evaluation.is_a?(Hash)
+
+        tokens = evaluation.try(:input_tokens) || usage_tokens(evaluation.try(:usage))
+        { "answers" => evaluation.answers.to_h, "model" => evaluation.try(:model),
+          "usage" => ({ "input_tokens" => tokens } if tokens) }.compact
+      end
+
+      def usage_tokens(usage)
+        usage.is_a?(Hash) ? usage.with_indifferent_access[:input_tokens] : usage.try(:input_tokens)
+      end
+    end
+  end
+end

data/lib/truffler/configuration.rb

@@ -19,6 +19,12 @@ module Truffler
       :smart_run_ttl, :smart_candidate_pool, :broadcaster
     # Generic nouns that are never required keywords on their own (Search::Filler).
     attr_accessor :filler_words
+    # ->(model, tenant_key) { true/false } for tenant-scoped models; nil indexes every tenant.
+    attr_accessor :tenant_enabled
+    # :tenant (a spend ledger per tenant for scoped models) or :app (one per model).
+    attr_accessor :backfill_spend_cap_scope
+    # Choice options below this probability store no row and read as 0.0 (nil stores every option).
+    attr_accessor :choice_min_probability
 
     def initialize(env: ENV)
       @model = "jev-latest"
@@ -53,6 +59,9 @@ module Truffler
       @smart_candidate_pool = 200
       @broadcaster = nil
       @filler_words = Search::Filler::DEFAULT_WORDS.dup
+      @tenant_enabled = nil
+      @backfill_spend_cap_scope = :tenant
+      @choice_min_probability = 0.05
     end
 
     def client

data/lib/truffler/definition.rb

@@ -116,6 +116,42 @@ module Truffler
       @weak_below || DEFAULT_WEAK_BELOW
     end
 
+    attr_accessor :index_if, :index_scope
+
+    # config.tenant_enabled is asked only for tenant-scoped models; an
+    # unscoped model is always enabled.
+    def tenant_enabled?(tenant_key)
+      check = Truffler.config.tenant_enabled
+      return true if check.nil? || !scoped?
+
+      check.call(model, tenant_key) ? true : false
+    end
+
+    # Whether the after-commit hooks, Queue, and EmbedJob handle this record:
+    # its tenant is enabled and `index_if` (when declared) accepts it.
+    def indexable?(record)
+      tenant_enabled?(tenant_key_for(record)) && (index_if.nil? || index_if.call(record) ? true : false)
+    end
+
+    # The relation the batch paths (backfills, sweeps) page over.
+    def index_relation(relation = model.all)
+      index_scope ? index_scope.call(relation) : relation
+    end
+
+    # The tenant a backfill spend ledger row belongs to: the tenant for
+    # scoped models under backfill_spend_cap_scope :tenant, else nil (app-wide).
+    def ledger_tenant(tenant_key)
+      tenant_key if scoped? && Truffler.config.backfill_spend_cap_scope.to_sym == :tenant
+    end
+
+    attr_writer :invite_on_pending_encoding
+
+    # Whether a model with a `keyword` source shows the Smart search row while
+    # the query's encoding is in flight (AE10). Default true.
+    def invite_on_pending_encoding
+      @invite_on_pending_encoding.nil? || @invite_on_pending_encoding
+    end
+
     private
 
     def table_available?
@@ -204,6 +240,22 @@ module Truffler
       def weak_below(count)
         @definition.weak_below = Integer(count)
       end
+
+      def index_if(callable)
+        raise DefinitionError, "index_if must be callable with the record" unless callable.respond_to?(:call)
+
+        @definition.index_if = callable
+      end
+
+      def index_scope(callable)
+        raise DefinitionError, "index_scope must be callable with a relation" unless callable.respond_to?(:call)
+
+        @definition.index_scope = callable
+      end
+
+      def invite_on_pending_encoding(enabled)
+        @definition.invite_on_pending_encoding = enabled ? true : false
+      end
     end
   end
 end