truffler 0.1.3 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bc0514a875fdc157249448de1514bb62560348ab4dfba22c75e3d0b0137b9072
-  data.tar.gz: 93b65a47fb89826536cdf03bd68efd824193488d20675789776400de2f19fb7a
+  metadata.gz: 41cc5bfdb1f4f8a5b18ab9388518140bb5b2c927c87f115094c43ffa1c37659a
+  data.tar.gz: 11ee26029552cfaaa18d68fed182a2ede4641f9491c942eecbf6aa86326baef8
 SHA512:
-  metadata.gz: 4cbcba6f0deaee8afecbbe3e72988a175422b6ce42ba0f3d42b6d30b77f4736f28f485c21e15aac956fb31c4853724c30402872c52f8e5aff44acec36c0850a1
-  data.tar.gz: df78717442f56984544151b2962f3b78d724e001b6111d2ea92c9fd41903cb53c57117f85bac698bac4726b66000608b6a5132352eaa3c0fa30c688c893c1d04
+  metadata.gz: b1bc305d0c1745424ce0c55e905a6f592a423f37274edb420def42073119de9f089ac52fded8f0af0319327e9e7e3738b41e6d62fe77ec1d398d1f7c7bddd170
+  data.tar.gz: 8dd567f12f014c36ac80e900a87965390cbaae8f3c3e63e588cd6668510e68afc020f8d0039198fea2c13a98aabff2425b6a5eef839c5b3be64864ca8be1e868

data/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [0.1.5]
+
+Fixes from the Cora integration at Postgres scale, and two from happyhappy's 0.1.4 upgrade.
+
+- Tenant-scoped indexing: `index_if ->(record) { ... }` and `index_scope ->(relation) { ... }` in `truffler do`, plus `config.tenant_enabled = ->(model, tenant_key) { ... }`. Records outside them, or in a disabled tenant, are never labeled, embedded or backfilled. The after-commit hooks, `Labeling::Queue`, the labeler, both backfills, lens backfills, `LabelFlushJob`, `ResumeJob` and `EmbedJob` all honor them. `BackfillJob` takes `tenant_key:`, over-cap demotion and `ResumeJob` enqueue one job per tenant, and `truffler:backfill` and `truffler:status` accept `TENANT=`.
+- `Embeddings::Backfill` pages per tenant with a limited `NOT EXISTS` anti-join instead of a whole-table `NOT IN`.
+- The backfill spend ledger is per tenant for tenant-scoped models (`truffler_backfill_spends.tenant_key`; `config.backfill_spend_cap_scope = :tenant`, the default, or `:app`), so `backfill_spend_cap` applies to each tenant. Existing installs: `rails g truffler:upgrade && rails db:migrate`.
+- `rails g truffler:upgrade` is safe to rerun. It skips migrations already present or applied and writes only the missing ones, with no conflicts.
+- Choice labels store a row only for options at or above `config.choice_min_probability` (default 0.05; `nil` stores all), plus the most likely option. Missing options read as 0.0 everywhere, and no migration is needed.
+- An out-of-shape `from:` answer settles: nothing is stored for that label, the record is marked labeled, and `truffler.supplied_label_failed` fires with `permanent: true`. A raising `from:` still retries and fails after `max_attempts`.
+- Postgres: upserts no longer assign `updated_at` twice (Rails 8.1 raised `PG::SyntaxError` on every live enqueue and embedding write). The suite now also runs on Postgres with pgvector in CI (`TRUFFLER_DATABASE_URL`). As a side effect, re-enqueuing an already pending row no longer refreshes its `updated_at`.
+- Keystroke SQL at scale on Postgres. A `keyword` or `exact` callable that returns a relation runs once as `id = ANY(ARRAY(subquery))` instead of a hashed `IN` over every tenant row; returning an Array of ids is the documented fast path. Label-only and filtered searches add up label scores in one grouped subquery joined on `record_id` instead of a subquery per row: 120 ms p50 before, 51 ms after, for 50k records. New installs get `INCLUDE (record_id)` on `index_truffler_labels_for_search`, and `truffler:upgrade` adds it on existing Postgres installs. On Postgres, `NeighborStore` scores text from the tenant's top-K neighbors (`ORDER BY embedding <=> q LIMIT k`, default 200); `NeighborStore.new(k:, top_k:)` is new, and `config.vector_store` accepts a store instance. Records outside the top `k` get no text score. The README covers `SET LOCAL jit = off` for the keystroke transaction.
+- New `invite_on_pending_encoding` declaration (default true): a model with a `keyword` source, such as a blind index, still shows the `:encoding_pending` Smart search row while a first-time query's encoding is in flight. `Result#local_weak?` keeps the backup provider off when the keyword list is strong.
+- New `Truffler::Clients::Evaluator` for clients that read `schema.questions` and return an evaluation object without `to_h`, such as Cora's `TypeSafeClient`. The Cora checklist in `docs/host-integration.md` now shows it instead of the broken `Callable` line.
+- A query word that names any declared label key, option key or option search-text word, applied or not, is never dropped as filler. `email` and `emails` are no longer default `config.filler_words`.
+- Removing the last label chip no longer makes filler nouns required keywords while a time phrase still applies.
+- A lens backfill never pages disabled tenants (filtered in SQL, so a capped `LensBackfillJob` cannot refetch them forever), and the labeler never deletes a disabled tenant's state rows; it returns them to pending at backfill priority, so re-enabling relabels only what is stale.
+- Removing chips keeps declared label and option words (option search text included) as keywords, the same as the encoder and the cold-cache path.
+- `truffler:status` and `Backfill.status(model, tenant_key:)` count only records inside `index_scope` and enabled tenants, and only their state rows, so a finished partial rollout reports nothing missing or pending.
+
+## [0.1.4]
+
+Fixes from happyhappy production.
+
+- Generic filler nouns are no longer required keywords. "customers in the last 3 hours" returned nothing while "in the last 3 hours" returned the records. New `config.filler_words` (default: customer, customers, people, person, user, users, message, messages, email, emails, item, items, stuff, thing, things; replace or extend it, matched ignoring plurals). One rule now serves both the encoder's word reconciliation and the cold-cache keywords: stopwords and filler words are dropped unless that would leave the search with no keyword, no applied label, and no time phrase, so a lone "customers" still searches text and "customers refund this week" requires only "refund".
+- `rake truffler:status` with no model prints the status of every registered Truffler model instead of " is not a Truffler model". An unknown model name still aborts with a message.
+- A choice option may be `{ description:, search: }`: the long description stays what Jev labels with, and the short search text drives query-word matching and the request state's `option_names`. Plain descriptions and per-tenant callables keep working, and callables may return the new shape. Labeling fingerprints now cover only what Jev labels with: asked labels are unchanged (search texts are not part of the question), and a supplied label's fingerprint digests its type, option keys, legend, and `version:` but no longer its descriptions. Descriptions and search texts go into a separate encoding version that keys the query-encoding cache, so rewording re-encodes queries without staling labels.
+- Removing a chip gives its words back to the keywords. The cached encoding records which applied labels each label-term word named (by token position, no query text); once every one of them is removed, the word is a keyword again, so "urgent refunds" without the urgent chip matches both words. A word that named a label or option key only by shared prefix adds a soft keyword score (a quarter of the keyword weight) and is never required.
+- Upgrading from 0.1.3: supplied (`from:`) labels get a new fingerprint, so their stored rows and record states read stale once. `rake truffler:backfill` (or the next watched change) rewrites them at no Jev cost; the new vocabulary version also starts a fresh backfill spend ledger. Query encodings cached by 0.1.3 miss once and are re-encoded.
+- Removing the chip that justified dropping a filler word (the time range, or the last applied label) brings the filler noun back as a keyword, cached or cold. When nothing anchors a search and only droppable words are left, filler nouns stay keywords and only pure stopwords are dropped ("the customers" searches "customers").
+
 ## [0.1.3]
 
 - Display-name and description words of an applied choice option now match query words exactly (ignoring case and plurals) instead of by shared three-letter prefix. Under a filter, ordinary search words such as "email", "inbox" and "summary" (with Cora applied) or "chat" and "change" (with billing applied) no longer become label terms and keep ranking results. Label keys and option keys still match by shared prefix, so "angry" names `anger`.

data/README.md

@@ -51,6 +51,10 @@ bin/rails db:migrate
 
 The migration creates only that table and skips it if it already exists. Until you run it, backfills log one warning and cap spend per run, as 0.1.1 did.
 
+0.1.5 adds `truffler_backfill_spends.tenant_key` for per-tenant spend ledgers. The same `truffler:upgrade` command writes that migration. Until you run it, every tenant shares the model's app-wide ledger and a warning is logged once.
+
+`truffler:upgrade` is safe to rerun. It skips any migration already in `db/migrate` or already applied to the database, and writes only the missing ones.
+
 Truffler digests user keys and query misses with `secret_key_base`. Rails supplies it automatically; outside Rails, set `config.secret_key_base`.
 
 ## Declaring a model
@@ -84,6 +88,7 @@ class Email < ApplicationRecord
     surface :inbox, explicit_action: :enter     # :enter, :key, or :row
     ranking label: 1.0, text: 1.0, keyword: 0.5, exact: 1.0
     weak_below 3                                # fewer keystroke results than this counts as weak
+    invite_on_pending_encoding true             # Smart search row while a first-time query's encoding is in flight
   end
 end
 ```
@@ -92,10 +97,12 @@ Here is what each option does:
 
 - `filter_at` makes a label a hard filter at that probability when the query asks for it. `boost` is the label's weight when the query prefers it. `filter_weight:` (default 0) is how much intent weight a filter adds on top.
 - Choice `options:` may be a callable of the tenant key, which gives each tenant its own vocabulary. A tenant it gives no options (`{}` or nil) simply lacks the label: it is neither asked nor encoded there. The option name `truffler:none` is reserved (see `Truffler::NO_OPTION` below).
+- An option's value may be `{ description: "...", search: "..." }` instead of a plain description, e.g. `options: { "prod-a1" => { description: "Cora, the AI email assistant that drafts replies...", search: "Cora email assistant" } }`. Jev labels records with the description; query words are matched against the short search text (else the description), and the request state's `option_names` carries it. A per-tenant callable may return the same shape. Editing a search text never stales labels or triggers a backfill; it only changes the query-encoding cache key, so queries are re-encoded.
 - `watch :column, ...` relabels every label when one of those columns changes. Saving a record only relabels on columns that changed, so a `reads` field backed by a method (a conversation built from messages, say) needs the columns it is built from in `watch`.
-- `keyword` also accepts a single callable, `->(scope, tokens) { relation }`.
+- `keyword` also accepts a single callable, `->(scope, tokens) { relation }`. It, and each `exact` callable, may return an Array of ids instead of a relation, which is far faster on a large tenant (see [Keystroke search at scale](#keystroke-search-at-scale)).
 - `embeddings column: :my_vector` searches a vector column you maintain yourself. Truffler never writes it.
 - `embeddings` refuses to send encrypted fields to the embedding provider unless you pass `allow_encrypted: true`.
+- `invite_on_pending_encoding` (default true) shows the Smart search row with reason `:encoding_pending` while a query's first encoding is in flight, even when a `keyword` source matched. A model with no `keyword` source always does. Pass `false` to invite only on weak or empty results.
 
 `label key, type, **options` takes:
 
@@ -103,7 +110,7 @@ Here is what each option does:
 |---|---|---|
 | `question:` | asked labels (required); optional with `from:` | What Jev is asked about each record. |
 | `criteria:` | `:noul` | `{ true => "...", false => "..." }` guidance for Jev. |
-| `options:` | `:choice` (required) | Option names, `{ option => description }`, or a callable of the tenant key. |
+| `options:` | `:choice` (required) | Option names, `{ option => description }`, `{ option => { description:, search: } }`, or a callable of the tenant key returning any of these. |
 | `legend:` | `:score` (required) | Two or more ordered levels, as an array or `{ index => description }`. |
 | `filter_at:`, `boost:`, `filter_weight:` | all | Filter threshold, boost weight, and the intent weight a filter adds. |
 | `description:` | all | The label's wording in query encoding. Defaults to the question, then to the key. |
@@ -148,7 +155,10 @@ end
 | `:choice` | one option string (meaning 1.0 for it), or `{ option => probability }` | one row per option, `label:option`; options left out store 0.0 |
 | `:score` | a level index into `legend:` | `index / (levels - 1)`, as for Jev scores |
 
-`nil` stores nothing: the record reads as missing that label, not as 0. An answer out of shape (an undeclared option, a probability outside 0..1, a level past the legend) or a `from:` that raises stores nothing for that label and emits `truffler.supplied_label_failed` with the label key and error class. Rows already stored keep serving until the next good write.
+`nil` stores nothing: the record reads as missing that label, not as 0. Both kinds of failure emit `truffler.supplied_label_failed` with the label key, the error class, and `permanent`:
+
+- An answer out of shape (an undeclared option, a probability outside 0..1, a level past the legend) is permanent (`permanent: true`). It is treated like `nil`: nothing is stored for that label, earlier rows are cleared, and the record is marked labeled, so backfills do not claim it again. A later valid answer, through `watch:` or `truffler_refresh_labels!`, is written as usual.
+- A `from:` that raises is transient (`permanent: false`). The record stays pending at backfill priority and is retried for free, then marked failed after `max_attempts`. Rows already stored keep serving until the next good write.
 
 Supplied labels never reach Jev. They are never in a labeling request, take no budget slot, cost nothing, and do not count toward a lens or backfill spend cap. A flush or backfill where only supplied labels are stale makes no Jev call. The labeling job writes them before it asks Jev anything, so a Jev outage or a budget denial never holds them back. Otherwise they behave like asked labels: same `truffler_labels` rows, the same filters, boosts, chips, and label vectors, and query encoding asks Jev how a query uses them (using `description:`), within the one encoding call per query.
 
@@ -158,7 +168,7 @@ To keep them fresh:
 - Call `record.truffler_refresh_labels!` when the answers change somewhere truffler cannot see, such as the job that runs your classifier after insert. It rewrites the record's supplied labels immediately, with no Jev call.
 - Bump `version:` when the logic behind `from:` changes. That changes the vocabulary version, so `rake truffler:backfill` rewrites the label for every record, again at no cost.
 
-A supplied label's fingerprint digests its type, options, legend, description, and `version:`, never the Jev model, so changing `config.model` does not rewrite supplied labels.
+A supplied label's fingerprint digests its type, option keys, legend, and `version:`, never the Jev model or its wording, so changing `config.model`, `description:`, or an option's description or search text does not rewrite supplied labels. Wording changes only re-key the query-encoding cache.
 
 ## Clients
 
@@ -167,11 +177,17 @@ The default client is `Truffler::Clients::RubyLLMTypeSafe`. It needs ruby_llm 2
 If you already have a TypeSafe client, wrap it in `Truffler::Clients::Callable`. The wrapped object must respond to `evaluate(state:, schema:)`, and may also accept `model:`. It returns either the answers hash or `{"answers" => ..., "model" => ..., "usage" => {"input_tokens" => ...}}`:
 
 ```ruby
-config.client = Truffler::Clients::Callable.new(TypeSafeClient.new)
+config.client = Truffler::Clients::Callable.new(MyJevClient.new)
 ```
 
 `schema:` holds the questions in TypeSafe wire shape. When a response carries no token count, Truffler estimates it from the request size and flags the estimate. Errors reach you as `Truffler::ClientError`, carrying only the HTTP status and the error class name.
 
+If your client takes a schema object (it calls `schema.questions`) and returns an evaluation object with `answers`, `model`, and `input_tokens` readers rather than a hash, use `Truffler::Clients::Evaluator` instead. For any other shape, subclass `Truffler::Clients::Base` and implement `perform` (see [docs/host-integration.md](docs/host-integration.md)).
+
+```ruby
+config.client = Truffler::Clients::Evaluator.new(TypeSafeClient.new)
+```
+
 ## Keystroke search
 
 ```ruby
@@ -183,7 +199,9 @@ result = Email.truffler(params[:q], tenant: Current.account.id, scope: Current.a
 
 A keystroke search makes no network call. It reads the query encoding and query vector from the cache. When the cache misses, it enqueues `EncodeQueryJob`, and the next keystroke or reload picks up the result.
 
-Query encoding sends Jev the label vocabulary (each label's description and a choice label's option names) next to the query, and asks for each label whether the query filters on it, prefers it, or ignores it, plus the role of each word. A choice label's option question also offers `Truffler::NO_OPTION` (`"truffler:none"`), meaning the query names none of its options, so a host option literally called `none` stays filterable. Word roles are then checked locally: a word that names a label the query applies (its key, a word of its key, or the chosen option key, ignoring case and plurals or sharing their first three letters when both words have four letters or more, so "angry" names `anger`; or, matched exactly, a word of that option's display name or description) counts as naming the label, and common stopwords are dropped. When the encoding applies a label filter, the filter decides which records match and keyword hits only rank them; without one, the remaining keywords must match.
+Query encoding sends Jev the label vocabulary (each label's description and a choice label's option names) next to the query, and asks for each label whether the query filters on it, prefers it, or ignores it, plus the role of each word. A choice label's option question also offers `Truffler::NO_OPTION` (`"truffler:none"`), meaning the query names none of its options, so a host option literally called `none` stays filterable. Word roles are then checked locally: a word that names a label the query applies (its key, a word of its key, or the chosen option key, ignoring case and plurals or sharing their first three letters when both words have four letters or more, so "angry" names `anger`; or, matched exactly, a word of that option's display name or description) counts as naming the label, and common stopwords and `config.filler_words` (generic nouns such as "customers", "users", "items") are dropped. A word that names any declared label (a word of its key, of an option key, or of an option's search text), applied or not, is never dropped as filler. Filler is kept only when dropping them would leave no keyword, no applied label, and no time phrase, so "customers in the last 3 hours" lists the window's records while a lone "customers" still searches text; the same rule applies before the encoding is cached. When the encoding applies a label filter, the filter decides which records match and keyword hits only rank them; without one, the remaining keywords must match.
+
+When the searcher removes a chip, the words that named only that label (or only labels that are now all removed) become keywords again, so removing the `urgent` chip from "urgent refunds" searches for both words. A word that named a label or option key only by shared prefix ("urgently" for `urgent`) also adds a small keyword score (a quarter of the keyword weight) while its label applies, ranking records whose text contains it higher without requiring it.
 
 Time phrases are handled in Ruby and never asked of Jev: `today`, `yesterday`, `this week`, `last week`, `this month`, `last month`, `past|last N hours|days|weeks|months`, `last hour`, `past hour`, `past week`, `past month`, and `since monday` through `since sunday`. "Past/last N units" is a rolling window ending now; "this/last week" and "this/last month" are calendar windows. The first one in a query limits results to records whose `arrived_at` column falls in that window, its words are not keywords, and it shows as a chip `{key: "time", label: "time", kind: :time, name: "This week"}`. Pass `"time"` in `suppressed:` to drop it. Weeks start on `Date.beginning_of_week`, and the window is computed from `Time.current` (or a `clock:` callable passed to the search, for tests).
 
@@ -192,6 +210,7 @@ The returned `Truffler::Search::Result` exposes:
 - `records` and `ids`.
 - `chips`: `[{key:, label:, kind: :filter | :boost | :time, name:}]`.
 - `invite_row`: `{query:, reason: :weak | :empty | :encoding_pending}` or nil.
+- `local_weak?`: whether the list is too weak to stand alone (starts the backup provider). A pending encoding on a model with a `keyword` source invites Smart search without making the list weak.
 - `encoding_status`: `:cached`, `:pending`, or `:none`.
 - `watermark` and `new_matches_count`.
 - `explicit_action`: the surface's declared action.
@@ -204,6 +223,24 @@ For the "N new matches" row on a later request, pass the watermark back in:
 Email.jev_new_matches_count(params[:q], tenant: account.id, scope: account.emails, user: current_user, since: Time.iso8601(params[:since]))
 ```
 
+### Keystroke search at scale
+
+The keystroke query is shaped so a large tenant (tens of thousands of records) stays fast on Postgres:
+
+- Return ids from `keyword` and `exact` callables when you can, e.g. `.limit(2_000).pluck(:id)` from a blind index. Ids become a literal `IN (...)` list. A returned relation is rendered as `id = ANY(ARRAY(subquery))` on Postgres (plain `IN (subquery)` elsewhere), so it runs once instead of as a hashed filter over every tenant row, but the id list is still faster.
+- Label-only and filtered searches, where every record past the tenant and filters is a candidate, read label scores from one grouped aggregate over `truffler_labels` joined on `record_id`, never a subquery per row. New installs get `index_truffler_labels_for_search` with `INCLUDE (record_id)`, which makes that aggregate an index-only scan. On an existing Postgres install, `bin/rails generate truffler:upgrade && bin/rails db:migrate` adds it (built concurrently, then the old index is dropped). It is optional: the grouped aggregate already replaces the per-row subquery, and the index only saves heap reads.
+- With embeddings on Postgres, text similarity comes from the tenant's top `k` neighbors (`ORDER BY embedding <=> q LIMIT k`), joined on `record_id`. An HNSW index on `truffler_embeddings.embedding` serves it. Records outside the top `k` score no text similarity; see `vector_store` under [Configuration](#configuration).
+- The plans these queries produce are costed high enough that Postgres JIT often compiles them, which can add 0.3 to 0.9 s to a query that runs in tens of milliseconds. Turn JIT off for the keystroke only:
+
+  ```ruby
+  result = Email.transaction do
+    Email.connection.execute("SET LOCAL jit = off")
+    Email.truffler(params[:q], tenant: account.id, scope: account.emails, user: current_user)
+  end
+  ```
+
+  `SET LOCAL` lasts until the transaction ends, so other queries keep JIT. `result.records` is loaded inside the block.
+
 ## Smart search and streaming
 
 The searcher's explicit action (the `explicit_action` their surface declares) starts a Smart run:
@@ -338,11 +375,12 @@ Set these in `Truffler.configure do |config| ... end`.
 | `max_field_chars`, `request_token_budget`, `max_questions_per_request` | 4,000, 48,000, 200 | Request packing limits. |
 | `queue_name` | `:default` | Queue for every Truffler job. |
 | `cost_per_million_tokens` | 0.042 | Jev input price, used in usage events and estimates. |
-| `backfill_spend_cap` | 5.0 | Dollar cap on backfill spend per model and vocabulary version, shared by `BackfillJob` chains, `ResumeJob` backfills, and `truffler:backfill` runs (see Backfill). `nil` disables it; for the rake task, `SPEND_CAP=none` does. Supplied labels cost nothing and are still written once it is reached. |
+| `backfill_spend_cap` | 5.0 | Dollar cap on backfill spend per model (per tenant for tenant-scoped models; see `backfill_spend_cap_scope`) and vocabulary version, shared by `BackfillJob` chains, `ResumeJob` backfills, and `truffler:backfill` runs (see Backfill). `nil` disables it; for the rake task, `SPEND_CAP=none` does. Supplied labels cost nothing and are still written once it is reached. |
 | `resume_pending_after` | 5 minutes | How long before `ResumeJob` treats work as stuck. |
 | `embedder` | `Embeddings::RubyLLMEmbedder.new` | Any `Embeddings::Embedder` subclass. The default calls `RubyLLM.embed`, which works on ruby_llm 1.x and 2. |
 | `embedding_cost_per_million_tokens` | 0.02 | Embedding price. |
-| `vector_store` | `:auto` | `:neighbor` (pgvector `<=>`, or sqlite-vec `vec_distance_cosine` when you load the extension), `:ruby` (exact cosine in Ruby), or `:auto` (neighbor when available, otherwise Ruby). A model declaring `embeddings column:` always reads its own column. |
+| `vector_store` | `:auto` | `:neighbor` (pgvector `<=>`, or sqlite-vec `vec_distance_cosine` when you load the extension), `:ruby` (exact cosine in Ruby), or `:auto` (neighbor when available, otherwise Ruby). Or a store instance, e.g. `Truffler::Embeddings::NeighborStore.new(k: 500)` or your own `VectorStore` subclass. On Postgres the neighbor store scores text from the tenant's top `k` (default 200) neighbors; `NeighborStore.new(top_k: false)` scores every row exactly. A model declaring `embeddings column:` always reads its own column. |
+| `filler_words` | `customer(s) people person user(s) message(s) item(s) stuff thing(s)` | Generic nouns never required as keywords on their own (matched ignoring plurals). Replace the list or extend it (`config.filler_words += %w[ticket]`). |
 | `encoding_prefetch` | `QueryEncoding::Prefetch.new` | Cache-miss hook, called as `call(model, query, cache_key:, tenant_key:, user_key:)`. |
 | `encoding_deadline` | 1.0 | Seconds a Smart run waits for an in-flight query encoding. |
 | `rerank_depth`, `rerank_chunk_size`, `rerank_max_field_chars` | 30, 10, 1,200 | Candidates reranked, candidates per Jev request, characters per field sent. |
@@ -354,6 +392,9 @@ Set these in `Truffler.configure do |config| ... end`.
 | `lenses.creators`, `lenses.authorize_lens`, `lenses.proposals` | `:developers`, nil, false | Lens policy (see Lenses). |
 | `lenses.spend_cap_usd`, `lenses.sample_size`, `lenses.max_questions`, `lenses.expire_after` | 1.0, 20, 8, 30 days | Lens limits. |
 | `lenses.generator`, `lenses.drafter_model`, `lenses.user_key` | `RubyLLMGenerator`, nil, `"User:42"` style | Drafting model seam and user key mapping. |
+| `tenant_enabled` | nil (every tenant) | `->(model, tenant_key) { ... }`, asked for tenant-scoped models only. A disabled tenant is never labeled, embedded, or backfilled (see Indexing only some tenants). |
+| `backfill_spend_cap_scope` | `:tenant` | `:tenant` keeps a spend ledger per tenant for tenant-scoped models, so `backfill_spend_cap` applies to each tenant. `:app` keeps one ledger per model. Unscoped models always use one. |
+| `choice_min_probability` | 0.05 | Choice labels store a row only for options at or above this probability, plus the most likely option. A missing option reads as 0.0 in filters, boosts, label vectors, and contributions. `nil` stores every option. No migration is needed: rows written earlier stay until their record is relabeled. |
 
 ## Jobs to schedule
 
@@ -367,7 +408,7 @@ Truffler enqueues most of its own jobs. Run a worker for `config.queue_name` and
 | `bin/rails "truffler:backfill[Email]"` (see Backfill) or `Truffler::Jobs::BackfillJob.perform_later("Email")` | After adopting Truffler, changing a declaration, or changing the model pin. |
 | `Truffler::Embeddings::Backfill.new(Email).enqueue` | After enabling embeddings or changing the embedding model, width, or fields, to re-embed everything now instead of through the `ResumeJob` sweep. It enqueues 1,000 jobs at a time; pass `limit:` to cap the total. |
 
-`bin/rails "truffler:status[Email]"` prints labeling counts and the backfill spend for the current vocabulary version. `bin/rails "truffler:suggestions[Email]"` prints candidate questions drawn from logged query misses.
+`bin/rails "truffler:status[Email]"` prints labeling counts and the backfill spend for the current vocabulary version; `bin/rails truffler:status` prints them for every registered Truffler model. `bin/rails "truffler:suggestions[Email]"` prints candidate questions drawn from logged query misses.
 
 With Solid Queue, for example:
 
@@ -392,10 +433,36 @@ truffler_expire_lenses:
 |---|---|
 | `SPEND_CAP=20` or `none` | Overrides `backfill_spend_cap` for this run. |
 | `MAX_DURATION=600` | Stops after that many seconds with `paused` and the cursor. Rerun to continue. |
-| `RESET_SPEND=1` | Zeroes the spend recorded for the current vocabulary version before starting. |
+| `RESET_SPEND=1` | Zeroes the spend recorded for the current vocabulary version before starting (every tenant's ledger when no `TENANT` is given). |
+| `TENANT=42` | Backfills only that tenant, against that tenant's ledger. `truffler:status` takes it too, to print that tenant's spend. |
 
 The spend cap holds across runs. Truffler records backfill spend per model and app-wide vocabulary version in `truffler_backfill_spends`, and reserves each request's estimate against the cap in SQL. A rerun, a `ResumeJob` backfill, and overlapping `BackfillJob` chains all draw on the same total, so together they stop at `backfill_spend_cap`. Changing the vocabulary (a reworded question, a new label, or an activated lens) starts a new total. Lens backfills are capped separately by each lens's `spend_cap_usd`. `BackfillJob` reschedules itself after a denial with the same backoff. In code, `Truffler::Labeling::Backfill.new(Email).run(wait: true, max_duration: 600)` does what the task does.
 
+For tenant-scoped models the ledger is per tenant (`backfill_spend_cap_scope :tenant`, the default), so the cap applies to each tenant. A whole-model run skips a tenant once its ledger reaches the cap, keeps labeling the others, and ends with `spend_cap_reached`. A tenant run (`TENANT=`, or `BackfillJob.perform_later("Email", tenant_key: "42")`) stops at that tenant's cap. `ResumeJob` and the flush job's over-cap demotion enqueue one `BackfillJob` per tenant.
+
+### Indexing only some tenants
+
+For a per-account rollout, or to leave some records out, declare which records Truffler indexes:
+
+```ruby
+truffler do
+  tenant :account_id
+  # ...
+  index_if ->(email) { !email.spam? }                  # single records: after-commit hooks, Queue, EmbedJob
+  index_scope ->(relation) { relation.where(spam: false) } # batch paths: backfills and sweeps
+end
+
+Truffler.configure do |config|
+  config.tenant_enabled = ->(model, tenant_key) { Account.search_enabled?(tenant_key) }
+end
+```
+
+- The after-commit label and embed hooks skip records that are not indexable: `index_if` rejects them or their tenant is disabled. No state row or job is created.
+- `Labeling::Backfill`, `Embeddings::Backfill`, and lens backfills page only over `index_scope` and skip disabled tenants. The labeler drops claimed records that are no longer indexable instead of labeling them.
+- `index_if` and `index_scope` should select the same records. Keep `index_scope` index-friendly, because backfills page over it.
+- The `ResumeJob` embedding sweep runs tenant by tenant over the enabled tenants that have records in `index_scope`. Each tenant pass is a `NOT EXISTS` anti-join limited to the sweep size, not a `NOT IN` over the table.
+- Stored labels of a tenant you disable stay in place and keep serving search. When you re-enable the tenant, run `truffler:backfill` with `TENANT=` for it.
+
 ## Privacy
 
 These guarantees hold for every model, and matter most for models that use Active Record encryption:

data/bench/cassettes/1037e0b63ba790c02d14d0ddfc18b68151c11d03ce88d764658044b6bab2e55b.json

@@ -0,0 +1 @@
+{"answers":{"c001__relevance":{"noul":0.0,"type":"noul"},"c002__relevance":{"noul":0.0,"type":"noul"},"c003__relevance":{"noul":0.0,"type":"noul"},"c004__relevance":{"noul":0.0,"type":"noul"},"c005__relevance":{"noul":0.0,"type":"noul"}},"model":"synthetic-jev","request_hash":"1037e0b63ba790c02d14d0ddfc18b68151c11d03ce88d764658044b6bab2e55b","usage":{"input_tokens":873}}

data/bench/cassettes/19bba849a846d30a69dc385a337d67704fbe5194506fd3c914d3270d86495fde.json

@@ -0,0 +1 @@
+{"answers":{"c001__relevance":{"noul":0.0,"type":"noul"},"c002__relevance":{"noul":0.0,"type":"noul"},"c003__relevance":{"noul":0.0,"type":"noul"},"c004__relevance":{"noul":0.0,"type":"noul"},"c005__relevance":{"noul":0.0,"type":"noul"},"c006__relevance":{"noul":0.0,"type":"noul"},"c007__relevance":{"noul":0.0,"type":"noul"}},"model":"synthetic-jev","request_hash":"19bba849a846d30a69dc385a337d67704fbe5194506fd3c914d3270d86495fde","usage":{"input_tokens":1164}}

data/bench/cassettes/3768847dc57df8bf2328771e7763c0b8bd013a167244e7849967dac144420601.json

@@ -0,0 +1 @@
+{"answers":{"c001__relevance":{"noul":0.0,"type":"noul"},"c002__relevance":{"noul":0.0,"type":"noul"}},"model":"synthetic-jev","request_hash":"3768847dc57df8bf2328771e7763c0b8bd013a167244e7849967dac144420601","usage":{"input_tokens":417}}

data/lib/generators/truffler/install/templates/migration.rb.tt

@@ -11,7 +11,7 @@ class CreateTrufflerTables < ActiveRecord::Migration<%= migration_version %>
     end
     add_index :truffler_labels, [:record_type, :record_id, :label_key], unique: true,
       name: "index_truffler_labels_on_record_and_label"
-    add_index :truffler_labels, [:record_type, :tenant_key, :label_key, :value],
+    add_index :truffler_labels, [:record_type, :tenant_key, :label_key, :value], include: [:record_id],
       name: "index_truffler_labels_for_search"
 
     create_table :truffler_record_states do |t|
@@ -108,12 +108,15 @@ class CreateTrufflerTables < ActiveRecord::Migration<%= migration_version %>
 
     create_table :truffler_backfill_spends do |t|
       t.string :record_type, null: false
+      t.string :tenant_key
       t.string :vocabulary_version, null: false
       t.float :spent_usd, null: false, default: 0.0
       t.integer :requests, null: false, default: 0
       t.timestamps
     end
-    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true,
-      name: "index_truffler_backfill_spends_on_ledger"
+    add_index :truffler_backfill_spends, [:record_type, :tenant_key, :vocabulary_version], unique: true,
+      name: "index_truffler_backfill_spends_on_tenant_ledger"
+    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true, where: "tenant_key IS NULL",
+      name: "index_truffler_backfill_spends_on_app_ledger"
   end
 end

data/lib/generators/truffler/upgrade/templates/backfill_spends_tenant_key_migration.rb.tt

@@ -0,0 +1,25 @@
+class AddTenantKeyToTrufflerBackfillSpends < ActiveRecord::Migration<%= migration_version %>
+  def up
+    add_column :truffler_backfill_spends, :tenant_key, :string unless column_exists?(:truffler_backfill_spends, :tenant_key)
+    if index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_ledger")
+      remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_ledger"
+    end
+    unless index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_tenant_ledger")
+      add_index :truffler_backfill_spends, [:record_type, :tenant_key, :vocabulary_version], unique: true,
+        name: "index_truffler_backfill_spends_on_tenant_ledger"
+    end
+    unless index_name_exists?(:truffler_backfill_spends, "index_truffler_backfill_spends_on_app_ledger")
+      add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true, where: "tenant_key IS NULL",
+        name: "index_truffler_backfill_spends_on_app_ledger"
+    end
+  end
+
+  def down
+    execute "DELETE FROM truffler_backfill_spends WHERE tenant_key IS NOT NULL" if column_exists?(:truffler_backfill_spends, :tenant_key)
+    remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_tenant_ledger", if_exists: true
+    remove_index :truffler_backfill_spends, name: "index_truffler_backfill_spends_on_app_ledger", if_exists: true
+    remove_column :truffler_backfill_spends, :tenant_key, if_exists: true
+    add_index :truffler_backfill_spends, [:record_type, :vocabulary_version], unique: true,
+      name: "index_truffler_backfill_spends_on_ledger", if_not_exists: true
+  end
+end

data/lib/generators/truffler/upgrade/templates/labels_search_covering_migration.rb.tt

@@ -0,0 +1,32 @@
+class CoverTrufflerLabelsForSearch < ActiveRecord::Migration<%= migration_version %>
+  disable_ddl_transaction!
+
+  COVERING = "index_truffler_labels_for_search_covering".freeze
+  COLUMNS = [ :record_type, :tenant_key, :label_key, :value ].freeze
+
+  def up
+    return unless postgres? && !covered?
+
+    add_index :truffler_labels, COLUMNS, include: [ :record_id ], name: COVERING, algorithm: :concurrently
+    remove_index :truffler_labels, name: "index_truffler_labels_for_search", algorithm: :concurrently, if_exists: true
+  end
+
+  def down
+    return unless postgres? && index_name_exists?(:truffler_labels, COVERING)
+
+    add_index :truffler_labels, COLUMNS, name: "index_truffler_labels_for_search", algorithm: :concurrently,
+      if_not_exists: true
+    remove_index :truffler_labels, name: COVERING, algorithm: :concurrently
+  end
+
+  private
+
+  def postgres?
+    connection.adapter_name.match?(/postg/i)
+  end
+
+  def covered?
+    index_name_exists?(:truffler_labels, COVERING) ||
+      select_value("SELECT indexdef FROM pg_indexes WHERE indexname = 'index_truffler_labels_for_search'").to_s.include?("INCLUDE")
+  end
+end

data/lib/generators/truffler/upgrade/upgrade_generator.rb

@@ -3,19 +3,65 @@ require "rails/generators/active_record"
 
 module Truffler
   module Generators
-    # Adds the tables a newer Truffler needs to an app installed with an older
-    # one. Each migration skips a table that already exists.
+    # Adds the tables and columns a newer Truffler needs to an app installed
+    # with an older one. Safe to rerun: a step is skipped when db/migrate
+    # already holds a migration of that name or the database already has
+    # what it adds, so only missing migrations are written. Each migration
+    # also skips a table, column, or index that already exists.
     class UpgradeGenerator < Rails::Generators::Base
       include ActiveRecord::Generators::Migration
 
       source_root File.expand_path("templates", __dir__)
 
+      class_attribute :schema_connection, default: -> { ActiveRecord::Base.connection }
+
       def create_backfill_spends_migration
-        migration_template "backfill_spends_migration.rb.tt", File.join(db_migrate_path, "create_truffler_backfill_spends.rb")
+        add_migration "backfill_spends_migration.rb.tt", "create_truffler_backfill_spends" do |connection|
+          connection.table_exists?(:truffler_backfill_spends)
+        end
+      end
+
+      def create_backfill_spends_tenant_key_migration
+        add_migration "backfill_spends_tenant_key_migration.rb.tt", "add_tenant_key_to_truffler_backfill_spends" do |connection|
+          connection.column_exists?(:truffler_backfill_spends, :tenant_key)
+        end
+      end
+
+      # Postgres only: the 0.1.5 covering index that makes label-only ranking
+      # an index-only scan. Other databases and installs that already have it
+      # skip the step.
+      def create_labels_search_covering_migration
+        add_migration "labels_search_covering_migration.rb.tt", "cover_truffler_labels_for_search" do |connection|
+          !connection.adapter_name.match?(/postg/i) || labels_search_covered?(connection)
+        end
       end
 
       private
 
+      def labels_search_covered?(connection)
+        connection.index_name_exists?(:truffler_labels, "index_truffler_labels_for_search_covering") ||
+          connection.select_value("SELECT indexdef FROM pg_indexes WHERE indexname = 'index_truffler_labels_for_search'").to_s.include?("INCLUDE")
+      end
+
+      def add_migration(template, name, &applied)
+        existing = self.class.migration_exists?(File.join(destination_root, db_migrate_path), name)
+        if existing
+          say_status :skip, "#{name} (#{File.basename(existing)} exists)", :yellow
+        elsif schema_applied?(&applied)
+          say_status :skip, "#{name} (already in the database)", :yellow
+        else
+          migration_template template, File.join(db_migrate_path, "#{name}.rb")
+        end
+      end
+
+      # Without a database (not configured, not created yet) every step counts
+      # as missing; the migrations themselves are no-ops where already applied.
+      def schema_applied?
+        yield schema_connection.call
+      rescue ActiveRecord::ActiveRecordError
+        false
+      end
+
       def migration_version
         "[#{ActiveRecord::Migration.current_version}]"
       end

data/lib/tasks/truffler.rake

@@ -27,38 +27,63 @@ namespace :truffler do
 
   describe_cursor = ->(cursor) { cursor.nil? ? "none" : cursor }
 
+  resolve_tenant = ->(value) { value.to_s.strip.presence }
+
+  describe_model = ->(model, tenant_key) { tenant_key ? "#{model.name} (tenant #{tenant_key})" : model.name }
+
   desc "Backfill stale, missing, and failed labels for a model, waiting out budget denials " \
-    "(SPEND_CAP=dollars or none; default config.backfill_spend_cap; MAX_DURATION=seconds; RESET_SPEND=1 for a fresh spend ledger)"
+    "(TENANT=key for one tenant; SPEND_CAP=dollars or none; default config.backfill_spend_cap; MAX_DURATION=seconds; " \
+    "RESET_SPEND=1 for a fresh spend ledger)"
   task :backfill, [ :model ] => :setup do |_, args|
     model = resolve_model.call(args[:model])
+    tenant_key = resolve_tenant.call(ENV.fetch("TENANT", nil))
+    name = describe_model.call(model, tenant_key)
     spend_cap = resolve_spend_cap.call(ENV.fetch("SPEND_CAP", nil))
     max_duration = resolve_max_duration.call(ENV.fetch("MAX_DURATION", nil))
     if ENV.fetch("RESET_SPEND", nil) == "1"
-      Truffler::Labeling::Backfill.reset_spend!(model)
-      puts "#{model.name}: fresh spend ledger for the current vocabulary version"
+      ledger_tenant = model.truffler_definition.ledger_tenant(tenant_key)
+      all_tenants = ledger_tenant.nil? && !model.truffler_definition.ledger_tenant("").nil?
+      Truffler::Labeling::Backfill.reset_spend!(model, tenant_key: ledger_tenant, all_tenants: all_tenants)
+      puts all_tenants ? "#{name}: fresh spend ledgers for every tenant" : "#{name}: fresh spend ledger for the current vocabulary version"
     end
     progress = lambda do |so_far, delay|
-      puts "#{model.name}: waiting #{format('%.1f', delay)}s for backfill budget (#{so_far.labeled} labeled, " \
+      puts "#{name}: waiting #{format('%.1f', delay)}s for backfill budget (#{so_far.labeled} labeled, " \
         "$#{format('%.6f', so_far.cost)} spent, cursor #{describe_cursor.call(so_far.cursor)})"
     end
-    result = Truffler::Labeling::Backfill.new(model, spend_cap: spend_cap).run(wait: true, max_duration: max_duration, progress: progress)
-    summary = "#{model.name}: #{result.status}, #{result.labeled} labeled in #{result.requests} requests, $#{format('%.6f', result.cost)}"
+    result = Truffler::Labeling::Backfill.new(model, tenant_key: tenant_key, spend_cap: spend_cap)
+      .run(wait: true, max_duration: max_duration, progress: progress)
+    summary = "#{name}: #{result.status}, #{result.labeled} labeled in #{result.requests} requests, $#{format('%.6f', result.cost)}"
     summary += ", cursor #{describe_cursor.call(result.cursor)}" if result.status == :paused
     puts summary
   end
 
-  desc "Print a model's labeling counts by status and staleness"
-  task :status, [ :model ] => :setup do |_, args|
-    model = resolve_model.call(args[:model])
-    puts model.name
+  print_status = lambda do |model, tenant_key = nil|
+    puts describe_model.call(model, tenant_key)
     Truffler::Labeling::Backfill.status(model).each { |key, count| puts format("  %-9s %d", key, count) }
-    if Truffler::Records::BackfillSpend.available?
-      ledger = Truffler::Labeling::Backfill.spend(model)
+    ledger_tenant = model.truffler_definition.ledger_tenant(tenant_key)
+    if Truffler::Records::BackfillSpend.available? && Truffler::Records::BackfillSpend.tenant_ledgers? &&
+        model.truffler_definition.ledger_tenant("") && ledger_tenant.nil?
+      puts format("  %-9s %s", "spent", "per tenant; pass TENANT=key")
+    elsif Truffler::Records::BackfillSpend.available?
+      ledger = Truffler::Labeling::Backfill.spend(model, tenant_key: ledger_tenant)
       cap = Truffler.config.backfill_spend_cap
       puts format("  %-9s $%.6f in %d requests (vocabulary %s, cap %s)", "spent", ledger&.spent_usd.to_f, ledger&.requests.to_i,
-        Truffler::Labeling::Backfill.ledger_version(model).first(12), cap ? format("$%.2f", cap) : "none")
+        Truffler::Labeling::Backfill.ledger_version(model, ledger_tenant).first(12), cap ? format("$%.2f", cap) : "none")
     else
       puts format("  %-9s %s", "spent", "not tracked across runs; run bin/rails g truffler:upgrade && bin/rails db:migrate")
     end
   end
+
+  desc "Print a model's labeling counts by status and staleness (every registered Truffler model when none is named; " \
+    "TENANT=key for a tenant's spend ledger)"
+  task :status, [ :model ] => :setup do |_, args|
+    if args[:model].to_s.strip.empty?
+      Rails.application.eager_load! if defined?(Rails.application) && Rails.application
+      models = Truffler.registry.models.select { |model| model.try(:truffler_definition) }.sort_by(&:name)
+      abort "No Truffler models are registered" if models.empty?
+      models.each(&print_status)
+    else
+      print_status.call(resolve_model.call(args[:model]), resolve_tenant.call(ENV.fetch("TENANT", nil)))
+    end
+  end
 end

data/lib/truffler/benchmark/runner.rb

@@ -223,7 +223,7 @@ module Truffler
         failures = @dataset.injections.filter_map do |injection|
           clean = stored[injection.clean_id]
           twin = stored[injection.record.id]
-          changed = (clean.keys | twin.keys).reject { |key| Metrics.agree?(clean[key], twin[key], tolerance) }
+          changed = (clean.keys | twin.keys).reject { |key| Metrics.agree?(clean.fetch(key, 0.0), twin.fetch(key, 0.0), tolerance) }
           next if changed.empty?
 
           { "fixture" => injection.id, "clean_id" => injection.clean_id, "twin_id" => injection.record.id,

data/lib/truffler/clients/evaluator.rb

@@ -0,0 +1,44 @@
+module Truffler
+  module Clients
+    # Wraps a host client whose `evaluate(state:, schema:)` takes a schema
+    # object (it reads `schema.questions`) and returns an evaluation object
+    # with `answers`, `model`, and `input_tokens` (or a `usage` hash) readers
+    # instead of a hash, such as Cora's TypeSafeClient and its
+    # TypeSafe::Evaluation. The schema also reads like the question hash, so
+    # a client that treats it as a Hash works too. The pinned model is passed
+    # when the host's method accepts `model:`.
+    #
+    #   config.client = Truffler::Clients::Evaluator.new(TypeSafeClient.new)
+    class Evaluator < Callable
+      Schema = Data.define(:questions) do
+        delegate :[], :each, :keys, :size, :empty?, :as_json, :to_json, to: :questions
+
+        def ids
+          questions.keys
+        end
+
+        def to_h
+          questions
+        end
+      end
+
+      def perform(state:, questions:, model:)
+        response(super(state: state, questions: Schema.new(questions: questions), model: model))
+      end
+
+      private
+
+      def response(evaluation)
+        return evaluation if evaluation.is_a?(Hash)
+
+        tokens = evaluation.try(:input_tokens) || usage_tokens(evaluation.try(:usage))
+        { "answers" => evaluation.answers.to_h, "model" => evaluation.try(:model),
+          "usage" => ({ "input_tokens" => tokens } if tokens) }.compact
+      end
+
+      def usage_tokens(usage)
+        usage.is_a?(Hash) ? usage.with_indifferent_access[:input_tokens] : usage.try(:input_tokens)
+      end
+    end
+  end
+end

data/lib/truffler/configuration.rb

@@ -17,6 +17,14 @@ module Truffler
     attr_reader :lenses
     attr_accessor :encoding_deadline, :rerank_depth, :rerank_chunk_size, :rerank_max_field_chars, :smart_thresholds,
       :smart_run_ttl, :smart_candidate_pool, :broadcaster
+    # Generic nouns that are never required keywords on their own (Search::Filler).
+    attr_accessor :filler_words
+    # ->(model, tenant_key) { true/false } for tenant-scoped models; nil indexes every tenant.
+    attr_accessor :tenant_enabled
+    # :tenant (a spend ledger per tenant for scoped models) or :app (one per model).
+    attr_accessor :backfill_spend_cap_scope
+    # Choice options below this probability store no row and read as 0.0 (nil stores every option).
+    attr_accessor :choice_min_probability
 
     def initialize(env: ENV)
       @model = "jev-latest"
@@ -50,6 +58,10 @@ module Truffler
       @smart_run_ttl = 15.minutes
       @smart_candidate_pool = 200
       @broadcaster = nil
+      @filler_words = Search::Filler::DEFAULT_WORDS.dup
+      @tenant_enabled = nil
+      @backfill_spend_cap_scope = :tenant
+      @choice_min_probability = 0.05
     end
 
     def client