query_translator 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: a760f47dec0d0d1a2169009be88518918d804a74a7b697f7ea63e55090d74602
4
+ data.tar.gz: cb92f84e6179b0e3e319fd3903fc7519786e2937a695bb72bc97413f8936001d
5
+ SHA512:
6
+ metadata.gz: 672432dad8a0b91af3b3e93b9a189cfbbd825c70eb5f785271d2c219574a835645d3c82d7a43dcce1fbe915afabca58a29a7ca6f52539a87762faf6e58185c98
7
+ data.tar.gz: 47485f0add0360ee44d66ae23bb0e4a88b795ffc60be700a7ff27167c2b31bb1449d4a0343a96d8da02d2d45b5d7f29402b67812d56f3ed0dc855316b9c9b2d2
data/AGENTS.md ADDED
@@ -0,0 +1,518 @@
1
+ ---
2
+ version: 1.2.0
3
+ updated: 2026-11-21
4
+ ---
5
+
6
+ # QueryTranslator — integration guide for coding agents
7
+
8
+ You are wiring `query_translator` into a Ruby/Rails application so its users can search in plain language. This file
9
+ tells you how to do that correctly. Follow it step by step and do not improvise around the rules marked **MUST**.
10
+
11
+ The human-facing overview is in [README.md](README.md). This file is the authority when the two differ.
12
+
13
+ ---
14
+
15
+ ## 1. What the gem does and does not do
16
+
17
+ **Does:** takes free text plus a list of resources (kinds of search) and their filters, asks an LLM through RubyLLM to
18
+ fill in a strict JSON form, checks every returned filter against the list, casts values to their types, and returns a
19
+ `QueryTranslator::Result`: which resource, which filters, how confident, what did not fit, and an optional
20
+ clarification question with answer options.
21
+
22
+ **Does not:** query your database, call your API, render anything, cache, authorize, rate-limit, or know anything about
23
+ your domain beyond what your filter descriptions say. All of that is the app's job and is described below.
24
+
25
+ Data flow:
26
+
27
+ ```text
28
+ user text ─► app builds resources ─► Translator.call ─► LLM (structured output, schema with enums)
29
+ ◄─ answer form
30
+ FilterCheck: known names only, values cast, wrong ones reported
31
+ ◄─ Result(resource, filters, confidence, unmapped, errors, question, options)
32
+ app ─► question? → show options, re-translate "<text> — <option>"
33
+ ─► unsure? → show the normal filter form prefilled with result.filters
34
+ ─► otherwise → call / redirect to result.path with result.filters (through the normal index endpoint)
35
+ ```
36
+
37
+ ---
38
+
39
+ ## 2. Hard rules
40
+
41
+ 1. **MUST** keep the LLM key in the app's credentials (or ENV through the project's credentials helper). Never in code,
42
+ specs, fixtures or logs.
43
+ 2. **MUST** name every filter exactly as the API endpoint accepts the query param. The gem returns those names as hash
44
+ keys and the app puts them into the URL unchanged.
45
+ 3. **MUST** take enum values from code (model `enum` definitions, constants), never from database rows of a production
46
+ system you can't see. Lists that live in the database (categories, tags) are built at request time inside the app.
47
+ 4. **MUST** send results through the normal index endpoint and its authorization/policy scope. The gem only builds
48
+ filters; it never widens what a user may see.
49
+ 5. **MUST NOT** pass filter names or values to `send`, `public_send`, `constantize`, `order(params...)` or SQL without
50
+ the endpoint's own whitelist. The gem guarantees names and enum values come from your list, but keep the endpoint's
51
+ whitelist (`PERMITTED_ATTRIBUTES`) as the second wall.
52
+ 6. **MUST** handle all three outcomes: `clarify?`, `low_confidence?`, confident. Showing results for an unsure answer
53
+ without a hint is a bug.
54
+ 7. **MUST** rescue `QueryTranslator::Error` and fall back to the normal filter UI (web) or a 503 with a plain sentence
55
+ (API). Never show the provider's error message to users.
56
+ 8. **MUST NOT** call the real model in the app's test suite. Stub `QueryTranslator::Translator.call`.
57
+ 9. **MUST** describe every boolean filter as something that is switched on (see §6.4). The gem only ever sends `true`.
58
+ 10. **MUST NOT** edit the gem's prompt to fit one app. App-specific wording goes into filter descriptions, resource
59
+ descriptions or `config.extra_instructions`.
60
+
61
+ ---
62
+
63
+ ## 3. Public API reference
64
+
65
+ ### 3.1 Configuration
66
+
67
+ ```ruby
68
+ QueryTranslator.configure do |config|
69
+ config.model = 'claude-haiku-5-5' # default; any RubyLLM model id with structured output support
70
+ config.confidence_threshold = 0.7 # default; Result#low_confidence? is true below it
71
+ config.max_query_length = 500 # default; longer text is cut before it reaches the model
72
+ config.extra_instructions = nil # default; text appended to the prompt as "More rules from the app"
73
+ end
74
+
75
+ QueryTranslator.config.model # read anywhere
76
+ ```
77
+
78
+ RubyLLM itself is configured by the app (provider key, timeouts):
79
+
80
+ ```ruby
81
+ RubyLLM.configure do |config|
82
+ config.anthropic_api_key = Rails.application.credentials.dig(:anthropic, :api_key)
83
+ config.request_timeout = 15
84
+ config.max_retries = 1
85
+ end
86
+ ```
87
+
88
+ The translation runs inside a web request, and RubyLLM's defaults (300 s per attempt, 3 retries) can hold a server
89
+ thread for many minutes when the provider is slow. Both settings are global: if the app already uses RubyLLM for long
90
+ jobs, agree the values with the developer.
91
+
92
+ Haiku is the default on purpose: the gem was tuned on it across eight projects. If Haiku handles a project, larger
93
+ models do too; do not switch models to fix a bad filter description.
94
+
95
+ ### 3.2 `QueryTranslator::Filter`
96
+
97
+ ```ruby
98
+ QueryTranslator::Filter.new(name:, type:, description:, values: nil, multiple: false)
99
+ ```
100
+
101
+ | field | meaning |
102
+ | --- | --- |
103
+ | `name` | the API query param, exactly (`by_status`, `by_price_gte`, `order_by[start_date]`) |
104
+ | `type` | one of `string integer decimal date datetime boolean enum` |
105
+ | `description` | one sentence the model reads; see §6 |
106
+ | `values` | allowed values; **required** for `enum`, ignored otherwise |
107
+ | `multiple` | `true` when the endpoint accepts several values (`by_x[]=a&by_x[]=b`) |
108
+
109
+ `Filter.new` raises `ArgumentError` for an unknown type or an `enum` without values. That happens when the resources
110
+ are built, so a broken list fails loudly at the first request, not silently.
111
+
112
+ ### 3.3 `QueryTranslator::Resource`
113
+
114
+ ```ruby
115
+ QueryTranslator::Resource.new(name:, path:, description:, filters:)
116
+ ```
117
+
118
+ `name` is what the model picks (`venues`, `bookings`); `path` is the index endpoint the result points to;
119
+ `description` tells the model when to choose this resource. Both are plain `Data` objects (immutable, `#with` works).
120
+
121
+ ### 3.4 `QueryTranslator::Translator`
122
+
123
+ ```ruby
124
+ result = QueryTranslator::Translator.call(query: text, resources: [res_a, res_b], today: Date.current)
125
+ ```
126
+
127
+ - `today` defaults to `Date.today`; pass the app's `Date.current` so "tomorrow" follows the app's time zone.
128
+ - A blank query returns `Result.empty` without calling the model.
129
+ - Raises `QueryTranslator::Error` when RubyLLM fails (network, provider error) or the model returns no JSON.
130
+ `RubyLLM::ConfigurationError` (no key) and `RubyLLM::ModelNotFoundError` (unknown model id) are **not** wrapped:
131
+ they are setup errors and should crash loudly.
132
+
133
+ ### 3.5 `QueryTranslator::Result`
134
+
135
+ | member | meaning |
136
+ | --- | --- |
137
+ | `resource` | the chosen `Resource` or `nil` |
138
+ | `path` | `resource&.path` |
139
+ | `filters` | `{ "name" => cast_value }`; an array for `multiple` filters |
140
+ | `confidence` | the model's own score, 0..1 |
141
+ | `unmapped` | parts of the request, in the user's words, that no filter can express |
142
+ | `errors` | what `FilterCheck` threw out (unknown filter, bad value, several values for a one-value filter, `false` for a boolean) |
143
+ | `question`, `options` | a clarification question and 2–4 answer options, in the request's language; empty strings/arrays when none |
144
+ | `clarify?` | a question is present |
145
+ | `low_confidence?` | `confidence < threshold` **or** no filters **or** anything unmapped **or** any error |
146
+
147
+ Cast values: `string` → stripped `String`, `integer` → `Integer` (base 10), `decimal` → `Float`, `date` → `Date`,
148
+ `datetime` → `Time` (ISO 8601), `boolean` → `true`, `enum` → `String` from `values`. Rails turns them into URL params
149
+ (`Date#to_param` → `2026-11-22`, arrays → `name[]=…`). A `Time` becomes `2026-11-01 00:00:00 UTC`; call `#iso8601` on
150
+ it first when the API or the client expects ISO strings.
151
+
152
+ ### 3.6 `QueryTranslator::FilterCheck`
153
+
154
+ Usable on its own, e.g. to validate filters that came from somewhere else:
155
+
156
+ ```ruby
157
+ check = QueryTranslator::FilterCheck.call(resource: res, filters: { 'by_status' => %w[draft paid], 'x' => 1 })
158
+ check.filters # => { "by_status" => ["draft", "paid"] }
159
+ check.errors # => ["unknown filter x"]
160
+ ```
161
+
162
+ `nil` values are skipped silently.
163
+
164
+ ---
165
+
166
+ ## 4. Integration, step by step
167
+
168
+ The examples below come from venue_hub, which keeps business objects under `business/`, models under `data/` and
169
+ search services on `action_scope`. In another layout keep the same separation: resources and translation in a
170
+ business object, thin controllers.
171
+
172
+ ### Step 1 — Gemfile
173
+
174
+ ```ruby
175
+ gem 'query_translator', '~> 0.1'
176
+ ```
177
+
178
+ Run `bundle install`. Never commit a `path:` source: CI and servers don't have it.
179
+
180
+ ### Step 2 — key and initializers
181
+
182
+ 1. Ask the developer to add the key to the credentials of development and of every deployed environment
183
+ (`anthropic: api_key:`), not test: specs stub the translator, and without a key a forgotten stub fails loudly
184
+ instead of spending money. You do not create or read credential keys yourself.
185
+ 2. `config/initializers/ruby_llm.rb` — provider key from credentials and short timeouts (see §3.1).
186
+ 3. `config/initializers/query_translator.rb` — model, threshold, length, and `extra_instructions` with domain facts
187
+ the model cannot guess (currency, units, market), one or two sentences.
188
+
189
+ ### Step 3 — inventory the searchable endpoints
190
+
191
+ For every index endpoint a user might want to search in words:
192
+
193
+ 1. Find the controller action (`app/controllers/api/**`), its route (`config/routes.rb`), the search service it calls
194
+ (`app/services/search/*.rb`), and the permitted params (`PERMITTED_ATTRIBUTES`, `params.permit(...)`).
195
+ 2. Read the model (`data/*.rb` or `app/models/*.rb`): enums, custom scopes, and what `action_scope` generates
196
+ (`Model.options_for_search` lists generated scopes; read the gem source when unsure).
197
+ 3. Note who may call the endpoint (guest, client, owner, admin) and what scope the policy applies.
198
+ 4. Pick the 2–5 resources users actually ask about. Not every index needs to be searchable in words.
199
+
200
+ Some APIs expose a metadata endpoint (for example `GET /api/v1/metadata`) listing every search service's filter
201
+ options: a quick way to see the real param names and enum values.
202
+
203
+ ### Step 4 — map params to filters
204
+
205
+ Common param conventions (the `action_scope` style) and how to describe them:
206
+
207
+ | API param | filter type | description pattern |
208
+ | --- | --- | --- |
209
+ | `by_<col>` on an enum column | `enum`, `values:` enum keys, `multiple: true` if arrays are accepted | "Booking status" |
210
+ | `by_<col>` on a string column | `string` | "Exact deal number, e.g. D-0042" — say **exact** |
211
+ | `by_<col>_match` | `string` | "Part of the city name" — say **part of** |
212
+ | `by_search_query` / `by_free_text` / `q` | `string` | "Free-text search by name, code and email" — list the fields |
213
+ | `by_<col>_gte` / `_lte` (numbers) | `integer` / `decimal` | "Lowest price in USD", "Highest price in USD" |
214
+ | `by_<col>_gte` / `_lte` (date column) | `date` | "Due on or after this date" |
215
+ | `by_<col>_gte` / `_lte` (datetime column) | `datetime` | "Created at or after this moment" |
216
+ | `<col>_from` / `<col>_to` | `date` | if `_to` excludes the day itself, **say so**: "before this date (the date itself is not included)" |
217
+ | `with_<thing>` / `has_<thing>` | `boolean` | "Only invoices past their due date" — see §6.4 |
218
+ | `sort_by_<col>` | `enum`, `values: %w[asc desc]` | "Sort by expected value" |
219
+ | `order_by[<col>]` (nested hash) | `enum`, name `"order_by[<col>]"`, `values: %w[asc desc]` | one filter per sortable field |
220
+ | `sort=<word>` | `enum` with the words | "Order of the results: newest, name or lowest price" |
221
+ | `by_<x>_within_days`, `by_stale_days` | `integer` | "Decision expected within this many days from today" |
222
+
223
+ Skip, unless the API accepts a human-readable value:
224
+
225
+ - foreign keys (`by_user`, `by_project`, `by_<association>`): the model can't know ids;
226
+ - `_gt` / `_lt` duplicates of `_gte` / `_lte`; exact-match datetimes; `_gte` on enum integers;
227
+ - params the gem cannot express (§9): geo tuples, positional pairs (`range[]=start&range[]=end`), nested hashes other
228
+ than one-key `order_by[...]`.
229
+
230
+ **Database-driven lists** (small shared lists the staff manages: categories, regions) become an `enum` of ids built
231
+ at request time, with the id–name pairs in the description, and hierarchy shown when the scope includes children.
232
+ Never build one from per-tenant or per-user records, names users type, or people: every name goes into the prompt of
233
+ every call and can come back in a clarification question.
234
+
235
+ ```ruby
236
+ def category_filters
237
+ return [] if categories.empty? # an enum without values raises
238
+
239
+ [ filter('by_category_tree', 'enum',
240
+ "Kind of place by category id. A category includes its subcategories, so pick the broader one unless " \
241
+ "the request names an exact kind: #{categories.map { |c| category_name(c) }.join(', ')}",
242
+ categories.map { |c| c.id.to_s }) ]
243
+ end
244
+
245
+ def category_name(category)
246
+ [ "#{category.id} #{category.name}", ("(in #{category.parent_id})" if category.parent_id) ].compact.join(' ')
247
+ end
248
+ ```
249
+
250
+ ### Step 5 — the business objects
251
+
252
+ Neither object writes to the database. venue_hub keeps both as **Actions** and its controllers call them; where the
253
+ project's rules let controllers call only Operations, make `Translate` an Operation that returns the same hash. Suggested files (replace `NaturalSearches` with the project's naming):
254
+
255
+ ```text
256
+ business/natural_searches/actions/build_resources.rb # returns [Resource, ...]
257
+ business/natural_searches/actions/translate.rb # cache + gem call + app-specific clarifications
258
+ ```
259
+
260
+ `BuildResources` builds `QueryTranslator::Resource` objects with private helpers (`venue_filters`, `filter(...)`),
261
+ reading constants (`Search::X::SORTS`, `Model.statuses.keys`, `Model::AMENITIES`) so the list follows the code.
262
+
263
+ `Translate` (sketch, adapt names):
264
+
265
+ ```ruby
266
+ class NaturalSearches::Actions::Translate < BaseAction
267
+ attr_reader :query, :venues
268
+
269
+ def call
270
+ translation
271
+ end
272
+
273
+ private
274
+
275
+ def translation
276
+ @translation ||= Rails.cache.fetch(cache_key, expires_in: 1.day) { answer(translate) }
277
+ end
278
+
279
+ def translate
280
+ QueryTranslator::Translator.call(query: query, resources: NaturalSearches::Actions::BuildResources.call,
281
+ today: Date.current)
282
+ end
283
+
284
+ def answer(result)
285
+ { resource: result.resource&.name, path: result.path, filters: result.filters, confidence: result.confidence,
286
+ low_confidence: result.low_confidence?, unmapped: result.unmapped, question: result.question,
287
+ options: result.options }
288
+ end
289
+
290
+ def cache_key
291
+ [ 'natural_search', Date.current.iso8601, Digest::SHA256.hexdigest(query.to_s.squish.downcase) ]
292
+ end
293
+ end
294
+ ```
295
+
296
+ - Cache a plain hash: it serializes cleanly to Redis and to JSON.
297
+ - The **date is part of the cache key** because relative dates ("on Saturday") change meaning every day.
298
+ - The key holds only the date and the text, so `BuildResources` must return the same list for every user. If it ever
299
+ depends on the user or role, add that to the key.
300
+ - Cache only the translation. Never cache search results here.
301
+ - App-specific questions that need the database belong here, after the gem: e.g. venue_hub asks "Which state?" when
302
+ the translated city has published venues in several states, and that question **wins** over a city question the
303
+ model may have asked (its options come from real data).
304
+
305
+ ### Step 6 — controllers
306
+
307
+ API (returns the ready call; the client makes the second request itself):
308
+
309
+ ```ruby
310
+ class Api::V1::NaturalSearchesController < Api::V1::BaseController
311
+ allow_guests # only if the searched endpoints are public
312
+
313
+ rescue_from QueryTranslator::Error do |error|
314
+ Rails.logger.error("Natural search failed: #{error.message}")
315
+ error_response('Service Unavailable', I18n.t('natural_searches.unavailable'), 503)
316
+ end
317
+
318
+ def index
319
+ render json: { natural_search: NaturalSearches::Actions::Translate.call(query: params[:query].to_s,
320
+ venues: scoped_collection) }
321
+ end
322
+
323
+ private
324
+
325
+ def record_class
326
+ Venue # the Authorizer concern needs a model for its policy check
327
+ end
328
+ end
329
+ ```
330
+
331
+ Web (redirects to the normal page with the filters, or renders the question):
332
+
333
+ ```ruby
334
+ rescue_from QueryTranslator::Error do |error|
335
+ Rails.logger.error("Natural search failed: #{error.message}")
336
+ redirect_to venues_path(query: params[:query].to_s.squish), alert: t('natural_searches.unavailable'),
337
+ status: :see_other
338
+ end
339
+
340
+ def index
341
+ @query = params[:query].to_s.squish
342
+ @search = NaturalSearches::Actions::Translate.call(query: @query, venues: scoped_collection)
343
+ return if @search[:question].present? # render the question page
344
+
345
+ redirect_to result_path, alert: (t('.unsure') if @search[:low_confidence]), status: :see_other
346
+ end
347
+ ```
348
+
349
+ - `result_path` maps the resource to the matching HTML page and merges the filters plus the original text, to show
350
+ "You searched for …". venue_hub carries the text in `query`; if the target page already filters by a param of that
351
+ name (a full-text search), use another one, such as `natural_query`. Keep the mapping in a private method; it is
352
+ routing, not business logic.
353
+ - The question page shows the question and one link per option to the same action with `"#{@query} — #{option}"` as
354
+ the text, plus the edit box (same `maxlength`) and a link to the plain filter form. The gem keeps only the first
355
+ `max_query_length` characters, so trim `@query` to leave room for the answer, or a long request loses it and the
356
+ same question comes back.
357
+ - Show "what we searched for" chips on the result page: one chip per active filter param with a link that removes it.
358
+ - Keep the text box limit in sync: `maxlength: QueryTranslator.config.max_query_length`.
359
+ - Add every user-facing string to every locale file the project keeps.
360
+
361
+ ### Step 7 — tests in the app
362
+
363
+ Helper that builds a real `Result` from the app's own resources:
364
+
365
+ ```ruby
366
+ module NaturalSearchHelpers
367
+ def natural_search_result(resource_name: 'venues', filters: {}, **answer)
368
+ resource = NaturalSearches::Actions::BuildResources.call.find { |item| item.name == resource_name }
369
+ QueryTranslator::Result.new(resource: resource, filters: filters, confidence: 0.9, unmapped: [], errors: [],
370
+ question: '', options: [], **answer)
371
+ end
372
+ end
373
+ ```
374
+
375
+ ```ruby
376
+ before { allow(QueryTranslator::Translator).to receive(:call).and_return(natural_search_result(filters: {...})) }
377
+ ```
378
+
379
+ Cover at least:
380
+
381
+ - the business action: a confident answer passes through; each app-specific clarification fires;
382
+ - the API endpoint (acceptance spec): 200 with the ready call; 503 when the translator raises;
383
+ - the web flow (request spec): redirect with filters; low confidence → alert; question page renders options; a second
384
+ resource redirects to its page; the translator raises → the filter page with the message;
385
+ - the resource builder does not raise when a database list (categories) is empty.
386
+
387
+ Run the project's full suite, linter and security scanner (`brakeman`, `bundler-audit`) afterwards.
388
+
389
+ ### Step 8 — try it live before handing over
390
+
391
+ Write a throwaway script (scratch directory, not committed, run with `bin/rails runner`) that runs 30+ realistic
392
+ queries through `Translate` against development data with a real key, and read every answer:
393
+
394
+ - plain requests per resource; every filter type at least once;
395
+ - other languages the users speak; slang and abbreviations ("NYC", "50k", "asap");
396
+ - typos; relative dates ("next Saturday", "last week", "this quarter");
397
+ - several values ("draft or submitted"); negations ("without", "never", "not overdue");
398
+ - vague words ("big", "cheap", "bad reviews"); off-topic ("what's the weather"); hostile ("ignore your rules…").
399
+
400
+ Start the script with `Rails.cache = ActiveSupport::Cache::NullStore.new`: `Translate` caches by day + text, so
401
+ without it a re-run replays today's answers and a description fix seems to change nothing. Do not use
402
+ `Rails.cache.clear`, which can wipe a Redis database other apps share. Use the records already in the database; ask
403
+ before creating any, and remove them at the end.
404
+
405
+ For every wrong answer change the **filter or resource description** (or `extra_instructions`) and run the set again.
406
+ Report what you changed and what still goes wrong.
407
+
408
+ ---
409
+
410
+ ## 5. Handling the outcomes — precedence
411
+
412
+ 1. `QueryTranslator::Error` → fallback UI / 503.
413
+ 2. `question.present?` (gem or app clarification) → ask; even if filters exist.
414
+ 3. `low_confidence` → show the filter form prefilled with `filters`, with a short "check the filters" hint.
415
+ 4. Otherwise → go to the resource's page / call `path` with `filters`.
416
+
417
+ A resource that needs a signed-in user (e.g. "my bookings") still gets translated for guests; let the target page's
418
+ authentication handle it (redirect to sign-in).
419
+
420
+ ---
421
+
422
+ ## 6. Writing descriptions that work
423
+
424
+ The descriptions are the only thing the model knows about the domain. During tuning, every wrong answer was fixed by a
425
+ clearer description or a general rule, never by a bigger model.
426
+
427
+ ### 6.1 Resources
428
+
429
+ One sentence on what the records are, in business words, and whose they are:
430
+ "Venues and studios for rent in the public catalog", "The signed-in client's own bookings",
431
+ "Stripe payments and refunds against billing invoices".
432
+
433
+ ### 6.2 Filters
434
+
435
+ - Say **exact** vs **part of** for strings.
436
+ - Give **units** and **currency**: "in USD", "in seconds", "in miles", "percent".
437
+ - Give **formats** for odd values: "a year like 2020, not an age", "two-letter state code".
438
+ - For ranges say **lowest/highest**, **on or after/on or before**, and whether an end date is excluded.
439
+ - For enums with cryptic values explain them in the description: "Priority: 0 is P0 (highest) … 3 is P3",
440
+ "net30 is an invoice due in 30 days".
441
+ - For free-text filters list the fields searched.
442
+ - Keep it to one sentence. The model sees every filter of every resource on each call.
443
+
444
+ ### 6.3 What the model already does well (don't fight it)
445
+
446
+ Abbreviations and slang for places, amounts like "50k" or "10 grand", relative dates and periods (it sets both ends of
447
+ a range), converting hours/days into seconds (8-hour working days), "over 40 hours" → `_gte 144001`, exact values
448
+ through `_gte` + `_lte`, quarters, "this week" with exclusive ends when the description says so.
449
+
450
+ ### 6.4 Booleans
451
+
452
+ Many `with_*` scopes switch on for **any** non-blank value, so `with_x=false` returns the opposite of what the user
453
+ asked. The gem therefore treats every boolean as a switch: the prompt tells the model to send only `"true"`, and
454
+ `FilterCheck` drops a `false` and reports it (the result becomes unsure). Describe booleans as what they turn on:
455
+ "Only payments that have at least one refund". A negation ("never refunded") goes to another filter when one fits (a
456
+ status) or ends up in `unmapped` — the user then sees the filter form, not wrong results.
457
+
458
+ ### 6.5 `extra_instructions`
459
+
460
+ For facts that apply to every filter: currency, units, market, time zone conventions. Not for per-filter wording.
461
+
462
+ ---
463
+
464
+ ## 7. Security
465
+
466
+ - **Prompt injection:** the user's text goes in as the user message; the system prompt says it is data. The response
467
+ schema only allows your resource names and filter names, and enum values are checked again in Ruby. A hostile
468
+ request ends up with no filters and low confidence.
469
+ - **Allowed values only:** enum values come from your list; never forward a free-form value into code that dispatches
470
+ on it (`send(kind)`). If the project already has such a scope, flag it to the developer.
471
+ - **Authorization** stays in the index endpoints (§2.4). The natural-search endpoint only returns filters.
472
+ - **Cost:** every uncached request is one LLM call (fractions of a cent on Haiku). Cache by day + text. If the
473
+ endpoint is public, ask the developer for a rate limit on the natural-search actions themselves; a site-wide page
474
+ limit is far above what paid calls need.
475
+ - **Privacy:** the model sees the user's text and your filter descriptions (including database-driven names like
476
+ categories). Don't put personal data into descriptions.
477
+
478
+ ---
479
+
480
+ ## 8. Testing the gem itself (when you change it)
481
+
482
+ - `bundle exec rake` runs specs and RuboCop.
483
+ - `spec/query_translator/realistic_queries_spec.rb` replays real model answers from `spec/cassettes/` (VCR). Any change
484
+ to the prompt or the schema changes the request body, so playback fails until you re-record: put
485
+ `ANTHROPIC_API_KEY=...` (a key that belongs to an Anthropic workspace) into the gem's gitignored `.env`, delete
486
+ `spec/cassettes/`, run `bundle exec rspec`, then check that no key is in the recordings
487
+ (`grep -ri "x-api-key\|sk-ant" spec/cassettes` must print nothing).
488
+ - Without a key the cassettes replay and nothing is recorded; non-200 answers are never recorded.
489
+
490
+ ---
491
+
492
+ ## 9. Known limitations
493
+
494
+ - One resource per request. "Show me deals and contacts" picks one or asks.
495
+ - No OR across different filters ("in Chicago or under $100").
496
+ - No geo search, positional pairs or arbitrary nested hashes; only one-key `order_by[...]`.
497
+ - Dates and datetimes come back without a time zone of the app (datetimes in UTC); `today` is the only clock input.
498
+ - The model's `confidence` varies a little between calls; that is why `low_confidence?` also uses hard rules.
499
+ - Free-form string enums in the database (a Jira status, an environment name) are best described as `string` with
500
+ examples, not as `enum`, unless the values are fixed in code.
501
+
502
+ ---
503
+
504
+ ## 10. Hand-over checklist
505
+
506
+ - [ ] Gem from rubygems (not `path:`) in the Gemfile, lockfile updated.
507
+ - [ ] Key in the credentials of development and every deployed environment (not test), both initializers present
508
+ (short RubyLLM timeout and retries), model set explicitly.
509
+ - [ ] Every filter name equals a permitted param of its endpoint; enum values come from code.
510
+ - [ ] Booleans described as switches; ranges say lowest/highest and whether ends are included.
511
+ - [ ] Database-driven enums come from shared lists the staff manages and skip themselves when empty.
512
+ - [ ] Translation cached by date + normalized text.
513
+ - [ ] API endpoint returns the ready call; 503 on `QueryTranslator::Error`, which is logged.
514
+ - [ ] Web flow: question page, low-confidence hint, redirect with filters, "what we searched for" chips, the filter
515
+ page on `QueryTranslator::Error`.
516
+ - [ ] All strings in every locale.
517
+ - [ ] App specs stub the translator; full suite, linter and security scan green.
518
+ - [ ] 30+ live queries tried and the results reported to the developer.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2026 AndriyAndriyovuch
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.