rails-ai-context 5.35.1 → 5.35.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +198 -0
  3. data/CONTEXT.md +2 -2
  4. data/CONTRIBUTING.md +1 -1
  5. data/docs/COMPATIBILITY.md +5 -0
  6. data/docs/RELEASING.md +3 -1
  7. data/docs/SECURITY.md +60 -32
  8. data/docs/TOOLS.md +20 -8
  9. data/docs/TROUBLESHOOTING.md +33 -6
  10. data/docs/adr/0008-query-runs-only-where-it-can-guard.md +3 -3
  11. data/lib/rails_ai_context/database_family/mysql_runtime.rb +2 -3
  12. data/lib/rails_ai_context/database_family/postgresql_plan.rb +30 -15
  13. data/lib/rails_ai_context/database_family/postgresql_runtime.rb +2 -2
  14. data/lib/rails_ai_context/database_family/sqlite_runtime.rb +39 -11
  15. data/lib/rails_ai_context/database_family.rb +9 -5
  16. data/lib/rails_ai_context/introspectors/declared_constant.rb +19 -7
  17. data/lib/rails_ai_context/introspectors/job_introspector.rb +6 -6
  18. data/lib/rails_ai_context/introspectors/model_introspector.rb +5 -12
  19. data/lib/rails_ai_context/introspectors/multi_database_introspector.rb +10 -1
  20. data/lib/rails_ai_context/introspectors/performance_introspector.rb +0 -1
  21. data/lib/rails_ai_context/payload.rb +13 -3
  22. data/lib/rails_ai_context/secret_columns.rb +65 -39
  23. data/lib/rails_ai_context/serializers/markdown_serializer.rb +1 -1
  24. data/lib/rails_ai_context/sql_gate.rb +68 -14
  25. data/lib/rails_ai_context/tools/get_context.rb +4 -4
  26. data/lib/rails_ai_context/tools/get_controllers.rb +5 -1
  27. data/lib/rails_ai_context/tools/onboard.rb +3 -6
  28. data/lib/rails_ai_context/tools/query.rb +1 -1
  29. data/lib/rails_ai_context/tools/runtime_info.rb +8 -4
  30. data/lib/rails_ai_context/tools/safe_call.rb +2 -3
  31. data/lib/rails_ai_context/tools/session_context.rb +2 -3
  32. data/lib/rails_ai_context/version.rb +1 -1
  33. data/server.json +1 -1
  34. metadata +1 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 534fe60bd5d8bcd7bc0e2df04cbaed12794c259bc663e969625dd6ccd2437237
4
- data.tar.gz: 0a5d91ff87c7c960539e6e60f3fea7d238cf3fcd79dc56905fea7a248b9d912a
3
+ metadata.gz: 40f690d7055b2ed848a3e2c381b9735a04a1e36daaf17b9b3be6ef8fb123b4e7
4
+ data.tar.gz: ef4a68d5e3cfaefa285f78f8292059fef488788c5368d6cf110dae1bfe6a0947
5
5
  SHA512:
6
- metadata.gz: e44f58f7a5728ab9cbb04429f49cfe4abd685ce37c6fa6cd2b59e30afdb27443a11db5e9a07733bc135572a5876993a1449ef04bfba206c15f6ff71b4b870baf
7
- data.tar.gz: d689733a334294b5f22a9ca839ee6171f881b2bcd7fdf3818097166ce8a2989ef5ceefd702b357d436c4796e7a3f93c8a11ce4adb3de85f9a8fe7191193153c5
6
+ metadata.gz: c4fd6c004371524c34c681922c2d7b8a2f0e5de79e93071a6a2cc34a93d54e70ce1c3565333109c4975cdb6763ea8f1db5adaba9dd8a39444a87e6329cb55367
7
+ data.tar.gz: 71d3fff40bd09fc9f92955c8f816fd455bf4cf3c3c36ea46eda3c4296bafa5bd3abc31290e717de53d11062b2fb96355bbb961267be18e121d7261c20ad7c59c
data/CHANGELOG.md CHANGED
@@ -5,6 +5,204 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [5.35.2] - 2026-10-14
9
+
10
+ ### Security
11
+
12
+ - **On MySQL and MariaDB a quote character inside a string does not change
13
+ which databases a statement is read to name.** The secret column rule reads
14
+ the columns of every database a statement names. In 5.35.1, the first release
15
+ with that read, a quote character inside a string could keep the rule from
16
+ seeing that the statement named another database. A secret column of a table
17
+ or of a view in that database then came back. Each name before a dot is now
18
+ read backward from its dot, so nothing written before the name changes how it
19
+ is read. The refusals are the ones 5.35.1 gives. This gap is in 5.35.1
20
+ alone: earlier versions read no other database, which the 5.35.1 notes cover.
21
+ - **On MySQL and MariaDB a `[` outside a string or a quoted name is refused.**
22
+ MariaDB reads `[name]` as a quoted name when the session's sql_mode holds
23
+ `MSSQL`, and the statement checks read a bracket as an ordinary character. In
24
+ a session with that mode the checks and the server could read one statement
25
+ two ways. `rails_query` does not read the session's mode, so it now refuses
26
+ the bracket on both servers. 5.35.1 and every earlier version had the gap,
27
+ on an app whose connection runs in that mode.
28
+ - **On MySQL and MariaDB a query leaves no value behind on the app's
29
+ connection.** A user variable assignment (`:=`) and `LAST_INSERT_ID` called
30
+ with an argument each set a value that stayed on the pooled connection after
31
+ the query returned, which a read-only transaction does not stop. Both are now
32
+ refused. 5.35.1 and every earlier version ran them.
33
+ - **On PostgreSQL what a foreign scan sends to the remote server is read like
34
+ what the plan runs itself.** A condition, a sort or a group on a secret
35
+ column of a `postgres_fdw` foreign table ran on the remote server, where the
36
+ plan checks did not read it. So a statement could test the column's value,
37
+ also through a view that renames the column and under a `column_name`
38
+ option. The remote statement the plan prints is now read, and such a
39
+ statement answers "Blocked: the query's plan uses the secret column
40
+ `<column>`". Fetching the column returns `[FILTERED]` as before. A statement
41
+ whose plan holds a foreign scan costs one more catalog read. Only the remote
42
+ statement of `postgres_fdw` is read: another foreign data wrapper prints
43
+ keys of its own. 5.35.1 and every earlier version had the gap.
44
+ - **On PostgreSQL a join that the remote server runs is traced to its
45
+ tables.** When `postgres_fdw` sends a whole join to the remote server, the
46
+ plan names the tables with their aliases in one line of the scan, which the
47
+ plan checks did not read, so the secret column rule did not reach those
48
+ tables. The tables and aliases of that line are now read like any scan's: a
49
+ secret column is `[FILTERED]` in place and a whole row is refused, as for a
50
+ local table. 5.35.1 and every earlier version had the gap.
51
+ - **On PostgreSQL the check for a secret column passed through a subquery
52
+ reads a name with two spaces in a row or a dotted capital I.** In 5.35.1
53
+ that check missed a secret column whose name holds either one. A statement
54
+ that passes such a column through a subquery, a CTE or a set operation is
55
+ now refused as for any other secret column. 5.35.0 refused it, so the gap is
56
+ in 5.35.1 alone.
57
+ - **On SQLite a virtual table built on a relation that is not read is refused
58
+ however its definition is written.** 5.35.1 read a relation's name in the
59
+ `content=` option of a full-text table and in the table a vocabulary lists.
60
+ A virtual table that named the relation another way ran, and returned the
61
+ rows or words of a refused view, a shadow table or the page modules. The
62
+ module and every argument can now name a relation, except the columns a
63
+ full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab`
64
+ table ends with (`row`, `col` or `instance`). An argument with `=` is read
65
+ whole, whatever its name, and comments are taken out first. The module is
66
+ read after the table's own name, so a name that spells a module changes
67
+ nothing: 5.35.1 read such a name as the module, and the rule for a virtual
68
+ table that holds a secret column did not reach that table. The answers are
69
+ the ones 5.35.1 gives. The gap needs a virtual table the app itself created.
70
+ 5.35.0 and earlier returned such a table's rows too.
71
+ - **On SQLite a relation whose name holds an apostrophe is matched where it
72
+ is written in single quotes.** SQLite reads a name in single quotes where a
73
+ table is expected, with the apostrophe doubled inside. The secret column
74
+ rule spelled a name in double quotes and in backticks and missed that
75
+ spelling, so a statement or a definition that used it was not held to the
76
+ rule for that relation. 5.35.1 and every earlier version had the gap, on an
77
+ app with a relation so named.
78
+
79
+ ### Changed
80
+
81
+ The three refusals below go out under a patch number. Each closes a guard gap,
82
+ and the maintainer chose 5.35.2 for them, outside step 1 of `docs/RELEASING.md`.
83
+
84
+ - **On MySQL and MariaDB a statement with a user variable assignment (`:=`) is
85
+ refused.** The answer is "Blocked: a user variable assignment (`:=`) is not
86
+ run. It leaves a value on the pooled connection after the query returns,
87
+ which a read-only transaction does not stop." A statement that numbers its
88
+ rows with `@n := @n + 1` ran before and is refused now. Use a window
89
+ function, `ROW_NUMBER() OVER (...)`, which MySQL 8 and MariaDB 10.2 and
90
+ later have. A `:=` inside a string or a quoted name is data and runs, with
91
+ one case apart: a statement with a backslash before a quote inside a string.
92
+ The second reading of a backslash ends the string at that quote, and where
93
+ that leaves the `:=` outside a string the statement is refused. Write the
94
+ quote doubled.
95
+ - **On MySQL and MariaDB `LAST_INSERT_ID` with an argument is refused.** The
96
+ answer is "Blocked: LAST_INSERT_ID with an argument is not run. It sets a
97
+ value that stays on the pooled connection after the query returns, which a
98
+ read-only transaction does not stop. LAST_INSERT_ID() with no argument
99
+ runs." The call is matched on the raw text, like the session functions, so
100
+ it is refused inside a string too.
101
+ - **On MySQL and MariaDB a statement with a `[` outside a string or a quoted
102
+ name is refused.** The answer is "Blocked: a `[` outside a string or a
103
+ quoted name is not run. MariaDB reads it as the start of a quoted name when
104
+ the session's sql_mode holds MSSQL, and rails_query does not read the
105
+ session's mode. Quote a name in backticks." MySQL, and MariaDB outside that
106
+ mode, run no statement that holds one. In a session with that mode, write
107
+ `` `name` `` for `[name]`. A `[` inside a string or a backticked name runs,
108
+ with one case apart: a statement with a backslash before a quote inside a
109
+ string, as in `'O\'Brien'` beside `REGEXP '[a-z]'`. The second reading of a
110
+ backslash ends the string at that quote, and where that leaves the `[`
111
+ outside a string the statement is refused. Such a statement ran before.
112
+ Write the quote doubled.
113
+
114
+ ### Fixed
115
+
116
+ - **A write that follows a `WITH` clause is refused as the write it is.**
117
+ `WITH ... INSERT INTO ...` answered "Blocked: SELECT INTO creates a table".
118
+ It now answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` after a
119
+ `WITH` clause are named the same way. A real `SELECT ... INTO` keeps its
120
+ sentence.
121
+ - **A refusal names a view, a table and a relation as the database spells
122
+ them.** It named them case folded, with a run of spaces as one space, so a
123
+ view called `Hq Odd View` was named `hq odd view`, which no relation is
124
+ called. Two names that differ by a run of spaces alone are one name to the
125
+ rule, and the refusal names the first of them it read.
126
+ - **A view refused over a table that is not read is said to name it.** The
127
+ answer ended "its definition reads `<table>`, which is not read", and the
128
+ rule tests whether the definition names the table. It now ends "its
129
+ definition names `<table>`, which is not read".
130
+ - **The answer for a view that leads too deep drops its count.** It reads "The
131
+ view `<view>` leads through more databases than rails_query follows, so the
132
+ secret columns it could read cannot be told and no statement that names it
133
+ runs." The "(3 rounds)" after "follows" told a user nothing.
134
+ - **The note for a row limit of one says "1 row shown".** It said "1 rows
135
+ shown; the query returned more".
136
+ - **On PostgreSQL a sequence can be queried.** `SELECT last_value FROM
137
+ <sequence>` and `SELECT * FROM <sequence>` ran on 5.35.0, and 5.35.1 answered
138
+ that the sequence's columns could not be read. A sequence is now read like a
139
+ table that holds no secret column.
140
+ - **On PostgreSQL a whole row of a table with no secret column runs when
141
+ another table shares its alias or its name with a number after it.** 5.35.1
142
+ refused such a statement when a subquery used the same alias for a table
143
+ that holds a secret column, or when a table named like `name_2` stood beside
144
+ it, and the refusal named the other table. 5.35.0 ran it. The number the
145
+ planner adds to an alias is now taken off for the members of an Append or a
146
+ Merge Append alone, and a refusal over a whole row names the relation whose
147
+ row the statement reads.
148
+ - **On PostgreSQL a block comment inside a block comment is read as the server
149
+ reads it.** PostgreSQL nests block comments, and the gate ended one at the
150
+ first `*/`, so `SELECT 1 /* a /* b */ c */ AS n` answered PostgreSQL's
151
+ syntax error. It now runs.
152
+ - **On SQLite a statement that runs past `query_timeout` answers the timeout
153
+ when a writer takes the lock while it runs.** It answered that the database
154
+ was locked and the query did not run. The query process now takes its read
155
+ lock before it prepares the statement, so no lock stops the statement from
156
+ there on. The locked answer is given for a lock the query process is still
157
+ waiting on when it gives up or when the deadline passes.
158
+ - **On a SQLite file database a statement that names something the app's
159
+ connection alone holds runs on that connection.** While that connection
160
+ holds a temp object or an attachment, a statement runs there when it names a
161
+ temp object, `sqlite_temp_master` or `sqlite_temp_schema`, the `temp` schema
162
+ or an attached database, or calls a pragma function (`pragma_table_info(...)`,
163
+ `pragma_table_list` and the rest) or `dbstat`, since an argument can name any
164
+ of them.
165
+ The query process has none of them, so it answered an empty list, the rows of
166
+ a main table of the same name, or "unknown database". The answer now ends
167
+ with the note that the query ran without a time limit. A name is matched
168
+ anywhere in the statement, in any letter case and inside a string too.
169
+ Finding this out takes two more reads before a statement on a file database
170
+ where a query process can run: eight where there were six.
171
+ - **The full generated file lists the first database's recent migrations
172
+ under its own name.** With a first database not named `primary` and another
173
+ one that is, the heading read `### Recent Migrations: primary`. It now names
174
+ the first database, as in `### Recent Migrations: animals`.
175
+ - **The database marks and the first database's own name do not depend on the
176
+ preset.** On `preset: standard`, or any `introspectors` list without
177
+ `multi_database`, `rails_get_schema`, `rails_onboard`,
178
+ `rails_performance_check` and the generated files lost the mark of a
179
+ database another environment configures, such as `cache (production only)`,
180
+ and called a first database with another name `primary`. They now read the
181
+ app databases list there, as on `full`, and `rails_onboard` names a
182
+ secondary's adapter.
183
+ - **The model, controller, job, mailer and channel lists hold a class only
184
+ while its own constant names it.** A class whose constant was removed stays
185
+ loaded until the garbage collector takes it, so a list could hold it in one
186
+ run and lose it in the next. Such a class is now left out. A constant still
187
+ waiting to autoload counts as naming its class. The lists of a class's
188
+ children follow the same rule: a model's STI children, the classes a base job
189
+ or mailer is inherited by, and the abstract bases.
190
+ - **A model file that raises before its class opens costs that model alone.**
191
+ A failing `require` at the top of the file, or a raise above the class line,
192
+ made the whole models section answer an error. The file is now named with
193
+ what it raised, and the other models stay listed.
194
+ - **The refusal over a secret column says where `query_allowed_columns` is
195
+ set on either install.** It said to add the name "in an initializer", which
196
+ does nothing on a standalone install. It now ends "add "x" to
197
+ query_allowed_columns in .rails-ai-context.yml (or in
198
+ config/initializers/rails_ai_context.rb where the app bundles the gem)."
199
+ - **`rails_runtime_info` says when table sizes or index usage could not be
200
+ read.** A read that failed or ran out of time left its part of the database
201
+ section out with no word. The section now says "_Table sizes were not read:
202
+ the read failed or took longer than query_timeout._", and at `detail:
203
+ "full"` "_Index usage was not read: the read failed or took longer than
204
+ query_timeout._"
205
+
8
206
  ## [5.35.1] - 2026-10-13
9
207
 
10
208
  ### Security
data/CONTEXT.md CHANGED
@@ -167,7 +167,7 @@ A gem the context calls out because it shapes how the app is built, counted only
167
167
 
168
168
  ## Payload
169
169
 
170
- The introspection context hash, in the sense consumers read it. `Payload` is the reading side: a section guard and pinned list readers whose key pairs a spec checks against the producing introspector's own output, so a renamed key fails a test instead of silently emptying a section everywhere. Facts rendered from it live once (`SectionFacts`); membership judgments are applied at the introspector seam, and readers trust the payload rather than re-filtering it.
170
+ The introspection context hash, in the sense consumers read it. `Payload` is the reading side: a section guard and pinned list readers whose key pairs a spec checks against the producing introspector's own output, so a renamed key fails a test instead of silently emptying a section everywhere. Facts rendered from it live once (`SectionFacts`); membership judgments are applied at the introspector seam, and readers trust the payload rather than re-filtering it. One reader fills a section a preset leaves out: `Payload.multi_database` reads the app databases list when the `multi_database` introspector did not run, so a database's mark and the first database's own name do not depend on the preset.
171
171
 
172
172
  ## Standalone
173
173
 
@@ -235,7 +235,7 @@ What decides whether `rails_query` runs a statement: the checks on its text, and
235
235
 
236
236
  ## Served tool
237
237
 
238
- A tool the server answers for under the app's configuration: the built-ins `skip_tools` leaves, then each custom tool whose name no tool before it claims. Every surface reads the one set: `tools/list`, the CLI's list and lookup, the tool guide, the context files, `rails_get_context`'s includes and every tool name an answer prints. A tool outside it is never offered, run or pointed at. An answer or a guide step names or runs a tool only when the app serves the gem's own tool under that name: a custom tool in a built-in's place is listed, and nothing gives it the built-in's arguments. A custom tool named `rails_get_config` takes the built-in's place when `skip_tools` lists that name, and the **config sections** go with the built-in.
238
+ A tool the server answers for under the app's configuration: the built-ins `skip_tools` leaves, then each custom tool whose name no tool before it claims. An app's own class that inherits the gem's base tool counts with the built-ins once it is loaded, whether or not `custom_tools` lists it. Every surface reads the one set: `tools/list`, the CLI's list and lookup, the tool guide, the context files, `rails_get_context`'s includes and every tool name an answer prints. A tool outside it is never offered, run or pointed at. An answer or a guide step names or runs a tool only when the app serves the gem's own tool under that name: a custom tool in a built-in's place is listed, and nothing gives it the built-in's arguments. A custom tool named `rails_get_config` takes the built-in's place when `skip_tools` lists that name, and the **config sections** go with the built-in.
239
239
 
240
240
  ## Config section
241
241
 
data/CONTRIBUTING.md CHANGED
@@ -106,7 +106,7 @@ RAC_TEST_MYSQL_URL=trilogy://root:secret@127.0.0.1:3306/rac \
106
106
  BUNDLE_GEMFILE=gemfiles/database_servers.gemfile bundle exec rspec spec/lib/rails_ai_context/database_family/database_servers_spec.rb
107
107
  ```
108
108
 
109
- The specs create and drop tables and views whose names start with `rac_`, so point them at a scratch database.
109
+ The specs create and drop tables and views whose names start with `rac_`, and leave the `postgres_fdw` extension installed on PostgreSQL, so point them at a scratch database.
110
110
  CI's `database_servers` job runs them on PostgreSQL 16 in every leg. Rails 8.0 runs two legs: MySQL 8.0 with MariaDB 10.6, and MySQL 8.4 with MariaDB 11.4. Rails 7.0 runs MySQL 8.0 with MariaDB 10.6, and Rails 8.1 runs MySQL 8.4 with MariaDB 11.4.
111
111
  Rails 7.0 has no Trilogy adapter, so its leg runs PostgreSQL and mysql2 only.
112
112
  The job builds mysql2 against MariaDB's client library (`libmariadb-dev`, through `BUNDLE_BUILD__MYSQL2`), which signs in to MySQL and to MariaDB. A MySQL client from 9.0 on cannot sign in to MariaDB.
@@ -312,6 +312,11 @@ Proof sources:
312
312
  the tool's own checks timed on Ruby 3.4 for statements up to a megabyte
313
313
  and on Ruby 3.1 for statements of 50 kilobytes; and the unit suite and the
314
314
  E2E harness on Linux, macOS and Windows.
315
+ In the v5.35.2 closing review: the statement files of the v5.35.1 QA sent
316
+ through the tool again on PostgreSQL 16, MySQL 8.4, MariaDB 11.4 and SQLite,
317
+ on the v5.35.1 code and on the fixed code, with every changed answer traced
318
+ to its fix; and for each fix an example that fails on the v5.35.1 code, on a
319
+ real server where the fix is a server's.
315
320
  2. Non-crash coverage for every built-in tool including `get_view` in
316
321
  `spec/e2e/in_gemfile_install_spec.rb`'s full-tool sweep; output correctness
317
322
  (ivar cross-check, render-form detection, partial interfaces) verified
data/docs/RELEASING.md CHANGED
@@ -32,7 +32,9 @@ pushed.
32
32
  commit anyway.
33
33
  4. **Prose that names the last version goes stale where no check can see it.**
34
34
  `docs/COMPATIBILITY.md` names the release its proof sources come from. Grep
35
- the tree for the previous version before committing.
35
+ the tree for the previous version before committing. The repository's About
36
+ text on GitHub names the tool count, and no check reads it:
37
+ `gh repo edit --description` changes it.
36
38
  5. **The copy-paste prompts name a version floor.** `docs/PROMPTS.md` (and the
37
39
  README's copy of the recommended prompt) tells the AI tool to install
38
40
  `>= 5.35.0`, the first release with the install flags. If the release that
data/docs/SECURITY.md CHANGED
@@ -50,14 +50,15 @@ flowchart LR
50
50
  Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`) reads it once and answers either the text to run or a refusal with the reason. It holds every check in this layer and, on PostgreSQL, the plan checks of Layer 1b. Comments and quotes are read the way the connection's database family reads them:
51
51
 
52
52
  - **Refuses a statement over 1,000,000 bytes**, before any other check: "Blocked: the statement is <N> bytes long, and rails_query checks a statement of 1000000 bytes at most." The length is checked before any scan of the text and before a connection opens. The text checks below read the whole statement, in time that grows in step with its length, and they give up when they have run longer than `query_timeout`: "Blocked: the checks on the statement's text ran longer than query_timeout (<N> seconds), so the statement was not run. Send a shorter statement."
53
- - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
53
+ - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. PostgreSQL nests block comments, so there a block comment inside a block comment needs a closer of its own, and the gate reads it that way: `SELECT 1 /* a /* b */ c */ AS n` runs. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
54
54
  - **On MySQL and MariaDB, reads a backslash in a string both ways**: MySQL reads a backslash in a string as an escape, unless the session's sql_mode holds `NO_BACKSLASH_ESCAPES`, and `rails_query` does not read the session's mode. So the text checks read a statement that holds a backslash once for each reading, and a statement that either reading refuses is refused. A statement with no backslash reads the same both ways and is checked once. A refusal that the second reading alone gives ends "That is how MySQL reads the statement when the session's sql_mode holds NO_BACKSLASH_ESCAPES, and rails_query does not read the session's mode. Write a quote inside a string doubled, so that both readings agree."
55
+ - **On MySQL and MariaDB, refuses a `[` outside a string or a quoted name**: MariaDB reads `[name]` as a quoted name when the session's sql_mode holds `MSSQL`, and `rails_query` does not read the session's mode. The checks in this list read a bracket as an ordinary character, so in that mode a name in brackets, or text after one, would be read one way here and another way by the server. MySQL, and MariaDB outside that mode, run no statement that holds one. The answer is "Blocked: a `[` outside a string or a quoted name is not run. MariaDB reads it as the start of a quoted name when the session's sql_mode holds MSSQL, and rails_query does not read the session's mode. Quote a name in backticks." A `[` inside a string or a backticked name is data and runs, unless a backslash before a quote in the statement leaves the bracket outside a string in the second reading of a backslash (above): then the statement is refused until the quote is written doubled
55
56
  - **Blocks write keywords**: INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, CREATE, GRANT, REVOKE, SET, COPY, MERGE, REPLACE. The keyword check (and the `SELECT ... INTO` table-creation check) reads **quote-masked text**, so a keyword inside a string literal is data: `... WHERE notes LIKE '%update%'` and `... = 'put into box'` are reads. `REPLACE` is matched only as the statement; the string function `replace(col, a, b)` every database has (a `REPLACE` immediately followed by `(`) is allowed. `SHOW CREATE TABLE` / `SHOW CREATE VIEW` are allowed despite the `CREATE` (see below)
56
57
  - **Blocks lock clauses**: FOR UPDATE, FOR SHARE, FOR NO KEY UPDATE, FOR KEY SHARE and LOCK IN SHARE MODE, with whatever follows them (NOWAIT, SKIP LOCKED, OF, WAIT). On PostgreSQL the clause is read on quote-masked text, so `LIKE '%ready for update%'` is data: its read-only transaction refuses a locking read anyway. MySQL's read-only transaction lets `FOR SHARE` and `LOCK IN SHARE MODE` through, so there and on SQLite the raw text decides
57
58
  - **Blocks SHOW forms that tell of other sessions, replication and logs, on MySQL and MariaDB**: `SHOW GRANTS`, `SHOW PROCESSLIST` and `SHOW FULL PROCESSLIST`, `SHOW BINLOG`, `SHOW BINARY LOGS` and `SHOW BINARY LOG STATUS`, `SHOW RELAYLOG EVENTS`, the `SHOW SLAVE` and `SHOW SLAVES` forms, `SHOW MASTER`, the `SHOW REPLICA` and `SHOW REPLICAS` forms, `SHOW ALL SLAVES STATUS` and `SHOW ALL REPLICAS STATUS`, `SHOW ENGINE ... STATUS`, `SHOW PROFILE` and `SHOW PROFILES`, `SHOW OPEN TABLES`, `SHOW EXPLAIN FOR` and `SHOW ANALYZE FOR`, `SHOW USER_STATISTICS`, `SHOW CLIENT_STATISTICS` and the `SHOW WSREP_` forms. The answer is "Blocked: sensitive SHOW command". They are matched on the raw text, so the words inside a string literal are refused too
58
59
  - **Blocks `EXPLAIN FOR CONNECTION`, on MySQL and MariaDB**: it shows the plan of the statement another session is running. It is refused also as `DESCRIBE` or `DESC`, with a `FORMAT`, and inside a version comment. The answer is "Blocked: EXPLAIN FOR CONNECTION is not run. It shows the plan of the statement another session is running." It is read on quote-masked text, so a string literal that holds the words runs
59
60
  - **Blocks SHOW with a WHERE clause**, on every family. MySQL's statement timeout holds `SELECT` only, so a `SHOW ... WHERE` filter would run past `query_timeout`. `SHOW ... LIKE` runs. The answer names the query to use: "Blocked: SHOW with a WHERE clause is not run, because MySQL's statement timeout does not hold it. SHOW ... LIKE runs. For any other filter, SELECT from information_schema, for example `SELECT table_name FROM information_schema.tables WHERE table_schema = DATABASE()`."
60
- - **Blocks SELECT INTO**: prevents table creation via SELECT. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
61
+ - **Blocks SELECT INTO**: prevents table creation via SELECT. After a `WITH` clause an `INTO` can belong to a write, and such a statement is answered as that write: `WITH ... INSERT INTO ...` answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` are named the same way. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
61
62
  - **Blocks file and network functions**, each on the database family that has it. PostgreSQL: `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file`, `lo_import`, `lo_export`, `dblink*` and their relatives. MySQL and MariaDB: `LOAD DATA` and `LOAD_FILE`. SQLite: `load_extension`. They are checked before and after comment stripping
62
63
  - **Blocks functions that return a whole table as one value**: PostgreSQL's `table_to_xml`, `query_to_xml`, `cursor_to_xml`, `schema_to_xml`, `database_to_xml` and their `_to_xmlschema` and `_to_xml_and_xmlschema` relatives. Such a value carries every column of the table, its secret columns included, under a name no column check reads. The name with `(` after it is refused on the raw text: "Blocked: function <name> returns a whole table as one value, secret columns included. SELECT the columns you need from the table itself."
63
64
  - **Blocks relations and settings that hold sampled values, other sessions' statements or stored credentials**: no check on a column's name applies to what they hold, so they are not read at all. "Relations and settings that are never read" below lists them by database family, and each name is refused on the family that has it
@@ -65,9 +66,10 @@ Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`)
65
66
  - **On SQLite, blocks the tables that hold another table's rows under other names**: the page modules, a shadow table of a virtual table, and a virtual table that holds a secret column. See Layer 4
66
67
  - **Blocks a function called by a quoted name**: a quoted name can spell any function, past every check in this list that reads a function's name. A name in double quotes, backticks or brackets with `(` after it is refused on every family, a `U&"..."` name with `UESCAPE` included. The answer is "Blocked: a quoted name is followed by a parenthesis, the way a function is called. rails_query reads a function's name to decide whether it runs, so write the name without quotes." A quoted column inside a call (`count("id")`) and a quoted table before `USING (...)` run
67
68
  - **Blocks session-effecting / administrative functions**: a read-only transaction does not stop these, and their effect lingers on the pooled connection, so a call is matched on the raw text: the name with `(` after it is refused, inside a string literal too. A bare name inside a literal is data. PostgreSQL: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_restore_point`, replication-slot and -origin functions, `pg_logical_emit_message`, `pg_advisory_lock*`, `set_config`, `pg_notify`, `txid_current` and their relatives; MySQL and MariaDB: `GET_LOCK`, `RELEASE_LOCK`, `RELEASE_ALL_LOCKS`, `IS_FREE_LOCK`, `IS_USED_LOCK`, `MASTER_POS_WAIT`, `SOURCE_POS_WAIT`. Each list is held to its own family. This name list is the first layer; on PostgreSQL the plan's VOLATILE-function check (below) closes the rest of the family
68
- - **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead."
69
+ - **On MySQL and MariaDB, blocks a statement that sets a value on the session**: the value stays on the pooled connection after the query returns, which a read-only transaction does not stop. A user variable assignment (`:=`) is read on quote-masked text, so one inside a string or a quoted name is data: "Blocked: a user variable assignment (`:=`) is not run. It leaves a value on the pooled connection after the query returns, which a read-only transaction does not stop." To number rows, use a window function (`ROW_NUMBER() OVER (...)`), which MySQL 8 and MariaDB 10.2 and later have. `LAST_INSERT_ID` called with an argument is read on the raw text, like the session functions above, so it is refused inside a string too: "Blocked: LAST_INSERT_ID with an argument is not run. It sets a value that stays on the pooled connection after the query returns, which a read-only transaction does not stop. LAST_INSERT_ID() with no argument runs."
70
+ - **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead." A stated limit: one name in parentheses after `GLOB`, `MATCH` or `OVER` that follows a subquery (`(SELECT ...) GLOB (title)`, `sum((SELECT 1)) OVER (w)`) has the shape of a list after a subquery's alias and is refused. Each of those words can name a table or a CTE on SQLite, so the gate cannot let them through without letting an alias list through. Write the name with no parentheses (`GLOB title`), or write the window inline (`OVER (ORDER BY id)`)
69
71
  - **Blocks multi-statements**: a semicolon with anything after it, read on quote-masked text so a `;` inside a string literal or a `DO $$ ... ; ... $$` block body is not mistaken for a second statement (a `DO` block is then refused plainly as a disallowed statement)
70
- - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data
72
+ - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data. A stated limit: a quoted name that spells the word, such as the alias in `SELECT 1 AS "union"`, is refused on every family, since the check reads quoted names. Give the alias another name
71
73
  - **Blocks secret column references**: see Layer 4 - a query whose text names a secret column, directly or through an alias or expression, is refused before execution
72
74
  - **Blocks a `*` inside a subquery of a statement that names a relation with a secret column, on MySQL, MariaDB and SQLite**: see Layer 4
73
75
  - **Allows only**: SELECT, WITH, SHOW, EXPLAIN, DESCRIBE, DESC. `SHOW CREATE TABLE` and `SHOW CREATE VIEW` are allowed so schema can be inspected; every other `SHOW CREATE` (`USER`, whose output includes credential hashes, `DATABASE`, `FUNCTION`, ...) keeps its `CREATE` and is refused. A returned definition is passed through secret redaction before it is shown, so a credential embedded in it - a FEDERATED table's `CONNECTION='mysql://user:pass@host/db'` - comes back as `mysql://[FILTERED]@host/db`
@@ -80,8 +82,8 @@ The textual layer cannot see a whole-row serialisation (`SELECT row_to_json(u.*)
80
82
  - reads a materialized view that the view rule of Layer 4 refuses, directly or through a view over it;
81
83
  - reads one of the relations or functions Layer 1 refuses by name (`pg_stats`, `pg_stat_activity`, `pg_settings` and the rest), which a view or an escaped name can reach where the text check saw nothing;
82
84
  - calls a function that returns a whole table as one value, or reads a setting that can hold a password (`current_setting` with a refused name, or with a name the plan does not show as one constant);
83
- - holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations;
84
- - **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included;
85
+ - holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations. The planner numbers an alias a statement uses twice (`p_1` for the `p` inside a subquery), and that name stands for its own relation alone. The number is taken off only for the members of an Append or a Merge Append, which is how the partitions of one table are named. The refusal names the relation whose row the statement reads;
86
+ - **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included. The statement a `postgres_fdw` foreign scan sends to the remote server (the plan's `Remote SQL`) is read the same way, so a condition, a sort or a group on a secret column of a foreign table is refused where the remote server would run it. A column that carries another name on the remote server (the `column_name` option) is told by one more catalog read, made only for a plan that holds a foreign scan. A stated limit: only the remote statement of `postgres_fdw` is read, and another foreign data wrapper prints keys of its own. A join that `postgres_fdw` runs on the remote server has one scan for all its tables, which names them with their aliases in its `Relations` line: they are read from there, so a whole row of one is refused and a column of one is traced and filtered like a local table's;
85
87
  - **passes a secret column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused, since no redaction could reach them. A column that a scan hands to an aggregate or a join is counted only when that node names it, so `count(*)` over a table with a secret column runs;
86
88
  - names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn`, `pg_current_wal_insert_lsn`, `pg_current_wal_flush_lsn`, `pg_last_wal_receive_lsn`, `pg_last_wal_replay_lsn`, `pg_last_xact_replay_timestamp`). The functions the statement's text names are checked beside those the plan prints, since a plan prints no row of a `VALUES` list and no `LIMIT` expression. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and outside the read-only list. This closes the admin-function family without a name list that has to grow with every PostgreSQL release.
87
89
 
@@ -110,7 +112,7 @@ A statement that names one of these on the database family that has it is refuse
110
112
 
111
113
  Each name belongs to one database family and is refused there alone. A PostgreSQL catalog name is not refused on MySQL, MariaDB or SQLite, and a MySQL name is not refused on PostgreSQL or SQLite. On its own family a name is matched as a whole word, so an app table that carries one (`slow_log` on MySQL) cannot be queried. The `information_schema` names in the MySQL row are matched only under that schema, so an app's own `user_statistics` table runs.
112
114
 
113
- The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. For an alias the answer ends " A name before a dot is read as that schema, so an alias named `sys` needs another name." A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
115
+ The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. Every such refusal outside double-quoted text, a real `FROM mysql.user` included, ends " A name before a dot is read as that schema, so an alias named `sys` needs another name.", with `mysql` where that is the name. A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
114
116
 
115
117
  On PostgreSQL the plan is checked for the same names, as relations and as functions, so a view over one is refused: `information_schema.user_mappings`, `information_schema.foreign_servers` and `pg_stat_subscription` are refused that way. `pg_authid`, `pg_shadow` and `pg_db_role_setting` are the exception. A view over them runs, and their `rolpassword` and `setconfig` columns are secret columns: `pg_roles`, `pg_user` and `information_schema.schemata` run, `SELECT rolconfig FROM pg_roles` comes back `[FILTERED]`, and an expression over either column is refused.
116
118
 
@@ -146,7 +148,7 @@ What bounds each statement a `rails_query` call sends:
146
148
  | The query, and `explain: true` | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | the child process's deadline |
147
149
  | The reads beside it: the schema read behind the secret column rule, and on PostgreSQL the plan, its `pg_proc` lookup and the column trace | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | no statement limit; a wait on another writer ends at the connection's busy timeout |
148
150
 
149
- The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. A statement or a view definition that names another schema or database adds the reads of that one.
151
+ The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. On a SQLite file database where a query process can run, two more reads ask what the app's own connection alone holds, so eight reads come before the statement there. On PostgreSQL a plan that holds a foreign scan adds one catalog read. A statement or a view definition that names another schema or database adds the reads of that one.
150
152
 
151
153
  A read that runs out of that time ends the call with an error that says which side of the statement it was on:
152
154
 
@@ -160,6 +162,7 @@ Stated limits of the time bound:
160
162
  - **A SQLite statement on the app's own connection cannot be ended in the middle of one long step.** The paragraph on that path below says when a statement runs there.
161
163
  - **PostgreSQL has no client read timeout.** `statement_timeout` is the server's to keep. A PostgreSQL server that goes silent after it connects is not ended by the client.
162
164
  - **On MySQL a `SLEEP()` the statement limit interrupts is no timeout.** MySQL returns 1 from an interrupted `SLEEP()` and raises no error, so the statement answers its rows.
165
+ - **On Ruby 3.1 a malformed statement of tens of kilobytes is slow to report.** When the database's error message carries the bad text (an unterminated quote, a run of `[`), ActiveRecord's handling of that message is slow on a Ruby with no regexp match cache. A 50 KB statement took 17 to 31 seconds to answer its SQL error there, run on the app's own SQLite connection. The time is spent after the database answered, so no limit above ends it. Ruby 3.2 and later report the error at once, and on a SQLite file database the query process ends at the deadline.
163
166
 
164
167
  A database leased by name connects with the `connect_timeout` of its `config/database.yml` entry, or `query_timeout` rounded up to whole seconds when the entry sets none, so a server that never answers the connection ends the call. On MySQL and MariaDB such a connection also gets a `read_timeout` of twice `query_timeout` plus 5 seconds unless its entry sets one, so a server that goes silent after the handshake ends the call. On PostgreSQL `connect_timeout` is the only client bound. Taking the lease down does not replace the outcome of the call: its connections are thrown away and its pool removed, and an error there is not raised. The primary uses the app's own pool and its settings.
165
168
 
@@ -177,33 +180,52 @@ A SQLite database whose file is not there is refused by name before anything
177
180
  connects, with or without a `database` argument, since connecting would create
178
181
  the file. The answer names `bin/rails db:prepare`.
179
182
 
180
- A SQLite statement runs on the app's own connection, under `PRAGMA query_only = ON`,
181
- in four cases: an in-memory database, a platform without `fork`, a statement
182
- the child process could not prepare, and a transaction open on the app's
183
- connection. The third is a statement that needs a function, virtual-table
184
- module, collation, temporary table or encryption key only the app's own
183
+ A SQLite statement runs on the app's own connection, under
184
+ `PRAGMA query_only = ON`, in five cases: an in-memory database, a platform
185
+ without `fork`, a statement the child process could not prepare, a transaction
186
+ open on the app's connection, and a statement that names something the app's
187
+ connection alone holds. The third is a statement that needs a table, function,
188
+ virtual-table module, collation or encryption key only the app's own
185
189
  connection has. That case is told from which step failed in the child, and the
186
- error's text is not read. A statement the child prepared and that fails while it
187
- runs is answered with its SQL error and is not run again. The fourth is there
188
- because the child opens the file itself and would answer without the rows of
189
- the open transaction. On the
190
- app's own connection a statement that returns rows slowly ends with the timeout
191
- answer a few rows after `query_timeout` passes. The sqlite3 driver holds the
192
- interpreter lock while a statement steps, and an interrupt lands only between
193
- two rows, so the end comes late by the time those rows take. One long step
194
- cannot be ended there at all. So the table and EXPLAIN answers of such a run say it ran
195
- without a time limit. CSV output stays plain data.
190
+ error's text is not read. A statement the child prepared and that fails while
191
+ it runs is answered with its SQL error and is not run again. The fourth is
192
+ there because the child opens the file itself and would answer without the
193
+ rows of the open transaction.
194
+
195
+ The fifth is told before the child starts, by two reads on the app's
196
+ connection: its temp objects and its attached databases. When it holds any of
197
+ them, a statement runs there if it names one of them, the temp catalog
198
+ (`sqlite_temp_master`, `sqlite_temp_schema`) or the `temp` schema, or if it
199
+ calls a pragma function (`pragma_table_info(...)`, `pragma_table_list` and the
200
+ rest) or `dbstat`, whose argument can name any of them. The child has none of
201
+ these and would answer an empty list, the rows of a main table of the same
202
+ name, or "unknown database". A table that only an attached database holds
203
+ needs no match: the child cannot prepare a statement that reads it, which is
204
+ the third case. A stated limit: a name is matched anywhere in the statement's
205
+ text, in any letter case and inside a string too, so a statement that holds
206
+ such a name as plain data runs on the app's connection as well.
207
+
208
+ On the app's own connection a statement that returns rows slowly ends with the
209
+ timeout answer a few rows after `query_timeout` passes. The sqlite3 driver
210
+ holds the interpreter lock while a statement steps, and an interrupt lands only
211
+ between two rows, so the end comes late by the time those rows take. One long
212
+ step cannot be ended there at all. So the table and EXPLAIN answers of such a
213
+ run say it ran without a time limit: "This SQLite query ran without a time
214
+ limit: query_timeout needs a file-backed database, a platform with fork, and
215
+ nothing that only the app's own connection has." CSV output stays plain data.
196
216
 
197
217
  A SQLite database that another connection keeps locked is answered as locked:
198
218
  "The database is locked by another connection, so the query did not run. Run it
199
219
  again when the other connection has finished." It is an error response. The
200
220
  child process waits on a lock for 80% of `query_timeout`, so a lock that stays
201
221
  is answered before the timeout. The child tells its parent the moment it meets
202
- a lock, so a child that started late on a slow machine is answered as locked
203
- too, and again when it is past the lock, so a statement that then runs past
204
- the deadline answers the timeout. A lock met while the connection opens gets
205
- the locked answer. The reads on the app's own connection wait for that
206
- connection's busy timeout.
222
+ a lock, so a child that started late on a slow machine and still waits at the
223
+ deadline is answered as locked too. The child takes its read lock before it
224
+ prepares the statement and tells its parent once it holds it. No writer can
225
+ stop the statement from there on, so a statement that then runs past the
226
+ deadline answers the timeout, also when a writer asks for the lock while it
227
+ runs. A lock met while the connection opens gets the locked answer. The reads
228
+ on the app's own connection wait for that connection's busy timeout.
207
229
 
208
230
  On sqlite3 1.x a query does not run when another connection in the same
209
231
  process held an exclusive lock on a rollback-journal database as it started
@@ -243,11 +265,13 @@ The rule follows what a statement reaches. It starts from these columns, by data
243
265
 
244
266
  | Family | The columns the rule reads |
245
267
  |:-------|:---------------------------|
246
- | PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views and foreign tables. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
268
+ | PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views, foreign tables and sequences. A sequence holds no secret column, so `SELECT last_value FROM <sequence>` runs. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
247
269
  | MySQL and MariaDB | The base tables of the connection's database. Then those of every database the statement's text names. `mysql`, `sys`, `performance_schema` and `information_schema` are left out: their relations are refused by name or hold no app data |
248
270
  | SQLite | The tables of `main`, of `temp` and of every attached database |
249
271
 
250
- A schema or database is told by a name before a dot, and the server says which of those names are schemas. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows (3 rounds), so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
272
+ A schema or database is told by a name before a dot, and the server says which of those names are schemas. Each name is read backward from its dot, bare or in backticks or double quotes, so nothing written before the name changes how it is read, a quote character inside a string included. A name in brackets is not read: SQLite reads every schema whatever a statement names, and on MySQL and MariaDB Layer 1 refuses the bracket. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows, so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
273
+
274
+ A refusal names a view, a table or a relation as the database spells it. The rule itself compares names case folded, with a run of spaces as one space. So two names that differ by a run of spaces alone are one name to the rule, and a refusal names the first of them it read.
251
275
 
252
276
  The schema read fails closed. When the columns of the database cannot be read, no query runs on it and the answer is "The columns of the <name> database could not be read, so its secret columns cannot be told and no query runs on it." Only the primary falls back to the cached context's schema, and it is refused the same way when that has no tables either.
253
277
 
@@ -261,11 +285,13 @@ If one of your own columns merely looks secret (an `oauth_applications.secret`,
261
285
  config.query_allowed_columns = %w[secret]
262
286
  ```
263
287
 
288
+ On a standalone install, where no initializer runs, put the name under `query_allowed_columns:` in `.rails-ai-context.yml`.
289
+
264
290
  Results are redacted as well: a returned column comes back as `[FILTERED]` when the rule flags its **output name**, and on PostgreSQL when its origin is a secret base column - either by **provenance** (`PG::Result#ftable` / `#ftablecol`, mapped to the base column) or by the **plan's top-node `Output`**, whose i-th expression is result column i. The plan path catches a view (or inlined CTE/subquery/join) that renamed a secret column to a harmless name: `CREATE VIEW user_tokens AS SELECT id, api_token AS t FROM users; SELECT t FROM user_tokens` returns `t` as `[FILTERED]`. A quoted mixed-case secret column (`"Salt"`) is filtered in place the same way. So is a secret column under a wildcard over a partitioned table or an inheritance parent: each partition is read as its own relation, a result column is filtered when any of them hands a secret column up in that place, and a whole-row read is refused when any of them holds one. An expression beside the wildcard (`SELECT *, id + 1 AS next`) changes nothing: each plain result column keeps its table column, and each partition's expressions are checked as expressions. The arms of a `UNION ALL` inside a view are read the same way: each arm against its own table, so no arm stands for another. A result column whose table's column names cannot be read is filtered when that table holds a secret column. When the table cannot be told either, it is filtered when any table holds one. On SQLite a second result column of the same name comes back as `name:1`, and the rule reads that name without its suffix. `SHOW`, `DESCRIBE` and `EXPLAIN` output skips column-name redaction; the one exception is `SHOW CREATE TABLE` / `SHOW CREATE VIEW`, whose definition text is run through the shared secret redaction so a credential embedded in it is filtered.
265
291
 
266
292
  Some relations are refused whole, because no result column of them is traced to a table column. MySQL, MariaDB and SQLite trace none at all, so there two refusals stand in for the plan checks:
267
293
 
268
- - **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition reads `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
294
+ - **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition names `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
269
295
  - **A `*` inside a subquery is refused when the statement names a relation that holds a secret column.** With no plan to read, a value a wildcard returns could be compared or renamed unseen. The test is those two facts together: the statement has a wildcard inside a subquery, and it names such a relation. Which relation the wildcard reads is not told without a parser. The answer is "Blocked: a `*` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. On <family> a value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery." A CTE body and a derived table count. So does `(TABLE t)` on MySQL and MariaDB, the two families that have the statement and read no plan, with an answer of its own: "Blocked: `TABLE` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `TABLE t` reads every column of `t`, and on <family> a value read that way can be compared or renamed past the redaction, so SELECT the columns you need in the subquery." `EXISTS (SELECT * ...)` returns no column and `count(*)` reads none, so both run. A relation's name inside a string literal is data on MySQL and MariaDB. SQLite reads a string as a table's name wherever a name can stand, so there a string equal to such a relation's name counts as that name, wherever it stands: "Blocked: a `*` inside a subquery is not run in a statement that holds the string '<relation>'. SQLite reads a string as a table's name where a name can stand, and the table `<relation>` holds the secret column `<column>`. A value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery."
270
296
  - **On SQLite `IN` before a relation's name is held to the same rule.** `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, a wildcard subquery with no `*` written. It is refused wherever it stands when the statement names a relation that holds a secret column, or holds a string equal to such a relation's name: "Blocked: `IN` before a table's name is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, and a value read through such a wildcard can be compared past the redaction, so write the subquery with its columns named." `IN (...)` with a list or a subquery in parentheses runs.
271
297
 
@@ -277,7 +303,7 @@ On SQLite a table that holds another table's rows or words under other names is
277
303
  - **A shadow table of a virtual table.** "Blocked: `notes_content` is not read: it is a shadow table, where SQLite keeps what the virtual table `notes` stores under other column names. Query `notes`." The last sentence is left out when the virtual table itself is refused. Before SQLite 3.37, which cannot type a shadow table, a table whose name starts with a virtual table's name and an underscore is read as one.
278
304
  - **A virtual table that holds a secret column.** A search, a highlight or a snippet over it can return or confirm that column's values, so it is refused whole: "Blocked: `docs` is not read: it is a virtual table that holds the secret column `api_token`, and a search or a function over a virtual table reads a column by its position, with no name written." Only the columns a virtual table declares decide this. A full-text table has a hidden column of its own name, and that one does not count, so a table named `token_search` runs. A virtual table with no secret column runs.
279
305
  - **An `fts5vocab` or `fts4aux` table over such a table.** "Blocked: `docs_words` is not read: it lists the words of the full-text table `docs`, which holds the secret column `api_token`."
280
- - **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table is built on a relation through its `content=` option, and a vocabulary table through the table it lists the words of; a column that shares a view's name changes nothing. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
306
+ - **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table's definition can name a relation with its module and with every argument, except the columns a full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab` table ends with (`row`, `col` or `instance`): a column or a type that shares a view's name changes nothing. An argument with `=` is an option and is read whole, whatever its name. Comments are taken out before any of it is read, and the module is read after the table's own name, so a name that spells a module changes nothing. A vocabulary table also names the table it lists the words of. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
281
307
  - **A view over any of these**, by the view rule above.
282
308
 
283
309
  `dbstat` runs: it gives sizes and no stored value.
@@ -288,6 +314,8 @@ On every family a generated column whose expression reads a secret column is a s
288
314
 
289
315
  A stored procedure or function is a stated limit: its body is not read, so a function that returns a secret column's value returns it, and the value is filtered by its output name only.
290
316
 
317
+ A column is known by the name it has in this database, which is a stated limit too. A foreign table that gives a secret column of the remote table another local name (the `column_name` option), a column a trigger fills from a secret column and a table built from another are read by their own names, so their values come back unless that name is secret itself. `config.query_redacted_columns` takes such a name.
318
+
291
319
  The exemption covers the results too: an allowed name comes back unredacted. A
292
320
  column declared with `encrypts` is refused by name and stays `[FILTERED]` either
293
321
  way: an encrypted-at-rest attribute is not meant to be read back raw.