rails-ai-context 5.35.1 → 5.35.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +198 -0
- data/CONTEXT.md +2 -2
- data/CONTRIBUTING.md +1 -1
- data/docs/COMPATIBILITY.md +5 -0
- data/docs/RELEASING.md +3 -1
- data/docs/SECURITY.md +60 -32
- data/docs/TOOLS.md +20 -8
- data/docs/TROUBLESHOOTING.md +33 -6
- data/docs/adr/0008-query-runs-only-where-it-can-guard.md +3 -3
- data/lib/rails_ai_context/database_family/mysql_runtime.rb +2 -3
- data/lib/rails_ai_context/database_family/postgresql_plan.rb +30 -15
- data/lib/rails_ai_context/database_family/postgresql_runtime.rb +2 -2
- data/lib/rails_ai_context/database_family/sqlite_runtime.rb +39 -11
- data/lib/rails_ai_context/database_family.rb +9 -5
- data/lib/rails_ai_context/introspectors/declared_constant.rb +19 -7
- data/lib/rails_ai_context/introspectors/job_introspector.rb +6 -6
- data/lib/rails_ai_context/introspectors/model_introspector.rb +5 -12
- data/lib/rails_ai_context/introspectors/multi_database_introspector.rb +10 -1
- data/lib/rails_ai_context/introspectors/performance_introspector.rb +0 -1
- data/lib/rails_ai_context/payload.rb +13 -3
- data/lib/rails_ai_context/secret_columns.rb +65 -39
- data/lib/rails_ai_context/serializers/markdown_serializer.rb +1 -1
- data/lib/rails_ai_context/sql_gate.rb +68 -14
- data/lib/rails_ai_context/tools/get_context.rb +4 -4
- data/lib/rails_ai_context/tools/get_controllers.rb +5 -1
- data/lib/rails_ai_context/tools/onboard.rb +3 -6
- data/lib/rails_ai_context/tools/query.rb +1 -1
- data/lib/rails_ai_context/tools/runtime_info.rb +8 -4
- data/lib/rails_ai_context/tools/safe_call.rb +2 -3
- data/lib/rails_ai_context/tools/session_context.rb +2 -3
- data/lib/rails_ai_context/version.rb +1 -1
- data/server.json +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 40f690d7055b2ed848a3e2c381b9735a04a1e36daaf17b9b3be6ef8fb123b4e7
|
|
4
|
+
data.tar.gz: ef4a68d5e3cfaefa285f78f8292059fef488788c5368d6cf110dae1bfe6a0947
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c4fd6c004371524c34c681922c2d7b8a2f0e5de79e93071a6a2cc34a93d54e70ce1c3565333109c4975cdb6763ea8f1db5adaba9dd8a39444a87e6329cb55367
|
|
7
|
+
data.tar.gz: 71d3fff40bd09fc9f92955c8f816fd455bf4cf3c3c36ea46eda3c4296bafa5bd3abc31290e717de53d11062b2fb96355bbb961267be18e121d7261c20ad7c59c
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,204 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [5.35.2] - 2026-10-14
|
|
9
|
+
|
|
10
|
+
### Security
|
|
11
|
+
|
|
12
|
+
- **On MySQL and MariaDB a quote character inside a string does not change
|
|
13
|
+
which databases a statement is read to name.** The secret column rule reads
|
|
14
|
+
the columns of every database a statement names. In 5.35.1, the first release
|
|
15
|
+
with that read, a quote character inside a string could keep the rule from
|
|
16
|
+
seeing that the statement named another database. A secret column of a table
|
|
17
|
+
or of a view in that database then came back. Each name before a dot is now
|
|
18
|
+
read backward from its dot, so nothing written before the name changes how it
|
|
19
|
+
is read. The refusals are the ones 5.35.1 gives. This gap is in 5.35.1
|
|
20
|
+
alone: earlier versions read no other database, which the 5.35.1 notes cover.
|
|
21
|
+
- **On MySQL and MariaDB a `[` outside a string or a quoted name is refused.**
|
|
22
|
+
MariaDB reads `[name]` as a quoted name when the session's sql_mode holds
|
|
23
|
+
`MSSQL`, and the statement checks read a bracket as an ordinary character. In
|
|
24
|
+
a session with that mode the checks and the server could read one statement
|
|
25
|
+
two ways. `rails_query` does not read the session's mode, so it now refuses
|
|
26
|
+
the bracket on both servers. 5.35.1 and every earlier version had the gap,
|
|
27
|
+
on an app whose connection runs in that mode.
|
|
28
|
+
- **On MySQL and MariaDB a query leaves no value behind on the app's
|
|
29
|
+
connection.** A user variable assignment (`:=`) and `LAST_INSERT_ID` called
|
|
30
|
+
with an argument each set a value that stayed on the pooled connection after
|
|
31
|
+
the query returned, which a read-only transaction does not stop. Both are now
|
|
32
|
+
refused. 5.35.1 and every earlier version ran them.
|
|
33
|
+
- **On PostgreSQL what a foreign scan sends to the remote server is read like
|
|
34
|
+
what the plan runs itself.** A condition, a sort or a group on a secret
|
|
35
|
+
column of a `postgres_fdw` foreign table ran on the remote server, where the
|
|
36
|
+
plan checks did not read it. So a statement could test the column's value,
|
|
37
|
+
also through a view that renames the column and under a `column_name`
|
|
38
|
+
option. The remote statement the plan prints is now read, and such a
|
|
39
|
+
statement answers "Blocked: the query's plan uses the secret column
|
|
40
|
+
`<column>`". Fetching the column returns `[FILTERED]` as before. A statement
|
|
41
|
+
whose plan holds a foreign scan costs one more catalog read. Only the remote
|
|
42
|
+
statement of `postgres_fdw` is read: another foreign data wrapper prints
|
|
43
|
+
keys of its own. 5.35.1 and every earlier version had the gap.
|
|
44
|
+
- **On PostgreSQL a join that the remote server runs is traced to its
|
|
45
|
+
tables.** When `postgres_fdw` sends a whole join to the remote server, the
|
|
46
|
+
plan names the tables with their aliases in one line of the scan, which the
|
|
47
|
+
plan checks did not read, so the secret column rule did not reach those
|
|
48
|
+
tables. The tables and aliases of that line are now read like any scan's: a
|
|
49
|
+
secret column is `[FILTERED]` in place and a whole row is refused, as for a
|
|
50
|
+
local table. 5.35.1 and every earlier version had the gap.
|
|
51
|
+
- **On PostgreSQL the check for a secret column passed through a subquery
|
|
52
|
+
reads a name with two spaces in a row or a dotted capital I.** In 5.35.1
|
|
53
|
+
that check missed a secret column whose name holds either one. A statement
|
|
54
|
+
that passes such a column through a subquery, a CTE or a set operation is
|
|
55
|
+
now refused as for any other secret column. 5.35.0 refused it, so the gap is
|
|
56
|
+
in 5.35.1 alone.
|
|
57
|
+
- **On SQLite a virtual table built on a relation that is not read is refused
|
|
58
|
+
however its definition is written.** 5.35.1 read a relation's name in the
|
|
59
|
+
`content=` option of a full-text table and in the table a vocabulary lists.
|
|
60
|
+
A virtual table that named the relation another way ran, and returned the
|
|
61
|
+
rows or words of a refused view, a shadow table or the page modules. The
|
|
62
|
+
module and every argument can now name a relation, except the columns a
|
|
63
|
+
full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab`
|
|
64
|
+
table ends with (`row`, `col` or `instance`). An argument with `=` is read
|
|
65
|
+
whole, whatever its name, and comments are taken out first. The module is
|
|
66
|
+
read after the table's own name, so a name that spells a module changes
|
|
67
|
+
nothing: 5.35.1 read such a name as the module, and the rule for a virtual
|
|
68
|
+
table that holds a secret column did not reach that table. The answers are
|
|
69
|
+
the ones 5.35.1 gives. The gap needs a virtual table the app itself created.
|
|
70
|
+
5.35.0 and earlier returned such a table's rows too.
|
|
71
|
+
- **On SQLite a relation whose name holds an apostrophe is matched where it
|
|
72
|
+
is written in single quotes.** SQLite reads a name in single quotes where a
|
|
73
|
+
table is expected, with the apostrophe doubled inside. The secret column
|
|
74
|
+
rule spelled a name in double quotes and in backticks and missed that
|
|
75
|
+
spelling, so a statement or a definition that used it was not held to the
|
|
76
|
+
rule for that relation. 5.35.1 and every earlier version had the gap, on an
|
|
77
|
+
app with a relation so named.
|
|
78
|
+
|
|
79
|
+
### Changed
|
|
80
|
+
|
|
81
|
+
The three refusals below go out under a patch number. Each closes a guard gap,
|
|
82
|
+
and the maintainer chose 5.35.2 for them, outside step 1 of `docs/RELEASING.md`.
|
|
83
|
+
|
|
84
|
+
- **On MySQL and MariaDB a statement with a user variable assignment (`:=`) is
|
|
85
|
+
refused.** The answer is "Blocked: a user variable assignment (`:=`) is not
|
|
86
|
+
run. It leaves a value on the pooled connection after the query returns,
|
|
87
|
+
which a read-only transaction does not stop." A statement that numbers its
|
|
88
|
+
rows with `@n := @n + 1` ran before and is refused now. Use a window
|
|
89
|
+
function, `ROW_NUMBER() OVER (...)`, which MySQL 8 and MariaDB 10.2 and
|
|
90
|
+
later have. A `:=` inside a string or a quoted name is data and runs, with
|
|
91
|
+
one case apart: a statement with a backslash before a quote inside a string.
|
|
92
|
+
The second reading of a backslash ends the string at that quote, and where
|
|
93
|
+
that leaves the `:=` outside a string the statement is refused. Write the
|
|
94
|
+
quote doubled.
|
|
95
|
+
- **On MySQL and MariaDB `LAST_INSERT_ID` with an argument is refused.** The
|
|
96
|
+
answer is "Blocked: LAST_INSERT_ID with an argument is not run. It sets a
|
|
97
|
+
value that stays on the pooled connection after the query returns, which a
|
|
98
|
+
read-only transaction does not stop. LAST_INSERT_ID() with no argument
|
|
99
|
+
runs." The call is matched on the raw text, like the session functions, so
|
|
100
|
+
it is refused inside a string too.
|
|
101
|
+
- **On MySQL and MariaDB a statement with a `[` outside a string or a quoted
|
|
102
|
+
name is refused.** The answer is "Blocked: a `[` outside a string or a
|
|
103
|
+
quoted name is not run. MariaDB reads it as the start of a quoted name when
|
|
104
|
+
the session's sql_mode holds MSSQL, and rails_query does not read the
|
|
105
|
+
session's mode. Quote a name in backticks." MySQL, and MariaDB outside that
|
|
106
|
+
mode, run no statement that holds one. In a session with that mode, write
|
|
107
|
+
`` `name` `` for `[name]`. A `[` inside a string or a backticked name runs,
|
|
108
|
+
with one case apart: a statement with a backslash before a quote inside a
|
|
109
|
+
string, as in `'O\'Brien'` beside `REGEXP '[a-z]'`. The second reading of a
|
|
110
|
+
backslash ends the string at that quote, and where that leaves the `[`
|
|
111
|
+
outside a string the statement is refused. Such a statement ran before.
|
|
112
|
+
Write the quote doubled.
|
|
113
|
+
|
|
114
|
+
### Fixed
|
|
115
|
+
|
|
116
|
+
- **A write that follows a `WITH` clause is refused as the write it is.**
|
|
117
|
+
`WITH ... INSERT INTO ...` answered "Blocked: SELECT INTO creates a table".
|
|
118
|
+
It now answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` after a
|
|
119
|
+
`WITH` clause are named the same way. A real `SELECT ... INTO` keeps its
|
|
120
|
+
sentence.
|
|
121
|
+
- **A refusal names a view, a table and a relation as the database spells
|
|
122
|
+
them.** It named them case folded, with a run of spaces as one space, so a
|
|
123
|
+
view called `Hq Odd View` was named `hq odd view`, which no relation is
|
|
124
|
+
called. Two names that differ by a run of spaces alone are one name to the
|
|
125
|
+
rule, and the refusal names the first of them it read.
|
|
126
|
+
- **A view refused over a table that is not read is said to name it.** The
|
|
127
|
+
answer ended "its definition reads `<table>`, which is not read", and the
|
|
128
|
+
rule tests whether the definition names the table. It now ends "its
|
|
129
|
+
definition names `<table>`, which is not read".
|
|
130
|
+
- **The answer for a view that leads too deep drops its count.** It reads "The
|
|
131
|
+
view `<view>` leads through more databases than rails_query follows, so the
|
|
132
|
+
secret columns it could read cannot be told and no statement that names it
|
|
133
|
+
runs." The "(3 rounds)" after "follows" told a user nothing.
|
|
134
|
+
- **The note for a row limit of one says "1 row shown".** It said "1 rows
|
|
135
|
+
shown; the query returned more".
|
|
136
|
+
- **On PostgreSQL a sequence can be queried.** `SELECT last_value FROM
|
|
137
|
+
<sequence>` and `SELECT * FROM <sequence>` ran on 5.35.0, and 5.35.1 answered
|
|
138
|
+
that the sequence's columns could not be read. A sequence is now read like a
|
|
139
|
+
table that holds no secret column.
|
|
140
|
+
- **On PostgreSQL a whole row of a table with no secret column runs when
|
|
141
|
+
another table shares its alias or its name with a number after it.** 5.35.1
|
|
142
|
+
refused such a statement when a subquery used the same alias for a table
|
|
143
|
+
that holds a secret column, or when a table named like `name_2` stood beside
|
|
144
|
+
it, and the refusal named the other table. 5.35.0 ran it. The number the
|
|
145
|
+
planner adds to an alias is now taken off for the members of an Append or a
|
|
146
|
+
Merge Append alone, and a refusal over a whole row names the relation whose
|
|
147
|
+
row the statement reads.
|
|
148
|
+
- **On PostgreSQL a block comment inside a block comment is read as the server
|
|
149
|
+
reads it.** PostgreSQL nests block comments, and the gate ended one at the
|
|
150
|
+
first `*/`, so `SELECT 1 /* a /* b */ c */ AS n` answered PostgreSQL's
|
|
151
|
+
syntax error. It now runs.
|
|
152
|
+
- **On SQLite a statement that runs past `query_timeout` answers the timeout
|
|
153
|
+
when a writer takes the lock while it runs.** It answered that the database
|
|
154
|
+
was locked and the query did not run. The query process now takes its read
|
|
155
|
+
lock before it prepares the statement, so no lock stops the statement from
|
|
156
|
+
there on. The locked answer is given for a lock the query process is still
|
|
157
|
+
waiting on when it gives up or when the deadline passes.
|
|
158
|
+
- **On a SQLite file database a statement that names something the app's
|
|
159
|
+
connection alone holds runs on that connection.** While that connection
|
|
160
|
+
holds a temp object or an attachment, a statement runs there when it names a
|
|
161
|
+
temp object, `sqlite_temp_master` or `sqlite_temp_schema`, the `temp` schema
|
|
162
|
+
or an attached database, or calls a pragma function (`pragma_table_info(...)`,
|
|
163
|
+
`pragma_table_list` and the rest) or `dbstat`, since an argument can name any
|
|
164
|
+
of them.
|
|
165
|
+
The query process has none of them, so it answered an empty list, the rows of
|
|
166
|
+
a main table of the same name, or "unknown database". The answer now ends
|
|
167
|
+
with the note that the query ran without a time limit. A name is matched
|
|
168
|
+
anywhere in the statement, in any letter case and inside a string too.
|
|
169
|
+
Finding this out takes two more reads before a statement on a file database
|
|
170
|
+
where a query process can run: eight where there were six.
|
|
171
|
+
- **The full generated file lists the first database's recent migrations
|
|
172
|
+
under its own name.** With a first database not named `primary` and another
|
|
173
|
+
one that is, the heading read `### Recent Migrations: primary`. It now names
|
|
174
|
+
the first database, as in `### Recent Migrations: animals`.
|
|
175
|
+
- **The database marks and the first database's own name do not depend on the
|
|
176
|
+
preset.** On `preset: standard`, or any `introspectors` list without
|
|
177
|
+
`multi_database`, `rails_get_schema`, `rails_onboard`,
|
|
178
|
+
`rails_performance_check` and the generated files lost the mark of a
|
|
179
|
+
database another environment configures, such as `cache (production only)`,
|
|
180
|
+
and called a first database with another name `primary`. They now read the
|
|
181
|
+
app databases list there, as on `full`, and `rails_onboard` names a
|
|
182
|
+
secondary's adapter.
|
|
183
|
+
- **The model, controller, job, mailer and channel lists hold a class only
|
|
184
|
+
while its own constant names it.** A class whose constant was removed stays
|
|
185
|
+
loaded until the garbage collector takes it, so a list could hold it in one
|
|
186
|
+
run and lose it in the next. Such a class is now left out. A constant still
|
|
187
|
+
waiting to autoload counts as naming its class. The lists of a class's
|
|
188
|
+
children follow the same rule: a model's STI children, the classes a base job
|
|
189
|
+
or mailer is inherited by, and the abstract bases.
|
|
190
|
+
- **A model file that raises before its class opens costs that model alone.**
|
|
191
|
+
A failing `require` at the top of the file, or a raise above the class line,
|
|
192
|
+
made the whole models section answer an error. The file is now named with
|
|
193
|
+
what it raised, and the other models stay listed.
|
|
194
|
+
- **The refusal over a secret column says where `query_allowed_columns` is
|
|
195
|
+
set on either install.** It said to add the name "in an initializer", which
|
|
196
|
+
does nothing on a standalone install. It now ends "add "x" to
|
|
197
|
+
query_allowed_columns in .rails-ai-context.yml (or in
|
|
198
|
+
config/initializers/rails_ai_context.rb where the app bundles the gem)."
|
|
199
|
+
- **`rails_runtime_info` says when table sizes or index usage could not be
|
|
200
|
+
read.** A read that failed or ran out of time left its part of the database
|
|
201
|
+
section out with no word. The section now says "_Table sizes were not read:
|
|
202
|
+
the read failed or took longer than query_timeout._", and at `detail:
|
|
203
|
+
"full"` "_Index usage was not read: the read failed or took longer than
|
|
204
|
+
query_timeout._"
|
|
205
|
+
|
|
8
206
|
## [5.35.1] - 2026-10-13
|
|
9
207
|
|
|
10
208
|
### Security
|
data/CONTEXT.md
CHANGED
|
@@ -167,7 +167,7 @@ A gem the context calls out because it shapes how the app is built, counted only
|
|
|
167
167
|
|
|
168
168
|
## Payload
|
|
169
169
|
|
|
170
|
-
The introspection context hash, in the sense consumers read it. `Payload` is the reading side: a section guard and pinned list readers whose key pairs a spec checks against the producing introspector's own output, so a renamed key fails a test instead of silently emptying a section everywhere. Facts rendered from it live once (`SectionFacts`); membership judgments are applied at the introspector seam, and readers trust the payload rather than re-filtering it.
|
|
170
|
+
The introspection context hash, in the sense consumers read it. `Payload` is the reading side: a section guard and pinned list readers whose key pairs a spec checks against the producing introspector's own output, so a renamed key fails a test instead of silently emptying a section everywhere. Facts rendered from it live once (`SectionFacts`); membership judgments are applied at the introspector seam, and readers trust the payload rather than re-filtering it. One reader fills a section a preset leaves out: `Payload.multi_database` reads the app databases list when the `multi_database` introspector did not run, so a database's mark and the first database's own name do not depend on the preset.
|
|
171
171
|
|
|
172
172
|
## Standalone
|
|
173
173
|
|
|
@@ -235,7 +235,7 @@ What decides whether `rails_query` runs a statement: the checks on its text, and
|
|
|
235
235
|
|
|
236
236
|
## Served tool
|
|
237
237
|
|
|
238
|
-
A tool the server answers for under the app's configuration: the built-ins `skip_tools` leaves, then each custom tool whose name no tool before it claims. Every surface reads the one set: `tools/list`, the CLI's list and lookup, the tool guide, the context files, `rails_get_context`'s includes and every tool name an answer prints. A tool outside it is never offered, run or pointed at. An answer or a guide step names or runs a tool only when the app serves the gem's own tool under that name: a custom tool in a built-in's place is listed, and nothing gives it the built-in's arguments. A custom tool named `rails_get_config` takes the built-in's place when `skip_tools` lists that name, and the **config sections** go with the built-in.
|
|
238
|
+
A tool the server answers for under the app's configuration: the built-ins `skip_tools` leaves, then each custom tool whose name no tool before it claims. An app's own class that inherits the gem's base tool counts with the built-ins once it is loaded, whether or not `custom_tools` lists it. Every surface reads the one set: `tools/list`, the CLI's list and lookup, the tool guide, the context files, `rails_get_context`'s includes and every tool name an answer prints. A tool outside it is never offered, run or pointed at. An answer or a guide step names or runs a tool only when the app serves the gem's own tool under that name: a custom tool in a built-in's place is listed, and nothing gives it the built-in's arguments. A custom tool named `rails_get_config` takes the built-in's place when `skip_tools` lists that name, and the **config sections** go with the built-in.
|
|
239
239
|
|
|
240
240
|
## Config section
|
|
241
241
|
|
data/CONTRIBUTING.md
CHANGED
|
@@ -106,7 +106,7 @@ RAC_TEST_MYSQL_URL=trilogy://root:secret@127.0.0.1:3306/rac \
|
|
|
106
106
|
BUNDLE_GEMFILE=gemfiles/database_servers.gemfile bundle exec rspec spec/lib/rails_ai_context/database_family/database_servers_spec.rb
|
|
107
107
|
```
|
|
108
108
|
|
|
109
|
-
The specs create and drop tables and views whose names start with `rac_`, so point them at a scratch database.
|
|
109
|
+
The specs create and drop tables and views whose names start with `rac_`, and leave the `postgres_fdw` extension installed on PostgreSQL, so point them at a scratch database.
|
|
110
110
|
CI's `database_servers` job runs them on PostgreSQL 16 in every leg. Rails 8.0 runs two legs: MySQL 8.0 with MariaDB 10.6, and MySQL 8.4 with MariaDB 11.4. Rails 7.0 runs MySQL 8.0 with MariaDB 10.6, and Rails 8.1 runs MySQL 8.4 with MariaDB 11.4.
|
|
111
111
|
Rails 7.0 has no Trilogy adapter, so its leg runs PostgreSQL and mysql2 only.
|
|
112
112
|
The job builds mysql2 against MariaDB's client library (`libmariadb-dev`, through `BUNDLE_BUILD__MYSQL2`), which signs in to MySQL and to MariaDB. A MySQL client from 9.0 on cannot sign in to MariaDB.
|
data/docs/COMPATIBILITY.md
CHANGED
|
@@ -312,6 +312,11 @@ Proof sources:
|
|
|
312
312
|
the tool's own checks timed on Ruby 3.4 for statements up to a megabyte
|
|
313
313
|
and on Ruby 3.1 for statements of 50 kilobytes; and the unit suite and the
|
|
314
314
|
E2E harness on Linux, macOS and Windows.
|
|
315
|
+
In the v5.35.2 closing review: the statement files of the v5.35.1 QA sent
|
|
316
|
+
through the tool again on PostgreSQL 16, MySQL 8.4, MariaDB 11.4 and SQLite,
|
|
317
|
+
on the v5.35.1 code and on the fixed code, with every changed answer traced
|
|
318
|
+
to its fix; and for each fix an example that fails on the v5.35.1 code, on a
|
|
319
|
+
real server where the fix is a server's.
|
|
315
320
|
2. Non-crash coverage for every built-in tool including `get_view` in
|
|
316
321
|
`spec/e2e/in_gemfile_install_spec.rb`'s full-tool sweep; output correctness
|
|
317
322
|
(ivar cross-check, render-form detection, partial interfaces) verified
|
data/docs/RELEASING.md
CHANGED
|
@@ -32,7 +32,9 @@ pushed.
|
|
|
32
32
|
commit anyway.
|
|
33
33
|
4. **Prose that names the last version goes stale where no check can see it.**
|
|
34
34
|
`docs/COMPATIBILITY.md` names the release its proof sources come from. Grep
|
|
35
|
-
the tree for the previous version before committing.
|
|
35
|
+
the tree for the previous version before committing. The repository's About
|
|
36
|
+
text on GitHub names the tool count, and no check reads it:
|
|
37
|
+
`gh repo edit --description` changes it.
|
|
36
38
|
5. **The copy-paste prompts name a version floor.** `docs/PROMPTS.md` (and the
|
|
37
39
|
README's copy of the recommended prompt) tells the AI tool to install
|
|
38
40
|
`>= 5.35.0`, the first release with the install flags. If the release that
|
data/docs/SECURITY.md
CHANGED
|
@@ -50,14 +50,15 @@ flowchart LR
|
|
|
50
50
|
Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`) reads it once and answers either the text to run or a refusal with the reason. It holds every check in this layer and, on PostgreSQL, the plan checks of Layer 1b. Comments and quotes are read the way the connection's database family reads them:
|
|
51
51
|
|
|
52
52
|
- **Refuses a statement over 1,000,000 bytes**, before any other check: "Blocked: the statement is <N> bytes long, and rails_query checks a statement of 1000000 bytes at most." The length is checked before any scan of the text and before a connection opens. The text checks below read the whole statement, in time that grows in step with its length, and they give up when they have run longer than `query_timeout`: "Blocked: the checks on the statement's text ran longer than query_timeout (<N> seconds), so the statement was not run. Send a shorter statement."
|
|
53
|
-
- Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
|
|
53
|
+
- Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. PostgreSQL nests block comments, so there a block comment inside a block comment needs a closer of its own, and the gate reads it that way: `SELECT 1 /* a /* b */ c */ AS n` runs. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
|
|
54
54
|
- **On MySQL and MariaDB, reads a backslash in a string both ways**: MySQL reads a backslash in a string as an escape, unless the session's sql_mode holds `NO_BACKSLASH_ESCAPES`, and `rails_query` does not read the session's mode. So the text checks read a statement that holds a backslash once for each reading, and a statement that either reading refuses is refused. A statement with no backslash reads the same both ways and is checked once. A refusal that the second reading alone gives ends "That is how MySQL reads the statement when the session's sql_mode holds NO_BACKSLASH_ESCAPES, and rails_query does not read the session's mode. Write a quote inside a string doubled, so that both readings agree."
|
|
55
|
+
- **On MySQL and MariaDB, refuses a `[` outside a string or a quoted name**: MariaDB reads `[name]` as a quoted name when the session's sql_mode holds `MSSQL`, and `rails_query` does not read the session's mode. The checks in this list read a bracket as an ordinary character, so in that mode a name in brackets, or text after one, would be read one way here and another way by the server. MySQL, and MariaDB outside that mode, run no statement that holds one. The answer is "Blocked: a `[` outside a string or a quoted name is not run. MariaDB reads it as the start of a quoted name when the session's sql_mode holds MSSQL, and rails_query does not read the session's mode. Quote a name in backticks." A `[` inside a string or a backticked name is data and runs, unless a backslash before a quote in the statement leaves the bracket outside a string in the second reading of a backslash (above): then the statement is refused until the quote is written doubled
|
|
55
56
|
- **Blocks write keywords**: INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, CREATE, GRANT, REVOKE, SET, COPY, MERGE, REPLACE. The keyword check (and the `SELECT ... INTO` table-creation check) reads **quote-masked text**, so a keyword inside a string literal is data: `... WHERE notes LIKE '%update%'` and `... = 'put into box'` are reads. `REPLACE` is matched only as the statement; the string function `replace(col, a, b)` every database has (a `REPLACE` immediately followed by `(`) is allowed. `SHOW CREATE TABLE` / `SHOW CREATE VIEW` are allowed despite the `CREATE` (see below)
|
|
56
57
|
- **Blocks lock clauses**: FOR UPDATE, FOR SHARE, FOR NO KEY UPDATE, FOR KEY SHARE and LOCK IN SHARE MODE, with whatever follows them (NOWAIT, SKIP LOCKED, OF, WAIT). On PostgreSQL the clause is read on quote-masked text, so `LIKE '%ready for update%'` is data: its read-only transaction refuses a locking read anyway. MySQL's read-only transaction lets `FOR SHARE` and `LOCK IN SHARE MODE` through, so there and on SQLite the raw text decides
|
|
57
58
|
- **Blocks SHOW forms that tell of other sessions, replication and logs, on MySQL and MariaDB**: `SHOW GRANTS`, `SHOW PROCESSLIST` and `SHOW FULL PROCESSLIST`, `SHOW BINLOG`, `SHOW BINARY LOGS` and `SHOW BINARY LOG STATUS`, `SHOW RELAYLOG EVENTS`, the `SHOW SLAVE` and `SHOW SLAVES` forms, `SHOW MASTER`, the `SHOW REPLICA` and `SHOW REPLICAS` forms, `SHOW ALL SLAVES STATUS` and `SHOW ALL REPLICAS STATUS`, `SHOW ENGINE ... STATUS`, `SHOW PROFILE` and `SHOW PROFILES`, `SHOW OPEN TABLES`, `SHOW EXPLAIN FOR` and `SHOW ANALYZE FOR`, `SHOW USER_STATISTICS`, `SHOW CLIENT_STATISTICS` and the `SHOW WSREP_` forms. The answer is "Blocked: sensitive SHOW command". They are matched on the raw text, so the words inside a string literal are refused too
|
|
58
59
|
- **Blocks `EXPLAIN FOR CONNECTION`, on MySQL and MariaDB**: it shows the plan of the statement another session is running. It is refused also as `DESCRIBE` or `DESC`, with a `FORMAT`, and inside a version comment. The answer is "Blocked: EXPLAIN FOR CONNECTION is not run. It shows the plan of the statement another session is running." It is read on quote-masked text, so a string literal that holds the words runs
|
|
59
60
|
- **Blocks SHOW with a WHERE clause**, on every family. MySQL's statement timeout holds `SELECT` only, so a `SHOW ... WHERE` filter would run past `query_timeout`. `SHOW ... LIKE` runs. The answer names the query to use: "Blocked: SHOW with a WHERE clause is not run, because MySQL's statement timeout does not hold it. SHOW ... LIKE runs. For any other filter, SELECT from information_schema, for example `SELECT table_name FROM information_schema.tables WHERE table_schema = DATABASE()`."
|
|
60
|
-
- **Blocks SELECT INTO**: prevents table creation via SELECT. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
|
|
61
|
+
- **Blocks SELECT INTO**: prevents table creation via SELECT. After a `WITH` clause an `INTO` can belong to a write, and such a statement is answered as that write: `WITH ... INSERT INTO ...` answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` are named the same way. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
|
|
61
62
|
- **Blocks file and network functions**, each on the database family that has it. PostgreSQL: `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file`, `lo_import`, `lo_export`, `dblink*` and their relatives. MySQL and MariaDB: `LOAD DATA` and `LOAD_FILE`. SQLite: `load_extension`. They are checked before and after comment stripping
|
|
62
63
|
- **Blocks functions that return a whole table as one value**: PostgreSQL's `table_to_xml`, `query_to_xml`, `cursor_to_xml`, `schema_to_xml`, `database_to_xml` and their `_to_xmlschema` and `_to_xml_and_xmlschema` relatives. Such a value carries every column of the table, its secret columns included, under a name no column check reads. The name with `(` after it is refused on the raw text: "Blocked: function <name> returns a whole table as one value, secret columns included. SELECT the columns you need from the table itself."
|
|
63
64
|
- **Blocks relations and settings that hold sampled values, other sessions' statements or stored credentials**: no check on a column's name applies to what they hold, so they are not read at all. "Relations and settings that are never read" below lists them by database family, and each name is refused on the family that has it
|
|
@@ -65,9 +66,10 @@ Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`)
|
|
|
65
66
|
- **On SQLite, blocks the tables that hold another table's rows under other names**: the page modules, a shadow table of a virtual table, and a virtual table that holds a secret column. See Layer 4
|
|
66
67
|
- **Blocks a function called by a quoted name**: a quoted name can spell any function, past every check in this list that reads a function's name. A name in double quotes, backticks or brackets with `(` after it is refused on every family, a `U&"..."` name with `UESCAPE` included. The answer is "Blocked: a quoted name is followed by a parenthesis, the way a function is called. rails_query reads a function's name to decide whether it runs, so write the name without quotes." A quoted column inside a call (`count("id")`) and a quoted table before `USING (...)` run
|
|
67
68
|
- **Blocks session-effecting / administrative functions**: a read-only transaction does not stop these, and their effect lingers on the pooled connection, so a call is matched on the raw text: the name with `(` after it is refused, inside a string literal too. A bare name inside a literal is data. PostgreSQL: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_restore_point`, replication-slot and -origin functions, `pg_logical_emit_message`, `pg_advisory_lock*`, `set_config`, `pg_notify`, `txid_current` and their relatives; MySQL and MariaDB: `GET_LOCK`, `RELEASE_LOCK`, `RELEASE_ALL_LOCKS`, `IS_FREE_LOCK`, `IS_USED_LOCK`, `MASTER_POS_WAIT`, `SOURCE_POS_WAIT`. Each list is held to its own family. This name list is the first layer; on PostgreSQL the plan's VOLATILE-function check (below) closes the rest of the family
|
|
68
|
-
- **
|
|
69
|
+
- **On MySQL and MariaDB, blocks a statement that sets a value on the session**: the value stays on the pooled connection after the query returns, which a read-only transaction does not stop. A user variable assignment (`:=`) is read on quote-masked text, so one inside a string or a quoted name is data: "Blocked: a user variable assignment (`:=`) is not run. It leaves a value on the pooled connection after the query returns, which a read-only transaction does not stop." To number rows, use a window function (`ROW_NUMBER() OVER (...)`), which MySQL 8 and MariaDB 10.2 and later have. `LAST_INSERT_ID` called with an argument is read on the raw text, like the session functions above, so it is refused inside a string too: "Blocked: LAST_INSERT_ID with an argument is not run. It sets a value that stays on the pooled connection after the query returns, which a read-only transaction does not stop. LAST_INSERT_ID() with no argument runs."
|
|
70
|
+
- **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead." A stated limit: one name in parentheses after `GLOB`, `MATCH` or `OVER` that follows a subquery (`(SELECT ...) GLOB (title)`, `sum((SELECT 1)) OVER (w)`) has the shape of a list after a subquery's alias and is refused. Each of those words can name a table or a CTE on SQLite, so the gate cannot let them through without letting an alias list through. Write the name with no parentheses (`GLOB title`), or write the window inline (`OVER (ORDER BY id)`)
|
|
69
71
|
- **Blocks multi-statements**: a semicolon with anything after it, read on quote-masked text so a `;` inside a string literal or a `DO $$ ... ; ... $$` block body is not mistaken for a second statement (a `DO` block is then refused plainly as a disallowed statement)
|
|
70
|
-
- **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data
|
|
72
|
+
- **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data. A stated limit: a quoted name that spells the word, such as the alias in `SELECT 1 AS "union"`, is refused on every family, since the check reads quoted names. Give the alias another name
|
|
71
73
|
- **Blocks secret column references**: see Layer 4 - a query whose text names a secret column, directly or through an alias or expression, is refused before execution
|
|
72
74
|
- **Blocks a `*` inside a subquery of a statement that names a relation with a secret column, on MySQL, MariaDB and SQLite**: see Layer 4
|
|
73
75
|
- **Allows only**: SELECT, WITH, SHOW, EXPLAIN, DESCRIBE, DESC. `SHOW CREATE TABLE` and `SHOW CREATE VIEW` are allowed so schema can be inspected; every other `SHOW CREATE` (`USER`, whose output includes credential hashes, `DATABASE`, `FUNCTION`, ...) keeps its `CREATE` and is refused. A returned definition is passed through secret redaction before it is shown, so a credential embedded in it - a FEDERATED table's `CONNECTION='mysql://user:pass@host/db'` - comes back as `mysql://[FILTERED]@host/db`
|
|
@@ -80,8 +82,8 @@ The textual layer cannot see a whole-row serialisation (`SELECT row_to_json(u.*)
|
|
|
80
82
|
- reads a materialized view that the view rule of Layer 4 refuses, directly or through a view over it;
|
|
81
83
|
- reads one of the relations or functions Layer 1 refuses by name (`pg_stats`, `pg_stat_activity`, `pg_settings` and the rest), which a view or an escaped name can reach where the text check saw nothing;
|
|
82
84
|
- calls a function that returns a whole table as one value, or reads a setting that can hold a password (`current_setting` with a refused name, or with a name the plan does not show as one constant);
|
|
83
|
-
- holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations;
|
|
84
|
-
- **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included;
|
|
85
|
+
- holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations. The planner numbers an alias a statement uses twice (`p_1` for the `p` inside a subquery), and that name stands for its own relation alone. The number is taken off only for the members of an Append or a Merge Append, which is how the partitions of one table are named. The refusal names the relation whose row the statement reads;
|
|
86
|
+
- **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included. The statement a `postgres_fdw` foreign scan sends to the remote server (the plan's `Remote SQL`) is read the same way, so a condition, a sort or a group on a secret column of a foreign table is refused where the remote server would run it. A column that carries another name on the remote server (the `column_name` option) is told by one more catalog read, made only for a plan that holds a foreign scan. A stated limit: only the remote statement of `postgres_fdw` is read, and another foreign data wrapper prints keys of its own. A join that `postgres_fdw` runs on the remote server has one scan for all its tables, which names them with their aliases in its `Relations` line: they are read from there, so a whole row of one is refused and a column of one is traced and filtered like a local table's;
|
|
85
87
|
- **passes a secret column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused, since no redaction could reach them. A column that a scan hands to an aggregate or a join is counted only when that node names it, so `count(*)` over a table with a secret column runs;
|
|
86
88
|
- names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn`, `pg_current_wal_insert_lsn`, `pg_current_wal_flush_lsn`, `pg_last_wal_receive_lsn`, `pg_last_wal_replay_lsn`, `pg_last_xact_replay_timestamp`). The functions the statement's text names are checked beside those the plan prints, since a plan prints no row of a `VALUES` list and no `LIMIT` expression. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and outside the read-only list. This closes the admin-function family without a name list that has to grow with every PostgreSQL release.
|
|
87
89
|
|
|
@@ -110,7 +112,7 @@ A statement that names one of these on the database family that has it is refuse
|
|
|
110
112
|
|
|
111
113
|
Each name belongs to one database family and is refused there alone. A PostgreSQL catalog name is not refused on MySQL, MariaDB or SQLite, and a MySQL name is not refused on PostgreSQL or SQLite. On its own family a name is matched as a whole word, so an app table that carries one (`slow_log` on MySQL) cannot be queried. The `information_schema` names in the MySQL row are matched only under that schema, so an app's own `user_statistics` table runs.
|
|
112
114
|
|
|
113
|
-
The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too.
|
|
115
|
+
The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. Every such refusal outside double-quoted text, a real `FROM mysql.user` included, ends " A name before a dot is read as that schema, so an alias named `sys` needs another name.", with `mysql` where that is the name. A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
|
|
114
116
|
|
|
115
117
|
On PostgreSQL the plan is checked for the same names, as relations and as functions, so a view over one is refused: `information_schema.user_mappings`, `information_schema.foreign_servers` and `pg_stat_subscription` are refused that way. `pg_authid`, `pg_shadow` and `pg_db_role_setting` are the exception. A view over them runs, and their `rolpassword` and `setconfig` columns are secret columns: `pg_roles`, `pg_user` and `information_schema.schemata` run, `SELECT rolconfig FROM pg_roles` comes back `[FILTERED]`, and an expression over either column is refused.
|
|
116
118
|
|
|
@@ -146,7 +148,7 @@ What bounds each statement a `rails_query` call sends:
|
|
|
146
148
|
| The query, and `explain: true` | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | the child process's deadline |
|
|
147
149
|
| The reads beside it: the schema read behind the secret column rule, and on PostgreSQL the plan, its `pg_proc` lookup and the column trace | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | no statement limit; a wait on another writer ends at the connection's busy timeout |
|
|
148
150
|
|
|
149
|
-
The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. A statement or a view definition that names another schema or database adds the reads of that one.
|
|
151
|
+
The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. On a SQLite file database where a query process can run, two more reads ask what the app's own connection alone holds, so eight reads come before the statement there. On PostgreSQL a plan that holds a foreign scan adds one catalog read. A statement or a view definition that names another schema or database adds the reads of that one.
|
|
150
152
|
|
|
151
153
|
A read that runs out of that time ends the call with an error that says which side of the statement it was on:
|
|
152
154
|
|
|
@@ -160,6 +162,7 @@ Stated limits of the time bound:
|
|
|
160
162
|
- **A SQLite statement on the app's own connection cannot be ended in the middle of one long step.** The paragraph on that path below says when a statement runs there.
|
|
161
163
|
- **PostgreSQL has no client read timeout.** `statement_timeout` is the server's to keep. A PostgreSQL server that goes silent after it connects is not ended by the client.
|
|
162
164
|
- **On MySQL a `SLEEP()` the statement limit interrupts is no timeout.** MySQL returns 1 from an interrupted `SLEEP()` and raises no error, so the statement answers its rows.
|
|
165
|
+
- **On Ruby 3.1 a malformed statement of tens of kilobytes is slow to report.** When the database's error message carries the bad text (an unterminated quote, a run of `[`), ActiveRecord's handling of that message is slow on a Ruby with no regexp match cache. A 50 KB statement took 17 to 31 seconds to answer its SQL error there, run on the app's own SQLite connection. The time is spent after the database answered, so no limit above ends it. Ruby 3.2 and later report the error at once, and on a SQLite file database the query process ends at the deadline.
|
|
163
166
|
|
|
164
167
|
A database leased by name connects with the `connect_timeout` of its `config/database.yml` entry, or `query_timeout` rounded up to whole seconds when the entry sets none, so a server that never answers the connection ends the call. On MySQL and MariaDB such a connection also gets a `read_timeout` of twice `query_timeout` plus 5 seconds unless its entry sets one, so a server that goes silent after the handshake ends the call. On PostgreSQL `connect_timeout` is the only client bound. Taking the lease down does not replace the outcome of the call: its connections are thrown away and its pool removed, and an error there is not raised. The primary uses the app's own pool and its settings.
|
|
165
168
|
|
|
@@ -177,33 +180,52 @@ A SQLite database whose file is not there is refused by name before anything
|
|
|
177
180
|
connects, with or without a `database` argument, since connecting would create
|
|
178
181
|
the file. The answer names `bin/rails db:prepare`.
|
|
179
182
|
|
|
180
|
-
A SQLite statement runs on the app's own connection, under
|
|
181
|
-
in
|
|
182
|
-
the child process could not prepare,
|
|
183
|
-
|
|
184
|
-
|
|
183
|
+
A SQLite statement runs on the app's own connection, under
|
|
184
|
+
`PRAGMA query_only = ON`, in five cases: an in-memory database, a platform
|
|
185
|
+
without `fork`, a statement the child process could not prepare, a transaction
|
|
186
|
+
open on the app's connection, and a statement that names something the app's
|
|
187
|
+
connection alone holds. The third is a statement that needs a table, function,
|
|
188
|
+
virtual-table module, collation or encryption key only the app's own
|
|
185
189
|
connection has. That case is told from which step failed in the child, and the
|
|
186
|
-
error's text is not read. A statement the child prepared and that fails while
|
|
187
|
-
runs is answered with its SQL error and is not run again. The fourth is
|
|
188
|
-
because the child opens the file itself and would answer without the
|
|
189
|
-
the open transaction.
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
190
|
+
error's text is not read. A statement the child prepared and that fails while
|
|
191
|
+
it runs is answered with its SQL error and is not run again. The fourth is
|
|
192
|
+
there because the child opens the file itself and would answer without the
|
|
193
|
+
rows of the open transaction.
|
|
194
|
+
|
|
195
|
+
The fifth is told before the child starts, by two reads on the app's
|
|
196
|
+
connection: its temp objects and its attached databases. When it holds any of
|
|
197
|
+
them, a statement runs there if it names one of them, the temp catalog
|
|
198
|
+
(`sqlite_temp_master`, `sqlite_temp_schema`) or the `temp` schema, or if it
|
|
199
|
+
calls a pragma function (`pragma_table_info(...)`, `pragma_table_list` and the
|
|
200
|
+
rest) or `dbstat`, whose argument can name any of them. The child has none of
|
|
201
|
+
these and would answer an empty list, the rows of a main table of the same
|
|
202
|
+
name, or "unknown database". A table that only an attached database holds
|
|
203
|
+
needs no match: the child cannot prepare a statement that reads it, which is
|
|
204
|
+
the third case. A stated limit: a name is matched anywhere in the statement's
|
|
205
|
+
text, in any letter case and inside a string too, so a statement that holds
|
|
206
|
+
such a name as plain data runs on the app's connection as well.
|
|
207
|
+
|
|
208
|
+
On the app's own connection a statement that returns rows slowly ends with the
|
|
209
|
+
timeout answer a few rows after `query_timeout` passes. The sqlite3 driver
|
|
210
|
+
holds the interpreter lock while a statement steps, and an interrupt lands only
|
|
211
|
+
between two rows, so the end comes late by the time those rows take. One long
|
|
212
|
+
step cannot be ended there at all. So the table and EXPLAIN answers of such a
|
|
213
|
+
run say it ran without a time limit: "This SQLite query ran without a time
|
|
214
|
+
limit: query_timeout needs a file-backed database, a platform with fork, and
|
|
215
|
+
nothing that only the app's own connection has." CSV output stays plain data.
|
|
196
216
|
|
|
197
217
|
A SQLite database that another connection keeps locked is answered as locked:
|
|
198
218
|
"The database is locked by another connection, so the query did not run. Run it
|
|
199
219
|
again when the other connection has finished." It is an error response. The
|
|
200
220
|
child process waits on a lock for 80% of `query_timeout`, so a lock that stays
|
|
201
221
|
is answered before the timeout. The child tells its parent the moment it meets
|
|
202
|
-
a lock, so a child that started late on a slow machine
|
|
203
|
-
|
|
204
|
-
the
|
|
205
|
-
the
|
|
206
|
-
|
|
222
|
+
a lock, so a child that started late on a slow machine and still waits at the
|
|
223
|
+
deadline is answered as locked too. The child takes its read lock before it
|
|
224
|
+
prepares the statement and tells its parent once it holds it. No writer can
|
|
225
|
+
stop the statement from there on, so a statement that then runs past the
|
|
226
|
+
deadline answers the timeout, also when a writer asks for the lock while it
|
|
227
|
+
runs. A lock met while the connection opens gets the locked answer. The reads
|
|
228
|
+
on the app's own connection wait for that connection's busy timeout.
|
|
207
229
|
|
|
208
230
|
On sqlite3 1.x a query does not run when another connection in the same
|
|
209
231
|
process held an exclusive lock on a rollback-journal database as it started
|
|
@@ -243,11 +265,13 @@ The rule follows what a statement reaches. It starts from these columns, by data
|
|
|
243
265
|
|
|
244
266
|
| Family | The columns the rule reads |
|
|
245
267
|
|:-------|:---------------------------|
|
|
246
|
-
| PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views
|
|
268
|
+
| PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views, foreign tables and sequences. A sequence holds no secret column, so `SELECT last_value FROM <sequence>` runs. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
|
|
247
269
|
| MySQL and MariaDB | The base tables of the connection's database. Then those of every database the statement's text names. `mysql`, `sys`, `performance_schema` and `information_schema` are left out: their relations are refused by name or hold no app data |
|
|
248
270
|
| SQLite | The tables of `main`, of `temp` and of every attached database |
|
|
249
271
|
|
|
250
|
-
A schema or database is told by a name before a dot, and the server says which of those names are schemas. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows
|
|
272
|
+
A schema or database is told by a name before a dot, and the server says which of those names are schemas. Each name is read backward from its dot, bare or in backticks or double quotes, so nothing written before the name changes how it is read, a quote character inside a string included. A name in brackets is not read: SQLite reads every schema whatever a statement names, and on MySQL and MariaDB Layer 1 refuses the bracket. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows, so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
|
|
273
|
+
|
|
274
|
+
A refusal names a view, a table or a relation as the database spells it. The rule itself compares names case folded, with a run of spaces as one space. So two names that differ by a run of spaces alone are one name to the rule, and a refusal names the first of them it read.
|
|
251
275
|
|
|
252
276
|
The schema read fails closed. When the columns of the database cannot be read, no query runs on it and the answer is "The columns of the <name> database could not be read, so its secret columns cannot be told and no query runs on it." Only the primary falls back to the cached context's schema, and it is refused the same way when that has no tables either.
|
|
253
277
|
|
|
@@ -261,11 +285,13 @@ If one of your own columns merely looks secret (an `oauth_applications.secret`,
|
|
|
261
285
|
config.query_allowed_columns = %w[secret]
|
|
262
286
|
```
|
|
263
287
|
|
|
288
|
+
On a standalone install, where no initializer runs, put the name under `query_allowed_columns:` in `.rails-ai-context.yml`.
|
|
289
|
+
|
|
264
290
|
Results are redacted as well: a returned column comes back as `[FILTERED]` when the rule flags its **output name**, and on PostgreSQL when its origin is a secret base column - either by **provenance** (`PG::Result#ftable` / `#ftablecol`, mapped to the base column) or by the **plan's top-node `Output`**, whose i-th expression is result column i. The plan path catches a view (or inlined CTE/subquery/join) that renamed a secret column to a harmless name: `CREATE VIEW user_tokens AS SELECT id, api_token AS t FROM users; SELECT t FROM user_tokens` returns `t` as `[FILTERED]`. A quoted mixed-case secret column (`"Salt"`) is filtered in place the same way. So is a secret column under a wildcard over a partitioned table or an inheritance parent: each partition is read as its own relation, a result column is filtered when any of them hands a secret column up in that place, and a whole-row read is refused when any of them holds one. An expression beside the wildcard (`SELECT *, id + 1 AS next`) changes nothing: each plain result column keeps its table column, and each partition's expressions are checked as expressions. The arms of a `UNION ALL` inside a view are read the same way: each arm against its own table, so no arm stands for another. A result column whose table's column names cannot be read is filtered when that table holds a secret column. When the table cannot be told either, it is filtered when any table holds one. On SQLite a second result column of the same name comes back as `name:1`, and the rule reads that name without its suffix. `SHOW`, `DESCRIBE` and `EXPLAIN` output skips column-name redaction; the one exception is `SHOW CREATE TABLE` / `SHOW CREATE VIEW`, whose definition text is run through the shared secret redaction so a credential embedded in it is filtered.
|
|
265
291
|
|
|
266
292
|
Some relations are refused whole, because no result column of them is traced to a table column. MySQL, MariaDB and SQLite trace none at all, so there two refusals stand in for the plan checks:
|
|
267
293
|
|
|
268
|
-
- **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition
|
|
294
|
+
- **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition names `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
|
|
269
295
|
- **A `*` inside a subquery is refused when the statement names a relation that holds a secret column.** With no plan to read, a value a wildcard returns could be compared or renamed unseen. The test is those two facts together: the statement has a wildcard inside a subquery, and it names such a relation. Which relation the wildcard reads is not told without a parser. The answer is "Blocked: a `*` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. On <family> a value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery." A CTE body and a derived table count. So does `(TABLE t)` on MySQL and MariaDB, the two families that have the statement and read no plan, with an answer of its own: "Blocked: `TABLE` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `TABLE t` reads every column of `t`, and on <family> a value read that way can be compared or renamed past the redaction, so SELECT the columns you need in the subquery." `EXISTS (SELECT * ...)` returns no column and `count(*)` reads none, so both run. A relation's name inside a string literal is data on MySQL and MariaDB. SQLite reads a string as a table's name wherever a name can stand, so there a string equal to such a relation's name counts as that name, wherever it stands: "Blocked: a `*` inside a subquery is not run in a statement that holds the string '<relation>'. SQLite reads a string as a table's name where a name can stand, and the table `<relation>` holds the secret column `<column>`. A value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery."
|
|
270
296
|
- **On SQLite `IN` before a relation's name is held to the same rule.** `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, a wildcard subquery with no `*` written. It is refused wherever it stands when the statement names a relation that holds a secret column, or holds a string equal to such a relation's name: "Blocked: `IN` before a table's name is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, and a value read through such a wildcard can be compared past the redaction, so write the subquery with its columns named." `IN (...)` with a list or a subquery in parentheses runs.
|
|
271
297
|
|
|
@@ -277,7 +303,7 @@ On SQLite a table that holds another table's rows or words under other names is
|
|
|
277
303
|
- **A shadow table of a virtual table.** "Blocked: `notes_content` is not read: it is a shadow table, where SQLite keeps what the virtual table `notes` stores under other column names. Query `notes`." The last sentence is left out when the virtual table itself is refused. Before SQLite 3.37, which cannot type a shadow table, a table whose name starts with a virtual table's name and an underscore is read as one.
|
|
278
304
|
- **A virtual table that holds a secret column.** A search, a highlight or a snippet over it can return or confirm that column's values, so it is refused whole: "Blocked: `docs` is not read: it is a virtual table that holds the secret column `api_token`, and a search or a function over a virtual table reads a column by its position, with no name written." Only the columns a virtual table declares decide this. A full-text table has a hidden column of its own name, and that one does not count, so a table named `token_search` runs. A virtual table with no secret column runs.
|
|
279
305
|
- **An `fts5vocab` or `fts4aux` table over such a table.** "Blocked: `docs_words` is not read: it lists the words of the full-text table `docs`, which holds the secret column `api_token`."
|
|
280
|
-
- **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table
|
|
306
|
+
- **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table's definition can name a relation with its module and with every argument, except the columns a full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab` table ends with (`row`, `col` or `instance`): a column or a type that shares a view's name changes nothing. An argument with `=` is an option and is read whole, whatever its name. Comments are taken out before any of it is read, and the module is read after the table's own name, so a name that spells a module changes nothing. A vocabulary table also names the table it lists the words of. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
|
|
281
307
|
- **A view over any of these**, by the view rule above.
|
|
282
308
|
|
|
283
309
|
`dbstat` runs: it gives sizes and no stored value.
|
|
@@ -288,6 +314,8 @@ On every family a generated column whose expression reads a secret column is a s
|
|
|
288
314
|
|
|
289
315
|
A stored procedure or function is a stated limit: its body is not read, so a function that returns a secret column's value returns it, and the value is filtered by its output name only.
|
|
290
316
|
|
|
317
|
+
A column is known by the name it has in this database, which is a stated limit too. A foreign table that gives a secret column of the remote table another local name (the `column_name` option), a column a trigger fills from a secret column and a table built from another are read by their own names, so their values come back unless that name is secret itself. `config.query_redacted_columns` takes such a name.
|
|
318
|
+
|
|
291
319
|
The exemption covers the results too: an allowed name comes back unredacted. A
|
|
292
320
|
column declared with `encrypts` is refused by name and stays `[FILTERED]` either
|
|
293
321
|
way: an encrypted-at-rest attribute is not meant to be read back raw.
|