rails-ai-context 5.35.1 → 5.36.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +375 -1
  3. data/CONTRIBUTING.md +2 -1
  4. data/docs/COMPATIBILITY.md +15 -1
  5. data/docs/SECURITY.md +70 -40
  6. data/docs/TOOLS.md +23 -10
  7. data/docs/TROUBLESHOOTING.md +33 -6
  8. data/lib/rails_ai_context/database_family/mysql_runtime.rb +19 -5
  9. data/lib/rails_ai_context/database_family/postgresql_plan.rb +35 -16
  10. data/lib/rails_ai_context/database_family/postgresql_runtime.rb +43 -8
  11. data/lib/rails_ai_context/database_family/sqlite_runtime.rb +47 -13
  12. data/lib/rails_ai_context/database_family.rb +36 -10
  13. data/lib/rails_ai_context/introspector.rb +12 -0
  14. data/lib/rails_ai_context/introspectors/declared_constant.rb +19 -7
  15. data/lib/rails_ai_context/introspectors/eager_load.rb +12 -2
  16. data/lib/rails_ai_context/introspectors/job_introspector.rb +60 -20
  17. data/lib/rails_ai_context/introspectors/model_introspector.rb +18 -23
  18. data/lib/rails_ai_context/introspectors/multi_database_introspector.rb +10 -1
  19. data/lib/rails_ai_context/introspectors/performance_introspector.rb +0 -1
  20. data/lib/rails_ai_context/introspectors/schema_introspector.rb +12 -1
  21. data/lib/rails_ai_context/payload.rb +12 -3
  22. data/lib/rails_ai_context/secret_columns.rb +120 -72
  23. data/lib/rails_ai_context/serializers/markdown_serializer.rb +1 -1
  24. data/lib/rails_ai_context/sql_gate.rb +183 -38
  25. data/lib/rails_ai_context/tools/get_context.rb +4 -4
  26. data/lib/rails_ai_context/tools/get_controllers.rb +5 -1
  27. data/lib/rails_ai_context/tools/get_conventions.rb +10 -5
  28. data/lib/rails_ai_context/tools/get_schema.rb +35 -11
  29. data/lib/rails_ai_context/tools/onboard.rb +3 -6
  30. data/lib/rails_ai_context/tools/query.rb +13 -3
  31. data/lib/rails_ai_context/tools/runtime_info.rb +23 -14
  32. data/lib/rails_ai_context/tools/safe_call.rb +2 -3
  33. data/lib/rails_ai_context/tools/session_context.rb +2 -3
  34. data/lib/rails_ai_context/version.rb +1 -1
  35. metadata +1 -23
  36. data/.rspec +0 -3
  37. data/.rubocop.yml +0 -16
  38. data/.ruby-version +0 -1
  39. data/AGENTS.md +0 -15
  40. data/CLAUDE.md +0 -3
  41. data/CODE_OF_CONDUCT.md +0 -44
  42. data/CONTEXT.md +0 -242
  43. data/Rakefile +0 -80
  44. data/docs/RELEASING.md +0 -70
  45. data/docs/_config.yml +0 -15
  46. data/docs/_includes/head-custom.html +0 -16
  47. data/docs/adr/0001-own-listener-registration.md +0 -9
  48. data/docs/adr/0002-static-tier-declared-not-detected.md +0 -5
  49. data/docs/adr/0003-shared-tool-cache-semantics.md +0 -31
  50. data/docs/adr/0004-install-e2e-stays-example-rich.md +0 -28
  51. data/docs/adr/0008-query-runs-only-where-it-can-guard.md +0 -38
  52. data/docs/adr/0009-dbhub-shapes-in-ruby.md +0 -26
  53. data/docs/adr/0010-one-config-tool-with-sections.md +0 -32
  54. data/docs/agents/domain.md +0 -38
  55. data/docs/agents/issue-tracker.md +0 -45
  56. data/docs/agents/triage-labels.md +0 -15
  57. data/server.json +0 -21
data/docs/SECURITY.md CHANGED
@@ -50,24 +50,26 @@ flowchart LR
50
50
  Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`) reads it once and answers either the text to run or a refusal with the reason. It holds every check in this layer and, on PostgreSQL, the plan checks of Layer 1b. Comments and quotes are read the way the connection's database family reads them:
51
51
 
52
52
  - **Refuses a statement over 1,000,000 bytes**, before any other check: "Blocked: the statement is <N> bytes long, and rails_query checks a statement of 1000000 bytes at most." The length is checked before any scan of the text and before a connection opens. The text checks below read the whole statement, in time that grows in step with its length, and they give up when they have run longer than `query_timeout`: "Blocked: the checks on the statement's text ran longer than query_timeout (<N> seconds), so the statement was not run. Send a shorter statement."
53
- - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
53
+ - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. PostgreSQL nests block comments, so there a block comment inside a block comment needs a closer of its own, and the gate reads it that way: `SELECT 1 /* a /* b */ c */ AS n` runs. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
54
54
  - **On MySQL and MariaDB, reads a backslash in a string both ways**: MySQL reads a backslash in a string as an escape, unless the session's sql_mode holds `NO_BACKSLASH_ESCAPES`, and `rails_query` does not read the session's mode. So the text checks read a statement that holds a backslash once for each reading, and a statement that either reading refuses is refused. A statement with no backslash reads the same both ways and is checked once. A refusal that the second reading alone gives ends "That is how MySQL reads the statement when the session's sql_mode holds NO_BACKSLASH_ESCAPES, and rails_query does not read the session's mode. Write a quote inside a string doubled, so that both readings agree."
55
+ - **On MySQL and MariaDB, refuses a `[` outside a string or a quoted name**: MariaDB reads `[name]` as a quoted name when the session's sql_mode holds `MSSQL`, and `rails_query` does not read the session's mode. The checks in this list read a bracket as an ordinary character, so in that mode a name in brackets, or text after one, would be read one way here and another way by the server. MySQL, and MariaDB outside that mode, run no statement that holds one. The answer is "Blocked: a `[` outside a string or a quoted name is not run. MariaDB reads it as the start of a quoted name when the session's sql_mode holds MSSQL, and rails_query does not read the session's mode. Quote a name in backticks." A `[` inside a string or a backticked name is data and runs, unless a backslash before a quote in the statement leaves the bracket outside a string in the second reading of a backslash (above): then the statement is refused until the quote is written doubled
55
56
  - **Blocks write keywords**: INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, CREATE, GRANT, REVOKE, SET, COPY, MERGE, REPLACE. The keyword check (and the `SELECT ... INTO` table-creation check) reads **quote-masked text**, so a keyword inside a string literal is data: `... WHERE notes LIKE '%update%'` and `... = 'put into box'` are reads. `REPLACE` is matched only as the statement; the string function `replace(col, a, b)` every database has (a `REPLACE` immediately followed by `(`) is allowed. `SHOW CREATE TABLE` / `SHOW CREATE VIEW` are allowed despite the `CREATE` (see below)
56
57
  - **Blocks lock clauses**: FOR UPDATE, FOR SHARE, FOR NO KEY UPDATE, FOR KEY SHARE and LOCK IN SHARE MODE, with whatever follows them (NOWAIT, SKIP LOCKED, OF, WAIT). On PostgreSQL the clause is read on quote-masked text, so `LIKE '%ready for update%'` is data: its read-only transaction refuses a locking read anyway. MySQL's read-only transaction lets `FOR SHARE` and `LOCK IN SHARE MODE` through, so there and on SQLite the raw text decides
57
58
  - **Blocks SHOW forms that tell of other sessions, replication and logs, on MySQL and MariaDB**: `SHOW GRANTS`, `SHOW PROCESSLIST` and `SHOW FULL PROCESSLIST`, `SHOW BINLOG`, `SHOW BINARY LOGS` and `SHOW BINARY LOG STATUS`, `SHOW RELAYLOG EVENTS`, the `SHOW SLAVE` and `SHOW SLAVES` forms, `SHOW MASTER`, the `SHOW REPLICA` and `SHOW REPLICAS` forms, `SHOW ALL SLAVES STATUS` and `SHOW ALL REPLICAS STATUS`, `SHOW ENGINE ... STATUS`, `SHOW PROFILE` and `SHOW PROFILES`, `SHOW OPEN TABLES`, `SHOW EXPLAIN FOR` and `SHOW ANALYZE FOR`, `SHOW USER_STATISTICS`, `SHOW CLIENT_STATISTICS` and the `SHOW WSREP_` forms. The answer is "Blocked: sensitive SHOW command". They are matched on the raw text, so the words inside a string literal are refused too
58
59
  - **Blocks `EXPLAIN FOR CONNECTION`, on MySQL and MariaDB**: it shows the plan of the statement another session is running. It is refused also as `DESCRIBE` or `DESC`, with a `FORMAT`, and inside a version comment. The answer is "Blocked: EXPLAIN FOR CONNECTION is not run. It shows the plan of the statement another session is running." It is read on quote-masked text, so a string literal that holds the words runs
59
60
  - **Blocks SHOW with a WHERE clause**, on every family. MySQL's statement timeout holds `SELECT` only, so a `SHOW ... WHERE` filter would run past `query_timeout`. `SHOW ... LIKE` runs. The answer names the query to use: "Blocked: SHOW with a WHERE clause is not run, because MySQL's statement timeout does not hold it. SHOW ... LIKE runs. For any other filter, SELECT from information_schema, for example `SELECT table_name FROM information_schema.tables WHERE table_schema = DATABASE()`."
60
- - **Blocks SELECT INTO**: prevents table creation via SELECT. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
61
+ - **Blocks SELECT INTO**: prevents table creation via SELECT. After a `WITH` clause an `INTO` can belong to a write, and such a statement is answered as that write: `WITH ... INSERT INTO ...` answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` are named the same way. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
61
62
  - **Blocks file and network functions**, each on the database family that has it. PostgreSQL: `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file`, `lo_import`, `lo_export`, `dblink*` and their relatives. MySQL and MariaDB: `LOAD DATA` and `LOAD_FILE`. SQLite: `load_extension`. They are checked before and after comment stripping
62
63
  - **Blocks functions that return a whole table as one value**: PostgreSQL's `table_to_xml`, `query_to_xml`, `cursor_to_xml`, `schema_to_xml`, `database_to_xml` and their `_to_xmlschema` and `_to_xml_and_xmlschema` relatives. Such a value carries every column of the table, its secret columns included, under a name no column check reads. The name with `(` after it is refused on the raw text: "Blocked: function <name> returns a whole table as one value, secret columns included. SELECT the columns you need from the table itself."
63
64
  - **Blocks relations and settings that hold sampled values, other sessions' statements or stored credentials**: no check on a column's name applies to what they hold, so they are not read at all. "Relations and settings that are never read" below lists them by database family, and each name is refused on the family that has it
64
- - **On SQLite, blocks functions that tell of the connection**: `rails_query` runs a statement on a connection of its own and sets some of that connection's values itself, so such an answer would not be the app's. A `pragma_*` function runs only when it describes the schema or the database file: `pragma_table_info`, `pragma_table_xinfo`, `pragma_table_list`, `pragma_index_list`, `pragma_index_info`, `pragma_index_xinfo`, `pragma_foreign_key_list`, `pragma_foreign_key_check`, `pragma_integrity_check`, `pragma_quick_check`, `pragma_user_version`, `pragma_application_id`, `pragma_schema_version`, `pragma_page_size`, `pragma_page_count`, `pragma_freelist_count`, `pragma_encoding`, `pragma_auto_vacuum`, `pragma_journal_mode`, `pragma_compile_options` and `pragma_pragma_list`. Every other `pragma_*` name is refused wherever it stands, inside a string literal too, and so is a call of `changes()`, `total_changes()` or `last_insert_rowid()`. The answer is "Blocked: `<name>` tells of the connection that runs it. rails_query runs a statement on a connection of its own and sets some of that connection's values itself, so the answer would not be the app's. A pragma function that describes the schema or the database file runs, for example `pragma_table_info`." An app table whose own name starts with `pragma_` runs
65
+ - **On SQLite, blocks functions that tell of the connection**: `rails_query` runs a statement on a connection of its own and sets some of that connection's values itself, so such an answer would not be the app's. A `pragma_*` function runs only when it describes the schema or the database file: `pragma_table_info`, `pragma_table_xinfo`, `pragma_table_list`, `pragma_index_list`, `pragma_index_info`, `pragma_index_xinfo`, `pragma_foreign_key_list`, `pragma_foreign_key_check`, `pragma_integrity_check`, `pragma_quick_check`, `pragma_user_version`, `pragma_application_id`, `pragma_schema_version`, `pragma_page_size`, `pragma_page_count`, `pragma_freelist_count`, `pragma_encoding`, `pragma_auto_vacuum`, `pragma_journal_mode`, `pragma_compile_options` and `pragma_pragma_list`. Every other `pragma_*` name is refused where SQLite reads it as the function: called, or where a table stands (after `FROM`, `JOIN`, `IN`, a comma of a `FROM` list, a parenthesis that opens one, or a schema's dot there), quoted or in a string literal too. A call of `changes()`, `total_changes()` or `last_insert_rowid()` is refused wherever it stands, inside a string literal too, whatever tables the app has. The answer is "Blocked: `<name>` tells of the connection that runs it. rails_query runs a statement on a connection of its own and sets some of that connection's values itself, so the answer would not be the app's. A pragma function that describes the schema or the database file runs, for example `pragma_table_info`." A view the statement names, and every view its definition names in turn, is read the same way, since SQLite runs its definition on the same connection. Such a view is refused with "Blocked: `<name>`, which the view `<view>` reads, tells of the connection that runs it." and the rest of the sentence above. A view over a pragma function that describes the schema or the database file runs. An app table whose own name starts with `pragma_` runs, since SQLite reads the table under that name, and so does a column whose own name starts with `pragma_`
65
66
  - **On SQLite, blocks the tables that hold another table's rows under other names**: the page modules, a shadow table of a virtual table, and a virtual table that holds a secret column. See Layer 4
66
67
  - **Blocks a function called by a quoted name**: a quoted name can spell any function, past every check in this list that reads a function's name. A name in double quotes, backticks or brackets with `(` after it is refused on every family, a `U&"..."` name with `UESCAPE` included. The answer is "Blocked: a quoted name is followed by a parenthesis, the way a function is called. rails_query reads a function's name to decide whether it runs, so write the name without quotes." A quoted column inside a call (`count("id")`) and a quoted table before `USING (...)` run
67
68
  - **Blocks session-effecting / administrative functions**: a read-only transaction does not stop these, and their effect lingers on the pooled connection, so a call is matched on the raw text: the name with `(` after it is refused, inside a string literal too. A bare name inside a literal is data. PostgreSQL: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_restore_point`, replication-slot and -origin functions, `pg_logical_emit_message`, `pg_advisory_lock*`, `set_config`, `pg_notify`, `txid_current` and their relatives; MySQL and MariaDB: `GET_LOCK`, `RELEASE_LOCK`, `RELEASE_ALL_LOCKS`, `IS_FREE_LOCK`, `IS_USED_LOCK`, `MASTER_POS_WAIT`, `SOURCE_POS_WAIT`. Each list is held to its own family. This name list is the first layer; on PostgreSQL the plan's VOLATILE-function check (below) closes the rest of the family
68
- - **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead."
69
+ - **On MySQL and MariaDB, blocks a statement that sets a value on the session**: the value stays on the pooled connection after the query returns, which a read-only transaction does not stop. A user variable assignment (`:=`) is read on quote-masked text, so one inside a string or a quoted name is data: "Blocked: a user variable assignment (`:=`) is not run. It leaves a value on the pooled connection after the query returns, which a read-only transaction does not stop." To number rows, use a window function (`ROW_NUMBER() OVER (...)`), which MySQL 8 and MariaDB 10.2 and later have. `LAST_INSERT_ID` called with an argument is read on the raw text, like the session functions above, so it is refused inside a string too: "Blocked: LAST_INSERT_ID with an argument is not run. It sets a value that stays on the pooled connection after the query returns, which a read-only transaction does not stop. LAST_INSERT_ID() with no argument runs."
70
+ - **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead." A stated limit: one name in parentheses after `GLOB`, `MATCH` or `OVER` that follows a subquery (`(SELECT ...) GLOB (title)`, `sum((SELECT 1)) OVER (w)`) has the shape of a list after a subquery's alias and is refused. Each of those words can name a table or a CTE on SQLite, so the gate cannot let them through without letting an alias list through. Write the name with no parentheses (`GLOB title`), or write the window inline (`OVER (ORDER BY id)`)
69
71
  - **Blocks multi-statements**: a semicolon with anything after it, read on quote-masked text so a `;` inside a string literal or a `DO $$ ... ; ... $$` block body is not mistaken for a second statement (a `DO` block is then refused plainly as a disallowed statement)
70
- - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data
72
+ - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data. A stated limit: a quoted name that spells the word, such as the alias in `SELECT 1 AS "union"`, is refused on every family, since the check reads quoted names. Give the alias another name
71
73
  - **Blocks secret column references**: see Layer 4 - a query whose text names a secret column, directly or through an alias or expression, is refused before execution
72
74
  - **Blocks a `*` inside a subquery of a statement that names a relation with a secret column, on MySQL, MariaDB and SQLite**: see Layer 4
73
75
  - **Allows only**: SELECT, WITH, SHOW, EXPLAIN, DESCRIBE, DESC. `SHOW CREATE TABLE` and `SHOW CREATE VIEW` are allowed so schema can be inspected; every other `SHOW CREATE` (`USER`, whose output includes credential hashes, `DATABASE`, `FUNCTION`, ...) keeps its `CREATE` and is refused. A returned definition is passed through secret redaction before it is shown, so a credential embedded in it - a FEDERATED table's `CONNECTION='mysql://user:pass@host/db'` - comes back as `mysql://[FILTERED]@host/db`
@@ -80,16 +82,16 @@ The textual layer cannot see a whole-row serialisation (`SELECT row_to_json(u.*)
80
82
  - reads a materialized view that the view rule of Layer 4 refuses, directly or through a view over it;
81
83
  - reads one of the relations or functions Layer 1 refuses by name (`pg_stats`, `pg_stat_activity`, `pg_settings` and the rest), which a view or an escaped name can reach where the text check saw nothing;
82
84
  - calls a function that returns a whole table as one value, or reads a setting that can hold a password (`current_setting` with a refused name, or with a name the plan does not show as one constant);
83
- - holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations;
84
- - **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included;
85
- - **passes a secret column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused, since no redaction could reach them. A column that a scan hands to an aggregate or a join is counted only when that node names it, so `count(*)` over a table with a secret column runs;
86
- - names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn`, `pg_current_wal_insert_lsn`, `pg_current_wal_flush_lsn`, `pg_last_wal_receive_lsn`, `pg_last_wal_replay_lsn`, `pg_last_xact_replay_timestamp`). The functions the statement's text names are checked beside those the plan prints, since a plan prints no row of a `VALUES` list and no `LIMIT` expression. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and outside the read-only list. This closes the admin-function family without a name list that has to grow with every PostgreSQL release.
85
+ - holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations. The planner numbers an alias a statement uses twice (`p_1` for the `p` inside a subquery), and that name stands for its own relation alone. The number is taken off only for the members of an Append or a Merge Append, which is how the partitions of one table are named. The refusal names the relation whose row the statement reads;
86
+ - **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included. The statement a `postgres_fdw` foreign scan sends to the remote server (the plan's `Remote SQL`) is read the same way, so a condition, a sort or a group on a secret column of a foreign table is refused where the remote server would run it. A column that carries another name on the remote server (the `column_name` option) is told by one more catalog read, made only for a plan that holds a foreign scan. A stated limit: only the remote statement of `postgres_fdw` is read, and another foreign data wrapper prints keys of its own. A join that `postgres_fdw` runs on the remote server has one scan for all its tables, which names them with their aliases in its `Relations` line: they are read from there, so a whole row of one is refused and a column of one is traced and filtered like a local table's;
87
+ - **passes a secret column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused, since no redaction could reach them. A column that a scan hands to an aggregate, a join or a Result node is counted only when that node names it, so `count(*)` over a table with a secret column runs, and so does an `EXISTS` inside an `EXISTS`, whose plan puts a Result over the inner scan;
88
+ - names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in functions and extension C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn`, `pg_current_wal_insert_lsn`, `pg_current_wal_flush_lsn`, `pg_last_wal_receive_lsn`, `pg_last_wal_replay_lsn`, `pg_last_xact_replay_timestamp`); and `pg_sequence_last_value`, which reads a sequence's stored value the way `SELECT last_value FROM a_sequence` does and is what the `pg_sequences` view reads. A name on the list vouches for the server's own function and an extension's C function alone, such as `uuid_generate_v4`: while the database holds a VOLATILE function of that name in another language outside `pg_catalog`, every call of the name is refused. The functions the statement's text names are checked beside those the plan prints, since a plan prints no row of a `VALUES` list and no `LIMIT` expression. The name after `TABLESAMPLE` is a sampling method, whose handler the statement never calls, so it is not read as a call; the method's arguments and its `REPEATABLE` seed are, as the plan prints them. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and outside the read-only list, or, for a name on the list, that the database holds another VOLATILE function of that name. This closes the admin-function family without a name list that has to grow with every PostgreSQL release.
87
89
 
88
- Every statement except `SHOW` is planned, and an `EXPLAIN` is planned as the statement it explains. A statement whose plan cannot be read does not run: when planning times out the answer is the timeout, when planning fails it is the SQL error, and when the server returns no plan the statement is refused.
90
+ Every statement except `SHOW` is planned, and an `EXPLAIN` is planned as the statement it explains. `EXPLAIN (GENERIC_PLAN)` is planned with that option too, read as the server reads it (its last value, quoted or not), so a statement with `$1` in it is planned as PostgreSQL 16 and later plan it. A statement whose plan cannot be read does not run: when planning times out the answer is the timeout, when planning fails it is the SQL error, and when the server returns no plan the statement is refused.
89
91
 
90
92
  A whole-row read of a relation with no secret column (`row_to_json(posts.*)`) is allowed. Before these checks scan a plan expression, its string constants are masked (PostgreSQL's deparser always doubles quotes and never emits an `E''` string, so a constant is matched exactly), so a secret word inside a `'%key%'::text` filter or a literal argument is not mistaken for a column or a function.
91
93
 
92
- A subquery or a join over a table that holds a secret column runs when it reads none of them. A scan under an aggregate or a join hands up every column of its table, and the plan check counts a secret column only when the aggregate or the join names it. So `SELECT (SELECT count(*) FROM users) AS n`, `count(id)`, `max(id)` and `EXISTS (SELECT 1 FROM users)` run, while `max(api_token)`, `count(api_token)`, `string_agg(api_token, ',')`, `row_to_json(users)`, a whole row and `count(users)` are refused.
94
+ A subquery or a join over a table that holds a secret column runs when it reads none of them. A scan under an aggregate, a join or a Result node hands up every column of its table, and the plan check counts a secret column only when that node names it. So `SELECT (SELECT count(*) FROM users) AS n`, `count(id)`, `max(id)` and `EXISTS (SELECT 1 FROM users)` run, while `max(api_token)`, `count(api_token)`, `string_agg(api_token, ',')`, `row_to_json(users)`, a whole row and `count(users)` are refused.
93
95
 
94
96
  #### Relations and settings that are never read
95
97
 
@@ -110,7 +112,7 @@ A statement that names one of these on the database family that has it is refuse
110
112
 
111
113
  Each name belongs to one database family and is refused there alone. A PostgreSQL catalog name is not refused on MySQL, MariaDB or SQLite, and a MySQL name is not refused on PostgreSQL or SQLite. On its own family a name is matched as a whole word, so an app table that carries one (`slow_log` on MySQL) cannot be queried. The `information_schema` names in the MySQL row are matched only under that schema, so an app's own `user_statistics` table runs.
112
114
 
113
- The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. For an alias the answer ends " A name before a dot is read as that schema, so an alias named `sys` needs another name." A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
115
+ The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. Every such refusal outside double-quoted text, a real `FROM mysql.user` included, ends " A name before a dot is read as that schema, so an alias named `sys` needs another name.", with `mysql` where that is the name. A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
114
116
 
115
117
  On PostgreSQL the plan is checked for the same names, as relations and as functions, so a view over one is refused: `information_schema.user_mappings`, `information_schema.foreign_servers` and `pg_stat_subscription` are refused that way. `pg_authid`, `pg_shadow` and `pg_db_role_setting` are the exception. A view over them runs, and their `rolpassword` and `setconfig` columns are secret columns: `pg_roles`, `pg_user` and `information_schema.schemata` run, `SELECT rolconfig FROM pg_roles` comes back `[FILTERED]`, and an expression over either column is refused.
116
118
 
@@ -146,7 +148,7 @@ What bounds each statement a `rails_query` call sends:
146
148
  | The query, and `explain: true` | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | the child process's deadline |
147
149
  | The reads beside it: the schema read behind the secret column rule, and on PostgreSQL the plan, its `pg_proc` lookup and the column trace | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | no statement limit; a wait on another writer ends at the connection's busy timeout |
148
150
 
149
- The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. A statement or a view definition that names another schema or database adds the reads of that one.
151
+ The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. On a SQLite file database where a query process can run, two more reads ask what the app's own connection alone holds, so eight reads come before the statement there. On PostgreSQL a plan that holds a foreign scan adds one catalog read. A statement or a view definition that names another schema or database adds the reads of that one.
150
152
 
151
153
  A read that runs out of that time ends the call with an error that says which side of the statement it was on:
152
154
 
@@ -160,16 +162,19 @@ Stated limits of the time bound:
160
162
  - **A SQLite statement on the app's own connection cannot be ended in the middle of one long step.** The paragraph on that path below says when a statement runs there.
161
163
  - **PostgreSQL has no client read timeout.** `statement_timeout` is the server's to keep. A PostgreSQL server that goes silent after it connects is not ended by the client.
162
164
  - **On MySQL a `SLEEP()` the statement limit interrupts is no timeout.** MySQL returns 1 from an interrupted `SLEEP()` and raises no error, so the statement answers its rows.
165
+ - **On Ruby 3.1 a malformed statement of tens of kilobytes is slow to report.** When the database's error message carries the bad text (an unterminated quote, a run of `[`), ActiveRecord's handling of that message is slow on a Ruby with no regexp match cache. A 50 KB statement took 17 to 31 seconds to answer its SQL error there, run on the app's own SQLite connection. The time is spent after the database answered, so no limit above ends it. Ruby 3.2 and later report the error at once, and on a SQLite file database the query process ends at the deadline.
163
166
 
164
167
  A database leased by name connects with the `connect_timeout` of its `config/database.yml` entry, or `query_timeout` rounded up to whole seconds when the entry sets none, so a server that never answers the connection ends the call. On MySQL and MariaDB such a connection also gets a `read_timeout` of twice `query_timeout` plus 5 seconds unless its entry sets one, so a server that goes silent after the handshake ends the call. On PostgreSQL `connect_timeout` is the only client bound. Taking the lease down does not replace the outcome of the call: its connections are thrown away and its pool removed, and an error there is not raised. The primary uses the app's own pool and its settings.
165
168
 
166
- The MySQL session variable bounds a statement that starts with `WITH` or `EXPLAIN ANALYZE` too; the `MAX_EXECUTION_TIME` hint binds only the outermost `SELECT`, so it is not used. `query_timeout` is a positive number, and PostgreSQL, MySQL and MariaDB bound a value under a millisecond at one millisecond. A timeout is read from the error's class or the database's native code alone, so an SQL error whose message names a column called `timeout` is reported as an SQL error. A wait that PostgreSQL's own `lock_timeout` ended is answered as the timeout. A connection the server dropped is no timeout: on Rails 7.1 and later the answer is an error that starts "The <name> database stopped answering during the query:".
169
+ The MySQL session variable bounds a statement that starts with `WITH` or `EXPLAIN ANALYZE` too; the `MAX_EXECUTION_TIME` hint binds only the outermost `SELECT`, so it is not used. `query_timeout` is a positive number, and PostgreSQL, MySQL and MariaDB bound a value under a millisecond at one millisecond. A timeout is read from the error's class or the database's native code alone, so an SQL error whose message names a column called `timeout` is reported as an SQL error. On PostgreSQL a statement another session cancels (`pg_cancel_backend`) raises the timeout's class too, and the server's message alone tells them apart: a cancel, whose message ends "due to user request", is answered as the SQL error it is. A server that writes its messages in another language answers such a cancel as the timeout. A wait that PostgreSQL's own `lock_timeout` ended is answered as the timeout. A connection the server dropped is no timeout: on Rails 7.1 and later the answer is an error that starts "The <name> database stopped answering during the query:".
167
170
 
168
171
  On MySQL and MariaDB the session that holds the read-only transaction and the time limit is the only one the statement is sent on. Rails replaces a connection it lost before the transaction begins, and the new session has neither guard. So the session's id is read inside the transaction, with one `SELECT CONNECTION_ID()`, and compared with the one the limits were set on. When they differ the answer is "The <name> database stopped answering during the query: the connection was replaced before the statement was sent, and the new session has no read-only transaction and no time limit, so the statement was not sent".
169
172
 
170
173
  On PostgreSQL and MySQL the query executes inside a transaction, then rolls back (even if it could write, it can't). MySQL's read-only transaction still lets `FOR SHARE`, `LOCK IN SHARE MODE` and `INTO` through, so the Layer 1 checks stay on every family.
171
174
 
172
- On a connection that is already inside a transaction (a tool called from a transactional test, a mounted server inside a request's transaction) the guard leaves that transaction as it found it. PostgreSQL guards inside a savepoint, so the outer transaction stays writable. MySQL and MariaDB cannot make an open transaction read-only, so the statement is not sent and the answer is "Query failed: MySQL cannot make a transaction that is already open read-only, so the statement was not sent. Run it outside the transaction." SQLite runs the statement on the app's own connection, so it reads the rows of the open transaction. It runs under `PRAGMA query_only = ON`, and the pragma is put back as it was found.
175
+ On a connection that is already inside a transaction (a tool called from a transactional test, a mounted server inside a request's transaction, a transaction the app opened with a raw `BEGIN` that Active Record does not count) the guard leaves that transaction open as it found it, with its writes. PostgreSQL guards inside a savepoint, chosen by the server's own transaction status, so the outer transaction stays writable. MySQL and MariaDB cannot make an open transaction read-only: the server refuses `SET TRANSACTION READ ONLY` inside one, and a `BEGIN` there would commit it, so no `BEGIN` is sent, the statement is not sent, and the answer is "Query failed: MySQL cannot make a transaction that is already open read-only, so the statement was not sent. Run it outside the transaction." SQLite runs the statement on the app's own connection, so it reads the rows of the open transaction. It runs under `PRAGMA query_only = ON`, and the pragma is put back as it was found.
176
+
177
+ A statement that reads a setting the guard sets for it answers the guard's value: `statement_timeout` and `transaction_read_only` on PostgreSQL, the statement limit (`max_execution_time` or `max_statement_time`) and `lock_wait_timeout` on MySQL and MariaDB. When the statement reads one of them (`SHOW name`, `@@name`, `@@SESSION.name`, `@@LOCAL.name` or `current_setting('name')`, the name backquoted or not), or a `SHOW VARIABLES` row or MariaDB's `information_schema.SESSION_VARIABLES` names one, the answer ends with a note that says so and gives the session's own value, for example "statement_timeout here is the value rails_query sets for each statement it runs. The session's own: statement_timeout 50s." A name used as an alias or a column, one in a string or a comment, and a global value on MySQL and MariaDB get no note. SQLite's `PRAGMA query_only` is refused by name.
173
178
 
174
179
  An adapter with no family (SQL Server, Oracle, CockroachDB and any other) is refused: `rails_query` answers with the adapter's name and sends no statement, because it cannot give that database a read-only transaction and a timeout it has verified. The family is read from the database's configuration, so the refusal comes before any connection is opened or verified. CockroachDB's adapter is built on PostgreSQL's, but the adapter's name alone chooses the family, so it is refused too.
175
180
 
@@ -177,33 +182,52 @@ A SQLite database whose file is not there is refused by name before anything
177
182
  connects, with or without a `database` argument, since connecting would create
178
183
  the file. The answer names `bin/rails db:prepare`.
179
184
 
180
- A SQLite statement runs on the app's own connection, under `PRAGMA query_only = ON`,
181
- in four cases: an in-memory database, a platform without `fork`, a statement
182
- the child process could not prepare, and a transaction open on the app's
183
- connection. The third is a statement that needs a function, virtual-table
184
- module, collation, temporary table or encryption key only the app's own
185
+ A SQLite statement runs on the app's own connection, under
186
+ `PRAGMA query_only = ON`, in five cases: an in-memory database, a platform
187
+ without `fork`, a statement the child process could not prepare, a transaction
188
+ open on the app's connection, and a statement that names something the app's
189
+ connection alone holds. The third is a statement that needs a table, function,
190
+ virtual-table module, collation or encryption key only the app's own
185
191
  connection has. That case is told from which step failed in the child, and the
186
- error's text is not read. A statement the child prepared and that fails while it
187
- runs is answered with its SQL error and is not run again. The fourth is there
188
- because the child opens the file itself and would answer without the rows of
189
- the open transaction. On the
190
- app's own connection a statement that returns rows slowly ends with the timeout
191
- answer a few rows after `query_timeout` passes. The sqlite3 driver holds the
192
- interpreter lock while a statement steps, and an interrupt lands only between
193
- two rows, so the end comes late by the time those rows take. One long step
194
- cannot be ended there at all. So the table and EXPLAIN answers of such a run say it ran
195
- without a time limit. CSV output stays plain data.
192
+ error's text is not read. A statement the child prepared and that fails while
193
+ it runs is answered with its SQL error and is not run again. The fourth is
194
+ there because the child opens the file itself and would answer without the
195
+ rows of the open transaction.
196
+
197
+ The fifth is told before the child starts, by two reads on the app's
198
+ connection: its temp objects and its attached databases. When it holds any of
199
+ them, a statement runs there if it names one of them, the temp catalog
200
+ (`sqlite_temp_master`, `sqlite_temp_schema`) or the `temp` schema, or if it
201
+ calls a pragma function (`pragma_table_info(...)`, `pragma_table_list` and the
202
+ rest) or `dbstat`, whose argument can name any of them. The child has none of
203
+ these and would answer an empty list, the rows of a main table of the same
204
+ name, or "unknown database". A table that only an attached database holds
205
+ needs no match: the child cannot prepare a statement that reads it, which is
206
+ the third case. A stated limit: a name is matched anywhere in the statement's
207
+ text, in any letter case and inside a string too, so a statement that holds
208
+ such a name as plain data runs on the app's connection as well.
209
+
210
+ On the app's own connection a statement that returns rows slowly ends with the
211
+ timeout answer a few rows after `query_timeout` passes. The sqlite3 driver
212
+ holds the interpreter lock while a statement steps, and an interrupt lands only
213
+ between two rows, so the end comes late by the time those rows take. One long
214
+ step cannot be ended there at all. So the table and EXPLAIN answers of such a
215
+ run say it ran without a time limit: "This SQLite query ran without a time
216
+ limit: query_timeout needs a file-backed database, a platform with fork, and
217
+ nothing that only the app's own connection has." CSV output stays plain data.
196
218
 
197
219
  A SQLite database that another connection keeps locked is answered as locked:
198
220
  "The database is locked by another connection, so the query did not run. Run it
199
221
  again when the other connection has finished." It is an error response. The
200
222
  child process waits on a lock for 80% of `query_timeout`, so a lock that stays
201
223
  is answered before the timeout. The child tells its parent the moment it meets
202
- a lock, so a child that started late on a slow machine is answered as locked
203
- too, and again when it is past the lock, so a statement that then runs past
204
- the deadline answers the timeout. A lock met while the connection opens gets
205
- the locked answer. The reads on the app's own connection wait for that
206
- connection's busy timeout.
224
+ a lock, so a child that started late on a slow machine and still waits at the
225
+ deadline is answered as locked too. The child takes its read lock before it
226
+ prepares the statement and tells its parent once it holds it. No writer can
227
+ stop the statement from there on, so a statement that then runs past the
228
+ deadline answers the timeout, also when a writer asks for the lock while it
229
+ runs. A lock met while the connection opens gets the locked answer. The reads
230
+ on the app's own connection wait for that connection's busy timeout.
207
231
 
208
232
  On sqlite3 1.x a query does not run when another connection in the same
209
233
  process held an exclusive lock on a rollback-journal database as it started
@@ -234,7 +258,7 @@ minus anything in `config.query_allowed_columns`. An `encrypts` column is the ex
234
258
 
235
259
  **Default redacted patterns:** `password_digest`, `encrypted_password`, `password_hash`, `reset_password_token`, `confirmation_token`, `unlock_token`, `otp_secret`, `session_data`, `secret_key`, `api_key`, `api_secret`, `access_token`, `refresh_token`, `jti`. Those are the defaults of `config.query_redacted_columns`. The fixed built-in list adds `password_reset_token`, `remember_token`, `secret`, `private_key`, `authentication_string`, `rolpassword` and `setconfig`. The last three are where MySQL and PostgreSQL keep password hashes and stored settings.
236
260
 
237
- The pre-execution refusal matches on two things, as whole identifiers. Names are folded the way the databases compare them: lowercase, with MySQL's reading of a dotted capital I as `i`.
261
+ The pre-execution refusal matches on two things, as whole identifiers. Names are folded the way the databases compare them. On SQLite that is ASCII letters alone, so `"é"` and `"É"` are two columns there and a refusal of one leaves the other. On PostgreSQL, MySQL and MariaDB every letter is folded to lowercase, with MySQL's reading of a dotted capital I as `i`, since a collation or a database encoding can fold more than ASCII there, and a false refusal is the safe side. What a name means is read with every letter folded on every family: a column whose name folds to a built-in name or to one the heuristic flags is a secret column, and so is one a configured name folds to when no column of the database spells that name as the database folds it.
238
262
 
239
263
  - **the fixed names above**, on every family whether or not they are real columns;
240
264
  - **every real column of your schema that the rule flags** - read from the connection the query runs on, so a database named with `database` is checked against its own columns. This catches an app-specific column the heuristic flags (`api_token`, `auth_token`) through any alias or expression. Rails' own bookkeeping tables (`ar_internal_metadata`, `schema_migrations`) are left out: their columns hold no secrets, and `ar_internal_metadata.key` ends in "key", so counting it would refuse every query whose text held the word "key".
@@ -243,11 +267,13 @@ The rule follows what a statement reaches. It starts from these columns, by data
243
267
 
244
268
  | Family | The columns the rule reads |
245
269
  |:-------|:---------------------------|
246
- | PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views and foreign tables. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
270
+ | PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views, foreign tables and sequences. A sequence holds no secret column, so `SELECT last_value FROM <sequence>` runs. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
247
271
  | MySQL and MariaDB | The base tables of the connection's database. Then those of every database the statement's text names. `mysql`, `sys`, `performance_schema` and `information_schema` are left out: their relations are refused by name or hold no app data |
248
272
  | SQLite | The tables of `main`, of `temp` and of every attached database |
249
273
 
250
- A schema or database is told by a name before a dot, and the server says which of those names are schemas. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows (3 rounds), so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
274
+ A schema or database is told by a name before a dot, and the server says which of those names are schemas. Each name is read backward from its dot, bare or in backticks or double quotes, so nothing written before the name changes how it is read, a quote character inside a string included. A name in brackets is not read: SQLite reads every schema whatever a statement names, and on MySQL and MariaDB Layer 1 refuses the bracket. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows, so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
275
+
276
+ A refusal names a view, a table or a relation as the database spells it. The rule itself compares names case folded as above, with a run of spaces as one space. So two names that differ by a run of spaces alone are one name to the rule, and a refusal names the first of them it read.
251
277
 
252
278
  The schema read fails closed. When the columns of the database cannot be read, no query runs on it and the answer is "The columns of the <name> database could not be read, so its secret columns cannot be told and no query runs on it." Only the primary falls back to the cached context's schema, and it is refused the same way when that has no tables either.
253
279
 
@@ -261,11 +287,13 @@ If one of your own columns merely looks secret (an `oauth_applications.secret`,
261
287
  config.query_allowed_columns = %w[secret]
262
288
  ```
263
289
 
290
+ On a standalone install, where no initializer runs, put the name under `query_allowed_columns:` in `.rails-ai-context.yml`.
291
+
264
292
  Results are redacted as well: a returned column comes back as `[FILTERED]` when the rule flags its **output name**, and on PostgreSQL when its origin is a secret base column - either by **provenance** (`PG::Result#ftable` / `#ftablecol`, mapped to the base column) or by the **plan's top-node `Output`**, whose i-th expression is result column i. The plan path catches a view (or inlined CTE/subquery/join) that renamed a secret column to a harmless name: `CREATE VIEW user_tokens AS SELECT id, api_token AS t FROM users; SELECT t FROM user_tokens` returns `t` as `[FILTERED]`. A quoted mixed-case secret column (`"Salt"`) is filtered in place the same way. So is a secret column under a wildcard over a partitioned table or an inheritance parent: each partition is read as its own relation, a result column is filtered when any of them hands a secret column up in that place, and a whole-row read is refused when any of them holds one. An expression beside the wildcard (`SELECT *, id + 1 AS next`) changes nothing: each plain result column keeps its table column, and each partition's expressions are checked as expressions. The arms of a `UNION ALL` inside a view are read the same way: each arm against its own table, so no arm stands for another. A result column whose table's column names cannot be read is filtered when that table holds a secret column. When the table cannot be told either, it is filtered when any table holds one. On SQLite a second result column of the same name comes back as `name:1`, and the rule reads that name without its suffix. `SHOW`, `DESCRIBE` and `EXPLAIN` output skips column-name redaction; the one exception is `SHOW CREATE TABLE` / `SHOW CREATE VIEW`, whose definition text is run through the shared secret redaction so a credential embedded in it is filtered.
265
293
 
266
294
  Some relations are refused whole, because no result column of them is traced to a table column. MySQL, MariaDB and SQLite trace none at all, so there two refusals stand in for the plan checks:
267
295
 
268
- - **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition reads `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
296
+ - **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition names `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
269
297
  - **A `*` inside a subquery is refused when the statement names a relation that holds a secret column.** With no plan to read, a value a wildcard returns could be compared or renamed unseen. The test is those two facts together: the statement has a wildcard inside a subquery, and it names such a relation. Which relation the wildcard reads is not told without a parser. The answer is "Blocked: a `*` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. On <family> a value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery." A CTE body and a derived table count. So does `(TABLE t)` on MySQL and MariaDB, the two families that have the statement and read no plan, with an answer of its own: "Blocked: `TABLE` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `TABLE t` reads every column of `t`, and on <family> a value read that way can be compared or renamed past the redaction, so SELECT the columns you need in the subquery." `EXISTS (SELECT * ...)` returns no column and `count(*)` reads none, so both run. A relation's name inside a string literal is data on MySQL and MariaDB. SQLite reads a string as a table's name wherever a name can stand, so there a string equal to such a relation's name counts as that name, wherever it stands: "Blocked: a `*` inside a subquery is not run in a statement that holds the string '<relation>'. SQLite reads a string as a table's name where a name can stand, and the table `<relation>` holds the secret column `<column>`. A value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery."
270
298
  - **On SQLite `IN` before a relation's name is held to the same rule.** `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, a wildcard subquery with no `*` written. It is refused wherever it stands when the statement names a relation that holds a secret column, or holds a string equal to such a relation's name: "Blocked: `IN` before a table's name is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, and a value read through such a wildcard can be compared past the redaction, so write the subquery with its columns named." `IN (...)` with a list or a subquery in parentheses runs.
271
299
 
@@ -277,7 +305,7 @@ On SQLite a table that holds another table's rows or words under other names is
277
305
  - **A shadow table of a virtual table.** "Blocked: `notes_content` is not read: it is a shadow table, where SQLite keeps what the virtual table `notes` stores under other column names. Query `notes`." The last sentence is left out when the virtual table itself is refused. Before SQLite 3.37, which cannot type a shadow table, a table whose name starts with a virtual table's name and an underscore is read as one.
278
306
  - **A virtual table that holds a secret column.** A search, a highlight or a snippet over it can return or confirm that column's values, so it is refused whole: "Blocked: `docs` is not read: it is a virtual table that holds the secret column `api_token`, and a search or a function over a virtual table reads a column by its position, with no name written." Only the columns a virtual table declares decide this. A full-text table has a hidden column of its own name, and that one does not count, so a table named `token_search` runs. A virtual table with no secret column runs.
279
307
  - **An `fts5vocab` or `fts4aux` table over such a table.** "Blocked: `docs_words` is not read: it lists the words of the full-text table `docs`, which holds the secret column `api_token`."
280
- - **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table is built on a relation through its `content=` option, and a vocabulary table through the table it lists the words of; a column that shares a view's name changes nothing. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
308
+ - **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table's definition can name a relation with its module and with every argument, except the columns a full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab` table ends with (`row`, `col` or `instance`): a column or a type that shares a view's name changes nothing. An argument with `=` is an option and is read whole, whatever its name. Comments are taken out before any of it is read, and the module is read after the table's own name, so a name that spells a module changes nothing. A vocabulary table also names the table it lists the words of. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
281
309
  - **A view over any of these**, by the view rule above.
282
310
 
283
311
  `dbstat` runs: it gives sizes and no stored value.
@@ -288,6 +316,8 @@ On every family a generated column whose expression reads a secret column is a s
288
316
 
289
317
  A stored procedure or function is a stated limit: its body is not read, so a function that returns a secret column's value returns it, and the value is filtered by its output name only.
290
318
 
319
+ A column is known by the name it has in this database, which is a stated limit too. A foreign table that gives a secret column of the remote table another local name (the `column_name` option), a column a trigger fills from a secret column and a table built from another are read by their own names, so their values come back unless that name is secret itself. `config.query_redacted_columns` takes such a name.
320
+
291
321
  The exemption covers the results too: an allowed name comes back unredacted. A
292
322
  column declared with `encrypts` is refused by name and stays `[FILTERED]` either
293
323
  way: an encrypted-at-rest attribute is not meant to be read back raw.
data/docs/TOOLS.md CHANGED
@@ -145,9 +145,9 @@ tables of every database the app's own schema dumps declare, as
145
145
  Cache and Solid Cable get one line, without their tables. A database whose
146
146
  dump was not read is named with "(tables not read)" and left out of the table
147
147
  count, and the summary says so: "5 tables in 2 databases, the warehouse
148
- database not read". It ends with the app's custom
149
- rake tasks (Rakefile, `lib/tasks`, `rakelib`), each with its arguments,
150
- description and file: the first 15 at `standard`, every one at `full`.
148
+ database not read". It ends with the app's custom rake tasks (Rakefile,
149
+ `lib/tasks`, `rakelib`), each with its arguments, description and file: the
150
+ first 15 at `standard`, every one at `full`.
151
151
  Then the app's own generators under `lib/generators` (the `bin/rails generate`
152
152
  command and its USAGE line), the `lib/templates` files that replace a built-in
153
153
  generator's template, and the Railties under `lib/` with their initializers.
@@ -167,7 +167,10 @@ generator's template, and the Railties under `lib/` with their initializers.
167
167
  Database schema with column types, indexes, defaults, encrypted hints. Booted,
168
168
  a table `db/schema.rb` declares and the connected database does not have is
169
169
  named as a migration that has not run, and the listing header says when the
170
- two table counts disagree.
170
+ two table counts disagree. Where two databases both dump to `db/schema.rb` (the
171
+ first one and the one named `primary`), the file may hold the other one's
172
+ tables, so such a table is named as not in the connected database, with no
173
+ word of a migration.
171
174
 
172
175
  The other databases are listed after the primary's tables: every database any
173
176
  environment in `config/database.yml` configures, so a Rails 8 app's queue,
@@ -178,6 +181,13 @@ its own path, else from the other format's file under its name
178
181
  (`db/queue_schema.rb` or `db/queue_structure.sql`), whatever `database_tasks`
179
182
  or `schema_format` says.
180
183
 
184
+ A database only another environment configures carries a mark, as in
185
+ `cache (production only)`, and a first database that is not named `primary`
186
+ is listed under its own name where another database is the one named
187
+ `primary`. Both are the same under every `preset`, and so is each database's
188
+ adapter: with the `multi_database` introspector off, as on `preset: standard`,
189
+ the context still carries the app databases list.
190
+
181
191
  Column hints follow the leftmost-prefix rule: `[indexed]` means some index leads
182
192
  with the column, so a lookup on it alone can use one; `[unique]` means a unique
183
193
  index covers the column alone; `[unique with x]` names the partners of a
@@ -471,7 +481,8 @@ database's name is read whatever `database_tasks` or `schema_dump` says. A
471
481
  database whose tables could not be read is listed under **Not checked** with
472
482
  the reason: its dump is not on disk or is over `max_schema_file_size`, or its
473
483
  `config/database.yml` entry turns `database_tasks` or `schema_dump` off and no
474
- dump is on disk. A database another environment configures carries its mark:
484
+ dump is on disk, or "`schema_dump` in its database.yml entry names no file this
485
+ check reads". A database another environment configures carries its mark:
475
486
  `**Not checked:** warehouse (production only, db/warehouse_schema.rb is not on disk)`.
476
487
 
477
488
  | Parameter | Type | Default | Description |
@@ -643,7 +654,7 @@ ViewComponent/Phlex components: props, slots, previews, sidecar assets, usage ex
643
654
 
644
655
  ### `rails_get_mailers`
645
656
 
646
- ActionMailer mailers: every mailer class with its delivery actions and delivery method. A mailer that declares no action of its own - one taking them from a gem base or a mixin, or one called through class methods - is listed with where its actions come from rather than left out. A base other mailers inherit from is not one: it leaves the listing, and a line above names the ones left out. Asked for by name, a base answers with what every mailer below it inherits - its layout, helpers, defaults, callbacks and mixins as the app wrote them, the methods it defines - and which mailers inherit it.
657
+ ActionMailer mailers: every mailer class with its delivery actions and delivery method. The actions include the public methods a mailer inherits from the app's own mailers below `ApplicationMailer`, on both tiers. A mailer that declares no action of its own - one taking them from a gem base or a mixin, or one called through class methods - is listed with where its actions come from. A base other mailers inherit from is not one: it leaves the listing, and a line above names the ones left out. Asked for by name, a base answers with what every mailer below it inherits - its layout, helpers, defaults, callbacks and mixins as the app wrote them, the methods it defines - and which mailers inherit it.
647
658
 
648
659
  Each mailer also shows its own `default`, `layout`, `helper` and action and
649
660
  deliver callbacks as written, the template formats of each action under
@@ -735,13 +746,13 @@ Safe read-only SQL with layered security: the SQL gate, a PostgreSQL plan check
735
746
 
736
747
  It runs on PostgreSQL, MySQL (with MariaDB) and SQLite, each with its own read-only transaction and statement timeout. On any other adapter it answers with that adapter's name and runs nothing. A statement that runs past `query_timeout` is answered as a timeout and any other database error as an SQL error. Both are error responses: `isError` over MCP and a non-zero exit on the CLI. So is a database no connection reaches, which the answer names with what to do about it. So are a locked SQLite database and the refusal in production.
737
748
 
738
- On PostgreSQL, MySQL and MariaDB every statement a call sends has a time limit. The query has `query_timeout` to itself and the reads beside it share a second one, so the statements of a call take about twice `query_timeout` at most. On SQLite the query has `query_timeout`, and the reads beside it run on the app's own connection with no statement limit. The [security model](SECURITY.md#layer-2---database-level-read-only) says what bounds each statement and lists the stated limits.
749
+ On PostgreSQL, MySQL and MariaDB every statement a call sends has a time limit. The query has `query_timeout` to itself and the reads beside it share a second one, so the statements of a call take about twice `query_timeout` at most. On SQLite the query has `query_timeout`, and the reads beside it run on the app's own connection with no statement limit. A SQLite statement that names something the app's connection alone holds, such as a temp table or an attached database, runs on that connection, and its answer says it ran without a time limit. The [security model](SECURITY.md#layer-2---database-level-read-only) says what bounds each statement and lists the stated limits.
739
750
 
740
751
  On MySQL, MariaDB and SQLite a statement that names a view whose definition names a secret column is refused. So is a `*` inside a subquery when the statement names a table that holds a secret column, and on SQLite `IN` before a table's name (`x IN t`). On PostgreSQL a materialized view whose definition names a secret column is refused. On SQLite a virtual table that holds a secret column, a shadow table and the page modules are refused, and only the pragma functions that describe the schema or the database file run.
741
752
 
742
753
  The secret column rule reads the columns of every schema and database a statement reaches: the ones its text names, on PostgreSQL the ones its plan names, and the ones the views it names lead to, three reads deep. So a secret column of a table in another one is refused by name too.
743
754
 
744
- On MySQL and MariaDB the text checks read a backslash in a string both ways, and a statement that either reading refuses is refused. On every family the text checks give up with a refusal when they have run longer than `query_timeout`.
755
+ On MySQL and MariaDB the text checks read a backslash in a string both ways, and a statement that either reading refuses is refused. There a `[` outside a string or a quoted name, a user variable assignment (`:=`) and `LAST_INSERT_ID` with an argument are refused too. On every family the text checks give up with a refusal when they have run longer than `query_timeout`.
745
756
 
746
757
  | Parameter | Type | Default | Description |
747
758
  |:----------|:-----|:--------|:------------|
@@ -755,7 +766,7 @@ On MySQL and MariaDB the text checks read a backslash in a string both ways, and
755
766
 
756
767
  - A name the app does not have is answered with the closest one and every name it has.
757
768
  - A database only another environment configures, a replica and one with `database_tasks` off included, is answered with the environments that do.
758
- - A top-level entry of `config/database.yml` that code connects to by name (`establish_connection(:reporting)`) is an environment to Rails, so `database` does not take its name: the answer lists the databases of the running environment.
769
+ - A top-level entry of `config/database.yml` that code connects to by name (`establish_connection(:reporting)`) is an environment to Rails, so `database` does not take its name: the answer lists every database the app has, as in "No database named reporting. This app has: primary, cache, queue".
759
770
  - One that only a committed dump names is answered as such.
760
771
  - On the static tier the whole tool is unavailable, with or without `database`.
761
772
  - A SQLite database whose file is not there is an error, with or without `database`. The message starts "The <name> database does not exist yet" and names `bin/rails db:prepare`, and nothing is created. A `file:` URI is checked at the path inside it.
@@ -818,10 +829,12 @@ Each diff passes through the redaction every tool's source does, read as the fil
818
829
 
819
830
  Live database pool stats, table sizes, pending migrations, cache stats, queue depth.
820
831
 
821
- Table sizes come from PostgreSQL, MySQL (base tables only) and SQLite, and index usage from PostgreSQL. On MySQL, MariaDB and SQLite the full detail says "_Index usage is not available for the MySQL database family._", or SQLite. On an adapter with no database family the database section says table sizes and index usage are not available for that adapter.
832
+ Table sizes come from PostgreSQL, MySQL (base tables only) and SQLite, and index usage from PostgreSQL. On MySQL, MariaDB and SQLite the full detail says "_Index usage is not available for the MySQL database family._", or SQLite. On an adapter with no database family the database section says table sizes and index usage are not available for that adapter. A read that failed or took longer than `query_timeout` is said in one line: "_Table sizes were not read: the read failed or took longer than query_timeout._", and at `detail: "full"` "_Index usage was not read: the read failed or took longer than query_timeout._"
822
833
 
823
834
  The pool and the cache are read inside the MCP server process, and the answer says so: the pool's counts are that process's own connections, and a `MemoryStore`'s entries are its own, since each app server process holds its own. A store the processes share (Redis, Memcached, Solid Cache, a file store) is the app's. Another environment that configures a shared store is named, as where the app's cache lives there. Solid Cache's stats are read from its database, so when that is a SQLite file that is not there, the cache section says the database does not exist yet and creates nothing.
824
835
 
836
+ The jobs section names the Active Job adapter the running environment uses. Queue counts are read from Sidekiq alone: they are the app's when Sidekiq is that adapter, and under another adapter they are shown under their own heading only when the app has Sidekiq workers of its own, as those workers' counts.
837
+
825
838
  | Parameter | Type | Default | Description |
826
839
  |:----------|:-----|:--------|:------------|
827
840
  | `detail` | enum | `standard` | `summary`, `standard`, `full` |
@@ -278,6 +278,7 @@ The SQL gate refuses write statements, file and admin functions and secret colum
278
278
 
279
279
  What a string literal holds decides some of these:
280
280
  - A write keyword or a semicolon inside a string (`WHERE note = 'please insert coin'`, `'a; b'`) is data on every database, and the query runs
281
+ - A write after a `WITH` clause (`WITH ... INSERT INTO ...`) is named as that write: "Blocked: contains INSERT"
281
282
  - A secret column's name, `FOR UPDATE` / `FOR SHARE` or `UNION` inside a string is data on PostgreSQL only. On MySQL, MariaDB and SQLite the raw text decides and the query is refused, since no later check there would catch a quote the masking misread
282
283
  - `INTO OUTFILE` / `INTO DUMPFILE` is refused wherever it stands, inside a string too, on every database
283
284
  - A file or network function (`pg_read_file`, `LOAD_FILE`), a refused `SHOW` form (`SHOW GRANTS`) and a relation that is not read (`pg_stats`, `slow_log`) are refused wherever they stand, inside a string too, each on the database that has it. `slow_log` is refused on MySQL and MariaDB, and `pg_stats` on PostgreSQL
@@ -292,7 +293,7 @@ MySQL, MariaDB and SQLite trace no result column back to its table column, so th
292
293
  - A `*` inside a subquery, a CTE body included, when the statement names a table that holds a secret column. The message names that table and the column. The rule tests the two facts together and does not tell which table the wildcard reads, so the statement is refused also when the wildcard is over another table. Name the columns in the subquery: `WITH x AS (SELECT id, email FROM users) SELECT * FROM x`. On MySQL and MariaDB `(TABLE t)` counts as a wildcard: select the columns you need. `EXISTS (SELECT * ...)` and `count(*)` run
293
294
  - On SQLite, `IN` before a table's name (`WHERE id IN users`), when the statement names a table that holds a secret column. It is SQLite's shorthand for `IN (SELECT * FROM users)`. Write the subquery with its columns named: `WHERE id IN (SELECT id FROM users)`
294
295
  - On SQLite, either of those two in a statement that holds a string equal to such a table's name (`WHERE role = 'users'`). SQLite reads a string as a table's name where a name can stand, and the rule cannot tell that use from plain data. The message says "in a statement that holds the string 'users'". Name the columns in the subquery, or compare with another value
295
- - A view whose definition names a secret column, or has a wildcard and names a table that holds one. Query the tables the view reads. When the column holds no secret, list its name in `config.query_allowed_columns`
296
+ - A view whose definition names a secret column, or has a wildcard and names a table that holds one. Query the tables the view reads. When the column holds no secret, exempt it by name as [Column values show [FILTERED]](#column-values-show-filtered) says
296
297
 
297
298
  On PostgreSQL a plain view runs and its secret columns come back `[FILTERED]`. A materialized view whose definition names a secret column gets the view message, since its stored rows cannot be traced to a table column. Query the tables it was built from.
298
299
 
@@ -306,11 +307,11 @@ On SQLite `rails_query` reads no table that holds another table's rows or words
306
307
  - An `fts5vocab` or `fts4aux` table over such a table lists its words, so it is refused too
307
308
  - A virtual table whose definition names a refused view, or a table that is not read, holds their rows or words. The message says "its definition names" and what it names
308
309
 
309
- A view over one of these is refused with "its definition reads `<table>`, which is not read". The name is refused inside a string too.
310
+ A view over one of these is refused with "its definition names `<table>`, which is not read". The name is refused inside a string too.
310
311
 
311
312
  ### "Blocked: `pragma_foreign_keys` tells of the connection that runs it"
312
313
 
313
- On SQLite `rails_query` runs a statement on a connection of its own, so a value that belongs to a connection would not be the app's. The pragma functions that describe the schema or the database file run: `pragma_table_info`, `pragma_index_list`, `pragma_foreign_key_list`, `pragma_integrity_check`, `pragma_page_count` and the others [SECURITY.md](SECURITY.md#layer-1---the-sql-gate-statement-checks) lists. Every other `pragma_*` function is refused, and so are `changes()`, `total_changes()` and `last_insert_rowid()`. Read those on the app's own connection, in `bin/rails runner` or the console.
314
+ On SQLite `rails_query` runs a statement on a connection of its own, so a value that belongs to a connection would not be the app's. The pragma functions that describe the schema or the database file run: `pragma_table_info`, `pragma_index_list`, `pragma_foreign_key_list`, `pragma_integrity_check`, `pragma_page_count` and the others [SECURITY.md](SECURITY.md#layer-1---the-sql-gate-statement-checks) lists. Every other `pragma_*` function is refused, and so are `changes()`, `total_changes()` and `last_insert_rowid()`. Read those on the app's own connection, in `bin/rails runner` or the console. A column whose own name starts with `pragma_` runs. Its name is refused only where SQLite would read the function: called, or where a table stands.
314
315
 
315
316
  ### "Blocked: the statement is ... bytes long"
316
317
 
@@ -318,7 +319,7 @@ On SQLite `rails_query` runs a statement on a connection of its own, so a value
318
319
 
319
320
  ### "Blocked: the checks on the statement's text ran longer than query_timeout"
320
321
 
321
- The checks that read a statement's text have `query_timeout` to themselves and give up when they pass it. Nothing was sent to the database. Send a shorter statement.
322
+ The checks that read a statement's text have `query_timeout` to themselves and give up when they pass it. The statement was not sent to the database. The schema read that comes before the checks was. Send a shorter statement.
322
323
 
323
324
  ### "Blocked: ... is not queried: its columns could not be read" and "leads through more databases than rails_query follows"
324
325
 
@@ -334,12 +335,32 @@ MySQL's statement timeout does not hold a `SHOW` filter, so it is refused on eve
334
335
 
335
336
  ### "A name before a dot is read as that schema, so an alias named `sys` needs another name"
336
337
 
337
- On MySQL and MariaDB `mysql.` and `sys.` are read as those schemas wherever they stand outside a single-quoted string. An alias cannot be told from a schema, so `FROM users sys ... sys.id` is refused. Give the alias another name. A double-quoted string that holds `mysql.` or `sys.` is refused too, with a sentence of its own: MySQL reads double-quoted text as a name when the session's sql_mode holds `ANSI_QUOTES`. Write it in single quotes.
338
+ On MySQL and MariaDB `mysql.` and `sys.` are read as those schemas wherever they stand outside a single-quoted string. An alias cannot be told from a schema, so `FROM users sys ... sys.id` is refused. Give the alias another name. The sentence ends every refusal over `mysql.` or `sys.` outside double-quoted text, so a real `FROM mysql.user` gets it too, with `mysql` as the name. A double-quoted string that holds `mysql.` or `sys.` is refused too, with a sentence of its own: MySQL reads double-quoted text as a name when the session's sql_mode holds `ANSI_QUOTES`. Write it in single quotes.
338
339
 
339
340
  ### "That is how MySQL reads the statement when the session's sql_mode holds NO_BACKSLASH_ESCAPES"
340
341
 
341
342
  On MySQL and MariaDB a backslash in a string is an escape, unless the session's sql_mode holds `NO_BACKSLASH_ESCAPES`. `rails_query` does not read the session's mode, so it checks the statement under both readings and refuses it when either one does. This message ends a refusal that the second reading alone gives: with `\'` read as a backslash and the end of the string, the text after it is code. Write a quote inside a string doubled (`'it''s'`), so that both readings agree.
342
343
 
344
+ ### "Blocked: a `[` outside a string or a quoted name is not run"
345
+
346
+ On MySQL and MariaDB a square bracket outside a string or a quoted name is refused. MariaDB reads `[name]` as a quoted name when the session's sql_mode holds `MSSQL`, and `rails_query` does not read the session's mode, so its checks could read such a statement another way than the server does. Quote a name in backticks: `` `name` ``. A `[` inside a string, such as a JSON path (`'$[0]'`), runs. One case apart: a string that holds a backslash before a quote (`'O\'Brien'`) is read a second way that ends it at that quote, and a `[` which that reading leaves outside a string is refused. Write the quote doubled (`'O''Brien'`).
347
+
348
+ ### "Blocked: a user variable assignment (`:=`) is not run"
349
+
350
+ On MySQL and MariaDB `SELECT @n := @n + 1` leaves `@n` set on the app's pooled connection after the query returns, and a read-only transaction does not stop that. To number rows, use a window function, which MySQL 8 and MariaDB 10.2 and later have: `SELECT ROW_NUMBER() OVER (ORDER BY id) AS n, title FROM posts`. A `:=` inside a string or a quoted name runs, unless a backslash before a quote in the statement leaves it outside a string in the second reading of a backslash: write that quote doubled.
351
+
352
+ ### "Blocked: LAST_INSERT_ID with an argument is not run"
353
+
354
+ On MySQL and MariaDB `LAST_INSERT_ID(<value>)` sets the value the next `LAST_INSERT_ID()` returns, and it stays on the app's pooled connection after the query returns. `LAST_INSERT_ID()` with no argument runs. The call is refused wherever it stands, inside a string too.
355
+
356
+ ### "Blocked: a column-alias list" on a statement that has none
357
+
358
+ One name in parentheses after `GLOB`, `MATCH` or `OVER` that follows a subquery has the shape of a column-alias list, so `WHERE (SELECT title FROM post_titles LIMIT 1) GLOB (title)` and `sum((SELECT 1)) OVER (w)` are refused. Write the name with no parentheses (`GLOB title`), or write the window inline (`OVER (ORDER BY id)`).
359
+
360
+ ### "Blocked: UNION is not run" on a statement with no UNION
361
+
362
+ A quoted name that spells the word is refused, as in `SELECT 1 AS "union"`. Give the alias another name. On PostgreSQL the word inside a string is data.
363
+
343
364
  ### "Blocked: the setting ... is not read", "Blocked: SHOW ALL is not run" and "Blocked: SHOW takes a setting's plain name on PostgreSQL"
344
365
 
345
366
  On PostgreSQL `rails_query` reads one setting at a time, by its plain name: `SHOW work_mem`. `SHOW ALL` and a quoted name (`SHOW "work_mem"`) are refused. Six settings can hold a password or a command line that carries one and are refused: `primary_conninfo`, `ssl_passphrase_command`, `archive_command`, `restore_command`, `archive_cleanup_command` and `recovery_end_command`. So is a setting whose own name reads as a secret, such as `app.jwt_secret`.
@@ -362,7 +383,11 @@ The secret column rule reads the database's columns before any query runs. When
362
383
 
363
384
  ### "The database is locked by another connection, so the query did not run"
364
385
 
365
- Another connection holds a lock on the SQLite database, such as a write transaction left open in a console. Nothing ran. End that transaction and run the query again. The query waits on a lock for 80% of `query_timeout` before it answers this. A lock met while the connection opens, as on the CLI or on a server's first call, gets the same answer.
386
+ Another connection holds a lock on the SQLite database, such as a write transaction left open in a console. Nothing ran. End that transaction and run the query again. The query waits on a lock for 80% of `query_timeout` before it answers this. A lock met while the connection opens, as on the CLI or on a server's first call, gets the same answer. A query that holds its read lock and then runs past `query_timeout` answers the timeout, also when a writer asks for the lock while it runs.
387
+
388
+ ### "This SQLite query ran without a time limit"
389
+
390
+ On SQLite `rails_query` runs a statement in a process of its own, which it ends at `query_timeout`. That process cannot see what the app's own connection alone holds. So the statement runs on the app's connection, with this note, on an in-memory database, on a platform without `fork`, inside an open transaction, and when it needs a table, a function, a module, a collation or an encryption key only that connection has. While the connection holds a temp object or an attachment, it runs there too when it names a temp object, the `temp` schema, `sqlite_temp_master` or an attached database, or calls a pragma function (`pragma_table_info(...)`, `pragma_table_list` and the rest) or `dbstat`. A name is matched anywhere in the statement, in any letter case and inside a string too. A statement that returns rows slowly still ends a few rows after `query_timeout`. One long step cannot be ended there.
366
391
 
367
392
  ### "The reads rails_query makes before a statement ... took the whole <N> second timeout"
368
393
 
@@ -376,6 +401,8 @@ Columns whose names look secret are redacted, and a query that names one is reje
376
401
  config.query_allowed_columns = %w[secret]
377
402
  ```
378
403
 
404
+ On a standalone install, where no initializer runs, put the name under `query_allowed_columns:` in `.rails-ai-context.yml`.
405
+
379
406
  ---
380
407
 
381
408
  ## Search issues