rails-ai-context 5.35.0 → 5.35.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (123) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +1024 -0
  3. data/CONTEXT.md +37 -7
  4. data/CONTRIBUTING.md +30 -1
  5. data/README.md +9 -9
  6. data/SECURITY.md +2 -2
  7. data/docs/ARCHITECTURE.md +12 -4
  8. data/docs/CLI.md +44 -15
  9. data/docs/COMPATIBILITY.md +34 -8
  10. data/docs/CONFIGURATION.md +28 -5
  11. data/docs/CUSTOM_TOOLS.md +8 -2
  12. data/docs/FAQ.md +5 -5
  13. data/docs/GUIDE.md +33 -65
  14. data/docs/INTROSPECTORS.md +4 -4
  15. data/docs/QUICKSTART.md +1 -1
  16. data/docs/RECIPES.md +5 -5
  17. data/docs/RELEASING.md +3 -1
  18. data/docs/SECURITY.md +255 -74
  19. data/docs/SETUP.md +1 -1
  20. data/docs/STANDALONE.md +2 -2
  21. data/docs/TOOLS.md +151 -90
  22. data/docs/TROUBLESHOOTING.md +123 -8
  23. data/docs/_config.yml +1 -1
  24. data/docs/adr/0008-query-runs-only-where-it-can-guard.md +38 -0
  25. data/docs/adr/0009-dbhub-shapes-in-ruby.md +26 -0
  26. data/docs/adr/0010-one-config-tool-with-sections.md +32 -0
  27. data/docs/index.md +2 -2
  28. data/exe/rails-ai-context +24 -9
  29. data/lib/generators/rails_ai_context/install/install_generator.rb +2 -0
  30. data/lib/rails_ai_context/app_databases.rb +377 -0
  31. data/lib/rails_ai_context/app_kind.rb +2 -0
  32. data/lib/rails_ai_context/cli/app_root.rb +1 -0
  33. data/lib/rails_ai_context/cli/tool_runner.rb +110 -99
  34. data/lib/rails_ai_context/configuration.rb +15 -8
  35. data/lib/rails_ai_context/database_family/mysql_runtime.rb +170 -0
  36. data/lib/rails_ai_context/database_family/postgresql_plan.rb +124 -0
  37. data/lib/rails_ai_context/database_family/postgresql_runtime.rb +211 -0
  38. data/lib/rails_ai_context/database_family/sqlite_runtime.rb +391 -0
  39. data/lib/rails_ai_context/database_family.rb +282 -0
  40. data/lib/rails_ai_context/database_file.rb +61 -7
  41. data/lib/rails_ai_context/database_yml.rb +7 -29
  42. data/lib/rails_ai_context/doctor.rb +15 -16
  43. data/lib/rails_ai_context/instrumentation.rb +1 -1
  44. data/lib/rails_ai_context/introspectors/connection_pool_introspector.rb +8 -23
  45. data/lib/rails_ai_context/introspectors/database_stats_introspector.rb +6 -70
  46. data/lib/rails_ai_context/introspectors/declared_constant.rb +19 -7
  47. data/lib/rails_ai_context/introspectors/env_introspector.rb +1 -1
  48. data/lib/rails_ai_context/introspectors/env_references.rb +1 -1
  49. data/lib/rails_ai_context/introspectors/job_introspector.rb +7 -7
  50. data/lib/rails_ai_context/introspectors/migration_introspector.rb +8 -11
  51. data/lib/rails_ai_context/introspectors/migration_replay/statements.rb +9 -9
  52. data/lib/rails_ai_context/introspectors/model_introspector.rb +33 -39
  53. data/lib/rails_ai_context/introspectors/multi_database_introspector.rb +33 -30
  54. data/lib/rails_ai_context/introspectors/node_source.rb +2 -0
  55. data/lib/rails_ai_context/introspectors/performance_introspector.rb +80 -24
  56. data/lib/rails_ai_context/introspectors/pg_partitions.rb +1 -4
  57. data/lib/rails_ai_context/introspectors/schema_conventions.rb +6 -8
  58. data/lib/rails_ai_context/introspectors/schema_dump_path.rb +12 -14
  59. data/lib/rails_ai_context/introspectors/schema_introspector.rb +47 -27
  60. data/lib/rails_ai_context/introspectors/schema_reader.rb +1 -1
  61. data/lib/rails_ai_context/introspectors/source_scan.rb +39 -4
  62. data/lib/rails_ai_context/migration_status.rb +18 -46
  63. data/lib/rails_ai_context/payload.rb +48 -1
  64. data/lib/rails_ai_context/pending_migrations.rb +3 -4
  65. data/lib/rails_ai_context/run_cache.rb +5 -0
  66. data/lib/rails_ai_context/schema_adapter.rb +13 -18
  67. data/lib/rails_ai_context/secret_columns.rb +506 -0
  68. data/lib/rails_ai_context/serializers/claude_rules_serializer.rb +2 -2
  69. data/lib/rails_ai_context/serializers/markdown_serializer.rb +14 -10
  70. data/lib/rails_ai_context/serializers/section_facts.rb +12 -2
  71. data/lib/rails_ai_context/serializers/stack_overview_helper.rb +5 -2
  72. data/lib/rails_ai_context/serializers/tool_guide_helper.rb +35 -7
  73. data/lib/rails_ai_context/server.rb +143 -29
  74. data/lib/rails_ai_context/sql_gate.rb +1007 -0
  75. data/lib/rails_ai_context/tasks/rails_ai_context.rake +1 -2
  76. data/lib/rails_ai_context/test_helper.rb +3 -1
  77. data/lib/rails_ai_context/tools/analyze_feature.rb +11 -7
  78. data/lib/rails_ai_context/tools/base_tool.rb +71 -48
  79. data/lib/rails_ai_context/tools/dependency_graph.rb +1 -1
  80. data/lib/rails_ai_context/tools/diagnose.rb +105 -42
  81. data/lib/rails_ai_context/tools/generate_test.rb +4 -6
  82. data/lib/rails_ai_context/tools/get_active_support.rb +1 -6
  83. data/lib/rails_ai_context/tools/get_api.rb +1 -6
  84. data/lib/rails_ai_context/tools/get_autoload.rb +1 -6
  85. data/lib/rails_ai_context/tools/get_callbacks.rb +5 -5
  86. data/lib/rails_ai_context/tools/get_component_catalog.rb +6 -7
  87. data/lib/rails_ai_context/tools/get_concern.rb +6 -6
  88. data/lib/rails_ai_context/tools/get_config.rb +189 -12
  89. data/lib/rails_ai_context/tools/get_context.rb +111 -105
  90. data/lib/rails_ai_context/tools/get_controllers.rb +8 -5
  91. data/lib/rails_ai_context/tools/get_conventions.rb +1 -6
  92. data/lib/rails_ai_context/tools/get_engines.rb +1 -6
  93. data/lib/rails_ai_context/tools/get_env.rb +1 -6
  94. data/lib/rails_ai_context/tools/get_env_config.rb +2 -8
  95. data/lib/rails_ai_context/tools/get_frontend_stack.rb +1 -6
  96. data/lib/rails_ai_context/tools/get_gems.rb +1 -6
  97. data/lib/rails_ai_context/tools/get_helper_methods.rb +4 -5
  98. data/lib/rails_ai_context/tools/get_i18n.rb +1 -7
  99. data/lib/rails_ai_context/tools/get_job_pattern.rb +1 -1
  100. data/lib/rails_ai_context/tools/get_mailers.rb +1 -1
  101. data/lib/rails_ai_context/tools/get_model_details.rb +5 -8
  102. data/lib/rails_ai_context/tools/get_partial_interface.rb +3 -4
  103. data/lib/rails_ai_context/tools/get_schema.rb +10 -8
  104. data/lib/rails_ai_context/tools/get_service_pattern.rb +60 -24
  105. data/lib/rails_ai_context/tools/get_turbo_map.rb +1 -1
  106. data/lib/rails_ai_context/tools/migration_advisor.rb +47 -32
  107. data/lib/rails_ai_context/tools/onboard.rb +24 -24
  108. data/lib/rails_ai_context/tools/performance_check.rb +15 -2
  109. data/lib/rails_ai_context/tools/query.rb +142 -1356
  110. data/lib/rails_ai_context/tools/read_logs.rb +1 -1
  111. data/lib/rails_ai_context/tools/review_changes.rb +6 -7
  112. data/lib/rails_ai_context/tools/runtime_info.rb +125 -88
  113. data/lib/rails_ai_context/tools/safe_call.rb +70 -28
  114. data/lib/rails_ai_context/tools/search_code.rb +0 -4
  115. data/lib/rails_ai_context/tools/search_docs.rb +3 -1
  116. data/lib/rails_ai_context/tools/security_scan.rb +9 -5
  117. data/lib/rails_ai_context/tools/session_context.rb +18 -12
  118. data/lib/rails_ai_context/tools/validate.rb +1 -0
  119. data/lib/rails_ai_context/tools/validate_semantics.rb +5 -5
  120. data/lib/rails_ai_context/version.rb +1 -1
  121. data/lib/rails_ai_context.rb +4 -7
  122. data/server.json +2 -2
  123. metadata +14 -3
data/docs/SECURITY.md CHANGED
@@ -11,11 +11,11 @@
11
11
  ---
12
12
 
13
13
  > [!CAUTION]
14
- > This gem is designed for **development environments**. The query tool, and MCP over HTTP from inside the app, are disabled in production by default. Sensitive files are blocked. All 45 tools are read-only.
14
+ > This gem is designed for **development environments**. The query tool, and MCP over HTTP from inside the app, are disabled in production by default. Sensitive files are blocked. All 35 tools are read-only.
15
15
 
16
16
  ## Design principles
17
17
 
18
- 1. **Read-only by design** - All 45 tools are annotated as non-destructive in the MCP protocol
18
+ 1. **Read-only by design** - All 35 tools are annotated as non-destructive in the MCP protocol
19
19
  2. **Defense in depth** - Multiple security layers, not single points of failure
20
20
  3. **Sensitive data blocking** - Configurable patterns prevent access to secrets
21
21
  4. **Offline by default** - No network calls except optional `rails_search_docs` with `fetch: true`
@@ -29,13 +29,13 @@ The `rails_query` tool uses a 4-layer security model:
29
29
 
30
30
  ```mermaid
31
31
  flowchart LR
32
- Q[SQL Query] --> L1{Layer 1\nRegex Validation}
32
+ Q[SQL Query] --> L1{Layer 1\nSQL gate}
33
33
  L1 -->|"Blocked:\nINSERT, DROP,\nUNION SELECT..."| R1[Rejected]
34
34
  L1 -->|SELECT only| L2{Layer 2\nDatabase Read-Only}
35
35
  L2 -->|"SET TRANSACTION\nREAD ONLY\n+ timeout"| L3{Layer 3\nRow Limit}
36
- L3 -->|"Cap: 1000 rows\nDefault: 100"| L4{Layer 4\nSensitive Columns}
36
+ L3 -->|"Cap: 1000 rows\nDefault: 100"| L4{Layer 4\nSecret Columns}
37
37
  L4 -->|"names password_digest,\napi_key, ..."| R1
38
- L4 -->|"no sensitive column"| OK[Safe Result]
38
+ L4 -->|"no secret column"| OK[Safe Result]
39
39
 
40
40
  style R1 fill:#e74c3c,stroke:#c0392b,color:#fff
41
41
  style OK fill:#27ae60,stroke:#1e8449,color:#fff
@@ -45,107 +45,288 @@ flowchart LR
45
45
  style L4 fill:#9b59b6,stroke:#8e44ad,color:#fff
46
46
  ```
47
47
 
48
- ### Layer 1 - SQL validation (regex-based)
49
-
50
- Before any query reaches the database:
51
-
52
- - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. The query that runs is this stripped text, never the raw input, so nothing the checks did not read reaches the database
53
- - **Blocks write keywords**: INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, CREATE, GRANT, REVOKE, SET, COPY, MERGE, REPLACE. The keyword check (and the `SELECT … INTO` table-creation check) reads **quote-masked text**, so a keyword inside a string literal is data: `… WHERE notes LIKE '%update%'` and `… = 'put into box'` are reads, not writes. `REPLACE` is matched only as the statement; the string function `replace(col, a, b)` every database has (a `REPLACE` immediately followed by `(`) is allowed. `SHOW CREATE TABLE` / `SHOW CREATE VIEW` are allowed despite the `CREATE` (see below)
54
- - **Blocks lock clauses**: FOR UPDATE, FOR SHARE, FOR NO KEY UPDATE
55
- - **Blocks dangerous SHOW**: GRANTS, PROCESSLIST, BINLOG, SLAVE, MASTER, REPLICAS
56
- - **Blocks SELECT INTO**: prevents table creation via SELECT. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file/network functions above, stay matched on the **raw (unmasked) text** - no read-only transaction stops a server-side file write, so these are never read as data
57
- - **Blocks file and network functions**: `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file`, `lo_import`, `lo_export`, `dblink*`, `LOAD DATA`, `LOAD_FILE`, `load_extension` and their relatives, checked before and after comment stripping
58
- - **Blocks session-effecting / administrative functions**: a read-only transaction does not stop these, and their effect lingers on the pooled connection. PostgreSQL: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_restore_point`, replication-slot and -origin functions, `pg_logical_emit_message`, `pg_advisory_lock*`, `set_config`, `pg_notify`, `txid_current` and their relatives; MySQL: `GET_LOCK`, `RELEASE_LOCK`, `RELEASE_ALL_LOCKS`, `IS_FREE_LOCK`, `IS_USED_LOCK`. This name list is the first layer; on PostgreSQL the plan's VOLATILE-function check (below) closes the rest of the family
59
- - **Blocks column-alias lists**: a CTE or FROM-item list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry sensitive columns out under harmless names. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`) and a single-column alias are left alone. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES …)`) renames no table column - `VALUES` has none - so it is allowed; the body is identified by balancing parentheses on quote-masked text, so one inside a literal does not move its bounds, and a sensitive name written inside the `VALUES` rows is still caught by the textual check. A FROM-item table-alias list is always a rename and is refused
60
- - **Blocks multi-statements**: multiple semicolons, read on quote-masked text so a `;` inside a string literal or a `DO $$ … ; … $$` block body is not mistaken for a second statement (a `DO` block is then refused plainly as a disallowed statement, not for the wrong reason)
61
- - **Blocks injection patterns**: OR 1=1, OR true, OR ''=''
62
- - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data
63
- - **Blocks sensitive column references**: see Layer 4 - a query whose text names a sensitive column, directly or through an alias or expression, is refused before execution
64
- - **Allows only**: SELECT, WITH, SHOW, EXPLAIN, DESCRIBE, DESC. `SHOW CREATE TABLE` and `SHOW CREATE VIEW` are allowed so schema can be inspected; every other `SHOW CREATE` (`USER`, whose output includes credential hashes, `DATABASE`, `FUNCTION`, …) keeps its `CREATE` and is refused. A returned definition is passed through secret redaction before it is shown, so a credential embedded in it - a FEDERATED table's `CONNECTION='mysql://user:pass@host/db'` - comes back as `mysql://[FILTERED]@host/db`
65
-
66
- ### Layer 1b - PostgreSQL semantic analysis (planning only)
67
-
68
- The textual layer cannot see a whole-row serialisation (`SELECT row_to_json(u.*) FROM users u`) or an admin function introduced through a view, because no sensitive column name appears in the query. On PostgreSQL, before the query runs, the tool plans it with `EXPLAIN (VERBOSE, FORMAT JSON)` inside the same read-only transaction - planning only, nothing executes, not even a VOLATILE function (the planner does not fold one). The plan is refused when it:
69
-
70
- - holds a **whole-row reference** (`alias.*`) to a relation that has a sensitive column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, …)`, a bare `SELECT u`, and the same through a view, subquery or CTE;
71
- - **uses a sensitive column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle;
72
- - **launders a sensitive column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused rather than redacted;
73
- - names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn` and the other `pg_current_wal_*` / `pg_last_wal_*` / `pg_last_xact_replay_timestamp` readers). `pg_stat_activity` reads through `pg_stat_get_activity`, which is STABLE, so it was never refused. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and not on the read-only list - not that it changes state. This is how the admin-function family is closed robustly, rather than by an ever-growing name list.
74
-
75
- A whole-row read of a relation with no sensitive column (`row_to_json(posts.*)`) is allowed. Before these checks scan a plan expression, its string constants are masked (PostgreSQL's deparser always doubles quotes and never emits an `E''` string, so a constant is matched exactly), so a sensitive word inside a `'%key%'::text` filter or a literal argument is not mistaken for a column or a function.
48
+ ### Layer 1 - The SQL gate: statement checks
49
+
50
+ Before any query reaches the database, the SQL gate (`RailsAiContext::SqlGate`) reads it once and answers either the text to run or a refusal with the reason. It holds every check in this layer and, on PostgreSQL, the plan checks of Layer 1b. Comments and quotes are read the way the connection's database family reads them:
51
+
52
+ - **Refuses a statement over 1,000,000 bytes**, before any other check: "Blocked: the statement is <N> bytes long, and rails_query checks a statement of 1000000 bytes at most." The length is checked before any scan of the text and before a connection opens. The text checks below read the whole statement, in time that grows in step with its length, and they give up when they have run longer than `query_timeout`: "Blocked: the checks on the statement's text ran longer than query_timeout (<N> seconds), so the statement was not run. Send a shorter statement."
53
+ - Strips comments: block (`/* */`), line (`--`), and on MySQL `#`. A comment marker inside a quoted string or identifier is data and stays. PostgreSQL nests block comments, so there a block comment inside a block comment needs a closer of its own, and the gate reads it that way: `SELECT 1 /* a /* b */ c */ AS n` runs. The query that runs is this stripped text, so nothing the checks did not read reaches the database. Dollar quoting (`$tag$ ... $tag$`) is a string on PostgreSQL only. SQLite and MySQL have none, so there the text between two such marks is read as code
54
+ - **On MySQL and MariaDB, reads a backslash in a string both ways**: MySQL reads a backslash in a string as an escape, unless the session's sql_mode holds `NO_BACKSLASH_ESCAPES`, and `rails_query` does not read the session's mode. So the text checks read a statement that holds a backslash once for each reading, and a statement that either reading refuses is refused. A statement with no backslash reads the same both ways and is checked once. A refusal that the second reading alone gives ends "That is how MySQL reads the statement when the session's sql_mode holds NO_BACKSLASH_ESCAPES, and rails_query does not read the session's mode. Write a quote inside a string doubled, so that both readings agree."
55
+ - **On MySQL and MariaDB, refuses a `[` outside a string or a quoted name**: MariaDB reads `[name]` as a quoted name when the session's sql_mode holds `MSSQL`, and `rails_query` does not read the session's mode. The checks in this list read a bracket as an ordinary character, so in that mode a name in brackets, or text after one, would be read one way here and another way by the server. MySQL, and MariaDB outside that mode, run no statement that holds one. The answer is "Blocked: a `[` outside a string or a quoted name is not run. MariaDB reads it as the start of a quoted name when the session's sql_mode holds MSSQL, and rails_query does not read the session's mode. Quote a name in backticks." A `[` inside a string or a backticked name is data and runs, unless a backslash before a quote in the statement leaves the bracket outside a string in the second reading of a backslash (above): then the statement is refused until the quote is written doubled
56
+ - **Blocks write keywords**: INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, CREATE, GRANT, REVOKE, SET, COPY, MERGE, REPLACE. The keyword check (and the `SELECT ... INTO` table-creation check) reads **quote-masked text**, so a keyword inside a string literal is data: `... WHERE notes LIKE '%update%'` and `... = 'put into box'` are reads. `REPLACE` is matched only as the statement; the string function `replace(col, a, b)` every database has (a `REPLACE` immediately followed by `(`) is allowed. `SHOW CREATE TABLE` / `SHOW CREATE VIEW` are allowed despite the `CREATE` (see below)
57
+ - **Blocks lock clauses**: FOR UPDATE, FOR SHARE, FOR NO KEY UPDATE, FOR KEY SHARE and LOCK IN SHARE MODE, with whatever follows them (NOWAIT, SKIP LOCKED, OF, WAIT). On PostgreSQL the clause is read on quote-masked text, so `LIKE '%ready for update%'` is data: its read-only transaction refuses a locking read anyway. MySQL's read-only transaction lets `FOR SHARE` and `LOCK IN SHARE MODE` through, so there and on SQLite the raw text decides
58
+ - **Blocks SHOW forms that tell of other sessions, replication and logs, on MySQL and MariaDB**: `SHOW GRANTS`, `SHOW PROCESSLIST` and `SHOW FULL PROCESSLIST`, `SHOW BINLOG`, `SHOW BINARY LOGS` and `SHOW BINARY LOG STATUS`, `SHOW RELAYLOG EVENTS`, the `SHOW SLAVE` and `SHOW SLAVES` forms, `SHOW MASTER`, the `SHOW REPLICA` and `SHOW REPLICAS` forms, `SHOW ALL SLAVES STATUS` and `SHOW ALL REPLICAS STATUS`, `SHOW ENGINE ... STATUS`, `SHOW PROFILE` and `SHOW PROFILES`, `SHOW OPEN TABLES`, `SHOW EXPLAIN FOR` and `SHOW ANALYZE FOR`, `SHOW USER_STATISTICS`, `SHOW CLIENT_STATISTICS` and the `SHOW WSREP_` forms. The answer is "Blocked: sensitive SHOW command". They are matched on the raw text, so the words inside a string literal are refused too
59
+ - **Blocks `EXPLAIN FOR CONNECTION`, on MySQL and MariaDB**: it shows the plan of the statement another session is running. It is refused also as `DESCRIBE` or `DESC`, with a `FORMAT`, and inside a version comment. The answer is "Blocked: EXPLAIN FOR CONNECTION is not run. It shows the plan of the statement another session is running." It is read on quote-masked text, so a string literal that holds the words runs
60
+ - **Blocks SHOW with a WHERE clause**, on every family. MySQL's statement timeout holds `SELECT` only, so a `SHOW ... WHERE` filter would run past `query_timeout`. `SHOW ... LIKE` runs. The answer names the query to use: "Blocked: SHOW with a WHERE clause is not run, because MySQL's statement timeout does not hold it. SHOW ... LIKE runs. For any other filter, SELECT from information_schema, for example `SELECT table_name FROM information_schema.tables WHERE table_schema = DATABASE()`."
61
+ - **Blocks SELECT INTO**: prevents table creation via SELECT. After a `WITH` clause an `INTO` can belong to a write, and such a statement is answered as that write: `WITH ... INSERT INTO ...` answers "Blocked: contains INSERT", and `REPLACE` and `MERGE` are named the same way. `INTO OUTFILE` / `INTO DUMPFILE` disk writes, and the file and network functions below, stay matched on the **raw (unmasked) text**. No read-only transaction stops a server-side file write, so `INTO OUTFILE` inside a string literal is refused on every database family, and a file function inside one on the family that has it
62
+ - **Blocks file and network functions**, each on the database family that has it. PostgreSQL: `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file`, `lo_import`, `lo_export`, `dblink*` and their relatives. MySQL and MariaDB: `LOAD DATA` and `LOAD_FILE`. SQLite: `load_extension`. They are checked before and after comment stripping
63
+ - **Blocks functions that return a whole table as one value**: PostgreSQL's `table_to_xml`, `query_to_xml`, `cursor_to_xml`, `schema_to_xml`, `database_to_xml` and their `_to_xmlschema` and `_to_xml_and_xmlschema` relatives. Such a value carries every column of the table, its secret columns included, under a name no column check reads. The name with `(` after it is refused on the raw text: "Blocked: function <name> returns a whole table as one value, secret columns included. SELECT the columns you need from the table itself."
64
+ - **Blocks relations and settings that hold sampled values, other sessions' statements or stored credentials**: no check on a column's name applies to what they hold, so they are not read at all. "Relations and settings that are never read" below lists them by database family, and each name is refused on the family that has it
65
+ - **On SQLite, blocks functions that tell of the connection**: `rails_query` runs a statement on a connection of its own and sets some of that connection's values itself, so such an answer would not be the app's. A `pragma_*` function runs only when it describes the schema or the database file: `pragma_table_info`, `pragma_table_xinfo`, `pragma_table_list`, `pragma_index_list`, `pragma_index_info`, `pragma_index_xinfo`, `pragma_foreign_key_list`, `pragma_foreign_key_check`, `pragma_integrity_check`, `pragma_quick_check`, `pragma_user_version`, `pragma_application_id`, `pragma_schema_version`, `pragma_page_size`, `pragma_page_count`, `pragma_freelist_count`, `pragma_encoding`, `pragma_auto_vacuum`, `pragma_journal_mode`, `pragma_compile_options` and `pragma_pragma_list`. Every other `pragma_*` name is refused wherever it stands, inside a string literal too, and so is a call of `changes()`, `total_changes()` or `last_insert_rowid()`. The answer is "Blocked: `<name>` tells of the connection that runs it. rails_query runs a statement on a connection of its own and sets some of that connection's values itself, so the answer would not be the app's. A pragma function that describes the schema or the database file runs, for example `pragma_table_info`." An app table whose own name starts with `pragma_` runs
66
+ - **On SQLite, blocks the tables that hold another table's rows under other names**: the page modules, a shadow table of a virtual table, and a virtual table that holds a secret column. See Layer 4
67
+ - **Blocks a function called by a quoted name**: a quoted name can spell any function, past every check in this list that reads a function's name. A name in double quotes, backticks or brackets with `(` after it is refused on every family, a `U&"..."` name with `UESCAPE` included. The answer is "Blocked: a quoted name is followed by a parenthesis, the way a function is called. rails_query reads a function's name to decide whether it runs, so write the name without quotes." A quoted column inside a call (`count("id")`) and a quoted table before `USING (...)` run
68
+ - **Blocks session-effecting / administrative functions**: a read-only transaction does not stop these, and their effect lingers on the pooled connection, so a call is matched on the raw text: the name with `(` after it is refused, inside a string literal too. A bare name inside a literal is data. PostgreSQL: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_restore_point`, replication-slot and -origin functions, `pg_logical_emit_message`, `pg_advisory_lock*`, `set_config`, `pg_notify`, `txid_current` and their relatives; MySQL and MariaDB: `GET_LOCK`, `RELEASE_LOCK`, `RELEASE_ALL_LOCKS`, `IS_FREE_LOCK`, `IS_USED_LOCK`, `MASTER_POS_WAIT`, `SOURCE_POS_WAIT`. Each list is held to its own family. This name list is the first layer; on PostgreSQL the plan's VOLATILE-function check (below) closes the rest of the family
69
+ - **On MySQL and MariaDB, blocks a statement that sets a value on the session**: the value stays on the pooled connection after the query returns, which a read-only transaction does not stop. A user variable assignment (`:=`) is read on quote-masked text, so one inside a string or a quoted name is data: "Blocked: a user variable assignment (`:=`) is not run. It leaves a value on the pooled connection after the query returns, which a read-only transaction does not stop." To number rows, use a window function (`ROW_NUMBER() OVER (...)`), which MySQL 8 and MariaDB 10.2 and later have. `LAST_INSERT_ID` called with an argument is read on the raw text, like the session functions above, so it is refused inside a string too: "Blocked: LAST_INSERT_ID with an argument is not run. It sets a value that stays on the pooled connection after the query returns, which a read-only transaction does not stop. LAST_INSERT_ID() with no argument runs."
70
+ - **Blocks column-alias lists**: a list such as `t(a, b, c)` renames the columns a wildcard returns, which would carry secret columns out under harmless names. The gate reads the list by its shape, on every family and in every spelling of a name (bare, double-quoted, backticked, bracketed, non-ASCII). The shapes are a parenthesised list of names after a CTE's name and before `AS`, `AS MATERIALIZED` or `AS NOT MATERIALIZED`; after `AS alias`; after the alias of a subquery written with no `AS`; and after a table's alias in a FROM clause. A type modifier (`numeric(10,2)`, `varchar(255)`), a record-function list (`AS t(a int, b text)`), a `JOIN ... USING (x, y)` list and a row comparison (`WHERE (a, b) IN (...)`) are no alias list and run. One name after a function call (`generate_series(1, 5) AS g(n)`) names the one column the function returns, so it runs. One name after a subquery is refused. A list whose body is a `VALUES` clause (`(VALUES (1,'a')) AS v(id, name)`, `WITH v(id, name) AS (VALUES ...)`) renames constants only, so it is allowed unless the body also holds a `SELECT`, a `TABLE`, a `WITH` or a set operation. A secret name written inside the `VALUES` rows is still caught by the name check. The answer is "Blocked: a column-alias list (e.g. `t(a, b, c)`) renames the columns a wildcard returns, so secret columns would leave under harmless names. Name the columns you want in the SELECT list instead." A stated limit: one name in parentheses after `GLOB`, `MATCH` or `OVER` that follows a subquery (`(SELECT ...) GLOB (title)`, `sum((SELECT 1)) OVER (w)`) has the shape of a list after a subquery's alias and is refused. Each of those words can name a table or a CTE on SQLite, so the gate cannot let them through without letting an alias list through. Write the name with no parentheses (`GLOB title`), or write the window inline (`OVER (ORDER BY id)`)
71
+ - **Blocks multi-statements**: a semicolon with anything after it, read on quote-masked text so a `;` inside a string literal or a `DO $$ ... ; ... $$` block body is not mistaken for a second statement (a `DO` block is then refused plainly as a disallowed statement)
72
+ - **Refuses any UNION**: a UNION takes its column names from its first SELECT, so another table's rows - a wildcard's columns included - would come out under those names, past the redaction that reads names. Every spelling is refused (`UNION`, `UNION ALL`, `UNION DISTINCT`, a parenthesised `UNION (SELECT ...)`), recursive CTEs with it; `INTERSECT` and `EXCEPT` return only their first SELECT's rows and stay. On PostgreSQL a "union" inside a string literal is data. A stated limit: a quoted name that spells the word, such as the alias in `SELECT 1 AS "union"`, is refused on every family, since the check reads quoted names. Give the alias another name
73
+ - **Blocks secret column references**: see Layer 4 - a query whose text names a secret column, directly or through an alias or expression, is refused before execution
74
+ - **Blocks a `*` inside a subquery of a statement that names a relation with a secret column, on MySQL, MariaDB and SQLite**: see Layer 4
75
+ - **Allows only**: SELECT, WITH, SHOW, EXPLAIN, DESCRIBE, DESC. `SHOW CREATE TABLE` and `SHOW CREATE VIEW` are allowed so schema can be inspected; every other `SHOW CREATE` (`USER`, whose output includes credential hashes, `DATABASE`, `FUNCTION`, ...) keeps its `CREATE` and is refused. A returned definition is passed through secret redaction before it is shown, so a credential embedded in it - a FEDERATED table's `CONNECTION='mysql://user:pass@host/db'` - comes back as `mysql://[FILTERED]@host/db`
76
+
77
+ ### Layer 1b - The SQL gate: PostgreSQL plan checks (planning only)
78
+
79
+ The textual layer cannot see a whole-row serialisation (`SELECT row_to_json(u.*) FROM users u`) or an admin function introduced through a view, because no secret column name appears in the query. On PostgreSQL, before the query runs, the tool plans it with `EXPLAIN (VERBOSE, FORMAT JSON)` inside the same read-only transaction - planning only, nothing executes, a VOLATILE function included (the planner does not fold one). The plan is refused when it:
80
+
81
+ - reads a relation whose columns the secret column rule could not read, in any schema: "Blocked: `<schema.relation>` is not queried: its columns could not be read, so rails_query cannot tell which of them are secret.";
82
+ - reads a materialized view that the view rule of Layer 4 refuses, directly or through a view over it;
83
+ - reads one of the relations or functions Layer 1 refuses by name (`pg_stats`, `pg_stat_activity`, `pg_settings` and the rest), which a view or an escaped name can reach where the text check saw nothing;
84
+ - calls a function that returns a whole table as one value, or reads a setting that can hold a password (`current_setting` with a refused name, or with a name the plan does not show as one constant);
85
+ - holds a **whole-row reference** (`alias.*`) to a relation that has a secret column - this closes `row_to_json(u)`, `to_json(u)`, `u::text`, `array_agg(u)`, `json_agg(u)`, `hstore(u)`, `concat(u.*)`, `string_agg(u::text, ...)`, a bare `SELECT u`, and the same through a view, subquery or CTE. An alias is read as the plan prints it, so two aliases that differ only by case are two relations. The planner numbers an alias a statement uses twice (`p_1` for the `p` inside a subquery), and that name stands for its own relation alone. The number is taken off only for the members of an Append or a Merge Append, which is how the partitions of one table are named. The refusal names the relation whose row the statement reads;
86
+ - **uses a secret column** anywhere but as a plain pass-through output - a `ROW(u.*)` the planner expanded into named columns, or a `WHERE` / `ORDER BY` oracle, an ordinal `ORDER BY 3` over a wildcard included. The statement a `postgres_fdw` foreign scan sends to the remote server (the plan's `Remote SQL`) is read the same way, so a condition, a sort or a group on a secret column of a foreign table is refused where the remote server would run it. A column that carries another name on the remote server (the `column_name` option) is told by one more catalog read, made only for a plan that holds a foreign scan. A stated limit: only the remote statement of `postgres_fdw` is read, and another foreign data wrapper prints keys of its own. A join that `postgres_fdw` runs on the remote server has one scan for all its tables, which names them with their aliases in its `Relations` line: they are read from there, so a whole row of one is refused and a column of one is traced and filtered like a local table's;
87
+ - **passes a secret column** through an intermediate the plan's top node does not name - a `MATERIALIZED` CTE, a subquery scan, or a set operation (`INTERSECT` / `EXCEPT`) - so its value would reach a result column that the output-provenance redaction (Layer 4) cannot pin. These are refused, since no redaction could reach them. A column that a scan hands to an aggregate or a join is counted only when that node names it, so `count(*)` over a table with a secret column runs;
88
+ - names any **VOLATILE function** (`pg_proc.provolatile = 'v'`) outside a short read-only allowlist. `pg_proc` marks a function VOLATILE when its answer may change within one statement, which is as true of a reader of live server state as of a writer, so VOLATILE alone does not mean "changes state". The allowlist holds only built-in C functions that write nothing: value generators (`random`, `clock_timestamp`, `gen_random_uuid`, `timeofday`, `uuid_generate_v4`, and `pg_sleep*`, which the statement timeout bounds); the size readers (`pg_total_relation_size`, `pg_relation_size`, `pg_table_size`, `pg_indexes_size`, `pg_database_size`, `pg_tablespace_size`); the lock-table snapshot readers (`pg_lock_status` - what the `pg_locks` view reads - `pg_blocking_pids`, `pg_safe_snapshot_blocking_pids`); and the recovery / WAL-position readers (`pg_is_in_recovery`, `pg_is_wal_replay_paused`, `pg_get_wal_replay_pause_state`, `pg_current_wal_lsn`, `pg_current_wal_insert_lsn`, `pg_current_wal_flush_lsn`, `pg_last_wal_receive_lsn`, `pg_last_wal_replay_lsn`, `pg_last_xact_replay_timestamp`). The functions the statement's text names are checked beside those the plan prints, since a plan prints no row of a `VALUES` list and no `LIMIT` expression. Anything else VOLATILE is not run, and the refusal says plainly that the function is VOLATILE and outside the read-only list. This closes the admin-function family without a name list that has to grow with every PostgreSQL release.
89
+
90
+ Every statement except `SHOW` is planned, and an `EXPLAIN` is planned as the statement it explains. A statement whose plan cannot be read does not run: when planning times out the answer is the timeout, when planning fails it is the SQL error, and when the server returns no plan the statement is refused.
91
+
92
+ A whole-row read of a relation with no secret column (`row_to_json(posts.*)`) is allowed. Before these checks scan a plan expression, its string constants are masked (PostgreSQL's deparser always doubles quotes and never emits an `E''` string, so a constant is matched exactly), so a secret word inside a `'%key%'::text` filter or a literal argument is not mistaken for a column or a function.
93
+
94
+ A subquery or a join over a table that holds a secret column runs when it reads none of them. A scan under an aggregate or a join hands up every column of its table, and the plan check counts a secret column only when the aggregate or the join names it. So `SELECT (SELECT count(*) FROM users) AS n`, `count(id)`, `max(id)` and `EXISTS (SELECT 1 FROM users)` run, while `max(api_token)`, `count(api_token)`, `string_agg(api_token, ',')`, `row_to_json(users)`, a whole row and `count(users)` are refused.
95
+
96
+ #### Relations and settings that are never read
97
+
98
+ A statement that names one of these on the database family that has it is refused. The name is matched on the raw text, so it is refused inside a string literal too. Names are folded the way MySQL folds them before they are matched. The answer is "Blocked: `<name>` is not read. It holds values sampled from other tables, other sessions' statements or stored credentials, and rails_query cannot tell which of them are secret." For a relation or function that lists settings it ends " To read one setting, run SHOW with its name."
99
+
100
+ | Database | Never read | What it holds |
101
+ |:---------|:-----------|:--------------|
102
+ | PostgreSQL | `pg_stats`, `pg_stats_ext`, `pg_stats_ext_exprs`, `pg_statistic`, `pg_statistic_ext`, `pg_statistic_ext_data`, `pg_largeobject` | values sampled or copied from tables |
103
+ | PostgreSQL | `pg_stat_activity` and the functions it reads through (`pg_stat_get_activity`, `pg_stat_get_backend_activity`), `pg_stat_statements`, `pg_prepared_statements`, `pg_cursors` | other sessions' and earlier statements |
104
+ | PostgreSQL | `pg_authid`, `pg_shadow`, `pg_db_role_setting` | password hashes, and the settings stored for a role or a database |
105
+ | PostgreSQL | `pg_user_mapping`, `pg_user_mappings`, `information_schema.user_mapping_options`, `pg_subscription`, `pg_foreign_server`, `pg_foreign_data_wrapper`, `pg_foreign_table` | connection options, which can carry a password |
106
+ | PostgreSQL | `pg_settings`, `pg_file_settings`, `pg_show_all_settings()`, `pg_show_all_file_settings()`, `pg_hba_file_rules`, `pg_ident_file_mappings` | every setting, and the authentication files |
107
+ | MySQL and MariaDB | anything after `mysql.` or `sys.`, `performance_schema`, `global_priv` | accounts and their password hashes, other sessions' statements |
108
+ | MySQL and MariaDB | `processlist`, `general_log`, `slow_log`, and in `information_schema`: `INNODB_TRX`, `INNODB_LOCKS`, `INNODB_LOCK_WAITS`, the `INNODB_FT_` tables, `USER_STATISTICS`, `CLIENT_STATISTICS`, `PROFILING`, `OPTIMIZER_TRACE` | other sessions' statements, locked rows, indexed words |
109
+ | MySQL and MariaDB | `column_statistics`, `column_stats` | values sampled from tables |
110
+ | SQLite | `sqlite_stat1`, `sqlite_stat3`, `sqlite_stat4` | values sampled from indexes |
111
+ | SQLite | `sqlite_stmt` | the connection's own statements with their literals |
112
+
113
+ Each name belongs to one database family and is refused there alone. A PostgreSQL catalog name is not refused on MySQL, MariaDB or SQLite, and a MySQL name is not refused on PostgreSQL or SQLite. On its own family a name is matched as a whole word, so an app table that carries one (`slow_log` on MySQL) cannot be queried. The `information_schema` names in the MySQL row are matched only under that schema, so an app's own `user_statistics` table runs.
114
+
115
+ The `mysql` and `sys` schemas are told by a name before a dot, on text with string literals masked, on MySQL and MariaDB alone. So `'mysql.com'` inside a single-quoted string runs. An alias cannot be told from a schema without a parser, so an alias named `sys` or `mysql` before a dot is refused too. Every such refusal outside double-quoted text, a real `FROM mysql.user` included, ends " A name before a dot is read as that schema, so an alias named `sys` needs another name.", with `mysql` where that is the name. A double-quoted text that holds `mysql.` or `sys.` is refused too, and its answer ends " The name stands before a dot inside double-quoted text, which MySQL reads as a name when the session's sql_mode holds ANSI_QUOTES, and rails_query does not read the session's mode. Write the text in single quotes."
116
+
117
+ On PostgreSQL the plan is checked for the same names, as relations and as functions, so a view over one is refused: `information_schema.user_mappings`, `information_schema.foreign_servers` and `pg_stat_subscription` are refused that way. `pg_authid`, `pg_shadow` and `pg_db_role_setting` are the exception. A view over them runs, and their `rolpassword` and `setconfig` columns are secret columns: `pg_roles`, `pg_user` and `information_schema.schemata` run, `SELECT rolconfig FROM pg_roles` comes back `[FILTERED]`, and an expression over either column is refused.
118
+
119
+ PostgreSQL settings are read one at a time, and the ones that can hold a secret are refused:
120
+
121
+ - `primary_conninfo`, `ssl_passphrase_command`, `archive_command`, `restore_command`, `archive_cleanup_command` and `recovery_end_command` are refused through `SHOW` and through `current_setting`. So is a setting whose own name reads as a secret by the rule of Layer 4, such as `app.jwt_secret`. The answer is "Blocked: the setting `<name>` is not read. It can hold a password, a key or a command line that carries one." On PostgreSQL the six names are refused wherever they stand in a statement, inside a string literal too
122
+ - `SHOW ALL` is refused on PostgreSQL: "Blocked: SHOW ALL is not run. It lists every setting, the ones that can hold a password among them. SHOW one setting by its name."
123
+ - `SHOW` has no plan, so on PostgreSQL its name has to be plain: "Blocked: SHOW takes a setting's plain name on PostgreSQL, with no quotes around it, for example `SHOW work_mem`." `SHOW TIME ZONE` and `SHOW TRANSACTION ISOLATION LEVEL` run
124
+ - `current_setting` is refused unless its first argument is one plain string, in the text and in the plan: "Blocked: current_setting runs only with the setting's name written as one plain string, for example `current_setting('work_mem')`, so that rails_query can read which setting it returns."
125
+
126
+ Stated limits:
127
+
128
+ - **A function whose name needs quotes cannot be called.** A mixed-case function on PostgreSQL is one. Its name cannot be written without quotes, and a quoted name before a parenthesis is refused.
129
+ - **MySQL and MariaDB variables still run.** `SHOW VARIABLES`, `@@name` and MariaDB's `information_schema.GLOBAL_VARIABLES`, `SYSTEM_VARIABLES` and `SESSION_VARIABLES` are read as written.
130
+ - **Relations of third-party PostgreSQL extensions are not named.** Beyond `pg_stat_statements`, an extension's relation that holds statement text or sampled values is not in the list.
131
+ - **On MySQL, MariaDB and SQLite a view the app defines over a never-read relation is read.** The never-read names are held to the statement's own text, so a view the app itself defines over a statistics table or a session table is read as the app defined it. PostgreSQL refuses it, because the plan names the relation.
76
132
 
77
133
  ### Layer 2 - Database-level read-only
78
134
 
79
- After validation, the query runs inside a transaction:
80
-
81
- | Database | Mechanism |
82
- |:---------|:----------|
83
- | PostgreSQL | `SET TRANSACTION READ ONLY` + `SET LOCAL statement_timeout` |
84
- | MySQL | `SET TRANSACTION READ ONLY` + `MAX_EXECUTION_TIME` hint |
85
- | MariaDB | `SET TRANSACTION READ ONLY` + `SET STATEMENT max_statement_time = <seconds> FOR <query>` (MariaDB ignores MySQL's hint, so a long query would otherwise run unbounded) |
86
- | SQLite | A read-only connection in a child process, killed at timeout |
87
-
88
- On PostgreSQL and MySQL the query executes inside a transaction, then rolls back (even if it could write, it can't). Any other adapter has no database-level guard: the query runs with Layer 1 validation and the row limit only.
89
-
90
- A SQLite query runs in-process under `PRAGMA query_only = ON`, with no time
91
- limit, for an in-memory database, on a platform without `fork`, or when it
92
- needs a function, virtual-table module, collation or encryption key only the
93
- app's own connection has. The table and EXPLAIN answers then say so; CSV output
94
- stays plain data.
95
-
96
- On sqlite3 1.x a query fails at its timeout, with the statement-timeout error,
97
- when another connection in the same process held an exclusive lock on a
98
- rollback-journal database as it started (`BEGIN EXCLUSIVE`): the child
99
- inherits that lock record and it never clears. A lock held
100
- by another process is waited on as usual, and an open write transaction or any
101
- lock in WAL mode does not block it.
135
+ After validation, the query runs through its database family, which knows how its database spells a read-only transaction and a statement timeout:
136
+
137
+ | Family | Mechanism | A timeout arrives as |
138
+ |:-------|:----------|:---------------------|
139
+ | PostgreSQL (and PostGIS) | `SET TRANSACTION READ ONLY` + `SET LOCAL statement_timeout` | `ActiveRecord::QueryCanceled` |
140
+ | MySQL (mysql2 and Trilogy) | `SET TRANSACTION READ ONLY` + the session's `max_execution_time`, read first, set for the statement and put back afterwards, also when the statement fails | `ActiveRecord::StatementTimeout` (3024), sometimes `QueryCanceled` (1317) |
141
+ | MariaDB | `SET TRANSACTION READ ONLY` + the session's `max_statement_time`, read first, set for the statement and put back afterwards (MariaDB has no `max_execution_time` and ignores MySQL's hint) | native code 1969 |
142
+ | SQLite | A read-only connection in a child process, killed at timeout | the gem's own timeout |
143
+
144
+ What bounds each statement a `rails_query` call sends:
145
+
146
+ | Statement | PostgreSQL | MySQL and MariaDB | SQLite |
147
+ |:----------|:-----------|:------------------|:-------|
148
+ | The query, and `explain: true` | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | the child process's deadline |
149
+ | The reads beside it: the schema read behind the secret column rule, and on PostgreSQL the plan, its `pg_proc` lookup and the column trace | `statement_timeout`, a wait on a lock included | the session's statement limit, and `lock_wait_timeout` for a wait on a lock | no statement limit; a wait on another writer ends at the connection's busy timeout |
150
+
151
+ The query has `query_timeout` to itself. The reads beside it share a second `query_timeout`, and each is charged the time it ran, so the statements of a call take about twice `query_timeout` at most. Building the app's context, which a cold call does first, is charged to neither, and so are the gate's checks on the statement's text, which the statement length limit bounds. The schema read is a few statements whatever the number of tables: four at most on MySQL and MariaDB, five on PostgreSQL and six on SQLite. On a SQLite file database where a query process can run, two more reads ask what the app's own connection alone holds, so eight reads come before the statement there. On PostgreSQL a plan that holds a foreign scan adds one catalog read. A statement or a view definition that names another schema or database adds the reads of that one.
152
+
153
+ A read that runs out of that time ends the call with an error that says which side of the statement it was on:
154
+
155
+ - "The reads rails_query makes before a statement (the columns of the schema, and on PostgreSQL the statement's plan) took the whole <N> second timeout, so the statement was not run." A planning timeout on PostgreSQL answers this too.
156
+ - "The statement ran, and the read that traces its result columns to their table columns took what was left of the <N> second timeout the reads beside a statement share, so its rows are not shown."
157
+
158
+ Stated limits of the time bound:
159
+
160
+ - **MySQL and MariaDB lock waits are whole seconds.** Their statement limit does not cover `EXPLAIN` or `SHOW`, so the session's `lock_wait_timeout` is set too and put back afterwards. It takes whole seconds, so a wait on a lock ends at `query_timeout` rounded up to the next second.
161
+ - **SQLite's reads beside the query have no statement limit.** They run on the app's own connection, where a wait on another writer ends at that connection's busy timeout (`timeout` in `config/database.yml`; with none, a busy database fails at once).
162
+ - **A SQLite statement on the app's own connection cannot be ended in the middle of one long step.** The paragraph on that path below says when a statement runs there.
163
+ - **PostgreSQL has no client read timeout.** `statement_timeout` is the server's to keep. A PostgreSQL server that goes silent after it connects is not ended by the client.
164
+ - **On MySQL a `SLEEP()` the statement limit interrupts is no timeout.** MySQL returns 1 from an interrupted `SLEEP()` and raises no error, so the statement answers its rows.
165
+ - **On Ruby 3.1 a malformed statement of tens of kilobytes is slow to report.** When the database's error message carries the bad text (an unterminated quote, a run of `[`), ActiveRecord's handling of that message is slow on a Ruby with no regexp match cache. A 50 KB statement took 17 to 31 seconds to answer its SQL error there, run on the app's own SQLite connection. The time is spent after the database answered, so no limit above ends it. Ruby 3.2 and later report the error at once, and on a SQLite file database the query process ends at the deadline.
166
+
167
+ A database leased by name connects with the `connect_timeout` of its `config/database.yml` entry, or `query_timeout` rounded up to whole seconds when the entry sets none, so a server that never answers the connection ends the call. On MySQL and MariaDB such a connection also gets a `read_timeout` of twice `query_timeout` plus 5 seconds unless its entry sets one, so a server that goes silent after the handshake ends the call. On PostgreSQL `connect_timeout` is the only client bound. Taking the lease down does not replace the outcome of the call: its connections are thrown away and its pool removed, and an error there is not raised. The primary uses the app's own pool and its settings.
168
+
169
+ The MySQL session variable bounds a statement that starts with `WITH` or `EXPLAIN ANALYZE` too; the `MAX_EXECUTION_TIME` hint binds only the outermost `SELECT`, so it is not used. `query_timeout` is a positive number, and PostgreSQL, MySQL and MariaDB bound a value under a millisecond at one millisecond. A timeout is read from the error's class or the database's native code alone, so an SQL error whose message names a column called `timeout` is reported as an SQL error. A wait that PostgreSQL's own `lock_timeout` ended is answered as the timeout. A connection the server dropped is no timeout: on Rails 7.1 and later the answer is an error that starts "The <name> database stopped answering during the query:".
170
+
171
+ On MySQL and MariaDB the session that holds the read-only transaction and the time limit is the only one the statement is sent on. Rails replaces a connection it lost before the transaction begins, and the new session has neither guard. So the session's id is read inside the transaction, with one `SELECT CONNECTION_ID()`, and compared with the one the limits were set on. When they differ the answer is "The <name> database stopped answering during the query: the connection was replaced before the statement was sent, and the new session has no read-only transaction and no time limit, so the statement was not sent".
172
+
173
+ On PostgreSQL and MySQL the query executes inside a transaction, then rolls back (even if it could write, it can't). MySQL's read-only transaction still lets `FOR SHARE`, `LOCK IN SHARE MODE` and `INTO` through, so the Layer 1 checks stay on every family.
174
+
175
+ On a connection that is already inside a transaction (a tool called from a transactional test, a mounted server inside a request's transaction) the guard leaves that transaction as it found it. PostgreSQL guards inside a savepoint, so the outer transaction stays writable. MySQL and MariaDB cannot make an open transaction read-only, so the statement is not sent and the answer is "Query failed: MySQL cannot make a transaction that is already open read-only, so the statement was not sent. Run it outside the transaction." SQLite runs the statement on the app's own connection, so it reads the rows of the open transaction. It runs under `PRAGMA query_only = ON`, and the pragma is put back as it was found.
176
+
177
+ An adapter with no family (SQL Server, Oracle, CockroachDB and any other) is refused: `rails_query` answers with the adapter's name and sends no statement, because it cannot give that database a read-only transaction and a timeout it has verified. The family is read from the database's configuration, so the refusal comes before any connection is opened or verified. CockroachDB's adapter is built on PostgreSQL's, but the adapter's name alone chooses the family, so it is refused too.
178
+
179
+ A SQLite database whose file is not there is refused by name before anything
180
+ connects, with or without a `database` argument, since connecting would create
181
+ the file. The answer names `bin/rails db:prepare`.
182
+
183
+ A SQLite statement runs on the app's own connection, under
184
+ `PRAGMA query_only = ON`, in five cases: an in-memory database, a platform
185
+ without `fork`, a statement the child process could not prepare, a transaction
186
+ open on the app's connection, and a statement that names something the app's
187
+ connection alone holds. The third is a statement that needs a table, function,
188
+ virtual-table module, collation or encryption key only the app's own
189
+ connection has. That case is told from which step failed in the child, and the
190
+ error's text is not read. A statement the child prepared and that fails while
191
+ it runs is answered with its SQL error and is not run again. The fourth is
192
+ there because the child opens the file itself and would answer without the
193
+ rows of the open transaction.
194
+
195
+ The fifth is told before the child starts, by two reads on the app's
196
+ connection: its temp objects and its attached databases. When it holds any of
197
+ them, a statement runs there if it names one of them, the temp catalog
198
+ (`sqlite_temp_master`, `sqlite_temp_schema`) or the `temp` schema, or if it
199
+ calls a pragma function (`pragma_table_info(...)`, `pragma_table_list` and the
200
+ rest) or `dbstat`, whose argument can name any of them. The child has none of
201
+ these and would answer an empty list, the rows of a main table of the same
202
+ name, or "unknown database". A table that only an attached database holds
203
+ needs no match: the child cannot prepare a statement that reads it, which is
204
+ the third case. A stated limit: a name is matched anywhere in the statement's
205
+ text, in any letter case and inside a string too, so a statement that holds
206
+ such a name as plain data runs on the app's connection as well.
207
+
208
+ On the app's own connection a statement that returns rows slowly ends with the
209
+ timeout answer a few rows after `query_timeout` passes. The sqlite3 driver
210
+ holds the interpreter lock while a statement steps, and an interrupt lands only
211
+ between two rows, so the end comes late by the time those rows take. One long
212
+ step cannot be ended there at all. So the table and EXPLAIN answers of such a
213
+ run say it ran without a time limit: "This SQLite query ran without a time
214
+ limit: query_timeout needs a file-backed database, a platform with fork, and
215
+ nothing that only the app's own connection has." CSV output stays plain data.
216
+
217
+ A SQLite database that another connection keeps locked is answered as locked:
218
+ "The database is locked by another connection, so the query did not run. Run it
219
+ again when the other connection has finished." It is an error response. The
220
+ child process waits on a lock for 80% of `query_timeout`, so a lock that stays
221
+ is answered before the timeout. The child tells its parent the moment it meets
222
+ a lock, so a child that started late on a slow machine and still waits at the
223
+ deadline is answered as locked too. The child takes its read lock before it
224
+ prepares the statement and tells its parent once it holds it. No writer can
225
+ stop the statement from there on, so a statement that then runs past the
226
+ deadline answers the timeout, also when a writer asks for the lock while it
227
+ runs. A lock met while the connection opens gets the locked answer. The reads
228
+ on the app's own connection wait for that connection's busy timeout.
229
+
230
+ On sqlite3 1.x a query does not run when another connection in the same
231
+ process held an exclusive lock on a rollback-journal database as it started
232
+ (`BEGIN EXCLUSIVE`): the child inherits that lock record and it never clears.
233
+ A lock held by another process is waited on as usual, and an open write
234
+ transaction or any lock in WAL mode does not block it.
102
235
 
103
236
  ### Layer 3 - Row limit
104
237
 
105
238
  - Default: 100 rows
106
239
  - Configurable: `config.query_row_limit` (hard cap: 1000)
107
- - Applied as a `LIMIT` clause appended to the query, which fetches **one row past the cap** (`cap + 1`): a `LIMIT` or `FETCH FIRST` that ends the query is rewritten to `min(n, cap + 1)`, and one inside a subquery is left as written
240
+ - Applied as a `LIMIT` clause appended to the query, which fetches **one row past the cap** (`cap + 1`): a `LIMIT` or `FETCH FIRST` that ends the query is rewritten to `min(n, cap + 1)`, and one inside a subquery is left as written. `LIMIT ALL`, `LIMIT NULL` and a negative limit mean no limit, so they are rewritten to `cap + 1`
108
241
  - The rows that come back are cut to the cap, so the answer holds it whatever the database made of the text
109
- - The note that rows were held back is added **only when more than the cap came back** - the extra row proves the cap actually cut the answer - then the extra row is dropped. A result that exactly fills the cap (every row fit) gets no note, which fixes the old "N rows shown; the query returned at least this many" appearing when nothing was held back. The note reads `100 rows shown; the query returned more (the row limit of 100; pass limit: up to 1000 to see more)` and rides every format; in CSV it sits after a blank line, outside the comma block, so a parser still reads clean rows
242
+ - The note that rows were held back is added **only when more than the cap came back** - the extra row proves the cap actually cut the answer - then the extra row is dropped. A result that exactly fills the cap (every row fit) gets no note. The note reads `100 rows shown; the query returned more (the row limit of 100; pass limit: up to 1000 to see more).` and rides every format; in CSV it sits after a blank line, outside the comma block, so a parser still reads clean rows
243
+ - CSV output with no rows is the header row alone
110
244
 
111
- ### Layer 4 - Sensitive column rejection
245
+ ### Layer 4 - Secret column rejection
112
246
 
113
- A query that names a sensitive column is **rejected before execution**, not redacted after it. One rule decides what is sensitive, used by the pre-execution refusal, the PostgreSQL plan check and result redaction alike. A column is sensitive when it is:
247
+ A query that names a secret column is **rejected before execution**, and a secret column's values are redacted where a `SELECT *` returns them. One rule (`RailsAiContext::SecretColumns`) decides what is secret, used by the pre-execution refusal, the PostgreSQL plan check and result redaction alike. A column is secret when it is:
114
248
 
115
249
  - in `config.query_redacted_columns`, or the fixed built-in list;
116
250
  - an `encrypts` column of one of your models;
117
- - a name the heuristic catches: it ends in `password`, `secret`, `token`, `key`, `digest` or `hash`, or contains `password`, `secret` or `token`
251
+ - a name the heuristic catches: it ends in `password`, `secret`, `token`, `key`, `digest` or `hash`, or contains `password`, `secret`, `token` or `nonce`, or is `iv` or ends in `_iv`;
252
+ - a name the shared log and config redaction treats as secret (`passwd`, `salt`, `credentials`, `pepper`, `signing_key` and the rest of its secret words), so a word added there is secret here too;
253
+ - a generated column whose expression reads a secret column (described further down)
254
+
255
+ minus anything in `config.query_allowed_columns`. An `encrypts` column is the exception: it is a secret column whatever that list holds. A query that names it is refused, the refusal says the allowed list does not release it, and its values stay `[FILTERED]`.
256
+
257
+ **Default redacted patterns:** `password_digest`, `encrypted_password`, `password_hash`, `reset_password_token`, `confirmation_token`, `unlock_token`, `otp_secret`, `session_data`, `secret_key`, `api_key`, `api_secret`, `access_token`, `refresh_token`, `jti`. Those are the defaults of `config.query_redacted_columns`. The fixed built-in list adds `password_reset_token`, `remember_token`, `secret`, `private_key`, `authentication_string`, `rolpassword` and `setconfig`. The last three are where MySQL and PostgreSQL keep password hashes and stored settings.
258
+
259
+ The pre-execution refusal matches on two things, as whole identifiers. Names are folded the way the databases compare them: lowercase, with MySQL's reading of a dotted capital I as `i`.
260
+
261
+ - **the fixed names above**, on every family whether or not they are real columns;
262
+ - **every real column of your schema that the rule flags** - read from the connection the query runs on, so a database named with `database` is checked against its own columns. This catches an app-specific column the heuristic flags (`api_token`, `auth_token`) through any alias or expression. Rails' own bookkeeping tables (`ar_internal_metadata`, `schema_migrations`) are left out: their columns hold no secrets, and `ar_internal_metadata.key` ends in "key", so counting it would refuse every query whose text held the word "key".
118
263
 
119
- minus anything in `config.query_allowed_columns`.
264
+ The rule follows what a statement reaches. It starts from these columns, by database family:
120
265
 
121
- **Default redacted patterns:** `password_digest`, `encrypted_password`, `password_hash`, `reset_password_token`, `confirmation_token`, `unlock_token`, `otp_secret`, `session_data`, `secret_key`, `api_key`, `api_secret`, `access_token`, `refresh_token`, `jti`. Those are the defaults of `config.query_redacted_columns`. The fixed built-in list adds `password_reset_token`, `remember_token`, `secret` and `private_key`.
266
+ | Family | The columns the rule reads |
267
+ |:-------|:---------------------------|
268
+ | PostgreSQL | Every relation on the search path that stores rows: tables, partitioned tables, materialized views, foreign tables and sequences. A sequence holds no secret column, so `SELECT last_value FROM <sequence>` runs. Then every schema the statement's text names, and every relation the statement's plan names, in any schema. The session's temporary tables are read like any table. The system schemas are left out: the never-read list and the catalog columns `rolpassword` and `setconfig` hold what is secret there |
269
+ | MySQL and MariaDB | The base tables of the connection's database. Then those of every database the statement's text names. `mysql`, `sys`, `performance_schema` and `information_schema` are left out: their relations are refused by name or hold no app data |
270
+ | SQLite | The tables of `main`, of `temp` and of every attached database |
122
271
 
123
- The pre-execution refusal matches on two things, case-insensitive and word-bounded:
272
+ A schema or database is told by a name before a dot, and the server says which of those names are schemas. Each name is read backward from its dot, bare or in backticks or double quotes, so nothing written before the name changes how it is read, a quote character inside a string included. A name in brackets is not read: SQLite reads every schema whatever a statement names, and on MySQL and MariaDB Layer 1 refuses the bracket. A view the statement names can name a further schema or database in its definition, and a view named there a further one. The rule follows those views three reads deep. A view no statement names is not followed, so a statement that names no other schema or database and no such view costs no extra read. On PostgreSQL the plan reads through a plain view, so only a materialized view's definition is followed. A statement that names a view that leads deeper is not run: "The view `<view>` leads through more databases than rails_query follows, so the secret columns it could read cannot be told and no statement that names it runs." On PostgreSQL the sentence says "schemas". A schema whose columns cannot be read stops the statement too: "The columns of `<name>` could not be read, so the secret columns there cannot be told and no query that reads it runs." On PostgreSQL a relation the plan names and the rule could not read is refused by the plan check of Layer 1b. So a statement that names a secret column of a table in another schema or database is refused by name, as one in the connection's own is.
124
273
 
125
- - **the fixed names above**, on every adapter whether or not they are real columns, so the check holds even with no connection;
126
- - **every real column of your schema that the rule flags** - read from the live connection, falling back to the cached schema. This catches an app-specific column the heuristic flags (`api_token`, `auth_token`) through any alias or expression, not only an exact configured name. Rails' own bookkeeping tables (`ar_internal_metadata`, `schema_migrations`) are left out: their columns are never secrets, and `ar_internal_metadata.key` ends in "key", so counting it refused every query whose text held the word "key".
274
+ A refusal names a view, a table or a relation as the database spells it. The rule itself compares names case folded, with a run of spaces as one space. So two names that differ by a run of spaces alone are one name to the rule, and a refusal names the first of them it read.
127
275
 
128
- So `SELECT password_digest AS pd FROM users`, `SELECT upper(api_token) FROM users` and `SELECT (SELECT api_token FROM users LIMIT 1) AS leaked` are all refused: post-execution redaction reads the output column names, which the caller controls through aliases and expressions, so it cannot be relied on. A table or alias that merely contains a sensitive word (a `tokens` table) does not trip the check - only a real column name does. `SELECT *` stays allowed, and its sensitive columns are redacted in the result.
276
+ The schema read fails closed. When the columns of the database cannot be read, no query runs on it and the answer is "The columns of the <name> database could not be read, so its secret columns cannot be told and no query runs on it." Only the primary falls back to the cached context's schema, and it is refused the same way when that has no tables either.
129
277
 
130
- A sensitive word inside a **string literal** - a `LIKE '%secret%'` pattern, a JSON key `->>'token'` - is data, not a column reference. On PostgreSQL the check runs on literal-masked text, and the plan layer (Layer 1b) then has to vouch: a query that passed only because a name sat inside a literal runs only when its plan was obtained and cleared every check, so a masker fooled by a quote trick (`standard_conforming_strings` off, an `E''` string) can never be the only gate - if planning fails, the raw-text refusal stands. On MySQL, MariaDB and SQLite the match stays on raw text with no masking, because a quote's meaning there depends on server modes (`NO_BACKSLASH_ESCAPES`, `ANSI_QUOTES`) or a dotted single-quoted name is an identifier (`t.'col'`).
278
+ So `SELECT password_digest AS pd FROM users`, `SELECT upper(api_token) FROM users` and `SELECT (SELECT api_token FROM users LIMIT 1) AS leaked` are all refused: post-execution redaction reads the output column names, which the caller controls through aliases and expressions, so it cannot be relied on. The refusal starts "Blocked: query references secret column `api_token`." and says how to release a name that holds no secret. A table or alias that merely contains a secret word (a `tokens` table) does not trip the check - only a real column name does. `SELECT *` stays allowed, and its secret columns are redacted in the result.
131
279
 
132
- If one of your own columns merely looks sensitive (an `oauth_applications.secret`, say), exempt it by name:
280
+ A secret word inside a **string literal** - a `LIKE '%secret%'` pattern, a JSON key `->>'token'` - is data. On PostgreSQL the check runs on literal-masked text, and the plan layer (Layer 1b) then has to vouch: a query that passed only because a name sat inside a literal runs only when its plan was obtained and cleared every check, so a masker fooled by a quote trick (`standard_conforming_strings` off, an `E''` string) is not the only guard. A statement with no plan to read, such as `SHOW`, keeps the raw-text refusal. On MySQL, MariaDB and SQLite the match stays on raw text with no masking, because a quote's meaning there depends on server modes (`NO_BACKSLASH_ESCAPES`, `ANSI_QUOTES`) or a dotted single-quoted name is an identifier (`t.'col'`).
281
+
282
+ If one of your own columns merely looks secret (an `oauth_applications.secret`, say), exempt it by name:
133
283
 
134
284
  ```ruby
135
285
  config.query_allowed_columns = %w[secret]
136
286
  ```
137
287
 
138
- Results are redacted as well: a returned column comes back as `[FILTERED]` when the sensitivity rule flags its **output name**, and on PostgreSQL when its origin is a sensitive base column - either by **provenance** (`PG::Result#ftable` / `#ftablecol`, mapped to the base column) or by the **plan's top-node `Output`**, whose i-th expression is result column i. The plan path catches a view (or inlined CTE/subquery/join) that renamed a sensitive column to a harmless name: `CREATE VIEW user_tokens AS SELECT id, api_token AS t FROM users; SELECT t FROM user_tokens` returns `t` as `[FILTERED]`. `SHOW`, `DESCRIBE` and `EXPLAIN` output skips column-name redaction; the one exception is `SHOW CREATE TABLE` / `SHOW CREATE VIEW`, whose definition text is run through the shared secret redaction so a credential embedded in it is filtered.
288
+ On a standalone install, where no initializer runs, put the name under `query_allowed_columns:` in `.rails-ai-context.yml`.
289
+
290
+ Results are redacted as well: a returned column comes back as `[FILTERED]` when the rule flags its **output name**, and on PostgreSQL when its origin is a secret base column - either by **provenance** (`PG::Result#ftable` / `#ftablecol`, mapped to the base column) or by the **plan's top-node `Output`**, whose i-th expression is result column i. The plan path catches a view (or inlined CTE/subquery/join) that renamed a secret column to a harmless name: `CREATE VIEW user_tokens AS SELECT id, api_token AS t FROM users; SELECT t FROM user_tokens` returns `t` as `[FILTERED]`. A quoted mixed-case secret column (`"Salt"`) is filtered in place the same way. So is a secret column under a wildcard over a partitioned table or an inheritance parent: each partition is read as its own relation, a result column is filtered when any of them hands a secret column up in that place, and a whole-row read is refused when any of them holds one. An expression beside the wildcard (`SELECT *, id + 1 AS next`) changes nothing: each plain result column keeps its table column, and each partition's expressions are checked as expressions. The arms of a `UNION ALL` inside a view are read the same way: each arm against its own table, so no arm stands for another. A result column whose table's column names cannot be read is filtered when that table holds a secret column. When the table cannot be told either, it is filtered when any table holds one. On SQLite a second result column of the same name comes back as `name:1`, and the rule reads that name without its suffix. `SHOW`, `DESCRIBE` and `EXPLAIN` output skips column-name redaction; the one exception is `SHOW CREATE TABLE` / `SHOW CREATE VIEW`, whose definition text is run through the shared secret redaction so a credential embedded in it is filtered.
291
+
292
+ Some relations are refused whole, because no result column of them is traced to a table column. MySQL, MariaDB and SQLite trace none at all, so there two refusals stand in for the plan checks:
293
+
294
+ - **A view is checked by its definition.** A statement that names a view is refused when the view's definition names a secret column, when the view is built on such a view, when its definition has a wildcard and names a relation that holds a secret column, or when its definition cannot be read on a database that holds secret columns. The wildcard test is those two facts together: the rule does not tell which relation the wildcard reads. The definition is read with its comments taken out, so a secret column's name inside a comment refuses nothing, and a view's name is matched as an identifier whatever characters it holds. The answer is "Blocked: the view `<view>` is not queried: <reason>. A view can rename a column past the redaction on <family>, so query the tables it reads." The reason is one of "its definition names the secret column `<column>`", "its definition has a wildcard and names `<relation>`, which holds the secret column `<column>`", "its definition could not be read, and this database holds secret columns", "it reads the view `<view>`:" with that view's reason, and on SQLite "its definition names `<table>`, which is not read". `SHOW CREATE VIEW` still prints the definition.
295
+ - **A `*` inside a subquery is refused when the statement names a relation that holds a secret column.** With no plan to read, a value a wildcard returns could be compared or renamed unseen. The test is those two facts together: the statement has a wildcard inside a subquery, and it names such a relation. Which relation the wildcard reads is not told without a parser. The answer is "Blocked: a `*` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. On <family> a value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery." A CTE body and a derived table count. So does `(TABLE t)` on MySQL and MariaDB, the two families that have the statement and read no plan, with an answer of its own: "Blocked: `TABLE` inside a subquery is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `TABLE t` reads every column of `t`, and on <family> a value read that way can be compared or renamed past the redaction, so SELECT the columns you need in the subquery." `EXISTS (SELECT * ...)` returns no column and `count(*)` reads none, so both run. A relation's name inside a string literal is data on MySQL and MariaDB. SQLite reads a string as a table's name wherever a name can stand, so there a string equal to such a relation's name counts as that name, wherever it stands: "Blocked: a `*` inside a subquery is not run in a statement that holds the string '<relation>'. SQLite reads a string as a table's name where a name can stand, and the table `<relation>` holds the secret column `<column>`. A value read through such a wildcard can be compared or renamed past the redaction, so name the columns in the subquery."
296
+ - **On SQLite `IN` before a relation's name is held to the same rule.** `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, a wildcard subquery with no `*` written. It is refused wherever it stands when the statement names a relation that holds a secret column, or holds a string equal to such a relation's name: "Blocked: `IN` before a table's name is not run in a statement that names `<relation>`, which holds the secret column `<column>`. `x IN t` is SQLite's shorthand for `x IN (SELECT * FROM t)`, and a value read through such a wildcard can be compared past the redaction, so write the subquery with its columns named." `IN (...)` with a list or a subquery in parentheses runs.
297
+
298
+ On PostgreSQL a plain view is traced through the plan and its secret columns are filtered in place. A **materialized view** stores its rows, so its result columns have no table column to be traced to, and it is held to the view rule above: a statement that reads one whose definition names a secret column, directly or through a view, is refused with the view answer. So is a statement that reads it through a view over it, since the plan names it. A materialized view is told by its schema and name, as the plan gives them, so a table of the same name in another schema runs.
299
+
300
+ On SQLite a table that holds another table's rows or words under other names is not read. The name is matched on the raw text, inside a string literal too:
301
+
302
+ - **The page modules.** `sqlite_dbpage`, `sqlite_dbdata` and `sqlite_dbptr`: "Blocked: `sqlite_dbpage` is not read: it returns the database file's raw pages, and every stored value is in them."
303
+ - **A shadow table of a virtual table.** "Blocked: `notes_content` is not read: it is a shadow table, where SQLite keeps what the virtual table `notes` stores under other column names. Query `notes`." The last sentence is left out when the virtual table itself is refused. Before SQLite 3.37, which cannot type a shadow table, a table whose name starts with a virtual table's name and an underscore is read as one.
304
+ - **A virtual table that holds a secret column.** A search, a highlight or a snippet over it can return or confirm that column's values, so it is refused whole: "Blocked: `docs` is not read: it is a virtual table that holds the secret column `api_token`, and a search or a function over a virtual table reads a column by its position, with no name written." Only the columns a virtual table declares decide this. A full-text table has a hidden column of its own name, and that one does not count, so a table named `token_search` runs. A virtual table with no secret column runs.
305
+ - **An `fts5vocab` or `fts4aux` table over such a table.** "Blocked: `docs_words` is not read: it lists the words of the full-text table `docs`, which holds the secret column `api_token`."
306
+ - **A virtual table built on a refused view, or on a table that is not read.** Its rows, or its words, are theirs: "Blocked: `ren_fts` is not read: its definition names the view `renamed`, which is not queried: its definition names the secret column `api_token`." For a table the reason is "its definition names `<table>`, which is not read". A virtual table's definition can name a relation with its module and with every argument, except the columns a full-text table (fts3, fts4, fts5) declares and the type an `fts5vocab` table ends with (`row`, `col` or `instance`): a column or a type that shares a view's name changes nothing. An argument with `=` is an option and is read whole, whatever its name. Comments are taken out before any of it is read, and the module is read after the table's own name, so a name that spells a module changes nothing. A vocabulary table also names the table it lists the words of. An `fts5vocab` or `fts4aux` table over such a virtual table is refused that way too.
307
+ - **A view over any of these**, by the view rule above.
308
+
309
+ `dbstat` runs: it gives sizes and no stored value.
310
+
311
+ Stated limits on MySQL, MariaDB and SQLite, where no plan is read: `SELECT * FROM t INTERSECT SELECT <a guessed row>` (and `EXCEPT`) tells whether a whole row equals the guess, and an ordinal `ORDER BY 3` over a wildcard tells the order of a secret column's values. Neither returns a value: the secret columns of the rows that come back are `[FILTERED]`. PostgreSQL refuses both by the plan. On SQLite a string literal equal to the name of a relation that holds a secret column refuses a statement with a wildcard in a subquery, also when the string is plain data there: the rule cannot tell the two uses apart without a parser.
312
+
313
+ On every family a generated column whose expression reads a secret column is a secret column itself: a statement that names it is refused and its values come back `[FILTERED]`. SQLite stores no expression per column, so each one is cut from its table's definition. A generated column whose part of the definition cannot be found is read by the whole definition, and is then secret once anything in it names a secret column. A generated column is known by its name. When two tables each have one of the same name, that name is secret when either reads a secret column, and it is held as an `encrypts` column when either reads one. `config.query_allowed_columns` releases a generated column, or a column a view reads, by name, unless it reads an `encrypts` column.
314
+
315
+ A stored procedure or function is a stated limit: its body is not read, so a function that returns a secret column's value returns it, and the value is filtered by its output name only.
316
+
317
+ A column is known by the name it has in this database, which is a stated limit too. A foreign table that gives a secret column of the remote table another local name (the `column_name` option), a column a trigger fills from a secret column and a table built from another are read by their own names, so their values come back unless that name is secret itself. `config.query_redacted_columns` takes such a name.
139
318
 
140
319
  The exemption covers the results too: an allowed name comes back unredacted. A
141
- column declared with `encrypts` stays `[FILTERED]` either way - an encrypted-at-rest
142
- attribute is never meant to be read back raw.
320
+ column declared with `encrypts` is refused by name and stays `[FILTERED]` either
321
+ way: an encrypted-at-rest attribute is not meant to be read back raw.
143
322
 
144
323
  ### Environment guard
145
324
 
146
325
  > [!WARNING]
147
326
  > Disabled in production by default. Only enable with `config.allow_query_in_production = true` if you understand the implications.
148
327
 
328
+ The refusal is an error response, `isError` over MCP and exit 1 from the CLI: "rails_query is disabled in production for data privacy. Set config.allow_query_in_production = true to override."
329
+
149
330
  ---
150
331
 
151
332
  ## Sensitive file blocking
@@ -252,7 +433,7 @@ never above the git root and never outside a git repository). Only `package.json
252
433
  lockfiles and the presence of a bundler config (`vite.config.*` and similar) are
253
434
  read there, plus a configured entry's `tsconfig.json` and the configs it extends
254
435
  inside that same directory. Sensitive patterns and symlink containment apply relative to that
255
- directory, and frontend_stack names it in its answer.
436
+ directory, and the `frontend` section of `rails_get_config` names it in its answer.
256
437
 
257
438
  ### The bundle config/boot.rb declares
258
439
 
@@ -412,7 +593,7 @@ The engine builds one transport per process, under a mutex, outside the controll
412
593
 
413
594
  ## Credential handling
414
595
 
415
- - `rails_get_env` returns credential **keys**, never values
596
+ - `rails_get_config(section:"env")` returns credential **keys** and no values
416
597
  - The process's environment variable values are never read. A default written in code, and a placeholder from `.env.example`, `.env.sample` or `.env.template`, is shown after redaction
417
598
  - `config/credentials/*.yml.enc` is in the sensitive patterns list
418
599
 
data/docs/SETUP.md CHANGED
@@ -144,7 +144,7 @@ The MCP tools rule uses `alwaysApply: false` with a descriptive `description:` f
144
144
 
145
145
  ```markdown
146
146
  ---
147
- description: "Rails MCP tools reference - 45 tools for schema, models, routes, controllers, search, testing, and more"
147
+ description: "Rails MCP tools reference - 35 tools for schema, models, routes, controllers, search, testing, and more"
148
148
  alwaysApply: false
149
149
  ---
150
150
  ```
data/docs/STANDALONE.md CHANGED
@@ -62,7 +62,7 @@ rails-ai-context version # Show version
62
62
  3. **YAML config** - uses `.rails-ai-context.yml` instead of a Ruby initializer
63
63
  4. **Steps aside for the app's own copy** - in an app whose bundle carries its own rails-ai-context (another installed version, or a `path:` copy), a command that boots the app runs through that copy, as `bundle exec rails-ai-context` would, and says so on stderr. That is the copy the app's MCP configs start, and two copies in one process would each load half the gem. The bundle is the one `BUNDLE_GEMFILE` names when you set it (a dual boot's `Gemfile.next`), and a command typed in a subdirectory runs from the app root it found. `init` asks its questions itself and has that copy write only the context files; an MCP-only setup, chosen with `--mcp-only` or at the prompt, runs nothing of the app's. So does `--no-boot`: bundle exec would evaluate the app's Gemfile and start whatever copy the app names, which on a repo just cloned is code nobody has read. A bundle not installed yet (`bundle check` fails) is left alone too: its boot fails, and the static tier answers. A copy installed only under a vendored bundle path or from git is not seen from outside the bundle, so there the installed copy runs
64
64
 
65
- This means you get the same 45 tools, same MCP server, same context generation - without touching the project's Gemfile.
65
+ This means you get the same 35 tools, same MCP server, same context generation - without touching the project's Gemfile.
66
66
 
67
67
  ## Configuration via YAML
68
68
 
@@ -158,7 +158,7 @@ rails-ai-context uninstall
158
158
  gem uninstall rails-ai-context # once no other app uses it
159
159
  ```
160
160
 
161
- The MCP config files are updated automatically. Both modes generate the same set of context files and provide the same 45 tools. The commands named inside the files follow the install mode.
161
+ The MCP config files are updated automatically. Both modes generate the same set of context files and provide the same 35 tools. The commands named inside the files follow the install mode.
162
162
 
163
163
  Both also take `--mcp-only`, which writes the MCP config and no context files
164
164
  at all. See [CONFIGURATION.md](CONFIGURATION.md#mcp-only).