@carllee1983/dbcli 1.58.0 → 2.1.0

package/assets/reference.md

@@ -334,9 +334,91 @@ dbcli query "SELECT day, dau FROM dau_daily" --ui                # open in brows
 dbcli query "SELECT * FROM orders" --format html > orders.html   # pipe to stdout
 ```
 
-**Options:** `--format <table|json|csv|html>`, `--ui` (open the dashboard in the system browser; implies `--format html`), `--limit <number>`, `--no-limit`, `--collection <name>` (MongoDB / Elasticsearch), `--index <name>` (Elasticsearch alias for `--collection`), `--fields <list>`, `--truncate <number>` / `--no-truncate`, `-f, --query-file <path>`, `--use <name[,name]>`, `--slow-ms <number>`, `--recovery`
+**Options:** `--format <table|json|csv|html>`, `--ui` (open the dashboard in the system browser; implies `--format html`), `--limit <number>`, `--no-limit`, `--collection <name>` (MongoDB / Elasticsearch), `--index <name>` (Elasticsearch alias for `--collection`), `--fields <list>`, `--truncate <number>` / `--no-truncate`, `-f, --query-file <path>`, `--use <name[,name]>`, `--slow-ms <number>`, `--yes`, `--recovery`
 **Permission:** query-only+ (Redis: per-command; Elasticsearch: per HTTP method/path)
 
+#### Write confirmation gate (2.0.0)
+
+A SQL write passes through a two-tier gate before the connection is opened. The gate is
+separate from the permission axis: permission says what the connection may do, the gate
+says whether this particular statement may run right now.
+
+| Tier | Statements | Interactive terminal | Non-interactive (or `--format json`) |
+| :--- | :--- | :--- | :--- |
+| One | `INSERT`, `UPDATE` / `DELETE` **with** a `WHERE` or `LIMIT`, `CREATE`, `ALTER` | Summary + `y/N`; `--yes` skips it | Runs, exactly as before |
+| Two | `UPDATE` / `DELETE` with **no** `WHERE`, `DROP`, `TRUNCATE`, unparseable statements, several statements in one string, a write nested inside another statement, a destructive `MERGE` | Type the target table name; **no flag skips it** | **Refused**: exit `1`, nothing sent to the database |
+
+A refusal message names a machine-readable reason — `reason=no_where`, `reason=multi_table`,
+`reason=ddl_destruction`, `reason=unparseable`, `reason=multiple_statements`, `reason=nested_write` — so a caller can tell it apart from a
+connection failure or a permission denial.
+
+**A write that joins another table is tier two, whatever its `WHERE` says.** Whether such a
+write is limited to particular rows is not a property of the statement:
+`DELETE p FROM p JOIN o ON p.id = o.ref WHERE o.x > 0` deleted 2 of 5 rows against one
+dataset and all 2000 against another, and a join's `ON` necessarily names the target, so it
+proves nothing. Five rounds of adversarial measurement went into trying to read this off the
+syntax before the rule became "a second table means tier two" (#80, #93). That refuses the
+standard `UPDATE … FROM`, `UPDATE … JOIN … SET` and multi-table `DELETE` idioms for
+unattended callers: rewrite the other table into a subquery in the `WHERE`, or run them
+where someone can confirm.
+
+**A statement is not the keyword it starts with.** PostgreSQL's data-modifying CTEs put an
+arbitrary write in front of the statement the first keyword names, and a `MERGE` carries its
+writes in a `WHEN … THEN` list — measured against a 2000-row table,
+`WITH moved AS (DELETE FROM p RETURNING *) INSERT INTO archive …` and
+`MERGE INTO p USING (SELECT 1 AS x) s ON true WHEN MATCHED THEN DELETE` each emptied it
+while reading as an ordinary `INSERT` (#94, #95). So a write nested inside another statement
+is tier two, `reason=nested_write`, and a `MERGE` is classified by its actions: a
+`THEN DELETE` or `THEN UPDATE` is tier two, `reason=multi_table`, while an insert-only or
+`DO NOTHING` one stays tier one. `ON CONFLICT … DO UPDATE`, `ON DUPLICATE KEY UPDATE` and
+`INSERT … SELECT` are untouched by this — they are not nested writes, whatever they read
+like. The cost is a CTE that deletes one row by primary key, and the ordinary `MERGE` upsert,
+both refused for unattended callers.
+
+**For a single-table write, the `WHERE` has to be about that table.** `UPDATE p SET c = 1
+WHERE id = 1` is an ordinary write; `UPDATE p SET c = (SELECT max(x) FROM o WHERE o.id = 1)`
+is not, because its only `WHERE` restricts the subquery and the write touches every row. A
+*correlated* reference back to the target does count, so
+`DELETE FROM t WHERE EXISTS (SELECT 1 FROM o WHERE o.tid = t.id)` is tier one while
+`DELETE FROM t WHERE EXISTS (SELECT 1 FROM o WHERE o.id = 1)` deletes every row and is tier
+two. A qualifier the subquery binds itself — including the target's own name, as in
+`DELETE FROM sessions WHERE EXISTS (SELECT 1 FROM sessions WHERE …)` — is about the
+subquery's rows, not the write.
+
+This is a lower bound, not a proof: `WHERE id IS NOT NULL` names the target and still touches
+every row, and no static check settles that. What it rules out is the class where nothing in
+the condition is about the table being written.
+
+**When the parser cannot read the statement**, there is no tree to judge and the same rule is
+applied to the text: tier one only when the statement reads as a write to one named table
+with a `WHERE` or `LIMIT` and contains no `SELECT`, `TABLE`, `VALUES`, `JOIN`, `USING`, a
+statement-level `WITH`, or an `UPDATE … FROM`. A keyword inside parentheses is not at
+statement level, so `SUBSTRING(x FROM 2)` and `AGAINST ('a' WITH QUERY EXPANSION)` do not
+trip it. This is written as an allowlist because the denylist that preceded it was defeated
+once per review round — `USING`, then `JOIN`, then a CTE, then a subquery, then `TABLE` as a
+subquery.
+
+**Escape routes.** For a statement that accepts a `WHERE`, put the intent in the SQL:
+
+```bash
+dbcli query "UPDATE users SET banned = 1"                    # refused, reason=no_where
+dbcli query "UPDATE users SET banned = 1 WHERE 1=1"          # runs — intent is explicit
+dbcli query "DELETE FROM sessions LIMIT 1000"                # runs — damage is bounded
+dbcli query "UPDATE users SET banned = 1 WHERE id = 3" --yes # tier one, question skipped
+dbcli query "UPDATE p SET x = 1 FROM o WHERE o.id = 1"       # refused — the WHERE is about o, not p
+```
+
+This is deliberately not a flag. `WHERE 1=1` appended to a statement that already has a
+`WHERE` is a syntax error, so a blanket "always add it" habit breaks on the first ordinary
+statement; a flag would be harmless everywhere and therefore added everywhere. For `DROP`
+and `TRUNCATE` there is no clause to add — the connection's `permission` level decides
+whether they are possible at all, and the typed confirmation must come from a person.
+
+Every tier-two evaluation is written to the audit log with
+`metadata.write_gate_outcome` (`allowed` / `declined` / `refused`) and
+`metadata.write_gate_reason`. `dbcli audit write-gate` summarizes them — that is
+the measurement ADR 0010 stakes this gate on.
+
 > **Elasticsearch tiers by scope.** A read (`GET` / `HEAD`, plus `POST _search` and
 > `POST _count`) is query-only. Indexing or updating a document is read-write.
 > `DELETE /<index>/_doc/<id>` — one document — is data-admin. Everything that removes or
@@ -955,6 +1037,14 @@ dbcli update users --where "id=1" --set '{"name":"Bob"}' --plan --format json
 > target primary keys first, then issue one `update` / `delete --where "id=<pk>"` per key.
 > (MongoDB `--where` accepts a full JSON filter and is exempt.)
 
+> **Tier two for structured writes (2.0.0)** — a `--where` that matches on no primary key
+> and no unique index selects an unknown number of rows, so it is treated the same as a
+> raw statement with no `WHERE`: the target table name must be typed at an interactive
+> terminal, and a non-interactive run is refused with exit `1` and
+> `reason=non_unique_where`. `--force` and `--dry-run` do not affect this; `--force` skips
+> the ordinary confirmation only. Select the primary keys first and write one row at a
+> time, or run it where a person can confirm it.
+
 ### delete
 
 Delete data from a table.
@@ -1946,6 +2036,7 @@ Audit entries are metadata-only by design — never raw SQL bodies, `--param` va
 | `audit show` | `readonly` | Print a single full entry by id prefix or `--recovery-ref`. |
 | `audit clear` | `local-write` | Delete `<conn>.jsonl` + rotated `.jsonl.1` from local disk. Requires `--yes` or interactive confirm. |
 | `audit health` | `readonly` | Render `AuditLogger.getHealth()` snapshot (writer state, lock state, rotation usage). |
+| `audit write-gate` | `readonly` | Summarize tier-two write-gate decisions: how often the gate was reached, by which reason, and how it was answered. |
 
 #### `audit tail`
 
@@ -2006,6 +2097,32 @@ Examples:
 
 Output reports: writer enabled/disabled, last write result, file-lock state, rotation cap usage (`max_bytes` / `max_entries`). When `audit.enabled = false` (D1 opt-out), `tail` / `show` / `health` still exit 0 and print `Audit is disabled (audit.enabled = false in .dbcli). Use 'dbcli audit health' for details.` (E note).
 
+#### `audit write-gate`
+
+Answers the question ADR 0010 stakes the two-tier write gate on: is tier two stopping anything, or has everyone routed around it? The data was always written — `metadata.write_gate_tier` / `write_gate_outcome` / `write_gate_reason` on every evaluation — but reading it meant hand-writing jq over the JSONL.
+
+| Flag | Purpose | Default |
+|---|---|---|
+| `--format <fmt>` | `table` \| `json`. | `table` |
+| `--all` | Merge every connection into one measurement. | off |
+| `--for-agent` | Shortcut for `--format json`. | off |
+
+```bash
+dbcli audit write-gate
+dbcli audit write-gate --format json
+dbcli audit write-gate --all
+```
+
+Reports, over the current connection's retained history (rotated `.jsonl.1` included):
+
+- `total` — tier-two evaluations, with `range` giving the interval they span.
+- `outcomes` — `allowed` / `declined` / `refused`. The ratio between them is the line between "the gate is stopping things" and "everyone confirms through it".
+- `reasons` — which criterion sent the statement to tier two. Every known reason is listed, **including the ones that never fired**: a criterion that triggers zero times is the finding, not a row to omit.
+- `scanned` + `window` — the denominator. Two decisions in a week and two in a year are different findings.
+- `tierOne` — counted apart, with its own outcome breakdown, because tier one is recorded only when the operator declined.
+
+When tier two was never reached, the summary says so and names the conclusion ADR 0010 draws from it (the criterion is wrong, not the gate unnecessary) rather than printing an empty table. An entirely empty audit log is reported as a separate case: it supports no conclusion either way.
+
 #### Boundaries
 
 - Entries are append-only JSONL; rotation triggers at `~10 MB` or `~1000` entries (whichever first). Previous segment is preserved as `.jsonl.1`.
@@ -2495,11 +2612,38 @@ dbcli shell --sql    # SQL-only mode
 Inside the shell:
 - Type SQL statements ending with `;` to execute
 - Type dbcli commands without the `dbcli` prefix (e.g., `schema users`)
+- Prefix a subcommand with `\` to force it: `\delete users --where id=1`. Required for the
+  subcommands whose names are also SQL keywords — `insert`, `update`, `delete`, `explain` —
+  because a line starting with one of those is read as SQL,
+  which is what a shell is for. The prefix works on every subcommand, so it is one rule
+  rather than a list of exceptions (#88).
+- A subcommand runs in its own process with no stdin, so its ordinary y/n confirmation reads
+  EOF and cancels. `\insert` / `\update` / `\delete` therefore need `--force` from inside
+  the shell — or type the SQL at the prompt, where the confirmation can actually be
+  answered. Tier two is unaffected either way: it is refused in the child and points you
+  back to the prompt.
 - Use Tab for auto-completion (SQL keywords, table names, column names)
 - Type `.help` for meta commands (.quit, .clear, .format, .history, .timing)
-- Multi-line SQL: keeps accumulating until `;` is found
+- Multi-line SQL: keeps accumulating until `;` is found. Meta commands still work while a
+  statement is accumulating — `.quit` quits, `.clear` abandons the buffer — and Ctrl-C
+  cancels. A line that reads as SQL but names a subcommand and carries a double-dash option prints
+  a note saying which prefix would have reached the subcommand.
 - History persists across sessions (~/.dbcli_history)
 
+> **Tier two applies here (2.0.0)** — on SQL connections a typed `UPDATE` / `DELETE` with
+> no `WHERE`, a `DROP` or a `TRUNCATE` asks for the target table name before it runs.
+> Anything else typed cancels the statement and returns to the prompt; the session stays
+> open, and Ctrl-C withdraws the question itself — the next line is read as a statement,
+> not as the answer. Tier one (the y/N on ordinary writes) is deliberately not wired here — every line
+> is typed by a person. Piped input (`dbcli shell < script.sql`) has nobody to answer, so a
+> tier-two statement is refused and the remaining lines still run. Redis, MongoDB and
+> Elasticsearch shells are unaffected.
+>
+> dbcli **subcommands** typed in the shell (`query "..."`, `delete ...`) run as separate
+> processes with no stdin, so they cannot ask anything: a tier-two statement there is
+> refused with a message pointing you back to the `dbcli>` prompt, where the confirmation
+> can actually be typed.
+
 The REPL flavor depends on the active engine: SQL engines and MongoDB use the
 form above; **Redis** opens a single-line command REPL (see [Redis › Interactive
 shell](#interactive-shell)); **Elasticsearch** opens a Kibana Dev Tools-style