@carllee1983/dbcli 2.0.0 → 3.0.0

package/assets/reference.md

@@ -346,12 +346,58 @@ says whether this particular statement may run right now.
 | Tier | Statements | Interactive terminal | Non-interactive (or `--format json`) |
 | :--- | :--- | :--- | :--- |
 | One | `INSERT`, `UPDATE` / `DELETE` **with** a `WHERE` or `LIMIT`, `CREATE`, `ALTER` | Summary + `y/N`; `--yes` skips it | Runs, exactly as before |
-| Two | `UPDATE` / `DELETE` with **no** `WHERE`, `DROP`, `TRUNCATE`, unparseable statements, several statements in one string | Type the target table name; **no flag skips it** | **Refused**: exit `1`, nothing sent to the database |
+| Two | `UPDATE` / `DELETE` with **no** `WHERE`, `DROP`, `TRUNCATE`, unparseable statements, several statements in one string, a write nested inside another statement, a destructive `MERGE` | Type the target table name; **no flag skips it** | **Refused**: exit `1`, nothing sent to the database |
 
-A refusal message names a machine-readable reason — `reason=no_where`,
-`reason=ddl_destruction`, `reason=unparseable`, `reason=multiple_statements` — so a caller can tell it apart from a
+A refusal message names a machine-readable reason — `reason=no_where`, `reason=multi_table`,
+`reason=ddl_destruction`, `reason=unparseable`, `reason=multiple_statements`, `reason=nested_write` — so a caller can tell it apart from a
 connection failure or a permission denial.
 
+**A write that joins another table is tier two, whatever its `WHERE` says.** Whether such a
+write is limited to particular rows is not a property of the statement:
+`DELETE p FROM p JOIN o ON p.id = o.ref WHERE o.x > 0` deleted 2 of 5 rows against one
+dataset and all 2000 against another, and a join's `ON` necessarily names the target, so it
+proves nothing. Five rounds of adversarial measurement went into trying to read this off the
+syntax before the rule became "a second table means tier two" (#80, #93). That refuses the
+standard `UPDATE … FROM`, `UPDATE … JOIN … SET` and multi-table `DELETE` idioms for
+unattended callers: rewrite the other table into a subquery in the `WHERE`, or run them
+where someone can confirm.
+
+**A statement is not the keyword it starts with.** PostgreSQL's data-modifying CTEs put an
+arbitrary write in front of the statement the first keyword names, and a `MERGE` carries its
+writes in a `WHEN … THEN` list — measured against a 2000-row table,
+`WITH moved AS (DELETE FROM p RETURNING *) INSERT INTO archive …` and
+`MERGE INTO p USING (SELECT 1 AS x) s ON true WHEN MATCHED THEN DELETE` each emptied it
+while reading as an ordinary `INSERT` (#94, #95). So a write nested inside another statement
+is tier two, `reason=nested_write`, and a `MERGE` is classified by its actions: a
+`THEN DELETE` or `THEN UPDATE` is tier two, `reason=multi_table`, while an insert-only or
+`DO NOTHING` one stays tier one. `ON CONFLICT … DO UPDATE`, `ON DUPLICATE KEY UPDATE` and
+`INSERT … SELECT` are untouched by this — they are not nested writes, whatever they read
+like. The cost is a CTE that deletes one row by primary key, and the ordinary `MERGE` upsert,
+both refused for unattended callers.
+
+**For a single-table write, the `WHERE` has to be about that table.** `UPDATE p SET c = 1
+WHERE id = 1` is an ordinary write; `UPDATE p SET c = (SELECT max(x) FROM o WHERE o.id = 1)`
+is not, because its only `WHERE` restricts the subquery and the write touches every row. A
+*correlated* reference back to the target does count, so
+`DELETE FROM t WHERE EXISTS (SELECT 1 FROM o WHERE o.tid = t.id)` is tier one while
+`DELETE FROM t WHERE EXISTS (SELECT 1 FROM o WHERE o.id = 1)` deletes every row and is tier
+two. A qualifier the subquery binds itself — including the target's own name, as in
+`DELETE FROM sessions WHERE EXISTS (SELECT 1 FROM sessions WHERE …)` — is about the
+subquery's rows, not the write.
+
+This is a lower bound, not a proof: `WHERE id IS NOT NULL` names the target and still touches
+every row, and no static check settles that. What it rules out is the class where nothing in
+the condition is about the table being written.
+
+**When the parser cannot read the statement**, there is no tree to judge and the same rule is
+applied to the text: tier one only when the statement reads as a write to one named table
+with a `WHERE` or `LIMIT` and contains no `SELECT`, `TABLE`, `VALUES`, `JOIN`, `USING`, a
+statement-level `WITH`, or an `UPDATE … FROM`. A keyword inside parentheses is not at
+statement level, so `SUBSTRING(x FROM 2)` and `AGAINST ('a' WITH QUERY EXPANSION)` do not
+trip it. This is written as an allowlist because the denylist that preceded it was defeated
+once per review round — `USING`, then `JOIN`, then a CTE, then a subquery, then `TABLE` as a
+subquery.
+
 **Escape routes.** For a statement that accepts a `WHERE`, put the intent in the SQL:
 
 ```bash
@@ -359,6 +405,7 @@ dbcli query "UPDATE users SET banned = 1"                    # refused, reason=n
 dbcli query "UPDATE users SET banned = 1 WHERE 1=1"          # runs — intent is explicit
 dbcli query "DELETE FROM sessions LIMIT 1000"                # runs — damage is bounded
 dbcli query "UPDATE users SET banned = 1 WHERE id = 3" --yes # tier one, question skipped
+dbcli query "UPDATE p SET x = 1 FROM o WHERE o.id = 1"       # refused — the WHERE is about o, not p
 ```
 
 This is deliberately not a flag. `WHERE 1=1` appended to a statement that already has a
@@ -369,7 +416,8 @@ whether they are possible at all, and the typed confirmation must come from a pe
 
 Every tier-two evaluation is written to the audit log with
 `metadata.write_gate_outcome` (`allowed` / `declined` / `refused`) and
-`metadata.write_gate_reason`.
+`metadata.write_gate_reason`. `dbcli audit write-gate` summarizes them — that is
+the measurement ADR 0010 stakes this gate on.
 
 > **Elasticsearch tiers by scope.** A read (`GET` / `HEAD`, plus `POST _search` and
 > `POST _count`) is query-only. Indexing or updating a document is read-write.
@@ -1525,7 +1573,7 @@ Opt-in flag trio that persists a **VerificationArtifact JSON** (schema v1) under
 
 **Planned vs Result evidence.** `dbcli skill tasks plan safe-backfill-verify --format json` returns a plan containing a `verification` block with `status: "planned"`. That block is the **planned** evidence definition — it describes which check will run. Running `assert --write-verification-artifact` on the actual data produces **result** evidence (`status: "verified"` or `status: "not_verified"`). The two records are distinct; `"planned"` does **not** indicate that verification has run or passed.
 
-> **Casting note:** Postgres returns `count(*)` and `sum()` as bigint (a string in the result set). `value ==` uses strict equality, so `"0" == 0` is false. Cast to `::int` (`count(*)::int`) to ensure numeric comparison works correctly.
+> **Numeric note:** Postgres returns `count(*)` and `sum()` as bigint, which arrives as a string. A numeric expectation is compared numerically, so `value == 0` matches it without a cast. A quoted expectation stays a text comparison: `value == "0"` matches the text, not the number. The `::int` casts in the examples below are harmless and no longer required.
 
 ```bash
 dbcli assert "SELECT count(*)::int FROM orders WHERE status IS NULL" \
@@ -1988,6 +2036,7 @@ Audit entries are metadata-only by design — never raw SQL bodies, `--param` va
 | `audit show` | `readonly` | Print a single full entry by id prefix or `--recovery-ref`. |
 | `audit clear` | `local-write` | Delete `<conn>.jsonl` + rotated `.jsonl.1` from local disk. Requires `--yes` or interactive confirm. |
 | `audit health` | `readonly` | Render `AuditLogger.getHealth()` snapshot (writer state, lock state, rotation usage). |
+| `audit write-gate` | `readonly` | Summarize tier-two write-gate decisions: how often the gate was reached, by which reason, and how it was answered. |
 
 #### `audit tail`
 
@@ -2048,6 +2097,32 @@ Examples:
 
 Output reports: writer enabled/disabled, last write result, file-lock state, rotation cap usage (`max_bytes` / `max_entries`). When `audit.enabled = false` (D1 opt-out), `tail` / `show` / `health` still exit 0 and print `Audit is disabled (audit.enabled = false in .dbcli). Use 'dbcli audit health' for details.` (E note).
 
+#### `audit write-gate`
+
+Answers the question ADR 0010 stakes the two-tier write gate on: is tier two stopping anything, or has everyone routed around it? The data was always written — `metadata.write_gate_tier` / `write_gate_outcome` / `write_gate_reason` on every evaluation — but reading it meant hand-writing jq over the JSONL.
+
+| Flag | Purpose | Default |
+|---|---|---|
+| `--format <fmt>` | `table` \| `json`. | `table` |
+| `--all` | Merge every connection into one measurement. | off |
+| `--for-agent` | Shortcut for `--format json`. | off |
+
+```bash
+dbcli audit write-gate
+dbcli audit write-gate --format json
+dbcli audit write-gate --all
+```
+
+Reports, over the current connection's retained history (rotated `.jsonl.1` included):
+
+- `total` — tier-two evaluations, with `range` giving the interval they span.
+- `outcomes` — `allowed` / `declined` / `refused`. The ratio between them is the line between "the gate is stopping things" and "everyone confirms through it".
+- `reasons` — which criterion sent the statement to tier two. Every known reason is listed, **including the ones that never fired**: a criterion that triggers zero times is the finding, not a row to omit.
+- `scanned` + `window` — the denominator. Two decisions in a week and two in a year are different findings.
+- `tierOne` — counted apart, with its own outcome breakdown, because tier one is recorded only when the operator declined.
+
+When tier two was never reached, the summary says so and names the conclusion ADR 0010 draws from it (the criterion is wrong, not the gate unnecessary) rather than printing an empty table. An entirely empty audit log is reported as a separate case: it supports no conclusion either way.
+
 #### Boundaries
 
 - Entries are append-only JSONL; rotation triggers at `~10 MB` or `~1000` entries (whichever first). Previous segment is preserved as `.jsonl.1`.
@@ -2537,11 +2612,38 @@ dbcli shell --sql    # SQL-only mode
 Inside the shell:
 - Type SQL statements ending with `;` to execute
 - Type dbcli commands without the `dbcli` prefix (e.g., `schema users`)
+- Prefix a subcommand with `\` to force it: `\delete users --where id=1`. Required for the
+  subcommands whose names are also SQL keywords — `insert`, `update`, `delete`, `explain` —
+  because a line starting with one of those is read as SQL,
+  which is what a shell is for. The prefix works on every subcommand, so it is one rule
+  rather than a list of exceptions (#88).
+- A subcommand runs in its own process with no stdin, so its ordinary y/n confirmation reads
+  EOF and cancels. `\insert` / `\update` / `\delete` therefore need `--force` from inside
+  the shell — or type the SQL at the prompt, where the confirmation can actually be
+  answered. Tier two is unaffected either way: it is refused in the child and points you
+  back to the prompt.
 - Use Tab for auto-completion (SQL keywords, table names, column names)
 - Type `.help` for meta commands (.quit, .clear, .format, .history, .timing)
-- Multi-line SQL: keeps accumulating until `;` is found
+- Multi-line SQL: keeps accumulating until `;` is found. Meta commands still work while a
+  statement is accumulating — `.quit` quits, `.clear` abandons the buffer — and Ctrl-C
+  cancels. A line that reads as SQL but names a subcommand and carries a double-dash option prints
+  a note saying which prefix would have reached the subcommand.
 - History persists across sessions (~/.dbcli_history)
 
+> **Tier two applies here (2.0.0)** — on SQL connections a typed `UPDATE` / `DELETE` with
+> no `WHERE`, a `DROP` or a `TRUNCATE` asks for the target table name before it runs.
+> Anything else typed cancels the statement and returns to the prompt; the session stays
+> open, and Ctrl-C withdraws the question itself — the next line is read as a statement,
+> not as the answer. Tier one (the y/N on ordinary writes) is deliberately not wired here — every line
+> is typed by a person. Piped input (`dbcli shell < script.sql`) has nobody to answer, so a
+> tier-two statement is refused and the remaining lines still run. Redis, MongoDB and
+> Elasticsearch shells are unaffected.
+>
+> dbcli **subcommands** typed in the shell (`query "..."`, `delete ...`) run as separate
+> processes with no stdin, so they cannot ask anything: a tier-two statement there is
+> refused with a message pointing you back to the `dbcli>` prompt, where the confirmation
+> can actually be typed.
+
 The REPL flavor depends on the active engine: SQL engines and MongoDB use the
 form above; **Redis** opens a single-line command REPL (see [Redis › Interactive
 shell](#interactive-shell)); **Elasticsearch** opens a Kibana Dev Tools-style
@@ -2619,7 +2721,7 @@ dbcli evidence compose --claims ./claims.json --verification ver_abcd --audit 1a
 
 | Flag | Purpose | Default |
 |---|---|---|
-| `--claims <file>` | Required JSON file with exactly `subject` and `claims`; every claim has an `id` and plain-language `text`. SQL-shaped text, credentials, error content, and blacklisted identifiers are rejected. | — |
+| `--claims <file>` | Required JSON file with exactly `subject` and `claims`; every claim has an `id` and plain-language `text`. SQL-shaped text, credentials, error content, and blacklisted identifiers are rejected. A blacklisted identifier has to appear as an identifier — a term like `id` matches `orders.id` but not the word "identifier" — and the refusal names the field, never the matched term. | — |
 | `--verification <selector...>` | One or more verification artifact ids, unique prefixes, filenames, or in-bounds paths. | none |
 | `--audit <selector...>` | One or more audit ids or unique prefixes (minimum four characters) from the active connection. | none |
 | `--receipt <path...>` | Explicit workspace-relative receipt path(s). Receipts contribute safe provenance only; they are not execution approval. | none |
@@ -2627,8 +2729,11 @@ dbcli evidence compose --claims ./claims.json --verification ver_abcd --audit 1a
 | `--format <json\|markdown>` | Compose receipt format printed to stdout. | `json` |
 
 At least one `--verification`, `--audit`, or `--receipt` reference is required. The resulting pack
-contains a canonical SHA-256 integrity digest and `coverage.completeForDeclaredEvidence:
-true` for its explicitly selected references.
+carries a canonical SHA-256 digest over its content, and its `id` is derived from that digest, so
+composing the same claims and references twice produces the same pack. `createdAt` records when the
+pack was written and is deliberately outside the digest — it is the one field a restamp can change
+without breaking validation. Whether a referenced source is still resolvable is reported by
+`evidence validate`, not stored in the pack.
 
 #### `evidence validate`