@optimystic/quereus-plugin-crypto 0.13.5 → 0.16.2

package/CHANGELOG.md

@@ -0,0 +1,91 @@
+# Changelog
+
+## 0.14.0 — BREAKING: `digest` API rework
+
+### What changed
+
+The exported `digest()` function signature changed from:
+
+```ts
+// OLD (≤ 0.13.x)
+digest(data: string | Uint8Array, algorithm?, inputEncoding?, outputEncoding?)
+```
+
+to:
+
+```ts
+// NEW (≥ 0.14.0)
+digest(fields: readonly DigestField[], algorithm?, encoding?)
+```
+
+Key differences:
+
+| | Old | New |
+|---|---|---|
+| First argument | A single scalar value (`string` \| `Uint8Array`) | An **array** of values |
+| `inputEncoding` | 3rd positional arg | **Removed** (the new API frames values by type, no string-decoding step) |
+| Output encoding | 4th positional arg | 2nd `encoding` arg (shifted left by one) |
+| Algorithm | 2nd positional arg | 2nd `algorithm` arg (unchanged position) |
+| Result | Bare hash of the decoded bytes | **Framed** injective digest — `digest(['hello'])` ≠ `sha256("hello")` |
+| Algorithm + encoding | Per-call | **Bound at plugin load time** for the SQL function |
+
+### Migration: JS/TypeScript callers
+
+```ts
+// OLD
+const hashBytes     = digest(payload, 'sha256', 'utf8', 'bytes')    as Uint8Array;
+const payloadDigest = digest(payload, 'sha256', 'utf8', 'base64url') as string;
+
+// NEW — wrap the value in an array, drop inputEncoding
+const hashBytes     = digest([payload], 'sha256', 'bytes')    as Uint8Array;
+const payloadDigest = digest([payload], 'sha256', 'base64url') as string;
+```
+
+> **The result value changes.** The new digest is *framed* (version byte + type tag +
+> length-prefixed payload per field), so `digest(['hello'])` is **not** the same bytes
+> as `sha256(utf8("hello"))`. If you need to match an externally-computed bare hash,
+> see the open question below.
+
+### Migration: SQL callers
+
+The SQL `digest(field1, field2, ...)` function is **variadic over data fields** — every
+argument is a field to hash, not a config option — so the signature is unchanged from the
+SQL perspective.
+
+However, if you were passing extra positional arguments to mimic `algo`/`inputEncoding`/`outputEncoding`
+(e.g. `digest(data, 'sha256', 'utf8', 'bytes')`), those are now treated as **additional
+data fields** and hashed into the result silently rather than interpreted as config. There
+is **no error** on the SQL path for this; it just hashes more fields. Check any SQL call
+sites that pass more than pure data arguments.
+
+Algorithm and encoding are now set via the plugin config at load time (see the
+[Digest configuration](README.md#digest-configuration) section of the README).
+
+### Why no compatibility shim?
+
+The old and new calling conventions cannot be cleanly disambiguated: the new first argument
+is always an array; the old's was always a scalar. Adding a scalar → old-API detection
+shim would silently re-enable the broken `inputEncoding` positional footgun and make the
+result value unpredictable. The clean break stays; instead, old-style JS calls now throw
+a clear, actionable error message naming this migration note.
+
+### Error message for old-style calls
+
+If you pass a non-array as the first argument to `digest()` in JS/TypeScript, you will now
+see:
+
+```
+digest(fields, algorithm?, encoding?): 'fields' must be an array of values.
+The digest API changed in v0.14: it is now variadic/injective over fields,
+the per-call inputEncoding was removed, and algorithm + output encoding are bound at plugin load time.
+Migrate digest(value, algo, inputEncoding, outputEncoding) → digest([value], algo, outputEncoding) —
+note the result is now a *framed* digest, not a bare hash of the bytes.
+```
+
+### Open question: bare hash helper
+
+The new `digest` has no function that returns a bare (un-framed) hash of a single value's
+bytes in a chosen encoding — what the old `digest(x, algo, inEnc, outEnc)` did. If you
+need bare-hash semantics (e.g. to match a hash stored before v0.14, or computed by an
+external system), there is currently no drop-in. File a separate issue/ticket if a
+`hash(data, algorithm, inputEncoding, outputEncoding)` helper is needed.

package/README.md

@@ -5,6 +5,8 @@ Quereus plugin providing cryptographic functions for SQL queries with base64url
 ## Features
 
 - **Hash Functions**: SHA-256, SHA-512, BLAKE3 hashing with base64url output
+- **Content Identifiers (CIDv1)**: Self-describing, interoperable IPFS/IPLD content addresses
+- **Selective Disclosure**: Salted-leaf set commitment — commit to a whole attribute set with one value, later reveal only a chosen subset with proof of authenticity
 - **Hash Modulo**: Fixed-size hash values (16-bit, 32-bit, etc.) for sharding and partitioning
 - **Random Bytes**: Generate cryptographically secure random bytes (default: 256 bits)
 - **Signature Functions**: secp256k1, P-256, Ed25519 signing with base64url encoding
@@ -28,6 +30,13 @@ import { loadPlugin } from '@quereus/quereus/util/plugin-loader.js';
 
 const db = new Database();
 await loadPlugin('npm:@optimystic/quereus-plugin-crypto', db);
+
+// Or configure the digest algorithm / output encoding once, at load time
+// (see "Digest configuration" below):
+await loadPlugin('npm:@optimystic/quereus-plugin-crypto', db, {
+  algorithm: 'sha256',     // 'sha256' (default) | 'sha512' | 'blake3'
+  encoding: 'base64url',   // 'base64url' (default) | 'base64' | 'hex'
+});
 ```
 
 ### Direct Import (for JavaScript/TypeScript code)
@@ -45,8 +54,8 @@ import {
   getPublicKey
 } from '@optimystic/quereus-plugin-crypto';
 
-// Hash data (base64url by default)
-const hash = digest('hello world', 'sha256', 'utf8', 'base64url');
+// Hash an ordered tuple of fields (injective — distinct tuples never collide)
+const hash = digest(['hello world', 42, null], 'sha256', 'base64url');
 
 // Get a 16-bit hash for sharding
 const shard = hashMod('user@example.com', 16, 'sha256', 'utf8');
@@ -68,24 +77,164 @@ const isValid = verify(hash, signature, publicKey, 'secp256k1', 'base64url', 'ba
 
 All SQL functions use **base64url encoding by default** for inputs and outputs. This is URL-safe and SQL-friendly.
 
-### digest(data, algorithm?, inputEncoding?, outputEncoding?)
+### digest(field1, field2, ..., fieldN)
+
+Hash an **ordered tuple of fields** into a single digest. `digest` is variadic over
+*data* — every argument is a field, not a configuration option. The algorithm and
+output encoding are chosen once, at load time (see [Digest configuration](#digest-configuration)),
+so they never have to be passed per call.
+
+```sql
+-- Hash a single value
+SELECT digest(Id) as hash;
+
+-- Hash a tuple of columns — distinct tuples never collide,
+-- NULL is distinguished from '', and 123 from '123'
+SELECT digest(Tid, Name, ImageRef, NumberRequiredTSAs) as commitment;
+```
+
+**Why variadic + injective?** Hashing several fields by joining them
+(`a || '|' || b`) or by `String()`-concatenation is not injective: `('a|b','c')`
+collides with `('a','b|c')`, `NULL` collides with `''`, and `123` collides with
+`'123'`. `digest` instead applies a canonical, length-prefixed, type-tagged framing
+to each field, so distinct tuples always produce distinct digests. This matters when
+the digest is signed or persisted as a commitment.
+
+The framing is also why `digest(x)` is **not** a bare `sha256(x)` — it is a framed
+digest of a one-element tuple. For sharding a single value, use `hash_mod`.
+
+### cid(data, codec?, hash?, base?) / cid_v1(digest, hash, codec?, base?) / cid_decode(cid)
 
-Hash data using SHA-256 (default), SHA-512, or BLAKE3.
+`digest` returns a **bare** hash — raw digest bytes in some text encoding, with no
+record of which base, content type, or hash algorithm produced it. `cid` instead
+produces a **self-describing CIDv1**, the same interoperable content address an
+IPFS/IPLD store computes for the same bytes:
+
+```
+CIDv1     = multibase( version ‖ multicodec(content-type) ‖ multihash )
+multihash = hashFnCode ‖ digestLength ‖ digestBytes
+```
+
+Because the multibase, content codec, and hash code all travel *inside* the value,
+a consumer can validate it without out-of-band knowledge, and an algorithm
+migration (e.g. sha2-256 → another hash) is unambiguous rather than a silent
+reinterpretation. The framing comes entirely from the audited
+[`multiformats`](https://github.com/multiformats/js-multiformats) library.
 
 ```sql
--- Hash base64url data (default)
-SELECT digest('aGVsbG8gd29ybGQ') as hash;
+-- Hash a blob and frame it as the canonical raw/sha2-256/base32 CID
+-- (== the CID IPFS shows for the same bytes)
+SELECT cid(SomeBlob) AS Cid;
 
--- Hash UTF-8 text
-SELECT digest('hello world', 'sha256', 'utf8', 'base64url') as hash;
+-- Pick content codec / hash / base
+SELECT cid(SomeBlob, 'dag-cbor', 'sha2-512', 'base58btc') AS Cid;
 
--- Hash with SHA-512
-SELECT digest('data', 'sha512', 'utf8', 'base64url') as sha512_hash;
+-- A self-describing content address over a field tuple: digest() canonically
+-- frames + hashes the fields; cid_v1() wraps that exact digest as a CIDv1
+-- (no double-hash). The asserted hash must match digest()'s configured algorithm.
+SELECT cid_v1(digest(ColA, ColB, ColC), 'sha2-256') AS Cid;
 
--- Hash with BLAKE3, output as hex
-SELECT digest('data', 'blake3', 'utf8', 'hex') as blake3_hex;
+-- Validate / inspect a stored CID (returns JSON: { version, codec, hashCode, digest })
+SELECT cid_decode(Cid) ->> 'codec' AS codec FROM T;
 ```
 
+- **`cid(data, codec?, hash?, base?)`** — hash `data` then frame. `data` is a BLOB,
+  or a base64url-encoded TEXT digest/blob (the plugin's canonical text encoding).
+  Defaults: `codec='raw'`, `hash='sha2-256'`, `base='base32'`.
+- **`cid_v1(digest, hash, codec?, base?)`** — wrap an **already-computed** digest
+  without re-hashing. `hash` is required and asserts which algorithm produced the
+  digest; the digest length is checked against it (sha2-256/blake3 = 32 bytes,
+  sha2-512 = 64) and a mismatch is rejected.
+- **`cid_decode(cid) → JSON`** — parse a CID back to `{ version, codec, hashCode,
+  digest }` (digest as base64url). Throws cleanly on malformed input.
+
+Selectable values: `codec` ∈ `raw`, `dag-cbor`; `hash` ∈ `sha2-256`, `sha2-512`,
+`blake3`; `base` ∈ `base32` (default), `base58btc`, `base64url`, `base16`.
+
+**Why `base32` by default?** A CID's whole purpose is to match what an external
+content-addressed store computes, and IPFS renders CIDv1 canonically in base32 (the
+`b…` prefix). This is deliberately different from the plugin's `digest`/`random_bytes`
+default of base64url: base64url is compact for *internal* values that live in memory,
+on the wire, or in JSON, whereas base32 is case-insensitive and DNS/URL/filename-safe
+where interoperable addresses are copied and read. Two audiences, two defaults — pass
+`'base64url'` explicitly if you want the compact form.
+
+### set_commit(leaves_json) / set_verify(root, disclosed_json, hidden_json)
+
+**Per-attribute selective disclosure.** An authority commits to a registrant's whole
+attribute set as a single root (which it signs / persists), then later reveals only a
+chosen *subset* to a recipient — with a proof the revealed values are genuinely the
+committed ones — **without** leaking the withheld attribute values. A flat
+`digest(whole set)` can't do this (verifying one field needs the whole pre-image, so
+it's all-or-nothing); `set_commit` supports *partial opening*.
+
+Each attribute becomes a salted leaf, and the root is the digest of all leaf digests in
+canonical (sort-by-leaf-digest-bytes) order:
+
+```
+leafDigest = digest([SD_LEAF_DOMAIN_V1, name, value, salt])   -- raw digest bytes
+root       = digest([SD_SET_DOMAIN_V1, sortedLeaf_0, sortedLeaf_1, ...])
+```
+
+This is the same flat salted-hash shape the IETF SD-JWT standard
+(`draft-ietf-oauth-selective-disclosure-jwt`) settled on rather than a Merkle tree —
+cited as conceptual precedent only; the framing here is Optimystic's own `digest`
+`encodeFields` (not SD-JWT wire-compatible). The `name` is hashed into the leaf so a
+`(value, salt)` proof can't be replayed under another attribute; the `salt` is per-leaf
+and **mandatory** (low-entropy values like a DOB or a boolean are brute-forceable from a
+bare hash, and independent salts defeat cross-registrant correlation).
+
+```sql
+-- Commit to a JSON array of [name, value, salt] leaves -> single root.
+-- Pair with cid() for the self-describing persisted/signed column representation:
+SELECT cid(set_commit(SelectiveDetails)) AS SelectiveCid;
+
+-- A schema CHECK makes a forged root impossible to store: the root is recomputed from
+-- the stored attribute triples, so SelectiveCid must equal the genuine commitment.
+CREATE TABLE Registrant (
+  ...,
+  SelectiveDetails TEXT,   -- JSON array of [name, value, salt] triples
+  SelectiveCid     TEXT,
+  CHECK (SelectiveCid = cid(set_commit(SelectiveDetails)))
+);
+
+-- A recipient verifies a disclosure: the disclosed [name, value, salt] triples plus
+-- the opaque leaf digests of the withheld leaves reconstruct the entire root.
+SELECT set_verify(root, disclosed_json, hidden_json) AS ok;
+```
+
+- **`set_commit(leaves_json) → TEXT`** — `leaves_json` is a JSON array; each element is a
+  leaf `[name, value, salt]` *or* `{ "name", "value", "salt" }`. Values follow `digest`'s
+  rules as parsed from JSON (INTEGER vs REAL by JS value, TEXT, BOOL, null, nested
+  object/array). Each leaf must carry all three fields — a missing `value` (object form) or
+  a `[name, value]` array (under three elements) throws; pass `value: null` for a null-valued
+  attribute. The salt is base64url TEXT (e.g. from `random_bytes`). **THROWS** on a duplicate
+  name, a missing value, a missing/empty salt, or unparseable/non-array JSON (invalid states
+  made impossible). `replicable` — the root is signed and persisted, same bar as `digest`.
+- **`set_verify(root, disclosed_json, hidden_json) → BOOLEAN`** — `disclosed_json` is the
+  opened triples (same leaf shape), `hidden_json` a JSON array of the withheld leaves'
+  opaque base64url digests. Reconstructs the **entire** root and compares — so the holder
+  cannot add, drop, or swap a leaf. Returns `false` on any mismatch or malformed input
+  (forgiving, like `verify`).
+
+> **Disclosure generation is JS-only** (`setDisclose`, below) — there's no SQL
+> `set_disclose`, because building a disclosure requires the full attribute set including
+> the secret salts, which lives engine-side, not in a query.
+
+**BLOB-valued attributes via SQL:** JSON has no blob type, so a blob attribute passed
+through `set_commit` is committed as its base64url TEXT. Callers needing a *true* BLOB
+value (committed as a BLOB field, `TAG_BLOB`) must use the JS `setCommit` with a
+`Uint8Array` value.
+
+**Privacy note:** a fixed commitment disclosed to two audiences exposes the same hidden
+digests and field count to both, so they can correlate that it's the same record.
+Re-randomizing per disclosure would require fresh salts → a new root → a new signature
+(out of scope here).
+
+**Framing coupling:** leaf and root reuse `digest`'s `encodeFields`, so a future digest
+framing-version bump changes `set_commit` output too — intentional (one canonical
+framing), but it means the pinned set-commitment vectors move with the digest vectors.
+
 ### hash_mod(data, bits, algorithm?, inputEncoding?)
 
 Hash data and return modulo 2^bits for fixed-size hash values.
@@ -121,6 +270,16 @@ SELECT random_bytes(64) as random_id;
 
 ### sign(data, privateKey, curve?, inputEncoding?, keyEncoding?, outputEncoding?)
 
+**Security note (replication):** passing a private key to `sign()` as a literal or
+bound parameter is **safe** with respect to Optimystic replication. The Quereus engine
+rebuilds the replicated statement from evaluated column values — not from source SQL —
+so the key argument is evaluated away and never reaches the record. Peers re-execute
+`INSERT ... VALUES (<signature>)`, never the original `sign(..., key)` call. The one
+thing to avoid is storing a raw private key **as a column value** in an
+optimystic-backed table, since any persisted column value is replicated. Sign or derive
+and store only the public result (signature, public key, commitment). See
+`docs/transactions.md` § "Secrets and the replicated statement record" for full detail.
+
 Sign data using secp256k1 (default), P-256, or Ed25519.
 
 ```sql
@@ -152,6 +311,40 @@ SELECT verify('hello', 'c2lnbmF0dXJl', 'cHVibGljS2V5', 'secp256k1', 'utf8') as i
 SELECT verify('data', 'c2lnbmF0dXJl', 'cHVibGljS2V5', 'p256', 'utf8') as is_valid;
 ```
 
+## Migration (v0.14 breaking change)
+
+The `digest()` JS/TypeScript API changed significantly in v0.14. See [CHANGELOG.md](CHANGELOG.md) for the
+full migration guide, including old → new call-site examples, what changed in the SQL function, and
+the open bare-hash-helper question.
+
+## Digest configuration
+
+Because `digest` is variadic over data, its **algorithm** and **output encoding** are
+not call arguments — they are bound once when the plugin is loaded, via the plugin
+config object:
+
+```ts
+import { registerPlugin } from '@quereus/quereus';
+import cryptoPlugin from '@optimystic/quereus-plugin-crypto/plugin';
+
+await registerPlugin(db, cryptoPlugin, {
+  algorithm: 'sha256',   // 'sha256' (default) | 'sha512' | 'blake3'
+  encoding: 'base64url', // 'base64url' (default) | 'base64' | 'hex'
+});
+```
+
+An unknown `algorithm` or a non-text `encoding` throws at registration (fail fast),
+and the algorithm/encoding are resolved once so the per-call path does no branching.
+
+**Why load-time and not per-connection?** The SQL `digest` is registered as
+`replicable` — its output must be bit-identical across peers, platforms, and app
+versions, because these digests are signed and persisted as commitments. That holds
+only if the configuration is fixed for every peer. *Mutable* per-connection
+configuration (e.g. a runtime `SET`/PRAGMA) would let two peers disagree and silently
+break signature validation, so it is intentionally not offered for this function. If a
+single database genuinely needs two digest configurations, register the plugin twice
+(or expose named variants) rather than flipping mutable session state.
+
 ## Supported Algorithms
 
 ### Hash Algorithms
@@ -216,7 +409,7 @@ const privateKey = generatePrivateKey('secp256k1', 'base64url');
 const publicKey = getPublicKey(privateKey);
 
 const message = 'Hello, World!';
-const hash = digest(message, 'sha256', 'utf8', 'base64url');
+const hash = digest([message], 'sha256', 'base64url') as string;
 const signature = sign(hash, privateKey);
 const isValid = verify(hash, signature, publicKey);
 
@@ -229,9 +422,59 @@ console.log('Random nonce:', nonce);
 
 ## JavaScript API Reference
 
-### digest(data, algorithm?, inputEncoding?, outputEncoding?)
-Hash data using SHA-256, SHA-512, or BLAKE3.
-- **Returns**: Hash as string (or Uint8Array if encoding is 'bytes')
+### digest(fields, algorithm?, encoding?)
+Injective digest over an ordered tuple of fields. `fields` is an array of values
+(any SQL value type). `algorithm` defaults to `'sha256'`, `encoding` to `'base64url'`.
+- **Returns**: Hash as string (or Uint8Array if encoding is `'bytes'`)
+- **Related**: `encodeFields(fields)` returns the canonical pre-hash byte framing;
+  `digestFields(fields, hasher, encode)` / `resolveHasher` / `resolveOutputEncoder`
+  are the building blocks the SQL function composes (resolve once, no per-call branching).
+
+### cid(data, codec?, hash?, base?)
+Hash `data` (a `Uint8Array`) and frame it as a self-describing CIDv1 string. Defaults:
+`codec='raw'`, `hash='sha2-256'`, `base='base32'`. Byte-identical to the CID an IPFS/IPLD
+store computes for the same bytes.
+- **Returns**: CIDv1 string
+
+### cidV1(digest, hash, codec?, base?)
+Frame an **already-computed** `digest` (a `Uint8Array`) as a CIDv1 without re-hashing.
+`hash` asserts which algorithm produced the digest; the digest length is validated
+against it. Use to turn a `digest(...)` result into a CID: `cidV1(digest(fields, 'sha256', 'bytes'), 'sha2-256')`.
+- **Returns**: CIDv1 string
+
+### cidDecode(cid)
+Parse a CID string into `{ version, codec, hashCode, digest }` for validation/migration.
+Recognized codec/hash codes are returned as names, otherwise as numbers; `digest` is a
+`Uint8Array`. Throws on malformed input.
+- **Returns**: `{ version: number, codec: Multicodec | number, hashCode: MultihashCode | number, digest: Uint8Array }`
+
+### setCommit(leaves, hasher?, encode?) / setDisclose(leaves, revealNames, hasher?) / setVerify(root, disclosure, hasher?, encode?)
+Salted-leaf set commitment for selective disclosure. `leaves` is an array of
+`{ name, value, salt }` (`salt` a base64url string or `Uint8Array`).
+- **`setCommit`** → the root (`string`, or `Uint8Array` with a bytes encoder). Throws on a
+  duplicate name or a missing/empty salt; the empty set is well-defined, not an error.
+- **`setDisclose`** → `{ disclosed, hidden }`: the revealed `{ name, value, salt }` triples
+  plus the opaque base64url leaf digests of the withheld leaves (withheld values/salts never
+  appear). This is the engine-side generator with no SQL equivalent.
+- **`setVerify`** → `boolean`: reconstructs the entire root from `disclosure` and compares to
+  `root`. `false` on mismatch or malformed input. `encode` is how the signed root is rendered
+  (default base64url); a `Uint8Array` root is compared by raw bytes.
+- **`leafDigest(leaf, hasher)`** → raw leaf digest bytes (the low-level building block).
+
+```typescript
+import { setCommit, setDisclose, setVerify, randomBytes } from '@optimystic/quereus-plugin-crypto';
+
+const leaves = [
+  { name: 'name',    value: 'Alice',     salt: randomBytes(256) as string },
+  { name: 'over18',  value: true,        salt: randomBytes(256) as string },
+  { name: 'zip',     value: '90210',     salt: randomBytes(256) as string },
+];
+const root = setCommit(leaves);                 // sign / persist this (often as cid(root))
+
+// Recipient gets only `over18`, with proof it belongs to the committed set:
+const disclosure = setDisclose(leaves, ['over18']);
+const ok = setVerify(root, disclosure);          // true — withheld values never left the engine
+```
 
 ### hashMod(data, bits, algorithm?, inputEncoding?)
 Hash data and return modulo 2^bits for fixed-size hash values.