@voltro/cli 0.1.0

package/templates/agent-docs/database/misc.md

@@ -0,0 +1,270 @@
+# Database
+
+> Spin up an isolated branch of a tenant's data — a namespace snapshot/restore by default (dialect-agnostic, no vendor lock) or a Neon copy-on-write fast-path on Postgres — then bind to it and tear it down. The basis for per-PR preview environments.
+
+
+
+---
+
+<!-- source: en/database/branching.md -->
+## Data branching
+
+_Spin up an isolated branch of a tenant's data — a namespace snapshot/restore by default (dialect-agnostic, no vendor lock) or a Neon copy-on-write fast-path on Postgres — then bind to it and tear it down. The basis for per-PR preview environments._
+
+Data branching creates an **isolated copy of a tenant's data** you can read and
+write without touching the source — the basis for per-PR preview environments and
+safe "try a migration against real-shaped data" workflows. The default mechanism
+is a **namespace snapshot/restore** (works on every dialect, no vendor lock); on
+Postgres backed by Neon, a **copy-on-write branch** fast-path is used instead.
+
+## The plan/execute split
+
+Branching is a **plan** (what to do) + an **executor** (do it), mirroring the
+migration engine. The plan is pure + inspectable; the executor performs the I/O.
+
+```ts
+import { planBranch, resolveBranchMechanism } from '@voltro/database'
+
+// Pick the mechanism: 'neon-cow' when seeding a copy on a Neon URL,
+// 'namespace' (schema-per-branch) otherwise.
+const mechanism = resolveBranchMechanism({ seed: 'copy', dbUrl: connectionString })
+const steps = planBranch({
+  mechanism,
+  branchId: 'branch_acme_pr128',
+  tableNames,
+  seed: 'copy',
+  parentNamespace: 'public',   // namespace mechanism: the schema to snapshot from
+})
+// steps: e.g. [{ kind: 'neon-branch-create', … }] or per-table namespace snapshot steps
+```
+
+## Provision + tear down
+
+`provisionBranch` / `teardownBranch` run a plan through an injected
+`BranchExecutor` (the dialect/Neon adapter). Injected so the lifecycle is
+unit-testable with a recording executor — no live DB:
+
+```ts
+import { provisionBranch, teardownBranch } from '@voltro/database'
+
+const result = await provisionBranch(
+  { appSlug: 'acme', prNumber: 128, seed: 'copy', tableNames, dbUrl: connectionString },
+  executor,
+)
+// result: { branchId, mechanism, steps, … } — for 'neon-cow' also the
+// branch's own connection string to bind the preview store to.
+
+// …run the preview against result, then:
+await teardownBranch(result.branchId, result.mechanism, executor)
+```
+
+`admitBranch` gates whether a branch may be served (e.g. it exists + is ready)
+so a request never binds to a half-provisioned branch.
+
+## Mechanisms
+
+| mechanism | when | how |
+|---|---|---|
+| `namespace-snapshot` | any dialect (default) | snapshot the tenant's namespace → restore into a new branch namespace |
+| `neon-branch` | Postgres on Neon | a copy-on-write Neon branch (instant, storage-shared) |
+
+The mechanism is resolved from the connection, so the same `planBranch` call
+produces the right steps for the deployment's database — dialect-agnostic by
+default, with the Neon fast-path when it's available. No code path is locked to a
+vendor.
+
+> Branching copies **data**, not the schema definition — apply migrations to a
+> branch the same way you would to any store. Teardown is explicit; a branch
+> left bound holds its namespace/Neon branch until torn down.
+
+
+
+---
+
+<!-- source: en/database/sensitivity.md -->
+## Data classification (.sensitive / .safe)
+
+_Classify columns as .sensitive(class) or .safe() so the export masker can replace PII with realistic, referentially-consistent fakes at the source. Fail-closed — an unclassified column blocks a masking export._
+
+Two column modifiers that carry **data-sensitivity metadata** into the
+schema — nothing at runtime, nothing in the DDL. They exist for one job:
+letting `voltro data export --profile dev` replace personal data with
+realistic, referentially-consistent fakes at the **source**, so a copy of
+prod that lands in dev/stage never contains real user data.
+
+- **`.sensitive(class)`** — this column holds personal/sensitive data of a
+  known CLASS (`email`, `phone`, `secret`, …). The class picks a
+  format-preserving fake.
+- **`.safe()`** — this column was reviewed and holds NO PII; copy it verbatim.
+
+They pair: under a masking profile every exported column must be **one or the
+other** (or the primary key / a foreign key, which are implicitly safe), or the
+export **refuses**. That is the whole point — see [why fail-closed](#why-fail-closed).
+
+## `.sensitive(class)` — mark a column as PII
+
+```ts
+import { id, text, table } from '@voltro/database'
+
+export const users = table('users', {
+  id:    id(),
+  email: text().sensitive('email'),
+  name:  text().sensitive('fullName'),
+})
+```
+
+**What it does:** nothing at runtime and nothing in the emitted DDL. It is
+pure metadata — the export masker reads it (from the DECLARED schema) and picks
+a format-preserving fake for the column.
+
+**Why you use it:** lower environments must never contain real user data — for
+GDPR, for least privilege, and to shrink the blast radius of a leaked dev dump.
+Classifying the PII columns lets the framework substitute realistic,
+referentially-consistent fakes **automatically at the source**, so real values
+never reach the bundle, transit, or a developer's machine.
+
+### Known classes
+
+Each known class maps to a format-preserving fake (a fake email is a valid
+email, a fake phone looks like a phone). The class is also what a
+[masking policy](/docs/cli/data#masking-prod-dev-stage-safely) keys its
+per-class overrides on.
+
+| Class | Fake |
+|---|---|
+| `email` | valid-looking address (`ada.lovelace4823@example.com`) |
+| `fullName` | `First Last` |
+| `firstName` | a first name |
+| `lastName` | a last name |
+| `username` | a handle |
+| `phone` | a phone number |
+| `address` | a street address |
+| `company` | a company name |
+| `url` | a URL |
+| `ip` | an IPv4 address |
+| `creditCard` | a Luhn-valid 16-digit number |
+| `date` | date-shifted (see below) |
+| `secret` | nulled |
+| `freeText` | redacted to `[redacted]` |
+
+`date`, `secret`, and `freeText` are the non-`fake` defaults: `date` shifts by a
+seed-derived offset (relative intervals and ordering survive), `secret` is set
+to `null`, `freeText` becomes `[redacted]`. Every default is overridable per
+class or per column in the [masking policy](/docs/cli/data#masking-prod-dev-stage-safely).
+
+**Custom classes.** The known set is only what gives editor autocomplete — any
+string is accepted. Use a custom class (e.g. `.sensitive('iban')`) and give it a
+transform in the policy's `classes` (an unmapped custom class falls back to
+`redact`).
+
+```ts
+account: text().sensitive('iban'),   // give 'iban' an action in the policy's `classes`
+```
+
+## `.safe()` — mark a column reviewed-safe
+
+```ts
+import { id, integer, text, table } from '@voltro/database'
+
+export const posts = table('posts', {
+  id:     id(),
+  title:  text().safe(),                       // public — copy verbatim
+  status: text().oneOf(['draft', 'live']).safe(),
+  views:  integer().safe(),
+})
+```
+
+**What it does:** metadata only — it copies verbatim under a masking profile.
+
+**Why it exists:** masking is **fail-closed**. Under a masking profile a column
+that is neither `.sensitive()` nor `.safe()` **blocks the export** until it is
+consciously classified — so adding a column later can never silently leak PII to
+dev. `.safe()` is the explicit *"I looked, it's fine"* acknowledgement (a public
+title, a status enum, a counter).
+
+## Primary keys and foreign keys are implicitly safe
+
+You do **not** annotate every id. `id()` (a primary key) and `reference()` (a
+foreign key) columns are opaque identifiers — keeping them verbatim is exactly
+what makes joins survive the copy — so they are treated as **safe by default**
+under a masking profile. No `.safe()` needed.
+
+An explicit `.sensitive()` still overrides this for the rare case where the key
+itself is PII (a natural key like an email-as-id):
+
+```ts
+// A table keyed by a natural PII value — classify it explicitly.
+subscription: table('subscription', {
+  email: text().sensitive('email'),   // this IS the PK, and it IS PII → fake it
+  plan:  text().safe(),
+})
+```
+
+## Interaction with `.encrypted()`
+
+This is the subtle trap. [`.encrypted()`](/docs/database/columns) protects data
+**at rest** (the stored value is opaque ciphertext), but the runtime
+**decrypts on read** — so by the time the export streams the row, the column is
+back to **plaintext**. Encryption at rest gives you *nothing* at export time.
+
+The framework therefore treats an `.encrypted()` column as
+**`.sensitive('secret')` automatically** (so it is nulled under a masking
+profile) — unless you classify it otherwise:
+
+```ts
+ssn:   text().encrypted(),                 // ⇒ implicitly sensitive('secret') → nulled on export
+phone: text().encrypted().sensitive('phone'), // explicit class wins → fake phone instead of null
+review: text().encrypted().safe(),         // encrypted at rest, but reviewed-safe → copied verbatim
+```
+
+Precedence, exactly:
+
+1. An explicit **`.sensitive(class)`** wins — the column is faked by that class.
+2. Otherwise **`.encrypted()`** (without `.safe()`) implies `sensitive('secret')`
+   → the column is nulled.
+3. **`.safe()`** on an encrypted column suppresses the implied secret and copies
+   it verbatim — say this only when you have genuinely reviewed the plaintext.
+
+## Worked example
+
+```ts
+import { id, text, table } from '@voltro/database'
+
+export const users = table('users', {
+  id:     id(),                                   // PK → implicitly safe
+  email:  text().sensitive('email'),              // → fake email
+  name:   text().sensitive('fullName'),           // → fake name
+  ssn:    text().encrypted(),                      // → implicitly sensitive('secret') → nulled
+  status: text().oneOf(['active', 'banned']).safe(), // reviewed → copied verbatim
+  bio:    text(),                                  // UNCLASSIFIED → blocks a masking export
+})
+```
+
+Under a masking profile this table exports fine **except** `bio`: it is neither
+`.sensitive()` nor `.safe()`, so the export refuses and names it. Classify it
+(`.sensitive('freeText')` if it may contain PII, `.safe()` if it can't) — or
+override it in the policy's `columns` — and the export proceeds.
+
+Classification is invisible everywhere else: it does not change the column's SQL
+type, its nullability, or any query. It is read **only** at export time.
+
+## Why fail-closed
+
+Fail-**open** masking is worse than no masking. If a newly-added, unclassified
+column were copied verbatim by default, the copy would *look* masked — giving
+false confidence — while leaking real PII into dev on the very next schema
+change. The failure is silent and the blast radius grows over time.
+
+Fail-**closed** flips that: the export **stops** and names the unreviewed column,
+forcing a conscious `.sensitive()` / `.safe()` decision before any data moves.
+The cost is a one-line annotation per new column; the payoff is that a PII leak
+to a lower environment can't happen by omission. Preview the exact set that
+would block with [`--dry-run`](/docs/cli/data#dry-run-preview-without-writing)
+before a real run.
+
+## See also
+
+- [`voltro data` — masking, profiles, subsetting](/docs/cli/data) — the
+  export command, the masking policy, `--dry-run`, and the audit manifest.
+- [Column types](/docs/database/columns) — `.encrypted()` and the other
+  column modifiers.

package/templates/agent-docs/database/overview.md

@@ -0,0 +1,108 @@
+# Database
+
+> How Voltro talks to Postgres — the schema DSL, the query builder, mixins, migrations, and the reactive engine's relationship to all of it.
+
+
+
+---
+
+<!-- source: en/database/overview.md -->
+## Overview
+
+_How Voltro talks to Postgres — the schema DSL, the query builder, mixins, migrations, and the reactive engine's relationship to all of it._
+
+Voltro's data layer is opinionated. One database (Postgres), one connection layer (`@effect/sql`), one schema DSL (built into the framework). No second ORM, no second query builder, no codegen step you have to remember to run.
+
+This section covers everything about how the framework reads + writes data.
+
+## The shape of it
+
+```text
+┌───────────────────────────────────────────────────────────┐
+│  Schema DSL  (apps/api/database/*.entity.ts)              │
+│   table(...) + columns + mixins                           │
+└────────────────┬──────────────────────────────────────────┘
+                 │  voltro migrate
+                 ▼
+┌───────────────────────────────────────────────────────────┐
+│  Postgres                                                 │
+└────────────────┬──────────────────────────────────────────┘
+                 │
+                 ▼
+┌───────────────────────────────────────────────────────────┐
+│  Runtime data layer  (ctx.store inside every executor)    │
+│   - select/insert/update/delete builders                  │
+│   - dependency tracking for subscriptions                 │
+│   - tenant-mixin AND-merging                              │
+└───────────────────────────────────────────────────────────┘
+```
+
+You declare tables once. The migrator produces SQL; the runtime gives you a typed `ctx.store`; the reactive engine watches the same rows.
+
+## A first table
+
+```ts
+// apps/api/database/notes.entity.ts
+import {
+  table, id, text, integer, timestamp,
+} from '@voltro/database'
+import { tenant } from '@voltro/plugin-multitenancy'
+
+export const notes = table('notes', {
+  id:        id(),
+  title:     text(),
+  body:      text(),
+  views:     integer().default(0),
+  createdAt: timestamp().default('now'),
+  updatedAt: timestamp().default('now').onUpdate('now'),
+})
+  .with(tenant())   // adds tenantId + scoping — mixins chain via .with(), not spread
+```
+
+After `voltro migrate`:
+
+- A `notes` table exists in Postgres with the columns above + tenant column from the mixin.
+- A typed `ctx.store.select('notes')` etc. is available everywhere.
+- Subscriptions reading `notes` get auto-scoped to the caller's tenant.
+
+## What's in this section
+
+### Schema basics
+- [Column types](/docs/database/columns) — every column function, their SQL type, modifiers
+- [Mixins](/docs/database/mixins) — `tenant()`, `audit()`, `softDelete()`, writing your own
+- [Indexes](/docs/database/indexes) — single-column, composite, partial, expression, GiST, HNSW
+- [Migrations](/docs/database/migrations) — the migration story end-to-end, including production
+
+### Specialized columns
+- [Enums (`dbEnum`)](/docs/database/enums) — postgres-native ENUM types
+- [Generated columns](/docs/database/generated-columns) — DB-computed values
+- [Arrays + intervals](/docs/database/arrays-intervals) — native postgres types + cross-dialect codec
+- [JSON columns](/docs/database/json) — typed `jsonb`, indexing, paths
+- [Vector columns](/docs/database/vectors) — pgvector, embedding, ANN search, HNSW
+- [PostGIS](/docs/database/postgis) — geography/geometry for location-aware apps
+- [Full-text search](/docs/database/full-text-search) — tsvector + GIN + `.matching()`
+
+### Reading data
+- [Query builder](/docs/database/query-builder) — `select`, `where`, aggregates, raw SQL escape hatch
+- [Joins & relations](/docs/database/joins) — explicit joins, eager loading, the runtime's tracking
+- [Aggregations](/docs/database/aggregations) — `count` / `sum` / `groupBy` / `having` + window functions
+- [Sub-queries](/docs/database/sub-queries) — `inSubquery` / `exists` predicates
+- [Set operations](/docs/database/set-operations) — `union` / `intersect` / `except`
+- [DISTINCT + DISTINCT ON](/docs/database/distinct) — dedup + one-per-group
+- [Self-joins](/docs/database/self-joins) — `.as(alias)` + `.innerJoin(table, alias, on)`
+- [Recursive CTEs](/docs/database/recursive-cte) — `WITH RECURSIVE` for tree walks
+- [SQL views](/docs/database/views) — `view(name, columns, select)` read-only named SELECTs
+
+### Writing data
+- [Transactions](/docs/database/transactions) — how `ctx.store` works inside mutations + workflows
+- [Bulk operations](/docs/database/bulk-operations) — `updateMany` / `upsert` / `insertIgnore`
+
+## Conventions enforced by the framework
+
+| | Why |
+|---|---|
+| One table per `database/*.entity.ts` file + a `database/index.ts` barrel | Discovery walks `*.entity.ts` / `*.schema.ts` / `schema.ts`; the entity-per-file model is the primary convention. |
+| All columns NOT NULL by default; `.nullable()` to opt in | Three-valued logic is the source of half of all SQL bugs. The default is the safer one. |
+| `id` columns default to **TypeID** (`<prefix>_<ulid>`), not serial integers | Sortable by time, URL-safe, branded to the table type, no leaking sequential row counts. |
+| Timestamps are `timestamptz` | Always UTC, never naive. The wire format is ISO 8601. |
+| Mixins are chained via `.with(mixin())` | Keeps the table declaration readable + composable; the dep resolver dedups shared mixins. |