@rawsql-ts/ztd-cli 0.16.0 → 0.17.0

package/templates/README.md

@@ -1,217 +1,14 @@
 # Zero Table Dependency Project
 
-This project organizes all SQL‑related artifacts under the `ztd/` directory, separating concerns so both humans and AI can collaborate effectively without interfering with each other's responsibilities.
-
-```
-/ztd
-  /ddl
-    *.sql            <- schema definitions (required)
-  README.md          <- documentation for the layout
-  AGENTS.md          <- combined guidance for DDL
-
-/src                 <- application & repository code
-/tests               <- ZTD tests, fixtures, generated maps
-```
-
-Only `ztd/ddl` is part of the template contract. Do not create or assume other `ztd` subdirectories unless the project explicitly adds them.
-
-## Generated files (important)
-
-`tests/generated/` is auto-generated and must never be committed to git.
-After cloning the repository (or in a clean environment), run (strongly recommended):
-
-```bash
-npx ztd ztd-config
-```
-
-If TypeScript reports missing modules or type errors because `tests/generated/` is missing, rerun `npx ztd ztd-config`.
-
-`tests/generated/ztd-layout.generated.ts` declares the directories above so the CLI and your tests always point at the intended files. The authoritative directory remains `ztd/ddl/`; do not read or assume additional `ztd` subdirectories.
-
----
-
-# Optional SqlClient seam
-
-If this project was initialized with `npx ztd init --with-sqlclient`, you'll also have `src/db/sql-client.ts`.
-It defines a minimal `SqlClient` interface that repositories can depend on:
-
-- Use it for tutorials and greenfield projects to keep repository SQL decoupled from drivers.
-- Skip it when you already have a database abstraction (Prisma, Drizzle, Kysely, custom adapters).
-- For `pg`, adapt `client.query(...)` so it returns a plain `T[]` row array that matches the interface.
-- Prefer a shared client per worker process so tests and scripts do not reconnect on every query.
-- Do not share a live connection across parallel workers; each worker should own its own shared client.
-
-Example (driver-agnostic):
-
-```ts
-let sharedClient: SqlClient | undefined;
-
-export function getSqlClient(): SqlClient {
-  if (!sharedClient) {
-    // Create the client once using your chosen driver (pg, mysql, etc.).
-    sharedClient = createSqlClientOnce();
-  }
-  return sharedClient;
-}
-```
-
----
-
-# Mapper + writer sample
-
-- This scaffold already exposes `src/repositories/user-accounts.ts`, which maps `public.user_account` rows together with optional `public.user_profile` data through `@rawsql-ts/sql-contract/mapper` and emits insert/update/remove helpers via `@rawsql-ts/sql-contract/writer`.
-- The SQL used here is defined in `src/repositories/user-accounts.ts`; the template tests exercise that implementation and `ztd/ddl` is the authoritative source for every column and constraint.
-- Two template tests demonstrate how to run the stitch:
-  - `tests/user-profiles.test.ts` seeds fixtures, executes the query through the mapper, and verifies the DTO shape.
-  - `tests/writer-constraints.test.ts` reads `userAccountWriterColumnSets` plus `tests/generated/ztd-row-map.generated.ts` so writer callers stay within the approved column set when referencing `public.user_account`.
-- Regenerate `tests/generated/ztd-row-map.generated.ts` (`npx ztd ztd-config`) before running the example tests so the row map reflects any schema changes.
-- The example tests require a real PostgreSQL connection via `DATABASE_URL`; they automatically skip when the variable is missing so local tooling stays fast.
-
-# Principles
-
-### 1. Humans own the *definitions*
-- DDL (physical schema)
-  Only `ztd/ddl` is part of the template contract; other subdirectories should not be assumed.
-
-### 2. AI owns the *implementation*
-- Repository SQL generation
-- Test fixture updates
-- Intermediate TypeScript structures
-- SQL rewriting, parameter binding, shape resolution
-
-### 3. ZTD ensures these stay in sync
-ZTD acts as the consistency layer ensuring:
-- DDL → SQL shape consistency
-- Do not rely on other directories unless the project explicitly adds them.
-
-If any part diverges, ZTD tests fail deterministically.
-
----
-
-# Workflow Overview
-
-Different tasks start from different entry points. Choose the workflow that matches what you want to change.
-
----
-
-# Workflow A — Starting From *DDL Changes*  
-(Adding tables/columns, changing constraints)
-
-1. Edit files under `ztd/ddl/`.
-2. Run:
-
-   ```bash
-   npx ztd ztd-config
-   ```
-
-   This regenerates `tests/generated/ztd-row-map.generated.ts` from the new schema.
-
-3. Update repository SQL so it matches the new schema.
-4. Update fixtures if shapes changed.
-5. Run tests. Any schema mismatch will fail fast.
-
-**Flow:**  
-**DDL -> repository SQL -> fixtures/tests -> application**
-
----
-
-# Workflow B — Starting From *Repository Interface Changes*  
-(Adding a method, changing return types, etc.)
-
-1. Modify the repository interface or class in `/src`.
-2. Allow AI to generate the SQL needed to satisfy the interface.
-3. If the query contradicts DDL, reconcile the authoritative definition before continuing.
-4. Run ZTD tests to confirm logic is consistent.
-5. Regenerate ZTD config if result shapes changed.
-
-**Flow:**  
-**repository interface -> SQL -> tests**
-
----
-
-# Workflow C — Starting From Repository SQL Logic Changes
-(Fixing a bug, optimizing logic, rewriting a query)
-
-1. Edit SQL inside the repository.
-2. Run ZTD tests.
-3. If the intended behavior changes, update the DDL before adjusting dependent logic and keep documentation aligned with the confirmed schema.
-4. Update fixtures as necessary.
-5. If SQL result shape changed, run:
-
-   ```bash
-   npx ztd ztd-config
-   ```
-
-**Flow:**  
-**SQL -> fixtures/tests**
-
----
-
-# Combined Real‑World Flow Examples
-
-- **Add a new contract status**  
-  DDL -> SQL -> config -> tests
-
-- **Add a new table**  
-  DDL -> config -> SQL -> fixtures -> tests
-
-- **Fix business logic**  
-  SQL -> tests
-
-ZTD ensures all changes converge into a consistent, validated workflow.
----
-
-# Human Responsibilities
-
-- Humans maintain:
-  - Physical schema (`ddl`)
-  - High‑level repository interfaces
-  - Acceptance of AI-generated changes
-
-Humans decide “what is correct.”
-
----
-
-# AI Responsibilities
-
-AI must:
-
-  - Use DDL as the **physical shape constraint** and primary source of truth.
-  - Do not assume any additional `ztd` directories exist unless a human explicitly creates them.
-  - Generate repository SQL consistent with DDL and the documented behavior.
-  - Regenerate fixtures and tests as instructed.
-  - Never modify `ztd/AGENTS.md` or `ztd/README.md` unless explicitly asked.
-
-AI decides “how to implement” within those constraints.
-
----
-
-# ZTD CLI Responsibilities
-
-ZTD CLI:
-
-- Parses DDL files to build accurate table/column shapes
-- Rewrites SQL with fixture-based CTE shadowing (via testkit adapters)
-- Generates `ztd-row-map.generated.ts`
-- Produces deterministic, parallelizable tests
-
-ZTD is the verification engine guaranteeing correctness.
-
-## Traditional execution mode
-
-- Set `ZTD_EXECUTION_MODE=traditional` or pass `{ mode: 'traditional', traditional: { isolation: 'schema', cleanup: 'drop_schema' } }` when you need to run the tests against a real Postgres schema (locking, isolation, constraints). The helper still applies the DDL inside `ztd/ddl/`, loads the fixture rows into the schema, optionally executes `setupSql`, and carries out the chosen cleanup strategy (`drop_schema`, `custom_sql`, or `none`).
-- Use `isolation: 'none'` if you need to target a schema that is already defined or if your SQL embeds schema qualifiers explicitly.
-
----
-
-# Summary
-
-ZTD enables a workflow where **humans define meaning**, **AI writes implementation**, and **tests guarantee correctness**.
-
-The project layout and workflows above ensure long-term maintainability, clarity, and full reproducibility of SQL logic independent of physical database state.
-
-## Recommended local verification
-
-- `npx ztd ztd-config` (Recommended)
-- `pnpm -C packages/ztd-cli test` (Recommended)
-- `templates/tests/user-profiles.test.ts` runs only when `DATABASE_URL` is configured; otherwise it skips automatically.
+This project uses Zero Table Dependency (ZTD) to keep SQL, DDL, and tests aligned.
+
+Key folders:
+- ztd/ddl: schema files (source of truth)
+- src: application SQL and repositories
+- tests: ZTD tests and support
+
+Next steps:
+1. Update `ztd/ddl/<schema>.sql` if needed.
+2. Run `npx ztd ztd-config`.
+3. Provide a SqlClient implementation.
+4. Run tests (`pnpm test` or `npx vitest run`).

package/templates/dist/ztd-cli/templates/src/db/sql-client.ts

@@ -0,0 +1,24 @@
+/**
+ * Promise that resolves to the array of rows produced by an SQL query.
+ * @template T Shape of each row yielded by the SQL client.
+ * @example
+ * const rows: SqlQueryRows<{ id: number; name: string }> = client.query('SELECT id, name FROM users');
+ */
+export type SqlQueryRows<T> = Promise<T[]>;
+
+/**
+ * Minimal SQL client interface required by the repository layer.
+ *
+ * - Production: adapt this interface to your preferred driver (pg, mysql2, etc.) and normalize the results to `T[]`.
+ * - Tests: replace the implementation with a mock, a fixture helper, or an adapter that follows this contract.
+ *
+ * Connection strategy note:
+ * - Prefer a shared client per worker process for better performance.
+ * - Do not share a live client across parallel workers without proper synchronization.
+ */
+export type SqlClient = {
+  query<T extends Record<string, unknown> = Record<string, unknown>>(
+    text: string,
+    values?: readonly unknown[]
+  ): SqlQueryRows<T>;
+};

package/templates/src/AGENTS.md

@@ -0,0 +1,26 @@
+# src AGENTS
+
+This directory is runtime application code.
+
+## Boundaries
+
+- Code under "src/" MUST NOT import from:
+  - "tests/"
+  - "tests/generated/"
+  - any test-only helpers
+- Runtime code MUST NOT depend on ZTD internals.
+- Generated artifacts are test-only signals, not runtime dependencies.
+
+## Implementation principles
+
+- Keep modules small and explicit.
+- Prefer explicit contracts over inference.
+- Favor deterministic behavior and clear error surfaces.
+
+## Verification (required)
+
+After changes:
+- Run the project typecheck command (example: "pnpm typecheck").
+- Run relevant tests (example: "pnpm test" or a filtered command).
+
+If you touched SQL contracts or catalog specs, run tests that exercise them.

package/templates/src/catalog/AGENTS.md

@@ -0,0 +1,37 @@
+# src/catalog AGENTS
+
+This directory is runtime code.
+
+## Purpose
+
+Catalog defines named query entry points by binding:
+- SQL assets ("src/sql/*.sql")
+- input parameter contracts
+- output DTO contracts
+- validation and mapping behavior
+- observability hooks (if supported)
+
+## Directory roles (important)
+
+- "src/catalog/specs": human-owned contracts (params + DTO + semantics)
+- "src/catalog/runtime": AI-assisted runtime wiring (executors, helpers, sinks)
+
+## Non-negotiable ownership
+
+- Specs are contracts. Do not infer, guess, widen, or narrow them.
+- Do not change params / DTO shapes in "specs" without explicit instruction.
+
+## Boundaries
+
+- Code under "src/catalog/" MUST NOT import from:
+  - "tests/"
+  - "tests/generated/"
+  - "ztd/"
+- Do not depend on ZTD internals at runtime.
+
+## Testing rule (required)
+
+Every spec MUST have tests that verify:
+- SQL executes under ZTD rewriting
+- mapping/validation behavior is correct (success and failure)
+- output DTO shape matches expectations

package/templates/src/catalog/runtime/AGENTS.md

@@ -0,0 +1,75 @@
+# src/catalog/runtime AGENTS
+
+This directory contains runtime wiring for catalog specs:
+- parameter validation entrypoints
+- row-to-DTO mapping
+- output validation
+
+## Runtime classification
+
+- This is a runtime directory.
+- Code here is executed/loaded by the application.
+
+## Responsibilities (critical)
+
+- Runtime MUST be the only place that:
+  - validates unknown inputs into typed params
+  - maps SQL rows (snake_case) into DTO objects
+  - validates DTO outputs before returning to repositories
+- Runtime output validation MUST follow the output contract type (DTO or scalar).
+- Command outputs MAY be scalar identifiers or `void`; DTO mapping is required only when the output contract is DTO.
+
+Repositories MUST call runtime helpers (ensure*/map*) and MUST NOT bypass them.
+
+## Validator requirement (required)
+
+- A validator library (zod or arktype) is always available.
+- Runtime MUST apply validators for:
+  - input params (unknown -> typed)
+  - outputs (mapped DTO -> validated DTO)
+
+Do not rely on TypeScript types alone.
+
+## SQL row normalization (important)
+
+SQL drivers may return different runtime representations for the same column types.
+Runtime MUST normalize driver-dependent values before validating DTOs.
+
+Required normalization rules:
+- Timestamp columns (e.g. timestamptz) may arrive as Date or string.
+  - Runtime MUST use `timestampFromDriver` from `@rawsql-ts/sql-contract`.
+  - Do NOT re-implement `normalizeTimestamp` locally.
+  - Do NOT call `new Date(...)` directly for driver-dependent timestamp normalization.
+- Numeric columns may arrive as number, string, or bigint depending on driver.
+  - Normalization rules MUST be explicit per contract and MUST NOT be silent.
+
+Never force SQL assets to encode driver-specific behavior.
+
+## Mapping rules (required)
+
+- Keep SQL assets snake_case and DTO-independent.
+- Mapping occurs in runtime:
+  - snake_case row -> DTO-shaped object
+  - apply normalization
+  - validate via spec-owned validator
+- DTO camelCase aliases in SQL are forbidden.
+
+## Entry points (recommended)
+
+Prefer these patterns:
+- `ensureXxxParams(value: unknown): XxxParams` (validate inputs)
+- `mapXxxRowToDto(row: XxxSqlRow): XxxDto` (normalize + validate outputs)
+
+The `map*` functions MUST always validate before returning.
+
+## Error behavior (required)
+
+- Validation errors must fail fast with clear messages.
+- Do not swallow validator errors.
+- Do not silently coerce invalid values unless explicitly defined by the contract.
+
+## Boundaries
+
+- Do not perform database I/O here.
+- Do not import from "tests/" or "tests/generated/".
+- Do not define human-owned contracts here; they live in "src/catalog/specs".

package/templates/src/catalog/runtime/_coercions.ts

@@ -0,0 +1 @@
+export { timestampFromDriver as normalizeTimestamp } from '@rawsql-ts/sql-contract';

package/templates/src/catalog/runtime/_smoke.runtime.ts

@@ -0,0 +1,21 @@
+import { parseSmokeOutput, type SmokeOutput } from '../specs/_smoke.spec';
+import { normalizeTimestamp } from './_coercions';
+
+/**
+ * Validate runtime output against the catalog smoke invariant.
+ */
+export function ensureSmokeOutput(value: unknown): SmokeOutput {
+  // Normalize driver-dependent timestamp representations before contract validation.
+  if (isRecord(value) && 'createdAt' in value) {
+    return parseSmokeOutput({
+      ...value,
+      createdAt: normalizeTimestamp(value.createdAt, 'createdAt')
+    });
+  }
+
+  return parseSmokeOutput(value);
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return typeof value === 'object' && value !== null;
+}

package/templates/src/catalog/specs/AGENTS.md

@@ -0,0 +1,48 @@
+# src/catalog/specs AGENTS
+
+This directory defines query and command specifications (contracts).
+
+## Runtime classification
+
+- This is a runtime directory.
+- Contents are human-owned contracts, not implementation details.
+
+## Ownership (critical)
+
+- Specs are human-owned.
+- Input parameters are part of the domain contract.
+- Output DTO shapes are part of the domain contract.
+- Cardinality and semantics are part of the domain contract.
+
+AI MUST NOT:
+- infer missing parameters
+- widen or narrow DTO shapes
+- change nullability or optionality
+- change CUD return shape implicitly
+
+Changes require explicit human instruction.
+
+## Validator requirement (required)
+
+- A validator library (zod or arktype) is always available.
+- Every spec MUST define validators for its public inputs/outputs.
+- Validators are part of the contract and are human-owned.
+
+## Driver variability policy (important)
+
+Specs MUST NOT require SQL assets to encode driver-specific workarounds.
+
+- Specs may acknowledge that SQL rows can vary in representation by driver
+  (e.g. timestamps as Date or string).
+- Normalization for driver-dependent values is implemented in catalog runtime,
+  then validated against the spec.
+
+Contracts define the final DTO shape.
+Runtime defines normalization steps to reach that shape.
+
+## What does NOT belong here
+
+- No executors
+- No database connections
+- No ZTD internals
+- No SQL camelCase aliasing rules (SQL rules live under src/sql)

package/templates/src/catalog/specs/_smoke.spec.arktype.ts

@@ -0,0 +1,21 @@
+import { type } from 'arktype';
+
+/**
+ * Validator invariant contract used to prove runtime validation wiring.
+ *
+ * This file is intentionally minimal and domain-agnostic.
+ */
+export const smokeOutputSchema = type({
+  id: 'number.integer',
+  createdAt: 'Date'
+});
+
+export type SmokeOutput = ReturnType<typeof smokeOutputSchema>;
+
+/**
+ * Parse and validate an unknown runtime payload.
+ */
+export function parseSmokeOutput(value: unknown): SmokeOutput {
+  smokeOutputSchema.assert(value);
+  return value as SmokeOutput;
+}

package/templates/src/catalog/specs/_smoke.spec.zod.ts

@@ -0,0 +1,20 @@
+import { z } from 'zod';
+
+/**
+ * Validator invariant contract used to prove runtime validation wiring.
+ *
+ * This file is intentionally minimal and domain-agnostic.
+ */
+export const smokeOutputSchema = z.object({
+  id: z.number().int(),
+  createdAt: z.date()
+});
+
+export type SmokeOutput = z.infer<typeof smokeOutputSchema>;
+
+/**
+ * Parse and validate an unknown runtime payload.
+ */
+export function parseSmokeOutput(value: unknown): SmokeOutput {
+  return smokeOutputSchema.parse(value);
+}

package/templates/src/db/sql-client.ts

@@ -9,16 +9,16 @@ export type SqlQueryRows<T> = Promise<T[]>;
 /**
  * Minimal SQL client interface required by the repository layer.
  *
- * - Production: adapt `pg` (or other drivers) to normalize results into `T[]`
- * - Tests: compatible with the `testkit-postgres` pipeline exposed by `@rawsql-ts/adapter-node-pg` clients returned by `createTestkitClient()`
+ * - Production: adapt this interface to your preferred driver (pg, mysql2, etc.) and normalize the results to `T[]`.
+ * - Tests: replace the implementation with a mock, a fixture helper, or an adapter that follows this contract.
  *
  * Connection strategy note:
- * - Prefer a shared client per worker process for performance.
- * - Do not share a live client across parallel workers.
+ * - Prefer a shared client per worker process for better performance.
+ * - Do not share a live client across parallel workers without proper synchronization.
  */
 export type SqlClient = {
   query<T extends Record<string, unknown> = Record<string, unknown>>(
     text: string,
-    values?: readonly unknown[]
+    values?: readonly unknown[] | Record<string, unknown>
   ): SqlQueryRows<T>;
 };

package/templates/src/jobs/AGENTS.md

@@ -0,0 +1,26 @@
+# src/jobs AGENTS
+
+This folder contains script-like operations (procedural SQL execution).
+
+## Scope
+
+- Maintenance jobs
+- Data backfills
+- Batch operations
+- Temporary-table driven workflows
+
+## Safety rules
+
+- Be explicit about transaction boundaries.
+- Prefer idempotent design (safe to rerun).
+- Emit clear logs/events at start and end of the job.
+
+## SQL usage
+
+- It is acceptable to run multiple statements in a job (including temp tables).
+- Avoid coupling job logic to tests-only helpers.
+
+## Testing
+
+- Prefer integration-style tests that verify observable outcomes.
+- If the job is heavy, test smaller units or use fixtures to limit scope.

package/templates/src/jobs/README.md

@@ -0,0 +1,3 @@
+# Jobs
+
+Job runners that orchestrate repository or SQL workflows.

package/templates/src/repositories/AGENTS.md

@@ -0,0 +1,118 @@
+# src/repositories AGENTS
+
+This directory contains repository implementations.
+
+## Runtime classification
+
+- This is a runtime directory.
+- Code here is executed by the application.
+
+## Responsibilities
+
+Repositories orchestrate execution only.
+
+They are responsible for:
+- loading SQL assets from `src/sql`
+- validating external inputs via catalog runtime
+- binding parameters
+- calling `SqlClient`
+- mapping SQL rows to DTOs via catalog runtime
+- returning application-facing results
+
+Repositories MUST remain thin adapters over SQL.
+
+Repositories MUST NOT:
+- embed business rules
+- infer schema or DTO structure
+- compensate for driver limitations with extra queries
+
+## Contract boundaries (important)
+
+- Human-owned contracts live under "src/catalog/specs".
+- Runtime normalization and validation live under "src/catalog/runtime".
+- Repositories MUST NOT invent, infer, or modify contracts.
+
+If a contract is insufficient, update the spec/runtime first.
+
+## Mandatory runtime usage (required)
+
+Repositories MUST use catalog runtime helpers:
+- `ensure*Params` for all external inputs (`unknown -> typed`)
+- `map*RowToDto` for row-to-DTO mapping and output validation
+
+Repositories MUST NOT:
+- call spec validators directly
+- bypass runtime normalization
+- return DTOs without runtime validation
+- assume driver-specific runtime types (e.g. timestamps always `Date`)
+
+## SQL rules
+
+- SQL assets MUST remain DTO-independent.
+- SQL files MUST use snake_case column names.
+- Repositories MUST NOT push DTO aliasing into SQL.
+- Inline SQL strings are forbidden.
+- Repository SQL loading MUST be centralized in shared runtime/repository infrastructure.
+- Repositories MUST reference SQL assets by stable logical key (example: `user/insert_user.sql`).
+- Repository modules MUST NOT pass caller-relative paths (`./`, `../`) or absolute filesystem paths to SQL loaders.
+- The shared loader MUST resolve logical keys against the `src/sql` root (or catalog SQL registry), never relative to the caller file location.
+- Repository modules MUST NOT implement ad-hoc SQL file loading with `readFileSync`, `__dirname`, `import.meta.url`, or direct path resolution.
+
+## CUD default policy (important)
+
+Default policy for table-oriented CUD is:
+- CREATE SQL MAY use `RETURNING` for identifier columns only.
+- CREATE repository methods MUST return the generated identifier by default.
+- CREATE repository methods MUST NOT perform a follow-up SELECT whose purpose is DTO return shaping.
+- UPDATE/DELETE SQL MUST NOT use `RETURNING`.
+- repositories do NOT perform follow-up SELECTs after UPDATE/DELETE.
+- UPDATE/DELETE methods return `Promise<void>` by default.
+
+If a CREATE method must return a DTO, that requirement MUST be explicitly declared in catalog specs by human instruction.
+
+## CUD row count rules (non-negotiable)
+
+- UPDATE and DELETE MUST rely on driver-reported affected row counts (`rowCount` or equivalent).
+- Repositories MUST NOT use SQL `RETURNING` result presence/count as a substitute for affected row counts.
+- Repositories MUST NOT emulate affected rows using:
+  - pre-check SELECT
+  - post-check SELECT
+  - retry loops
+  - shadow queries / CTE hacks
+
+If `rowCount === 0`, repositories MUST throw an explicit error.
+
+If the driver cannot report affected row counts:
+- CUD verification is unsupported
+- tests MUST fail fast
+- do NOT add workaround logic in repositories
+
+## Read (SELECT) rules
+
+- SELECT methods may return:
+  - `T | null` for single-row queries
+  - `T[]` for multi-row queries
+- Absence of rows is NOT an error unless specified by the contract.
+
+## Error behavior
+
+- Propagate validation errors as failures.
+- Do not silently coerce invalid data.
+- Error messages SHOULD include operation name and identifying parameters.
+
+## Testing expectations
+
+- Every public repository method MUST be covered by tests.
+- Update/Delete tests MUST verify rowCount handling explicitly.
+- Repository changes without tests are incomplete.
+
+## Guiding principle
+
+Repositories execute contracts.
+They do not interpret intent.
+
+If correctness depends on extra queries to "confirm" changes:
+- the contract is wrong
+- or the driver is unsupported
+
+Fix the contract, not the repository.