ts-procedures 5.7.2 → 5.9.0

package/docs/superpowers/specs/2026-03-30-client-codegen-design.md

@@ -0,0 +1,632 @@
+# ts-procedures Client Code Generation
+
+**Date:** 2026-03-30
+**Status:** Draft
+
+## Problem
+
+Downstream applications that consume ts-procedures APIs must manually match server contracts on the client side. There is no type-safe bridge between the server's schema definitions and the client's HTTP calls. The DocRegistry already produces a JSON spec of all procedures (served via `GET /docs`), but this spec provides documentation only — no generated TypeScript types, no callable functions, no type safety.
+
+## Solution
+
+A static code generation system that reads the DocRegistry's `DocEnvelope` output and produces per-scope TypeScript files containing typed params/response/yield types and callable functions for every procedure. The generated code imports a lightweight runtime client that handles HTTP execution, hook pipelines, and stream consumption. App developers plug in their own HTTP library (fetch, axios, etc.) via an adapter interface.
+
+## Architecture
+
+Three layers, two new package exports:
+
+```
+┌─────────────────────────────────────────────────────────┐
+│  ts-procedures/client  (runtime — imported by app code) │
+│  - ClientAdapter interface (request + stream methods)   │
+│  - Hook types (onBeforeRequest, onAfterResponse, etc.)  │
+│  - createClient() factory                               │
+│  - Stream consumption utilities (SSE parser)            │
+│  - Client-side error classes                            │
+└─────────────────────────────────────────────────────────┘
+                          ▲
+                          │ imports
+┌─────────────────────────────────────────────────────────┐
+│  Generated code  (per-scope .ts files in app repo)      │
+│  - TypeScript types (params, response, yield) via ajsc  │
+│  - Callable functions per procedure                     │
+│  - One file per scope + barrel index.ts                 │
+└─────────────────────────────────────────────────────────┘
+                          ▲
+                          │ produces
+┌─────────────────────────────────────────────────────────┐
+│  ts-procedures/codegen  (build-time tool)               │
+│  - generateClient() programmatic API                    │
+│  - CLI: npx ts-procedures-codegen                       │
+│  - Uses ajsc for JSON Schema → TypeScript conversion    │
+│  - Reads DocEnvelope from URL, file, or object          │
+└─────────────────────────────────────────────────────────┘
+```
+
+- `ts-procedures/client` — zero external dependencies, pure TypeScript runtime
+- `ts-procedures/codegen` — depends on `ajsc` for type generation, build-time only
+
+## Runtime Client (`ts-procedures/client`)
+
+### ClientInstance
+
+The core internal type that generated scope bindings receive. This is constructed by `createClient` and passed to each `bind*Scope` factory:
+
+```ts
+interface ClientInstance {
+  basePath: string;
+  adapter: ClientAdapter;
+  hooks: ClientHooks;
+
+  /** Execute a request/response procedure call */
+  call<TResponse>(
+    descriptor: CallDescriptor,
+    options?: ProcedureCallOptions,
+  ): Promise<TResponse>;
+
+  /** Execute a streaming procedure call */
+  stream<TYield, TReturn>(
+    descriptor: StreamDescriptor,
+    options?: ProcedureCallOptions,
+  ): TypedStream<TYield, TReturn>;
+}
+
+interface CallDescriptor {
+  name: string;
+  scope: string;
+  path: string;
+  method: string;
+  params: unknown;
+}
+
+interface StreamDescriptor extends CallDescriptor {
+  streamMode: 'sse' | 'text';
+}
+
+type ProcedureCallOptions = ClientHooks;
+```
+
+### Adapter Interface
+
+App developers implement this to wrap their HTTP library of choice:
+
+```ts
+interface ClientAdapter {
+  request(config: AdapterRequest): Promise<AdapterResponse>;
+  stream(config: AdapterRequest): Promise<AdapterStreamResponse>;
+}
+
+interface AdapterRequest {
+  url: string;
+  method: string;
+  headers?: Record<string, string>;
+  body?: unknown;
+  signal?: AbortSignal;
+}
+
+interface AdapterResponse {
+  status: number;
+  headers: Record<string, string>;
+  body: unknown;
+}
+
+interface AdapterStreamResponse {
+  status: number;
+  headers: Record<string, string>;
+  body: AsyncIterable<unknown>;
+}
+```
+
+Two methods — `request()` for RPC/API calls, `stream()` for SSE/text streaming. A user who does not use streaming only needs to implement `request()`.
+
+The adapter's `stream()` method returns a raw `AsyncIterable<unknown>` of whatever the transport provides (raw SSE frames, text chunks, etc.). The runtime client handles parsing and type-narrowing — see the Stream Consumption section below.
+
+### Hook System
+
+Global hooks configured on the client, with per-procedure overrides available on any call:
+
+```ts
+interface ClientHooks {
+  onBeforeRequest?(context: BeforeRequestContext): BeforeRequestContext | Promise<BeforeRequestContext>;
+  onAfterResponse?(context: AfterResponseContext): void | Promise<void>;
+  onError?(context: ErrorContext): void | Promise<void>;
+}
+
+interface BeforeRequestContext {
+  procedureName: string;
+  scope: string;
+  request: AdapterRequest;        // mutable — hooks modify this
+}
+
+interface AfterResponseContext {
+  procedureName: string;
+  scope: string;
+  request: AdapterRequest;        // as-sent (after onBeforeRequest)
+  response: AdapterResponse;      // mutable — hooks can transform
+}
+
+interface ErrorContext {
+  procedureName: string;
+  scope: string;
+  request: AdapterRequest;
+  error: unknown;
+}
+```
+
+**Execution order for request/response calls:**
+
+1. Build `AdapterRequest` from procedure metadata + caller args
+2. Run global `onBeforeRequest` then per-procedure `onBeforeRequest`
+3. Call `adapter.request()`
+4. If non-2xx and no hook swallows it, throw
+5. Run global `onAfterResponse` then per-procedure `onAfterResponse`
+6. Return typed result
+
+**Execution order for streaming calls:**
+
+1. Build `AdapterRequest` from procedure metadata + caller args
+2. Run global `onBeforeRequest` then per-procedure `onBeforeRequest`
+3. Call `adapter.stream()`
+4. `onAfterResponse` fires immediately after the initial HTTP response (status + headers), before stream iteration begins — this lets hooks check status codes, read headers, or reject the stream early
+5. If non-2xx and no hook swallows it, throw
+6. Return `TypedStream` — iteration begins when the consumer starts reading
+7. If an error occurs mid-stream, `onError` fires with the error context
+
+**Error strategy:** Hooks decide. Default behavior is throw on non-2xx. Generated callables return `Promise<T>` on the happy path. Apps implement their own error handling via `onAfterResponse` and `onError` hooks.
+
+### Stream Consumption and TypedStream
+
+```ts
+interface TypedStream<TYield, TReturn = void> extends AsyncIterable<TYield> {
+  /** Resolves when the stream completes. Contains the final return value if the
+   *  stream's returnType schema is defined, otherwise void.
+   *  Rejects if the stream errors before completing. */
+  result: Promise<TReturn>;
+}
+```
+
+**How the runtime distinguishes yields from the final return:**
+
+For SSE streams, the server's Hono stream builder wraps each yield in an SSE envelope (`{ data, event, id, retry }`). The runtime SSE parser strips this envelope and emits only the `data` payload as each `TYield` value. The final return value (if the procedure defines `returnType`) is sent as an SSE event with `event: 'return'` — the runtime captures this, resolves `.result`, and does not emit it as a yield.
+
+For text streams, the runtime emits each chunk as a `TYield` value. `TReturn` is `void` for text streams (no final return mechanism).
+
+**When `TReturn` is void:** If the server procedure has no `returnType` schema, the generated callable uses `TypedStream<TYield, void>` and `.result` resolves to `void` on stream completion. The `.result` promise always exists for uniform error handling (`.result.catch(...)`) regardless of whether there is a return value.
+
+### Path Parameter Interpolation
+
+The runtime `call()` method interpolates path parameters using simple `:param` replacement:
+
+```ts
+// path: '/users/:id/posts/:postId'
+// params.pathParams: { id: '123', postId: '456' }
+// result: '/users/123/posts/456'
+```
+
+Each `:paramName` segment in the path is replaced with the corresponding value from `params.pathParams`. Values are URI-encoded. Unknown params are ignored. Missing params throw a client-side error.
+
+### createClient Factory
+
+```ts
+import { createClient } from 'ts-procedures/client';
+import { createScopeBindings } from './generated';
+
+const client = createClient({
+  adapter: myFetchAdapter,
+  basePath: 'http://localhost:3000/api/v1',
+  scopes: createScopeBindings,
+  hooks: {
+    onBeforeRequest({ request, ...rest }) {
+      request.headers = {
+        ...request.headers,
+        Authorization: `Bearer ${getToken()}`,
+      };
+      return { ...rest, request };
+    },
+    onAfterResponse({ response }) {
+      if (response.status === 401) redirect('/login');
+    },
+  },
+});
+
+// Fully typed usage:
+const user = await client.users.GetUser({ pathParams: { id: '123' } });
+
+// Per-procedure hook override:
+await client.users.GetUser({ pathParams: { id: '123' } }, {
+  onAfterResponse({ response }) {
+    const rateLimit = response.headers['x-rate-limit-remaining'];
+  },
+});
+
+// Streaming:
+const stream = client.notifications.WatchNotifications({ accountId: 'abc' });
+for await (const event of stream) {
+  console.log(event);
+}
+const finalResult = await stream.result;
+```
+
+## Generated Code
+
+### Scope Normalization
+
+RPC and Stream routes define `scope` as `string | string[]` (e.g., `'users'` or `['admin', 'users']`). API routes will add an optional `scope` field. The codegen normalizes scope to a canonical string key for file grouping:
+
+- **String scope:** used as-is → `'users'` → `users.ts`
+- **Array scope:** joined with `-` → `['admin', 'users']` → `admin-users.ts`
+- **Missing scope (API routes):** falls back to `'default'` → `default.ts`. The codegen emits a warning so developers know to add explicit scopes.
+
+The `bind*Scope` function name and the property key in `createScopeBindings` use the normalized scope converted to camelCase (e.g., `admin-users` → `bindAdminUsersScope`, `adminUsers` property).
+
+### File Layout
+
+One file per normalized scope, plus a barrel index:
+
+```
+generated/
+  users.ts
+  billing.ts
+  admin-users.ts
+  notifications.ts
+  index.ts
+```
+
+### Scope File Structure
+
+Each scope file contains types (via ajsc) and a `bind*Scope` factory:
+
+```ts
+// Auto-generated by ts-procedures-codegen — do not edit
+import type { ClientInstance, ProcedureCallOptions } from 'ts-procedures/client';
+
+// ── Types ────────────────────────────────────────────────────
+
+export type GetUserParams = {
+  pathParams: {
+    id: string;
+  };
+  query?: {
+    include?: string;
+  };
+};
+
+export type GetUserResponse = {
+  id: string;
+  name: string;
+  email: string;
+  createdAt: string;
+};
+
+export type CreateUserParams = {
+  body: {
+    name: string;
+    email: string;
+    role?: 'admin' | 'member';
+  };
+};
+
+export type CreateUserResponse = {
+  id: string;
+  name: string;
+  email: string;
+  role: 'admin' | 'member';
+};
+
+// ── Callables ────────────────────────────────────────────────
+
+export function bindUsersScope(client: ClientInstance) {
+  return {
+    /** GET /api/v1/users/:id */
+    GetUser(params: GetUserParams, options?: ProcedureCallOptions): Promise<GetUserResponse> {
+      return client.call({
+        name: 'GetUser',
+        scope: 'users',
+        path: '/users/:id',
+        method: 'get',
+        params,
+      }, options);
+    },
+
+    /** POST /api/v1/users */
+    CreateUser(params: CreateUserParams, options?: ProcedureCallOptions): Promise<CreateUserResponse> {
+      return client.call({
+        name: 'CreateUser',
+        scope: 'users',
+        path: '/users',
+        method: 'post',
+        params,
+      }, options);
+    },
+  };
+}
+```
+
+### Stream Scope Example
+
+Generated yield types represent the **data payload only** — the SSE envelope (event, id, retry) is handled by the runtime and is not part of the generated type:
+
+```ts
+// Auto-generated by ts-procedures-codegen — do not edit
+import type { ClientInstance, ProcedureCallOptions, TypedStream } from 'ts-procedures/client';
+
+export type WatchNotificationsParams = {
+  accountId: string;
+};
+
+export type WatchNotificationsYield = {
+  id: string;
+  type: 'invoice' | 'payment' | 'alert';
+  message: string;
+};
+
+export type WatchNotificationsReturn = {
+  total: number;
+};
+
+export function bindNotificationsScope(client: ClientInstance) {
+  return {
+    /** SSE /api/v1/notifications/WatchNotifications/1 */
+    WatchNotifications(
+      params: WatchNotificationsParams,
+      options?: ProcedureCallOptions,
+    ): TypedStream<WatchNotificationsYield, WatchNotificationsReturn> {
+      return client.stream({
+        name: 'WatchNotifications',
+        scope: 'notifications',
+        path: '/notifications/WatchNotifications/1',
+        method: 'post',
+        streamMode: 'sse',
+        params,
+      }, options);
+    },
+  };
+}
+```
+
+For streams with no `returnType` schema, the callable returns `TypedStream<TYield, void>`.
+
+### Barrel Index
+
+```ts
+// Auto-generated by ts-procedures-codegen — do not edit
+import type { ClientInstance } from 'ts-procedures/client';
+import { bindUsersScope } from './users';
+import { bindBillingScope } from './billing';
+import { bindNotificationsScope } from './notifications';
+
+export * from './users';
+export * from './billing';
+export * from './notifications';
+
+export function createScopeBindings(client: ClientInstance) {
+  return {
+    users: bindUsersScope(client),
+    billing: bindBillingScope(client),
+    notifications: bindNotificationsScope(client),
+  };
+}
+```
+
+## Code-Gen Engine (`ts-procedures/codegen`)
+
+### Programmatic API
+
+```ts
+import { generateClient } from 'ts-procedures/codegen';
+
+await generateClient({
+  // Input — one of:
+  url: 'http://localhost:3000/docs',
+  file: './docs.json',
+  envelope: docEnvelopeObject,
+
+  // Output
+  outDir: './src/generated/api',
+
+  // Options
+  ajsc: {
+    enumStyle: 'union',
+    depluralize: true,
+  },
+});
+```
+
+### Pipeline
+
+1. **Resolve DocEnvelope** — fetch from URL, read file, or accept object directly
+2. **Normalize scopes** — for each route, normalize `scope` to a canonical string key (join arrays with `-`, default to `'default'` for API routes without scope)
+3. **Group routes by normalized scope**
+4. **For each scope, for each route:**
+   - Detect route kind via the `kind` discriminant field
+   - Convert each JSON Schema channel through ajsc `TypescriptConverter` to type strings
+   - For SSE streams: use the inner `data` schema from the yieldType SSE envelope, not the envelope itself
+   - Build callable function with hardcoded route metadata and typed signature
+5. **Assemble scope file** — imports + types + bind function
+6. **Generate index.ts** — barrel exports + `createScopeBindings`
+7. **Write all files** to `outDir`
+
+### Route Type Handling
+
+| Route Kind | Detected By | Params Source | Return Source | Callable |
+|---|---|---|---|---|
+| `rpc` | `kind: 'rpc'` | `jsonSchema.body` → single params type | `jsonSchema.response` | `client.call()` |
+| `api` | `kind: 'api'` | `jsonSchema.pathParams/query/body/headers` → structured params type | `jsonSchema.response` | `client.call()` |
+| `stream` | `kind: 'stream'` | `jsonSchema.params` → single params type | `jsonSchema.yieldType` + `jsonSchema.returnType` | `client.stream()` |
+
+For API routes with structured input, `client.call()` unpacks the params object:
+- `pathParams` → interpolated into URL path via `:param` replacement, values URI-encoded
+- `query` → serialized to query string
+- `body` → request body (JSON-serialized)
+- `headers` → merged into request headers
+
+### CLI
+
+```bash
+# From URL (most common)
+npx ts-procedures-codegen --url http://localhost:3000/docs --out ./src/generated/api
+
+# From file
+npx ts-procedures-codegen --file ./docs.json --out ./src/generated/api
+
+# With ajsc options
+npx ts-procedures-codegen --url http://localhost:3000/docs --out ./src/generated/api --enum-style union
+
+# Watch mode — polls /docs endpoint, skips write if output unchanged
+npx ts-procedures-codegen --url http://localhost:3000/docs --out ./src/generated/api --watch --interval 3000
+```
+
+The `--watch` flag polls at a configurable interval (default 3000ms). It hashes the DocEnvelope response and skips file writes when the hash matches the previous run.
+
+### Code-Gen Error Handling
+
+Fails with clear messages on:
+- Routes missing `scope` where `kind` is `'rpc'` or `'stream'`: "Route 'X' has no scope — this should not happen for RPC/Stream routes"
+- API routes missing `scope`: warning (falls back to `'default'`), not an error
+- Unreachable URL or malformed JSON
+- ajsc conversion failures: includes procedure name and the schema that failed
+- Empty DocEnvelope (no routes)
+
+## Changes to Existing ts-procedures
+
+### Add Optional `scope` to API Route Config
+
+API routes currently lack `scope`. Add it as **optional** to avoid a breaking change to all existing API builder consumers:
+
+```ts
+// In src/implementations/types.ts
+interface APIConfig {
+  path: string;
+  method: HttpMethod;
+  scope?: string;            // NEW — optional, used by codegen for file grouping
+  successStatus?: number;
+}
+```
+
+The `scope` flows through to `APIHttpRouteDoc`. Existing API builder consumers are unaffected. The codegen warns (not errors) when API routes lack scope, falling back to the `'default'` group.
+
+### Add `kind` Discriminant to Route Docs
+
+Add a `kind` field to each existing route doc interface. This is an additive change — existing fields are untouched:
+
+```ts
+// In src/implementations/types.ts — additive fields only
+
+interface RPCHttpRouteDoc extends RPCConfig {
+  kind: 'rpc';               // NEW
+  name: string;
+  path: string;
+  method: 'post';
+  jsonSchema: { /* unchanged */ };
+}
+
+interface APIHttpRouteDoc extends APIConfig {
+  kind: 'api';               // NEW
+  name: string;
+  fullPath: string;
+  jsonSchema: { /* unchanged */ };
+}
+
+interface StreamHttpRouteDoc extends RPCConfig {
+  kind: 'stream';            // NEW
+  name: string;
+  path: string;
+  methods: ('get' | 'post')[];
+  streamMode: StreamMode;
+  jsonSchema: { /* unchanged */ };
+}
+```
+
+Each builder sets `kind` when constructing its doc object. The `AnyHttpRouteDoc` union (`RPCHttpRouteDoc | APIHttpRouteDoc | StreamHttpRouteDoc`) can now be narrowed via `kind`.
+
+**Note:** `StreamHttpRouteDoc` is currently defined in two places — `src/implementations/types.ts` (line 78) and `src/implementations/http/hono-stream/types.ts` (line 7). The `kind` field must be added to both. The implementer should consider consolidating to a single source (the canonical type in `types.ts`, re-exported by the hono-stream module).
+
+### Scope Type Consideration
+
+`RPCConfig.scope` is typed `string | string[]`. The array form represents multi-segment scope paths (e.g., `['admin', 'users']` produces path `/admin/users/ProcName/1`). The codegen normalizes this to a string key for file grouping (joined with `-`). No changes to `RPCConfig` are needed — the normalization happens entirely in the codegen pipeline.
+
+### New Source Structure
+
+```
+src/
+  client/
+    index.ts                 # createClient, ClientInstance
+    types.ts                 # ClientAdapter, hooks, AdapterRequest/Response, TypedStream
+    call.ts                  # request execution + hook pipeline
+    stream.ts                # stream execution + SSE parsing
+    errors.ts                # client-side error classes
+  codegen/
+    index.ts                 # generateClient() public API
+    pipeline.ts              # orchestrates generation steps
+    resolve-envelope.ts      # fetch URL / read file / accept object
+    group-routes.ts          # normalize scopes + group routes
+    emit-scope.ts            # generate a single scope file
+    emit-index.ts            # generate the barrel index
+    emit-types.ts            # ajsc integration
+    bin/
+      cli.ts                 # CLI entry point
+```
+
+### New Package Exports
+
+Following the existing dual `types`/`import` pattern:
+
+```json
+{
+  "exports": {
+    "./client": {
+      "types": "./build/client/index.d.ts",
+      "import": "./build/client/index.js"
+    },
+    "./codegen": {
+      "types": "./build/codegen/index.d.ts",
+      "import": "./build/codegen/index.js"
+    }
+  }
+}
+```
+
+A new `bin` entry for the CLI:
+
+```json
+{
+  "bin": {
+    "ts-procedures-setup": "./agent_config/bin/setup.mjs",
+    "ts-procedures-codegen": "./build/codegen/bin/cli.js"
+  }
+}
+```
+
+### New Dependency
+
+- **ajsc** — added as an `optionalDependency` of ts-procedures (following the existing pattern used for `express` and `hono`). The codegen export uses a dynamic `import('ajsc')` so the dependency is only required when the codegen is actually invoked. Consumers who never run codegen do not need ajsc installed.
+
+### Impact on Existing Code
+
+**Additive changes only — no breaking changes:**
+
+- `APIConfig` gains an optional `scope` field (non-breaking)
+- `RPCHttpRouteDoc`, `APIHttpRouteDoc`, `StreamHttpRouteDoc` each gain a `kind` field (additive)
+- Each builder (hono-rpc, hono-api, hono-stream, express-rpc) must be updated to set `kind` when constructing route docs
+- The `./http` types-only export (`build/implementations/http/types.d.ts`) will include the new `kind` fields on the union members — consumers who type-narrow `AnyHttpRouteDoc` may benefit from the new discriminant but are not required to use it
+- Existing tests continue to pass; new tests are added for the `kind` field and `scope` on API routes
+
+## Decision Summary
+
+| Decision | Choice |
+|---|---|
+| Trigger | CLI + programmatic API |
+| File grouping | Intentional scope on all routes, array scopes joined with `-` |
+| Route coverage | RPC + API + Stream from v1 |
+| Adapter | Two-method: `request()` + `stream()` |
+| Hooks | Global + per-procedure overrides |
+| Hook timing (streams) | `onAfterResponse` fires on initial HTTP response; `onError` fires on mid-stream errors |
+| Error strategy | Hooks decide, default throw, callables return `Promise<T>` |
+| Runtime location | `ts-procedures/client` export |
+| Codegen approach | Full static — types + callables in generated .ts files |
+| Type generation | ajsc `TypescriptConverter` (optionalDependency, dynamic import) |
+| Route discrimination | Explicit `kind` field on all route docs (additive) |
+| Yield types | Data payload only — SSE envelope stripped by runtime |
+| Stream completion | `TypedStream.result` promise; `event: 'return'` for SSE final value |
+| API scope | Optional on `APIConfig`, defaults to `'default'` in codegen with warning |
+| Breaking changes | None — all changes are additive to existing interfaces |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ts-procedures",
-  "version": "5.7.2",
+  "version": "5.9.0",
   "description": "A TypeScript RPC framework that creates type-safe, schema-validated procedure calls with a single function definition. Define your procedures once and get full type inference, runtime validation, and framework integration hooks.",
   "main": "build/exports.js",
   "types": "build/exports.d.ts",
@@ -59,7 +59,12 @@
   "files": [
     "assets",
     "build",
+    "docs",
     "agent_config",
+    "src/implementations/http/README.md",
+    "src/implementations/http/express-rpc/README.md",
+    "src/implementations/http/hono-rpc/README.md",
+    "src/implementations/http/hono-stream/README.md",
     "src/client/types.ts",
     "src/client/errors.ts",
     "src/client/request-builder.ts",