@db-lyon/flowkit 0.13.0 → 0.15.0

package/docs/custom-tasks.md

@@ -1,258 +1,271 @@
-# Custom tasks
-
-Tasks are the building blocks of flowkit. Each task is a class that extends `BaseTask` and implements an `execute()` method.
-
-## Anatomy of a task
-
-```typescript
-import { BaseTask, type TaskResult } from '@db-lyon/flowkit';
-
-interface MyOptions {
-  url: string;
-  retries?: number;
-}
-
-export default class FetchData extends BaseTask<MyOptions> {
-  get taskName() {
-    return 'fetch_data';
-  }
-
-  protected validate() {
-    if (!this.options.url) {
-      throw new Error('url option is required');
-    }
-  }
-
-  async execute(): Promise<TaskResult> {
-    const { url, retries = 3 } = this.options;
-
-    const response = await fetch(url);
-    if (!response.ok) {
-      return {
-        success: false,
-        error: new Error(`HTTP ${response.status}`),
-      };
-    }
-
-    const data = await response.json();
-    return {
-      success: true,
-      data: { body: data, status: response.status },
-    };
-  }
-}
-```
-
-### Required members
-
-| Member | Description |
-|--------|-------------|
-| `get taskName()` | A human-readable name used in logging |
-| `execute()` | Async method that performs the work and returns a `TaskResult` |
-
-### Optional members
-
-| Member | Description |
-|--------|-------------|
-| `validate()` | Called before `execute()`. Throw to abort with a validation error. |
-
-### Available on `this`
-
-| Property | Description |
-|----------|-------------|
-| `this.options` | The merged options (task defaults + step overrides), typed as `TOptions` |
-| `this.ctx` | The `TaskContext` passed to the flow runner — read-only, see below |
-| `this.logger` | A child logger scoped to this task instance |
-| `this.resolve(name, options?)` | Build another task by configured name or class path, unexecuted |
-| `this.call(name, options?)` | `resolve()` plus `run()`, returning its `TaskResult` |
-
-## Calling other tasks
-
-`this.call(name)` resolves `name` the same way a flow step does. A configured task name is looked up in the `tasks:` config and dispatched through its `class_path`, inheriting its configured `options` as defaults; anything you pass as `options` merges over them and wins. A name with no configured entry resolves as a class path directly, so a bare `vendor.tasks.Thing` still works.
-
-```yaml
-tasks:
-  soql_query:
-    class_path: caseops.tasks.SoqlQuery
-    options:
-      org: ${org.username}
-```
-
-```typescript
-// Runs caseops.tasks.SoqlQuery with { org: 'admin@example.com', query: 'SELECT ...' }
-const result = await this.call('soql_query', { query: 'SELECT Id FROM Case' });
-```
-
-The `${ns.path}` references in those configured defaults are interpolated for you, against the same scope the calling task itself runs under. The `options` you pass are your own runtime data and are **never** interpolated, so a `${...}` you computed reaches the task verbatim rather than being reinterpreted as configuration.
-
-Calling a task requires a registry on the context, which `FlowRunner` supplies. A task constructed by hand without one throws.
-
-## The task lifecycle
-
-When `task.run()` is called (by the flow runner):
-
-1. `validate()` runs — throw here to reject bad options
-2. `execute()` runs — return a `TaskResult`
-3. The result gets a `duration` field added automatically
-4. If `validate()` or `execute()` throws, the error is caught and returned as `{ success: false, error }`
-
-You never call `run()` yourself in normal usage — the flow runner handles it.
-
-## TaskResult
-
-```typescript
-interface TaskResult {
-  success: boolean;
-  data?: Record<string, unknown>;  // arbitrary output data
-  error?: Error;                    // populated on failure
-  duration?: number;                // milliseconds, set by run()
-}
-```
-
-Return `{ success: true }` for success and `{ success: false, error }` for expected failures. Unexpected exceptions are caught automatically.
-
-## TaskContext
-
-The context carries host-supplied state to every task in a flow run — database connections, API clients, configuration:
-
-```typescript
-const runner = new FlowRunner({
-  // ...
-  context: {
-    logger: myLogger,
-    db: databaseConnection,
-    apiKey: process.env.API_KEY,
-  },
-});
-```
-
-Inside a task:
-
-```typescript
-async execute(): Promise<TaskResult> {
-  const db = this.ctx.db as Database;
-  // ...
-}
-```
-
-Treat the context as read-only. Each task is handed its own derived context, so assigning a key inside a task (`this.ctx.cached = x`) does not reach the next step, another task, or a sub-agent. To share mutable state, put a mutable object on the context up front and write into that:
-
-```typescript
-context: { cache: new Map() }   // this.ctx.cache.set(...) is visible everywhere
-```
-
-## Registering tasks
-
-### By name
-
-```typescript
-const registry = new TaskRegistry();
-registry.register('fetch_data', FetchData as any);
-```
-
-The YAML can then reference it directly:
-
-```yaml
-tasks:
-  fetch_data:
-    class_path: fetch_data
-```
-
-### By class path
-
-```typescript
-registry.registerClassPath('my.tasks.FetchData', FetchData as any);
-```
-
-### Bulk registration
-
-```typescript
-registry.registerAll({
-  fetch_data: FetchData as any,
-  transform: TransformData as any,
-  upload: Upload as any,
-});
-```
-
-### Dynamic resolution
-
-If a `class_path` isn't found in the registry, flowkit converts dots to path separators and looks for a file on disk:
-
-| class_path | Files checked |
-|------------|---------------|
-| `tasks.FetchData` | `tasks/FetchData.ts`, `tasks/FetchData.js`, `tasks/FetchData/index.ts`, `tasks/FetchData/index.js` |
-| `lib.etl.Extract` | `lib/etl/Extract.ts`, `lib/etl/Extract.js`, ... |
-
-The module must have either a `default` export or a named export matching the last segment of the path (e.g., `FetchData`). The export must extend `BaseTask`.
-
-## Built-in: ShellTask
-
-`ShellTask` executes shell commands through the platform shell and streams output.
-Register it under any name you like:
-
-```typescript
-import { ShellTask } from '@db-lyon/flowkit';
-
-registry.register('shell', ShellTask as any);
-```
-
-Then use it in YAML:
-
-```yaml
-tasks:
-  lint:
-    class_path: shell
-    description: Run the linter
-    options:
-      command: npm run lint
-
-  build:
-    class_path: shell
-    description: Build the project
-    options:
-      command: npm run build
-      cwd: /path/to/project
-      timeout: 120000
-```
-
-### ShellTask options
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `command` | `string` | (required) | The shell command to execute |
-| `cwd` | `string` | `undefined` | Working directory |
-| `timeout` | `number` | `300000` (5 min) | Timeout in milliseconds |
-| `signal` | `AbortSignal` | `undefined` | Cancels one programmatic invocation; cannot be specified in YAML |
-
-On success, `result.data.output` contains the trimmed stdout. On failure, `result.data` includes `exitCode`, `stderr`, and `stdout`.
-
-Existing YAML consumers need no change. Programmatic callers that need
-cancellation pass an invocation-specific `AbortSignal` in the task options.
-After cancellation, Flowkit waits for the shell to close, with a bounded
-fallback if the operating system does not report closure (one second on POSIX,
-three seconds on Windows). That fallback does not guarantee all descendants
-have exited. At the deadline Flowkit requests force termination and releases
-its Node handles for the root shell and any Windows `taskkill` helper before
-returning. On POSIX,
-signal-bearing invocations use a dedicated process group: Flowkit requests
-`SIGTERM`, waits 250ms for cooperative cleanup, then escalates the group to
-`SIGKILL`. Terminal Ctrl+C is not delivered to that separate group, so use the
-supplied `AbortSignal` for cancellation.
-
-On Windows, Flowkit asks `taskkill /T /F` to terminate the shell tree and does
-not kill the shell while that traversal is in progress. If `taskkill` fails or
-the three-second deadline expires, Flowkit requests force termination and
-releases its Node handles for the root and helper. Windows and POSIX descendants that escape the managed process
-tree or process group may still survive; Node does not provide a portable
-guarantee of complete descendant termination.
-
-Trailing stdout and stderr fragments are captured once, including when no
-`signal` is supplied. This corrects the duplicate final-partial-line output in
-earlier releases.
-
-## Listing registered tasks
-
-```typescript
-const names = registry.listRegistered();
-// ['fetch_data', 'shell', 'my.tasks.Transform', ...]
-```
+# Custom tasks
+
+Tasks are the building blocks of flowkit. Each task is a class that extends `BaseTask` and implements an `execute()` method.
+
+## Anatomy of a task
+
+```typescript
+import { BaseTask, type TaskResult } from '@db-lyon/flowkit';
+
+interface MyOptions {
+  url: string;
+  retries?: number;
+}
+
+export default class FetchData extends BaseTask<MyOptions> {
+  get taskName() {
+    return 'fetch_data';
+  }
+
+  protected validate() {
+    if (!this.options.url) {
+      throw new Error('url option is required');
+    }
+  }
+
+  async execute(): Promise<TaskResult> {
+    const { url, retries = 3 } = this.options;
+
+    const response = await fetch(url);
+    if (!response.ok) {
+      return {
+        success: false,
+        error: new Error(`HTTP ${response.status}`),
+      };
+    }
+
+    const data = await response.json();
+    return {
+      success: true,
+      data: { body: data, status: response.status },
+    };
+  }
+}
+```
+
+### Required members
+
+| Member | Description |
+|--------|-------------|
+| `get taskName()` | A human-readable name used in logging |
+| `execute()` | Async method that performs the work and returns a `TaskResult` |
+
+### Optional members
+
+| Member | Description |
+|--------|-------------|
+| `validate()` | Called before `execute()`. Throw to abort with a validation error. |
+
+### Available on `this`
+
+| Property | Description |
+|----------|-------------|
+| `this.options` | The merged options (task defaults + step overrides), typed as `TOptions` |
+| `this.ctx` | The `TaskContext` passed to the flow runner — read-only, see below |
+| `this.logger` | A child logger scoped to this task instance |
+| `this.resolve(name, options?)` | Build another task by configured name or class path, unexecuted |
+| `this.call(name, options?)` | `resolve()` plus `run()`, returning its `TaskResult` |
+
+## Calling other tasks
+
+`this.call(name)` resolves `name` the same way a flow step does. A configured task name is looked up in the `tasks:` config and dispatched through its `class_path`, inheriting its configured `options` as defaults; anything you pass as `options` merges over them and wins. A name with no configured entry resolves as a class path directly, so a bare `vendor.tasks.Thing` still works.
+
+```yaml
+tasks:
+  soql_query:
+    class_path: caseops.tasks.SoqlQuery
+    options:
+      org: ${org.username}
+```
+
+```typescript
+// Runs caseops.tasks.SoqlQuery with { org: 'admin@example.com', query: 'SELECT ...' }
+const result = await this.call('soql_query', { query: 'SELECT Id FROM Case' });
+```
+
+The `${ns.path}` references in those configured defaults are interpolated for you, against the same scope the calling task itself runs under. The `options` you pass are your own runtime data and are **never** interpolated, so a `${...}` you computed reaches the task verbatim rather than being reinterpreted as configuration.
+
+Calling a task requires a registry on the context, which `FlowRunner` supplies. A task constructed by hand without one throws.
+
+## The task lifecycle
+
+When `task.run()` is called (by the flow runner):
+
+1. `validate()` runs — throw here to reject bad options
+2. `execute()` runs — return a `TaskResult`
+3. The result gets a `duration` field added automatically
+4. If `validate()` or `execute()` throws, the error is caught and returned as `{ success: false, error }`
+
+You never call `run()` yourself in normal usage — the flow runner handles it.
+
+## TaskResult
+
+```typescript
+interface TaskResult {
+  success: boolean;
+  data?: Record<string, unknown>;  // arbitrary output data
+  error?: Error;                    // populated on failure
+  duration?: number;                // milliseconds, set by run()
+}
+```
+
+Return `{ success: true }` for success and `{ success: false, error }` for expected failures. Unexpected exceptions are caught automatically.
+
+## TaskContext
+
+The context carries host-supplied state to every task in a flow run — database connections, API clients, configuration:
+
+```typescript
+const runner = new FlowRunner({
+  // ...
+  context: {
+    logger: myLogger,
+    db: databaseConnection,
+    apiKey: process.env.API_KEY,
+  },
+});
+```
+
+Inside a task:
+
+```typescript
+async execute(): Promise<TaskResult> {
+  const db = this.ctx.db as Database;
+  if (this.executionPhase === 'rollback') {
+    // This invocation is compensating for earlier successful work.
+  }
+  // ...
+}
+```
+
+`this.executionPhase` is a public, read-only lifecycle value:
+`'task' | 'on_start' | 'on_success' | 'on_failure' | 'finally' | 'rollback'`.
+Ordinary steps (including nested-flow steps), direct `runTask` calls,
+task-to-task calls, and agent/tool work receive `'task'`. Hook tasks receive
+their hook phase, and rollback-record invocations receive `'rollback'`.
+Flowkit supplies the value; existing runner configurations do not need to add
+it to `context`. `ctx.executionPhase` holds the same value but is typed
+optional, because `TaskContext` is also the shape hosts build their own context
+interfaces from; prefer the `this.executionPhase` accessor.
+
+Treat the context as read-only. Each task is handed its own derived context, so assigning a key inside a task (`this.ctx.cached = x`) does not reach the next step, another task, or a sub-agent. To share mutable state, put a mutable object on the context up front and write into that:
+
+```typescript
+context: { cache: new Map() }   // this.ctx.cache.set(...) is visible everywhere
+```
+
+## Registering tasks
+
+### By name
+
+```typescript
+const registry = new TaskRegistry();
+registry.register('fetch_data', FetchData as any);
+```
+
+The YAML can then reference it directly:
+
+```yaml
+tasks:
+  fetch_data:
+    class_path: fetch_data
+```
+
+### By class path
+
+```typescript
+registry.registerClassPath('my.tasks.FetchData', FetchData as any);
+```
+
+### Bulk registration
+
+```typescript
+registry.registerAll({
+  fetch_data: FetchData as any,
+  transform: TransformData as any,
+  upload: Upload as any,
+});
+```
+
+### Dynamic resolution
+
+If a `class_path` isn't found in the registry, flowkit converts dots to path separators and looks for a file on disk:
+
+| class_path | Files checked |
+|------------|---------------|
+| `tasks.FetchData` | `tasks/FetchData.ts`, `tasks/FetchData.js`, `tasks/FetchData/index.ts`, `tasks/FetchData/index.js` |
+| `lib.etl.Extract` | `lib/etl/Extract.ts`, `lib/etl/Extract.js`, ... |
+
+The module must have either a `default` export or a named export matching the last segment of the path (e.g., `FetchData`). The export must extend `BaseTask`.
+
+## Built-in: ShellTask
+
+`ShellTask` executes shell commands through the platform shell and streams output.
+Register it under any name you like:
+
+```typescript
+import { ShellTask } from '@db-lyon/flowkit';
+
+registry.register('shell', ShellTask as any);
+```
+
+Then use it in YAML:
+
+```yaml
+tasks:
+  lint:
+    class_path: shell
+    description: Run the linter
+    options:
+      command: npm run lint
+
+  build:
+    class_path: shell
+    description: Build the project
+    options:
+      command: npm run build
+      cwd: /path/to/project
+      timeout: 120000
+```
+
+### ShellTask options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `command` | `string` | (required) | The shell command to execute |
+| `cwd` | `string` | `undefined` | Working directory |
+| `timeout` | `number` | `300000` (5 min) | Timeout in milliseconds |
+| `signal` | `AbortSignal` | `undefined` | Cancels one programmatic invocation; cannot be specified in YAML |
+
+On success, `result.data.output` contains the trimmed stdout. On failure, `result.data` includes `exitCode`, `stderr`, and `stdout`.
+
+Existing YAML consumers need no change. Programmatic callers that need
+cancellation pass an invocation-specific `AbortSignal` in the task options.
+After cancellation, Flowkit waits for the shell to close, with a bounded
+fallback if the operating system does not report closure (one second on POSIX,
+three seconds on Windows). That fallback does not guarantee all descendants
+have exited. At the deadline Flowkit requests force termination and releases
+its Node handles for the root shell and any Windows `taskkill` helper before
+returning. On POSIX,
+signal-bearing invocations use a dedicated process group: Flowkit requests
+`SIGTERM`, waits 250ms for cooperative cleanup, then escalates the group to
+`SIGKILL`. Terminal Ctrl+C is not delivered to that separate group, so use the
+supplied `AbortSignal` for cancellation.
+
+On Windows, Flowkit asks `taskkill /T /F` to terminate the shell tree and does
+not kill the shell while that traversal is in progress. If `taskkill` fails or
+the three-second deadline expires, Flowkit requests force termination and
+releases its Node handles for the root and helper. Windows and POSIX descendants that escape the managed process
+tree or process group may still survive; Node does not provide a portable
+guarantee of complete descendant termination.
+
+Trailing stdout and stderr fragments are captured once, including when no
+`signal` is supplied. This corrects the duplicate final-partial-line output in
+earlier releases.
+
+## Listing registered tasks
+
+```typescript
+const names = registry.listRegistered();
+// ['fetch_data', 'shell', 'my.tasks.Transform', ...]
+```

package/docs/guards.md

@@ -0,0 +1,133 @@
+# Guards
+
+A guard is a `before`/`after` pipeline around one host operation. It sits on a seam the host already has (an RPC call, a write, a command dispatch) and may veto that operation, act on it, or observe its result.
+
+The pipeline knows nothing about what any guard does. Access policy, source control checkout, audit logging, rate limiting and approval gating are all just guards.
+
+Guards are distinct from [`FlowRunnerHooks`](api-reference.md#flow), which fire around flow steps. A hook observes a step; a guard wraps one host operation and can deny it.
+
+```ts
+import { GuardRegistry, runGuarded, guardContextBase, lazy } from '@db-lyon/flowkit/guard';
+```
+
+## The context
+
+Flowkit requires only a scratch map. Everything else belongs to the host.
+
+```ts
+interface GuardContext {
+  readonly meta: Map<string, unknown>;
+}
+```
+
+Extend it with whatever your operation carries, and build it with `guardContextBase()`:
+
+```ts
+interface CallContext extends GuardContext {
+  readonly method: string;
+  readonly params: Record<string, unknown>;
+  /** Files this call will modify. Computed on demand. */
+  files(): string[];
+}
+
+function makeCallContext(method: string, params: Record<string, unknown>): CallContext {
+  const ctx = { ...guardContextBase(), method, params } as CallContext;
+  ctx.files = lazy(ctx, 'files', () => classify(method, params));
+  return ctx;
+}
+```
+
+`lazy(ctx, key, compute)` caches into `meta`, so enrichment that is expensive to compute and that most guards never consult costs nothing when it is ignored and is computed once when several guards want it.
+
+`meta` is also how guards talk to each other. A `before` hook can stash what it did and the matching `after` hook, or a later guard, can read it back.
+
+## Writing a guard
+
+```ts
+interface Guard<Ctx extends GuardContext, TResult> {
+  readonly name: string;
+  readonly order?: number;
+  appliesTo?(ctx: Ctx): boolean | Promise<boolean>;
+  before?(ctx: Ctx): Promise<void>;
+  after?(ctx: Ctx, result: TResult): Promise<TResult | void>;
+}
+```
+
+- **`before`** runs before the operation. Throw to deny it: the operation never happens and your error propagates unchanged, so the host's own error type survives.
+- **`after`** runs after a successful operation. Return a value to replace the result; return nothing to leave it alone.
+- **`appliesTo`** decides whether the guard participates at all. Default is always.
+- **`order`** sorts the chain, lower first. Ties break by name, so a registry built from an unordered source still runs deterministically.
+
+```ts
+const sourceControl: Guard<CallContext, unknown> = {
+  name: 'source-control',
+  order: 10,
+  appliesTo: (ctx) => ctx.files().length > 0,
+  before: async (ctx) => {
+    const denied = await checkout(ctx.files());
+    if (denied.length) throw new Error(`locked by another user: ${denied.join(', ')}`);
+  },
+};
+```
+
+## Running the pipeline
+
+```ts
+const guards = new GuardRegistry<CallContext, Result>().registerAll([sourceControl, audit]);
+
+async function call(method: string, params: Record<string, unknown>): Promise<Result> {
+  return runGuarded(makeCallContext(method, params), guards, () => transport.send(method, params));
+}
+```
+
+`before` hooks run in registration order, then `invoke`, then `after` hooks in reverse order, so a guard's two halves nest rather than interleave.
+
+Applicability resolves once, up front. A guard whose `before` changes the answer to its own `appliesTo` (a source-control guard that checks a file out, making it writable) still gets its `after` half.
+
+With an empty registry `runGuarded` is exactly `invoke()`, so it is safe to install on a seam before any guard exists.
+
+## Guards from tasks
+
+A host that already loads tasks from config or from plugins gets guards for free. Name a task `guard.<name>.<phase>` and `discoverTaskGuards` turns it into a guard. No separate activation concept is needed, because the task registry is already the list of everything the host was given.
+
+```yaml
+tasks:
+  guard.p4.beforeWrite:
+    class_path: ./guards/perforce.js
+  guard.audit.after:
+    class_path: ./guards/audit.js
+```
+
+`<phase>` is `before` or `after`, optionally suffixed with a scope the host declared:
+
+| Task name | Runs |
+| --- | --- |
+| `guard.audit.before` | before every operation |
+| `guard.audit.after` | after every successful operation |
+| `guard.p4.beforeWrite` | before an operation the `write` scope claims |
+
+```ts
+const guards = discoverTaskGuards<CallContext, Result>(taskRegistry, {
+  scopes: { write: (ctx) => ctx.files().length > 0 },
+  contextFor: (ctx) => ({ logger, registry: taskRegistry, transport: ctx.transport }),
+  optionsFor: (ctx, result) => ({
+    method: ctx.method,
+    params: ctx.params,
+    paths: ctx.files(),
+    ...(result !== undefined ? { result } : {}),
+  }),
+  onDeny: (info) => new PolicyError(`blocked (${info.ctx.method}): ${info.reason}`),
+});
+```
+
+`contextFor` is called per operation, so a guard can be bound to whatever the operation belongs to rather than to a single ambient target. A host driving several connections should hand each guard the one serving the call it is guarding.
+
+A scope named by a task but not registered by the host is an error at discovery time. A typo in a plugin's task name surfaces at startup rather than becoming a guard that silently runs on everything.
+
+### Denial and failure
+
+A `before` guard denies by returning `success: false` or by throwing. Both route to `onDeny`, because `BaseTask.run` turns an exception into a failed result: a guard that crashes denies the operation rather than waving it through, which is the safe direction for the thing standing between a caller and a mutation.
+
+`onError` is separate and narrower. It fires only when the guard task cannot be constructed at all, an unresolvable class path or a module that fails to import, where nothing about the operation was evaluated.
+
+An `after` guard observes the result and cannot replace it, since a task returns a `TaskResult` rather than the host's result type. A failure is reported through `onAfterFailure` instead of failing an operation that already happened. Reach for a hand-written `Guard` when you need an `after` hook that transforms the result.