@pgflow/core 0.0.0-array-map-steps-302d00a8-20250922101336 → 0.0.0-test-snapshot-releases-8d5d9bc1-20250922101013

package/dist/PLAN_race_condition_testing.md

@@ -1,176 +0,0 @@
-# PLAN: Race Condition Testing for Type Violations
-
-## Background
-
-When a type violation occurs (e.g., single step produces non-array for dependent map), the system must archive ALL active messages to prevent orphaned messages that cycle through workers indefinitely.
-
-## Current Issue
-
-The fix archives both `'queued'` AND `'started'` tasks, but existing tests don't properly validate the race condition scenarios.
-
-## Test Scenarios Needed
-
-### 1. Basic Type Violation (✅ Already Covered)
-**Scenario**: Single task causes type violation
-```
-step1 (single) → step2 (single) → map_step
-```
-- Worker completes step2 with non-array
-- Verify run fails and current task's message is archived
-- **Coverage**: `non_array_to_map_should_fail.test.sql`
-
-### 2. Concurrent Started Tasks (❌ Not Covered)
-**Scenario**: Multiple workers have tasks in 'started' state when violation occurs
-```
-producer (single) → map_consumer (map, expects array)
-producer (single) → parallel_task1 (single)
-producer (single) → parallel_task2 (single)
-```
-
-**Test Flow**:
-1. Complete producer with `[1, 2, 3]` (spawns 3 map tasks + 2 parallel tasks)
-2. Worker A starts `map_consumer[0]`
-3. Worker B starts `map_consumer[1]`
-4. Worker C starts `parallel_task1`
-5. Worker D starts `parallel_task2`
-6. Worker C completes `parallel_task1` with non-array (violates some other map dependency)
-7. **Verify**: ALL started tasks (map_consumer[0], map_consumer[1], parallel_task2) get archived
-
-### 3. Mixed Queue States (❌ Not Covered)
-**Scenario**: Mix of queued and started tasks across different steps
-```
-step1 → step2 → step3 → map_step
-     ↘ step4 → step5
-```
-
-**Test Flow**:
-1. Complete step1
-2. Worker A starts step2
-3. Worker B starts step4
-4. Step3 and step5 remain queued
-5. Worker A completes step2 with type violation
-6. **Verify**: Both started (step4) AND queued (step3, step5) messages archived
-
-### 4. Map Task Partial Processing (❌ Not Covered)
-**Scenario**: Some map tasks started, others queued when violation occurs
-```
-producer → large_map (100 elements)
-```
-
-**Test Flow**:
-1. Producer outputs array of 100 elements
-2. Workers start processing first 10 tasks
-3. 90 tasks remain queued
-4. One of the started tasks detects downstream type violation
-5. **Verify**: All 100 messages (10 started + 90 queued) get archived
-
-### 5. Visibility Timeout Verification (❌ Not Covered)
-**Scenario**: Ensure orphaned messages don't reappear after timeout
-```
-step1 → step2 → map_step
-```
-
-**Test Flow**:
-1. Worker starts step2 (30s visibility timeout)
-2. Type violation occurs but message NOT archived (simulate bug)
-3. Wait 31 seconds
-4. **Verify**: Message reappears in queue (demonstrates the bug)
-5. Apply fix and verify message doesn't reappear
-
-### 6. Nested Map Chains (❌ Not Covered)
-**Scenario**: Type violation in middle of map chain
-```
-map1 (produces arrays) → map2 (expects arrays) → map3
-```
-
-**Test Flow**:
-1. map1 task completes with non-array (violates map2 expectation)
-2. Other map1 tasks are in various states (started/queued)
-3. **Verify**: All map1 tasks archived, map2 never starts
-
-### 7. Race During Archival (❌ Not Covered)
-**Scenario**: Worker tries to complete task while archival is happening
-```
-step1 → step2 → map_step
-```
-
-**Test Flow**:
-1. Worker A detects type violation, begins archiving
-2. Worker B tries to complete its task during archival
-3. **Verify**: Worker B's completion is rejected (guard clause)
-4. **Verify**: No duplicate archival attempts
-
-## Implementation Strategy
-
-### Test Utilities Needed
-
-1. **Multi-worker simulator**:
-```sql
-CREATE FUNCTION pgflow_tests.simulate_worker(
-  worker_id uuid,
-  flow_slug text
-) RETURNS TABLE(...);
-```
-
-2. **Queue state inspector**:
-```sql
-CREATE FUNCTION pgflow_tests.inspect_queue_state(
-  flow_slug text
-) RETURNS TABLE(
-  message_id bigint,
-  task_status text,
-  visibility_timeout timestamptz
-);
-```
-
-3. **Time manipulation** (for visibility timeout tests):
-```sql
--- May need to mock pgmq visibility behavior
-```
-
-### Test File Organization
-
-```
-supabase/tests/type_violations/
-├── basic_violation.test.sql                 # Existing coverage
-├── concurrent_started_tasks.test.sql        # NEW: Scenario 2
-├── mixed_queue_states.test.sql              # NEW: Scenario 3
-├── map_partial_processing.test.sql          # NEW: Scenario 4
-├── visibility_timeout_recovery.test.sql     # NEW: Scenario 5
-├── nested_map_chains.test.sql               # NEW: Scenario 6
-└── race_during_archival.test.sql            # NEW: Scenario 7
-```
-
-## Success Criteria
-
-1. **No orphaned messages**: Queue must be empty after type violation
-2. **No message resurrection**: Archived messages don't reappear after timeout
-3. **Complete cleanup**: ALL tasks (queued + started) for the run are handled
-4. **Atomic operation**: Archival happens in single transaction
-5. **Guard effectiveness**: No operations on failed runs
-
-## Performance Considerations
-
-- Test with large numbers of tasks (1000+) to verify batch archival performance
-- Ensure archival doesn't lock tables for extended periods
-- Verify index usage on `(run_id, status, message_id)`
-
-## Current Gap Analysis
-
-**What we have**:
-- Basic type violation detection ✅
-- Single task archival ✅
-- Run failure on violation ✅
-
-**What we need**:
-- True concurrent worker simulation ❌
-- Multi-task race condition validation ❌
-- Visibility timeout verification ❌
-- Performance under load testing ❌
-
-## Priority
-
-1. **HIGH**: Concurrent started tasks (Scenario 2) - Most common real-world case
-2. **HIGH**: Map partial processing (Scenario 4) - Critical for large arrays
-3. **MEDIUM**: Mixed queue states (Scenario 3) - Complex flows
-4. **LOW**: Other scenarios - Edge cases but important for robustness

package/dist/PgflowSqlClient.d.ts

@@ -1,17 +0,0 @@
-import type postgres from 'postgres';
-import type { StepTaskRecord, IPgflowClient, StepTaskKey, RunRow, MessageRecord } from './types.js';
-import type { Json } from './types.js';
-import type { AnyFlow, ExtractFlowInput } from '@pgflow/dsl';
-/**
- * Implementation of IPgflowClient that uses direct SQL calls to pgflow functions
- */
-export declare class PgflowSqlClient<TFlow extends AnyFlow> implements IPgflowClient<TFlow> {
-    private readonly sql;
-    constructor(sql: postgres.Sql);
-    readMessages(queueName: string, visibilityTimeout: number, batchSize: number, maxPollSeconds?: number, pollIntervalMs?: number): Promise<MessageRecord[]>;
-    startTasks(flowSlug: string, msgIds: number[], workerId: string): Promise<StepTaskRecord<TFlow>[]>;
-    completeTask(stepTask: StepTaskKey, output?: Json): Promise<void>;
-    failTask(stepTask: StepTaskKey, error: unknown): Promise<void>;
-    startFlow<TFlow extends AnyFlow>(flow_slug: string, input: ExtractFlowInput<TFlow>, run_id?: string): Promise<RunRow>;
-}
-//# sourceMappingURL=PgflowSqlClient.d.ts.map

package/dist/PgflowSqlClient.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"PgflowSqlClient.d.ts","sourceRoot":"","sources":["../src/PgflowSqlClient.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,QAAQ,MAAM,UAAU,CAAC;AACrC,OAAO,KAAK,EACV,cAAc,EACd,aAAa,EACb,WAAW,EACX,MAAM,EACN,aAAa,EACd,MAAM,YAAY,CAAC;AACpB,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,YAAY,CAAC;AACvC,OAAO,KAAK,EAAE,OAAO,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAE7D;;GAEG;AACH,qBAAa,eAAe,CAAC,KAAK,SAAS,OAAO,CAChD,YAAW,aAAa,CAAC,KAAK,CAAC;IAEnB,OAAO,CAAC,QAAQ,CAAC,GAAG;gBAAH,GAAG,EAAE,QAAQ,CAAC,GAAG;IAExC,YAAY,CAChB,SAAS,EAAE,MAAM,EACjB,iBAAiB,EAAE,MAAM,EACzB,SAAS,EAAE,MAAM,EACjB,cAAc,SAAI,EAClB,cAAc,SAAM,GACnB,OAAO,CAAC,aAAa,EAAE,CAAC;IAarB,UAAU,CACd,QAAQ,EAAE,MAAM,EAChB,MAAM,EAAE,MAAM,EAAE,EAChB,QAAQ,EAAE,MAAM,GACf,OAAO,CAAC,cAAc,CAAC,KAAK,CAAC,EAAE,CAAC;IAW7B,YAAY,CAAC,QAAQ,EAAE,WAAW,EAAE,MAAM,CAAC,EAAE,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;IAWjE,QAAQ,CAAC,QAAQ,EAAE,WAAW,EAAE,KAAK,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC;IAkB9D,SAAS,CAAC,KAAK,SAAS,OAAO,EACnC,SAAS,EAAE,MAAM,EACjB,KAAK,EAAE,gBAAgB,CAAC,KAAK,CAAC,EAC9B,MAAM,CAAC,EAAE,MAAM,GACd,OAAO,CAAC,MAAM,CAAC;CAiBnB"}

package/dist/PgflowSqlClient.js

@@ -1,70 +0,0 @@
-/**
- * Implementation of IPgflowClient that uses direct SQL calls to pgflow functions
- */
-export class PgflowSqlClient {
-    sql;
-    constructor(sql) {
-        this.sql = sql;
-    }
-    async readMessages(queueName, visibilityTimeout, batchSize, maxPollSeconds = 5, pollIntervalMs = 200) {
-        return await this.sql `
-      SELECT *
-      FROM pgflow.read_with_poll(
-        queue_name => ${queueName},
-        vt => ${visibilityTimeout},
-        qty => ${batchSize},
-        max_poll_seconds => ${maxPollSeconds},
-        poll_interval_ms => ${pollIntervalMs}
-      );
-    `;
-    }
-    async startTasks(flowSlug, msgIds, workerId) {
-        return await this.sql `
-      SELECT *
-      FROM pgflow.start_tasks(
-        flow_slug => ${flowSlug},
-        msg_ids => ${msgIds}::bigint[],
-        worker_id => ${workerId}::uuid
-      );
-    `;
-    }
-    async completeTask(stepTask, output) {
-        await this.sql `
-      SELECT pgflow.complete_task(
-        run_id => ${stepTask.run_id}::uuid,
-        step_slug => ${stepTask.step_slug}::text,
-        task_index => ${stepTask.task_index}::int,
-        output => ${this.sql.json(output || null)}::jsonb
-      );
-    `;
-    }
-    async failTask(stepTask, error) {
-        const errorString = typeof error === 'string'
-            ? error
-            : error instanceof Error
-                ? error.message
-                : JSON.stringify(error);
-        await this.sql `
-      SELECT pgflow.fail_task(
-        run_id => ${stepTask.run_id}::uuid,
-        step_slug => ${stepTask.step_slug}::text,
-        task_index => ${stepTask.task_index}::int,
-        error_message => ${errorString}::text
-      );
-    `;
-    }
-    async startFlow(flow_slug, input, run_id) {
-        const results = await this.sql `
-      SELECT * FROM pgflow.start_flow(
-        flow_slug => ${flow_slug}::text,
-        input => ${this.sql.json(input)}::jsonb
-        ${run_id ? this.sql `, run_id => ${run_id}::uuid` : this.sql ``}
-      );
-    `;
-        if (results.length === 0) {
-            throw new Error(`Failed to start flow ${flow_slug}`);
-        }
-        const [flowRun] = results;
-        return flowRun;
-    }
-}

package/dist/README.md

@@ -1,399 +0,0 @@
-# pgflow SQL Core
-
-PostgreSQL-native workflow engine for defining, managing, and tracking DAG-based workflows directly in your database.
-
-> [!NOTE]
-> This project and all its components are licensed under [Apache 2.0](./LICENSE) license.
-
-> [!WARNING]
-> This project uses [Atlas](https://atlasgo.io/docs) to manage the schemas and migrations.
-> See [ATLAS.md](ATLAS.md) for more details.
-
-## Table of Contents
-
-- [Overview](#overview)
-- [Key Features](#key-features)
-- [Architecture](#architecture)
-  - [Schema Design](#schema-design)
-  - [Execution Model](#execution-model)
-- [Example Flow and its life](#example-flow-and-its-life)
-  - [Defining a Workflow](#defining-a-workflow)
-  - [Starting a Workflow Run](#starting-a-workflow-run)
-  - [Workflow Execution](#workflow-execution)
-    - [Task Polling](#task-polling)
-    - [Task Completion](#task-completion)
-    - [Error Handling](#error-handling)
-    - [Retries and Timeouts](#retries-and-timeouts)
-- [TypeScript Flow DSL](#typescript-flow-dsl)
-  - [Overview](#overview-1)
-  - [Type Inference System](#type-inference-system)
-  - [Basic Example](#basic-example)
-  - [How Payload Types Are Built](#how-payload-types-are-built)
-  - [Benefits of Automatic Type Inference](#benefits-of-automatic-type-inference)
-- [Data Flow](#data-flow)
-  - [Input and Output Handling](#input-and-output-handling)
-  - [Run Completion](#run-completion)
-
-## Overview
-
-The pgflow SQL Core provides the data model, state machine, and transactional functions for workflow management. It treats workflows as Directed Acyclic Graphs (DAGs) of steps, each step being a simple state machine.
-
-This package focuses on:
-
-- Defining and storing workflow shapes
-- Managing workflow state transitions
-- Exposing transactional functions for workflow operations
-- Providing two-phase APIs for reliable task polling and status updates
-
-The actual execution of workflow tasks is handled by the [Edge Worker](../edge-worker/README.md), which calls back to the SQL Core to acknowledge task completion or failure.
-
-## Key Features
-
-- **Declarative Workflows**: Define flows and steps via SQL tables
-- **Dependency Management**: Explicit step dependencies with atomic transitions
-- **Configurable Behavior**: Per-flow and per-step options for timeouts, retries, and delays
-- **Queue Integration**: Built on pgmq for reliable task processing
-- **Transactional Guarantees**: All state transitions are ACID-compliant
-
-## Architecture
-
-### Schema Design
-
-[Schema ERD Diagram (click to enlarge)](./assets/schema.svg)
-
-<a href="./assets/schema.svg">
-  <img src="./assets/schema.svg" alt="Schema ERD Diagram" width="25%" height="25%">
-</a>
-
----
-
-The schema consists of two main categories of tables:
-
-#### Static definition tables
-
-- `flows` (just an identity for the workflow with some global options)
-- `steps` (DAG nodes belonging to particular `flows`, with option overrides)
-- `deps` (DAG edges between `steps`)
-
-#### Runtime state tables
-
-- `runs` (execution instances of `flows`)
-- `step_states` (states of individual `steps` within a `run`)
-- `step_tasks` (units of work for individual `steps` within a `run`, so we can have fanouts)
-
-### Execution Model
-
-The SQL Core handles the workflow lifecycle through these key operations:
-
-1. **Definition**: Workflows are defined using `create_flow` and `add_step`
-2. **Instantiation**: Workflow instances are started with `start_flow`, creating a new run
-3. **Task Retrieval**: The [Edge Worker](../edge-worker/README.md) uses two-phase polling - first `read_with_poll` to reserve queue messages, then `start_tasks` to convert them to executable tasks
-4. **State Transitions**: When the Edge Worker reports back using `complete_task` or `fail_task`, the SQL Core handles state transitions and schedules dependent steps
-
-[Flow lifecycle diagram (click to enlarge)](./assets/flow-lifecycle.svg)
-
-<a href="./assets/flow-lifecycle.svg"><img src="./assets/flow-lifecycle.svg" alt="Flow Lifecycle" width="25%" height="25%"></a>
-
-## Example flow and its life
-
-Let's walk through creating and running a workflow that fetches a website,
-does summarization and sentiment analysis in parallel steps
-and saves the results to a database.
-
-![example flow graph](./assets/example-flow.svg)
-
-### Defining a Workflow
-
-Workflows are defined using two SQL functions: `create_flow` and `add_step`.
-
-In this example, we'll create a workflow with:
-
-- `website` as the entry point ("root step")
-- `sentiment` and `summary` as parallel steps that depend on `website`
-- `saveToDb` as the final step, depending on both parallel steps
-
-```sql
--- Define workflow with parallel steps
-SELECT pgflow.create_flow('analyze_website');
-SELECT pgflow.add_step('analyze_website', 'website');
-SELECT pgflow.add_step('analyze_website', 'sentiment', deps_slugs => ARRAY['website']);
-SELECT pgflow.add_step('analyze_website', 'summary', deps_slugs => ARRAY['website']);
-SELECT pgflow.add_step('analyze_website', 'saveToDb', deps_slugs => ARRAY['sentiment', 'summary']);
-```
-
-> [!WARNING]
-> You need to call `add_step` in topological order, which is enforced by foreign key constraints.
-
-> [!NOTE]
-> You can have multiple "root steps" in a workflow. You can even create a root-steps-only workflow
-> to process a single input in parallel, because at the end, all of the outputs from steps
-> that does not have dependents ("final steps") are aggregated and saved as run's `output`.
-
-### Starting a Workflow Run
-
-To start a workflow, call `start_flow` with a flow slug and input arguments:
-
-```sql
-SELECT * FROM pgflow.start_flow(
-  flow_slug => 'analyze_website',
-  input => '{"url": "https://example.com"}'::jsonb
-);
-
---     run_id  | flow_slug       | status  |  input                         | output | remaining_steps
--- ------------+-----------------+---------+--------------------------------+--------+-----------------
---  <run uuid> | analyze_website | started | {"url": "https://example.com"} | [NULL] |               4
-```
-
-When a workflow starts:
-
-- A new `run` record is created
-- Initial states for all steps are created
-- Root steps are marked as `started`
-- Tasks are created for root steps
-- Messages are enqueued on PGMQ for worker processing
-
-> [!NOTE]
-> The `input` argument must be a valid JSONB object: string, number, boolean, array, object or null.
-
-### Workflow Execution
-
-#### Task Polling
-
-The Edge Worker uses a two-phase approach to retrieve and start tasks:
-
-**Phase 1 - Reserve Messages:**
-```sql
-SELECT * FROM pgflow.read_with_poll(
-  queue_name => 'analyze_website',
-  vt => 60, -- visibility timeout in seconds
-  qty => 5  -- maximum number of messages to fetch
-);
-```
-
-**Phase 2 - Start Tasks:**
-```sql
-SELECT * FROM pgflow.start_tasks(
-  flow_slug => 'analyze_website',
-  msg_ids => ARRAY[101, 102, 103], -- message IDs from phase 1
-  worker_id => '550e8400-e29b-41d4-a716-446655440000'::uuid
-);
-```
-
-**How it works:**
-
-1. **read_with_poll** reserves raw queue messages and hides them from other workers
-2. **start_tasks** finds matching step_tasks, increments attempts counter, and builds task inputs
-3. Task metadata and input are returned to the worker for execution
-
-This two-phase approach ensures tasks always exist before processing begins, eliminating race conditions that could occur with single-phase polling.
-
-#### Task Completion
-
-After successful processing, the worker acknowledges completion:
-
-```sql
-SELECT pgflow.complete_task(
-  run_id => '<run_uuid>',
-  step_slug => 'website',
-  task_index => 0, -- we will have multiple tasks for a step in the future
-  output => '{"content": "HTML content", "status": 200}'::jsonb
-);
-```
-
-When a task completes:
-
-1. The task status is updated to 'completed' and the output is saved
-2. The message is archived in PGMQ
-3. The step state is updated to 'completed'
-4. Dependent steps with all dependencies completed are automatically started
-5. The run's remaining_steps counter is decremented
-6. If all steps are completed, the run is marked as completed with aggregated outputs
-
-#### Error Handling
-
-If a task fails, the worker acknowledges this using `fail_task`:
-
-```sql
-SELECT pgflow.fail_task(
-  run_id => '<run_uuid>',
-  step_slug => 'website',
-  task_index => 0,
-  error_message => 'Connection timeout when fetching URL'::text
-);
-```
-
-The system handles failures by:
-
-1. Checking if retry attempts are available
-2. For available retries:
-   - Keeping the task in 'queued' status
-   - Applying exponential backoff for visibility
-   - Preventing processing until the visibility timeout expires
-3. When retries are exhausted:
-   - Marking the task as 'failed'
-   - Storing the task output (even for failed tasks)
-   - Marking the step as 'failed'
-   - Marking the run as 'failed'
-   - Archiving the message in PGMQ
-   - **Archiving all queued messages for the failed run** (preventing orphaned messages)
-4. Additional failure handling:
-   - **No retries on already-failed runs** - tasks are immediately marked as failed
-   - **Graceful type constraint violations** - handled without exceptions when single steps feed map steps
-   - **Stores invalid output on type violations** - captures the output that caused the violation for debugging
-   - **Performance-optimized message archiving** using indexed queries
-
-#### Retries and Timeouts
-
-Retry behavior can be configured at both the flow and step level:
-
-```sql
--- Flow-level defaults
-SELECT pgflow.create_flow(
-  flow_slug => 'analyze_website',
-  max_attempts => 3,    -- Maximum retry attempts (including first attempt)
-  base_delay => 5,      -- Base delay in seconds for exponential backoff
-  timeout => 60         -- Task timeout in seconds
-);
-
--- Step-level overrides
-SELECT pgflow.add_step(
-  flow_slug => 'analyze_website',
-  step_slug => 'sentiment',
-  deps_slugs => ARRAY['website']::text[],
-  max_attempts => 5,    -- Override max attempts for this step
-  base_delay => 2,      -- Override base delay for exponential backoff
-  timeout => 30         -- Override timeout for this step
-);
-```
-
-The system applies exponential backoff for retries using the formula:
-
-```
-delay = base_delay * (2 ^ attempts_count)
-```
-
-Timeouts are enforced by setting the message visibility timeout to the step's timeout value plus a small buffer. If a worker doesn't acknowledge completion or failure within this period, the task becomes visible again and can be retried.
-
-## TypeScript Flow DSL
-
-> [!NOTE]
-> TypeScript Flow DSL is a Work In Progress and is not ready yet!
-
-### Overview
-
-While the SQL Core engine handles workflow definitions and state management, the primary way to define and work with your workflow logic is via the Flow DSL in TypeScript. This DSL offers a fluent API that makes it straightforward to outline the steps in your flow with full type safety.
-
-### Type Inference System
-
-The most powerful feature of the Flow DSL is its **automatic type inference system**:
-
-1. You only need to annotate the initial Flow input type
-2. The return type of each step is automatically inferred from your handler function
-3. These return types become available in the payload of dependent steps
-4. The TypeScript compiler builds a complete type graph matching your workflow DAG
-
-This means you get full IDE autocompletion and type checking throughout your workflow without manual type annotations.
-
-### Basic Example
-
-Here's an example that matches our website analysis workflow:
-
-```ts
-// Provide a type for the input of the Flow
-type Input = {
-  url: string;
-};
-
-const AnalyzeWebsite = new Flow<Input>({
-  slug: 'analyze_website',
-  maxAttempts: 3,
-  baseDelay: 5,
-  timeout: 10,
-})
-  .step(
-    { slug: 'website' },
-    async (input) => await scrapeWebsite(input.run.url)
-  )
-  .step(
-    { slug: 'sentiment', dependsOn: ['website'], timeout: 30, maxAttempts: 5 },
-    async (input) => await analyzeSentiment(input.website.content)
-  )
-  .step(
-    { slug: 'summary', dependsOn: ['website'] },
-    async (input) => await summarizeWithAI(input.website.content)
-  )
-  .step(
-    { slug: 'saveToDb', dependsOn: ['sentiment', 'summary'] },
-    async (input) =>
-      await saveToDb({
-        websiteUrl: input.run.url,
-        sentiment: input.sentiment.score,
-        summary: input.summary,
-      }).status
-  );
-```
-
-### How Payload Types Are Built
-
-The payload object for each step is constructed dynamically based on:
-
-1. **The `run` property**: Always contains the original workflow input
-2. **Dependency outputs**: Each dependency's output is available under a key matching the dependency's ID
-3. **DAG structure**: Only outputs from direct dependencies are included in the payload
-
-This means your step handlers receive exactly the data they need, properly typed, without any manual type declarations beyond the initial Flow input type.
-
-### Benefits of Automatic Type Inference
-
-- **Refactoring safety**: Change a step's output, and TypeScript will flag all dependent steps that need updates
-- **Discoverability**: IDE autocompletion shows exactly what data is available in each step
-- **Error prevention**: Catch typos and type mismatches at compile time, not runtime
-- **Documentation**: The types themselves serve as living documentation of your workflow's data flow
-
-## Data Flow
-
-### Input and Output Handling
-
-Handlers in pgflow **must return** JSON-serializable values that are captured and saved when `complete_task` is called. These outputs become available as inputs to dependent steps, allowing data to flow through your workflow pipeline.
-
-When a step is executed, it receives an input object where:
-
-- Each key is a step_slug of a completed dependency
-- Each value is that step's output
-- A special "run" key contains the original workflow input
-
-#### Example: `sentiment`
-
-When the `sentiment` step runs, it receives:
-
-```json
-{
-  "run": { "url": "https://example.com" },
-  "website": { "content": "HTML content", "status": 200 }
-}
-```
-
-#### Example: `saveToDb`
-
-The `saveToDb` step depends on both `sentiment` and `summary`:
-
-```json
-{
-  "run": { "url": "https://example.com" },
-  "sentiment": { "score": 0.85, "label": "positive" },
-  "summary": "This website discusses various topics related to technology and innovation."
-}
-```
-
-### Run Completion
-
-When all steps in a run are completed, the run status is automatically updated to 'completed' and its output is set. The output is an aggregation of all the outputs from final steps (steps that have no dependents):
-
-```sql
--- Example of a completed run with output
-SELECT run_id, status, output FROM pgflow.runs WHERE run_id = '<run_uuid>';
-
---     run_id  | status    | output
--- ------------+-----------+-----------------------------------------------------
---  <run uuid> | completed | {"saveToDb": {"status": "success"}}
-```