@brandboostinggmbh/observable-workflows 0.22.0 → 0.23.0

package/CHANGELOG.md

@@ -0,0 +1,839 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## NPM Changelog Conventions
+
+For NPM packages, changelogs are conventionally stored in a `CHANGELOG.md` file at the root of the repository. When publishing to NPM:
+
+- The changelog is included in the package if the file is not excluded in `.npmignore` or `package.json` files
+- Many tools and package registries (including npm's website) will display the changelog from this file
+- Users can view the changelog with `npm view @brandboostinggmbh/observable-workflows`
+- The changelog file is also visible in the GitHub repository for easy reference
+
+## [Unreleased]
+
+### 🚧 Work in Progress
+
+Changes that will be included in the next release will be documented here.
+
+---
+
+## [v0.23.0] - 2026-03-16
+
+### ✨ Features
+
+- `call()` and `retry()` now return a `WorkflowCallResult<O>` instead of `void`, containing `instanceId`, `workflowType`, `tenantId`, `result`, `workflowStatus`, `startTime`, `endTime`, `parentInstanceId`, and `triggerId`
+- New exported type `WorkflowCallResult<O, TYPE>` built via `Pick` from `TypedWorkflowRun` with narrowed `workflowStatus: 'completed'` and non-null `result: O` / `endTime: number`
+
+---
+
+## [v0.22.1] - 2026-02-19
+
+### 📚 Documentation
+
+- Add comprehensive CHANGELOG.md with full project history from v0.1.0 to v0.22.0
+- Include CHANGELOG.md in published NPM package
+- Add changelog reference section to README.md
+
+---
+
+## [v0.22.0] - 2026-02-19
+
+### ✨ Features
+
+- **Delayed Workflow Scheduling**: Workflows can now be scheduled with a delay before execution
+  - New `delaySeconds` parameter in `enqueueWorkflow()` and `enqueueWorkflowBatch()` for specifying execution delays (0-2592000 seconds / 30 days)
+  - New `scheduledFor` timestamp field in `WorkflowQueueMessage` and `WorkflowRun` tracking when delayed workflows should execute
+  - Database schema V8 adds `delaySeconds` (INTEGER) and `scheduledFor` (INTEGER) columns to WorkflowTable
+  - Automatic migration from V6/V7 to V8 preserves existing data while adding new columns
+  - Queue consumer honors delay by using `delaySeconds` in Cloudflare Queue's batch send API
+  - Workflows remain in 'scheduled' status until delay expires and execution begins
+- New `getWorkflowsScheduledBefore()` function for querying workflows scheduled to execute before a specific timestamp
+- Enhanced `listWorkflows()` with `scheduledBefore` and `scheduledAfter` filter options for time-based scheduled workflow queries
+
+### 🔧 Improvements
+
+- Database schema improvements with new columns for delayed execution tracking
+- Migration system enhanced to support V6→V8 and V7→V8 upgrade paths
+- Queue message validation ensures delaySeconds is within valid range (0-2592000)
+
+## [v0.21.1] - 2026-02-11
+
+*Production release with no additional changes from v0.21.1-beta.2*
+
+## [v0.21.1-beta.2] - 2026-02-11
+
+### 💥 Breaking Changes
+
+- `enqueueWorkflowBatch()` now enforces a maximum batch size limit of 100 workflows and throws an error if exceeded, preventing potential performance issues and queue system overload
+
+### 🔧 Improvements
+
+- Batch size validation added before processing to fail fast with clear error message
+- Improved error messaging for batch size violations
+
+## [v0.21.1-beta.1] - 2026-02-11
+
+*Version bump with no user-facing changes*
+
+## [v0.21.0-beta.1] - 2026-02-05
+
+### ✨ Features
+
+- **Workflow Trigger Identification**: Support for tracking the origin or cause of workflow execution via `triggerId`
+  - New `triggerId` parameter in `enqueueWorkflow()` and `enqueueWorkflowBatch()` for identifying what triggered the workflow execution
+  - New `getWorkflowByTriggerId()` function in log accessor for retrieving workflows by their trigger identifier
+  - `WorkflowQueueMessage` includes `triggerId` field for queue-based workflow execution tracking
+  - `WorkflowRun` type includes `triggerId: string | null` field for trigger correlation in workflow records
+
+### 💥 Breaking Changes
+
+- **Function Signature Change**: `enqueueRetryWorkflow()` now accepts an object parameter instead of positional parameters
+  - Before: `enqueueRetryWorkflow(instanceId, retryConfig)`
+  - After: `enqueueRetryWorkflow({ instanceId, retryConfig, triggerId })`
+  - Migration: Wrap existing calls in object syntax, e.g., `enqueueRetryWorkflow({ instanceId, retryConfig })`
+
+### 🔧 Improvements
+
+- Database schema V6 adds `triggerId` (TEXT, UNIQUE, nullable) column to WorkflowTable for trigger identification
+- Automatic migration from V5 to V6 preserves existing data while adding trigger tracking capability
+- Enhanced workflow enqueue operations to propagate `triggerId` through the execution pipeline
+
+## [v0.20.3-beta.1] - 2026-01-26
+
+### 🔧 Improvements
+
+- **Query Optimization**: `getPropertiesKeys()` split into two separate optimized queries instead of using `(? IS NULL OR instanceId = ?)` pattern
+  - When `instanceId` provided: Uses `instanceId` index directly for targeted query
+  - When `instanceId` omitted: Uses `tenant_key` composite index for tenant-wide query
+  - Eliminates OR pattern that prevented SQLite from using indexes effectively
+  - Improves query performance by allowing index usage in both code paths
+
+## [v0.20.2-beta.1] - 2026-01-26
+
+### 🔧 Improvements
+
+- **Query Optimization**: `listWorkflows()` now only uses `DISTINCT` when JOIN operations are present (property filters)
+  - Removes unnecessary `DISTINCT` overhead when querying workflows without property filters
+  - Significantly improves query performance for simple workflow listings by avoiding full result set processing
+  - Maintains `DISTINCT` when needed (with JOINs) to prevent duplicate results
+  - Added performance tracking with D1 metadata logging for query analysis
+
+## [v0.20.1-beta.1] - 2026-01-20
+
+### ✨ Features
+
+- **Cleanup Manager Enhancement**: Add optional support for cleaning up scheduled and waiting workflows
+  - New `scheduledWorkflows` option (defaults to `false`) in `DeleteConfig` for including workflows in 'scheduled' status
+  - New `waitingWorkflows` option (defaults to `false`) in `DeleteConfig` for including workflows in 'waiting' status
+  - Enables selective cleanup of workflows in pre-execution states when explicitly opted in
+  - Maintains backward compatibility by keeping these statuses excluded by default
+
+### 🔧 Improvements
+
+- Comprehensive test coverage for scheduled and waiting workflow cleanup scenarios
+- Test coverage for multi-status cleanup operations with all flags enabled
+
+## [v0.20.0-beta.6] - 2026-01-19
+
+### 🔧 Improvements
+
+- **Batch Performance Optimization**: `enqueueWorkflowBatch()` now uses `prepareWorkflowInsertStatements()` helper for efficient batch workflow insertion
+  - Pre-prepares all workflow insert statements before executing batch operation
+  - Reduces code duplication between single and batch workflow insertion logic
+  - Maintains consistent workflow record creation across enqueue operations
+
+## [v0.20.0-beta.5] - 2026-01-19
+
+### ✨ Features
+
+- **Customizable D1 Retry Configuration**: Users can now override the default D1 retry behavior
+  - New `retryConfig` optional parameter in `createWorkflowContext()`, `createQueueWorkflowContext()`, and `createLogAccessor()`
+  - Allows customizing `maxRetries`, `initialDelayMs`, and `maxDelayMs` for D1 operations
+  - Configuration is propagated to all internal operations that use D1 retry logic
+  - Defaults to existing behavior when not specified (5 retries, 100ms initial delay, 5000ms max delay)
+
+### 🔧 Improvements
+
+- Export `RetryConfig` type from library for users to define custom retry configurations
+- Retry configuration is now threaded through all context creation functions for consistent retry behavior
+
+## [v0.20.0-beta.4] - 2026-01-19
+
+*Version bump with no user-facing changes*
+
+## [v0.20.0-beta.3] - 2026-01-19
+
+*Version bump with no user-facing changes*
+
+## [v0.20.0-beta.2] - 2026-01-19
+
+*Version bump with no user-facing changes*
+
+## [v0.20.0-beta.1] - 2026-01-19
+
+### ✨ Features
+
+- **Comprehensive D1 Retry Logic**: All D1 database operations now include automatic retry with exponential backoff and jitter
+  - New `retryD1Operation()` helper function wraps all D1 queries with configurable retry logic
+  - Handles transient network errors (SQLITE_BUSY, connection timeouts, network failures)
+  - Default configuration: 5 max retries, 100ms initial delay, 5s max delay, exponential backoff with jitter
+  - Applied to all database operations: workflow CRUD, step CRUD, property CRUD, cleanup operations, statistics queries
+  - `RetryConfig` type allows customization of retry behavior (maxRetries, initialDelayMs, maxDelayMs)
+
+### 🔧 Improvements
+
+- Improved resilience for all database operations across workflow lifecycle (creation, execution, logging, cleanup)
+- Retry logic added to workflow insertion, step insertion, log insertion, property operations, and query operations
+- Enhanced error handling for transient D1 failures with automatic recovery
+- Consistent retry behavior across all database interaction points in the library
+
+## [v0.19.0-beta.4] - 2025-11-04
+
+### 🔧 Improvements
+
+- Add composite database index on WorkflowTable (parentInstanceId, tenantId) to optimize queries filtering by parent workflows within a tenant, improving performance for hierarchical workflow queries
+
+## [v0.19.0-beta.3] - 2025-11-04
+
+### 🔧 Improvements
+
+- Add database indexes on WorkflowTable for improved query performance:
+  - Index on `tenantId` column for tenant-scoped queries
+  - Index on `workflowStatus` column for status-based filtering
+  - Index on `parentInstanceId` column for parent-child workflow relationship queries
+- Add `release:beta` npm script for publishing beta releases to NPM
+
+## [v0.19.0-beta.2] - 2025-10-23
+
+*Version bump with no user-facing changes*
+
+## [v0.19.0-beta.1] - 2025-10-20
+
+### ✨ Features
+
+- **Workflow Dependencies System**: Workflows can now depend on other workflows with full dependency management support
+  - New `createWorkflowDependency()` function to create a single dependency relationship between workflows
+  - New `createWorkflowDependencies()` function to create multiple dependency relationships atomically with workflow creation
+  - New `deleteWorkflowDependency()` function to remove dependency relationships between workflows
+  - New `WorkflowDependencies` database table with composite primary key and tenant-scoped queries
+  - Workflows in 'waiting' status hold execution until all dependencies complete
+  - New `populateDependencies` and `populateDependents` options in `getWorkflow()` and `listWorkflows()` for fetching dependency relationships
+- **Scheduled Workflow Support**: Workflows can now be pre-created and scheduled for later execution
+  - New `scheduledInstanceId` parameter in `WorkflowQueueMessage` for pre-creating workflow records before execution
+  - Workflows created in 'scheduled' status when pre-defined, or 'waiting' status when dependencies exist
+  - `enqueueWorkflow()` and `enqueueWorkflowBatch()` now return `ScheduledWorkflowExecutionStub` with `instanceId` and `workflowType`
+  - Enables workflow scheduling patterns where workflow records are created ahead of time
+- **Workflow Dependency Polling**: New `handlePollWaitingWorkflows()` function for automatic dependency resolution
+  - Polls waiting workflows from the database
+  - Automatically enqueues workflows when all their dependencies complete
+  - Transitions 'waiting' → 'scheduled' status when dependencies are satisfied
+  - Supports cross-tenant waiting workflow discovery with `ignoreTenant` option
+- **Enhanced listWorkflows Options**:
+  - New `populateResult` option to include workflow results in responses
+  - New `populateDependencies` option to fetch and attach all dependency relationships
+  - New `populateDependents` option to fetch and attach all dependent relationships
+  - New `ignoreTenant` option for cross-tenant workflow queries (useful for dependency polling)
+- **New WorkflowStatus Types**: Added 'scheduled' and 'waiting' workflow status values to support pre-execution workflow states
+- **New Exports**:
+  - `createCleanupManager` function for workflow data retention management
+  - `WorkflowStatus` type for type-safe status usage
+  - `WorkflowDependency` and `WorkflowDependencyRelation` types for dependency handling
+  - `updateWorkflow()` function for updating workflow records after creation
+  - `workflowTableRowToWorkflowRun()` helper for converting database rows to workflow objects
+  - Workflow dependency helper functions: `prepareWorkflowDependencyStatement()`, `createWorkflowDependency()`, `createWorkflowDependencies()`, `deleteWorkflowDependency()`
+
+### 🔧 Improvements
+
+- **Database Schema Enhancement**: New WorkflowDependencies table (V7) with efficient many-to-many relationship tracking
+  - Composite primary key on (dependencyWorkflowId, dependentWorkflowId) ensures unique relationships
+  - Foreign key constraints with CASCADE DELETE for referential integrity
+  - Optimized indexes on dependencyWorkflowId, dependentWorkflowId, and tenantId for fast queries
+  - createdAt timestamp for audit trail and relationship history
+- **Enhanced insertWorkflowRecord()**: Now supports atomic creation of workflows with their dependencies
+  - Accepts optional dependencies array
+  - Automatically creates appropriate status based on dependency presence
+  - Returns success metadata for both workflow and dependency inserts
+- **Improved getWorkflow()**: Enhanced with dependency fetching capabilities
+  - New `populateDependencies` option to fetch all workflows this workflow depends on
+  - New `populateDependents` option to fetch all workflows that depend on this workflow
+  - Maintains clean separation between data fetching and population options
+- **TypeScript Type Improvements**: Enhanced type system for workflow dependencies
+  - New `ExtractDependencyResult<W>` type for typed dependency extraction
+  - New `WorkflowDependencyRun<I, O, TYPE>` type for typed dependency relationships
+  - Typed `getDependencies()` in workflow context for strong type safety across dependency workflows
+- **Batch Enqueue Performance**: `enqueueWorkflowBatch()` maintains order while efficiently handling dependencies and scheduling
+
+## [v0.18.2] - 2025-10-07
+
+### 🐛 Bug Fixes
+
+- Fix `getPropertiesKeys()` function to use `deserializeWorkflowPropertyValue()` instead of `tryDeserializeObj()` for consistent property value deserialization across workflow properties
+
+## [v0.18.1] - 2025-10-07
+
+### ✨ Features
+
+- Export new helper functions `serializeWorkflowPropertyValue()` and `deserializeWorkflowPropertyValue()` for handling workflow property serialization and deserialization
+
+### 🔧 Improvements
+
+- Improve property deserialization in `listWorkflows()` by using `deserializeWorkflowPropertyValue()` instead of `tryDeserializeObj()` for proper type-specific deserialization
+- Enhance property value deserialization to handle different value types (string, number, boolean, object) with appropriate type conversion and JSON parsing with fallback error handling
+
+## [v0.18.0] - 2025-10-06
+
+### 🔧 Improvements
+
+- Index creation is now idempotent - checks for existing indexes before attempting to create them, eliminating redundant creation attempts and preventing potential conflicts when re-running migrations
+- Optimized index creation performance by batching index creation statements together for execution, reducing the number of database round-trips
+- Fine-tuned query batch size for stability by reducing property loading batch size from 100 to 80 to provide additional safety margin below SQLite's variable limit
+
+## [v0.17.7] - 2025-10-05
+
+*Version bump with no user-facing changes*
+
+## [v0.17.6] - 2025-10-05
+
+### 🔧 Improvements
+
+- Property loading in `listWorkflows` now processes in batches of 100 instanceIds at a time to avoid hitting SQLite's 999 variable limit when populating properties for large result sets
+
+## [v0.17.5] - 2025-10-05
+
+*Version bump with no user-facing changes*
+
+## [v0.17.4] - 2025-10-04
+
+### ✨ Features
+
+- Add `populateProperties` option to `listWorkflows()` method for efficient workflow property fetching - when enabled, properties are fetched and attached to each workflow in a single additional query
+
+### 🔧 Improvements
+
+- Efficient property fetching using IN clause with all instanceIds in one query, then grouped by instanceId and attached to workflows, improving performance for bulk workflow retrieval with properties
+
+## [v0.17.3] - 2025-09-16
+
+### 🔧 Improvements
+
+- Add composite database index `idx_workflows_tenant_starttime` on WorkflowTable (tenantId, startTime DESC) to optimize tenant and time-based workflow queries
+
+## [v0.17.2] - 2025-09-15
+
+### 🔧 Improvements
+
+- Add `debugLogs` option to `listWorkflows()` function for SQL query debugging - when enabled, logs the SQL query being executed, the bindings used, and a message when the query completes
+
+## [v0.17.1] - 2025-09-14
+
+### 🔧 Improvements
+
+- Add console logging to cleanup manager for better visibility during deletion operations, including workflow deletion progress, associated steps deletion, external storage key deletion, database deletion phase, and messages when no workflows match deletion criteria
+
+## [v0.17.0] - 2025-09-10
+
+### 💥 Breaking Changes
+
+- `queueIdentifier` is now a required field in `QueueWorkflowContextOptions` - must be provided when creating queue workflow contexts
+
+### ✨ Features
+
+- Queue handlers now validate and track workflow queue routing with `queueIdentifier` support in `WorkflowQueueMessage` - a warning is logged if a workflow message is processed by the incorrect queue handler
+
+## [v0.16.4] - 2025-09-09
+
+### 🔧 Improvements
+
+- Optimize `listSteps` query performance by refactoring SQL query construction to use dynamic WHERE clause building instead of null-checking pattern
+- Add composite database indexes for improved query performance: `idx_steps_instance_starttime` on StepTable (instanceId, startTime) and `idx_workflow_properties_tenant_key` on WorkflowProperties (tenantId, key, valueType)
+
+## [v0.16.3] - 2025-09-06
+
+### ✨ Features
+
+- Add `workflowAverageWallTimeMilis` property to workflow statistics for tracking average workflow execution time in aggregated results
+
+## [v0.16.2] - 2025-09-05
+
+*Version bump with no user-facing changes*
+
+## [v0.16.1] - 2025-09-05
+
+### ✨ Features
+
+- Add `successfulWorkflowCount` and `failedWorkflowCount` fields to workflow statistics query results for better insights into workflow completion status
+
+## [v0.16.0] - 2025-09-04
+
+### ✨ Features
+
+- Add `getWorkflowTypesStatistics()` function to retrieve aggregated statistics for workflow types including counts, wall time totals, and incomplete workflow counts with optional filtering by tenant ID and time range
+
+## [v0.15.6] - 2025-09-01
+
+### 🔧 Improvements
+
+- Add database index on StepTable startTime column for improved query performance when filtering or sorting steps by start time
+
+## [v0.15.5] - 2025-09-01
+
+### 🔧 Improvements
+
+- Optimize SQL query for orphaned log deletion by querying StepTable directly instead of using NOT EXISTS subquery with WorkflowTable, improving performance for cleanup operations
+
+## [v0.15.4] - 2025-08-31
+
+### ✨ Features
+
+- Add `deleteOrphanedSteps()` function to cleanup manager for removing steps that reference non-existent workflows
+- Add `deleteOrphanedLogs()` function to cleanup manager for removing logs that reference non-existent steps or workflows
+
+## [v0.15.3] - 2025-08-29
+
+### 🔧 Improvements
+
+- Add database indexes on foreign key columns (StepTable.instanceId and LogTable.instanceId) for improved query performance, particularly for cascade deletion and JOIN operations
+
+## [v0.15.1] - 2025-08-28
+
+### 🔧 Improvements
+
+- Add specific index on WorkflowTable for status and time-based filtering to improve query performance when filtering by workflow status with time ranges
+- Optimize workflow deletion SQL query in cleanup manager to use explicit filter predicate instead of subquery for better performance
+
+## [v0.15.0] - 2025-08-28
+
+### ✨ Features
+
+- Add `deleteRefsFromExternalStorage` configuration option to cleanup manager for selective external storage cleanup during workflow deletion
+- Add `delete()` method to `ExternalBlobStorage` interface for batch deletion of external storage keys
+- Implement bulk delete capability in R2 external blob storage with fallback to individual deletes for resilience
+
+### 🔧 Improvements
+
+- Enhance `countAffectedWorkflows()` to return detailed information including affected steps array and count of external storage keys to be deleted
+- Enhance `deleteOldWorkflows()` to return deleted steps array and count of external storage keys actually deleted, enabling better tracking of cleanup operations
+- Improve error handling in external storage deletion - errors are logged but do not fail the entire cleanup operation, allowing partial deletions to succeed
+- Refactor cleanup manager with new helper functions `getAffectedWorkflows()`, `getAffectedSteps()`, and `collectExternalStorageKeys()` for better code organization and maintainability
+- Add `limit` parameter to cleanup functions to prevent out-of-memory errors when processing large numbers of workflows
+
+### 💥 Breaking Changes
+
+- `ExternalBlobStorage` interface now requires implementing the `delete(...keys: string[]): Promise<number>` method. Existing implementations must add this method.
+- `countAffectedWorkflows()` now accepts a `limit` parameter as the second argument
+- `deleteOldWorkflows()` now accepts a `limit` parameter as the second argument
+
+---
+
+## [v0.14.0] - 2025-08-27
+
+### ✨ Features
+
+- Add Cleanup Manager system with `createCleanupManager()` for automated workflow data retention management
+- Add `countAffectedWorkflows()` function to preview workflows matching cleanup criteria before deletion
+- Add `deleteOldWorkflows()` function to delete workflows matching time-based, count-based, or type-based criteria
+- Add database schema V5 with CASCADE DELETE constraints to automatically remove related steps and logs when workflows are deleted
+
+### 🔧 Improvements
+
+- Improve data consistency with foreign key constraints ensuring referential integrity between workflows, steps, and logs
+- Add migration system supporting upgrade from schema V2 and V3 to V5
+
+## [v0.13.3] - 2025-08-23
+
+*Internal debugging release with temporary diagnostic logging*
+
+## [v0.13.2] - 2025-08-23
+
+### 🐛 Bug Fixes
+
+- Fix incorrect table name 'WorkflowProperties' to correct table name 'WorkflowProperty' in deleteWorkflowProperties query
+
+## [v0.13.1] - 2025-08-22
+
+### 💥 Breaking Changes
+
+- Move `populateInput` parameter from function signature to filter options object in `listWorkflows()` for consistent parameter structure
+
+## [v0.13.0] - 2025-08-22
+
+### ✨ Features
+
+- Add `populateInput` parameter to `listWorkflows()` to control whether workflow inputs are deserialized and loaded (defaults to `false` to reduce memory usage when listing many workflows)
+
+## [v0.12.2] - 2025-08-20
+
+### 💥 Breaking Changes
+
+- Remove `WorkflowBatchItem` type alias export - users should directly type workflow parameters with required `workflow`, `input`, `workflowName`, and optional `triggerId` fields
+
+## [v0.12.1] - 2025-08-20
+
+### 🔧 Improvements
+
+- Export `WorkflowBatchItem` type alias for improved type safety when using `enqueueWorkflowBatch`
+
+## [v0.12.0] - 2025-08-19
+
+### ✨ Features
+
+- Add `reuseSuccessfulSteps` option to `enqueueRetryWorkflow()` for controlling whether retry operations reuse successful step results from the parent workflow instance (defaults to `true`)
+
+---
+
+## [v0.15.2] - 2026-02-19
+
+### 🐛 Bug Fixes
+
+- Fix workflow batch deletion to process deletes in batches of 100 to prevent hitting D1's variable binding limit when deleting large numbers of workflows
+
+## [v0.11.5] - 2026-02-19
+
+### 🐛 Bug Fixes
+
+- Remove debug console.log statements from listWorkflows method to reduce noise in logs
+
+## [v0.11.4] - 2025-08-06
+
+### 🔧 Improvements
+
+- Add database index on WorkflowTable startTime column for improved query performance
+
+## [v0.11.3] - 2026-02-19
+
+### 🔧 Improvements
+
+- Enhanced debug logging in `listWorkflows` to include the SQL query being executed
+
+## [v0.11.2] - 2025-08-06
+
+### 🔧 Improvements
+
+- Added debug logging to `listWorkflows` function to help track query execution and result processing
+
+## [v0.11.1] - 2026-02-19
+
+### ✨ Features
+
+- Export `HandleWorkflowQueueMessageParams` type for better type safety when handling queue messages
+- Add support for `triggerId` parameter in workflow retry operations to improve workflow traceability
+
+### 🔧 Improvements
+
+- Refactor `handleWorkflowQueueMessage` function to use object destructuring for cleaner API
+- Refactor `retry` function to use object parameters for better maintainability
+- Improve error handling to check for existing workflows before throwing error messages
+- Add `triggerId` generation for workflow runs and retries to enable better request tracking
+
+## [v0.11.0] - 2025-07-25
+
+### ✨ Features
+
+- Add workflow trigger identification support via new `triggerId` parameter in `WorkflowCallParams`
+- Add `getWorkflowByTriggerId()` function to log accessor for retrieving workflows by trigger ID
+
+### 💥 Breaking Changes
+
+- `WorkflowRun` type now includes `triggerId: string | null` field for trigger correlation
+
+### 🔧 Improvements
+
+- Add duplicate workflow prevention by checking for existing workflows with same `triggerId` before execution
+- Enhance database schema with V4 migration to support `triggerId` with UNIQUE constraint on WorkflowTable
+- Add comprehensive migration documentation explaining versioning system and table evolution strategy
+- Improve database migration system with specific migration functions `migrateWorkflowTableV1ToV2()` and `migrateWorkflowTableV2V3ToV4()`
+
+## [v0.10.1] - 2026-02-19
+
+### 🐛 Bug Fixes
+
+- Add log message truncation to prevent database storage errors for oversized logs. Messages exceeding 64KB are truncated with a UTF-8 safe truncation point and an indicator appended.
+
+## [v0.10.0] - 2026-02-19
+
+### ✨ Features
+
+- Add `enqueueWorkflowBatch` function to batch-enqueue multiple workflows at once, improving efficiency when processing multiple workflow requests
+
+## [v0.9.0] - 2025-02-19
+
+### ✨ Features
+
+- Added `LogBatcher` class for efficient batch processing of log entries with automatic periodic flushing and size-based triggers
+- Added `populateData` parameter to `listSteps()` function to control whether result/error fields are populated from external storage
+
+### 🔧 Improvements
+
+- Improved performance of log operations by batching database writes using D1's batch API instead of individual pushes
+- Optimized memory usage by defaulting to not populating result/error fields in `listSteps()` - can be enabled via `populateData` option when needed
+- Updated `getWorkflow()` to support `populateData` parameter for consistent data retrieval behavior
+
+## [v0.8.6] - 2025-07-04
+
+### 🔧 Improvements
+
+- `listSteps()` now returns `null` for result and error fields to optimize memory usage. Results and errors are not populated in this function and should be retrieved separately using `getStep()` if needed.
+
+### 🐛 Bug Fixes
+
+- Fixed workflow retry functionality to properly deserialize workflow input from external blob storage when retrying with `reuseSuccessfulSteps` option
+
+## [v0.8.5] - 2025-07-04
+
+### 🐛 Bug Fixes
+
+- Removed debug logging from deserializeWithExternalStorage that was cluttering console output
+
+## [v0.8.4] - 2025-07-04
+
+### 🐛 Bug Fixes
+
+- Added debug logging to external storage deserialization to help diagnose data loading issues
+
+## [v0.8.3] - 2025-07-04
+
+### 🐛 Bug Fixes
+
+- Fixed step result retrieval to properly check external blob storage when accessing step data through `getStep()` method
+
+## [v0.8.2] - 2025-07-04
+
+### 🐛 Bug Fixes
+
+- Simplified error logging to improve clarity and reduce noise in error reports
+
+## [v0.8.1] - 2025-07-04
+
+### 🐛 Bug Fixes
+
+- Improved error logging for serialization/deserialization failures with structured error details including message, cause, and stack trace
+
+## [v0.8.0] - 2025-07-03
+
+### ✨ Features
+
+- Enhanced `listSteps()` function with flexible API - now supports both object-based parameters (`listSteps({ limit, offset, instanceId })`) and maintains backward compatibility with the original positional parameter signature. The new API allows omitting limit/offset parameters to retrieve all steps without pagination
+
+## [v0.7.3] - 2025-07-03
+
+### ✨ Features
+
+- Added comprehensive database schema migration system with automatic version detection and idempotent migrations that preserve existing data
+
+## [v0.7.2] - 2025-07-02
+
+*Internal refactoring release with no user-facing changes*
+
+## [v0.7.1] - 2025-07-02
+
+### 🐛 Bug Fixes
+
+- Remove console output during database table initialization to reduce unnecessary logging noise
+
+## [v0.7.0] - 2025-07-02
+
+### ✨ Features
+
+- **External Blob Storage Support**: Added external storage capability for large workflow data (step results, errors, and inputs) that exceeds D1 database size limits
+  - New `ExternalBlobStorage` interface for pluggable storage backends
+  - New `createR2ExternalBlobStorage()` function for Cloudflare R2 integration with configurable threshold, key prefix, and custom ID generation
+  - New helper functions `serializeWithExternalStorage()` and `deserializeWithExternalStorage()` for transparent external storage handling
+  - Automatic fallback to direct database storage when external storage fails
+  - Optional `externalBlobStorage` parameter in `createWorkflowContext()` and `createLogAccessor()`
+- **Comprehensive Documentation**: Complete README.md rewrite with extensive usage examples, API reference, best practices, and quick start guide
+
+### 🔧 Improvements
+
+- **Database Schema Migration**: Enhanced `ensureTables()` function with automatic schema migration support to add external blob storage columns (`inputRef`, `resultRef`, `errorRef`) to existing tables
+- **Data Retrieval**: Updated `createLogAccessor()` to automatically handle external blob storage references when retrieving workflow data, steps, and logs
+
+### 📚 Exports
+
+- `ExternalBlobStorage` (type)
+- `R2ExternalBlobStorageOptions` (type)
+- `createR2ExternalBlobStorage()` (function)
+- `serializeWithExternalStorage()` (function)
+- `deserializeWithExternalStorage()` (function)
+
+## [v0.6.0] - 2025-06-05
+
+### ✨ Features
+
+- Add comprehensive filtering capabilities to `listWorkflows()` function with support for:
+  - Text search across workflow names and types
+  - Status and type filtering with single or multiple values
+  - Date range filtering for start and end times (with `gte`, `lte`, `gt`, `lt` operators)
+  - Custom property filtering with comparison operators (`equals`, `contains`, `gt`, `gte`, `lt`, `lte`, `in`)
+  - Combined multi-criteria filtering
+- Export new filter types: `WorkflowFilter`, `DateRangeFilter`, `StringFilter`, and `PropertyFilter`
+
+### 🔧 Development
+
+- Add Cloudflare Workers testing environment with `@cloudflare/vitest-pool-workers` and `wrangler` for realistic D1 database testing
+- Add comprehensive test coverage for filtering functionality and logging behavior
+- Temporarily disable lint job in CI workflow
+
+## [v0.5.0] - 2025-06-04
+
+### ✨ Features
+
+- Console methods (log, info, error, warn) now accept multiple arguments of any type, matching the standard JavaScript console API
+- Console methods automatically serialize objects, arrays, and other non-string values to strings for logging
+- Added support for circular reference detection in logged objects to prevent serialization errors
+
+## [v0.4.8] - 2025-05-22
+
+### 🐛 Bug Fixes
+
+- Removed temporary debug logging from error handler that was cluttering console output
+
+## [v0.4.7] - 2025-05-22
+
+### 🐛 Bug Fixes
+
+- Added temporary debug logging to error handler in step context to diagnose error serialization issues
+
+## [v0.4.6] - 2025-05-21
+
+### 🐛 Bug Fixes
+
+- Fixed step result retrieval to return raw cached results without unnecessary deserialization
+
+## [v0.4.5] - 2025-05-21
+
+### 🐛 Bug Fixes
+
+- Fixed serialization of step execution results and errors in workflow history to ensure consistent data format when persisting to storage
+
+## [v0.4.4] - 2025-05-21
+
+### 🐛 Bug Fixes
+
+- Fixed step reuse feature to correctly deserialize metadata and results when reusing successful steps from previous workflow runs
+
+## [v0.4.3] - 2025-05-21
+
+### 🐛 Bug Fixes
+
+- Added debug logging to step context creation to help diagnose workflow step reuse behavior
+
+## [v0.4.2] - 2025-05-21
+
+### 🐛 Bug Fixes
+
+- Added temporary debug logging to step reuse mechanism to help diagnose issues with workflow step caching
+
+## [v0.4.1] - 2025-05-21
+
+### ✨ Features
+
+- Added `reuseSuccessfulSteps` option to control whether workflow retries reuse results from successful steps in the parent instance (defaults to `true`)
+- Added `RetryWorkflowOptions` parameter to the `retry()` method, allowing granular control over retry behavior per workflow
+
+### 🐛 Bug Fixes
+
+- Fixed missing `parentInstanceId` being passed to step context during workflow execution
+
+## [v0.4.0] - 2025-05-20
+
+### 🐛 Bug Fixes
+
+- Fix input deserialization in workflow retry to use configured serializer instead of JSON.parse
+
+## [v0.3.8] - 2025-05-20
+
+### ✨ Features
+
+- Add `WorkflowContextInstance` type export for typed workflow context references
+- Update `createWorkflowContext()` to return `WorkflowContextInstance` for improved type safety
+
+### 💥 Breaking Changes
+
+- **QueueWorkflowContextOptions:** Replace `serializer` and `idFactory` options with `workflowContext` instance parameter - queue contexts now require a pre-configured workflow context instead of individual configuration options
+
+## [v0.3.7] - 2025-05-20
+
+*Version bump with no functional changes*
+
+## [v0.3.6] - 2025-05-20
+
+### ⚠️ BREAKING CHANGES
+
+- Remove `valueType` parameter from `setWorkflowProperty()` - type is now auto-detected from value
+
+## [v0.3.5] - 2025-05-19
+
+### 🐛 Bug Fixes
+
+- Remove unnecessary generic type parameter from `handleWorkflowQueueMessage` to fix type inference issues with workflow resolver
+
+## [v0.3.4] - 2025-05-19
+
+*Maintenance release with no user-facing changes*
+
+## [v0.3.3] - 2025-05-19
+
+### ✨ Features
+
+- Add `tenantId` to workflow context for multi-tenant support
+
+## [v0.3.2] - 2025-05-19
+
+### ✨ Features
+
+- Add simplified step syntax: `step('stepName', callback)` in addition to `step({ name, metadata }, callback)`
+
+## [v0.3.1] - 2025-05-19
+
+### ⚠️ BREAKING CHANGES
+
+- Change `createWorkflowContext()` from async to sync function (tables now created lazily on first execution)
+
+## [v0.3.0] - 2025-05-19
+
+### ✨ Features
+
+- Add default serializer and ID factory implementations
+- Make serializer parameter optional in `createLogAccessor()` with automatic fallback to defaults
+
+## [v0.2.2] - 2025-05-19
+
+*Minor release with internal improvements*
+
+## [v0.2.1] - 2025-05-19
+
+*Minor release with internal improvements*
+
+## [v0.2.0] - 2025-05-16
+
+### ✨ Features
+
+- Add parent-child workflow relationship support with `parentInstanceId` parameter
+
+## [v0.1.0] - 2025-05-16
+
+### ✨ Features
+
+- Add `defineWorkflow()` to define typed workflow functions with metadata
+- Add `createWorkflowContext()` for direct workflow execution with step tracking, logging, and retry capabilities
+- Add `createQueueWorkflowContext()` for queue-based workflow execution with batch enqueueing and dependency management
+- Add `createStepContext()` for granular step-level execution, logging, and result reuse
+- Add `createLogAccessor()` with comprehensive query methods: `listWorkflows()`, `getWorkflow()`, `listSteps()`, `getStep()`, and `getWorkflowTypesStatistics()`
+- Add workflow filtering with advanced query capabilities including property filters, date ranges, and text search
+- Add workflow dependency system for coordinating workflows that must complete before others start
+- Add workflow retry functionality with optional step result reuse
+- Add workflow properties system for attaching custom metadata to workflows
+- Add external blob storage support for large data that exceeds D1 size limits
+- Add retry configuration for D1 operations with exponential backoff and jitter
+- Add batch workflow enqueueing for efficient bulk operations (up to 100 workflows per batch)
+- Add `handleWorkflowQueueMessage()` for processing queued workflow execution messages
+- Add `handlePollWaitingWorkflows()` to transition waiting workflows to scheduled state when dependencies complete
+
+## [v0.0.2] - 2025-05-15
+
+*Minor release with internal improvements*

package/README.md

@@ -429,6 +429,10 @@ const typedWorkflow = defineWorkflow<MyInput>(
 )
 ```
 
+## Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md) for a detailed history of changes to this project.
+
 ## Contributing
 
 We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.

package/dist/index.d.ts

@@ -671,8 +671,8 @@ type WorkflowContextOptions = {
 };
 type InternalWorkflowContextOptions = WorkflowContextOptions & Required<Pick<WorkflowContextOptions, 'serializer' | 'idFactory'>>;
 type WorkflowContextInstance = {
-  call: <I, O>(args: WorkflowCallParams<I, O>) => Promise<void>;
-  retry: <I, O>(params: RetryWorkflowParams<I, O>) => Promise<void>;
+  call: <I, O>(args: WorkflowCallParams<I, O>) => Promise<WorkflowCallResult<O>>;
+  retry: <I, O>(params: RetryWorkflowParams<I, O>) => Promise<WorkflowCallResult<O>>;
 };
 type QueueWorkflowContextOptions = {
   QUEUE: Queue<WorkflowQueueMessage>;
@@ -778,6 +778,11 @@ type WorkflowCallParams<I, O> = {
   /** Optional: Provide an existing workflow instance ID. When provided, this workflow will update the existing instance. This is used when you have an workflow that is marked as waiting or scheduled and want to execute it. */
   scheduledInstanceId?: string | undefined;
 };
+type WorkflowCallResult<O, TYPE extends string = string> = Pick<TypedWorkflowRun<never, O, TYPE>, 'instanceId' | 'workflowType' | 'tenantId' | 'parentInstanceId' | 'triggerId' | 'startTime'> & {
+  result: O;
+  workflowStatus: 'completed';
+  endTime: number;
+};
 type ScheduledWorkflowExecutionStub<O, TYPE extends string = string> = {
   instanceId: string;
   workflowType: TYPE;
@@ -983,4 +988,4 @@ type R2ExternalBlobStorageOptions = {
  */
 declare function createR2ExternalBlobStorage(options: R2ExternalBlobStorageOptions): ExternalBlobStorage;
 //#endregion
-export { ConsoleWrapper, DateRangeFilter, ExternalBlobStorage, HandleWorkflowQueueMessageParams, InternalWorkflowContextOptions, Log, PossibleValueTypeNames, PossibleValueTypes, PropertyFilter, QueueWorkflowContextOptions, R2ExternalBlobStorageOptions, RetryConfig, Serializer, Step, StepContextOptions, StepCtx, StepWorkflowStatus, StringFilter, ValueTypeMap, WorkflowContext, WorkflowContextInstance, WorkflowContextOptions, WorkflowEnqueueBatchItem, WorkflowFilter, WorkflowFunction, WorkflowProperty, WorkflowPropertyDefinition, WorkflowQueueMessage, WorkflowRun, WorkflowStatus, createCleanupManager, createLogAccessor, createQueueWorkflowContext, createR2ExternalBlobStorage, createStepContext, createWorkflowContext, createWorkflowDependencies, createWorkflowDependency, defaultIdFactory, defaultSerializer, defineWorkflow, deleteWorkflowDependency, deserializeWithExternalStorage, ensureTables, finalizeWorkflowRecord, insertStepRecordFull, insertWorkflowRecord, pushLogToDB, retryD1Operation, serializeWithExternalStorage, tryDeserializeObj, updateWorkflow, updateWorkflowName, upsertWorkflowProperty, workflowTableRowToWorkflowRun };
+export { ConsoleWrapper, DateRangeFilter, ExternalBlobStorage, HandleWorkflowQueueMessageParams, InternalWorkflowContextOptions, Log, PossibleValueTypeNames, PossibleValueTypes, PropertyFilter, QueueWorkflowContextOptions, R2ExternalBlobStorageOptions, RetryConfig, Serializer, Step, StepContextOptions, StepCtx, StepWorkflowStatus, StringFilter, ValueTypeMap, WorkflowCallResult, WorkflowContext, WorkflowContextInstance, WorkflowContextOptions, WorkflowEnqueueBatchItem, WorkflowFilter, WorkflowFunction, WorkflowProperty, WorkflowPropertyDefinition, WorkflowQueueMessage, WorkflowRun, WorkflowStatus, createCleanupManager, createLogAccessor, createQueueWorkflowContext, createR2ExternalBlobStorage, createStepContext, createWorkflowContext, createWorkflowDependencies, createWorkflowDependency, defaultIdFactory, defaultSerializer, defineWorkflow, deleteWorkflowDependency, deserializeWithExternalStorage, ensureTables, finalizeWorkflowRecord, insertStepRecordFull, insertWorkflowRecord, pushLogToDB, retryD1Operation, serializeWithExternalStorage, tryDeserializeObj, updateWorkflow, updateWorkflowName, upsertWorkflowProperty, workflowTableRowToWorkflowRun };

package/dist/index.js

@@ -2320,6 +2320,17 @@ function createWorkflowContext(options) {
 				result
 			});
 			await logBatcher.destroy();
+			return {
+				instanceId,
+				workflowType: workflow.workflowType,
+				tenantId,
+				parentInstanceId: parentInstanceId ?? null,
+				triggerId: triggerId ?? null,
+				startTime,
+				result,
+				workflowStatus,
+				endTime
+			};
 		} catch (error) {
 			const endTime = Date.now();
 			const workflowStatus = "failed";
@@ -2345,7 +2356,7 @@ function createWorkflowContext(options) {
 		const encodedInput = oldRun?.input;
 		if (inputRef === void 0 || encodedInput === void 0) throw new Error(`No input found for instanceId ${retryInstanceId}`);
 		const input = await deserializeWithExternalStorage(encodedInput, inputRef, internalContext.serializer, options.externalBlobStorage);
-		await call({
+		return await call({
 			workflow,
 			input,
 			workflowName: oldWorkflowName ?? "unknown",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@brandboostinggmbh/observable-workflows",
-  "version": "0.22.0",
+  "version": "0.23.0",
   "description": "My awesome typescript library",
   "type": "module",
   "license": "MIT",
@@ -14,7 +14,8 @@
   },
   "author": "Tim <tim.stepanov@brand-boosting.de>",
   "files": [
-    "dist"
+    "dist",
+    "CHANGELOG.md"
   ],
   "main": "./dist/index.js",
   "module": "./dist/index.js",