@multiplier-labs/stepflow 0.3.3 → 0.3.4

package/.codekin/reports/code-review/2026-04-11_code-review-daily.md

@@ -0,0 +1,146 @@
+# Daily Code Review: stepflow
+
+**Date**: 2026-04-11T02:46:56.903Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: 592ed81f-2d62-4608-bf34-98470c42c55b
+**Session**: 99646d28-b2e3-4d3a-bcdf-1478c0bc9eff
+
+---
+
+Let me verify the most critical findings before writing the final report.Now I have enough detail to write the full report.
+
+---
+
+## Daily Automated Code Review — `@multiplier-labs/stepflow` v0.2.6
+
+**Date**: 2026-04-11 | **Commit**: `bd16961` (Initial commit — only commit in repo)
+
+---
+
+### Critical
+
+**None found.**
+
+---
+
+### Warnings
+
+**[W1] Checkpoint resume derives completedStepKeys from context keys — fragile assumption**
+`src/core/engine.ts:569`
+```typescript
+const completedStepKeys = new Set(Object.keys(run.context));
+```
+`run.context` stores step results keyed by step key. If a step writes to context before throwing, its key will appear in context but the step didn't complete. On resume, that step would be silently skipped. There is no separate `completedSteps` log to distinguish "wrote result" from "finished cleanly."
+**Recommendation**: Track a dedicated `completedSteps: string[]` field in the persisted run record, written atomically with the step result, and use that set during resume.
+
+---
+
+**[W2] `isBlockedIp` does not cover IPv6 reserved ranges beyond loopback and link-local**
+`src/events/webhook.ts:533–557`
+The resolved-IP check (`isBlockedIp`) handles IPv4 private ranges well, and blocks `::1` and `fe80:` for IPv6. However it does not block:
+- `fc00::/7` — Unique Local Addresses (ULA, the IPv6 equivalent of RFC 1918)
+- `ff00::/8` — Multicast
+- `2001:db8::/32` — Documentation range
+- `::ffff:0:0/96` — IPv4-mapped IPv6 addresses (could bypass IPv4 private range checks)
+
+`isBlockedHost` is more aggressive (blocks all bracket-wrapped IPv6 literals), so the static check is safe. But if `validateResolvedHost` receives a resolved IPv6 address in any format other than `[addr]`, the gap applies.
+**Recommendation**: In `isBlockedIp`, add IPv4-mapped check (`::ffff:` prefix) and ULA check (`/^f[cd]/i`).
+
+---
+
+**[W3] `enqueueRequest` doesn't guard against `maxConcurrentRequests <= 0`**
+`src/events/webhook.ts:475–479`
+```typescript
+if (this.activeRequests < this.maxConcurrentRequests) {
+  execute();
+} else {
+  this.requestQueue.push(execute);
+}
+```
+If `maxConcurrentRequests` is set to `0` (or negative), the condition is never true and all requests queue forever — effectively a deadlock. There is no constructor validation.
+**Recommendation**: Add `if (maxConcurrentRequests <= 0) throw new Error('maxConcurrentRequests must be > 0')` in the constructor.
+
+---
+
+**[W4] `withRetry` abort throws a generic `Error('Aborted')` instead of `WorkflowCanceledError`**
+`src/utils/retry.ts:83–85`
+```typescript
+if (opts.signal?.aborted) {
+  throw new Error('Aborted');
+}
+```
+The `sleep()` helper (line 58) correctly throws `WorkflowCanceledError` on abort, but the pre-attempt abort check throws a plain `Error`. Downstream error classification (e.g. `instanceof WorkflowCanceledError`) will fail for aborts caught before the first sleep.
+**Recommendation**: Change to `throw new WorkflowCanceledError('run')` to match the `sleep()` path.
+
+---
+
+**[W5] Hook errors log without run context**
+`src/core/orchestrator.ts:258`
+```typescript
+logger.error('afterRun hook failed:', hookError);
+```
+The error is intentionally swallowed (correct behavior), but the log line has no `runId` or `kind`, making production debugging unnecessarily hard.
+**Recommendation**: `logger.error(\`afterRun hook failed for run ${runId}:\`, hookError)`
+
+---
+
+**[W6] Integration and edge-case test coverage gaps**
+
+No integration tests exercise the full lifecycle across multiple adapters. Specific gaps:
+- Resume of a partially-completed run (directly relevant to W1 above)
+- `maxConcurrency` queuing with priority ordering
+- Webhook retries under network failure simulations
+- Scheduler DST-boundary cron behavior
+- Hook error recovery scenarios (`beforeRun`, `afterRun`, `onStepError` throwing)
+- SQLite transaction rollback verification
+
+The current test suite is unit-level and covers the happy path well, but these edge cases are where durable-execution bugs hide.
+
+---
+
+### Info
+
+**[I1] CI workflow publishes without running tests**
+`.github/workflows/publish.yml:23–29`
+The publish job runs `npm ci` → `npm run build` → `npm publish`. There is no `npm test` or `npm run typecheck` step. A broken release could be published if tests are skipped.
+**Recommendation**: Add `- run: npm test` and `- run: npm run typecheck` between Install and Publish steps.
+
+---
+
+**[I2] `calculateRetryDelay` is defined but never called internally**
+`src/utils/retry.ts:115–121`
+`withRetry` computes delay inline (`currentDelay * opts.backoff`) rather than calling `calculateRetryDelay`. The exported helper is a public API, but the internal inconsistency means the two can drift.
+**Recommendation**: Refactor `withRetry` to call `calculateRetryDelay` for the delay computation, or document that they are intentionally separate.
+
+---
+
+**[I3] Peer dependency lower bounds are not tested**
+`package.json:57–71`
+`better-sqlite3>=11`, `kysely>=0.27`, `pg>=8.13` are wide ranges. Only the devDependency pinned versions are tested. If a consumer uses the minimum allowed version and it has a breaking API difference, the error surface is silent.
+**Recommendation**: Add a CI matrix job that installs minimum peer dependency versions and runs the test suite.
+
+---
+
+**[I4] Short webhook secrets not warned against**
+`src/events/webhook.ts:24`
+The `secret` field accepts any string. A very short secret (< 16 bytes) produces a weak HMAC. The JSDoc example (`process.env.WEBHOOK_SECRET`) is good, but there is no runtime warning or minimum-length enforcement.
+**Recommendation**: Add a `secret.length < 16` warning log in `signPayload` at minimum, or throw in the constructor.
+
+---
+
+**[I5] `retryDelay` in webhook config is not applied with backoff**
+`src/events/webhook.ts` — webhook retries use a fixed `retryDelay` with no backoff, while the core `withRetry` utility supports exponential backoff. This inconsistency means repeated webhook failures hammer the endpoint at a fixed rate.
+**Recommendation**: Either apply backoff to webhook retries (using the existing `withRetry` utility) or document the fixed-delay intent.
+
+---
+
+### Summary
+
+| Severity | Count |
+|---|---|
+| Critical | 0 |
+| Warning | 6 |
+| Info | 5 |
+
+The codebase is well-structured with good TypeScript discipline, solid SSRF protection, and thoughtful SQLite query design. The top priorities are **W1** (checkpoint resume correctness) and **W4** (abort error type mismatch), both of which are correctness bugs rather than just style issues. **W2** is a security hardening gap. The CI publish-without-tests gap (**I1**) is a process risk worth closing quickly.

package/.codekin/reports/code-review/2026-04-13_code-review-daily.md

@@ -0,0 +1,74 @@
+# Daily Code Review: stepflow
+
+**Date**: 2026-04-13T01:32:56.537Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: 12fb929b-3da3-4e1b-a8fc-e9b497a0eb42
+**Session**: c3af596a-a1f0-43c3-922e-8ed0cd0562f0
+
+---
+
+---
+
+## Daily Code Review — 2026-04-13
+
+**Project:** `@multiplier-labs/stepflow` — TypeScript workflow orchestration library
+**Coverage:** 93.88% | **Recent changes:** Initial commit only (no incremental changes in 7 days)
+
+---
+
+### CRITICAL (4)
+
+**C1 — Unhandled `JSON.parse()` in all storage adapters**
+`src/storage/sqlite.ts`, `src/storage/postgres.ts`, `src/scheduler/*-persistence.ts`
+Every row deserialization call is bare `JSON.parse()`. Corrupted DB rows will throw unhandled `SyntaxError` and crash the process. Wrap all calls in try-catch with safe fallbacks.
+
+**C2 — DNS lookup blocks event loop on every webhook send**
+`src/events/webhook.ts` ~line 417
+`validateResolvedHost()` calls `dns.lookup()` with no timeout, no caching. A stalling DNS resolver can hang the entire event loop for 30+ seconds. Add a per-call timeout (via `AbortController`) and a short-TTL cache.
+
+**C3 — No `shutdown()` method; timers are untracked**
+`src/core/engine.ts` ~lines 369–374
+`setTimeout` / `setImmediate` handles in `launchRun()` are never stored or cleared. The process cannot exit cleanly while delayed runs are pending. Implement `WorkflowEngine.shutdown()` that cancels all tracked handles.
+
+**C4 — Race condition in `waitForRun`**
+`src/core/engine.ts` ~lines 480–527
+Storage is polled _before_ subscribing to events. A run completing in that window emits an event that is never received, causing an infinite wait. Fix: subscribe first, then check storage; unsubscribe immediately if terminal.
+
+---
+
+### WARNING (10)
+
+| # | File | Issue |
+|---|------|-------|
+| W1 | `package.json` deps | `npm audit`: `picomatch` (HIGH ReDoS) + `brace-expansion` (MODERATE hang). Run `npm audit fix`. |
+| W2 | `src/storage/postgres.ts:13-16` | 48 `any` casts — concentrated in lazy Kysely imports. Use conditional types or `unknown` + guards. |
+| W3 | `src/events/webhook.ts` | Webhook secret length not enforced despite docs recommending ≥ 32 bytes. Validate in `addEndpoint()`. |
+| W4 | `src/events/webhook.ts` | DNS validation is outside the retry loop — DNS failures skip all retry logic. Move inside loop. |
+| W5 | `src/scheduler/cron.ts:96-97` | `start()` after `stop()` can create duplicate subscriptions. Unconditionally unsubscribe at top of `start()`. |
+| W6 | `src/storage/postgres.ts` | PG connection pool not closed if adapter goes out of scope without explicit `close()`. Document + warn. |
+| W7 | `src/core/engine.ts:405-412` | No depth/count limit on `spawnChild()` — recursive workflows can spawn unboundedly. Add `maxChildDepth`. |
+| W8 | `src/core/orchestrator.ts:87-93` | `emitEvent()` called without error handling — transport failures can abort workflow execution. Wrap in try-catch. |
+| W9 | `src/scheduler/cron.ts` | Invalid persisted cron strings crash the scheduler on startup. Catch `CronExpressionParser.parse()` errors. |
+| W10 | `src/storage/sqlite.ts` | SQLite single-writer limitation not documented. Add prominent warning; recommend Postgres for multi-worker. |
+
+---
+
+### INFO (5)
+
+- **I1** — Test files are very large (engine.test.ts: 1364 lines). Split by feature for maintainability.
+- **I2** — Resume/checkpoint semantics undocumented for distributed use (single-process only).
+- **I3** — `registerWorkflow()` doesn't validate `kind` format or duplicate step keys.
+- **I4** — Error stacks sanitized on write but not on read — defense-in-depth gap.
+- **I5** — No circuit breaker for consistently failing webhook endpoints.
+
+---
+
+**Top 5 priorities:**
+1. `JSON.parse()` error handling in all storage adapters (C1)
+2. `WorkflowEngine.shutdown()` with timer cleanup (C3)
+3. Fix subscribe-then-check ordering in `waitForRun` (C4)
+4. `npm audit fix` for dep vulnerabilities (W1)
+5. DNS caching + timeout in webhook transport (C2/W4)
+
+Report saved to `.codekin/reports/code-review/2026-04-13_code-review-daily.md`.

package/.codekin/reports/comments/2026-04-11_comment-assessment.md

@@ -0,0 +1,161 @@
+# Comment Assessment: stepflow
+
+**Date**: 2026-04-11T02:48:00.194Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: 98e67949-d200-491e-8d2e-333880bba263
+**Session**: 1f4952f0-14a9-4677-a164-d0cd05cc6d35
+
+---
+
+## Summary
+
+**Stepflow** is a TypeScript workflow orchestration library (~43 source files, ~30 non-test). Overall documentation quality is **Good to Excellent**, with approximately **67% of source files** carrying JSDoc/TSDoc comments. Public APIs are thoroughly documented; the core engine and utility modules set a high bar. Test files carry no comments, which is appropriate given self-descriptive `describe`/`it` naming. The most notable gaps are confined to a handful of interface implementation methods in the scheduler persistence layer, and two non-obvious semantic behaviors in the planning module that lack any explanatory comment.
+
+**Rating: 8/10** — above industry average; minor targeted additions would close all meaningful gaps.
+
+---
+
+## Well-Documented Areas
+
+### `src/index.ts`
+A 56-line module-level JSDoc block enumerates every major feature (workflow registration, async execution, state management, event system, pluggable storage, retry/error handling) followed by a complete 45-line working code example. Serves as the library's living documentation entry point.
+
+### `src/core/engine.ts` (693 lines)
+Every public method includes `@param`, `@returns`, and `@throws` tags. Logical sections are separated by banner dividers (`// ========…`). Example:
+```ts
+/**
+ * Register a workflow definition with the engine.
+ * @param definition - The workflow definition to register.
+ * @throws {WorkflowEngineError} If a workflow with the same kind is already registered.
+ */
+registerWorkflow(definition: WorkflowDefinition): void
+```
+
+### `src/core/orchestrator.ts` — timeout / memory-leak rationale (lines 482–518)
+The `executeWithTimeout` helper explains *why* the `finally` block exists, not just what it does:
+```ts
+/**
+ * …The `finally` block is critical: without it, the losing promise's timer/listener
+ * would remain active, leaking memory — especially problematic in long-running engines
+ * processing many steps.
+ */
+```
+This is exemplary "why" commenting.
+
+### `src/utils/errors.ts`
+Every custom error class carries a one-sentence JSDoc stating its purpose and the specific condition that triggers it.
+
+### `src/utils/retry.ts`
+`RetryOptions` fields are individually documented. The exponential backoff formula includes an example showing calculated delays at each attempt.
+
+### `src/core/types.ts`
+Every interface field has a one-line `/** … */` doc comment. `WorkflowContext`, `WorkflowDefinition`, `WorkflowHooks`, `RunStatus`, and `StepStatus` are all self-contained.
+
+### `src/storage/postgres.ts` — schema type docs (lines 46–128)
+Kysely row-type interfaces (`WorkflowRunsTable`, `WorkflowRunStepsTable`, etc.) document every column with field-level comments explaining nullability, units, and JSON encoding:
+```ts
+/** JSON-serialized output payload, null until run completes. */
+output_json: string | null;
+/** Execution priority (lower number = higher precedence). */
+priority: number;
+```
+
+### `src/scheduler/postgres-persistence.ts` — class-level JSDoc with two `@example` blocks
+Includes a basic usage example and a "Sharing connection pool" example, covering the two main integration patterns.
+
+---
+
+## Underdocumented Areas
+
+| File | Issue | Severity |
+|------|-------|----------|
+| `src/planning/planner.ts` | `evaluateCondition()` (line 44): uses `RE2` instead of native `RegExp` for ReDoS safety — security-relevant choice is entirely uncommented | **High** |
+| `src/planning/planner.ts` | `exists`/`notExists` operators (lines 95–99): treat empty string `''` as non-existent — opinionated semantic behaviour with no comment | **High** |
+| `src/scheduler/postgres-persistence.ts` | `loadSchedules()` (line 306), `saveSchedule()` (line 316), `updateSchedule()` (line 340), `deleteSchedule()` (line 402): public interface methods with no JSDoc | **Medium** |
+| `src/planning/planner.ts` | `applyConstraints()` (line 466): `perStepTimeout = maxDuration / steps.length` — evenly distributes budget across steps without noting the equal-duration assumption | **Medium** |
+| `src/planning/planner.ts` | `RuleBasedPlanner` class (line 170): single-line class JSDoc; scoring algorithm overview (condition-score vs priority-score vs forced/preferred-variant paths) not explained | **Medium** |
+| `src/scheduler/postgres-persistence.ts` | `rowToSchedule()` (line 480): dual JSON.parse path (`typeof … === 'string'` guard) is non-obvious with no comment explaining why it handles both string and pre-parsed JSONB | **Medium** |
+| `src/storage/postgres.ts` | `StepflowDatabase` interface (line 130): no doc comment explaining that this is the Kysely schema map tying table names to row types | **Low** |
+| `src/scheduler/postgres-persistence.ts` | `ensureInitialized()` (line 166): private guard method with no doc comment | **Low** |
+| `src/planning/planner.ts` | `scoreConditions()` scoring formula (lines 127–134) has inline comments but no JSDoc — callers of the public `RuleBasedPlanner` API cannot understand selection scoring from public docs alone | **Low** |
+| `src/storage/postgres.ts` | `stripStack()` helper (line 37): has a one-liner but does not explain *why* `stack` is stripped (prevent large stack strings from bloating persisted JSON) | **Low** |
+| `src/utils/id.ts` | `generateId()` explains ULID advantages but omits the practical consequence: IDs are lexicographically sortable by creation time | **Low** |
+| `src/events/memory.ts` | Channel naming convention explained in code (lines 34–41) but not in any exported type or JSDoc for consumers building custom transports | **Low** |
+| `src/scheduler/sqlite-persistence.ts` | `createTables()` has schema comments; `loadSchedules()`, `saveSchedule()`, `deleteSchedule()` interface methods have no JSDoc (same gap as postgres counterpart) | **Low** |
+| `src/planning/registry.ts` | `MemoryRecipeRegistry` class has no class-level JSDoc; only individual methods are commented | **Low** |
+| `src/core/orchestrator.ts` | `WorkflowCheckpoint` interface is documented, but no comment explains when/why checkpoints are created (only on step completion, not at arbitrary points) | **Low** |
+
+---
+
+## Comment Quality Issues
+
+No clearly **inaccurate** or **misleading** comments were found. The following are cases where comments exist but create a gap between stated and actual behaviour:
+
+### 1. `src/planning/planner.ts:95–99` — `exists`/`notExists` undocumented empty-string treatment
+```ts
+case 'exists':
+  return fieldValue !== undefined && fieldValue !== null && fieldValue !== '';
+
+case 'notExists':
+  return fieldValue === undefined || fieldValue === null || fieldValue === '';
+```
+The corresponding type definition in `src/planning/types.ts` describes `exists` as "field exists and is non-null." It does not mention that `''` is also treated as absent. A consumer who stores an empty string legitimately would get a surprising false from `exists`. No comment at either the operator definition or the implementation site flags this.
+
+### 2. `src/planning/planner.ts:85–93` — RE2 usage is silent
+```ts
+case 'matches':
+  if (typeof fieldValue === 'string' && typeof conditionValue === 'string') {
+    try {
+      return new RE2(conditionValue).test(fieldValue);
+    } catch {
+      return false;
+    }
+  }
+```
+`RE2` is a linear-time regex engine that prevents ReDoS attacks — a meaningful security property. The comment on `evaluateCondition` (line 42) says only "Evaluate a single condition against a value." Callers writing user-supplied patterns need to know RE2 syntax differs from ECMA regex (e.g., no lookaheads), and security-conscious reviewers need to know the DoS protection is intentional.
+
+### 3. `src/scheduler/postgres-persistence.ts:387` — condition always true for `domainKey`
+```ts
+if (updates[domainKey] !== undefined || merged[domainKey] !== undefined) {
+```
+Because `merged` is `{ ...existing, ...updates }`, `merged[domainKey]` is always defined for fields present in the existing row. The inline comment above (lines 342–345) correctly describes intent ("only included if explicitly provided"), but the condition as written effectively includes every field on every update. The comment describes the goal, not the actual behaviour — a subtle mislead.
+
+---
+
+## Recommendations
+
+1. **`src/planning/planner.ts:85–93` — Document RE2's ReDoS safety in `evaluateCondition`**
+   Add to the `matches` case a brief comment: `// RE2 provides linear-time matching, preventing ReDoS attacks from user-supplied patterns.` Also note in the `evaluateCondition` JSDoc that RE2 syntax is used (no lookaheads, no backreferences), so callers know pattern compatibility constraints.
+
+2. **`src/planning/planner.ts:95–99` and `src/planning/types.ts` — Align `exists` docs with empty-string behaviour**
+   Either update the `ConditionOperator` type definition to read "field is non-null and non-empty-string," or add a `// NOTE:` inline explaining the empty-string-as-absent convention. Doing both ensures consumers reading either the type or the implementation get consistent expectations.
+
+3. **`src/scheduler/postgres-persistence.ts:306–408` — Add JSDoc to all four public `SchedulePersistence` interface methods**
+   `loadSchedules`, `saveSchedule`, `updateSchedule`, and `deleteSchedule` are part of the persistence contract but have no documentation. At minimum add `@returns` and `@throws` tags to match the documented `getSchedule`, `getDueSchedules`, and `getCompletionTriggers` below them. Apply the same fix to `src/scheduler/sqlite-persistence.ts` for consistency.
+
+4. **`src/planning/planner.ts:170` — Expand `RuleBasedPlanner` class JSDoc**
+   The current one-line class comment ("Rule-based planner that selects recipes based on condition matching") does not explain the selection algorithm. Add a paragraph describing the scoring hierarchy: forced recipe → preferred variant → condition scoring (50 + 10×N conditions, capped at 100) → default recipe → first available. This allows users to understand and predict selection without reading the source.
+
+5. **`src/planning/planner.ts:462–468` — Document the per-step timeout distribution assumption**
+   ```ts
+   const perStepTimeout = constraints.maxDuration
+     ? Math.floor(constraints.maxDuration / steps.length)
+     : undefined;
+   ```
+   Add a comment: `// Budget is divided equally across steps; steps with inherently longer durations may time out even if the overall run would complete within maxDuration.` This sets expectations and helps users tune `maxDuration` correctly.
+
+6. **`src/scheduler/postgres-persistence.ts:480` — Document the dual JSON.parse guard in `rowToSchedule`**
+   The pattern `typeof row.X === 'string' ? JSON.parse(row.X) : row.X` appears six times without explanation. Add a single comment near the first occurrence: `// Kysely returns JSONB columns pre-parsed in some drivers; guard ensures correct deserialization regardless of driver behaviour.`
+
+7. **`src/planning/types.ts` — Document each `ConditionOperator` value in the enum/union**
+   The `ConditionOperator` type has a comment block, but individual values like `'matches'`, `'contains'`, `'exists'` have no per-value docs. Add a TSDoc `@remarks` or switch to a documented string union listing what each operator accepts (e.g., `'matches'` works on strings using RE2 syntax; `'contains'` works on both strings and arrays).
+
+8. **`src/core/orchestrator.ts` — Document when checkpoints are written**
+   `WorkflowCheckpoint` is documented, but `executeWorkflow` does not explain that checkpoints capture state only after a step fully completes. Add a note in the `ExecuteOptions.checkpoint` field doc: `// Checkpoint is built incrementally as steps complete; a partially executed step is NOT checkpointed and will re-run on resume.`
+
+9. **`src/storage/postgres.ts:130` — Add JSDoc to `StepflowDatabase`**
+   This interface is the Kysely schema map. A one-line comment — `/** Kysely database schema mapping table names to their row types. */` — makes the architectural role immediately clear to contributors adding new tables.
+
+10. **`src/utils/id.ts` — Note sort-by-creation-time property in `generateId` JSDoc**
+    The existing comment explains ULID advantages but omits the most practically useful one: generated IDs are lexicographically sortable by creation time, making them suitable as primary keys with implicit time-ordering. Adding this helps consumers decide whether to add a separate `created_at` sort column.

package/.codekin/reports/comments/2026-04-14_comment-assessment.md

@@ -0,0 +1,34 @@
+# Comment Assessment: stepflow
+
+**Date**: 2026-04-14T01:18:27.171Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: af53bfdc-9706-4e22-bafa-e5954c9c9875
+**Session**: 27222af5-7c69-4c33-af75-7ad0318122a4
+
+---
+
+## Summary
+
+**Overall rating: A+ (Excellent)**
+
+The Stepflow codebase has outstanding documentation practices. The report has been saved to `.codekin/reports/comments/2026-04-14_comment-assessment.md`.
+
+Key findings:
+
+**Strengths (~94% coverage):**
+- Every exported function, interface, class, and type carries JSDoc
+- File-level module comments in 19/20 sampled files
+- Critical complex logic (abort handling, memory leak prevention, retry backoff) is thoroughly explained
+- Consistent use of `@param`, `@returns`, `@throws`, `@example` throughout
+- Zero stale, outdated, or misleading comments detected
+- External docs (`README.md`, `docs/stepflow-api-reference.md`) are comprehensive
+
+**Minor gaps (all Low/Medium severity):**
+- `QueuedRun` internal interface in `engine.ts` lacks JSDoc
+- `MemoryStorageAdapter` methods have one-liner summaries but no `@param`/`@returns`
+- Test files have no file-level comments
+- `postgres.ts` and persistence files not fully verified for comment parity
+- ID generation strategy in `utils/id.ts` undocumented
+
+Top recommendations: add JSDoc to `QueuedRun`, document the SQLite statement cache strategy, audit Postgres implementation for parity with SQLite docs, and add `@param`/`@returns` to storage implementation methods.

package/.codekin/reports/complexity/2026-04-11_complexity-report.md

@@ -0,0 +1,152 @@
+# Complexity Report: stepflow
+
+**Date**: 2026-04-11T02:46:40.781Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: 7c4174ae-8cba-4ad8-9a30-b2f56cf4c98b
+**Session**: 474ba458-5157-4ff3-8ef4-802b1bc389e5
+
+---
+
+Now I have enough information to produce the report.
+
+## Summary
+
+**Overall Complexity Rating: Medium**
+
+The codebase is a TypeScript workflow orchestration library (~14,300 lines across source + tests). The implementation is generally clean and well-structured, but several files are oversized, the storage layer has two parallel interface lineages with significant duplication, and the orchestrator's `executeStep` function carries the highest cyclomatic complexity. No deeply pathological nesting exists, but coupling between the `postgres.ts` adapter and two different storage interfaces is the most acute maintenance risk.
+
+Key metrics:
+- **Largest file:** `src/storage/postgres.ts` — 1,354 lines
+- **Deepest nesting:** `executeStep` in `orchestrator.ts` — 4–5 levels (`while → try → if → if → try`)
+- **Most complex function:** `executeStep` — retry loop, cancellation guards, hooks, event emission, 4 error-handling branches
+- **Duplicate interface footprint:** `StorageAdapter` vs. `WorkflowStorage` — two complete CRUD contracts implemented in the same `postgres.ts` class
+
+---
+
+## Largest Files
+
+| File | Lines | Primary Responsibility | Refactor Priority |
+|------|-------|------------------------|-------------------|
+| `src/storage/postgres.ts` | 1,354 | PostgreSQL adapter (implements both `StorageAdapter` and `WorkflowStorage`) | **High** |
+| `src/core/engine.test.ts` | 1,364 | Engine integration tests | Low (test file) |
+| `src/planning/planning.test.ts` | 727 | Planner integration tests | Low (test file) |
+| `src/events/webhook.test.ts` | 657 | Webhook transport tests | Low (test file) |
+| `src/core/engine.ts` | 692 | `WorkflowEngine` class — run lifecycle, queue, resume | Medium |
+| `src/storage/sqlite.ts` | 621 | SQLite adapter | Medium |
+| `src/planning/planner.ts` | 576 | `RuleBasedPlanner` — recipe selection, plan generation | Low |
+| `src/planning/types.ts` | 566 | Planning domain types (Recipe, Plan, Planner interfaces) | Low |
+| `src/core/orchestrator.ts` | 564 | Step execution loop, retry logic, hooks | **High** |
+| `src/events/webhook.ts` | 557 | WebhookEventTransport — SSRF guards, signing, concurrency | Medium |
+| `src/scheduler/postgres-persistence.ts` | 512 | PostgreSQL schedule persistence | Low |
+| `src/scheduler/cron.ts` | 414 | CronScheduler — polling + event-driven triggers | Low |
+| `src/storage/types.ts` | 342 | Storage type definitions (two parallel interface families) | **High** |
+| `src/events/socketio.ts` | 262 | Socket.IO event transport | Low |
+| `src/index.ts` | 270 | Public API barrel file | Low |
+
+---
+
+## Most Complex Functions
+
+| File:Function | Estimated Complexity | Issue Description | Refactor Suggestion |
+|---------------|---------------------|-------------------|---------------------|
+| `orchestrator.ts:executeStep` | Very High (CC ~12) | Single `while(true)` loop wraps: `AbortSignal` check, step record creation, `beforeStep` hook, conditional timeout vs. `raceWithAbort`, `afterStep` hook, cancellation detection, `onStepError` hook, three error branches (`skip`/`retry`/`fail`), sleep-with-abort, and re-throw. 4–5 nesting levels. | Extract `handleStepError(error, attempt, strategy)` and `executeStepOnce(step, context, options)` as separate functions |
+| `orchestrator.ts:executeWorkflow` | High (CC ~8) | Combines run status update, event emission, context construction, timeout setup, `beforeRun` hook, step loop with skip checks, `afterRun` hook, success/failure/timeout/cancel branching in one try/catch | Split success path and failure classification into `finalizeSuccess()` / `finalizeFailure()` helpers |
+| `storage/postgres.ts:createTables` | High (CC ~7) | 150-line method issues 15+ raw SQL statements sequentially with ad-hoc `catch(() => {})` for ALTER TABLE idempotency. No migration versioning. | Replace with a proper migration runner (numbered migration files or Kysely-migrate) |
+| `storage/postgres.ts:mapRunRow` | Medium (CC ~6) | Repeated `typeof x === 'string' ? JSON.parse(x) : x` ternary for every JSONB column (6 occurrences in `mapRunRow` alone, duplicated in `mapExtendedRunRow`) | Extract `parseJsonColumn<T>(col: string | T): T` helper; used in all `mapXRow` methods |
+| `events/webhook.ts:sendWebhook` | Medium (CC ~6) | Retry loop with DNS validation re-run on every attempt, payload size check, dynamic header assembly, HMAC signing, and `AbortController` per attempt | Extract `deliverOnce(url, headers, body, timeout)` and call from `sendWebhookWithRetry` |
+| `events/webhook.ts:isBlockedHost` | Medium (CC ~9) | Flat function with 9 early-return branches testing IPv4 ranges, IPv6 literals, loopback, link-local — logic duplicated almost identically in `isBlockedIp` | Merge `isBlockedHost` and `isBlockedIp` into a single `isBlockedAddress(address: string)` that handles both hostname and resolved-IP cases |
+| `planner.ts:selectRecipe` | Medium (CC ~6) | Three sequential priority checks (forced, preferred, scored) each with early returns, plus a fallback chain of two more returns | Already reasonably structured; document the scoring ladder with inline comments |
+| `storage/postgres.ts:updateRun` | Medium (CC ~5) | Builds `updateData` object via 6 independent `if` guards covering two different input shapes (`UpdateRunInput` and `Partial<WorkflowRunRecord>`) simultaneously | Accept only `UpdateRunInput`; deprecate the `Partial<WorkflowRunRecord>` overload |
+| `engine.ts:waitForRun` | Medium (CC ~5) | Interleaves early-return for already-completed runs with Promise-based event subscription, dual cleanup via `cleanup()` closure, nested `try/catch` inside the event callback | Extract the Promise setup into a `awaitRunEvent(runId, timeout)` utility |
+| `planner.ts:evaluateCondition` | Low-Medium (CC ~11) | Large `switch` on 9 operators; each case is self-contained but the function is called in a hot loop for every condition on every recipe | Acceptable as-is; consider a `conditionEvaluators: Record<ConditionOperator, Evaluator>` map if operators are user-extensible |
+
+---
+
+## Coupling & Cohesion Issues
+
+**1. `storage/postgres.ts` implements two unrelated storage interfaces**
+
+`PostgresStorageAdapter` simultaneously implements `StorageAdapter` (the legacy interface with `WorkflowRunRecord`, `WorkflowRunStepRecord`) and `WorkflowStorage` (the newer interface with `ExtendedWorkflowRunRecord`, `StepResult`). This means `postgres.ts` contains two nearly identical sets of CRUD methods (`createRun`/`getRun`/`updateRun` in both shapes), two `mapRunRow` functions (`mapRunRow` and `mapExtendedRunRow`) that differ only in the target type, and two separate status type hierarchies (`RunStatus` vs. `ExtendedRunStatus`). Any schema change must be applied in multiple places.
+
+*Suggested fix:* Decide on one canonical interface. If `WorkflowStorage` is the future, mark `StorageAdapter` as deprecated and migrate callers. Provide an adapter shim for backward compatibility during the transition rather than carrying both implementations indefinitely.
+
+**2. `storage/types.ts` defines two parallel type families**
+
+`StorageAdapter` + `WorkflowRunRecord`/`WorkflowRunStepRecord` and `WorkflowStorage` + `ExtendedWorkflowRunRecord`/`StepResult` overlap heavily. `ExtendedRunStatus` adds `'pending'` and `'timeout'` to `RunStatus`; `ExtendedStepStatus` renames `'succeeded'` to `'completed'`. These differences are not documented as deliberate — they appear to be the result of incremental additions without consolidation.
+
+*Suggested fix:* Unify the status enums (`RunStatus` should include `'timeout'`; step status should use consistent terminology). Remove the `Extended*` types once the single interface is adopted.
+
+**3. `engine.ts` holds both scheduling concerns and run lifecycle concerns**
+
+`WorkflowEngine` manages the in-memory run queue with priority insertion (`queueRun`, `processQueue`), active-run tracking (`activeRuns` map), child spawning delegation, event subscription forwarding, and shutdown. This is reasonable but the queue and concurrency control logic (`hasCapacity`, `queueRun`, `processQueue`) could become an independent `RunQueue` class, making `WorkflowEngine` a thinner coordinator.
+
+**4. `events/webhook.ts` duplicates IP-range logic across two functions**
+
+`isBlockedHost` and `isBlockedIp` both check the same IPv4 private ranges (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x, 0.0.0.0) with near-identical code. A future range addition or bug fix must be applied twice.
+
+**5. `scheduler/postgres-persistence.ts` repeats the Kysely lazy-load pattern from `storage/postgres.ts`**
+
+Both files declare module-level `let Kysely`, `let PostgresDialect`, `let sql`, `let pgModule` and both call `loadPostgresDeps()` in `initialize()`. This cross-file duplication of boilerplate means any change to the lazy-loading pattern (e.g., adding a new exported symbol) must be applied in two places.
+
+*Suggested fix:* Centralize the Kysely instantiation in `utils/postgres-deps.ts` and return a ready-to-use `Kysely` instance, rather than returning the constructor for callers to instantiate manually.
+
+---
+
+## Refactoring Candidates
+
+**1. Consolidate dual storage interfaces (`StorageAdapter` / `WorkflowStorage`)**
+
+- **Location:** `src/storage/types.ts`, `src/storage/postgres.ts`
+- **Problem:** Two overlapping interface families with near-duplicate type definitions, status enums that differ by one or two values, and two sets of CRUD methods in `PostgresStorageAdapter`. Every schema or status change requires dual edits.
+- **Suggested approach:** Choose `WorkflowStorage` as the single target. Introduce a `LegacyStorageAdapterBridge` shim that wraps a `WorkflowStorage` into the `StorageAdapter` shape for backward compatibility. Unify `RunStatus` to include `'timeout'`; unify step status terminology to `'succeeded'`. Remove `Extended*` type aliases.
+- **Effort:** Large
+
+**2. Extract step execution helpers from `orchestrator.ts:executeStep`**
+
+- **Location:** `src/core/orchestrator.ts:296–479`
+- **Problem:** The `executeStep` function is a 183-line `while(true)` loop combining timeout handling, cancellation, hook invocation, event emission, and three error recovery strategies. High cyclomatic complexity makes it difficult to unit-test individual error paths.
+- **Suggested approach:** Extract `executeStepHandler(step, context, abortController)` (pure execution + timeout), `handleStepSuccess(stepRecord, ...)`, and `handleStepError(step, error, attempt, strategy, ...)` (retry/skip/fail branching). `executeStep` becomes a thin coordinator.
+- **Effort:** Medium
+
+**3. Merge `isBlockedHost` and `isBlockedIp` into one utility**
+
+- **Location:** `src/events/webhook.ts:487–555`
+- **Problem:** Two functions with near-identical IPv4-range checks. A missed range in one is a security regression.
+- **Suggested approach:** Consolidate into `isBlockedAddress(address: string): boolean` operating on dotted-decimal IPv4 or hostname strings. Call it from both `validateWebhookUrl` and `validateResolvedHost`.
+- **Effort:** Small
+
+**4. Extract a `parseJsonColumn` helper for storage mappers**
+
+- **Location:** `src/storage/postgres.ts:861–940`, `src/storage/sqlite.ts`
+- **Problem:** The pattern `typeof col === 'string' ? JSON.parse(col) : col` appears 10+ times across `mapRunRow`, `mapExtendedRunRow`, `mapStepRow`, `mapStepResultRow`, and `mapEventRow` in postgres.ts, and again in sqlite.ts's mapper.
+- **Suggested approach:** `function parseJsonColumn<T>(col: string | T | null | undefined, fallback?: T): T | undefined` — extracted to a shared `src/storage/utils.ts`. Eliminates all inline ternaries.
+- **Effort:** Small
+
+**5. Replace `createTables` ad-hoc migration with a versioned migration runner**
+
+- **Location:** `src/storage/postgres.ts:314–465`
+- **Problem:** The 150-line `createTables` method issues 15+ raw SQL statements and silently swallows `ALTER TABLE` errors as idempotency. There is no migration version tracking, so it is impossible to determine the current schema version or safely evolve the schema.
+- **Suggested approach:** Introduce a `schema_migrations` table tracking applied versions. Replace the `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` hack with numbered migration files applied sequentially. This also enables rollback logic.
+- **Effort:** Large
+
+**6. Refactor `WorkflowEngine` queue management into a `RunQueue` class**
+
+- **Location:** `src/core/engine.ts:112–399`
+- **Problem:** `WorkflowEngine` mixes run-lifecycle concerns (start, cancel, resume, wait) with an in-memory priority queue implementation (`queueRun`, `processQueue`, `hasCapacity`, `runQueue` array, `activeRuns` map). As concurrency features grow, this class will accumulate more queue-specific logic.
+- **Suggested approach:** Extract a `RunQueue` class that owns `runQueue`, `activeRuns`, `hasCapacity`, `queueRun`, `processQueue`, and the `QueuedRun` interface. `WorkflowEngine` delegates to it.
+- **Effort:** Medium
+
+**7. Centralize Kysely lazy-loading in `utils/postgres-deps.ts`**
+
+- **Location:** `src/utils/postgres-deps.ts`, `src/storage/postgres.ts:13–16`, `src/scheduler/postgres-persistence.ts:12–15`
+- **Problem:** Module-level `let Kysely`, `let PostgresDialect`, `let sql`, `let pgModule` declarations and `loadPostgresDeps()` calls are copy-pasted into two different files, and the callers then call `new Kysely(...)` themselves.
+- **Suggested approach:** Have `loadPostgresDeps` return an already-constructed `Kysely<any>` factory or a `buildKysely(pool)` helper. Callers receive a usable query builder, not raw constructors.
+- **Effort:** Small
+
+**8. Add a `retryUntilAbort` helper to eliminate duplicated sleep-abort patterns**
+
+- **Location:** `src/core/orchestrator.ts:462–473`, `src/utils/retry.ts`
+- **Problem:** The `sleep(delay, abortController.signal)` call inside the retry loop is followed by a catch that checks `WorkflowCanceledError` and updates the step record — a pattern that will be repeated if a second retry site is ever added.
+- **Suggested approach:** Provide a `sleepWithAbort(ms, signal, onAbort?)` utility in `src/utils/retry.ts` that handles the cancellation check and throws consistently, removing the need for a catch at the call site.
+- **Effort:** Small

package/.codekin/reports/complexity/2026-04-14_complexity-report.md

@@ -0,0 +1,34 @@
+# Complexity Report: stepflow
+
+**Date**: 2026-04-14T01:04:23.188Z
+**Repository**: /srv/repos/stepflow
+**Branch**: main
+**Workflow Run**: 21addb61-3850-4a9f-9504-0c0627b6a8d5
+**Session**: 01abcef0-2ca7-4ee5-b46f-a7959609ab9a
+
+---
+
+## Code Complexity Report — Stepflow
+
+The report has been saved to `.codekin/reports/complexity/2026-04-14_complexity-report.md`.
+
+**Overall rating: Medium-High**
+
+Key findings:
+
+**Largest files** (top 3):
+- `src/storage/postgres.ts` — 1,354 lines (4.5× the 300-line guideline), a God object with 20+ methods across 8 distinct responsibilities
+- `src/core/engine.ts` — 692 lines, mixes queue management with lifecycle orchestration
+- `src/core/orchestrator.ts` — 564 lines, hosts the two most complex functions in the codebase
+
+**Hottest complexity spots:**
+- `orchestrator.ts: executeStep()` — CC ≈ 20, 7 levels of nesting in the retry loop
+- `orchestrator.ts: executeWorkflow()` — 217-line function with inline timeout scheduling, checkpointing, and lifecycle hooks all interleaved
+- `postgres.ts: createTables()` — 152 lines of undifferentiated SQL blocks
+
+**Highest-impact refactoring opportunities (quick wins):**
+1. Extract a `parseJsonColumn<T>()` utility to eliminate the 8+ identical JSON parsing idioms duplicated across postgres and sqlite adapters (Small effort)
+2. Replace `evaluateCondition()` switch with a strategy map (Small effort)
+3. Extract event/status string literals into a `src/core/constants.ts` file (Small effort)
+4. Replace the O(n) array-splice run queue with a priority heap (Small effort)
+5. Introduce a shared `Semaphore` utility — both `engine.ts` and `webhook.ts` have hand-rolled concurrency controllers (Small effort)