@pikku/mongodb 0.12.10 → 0.12.12

package/CHANGELOG.md

@@ -1,5 +1,115 @@
 # @pikku/mongodb
 
+## 0.12.12
+
+### Patch Changes
+
+- 8a2c993: Make the workflow service cheaper to run, and fix two ways it lost state.
+
+  The SQL workflow tables had no indexes at all, so every step read, every
+  history walk and every orchestrator tick was a sequential scan; five indexes
+  now cover the columns the engine actually queries by. A replay used to ask for
+  each step's row individually — O(N) reads per replay, O(N^2) over a run — and
+  now takes one read of the run's steps and serves the walk from it. A step
+  transition wrote the step row and its history row as two separate statements,
+  so a crash between them left a step saying `succeeded` whose history still said
+  `running`; both halves are now one transaction, and the history row is found by
+  attempt number rather than by sorting on `created_at`, which two attempts can
+  share. Resolving a dynamic workflow read and parsed every AI-generated workflow
+  in the deployment to `.find()` one by name; it is a point lookup now.
+
+  Waiting on a run no longer polls at a fixed interval. `pollIntervalMs` became a
+  ceiling rather than a cadence: polling starts at 10ms and widens towards it, so
+  a workflow that finishes in milliseconds is no longer held for a full second,
+  and a long-running one is not read at full rate for its whole life.
+
+  Two backend-specific defects: Redis kept a run's state as one JSON blob and
+  read-modified-wrote it, so parallel branches setting different variables
+  overwrote each other — state is a field per variable now, with the old blob
+  still read underneath so runs in flight keep what they had. Mongo's
+  `setStepScheduled` never wrote history, leaving a queued step reading as never
+  dispatched.
+
+  Also: dispatch no longer JSON round-trips every step payload before handing it
+  to a queue that serialises it anyway — the in-process dev queue, which is the
+  only one that was relying on it, does it itself now.
+
+  Two more defects. A transition whose step had no live attempt wrote its status
+  to the step row and silently nothing to history — the exact divergence the
+  transaction exists to prevent — and now repairs the step and writes the
+  missing row. And resolving a dynamic workflow was non-deterministic on all
+  three backends: a name can hold several active versions, and none of them
+  ordered the candidates, so which one ran could change between two calls
+  reading identical data. The newest version wins, with the graph hash breaking
+  a tie.
+
+  The two attempt columns and the five indexes are declared in the workflow
+  schema, so a fresh database gets them at boot. An existing one gets them from
+  a migration — `pikku db generate` writes the declaration down — rather than
+  from DDL issued at boot.
+
+- a261006: **Breaking:** removed dynamic workflows — runtime-defined workflow graphs stored in the database and resolved by name instead of by codegen.
+
+  The feature was already half-gone. Its authoring surface (`createAgentWorkflow`, `saveAgentWorkflow`, `listAgentWorkflows`, `executeAgentWorkflow`, and the AI-agent instruction builder) was deleted in April 2026 along with its entire e2e suite, and nothing has written a dynamic workflow since. What remained could not execute one either: `executeAgentWorkflow` gated on `pikkuState('workflows', 'meta')`, which only codegen ever populates, so a graph that existed solely in the database was never findable. The two backend families had also drifted onto different `source` sentinels (`'ai-agent'` vs `'dynamic-workflow'`), and the two Redis implementations disagreed on key escaping — so at least one of them matched nothing. Rather than keep shipping plumbing for a path no caller could complete, it is removed until it can be reintroduced deliberately.
+
+  Removed:
+  - `getAIGeneratedWorkflows` from `WorkflowService` and `WorkflowRunService`, and from every backend (in-memory, Redis, MongoDB, Kysely, and the Cloudflare Durable Object service and client — the last two were already a `return []` stub and a rejection).
+  - The database-lookup fallbacks in `startWorkflow` and `runWorkflowJob` that resolved a workflow name against stored graphs when static meta had no match.
+  - `'dynamic-workflow'` from the `WorkflowRuntimeMeta['source']` union.
+  - `validateWorkflowWiring` and `computeEntryNodeIds` from `@pikku/core/workflow`. These validated AI-authored graphs and had no callers in core; the inspector keeps its own private entry-node computation for static graph wiring, which is unaffected.
+  - The `workflow-created` AI stream event and its AG-UI `pikku:workflow-created` custom event. Its only emitter went with the April deletion, so it could never fire.
+  - The console's `console:getAIWorkflows` RPC, the `useAIWorkflows` hook, the "Dynamic" workflow filter and badge, and the trigger-schema scraper that derived an input form from a stored graph's `$ref` bindings.
+
+  Kept, because static graph workflows depend on them and this is not a change to versioning:
+  - `upsertWorkflowVersion`, `getWorkflowVersion`, `updateWorkflowVersionStatus`, and the `workflowVersions` storage in every backend. These back version-mismatch replay: when a deployed graph's hash changes, in-flight runs continue against the exact graph they started on. No schema migration is needed — the table, its columns, and its `(workflowName, graphHash)` upsert key are unchanged.
+  - `generateMermaidDiagram`, which renders any workflow graph and is not specific to dynamic ones.
+
+  Static `pikkuWorkflowGraph` and DSL workflows are entirely unaffected: they resolve from codegen'd meta, which was always the only path that worked.
+
+  To revive this post-MVP, the deleted authoring code is recoverable in full — its prompt engineering (a compact tool table upfront, full schemas with flattened dotted output paths returned only after a validation failure) is worth reading before rewriting:
+
+  ```
+  git show f52f3308b^:packages/core/src/wirings/ai-agent/agent-dynamic-workflow.ts
+  git show f52f3308b^:packages/core/src/wirings/workflow/graph/graph-validation.ts
+  git show f52f3308b --stat   # the April removal, incl. the three e2e feature files
+  ```
+
+  Note that reviving it needs more than restoring those files: the queued-step path (`executeWorkflowStep`), `onError` compensation, and sub-workflow resolution all read static meta only and would need a fallback for a graph that exists solely in the database.
+
+- Updated dependencies [8a2c993]
+- Updated dependencies [a261006]
+- Updated dependencies [09973b9]
+  - @pikku/core@0.12.71
+
+## 0.12.11
+
+### Patch Changes
+
+- e3d4454: Add job groups, so one shared queue can stay fair without splitting into one
+  queue per producer.
+
+  A job may now carry `group: { id, tier }`, and a worker may cap how many jobs
+  of any one group run at once via `groupConcurrency`. On pg-boss this maps to
+  `localGroupConcurrency`, which excludes at-capacity groups from the fetch query
+  itself, so a capped group costs nothing rather than being fetched and restored.
+  BullMQ declares it unsupported (groups are a BullMQ Pro feature) — being
+  push-based, it can simply use a queue per group at no polling cost.
+
+  Workflow services accept a `queueStrategy`. The default `'per-workflow'` is
+  unchanged: every workflow gets its own `wf-orchestrator-*` / `wf-step-*` queue,
+  which is also what lets serverless providers deploy one unit per workflow. The
+  new `'shared-groups'` routes every workflow through the shared
+  orchestrator/step-worker queues and isolates them by group instead, so a
+  monolith runs one set of pollers rather than one per workflow — on a
+  pull-based backend with dozens of workflows that is the difference between
+  hundreds of poll loops and twenty. It is for single-process runtimes only; a
+  per-unit serverless deploy still needs the per-workflow queues to route to its
+  units.
+
+- Updated dependencies [24252b8]
+- Updated dependencies [e3d4454]
+  - @pikku/core@0.12.69
+
 ## 0.12.10
 
 ### Patch Changes

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 - present Yasser Fadl and Pikku contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,27 @@
+# @pikku/mongodb
+
+MongoDB implementations of the Pikku service interfaces — workflow state, agent
+runs, secrets, and the channel and event-hub stores.
+
+## Install
+
+```bash
+npm install @pikku/mongodb
+```
+
+## Usage
+
+```typescript
+import { PikkuMongoDB, MongoDBWorkflowService } from '@pikku/mongodb'
+
+const { db } = new PikkuMongoDB(logger, 'mongodb://localhost:27017')
+
+const workflowService = new MongoDBWorkflowService(db)
+```
+
+`PikkuMongoDB` also accepts an existing `MongoClient`, in which case it will not
+close the connection on shutdown.
+
+## Docs
+
+https://pikku.dev/docs

package/dist/src/mongodb-workflow-run-service.d.ts

@@ -26,11 +26,6 @@ export declare class MongoDBWorkflowRunService implements WorkflowRunService {
         graph: any;
         source: string;
     } | null>;
-    getAIGeneratedWorkflows(agentName?: string): Promise<Array<{
-        workflowName: string;
-        graphHash: string;
-        graph: any;
-    }>>;
     deleteRun(id: string): Promise<boolean>;
     private mapRunRow;
 }

package/dist/src/mongodb-workflow-run-service.js

@@ -105,19 +105,6 @@ export class MongoDBWorkflowRunService {
             source: row.source,
         };
     }
-    async getAIGeneratedWorkflows(agentName) {
-        const filter = { source: 'ai-agent', status: 'active' };
-        if (agentName) {
-            const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
-            filter.workflowName = { $regex: `^ai:${escaped}:` };
-        }
-        const rows = await this.versions.find(filter).toArray();
-        return rows.map((row) => ({
-            workflowName: row.workflowName,
-            graphHash: row.graphHash,
-            graph: row.graph,
-        }));
-    }
     async deleteRun(id) {
         const steps = await this.steps.find({ workflowRunId: id }).toArray();
         const stepIds = steps.map((s) => s._id);

package/dist/src/mongodb-workflow-service.d.ts

@@ -1,5 +1,5 @@
 import type { SerializedError } from '@pikku/core';
-import { PikkuWorkflowService, type WorkflowPlannedStep, type WorkflowRun, type WorkflowRunWire, type StepState, type StepStatus, type WorkflowStatus, type WorkflowVersionStatus } from '@pikku/core/workflow';
+import { PikkuWorkflowService, type WorkflowPlannedStep, type WorkflowQueueOptions, type WorkflowRun, type WorkflowRunWire, type StepState, type StepStatus, type WorkflowStatus, type WorkflowVersionStatus } from '@pikku/core/workflow';
 import type { Db } from 'mongodb';
 export declare class MongoDBWorkflowService extends PikkuWorkflowService {
     private initialized;
@@ -8,7 +8,7 @@ export declare class MongoDBWorkflowService extends PikkuWorkflowService {
     private steps;
     private stepHistory;
     private versions;
-    constructor(db: Db);
+    constructor(db: Db, options?: WorkflowQueueOptions);
     init(): Promise<void>;
     protected createRunImpl(workflowName: string, input: any, inline: boolean, graphHash: string, wire: WorkflowRunWire, options?: {
         deterministic?: boolean;
@@ -26,6 +26,16 @@ export declare class MongoDBWorkflowService extends PikkuWorkflowService {
     }>>;
     protected setStepRunningImpl(stepId: string): Promise<void>;
     protected setStepScheduledImpl(stepId: string): Promise<void>;
+    /**
+     * Move the current attempt of a step's history to a status.
+     *
+     * Finding the newest attempt and then updating it by `_id` was two round
+     * trips with a window in between; `findOneAndUpdate` applies the same sort
+     * server-side and writes the row it selected, so nothing can move underneath
+     * it. The attempt is still identified by `createdAt` because Mongo keeps no
+     * attempt number — see `getStepState`, which counts history documents.
+     */
+    private writeLatestHistory;
     private insertHistoryRecord;
     private getTimestampFieldForStatus;
     protected setStepChildRunIdImpl(stepId: string, childRunId: string): Promise<void>;
@@ -55,10 +65,5 @@ export declare class MongoDBWorkflowService extends PikkuWorkflowService {
         graph: any;
         source: string;
     } | null>;
-    getAIGeneratedWorkflows(agentName?: string): Promise<Array<{
-        workflowName: string;
-        graphHash: string;
-        graph: any;
-    }>>;
     close(): Promise<void>;
 }

package/dist/src/mongodb-workflow-service.js

@@ -7,8 +7,8 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
     steps;
     stepHistory;
     versions;
-    constructor(db) {
-        super();
+    constructor(db, options = {}) {
+        super(options);
         this.runService = new MongoDBWorkflowRunService(db);
         this.runs = db.collection('workflow_runs');
         this.steps = db.collection('workflow_step');
@@ -124,17 +124,29 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
     }
     async setStepRunningImpl(stepId) {
         await this.steps.updateOne({ _id: stepId }, { $set: { status: 'running', updatedAt: new Date() } });
-        const latestHistory = await this.stepHistory
-            .find({ workflowStepId: stepId })
-            .sort({ createdAt: -1 })
-            .limit(1)
-            .toArray();
-        if (latestHistory.length > 0) {
-            await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'running', runningAt: new Date() } });
-        }
+        await this.writeLatestHistory(stepId, 'running');
     }
     async setStepScheduledImpl(stepId) {
         await this.steps.updateOne({ _id: stepId }, { $set: { status: 'scheduled', updatedAt: new Date() } });
+        await this.writeLatestHistory(stepId, 'scheduled');
+    }
+    /**
+     * Move the current attempt of a step's history to a status.
+     *
+     * Finding the newest attempt and then updating it by `_id` was two round
+     * trips with a window in between; `findOneAndUpdate` applies the same sort
+     * server-side and writes the row it selected, so nothing can move underneath
+     * it. The attempt is still identified by `createdAt` because Mongo keeps no
+     * attempt number — see `getStepState`, which counts history documents.
+     */
+    async writeLatestHistory(stepId, status, values = {}) {
+        const now = new Date();
+        const update = { status, ...values };
+        const timestampField = this.getTimestampFieldForStatus(status);
+        if (timestampField !== 'createdAt') {
+            update[timestampField] = now;
+        }
+        await this.stepHistory.findOneAndUpdate({ workflowStepId: stepId }, { $set: update }, { sort: { createdAt: -1 } });
     }
     async insertHistoryRecord(stepId, status, result, error) {
         const now = new Date();
@@ -187,14 +199,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
                 updatedAt: new Date(),
             },
         });
-        const latestHistory = await this.stepHistory
-            .find({ workflowStepId: stepId })
-            .sort({ createdAt: -1 })
-            .limit(1)
-            .toArray();
-        if (latestHistory.length > 0) {
-            await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'succeeded', result, succeededAt: new Date() } });
-        }
+        await this.writeLatestHistory(stepId, 'succeeded', { result });
     }
     async setStepErrorImpl(stepId, error) {
         const serializedError = {
@@ -210,14 +215,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
                 updatedAt: new Date(),
             },
         });
-        const latestHistory = await this.stepHistory
-            .find({ workflowStepId: stepId })
-            .sort({ createdAt: -1 })
-            .limit(1)
-            .toArray();
-        if (latestHistory.length > 0) {
-            await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'failed', error: serializedError, failedAt: new Date() } });
-        }
+        await this.writeLatestHistory(stepId, 'failed', {
+            error: serializedError,
+        });
     }
     async createRetryAttemptImpl(stepId, status) {
         await this.steps.updateOne({ _id: stepId }, {
@@ -362,18 +362,5 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
     async getWorkflowVersion(name, graphHash) {
         return this.runService.getWorkflowVersion(name, graphHash);
     }
-    async getAIGeneratedWorkflows(agentName) {
-        const filter = { source: 'ai-agent', status: 'active' };
-        if (agentName) {
-            const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
-            filter.workflowName = { $regex: `^ai:${escaped}:` };
-        }
-        const docs = await this.versions.find(filter).toArray();
-        return docs.map((doc) => ({
-            workflowName: doc.workflowName,
-            graphHash: doc.graphHash,
-            graph: doc.graph,
-        }));
-    }
     async close() { }
 }

package/package.json

@@ -1,6 +1,7 @@
 {
   "name": "@pikku/mongodb",
-  "version": "0.12.10",
+  "version": "0.12.12",
+  "description": "MongoDB services for Pikku — workflow state, agent runs, secrets and channel stores",
   "author": "yasser.fadl@gmail.com",
   "license": "MIT",
   "module": "dist/src/index.js",
@@ -17,7 +18,7 @@
     "prepublishOnly": "yarn build"
   },
   "peerDependencies": {
-    "@pikku/core": "^0.12.66"
+    "@pikku/core": "^0.12.71"
   },
   "dependencies": {
     "mongodb": "^6.12.0"

package/src/mongodb-workflow-run-service.ts

@@ -202,22 +202,6 @@ export class MongoDBWorkflowRunService implements WorkflowRunService {
     }
   }
 
-  async getAIGeneratedWorkflows(
-    agentName?: string
-  ): Promise<Array<{ workflowName: string; graphHash: string; graph: any }>> {
-    const filter: Record<string, any> = { source: 'ai-agent', status: 'active' }
-    if (agentName) {
-      const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
-      filter.workflowName = { $regex: `^ai:${escaped}:` }
-    }
-    const rows = await this.versions.find(filter).toArray()
-    return rows.map((row) => ({
-      workflowName: row.workflowName,
-      graphHash: row.graphHash,
-      graph: row.graph,
-    }))
-  }
-
   async deleteRun(id: string): Promise<boolean> {
     const steps = await this.steps.find({ workflowRunId: id }).toArray()
     const stepIds = steps.map((s) => s._id)

package/src/mongodb-workflow-service.test.ts

@@ -0,0 +1,121 @@
+import { describe, test, before, after, beforeEach } from 'node:test'
+import assert from 'node:assert/strict'
+import { MongoClient, type Db } from 'mongodb'
+import { MongoMemoryServer } from 'mongodb-memory-server'
+
+import { MongoDBWorkflowService } from './mongodb-workflow-service.js'
+
+let mongod: MongoMemoryServer
+let client: MongoClient
+let db: Db
+let ws: MongoDBWorkflowService
+let dbCount = 0
+
+before(async () => {
+  mongod = await MongoMemoryServer.create()
+  client = new MongoClient(mongod.getUri())
+  await client.connect()
+})
+
+after(async () => {
+  await client?.close()
+  await mongod?.stop()
+})
+
+beforeEach(async () => {
+  db = client.db(`wf-${++dbCount}`)
+  ws = new MongoDBWorkflowService(db)
+  await ws.init()
+})
+
+const seedStep = async () => {
+  const runId = await ws.createRun('flow', {}, false, 'hash', {
+    type: 'test',
+  } as any)
+  const step = await ws.insertStepState(runId, 'step-1', 'rpc.fn', { x: 1 })
+  return { runId, step }
+}
+
+/** The history rows of a run, oldest first. */
+const historyOf = async (runId: string) => ws.getRunHistory(runId)
+
+describe('a step transition reaches the history', () => {
+  test('running stamps the attempt it is running', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepRunning(step.stepId)
+
+    const [attempt] = await historyOf(runId)
+    assert.equal(attempt!.status, 'running')
+    assert.ok(attempt!.runningAt, 'no runningAt, so the step has no duration')
+  })
+
+  test('scheduled stamps the attempt it is queueing', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepScheduled(step.stepId)
+
+    const [attempt] = await historyOf(runId)
+    assert.equal(
+      attempt!.status,
+      'scheduled',
+      'the history still says pending, so a queued step reads as never dispatched'
+    )
+    assert.ok(attempt!.scheduledAt, 'no scheduledAt on the queued attempt')
+  })
+
+  test('succeeded carries the result', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepRunning(step.stepId)
+    await ws.setStepResult(step.stepId, { ok: true })
+
+    const [attempt] = await historyOf(runId)
+    assert.equal(attempt!.status, 'succeeded')
+    assert.deepEqual(attempt!.result, { ok: true })
+    assert.ok(attempt!.succeededAt)
+  })
+
+  test('failed carries the error', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepRunning(step.stepId)
+    await ws.setStepError(step.stepId, new Error('exploded'))
+
+    const [attempt] = await historyOf(runId)
+    assert.equal(attempt!.status, 'failed')
+    assert.equal(attempt!.error?.message, 'exploded')
+    assert.ok(attempt!.failedAt)
+  })
+
+  test('a retry writes to the new attempt, leaving the failed one intact', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepRunning(step.stepId)
+    await ws.setStepError(step.stepId, new Error('first go'))
+    await ws.createRetryAttempt(step.stepId, 'pending')
+    await ws.setStepRunning(step.stepId)
+    await ws.setStepResult(step.stepId, 'second go')
+
+    const history = await historyOf(runId)
+    assert.equal(history.length, 2)
+    assert.equal(history[0]!.status, 'failed')
+    assert.equal(history[0]!.error?.message, 'first go')
+    assert.equal(history[1]!.status, 'succeeded')
+    assert.equal(history[1]!.result, 'second go')
+  })
+
+  test('the step row keeps only its latest outcome', async () => {
+    const { runId, step } = await seedStep()
+
+    await ws.setStepError(step.stepId, new Error('first go'))
+    await ws.createRetryAttempt(step.stepId, 'pending')
+    await ws.setStepResult(step.stepId, 'second go')
+
+    const state = await ws.getStepState(runId, 'step-1')
+    assert.equal(state.status, 'succeeded')
+    assert.equal(state.result, 'second go')
+    assert.equal(state.error, undefined)
+    assert.equal(state.attemptCount, 2)
+  })
+})

package/src/mongodb-workflow-service.ts

@@ -2,6 +2,7 @@ import type { SerializedError } from '@pikku/core'
 import {
   PikkuWorkflowService,
   type WorkflowPlannedStep,
+  type WorkflowQueueOptions,
   type WorkflowRun,
   type WorkflowRunWire,
   type StepState,
@@ -77,8 +78,8 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
   private stepHistory: Collection<WorkflowStepHistoryDoc>
   private versions: Collection<WorkflowVersionDoc>
 
-  constructor(db: Db) {
-    super()
+  constructor(db: Db, options: WorkflowQueueOptions = {}) {
+    super(options)
     this.runService = new MongoDBWorkflowRunService(db)
     this.runs = db.collection<WorkflowRunDoc>('workflow_runs')
     this.steps = db.collection<WorkflowStepDoc>('workflow_step')
@@ -254,19 +255,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
       { _id: stepId },
       { $set: { status: 'running', updatedAt: new Date() } }
     )
-
-    const latestHistory = await this.stepHistory
-      .find({ workflowStepId: stepId })
-      .sort({ createdAt: -1 })
-      .limit(1)
-      .toArray()
-
-    if (latestHistory.length > 0) {
-      await this.stepHistory.updateOne(
-        { _id: latestHistory[0]!._id },
-        { $set: { status: 'running', runningAt: new Date() } }
-      )
-    }
+    await this.writeLatestHistory(stepId, 'running')
   }
 
   protected async setStepScheduledImpl(stepId: string): Promise<void> {
@@ -274,6 +263,35 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
       { _id: stepId },
       { $set: { status: 'scheduled', updatedAt: new Date() } }
     )
+    await this.writeLatestHistory(stepId, 'scheduled')
+  }
+
+  /**
+   * Move the current attempt of a step's history to a status.
+   *
+   * Finding the newest attempt and then updating it by `_id` was two round
+   * trips with a window in between; `findOneAndUpdate` applies the same sort
+   * server-side and writes the row it selected, so nothing can move underneath
+   * it. The attempt is still identified by `createdAt` because Mongo keeps no
+   * attempt number — see `getStepState`, which counts history documents.
+   */
+  private async writeLatestHistory(
+    stepId: string,
+    status: string,
+    values: Record<string, unknown> = {}
+  ): Promise<void> {
+    const now = new Date()
+    const update: Record<string, unknown> = { status, ...values }
+    const timestampField = this.getTimestampFieldForStatus(status)
+    if (timestampField !== 'createdAt') {
+      update[timestampField] = now
+    }
+
+    await this.stepHistory.findOneAndUpdate(
+      { workflowStepId: stepId },
+      { $set: update as any },
+      { sort: { createdAt: -1 } }
+    )
   }
 
   private async insertHistoryRecord(
@@ -350,18 +368,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
       }
     )
 
-    const latestHistory = await this.stepHistory
-      .find({ workflowStepId: stepId })
-      .sort({ createdAt: -1 })
-      .limit(1)
-      .toArray()
-
-    if (latestHistory.length > 0) {
-      await this.stepHistory.updateOne(
-        { _id: latestHistory[0]!._id },
-        { $set: { status: 'succeeded', result, succeededAt: new Date() } }
-      )
-    }
+    await this.writeLatestHistory(stepId, 'succeeded', { result })
   }
 
   protected async setStepErrorImpl(
@@ -386,18 +393,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
       }
     )
 
-    const latestHistory = await this.stepHistory
-      .find({ workflowStepId: stepId })
-      .sort({ createdAt: -1 })
-      .limit(1)
-      .toArray()
-
-    if (latestHistory.length > 0) {
-      await this.stepHistory.updateOne(
-        { _id: latestHistory[0]!._id },
-        { $set: { status: 'failed', error: serializedError, failedAt: new Date() } }
-      )
-    }
+    await this.writeLatestHistory(stepId, 'failed', {
+      error: serializedError,
+    })
   }
 
   protected async createRetryAttemptImpl(
@@ -507,7 +505,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
     return nodeIds.filter((id) => !existingStepNames.has(id))
   }
 
-  async getStepInstances(runId: string): Promise<
+  async getStepInstances(
+    runId: string
+  ): Promise<
     Array<{ stepName: string; status: StepStatus; fromStepName?: string }>
   > {
     const rows = await this.steps
@@ -621,21 +621,5 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
     return this.runService.getWorkflowVersion(name, graphHash)
   }
 
-  async getAIGeneratedWorkflows(
-    agentName?: string
-  ): Promise<Array<{ workflowName: string; graphHash: string; graph: any }>> {
-    const filter: Record<string, any> = { source: 'ai-agent', status: 'active' }
-    if (agentName) {
-      const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
-      filter.workflowName = { $regex: `^ai:${escaped}:` }
-    }
-    const docs = await this.versions.find(filter).toArray()
-    return docs.map((doc) => ({
-      workflowName: doc.workflowName,
-      graphHash: doc.graphHash,
-      graph: doc.graph,
-    }))
-  }
-
   async close(): Promise<void> {}
 }