@pikku/mongodb 0.12.11 → 0.12.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +81 -0
- package/LICENSE +21 -0
- package/README.md +27 -0
- package/dist/src/mongodb-workflow-run-service.d.ts +0 -5
- package/dist/src/mongodb-workflow-run-service.js +0 -13
- package/dist/src/mongodb-workflow-service.d.ts +10 -5
- package/dist/src/mongodb-workflow-service.js +24 -37
- package/package.json +3 -2
- package/src/mongodb-workflow-run-service.ts +0 -16
- package/src/mongodb-workflow-service.test.ts +121 -0
- package/src/mongodb-workflow-service.ts +37 -54
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,86 @@
|
|
|
1
1
|
# @pikku/mongodb
|
|
2
2
|
|
|
3
|
+
## 0.12.12
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 8a2c993: Make the workflow service cheaper to run, and fix two ways it lost state.
|
|
8
|
+
|
|
9
|
+
The SQL workflow tables had no indexes at all, so every step read, every
|
|
10
|
+
history walk and every orchestrator tick was a sequential scan; five indexes
|
|
11
|
+
now cover the columns the engine actually queries by. A replay used to ask for
|
|
12
|
+
each step's row individually — O(N) reads per replay, O(N^2) over a run — and
|
|
13
|
+
now takes one read of the run's steps and serves the walk from it. A step
|
|
14
|
+
transition wrote the step row and its history row as two separate statements,
|
|
15
|
+
so a crash between them left a step saying `succeeded` whose history still said
|
|
16
|
+
`running`; both halves are now one transaction, and the history row is found by
|
|
17
|
+
attempt number rather than by sorting on `created_at`, which two attempts can
|
|
18
|
+
share. Resolving a dynamic workflow read and parsed every AI-generated workflow
|
|
19
|
+
in the deployment to `.find()` one by name; it is a point lookup now.
|
|
20
|
+
|
|
21
|
+
Waiting on a run no longer polls at a fixed interval. `pollIntervalMs` became a
|
|
22
|
+
ceiling rather than a cadence: polling starts at 10ms and widens towards it, so
|
|
23
|
+
a workflow that finishes in milliseconds is no longer held for a full second,
|
|
24
|
+
and a long-running one is not read at full rate for its whole life.
|
|
25
|
+
|
|
26
|
+
Two backend-specific defects: Redis kept a run's state as one JSON blob and
|
|
27
|
+
read-modified-wrote it, so parallel branches setting different variables
|
|
28
|
+
overwrote each other — state is a field per variable now, with the old blob
|
|
29
|
+
still read underneath so runs in flight keep what they had. Mongo's
|
|
30
|
+
`setStepScheduled` never wrote history, leaving a queued step reading as never
|
|
31
|
+
dispatched.
|
|
32
|
+
|
|
33
|
+
Also: dispatch no longer JSON round-trips every step payload before handing it
|
|
34
|
+
to a queue that serialises it anyway — the in-process dev queue, which is the
|
|
35
|
+
only one that was relying on it, does it itself now.
|
|
36
|
+
|
|
37
|
+
Two more defects. A transition whose step had no live attempt wrote its status
|
|
38
|
+
to the step row and silently nothing to history — the exact divergence the
|
|
39
|
+
transaction exists to prevent — and now repairs the step and writes the
|
|
40
|
+
missing row. And resolving a dynamic workflow was non-deterministic on all
|
|
41
|
+
three backends: a name can hold several active versions, and none of them
|
|
42
|
+
ordered the candidates, so which one ran could change between two calls
|
|
43
|
+
reading identical data. The newest version wins, with the graph hash breaking
|
|
44
|
+
a tie.
|
|
45
|
+
|
|
46
|
+
The two attempt columns and the five indexes are declared in the workflow
|
|
47
|
+
schema, so a fresh database gets them at boot. An existing one gets them from
|
|
48
|
+
a migration — `pikku db generate` writes the declaration down — rather than
|
|
49
|
+
from DDL issued at boot.
|
|
50
|
+
|
|
51
|
+
- a261006: **Breaking:** removed dynamic workflows — runtime-defined workflow graphs stored in the database and resolved by name instead of by codegen.
|
|
52
|
+
|
|
53
|
+
The feature was already half-gone. Its authoring surface (`createAgentWorkflow`, `saveAgentWorkflow`, `listAgentWorkflows`, `executeAgentWorkflow`, and the AI-agent instruction builder) was deleted in April 2026 along with its entire e2e suite, and nothing has written a dynamic workflow since. What remained could not execute one either: `executeAgentWorkflow` gated on `pikkuState('workflows', 'meta')`, which only codegen ever populates, so a graph that existed solely in the database was never findable. The two backend families had also drifted onto different `source` sentinels (`'ai-agent'` vs `'dynamic-workflow'`), and the two Redis implementations disagreed on key escaping — so at least one of them matched nothing. Rather than keep shipping plumbing for a path no caller could complete, it is removed until it can be reintroduced deliberately.
|
|
54
|
+
|
|
55
|
+
Removed:
|
|
56
|
+
- `getAIGeneratedWorkflows` from `WorkflowService` and `WorkflowRunService`, and from every backend (in-memory, Redis, MongoDB, Kysely, and the Cloudflare Durable Object service and client — the last two were already a `return []` stub and a rejection).
|
|
57
|
+
- The database-lookup fallbacks in `startWorkflow` and `runWorkflowJob` that resolved a workflow name against stored graphs when static meta had no match.
|
|
58
|
+
- `'dynamic-workflow'` from the `WorkflowRuntimeMeta['source']` union.
|
|
59
|
+
- `validateWorkflowWiring` and `computeEntryNodeIds` from `@pikku/core/workflow`. These validated AI-authored graphs and had no callers in core; the inspector keeps its own private entry-node computation for static graph wiring, which is unaffected.
|
|
60
|
+
- The `workflow-created` AI stream event and its AG-UI `pikku:workflow-created` custom event. Its only emitter went with the April deletion, so it could never fire.
|
|
61
|
+
- The console's `console:getAIWorkflows` RPC, the `useAIWorkflows` hook, the "Dynamic" workflow filter and badge, and the trigger-schema scraper that derived an input form from a stored graph's `$ref` bindings.
|
|
62
|
+
|
|
63
|
+
Kept, because static graph workflows depend on them and this is not a change to versioning:
|
|
64
|
+
- `upsertWorkflowVersion`, `getWorkflowVersion`, `updateWorkflowVersionStatus`, and the `workflowVersions` storage in every backend. These back version-mismatch replay: when a deployed graph's hash changes, in-flight runs continue against the exact graph they started on. No schema migration is needed — the table, its columns, and its `(workflowName, graphHash)` upsert key are unchanged.
|
|
65
|
+
- `generateMermaidDiagram`, which renders any workflow graph and is not specific to dynamic ones.
|
|
66
|
+
|
|
67
|
+
Static `pikkuWorkflowGraph` and DSL workflows are entirely unaffected: they resolve from codegen'd meta, which was always the only path that worked.
|
|
68
|
+
|
|
69
|
+
To revive this post-MVP, the deleted authoring code is recoverable in full — its prompt engineering (a compact tool table upfront, full schemas with flattened dotted output paths returned only after a validation failure) is worth reading before rewriting:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
git show f52f3308b^:packages/core/src/wirings/ai-agent/agent-dynamic-workflow.ts
|
|
73
|
+
git show f52f3308b^:packages/core/src/wirings/workflow/graph/graph-validation.ts
|
|
74
|
+
git show f52f3308b --stat # the April removal, incl. the three e2e feature files
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Note that reviving it needs more than restoring those files: the queued-step path (`executeWorkflowStep`), `onError` compensation, and sub-workflow resolution all read static meta only and would need a fallback for a graph that exists solely in the database.
|
|
78
|
+
|
|
79
|
+
- Updated dependencies [8a2c993]
|
|
80
|
+
- Updated dependencies [a261006]
|
|
81
|
+
- Updated dependencies [09973b9]
|
|
82
|
+
- @pikku/core@0.12.71
|
|
83
|
+
|
|
3
84
|
## 0.12.11
|
|
4
85
|
|
|
5
86
|
### Patch Changes
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2021 - present Yasser Fadl and Pikku contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# @pikku/mongodb
|
|
2
|
+
|
|
3
|
+
MongoDB implementations of the Pikku service interfaces — workflow state, agent
|
|
4
|
+
runs, secrets, and the channel and event-hub stores.
|
|
5
|
+
|
|
6
|
+
## Install
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
npm install @pikku/mongodb
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Usage
|
|
13
|
+
|
|
14
|
+
```typescript
|
|
15
|
+
import { PikkuMongoDB, MongoDBWorkflowService } from '@pikku/mongodb'
|
|
16
|
+
|
|
17
|
+
const { db } = new PikkuMongoDB(logger, 'mongodb://localhost:27017')
|
|
18
|
+
|
|
19
|
+
const workflowService = new MongoDBWorkflowService(db)
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`PikkuMongoDB` also accepts an existing `MongoClient`, in which case it will not
|
|
23
|
+
close the connection on shutdown.
|
|
24
|
+
|
|
25
|
+
## Docs
|
|
26
|
+
|
|
27
|
+
https://pikku.dev/docs
|
|
@@ -26,11 +26,6 @@ export declare class MongoDBWorkflowRunService implements WorkflowRunService {
|
|
|
26
26
|
graph: any;
|
|
27
27
|
source: string;
|
|
28
28
|
} | null>;
|
|
29
|
-
getAIGeneratedWorkflows(agentName?: string): Promise<Array<{
|
|
30
|
-
workflowName: string;
|
|
31
|
-
graphHash: string;
|
|
32
|
-
graph: any;
|
|
33
|
-
}>>;
|
|
34
29
|
deleteRun(id: string): Promise<boolean>;
|
|
35
30
|
private mapRunRow;
|
|
36
31
|
}
|
|
@@ -105,19 +105,6 @@ export class MongoDBWorkflowRunService {
|
|
|
105
105
|
source: row.source,
|
|
106
106
|
};
|
|
107
107
|
}
|
|
108
|
-
async getAIGeneratedWorkflows(agentName) {
|
|
109
|
-
const filter = { source: 'ai-agent', status: 'active' };
|
|
110
|
-
if (agentName) {
|
|
111
|
-
const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
112
|
-
filter.workflowName = { $regex: `^ai:${escaped}:` };
|
|
113
|
-
}
|
|
114
|
-
const rows = await this.versions.find(filter).toArray();
|
|
115
|
-
return rows.map((row) => ({
|
|
116
|
-
workflowName: row.workflowName,
|
|
117
|
-
graphHash: row.graphHash,
|
|
118
|
-
graph: row.graph,
|
|
119
|
-
}));
|
|
120
|
-
}
|
|
121
108
|
async deleteRun(id) {
|
|
122
109
|
const steps = await this.steps.find({ workflowRunId: id }).toArray();
|
|
123
110
|
const stepIds = steps.map((s) => s._id);
|
|
@@ -26,6 +26,16 @@ export declare class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
26
26
|
}>>;
|
|
27
27
|
protected setStepRunningImpl(stepId: string): Promise<void>;
|
|
28
28
|
protected setStepScheduledImpl(stepId: string): Promise<void>;
|
|
29
|
+
/**
|
|
30
|
+
* Move the current attempt of a step's history to a status.
|
|
31
|
+
*
|
|
32
|
+
* Finding the newest attempt and then updating it by `_id` was two round
|
|
33
|
+
* trips with a window in between; `findOneAndUpdate` applies the same sort
|
|
34
|
+
* server-side and writes the row it selected, so nothing can move underneath
|
|
35
|
+
* it. The attempt is still identified by `createdAt` because Mongo keeps no
|
|
36
|
+
* attempt number — see `getStepState`, which counts history documents.
|
|
37
|
+
*/
|
|
38
|
+
private writeLatestHistory;
|
|
29
39
|
private insertHistoryRecord;
|
|
30
40
|
private getTimestampFieldForStatus;
|
|
31
41
|
protected setStepChildRunIdImpl(stepId: string, childRunId: string): Promise<void>;
|
|
@@ -55,10 +65,5 @@ export declare class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
55
65
|
graph: any;
|
|
56
66
|
source: string;
|
|
57
67
|
} | null>;
|
|
58
|
-
getAIGeneratedWorkflows(agentName?: string): Promise<Array<{
|
|
59
|
-
workflowName: string;
|
|
60
|
-
graphHash: string;
|
|
61
|
-
graph: any;
|
|
62
|
-
}>>;
|
|
63
68
|
close(): Promise<void>;
|
|
64
69
|
}
|
|
@@ -124,17 +124,29 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
124
124
|
}
|
|
125
125
|
async setStepRunningImpl(stepId) {
|
|
126
126
|
await this.steps.updateOne({ _id: stepId }, { $set: { status: 'running', updatedAt: new Date() } });
|
|
127
|
-
|
|
128
|
-
.find({ workflowStepId: stepId })
|
|
129
|
-
.sort({ createdAt: -1 })
|
|
130
|
-
.limit(1)
|
|
131
|
-
.toArray();
|
|
132
|
-
if (latestHistory.length > 0) {
|
|
133
|
-
await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'running', runningAt: new Date() } });
|
|
134
|
-
}
|
|
127
|
+
await this.writeLatestHistory(stepId, 'running');
|
|
135
128
|
}
|
|
136
129
|
async setStepScheduledImpl(stepId) {
|
|
137
130
|
await this.steps.updateOne({ _id: stepId }, { $set: { status: 'scheduled', updatedAt: new Date() } });
|
|
131
|
+
await this.writeLatestHistory(stepId, 'scheduled');
|
|
132
|
+
}
|
|
133
|
+
/**
|
|
134
|
+
* Move the current attempt of a step's history to a status.
|
|
135
|
+
*
|
|
136
|
+
* Finding the newest attempt and then updating it by `_id` was two round
|
|
137
|
+
* trips with a window in between; `findOneAndUpdate` applies the same sort
|
|
138
|
+
* server-side and writes the row it selected, so nothing can move underneath
|
|
139
|
+
* it. The attempt is still identified by `createdAt` because Mongo keeps no
|
|
140
|
+
* attempt number — see `getStepState`, which counts history documents.
|
|
141
|
+
*/
|
|
142
|
+
async writeLatestHistory(stepId, status, values = {}) {
|
|
143
|
+
const now = new Date();
|
|
144
|
+
const update = { status, ...values };
|
|
145
|
+
const timestampField = this.getTimestampFieldForStatus(status);
|
|
146
|
+
if (timestampField !== 'createdAt') {
|
|
147
|
+
update[timestampField] = now;
|
|
148
|
+
}
|
|
149
|
+
await this.stepHistory.findOneAndUpdate({ workflowStepId: stepId }, { $set: update }, { sort: { createdAt: -1 } });
|
|
138
150
|
}
|
|
139
151
|
async insertHistoryRecord(stepId, status, result, error) {
|
|
140
152
|
const now = new Date();
|
|
@@ -187,14 +199,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
187
199
|
updatedAt: new Date(),
|
|
188
200
|
},
|
|
189
201
|
});
|
|
190
|
-
|
|
191
|
-
.find({ workflowStepId: stepId })
|
|
192
|
-
.sort({ createdAt: -1 })
|
|
193
|
-
.limit(1)
|
|
194
|
-
.toArray();
|
|
195
|
-
if (latestHistory.length > 0) {
|
|
196
|
-
await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'succeeded', result, succeededAt: new Date() } });
|
|
197
|
-
}
|
|
202
|
+
await this.writeLatestHistory(stepId, 'succeeded', { result });
|
|
198
203
|
}
|
|
199
204
|
async setStepErrorImpl(stepId, error) {
|
|
200
205
|
const serializedError = {
|
|
@@ -210,14 +215,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
210
215
|
updatedAt: new Date(),
|
|
211
216
|
},
|
|
212
217
|
});
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
.limit(1)
|
|
217
|
-
.toArray();
|
|
218
|
-
if (latestHistory.length > 0) {
|
|
219
|
-
await this.stepHistory.updateOne({ _id: latestHistory[0]._id }, { $set: { status: 'failed', error: serializedError, failedAt: new Date() } });
|
|
220
|
-
}
|
|
218
|
+
await this.writeLatestHistory(stepId, 'failed', {
|
|
219
|
+
error: serializedError,
|
|
220
|
+
});
|
|
221
221
|
}
|
|
222
222
|
async createRetryAttemptImpl(stepId, status) {
|
|
223
223
|
await this.steps.updateOne({ _id: stepId }, {
|
|
@@ -362,18 +362,5 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
362
362
|
async getWorkflowVersion(name, graphHash) {
|
|
363
363
|
return this.runService.getWorkflowVersion(name, graphHash);
|
|
364
364
|
}
|
|
365
|
-
async getAIGeneratedWorkflows(agentName) {
|
|
366
|
-
const filter = { source: 'ai-agent', status: 'active' };
|
|
367
|
-
if (agentName) {
|
|
368
|
-
const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
369
|
-
filter.workflowName = { $regex: `^ai:${escaped}:` };
|
|
370
|
-
}
|
|
371
|
-
const docs = await this.versions.find(filter).toArray();
|
|
372
|
-
return docs.map((doc) => ({
|
|
373
|
-
workflowName: doc.workflowName,
|
|
374
|
-
graphHash: doc.graphHash,
|
|
375
|
-
graph: doc.graph,
|
|
376
|
-
}));
|
|
377
|
-
}
|
|
378
365
|
async close() { }
|
|
379
366
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@pikku/mongodb",
|
|
3
|
-
"version": "0.12.
|
|
3
|
+
"version": "0.12.12",
|
|
4
|
+
"description": "MongoDB services for Pikku — workflow state, agent runs, secrets and channel stores",
|
|
4
5
|
"author": "yasser.fadl@gmail.com",
|
|
5
6
|
"license": "MIT",
|
|
6
7
|
"module": "dist/src/index.js",
|
|
@@ -17,7 +18,7 @@
|
|
|
17
18
|
"prepublishOnly": "yarn build"
|
|
18
19
|
},
|
|
19
20
|
"peerDependencies": {
|
|
20
|
-
"@pikku/core": "^0.12.
|
|
21
|
+
"@pikku/core": "^0.12.71"
|
|
21
22
|
},
|
|
22
23
|
"dependencies": {
|
|
23
24
|
"mongodb": "^6.12.0"
|
|
@@ -202,22 +202,6 @@ export class MongoDBWorkflowRunService implements WorkflowRunService {
|
|
|
202
202
|
}
|
|
203
203
|
}
|
|
204
204
|
|
|
205
|
-
async getAIGeneratedWorkflows(
|
|
206
|
-
agentName?: string
|
|
207
|
-
): Promise<Array<{ workflowName: string; graphHash: string; graph: any }>> {
|
|
208
|
-
const filter: Record<string, any> = { source: 'ai-agent', status: 'active' }
|
|
209
|
-
if (agentName) {
|
|
210
|
-
const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
|
211
|
-
filter.workflowName = { $regex: `^ai:${escaped}:` }
|
|
212
|
-
}
|
|
213
|
-
const rows = await this.versions.find(filter).toArray()
|
|
214
|
-
return rows.map((row) => ({
|
|
215
|
-
workflowName: row.workflowName,
|
|
216
|
-
graphHash: row.graphHash,
|
|
217
|
-
graph: row.graph,
|
|
218
|
-
}))
|
|
219
|
-
}
|
|
220
|
-
|
|
221
205
|
async deleteRun(id: string): Promise<boolean> {
|
|
222
206
|
const steps = await this.steps.find({ workflowRunId: id }).toArray()
|
|
223
207
|
const stepIds = steps.map((s) => s._id)
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
import { describe, test, before, after, beforeEach } from 'node:test'
|
|
2
|
+
import assert from 'node:assert/strict'
|
|
3
|
+
import { MongoClient, type Db } from 'mongodb'
|
|
4
|
+
import { MongoMemoryServer } from 'mongodb-memory-server'
|
|
5
|
+
|
|
6
|
+
import { MongoDBWorkflowService } from './mongodb-workflow-service.js'
|
|
7
|
+
|
|
8
|
+
let mongod: MongoMemoryServer
|
|
9
|
+
let client: MongoClient
|
|
10
|
+
let db: Db
|
|
11
|
+
let ws: MongoDBWorkflowService
|
|
12
|
+
let dbCount = 0
|
|
13
|
+
|
|
14
|
+
before(async () => {
|
|
15
|
+
mongod = await MongoMemoryServer.create()
|
|
16
|
+
client = new MongoClient(mongod.getUri())
|
|
17
|
+
await client.connect()
|
|
18
|
+
})
|
|
19
|
+
|
|
20
|
+
after(async () => {
|
|
21
|
+
await client?.close()
|
|
22
|
+
await mongod?.stop()
|
|
23
|
+
})
|
|
24
|
+
|
|
25
|
+
beforeEach(async () => {
|
|
26
|
+
db = client.db(`wf-${++dbCount}`)
|
|
27
|
+
ws = new MongoDBWorkflowService(db)
|
|
28
|
+
await ws.init()
|
|
29
|
+
})
|
|
30
|
+
|
|
31
|
+
const seedStep = async () => {
|
|
32
|
+
const runId = await ws.createRun('flow', {}, false, 'hash', {
|
|
33
|
+
type: 'test',
|
|
34
|
+
} as any)
|
|
35
|
+
const step = await ws.insertStepState(runId, 'step-1', 'rpc.fn', { x: 1 })
|
|
36
|
+
return { runId, step }
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/** The history rows of a run, oldest first. */
|
|
40
|
+
const historyOf = async (runId: string) => ws.getRunHistory(runId)
|
|
41
|
+
|
|
42
|
+
describe('a step transition reaches the history', () => {
|
|
43
|
+
test('running stamps the attempt it is running', async () => {
|
|
44
|
+
const { runId, step } = await seedStep()
|
|
45
|
+
|
|
46
|
+
await ws.setStepRunning(step.stepId)
|
|
47
|
+
|
|
48
|
+
const [attempt] = await historyOf(runId)
|
|
49
|
+
assert.equal(attempt!.status, 'running')
|
|
50
|
+
assert.ok(attempt!.runningAt, 'no runningAt, so the step has no duration')
|
|
51
|
+
})
|
|
52
|
+
|
|
53
|
+
test('scheduled stamps the attempt it is queueing', async () => {
|
|
54
|
+
const { runId, step } = await seedStep()
|
|
55
|
+
|
|
56
|
+
await ws.setStepScheduled(step.stepId)
|
|
57
|
+
|
|
58
|
+
const [attempt] = await historyOf(runId)
|
|
59
|
+
assert.equal(
|
|
60
|
+
attempt!.status,
|
|
61
|
+
'scheduled',
|
|
62
|
+
'the history still says pending, so a queued step reads as never dispatched'
|
|
63
|
+
)
|
|
64
|
+
assert.ok(attempt!.scheduledAt, 'no scheduledAt on the queued attempt')
|
|
65
|
+
})
|
|
66
|
+
|
|
67
|
+
test('succeeded carries the result', async () => {
|
|
68
|
+
const { runId, step } = await seedStep()
|
|
69
|
+
|
|
70
|
+
await ws.setStepRunning(step.stepId)
|
|
71
|
+
await ws.setStepResult(step.stepId, { ok: true })
|
|
72
|
+
|
|
73
|
+
const [attempt] = await historyOf(runId)
|
|
74
|
+
assert.equal(attempt!.status, 'succeeded')
|
|
75
|
+
assert.deepEqual(attempt!.result, { ok: true })
|
|
76
|
+
assert.ok(attempt!.succeededAt)
|
|
77
|
+
})
|
|
78
|
+
|
|
79
|
+
test('failed carries the error', async () => {
|
|
80
|
+
const { runId, step } = await seedStep()
|
|
81
|
+
|
|
82
|
+
await ws.setStepRunning(step.stepId)
|
|
83
|
+
await ws.setStepError(step.stepId, new Error('exploded'))
|
|
84
|
+
|
|
85
|
+
const [attempt] = await historyOf(runId)
|
|
86
|
+
assert.equal(attempt!.status, 'failed')
|
|
87
|
+
assert.equal(attempt!.error?.message, 'exploded')
|
|
88
|
+
assert.ok(attempt!.failedAt)
|
|
89
|
+
})
|
|
90
|
+
|
|
91
|
+
test('a retry writes to the new attempt, leaving the failed one intact', async () => {
|
|
92
|
+
const { runId, step } = await seedStep()
|
|
93
|
+
|
|
94
|
+
await ws.setStepRunning(step.stepId)
|
|
95
|
+
await ws.setStepError(step.stepId, new Error('first go'))
|
|
96
|
+
await ws.createRetryAttempt(step.stepId, 'pending')
|
|
97
|
+
await ws.setStepRunning(step.stepId)
|
|
98
|
+
await ws.setStepResult(step.stepId, 'second go')
|
|
99
|
+
|
|
100
|
+
const history = await historyOf(runId)
|
|
101
|
+
assert.equal(history.length, 2)
|
|
102
|
+
assert.equal(history[0]!.status, 'failed')
|
|
103
|
+
assert.equal(history[0]!.error?.message, 'first go')
|
|
104
|
+
assert.equal(history[1]!.status, 'succeeded')
|
|
105
|
+
assert.equal(history[1]!.result, 'second go')
|
|
106
|
+
})
|
|
107
|
+
|
|
108
|
+
test('the step row keeps only its latest outcome', async () => {
|
|
109
|
+
const { runId, step } = await seedStep()
|
|
110
|
+
|
|
111
|
+
await ws.setStepError(step.stepId, new Error('first go'))
|
|
112
|
+
await ws.createRetryAttempt(step.stepId, 'pending')
|
|
113
|
+
await ws.setStepResult(step.stepId, 'second go')
|
|
114
|
+
|
|
115
|
+
const state = await ws.getStepState(runId, 'step-1')
|
|
116
|
+
assert.equal(state.status, 'succeeded')
|
|
117
|
+
assert.equal(state.result, 'second go')
|
|
118
|
+
assert.equal(state.error, undefined)
|
|
119
|
+
assert.equal(state.attemptCount, 2)
|
|
120
|
+
})
|
|
121
|
+
})
|
|
@@ -255,19 +255,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
255
255
|
{ _id: stepId },
|
|
256
256
|
{ $set: { status: 'running', updatedAt: new Date() } }
|
|
257
257
|
)
|
|
258
|
-
|
|
259
|
-
const latestHistory = await this.stepHistory
|
|
260
|
-
.find({ workflowStepId: stepId })
|
|
261
|
-
.sort({ createdAt: -1 })
|
|
262
|
-
.limit(1)
|
|
263
|
-
.toArray()
|
|
264
|
-
|
|
265
|
-
if (latestHistory.length > 0) {
|
|
266
|
-
await this.stepHistory.updateOne(
|
|
267
|
-
{ _id: latestHistory[0]!._id },
|
|
268
|
-
{ $set: { status: 'running', runningAt: new Date() } }
|
|
269
|
-
)
|
|
270
|
-
}
|
|
258
|
+
await this.writeLatestHistory(stepId, 'running')
|
|
271
259
|
}
|
|
272
260
|
|
|
273
261
|
protected async setStepScheduledImpl(stepId: string): Promise<void> {
|
|
@@ -275,6 +263,35 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
275
263
|
{ _id: stepId },
|
|
276
264
|
{ $set: { status: 'scheduled', updatedAt: new Date() } }
|
|
277
265
|
)
|
|
266
|
+
await this.writeLatestHistory(stepId, 'scheduled')
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
/**
|
|
270
|
+
* Move the current attempt of a step's history to a status.
|
|
271
|
+
*
|
|
272
|
+
* Finding the newest attempt and then updating it by `_id` was two round
|
|
273
|
+
* trips with a window in between; `findOneAndUpdate` applies the same sort
|
|
274
|
+
* server-side and writes the row it selected, so nothing can move underneath
|
|
275
|
+
* it. The attempt is still identified by `createdAt` because Mongo keeps no
|
|
276
|
+
* attempt number — see `getStepState`, which counts history documents.
|
|
277
|
+
*/
|
|
278
|
+
private async writeLatestHistory(
|
|
279
|
+
stepId: string,
|
|
280
|
+
status: string,
|
|
281
|
+
values: Record<string, unknown> = {}
|
|
282
|
+
): Promise<void> {
|
|
283
|
+
const now = new Date()
|
|
284
|
+
const update: Record<string, unknown> = { status, ...values }
|
|
285
|
+
const timestampField = this.getTimestampFieldForStatus(status)
|
|
286
|
+
if (timestampField !== 'createdAt') {
|
|
287
|
+
update[timestampField] = now
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
await this.stepHistory.findOneAndUpdate(
|
|
291
|
+
{ workflowStepId: stepId },
|
|
292
|
+
{ $set: update as any },
|
|
293
|
+
{ sort: { createdAt: -1 } }
|
|
294
|
+
)
|
|
278
295
|
}
|
|
279
296
|
|
|
280
297
|
private async insertHistoryRecord(
|
|
@@ -351,18 +368,7 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
351
368
|
}
|
|
352
369
|
)
|
|
353
370
|
|
|
354
|
-
|
|
355
|
-
.find({ workflowStepId: stepId })
|
|
356
|
-
.sort({ createdAt: -1 })
|
|
357
|
-
.limit(1)
|
|
358
|
-
.toArray()
|
|
359
|
-
|
|
360
|
-
if (latestHistory.length > 0) {
|
|
361
|
-
await this.stepHistory.updateOne(
|
|
362
|
-
{ _id: latestHistory[0]!._id },
|
|
363
|
-
{ $set: { status: 'succeeded', result, succeededAt: new Date() } }
|
|
364
|
-
)
|
|
365
|
-
}
|
|
371
|
+
await this.writeLatestHistory(stepId, 'succeeded', { result })
|
|
366
372
|
}
|
|
367
373
|
|
|
368
374
|
protected async setStepErrorImpl(
|
|
@@ -387,18 +393,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
387
393
|
}
|
|
388
394
|
)
|
|
389
395
|
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
.limit(1)
|
|
394
|
-
.toArray()
|
|
395
|
-
|
|
396
|
-
if (latestHistory.length > 0) {
|
|
397
|
-
await this.stepHistory.updateOne(
|
|
398
|
-
{ _id: latestHistory[0]!._id },
|
|
399
|
-
{ $set: { status: 'failed', error: serializedError, failedAt: new Date() } }
|
|
400
|
-
)
|
|
401
|
-
}
|
|
396
|
+
await this.writeLatestHistory(stepId, 'failed', {
|
|
397
|
+
error: serializedError,
|
|
398
|
+
})
|
|
402
399
|
}
|
|
403
400
|
|
|
404
401
|
protected async createRetryAttemptImpl(
|
|
@@ -508,7 +505,9 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
508
505
|
return nodeIds.filter((id) => !existingStepNames.has(id))
|
|
509
506
|
}
|
|
510
507
|
|
|
511
|
-
async getStepInstances(
|
|
508
|
+
async getStepInstances(
|
|
509
|
+
runId: string
|
|
510
|
+
): Promise<
|
|
512
511
|
Array<{ stepName: string; status: StepStatus; fromStepName?: string }>
|
|
513
512
|
> {
|
|
514
513
|
const rows = await this.steps
|
|
@@ -622,21 +621,5 @@ export class MongoDBWorkflowService extends PikkuWorkflowService {
|
|
|
622
621
|
return this.runService.getWorkflowVersion(name, graphHash)
|
|
623
622
|
}
|
|
624
623
|
|
|
625
|
-
async getAIGeneratedWorkflows(
|
|
626
|
-
agentName?: string
|
|
627
|
-
): Promise<Array<{ workflowName: string; graphHash: string; graph: any }>> {
|
|
628
|
-
const filter: Record<string, any> = { source: 'ai-agent', status: 'active' }
|
|
629
|
-
if (agentName) {
|
|
630
|
-
const escaped = agentName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
|
631
|
-
filter.workflowName = { $regex: `^ai:${escaped}:` }
|
|
632
|
-
}
|
|
633
|
-
const docs = await this.versions.find(filter).toArray()
|
|
634
|
-
return docs.map((doc) => ({
|
|
635
|
-
workflowName: doc.workflowName,
|
|
636
|
-
graphHash: doc.graphHash,
|
|
637
|
-
graph: doc.graph,
|
|
638
|
-
}))
|
|
639
|
-
}
|
|
640
|
-
|
|
641
624
|
async close(): Promise<void> {}
|
|
642
625
|
}
|