@backendkit-labs/agent-flow-queue 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -8
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,15 +1,53 @@
|
|
|
1
1
|
# @backendkit-labs/agent-flow-queue
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Run [`agent-core` flows](../core/README.md#flows--declarative-and-durable) across processes: a BullMQ `StepExecutor` (each step runs on a worker, maybe on another machine) and a Redis `RunJournal` (a run survives the process and any process can resume it).
|
|
4
4
|
|
|
5
|
-
```
|
|
5
|
+
```bash
|
|
6
|
+
npm install @backendkit-labs/agent-flow-queue @backendkit-labs/agent-core
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Needs a Redis. The flow engine stays in one process; only the steps travel.
|
|
10
|
+
|
|
11
|
+
## Run steps on workers
|
|
12
|
+
|
|
13
|
+
```typescript
|
|
6
14
|
// orchestrator process
|
|
7
|
-
|
|
8
|
-
|
|
15
|
+
import { loadFlow, runFlow } from '@backendkit-labs/agent-core';
|
|
16
|
+
import { QueueStepExecutor } from '@backendkit-labs/agent-flow-queue';
|
|
17
|
+
|
|
18
|
+
const executor = new QueueStepExecutor({ redis: process.env.REDIS_URL!, timeoutMs: 15 * 60_000 });
|
|
19
|
+
const result = await runFlow(await loadFlow('flows/close-month.yaml'), { executor, payload: { month: '2026-10' } });
|
|
20
|
+
await executor.close();
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
```typescript
|
|
24
|
+
// worker process: one queue `flow.<agent>` each, `concurrency` steps at a time
|
|
25
|
+
import { createStepWorker } from '@backendkit-labs/agent-flow-queue';
|
|
9
26
|
|
|
10
|
-
|
|
11
|
-
|
|
27
|
+
const workers = createStepWorker({
|
|
28
|
+
redis: process.env.REDIS_URL!,
|
|
29
|
+
agents: ['sales', 'finance'],
|
|
30
|
+
concurrency: 4,
|
|
31
|
+
executor: { run: async (step, input) => ({ content: await localAgent(step.agent).run(input.prompt) }) },
|
|
32
|
+
});
|
|
33
|
+
// on shutdown: await workers.close();
|
|
12
34
|
```
|
|
13
35
|
|
|
14
|
-
At
|
|
15
|
-
|
|
36
|
+
- **At least once.** The job id is the step's idempotency key (`runId:stepId:attempt`), so enqueuing the same step twice is one job. A step with an external effect (payment, email) must use `input.idempotencyKey` in the worker too.
|
|
37
|
+
- **Retries belong to the engine.** `attempts` is 1 by default; BullMQ retries of the same job keep the same key.
|
|
38
|
+
- **Cancel.** Aborting the run removes a step no worker has taken yet; one already running finishes (the worker's signal never aborts).
|
|
39
|
+
- **Deadline.** `timeoutMs` (default 900 s) per step; on expiry the step fails and the flow handles it like any failed step.
|
|
40
|
+
- **Names.** `prefix` (default `flow`) lets several environments share a Redis.
|
|
41
|
+
|
|
42
|
+
## A run that survives the process
|
|
43
|
+
|
|
44
|
+
```typescript
|
|
45
|
+
import { RedisRunJournal } from '@backendkit-labs/agent-flow-queue';
|
|
46
|
+
|
|
47
|
+
const journal = new RedisRunJournal({ redis: process.env.REDIS_URL!, prefix: 'flow' });
|
|
48
|
+
await runFlow(flow, { executor, journal, runId: 'close-2026-10', payload }); // run again with the same runId to resume
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
One Lua script per mutation makes "create only if absent", "append only if it exists" and "take the claim only if free or mine" atomic. The claim expires by Redis TTL, and releasing with an old token never frees a renewed claim. Logs do not expire on their own: delete finished runs with your own retention job.
|
|
52
|
+
|
|
53
|
+
> The scripts are covered by a contract suite that runs against a real Redis when `localhost:6379` answers (`REDIS_HOST`, `REDIS_PORT`) and against an in-memory stand-in otherwise.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@backendkit-labs/agent-flow-queue",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.1",
|
|
4
4
|
"description": "BullMQ StepExecutor for @backendkit-labs/agent-core flows: run each step on a worker in another process",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"types": "dist/index.d.ts",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"ioredis": "^5.0.0"
|
|
27
27
|
},
|
|
28
28
|
"peerDependencies": {
|
|
29
|
-
"@backendkit-labs/agent-core": ">=0.42.0 <0.
|
|
29
|
+
"@backendkit-labs/agent-core": ">=0.42.0 <0.45.0"
|
|
30
30
|
},
|
|
31
31
|
"devDependencies": {
|
|
32
32
|
"@types/jest": "^29.0.0",
|