@backendkit-labs/agent-flow-queue 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +46 -8
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -1,15 +1,53 @@
1
1
  # @backendkit-labs/agent-flow-queue
2
2
 
3
- BullMQ `StepExecutor` for `runFlow` (SPEC-0023 D5): each step runs on a worker, maybe in another process.
3
+ Run [`agent-core` flows](../core/README.md#flows--declarative-and-durable) across processes: a BullMQ `StepExecutor` (each step runs on a worker, maybe on another machine) and a Redis `RunJournal` (a run survives the process and any process can resume it).
4
4
 
5
- ```ts
5
+ ```bash
6
+ npm install @backendkit-labs/agent-flow-queue @backendkit-labs/agent-core
7
+ ```
8
+
9
+ Needs a Redis. The flow engine stays in one process; only the steps travel.
10
+
11
+ ## Run steps on workers
12
+
13
+ ```typescript
6
14
  // orchestrator process
7
- const executor = new QueueStepExecutor({ redis: process.env.REDIS_URL! });
8
- await runFlow(flow, { executor });
15
+ import { loadFlow, runFlow } from '@backendkit-labs/agent-core';
16
+ import { QueueStepExecutor } from '@backendkit-labs/agent-flow-queue';
17
+
18
+ const executor = new QueueStepExecutor({ redis: process.env.REDIS_URL!, timeoutMs: 15 * 60_000 });
19
+ const result = await runFlow(await loadFlow('flows/close-month.yaml'), { executor, payload: { month: '2026-10' } });
20
+ await executor.close();
21
+ ```
22
+
23
+ ```typescript
24
+ // worker process: one queue `flow.<agent>` each, `concurrency` steps at a time
25
+ import { createStepWorker } from '@backendkit-labs/agent-flow-queue';
9
26
 
10
- // worker process (one queue flow.<agent> each)
11
- createStepWorker({ redis, agents: ["writer"], executor: localExecutor, concurrency: 4 });
27
+ const workers = createStepWorker({
28
+ redis: process.env.REDIS_URL!,
29
+ agents: ['sales', 'finance'],
30
+ concurrency: 4,
31
+ executor: { run: async (step, input) => ({ content: await localAgent(step.agent).run(input.prompt) }) },
32
+ });
33
+ // on shutdown: await workers.close();
12
34
  ```
13
35
 
14
- At-least-once: job id = `runId:stepId:attempt`; steps with external effects must use `input.idempotencyKey`.
15
- Cancelling the run removes a step no worker took yet. Default wait 900 s per step (`timeoutMs`).
36
+ - **At least once.** The job id is the step's idempotency key (`runId:stepId:attempt`), so enqueuing the same step twice is one job. A step with an external effect (payment, email) must use `input.idempotencyKey` in the worker too.
37
+ - **Retries belong to the engine.** `attempts` is 1 by default; BullMQ retries of the same job keep the same key.
38
+ - **Cancel.** Aborting the run removes a step no worker has taken yet; one already running finishes (the worker's signal never aborts).
39
+ - **Deadline.** `timeoutMs` (default 900 s) per step; on expiry the step fails and the flow handles it like any failed step.
40
+ - **Names.** `prefix` (default `flow`) lets several environments share a Redis.
41
+
42
+ ## A run that survives the process
43
+
44
+ ```typescript
45
+ import { RedisRunJournal } from '@backendkit-labs/agent-flow-queue';
46
+
47
+ const journal = new RedisRunJournal({ redis: process.env.REDIS_URL!, prefix: 'flow' });
48
+ await runFlow(flow, { executor, journal, runId: 'close-2026-10', payload }); // run again with the same runId to resume
49
+ ```
50
+
51
+ One Lua script per mutation makes "create only if absent", "append only if it exists" and "take the claim only if free or mine" atomic. The claim expires by Redis TTL, and releasing with an old token never frees a renewed claim. Logs do not expire on their own: delete finished runs with your own retention job.
52
+
53
+ > The scripts are covered by a contract suite that runs against a real Redis when `localhost:6379` answers (`REDIS_HOST`, `REDIS_PORT`) and against an in-memory stand-in otherwise.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@backendkit-labs/agent-flow-queue",
3
- "version": "0.1.0",
3
+ "version": "0.1.1",
4
4
  "description": "BullMQ StepExecutor for @backendkit-labs/agent-core flows: run each step on a worker in another process",
5
5
  "main": "dist/index.js",
6
6
  "types": "dist/index.d.ts",
@@ -26,7 +26,7 @@
26
26
  "ioredis": "^5.0.0"
27
27
  },
28
28
  "peerDependencies": {
29
- "@backendkit-labs/agent-core": ">=0.42.0 <0.44.0"
29
+ "@backendkit-labs/agent-core": ">=0.42.0 <0.45.0"
30
30
  },
31
31
  "devDependencies": {
32
32
  "@types/jest": "^29.0.0",