@antelopejs/interface-redis-scheduler 0.0.9 → 0.0.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -20
- package/dist/index.d.ts +5 -0
- package/dist/index.js +45 -38
- package/dist/index.js.map +1 -1
- package/dist/tests/index.test.js +322 -144
- package/dist/tests/index.test.js.map +1 -1
- package/docs/1.introduction.md +95 -73
- package/docs/assets/task-delivery.svg +3 -3
- package/docs/tsconfig.json +7 -0
- package/package.json +11 -8
- package/skills/redis-scheduler-interface/SKILL.md +57 -29
- package/skills/redis-scheduler-interface/cases.json +57 -0
package/docs/1.introduction.md
CHANGED
|
@@ -1,37 +1,45 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
title: Redis scheduler
|
|
3
|
+
description: Schedule tasks in a Redis sorted set and run them with a listener, with the delivery, retry, and shutdown rules of each export.
|
|
4
|
+
mode: reference
|
|
5
|
+
---
|
|
2
6
|
|
|
3
7
|
## Overview
|
|
4
8
|
|
|
5
|
-
|
|
9
|
+
`@antelopejs/interface-redis-scheduler` stores tasks in a Redis sorted set, with each due timestamp as the score. A local timer checks for due tasks, and Redis pub/sub tells listeners when the schedule changes. A handler that throws is retried. The scheduler does not give exactly-once delivery.
|
|
6
10
|
|
|
7
|
-
|
|
11
|
+
The scheduler code runs in your own module. It reaches Redis through `GetClient()` from `@antelopejs/interface-redis`, so the project must load a module that implements that interface, such as `@antelopejs/redis`. Where Redis keeps its data, and whether queued entries survive a Redis restart, depends on that module's connection settings and the Redis server's persistence configuration.
|
|
12
|
+
|
|
13
|
+
A listener takes due tasks with one Lua script that reads and removes them in a single step. Several listeners can share one queue, and each due entry goes to one of them. The entry leaves Redis before its handler runs, so a process failure after the removal and before the handler completes loses the task. Make handlers idempotent, because a retry runs the handler again after a failure that may already have had effects.
|
|
14
|
+
|
|
15
|
+
A listener that takes an entry whose handler name it has not registered removes the entry and does not run it, retry it, or log it. Register every handler in every process that calls `enableListener()`.
|
|
8
16
|
|
|
9
17
|
## Schedule a task safely
|
|
10
18
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
19
|
+
A task runs as expected when these four conditions hold:
|
|
20
|
+
|
|
21
|
+
1. Every process that adds, removes, or runs tasks of a type registers that type with [`setHandler`](#sethandler).
|
|
22
|
+
2. Each listener starts with [`enableListener`](#enablelistener) after its handlers are registered.
|
|
23
|
+
3. The application saves the serialized payload, then calls [`addTask`](#addtask) with a due time in milliseconds.
|
|
24
|
+
4. [`removeTask`](#removetask) receives that saved payload. A payload rebuilt with a new timestamp does not match.
|
|
15
25
|
|
|
16
|
-
{width="480"}
|
|
17
27
|
|
|
18
|
-
|
|
28
|
+
The scheduler fits work whose loss or repetition your application can tolerate or reconcile. It has no exactly-once execution, no durable record of which process runs a task, and no API that stops or drains processing. Read [retry logic](#retry-logic), [shutdown limits](#disablelistener), and [known limitations](#known-limitations) before you depend on it.
|
|
19
29
|
|
|
20
|
-
## Getting
|
|
30
|
+
## Getting started
|
|
21
31
|
|
|
22
|
-
Install the package in your
|
|
32
|
+
Install the package in your module:
|
|
23
33
|
|
|
24
34
|
```bash
|
|
25
35
|
npm install @antelopejs/interface-redis-scheduler
|
|
26
36
|
```
|
|
27
37
|
|
|
28
|
-
> **Note:**
|
|
38
|
+
> **Note:** `@antelopejs/interface-redis` is a peer dependency. Install it too, and load a module that implements it, such as `@antelopejs/redis`, so the scheduler can connect to Redis.
|
|
29
39
|
|
|
30
40
|
## Try one scheduled task
|
|
31
41
|
|
|
32
|
-
|
|
33
|
-
with no other scheduler listener. It logs a reminder after about one minute;
|
|
34
|
-
it does not send an email or charge a payment.
|
|
42
|
+
This example runs in an initialized application against a disposable Redis database with no other scheduler listener. It logs a reminder after about one minute. It does not send an email or charge a payment.
|
|
35
43
|
|
|
36
44
|
```typescript
|
|
37
45
|
import {
|
|
@@ -52,26 +60,29 @@ const taskInfo = JSON.stringify({
|
|
|
52
60
|
await addTask("docs-reminder", Date.now() + 60_000, taskInfo);
|
|
53
61
|
```
|
|
54
62
|
|
|
55
|
-
|
|
56
|
-
in milliseconds, not a delay. Save `taskInfo` if you may later remove the queued
|
|
57
|
-
entry; [removal is not a cancellation guarantee](#removetask). The public API does
|
|
58
|
-
not fully stop timers, so end this disposable process when finished.
|
|
63
|
+
The process must stay alive for the reminder to appear. The due time is a Unix timestamp in milliseconds, not a delay. Save `taskInfo` if you may later remove the queued entry, and note that [removal does not guarantee cancellation](#removetask). The API cannot fully stop the timers, so end this disposable process when you are done.
|
|
59
64
|
|
|
60
|
-
## Core
|
|
65
|
+
## Core concepts
|
|
61
66
|
|
|
62
|
-
The scheduler
|
|
67
|
+
The scheduler has three parts:
|
|
63
68
|
|
|
64
|
-
- **Handlers** process tasks of
|
|
65
|
-
- **Tasks** are
|
|
66
|
-
- **
|
|
69
|
+
- **Handlers** process tasks of one type. Register them with `setHandler()` before you schedule tasks of that type.
|
|
70
|
+
- **Tasks** are entries in a Redis sorted set, scored by their due time. When a task is due, a listener calls the matching handler.
|
|
71
|
+
- **Listeners** watch the sorted set and run tasks when they fall due. Start one with `enableListener()`.
|
|
67
72
|
|
|
68
73
|
## `setHandler`
|
|
69
74
|
|
|
70
|
-
|
|
75
|
+
`setHandler` registers the handler for one task type. The name identifies the type, and the callback receives the task data as a string. The registration is local to the process and makes no Redis call. `sendEmail` below stands for a function in your application. It is not a scheduler export.
|
|
71
76
|
|
|
72
77
|
```typescript
|
|
73
78
|
import { setHandler } from "@antelopejs/interface-redis-scheduler";
|
|
74
79
|
|
|
80
|
+
declare function sendEmail(message: {
|
|
81
|
+
to: string;
|
|
82
|
+
subject: string;
|
|
83
|
+
body: string;
|
|
84
|
+
}): Promise<void>;
|
|
85
|
+
|
|
75
86
|
setHandler("send-email", async (taskInfo) => {
|
|
76
87
|
const emailData = JSON.parse(taskInfo);
|
|
77
88
|
await sendEmail({
|
|
@@ -82,7 +93,7 @@ setHandler("send-email", async (taskInfo) => {
|
|
|
82
93
|
});
|
|
83
94
|
```
|
|
84
95
|
|
|
85
|
-
|
|
96
|
+
A handler name must not contain the `:` character, because the scheduler uses it as a delimiter. Otherwise `setHandler` throws an `Error` with the message ``SchedulerUtil handler name may not contain `:` ``.
|
|
86
97
|
|
|
87
98
|
### Parameters
|
|
88
99
|
|
|
@@ -93,28 +104,27 @@ Handler names must not contain the `:` character, as the scheduler uses it inter
|
|
|
93
104
|
|
|
94
105
|
## `enableListener`
|
|
95
106
|
|
|
96
|
-
|
|
107
|
+
`enableListener` starts task processing in this process. It opens a dedicated Redis subscriber connection, which receives the update messages that `addTask` publishes. It then arms a timer for the next due task, or runs the tasks that are already due.
|
|
97
108
|
|
|
98
109
|
```typescript
|
|
99
110
|
import { enableListener } from "@antelopejs/interface-redis-scheduler";
|
|
100
111
|
|
|
101
112
|
async function startApp() {
|
|
102
|
-
// Register all handlers first
|
|
113
|
+
// Register all handlers first.
|
|
103
114
|
// ...
|
|
104
115
|
|
|
105
|
-
// Then start the listener
|
|
106
116
|
await enableListener();
|
|
107
117
|
console.log("Scheduler is active");
|
|
108
118
|
}
|
|
109
119
|
```
|
|
110
120
|
|
|
111
|
-
> **Important:** Register all task handlers before
|
|
121
|
+
> **Important:** Register all task handlers before you call `enableListener()`. The listener starts processing due tasks right away.
|
|
112
122
|
|
|
113
123
|
## `disableListener`
|
|
114
124
|
|
|
115
|
-
`disableListener()` returns `Promise<void>` and only calls `quit()` on the notification subscriber. Call it only after `enableListener()` has completed
|
|
125
|
+
`disableListener()` returns `Promise<void>` and only calls `quit()` on the notification subscriber. Call it only after `enableListener()` has completed. Before that, the subscriber does not exist and the call rejects.
|
|
116
126
|
|
|
117
|
-
> **Warning:** It does not clear the pending timer, cancel running handlers, or wait for
|
|
127
|
+
> **Warning:** It does not clear the pending timer, cancel running handlers, or wait for tasks to complete. A timer that is already armed can still run tasks and arm another timer through `updateTimer()`. Do not use this function as a pause, a drain, or a guarantee that no more tasks run. The public API has no operation that stops the timer.
|
|
118
128
|
|
|
119
129
|
```typescript
|
|
120
130
|
import { disableListener } from "@antelopejs/interface-redis-scheduler";
|
|
@@ -127,13 +137,13 @@ async function stopApp() {
|
|
|
127
137
|
|
|
128
138
|
## `addTask`
|
|
129
139
|
|
|
130
|
-
|
|
140
|
+
`addTask` schedules a task for a given time. It adds the task to the Redis sorted set with the due time as its score, then publishes an update message so listeners refresh their timers. While this process runs a batch of tasks, `addTask` does not publish. That process picks up the new task when its batch ends.
|
|
131
141
|
|
|
132
142
|
```typescript
|
|
133
143
|
import { addTask } from "@antelopejs/interface-redis-scheduler";
|
|
134
144
|
|
|
135
145
|
async function scheduleReminder(recipient: string) {
|
|
136
|
-
//
|
|
146
|
+
// Due in 3 days.
|
|
137
147
|
const dueTime = Date.now() + 3 * 24 * 60 * 60 * 1000;
|
|
138
148
|
|
|
139
149
|
const taskInfo = JSON.stringify({
|
|
@@ -148,21 +158,21 @@ async function scheduleReminder(recipient: string) {
|
|
|
148
158
|
}
|
|
149
159
|
```
|
|
150
160
|
|
|
151
|
-
|
|
161
|
+
`addTask` rejects with `Unknown SchedulerUtil handler: <name>` when this process has not registered the handler name.
|
|
152
162
|
|
|
153
|
-
The pair `handlerName`
|
|
163
|
+
The pair of `handlerName` and `taskInfo` identifies the queued entry. Adding the same pair again changes its due time and does not create a second task. To keep two otherwise identical jobs apart, put a stable job ID in the payload.
|
|
154
164
|
|
|
155
165
|
### Parameters
|
|
156
166
|
|
|
157
|
-
| Parameter | Type | Description
|
|
158
|
-
| ------------- | -------- |
|
|
159
|
-
| `handlerName` | `string` | The registered handler name that processes this task
|
|
160
|
-
| `dueTime` | `number` | Unix timestamp in milliseconds when the task
|
|
161
|
-
| `taskInfo` | `string` | Data passed to the handler
|
|
167
|
+
| Parameter | Type | Description |
|
|
168
|
+
| ------------- | -------- | --------------------------------------------------------- |
|
|
169
|
+
| `handlerName` | `string` | The registered handler name that processes this task |
|
|
170
|
+
| `dueTime` | `number` | Unix timestamp in milliseconds when the task is due |
|
|
171
|
+
| `taskInfo` | `string` | Data passed to the handler, usually a JSON-encoded string |
|
|
162
172
|
|
|
163
173
|
## `removeTask`
|
|
164
174
|
|
|
165
|
-
|
|
175
|
+
`removeTask` removes the original queued entry if it is still present. Pass the saved `taskInfo` string that `scheduleReminder` returned. Do not rebuild the JSON.
|
|
166
176
|
|
|
167
177
|
```typescript
|
|
168
178
|
import { removeTask } from "@antelopejs/interface-redis-scheduler";
|
|
@@ -172,11 +182,11 @@ async function removeQueuedReminder(savedTaskInfo: string) {
|
|
|
172
182
|
}
|
|
173
183
|
```
|
|
174
184
|
|
|
175
|
-
|
|
185
|
+
`removeTask` rejects with `Unknown SchedulerUtil handler: <name>` when this process has not registered the handler name.
|
|
176
186
|
|
|
177
|
-
> **Warning:** The `taskInfo` must
|
|
187
|
+
> **Warning:** The `taskInfo` must match the original string exactly. If the serialized JSON differs in key order or whitespace, nothing is removed and no error is raised.
|
|
178
188
|
|
|
179
|
-
Removal targets the original queued
|
|
189
|
+
Removal targets the original queued entry only. It does not stop a running handler, and it does not remove a retry stored with a `RETRY-` prefix. If a cancellation must survive races and retries, make the handler check cancellation state that your application owns before it applies effects.
|
|
180
190
|
|
|
181
191
|
### Parameters
|
|
182
192
|
|
|
@@ -185,64 +195,63 @@ Removal targets the original queued member only. It does not stop a running hand
|
|
|
185
195
|
| `handlerName` | `string` | The handler name associated with the task |
|
|
186
196
|
| `taskInfo` | `string` | The exact task info string used when adding the task |
|
|
187
197
|
|
|
188
|
-
## Retry
|
|
189
|
-
|
|
190
|
-
The scheduler automatically retries failed tasks. When a handler throws an error, the scheduler reschedules the task with a retry counter.
|
|
198
|
+
## Retry logic
|
|
191
199
|
|
|
192
|
-
|
|
193
|
-
- **Retry delay:** 5 seconds between each attempt
|
|
194
|
-
- **Behavior on exhaustion:** After 3 failed retries, the task is discarded; failures are reported through `Logging.Error`
|
|
200
|
+
The scheduler retries failed tasks with no extra code. When a handler throws or its promise rejects, the scheduler logs the error with `Logging.Error` and queues the task again with a retry counter.
|
|
195
201
|
|
|
196
|
-
|
|
202
|
+
- **Maximum retries.** 3 retries after the first attempt, so 4 attempts in total.
|
|
203
|
+
- **Retry delay.** Each retry is due 5 seconds after the attempt that failed.
|
|
204
|
+
- **Exhaustion.** After the third retry fails, the scheduler logs the error and discards the task.
|
|
205
|
+
- **Failed retry write.** When Redis rejects the write that queues a retry, the scheduler logs that error, the task gets no retry, and the batch goes on with its next task.
|
|
197
206
|
|
|
198
207
|
```typescript
|
|
199
208
|
import { setHandler } from "@antelopejs/interface-redis-scheduler";
|
|
200
209
|
|
|
201
|
-
|
|
210
|
+
declare function refreshSummary(data: unknown): Promise<void>;
|
|
211
|
+
|
|
212
|
+
// If this handler throws, the scheduler retries up to 3 times.
|
|
202
213
|
setHandler("refresh-summary", async (taskInfo) => {
|
|
203
214
|
const data = JSON.parse(taskInfo);
|
|
204
215
|
await refreshSummary(data);
|
|
205
|
-
// On failure: retry after 5s, then 5s, then 5s, then discard
|
|
216
|
+
// On failure: retry after 5s, then 5s, then 5s, then discard.
|
|
206
217
|
});
|
|
207
218
|
```
|
|
208
219
|
|
|
209
|
-
`refreshSummary` is an application function that recomputes a replaceable summary.
|
|
210
|
-
Choose retryable, idempotent work; a retry is not proof that the previous attempt
|
|
211
|
-
had no effect. Do not use this sketch as a durable payment-processing queue.
|
|
220
|
+
`refreshSummary` is an application function that recomputes a replaceable summary. Choose work that is safe to retry and idempotent, because a retry does not prove that the failed attempt had no effect. Do not use this pattern as a durable payment-processing queue.
|
|
212
221
|
|
|
213
|
-
## Redis
|
|
222
|
+
## Redis storage
|
|
214
223
|
|
|
215
|
-
The scheduler uses
|
|
224
|
+
The scheduler uses one Redis sorted set with the key `SchedulerUtil.Tasks`. Each entry has the form `handlerName:taskInfo`, and its score is the due timestamp in milliseconds.
|
|
216
225
|
|
|
217
|
-
|
|
226
|
+
A retried task has the form `RETRY-{count}:handlerName:taskInfo`, where `{count}` is the number of the retry, from 1 to 3.
|
|
218
227
|
|
|
219
|
-
The scheduler also uses a Redis pub/sub channel named `SchedulerUtil`
|
|
228
|
+
The scheduler also uses a Redis pub/sub channel named `SchedulerUtil`. `addTask` publishes the message `update` on it, and each listener that receives it refreshes its timer. The messages do not assign tasks to a listener. The Lua script that takes due tasks does that.
|
|
220
229
|
|
|
221
230
|
## `updateTimer`
|
|
222
231
|
|
|
223
|
-
|
|
232
|
+
`updateTimer` reads the first queued task. If that task is already due, `updateTimer` runs the due tasks. Otherwise it arms a timer for the task's due time, capped at 24 hours. The timer does not keep the process alive on its own. A process with nothing else pending exits, and the task stays queued in Redis for the next listener. A listener calls `updateTimer` when it starts, when it receives an update message, and after each batch of tasks. While this process runs a batch, `updateTimer` returns without doing anything. You do not need to call it in normal usage.
|
|
224
233
|
|
|
225
234
|
```typescript
|
|
226
235
|
import { updateTimer } from "@antelopejs/interface-redis-scheduler";
|
|
227
236
|
|
|
228
|
-
// Force a timer refresh (advanced usage)
|
|
237
|
+
// Force a timer refresh (advanced usage).
|
|
229
238
|
await updateTimer();
|
|
230
239
|
```
|
|
231
240
|
|
|
232
241
|
## `runTasks`
|
|
233
242
|
|
|
234
|
-
|
|
243
|
+
`runTasks` takes every task that is due and runs it. It removes the due tasks from Redis in one step, then calls their handlers one after another, earliest due time first. A task whose handler is not registered in this process is dropped. The retry logic applies to each handler that fails. When the batch ends, `runTasks` calls `updateTimer`. One process runs one batch at a time: a call to `runTasks` while a batch runs, from the timer, an update message or your code, returns at once, so the handlers of one listener never overlap. When the running batch resolves, its closing `updateTimer` runs the tasks that fell due meanwhile. When the batch rejects, for example because Redis fails while it takes due tasks, `runTasks` releases its lock and rejects without arming a timer, and a timer that fired during that batch is spent, so the due tasks run on the next update message or the next `updateTimer` call. The timer calls `runTasks` when it fires, so you do not need to call it in normal usage.
|
|
235
244
|
|
|
236
245
|
```typescript
|
|
237
246
|
import { runTasks } from "@antelopejs/interface-redis-scheduler";
|
|
238
247
|
|
|
239
|
-
//
|
|
248
|
+
// Run the due tasks now (advanced usage).
|
|
240
249
|
await runTasks();
|
|
241
250
|
```
|
|
242
251
|
|
|
243
|
-
## Complete
|
|
252
|
+
## Complete example
|
|
244
253
|
|
|
245
|
-
This
|
|
254
|
+
This sketch registers handlers, starts the listener, schedules tasks, and closes the subscriber. `sendEmail`, `deleteSession`, and `refreshSummary` in these examples stand for application functions you supply. They are not scheduler exports. The example needs a configured Redis implementation and does not start a service from this page.
|
|
246
255
|
|
|
247
256
|
```typescript
|
|
248
257
|
import {
|
|
@@ -253,7 +262,14 @@ import {
|
|
|
253
262
|
removeTask,
|
|
254
263
|
} from "@antelopejs/interface-redis-scheduler";
|
|
255
264
|
|
|
256
|
-
|
|
265
|
+
declare function sendEmail(
|
|
266
|
+
to: string,
|
|
267
|
+
subject: string,
|
|
268
|
+
body: string,
|
|
269
|
+
): Promise<void>;
|
|
270
|
+
declare function deleteSession(sessionId: string): Promise<void>;
|
|
271
|
+
|
|
272
|
+
// 1. Register task handlers.
|
|
257
273
|
setHandler("send-email", async (taskInfo) => {
|
|
258
274
|
const data = JSON.parse(taskInfo);
|
|
259
275
|
await sendEmail(data.to, data.subject, data.body);
|
|
@@ -264,14 +280,14 @@ setHandler("cleanup-session", async (taskInfo) => {
|
|
|
264
280
|
await deleteSession(sessionId);
|
|
265
281
|
});
|
|
266
282
|
|
|
267
|
-
// 2. Start the listener
|
|
283
|
+
// 2. Start the listener.
|
|
268
284
|
async function start() {
|
|
269
285
|
await enableListener();
|
|
270
286
|
}
|
|
271
287
|
|
|
272
|
-
// 3. Schedule tasks from your application logic
|
|
288
|
+
// 3. Schedule tasks from your application logic.
|
|
273
289
|
async function onUserSignup(userId: string, email: string) {
|
|
274
|
-
// Send a welcome email in 1 hour
|
|
290
|
+
// Send a welcome email in 1 hour.
|
|
275
291
|
const dueTime = Date.now() + 60 * 60 * 1000;
|
|
276
292
|
await addTask(
|
|
277
293
|
"send-email",
|
|
@@ -285,13 +301,19 @@ async function onUserSignup(userId: string, email: string) {
|
|
|
285
301
|
}
|
|
286
302
|
|
|
287
303
|
async function onUserLogout(sessionId: string) {
|
|
288
|
-
// Clean up session data in 24 hours
|
|
304
|
+
// Clean up session data in 24 hours.
|
|
289
305
|
const dueTime = Date.now() + 24 * 60 * 60 * 1000;
|
|
290
306
|
await addTask("cleanup-session", dueTime, JSON.stringify({ sessionId }));
|
|
291
307
|
}
|
|
292
308
|
|
|
293
|
-
// 4. Close notifications on shutdown
|
|
309
|
+
// 4. Close notifications on shutdown. Pending timers and handlers may continue.
|
|
294
310
|
async function shutdown() {
|
|
295
311
|
await disableListener();
|
|
296
312
|
}
|
|
297
313
|
```
|
|
314
|
+
|
|
315
|
+
## Known limitations
|
|
316
|
+
|
|
317
|
+
These behaviours are tracked as open issues:
|
|
318
|
+
|
|
319
|
+
- When `removeTask` deletes the first queued task while `updateTimer` reads it, no timer is armed, and the remaining tasks wait for the next update message ([#38](https://github.com/AntelopeJS/interface-redis-scheduler/issues/38)).
|
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
<svg xmlns="http://www.w3.org/2000/svg" width="480" height="760" viewBox="0 0 480 760" role="img" aria-labelledby="title desc">
|
|
2
2
|
<title id="title">Task delivery and the in-flight gap</title>
|
|
3
|
-
<desc id="desc">A timer reads due tasks from a sorted set
|
|
3
|
+
<desc id="desc">A timer reads and removes due tasks from a sorted set in one Lua script before calling handlers. Successful handlers finish. Thrown errors requeue after five seconds, up to three retries. A crash after removal can lose work; pub/sub only refreshes timers.</desc>
|
|
4
4
|
<defs><marker id="arrow" markerWidth="8" markerHeight="8" refX="7" refY="4" orient="auto"><path d="M0 0 8 4 0 8" fill="#627d98"/></marker></defs>
|
|
5
5
|
<rect width="480" height="760" rx="16" fill="#f4f7fb"/>
|
|
6
6
|
<g font-family="system-ui, sans-serif" fill="#17324d" font-size="18">
|
|
7
|
-
<text x="24" y="36" font-size="24" font-weight="700">Remove first. Run the handler.</text><text x="24" y="70">
|
|
7
|
+
<text x="24" y="36" font-size="24" font-weight="700">Remove first. Run the handler.</text><text x="24" y="70">Every listener registers every handler</text><text x="24" y="99">Application-owned recovery</text>
|
|
8
8
|
<rect x="24" y="126" width="400" height="86" rx="10" fill="#e2edf8" stroke="#b0c9e0"/><text x="44" y="160" font-size="22" font-weight="700">Queued in Redis</text><text x="44" y="190">Score = due time (milliseconds)</text>
|
|
9
9
|
<path d="M224 212V246" stroke="#627d98" stroke-width="2" marker-end="url(#arrow)"/>
|
|
10
|
-
<rect x="24" y="258" width="400" height="86" rx="10" fill="#fff0cf"/><text x="44" y="292" font-size="22" font-weight="700">Read + remove due tasks</text><text x="44" y="322">
|
|
10
|
+
<rect x="24" y="258" width="400" height="86" rx="10" fill="#fff0cf"/><text x="44" y="292" font-size="22" font-weight="700">Read + remove due tasks</text><text x="44" y="322">One Lua script, one listener per task</text>
|
|
11
11
|
<path d="M224 344V378" stroke="#627d98" stroke-width="2" marker-end="url(#arrow)"/>
|
|
12
12
|
<rect x="24" y="390" width="400" height="86" rx="10" fill="#17324d"/><text x="44" y="424" fill="#fff" font-size="22" font-weight="700">Run handler</text><text x="44" y="454" fill="#d5e5f3">Success → done</text>
|
|
13
13
|
<path d="M424 433H448V169H424" fill="none" stroke="#627d98" stroke-width="2" stroke-dasharray="5 5" marker-end="url(#arrow)"/>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@antelopejs/interface-redis-scheduler",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.11",
|
|
4
4
|
"description": "AntelopeJS interface for scheduling tasks on Redis, with retries on failure",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"antelopejs",
|
|
@@ -46,23 +46,23 @@
|
|
|
46
46
|
"test": "pnpm run build && ajs module test .",
|
|
47
47
|
"format:check": "oxfmt --check .",
|
|
48
48
|
"knip": "knip",
|
|
49
|
-
"prepare": "if
|
|
49
|
+
"prepare": "if [ -e .git ] && command -v git >/dev/null 2>&1 && command -v lefthook >/dev/null 2>&1 && GIT_DIR=.git GIT_WORK_TREE=. git rev-parse --git-dir >/dev/null 2>&1; then GIT_DIR=.git GIT_WORK_TREE=. lefthook install; fi",
|
|
50
50
|
"checks": "antelopejs-checks",
|
|
51
51
|
"size": "antelopejs-check-size",
|
|
52
|
-
"ratchet": "antelopejs-ratchet",
|
|
53
52
|
"sync": "antelopejs-sync",
|
|
54
|
-
"ratchet:update": "antelopejs-ratchet --update",
|
|
55
53
|
"sync:check": "antelopejs-sync --check",
|
|
56
|
-
"typecheck": "tsc --noEmit"
|
|
54
|
+
"typecheck": "tsc --noEmit",
|
|
55
|
+
"suppressions": "antelopejs-suppressions",
|
|
56
|
+
"suppressions:update": "antelopejs-suppressions --update"
|
|
57
57
|
},
|
|
58
58
|
"dependencies": {
|
|
59
59
|
"ioredis": "^5.10.1"
|
|
60
60
|
},
|
|
61
61
|
"devDependencies": {
|
|
62
|
-
"@antelopejs/core": "^1.
|
|
62
|
+
"@antelopejs/core": "^1.21.0",
|
|
63
63
|
"@antelopejs/interface-core": "^0.2.4",
|
|
64
64
|
"@antelopejs/interface-redis": "^0.0.9",
|
|
65
|
-
"@antelopejs/tooling-configs": "^0.
|
|
65
|
+
"@antelopejs/tooling-configs": "^0.4.0",
|
|
66
66
|
"@types/chai": "^4.3.20",
|
|
67
67
|
"@types/mocha": "^10.0.10",
|
|
68
68
|
"@types/node": "^22.19.15",
|
|
@@ -87,6 +87,9 @@
|
|
|
87
87
|
"test": "src/antelope.test.ts",
|
|
88
88
|
"skills": [
|
|
89
89
|
"./skills"
|
|
90
|
-
]
|
|
90
|
+
],
|
|
91
|
+
"docs": {
|
|
92
|
+
"standard": true
|
|
93
|
+
}
|
|
91
94
|
}
|
|
92
95
|
}
|
|
@@ -1,20 +1,25 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: redis-scheduler-interface
|
|
3
|
-
description: Distributed Redis-backed task scheduling for AntelopeJS
|
|
3
|
+
description: Distributed Redis-backed task scheduling for AntelopeJS. Register named handlers, schedule tasks at a due timestamp in a Redis sorted set, and execute them with automatic retries; listeners across instances claim due tasks atomically, and pub/sub only re-arms their timers. Use when code imports '@antelopejs/interface-redis-scheduler', when a task involves scheduling delayed/deferred jobs, reminders, or background work in an AntelopeJS module, or when symbols like setHandler, addTask, removeTask, enableListener, disableListener, or the SchedulerUtil.Tasks Redis key appear.
|
|
4
4
|
category: antelopejs-interface
|
|
5
5
|
tags: [antelopejs, redis, scheduler, tasks, background-jobs]
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Redis Scheduler Interface
|
|
9
9
|
|
|
10
|
-
Distributed task scheduler on top of Redis
|
|
11
|
-
(`SchedulerUtil.Tasks`) scored by their due timestamp
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
10
|
+
Distributed task scheduler on top of Redis. Tasks live in one sorted set
|
|
11
|
+
(`SchedulerUtil.Tasks`), scored by their due timestamp. A listener takes the
|
|
12
|
+
due tasks with one Lua script that reads and removes them in a single step, so
|
|
13
|
+
each due entry goes to exactly one listener, and several listeners can share
|
|
14
|
+
the queue. The `SchedulerUtil` pub/sub channel only carries an `update`
|
|
15
|
+
message that tells listeners to re-arm their timers; it does not decide who
|
|
16
|
+
runs a task.
|
|
17
|
+
|
|
18
|
+
Mental model: this is a consumer-side utility, not a proxied service. All
|
|
19
|
+
exported functions run in your own module. Redis access goes through
|
|
20
|
+
`GetClient()` from `@antelopejs/interface-redis` (`setHandler` itself is local
|
|
21
|
+
and makes no Redis call). There is no provider to write with
|
|
22
|
+
`ImplementInterface`. A Redis interface implementation (for example the
|
|
18
23
|
`@antelopejs/redis` module) must be loaded in the app.
|
|
19
24
|
|
|
20
25
|
## Imports
|
|
@@ -31,8 +36,8 @@ import {
|
|
|
31
36
|
} from "@antelopejs/interface-redis-scheduler";
|
|
32
37
|
```
|
|
33
38
|
|
|
34
|
-
|
|
35
|
-
execution loop
|
|
39
|
+
`updateTimer` and `runTasks` are also exported, but they belong to the internal
|
|
40
|
+
execution loop. Call them only in tests or advanced scenarios.
|
|
36
41
|
|
|
37
42
|
## Consumption example
|
|
38
43
|
|
|
@@ -66,28 +71,51 @@ await disableListener();
|
|
|
66
71
|
|
|
67
72
|
## Gotchas
|
|
68
73
|
|
|
69
|
-
- Handler names must not contain
|
|
74
|
+
- Handler names must not contain `:`, which delimits the sorted-set member
|
|
70
75
|
format `handlerName:taskInfo`. `setHandler` throws otherwise.
|
|
71
|
-
- `addTask` and `removeTask`
|
|
72
|
-
current process.
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
76
|
+
- `addTask` and `removeTask` reject with `Unknown SchedulerUtil handler: <name>`
|
|
77
|
+
for a handler name not registered in the current process. Call `setHandler`
|
|
78
|
+
before you schedule.
|
|
79
|
+
- Register every handler in every listener process. A listener that claims a
|
|
80
|
+
due entry whose handler it has not registered drops the entry: it does not
|
|
81
|
+
run it, retry it or log it.
|
|
82
|
+
- Register all handlers before `enableListener()`. The listener checks for due
|
|
83
|
+
tasks as soon as it starts.
|
|
84
|
+
- A claimed entry is removed from Redis before its handler runs. If the
|
|
85
|
+
process stops during the run, the task is lost. There is no exactly-once
|
|
86
|
+
execution.
|
|
87
|
+
- A batch runs its handlers one after another, earliest due time first.
|
|
88
|
+
- One process runs one batch at a time, so its handlers never overlap. A
|
|
89
|
+
`runTasks()` call while a batch runs returns at once; that batch picks up
|
|
90
|
+
the tasks that fell due meanwhile when it resolves. A batch that rejects
|
|
91
|
+
arms no timer, so those tasks wait for the next update message.
|
|
92
|
+
- `taskInfo` is a plain string. Serialize and parse it (typically JSON)
|
|
93
|
+
yourself.
|
|
94
|
+
- `removeTask` needs the exact original `taskInfo` string. A JSON payload with
|
|
95
|
+
a different key order or whitespace does not match, and nothing is removed.
|
|
96
|
+
- A handler that throws or rejects is retried up to 3 times. Each retry is due
|
|
97
|
+
5 seconds after the attempt that failed, and its member becomes
|
|
98
|
+
`RETRY-{n}:handlerName:taskInfo`. After the third retry fails, the task is
|
|
99
|
+
discarded. When Redis rejects the retry write, that task gets no retry and
|
|
100
|
+
the batch runs its next task. Every failure is logged through
|
|
101
|
+
interface-core `Logging.Error`.
|
|
102
|
+
- The listener's timer waits at most 24 hours before it checks again. It does
|
|
103
|
+
not keep the process alive on its own: a process with nothing else pending
|
|
104
|
+
exits, and its queued tasks stay in Redis for the next listener.
|
|
81
105
|
- `enableListener` duplicates the shared Redis client into a dedicated
|
|
82
|
-
subscriber connection
|
|
83
|
-
- `disableListener` only quits the subscriber connection
|
|
84
|
-
pending
|
|
85
|
-
|
|
106
|
+
subscriber connection. `disableListener` quits that subscriber.
|
|
107
|
+
- `disableListener` only quits the subscriber connection. It does not clear
|
|
108
|
+
the pending timer, cancel running handlers, or wait for tasks to complete.
|
|
109
|
+
An armed timer keeps running due tasks and arming the next timer through
|
|
110
|
+
`updateTimer()`, so tasks can keep running after it returns. Do not use it as
|
|
111
|
+
a pause, a drain, or a guarantee that no more tasks run: the public API has
|
|
112
|
+
no operation that stops the timer. It rejects if `enableListener` was never
|
|
113
|
+
called.
|
|
86
114
|
- Peer interfaces `@antelopejs/interface-core` and `@antelopejs/interface-redis`
|
|
87
115
|
must be installed alongside this package.
|
|
88
116
|
|
|
89
117
|
## Deeper reference
|
|
90
118
|
|
|
91
|
-
See this package's `docs/1.introduction.md` (
|
|
92
|
-
|
|
93
|
-
and `dist/index.d.ts`. Do not duplicate them here.
|
|
119
|
+
See this package's `docs/1.introduction.md` (overview, getting started, core
|
|
120
|
+
concepts, one section per function, retry logic, Redis storage, complete
|
|
121
|
+
example, known limitations) and `dist/index.d.ts`. Do not duplicate them here.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 1,
|
|
3
|
+
"cases": [
|
|
4
|
+
{
|
|
5
|
+
"id": "redis-scheduler-interface-delayed-job",
|
|
6
|
+
"area": "framework",
|
|
7
|
+
"style": "clean",
|
|
8
|
+
"prompt": "In my AntelopeJS module, schedule a reminder email to run 24 hours after a booking is created, and have it run on whichever instance is listening.",
|
|
9
|
+
"expect": {
|
|
10
|
+
"skills": ["redis-scheduler-interface"],
|
|
11
|
+
"acceptable": [
|
|
12
|
+
"redis-interface",
|
|
13
|
+
"ajs-consume-interface",
|
|
14
|
+
"antelopejs"
|
|
15
|
+
],
|
|
16
|
+
"mustMention": ["setHandler", "addTask"]
|
|
17
|
+
}
|
|
18
|
+
},
|
|
19
|
+
{
|
|
20
|
+
"id": "redis-scheduler-interface-delayed-job-casual",
|
|
21
|
+
"area": "framework",
|
|
22
|
+
"style": "casual",
|
|
23
|
+
"prompt": "need to run a job later, like in an hour, across my instances. whats the antelope way",
|
|
24
|
+
"expect": {
|
|
25
|
+
"skills": ["redis-scheduler-interface"],
|
|
26
|
+
"acceptable": [
|
|
27
|
+
"redis-interface",
|
|
28
|
+
"ajs-consume-interface",
|
|
29
|
+
"antelopejs"
|
|
30
|
+
],
|
|
31
|
+
"mustMention": ["addTask"]
|
|
32
|
+
}
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"id": "redis-scheduler-interface-redis-neighbour",
|
|
36
|
+
"area": "framework",
|
|
37
|
+
"style": "clean",
|
|
38
|
+
"prompt": "Cache the result of an expensive lookup in Redis for 10 minutes from my AntelopeJS module.",
|
|
39
|
+
"expect": {
|
|
40
|
+
"skills": ["redis-interface"],
|
|
41
|
+
"acceptable": ["ajs-consume-interface", "antelopejs"],
|
|
42
|
+
"avoid": ["redis-scheduler-interface"]
|
|
43
|
+
}
|
|
44
|
+
},
|
|
45
|
+
{
|
|
46
|
+
"id": "redis-scheduler-interface-two-listeners",
|
|
47
|
+
"area": "framework",
|
|
48
|
+
"style": "clean",
|
|
49
|
+
"prompt": "Three instances of my module call enableListener from @antelopejs/interface-redis-scheduler. Can a due task run on more than one of them?",
|
|
50
|
+
"expect": {
|
|
51
|
+
"skills": ["redis-scheduler-interface"],
|
|
52
|
+
"acceptable": ["redis-interface", "antelopejs"],
|
|
53
|
+
"mustMention": ["Lua"]
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
]
|
|
57
|
+
}
|