effect-mq 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +305 -25
- package/dist/Job.d.ts +118 -5
- package/dist/Job.d.ts.map +1 -1
- package/dist/Job.js +119 -4
- package/dist/Job.js.map +1 -1
- package/dist/JobStore.d.ts +260 -9
- package/dist/JobStore.d.ts.map +1 -1
- package/dist/JobStore.js +115 -1
- package/dist/JobStore.js.map +1 -1
- package/dist/MemoryJobStore.d.ts +38 -6
- package/dist/MemoryJobStore.d.ts.map +1 -1
- package/dist/MemoryJobStore.js +351 -47
- package/dist/MemoryJobStore.js.map +1 -1
- package/dist/Worker.d.ts +5 -1
- package/dist/Worker.d.ts.map +1 -1
- package/dist/Worker.js +117 -10
- package/dist/Worker.js.map +1 -1
- package/dist/{drizzle → drizzle-postgres}/DrizzleJobStore.d.ts +35 -2
- package/dist/drizzle-postgres/DrizzleJobStore.d.ts.map +1 -0
- package/dist/drizzle-postgres/DrizzleJobStore.js +941 -0
- package/dist/drizzle-postgres/DrizzleJobStore.js.map +1 -0
- package/dist/drizzle-postgres/index.d.ts.map +1 -0
- package/dist/drizzle-postgres/index.js.map +1 -0
- package/dist/drizzle-postgres/schema.d.ts +670 -0
- package/dist/drizzle-postgres/schema.d.ts.map +1 -0
- package/dist/drizzle-postgres/schema.js +150 -0
- package/dist/drizzle-postgres/schema.js.map +1 -0
- package/dist/redis/RedisJobStore.d.ts +58 -0
- package/dist/redis/RedisJobStore.d.ts.map +1 -0
- package/dist/redis/RedisJobStore.js +424 -0
- package/dist/redis/RedisJobStore.js.map +1 -0
- package/dist/redis/index.d.ts +9 -0
- package/dist/redis/index.d.ts.map +1 -0
- package/dist/redis/index.js +9 -0
- package/dist/redis/index.js.map +1 -0
- package/dist/redis/scripts.d.ts +181 -0
- package/dist/redis/scripts.d.ts.map +1 -0
- package/dist/redis/scripts.js +940 -0
- package/dist/redis/scripts.js.map +1 -0
- package/dist/testing/conformance.d.ts.map +1 -1
- package/dist/testing/conformance.js +502 -8
- package/dist/testing/conformance.js.map +1 -1
- package/package.json +8 -4
- package/src/Job.ts +301 -10
- package/src/JobStore.ts +373 -9
- package/src/MemoryJobStore.ts +440 -53
- package/src/Worker.ts +153 -10
- package/src/drizzle-postgres/DrizzleJobStore.ts +1311 -0
- package/src/drizzle-postgres/schema.ts +279 -0
- package/src/redis/RedisJobStore.ts +652 -0
- package/src/redis/index.ts +8 -0
- package/src/redis/scripts.ts +1055 -0
- package/src/testing/conformance.ts +665 -8
- package/dist/drizzle/DrizzleJobStore.d.ts.map +0 -1
- package/dist/drizzle/DrizzleJobStore.js +0 -426
- package/dist/drizzle/DrizzleJobStore.js.map +0 -1
- package/dist/drizzle/index.d.ts.map +0 -1
- package/dist/drizzle/index.js.map +0 -1
- package/dist/drizzle/schema.d.ts +0 -464
- package/dist/drizzle/schema.d.ts.map +0 -1
- package/dist/drizzle/schema.js +0 -68
- package/dist/drizzle/schema.js.map +0 -1
- package/src/drizzle/DrizzleJobStore.ts +0 -599
- package/src/drizzle/schema.ts +0 -116
- /package/dist/{drizzle → drizzle-postgres}/index.d.ts +0 -0
- /package/dist/{drizzle → drizzle-postgres}/index.js +0 -0
- /package/src/{drizzle → drizzle-postgres}/index.ts +0 -0
package/README.md
CHANGED
|
@@ -14,7 +14,8 @@ One package, tree-shakeable modules:
|
|
|
14
14
|
| Import | Contents | Extra peers |
|
|
15
15
|
| --- | --- | --- |
|
|
16
16
|
| `effect-mq` | `Job`, `JobStore`, `MemoryJobStore`, `Worker` | — |
|
|
17
|
-
| `effect-mq/drizzle` | drizzle schema factories + the Postgres `JobStore` | `drizzle-orm` (v1), `@effect/sql-pg` |
|
|
17
|
+
| `effect-mq/drizzle-postgres` | drizzle-postgres schema factories + the Postgres `JobStore` | `drizzle-orm` (v1), `@effect/sql-pg` |
|
|
18
|
+
| `effect-mq/redis` | the Redis `JobStore` (Lua-script atomicity) | a `Redis` service (`@effect/platform-node`/`-bun`) |
|
|
18
19
|
| `effect-mq/testing` | the `JobStore` conformance suite for driver authors | `@effect/vitest` |
|
|
19
20
|
|
|
20
21
|
## Five-minute tour
|
|
@@ -73,6 +74,156 @@ What you get out of the box:
|
|
|
73
74
|
and per-job retention via `keep: { count, age }`.
|
|
74
75
|
- **Graceful shutdown** — interrupting a worker releases in-flight jobs back
|
|
75
76
|
to `waiting` without consuming an attempt.
|
|
77
|
+
- **Repeatable jobs** — durable cron/interval schedules
|
|
78
|
+
(`MyJob.schedule(key, { cron })`) that fire exactly once per occurrence
|
|
79
|
+
across any number of workers.
|
|
80
|
+
- **Admin verbs** — `cancel` (including *running* jobs, whose handler fiber is
|
|
81
|
+
interrupted cross-process), `promote` (delayed → now), and queue-level
|
|
82
|
+
`pause`/`resume`.
|
|
83
|
+
- **Handler timeouts** — `defaults: { timeout: "30 seconds" }` interrupts the
|
|
84
|
+
handler cleanly and routes through normal retry accounting; something BullMQ
|
|
85
|
+
can't do to a running processor.
|
|
86
|
+
- **Unrecoverable errors** — `Job.unrecoverable(error)` or a `retryable`
|
|
87
|
+
predicate skips the remaining retry budget when retrying can't help.
|
|
88
|
+
- **Deduplication** — pending-dedup, throttle, debounce, and
|
|
89
|
+
replace-while-delayed via a dedup key that never touches your job ids.
|
|
90
|
+
|
|
91
|
+
## Repeatable jobs
|
|
92
|
+
|
|
93
|
+
Schedules are durable rows in the store — not process-local timers — so they
|
|
94
|
+
survive restarts and coordinate across workers. Ticks enqueue with a
|
|
95
|
+
slot-deterministic job id (`sched/<key>/<slot>`), which makes every occurrence
|
|
96
|
+
exactly-once no matter how many workers sweep:
|
|
97
|
+
|
|
98
|
+
```ts
|
|
99
|
+
// Create or replace (same key = replace; upsert is idempotent to deploy).
|
|
100
|
+
yield* SendDigest.schedule("daily", {
|
|
101
|
+
cron: "0 9 * * *", // or: every: "10 minutes"
|
|
102
|
+
tz: "America/New_York", // IANA zone, cron only
|
|
103
|
+
payload: { edition: "morning" }
|
|
104
|
+
})
|
|
105
|
+
|
|
106
|
+
yield* SendDigest.unschedule("daily")
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`cron` fires at the next matching occurrence; `every` first fires one interval
|
|
110
|
+
from now and stays on its original grid. If workers are down over several
|
|
111
|
+
occurrences, missed slots collapse into one run (the next sweep enqueues the
|
|
112
|
+
overdue slot once, then advances past `now`). Options mirror `enqueue`:
|
|
113
|
+
`metadata`, `priority`, `attempts`, `backoff`, `keep`, `timeout`.
|
|
114
|
+
|
|
115
|
+
## Timeouts, cancellation, and unrecoverable errors
|
|
116
|
+
|
|
117
|
+
Because handlers are Effect fibers, the runtime can *actually stop them* —
|
|
118
|
+
these verbs interrupt cleanly (finalizers run) instead of abandoning work:
|
|
119
|
+
|
|
120
|
+
```ts
|
|
121
|
+
class GenerateInvoice extends Job.make("generate-invoice", {
|
|
122
|
+
payload: { invoiceId: Schema.String },
|
|
123
|
+
error: InvoiceError,
|
|
124
|
+
defaults: { attempts: 5, timeout: "2 minutes" }, // per-run limit
|
|
125
|
+
retryable: (e) => e.reason !== "invoice-voided" // skip retries when futile
|
|
126
|
+
}) {}
|
|
127
|
+
|
|
128
|
+
// Inside a handler, mark a specific failure as not worth retrying:
|
|
129
|
+
Effect.fail(Job.unrecoverable(new InvoiceError({ reason: "customer deleted" })))
|
|
130
|
+
|
|
131
|
+
// From anywhere (a dashboard, another process):
|
|
132
|
+
yield* GenerateInvoice.cancel(jobId) // waiting/delayed: terminal immediately;
|
|
133
|
+
// running: the worker's next heartbeat
|
|
134
|
+
// interrupts the handler fiber
|
|
135
|
+
yield* GenerateInvoice.promote(jobId) // delayed -> runnable now
|
|
136
|
+
|
|
137
|
+
// Store-level (definition-free) equivalents for generic dashboards:
|
|
138
|
+
const store = yield* JobStore.JobStore
|
|
139
|
+
yield* store.cancel(jobId)
|
|
140
|
+
yield* store.pause(QueueName("email")) // claims stop; producers unaffected
|
|
141
|
+
yield* store.resume(QueueName("email")) // wakes idle workers immediately
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
A timed-out run records a `TimeoutError` defect in the attempt ledger and
|
|
145
|
+
consumes an attempt like any other failure. A cancelled job lands in the
|
|
146
|
+
terminal `cancelled` state with a `cancelled` ledger entry; `awaitResult`
|
|
147
|
+
treats it as a defect (`JobCancelledError`), not a typed failure.
|
|
148
|
+
|
|
149
|
+
## History retention
|
|
150
|
+
|
|
151
|
+
Two layers of control, both splittable by terminal state (completed jobs are
|
|
152
|
+
usually noise, failed ones evidence):
|
|
153
|
+
|
|
154
|
+
- **Per job**: `keep` prunes terminal records per name+state — flat
|
|
155
|
+
`{ count, age }` applies to all states, or split per state:
|
|
156
|
+
|
|
157
|
+
```ts
|
|
158
|
+
keep: { count: 100 } // all states
|
|
159
|
+
keep: { completed: { age: "1 day" }, failed: { age: "30 days" } } // split
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
- **Per store**: a retention ceiling swept periodically in the background —
|
|
163
|
+
one duration or a per-state split (an absent state has no ceiling; the
|
|
164
|
+
timer touches its rows only when they carry their own `keep.age`):
|
|
165
|
+
|
|
166
|
+
```ts
|
|
167
|
+
MemoryJobStore.layerWith({ historyTtl: "7 days" })
|
|
168
|
+
DrizzleJobStore.layer({ ...tables, historyTtl: { completed: "1 day", failed: "90 days" } })
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
The sweep honours `min(per-row keep.age, ceiling)`, so a job name that goes
|
|
172
|
+
quiet is still pruned on the timer, not only when its group is next acked.
|
|
173
|
+
|
|
174
|
+
## Custom job ids
|
|
175
|
+
|
|
176
|
+
Store-assigned ids default to a compact sequence (`j-<n>`). Bring your own
|
|
177
|
+
generator when you want globally unique or prefixed ids:
|
|
178
|
+
|
|
179
|
+
```ts
|
|
180
|
+
import { ulid } from "ulid"
|
|
181
|
+
|
|
182
|
+
DrizzleJobStore.layer({
|
|
183
|
+
jobs, attempts, schedules, queues,
|
|
184
|
+
idGenerator: ({ name }) => `${name}_${ulid()}` // sync or Effect-returning
|
|
185
|
+
})
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
The generator only runs for store-assigned ids — `idempotencyKey` ids and
|
|
189
|
+
repeatable-schedule tick ids stay deterministic (exactly-once depends on
|
|
190
|
+
them). Collisions are retried a bounded number of times, then the enqueue
|
|
191
|
+
fails; bring real entropy.
|
|
192
|
+
|
|
193
|
+
## Deduplication
|
|
194
|
+
|
|
195
|
+
Dedup is a **separate key**, not id derivation — your ids (explicit,
|
|
196
|
+
`idGenerator`, or store-assigned) are never rewritten. Keys are scoped per
|
|
197
|
+
job name and picked per definition or per enqueue:
|
|
198
|
+
|
|
199
|
+
```ts
|
|
200
|
+
class SyncBenefits extends Job.make("sync-benefits", {
|
|
201
|
+
payload: { employerId: Schema.String },
|
|
202
|
+
dedupe: ({ employerId }) => employerId // string shorthand = { key }
|
|
203
|
+
}) {}
|
|
204
|
+
|
|
205
|
+
// Per-enqueue, with modes:
|
|
206
|
+
yield* SyncBenefits.enqueue(payload, { dedupe: { key: "emp-1", ttl: "1 minute" } })
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
Four behaviors, composable from three fields:
|
|
210
|
+
|
|
211
|
+
| Options | Behavior |
|
|
212
|
+
| --- | --- |
|
|
213
|
+
| `{ key }` | dedupe while the keyed job is pending; finishing frees the key |
|
|
214
|
+
| `{ key, ttl }` | throttle: at most one job per window, even after completion |
|
|
215
|
+
| `{ key, ttl, extend: true }` | debounce: every dropped enqueue pushes the window out |
|
|
216
|
+
| `{ key, replace: true }` | while the keyed job is still delayed, the newest payload/metadata/priority/attempts/backoff/keep/timeout/delay replace it (same id; a `ttl` window re-arms) |
|
|
217
|
+
|
|
218
|
+
A deduplicated enqueue returns the keyed job's id (`duplicate: true` at the
|
|
219
|
+
store level). `idempotencyKey` still exists and is different on purpose: it
|
|
220
|
+
makes the job id *itself* deterministic (permanent identity, joinable from
|
|
221
|
+
your domain tables), while `dedupe` is temporal policy with its own lifecycle.
|
|
222
|
+
|
|
223
|
+
Postgres users: dedup adds one table and one jobs column — add
|
|
224
|
+
`export const jobDedupe = mqDedupe()` to your schema and `drizzle-kit
|
|
225
|
+
generate` diffs both (the table and the new `dedupe_key` column) into one
|
|
226
|
+
migration. Memory and Redis need nothing.
|
|
76
227
|
|
|
77
228
|
## Postgres through drizzle
|
|
78
229
|
|
|
@@ -93,15 +244,62 @@ migrations** — no library-run DDL, no parallel migration system:
|
|
|
93
244
|
|
|
94
245
|
```ts
|
|
95
246
|
// db/schema.ts
|
|
96
|
-
import { mqJobAttempts, mqJobs } from "effect-mq/drizzle"
|
|
247
|
+
import { mqDedupe, mqJobAttempts, mqJobs, mqQueueControl, mqSchedules } from "effect-mq/drizzle-postgres"
|
|
97
248
|
|
|
98
249
|
// The `name` column is typed to your job tags (derived, not hand-written):
|
|
99
|
-
type JobNames = typeof
|
|
250
|
+
type JobNames = typeof GenerateInvoice._tag | typeof SendEmail._tag
|
|
100
251
|
|
|
101
252
|
export const jobs = mqJobs<JobNames>() // default table: effect_mq_jobs
|
|
102
253
|
export const jobAttempts = mqJobAttempts(jobs) // default: effect_mq_job_attempts
|
|
254
|
+
export const jobSchedules = mqSchedules() // default: effect_mq_schedules
|
|
255
|
+
export const jobQueues = mqQueueControl() // default: effect_mq_queue_control
|
|
256
|
+
export const jobDedupe = mqDedupe() // default: effect_mq_dedupe
|
|
103
257
|
```
|
|
104
258
|
|
|
259
|
+
Need more indexes (the built-ins cover claiming, listing, metadata
|
|
260
|
+
containment, and `name`/`state`/`finishedAt` history)? Every factory takes an
|
|
261
|
+
`extraConfig` callback — the same shape as drizzle's third `pgTable` argument —
|
|
262
|
+
appended after the built-in indexes and owned by your migrations like
|
|
263
|
+
everything else:
|
|
264
|
+
|
|
265
|
+
```ts
|
|
266
|
+
export const jobs = mqJobs<JobNames>("effect_mq_jobs", {
|
|
267
|
+
extraConfig: (t) => [index("jobs_name_recent_idx").on(t.name, t.enqueuedAt.desc())]
|
|
268
|
+
})
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
### Custom columns
|
|
272
|
+
|
|
273
|
+
Need real columns — a tenant id you can FK, RLS-scope, and join — populated
|
|
274
|
+
at job creation instead of patched in from job logic? `extend` the jobs
|
|
275
|
+
table; at enqueue the store fills each extended column from the job's
|
|
276
|
+
`metadata` entry with the same TS key (the definition's
|
|
277
|
+
`metadata: (payload) => ...` is your creation-time hook), NULL when absent:
|
|
278
|
+
|
|
279
|
+
```ts
|
|
280
|
+
class SyncBenefits extends Job.make("sync-benefits", {
|
|
281
|
+
payload: { companyId: Schema.String, objectId: Schema.String },
|
|
282
|
+
metadata: ({ companyId, objectId }) => ({ companyId, objectId })
|
|
283
|
+
}) {}
|
|
284
|
+
|
|
285
|
+
export const jobs = mqJobs<JobNames>("effect_mq_jobs", {
|
|
286
|
+
extend: {
|
|
287
|
+
companyId: text("company_id").notNull(),
|
|
288
|
+
objectId: text("object_id")
|
|
289
|
+
},
|
|
290
|
+
extraConfig: (t) => [index("jobs_company_idx").on(t.companyId, t.state)]
|
|
291
|
+
})
|
|
292
|
+
|
|
293
|
+
// Coercion/renames when the metadata convention isn't enough:
|
|
294
|
+
DrizzleJobStore.layer({ ...tables, extraValues: ({ metadata }) => ({ companyId: metadata.companyId }) })
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
`db.select().from(jobs).where(eq(jobs.companyId, tenant))` is fully typed; a
|
|
298
|
+
dedupe `replace` rewrites the extended columns with the latest values; a
|
|
299
|
+
missing metadata key on a `NOT NULL` column fails the enqueue loudly. Memory
|
|
300
|
+
and Redis need nothing — there the metadata projection is already the
|
|
301
|
+
queryable surface (`store.list({ metadata: { companyId } })`).
|
|
302
|
+
|
|
105
303
|
```
|
|
106
304
|
drizzle-kit generate # emits the CREATE TABLE migration next to your others
|
|
107
305
|
drizzle-kit migrate
|
|
@@ -113,12 +311,18 @@ When a future effect-mq version changes the layout, the factory changes and
|
|
|
113
311
|
### 2. Provide the store layer
|
|
114
312
|
|
|
115
313
|
```ts
|
|
116
|
-
import { DrizzleJobStore } from "effect-mq/drizzle"
|
|
314
|
+
import { DrizzleJobStore } from "effect-mq/drizzle-postgres"
|
|
117
315
|
import { PgClient } from "@effect/sql-pg"
|
|
118
316
|
import { Layer, Redacted } from "effect"
|
|
119
|
-
import { jobAttempts, jobs } from "./db/schema.ts"
|
|
120
|
-
|
|
121
|
-
const JobStoreLive = DrizzleJobStore.layer({
|
|
317
|
+
import { jobAttempts, jobDedupe, jobQueues, jobs, jobSchedules } from "./db/schema.ts"
|
|
318
|
+
|
|
319
|
+
const JobStoreLive = DrizzleJobStore.layer({
|
|
320
|
+
jobs,
|
|
321
|
+
attempts: jobAttempts,
|
|
322
|
+
schedules: jobSchedules,
|
|
323
|
+
queues: jobQueues,
|
|
324
|
+
dedupe: jobDedupe
|
|
325
|
+
}).pipe(
|
|
122
326
|
Layer.provide(PgClient.layer({ url: Redacted.make(process.env.DATABASE_URL!) }))
|
|
123
327
|
)
|
|
124
328
|
```
|
|
@@ -132,8 +336,8 @@ Product UIs read the tables directly with drizzle — fully typed:
|
|
|
132
336
|
|
|
133
337
|
```ts
|
|
134
338
|
db.select().from(jobs).where(and(
|
|
135
|
-
eq(jobs.name, "
|
|
136
|
-
sql`${jobs.metadata} @> ${{
|
|
339
|
+
eq(jobs.name, "generate-invoice"), // a typo is a compile error
|
|
340
|
+
sql`${jobs.metadata} @> ${{ customerId }}::jsonb` // GIN-indexed containment
|
|
137
341
|
))
|
|
138
342
|
|
|
139
343
|
db.select().from(jobAttempts)
|
|
@@ -145,9 +349,44 @@ db.select().from(jobAttempts)
|
|
|
145
349
|
`MyJob.retry(id)`, `store.remove(id)` — so lock tokens, attempt accounting,
|
|
146
350
|
and wake-up notifications stay coherent.
|
|
147
351
|
|
|
148
|
-
Worker tip: `awaitWake` is LISTEN/NOTIFY-driven
|
|
149
|
-
|
|
150
|
-
|
|
352
|
+
Worker tip: `awaitWake` is LISTEN/NOTIFY-driven and **queue-filtered** (the
|
|
353
|
+
NOTIFY payload names the queue, so an enqueue wakes only the takers watching
|
|
354
|
+
that queue — many queues don't amplify into many claims). The worker's
|
|
355
|
+
`pollInterval` is just the fallback; the 5s default is fine.
|
|
356
|
+
|
|
357
|
+
## Redis store
|
|
358
|
+
|
|
359
|
+
The Redis store (`effect-mq/redis`) implements every `JobStore` operation as
|
|
360
|
+
one atomic Lua script, so it is safe across any number of producer and worker
|
|
361
|
+
processes. It builds on Effect's client-agnostic `Redis` service — provide it
|
|
362
|
+
from your platform package (no extra peers beyond what you already run):
|
|
363
|
+
|
|
364
|
+
```ts
|
|
365
|
+
import { RedisJobStore } from "effect-mq/redis"
|
|
366
|
+
import { NodeRedis } from "@effect/platform-node" // node-redis under the hood
|
|
367
|
+
// import { BunRedis } from "@effect/platform-bun" // Bun.redis under the hood
|
|
368
|
+
import { Layer } from "effect"
|
|
369
|
+
|
|
370
|
+
const JobStoreLive = RedisJobStore.layer({
|
|
371
|
+
prefix: "myapp-jobs", // key namespace (default "effect-mq")
|
|
372
|
+
historyTtl: "30 days" // optional retention ceiling
|
|
373
|
+
}).pipe(
|
|
374
|
+
Layer.provide(NodeRedis.layer({ url: process.env.REDIS_URL }))
|
|
375
|
+
)
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Wake-ups ride pub/sub (`<prefix>:wake`) with queue-filtered messages, so
|
|
379
|
+
idle workers in other processes pick new jobs up promptly and an enqueue
|
|
380
|
+
wakes only the takers watching its queue; the worker's `pollInterval` is the
|
|
381
|
+
fallback. Notes:
|
|
382
|
+
|
|
383
|
+
- Keys are plain-prefixed (no hash tags) — point it at a single Redis /
|
|
384
|
+
Valkey node or a cluster-unaware proxy, not Redis Cluster.
|
|
385
|
+
- `list` filters scan server-side in Lua: fine for dashboards, not for
|
|
386
|
+
millions of terminal rows — set `historyTtl`/`keep` accordingly. `counts`
|
|
387
|
+
is O(1) (maintained counters).
|
|
388
|
+
- The same conformance suite that runs against Postgres runs against a real
|
|
389
|
+
Redis in this repo, TestClock included (scripts take time via ARGV).
|
|
151
390
|
|
|
152
391
|
## Multiple stores on different infrastructure
|
|
153
392
|
|
|
@@ -158,17 +397,17 @@ disposable ones live elsewhere — enforced by the type system:
|
|
|
158
397
|
import { Job, JobStore, MemoryJobStore, Worker } from "effect-mq"
|
|
159
398
|
|
|
160
399
|
const Durable = JobStore.named("durable") // -> Postgres in prod
|
|
161
|
-
const Ephemeral = JobStore.named("ephemeral") // -> memory
|
|
400
|
+
const Ephemeral = JobStore.named("ephemeral") // -> Redis or memory
|
|
162
401
|
|
|
163
|
-
class
|
|
164
|
-
payload: {
|
|
165
|
-
idempotencyKey: ({
|
|
402
|
+
class GenerateInvoice extends Job.make("generate-invoice", {
|
|
403
|
+
payload: { invoiceId: Schema.String },
|
|
404
|
+
idempotencyKey: ({ invoiceId }) => invoiceId,
|
|
166
405
|
store: Durable
|
|
167
406
|
}) {}
|
|
168
407
|
|
|
169
408
|
// Forgetting the Durable layer is now a COMPILE error at every enqueue site.
|
|
170
409
|
// Workers bind to one store:
|
|
171
|
-
const durableWorkers =
|
|
410
|
+
const durableWorkers = GenerateInvoice.toLayer(handler).pipe(
|
|
172
411
|
Layer.provide(Worker.layer({ store: Durable })) // local provide: several
|
|
173
412
|
) // workers can coexist
|
|
174
413
|
```
|
|
@@ -181,11 +420,11 @@ a **store** is an infrastructure/durability domain hosting many queues.
|
|
|
181
420
|
|
|
182
421
|
Two kinds of "business context", two homes:
|
|
183
422
|
|
|
184
|
-
- **Ops UI** ("list
|
|
423
|
+
- **Ops UI** ("list invoice runs for customer X, retry that one"): use the
|
|
185
424
|
`metadata` projection — a flat `Record<string, string>` derived from the
|
|
186
425
|
payload, indexed by every driver, filterable via `store.list` or raw SQL.
|
|
187
|
-
- **Domain history** ("what did this
|
|
188
|
-
joined by the *deterministic* job id from `idempotencyKey`. The queue's
|
|
426
|
+
- **Domain history** ("what did this invoice run actually produce"): your own
|
|
427
|
+
table, joined by the *deterministic* job id from `idempotencyKey`. The queue's
|
|
189
428
|
retention (`keep`) can then prune freely while your business history lives
|
|
190
429
|
forever. Don't let queue infrastructure own business data lifecycles.
|
|
191
430
|
|
|
@@ -205,12 +444,51 @@ store un-encoded. For `Schema.Redacted` fields, Effect's semantics apply:
|
|
|
205
444
|
|
|
206
445
|
Both behaviors are pinned by tests.
|
|
207
446
|
|
|
447
|
+
## Reference: knobs at a glance
|
|
448
|
+
|
|
449
|
+
Everything a job definition, an enqueue, and a worker can be tuned with:
|
|
450
|
+
|
|
451
|
+
**`Job.make(name, options)`** — `payload` (schema or struct fields),
|
|
452
|
+
`success`/`error` schemas, `idempotencyKey(payload)`, `dedupe(payload)`,
|
|
453
|
+
`metadata(payload)`, `retryable(error)`, `queue`, `store`, and `defaults`
|
|
454
|
+
(any per-enqueue option below).
|
|
455
|
+
|
|
456
|
+
**Per enqueue** (`enqueue`/`execute` options) — `jobId`, `queue`,
|
|
457
|
+
`metadata`, `dedupe`, `delay`, `priority` (higher first), `attempts`,
|
|
458
|
+
`backoff` (`fixed`/`exponential`), `keep` (`count`/`age`), `timeout`.
|
|
459
|
+
|
|
460
|
+
**Job verbs** — `enqueue`, `execute` (enqueue + await the typed result),
|
|
461
|
+
`poll`, `awaitResult`, `attempts` (the decoded run ledger), `retry`,
|
|
462
|
+
`cancel`, `promote`, `schedule`/`unschedule`, `toLayer` (register the
|
|
463
|
+
handler).
|
|
464
|
+
|
|
465
|
+
**`Worker.layer(options)`** — all durations take `Duration.Input`:
|
|
466
|
+
|
|
467
|
+
| Option | Default | Meaning |
|
|
468
|
+
| --- | --- | --- |
|
|
469
|
+
| `store` | default `JobStore` | which named store this worker claims from |
|
|
470
|
+
| `concurrency` | 1 | taker fibers per queue |
|
|
471
|
+
| `queues` | — | per-queue overrides: `{ email: { concurrency: 5 } }` |
|
|
472
|
+
| `lockDuration` | 30s | how long a claim's lock lasts before it counts as stalled |
|
|
473
|
+
| `lockRenewInterval` | half of `lockDuration` | heartbeat cadence (also delivers cross-process cancels) |
|
|
474
|
+
| `stalledInterval` | 30s | how often to sweep for stalled jobs |
|
|
475
|
+
| `maxStalledCount` | 1 | stalls tolerated before a job is failed outright |
|
|
476
|
+
| `pollInterval` | 5s | idle fallback when no wake-up arrives (wake-ups are queue-filtered and push-based, so the default is fine even on Postgres) |
|
|
477
|
+
| `scheduleSweepInterval` | 15s | how often to tick due repeatable-job schedules |
|
|
478
|
+
| `id` | random | identifier used in lock tokens |
|
|
479
|
+
|
|
480
|
+
**Store construction** — every driver accepts `idGenerator`, `historyTtl`,
|
|
481
|
+
and `historySweepInterval`; drizzle-postgres additionally takes the table
|
|
482
|
+
instances (+ `validate`), Redis a key `prefix`.
|
|
483
|
+
|
|
208
484
|
## Writing a storage driver
|
|
209
485
|
|
|
210
486
|
Implement the `JobStore` service (one atomic seam: `enqueue`, `claim`, `ack`,
|
|
211
487
|
`release`, `extendLocks`, `recoverStalled`, `awaitWake`, `getJob`,
|
|
212
|
-
`getAttempts`, `list`, `retry`, `counts`, `remove`
|
|
213
|
-
|
|
488
|
+
`getAttempts`, `list`, `retry`, `counts`, `remove`, `cancel`, `promote`,
|
|
489
|
+
`pause`/`resume`/`pausedQueues`, and the schedule ops
|
|
490
|
+
`upsertSchedule`/`removeSchedule`/`listSchedules`/`dueSchedules`/`advanceSchedule`)
|
|
491
|
+
and run the conformance suite against it:
|
|
214
492
|
|
|
215
493
|
```ts
|
|
216
494
|
import { jobStoreConformance } from "effect-mq/testing"
|
|
@@ -226,9 +504,11 @@ suite in this repo runs the same conformance tests against a real database.
|
|
|
226
504
|
|
|
227
505
|
## Roadmap
|
|
228
506
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
-
|
|
507
|
+
Next up: trace propagation, a cross-process event stream, batch enqueue,
|
|
508
|
+
custom drizzle columns, and parent-child fan-out. Full prioritized list:
|
|
509
|
+
[ROADMAP.md](https://github.com/TeamWarp/effect-mq/blob/main/ROADMAP.md);
|
|
510
|
+
release history:
|
|
511
|
+
[CHANGELOG.md](https://github.com/TeamWarp/effect-mq/blob/main/CHANGELOG.md).
|
|
232
512
|
|
|
233
513
|
## License
|
|
234
514
|
|
package/dist/Job.d.ts
CHANGED
|
@@ -31,7 +31,15 @@
|
|
|
31
31
|
* @since 0.1.0
|
|
32
32
|
*/
|
|
33
33
|
import { type Context, Duration, Effect, type Exit, Layer, Option, Schedule, Schema } from "effect";
|
|
34
|
-
import { type BackoffPolicy, JobId, JobNotFoundError, type JobNotRetryableError, type JobState, JobStore, type KeepPolicy, QueueName, type Service as StoreService } from "./JobStore.ts";
|
|
34
|
+
import { type BackoffPolicy, JobId, type JobNotCancellableError, JobNotFoundError, type JobNotPromotableError, type JobNotRetryableError, type JobState, JobStore, type KeepPolicy, QueueName, ScheduleKey, type Service as StoreService, unrecoverable } from "./JobStore.ts";
|
|
35
|
+
export {
|
|
36
|
+
/**
|
|
37
|
+
* Mark an error value as unrecoverable: the worker skips the remaining
|
|
38
|
+
* retry budget and fails the job immediately. Returns the error unchanged.
|
|
39
|
+
*
|
|
40
|
+
* @since 0.2.0
|
|
41
|
+
*/
|
|
42
|
+
unrecoverable };
|
|
35
43
|
import { type JobContext, type RegisterOptions, Worker } from "./Worker.ts";
|
|
36
44
|
declare const TypeId: "~effect-mq/Job";
|
|
37
45
|
/**
|
|
@@ -58,12 +66,30 @@ export interface BackoffInput {
|
|
|
58
66
|
*
|
|
59
67
|
* @since 0.1.0
|
|
60
68
|
*/
|
|
61
|
-
export interface
|
|
69
|
+
export interface KeepStateInput {
|
|
62
70
|
/** Keep at most this many terminal records (per name + state). */
|
|
63
71
|
readonly count?: number | undefined;
|
|
64
72
|
/** Remove terminal records older than this. */
|
|
65
73
|
readonly age?: Duration.Input | undefined;
|
|
66
74
|
}
|
|
75
|
+
/**
|
|
76
|
+
* Retention input: a flat `{ count, age }` applies to all terminal states;
|
|
77
|
+
* the split form sets independent rules per state (completed jobs are
|
|
78
|
+
* usually noise, failed ones evidence) — an absent state keeps its records
|
|
79
|
+
* until the store's `historyTtl` ceiling:
|
|
80
|
+
*
|
|
81
|
+
* ```ts
|
|
82
|
+
* keep: { count: 100 } // all states
|
|
83
|
+
* keep: { completed: { age: "1 day" }, failed: { age: "30 days" } }
|
|
84
|
+
* ```
|
|
85
|
+
*
|
|
86
|
+
* @since 0.1.0
|
|
87
|
+
*/
|
|
88
|
+
export type KeepInput = KeepStateInput | {
|
|
89
|
+
readonly completed?: KeepStateInput | undefined;
|
|
90
|
+
readonly failed?: KeepStateInput | undefined;
|
|
91
|
+
readonly cancelled?: KeepStateInput | undefined;
|
|
92
|
+
};
|
|
67
93
|
/**
|
|
68
94
|
* Options shared between job defaults and per-enqueue overrides.
|
|
69
95
|
*
|
|
@@ -79,7 +105,35 @@ export interface JobOptions {
|
|
|
79
105
|
readonly backoff?: BackoffInput | undefined;
|
|
80
106
|
/** Retention for terminal records. Default: keep forever. */
|
|
81
107
|
readonly keep?: KeepInput | undefined;
|
|
108
|
+
/**
|
|
109
|
+
* Per-run execution time limit; the worker interrupts the handler fiber
|
|
110
|
+
* past it and the run counts as a failed attempt (retried per backoff).
|
|
111
|
+
*/
|
|
112
|
+
readonly timeout?: Duration.Input | undefined;
|
|
82
113
|
}
|
|
114
|
+
/**
|
|
115
|
+
* @since 0.1.0
|
|
116
|
+
*/
|
|
117
|
+
/**
|
|
118
|
+
* Deduplication input for `Job.make({ dedupe })` and per-enqueue overrides.
|
|
119
|
+
* A bare string is shorthand for `{ key }`. Dedup never changes the job id —
|
|
120
|
+
* it is a separate, name-scoped key:
|
|
121
|
+
*
|
|
122
|
+
* - `{ key }`: dedupe while the keyed job is pending; done jobs free the key.
|
|
123
|
+
* - `{ key, ttl }`: throttle — at most one job per window.
|
|
124
|
+
* - `{ key, ttl, extend: true }`: debounce — dropped enqueues push the
|
|
125
|
+
* window out.
|
|
126
|
+
* - `{ key, replace: true }`: while the keyed job is still delayed, the new
|
|
127
|
+
* enqueue's content replaces it (latest wins).
|
|
128
|
+
*
|
|
129
|
+
* @since 0.3.0
|
|
130
|
+
*/
|
|
131
|
+
export type DedupeInput = string | {
|
|
132
|
+
readonly key: string;
|
|
133
|
+
readonly ttl?: Duration.Input | undefined;
|
|
134
|
+
readonly extend?: boolean | undefined;
|
|
135
|
+
readonly replace?: boolean | undefined;
|
|
136
|
+
};
|
|
83
137
|
/**
|
|
84
138
|
* @since 0.1.0
|
|
85
139
|
*/
|
|
@@ -94,6 +148,8 @@ export interface EnqueueOptions extends JobOptions {
|
|
|
94
148
|
readonly queue?: string | undefined;
|
|
95
149
|
/** Queryable business context, merged over the definition's `metadata`. */
|
|
96
150
|
readonly metadata?: Readonly<Record<string, string>> | undefined;
|
|
151
|
+
/** Deduplicate by key (see `DedupeInput`); overrides the definition's `dedupe`. */
|
|
152
|
+
readonly dedupe?: DedupeInput | undefined;
|
|
97
153
|
}
|
|
98
154
|
interface ResolvedDefaults {
|
|
99
155
|
readonly delayMs: number;
|
|
@@ -101,6 +157,27 @@ interface ResolvedDefaults {
|
|
|
101
157
|
readonly attempts: number;
|
|
102
158
|
readonly backoff: BackoffPolicy | undefined;
|
|
103
159
|
readonly keep: KeepPolicy | undefined;
|
|
160
|
+
readonly timeoutMs: number | undefined;
|
|
161
|
+
}
|
|
162
|
+
/**
|
|
163
|
+
* Options for `Job.schedule`. Exactly one of `cron` (with optional IANA `tz`)
|
|
164
|
+
* or `every` must be set. `cron` schedules first fire at the next matching
|
|
165
|
+
* occurrence; `every` schedules first fire one interval from now.
|
|
166
|
+
*
|
|
167
|
+
* @since 0.2.0
|
|
168
|
+
*/
|
|
169
|
+
export interface ScheduleOptions<PayloadInput> {
|
|
170
|
+
readonly cron?: string | undefined;
|
|
171
|
+
readonly tz?: string | undefined;
|
|
172
|
+
readonly every?: Duration.Input | undefined;
|
|
173
|
+
readonly payload: PayloadInput;
|
|
174
|
+
/** Queryable business context, merged over the definition's `metadata`. */
|
|
175
|
+
readonly metadata?: Readonly<Record<string, string>> | undefined;
|
|
176
|
+
readonly priority?: number | undefined;
|
|
177
|
+
readonly attempts?: number | undefined;
|
|
178
|
+
readonly backoff?: BackoffInput | undefined;
|
|
179
|
+
readonly keep?: KeepInput | undefined;
|
|
180
|
+
readonly timeout?: Duration.Input | undefined;
|
|
104
181
|
}
|
|
105
182
|
/**
|
|
106
183
|
* The status of a job as seen by `poll`. Fetch the full run ledger with
|
|
@@ -125,8 +202,8 @@ export interface JobAttempt<A, E> {
|
|
|
125
202
|
readonly attempt: number;
|
|
126
203
|
readonly startedAt: number | undefined;
|
|
127
204
|
readonly finishedAt: number;
|
|
128
|
-
readonly outcome: "completed" | "retried" | "failed" | "stalled";
|
|
129
|
-
/** Absent for `stalled` entries. */
|
|
205
|
+
readonly outcome: "completed" | "retried" | "failed" | "stalled" | "cancelled";
|
|
206
|
+
/** Absent for `stalled` and `cancelled` entries. */
|
|
130
207
|
readonly exit: Option.Option<Exit.Exit<A, E>>;
|
|
131
208
|
}
|
|
132
209
|
/**
|
|
@@ -151,7 +228,9 @@ export interface Job<Name extends string, Payload extends AnyStructSchema, Succe
|
|
|
151
228
|
/** JSON codec for handler exits — what is actually persisted. */
|
|
152
229
|
readonly exitSchema: Schema.toCodecJson<Schema.Exit<Schema.toCodecJson<Success>, Schema.toCodecJson<Error>, Schema.Defect>>;
|
|
153
230
|
readonly idempotencyKey: ((payload: Payload["Type"]) => string) | undefined;
|
|
231
|
+
readonly dedupe: ((payload: Payload["Type"]) => DedupeInput) | undefined;
|
|
154
232
|
readonly metadata: ((payload: Payload["Type"]) => Readonly<Record<string, string>>) | undefined;
|
|
233
|
+
readonly retryable: ((error: Error["Type"]) => boolean) | undefined;
|
|
155
234
|
readonly defaults: ResolvedDefaults;
|
|
156
235
|
/**
|
|
157
236
|
* Queue this job. Returns the job id. Duplicate ids (via `jobId` or
|
|
@@ -176,6 +255,30 @@ export interface Job<Name extends string, Payload extends AnyStructSchema, Succe
|
|
|
176
255
|
* preserved. (The admin op behind a dashboard's "retry" button.)
|
|
177
256
|
*/
|
|
178
257
|
readonly retry: (jobId: JobId) => Effect.Effect<void, JobNotFoundError | JobNotRetryableError, StoreId>;
|
|
258
|
+
/**
|
|
259
|
+
* Cancel a job: waiting/delayed become terminal (`cancelled`) immediately;
|
|
260
|
+
* a running job's handler fiber is interrupted by its worker on the next
|
|
261
|
+
* heartbeat (latency ≤ `lockRenewInterval`).
|
|
262
|
+
*/
|
|
263
|
+
readonly cancel: (jobId: JobId) => Effect.Effect<void, JobNotFoundError | JobNotCancellableError, StoreId>;
|
|
264
|
+
/** Run a delayed job now. */
|
|
265
|
+
readonly promote: (jobId: JobId) => Effect.Effect<void, JobNotFoundError | JobNotPromotableError, StoreId>;
|
|
266
|
+
/**
|
|
267
|
+
* Create or replace a durable repeatable schedule for this job. Ticks
|
|
268
|
+
* enqueue with a slot-deterministic id, so schedules are exactly-once per
|
|
269
|
+
* occurrence across all workers (assuming history retention windows
|
|
270
|
+
* comfortably exceed the sweep interval — a pruned tick job cannot dedup a
|
|
271
|
+
* pathologically stale sweeper). Missed occurrences (downtime) collapse
|
|
272
|
+
* into a single catch-up run.
|
|
273
|
+
*
|
|
274
|
+
* Re-registering with an *unchanged* cadence (same `cron`/`tz`/`every`) is
|
|
275
|
+
* a no-op for the next occurrence — deploy-time re-registration neither
|
|
276
|
+
* re-anchors `every` grids nor drops a pending catch-up run. Changing the
|
|
277
|
+
* cadence resets the next occurrence.
|
|
278
|
+
*/
|
|
279
|
+
readonly schedule: (key: string, options: ScheduleOptions<Payload["~type.make.in"]>) => Effect.Effect<ScheduleKey, never, StoreId | Payload["EncodingServices"]>;
|
|
280
|
+
/** Remove a schedule created by `schedule`. False when it did not exist. */
|
|
281
|
+
readonly unschedule: (key: string) => Effect.Effect<boolean, never, StoreId>;
|
|
179
282
|
/**
|
|
180
283
|
* Attach the handler that processes this job, as a layer to provide on top
|
|
181
284
|
* of `Worker.layer` (bound to the same store).
|
|
@@ -204,11 +307,22 @@ export declare const make: <const Name extends string, Payload extends Schema.St
|
|
|
204
307
|
* while the first job still exists is a no-op (returns the existing id).
|
|
205
308
|
*/
|
|
206
309
|
readonly idempotencyKey?: ((payload: Payload extends Schema.Struct.Fields ? Schema.Struct.Type<Payload> : Payload["Type"]) => string) | undefined;
|
|
310
|
+
/**
|
|
311
|
+
* Derive a dedup key (and optional throttle/debounce/replace behavior)
|
|
312
|
+
* from the payload. Unlike `idempotencyKey`, this never changes the job
|
|
313
|
+
* id — see `DedupeInput` for the mode semantics.
|
|
314
|
+
*/
|
|
315
|
+
readonly dedupe?: ((payload: Payload extends Schema.Struct.Fields ? Schema.Struct.Type<Payload> : Payload["Type"]) => DedupeInput) | undefined;
|
|
207
316
|
/**
|
|
208
317
|
* Derive queryable business context from the payload (flat string map, so
|
|
209
318
|
* every driver can index it). Merged with per-enqueue `metadata`.
|
|
210
319
|
*/
|
|
211
320
|
readonly metadata?: ((payload: Payload extends Schema.Struct.Fields ? Schema.Struct.Type<Payload> : Payload["Type"]) => Readonly<Record<string, string>>) | undefined;
|
|
321
|
+
/**
|
|
322
|
+
* When present, a typed handler failure for which this returns false
|
|
323
|
+
* skips the remaining retry budget (see also `Job.unrecoverable`).
|
|
324
|
+
*/
|
|
325
|
+
readonly retryable?: ((error: Error["Type"]) => boolean) | undefined;
|
|
212
326
|
/** The queue this job runs on. Default `"default"`. */
|
|
213
327
|
readonly queue?: string | undefined;
|
|
214
328
|
/**
|
|
@@ -218,5 +332,4 @@ export declare const make: <const Name extends string, Payload extends Schema.St
|
|
|
218
332
|
readonly store?: Context.Key<StoreId, StoreService> | undefined;
|
|
219
333
|
readonly defaults?: JobOptions | undefined;
|
|
220
334
|
}) => Job<Name, Payload extends Schema.Struct.Fields ? Schema.Struct<Payload> : Payload, Success, Error, StoreId>;
|
|
221
|
-
export {};
|
|
222
335
|
//# sourceMappingURL=Job.d.ts.map
|
package/dist/Job.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Job.d.ts","sourceRoot":"","sources":["../src/Job.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA+BG;AACH,OAAO,
|
|
1
|
+
{"version":3,"file":"Job.d.ts","sourceRoot":"","sources":["../src/Job.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA+BG;AACH,OAAO,EAAS,KAAK,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,IAAI,EAAE,KAAK,EAAE,MAAM,EAAa,QAAQ,EAAE,MAAM,EAAE,MAAM,QAAQ,CAAA;AACrH,OAAO,EACL,KAAK,aAAa,EAIlB,KAAK,EACL,KAAK,sBAAsB,EAC3B,gBAAgB,EAChB,KAAK,qBAAqB,EAC1B,KAAK,oBAAoB,EACzB,KAAK,QAAQ,EACb,QAAQ,EACR,KAAK,UAAU,EAEf,SAAS,EACT,WAAW,EAEX,KAAK,OAAO,IAAI,YAAY,EAC5B,aAAa,EACd,MAAM,eAAe,CAAA;AAEtB,OAAO;AACL;;;;;GAKG;AACH,aAAa,EACd,CAAA;AACD,OAAO,EAAE,KAAK,UAAU,EAAE,KAAK,eAAe,EAAE,MAAM,EAAE,MAAM,aAAa,CAAA;AAE3E,QAAA,MAAM,MAAM,EAAG,gBAAyB,CAAA;AAExC;;;;GAIG;AACH,MAAM,WAAW,eAAgB,SAAQ,MAAM,CAAC,GAAG;IACjD,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,CAAA;CACtC;AAED;;;;GAIG;AACH,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,IAAI,EAAE,OAAO,GAAG,aAAa,CAAA;IACtC,QAAQ,CAAC,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAA;IAC9B,6CAA6C;IAC7C,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;CACrC;AAED;;;;GAIG;AACH,MAAM,WAAW,cAAc;IAC7B,kEAAkE;IAClE,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACnC,+CAA+C;IAC/C,QAAQ,CAAC,GAAG,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;CAC1C;AAED;;;;;;;;;;;;GAYG;AACH,MAAM,MAAM,SAAS,GAAG,cAAc,GAAG;IACvC,QAAQ,CAAC,SAAS,CAAC,EAAE,cAAc,GAAG,SAAS,CAAA;IAC/C,QAAQ,CAAC,MAAM,CAAC,EAAE,cAAc,GAAG,SAAS,CAAA;IAC5C,QAAQ,CAAC,SAAS,CAAC,EAAE,cAAc,GAAG,SAAS,CAAA;CAChD,CAAA;AAED;;;;GAIG;AACH,MAAM,WAAW,UAAU;IACzB,qDAAqD;IACrD,QAAQ,CAAC,KAAK,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;IAC3C,mDAAmD;IACnD,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACtC,sEAAsE;IACtE,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACtC,QAAQ,CAAC,OAAO,CAAC,EAAE,YAAY,GAAG,SAAS,CAAA;IAC3C,6DAA6D;IAC7D,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,GAAG,SAAS,CAAA;IACrC;;;OAGG;IACH,QAAQ,CAAC,OAAO,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;CAC9C;AAED;;GAEG;AACH;;;;;;;;;;;;;GAaG;AACH,MAAM,MAAM,WAAW,GAAG,MAAM,GAAG;IACjC,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;IACpB,QAAQ,CAAC,GAAG,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;IACzC,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,GAAG,SAAS,CAAA;IACrC,QAAQ,CAAC,OAAO,CAAC,EAAE,OAAO,GAAG,SAAS,CAAA;CACvC,CAAA;AAmBD;;GAEG;AACH,MAAM,WAAW,cAAe,SAAQ,UAAU;IAChD;;;;OAIG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACnC,uDAAuD;IACvD,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACnC,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,GAAG,SAAS,CAAA;IAChE,mFAAmF;IACnF,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,GAAG,SAAS,CAAA;CAC1C;AAED,UAAU,gBAAgB;IACxB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAA;IACzB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAA;IACzB,QAAQ,CAAC,OAAO,EAAE,aAAa,GAAG,SAAS,CAAA;IAC3C,QAAQ,CAAC,IAAI,EAAE,UAAU,GAAG,SAAS,CAAA;IACrC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,SAAS,CAAA;CACvC;AAED;;;;;;GAMG;AACH,MAAM,WAAW,eAAe,CAAC,YAAY;IAC3C,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IAClC,QAAQ,CAAC,EAAE,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IAChC,QAAQ,CAAC,KAAK,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;IAC3C,QAAQ,CAAC,OAAO,EAAE,YAAY,CAAA;IAC9B,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,GAAG,SAAS,CAAA;IAChE,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACtC,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACtC,QAAQ,CAAC,OAAO,CAAC,EAAE,YAAY,GAAG,SAAS,CAAA;IAC3C,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,GAAG,SAAS,CAAA;IACrC,QAAQ,CAAC,OAAO,CAAC,EAAE,QAAQ,CAAC,KAAK,GAAG,SAAS,CAAA;CAC9C;AAED;;;;;GAKG;AACH,MAAM,WAAW,SAAS,CAAC,CAAC,EAAE,CAAC;IAC7B,QAAQ,CAAC,KAAK,EAAE,QAAQ,CAAA;IACxB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAA;IACnD,oFAAoF;IACpF,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAA;IAC7C,QAAQ,CAAC,YAAY,EAAE,MAAM,GAAG,SAAS,CAAA;CAC1C;AAED;;;;GAIG;AACH,MAAM,WAAW,UAAU,CAAC,CAAC,EAAE,CAAC;IAC9B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,SAAS,CAAA;IACtC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,OAAO,EAAE,WAAW,GAAG,SAAS,GAAG,QAAQ,GAAG,SAAS,GAAG,WAAW,CAAA;IAC9E,oDAAoD;IACpD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAA;CAC9C;AAED;;GAEG;AACH,MAAM,WAAW,GAAG,CAClB,IAAI,SAAS,MAAM,EACnB,OAAO,SAAS,eAAe,EAC/B,OAAO,SAAS,MAAM,CAAC,GAAG,EAC1B,KAAK,SAAS,MAAM,CAAC,GAAG,EACxB,OAAO,GAAG,QAAQ;IAElB,KAAI,CAAC,EAAE,KAAK,GAAG,EAAE,CAAA;IAEjB,QAAQ,CAAC,CAAC,MAAM,CAAC,EAAE,OAAO,MAAM,CAAA;IAChC;;;OAGG;IACH,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAA;IACnB,QAAQ,CAAC,KAAK,EAAE,SAAS,CAAA;IACzB,yCAAyC;IACzC,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,OAAO,EAAE,YAAY,CAAC,CAAA;IAClD,QAAQ,CAAC,aAAa,EAAE,OAAO,CAAA;IAC/B,QAAQ,CAAC,aAAa,EAAE,OAAO,CAAA;IAC/B,QAAQ,CAAC,WAAW,EAAE,KAAK,CAAA;IAC3B,+DAA+D;IAC/D,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAC,WAAW,CAAC,OAAO,CAAC,CAAA;IACvD,iEAAiE;IACjE,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC,WAAW,CACrC,MAAM,CAAC,IAAI,CACT,MAAM,CAAC,WAAW,CAAC,OAAO,CAAC,EAC3B,MAAM,CAAC,WAAW,CAAC,KAAK,CAAC,EACzB,MAAM,CAAC,MAAM,CACd,CACF,CAAA;IACD,QAAQ,CAAC,cAAc,EAAE,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,KAAK,MAAM,CAAC,GAAG,SAAS,CAAA;IAC3E,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,KAAK,WAAW,CAAC,GAAG,SAAS,CAAA;IACxE,QAAQ,CAAC,QAAQ,EAAE,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,KAAK,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC,GAAG,SAAS,CAAA;IAC/F,QAAQ,CAAC,SAAS,EAAE,CAAC,CAAC,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,KAAK,OAAO,CAAC,GAAG,SAAS,CAAA;IACnE,QAAQ,CAAC,QAAQ,EAAE,gBAAgB,CAAA;IAEnC;;;OAGG;IACH,QAAQ,CAAC,OAAO,EAAE,CAChB,OAAO,EAAE,OAAO,CAAC,eAAe,CAAC,EACjC,OAAO,CAAC,EAAE,cAAc,GAAG,SAAS,KACjC,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,OAAO,GAAG,OAAO,CAAC,kBAAkB,CAAC,CAAC,CAAA;IAEvE,4DAA4D;IAC5D,QAAQ,CAAC,IAAI,EAAE,CACb,KAAK,EAAE,KAAK,KACT,MAAM,CAAC,MAAM,CAChB,MAAM,CAAC,MAAM,CAAC,SAAS,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EACxD,KAAK,EACL,OAAO,GAAG,OAAO,CAAC,kBAAkB,CAAC,GAAG,KAAK,CAAC,kBAAkB,CAAC,CAClE,CAAA;IAED,kDAAkD;IAClD,QAAQ,CAAC,QAAQ,EAAE,CACjB,KAAK,EAAE,KAAK,KACT,MAAM,CAAC,MAAM,CAChB,aAAa,CAAC,UAAU,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EACzD,KAAK,EACL,OAAO,GAAG,OAAO,CAAC,kBAAkB,CAAC,GAAG,KAAK,CAAC,kBAAkB,CAAC,CAClE,CAAA;IAED;;;OAGG;IACH,QAAQ,CAAC,WAAW,EAAE,CACpB,KAAK,EAAE,KAAK,EACZ,OAAO,CAAC,EAAE;QAAE,QAAQ,CAAC,YAAY,CAAC,EAAE,QAAQ,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,SAAS,CAAA;KAAE,GAAG,SAAS,KACrF,MAAM,CAAC,MAAM,CAChB,OAAO,CAAC,MAAM,CAAC,EACf,KAAK,CAAC,MAAM,CAAC,EACb,OAAO,GAAG,OAAO,CAAC,kBAAkB,CAAC,GAAG,KAAK,CAAC,kBAAkB,CAAC,CAClE,CAAA;IAED,6CAA6C;IAC7C,QAAQ,CAAC,OAAO,EAAE,CAChB,OAAO,EAAE,OAAO,CAAC,eAAe,CAAC,EACjC,OAAO,CAAC,EAAE,cAAc,GAAG,SAAS,KACjC,MAAM,CAAC,MAAM,CAChB,OAAO,CAAC,MAAM,CAAC,EACf,KAAK,CAAC,MAAM,CAAC,EACX,OAAO,GACP,OAAO,CAAC,kBAAkB,CAAC,GAC3B,OAAO,CAAC,kBAAkB,CAAC,GAC3B,KAAK,CAAC,kBAAkB,CAAC,CAC5B,CAAA;IAED;;;OAGG;IACH,QAAQ,CAAC,KAAK,EAAE,CACd,KAAK,EAAE,KAAK,KACT,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,gBAAgB,GAAG,oBAAoB,EAAE,OAAO,CAAC,CAAA;IAE1E;;;;OAIG;IACH,QAAQ,CAAC,MAAM,EAAE,CACf,KAAK,EAAE,KAAK,KACT,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,gBAAgB,GAAG,sBAAsB,EAAE,OAAO,CAAC,CAAA;IAE5E,6BAA6B;IAC7B,QAAQ,CAAC,OAAO,EAAE,CAChB,KAAK,EAAE,KAAK,KACT,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,gBAAgB,GAAG,qBAAqB,EAAE,OAAO,CAAC,CAAA;IAE3E;;;;;;;;;;;;OAYG;IACH,QAAQ,CAAC,QAAQ,EAAE,CACjB,GAAG,EAAE,MAAM,EACX,OAAO,EAAE,eAAe,CAAC,OAAO,CAAC,eAAe,CAAC,CAAC,KAC/C,MAAM,CAAC,MAAM,CAAC,WAAW,EAAE,KAAK,EAAE,OAAO,GAAG,OAAO,CAAC,kBAAkB,CAAC,CAAC,CAAA;IAE7E,4EAA4E;IAC5E,QAAQ,CAAC,UAAU,EAAE,CACnB,GAAG,EAAE,MAAM,KACR,MAAM,CAAC,MAAM,CAAC,OAAO,EAAE,KAAK,EAAE,OAAO,CAAC,CAAA;IAE3C;;;OAGG;IACH,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC,EAClB,OAAO,EAAE,CACP,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,EACxB,OAAO,EAAE,UAAU,KAChB,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,EACrD,OAAO,CAAC,EAAE,eAAe,GAAG,SAAS,KAClC,KAAK,CAAC,KAAK,CACd,KAAK,EACL,KAAK,EACH,MAAM,GACN,CAAC,GACD,OAAO,CAAC,kBAAkB,CAAC,GAC3B,OAAO,CAAC,kBAAkB,CAAC,GAC3B,KAAK,CAAC,kBAAkB,CAAC,CAC5B,CAAA;CACF;AAED;;GAEG;AACH,MAAM,WAAW,GAAG;IAClB,QAAQ,CAAC,CAAC,MAAM,CAAC,EAAE,OAAO,MAAM,CAAA;IAChC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,KAAK,EAAE,SAAS,CAAA;CAC1B;AAoWD;;;;GAIG;AACH,eAAO,MAAM,IAAI,GACf,KAAK,CAAC,IAAI,SAAS,MAAM,EACzB,OAAO,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,GAAG,eAAe,EACtD,OAAO,SAAS,MAAM,CAAC,GAAG,GAAG,MAAM,CAAC,IAAI,EACxC,KAAK,SAAS,MAAM,CAAC,GAAG,GAAG,MAAM,CAAC,KAAK,EACvC,OAAO,GAAG,QAAQ,QAEZ,IAAI,WACD;IACP,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAA;IACzB,QAAQ,CAAC,OAAO,CAAC,EAAE,OAAO,GAAG,SAAS,CAAA;IACtC,QAAQ,CAAC,KAAK,CAAC,EAAE,KAAK,GAAG,SAAS,CAAA;IAClC;;;OAGG;IACH,QAAQ,CAAC,cAAc,CAAC,EACpB,CAAC,CACD,OAAO,EAAE,OAAO,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,GACvE,OAAO,CAAC,MAAM,CAAC,KAChB,MAAM,CAAC,GACV,SAAS,CAAA;IACb;;;;OAIG;IACH,QAAQ,CAAC,MAAM,CAAC,EACZ,CAAC,CACD,OAAO,EAAE,OAAO,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,GACvE,OAAO,CAAC,MAAM,CAAC,KAChB,WAAW,CAAC,GACf,SAAS,CAAA;IACb;;;OAGG;IACH,QAAQ,CAAC,QAAQ,CAAC,EACd,CAAC,CACD,OAAO,EAAE,OAAO,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,GACvE,OAAO,CAAC,MAAM,CAAC,KAChB,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC,GACpC,SAAS,CAAA;IACb;;;OAGG;IACH,QAAQ,CAAC,SAAS,CAAC,EACf,CAAC,CACD,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,KACjB,OAAO,CAAC,GACX,SAAS,CAAA;IACb,uDAAuD;IACvD,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAA;IACnC;;;OAGG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC,GAAG,CAAC,OAAO,EAAE,YAAY,CAAC,GAAG,SAAS,CAAA;IAC/D,QAAQ,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,SAAS,CAAA;CAC3C,KACA,GAAG,CACJ,IAAI,EACJ,OAAO,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,GAAG,OAAO,EACvE,OAAO,EACP,KAAK,EACL,OAAO,CA4CR,CAAA"}
|