vouchington-tooling 0.72.3 → 0.73.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "vouchington-tooling",
3
- "version": "0.72.3",
3
+ "version": "0.73.0",
4
4
  "description": "Vouchington CLI and extractable tooling libraries.",
5
5
  "homepage": "https://github.com/vouchington/vouchington-tooling/tree/main/packages/vouchington-tooling#readme",
6
6
  "bugs": {
@@ -0,0 +1,51 @@
1
+ ---
2
+ name: glide-mq
3
+ description: Design glide-mq queue jobs, ordering lanes, worker concurrency, and cross-job caps when writing or reviewing queue producers, workers, and dispatchers.
4
+ ---
5
+
6
+ # glide-mq queues
7
+
8
+ Read applicable `AGENTS.md` and the repository's local queue adapter when present.
9
+ Local policy overrides these portable conventions.
10
+
11
+ Names below are the glide-mq API: the job option `ordering: { key, concurrency }`, the worker
12
+ option `concurrency`, `lockDuration`, and `job.moveToDelayed(timestamp)`. Read
13
+ [lease patterns](references/lease-patterns.md) for the lease shape.
14
+
15
+ 1. **Every shared-queue job sets `ordering`.** An ungrouped job in a shared queue competes for
16
+ every worker slot, so a flood of it starves other families. Give each job an
17
+ `ordering.key` prefixed with its domain name and an explicit `ordering.concurrency`.
18
+ 2. **`ordering.concurrency` is a global cap; the worker `concurrency` is per process.** The key's
19
+ cap holds across all workers and processes. The worker option only bounds one process's
20
+ active jobs. Never size lane caps to sum to the worker option. Take caps from runtime
21
+ configuration, and give batched families a low cap.
22
+ 3. **Group FIFO ignores `priority`.** Jobs sharing an ordering key run in enqueue order whatever
23
+ their priority. Work that must not wait behind a flood, and dispatchers that rely on
24
+ priority, get their own lane. Put work that outsiders can flood in a lane of its own.
25
+ Ordering keys are the orchestration layer; add none on top.
26
+ 4. **`globalConcurrency` caps the whole queue.** The worker option `globalConcurrency` limits
27
+ every job on the queue together, so on a shared queue it throttles all families at once. Use
28
+ ordering lanes for per-family caps, and use `globalConcurrency` only when one resource bounds
29
+ the whole queue.
30
+ 5. **A job moved to delayed keeps its ordering slot until it completes.** A deferred job still
31
+ counts against `ordering.concurrency`, so deferred jobs can fill a key and block ready work
32
+ on it. A cap that must survive deferral belongs in a token-owned lease, released before
33
+ `moveToDelayed`, not in the ordering key. Never fix starvation by raising the key's
34
+ concurrency: overlap grows and the same number of deferred jobs still fill the slots.
35
+ 6. **Lease mechanics.** Acquire a token-owned lease with `SET key token NX PX ttl` before the
36
+ guarded work. Set the TTL at or above the job's `lockDuration`, renew it while the job runs,
37
+ and release it with compare-and-delete in `finally`. On contention, `moveToDelayed` only this
38
+ job by a short, bounded, jittered delay so it keeps its id, cursor, and dedup. Release the
39
+ lease before any `moveToDelayed` for quota deferral. Reuse the repository's lease helper
40
+ rather than adding a second. See [lease patterns](references/lease-patterns.md).
41
+ 7. **Job count is not a KPI.** Many small jobs with a dispatcher and a sweep are fine. Do not
42
+ merge or remove jobs to cut the count. Judge a design by model calls or spend per item and by
43
+ how simply the jobs share a lifecycle.
44
+
45
+ Dispatchers, paged fan-out, and job-id deduplication follow
46
+ [bounded iteration](../bounded-iteration/SKILL.md). A lease for work that spans an external call
47
+ replaces holding a PostgreSQL transaction or advisory lock open across it; see
48
+ [performance patterns](../postgres-node-performance-tuning/references/performance-patterns.md).
49
+
50
+ Consumer policy owns queue names, lane keys, caps and their configuration source, the lease helper,
51
+ and enforcement configuration.
@@ -0,0 +1,44 @@
1
+ # Lease patterns
2
+
3
+ A lease caps work across jobs and survives deferral, which an `ordering` slot does not: a job
4
+ moved to delayed keeps its slot until it completes.
5
+
6
+ ## Acquire, renew, release
7
+
8
+ ```ts
9
+ const token = randomUUID()
10
+ // SET leaseKey token NX PX ttlMs -> null when another job holds the lease
11
+ if (!(await setIfAbsent(leaseKey, token, ttlMs))) {
12
+ return job.moveToDelayed(Date.now() + jitteredDelayMs) // this job only; never falls through
13
+ }
14
+ try {
15
+ // renew while running, e.g. every ttlMs / 3, only while the stored value equals token
16
+ await doGuardedWork()
17
+ } finally {
18
+ await compareAndDelete(leaseKey, token) // script: delete only when the value equals token
19
+ }
20
+ ```
21
+
22
+ - The TTL is at least the job's `lockDuration`. A shorter lease can expire while the job is
23
+ still active, and a second job then enters the guarded section.
24
+ - Renewal and release are scripts that compare the stored value with the token. A plain `DEL`
25
+ can remove a lease another job has since acquired.
26
+ - The delay on contention is short, bounded, and jittered so deferred jobs do not wake together.
27
+ `moveToDelayed` defers this job only and keeps its id, cursor, and dedup.
28
+
29
+ ## Quota deferral
30
+
31
+ When the work defers itself for a quota or rate, release the lease first, then call
32
+ `moveToDelayed`. `moveToDelayed` never returns, so a release placed after it does not run, and a
33
+ deferred job that keeps the lease blocks the jobs the lease exists to admit.
34
+
35
+ ```ts
36
+ await compareAndDelete(leaseKey, token)
37
+ await job.moveToDelayed(resumeAt)
38
+ ```
39
+
40
+ ## Why not the ordering key
41
+
42
+ Concurrency 1 on a key serializes ready work but lets deferred jobs hold the slot. Raising it to
43
+ 2 lets two guarded sections overlap, and two deferred jobs still fill both slots. A lease
44
+ separates the cap on active work from the queue-order slot.
@@ -30,7 +30,7 @@
30
30
  {
31
31
  "name": "bounded-iteration",
32
32
  "plugin": "vouchington-database",
33
- "pluginVersion": "0.8.0",
33
+ "pluginVersion": "0.9.0",
34
34
  "path": "bounded-iteration/SKILL.md"
35
35
  },
36
36
  {
@@ -72,6 +72,12 @@
72
72
  "pluginVersion": "0.14.2",
73
73
  "path": "github-issue/SKILL.md"
74
74
  },
75
+ {
76
+ "name": "glide-mq",
77
+ "plugin": "vouchington-database",
78
+ "pluginVersion": "0.9.0",
79
+ "path": "glide-mq/SKILL.md"
80
+ },
75
81
  {
76
82
  "name": "nextjs-vitest-test-authoring",
77
83
  "plugin": "vouchington-testing",
@@ -115,19 +121,19 @@
115
121
  {
116
122
  "name": "postgres-node-performance-tuning",
117
123
  "plugin": "vouchington-database",
118
- "pluginVersion": "0.8.0",
124
+ "pluginVersion": "0.9.0",
119
125
  "path": "postgres-node-performance-tuning/SKILL.md"
120
126
  },
121
127
  {
122
128
  "name": "postgres-partitioning-uuid-v7",
123
129
  "plugin": "vouchington-database",
124
- "pluginVersion": "0.8.0",
130
+ "pluginVersion": "0.9.0",
125
131
  "path": "postgres-partitioning-uuid-v7/SKILL.md"
126
132
  },
127
133
  {
128
134
  "name": "postgres-schema-design",
129
135
  "plugin": "vouchington-database",
130
- "pluginVersion": "0.8.0",
136
+ "pluginVersion": "0.9.0",
131
137
  "path": "postgres-schema-design/SKILL.md"
132
138
  },
133
139
  {