@open-mercato/queue 0.6.6-develop.5586.1.c9ed1d68a8 → 0.6.6-develop.5588.1.a8f6c51d1f
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +15 -0
- package/package.json +2 -2
package/AGENTS.md
CHANGED
|
@@ -42,6 +42,21 @@ yarn workspace @open-mercato/queue build
|
|
|
42
42
|
| CPU-bound (calculations, parsing) | 1–2 | Avoid blocking the event loop |
|
|
43
43
|
| Database-heavy (bulk writes) | 3–5 | Balance throughput with connection pool |
|
|
44
44
|
|
|
45
|
+
## Connection Budget (MUST keep workers within the DB pool)
|
|
46
|
+
|
|
47
|
+
Since each worker job runs in its own request container (one `EntityManager`, so
|
|
48
|
+
one pooled DB connection checked out for the job's duration), the peak DB
|
|
49
|
+
connection demand of `worker --all` is **`Σ(per-queue concurrency)`**. That sum
|
|
50
|
+
MUST stay within the database's connection budget, or background jobs starve the
|
|
51
|
+
connections the request/onboarding path needs — the failure mode behind the
|
|
52
|
+
2026-06 onboarding stall.
|
|
53
|
+
|
|
54
|
+
- **Invariant:** `web_pool_max + worker_pool_max + scheduler/overhead ≤ Postgres max_connections` (leave headroom). `*_pool_max` is each process's `DB_POOL_MAX` (default 20).
|
|
55
|
+
- `worker --all` fits `Σconcurrency` to the worker's connection budget at startup and logs the resolved plan (`[worker] DB connection budget: …`). The budget defaults to the resolved `DB_POOL_MAX`; override with `OM_WORKERS_DB_CONNECTION_BUDGET`. Every queue keeps a floor of 1, and no queue exceeds its declared concurrency — so clamping only removes over-subscription, never real throughput.
|
|
56
|
+
- When you change a worker's `concurrency`, add a queue, or raise a pool size, re-check the invariant. A change that multiplies per-job resource usage (e.g. moving to per-job containers) MUST state and verify the new connection ceiling.
|
|
57
|
+
- Give long-running background processes (the worker) a **smaller** `DB_POOL_MAX` than the web process when they share one database, so a worker storm can never consume the web pool's share of `max_connections`.
|
|
58
|
+
- Workers that call external services MUST bound those calls with a timeout (e.g. `VECTOR_EMBEDDING_TIMEOUT_MS` for embeddings) so a dead dependency releases its connection promptly instead of pinning pool capacity.
|
|
59
|
+
|
|
45
60
|
## Adding a New Worker
|
|
46
61
|
|
|
47
62
|
1. Create worker file in `src/modules/<module>/workers/<worker-name>.ts`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@open-mercato/queue",
|
|
3
|
-
"version": "0.6.6-develop.
|
|
3
|
+
"version": "0.6.6-develop.5588.1.a8f6c51d1f",
|
|
4
4
|
"description": "Multi-strategy job queue with local and BullMQ support",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -50,7 +50,7 @@
|
|
|
50
50
|
"access": "public"
|
|
51
51
|
},
|
|
52
52
|
"dependencies": {
|
|
53
|
-
"@open-mercato/shared": "0.6.6-develop.
|
|
53
|
+
"@open-mercato/shared": "0.6.6-develop.5588.1.a8f6c51d1f"
|
|
54
54
|
},
|
|
55
55
|
"repository": {
|
|
56
56
|
"type": "git",
|