vouchington-tooling 0.72.2 → 0.72.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "vouchington-tooling",
3
- "version": "0.72.2",
3
+ "version": "0.72.3",
4
4
  "description": "Vouchington CLI and extractable tooling libraries.",
5
5
  "homepage": "https://github.com/vouchington/vouchington-tooling/tree/main/packages/vouchington-tooling#readme",
6
6
  "bugs": {
@@ -29,7 +29,8 @@ A run's cost is the rows it reads and the work it starts, not the memory it hold
29
29
  per run.
30
30
  5. **Resume without restarting.** Keep a keyset position on the UUIDv7 key or a
31
31
  `<verb>_through_at` time position, fix the sweep's end when it starts, and rescan an overlap
32
- when insert order differs from commit order. Cursor tables follow
32
+ when insert order differs from commit order. Prefer the position in job data; a cursor table is
33
+ for a position that must be transactional with domain rows, and follows
33
34
  [table shapes](../postgres-schema-design/references/table-shapes.md).
34
35
  (`postgres-generated-column-predicates`)
35
36
  6. **No unbounded fan-out.** Use `Promise.all` only over a bounded result, with bounded
@@ -81,6 +81,10 @@ parent id, `WHERE account_id = $1`, for the one account it was started for.
81
81
 
82
82
  ## High-water-mark window with an overlap
83
83
 
84
+ Prefer keeping the position and the sweep's fixed end in job data for a single-writer continuation.
85
+ Use a persisted cursor table only when the position must be transactional with domain rows, as
86
+ here.
87
+
84
88
  An export reads `invoices` added since the last sweep. The cursor row holds the position and the
85
89
  sweep's fixed end, and the job seeds its own row:
86
90
 
@@ -30,7 +30,7 @@
30
30
  {
31
31
  "name": "bounded-iteration",
32
32
  "plugin": "vouchington-database",
33
- "pluginVersion": "0.7.0",
33
+ "pluginVersion": "0.8.0",
34
34
  "path": "bounded-iteration/SKILL.md"
35
35
  },
36
36
  {
@@ -115,19 +115,19 @@
115
115
  {
116
116
  "name": "postgres-node-performance-tuning",
117
117
  "plugin": "vouchington-database",
118
- "pluginVersion": "0.7.0",
118
+ "pluginVersion": "0.8.0",
119
119
  "path": "postgres-node-performance-tuning/SKILL.md"
120
120
  },
121
121
  {
122
122
  "name": "postgres-partitioning-uuid-v7",
123
123
  "plugin": "vouchington-database",
124
- "pluginVersion": "0.7.0",
124
+ "pluginVersion": "0.8.0",
125
125
  "path": "postgres-partitioning-uuid-v7/SKILL.md"
126
126
  },
127
127
  {
128
128
  "name": "postgres-schema-design",
129
129
  "plugin": "vouchington-database",
130
- "pluginVersion": "0.7.0",
130
+ "pluginVersion": "0.8.0",
131
131
  "path": "postgres-schema-design/SKILL.md"
132
132
  },
133
133
  {
@@ -18,7 +18,8 @@ Measure the query plan and the workload shape before changing code.
18
18
  - Keep connection-pool usage bounded.
19
19
  - Validate a query change at representative cardinality. Watch latency, memory, lock time, and
20
20
  connection pressure together.
21
- - Cut round trips: fold dependent reads, batch per-item lookups, and overlap independent reads; see
21
+ - Cut round trips without weakening correctness: fold dependent reads, batch per-item lookups, and
22
+ overlap independent reads; see
22
23
  [round trips](references/performance-patterns.md#round-trips).
23
24
  - Read the plan for hidden per-row work before trusting a change; see
24
25
  [reading plans](references/reading-plans.md).
@@ -3,10 +3,16 @@
3
3
  Use pools and release checked-out clients in `finally`, especially for cursors, streams, and COPY.
4
4
  Route ordinary reads to a replica when its lag is acceptable; route read-after-write, locking, and
5
5
  transaction-consistent reads to the writer. Do not hold a client across unrelated application work.
6
+ Never hold a transaction, connection, or advisory lock across an external call; take a token lease
7
+ in a shared store instead, with a TTL, renewal, and compare-and-delete release. Stop work when
8
+ renewal fails, and guard any write a stale owner could make with the token.
6
9
 
7
10
  For large reads, use keyset pagination, cursors, or streams with cancellation and bounded batches.
8
11
  For writes, prefer set-based batches such as UNNEST or COPY when they preserve validation and error
9
- handling. Measure query plans with representative cardinality before changing an index or query.
12
+ handling. Do large list diffs and dedupes in the worker as a streaming comparison of sorted
13
+ inputs, not as writer-side set operations. Measure query plans with representative cardinality
14
+ before changing an index or query. Adding an index for a measured plan is a default: do not ask
15
+ first and do not make it a spec stop condition.
10
16
 
11
17
  Check query predicates, joins, ordering, and selected columns against index shape. Use EXPLAIN
12
18
  evidence to confirm planner behavior. Add extended statistics only when observed estimates show a
@@ -126,13 +132,23 @@ lateral subquery, rather than reading the parent and then the relation.
126
132
  `Promise.all` over statements on one transaction client is still serial, because one connection
127
133
  runs one statement at a time. Merge them instead: CTEs, `UNNEST`, or one multi-row statement.
128
134
 
129
- Give every per-item lookup helper a batch twin that takes an array of keys and returns a keyed
130
- map, and call the twin from loops, `.map` callbacks, and `Promise.all` fan-outs. `no-await-in-loop`
131
- misses a helper that queries internally and `Promise.all(items.map(helper))`; review those call
132
- sites by hand.
135
+ Write the batch implementation first: it takes an array of keys and returns a keyed map. The
136
+ single-item function is a thin wrapper over it (`addUrl(url)` calls `addUrls([url])`), with no
137
+ parallel per-item SQL path, for lookups and mutations alike. The batch must give a one-item array the
138
+ same atomicity, partial-failure, ordering, duplicate-key, and result-shape semantics as the single
139
+ call; if it doesn't, fix the batch rather than keep a per-item path. Call the batch from loops, `.map`
140
+ callbacks, and `Promise.all` fan-outs. `no-await-in-loop` misses a helper that queries internally
141
+ and `Promise.all(items.map(helper))`; review those call sites by hand.
133
142
 
134
143
  Overlap independent cache and PostgreSQL reads with `Promise.all`, such as a rate-limiter check
135
- alongside the user fetch. Serialize only true dependencies, where one result feeds the next call.
144
+ alongside the user fetch. Serialize only true dependencies, where one result feeds the next call. Suppress the
145
+ sequential-await lint only when each step consumes the previous step's result; a shared
146
+ transaction client or lock ordering is not a data dependency. A lock that must be taken first
147
+ stays a separate, earlier statement; never reorder or parallelize it. Audit every added suppression.
148
+
149
+ Correctness comes before round trips. Before folding a statement away, state the guarantee it
150
+ gave (what it locked, which snapshot it read) and reject the fold if that weakens. An `EXISTS`
151
+ inside a `FOR UPDATE` statement reads a snapshot taken before the lock wait, so it can be stale.
136
152
 
137
153
  Run a symmetric check, such as "is either party blocking the other", as one query with an `OR`
138
154
  or both-direction predicate, not twice with the arguments swapped.
@@ -4,8 +4,11 @@ UUIDv7 ordering supports range bounds and index-friendly time windows. Generate
4
4
  instead of extracting timestamps in predicates; use a generated timestamp only when application
5
5
  semantics require one. Partition on the range key that queries and retention actually constrain.
6
6
 
7
- Create future ranges before writes need them and maintain a default partition only with an explicit
8
- attachment plan. Before attaching a populated range, prove the default partition excludes that range
7
+ Choose a lifecycle. Pre-create future ranges before writes need them, or use on-demand ranges:
8
+ `RANGE (id)` with a DEFAULT partition, split into explicit ranges on demand, and a guarded drop
9
+ that refuses drops inside the retention window or under legal hold. Move or check rows in the
10
+ DEFAULT partition that belong to the range before dropping it. Maintain a default partition
11
+ only with an explicit attachment plan. Before attaching a populated range, prove the default partition excludes that range
9
12
  or move conflicting rows; otherwise attachment can scan and lock the default partition.
10
13
 
11
14
  Every primary or unique constraint on a partitioned table must include its partition key. Verify
@@ -13,6 +13,7 @@ Local policy overrides these portable conventions.
13
13
  - Joined identities use foreign keys; read [relations](references/relations.md) before modeling references.
14
14
  - Split differing shapes, share identical values, and keep history documents as JSON.
15
15
  - Use canonical [table shapes](references/table-shapes.md) for revisions, changes, work, and cursors.
16
+ - Default to finite retention for everything except user content; see [table shapes](references/table-shapes.md#retention).
16
17
  - Reuse parameterized trigger functions instead of copying bodies.
17
18
  - Comment every table, column, and view; document reviewed exceptions explicitly.
18
19
 
@@ -2,7 +2,9 @@
2
2
 
3
3
  An identity that readers join is an FK column or an FK child row. JSON holds structured,
4
4
  schemaless data and history snapshots, including identities recorded as they were at the time.
5
- Do not use UUID arrays, JSON ids, or type/id pairs for live relations. A column never names a
5
+ Do not use UUID arrays, JSON ids, or type/id pairs for live relations. A relation id that
6
+ appears in JSON or in a hashed key also gets an FK column on the same row; the JSON copy may stay,
7
+ and dedup keys on the FK columns, plus any other identity fields, rather than a hash of the document. A column never names a
6
8
  table: use one table per target and generate any per-relation progress enum from the same metadata.
7
9
  Cursor positions and random fencing tokens are intentionally not live entity references; record
8
10
  their reviewed exception instead of inventing a foreign key.
@@ -7,14 +7,14 @@ explicit sweeper-written timestamp; `now()` cannot appear in an immutable genera
7
7
  When state keeps returning to earlier states, record append-only attempts or changes and derive
8
8
  current state from the latest row rather than erasing the previous lifecycle.
9
9
 
10
- | Concept | Shape |
11
- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
12
- | Field-diff history | `<entity>_revisions`: UUIDv7 `id`, entity FK, nullable revision actor FK with `ON DELETE SET NULL`, revision-type enum, `changes jsonb`. |
13
- | Typed lifecycle ledger | `<entity>_changes`: UUIDv7 `id`, entity FK, change-type enum, retained actor identity FK with `ON DELETE RESTRICT`, typed joined payload and JSON for unjoined data. Time comes from `id`. |
14
- | Current state from a log | Latest change by `id DESC`, projected through a shared trigger or read through a view. |
15
- | Per-entity work | `<entity>_<job>_work_items`: entity FK, lease and final-outcome columns, claim index, retention. |
16
- | Job progress | `<job>_cursors`: finite enum key or checked singleton, keyset positions without FKs, optional sweep upper bound. |
17
- | Inbound or protocol events | Append-only `<source>_events` with `occurred_at` recording the event's business time. |
10
+ | Concept | Shape |
11
+ | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
12
+ | Field-diff history | `<entity>_revisions`: UUIDv7 `id`, entity FK, nullable revision actor FK with `ON DELETE SET NULL`, revision-type enum, `changes jsonb`. |
13
+ | Typed lifecycle ledger | `<entity>_changes`: UUIDv7 `id`, entity FK, change-type enum, retained actor identity FK with `ON DELETE RESTRICT`, typed joined payload and JSON for unjoined data. Time comes from `id`. |
14
+ | Current state from a log | Latest change by `id DESC`, projected through a shared trigger or read through a view. |
15
+ | Per-entity work | Default to a queue job id plus a bounded re-enqueue sweep. When justified, `<entity>_<job>_work_items`: entity FK, lease and final-outcome columns, claim index, retention. |
16
+ | Job progress | Prefer the position in job data. Use `<job>_cursors` only when it must be transactional with domain rows: finite enum key or checked singleton, keyset positions without FKs, optional sweep upper bound. |
17
+ | Inbound or protocol events | Append-only `<source>_events` with `occurred_at` recording the event's business time. |
18
18
 
19
19
  Revisions and changes are append-only through a generic `fn_reject_mutation`, parameterized by
20
20
  actor columns that may be erased to NULL. Permit only those erasures; parent-cascade deletion
@@ -25,20 +25,40 @@ A system change has a NULL actor. Retained actors remain historical identities,
25
25
 
26
26
  ## Work leases
27
27
 
28
- The lease lives on a row that exists only for the work. If an invoice outlives a delivery job,
28
+ Default to a queue job whose job id is the durable row id, enqueued after commit, plus a bounded
29
+ sweep that re-enqueues rows with no recorded outcome. The queue already supplies attempts,
30
+ backoff, locking, stall recovery and unique ids. A lease or outbox work-item table needs a stated
31
+ reason: legal or payment records, old values that vanish at commit, or an outcome that must
32
+ commit with domain rows.
33
+
34
+ When a table is justified, fence with one compare-and-set on the outcome or a generation
35
+ (`WHERE completed_at IS NULL` / `AND generation = $n`) instead of a full set of lease columns.
36
+ Idempotent database-only work needs no fence. A unique job id stops duplicate enqueues, not
37
+ duplicate runs.
38
+
39
+ When a full lease is needed, the lease lives on a row that exists only for the work. If an invoice outlives a delivery job,
29
40
  use `invoice_delivery_work_items`, with an invoice FK and `ON DELETE CASCADE`, rather than putting
30
41
  the lease on the invoice. Do not create a shared polymorphic lease table.
31
42
 
32
- Use `lease_token uuid`, `leased_at`, `lease_expires_at`, `attempt_count`, `available_at`, and
43
+ A full lease uses `lease_token uuid`, `leased_at`, `lease_expires_at`, `attempt_count`, `available_at`, and
33
44
  `completed_at` plus timestamps for final outcomes. Add `generation` only when re-enqueue can race
34
45
  a running worker. Each claim or reclaim gets a fresh opaque token. Every renewal, completion,
35
46
  and failure must include `WHERE lease_token = $mine`; zero affected rows means ownership was lost.
36
47
  Keep retries in an optional `_attempts` ledger with `attempt_number` when their lifecycle must
37
48
  survive. One-shot compare-and-set claims and human assignments need their own shapes.
38
49
 
50
+ ## Retention
51
+
52
+ Give every durable row a finite retention window, except user content. Prefer a compact per-entity
53
+ summary over raw event rows when no reader needs the individual events; append-only ledgers still
54
+ get a retention window. Delete a cache's older rows when a new version is written.
55
+ A bounded post-commit enqueue-loss window is acceptable when the stale result heals itself through
56
+ a TTL; no transactional dirty-work table is needed.
57
+
39
58
  ## Cursors and shared functions
40
59
 
41
- A cursor key names the finite thing being swept. A singleton uses
60
+ Prefer keeping the position in job data; a cursor table is for a sweep whose position must commit with
61
+ its domain rows. A cursor key names the finite thing being swept. A singleton uses
42
62
  `is_singleton boolean PRIMARY KEY DEFAULT true CHECK (is_singleton)`; the job seeds its own row
43
63
  with `ON CONFLICT DO NOTHING`. `cursor_<thing>_id` and optional `sweep_upper_bound_<thing>_id` are
44
64
  FK-free UUIDv7 positions; NULL means the start. Per-unit work progress belongs in work items.