@byok-sdk/cloud-dataplane 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,416 @@
1
+ -- 0002_core_domain.sql — the eleven core-domain port tables.
2
+ --
3
+ -- Frozen at merge, like 0001. Migrations are forward-only (sprint S4A.6): a
4
+ -- correction is a NEW file, never an edit to this one, and the runner
5
+ -- checksums every applied file against its ledger row
6
+ -- (packages/cloud-postgres/src/migrate.ts). 0001 is not touched here, by
7
+ -- construction and by a machine-checked zero diff.
8
+ --
9
+ -- Key design follows docs/researches/s4a-dataplane-design.md §5 and the
10
+ -- tenant-first discipline in docs/architecture/sdk-architecture.md §12.6.2.
11
+ -- The rule 0001 states and this file inherits:
12
+ --
13
+ -- EVERY unique index or constraint on a tenant-owned table starts with
14
+ -- tenant_id.
15
+ --
16
+ -- This file adds no exception to it. The two whitelisted ones both live in
17
+ -- 0001 (device.device_id, pairing_code.code) and both are single-step
18
+ -- pre-tenant resolutions of a cloud-minted credential.
19
+ -- tests/sql/control_plane_invariants.sql turns that rule into an executable
20
+ -- catalog assertion with the same two-entry whitelist.
21
+ --
22
+ -- ---------------------------------------------------------------------------
23
+ -- Two type decisions that apply to every table below
24
+ -- ---------------------------------------------------------------------------
25
+ --
26
+ -- 1. Canonical instants are `text`, not `timestamptz`. Every instant that
27
+ -- crosses a `@byok/core` port is a canonical ISO-8601 UTC string
28
+ -- (YYYY-MM-DDTHH:mm:ss.sssZ) and the contract compares them
29
+ -- lexicographically -- `packages/core/src/time.ts` exists to pin exactly
30
+ -- that and to REJECT anything else rather than normalize it. `text` stores
31
+ -- what the port produced, byte for byte, and `<` / `>=` over it is the same
32
+ -- comparison the in-memory reference performs. `timestamptz` would round
33
+ -- trip through Postgres' own serializer, so every read would owe a proof
34
+ -- that the canonical form survived, and a formatting divergence between the
35
+ -- two compositions would be silent rather than loud. The conformance
36
+ -- suite's `canonical instants` dimension asserts a composition's own output
37
+ -- feeds straight back in; `text` makes that unrepresentably true.
38
+ --
39
+ -- 0001 stores its instants as `timestamptz` on purpose and that stays
40
+ -- correct: those are cloud-local expiry fields with no cross-composition
41
+ -- string contract behind them.
42
+ --
43
+ -- 2. Byte counts are `bigint`. `byteSize`, `expectedBytes`, `hardLimitBytes`,
44
+ -- `releasedBytes` and the whole usage surface are declared `bigint` in
45
+ -- `@byok/core`, and the pool installs an int8 parser so they arrive as JS
46
+ -- `bigint` rather than strings (packages/cloud-postgres/src/pool.ts).
47
+ -- `integer` would put a silent 2^31 ceiling under a storage quota; a string
48
+ -- comparison would answer a different question than `>` without throwing.
49
+ --
50
+ -- Row counts the port types declare as `number` (`rev`, `refCount`, `dropped`,
51
+ -- `capacity`) stay `integer` for the same reason in reverse: they are small by
52
+ -- contract, and decoding them as `bigint` would force a cast at every boundary.
53
+
54
+ -- ---------------------------------------------------------------------------
55
+ -- device_stream.acked_at — the one column 0001 could not foresee
56
+ -- ---------------------------------------------------------------------------
57
+ --
58
+ -- 0001 created `device_stream.acked_seq` ahead of its consumer precisely so
59
+ -- this slice would not have to widen a frozen table. It did not create a
60
+ -- companion instant, and `MailboxCursorState` carries a required `updatedAt`
61
+ -- (packages/core/src/mailbox.ts). There are exactly three ways to supply it:
62
+ -- add the column here, invent a second per-device table that would leave
63
+ -- `device_stream.acked_seq` half-owned, or return the reader's own clock and
64
+ -- call a value that was never written "the instant the cursor moved".
65
+ --
66
+ -- The third is a fabricated fact, and the second contradicts the §5 mapping
67
+ -- that puts the mailbox cursor on `device_stream`. So: one additive, nullable
68
+ -- column. Additive in the forward-only sense -- 0001's bytes are untouched, and
69
+ -- a row written before this migration simply has no recorded ack instant until
70
+ -- its next `advanceCursor`.
71
+ ALTER TABLE device_stream ADD COLUMN acked_at text;
72
+
73
+ -- ---------------------------------------------------------------------------
74
+ -- outbox (core.mailbox) — the hosted mailbox rows
75
+ -- ---------------------------------------------------------------------------
76
+ --
77
+ -- `seq` is allocated from `device_stream.next_seq`, the same allocator
78
+ -- `cloud.sequence` uses, because the daemon's redelivery cursor IS that number
79
+ -- (0001's note on that table). Gaps are legal and expected: the allocator bumps
80
+ -- in its own statement, so a rejected append (a replayed `message_id`) burns a
81
+ -- number rather than reusing one. A reused number would make the cursor
82
+ -- ambiguous, which is the failure the allocator exists to prevent; a gap is
83
+ -- merely a gap.
84
+ --
85
+ -- `state` carries MAILBOX_MESSAGE_STATES. Deliberately no CHECK constraint
86
+ -- restating that list -- the port type is the vocabulary's single authority and
87
+ -- a copy here could drift from it silently (the same call 0001 makes for
88
+ -- `task.status`).
89
+ --
90
+ -- UNIQUE (tenant_id, device_id, message_id) is the producer-supplied
91
+ -- idempotency key from `MailboxAppendInput`. Tenant-first, so it is not an
92
+ -- exception to the rule above: a second append of the same envelope returns the
93
+ -- row that already exists instead of enqueuing a duplicate the device would
94
+ -- execute twice.
95
+ CREATE TABLE outbox (
96
+ tenant_id text NOT NULL,
97
+ device_id text NOT NULL,
98
+ seq bigint NOT NULL,
99
+ message_id text NOT NULL,
100
+ body text NOT NULL,
101
+ body_hash text NOT NULL,
102
+ byte_size bigint NOT NULL,
103
+ state text NOT NULL,
104
+ appended_at text NOT NULL,
105
+ PRIMARY KEY (tenant_id, device_id, seq),
106
+ CONSTRAINT outbox_tenant_device_message_key UNIQUE (tenant_id, device_id, message_id)
107
+ );
108
+
109
+ -- The retention sweep's index: `collectRetired` deletes acked rows older than
110
+ -- one cutoff and marks unacked rows older than another, both scoped to a tenant
111
+ -- and optionally a device. Not unique -- an index that made
112
+ -- (tenant, device, state, instant) unique would forbid two messages appended in
113
+ -- the same millisecond.
114
+ CREATE INDEX outbox_retention_idx
115
+ ON outbox (tenant_id, device_id, state, appended_at);
116
+
117
+ -- ---------------------------------------------------------------------------
118
+ -- tenant_stream (core.board) — the per-tenant board sequence
119
+ -- ---------------------------------------------------------------------------
120
+ --
121
+ -- `board_seq` is monotonic PER TENANT and bumps on every board mutation, which
122
+ -- is what makes `list({ afterSeq })` an incremental feed. A global sequence
123
+ -- would work mechanically and leak every other tenant's write rate through the
124
+ -- gaps, which is why §12.3 pins it per tenant.
125
+ --
126
+ -- Allocation is its own statement (`INSERT ... ON CONFLICT DO UPDATE SET
127
+ -- board_seq = board_seq + 1 RETURNING board_seq`), deliberately NOT a
128
+ -- data-modifying CTE inside the board write it feeds. Two concurrent claims
129
+ -- would then lock this row and `board_item` in an order the planner chooses,
130
+ -- and two sessions choosing opposite orders is a deadlock. Allocating first, in
131
+ -- an autocommitted statement that releases its lock immediately, gives every
132
+ -- writer the same lock order and no deadlock edge. The cost is that a rejected
133
+ -- board write burns a number -- the same trade `outbox.seq` makes, and for the
134
+ -- same reason: monotonic is the contract, gapless is not.
135
+ CREATE TABLE tenant_stream (
136
+ tenant_id text NOT NULL,
137
+ board_seq bigint NOT NULL DEFAULT 0,
138
+ PRIMARY KEY (tenant_id)
139
+ );
140
+
141
+ -- ---------------------------------------------------------------------------
142
+ -- board_item (core.board) — the coordination lifecycle
143
+ -- ---------------------------------------------------------------------------
144
+ --
145
+ -- `status` and `holder_id` are two columns, not one enum, because "where it is"
146
+ -- and "who holds it" change independently (§12.3). Every mutation is a
147
+ -- compare-and-set expressed as an `UPDATE ... WHERE` guard whose zero-row
148
+ -- result is the typed rejection; there is no last-write-wins path, and a loser
149
+ -- re-reads the row it lost to so it can re-decide against real state.
150
+ --
151
+ -- `held_since` is NULL exactly when `holder_id` is NULL: the pair is
152
+ -- `BoardAssignee`, and an unheld item has no assignee rather than an assignee
153
+ -- with an empty holder.
154
+ --
155
+ -- No CHECK on `status` -- BOARD_STATUSES and BOARD_TRANSITIONS in
156
+ -- packages/core/src/board.ts are the vocabulary's authority, and the legality
157
+ -- of a move is a two-value question (from, to) that a column constraint cannot
158
+ -- express anyway.
159
+ CREATE TABLE board_item (
160
+ tenant_id text NOT NULL,
161
+ item_id text NOT NULL,
162
+ channel text NOT NULL,
163
+ title text NOT NULL,
164
+ status text NOT NULL,
165
+ holder_id text,
166
+ held_since text,
167
+ board_seq bigint NOT NULL,
168
+ created_at text NOT NULL,
169
+ updated_at text NOT NULL,
170
+ PRIMARY KEY (tenant_id, item_id)
171
+ );
172
+
173
+ -- The incremental feed's ordering index. Not unique: nothing in the contract
174
+ -- promises two items never share a seq, and a unique index here would turn a
175
+ -- harmless allocation race into a write failure.
176
+ CREATE INDEX board_item_feed_idx ON board_item (tenant_id, board_seq);
177
+
178
+ -- ---------------------------------------------------------------------------
179
+ -- attested_record (core.truth) — two write models, one table
180
+ -- ---------------------------------------------------------------------------
181
+ --
182
+ -- `task.terminal` is first-write-wins and immutable; `profile`/`memory` are
183
+ -- per-key snapshots under an `expectedRev` CAS. Both are keyed
184
+ -- (tenant_id, kind, subject_id), so the terminal path's "first fact is never
185
+ -- overwritten" (§12.6.4) is enforced by the primary key itself:
186
+ -- `INSERT ... ON CONFLICT DO NOTHING` plus an equality re-read, never an upsert
187
+ -- that would restamp history while passing a naive "write twice" check.
188
+ --
189
+ -- `subject_id` is the port's `recordKey` -- the task id for a terminal, the
190
+ -- host's key for a snapshot. Named for what the column IS rather than for the
191
+ -- port's accessor, so `kind` reads as the discriminator it is.
192
+ --
193
+ -- The body is stored as a discriminated pair rather than as JSON: `body_kind`
194
+ -- is 'inline' or 'object', and exactly one of `body_inline` /
195
+ -- `body_object_hash` is populated. `TruthBodyRef` is a two-case union in the
196
+ -- contract, and a JSON blob here would let the store hold shapes the contract
197
+ -- cannot name.
198
+ CREATE TABLE attested_record (
199
+ tenant_id text NOT NULL,
200
+ kind text NOT NULL,
201
+ subject_id text NOT NULL,
202
+ rev integer NOT NULL,
203
+ content_hash text NOT NULL,
204
+ byte_size bigint NOT NULL,
205
+ body_kind text NOT NULL,
206
+ body_inline text,
207
+ body_object_hash text,
208
+ label text,
209
+ request_id text,
210
+ written_at text NOT NULL,
211
+ PRIMARY KEY (tenant_id, kind, subject_id)
212
+ );
213
+
214
+ -- ---------------------------------------------------------------------------
215
+ -- device_presence (core.presence) — lossy, TTL-bounded hints
216
+ -- ---------------------------------------------------------------------------
217
+ --
218
+ -- One row per device, overwritten on every publish: presence is a hint, not a
219
+ -- history, and §12.3 forbids deriving coordination, execution, authorization,
220
+ -- billing or recovery state from it.
221
+ --
222
+ -- `expires_at` is authoritative and expiry means ABSENCE. Reads filter on it
223
+ -- rather than deleting the row, so an expired hint is indistinguishable from
224
+ -- one that was never written -- and a read path stays a read path. The
225
+ -- in-memory reference drops the entry lazily on read; both produce the same
226
+ -- observable answer, and only one of them writes to satisfy a `SELECT`.
227
+ CREATE TABLE device_presence (
228
+ tenant_id text NOT NULL,
229
+ device_id text NOT NULL,
230
+ level text NOT NULL,
231
+ detail text,
232
+ observed_at text NOT NULL,
233
+ expires_at text NOT NULL,
234
+ PRIMARY KEY (tenant_id, device_id)
235
+ );
236
+
237
+ -- ---------------------------------------------------------------------------
238
+ -- activity_tail (core.activity) — a bounded, explicitly lossy tail
239
+ -- ---------------------------------------------------------------------------
240
+ --
241
+ -- The whole tail is one row. `entries` is `jsonb` because the tail is read and
242
+ -- written as a unit and is bounded by `capacity` -- a row-per-entry table would
243
+ -- buy per-entry queries the port does not expose and cost a trim on every
244
+ -- append.
245
+ --
246
+ -- `dropped` is the point of the design: lossiness is IN the data, so a reader
247
+ -- can tell "nothing happened" from "we lost the middle" instead of inferring it
248
+ -- from a gap it has to notice. The append is a single upsert that recomputes
249
+ -- both `entries` and `dropped` from the stored row; a concurrent second append
250
+ -- can lose an entry, which is within contract for a store §12.3 declares lossy
251
+ -- and non-authoritative, and is emphatically not within contract anywhere else
252
+ -- in this file.
253
+ CREATE TABLE activity_tail (
254
+ tenant_id text NOT NULL,
255
+ task_id text NOT NULL,
256
+ entries jsonb NOT NULL,
257
+ dropped integer NOT NULL,
258
+ capacity integer NOT NULL,
259
+ expires_at text NOT NULL,
260
+ PRIMARY KEY (tenant_id, task_id)
261
+ );
262
+
263
+ -- ---------------------------------------------------------------------------
264
+ -- object_manifest (core.objects) — metadata only, no bytes
265
+ -- ---------------------------------------------------------------------------
266
+ --
267
+ -- The manifest is the transaction authority; the object store holds the payload
268
+ -- (§12.7.4, §12.7.8). `pending` exists because Postgres and R2 have no shared
269
+ -- transaction: the row is written before the bytes land, and only `committed`
270
+ -- rows may be referenced. `delete_pending` is the tombstone the GC worker
271
+ -- drives, so a failed object-store delete is retryable instead of leaving usage
272
+ -- silently wrong.
273
+ --
274
+ -- PK (tenant_id, hash) -- never (hash). A global content-address key space
275
+ -- would deduplicate better and turn object existence into a cross-tenant
276
+ -- oracle, which is why §12.7.4 forbids it. Two tenants holding the same bytes
277
+ -- get two rows and are billed twice, on purpose.
278
+ --
279
+ -- `ref_count` is DERIVED, never incremented: every reference mutation
280
+ -- recomputes it as `count(*)` over `object_reference`. An increment can drift
281
+ -- -- a retried `addReference` inflates it and strands the object forever, which
282
+ -- is exactly the bug the "idempotent per (hash, refKind, refId)" contract
283
+ -- exists to prevent. A recomputation cannot.
284
+ CREATE TABLE object_manifest (
285
+ tenant_id text NOT NULL,
286
+ hash text NOT NULL,
287
+ byte_size bigint NOT NULL,
288
+ content_type text NOT NULL,
289
+ state text NOT NULL,
290
+ ref_count integer NOT NULL DEFAULT 0,
291
+ created_at text NOT NULL,
292
+ updated_at text NOT NULL,
293
+ delete_pending_at text,
294
+ PRIMARY KEY (tenant_id, hash)
295
+ );
296
+
297
+ -- The GC sweep's index: list by state, and by tombstone age within
298
+ -- `delete_pending`. Not unique.
299
+ CREATE INDEX object_manifest_state_idx
300
+ ON object_manifest (tenant_id, state, delete_pending_at);
301
+
302
+ -- ---------------------------------------------------------------------------
303
+ -- object_reference (core.objects) — what points at an object
304
+ -- ---------------------------------------------------------------------------
305
+ --
306
+ -- `ref_kind` / `ref_id` are opaque to core. The primary key IS the idempotency
307
+ -- contract: re-adding the same reference conflicts instead of double-counting,
308
+ -- which is what lets `ref_count` be a recomputation rather than a guess.
309
+ CREATE TABLE object_reference (
310
+ tenant_id text NOT NULL,
311
+ hash text NOT NULL,
312
+ ref_kind text NOT NULL,
313
+ ref_id text NOT NULL,
314
+ created_at text NOT NULL,
315
+ PRIMARY KEY (tenant_id, hash, ref_kind, ref_id)
316
+ );
317
+
318
+ -- ---------------------------------------------------------------------------
319
+ -- storage_entitlement (core.quota) — the host-issued numeric entitlement
320
+ -- ---------------------------------------------------------------------------
321
+ --
322
+ -- The SDK does not know what a plan is: no tier name, no price, no currency
323
+ -- (§12.7.6, and `packages/core/src/__tests__/constraints.test.ts` asserts the
324
+ -- same about the port source). What crosses this boundary is numbers plus a
325
+ -- monotonic `version` that is CAS-checked on write, so a delayed control-plane
326
+ -- update cannot resurrect an older plan over a newer one.
327
+ --
328
+ -- `downgrade_grace_until` is a canonical instant compared as a string, per the
329
+ -- type note at the top of this file. It is also the column that decides
330
+ -- `blocked` (507, over limit but inside grace) from `suspended` (423, over
331
+ -- limit and grace has ended), so a comparison that read it differently than the
332
+ -- in-memory reference does would change an HTTP status.
333
+ CREATE TABLE storage_entitlement (
334
+ tenant_id text NOT NULL,
335
+ version bigint NOT NULL,
336
+ hard_limit_bytes bigint NOT NULL,
337
+ max_object_bytes bigint NOT NULL,
338
+ max_inline_bytes bigint NOT NULL,
339
+ mailbox_limit_bytes bigint NOT NULL,
340
+ retention_policy_id text NOT NULL,
341
+ downgrade_grace_until text,
342
+ PRIMARY KEY (tenant_id)
343
+ );
344
+
345
+ -- ---------------------------------------------------------------------------
346
+ -- storage_usage (core.quota) — measured usage
347
+ -- ---------------------------------------------------------------------------
348
+ --
349
+ -- Note what is NOT here: `reserved_bytes`. `TenantStorageUsage.reservedBytes`
350
+ -- is DERIVED as `SUM(expected_bytes)` over this tenant's `reserved` rows in
351
+ -- `storage_reservation`. A stored counter has to be incremented on reserve and
352
+ -- decremented on every one of finalize / abort / expire, and any path that
353
+ -- settles a reservation without the matching decrement leaves a tenant
354
+ -- permanently short of quota it has actually released. Deriving it makes that
355
+ -- class of drift unrepresentable, and it makes settlement a pure state
356
+ -- transition on one row.
357
+ --
358
+ -- One row per entitled tenant, seeded by `writeEntitlement` in the same
359
+ -- statement that accepts the entitlement. The invariant that buys:
360
+ -- a `storage_usage` row exists whenever a `storage_entitlement` row does, so
361
+ -- every accounting write is an `UPDATE` with a guard rather than an upsert that
362
+ -- has to restate the guard twice. A tenant with no entitlement reads as all
363
+ -- zeros without a row being written for it.
364
+ CREATE TABLE storage_usage (
365
+ tenant_id text NOT NULL,
366
+ committed_object_bytes bigint NOT NULL DEFAULT 0,
367
+ committed_inline_bytes bigint NOT NULL DEFAULT 0,
368
+ mailbox_bytes bigint NOT NULL DEFAULT 0,
369
+ object_count bigint NOT NULL DEFAULT 0,
370
+ updated_at text NOT NULL,
371
+ PRIMARY KEY (tenant_id)
372
+ );
373
+
374
+ -- ---------------------------------------------------------------------------
375
+ -- storage_reservation (core.quota) — the no-oversell ledger
376
+ -- ---------------------------------------------------------------------------
377
+ --
378
+ -- Reservation exists because Postgres and the object store have no shared
379
+ -- transaction (§12.7.7): every durable write reserves first, uploads second,
380
+ -- finalizes third, and the invariant
381
+ -- `committed + reserved + expected <= hardLimitBytes` is checked at reserve
382
+ -- time. Two concurrent uploads must not both pass that check, which is why the
383
+ -- admission query runs behind a `FOR UPDATE` on the tenant's entitlement row --
384
+ -- see the header of `packages/cloud-postgres/src/stores/core/quota.ts` for why
385
+ -- a lone statement cannot get this right under READ COMMITTED.
386
+ --
387
+ -- `deduplicated` is stored rather than recomputed because
388
+ -- `StorageFinalizeResult.deduplicated` has to answer the same way when a
389
+ -- finalize is replayed against an already-committed reservation. Recomputing it
390
+ -- then would look at a set that now includes this very row and answer `true`
391
+ -- for the write that actually added the bytes.
392
+ --
393
+ -- `settled_at` is NULL exactly while `state = 'reserved'`.
394
+ CREATE TABLE storage_reservation (
395
+ tenant_id text NOT NULL,
396
+ reservation_id text NOT NULL,
397
+ state text NOT NULL,
398
+ kind text NOT NULL,
399
+ expected_bytes bigint NOT NULL,
400
+ content_hash text NOT NULL,
401
+ content_type text NOT NULL,
402
+ created_at text NOT NULL,
403
+ expires_at text NOT NULL,
404
+ settled_at text,
405
+ deduplicated boolean NOT NULL DEFAULT false,
406
+ PRIMARY KEY (tenant_id, reservation_id)
407
+ );
408
+
409
+ -- Two non-unique indexes, one per sweep the port performs: the live-reservation
410
+ -- sum and the TTL expiry both scan (tenant_id, state), and the per-tenant hash
411
+ -- deduplication check scans (tenant_id, content_hash, state).
412
+ CREATE INDEX storage_reservation_state_idx
413
+ ON storage_reservation (tenant_id, state, expires_at);
414
+
415
+ CREATE INDEX storage_reservation_hash_idx
416
+ ON storage_reservation (tenant_id, content_hash, state);
@@ -0,0 +1,97 @@
1
+ -- 0003_cloud_cleanup.sql — S4B-c retention and cross-system GC authority.
2
+ --
3
+ -- Forward-only and additive. 0001/0002 are immutable and checksum-locked by
4
+ -- the migration runner. This file adds no hash-verification state: ADR-024
5
+ -- remains authoritative, so cleanup may observe only key/existence/size/type.
6
+ --
7
+ -- Every table below is tenant-owned and every unique key starts with
8
+ -- tenant_id. tests/sql/control_plane_invariants.sql enforces that catalog rule.
9
+
10
+ -- Replayed mailbox rows retain the exact expired source sequence. The existing
11
+ -- tenant/device/message-id unique key remains the idempotency authority; this
12
+ -- provenance prevents equal bytes from two different dead letters aliasing the
13
+ -- same operator replay id. The source row remains independently discardable.
14
+ ALTER TABLE outbox
15
+ ADD COLUMN replay_source_seq bigint;
16
+
17
+ -- The host names a retention policy in storage_entitlement. Missing policy is
18
+ -- an operator error and cleanup fails closed; there is no hidden default that
19
+ -- can delete data under a window nobody selected.
20
+ CREATE TABLE tenant_retention_policy (
21
+ tenant_id text NOT NULL,
22
+ policy_id text NOT NULL,
23
+ mailbox_acked_retention_ms bigint NOT NULL,
24
+ mailbox_unacked_retention_ms bigint NOT NULL,
25
+ request_receipt_retention_ms bigint NOT NULL,
26
+ object_orphan_grace_ms bigint NOT NULL,
27
+ updated_at text NOT NULL,
28
+ PRIMARY KEY (tenant_id, policy_id),
29
+ CONSTRAINT tenant_retention_policy_nonnegative CHECK (
30
+ mailbox_acked_retention_ms >= 0
31
+ AND mailbox_unacked_retention_ms >= 0
32
+ AND request_receipt_retention_ms >= 0
33
+ AND object_orphan_grace_ms >= 0
34
+ )
35
+ );
36
+
37
+ -- One durable readback row per host-issued job id. The counters are the
38
+ -- provider-neutral metrics surface; a deployment may export them to its own
39
+ -- telemetry system without making that system the cleanup authority.
40
+ CREATE TABLE cleanup_job (
41
+ tenant_id text NOT NULL,
42
+ job_id text NOT NULL,
43
+ kind text NOT NULL,
44
+ state text NOT NULL,
45
+ started_at text NOT NULL,
46
+ finished_at text,
47
+ mailbox_deleted_count bigint NOT NULL DEFAULT 0,
48
+ mailbox_expired_count bigint NOT NULL DEFAULT 0,
49
+ mailbox_released_bytes bigint NOT NULL DEFAULT 0,
50
+ reservations_expired bigint NOT NULL DEFAULT 0,
51
+ ttl_rows_deleted bigint NOT NULL DEFAULT 0,
52
+ objects_tombstoned bigint NOT NULL DEFAULT 0,
53
+ objects_deleted bigint NOT NULL DEFAULT 0,
54
+ object_released_bytes bigint NOT NULL DEFAULT 0,
55
+ orphan_witnesses_created bigint NOT NULL DEFAULT 0,
56
+ missing_objects bigint NOT NULL DEFAULT 0,
57
+ shape_drift bigint NOT NULL DEFAULT 0,
58
+ invalid_object_keys bigint NOT NULL DEFAULT 0,
59
+ operation_errors bigint NOT NULL DEFAULT 0,
60
+ error_message text,
61
+ PRIMARY KEY (tenant_id, job_id)
62
+ );
63
+
64
+ CREATE INDEX cleanup_job_state_idx
65
+ ON cleanup_job (tenant_id, state, started_at);
66
+
67
+ -- Opaque cursor values only. R2 continuation tokens and manifest hashes are
68
+ -- not interchangeable, so cursor_kind is part of the tenant-first key.
69
+ CREATE TABLE gc_cursor (
70
+ tenant_id text NOT NULL,
71
+ cursor_kind text NOT NULL,
72
+ cursor_value text,
73
+ updated_at text NOT NULL,
74
+ PRIMARY KEY (tenant_id, cursor_kind)
75
+ );
76
+
77
+ -- `gc_accounted_bytes` records the accounting fact at the moment a manifest
78
+ -- becomes a tombstone: committed -> byte_size, pending -> 0. It stays NULL for
79
+ -- any legacy/manual delete_pending row whose origin is unknowable, which makes
80
+ -- the worker fail closed instead of guessing whether usage should be reduced.
81
+ -- It is metadata, not a fifth manifest state.
82
+ ALTER TABLE object_manifest
83
+ ADD COLUMN gc_accounted_bytes bigint;
84
+
85
+ ALTER TABLE object_manifest
86
+ ADD COLUMN gc_accounted_object boolean;
87
+
88
+ ALTER TABLE object_manifest
89
+ ADD CONSTRAINT object_manifest_gc_accounted_nonnegative
90
+ CHECK (gc_accounted_bytes IS NULL OR gc_accounted_bytes >= 0);
91
+
92
+ -- Candidate scan: tenant, state and age; the partial predicate keeps live
93
+ -- referenced objects out of the maintenance index. Reference rows are still
94
+ -- scanned again before tombstoning — ref_count alone is not the safety proof.
95
+ CREATE INDEX object_manifest_gc_candidate_idx
96
+ ON object_manifest (tenant_id, state, updated_at, hash)
97
+ WHERE ref_count = 0;
@@ -0,0 +1,39 @@
1
+ -- 0004_device_proof_truth.sql — S6 device proof key and replay authority.
2
+ --
3
+ -- Forward-only. Existing device_public_key rows are the identity proof key;
4
+ -- this migration projects that already-shipped fact into explicit key id and
5
+ -- epoch columns. Runtime verification never supplies a missing default.
6
+
7
+ ALTER TABLE device
8
+ ADD COLUMN proof_key_id text NOT NULL DEFAULT 'identity',
9
+ ADD COLUMN proof_key_epoch integer NOT NULL DEFAULT 0;
10
+
11
+ ALTER TABLE device
12
+ ALTER COLUMN proof_key_id DROP DEFAULT,
13
+ ALTER COLUMN proof_key_epoch DROP DEFAULT,
14
+ ADD CONSTRAINT device_proof_key_epoch_nonnegative CHECK (proof_key_epoch >= 0);
15
+
16
+ -- Dedicated request-bound result authority. The older
17
+ -- device_request_receipts table is the frozen protocol terminal seam and lacks
18
+ -- device/operation/resource/hash fields; widening or reinterpreting it would
19
+ -- create two meanings for one row shape.
20
+ CREATE TABLE proof_request_receipt (
21
+ tenant_id text NOT NULL,
22
+ device_id text NOT NULL,
23
+ request_id text NOT NULL,
24
+ operation text NOT NULL,
25
+ resource text NOT NULL,
26
+ body_sha256 text NOT NULL,
27
+ body_size bigint NOT NULL,
28
+ response_status integer NOT NULL,
29
+ response_body text NOT NULL,
30
+ recorded_at timestamptz NOT NULL,
31
+ PRIMARY KEY (tenant_id, device_id, request_id),
32
+ CONSTRAINT proof_request_receipt_body_size_nonnegative CHECK (body_size >= 0),
33
+ CONSTRAINT proof_request_receipt_status_range CHECK (
34
+ response_status >= 100 AND response_status <= 599
35
+ )
36
+ );
37
+
38
+ CREATE INDEX proof_request_receipt_recorded_idx
39
+ ON proof_request_receipt (tenant_id, recorded_at);
@@ -0,0 +1,78 @@
1
+ -- 0005_skill_packs.sql — the core.skillPacks port tables.
2
+ --
3
+ -- Forward-only, like every migration before it (sprint S4A.6): a correction is
4
+ -- a NEW file, never an edit to this one, and the runner checksums every applied
5
+ -- file against its ledger row (packages/cloud-postgres/src/migrate.ts). 0001
6
+ -- through 0004 are untouched here.
7
+ --
8
+ -- Phase 2 of the skill-pack-delivery-channel plan promotes `skillPacks` from a
9
+ -- bridge port to a mandatory `CoreStores` member, so every Postgres deployment
10
+ -- creates these tables. They are empty on a deployment that never declares the
11
+ -- `skills.pack` capability — storage presence is not capability advertisement.
12
+ --
13
+ -- The type discipline of 0002 applies:
14
+ --
15
+ -- EVERY unique index or constraint on a tenant-owned table starts with
16
+ -- tenant_id.
17
+ --
18
+ -- Both primary keys below start with tenant_id, and this file adds no exception
19
+ -- to the two-entry whitelist 0001 owns.
20
+ --
21
+ -- ---------------------------------------------------------------------------
22
+ -- skill_pack (core.skillPacks) — one row per published pack
23
+ -- ---------------------------------------------------------------------------
24
+ --
25
+ -- The pack-level fields a `SkillPackManifest` needs to be reconstructed on read:
26
+ -- `version` orders two publications of one name, `description` is the catalogue
27
+ -- text, and `content_hash` is the manifest's whole-pack content address (the
28
+ -- installer re-derives and verifies it; this store never hashes). The schema id
29
+ -- is a constant the store supplies from `SKILL_PACK_MANIFEST_SCHEMA_ID` rather
30
+ -- than a stored column, so it cannot drift per row.
31
+ --
32
+ -- PK (tenant_id, name): a pack name is unique within a tenant and a re-publish
33
+ -- under the same name replaces the pack, which is the store's idempotent upsert.
34
+ -- Never (name) alone — a global name space would turn pack existence into a
35
+ -- cross-tenant oracle, the same call object_manifest makes for content hashes.
36
+ --
37
+ -- No stored timestamp: the manifest carries none, so the store writes none and
38
+ -- reads the database clock nowhere.
39
+ CREATE TABLE skill_pack (
40
+ tenant_id text NOT NULL,
41
+ name text NOT NULL,
42
+ version text NOT NULL,
43
+ description text NOT NULL,
44
+ content_hash text NOT NULL,
45
+ PRIMARY KEY (tenant_id, name)
46
+ );
47
+
48
+ -- ---------------------------------------------------------------------------
49
+ -- skill_pack_file (core.skillPacks) — one row per declared file
50
+ -- ---------------------------------------------------------------------------
51
+ --
52
+ -- Each file the pack declares, with the metadata the manifest addresses it by
53
+ -- (`content_hash`, `byte_size`) and the UTF-8 text itself (`content`). A pack
54
+ -- carries Markdown, YAML and static text assets — never binaries, never
55
+ -- archives — so `content` is `text`, not `bytea`: it stores exactly the string
56
+ -- `SkillPackFileContent.content` is typed as and hands the wire handler back
57
+ -- byte for byte.
58
+ --
59
+ -- `byte_size` is `integer`, not `bigint`. `SkillPackFile.byteSize` is a core
60
+ -- `number` bounded by SKILL_PACK_FILE_MAX_BYTES (256 KiB), unlike the quota and
61
+ -- truth byte fields the contract declares as `bigint` (and that the pool's int8
62
+ -- parser therefore decodes as `BigInt`). `integer` decodes to a JS `number`
63
+ -- with no cast at the boundary — the same call 0002 makes for the small
64
+ -- integer row counts (`rev`, `ref_count`).
65
+ --
66
+ -- PK (tenant_id, pack_name, path): a path is unique within a pack, and the
67
+ -- reference store's publish rejects a duplicate path before it reaches here.
68
+ -- The publish replaces the whole file set in one transaction (DELETE then
69
+ -- INSERT), so a re-publish that drops a file leaves no orphan row behind.
70
+ CREATE TABLE skill_pack_file (
71
+ tenant_id text NOT NULL,
72
+ pack_name text NOT NULL,
73
+ path text NOT NULL,
74
+ content_hash text NOT NULL,
75
+ byte_size integer NOT NULL,
76
+ content text NOT NULL,
77
+ PRIMARY KEY (tenant_id, pack_name, path)
78
+ );
@@ -0,0 +1,18 @@
1
+ -- Device-local logical toolset inventory projected through presence.
2
+ --
3
+ -- Nullable is intentional: NULL means a legacy daemon or an inventory that
4
+ -- has not yet been observed, while [] means a current daemon explicitly
5
+ -- reports no configured toolsets. Executable definitions and credentials
6
+ -- never enter this table.
7
+ ALTER TABLE device_presence
8
+ ADD COLUMN configured_toolsets jsonb;
9
+
10
+ ALTER TABLE device_presence
11
+ ADD CONSTRAINT device_presence_configured_toolsets_shape
12
+ CHECK (
13
+ configured_toolsets IS NULL
14
+ OR (
15
+ jsonb_typeof(configured_toolsets) = 'array'
16
+ AND jsonb_array_length(configured_toolsets) <= 64
17
+ )
18
+ );