@stackbone/cli 0.4.0-alpha.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,20 @@
1
+ -- 0056_trigger_subscriptions_scope.sql — the per-subscription values a trigger
2
+ -- declares in `scopeParams`, stored as one jsonb map keyed by the arg each value
3
+ -- is written into (GitHub's `commit-pushed`: `owner`, `repo`, `sha`).
4
+ --
5
+ -- Why a new column instead of reusing `mailbox` (0022): the mailbox is ONE string
6
+ -- and names ONE arg. GitHub's commit list needs three values, and folding them
7
+ -- into `stackbone/stackbone:next` would leave a column called `mailbox` holding a
8
+ -- repository — a lie every later reader pays for. `mailbox` stays exactly as it
9
+ -- is for the Outlook triggers; `scope` is the general form beside it.
10
+ --
11
+ -- Nullable, and no CHECK constraint. 0022's `chk_outlook_app_only_requires_mailbox`
12
+ -- hardcodes one connector id because SQL cannot read a connector definition, and
13
+ -- it already has to be edited whenever another mailbox-scoped connector ships.
14
+ -- Repeating that shape here would mean a migration per connector. The source of
15
+ -- truth is the descriptor's `scopeParams`, enforced where it can actually be read:
16
+ -- the register endpoint rejects a link missing a required value, and the poll
17
+ -- engine throws rather than sending a request with an unsubstituted path.
18
+
19
+ ALTER TABLE stackbone_platform.trigger_subscriptions
20
+ ADD COLUMN scope jsonb;
@@ -0,0 +1,35 @@
1
+ -- 0057_trigger_subscriptions_fold_mailbox_into_scope.sql — retire `mailbox` (0022)
2
+ -- into `scope` (0056).
3
+ --
4
+ -- Both columns hold the same idea: a value the LINK owns, injected into a list arg
5
+ -- on every poll. `mailbox` holds one string and names one arg (Outlook app-only's
6
+ -- `user-id`); `scope` holds a map and names any number (GitHub's `owner`, `repo`,
7
+ -- `sha`). The second is strictly the general case of the first, so keeping both
8
+ -- meant parallel plumbing in six places and a guess for the next scoped connector
9
+ -- about which one to use.
10
+ --
11
+ -- The COPY comes first and the DROP second, in one transaction, because an Outlook
12
+ -- link armed before this migration has to keep polling the same mailbox after it.
13
+ -- `user-id` is the key, not `mailbox`: the key IS the arg the value is written
14
+ -- into, which is what makes the map generic. It matches the `scopeParams` entry the
15
+ -- Outlook trigger now declares.
16
+ --
17
+ -- `scope IS NULL` guards the copy. Nothing writes both columns today, but a row
18
+ -- that already carries its own scope must not lose it to a stale mailbox.
19
+
20
+ UPDATE stackbone_platform.trigger_subscriptions
21
+ SET scope = jsonb_build_object('user-id', mailbox)
22
+ WHERE mailbox IS NOT NULL
23
+ AND scope IS NULL;
24
+
25
+ -- The CHECK goes with the column. It hardcoded `outlook-app-only` because SQL
26
+ -- cannot read a connector definition, so it had to be edited every time a
27
+ -- mailbox-scoped connector shipped. There is deliberately no `scope` equivalent:
28
+ -- the rule lives where the declaration is readable — the register endpoint rejects
29
+ -- a link missing a required value, and the poll engine throws rather than sending a
30
+ -- request with an unsubstituted path.
31
+ ALTER TABLE stackbone_platform.trigger_subscriptions
32
+ DROP CONSTRAINT IF EXISTS chk_outlook_app_only_requires_mailbox;
33
+
34
+ ALTER TABLE stackbone_platform.trigger_subscriptions
35
+ DROP COLUMN mailbox;
@@ -0,0 +1,9 @@
1
+ -- 0058_trigger_subscriptions_cursor_column.sql — the column the poll cursor
2
+ -- advances on (PLA-888). For a trigger whose descriptor declares `cursorColumn`
3
+ -- (Supabase's row-inserted), the OPERATOR chooses which column orders the rows —
4
+ -- the generated document carries no constraint metadata, so nothing could
5
+ -- honestly detect one. NULL = the descriptor's own default (`id`); every
6
+ -- pre-existing subscription keeps exactly the behaviour it had.
7
+
8
+ ALTER TABLE stackbone_platform.trigger_subscriptions
9
+ ADD COLUMN cursor_column text;
@@ -0,0 +1,26 @@
1
+ -- 0059_trigger_subscriptions_last_error.sql — a link that cannot poll stops
2
+ -- reading as healthy (PLA-1095).
3
+ --
4
+ -- Before this the row had nowhere to put a failure: the whole shape was id,
5
+ -- connector, trigger_id, principal_key, workflow_name, mapping, enabled,
6
+ -- poll_cursor, remote_subscription_id, last_polled_at, created_at, updated_at,
7
+ -- mailbox, mapping_graph, correlation_key, cursor_column. A revoked connection
8
+ -- made every tick fail forever and the only trace was one stdout line per tick.
9
+ --
10
+ -- `last_polled_at` is deliberately NOT touched. It means "the last poll that
11
+ -- CLOSED", and that is the honest answer to "when was this data last really
12
+ -- read". Making it advance on failed attempts too would turn it into "last
13
+ -- attempt" and a broken link would keep reading fresh — the same lie with a
14
+ -- better alibi. The pair is what disambiguates: polled at 07:48, failing since
15
+ -- 07:49.
16
+ --
17
+ -- All three are nullable / defaulted so every existing row is born healthy and
18
+ -- behaves exactly as it did.
19
+
20
+ ALTER TABLE stackbone_platform.trigger_subscriptions
21
+ ADD COLUMN last_error text,
22
+ ADD COLUMN last_error_at timestamptz,
23
+ -- The counter is what separates "one blip on a flaky network" from "broken since
24
+ -- Tuesday". Without it a screen can only ever say "failing", and a single
25
+ -- transient failure would read as loudly as a fortnight of them.
26
+ ADD COLUMN consecutive_failures integer NOT NULL DEFAULT 0;
@@ -0,0 +1,39 @@
1
+ -- 0060_trigger_subscriptions_capped_state.sql — a link that hits the per-tick
2
+ -- delivery ceiling on every tick stops reading as healthy (PLA-1213).
3
+ --
4
+ -- PLA-1210 gave the intake a ceiling: a tick forwards at most
5
+ -- `DEFAULT_MAX_DELIVERIES_PER_TICK` new deliveries and, on overflow, re-saves the
6
+ -- cursor it POLLED WITH so the next tick re-polls the same window. That bounds a
7
+ -- runaway burst, and it terminates, because the delivered prefix comes back as
8
+ -- `duplicate` and spends no ceiling.
9
+ --
10
+ -- What it does not do is say anything. The overflow lives in the tick's returned
11
+ -- outcome and in one stdout line; the ROW is written by `saveCursor`, which
12
+ -- clears `last_error`, zeroes `consecutive_failures` and stamps `last_polled_at`.
13
+ -- So a link whose new-item rate exceeds the ceiling — exactly the self-retriggering
14
+ -- case the ceiling was written for — holds its cursor forever while its row renders
15
+ -- as a link that just polled cleanly. The runaway went from loud to silent.
16
+ --
17
+ -- The trio mirrors 0059's failure trio deliberately, and for the same reasons:
18
+ --
19
+ -- * `last_capped_at` pairs with `last_polled_at` the way `last_error_at` does.
20
+ -- Polled at 07:48, capping since 07:49 is the sentence an operator needs.
21
+ -- * `last_deferred_count` is how many items the tick left behind. One number
22
+ -- separates "203 arrived, 3 waited" from "5000 arrived and this will run for
23
+ -- 25 ticks", and those want different answers.
24
+ -- * `consecutive_capped_ticks` separates an arming backfill — which caps for
25
+ -- one or two ticks and is the ceiling working as designed — from a link that
26
+ -- has been capping since Tuesday. Without it a screen can only say "capping",
27
+ -- and the backfill would read as loudly as the runaway.
28
+ --
29
+ -- Cleared rather than kept on an uncapped tick, same bargain as `last_error`: the
30
+ -- row means CURRENT state, and a link that has stopped capping must not read as
31
+ -- capping. The history of laps lives in `trigger_deliveries`.
32
+ --
33
+ -- All three are nullable / defaulted, so every existing row is born uncapped and
34
+ -- behaves exactly as it did.
35
+
36
+ ALTER TABLE stackbone_platform.trigger_subscriptions
37
+ ADD COLUMN last_capped_at timestamptz,
38
+ ADD COLUMN last_deferred_count integer,
39
+ ADD COLUMN consecutive_capped_ticks integer NOT NULL DEFAULT 0;
@@ -0,0 +1,35 @@
1
+ -- 0061_trigger_subscriptions_cursor_held.sql — `poll_cursor IS NULL` stops
2
+ -- meaning two different things (PLA-1215).
3
+ --
4
+ -- The overload. `poll_cursor = NULL` used to mean exactly one thing, "this link
5
+ -- has never polled", and one write depends on that: `seedCursorIfUnset` arms a
6
+ -- link "from now" with
7
+ --
8
+ -- poll_cursor = COALESCE(poll_cursor, $seed)
9
+ --
10
+ -- so a link that HAS polled resumes where it left off instead of replaying, and a
11
+ -- link whose scope changed (PLA-1144 clears the cursor deliberately) gets a fresh
12
+ -- bookmark. Both correct.
13
+ --
14
+ -- PLA-1210's ceiling added a third case. When a provider rejects a stale opaque
15
+ -- cursor the tick re-polls with `null`, and if THAT window is over the ceiling the
16
+ -- tick holds the cursor it polled with — which is `null`. So a link with a known
17
+ -- multi-tick backlog now sits at NULL, and the COALESCE above cannot tell it from
18
+ -- never-polled: any patch that arms or re-arms the link mid-drain seeds it "from
19
+ -- now" and every deferred item is dropped, silently. That is PLA-1005 through a
20
+ -- different door.
21
+ --
22
+ -- Why a column of its own rather than reading PLA-1213's `consecutive_capped_ticks`,
23
+ -- which is NULL-vs-held information too: the two want opposite clearing rules.
24
+ -- The capped trio is observability ABOUT THE LINK and 0060 clears it when a link is
25
+ -- armed, because a link being armed has not capped yet. This is a fact ABOUT THE
26
+ -- CURSOR and must survive arming — otherwise arming twice reads a cleared counter
27
+ -- the second time and seeds over the backlog anyway, which is the bug with an extra
28
+ -- step.
29
+ --
30
+ -- Written by the same `saveCursor` statement that saves the cursor, so the flag and
31
+ -- the value it describes can never disagree. `false` for every existing row, which
32
+ -- is what they all are: nothing has held a cursor before this ships.
33
+
34
+ ALTER TABLE stackbone_platform.trigger_subscriptions
35
+ ADD COLUMN poll_cursor_held boolean NOT NULL DEFAULT false;
@@ -0,0 +1,40 @@
1
+ -- 0062_trigger_deliveries_retention.sql — the delivery journal gets a retention
2
+ -- sweep, and the two things an hourly DELETE by age needs on this table (PLA-1209).
3
+ --
4
+ -- `trigger_deliveries` is the at-least-once dedup journal (0021): one row per
5
+ -- `(subscription_id, provider_item_id)`, claimed with an INSERT that collapses on
6
+ -- the unique pair. Nothing pruned it by age. That was fine while every trigger was
7
+ -- creation-shaped — one row per message, per file, per commit — so the table
8
+ -- settled. PLA-878's `dedup.versionPath` (Linear `issue-updated`) keys a row per
9
+ -- CHANGE to an object, and the table stopped being finite on the very index the
10
+ -- dedup reads.
11
+ --
12
+ -- The sweep itself lives in the runtime (`delivery-journal-retention-job.ts`),
13
+ -- on the log-retention job's cadence pattern. This migration is only what the
14
+ -- sweep needs from the schema:
15
+ --
16
+ -- * An index on `created_at`. The age pass deletes `WHERE created_at < cutoff`
17
+ -- in bounded batches, hourly, and without an index every one of those
18
+ -- batches is a sequential scan of a table whose whole problem is its size.
19
+ -- The budget pass reads the same index: it takes ONE subscription's oldest
20
+ -- rows first (`ORDER BY created_at`), and the subscription over budget is by
21
+ -- definition the one holding most of the old rows, so the ascending scan
22
+ -- finds them at the front. No `(subscription_id, created_at)` index on top:
23
+ -- a third index is paid for on every claim, and the dedup index's
24
+ -- `subscription_id` prefix already answers the per-link count.
25
+ --
26
+ -- * Per-table autovacuum, for the reason 0051 gives the log table: the cluster
27
+ -- default waits for dead tuples to reach 20% of the table, and this is now a
28
+ -- table that is bulk-deleted on a schedule. Left on the defaults, an hourly
29
+ -- sweep leaves its dead rows in place for hours under the unique index every
30
+ -- claim probes. The threshold is lower than the log table's because the rows
31
+ -- are far fewer: a journal that only ever held a hundred rows should still
32
+ -- vacuum after its first sweep.
33
+
34
+ CREATE INDEX IF NOT EXISTS trigger_deliveries_created_at_idx
35
+ ON stackbone_platform.trigger_deliveries (created_at);
36
+
37
+ ALTER TABLE stackbone_platform.trigger_deliveries SET (
38
+ autovacuum_vacuum_scale_factor = 0.02,
39
+ autovacuum_vacuum_threshold = 500
40
+ );
@@ -0,0 +1,42 @@
1
+ -- A SEVENTH ENDING: `not_evaluable` (PLA-1228).
2
+ --
3
+ -- A measured run answers its connector calls with stand-ins, and only Outlook has
4
+ -- one. Every other connector is REFUSED rather than passed through to the real
5
+ -- provider — which is the safe choice and stays — but the refusal arrives as an
6
+ -- ordinary failure, so the case ends `errored` and an operator cannot tell "this
7
+ -- workflow is not measurable on this box" from "this workflow is broken".
8
+ --
9
+ -- WHY NOT REUSE `blocked_by_guardrail`. It is the obvious candidate and it is
10
+ -- wrong. That value does not mean "a rule said no"; it means NOTHING RAN. It is
11
+ -- written on one path only — the start path's catch, which throws before anything
12
+ -- is enqueued (`workflow-case-runner.ts`, and `eval-case-runner.ts` says the same
13
+ -- in different words). A connector refusal happens MID-RUN: the workflow started,
14
+ -- and the steps before the refused call may have had effects. Widening the value
15
+ -- would silently falsify it for anything already reading it as "no side effects".
16
+ --
17
+ -- So the vocabulary grows, which the 0042 comment above deliberately made
18
+ -- expensive. This is the case that pays for it: `escalated` and `asked_user` were
19
+ -- kept apart because folding them made the pause criteria unmeasurable, and the
20
+ -- same test applies here — an operator's next action after "not measurable here"
21
+ -- (register a stand-in, or measure something else) is not the action after
22
+ -- "errored" (read the trace, fix the workflow).
23
+ --
24
+ -- NULLABLE and never backfilled, for the same reason as 0042: the table is
25
+ -- append-only and every row already in it belongs to a sealed run. A case that
26
+ -- ended `errored` because of a missing stand-in stays `errored`, because that is
27
+ -- what was recorded and there is no honest way to tell those rows apart now.
28
+
29
+ ALTER TABLE stackbone_platform.eval_case_results
30
+ DROP CONSTRAINT IF EXISTS eval_case_results_ended_check;
31
+
32
+ ALTER TABLE stackbone_platform.eval_case_results
33
+ ADD CONSTRAINT eval_case_results_ended_check
34
+ CHECK (ended IS NULL OR ended IN (
35
+ 'completed',
36
+ 'escalated',
37
+ 'asked_user',
38
+ 'blocked_by_guardrail',
39
+ 'not_evaluable',
40
+ 'errored',
41
+ 'cancelled'
42
+ ));
@@ -0,0 +1,90 @@
1
+ -- 0064_workflow_file_outputs.sql — where a workflow's declared file outputs go,
2
+ -- and where a document waits when they cannot (PLA-1092 slices 3 and 4).
3
+ --
4
+ -- Before this there was no table at all. A workflow declares
5
+ -- `export const fileOutputs = ['prd']` and the boot manifest carries the names
6
+ -- (slice 1); `drive.file-write` knows how to file a document into a Drive folder
7
+ -- (slice 2); and nothing on the box connected the two. The creator's code named
8
+ -- a folder id, a connector and a principal in the workflow body, and moving the
9
+ -- output meant a redeploy.
10
+ --
11
+ -- Two tables, and they are deliberately not one.
12
+ --
13
+ -- `workflow_file_output_bindings` is the OPERATOR's decision: this output of this
14
+ -- workflow lands THERE, through THAT connection. One row per (workflow, output),
15
+ -- which is what the UNIQUE says — a second binding for the same output is an
16
+ -- edit of the first, never a sibling.
17
+ --
18
+ -- `destination jsonb`, not `folder_id text`. Only the writer knows what its
19
+ -- provider calls a destination — a Drive folder id today, a Notion page or an
20
+ -- S3 prefix tomorrow — and `FileWriteRequest.destination` is documented
21
+ -- opaque for exactly that reason. Naming the column after one provider's noun
22
+ -- is the `mailbox` column 0056 argued against, one table over. The one rule
23
+ -- the box holds it to lives in code (`file-output-writer.ts`): the object
24
+ -- carries an `id` string, and that string is what the writer receives.
25
+ --
26
+ -- No `installation_id`. Every read here is `WHERE id = $1` or by the
27
+ -- (workflow, output) pair: one box is one installation, and the box's own
28
+ -- database is the scope — the same reason `trigger_subscriptions` carries none.
29
+ --
30
+ -- `parked_file_outputs` is the DOCUMENT that could not be filed. By the time a
31
+ -- run reaches its output the expensive work is done — the model has run, the
32
+ -- bytes are staged — so a missing binding, or a provider that said no, must not
33
+ -- throw the document away. The row keeps the staged key and everything needed
34
+ -- to file it later; a drain (slice 5, not here) reads it back.
35
+ --
36
+ -- `reason` is `'unbound' | 'write_failed'`, text with no CHECK. 0056's rule:
37
+ -- SQL cannot read a connector definition, and a constraint that needs one
38
+ -- lives in code. `unbound` says nobody decided where this goes yet;
39
+ -- `write_failed` says somebody did and the provider refused.
40
+ --
41
+ -- `last_error` / `last_error_at` / `attempts` are lifted from 0059 for its own
42
+ -- reason: a counter is what separates one blip from broken-since-Tuesday, and
43
+ -- a screen without it can only ever say "failing". `attempts` is incremented
44
+ -- in SQL by the store, never read-then-written, so two drains racing on one
45
+ -- row cannot both write 1.
46
+ --
47
+ -- `filed_file_id` is the row's terminal state: `NULL` while it waits, the
48
+ -- provider's id once a later attempt landed it. The row is KEPT rather than
49
+ -- deleted on success, so an operator can still answer "where did that PRD
50
+ -- from Tuesday end up" after the fact.
51
+ --
52
+ -- Indexes: one partial index on the parked table. The only list read is "what
53
+ -- is still waiting", newest first — `WHERE filed_file_id IS NULL ORDER BY
54
+ -- created_at DESC` — and a filed row is never listed again, so indexing only
55
+ -- the unfiled ones keeps the index the size of the backlog rather than of the
56
+ -- history. The bindings table needs nothing beyond its UNIQUE: the write path
57
+ -- reads exactly by that pair, and the id reads hit the primary key.
58
+
59
+ CREATE TABLE stackbone_platform.workflow_file_output_bindings (
60
+ id uuid PRIMARY KEY,
61
+ workflow_name text NOT NULL,
62
+ output_name text NOT NULL,
63
+ connector text NOT NULL,
64
+ principal_key text NOT NULL,
65
+ destination jsonb NOT NULL,
66
+ created_at timestamptz NOT NULL DEFAULT now(),
67
+ updated_at timestamptz NOT NULL DEFAULT now(),
68
+ UNIQUE (workflow_name, output_name)
69
+ );
70
+
71
+ CREATE TABLE stackbone_platform.parked_file_outputs (
72
+ id uuid PRIMARY KEY,
73
+ workflow_name text NOT NULL,
74
+ output_name text NOT NULL,
75
+ run_id text,
76
+ file_name text NOT NULL,
77
+ storage_key text NOT NULL,
78
+ content_type text NOT NULL,
79
+ reason text NOT NULL,
80
+ last_error text,
81
+ last_error_at timestamptz,
82
+ attempts integer NOT NULL DEFAULT 0,
83
+ filed_file_id text,
84
+ created_at timestamptz NOT NULL DEFAULT now(),
85
+ updated_at timestamptz NOT NULL DEFAULT now()
86
+ );
87
+
88
+ CREATE INDEX parked_file_outputs_waiting_idx
89
+ ON stackbone_platform.parked_file_outputs (created_at DESC)
90
+ WHERE filed_file_id IS NULL;
@@ -0,0 +1,36 @@
1
+ -- 0065_parked_file_outputs_filing_claim.sql — make "this row is being filed"
2
+ -- a fact Postgres holds, not one node believed (PLA-1092 slice 5).
3
+ --
4
+ -- The drain read the row, saw `filed_file_id IS NULL`, spent a few hundred
5
+ -- milliseconds at the provider, and only then wrote the id back with an
6
+ -- unconditional UPDATE. Nothing claimed the row in between, so two drains 200 ms
7
+ -- apart both passed the check, both filed, and both marked: two documents in the
8
+ -- operator's folder, one `filed_file_id` overwriting the other, and the orphaned
9
+ -- id recorded nowhere. The comment above the guard named "a double click" as the
10
+ -- exact case it prevented, which is what a read-then-act guard cannot do.
11
+ --
12
+ -- `filing_claimed_at` is that claim. One UPDATE takes it — `WHERE filed_file_id
13
+ -- IS NULL AND (filing_claimed_at IS NULL OR filing_claimed_at < <stale>)` — and
14
+ -- its row count is the answer: the loser never reaches the provider.
15
+ --
16
+ -- WHY A COLUMN AND NOT `filed_file_id` ITSELF. The claim has to be taken BEFORE
17
+ -- the provider call, and before the provider call there is no file id to write.
18
+ -- Writing a sentinel into the column that means "landed here" would make the
19
+ -- terminal state unreadable, and `listParked`'s partial index is built on it.
20
+ --
21
+ -- WHY IT EXPIRES. A box that dies between the claim and the release would
22
+ -- otherwise leave the row claimed forever and the button dead forever, with no
23
+ -- way out but SQL. The staleness window is the writer's
24
+ -- (`FILING_CLAIM_TTL_MS`), not a CHECK here: the number is a judgement about how
25
+ -- long a provider round-trip can take, and that belongs beside the code that
26
+ -- makes it. Retaking a stale claim can still file a second copy — a claim is not
27
+ -- an idempotency record — but it bounds that to a box that crashed mid-write
28
+ -- rather than to a double click.
29
+ --
30
+ -- Nullable, no default, no backfill: an existing parked row is by definition not
31
+ -- being filed right now, and NULL says exactly that. The partial index from 0064
32
+ -- is untouched — every read that uses it still filters on `filed_file_id IS
33
+ -- NULL`, and the claim is only ever read by id.
34
+
35
+ ALTER TABLE stackbone_platform.parked_file_outputs
36
+ ADD COLUMN filing_claimed_at timestamptz;
@@ -0,0 +1,70 @@
1
+ -- 0066_filed_file_outputs.sql — a file output that SUCCEEDED leaves a record, so
2
+ -- the retry after a lost response does not file a second copy (PLA-1247 slice 1).
3
+ --
4
+ -- 0064 recorded only the documents that could NOT be filed. The success path was
5
+ -- write-only by decision: `file-output-writer.ts` said "nothing is recorded here:
6
+ -- the run's own timeline already says a document went out". That holds exactly as
7
+ -- long as the response arrives.
8
+ --
9
+ -- When it does not — the box restarts between the writer returning and `c.json`,
10
+ -- a proxy times out, a connection resets — the SDK's `fetch` throws, the step
11
+ -- fails, and the durable step retries. The retry stages fresh bytes under a fresh
12
+ -- uuid and files again, and `drive.files.create` is not idempotent: two identical
13
+ -- documents in the customer's folder, with nothing on either side able to say the
14
+ -- first one landed. The create-then-fill order 0064's writers use exists to stop
15
+ -- a crash leaving a confusing artefact; a lost response reintroduced it through
16
+ -- the front door.
17
+ --
18
+ -- THE KEY IS THE PRIMARY KEY. `(run_id, workflow_name, output_name)` is the
19
+ -- identity of "this run already filed this output", so it is the row's identity
20
+ -- too — there is no surrogate uuid, because no second row for that triple is ever
21
+ -- meaningful and a surrogate id would be a second way to name the same fact. A
22
+ -- repeat write reads it and answers the recorded `file_id` without touching the
23
+ -- provider.
24
+ --
25
+ -- WHY NOT `parked_file_outputs`. That table holds one row per FAILED ATTEMPT and
26
+ -- carries no unique key on the triple, deliberately: a run that parks `unbound`
27
+ -- and is then retried parks again, and both rows are real. Adding the unique key
28
+ -- an idempotency record needs would refuse the second park — turning a lost
29
+ -- document into a lost document AND a failed insert. Nor is 0065's
30
+ -- `filing_claimed_at` this record: a claim has a TTL so a dead box cannot hold a
31
+ -- row forever, and a retaken stale claim can still file a second copy. A claim
32
+ -- bounds a duplicate; only a recorded file id removes one.
33
+ --
34
+ -- `run_id` IS `NOT NULL`, and that is the whole statement of what happens without
35
+ -- one. `parked_file_outputs.run_id` is nullable because a parked document is worth
36
+ -- keeping whether or not a run claimed it; an idempotency key with a null
37
+ -- component is not a key. A write carrying no run id is not deduped at all — see
38
+ -- `file-output-writer.ts`, which decides that in code rather than discovering it
39
+ -- here as a constraint violation. Collapsing every run-less write of one output
40
+ -- into a single row would short-circuit the SECOND real document and never file
41
+ -- it, which is worse than the duplicate this table exists to prevent.
42
+ --
43
+ -- `file_name` and `storage_key` ride along for the audit question 0064 opened and
44
+ -- answered only for the failed half: "where did Tuesday's PRD end up". Without
45
+ -- them a filed row names a provider id and nothing a human recognises.
46
+ --
47
+ -- RETENTION: none, on purpose, and this is the paragraph 0062 earned. The
48
+ -- delivery journal needed a sweep because its key space grew with provider
49
+ -- CHATTER — one row per observed change, most of which produce no work — with no
50
+ -- ceiling on an index every claim probes. This table grows one row per document
51
+ -- that actually exists in a customer's folder: the row count IS the folder count,
52
+ -- there is no amplification, and a workspace with enough rows here to notice has
53
+ -- a filing problem long before it has a table problem. Every read is a point
54
+ -- lookup on the primary key, so size costs nothing at read time, and the row is
55
+ -- the only surviving answer to where a filed document went — deleting it by age
56
+ -- would make the SUCCESSFUL half less answerable than the parked half 0064 keeps
57
+ -- forever. If a workspace ever files at machine cadence, the age + budget sweep in
58
+ -- `delivery-journal-retention-job.ts` is the shape to copy, and it would want an
59
+ -- index on `filed_at` that is deliberately not paid for here.
60
+
61
+ CREATE TABLE stackbone_platform.filed_file_outputs (
62
+ run_id text NOT NULL,
63
+ workflow_name text NOT NULL,
64
+ output_name text NOT NULL,
65
+ file_id text NOT NULL,
66
+ file_name text NOT NULL,
67
+ storage_key text NOT NULL,
68
+ filed_at timestamptz NOT NULL DEFAULT now(),
69
+ PRIMARY KEY (run_id, workflow_name, output_name)
70
+ );
@@ -0,0 +1,72 @@
1
+ -- 0067_parked_outputs_carry_a_kind.sql — what waits on the parked list can now
2
+ -- be a call as well as a document (PLA-1253).
3
+ --
4
+ -- 0064 built `parked_file_outputs` around one thing: a document the run already
5
+ -- produced, kept by its staged key so a missing binding could not throw it away.
6
+ -- An action parks for the same reason and keeps something different. It has no
7
+ -- bytes. What has to survive is the ARGUMENT OBJECT the run composed, because
8
+ -- recomputing it means re-running the whole workflow — the model call included.
9
+ --
10
+ -- ONE TABLE, NOT TWO. The operator's question is "what has not gone out yet",
11
+ -- and it is one question. A second table would mean two lists to merge, two
12
+ -- claims, two partial indexes and two drains, for two rows that differ in three
13
+ -- columns. The screen renders each as its own kind; the storage does not have to.
14
+ --
15
+ -- The table KEEPS ITS NAME for the reason 0064's siblings kept theirs (PLA-1250):
16
+ -- renaming a table is a migration plus a version bump for a gain nobody outside
17
+ -- this repo sees, and the store carries a comment explaining the mismatch.
18
+ --
19
+ -- `kind` DEFAULTS to 'file' and every existing row is one. No CHECK, following
20
+ -- 0064's rule for `reason`: the set of kinds is declared in the manifest a box
21
+ -- boots with, and SQL cannot read it.
22
+ --
23
+ -- THE THREE FILE COLUMNS LOSE THEIR NOT NULL. `file_name`, `storage_key` and
24
+ -- `content_type` describe a document, and an action has none of them. Dropping
25
+ -- the constraint is what makes one table honest rather than one table with three
26
+ -- empty strings pretending to be a filename. The code holds what SQL now cannot:
27
+ -- the writer only ever reads them on a `kind = 'file'` row.
28
+ --
29
+ -- `settled_at` IS THE NEW TERMINAL STATE, AND IT REPLACES A PREDICATE RATHER
30
+ -- THAN ADDING ONE. 0064 made `filed_file_id` the terminal state — NULL while it
31
+ -- waits, the provider's id once it landed — and 0065 argued in writing why a
32
+ -- sentinel must never be written into it: "writing a sentinel into the column
33
+ -- that means 'landed here' would make the terminal state unreadable, and
34
+ -- `listParked`'s partial index is built on it." That argument holds for an
35
+ -- action, which has no file id to write and may have no id at all: an operation
36
+ -- can answer a list, a boolean, or nothing.
37
+ --
38
+ -- So the two facts are separated. `filed_file_id` goes back to being ONLY the
39
+ -- file's payload; `settled_at` is "this row is done", for both kinds, and it is
40
+ -- the single predicate the claim, the update and the list now share. The
41
+ -- alternative — `filed_file_id IS NULL AND fired_at IS NULL` — encodes one
42
+ -- concept in two columns and makes every future reader hold both.
43
+ --
44
+ -- The BACKFILL is `updated_at`, and it is approximate on purpose. A row filed
45
+ -- before this migration has no record of the instant it landed; its last write
46
+ -- is the closest true thing, and nothing reads `settled_at` as a moment — every
47
+ -- read is `IS NULL`. Saying so here is cheaper than a column that looks precise
48
+ -- and is not.
49
+
50
+ ALTER TABLE stackbone_platform.parked_file_outputs
51
+ ADD COLUMN kind text NOT NULL DEFAULT 'file',
52
+ ADD COLUMN operation text,
53
+ ADD COLUMN args jsonb,
54
+ ADD COLUMN settled_at timestamptz;
55
+
56
+ ALTER TABLE stackbone_platform.parked_file_outputs
57
+ ALTER COLUMN file_name DROP NOT NULL,
58
+ ALTER COLUMN storage_key DROP NOT NULL,
59
+ ALTER COLUMN content_type DROP NOT NULL;
60
+
61
+ UPDATE stackbone_platform.parked_file_outputs
62
+ SET settled_at = updated_at
63
+ WHERE filed_file_id IS NOT NULL;
64
+
65
+ -- The waiting list, on the predicate both kinds share. Dropped and recreated
66
+ -- rather than left beside a new one: two partial indexes over the same rows on
67
+ -- two spellings of "still waiting" is how the two spellings drift apart.
68
+ DROP INDEX IF EXISTS stackbone_platform.parked_file_outputs_waiting_idx;
69
+
70
+ CREATE INDEX parked_file_outputs_waiting_idx
71
+ ON stackbone_platform.parked_file_outputs (created_at DESC)
72
+ WHERE settled_at IS NULL;
@@ -0,0 +1,12 @@
1
+ -- Fence poll results against operator edits without holding transactions over provider I/O.
2
+ ALTER TABLE stackbone_platform.trigger_subscriptions
3
+ ADD COLUMN poll_revision bigint NOT NULL DEFAULT 0;
4
+
5
+ -- Existing claims get a full retention grace period from migration time.
6
+ -- Encountering a duplicate refreshes this timestamp without changing its delivery history.
7
+ ALTER TABLE stackbone_platform.trigger_deliveries
8
+ ADD COLUMN last_seen_at timestamptz NOT NULL DEFAULT now();
9
+ CREATE INDEX trigger_deliveries_last_seen_at_idx
10
+ ON stackbone_platform.trigger_deliveries (last_seen_at);
11
+ CREATE INDEX trigger_deliveries_subscription_last_seen_at_idx
12
+ ON stackbone_platform.trigger_deliveries (subscription_id, last_seen_at);
@@ -0,0 +1,50 @@
1
+ -- 0069_trigger_deliveries_queued_and_starting_status.sql — give `claimed` back its
2
+ -- literal meaning of "claimed and never attempted", so an abandoned claim can be
3
+ -- told apart from a live one and released (PLA-1358).
4
+ --
5
+ -- THE BUG. The inbound forwarder claims the (subscription, provider item) delivery
6
+ -- BEFORE it maps or starts anything. A process that dies in that gap leaves a
7
+ -- `claimed` row with no run; the next poll's claim collapses on the dedup index,
8
+ -- the tick counts a duplicate, and the cursor advances past an event that never
9
+ -- started. A sweep that released old `claimed` rows would fix it — except that
10
+ -- today three different situations all leave exactly `status = 'claimed'`,
11
+ -- `run_id` NULL, and nothing in the row separates them:
12
+ --
13
+ -- 1. the abandoned claim (a crash between the claim and the start)
14
+ -- 2. a delivery durably ENQUEUED behind an active serial run — accepted, no run
15
+ -- yet, deliberately not marked started so it never presents a run that does
16
+ -- not exist
17
+ -- 3. a run that DID start whose best-effort `markStarted` write was swallowed
18
+ --
19
+ -- Releasing (2) re-delivers an item that is already queued to run; releasing (3)
20
+ -- runs the customer's workflow a second time. Age does not separate them either.
21
+ -- So the journal gains two statuses and `claimed` stops being ambiguous:
22
+ --
23
+ -- `starting` — written immediately BEFORE the forwarder calls start or resume.
24
+ -- A row still `claimed` therefore provably never reached either,
25
+ -- which is what makes case 3 impossible and the sweep safe.
26
+ -- `queued` — case 2, labelled instead of left claimed. Still no `run_id`: the
27
+ -- real run surfaces on its own when the serial gate admits it.
28
+ --
29
+ -- The 0021 CHECK was declared INLINE (unnamed), so Postgres system-named it
30
+ -- `trigger_deliveries_status_check`; 0027 dropped that and re-added it with
31
+ -- `resumed`. Same move here, carrying the WHOLE accepted set forward — a widening
32
+ -- that re-derives the list instead of carrying it drops a status the forwarder
33
+ -- still writes, and the resulting constraint violation is swallowed by the
34
+ -- best-effort journal write, so it surfaces as a mislabelled row and not an error.
35
+ --
36
+ -- ONE-TIME COST ON DEPLOY, RECORDED AND ACCEPTED. Every `claimed` row that
37
+ -- predates this migration is ambiguous by construction — that is the bug — and
38
+ -- nothing in the row can say which of the three cases it was. The reclaim sweep
39
+ -- releases them like any other row. So an item that was sitting in a World serial
40
+ -- queue from before this migration can run once from that queue and once from the
41
+ -- re-delivery. It is bounded: one time, only `serial` workflows, only queues older
42
+ -- than the sweep's threshold, and only where the cursor has not already advanced
43
+ -- past the item. The alternative — a predicate skipping rows older than this
44
+ -- migration — would permanently forfeit recovery for exactly the rows most likely
45
+ -- to be abandoned, so it was rejected deliberately.
46
+ ALTER TABLE stackbone_platform.trigger_deliveries
47
+ DROP CONSTRAINT IF EXISTS trigger_deliveries_status_check;
48
+ ALTER TABLE stackbone_platform.trigger_deliveries
49
+ ADD CONSTRAINT trigger_deliveries_status_check
50
+ CHECK (status IN ('claimed', 'started', 'resumed', 'rejected', 'queued', 'starting'));
@@ -0,0 +1,39 @@
1
+ -- 0070_trigger_deliveries_takeover_count.sql — bound the poison item the cursor
2
+ -- hold creates: count how many times a delivery's abandoned claim has been taken
3
+ -- over, so the caller can stop taking it over (PLA-1380).
4
+ --
5
+ -- THE REGRESSION THIS BOUNDS. 0069 plus the in-`claim` takeover made an abandoned
6
+ -- claim recoverable by the next poll, and the intake tick now HOLDS ITS CURSOR
7
+ -- while a duplicate lands on a row still `claimed`. For an item whose forward
8
+ -- kills the process that combination never ends:
9
+ --
10
+ -- 1. `claim` takes the aged row over and re-stamps `created_at`, so the hourly
11
+ -- `releaseAbandonedClaims` (15 min on `created_at`) never matches it.
12
+ -- 2. Every re-list refreshes `last_seen_at`, so `deleteDeliveriesOlderThan`
13
+ -- never ages it out.
14
+ -- 3. The tick holds its cursor, so every item BEHIND this one is blocked too.
15
+ -- 4. The forward kills the process again on the next takeover. Forever, once
16
+ -- per takeover interval.
17
+ --
18
+ -- Nothing in the row could break that: age cannot, because a takeover resets it,
19
+ -- and that reset is load-bearing (it is what stops two forwarders in one drain
20
+ -- loop from both taking the same row over). So the row gains a counter that only
21
+ -- moves forward. `claim` increments it inside the SAME `CASE` that re-stamps
22
+ -- `created_at`, so it advances on an actual takeover and never on a duplicate or
23
+ -- a fresh insert, and the CALLER refuses to forward past
24
+ -- `DELIVERY_CLAIM_MAX_TAKEOVERS` — journalling the row `rejected` with reason
25
+ -- `claim_abandoned`, which is terminal, so the poll after it reads a plain
26
+ -- `duplicate` and the cursor finally advances.
27
+ --
28
+ -- NO CHECK REBUILD. The abandonment reuses `rejected` rather than minting a
29
+ -- status, so `trigger_deliveries_status_check` (re-added by 0069) stands as is.
30
+ --
31
+ -- NO TABLE REWRITE. `ADD COLUMN ... NOT NULL DEFAULT <constant>` is metadata-only
32
+ -- since PostgreSQL 11: the default is recorded in the catalog and read for
33
+ -- existing rows instead of being written into each one. It takes an ACCESS
34
+ -- EXCLUSIVE lock for the catalog update and releases it immediately, so the cost
35
+ -- is the lock wait behind whatever else is touching the table, not a rewrite of
36
+ -- it. Existing rows read 0, which is the honest value — nothing counted their
37
+ -- takeovers before this migration, so they get the full budget from here.
38
+ ALTER TABLE stackbone_platform.trigger_deliveries
39
+ ADD COLUMN takeover_count integer NOT NULL DEFAULT 0;