@fayz-ai/db 0.10.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,651 @@
1
+ -- ============================================================================
2
+ -- 033_sync_schedule.sql — the clock. One, generic, per connection.
3
+ --
4
+ -- Until this file nothing woke a connector. `SyncTrigger` has carried
5
+ -- `'scheduled'` since 029, `rateLimit` and `firstLoad` are declared on every
6
+ -- ConnectorDefinition, cursors are typed and asserted in the bench — and the
7
+ -- only use of the word `scheduled` in the codebase was a default value for a log
8
+ -- field. A merchant authorised Bling, closed the browser, and nothing ever
9
+ -- pulled the catalogue.
10
+ --
11
+ -- ── Why every mirroring connector needs this, not only the ones without push ──
12
+ -- Webhook is best-effort. The endpoint is down, the watch channel expires, the
13
+ -- provider gives up after N retries, the network eats one. With only a webhook,
14
+ -- ONE missed delivery is permanent silent divergence: the mirror is wrong and
15
+ -- nobody finds out until a customer is sold stock that does not exist.
16
+ --
17
+ -- The pull is not an alternative to the webhook. It is what makes the webhook
18
+ -- safe to trust. Push changes the FREQUENCY this has to run at; it never
19
+ -- changes whether it has to run. Google Calendar survives today by accident.
20
+ --
21
+ -- ── Why the clock is here and not in the platform ─────────────────────────────
22
+ -- The inverse symmetry of FAY-1388. The credential lives in the platform
23
+ -- because that is where it is audited. Execution lives in the pool because that
24
+ -- is where the edge function and `plg_sync_runs` already are, and because a
25
+ -- mirror must not stop diverging-and-repairing when fayz is down. A platform
26
+ -- job would make every tenant's freshness depend on one control plane's uptime.
27
+ --
28
+ -- ── Two jobs, both generic, neither belonging to any connector ────────────────
29
+ -- reconcile wake the connector to read from its cursor, apply, advance.
30
+ -- renew renew the provider-side subscription before it expires (Google's
31
+ -- watch channel, Bling's webhook registration).
32
+ -- A third, `prune`, retires the note 029 left on plg_prune_sync_runs — "no pool
33
+ -- runs pg_cron, so the caller is the sync ingress". A pool runs pg_cron now.
34
+ --
35
+ -- ── ORDERING ─────────────────────────────────────────────────────────────────
36
+ -- This file REQUIRES the connector spine: it references public.plg_connections
37
+ -- and public.plg_sync_runs, so it must be applied AFTER 029 (and, for the
38
+ -- pruner comment, after that same file). It does NOT touch 030/031/032. Four of
39
+ -- the seven pools (agency, restaurant, dentist, creators) have no spine at all
40
+ -- today; on those, `fayz db apply` reaches 029 before 033 in file order and the
41
+ -- requirement is satisfied by the ordering alone. Applying 033 to a pool with no
42
+ -- 029 fails loudly at the first CREATE — which is the correct outcome, not a
43
+ -- case to be guarded away: a clock with nothing to wake is a bug, not a state.
44
+ --
45
+ -- Additive and idempotent throughout: every statement is IF NOT EXISTS, CREATE
46
+ -- OR REPLACE, or a guarded DO. `fayz db apply` re-runs a file whose checksum
47
+ -- moved, so it must be safe twice.
48
+ -- ============================================================================
49
+
50
+ -- ── §0 provider_state: declared by FAY-1388, never built ────────────────────
51
+ -- FAY-1405 says the subscription expiry "lives in plg_connections.provider_state
52
+ -- (FAY-1388)". It does not: nothing in this repository has ever created that
53
+ -- column. It is added here because the renewal job cannot exist without it.
54
+ --
55
+ -- What belongs in it: what the PROVIDER knows about this connection and we only
56
+ -- mirror — the watch channel id and its expiry, the registered webhook id, the
57
+ -- resource id an unsubscribe needs. What does not: anything the tenant chose
58
+ -- (that is `settings`), anything about resume position (`cursors`), and above
59
+ -- all any credential — 029 and 032 both turn on there being no secret in this
60
+ -- table, and the channel token of a watch is not an exception worth making.
61
+ --
62
+ -- IT IS TENANT-WRITABLE, and that is deliberate rather than overlooked: 029
63
+ -- grants `authenticated` UPDATE on the whole table, and a column-level revoke on
64
+ -- top of a table-level grant is a no-op in Postgres. The renewal job therefore
65
+ -- treats `expiresAt` as a HINT and floors it (§4, `floor_seconds`) — a tenant who
66
+ -- writes an expiry one second in the future gets one renewal per floor, not a
67
+ -- loop that eats the pool's share of the provider's quota.
68
+ ALTER TABLE public.plg_connections
69
+ ADD COLUMN IF NOT EXISTS provider_state jsonb NOT NULL DEFAULT '{}'::jsonb;
70
+
71
+ COMMENT ON COLUMN public.plg_connections.provider_state IS
72
+ 'What the PROVIDER knows about this connection and we only mirror: watch '
73
+ 'channel id and expiry (`expiresAt`, ISO 8601), registered webhook id, '
74
+ 'resource id. Not settings, not cursors, and never a credential. Written '
75
+ 'server-side; the scheduler reads `expiresAt` as a hint and floors it.';
76
+
77
+ -- ── §1 extensions ───────────────────────────────────────────────────────────
78
+ -- Guarded one at a time so a pool that has one and not the other still gets it,
79
+ -- and so this file applies on a bare Postgres (the migration bench) instead of
80
+ -- aborting the chain. A pool missing either degrades to "nothing is dispatched
81
+ -- and every attempt says why" (§4), never to a broken migration.
82
+ DO $do$
83
+ BEGIN
84
+ BEGIN
85
+ EXECUTE 'CREATE EXTENSION IF NOT EXISTS pg_net WITH SCHEMA extensions';
86
+ EXCEPTION WHEN OTHERS THEN
87
+ RAISE WARNING 'pg_net could not be installed (%) — the clock will file every attempt as an error until it is', SQLERRM;
88
+ END;
89
+ BEGIN
90
+ -- No SCHEMA clause: Supabase ships a control file that pins pg_cron to
91
+ -- pg_catalog, and naming a schema there fails.
92
+ EXECUTE 'CREATE EXTENSION IF NOT EXISTS pg_cron';
93
+ EXCEPTION WHEN OTHERS THEN
94
+ RAISE WARNING 'pg_cron could not be installed (%) — NOTHING WILL WAKE A CONNECTOR on this pool', SQLERRM;
95
+ END;
96
+ END
97
+ $do$;
98
+
99
+ -- ── §2 the registry: how a connector declares a cadence ─────────────────────
100
+ -- One row per (connector, kind). This is the ONLY place a connector says how
101
+ -- often it needs waking, and it says it in SQL — shipped by the connector's own
102
+ -- migration, next to the tables it already ships.
103
+ --
104
+ -- WHY NOT A FIELD ON ConnectorDefinition. Because nothing would read it. The
105
+ -- clock runs inside Postgres with no view of a TypeScript manifest, and there is
106
+ -- no manifest→pool projection in this repository. A declared field nothing
107
+ -- consumes is prose that writes the future in the present; when that projection
108
+ -- exists, this table is its target and the connector's migration stops being
109
+ -- hand-written. Until then the declaration lives where the reader is.
110
+ --
111
+ -- `function_name` is CHECKed to a slug, not merely validated in the register
112
+ -- function: it is concatenated onto the pool's functions base URL, and a
113
+ -- constraint is what stops a row containing `../` from ever existing, however
114
+ -- it was inserted.
115
+ CREATE TABLE IF NOT EXISTS public.plg_connector_schedules (
116
+ connector_id text NOT NULL,
117
+ kind text NOT NULL CHECK (kind IN ('reconcile', 'renew')),
118
+ -- The edge function to wake, and the `action` inside its body. Two fields
119
+ -- because one function serves both kinds and several actions.
120
+ function_name text NOT NULL CHECK (function_name ~ '^[a-z0-9][a-z0-9_-]{0,62}$'),
121
+ action text NOT NULL CHECK (action ~ '^[a-z][a-z0-9_]{0,62}$'),
122
+ -- The cadence. Floored at 60s because the clock ticks once a minute; anything
123
+ -- smaller is a number that lies to whoever reads it.
124
+ interval_seconds integer NOT NULL CHECK (interval_seconds >= 60),
125
+ -- The spread. Each connection gets a deterministic offset in [0, jitter) so a
126
+ -- thousand connections enrolled in the same second do not come due together,
127
+ -- and do not re-converge after the first run.
128
+ jitter_seconds integer NOT NULL DEFAULT 60 CHECK (jitter_seconds >= 0),
129
+ -- The hard ceiling on dispatches per tick, per connector. This is what makes
130
+ -- a cold start drain instead of stampede: a pool where every connection is due
131
+ -- at once still issues at most this many calls a minute.
132
+ max_per_tick integer NOT NULL DEFAULT 25 CHECK (max_per_tick BETWEEN 1 AND 500),
133
+ -- The fastest this connector may EVER be woken, whatever any other input says.
134
+ -- The renewal job reads a tenant-writable expiry; this is what makes that safe.
135
+ floor_seconds integer NOT NULL DEFAULT 300 CHECK (floor_seconds >= 60),
136
+ -- Renewal only: how long before `provider_state.expiresAt` to renew. A watch
137
+ -- renewed at the last second is a watch that expires during a retry.
138
+ lead_seconds integer NOT NULL DEFAULT 86400 CHECK (lead_seconds >= 0),
139
+ -- After this, a run with no verdict is declared dead (§4, the reaper).
140
+ timeout_seconds integer NOT NULL DEFAULT 900 CHECK (timeout_seconds >= 60),
141
+ enabled boolean NOT NULL DEFAULT true,
142
+ created_at timestamptz NOT NULL DEFAULT now(),
143
+ updated_at timestamptz NOT NULL DEFAULT now(),
144
+ PRIMARY KEY (connector_id, kind)
145
+ );
146
+
147
+ COMMENT ON TABLE public.plg_connector_schedules IS
148
+ 'How often each connector needs waking, and which edge function + action to '
149
+ 'wake. Written by the connector''s own migration through '
150
+ 'plg_register_connector_schedule. Granted to no client role: it decides how '
151
+ 'much of a provider''s quota the whole pool spends.';
152
+
153
+ -- ── §3 the clock's own bookkeeping ──────────────────────────────────────────
154
+ -- One row per (connection, kind). NOT a second history — plg_sync_runs is the
155
+ -- history and stays the only one. This holds the single question the selection
156
+ -- asks ("when is this one next due?") so that question is an index range scan
157
+ -- over the DUE rows rather than a per-connection lookup over every connection in
158
+ -- the pool. That is the difference between cheap at a thousand connections and
159
+ -- a per-minute scan that grows with the pool.
160
+ --
161
+ -- It is also why "due" is not computed from plg_connections.last_sync_at:
162
+ -- `authenticated` may UPDATE that column, so a due-test built on it is a due-test
163
+ -- the tenant can drive. Everything the clock decides on lives either here (the
164
+ -- clock writes it) or in plg_sync_runs (029 grants the app no write at all).
165
+ CREATE TABLE IF NOT EXISTS public.plg_sync_schedule (
166
+ connection_id uuid NOT NULL REFERENCES public.plg_connections(id) ON DELETE CASCADE,
167
+ kind text NOT NULL CHECK (kind IN ('reconcile', 'renew')),
168
+ next_attempt_at timestamptz NOT NULL DEFAULT now(),
169
+ last_attempt_at timestamptz,
170
+ -- The run this connection was last dispatched with, until it gets a verdict.
171
+ -- Non-NULL means "in flight": the fold and the reaper both key off it.
172
+ last_run_id uuid,
173
+ -- Consecutive failures, for the backoff. Reset by the fold on any run that
174
+ -- came back success or partial.
175
+ failures integer NOT NULL DEFAULT 0,
176
+ PRIMARY KEY (connection_id, kind)
177
+ );
178
+
179
+ CREATE INDEX IF NOT EXISTS idx_plg_sync_schedule_due
180
+ ON public.plg_sync_schedule (kind, next_attempt_at);
181
+
182
+ -- Small and only ever read by the fold/reaper, but without it those two do a
183
+ -- seq scan on every tick of a pool with a thousand idle connections.
184
+ CREATE INDEX IF NOT EXISTS idx_plg_sync_schedule_inflight
185
+ ON public.plg_sync_schedule (last_run_id)
186
+ WHERE last_run_id IS NOT NULL;
187
+
188
+ COMMENT ON TABLE public.plg_sync_schedule IS
189
+ 'The clock''s bookkeeping: when each connection is next due, what is in '
190
+ 'flight, how many consecutive failures. Not a history — plg_sync_runs is. '
191
+ 'Rows are enrolled lazily by plg_sync_tick, with a deterministic offset so a '
192
+ 'cold start spreads instead of stampeding.';
193
+
194
+ -- ── §4 registration ─────────────────────────────────────────────────────────
195
+ -- Upsert, so a connector's migration re-applied (or its cadence changed in a
196
+ -- later migration) converges instead of erroring.
197
+ CREATE OR REPLACE FUNCTION public.plg_register_connector_schedule(
198
+ p_connector_id text,
199
+ p_kind text,
200
+ p_function_name text,
201
+ p_action text,
202
+ p_interval_seconds integer,
203
+ p_jitter_seconds integer DEFAULT NULL,
204
+ p_max_per_tick integer DEFAULT 25,
205
+ p_floor_seconds integer DEFAULT 300,
206
+ p_lead_seconds integer DEFAULT 86400,
207
+ p_timeout_seconds integer DEFAULT 900,
208
+ p_enabled boolean DEFAULT true
209
+ )
210
+ RETURNS void
211
+ LANGUAGE plpgsql
212
+ SECURITY DEFINER
213
+ SET search_path = public
214
+ AS $fn$
215
+ DECLARE
216
+ -- A quarter of the cadence by default: enough to break a cohort apart, small
217
+ -- enough that "every 15 minutes" is still true to within a rounding a merchant
218
+ -- would not notice.
219
+ v_jitter integer := COALESCE(p_jitter_seconds, GREATEST(p_interval_seconds / 4, 1));
220
+ BEGIN
221
+ INSERT INTO public.plg_connector_schedules AS s
222
+ (connector_id, kind, function_name, action, interval_seconds, jitter_seconds,
223
+ max_per_tick, floor_seconds, lead_seconds, timeout_seconds, enabled)
224
+ VALUES
225
+ (p_connector_id, p_kind, p_function_name, p_action, p_interval_seconds, v_jitter,
226
+ p_max_per_tick, p_floor_seconds, p_lead_seconds, p_timeout_seconds, p_enabled)
227
+ ON CONFLICT (connector_id, kind) DO UPDATE SET
228
+ function_name = EXCLUDED.function_name,
229
+ action = EXCLUDED.action,
230
+ interval_seconds = EXCLUDED.interval_seconds,
231
+ jitter_seconds = EXCLUDED.jitter_seconds,
232
+ max_per_tick = EXCLUDED.max_per_tick,
233
+ floor_seconds = EXCLUDED.floor_seconds,
234
+ lead_seconds = EXCLUDED.lead_seconds,
235
+ timeout_seconds = EXCLUDED.timeout_seconds,
236
+ enabled = EXCLUDED.enabled,
237
+ updated_at = now();
238
+ END;
239
+ $fn$;
240
+
241
+ -- ── §5 the endpoint, in Vault ───────────────────────────────────────────────
242
+ -- The same shape the Google Calendar outbound trigger has run since 2026-07-28
243
+ -- (plugin-agenda .../005_outbound_vault_delivery.sql): the credential is
244
+ -- encrypted at rest, outside pg_dump's plaintext, readable only by a definer
245
+ -- function, and it authorises exactly one thing — calling this pool's own edge
246
+ -- functions. Losing it does not hand anyone a tenant's data.
247
+ --
248
+ -- Emphatically NOT the service-role key. 005's header spells out why that was
249
+ -- an unrestricted read/write over every tenant of the cluster sitting in
250
+ -- pg_db_role_setting in plain text. The clock never needs a tenant identity: the
251
+ -- tenant is resolved from the connection row and travels in the BODY, the same
252
+ -- discipline plg_claim_effect follows. The header proves only "not a browser".
253
+ --
254
+ -- No secret is created here — nothing in this repository may contain one. The
255
+ -- operator calls this once per pool.
256
+ CREATE OR REPLACE FUNCTION public.plg_set_scheduler_endpoint(
257
+ p_functions_base_url text,
258
+ p_secret text
259
+ )
260
+ RETURNS void
261
+ LANGUAGE plpgsql
262
+ SECURITY DEFINER
263
+ SET search_path = public
264
+ AS $fn$
265
+ DECLARE
266
+ v_id uuid;
267
+ BEGIN
268
+ IF COALESCE(p_functions_base_url, '') = '' OR COALESCE(p_secret, '') = '' THEN
269
+ RAISE EXCEPTION 'both the functions base url and the scheduler secret are required';
270
+ END IF;
271
+ IF p_functions_base_url !~ '^https://' THEN
272
+ -- The bearer travels on this request. A plaintext scheme would put it on the
273
+ -- wire; refusing is the only place that can be caught before it happens.
274
+ RAISE EXCEPTION 'the functions base url must be https';
275
+ END IF;
276
+
277
+ -- Replace, never accumulate — 032's rule. A rotated secret that leaves the old
278
+ -- one decryptable in the vault is not a rotation.
279
+ SELECT id INTO v_id FROM vault.secrets WHERE name = 'fayz_functions_url';
280
+ IF v_id IS NULL THEN
281
+ PERFORM vault.create_secret(rtrim(p_functions_base_url, '/'), 'fayz_functions_url', 'fayz pool edge functions base url');
282
+ ELSE
283
+ PERFORM vault.update_secret(v_id, rtrim(p_functions_base_url, '/'));
284
+ END IF;
285
+
286
+ SELECT id INTO v_id FROM vault.secrets WHERE name = 'fayz_scheduler_secret';
287
+ IF v_id IS NULL THEN
288
+ PERFORM vault.create_secret(p_secret, 'fayz_scheduler_secret', 'fayz sync scheduler bearer');
289
+ ELSE
290
+ PERFORM vault.update_secret(v_id, p_secret);
291
+ END IF;
292
+ END;
293
+ $fn$;
294
+
295
+ -- ── §6 the tick ─────────────────────────────────────────────────────────────
296
+ -- One function, two kinds, four phases. Runs once a minute under pg_cron as
297
+ -- `postgres`; SECURITY DEFINER so it can write plg_sync_runs, which 029 grants
298
+ -- no application role a write on.
299
+ --
300
+ -- reap a dispatch that never came back is a failure, not a silence.
301
+ -- fold a settled run becomes backoff (or clears it).
302
+ -- enroll connections with a registered schedule and no state row.
303
+ -- dispatch open a run, post, advance.
304
+ --
305
+ -- THE ORDER MATTERS: reap before fold means a dead run is folded in the same
306
+ -- tick it is declared dead, so a connector whose function is down backs off
307
+ -- immediately instead of being re-dispatched every minute for the timeout.
308
+ --
309
+ -- A RUN ROW FOR EVERY ATTEMPT, INCLUDING ONE THAT NEVER REACHED THE FUNCTION.
310
+ -- Missing pg_net, no Vault secret, a rejected post: each files a run with
311
+ -- status 'error' and a sentence saying which. A schedule that silently does
312
+ -- nothing is indistinguishable from one that works, and this is the whole
313
+ -- difference.
314
+ CREATE OR REPLACE FUNCTION public.plg_sync_tick(p_kind text)
315
+ RETURNS integer
316
+ LANGUAGE plpgsql
317
+ SECURITY DEFINER
318
+ SET search_path = public, extensions, net, vault
319
+ AS $fn$
320
+ DECLARE
321
+ v_base text;
322
+ v_secret text;
323
+ v_have_net boolean;
324
+ v_row record;
325
+ v_run_id uuid;
326
+ v_next timestamptz;
327
+ v_expires timestamptz;
328
+ v_offset integer;
329
+ v_dispatched integer := 0;
330
+ v_reason text;
331
+ BEGIN
332
+ IF p_kind NOT IN ('reconcile', 'renew') THEN
333
+ RAISE EXCEPTION 'plg_sync_tick: unknown kind %', p_kind;
334
+ END IF;
335
+
336
+ -- A tick that outlives its minute must not overlap the next one: two ticks
337
+ -- would both see the same due rows and dispatch each twice. Skipping is right
338
+ -- — the work is not lost, it is due a minute later.
339
+ IF NOT pg_try_advisory_xact_lock(hashtext('plg_sync_tick:' || p_kind)) THEN
340
+ RETURN 0;
341
+ END IF;
342
+
343
+ v_have_net := to_regprocedure('net.http_post(text, jsonb, jsonb, jsonb, integer)') IS NOT NULL;
344
+
345
+ -- Wrapped: a pool without the vault extension must degrade to "every attempt
346
+ -- says it is not configured", not abort the tick.
347
+ BEGIN
348
+ SELECT decrypted_secret INTO v_base
349
+ FROM vault.decrypted_secrets WHERE name = 'fayz_functions_url'
350
+ ORDER BY created_at DESC LIMIT 1;
351
+ SELECT decrypted_secret INTO v_secret
352
+ FROM vault.decrypted_secrets WHERE name = 'fayz_scheduler_secret'
353
+ ORDER BY created_at DESC LIMIT 1;
354
+ EXCEPTION WHEN OTHERS THEN
355
+ v_base := NULL;
356
+ v_secret := NULL;
357
+ END;
358
+
359
+ -- ── reap ──────────────────────────────────────────────────────────────────
360
+ UPDATE public.plg_sync_runs r
361
+ SET status = 'error',
362
+ finished_at = now(),
363
+ error = COALESCE(r.error, 'the connector function returned no verdict within the schedule timeout')
364
+ FROM public.plg_sync_schedule s
365
+ JOIN public.plg_connections c ON c.id = s.connection_id
366
+ JOIN public.plg_connector_schedules d
367
+ ON d.connector_id = c.connector_id AND d.kind = s.kind
368
+ WHERE s.kind = p_kind
369
+ AND r.id = s.last_run_id
370
+ AND r.status = 'running'
371
+ AND r.started_at < now() - make_interval(secs => d.timeout_seconds);
372
+
373
+ -- ── fold ──────────────────────────────────────────────────────────────────
374
+ -- Exponential backoff on consecutive failures, capped at a day. The cap is not
375
+ -- decoration: without it a connection that failed twenty times would next be
376
+ -- tried after the heat death of the merchant's patience, and a provider that
377
+ -- came back would never be noticed.
378
+ UPDATE public.plg_sync_schedule s
379
+ SET failures = CASE WHEN r.status IN ('success', 'partial') THEN 0 ELSE s.failures + 1 END,
380
+ last_run_id = NULL,
381
+ next_attempt_at = CASE
382
+ WHEN r.status IN ('success', 'partial') THEN s.next_attempt_at
383
+ ELSE GREATEST(
384
+ s.next_attempt_at,
385
+ now() + make_interval(secs => LEAST(
386
+ d.interval_seconds::bigint * (2 ^ LEAST(s.failures + 1, 6))::bigint,
387
+ 86400)::double precision)
388
+ )
389
+ END
390
+ FROM public.plg_sync_runs r,
391
+ public.plg_connections c,
392
+ public.plg_connector_schedules d
393
+ WHERE s.kind = p_kind
394
+ AND r.id = s.last_run_id
395
+ AND r.finished_at IS NOT NULL
396
+ AND c.id = s.connection_id
397
+ AND d.connector_id = c.connector_id
398
+ AND d.kind = s.kind;
399
+
400
+ -- ── enroll ────────────────────────────────────────────────────────────────
401
+ -- Lazy, so neither the connect flow nor a backfill script has to know the
402
+ -- clock exists — a connection made before this migration and one made a minute
403
+ -- from now enter the same way.
404
+ --
405
+ -- THE OFFSET IS THE ANTI-STAMPEDE, and it is applied HERE because this is the
406
+ -- one moment a whole pool can enter the schedule at once: the first tick after
407
+ -- this file is applied enrolls every existing connection in the same second.
408
+ -- Without the offset all of them would be due at the same instant forever
409
+ -- after, since each advance adds the same interval to the same base.
410
+ -- Deterministic from the connection id, so it survives a re-enrollment.
411
+ INSERT INTO public.plg_sync_schedule (connection_id, kind, next_attempt_at)
412
+ SELECT q.id, p_kind,
413
+ now() + make_interval(secs => mod(abs(hashtext(q.id::text)::bigint), GREATEST(q.jitter_seconds, 1))::double precision)
414
+ FROM (
415
+ SELECT c.id, d.jitter_seconds
416
+ FROM public.plg_connections c
417
+ JOIN public.plg_connector_schedules d
418
+ ON d.connector_id = c.connector_id AND d.kind = p_kind AND d.enabled
419
+ LEFT JOIN public.plg_sync_schedule s
420
+ ON s.connection_id = c.id AND s.kind = p_kind
421
+ WHERE s.connection_id IS NULL
422
+ AND c.active
423
+ -- Bounded so the first tick on a large pool does not enrol ten thousand
424
+ -- rows in one transaction; the rest arrive on the next tick.
425
+ LIMIT 500
426
+ ) q
427
+ ON CONFLICT (connection_id, kind) DO NOTHING;
428
+
429
+ -- ── dispatch ──────────────────────────────────────────────────────────────
430
+ FOR v_row IN
431
+ SELECT *
432
+ FROM (
433
+ SELECT s.connection_id, s.failures, c.tenant_id, c.connector_id, c.provider_state,
434
+ d.function_name, d.action, d.interval_seconds, d.jitter_seconds,
435
+ d.floor_seconds, d.lead_seconds, d.timeout_seconds, d.max_per_tick,
436
+ row_number() OVER (PARTITION BY c.connector_id ORDER BY s.next_attempt_at, s.connection_id) AS rn
437
+ FROM (
438
+ SELECT s2.*
439
+ FROM public.plg_sync_schedule s2
440
+ WHERE s2.kind = p_kind AND s2.next_attempt_at <= now()
441
+ ORDER BY s2.next_attempt_at
442
+ -- A ceiling on the work ONE tick can consider, independent of any
443
+ -- connector's max_per_tick. Ten thousand due rows must not become a
444
+ -- ten-thousand-row sort every minute.
445
+ LIMIT 2000
446
+ ) s
447
+ JOIN public.plg_connections c ON c.id = s.connection_id
448
+ JOIN public.plg_connector_schedules d
449
+ ON d.connector_id = c.connector_id AND d.kind = s.kind
450
+ WHERE d.enabled
451
+ AND c.active
452
+ -- 'pending' is "connected but not configured yet" — waking it just
453
+ -- files an error a minute. 'revoked' is the tenant saying stop.
454
+ AND c.status IN ('connected', 'error')
455
+ ) ranked
456
+ WHERE ranked.rn <= ranked.max_per_tick
457
+ LOOP
458
+ -- The run row is opened BEFORE the post, so a post that throws still leaves
459
+ -- an attempt on the record. tenant_id comes from the connection, never from
460
+ -- a caller — plg_claim_effect's discipline, for the same reason.
461
+ INSERT INTO public.plg_sync_runs
462
+ (tenant_id, connection_id, connector_id, direction, trigger_kind, status, stats)
463
+ VALUES
464
+ (v_row.tenant_id, v_row.connection_id, v_row.connector_id,
465
+ CASE WHEN p_kind = 'renew' THEN 'outbound' ELSE 'inbound' END,
466
+ 'scheduled', 'running',
467
+ jsonb_build_object('schedule_kind', p_kind, 'attempt', v_row.failures + 1))
468
+ RETURNING id INTO v_run_id;
469
+
470
+ v_reason := NULL;
471
+ IF NOT v_have_net THEN
472
+ v_reason := 'pg_net is not installed on this pool, so the scheduler cannot call the connector function';
473
+ ELSIF COALESCE(v_base, '') = '' OR COALESCE(v_secret, '') = '' THEN
474
+ v_reason := 'the scheduler endpoint is not configured: run plg_set_scheduler_endpoint on this pool';
475
+ ELSE
476
+ BEGIN
477
+ PERFORM net.http_post(
478
+ url := v_base || '/' || v_row.function_name,
479
+ headers := jsonb_build_object(
480
+ 'Content-Type', 'application/json',
481
+ -- Proves "not a browser" and nothing else. It is NOT an identity:
482
+ -- the tenant is in the body, resolved from the connection row.
483
+ 'Authorization', 'Bearer ' || v_secret,
484
+ 'X-Fayz-Scheduler', '1'
485
+ ),
486
+ body := jsonb_build_object(
487
+ 'action', v_row.action,
488
+ 'kind', p_kind,
489
+ 'connectionId', v_row.connection_id,
490
+ 'tenantId', v_row.tenant_id,
491
+ 'connectorId', v_row.connector_id,
492
+ 'runId', v_run_id,
493
+ 'attempt', v_row.failures + 1,
494
+ 'scheduledAt', to_char(now() AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
495
+ ),
496
+ timeout_milliseconds := v_row.timeout_seconds * 1000
497
+ );
498
+ EXCEPTION WHEN OTHERS THEN
499
+ v_reason := 'the scheduler could not post to the connector function: ' || SQLERRM;
500
+ END;
501
+ END IF;
502
+
503
+ IF v_reason IS NOT NULL THEN
504
+ -- Settled here rather than left to the reaper: this attempt provably never
505
+ -- left the database, so waiting out the timeout would only delay the
506
+ -- backoff and hide the reason.
507
+ UPDATE public.plg_sync_runs
508
+ SET status = 'error', error = v_reason, finished_at = now()
509
+ WHERE id = v_run_id;
510
+ END IF;
511
+
512
+ -- ── advance ─────────────────────────────────────────────────────────────
513
+ v_offset := mod(abs(hashtext(v_row.connection_id::text)::bigint),
514
+ GREATEST(v_row.jitter_seconds, 1))::integer;
515
+ v_next := now() + make_interval(secs => (v_row.interval_seconds + v_offset)::double precision);
516
+
517
+ IF p_kind = 'renew' THEN
518
+ -- The expiry the provider gave us pulls the next attempt EARLIER, never
519
+ -- later: a watch that dies in an hour must not wait for a daily cadence.
520
+ BEGIN
521
+ v_expires := (v_row.provider_state ->> 'expiresAt')::timestamptz;
522
+ EXCEPTION WHEN OTHERS THEN
523
+ v_expires := NULL; -- a malformed hint is no hint
524
+ END;
525
+ IF v_expires IS NOT NULL THEN
526
+ v_next := LEAST(v_next, v_expires - make_interval(secs => v_row.lead_seconds::double precision));
527
+ END IF;
528
+ END IF;
529
+
530
+ -- The floor, last and unconditional. provider_state is tenant-writable
531
+ -- (§0); this is the line that makes that harmless.
532
+ v_next := GREATEST(v_next, now() + make_interval(secs => v_row.floor_seconds::double precision));
533
+
534
+ UPDATE public.plg_sync_schedule
535
+ SET next_attempt_at = v_next,
536
+ last_attempt_at = now(),
537
+ last_run_id = v_run_id
538
+ WHERE connection_id = v_row.connection_id AND kind = p_kind;
539
+
540
+ v_dispatched := v_dispatched + 1;
541
+ END LOOP;
542
+
543
+ RETURN v_dispatched;
544
+ END;
545
+ $fn$;
546
+
547
+ -- ── §7 who may reach any of this ────────────────────────────────────────────
548
+ -- 011 shut `anon` out of every new table in this schema and left `authenticated`
549
+ -- born with the full set — the trap 029 and 032 both documented. Both new tables
550
+ -- are stripped and nothing is granted back, and RLS with no policy makes the
551
+ -- grant and the policy set agree, so "fixing the grant" alone reopens nothing.
552
+ --
553
+ -- Neither table is tenant data, and that is exactly why they are shut. A tenant
554
+ -- who could write plg_connector_schedules would set interval_seconds to its
555
+ -- floor for every tenant on the pool and spend the provider's shared quota; one
556
+ -- who could write plg_sync_schedule would put itself first in a queue everyone
557
+ -- shares. "Next sync at" is a legitimate thing for a panel to want — it can be
558
+ -- served later by a definer function that returns one connection's row, not by
559
+ -- opening the table.
560
+ REVOKE ALL ON public.plg_connector_schedules FROM PUBLIC;
561
+ REVOKE ALL ON public.plg_connector_schedules FROM anon, authenticated;
562
+ REVOKE ALL ON public.plg_sync_schedule FROM PUBLIC;
563
+ REVOKE ALL ON public.plg_sync_schedule FROM anon, authenticated;
564
+
565
+ ALTER TABLE public.plg_connector_schedules ENABLE ROW LEVEL SECURITY;
566
+ ALTER TABLE public.plg_sync_schedule ENABLE ROW LEVEL SECURITY;
567
+
568
+ -- PUBLIC first: EXECUTE on a new function is granted to PUBLIC by default, and
569
+ -- revoking from anon/authenticated alone leaves it reachable through that.
570
+ --
571
+ -- plg_sync_tick is the sharpest of the three. Anyone who can call it can make
572
+ -- the database issue a burst of authenticated calls to every connector function
573
+ -- on the pool, on demand, as fast as they can loop — the stampede this whole
574
+ -- file is built to prevent, handed to a browser.
575
+ REVOKE ALL ON FUNCTION public.plg_sync_tick(text) FROM PUBLIC;
576
+ REVOKE ALL ON FUNCTION public.plg_sync_tick(text) FROM anon, authenticated;
577
+ GRANT EXECUTE ON FUNCTION public.plg_sync_tick(text) TO service_role;
578
+
579
+ REVOKE ALL ON FUNCTION public.plg_register_connector_schedule(text, text, text, text, integer, integer, integer, integer, integer, integer, boolean) FROM PUBLIC;
580
+ REVOKE ALL ON FUNCTION public.plg_register_connector_schedule(text, text, text, text, integer, integer, integer, integer, integer, integer, boolean) FROM anon, authenticated;
581
+ GRANT EXECUTE ON FUNCTION public.plg_register_connector_schedule(text, text, text, text, integer, integer, integer, integer, integer, integer, boolean) TO service_role;
582
+
583
+ REVOKE ALL ON FUNCTION public.plg_set_scheduler_endpoint(text, text) FROM PUBLIC;
584
+ REVOKE ALL ON FUNCTION public.plg_set_scheduler_endpoint(text, text) FROM anon, authenticated;
585
+ GRANT EXECUTE ON FUNCTION public.plg_set_scheduler_endpoint(text, text) TO service_role;
586
+
587
+ COMMENT ON FUNCTION public.plg_sync_tick(text) IS
588
+ 'One tick of the sync clock: reap dead dispatches, fold verdicts into backoff, '
589
+ 'enrol new connections with a deterministic offset, dispatch what is due up to '
590
+ 'max_per_tick per connector. Files a plg_sync_runs row for EVERY attempt, '
591
+ 'including one that never reached the function. service_role and cron only.';
592
+ COMMENT ON FUNCTION public.plg_register_connector_schedule(text, text, text, text, integer, integer, integer, integer, integer, integer, boolean) IS
593
+ 'How a connector declares its cadence. Called from the connector''s own '
594
+ 'migration; upsert, so re-applying converges.';
595
+ COMMENT ON FUNCTION public.plg_set_scheduler_endpoint(text, text) IS
596
+ 'Place the pool''s edge-functions base URL and the scheduler bearer in Vault. '
597
+ 'Run once per pool by an operator; no secret is ever committed.';
598
+
599
+ -- ── §8 cron, and who may not touch it ───────────────────────────────────────
600
+ -- MEASURED, not assumed, on cluster-salon-br-01 and cluster-ecommerce-br-01:
601
+ --
602
+ -- schema cron {supabase_admin=UC, postgres=U*} -- no anon/authenticated
603
+ -- cron.schedule {=X/supabase_admin, ...} -- EXECUTE to PUBLIC
604
+ -- cron.unschedule {=X/supabase_admin, ...} -- EXECUTE to PUBLIC
605
+ -- cron.job {=r/supabase_admin, ...} -- SELECT to PUBLIC
606
+ -- cron.job_run_details {=rd/supabase_admin, ...} -- SELECT+DELETE to PUBLIC
607
+ --
608
+ -- So the ONLY thing standing between `authenticated` and rescheduling this pool's
609
+ -- clock is USAGE on the `cron` schema, which Supabase does not grant. The
610
+ -- function and table ACLs behind it are wide open, and every one of those objects
611
+ -- is owned by `supabase_admin` — a migration running as `postgres` cannot revoke
612
+ -- a grant it did not make, so this file does NOT pretend to close them. What it
613
+ -- does instead is make the one line that matters a graded assertion
614
+ -- (sync-schedule.regression.sql S8): the day anyone runs
615
+ -- `GRANT USAGE ON SCHEMA cron TO authenticated` "for the dashboard", CI goes red.
616
+ --
617
+ -- Idempotent: cron.schedule upserts by job name.
618
+ DO $do$
619
+ BEGIN
620
+ IF to_regprocedure('cron.schedule(text, text, text)') IS NULL THEN
621
+ RAISE WARNING 'pg_cron is not available: the sync clock is NOT scheduled on this pool and no connector will be woken';
622
+ RETURN;
623
+ END IF;
624
+
625
+ -- Every minute. The per-connector cadence lives in the registry, not here:
626
+ -- a job per connector would put the connector's name in the platform's clock,
627
+ -- which is the coupling this file exists to avoid.
628
+ PERFORM cron.schedule('fayz_sync_reconcile', '* * * * *',
629
+ $cmd$SELECT public.plg_sync_tick('reconcile')$cmd$);
630
+
631
+ -- Renewal windows are hours and days; a minute of granularity buys nothing and
632
+ -- costs a query. `lead_seconds` is what makes five minutes safe.
633
+ PERFORM cron.schedule('fayz_sync_renew', '*/5 * * * *',
634
+ $cmd$SELECT public.plg_sync_tick('renew')$cmd$);
635
+
636
+ -- 029 shipped the pruner and wrote on it "no pool runs pg_cron, so the caller
637
+ -- is the sync ingress — it prunes the connection it just wrote". That was true
638
+ -- and is not any more; leaving retention to whoever happens to sync means a
639
+ -- connector that stops syncing keeps its rows forever.
640
+ PERFORM cron.schedule('fayz_sync_prune', '17 4 * * *',
641
+ $cmd$SELECT public.plg_prune_sync_runs()$cmd$);
642
+ EXCEPTION WHEN OTHERS THEN
643
+ RAISE WARNING 'could not schedule the fayz sync clock (%) — no connector will be woken on this pool', SQLERRM;
644
+ END
645
+ $do$;
646
+
647
+ -- The note 029 left on the pruner — "No pool runs pg_cron, so the caller is the
648
+ -- sync ingress" — is now wrong. Restated rather than edited in place: 029 is
649
+ -- applied on every pool and editing an applied migration is a ledger hard stop.
650
+ COMMENT ON FUNCTION public.plg_prune_sync_runs(integer, integer) IS
651
+ 'Retention for plg_sync_runs: deletes runs that are both outside the newest N of their connection and older than the window. Scheduled nightly by 033 (cron job fayz_sync_prune); the sync ingress may still call it directly.';