@pikku/kysely 0.13.31 → 0.13.34

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/CHANGELOG.md +84 -0
  2. package/README.md +12 -2
  3. package/dist/src/index.d.ts +2 -1
  4. package/dist/src/index.js +2 -1
  5. package/dist/src/kysely-lease-clock.d.ts +14 -0
  6. package/dist/src/kysely-lease-clock.js +14 -0
  7. package/dist/src/kysely-lease-service.d.ts +37 -0
  8. package/dist/src/kysely-lease-service.js +116 -0
  9. package/dist/src/kysely-tables.d.ts +16 -0
  10. package/dist/src/kysely-workflow-service.d.ts +36 -20
  11. package/dist/src/kysely-workflow-service.js +195 -102
  12. package/dist/src/schema/agent.schema.js +43 -36
  13. package/dist/src/schema/analytics.schema.js +17 -17
  14. package/dist/src/schema/audit.schema.js +22 -22
  15. package/dist/src/schema/channel.schema.js +13 -10
  16. package/dist/src/schema/credential.schema.js +6 -6
  17. package/dist/src/schema/deployment.schema.js +12 -12
  18. package/dist/src/schema/flag.schema.js +15 -12
  19. package/dist/src/schema/incoming-webhook.schema.js +7 -7
  20. package/dist/src/schema/index.d.ts +1 -0
  21. package/dist/src/schema/index.js +33 -7
  22. package/dist/src/schema/lease.schema.d.ts +2 -0
  23. package/dist/src/schema/lease.schema.js +12 -0
  24. package/dist/src/schema/pikku-schema.types.d.ts +42 -2
  25. package/dist/src/schema/pikku-schema.types.js +23 -7
  26. package/dist/src/schema/scope.schema.js +34 -22
  27. package/dist/src/schema/secret.schema.js +5 -5
  28. package/dist/src/schema/session.schema.js +3 -3
  29. package/dist/src/schema/trigger-source.schema.js +8 -8
  30. package/dist/src/schema/virtual-user-schedule.schema.js +4 -4
  31. package/dist/src/schema/virtual-user.schema.js +14 -14
  32. package/dist/src/schema/webhook.schema.js +17 -17
  33. package/dist/src/schema/workflow.schema.js +50 -49
  34. package/knowledge/decisions/a-lease-is-taken-by-one-guarded-update.md +30 -0
  35. package/knowledge/decisions/index.md +3 -0
  36. package/knowledge/decisions/leases-are-judged-by-the-database-clock.md +42 -0
  37. package/knowledge/decisions/workflows-own-the-lease-table-too.md +22 -0
  38. package/package.json +2 -2
package/CHANGELOG.md CHANGED
@@ -1,3 +1,87 @@
1
+ ## 0.13.34
2
+
3
+ ### Patch Changes
4
+
5
+ - 80c94f9: MySQL is a third dialect for the `pikku db` commands, beside sqlite and postgres.
6
+
7
+ Set `mysqlUrl` in `createConfig` and `db migrate`, `generate`, `check`, `baseline`, `reset`, `codegen` and the dev seed behave as they do elsewhere, with migrations in `db/mysql/` and the seed in `db/mysql-dev-seed.sql`. MySQL has no embedded engine, so a command that needs a throwaway database creates and drops a `pikku_scratch_<hex>` one on your server — the account needs `CREATE` and `DROP`. DDL is not transactional there, so a migration that fails halfway is replayed from the top. `db.schema` is refused, because a MySQL schema is a database.
8
+
9
+ `@pikku/migrator-sql` gains `@pikku/migrator-sql/mysql`. The runtime schemas in `@pikku/kysely` now compile for MySQL without changing what sqlite and postgres get: keys are `varchar(255)`, free text is `longtext`, literal text defaults are expressions, and foreign keys are table-level constraints, because MySQL parses an inline `REFERENCES` and silently drops it. A foreign key onto a column another source owns takes that column's full type (`varchar(36)`), not the bare `varchar` the introspector reports.
10
+
11
+ An addon that ships `db/mysql` migrations needs `db.mysqlUrl` in its `pikku.config.json` for `pikku all` to publish them. The standalone bundle does not support MySQL yet and refuses a MySQL project by name.
12
+
13
+ - Updated dependencies [149faae]
14
+ - @pikku/core@0.12.136
15
+
16
+ ## 0.13.33
17
+
18
+ ### Patch Changes
19
+
20
+ - a16b0ba: The workflow run lease takes the app's `leaseService` (`holdLease(leaseService, 'workflow-run:<id>', …)`) in place of `pg_advisory_lock` / `GET_LOCK`, for DSL and graph runs alike. A message for a run already being orchestrated elsewhere wakes the run again a second later instead of failing, so a long pass never spends the retry budget the queue keeps for real failures. An inline run whose lease was lost after its body finished keeps the outcome it wrote. `PgWorkflowQueueOptions` and `RunLockHoldTimeoutError` are removed. `WorkflowService.withRunLock` is renamed `withRunLease`, since what it holds runs out unless renewed; a custom workflow service overriding it renames the method.
21
+
22
+ Pass the lease service to the workflow service — `new PgKyselyWorkflowService(db, { leaseService })` — register the same instance as the app's `leaseService`, and migrate `pikku_lease`.
23
+
24
+ A step lease is renewed by the same loop as the run lease. A graph node whose worker died is dispatched again once its lease lapses, instead of leaving the run waiting on it. A replayed step on Kysely now sees its lease, as `getStepState` does. The in-memory store refuses a superseded worker's outcome, as the database stores do. The MySQL workflow service can now record a workflow version; it used `ON CONFLICT`, which MySQL does not have.
25
+
26
+ `pikku db generate` now writes the `pikku_lease` table for any project that runs workflows, not only one that reaches `leaseService`, since every persistent workflow service holds its runs and steps there.
27
+
28
+ - a16b0ba: Give a running workflow step a lease, so a step whose worker dies can be claimed again
29
+
30
+ A step moving from `pending` to `running` only proves it was dispatched; it says
31
+ nothing about whether the worker holding it is still alive. A worker that died
32
+ mid-step therefore left the step wedged in `running` forever: unclaimable by the
33
+ next dispatch, and invisible to the stalled-run sweep.
34
+
35
+ The dispatch that claims a step now stamps a lease on it, refreshes that lease on
36
+ an interval for as long as it is working, and releases it when the step parks on
37
+ a child run. A step whose lease has lapsed is claimable again, and counts as
38
+ in-flight no longer, so the stalled sweep can pick its run back up. A step that
39
+ keeps losing its worker fails with `WorkflowStepLeaseExpiredError` once its
40
+ attempts run out, rather than looping.
41
+
42
+ The lease duration is taken from the queue the step is dispatched through, so the
43
+ lease and the queue lock never disagree about who owns the job.
44
+
45
+ The refresh runs at half the lease and never later, even when that is under the
46
+ one-second floor the interval otherwise respects: applying that floor as a
47
+ maximum would renew a shorter lease for the first time after it had already
48
+ lapsed, which is the double-claim the lease exists to prevent. A lease that
49
+ short is logged once, pointing at the queue's `lockDuration`.
50
+
51
+ The lease is written and judged on the database's clock, never the worker's,
52
+ so a worker whose clock runs fast cannot claim a step another worker still
53
+ holds. A dispatch whose step has since been claimed by another stops renewing,
54
+ and says so: `refreshStepLease(stepId, leaseMs, attempt)` resolves `false` once
55
+ the step is no longer that attempt's.
56
+
57
+ The lease lives in a new `workflow_step.lease_expires_at` column, in epoch
58
+ milliseconds (`bigint`). Run `pikku db generate` and `pikku db migrate` after
59
+ upgrading: the runtime does not add columns, and every step read selects this
60
+ one.
61
+
62
+ - a16b0ba: Persistent workflow services take a required `leaseService` (`WorkflowServiceOptions`) and lock runs and steps on it. The Postgres advisory and MySQL `GET_LOCK` step locks, and Redis's own `SET NX` run and step locks, are removed; `RedisWorkflowService` now takes `(connection, { leaseService, keyPrefix? })`.
63
+ - Updated dependencies [a16b0ba]
64
+ - Updated dependencies [21e9c3a]
65
+ - Updated dependencies [a16b0ba]
66
+ - Updated dependencies [a16b0ba]
67
+ - @pikku/core@0.12.133
68
+
69
+ ## 0.13.32
70
+
71
+ ### Patch Changes
72
+
73
+ - 942ebdd: Add `LeaseService`: named leases shared by every process on the same store. A lease is not a mutex: a holder that stalls past its expiry loses the key without knowing, so whatever the body writes to should check the lease's `token` and refuse an older one. `acquire` never blocks and answers `null` while someone else holds the key; a lease lapses on its own if its holder dies; and each lease carries a `token` that rises every time the key changes hands, so a stale holder cannot refresh or release its successor's lease. `holdLease` refreshes the lease while its body runs, hands the body an `AbortSignal` that fires the moment the lease is lost, and throws `LeaseLostError` once the body finishes if the lease lapsed or changed hands meanwhile, so a caller never takes that result as produced under the lease.
74
+
75
+ Ships as `InMemoryLeaseService`, `PgKyselyLeaseService` (`@pikku/kysely-postgres`), `MySQLKyselyLeaseService` (`@pikku/kysely-mysql`) and `KyselyLeaseService` (SQLite), all on `pikku_lease`. The Postgres and MySQL services judge every lease by the database's clock, so a worker whose clock runs fast cannot take a live lease or stretch its own; `KyselyLeaseService` reads the process clock and is for a single-host SQLite app. `pikku db generate` writes the table for any project that reaches `leaseService`.
76
+
77
+ `RedisLeaseService` (`@pikku/redis`) keeps each lease in a hash Redis expires itself, with every check-and-set in one Lua script that reads Redis's `TIME`, so it too judges leases by the server's clock; its token counter outlives release, so the next holder always gets a higher token.
78
+
79
+ - Updated dependencies [d327fa5]
80
+ - Updated dependencies [942ebdd]
81
+ - Updated dependencies [8ac25a8]
82
+ - Updated dependencies [6606777]
83
+ - @pikku/core@0.12.132
84
+
1
85
  ## 0.13.31
2
86
 
3
87
  ### Patch Changes
package/README.md CHANGED
@@ -15,16 +15,26 @@ npm install @pikku/kysely kysely
15
15
  ## Usage
16
16
 
17
17
  ```typescript
18
- import { KyselyWorkflowService, KyselySecretService } from '@pikku/kysely'
18
+ import {
19
+ KyselyLeaseService,
20
+ KyselySecretService,
21
+ KyselyWorkflowService,
22
+ } from '@pikku/kysely'
19
23
  import type { KyselyPikkuDB } from '@pikku/kysely'
20
24
  import type { Kysely } from 'kysely'
21
25
 
22
26
  declare const db: Kysely<KyselyPikkuDB>
23
27
 
24
- const workflowService = new KyselyWorkflowService(db)
28
+ const leaseService = new KyselyLeaseService(db)
29
+ await leaseService.init()
30
+
31
+ const workflowService = new KyselyWorkflowService(db, { leaseService })
25
32
  const secretService = new KyselySecretService(db, { key: encryptionKey })
26
33
  ```
27
34
 
35
+ The workflow service locks runs and steps on the `leaseService` you give it;
36
+ register the same instance as the app's `leaseService`.
37
+
28
38
  ## Docs
29
39
 
30
40
  https://pikku.dev/docs
@@ -15,6 +15,7 @@ export { KyselyFeatureFlagStore } from './kysely-feature-flag-store.js';
15
15
  export type { KyselyFeatureFlagStoreOptions } from './kysely-feature-flag-store.js';
16
16
  export { KyselyWebhookService } from './kysely-webhook-service.js';
17
17
  export { KyselyTriggerSourceStore } from './kysely-trigger-source-store.js';
18
+ export { KyselyLeaseService } from './kysely-lease-service.js';
18
19
  export { KyselyIncomingWebhookService } from './kysely-incoming-webhook-service.js';
19
20
  export { createAuditedKysely, type CreateAuditedKyselyOptions, } from './create-audited-kysely.js';
20
21
  export { KyselyAuditService } from './kysely-audit-service.js';
@@ -26,7 +27,7 @@ export { SerializePlugin, BaseSerializePlugin, type Serializer, type Deserialize
26
27
  export { createCoercionPlugin, type ColumnKind, type CoercionMap, type CreateCoercionPluginOptions, } from './coercion-plugin.js';
27
28
  export { createClassificationPlugin, type CreateClassificationPluginOptions, } from './classification-plugin.js';
28
29
  export { ClassificationCrypto, DEFAULT_KEY_ID, isColumnEnvelope, parseColumnEnvelope, type ClassificationCryptoOptions, type ColumnEnvelope, type KEKResolver, type ResolvedKEK, } from './classification-crypto.js';
29
- export { agentSchema, analyticsSchema, auditSchema, channelSchema, credentialSchema, deploymentSchema, flagSchema, incomingWebhookSchema, scopeSchema, secretSchema, sessionSchema, virtualUserSchema, webhookSchema, workflowSchema, pikkuSchemas, requiredPikkuSchemas, applyPikkuSchemas, compilePikkuSchemas, requirePikkuSchema, resolveRequirements, type PikkuSchema, type RequiredTypes, type UnmetRequirement, type SchemaStatementFactory, } from './schema/index.js';
30
+ export { agentSchema, analyticsSchema, auditSchema, channelSchema, credentialSchema, deploymentSchema, flagSchema, incomingWebhookSchema, leaseSchema, scopeSchema, secretSchema, sessionSchema, virtualUserSchema, webhookSchema, workflowSchema, pikkuSchemas, requiredPikkuSchemas, applyPikkuSchemas, compilePikkuSchemas, requirePikkuSchema, resolveRequirements, type PikkuSchema, type RequiredTypes, type UnmetRequirement, type SchemaStatementFactory, } from './schema/index.js';
30
31
  export type { KyselyPikkuDB } from './kysely-tables.js';
31
32
  export type { WorkflowRunService } from '@pikku/core/workflow';
32
33
  export type { AgentRunService, AgentRunRow } from '@pikku/core/agent';
package/dist/src/index.js CHANGED
@@ -14,6 +14,7 @@ export { KyselyScopeService } from './kysely-scope-service.js';
14
14
  export { KyselyFeatureFlagStore } from './kysely-feature-flag-store.js';
15
15
  export { KyselyWebhookService } from './kysely-webhook-service.js';
16
16
  export { KyselyTriggerSourceStore } from './kysely-trigger-source-store.js';
17
+ export { KyselyLeaseService } from './kysely-lease-service.js';
17
18
  export { KyselyIncomingWebhookService } from './kysely-incoming-webhook-service.js';
18
19
  export { createAuditedKysely, } from './create-audited-kysely.js';
19
20
  export { KyselyAuditService } from './kysely-audit-service.js';
@@ -28,4 +29,4 @@ export { SerializePlugin, BaseSerializePlugin, } from './serialize-plugin.js';
28
29
  export { createCoercionPlugin, } from './coercion-plugin.js';
29
30
  export { createClassificationPlugin, } from './classification-plugin.js';
30
31
  export { ClassificationCrypto, DEFAULT_KEY_ID, isColumnEnvelope, parseColumnEnvelope, } from './classification-crypto.js';
31
- export { agentSchema, analyticsSchema, auditSchema, channelSchema, credentialSchema, deploymentSchema, flagSchema, incomingWebhookSchema, scopeSchema, secretSchema, sessionSchema, virtualUserSchema, webhookSchema, workflowSchema, pikkuSchemas, requiredPikkuSchemas, applyPikkuSchemas, compilePikkuSchemas, requirePikkuSchema, resolveRequirements, } from './schema/index.js';
32
+ export { agentSchema, analyticsSchema, auditSchema, channelSchema, credentialSchema, deploymentSchema, flagSchema, incomingWebhookSchema, leaseSchema, scopeSchema, secretSchema, sessionSchema, virtualUserSchema, webhookSchema, workflowSchema, pikkuSchemas, requiredPikkuSchemas, applyPikkuSchemas, compilePikkuSchemas, requirePikkuSchema, resolveRequirements, } from './schema/index.js';
@@ -0,0 +1,14 @@
1
+ import { type RawBuilder } from 'kysely';
2
+ /**
3
+ * Now, in epoch milliseconds, as a SQL expression: the clock every lease in
4
+ * this package is granted and judged on.
5
+ *
6
+ * A lease is compared by whichever worker reads it next, so judging it on each
7
+ * worker's own clock lets a worker running ahead take a lease that is still
8
+ * held. The dialect services read the database's clock instead, which every
9
+ * worker shares. This default binds the process clock, for SQLite, where the
10
+ * database lives in the process anyway.
11
+ */
12
+ export declare const appNowMs: () => RawBuilder<number>;
13
+ /** `now + ms`, for a lease that runs `ms` from the clock's now. */
14
+ export declare const leaseUntil: (now: RawBuilder<number>, ms: number) => RawBuilder<number>;
@@ -0,0 +1,14 @@
1
+ import { sql } from 'kysely';
2
+ /**
3
+ * Now, in epoch milliseconds, as a SQL expression: the clock every lease in
4
+ * this package is granted and judged on.
5
+ *
6
+ * A lease is compared by whichever worker reads it next, so judging it on each
7
+ * worker's own clock lets a worker running ahead take a lease that is still
8
+ * held. The dialect services read the database's clock instead, which every
9
+ * worker shares. This default binds the process clock, for SQLite, where the
10
+ * database lives in the process anyway.
11
+ */
12
+ export const appNowMs = () => sql `cast(${Date.now()} as bigint)`;
13
+ /** `now + ms`, for a lease that runs `ms` from the clock's now. */
14
+ export const leaseUntil = (now, ms) => sql `${now} + ${ms}`;
@@ -0,0 +1,37 @@
1
+ import type { Lease, LeaseService } from '@pikku/core/services';
2
+ import type { Kysely } from 'kysely';
3
+ import type { KyselyPikkuDB } from './kysely-tables.js';
4
+ /**
5
+ * Leases on the `pikku_lease` table.
6
+ *
7
+ * Use it directly on SQLite. On PostgreSQL and MySQL use
8
+ * `PgKyselyLeaseService` and `MySQLKyselyLeaseService`, which judge every lease
9
+ * on the database's clock rather than on each worker's own.
10
+ *
11
+ * Every statement is one MySQL can run too — no `RETURNING`, no conditional
12
+ * upsert — so the dialects differ only in `nowMs` and `insertIfAbsent`.
13
+ */
14
+ export declare class KyselyLeaseService implements LeaseService {
15
+ protected db: Kysely<KyselyPikkuDB>;
16
+ private initialized;
17
+ constructor(db: Kysely<KyselyPikkuDB>);
18
+ init(): Promise<void>;
19
+ /** Now, in epoch milliseconds, on the clock leases are judged by. */
20
+ protected nowMs(): import("kysely").RawBuilder<number>;
21
+ /** Create the key's row, lapsed, unless it already has one. */
22
+ protected insertIfAbsent(key: string, holder: string): Promise<void>;
23
+ /**
24
+ * The row is created lapsed and then taken by the same guarded `UPDATE` as
25
+ * any other lapsed lease, so there is one path to holding a key. `token` is
26
+ * assigned first: MySQL evaluates `SET` left to right against the values
27
+ * already assigned, and the holder test must see the previous holder.
28
+ *
29
+ * Whether the update won is read back rather than taken from its row count,
30
+ * because MySQL counts rows changed, not rows matched.
31
+ */
32
+ acquire(key: string, holder: string, ttlMs: number): Promise<Lease | null>;
33
+ refresh(lease: Lease, ttlMs: number): Promise<Lease | null>;
34
+ release(lease: Lease): Promise<void>;
35
+ get(key: string): Promise<Lease | null>;
36
+ private readHeld;
37
+ }
@@ -0,0 +1,116 @@
1
+ import { appNowMs, leaseUntil } from './kysely-lease-clock.js';
2
+ import { requirePikkuSchema } from './schema/index.js';
3
+ import { leaseSchema } from './schema/lease.schema.js';
4
+ const toLease = (row) => ({
5
+ key: row.key,
6
+ holder: row.holder,
7
+ token: Number(row.token),
8
+ expiresAt: new Date(Number(row.expiresAt)),
9
+ });
10
+ /**
11
+ * Leases on the `pikku_lease` table.
12
+ *
13
+ * Use it directly on SQLite. On PostgreSQL and MySQL use
14
+ * `PgKyselyLeaseService` and `MySQLKyselyLeaseService`, which judge every lease
15
+ * on the database's clock rather than on each worker's own.
16
+ *
17
+ * Every statement is one MySQL can run too — no `RETURNING`, no conditional
18
+ * upsert — so the dialects differ only in `nowMs` and `insertIfAbsent`.
19
+ */
20
+ export class KyselyLeaseService {
21
+ db;
22
+ initialized = false;
23
+ constructor(db) {
24
+ this.db = db;
25
+ }
26
+ async init() {
27
+ if (this.initialized)
28
+ return;
29
+ await requirePikkuSchema(this.db, leaseSchema);
30
+ this.initialized = true;
31
+ }
32
+ /** Now, in epoch milliseconds, on the clock leases are judged by. */
33
+ nowMs() {
34
+ return appNowMs();
35
+ }
36
+ /** Create the key's row, lapsed, unless it already has one. */
37
+ async insertIfAbsent(key, holder) {
38
+ await this.db
39
+ .insertInto('pikkuLease')
40
+ .values({ key, holder, token: 0, expiresAt: 0 })
41
+ .onConflict((oc) => oc.column('key').doNothing())
42
+ .execute();
43
+ }
44
+ /**
45
+ * The row is created lapsed and then taken by the same guarded `UPDATE` as
46
+ * any other lapsed lease, so there is one path to holding a key. `token` is
47
+ * assigned first: MySQL evaluates `SET` left to right against the values
48
+ * already assigned, and the holder test must see the previous holder.
49
+ *
50
+ * Whether the update won is read back rather than taken from its row count,
51
+ * because MySQL counts rows changed, not rows matched.
52
+ */
53
+ async acquire(key, holder, ttlMs) {
54
+ await this.insertIfAbsent(key, holder);
55
+ const now = this.nowMs();
56
+ await this.db
57
+ .updateTable('pikkuLease')
58
+ .set((eb) => ({
59
+ token: eb
60
+ .case()
61
+ .when(eb.and([eb('holder', '=', holder), eb('expiresAt', '>', now)]))
62
+ .then(eb.ref('token'))
63
+ .else(eb('token', '+', 1))
64
+ .end(),
65
+ holder,
66
+ expiresAt: leaseUntil(now, ttlMs),
67
+ }))
68
+ .where('key', '=', key)
69
+ .where((eb) => eb.or([eb('holder', '=', holder), eb('expiresAt', '<=', now)]))
70
+ .execute();
71
+ return this.readHeld(key, holder);
72
+ }
73
+ async refresh(lease, ttlMs) {
74
+ const now = this.nowMs();
75
+ await this.db
76
+ .updateTable('pikkuLease')
77
+ .set({ expiresAt: leaseUntil(now, ttlMs) })
78
+ .where('key', '=', lease.key)
79
+ .where('holder', '=', lease.holder)
80
+ .where('token', '=', lease.token)
81
+ .where('expiresAt', '>', now)
82
+ .execute();
83
+ return this.readHeld(lease.key, lease.holder, lease.token);
84
+ }
85
+ async release(lease) {
86
+ await this.db
87
+ .updateTable('pikkuLease')
88
+ .set({ expiresAt: 0 })
89
+ .where('key', '=', lease.key)
90
+ .where('holder', '=', lease.holder)
91
+ .where('token', '=', lease.token)
92
+ .execute();
93
+ }
94
+ async get(key) {
95
+ const row = await this.db
96
+ .selectFrom('pikkuLease')
97
+ .selectAll()
98
+ .where('key', '=', key)
99
+ .where('expiresAt', '>', this.nowMs())
100
+ .executeTakeFirst();
101
+ return row ? toLease(row) : null;
102
+ }
103
+ async readHeld(key, holder, token) {
104
+ let query = this.db
105
+ .selectFrom('pikkuLease')
106
+ .selectAll()
107
+ .where('key', '=', key)
108
+ .where('holder', '=', holder)
109
+ .where('expiresAt', '>', this.nowMs());
110
+ if (token !== undefined) {
111
+ query = query.where('token', '=', token);
112
+ }
113
+ const row = await query.executeTakeFirst();
114
+ return row ? toLease(row) : null;
115
+ }
116
+ }
@@ -46,6 +46,13 @@ export interface WorkflowStepTable {
46
46
  fromStepName: string | null;
47
47
  /** The history attempt currently in flight; see WorkflowStepHistoryTable.attempt. */
48
48
  currentAttempt: number | null;
49
+ /**
50
+ * When the dispatch holding a `running` step stops owning it. Null is a step
51
+ * nothing is expected to hand back — one parked on a child run, or written
52
+ * before it was ever claimed. Epoch milliseconds, on the clock leases are
53
+ * judged by; see `appNowMs`.
54
+ */
55
+ leaseExpiresAt: number | null;
49
56
  createdAt: Generated<Date>;
50
57
  updatedAt: Generated<Date>;
51
58
  }
@@ -250,6 +257,14 @@ export interface PikkuTriggerSourceTable {
250
257
  detail: string | null;
251
258
  updatedAt: Generated<Date>;
252
259
  }
260
+ /** A named lease, current or last. */
261
+ export interface PikkuLeaseTable {
262
+ key: string;
263
+ holder: string;
264
+ token: number;
265
+ /** Epoch milliseconds on the clock leases are judged by; see `appNowMs`. */
266
+ expiresAt: number;
267
+ }
253
268
  /** One subject's short-circuit, in either direction, over a flag's rollout. */
254
269
  export interface PikkuFeatureFlagOverridesTable {
255
270
  flag: string;
@@ -436,6 +451,7 @@ export interface KyselyPikkuDB {
436
451
  pikkuFeatureFlags: PikkuFeatureFlagsTable;
437
452
  pikkuFeatureFlagOverrides: PikkuFeatureFlagOverridesTable;
438
453
  pikkuTriggerSource: PikkuTriggerSourceTable;
454
+ pikkuLease: PikkuLeaseTable;
439
455
  channels: ChannelsTable;
440
456
  channelSubscriptions: ChannelSubscriptionsTable;
441
457
  workflowRuns: WorkflowRunsTable;
@@ -1,13 +1,13 @@
1
1
  import type { SerializedError } from '@pikku/core/errors';
2
2
  import { PikkuWorkflowService } from '@pikku/core/workflow';
3
- import type { WorkflowPlannedStep, WorkflowQueueOptions, WorkflowRun, WorkflowRunWire, StepState, StepStatus, WorkflowStatus, WorkflowVersionStatus } from '@pikku/core/workflow';
3
+ import type { WorkflowPlannedStep, WorkflowServiceOptions, WorkflowRun, WorkflowRunWire, StepState, StepStatus, WorkflowStatus, WorkflowVersionStatus } from '@pikku/core/workflow';
4
4
  import { type Kysely } from 'kysely';
5
5
  import type { KyselyPikkuDB } from './kysely-tables.js';
6
6
  export declare class KyselyWorkflowService extends PikkuWorkflowService {
7
7
  protected db: Kysely<KyselyPikkuDB>;
8
8
  private initialized;
9
9
  private runService;
10
- constructor(db: Kysely<KyselyPikkuDB>, options?: WorkflowQueueOptions);
10
+ constructor(db: Kysely<KyselyPikkuDB>, options: WorkflowServiceOptions);
11
11
  init(): Promise<void>;
12
12
  protected createRunImpl(workflowName: string, input: any, inline: boolean, graphHash: string, wire: WorkflowRunWire, options?: {
13
13
  deterministic?: boolean;
@@ -23,10 +23,18 @@ export declare class KyselyWorkflowService extends PikkuWorkflowService {
23
23
  protected listStepStates(runId: string): Promise<Array<StepState & {
24
24
  stepName: string;
25
25
  }>>;
26
+ getRunSteps(runId: string): Promise<Array<StepState & {
27
+ stepName: string;
28
+ rpcName?: string;
29
+ data?: any;
30
+ }>>;
26
31
  getRunHistory(runId: string): Promise<Array<StepState & {
27
32
  stepName: string;
28
33
  }>>;
29
34
  protected setStepRunningImpl(stepId: string): Promise<void>;
35
+ refreshStepLease(stepId: string, leaseMs: number | null, attempt?: number): Promise<boolean>;
36
+ /** Now, in epoch milliseconds, on the clock step leases are judged by. */
37
+ protected nowMs(): import("kysely").RawBuilder<number>;
30
38
  protected setStepScheduledImpl(stepId: string): Promise<void>;
31
39
  /**
32
40
  * Move a step and its current history attempt to the same status in one
@@ -43,21 +51,10 @@ export declare class KyselyWorkflowService extends PikkuWorkflowService {
43
51
  private insertHistoryRecord;
44
52
  private getTimestampFieldForStatus;
45
53
  protected setStepChildRunIdImpl(stepId: string, childRunId: string): Promise<void>;
46
- protected setStepResultImpl(stepId: string, result: any): Promise<void>;
47
- protected setStepErrorImpl(stepId: string, error: Error): Promise<void>;
54
+ protected setStepResultImpl(stepId: string, result: any, attempt?: number): Promise<void>;
55
+ protected setStepErrorImpl(stepId: string, error: Error, attempt?: number): Promise<void>;
48
56
  protected createRetryAttemptImpl(stepId: string, status: 'pending' | 'running'): Promise<StepState>;
49
- withRunLock<T>(_id: string, fn: () => Promise<T>): Promise<T>;
50
- /**
51
- * A pass-through, and deliberately so: the one decision that must exclude —
52
- * claiming a step to execute it — is made by `claimStepForExecution` below as
53
- * a single conditional write, which needs no lock to be exclusive.
54
- *
55
- * What is left are the suspend and approval sections in the engine, where the
56
- * lock is genuine mutual exclusion rather than a claim. A dialect with a real
57
- * primitive overrides this to cover them — `kysely-postgres` with
58
- * `pg_advisory_xact_lock`, `kysely-mysql` with `GET_LOCK`.
59
- */
60
- withStepLock<T>(_runId: string, _stepName: string, fn: () => Promise<T>): Promise<T>;
57
+ private readStepAttempt;
61
58
  /**
62
59
  * Claim the step with a status-guarded `UPDATE` and read the affected-row
63
60
  * count: the database decides the winner in one statement, so two dispatches
@@ -70,11 +67,29 @@ export declare class KyselyWorkflowService extends PikkuWorkflowService {
70
67
  *
71
68
  * The winner then goes through the ordinary transition methods, so history
72
69
  * rows and the mirror see exactly what they saw before.
70
+ *
71
+ * A `running` step is taken back only once its lease has lapsed, and the
72
+ * lease is part of the same guarded `UPDATE` — checking it here and writing
73
+ * it afterwards would let two dispatches read one lapsed lease and both
74
+ * proceed. Whether it has lapsed is the database's call, on its own clock:
75
+ * a live lease simply matches no row.
76
+ */
77
+ protected claimStepForExecution(runId: string, stepName: string, rpcName: string, leaseMs: number): Promise<StepState | null>;
78
+ /**
79
+ * Take back a failed or lapsed step as a new attempt. The attempt number is
80
+ * advanced by the same guarded `UPDATE` that grants the lease, so the worker
81
+ * it is taken from is fenced out of its step at once — deferring that to
82
+ * `createRetryAttempt` would leave a window in which its result still lands,
83
+ * only to be cleared and the step run again.
73
84
  */
74
- protected claimStepForExecution(runId: string, stepName: string, rpcName: string): Promise<StepState | null>;
85
+ private reclaimStepAttempt;
75
86
  /**
76
- * Move a step to `running` only if it is still in one of `from`, reporting
77
- * whether this caller is the one that moved it.
87
+ * Move a step to `running` under a fresh lease, only if it is still in one of
88
+ * `from`, reporting whether this caller is the one that moved it.
89
+ *
90
+ * Claiming back a `running` step additionally requires its lease to have
91
+ * lapsed. A null lease never matches, which is what keeps a step parked on a
92
+ * child run — running, with no worker on it by design — out of reach.
78
93
  */
79
94
  private claimStepStatus;
80
95
  getCompletedGraphState(runId: string): Promise<{
@@ -86,6 +101,7 @@ export declare class KyselyWorkflowService extends PikkuWorkflowService {
86
101
  stepName: string;
87
102
  status: StepStatus;
88
103
  fromStepName?: string;
104
+ leaseExpiresAt?: Date;
89
105
  }>>;
90
106
  getNodeResults(runId: string, nodeIds: string[]): Promise<Record<string, any>>;
91
107
  protected setBranchTakenImpl(stepId: string, branchKey: string): Promise<void>;
@@ -113,7 +129,7 @@ export declare class KyselyWorkflowService extends PikkuWorkflowService {
113
129
  * The undispatched-step query, shared by every dialect.
114
130
  *
115
131
  * It used to be deliberately NOT an override, because this class's
116
- * `withStepLock` is a pass-through: a subclass inheriting it (kysely-sqlite)
132
+ * `withStepLock` was a pass-through: a subclass inheriting it (kysely-sqlite)
117
133
  * had no atomic claim, so the relay's redundant dispatches would have become
118
134
  * double executions, and only `kysely-postgres` and `kysely-mysql` opted in
119
135
  * on the strength of their real locks. `claimStepForExecution` no longer