@sanoma-ai/workflows 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +72 -21
- package/dist/apply-steps.d.ts +6 -0
- package/dist/apply-steps.js +93 -0
- package/dist/apply-steps.js.map +1 -0
- package/dist/apply.d.ts +197 -0
- package/dist/apply.js +470 -0
- package/dist/apply.js.map +1 -0
- package/dist/call.d.ts +4 -0
- package/dist/call.js +19 -4
- package/dist/call.js.map +1 -1
- package/dist/client.d.ts +38 -2
- package/dist/client.js +67 -10
- package/dist/client.js.map +1 -1
- package/dist/config.d.ts +3 -3
- package/dist/config.js +33 -16
- package/dist/config.js.map +1 -1
- package/dist/credentials.d.ts +25 -0
- package/dist/credentials.js +73 -5
- package/dist/credentials.js.map +1 -1
- package/dist/datafile.d.ts +11 -2
- package/dist/datafile.js +56 -2
- package/dist/datafile.js.map +1 -1
- package/dist/describe.d.ts +11 -5
- package/dist/describe.js +13 -5
- package/dist/describe.js.map +1 -1
- package/dist/drift.d.ts +64 -12
- package/dist/drift.js +118 -41
- package/dist/drift.js.map +1 -1
- package/dist/expectations.d.ts +2 -2
- package/dist/expectations.js +28 -3
- package/dist/expectations.js.map +1 -1
- package/dist/host.d.ts +56 -0
- package/dist/host.js +45 -0
- package/dist/host.js.map +1 -0
- package/dist/index.d.ts +6 -2
- package/dist/index.js +5 -1
- package/dist/index.js.map +1 -1
- package/dist/lockfile.d.ts +36 -0
- package/dist/lockfile.js +69 -0
- package/dist/lockfile.js.map +1 -0
- package/dist/op.d.ts +3 -0
- package/dist/op.js.map +1 -1
- package/dist/resource.d.ts +188 -22
- package/dist/resource.js +173 -28
- package/dist/resource.js.map +1 -1
- package/dist/resources.d.ts +18 -3
- package/dist/resources.js +53 -6
- package/dist/resources.js.map +1 -1
- package/dist/run.d.ts +14 -0
- package/dist/run.js +6 -1
- package/dist/run.js.map +1 -1
- package/dist/sandbox.js +5 -1
- package/dist/sandbox.js.map +1 -1
- package/dist/scenario.d.ts +12 -3
- package/dist/scenario.js +33 -8
- package/dist/scenario.js.map +1 -1
- package/dist/secret.d.ts +42 -0
- package/dist/secret.js +58 -0
- package/dist/secret.js.map +1 -0
- package/dist/shared.d.ts +8 -0
- package/dist/shared.js +7 -0
- package/dist/shared.js.map +1 -1
- package/dist/source.d.ts +58 -0
- package/dist/source.js +54 -0
- package/dist/source.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/dist/worker.d.ts +3 -0
- package/dist/worker.js +13 -4
- package/dist/worker.js.map +1 -1
- package/dist/writer.d.ts +35 -0
- package/dist/writer.js +201 -0
- package/dist/writer.js.map +1 -0
- package/package.json +6 -1
- package/src/apply-steps.ts +110 -0
- package/src/apply.ts +626 -0
- package/src/call.ts +19 -4
- package/src/client.ts +83 -13
- package/src/config.ts +48 -25
- package/src/credentials.ts +90 -5
- package/src/datafile.ts +76 -4
- package/src/describe.ts +36 -16
- package/src/drift.ts +155 -45
- package/src/expectations.ts +30 -4
- package/src/host.ts +96 -0
- package/src/index.ts +24 -1
- package/src/lockfile.ts +95 -0
- package/src/op.ts +3 -0
- package/src/resource.ts +270 -47
- package/src/resources.ts +63 -8
- package/src/run.ts +15 -0
- package/src/sandbox.ts +5 -1
- package/src/scenario.ts +50 -13
- package/src/secret.ts +92 -0
- package/src/shared.ts +10 -0
- package/src/source.ts +82 -0
- package/src/version.ts +1 -1
- package/src/worker.ts +16 -5
- package/src/writer.ts +225 -0
package/README.md
CHANGED
|
@@ -80,7 +80,7 @@ That is the whole config; the rest is convention, from its `root` (`defineConfig
|
|
|
80
80
|
|
|
81
81
|
The [data files](#data-files) are `resources/` in the root. `ledger` and `databaseUrl` are the only other fields. `drivers`, `fakes`, `appName` and `scenarios` are typed `never` on `SanomaConfig`, each deprecated with where it went, so the compiler and the editor say so; an untyped config that still names `drivers` or `fakes` is refused at run time rather than run live on what it no longer reads.
|
|
82
82
|
|
|
83
|
-
A config must name its policy. `policy: allowAll` allows every operation call, and says so. The audit record goes to the ledger, by default `postgresLedger(databaseUrl)` on the config's database (see [Ledger](#ledger)); `ledger: jsonlLedger(dir)` keeps it in files instead, and `memoryLedger()` in this process only, for tests. `resolveConfig(config)` checks a config and derives what the runtime uses from it: the app name, database, version and queue (`sanoma:<appName>`, for every run, [sandbox runs](#scenarios-and-sandbox-runs) included), the operations by id, the driver each vendor's connector attaches, made once for the resolution (`driversByVendor`, in `connectors` order, and their functions by operation id, `drivers`), the makers of their fakes (`fakes`, by vendor), the scenarios directory, and the config's `ledger` when it gives one; it opens nothing and reads no file but the root's `package.json` and whether `scenarios/` is a directory. It throws, with one message, whatever the worker would refuse: a workflow or driver naming an operation the `connectors` don't declare, a workflow redeclaring an operation with another effect, an operation with no driver (`` `shop.order.refund` has no driver: the connector package must attach one ``), two different drivers or fakes for one vendor, two workflows with one name, no policy, a `ledger` that is not a store, a `drivers` or `fakes` an untyped config still names. `startWorker`, `SanomaClient.connect` and `describeConfig` all call it, so they fail the same way. It adds the built-in [`drift`](#drift)
|
|
83
|
+
A config must name its policy. `policy: allowAll` allows every operation call, and says so. The audit record goes to the ledger, by default `postgresLedger(databaseUrl)` on the config's database (see [Ledger](#ledger)); `ledger: jsonlLedger(dir)` keeps it in files instead, and `memoryLedger()` in this process only, for tests. `resolveConfig(config)` checks a config and derives what the runtime uses from it: the app name, database, version and queue (`sanoma:<appName>`, for every run, [sandbox runs](#scenarios-and-sandbox-runs) included), the operations by id, the driver each vendor's connector attaches, made once for the resolution (`driversByVendor`, in `connectors` order, and their functions by operation id, `drivers`), the makers of their fakes (`fakes`, by vendor), the scenarios directory, and the config's `ledger` when it gives one; it opens nothing and reads no file but the root's `package.json` and whether `scenarios/` is a directory. It throws, with one message, whatever the worker would refuse: a workflow or driver naming an operation the `connectors` don't declare, a workflow redeclaring an operation with another effect, an operation with no driver (`` `shop.order.refund` has no driver: the connector package must attach one ``), two different drivers or fakes for one vendor, two workflows with one name, no policy, a `ledger` that is not a store, a `drivers` or `fakes` an untyped config still names. `startWorker`, `SanomaClient.connect` and `describeConfig` all call it, so they fail the same way. It adds the built-in [`drift`](#drift) and [`apply`](#apply) workflows when the connectors declare resource types. The worker takes each operation's effect, schemas and retry setting from `connectors`, never from the workflow.
|
|
84
84
|
|
|
85
85
|
A policy returns `allow()`, `deny(reason)` (the run fails) or `approve(who)` (the run waits for that person, or for anyone in `{ group }`; see [Approvals](#approvals)). `allow(reasons)` and `deny(reason, reasons)` may add a list of reasons for whoever reads the ledger. The policy must decide the same way on every replay: no clock, randomness or network. What it sees is plain data, so a test can build one by hand, with `policyOpOf(op)` for the operation as the runtime passes it:
|
|
86
86
|
|
|
@@ -132,7 +132,7 @@ const run = await client.run(runId); // run.status: "queued" | "running" | "wait
|
|
|
132
132
|
|
|
133
133
|
`workflow` is a definition or the name of one in the config's `workflows` (`client.start("refund", …)`); a name the config does not have is refused with `invalid_input`, naming the ones it has. `startedBy` is required. `start(workflow, input, { startedBy, runId })` with a `runId` makes a retried start idempotent: an id that exists returns that run when the workflow, the input (as JSON) and `startedBy` match, and is refused with `invalid_input`, naming what differs, when they do not. Of two starts racing with one new id, the first stands and the second is answered the same way. Errors the runtime and the client throw carry a `code` (`policy_denied`, `approval_rejected`, `not_approver`, `no_pending_approval`, `already_decided`, `run_not_found`, `driver_failed`, `invalid_input`, `run_ended`, `run_running`: `client.result` timed out with the run still going) and `data`. Read it with `errorCode(err)`, not `instanceof`: a run's error comes back from the database as a copy, so `errorCode(await client.result(runId).catch((e) => e))` is `"policy_denied"` for a denied call.
|
|
134
134
|
|
|
135
|
-
`startWorker(config, { logLevel })` runs workflows and recovers interrupted runs. `SanomaClient` starts runs, lists them, records approval decisions and reads a run's ledger. `describeConfig(config)`, from `@sanoma-ai/workflows/describe`, returns the same config as plain JSON (its version, each workflow's input as JSON Schema, the operations it may call and its [outline](#outline), each operation's effect and contract: its input as a caller sends it, `io: "input"`, and its output as parsed, `io: "output"`), each vendor's title, logo, package and homepage and whether its connector attaches a fake (`vendors`, by vendor id, the logo as `data:image/svg+xml` URLs), the resource types (`resourceTypes`, see [Resources](#resources)), and the resources the data files declare with what is wrong in them (`resources` and `problems`, see [Data files](#data-files); a workflow's file's problems are on its entry, beside its `file` and `outline`), which is what a UI renders from.
|
|
135
|
+
`startWorker(config, { logLevel })` runs workflows and recovers interrupted runs. `SanomaClient` starts runs, lists them, records approval decisions and reads a run's ledger. `describeConfig(config)`, from `@sanoma-ai/workflows/describe`, returns the same config as plain JSON (its version, each workflow's input as JSON Schema, the operations it may call and its [outline](#outline), each operation's effect and contract: its input as a caller sends it, `io: "input"`, and its output as parsed, `io: "output"`), each vendor's title, logo, package and homepage and whether its connector attaches a fake (`vendors`, by vendor id, the logo as `data:image/svg+xml` URLs), the resource types (`resourceTypes`, see [Resources](#resources)), and the resources the data files declare, each with its binding, the lockfile's orphans and what is wrong in them (`resources`, `orphans` and `problems`, see [Data files](#data-files); a workflow's file's problems are on its entry, beside its `file` and `outline`), which is what a UI renders from.
|
|
136
136
|
|
|
137
137
|
The worker, the client and the app find Postgres at the config's `databaseUrl`, else the `SANOMA_DATABASE_URL` environment variable, else `postgresql://postgres:dbos@localhost:5433/sanoma`, the database `pnpm db:up` starts from this repo's docker compose file. Set one of the first two anywhere but on your own machine.
|
|
138
138
|
|
|
@@ -186,7 +186,7 @@ Every call is a step with a name, which a test and the graph name it by: `ctx.gi
|
|
|
186
186
|
|
|
187
187
|
### Drivers
|
|
188
188
|
|
|
189
|
-
A driver implements a connector's operations with `defineDriver(connector, { resource: { name: (input, call) => … } })`, typed by the connector's schemas, and its package attaches it to the connector it exports (`withSetup`). `call` carries `idempotencyKey`, the same on every retry and replay of one call: pass it to the vendor (or dedupe on it) so a crash between the vendor's reply and the checkpoint does not repeat the side effect. A driver reads its credentials when it is called, never from the config, and declares them (see [Credentials](#credentials)).
|
|
189
|
+
A driver implements a connector's operations with `defineDriver(connector, { resource: { name: (input, call) => … } })`, typed by the connector's schemas, and its package attaches it to the connector it exports (`withSetup`). `call` carries `idempotencyKey`, the same on every retry and replay of one call: pass it to the vendor (or dedupe on it) so a crash between the vendor's reply and the checkpoint does not repeat the side effect. It carries `secrets` too, the store `secret()` markers name (see [Secrets](#secrets)). A driver reads its credentials when it is called, never from the config, and declares them (see [Credentials](#credentials)).
|
|
190
190
|
|
|
191
191
|
When the vendor says no, throw a `DriverError(message, { retryable, status?, vendorCode? })`. An operation declared `idempotent` is tried up to three times (after 1 and 2 seconds) unless the error says `retryable: false` or the reply fails the output schema; any other operation is tried once. When the tries run out, the run fails with the last try's error, not a wrapper: for a `DriverError`, `errorCode(err)` is `"driver_failed"` and the ledger records the vendor's message, status and code. Anything else a driver throws (a `TypeError`, a plain `Error`) fails the run as it is, with no code.
|
|
192
192
|
|
|
@@ -194,7 +194,7 @@ When the vendor says no, throw a `DriverError(message, { retryable, status?, ven
|
|
|
194
194
|
|
|
195
195
|
A driver reads its credentials from the environment when it is called, and declares them in `defineDriver`'s third argument: `defineDriver(acme, impl, { env: z.object({ ACME_API_KEY: z.string().describe("an API key, from Settings, API keys"), ACME_REGION: z.enum(["us", "eu"]).optional().describe("the region; default us") }) })`. Each key is a variable's name, its `.describe()` the one-line manual, `.optional()` marks one that may be unset. The driver reads the raw environment, so `.default()` and `.transform()` are not applied to what it reads (use `.optional()` and name the default in `.describe()`), and a `.refine()` or `.superRefine()` on the `z.object` is not evaluated, only its fields. Only the declared names are read, an empty value counts as unset, and a value is never kept: each variable is set, missing or invalid, with the schema's reason.
|
|
196
196
|
|
|
197
|
-
The environment stays the only interface a driver sees, but a value need not come from a file. Locally, set values live in the `sanoma_credentials` table in the config's database (`databaseUrl`), which the runtime creates on first use. Set one from the app's connector page, or with `client.setCredential("GHOST_ADMIN_API_KEY", value, { by: "alice" })`, and delete it with `client.clearCredential(name, { by })`; `client.credentials()` gives each vendor's statuses, each with its `source` (`environment` or `stored`) and, for a stored one, who set it and when. A name no driver declares, or a value its schema refuses, is refused with the reason, never the value. `startWorker` loads the stored values of the declared variables its environment leaves unset into `process.env` and listens for changes, so a value set or cleared is live without a restart: drivers read `process.env` when they are called. A worker whose listener drops reconnects and loads them again. A variable the process environment sets wins over a stored one, always. The worker starts whether or not every variable is there: it prints one line naming each vendor with a variable missing (and not optional) or invalid, `sanoma: 2 of 5 connectors not configured yet: ghost, resend; set their variables on the connector pages or in the environment` (`unconfigured` names them too; `quiet: true` prints nothing, as `runScenarios` does, its runs calling only the fakes). A call to that vendor then fails, as the call's own failure: a `DriverError` that is not retried, saying why, `<vendor> is not configured: <NAME> (<description>), <NAME> is invalid: <why>; set it on the connector page` (or `set in the environment`, when an invalid value is there: the environment wins over a stored value), checked against the environment at the moment of the call, so a value set on the connector page (or in `process.env`) takes effect on the next call, with no restart. The call's `op.called` record carries it, and the run fails with it like any other final driver failure. `startApp` starts the same way, and its connector pages show and set them.
|
|
197
|
+
The environment stays the only interface a driver sees, but a value need not come from a file. Locally, set values live in the `sanoma_credentials` table in the config's database (`databaseUrl`), which the runtime creates on first use, encrypted with AES-256-GCM under `SANOMA_SECRETS_KEY`, bound to its name (a value moved to another row does not decrypt) (32 random bytes, base64; `sanoma` writes one to `.env` when there is none): storing a value without the key fails, naming it. A value stored before encryption is read as it is, and encrypted on its next write. Set one from the app's connector page, or with `client.setCredential("GHOST_ADMIN_API_KEY", value, { by: "alice" })`, and delete it with `client.clearCredential(name, { by })`; `client.credentials()` gives each vendor's statuses, each with its `source` (`environment` or `stored`) and, for a stored one, who set it and when. A name no driver declares, or a value its schema refuses, is refused with the reason, never the value. `client.setSecret(name, value, { by })` stores a value under any variable's name, declared or not (a data file's `secret()`, or the app's own, such as its GitHub App's keys), checking a declared one as `setCredential` does, and `client.secrets(names)` reads values back for server code, the environment's first, with their `source` and who stored them. `startWorker` loads the stored values of the declared variables its environment leaves unset into `process.env` and listens for changes, so a value set or cleared is live without a restart: drivers read `process.env` when they are called. A worker whose listener drops reconnects and loads them again. A variable the process environment sets wins over a stored one, always. The worker starts whether or not every variable is there: it prints one line naming each vendor with a variable missing (and not optional) or invalid, `sanoma: 2 of 5 connectors not configured yet: ghost, resend; set their variables on the connector pages or in the environment` (`unconfigured` names them too; `quiet: true` prints nothing, as `runScenarios` does, its runs calling only the fakes). A call to that vendor then fails, as the call's own failure: a `DriverError` that is not retried, saying why, `<vendor> is not configured: <NAME> (<description>), <NAME> is invalid: <why>; set it on the connector page` (or `set in the environment`, when an invalid value is there: the environment wins over a stored value), checked against the environment at the moment of the call, so a value set on the connector page (or in `process.env`) takes effect on the next call, with no restart. The call's `op.called` record carries it, and the run fails with it like any other final driver failure. `startApp` starts the same way, and its connector pages show and set them.
|
|
198
198
|
|
|
199
199
|
The values are plain text in the local database: the same exposure as a `.env` file on the same disk. Keep `.env` (`node --env-file-if-exists=.env`) for the bootstrap values a worker needs before it can read the database: the database URL, the bridge binary, the port. A value never reaches the ledger or the description; see [Secrets](#secrets) for why one must never be an input either.
|
|
200
200
|
|
|
@@ -209,22 +209,28 @@ export const repository = defineResource({
|
|
|
209
209
|
vendor: "github",
|
|
210
210
|
type: "repository",
|
|
211
211
|
title: "Repository",
|
|
212
|
-
identity: "name", // how `find` makes the
|
|
212
|
+
identity: "name", // optional, with `find`: how `find` makes the identity, for people
|
|
213
213
|
schema, // a z.object: the fields a read returns, and a data file declares (less the vendor-owned ones)
|
|
214
|
-
fields: { immutable: [], vendorOwned: ["html_url"], writeOnly: [], unordered: ["topics"] }, // and `references`, below
|
|
215
|
-
find: ({ name }) => name, // the
|
|
214
|
+
fields: { immutable: [], vendorOwned: ["html_url", "node_id"], writeOnly: [], unordered: ["topics"] }, // and `references`, below
|
|
215
|
+
find: ({ name }) => name, // optional: the import id that finds a declared resource before it is bound
|
|
216
|
+
bindingId: (state) => String(state.node_id), // optional: the stable id a binding keeps; default `String(state.id)`
|
|
216
217
|
normalize, // optional: the part of a state a drift check compares; default `compareDeclared`
|
|
218
|
+
writable: true, // optional: apply may change its objects (the change operations, below)
|
|
219
|
+
stateless: true, // optional: its objects hold no data, so apply makes one gone again without asking
|
|
220
|
+
afterCreate: [{ op: "acme.repository.protect", input: ({ id, export: name }) => ({ id }), until: (out) => out.ok }], // optional
|
|
217
221
|
});
|
|
218
222
|
export const github = defineConnector("github", { repository });
|
|
219
223
|
```
|
|
220
224
|
|
|
221
|
-
A resource type is a connector's group under its own `type`; `defineConnector` refuses one of another vendor, or under another key, or on a connector without `info.package` (data files import the types from `<package>/resources`), and keeps the list on the connector, beside its vendor info. Its operations are `<vendor>.<type>.import` (`{ id }`,
|
|
225
|
+
A resource type is a connector's group under its own `type`; `defineConnector` refuses one of another vendor, or under another key, or on a connector without `info.package` (data files import the types from `<package>/resources`), and keeps the list on the connector, beside its vendor info. Its operations are `<vendor>.<type>.import` (`{ id }`, an identity or a binding id) and `<vendor>.<type>.read` (`{ id, state?, handle? }`, a state from an earlier call), both answering `{ id, gone: true }` when the vendor has no such object and `{ id, gone: false, state, handle? }` when it has, both effect `read`, idempotent, with the id as the policy's `target`. "Missing" is part of the contract: the driver decides it, so a caller never parses a vendor's error for it. A driver implements them like any other. `handle` is the driver's own data about the object, opaque to everyone else, passed back unchanged (an OpenTofu driver keeps the provider's private data and state version in it); the operations declare it `opaque`, so the ledger records it as `"<handle>"`, never its bytes. Any operation may name such fields (`opaque` in its spec).
|
|
222
226
|
|
|
223
|
-
|
|
227
|
+
A `writable` type has four change operations besides, which the built-in apply workflow (`APPLY_WORKFLOW`, `"apply"`, a name a config's own workflow may not take) calls and no other: `resolveConfig` refuses a workflow whose `uses` names `create`, `update` or `delete` of a declared type, and a run of any other workflow is refused the call. `plan` (effect `read`, idempotent) takes `{ id?, state?, handle?, desired? }` (no `id` for a create, no `desired` for a delete) and returns `{ action, planned?, unknown, replace }`: `none`, `create`, `update`, `replace` or `delete`, the new state without write-only fields, the dotted paths known only after apply, and those whose change replaces the object. `create` (effect `write`, `target` `new:<vendor>.<type>`) takes `{ desired }` and returns `{ id, state, handle? }`, `id` its binding id; `update` takes `{ id, state?, handle?, desired, expect? }` and returns the same, refusing with a `DriverError` whose `vendorCode` is `plan_changed` when the fresh plan's `action` or `replace` differs from `expect`; `delete` (idempotent) takes `{ id, state?, handle? }` and returns `{ id, gone: true }`, an object already gone included. `desired` may hold secret markers at write-only fields, which the driver resolves (`resolveSecrets`) just before the vendor call (`plan` stores no generated secret: one not stored yet gets a throwaway value); no output holds a write-only value.
|
|
228
|
+
|
|
229
|
+
Calling the type declares one resource, which is what a data file does: `repository({ name: "sanoma" })` returns a frozen `Declared`, `{ vendor: "github", type: "repository", name: "sanoma", desired: { name: "sanoma" }, refs: {} }`, branded with its `<vendor>.<type>` under the `DECLARED` symbol. It refuses a field the schema does not have, a field the vendor owns (unless `identity` names it), a value the schema rejects, or an empty identity. A type without `find` (Stripe's, whose objects Stripe names) declares resources without a `name`: they are found only by their bindings ([below](#bindings)).
|
|
224
230
|
|
|
225
231
|
A field may name another declared resource where the type's `fields.references` says so, by dotted path, with the type it names and, in `by`, the fields of that resource the vendor may hold there instead: GitHub's `branch_protection` has `{ repository_id: { type: "github.repository", by: ["name", "node_id"] } }`, since GitHub holds a rule's repository by its node id. So `branch_protection({ repository_id: site, pattern: "main" })` takes the repository `site`. It stands for that resource's `name`: `desired` holds `repository_id: "website"`, which the schema and `find` (the rule is `website:main`) read, and a drift check takes any of the repository's `by` values, read in the same run, as that name; and `refs` says where a resource was named, `{ repository_id: "github.repository:website" }` (a list item's path has its index, `teams.0`). Anywhere else, or of another type, a resource is refused, and the constructor's type (`Declaring`) says the same, so the editor marks it first.
|
|
226
232
|
|
|
227
|
-
Only declared fields are compared for drift, as CloudFormation does: `compareDeclared(fields, state, desired)` keeps the fields `desired` declares, minus `vendorOwned` and `writeOnly` ones, picking objects field by field, list items against the declared item at the same index, and the items of an `unordered` list (a set) against what any declared item declares, sorted, so a set is compared as a multiset whatever order the vendor returns it in. A drift check compares `normalize(actual, desired)` with `normalize(desired, desired)`, leaf by leaf: `diffDeclared(resource, state, desired, accept?)` returns each declared field that differs, `{ path, desired, actual }`, where `accept` gives, by a reference's path, the values that stand for the resource it names. `withoutWriteOnly(fields, desired)` drops write-only values from a declaration. So a field left out is never drift, which covers attributes an OpenTofu provider marks `computed` and `optional` (GitHub's `etag`, `topics`): they are the user's to set, so not `vendorOwned`, but the vendor fills them in when nobody does.
|
|
233
|
+
Only declared fields are compared for drift, as CloudFormation does: `compareDeclared(fields, state, desired)` keeps the fields `desired` declares, minus `vendorOwned` and `writeOnly` ones, picking objects field by field, list items against the declared item at the same index, and the items of an `unordered` list (a set) against what any declared item declares, sorted, so a set is compared as a multiset whatever order the vendor returns it in. A drift check compares `normalize(actual, desired)` with `normalize(desired, desired)`, leaf by leaf: `diffDeclared(resource, state, desired, accept?)` returns each declared field that differs, `{ path, desired, actual }`, where `accept` gives, by a reference's path, the values that stand for the resource it names. `withoutWriteOnly(fields, desired)` drops write-only values from a declaration, keeping a secret's marker, which names a stored value and holds none. So a field left out is never drift, which covers attributes an OpenTofu provider marks `computed` and `optional` (GitHub's `etag`, `topics`): they are the user's to set, so not `vendorOwned`, but the vendor fills them in when nobody does.
|
|
228
234
|
|
|
229
235
|
`describeConfig` lists the resource types as `resourceTypes`, each `{ id: "<vendor>.<type>", vendor, type, title, identity, fields, schema, ops }`: `schema` is its state as JSON Schema, with `$id` `sanoma:resource-type/<id>`, which its operations' contracts `$ref` rather than repeat. `@sanoma-ai/workflows/shared` exports the `Resource`, `ResourceSpec`, `ResourceFields`, `References`, `FieldReference`, `Declared`, `Declaring`, `DeclaredResource` and `ResourceProblem` types.
|
|
230
236
|
|
|
@@ -249,11 +255,11 @@ export default [website, websiteMain];
|
|
|
249
255
|
|
|
250
256
|
Another data file names `website` by importing it: `import { website } from "./github.ts"`.
|
|
251
257
|
|
|
252
|
-
What a data file may hold: named imports, of a connector's constructors from its `<package>/resources` entry (exported under the vendor's id)
|
|
258
|
+
What a data file may hold: named imports, of a connector's constructors from its `<package>/resources` entry (exported under the vendor's id), of resources from other `.ts` data files in `resources/`, and of `secret` from `sanoma/resources` (or `@sanoma-ai/workflows/resources`); `export const <name> = <vendor>.<type>({ … })`, one resource per statement and each name once, whose fields are strings (and template literals without `${}`), numbers, booleans, null, objects, arrays and, where the type's `fields.references` allows one, names of resources the file declares or imports, and, where the type's `fields.writeOnly` lists the field, `secret("NAME")` or `secret("NAME", { generate: true })` with literal arguments (see [Secrets](#secrets)); and `export default [a, b]`, the file's resources. Nothing else: no other calls, `new`, functions, spreads, computed keys, member access (`site.name`: name `site`), `process`, `undefined` (leave the field out) or type assertions. Each refusal says what to write instead.
|
|
253
259
|
|
|
254
260
|
The data files are found from the config's root: `defineConfig` records the file it is called from as the config's `file`, and `resolveConfig` gives its directory as `root`. A config bundled into one file sets `root` itself. A config with neither has no data files, and says so, rather than guess the working directory.
|
|
255
261
|
|
|
256
|
-
`readDataFiles(root, connectors)`, from `@sanoma-ai/workflows/describe`, reads every regular `.ts` file under `<root>/resources` (but `*.test.ts`, `*.d.ts` and `node_modules`), parsing it with oxc-parser and never importing it. A constructors import must be exactly a connector's `<info.package>/resources`, matched among the config's `connectors`; each resource is checked by calling its type, as running the file would, with each resource it names given as that resource. It returns `{ resources, problems }`: one `DeclaredResource` per `export const`, by file and then in file order, less any with a problem and any that names one, and the problems, by file, line and column:
|
|
262
|
+
`readDataFiles(root, connectors)`, from `@sanoma-ai/workflows/describe`, reads every regular `.ts` file under `<root>/resources` (but `*.test.ts`, `*.d.ts` and `node_modules`), parsing it with oxc-parser and never importing it. A constructors import must be exactly a connector's `<info.package>/resources`, matched among the config's `connectors`; each resource is checked by calling its type, as running the file would, with each resource it names given as that resource. It returns `{ resources, bindings, orphans, problems }`: one `DeclaredResource` per `export const`, by file and then in file order, less any with a problem and any that names one, and the problems, by file, line and column:
|
|
257
263
|
|
|
258
264
|
```ts
|
|
259
265
|
interface DeclaredResource {
|
|
@@ -261,7 +267,9 @@ interface DeclaredResource {
|
|
|
261
267
|
file: string; // the file, as `id` names it: "resources/identity/github.ts"
|
|
262
268
|
vendor: string; // "github"
|
|
263
269
|
type: string; // "branch_protection"
|
|
264
|
-
name: string; // its
|
|
270
|
+
name: string; // what people call it: its identity, else its export's name: "website:main"
|
|
271
|
+
identity?: string; // the import id that finds it unbound, from `find`: "website:main"
|
|
272
|
+
binding?: string; // the id of the object the lockfile binds it to: "BPR_kwDOabc123"
|
|
265
273
|
span: readonly [start: number, end: number]; // its `export const`, as UTF-16 offsets into the file
|
|
266
274
|
desired: Record<string, unknown>; // its fields as its type checked them: { repository_id: "website", … }
|
|
267
275
|
refs: Record<string, string>; // where it names a resource, to that resource's id: { repository_id: "resources/identity/github.ts#website" }
|
|
@@ -275,10 +283,25 @@ interface ResourceProblem {
|
|
|
275
283
|
}
|
|
276
284
|
```
|
|
277
285
|
|
|
278
|
-
A problem is anything outside the subset, a constructors import of no connector in the config or a data file that is not one, a type the connector does not declare, a value the type refuses (a resource where it takes none among them), a reference cycle, or two resources of one type with one
|
|
286
|
+
A problem is anything outside the subset, a constructors import of no connector in the config or a data file that is not one, a type the connector does not declare, a value the type refuses (a resource where it takes none among them), a reference cycle, or two resources of one type with one identity, and, from the lockfile ([below](#bindings)), a lockfile that is not JSON or not bindings, a binding of another type than its declaration's, or two declarations bound to one object. `describeConfig` lists both, as `resources` and `problems` (and the lockfile's `orphans`), so the app starts with the resources it could read and shows what is wrong in the rest. `readResources(config)` is the strict reading, for a test or CI: it returns the resources, or throws one error listing every problem at `file:line:column`.
|
|
287
|
+
|
|
288
|
+
#### Bindings
|
|
289
|
+
|
|
290
|
+
Which vendor object a declaration means lives in a lockfile beside the data files, `resources/sanoma.lock.json` (`LOCKFILE`), not in the declaration, as `package-lock.json` is beside `package.json`: data files hold intent only, so an edit to a field cannot point a declaration at another object, copying a declaration copies no binding, and a rename is an update. Only apply writes it; it is committed.
|
|
291
|
+
|
|
292
|
+
```jsonc
|
|
293
|
+
{
|
|
294
|
+
"resources/identity/github.ts#website": { "type": "github.repository", "id": "R_kgDOabc123" },
|
|
295
|
+
"resources/billing/stripe.ts#pro": { "type": "stripe.product", "id": "prod_Rx2" },
|
|
296
|
+
}
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
`readDataFiles` joins each declaration to its binding by its key, `<file>#<export>`, as `binding`, returns the lockfile's `bindings`, and lists as `orphans` (`{ key, type, id }`) the bindings no data file declares. A missing lockfile binds nothing. `@sanoma-ai/workflows/describe` exports `readLockfile(root)` (`{ bindings, problems }`), `serializeLockfile(bindings)` (keys sorted, two-space JSON, a trailing newline), `writeLockfile(root, bindings)` (whole or not at all: written beside it, then renamed), and `diffLockfiles(before, after)`, each change a `Rebind`: `{ key, type, kind: "bind" | "rebind" | "unbind", from?, to? }`.
|
|
279
300
|
|
|
280
301
|
oxlint holds `resources/**` to the subset with the rule `sanoma/data-file`, an oxlint JS plugin the package's `oxlint.json` loads from `dist/plugin.js`, so extending it (see [Keeping workflows replay-safe](#keeping-workflows-replay-safe)) covers data files as well; it does not know the config, so whether a connector, type or value is known is the reader's to say, in `describeConfig(config).problems`, which `sanoma check` prints.
|
|
281
302
|
|
|
303
|
+
Sanoma writes data files too, from `@sanoma-ai/workflows/describe`. `setFields(text, exportName, fields, options)` returns the file's text with fields of one resource set, by dotted path as a drift report names them (`{ has_wiki: true, "metadata.owner": "ops" }`; a field the declaration lacks is added): only that `export const` changes, formatted with [oxfmt](https://oxc.rs/docs/guide/usage/formatter), and every other byte of the file stays as it was. It refuses a resource the file does not declare, a path through a value that is not an object or a list, and a value that is or holds a reference (a person changes those); a path whose segments name a field with a dot in its key (`metadata: { "app.owner": … }`) finds that field. `renderDataFile({ imports, exports }, options)` renders a whole new file, which `readDataFile` reads back as given (what it returns renders the file again, less its comments); a reference is a `Reference`, and a secret's marker is written as `secret("NAME")` (or with `{ generate: true }`), the file then importing `secret` from `"sanoma/resources"` (`setFields` adds that import after the file's own when it sets one). `formatOptions(root)` reads the options from `<root>/.oxfmtrc.json`, oxfmt's defaults without one. oxfmt is a dependency of this package, loaded only when a file is written.
|
|
304
|
+
|
|
282
305
|
### Drift
|
|
283
306
|
|
|
284
307
|
`drift` is a built-in workflow: `resolveConfig` adds it to every config whose connectors declare resource types, so the worker runs it, `describeConfig` lists it (its entry has `builtin: true`, a label) and it starts like any other workflow, with the input `{}`. A config's own workflow may not take its name. It `uses` the `import` of each resource type that has a driver, so its version moves with the connectors and drivers, not with the data files, and it is held to the same lint as any workflow: every vendor call goes through `ctx`, so the policy sees each as effect `read` and the ledger records it.
|
|
@@ -286,17 +309,43 @@ oxlint holds `resources/**` to the subset with the rule `sanoma/data-file`, an o
|
|
|
286
309
|
```ts
|
|
287
310
|
const runId = await client.drift({ startedBy: { id: "alice" } }); // start(drift, {}, …)
|
|
288
311
|
const report = await client.driftReport(runId); // waits for it, as `result` does
|
|
289
|
-
// { startedAt, finishedAt, problems: [], resources: [{ id, type, vendor, name, status: "drifted", fields: [{ path: "has_wiki", desired: false, actual: true }] }, …] }
|
|
312
|
+
// { startedAt, finishedAt, problems: [], resources: [{ id, type, vendor, name, binding, status: "drifted", fields: [{ path: "has_wiki", desired: false, actual: true }] }, …] }
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
The run reads the data files and the lockfile itself, in its first step (`drift:data-files`, recorded, so a replay checks the same list): its input is `{}`, and any other is refused, so no caller can hand it other resources. The step records each resource without its write-only values, and the files' problems, which the report carries (`problems`; a file with problems contributes no resources, as in `describeConfig`). The worker loads the reader only then. Then the run imports each resource by its binding, else by its identity, one vendor call each (none for a resource with neither), in the files' order (every run imports: no state is kept between runs yet), and compares the declared fields with `diffDeclared` (above), a reference against the state of the resource it names from the same run. Each resource's `status` is `clean`, `drifted` (with `fields`, each leaf that differs: `{ path, desired, actual }`, a list item's path with its index, a list whose length changed or an `unordered` one whole), `gone` (the import of its binding says so), `absent` (it is not bound; `found` is the binding id of the object its identity found, if any, which apply may bind it to) or `error` (with `error`: the vendor's error, a policy denial, or no driver for its type); one resource's failure does not stop the others, but a failure that ends the run (`isRunControlError`: DBOS cancelling or stopping it) does. `client.drift` throws `invalid_input` when the connectors declare no resource types; `client.driftReport(runId)` throws `invalid_input` for another workflow's run. `DriftReport`, `DriftResult`, `DriftField` and `DriftStatus` are the report's types, also in `@sanoma-ai/workflows/shared`; `client.runs({ workflow: "drift" })` lists drift runs.
|
|
316
|
+
|
|
317
|
+
A host's revision is drift too (see [Hosts](#hosts)): the run resolves its ref with the built-in `sanoma.source.resolve`, and a host whose state runs another commit is `drifted`, with a field at its `revision` path (`desired` the commit the ref names now).
|
|
318
|
+
|
|
319
|
+
### Apply
|
|
320
|
+
|
|
321
|
+
`apply` is the built-in workflow that makes the vendors match the data files, beside `drift` and listed the same way (`builtin: true`). It `uses` the `import` of each resource type with a driver, the change operations of each writable one, and `approval`; it is the only workflow that may call a change operation. Its input is `{ only?, bind?, answers? }` (any other field refused): `only` the declared ids (`<file>#<export>`) or orphaned lockfile keys to plan, else all; `bind`, a declared id to the vendor's id of the object to bind it to; `answers`, by id or key, `recreate` or `forget` for a gone object, `forget` or `delete` for an orphaned binding.
|
|
322
|
+
|
|
323
|
+
```ts
|
|
324
|
+
const runId = await client.apply(
|
|
325
|
+
{ answers: { "resources/infra.ts#store": "recreate" } },
|
|
326
|
+
{ startedBy: { id: "alice" } },
|
|
327
|
+
);
|
|
328
|
+
const report = await client.applyReport(runId); // { plan: { problems, rebinds, steps }, results, lockfile: { written, revision?, bindings } }
|
|
290
329
|
```
|
|
291
330
|
|
|
292
|
-
The run reads the data files
|
|
331
|
+
The run reads the data files and the lockfile in its first step (`apply:data-files`; a secret's marker stays in `desired`, a plain write-only value does not), with the lockfile's `rebinds`: what changed in it since apply last wrote it (`diffLockfiles` against the bindings the latest live apply run's last `apply:lockfile` step wrote), changes someone else made. It imports each declared resource as drift does (by the binding `bind` gives, else the lockfile's, else its identity), then plans each one `only` wants, each after the resources it names, as a `PlanStep` (`{ id, vendor, type, action, … }`): `bind` (to the object `bind` gives, or the one its identity `found`, by its binding id), `create` (unbound and nothing found: `planned` and `unknown` from its type's `plan`), `update` or `replace` (its bound object as `plan` says, with drift's `fields` and the paths that force a replace), `none`, `gone` (its bound object is no longer there: skipped unless answered, since a recreated database looks fine and has lost its data; a type with `stateless: true`, holding no data, is planned as a `create` instead, unless answered), and `orphan` for each binding no declaration has (skipped unless answered: deleting a declaration never deletes its object). What cannot be planned (a read-only type that is absent or drifted, an answer that cannot be done, an id the input names that nothing declares) is one of the plan's `problems`, with no step.
|
|
332
|
+
|
|
333
|
+
When the policy would hold any change call the plan makes (asked ahead, in the step `apply:policy`, as each call would ask it), the run asks one approval of the first approver it names, titled "Apply resource changes", covering every change operation in the plan, with the plan as JSON in its `details`; a policy that lets a covered call through once approved (`approvedFor`) then holds none of them again. Then each step runs, in order: `create`, `update` (with `expect`, the plan's action and replace, so a vendor that now plans otherwise refuses with `plan_changed`), `delete`, or only the lockfile changed. A reference to a resource the vendor holds by another field (`by`) is given that resource's binding id, one made earlier in the run included. Right after any step that changes a binding, that binding is changed on the lockfile as it is then (so another run's change made meanwhile stays), written whole and committed through the worker's `source` (`apply:lockfile`: `Apply: create resources/infra.ts#store (supabase.project abcd…)`, by the person who started the run), so an object made is never left unbound. Right after a create's binding is written, the run calls its type's `afterCreate` operations in order, by id, each given the new object's binding id and its export (a Supabase project waits for its health, then stores its database URL); one with `until` is called again every 5 seconds until its output passes, for up to 10 minutes, and one that never does fails the step, its object still bound. The approval covers the `write` ones among them too. Each step's result is `done`, `skipped` (unanswered) or `failed` (with `error`; the run goes on); a host's that it made or changed carries `host`, its `outputs`. A sandbox run writes nothing: the report's `lockfile.bindings` is its copy. `ApplyPlan`, `PlanStep`, `ApplyReport` and `ApplyResult` are the types, also in `@sanoma-ai/workflows/describe` and `/shared`.
|
|
293
334
|
|
|
294
335
|
This package knows nothing of OpenTofu. Resource types from an OpenTofu provider are generated by [`@sanoma-ai/bridge/tfschema`](../bridge/src/tfschema/AGENTS.md), which turns the provider's schema document into zod schemas and flagged fields (`resources.gen.ts`); `@sanoma-ai/bridge` depends on this package, never the other way round.
|
|
295
336
|
|
|
337
|
+
### Hosts
|
|
338
|
+
|
|
339
|
+
A host is a resource type that runs the company (a container, a Render service, a Fly app): `defineResource({ …, writable: true, host })`, where `host` (`HostSpec`) says where the type takes the runtime's inputs, by dotted path into its fields (`inputs: { source, ref, secrets, port }`, `secrets` a write-only field), which field apply fills with the commit (`revision`, which a data file may not declare: it names a branch, never a commit), what it can do (`capabilities: { lifetime, publicUrl, buildsFromGit, staticEgress }`), and how its state reads (`outputs(state) → { url, revision?, healthy }`). `defineResource` refuses a `host` that names no field, or whose `secrets` is not write-only. A write-only field may be a map of secrets (`secrets: { SANOMA_DATABASE_URL: secret("DATABASE_URL") }`): its markers are kept as one at the field is. A host type is `stateless` (a `ResourceSpec` field, default true for a host): its objects keep no data, so replacing or recreating one is routine. `source` is a clone URL, or a reference to a declared repository with a `by` field holding one.
|
|
340
|
+
|
|
341
|
+
A config whose connectors declare a host type has the built-in operation `sanoma.source.resolve` (`SOURCE_RESOLVE`, effect `read`): `{ url, ref } → { revision }`, by `git ls-remote` on this machine (a branch, then a tag, peeled; a full commit id is itself). Apply calls it for each host it plans, and gives the host the commit as its `revision` and the clone URL as its `source` (a referenced repository's, from its `by` field); a host whose ref does not resolve, or whose repository has no clone URL yet (apply the repository first), is a problem of the plan. The drift run calls it for every host it reads. A host's `secrets` map may name sanoma's own secrets (`SANOMA_…`), and no other field of any type may. `resolveRevision(url, ref)` is the same, outside a run.
|
|
342
|
+
|
|
296
343
|
### Secrets
|
|
297
344
|
|
|
298
345
|
Never put a secret in a workflow's input or an operation's input or output. They are persisted verbatim: DBOS keeps every step's input and output in Postgres, the ledger records each call's input and output, an approval a policy asks for carries the held call's input, and the app shows all of it to anyone who can reach it. A driver reads its credentials when it is called (from the environment or a secret store), and an operation that creates a secret (an API key, a password reset link) returns a reference to where it is stored, not the secret.
|
|
299
346
|
|
|
347
|
+
A secret a resource needs is named, never written. A data file gives `secret("NAME")` (from `sanoma/resources`; `NAME` upper case, as a variable's; sanoma's own, `SANOMA_…`, only in a host's `secrets`) at a write-only field (or an entry of a write-only map), and `secret("NAME", { generate: true })` for one apply makes when none is stored: `database_password: secret("SUPABASE_DB_PASSWORD", { generate: true })`. Declarations, checkpoints and the ledger hold its marker, `{ "$secret": "NAME", "generate": true }`; the resource type refuses one at any other field, and its schema does not judge it. The driver resolves the markers just before the vendor call, `await resolveSecrets(input, call.secrets)`: each is replaced by the stored value, a `generate` one with none stored first made (32 random bytes, base64url) and stored, and one with nothing stored fails the call for good. `call.secrets` is `{ get(name), ensure(name, make) }`: the environment wins, else the `sanoma_credentials` table (see [Credentials](#credentials)); a fake's and a sandbox run's are in memory (`memorySecrets()`, which a driver's tests use too).
|
|
348
|
+
|
|
300
349
|
### Versions
|
|
301
350
|
|
|
302
351
|
Every run is stamped with the application version of the worker that runs it, `<appName>@<version>`, and only a worker on that version recovers it after a restart. Versioning is automatic: the version is a hash of each workflow's name, the source of its `run` function, its operations and its input schema, which is DBOS's own scheme applied to the workflow code rather than to the runtime's registration wrapper (DBOS would otherwise see one identical function for every workflow and never change). The hash cannot see functions `run` calls that live elsewhere, op schemas, drivers or the policy: edit those with no run in flight, or restart on the old code first. To name a version instead (a git commit, say), set DBOS's `DBOS__APPVERSION` environment variable; it is prefixed with the app name the same way. A new version becomes the app's latest when it first starts, and runs queued without a version go to the latest; a worker started on an older version (a rollback) warns instead, unless started with `{ promote: true }`. The hash is of the code as it runs, so TypeScript source and the compiled JavaScript of the same workflow have different versions.
|
|
@@ -384,12 +433,14 @@ The steps, as [Cucumber expressions](https://github.com/cucumber/cucumber-expres
|
|
|
384
433
|
| Then | `{op} was denied` | Expects the policy to have denied a call to it: `Then github.repository.import was denied` |
|
|
385
434
|
| Then | `{op} was held for {who}` | Expects an approval for that approver to have held it: `Then stripe.product.import was held for finance` |
|
|
386
435
|
| Then | `{declared} is drifted in {field}` | For a drift run: `Then resources/identity/github.ts#website is drifted in has_wiki` |
|
|
387
|
-
| Then | `{declared} is clean`, `{declared} is gone`
|
|
436
|
+
| Then | `{declared} is clean`, `{declared} is gone`, `{declared} is absent` | For a drift run: `Then resources/identity/github.ts#docs is clean` |
|
|
437
|
+
| Then | `apply creates {declared}` | For an apply run: it created the resource (or made a gone one again): `Then apply creates resources/identity/github.ts#docs` |
|
|
438
|
+
| Then | `{declared} is bound to {string}` | For an apply run: the lockfile it ended with binds it so: `Then resources/identity/rules.ts#docsMain is bound to "BPR_docs:main"` |
|
|
388
439
|
| Then | `the run succeeds`, `the run fails`, `the run fails with {string}` | Expects the run to finish, or to fail (with that error code, such as `"approval_rejected"`) |
|
|
389
440
|
|
|
390
|
-
A resource step names an object a vendor's fake holds, by resource type and import id (
|
|
441
|
+
A resource step names an object a vendor's fake holds, by resource type and import id (a binding id, or an identity: a GitHub repository's name, a branch protection rule's `repository:pattern`; a fake also finds an object by its binding id, such as a repository's `node_id`), for a fake that holds objects (`@sanoma-ai/bridge`'s `tfFake`, as the GitHub and Stripe fakes are; a sandbox run whose scenario seeds a resource of another vendor's fake fails with `invalid_input`). `has` puts the fields on the object (`put`: over the one the vendor has, keeping its other fields, or a new one), each read as a table cell is for an operation's input and checked against the type's schema; `is gone` removes it (`remove`), and is nothing when there is none. The run's own fake is what is seeded. A fake starts a sandbox run with what its replies were recorded with, so a declared resource the recording lacks is `gone` to a drift run until a scenario puts it.
|
|
391
442
|
|
|
392
|
-
The policy steps read the run's ledger: `was denied` holds when the policy's decision on some call to the operation is `deny`, and `was held for {who}` when an approval addressed to that person (by id) or group (by name) holds it, whether the policy asked for it on that call or the workflow asked for one that `covers` it. A failed step says what happened instead: `github.repository.import was allowed, not denied`, `stripe.product.import was held for finance, not marketing-lead`. The drift steps read the report a finished [drift](#drift) run returned, so their scenario must run `drift` (a parse error otherwise); a failed one says how the report found the resource: `resources/identity/github.ts#website is drifted in has_wiki, not clean`, `… is gone, not drifted in has_wiki`, or that the report has no such resource.
|
|
443
|
+
The policy steps read the run's ledger: `was denied` holds when the policy's decision on some call to the operation is `deny`, and `was held for {who}` when an approval addressed to that person (by id) or group (by name) holds it, whether the policy asked for it on that call or the workflow asked for one that `covers` it. A failed step says what happened instead: `github.repository.import was allowed, not denied`, `stripe.product.import was held for finance, not marketing-lead`. The drift steps read the report a finished [drift](#drift) run returned, so their scenario must run `drift` (a parse error otherwise), as the apply steps read an apply run's report, and an apply run's input is only what `runs with` gives, never made up; a failed one says how the report found the resource: `resources/identity/github.ts#website is drifted in has_wiki, not clean`, `… is gone, not drifted in has_wiki`, or that the report has no such resource.
|
|
393
444
|
|
|
394
445
|
The `step` steps name a [step](#step-names) of the scenario's workflow, after its `When`: one whose name is no step's, or several's (a default an operation called twice shares), is an error listing the names it has, so a test names one call (an unnamed call with a computed segment, `ctx.ghost.post[action](…)`, is named in a run by the operation it calls, so it shares that operation's id; an approval whose title is no literal is named by its title in a run, which the outline cannot tell apart), and `was called with` checks its operation's input fields. They read the ledger by the `name` its records carry: a step ran when a call by it was made (not denied, rejected or failed for good), a sleep by it started, or the workflow asked for an approval of that title; a policy's hold on a call is that call's, not a run of its own. A call in a loop runs once per pass: `ran 3 times` for three items. A failed one says what happened: `step "Draft the post" ran twice, not 3 times`.
|
|
395
446
|
|
|
@@ -460,8 +511,8 @@ A sandbox run does not survive a worker restart: its fakes keep their state in t
|
|
|
460
511
|
|
|
461
512
|
`@sanoma-ai/app` is one UI over a config; another (a Slack bot, an internal tool) can be built on the same pieces, which the package exports for that:
|
|
462
513
|
|
|
463
|
-
- `describeConfig(config)` and its types (`ConfigDescription`, `WorkflowEntry`, `OpEntry`, `VendorEntry`, `ResourceTypeEntry`, `DeclaredResource`, `ResourceProblem`), and `readDataFiles(root, connectors)` and `readResources(config)`, from `@sanoma-ai/workflows/describe`: what to render, as plain JSON, with each workflow's outline. It is a separate entry so the worker never loads the parser the outline uses, and the lint refuses it in workflow files. `resolveConfig(config)` returns the checked config as a `ResolvedConfig`, with the operations and drivers by id, the resource types by `<vendor>.<type>` (`types`), the workflows by name (the built-
|
|
464
|
-
- `SanomaClient`: start runs, list them (`runs({ limit, status, workflow })`), read a run's ledger and approvals, decide approvals, with the checks described above, and start a drift check and read its report (`drift`, `driftReport`, see [Drift](#drift)).
|
|
514
|
+
- `describeConfig(config)` and its types (`ConfigDescription`, `WorkflowEntry`, `OpEntry`, `VendorEntry`, `ResourceTypeEntry`, `DeclaredResource`, `ResourceProblem`), and `readDataFiles(root, connectors)` and `readResources(config)`, from `@sanoma-ai/workflows/describe`: what to render, as plain JSON, with each workflow's outline. It is a separate entry so the worker never loads the parser the outline uses, and the lint refuses it in workflow files. `resolveConfig(config)` returns the checked config as a `ResolvedConfig`, with the operations and drivers by id, the resource types by `<vendor>.<type>` (`types`), the workflows by name (the built-ins `drift` and `apply` among them), its `root` and each vendor's `credentials` (`CredentialStatus`); `isOp(x)` tells an operation from a built-in in a workflow's `uses`.
|
|
515
|
+
- `SanomaClient`: start runs, list them (`runs({ limit, status, workflow })`), read a run's ledger and approvals, decide approvals, with the checks described above, and start a drift check or an apply and read its report (`drift`, `driftReport`, `apply`, `applyReport`, see [Drift](#drift) and [Apply](#apply)).
|
|
465
516
|
- `APPROVALS_EVENT` and `decisionEventOf(approvalId)`: the DBOS events a run publishes its approvals and each decision on, for a UI that reads DBOS directly. `ApprovalMessage` is the zod schema of a decision as a run reads it. `RunArgs` is what a run receives: its input, `startedBy` and, for a sandbox run, its scenario as `sandbox`.
|
|
466
517
|
- `mayDecide(approval, principal)` and `approverLabel(approver)`: who may decide, and how to name them, the same way the run does. `isEnded(status)` and `ENDED_STATUSES`: the run statuses that read no more decisions; `RUN_STATUSES`: every run status, in the order a run may pass through them.
|
|
467
518
|
- Errors: `errorCode(err)` for the code to branch on, `errorMessage(err)` for the text of anything thrown, `invalidInput(what, issues)` to build an `invalid_input` error from zod issues (`InputIssue` is one issue, without symbols in its path), and the classes `SanomaError`, `PolicyDeniedError`, `RejectedError` and `DriverError`. Read codes with `errorCode`, never `instanceof`.
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
import type { ApplyDeclared, ApplySteps } from "./apply.ts";
|
|
2
|
+
/**
|
|
3
|
+
* The apply workflow's steps for a config: `read`, the data files and the lockfile as drift reads
|
|
4
|
+
* them, under `root`; `appName` scopes the search for the lockfile apply last wrote.
|
|
5
|
+
*/
|
|
6
|
+
export declare function applySteps(appName: string, root: string | undefined, read: () => Promise<Omit<ApplyDeclared, "rebinds">>): ApplySteps;
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
import { join, relative, sep } from "node:path";
|
|
2
|
+
import { DBOS } from "@dbos-inc/dbos-sdk";
|
|
3
|
+
import { decide } from "./call.js";
|
|
4
|
+
import { diffLockfiles, LOCKFILE, readLockfile, writeLockfile } from "./lockfile.js";
|
|
5
|
+
import { currentRun } from "./run.js";
|
|
6
|
+
import { APPLY_WORKFLOW } from "./shared.js";
|
|
7
|
+
// The apply workflow's steps that call no vendor: reading the data files, asking the policy
|
|
8
|
+
// ahead, and writing the lockfile. Here, not in apply.ts, which is held to the workflow lint and
|
|
9
|
+
// imports no DBOS; each reads its run from `currentRun`.
|
|
10
|
+
/** The name of the step that writes the lockfile and commits it: its output has the bindings it wrote. */
|
|
11
|
+
const WRITE_STEP = "apply:lockfile";
|
|
12
|
+
/** How many of the latest apply runs are searched for the lockfile one last wrote. */
|
|
13
|
+
const RUNS_SEARCHED = 100;
|
|
14
|
+
const runNow = () => {
|
|
15
|
+
const run = currentRun.getStore();
|
|
16
|
+
if (!run)
|
|
17
|
+
throw new Error("An apply step ran outside its run");
|
|
18
|
+
return run;
|
|
19
|
+
};
|
|
20
|
+
/**
|
|
21
|
+
* The bindings the latest apply run that wrote the lockfile wrote last, from its steps; undefined
|
|
22
|
+
* when none of the latest `RUNS_SEARCHED` did. A sandbox run writes none.
|
|
23
|
+
*/
|
|
24
|
+
async function lastWritten(appName) {
|
|
25
|
+
const runs = await DBOS.listWorkflows({
|
|
26
|
+
workflowName: APPLY_WORKFLOW,
|
|
27
|
+
applicationName: appName,
|
|
28
|
+
sortDesc: true,
|
|
29
|
+
limit: RUNS_SEARCHED,
|
|
30
|
+
loadInput: false,
|
|
31
|
+
loadOutput: false,
|
|
32
|
+
});
|
|
33
|
+
for (const { workflowID } of runs) {
|
|
34
|
+
const steps = await DBOS.listWorkflowSteps(workflowID);
|
|
35
|
+
const last = steps?.findLast((step) => step.name === WRITE_STEP && !step.error);
|
|
36
|
+
if (last)
|
|
37
|
+
return last.output.bindings;
|
|
38
|
+
}
|
|
39
|
+
return undefined;
|
|
40
|
+
}
|
|
41
|
+
/**
|
|
42
|
+
* The apply workflow's steps for a config: `read`, the data files and the lockfile as drift reads
|
|
43
|
+
* them, under `root`; `appName` scopes the search for the lockfile apply last wrote.
|
|
44
|
+
*/
|
|
45
|
+
export function applySteps(appName, root, read) {
|
|
46
|
+
return {
|
|
47
|
+
// The data files, and the lockfile against the one apply last wrote: a change someone else made.
|
|
48
|
+
declared: () => DBOS.runStep(async () => {
|
|
49
|
+
const declared = await read();
|
|
50
|
+
const before = await lastWritten(appName);
|
|
51
|
+
return { ...declared, rebinds: before ? diffLockfiles(before, declared.bindings) : [] };
|
|
52
|
+
}, { name: "apply:data-files" }),
|
|
53
|
+
// The policy asked as each call would ask it, now, so one approval can cover them all.
|
|
54
|
+
holds: (calls) => DBOS.runStep(async () => {
|
|
55
|
+
const run = runNow();
|
|
56
|
+
for (const { op: id, input } of calls) {
|
|
57
|
+
const op = run.state.ops.get(id);
|
|
58
|
+
// Parsed as the call will be, so the policy sees the same input with its defaults.
|
|
59
|
+
const parsed = op.input.safeParse(input);
|
|
60
|
+
const decision = await decide(run, op, parsed.success ? parsed.data : input);
|
|
61
|
+
if (decision.kind === "approve")
|
|
62
|
+
return decision.approver;
|
|
63
|
+
}
|
|
64
|
+
return undefined;
|
|
65
|
+
}, { name: "apply:policy" }),
|
|
66
|
+
// The one binding changed on the lockfile as it is now, written whole, then committed, in one
|
|
67
|
+
// step whose output is what it wrote. A sandbox run keeps its lockfile in memory: the run's
|
|
68
|
+
// report has it.
|
|
69
|
+
write: async (key, binding, message) => {
|
|
70
|
+
const run = runNow();
|
|
71
|
+
if (run.sandbox !== undefined || root === undefined)
|
|
72
|
+
return { written: false };
|
|
73
|
+
const { source } = run.state;
|
|
74
|
+
const { revision } = await DBOS.runStep(async () => {
|
|
75
|
+
const now = readLockfile(root);
|
|
76
|
+
if (now.problems.length)
|
|
77
|
+
throw new Error(`${LOCKFILE} cannot be updated: ${now.problems[0].message}`);
|
|
78
|
+
const { [key]: _, ...others } = now.bindings;
|
|
79
|
+
const bindings = binding ? { ...others, [key]: binding } : others;
|
|
80
|
+
writeLockfile(root, bindings);
|
|
81
|
+
const paths = [
|
|
82
|
+
relative(source?.root ?? root, join(root, LOCKFILE))
|
|
83
|
+
.split(sep)
|
|
84
|
+
.join("/"),
|
|
85
|
+
];
|
|
86
|
+
const committed = await source?.commit({ paths, message, author: run.actor.id });
|
|
87
|
+
return { bindings, ...(committed && { revision: committed.revision }) };
|
|
88
|
+
}, { name: WRITE_STEP });
|
|
89
|
+
return { written: true, ...(revision !== undefined && { revision }) };
|
|
90
|
+
},
|
|
91
|
+
};
|
|
92
|
+
}
|
|
93
|
+
//# sourceMappingURL=apply-steps.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"apply-steps.js","sourceRoot":"","sources":["../src/apply-steps.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,GAAG,EAAE,MAAM,WAAW,CAAC;AAChD,OAAO,EAAE,IAAI,EAAE,MAAM,oBAAoB,CAAC;AAE1C,OAAO,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AAEnC,OAAO,EAA+B,aAAa,EAAE,QAAQ,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAClH,OAAO,EAAE,UAAU,EAAY,MAAM,UAAU,CAAC;AAChD,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAE7C,4FAA4F;AAC5F,iGAAiG;AACjG,yDAAyD;AAEzD,0GAA0G;AAC1G,MAAM,UAAU,GAAG,gBAAgB,CAAC;AAEpC,sFAAsF;AACtF,MAAM,aAAa,GAAG,GAAG,CAAC;AAE1B,MAAM,MAAM,GAAG,GAAQ,EAAE;IACvB,MAAM,GAAG,GAAG,UAAU,CAAC,QAAQ,EAAE,CAAC;IAClC,IAAI,CAAC,GAAG;QAAE,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IAC/D,OAAO,GAAG,CAAC;AACb,CAAC,CAAC;AAEF;;;GAGG;AACH,KAAK,UAAU,WAAW,CAAC,OAAe;IACxC,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,aAAa,CAAC;QACpC,YAAY,EAAE,cAAc;QAC5B,eAAe,EAAE,OAAO;QACxB,QAAQ,EAAE,IAAI;QACd,KAAK,EAAE,aAAa;QACpB,SAAS,EAAE,KAAK;QAChB,UAAU,EAAE,KAAK;KAClB,CAAC,CAAC;IACH,KAAK,MAAM,EAAE,UAAU,EAAE,IAAI,IAAI,EAAE,CAAC;QAClC,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,iBAAiB,CAAC,UAAU,CAAC,CAAC;QACvD,MAAM,IAAI,GAAG,KAAK,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,KAAK,UAAU,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAChF,IAAI,IAAI;YAAE,OAAQ,IAAI,CAAC,MAAiC,CAAC,QAAQ,CAAC;IACpE,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,UAAU,CACxB,OAAe,EACf,IAAwB,EACxB,IAAmD;IAEnD,OAAO;QACL,iGAAiG;QACjG,QAAQ,EAAE,GAAG,EAAE,CACb,IAAI,CAAC,OAAO,CACV,KAAK,IAAI,EAAE;YACT,MAAM,QAAQ,GAAG,MAAM,IAAI,EAAE,CAAC;YAC9B,MAAM,MAAM,GAAG,MAAM,WAAW,CAAC,OAAO,CAAC,CAAC;YAC1C,OAAO,EAAE,GAAG,QAAQ,EAAE,OAAO,EAAE,MAAM,CAAC,CAAC,CAAC,aAAa,CAAC,MAAM,EAAE,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;QAC1F,CAAC,EACD,EAAE,IAAI,EAAE,kBAAkB,EAAE,CAC7B;QACH,uFAAuF;QACvF,KAAK,EAAE,CAAC,KAAoB,EAAE,EAAE,CAC9B,IAAI,CAAC,OAAO,CACV,KAAK,IAAmC,EAAE;YACxC,MAAM,GAAG,GAAG,MAAM,EAAE,CAAC;YACrB,KAAK,MAAM,EAAE,EAAE,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,KAAK,EAAE,CAAC;gBACtC,MAAM,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAE,CAAC;gBAClC,mFAAmF;gBACnF,MAAM,MAAM,GAAG,EAAE,CAAC,KAAK,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC;gBACzC,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,GAAG,EAAE,EAAE,EAAE,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;gBAC7E,IAAI,QAAQ,CAAC,IAAI,KAAK,SAAS;oBAAE,OAAO,QAAQ,CAAC,QAAQ,CAAC;YAC5D,CAAC;YACD,OAAO,SAAS,CAAC;QACnB,CAAC,EACD,EAAE,IAAI,EAAE,cAAc,EAAE,CACzB;QACH,8FAA8F;QAC9F,4FAA4F;QAC5F,iBAAiB;QACjB,KAAK,EAAE,KAAK,EAAE,GAAW,EAAE,OAA4B,EAAE,OAAe,EAAE,EAAE;YAC1E,MAAM,GAAG,GAAG,MAAM,EAAE,CAAC;YACrB,IAAI,GAAG,CAAC,OAAO,KAAK,SAAS,IAAI,IAAI,KAAK,SAAS;gBAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;YAC/E,MAAM,EAAE,MAAM,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC;YAC7B,MAAM,EAAE,QAAQ,EAAE,GAAG,MAAM,IAAI,CAAC,OAAO,CACrC,KAAK,IAAwD,EAAE;gBAC7D,MAAM,GAAG,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;gBAC/B,IAAI,GAAG,CAAC,QAAQ,CAAC,MAAM;oBAAE,MAAM,IAAI,KAAK,CAAC,GAAG,QAAQ,uBAAuB,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAE,CAAC,OAAO,EAAE,CAAC,CAAC;gBACvG,MAAM,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,GAAG,MAAM,EAAE,GAAG,GAAG,CAAC,QAAQ,CAAC;gBAC7C,MAAM,QAAQ,GAAG,OAAO,CAAC,CAAC,CAAC,EAAE,GAAG,MAAM,EAAE,CAAC,GAAG,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC;gBAClE,aAAa,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC;gBAC9B,MAAM,KAAK,GAAG;oBACZ,QAAQ,CAAC,MAAM,EAAE,IAAI,IAAI,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC;yBACjD,KAAK,CAAC,GAAG,CAAC;yBACV,IAAI,CAAC,GAAG,CAAC;iBACb,CAAC;gBACF,MAAM,SAAS,GAAG,MAAM,MAAM,EAAE,MAAM,CAAC,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,CAAC,KAAK,CAAC,EAAE,EAAE,CAAC,CAAC;gBACjF,OAAO,EAAE,QAAQ,EAAE,GAAG,CAAC,SAAS,IAAI,EAAE,QAAQ,EAAE,SAAS,CAAC,QAAQ,EAAE,CAAC,EAAE,CAAC;YAC1E,CAAC,EACD,EAAE,IAAI,EAAE,UAAU,EAAE,CACrB,CAAC;YACF,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,GAAG,CAAC,QAAQ,KAAK,SAAS,IAAI,EAAE,QAAQ,EAAE,CAAC,EAAE,CAAC;QACxE,CAAC;KACF,CAAC;AACJ,CAAC"}
|
package/dist/apply.d.ts
ADDED
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
import { z } from "zod";
|
|
2
|
+
import { type Approver, type WorkflowDefinition } from "./define.ts";
|
|
3
|
+
import { type DriftDeclared, type DriftField, type ResolveOp } from "./drift.ts";
|
|
4
|
+
import type { HostOutputs } from "./host.ts";
|
|
5
|
+
import type { Binding, Bindings, Rebind } from "./lockfile.ts";
|
|
6
|
+
import type { Op } from "./op.ts";
|
|
7
|
+
import { type PlanAction, type Resource } from "./resource.ts";
|
|
8
|
+
import type { ResourceProblem } from "./resources.ts";
|
|
9
|
+
export { APPLY_WORKFLOW } from "./shared.ts";
|
|
10
|
+
/** What a person answers for a bound object the vendor no longer has (`gone`), or a binding no declaration has (`orphan`). */
|
|
11
|
+
export declare const PLAN_ANSWERS: readonly ["recreate", "forget", "delete"];
|
|
12
|
+
export type PlanAnswer = (typeof PLAN_ANSWERS)[number];
|
|
13
|
+
/**
|
|
14
|
+
* The apply workflow's input: `only` the declared ids (`<file>#<export>`) or orphaned lockfile
|
|
15
|
+
* keys to plan, else all; `bind`, a declared id to the vendor's id of the object to bind it to;
|
|
16
|
+
* `answers`, by declared id or orphaned key: `recreate` or `forget` a gone object, `forget` or
|
|
17
|
+
* `delete` an orphaned binding's.
|
|
18
|
+
*/
|
|
19
|
+
export declare const ApplyInput: z.ZodObject<{
|
|
20
|
+
only: z.ZodOptional<z.ZodArray<z.ZodString>>;
|
|
21
|
+
bind: z.ZodOptional<z.ZodRecord<z.ZodString, z.ZodString>>;
|
|
22
|
+
answers: z.ZodOptional<z.ZodRecord<z.ZodString, z.ZodEnum<{
|
|
23
|
+
delete: "delete";
|
|
24
|
+
forget: "forget";
|
|
25
|
+
recreate: "recreate";
|
|
26
|
+
}>>>;
|
|
27
|
+
}, z.core.$strict>;
|
|
28
|
+
export type ApplyInput = z.infer<typeof ApplyInput>;
|
|
29
|
+
/**
|
|
30
|
+
* One declared resource, or orphaned binding, by what apply does with it: `create` it (as the
|
|
31
|
+
* vendor plans: `planned`, the paths `unknown` until it exists), `update` or `replace` its bound
|
|
32
|
+
* object (the declared `fields` that differ, the paths that force the replace), `bind` it to an
|
|
33
|
+
* object its identity `found` or the input `given`, ask about a `gone` object or an `orphan`ed
|
|
34
|
+
* binding (unless `answer`ed), or `none`.
|
|
35
|
+
*/
|
|
36
|
+
export type PlanStep = {
|
|
37
|
+
id: string;
|
|
38
|
+
vendor: string;
|
|
39
|
+
type: string;
|
|
40
|
+
} & ({
|
|
41
|
+
action: "create";
|
|
42
|
+
planned: Record<string, unknown>;
|
|
43
|
+
unknown: string[];
|
|
44
|
+
} | {
|
|
45
|
+
action: "update" | "replace";
|
|
46
|
+
binding: string;
|
|
47
|
+
fields: DriftField[];
|
|
48
|
+
replace: string[];
|
|
49
|
+
} | {
|
|
50
|
+
action: "bind";
|
|
51
|
+
binding: string;
|
|
52
|
+
source: "found" | "given";
|
|
53
|
+
} | {
|
|
54
|
+
action: "gone";
|
|
55
|
+
binding: string;
|
|
56
|
+
answer?: PlanAnswer;
|
|
57
|
+
} | {
|
|
58
|
+
action: "orphan";
|
|
59
|
+
key: string;
|
|
60
|
+
binding: string;
|
|
61
|
+
answer?: PlanAnswer;
|
|
62
|
+
} | {
|
|
63
|
+
action: "none";
|
|
64
|
+
binding: string;
|
|
65
|
+
});
|
|
66
|
+
/** What apply would change: the problems, the lockfile's changes it did not make, and each step in dependency order. */
|
|
67
|
+
export interface ApplyPlan {
|
|
68
|
+
problems: ResourceProblem[];
|
|
69
|
+
/** The committed lockfile against the one apply last wrote: changes someone else made. */
|
|
70
|
+
rebinds: Rebind[];
|
|
71
|
+
steps: PlanStep[];
|
|
72
|
+
}
|
|
73
|
+
/** What became of one step: `done` (a `none` too), `skipped` (a question unanswered), or `failed`, with why. */
|
|
74
|
+
export interface ApplyResult {
|
|
75
|
+
id: string;
|
|
76
|
+
action: PlanStep["action"];
|
|
77
|
+
status: "done" | "skipped" | "failed";
|
|
78
|
+
/** Its binding after the step, when it has one. */
|
|
79
|
+
binding?: string;
|
|
80
|
+
/** For a host it created or updated: where it answers, as its state says. */
|
|
81
|
+
host?: HostOutputs;
|
|
82
|
+
error?: string;
|
|
83
|
+
}
|
|
84
|
+
/** The apply workflow's output. */
|
|
85
|
+
export interface ApplyReport {
|
|
86
|
+
plan: ApplyPlan;
|
|
87
|
+
results: ApplyResult[];
|
|
88
|
+
/**
|
|
89
|
+
* Whether the run wrote the lockfile (a sandbox run keeps its copy in memory), the revision of
|
|
90
|
+
* its last commit, and the bindings it ended with.
|
|
91
|
+
*/
|
|
92
|
+
lockfile: {
|
|
93
|
+
written: boolean;
|
|
94
|
+
revision?: string;
|
|
95
|
+
bindings: Bindings;
|
|
96
|
+
};
|
|
97
|
+
}
|
|
98
|
+
/** What an apply run reads first: the data files and the lockfile, as drift does, and the lockfile's rebinds. */
|
|
99
|
+
export interface ApplyDeclared extends DriftDeclared {
|
|
100
|
+
bindings: Bindings;
|
|
101
|
+
orphans: (Binding & {
|
|
102
|
+
key: string;
|
|
103
|
+
})[];
|
|
104
|
+
rebinds: Rebind[];
|
|
105
|
+
}
|
|
106
|
+
/** One change call apply plans to make, as the policy would see it. */
|
|
107
|
+
export interface PlannedCall {
|
|
108
|
+
op: string;
|
|
109
|
+
input: unknown;
|
|
110
|
+
}
|
|
111
|
+
/** What apply does that is no vendor call, each a step of its own (`apply-steps.ts`). */
|
|
112
|
+
export interface ApplySteps {
|
|
113
|
+
declared(): Promise<ApplyDeclared>;
|
|
114
|
+
/** Who the policy would hold any of these calls for, the first it holds; undefined when it holds none. */
|
|
115
|
+
holds(calls: PlannedCall[]): Promise<Approver | undefined>;
|
|
116
|
+
/**
|
|
117
|
+
* Binds `key` to `binding` (unbinds it when undefined) in the lockfile as it is now, so a change
|
|
118
|
+
* another run made since this one read it stays, and commits it with `message`; a sandbox run
|
|
119
|
+
* writes nothing.
|
|
120
|
+
*/
|
|
121
|
+
write(key: string, binding: Binding | undefined, message: string): Promise<{
|
|
122
|
+
written: boolean;
|
|
123
|
+
revision?: string;
|
|
124
|
+
}>;
|
|
125
|
+
}
|
|
126
|
+
type State = Record<string, unknown>;
|
|
127
|
+
type Expect = {
|
|
128
|
+
action: PlanAction;
|
|
129
|
+
replace: string[];
|
|
130
|
+
};
|
|
131
|
+
type ImportOp = Op<string, string, "import", {
|
|
132
|
+
id: string;
|
|
133
|
+
}, {
|
|
134
|
+
id: string;
|
|
135
|
+
gone: boolean;
|
|
136
|
+
state?: State;
|
|
137
|
+
}>;
|
|
138
|
+
type PlanOp = Op<string, string, "plan", {
|
|
139
|
+
id?: string;
|
|
140
|
+
desired?: State;
|
|
141
|
+
}, {
|
|
142
|
+
action: PlanAction;
|
|
143
|
+
planned?: State;
|
|
144
|
+
unknown: string[];
|
|
145
|
+
replace: string[];
|
|
146
|
+
}>;
|
|
147
|
+
type CreateOp = Op<string, string, "create", {
|
|
148
|
+
desired: State;
|
|
149
|
+
}, {
|
|
150
|
+
id: string;
|
|
151
|
+
state: State;
|
|
152
|
+
}>;
|
|
153
|
+
type UpdateOp = Op<string, string, "update", {
|
|
154
|
+
id: string;
|
|
155
|
+
desired: State;
|
|
156
|
+
expect?: Expect;
|
|
157
|
+
}, {
|
|
158
|
+
id: string;
|
|
159
|
+
state: State;
|
|
160
|
+
}>;
|
|
161
|
+
type DeleteOp = Op<string, string, "delete", {
|
|
162
|
+
id: string;
|
|
163
|
+
}, {
|
|
164
|
+
id: string;
|
|
165
|
+
gone: true;
|
|
166
|
+
}>;
|
|
167
|
+
type ApplyUse = ImportOp | PlanOp | CreateOp | UpdateOp | DeleteOp | ResolveOp | "approval" | "sleep";
|
|
168
|
+
/**
|
|
169
|
+
* The apply workflow for a config: it `uses` the `import` of every resource type the connectors
|
|
170
|
+
* declare that has a driver, and the change operations (`plan`, `create`, `update`, `delete`) of
|
|
171
|
+
* each writable one, and `approval`. `resolveConfig` adds it beside drift, with `steps`, the
|
|
172
|
+
* steps that read the data files, ask the policy and write the lockfile.
|
|
173
|
+
*
|
|
174
|
+
* A run reads the data files, the lockfile and its rebinds (`steps.declared`), and observes each
|
|
175
|
+
* declared resource as drift does: one import each, by the binding the input gives, else the
|
|
176
|
+
* lockfile's, else its identity. Then, for each resource the input wants (`only`), each after the
|
|
177
|
+
* ones it names, it plans:
|
|
178
|
+
*
|
|
179
|
+
* - `bind` when the input gives an object (`bind`) or, unbound, its identity finds one;
|
|
180
|
+
* - `create` when unbound and nothing is found, as its type's `plan` says (a gone object of a
|
|
181
|
+
* `stateless` type too, unless answered);
|
|
182
|
+
* - `gone` when its bound object is no longer there: skipped unless `answers` says `recreate`
|
|
183
|
+
* (created again) or `forget` (unbound);
|
|
184
|
+
* - `update`, `replace` or `none`, as its type's `plan` says for its bound object;
|
|
185
|
+
* - `orphan` for a binding no declaration has: skipped unless `answers` says `forget` (unbound) or
|
|
186
|
+
* `delete` (deleted at the vendor, then unbound).
|
|
187
|
+
*
|
|
188
|
+
* What cannot be planned is a problem, and has no step. When the policy would hold any of the
|
|
189
|
+
* change calls the plan makes (`steps.holds`), one approval asks for them all, covering each
|
|
190
|
+
* change operation, with the plan as its details. Then each step runs in order: a change call,
|
|
191
|
+
* and right after one that changes a binding, the lockfile written and committed
|
|
192
|
+
* (`steps.write`), so an object made is never left unbound. An update is given the plan's action
|
|
193
|
+
* and replace (`expect`), and refuses (`plan_changed`) when the vendor now plans otherwise; a
|
|
194
|
+
* reference to a resource made in this run is given its new binding. A failed step is reported
|
|
195
|
+
* and the run goes on; a failure that ends the run (`isRunControlError`) ends it.
|
|
196
|
+
*/
|
|
197
|
+
export declare function applyWorkflow(types: ReadonlyMap<string, Resource>, ops: ReadonlyMap<string, Op>, drivers: ReadonlyMap<string, unknown>, steps: ApplySteps): WorkflowDefinition<readonly ApplyUse[], typeof ApplyInput>;
|