@xeplr/workflow-engine 0.0.0-stage → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +182 -2
- package/db/xcfgSetup.js +8 -0
- package/index.js +290 -0
- package/lib/actionCatalog.js +197 -0
- package/lib/actions/jobChainSteps.js +79 -0
- package/lib/actions/jobRun.js +171 -0
- package/lib/actions/screenShow.js +71 -0
- package/lib/auth.js +19 -0
- package/lib/db.js +82 -0
- package/lib/envExposed.js +87 -0
- package/lib/flows.js +916 -0
- package/lib/jobChains.js +426 -0
- package/lib/workflowRunner.js +652 -0
- package/migrations/0003_workflows.sql +35 -0
- package/migrations/0004_workflow_steps.sql +79 -0
- package/migrations/0005_workflow_runs.sql +49 -0
- package/migrations/0006_workflow_step_runs.sql +42 -0
- package/migrations/0007_workflow_resume_keys.sql +42 -0
- package/migrations/0008_workflow_run_edges.sql +43 -0
- package/migrations/0009_workflow_steps_layout.sql +11 -0
- package/migrations/0010_workflow_steps_sample_output.sql +17 -0
- package/migrations/0011_workflow_steps_params.sql +27 -0
- package/migrations/0012_workflows_kind.sql +27 -0
- package/migrations/0013_workflows_key.sql +48 -0
- package/migrations/0014_workflows_trigger.sql +30 -0
- package/migrations-tenants/0001_companies.sql +23 -0
- package/migrations-tenants/0002_workspaces.sql +22 -0
- package/models/Company.js +82 -0
- package/models/Workflow.js +67 -0
- package/models/WorkflowResumeKey.js +32 -0
- package/models/WorkflowRun.js +64 -0
- package/models/WorkflowRunEdge.js +49 -0
- package/models/WorkflowStep.js +66 -0
- package/models/WorkflowStepRun.js +49 -0
- package/models/Workspace.js +75 -0
- package/models/index.js +25 -0
- package/package.json +47 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Xeplr
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,183 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @xeplr/workflow-engine
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**The workflow engine, with no HTTP in it.** It owns workflow's database and tables, the action catalog, starting and resuming runs, and the flows facade (screens as steps). [`@xeplr/workflow-api`](https://github.com/Xeplr/xeplr-workflow-api) puts the routes on top.
|
|
4
|
+
|
|
5
|
+
**Free and open source (MIT, [LICENSE](LICENSE))**, on npm as `@xeplr/workflow-engine` and on GitHub at [Xeplr/xeplr-workflow-engine](https://github.com/Xeplr/xeplr-workflow-engine). With [`@xeplr/workflow-api`](https://github.com/Xeplr/xeplr-workflow-api) it is a complete base to automate with and to build your own workflow screens on: every action in `@xeplr/actions` (databases, files, email…), branching, fan-out, waits, job chains and screens as steps. **X-Flow**, Xeplr's premium workflow product, is built on these two packages and adds the visual designer (steps drawn and connected on a canvas) and data operations between steps; neither is part of this package.
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install @xeplr/workflow-engine
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
## Use
|
|
12
|
+
|
|
13
|
+
```js
|
|
14
|
+
var engine = require('@xeplr/workflow-engine');
|
|
15
|
+
|
|
16
|
+
await engine.startEngine({
|
|
17
|
+
db: { name: 'xeplr_bi_workflow', connection: conn, connectionName: 'workflow' }
|
|
18
|
+
});
|
|
19
|
+
var run = await engine.runner.startRun(workflowId, params, { user });
|
|
20
|
+
await engine.resumeByKey(key, { confirmedBy: userId });
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Most hosts never call this directly: the API's `registerWorkflow()` calls `startEngine()` first and passes `resumeByKey` and the env lists through.
|
|
24
|
+
|
|
25
|
+
The shared `@xeplr/*` libraries (`actions`, `db`, `expression-handler`, `schema-handler`, `utils`) are **peer dependencies**. The host supplies them, so the engine reads the same tenant context and the same action registry as the host. Two copies of `@xeplr/db` would mean two tenant contexts, and a run started in one would not see the other's company.
|
|
26
|
+
|
|
27
|
+
## Exports (`index.js`)
|
|
28
|
+
|
|
29
|
+
| Export | Purpose |
|
|
30
|
+
|---|---|
|
|
31
|
+
| `startEngine(config)` | Connect workflow's database (creating it if missing), run its migrations, ready `xeplr_configs`, register the action catalog. Idempotent. `config.db` is required; throws if no application id is registered. Config: `db` (`{ name, connection, connectionName?, mts?, migrationsTable? }`), `applicationId`, `tenantTables`, `migrations`, `migrate` (`false`: connect without migrating — a second process of an install whose API migrates), `startJob` (below) |
|
|
32
|
+
| `resumeByKey(key, output?, opts?)` | Server-side, in-process resume of a run parked on a `wait` step. `opts.status: 'failed'` resolves the step as failed. Throws `err.code` `RESUME_KEY_INVALID` (used/unknown, normal for host routes) or `RESUME_KEY_NOT_READY` (step still running; nothing consumed, retry) |
|
|
33
|
+
| `embedRequiredEnv` | `['DB_WORKFLOW']`, spread into a host's `env.required.js` |
|
|
34
|
+
| `callbackRequiredEnv` | `['WORKFLOW_PUBLIC_URL']`, only for hosts using steps that call out (`job-run`) |
|
|
35
|
+
| `MIGRATIONS_DIR`, `TENANT_MIGRATIONS_DIR` | Where the SQL is, for the standalone server and for a bundle that ships the folders |
|
|
36
|
+
| `db` | `connectDb`, with `.model(name)` bound to workflow's own connection |
|
|
37
|
+
| `runner` | `lib/workflowRunner.js`: `startRun`, `resumeByKey`, `resolveParams`, `collectParams` |
|
|
38
|
+
| `flows` | `lib/flows.js`: the flows facade (list, create, put, publish, start, submit, runs) and `touchesScreens` |
|
|
39
|
+
| `jobChains` | `lib/jobChains.js`: a job's "After this job" rules as a workflow — `getChain`, `putChain`, `jobFinished`, `setJobStarter`, `touchesChains` ([Job chains](#job-chains-libjobchainsjs)) |
|
|
40
|
+
| `actionCatalog`, `envExposed`, `xcfgSetup` | The action whitelist, the `WORKFLOW_ENV_EXPOSED` allowlist, the shared `xeplr_configs` attachment |
|
|
41
|
+
|
|
42
|
+
The package reads `migrations/` and `migrations-tenants/` from disk (`"xeplr": { "readsFromDisk": [...] }` in `package.json`), so a build that compiles it must ship those folders beside it.
|
|
43
|
+
|
|
44
|
+
## Structure
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
engine/
|
|
48
|
+
├─ index.js startEngine, resumeByKey, the env lists, and the parts below
|
|
49
|
+
├─ lib/
|
|
50
|
+
│ ├─ workflowRunner.js startRun, resumeByKey, resolveParams, collectParams
|
|
51
|
+
│ ├─ flows.js the flows facade: screens as steps
|
|
52
|
+
│ ├─ jobChains.js a job's "After this job" rules, kept as a workflow
|
|
53
|
+
│ ├─ actionCatalog.js whitelist of @xeplr/actions built-ins + local actions
|
|
54
|
+
│ ├─ actions/ local actions: job-run, screen-show, job-chain-decide/-next
|
|
55
|
+
│ ├─ envExposed.js WORKFLOW_ENV_EXPOSED allowlist for {env.*}
|
|
56
|
+
│ └─ db.js connectDb + model(name) bound to workflow's own connection
|
|
57
|
+
├─ db/xcfgSetup.js shared @xeplr/actions attachConfig({ service: 'xeplr-workflow' })
|
|
58
|
+
├─ models/ Company, Workspace, Workflow, WorkflowStep, WorkflowRun,
|
|
59
|
+
│ WorkflowStepRun, WorkflowResumeKey, WorkflowRunEdge
|
|
60
|
+
├─ migrations/ 0003–0014, workflow's own tables
|
|
61
|
+
├─ migrations-tenants/ 0001–0002: standalone workflow's companies / workspaces
|
|
62
|
+
└─ test/ run.mjs + *.test.mjs
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Data model and migrations
|
|
66
|
+
|
|
67
|
+
Both run against workflow's own database:
|
|
68
|
+
|
|
69
|
+
| Directory | Target DB | How it runs |
|
|
70
|
+
|---|---|---|
|
|
71
|
+
| `migrations/` | workflow's own (`DB_API` standalone, `DB_WORKFLOW` in BI) | On every boot inside `startEngine` (and once more in the API's `standalone.js`); idempotent. Also the API's `npm run migrate:up`. Plain `.sql`, ledger-tracked by filename, `precede` order |
|
|
72
|
+
| `migrations-tenants/` | the same | Inside `startEngine` unless `tenantTables: false` |
|
|
73
|
+
|
|
74
|
+
| Migration | Creates |
|
|
75
|
+
|---|---|
|
|
76
|
+
| `0001_companies` | `companies` (l1 tenant) |
|
|
77
|
+
| `0002_workspaces` | `workspaces` (l2, under a company) |
|
|
78
|
+
| `0003_workflows` | `workflows` — `name`, `description`, `params` (jsonb), `status` (default `draft`) |
|
|
79
|
+
| `0004_workflow_steps` | `workflow_steps` — `stepKey` (unique per workflow), `actionName`, `values`, `kind` (`auto`/`wait`), `timeoutMs`, `onError` (`stop`/`continue`), `transitions`, `joinStep`, `position` |
|
|
80
|
+
| `0005_workflow_runs` | `workflow_runs` — `status` (`queued`/`running`/`waiting`/`success`/`failed`), `params`, `item`, `trigger`, timings, `error` |
|
|
81
|
+
| `0006_workflow_step_runs` | `workflow_step_runs` — copied `stepKey`/`actionName`, resolved `input`, `output`, `error` |
|
|
82
|
+
| `0007_workflow_resume_keys` | `workflow_resume_keys` — unique random `key`, `consumedDate` |
|
|
83
|
+
| `0008_workflow_run_edges` | `workflow_run_edges` — parent → child run for `each` fan-out; `childRunId` unique |
|
|
84
|
+
| `0009` | `workflow_steps.layout` (canvas `{x, y}`) |
|
|
85
|
+
| `0010` | `workflow_steps.sampleOutput` |
|
|
86
|
+
| `0011` | `workflow_steps.params` |
|
|
87
|
+
| `0012` | `workflows.kind` (`workflow` \| `jobs`, default `workflow`) |
|
|
88
|
+
| `0013` | `workflows.key` — a stable name a host addresses a flow by, unique per tenant while it is set |
|
|
89
|
+
| `0014` | `workflows.trigger` (jsonb) — **what is supposed to start a run**, beside what comes in with it. One shape per kind: `{kind:'manual'}`, `{kind:'api'}`, `{kind:'schedule',cron}`, `{kind:'file',folder,named}`, `{kind:'email',mailbox,folder}`. NULL means "started however it is called", so every existing workflow behaves exactly as it did. **Nothing arms these yet** — the designer records the intent; the pollers and the hook endpoint read it when they land |
|
|
90
|
+
|
|
91
|
+
Every table carries `mtId1`–`mtId4` and **no `applicationId`**: the database a
|
|
92
|
+
host gives workflow is the boundary.
|
|
93
|
+
|
|
94
|
+
## How a run behaves (`lib/workflowRunner.js`)
|
|
95
|
+
|
|
96
|
+
- **Step values** are interpolated with `@xeplr/schema-handler` against
|
|
97
|
+
`{ params, item, steps.<key>.output, previous_step.output, env, resumeKey, resumeUrl }`.
|
|
98
|
+
`previous_step`/`steps` come from what ran in this run, not array position.
|
|
99
|
+
- **Routing**: no `transitions` → next step by `position`. With transitions →
|
|
100
|
+
first match wins (`@xeplr/expression-handler`); a blank condition is the
|
|
101
|
+
catch-all; targets are a `stepKey`, `end_success` or `end_failed`. No match
|
|
102
|
+
and no catch-all **fails the run**.
|
|
103
|
+
- **Fan-out**: a matching `each` transition creates one child run per matching
|
|
104
|
+
element (`item`), recorded in `workflow_run_edges`. With `joinStep` the parent
|
|
105
|
+
waits and resumes at the join once all children are terminal (guarded by an
|
|
106
|
+
atomic `waiting → running` patch); without it the parent finishes.
|
|
107
|
+
- **Wait steps**: the resume key row is written **before** the action runs;
|
|
108
|
+
the run parks with status `waiting`. If the action fails the key is marked
|
|
109
|
+
consumed.
|
|
110
|
+
- **Action failure**: `onError: 'continue'` advances by position; otherwise the
|
|
111
|
+
run fails.
|
|
112
|
+
- **Run params**: the run schema is the union of every step's `params` (plus
|
|
113
|
+
legacy `workflows.params` first). Any step requiring a name makes it required;
|
|
114
|
+
otherwise first declaration by position wins. Validated with `applySchema`
|
|
115
|
+
**before** the run row is inserted; undeclared keys are dropped; failure has
|
|
116
|
+
`code: 'PARAMS_INVALID'` and `details`. A workflow that declares nothing
|
|
117
|
+
accepts params unchanged.
|
|
118
|
+
- **A wait step's own key** reaches its action as `system.resumeKey` (and
|
|
119
|
+
`{resumeKey}` / `{resumeUrl}` in its values), so an action can hand it to
|
|
120
|
+
something in the same process instead of over a URL.
|
|
121
|
+
- **Tenant context**: after the first lookup, all DB work runs inside
|
|
122
|
+
`runWithMt()` with the run's stored `mtId`s, because a resume can arrive long
|
|
123
|
+
after the starting request.
|
|
124
|
+
|
|
125
|
+
Not read by the runner: `workflows.status`, `workflows.kind`,
|
|
126
|
+
`workflow_steps.layout`, `workflow_steps.sampleOutput`, `workflow_steps.timeoutMs`.
|
|
127
|
+
|
|
128
|
+
## Job chains (`lib/jobChains.js`)
|
|
129
|
+
|
|
130
|
+
**Every hand-off between jobs goes through this engine.** `@xeplr/jobs` never
|
|
131
|
+
starts another job; a job's screen keeps a list of "After this job" rules, and
|
|
132
|
+
`putChain(jobId, { rules, jobName })` turns them into one hidden workflow,
|
|
133
|
+
kind `job-chain`, key `job-chain-<jobId>`:
|
|
134
|
+
|
|
135
|
+
- `decide` (`job-chain-decide`, does nothing): its transitions are the rules, first match wins, and no match ends the run successfully;
|
|
136
|
+
- `run_1 … run_n` (`job-chain-next`, wait steps): start that rule's job and park until it reports back.
|
|
137
|
+
|
|
138
|
+
A rule is `{ when, run, inputs?, carry? }`:
|
|
139
|
+
|
|
140
|
+
```json
|
|
141
|
+
[
|
|
142
|
+
{ "when": { "field": "output.totalRows", "op": ">", "value": 0 }, "run": "<jobId>",
|
|
143
|
+
"inputs": { "region": "{output.region}" }, "carry": { "rows": "{output.totalRows}" } },
|
|
144
|
+
{ "when": { "all": [ { "field": "status", "op": "=", "value": "failed" } ] }, "run": "<jobId>" },
|
|
145
|
+
{ "run": "<jobId>" }
|
|
146
|
+
]
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
- **`when`** is `{ field, op, value }`, `{ all: [...] }`, `{ any: [...] }`, or empty for "otherwise" (last only). Fields start with `status` (`success`/`failed`), `output.` (what the finished job's action returned), `window.`, `carry.`, `error.` or `jobId`. The comparisons are the flows facade's (`flows.comparisonNode`), so `=` means one thing everywhere.
|
|
150
|
+
- **The period flows on.** The next job is started with the finished job's `window` unless the rule's `inputs` set one, so a chain covers one period from its first job to its last.
|
|
151
|
+
- **`inputs`** fill inputs of the next job — only `window` and the names that job lists in `acceptsInputs` (`@xeplr/jobs` refuses the rest, and the run fails saying which).
|
|
152
|
+
- **`carry`** adds values for later jobs, read as `carry.<name>`. Bindings like `{output.x}` keep their type when they are the whole value.
|
|
153
|
+
- **Loops stop**: a job that has already run in this chain, or a chain 25 jobs long, fails the step with a message.
|
|
154
|
+
|
|
155
|
+
`putChain` with an empty list sets the workflow back to draft (its runs stay). `getChain` answers the rules as written (kept on the `decide` step). A `job-chain` workflow cannot be saved through `/workflows/save` (`touchesChains`), like a screens flow.
|
|
156
|
+
|
|
157
|
+
**The host connects jobs, in-process.** The engine never imports jobs:
|
|
158
|
+
|
|
159
|
+
```js
|
|
160
|
+
await engine.startEngine({ db, startJob: (jobId, body) => jobs.trigger(jobId, body) });
|
|
161
|
+
await jobs.init({ /* … */ onFinish: (report) => engine.jobChains.jobFinished(report) });
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
`jobFinished(report)` takes `@xeplr/jobs`' finish envelope. If the run was started by a chain step it resumes that step (retrying while the step parks), and if the finished job has rules it starts its chain with params `{ jobId, status, error, output, window, carry, path }`. With `startJob` set, the canvas's `job-run` step starts jobs the same way, with no `jobsUrl` and no HTTP call.
|
|
165
|
+
|
|
166
|
+
## Tests
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
npm test # node test/run.mjs — each *.test.mjs in its own process
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
No database or network needed, except `jobChains.test.mjs`, which runs on a real Postgres when `WORKFLOW_TEST_PG='{"host":…,"port":…,"user":…,"password":…}'` is set (scratch databases, created and dropped) and is skipped otherwise. CI and the release start a throwaway Postgres and set it, so neither ships on a skipped suite. `npm install` first: the peers install with it.
|
|
173
|
+
|
|
174
|
+
| Suite | Covers |
|
|
175
|
+
|---|---|
|
|
176
|
+
| `jobChains.test.mjs` | "After this job" with the real `@xeplr/jobs` (a dev dependency) wired as a host wires it: rules ⇄ workflow, every refusal, A → B → D with A's period and carried values, otherwise, no match, a failed job followed, loops and over-long chains stopped, an input the next job does not accept, `job-run` in-process, rules removed |
|
|
177
|
+
| `actionCatalog.test.mjs` | Ready vs placeholder detection; registered names are kebab-case; `spawnProgram` not offered |
|
|
178
|
+
| `envExposed.test.mjs` | Nothing exposed by default; exact, case-sensitive matching; globs fail closed (against real `interpolateAll`) |
|
|
179
|
+
| `requires.test.mjs` | Every relative `require` in this package points at a file (Node only notices a missing one when the line runs; a bundler refuses it up front) |
|
|
180
|
+
| `paths.test.mjs` | `MIGRATIONS_DIR` and `TENANT_MIGRATIONS_DIR` exist inside the package, hold SQL, and are listed in `readsFromDisk` (an undefined one makes the migrator run the HOST's `./migrations` against workflow's database) |
|
|
181
|
+
| `runParams.test.mjs` | `resolveParams` / `collectParams`: required, defaults, types, typos, union rules |
|
|
182
|
+
|
|
183
|
+
The flows facade is tested from `@xeplr/workflow-api` (`test/flows.test.mjs`), through its real router.
|
package/db/xcfgSetup.js
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
// Single shared instance of @xeplr/actions' attachConfig() — so bin/www (which
|
|
2
|
+
// calls ready()) and anything else reading control-plane metadata see the same
|
|
3
|
+
// connection. All the bootstrap/bind/metaStore logic lives in @xeplr/actions;
|
|
4
|
+
// this file is just the one place xeplr-workflow holds its instance, naming
|
|
5
|
+
// itself 'xeplr-workflow'.
|
|
6
|
+
var { attachConfig } = require('@xeplr/actions');
|
|
7
|
+
|
|
8
|
+
module.exports = attachConfig({ service: 'xeplr-workflow' });
|
package/index.js
ADDED
|
@@ -0,0 +1,290 @@
|
|
|
1
|
+
var path = require('path');
|
|
2
|
+
var { up } = require('@xeplr/db').sqlMigrator;
|
|
3
|
+
var { resolveConfig, registerApplication, getApplicationId, migrationsFor, ensureDatabaseFor } = require('@xeplr/db');
|
|
4
|
+
var connectDb = require('./lib/db');
|
|
5
|
+
var actionCatalog = require('./lib/actionCatalog');
|
|
6
|
+
var envExposed = require('./lib/envExposed');
|
|
7
|
+
var workflowRunner = require('./lib/workflowRunner');
|
|
8
|
+
var flows = require('./lib/flows');
|
|
9
|
+
var jobChains = require('./lib/jobChains');
|
|
10
|
+
var xcfgSetup = require('./db/xcfgSetup');
|
|
11
|
+
|
|
12
|
+
// THE WORKFLOW ENGINE — everything that runs a workflow, with no HTTP in it:
|
|
13
|
+
// its database and tables, the action catalog, starting and resuming runs, and
|
|
14
|
+
// the flows facade (screens as steps). @xeplr/workflow-api puts routes on top
|
|
15
|
+
// of this; a host that only RUNS workflows (a job that starts one, a link that
|
|
16
|
+
// resumes one) can use this alone.
|
|
17
|
+
|
|
18
|
+
// Where the engine's SQL lives. Exported so a deployment that ships these
|
|
19
|
+
// folders somewhere else (a compiled bundle) and the standalone server read
|
|
20
|
+
// the same paths rather than each re-deriving them.
|
|
21
|
+
var MIGRATIONS_DIR = path.join(__dirname, 'migrations');
|
|
22
|
+
var TENANT_MIGRATIONS_DIR = path.join(__dirname, 'migrations-tenants');
|
|
23
|
+
|
|
24
|
+
/**
|
|
25
|
+
* Connect workflow's own database, bring its tables up to date and register
|
|
26
|
+
* its actions. Idempotent; the API's registerWorkflow() calls it first.
|
|
27
|
+
*
|
|
28
|
+
* @param {object} config
|
|
29
|
+
* @param {object} config.db - { name, connection, mts?, connectionName?, migrationsTable? } —
|
|
30
|
+
* see lib/db.js. migrationsTable: workflow's own migration ledger, for a
|
|
31
|
+
* database it SHARES with its host (default: @xeplr/db's xeplr_migrations).
|
|
32
|
+
* @param {boolean} [config.tenantTables=true] - false: do not create
|
|
33
|
+
* standalone workflow's own companies / workspaces tables.
|
|
34
|
+
* @param {string} [config.applicationId] - the PRODUCT running this; see below.
|
|
35
|
+
* @param {string|string[]} [config.migrations] - a host's extra migrations
|
|
36
|
+
* for workflow's database (default: XEPLR_WORKFLOW_MIGRATIONS).
|
|
37
|
+
* @param {boolean} [config.migrate=true] - false: connect without running
|
|
38
|
+
* migrations, for a second process of an install whose API migrates.
|
|
39
|
+
* @param {Function} [config.startJob] - async (jobId, { inputs, chain,
|
|
40
|
+
* callbackUrl }) → { occurrenceId }: how this host starts a job. With
|
|
41
|
+
* @xeplr/jobs in the same process, `(id, body) => jobs.trigger(id, body)`.
|
|
42
|
+
* Used by "After this job" chains and by the job-run step. Unset, those
|
|
43
|
+
* steps fail and say so (job-run falls back to its jobsUrl over HTTP).
|
|
44
|
+
*/
|
|
45
|
+
async function startEngine(config) {
|
|
46
|
+
config = config || {};
|
|
47
|
+
if (config.startJob !== undefined) jobChains.setJobStarter(config.startJob);
|
|
48
|
+
if (!config.db) {
|
|
49
|
+
throw new Error('startEngine: config.db is required — workflow owns its own domain data regardless of how it is mounted');
|
|
50
|
+
}
|
|
51
|
+
// WORKFLOW'S OWN TABLES CARRY NO applicationId. Each host supplies its own
|
|
52
|
+
// database (config.db), so that database is the boundary — a column holding
|
|
53
|
+
// one constant value in every row would buy nothing, which is the same
|
|
54
|
+
// reason xeplr_bi and the auth DBs do not have one either.
|
|
55
|
+
//
|
|
56
|
+
// The identity is still REQUIRED, because this mount also writes to
|
|
57
|
+
// xeplr_configs — the one store that genuinely is shared by every app and
|
|
58
|
+
// cannot be split per consumer (reference data, plus import_meta). Rows
|
|
59
|
+
// there are attributed by applicationId, so a mount that skipped this would
|
|
60
|
+
// quietly file its movement metadata under the package name instead of the
|
|
61
|
+
// product's.
|
|
62
|
+
//
|
|
63
|
+
// Inherited by default: a host registers itself once at boot and every
|
|
64
|
+
// sub-product it mounts picks it up with no further wiring. config
|
|
65
|
+
// .applicationId is accepted for a host that would rather be explicit, and
|
|
66
|
+
// is idempotent under the same value.
|
|
67
|
+
if (config.applicationId) registerApplication(config.applicationId);
|
|
68
|
+
if (!getApplicationId()) {
|
|
69
|
+
throw new Error(
|
|
70
|
+
'startEngine: no application registered. Call registerApplication("<this-product>") ' +
|
|
71
|
+
'from @xeplr/db once at host startup (or pass config.applicationId). It attributes the ' +
|
|
72
|
+
'rows this mount writes to the SHARED xeplr_configs database; workflow\'s own tables are ' +
|
|
73
|
+
'separated by having their own database, not by this.');
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
await connectDb(config.db);
|
|
77
|
+
|
|
78
|
+
// ITS OWN TABLES, ITS OWN JOB — for the same reason config.db is required in
|
|
79
|
+
// both modes. This lived in orchestration/standalone.js, so a host that
|
|
80
|
+
// embedded workflow got a connection to a database with none of workflow's
|
|
81
|
+
// tables in it, and the first request failed on a missing relation.
|
|
82
|
+
//
|
|
83
|
+
// Idempotent, so a host that also runs them loses nothing by this.
|
|
84
|
+
//
|
|
85
|
+
// The migrator resolves its connection BY NAME from a registry that
|
|
86
|
+
// getConnection above does not populate, so the name has to be resolved
|
|
87
|
+
// explicitly first. Without this an embedded mount died on
|
|
88
|
+
// 'No resolved config for "WORKFLOW"' and the whole product silently did
|
|
89
|
+
// not appear — while every /workflow/* path still answered 401, because the
|
|
90
|
+
// host's auth middleware runs before routing and a missing route never got
|
|
91
|
+
// the chance to 404.
|
|
92
|
+
var migrationConnection = config.db.connectionName || 'workflow';
|
|
93
|
+
await resolveConfig(migrationConnection, config.db.connection);
|
|
94
|
+
|
|
95
|
+
// CREATE IT IF IT IS NOT THERE. Each host gives workflow its own database
|
|
96
|
+
// (xeplr_bi_workflow, xeplr_erp_workflow) — that separation is the entire
|
|
97
|
+
// isolation model, so pointing a host at a name that does not exist yet is
|
|
98
|
+
// the NORMAL first boot, not an error state. Without this it failed on
|
|
99
|
+
// "database does not exist" and every host had to run a manual createdb
|
|
100
|
+
// that nothing in the config hinted at. Same as @xeplr/email does.
|
|
101
|
+
// migrate: false — connect only. For a second process of the same install
|
|
102
|
+
// (BI's separate jobs service) that must not race the API's own migration
|
|
103
|
+
// run at deploy: the API migrates, this one uses the tables it made.
|
|
104
|
+
if (config.migrate !== false) {
|
|
105
|
+
var ensured = await ensureDatabaseFor(config.db.connection, config.db.name);
|
|
106
|
+
if (ensured.created) console.log('[workflow] created database ' + config.db.name);
|
|
107
|
+
// XEPLR_WORKFLOW_MIGRATIONS — a host adds its own tables or seed rows to
|
|
108
|
+
// workflow's database without editing this package. Same convention as
|
|
109
|
+
// every other xeplr library (see @xeplr/db's app-migrations.js); config
|
|
110
|
+
// .migrations lets an embedding host pass them directly instead, since a
|
|
111
|
+
// host that already has the paths in code should not have to route them
|
|
112
|
+
// back out through the environment.
|
|
113
|
+
//
|
|
114
|
+
// SHARING A DATABASE. A host may give workflow its OWN database (BI's
|
|
115
|
+
// xeplr_bi_workflow) or put it in the host's (an app from @xeplr/cli). In a
|
|
116
|
+
// shared one, config.db.migrationsTable keeps workflow's record of what ran
|
|
117
|
+
// apart from the host's — both use @xeplr/db's migrator, and one ledger
|
|
118
|
+
// would mix the two histories by filename.
|
|
119
|
+
var ledger = config.db.migrationsTable || undefined;
|
|
120
|
+
|
|
121
|
+
// companies / workspaces are STANDALONE workflow's own tenant lists — the
|
|
122
|
+
// lists behind its "pick a company, then a workspace" screens. Nothing in a
|
|
123
|
+
// workflow, step or run references them (runs carry mtId1/mtId2 as plain
|
|
124
|
+
// ids), and an embedded host has its own tenants, or none — so a host says
|
|
125
|
+
// tenantTables: false and they are not created. That is also what lets
|
|
126
|
+
// workflow share a database that already has a "companies" table.
|
|
127
|
+
if (config.tenantTables !== false) {
|
|
128
|
+
var tenants = await up({
|
|
129
|
+
db: config.db.name,
|
|
130
|
+
dir: TENANT_MIGRATIONS_DIR,
|
|
131
|
+
type: 'precede',
|
|
132
|
+
connectionName: migrationConnection,
|
|
133
|
+
tableName: ledger
|
|
134
|
+
});
|
|
135
|
+
if (tenants.migrations.length) console.log('[workflow] ran ' + tenants.migrations.length + ' tenant-table migrations');
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
var migrated = await up({
|
|
139
|
+
db: config.db.name,
|
|
140
|
+
dir: MIGRATIONS_DIR,
|
|
141
|
+
extDir: config.migrations || migrationsFor('workflow'),
|
|
142
|
+
type: 'precede',
|
|
143
|
+
connectionName: migrationConnection,
|
|
144
|
+
tableName: ledger
|
|
145
|
+
});
|
|
146
|
+
if (migrated.migrations.length) {
|
|
147
|
+
console.log('[workflow] ran ' + migrated.migrations.length + ' migrations');
|
|
148
|
+
}
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
// THE ACTION CATALOG, for the same reason the migrations above moved here.
|
|
154
|
+
// This lived only in orchestration/standalone.js, so an embedded host got a
|
|
155
|
+
// mounted product whose registry was empty: GET /actions answered 200 with
|
|
156
|
+
// an empty list, the step editor's Action dropdown drew nothing, and there
|
|
157
|
+
// was no error anywhere to explain why — the one failure mode the whitelist
|
|
158
|
+
// in lib/actionCatalog.js is otherwise careful to make loud.
|
|
159
|
+
//
|
|
160
|
+
// Idempotent (register() overwrites by name), so a host that somehow also
|
|
161
|
+
// registers them loses nothing.
|
|
162
|
+
//
|
|
163
|
+
// xeplr_configs is best-effort HERE and fail-fast in standalone, and the
|
|
164
|
+
// difference is deliberate: standalone owns its whole process and should
|
|
165
|
+
// refuse to boot half-configured, whereas failing a host's mount over one
|
|
166
|
+
// action that records movement metadata would take down a product that is
|
|
167
|
+
// otherwise entirely functional. The warning names what was lost.
|
|
168
|
+
try {
|
|
169
|
+
await xcfgSetup.ready();
|
|
170
|
+
} catch (err) {
|
|
171
|
+
console.warn('[workflow] xeplr_configs unavailable (' + err.message + ') — actions needing it are not offered');
|
|
172
|
+
}
|
|
173
|
+
var registered = actionCatalog.registerAll();
|
|
174
|
+
console.log('[workflow] registered ' + registered.length + ' actions: ' + registered.join(', '));
|
|
175
|
+
|
|
176
|
+
// Say out loud what step templates can read out of the environment. Silent
|
|
177
|
+
// by default (nothing exposed unless WORKFLOW_ENV_EXPOSED names it), so a
|
|
178
|
+
// line here means somebody opted in — and a list that accidentally names a
|
|
179
|
+
// secret is only catchable if it is visible somewhere.
|
|
180
|
+
envExposed.logExposure();
|
|
181
|
+
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
// Env vars a HOST embedding workflow must supply — spread into the host's
|
|
185
|
+
// env.required.js so a missing one fails at startup rather than at mount:
|
|
186
|
+
//
|
|
187
|
+
// ...require('@xeplr/workflow-engine').embedRequiredEnv, // DB_WORKFLOW
|
|
188
|
+
//
|
|
189
|
+
// Named `embedRequiredEnv`, not `requiredEnv`, because it is not the same
|
|
190
|
+
// list as this package's OWN standalone process needs (see env.required.js,
|
|
191
|
+
// which uses DB_API for the very same database — a standalone deployment
|
|
192
|
+
// names its own store, an embedded one is given a separate one by its host).
|
|
193
|
+
// Two deployments, two questions, two lists.
|
|
194
|
+
//
|
|
195
|
+
// DB_WORKFLOW only: the SERVER login comes from the shared
|
|
196
|
+
// XEPLR_DB_CONNECTION, and workflow never reads either itself — the host
|
|
197
|
+
// passes both into registerWorkflow({ db }). This list exists so the host is
|
|
198
|
+
// told what to pass before it boots.
|
|
199
|
+
var embedRequiredEnv = [
|
|
200
|
+
'DB_WORKFLOW'
|
|
201
|
+
];
|
|
202
|
+
|
|
203
|
+
/**
|
|
204
|
+
* Resume a run parked on a 'wait' step. THE ENTIRE consumer-side contract for
|
|
205
|
+
* waiting — the caller never needs to know which run or step the key belongs
|
|
206
|
+
* to, because that lookup lives in workflow_resume_keys alone.
|
|
207
|
+
*
|
|
208
|
+
* var { resumeByKey } = require('@xeplr/workflow-engine');
|
|
209
|
+
* await resumeByKey(key, { confirmedBy: userId });
|
|
210
|
+
*
|
|
211
|
+
* SERVER-SIDE, IN-PROCESS, DELIBERATELY. This is not something a browser
|
|
212
|
+
* calls. The host's own route — the activation link's landing route, the
|
|
213
|
+
* confirm button's POST — does its own work (activate the account, record the
|
|
214
|
+
* approval) and then calls this in the same request. The key travels out
|
|
215
|
+
* bound into that route's URL via `{resumeKey}` and comes back to the host,
|
|
216
|
+
* never to workflow's own HTTP surface.
|
|
217
|
+
*
|
|
218
|
+
* That is the whole reason it is exported here rather than left as a route: a
|
|
219
|
+
* resume is a step in the host's transaction, not a separate thing a client
|
|
220
|
+
* is trusted to trigger.
|
|
221
|
+
*
|
|
222
|
+
* Throws with `err.code` set, which is what a host should branch on:
|
|
223
|
+
*
|
|
224
|
+
* RESUME_KEY_INVALID no live key — already used, unknown, or the run is
|
|
225
|
+
* gone. NORMAL for a host route that also serves
|
|
226
|
+
* users who never came from a workflow: catch it and
|
|
227
|
+
* carry on with your own work.
|
|
228
|
+
* RESUME_KEY_NOT_READY the key is real but its step has not parked yet
|
|
229
|
+
* (the action that delivers it is still running).
|
|
230
|
+
* Nothing is consumed — retry shortly.
|
|
231
|
+
*
|
|
232
|
+
* Requires startEngine() (or the API's registerWorkflow(), which calls it) to have run in this process: it is what
|
|
233
|
+
* connects workflow's database, and this reads through that same connection.
|
|
234
|
+
* Needs no ambient tenant context — the run carries its own mtIds and this
|
|
235
|
+
* re-enters them itself.
|
|
236
|
+
*
|
|
237
|
+
* @param {string} key the value bound into the step as {resumeKey}
|
|
238
|
+
* @param {object} [output] merged into the wait step's output, so later
|
|
239
|
+
* steps can read it as {steps.<stepKey>.output.<name>}
|
|
240
|
+
* @param {object} [opts]
|
|
241
|
+
* @param {'success'|'failed'} [opts.status] resolve the step as FAILED rather
|
|
242
|
+
* than succeeded — from there the step's own `onError` decides, defaulting
|
|
243
|
+
* to stopping the run. Omit for success, which is what every caller that
|
|
244
|
+
* predates this means (an approval click, a confirmation link).
|
|
245
|
+
* @param {object} [opts.error] recorded on the step run when status is
|
|
246
|
+
* 'failed'.
|
|
247
|
+
* @returns {Promise<{ runId: string, stepKey: string, status: string }>}
|
|
248
|
+
*/
|
|
249
|
+
async function resumeByKey(key, output, opts) {
|
|
250
|
+
return workflowRunner.resumeByKey(key, output, opts);
|
|
251
|
+
}
|
|
252
|
+
|
|
253
|
+
// REQUIRED ONLY IF YOU USE STEPS THAT CALL SOMETHING OUTSIDE THIS PROCESS —
|
|
254
|
+
// today that is `job-run`. Kept as its own list, the same way @xeplr/email
|
|
255
|
+
// separates `requiredEnv` (sending) from `templatesRequiredEnv` (the template
|
|
256
|
+
// store): a host that only runs self-contained workflows is never made to
|
|
257
|
+
// configure something it does not use.
|
|
258
|
+
//
|
|
259
|
+
// WORKFLOW_PUBLIC_URL is the base a CALLER can reach this service on, mount
|
|
260
|
+
// path included — https://bi.example.com/workflow for an embedded mount. It is
|
|
261
|
+
// what {resumeUrl} is built from, so a step can hand a job an address to
|
|
262
|
+
// report back to.
|
|
263
|
+
//
|
|
264
|
+
// NOT DEFAULTED, and that is the point of it being here rather than having a
|
|
265
|
+
// fallback: a wrong-but-present address gives a job somewhere to POST that
|
|
266
|
+
// nothing is listening on, so the run waits forever on work that actually
|
|
267
|
+
// finished. Unset, {resumeUrl} is null and job-run refuses by name before
|
|
268
|
+
// starting anything.
|
|
269
|
+
var callbackRequiredEnv = [
|
|
270
|
+
'WORKFLOW_PUBLIC_URL'
|
|
271
|
+
];
|
|
272
|
+
|
|
273
|
+
module.exports = {
|
|
274
|
+
startEngine: startEngine,
|
|
275
|
+
resumeByKey: resumeByKey,
|
|
276
|
+
embedRequiredEnv: embedRequiredEnv,
|
|
277
|
+
callbackRequiredEnv: callbackRequiredEnv,
|
|
278
|
+
MIGRATIONS_DIR: MIGRATIONS_DIR,
|
|
279
|
+
TENANT_MIGRATIONS_DIR: TENANT_MIGRATIONS_DIR,
|
|
280
|
+
|
|
281
|
+
// The parts the API (and a host's own routes) build on.
|
|
282
|
+
db: connectDb,
|
|
283
|
+
actionCatalog: actionCatalog,
|
|
284
|
+
envExposed: envExposed,
|
|
285
|
+
runner: workflowRunner,
|
|
286
|
+
flows: flows,
|
|
287
|
+
jobChains: jobChains,
|
|
288
|
+
xcfgSetup: xcfgSetup,
|
|
289
|
+
auth: require('./lib/auth')
|
|
290
|
+
};
|