@flowrunner/cli 0.1.14 → 0.1.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/ai-docs/instructions/service-code-format/02-zod-types.md +59 -15
  2. package/ai-docs/instructions/service-code-format/04-http-requests.md +9 -0
  3. package/ai-docs/instructions/service-code-format/06-oauth2.md +59 -9
  4. package/ai-docs/instructions/service-code-format/07-actions.md +1 -1
  5. package/ai-docs/instructions/service-code-format/08-polling-triggers.md +13 -0
  6. package/ai-docs/instructions/service-code-format/09-realtime-triggers.md +110 -4
  7. package/ai-docs/instructions/service-code-format/16-gotchas.md +28 -0
  8. package/ai-docs/instructions/service-code-format/18-test-recipes.md +16 -0
  9. package/ai-docs/instructions/service-code-format/19-security.md +121 -0
  10. package/ai-docs/instructions/service-code-format/README.md +3 -2
  11. package/ai-docs/providers/claude/agents/flowrunner-service-code-engineer.md +1 -1
  12. package/dist/cli.js +104 -16
  13. package/dist/cli.js.map +2 -2
  14. package/dist/index.js +103 -15
  15. package/dist/index.js.map +2 -2
  16. package/dist/types/flow-extension-runner/src/v2/oauth/types.d.ts +9 -7
  17. package/dist/types/flow-extension-runner/src/v2/params/schema/field-plan.d.ts +9 -4
  18. package/dist/types/flow-extension-runner/src/v2/params/zod.d.ts +12 -0
  19. package/dist/types/flow-extension-runner/src/v2/realtime-triggers/types.d.ts +2 -1
  20. package/dist/types/shared/src/constants/activity-log.d.ts +77 -0
  21. package/dist/types/shared/src/constants/app.d.ts +0 -1
  22. package/dist/types/shared/src/constants/notification-types.d.ts +117 -6
  23. package/dist/types/shared/src/scheme/account/dto.d.ts +3 -0
  24. package/dist/types/shared/src/scheme/crm/dto.d.ts +6 -0
  25. package/dist/types/shared/src/scheme/index.d.ts +1 -0
  26. package/dist/types/shared/src/scheme/integration-connection/dto.d.ts +27 -15
  27. package/dist/types/shared/src/scheme/notifications/dto.d.ts +3 -0
  28. package/dist/types/shared/src/scheme/notifications/rest.d.ts +6 -0
  29. package/dist/types/shared/src/scheme/onboarding/dto.d.ts +29 -0
  30. package/dist/types/shared/src/scheme/onboarding/index.d.ts +1 -0
  31. package/package.json +1 -1
  32. package/runtime/index.js +103 -15
  33. package/runtime/index.js.map +2 -2
  34. package/runtime/types/flow-extension-runner/src/v2/oauth/types.d.ts +9 -7
  35. package/runtime/types/flow-extension-runner/src/v2/params/schema/field-plan.d.ts +9 -4
  36. package/runtime/types/flow-extension-runner/src/v2/params/zod.d.ts +12 -0
  37. package/runtime/types/flow-extension-runner/src/v2/realtime-triggers/types.d.ts +2 -1
  38. package/runtime/types/shared/src/constants/activity-log.d.ts +77 -0
  39. package/runtime/types/shared/src/constants/app.d.ts +0 -1
  40. package/runtime/types/shared/src/constants/notification-types.d.ts +117 -6
  41. package/runtime/types/shared/src/scheme/account/dto.d.ts +3 -0
  42. package/runtime/types/shared/src/scheme/crm/dto.d.ts +6 -0
  43. package/runtime/types/shared/src/scheme/index.d.ts +1 -0
  44. package/runtime/types/shared/src/scheme/integration-connection/dto.d.ts +27 -15
  45. package/runtime/types/shared/src/scheme/notifications/dto.d.ts +3 -0
  46. package/runtime/types/shared/src/scheme/notifications/rest.d.ts +6 -0
  47. package/runtime/types/shared/src/scheme/onboarding/dto.d.ts +29 -0
  48. package/runtime/types/shared/src/scheme/onboarding/index.d.ts +1 -0
@@ -17,6 +17,7 @@ and behaviors.
17
17
  | `z.array(inner)` | list | | `.labels({ value: 'Label' })` | `z.enum` | UI shows the label; the flow stores and `execute` receives the VALUE |
18
18
  | `z.union([...])` | the **first** branch's widget — see below | | | | |
19
19
  | `z.filesUploadOptions([...])`, `z.filesListOptions([...])`, `z.filesGetOptions([...])`, `z.filesDeleteOptions([...])` | nested form — the platform's own file-storage options, one group per Files call (see [03-parameters.md](03-parameters.md)) | | | | |
20
+ | | | | `.required()` | `z.unknown()` / `z.any()` field | required, any value — the open types are the only ones optional by default (see [Any value, but required](#any-value-but-required)) |
20
21
  | | | | `.secret()` | `z.string()` config field | Masks the input, with a reveal toggle. **Display only** — see below |
21
22
  | | | | `.dynamicProps({ dependsOn, execute })` | top-level **action** param: `z.looseObject()` or `z.array(z.looseObject())` | the object's inner fields are loaded at design time from other params (see [03-parameters.md](03-parameters.md)) |
22
23
 
@@ -27,7 +28,8 @@ Notes:
27
28
  config field is the `type` it **publishes**: a closed set in which `z.number()` lands on `string`. That is
28
29
  what a test asserts, not what the form renders. See
29
30
  [Config fields publish a smaller set of types](#config-fields-publish-a-smaller-set-of-types).
30
- - **Required is the default** — mark optional fields with `.optional()`.
31
+ - **Required is the default** — mark optional fields with `.optional()`. The exception is `z.unknown()` / `z.any()`,
32
+ optional unless marked `.required()`.
31
33
  - `.shared()` marks a config field as a **shared** (global) config value rather than per-app — used for
32
34
  OAuth client id/secret and other secrets. It is a **UI directive**: it decides who fills the field in and
33
35
  in which dialog, not how your handler reads it. `config.clientId` is the same either way, and a service
@@ -98,7 +100,7 @@ The table above is what a service declares day to day. A few more types are acce
98
100
  | `z.bigint()` | numeric stepper | your handler receives a `BigInt`. A fractional value is refused rather than rounded, since the conversion would lose information |
99
101
  | `z.literal(v)` | the widget of `v`'s own type — a toggle for `z.literal(true)`, a stepper for `z.literal(7)` | a literal of several values of *different* types has no such type and publishes as text |
100
102
  | `z.record(k, v)` | nested form with no declared fields, because the keys are open | the values are converted against `v` like any other declaration |
101
- | `z.any()` / `z.unknown()` | single-line | declares nothing: the value reaches your handler exactly as it arrived, unconverted |
103
+ | `z.any()` / `z.unknown()` | single-line | declares nothing: the value reaches your handler exactly as it arrived, unconverted. **Optional unless marked** — the one exception to *required is the default*; see [Any value, but required](#any-value-but-required) |
102
104
 
103
105
  **Those two tables together are the whole surface.** A declaration outside them — `z.tuple()`,
104
106
  `z.intersection()`, a modifier a future zod release adds — **fails when the extension loads**, naming the
@@ -118,12 +120,33 @@ These modifiers are looked THROUGH — the parameter publishes and coerces as th
118
120
  | Modifier | Published as |
119
121
  |---|---|
120
122
  | `.optional()` / `.nullable()` / `.nullish()` | the inner type, not required |
121
- | `.default(v)` / `.prefault(v)` | the inner type, not required, with `defaultValue` |
123
+ | `.default(v)` / `.prefault(v)` | the inner type, **not required**, with `defaultValue` — always, even with `.nonoptional()` around it: the runner fills the default, so an empty field never reaches the handler |
122
124
  | `.readonly()` | the inner type |
123
- | `.nonoptional()` | the inner type, required — it overrides an `.optional()` inside it |
125
+ | `.nonoptional()` / `.required()` | the inner type, required — it overrides an `.optional()` inside it, but not a `.default(v)`. `.required()` is the FlowRunner spelling for `z.unknown()` / `z.any()`; on `z.object()` it is zod's own (every key required) |
124
126
  | `.pipe(…)` / `.transform(fn)` | the **input** side. `bodyParams` describes what a caller supplies, which is what the console renders a form for; what your handler receives afterwards is the pipe's business |
125
127
  | `z.lazy(() => …)` | the type the thunk returns |
126
128
 
129
+ ### Any value, but required
130
+
131
+ `z.unknown()` and `z.any()` pass a value through **unconverted** — a number or a boolean bound from an
132
+ expression keeps its type, an object stays an object — which is what you want for a field the vendor takes
133
+ in any type (a Redis value, a Cosmos DB partition key, a custom-field value). Unlike every other type they
134
+ are **optional unless marked**, so mark the ones that must be set:
135
+
136
+ ```js
137
+ value: z.unknown().required().label('Value') // or .nonoptional() — the same thing
138
+ ```
139
+
140
+ | The flow sends | The handler receives |
141
+ |---|---|
142
+ | text (`''` included), a number, `0`, `false`, an object, a list, a date | the value, as given |
143
+ | `null` | `null` — a value the flow passed on purpose |
144
+ | nothing (`undefined`) | refused: `«Value» is required` — the handler never runs |
145
+
146
+ A bare `z.unknown()` is optional: a missing value reaches the handler as `undefined`, while `''` and `null` arrive
147
+ as given, as on a required one. Use `.required()` rather than guarding for `undefined` in the handler — it also marks the field required in the
148
+ form, where a guard cannot.
149
+
127
150
  ### `.catch(v)` is refused — use `.default(v)`
128
151
 
129
152
  `.catch(v)` looks like the natural way to say "if this isn't a valid list, use an empty one", so it is what
@@ -155,22 +178,42 @@ Both declarations validate identically. They **publish differently**, and nothin
155
178
  — so reordering branches to read more nicely silently changes what the flow author is shown. Put the shape
156
179
  you want on the form first, and treat the order as part of the contract.
157
180
 
158
- ### Coercion tries the branches in that same order, and the first that fits wins
181
+ ### A value that already fits a branch is kept as given
182
+
183
+ The branches are checked in that same order against the value **exactly as it arrived**, and the first that
184
+ accepts it wins — nothing is converted. Only a value that fits no branch is converted, into the first branch
185
+ it can become.
186
+
187
+ ```js
188
+ badge: z.union([z.number(), z.string()]) // 5 → 5 '+1' → '+1' true → 'true' (fits neither, converted)
189
+ id : z.union([z.string(), z.number()]) // 5 → 5 '5' → '5'
190
+ ```
191
+
192
+ So one field can keep both forms of a mixed value — a count and an increment, a number and a vendor's text
193
+ decimal — whichever order you declare.
194
+
195
+ **Text always fits a `z.string()` branch.** A value bound from a template or typed into an expression is
196
+ text, so a union with a `z.string()` branch never turns it into a number or a boolean, whatever the order.
197
+ That is worth spelling out for booleans, because the failure is silent: in `z.union([z.boolean(),
198
+ z.string()])` the expression text `'false'` reaches `execute` as the string `'false'`, which is truthy —
199
+ `if (params.flag)` runs the *on* path for an explicit *off*. (The toggle itself sends a real `false`.)
159
200
 
160
- A flow supplies every value as text, so a union has to convert before it can tell which branch applies.
201
+ When the other accepted forms are **a few fixed words**, declare the words, not `z.string()`. A
202
+ `z.literal()` or a `z.enum()` does not fit arbitrary text, so text that is a number or a boolean is
203
+ converted:
161
204
 
162
205
  ```js
163
- flag: z.union([z.boolean(), z.string()]) // 'true' → your handler receives the boolean true
164
- flag: z.union([z.string(), z.boolean()]) // 'true' → your handler receives the string 'true'
206
+ limit: z.union([z.number(), z.literal('max')]) // '5' → 5 'max' → 'max'
207
+ mode : z.union([z.boolean(), z.literal('auto')]) // 'false' → false 'auto' → 'auto'
208
+ scope: z.union([z.number(), z.enum(['all', 'none'])]) // '5' → 5 'all' → 'all' 'x' → «Scope» is invalid
165
209
  ```
166
210
 
167
- **A branch that accepts anything hides every branch after it.** `z.string()` takes `'true'` as-is, so the
168
- boolean branch is never reached. If one branch is a fallback, declare it last.
211
+ Keep `z.string()` in a union only when free text really is one of the forms, and convert in the handler if
212
+ you need a number out of it. For a value that must pass through untouched whatever its type, use
213
+ `z.unknown()` (see [Any value, but required](#any-value-but-required)).
169
214
 
170
- That is worth spelling out for booleans, because the failure is silent. In `z.union([z.string(),
171
- z.boolean()])` the field draws as the **first** branch's widget — a text box, not a toggle — the boolean
172
- vocabulary below is never applied, and `execute` receives the string `'false'`, which is truthy: `if
173
- (params.flag)` runs the *on* path for an explicit *off*. Put the boolean branch first.
215
+ A constraint is part of "fits": `z.union([z.number().min(10), z.string()])` given `5` hands over `'5'`,
216
+ because the number branch refuses it and the string branch takes it converted.
174
217
 
175
218
  ### A blank means "not set"
176
219
 
@@ -272,5 +315,6 @@ knowing:
272
315
  - **Absence follows your modifiers, not the transport.** Whether an empty field arrives as `undefined`,
273
316
  `null`, or the declared `.default(v)` is decided by what your schema accepts, so you write `.optional()` or
274
317
  `.nullable()` — whichever reads naturally — instead of guarding for all three. One exception: an empty
275
- string on a `z.string()` field is a *value*, not absence, so it does not fail a required field. See
318
+ string on a `z.string()` field — and on `z.unknown()` / `z.any()` — is a *value*, not absence, so it does not
319
+ fail a required field. See
276
320
  [16-gotchas.md](16-gotchas.md).
@@ -153,6 +153,15 @@ await Flowrunner.Request.post(url).send({ name: 'Bob', tags: ['a', 'b'] })
153
153
  await Flowrunner.Request.post(url).type('text/plain').send('raw text')
154
154
  ```
155
155
 
156
+ That serialization happens **only when the content type is exactly `application/json`**. If it is any other
157
+ type, including `application/json; charset=utf-8`, serialize the body yourself. An object passed with any other
158
+ type throws before the request is sent, so the vendor never receives anything:
159
+
160
+ ```js
161
+ .type('application/x-www-form-urlencoded').send(new URLSearchParams(body).toString())
162
+ .type('application/json; charset=utf-8').send(JSON.stringify(body))
163
+ ```
164
+
156
165
  ### Forms and file uploads
157
166
 
158
167
  `.form(data)` sets a form-data body — this is the multipart path, and there is no `.attach()`. It takes a
@@ -11,11 +11,59 @@ for the common case; any handler below overrides its step. Combined scopes = `sc
11
11
  | `pkce` | boolean | Runtime generates verifier/challenge + threads state. |
12
12
  | `clientAuth` | `'basic' \| 'body'` | Client credential placement. |
13
13
  | `identityRequest` | fn | `({ apiRequest }) => raw` — fetch the account identity. |
14
- | `identity` | fn | `(raw) => { name, imageUrl?, userData? }`. |
14
+ | `identity` | fn | `(raw) => { name, imageUrl?, userData? }`. `userData` is shown with the connection and is **never handed back** — store what you need later in the token (see [Composite tokens](#composite-tokens)). |
15
15
  | `getConnectionURL` (esc) | fn | see below. |
16
16
  | `executeCallback` (esc) | fn | see below. |
17
17
  | `refreshToken` (esc) | fn | see below. |
18
- | `serializeToken` / `deserializeToken` | fn | Composite-token (de)serialization; runtime deserializes the header token into `oauth.token` before `initContext`. |
18
+ | `serializeToken` / `deserializeToken` | fn | Composite-token (de)serialization over one shape, `{ accessToken, refreshToken?, userData? }`; runtime deserializes the header token into `oauth.token` (and `oauth.userData`) before `initContext`. See [Composite tokens](#composite-tokens). |
19
+
20
+ ## Composite tokens
21
+
22
+ The platform stores the token a connection returns and sends **only that token** back on every later call —
23
+ handlers and refresh alike. The `userData` a connection returns is kept for display and **never reaches your
24
+ code again**. So anything a later call needs — a region host, a tenant or vendor account id, an environment —
25
+ goes **inside the token**.
26
+
27
+ The standard is one shape in both directions: `serializeToken` receives `{ accessToken, refreshToken, userData }`
28
+ and turns it into the stored string, and `deserializeToken` turns that string back into the same shape.
29
+
30
+ ```js
31
+ ext.setupOauth2({
32
+ authorizeUrl, tokenUrl, identityRequest,
33
+ identity : me => ({ name: me.email, userData: { accountId: me.id, apiHost: me.api_domain } }),
34
+ serializeToken : stored => JSON.stringify(stored), // { accessToken, refreshToken, userData }
35
+ deserializeToken: token => {
36
+ try {
37
+ return JSON.parse(token)
38
+ } catch {
39
+ return { accessToken: token } // a connection made before the wrapper
40
+ }
41
+ },
42
+ })
43
+
44
+ // in any handler or initContext
45
+ initContext: ({ oauth }) => ({ accessToken: oauth.token?.accessToken, apiHost: oauth.userData?.apiHost }),
46
+ ```
47
+
48
+ What the runtime does with it:
49
+
50
+ | Step | What happens |
51
+ |---|---|
52
+ | Connect (built-in `executeCallback`) | `serializeToken({ accessToken, refreshToken, userData })` — `userData` is what `identity` returned |
53
+ | Refresh (built-in, declarative `tokenUrl`) | the current token is deserialized and its `userData` is passed to `serializeToken` with the new tokens, so it **survives every refresh** with no code in the service |
54
+ | Every handler and `initContext` | `oauth.token` = `deserializeToken(<stored token>)`, `oauth.userData` = `oauth.token.userData` |
55
+
56
+ - **Send the vendor `oauth.token.accessToken`**, never `oauth.accessToken` — that is the stored string, wrapper
57
+ and all. The vendor only ever receives the bare token.
58
+ - **A bare token still reads as one.** Connections made before a service wrapped its tokens hold the vendor's
59
+ token unwrapped, so `deserializeToken` must accept it (the `catch` above). Such a connection has no
60
+ `userData` until the user reconnects — the refresh has nothing to carry over — so fall back to the
61
+ configured default for it.
62
+ - **A stored host that receives a secret is validated anyway.** A refresh POSTs the client secret, so check a
63
+ host read from `userData` against the vendor's known set, even though the service wrote the value itself.
64
+ - **An escape-hatch `executeCallback` or `refreshToken` returns the token string itself**: build it with the
65
+ same `serializeToken`, and in `refreshToken` carry `oauth.userData` over — the runtime cannot do that for a
66
+ hook it does not run.
19
67
 
20
68
  ## Escape-hatch handlers → returns
21
69
 
@@ -28,17 +76,18 @@ for the common case; any handler below overrides its step. Combined scopes = `sc
28
76
  `getConnectionURL` and `executeCallback` run before any token exists, so their `oauth` is empty. `refreshToken`
29
77
  is different: the platform sends the connection's **current (expired) token** in the `oauth-access-token`
30
78
  header, so `oauth = { accessToken, token? }` and `initContext` see it exactly as in any other handler. That is
31
- how a composite token survives a refresh — rebuild the next one from the data in the current one, because the
32
- provider's refresh response does not repeat what it returned at connect time:
79
+ how a composite token survives a custom refresh — rebuild the next one from the data in the current one,
80
+ because the provider's refresh response does not repeat what it returned at connect time:
33
81
 
34
82
  ```js
35
83
  refreshToken: async ({ refreshToken, config, oauth }) => {
36
84
  const res = await exchangeRefreshToken(config, refreshToken)
85
+ const next = res.refresh_token ?? refreshToken
37
86
 
38
- // `oauth.token` is `deserializeToken(<current token>)` — carry the realm id over
87
+ // `oauth.userData` is the current token's `userData` — carry it over
39
88
  return {
40
- token : `${ res.access_token }::realm::${ oauth.token.realmId }`,
41
- refreshToken: res.refresh_token ?? refreshToken,
89
+ token : serializeToken({ accessToken: res.access_token, refreshToken: next, userData: oauth.userData }),
90
+ refreshToken: next,
42
91
  expirationInSeconds: res.expires_in,
43
92
  }
44
93
  },
@@ -55,8 +104,9 @@ error propagate.
55
104
 
56
105
  ## Using OAuth in handlers
57
106
 
58
- `oauth` derives only from the `oauth-access-token` header → `oauth = { accessToken, token? }` (`token` =
59
- `deserializeToken(accessToken)`). Per-op opt-in for extra scopes: set `oauth2Scopes: ['extra.scope']` on
107
+ `oauth` derives only from the `oauth-access-token` header → `oauth = { accessToken, token?, userData? }`
108
+ (`token` = `deserializeToken(accessToken)`, `userData` = `token.userData`). The connection's own `userData` is
109
+ not part of it — only what the service stored in the token (see [Composite tokens](#composite-tokens)). Per-op opt-in for extra scopes: set `oauth2Scopes: ['extra.scope']` on
60
110
  any action, polling or realtime trigger, or dictionary.
61
111
 
62
112
  `token` is the one slot in the handler bag left open rather than inferred: `setupOauth2` runs after
@@ -115,7 +115,7 @@ where each type comes from:
115
115
  | `params` | `z.infer` of **this handler's own** `params` | Parsed input — defaults applied, values coerced, unknown fields stripped. `{}` when the method declares none. |
116
116
  | `context` | return type of `initContext` | Whatever the factory built (see [01-service-structure.md](01-service-structure.md)). |
117
117
  | `config` | `z.infer` of `configItems` | Resolved config values. |
118
- | `oauth` | `{ accessToken?: string, token?: unknown }` | From the `oauth-access-token` header only; see [06-oauth2.md](06-oauth2.md). |
118
+ | `oauth` | `{ accessToken?: string, token?: unknown, userData?: unknown }` | From the `oauth-access-token` header only — `userData` is the one stored inside the token; see [06-oauth2.md](06-oauth2.md#composite-tokens). |
119
119
  | `apiRequest` | your own `apiRequest`, minus `context` / `logger` | Shared HTTP helper — the runtime injects those two, so you never pass them (see [04-http-requests.md](04-http-requests.md)). |
120
120
  | `files` | the file-service client | Present only when `usesFileStorage:true` (else a proxy that throws; see [10-files-api.md](10-files-api.md)). |
121
121
  | `filesScope` | `{ WORKSPACE, FLOW, EXECUTION }` | The storage layers, so you name one instead of spelling the string. Always present, opt-in or not. |
@@ -72,6 +72,19 @@ poll: async ({ params, state, isLearningMode, apiRequest }) => {
72
72
 
73
73
  This is distinct from `result` (which describes shape only).
74
74
 
75
+ ## `state` — keep only the comparison keys
76
+
77
+ The returned `state` is sent back in the request body of the next poll, and that body has a size limit.
78
+ Storing whole items (`state: { records }`) eventually fails every poll with `PayloadTooLargeError`, and the
79
+ trigger silently stops. Store only what the next poll compares — an id, an id plus its modified value, a
80
+ watermark or a cursor — and bound any list carried across polls:
81
+
82
+ ```js
83
+ state: { records: records.map(({ id, modifiedAt }) => ({ id, modifiedAt })) }
84
+ ```
85
+
86
+ When fixing an existing trigger, keep its state key so state already stored still reads.
87
+
75
88
  ## Two ways a watermark drops events silently
76
89
 
77
90
  Both of these lose events permanently, with no error and nothing in the logs — the trigger simply never
@@ -48,7 +48,7 @@ running and the error appears in the customer's **workspace log** on each refres
48
48
 
49
49
  | Property | Type | Description |
50
50
  |---|---|---|
51
- | `scope` | `'SINGLE_APP' \| 'ALL_APPS'` | Per-app webhook vs one shared callback. |
51
+ | `scope` | `'SINGLE_APP' \| 'ALL_APPS'` | Per-app webhook vs one shared callback — see [Choosing `scope`](#choosing-scope). |
52
52
  | `extractProviderEventName` | path \| fn `({requestBody})=>name\|null` | Discriminator → routes to triggers by `providerEventName` (fan-out: matches every trigger with that value). |
53
53
  | `extractProviderEventData` | path \| fn | Base `providerEventData` handed to matched triggers. Default = whole body. |
54
54
  | `scopeAt` | path | ALL_APPS routing key path (route where `scopeAt === eventScopeId`). |
@@ -60,6 +60,57 @@ running and the error appears in the customer's **workspace log** on each refres
60
60
  | `resolveEvents` | fn | Escape hatch — replaces `extract*`/`handshake` for irregular/multi-shape payloads. |
61
61
  | `verifyRequest` | fn | Escape hatch — replaces `verify`. |
62
62
 
63
+ ### Choosing `scope`
64
+
65
+ **`SINGLE_APP` is the default choice.** Each app registers its own webhook with the provider (`subscribe` gets a
66
+ `callbackUrl` per app), so a delivery arrives on the URL of the app it belongs to.
67
+
68
+ **`ALL_APPS` is for a provider that has one callback URL for the whole integration**, set once in its developer
69
+ console and shared by every customer. All deliveries for all apps arrive on that one URL, and the platform routes
70
+ each one by **`eventScopeId`**: `subscribe` returns this app's value, each delivery yields one (read at `scopeAt`,
71
+ or returned by `resolveEvents`), and the delivery goes to the apps whose `subscribe` returned the same value.
72
+
73
+ Choose `ALL_APPS` only when **both** hold:
74
+
75
+ 1. the provider offers one shared callback URL, not a URL per registered webhook, **and**
76
+ 2. every delivery carries a value that identifies the customer's account (a team, tenant or account id), and
77
+ `subscribe` can learn the same value for the app's connection.
78
+
79
+ If the provider lets you register a URL per webhook, use `SINGLE_APP`, even when its deliveries also carry a
80
+ tenant id. If its deliveries carry no tenant id, `ALL_APPS` cannot route them correctly: use `SINGLE_APP`.
81
+
82
+ ⚠️ **`eventScopeId` must identify a tenant, never be a constant.** A fixed value returned from `subscribe` (the
83
+ extension's own name, a hardcoded string) puts **every customer of the extension on one routing key**, so each
84
+ delivery reaches every app that uses the extension. The service still looks like it works, because each app does
85
+ get its own events, along with everyone else's. The runner logs a warning when an `ALL_APPS` `subscribe` returns no
86
+ `eventScopeId`, or returns the extension's own `id` or `name`. It does not refuse either one.
87
+
88
+ ⚠️ **On `ALL_APPS`, the inbound seams run before the app is known.** `handshake`, `verify`/`verifyRequest` and
89
+ `resolveEvents` are what produce the `eventScopeId`, and that value is what decides which app the delivery belongs
90
+ to. So nothing stored per app is available to them yet: `webhookData` is `null` (see
91
+ [`webhookData` on the inbound path](#webhookdata-on-the-inbound-path--read-only)), and there is no app connection
92
+ to call the provider's API with. Derive the event name, its data and the `eventScopeId` from the delivery itself.
93
+ A signing secret has to be app-level: a `.shared()` config item, the same for every customer.
94
+
95
+ ```js
96
+ // Slack-style: one Events API URL for every workspace; each delivery names its workspace in `team_id`.
97
+ svc.setupRealtimeTriggers({
98
+ scope : 'ALL_APPS',
99
+ scopeAt : 'team_id', // this delivery's tenant
100
+ extractProviderEventName: 'event.type',
101
+
102
+ // The secret comes from the provider's app settings, so it is shared, not per customer.
103
+ verifyRequest: ({ rawBody, headers, config }) => verifySignature(rawBody, headers, config.signingSecret),
104
+
105
+ subscribe: async ({ apiRequest }) => {
106
+ const { team_id } = await apiRequest({ url: '/auth.test' })
107
+
108
+ return { eventScopeId: team_id } // the same value `scopeAt` reads from this customer's deliveries
109
+ },
110
+ unsubscribe: () => ({}),
111
+ })
112
+ ```
113
+
63
114
  **Function args → returns.** All of these also receive the full ctx bag — `config, context, apiRequest,
64
115
  oauth, logger, helpers` — in the *same* single object as their own arguments. That includes `verifyRequest`,
65
116
  so a value derived in `initContext` is reachable there; prefer it over re-deriving from `config`
@@ -74,7 +125,7 @@ does its own lowercased retry; that behaviour does not extend to `verifyRequest`
74
125
  | `subscribe` | `callbackUrl`, `triggers:[{ id, params, providerEventName? }]`, `webhookData` (null first), `connectionId` | `{ webhookData?, eventScopeId?, refreshIntervalInSeconds? }` |
75
126
  | `unsubscribe` | `callbackUrl`, `webhookData`, `connectionId` | `{ webhookData? }` |
76
127
  | `refresh` | `callbackUrl`, `triggers:[{ id, params, providerEventName? }]`, `webhookData` | `{ webhookData?, refreshIntervalInSeconds }` |
77
- | `handshake` | `{ requestBody, headers, rawBody, webhookData }` | `true` → ack with a plain 200; `{ responseToWebhook }` → ack with that response (runtime adds `handshake:true`); nothing / `false` / `{}` → proceed. See [Handshakes](#handshakes). |
128
+ | `handshake` | `{ requestBody, headers, query, rawBody, webhookData }` | `true` → ack with a plain 200; `{ responseToWebhook }` → ack with that response (runtime adds `handshake:true`); nothing / `false` / `{}` → proceed. See [Handshakes](#handshakes). |
78
129
  | `resolveEvents` | `rawBody`, `headers`, `query`, `body`, `webhookData`, `connectionId` | `{ events:[{ id, providerEventData }], eventScopeId?, handshake?, responseToWebhook? }` |
79
130
 
80
131
  Both also still accept the **deprecated** `responseToExternalService` / `responseHeaders` — see
@@ -122,6 +173,9 @@ handshake: ({ requestBody }) => {
122
173
 
123
174
  return requestBody == null || Object.keys(requestBody).length === 0
124
175
  },
176
+
177
+ // A challenge sent in the query string: `query` holds the callback URL's query parameters ({} when none).
178
+ handshake: ({ query }) => query.challenge ? { responseToWebhook: { rawBody: String(query.challenge) } } : false,
125
179
  ```
126
180
 
127
181
  ⚠️ **`handshake` runs before signature verification**, so it sees an unverified request. Acking one is harmless
@@ -247,16 +301,54 @@ verifyRequest: ({ rawBody, headers }) => {
247
301
  Hashing `''` instead produces a digest that can never match, which reads as a *signature mismatch* rather
248
302
  than the missing input it is — and sends you hunting for a wrong secret.
249
303
 
304
+ ### Who owns the secret decides what a missing signature means
305
+
306
+ A signature check makes two decisions: whether to verify, and **what to do with a delivery that carries no
307
+ signature at all**. The second is where the mistakes are, and it follows from one question — **is there a
308
+ secret on record for this delivery?** `secret` answers it: return the key, or nothing (`null`, `undefined`,
309
+ `''`) when there is none.
310
+
311
+ | Situation | `secret` returns | Header absent | Header present but wrong |
312
+ |---|---|---|---|
313
+ | The provider signs with a credential you hold — OAuth client secret, account auth token, app key | the key | ⛔ 401 | ⛔ 401 |
314
+ | You minted the secret, or the provider minted it, and `subscribe` stored it in `webhookData` | the stored secret | ⛔ 401 | ⛔ 401 |
315
+ | A subscription registered before the service stored a secret, or an optional key the customer left empty | nothing | ✓ accepted — nothing to check against | ✓ accepted |
316
+
317
+ ```js
318
+ verify: {
319
+ header : 'x-provider-signature',
320
+ algo : 'sha256',
321
+ encoding: 'hex',
322
+ secret : (config, { webhookData }) => webhookData?.signingSecret ?? null, // nothing stored → not checked
323
+ },
324
+ ```
325
+
326
+ - **While a secret exists, a missing header is never accepted.** An early `return true` when the header is
327
+ absent turns the check off for anyone who simply leaves it out — a secret that only protects you when the
328
+ attacker chooses to send a signature protects nothing. There is no switch to accept unsigned deliveries;
329
+ return nothing from `secret` only where there genuinely is no secret.
330
+ - **A missing secret self-heals.** `subscribe` is an upsert that re-runs on flow start, so an old subscription
331
+ gets a stored secret the next time it is registered, and from then on its deliveries are checked.
332
+ - **Prefer a secret you can derive over one you have to store.** Where the provider lets you supply the secret,
333
+ derive it from something present at delivery time — `HMAC(clientSecret, channelId)`, `HMAC(botToken, '<fixed
334
+ context>')` — so there is nothing to store or lose. A derived secret can always be computed, though, so mark
335
+ the subscriptions that were registered with it (a flag in `webhookData` at subscribe time) and return nothing
336
+ for the ones that were not; otherwise every subscription made before the change is answered 401.
337
+ - **An app-level key is not a reason to skip verification.** Declare it `.optional()` and return it as is:
338
+ an empty key means nothing to check, so a connection that never sets one behaves as before. Where the
339
+ provider issues one key per app, mark it `.shared()` — it is filled in once, next to the OAuth client id,
340
+ and no customer has to do anything.
341
+
250
342
  ### `verify` fields
251
343
 
252
344
  HMACs the **raw body alone** with the key `secret` returns and compares it to the header in constant time.
253
345
 
254
346
  | Field | Req | Description |
255
347
  |---|---|---|
256
- | `header` | ✓ | Signature header. Read as declared, then retried lowercased. Absent header → fail. |
348
+ | `header` | ✓ | Signature header. Read as declared, then retried lowercased. Absent header → fail, when a secret is on record. |
257
349
  | `algo` | ✓ | Hash algorithm, e.g. `'sha256'`. |
258
350
  | `encoding` | ✓ | `'hex' \| 'base64'` — the digest encoding the provider sends. |
259
- | `secret` | ✓ | `(config, { webhookData }) => key`. The second argument is the stored webhook document — `null` when there is none — for a provider that mints the secret at registration time. |
351
+ | `secret` | ✓ | `(config, { webhookData }) => key`. The second argument is the stored webhook document — `null` when there is none — for a provider that mints the secret at registration time. Returning nothing (`null`, `undefined`, `''`) means no secret is on record: the delivery is accepted unchecked. See [Who owns the secret](#who-owns-the-secret-decides-what-a-missing-signature-means). |
260
352
  | `timestampHeader` | – | Header carrying the request timestamp. **Required for `freshness` to run.** |
261
353
  | `freshness` | – | Max age in seconds. **Only checked when `timestampHeader` is also declared** — on its own it is silently inert. A declared-but-absent timestamp header passes. |
262
354
 
@@ -270,6 +362,20 @@ on flow stop). Two strategies:
270
362
  - **Recreate** (Calendly, Backendless, joggai) — delete prior `webhookData` first, then create.
271
363
  - **Reconcile/diff** (Slack, Box) — apply only additions/removals (no teardown).
272
364
 
365
+ **The prior `webhookData` is an input, never something to ignore.** A `subscribe` that mints a fresh signing
366
+ secret (or creates a fresh webhook) on every call, without reading the stored document, is neither strategy:
367
+
368
+ 1. Flow enabled → webhook created with secret S1, S1 stored.
369
+ 2. Flow re-enabled — any edit, any redeploy → S2 minted, a **second** webhook created, S2 stored.
370
+ 3. The first webhook is still live and still signs with S1, so every one of its deliveries now fails
371
+ verification — and the events arrive twice besides.
372
+
373
+ It degrades on its own over time rather than failing on the first run, and shows up as intermittent
374
+ duplicate events plus intermittent signature failures, neither of which points at registration. Reuse what is
375
+ stored — `const secret = webhookData?.secret || newSecret()` — or tear the prior webhook down first. Test it
376
+ with the recipe in [18-test-recipes.md](18-test-recipes.md#triggers): a second upsert fed the first one's
377
+ `webhookData`.
378
+
273
379
  Runtime provides `Flowrunner.crypto.hmac(algo,key,data,'hex'|'base64')` — **synchronous**, returning the
274
380
  encoded digest as a string, not a Promise — `timingSafeEqual(a,b)` (false — not a throw — on nullish or
275
381
  length mismatch, so a caller can `return` it), and `ageSeconds(timestamp)`.
@@ -6,6 +6,10 @@ in full.
6
6
 
7
7
  ## An empty string is a value, not an absence
8
8
 
9
+ On a plain `z.string()`, and on `z.unknown()` / `z.any()`, which take any value as given. On every other type
10
+ `''` is absence; and a `.default(v)` — on any type, an enum included — fills `''` and `null` as well as a missing
11
+ value, so a defaulted field never sees either.
12
+
9
13
  Request bodies are usually built by dropping unset params. Drop only `undefined` and `null`:
10
14
 
11
15
  ```js
@@ -47,6 +51,29 @@ webhookUrl: z.union([z.url(), z.literal('')]).optional().label('Webhook URL')
47
51
 
48
52
  Declare the union wherever blank has to mean "not set". A non-empty value still has to satisfy the format.
49
53
 
54
+ ## An optional object the author never filled in is absent — but may still arrive as `{}`
55
+
56
+ An optional `z.object({ … })` param whose fields the flow author left empty reaches your handler **absent**
57
+ (`undefined`, or `null` under `.nullable()`, or its `.default()`), the same as a boolean nobody set. Once the
58
+ author fills in **any** field, the whole object is sent, and the flow editor then holds the author to that
59
+ object's required fields, showing `Bar is required` against an empty one.
60
+
61
+ So read an optional object defensively — `params.data?.foo`, never `params.data.foo`.
62
+
63
+ **A flow saved before this behaviour still sends `{}`** for the same untouched object, until someone opens and
64
+ saves it again. A handler that forwards an optional object to a vendor must therefore treat `{}` as absent
65
+ too, or it sends an empty object the vendor reads as an instruction — an input context with no name, an
66
+ entity list replaced by nothing:
67
+
68
+ ```js
69
+ const isEmpty = value => !value || (typeof value === 'object' && !Object.keys(value).length)
70
+
71
+ body: { name: params.name, context: isEmpty(params.context) ? undefined : params.context }
72
+ ```
73
+
74
+ Test it both ways: pass no value and pass `{}`, and assert the field is missing from the request. A list of
75
+ objects is unaffected — an empty list is still sent as `[]`.
76
+
50
77
  ## Guard a lookup table keyed by user input
51
78
 
52
79
  An object literal used as a lookup table inherits from `Object.prototype`, so a key like `constructor`,
@@ -243,6 +270,7 @@ Do not restate these here; they are covered where they belong.
243
270
  | A binary download is `.toBuffer()`, not a response encoding — an encoded body is parsed and corrupted | [04-http-requests.md](04-http-requests.md) |
244
271
  | `ResponseError.message` is the response *body*, so it can be an object | [04-http-requests.md](04-http-requests.md) |
245
272
  | A blank (`''`) fails `z.url().optional()` — `.optional()` does not cover it | this document |
273
+ | Secrets in logs, credentials in the URL, credentials or caller headers sent to an origin the service didn't choose, upload MIME type | [19-security.md](19-security.md) |
246
274
  | `.default()` is not applied to realtime-trigger `match` params | [13-runtime-contracts.md](13-runtime-contracts.md) |
247
275
  | `rawBody` is **absent**, not empty, when there is nothing to verify — hashing `''` looks like a bad signature | [09-realtime-triggers.md](09-realtime-triggers.md) |
248
276
  | `FLOW`/`EXECUTION` file scopes fail in a method that runs outside a flow execution | [10-files-api.md](10-files-api.md) |
@@ -116,6 +116,22 @@ history entry for the registration call, the whole returned `webhookData` — an
116
116
  `describe` like everything else. One registration serves every trigger, so exercise it under the trigger
117
117
  whose delivery the rest of that group covers rather than repeating it under all of them.
118
118
 
119
+ **Upsert twice, feeding the first `webhookData` back.** The platform re-runs the upsert whenever a flow is
120
+ re-enabled, with the document the last one stored. A registration that mints a new secret or a new webhook
121
+ every time passes a single-call test and fails here:
122
+
123
+ ```js
124
+ it('reuses its webhook and secret when the flow is enabled again', async () => {
125
+ const first = await runHandleTriggerUpsertWebhookMethod({ callbackUrl: 'https://cb/x', triggers: [] })
126
+ const second = await runHandleTriggerUpsertWebhookMethod({
127
+ callbackUrl: 'https://cb/x', triggers: [], webhookData: first.webhookData,
128
+ })
129
+
130
+ expect(second.webhookData.secret).toBe(first.webhookData.secret)
131
+ // and the history shows no second "create webhook" call (or the prior one deleted first, for recreate)
132
+ })
133
+ ```
134
+
119
135
  > **The live trigger set is out of reach.** `triggers` is resolved from persisted flow instances, which a
120
136
  > test has none of, so whatever array a subscribe-path invocation carries is discarded silently. Do not
121
137
  > build a test on it — see *What the harness cannot reach* in
@@ -0,0 +1,121 @@
1
+ # 19 — Security
2
+
3
+ Four mistakes a service makes **silently**: nothing fails, every test passes, and a credential or a request
4
+ ends up somewhere it should not. Each has shipped in many services. Check a service against all four by name
5
+ before calling it done — they do not show up any other way.
6
+
7
+ Verifying inbound webhook signatures is a separate topic: see
8
+ [09-realtime-triggers.md](09-realtime-triggers.md), *Who owns the secret decides what a missing signature
9
+ means*.
10
+
11
+ ## Never log a secret
12
+
13
+ Logs are kept, shared with support and read by people who never hold the credential. Never log:
14
+
15
+ - a token-endpoint response, an access token or a refresh token;
16
+ - the OAuth callback object — it carries the authorization `code`;
17
+ - a credential config value (an API key, a client secret, a password);
18
+ - an outgoing request's headers, where `Authorization` lives.
19
+
20
+ Log the field **names** instead, which is what a debugging session actually needs:
21
+
22
+ ```js
23
+ logger.debug(`token response fields: ${ Object.keys(response).join(', ') }`) // ✓
24
+ logger.debug(`token response: ${ JSON.stringify(response) }`) // ✗ logs the tokens
25
+ ```
26
+
27
+ A rejected request's `ResponseError` is safe to log whole: it holds the vendor's **response**, not the
28
+ request that carried the credentials.
29
+
30
+ To test it, run the method with a recognisable fake secret and assert that no log line contains it.
31
+
32
+ ## Never put a credential in the URL
33
+
34
+ A URL is written to access logs, proxies and browser history on every hop. So:
35
+
36
+ - **An OAuth token or refresh request** sends `client_secret`, `code` and `refresh_token` in an
37
+ `application/x-www-form-urlencoded` **body**, never in the query string (RFC 6749 §4.1.3 and §6). The
38
+ declarative `tokenUrl` path already does this; a hand-written `executeCallback` or `refreshToken` must too.
39
+ - **A vendor API key** goes in exactly one place: a header when the vendor accepts one, otherwise the body
40
+ when the request has one. The query string only for a body-less request the vendor documents that way.
41
+
42
+ ```js
43
+ // ✓ refresh grant in the body
44
+ Flowrunner.Request.post(tokenUrl)
45
+ .type('application/x-www-form-urlencoded')
46
+ .send(new URLSearchParams({ grant_type: 'refresh_token', refresh_token, client_id, client_secret }).toString())
47
+
48
+ // ✗ the secret in the query string
49
+ Flowrunner.Request.post(`${ tokenUrl }?client_secret=${ client_secret }&refresh_token=${ refresh_token }`)
50
+ ```
51
+
52
+ ## Send credentials only to the origin you configured
53
+
54
+ A URL that arrives from a **param or a response** — a pagination cursor, a download link, an upload chunk
55
+ URL, a "next" link — is not yours. Attach credentials to it only when its origin is exactly the configured
56
+ instance's origin. Check before any request is made:
57
+
58
+ ```js
59
+ function assertOwnOrigin(url, baseUrl) {
60
+ let parsed
61
+
62
+ try {
63
+ parsed = new URL(url)
64
+ } catch {
65
+ throw new Error('Refusing a URL that does not parse')
66
+ }
67
+
68
+ if (parsed.origin !== new URL(baseUrl).origin || parsed.username || parsed.password) {
69
+ throw new Error(`Refusing to send credentials to ${ parsed.origin }`)
70
+ }
71
+
72
+ return parsed.toString()
73
+ }
74
+ ```
75
+
76
+ Comparing `origin` (scheme + host + port) refuses, in one test, a foreign host, a lookalike
77
+ (`api.vendor.com.evil.io`), an `http` downgrade of an `https` instance and a different port; the
78
+ `username`/`password` check refuses `https://api.vendor.com@evil.io`. Never compare with `startsWith` or
79
+ `includes` — every one of those passes the lookalike.
80
+
81
+ If the link must be followed and the vendor hosts it elsewhere (a CDN download), fetch it **without**
82
+ credentials.
83
+
84
+ **The same applies to headers the caller supplies.** A param that becomes request headers or properties
85
+ must not be able to set `Authorization`, `Content-Type` or the vendor's own auth headers, and a value
86
+ containing `\r` or `\n` is refused — otherwise a flow can rewrite the service's authentication or inject a
87
+ header line:
88
+
89
+ ```js
90
+ const RESERVED = new Set(['authorization', 'content-type', 'x-api-key']) // + the vendor's auth headers
91
+
92
+ function callerHeaders(headers = {}) {
93
+ return Object.fromEntries(Object.entries(headers).map(([name, value]) => {
94
+ if (RESERVED.has(name.toLowerCase())) {
95
+ throw new Error(`Header "${ name }" is set by the service and cannot be overridden`)
96
+ }
97
+
98
+ if (/[\r\n]/.test(`${ name }${ value }`)) {
99
+ throw new Error(`Header "${ name }" contains a line break`)
100
+ }
101
+
102
+ return [name, String(value)]
103
+ }))
104
+ }
105
+ ```
106
+
107
+ ## Pick an upload's MIME type in a fixed order
108
+
109
+ When the service sends a file **to the vendor**, the type it declares decides how the vendor stores and
110
+ serves it. A wrong one is accepted silently and surfaces later as a file that will not open. Pick it in this
111
+ order and stop at the first that applies:
112
+
113
+ 1. the type the caller passed;
114
+ 2. the download's `Content-Type`, with parameters stripped (`image/png; charset=binary` → `image/png`),
115
+ ignoring an empty one and `application/octet-stream`;
116
+ 3. the type implied by the file name's extension;
117
+ 4. `application/octet-stream`.
118
+
119
+ A file name the service generates takes the extension of the type it chose, so the two never disagree. An
120
+ endpoint that accepts one kind only (an image API, say) keeps that kind as its last fallback instead of
121
+ `application/octet-stream`.