@withgateway/sdk 0.3.3 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (158) hide show
  1. package/README.md +4 -3
  2. package/dist/cli.js +3005 -1401
  3. package/dist/cli.js.map +1 -1
  4. package/dist/cli.mjs +2996 -1392
  5. package/dist/cli.mjs.map +1 -1
  6. package/dist/files/index.js +128 -37
  7. package/dist/files/index.js.map +1 -1
  8. package/dist/files/index.mjs +126 -35
  9. package/dist/files/index.mjs.map +1 -1
  10. package/dist/hooks/index.d.mts +1 -1
  11. package/dist/hooks/index.d.ts +1 -1
  12. package/dist/hooks/index.js +124 -11
  13. package/dist/hooks/index.js.map +1 -1
  14. package/dist/hooks/index.mjs +124 -11
  15. package/dist/hooks/index.mjs.map +1 -1
  16. package/dist/{index-C3UzD0G3.d.mts → index-CDPQBsU5.d.mts} +19 -3
  17. package/dist/{index-C3UzD0G3.d.ts → index-CDPQBsU5.d.ts} +19 -3
  18. package/dist/{index-mJhG3vZU.d.mts → index-Di9nNqOY.d.mts} +26 -3
  19. package/dist/{index-DUHmS74p.d.ts → index-LA-PMRJT.d.ts} +26 -3
  20. package/dist/index.d.mts +4 -4
  21. package/dist/index.d.ts +4 -4
  22. package/dist/index.js +1078 -260
  23. package/dist/index.js.map +1 -1
  24. package/dist/index.mjs +1040 -251
  25. package/dist/index.mjs.map +1 -1
  26. package/dist/replay/index.d.mts +1 -1
  27. package/dist/replay/index.d.ts +1 -1
  28. package/dist/replay/index.js.map +1 -1
  29. package/dist/replay/index.mjs.map +1 -1
  30. package/dist/traces/index.js +125 -34
  31. package/dist/traces/index.js.map +1 -1
  32. package/dist/traces/index.mjs +123 -32
  33. package/dist/traces/index.mjs.map +1 -1
  34. package/dist/tracing/index.d.mts +1 -1
  35. package/dist/tracing/index.d.ts +1 -1
  36. package/dist/tracing/index.js +7 -1
  37. package/dist/tracing/index.js.map +1 -1
  38. package/dist/tracing/index.mjs +7 -1
  39. package/dist/tracing/index.mjs.map +1 -1
  40. package/dist/worlds/index.d.mts +819 -438
  41. package/dist/worlds/index.d.ts +819 -438
  42. package/dist/worlds/index.js +912 -236
  43. package/dist/worlds/index.js.map +1 -1
  44. package/dist/worlds/index.mjs +908 -232
  45. package/dist/worlds/index.mjs.map +1 -1
  46. package/package.json +1 -1
  47. package/runtime/gatewaysdk/environment/_bundle.py +4 -6
  48. package/runtime/gatewaysdk/environment/_materialize.py +5 -25
  49. package/runtime/gatewaysdk/environment/_world.py +82 -66
  50. package/runtime/gatewaysdk/environment/backing.py +20 -24
  51. package/runtime/gatewaysdk/environment/core.py +1 -16
  52. package/runtime/gatewaysdk/environment/legacy_sqlite.py +14 -27
  53. package/runtime/gatewaysdk/environment/runtime.py +45 -2
  54. package/runtime/gatewaysdk/environment/schema/admission.py +174 -0
  55. package/runtime/gatewaysdk/environment/schema/api.py +198 -474
  56. package/runtime/gatewaysdk/environment/schema/assertions.py +458 -0
  57. package/runtime/gatewaysdk/environment/schema/batch.py +3 -3
  58. package/runtime/gatewaysdk/environment/schema/bundle_runtime.py +1 -0
  59. package/runtime/gatewaysdk/environment/schema/cache_layout.py +5 -0
  60. package/runtime/gatewaysdk/environment/schema/call_capture.py +321 -0
  61. package/runtime/gatewaysdk/environment/schema/call_log.py +276 -18
  62. package/runtime/gatewaysdk/environment/schema/call_redaction.py +207 -19
  63. package/runtime/gatewaysdk/environment/schema/cast.py +15 -35
  64. package/runtime/gatewaysdk/environment/schema/compiler.py +4 -1
  65. package/runtime/gatewaysdk/environment/schema/conform.py +38 -35
  66. package/runtime/gatewaysdk/environment/schema/connector.py +94 -152
  67. package/runtime/gatewaysdk/environment/schema/connectors/confluence/connector.toml +1 -12
  68. package/runtime/gatewaysdk/environment/schema/connectors/custom/handlers.py +2 -1
  69. package/runtime/gatewaysdk/environment/schema/connectors/github/api/handlers.py +217 -58
  70. package/runtime/gatewaysdk/environment/schema/connectors/github/connector.toml +10 -19
  71. package/runtime/gatewaysdk/environment/schema/connectors/github/scenarios/default/seed/github_webhooks.json +10 -2
  72. package/runtime/gatewaysdk/environment/schema/connectors/github/tests/webhooks.json +47 -0
  73. package/runtime/gatewaysdk/environment/schema/connectors/github/world.json +21 -0
  74. package/runtime/gatewaysdk/environment/schema/connectors/jamf/api/handlers.py +185 -18
  75. package/runtime/gatewaysdk/environment/schema/connectors/jamf/connector.toml +29 -14
  76. package/runtime/gatewaysdk/environment/schema/connectors/jamf/provenance.json +43 -12
  77. package/runtime/gatewaysdk/environment/schema/connectors/jamf/scenarios/default/seed/jamf_webhooks.json +2 -2
  78. package/runtime/gatewaysdk/environment/schema/connectors/jamf/scope.toml +4 -17
  79. package/runtime/gatewaysdk/environment/schema/connectors/jamf/tests/webhooks.json +107 -0
  80. package/runtime/gatewaysdk/environment/schema/connectors/jamf/vendor-openapi.json +1 -1
  81. package/runtime/gatewaysdk/environment/schema/connectors/linear/api/handlers.py +85 -1
  82. package/runtime/gatewaysdk/environment/schema/connectors/linear/connector.toml +19 -31
  83. package/runtime/gatewaysdk/environment/schema/connectors/linear/provenance.json +18 -8
  84. package/runtime/gatewaysdk/environment/schema/connectors/linear/scenarios/default/seed/webhooks.json +19 -3
  85. package/runtime/gatewaysdk/environment/schema/connectors/linear/tests/test_webhooks.py +71 -0
  86. package/runtime/gatewaysdk/environment/schema/connectors/servicenow/connector.toml +2 -5
  87. package/runtime/gatewaysdk/environment/schema/cors.py +54 -0
  88. package/runtime/gatewaysdk/environment/schema/credentials.py +313 -8
  89. package/runtime/gatewaysdk/environment/schema/data.py +14 -13
  90. package/runtime/gatewaysdk/environment/schema/database/base.py +106 -165
  91. package/runtime/gatewaysdk/environment/schema/database/guard.py +45 -37
  92. package/runtime/gatewaysdk/environment/schema/database/reads.py +169 -0
  93. package/runtime/gatewaysdk/environment/schema/database/sqlite.py +57 -31
  94. package/runtime/gatewaysdk/environment/schema/declared_routes.py +210 -0
  95. package/runtime/gatewaysdk/environment/schema/door.py +899 -0
  96. package/runtime/gatewaysdk/environment/schema/envelope.py +86 -28
  97. package/runtime/gatewaysdk/environment/schema/grading.py +58 -86
  98. package/runtime/gatewaysdk/environment/schema/handler_errors.py +52 -5
  99. package/runtime/gatewaysdk/environment/schema/host.py +229 -1695
  100. package/runtime/gatewaysdk/environment/schema/host_control.py +144 -0
  101. package/runtime/gatewaysdk/environment/schema/host_handoff.py +370 -0
  102. package/runtime/gatewaysdk/environment/schema/host_loops.py +97 -0
  103. package/runtime/gatewaysdk/environment/schema/host_owners.py +121 -0
  104. package/runtime/gatewaysdk/environment/schema/host_proxy.py +111 -0
  105. package/runtime/gatewaysdk/environment/schema/host_requests.py +6 -0
  106. package/runtime/gatewaysdk/environment/schema/host_session.py +126 -0
  107. package/runtime/gatewaysdk/environment/schema/host_surfaces.py +79 -21
  108. package/runtime/gatewaysdk/environment/schema/host_transfers.py +127 -0
  109. package/runtime/gatewaysdk/environment/schema/host_versions.py +242 -0
  110. package/runtime/gatewaysdk/environment/schema/host_workers.py +577 -0
  111. package/runtime/gatewaysdk/environment/schema/journal.py +438 -0
  112. package/runtime/gatewaysdk/environment/schema/ledger_spill.py +828 -0
  113. package/runtime/gatewaysdk/environment/schema/local.py +32 -116
  114. package/runtime/gatewaysdk/environment/schema/openapi_schemas.py +20 -5
  115. package/runtime/gatewaysdk/environment/schema/platform.py +29 -234
  116. package/runtime/gatewaysdk/environment/schema/request_body.py +238 -0
  117. package/runtime/gatewaysdk/environment/schema/request_ledger.py +958 -0
  118. package/runtime/gatewaysdk/environment/schema/route_bindings.py +25 -3
  119. package/runtime/gatewaysdk/environment/schema/route_miss.py +54 -0
  120. package/runtime/gatewaysdk/environment/schema/runtime.py +136 -176
  121. package/runtime/gatewaysdk/environment/schema/scaffold.py +2 -1
  122. package/runtime/gatewaysdk/environment/schema/session_files.py +133 -24
  123. package/runtime/gatewaysdk/environment/schema/session_move.py +275 -0
  124. package/runtime/gatewaysdk/environment/schema/skills/agents/connector-template-author.md +5 -1
  125. package/runtime/gatewaysdk/environment/schema/skills/benchmarks-getting-started/SKILL.md +8 -9
  126. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONNECTORS.md +3 -3
  127. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +50 -28
  128. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +12 -8
  129. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +20 -22
  130. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +25 -25
  131. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/cli-mcp-map.md +24 -16
  132. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/gotchas.md +71 -7
  133. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/journal-checks.md +182 -0
  134. package/runtime/gatewaysdk/environment/schema/snapshot.py +150 -30
  135. package/runtime/gatewaysdk/environment/schema/spill_files.py +228 -0
  136. package/runtime/gatewaysdk/environment/schema/tasks.py +229 -323
  137. package/runtime/gatewaysdk/environment/schema/tools_world.py +103 -203
  138. package/runtime/gatewaysdk/environment/schema/validation.py +69 -1
  139. package/runtime/gatewaysdk/environment/schema/verbs.py +38 -36
  140. package/runtime/gatewaysdk/environment/schema/webhooks.py +687 -0
  141. package/runtime/gatewaysdk/environment/schema/worker.py +6 -4
  142. package/runtime/gatewaysdk/environment/schema/workers.py +203 -39
  143. package/runtime/gatewaysdk/environment/schema/world_host.py +970 -0
  144. package/runtime/gatewaysdk/environment/schema/world_tests.py +83 -11
  145. package/runtime/gatewaysdk/environment/world.py +1 -1
  146. package/runtime/gatewaysdk/environment/world_session_contract.py +14 -0
  147. package/skills/agents/connector-template-author.md +5 -1
  148. package/skills/benchmarks-getting-started/SKILL.md +8 -9
  149. package/skills/connector-schema-authoring/CONNECTORS.md +3 -3
  150. package/skills/connector-schema-authoring/SKILL.md +50 -28
  151. package/skills/connector-schema-authoring/connector-worlds.md +282 -18
  152. package/skills/connector-schema-authoring/world-schema.md +7 -6
  153. package/skills/world-data-ingestion/ROWS.md +12 -8
  154. package/skills/world-data-ingestion/SKILL.md +20 -22
  155. package/skills/worlds-getting-started/SKILL.md +25 -25
  156. package/skills/worlds-getting-started/references/cli-mcp-map.md +24 -16
  157. package/skills/worlds-getting-started/references/gotchas.md +71 -7
  158. package/skills/worlds-getting-started/references/journal-checks.md +182 -0
@@ -40,9 +40,9 @@ layout: opt-in on create (`layout: "api-bundle"` / `--layout api-bundle`, the an
40
40
  `deprecations: [{code: "api-bundle-layout", …}]`), still served as an api session for the worlds
41
41
  that already use it, and every platform reader still resolves it through `schemaWorldPrefix` /
42
42
  `schemaWorldFiles` (`packages/shared/src/domain/worldSchema.ts`, the twin of the runtime's
43
- `world_root`). `gateway bench push` in the TypeScript CLI still packages a `connector.toml` tree on
44
- the way up (`isConnectorWorld` in `bench.ts`), as does `publishWorldTree` (the agent's
45
- `create_custom_world`); those are the remaining packaging sites. `gateway worlds migrate <slug>`
43
+ `world_root`). Nothing packages a `connector.toml` tree on the way up any more: `gateway bench push`
44
+ and `gateway worlds push` push the tree as it is (`versionVerbs.ts`), and `publishWorldTree`
45
+ (`worldPublicationService.ts`, now behind the world fixture seeder only) commits it as given. `gateway worlds migrate <slug>`
46
46
  (`POST /api/public/worlds/{slug}/migrate`, MCP `migrate_world`; `planWorldMigration` in
47
47
  `packages/shared/src/domain/worldMigration.ts`) restores a stored bundle to its tree — the source is
48
48
  inside the bundle under `server/world/`; the bundle's own scenarios are carried, its derived files
@@ -86,10 +86,131 @@ when that runs out, prints the session with `ready: false` and exits 1 while it
86
86
  on a cold version pass `--no-wait` and poll `gateway worlds session status <sessionId>` until
87
87
  `ready`.
88
88
 
89
- The world's API honours the vendor's auth scheme: a hosted session accepts its own api token (as a
90
- bearer or in the vendor's header), `worlds serve` the credential `--api-key` pins (default `conform`),
91
- and `WORLD_API_KEY` applies only to a legacy api bundle. Anything else answers 401 with the body configured under
92
- `[api].unauthorized` (`""` or `'null'` for a bare status with no body, the way some vendors answer).
89
+ The world's API mirrors the vendor's auth as `[auth]` declares it (`door.py`, "The door" below). A
90
+ hosted session's credential is its own api token, `worlds serve`'s the value `--api-key` pins
91
+ (default `conform`), and `WORLD_API_KEY` applies only to a legacy api bundle. A hosted session takes
92
+ that token the way the vendor takes its key: after an `Authorization` word the vendor takes, in the
93
+ declared header, as the Basic user (or password), in the declared query parameter, a token the
94
+ declared login minted for this session, or a signed download link. Platform tooling sends it in the
95
+ platform's own `X-World-Session` header, which every world takes and no vendor's client sends
96
+ (both SDKs' `apiHeaders`); the platform's `Authorization: Bearer` of the session token reaches
97
+ every world for good (the host counts it as `requests.platform_bearer` where the vendor takes its
98
+ key another way). An **open** session asks for nothing: a world sets `mode = "open"` under `[auth]`
99
+ as its sessions' default, and a session's open (`auth: "open"`, `--auth open`) chooses either way.
100
+ Its api and ui answer without a credential on a capability hostname,
101
+ `o-<sid>-<label>[-api].<worlds domain>`, the label a 130-bit secret derived from the session token
102
+ and kept for the session's life (a reopen, a reset, a restart keep the link); the session's own
103
+ `w-<sid>` hostname keeps asking for the credential. On the capability hostname the host is the
104
+ proof, so `ttl_seconds` bounds nothing there; a link the world signs still names the `w-` host. Anything
105
+ else answers the vendor's refusal configured under `[api].unauthorized_status` and
106
+ `[api].unauthorized` (`""` or `'null'` for a bare status with no body, the way some vendors
107
+ answer: `unauthorized_status = 403` with `'{"message": "Invalid API key"}'`); a world that declares
108
+ neither gets the host's own `401 {"error": "invalid session token"}`.
109
+
110
+ ### The door
111
+
112
+ `[auth]` says where the credential rides and, optionally, how the vendor trades it for a token and
113
+ which files it hands out as links (synthetic examples; every name is invented):
114
+
115
+ ```toml
116
+ [auth] # a key in the URL (?key=...)
117
+ scheme = "query"
118
+ name = "key"
119
+ secret_env = "VENDOR_API_KEY"
120
+ ```
121
+
122
+ ```toml
123
+ [auth] # Authorization: Bearer <key> or token <key>, as GitHub takes it
124
+ scheme = "bearer"
125
+ prefixes = ["Bearer", "token"] # the words the vendor takes, any case; calls carry the first
126
+ secret_env = "VENDOR_TOKEN"
127
+ ```
128
+
129
+ ```toml
130
+ [auth] # Basic email:api_token, the key the password under any user
131
+ scheme = "basic"
132
+ credential = "password" # default "user": the key the user, the password empty
133
+ secret_env = "VENDOR_API_TOKEN"
134
+ ```
135
+
136
+ ```toml
137
+ [auth] # a JSON login that answers a token; calls carry it as a bearer
138
+ scheme = "bearer"
139
+ secret_env = "VENDOR_PASSWORD"
140
+
141
+ [auth.login]
142
+ method = "POST" # the default
143
+ path = "/token" # under base_url's path, as routes are; never /__…
144
+ credential = { json = "password" } # or { form = "secret" }, { header = "X-Api-Key" }; a dotted json path reaches inside
145
+ required = ["username", "realm"] # what else the vendor demands, read from the same place
146
+ reply = '{"access_token": "{token}", "expires_in": "{expires_in}", "token_type": "Bearer"}'
147
+ ttl_seconds = 86400 # opt in to expiry; absent, a token lives as long as the session
148
+ ```
149
+
150
+ ```toml
151
+ [auth.login] # an api-key login whose reply nests the token: {"data": {"token": …}}
152
+ path = "/login"
153
+ credential = { json = "api_key" }
154
+ reply = '{"data": {"token": "{token}"}}'
155
+ ```
156
+
157
+ ```toml
158
+ [auth.downloads] # presigned links: these GET routes answer a signed URL without the key
159
+ routes = ["download_results"]
160
+ ttl_seconds = 900 # opt in to expiry; absent, a link lives as long as the session
161
+ ```
162
+
163
+ - **The session's credential is the key.** On the login route the session token (or the pin) goes
164
+ where `credential` says; a wrong or missing one is the vendor's 401 (`[api].unauthorized`), a
165
+ missing `required` field or an unreadable body its 400 (`[api].invalid`). The right one is
166
+ answered `reply` (default `{"access_token", "token_type", "expires_in"}`) with the token at
167
+ `"{token}"` and the lifetime at `"{expires_in}"`, a key whose value is `"{expires_in}"` left out
168
+ when the world declares no `ttl_seconds`; `status` (default 200) is the vendor's. On an open
169
+ session's capability hostname the login answers whatever credential the client sends, or none.
170
+ - **Later calls carry the token the way `scheme` says** (bearer, header, Basic, query parameter). A
171
+ login world takes only tokens it minted, never the raw key, as the vendor does; the platform's
172
+ `X-World-Session` header still reaches a hosted session (the SDKs send it), and the host then
173
+ presents the world a freshly minted token and drops the header. A refresh token an oauth2 world
174
+ minted answers every time it is presented, wherever the session runs, and lives as long as the
175
+ session even where `[auth]` declares `ttl_seconds`, which bounds the access token alone: the
176
+ short access token and long refresh real vendors hand out. An oauth2
177
+ `token_url` path may name a segment (`/ccx/oauth2/{tenant}/token`), which any value fills.
178
+ - **Tokens and links are bound to the session.** A token is an HS256 JWT, a link an HMAC-SHA256 over
179
+ its path and expiry, each under a key derived from the session's credential (no new secret): a
180
+ token or link from another session never verifies, a tampered one is refused, one past a declared
181
+ lifetime is refused, and a supervisor, its version worker and any host the session moves to check
182
+ them alike. Nothing expires unless the world declares `ttl_seconds`. Comparisons run in constant
183
+ time.
184
+ - **A download route is never open.** A handler hands out a link with
185
+ `ctx.signed_url("/exports/<id>/results.zip")` (the path as the client sends it under base_url's
186
+ path; an optional second argument gives it a lifetime in seconds); the route answers that link
187
+ without the key for the session's life, or until its declared lifetime ends, and the credential
188
+ as any other route does. The route names its object in the path and declares no `params`, and a
189
+ link carries `signature` (and `expires`, when it has a lifetime) alone, so it reaches the one
190
+ object it was signed for.
191
+ - **Nothing logs a key.** The call log and the request ledger drop a query key and a link's
192
+ `signature`; a token or a link's signature inside a longer string reads `[redacted]` in place;
193
+ the ledger keeps no login request body and scrubs every body it keeps (JSON or text, the answer
194
+ that hands out a link and a page link the world echoes included) of the session credential, a
195
+ carried parameter's value, a minted token and a signature; the host writes no request URL to its
196
+ log, and the chart's World Host Ingress keeps no nginx access log (an ALB access log you turn on,
197
+ or another controller's default access log, would record URLs).
198
+ - **A link reaches its own route.** A request a signed link admits is routed among the
199
+ `[auth.downloads]` routes alone, so a sibling route on the same path (one that takes `params`)
200
+ never answers a link.
201
+ - **Capture does not log in.** `connector capture` and the live hook lanes send `secret_env` in the
202
+ scheme's carrier as it is (oauth2 alone exchanges first): for a vendor that trades its key for a
203
+ token, set `secret_env` to a token its own login minted.
204
+ - `scheme = "oauth2"` is the RFC 6749 token route (below); it takes no `[auth.login]`, and neither
205
+ does `none`. A login path that is also a declared route of the same method is refused, as is a
206
+ download route that answers anything but GET or declares any `params`, and a setting of another
207
+ scheme (`prefixes` on `header`, `credential` on `bearer`).
208
+ - **Uploads reach handlers.** A write's body is read by its media type: JSON, a form, a multipart
209
+ form (each field's first part; a file part reads `{"filename", "content_type", "size"}` with its
210
+ bytes in `ctx.files[name].content`) or multipart/related (the first part as JSON, the rest in
211
+ `ctx.files["1"]`, …). Any other type (`application/octet-stream`, `text/csv`) is an upload: a
212
+ handler route reads `body` as `{}` and the bytes in `ctx.raw`; a declared route refuses it as the
213
+ vendor's 400, as it refuses JSON that does not parse.
93
214
 
94
215
  The runner's own routes (`/__world/seed|reset|state|calls`) answer only `X-World-Admin-Token`. A session
95
216
  pod and a Batch run mint that token themselves. A twin you host yourself (`hostedUrl`) gets it from
@@ -126,13 +247,14 @@ least one `[[operations]]` entry.
126
247
  | Table | Keys | Notes |
127
248
  | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
128
249
  | `[connector]` | `slug`, `base_url` | `base_url` is an http(s) origin plus optional path, no query. Requests carrying that path prefix are accepted too. |
129
- | `[auth]` | `scheme` (`header`, `bearer`, `basic`, `query`, `oauth2`, `none`), `name`, `secret_env`, `headers`, `token_url`, `client_id_env`, `scope` | The secret is only ever named by an env var. `headers = { "X-App-Token" = "VENDOR_APP_TOKEN" }` adds more credential headers. `oauth2` is the client-credentials grant: capture exchanges `client_id_env` + `secret_env` at `token_url` (a path under `base_url` or an absolute URL, default `/oauth/token`) for the bearer it then sends, and the served world mints bearers on the same path. |
250
+ | `[auth]` | `scheme` (`header`, `bearer`, `basic`, `query`, `oauth2`, `none`), `name`, `secret_env`, `headers`, `prefixes`, `credential`, `token_url`, `client_id_env`, `scope`, `[auth.login]`, `[auth.downloads]` | The secret is only ever named by an env var. `prefixes` (bearer, oauth2) lists the `Authorization` words the vendor takes; `credential = "password"` (basic) reads the key as the Basic password. A token path (`token_url`'s, `[api] token_path`) is a route path (each `{name}` once) that no declared POST operation answers too. `headers = { "X-App-Token" = "VENDOR_APP_TOKEN" }` adds more credential headers. `oauth2` is the client-credentials grant: capture exchanges `client_id_env` + `secret_env` at `token_url` (a path under `base_url` or an absolute URL, default `/oauth/token`) for the bearer it then sends, and the served world mints bearers on the same path. `[auth.login]` is the vendor's own login route and `[auth.downloads]` its presigned download routes ("The door" above). |
130
251
  | `[capture]` | `requests_per_minute`, `timeout_seconds`, `max_pages`, `cursor_param`, `cursor_field`, `cursor_header` | Pacing between pages; providers ask us to be sparing. The next cursor is read from a body field or a response header (Torchlight sends `Cursor:`), never both, and is sent back as `cursor_param`; a cursor that is an absolute URL on the connection's host (DRF's `next`) is followed by its query. |
131
252
  | `[redact]` | `hash`, `drop`, `round`, `preserve` | Applied before a body reaches disk. `preserve` hashes shape-preservingly (same length, digits stay digits, letters keep their case, separators stay; slug-salted, deterministic) so an account number still meets the field's format — `hash` made it a 71-char `sha256:` string the contract refused (Brex). `hash` writes `sha256:<hex>` salted with the slug so joins survive, element by element for arrays so the shape the contract expects survives too; `drop` removes; `round = { lat = 2, lon = 2 }` keeps numbers numeric but coarsens them. A bare name matches at any depth; a dotted path (`results.*.email`, `*` for list items) matches that place only, so a catalog's `assets` count map keeps its `email` key. Field names keep the vendor's casing. |
132
- | `[api]` | `envelope`, `hits_key`, `cursor_key`, `page`, `limit_param`, `offset_param`, `cursor_param`, `page_size`, `previous_key`, `page_links`, `page_headers`, `cursor_absent_when_done`, `unauthorized`, `not_found`, `invalid`, `*_status`, `error_content_type`, `token_path`, `handlers`, `before` | `envelope = "results"` wraps rows as `{results, hits, cursor}` under the vendor's key names; empty means a bare array. `hits_key = ""` leaves the count out and `cursor_absent_when_done = true` leaves the cursor key out of the last page (instead of `""`), together the `{items, next_cursor?}` shape Brex-style vendors answer and clients test with `"next_cursor" in body`. `page` is `none`, `offset` or `cursor`, with the vendor's parameter names; `previous_key` adds the previous page and `page_links = true` renders next/previous as absolute URLs of the served host, the way Django REST Framework answers; `page_headers = { cursor = "Cursor", total = "Total-Count" }` sends the same values as response headers for vendors that page that way (Torchlight). Error bodies are JSON strings replayed verbatim with the vendor's statuses and content type (`application/problem+json` for RFC 7807 vendors); `""` (or `'null'`) means no body at all, for vendors that answer a bare 401. An oauth2 world mints bearers on `token_path` (default the spec's `token_url`) and honours only those. `handlers` names the world's Python module (default `api/handlers.py` when present); `before` names a gate in it that runs before every operation, declarative ones included, and returns `None` to let the request through or an answer (quota, allow-list, key state) to stop it. |
253
+ | `[api]` | `envelope`, `hits_key`, `cursor_key`, `page`, `limit_param`, `offset_param`, `cursor_param`, `page_size`, `previous_key`, `page_links`, `page_headers`, `link_header`, `cursor_absent_when_done`, `unauthorized`, `not_found`, `invalid`, `wrong_method`, `*_status`, `error_content_type`, `cors`, `token_path`, `handlers`, `before` | `envelope = "results"` wraps rows as `{results, hits, cursor}` under the vendor's key names; empty means a bare array. `hits_key = ""` leaves the count out and `cursor_absent_when_done = true` leaves the cursor key out of the last page (instead of `""`), together the `{items, next_cursor?}` shape Brex-style vendors answer and clients test with `"next_cursor" in body`. `page` is `none`, `offset` or `cursor`, with the vendor's parameter names; `previous_key` adds the previous page and `page_links = true` renders next/previous as absolute URLs of the served host, the way Django REST Framework answers; `page_headers = { cursor = "Cursor", total = "Total-Count" }` sends the same values as response headers for vendors that page that way (Torchlight). Error bodies are JSON strings replayed verbatim with the vendor's statuses and content type (`application/problem+json` for RFC 7807 vendors); `""` (or `'null'`) means no body at all, for vendors that answer a bare 401. A path the world does not serve answers `not_found`; a path it serves, called with a method none of its routes answer, answers `wrong_method` (405 by default with the `Allow` header RFC 9110 requires, no body unless declared; `wrong_method_status` for a vendor that answers another status, which carries no `Allow`). A write the contract refuses answers `invalid` exactly as declared, and the refusal's rule and schema path go to the host's log, never the answer (the request ledger keeps the request). `link_header = true` sends RFC 8288 `Link: <next>; rel="next"` (and `rel="prev"`) on list answers, each URL the request's own with the page's paging parameters, as GitHub pages. `cors = true` is a vendor whose API answers browsers: an `OPTIONS` preflight is answered 204 before any credential (`Access-Control-Allow-Origin: *`, the methods the world serves, the headers the page asked to send) and every answer names any origin and exposes its headers (`Link`); without it an `OPTIONS` is routed like any other method. `gateway worlds serve` (and `worlds test`, which opens it) answers only this machine's pages: a request from another site's `Origin` is refused 403, since its pin is public. A `HEAD` answers as `GET` without a body, and a `DELETE` that carries a body is read like a `POST`'s. An oauth2 world mints bearers on `token_path` (default the spec's `token_url`) and honours only those. `handlers` names the world's Python module (default `api/handlers.py` when present); `before` names a gate in it that runs before every operation, declarative ones included, and returns `None` to let the request through or an answer (quota, allow-list, key state) to stop it. |
133
254
  | `[[operations]]` | `name`, `method`, `path`, `params`, `entity`, `results` | `path` may carry `{args}`; a `params` name keeps the vendor's spelling, dots included (`query.bibliographic`, `filter.overallStatus`), and may end in `[]` (`expand[]`, `status[]`) for a key the vendor repeats, which capture then sends repeated and the served route filters on any value of; `results` is the dotted path to the row array (empty when the body is the row). An operation without `entity` never seeds and answers 501 `not_modelled`. Routes are never tools: an agent calls them over HTTP (the api url and token `session open --surface api` prints), and a tool exists only where `x-gateway.tools` declares one with a handler. An `entity` (or an `explode` target) that `schema/world.json` does not declare is refused by `connector validate`, `schema compile`, `schema check` and the platform's describe (`gateway worlds describe`, local or platform) alike — `operation X lands on entity Y, which schema/world.json does not declare (declared: …)` at `/operations/<i>/entity` — and nothing is written; hand-edited TOML meets the same gate `route add` applies. |
134
255
  | `[operations.ingest]` | `drop`, `explode`, `carry_args` | `drop` removes vendor fields the entity does not carry; each `[[operations.ingest.explode]]` (`field`, `entity`, `carry`, `take`) turns a nested array into child rows; `carry_args = { account_id = "id" }` stamps the capture's path argument or query parameter onto every row before projection (a statement only knows its account from the URL it was fetched under). `connector ingest --dry-run [--report FILE]` lists every violation of every capture and writes nothing. |
135
256
  | `[operations.api]` | `filter`, `match`, `action`, `status`, `single`, `envelope`, `wrapper`, `page`, `body`, `defaults`, `generate`, `handler`, `sample`, `sample_headers`, `sample_body`, `not_found`, `not_found_status`, `cursor_absent_when_done` | `filter` maps a path argument or declared query parameter to the row field it selects on (case-insensitive, any depth; an absent parameter does not narrow, one sent more than once matches rows carrying any of its values); `match = { name = "contains" }` switches an argument from equality to substring or `prefix`, and a list field matches when any element does. `action` is `read` (default), `create`, `update`, `delete` or `handler`: writes persist through the contract, `body` maps row fields (dotted for nested) to request-body paths, `defaults` fixes constants, `generate` fills `uuid`, `int` or `now`; `update` and `delete` find their row through `filter` and answer the vendor's 404 otherwise; they, and a `single` read with a path argument, refuse (`finds its row by nothing`, 400) when neither `filter` nor a path argument naming a field selects a row, rather than act on some row; a singleton served under a scoped path answers through `action = "handler"`. `status` is the 2xx the vendor answers (201 for create, 204 for delete by default), `single` returns the row itself instead of an envelope, `envelope` and `page` override the world-wide setting per route. A dotted `envelope` nests the answer the way the vendor does (`message.items` answers `{message: {items, total-results, next-cursor}}`), and a single row is wrapped only by its own route's `envelope` (`message` answers `{message: row}`). `wrapper` is the rest of the vendor's envelope as a JSON object string (`'{"status": "ok", "message-type": "work-list"}'`), the answer laid onto it key by key; `schema init --from-openapi` writes it from the document. `handler` names a function `def name(ctx, args, query, body)` in the handlers module that returns a body, `(status, body)` or `(status, body, headers)`; `ctx` reads and writes the store (`rows`, `get`, `find`, `insert`, `update`, `delete`), keeps time (`now`, `elapsed`, `ladder` for status machines, frozen by `GATEWAY_WORLD_CLOCK`), mints ids (`uuid`) and shapes answers (`envelope`, `not_found`, `invalid`) and sees the request's `headers` and `query_all` (every value of every parameter, `{"expand[]": ["a", "b"]}`; the `query` argument keeps the first value only). `not_found` / `not_found_status` answer this route's missing row the vendor's own way (a 403 with its own message for one resource) and fall back to `[api]`; `cursor_absent_when_done` overrides `[api]` for this route. `sample = { ip = "203.0.113.5" }` gives the conformance run a value a validating route accepts when no seeded row can supply one; `sample_headers = { "Idempotency-Key" = "conform-1" }` and `sample_body = { reason = "conform" }` give it the headers and body a route demands (read by `connector conform` only; the served API never sends requests). |
257
+ | `[webhooks]` | `subscriptions`, `url`, `events`, `secret`, `enabled`, `scope`, `signing`, `signature_header` | Where the world's webhook subscriptions live and how the vendor signs what it sends (see Webhooks below). `subscriptions` names the entity agents register endpoints in through the vendor's own routes; `url`, `events`, `secret`, `enabled` and each `scope` value name its fields (checked against the contract at compile). `signing` is `svix`, `github`, `linear`, `stripe`, `hmac-sha256` (with `signature_header`) or `none`. |
136
258
 
137
259
  ### Scaffolding a route
138
260
 
@@ -192,11 +314,149 @@ does all of that from the same `connector.toml`:
192
314
  handlers and the gate may answer with headers too (`SpyCloud-Error`, `Retry-After`).
193
315
  - An oauth2 world answers the vendor's token route before any authorization, so a client that
194
316
  performs the grant first works unchanged; a hosted session checks the client secret against its
195
- own api token, `worlds serve` against the `--api-key` pin.
317
+ own api token, `worlds serve` against the `--api-key` pin. A declared `[auth.login]` route is
318
+ answered the same way, ahead of every handler.
196
319
 
197
320
  The handlers ship inside the api bundle beside the world, so the hosted container runs them
198
321
  with nothing installed. The abstraction is the platform's; the handlers are the customer's.
199
322
 
323
+ ## Webhooks
324
+
325
+ A vendor that notifies (AgentMail's `message.received`, GitHub's `push`) declares where its
326
+ subscriptions live, and a handler sends:
327
+
328
+ ```toml
329
+ [webhooks]
330
+ subscriptions = "webhooks" # the entity POST /v0/webhooks writes
331
+ url = "url"
332
+ events = "event_types" # absent or empty: every event
333
+ secret = "secret"
334
+ enabled = "enabled" # absent: on; false: off
335
+ scope = { inbox_id = "inbox_ids" } # a scoped event reaches a row whose list holds it, or is empty
336
+ signing = "svix"
337
+ ```
338
+
339
+ ```python
340
+ def send_message(ctx, args, query, body):
341
+ ...
342
+ ctx.deliver("message.sent", {"type": "event", "event_type": "message.sent", "message": message},
343
+ scope={"inbox_id": args["inbox_id"]})
344
+ ```
345
+
346
+ A tool sends the same way with `from gateway_env import webhooks` and `webhooks.deliver(...)`.
347
+ The GitHub (`ping`, `issues`, `issue_comment`, `pull_request`, `push`), Linear (`Issue`, `Comment`)
348
+ and Jamf (`ComputerAdded`) templates send from their routes; their agent tools write the same rows
349
+ and send nothing, so an agent that must be notified works through the routes.
350
+ Each subscription that is enabled, wants the event and holds its scope gets one delivery: the
351
+ body as JSON, signed with that subscription's secret the vendor's way (`svix-id`,
352
+ `svix-timestamp`, `svix-signature`; GitHub's `X-Hub-Signature-256`; Linear's bare hex `linear-signature`;
353
+ Stripe's `stripe-signature: t=…,v1=…` over `timestamp.body`; `sha256=` under a named
354
+ header). `deliver` answers the delivery ids. A delivery is kept only when the call that sent it
355
+ commits: a call that answers 4xx/5xx, breaks the contract or raises sends nothing, like its
356
+ writes.
357
+
358
+ On a World Host session every delivery lands in the world's request ledger with `via`
359
+ `webhook`: the event type as `operation`, the receiver's URL without its userinfo or query as
360
+ `path` (a segment carrying a credential `[redacted]`), the body, and status `0` until a receiver
361
+ answers. Of the headers the ledger keeps `content-type`, `svix-id`, `svix-timestamp`,
362
+ `x-github-event` and `x-github-delivery`; the signature is `[redacted]`.
363
+
364
+ Bounds: at most 100 deliveries per event (a subscription past the bound is skipped and logged),
365
+ 500 per call, and 256 KB per body. The body is frozen as JSON when `deliver` is called: what the
366
+ ledger keeps and a receiver gets is what was signed, whatever the handler does to its object after.
367
+ A handler that calls `deliver` wrongly (no `[webhooks]`, an empty event type, a scope key the table
368
+ does not declare, a body JSON cannot carry such as `NaN`, a body too large, a call past 500) gets
369
+ its own `400 handler` answer. A subscription whose secret the scheme cannot sign with, or whose
370
+ URL no receiver could be at (not http(s), no host, a port that does not parse), is skipped, never
371
+ the agent's refusal.
372
+ `gateway worlds requests <slug> --via webhook` lists them, and a check reads them like any
373
+ request: `{entity: "gateway_requests", where: {via: "webhook", operation: "message.received"}}`. A grader's `calls` never holds them: it is what the agent called.
374
+ ### Seeing and receiving them
375
+
376
+ - `gateway worlds session webhooks <id>` lists the subscriptions the session's world holds (url
377
+ without its userinfo or query, events, scope, on or off, how many deliveries each was sent since
378
+ the session's feed began; never a secret; the first 200). The session page shows the same as its
379
+ Webhooks section.
380
+ - `gateway worlds session listen <id> --forward-to http://localhost:3000/webhooks` forwards every
381
+ delivery to a receiver on your machine, each with its signed headers and exact body, so the
382
+ receiver verifies it with the secret the world returned when the webhook was registered. The
383
+ receiver's status (or `0` and why it never answered) goes back to the world's request ledger, once
384
+ per delivery. Only deliveries sent from now are forwarded unless `--replay` (a Svix receiver refuses
385
+ a delivery older than about 5 minutes). `--once` forwards every delivery the session holds and
386
+ exits; without it a failed poll is retried with backoff until Ctrl-C, except a session that ended, is
387
+ another project's or another engine's, which stops it with exit 1 (a session starting or being
388
+ recovered is retried).
389
+ - In the ledger a delivery's entry has the event type as `operation` and its id in
390
+ `query.delivery`; a receiver's answer is recorded under `<event type>.answer` with
391
+ `query.answer` naming the delivery. So `{entity: "gateway_requests", where: {via: "webhook",
392
+ operation: "message.received"}}` counts deliveries whether or not anyone was listening, and
393
+ `operation: "message.received.answer"` counts what receivers answered.
394
+ - The routes: `GET /api/public/world-sessions/{id}/webhooks?after&epoch&tail&limit` (read key:
395
+ subscriptions and which event went to which subscription and when; no signed header, payload or
396
+ URL) and `POST` the same with `{after, epoch, tail, limit, answers, subscriptions}` (secret key:
397
+ deliveries whole, the receivers' answers kept first). A session another project holds is 404; one
398
+ that is not running 409. A session on another engine answers 409 `webhooks_not_served`.
399
+ - The World Host keeps each session's last 500 deliveries, at most 8 MiB, whole for this; a page is at
400
+ most 100 deliveries and 4 MiB. The feed is in memory: a session that starts again (a restart, a move)
401
+ starts a fresh feed under a new `epoch`, and a poll naming the old one is read from the fresh feed's
402
+ start with `reset: true`; a page says how many deliveries left the feed unread (`missed`). The
403
+ ledger keeps every delivery. A delivered body is the body as signed when the handler sent it.
404
+
405
+ ### Pushing them to receivers on the internet
406
+
407
+ A project can have the platform deliver its worlds' webhooks itself: **Settings → General → World
408
+ webhooks → Send webhooks to their receivers** (off by default; a project admin or owner turns it on;
409
+ the change is in the audit log). While it is on, a worker job every 5 seconds reads each running World
410
+ Host session of the project, posts every new delivery to its subscription's URL with its signed headers
411
+ and exact body, and hands the receiver's answer back to the world's request ledger (`<event>.answer`,
412
+ the receiver's status or `0` and why). A session opened while push is on is pushed from its first
413
+ delivery; one that was already running when push was turned on starts at its feed's head, so turning
414
+ push on never replays what was sent before.
415
+
416
+ Only https on port 443 is pushed to. Every URL passes the platform's SSRF guard and the host is resolved
417
+ once: every address must be public (private, loopback, link-local, metadata, CGNAT, reserved, NAT64,
418
+ 6to4 and IPv6-local ranges are refused; an operator's `LANGFUSE_WEBHOOK_WHITELISTED_IPS` and
419
+ `_IP_SEGMENTS` also let a world's subscriptions reach the addresses they name, while
420
+ `_WHITELISTED_HOST` only skips the URL check: a listed host that resolves to a refused address is still
421
+ refused), and the connection goes to the address that was checked, so a DNS answer that changes
422
+ afterwards is never used. The world's `host` and connection headers are not forwarded. No redirect is
423
+ followed, an answer outside HTTP's 100–599 is recorded as `0` and why, a receiver has 10 seconds in all, its DNS included (however slowly it answers), and its
424
+ answer is read to 64 KB and dropped.
425
+
426
+ At most once per delivery: the relay moves its place past a delivery before sending it, so a failure
427
+ is recorded with its reason and never retried. Each session is sent by one lease holder at a time, and a
428
+ holder whose lease ran out can neither move the session's place nor send again, so two workers never
429
+ send one delivery twice. A session gets 20 seconds of sending per tick; the rest waits for the next.
430
+ One tick pushes at most 200 sessions: the share is split evenly between opted-in projects, a project
431
+ with fewer live sessions than its share passing the rest on, and each project's share starts after the
432
+ last session its previous share ended on (kept in Redis, whatever a tick's length, sessions opening and
433
+ closing between ticks included), so every live session is reached within about `ceil(live / share)`
434
+ ticks; when more are live than one tick takes, the worker
435
+ logs an error. Ticks repeat every 5 seconds, one at a time on each worker replica (replicas may overlap; the
436
+ per-session lease keeps them apart), and the project's setting is read again as each session's lease
437
+ is taken, so turning push off stops every session not already sending; a tick lasts as long as its slowest
438
+ sessions (16 in parallel, each up to 10 seconds to poll and 20 of sending), so a delivery usually
439
+ leaves within seconds and, under full load, within minutes. Turning push off and on again starts every
440
+ session that already existed at its feed's head: nothing a world sent while push was off is pushed
441
+ (saving the setting unchanged moves nothing). The first time the relay writes a session's place is recorded on
442
+ the session, so a session whose place in Redis is lost (a day without a tick, a Redis failover) starts
443
+ again at its feed's head, logged at error, and is never sent its feed a second time; deliveries in that
444
+ gap are not pushed. Answers to a session's last deliveries are lost if
445
+ the session closes before the next tick (the deliveries stay in the ledger at status 0). There is no
446
+ platform-wide off switch beyond each project's own. Every enabled subscription row receives, including rows a seed or
447
+ an import put there. `session listen` and push can run together; the first answer to a delivery is
448
+ the one kept.
449
+
450
+ A world can send a registered webhook somewhere else without touching the world:
451
+ **World settings → Webhook destinations**, or the pencil on a subscription in the session's Webhooks
452
+ section, maps a registered URL (its scheme, host, port and path; never its query or userinfo) to a
453
+ destination (`world_webhook_destinations`, `WorldWebhookDestinations`, project admins and owners,
454
+ audit-logged). The push relay sends every delivery whose subscription registered that URL, in every
455
+ session of the world, to the destination instead, signed as before; the subscription row, what the
456
+ agent reads back and the ledger's `path` stay the registered URL. A destination passes the same checks
457
+ as a push (https on 443, no credentials, the platform's URL guard) when it is saved and again at every
458
+ send. At most 100 per world; `session listen` is unaffected (it sends to its own `--forward-to`).
459
+
200
460
  ## Capturing from the platform
201
461
 
202
462
  A vendor that allow-lists a caller's IP (SpyCloud) cannot be captured from a laptop. The same
@@ -227,11 +487,14 @@ author `connector.toml` + `schema/` in the sandbox → `run_gateway_cli connecto
227
487
  (`environments:write`; the key comes from the world's Secrets tab — `list_world_secrets` shows
228
488
  names only, and a `connector.toml` that inlines a credential is refused) → `get_connector_capture`
229
489
  (status, redacted file metadata, and once complete a copy of the pages into
230
- `<sandbox dir>/captures/`) → `run_gateway_cli connector ingest` → `ingest_to_world`. The
231
- sandbox never holds the vendor key or a platform credential: capture runs server-side exactly as
232
- the `--remote` path above, and the CLI wrapper runs the same TypeScript `gateway` CLI this page
233
- documents (`node /tmp/gateway-cli/cli.js worlds …`, staged from the web image's build) pinned to
234
- its local runtime, with `GATEWAY_*`/`LANGFUSE_*` stripped and `--host`/`--remote` refused. Details in
490
+ `<sandbox dir>/captures/`) → `run_gateway_cli connector ingest ./dir` → `run_gateway_cli push ./dir`
491
+ (the world checked out with `pull <slug> --dir ./dir`). The sandbox never holds the vendor key:
492
+ capture runs server-side exactly as the `--remote` path above. The CLI wrapper runs the same
493
+ TypeScript `gateway` CLI this page documents (`node /tmp/gateway-cli/cli.js worlds …`, staged
494
+ from the web image's build) pinned to its local runtime. A command on `./dir` runs with
495
+ `GATEWAY_*`/`LANGFUSE_*` stripped; a command on a slug reaches the platform as the user through a
496
+ project key minted for that one call (valid only on the routes that verb calls, deleted when the call
497
+ ends); `--host` and key flags are refused. Details in
235
498
  [agent.md](agent.md#platform-actions-worlds-tasks-rollouts-connectors-ingestion-self-schedules).
236
499
 
237
500
  ## Conformance and scenarios
@@ -369,10 +632,11 @@ gateway worlds connector skill --install . # the authoring skill + agen
369
632
  unless it is cloned again.
370
633
  - **Initializing from the platform's copy.** `--connector workspace:<slug>` pulls the head commit
371
634
  and calls `init_bundle_from_files`, the same path the platform's `init-template-files` bridge
372
- uses; the manifest records `connector = "<slug>"`. Logged out (or under the agent's
373
- local-only `run_gateway_cli`), `gateway worlds schema init --connector workspace:<slug>` refuses
635
+ uses; the manifest records `connector = "<slug>"`. Logged out,
636
+ `gateway worlds schema init --connector workspace:<slug>` refuses
374
637
  and names `gateway auth login`; the offline `python -m gatewaysdk.environment.schema init`
375
- refuses it the same way.
638
+ refuses it the same way. The agent's `run_gateway_cli` refuses it before running and names
639
+ `connector pull <slug> --out ./dir` instead.
376
640
  - **The agent CTA.** `gateway worlds connector skill` prints the `connector-schema-authoring`
377
641
  skill; `--install DIR` writes SKILL.md, CONVENTIONS.md and CONNECTORS.md plus the
378
642
  `connector-template-author` agent under `DIR/.claude/`, where Claude Code, Cursor and Codex
@@ -25,7 +25,7 @@ session](#raw-ingestion-schema-gated-sql), and connector
25
25
  | SDK/public and platform publication, including READY/dedup/fork/ref promotion | Verifies the stored SHA-256, generated artifacts and exact lock through the trusted compiler/materializer using actual archive bytes; uploaded Python is parsed, not imported during publication. |
26
26
  | Worker builder, bundle tester, and workbench | Materialize through the shared schema store; tester/workbench execution uses an isolated sandbox with an installed trusted runtime. No legacy seed-SQL fallback. |
27
27
  | Hosted `mock-tools` sessions | Use the shared `World` in a child process for validated calls, reset and grading; dependency worlds are unsupported. |
28
- | Platform agent (`create_custom_world`, `ingest_records_to_world`, `ingest_to_world`) | Authors and fills arbitrary schema worlds through the trusted bridge (`platform.py compile` / `apply-batch`) and publishes each result as one immutable READY version via `worldPublicationService`; see [agent.md](./agent.md#platform-actions-worlds-tasks-rollouts-connectors-ingestion-self-schedules). |
28
+ | Platform agent (`run_gateway_cli`) | Edits schema worlds in its sandbox with the `gateway worlds` CLI and publishes as the user through its doors (`create`, `push`, `data import <slug>`), checked like any CLI push; see [agent.md](./agent.md#platform-actions-worlds-tasks-rollouts-connectors-ingestion-self-schedules). |
29
29
  | SDK Harbor worlds and Harbor/API/verifiers engines | Retain their separate unsupported-mode guards; do not combine their bundle markers with schema mode. |
30
30
 
31
31
  Environment Actions remain advisory. See [Environment Builder](./environment-builder.md) and
@@ -280,7 +280,8 @@ facade over it admitting exactly `get, list, list_where, insert, update, delete,
280
280
  lineage, counts` — the one list in `environment/schema/store_contract.py` (`STORE_METHODS`, read off
281
281
  `WorldDatabase` so it cannot drift). `gateway worlds schema check` refuses any other `store.<name>`
282
282
  statically at file:line with that list; a hosted call into a handler whose own code raises
283
- `AttributeError`/`TypeError`/`NameError` answers 400 `{error: "handler", detail, handler}`, any other
283
+ `AttributeError`/`TypeError`/`NameError` answers 400 `{error: "handler", detail, handler}`,
284
+ `raise Refusal(status, body)` (from `gateway_env`, a 4xx) answers that status and body, any other
284
285
  handler failure 500 `{error: "engine", detail, handler}` — never a bare exception string. There is
285
286
  no `upsert`: `get`, then `insert` or `update`.
286
287
 
@@ -616,13 +617,13 @@ as a credential-free subprocess (`GATEWAY_SCHEMA_PYTHON`, `GATEWAY_SCHEMA_RUNTIM
616
617
  | Command | Used by | Effect |
617
618
  | ------------------------------ | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
618
619
  | `validate`, `validate-archive` | publication | Verify SHA-256, regenerated artifacts and exact lock of an existing tree/archive. |
619
- | `init-template` | `create_connector_world`, `create_custom_world --template` | Scaffold a connector template into a tree. |
620
+ | `init-template` | `create_connector_world`, `worlds create --from TEMPLATE` | Scaffold a connector template into a tree. |
620
621
  | `init-openapi` | `worlds schema init --from-openapi` (both CLIs; the TypeScript CLI through `POST /api/public/worlds/runtime` only when no runtime is here) | `{source, document (base64), destination, plan_only?, include?, exclude?, name?}` → the import plan and, when written, the compiled tree as `files[]` (at most 2000). The bridge never opens a caller-named path or URL; the CLI stages the bytes. A YAML document needs PyYAML in the Python that reads it, and is refused naming the fix without it. |
621
- | `compile` | `create_custom_world`, `ingest_to_world` on schema worlds | Compile an authored tree (`compileWorldFiles`): returns the full entry map with generated artifacts + lock, the schema report and which files were added/retargeted. |
622
- | `apply-batch` | `ingest_records_to_world` | Apply `insert\|update\|delete` mutations as one batch (`applyWorldBatch`): validated row by row against the compiled schema, appended as `data/batches/<batchId>.json` with `source` provenance; replaying the same `batchId` is a no-op. |
622
+ | `compile` | `POST /api/public/worlds/runtime` | Compile an authored tree (`compileWorldFiles`): returns the full entry map with generated artifacts + lock, the schema report and which files were added/retargeted. |
623
+ | `import-data` | `importWorldData`: `worlds create --rows`, `import_world_data` on a world without a contract, `gateway worlds data import <dir>` (the CLI's bundled runtime, else `POST /api/public/worlds/runtime`) | Lay `{entity: [row]}` rows into the tree through the world's own gate (`data.import_rows`, `append` or `replace`: the contract of a schema world, `db/schema.sql` of a mock-tools one), rebuild a schema world's snapshot and return the updated tree; a refused row writes nothing. |
623
624
  | `build-slack`, `refresh-slack` | connector worlds | Slack capture → batches (durable job). |
624
625
 
625
- Every agent write therefore passes through the schema: `ingest_to_world` recompiles a schema world before it becomes head, and `ingest_records_to_world` refuses invalid rows with the field path instead of publishing a partial batch.
626
+ Every agent write still passes through the schema, now through the CLI's own doors (`run_gateway_cli`, 2026-10-10): `create` and `push` land as versions checked like any CLI push, and `data import <slug>` validates every row through the contract in a data batch, with refusals naming the field.
626
627
 
627
628
  ## Compilation, locks, and compatibility
628
629
 
@@ -264,17 +264,18 @@ schema type, the same rule in the TypeScript CLI (`worldData/cast.ts`) and the P
264
264
 
265
265
  | Field type | Cell | Value |
266
266
  | --- | --- | --- |
267
- | `integer` | `42`, `-7` | the number; `4.5` or `abc` is refused |
267
+ | `integer` | `42`, `-7` | the number; `4.5` or `abc` stays text, refused by the contract |
268
268
  | `number` | `12.5`, `1e3`, `3` | the number (always a float) |
269
- | `boolean` | `true`/`false`/`1`/`0`, any case | the boolean; `yes` is refused |
269
+ | `boolean` | `true`/`false`/`1`/`0`, any case | the boolean; `yes` stays text, refused by the contract |
270
270
  | `array` / `object` | JSON text of that shape (quote it in the CSV: `"[""a"",""b""]"`) | the parsed value |
271
271
  | `string` | anything | the text as given |
272
272
  | `integer \| string` (union) | each member tried in order, `string` last | the first that fits |
273
- | any, empty cell | `` | null when the type admits null; left out when the field is optional; `""` for a required string; refused otherwise |
273
+ | any, empty cell | (nothing) | null when the type admits null; `""` for a required string; left out otherwise (a required one is then refused by the contract) |
274
274
 
275
- A cell that fits none is a row-level refusal naming the field and the cell —
276
- `/taxi_trip/3: fare = "n/a" is not a number` — with the usual pointer; nothing is written. A
277
- column the schema does not name stays text and the gate refuses it as an unknown field. A CSV
275
+ A cell that fits none stays its text and the contract refuses the row as it refuses a JSON row
276
+ — `taxi_trip {"id":3}.fare: Expected number`, with the usual pointer — so a file reports every
277
+ bad row at once (`--report` writes them all) and nothing is written. A column the schema does
278
+ not name stays text and the gate refuses it as an unknown field. A CSV
278
279
  on the drive (`drive:<path>`) is still refused: import it from the machine that holds it, or
279
280
  through `worlds sandbox import` when it sits in a sandbox.
280
281
 
@@ -311,8 +312,11 @@ the package version keeps every snapshot (local, hosted, pinned), with no re-imp
311
312
  that only adds to the domain (a new entity, a new field that is not required, another
312
313
  `domain_version`) carries the rows: compile migrates the snapshots the tree holds and keeps the
313
314
  pin with the stored domain's `lock` and `entities`, `sandbox check` says the commit migrates the
314
- hosted snapshot, and `sandbox commit` migrates it and re-pins the tree (a CLI push of such a
315
- change is refused, naming the sandbox). Any other change to an entity, a field (its description
315
+ hosted snapshot, and `sandbox commit` migrates it as a job (a `migration` batch: the CLI follows it,
316
+ MCP answers `{batchId}` for `get_data_batch`) and re-pins the tree (a CLI push of such a change is
317
+ refused, naming the sandbox). Importing exactly the batch behind the served snapshot again (the
318
+ same chunks in the same order, with `redact` off) is a no-op: the batch lands on the same version
319
+ and names the one it repeats (`unchangedFrom`). Any other change to an entity, a field (its description
316
320
  or title too), a key or a relationship refuses it, naming each change:
317
321
 
318
322
  ```text