@withgateway/sdk 0.1.20 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (130) hide show
  1. package/README.md +141 -2
  2. package/dist/cli.js +8683 -1343
  3. package/dist/cli.js.map +1 -1
  4. package/dist/cli.mjs +8683 -1344
  5. package/dist/cli.mjs.map +1 -1
  6. package/dist/index.d.mts +1 -1
  7. package/dist/index.d.ts +1 -1
  8. package/dist/index.js +979 -46
  9. package/dist/index.js.map +1 -1
  10. package/dist/index.mjs +943 -38
  11. package/dist/index.mjs.map +1 -1
  12. package/dist/tracing/index.js +9 -2
  13. package/dist/tracing/index.js.map +1 -1
  14. package/dist/tracing/index.mjs +9 -2
  15. package/dist/tracing/index.mjs.map +1 -1
  16. package/dist/worlds/index.d.mts +738 -28
  17. package/dist/worlds/index.d.ts +738 -28
  18. package/dist/worlds/index.js +940 -42
  19. package/dist/worlds/index.js.map +1 -1
  20. package/dist/worlds/index.mjs +934 -36
  21. package/dist/worlds/index.mjs.map +1 -1
  22. package/package.json +9 -2
  23. package/runtime/gatewaysdk/__init__.py +0 -0
  24. package/runtime/gatewaysdk/environment/__init__.py +108 -0
  25. package/runtime/gatewaysdk/environment/_bundle.py +281 -0
  26. package/runtime/gatewaysdk/environment/_harbor.py +276 -0
  27. package/runtime/gatewaysdk/environment/_materialize.py +97 -0
  28. package/runtime/gatewaysdk/environment/_tar.py +33 -0
  29. package/runtime/gatewaysdk/environment/_world.py +346 -0
  30. package/runtime/gatewaysdk/environment/core.py +527 -0
  31. package/runtime/gatewaysdk/environment/errors.py +58 -0
  32. package/runtime/gatewaysdk/environment/runtime.py +150 -0
  33. package/runtime/gatewaysdk/environment/schema/__init__.py +35 -0
  34. package/runtime/gatewaysdk/environment/schema/__main__.py +225 -0
  35. package/runtime/gatewaysdk/environment/schema/_slack_fidelity.py +116 -0
  36. package/runtime/gatewaysdk/environment/schema/_slack_scenario.py +183 -0
  37. package/runtime/gatewaysdk/environment/schema/api.py +1628 -0
  38. package/runtime/gatewaysdk/environment/schema/batch.py +755 -0
  39. package/runtime/gatewaysdk/environment/schema/compiler.py +615 -0
  40. package/runtime/gatewaysdk/environment/schema/conform.py +428 -0
  41. package/runtime/gatewaysdk/environment/schema/connector.py +777 -0
  42. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/handlers.py +17 -0
  43. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/parity.json +28 -0
  44. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/provenance.json +233 -0
  45. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/scope.toml +112 -0
  46. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/world.json +1380 -0
  47. package/runtime/gatewaysdk/environment/schema/connectors/base.json +28 -0
  48. package/runtime/gatewaysdk/environment/schema/connectors/custom/handlers.py +18 -0
  49. package/runtime/gatewaysdk/environment/schema/connectors/custom/world.json +123 -0
  50. package/runtime/gatewaysdk/environment/schema/connectors/github/handlers.py +81 -0
  51. package/runtime/gatewaysdk/environment/schema/connectors/github/parity.json +37 -0
  52. package/runtime/gatewaysdk/environment/schema/connectors/github/provenance.json +134 -0
  53. package/runtime/gatewaysdk/environment/schema/connectors/github/scope.toml +158 -0
  54. package/runtime/gatewaysdk/environment/schema/connectors/github/world.json +5869 -0
  55. package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/handlers.py +23 -0
  56. package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/world.json +209 -0
  57. package/runtime/gatewaysdk/environment/schema/connectors/google-drive/handlers.py +18 -0
  58. package/runtime/gatewaysdk/environment/schema/connectors/google-drive/world.json +286 -0
  59. package/runtime/gatewaysdk/environment/schema/connectors/jamf/handlers.py +34 -0
  60. package/runtime/gatewaysdk/environment/schema/connectors/jamf/parity.json +49 -0
  61. package/runtime/gatewaysdk/environment/schema/connectors/jamf/provenance.json +181 -0
  62. package/runtime/gatewaysdk/environment/schema/connectors/jamf/scope.toml +128 -0
  63. package/runtime/gatewaysdk/environment/schema/connectors/jamf/world.json +2218 -0
  64. package/runtime/gatewaysdk/environment/schema/connectors/jira/handlers.py +28 -0
  65. package/runtime/gatewaysdk/environment/schema/connectors/jira/world.json +389 -0
  66. package/runtime/gatewaysdk/environment/schema/connectors/kandji/handlers.py +38 -0
  67. package/runtime/gatewaysdk/environment/schema/connectors/kandji/parity.json +29 -0
  68. package/runtime/gatewaysdk/environment/schema/connectors/kandji/provenance.json +152 -0
  69. package/runtime/gatewaysdk/environment/schema/connectors/kandji/scope.toml +120 -0
  70. package/runtime/gatewaysdk/environment/schema/connectors/kandji/world.json +765 -0
  71. package/runtime/gatewaysdk/environment/schema/connectors/linear/handlers.py +20 -0
  72. package/runtime/gatewaysdk/environment/schema/connectors/linear/world.json +363 -0
  73. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/fidelity.json +45 -0
  74. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/handlers.py +1 -0
  75. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/provenance.json +851 -0
  76. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/scope.md +231 -0
  77. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/teams-types.json +1710 -0
  78. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/world.json +858 -0
  79. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/handlers.py +32 -0
  80. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/parity.json +50 -0
  81. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/provenance.json +137 -0
  82. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/scope.toml +78 -0
  83. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/world.json +16243 -0
  84. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/handlers.py +806 -0
  85. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/parity.json +46 -0
  86. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/provenance.json +143 -0
  87. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/scope.toml +174 -0
  88. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/world.json +4695 -0
  89. package/runtime/gatewaysdk/environment/schema/connectors/slack/MODEL.md +236 -0
  90. package/runtime/gatewaysdk/environment/schema/connectors/slack/capabilities.json +1039 -0
  91. package/runtime/gatewaysdk/environment/schema/connectors/slack/handlers.py +762 -0
  92. package/runtime/gatewaysdk/environment/schema/connectors/slack/parity.json +65 -0
  93. package/runtime/gatewaysdk/environment/schema/connectors/slack/provenance.json +294 -0
  94. package/runtime/gatewaysdk/environment/schema/connectors/slack/scope.toml +44 -0
  95. package/runtime/gatewaysdk/environment/schema/connectors/slack/world.json +4993 -0
  96. package/runtime/gatewaysdk/environment/schema/connectors/workday/handlers.py +75 -0
  97. package/runtime/gatewaysdk/environment/schema/connectors/workday/parity.json +36 -0
  98. package/runtime/gatewaysdk/environment/schema/connectors/workday/provenance.json +179 -0
  99. package/runtime/gatewaysdk/environment/schema/connectors/workday/scope.toml +180 -0
  100. package/runtime/gatewaysdk/environment/schema/connectors/workday/world.json +806 -0
  101. package/runtime/gatewaysdk/environment/schema/data.py +593 -0
  102. package/runtime/gatewaysdk/environment/schema/extract.py +263 -0
  103. package/runtime/gatewaysdk/environment/schema/host.py +2407 -0
  104. package/runtime/gatewaysdk/environment/schema/host_surfaces.py +275 -0
  105. package/runtime/gatewaysdk/environment/schema/platform.py +1272 -0
  106. package/runtime/gatewaysdk/environment/schema/scaffold.py +250 -0
  107. package/runtime/gatewaysdk/environment/schema/skill.py +118 -0
  108. package/runtime/gatewaysdk/environment/schema/skills/agents/connector-template-author.md +202 -0
  109. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONNECTORS.md +331 -0
  110. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
  111. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +313 -0
  112. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +223 -0
  113. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +293 -0
  114. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +390 -0
  115. package/runtime/gatewaysdk/environment/schema/slack.py +757 -0
  116. package/runtime/gatewaysdk/environment/schema/snapshot.py +479 -0
  117. package/runtime/gatewaysdk/environment/schema/store.py +904 -0
  118. package/runtime/gatewaysdk/environment/schema/tasks.py +525 -0
  119. package/runtime/gatewaysdk/environment/schema/tools_world.py +322 -0
  120. package/runtime/gatewaysdk/environment/schema/validation.py +577 -0
  121. package/runtime/gatewaysdk/environment/schema/worker.py +78 -0
  122. package/runtime/gatewaysdk/environment/schema/workers.py +500 -0
  123. package/runtime/gatewaysdk/environment/schema/world_tests.py +336 -0
  124. package/skills/agents/connector-template-author.md +202 -0
  125. package/skills/connector-schema-authoring/CONNECTORS.md +331 -0
  126. package/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
  127. package/skills/connector-schema-authoring/SKILL.md +313 -0
  128. package/skills/world-data-ingestion/ROWS.md +223 -0
  129. package/skills/world-data-ingestion/SKILL.md +293 -0
  130. package/skills/worlds-getting-started/SKILL.md +390 -0
@@ -0,0 +1,390 @@
1
+ ---
2
+ name: worlds-getting-started
3
+ description: Starter guide for building worlds and environments on the platform, end to end, for an agent (Claude Code, Cursor, Codex). Use when starting a world from scratch, building a world or environment, mocking a vendor, choosing between a shipped template, a workspace connector, a vendor API capture or a custom contract, provisioning worlds for tenants in one call (worlds create), publishing with bench push, opening a session, running and grading, or iterating with pull, fork, diff and propose. Every CLI step names its MCP twin (list_connectors, publish_connector, create_world, init_world_from_connector, env_write_file, env_commit, import_world_data, run_evaluation and the rest) and where the result shows in the UI.
4
+ ---
5
+
6
+ # Worlds: getting started
7
+
8
+ > **Two copies, kept identical.** This file ships inside the `gatewaysdk` Python package
9
+ > (`gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md`) and the `@withgateway/sdk`
10
+ > npm package (copied into `skills/` at build) and is what
11
+ > `gateway worlds skill worlds-getting-started` prints and `--install DIR` writes into a
12
+ > repository's `.claude/skills/`. The repository copy is `.claude/skills/worlds-getting-started/SKILL.md`.
13
+ > Edit one, copy to the other; a test compares them.
14
+
15
+
16
+ > **Two copies, kept identical.** This file ships inside the `gatewaysdk` Python package
17
+ > (`gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md`) and the `@withgateway/sdk`
18
+ > npm package (copied into `skills/` at build) and is what
19
+ > `gateway worlds skill worlds-getting-started` prints and `--install DIR` writes into a
20
+ > repository's `.claude/skills/`. The repository copy is `.claude/skills/worlds-getting-started/SKILL.md`.
21
+ > Edit one, copy to the other; a test compares them.
22
+
23
+ ## Purpose
24
+
25
+ The shortest correct path from nothing to a hosted world with data, a session, a graded
26
+ run and a next version. CLI first; the MCP twin sits under each step for an agent that
27
+ talks to the platform over MCP instead of a shell. Deeper references are linked by path,
28
+ not repeated.
29
+
30
+ Vocabulary (from `gateway_documentation/pages/glossary.mdx`): **world** (the sealed
31
+ replica), **scenario** (a task in it), **rollout** (one attempt), **run** (a filed set of
32
+ rollouts), **eval** (the grade), **gate** (the bar). Identifiers keep the old words:
33
+ `environment`, `container`, `task set`, `bench`.
34
+
35
+ ## 0. Authenticate
36
+
37
+ ```bash
38
+ npm install -g @withgateway/sdk # the gateway CLI (TypeScript; no Python needed)
39
+ gateway auth login --host https://<your-gateway> --public-key pk-lf-... --api-key sk-lf-...
40
+ gateway auth status # or export GATEWAY_HOST / GATEWAY_PUBLIC_KEY / GATEWAY_SECRET_KEY
41
+ ```
42
+
43
+ The npm package bundles the schema runtime (stdlib-only Python). With a `python3 >= 3.11` on the
44
+ machine every runtime command runs locally, including `worlds serve` and a local `connector
45
+ capture`, with no pip and no login; without one, the CLI sends the world's files to the platform
46
+ (`POST /api/public/worlds/runtime`) and the same code runs there (`worlds serve` then opens a hosted
47
+ api session, `connector capture` needs `--remote`). `GATEWAY_RUNTIME=local|platform` forces a side.
48
+
49
+ Or export `GATEWAY_HOST`, `GATEWAY_PUBLIC_KEY`, `GATEWAY_SECRET_KEY` (env wins over the
50
+ config file). Every command below accepts `--host`. Keys are project keys: the world lands
51
+ in that project.
52
+
53
+ MCP: the server is authenticated by the same project API key (or a PAT). Call
54
+ `describe_project({})` first, then `guide({ topic: "connectors" })` for the connector loop
55
+ and `guide({ topic: "world-data" })` for data.
56
+
57
+ ## 1. Pick a path
58
+
59
+ | You have | Path | First command |
60
+ | --- | --- | --- |
61
+ | A vendor the SDK ships a template for (`gateway worlds schema templates`: github, slack, salesforce, jira, linear, workday, netsuite, microsoft-teams, google-drive, google-calendar, jamf, kandji, apple-business-manager, constella) | shipped template | `gateway worlds schema init ./x --connector github` |
62
+ | A connector someone in your org (or any org) already published | workspace connector | `gateway worlds connector list --hub` then `schema init ./x --connector workspace:<slug>` |
63
+ | A vendor API with a dev key and no template | vendor API capture | `schema init ./x --connector custom`, write `connector.toml`, capture, ingest (`documentation/features/connector-worlds.md`) |
64
+ | A data model and nothing else | custom contract | `schema init ./x --connector custom`, author `schema/world.json` (`connector-schema-authoring`) |
65
+ | SQL tables and Python tool handlers | mock-tools world | `gateway-env.toml` + `db/schema.sql` + `db/seed.sql` + `tools/`; no schema contract |
66
+ | A Harbor task suite | harbor world | the task dirs as they are; `bench push` |
67
+ | A tenant to provision for, no local edit loop (a template, a hub connector, optionally your overlay and their rows) | one-call create | `gateway worlds create acme-slack --from slack --rows rows.json`; see §5 |
68
+
69
+ Rule from the owner: a world is read and write 1:1. Reads from the store alone are a lookup
70
+ mock; a vendor with workflows needs `action = "create|update|delete|handler"` routes and
71
+ `api/handlers.py`. Do not ship a read-only world and call it done.
72
+
73
+ MCP: `list_connectors({ query? })` (own + public), `get_connector({ slug, includeFiles: true })`
74
+ to read a template, `clone_connector({ slug })` to take a public one into this org.
75
+
76
+ ## 2. Author the contract
77
+
78
+ Layout of a schema world after `init`:
79
+
80
+ ```
81
+ x/
82
+ schema/world.json # gateway-world/1: entities, fields, keys, relationships (the deliverable)
83
+ schema/provenance.json # where the model came from, pinned
84
+ connector.toml # vendor connection + the routes the served world answers (optional)
85
+ tools/handlers.py # one handler per tool in x-gateway.tools (optional)
86
+ api/handlers.py # workflow handlers for action = "handler" routes (optional)
87
+ scenarios/<name>/ # scenario.toml + seed/*.json overlays + instruction.md
88
+ data/initial.json # the seed batch (written by ingest / data import)
89
+ captures/ # raw vendor pages; never committed to a template
90
+ ```
91
+
92
+ Author `schema/world.json` per `connector-schema-authoring` (conventions, prefixes, keys).
93
+ Write `connector.toml` per `documentation/features/connector-worlds.md` ("The spec, in
94
+ full"). Both have scaffolds so the declaration shape cannot be wrong; only payload
95
+ definitions and handler bodies are hand-written:
96
+
97
+ ```bash
98
+ gateway worlds schema entity add ./x crm_accounts --fields "id:string,name:string,tier:string?" --key id
99
+ gateway worlds schema entity add ./x crm_contacts --ref CrmContact --key id --relationship account_id->crm_accounts
100
+ gateway worlds connector route add ./x list_accounts --path /v1/accounts --param page --entity crm_accounts --results data
101
+ gateway worlds connector route add ./x get_account --path /v1/accounts/{id} --entity crm_accounts --filter id=id --single
102
+ gateway worlds connector route add ./x create_account -X POST --path /v1/accounts --entity crm_accounts --action create --body name=name
103
+ ```
104
+
105
+ Each refuses (and writes nothing) on a duplicate name, an undeclared entity or `$defs`
106
+ reference, a key that is not a required string/integer field, or a filter on an unknown
107
+ argument. Handlers: `def name(ctx, args, query, body)`; `ctx` reads and writes the store and
108
+ keeps the clock.
109
+
110
+ MCP: `get_connector_authoring_skill({})` returns the authoring skill and conventions.
111
+ Author files locally or with `env_write_file({ containerId, path, newContent })` on an existing
112
+ environment; land them with `env_commit({ containerId, changeReason })` (one version for the batch).
113
+
114
+ ## 3. Check
115
+
116
+ ```bash
117
+ gateway worlds schema compile ./x
118
+ gateway worlds schema check ./x
119
+ gateway worlds connector validate ./x # only when connector.toml exists
120
+ gateway worlds connector conform ./x --openapi vendor.yaml # only when the vendor publishes OpenAPI
121
+ gateway worlds serve ./x --port 8080 # answers the vendor's routes locally
122
+ ```
123
+
124
+ `check` validates the compiled bundle; `conform` drives every declared route against the
125
+ vendor's document and exits non-zero on a mismatch. Its FAIL list is the work list.
126
+
127
+ MCP: `validate_connector_template({ files, slug? })` compiles without writing and answers
128
+ `{ valid, reason? }`.
129
+
130
+ ## 4. Data
131
+
132
+ ```bash
133
+ gateway worlds data show ./x # kind + entity counts
134
+ gateway worlds data import ./x rows.json # rows file: {entity: [rows]} | JSONL | --entity
135
+ gateway worlds connector capture ./x --op <name> --arg k=v # from the real vendor, redacted
136
+ gateway worlds connector ingest ./x # captures -> data/initial.json
137
+ ```
138
+
139
+ Rows file contract, modes, upsert and refusals: `world-data-ingestion`. Captures: pace
140
+ with `[capture] requests_per_minute`; a vendor that allow-lists IPs is captured with
141
+ `--remote` after `gateway bench secrets set --env x --key VENDOR_API_KEY`.
142
+
143
+ MCP: `import_world_data({ containerId | slug, rows, mode?, message? })` on a pushed world.
144
+
145
+ ## 5. Publish
146
+
147
+ ```bash
148
+ gateway bench push ./x -m "first cut" # a world version; api bundle packaged when connector.toml exists
149
+ gateway worlds connector publish ./x --slug x -m "first cut" # the template as a workspace connector (org-owned, versioned)
150
+ ```
151
+
152
+ `bench push` makes the version sessions and runs address (`slug@version`). `connector publish`
153
+ saves the contract (never `data/`, `captures/`, `db/`) so anyone in the org can
154
+ `schema init --connector workspace:x`. Both, for a world you expect to rebuild.
155
+
156
+ MCP: `publish_connector({ slug, files, message? })` is the template write path.
157
+ `init_world_from_connector({ slug, name })` makes the world under a PAT session; under an API
158
+ key it returns the initialized files plus the publish steps (`env_write_file` per file, then
159
+ `env_commit`, or `create_environment_version({ containerId, path, newContent, changeReason })`
160
+ for one file).
161
+
162
+ ### One call, no local tree: `worlds create`
163
+
164
+ For a world you provision rather than iterate on (a tenant's Slack, one per customer):
165
+
166
+ ```bash
167
+ gateway worlds create acme-slack --from slack --rows rows.json --name "Acme Slack" # shipped template + their rows
168
+ gateway worlds create acme-crm --from workspace:acme-crm --overlay ./overlay --json # hub connector + your edits
169
+ gateway worlds create acme-custom --from ./x # a directory, any kind
170
+ gateway worlds create --batch worlds.jsonl # {slug, from, overlay?, data?, name?} per line; exit 2 if any failed
171
+ ```
172
+
173
+ One request (`POST /api/public/worlds`, `documentation/api/rest/worlds-create.md`): the source,
174
+ an optional overlay laid over it, optional rows through the world's contract. An overlay or a
175
+ directory is compiled by the schema runtime before anything is stored, so a broken edit is a
176
+ 400 naming the path, not a broken world. The result is what `bench push` lands: a container
177
+ and a READY first version, attributed to the key (`apikey:<id>`), owned by nobody. 409 when
178
+ the slug is taken in the project.
179
+
180
+ MCP: `create_world({ slug, from: { connector } | { files }, overlay?, data?, name?, message? })`
181
+ is the same request and answer. `init_world_from_connector` stays for the PAT flow that wants a
182
+ human owner.
183
+
184
+ ## 6. Open a session
185
+
186
+ ```bash
187
+ gateway worlds session open x -t default # tools surface; prints {sessionId, surfaces}
188
+ gateway worlds session open x -t default --surface api # adds api.url: the vendor's routes, hosted
189
+ gateway worlds session open x -t default --no-wait # return at once on a cold version
190
+ gateway worlds session open x --task-file task.json --surface api # the session's own task (below)
191
+ gateway worlds session status <SESSION_ID> # poll until ready
192
+ gateway worlds session seed <SESSION_ID> rows.json # rows into this session only
193
+ gateway worlds session state <SESSION_ID> # every entity's rows, the dump a grader reads
194
+ gateway worlds session reset <SESSION_ID> # back to the snapshot (+ the task's seed)
195
+ gateway worlds session grade <SESSION_ID> [--timeout S] # {reward, rewards, raw}; a rubric judge may take a while
196
+ gateway worlds session export <SESSION_ID> --out calls.jsonl
197
+ gateway worlds session close <SESSION_ID>
198
+ ```
199
+
200
+ **A task outside the bundle.** `-t <name>` picks a scenario the version ships. On a World Host
201
+ world (every schema / handler / mock-tools world once the host is configured; a pod world
202
+ answers 400) `--task-file task.json` supplies the task with the open instead — one published
203
+ world, a task per end user, no `bench push` per goal:
204
+
205
+ ```json
206
+ { "name": "acme-q3", "instruction": "Mark every open Q3 invoice for Acme paid.",
207
+ "seed": { "rows": { "invoices": [ { "id": "INV-7", "status": "open" } ] } },
208
+ "grader": { "kind": "assertions", "checks": [
209
+ { "id": "paid", "entity": "invoices", "where": { "id": "INV-7", "status": "paid" } } ] },
210
+ "metadata": { "externalUserId": "u_42" } }
211
+ ```
212
+
213
+ `instruction` is what the agent is told; `seed` (`{mode?, rows}`, the live-seed contract) is
214
+ laid on the clone and re-applied by `reset`; `grader` is `assertions` (`entity`, `where`,
215
+ `exists|absent|count|all|any`, `field op value`; each check's `id` becomes a `rewards` key),
216
+ `{kind: "python", source}` (a `verify.py` body) or `{kind: "rubric", rubric, criteria?, model?}`
217
+ (an LLM judge over end state + call log, graded on the platform: `grade` answers pending until
218
+ the worker finalizes it); `tools: [names]` narrows the operations the session may call to
219
+ exactly those (a name the world does not serve is a 400); `metadata` rides the session. Exactly one of
220
+ `-t` / `--task-id` / `--task-file`. The output's `task` block is `{source: "inline", name, hash, metadata}`.
221
+ SDK: `openSession(slug, { task: spec })` (TS), `open_session(slug, spec)` (Python);
222
+ `runSessions` / `run_sessions` take a mixed list of names and specs.
223
+
224
+ **A stored task.** The same spec kept as a project resource, versioned by content, so many
225
+ sessions open it by id and grades attribute to the task and its version:
226
+
227
+ ```bash
228
+ gateway worlds task create task.json --world x # version 1; --name overrides the spec's name
229
+ gateway worlds task list --world x --metadata externalUserId=u_42
230
+ gateway worlds task show <TASK_ID> [--version 2] # spec + version history
231
+ gateway worlds task validate <TASK_ID> # seed/grader entities against the pinned world's version
232
+ gateway worlds task update <TASK_ID> --spec task2.json # a changed spec is the next version; a rename is not
233
+ gateway worlds session open x --task-id <TASK_ID> [--task-version 1] --surface api
234
+ gateway worlds session task save <SESSION_ID> --name acme-q3 # promote a session's inline task into a stored one
235
+ gateway worlds task delete <TASK_ID>
236
+ ```
237
+
238
+ SDK: `createTask / listTasks / getTask / updateTask / validateTask / deleteTask`,
239
+ `openSession(slug, { task: { id, version? } })`, `session.saveTask(name)` (TS);
240
+ `create_task` … `delete_task`, `open_session(slug, {"id": …})`, `session.save_task(name)`
241
+ (Python). MCP: `create_world_task`, `update_world_task`, `list_world_tasks`, `get_world_task`,
242
+ `validate_world_task`, `delete_world_task`, `save_world_session_task`. The world's Scenarios
243
+ tab lists the tasks pinned to it under a `world` kind chip.
244
+
245
+ **Stored tasks as a hosted eval.** A run config dispatches EITHER the bundle's scenarios
246
+ (`--tasks`) OR stored tasks (`--world-tasks`), never both:
247
+
248
+ ```bash
249
+ gateway bench run-configs create --name nightly --models openai/gpt-5-mini --world-tasks <TASK_ID>,<OTHER_ID>@2
250
+ gateway bench dispatch <RUN_CONFIG_ID> # each stored task is a sample; rubric tasks are refused at dispatch
251
+ ```
252
+
253
+ `<id>` runs the task's current version at dispatch time, `<id>@N` that version forever. SDK:
254
+ `upsertRunConfig({ name, models, worldTasks: [{ id }, { id, version: 2 }] })` (TS),
255
+ `upsert_run_config(name=…, models=…, worldTasks=[…])` (Python); `parseWorldTaskRefs` /
256
+ `parse_world_task_refs` turn the `--world-tasks` spelling into refs.
257
+
258
+ The first session on a freshly pushed version takes about fifteen minutes: the pod is
259
+ scheduled and the runner sidecar pulled; the session reads `PROVISIONING` and the api URL
260
+ answers 503. That is a cold start, not a fault, until about twenty minutes have passed.
261
+ `session open` waits ten minutes by default; on a cold version use `--no-wait` and poll.
262
+
263
+ The world's API honours the vendor's auth scheme; `WORLD_API_KEY` on the host pins the
264
+ credential, otherwise any non-empty one is accepted.
265
+
266
+ MCP: `seed_world_session({ sessionId, rows, mode? })`, `export_world_session({ sessionId })`.
267
+ Opening a session has no MCP tool; use the CLI or the public sessions route.
268
+
269
+ ## Test the world (not the agent)
270
+
271
+ A world ships its own tests under `tests/` and they run on every push. Tests assert the WORLD
272
+ answers as you say it must; evals (below) grade an agent on a task. Keep them apart.
273
+
274
+ ```bash
275
+ gateway worlds test ./my-world # tests/*.json HTTP cases + tests/test_*.py, served locally
276
+ gateway worlds test my-world # the same suite on the platform, against the pushed head
277
+ gateway bench tests my-world # every run the platform kept (push, CLI, API)
278
+ ```
279
+
280
+ A case: `{"name": "...", "request": {"method": "GET", "path": "/v1/accounts", "auth": true},
281
+ "expect": {"status": 200, "length": {"path": "$", "min": 1}}}`. Put `[actions]` `on_push = ["tests"]`
282
+ in `gateway-env.toml` and the platform runs the suite on each push (world page → Tests tab). Public
283
+ API: `POST/GET /api/public/worlds/{slug}/tests`. Details: `documentation/features/world-tests.md`.
284
+
285
+ ## 7. Run and grade
286
+
287
+ ```bash
288
+ gateway worlds run x -a agent.py:agent -m <model> # pull, run the callable against the world, file a run
289
+ gateway worlds run --path ./x -a agent.py:agent --no-file # local, unfiled
290
+ gateway worlds ci --path ./x -m <model> --fail-under 0.8 # push the working tree, run, gate (CI)
291
+ gateway environment run x -m <provider/model> # hosted runner, one job per model
292
+ gateway environment run-status <run-id>
293
+ ```
294
+
295
+ Scenarios (task sets):
296
+
297
+ ```bash
298
+ gateway environment link-task-set x --tasks tasks.jsonl # rows: id, prompt, metadata, worlds; idempotent
299
+ gateway environment tasks <container-id> # the linked set and its tasks
300
+ gateway environment create-task <dataset-id> --name n --prompt p
301
+ gateway environment performance <container-id>
302
+ gateway environment rollouts <container-id>
303
+ ```
304
+
305
+ MCP: `run_evaluation({ models | runConfigId, versionId?, numExamples?, rolloutsPerExample? })`,
306
+ `create_task_set({...})`, `create_task({...})`, `link_task_set({...})`, `list_rollouts({...})`,
307
+ `get_rollout_samples({...})`, `get_task_performance({...})`,
308
+ `promote_failure_to_scenario({...})` turns a failed rollout into a scenario.
309
+
310
+ ## 8. Iterate
311
+
312
+ ```bash
313
+ gateway bench pull x # reconstruct the head version's tree
314
+ gateway bench status ./x # branch, base version, local edits
315
+ gateway bench fork x --into x-experiment # a new container sharing content
316
+ gateway bench diff <base-version-id> <head-version-id> # per-file diff
317
+ gateway bench propose ./x --title t -m why # a change proposal instead of a push
318
+ gateway bench log x
319
+ ```
320
+
321
+ `worlds run --path` on a clean `bench pull` checkout files runs pinned to the base version;
322
+ a drifted directory runs unfiled. Push a new version when the change is meant to stick.
323
+
324
+ MCP: `env_diff({ containerId })`, `env_propose({ containerId, ... })`,
325
+ `env_list_proposals`, `env_get_proposal`, `env_apply_proposal`, `env_dismiss_proposal`,
326
+ `env_run_tests`, `env_get_run`.
327
+
328
+ ## 9. Where it shows in the UI
329
+
330
+ | Thing | Where |
331
+ | --- | --- |
332
+ | Connectors, shipped and workspace, with Create world / Clone; New connector | Connectors page `/project/{projectId}/connectors` |
333
+ | One connector: Overview (verdict + schema), API (served routes, tools), Worlds (built from it, Cancel/Retry) | `/project/{projectId}/connectors/{slug}` |
334
+ | A world: tabs Overview, Scenarios, Schema, Results, Evals, History | `/project/{projectId}/world?env={containerId}&tab=<tab>` |
335
+ | Schema tab: entities, tool schemas, the pin back to the connector | `...&tab=schema` |
336
+ | Versions, change reasons, proposals | `...&tab=history` |
337
+ | Runs and grades | `...&tab=results`, `...&tab=evals` |
338
+ | Sessions | `/project/{projectId}/world/sessions` |
339
+
340
+ ## CLI to MCP map
341
+
342
+ | CLI | MCP |
343
+ | --- | --- |
344
+ | `worlds connector list [--hub]` | `list_connectors` |
345
+ | `worlds connector pull <slug>` | `get_connector({ includeFiles: true })` |
346
+ | `worlds connector clone <slug>` | `clone_connector` |
347
+ | `worlds connector validate` / `schema check` | `validate_connector_template` |
348
+ | `worlds connector publish` | `publish_connector` |
349
+ | `worlds schema init --connector workspace:<slug>` | `init_world_from_connector` |
350
+ | `worlds create <slug> --from … [--overlay] [--rows] [--batch]` | `create_world` |
351
+ | `bench push` | `env_write_file` + `env_commit`, or `create_environment_version` (one file) |
352
+ | `worlds data import` | `import_world_data` |
353
+ | `worlds session seed` | `seed_world_session` |
354
+ | `worlds session export` | `export_world_session` |
355
+ | `environment run` / `worlds run` | `run_evaluation` |
356
+ | `environment link-task-set` / `create-task` | `create_task_set`, `create_task`, `link_task_set` |
357
+ | `bench diff` / `propose` | `env_diff`, `env_propose` |
358
+ | `worlds skill <name>` / `worlds connector skill` | `get_connector_authoring_skill`, `guide` |
359
+ | `worlds connector capture --remote` / poll + download | `start_connector_capture`, `get_connector_capture` (platform agent only) |
360
+
361
+ The platform (console) agent has every MCP tool in this table as an agent tool of the
362
+ same name, plus `run_gateway_cli` for the *local* CLI rows (`schema init|templates|compile|check|describe|api-bundle`,
363
+ `connector validate|conform|ingest|skill`, `data show|import|check|extract`, `skill`) inside its sandbox
364
+ — structured `{ command, args }` with this CLI's exact flags (the TypeScript `gateway`, run as
365
+ `node /tmp/gateway-cli/cli.js worlds …`), paths under `/workspace`, no remote flags, no platform
366
+ credentials — and `spawn_subagent` / `wait_for_subagent` to hand one world or connector
367
+ each to a tool-restricted child. Everything that writes to the platform goes through the
368
+ MCP-twin tools, gated by capabilities and the session's autonomy settings.
369
+
370
+ ## Gotchas
371
+
372
+ - `--connector workspace:<slug>` needs a login (`gateway auth login`) or the `GATEWAY_*` keys;
373
+ offline — including the agent's local-only `run_gateway_cli` — `schema init` refuses it and
374
+ names the command to use.
375
+ - `bench push` on a world with `connector.toml` packages an api-kind bundle; the root is
376
+ legacy-shaped on purpose. Edit the schema world, not the bundle.
377
+ - Sharing is publish + clone, never a live reference. A publisher's later edits reach a clone
378
+ only on re-clone.
379
+ - Never commit `captures/`; never merge a customer's world as an SDK template.
380
+ - A seed (scenario overlay or runner admin endpoint) is the whole state, not an increment.
381
+ - Never set `max_tokens` on any model call used to run or grade.
382
+
383
+ ## Related
384
+
385
+ - `world-data-ingestion`: rows in, calls out.
386
+ - `connector-schema-authoring`: the contract, fidelity shapes, conventions, definition of done.
387
+ - `documentation/features/connector-worlds.md`: `connector.toml` in full, capture, conform, serving.
388
+ - `documentation/features/workspace-connectors.md`: the connector object, sharing, MCP surface.
389
+ - `documentation/features/world-schema.md`: the `gateway-world/1` contract and store API.
390
+ - Consumer pages: `gateway_documentation/pages/worlds/{getting-started,data,custom-connections,slack}.mdx`.