@withgateway/sdk 0.1.20 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +141 -2
- package/dist/cli.js +8683 -1343
- package/dist/cli.js.map +1 -1
- package/dist/cli.mjs +8683 -1344
- package/dist/cli.mjs.map +1 -1
- package/dist/index.d.mts +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.js +979 -46
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +943 -38
- package/dist/index.mjs.map +1 -1
- package/dist/tracing/index.js +9 -2
- package/dist/tracing/index.js.map +1 -1
- package/dist/tracing/index.mjs +9 -2
- package/dist/tracing/index.mjs.map +1 -1
- package/dist/worlds/index.d.mts +738 -28
- package/dist/worlds/index.d.ts +738 -28
- package/dist/worlds/index.js +940 -42
- package/dist/worlds/index.js.map +1 -1
- package/dist/worlds/index.mjs +934 -36
- package/dist/worlds/index.mjs.map +1 -1
- package/package.json +9 -2
- package/runtime/gatewaysdk/__init__.py +0 -0
- package/runtime/gatewaysdk/environment/__init__.py +108 -0
- package/runtime/gatewaysdk/environment/_bundle.py +281 -0
- package/runtime/gatewaysdk/environment/_harbor.py +276 -0
- package/runtime/gatewaysdk/environment/_materialize.py +97 -0
- package/runtime/gatewaysdk/environment/_tar.py +33 -0
- package/runtime/gatewaysdk/environment/_world.py +346 -0
- package/runtime/gatewaysdk/environment/core.py +527 -0
- package/runtime/gatewaysdk/environment/errors.py +58 -0
- package/runtime/gatewaysdk/environment/runtime.py +150 -0
- package/runtime/gatewaysdk/environment/schema/__init__.py +35 -0
- package/runtime/gatewaysdk/environment/schema/__main__.py +225 -0
- package/runtime/gatewaysdk/environment/schema/_slack_fidelity.py +116 -0
- package/runtime/gatewaysdk/environment/schema/_slack_scenario.py +183 -0
- package/runtime/gatewaysdk/environment/schema/api.py +1628 -0
- package/runtime/gatewaysdk/environment/schema/batch.py +755 -0
- package/runtime/gatewaysdk/environment/schema/compiler.py +615 -0
- package/runtime/gatewaysdk/environment/schema/conform.py +428 -0
- package/runtime/gatewaysdk/environment/schema/connector.py +777 -0
- package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/handlers.py +17 -0
- package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/parity.json +28 -0
- package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/provenance.json +233 -0
- package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/scope.toml +112 -0
- package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/world.json +1380 -0
- package/runtime/gatewaysdk/environment/schema/connectors/base.json +28 -0
- package/runtime/gatewaysdk/environment/schema/connectors/custom/handlers.py +18 -0
- package/runtime/gatewaysdk/environment/schema/connectors/custom/world.json +123 -0
- package/runtime/gatewaysdk/environment/schema/connectors/github/handlers.py +81 -0
- package/runtime/gatewaysdk/environment/schema/connectors/github/parity.json +37 -0
- package/runtime/gatewaysdk/environment/schema/connectors/github/provenance.json +134 -0
- package/runtime/gatewaysdk/environment/schema/connectors/github/scope.toml +158 -0
- package/runtime/gatewaysdk/environment/schema/connectors/github/world.json +5869 -0
- package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/handlers.py +23 -0
- package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/world.json +209 -0
- package/runtime/gatewaysdk/environment/schema/connectors/google-drive/handlers.py +18 -0
- package/runtime/gatewaysdk/environment/schema/connectors/google-drive/world.json +286 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jamf/handlers.py +34 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jamf/parity.json +49 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jamf/provenance.json +181 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jamf/scope.toml +128 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jamf/world.json +2218 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jira/handlers.py +28 -0
- package/runtime/gatewaysdk/environment/schema/connectors/jira/world.json +389 -0
- package/runtime/gatewaysdk/environment/schema/connectors/kandji/handlers.py +38 -0
- package/runtime/gatewaysdk/environment/schema/connectors/kandji/parity.json +29 -0
- package/runtime/gatewaysdk/environment/schema/connectors/kandji/provenance.json +152 -0
- package/runtime/gatewaysdk/environment/schema/connectors/kandji/scope.toml +120 -0
- package/runtime/gatewaysdk/environment/schema/connectors/kandji/world.json +765 -0
- package/runtime/gatewaysdk/environment/schema/connectors/linear/handlers.py +20 -0
- package/runtime/gatewaysdk/environment/schema/connectors/linear/world.json +363 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/fidelity.json +45 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/handlers.py +1 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/provenance.json +851 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/scope.md +231 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/teams-types.json +1710 -0
- package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/world.json +858 -0
- package/runtime/gatewaysdk/environment/schema/connectors/netsuite/handlers.py +32 -0
- package/runtime/gatewaysdk/environment/schema/connectors/netsuite/parity.json +50 -0
- package/runtime/gatewaysdk/environment/schema/connectors/netsuite/provenance.json +137 -0
- package/runtime/gatewaysdk/environment/schema/connectors/netsuite/scope.toml +78 -0
- package/runtime/gatewaysdk/environment/schema/connectors/netsuite/world.json +16243 -0
- package/runtime/gatewaysdk/environment/schema/connectors/salesforce/handlers.py +806 -0
- package/runtime/gatewaysdk/environment/schema/connectors/salesforce/parity.json +46 -0
- package/runtime/gatewaysdk/environment/schema/connectors/salesforce/provenance.json +143 -0
- package/runtime/gatewaysdk/environment/schema/connectors/salesforce/scope.toml +174 -0
- package/runtime/gatewaysdk/environment/schema/connectors/salesforce/world.json +4695 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/MODEL.md +236 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/capabilities.json +1039 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/handlers.py +762 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/parity.json +65 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/provenance.json +294 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/scope.toml +44 -0
- package/runtime/gatewaysdk/environment/schema/connectors/slack/world.json +4993 -0
- package/runtime/gatewaysdk/environment/schema/connectors/workday/handlers.py +75 -0
- package/runtime/gatewaysdk/environment/schema/connectors/workday/parity.json +36 -0
- package/runtime/gatewaysdk/environment/schema/connectors/workday/provenance.json +179 -0
- package/runtime/gatewaysdk/environment/schema/connectors/workday/scope.toml +180 -0
- package/runtime/gatewaysdk/environment/schema/connectors/workday/world.json +806 -0
- package/runtime/gatewaysdk/environment/schema/data.py +593 -0
- package/runtime/gatewaysdk/environment/schema/extract.py +263 -0
- package/runtime/gatewaysdk/environment/schema/host.py +2407 -0
- package/runtime/gatewaysdk/environment/schema/host_surfaces.py +275 -0
- package/runtime/gatewaysdk/environment/schema/platform.py +1272 -0
- package/runtime/gatewaysdk/environment/schema/scaffold.py +250 -0
- package/runtime/gatewaysdk/environment/schema/skill.py +118 -0
- package/runtime/gatewaysdk/environment/schema/skills/agents/connector-template-author.md +202 -0
- package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONNECTORS.md +331 -0
- package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
- package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +313 -0
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +223 -0
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +293 -0
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +390 -0
- package/runtime/gatewaysdk/environment/schema/slack.py +757 -0
- package/runtime/gatewaysdk/environment/schema/snapshot.py +479 -0
- package/runtime/gatewaysdk/environment/schema/store.py +904 -0
- package/runtime/gatewaysdk/environment/schema/tasks.py +525 -0
- package/runtime/gatewaysdk/environment/schema/tools_world.py +322 -0
- package/runtime/gatewaysdk/environment/schema/validation.py +577 -0
- package/runtime/gatewaysdk/environment/schema/worker.py +78 -0
- package/runtime/gatewaysdk/environment/schema/workers.py +500 -0
- package/runtime/gatewaysdk/environment/schema/world_tests.py +336 -0
- package/skills/agents/connector-template-author.md +202 -0
- package/skills/connector-schema-authoring/CONNECTORS.md +331 -0
- package/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
- package/skills/connector-schema-authoring/SKILL.md +313 -0
- package/skills/world-data-ingestion/ROWS.md +223 -0
- package/skills/world-data-ingestion/SKILL.md +293 -0
- package/skills/worlds-getting-started/SKILL.md +390 -0
|
@@ -0,0 +1,390 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: worlds-getting-started
|
|
3
|
+
description: Starter guide for building worlds and environments on the platform, end to end, for an agent (Claude Code, Cursor, Codex). Use when starting a world from scratch, building a world or environment, mocking a vendor, choosing between a shipped template, a workspace connector, a vendor API capture or a custom contract, provisioning worlds for tenants in one call (worlds create), publishing with bench push, opening a session, running and grading, or iterating with pull, fork, diff and propose. Every CLI step names its MCP twin (list_connectors, publish_connector, create_world, init_world_from_connector, env_write_file, env_commit, import_world_data, run_evaluation and the rest) and where the result shows in the UI.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Worlds: getting started
|
|
7
|
+
|
|
8
|
+
> **Two copies, kept identical.** This file ships inside the `gatewaysdk` Python package
|
|
9
|
+
> (`gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md`) and the `@withgateway/sdk`
|
|
10
|
+
> npm package (copied into `skills/` at build) and is what
|
|
11
|
+
> `gateway worlds skill worlds-getting-started` prints and `--install DIR` writes into a
|
|
12
|
+
> repository's `.claude/skills/`. The repository copy is `.claude/skills/worlds-getting-started/SKILL.md`.
|
|
13
|
+
> Edit one, copy to the other; a test compares them.
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
> **Two copies, kept identical.** This file ships inside the `gatewaysdk` Python package
|
|
17
|
+
> (`gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md`) and the `@withgateway/sdk`
|
|
18
|
+
> npm package (copied into `skills/` at build) and is what
|
|
19
|
+
> `gateway worlds skill worlds-getting-started` prints and `--install DIR` writes into a
|
|
20
|
+
> repository's `.claude/skills/`. The repository copy is `.claude/skills/worlds-getting-started/SKILL.md`.
|
|
21
|
+
> Edit one, copy to the other; a test compares them.
|
|
22
|
+
|
|
23
|
+
## Purpose
|
|
24
|
+
|
|
25
|
+
The shortest correct path from nothing to a hosted world with data, a session, a graded
|
|
26
|
+
run and a next version. CLI first; the MCP twin sits under each step for an agent that
|
|
27
|
+
talks to the platform over MCP instead of a shell. Deeper references are linked by path,
|
|
28
|
+
not repeated.
|
|
29
|
+
|
|
30
|
+
Vocabulary (from `gateway_documentation/pages/glossary.mdx`): **world** (the sealed
|
|
31
|
+
replica), **scenario** (a task in it), **rollout** (one attempt), **run** (a filed set of
|
|
32
|
+
rollouts), **eval** (the grade), **gate** (the bar). Identifiers keep the old words:
|
|
33
|
+
`environment`, `container`, `task set`, `bench`.
|
|
34
|
+
|
|
35
|
+
## 0. Authenticate
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npm install -g @withgateway/sdk # the gateway CLI (TypeScript; no Python needed)
|
|
39
|
+
gateway auth login --host https://<your-gateway> --public-key pk-lf-... --api-key sk-lf-...
|
|
40
|
+
gateway auth status # or export GATEWAY_HOST / GATEWAY_PUBLIC_KEY / GATEWAY_SECRET_KEY
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
The npm package bundles the schema runtime (stdlib-only Python). With a `python3 >= 3.11` on the
|
|
44
|
+
machine every runtime command runs locally, including `worlds serve` and a local `connector
|
|
45
|
+
capture`, with no pip and no login; without one, the CLI sends the world's files to the platform
|
|
46
|
+
(`POST /api/public/worlds/runtime`) and the same code runs there (`worlds serve` then opens a hosted
|
|
47
|
+
api session, `connector capture` needs `--remote`). `GATEWAY_RUNTIME=local|platform` forces a side.
|
|
48
|
+
|
|
49
|
+
Or export `GATEWAY_HOST`, `GATEWAY_PUBLIC_KEY`, `GATEWAY_SECRET_KEY` (env wins over the
|
|
50
|
+
config file). Every command below accepts `--host`. Keys are project keys: the world lands
|
|
51
|
+
in that project.
|
|
52
|
+
|
|
53
|
+
MCP: the server is authenticated by the same project API key (or a PAT). Call
|
|
54
|
+
`describe_project({})` first, then `guide({ topic: "connectors" })` for the connector loop
|
|
55
|
+
and `guide({ topic: "world-data" })` for data.
|
|
56
|
+
|
|
57
|
+
## 1. Pick a path
|
|
58
|
+
|
|
59
|
+
| You have | Path | First command |
|
|
60
|
+
| --- | --- | --- |
|
|
61
|
+
| A vendor the SDK ships a template for (`gateway worlds schema templates`: github, slack, salesforce, jira, linear, workday, netsuite, microsoft-teams, google-drive, google-calendar, jamf, kandji, apple-business-manager, constella) | shipped template | `gateway worlds schema init ./x --connector github` |
|
|
62
|
+
| A connector someone in your org (or any org) already published | workspace connector | `gateway worlds connector list --hub` then `schema init ./x --connector workspace:<slug>` |
|
|
63
|
+
| A vendor API with a dev key and no template | vendor API capture | `schema init ./x --connector custom`, write `connector.toml`, capture, ingest (`documentation/features/connector-worlds.md`) |
|
|
64
|
+
| A data model and nothing else | custom contract | `schema init ./x --connector custom`, author `schema/world.json` (`connector-schema-authoring`) |
|
|
65
|
+
| SQL tables and Python tool handlers | mock-tools world | `gateway-env.toml` + `db/schema.sql` + `db/seed.sql` + `tools/`; no schema contract |
|
|
66
|
+
| A Harbor task suite | harbor world | the task dirs as they are; `bench push` |
|
|
67
|
+
| A tenant to provision for, no local edit loop (a template, a hub connector, optionally your overlay and their rows) | one-call create | `gateway worlds create acme-slack --from slack --rows rows.json`; see §5 |
|
|
68
|
+
|
|
69
|
+
Rule from the owner: a world is read and write 1:1. Reads from the store alone are a lookup
|
|
70
|
+
mock; a vendor with workflows needs `action = "create|update|delete|handler"` routes and
|
|
71
|
+
`api/handlers.py`. Do not ship a read-only world and call it done.
|
|
72
|
+
|
|
73
|
+
MCP: `list_connectors({ query? })` (own + public), `get_connector({ slug, includeFiles: true })`
|
|
74
|
+
to read a template, `clone_connector({ slug })` to take a public one into this org.
|
|
75
|
+
|
|
76
|
+
## 2. Author the contract
|
|
77
|
+
|
|
78
|
+
Layout of a schema world after `init`:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
x/
|
|
82
|
+
schema/world.json # gateway-world/1: entities, fields, keys, relationships (the deliverable)
|
|
83
|
+
schema/provenance.json # where the model came from, pinned
|
|
84
|
+
connector.toml # vendor connection + the routes the served world answers (optional)
|
|
85
|
+
tools/handlers.py # one handler per tool in x-gateway.tools (optional)
|
|
86
|
+
api/handlers.py # workflow handlers for action = "handler" routes (optional)
|
|
87
|
+
scenarios/<name>/ # scenario.toml + seed/*.json overlays + instruction.md
|
|
88
|
+
data/initial.json # the seed batch (written by ingest / data import)
|
|
89
|
+
captures/ # raw vendor pages; never committed to a template
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Author `schema/world.json` per `connector-schema-authoring` (conventions, prefixes, keys).
|
|
93
|
+
Write `connector.toml` per `documentation/features/connector-worlds.md` ("The spec, in
|
|
94
|
+
full"). Both have scaffolds so the declaration shape cannot be wrong; only payload
|
|
95
|
+
definitions and handler bodies are hand-written:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
gateway worlds schema entity add ./x crm_accounts --fields "id:string,name:string,tier:string?" --key id
|
|
99
|
+
gateway worlds schema entity add ./x crm_contacts --ref CrmContact --key id --relationship account_id->crm_accounts
|
|
100
|
+
gateway worlds connector route add ./x list_accounts --path /v1/accounts --param page --entity crm_accounts --results data
|
|
101
|
+
gateway worlds connector route add ./x get_account --path /v1/accounts/{id} --entity crm_accounts --filter id=id --single
|
|
102
|
+
gateway worlds connector route add ./x create_account -X POST --path /v1/accounts --entity crm_accounts --action create --body name=name
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Each refuses (and writes nothing) on a duplicate name, an undeclared entity or `$defs`
|
|
106
|
+
reference, a key that is not a required string/integer field, or a filter on an unknown
|
|
107
|
+
argument. Handlers: `def name(ctx, args, query, body)`; `ctx` reads and writes the store and
|
|
108
|
+
keeps the clock.
|
|
109
|
+
|
|
110
|
+
MCP: `get_connector_authoring_skill({})` returns the authoring skill and conventions.
|
|
111
|
+
Author files locally or with `env_write_file({ containerId, path, newContent })` on an existing
|
|
112
|
+
environment; land them with `env_commit({ containerId, changeReason })` (one version for the batch).
|
|
113
|
+
|
|
114
|
+
## 3. Check
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
gateway worlds schema compile ./x
|
|
118
|
+
gateway worlds schema check ./x
|
|
119
|
+
gateway worlds connector validate ./x # only when connector.toml exists
|
|
120
|
+
gateway worlds connector conform ./x --openapi vendor.yaml # only when the vendor publishes OpenAPI
|
|
121
|
+
gateway worlds serve ./x --port 8080 # answers the vendor's routes locally
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`check` validates the compiled bundle; `conform` drives every declared route against the
|
|
125
|
+
vendor's document and exits non-zero on a mismatch. Its FAIL list is the work list.
|
|
126
|
+
|
|
127
|
+
MCP: `validate_connector_template({ files, slug? })` compiles without writing and answers
|
|
128
|
+
`{ valid, reason? }`.
|
|
129
|
+
|
|
130
|
+
## 4. Data
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
gateway worlds data show ./x # kind + entity counts
|
|
134
|
+
gateway worlds data import ./x rows.json # rows file: {entity: [rows]} | JSONL | --entity
|
|
135
|
+
gateway worlds connector capture ./x --op <name> --arg k=v # from the real vendor, redacted
|
|
136
|
+
gateway worlds connector ingest ./x # captures -> data/initial.json
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Rows file contract, modes, upsert and refusals: `world-data-ingestion`. Captures: pace
|
|
140
|
+
with `[capture] requests_per_minute`; a vendor that allow-lists IPs is captured with
|
|
141
|
+
`--remote` after `gateway bench secrets set --env x --key VENDOR_API_KEY`.
|
|
142
|
+
|
|
143
|
+
MCP: `import_world_data({ containerId | slug, rows, mode?, message? })` on a pushed world.
|
|
144
|
+
|
|
145
|
+
## 5. Publish
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
gateway bench push ./x -m "first cut" # a world version; api bundle packaged when connector.toml exists
|
|
149
|
+
gateway worlds connector publish ./x --slug x -m "first cut" # the template as a workspace connector (org-owned, versioned)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
`bench push` makes the version sessions and runs address (`slug@version`). `connector publish`
|
|
153
|
+
saves the contract (never `data/`, `captures/`, `db/`) so anyone in the org can
|
|
154
|
+
`schema init --connector workspace:x`. Both, for a world you expect to rebuild.
|
|
155
|
+
|
|
156
|
+
MCP: `publish_connector({ slug, files, message? })` is the template write path.
|
|
157
|
+
`init_world_from_connector({ slug, name })` makes the world under a PAT session; under an API
|
|
158
|
+
key it returns the initialized files plus the publish steps (`env_write_file` per file, then
|
|
159
|
+
`env_commit`, or `create_environment_version({ containerId, path, newContent, changeReason })`
|
|
160
|
+
for one file).
|
|
161
|
+
|
|
162
|
+
### One call, no local tree: `worlds create`
|
|
163
|
+
|
|
164
|
+
For a world you provision rather than iterate on (a tenant's Slack, one per customer):
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
gateway worlds create acme-slack --from slack --rows rows.json --name "Acme Slack" # shipped template + their rows
|
|
168
|
+
gateway worlds create acme-crm --from workspace:acme-crm --overlay ./overlay --json # hub connector + your edits
|
|
169
|
+
gateway worlds create acme-custom --from ./x # a directory, any kind
|
|
170
|
+
gateway worlds create --batch worlds.jsonl # {slug, from, overlay?, data?, name?} per line; exit 2 if any failed
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
One request (`POST /api/public/worlds`, `documentation/api/rest/worlds-create.md`): the source,
|
|
174
|
+
an optional overlay laid over it, optional rows through the world's contract. An overlay or a
|
|
175
|
+
directory is compiled by the schema runtime before anything is stored, so a broken edit is a
|
|
176
|
+
400 naming the path, not a broken world. The result is what `bench push` lands: a container
|
|
177
|
+
and a READY first version, attributed to the key (`apikey:<id>`), owned by nobody. 409 when
|
|
178
|
+
the slug is taken in the project.
|
|
179
|
+
|
|
180
|
+
MCP: `create_world({ slug, from: { connector } | { files }, overlay?, data?, name?, message? })`
|
|
181
|
+
is the same request and answer. `init_world_from_connector` stays for the PAT flow that wants a
|
|
182
|
+
human owner.
|
|
183
|
+
|
|
184
|
+
## 6. Open a session
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
gateway worlds session open x -t default # tools surface; prints {sessionId, surfaces}
|
|
188
|
+
gateway worlds session open x -t default --surface api # adds api.url: the vendor's routes, hosted
|
|
189
|
+
gateway worlds session open x -t default --no-wait # return at once on a cold version
|
|
190
|
+
gateway worlds session open x --task-file task.json --surface api # the session's own task (below)
|
|
191
|
+
gateway worlds session status <SESSION_ID> # poll until ready
|
|
192
|
+
gateway worlds session seed <SESSION_ID> rows.json # rows into this session only
|
|
193
|
+
gateway worlds session state <SESSION_ID> # every entity's rows, the dump a grader reads
|
|
194
|
+
gateway worlds session reset <SESSION_ID> # back to the snapshot (+ the task's seed)
|
|
195
|
+
gateway worlds session grade <SESSION_ID> [--timeout S] # {reward, rewards, raw}; a rubric judge may take a while
|
|
196
|
+
gateway worlds session export <SESSION_ID> --out calls.jsonl
|
|
197
|
+
gateway worlds session close <SESSION_ID>
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
**A task outside the bundle.** `-t <name>` picks a scenario the version ships. On a World Host
|
|
201
|
+
world (every schema / handler / mock-tools world once the host is configured; a pod world
|
|
202
|
+
answers 400) `--task-file task.json` supplies the task with the open instead — one published
|
|
203
|
+
world, a task per end user, no `bench push` per goal:
|
|
204
|
+
|
|
205
|
+
```json
|
|
206
|
+
{ "name": "acme-q3", "instruction": "Mark every open Q3 invoice for Acme paid.",
|
|
207
|
+
"seed": { "rows": { "invoices": [ { "id": "INV-7", "status": "open" } ] } },
|
|
208
|
+
"grader": { "kind": "assertions", "checks": [
|
|
209
|
+
{ "id": "paid", "entity": "invoices", "where": { "id": "INV-7", "status": "paid" } } ] },
|
|
210
|
+
"metadata": { "externalUserId": "u_42" } }
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
`instruction` is what the agent is told; `seed` (`{mode?, rows}`, the live-seed contract) is
|
|
214
|
+
laid on the clone and re-applied by `reset`; `grader` is `assertions` (`entity`, `where`,
|
|
215
|
+
`exists|absent|count|all|any`, `field op value`; each check's `id` becomes a `rewards` key),
|
|
216
|
+
`{kind: "python", source}` (a `verify.py` body) or `{kind: "rubric", rubric, criteria?, model?}`
|
|
217
|
+
(an LLM judge over end state + call log, graded on the platform: `grade` answers pending until
|
|
218
|
+
the worker finalizes it); `tools: [names]` narrows the operations the session may call to
|
|
219
|
+
exactly those (a name the world does not serve is a 400); `metadata` rides the session. Exactly one of
|
|
220
|
+
`-t` / `--task-id` / `--task-file`. The output's `task` block is `{source: "inline", name, hash, metadata}`.
|
|
221
|
+
SDK: `openSession(slug, { task: spec })` (TS), `open_session(slug, spec)` (Python);
|
|
222
|
+
`runSessions` / `run_sessions` take a mixed list of names and specs.
|
|
223
|
+
|
|
224
|
+
**A stored task.** The same spec kept as a project resource, versioned by content, so many
|
|
225
|
+
sessions open it by id and grades attribute to the task and its version:
|
|
226
|
+
|
|
227
|
+
```bash
|
|
228
|
+
gateway worlds task create task.json --world x # version 1; --name overrides the spec's name
|
|
229
|
+
gateway worlds task list --world x --metadata externalUserId=u_42
|
|
230
|
+
gateway worlds task show <TASK_ID> [--version 2] # spec + version history
|
|
231
|
+
gateway worlds task validate <TASK_ID> # seed/grader entities against the pinned world's version
|
|
232
|
+
gateway worlds task update <TASK_ID> --spec task2.json # a changed spec is the next version; a rename is not
|
|
233
|
+
gateway worlds session open x --task-id <TASK_ID> [--task-version 1] --surface api
|
|
234
|
+
gateway worlds session task save <SESSION_ID> --name acme-q3 # promote a session's inline task into a stored one
|
|
235
|
+
gateway worlds task delete <TASK_ID>
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
SDK: `createTask / listTasks / getTask / updateTask / validateTask / deleteTask`,
|
|
239
|
+
`openSession(slug, { task: { id, version? } })`, `session.saveTask(name)` (TS);
|
|
240
|
+
`create_task` … `delete_task`, `open_session(slug, {"id": …})`, `session.save_task(name)`
|
|
241
|
+
(Python). MCP: `create_world_task`, `update_world_task`, `list_world_tasks`, `get_world_task`,
|
|
242
|
+
`validate_world_task`, `delete_world_task`, `save_world_session_task`. The world's Scenarios
|
|
243
|
+
tab lists the tasks pinned to it under a `world` kind chip.
|
|
244
|
+
|
|
245
|
+
**Stored tasks as a hosted eval.** A run config dispatches EITHER the bundle's scenarios
|
|
246
|
+
(`--tasks`) OR stored tasks (`--world-tasks`), never both:
|
|
247
|
+
|
|
248
|
+
```bash
|
|
249
|
+
gateway bench run-configs create --name nightly --models openai/gpt-5-mini --world-tasks <TASK_ID>,<OTHER_ID>@2
|
|
250
|
+
gateway bench dispatch <RUN_CONFIG_ID> # each stored task is a sample; rubric tasks are refused at dispatch
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
`<id>` runs the task's current version at dispatch time, `<id>@N` that version forever. SDK:
|
|
254
|
+
`upsertRunConfig({ name, models, worldTasks: [{ id }, { id, version: 2 }] })` (TS),
|
|
255
|
+
`upsert_run_config(name=…, models=…, worldTasks=[…])` (Python); `parseWorldTaskRefs` /
|
|
256
|
+
`parse_world_task_refs` turn the `--world-tasks` spelling into refs.
|
|
257
|
+
|
|
258
|
+
The first session on a freshly pushed version takes about fifteen minutes: the pod is
|
|
259
|
+
scheduled and the runner sidecar pulled; the session reads `PROVISIONING` and the api URL
|
|
260
|
+
answers 503. That is a cold start, not a fault, until about twenty minutes have passed.
|
|
261
|
+
`session open` waits ten minutes by default; on a cold version use `--no-wait` and poll.
|
|
262
|
+
|
|
263
|
+
The world's API honours the vendor's auth scheme; `WORLD_API_KEY` on the host pins the
|
|
264
|
+
credential, otherwise any non-empty one is accepted.
|
|
265
|
+
|
|
266
|
+
MCP: `seed_world_session({ sessionId, rows, mode? })`, `export_world_session({ sessionId })`.
|
|
267
|
+
Opening a session has no MCP tool; use the CLI or the public sessions route.
|
|
268
|
+
|
|
269
|
+
## Test the world (not the agent)
|
|
270
|
+
|
|
271
|
+
A world ships its own tests under `tests/` and they run on every push. Tests assert the WORLD
|
|
272
|
+
answers as you say it must; evals (below) grade an agent on a task. Keep them apart.
|
|
273
|
+
|
|
274
|
+
```bash
|
|
275
|
+
gateway worlds test ./my-world # tests/*.json HTTP cases + tests/test_*.py, served locally
|
|
276
|
+
gateway worlds test my-world # the same suite on the platform, against the pushed head
|
|
277
|
+
gateway bench tests my-world # every run the platform kept (push, CLI, API)
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
A case: `{"name": "...", "request": {"method": "GET", "path": "/v1/accounts", "auth": true},
|
|
281
|
+
"expect": {"status": 200, "length": {"path": "$", "min": 1}}}`. Put `[actions]` `on_push = ["tests"]`
|
|
282
|
+
in `gateway-env.toml` and the platform runs the suite on each push (world page → Tests tab). Public
|
|
283
|
+
API: `POST/GET /api/public/worlds/{slug}/tests`. Details: `documentation/features/world-tests.md`.
|
|
284
|
+
|
|
285
|
+
## 7. Run and grade
|
|
286
|
+
|
|
287
|
+
```bash
|
|
288
|
+
gateway worlds run x -a agent.py:agent -m <model> # pull, run the callable against the world, file a run
|
|
289
|
+
gateway worlds run --path ./x -a agent.py:agent --no-file # local, unfiled
|
|
290
|
+
gateway worlds ci --path ./x -m <model> --fail-under 0.8 # push the working tree, run, gate (CI)
|
|
291
|
+
gateway environment run x -m <provider/model> # hosted runner, one job per model
|
|
292
|
+
gateway environment run-status <run-id>
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
Scenarios (task sets):
|
|
296
|
+
|
|
297
|
+
```bash
|
|
298
|
+
gateway environment link-task-set x --tasks tasks.jsonl # rows: id, prompt, metadata, worlds; idempotent
|
|
299
|
+
gateway environment tasks <container-id> # the linked set and its tasks
|
|
300
|
+
gateway environment create-task <dataset-id> --name n --prompt p
|
|
301
|
+
gateway environment performance <container-id>
|
|
302
|
+
gateway environment rollouts <container-id>
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
MCP: `run_evaluation({ models | runConfigId, versionId?, numExamples?, rolloutsPerExample? })`,
|
|
306
|
+
`create_task_set({...})`, `create_task({...})`, `link_task_set({...})`, `list_rollouts({...})`,
|
|
307
|
+
`get_rollout_samples({...})`, `get_task_performance({...})`,
|
|
308
|
+
`promote_failure_to_scenario({...})` turns a failed rollout into a scenario.
|
|
309
|
+
|
|
310
|
+
## 8. Iterate
|
|
311
|
+
|
|
312
|
+
```bash
|
|
313
|
+
gateway bench pull x # reconstruct the head version's tree
|
|
314
|
+
gateway bench status ./x # branch, base version, local edits
|
|
315
|
+
gateway bench fork x --into x-experiment # a new container sharing content
|
|
316
|
+
gateway bench diff <base-version-id> <head-version-id> # per-file diff
|
|
317
|
+
gateway bench propose ./x --title t -m why # a change proposal instead of a push
|
|
318
|
+
gateway bench log x
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
`worlds run --path` on a clean `bench pull` checkout files runs pinned to the base version;
|
|
322
|
+
a drifted directory runs unfiled. Push a new version when the change is meant to stick.
|
|
323
|
+
|
|
324
|
+
MCP: `env_diff({ containerId })`, `env_propose({ containerId, ... })`,
|
|
325
|
+
`env_list_proposals`, `env_get_proposal`, `env_apply_proposal`, `env_dismiss_proposal`,
|
|
326
|
+
`env_run_tests`, `env_get_run`.
|
|
327
|
+
|
|
328
|
+
## 9. Where it shows in the UI
|
|
329
|
+
|
|
330
|
+
| Thing | Where |
|
|
331
|
+
| --- | --- |
|
|
332
|
+
| Connectors, shipped and workspace, with Create world / Clone; New connector | Connectors page `/project/{projectId}/connectors` |
|
|
333
|
+
| One connector: Overview (verdict + schema), API (served routes, tools), Worlds (built from it, Cancel/Retry) | `/project/{projectId}/connectors/{slug}` |
|
|
334
|
+
| A world: tabs Overview, Scenarios, Schema, Results, Evals, History | `/project/{projectId}/world?env={containerId}&tab=<tab>` |
|
|
335
|
+
| Schema tab: entities, tool schemas, the pin back to the connector | `...&tab=schema` |
|
|
336
|
+
| Versions, change reasons, proposals | `...&tab=history` |
|
|
337
|
+
| Runs and grades | `...&tab=results`, `...&tab=evals` |
|
|
338
|
+
| Sessions | `/project/{projectId}/world/sessions` |
|
|
339
|
+
|
|
340
|
+
## CLI to MCP map
|
|
341
|
+
|
|
342
|
+
| CLI | MCP |
|
|
343
|
+
| --- | --- |
|
|
344
|
+
| `worlds connector list [--hub]` | `list_connectors` |
|
|
345
|
+
| `worlds connector pull <slug>` | `get_connector({ includeFiles: true })` |
|
|
346
|
+
| `worlds connector clone <slug>` | `clone_connector` |
|
|
347
|
+
| `worlds connector validate` / `schema check` | `validate_connector_template` |
|
|
348
|
+
| `worlds connector publish` | `publish_connector` |
|
|
349
|
+
| `worlds schema init --connector workspace:<slug>` | `init_world_from_connector` |
|
|
350
|
+
| `worlds create <slug> --from … [--overlay] [--rows] [--batch]` | `create_world` |
|
|
351
|
+
| `bench push` | `env_write_file` + `env_commit`, or `create_environment_version` (one file) |
|
|
352
|
+
| `worlds data import` | `import_world_data` |
|
|
353
|
+
| `worlds session seed` | `seed_world_session` |
|
|
354
|
+
| `worlds session export` | `export_world_session` |
|
|
355
|
+
| `environment run` / `worlds run` | `run_evaluation` |
|
|
356
|
+
| `environment link-task-set` / `create-task` | `create_task_set`, `create_task`, `link_task_set` |
|
|
357
|
+
| `bench diff` / `propose` | `env_diff`, `env_propose` |
|
|
358
|
+
| `worlds skill <name>` / `worlds connector skill` | `get_connector_authoring_skill`, `guide` |
|
|
359
|
+
| `worlds connector capture --remote` / poll + download | `start_connector_capture`, `get_connector_capture` (platform agent only) |
|
|
360
|
+
|
|
361
|
+
The platform (console) agent has every MCP tool in this table as an agent tool of the
|
|
362
|
+
same name, plus `run_gateway_cli` for the *local* CLI rows (`schema init|templates|compile|check|describe|api-bundle`,
|
|
363
|
+
`connector validate|conform|ingest|skill`, `data show|import|check|extract`, `skill`) inside its sandbox
|
|
364
|
+
— structured `{ command, args }` with this CLI's exact flags (the TypeScript `gateway`, run as
|
|
365
|
+
`node /tmp/gateway-cli/cli.js worlds …`), paths under `/workspace`, no remote flags, no platform
|
|
366
|
+
credentials — and `spawn_subagent` / `wait_for_subagent` to hand one world or connector
|
|
367
|
+
each to a tool-restricted child. Everything that writes to the platform goes through the
|
|
368
|
+
MCP-twin tools, gated by capabilities and the session's autonomy settings.
|
|
369
|
+
|
|
370
|
+
## Gotchas
|
|
371
|
+
|
|
372
|
+
- `--connector workspace:<slug>` needs a login (`gateway auth login`) or the `GATEWAY_*` keys;
|
|
373
|
+
offline — including the agent's local-only `run_gateway_cli` — `schema init` refuses it and
|
|
374
|
+
names the command to use.
|
|
375
|
+
- `bench push` on a world with `connector.toml` packages an api-kind bundle; the root is
|
|
376
|
+
legacy-shaped on purpose. Edit the schema world, not the bundle.
|
|
377
|
+
- Sharing is publish + clone, never a live reference. A publisher's later edits reach a clone
|
|
378
|
+
only on re-clone.
|
|
379
|
+
- Never commit `captures/`; never merge a customer's world as an SDK template.
|
|
380
|
+
- A seed (scenario overlay or runner admin endpoint) is the whole state, not an increment.
|
|
381
|
+
- Never set `max_tokens` on any model call used to run or grade.
|
|
382
|
+
|
|
383
|
+
## Related
|
|
384
|
+
|
|
385
|
+
- `world-data-ingestion`: rows in, calls out.
|
|
386
|
+
- `connector-schema-authoring`: the contract, fidelity shapes, conventions, definition of done.
|
|
387
|
+
- `documentation/features/connector-worlds.md`: `connector.toml` in full, capture, conform, serving.
|
|
388
|
+
- `documentation/features/workspace-connectors.md`: the connector object, sharing, MCP surface.
|
|
389
|
+
- `documentation/features/world-schema.md`: the `gateway-world/1` contract and store API.
|
|
390
|
+
- Consumer pages: `gateway_documentation/pages/worlds/{getting-started,data,custom-connections,slack}.mdx`.
|