@withgateway/sdk 0.2.6 → 0.2.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -8
- package/dist/cli.js +15468 -11263
- package/dist/cli.js.map +1 -1
- package/dist/cli.mjs +15233 -11028
- package/dist/cli.mjs.map +1 -1
- package/dist/files/index.d.mts +1 -1
- package/dist/files/index.d.ts +1 -1
- package/dist/files/index.js +291 -40
- package/dist/files/index.js.map +1 -1
- package/dist/files/index.mjs +284 -37
- package/dist/files/index.mjs.map +1 -1
- package/dist/hooks/index.d.mts +1 -1
- package/dist/hooks/index.d.ts +1 -1
- package/dist/hooks/index.js +203 -47
- package/dist/hooks/index.js.map +1 -1
- package/dist/hooks/index.mjs +198 -42
- package/dist/hooks/index.mjs.map +1 -1
- package/dist/{index-BaKu5ZSU.d.mts → index-C3UzD0G3.d.mts} +75 -1
- package/dist/{index-BaKu5ZSU.d.ts → index-C3UzD0G3.d.ts} +75 -1
- package/dist/{index-CLkMr6VG.d.ts → index-D-_4Ibcn.d.ts} +36 -4
- package/dist/{index-CdbmO4aX.d.mts → index-FfK4Nump.d.mts} +36 -4
- package/dist/index.d.mts +3 -3
- package/dist/index.d.ts +3 -3
- package/dist/index.js +2149 -615
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +2053 -566
- package/dist/index.mjs.map +1 -1
- package/dist/traces/index.js +268 -31
- package/dist/traces/index.js.map +1 -1
- package/dist/traces/index.mjs +262 -28
- package/dist/traces/index.mjs.map +1 -1
- package/dist/tracing/index.js +19 -11
- package/dist/tracing/index.js.map +1 -1
- package/dist/tracing/index.mjs +19 -11
- package/dist/tracing/index.mjs.map +1 -1
- package/dist/worlds/index.d.mts +886 -117
- package/dist/worlds/index.d.ts +886 -117
- package/dist/worlds/index.js +1770 -452
- package/dist/worlds/index.js.map +1 -1
- package/dist/worlds/index.mjs +1752 -434
- package/dist/worlds/index.mjs.map +1 -1
- package/package.json +14 -10
- package/runtime/gatewaysdk/environment/_toml.py +3 -10
- package/runtime/gatewaysdk/environment/_world.py +95 -28
- package/runtime/gatewaysdk/environment/runtime.py +10 -6
- package/runtime/gatewaysdk/environment/schema/__main__.py +35 -0
- package/runtime/gatewaysdk/environment/schema/api.py +19 -23
- package/runtime/gatewaysdk/environment/schema/batch.py +68 -14
- package/runtime/gatewaysdk/environment/schema/bundle_runtime.py +4 -2
- package/runtime/gatewaysdk/environment/schema/cast.py +264 -0
- package/runtime/gatewaysdk/environment/schema/compiler.py +35 -1
- package/runtime/gatewaysdk/environment/schema/conform.py +28 -10
- package/runtime/gatewaysdk/environment/schema/connector.py +59 -14
- package/runtime/gatewaysdk/environment/schema/connectors/custom/handlers.py +38 -1
- package/runtime/gatewaysdk/environment/schema/connectors/google-drive/readers/merge.toml +3 -1
- package/runtime/gatewaysdk/environment/schema/credentials.py +20 -0
- package/runtime/gatewaysdk/environment/schema/csv_rows.py +117 -0
- package/runtime/gatewaysdk/environment/schema/cycles.py +121 -0
- package/runtime/gatewaysdk/environment/schema/data.py +21 -5
- package/runtime/gatewaysdk/environment/schema/grading.py +95 -13
- package/runtime/gatewaysdk/environment/schema/handler_errors.py +44 -0
- package/runtime/gatewaysdk/environment/schema/handler_scan.py +175 -0
- package/runtime/gatewaysdk/environment/schema/host.py +68 -115
- package/runtime/gatewaysdk/environment/schema/host_requests.py +78 -0
- package/runtime/gatewaysdk/environment/schema/host_surfaces.py +12 -0
- package/runtime/gatewaysdk/environment/schema/ingest.py +5 -18
- package/runtime/gatewaysdk/environment/schema/local.py +7 -10
- package/runtime/gatewaysdk/environment/schema/openapi_document.py +322 -0
- package/runtime/gatewaysdk/environment/schema/openapi_import.py +382 -0
- package/runtime/gatewaysdk/environment/schema/openapi_routes.py +441 -0
- package/runtime/gatewaysdk/environment/schema/openapi_schemas.py +629 -0
- package/runtime/gatewaysdk/environment/schema/openapi_source.py +210 -0
- package/runtime/gatewaysdk/environment/schema/platform.py +110 -299
- package/runtime/gatewaysdk/environment/schema/platform_import.py +182 -0
- package/runtime/gatewaysdk/environment/schema/platform_query.py +95 -0
- package/runtime/gatewaysdk/environment/schema/route_bindings.py +176 -0
- package/runtime/gatewaysdk/environment/schema/runtime.py +56 -7
- package/runtime/gatewaysdk/environment/schema/scaffold.py +64 -5
- package/runtime/gatewaysdk/environment/schema/skills/benchmarks-getting-started/SKILL.md +70 -22
- package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +54 -4
- package/runtime/gatewaysdk/environment/schema/skills/hooks-getting-started/SKILL.md +27 -2
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +62 -0
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +91 -54
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +146 -146
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/cli-mcp-map.md +11 -7
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/gotchas.md +30 -0
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/new-world-sandbox.md +43 -0
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/openapi-import.md +90 -0
- package/runtime/gatewaysdk/environment/schema/store.py +15 -6
- package/runtime/gatewaysdk/environment/schema/store_contract.py +119 -0
- package/runtime/gatewaysdk/environment/schema/tasks.py +121 -32
- package/runtime/gatewaysdk/environment/schema/tools_world.py +32 -10
- package/runtime/gatewaysdk/environment/schema/tree_import.py +228 -0
- package/runtime/gatewaysdk/environment/schema/ui.py +18 -2
- package/runtime/gatewaysdk/environment/schema/validation.py +35 -0
- package/runtime/gatewaysdk/environment/schema/verbs.py +8 -2
- package/runtime/gatewaysdk/environment/world.py +42 -3
- package/runtime/gatewaysdk/environment/world_session_contract.py +57 -0
- package/skills/benchmarks-getting-started/SKILL.md +70 -22
- package/skills/connector-schema-authoring/SKILL.md +54 -4
- package/skills/connector-schema-authoring/connector-worlds.md +21 -7
- package/skills/connector-schema-authoring/world-schema.md +144 -83
- package/skills/hooks-getting-started/SKILL.md +27 -2
- package/skills/world-data-ingestion/ROWS.md +62 -0
- package/skills/world-data-ingestion/SKILL.md +91 -54
- package/skills/worlds-getting-started/SKILL.md +146 -146
- package/skills/worlds-getting-started/references/cli-mcp-map.md +11 -7
- package/skills/worlds-getting-started/references/gotchas.md +30 -0
- package/skills/worlds-getting-started/references/new-world-sandbox.md +43 -0
- package/skills/worlds-getting-started/references/openapi-import.md +90 -0
|
@@ -1,29 +1,25 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: worlds-getting-started
|
|
3
|
-
description: Starter guide for building worlds and environments on the platform, end to end, for an agent (Claude Code, Cursor, Codex). Use when starting a world from scratch, building a world or environment, mocking a vendor, choosing between a shipped template, a workspace connector, a vendor API capture or a custom contract, provisioning worlds for tenants in one call (worlds create), publishing with bench push, opening a session, running and grading, or iterating with pull, fork, diff and
|
|
3
|
+
description: Starter guide for building worlds and environments on the platform, end to end, for an agent (Claude Code, Cursor, Codex). Use when starting a world from scratch, building a world or environment, mocking a vendor, importing a Swagger or OpenAPI spec into a world (schema init --from-openapi), choosing between a shipped template, a workspace connector, an OpenAPI document, a vendor API capture or a custom contract, provisioning worlds for tenants in one call (worlds create), publishing with bench push, opening a session, running and grading, or iterating with pull, fork, diff and edit sessions (worlds edit status/close/rollback: version control without PRs). Every CLI step names its MCP twin (list_connectors, publish_connector, create_world, init_world_from_connector, env_write_file, env_commit, import_world_data, run_evaluation and the rest) and where the result shows in the UI.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Worlds: getting started
|
|
7
7
|
|
|
8
|
-
> **Two copies, kept identical.** This directory (`SKILL.md` + `references/`) ships inside the
|
|
9
|
-
>
|
|
10
|
-
>
|
|
11
|
-
>
|
|
12
|
-
>
|
|
13
|
-
> Edit one, copy to the other; `sdk-ts/src/cli/__tests__/skill.test.ts` compares every file
|
|
14
|
-
> and holds this file under 500 lines.
|
|
8
|
+
> **Two copies, kept identical.** This directory (`SKILL.md` + `references/`) ships inside the `gatewaysdk` Python
|
|
9
|
+
> package (`gatewaysdk/environment/schema/skills/worlds-getting-started/`) and the `@withgateway/sdk` npm package
|
|
10
|
+
> (copied into `skills/` at build). `gateway worlds skill worlds-getting-started` prints this file; `--install DIR`
|
|
11
|
+
> writes the whole directory into a repository's `.claude/skills/`. The repository copy is `.claude/skills/worlds-getting-started/`.
|
|
12
|
+
> Edit one, copy to the other; `sdk-ts/src/cli/__tests__/skill.test.ts` compares every file and holds this file under 500 lines.
|
|
15
13
|
|
|
16
14
|
## Purpose
|
|
17
15
|
|
|
18
|
-
The shortest correct path from nothing to a hosted world with data, a session, a graded
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
not repeated.
|
|
16
|
+
The shortest correct path from nothing to a hosted world with data, a session, a graded run and a next version.
|
|
17
|
+
CLI first; the MCP twin sits under each step for an agent that talks to the platform over MCP instead of a shell.
|
|
18
|
+
Deeper references are linked by path, not repeated.
|
|
22
19
|
|
|
23
|
-
Vocabulary (from `gateway_documentation/pages/glossary.mdx`): **world** (the sealed
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
`environment`, `container`, `task set`, `bench`.
|
|
20
|
+
Vocabulary (from `gateway_documentation/pages/glossary.mdx`): **world** (the sealed replica), **scenario** (a task in
|
|
21
|
+
it), **rollout** (one attempt), **run** (a filed set of rollouts), **eval** (the grade), **gate** (the bar).
|
|
22
|
+
Identifiers keep the old words: `environment`, `container`, `task set`, `bench`.
|
|
27
23
|
|
|
28
24
|
## 0. Authenticate
|
|
29
25
|
|
|
@@ -33,28 +29,25 @@ gateway auth login --host https://<your-gateway> --public-key pk-lf-... --api-ke
|
|
|
33
29
|
gateway auth status # or export GATEWAY_HOST / GATEWAY_PUBLIC_KEY / GATEWAY_SECRET_KEY
|
|
34
30
|
```
|
|
35
31
|
|
|
36
|
-
The npm package bundles the schema runtime (stdlib-only Python). With a `python3 >= 3.
|
|
32
|
+
The npm package bundles the schema runtime (stdlib-only Python). With a `python3 >= 3.12` on the
|
|
37
33
|
machine every runtime command runs locally, including `worlds serve` and a local `connector
|
|
38
34
|
capture`, with no pip and no login; without one, the CLI sends the world's files to the platform
|
|
39
35
|
(`POST /api/public/worlds/runtime`) and the same code runs there (`worlds serve` then opens a hosted
|
|
40
36
|
api session, `connector capture` needs `--remote`). `GATEWAY_RUNTIME=local|platform` forces a side.
|
|
41
37
|
|
|
42
|
-
The interpreter: `GATEWAY_PYTHON` when set, else the first of `python3.13`, `python3.12`,
|
|
43
|
-
`
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
`GATEWAY_PYTHON="$(uv python find 3.12)"` (Brex retrospective: this used to be discovered by hand).
|
|
38
|
+
The interpreter: `GATEWAY_PYTHON` when set, else the first of `python3.13`, `python3.12`, `python3`,
|
|
39
|
+
`python`, then `uv python find` that is 3.12 or newer. A Mac whose `python3` is the system 3.9 is not a dead end: the
|
|
40
|
+
CLI prints one line naming what it found and what to set — `GATEWAY_PYTHON="$(brew --prefix python@3.12)/bin/python3.12"`
|
|
41
|
+
or `GATEWAY_PYTHON="$(uv python find 3.12)"` (Brex retrospective: this used to be discovered by hand).
|
|
47
42
|
|
|
48
|
-
Or export `GATEWAY_HOST`, `GATEWAY_PUBLIC_KEY`, `GATEWAY_SECRET_KEY` (env wins over the
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
accepts `--host`. Keys are project keys: the world lands in that project.
|
|
43
|
+
Or export `GATEWAY_HOST`, `GATEWAY_PUBLIC_KEY`, `GATEWAY_SECRET_KEY` (env wins over the config file). A file of them
|
|
44
|
+
works as written by a person, not only as the shell sources it: `GATEWAY_ENV_FILE=.env gateway …` accepts `KEY = value`
|
|
45
|
+
(spaces around `=`), quoted values, `export KEY=value` and `#` comments; a variable already in the shell's environment
|
|
46
|
+
wins, and vendor keys named by `connector.toml` ride along for a local capture. Every command below accepts `--host`; `GATEWAY_TIMEOUT_MS` (default 120000) bounds one request, and a host that never answers is one line: cause + next step.
|
|
47
|
+
Keys are project keys: the world lands in that project. Every verb whose output is JSON takes `--json` as a no-op.
|
|
54
48
|
|
|
55
|
-
MCP: the server is authenticated by the same project API key (or a PAT). Call
|
|
56
|
-
|
|
57
|
-
and `guide({ topic: "world-data" })` for data.
|
|
49
|
+
MCP: the server is authenticated by the same project API key (or a PAT). Call `describe_project({})`
|
|
50
|
+
first, then `guide({ topic: "connectors" })` for the connector loop and `guide({ topic: "world-data" })` for data.
|
|
58
51
|
|
|
59
52
|
## 1. Pick a path
|
|
60
53
|
|
|
@@ -68,9 +61,8 @@ your own, `[ui] generated = false` keeps a hand-written page — the UI-twin tem
|
|
|
68
61
|
notion, servicenow and zendesk ship a built twin under `ui/static` that way). It is stored as it is, and
|
|
69
62
|
one World Host session serves ALL its surfaces — tools, routes, the page and a pool browser
|
|
70
63
|
— over one store (`--surface api` adds the URL, `--surface ui,browser` the page and the
|
|
71
|
-
browser; the tools are always there). Everything else is derived or retired (owner,
|
|
72
|
-
|
|
73
|
-
deprecated completely"):
|
|
64
|
+
browser; the tools are always there). Everything else is derived or retired (owner, 2026-09-15:
|
|
65
|
+
"all worlds should default to the schema worlds … the tools worlds will be deprecated completely"):
|
|
74
66
|
|
|
75
67
|
- The **api bundle** (`gateway-api-world.toml` over `server/world/`) is a derived layout of a
|
|
76
68
|
schema world: opt-in on the create paths (`--layout api-bundle`, deprecated, the answer says
|
|
@@ -85,16 +77,18 @@ deprecated completely"):
|
|
|
85
77
|
| --- | --- | --- |
|
|
86
78
|
| A vendor the SDK ships a template for (`gateway worlds schema templates`: github, slack, salesforce, jira, linear, workday, netsuite, microsoft-teams, google-drive, google-calendar, jamf, kandji, apple-business-manager, constella) | shipped template | `gateway worlds schema init ./x --connector github` |
|
|
87
79
|
| A connector someone in your org (or any org) already published | workspace connector | `gateway worlds connector list --hub` then `schema init ./x --connector workspace:<slug>` |
|
|
80
|
+
| A Swagger 2.0 / OpenAPI 3.x document (JSON or YAML; a file, `drive:<path>` or a URL) | OpenAPI import | `gateway worlds schema init ./x --from-openapi petstore.yaml` — entities + routes from the spec, plan printed first; §2 "From an OpenAPI spec" |
|
|
88
81
|
| A vendor API with a dev key and no template | vendor API capture | `schema init ./x --connector custom`, write `connector.toml`, capture, ingest (`documentation/features/connector-worlds.md`) |
|
|
89
82
|
| A data model and nothing else | custom contract | `schema init ./x --connector custom`, author `schema/world.json` (`connector-schema-authoring`) |
|
|
90
83
|
| SQL tables and Python tool handlers (a legacy tools world) | retired — author the contract | `schema init ./x --connector custom`, `schema entity add` per table, port the handlers to `tools/handlers.py`; `worlds migrate <slug> --dry-run` on a stored one says exactly this |
|
|
84
|
+
| A set of stored tasks to run with one model, tracked as one run (who, branch, PR, progress, score) | set run | `gateway worlds task run-set <set-id> --agent ./agent.mjs -m <model>`; the set page's Runs tab |
|
|
91
85
|
| A benchmark to build: tasks graded on a world, run k times per model | a world with `[[tasks]]` | `gateway benchmarks init ./x` then `benchmarks check`, `push`, `run`, `results`; skill `benchmarks-getting-started` |
|
|
92
86
|
| A Harbor task suite or a verifiers package | environment (benchmark) project, not a world | `bench push` into an environment project; the worlds paths refuse it; `gateway benchmarks import ./x --from harbor\|verifiers` reports what is expressible as world tasks |
|
|
93
87
|
| A tenant to provision for, no local edit loop (a template, a hub connector, optionally your overlay and their rows) | one-call create | `gateway worlds create acme-slack --from slack --rows rows.json`; see §5 |
|
|
88
|
+
| A new world with nothing local at all (no checkout, no `schema init` on this disk) | new-world sandbox | `gateway worlds sandbox open --new x --from custom\|<template>`, then `sandbox exec <id> "gateway worlds schema entity add …"`, `sandbox check`, `sandbox commit -m "first cut"` creates the world; see §8 |
|
|
94
89
|
|
|
95
|
-
Rule from the owner: a world is read and write 1:1. Reads from the store alone are a lookup
|
|
96
|
-
|
|
97
|
-
`api/handlers.py`. Do not ship a read-only world and call it done.
|
|
90
|
+
Rule from the owner: a world is read and write 1:1. Reads from the store alone are a lookup mock; a vendor
|
|
91
|
+
with workflows needs `action = "create|update|delete|handler"` routes and `api/handlers.py`. Do not ship a read-only world and call it done.
|
|
98
92
|
|
|
99
93
|
MCP: `list_connectors({ query? })` (own + public), `get_connector({ slug, includeFiles: true })`
|
|
100
94
|
to read a template, `clone_connector({ slug })` to take a public one into this org.
|
|
@@ -138,31 +132,58 @@ gateway worlds connector route add ./x get_account --path /v1/accounts/{id} --en
|
|
|
138
132
|
gateway worlds connector route add ./x create_account -X POST --path /v1/accounts --entity crm_accounts --action create --body name=name
|
|
139
133
|
```
|
|
140
134
|
|
|
141
|
-
Each refuses (and writes nothing) on a duplicate name, an undeclared entity or `$defs`
|
|
142
|
-
|
|
143
|
-
argument. A create route mints its single-field primary key (`uuid` for strings, `int` for integers) unless `--body` maps it;
|
|
135
|
+
Each refuses (and writes nothing) on a duplicate name, an undeclared entity or `$defs` reference, a key that is not a
|
|
136
|
+
required string/integer field, or a filter on an unknown argument. A create route mints its single-field primary key (`uuid` for strings, `int` for integers) unless `--body` maps it;
|
|
144
137
|
`--generate field=uuid|int|now` and `--default field=value` fill the rest. A custom world's first `route add` creates `connector.toml`; `schema init` prints `kind: "schema"`, `api` and the `next` commands in order (custom: entity add → route add → task add → compile → check → import → serve → `worlds create`).
|
|
145
138
|
Manifest `kind` is `schema` (older trees: `mock-tools`). Handlers: `def name(ctx, args, query, body)`; `ctx` reads and writes the store and keeps the clock.
|
|
146
139
|
|
|
140
|
+
### From an OpenAPI spec (Swagger 2.0, OpenAPI 3.x)
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
gateway worlds schema init ./x --from-openapi petstore.yaml --plan-only # a file, drive:<path> or an https URL; --plan-only prints {entities, operations, skipped, warnings} and writes nothing
|
|
144
|
+
gateway worlds schema init ./x --from-openapi github.json --include '/repos/{owner}/{repo}/issues*' # --include/--exclude GLOB|TAG (repeatable) narrow a big spec; then the tree is written
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Keyed schemas become entities, path+method pairs routes (list/get/create/update/delete by response shape and method; other responses stay passthrough); `schema/provenance.json` records source, version, sha256. Refusals (no `paths`, unknown `$ref`, cyclic entity root, undeclared path parameter) name a JSON pointer and write nothing. Rules, plan shape, limits: `references/openapi-import.md`; then the printed `next` steps (compile, check, `data import`, serve, `sandbox commit`).
|
|
148
|
+
|
|
147
149
|
MCP: `get_connector_authoring_skill({})` returns the authoring skill and conventions.
|
|
148
150
|
Author files locally or with `env_write_file({ containerId, path, newContent })` on an existing
|
|
149
151
|
environment; land them with `env_commit({ containerId, changeReason })` (one version for the batch).
|
|
150
152
|
|
|
153
|
+
### Store API
|
|
154
|
+
|
|
155
|
+
Tool handlers (`tools/*.py`) reach the world's rows through `from gateway_env import store`. It has
|
|
156
|
+
exactly these methods; `gateway worlds schema check .` refuses any other `store.<name>` at file:line
|
|
157
|
+
with this list, and a hosted call into such a handler answers 400 `{error: "handler", detail, handler}`
|
|
158
|
+
(an `AttributeError`/`TypeError`/`NameError` is the author's bug; anything else a handler raises is a
|
|
159
|
+
500 `{error: "engine", detail, handler}`; never a bare exception string).
|
|
160
|
+
|
|
161
|
+
- `get(entity: str, key: dict) -> dict | None` — one row by primary key, or `None`.
|
|
162
|
+
- `list(entity: str) -> list[dict]` — every row, ordered by key.
|
|
163
|
+
- `list_where(entity: str, where: dict[str, list], limit: int | None = None, offset: int = 0) -> tuple[list[dict], int]` — a page of rows whose `where` fields hold one of the listed values, plus the total.
|
|
164
|
+
- `insert(entity: str, value: dict, *, source: dict | None = None) -> dict` — validate against the entity's schema and add; the key must be new.
|
|
165
|
+
- `update(entity: str, key: dict, patch: dict, *, source: dict | None = None) -> dict` — merge `patch` into the row at `key`, validate, return the row.
|
|
166
|
+
- `delete(entity: str, key: dict, *, source: dict | None = None) -> None` — remove the row at `key`; a missing row is a contract error.
|
|
167
|
+
- `apply_batch(batch: dict) -> dict` — one ingest batch `{id, target, mapping, coverage, mutations}`, atomically; an exact retry is a no-op.
|
|
168
|
+
- `lineage(entity: str, key: dict) -> list[dict]` — the batches that wrote the row at `key`, oldest first.
|
|
169
|
+
- `counts() -> dict[str, int]` — rows per entity, `{entity: count}`.
|
|
170
|
+
|
|
171
|
+
No `upsert`: `get`, then `insert` or `update`. Read-only SQL: `gateway_env.db.one/all`; raw SQL writes are refused. The scaffolded `tools/handlers.py` docstring carries the same table.
|
|
172
|
+
|
|
151
173
|
## 3. Check
|
|
152
174
|
|
|
153
175
|
```bash
|
|
154
176
|
gateway worlds schema compile ./x
|
|
155
177
|
gateway worlds schema check ./x
|
|
156
178
|
gateway worlds connector validate ./x # only when connector.toml exists
|
|
157
|
-
gateway worlds connector conform ./x --openapi vendor.yaml #
|
|
179
|
+
gateway worlds connector conform ./x --openapi vendor.yaml # checks an existing world against the vendor's spec; no world yet? `schema init --from-openapi` (§2)
|
|
158
180
|
gateway worlds serve ./x --port 8080 # answers the vendor's routes locally
|
|
159
181
|
```
|
|
160
182
|
|
|
161
183
|
`check` validates the compiled bundle; `conform` drives every declared route against the
|
|
162
184
|
vendor's document and exits non-zero on a mismatch. Its FAIL list is the work list.
|
|
163
185
|
|
|
164
|
-
MCP: `validate_connector_template({ files, slug? })` compiles without writing
|
|
165
|
-
`{ valid, reason? }`.
|
|
186
|
+
MCP: `validate_connector_template({ files, slug? })` compiles without writing; answers `{ valid, reason? }`.
|
|
166
187
|
|
|
167
188
|
## 4. Data
|
|
168
189
|
|
|
@@ -170,25 +191,23 @@ MCP: `validate_connector_template({ files, slug? })` compiles without writing an
|
|
|
170
191
|
gateway worlds describe ./x # entities, keys, relationships, row counts, [redact] per field
|
|
171
192
|
gateway worlds data show ./x # kind + entity counts
|
|
172
193
|
gateway worlds data query ./x users --where email=jo@acme.test --resolve # the rows you mean; plaintext into a hashed field
|
|
173
|
-
gateway worlds data import ./x rows.json # rows file: {entity: [rows]} | JSONL | --entity
|
|
194
|
+
gateway worlds data import ./x rows.json # rows file: {entity: [rows]} | JSONL | .csv/.tsv --entity (cells cast by the schema; --delimiter)
|
|
195
|
+
gateway files fetch https://example.test/rows.csv --to imports/rows.csv --header "Authorization: Bearer {{secret:API_KEY}}" # the platform fetches; the secret never leaves it
|
|
174
196
|
gateway worlds connector capture ./x --op <name> --arg k=v # from the real vendor, redacted
|
|
175
|
-
gateway worlds connector ingest ./x # captures -> data/initial.json
|
|
176
|
-
gateway worlds connector ingest ./x --dry-run --report # check only, nothing written; report by capture / entity / pointer
|
|
197
|
+
gateway worlds connector ingest ./x # captures -> data/initial.json; `--dry-run --report` checks only, nothing written
|
|
177
198
|
gateway worlds data calls pull calls.jsonl --from traces --session <id> # an agent's real calls, raw; prints records per tool + result shapes
|
|
178
199
|
gateway worlds data ingest ./x calls.jsonl --transform shape.py --dry-run # your code shapes them -> rows through the same gate (world-data-ingestion §2c)
|
|
179
200
|
gateway worlds data ingest ./x calls.jsonl --map ingest.toml # or a declared mapping; `data ingest map init` writes the skeleton
|
|
180
201
|
```
|
|
181
202
|
|
|
182
|
-
`describe` is the map before any query: every field says whether it is stored in the clear,
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
`@withgateway/sdk/worlds`) and the served handlers all compute it from that one rule. A dropped
|
|
203
|
+
`describe` is the map before any query: every field says whether it is stored in the clear, hashed (`[hash]`),
|
|
204
|
+
shape-preserved (`[preserve]`), rounded, or dropped (`[drop]`, never on disk). A hashed field is still looked up by
|
|
205
|
+
its plaintext: the world's `[redact]` policy fixes the stored form (`sha256:` of `<slug>:<value>`), and the CLI, the
|
|
206
|
+
SDK (`storedForms` in `@withgateway/sdk/worlds`) and the served handlers all compute it from that one rule. A dropped
|
|
187
207
|
field is a documented miss, not an error.
|
|
188
208
|
|
|
189
|
-
Rows file contract, modes, upsert and refusals: `world-data-ingestion`. Captures: pace
|
|
190
|
-
|
|
191
|
-
`--remote` after `gateway bench secrets set --env x --key VENDOR_API_KEY`.
|
|
209
|
+
Rows file contract, modes, upsert and refusals: `world-data-ingestion`. Captures: pace with `[capture] requests_per_minute`;
|
|
210
|
+
a vendor that allow-lists IPs is captured with `--remote` after `gateway bench secrets set --env x --key VENDOR_API_KEY`.
|
|
192
211
|
|
|
193
212
|
MCP: `import_world_data({ containerId | slug, rows, mode?, message? })` on a pushed world.
|
|
194
213
|
|
|
@@ -199,18 +218,14 @@ gateway bench push ./x -m "first cut" # a world version (t
|
|
|
199
218
|
gateway worlds connector publish ./x --slug x -m "first cut" # the template as a workspace connector (org-owned, versioned)
|
|
200
219
|
```
|
|
201
220
|
|
|
202
|
-
`bench push` makes the version sessions and runs address (`slug@version`). `connector publish`
|
|
203
|
-
|
|
204
|
-
`
|
|
205
|
-
|
|
206
|
-
retrospective: they used to be left behind, so the new world had no suite to run). Both, for a
|
|
207
|
-
world you expect to rebuild.
|
|
221
|
+
`bench push` makes the version sessions and runs address (`slug@version`). `connector publish` saves the contract
|
|
222
|
+
(never `data/`, `captures/`, `db/`) so anyone in the org can `schema init --connector workspace:x`. The world's
|
|
223
|
+
`tests/`, `tasks.json`, `verifiers/` and `openapi/` travel with the connector into every world initialized from it
|
|
224
|
+
(they used to be left behind, so the new world had no suite to run). Both, for a world you expect to rebuild.
|
|
208
225
|
|
|
209
|
-
MCP: `publish_connector({ slug, files, message? })` is the template write path.
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
`env_commit`, or `create_environment_version({ containerId, path, newContent, changeReason })`
|
|
213
|
-
for one file).
|
|
226
|
+
MCP: `publish_connector({ slug, files, message? })` is the template write path. `init_world_from_connector({ slug, name })`
|
|
227
|
+
makes the world under a PAT session; under an API key it returns the initialized files plus the publish steps
|
|
228
|
+
(`env_write_file` per file, then `env_commit`, or `create_environment_version({ containerId, path, newContent, changeReason })` for one file).
|
|
214
229
|
|
|
215
230
|
### One call, no local tree: `worlds create`
|
|
216
231
|
|
|
@@ -220,21 +235,25 @@ For a world you provision rather than iterate on (a tenant's Slack, one per cust
|
|
|
220
235
|
gateway worlds create acme-slack --from slack --rows rows.json --name "Acme Slack" # shipped template + their rows
|
|
221
236
|
gateway worlds create acme-crm --from workspace:acme-crm --overlay ./overlay --json # hub connector + your edits
|
|
222
237
|
gateway worlds create acme-custom --from ./x # a directory, any kind
|
|
238
|
+
gateway worlds create acme-drive --from-connection google_drive # the connector's template filled from an active connection (id or slug); `--from <template>` picks the template
|
|
223
239
|
gateway worlds create --batch worlds.jsonl # {slug, from, overlay?, data?, name?} per line; exit 2 if any failed
|
|
224
240
|
```
|
|
225
241
|
|
|
226
|
-
One request (`POST /api/public/worlds`, `documentation/api/rest/worlds-create.md`): the source,
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
242
|
+
One request (`POST /api/public/worlds`, `documentation/api/rest/worlds-create.md`): the source, an
|
|
243
|
+
optional overlay laid over it, optional rows through the world's contract. An overlay or a directory
|
|
244
|
+
is compiled by the schema runtime before anything is stored, so a broken edit is a 400 naming the
|
|
245
|
+
path, not a broken world. What lands is the schema tree as it is: a container and a READY first
|
|
246
|
+
version, attributed to the key (`apikey:<id>`), owned by nobody. 409 when the slug is taken in the
|
|
247
|
+
project. `--layout api-bundle` (body `layout`) packages the tree as its api bundle instead —
|
|
248
|
+
deprecated, and the answer's `deprecations` says so. A legacy tools bundle, a Harbor tree or a
|
|
249
|
+
verifiers package is a 400 `world_kind_retired`. `--from-connection` binds an active connection
|
|
250
|
+
(Connectors page) shared with the key's project — a personal one serves its owner alone (MCP as
|
|
251
|
+
that user); an inactive connection or a template that does not read the connector is refused,
|
|
252
|
+
nothing created. Refresh later with `gateway worlds data pull <slug> --connection …`.
|
|
253
|
+
|
|
254
|
+
MCP: `create_world({ slug, from: { connector } | { connection, connector? } | { files }, overlay?, data?, name?, message?, layout? })`
|
|
255
|
+
is the same request and answer; a PAT session reaches its own personal connections.
|
|
256
|
+
`init_world_from_connector` (also `layout?`) stays for the PAT flow that wants a human owner.
|
|
238
257
|
|
|
239
258
|
### Migrate a stored world: `worlds migrate`
|
|
240
259
|
|
|
@@ -293,18 +312,18 @@ stored join. `worlds describe <slug>` lists links.
|
|
|
293
312
|
gateway worlds describe x # a platform world: the same map, from the head version
|
|
294
313
|
gateway worlds session open x -t default # prints {sessionId, surfaces}; on a schema world with connector.toml the host serves tools AND api
|
|
295
314
|
gateway worlds session open x -t default --surface api # asks for api.url explicitly: the vendor's routes, hosted (400 on a tree with no connector.toml)
|
|
296
|
-
gateway worlds session open x -t default --surface ui,browser # + ui.entry (the page, token once then cookie) and browser.wsUrl (a pool Chromium); 400 without [ui] or without a pool
|
|
315
|
+
gateway worlds session open x -t default --surface ui,browser # + ui.entry (the page, token once then cookie) and browser.wsUrl (a pool Chromium); 400 without [ui] or without a pool; --browser-tier standard (spot, may be reclaimed; the SDK repairs + reconnects, raises BrowserReconnected on a lost click/type) | premium (on-demand), default: the org's setting
|
|
297
316
|
gateway worlds session browse <SESSION_ID> [--path /#/accounts] [--screenshot p.png] # open the page in the browser surface; prints url, title, text
|
|
298
|
-
gateway worlds session open x -
|
|
317
|
+
gateway worlds session open x [--no-wait] # EXPLORATORY: no task, no grader (task.source "none"); --no-wait returns at once on a cold version
|
|
299
318
|
gateway worlds session open x --task-file task.json --surface api # the session's own task (below)
|
|
300
319
|
gateway worlds session status <SESSION_ID> # poll until ready
|
|
301
320
|
gateway worlds session seed <SESSION_ID> rows.json # rows into this session only
|
|
302
321
|
gateway worlds session state <SESSION_ID> # every entity's rows, the dump a grader reads
|
|
303
322
|
gateway worlds session reset <SESSION_ID> # back to the snapshot (+ the task's seed)
|
|
304
|
-
gateway worlds session grade <SESSION_ID> [--timeout S] # {reward, rewards, raw};
|
|
323
|
+
gateway worlds session grade <SESSION_ID> [--timeout S] # {reward, rewards, raw}; exploratory → {reward: null, reason: "no grader"}
|
|
305
324
|
gateway worlds session export <SESSION_ID> --out calls.jsonl
|
|
306
325
|
gateway worlds data ingest x calls.jsonl --transform shape.py # the calls back in as rows, shaped by your code or --map ingest.toml (world-data-ingestion §2c)
|
|
307
|
-
gateway worlds session close <SESSION_ID>
|
|
326
|
+
gateway worlds session close <SESSION_ID> # a closed session reopens by id, same URLs + token, from its last checkpoint: gateway worlds session reopen <SESSION_ID>
|
|
308
327
|
```
|
|
309
328
|
|
|
310
329
|
**A task outside the bundle.** `-t <name>` picks a scenario the version ships. On a World Host
|
|
@@ -328,10 +347,12 @@ laid on the clone and re-applied by `reset`; `grader` is `assertions` (`entity`,
|
|
|
328
347
|
`{kind: "python", source}` (a `verify.py` body) or `{kind: "rubric", rubric, criteria?, model?}`
|
|
329
348
|
(an LLM judge over end state + call log, graded on the platform: `grade` answers pending until
|
|
330
349
|
the worker finalizes it); `tools: [names]` narrows the operations the session may call to
|
|
331
|
-
exactly those (a name the world does not serve is a 400); `metadata` rides the session.
|
|
332
|
-
`-t` / `--task-id` / `--task-file
|
|
333
|
-
SDK: `openSession(slug, { task: spec })` (TS), `open_session(slug, spec)` (Python);
|
|
334
|
-
`runSessions` / `run_sessions` take a mixed list of names and specs.
|
|
350
|
+
exactly those (a name the world does not serve is a 400); `metadata` rides the session. At most one of
|
|
351
|
+
`-t` / `--task-id` / `--task-file`; none is an exploratory session (`{source: "none"}`, no grader). The output's
|
|
352
|
+
`task` block is `{source: "inline", name, hash, metadata}`. SDK: `openSession(slug, { task: spec })` / `openSession(slug)` (TS), `open_session(slug, spec)` / `open_session(slug)` (Python);
|
|
353
|
+
`runSessions` / `run_sessions` take a mixed list of names and specs. Over several worlds, `worlds: {alias: {slug, …}}` opens one session per alias
|
|
354
|
+
(`task up|grade|export|down`, `open_task`/`openTask`); a question spanning them is a `verifiers[]` entry (`name`, `scope`, assertions|python over
|
|
355
|
+
`<alias>.<entity>` / `<alias>.calls`), graded on the platform only, rewards `x.<name>` (worlds/sessions.mdx).
|
|
335
356
|
|
|
336
357
|
**A stored task.** The same spec kept as a project resource, versioned by content, so many
|
|
337
358
|
sessions open it by id and grades attribute to the task and its version:
|
|
@@ -370,13 +391,15 @@ gateway bench dispatch <RUN_CONFIG_ID> # each stored task
|
|
|
370
391
|
The first session on a freshly pushed version takes about fifteen minutes: the pod is
|
|
371
392
|
scheduled and the runner sidecar pulled; the session reads `PROVISIONING` and the api URL
|
|
372
393
|
answers 503. That is a cold start, not a fault, until about twenty minutes have passed.
|
|
373
|
-
`session open` waits ten minutes
|
|
394
|
+
`session open` prints `session <id> opening — gateway worlds session status <id>` on stderr at once, then waits ten minutes; on a cold version use `--no-wait` and poll. `open` and
|
|
395
|
+
`status` print `idleExpiresAt` (null once pinned) + `pinned`; `status` warns under 5 minutes. A call
|
|
396
|
+
body over 4mb is refused client-side: `{status: "error", code: "payload_too_large"}`, exit 2.
|
|
374
397
|
|
|
375
398
|
The world's API honours the vendor's auth scheme; `WORLD_API_KEY` on the host pins the
|
|
376
399
|
credential, otherwise any non-empty one is accepted.
|
|
377
400
|
|
|
378
|
-
MCP: `seed_world_session
|
|
379
|
-
|
|
401
|
+
MCP: `seed_world_session`, `export_world_session`, `save_world_session_task`; open/call/close are CLI or REST only.
|
|
402
|
+
Find what you opened: `gateway worlds session list` / `sandbox list`; `session pin <id>` stops the reaper.
|
|
380
403
|
|
|
381
404
|
## Test the world (not the agent)
|
|
382
405
|
|
|
@@ -415,11 +438,11 @@ its `api.url` takes the session's bearer): act on it by id, `gateway worlds call
|
|
|
415
438
|
## 7. Run and grade
|
|
416
439
|
|
|
417
440
|
```bash
|
|
418
|
-
gateway worlds run x -a agent.
|
|
419
|
-
gateway worlds run
|
|
420
|
-
gateway worlds ci
|
|
421
|
-
gateway environment run x -m <provider/model>
|
|
422
|
-
gateway environment run-status <run-id>
|
|
441
|
+
gateway worlds run x -a agent.mjs --all # TS CLI: agent module (default export async (session, task) => void), every task; -t <task> for one; x is a slug[@ref] or a pushed dir
|
|
442
|
+
gateway worlds run x -a agent.py:agent # Python CLI: agent callable module:function; --path ./x runs a local dir, --no-file grades without filing
|
|
443
|
+
gateway worlds ci ./x -a agent.mjs --min-score 0.8 # push the working tree, run, gate (CI); --run-config <file> instead of -a; Python: --path ./x -a agent.py:agent
|
|
444
|
+
gateway environment run x -m <provider/model> -k 3 # hosted runner, one job per model; -k is 1–64
|
|
445
|
+
gateway environment run-status <run-id> [--cancel] # progress while running; --cancel stops it
|
|
423
446
|
```
|
|
424
447
|
|
|
425
448
|
Scenarios (task sets):
|
|
@@ -433,68 +456,45 @@ gateway environment rollouts <container-id>
|
|
|
433
456
|
```
|
|
434
457
|
|
|
435
458
|
MCP: `run_evaluation({ models | runConfigId, versionId?, numExamples?, rolloutsPerExample? })`,
|
|
436
|
-
`create_task_set
|
|
437
|
-
`
|
|
438
|
-
`promote_failure_to_scenario({...})` turns a failed rollout into a scenario.
|
|
459
|
+
`create_task_set`, `create_task`, `link_task_set`, `list_rollouts`, `get_rollout_samples`,
|
|
460
|
+
`get_task_performance`; `promote_failure_to_scenario({...})` turns a failed rollout into a scenario.
|
|
439
461
|
|
|
440
462
|
## 8. Iterate
|
|
441
463
|
|
|
442
464
|
```bash
|
|
443
|
-
gateway bench pull x # reconstruct the head version's tree
|
|
444
|
-
gateway bench
|
|
445
|
-
gateway
|
|
446
|
-
gateway
|
|
447
|
-
gateway
|
|
448
|
-
gateway bench
|
|
449
|
-
gateway worlds sandbox
|
|
450
|
-
gateway worlds sandbox
|
|
465
|
+
gateway bench pull x # reconstruct the head version's tree; `bench status ./x` = branch, base, local edits
|
|
466
|
+
gateway bench fork x --into x-experiment # a new container sharing content; `bench diff <base> <head>` per-file, `bench log x` versions
|
|
467
|
+
gateway worlds edit status x # your open EDIT SESSION: every push / sandbox commit / data import after the first version lands there; only this credential reads it as main
|
|
468
|
+
gateway worlds edit close x -m "why" # seal it: main moves to the session's tip for everyone; each version stays in `log`, `edit rollback` undoes the whole session (60 idle minutes seal it too). `edit list x` = history, `edit discard x` drops it unseen
|
|
469
|
+
gateway worlds edit rollback x <sessionId> # undo a sealed session (source + data together) as a new sealed session; --force when later sessions would go too
|
|
470
|
+
gateway worlds sandbox open x@main # no local checkout: write|exec|check|commit|close <id> edit the tree server-side; `bench sandbox` is the same set
|
|
471
|
+
gateway worlds sandbox exec <id> 'gateway worlds schema entity add . tickets --fields id:string,title:string -k id && gateway worlds schema compile . && gateway worlds data import . rows.json && gateway worlds schema check . && gateway worlds test .'
|
|
472
|
+
gateway worlds sandbox open --new y --from custom # a world that does not exist yet (no base); its first `commit` creates it — references/new-world-sandbox.md
|
|
473
|
+
gateway worlds sandbox import <id> data/trips.csv --entity trip # rows a sandbox made or curl'ed → the PLATFORM world, read by the platform; `commit` carries the source tree only
|
|
451
474
|
```
|
|
452
475
|
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
`
|
|
460
|
-
|
|
461
|
-
## 9. Where it shows in the UI
|
|
462
|
-
|
|
463
|
-
Every route, per thing (connector page, world tabs, Worlds › Map, sessions, tests, stored
|
|
464
|
-
tasks): `references/ui-map.md`.
|
|
465
|
-
|
|
466
|
-
## CLI to MCP map
|
|
476
|
+
The sandbox carries the platform's `gateway` CLI and `sqlite3` on PATH: `exec` runs `worlds
|
|
477
|
+
schema|data|test|serve|call …` on the working copy with nothing on your machine. `open` prints
|
|
478
|
+
`cli: {version, digest}`, or `cli: null` plus the reason under `warnings`. Remote verbs (`push`, `run`,
|
|
479
|
+
`session open`) stop inside the sandbox by design: it holds no platform credential, and never write one
|
|
480
|
+
into its tree — `check`/`commit` hold back scratch, credential-shaped files (`.env`, `.gateway-env`, `*.env`,
|
|
481
|
+
`*.pem`…) and symlinks (`ignored` + `warnings`, paths only); over 2000 files or 128 MB is refused before
|
|
482
|
+
any read. `sandbox commit <id> -m why --rebase` replays your edits onto a branch that moved since `open`; a path changed on both sides is a 409, one `conflict: <path> (<reason>)` line each on stderr, exit 1. `worlds run --path` on a clean `bench pull` checkout files runs pinned to the base version.
|
|
483
|
+
Work is saved while a sandbox sits idle and resumes by id, but after 7 idle days (`status` shows `idleExpiresAt`) the platform closes it even with uncommitted files, and a closed sandbox cannot be reopened: commit before you walk away. `sandbox close <id>` refuses uncommitted files (exit 1, their paths on stderr) unless `--force`. Sandbox twins and every limit: `references/cli-mcp-map.md`.
|
|
467
484
|
|
|
468
|
-
|
|
469
|
-
reaches the same surface (`run_gateway_cli`, `spawn_subagent`): `references/cli-mcp-map.md`.
|
|
485
|
+
MCP: `list_world_edits({ slug })`, `close_world_edit`, `discard_world_edit`, `rollback_world_edit({ slug, sessionId })`; `env_diff({ containerId })`, `env_run_tests`, `env_get_run`; proposals (`env_propose`, `env_list_proposals`, `env_apply_proposal`, `env_dismiss_proposal`) are the legacy PR-style door.
|
|
470
486
|
|
|
471
|
-
##
|
|
487
|
+
## 9. Where it shows in the UI, the CLI to MCP map, gotchas
|
|
472
488
|
|
|
473
|
-
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
the agent's local-only `run_gateway_cli`, `schema init` refuses it and names `--host`,
|
|
477
|
-
`GATEWAY_HOST` and the login command. A machine with no `python3 >= 3.11` scaffolds a shipped
|
|
478
|
-
template on the platform instead, so there it does need a host.
|
|
479
|
-
- `bench push` (the TS CLI) on a world with `connector.toml` still packages an api-kind bundle
|
|
480
|
-
on the way up; `worlds create --from ./x` stores the tree as it is, and `worlds migrate`
|
|
481
|
-
turns a pushed bundle back into the tree. Either way, edit the schema world, not the bundle.
|
|
482
|
-
- The World Host is the engine for every world, `[dependencies]` included (it opens each linked
|
|
483
|
-
world beside the root); a session never falls back to a slower engine, and a deployment
|
|
484
|
-
without a host refuses the open by name.
|
|
485
|
-
- Sharing is publish + clone, never a live reference. A publisher's later edits reach a clone
|
|
486
|
-
only on re-clone.
|
|
487
|
-
- Never commit `captures/`; never merge a customer's world as an SDK template.
|
|
488
|
-
- A seed (scenario overlay or runner admin endpoint) is the whole state, not an increment.
|
|
489
|
-
- Never set `max_tokens` on any model call used to run or grade.
|
|
489
|
+
- `references/ui-map.md`: every route, per thing (connector page, world tabs, Worlds › Map, sessions, tests, stored tasks).
|
|
490
|
+
- `references/cli-mcp-map.md`: every CLI step with its MCP twin, the steps that have none, and how the platform (console) agent reaches the same surface (`run_gateway_cli`, `spawn_subagent`).
|
|
491
|
+
- `references/gotchas.md`: offline vs platform scaffolding, `bench push` vs `worlds create`, the World Host as the only engine, sharing, seeds, `max_tokens`, secret preflight, `secret set` upsert.
|
|
490
492
|
|
|
491
493
|
## Related
|
|
492
494
|
|
|
493
|
-
- `benchmarks-getting-started
|
|
494
|
-
|
|
495
|
-
- `documentation/features/connector-worlds.md
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
-
|
|
499
|
-
the authoring skill: `gateway worlds skill connector-schema-authoring --install .` installs them.
|
|
500
|
-
- Consumer pages: `gateway_documentation/pages/worlds/{getting-started,data,custom-connections,slack}.mdx`.
|
|
495
|
+
- `benchmarks-getting-started` (a benchmark is a world with tasks: init, check, push, run, results);
|
|
496
|
+
`world-data-ingestion` (rows in, calls out); `connector-schema-authoring` (contract, fidelity shapes, definition of done).
|
|
497
|
+
- `documentation/features/connector-worlds.md` (`connector.toml`, capture, conform, serving), `workspace-connectors.md`
|
|
498
|
+
(the connector object, sharing, MCP surface), `world-schema.md` (the `gateway-world/1` contract and store API);
|
|
499
|
+
the first and last ship with the npm package (`gateway worlds skill connector-schema-authoring --install .`).
|
|
500
|
+
- Consumer pages: `gateway_documentation/pages/worlds/{getting-started,sessions,data,custom-connections,slack}.mdx`.
|
|
@@ -17,13 +17,16 @@ Reference for `worlds-getting-started`: every CLI step and its MCP twin, then ho
|
|
|
17
17
|
| `worlds migrate <slug> [--apply] [--force]` / `--all` | `migrate_world` |
|
|
18
18
|
| `bench push` | `env_write_file` + `env_commit`, or `create_environment_version` (one file) |
|
|
19
19
|
| `worlds data import` | `import_world_data` |
|
|
20
|
+
| `worlds session open [-t\|--task-id\|--task-file]` (none: exploratory) / `status` / `call` / `close` / `list` / `pin` | none on the public MCP server — CLI or REST only (`/api/public/world-sessions[/{id}[/calls]]`; the open answers `idleExpiresAt` + `pinned`) |
|
|
20
21
|
| `worlds session seed` | `seed_world_session` |
|
|
21
22
|
| `worlds session export` | `export_world_session` |
|
|
23
|
+
| `worlds session reopen <id>` | — (no MCP twin; `POST /api/public/world-sessions/{id}/reopen`: a closed session back under the same id, URLs and token from its last checkpoint, answering what `status` answers; 409 `session_not_closed` / `session_no_checkpoint` / `session_reopen_in_progress` / `session_version_unavailable`) |
|
|
22
24
|
| `worlds data calls pull` / `worlds data ingest [--transform\|--map]` | — (no MCP twin yet; `run_gateway_cli data ingest` on a sandboxed directory) |
|
|
23
25
|
| `files upload <local>... [--to]` | `request_file_upload` + `confirm_file_upload` (`upload_file` for ≤25 MB inline) |
|
|
24
26
|
| `files list [path]` / `files get <path>` / `files download\|cat <path>` | `list_files` / `get_file` (a presigned URL; the CLI streams it) |
|
|
25
27
|
| `files delete` / `files move` / `files mkdir` | `delete_file` / `move_file` / `create_folder` |
|
|
26
|
-
| `files fetch <url> --to <path
|
|
28
|
+
| `files fetch <url> --to <path> [--header "Name: value"]...` | `download_url_to_drive` (`POST /api/public/files/fetch`; the platform fetches, nothing passes through the caller's machine; a header value may name a project secret as `{{secret:NAME}}`, expanded on the platform and never returned; `--local` fetches from the caller's machine instead, without secrets) |
|
|
29
|
+
| `worlds sandbox import <id> <path-in-sandbox> [--entity] [--mode]` | `import_world_sandbox_data` (`POST /api/public/benchmark-containers/sandboxes/{id}/import`; the platform reads the file from the sandbox tree into the platform world — the sandbox never holds credentials; `commit` versions the source tree only) |
|
|
27
30
|
| `files connectors list\|sync <id>` | `list_file_connectors` / `sync_file_connector` (`GET\|POST /api/public/files/connectors[/sync]`) |
|
|
28
31
|
| `traces list\|get\|export [--observations]` | `get_trace`, `get_observations` (`GET /api/public/traces[/{id}]`, `/observations`) |
|
|
29
32
|
| `traces sessions list\|get [--io]` | `get_session` (`GET /api/public/sessions[/{id}?includeIO=true]`) |
|
|
@@ -31,14 +34,13 @@ Reference for `worlds-getting-started`: every CLI step and its MCP twin, then ho
|
|
|
31
34
|
| `worlds data … drive:<path>` / `--to <path>` (other data commands) | `get_file` + the command (the CLI downloads, then runs the command as on a local file) |
|
|
32
35
|
| `environment run` / `worlds run` | `run_evaluation` |
|
|
33
36
|
| `environment link-task-set` / `create-task` | `create_task_set`, `create_task`, `link_task_set` |
|
|
34
|
-
| `bench diff` / `propose` | `env_diff`, `env_propose` |
|
|
35
|
-
| `worlds\|bench
|
|
37
|
+
| `bench diff` / `propose` | `env_diff`, `env_propose` (proposals are the legacy PR-style door; edit sessions below are the version-control loop) |
|
|
38
|
+
| `worlds\|bench edit status\|list\|close\|discard\|rollback <slug>` | `list_world_edits({ slug, status?, limit? })`, `close_world_edit({ slug, sessionId?, force?, message? })`, `discard_world_edit`, `rollback_world_edit({ slug, sessionId, force?, message? })` (`GET /api/public/worlds/{slug}/edits`, `POST …/edits/close\|discard\|rollback`). After a world's first version every push, sandbox commit, data import, agent `env_commit` and UI save lands on the credential's open edit session — a private `edit/<id>` branch only that credential reads as `main` (`describe`, `pull`, `log`, `session open`, `sandbox open`); the write's JSON carries `editSessionId` + a hint. `close`, or 60 minutes without an edit, seals the session: the shared `main` moves to its tip (every version stays in `log`; the session is the rollback unit); a `main` that moved meanwhile is rebased when the paths differ, else the session parks as `conflict` (`close --force` takes its content). `rollback` restores source + data snapshot from before a sealed session as a new sealed session (`--force` when later sessions would be undone too). UI: world › History › Edit sessions |
|
|
39
|
+
| `worlds\|bench sandbox open\|status\|read\|write\|exec\|check\|commit\|close` | `open_world_sandbox`, `get_world_sandbox`, `read_world_sandbox_file`, `write_world_sandbox_file` (base64), `exec_in_world_sandbox`, `check_world_sandbox`, `commit_world_sandbox`, `close_world_sandbox({ sandboxId, force? })` (`/api/public/benchmark-containers/sandboxes[/{id}[/file\|exec\|check\|commit]]`; `close` with uncommitted work is a 409 `sandbox_uncommitted` naming the paths, nothing closed, unless `force: true` / `--force`; the source tree only — no data snapshot, nothing on the caller's disk; `commit` lands on the recorded base; `commit --rebase` sends `rebase: true` and replays onto a branch head that moved, a 409 naming each conflicting path). `open` reports `cli: {version, digest}` — the platform's own `gateway` CLI and a `sqlite3` are on PATH inside `exec`, so `gateway worlds schema entity add\|verifier add\|task add\|compile\|check .`, `gateway worlds data import\|show\|query .`, `gateway worlds test .`, `gateway worlds serve . --port 8080 & sleep 2; gateway worlds call http://localhost:8080 <tool> …` (one `exec`; a served world lives only as long as that command) and `sqlite3 db/state.sqlite "…"` run on the working copy; remote verbs (push, run, session open) stop in the sandbox by design — `check`/`commit` from your machine. `cli: null` + `warnings[]` means the image cannot host the runtime (python3 < 3.12 or node < 18); plain shell still runs |
|
|
40
|
+
| `worlds\|bench sandbox open --new <slug> --from custom\|<template>` | `open_world_sandbox({ new, from })` (same route, body `{new, from}`; a world that does not exist yet — `baseVersionId`/`containerId`/`baseContentHash` null, `branch: main`, `pendingSlug`; `custom` = the `schema init` scaffold, else a shipped template (`GET /api/public/worlds/templates`; unknown → 400 naming the list); taken slug → 409 `world_exists`, never renamed; `check_world_sandbox` runs the first commit's gate; the first `commit_world_sandbox` creates the world on `main` through the `worlds create --from <dir>` door, later commits land on the new base; the same staged CLI runs on the scaffold) |
|
|
36
41
|
| `worlds skill <name>` / `worlds connector skill` | `get_connector_authoring_skill`, `guide` |
|
|
37
42
|
| `worlds connector capture --remote` / poll + download | `start_connector_capture`, `get_connector_capture` (platform agent only) |
|
|
38
43
|
|
|
39
|
-
Steps with no MCP tool: opening a session (`worlds session open`) — use the CLI or the public
|
|
40
|
-
sessions route.
|
|
41
|
-
|
|
42
44
|
## Size and time limits (one source: `packages/shared/src/world/limits.ts`)
|
|
43
45
|
|
|
44
46
|
Every door enforces the same number and names it in the refusal; the CLI checks the file
|
|
@@ -48,7 +50,9 @@ before it makes a request.
|
|
|
48
50
|
|---|---|---|
|
|
49
51
|
| `sandbox write`, `write_world_sandbox_file`, `env_write_file`, `propose_task_fix` | 256 KB per file | refused, naming the limit; rows → `worlds data import <slug> <file>`; a big source file → `gateway files upload`, then fetch it inside the sandbox with `sandbox exec`; the whole tree → `bench push` |
|
|
50
52
|
| `sandbox exec --timeout`, `exec_in_world_sandbox timeoutMs` | default 60 s, at most 600 s | refused, not clamped |
|
|
51
|
-
| Sandbox tree at `check`/`commit`, `bench push` unpacked | 128 MB of source | refused, naming the limit; move data to `worlds data import`, delete scratch files |
|
|
53
|
+
| Sandbox tree at `check`/`commit`, `bench push` unpacked | 128 MB of source, 2000 files | refused, naming the limit, before a byte is read; move data to `worlds data import`, delete scratch files or list them in `.gatewayignore` |
|
|
54
|
+
| Sandbox tree at `check`/`commit`: credential-shaped files (`.env`, `*.pem`, `id_rsa`, `credentials.json`…; `.env.example` stays), symlinks, sockets/devices | held back, never in the version | listed under `ignored`, each named in `warnings` (path only); use `bench secrets set` for a key the world needs |
|
|
55
|
+
| `worlds data query` / `data show` on a stored version | none beyond the version's own caps | the platform opens the version's tree and snapshot in a prepared directory it owns; no size travels through the request |
|
|
52
56
|
| `bench push` / `worlds create --from` compressed bundle | 64 MB (16 MB from the browser) | refused, naming the limit |
|
|
53
57
|
| A single manifest / metadata file in a bundle | 1 MB | refused |
|
|
54
58
|
| `sandbox write` HTTP body | 1 MB (framework cap; only reachable by a body no schema would accept) | 413 |
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Worlds: gotchas
|
|
2
|
+
|
|
3
|
+
The things that bite once and never again, moved out of `SKILL.md` to keep it under 500 lines.
|
|
4
|
+
|
|
5
|
+
- A shipped template (`schema init ./x --connector salesforce`) scaffolds offline through the
|
|
6
|
+
bundled runtime: no host, no keys, no config file read. Only `--connector workspace:<slug>`
|
|
7
|
+
needs the platform — without `gateway auth login` or the `GATEWAY_*` keys (the agent's
|
|
8
|
+
local-only `run_gateway_cli` included) `schema init` refuses it and names `--host`,
|
|
9
|
+
`GATEWAY_HOST` and the login command. No `python3 >= 3.12`? A shipped template scaffolds
|
|
10
|
+
on the platform instead, so there it does need a host.
|
|
11
|
+
- `bench push` (the TS CLI) on a world with `connector.toml` still packages an api-kind bundle
|
|
12
|
+
on the way up; `worlds create --from ./x` stores the tree as it is, and `worlds migrate`
|
|
13
|
+
turns a pushed bundle back into the tree. Either way, edit the schema world, not the bundle.
|
|
14
|
+
- The World Host is the engine for every world, `[dependencies]` included (it opens each linked
|
|
15
|
+
world beside the root); a session never falls back to a slower engine, and a deployment
|
|
16
|
+
without a host refuses the open by name.
|
|
17
|
+
- Sharing is publish + clone, never a live reference; later edits reach a clone only on re-clone.
|
|
18
|
+
- Never commit `captures/`; never merge a customer's world as an SDK template.
|
|
19
|
+
- A seed (scenario overlay or runner admin endpoint) is the whole state, not an increment.
|
|
20
|
+
- Never set `max_tokens` on any model call used to run or grade.
|
|
21
|
+
- A platform request that never gets an answer fails in one line: the request, the host, the cause
|
|
22
|
+
(`connection refused`, `DNS lookup failed`, `timed out after 120s`) and the next step, exit 1. Each
|
|
23
|
+
request waits `GATEWAY_TIMEOUT_MS` (default 120000); polls (session ready, grades) are many short
|
|
24
|
+
requests, so a long wait is never cut by it, and sandbox requests always allow 660 s.
|
|
25
|
+
|
|
26
|
+
- A world that declares an environment secret (`environment_secret_name` in `release/manifest.json`, or `${NAME:?}` in a task's `docker-compose.yaml`) is refused at `session open`
|
|
27
|
+
with 400 `missing_env_secrets` naming `gateway bench secrets set --env <slug> --key <NAME>` before
|
|
28
|
+
anything is provisioned — the same preflight a run dispatch applies; set the secret, then open.
|
|
29
|
+
- `gateway secret set NAME` is an upsert: the same name again rotates the value and answers
|
|
30
|
+
`updated: true`; `bench secrets set` does the same for a benchmark's environment.
|