@withgateway/sdk 0.3.0 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -1
- package/dist/cli.js +3971 -2338
- package/dist/cli.js.map +1 -1
- package/dist/cli.mjs +3950 -2318
- package/dist/cli.mjs.map +1 -1
- package/dist/files/index.d.mts +1 -1
- package/dist/files/index.d.ts +1 -1
- package/dist/files/index.js +214 -41
- package/dist/files/index.js.map +1 -1
- package/dist/files/index.mjs +206 -35
- package/dist/files/index.mjs.map +1 -1
- package/dist/{index-DAb-FCMg.d.ts → index-DUHmS74p.d.ts} +3 -13
- package/dist/{index-D6oi1TWE.d.mts → index-mJhG3vZU.d.mts} +3 -13
- package/dist/index.d.mts +4 -4
- package/dist/index.d.ts +4 -4
- package/dist/index.js +1456 -564
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +1433 -557
- package/dist/index.mjs.map +1 -1
- package/dist/{media-CbcAfQvd.d.mts → media-BIApiPzS.d.mts} +21 -2
- package/dist/{media-CbcAfQvd.d.ts → media-BIApiPzS.d.ts} +21 -2
- package/dist/traces/index.d.mts +1 -1
- package/dist/traces/index.d.ts +1 -1
- package/dist/traces/index.js +211 -38
- package/dist/traces/index.js.map +1 -1
- package/dist/traces/index.mjs +203 -32
- package/dist/traces/index.mjs.map +1 -1
- package/dist/tracing/index.d.mts +2 -2
- package/dist/tracing/index.d.ts +2 -2
- package/dist/tracing/index.js +96 -21
- package/dist/tracing/index.js.map +1 -1
- package/dist/tracing/index.mjs +96 -21
- package/dist/tracing/index.mjs.map +1 -1
- package/dist/worlds/index.d.mts +300 -123
- package/dist/worlds/index.d.ts +300 -123
- package/dist/worlds/index.js +1385 -508
- package/dist/worlds/index.js.map +1 -1
- package/dist/worlds/index.mjs +1377 -501
- package/dist/worlds/index.mjs.map +1 -1
- package/package.json +1 -1
- package/runtime/gatewaysdk/environment/_bundle.py +2 -27
- package/runtime/gatewaysdk/environment/_materialize.py +49 -69
- package/runtime/gatewaysdk/environment/_world.py +122 -197
- package/runtime/gatewaysdk/environment/backing.py +137 -0
- package/runtime/gatewaysdk/environment/core.py +1 -1
- package/runtime/gatewaysdk/environment/legacy_sqlite.py +301 -0
- package/runtime/gatewaysdk/environment/reward_keys.py +25 -0
- package/runtime/gatewaysdk/environment/runtime.py +8 -13
- package/runtime/gatewaysdk/environment/schema/__init__.py +4 -4
- package/runtime/gatewaysdk/environment/schema/admin_routes.py +86 -0
- package/runtime/gatewaysdk/environment/schema/api.py +266 -515
- package/runtime/gatewaysdk/environment/schema/batch.py +42 -37
- package/runtime/gatewaysdk/environment/schema/bundle_runtime.py +17 -5
- package/runtime/gatewaysdk/environment/schema/call_log.py +291 -0
- package/runtime/gatewaysdk/environment/schema/call_redaction.py +344 -0
- package/runtime/gatewaysdk/environment/schema/cast.py +31 -4
- package/runtime/gatewaysdk/environment/schema/compiler.py +334 -158
- package/runtime/gatewaysdk/environment/schema/conform.py +8 -12
- package/runtime/gatewaysdk/environment/schema/connector.py +27 -62
- package/runtime/gatewaysdk/environment/schema/data.py +50 -207
- package/runtime/gatewaysdk/environment/schema/database/__init__.py +24 -0
- package/runtime/gatewaysdk/environment/schema/database/audit.py +121 -0
- package/runtime/gatewaysdk/environment/schema/database/base.py +995 -0
- package/runtime/gatewaysdk/environment/schema/database/budget.py +33 -0
- package/runtime/gatewaysdk/environment/schema/database/connections.py +101 -0
- package/runtime/gatewaysdk/environment/schema/database/engines.py +8 -0
- package/runtime/gatewaysdk/environment/schema/database/guard.py +208 -0
- package/runtime/gatewaysdk/environment/schema/database/ingest.py +154 -0
- package/runtime/gatewaysdk/environment/schema/database/matching.py +58 -0
- package/runtime/gatewaysdk/environment/schema/database/provenance.py +30 -0
- package/runtime/gatewaysdk/environment/schema/database/rows.py +53 -0
- package/runtime/gatewaysdk/environment/schema/database/runtime_rows.py +39 -0
- package/runtime/gatewaysdk/environment/schema/database/sqlite.py +367 -0
- package/runtime/gatewaysdk/environment/schema/domain_migration.py +236 -0
- package/runtime/gatewaysdk/environment/schema/envelope.py +169 -0
- package/runtime/gatewaysdk/environment/schema/extract.py +6 -13
- package/runtime/gatewaysdk/environment/schema/grading.py +45 -30
- package/runtime/gatewaysdk/environment/schema/handler_errors.py +142 -12
- package/runtime/gatewaysdk/environment/schema/host.py +158 -250
- package/runtime/gatewaysdk/environment/schema/lanes.py +53 -0
- package/runtime/gatewaysdk/environment/schema/local.py +44 -71
- package/runtime/gatewaysdk/environment/schema/mapping.py +5 -25
- package/runtime/gatewaysdk/environment/schema/openapi_import.py +112 -73
- package/runtime/gatewaysdk/environment/schema/openapi_paging.py +147 -0
- package/runtime/gatewaysdk/environment/schema/openapi_payloads.py +288 -0
- package/runtime/gatewaysdk/environment/schema/openapi_routes.py +97 -102
- package/runtime/gatewaysdk/environment/schema/openapi_schemas.py +113 -30
- package/runtime/gatewaysdk/environment/schema/platform.py +124 -178
- package/runtime/gatewaysdk/environment/schema/platform_import.py +2 -3
- package/runtime/gatewaysdk/environment/schema/platform_query.py +61 -5
- package/runtime/gatewaysdk/environment/schema/primary_key.py +61 -0
- package/runtime/gatewaysdk/environment/schema/proof.py +19 -12
- package/runtime/gatewaysdk/environment/schema/runtime.py +77 -160
- package/runtime/gatewaysdk/environment/schema/served_names.py +58 -0
- package/runtime/gatewaysdk/environment/schema/session_files.py +166 -0
- package/runtime/gatewaysdk/environment/schema/skills/benchmarks-getting-started/SKILL.md +2 -2
- package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +22 -14
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +29 -3
- package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +16 -16
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +18 -18
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/cli-mcp-map.md +3 -2
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/gotchas.md +15 -0
- package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/references/openapi-import.md +50 -21
- package/runtime/gatewaysdk/environment/schema/slack.py +4 -8
- package/runtime/gatewaysdk/environment/schema/snapshot.py +385 -129
- package/runtime/gatewaysdk/environment/schema/store_contract.py +8 -14
- package/runtime/gatewaysdk/environment/schema/tasks.py +140 -55
- package/runtime/gatewaysdk/environment/schema/tools_world.py +110 -87
- package/runtime/gatewaysdk/environment/schema/ui.py +24 -5
- package/runtime/gatewaysdk/environment/schema/ui_static/app.js +100 -130
- package/runtime/gatewaysdk/environment/schema/ui_static/index.html +1 -8
- package/runtime/gatewaysdk/environment/schema/validation.py +166 -40
- package/runtime/gatewaysdk/environment/schema/workers.py +142 -57
- package/runtime/gatewaysdk/environment/schema/world_sql.py +81 -0
- package/runtime/gatewaysdk/environment/schema/world_tests.py +71 -39
- package/runtime/gatewaysdk/environment/score.py +57 -0
- package/runtime/gatewaysdk/environment/secret_values.py +36 -0
- package/runtime/gatewaysdk/environment/world.py +42 -3
- package/runtime/gatewaysdk/environment/world_session_contract.py +32 -5
- package/skills/benchmarks-getting-started/SKILL.md +2 -2
- package/skills/connector-schema-authoring/SKILL.md +22 -14
- package/skills/connector-schema-authoring/connector-worlds.md +26 -10
- package/skills/connector-schema-authoring/world-schema.md +111 -51
- package/skills/world-data-ingestion/ROWS.md +29 -3
- package/skills/world-data-ingestion/SKILL.md +16 -16
- package/skills/worlds-getting-started/SKILL.md +18 -18
- package/skills/worlds-getting-started/references/cli-mcp-map.md +3 -2
- package/skills/worlds-getting-started/references/gotchas.md +15 -0
- package/skills/worlds-getting-started/references/openapi-import.md +50 -21
- package/runtime/gatewaysdk/environment/schema/store.py +0 -1013
|
@@ -292,9 +292,10 @@ A schema world's validated database is kept, not rebuilt. `gateway worlds schema
|
|
|
292
292
|
builds `db/state.sqlite` (the default snapshot: `data/initial.json` applied through the
|
|
293
293
|
contract) and `scenarios/<name>/state.sqlite` (the default with the scenario's overlays),
|
|
294
294
|
each with a `state.json` sidecar `{contractHash, batchDigest, entityCounts, builtAt,
|
|
295
|
-
runtime, fileSha256}`; the same
|
|
296
|
-
and a data batch the contract refuses leaves no
|
|
297
|
-
|
|
295
|
+
runtime, fileSha256}`; a snapshot that already holds the same data is never rebuilt,
|
|
296
|
+
`schema check` verifies the sidecars, and a data batch the contract refuses leaves no
|
|
297
|
+
snapshot behind. `gateway worlds serve` and
|
|
298
|
+
the api bundle open the snapshot (copy + `WorldDatabase.open`, no row replay) and reset by
|
|
298
299
|
copying it again; without one the JSON is replayed as before. A hosted pod gets
|
|
299
300
|
`WORLD_SNAPSHOT_URL` (presigned GET, `.gz` or `Content-Encoding: gzip` is gunzipped),
|
|
300
301
|
`WORLD_SNAPSHOT_SHA256` and `WORLD_SNAPSHOT_DIGEST`; the server downloads, verifies, writes
|
|
@@ -303,3 +304,28 @@ reports `"snapshot": {batchDigest, contractHash}` (or null) and `/__world/seed`
|
|
|
303
304
|
overlay form `{"base": digest, "rows": {...}, "mode": append|replace}` beside the legacy
|
|
304
305
|
whole-state body. Snapshot files are build outputs: gitignore `db/state.sqlite`,
|
|
305
306
|
`db/state.json`, `scenarios/*/state.*`.
|
|
307
|
+
|
|
308
|
+
**Which edits keep a snapshot.** Rows are bound to the domain they were stored under, not to
|
|
309
|
+
the whole contract: adding a tool, editing a handler, a route, a test, a documentation file or
|
|
310
|
+
the package version keeps every snapshot (local, hosted, pinned), with no re-import. A change
|
|
311
|
+
that only adds to the domain (a new entity, a new field that is not required, another
|
|
312
|
+
`domain_version`) carries the rows: compile migrates the snapshots the tree holds and keeps the
|
|
313
|
+
pin with the stored domain's `lock` and `entities`, `sandbox check` says the commit migrates the
|
|
314
|
+
hosted snapshot, and `sandbox commit` migrates it and re-pins the tree (a CLI push of such a
|
|
315
|
+
change is refused, naming the sandbox). Any other change to an entity, a field (its description
|
|
316
|
+
or title too), a key or a relationship refuses it, naming each change:
|
|
317
|
+
|
|
318
|
+
```text
|
|
319
|
+
snapshot_pinned at /db/snapshot.json: The rows here were stored under the previous domain, and
|
|
320
|
+
this change requires field users.age. Stored rows carry over only when a change adds to their
|
|
321
|
+
domain (a new entity, a new field that is not required, or another domain_version): ...
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
Then keep those as they were, or remove `db/snapshot.json` and import every row again. A tree
|
|
325
|
+
pinned to a hosted snapshot (`db/snapshot.json`, what a data import publishes) compiles and checks
|
|
326
|
+
without the snapshot itself, as in a sandbox: compile keeps the pin, never rebuilds state from
|
|
327
|
+
the data files, and records in the pin (`lock`) the contract the snapshot was built under, which
|
|
328
|
+
is how the platform proves the next edit still fits it. If `schema/lock.json` was replaced by
|
|
329
|
+
artifacts compiled elsewhere before that, compile says the domain cannot be proven: restore
|
|
330
|
+
`schema/lock.json` from the version you started from and compile again. A `[redact]` change is
|
|
331
|
+
not part of it: the store never applies the policy; capture and `--redact apply` do.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: world-data-ingestion
|
|
3
|
-
description: Put rows into a world from any agent. Use when uploading data to a world, seeding a world, importing rows (a few or millions), adding records to a schema world or mock-tools world, pulling a connector's data through an active connection (`gateway worlds data pull --connection`), seeding a live session, exporting a session's calls, turning a vendor export or a real run into world data, turning an agent's real tool calls, a trace's TOOL observations or a saved ClickHouse result into rows (`gateway worlds data calls pull`, then `data ingest --transform` with your own code or `--map ingest.toml`, `captureToolCalls`), or asking why a served world starts from a snapshot. Covers the rows file contract (JSON object, JSONL, --entity), the tool-call record contract, the transform contract, the `ingest.toml` grammar and `map init`, append vs replace, primary-key upsert, `gateway worlds describe`, `gateway worlds data show|import|pull|check|extract|calls pull|ingest|status|batches|publish`, chunked resumable uploads, refusal reports, data snapshots (`db/state.sqlite`, `WORLD_SNAPSHOT_*`), `gateway worlds session seed|export`, the MCP twins import_world_data / pull_world_data / seed_world_session / export_world_session, the refusals, and the definition of done.
|
|
3
|
+
description: Put rows into a world from any agent. Use when uploading data to a world, seeding a world, importing rows (a few or millions), adding records to a schema world or mock-tools world, pulling a connector's data through an active connection (`gateway worlds data pull --connection`), seeding a live session, exporting a session's calls, turning a vendor export or a real run into world data, turning an agent's real tool calls, a trace's TOOL observations or a saved ClickHouse result into rows (`gateway worlds data calls pull`, then `data ingest --transform` with your own code or `--map ingest.toml`, `captureToolCalls`), or asking why a served world starts from a snapshot. Covers the rows file contract (JSON object, JSONL, --entity), the tool-call record contract, the transform contract, the `ingest.toml` grammar and `map init`, append vs replace, primary-key upsert, `gateway worlds describe`, `gateway worlds data show|import|sql|pull|check|extract|calls pull|ingest|status|batches|publish`, chunked resumable uploads, refusal reports, data snapshots (`db/state.sqlite`, `WORLD_SNAPSHOT_*`), `gateway worlds session seed|export`, the MCP twins import_world_data / pull_world_data / sql_world_data / seed_world_session / export_world_session, the refusals, and the definition of done.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# World data ingestion
|
|
@@ -71,8 +71,7 @@ Rules:
|
|
|
71
71
|
(`--atomic`, the default); `--per-chunk` keeps the good chunks of a large upload and lists
|
|
72
72
|
the refused ones.
|
|
73
73
|
- Inside one batch, two rows with the same key merge by `--merge`: `upsert` (default, the
|
|
74
|
-
later row replaces the earlier) or `fill` (the first row wins, later rows only add fields
|
|
75
|
-
it lacks).
|
|
74
|
+
later row replaces the earlier) or `fill` (the first row wins, later rows only add fields it lacks).
|
|
76
75
|
|
|
77
76
|
## Decision table by world kind
|
|
78
77
|
|
|
@@ -95,7 +94,7 @@ Every CLI step has its MCP twin on the next line. `<WORLD>` is a local world dir
|
|
|
95
94
|
platform world slug (the CLI pulls it to a temp dir, imports, and pushes a new version unless
|
|
96
95
|
`--no-push`). Auth and host: `GATEWAY_HOST`, `GATEWAY_PUBLIC_KEY`, `GATEWAY_SECRET_KEY`, or
|
|
97
96
|
`gateway auth login`. Third place: a CLI sandbox (`worlds sandbox open <slug>@main`) has `gateway` + `sqlite3` on PATH, so
|
|
98
|
-
`sandbox exec <id> 'gateway worlds data import . rows.json'` (`show`, `query`) runs on the uncommitted copy with nothing local; `check` then `commit` lands it. Same refusals as a directory (exit 2).
|
|
97
|
+
`sandbox exec <id> 'gateway worlds data import . rows.json'` (`show`, `query`, `sql`) runs on the uncommitted copy with nothing local; `check` then `commit` lands it. Same refusals as a directory (exit 2).
|
|
99
98
|
The sandbox holds **no platform credentials, by design**: never write keys into its tree (`.gateway-env`, `*.env` are held out of every
|
|
100
99
|
commit, with a warning); bulk rows reach the platform world through `worlds sandbox import <id> <path>` from where the keys are (2d).
|
|
101
100
|
|
|
@@ -108,8 +107,13 @@ gateway worlds data show <WORLD> # {world, kind, entity_count
|
|
|
108
107
|
gateway worlds data show <WORLD> --entity users # the rows of one entity
|
|
109
108
|
gateway worlds data query <WORLD> users --where plan=pro --limit 50 # the rows you mean; a repeated --where key is IN
|
|
110
109
|
gateway worlds data query <WORLD> users --where email=jo@acme.test --resolve # plaintext into a hashed field
|
|
110
|
+
gateway worlds data sql <WORLD> 'SELECT id, name FROM users LIMIT 5' # read-only SQL, any engine
|
|
111
111
|
```
|
|
112
112
|
|
|
113
|
+
`data sql` runs ONE read-only statement (joins, `GROUP BY`, counts) over a schema world's entity tables,
|
|
114
|
+
whatever its database engine: a write, `ATTACH`, `PRAGMA` or a second statement is refused; at most 1000 rows
|
|
115
|
+
(`--limit`). MCP twin: `sql_world_data({ slug, sql, limit? })`. A db/schema.sql world is a SQLite file: `sqlite3`.
|
|
116
|
+
|
|
113
117
|
`data query --resolve` turns each `--where` value into the form the store holds under the
|
|
114
118
|
`[redact]` policy before matching, and prints `resolved <field> -> <form>`; a dropped field
|
|
115
119
|
resolves to nothing, so zero rows there is the policy, not a missing import. MCP twin:
|
|
@@ -269,9 +273,8 @@ level}]}`; text before the JSON is skipped; `input`/`output` decoded as JSON or
|
|
|
269
273
|
```
|
|
270
274
|
|
|
271
275
|
A `gateway worlds session export` line (`{seq, kind, tool, args, result, error, completedAt}`)
|
|
272
|
-
is accepted as it is: `kind != "call"` and a failed call are skipped and counted, never
|
|
273
|
-
silently dropped. `captureToolCalls` (TS) / `capture_tool_calls` (Python) write the same
|
|
274
|
-
record at the source.
|
|
276
|
+
is accepted as it is: `kind != "call"` (a grade, or a `--requests` line of kind `request`) and a failed call are skipped and counted, never
|
|
277
|
+
silently dropped. `captureToolCalls` (TS) / `capture_tool_calls` (Python) write the same record at the source.
|
|
275
278
|
|
|
276
279
|
**2. Transform.** `--transform <script>`: a `.py` file, or `.ts`/`.mjs`/`.js` for node, that
|
|
277
280
|
exports `transform(records) -> {entity: [rows]}` (node: `transform` or the default export),
|
|
@@ -380,26 +383,23 @@ state}` to poll with `data status`. A new-world sandbox commits once first. MCP:
|
|
|
380
383
|
|
|
381
384
|
### 3. Verify on a live session
|
|
382
385
|
|
|
383
|
-
Open a session, seed it, hit it. Live seeding does not change the world's version; it
|
|
384
|
-
changes this session only.
|
|
386
|
+
Open a session, seed it, hit it. Live seeding does not change the world's version; it changes this session only.
|
|
385
387
|
|
|
386
388
|
```bash
|
|
387
389
|
gateway worlds session open <slug> -t <scenario> --surface api # prints sessionId and api.url
|
|
388
390
|
gateway worlds session seed <SESSION_ID> rows.json # kind:"seed" call, waits for the result
|
|
389
391
|
gateway worlds session seed <SESSION_ID> rows.json --redact refuse # same, refusing plaintext in a redacted field (apply hashes it first)
|
|
390
392
|
gateway worlds session seed <SESSION_ID> rows.json --mode replace # swap the named entities
|
|
391
|
-
curl -H
|
|
393
|
+
curl -H "Authorization: Bearer $API_TOKEN" "$API_URL/<route>" # read it back (api.token from session open)
|
|
392
394
|
```
|
|
393
395
|
|
|
394
396
|
Prints `{sessionId, mode, result}`. MCP: `seed_world_session({ sessionId, rows, mode? })` waits
|
|
395
397
|
up to 60 seconds for the engine and otherwise returns a `callId` to poll. Same caps as the
|
|
396
|
-
import. Public API: `POST /api/public/world-sessions/{sessionId}/calls` with
|
|
397
|
-
`{ kind: "seed", rows, mode }`.
|
|
398
|
+
import. Public API: `POST /api/public/world-sessions/{sessionId}/calls` with `{ kind: "seed", rows, mode }`.
|
|
398
399
|
|
|
399
400
|
The tools surface is open by default; `--surface api` adds the vendor's HTTP routes. A fresh
|
|
400
401
|
version's first session takes about fifteen minutes to come up (`PROVISIONING`, 503 on the
|
|
401
|
-
URL); that is a cold pod, not a fault. Pass `--no-wait` and poll
|
|
402
|
-
`gateway worlds session status <SESSION_ID>`.
|
|
402
|
+
URL); that is a cold pod, not a fault. Pass `--no-wait` and poll `gateway worlds session status <SESSION_ID>`.
|
|
403
403
|
|
|
404
404
|
Which kinds can seed live: schema worlds (the store applies the batch through the contract),
|
|
405
405
|
api worlds (the runner merges the rows into the whole state and posts it to the world's seed
|
|
@@ -478,8 +478,8 @@ Sanitize before importing: hash or drop personal fields the world's `[redact]` p
|
|
|
478
478
|
|
|
479
479
|
## Data snapshots
|
|
480
480
|
|
|
481
|
-
Why a served world starts in under a second (`db/state.sqlite`, `WORLD_SNAPSHOT_*`,
|
|
482
|
-
|
|
481
|
+
Why a served world starts in under a second (`db/state.sqlite`, `WORLD_SNAPSHOT_*`, `/__world/seed`), and why a tool,
|
|
482
|
+
handler or route edit keeps the imported data while an entity, field or key edit does not: [ROWS.md — Data snapshots](ROWS.md#data-snapshots).
|
|
483
483
|
|
|
484
484
|
## Definition of done
|
|
485
485
|
|
|
@@ -134,7 +134,7 @@ gateway worlds connector route add ./x create_account -X POST --path /v1/account
|
|
|
134
134
|
|
|
135
135
|
Each refuses (and writes nothing) on a duplicate name, an undeclared entity or `$defs` reference, a key that is not a
|
|
136
136
|
required string/integer field, or a filter on an unknown argument. A create route mints its single-field primary key (`uuid` for strings, `int` for integers) unless `--body` maps it;
|
|
137
|
-
`--generate field=uuid|int|now` and `--default field=value` fill the rest. A custom world's first `route add` creates `connector.toml
|
|
137
|
+
`--generate field=uuid|int|now` and `--default field=value` fill the rest. A custom world's first `route add` creates `connector.toml` with `[auth] scheme = "bearer"` (a hosted session's api token; `worlds serve` answers 401 without one); `--action handler --handler fn` writes a 501 stub `fn(ctx, args, query, body)` into `api/handlers.py` unless it is defined. The custom scaffold's example entity is `records`: `gateway worlds schema entity remove ./x records` once yours are declared (refused, naming each reference, while a route, verifier, row, dependency map or relationship uses it). `schema init` prints `kind: "schema"`, `api` and the `next` commands in order (custom: entity add → entity remove records → route add → task add → compile → check → import → serve → `worlds create`).
|
|
138
138
|
Manifest `kind` is `schema` (older trees: `mock-tools`). Handlers: `def name(ctx, args, query, body)`; `ctx` reads and writes the store and keeps the clock.
|
|
139
139
|
|
|
140
140
|
### From an OpenAPI spec (Swagger 2.0, OpenAPI 3.x)
|
|
@@ -168,7 +168,7 @@ with this list, and a hosted call into such a handler answers 400 `{error: "hand
|
|
|
168
168
|
- `lineage(entity: str, key: dict) -> list[dict]` — the batches that wrote the row at `key`, oldest first.
|
|
169
169
|
- `counts() -> dict[str, int]` — rows per entity, `{entity: count}`.
|
|
170
170
|
|
|
171
|
-
No `upsert`: `get`, then `insert` or `update`. Read-only SQL: `gateway_env.db.one/all`; raw SQL writes are refused. The scaffolded `tools/handlers.py` docstring carries the same table.
|
|
171
|
+
No `upsert`: `get`, then `insert` or `update`. Read-only SQL: `gateway_env.db.one/all`; raw SQL writes are refused. The scaffolded `tools/handlers.py` docstring carries the same table. Every tool call and every route call (handler routes too) is one transaction: one that raises, breaks the contract or answers 4xx/5xx keeps no writes or `ctx.emit` events and is still logged with its status. A route's handler answers like a tool's on every door, an answer JSON cannot carry (a `datetime`, a set, NaN) is its 400, and a route answering bytes (a `404 text/plain`) comes back through `call` as `{status, body}` with the body as text.
|
|
172
172
|
|
|
173
173
|
## 3. Check
|
|
174
174
|
|
|
@@ -181,14 +181,14 @@ gateway worlds serve ./x --port 8080 # answers the vendor
|
|
|
181
181
|
```
|
|
182
182
|
|
|
183
183
|
`check` validates the compiled bundle; `conform` drives every declared route against the
|
|
184
|
-
vendor's document and exits non-zero on a mismatch. Its FAIL list is the work list.
|
|
184
|
+
vendor's document and exits non-zero on a mismatch. Its FAIL list is the work list. Routes are never tools: an agent reaches a world's routes over HTTP (the api url and token `session open --surface api` prints, or `worlds serve` and a request to `http://127.0.0.1:<port>/<path>`), and gets a tool only where `x-gateway.tools` declares one with a handler. A call by a route's name is refused (`not_a_tool`, naming the api URL). A task's `tools` lists declared tools only, and a task that lists any closes the session's routes (403 `route_not_allowed`). An api bundle serves no tools, so the benchmark runner refuses to run one: `gateway worlds migrate <slug> --apply` turns it into a schema world.
|
|
185
185
|
|
|
186
186
|
MCP: `validate_connector_template({ files, slug? })` compiles without writing; answers `{ valid, reason? }`.
|
|
187
187
|
|
|
188
188
|
## 4. Data
|
|
189
189
|
|
|
190
190
|
```bash
|
|
191
|
-
gateway worlds describe ./x # entities, keys, relationships, row counts, [redact] per field
|
|
191
|
+
gateway worlds describe ./x # entities, keys, relationships, row counts, enum values, [redact] per field
|
|
192
192
|
gateway worlds data show ./x # kind + entity counts
|
|
193
193
|
gateway worlds data query ./x users --where email=jo@acme.test --resolve # the rows you mean; plaintext into a hashed field
|
|
194
194
|
gateway worlds data import ./x rows.json # rows file: {entity: [rows]} | JSONL | .csv/.tsv --entity (cells cast by the schema; --delimiter)
|
|
@@ -314,14 +314,14 @@ gateway worlds session open x -t default # prints {sessionId,
|
|
|
314
314
|
gateway worlds session open x -t default --surface api # asks for api.url explicitly: the vendor's routes, hosted (400 on a tree with no connector.toml)
|
|
315
315
|
gateway worlds session open x -t default --surface ui,browser # + ui.entry (the page, token once then cookie) and browser.wsUrl (a pool Chromium); 400 without [ui] or without a pool; --browser-tier standard (spot, may be reclaimed; the SDK repairs + reconnects, raises BrowserReconnected on a lost click/type) | premium (on-demand), default: the org's setting
|
|
316
316
|
gateway worlds session browse <SESSION_ID> [--path /#/accounts] [--screenshot p.png] # open the page in the browser surface; prints url, title, text
|
|
317
|
-
gateway worlds session open x [--no-wait]
|
|
317
|
+
gateway worlds session open x [--no-wait|--timeout S] # EXPLORATORY: no task, no grader (task.source "none"); waits up to S seconds (600) for ready, --no-wait not at all
|
|
318
318
|
gateway worlds session open x --task-file task.json --surface api # the session's own task (below)
|
|
319
319
|
gateway worlds session status <SESSION_ID> # poll until ready
|
|
320
320
|
gateway worlds session seed <SESSION_ID> rows.json # rows into this session only
|
|
321
321
|
gateway worlds session state <SESSION_ID> # every entity's rows, the dump a grader reads
|
|
322
322
|
gateway worlds session reset <SESSION_ID> # back to the snapshot (+ the task's seed)
|
|
323
323
|
gateway worlds session grade <SESSION_ID> [--timeout S] # {reward, rewards, raw}; exploratory → {reward: null, reason: "no grader"}
|
|
324
|
-
gateway worlds session export <SESSION_ID> --out calls.jsonl
|
|
324
|
+
gateway worlds session export <SESSION_ID> --out calls.jsonl [--requests] # the platform's ledger; --requests (live session, one state call) adds the world's own request log, kind "request" with via
|
|
325
325
|
gateway worlds data ingest x calls.jsonl --transform shape.py # the calls back in as rows, shaped by your code or --map ingest.toml (world-data-ingestion §2c)
|
|
326
326
|
gateway worlds session close <SESSION_ID> # a closed session reopens by id, same URLs + token, from its last checkpoint: gateway worlds session reopen <SESSION_ID>
|
|
327
327
|
```
|
|
@@ -344,7 +344,7 @@ verifiers) is refused `world_kind_retired`; one on an api bundle opens and lists
|
|
|
344
344
|
`instruction` is what the agent is told; `seed` (`{mode?, rows}`, the live-seed contract) is laid on the clone and re-applied by `reset`;
|
|
345
345
|
`grader` is `assertions` (`entity`, `where`, `exists|absent|count|all|any`, `field op value`; each check's `id` becomes a `rewards` key;
|
|
346
346
|
`entity: "calls"` reads the call log — `operation`, `args`, `selectors`, `status`, `rows`, `found`, and `via`: `ui` from the page's `api_path`,
|
|
347
|
-
`api` from the api origin, `tool`
|
|
347
|
+
`api` from the api origin, `tool` for a declared tool's call; a credential-named or secret-shaped field reads `[redacted]`; `rows`/`found` are null where the answer declares no rows, a request too large to keep names what it lacks in `withheld`, and a check grades whenever the calls it can read decide it, ungraded only where those could change the result), `{kind: "python", source, timeoutSeconds?}` (a `verify.py` body; it reads the state as JSON on stdin — `{"<entity>": {"<primary key>": row}, "calls": [...], "report": "<the agent's report>"}`, a verifier's entities as `<alias>.<entity>` — and prints ONE flat JSON object as the last line of stdout, `reward` plus a number from 0 to 1 per check — `true`/`false` read as 1/0, `null` or a string as ungraded, no `reward` leaves the task ungraded, and a nested object or array (a `rewards` map included), a number outside 0..1 or NaN/Infinity fails the grade by key and value; it must exit 0 within `timeoutSeconds`, default and most 30, or the grade fails with the reason) or `{kind: "rubric", rubric, criteria?, model?}`
|
|
348
348
|
(an LLM judge over end state + call log, graded on the platform: `grade` answers pending until
|
|
349
349
|
the worker finalizes it); `tools: [names]` narrows the operations the session may call to
|
|
350
350
|
exactly those (a name the world does not serve is a 400); `metadata` rides the session. At most one of
|
|
@@ -352,7 +352,7 @@ exactly those (a name the world does not serve is a 400); `metadata` rides the s
|
|
|
352
352
|
`task` block is `{source: "inline", name, hash, metadata}`. SDK: `openSession(slug, { task: spec })` / `openSession(slug)` (TS), `open_session(slug, spec)` / `open_session(slug)` (Python);
|
|
353
353
|
`runSessions` / `run_sessions` take a mixed list of names and specs. Over several worlds, `worlds: {alias: {slug, …}}` opens one session per alias
|
|
354
354
|
(`task up|grade|export|down`, `open_task`/`openTask`); a question spanning them is a `verifiers[]` entry (`name`, `scope`, assertions|python over
|
|
355
|
-
`<alias>.<entity>` / `<alias>.calls`), graded on the platform only, rewards `x.<name
|
|
355
|
+
`<alias>.<entity>` / `<alias>.calls`), graded on the platform only, rewards `x.<name>`, so no world is aliased `x` (worlds/sessions.mdx).
|
|
356
356
|
|
|
357
357
|
**A stored task.** The same spec kept as a project resource, versioned by content, so many
|
|
358
358
|
sessions open it by id and grades attribute to the task and its version:
|
|
@@ -391,15 +391,15 @@ gateway bench dispatch <RUN_CONFIG_ID> # each stored task
|
|
|
391
391
|
The first session on a freshly pushed version takes about fifteen minutes: the pod is
|
|
392
392
|
scheduled and the runner sidecar pulled; the session reads `PROVISIONING` and the api URL
|
|
393
393
|
answers 503. That is a cold start, not a fault, until about twenty minutes have passed.
|
|
394
|
-
`session open` prints `session <id> opening — gateway worlds session status <id>` on stderr at once, then waits
|
|
394
|
+
`session open` prints `session <id> opening — gateway worlds session status <id>` on stderr at once while the World Host opens the session, then waits for ready up to `--timeout` seconds (600): a failed open exits 1 naming the reason and code (`browser_capacity` and `host_unavailable` are worth retrying), and a wait that runs out prints `ready: false`, exits 1 and leaves it opening; on a cold version use `--no-wait` and poll. A call on a session still opening waits up to 30 s, then 409 `session_starting`. `open` and
|
|
395
395
|
`status` print `idleExpiresAt` (null once pinned) + `pinned`; `status` warns under 5 minutes. A call
|
|
396
396
|
body over 4mb is refused client-side: `{status: "error", code: "payload_too_large"}`, exit 2.
|
|
397
397
|
|
|
398
|
-
The world's API honours the vendor's auth scheme
|
|
399
|
-
|
|
398
|
+
The world's API honours the vendor's auth scheme and accepts the session's own api token (bearer or
|
|
399
|
+
the vendor's header); `worlds serve` accepts the `--api-key` pin (default `conform`).
|
|
400
400
|
|
|
401
401
|
MCP: `seed_world_session`, `export_world_session`, `save_world_session_task`; open/call/close are CLI or REST only.
|
|
402
|
-
Find what you opened: `gateway worlds session list` / `sandbox list`; `session pin <id>` stops the reaper.
|
|
402
|
+
Find what you opened: `gateway worlds session list` (every state; the live ones are `--state RUNNING` and `--state IDLE`) / `sandbox list`; `session pin <id>` stops the reaper.
|
|
403
403
|
|
|
404
404
|
## Test the world (not the agent)
|
|
405
405
|
|
|
@@ -412,10 +412,10 @@ gateway worlds test my-world # the same suite on the platform,
|
|
|
412
412
|
gateway bench tests my-world # every run the platform kept (push, CLI, API)
|
|
413
413
|
```
|
|
414
414
|
|
|
415
|
-
A case: `{"name": "...", "request": {"method": "GET", "path": "/v1/accounts", "
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
415
|
+
A case: `{"name": "...", "request": {"method": "GET", "path": "/v1/accounts"}, "expect": {"status": 200}}`.
|
|
416
|
+
A `test_*.py` gets `WORLD_URL`, `WORLD_API_KEY` and, with `[ui]`, `WORLD_UI_URL` (the page on its own origin; the
|
|
417
|
+
platform's runner has no browser, so check the page over HTTP there). `[actions]` `on_push = ["tests"]` in
|
|
418
|
+
`gateway-env.toml` runs the suite on each push (Tests tab); `POST/GET /api/public/worlds/{slug}/tests`; `documentation/features/world-tests.md`.
|
|
419
419
|
|
|
420
420
|
## Act on a world as an outside agent (local)
|
|
421
421
|
|
|
@@ -438,7 +438,7 @@ its `api.url` takes the session's bearer): act on it by id, `gateway worlds call
|
|
|
438
438
|
## 7. Run and grade
|
|
439
439
|
|
|
440
440
|
```bash
|
|
441
|
-
gateway worlds run x -a agent.mjs --all # TS CLI: agent
|
|
441
|
+
gateway worlds run x -a agent.mjs --all # TS CLI: agent.mjs[#export] (the default export unless #name) async (session, task) => void, every task; -t <task> for one; x is a slug[@ref] or a pushed dir
|
|
442
442
|
gateway worlds run x -a agent.py:agent # Python CLI: agent callable module:function; --path ./x runs a local dir, --no-file grades without filing
|
|
443
443
|
gateway worlds ci ./x -a agent.mjs --min-score 0.8 # push the working tree, run, gate (CI); --run-config <file> instead of -a; Python: --path ./x -a agent.py:agent
|
|
444
444
|
gateway environment run x -m <provider/model> -k 3 # hosted runner, one job per model; -k is 1–64
|
|
@@ -474,7 +474,7 @@ gateway worlds sandbox import <id> data/trips.csv --entity trip # rows a sandbo
|
|
|
474
474
|
```
|
|
475
475
|
|
|
476
476
|
The sandbox carries the platform's `gateway` CLI and `sqlite3` on PATH: `exec` runs `worlds
|
|
477
|
-
schema|data|test|serve|call …` on the working copy with nothing on your machine. `open` prints
|
|
477
|
+
schema|data|test|serve|call …` on the working copy with nothing on your machine; on a world whose data is a hosted snapshot, a tool, handler or route edit compiles and keeps that data, an entity or key edit is refused (`references/gotchas.md`). `open` prints
|
|
478
478
|
`cli: {version, digest}`, or `cli: null` plus the reason under `warnings`. Remote verbs (`push`, `run`,
|
|
479
479
|
`session open`) stop inside the sandbox by design: it holds no platform credential, and never write one
|
|
480
480
|
into its tree — `check`/`commit` hold back scratch, credential-shaped files (`.env`, `.gateway-env`, `*.env`,
|
|
@@ -17,6 +17,7 @@ Reference for `worlds-getting-started`: every CLI step and its MCP twin, then ho
|
|
|
17
17
|
| `worlds migrate <slug> [--apply] [--force]` / `--all` | `migrate_world` |
|
|
18
18
|
| `bench push` | `env_write_file` + `env_commit`, or `create_environment_version` (one file) |
|
|
19
19
|
| `worlds data import` | `import_world_data` |
|
|
20
|
+
| `worlds data sql <world> '<sql>' [--limit] [--scenario] [--json]` | `sql_world_data` (`POST /api/public/worlds/{slug}/data/sql`; ONE read-only statement over a schema world's rows, whatever its engine; a write, `ATTACH`, `PRAGMA` or a second statement is refused; at most 1000 rows. A db/schema.sql world is a SQLite file: `sqlite3` in a sandbox) |
|
|
20
21
|
| `worlds session open [-t\|--task-id\|--task-file]` (none: exploratory) / `status` / `call` / `close` / `list` / `pin` | none on the public MCP server — CLI or REST only (`/api/public/world-sessions[/{id}[/calls]]`; the open answers `idleExpiresAt` + `pinned`) |
|
|
21
22
|
| `worlds session seed` | `seed_world_session` |
|
|
22
23
|
| `worlds session export` | `export_world_session` |
|
|
@@ -37,7 +38,7 @@ Reference for `worlds-getting-started`: every CLI step and its MCP twin, then ho
|
|
|
37
38
|
| `bench diff` / `propose` | `env_diff`, `env_propose` (proposals are the legacy PR-style door; edit sessions below are the version-control loop) |
|
|
38
39
|
| `worlds\|bench edit status\|list\|close\|discard\|rollback <slug>` | `list_world_edits({ slug, status?, limit? })`, `close_world_edit({ slug, sessionId?, force?, message? })`, `discard_world_edit`, `rollback_world_edit({ slug, sessionId, force?, message? })` (`GET /api/public/worlds/{slug}/edits`, `POST …/edits/close\|discard\|rollback`). After a world's first version every push, sandbox commit, data import, agent `env_commit` and UI save lands on the credential's open edit session — a private `edit/<id>` branch only that credential reads as `main` (`describe`, `pull`, `log`, `session open`, `sandbox open`); the write's JSON carries `editSessionId` + a hint. `close`, or 60 minutes without an edit, seals the session: the shared `main` moves to its tip (every version stays in `log`; the session is the rollback unit); a `main` that moved meanwhile is rebased when the paths differ, else the session parks as `conflict` (`close --force` takes its content). `rollback` restores source + data snapshot from before a sealed session as a new sealed session (`--force` when later sessions would be undone too). UI: world › History › Edit sessions |
|
|
39
40
|
| `worlds task edit status\|close\|rollback <task>` (task id, spec file with `worlds`, or `task up` manifest) | `get_task_edit({ task })`, `close_task_edit({ task, requireAll?, force?, message? })`, `rollback_task_edit({ task, groupId, force?, message? })` (`POST /api/public/world-task-edits/status\|close\|rollback`). The same edit sessions across every world a task names: `status` shows each world's unsealed edits (open and `parked` in conflict) and what closing would do (`fast-forward`, `rebase`, `conflict`) — your own `task up` opens them at your tip (`private: true`), everyone else still gets shared `main`; `close` seals every world in ONE transaction (a conflict, a missing world, a ref other than `main` or a parked session anywhere seals nothing, and the 409 names every world's problem; worlds with no edit are skipped unless `--require-all`), and the sessions share a `groupId`; `rollback --group` restores every world together to the `main` the group replaced, refused when anything reached a world's `main` after the group (a later session or a write outside sessions) unless `--force` |
|
|
40
|
-
| `worlds\|bench sandbox open\|status\|read\|write\|exec\|check\|commit\|close` | `open_world_sandbox`, `get_world_sandbox`, `read_world_sandbox_file`, `write_world_sandbox_file` (base64), `exec_in_world_sandbox`, `check_world_sandbox`, `commit_world_sandbox`, `close_world_sandbox({ sandboxId, force? })` (`/api/public/benchmark-containers/sandboxes[/{id}[/file\|exec\|check\|commit]]`; `close` with uncommitted work is a 409 `sandbox_uncommitted` naming the paths, nothing closed, unless `force: true` / `--force`; the source tree only — no data snapshot, nothing on the caller's disk; `commit` lands on the recorded base; `commit --rebase` sends `rebase: true` and replays onto a branch head that moved, a 409 naming each conflicting path). `open` reports `cli: {version, digest}` — the platform's own `gateway` CLI and a `sqlite3` are on PATH inside `exec`, so `gateway worlds schema entity add\|verifier add\|task add\|compile\|check .`, `gateway worlds data import\|show\|query .`, `gateway worlds test .`, `gateway worlds serve . --port 8080 & sleep 2; gateway worlds call http://localhost:8080 <tool> …` (one `exec`; a served world lives only as long as that command) and `sqlite3 db/state.sqlite "…"` run on the working copy; remote verbs (push, run, session open) stop in the sandbox by design — `check`/`commit` from your machine. `cli: null` + `warnings[]` means the image cannot host the runtime (python3 < 3.12 or node < 18); plain shell still runs |
|
|
41
|
+
| `worlds\|bench sandbox open\|status\|read\|write\|exec\|check\|commit\|close` | `open_world_sandbox`, `get_world_sandbox`, `read_world_sandbox_file`, `write_world_sandbox_file` (base64), `exec_in_world_sandbox`, `check_world_sandbox`, `commit_world_sandbox`, `close_world_sandbox({ sandboxId, force? })` (`/api/public/benchmark-containers/sandboxes[/{id}[/file\|exec\|check\|commit]]`; `close` with uncommitted work is a 409 `sandbox_uncommitted` naming the paths, nothing closed, unless `force: true` / `--force`; a sandbox whose machine was replaced gets its tree back (latest snapshot, or the base) and the next verb answers 409 `sandbox_recreated` once — look at the tree, redo lost edits, retry; 503 `sandbox_restore_failed` means retry; 409 `sandbox_tree_missing` means the world root was deleted, so close and reopen; the source tree only — no data snapshot, nothing on the caller's disk; `commit` lands on the recorded base; `commit --rebase` sends `rebase: true` and replays onto a branch head that moved, a 409 naming each conflicting path). `open` reports `cli: {version, digest}` — the platform's own `gateway` CLI and a `sqlite3` are on PATH inside `exec`, so `gateway worlds schema entity add\|verifier add\|task add\|compile\|check .`, `gateway worlds data import\|show\|query\|sql .`, `gateway worlds test .`, `gateway worlds serve . --port 8080 & sleep 2; gateway worlds call http://localhost:8080 <tool> …` (one `exec`; a served world lives only as long as that command) and `sqlite3 db/state.sqlite "…"` run on the working copy; remote verbs (push, run, session open) stop in the sandbox by design — `check`/`commit` from your machine. `cli: null` + `warnings[]` means the image cannot host the runtime (python3 < 3.12 or node < 18); plain shell still runs |
|
|
41
42
|
| `worlds\|bench sandbox open --new <slug> --from custom\|<template>` | `open_world_sandbox({ new, from })` (same route, body `{new, from}`; a world that does not exist yet — `baseVersionId`/`containerId`/`baseContentHash` null, `branch: main`, `pendingSlug`; `custom` = the `schema init` scaffold, else a shipped template (`GET /api/public/worlds/templates`; unknown → 400 naming the list); taken slug → 409 `world_exists`, never renamed; `check_world_sandbox` runs the first commit's gate; the first `commit_world_sandbox` creates the world on `main` through the `worlds create --from <dir>` door, later commits land on the new base; the same staged CLI runs on the scaffold) |
|
|
42
43
|
| `worlds skill <name>` / `worlds connector skill` | `get_connector_authoring_skill`, `guide` |
|
|
43
44
|
| `worlds connector capture --remote` / poll + download | `start_connector_capture`, `get_connector_capture` (platform agent only) |
|
|
@@ -53,7 +54,7 @@ before it makes a request.
|
|
|
53
54
|
| `sandbox exec --timeout`, `exec_in_world_sandbox timeoutMs` | default 60 s, at most 600 s | refused, not clamped |
|
|
54
55
|
| Sandbox tree at `check`/`commit`, `bench push` unpacked | 128 MB of source, 2000 files | refused, naming the limit, before a byte is read; move data to `worlds data import`, delete scratch files or list them in `.gatewayignore` |
|
|
55
56
|
| Sandbox tree at `check`/`commit`: credential-shaped files (`.env`, `*.pem`, `id_rsa`, `credentials.json`…; `.env.example` stays), symlinks, sockets/devices | held back, never in the version | listed under `ignored`, each named in `warnings` (path only); use `bench secrets set` for a key the world needs |
|
|
56
|
-
| `worlds data query` / `data show` on a stored version | none beyond the version's own caps | the platform opens the version's tree and snapshot in a prepared directory it owns; no size travels through the request |
|
|
57
|
+
| `worlds data query` / `data sql` / `data show` on a stored version | none beyond the version's own caps | the platform opens the version's tree and snapshot in a prepared directory it owns; no size travels through the request |
|
|
57
58
|
| `bench push` / `worlds create --from` compressed bundle | 64 MB (16 MB from the browser) | refused, naming the limit |
|
|
58
59
|
| A single manifest / metadata file in a bundle | 1 MB | refused |
|
|
59
60
|
| `sandbox write` HTTP body | 1 MB (framework cap; only reachable by a body no schema would accept) | 413 |
|
|
@@ -28,3 +28,18 @@ The things that bite once and never again, moved out of `SKILL.md` to keep it un
|
|
|
28
28
|
anything is provisioned — the same preflight a run dispatch applies; set the secret, then open.
|
|
29
29
|
- `gateway secret set NAME` is an upsert: the same name again rotates the value and answers
|
|
30
30
|
`updated: true`; `bench secrets set` does the same for a benchmark's environment.
|
|
31
|
+
- Imported data survives every edit that cannot touch a stored row. After `gateway worlds data import
|
|
32
|
+
<slug>` the world's tree carries `db/snapshot.json` (the hosted snapshot) and not the rows; in
|
|
33
|
+
`worlds sandbox open <slug>@main`, adding a tool, a handler, a route, a test or a documentation
|
|
34
|
+
file then `schema compile .` and `schema check .` pass and the commit keeps the snapshot, no
|
|
35
|
+
re-import, and `gateway bench fork <slug>` takes the data with it. Adding an entity or a field
|
|
36
|
+
that is not required (or another `domain_version`) carries the data too: `sandbox check` says
|
|
37
|
+
the commit migrates the snapshot, and `sandbox commit` migrates it and re-pins the tree.
|
|
38
|
+
Removing, retyping or requiring a field, or changing a key, a relationship or a description, is
|
|
39
|
+
refused, naming what changed: keep it as it was, or remove `db/snapshot.json` and import every
|
|
40
|
+
row again. `check` reports
|
|
41
|
+
the handlers `not_run` there, since a session needs the hosted rows; `gateway worlds session open
|
|
42
|
+
<slug>` runs them, so a handler that cannot be imported (a syntax error, a missing module) passes
|
|
43
|
+
`check` and is first refused at that open: open a session on the version after a handler edit.
|
|
44
|
+
Compile from a fresh `sandbox open` rather than copying in artifacts compiled
|
|
45
|
+
elsewhere: the pin proves its data's domain from the `schema/lock.json` it started with.
|
|
@@ -26,6 +26,12 @@ gateway worlds schema init ./x --from-openapi github.json --include '/repos/{own
|
|
|
26
26
|
Inside a world sandbox (`run_gateway_cli`, `sandbox exec`) only a workspace path is accepted:
|
|
27
27
|
write the document into the workspace first (`sandbox write --from spec.yaml`), then import.
|
|
28
28
|
|
|
29
|
+
The import itself runs in this machine's schema runtime whenever it has one; `GATEWAY_RUNTIME=local`
|
|
30
|
+
demands it and sends nothing to the platform. A YAML document is read by PyYAML in the Python that
|
|
31
|
+
runs the runtime: without it the command stops before anything is sent and names the fix
|
|
32
|
+
(`pip install pyyaml` there, point `GATEWAY_PYTHON` at a Python that has it, or convert the
|
|
33
|
+
document to JSON).
|
|
34
|
+
|
|
29
35
|
## What the plan says
|
|
30
36
|
|
|
31
37
|
`--from-openapi` always prints the plan first (JSON, one object):
|
|
@@ -50,28 +56,51 @@ With `--plan-only` the same object carries `"written": false` and no `domain`/`n
|
|
|
50
56
|
|
|
51
57
|
## Conversion rules (decided; the importer documents them in `schema/provenance.json`)
|
|
52
58
|
|
|
53
|
-
- **Entities
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
`
|
|
57
|
-
a
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
59
|
+
- **Entities are the rows the routes carry.** The importer reads each selected operation's
|
|
60
|
+
response and request body through the vendor's envelope to the item schema: an array of
|
|
61
|
+
objects, or an object that declares no key and holds exactly one array (Crossref's
|
|
62
|
+
`{status, message: {items: [Work], total-results}}`, ClinicalTrials.gov's
|
|
63
|
+
`{studies: [Study], nextPageToken}`); a single-row response is the item when some list carries
|
|
64
|
+
it, it declares a key, or it names the field the route's own `{segment}` reads it by, else the
|
|
65
|
+
one object its wrapper holds. Wrappers never become entities. An item no planned route reads or
|
|
66
|
+
writes is no entity either (it stays an inline `$def`). Each entity is a `$defs` entry plus
|
|
67
|
+
`x-gateway.entities.<snake_plural>` with `primary_key` (snake_case, at most 64 characters, the
|
|
68
|
+
original kept in provenance); `allOf` is merged; `oneOf`/`anyOf` become a union the world
|
|
69
|
+
validator accepts. Entity roots close (`additionalProperties: false`); the vendor's field names
|
|
70
|
+
are kept exactly as spelled, hyphens included (`reference-count`), so a rows file shaped like
|
|
71
|
+
the vendor's response imports as it is.
|
|
72
|
+
- **Keys.** A declared key first (`id`, `<name>Id`, `<name>_id`, a `readOnly` field the vendor
|
|
73
|
+
marks `x-unique`/`x-primary-key`), else the field the single-row route reads the row by:
|
|
74
|
+
top-level and spelled the vendor's way (`/works/{doi}` → `DOI`), or nested in the row's
|
|
75
|
+
objects (`/studies/{nctId}` → the JSON Pointer `/protocolSection/identificationModule/nctId`,
|
|
76
|
+
every object on the path made required; never through an array or a nullable object). An item
|
|
77
|
+
only list routes carry falls back to its first required string/integer field, with a warning
|
|
78
|
+
that rows sharing it collapse: mark the key with `x-primary-key` or include the read route.
|
|
63
79
|
- **Routes.** Every path+method in `get|post|put|patch|delete` becomes an `[[operations]]` entry:
|
|
64
80
|
`name` = `operationId` snake_cased, at most 64 characters, deduped with `_2`, `_3` and a
|
|
65
|
-
warning; `entity` = the
|
|
66
|
-
|
|
67
|
-
parameters become `params
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
81
|
+
warning; `entity` = the entity its response (or, for a write, its body or the path's `GET`)
|
|
82
|
+
carries, a list route or `read|create|update|delete` by method; path parameters and declared
|
|
83
|
+
query parameters become `params`, the vendor's names kept, dots included (`query.bibliographic`,
|
|
84
|
+
`filter.overallStatus`). The envelope's path becomes `results` and the route's dotted
|
|
85
|
+
`envelope` (`message.items`), and the rest of the envelope its `wrapper`, so the served world
|
|
86
|
+
answers the vendor's shape (`{status: "ok", message-type: "work-list", message: {items, …}}`).
|
|
87
|
+
On a list route a declared query parameter that names an entity field becomes a `filter`
|
|
88
|
+
(paging names never do). A single read whose path argument names no field of the row, and an
|
|
89
|
+
operation whose response is not an entity (login, upload, HTML), stay **passthrough** routes with
|
|
90
|
+
no entity, the reason in the plan — served, answering `501 {"error": "not_modelled"}` until you
|
|
91
|
+
give them a handler.
|
|
92
|
+
- **Paging.** A cursor parameter (`cursor`, `pageToken`) with a next-cursor field beside the rows
|
|
93
|
+
(`next-cursor`, `nextPageToken`) pages by cursor; an `offset` parameter by offset; the limit's
|
|
94
|
+
`default` becomes `page_size`; the count beside the rows (`total-results`, `totalCount`) is the
|
|
95
|
+
hits key. `[api]` pages the way most list routes do, a route paging otherwise names its own
|
|
96
|
+
`page`, and page-number paging (`page` + `per_page`) is named in a warning (that route answers
|
|
97
|
+
every row on one page).
|
|
98
|
+
- **Tools.** An import declares none: routes are never tools. An agent reaches the world's routes
|
|
99
|
+
over HTTP (the api url and token `session open --surface api` prints); declare a tool in
|
|
100
|
+
`x-gateway.tools`, with a handler, where an agent needs one.
|
|
73
101
|
- **Skipped, with a reason:** `HEAD`/`OPTIONS`/`TRACE`; a path argument that is not an
|
|
74
|
-
identifier; a
|
|
102
|
+
identifier; a path that repeats a placeholder (`/a/{id}/{id}`); a second operation on the same
|
|
103
|
+
method+path shape. **Refused, with a JSON pointer
|
|
75
104
|
into the document, nothing written:** no `paths`; an unknown or external `$ref`; an entity root
|
|
76
105
|
that is (transitively) cyclic — model the recursion as a relationship field (`parent_id`)
|
|
77
106
|
instead; a path parameter nothing declares (`--param <name>` on a hand-written route); a
|
|
@@ -85,6 +114,6 @@ With `--plan-only` the same object carries `"written": false` and no `domain`/`n
|
|
|
85
114
|
|
|
86
115
|
The printed `next` steps are the loop, in order: `schema compile`, `schema check`, `data import
|
|
87
116
|
<rows.json> --entity <first list entity>` (rows shaped like the vendor's response), `worlds serve`
|
|
88
|
-
then `
|
|
89
|
-
tree lives in a sandbox. `connector conform ./x --openapi <the same document>` then checks the
|
|
117
|
+
then `curl -s http://127.0.0.1:<port>/<list path>` (the route over HTTP), and `sandbox commit` when
|
|
118
|
+
the tree lives in a sandbox. `connector conform ./x --openapi <the same document>` then checks the
|
|
90
119
|
served world back against the spec.
|