@withgateway/sdk 0.1.20 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (130) hide show
  1. package/README.md +141 -2
  2. package/dist/cli.js +8683 -1343
  3. package/dist/cli.js.map +1 -1
  4. package/dist/cli.mjs +8683 -1344
  5. package/dist/cli.mjs.map +1 -1
  6. package/dist/index.d.mts +1 -1
  7. package/dist/index.d.ts +1 -1
  8. package/dist/index.js +979 -46
  9. package/dist/index.js.map +1 -1
  10. package/dist/index.mjs +943 -38
  11. package/dist/index.mjs.map +1 -1
  12. package/dist/tracing/index.js +9 -2
  13. package/dist/tracing/index.js.map +1 -1
  14. package/dist/tracing/index.mjs +9 -2
  15. package/dist/tracing/index.mjs.map +1 -1
  16. package/dist/worlds/index.d.mts +738 -28
  17. package/dist/worlds/index.d.ts +738 -28
  18. package/dist/worlds/index.js +940 -42
  19. package/dist/worlds/index.js.map +1 -1
  20. package/dist/worlds/index.mjs +934 -36
  21. package/dist/worlds/index.mjs.map +1 -1
  22. package/package.json +9 -2
  23. package/runtime/gatewaysdk/__init__.py +0 -0
  24. package/runtime/gatewaysdk/environment/__init__.py +108 -0
  25. package/runtime/gatewaysdk/environment/_bundle.py +281 -0
  26. package/runtime/gatewaysdk/environment/_harbor.py +276 -0
  27. package/runtime/gatewaysdk/environment/_materialize.py +97 -0
  28. package/runtime/gatewaysdk/environment/_tar.py +33 -0
  29. package/runtime/gatewaysdk/environment/_world.py +346 -0
  30. package/runtime/gatewaysdk/environment/core.py +527 -0
  31. package/runtime/gatewaysdk/environment/errors.py +58 -0
  32. package/runtime/gatewaysdk/environment/runtime.py +150 -0
  33. package/runtime/gatewaysdk/environment/schema/__init__.py +35 -0
  34. package/runtime/gatewaysdk/environment/schema/__main__.py +225 -0
  35. package/runtime/gatewaysdk/environment/schema/_slack_fidelity.py +116 -0
  36. package/runtime/gatewaysdk/environment/schema/_slack_scenario.py +183 -0
  37. package/runtime/gatewaysdk/environment/schema/api.py +1628 -0
  38. package/runtime/gatewaysdk/environment/schema/batch.py +755 -0
  39. package/runtime/gatewaysdk/environment/schema/compiler.py +615 -0
  40. package/runtime/gatewaysdk/environment/schema/conform.py +428 -0
  41. package/runtime/gatewaysdk/environment/schema/connector.py +777 -0
  42. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/handlers.py +17 -0
  43. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/parity.json +28 -0
  44. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/provenance.json +233 -0
  45. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/scope.toml +112 -0
  46. package/runtime/gatewaysdk/environment/schema/connectors/apple-business-manager/world.json +1380 -0
  47. package/runtime/gatewaysdk/environment/schema/connectors/base.json +28 -0
  48. package/runtime/gatewaysdk/environment/schema/connectors/custom/handlers.py +18 -0
  49. package/runtime/gatewaysdk/environment/schema/connectors/custom/world.json +123 -0
  50. package/runtime/gatewaysdk/environment/schema/connectors/github/handlers.py +81 -0
  51. package/runtime/gatewaysdk/environment/schema/connectors/github/parity.json +37 -0
  52. package/runtime/gatewaysdk/environment/schema/connectors/github/provenance.json +134 -0
  53. package/runtime/gatewaysdk/environment/schema/connectors/github/scope.toml +158 -0
  54. package/runtime/gatewaysdk/environment/schema/connectors/github/world.json +5869 -0
  55. package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/handlers.py +23 -0
  56. package/runtime/gatewaysdk/environment/schema/connectors/google-calendar/world.json +209 -0
  57. package/runtime/gatewaysdk/environment/schema/connectors/google-drive/handlers.py +18 -0
  58. package/runtime/gatewaysdk/environment/schema/connectors/google-drive/world.json +286 -0
  59. package/runtime/gatewaysdk/environment/schema/connectors/jamf/handlers.py +34 -0
  60. package/runtime/gatewaysdk/environment/schema/connectors/jamf/parity.json +49 -0
  61. package/runtime/gatewaysdk/environment/schema/connectors/jamf/provenance.json +181 -0
  62. package/runtime/gatewaysdk/environment/schema/connectors/jamf/scope.toml +128 -0
  63. package/runtime/gatewaysdk/environment/schema/connectors/jamf/world.json +2218 -0
  64. package/runtime/gatewaysdk/environment/schema/connectors/jira/handlers.py +28 -0
  65. package/runtime/gatewaysdk/environment/schema/connectors/jira/world.json +389 -0
  66. package/runtime/gatewaysdk/environment/schema/connectors/kandji/handlers.py +38 -0
  67. package/runtime/gatewaysdk/environment/schema/connectors/kandji/parity.json +29 -0
  68. package/runtime/gatewaysdk/environment/schema/connectors/kandji/provenance.json +152 -0
  69. package/runtime/gatewaysdk/environment/schema/connectors/kandji/scope.toml +120 -0
  70. package/runtime/gatewaysdk/environment/schema/connectors/kandji/world.json +765 -0
  71. package/runtime/gatewaysdk/environment/schema/connectors/linear/handlers.py +20 -0
  72. package/runtime/gatewaysdk/environment/schema/connectors/linear/world.json +363 -0
  73. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/fidelity.json +45 -0
  74. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/handlers.py +1 -0
  75. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/provenance.json +851 -0
  76. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/scope.md +231 -0
  77. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/teams-types.json +1710 -0
  78. package/runtime/gatewaysdk/environment/schema/connectors/microsoft-teams/world.json +858 -0
  79. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/handlers.py +32 -0
  80. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/parity.json +50 -0
  81. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/provenance.json +137 -0
  82. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/scope.toml +78 -0
  83. package/runtime/gatewaysdk/environment/schema/connectors/netsuite/world.json +16243 -0
  84. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/handlers.py +806 -0
  85. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/parity.json +46 -0
  86. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/provenance.json +143 -0
  87. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/scope.toml +174 -0
  88. package/runtime/gatewaysdk/environment/schema/connectors/salesforce/world.json +4695 -0
  89. package/runtime/gatewaysdk/environment/schema/connectors/slack/MODEL.md +236 -0
  90. package/runtime/gatewaysdk/environment/schema/connectors/slack/capabilities.json +1039 -0
  91. package/runtime/gatewaysdk/environment/schema/connectors/slack/handlers.py +762 -0
  92. package/runtime/gatewaysdk/environment/schema/connectors/slack/parity.json +65 -0
  93. package/runtime/gatewaysdk/environment/schema/connectors/slack/provenance.json +294 -0
  94. package/runtime/gatewaysdk/environment/schema/connectors/slack/scope.toml +44 -0
  95. package/runtime/gatewaysdk/environment/schema/connectors/slack/world.json +4993 -0
  96. package/runtime/gatewaysdk/environment/schema/connectors/workday/handlers.py +75 -0
  97. package/runtime/gatewaysdk/environment/schema/connectors/workday/parity.json +36 -0
  98. package/runtime/gatewaysdk/environment/schema/connectors/workday/provenance.json +179 -0
  99. package/runtime/gatewaysdk/environment/schema/connectors/workday/scope.toml +180 -0
  100. package/runtime/gatewaysdk/environment/schema/connectors/workday/world.json +806 -0
  101. package/runtime/gatewaysdk/environment/schema/data.py +593 -0
  102. package/runtime/gatewaysdk/environment/schema/extract.py +263 -0
  103. package/runtime/gatewaysdk/environment/schema/host.py +2407 -0
  104. package/runtime/gatewaysdk/environment/schema/host_surfaces.py +275 -0
  105. package/runtime/gatewaysdk/environment/schema/platform.py +1272 -0
  106. package/runtime/gatewaysdk/environment/schema/scaffold.py +250 -0
  107. package/runtime/gatewaysdk/environment/schema/skill.py +118 -0
  108. package/runtime/gatewaysdk/environment/schema/skills/agents/connector-template-author.md +202 -0
  109. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONNECTORS.md +331 -0
  110. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
  111. package/runtime/gatewaysdk/environment/schema/skills/connector-schema-authoring/SKILL.md +313 -0
  112. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/ROWS.md +223 -0
  113. package/runtime/gatewaysdk/environment/schema/skills/world-data-ingestion/SKILL.md +293 -0
  114. package/runtime/gatewaysdk/environment/schema/skills/worlds-getting-started/SKILL.md +390 -0
  115. package/runtime/gatewaysdk/environment/schema/slack.py +757 -0
  116. package/runtime/gatewaysdk/environment/schema/snapshot.py +479 -0
  117. package/runtime/gatewaysdk/environment/schema/store.py +904 -0
  118. package/runtime/gatewaysdk/environment/schema/tasks.py +525 -0
  119. package/runtime/gatewaysdk/environment/schema/tools_world.py +322 -0
  120. package/runtime/gatewaysdk/environment/schema/validation.py +577 -0
  121. package/runtime/gatewaysdk/environment/schema/worker.py +78 -0
  122. package/runtime/gatewaysdk/environment/schema/workers.py +500 -0
  123. package/runtime/gatewaysdk/environment/schema/world_tests.py +336 -0
  124. package/skills/agents/connector-template-author.md +202 -0
  125. package/skills/connector-schema-authoring/CONNECTORS.md +331 -0
  126. package/skills/connector-schema-authoring/CONVENTIONS.md +100 -0
  127. package/skills/connector-schema-authoring/SKILL.md +313 -0
  128. package/skills/world-data-ingestion/ROWS.md +223 -0
  129. package/skills/world-data-ingestion/SKILL.md +293 -0
  130. package/skills/worlds-getting-started/SKILL.md +390 -0
package/README.md CHANGED
@@ -324,12 +324,151 @@ is a complete local production harness — a real tool-calling agent loop,
324
324
  parallel execution, a pass gate, a JSON artifact and a CI exit code. The
325
325
  Python twin is [`gatewaysdk/examples/worlds/world_suite.py`](../sdk/examples/worlds/world_suite.py).
326
326
 
327
- There is also a CLI for pushing, pulling and running worlds:
327
+ ## The `gateway` CLI
328
+
329
+ The package ships the `gateway` command — the full TypeScript port of the
330
+ Python CLI, group for group, with the same command names, positionals and
331
+ flags (`npm i -g @withgateway/sdk`, or `npx -p @withgateway/sdk gateway …`).
332
+
333
+ The package bundles the schema runtime (the SDK's stdlib-only Python
334
+ `gatewaysdk/environment` directory, the same files the platform executes). With
335
+ any `python3 >= 3.11` on the machine the CLI runs it locally — `worlds serve`,
336
+ `connector capture`, `schema check|compile|describe`, `data import` on a
337
+ directory — with no pip and no login. Without one, those commands send the
338
+ world's files to `POST /api/public/worlds/runtime` and the platform runs the
339
+ same code. `GATEWAY_RUNTIME=local|platform` forces a side; `GATEWAY_PYTHON`
340
+ names the interpreter. Only `bench eval|install|harvest-harbor` and the Python
341
+ agent scaffolds (`algorithm`, `deploy`, `init`) stay Python-SDK-only.
328
342
 
329
343
  ```bash
330
- npx gateway bench --help
344
+ gateway auth login --host https://withgateway.ai --public-key pk-lf-… --api-key sk-lf-…
345
+ gateway --help # every group
346
+ gateway bench --help # every command in a group
347
+ gateway bench push --help # a command's positionals and flags
331
348
  ```
332
349
 
350
+ Credentials resolve in this order: `--host` / `--api-key` flags, then
351
+ `GATEWAY_HOST` / `GATEWAY_PUBLIC_KEY` / `GATEWAY_SECRET_KEY`, then
352
+ `~/.gateway/config.json` written by `gateway auth login`. Commands print JSON
353
+ where the Python CLI does and exit non-zero with the reason on stderr.
354
+
355
+ | Group | Commands |
356
+ | --- | --- |
357
+ | `auth` | `login`, `logout`, `status` |
358
+ | `secret` | `set`, `list`, `delete` |
359
+ | `agent` | `list` |
360
+ | `run` | `start`, `verify`, `list` |
361
+ | `dataset` | `upload FILE --name …` |
362
+ | `bench` | `push`, `pull`, `checkout`, `fork`, `diff`, `propose`, `status`, `log`, `branch`, `tag`, `dispatch`, `run`, `run-configs`, `secrets`, `validate`, `resolve`, `tasks`, `refs`, `proposals` |
363
+ | `worlds session` | `open`, `status`, `browse`, `close`, `seed`, `export` |
364
+ | `worlds schema` | `templates`, `init`, `compile`, `check`, `describe` (the schema runtime runs on the platform) |
365
+ | `worlds connector` | `validate`, `capture`, `conform`, `ingest`, `publish`, `list`, `pull`, `clone`, `skill` |
366
+ | `worlds data` | `import`, `check`, `extract`, `status`, `batches`, `publish`, `show` — rows into a world (streamed, chunked, resumable for a slug), out of vendor exports, and the batch ledger |
367
+ | `worlds skill` | `[NAME] [--list] [--install DIR] [--force]` — the packaged agent skills; `--install DIR` alone lands all of them |
368
+ | `worlds run` / `ci` | a TypeScript agent module against hosted sessions |
369
+ | `environment` | `list`, `get`, `tasks`, `run`, `run-status`, `rollouts`, `samples`, `performance`, `link-task-set`, `create-task`, `agent-status`, `agent-trigger` |
370
+ | `release` | `gate` — exit 0 pass, 1 blocked, 2 untested |
371
+
372
+ Python-only, and said so by the command itself (exit 2 with the equivalent):
373
+ `bench eval` (local engines → `bench dispatch`), `bench install`,
374
+ `bench harvest-harbor`, `bench push --build-wheel`, `deploy`, `init`,
375
+ `algorithm list|new`. `worlds serve` runs the bundled runtime with any
376
+ `python3 >= 3.11`, and opens a hosted `api` session when there is none.
377
+
378
+ Rows files for `worlds data import` and `worlds session seed` are a JSON
379
+ object `{"<entity>": [row, …]}`, JSONL of `{"entity", "row"}` lines, or bare
380
+ rows with `--entity`; `-` reads stdin and a `.gz` path is gunzipped.
381
+ `worlds session export` writes the session's call log as JSONL
382
+ (`{seq, kind, tool, args, result, error, completedAt}` per line).
383
+
384
+ ### Rows in bulk: `worlds data`
385
+
386
+ The same commands and flags as the Python CLI. A platform slug streams the
387
+ rows — never held whole — as gzipped JSONL chunks into a data batch the
388
+ platform validates against the world's contract, applies onto the world's
389
+ snapshot and turns into a data-only version; a local directory keeps the
390
+ small inline import through the runtime bridge (at most 5000 rows / 4 MiB —
391
+ larger inputs go to the platform world).
392
+
393
+ ```bash
394
+ gateway worlds data import <slug|dir> <rows…|-> [--entity E] [--mode append|replace] [--merge upsert|fill]
395
+ [--chunk-rows N] [--chunk-bytes SIZE] [--parallel N] [--resume] [--atomic|--per-chunk]
396
+ [--publish now|later] [--wait|--no-wait] [--report PATH] [--dry-run] [-m MESSAGE]
397
+ gateway worlds data check <slug|dir> <rows…|-> [--entity E] [--mode …] [--merge …] [--report PATH]
398
+ gateway worlds data extract <dir|slug> <file|-> --shape tool-results|vendor-envelope|session-export
399
+ --entity E [--out PATH|-] [--merge fill|upsert] [--no-redact]
400
+ gateway worlds data status <batchId> [--world SLUG]
401
+ gateway worlds data batches <slug> [--limit N]
402
+ gateway worlds data publish <slug> [-m MESSAGE]
403
+ gateway worlds data show <slug|dir> [--entity E]
404
+ ```
405
+
406
+ - `import` cuts chunks on row boundaries at `--chunk-bytes` (64MiB before
407
+ gzip by default) or `--chunk-rows`, uploads `--parallel` (4) at a time to
408
+ the presigned URLs the platform hands out, keeps the manifest at
409
+ `.gateway/imports/<batchId>.json` after every chunk (the same schema as the
410
+ Python CLI, so either CLI can `--resume` the other's upload), completes the
411
+ batch and follows the job: one line per phase on a TTY, then the settled
412
+ batch as JSON. Exit 0 applied or published, 2 refused (`--report` writes
413
+ every refusal as JSONL), 1 error. `--no-wait` prints `{batchId, state}`;
414
+ `--publish later` leaves the batch for `data publish`. `--merge` decides
415
+ the same primary key inside one batch (`upsert`: last wins; `fill`: first
416
+ wins, later rows add missing fields); `--per-chunk` applies the good chunks
417
+ and reports the bad ones.
418
+ - `check` (and `import --dry-run`) is the dry run: a slug runs a dry-run
419
+ batch on the platform, a directory the runtime bridge's import over a
420
+ scratch copy; nothing is written. Exit 2 when any row is refused.
421
+ - `extract` runs here, natively, on files of any size: rows out of a
422
+ tool-results dump (`raw_output.results[]`), the vendor's own responses
423
+ (`[api].envelope` of `connector.toml`) or a session export
424
+ (`result.<envelope>[]` of successful calls), with null and empty fields
425
+ dropped, undeclared fields stripped and counted, rows deduped by primary
426
+ key and the world's `[redact]` policy applied (`hash` fields become
427
+ `sha256:<hex>` over `<connector slug>:<value>`, `drop` fields go, `round`
428
+ fields round) — byte for byte what the Python CLI writes. The summary
429
+ `{"read", "rows", "unknownFields", "redacted"}` goes to stderr.
430
+
431
+ Chunks and rows are serialised exactly as the Python CLI serialises them
432
+ (`json.dumps(row, ensure_ascii=False, separators=(",", ":"))`, key order
433
+ kept, `1.0` kept a float), so extract output and chunk content digests
434
+ match across the two CLIs, and gzip digests too wherever the two zlibs
435
+ deflate alike (Node 20 and 24 bundle Chromium's zlib, which deflates
436
+ differently from CPython's stock zlib); the shared fixtures under
437
+ `gatewaysdk/fixtures/world-data/` are asserted by both test suites. `GATEWAY_WORLD_DATA_POLL_MS` shortens the
438
+ status poll (CI), `GATEWAY_PROGRESS=1|0` forces the phase lines on or off.
439
+
440
+ The packaged skills (`gateway worlds skill --list`) are copied from the Python
441
+ package into `skills/` by `npm run build` (`scripts/copy-skills.mjs`) and ship
442
+ with the npm package.
443
+
444
+ ### For agents
445
+
446
+ The CLI is built to be driven by a coding agent (Claude Code, Cursor, Codex):
447
+ every command has `--help` with its positionals and flags, output is JSON
448
+ where a program would read it, and failures exit non-zero with the reason on
449
+ stderr. The guides an agent needs ship inside the package as skills:
450
+
451
+ ```bash
452
+ npm i -g @withgateway/sdk # or: npx -p @withgateway/sdk gateway …
453
+ gateway auth login --host https://withgateway.ai --public-key pk-lf-… --api-key sk-lf-…
454
+ gateway worlds skill --list # the packaged guides
455
+ gateway worlds skill worlds-getting-started # print one: auth → pick a path → contract → data → push → session → run
456
+ gateway worlds skill --install . # land every guide (and its agent) in this repository's .claude/
457
+ ```
458
+
459
+ | Skill | Use it when |
460
+ | --- | --- |
461
+ | `worlds-getting-started` | building a world or environment from scratch: templates, workspace connectors, vendor captures, custom contracts; publishing, sessions, running, iterating |
462
+ | `world-data-ingestion` | putting rows into a world from any source: rows-file contract, `worlds data import\|check\|extract`, session seeding and export, refusals |
463
+ | `connector-schema-authoring` | writing a connector schema world (`gateway-world/1`) by hand or from a vendor spec |
464
+
465
+ `--install DIR` writes the skills under `DIR/.claude/skills/` (and the
466
+ connector-template-author agent under `DIR/.claude/agents/`), so the agent
467
+ reads them the way it reads any project skill; `--force` overwrites a copy
468
+ that drifted. Every CLI step in the guides names its MCP twin, so an agent
469
+ that has the [MCP server](https://withgateway.ai/docs/mcp/setup) attached can
470
+ do the same work without a shell.
471
+
333
472
  ## Cross-Language Compatibility
334
473
 
335
474
  This SDK produces **identical assignments** to the Python SDK (`gatewaysdk`) for the same `(experimentKey, userId)` pair. You can run experiments across services in different languages.