@elaraai/east-codex-plugin 1.0.107 → 1.0.108

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "east-codex-plugin",
3
- "version": "1.0.107",
3
+ "version": "1.0.108",
4
4
  "description": "East programming language ecosystem - a statically typed, expression-based language for business logic, data processing, UI definitions, and durable execution",
5
5
  "author": {
6
6
  "name": "Elara AI",
package/index.json CHANGED
@@ -48515,12 +48515,16 @@
48515
48515
  "policy",
48516
48516
  "spill",
48517
48517
  "rack-only",
48518
+ "memory-limit",
48519
+ "gpu",
48520
+ "environment",
48521
+ "layers",
48518
48522
  "dataflow",
48519
48523
  "watch",
48520
48524
  "e3-api-server"
48521
48525
  ],
48522
48526
  "imports": [],
48523
- "source": "# Local machine: bind a VPN address the worker can reach.\ne3 rack enroll --listen 100.64.0.10:7331 --label workshop\n# Inside a prepared worker VM, run the printed matching-release configure/enroll\n# commands with its host-provisioned allocation path. Enter the origin and token\n# at their hidden prompts; never put them in shell arguments or plaintext files.\n# HTTP needs --allow-vpn-http plus authenticated VPN and exact host endpoint policy.\n# Enrollment observes a new registration, not activation or task readiness.\n# Independent agents use separate worker VMs; resources/GPU ownership belong to\n# the host lifecycle. Final host/source management and capacity activation remain\n# companion cloud #253 work. Once the service and capacity are active:\ne3 rack policy . set 'train_*' rack --size large\ne3 dataflow run . dev --rack -j 4 --memory 8G\ne3 watch src/main.ts . dev --start --rack\ne3 rack status\ne3 task logs . dev.train_model --follow\n# Placement preserves hashes/cache. Local planner and fallback retain the budget.\n# --rack-only waits for busy compatible capacity; infrastructure fallback remains.\n# Custom commands/environments and host-dependent calls stay local by default.\n# For a local API server with the matching optional @elaraai/e3-rack installed:\ne3-api-server --repos ./repos --rack",
48527
+ "source": "# Local machine: bind a VPN address the worker can reach.\ne3 rack enroll --listen 100.64.0.10:7331 --label workshop\n# Inside a prepared worker VM, run the printed matching-release configure/enroll\n# commands with its host-provisioned allocation path. Enter the origin and token\n# at their hidden prompts; never put them in shell arguments or plaintext files.\n# HTTP needs --allow-vpn-http plus authenticated VPN and exact host endpoint policy.\n# Enrollment observes a new registration, not activation or task readiness.\n# Independent agents use separate worker VMs; resources/GPU ownership belong to\n# the host lifecycle. Final host/source management and capacity activation remain\n# companion cloud #253 work. Once the service and capacity are active:\ne3 rack policy . set 'train_*' rack --size large --gpu\ne3 rack policy . set 'score_*' rack --memory-limit 8G\ne3 dataflow run . dev --rack -j 4 --memory 8G\ne3 watch src/main.ts . dev --start --rack\ne3 rack list\ne3 rack status\ne3 task logs . dev.train_model --follow\n# Placement preserves hashes/cache. Local planner and fallback retain the budget.\n# A split task's pieces are leased together, each at its memory cap (half as much\n# again as the task was measured to need): a rack books the caps it holds, so it\n# never books more than it has. --memory-limit is a ceiling on one piece's cap;\n# --gpu asks for a rack's GPU (east-py tasks). A piece that outgrows its cap\n# runs again under a larger one; a whole task that does fails.\n# A task that declares a python or node environment runs on a rack that composes\n# environments: this machine installs it for the rack's platform and sends it as\n# layers ([ENV] lines), with no container runtime; tools/image environments, a\n# package with no wheel for the platform or with native code stay local, with\n# the reason on the task's [LOCAL] line.\n# --rack-only waits for busy compatible capacity; infrastructure fallback remains.\n# Custom commands and host-dependent calls stay local by default.\n# For a local API server with the matching optional @elaraai/e3-rack installed:\ne3-api-server --repos ./repos --rack",
48524
48528
  "languages": [
48525
48529
  "typescript"
48526
48530
  ]
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@elaraai/east-codex-plugin",
3
- "version": "1.0.107",
3
+ "version": "1.0.108",
4
4
  "description": "East programming language ecosystem - a statically typed, expression-based language for business logic, data processing, UI definitions, and durable execution",
5
5
  "license": "AGPL-3.0-or-later",
6
6
  "type": "module",
@@ -85,6 +85,8 @@ What do you need?
85
85
  │ └─ Bundle and export → e3.package(name, version, ...items); e3.export(pkg, zipPath)
86
86
  ├─ Operate it from the shell → the e3 CLI (reference/cli.md): deploy, run, read and write datasets, mutate records, gc
87
87
  ├─ Delegate local task bodies to a rack → e3 rack enroll + policy; dataflow run --rack, watch --start --rack (reference/rack.md)
88
+ │ ├─ A split task's pieces fill the rack → booked by what each reserves, capped half as much again; --memory-limit bounds the cap, --gpu asks for the GPU
89
+ │ └─ A task with its own python/node environment → nothing new in the package: its layers are built for the rack's platform
88
90
  └─ Drive it from code
89
91
  ├─ A server, over HTTP → @elaraai/e3-api-client (reference/from-code.md)
90
92
  ├─ A local repository → @elaraai/e3-core
@@ -236,6 +238,10 @@ const shout = e3.task('shout', [greet.output],
236
238
  derivation: `{ python: { project } }`, `{ node: { project } }`,
237
239
  `{ tools: { files: [...] } }` (prebuilt binaries on PATH, such as a C runner)
238
240
  or `{ image: { digest: 'repo@sha256:<64 hex>' } }` (cloud; a tag is refused).
241
+ A task with a `python` or `node` environment also runs on a rack
242
+ (`e3 dataflow run --rack`): the launching machine installs the environment
243
+ for the rack's platform and sends it as layers, with no container runtime;
244
+ `tools` and `image` environments stay local there (reference/rack.md).
239
245
 
240
246
  ```python
241
247
  # packages/pricing/src/pricing/__init__.py — the member's root module
@@ -518,7 +524,7 @@ Read the one a task needs; each holds its subject whole.
518
524
  | [reference/cli.md](reference/cli.md) | Every `e3` command: repositories, packages, workspaces, datasets, tasks, runs, calls, records, watch, convert, auth, gc |
519
525
  | [reference/from-code.md](reference/from-code.md) | `@elaraai/e3-api-client`, `@elaraai/e3-core`, `e3-api-server` |
520
526
  | [reference/browser.md](reference/browser.md) | `@elaraai/e3-web`: serveUnits, serveE3, createWebE3, and what a browser cannot do |
521
- | [reference/rack.md](reference/rack.md) | `e3 rack enroll` / `policy`, routing a run to a rack, `@elaraai/e3-rack` |
527
+ | [reference/rack.md](reference/rack.md) | `e3 rack enroll` / `policy`, routing a run to a rack, how a rack is filled and what a run prints, tasks with an environment on a rack, `@elaraai/e3-rack` |
522
528
  | [reference/repository.md](reference/repository.md) | A local repository's layout, and upgrades |
523
529
 
524
530
  ## Related skills
@@ -7,13 +7,13 @@ Part of the `e3` skill.
7
7
  | Signature | Description | Example |
8
8
  | --- | --- | --- |
9
9
  | `e3 rack enroll --listen <host:port> [--allow <ip,...>] [--advertise-url <origin>] [--label <label>] [--no-wait]` | Starts the user-wide hub and prints the source origin, the hub's certificate SHA-256, the matching release and a single-use token for the companion's hidden prompts. Waits for a new registration, not activation. Behind an SSH reverse tunnel, listen on the loopback and advertise the relay's origin. | `e3 rack enroll --listen 127.0.0.1:7331 --allow 127.0.0.1 --advertise-url https://relay.example:7401` |
10
- | `e3 rack policy [repo] set <glob> rack\|local [--size <size>] [--timeout <min>] [--allow-host-access]` | First whole-task-name match wins; `*` and `?` are wildcards. Size: serverless, small, medium, large, xlarge. | `e3 rack policy . set 'train_*' rack --size large` |
10
+ | `e3 rack policy [repo] set <glob> rack\|local [--size <size>] [--memory-limit <size>] [--gpu] [--timeout <min>] [--allow-host-access]` | First whole-task-name match wins; `*` and `?` are wildcards; setting a rule again replaces it whole. Size: serverless (2 cores, 2 GiB), small (2, 4 GiB), medium (4, 8 GiB), large (8, 16 GiB), xlarge (16, 32 GiB). `--memory-limit 8G` is a ceiling on one piece's cap, never a request; `--gpu` asks for a rack's GPU. | `e3 rack policy . set 'train_*' rack --size large` |
11
11
  | `e3 rack policy [repo] enable\|disable\|mode <opt-in\|auto>` | Enable/disable saved opt-in; `mode auto` selects eligible unmatched tasks. Defaults: disabled, opt-in. | `e3 rack policy . enable` |
12
12
  | `e3 rack policy [repo] defaults [--no-spill] [--claim-timeout <sec>] [--retry-task-failures-locally]` | Defaults: large, 1440-minute timeout, spill on, 30-second claim recheck, no task-failure retry. | `e3 rack policy . defaults --no-spill` |
13
13
  | `e3 rack policy [repo] show\|unset <glob>\|reset` | Inspect rules, remove an exact pattern, or delete the saved policy. | `e3 rack policy . show` |
14
- | `e3 dataflow run <repo> <ws> --rack\|--no-rack\|--rack-only` | Select rack routing for this run; rack-only disables capacity spill but keeps infrastructure fallback. ❗ Refused for remote URLs. | `e3 dataflow run . dev --rack` |
14
+ | `e3 dataflow run <repo> <ws> --rack\|--no-rack\|--rack-only` | Select rack routing for this run; rack-only disables capacity spill but keeps infrastructure fallback: work waits two minutes for an unhealthy rack to come back, and is leased once more after a failed transfer, before it runs locally. ❗ Refused for remote URLs. | `e3 dataflow run . dev --rack` |
15
15
  | `e3 watch <source> <repo> <ws> --start --rack` | Reuses a rack session across watch runs and refreshes capacity; a second Ctrl-C stops the watch at once. ❗ `--rack` and `--rack-only` require `--start`. | `e3 watch src/main.ts . dev --start --rack` |
16
- | `e3 rack status\|list\|remove <id-or-label>\|approve <id-or-label>\|stop` | Inspect fleet, revoke a rack, admit the boot a quarantined rack last reported, or stop the shared hub. Status never starts an absent hub. | `e3 rack status` |
16
+ | `e3 rack status\|list\|remove <id-or-label>\|approve <id-or-label>\|stop` | Inspect fleet, revoke a rack, admit the boot a quarantined rack last reported, or stop the shared hub. Status never starts an absent hub, and lists how its last leases ended, each with its reason. | `e3 rack status` |
17
17
  | `e3 rack config [--listen <host:port>] [--allow <ip,...>] [--idle-exit <min>] [--advertise-url <origin>] [--tls\|--no-tls] [--require-approval\|--no-require-approval]` | Configure the shared listener (off by default; HTTPS with the hub's own certificate unless `--no-tls`); every rack and data URL is built on `--advertise-url` when set; zero idle-exit disables automatic shutdown; `--require-approval` quarantines a rack after each reboot until `e3 rack approve` admits it (enrollment admits the first boot). ❗ Plain HTTP only on the loopback or a private address, with `--allow`. | `e3 rack config --listen 10.0.0.5:7331` |
18
18
 
19
19
  The local process owns graph planning, repository and cache. After a cache
@@ -21,10 +21,60 @@ miss, eligible plain task, map and merge bodies can execute on the rack; input
21
21
  closures and every output object transfer by hash. The session verifies the
22
22
  whole output graph before success. Placement never changes task/cache hashes.
23
23
  Local bodies and fallback share the original cores/memory budget; rack bodies
24
- use remote capacity. Calls, mutations and intake stay local. Stock runner
25
- configurations qualify; commands, custom environments/platforms and unscannable
26
- IR stay local. Host file/environment/network access stays local unless a rule
27
- deliberately grants `--allow-host-access` and its resources exist on the rack.
24
+ use remote capacity: each lease asks its cores, what it reserves and its cap
25
+ (a split task's piece one core, reserving what it was measured to need, capped
26
+ half as much again; a task its size's cores, at most the most a rack has,
27
+ reserving its cap, its `--memory-limit`, else its size's memory), a rack is given only what its free
28
+ resources hold by what each lease reserves, and only work whose cap all its
29
+ memory holds, and a run is as much wider as the racks have cores. `e3 rack list` shows each rack's resources and what is free. A run prints `[RACK] <task> → <rack>` when a rack claims a
30
+ task, `[RACK] <task> input transfer retried (2 of 5): <cause>` for a transfer
31
+ given another try, `[LOCAL] <task> (<reason>)` for a rack's work this machine
32
+ runs after all (`spill`, `no-rack-for-tier:<tier>`, or the rack's reason, such
33
+ as `input-transfer-failed: <cause>` or `guest-crashed: <detail>`, or
34
+ `memory-limit-exceeded: needs 3 GB, limit 2 GB` for work whose cap passes its
35
+ rule's limit or every rack's memory),
36
+ `[RACK] <task> waiting: needs 1 core and 4 GB, <rack> has 0 cores and 1 GB
37
+ free` for work queued with no rack having room for what it reserves,
38
+ `[RACK] <task> outgrew its 3 GB cap on <rack>: runs again under 4.5 GB` for a
39
+ split task's piece its rack stopped at its cap (a whole task fails instead:
40
+ `memory-limit-exceeded (cap 16 GB); raise --memory-limit or --size`),
41
+ `[RACK] <task> stopped on <rack> at 2.6 GB, its host short of memory: runs
42
+ again reserving 2.6 GB` for a piece a rack stopped to keep its host from
43
+ running out, as a local run's guard stops one, and ends
44
+ with `Rack summary: 1 task and 12 units on <rack> · 0 waited for capacity · 0
45
+ ran again · 1 transfer retried · 0 ran locally` (each task or unit once,
46
+ though it was leased again). On a rack that admits by
47
+ its resources a split task's pieces are queued together, booked by what each
48
+ reserves, and the rack's guard keeps them from running its host out of memory;
49
+ a first run's piece reserves e3's estimate, as a local run does, so it is as
50
+ wide as a later run and measures no piece first. Calls, mutations and intake stay local. Stock runner
51
+ configurations qualify, and so does a task that declares a `python` or `node`
52
+ environment, with the project platform packages its runner names; commands, a
53
+ platform package with no environment (`non-stock-platform:<name>`) and
54
+ unscannable IR stay local. Host file/environment/network access stays local
55
+ unless a rule deliberately grants `--allow-host-access` and its resources exist
56
+ on the rack.
57
+
58
+ **A task that declares an environment** needs nothing new in its package. On a
59
+ rack whose agent composes environments (`environments` among its features in
60
+ `e3 rack list`; `Rack: <rack> (…, environments for x86_64-manylinux_2_28)` as
61
+ a run starts), the launching machine installs the environment a second time
62
+ for the rack's platform, with the same frozen installs, packs it as layers the
63
+ rack fetches through the hub, and prints `[ENV] 3f9a2b1c0d4e for
64
+ x86_64-manylinux_2_28: building` then `[ENV] 3f9a2b1c0d4e built in 48 s: 4
65
+ layers, 2.1 GB (1 new)`. No container runtime on either machine; nothing is
66
+ compiled for the rack; a second run builds nothing, and a change to the
67
+ project's code makes one small new layer. The layers are kept in the
68
+ repository's `rack/env-layers`, which `e3 repo gc` sweeps. ❗ What cannot go
69
+ stays local, with its reason on the task's `[LOCAL]` line: `environment:tools`
70
+ / `environment:image` (its kind), `environment:windows-host` (a Windows
71
+ launching machine), `no-rack-for-environment:<tier>` (no rack composes them),
72
+ `environment:no-wheel:<package>` and `environment:native-package:<package>`
73
+ (nothing is compiled for another platform), `environment:pnpm` (a pnpm before
74
+ 11, for another platform), `environment:build-failed` (the tool's message is
75
+ in `e3 task logs`; a watch's next run builds it again). A task's tier is its runner's; `--gpu` puts an `east-py`
76
+ task on `py-datascience`, and is refused for another runner
77
+ (`gpu-not-east-py`).
28
78
 
29
79
  The listener serves HTTPS with the hub's own self-signed certificate, made on
30
80
  first use and kept in `hub-tls.beast2`; a rack pins it by its SHA-256, which
@@ -118,7 +168,14 @@ run waits up to 10 seconds for an enrolled rack to report healthy (with none
118
168
  enrolled, or a policy that selects no task for a rack, it starts at once): run
119
169
  `e3 rack hub` as a service, or set `--idle-exit 0`, for a hub racks can always
120
170
  reach. A data URL lasts 15 minutes, which a transfer may use whole; an object is
121
- at most 5 GiB. The control schema currently requires protocol 2 exactly.
171
+ at most 5 GiB. The hub serves one rack eight transfers at once (one more is told
172
+ to ask again), and reads an object from a byte on (`Range`), so a transfer a slow
173
+ link cut short resumes; it keeps a rack's idle connection for 75 seconds.
174
+ A hub and the e3 that calls it speak control protocol 3 exactly:
175
+ a hub an older e3 started is asked to exit once idle, and `e3 rack stop` with
176
+ that e3 replaces it now. A rack's agent and its source speak rack protocol 3:
177
+ an agent of another protocol is refused by name each time it asks, and
178
+ `e3 rack list` shows its rack as `incompatible: …; update the rack` until it is.
122
179
  Agents must report bundled e3 at least the project's release. Infrastructure
123
180
  failure falls back locally with a separate execution attempt, as does an
124
181
  attempt the rack stopped on its own; the run's own cancellation and the task's
@@ -129,11 +186,11 @@ sessions of the same canonical repository and e3 release.
129
186
 
130
187
  | Signature | Description | Example |
131
188
  | --- | --- | --- |
132
- | `openRackRunner({ repoPath, workspace, storage, label, budget?, settings?, policy?, rackOnly? })` | Opens a runner/session; preserve the caller's existing budget and process settings, use `rackSlots` to extend orchestrator width, and close in `finally`. | `const rack = await openRackRunner({ repoPath, workspace: 'dev', storage, label: 'run', budget });` |
189
+ | `openRackRunner({ repoPath, workspace, storage, label, budget?, settings?, policy?, rackOnly?, unreachableWaitMs?, onPlacement?, onEnvironment? })` | Opens a runner/session; preserve the caller's existing budget and process settings, use `rackSlots` to extend orchestrator width, and close in `finally`. Under `rackOnly`, work waits `unreachableWaitMs` (two minutes) for an unhealthy rack before it runs locally. `onPlacement` is told where each task or unit runs; `onEnvironment` that an environment's layers are being built for a rack, and what the build came to. | `const rack = await openRackRunner({ repoPath, workspace: 'dev', storage, label: 'run', budget });` |
133
190
  | `createDataflowRunnerFactory(options?)` | Pass to `createServer({ dataflowRunner })`; one session per API dataflow, using the server's shared budget. Explicit server opt-in overrides policy.enabled; rules still apply. | `const server = await createServer({ reposDir: './repos', dataflowRunner: createDataflowRunnerFactory() });` |
134
191
 
135
192
  The API server's optional rack package is needed only for
136
193
  `e3-api-server --repos ./repos --rack`. Portable handlers receive a runner
137
- factory and do not import hub code. Custom environments, remote function
138
- delegation, automatic reconnect after hub restart and embedded UI integration
139
- are deferred.
194
+ factory and do not import hub code. `tools` and `image` environments on a
195
+ rack, remote function delegation, automatic reconnect after hub restart and
196
+ embedded UI integration are deferred.
@@ -18,6 +18,7 @@ repo/
18
18
  ├── locks/ # locks, their holders, and how far each says it has got
19
19
  ├── gc/ # gc beside running work: unreachable notes, a stepped run's parts
20
20
  ├── envs/ # built execution environments
21
+ ├── rack/ # rack delegation: its policy, and the layers kept of environments built for a rack
21
22
  └── tmp/ # scratch and staged uploads
22
23
  ```
23
24