@elaraai/east-codex-plugin 1.0.109 → 1.0.110
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +1 -1
- package/index.json +90 -11
- package/package.json +1 -1
- package/skills/e3/SKILL.md +13 -9
- package/skills/e3/reference/cli.md +3 -2
- package/skills/e3/reference/rack.md +20 -6
- package/skills/e3-create/SKILL.md +7 -2
- package/skills/east-py-datascience/SKILL.md +8 -0
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@elaraai/east-codex-plugin",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.110",
|
|
4
4
|
"description": "East programming language ecosystem - a statically typed, expression-based language for business logic, data processing, UI definitions, and durable execution",
|
|
5
5
|
"license": "AGPL-3.0-or-later",
|
|
6
6
|
"type": "module",
|
package/skills/e3/SKILL.md
CHANGED
|
@@ -1,8 +1,12 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: e3
|
|
3
|
-
description: "East Execution Engine (e3): durable, content-addressed dataflow for East programs. Use when (1) authoring a package with @elaraai/e3 — e3.input, e3.task, e3.streamTask with e3.output and e3.partition (bounded memory, work split over an input), e3.customTask, e3.function, e3.package, e3.export, East.importFunction; (2) records — audited state written only through mutations (e3.record with mutation.reduce/edit/patch), their indexes (index.on/key/keys, interval, area, series, summary, nested, entries; e3.index reads one in a task or function) and migrations (migration.value/rows/rekey); (3) the e3 CLI — repo, package, workspace, dataset, task, dataflow run, run, call, mutate, seed, history, watch, convert, auth, and the cores and memory a run may use; (4) driving e3 from code (@elaraai/e3-api-client, @elaraai/e3-core, e3-api-server); (5) caching, reactive re-runs and garbage collection; (6) delegating task bodies
|
|
3
|
+
description: "East Execution Engine (e3): durable, content-addressed dataflow for East programs. Use when (1) authoring a package with @elaraai/e3 — e3.input, e3.task, e3.streamTask with e3.output and e3.partition (bounded memory, work split over an input), e3.customTask, e3.function, e3.package, e3.export, East.importFunction; (2) records — audited state written only through mutations (e3.record with mutation.reduce/edit/patch), their indexes (index.on/key/keys, interval, area, series, summary, nested, entries; e3.index reads one in a task or function) and migrations (migration.value/rows/rekey); (3) the e3 CLI — repo, package, workspace, dataset, task, dataflow run, run, call, mutate, seed, history, watch, convert, auth, and the cores and memory a run may use; (4) driving e3 from code (@elaraai/e3-api-client, @elaraai/e3-core, e3-api-server); (5) caching, reactive re-runs and garbage collection; (6) delegating task bodies, ad-hoc runs… See the detailed scope below."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
+
## Detailed skill scope
|
|
7
|
+
|
|
8
|
+
East Execution Engine (e3): durable, content-addressed dataflow for East programs. Use when (1) authoring a package with @elaraai/e3 — e3.input, e3.task, e3.streamTask with e3.output and e3.partition (bounded memory, work split over an input), e3.customTask, e3.function, e3.package, e3.export, East.importFunction; (2) records — audited state written only through mutations (e3.record with mutation.reduce/edit/patch), their indexes (index.on/key/keys, interval, area, series, summary, nested, entries; e3.index reads one in a task or function) and migrations (migration.value/rows/rekey); (3) the e3 CLI — repo, package, workspace, dataset, task, dataflow run, run, call, mutate, seed, history, watch, convert, auth, and the cores and memory a run may use; (4) driving e3 from code (@elaraai/e3-api-client, @elaraai/e3-core, e3-api-server); (5) caching, reactive re-runs and garbage collection; (6) delegating task bodies, ad-hoc runs and function calls to a rack (e3 rack); (7) e3 in a browser with @elaraai/e3-web. UI tasks are e3-ui's ui().
|
|
9
|
+
|
|
6
10
|
# East Execution Engine (e3)
|
|
7
11
|
|
|
8
12
|
e3 runs East programs as a durable dataflow over content-addressed data: every
|
|
@@ -85,8 +89,8 @@ What do you need?
|
|
|
85
89
|
│ ├─ A UI task → ui(name, [inputs], fn) from @elaraai/e3-ui (the e3-ui skill)
|
|
86
90
|
│ ├─ A function written in python or another node package → East.importFunction(pkg, name, FunctionType) in the body
|
|
87
91
|
│ └─ Bundle and export → e3.package(name, version, ...items); e3.export(pkg, zipPath)
|
|
88
|
-
├─ Operate it from the shell → the e3 CLI (reference/cli.md): deploy, run, read and write datasets, mutate records, gc
|
|
89
|
-
├─ Delegate local task bodies to a rack → e3 rack enroll + policy; dataflow run --rack, watch --start --rack (reference/rack.md)
|
|
92
|
+
├─ Operate it from the shell → the e3 CLI (reference/cli.md): deploy, run, read and write datasets, call functions (named, or `call --ir` a function's IR file), mutate records, gc
|
|
93
|
+
├─ Delegate local task bodies to a rack → e3 rack enroll + policy; dataflow run --rack, watch --start --rack; one task or one call: run --rack, call --rack (reference/rack.md)
|
|
90
94
|
│ ├─ A split task's pieces fill the rack → booked by what each reserves, capped half as much again; --memory-limit bounds the cap, --gpu asks for the GPU
|
|
91
95
|
│ └─ A task with its own python/node environment → nothing new in the package: its layers are built for the rack's platform
|
|
92
96
|
└─ Drive it from code
|
|
@@ -127,15 +131,15 @@ A `file` source:
|
|
|
127
131
|
naming the input and the first differing field) and again at deploy, before
|
|
128
132
|
the workspace is touched;
|
|
129
133
|
- is hashed first, so a delivery the store already holds costs that one read;
|
|
130
|
-
- is taken in without being read whole or modified
|
|
131
|
-
a new file, never edit it in place. A collection is taken in on the runners —
|
|
134
|
+
- is taken in without being read whole or modified. A collection is taken in on the runners —
|
|
132
135
|
east-c, or east-node where there is no east-c — in pieces of its segments, as
|
|
133
136
|
many at once as the budget allows: each row is walked, the bytes the Writer
|
|
134
137
|
writes for it kept as they stand and any other row written again, so the
|
|
135
138
|
delivery is stored as the Writer's value whichever writer wrote it, and a new
|
|
136
139
|
delivery stores only the segments that changed. A deploy or an upload
|
|
137
140
|
stopped part way takes up again from the pieces it finished. Any other value
|
|
138
|
-
is taken in
|
|
141
|
+
is taken in as a copy, unless the store holds it already, so the file stays
|
|
142
|
+
its user's to write again;
|
|
139
143
|
- may hold no segment of more than 64 MiB, the most a collection is read in at
|
|
140
144
|
once. A value encoded whole is one such segment, and so can be a segment of
|
|
141
145
|
wide rows from an older Writer, which bounded a segment by its element count
|
|
@@ -393,7 +397,7 @@ const change = e3.streamTask('change', {
|
|
|
393
397
|
| Definition | What it is |
|
|
394
398
|
|---|---|
|
|
395
399
|
| `e3.customTask(name, inputs, outputType, command, { environment? })` | A task that runs a bash script: `command($, inputPaths, outputPath)` builds it from the staged beast2 input files, and it writes the output as a beast2 file. |
|
|
396
|
-
| `e3.function(name, fn, { runner?, environment? })` | A named function, called by name with arguments (`e3 call`, the API, `Func.bind`) and its result returned inline: not in the dataflow. An argument is a value, or in a call through a workspace one of its datasets, read where it is stored. A call through a workspace is the server's: it outlives the request that starts it — the client starts it and polls it to its end — and runs until its timeout, the server's ceiling (ten minutes locally) unless it asks for less; it can be cancelled; and the server keeps it for its lifetime, an hour for a success or a cancel and a week for any other end: who made it, its arguments and drafts, the versions of the inputs it read, its value and its logs (reference/from-code.md). A call through the package runs within one request and keeps nothing. A result is at most 1 MiB, so a large one belongs in a task. A `custom` runner must speak the runner CLI (`<command> -i <arg>… -o <out> <ir>`). |
|
|
400
|
+
| `e3.function(name, fn, { runner?, environment? })` | A named function, called by name with arguments (`e3 call`, the API, `Func.bind`) and its result returned inline: not in the dataflow. An argument is a value, or in a call through a workspace one of its datasets, read where it is stored. A call through a workspace is the server's: it outlives the request that starts it — the client starts it and polls it to its end — and runs until its timeout, the server's ceiling (ten minutes locally) unless it asks for less; it can be cancelled; and the server keeps it for its lifetime, an hour for a success or a cancel and a week for any other end: who made it, its arguments and drafts, the versions of the inputs it read, its value and its logs (reference/from-code.md). A call through the package runs within one request and keeps nothing. A result is at most 1 MiB, so a large one belongs in a task. A `custom` runner must speak the runner CLI (`<command> -i <arg>… -o <out> <ir>`). `e3 call --rack` runs a local repository's call on an enrolled rack (reference/rack.md), and `e3 call --ir <file>` calls a function from its IR file, with no package (reference/cli.md). |
|
|
397
401
|
| `e3.function(name, { inputs, drafts?, args, output, runner?, environment?, async? }, fn)` | A function that reads the workspace as well: `inputs` (an input, a task's output, a record or an index of one, `e3.index(record.indexes.<name>)`, as a task takes them) are read as stored when each call launches, pinned at their hashes and named in the result's `inputs`; the caller passes `args` alone. The body takes the inputs, then the arguments. `e3.package` collects the inputs and what they depend on; nothing reruns when one changes. The inputs are read one at a time and the call waits for no run, so a call during a run may read one input before the run rewrites it and another after: the hashes in `inputs` say which. A record and its indexes are read at one state of it. Called through a workspace only (`e3 call -w`); an unassigned input is `invalid`, naming it and why. `drafts` lists the records, each read by one of `inputs` (itself, or an index of it), that a call may lay **unsaved edits** over: a draft is a mutation call that is not committed, `{ record, mutation, args }` (`e3 call -w <ws> <fn> [args…] --draft <record>.<mutation> [args…]`, reference/cli.md). A call's drafts run in the order passed, as a Save runs them, each over the state the one before it left, and the body reads each drafted record, and its indexes, at the state that leaves; a dataset argument naming a drafted record is read as saved, at the state the drafts were laid over. Nothing is saved: no commit, no record moved, no task re-run — the record and what reads it stay as they were for every reader. The call keeps its drafts as it keeps its arguments, for whoever may read the call, and each draft's mutation runs as a Save's does, an execution recorded and cached. The result's `inputs` name the drafted state's hash, and its `drafts` the commit each drafted record stood at. A draft of a record the function does not list, or one the record does not take (an unknown mutation, arguments of the wrong number or type), is `invalid`, naming the draft; one the record refuses ends the call as its Save would end, the body not run: `failed` or `timed_out` (the first line of stderr says which draft, or that the drafts were all laid as the timeout ended), or `conflict` when the record changed under the edit. Listing a record is the author's word that the body is fit to run over edits nobody saved, so a function that sends a message or writes elsewhere lists none. ❗ Refused: an `e3.partition` input, the `custom` runtime with inputs, and in `drafts` what is no record or a record no input reads. `FunctionSpec` names the spec's type. |
|
|
398
402
|
| `e3.package(name, version, ...items)` | Bundles the items and everything they depend on — pass the last tasks, records and functions; a package passed in contributes its contents. The name and version must be valid file names. |
|
|
399
403
|
| `e3.export(pkg, zipPath, { functions?, cores? })` | Writes the package zip, resolving every `East.importFunction` (a workspace package is exported by the export itself, `cores` packages at once, the CPUs by default; `functions` lists manifests built elsewhere). |
|
|
@@ -523,10 +527,10 @@ Read the one a task needs; each holds its subject whole.
|
|
|
523
527
|
|---|---|
|
|
524
528
|
| [reference/record-indexes.md](reference/record-indexes.md) | Secondary indexes, interval and area indexes, series and summary indexes, nested series indexes, entries indexes, and reading an index inside a task or a function (`e3.index`) |
|
|
525
529
|
| [reference/record-migrations.md](reference/record-migrations.md) | `migration.value` / `rows` / `rekey`, and how a record's type changes |
|
|
526
|
-
| [reference/cli.md](reference/cli.md) | Every `e3` command: repositories, packages, workspaces, datasets, tasks, runs, calls, records, watch, convert, auth, gc |
|
|
530
|
+
| [reference/cli.md](reference/cli.md) | Every `e3` command: repositories, packages, workspaces, datasets, tasks, runs, calls (named, or from an IR file), records, watch, convert, auth, gc |
|
|
527
531
|
| [reference/from-code.md](reference/from-code.md) | `@elaraai/e3-api-client`, `@elaraai/e3-core`, `e3-api-server` |
|
|
528
532
|
| [reference/browser.md](reference/browser.md) | `@elaraai/e3-web`: serveUnits, serveE3, createWebE3, and what a browser cannot do |
|
|
529
|
-
| [reference/rack.md](reference/rack.md) | `e3 rack enroll` / `policy`, routing a run to a rack, how a rack is filled and what a run prints, tasks with an environment on a rack, `@elaraai/e3-rack` |
|
|
533
|
+
| [reference/rack.md](reference/rack.md) | `e3 rack enroll` / `policy`, routing a run to a rack, one task or one function call on a rack (`run --rack`, `call --rack`), how a rack is filled and what a run prints, tasks with an environment on a rack, `@elaraai/e3-rack` |
|
|
530
534
|
| [reference/repository.md](reference/repository.md) | A local repository's layout, and upgrades |
|
|
531
535
|
|
|
532
536
|
## Related skills
|
|
@@ -33,9 +33,10 @@ a task is `<ws>.<task>` and a mutation `<record>.<mutation>`.
|
|
|
33
33
|
| `e3 dataflow run [repo] <ws> [--filter <task>] [--force \| --force-task <task>…] [-j <n>] [--memory <size>] [-v]` | Run what is stale, then print the outputs' paths. `--filter`: one task, by its exact name, and the tasks it depends on. `--force`: re-run every task the run runs (under `--filter`, its task); `--force-task`, repeated for each: re-run the tasks it names, whose dependents re-run when an output changes. A task the graph lacks, or one `--filter` leaves out, refuses the run. A task whose input nothing has set — an input never set, or a record declared with no initial value and not yet seeded — is left waiting, with its dependents (`[WAIT]`, `Waiting:`), and the run completes without them; they run once it is set. |
|
|
34
34
|
| `e3 task list [repo] <ws>` | Tasks with the status of each one's latest run. |
|
|
35
35
|
| `e3 task logs [repo] <ws.task> [-n <lines>] [--all] [--follow] [--execution <task>/<inputs>/<id>]` | The last 200 lines by default, of the latest attempt over the task's current inputs, or its latest run; `--follow` stays on that execution; `--execution`: one unit's log, as a split task's log names it (local). |
|
|
36
|
-
| `e3 run <repo> <pkg[@ver].task> [inputs.beast2…] -o <out> [--force] [-v] [-j <n>] [--memory <size>]` | Run one task ad hoc. |
|
|
37
|
-
| `e3 call <repo> <pkg[@ver].fn> [args…] [-o <out.beast2>] [-j <n>] [--memory <size>] [-v]` · `e3 call <repo> -w <ws> <fn> [args…]` | Call a function; each argument is an `.east` literal or a `.beast2`/`.json`/`.east` file, and with `-w` one written `@<keypath>` (`@.inputs.sales`) is that dataset of the workspace, pinned at its hash when the call starts. A function that declares `inputs` reads them from the workspace itself, so it takes its arguments alone and is called with `-w`; without it, it is refused, naming its inputs. The value goes to stdout, and the function's stdout and stderr to stderr. Through a server's workspace the call is the server's, kept for its lifetime: it runs as long as its timeout allows, a call longer than a second is named (`Call <id> runs on`), and Ctrl-C asks the server to stop it (exit 130). |
|
|
36
|
+
| `e3 run <repo> <pkg[@ver].task> [inputs.beast2…] -o <out> [--force] [-v] [-j <n>] [--memory <size>] [--rack\|--rack-only]` | Run one task ad hoc (a bare package name runs its greatest version). Each input file is taken in as `--from-file` takes one, a collection never read whole, so a task split over it runs in pieces, and any other value as a copy, so the file stays yours to write again; the output is written as the value, a segment at a time, the file at `-o` replaced and never written into (a link there is written where it leads; a file you may not write is refused; a pipe or a terminal, as `/dev/stdout`, is written into; in a directory you may not write, a file nothing else links to is written into, and one another name links to is refused). `--rack`: on an enrolled rack (local; reference/rack.md). |
|
|
37
|
+
| `e3 call <repo> <pkg[@ver].fn> [args…] [-o <out.beast2>] [-j <n>] [--memory <size>] [-v]` · `e3 call <repo> -w <ws> <fn> [args…]` | Call a function; each argument is an `.east` literal or a `.beast2`/`.json`/`.east` file, and with `-w` one written `@<keypath>` (`@.inputs.sales`) is that dataset of the workspace, pinned at its hash when the call starts. A function that declares `inputs` reads them from the workspace itself, so it takes its arguments alone and is called with `-w`; without it, it is refused, naming its inputs. The value goes to stdout, and the function's stdout and stderr to stderr. Through a server's workspace the call is the server's, kept for its lifetime: it runs as long as its timeout allows, a call longer than a second is named (`Call <id> runs on`), and Ctrl-C asks the server to stop it (exit 130). `--rack`/`--rack-only`: on an enrolled rack (local; reference/rack.md), its drafts laid here. |
|
|
38
38
|
| `… -w <ws> <fn> [args…] --draft <record>.<mutation> [args…]` | Lay an unsaved edit over a record the function lists in `drafts`, and save nothing: the mutation runs as `e3 mutate` would run it, its arguments read the same way, and the function reads the record at the state that leaves. Repeatable, laid in the order written, each over the state the one before it left. `--draft` reads every word up to the next option, so it is written after the call's own arguments, or the drafts are ended with `--`; a draft ends where its mutation's arguments do, and a word that starts with a dash is written `--draft=<word>`. Each draft is read against its mutation before anything runs: one that names no record or mutation of the workspace, or has too few arguments or one of the wrong type, is refused, named by its place (`Draft 2 (jobs.move): …`). A draft the record refuses ends the call before the function runs, and exits 1: `Draft jobs.patch (2 of 3) failed with exit code: 1; the function did not run`, `… had not been laid when the call timed out after <n>ms; …`, `The call's drafts were laid, and its timeout of <n>ms ended before the function ran: …` when laying them took all of the call's time, or, when the record changed under the edit, `Draft jobs.patch (2 of 3) conflicted: … — the edit no longer matches the record; re-read it and call again`. |
|
|
39
|
+
| `e3 call <repo> [-w <ws>] --ir <fn.beast2> [args…] [--runner east-node\|east-c\|east-py] [--platform <pkg>…\|none]` | Call the function an IR file holds (`.beast2`, `.json` or `.east`), its signature read from the IR, named by the file's name. It runs on `--runner`'s runtime (east-node by default) with that runtime's standard package, or the packages `--platform` names (`none`: none, a platform-free call). Through a server it needs `-w` and is one of the workspace's one-shot calls, under the caller's grant: a reader runs a platform-free one alone. It reads no record, so it takes no `--draft`. |
|
|
39
40
|
| **Records** (`-w <ws>` required) | |
|
|
40
41
|
| `e3 mutate <repo> <record.mutation> [args…] -w <ws> [-j <n>] [--memory <size>] [-v]` | Apply a mutation. |
|
|
41
42
|
| `e3 seed <repo> <record> <file.beast2> -w <ws> [-j <n>] [--memory <size>]` | Commit a record's first state from a beast2 file, as its `$init`: for a record declared with no initial value, unassigned until then (local). The file is taken in as a `file` source is, never read whole; a record that holds a state is refused. |
|
|
@@ -13,6 +13,8 @@ Part of the `e3` skill.
|
|
|
13
13
|
| `e3 rack policy [repo] show\|unset <glob>\|reset` | Inspect rules, remove an exact pattern, or delete the saved policy. | `e3 rack policy . show` |
|
|
14
14
|
| `e3 dataflow run <repo> <ws> --rack\|--no-rack\|--rack-only` | Select rack routing for this run; rack-only disables capacity spill but keeps infrastructure fallback: work waits two minutes for an unhealthy rack to come back, and is leased once more after a failed transfer, before it runs locally. ❗ Refused for remote URLs. | `e3 dataflow run . dev --rack` |
|
|
15
15
|
| `e3 watch <source> <repo> <ws> --start --rack` | Reuses a rack session across watch runs and refreshes capacity; a second Ctrl-C stops the watch at once. ❗ `--rack` and `--rack-only` require `--start`. | `e3 watch src/main.ts . dev --start --rack` |
|
|
16
|
+
| `e3 run <repo> <pkg.task> <inputs…> -o <out> --rack\|--rack-only` | Runs one task ad hoc on a rack, whatever the policy selects (a rule naming the task still sizes it); a split task's pieces and merges are leased, its planning stays here. Writes the cache and history a local run writes, says where it ran and why, and ends with the rack summary. A run the cache answers opens no rack session. Ctrl-C leaves its leases and stops the output's write, the file at `-o` left as it was (exit 130). ❗ Refused for remote URLs. | `e3 run . acme-forecast.train ./sales.beast2 -o ./model.beast2 --rack` |
|
|
17
|
+
| `e3 call <repo> [-w <ws>] <fn> [args…] --rack\|--rack-only` · `e3 call <repo> --ir <file> [args…] --rack` | Runs one call on a rack, whatever the policy selects, named by `fn:<name>` (an IR file's by its name): its own lease, sized as a whole task, its value back inline or, past 192 KiB, as an object. What no rack can run runs here; `[RACK] fn:<name> → <rack>` or `[LOCAL] fn:<name> (<reason>)` goes to stderr. Its time is its `fn:<name>` rule's `--timeout` when a rule sets one, longer or shorter than the CLI's ten minutes, however long: its drafts and its function share it, the function's part counted from when its lease is queued. A call no rack may run is judged before a rack session opens, and runs here at once, starting no hub. Ctrl-C leaves the lease; a value that came as Ctrl-C did is still printed. ❗ Refused for remote URLs. | `e3 call . -w dev train_once @.inputs.sales --rack` |
|
|
16
18
|
| `e3 rack status\|list\|remove <id-or-label>\|approve <id-or-label>\|stop` | Inspect fleet, revoke a rack, admit the boot a quarantined rack last reported, or stop the shared hub. Status never starts an absent hub, and lists how its last leases ended, each with its reason. | `e3 rack status` |
|
|
17
19
|
| `e3 rack config [--listen <host:port>] [--allow <ip,...>] [--idle-exit <min>] [--advertise-url <origin>] [--tls\|--no-tls] [--require-approval\|--no-require-approval]` | Configure the shared listener (off by default; HTTPS with the hub's own certificate unless `--no-tls`); every rack and data URL is built on `--advertise-url` when set; zero idle-exit disables automatic shutdown; `--require-approval` quarantines a rack after each reboot until `e3 rack approve` admits it (enrollment admits the first boot). ❗ Plain HTTP only on the loopback or a private address, with `--allow`. | `e3 rack config --listen 10.0.0.5:7331` |
|
|
18
20
|
|
|
@@ -47,7 +49,11 @@ though it was leased again). On a rack that admits by
|
|
|
47
49
|
its resources a split task's pieces are queued together, booked by what each
|
|
48
50
|
reserves, and the rack's guard keeps them from running its host out of memory;
|
|
49
51
|
a first run's piece reserves e3's estimate, as a local run does, so it is as
|
|
50
|
-
wide as a later run and measures no piece first.
|
|
52
|
+
wide as a later run and measures no piece first. A function call goes to a rack
|
|
53
|
+
only from `e3 call --rack`, and is fit for one as a task is: its runner's tier,
|
|
54
|
+
the environment it declares, and its program, scanned for host access (`[LOCAL]
|
|
55
|
+
fn:read_config (host-coupled:fs_read_file)`). A local API server's calls,
|
|
56
|
+
mutations and intake stay local. Stock runner
|
|
51
57
|
configurations qualify, and so does a task that declares a `python` or `node`
|
|
52
58
|
environment, with the project platform packages its runner names; commands, a
|
|
53
59
|
platform package with no environment (`non-stock-platform:<name>`) and
|
|
@@ -65,7 +71,15 @@ x86_64-manylinux_2_28: building` then `[ENV] 3f9a2b1c0d4e built in 48 s: 4
|
|
|
65
71
|
layers, 2.1 GB (1 new)`. No container runtime on either machine; nothing is
|
|
66
72
|
compiled for the rack; a second run builds nothing, and a change to the
|
|
67
73
|
project's code makes one small new layer. The layers are kept in the
|
|
68
|
-
repository's `rack/env-layers`, which `e3 repo gc` sweeps.
|
|
74
|
+
repository's `rack/env-layers`, which `e3 repo gc` sweeps. On a rack that
|
|
75
|
+
composes environments over its tier (`environments-over-tier`), a python
|
|
76
|
+
environment is built over the tier's image, without an interpreter and
|
|
77
|
+
without what the image holds the same (`[ENV] … over py-datascience: building`,
|
|
78
|
+
`[ENV] … built in 41 s: 2 layers, 6.2 MB (2 new), 31 distributions from the
|
|
79
|
+
tier`); one whose lock leaves out the image's Python is built whole, and so is
|
|
80
|
+
one whose build over the image fails. torch
|
|
81
|
+
stays in the tier for a member that asks for it through
|
|
82
|
+
`elaraai-east-py-datascience[torch,lightning]` and lists no torch of its own. ❗ What cannot go
|
|
69
83
|
stays local, with its reason on the task's `[LOCAL]` line: `environment:tools`
|
|
70
84
|
/ `environment:image` (its kind), `environment:windows-host` (a Windows
|
|
71
85
|
launching machine), `no-rack-for-environment:<tier>` (no rack composes them),
|
|
@@ -171,7 +185,7 @@ reach. A data URL lasts 15 minutes, which a transfer may use whole; an object is
|
|
|
171
185
|
at most 5 GiB. The hub serves one rack eight transfers at once (one more is told
|
|
172
186
|
to ask again), and reads an object from a byte on (`Range`), so a transfer a slow
|
|
173
187
|
link cut short resumes; it keeps a rack's idle connection for 75 seconds.
|
|
174
|
-
A hub and the e3 that calls it speak control protocol
|
|
188
|
+
A hub and the e3 that calls it speak control protocol 5 exactly:
|
|
175
189
|
a hub an older e3 started is asked to exit once idle, and `e3 rack stop` with
|
|
176
190
|
that e3 replaces it now. A rack's agent and its source speak rack protocol 3:
|
|
177
191
|
an agent of another protocol is refused by name each time it asks, and
|
|
@@ -186,11 +200,11 @@ sessions of the same canonical repository and e3 release.
|
|
|
186
200
|
|
|
187
201
|
| Signature | Description | Example |
|
|
188
202
|
| --- | --- | --- |
|
|
189
|
-
| `openRackRunner({ repoPath, workspace, storage, label, budget?, settings?, policy?, rackOnly?, unreachableWaitMs?, onPlacement?, onEnvironment? })` | Opens a runner/session; preserve the caller's existing budget and process settings, use `rackSlots` to extend orchestrator width, and close in `finally`. Under `rackOnly`, work waits `unreachableWaitMs` (two minutes) for an unhealthy rack before it runs locally. `onPlacement` is told where each task or
|
|
203
|
+
| `openRackRunner({ repoPath, workspace, storage, label, budget?, settings?, policy?, rackOnly?, selected?, unreachableWaitMs?, onPlacement?, onEnvironment? })` | Opens a runner/session; preserve the caller's existing budget and process settings, use `rackSlots` to extend orchestrator width, and close in `finally`. The runner's `runDetached` leases a call (`{ functionName }` names it) for the time its spec's limits give it; `rackCallTimeoutMs(policy, name)` is the time a `fn:<name>` rule gives a call, or null. `selected` takes every task and call to a rack whatever the policy's rules and `enabled` say, as `e3 run`/`e3 call --rack` do. Under `rackOnly`, work waits `unreachableWaitMs` (two minutes) for an unhealthy rack before it runs locally. `onPlacement` is told where each task, unit or call runs; `onEnvironment` that an environment's layers are being built for a rack, and what the build came to; `unattached` says why no session opened. | `const rack = await openRackRunner({ repoPath, workspace: 'dev', storage, label: 'run', budget });` |
|
|
190
204
|
| `createDataflowRunnerFactory(options?)` | Pass to `createServer({ dataflowRunner })`; one session per API dataflow, using the server's shared budget. Explicit server opt-in overrides policy.enabled; rules still apply. | `const server = await createServer({ reposDir: './repos', dataflowRunner: createDataflowRunnerFactory() });` |
|
|
191
205
|
|
|
192
206
|
The API server's optional rack package is needed only for
|
|
193
207
|
`e3-api-server --repos ./repos --rack`. Portable handlers receive a runner
|
|
194
208
|
factory and do not import hub code. `tools` and `image` environments on a
|
|
195
|
-
rack,
|
|
196
|
-
embedded UI integration are deferred.
|
|
209
|
+
rack, a local API server's calls on a rack, split calls on a rack, automatic
|
|
210
|
+
reconnect after hub restart and embedded UI integration are deferred.
|
|
@@ -226,8 +226,13 @@ and the task runs anywhere, with no python or Node package where it runs. To
|
|
|
226
226
|
add one, build it and list it under the name the app imports; e3 finds the
|
|
227
227
|
member in the uv / npm workspace and exports it itself (the **e3** skill).
|
|
228
228
|
|
|
229
|
-
**Build order / semantics.**
|
|
230
|
-
|
|
229
|
+
**Build order / semantics.** A python member's environment installs the
|
|
230
|
+
member's own locked dependencies, the runtime, and the stock platform packages
|
|
231
|
+
its task's runner names, each with the extras the workspace declares it with —
|
|
232
|
+
not what the workspace root or another member lists, nor the member's
|
|
233
|
+
development groups, so a heavy dependency costs only the members that use it;
|
|
234
|
+
node members must be
|
|
235
|
+
`tsc`-built before export
|
|
231
236
|
(the root `build` runs `--workspaces` first — both the `./platform` and the
|
|
232
237
|
`./functions` entries are read from `dist/`); C tools must be `make`-built
|
|
233
238
|
before `npm run start`. Add more packages later with the same flags — each new
|
|
@@ -239,6 +239,14 @@ Rules:
|
|
|
239
239
|
config)`.
|
|
240
240
|
- Optional dependencies gate at call time: a function raises
|
|
241
241
|
`NotImplementedError` naming the missing extra (`east-py-datascience[causal]`).
|
|
242
|
+
- **Ask for torch through the extras**
|
|
243
|
+
(`elaraai-east-py-datascience[torch,lightning]`), and list no `torch` of the
|
|
244
|
+
project's own: the extras pin the torch the release was tested with, which
|
|
245
|
+
its rack tier image holds, so a task on a rack is not sent torch. That holds
|
|
246
|
+
for torch from PyPI: a pin with no local label is met by a PyTorch index's
|
|
247
|
+
`+cpu` or `+cu130` build too, which is not the image's, and is sent.
|
|
248
|
+
- **Python 3.12 or later**, and xgboost 3.4 or later beside torch: an earlier
|
|
249
|
+
xgboost brings another NCCL library over torch's, and `import torch` fails.
|
|
242
250
|
|
|
243
251
|
For the East-value API (eager methods, `coerce_to`, `to_numpy` / `to_torch`,
|
|
244
252
|
`@East.platform_function`), load the **east-py** skill.
|