@elaraai/east-claude-plugin 1.0.99 → 1.0.100

package/skills/e3/SKILL.md

@@ -29,6 +29,10 @@ writing or changing East code:
    programs, exact and far cheaper, and the signatures omit the runtime rules
    that make East code correct.
 
+Tested e3 packages (inputs, tasks, stream tasks and partitions, custom tasks,
+functions, records with their mutations and migrations) are in the index too:
+search with `package: "e3"`.
+
 Nothing is injected for you; the search is the step.
 
 ## Quick Start
@@ -68,162 +72,24 @@ What do you need?
 │   ├─ A named function (RPC)    → e3.function(name, fn, config?) — called by name, persists nothing
 │   ├─ Audited state             → e3.record(name, type, { initial?, indexes?, mutations?, migrations? }), declared whole
 │   │   ├─ Its writes            → mutations: (mutation) => ({ name: mutation.reduce | edit | patch })
-│   │   ├─ An order to page in   → indexes: (index) => ({ name: index.on([...fields]) | index.key(fn) | index.keys(fn) })
+│   │   ├─ An order to page in   → indexes: (index) => ({ name: index.on([...fields]) | index.key(fn) | index.keys(fn) }) (reference/record-indexes.md)
 │   │   ├─ Read by a time window or a 2D window → index.interval | index.area
 │   │   ├─ Read a block of keys over time tiles, or their totals → index.series | index.summary
 │   │   └─ Read what rows nest (jobs, a value a day, child rows, trees) a tile at a time, with totals and subtotals
 │   │       → index.nested({ collections, tile, summary? })
-│   ├─ A record's type changes   → migrations: (migration) => [migration.value | rows | rekey(name, fn), …], in order
+│   ├─ A record's type changes   → migrations: (migration) => [migration.value | rows | rekey(name, fn), …], in order (reference/record-migrations.md)
 │   ├─ A UI task                 → ui(name, [inputs], fn) from @elaraai/e3-ui (the e3-ui skill)
 │   ├─ A function written in python or another node package → East.importFunction(pkg, name, FunctionType) in the body
 │   └─ Bundle and export         → e3.package(name, version, ...items); e3.export(pkg, zipPath)
-├─ Operate it from the shell     → the e3 CLI (below): deploy, run, read and write datasets, mutate records, gc
-├─ Delegate local task bodies to a rack → e3 rack enroll + policy; dataflow run --rack, watch --start --rack
+├─ Operate it from the shell     → the e3 CLI (reference/cli.md): deploy, run, read and write datasets, mutate records, gc
+├─ Delegate local task bodies to a rack → e3 rack enroll + policy; dataflow run --rack, watch --start --rack (reference/rack.md)
 └─ Drive it from code
-    ├─ A server, over HTTP       → @elaraai/e3-api-client
+    ├─ A server, over HTTP       → @elaraai/e3-api-client (reference/from-code.md)
     ├─ A local repository        → @elaraai/e3-core
     ├─ Serve repositories        → e3-api-server --repos <dir>, or createServer()
-    └─ e3 in a browser, no server → @elaraai/e3-web: serveUnits, serveE3, createWebE3 ("Running e3 in a browser")
+    └─ e3 in a browser, no server → @elaraai/e3-web: serveUnits, serveE3, createWebE3 (reference/browser.md)
 ```
 
-## Rack delegation from a local repository
-
-**Setup and routing**
-
-| Signature | Description | Example |
-| --- | --- | --- |
-| `e3 rack enroll --listen <host:port> [--allow <ip,...>] [--advertise-url <origin>] [--label <label>] [--no-wait]` | Starts the user-wide hub and prints the source origin, the hub's certificate SHA-256, the matching release and a single-use token for the companion's hidden prompts. Waits for a new registration, not activation. Behind an SSH reverse tunnel, listen on the loopback and advertise the relay's origin. | `e3 rack enroll --listen 127.0.0.1:7331 --allow 127.0.0.1 --advertise-url https://relay.example:7401` |
-| `e3 rack policy [repo] set <glob> rack\|local [--size <size>] [--timeout <min>] [--allow-host-access]` | First whole-task-name match wins; `*` and `?` are wildcards. Size: serverless, small, medium, large, xlarge. | `e3 rack policy . set 'train_*' rack --size large` |
-| `e3 rack policy [repo] enable\|disable\|mode <opt-in\|auto>` | Enable/disable saved opt-in; `mode auto` selects eligible unmatched tasks. Defaults: disabled, opt-in. | `e3 rack policy . enable` |
-| `e3 rack policy [repo] defaults [--no-spill] [--claim-timeout <sec>] [--retry-task-failures-locally]` | Defaults: large, 1440-minute timeout, spill on, 30-second claim recheck, no task-failure retry. | `e3 rack policy . defaults --no-spill` |
-| `e3 rack policy [repo] show\|unset <glob>\|reset` | Inspect rules, remove an exact pattern, or delete the saved policy. | `e3 rack policy . show` |
-| `e3 dataflow run <repo> <ws> --rack\|--no-rack\|--rack-only` | Select rack routing for this run; rack-only disables capacity spill but keeps infrastructure fallback. ❗ Refused for remote URLs. | `e3 dataflow run . dev --rack` |
-| `e3 watch <source> <repo> <ws> --start --rack` | Reuses a rack session across watch runs and refreshes capacity; a second Ctrl-C stops the watch at once. ❗ `--rack` and `--rack-only` require `--start`. | `e3 watch src/main.ts . dev --start --rack` |
-| `e3 rack status\|list\|remove <id-or-label>\|approve <id-or-label>\|stop` | Inspect fleet, revoke a rack, admit the boot a quarantined rack last reported, or stop the shared hub. Status never starts an absent hub. | `e3 rack status` |
-| `e3 rack config [--listen <host:port>] [--allow <ip,...>] [--idle-exit <min>] [--advertise-url <origin>] [--tls\|--no-tls] [--require-approval\|--no-require-approval]` | Configure the shared listener (off by default; HTTPS with the hub's own certificate unless `--no-tls`); every rack and data URL is built on `--advertise-url` when set; zero idle-exit disables automatic shutdown; `--require-approval` quarantines a rack after each reboot until `e3 rack approve` admits it (enrollment admits the first boot). ❗ Plain HTTP only on the loopback or a private address, with `--allow`. | `e3 rack config --listen 10.0.0.5:7331` |
-
-The local process owns graph planning, repository and cache. After a cache
-miss, eligible plain task, map and merge bodies can execute on the rack; input
-closures and every output object transfer by hash. The session verifies the
-whole output graph before success. Placement never changes task/cache hashes.
-Local bodies and fallback share the original cores/memory budget; rack bodies
-use remote capacity. Calls, mutations and intake stay local. Stock runner
-configurations qualify; commands, custom environments/platforms and unscannable
-IR stay local. Host file/environment/network access stays local unless a rule
-deliberately grants `--allow-host-access` and its resources exist on the rack.
-
-The listener serves HTTPS with the hub's own self-signed certificate, made on
-first use and kept in `hub-tls.beast2`; a rack pins it by its SHA-256, which
-enrollment prints, so TLS runs end to end through any relay or tunnel. The
-usual path is an SSH reverse tunnel from the developer machine to a relay the
-rack reaches, with the hub on the loopback (`--allow 127.0.0.1`) and the relay's
-origin advertised. Removing the certificate file, with the hub stopped, makes a
-new one, which every worker must then pin again. `--no-tls` is for a TLS
-terminator in front of the hub; the hub refuses plain HTTP on a wildcard or
-public address, or without `--allow`. A listener on every address needs
-`--advertise-url` to enroll.
-
-The handoff targets a prepared independent worker VM with the matching verified
-`e3-rack` release. Its `configure --allocation <host-provisioned-path>` prompts
-for the source origin, the certificate SHA-256 for an HTTPS origin the hub
-serves, the worker label and settings; `enroll --allocation <path>` prompts for
-the one-time token. Origins, pins and credentials are encrypted together. A
-plain-HTTP origin needs `--allow-vpn-http` on both commands and host
-authorization for that exact endpoint, and belongs only on a path encrypted end
-to end, such as an SSH tunnel, never across a VPN hub that decrypts traffic.
-Never put a source address or token in worker command arguments or plaintext
-files. The local mint intentionally displays its token; do not log that output.
-
-Enrollment does not start services or authorize task claims. The local CLI
-observes a new registration (optionally filtered by label); compare it with the
-worker's result. Before a first heartbeat, status is `enrolled-not-started`.
-After managed resume, a ready heartbeat advertises free plus occupied slots;
-busy workers stay eligible. Fenced, draining or stopping workers withdraw their
-capacity and show `unavailable`. After 45 seconds without a heartbeat, readiness
-expires; lease polling cannot prolong it. Source boot approval, when required,
-remains an additional gate. This advertisement does not authorize host resource
-admission or removal of service/network quarantine.
-Resources and mutable whole-GPU assignments belong to the host lifecycle, not
-local enrollment flags. The companion accepts up to 32 mutually trusted sources
-in one worker. After draining and stopping its services and sockets, use
-`configure --allocation <path> --add-source`, confirm the shared trust boundary,
-and enter the additional origin privately. It prints the new source UUID for
-`enroll --allocation <path> --source <UUID>`; add `--allow-vpn-http` to both for a
-plain-HTTP origin. All sources must be enrolled before startup. Each has its own identity,
-client, cache and completion journal; actual execution shares one worker-wide
-task/GPU limit. Per-source capacity reports are not additive reservations.
-A worker is the trust boundary: a shared agent process can access all its
-sources' credentials, so separate clients, and a cloud deployment and developer
-hubs, need separate worker VMs. The hub keeps no task data; what a worker keeps
-between tasks is its own policy, and an agent that keeps nothing works unchanged. Worker-local `source list --allocation <path>`
-prints UUIDs/enrollment status; `--show-origins` adds origins only on a terminal.
-After drain/stop and source-side revocation, `source remove` or `source reset`
-with `--allocation <path> --source <UUID> --acknowledge-revocation` asks for UUID
-confirmation. Reset prints a new UUID for fresh enrollment; old pending results
-and caches stay in the retired namespace. Other sources and retained data are
-preserved, and host endpoint policy must be reconciled before activation. The
-flag acknowledges revocation; it does not revoke remotely.
-
-Worker-local `registry status --allocation <path>` reports only whether encrypted
-registry credentials are present; it can run while active and does not test access.
-After drain/stop, `registry configure --allocation <path>` privately prompts for
-username/token without re-entering settings. `registry clear --allocation <path>`
-asks for the installation UUID and removes only the stored access. Both require
-the same release, the writer lock and no unresolved enrollment/service transition;
-they preserve current identities, rotations, settings, journals and cached data.
-Credentials stay in the existing encrypted bundle, with no Docker login file.
-A later activation receives them; services remain stopped and the next boot fenced.
-Repeating configure with the same values or clear when absent does not write.
-Clearing/replacing does not revoke a token remotely or purge cached images.
-
-The companion's
-`reconcile-services` applies a host-provided next allocation and service budgets
-after a completed drain, with stopped units, the previous plan hash and exact
-operation/boot binding. It retains ciphertext/journals and network quarantine;
-it does not move a GPU, start a worker or change its executable release. For a
-compatible release change, invoke the target verified executable's `switch-release`
-with the same drain/plan bindings and unchanged resource budgets at the next
-allocation generation. It checks both managed-state contracts, updates only the
-runtime version in the current encrypted configuration, and preserves identities
-and journals through interruption or downgrade. Both four-file releases must stay
-available; a downgrade is a new operation, never a saved-state restore. Services
-stay stopped and quarantined. If the selected release never resumed, the exact
-recorded previous executable's `recover-release` binds the completed release
-receipt, current plan and new operation at another generation increase. It
-requires the original drain or a valid fenced journal with no operation at the
-failed generation, verifies both releases' recorded hashes, and preserves current
-credentials and journals. A prior resume attempt, even across reboot, refuses
-recovery. General journal repair remains unfinished. Named host management, source aliases, network
-activation, incompatible-state migrations and hardware acceptance remain
-companion cloud #253 work. This and durable completion retry are implemented in
-companion PR #254, separately from the original main protocol pin.
-
-Policy is `<repo>/rack/policy.beast2`; hub state is `~/.e3/rack` (or
-`E3_RACK_HOME`). A hub `dataflow run` starts exits after 15 idle minutes, and a
-run waits up to 10 seconds for an enrolled rack to report healthy (with none
-enrolled, or a policy that selects no task for a rack, it starts at once): run
-`e3 rack hub` as a service, or set `--idle-exit 0`, for a hub racks can always
-reach. A data URL lasts 15 minutes, which a transfer may use whole; an object is
-at most 5 GiB. The control schema currently requires protocol 2 exactly.
-Agents must report bundled e3 at least the project's release. Infrastructure
-failure falls back locally with a separate execution attempt, as does an
-attempt the rack stopped on its own; the run's own cancellation and the task's
-timeout never do. One cancelled subscriber does not stop work shared by other
-sessions of the same canonical repository and e3 release.
-
-**Programmatic attachment** (`@elaraai/e3-rack`)
-
-| Signature | Description | Example |
-| --- | --- | --- |
-| `openRackRunner({ repoPath, workspace, storage, label, budget?, settings?, policy?, rackOnly? })` | Opens a runner/session; preserve the caller's existing budget and process settings, use `rackSlots` to extend orchestrator width, and close in `finally`. | `const rack = await openRackRunner({ repoPath, workspace: 'dev', storage, label: 'run', budget });` |
-| `createDataflowRunnerFactory(options?)` | Pass to `createServer({ dataflowRunner })`; one session per API dataflow, using the server's shared budget. Explicit server opt-in overrides policy.enabled; rules still apply. | `const server = await createServer({ reposDir: './repos', dataflowRunner: createDataflowRunnerFactory() });` |
-
-The API server's optional rack package is needed only for
-`e3-api-server --repos ./repos --rack`. Portable handlers receive a runner
-factory and do not import hub code. Custom environments, remote function
-delegation, automatic reconnect after hub restart and embedded UI integration
-are deferred.
-
 ## Authoring a package
 
 ### Inputs — `e3.input(name, type, source?)`
@@ -604,665 +470,12 @@ encoded size, so wide rows get segments of a few rows and a one-row edit stays
 cheap. A view reads every field of the rows it shows, so keep a bulky payload it
 does not display in a second record keyed the same way.
 
-#### Secondary indexes — `index.on([...fields])`, `index.key(fn)`, `index.keys(fn)`
-
-A second collection over a Dict record, in another sort order, maintained in the
-same commit, so a view by an attribute is a page rather than a scan.
-`index.on([...])` names row fields in index order: one field uses its value as
-the key, several use a Struct in that order, and a Set field gives an entry per
-element. It also carries field metadata for `Data.bindPaged(record, { indexes })`
-and `Slice.bindRecord` (see **e3-ui**). `index.key(fn)` gives one computed entry
-per row; `index.keys(fn)` returns a Set, one entry per element (a row naming five
-resources appears five times, an empty set not at all); the `value` option of
-each is a projection a view renders without touching the record. The functions
-are pure and synchronous, since an index maintained commit by commit must equal
-its rebuild to the byte. Its name — its key in `indexes` — is an identifier,
-never `primary` (the record's own order). Read through one with `index=<name>` on a page or key search; `e3
-reindex` rebuilds. A deploy builds an index the package declares and the state
-lacks, drops one it no longer declares, and runs nothing for the rest; a build
-is a split task over the record.
-
-```typescript
-const plans = e3.record('plans', PlansType, {
-  initial: new Map(),
-  indexes: (index) => ({
-    by_due: index.on(['due']),
-    by_owner_due: index.on(['owner', 'due']),
-    by_owner: index.key(East.function([StringType, PlanType], StringType, ($, _id, plan) => plan.owner), {
-      value: East.function([StringType, PlanType], StringType, ($, _id, plan) => plan.title),
-    }),
-    by_resource: index.keys(East.function([StringType, PlanType], SetType(StringType), ($, _id, plan) => plan.resources)),
-  }),
-  mutations: (mutation) => ({ /* add_plan, reschedule, patch — above */ }),
-});
-const pkg = e3.package('planning', '1.0.0', plans);   // the record whole: its indexes and mutations with it
-```
-
-#### Interval and area indexes — `index.interval`, `index.area`
-
-These indexes answer more than an order — as do the series and summary indexes
-below — each with a builder method of its own whose keys the SDK generates.
-Underneath each is an ordinary index — the same build,
-mutation, storage, paging and `e3 reindex` — and the record Slice reads a
-window through either (see **e3-ui**). Their generated keys are query
-machinery: edits and sessions always target the record's primary key.
-
-A generated key is a struct whose own fields are e3's: `on` holds the
-declaration's leading fields, by their names, at their key types (a Set field's
-element type) — a struct with no fields when it lists none — and the kind's own
-parts sit beside it. So no name is reserved: a row may name a field anything,
-`area`, `tile` and `__area` among them, and an index may be led by it.
-
-| Index | Key | e3's parts |
-|---|---|---|
-| interval | `{ on, node, at }` | the tree node an entry is filed at; the instant it is ordered by, at the `from` field's type |
-| area | `{ on, area }` | the cell's Morton code times 32 plus its level; −1 for a row not yet placed |
-| series | `{ on, key, level, tile }` | the value of the field `key` names; the level an event is filed at, and the start of that level's tile (#1433) |
-| summary | `{ on, key, tile }` | the value of the field `key` names; the tile's start |
-
-A definition's `on` is the leading fields alone; its kind's options say the
-rest (`events.indexes.by_time.interval`, `racks.indexes.racks_by_area.area`,
-`jobs.indexes.jobs_by_press_week.series`).
-
-**`index.interval({ on?, from, to })`** keeps events visible while they
-overlap a selected time window. `from` and `to` are the row's DateTime or
-Option<DateTime> fields; optional `on` fields are exact leading choices:
-
-```typescript
-const events = e3.record('events', EventsType, {
-  initial: new Map(),
-  indexes: (index) => ({
-    by_time: index.interval({ from: 'start', to: 'end' }),
-    by_status_time: index.interval({ on: ['status'], from: 'start', to: 'end' }),
-  }),
-});
-```
+A record's indexes (`index.on` / `key` / `keys`, `interval`, `area`,
+`series`, `summary`, `nested`) are in
+[reference/record-indexes.md](reference/record-indexes.md), and its migrations
+in [reference/record-migrations.md](reference/record-migrations.md).
 
-Its key is `{ on, node, at }`. It stores at most three entries per event per
-leading-field combination, independent of duration. The record Slice adapter resolves a half-open
-`[from, to)` window with at most 55 disjoint bounded queries per exact prefix;
-each matching original record occurs once, including events that started
-earlier or span the whole window. It never scans the record or guesses a
-maximum duration. An event ending at `from` is outside; an instant at `from`
-is inside and one at `to` is outside. A missing or non-positive end is an
-instant. A missing start remains selectable as Unset/backlog.
-
-Use `on: ['start']` only when the intended query is **starts within**, since it
-cannot preserve carry-in events. When both declarations are available, a Slice
-range on the start field uses the interval declaration. The generated interval
-keys are query machinery; record edits and sessions always target the original
-primary key. Calendar's `Calendar.events(binding, { slice, entries, … })`
-requires a matching interval declaration and the primary `entries` binding to
-confirm saves even after an edit leaves the selected window.
-
-**`index.area({ on?, x, y, width?, height?, at?, cell })`** reads the rows
-whose box meets a 2D window — what a floor plan's viewport shows — the interval
-index's twin:
-
-```typescript
-// RacksType: Dict<String, { hall: String, x: Option<Float>, y: Option<Float>, w: Float, h: Float }>
-const racks = e3.record('racks', RacksType, {
-  initial: new Map(),
-  indexes: (index) => ({
-    racks_by_area: index.area({ x: 'x', y: 'y', width: 'w', height: 'h', at: 'centre', cell: 0.01 }),
-    racks_by_hall_area: index.area({ on: ['hall'], x: 'x', y: 'y', width: 'w', height: 'h', at: 'centre', cell: 0.01 }),
-  }),
-});
-```
-
-| Option | The box's |
-|---|---|
-| `x`, `y` | place: two Float fields, or two Option<Float> fields whose `none` is a row not yet placed |
-| `width`, `height` | size: two Float fields, or neither for a point |
-| `at` | `'corner'`, the default — `x`, `y` is the box's minimum corner, as a zone's is — or `'centre'`, as an item's is |
-| `cell` | finest cell: its side in world units, about the smallest thing drawn — 0.01 for a floor in metres, 1 for one in millimetres. It is rounded down to a power of two, so the keys' arithmetic is exact on every runtime, and has no default: a wrong one costs a silent tenfold read |
-| `on` | exact leading fields, as the interval index's; a Set field fans out |
-
-Each option takes only the row's fields of its type, so a wrong one does not
-compile, and the definition checks them again, naming the option, the field and
-its type. The domain is the square ±`cell`·2²⁷ world units around zero
-(±1,048 km at a 0.01 cell, which rounds to 2⁻⁷), in 29 levels. Each row is
-filed once, under the deepest level whose cell is at least twice its box's
-larger side, in the cell holding its box's centre: the key
-`{ on, area: Integer }`. A row not yet placed files under `area` −1, which the
-record Slice reads as the unplaced clause; one outside the domain, not finite,
-or larger than half of it files under the root, which every window reads. The index **carries its rows**: each
-entry's projection is the whole row, so a window reads the index alone and no
-record segment, and it takes no `value`. The record Slice reads a viewport,
-grown by an eighth of its larger side on each side, as at most 266 disjoint
-searches per exact prefix, sent in one request, which together hold every row
-whose box meets it, each once — beside rows of the cells searched that it
-misses, which the reader draws or not.
-
-#### Series and summary indexes — `index.series`, `index.summary`
-
-What a planning board reads: the events on a block of keys — a Plan's
-resources — over a span of time, and, zoomed out past what a screen can draw,
-how busy each key is. Both file a row's event by the key it is on and by time
-tiles — `day`, `week` (ISO, from Monday), `month`, `quarter` or `year`, in UTC,
-half-open — the key before the time, so a key's entries over any span are
-contiguous: a series index's `{ on, key, level, tile }`, a summary index's
-`{ on, key, tile }`.
-
-```typescript
-// JobsType: Dict<String, { press: String, start: Option<DateTime>, end: Option<DateTime>, … }>
-const jobs = e3.record('jobs', JobsType, {
-  initial: new Map(),
-  indexes: (index) => ({
-    jobs_by_press_week: index.series({ key: 'press', from: 'start', to: 'end', tile: 'week' }),
-    jobs_by_press_month: index.summary({ key: 'press', from: 'start', to: 'end', tile: 'month' }),
-  }),
-});
-```
-
-| Option | |
-|---|---|
-| `key` | the field naming what an event is on: any field but a Set |
-| `from`, `to` | the event's start and end: DateTime or Option<DateTime> fields. A row with no start is filed nowhere; no end, or one at or before the start, is an instant |
-| `tile` | a series index's finest level, and a summary index's tile: `day`, `week`, `month`, `quarter` or `year` |
-| `on` | exact leading fields, before the key; a Set field fans out |
-
-**A series index** is an ordinary index whose keys the SDK generates, and it
-**carries its rows**, as the area index does. It files each event **once**, at
-the finest level from its `tile` up to the year whose shortest tile holds the
-event's length — a day, a week, 28 days, 90 days, 365 days — under that level's
-tile holding its start; an instant at `tile`; past 365 days, under each year it
-touches. An event filed at a level ends within the tile after the one holding
-its start, so a reader reads a block of keys over any span from the index alone
-as one range a key a level — that level's tiles from two before the span to one
-after it — in one request, whatever the span's width. **A summary index** holds,
-under each key and tile, `{ count, ms }`: how many rows touch the tile, and the
-milliseconds of it they cover — a row's shares
-add up to its length, and an instant's is none. Each entry stands for every row
-of its key and names none, so its window's `key` is Null, a read of it joins
-nothing, and a read that needs its rows — a joined read, a placement
-(`datasetPlace`) or a membership read (`datasetWithin`) — is refused, naming the
-index; the record's describe says it is a summary. A write
-changes its totals by what its rows took away and gave — its program reads the
-record, never the summary — and the apply adds that to each total, dropping one
-whose count reaches zero; the totals are Integers, so a maintained summary is the
-one `e3 reindex` sums, byte for byte, and a build split into pieces adds the
-pieces' totals of a key. A change that would leave a total no rebuild sums — a
-count below zero, or time with no row — fails the write, naming the total and
-`e3 reindex`, which rebuilds it; a total past the largest Integer fails the
-write or the build, naming a finer tile.
-
-A series index files an event past a year under each year it touches, as many
-as 4,096: one touching more is refused where it is written, naming it and its
-span. A summary splits an event over its tiles, as many as 4,096 — eleven years
-of days — and one touching more is refused the same way, naming the tile:
-declare a coarser tile. Each option takes only the row's fields of its type, as
-the area index's do, and the definition checks them again; a `value` is
-refused, and so is an `on` field that is the key, the start or the end: the key
-follows the leading fields, and the tiles are cut by the start and the end.
-
-#### Nested series indexes — `index.nested`
-
-What a planning board reads when a record's rows nest what it draws: a
-machine's jobs and its load a day at a time, a hall's members, an assembly's
-parts to any depth. One declaration a record lists every collection its rows
-nest, each field with a builder saying what it holds. e3 files each item apart
-from its row — once, by its collection, its row, its node and its time — keeps
-each node's totals a summary tile at a time, and each parent's subtotals at
-every level; and the record's other indexes carry the rows without those
-collections, so a page of rows reads none of them and a window of items reads
-no row.
-
-```typescript
-// MachinesType: Dict<String, { name, site, status, jobs: Dict<String, Job>, load: Dict<DateTime, Float> }>
-const machines = e3.record('machines', MachinesType, {
-  initial: new Map(),
-  indexes: (index) => ({
-    machines_in_time: index.nested({
-      collections: {
-        jobs: index.nested.items({ from: 'start', to: 'end' }),  // things people drag: by id, their times fields
-        load: index.nested.values(),                              // a value per instant: keyed by the instant
-      },
-      tile: 'week',
-      summary: 'month',
-    }),
-    machines_by_site: index.on(['site']),                         // pages each machine without its jobs and load
-  }),
-});
-// Each member's load, and each hall's mean of its members', at every level from the day up.
-const halls = e3.record('halls', HallsType, {
-  indexes: (index) => ({
-    halls_in_time: index.nested({
-      collections: { members: index.nested.rows({ load: index.nested.values({ rollup: 'mean' }) }) },
-      tile: 'day',
-      summary: 'month',
-    }),
-  }),
-});
-// Parts to any depth, each part's jobs filed under every part above it too.
-const assemblies = e3.record('assemblies', AssembliesType, {
-  indexes: (index) => ({
-    assemblies_in_time: index.nested({
-      collections: {
-        parts: index.nested.rows({
-          jobs: index.nested.items({ from: 'start', to: 'end' }, { rollup: true }),
-          parts: index.nested.recursive(),
-        }),
-      },
-      tile: 'week',
-      summary: 'month',
-    }),
-  }),
-});
-```
-
-| Builder (`index.nested.…`) | A field holding |
-|---|---|
-| `items({ from, to })`, `items({ at })` | `Dict<String, Struct>`: things people drag, by id; their times are DateTime or Option<DateTime> fields — no start files an item nowhere, no end (or one at or before its start) makes it an instant. `{ rollup: true }` files each under every ancestor too |
-| `values({ rollup? })` | `Dict<DateTime, V>`: a value per instant. `rollup` keeps every parent's subtotals |
-| `rows({ … })` | `Dict<String, Struct>`: child rows, their own collections declared the same way |
-| `recursive()` | `Dict<String, Self>`: more rows like the one listing it, to any depth |
-
-TypeScript types `collections` field by field from the record's type alone: a
-misnamed time, or a builder on the wrong kind of field, is an error where it is
-written, and its message says what fits. The definition checks the same again,
-in words, for a declaration TypeScript could not see. A record has one nested
-series index, which takes no `on` and no `value`.
-
-**What it files.** Every entry is keyed `{ collection, row, path, run, level,
-tile, item }` (e3-types' `NestedEntryKeyType`): the collection's place, the
-field names from the row down (`members` then `load`); the row's key; the Dict
-keys from the row down to the node (`[]` for the row, `m-3` for a member); the
-run; the level and tile; and the item. A place and a path are held as a key
-holds a list, folded from the last: `.step (key="members", rest=.step
-(key="load", rest=.end))`. Each run of a node is one contiguous range a level:
-
-| Run | Holds |
-|---|---|
-| `items` | each item once, as a series index files an event — at the level its length fits, past a year under each year — carrying it (`.item .jobs (…)`); a value, and an `{ at }` item, at the declared tile alone |
-| `totals` | where `summary` is declared, each node's `{ count, ms, values }` a summary tile at a time; for a Float or Integer collection, `values` is the sum, the least, the greatest and the last there, taken in key order |
-| `subtotals(rollup)` | for `values({ rollup })`, every parent's subtotal by each rollup at each level from `tile` to the year, where a descendant's value touches the tile: a node holding values folds its own, a parent aggregates its children's, in key order |
-| `subtree`, `subtreeTotals` | for `items(…, { rollup: true })`, each item under every ancestor too, `.under (path, id)`, and each ancestor's totals of them |
-
-A rollup is `'sum'`, `'mean'` (a mean of means up the tree), `'min'`, `'max'`,
-`'count'` (the leaf rows beneath that hold a value), a fold and an aggregate
-named apart (`{ fold: 'last', aggregate: 'sum' }`, a balance), or several.
-
-**Reading.** A reader searches the index's leading fields — `[collection, row,
-path, run, level, tile]`, each as East prints it — as a range a row a level: a
-block of fifty machines' jobs over three weeks is fifty times four ranges in one
-`datasetFindKeys` request, and one `datasetGetRuns` read of the entries they
-select, each job carried whole; a parent's week subtotals are one range; a
-part's subtree is one range a level.
-
-**The lean rows.** Beside a nested series index, every other index of the
-record that carries nothing or the row carries the row without the listed
-collections — each listed collection empty, child rows kept with theirs empty —
-so a page of rows read without a join reads no nested item and no segment of
-the record's own. An index with a `value` of its own carries its own. A
-binding of the record's indexes lists the nested index by name beside any that
-carry the lean rows — `Data.bindPaged(machines, { indexes: ['machines_by_site',
-'machines_in_time'] })` (see **e3-ui**).
-
-**Its span.** Beside every nested series index e3 keeps its span, an index of
-the record's named `<index>_span` (e3-types' `nestedSpanName`), built with it
-and never declared: under each collection's place and each month its items or
-values touch (`NestedSpanKeyType`, `{ collection, tile }`), how many of them
-touch it and the milliseconds of it its items cover, summed across the rows as a
-summary index's totals are, and a month let go once nothing touches it. A
-collection's first and last month held are its span across the record — what a
-Plan's horizon covers for the series reading it — and its months' counts are
-how much it holds in each, which the horizon's strip draws as its bars (see
-**e3-ui**). Reading it is one search a collection, an exact match on its
-place, and one read of the first and last entry of each. An index the record
-declares under that name is refused.
-
-**Writes.** A row a write updates by a patch is maintained by the patch: e3
-walks it along the declaration — each collection's Dict patch, and each child
-rows' — and compares the entries of only the items it names, with the totals of
-the summary tiles they touch and the subtotals of the tiles they touch at each
-level. A replace on the way, which a recursive type's patch always is, and an
-inserted or deleted row are compared whole. Both ways give the same delta, and a maintained
-index is the one `e3 reindex` rebuilds, byte for byte.
-
-#### Migrations — how a record's type changes
-
-| Form | Function | Runs as |
-|---|---|---|
-| `migration.value(name, fn)` | `(Old) => New`, over any record | one unit; its runner opens the state lazily |
-| `migration.rows(name, fn)` | a Dict's rows, `(K, V1) => V2` with the keys kept, or an Array's elements, `(T1) => T2` | a task split over the state: a piece at a time, in parallel, each piece cached |
-| `migration.rekey(name, fn)` | a Dict's entries, `(K1, V1) => { key: K2, value: V2 }`, or a Set's elements, `(T1) => T2` | the same split task, its pieces merged by key: two Dict rows landing on one key fail the deploy, naming it; two Set elements are one |
-
-A record's steps are one chain: the list `migrations` returns, in order, each
-step taking the record as the step before it leaves it, and the last leaving it
-as its declared type — each checked where the record is declared.
-A step's name is an identifier, unique on the record, and a workspace records the
-steps it has applied by name — so moving a step's code, or re-exporting it with a
-newer e3, does not run it again, and an edit to an applied step's body is not
-detected. The functions are pure, synchronous East.
-
-```typescript
-const RowV1Type = StructType({ title: StringType });
-const RowV2Type = StructType({ title: StringType, owner: StringType });
-const plans = e3.record('plans', DictType(StringType, RowV2Type), {
-  initial: new Map(),
-  migrations: (migration) => [
-    migration.rows('add_owner', East.function([StringType, RowV1Type], RowV2Type, ($, _id, row) => ({ title: row.title, owner: 'unassigned' }))),
-    // The next version adds its step here, after this one.
-  ],
-});
-const pkg = e3.package('planning', '2.0.0', plans);
-```
-
-What a deploy does with each record, decided before it writes anything:
-
-| The workspace holds | Against the package's chain | The deploy |
-|---|---|---|
-| no state | — | mints `$init` from the initial value, the whole chain counted applied; declared with no initial value, leaves it unassigned (`unassigned`) for `e3 seed` |
-| state | every step applied, the type unchanged | keeps it, with a `$deploy` commit when the package changed |
-| state | a proper prefix applied | runs the rest, a `$migrate:<name>` commit each |
-| state | every step applied, but the type changed | refused: a type change needs a migration, or `--schema reset` |
-| state | applied steps the chain does not start with | refused, naming the steps applied and those declared |
-| a record the package no longer declares | — | refused unless `--allow-drop-records` |
-
-`--schema <policy>` on `e3 workspace deploy` and `e3 watch`: `migrate` (the
-default), `fail` (run no migration; refuse) or `reset` (reset the record to its
-initial value, with a `$reset` commit (`reset`); one declared with no initial
-value goes back to unassigned (`unassign`), for `e3 seed` again). `--plan` prints what the deploy would do
-to each record, index and input and writes nothing — from a zip or a source it
-imports nothing, and a server plans only a package it holds. The steps run
-before the deploy writes: a failed step leaves the workspace as it was, and a
-deploy run again is served the steps that finished from the cache.
-
-What a deploy does with each input — a dataset an `e3.input` declares, which
-people set:
-
-| The input now | Its type | The deploy |
-|---|---|---|
-| holds the value the deployed package gave it, or is new | — | the new package's value (`package`) |
-| someone set it | — | the new package's value under `--inputs reset`, the default, and says it reset it (`reset`) |
-| someone set it | the new package's | kept, under `--inputs keep-edited` (`keep`) |
-| someone set it | changed | the new package's value, and says why (`reset`) |
-| the deployed package took it from a file, and it holds a value | — | the new package's value under either policy, since a value set since cannot be told from the file's, and says so (`reset`) |
-| the new package takes it from a file | — | its file, as every deploy does (`file`); a deploy that reads no file, a server's, leaves it unset for its client to send |
-| the new package no longer declares it | — | gone, with its value (`drop`) |
-
-## The e3 CLI
-
-Every `<repo>` is a local path or a server URL, `http(s)://<host>/repos/<name>`;
-where `[repo]` is optional it defaults to `$E3_REPO`, then `.`. Dataset paths are
-flat, `<ws>.<name>` — an input or a task output, with "did you mean" suggestions;
-a task is `<ws>.<task>` and a mutation `<record>.<mutation>`.
-
-| Command | Does |
-|---|---|
-| **Repositories** | |
-| `e3 repo create [repo] [--exist-ok]` | Create one (`--exist-ok`: succeed if it exists). |
-| `e3 repo status [repo]` · `e3 repo remove [repo] [-r]` | Show objects, packages, workspaces · remove (`-r`: its workspaces first). |
-| `e3 repo gc [repo] [--dry-run] [--keep-runs <n>] [--keep-days <d>] [--min-age <ms>]` | Drop old run history, then what nothing names (below). |
-| `e3 repo list <server>` | The repositories a server holds. |
-| **Packages** | |
-| `e3 package import [repo] <zip>` · `export [repo] <pkg[@ver]> <zip>` | Import · export (`--quiet`: errors only). |
-| `e3 package list [repo]` · `remove [repo] <pkg[@ver]>` | |
-| **Workspaces** | |
-| `e3 workspace create [repo] <ws>` · `list [repo]` · `status [repo] <ws>` · `remove [repo] <ws>` | `status` shows tasks, datasets and locks. |
-| `e3 workspace copy [repo] <from> <to>` | Make `<to>` what `<from>` is now — its package and every input, record and task output, so nothing runs again — made, or replaced whole with its runs. It writes refs only, so its cost follows the number of datasets, never their size; a write to either afterwards leaves the other as it was. Refused while a dataflow runs in `<to>`. |
-| `e3 workspace deploy [repo] <ws> <pkg[@ver]>` | Deploy an imported package. |
-| `… --from-zip <zip>` · `… --from-source <src.ts> [--functions <manifest…>]` | Import the zip (or bundle the source) and deploy, creating the workspace. |
-| `… [--schema <policy>] [--inputs <policy>] [--allow-drop-records] [--plan] [--skip-file-sources] [-j <n>] [--memory <size>] [--quiet]` | A record's policy, an input's, and `file` sources; says how far it has got (below), `--quiet` aside. `-j`: the `file` sources taken in at once, and the runner processes their intake units, the migrations and the index builds run on, and with `--from-source` the workspace packages exporting their functions at once; `--memory`: what those may reserve (local). |
-| `e3 workspace export [repo] <ws> <zip> [--name <n>] [--version <v>]` | The workspace's state as a package. |
-| **Datasets** | |
-| `e3 dataset get [repo] <ws.name> [-f east\|json\|beast2]` | Print a value. |
-| `e3 dataset set [repo] <ws.name> <file> [--type <spec> \| --type-file <path>]` | Write from `.east`, `.beast2`, `.json` or `.csv` (JSON and CSV need the type). |
-| `e3 dataset set [repo] <ws.name> --from-file <path.beast2> [-j <n>] [--memory <size>]` | Take a beast2 file in as the value, as a deploy takes a `file` source in, saying how far it has got; `-j`, `--memory`: the budget its intake units run under (local). |
-| `e3 dataset list [repo] <ws> [-l]` · `status [repo] <ws.name>` · `find [repo] <ws> <pattern>` | Paths (`-l`: kind, type, status, size) · one dataset · by substring or glob. |
-| **Running** | |
-| `e3 dataflow run [repo] <ws> [--filter <task>] [--force \| --force-task <task>…] [-j <n>] [--memory <size>] [-v]` | Run what is stale, then print the outputs' paths. `--filter`: one task, by its exact name, and the tasks it depends on. `--force`: re-run every task the run runs (under `--filter`, its task); `--force-task`, repeated for each: re-run the tasks it names, whose dependents re-run when an output changes. A task the graph lacks, or one `--filter` leaves out, refuses the run. A task whose input nothing has set — an input never set, or a record declared with no initial value and not yet seeded — is left waiting, with its dependents (`[WAIT]`, `Waiting:`), and the run completes without them; they run once it is set. |
-| `e3 task list [repo] <ws>` | Tasks with the status of each one's latest run. |
-| `e3 task logs [repo] <ws.task> [-n <lines>] [--all] [--follow] [--execution <task>/<inputs>/<id>]` | The last 200 lines by default, of the latest attempt over the task's current inputs, or its latest run; `--follow` stays on that execution; `--execution`: one unit's log, as a split task's log names it (local). |
-| `e3 run <repo> <pkg[@ver].task> [inputs.beast2…] -o <out> [--force] [-v] [-j <n>] [--memory <size>]` | Run one task ad hoc. |
-| `e3 call <repo> <pkg[@ver].fn> [args…] [-o <out.beast2>] [-j <n>] [--memory <size>] [-v]` · `e3 call <repo> -w <ws> <fn> [args…]` | Call a function; each argument is an `.east` literal or a `.beast2`/`.json`/`.east` file, and with `-w` one written `@<keypath>` (`@.inputs.sales`) is that dataset of the workspace, pinned at its hash when the call starts. |
-| **Records** (`-w <ws>` required) | |
-| `e3 mutate <repo> <record.mutation> [args…] -w <ws> [-j <n>] [--memory <size>] [-v]` | Apply a mutation. |
-| `e3 seed <repo> <record> <file.beast2> -w <ws> [-j <n>] [--memory <size>]` | Commit a record's first state from a beast2 file, as its `$init`: for a record declared with no initial value, unassigned until then (local). The file is taken in as a `file` source is, never read whole; a record that holds a state is refused. |
-| `e3 history <repo> <record> -w <ws> [--limit <n>] [--from <hash>] [--delta]` | Commits, newest first (`--delta`: what each changed, per target — a summary's totals added to as `±n`; local). |
-| `e3 reindex <repo> <record> -w <ws> [--index <name>] [-j <n>] [--memory <size>]` · `e3 compact <repo> <record> -w <ws>` | Rebuild indexes (local) · collapse the history to a `$compact` root, the state kept. |
-| **Development** | |
-| `e3 watch <src.ts> <repo> <ws> [--start] [--schema <p>] [--inputs <p>] [--abort-on-change] [--functions <manifest…>] [-j <n>] [--memory <size>]` | Redeploy on each change (and run, with `--start`). |
-| `e3 convert [input] [--from <f>] [--to <f>] [--type <spec>] [-o <out>]` | Convert between `.east`, `.json` and `.beast2`. |
-| `e3 completion install [--shell <s>]` · `uninstall` · `bash` \| `zsh` \| `fish` | Shell completion. |
-| **Servers** | |
-| `e3 auth login <server> [--no-browser]` · `logout <server>` · `status` · `token <server>` · `whoami [server]` | OAuth2 device flow; credentials per server (`token` prints one for curl). |
-
-**The budget.** `-j`/`--jobs <n>` is the cores — runner processes in flight, a
-task or a unit each — and `--memory <size>` (`8G`, `512M`) what they may reserve
-between them. The defaults are `E3_JOBS` and `E3_MEMORY`, else what e3 may use:
-its CPU affinity capped by a cgroup quota, and the cgroup's `memory.max` or
-physical memory, less a reserve. A unit of a split task reserves the largest
-peak a unit of its stage has reached in the run, so a stage runs its first unit
-alone and then fans out; on Linux and macOS a guard stops the newest unit when
-the runners together pass the budget, and runs it again once it fits. A deploy
-takes its `file` sources in `-j` at a time, their pieces' intake units running
-under the same budget. A source deploy, or a `watch`, exports the workspace
-packages its imported functions come from `-j` at a time, a process each, on
-this machine; against a server, as many at once as this machine has CPUs. The
-commands that run East — `workspace deploy`,
-`dataset set --from-file`, `dataflow run`, `run`, `call`, `mutate`, `seed`,
-`reindex` and `watch` — take the flags for a local repository; against a server the CLI
-refuses them, since the server runs the work under its own budget
-(`e3-api-server -j`, `--memory`).
-
-**A deploy says how far it has got**, on stderr. Each `file` source prints a
-line once it is in: its size, time and rate, and how it was taken in
-(`unchanged, already in the store`, `taken in by east-c`, or `carried` for a
-value that is not a collection), with why the first time a runner fell back to
-another. A terminal also keeps a live line for the files in flight, with their
-pieces, the rate and the time left across them. A source deploy prints a line
-for each workspace package whose functions it links: `exported <n> function(s)
-of <pkg> (east-py export-functions)`, or `reused <n> function(s) of <pkg>,
-unchanged since its last export` when the export cache held them. Against a
-server, the deploy line
-says what the job is doing, such as migrating a record or building an index;
-the deliveries then upload a few at a time, each printing a line once it is in,
-and the live line says what the server's commit is doing with each in flight.
-While a deploy runs, its lock carries the same progress for `e3-ui` or any
-client to read (`workspaceLockStatus`). A workspace deployed for the first time
-has no status until its deploy ends, so the lock is the only place to see it.
-
-**`-v`** passes `-v` to the runners, which print to the task's logs, identically
-on every runtime, how each input was read — opened lazily, and what reading it
-came to, or decoded whole, and the resident memory that added — and a timing
-and peak-memory block. It never changes hashes or caching (add `--force` to see
-it for a cached task), and works against a server.
-**Where a task's time goes.** `EAST_PROFILE=1` in the environment of a local
-`e3 dataflow run` (or `run`, `call`, `mutate`) has the east-c and east-py
-runners profile each unit — the runners inherit e3's environment — and the
-task's log (`e3 task logs`) then ends with every East function and platform
-function it called, by self time, with its calls and where it was written and
-called: the python lines of a function `East.importFunction` linked in
-included. Each input read lazily is an entry too, named as `-v` names it
-(`input 1`): the time its reads spent decoding segments, a call each — keyed
-reads at random beyond the segments the runner keeps, or a scan repeated,
-which `-v`'s account counts — and not the reading function's own time.
-`EAST_PROFILE_INTERVAL=300` adds the profile so far every five minutes while a
-long task runs. east-node does not profile. Like `-v` it changes no hash: add
-`--force` for a cached task.
-**`E3_SCRATCH_DIR`** moves a local run's per-execution scratch directories
-(default `<repo>/tmp/scratch`, on the object store's disk; on tmpfs, outputs sit
-in memory until stored).
-
-**`repo gc`** keeps each workspace's last 10 runs (`--keep-runs`), every run
-from the last 7 days (`--keep-days`) and the run its current state came from,
-with every execution those runs used; every execution a workspace's current
-state is served from, so a re-run stays cached; every execution from the last 7
-days; and whatever is still running. An attempt recorded running that the local
-runner says cannot finish is recorded interrupted, and goes as any that ended.
-It then removes the objects nothing names, and staging files older than
-`--min-age` (60 s). `--dry-run` reports what would go.
-
-A package zip names the release of e3 that exported it, and an import refuses a
-zip a newer release exported, naming that release: import it with an e3 at least
-as new as the SDK that exported it. A dataset upload names the transfer protocol
-its e3 speaks, and a server refuses one that speaks another, naming both
-releases and which of them to upgrade.
-
-## Driving e3 from code
-
-**`@elaraai/e3-api-client`** talks to a server over HTTP. Every call is
-`(url, repo, …, options)`, `options` being
-`{ token: string | null, retry?, verbose?, fetch? }` (`e3 auth token <server>`
-prints a token); `fetch` is the `fetch` every request of the call goes
-through, the global one unless given — an in-page e3's `e3.fetch` (below).
-Dataset paths are `TreePath`s — an input definition's `.path`, for one. A
-request a workspace's lock refuses throws `ApiError` `workspace_locked`,
-naming the holder: its pid, boot id, command and when it took the lock.
-
-| Area | Functions |
-|---|---|
-| Repositories | `repoList(url, opts)`, `repoCreate(url, name, opts)`, `repoRemove(url, name, opts)`, `repoStatus(url, repo, opts)`, `repoRecord` (its release and upgrades), `repoGc(url, repo, gcRequest, opts)` — a job it polls; `repoGcStart` and `repoGcStatus` apart |
-| Packages | `packageList(url, repo, opts)`, `packageGet(url, repo, name, version, opts)`, `packageImport(url, repo, zipBytes, opts)`, `packageExport(url, repo, name, version, opts)` → zip bytes, `packageRemove` |
-| Workspaces | `workspaceList`, `workspaceCreate(url, repo, ws, opts)`, `workspaceCopy(url, repo, from, to, opts)` → the target, made or replaced whole, `workspaceBundle(url, repo, ws, opts)` → the workspace as it is, as a bundle's bytes — its package, every dataset's ref, the executions its tasks' outputs come from, and every object those reach — and `workspaceBundleImport(url, repo, ws, bundle, opts)` → the workspace made from one under a name no workspace has, in another repository or another e3 (`workspace_exists`, `package_exists`, `bundle_invalid`), its tasks up to date with no run; a host mounts these routes where a bundle fits in a request: the local server, and e3-web, `workspaceGet`, `workspaceStatus(url, repo, ws, opts, { paths? })` — the whole workspace's, or the datasets named and the tasks producing them, `workspaceStatusSince(url, repo, ws, etag, opts, { paths? })` → `{ status, etag }`, or `null` while the status answered under `etag` is current (a poll sends the last answer's tag back; `null` asks in full), `workspaceLockStatus(url, repo, ws, opts)` → what holds the workspace and how far it says it has got (a deploy's files and records), or `null`, `workspaceRemove`, `workspaceDeploy(url, repo, ws, 'pkg@ver', opts, { schema?, inputs?, allowDropRecords?, plan?, onProgress? })` — a job it polls, whose progress while `deploying` is the deploy's own, answering what it did with each record, index and input, `workspaceExport(url, repo, ws, opts, { name?, version? })` → zip bytes. A poll's request takes `pollRequestOptions(opts)`: one attempt, its next poll being its retry |
-| Datasets | `datasetGet(url, repo, ws, path, opts)` → `{ data, hash, size }` (a collection downloads as its segments), `datasetGetStream`, `datasetGetPage(…, window, opts)`, `datasetFindKey(…, query, opts)`, and several reads in one request (at most `DATASET_BATCH_MAX`, 10,000, each at one content hash, in a body of at most 16 MiB: a larger one is refused 413, `payload_too_large`): `datasetFindKeys(…, { queries, index?, hash? }, opts)` — key searches, `datasetGetRows(…, { keys, hash? }, opts)` — a Dict's rows by `.east` key, each with its position — or `datasetGetRows(…, { keysOf: { path, index, runs, hash }, hash? }, opts)`, by the keys the entries of some runs of another record's index name, which the server reads from the index so none is sent (a drawing's details for its window's cells), a stale `keysOf.hash` thrown as `RunsHashMismatchError` naming the record's content — and `datasetGetRuns(…, { runs, hash, index?, join? }, opts)` — the entries of several runs as one window, a rows or runs answer holding the first `count` the page budget reaches (by runs, of `runEntries`); and where a narrowing's rows sit in a record, from its index alone: `datasetPlace(…, { index, runs, offset?, limit?, hash }, opts)` → `{ segments, placed, runEntries, totalElements, hash }` — how many of the runs' entries each record segment holds (`{ segment, offset, count, rows }`), for the first `placed` of a window from `offset` (the next window from `offset + placed`), and `datasetWithin(…, { index, runs, keys, hash }, opts)` → `{ within, hash }` — whether each index entry key (`(ik=…, k=…)`) lies in one of the runs, `datasetSet(url, repo, ws, path, beast2Bytes, opts)`, `datasetSetStream(url, repo, ws, path, { size, hash, slice }, opts, { onCommitProgress? })` — a file of any size, the server's commit saying how far it has taken it in, `datasetList`, `datasetListAt`, `datasetListRecursive`, `datasetListWithStatus`, `datasetGetStatus` |
-| Runs and tasks | `dataflowExecute(url, repo, ws, { force?, filter? }, opts, { pollInterval?, timeout? })` → the result (or `dataflowExecuteLaunch` and `dataflowExecutePoll`) — `force` is `true` or the names of the tasks to re-run, `filter` one task's exact name, `dataflowCancel` — with nothing running, `dataflow_error` ("No active execution for this workspace"), `dataflowGraph`, `dataflowBudget`, `taskList`, `taskGet`, `taskExecutionList(url, repo, ws, task, opts, { limit?, before? })` — a page of the task's runs, the latest first, each naming its `executionId`, the next page the runs `before` the last one's (100 a page unless `limit` says, 1,000 at most), `taskLogs(url, repo, ws, task, { stream?, offset?, limit?, execution? }, opts)` — a chunk of the log of the task's current execution, or of the one `execution` names (`{ inputsHash, executionId }`), the chunk naming its execution and whether it has `ended` |
-| Functions | `functionList`, `functionDescribe`, `functionCall(url, repo, pkg, version, fn, { args, runner, limits }, opts)` — each argument `variant('value', beast2Bytes)`; the function runs on its own runner for any caller (`runner: none`); a runner the call names is never `custom`, and loads a platform package the function's does not only for an elevated caller; `workspaceFunctionList`, `…Describe`, `…Call(url, repo, ws, fn, request, opts)` — where an argument may also be `variant('dataset', path)`, a dataset of the workspace pinned at its hash, which the result names (`inputs`); `oneShotExecute` — a reader runs a platform-free one (stock runner, `platforms: []`, no platform call), and its result names the datasets it read (`inputs`); `splitCall(url, repo, ws, { bodyIr, args, output, then, runner, limits }, opts, { onProgress? })` — a program over a dataset's pieces, as a job it polls, under the same rule (or `splitCallLaunch`, `splitCallStatus` and `splitCallExplain`) |
-| Records | `workspaceRecordDescribe`, `workspaceRecordMutate(url, repo, ws, record, mutation, { args, actor, limits }, opts, idempotencyKey?)`, `workspaceRecordHistory(url, repo, ws, record, limit, opts, from?)`, `workspaceRecordCompact` |
-| From East | `Platform` and the `platform_*` functions (`platform_dataset_get`, `platform_dataflow_execute`, …): the same calls as platform functions, for an East program that drives a server |
-
-**`@elaraai/e3-core`** does the same on a local repository, given a storage
-backend (`new LocalStorage()`) and the repository's path; work that runs East
-takes a runner, `new LocalTaskRunner(repo)`.
-
-| Area | Functions |
-|---|---|
-| Repositories | `repoInit(path)`, `repoFind(startPath?)`, `repositoryOpen(storage, repo)` (checks the repository and applies the upgrades it owes), `repositoryUpgradeStep(storage, repo, { budgetMs, waitMs? })` → `{ owed }` — a host's job applies them so, a part per run under its time limit, each part taking a step up where the last stopped, `repoGc(storage, repo, { dryRun?, minAge?, keepRuns?, keepDays?, retention?, executionAlive? })` — holding the repository still, or, with `retention: { windowMs }`, beside running work: an object goes once unreachable for the window and not written or re-referenced since; given a runner's `executionAlive`, an attempt recorded running that cannot finish is recorded interrupted and pruned; `repoGcStep(storage, repo, step, { windowMs, pruneMs?, markMs?, concurrency?, executionAlive?, … })` runs that one step at a time, each returning the next (`GcStepType`), `pruneMs` and `markMs` bounding a prune step and a mark step for a host with a time limit |
-| Packages | `packageImport(storage, repo, zipPath)`, `packageExport(storage, repo, name, version, zipPath)`, `packageList`, `packageRemove` |
-| Workspaces | `workspaceCreate(storage, repo, ws)`, `workspaceCopy(storage, repo, from, to)`, `workspaceBundleExport(storage, repo, ws, writable, { have? })` — the workspace as a bundle to a WHATWG stream, leaving out the objects `have` says the receiving repository holds — and `workspaceBundleImport(storage, repo, ws, source)` — the workspace made from a bundle (a stream or chunks) under a name no workspace has, its objects stored as they come and its refs and record written once the repository holds everything they name, `workspaceDeploy(storage, repo, ws, pkgName, pkgVersion, options?)` (`runner`: its migrations, index builds and intake units; `schema` and `inputs`: its policies; `onRecordPlan`, `onRecordIndex`, `onInputPlan`: what it decided; `sourceConcurrency`: the `file` sources taken in at once; `onSourceProgress`, `onDeployProgress`: how far it has got), `workspaceExport(storage, repo, ws, zipPath, name?, version?)`, `workspaceStatus(storage, runner, repo, ws)`, `workspaceStatusRevision(storage, repo, ws)` — a token that moves whenever the status might, which the API's `ETag` names, `workspaceLockStatus(storage, repo, ws)`, `workspaceListStates(storage, repo)` — every workspace with its state in one read of the store, `workspaceRemove(storage, repo, ws, { sweep? })` — the record and what a store removes at once go in one call, and the rest is swept in bounded batches unless `sweep: false` leaves them to `workspaceSweep(storage, repo, cursor?)`, which a host with a time limit runs from a job |
-| Datasets | `workspaceGetDataset(storage, repo, ws, treePath)`, `workspaceSetDataset(storage, repo, ws, treePath, value, type)`, `datasetAdoptFile(storage, repo, ws, treePath, file, { runner, onProgress? })` → `{ hash, size, segments, rows, taken, runners? }`, `taken` being `known`, `carried` or `taken` (by the `runners` named); `deliveryAdoptStored(storage, repo, ws, { address, type }, sha256, { runner?, onProgress?, signal? })` — a delivery the store holds taken in as a stored value of the type, naming no dataset, for a record's first state |
-| Runs | `dataflowExecute(storage, repo, ws, options?)`; `LocalOrchestrator` to start, poll and cancel a run |
-| Records | `recordMutate(storage, runner, repo, ws, record, mutation, args, { actor })`, `recordHistory`, `recordDescribe`, `recordCompact`, `recordReindex`; `recordSystemCommit(storage, runner, repo, ws, record, { name, target, actor, args? })` — `$rollback` to a commit, `$restore` of a state at the record's migrations, and `$init` on a record declared with no initial value: its first state, from a stored value, with the whole chain applied |
-
-`@elaraai/e3-core/test` exports the contract suites another storage backend runs
-over itself.
-
-**`e3-api-server`** serves repositories: `--repos <dir>` (each subdirectory a
-repository) or `--repo <path>` (one, served as `default`), `-p`/`--port` (3000),
-`-H`/`--host` (localhost), `--cors`, `-j`/`--memory` (the server's budget, for
-every run and call it serves and every upload it takes in), and auth — `--oidc`, a built-in provider for `e3
-auth login` (`--token-expiry`, `--refresh-token-expiry`), or an external JWT
-issuer (`--auth-key`, `--auth-issuer`, `--auth-audience`). In code,
-`createServer({ reposDir | singleRepoPath, port, host, … })` starts one, and its
-route factories mount on another host.
-
-## Running e3 in a browser — `@elaraai/e3-web`
-
-e3-web runs e3 entirely in a page, with no server: a repository kept in the
-browser's own storage (records in IndexedDB, objects in OPFS, locks through
-Web Locks), every East program on a pool of Web Workers, and e3's whole API
-answered in the page by e3-api-server's own routes. An app is three scripts:
-
-```typescript
-// unit.worker.ts — where East programs run: east-web-std answers for east-node-std's
-// platform functions, and e3's own (e3-api-client's Platform) reach the e3 worker
-import { serveUnits } from '@elaraai/e3-web/units';
-serveUnits();
-```
-
-```typescript
-// e3.worker.ts — e3 itself: its storage, orchestrator, runner and API
-import { serveE3 } from '@elaraai/e3-web/worker';
-serveE3({
-  units: () => new Worker(new URL('./unit.worker.ts', import.meta.url), { type: 'module' }),
-});
-```
-
-```typescript
-// main.ts — the page: e3-api-client's calls, every one answered by the e3 worker
-import { createWebE3 } from '@elaraai/e3-web';
-import {
-  dataflowExecute, packageImport, repoCreate, repoList, workspaceCreate, workspaceDeploy,
-} from '@elaraai/e3-api-client';
-
-const e3 = await createWebE3(new Worker(new URL('./e3.worker.ts', import.meta.url), { type: 'module' }));
-const opts = { token: null, fetch: e3.fetch };
-
-// The first visit makes the repository; later visits find it where they left it.
-if (!(await repoList(e3.apiUrl, opts)).includes('default')) {
-  await repoCreate(e3.apiUrl, 'default', opts);
-  // The package zip e3.export wrote at build time, served beside the page
-  const zip = new Uint8Array(await (await fetch('/sales-planning-1.0.0.zip')).arrayBuffer());
-  const { name, version } = await packageImport(e3.apiUrl, 'default', zip, opts);
-  await workspaceCreate(e3.apiUrl, 'default', 'main', opts);
-  await workspaceDeploy(e3.apiUrl, 'default', 'main', `${name}@${version}`, opts);
-  await dataflowExecute(e3.apiUrl, 'default', 'main', {}, opts);
-}
-```
-
-`e3.apiUrl` is `https://e3-web.invalid`, an origin reserved so that a request
-sent without `e3.fetch` fails rather than reaching the network, and
-`e3.fetch`, the global `fetch`'s signature, is answered by the e3 worker. Both
-go to e3-api-client (`RequestOptions.fetch`) and to e3-ui-components'
-`<E3Provider>` (`E3Config.fetch`: the **e3-ui** skill). An app's own platform
-package is served beside the standard ones:
-`serveUnits({ platforms: { '@acme/pricing': PricingPlatform } })`.
-
-| `serveE3` option | Default | Effect |
-|---|---|---|
-| `units` | — | starts a unit worker: a Web Worker whose script calls `serveUnits()` |
-| `width` | the cores | the most units that run at once, and so the most unit workers, each with a heap of its own |
-| `persist` | `true` | `false` keeps repositories in memory, gone with the page |
-| `name` | `'e3'` | the storage's name: e3 workers of one name share their repositories, in one tab or several |
-| `identify` | none | each request's caller, `{ sub, email?, roles }`, from the request; a request to a repository whose caller it does not identify is answered 401. None: the page's one caller may do everything |
-| `access` | `oneShotAccessByRoles()` with `identify`, else `any` | the grant a caller holds for a one-shot, a split call, or a function call naming its own runner |
-| `wholeIntakeLimit` | 256 MiB | the largest delivery one intake unit takes in whole |
-
-What works is the API, as the local server answers it: every API suite of
-e3-api-tests passes against it in Chromium, as an admin and, in its reader
-cases, as a reader. Paged reads and key search, function calls, one-shot and
-split calls, record mutations, deploys that migrate records or build indexes,
-uploads, package import and export, gc, and dataflow runs — tasks in parallel
-up to the pool's width (the cores), split tasks in pieces, re-run reactively.
-A repository outlives a reload, and two tabs share an origin's repositories as
-two processes share a directory. A closed tab's locks are free; a run it left
-is taken as one whose process died, the next run served what finished from
-the cache; and a job or an upload's commit it left is recorded failed, never
-run again.
-
-What a browser cannot do — each fails naming it, never a partial answer:
-
-- **Commands.** A custom task, or a task on the `custom` runtime, is recorded
-  `error`: "a browser runs no commands".
-- **Platform functions with no browser meaning.** east-web-std has no
-  FileSystem, Env or large-JSON reader, and east-node-io and the python
-  packages have no browser equivalent: a unit calling one fails naming the
-  function, and one listing a package its worker does not serve fails naming
-  the package.
-- **Segments on demand.** A unit's inputs are staged whole into its worker:
-  waiting for a segment needs a cross-origin isolated page, which GitHub Pages
-  cannot give. Split a large input into pieces with `e3.partition`, which
-  bounds a unit's memory anywhere; a delivery that cannot be cut into pieces
-  is refused above `wholeIntakeLimit`, naming the fix.
-- **A memory budget.** Memory is not measured: the pool is as wide as the
-  cores, there is no `--memory`, and no record names a peak.
-- **The CLI.** `e3` drives e3 over HTTP, which an in-page e3 does not serve.
-
-## A local repository
-
-```
-repo/
-├── repository.beast2  # the release that last wrote it, and the upgrades it has had
-├── repository-upgrade.beast2  # while an upgrade is under way: where its last part stopped
-├── metadata.beast2    # its name and status
-├── objects/           # content-addressed: values, segments, manifests, programs
-├── packages/          # package refs
-├── workspaces/        # each workspace's state, dataset refs, and its latest run's state and events
-├── dataflows/         # run records
-├── executions/        # execution attempts: status, owner, logs
-├── running/           # an index of the attempts recorded running, by task: what a status reads
-├── runs/              # an index of each task's runs, its units left out: what its history and logs read
-├── adoptions/         # the manifest each delivered file, or piece of one, became
-├── locks/             # locks, their holders, and how far each says it has got
-├── gc/                # gc beside running work: unreachable notes, a stepped run's parts
-├── envs/              # built execution environments
-└── tmp/               # scratch and staged uploads
-```
-
-An e3 opening a repository an older release wrote applies the upgrades it has
-not had, in place; one a newer e3 upgraded is refused, naming that release.
+## Caching
 
 **Caching.** An execution is keyed by its task — program, runner and
 environment — and its inputs' hashes, so a task re-runs only when one of those
@@ -1280,6 +493,20 @@ consumers, even during a run. A split task is cached a unit at a time: each piec
 and each merge is its own execution, and pieces are cut by content, so an append
 or an insertion re-runs only the pieces it reaches and the merges above them.
 
+## Reference files
+
+Read the one a task needs; each holds its subject whole.
+
+| File | Holds |
+|---|---|
+| [reference/record-indexes.md](reference/record-indexes.md) | Secondary indexes, interval and area indexes, series and summary indexes, nested series indexes |
+| [reference/record-migrations.md](reference/record-migrations.md) | `migration.value` / `rows` / `rekey`, and how a record's type changes |
+| [reference/cli.md](reference/cli.md) | Every `e3` command: repositories, packages, workspaces, datasets, tasks, runs, calls, records, watch, convert, auth, gc |
+| [reference/from-code.md](reference/from-code.md) | `@elaraai/e3-api-client`, `@elaraai/e3-core`, `e3-api-server` |
+| [reference/browser.md](reference/browser.md) | `@elaraai/e3-web`: serveUnits, serveE3, createWebE3, and what a browser cannot do |
+| [reference/rack.md](reference/rack.md) | `e3 rack enroll` / `policy`, routing a run to a rack, `@elaraai/e3-rack` |
+| [reference/repository.md](reference/repository.md) | A local repository's layout, and upgrades |
+
 ## Related skills
 
 - **east** — the language task bodies are written in.