@elaraai/east-codex-plugin 0.0.0-stage → 1.0.89

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,2023 @@
1
+ ---
2
+ name: east-py
3
+ description: "East in Python (east-py): (A) East EXPRESSIONS — East functions written in python with East.function / East.asyncFunction, the block `b` (TypeScript's `$`), the TypeScript methods snake_cased, the standard library, East.compile; (B) East VALUES — East data as ordinary python (EastArray/Set/Dict/Vector/Matrix/Struct/Variant/Ref/Blob) whose eager methods run in east-c under the same names. Use when writing python against east-py: (1) an East function in python (b.let/b.if_/b.for_, expression methods, the stdlib), (2) building and validating values (array/struct/variant/some/none, coerce_to/assert_value_of), (3) transforming values with eager methods (sort/map/filter/reduce/group_*/set algebra/dict merge), (4) scalar builtins via East.(Type), (5) @East.platform_function to expose python to East, (6) beast2 files larger than memory, numpy/torch through EastVector/EastMatrix, (7) porting a plain-python POC into a platform… See the detailed scope below."
4
+ ---
5
+
6
+ ## Detailed skill scope
7
+
8
+ East in Python (east-py): (A) East EXPRESSIONS — East functions written in python with East.function / East.asyncFunction, the block `b` (TypeScript's `$`), the TypeScript methods snake_cased, the standard library, East.compile; (B) East VALUES — East data as ordinary python (EastArray/Set/Dict/Vector/Matrix/Struct/Variant/Ref/Blob) whose eager methods run in east-c under the same names. Use when writing python against east-py: (1) an East function in python (b.let/b.if_/b.for_, expression methods, the stdlib), (2) building and validating values (array/struct/variant/some/none, coerce_to/assert_value_of), (3) transforming values with eager methods (sort/map/filter/reduce/group_*/set algebra/dict merge), (4) scalar builtins via East.<Type>, (5) @East.platform_function to expose python to East, (6) beast2 files larger than memory, numpy/torch through EastVector/EastMatrix, (7) porting a plain-python POC into a platform function, (8) the east-py CLI — run, exec, convert, transpile, export-functions, lint, check, lsp, version, (9) exporting functions for TypeScript / e3 and importing TypeScript-authored ones (East.export_functions / East.import_function), (10) edit-time diagnostics (east-py lint, flake8 EAS codes, east-py check, east-py lsp), (11) JSON Schema contracts (json_schema_for, type_from_json_schema), (12) typed jq queries over East values (East.jq, and east.query's check_jq / evaluate_jq).
9
+
10
+ # East.py — East expressions and East values in Python
11
+
12
+ `east-py` is the Python runtime for East: a Cython bridge to the native **east-c**
13
+ runtime, where IR compilation, the builtin library, execution and serialization
14
+ all run. Its two surfaces share one vocabulary — the TypeScript method names,
15
+ snake_cased:
16
+
17
+ - **East expressions** — East *functions* written in python, as the TypeScript
18
+ `east` skill writes them. `East.function(param_types, out, body)` runs `body`
19
+ ONCE over typed expression proxies (the block `b` first — TypeScript's `$`),
20
+ records IR, and east-c compiles it. Every East type has an expression class
21
+ (`ArrayExpression`, `IntegerExpression`, …) mirroring `libs/east/src/expr/*.ts`
22
+ method for method, plus the standard library (`East.Integer.print_compact`,
23
+ `East.DateTime.round_down_week`, …) and the statements (`b.let`, `b.if_`,
24
+ `b.while_`, `b.for_`, `b.match_`, `b.try_`, `b.return_`). What East cannot
25
+ express raises at build time — there is no interpreter behind it.
26
+ - **East values** — the data. `EastArray`/`EastSet`/`EastDict`/`EastVector`/
27
+ `EastMatrix`/`EastStruct`/`EastVariant`/`EastRef`/`EastBlob` are handles into
28
+ the east-c value slab; scalars are plain `int`/`float`/`str`/`bool`/`datetime`.
29
+ Their **eager methods** — the same names — run in east-c now, and chain.
30
+ `@East.platform_function` exposes a python function to East.
31
+
32
+ They meet twice: a **callback** handed to an eager method (`items.map(lambda b,
33
+ r: …)`) is an East function body — captured once, compiled, run natively per
34
+ element, never per-element python — and an `East.function` is a plain callable
35
+ on values that every eager method accepts. (The TypeScript DSL: the `east`
36
+ skill. ML and optimisation platform functions: `east-py-datascience`.)
37
+
38
+ ## Before writing code — search the example index
39
+
40
+ Every East API has a tested example in the plugin's index — the index IS the
41
+ API reference, printed from each example's IR in TypeScript or python. Before
42
+ writing or changing East code:
43
+
44
+ 1. Call `search_east_examples` for each capability you
45
+ are about to use — `language: "python"` for east-py, `"typescript"`
46
+ otherwise. Summaries come back first: id, signature, the inputs and the
47
+ expected result, a few hundred bytes each.
48
+ 2. Fetch the one or two that match with `get_east_example`
49
+ and pattern your code on them.
50
+ 3. Do not read `node_modules/@elaraai/**` or `*.examples.ts` files wholesale,
51
+ and do not reason from `.d.ts` signatures: the index holds the same
52
+ programs, exact and far cheaper, and the signatures omit the runtime rules
53
+ that make East code correct.
54
+
55
+ Nothing is injected for you; the search is the step.
56
+
57
+ ## Quick Start
58
+
59
+ ```python
60
+ from east import East, ArrayType, FloatType, StringType, StructType, array, struct
61
+
62
+ LineItem = StructType([("name", StringType), ("price", FloatType)]) # types take PAIRS
63
+
64
+ # ── East EXPRESSIONS: an East function — the block `b` first, then the parameters ──
65
+ @East.function([ArrayType(LineItem), FloatType], StringType)
66
+ def receipt(b, items, fx):
67
+ total = b.let(0.0) # $.let
68
+ def add(b, r, _i): # a body: the block, then the callback's arguments
69
+ b.assign(total, total + r.price * fx) # $.assign
70
+ b.for_(items, add) # $.for
71
+ b.if_(total > 1000.0, # $.if … $.return
72
+ lambda b: b.return_(East.str("big order: ", East.Float.print_currency(total))))
73
+ return East.str("total: ", East.Float.print_currency(total)) # East.str`…`; the stdlib is the TS one
74
+
75
+ # ── East VALUES: East data as ordinary python; eager methods run NOW in east-c ──
76
+ items = array(LineItem, [{"name": "a", "price": 1}, {"name": "b", "price": 2.0}]) # coerced + validated
77
+ dear = items.filter(lambda b, r: r.price >= 2.0).sort(lambda b, r: r.price) # callbacks are bodies
78
+ receipt(items, 1.5) # "total: $4.50" — an artifact is a callable on values
79
+ East.Float.sqrt(2.0); East.String.upper_case("hi") # scalar builtins live on East.<Type>
80
+
81
+ # ── Expose python to East — typed, validated, auto-collected ──
82
+ @East.platform_function(inputs=[FloatType, ArrayType(LineItem)], output=ArrayType(LineItem))
83
+ def convert_prices(fx_rate, items):
84
+ return items.map(lambda b, r: struct({"name": r.name, "price": r.price * fx_rate}, LineItem))
85
+
86
+ platform = East.platform_functions(__name__) # pass to East.compile(fn, platform=…) to register
87
+ ```
88
+
89
+ ## The two surfaces
90
+
91
+ | | East expressions | East values |
92
+ |---|---|---|
93
+ | What you hold | typed expression proxies: `IntegerExpression`, `ArrayExpression`, … (the TS `IntegerExpr`, `ArrayExpr`, …) | runtime data: `int`/`float`/`str`/`bool`/`datetime`, `EastArray`, `EastDict`, `EastStruct`, `EastVariant`, … |
94
+ | Where | inside an `East.function` / `East.asyncFunction` body, a statement body, and EVERY callback handed to an eager method | everywhere else — a `@East.platform_function` body, a script, a test |
95
+ | A method call | records an IR node (`xs.map(f)` is `ArrayMap`); the body runs ONCE, at build time | executes now in east-c (`xs.map(f)` runs the loop natively) and returns a live value |
96
+ | Names | the TypeScript names, snake_cased (`push_last`, `flatten_to_set`, `print_formatted`); python-idiom spellings (`fold`, `sorted`, `keys_set`, `upper`, …) are DEPRECATED aliases that warn | the same names — the eager class and the expression class agree method for method |
97
+ | Scalars | methods on the expression (`x.abs()`, `s.upper_case()`, `d.add_days(1)`) — or the `East.<Type>` namespace | the `East.<Type>` namespaces only (you cannot add methods to `float`/`str`) |
98
+ | Conditionals / loops | `East.if_else`, `b.if_`, `b.while_`, `b.for_`, `East.while_`/`for_` — python `if`/`while` collapse to a `bool` before the build sees them | plain python |
99
+ | An option | `.is_some()` / `.is_none()` / `.unwrap_or(d)` / `.match({…})` | an `EastVariant` — the SAME accessors, plus `.type` / `.value` / `.unwrap(tag)` |
100
+ | Errors | an operation East cannot express raises `ExpressionError` at BUILD time, naming the binding or method | a runtime `EastError` / `EastTypeError` |
101
+
102
+ A body's parameters are expressions already. A python scalar or `datetime` inside
103
+ a body lifts to a literal; `East.value(v, T)` / `b.const(v)` lift anything else
104
+ (TS `East.value` / `$.const`); an East COLLECTION an `East.function` body closes
105
+ over is snapshot into the IR (`.bind(table)` it as a trailing parameter to keep it
106
+ live). The other way, an artifact is a plain callable on values, every eager
107
+ method accepts one (a VALUE takes no block), and one called inside another body
108
+ is a call of its inline Function node — see [Python values vs East expressions](#python-values-vs-east-expressions).
109
+
110
+ ## Decision Tree: What Do You Need?
111
+
112
+ Each leaf names the section holding the whole surface for it.
113
+
114
+ ```
115
+ Task → What do you need?
116
+ │
117
+ ├─ A. Write an East function in python — East expressions (the `east` skill's twin, name for name)
118
+ │ ├─ Types (PAIRS, not a dict) → IntegerType · FloatType · StringType · BooleanType · DateTimeType · BlobType · NullType ·
119
+ │ │ ArrayType(T) · SetType(K) · DictType(K, V) · RefType(T) · VectorType(T) · MatrixType(T) (T ∈ Float/Integer/Boolean) ·
120
+ │ │ StructType([("f", T), …]) · VariantType([("case", T), …]) · OptionType(T) · recursive_type(lambda self: …) ·
121
+ │ │ FunctionType(I, O) · AsyncFunctionType(I, O) → "Type System Summary"
122
+ │ ├─ Author → East.function([T…], Out, lambda b, x: …) · @East.function([T…], Out) def f(b, x) · East.asyncFunction ·
123
+ │ │ ❗ out is required · a pure one is a callable on values · f.bind(table) pre-binds trailing parameters BY REFERENCE
124
+ │ │ → "Declare, implement, build, compile"
125
+ │ ├─ Statements on the block `b` (TS `$`) → b.let · b.const · b.assign · b.do · b.return_ · b.error · b.if_ … .else_if … .else_ ·
126
+ │ │ b.while_ · b.for_ · b.break_ / b.continue_ · b.match_ · b.try_ … .catch … .finally_ · East.block(lambda b: …)
127
+ │ │ → "Every body takes the block first"
128
+ │ ├─ Methods and operators (the TS methods, snake_cased) ❗ // % ** / on an Integer RAISE, with the fix-it
129
+ │ │ → "The expression surface, type by type"
130
+ │ ├─ The standard library, dual-mode → East.Integer / Float / DateTime / String / Blob / Boolean .* · East.str · East.print ·
131
+ │ │ East.min / max / clamp · East.equal / less / compare(T, a, b) → "The standard library" and "East.<Type> namespaces"
132
+ │ ├─ The next step depends on the last (worklist · BFS · fixpoint · replay) → b.while_ / b.for_, or the expression forms
133
+ │ │ East.while_ / for_ / if_else / let / block / try_catch · East.new_array / new_set / new_dict accumulators ·
134
+ │ │ East.break_ / continue_ / label → "Control flow — the expression forms", "Sequential logic that stays in east-c"
135
+ │ ├─ Nest flat parent-keyed rows (any depth) → xs.to_tree(Node, key, parent, build) — not recursion, which is for
136
+ │ │ shallow depths → "Trees from flat rows, and recursion"
137
+ │ ├─ Query a value with jq → East.jq(input, "jq", result_type) ❗ result_type is required, and types the result ·
138
+ │ │ a dict of inputs is read as an e3 root → "Queries — East.jq"
139
+ │ ├─ What a body may capture — and what it refuses, naming the binding → "What a body may reference"
140
+ │ ├─ Call python from a body → the @East.platform_function you hold IS the Platform node (no declaration) · East.platform /
141
+ │ │ asyncPlatform / genericPlatform only for one implemented elsewhere · East.compile(fn, platform=East.platform_functions(__name__))
142
+ │ │ → "Declare, implement, build, compile", "Platform functions"
143
+ │ ├─ Make it fast → data stays in East · chain · CSE by reusing a python variable · capture small tables, .bind big ones
144
+ │ │ → "Performance — the levers, ranked"
145
+ │ ├─ IR ↔ python → to_python_source · east-py transpile · compile_from_beast2 / json / east / value · east-c ir … → "IR ↔ python"
146
+ │ ├─ Refusals at edit time → east-py lint · flake8 --select EAS · east-py lsp; TYPE errors → east-py check
147
+ │ │ → "Diagnostics at edit time"
148
+ │ └─ Across languages → east_functions = {"name": fn} · East.import_function · East.link_imports · east-py export-functions
149
+ │ → "Cross-language functions"
150
+ │
151
+ ├─ B. Work with East values — the eager runtime (the same names, run NOW in east-c; results stay C-side and chain)
152
+ │ ├─ Build one → array · struct · variant · some / none · east_ref · EastBlob(b"…") · EastVector / EastMatrix (numpy) ·
153
+ │ │ coerce_to(value, T) (type-driven: int→Float, dict→Struct, 1-D numpy→Array) · validate at a python ↔ East boundary with
154
+ │ │ assert_value_of ❗ · explain_value_of · is_value_of · type_of → "Construction & validation"
155
+ │ ├─ Transform → map · filter · reduce · scan · sort · group_* · to_dict · to_tree · set algebra · dict union / merge · …, each callback
156
+ │ │ an East function body — and keep the data in East: never down-convert to list/dict/set, loop, and rebuild
157
+ │ │ → "Work in East values", "Eager callbacks", then "EastArray — complete method surface" and its siblings
158
+ │ ├─ Generate → EastArray.range / linspace / generate · EastSet / EastDict.generate · EastVector / EastMatrix.zeros / ones /
159
+ │ │ fill / from_array / from_numpy / from_torch → "Container generators"
160
+ │ ├─ Scalars (float / int / str / bool / datetime take no methods) → East.Float / Integer / String / DateTime / Boolean.* ·
161
+ │ │ East.less / compare / equal(T, a, b) — never python's str / re / datetime / // / < for these
162
+ │ │ → "East.<Type> namespaces", "Scalars: use the `East.<Type>` utilities"
163
+ │ ├─ Diff / patch two values → East.diff / apply_patch / compose_patch / invert_patch(T, …) → "East.<Type> namespaces"
164
+ │ ├─ Query one with jq → East.jq(value, "jq", T) runs now · evaluate_jq(program, value, input_type=T) · check_jq checks
165
+ │ │ without running → "Queries — East.jq"
166
+ │ ├─ A JSON contract another system validates against → json_schema_for · type_from_json_schema → "A JSON contract"
167
+ │ ├─ A collection file larger than memory → write_beast2_file · open_beast2_file (a read-only East collection value) ·
168
+ │ │ write_beast2_file_parallel · splice_beast2_files · Beast2ManifestWriter · open_beast2_pages_for → "Beast2 streaming"
169
+ │ ├─ Logic genuinely needs python (numpy, a model, a solver) → to_columns · from_columns · map_batches · update_many ·
170
+ │ │ extend · to_numpy / to_torch → "Columnar escape hatches"
171
+ │ └─ Let East call python → @East.platform_function · @East.generic_platform_function · @memoize → "Platform functions"
172
+ │
173
+ ├─ C. Cross between them → a body's parameters are expressions · python scalars and datetimes lift · East.value(v, T) /
174
+ │ b.const(v) · a captured East collection is a build-time SNAPSHOT (an eager callback refuses it), .bind(table) keeps it
175
+ │ live · an East function is a callable on values, and a VALUE takes no block · east.runtime.compiler.eager_stats()
176
+ │ shows how a hot call ran → "Python values vs East expressions", "Performance"
177
+ │
178
+ └─ D. Run a program from the shell → east-py run (IR + input files → a result) · exec (a unit: the runner protocol e3
179
+ speaks) · convert (a beast2 value → East text) · version · where its time goes: --profile / EAST_PROFILE=1
180
+ → "The east-py CLI"
181
+ ```
182
+
183
+ ## Type System Summary
184
+
185
+ Every East type has an expression class (what a body holds — the TS
186
+ `IntegerExpr`/`ArrayExpr`/… twins, from `east.expression.expr`) and a python
187
+ value (what you hold outside). Scalars are plain python objects, so their
188
+ builtins live on the `East.<Type>` namespaces; the rest are `East*` containers
189
+ that carry their element type and have eager methods. Only
190
+ `EastArray`/`EastSet`/`EastDict`/`EastRef` mutate in place — `EastStruct`,
191
+ `EastVariant`, `EastVector` and `EastMatrix` are immutable (`set` returns a new
192
+ tensor).
193
+
194
+ | East type | Expression class (in a body) | Python value (eager) | Mutability |
195
+ |-----------|------------------------------|----------------------|------------|
196
+ | `NullType` | `NullExpression` | `None` (validates); the canonical value is the `east_null` sentinel | Immutable |
197
+ | `BooleanType` | `BooleanExpression` | `bool` | Immutable |
198
+ | `IntegerType` | `IntegerExpression` | `int` (i64) | Immutable |
199
+ | `FloatType` | `FloatExpression` | `float` (f64) | Immutable |
200
+ | `StringType` | `StringExpression` | `str` | Immutable |
201
+ | `DateTimeType` | `DateTimeExpression` | `datetime.datetime` (UTC) | Immutable |
202
+ | `BlobType` | `BlobExpression` | `EastBlob` (a `bytes` subclass); `.data` is `bytes` | Immutable |
203
+ | `ArrayType(T)` | `ArrayExpression` | `EastArray` (indexable, iterable) | **Mutable** |
204
+ | `SetType(K)` | `SetExpression` | `EastSet` (East-sorted) | **Mutable** |
205
+ | `DictType(K, V)` | `DictExpression` | `EastDict` (East-sorted by key) | **Mutable** |
206
+ | `VectorType(T)` | `VectorExpression` | `EastVector`; 1-D numpy buffer via `.to_numpy()` | Immutable |
207
+ | `MatrixType(T)` | `MatrixExpression` | `EastMatrix`; 2-D row-major numpy buffer via `.to_numpy()` | Immutable |
208
+ | `StructType([("field", T), …])` | `StructExpression` (`r.field` / `r["field"]`) | `EastStruct` (`s["name"]` / `s.name`) | Immutable (frozen) |
209
+ | `VariantType([("case", T), …])` | `VariantExpression` | `EastVariant` (`.type` tag, `.value`; compared **by case name**) | Immutable (frozen) |
210
+ | `RefType(T)` | `RefExpression` | `EastRef` (cell; `.get()` / `.update(v)` / `.merge()`) | **Mutable** |
211
+ | `FunctionType(I, O)` / `AsyncFunctionType(I, O)` | `FunctionExpression` / `AsyncFunctionExpression` (callable in a body) | an `East.function` artifact / `EastFunction` | Immutable |
212
+ | `recursive_type(…)` / a diverging body | `RecursiveExpression` / `NeverExpression` | the wrapped value / — | — |
213
+
214
+ `VectorType`/`MatrixType` elements are `FloatType`, `IntegerType` or `BooleanType`;
215
+ a narrower numpy dtype (f32, …) is canonicalized to East's width (Float→f64,
216
+ Integer→i64, Boolean→u8) crossing into east-c. `PatchType(T)` (from
217
+ `east.types.types`) is the type of a structural patch of a `T`.
218
+
219
+ ## East expressions — writing East functions in Python
220
+
221
+ A python body becomes East IR by being CAPTURED: it runs ONCE against typed
222
+ expression proxies, exactly like the TypeScript builder; east-c compiles the
223
+ recorded IR, and from then on only the compiled function runs. A body East
224
+ cannot express raises at build time, so the same source always costs and means
225
+ the same thing.
226
+
227
+ ### Declare, implement, build, compile — the four authoring calls
228
+
229
+ The TypeScript names, name for name. Two are about **platform functions** —
230
+ python the program calls out to, paired **by name** — and two about **East
231
+ functions**, the program itself:
232
+
233
+ | Step | Call | What it is |
234
+ |---|---|---|
235
+ | **Implement** a platform function in python | `@East.platform_function(inputs=…, output=…)` (`@East.generic_platform_function(type_parameters=, inputs=, output=)`; `East.platform_functions(__name__)` collects a module's) | the host side AND the body's spelling: a plain python function over East VALUES, its result validated against `output` — and, called inside a body, the `Platform` node with this very signature. Its name — the `def`'s, or `name=` — is what every runtime pairs the call with |
236
+ | **Declare** one implemented elsewhere | `East.platform(name, inputs, output)` (`East.asyncPlatform`, `East.genericPlatform`, `East.asyncGenericPlatform`) | a handle a body CALLS — it emits the `Platform` node; nothing runs here. For a function implemented in another package or runtime; a python implementation you hold needs no declaration |
237
+ | **Build** an East function | `East.function(inputs, out, body)` / `@East.function(inputs, out)` (`East.asyncFunction`) | runs `body` once over expression proxies and records IR; a pure one is already callable |
238
+ | **Compile** with the implementations | `East.compile(fn, platform=[…])` (`East.compileAsync`) | analyzes the IR against the implementations — a declaration no implementation matches by name is `Platform function '<name>' not found` — and returns the native callable |
239
+
240
+ An `East.function` has no name of its own: it is a VALUE, called through the
241
+ binding that holds it (`score(x)`, `rows.map(score)`), stored in a struct or an
242
+ array, exported under a name (`east_functions = {"score": score}`) — its IR
243
+ carries its parameters' names, never its own, as in TypeScript. A platform
244
+ function is the opposite: the name IS the pairing, on every runtime.
245
+
246
+ ```python
247
+ from east import East, ArrayType, FloatType, IntegerType, NullType, StringType, StructType
248
+ from east_py_std import fs_read_file, platform as std
249
+
250
+ Row = StructType([("sku", StringType), ("qty", IntegerType)])
251
+
252
+ # BUILD — `out` is REQUIRED and enforced; every body takes the block `b` first
253
+ score = East.function([Row], FloatType, lambda b, r: r.qty.to_float() * 1.5)
254
+ score({"sku": "a", "qty": 2}) # 3.0 — a pure artifact is a callable on values
255
+
256
+ # IMPLEMENT — python over East values; `name=` because "t.log" is not an identifier.
257
+ # The decorated function is DUAL-MODE: log_line("x") runs the python;
258
+ # inside a body, log_line(name) IS the Platform node 't.log' with this signature.
259
+ @East.platform_function(inputs=[StringType], output=NullType, name="t.log")
260
+ def log_line(line):
261
+ print(line)
262
+
263
+ greet = East.function([StringType], NullType,
264
+ lambda b, name: log_line(East.String.concat("hello ", name)))
265
+ greet("bob") # EastError: Platform function 't.log' is not
266
+ # available — compile with East.compile(fn, platform=[...])
267
+
268
+ # COMPILE — the implementation, by name (the same record the decorator registered)
269
+ run = East.compile(greet, platform=East.platform_functions(__name__))
270
+ run("bob") # prints "hello bob"
271
+
272
+ # A stock implementation is called the same way — no East.platform line restates it
273
+ first = East.function([StringType], StringType, lambda b, path: fs_read_file(path).split("\n").get(0))
274
+ East.compile(first, platform=std)("hosts.txt")
275
+
276
+ # DECLARE — only for a function implemented elsewhere (another package or runtime):
277
+ remote = East.platform("t.remote", [StringType], NullType)
278
+ ```
279
+
280
+ **An East function inside a platform function.** The implementation is
281
+ ordinary python over East values, and an `East.function` is a callable on
282
+ values that every eager method accepts — so that is how the implementation
283
+ does its East work, with the loop and the body running in east-c:
284
+
285
+ ```python
286
+ Rows = ArrayType(Row)
287
+
288
+ @East.platform_function(inputs=[Rows], output=FloatType) # its name: "total", the def's
289
+ def total(rows):
290
+ return rows.map(score).sum() # `score` from above — an East function on values
291
+
292
+ report = East.function([Rows], StringType, # `total(rows)` in the body IS the
293
+ lambda b, rows: East.str("total: ", # Platform node — no declaration
294
+ East.Float.print_currency(total(rows))))
295
+ East.compile(report, platform=East.platform_functions(__name__))(rows) # "total: $9.00"
296
+ ```
297
+
298
+ Each call in detail:
299
+
300
+ | Call | Builds | Notes |
301
+ |---|---|---|
302
+ | `East.function(param_types, out, body)` | a `Function` artifact | `param_types` is a LIST (`[]` for none); the body takes the block first — `lambda b, x: …` / `def f(b, x)`, as EVERY body does (`lambda x: …` is refused with the fix-it); `out` is required, and a body whose expression has another type raises naming both; with `body` omitted it is a DECORATOR |
303
+ | `East.asyncFunction(param_types, out, body)` | an `AsyncFunction` artifact | for bodies calling async platform declarations; compile with `East.compileAsync` |
304
+ | `East.platform(name, inputs, output, optional=False)` | a declaration handle | callable INSIDE a body (emits the `Platform` node); calling one outside raises `expression-level`; `East.genericPlatform(name, ["T"], inputs, output)` is the type-parameterised form (`East.asyncGenericPlatform` the async one). Needed only for a function implemented elsewhere: a `@East.platform_function` you hold emits the same node itself |
305
+ | `East.asyncPlatform(name, inputs, output)` | an async declaration | calling it from a SYNC body is a build error naming `East.asyncFunction` |
306
+ | `East.compile(fn, platform=[])` / `East.compileAsync(...)` | a native callable | takes an artifact or a raw IR value; the IR is analyzed against `platform` first (`east.ir.analyze`, the TS `analyzeIR`): a signature mismatch or a missing implementation is an `EastError` naming the call — unless the declaration is `optional=True`, which compiles to a stub that raises at the call (TS parity) |
307
+
308
+ A **pure** artifact needs no compile step: it is already a native callable,
309
+ `.bind(*values)` pre-binds trailing parameters by reference, and calling it
310
+ inside another body is a `Call` of its inline `Function` node — TypeScript's
311
+ shape — so it runs in ITS OWN frame: a `b.return_` returns from it, an
312
+ artifact used as an `East.if_else` arm evaluates only when that arm is taken,
313
+ an error inside it names its own lines, and a profile lists it as an entry of
314
+ its own, placed where it was written and where it was called. (One called with
315
+ arguments narrower than its parameters — a column projection's narrowed rows —
316
+ re-runs its source in its own frame and splices in instead, as does one whose
317
+ build hoisted constants.) An artifact that mutates or raises, called and
318
+ DISCARDED, is the build's "evaluated and thrown away" error — spell it
319
+ `b.do(push(xs))` / `East.block(push(xs), result)`, as with any mutation. An
320
+ artifact that calls platform functions stays first-class (composable,
321
+ serializable) but raises until `East.compile` pairs it with implementations.
322
+ `East.function(...)` spelled INSIDE a body is the inline `Function` node, a
323
+ Function-typed expression: bind it with `b.const`, hand it to a callback slot,
324
+ or call it (a `Call`; in an `East.asyncFunction`, calling an async one is a
325
+ `CallAsync`).
326
+
327
+ **Error locations.** A build records the python frames that built each node,
328
+ so a runtime error names its authoring site: `EastError.location` is the stack —
329
+ the lambda's `file:line:column` first, then the `East.function(...)` call and
330
+ its callers — and a platform-signature mismatch at `East.compile` names the call
331
+ the same way. The map rides the function's beast2 encoding, so the error reads
332
+ the same on east-c or east-node. Paths are relative to the working directory;
333
+ `set_location_base_path(dir)` (from `east`) pins the base for reproducible
334
+ fixtures. An error inside a callback a builtin invokes (`arr.map(...)` and
335
+ friends) resolves to the builtin's call site, on every runner.
336
+ `set_location_capture(False)` (from `east`) builds without recording any
337
+ locations: there is then no stack walk per node, and an error raised by such a
338
+ function carries no location. Binding names are read separately and keep
339
+ working. The TypeScript twin is `setLocationCapture`.
340
+
341
+ ### Every body takes the block first — the TypeScript `$` twin
342
+
343
+ A TypeScript body receives `$` and appends statements to it; a python body
344
+ receives **`b`**, the block, as its FIRST parameter and does the same. EVERY
345
+ body does — an `East.function` body, a builtin's callback, a branch, a loop, a
346
+ handler — exactly as every TypeScript body is `($, …) => …`: `lambda x: …` is
347
+ refused with the fix-it, a body that uses the block as the element fails on the
348
+ block's first use, and each nested body receives ITS OWN block first, so a
349
+ statement on any other block is a build-time error. Python `None` returned from a
350
+ body is TypeScript's "no return"; the `east_null` sentinel is an explicit `null`:
351
+
352
+ ```python
353
+ from east import East, IntegerType, StringType
354
+
355
+ @East.function([IntegerType], StringType)
356
+ def classify(b, n):
357
+ acc = b.let(0) # $.let — reassignable
358
+ limit = b.const(n * 2) # $.const
359
+ def loop(b, label): # $.while body: (b, label)
360
+ b.if_(acc >= limit, lambda b: b.break_(label))
361
+ b.assign(acc, acc + 1) # $.assign
362
+ b.while_(True, loop)
363
+ b.if_(acc > 10, lambda b: b.return_("big")) \
364
+ .else_if(acc > 5, lambda b: b.return_("mid")) \
365
+ .else_(lambda b: b.return_("small")) # every arm returns → Never
366
+
367
+ classify(3) # "mid"
368
+ ```
369
+
370
+ A one-statement body is a `lambda b: …`; a longer one is a `def` written
371
+ just before the statement that uses it. Name the block `_b` when the body
372
+ does not use it.
373
+
374
+ | Body | Spelling |
375
+ |---|---|
376
+ | `East.function` / `East.asyncFunction` body | `lambda b, x: …`, `def f(b, x)` — the block plus exactly the function's parameters; `lambda x: …` is refused (`a body takes the block first`) |
377
+ | a builtin's callback (`xs.map(...)`, `reduce`, …) | `xs.map(lambda b, el, i: …)` — the block, then the builtin's callback signature (map's `(element, index)`; `xs.reduce(lambda b, acc, el, i: …, 0)`; a Dict's `(value, key)`); trailing parameters may be omitted, the block cannot |
378
+ | a statement construct's body (`b.if_`/`.else_if`/`.else_`, `b.match_`, `b.while_`, `b.for_`, `b.try_`/`.catch`/`.finally_`) and `East.block(fn)` | `b.for_(xs, lambda b, v, i, label: …)`, `b.match_(v, {"some": lambda b, x: …})`, `East.block(lambda b: …)` |
379
+ | an expression form's handler (`.match({...})`, `East.try_catch`, `East.let`, the `East.while_`/`for_` bodies, `.and_`/`.or_`/`.if_else`) | bodies too: `v.match({"some": lambda b, x: …})`, `East.try_catch(lambda b: …, lambda b, msg: …)` — statements inside them go in `East.block(lambda b: …)`; `East.if_else` arms are expressions, not bodies |
380
+ | a function VALUE — a compiled `East.function`, a `.bind` result, a Function-typed expression, a platform declaration | takes NO block: a slot invokes what it holds body-style and the value drops the block, so `xs.map(amount)` and `xs.map(lambda b, el: amount(el))` both work (TS: an `Expr<FunctionType>` wherever a `($, …) => …` is accepted); a value declaring fewer parameters than the slot passes takes the prefix |
381
+
382
+ | Statement | Emits | Notes |
383
+ |---|---|---|
384
+ | `b.let(value[, type])` / `b.const(value[, type])` | `Let` (Null) | the variable, mutable / not; a declared `type` widens a narrower literal (a `Variable` of a subtype gets an `As`) |
385
+ | `b.assign(var, value)` | `Assign` (Null) | `var` must come from `b.let` — a python `x = …` rebinds the NAME and changes nothing |
386
+ | `b.return_([value])` | `Return` (Never) | checked against the declared output; a `do` of the same expression just before is not duplicated |
387
+ | `b.if_(pred, fn).else_if(pred, fn).else_(fn)` | `IfElse` (Null; Never when every arm diverges) | each `fn(b)` runs in its own frame; a branch ending in a non-Null value pads with `null`; a body may return the chain itself (`lambda b: b.if_(…)`) |
388
+ | `b.match_(variant, {case: fn(b, data)})` | `Match` (Null) | a case without a handler does nothing |
389
+ | `b.while_(pred, fn)` / `b.for_(coll, fn)` | `While` / `ForArray`/`ForSet`/`ForDict` (Null) | `fn(b, label)`; Array `fn(b, value, index, label)`, Set `fn(b, key, label)`, Dict `fn(b, value, key, label)` |
390
+ | `b.break_(label)` / `b.continue_(label)` | `Break` / `Continue` (Never) | the loop's `label` is what the body received; a bare `East.break_()` (the expression form's jump) inside a statement loop targets it too |
391
+ | `b.try_(fn).catch(fn(b, message, stack)).finally_(fn)` | `TryCatch` (Null) | `.catch` at most once; Never when both bodies diverge |
392
+ | `b.do(expr)` | the expression as a statement | `$(expr)` — a platform call or mutation evaluated for its effect; a bare `arr.push_last(x)` line is thrown away, and the build says so |
393
+ | `b.error(msg)` | `Error` (Never) | `$.error` — raise now; `East.error(msg)` is the expression twin (return it, or use it as an `if_else` arm) |
394
+ | `East.block(fn)` | `Block` | the EXPRESSION form: `fn(b)`'s statements, then the value it returns (a block returning nothing must diverge) |
395
+
396
+ A statement after one that never completes raises `Unreachable statement
397
+ detected`, as in TypeScript; a statement on an OUTER block from inside a nested
398
+ body, or on a block whose body has returned, raises naming it (TypeScript's
399
+ `no-cross-block-builder` lint is a hard error here). Every other IR node kind has
400
+ a spelling too, so any program TypeScript can build, python builds name for
401
+ name: `East.value(v, T)` (a typed literal / struct / list / dict), `East.as_(v,
402
+ T)` (an explicit widening `As`), `East.wrap_recursive(v, R)` and `expr.unwrap()`
403
+ on a recursive-typed expression, and `East.builtin(name, [T…], [args], out)` (a
404
+ raw builtin, for the few with no named spelling — `east.codegen.RAW_ONLY`).
405
+
406
+ **The analyzer.** Every build and every `East.compile` runs
407
+ `east.ir.analyze.analyze_ir`, the twin of TypeScript's `analyzeIR`: scope, exact
408
+ types (a `Let`/`Assign`/argument/element/field has exactly its slot's type;
409
+ subtyping is spelled with `As`), divergence and node well-formedness, with the
410
+ TypeScript messages at the python `file:line:column`. A body that fails it never
411
+ compiles.
412
+
413
+ ### The expression surface, type by type
414
+
415
+ One class per East type in `east.expression.expr`, each the twin of
416
+ `libs/east/src/expr/<type>.ts`: the same methods snake_cased, the same builtin
417
+ and argument order behind each, and the same aliases where TypeScript has them
418
+ (`plus`/`minus`/…, `eq`/`equal`/`equals`). Every TypeScript method exists on the
419
+ expression class AND on the eager value class (a scalar's eager twin is the
420
+ `East.<Type>` function of the same name, value first: `d.add_days(n)` ↔
421
+ `East.DateTime.add_days(d, n)`), and each python-only name is a deprecated
422
+ python-idiom alias, a python protocol twin, or a convenience. Python departs from
423
+ TypeScript only where the language forces it: operators where they agree (`+ - *
424
+ / ** & | ^ ~`, the comparisons), `and_`/`or_`/`not_`, `as_`, and keyword
425
+ arguments (`out=`, `key=`, `combine=`) where TypeScript overloads.
426
+
427
+ | Type | Operators | Methods (the TypeScript names) |
428
+ |---|---|---|
429
+ | **Boolean** | `&` `\|` `^` `~` — never `and`/`or`/`not`/`if` (python collapses them to `bool`) · `==` `!=` | `.bit_and(y) .bit_or(y) .bit_xor(y) .not_()` (the builtins — both operands evaluate) · `.and_(fn(b))` `.or_(fn(b))` (TS `and`/`or`: SHORT-CIRCUIT, the other operand is a body) · `.if_else(fn(b), fn(b))` (TS `ifElse`; `East.if_else(cond, value, …, otherwise)` is the value form — pairs then the else, one `IfElse` node) · `.equals/.equal/.eq` `.not_equals/.not_equal/.ne` |
430
+ | **Integer** | `+` `-` `*` and unary `-`; `==` `!=` `<` `<=` `>` `>=` (East total order) · ❗ `//` `%` `**` `/` RAISE at build time with the fix-it: python floors/takes the divisor's sign/promotes a negative exponent where East truncates/takes the dividend's sign/yields 0 — spell `.divide(y)` / `East.Integer.divide`, `.remainder(y)` / `East.Integer.remainder`, `.pow(y)` / `East.Integer.pow`, `.to_float() / y` | `.add .subtract .multiply .divide .remainder .pow` (a Float argument widens `self`, like TS; aliases `.plus .sub .minus .mul .times .div .mod .rem .modulo`) · `.negate() .abs() .sign() .log(base)` · `.to_float()` · `.less_than/.less/.lt .greater_than/.greater/.gt .less_than_or_equal/.less_equal/.lte/.le .greater_than_or_equal/.greater_equal/.gte/.ge .equals/.equal/.eq .not_equals/.not_equal/.ne` |
431
+ | **Float** | `+` `-` `*` `/` `**` and unary `-`; comparisons · ❗ `//` `%` RAISE (`.remainder(y)` / `East.Float.remainder`) · `math.floor/ceil/trunc(x)` build the stdlib `East.Float.round_floor/round_ceil/round_trunc`; python `round(x)` raises (its tie rule differs — `East.Float.round_half(x)`) | the same named arithmetic and aliases (an Integer argument widens) · `.negate .abs .sign .sqrt .exp .log .sin .cos .tan` · `.to_integer()` ❗ errors at run time on a non-integral/NaN/infinite value · Float → Integer rounding is the stdlib `East.Float.round_floor/round_ceil/round_trunc/round_half(x)` (`round_half` = ties AWAY from zero; `.floor() .ceil() .trunc() .round()` are deprecated aliases) · comparisons as Integer |
432
+ | **String** | `+` (concat); comparisons | `.concat(s) .repeat(n) .substring(a, b) .upper_case() .lower_case() .trim() .trim_start() .trim_end()` · `.replace(old, new) .split(sep)` · `.length() .starts_with(p) .ends_with(s) .contains(s) .index_of(s)` · `.parse(T)` ❗ (strict whole-string) `.parse_json(T)` · `.try_parse(T) -> Option<T>` (none on any failure) · `.encode_utf8() .encode_utf16()` · `.regex_contains(pat, flags="") .regex_index_of(pat, flags="") .regex_replace(pat, repl, flags="")` (the builtins TypeScript has no method for) · never an f-string (it would constant-fold the proxy) — `East.str(…)` or `+` |
433
+ | **DateTime** | comparisons; a python `datetime` literal lifts as a DateTime | `.get_year() .get_month() .get_day_of_month() .get_day_of_week()` (Monday = 1) `.get_hour() .get_minute() .get_second() .get_millisecond()` · `.add_milliseconds(n) .add_seconds .add_minutes .add_hours .add_days .add_weeks` and `.subtract_milliseconds … .subtract_weeks` (an Integer or Float `n`) · `.duration_milliseconds(other) -> Integer`, `.duration_seconds/minutes/hours/days/weeks(other) -> Float` ❗ `a.duration_days(b)` is `b − a` (positive when `b` is later — the TS method; the namespace `East.DateTime.duration_milliseconds(a, b)` is the raw builtin, `a − b`) · `.to_epoch_milliseconds()` · `.print_formatted(fmt)` (Day.js tokens) |
434
+ | **Blob** | comparisons | `.size() .get_uint8(i)` · `.decode_utf8() .decode_utf16()` · `.decode_beast(T, version="v1")` (`"v2"` = the beast2 family) · `.open_beast(T)` (TS `openBeast`: an indexed beast2 `Array`/`Set`/`Dict` blob as a FROZEN paged value — `size`, keyed reads and `for_` decode one segment; an index-less blob or a Ref-/function-bearing element shape decodes whole; the header's type must equal `T`, checked before any decode) · `.decode_csv(RowT, config=None, **options)` |
435
+ | **Array / Set / Dict / Vector / Matrix** | `xs[i]` (a negative LITERAL index raises — spell `xs.get(xs.size() - 1)`) · `d[k]` ❗missing · comparisons | The eager surface, name for name and signature for signature — [EastArray](#eastarray--complete-method-surface) · [EastSet](#eastset--complete-method-surface) · [EastDict](#eastdict--complete-method-surface) · [EastVector](#eastvector--complete-method-surface) · [EastMatrix](#eastmatrix--complete-method-surface) — less what only a value has: the python protocol (Array `insert`/`remove`/`count`/`index`, Set `add`/`remove`/`discard`, Dict `values`/`items`), the columnar hatches (`to_columns`, `from_columns`, `map_batches`, `update_many`), the numpy/torch bridges, the classmethod generators (`East.Array.range` and kin are their twins) and the deprecated `map_elements`. A mutator yields Null or Boolean — sequence it with `b.do` or `East.block`. A Dict callback is `(value, key)`: `fn(b, v)` or `fn(b, v, k)`, a fold step `fn(b, acc, v[, k])`, a collision handler `combine(b, existing, incoming[, key])` |
436
+ | **Struct** | `r.field` / `r["field"]` (both build IR, both work on real rows) | build a row with a dict literal `{"k": expr, …}` or `struct({…}, T)` (dual-mode) |
437
+ | **Variant** | — | `.match({case: fn(b, payload)}, default=fn(b))` (TS's partial match; the arms must agree on one East type — a `some(x)` arm types its `none` sibling) · `.match_tag(tag, fn, default)` · `.unwrap(tag="some", on_other=None)` ❗ · `.has_tag(tag) .get_tag()` · Option: `.is_some() .is_none() .unwrap_or(d)` · build with `some(expr)` / `none` / `variant(case, payload)`, typed from context (the build's declared output, a typed `if_else` sibling, a declared struct field, or an `out=` pin) |
438
+ | **Ref** | — | `East.ref(v)` builds one · `.get()` · `.update(v)` (TS `RefUpdate`; `.set` and the read-modify-write `update(fn)` are deprecated — write `r.update(f(r.get()))`) · `.merge(v, fn(b, current, patch))` |
439
+ | **Function** | `f(x, …)` — a `Call` node (a `CallAsync` inside an async body) | a Function-typed parameter or a `b.const(East.function(…))` is callable in the body; `FunctionType` parameters bind with `.bind(fn_value)` |
440
+
441
+ The expression methods take the eager keywords — `out=` (`map`, `filter_map`,
442
+ `map_reduce`, `flat_map`, `flatten_to_set`, `to_array`, `to_set`), `key_out=` /
443
+ `value_out=` (`to_dict`), `key_out=` / `acc_out=` (`group_reduce`), `pred=`,
444
+ `key=`, `value_fn=` — and an `out=`-family pin also TYPES the callback's build, so
445
+ a pinned callback can build a general variant with no other context. Result
446
+ types come from the build, never a data sample: `out=` is optional everywhere,
447
+ and an EMPTY collection derives what a full one does. `some`/`every`/`first_map`
448
+ compile to the native short-circuiting FirstMap scans (`some([])` is False,
449
+ `every([])` True). An unsupported method raises `ExpressionError` naming the
450
+ supported set.
451
+
452
+ ### The standard library — the TypeScript `East.<Type>.*` functions
453
+
454
+ Every namespace carries the East standard library — TypeScript's
455
+ `libs/east/src/expr/libs/*.ts`, body for body, snake_cased (`printCompact` →
456
+ `print_compact`; the TS misspelling `printCommaSeperated` stays
457
+ `print_comma_seperated`, with `print_comma_separated` as a python twin). Each is
458
+ an `East.function` built on first use: on plain values it runs natively, inside a
459
+ body it is a call of its inline Function node like any artifact — as
460
+ TypeScript's is:
461
+
462
+ ```python
463
+ @East.function([IntegerType, FloatType, DateTimeType], StringType)
464
+ def label(b, n, x, d):
465
+ return East.str(East.Integer.print_compact(n), " / ", # "1.23M"
466
+ East.Float.print_currency(x), " @ ", # "$1,234.57"
467
+ East.DateTime.print_formatted(East.DateTime.round_down_hour(d, 6), "HH:mm"))
468
+ ```
469
+
470
+ The outputs are TypeScript's exactly — a Float prints with its point
471
+ (`East.Float.print_percentage(1.0, 0)` is `"100.0%"`), the Integer compact forms
472
+ carry two decimals below ten units (`print_compact(1500)` is `"1.50K"`), and
473
+ `round_nearest`'s Integer half-step truncates (`round_nearest(17, 5)` is `20`).
474
+ Every function, with its outputs, is in
475
+ [East.<Type> namespaces](#easttype-namespaces--the-builtins-and-the-standard-library);
476
+ the dual-mode generators (`East.Array.range`, `East.Set.generate`,
477
+ `East.Vector.zeros`, …) are in [Container generators](#container-generators-classmethods).
478
+
479
+ ### Control flow — the expression forms (`East.while_`, `East.for_`, …)
480
+
481
+ Prefer the collection methods: `map`/`filter`/`reduce`/`group_reduce`
482
+ express most work and are the fastest thing in the runtime. Reach here when
483
+ the next step **depends on the last** — a worklist, a BFS, a fixpoint, a
484
+ topological replay — which no data-parallel method can express. The
485
+ statement forms above (`b.while_`, `b.for_`) build the same nodes; these are
486
+ their EXPRESSION twins, usable inside a callback or a one-expression body.
487
+
488
+ Python cannot overload `=`, and `while`/`if` collapse to a `bool` before any
489
+ build sees them. That is the constraint that produced `East.if_else`, and the
490
+ answer is the same shape: **`while_` is to `while` what `if_else` is to `if`**.
491
+ The body is a pure function of the state that RETURNS the next state, and the
492
+ state struct IS the loop's local variables:
493
+
494
+ ```python
495
+ total = East.while_({"i": 0, "acc": 0},
496
+ cond=lambda b, s: s.i < n,
497
+ body=lambda b, s: {"acc": s.acc + s.i, "i": s.i + 1}).acc
498
+
499
+ # `East.if_else` is the `if`; a field left as `s.field` is the empty else;
500
+ # `{**s, …}` changes one field and keeps the rest
501
+ East.for_(rows, {"n": 0, "hi": 0.0},
502
+ lambda b, s, r: {"n": s.n + 1,
503
+ "hi": East.if_else(r.price > s.hi, r.price, s.hi)})
504
+ ```
505
+
506
+ This lowers to a `Ref` holding the state, a `While`/`For*` node whose body is
507
+ one `RefUpdate`, and a final read — the whole loop runs inside east-c. Every
508
+ construct is dual-mode like `East.if_else`: outside a build it runs the plain
509
+ python loop, so one body serves both a callback and a direct call on plain
510
+ East values. A worked example is in
511
+ [Key Patterns](#sequential-logic-that-stays-in-east-c-worklist--replay).
512
+
513
+ | Call | Emits | Notes |
514
+ |---|---|---|
515
+ | `East.if_else(cond, value, …, otherwise)` | `IfElse` | cond/value pairs then the else — an if/elif/else chain is ONE node; exactly one arm evaluates |
516
+ | `East.while_(state, cond, body, label=…)` | `While` | `cond(b, s) -> Boolean`, `body(b, s) -> next state` |
517
+ | `East.for_(coll, state, body, label=…)` | `ForArray`/`ForSet`/`ForDict` | Array `body(b, s, el[, i])`, Set `body(b, s, el)`, Dict `body(b, s, k, v)` |
518
+ | `East.block(a, b, …)` | `Block` | evaluates in order, yields the last — the sequencing point for mutators (`East.block(lambda b: …)` is the statement form) |
519
+ | `East.let(value, fn)` | `Let` | `fn(b, bound)` — bind once, use many times (explicit CSE inside a loop) |
520
+ | `East.ref(v)` | `NewRef` | a cell — `.get()` / `.update(v)` / `.merge(v, fn(b, current, patch))` |
521
+ | `East.label(name=None)` | — | names a loop, for `break_`/`continue_` from a NESTED one |
522
+ | `East.break_(state=…, label=…)` / `East.continue_(state=…, label=…)` | `Break` / `Continue` | leave / next iteration, optionally committing a last state |
523
+ | `East.try_catch(body, handler, finally_=None)` | `TryCatch` | `body(b)`, `handler(b, message[, stack])`, same East type as `body` |
524
+ | `East.new_array/new_set/new_dict(…)` | `NewArray`/`NewSet`/`NewDict` | a FRESH collection per evaluation — the loop accumulator |
525
+ | `East.new_vector(T, values)` / `East.new_matrix(T, rows, cols, values)` | `NewVector`/`NewMatrix` | a fresh tensor from scalar expressions |
526
+
527
+ **Accumulate in place.** Threading a collection through the state rebuilds it
528
+ every iteration (`order.concat(…)` copies — O(n²) over the loop). The
529
+ mutators extend it in O(1). Each returns what its EAGER twin returns, so
530
+ sequence with `East.block`; a mutation written as a bare statement is
531
+ evaluated and thrown away, and the build says so rather than compiling a
532
+ loop that silently does nothing:
533
+
534
+ ```python
535
+ counts = East.for_(rows, {"counts": East.new_dict(StringType, IntegerType)},
536
+ lambda b, s, r: East.block(
537
+ s.counts.insert_or_update(r.sku, 1, lambda b, x, y: x + y),
538
+ s)).counts
539
+ ```
540
+
541
+ ⚠️ Seed accumulators with `East.new_array/new_set/new_dict`. A captured
542
+ `EastArray(T)` works too — a loop's seed is built fresh per call — but
543
+ anywhere ELSE in a function a captured collection is a build-time snapshot
544
+ shared by every call, and mutating one raises rather than leaking state
545
+ between calls.
546
+
547
+ ### Queries — `East.jq`
548
+
549
+ `East.jq(input, program, result_type)` is a jq query as East code: the jq is
550
+ parsed, checked against the input's East type and translated to ordinary East
551
+ IR when the program is built — loops, `Match`, the builtins — so it runs
552
+ wherever East runs, with no interpreter. In a body it is a call of the `Query`
553
+ builtin, which carries the program as written beside its translation and gives
554
+ the translation, so `to_python_source` prints it back as `East.jq(…)`. It is
555
+ the twin of TypeScript's `East.jq` and builds the same IR. Dual-mode, like the
556
+ standard library: in a body it builds the call; on values it compiles the
557
+ translation (once per query and input type) and runs it now.
558
+
559
+ ```python
560
+ from east import East, ArrayType, FloatType, IntegerType, StructType, array
561
+
562
+ Order = StructType([("id", IntegerType), ("total", FloatType)])
563
+ orders = array(Order, [{"id": 1, "total": 250.0}, {"id": 2, "total": 1200.0}])
564
+
565
+ # in a body: an expression of `result_type` (here an ArrayExpression), so its methods chain
566
+ count_big = East.function([ArrayType(Order)], IntegerType,
567
+ lambda b, orders: East.jq(orders, "[.[] | select(.total > 1000) | .id]",
568
+ ArrayType(IntegerType)).size())
569
+ count_big(orders) # 1
570
+
571
+ East.jq(orders, "map(.total) | add", FloatType) # 1450.0 — on values, run now
572
+
573
+ # several inputs: a dict, checked as an e3 root — `.orders` reads the `orders` input alone
574
+ East.jq({"orders": orders, "limit": 1000.0},
575
+ ".limit as $l | [.orders[] | select(.total > $l) | .id]", ArrayType(IntegerType)) # [2]
576
+ ```
577
+
578
+ | | |
579
+ |---|---|
580
+ | ❗ `result_type` | Required: the outputs' type for a query that gives one output (`length`, `[…]`, `reduce`), an `Option` of it for at most one (`first(f)`), an `Array` of it for any number (`.[] \| .id`). The query must check to exactly it, or the build raises `type_mismatch: the query gives Array<Float>, not the Integer it was given.` |
581
+ | A query that does not check | `QueryError` when the program is built (a `QueryBuildError`, an `ExpressionError` too), its diagnostics each `jq <line>:<column>: <message>`: `unknown_field: .totl is not a field of Struct{id: Integer, total: Float}. Did you mean .total?` |
582
+ | An error the query raises | `error(v)`, an integer `% 0`, a date that does not parse: an East runtime error located in the jq text (`jq 1:5`), which `b.try_` catches; on values, a `QueryError` with one `runtime` diagnostic |
583
+ | The input's type | In a body, the expression's; on values, `type_of(value)`. Where that is narrower than the type meant (a variant's is its one case), run `evaluate_jq(program, value, input_type=T)` |
584
+ | The language | jq 1.8 over East's types: Dict keys of any type, DateTime a type (an ISO literal compares with it), Integers exact, a variant read as `{type, value}`. Every departure is listed in `libs/east/devdocs/QUERY.md` §13 |
585
+ | Host tools (`from east.query import …`) | `check_jq(program, input_type, root=False)`, without building: the program as written (`.program`, a `JqType` tree whose constants are typed East values; `None` when it does not check), the type it was checked against (`.input_type`), its `.element_type` and `.multiplicity` (`"one"` / `"maybe"` / `"many"`; `None` when it does not check), the datasets a root query reads (`.reads`) and its `.diagnostics`. A query's types are never stored: check the program against its data wherever they are needed · `evaluate_jq(program, value, input_type=T)`: a query whose input type is data · `translate_jq(checked)` · `parse_jq` / `print_jq` / `lex_jq` · `QueryError` |
586
+
587
+ ### What a body may reference
588
+
589
+ - Its own parameters; plain scalar constants (closure floats / ints / strings /
590
+ datetimes bake in — the same value per element either way); East types and
591
+ values (`east_null` included); the `East` namespace, `East.if_else` included;
592
+ the `struct`/`variant`/`some`/`none` constructors (dual-mode: they build IR
593
+ when handed expression fields); **East.function artifacts** (dual-mode: a
594
+ call of their inline Function node at any nesting depth); **compiled East function values**
595
+ (`.bind` results, `compile_from_*` functions — a CALL on one lowers to a native
596
+ IR `Call`); and, two wrapper levels deep — enough for helper lambdas that
597
+ compose a callback — other python functions that pass the same rules.
598
+ - Anything else RAISES an `ExpressionError` **naming the binding**: a module
599
+ (`random.…`, `np.…`), a python builtin (`len`, `str`), a mutable python
600
+ capture, closure mutation (`nonlocal x; x += 1`). A closed-over East
601
+ *collection* raises in an eager callback too — a capture snapshots, `.bind`
602
+ stays live, and which you meant is your choice: an explicit `East.function`
603
+ snapshots a side table, `.bind(table)` keeps it live. For genuine python
604
+ semantics write an explicit python loop outside the body. Side effects are
605
+ therefore never lost or silently doubled: the body runs once, at build time, or
606
+ not at all.
607
+ - Arithmetic follows East types exactly: no implicit Integer↔Float mixing
608
+ (`.to_float()` / `.to_integer()` convert), and `/` is Float division.
609
+ - Captures are CACHED: an eager callback whose code object, captured bindings
610
+ and declared signature match an earlier call reuses its compiled function, so
611
+ the per-group aggregate `group_to_arrays(key).to_array(lambda b, es, k: {…})`
612
+ builds each inner lambda once, not once per group. ⚠️ A lambda whose CAPTURES
613
+ change per call (`lambda b, r: r.v > g` in a python loop over `g`) rebuilds
614
+ every call, since each capture bakes into a different function — hoist an
615
+ `East.function(...)` and pass the varying value as a bound parameter.
616
+
617
+ ### Performance — the levers, ranked
618
+
619
+ The point of the machinery: **data stays in east-c and python never runs per
620
+ element**.
621
+
622
+ 1. **Keep values East end-to-end.** `EastArray`/`EastSet`/`EastDict` are
623
+ C-backed, and every eager method runs its loop natively however deeply the
624
+ element type nests. `list(...)`/`dict(...)` or a python loop pays a boxing
625
+ crossing per element — convert at most once, at the very end.
626
+ 2. **Let the callbacks build, and chain on the results** — a chain of
627
+ `map`/`filter`/`reduce` stays native between steps. Return a dict literal
628
+ (`lambda b, r: {"a": …, "b": …}`) to derive every column in ONE pass. **Reuse
629
+ a python variable to share work** (build-time CSE): `fields =
630
+ r.data.split("|")` read for 30 columns compiles to ONE `Let` — the split runs
631
+ once per row (re-calling `.split()` per column re-emits it). Loop-invariant
632
+ subexpressions hoist out of nested lambdas to the function body, a value read
633
+ once inside a callback included, so `table = derive(rec)` before a `.map` runs
634
+ once. A hoist never leaves a guard: a subexpression shared inside `b.try_` /
635
+ `East.try_catch` binds inside that body, so `s.try_parse(T)` still answers
636
+ `none` when CSE'd. `East.function(..., cse=False)` switches the pass off and
637
+ builds exactly what the body spells (what the transpiler emits).
638
+ 3. **Side tables: capture small ones, `bind` big ones** — opposite contracts:
639
+ - **Capture (snapshot).** A collection captured by an `East.function` body
640
+ is snapshot into its IR — hoisted and identity-deduped, so it builds **once
641
+ per compiled function** and each lookup runs in C. Right for tables up to
642
+ ~10⁴–10⁵ entries; the snapshot's build cost and memory ride the function (a
643
+ 1M-entry dict takes ~10 s), and later mutations are **not** seen. Only an
644
+ EXPLICIT build snapshots: an eager callback refuses a mutable capture.
645
+ - **`East.function(...).bind(table)` (by reference, live).** Declare the
646
+ table as a trailing parameter and pre-bind it: zero copy, O(1) at any size
647
+ (1M entries: ~0.1 ms), the per-row cost of the hoisted case, and the
648
+ function **sees later mutations**. Rebinding gives independent callables,
649
+ the unbound function stays usable, and a wrong-typed value raises
650
+ `TypeError`.
651
+
652
+ ```python
653
+ fx = EastDict(StringType, FloatType, rates) # small table → capture
654
+ to_usd = East.function([Row], FloatType, # (the EXPLICIT build
655
+ lambda b, r: r.amount * fx.get_or_default(r.ccy, 1.0)) # opts into snapshot)
656
+
657
+ TableT = DictType(StringType, FloatType) # huge table → bind
658
+ conv = East.function([Row, TableT], FloatType,
659
+ lambda b, r, t: r.amount * t.get_or_default(r.ccy, 1.0))
660
+ rows.map(conv.bind(big_table)) # loop + lookup stay in east-c
661
+ conv.bind(t1, t2) # multi-table: binds the TRAILING parameters in order
662
+ ```
663
+
664
+ Function *parameters* always cross by reference (zero copy, any size, any
665
+ depth); `bind` is what lets an eager method use a parameter-taking function
666
+ where the callback signature is fixed.
667
+ 4. **Hoist `East.function(...)` out of python loops** and reuse it — building
668
+ is cheap, not free.
669
+ 5. **When logic must stay python, go columnar** — one crossing per column or
670
+ batch instead of per row × field, then back with `from_columns`/`extend`
671
+ ([Columnar escape hatches](#columnar-escape-hatches--when-the-logic-must-stay-python)).
672
+ 6. **See how a hot call ran** — `east.runtime.compiler.eager_stats()`:
673
+ `function_direct` counts callbacks that rode a precompiled function value
674
+ straight in, `c_to_py_decodes` counts values boxed C→python (an eager method
675
+ quietly decoding a whole collection shows here), and the `beast2_*` counters
676
+ report column projection. There is no per-element python counter because
677
+ there is no per-element python path.
678
+
679
+ ### IR ↔ python: `east-py transpile` and the `east-c ir` toolbox
680
+
681
+ Any East IR — an `East.function` artifact, or a `.json` / `.beast2` export from
682
+ TypeScript, east-c or east-node — prints as an idiomatic python module that
683
+ REBUILDS it through the surface above:
684
+
685
+ ```python
686
+ from east.codegen import to_python_source
687
+ print(to_python_source(classify)) # `@East.function(..., cse=False)` / `def main(b, …)`
688
+ ```
689
+
690
+ ```bash
691
+ east-py transpile program.json -o program.py --name main # the same, from a file
692
+ east-py transpile program.json -p east-py-std -p east-py-io # platform calls print as the packages' own
693
+ # functions and named types, not restated declarations
694
+ east-c ir normalize program.json -o canonical.json # the canonical form
695
+ east-c ir diff a.json b.beast2 # first difference, or "identical"
696
+ east-c ir convert program.json -o program.beast2 # json <-> beast2, source map intact
697
+ ```
698
+
699
+ - **The contract:** `build(print(IR))` equals `IR` under `east-c ir normalize`
700
+ (location ids stripped, variables and labels renamed in the TypeScript
701
+ lowering's order, captures recomputed, recursive type ids renumbered — one
702
+ normalizer, in libeast-c). The TypeScript printer (`East.toSource`, `east-node
703
+ transpile`) is its twin: IR → python → IR → TypeScript → IR is equal at every
704
+ leg, over the whole compliance corpus and every example
705
+ (`docs/conventions/EAST_CODEGEN.md`).
706
+ - **Builtins print through one spelling table** (`east.codegen.spellings`):
707
+ operators only where they are exact — `+ - *` on numbers, `/` on Floats, the
708
+ comparisons, `& | ^ ~` on Booleans, `+` on Strings — named `East.<Type>.*` or
709
+ method spellings elsewhere, and `East.builtin(...)` for `RAW_ONLY`.
710
+ - **A function used as a VALUE** (`$.let(East.DateTime.roundDownWeek)`) prints
711
+ as the inline `@East.function` it is — the IR carries the body, not the name.
712
+ - **A platform call** prints as a hoisted `East.platform(...)` declaration — or,
713
+ given the implementing packages
714
+ (`to_python_source(ir, providers=providers_for(["east_py_std"]))`, `east-py
715
+ transpile -p east-py-std`), as `from east_py_std import fs_read_file` and the
716
+ call itself, with a type the package names (`GzipOptionsType`) printed by its
717
+ name; the plugin's example index is rendered this way. A provider is used only
718
+ where its declared signature IS the node's, so a drifted signature keeps the
719
+ declaration rather than printing a wrong import. A package whose function is
720
+ implemented in C exports the DECLARATION under the same name
721
+ (`simulation_run`, `optimization_iterative`), which a body calls identically.
722
+
723
+ ### Diagnostics at edit time — `east-py lint`, `east-py check`, flake8, `east-py lsp`
724
+
725
+ Everything the build refuses — a body without the block, `//` on an
726
+ expression, an f-string over one, `if` on one, a callback reaching for `np` — is
727
+ also a **rule** in `east.diagnostics` (the python twin of
728
+ `@elaraai/east-diagnostics`). The rules read a file's `ast`, find its East bodies
729
+ (an `East.function` body and everything nested in it, an eager callback on an
730
+ East value, a `@East.platform_function`'s East inputs), and say at edit time what
731
+ the build would say: every rule's message IS the build's refusal for the same
732
+ code, pinned by building the very source the rules read.
733
+
734
+ Two tiers, at two costs. The **rules** read the `ast` — instant, and useful even
735
+ on a file that does not parse. The **check** runs the build — the only type
736
+ checker a python East body has, since East's type checker IS the builder — at
737
+ the cost of importing the module.
738
+
739
+ ```bash
740
+ east-py lint src/ # file:line:col: category [rule] message — exit 1 on any finding
741
+ east-py lint src/ --format json # the findings as records
742
+ east-py lint src/ --disable no-deprecated-alias --exclude fixtures
743
+ east-py lint --list-rules # every rule, with its EAS code
744
+ east-py check src/ # the BUILD's errors — a file, a directory, or a module; --format json
745
+ flake8 --select EAS src/ # the same rules inside flake8 (east-py-cli registers the plugin)
746
+ east-py lsp # a Language Server over stdio, both tiers (--probe: can it start here?)
747
+ # pylsp # python-lsp-server runs the rules too (east-py-cli registers the plugin)
748
+ ```
749
+
750
+ - **`east-py check`** imports each target — a `.py` file, a directory (walked
751
+ as `lint` walks it) or a dotted module name, each executed afresh — builds
752
+ every East function in it, and reports EVERY failure at its authoring line: a
753
+ body whose type differs from the declared `out`, a callback the capture
754
+ refuses, an `IRAnalysisError`. Importing runs the module, so `EAST_CHECK=1` is
755
+ set for the duration — skip import-time work when you see it.
756
+ `--only-if-enabled` honours the project's opt-in (below), as the plugin's read
757
+ hook does, so the hook and the language server agree.
758
+ - **`east-py lsp`** (needs `pygls`) runs the rules on every change and the build
759
+ tier on **open** (debounced) and **save** (at once), on one long-lived worker
760
+ thread, and only when the project opts in: the build reads the module from
761
+ DISK, so on an unsaved buffer it would report the saved version's errors. A
762
+ save evicts the module from the warm process, so its importers are checked
763
+ against the new version. Pyright/Pylance has no plugin API, so flake8 and
764
+ pylsp are the editor paths that need no East-specific server; the Codex
765
+ plugin launches the project's own `east-py lsp` for `.py` files, and falls back
766
+ to `east-py lint` per change when that server cannot start.
767
+
768
+ The rules — `east-py lint --list-rules` prints them:
769
+ | Rule | Flags | What the build says |
770
+ |---|---|---|
771
+ | `body-takes-block-first` (EAS001) | `lambda x: …`, a body whose parameter count is not the declared count plus the block, `b.price`, `x + b` | `a body takes the block first` |
772
+ | `no-operator-fork` (EAS002) | `//` `%` `**` on an expression, `xs[-1]` | the operator-fork texts — `East.Integer.divide` / `remainder` / `pow`, `xs.get(xs.size() - 1)` |
773
+ | `no-python-formatting` (EAS003) | `f"{x}"`, `str(x)`, `print(x)`, `format(x)`, `"{}".format(x)`, `"%d" % x` | `f-strings / str() cannot be traced … East.String.print(T, value)` |
774
+ | `no-python-boolean` (EAS004) | `if`/`while`/`assert`/`and`/`or`/`not`/`x if c else y` on an expression, `in`, `for … in xs`, `len`/`int`/`float`/`bool`/`sum`/`sorted`/`max`… over one | `python \`if/and/or/not\` cannot be traced …` (`iteration`, `len()`, …) |
775
+ | `no-python-round` (EAS005) | `round(x)` on a Float expression | `East.Float.round_half(x) rounds half away from zero …` |
776
+ | `no-python-work` (EAS006) | an EAGER callback loading a module (`np`, `math`), a python builtin the capture does not admit (all but `abs`/`bool`/`isinstance`), a name imported from the standard library or an installed package that has no East form (`from math import floor`; a constant like `pi` lifts), a mutable East collection, or a `def` doing any of those (a clean macro `def` is fine) | `the callback cannot be captured automatically: it references np …` |
777
+ | `no-statement-on-outer-block` (EAS007) | `b.if_(p, lambda _b: b.assign(…))` — a statement on an enclosing body's block | `b.assign() was called on an OUTER block …` |
778
+ | `no-deprecated-alias` (EAS008, warning) | `.fold`, `.lower`, `East.Boolean.and_`, … (read off the surface's own deprecation docstrings) | `.fold() is deprecated: the spelling is .reduce() (the TypeScript name)` |
779
+ | `no-discarded-expression` (EAS009) | a bare `acc.push_last(x)` / `East.error(…)` / `xs.size()` line in a body | `.push_last() was evaluated and thrown away … b.do(…)` |
780
+ | `prefer-some-none` (EAS010, warning) | `variant("some", x)` / `variant("none", None)` | use `some(value)` / `none` |
781
+ | `no-handrolled-variant` (EAS011, warning) | a `{"type": …, "value": …}` dict | `variant("Tag", value, Type)` — the encoder needs what it constructs |
782
+ | `prefer-explicit-east-type` (EAS012) | `b.let([])` / `b.let([1, 2])` / `b.const(list())` — a bare python list, set or tuple, empty or not | `cannot lift python value of type list …` — pass the East type: `b.let([], ArrayType(IntegerType))`, or build it with `East.new_array` |
783
+ | `no-let-const-in-expression` (EAS013, warning) | `b.let` / `b.const` buried in a call argument, an element, a chain target (a tuple of declarations is fine) | give the declaration its own statement |
784
+ | `no-untracked-east-data` (EAS014, suggestion) | a plain python list/dict local reaching an expression's method | `b.const(rows, Type)` — a local carries no East type and re-inlines |
785
+ | `no-redundant-east-cast` (EAS015, warning) | `b.let(East.value(x, T), T)` | pass the value and type to `b.let` directly |
786
+ | `prefer-let-const-over-east-value` (EAS016, suggestion) | `East.value(...)` assigned or returned inside a body | `b.const(value, Type)` — `East.value` erases the type at the binding |
787
+ | `no-host-comparison-on-east-values` (EAS017) | `<` / `>` on a decoded VARIANT or OPTION outside a body — it raises `TypeError` (`==` is structural and fine; a decoded Integer is an `int`) | `compare_for(T)` / `less_for(T)`, `make_east_key(T)` for `sorted` |
788
+ | `no-module-scope-east-macro` (EAS018, warning) | a module-scope helper a body calls to build IR, or to assemble a composite `f"{a}|{b}"` key | make it an `East.function`; model typed / nested East data |
789
+ | `no-build-time-clock` (EAS019, warning) | `datetime.now()` / `time.time()` at module scope (a script's `if __name__ == "__main__":` is not module scope) | author the constant, or read the clock at runtime — a platform function, or `east_py_std`'s `time_now()` in a body |
790
+ | `no-inline-credentials` (EAS020, warning) | a literal `password` / `token` / `secret_access_key` (a localhost sibling host, a header NAME like `X-Api-Key`, and a `${TEMPLATE}` are exempt) | read it at runtime: `east_py_std`'s `env_get("VAR")` in a body, `os.environ` in a platform function — IR is content-addressed and replicated |
791
+ | `no-compile-time-data-injection` (EAS021, warning) | `open()` / `json.load` / `Path.read_text` / `os.environ` at module import, when the result REACHES East (a body reads it, or a boundary call takes it) | load at runtime: an e3 input, a dataset, a platform function |
792
+ | `no-python-east-data` (EAS022, warning) | East rows assembled by a module-scope comprehension or loop, then handed to a body | write the rows out, or produce them at runtime |
793
+ | `no-python-string-building` (EAS023, warning) | an f-string assembling an East string CONSTANT — a regex, a template, a key | spell the constant out |
794
+ | `no-derived-struct-fields` (EAS024, warning) | `Derived = StructType([… for f in Other.value])` | a type declaration is a wire format — spell the fields |
795
+ | `no-python-data-work` (EAS025, warning) | a python helper doing parse / strip / null-check / coerce work on an EXPRESSION a body hands it | express it in East, where it runs on every row |
796
+
797
+ Configure it once in the project's `pyproject.toml`; every surface reads it, and
798
+ `east-py lint`'s own flags add to it:
799
+
800
+ ```toml
801
+ [tool.east-py]
802
+ check = true # let an EDITOR run the build tier (off by default)
803
+ disable = ["no-deprecated-alias"] # rules to skip
804
+ exclude = ["fixtures", "vendor"] # extra directory names not to walk
805
+ ```
806
+
807
+ `check` is off unless a project asks: the rules READ a file, the build RUNS it,
808
+ and an editor should not import someone's modules on save because a language
809
+ server happened to be installed. `east-py check` on the command line is consent
810
+ in itself and ignores the setting. Unreadable configuration falls back to the
811
+ defaults — a diagnostics tool must never stop a project building.
812
+
813
+ A file that does not import `east` is never diagnosed; a line ending in `# noqa`
814
+ (or `# noqa: EAS002` / `# noqa: no-operator-fork`) is skipped; `.venv`, `venv`,
815
+ `node_modules`, `dist`, `build`, `.git`, `__pycache__`, `tests` and `test` are not
816
+ walked. The rules are syntactic, so a clean `lint` is necessary, not sufficient —
817
+ the type errors live behind `east-py check`. The rules are disjoint (each
818
+ mistake is one rule's to report). One TypeScript rule has no python twin on
819
+ purpose: `no-reinlined-east-binding` warns that a JS `const` holding an `Expr`
820
+ re-inlines it at each use, but the python build's CSE binds a reused python local
821
+ to ONE `Let`, so the hazard does not exist here.
822
+
823
+ ### Cross-language functions: `east-py export-functions` and `East.import_function`
824
+
825
+ A python `East.function` is *called* from a TypeScript e3 task — and a
826
+ TypeScript one from python — as pure IR: no python at run time, no platform
827
+ bridge. The functions travel as a **function manifest** (each function's IR,
828
+ declared type and platform dependencies with the package providing each);
829
+ the importer names the function and declares its type; linking checks the
830
+ type exactly and embeds the IR. Declare them on the package's root module:
831
+
832
+ ```python
833
+ # packages/pricing/src/pricing/__init__.py — the package's root module
834
+ from east import East
835
+ from east.types.types import FloatType, StructType, IntegerType
836
+
837
+ Row = StructType([("qty", IntegerType), ("price", FloatType)])
838
+ score = East.function([Row], FloatType, lambda b, r: r.qty.to_float() * r.price)
839
+ east_functions = {"score": score} # name -> East.function artifact. CLOSED values: the
840
+ # exported IR must BIND every name it reads. A build's
841
+ # own hoisted constants — a captured lookup table, a
842
+ # stdlib format string — ride along and close over
843
+ # nothing; a .bind result (no IR of its own) and
844
+ # an unresolved import are refused, and a genuinely free
845
+ # variable is refused BY NAME
846
+ ```
847
+
848
+ ```typescript
849
+ // the e3 task, in TypeScript (see the e3 skill) — the reference is all there is to write:
850
+ // e3.export finds `pricing` in the uv workspace, exports it (east-py export-functions, in the
851
+ // project's .venv) and links; the providers come from the task's runner
852
+ const score = East.importFunction("pricing", "score", FunctionType([RowType], FloatType));
853
+ await e3.export(pkg, "out.zip");
854
+ ```
855
+
856
+ For a package built elsewhere (published, another repo), or to link
857
+ in-process, write the manifest where the package lives and pass it:
858
+
859
+ ```bash
860
+ east-py export-functions pricing -o pricing.functions.beast2 -p east-py-std
861
+ # -p names the platform package implementing each platform call the functions make;
862
+ # a call no package provides fails the export (the manifest records the provider)
863
+ # → e3.export(pkg, "out.zip", { functions: ["./pricing.functions.beast2"] })
864
+ ```
865
+
866
+ The other direction, in python:
867
+
868
+ ```python
869
+ manifest = East.decode_function_manifest(Path("maths.functions.beast2").read_bytes()) # east-node export-functions wrote it
870
+ double = East.import_function("maths", "double", FunctionType([IntegerType], IntegerType))
871
+ user = East.function([IntegerType], IntegerType, lambda b, x: double(x) + 1) # callable in a body
872
+ ir, imports = East.link_imports(user, [manifest]) # exact type check; the IR embedded as a Let
873
+ compile_from_value(ir, [])(20) # 41 — pure IR, runs on any runner
874
+ ```
875
+
876
+ Unlinked, the reference is a `Platform` node named `east.importFunction`, and
877
+ compiling it raises naming that platform. `East.export_functions(pkg, version,
878
+ {…}, providers)` and `East.encode_function_manifest` are the API behind the CLI,
879
+ and `East.platform_dependencies(fn)` lists the platform functions an artifact
880
+ calls, in first-use order (unresolved imports skipped). Each also answers to its
881
+ TypeScript camelCase name (`East.exportFunctions`, `East.linkImports`, …). The
882
+ contract and the runner rules: `docs/conventions/EAST_CODEGEN.md`.
883
+
884
+ ## East values — the eager runtime
885
+
886
+ ### Work in East values — don't round-trip through Python
887
+
888
+ Inside a `@East.platform_function`, or any east-py code over runtime data, **do
889
+ the work with the East values you were handed and their chained eager methods.
890
+ Never down-convert to a python `list`/`dict`/`set`, loop in the interpreter, and
891
+ rebuild an East value** — the most common way east-py gets used badly, and it
892
+ changes both cost and correctness:
893
+
894
+ - **Speed.** An eager method hands the value's pointer to the native builtin
895
+ with no copy and returns a new handle, so
896
+ `a.filter(...).group_by(...).map(...)` runs traversal, allocation, ordering and
897
+ set/dict algebra in C and the data never leaves it. Down-converting is an O(n)
898
+ decode, an interpreter loop and an O(n) re-encode, every time. (A tensor's
899
+ `to_numpy()` is a zero-copy view; a python element loop is not.)
900
+ - **Correctness.** East methods use East's *total order* and equality (right
901
+ for floats and NaN, mixed types, variants by name) and keep Sets and Dicts
902
+ deterministically ordered; `sorted()`, a bare `dict` or `set()` get these
903
+ subtly wrong and diverge from the other runtimes.
904
+ - **Intermediates too, not just the boundary.** The failure is the *sandwich* —
905
+ East in → python `list`/`dict` logic → East out. If the logic is
906
+ East-expressible (mapping, filtering, grouping, joining, reducing, set/dict
907
+ algebra), every intermediate stays an east-c value. Prefer the declarative
908
+ reducers (`reduce`/`map_reduce`/`group_reduce`/`to_dict(..., combine=…)`) over
909
+ ANY hand-rolled accumulation: a reducer is one native call, while a manual loop
910
+ — python, or repeated `EastRef`/`EastDict` updates — pays an FFI crossing per
911
+ element. Cross to numpy/torch only for work East cannot express, and wrap the
912
+ result back. Bare scalars stay plain python — an East `Float` *is* a `float`;
913
+ don't wrap a running sum in an `EastRef`.
914
+ - **Chain, don't stage.** Each method returns a live east-c value: keep piping
915
+ (`arr.filter(...).to_dict(...).map(...)`) rather than binding intermediates to
916
+ python names and re-wrapping them. Cross to python only at the edges — a
917
+ scalar for `East.Float.*` math, a buffer via `to_numpy()`/`to_torch()`.
918
+ - **Scalars and dates** — see [Scalars](#scalars-use-the-easttype-utilities-for-consistency--above-all-string--datetime):
919
+ the `East.<Type>` utilities for anything whose semantics diverge, and ALWAYS
920
+ `East.DateTime.*` for dates.
921
+
922
+ | Instead of (pure Python) | Write (East values) |
923
+ |---|---|
924
+ | `[dict(r) for r in items]` then Python loops | keep `items` as the `EastArray`; `.map`/`.filter`/`.group_by`/`.reduce` |
925
+ | `sorted(items, key=…)` / `sorted(list(arr))` | `items.sort(lambda b, r: …)` (East order, in C) |
926
+ | `{r["k"]: r for r in items}` | `items.to_dict(lambda b, r: r["k"])` |
927
+ | `set(a) & set(b)` / `set(a) - set(b)` | `a.intersection(b)` / `a.difference(b)` |
928
+ | `EastArray(T, [f(x) for x in arr])` | `arr.map(f)` (pin `out=` for a widening map) |
929
+ | a `for` loop that sums/accumulates | `arr.reduce(lambda b, acc, x: …, init)` / `arr.map_reduce(…)` |
930
+ | a helper that takes/returns `list`/`dict`/`set` | a helper over East values, or inline the eager chain |
931
+
932
+ ```python
933
+ # WRONG — decodes the whole array to Python, loops in the interpreter,
934
+ # re-encodes, and uses Python's ordering (wrong for floats/NaN, non-deterministic)
935
+ def totals_by_region(items):
936
+ acc = {}
937
+ for r in [dict(x) for x in items]:
938
+ acc[r["region"]] = acc.get(r["region"], 0.0) + r["amount"]
939
+ return array(Row, [{"region": k, "total": v} for k, v in sorted(acc.items())])
940
+
941
+ # CORRECT — stays C-side, East-ordered, no round trip
942
+ def totals_by_region(items):
943
+ return (items
944
+ .group_by(lambda b, r: r["region"]) # Dict<region, Array<row>>
945
+ .map(lambda b, rows: rows.reduce(lambda b, acc, r: acc + r["amount"], 0.0)) # Dict<region, total>
946
+ .to_array(lambda b, total, region: struct({"region": region, "total": total}, Row)))
947
+ ```
948
+
949
+ ### Eager callbacks are East function bodies
950
+
951
+ An eager method and its expression twin are ONE builtin with two entry points:
952
+ `xs.map(f)` on an `EastArray` runs `ArrayMap` in east-c now; on an
953
+ `ArrayExpression` it records the `ArrayMap` node. The callback is a body either
954
+ way, and an eager callback slot takes exactly two kinds of function:
955
+
956
+ 1. **A python body, captured automatically** — the block first, then the
957
+ builtin's callback arguments (`lambda b, el: …`; trailing arguments may be
958
+ omitted, the block cannot), built as an `East.function` against the builtin's
959
+ declared signature. The whole expression surface is available; east-c
960
+ compiles it, and the loop AND the body run natively, zero python per element.
961
+ Every transform is pure, so a `record → legs → values` descent, or a
962
+ `group_by` + `to_dict(combine=)` + `sort` aggregate, is ONE compiled function
963
+ with no materialised intermediate:
964
+
965
+ ```python
966
+ rows = EastBlob(csv_bytes).decode_csv(Row) # C-backed Array<Row>
967
+ amounts = rows.map(lambda b, r: r.price * r.qty) # a body -> native
968
+ hot = rows.filter(lambda b, r: (r.sku == "A-1") & (r.price > 100.0))
969
+ total = rows.reduce(lambda b, acc, r: acc + r.price, 0.0) # multi-param bodies too
970
+ by_sku = rows.group_by(lambda b, r: r.sku) # fully native grouping
971
+ top = rows.sort(lambda b, r: -r.price)
972
+ spend = rows.to_dict(key=lambda b, r: r.sku,
973
+ value=lambda b, r: r.price * r.qty,
974
+ combine=lambda b, x, y: x + y)
975
+ ```
976
+
977
+ 2. **A precompiled East function** — `East.function(...)`, a `.bind(...)`
978
+ result, or one compiled elsewhere (`compile_from_beast2/json/east`, or
979
+ `compile_from_value` for IR built with `east.ir.builders`). Its native
980
+ function value passes **straight through** every eager method — the loop runs
981
+ entirely in east-c, and the output type comes from its own signature (no
982
+ `out=`, no sampling; an `out=` or declared type that contradicts it raises
983
+ `EastTypeError` at the call). An artifact is **dual-mode**: on values it runs
984
+ natively, and inside another body it is a call of its inline Function node
985
+ (`East.function([Row], FloatType, lambda b, r: amount(r) * 1.1)`), whose
986
+ field reads still narrow a column projection.
987
+ A compiled value (a `.bind` result) cannot re-run a body, so a body that CALLS
988
+ one lowers to an IR `Call`: the callee rides as a hidden bound parameter, and
989
+ loop, body and callee all run in east-c. `FunctionType` PARAMETERS are
990
+ first-class — callable in the body, bindable with function values (calling an
991
+ `AsyncFunctionType` value in a sync body is a named `ExpressionError`):
992
+
993
+ ```python
994
+ amount = East.function([Row], FloatType, lambda b, r: r.price * r.qty)
995
+ for blob in batches:
996
+ out = blob.decode_csv(Row).map(amount) # reuse — no rebuild
997
+ # (a VALUE takes no block)
998
+ step = East.function([FloatType, Row], FloatType, # fold arity
999
+ lambda b, acc, r: acc + r.price)
1000
+ k = compile_from_beast2(bytes_) # compiled in TS
1001
+
1002
+ conv = East.function([Row, TableT], FloatType,
1003
+ lambda b, r, t: t.get_or_default(r.sku, 0.0))
1004
+ rows.map(conv.bind(table)) # bind a live side-table BY REFERENCE
1005
+ ```
1006
+
1007
+ Anything else in a callback slot RAISES — there is no third kind. The
1008
+ callback-free operations (`sort`, `unique`, `union`/`intersection`/`difference`,
1009
+ `concat`, `group_by`, `to_dict`/`to_set`, `find_sorted_*`) always run the whole
1010
+ loop in east-c. The TypeScript compliance corpus is replayed through these eager
1011
+ methods, builtin by builtin, and must give the compiled programs' answers.
1012
+
1013
+ ### Construction & validation (`from east import ...`)
1014
+
1015
+ | Signature | Description | Example |
1016
+ |-----------|-------------|---------|
1017
+ | **Ergonomic constructors** |
1018
+ | `array(element_type, items, *, validate=True) -> EastArray` | Each item coerced/validated to `element_type` (dict→struct, int→Float, …); `validate=False` stores as-is | `array(LineItem, [{"name":"a","price":1}])` |
1019
+ | `struct(fields: dict, typ: StructType\|None=None) -> EastStruct` | Reorders/coerces keys to `typ` (else infers from fields); dual-mode — a field holding an expression (at any depth) builds Struct IR instead | `struct({"price":1,"name":"a"}, LineItem)` |
1020
+ | `variant(case: str, value, typ: VariantType\|None=None) -> EastVariant` | Tagged value; validates `value` against case `case` (matched **by name**); read back via `.type`/`.value`; dual-mode — an expression payload builds Variant IR, typed by `typ` or by context | `variant("named", "red", Color)` |
1021
+ | `some(value) -> EastVariant` / `none` | Option `some`; `none` is a **constant**, not a function | `some(5)` / `none` |
1022
+ | `match(v, cases: dict, default=None)` | Dispatch on `v.type`; the handler is a body, **always** called `handler(b, v.value)` — the `none` arm is `lambda b, v: …`, not `lambda: …`; `default` is a `default(b)` body for the cases without a handler (a plain value is returned as is) | `match(o, {"some": lambda b, x: x, "none": lambda b, v: -1})` |
1023
+ | `east_ref(value) -> EastRef` | Make a mutable ref cell (same as `EastRef(value)`) | `east_ref(0)` |
1024
+ | **Validation / coercion** — raise `EastTypeError` (`expected X, got Y (at $.path)`) |
1025
+ | `coerce_to(value, typ, *, path="$") -> EastValue` | Canonicalize any Python value to a bridge-ready East value, type-driven; a 1-D numpy array fills `Array<Float/Integer/Boolean>` C-side in one bulk crossing (hand struct fields numpy columns directly) | `coerce_to({"qty": np_i64}, InputType)` |
1026
+ | `assert_value_of(value, typ, *, path="$") -> value` ❗ | Validate; return value, or raise path-pinpointed `EastTypeError` on first mismatch | `assert_value_of(s, LineItem)` |
1027
+ | `explain_value_of(value, typ) -> list[(path, reason)]` | Every mismatch; `[]` == conforms | `explain_value_of(s, LineItem)` |
1028
+ | `is_value_of(value, typ) -> bool` | Boolean conformance check | `is_value_of(items, ArrayType(LineItem))` |
1029
+ | `type_of(value) -> EastType` | Infer the East type of a value | `type_of(items)` |
1030
+
1031
+ Container constructors are also direct: `EastArray(elem, items=None)`, `EastSet(elem, items=None)`,
1032
+ `EastDict(key, value, items=None)`, `EastVector(elem, data=None, length=0)`,
1033
+ `EastMatrix(elem, data=None, rows=0, cols=0)`, `EastRef(value)`.
1034
+
1035
+ ### A JSON contract for a type — `json_schema_for` / `type_from_json_schema`
1036
+
1037
+ `East.String.print_json` writes **East JSON**, lossless and self-describing.
1038
+ `json_schema_for(T)` emits the JSON Schema of exactly that encoding, so another
1039
+ system can validate a payload before sending it; `type_from_json_schema` goes the
1040
+ other way. Host-side functions, like `compare_for` — no expression, no IR:
1041
+
1042
+ ```python
1043
+ import json
1044
+ from pathlib import Path
1045
+
1046
+ from east import ArrayType, IntegerType, StringType, StructType
1047
+ from east.serialization import json_schema_for, type_from_json_schema
1048
+
1049
+ Reading = StructType([("sensor", StringType), ("litres", IntegerType)])
1050
+ Path("contract.schema.json").write_text(
1051
+ json.dumps(json_schema_for(ArrayType(Reading), draft="draft-07"), indent=2))
1052
+
1053
+ T = type_from_json_schema(json.loads(Path("partner.schema.json").read_text()))
1054
+ ```
1055
+
1056
+ | Signature | Notes |
1057
+ |-----------|-------|
1058
+ | `json_schema_for(typ, draft="2020-12") -> JsonSchema` | `draft`: `"2020-12"`, `"draft-07"` or `"openapi-3.0"`, as a consumer's validator pins. `Never`, `Function` and `AsyncFunction` raise `TypeError` naming what has no JSON form |
1059
+ | `type_from_json_schema(schema) -> EastType` ❗ | Raises `JsonSchemaUnsupportedError` — its `.pointer` the RFC 6901 location, also in the message — on what East cannot express: `allOf`, `not`, `if`/`then`/`else`, `anyOf`, `patternProperties`, `prefixItems`, a union of more than one type, an open record, an optional property, an untagged `oneOf`, a non-local `$ref`, a cycle of definitions no single definition breaks |
1060
+ | `EAST_JSON_PATTERNS` | The lexical forms of East JSON's scalars — `.integer` `.blob` `.float_specials` (TypeScript: `floatSpecials`) — so a reader enforces exactly what the schema describes. A DateTime has none: its form is `format: "date-time"` |
1061
+
1062
+ - **An `Option<T>` is `null` or `T`'s own encoding** wherever `T` can never
1063
+ encode as `null`: `none` → `null`, `some(7)` → `"7"`, a struct field
1064
+ `"note": null` or `"note": "x"` (an absent key is still an error). Only
1065
+ `Option<Null>` and `Option<Option<T>>` keep the tagged `{"type": …, "value":
1066
+ …}` object, which keeps `some(none)` apart from `none`; a recursive payload is
1067
+ judged by what it wraps. The rule is a total function of the type, applied by
1068
+ every codec and reader and by `json_schema_for` (a flat Option is `oneOf
1069
+ [null, T]` annotated `x-east-type: "Option"`), and the tagged object under a
1070
+ flat Option is refused by the payload's own decoder (`expected string, got
1071
+ {"type":"none","value":null}`).
1072
+ - **A `DateTime` is `format: "date-time"`**: every decoder, on every runtime,
1073
+ reads any RFC 3339 date-time — `Z` or any offset (the instant is UTC), `t`/`z`
1074
+ in either case, any fractional digits (past the millisecond dropped, never
1075
+ rounded, so `isoformat()`'s microseconds read to the millisecond), a leap
1076
+ second as the Unix time its fields add up to — in years 0001–9999, the range
1077
+ every runtime holds. `print_json` writes UTC, three fractional digits, `+00:00`.
1078
+ - **Other scalars are described as the encoder emits them**: an `Integer` is a
1079
+ quoted i64 decimal, a `Blob` lowercase hex (`parse_json` also takes uppercase,
1080
+ which the contract does not), so a producer that validates cannot send what a
1081
+ strict reader rejects. Every pattern spells digits `[0-9]` — python's `\d`
1082
+ matches any Unicode digit — so validators here and in JavaScript agree.
1083
+ - **The document is deterministic and byte-identical to TypeScript's
1084
+ `jsonSchemaFor`** (key order, `$defs` names in first-encounter order, variant
1085
+ case order). Its `x-east-type` annotations make the inverse exact — `Set` vs
1086
+ `Array`, `Dict` vs an array of pairs, a flat Option vs any other `oneOf`. A
1087
+ foreign schema without them converts under a structural mapping that does not
1088
+ promise to round-trip: `format: "date-time"` reads as a `DateTime` (any other
1089
+ `format` is a `String`), and `nullable: true`, `["string", "null"]` and a
1090
+ `oneOf` of null and one schema read as an Option.
1091
+ - **Recursion binds one `recursive_type` per cycle group**: definitions that
1092
+ reference each other convert when every cycle passes through one definition,
1093
+ the binder; three that each reference the other two need two binders and are
1094
+ refused, naming them.
1095
+ - To READ a document larger than memory under this contract: `json_open` /
1096
+ `json_next` in **east-py-std**.
1097
+
1098
+ ### EastArray — complete method surface
1099
+
1100
+ Eager; results are live east-c-backed values that chain. `arr[i]`, `len(arr)`, `for x in arr`
1101
+ work via the sequence protocol. Callback methods accept a python body — the block first, then
1102
+ the arguments the table lists, so `fn(el)` is written `lambda b, el: …` — (built into a native
1103
+ function) or a precompiled `East.function(...)` (a value: it takes no block); anything East cannot
1104
+ express raises with the binding named. Result types come from the build, so `out`/`element_type` is optional — pass it
1105
+ to PIN a type (a widening map, an Option payload) and a contradicting precompiled function raises
1106
+ at the call. `.element_type` is the logical element type.
1107
+
1108
+ | Group | Methods |
1109
+ |-------|----------------------|
1110
+ | Access | `get(i[, default_fn(i)])` (errors when out of bounds — `arr[i]` is the pythonic read; TS `get(index, onMissing?)`) · `at(i)` · `get_or_default(i, default)` · `try_get(i) -> some/none` · `has(i)` · `size()`/`length()` · `get_keys(indices: EastArray)` |
1111
+ | Reorder (new array) | `sort(by=None, *, reverse=False)` (TS `sort`; `sorted` is the deprecated spelling) · `reverse()` (`reversed` deprecated) |
1112
+ | Slice & combine | `slice(start, end)` · `concat(other)` · `copy()` |
1113
+ | Per-element | `map(fn(el), out=None)` · `filter(pred(el))` · `filter_map(fn(el)->some/none, out=None)` · `for_each(fn(el)) -> None` — every element callback may also declare `(el, idx)` to receive the builtin's index |
1114
+ | Reduce | `reduce(fn(acc, el), init)` (TS order; `fold(init, fn)` is the deprecated spelling) · `scan(fn(acc, el), init) -> Array` (running fold: one accumulator per element, seed not emitted, `scan(...)[n-1] == reduce(...)`) · `map_reduce(map_fn(el), reduce_fn(acc, m), out=None)` · `sum(fn=None)` · `mean(fn=None) -> float` (NaN when empty) · `maximum(by=None)` ❗empty · `minimum(by=None)` ❗empty (the ELEMENT — of `by(el)` when given — a tie keeps the first, like TS) · `every(pred=None) -> bool` · `some(pred=None) -> bool` (native short-circuit) |
1115
+ | Group & index | `group_by(key(el)) -> Dict` · `group_reduce(key, init(gk), fold(acc, el)) -> Dict` · `group_size(key=None)` · `group_sum(key, fn=None)` · `group_mean(key, fn=None)` · `group_maximum/group_minimum(key, by=None)` (Dict of the ELEMENT per group, like TS — `group_find_maximum` is the index finder) · `group_every/group_some(key, pred)` · `group_find_all(key, value, by=None) -> Dict<K, Array<Integer>>` / `group_find_first(key, value, by=None) -> Dict<K, some/none>` (GLOBAL row indices; every group appears, so a group with no match maps to `[]` / `none`) · `group_find_maximum/group_find_minimum(key, by=None) -> Dict<K, Integer>` (the INDEX per group; a tie keeps the earliest) · `group_to_arrays(key, value=None)` · `group_to_sets(key, value=None)` · `group_to_dicts(key, key2, value=None, combine=None)` · `to_dict(key(el), value=None, combine=None) -> Dict` (duplicate key errors without `combine`) · `to_set(key=None) -> Set` · `unique() -> Set` |
1116
+ | Search | `find_first(target, key=None) -> some/none` · `find_all(value, by=None) -> Array<Integer>` · `find_maximum/find_minimum(by=None) -> some(index)/none` · `find_sorted_first/last(target, key=None) -> int` · `find_sorted_range(target, key=None) -> {start,end}` · `first_map(fn(el)->some/none, out=None)` · `is_sorted(key=None) -> bool` |
1117
+ | Flatten | `flat_map(fn(el)->arr, out=None)` (`flatten_to_array` deprecated) · `flatten_to_set(fn(el)->set, out=None)` · `flatten_to_dict(fn(el)->dict, combine=None)` (a duplicate key errors without `combine`) |
1118
+ | Columnar | `to_columns(fields=None) -> dict` (numpy per numeric/bool column, `Option<Float>`→NaN, interned strings) · `EastArray.from_columns(element_type, columns)` *(static)* (C-side fill needs numpy columns — float64/int64/bool, `Option<Float>` as float64+NaN; python lists convert per cell) · `map_batches(fn(cols)->cols, out=None, batch_size=100_000)` |
1119
+ | Convert | `to_tree(node, key(el[, i]), parent(el[, i])->some/none, build(el, i, children)) -> Array<node>` (flat parent-keyed rows → nested nodes of any depth, iterative — [Trees from flat rows](#trees-from-flat-rows-and-recursion)) · `string_join(sep) -> str` (String arrays) · `to_vector() -> EastVector` (Float/Integer/Boolean elements, in order — east-c VectorFromArray; the expression method emits the same builtin) · `encode_csv(config=None, **options) -> EastBlob` (Array of structs; east-c ArrayEncodeCsv) |
1120
+ | Mutate (in place, the TS names) | `push_last(item)` · `push_first(item)` · `append(array)` (ArrayAppend — a whole array, NOT one element) · `prepend(array)` · `pop_last()` · `pop_first()` · `update(i, item)` ❗bounds · `merge(i, value, update_fn(existing, incoming[, i]))` · `merge_all(array, merge_fn(existing, incoming[, i]))` · `clear()` · `sort_in_place(by=None)` · `reverse_in_place()` — plus the python protocol: `extend(iterable)` (bulk: one crossing; C-to-C for same-type East arrays, raw buffers for numpy) · `insert(i, item)` · `pop(i=-1)` · `remove(item)` · `count(value) -> int` · `index(value) -> int` |
1121
+
1122
+ ### EastSet — complete method surface
1123
+
1124
+ Mutable, unique, **East-sorted**. `.element_type` is the element type; iteration / `to_array` /
1125
+ `reduce` visit in East order. Every callback takes the block first (`lambda b, el: …`); the
1126
+ signatures below list the arguments after it. **`map` and `filter_map` return an `EastDict`** keyed by the element
1127
+ (Set→Set is `to_set`).
1128
+
1129
+ | Group | Methods |
1130
+ |-------|---------|
1131
+ | Access | `len(s)` · `value in s` · `has(value)` · `for el in s` |
1132
+ | Algebra (vs another set) | `union(other)` · `intersection(other)` · `difference(other)` · `symmetric_difference(other)` · `is_subset_of(other) -> bool` · `is_superset_of(other) -> bool` · `is_disjoint_from(other) -> bool` (the TS names; `intersect`/`diff`/`sym_diff`/`is_subset`/`is_disjoint` are deprecated spellings) |
1133
+ | Per-element | `map(fn(el)) -> Dict` · `filter(pred(el))` · `filter_map(fn(el)->some/none, out=None) -> Dict` · `first_map(fn(el)->some/none, out=None)` · `to_set(fn(el), out=None)` · `to_array(key=None)` · `to_dict(key(el), value(el), combine=None)` (duplicate key errors without `combine`) · `to_tree(node, parent(el)->some/none, build(el, children)) -> Array<node>` (each element its own key) · `for_each(fn(el)) -> None` |
1134
+ | Reduce | `reduce(fn(acc, el), init)` (TS order) · `scan(fn(acc, el), init) -> Array` (running fold in East order) · `map_reduce(fn(el), reduce(a,b))` (raises on empty) · `sum(fn=None)` · `mean(fn=None) -> float` · `every(pred=None)` · `some(pred=None)` (native short-circuit) |
1135
+ | Group | `group_reduce(key(el), initial(gk), fold(acc, el)) -> Dict` · `group_size(key)` · `group_sum(key, fn=None)` · `group_mean(key, fn=None)` · `group_every/group_some(key, pred)` · `group_to_arrays/group_to_sets(key, value=None)` · `group_to_dicts(key, key2, value=None, combine=None)` · ⚠️ `group_fold(...)` is the DEPRECATED alias of `group_reduce` |
1136
+ | Flatten | `flatten_to_array(fn(el)->arr, out=…)` (the TS Set name; `flat_map` is Array's and a deprecated spelling here) · `flatten_to_set(fn(el)->set, out=…)` · `flatten_to_dict(fn(el)->dict, combine=None)` (duplicate key errors without `combine`) |
1137
+ | Mutate (in place) | `add(item)` · `insert(value)` (errors if present) · `try_insert(value) -> bool` (True if newly added) · `remove(item)` · `delete(value)` (errors if absent) · `try_delete(value) -> bool` · `discard(item)` · `union_in_place(other)` (adds all of `other`) · `clear()` · `copy()` |
1138
+
1139
+ ### EastDict — complete method surface
1140
+
1141
+ Mutable, **East-sorted by key**. `.key_type` / `.value_type`. **Every Dict callback takes
1142
+ the builtin's own `(value, key)` order — the TypeScript order** (the block first, as always):
1143
+ `fn(b, value)` when the key is not needed, `fn(b, value, key)` when it is — for `map`,
1144
+ `filter`, `first_map`, `to_*`, `flatten_to_*`, `map_reduce`, the projections of
1145
+ `every`/`some`/`sum`/`mean` and `group_*`'s key/value projections alike; a fold step
1146
+ (`reduce`/`scan`/`group_reduce`) is `fn(b, acc, value[, key])`; collision `combine` is
1147
+ `combine(existing, incoming, key)` — for `union`/`union_in_place`/`to_dict` a 2-arg
1148
+ `combine(existing, incoming)` is also accepted (a 3-arg one still receives the key).
1149
+ `union`/`union_in_place` require `other` to have the same key AND value type (a
1150
+ mismatch is refused); `merge_all` needs only the same KEYS and is generic
1151
+ in `other`'s values; `merge` takes a single differently-typed value.
1152
+
1153
+ | Group | Methods |
1154
+ |-------|---------|
1155
+ | Access | `d[k]` · `k in d` · `has(k)` · `len(d)`/`size()` · `get(k)` ❗missing · `get(k, default)` (a value) / `get(k, fn(k))` (TS `onMissing`) · `get_or_default(k, default)` · `try_get(k) -> some/none` · `keys() -> Set` (TS; east-c DictKeys — `keys_set` deprecated) · `values()`/`items()` (python views) |
1156
+ | Combine | `union(other, combine=None) -> Dict` (NEW dict, both inputs untouched; a shared key errors without `combine`) · `union_in_place(other, combine=None) -> None` · `merge_all(other, merge(existing, incoming[, key]), default(key)) -> None` (fold `other` into self in place; `default` seeds absent keys; **generic in `other`'s value type** — only the KEYS must match) · `merge(key, value, update_fn(existing, incoming[, key]), initial_fn=None) -> None` (ONE key, in place; `value` may be a different type — TS `merge`; `merge_key` deprecated) · `get_keys(keys: Set, fill(k)) -> Dict` |
1157
+ | Per-entry | `map(fn(value[, key]), out=None)` · `filter(pred(value[, key]))` · `filter_map(fn(value[, key])->some/none, out=None)` · `first_map(fn(value[, key])->some/none, out=None)` · `for_each(fn(value[, key])) -> None` |
1158
+ | Reduce | `reduce(fn(acc, value[, key]), init)` (TS order) · `scan(fn(acc, value[, key]), init) -> Array` (running fold in key order) · `map_reduce(map_fn(value[, key]), reduce_fn(a, b), out=None)` (raises on empty) · `sum(fn(value[, key])=None)` · `mean(fn(value[, key])=None) -> float` · `every(pred(value[, key])=None) -> bool` · `some(pred(value[, key])=None) -> bool` (native short-circuit) |
1159
+ | Group | `group_reduce(key_fn(value[, key]), init_fn(gk), fold_fn(acc, value[, key]), key_out=None, acc_out=None) -> Dict` · `group_size(key_fn)` · `group_sum(key_fn, fn=None)` · `group_mean(key_fn, fn=None)` · `group_every/group_some(key_fn, pred(value[, key]))` · `group_to_arrays/group_to_sets(key_fn, value_fn=None)` · `group_to_dicts(key_fn, key2_fn, value_fn=None, combine=None)` · ⚠️ `group_fold(...)` is the DEPRECATED alias of `group_reduce` |
1160
+ | Flatten | `flatten_to_array(fn(value[, key])->arr, out=None)` (the TS Dict name; `flat_map` is Array's and a deprecated spelling here) · `flatten_to_set(fn(value[, key])->set, out=None)` · `flatten_to_dict(fn(value[, key])->dict, combine=None)` (a duplicate key errors without `combine`) |
1161
+ | Convert | `keys() -> Set` · `to_array(fn(value[, key]), out=None)` · `to_set(fn(value[, key]), out=None)` · `to_dict(key_fn, value_fn=None, combine=None, key_out=None, value_out=None)` (the value itself when `value_fn` is omitted; a duplicate key errors without `combine`) · `to_tree(node, parent(value[, key])->some/none, build(value, key, children)) -> Array<node>` (each entry's key its key) · `copy()` |
1162
+ | Mutate (in place) | `d[k]=v` · `del d[k]` · `insert(k, v)` (errors if present) · `get_or_insert(k, fn(k))` · `insert_or_update(k, v, combine(existing, incoming, k))` · `update(k, v)` (TS; the read-modify-write `update(k, fn(current))` is deprecated) · `swap(k, v) -> prev` · `delete(k)` · `try_delete(k) -> bool` (`update`/`swap`/`delete` error on a missing key) · `pop(k, *default)` · `clear()` |
1163
+ | Bulk (in place) | `update_many(keys, values, combine(existing, incoming)=None)` — the whole batch crosses once; a pure/precompiled `combine` resolves collisions C-to-C (dicts as hot-loop accumulators) |
1164
+
1165
+ ### EastVector — complete method surface
1166
+
1167
+ Immutable 1-D numeric value: logical element type `Float`/`Integer`/`Boolean`,
1168
+ backed by a contiguous NumPy buffer for zero-copy ML interop; `.element_type` is
1169
+ fixed, while the storage `.dtype` may be any compatible width (f32, …). The
1170
+ **arithmetic surface delegates to the east-c builtins** — reductions fold in
1171
+ strict left-to-right index order and comparisons use East's total order (NaN
1172
+ greatest, `-0.0 < 0.0`) — bit-identical across the TypeScript, C and python
1173
+ runtimes, which numpy's reassociating reductions are not; free-form math goes via
1174
+ `to_numpy()`/`to_torch()`. `set` and every transform return a NEW vector. Not
1175
+ hashable, but a valid East Set/Dict key (ordered by value). Construct with the
1176
+ `EastVector.*` classmethods ([Container generators](#container-generators-classmethods));
1177
+ `from_numpy`/`from_torch` infer `element_type` from the dtype when omitted.
1178
+ Structural access with an **expression** argument (inside a body) lifts the
1179
+ vector as a constant and emits IR, like the eager collections.
1180
+
1181
+ | Group | Methods |
1182
+ |-------|---------|
1183
+ | Access | `get(i)` ❗bounds (promoted Python scalar) · `length() -> int` · `slice(start, end) -> EastVector` (half-open, contiguous copy) |
1184
+ | Transform (returns new) | `set(i, v) -> EastVector` ❗bounds (original unchanged) · `concat(other) -> EastVector` (takes this vector's element type) |
1185
+ | Arithmetic (east-c; Float/Integer elements) | `scale(alpha)` · `add_scaled(other, alpha)` (`self + alpha*other`) ❗len · `mul(other)` ❗len · `add_scalar(c)` · `abs()` · `clamp(lo, hi)` (East order) · `cum_sum()` (running sum, left to right) |
1186
+ | Reduce (east-c; strict left-to-right order) | `sum()` (0 when empty) · `dot(other)` ❗len · `max()`/`min()` ❗empty (TS names; `maximum`/`minimum` deprecated) · `arg_max()`/`arg_min()` ❗empty (ties keep the first index; NaN is greatest) · `mean() -> float` (Integer widens per element; NaN when empty) |
1187
+ | Masks & selection (east-c) | `eq(other)`/`lt(other)`/`gt(other)` ❗len `-> Vector<Boolean>` (East equality/order) · `mask.select(a, b)` ❗len (mask receiver) · `v.compress(mask)` ❗len · `mask.count_true() -> int` |
1188
+ | Gather / scatter / search (east-c) | `gather(indices)` ❗bounds · `scatter_add(indices, src)` ❗len/bounds (duplicates accumulate in order) · `search_sorted(needles) -> Vector<Integer>` (leftmost insertion index; assumes sorted) |
1189
+ | Sparse accumulators (east-c; `East.Vector.*`) | `sparse_axpy(ixA, vA, ixB, vB, alpha)` (union merge `vA + alpha*vB`; absent entries stay absent) ❗ascending/len · `sparse_from_pairs(ix, v)` (sorts + sums duplicates stably, in input order) ❗len · `sparse_filter_gt(ix, v, threshold)` ❗ascending/len — all return `Struct{ix: Vector<Integer>, v: Vector<T>}` with strictly ascending `ix`; every `ix`/`v` input takes a Vector OR an Array |
1190
+ | Per-element (east-c; the native callback builtins) | `map(fn(el[, i]), out=None) -> EastVector` (VectorMap; Float/Integer/Boolean results) · `reduce(fn(acc, el[, i]), init)` (VectorFold, index order; `fold(init, fn)` is the deprecated spelling) — a body like every other callback, zero python per element |
1191
+ | Convert | `to_array() -> EastArray` (promotes scalars; severs the zero-copy link) · `to_matrix(rows, cols) -> EastMatrix` (row-major reshape; `rows*cols == length`) |
1192
+ | NumPy / torch | `to_numpy(dtype=None, copy=False) -> ndarray` (read-only view by default; a cast or `copy=True` is writeable) · `to_torch(dtype=None) -> torch.Tensor` (always a writeable copy) · `np.asarray(v)` via `__array__` · props `.dtype` (storage) / `.element_type` (logical) |
1193
+
1194
+ ### EastMatrix — complete method surface
1195
+
1196
+ Immutable 2-D row-major numeric value on a contiguous NumPy buffer, with the
1197
+ `EastVector` contract: logical `.element_type` apart from storage `.dtype`, the
1198
+ arithmetic surface in east-c with the strict reduction order, free-form math via
1199
+ `to_numpy()`/`to_torch()`, NEW matrices from `set` and every transform, not
1200
+ hashable but a valid Set/Dict key, the `EastMatrix.*` classmethods to construct,
1201
+ and an expression argument lifting the matrix as a constant.
1202
+
1203
+ | Group | Methods |
1204
+ |-------|---------|
1205
+ | Access | `get(r, c)` ❗bounds (Python scalar) · `rows() -> int` · `cols() -> int` (TS names; `num_rows`/`num_cols` deprecated) |
1206
+ | Transform (returns new) | `set(r, c, v) -> EastMatrix` ❗bounds (original unchanged) · `transpose() -> EastMatrix` (new cols×rows, contiguous) |
1207
+ | Arithmetic (east-c; Float/Integer elements) | `scale(alpha)` · `add_scaled(other, alpha)` ❗dims · `mul_elementwise(other)` ❗dims (Hadamard) |
1208
+ | Reduce (east-c; ascending index order) | `row_sums() -> EastVector` (ascending column order per row) · `col_sums() -> EastVector` (ascending row order per column) · `vec_mul(v) -> EastVector` ❗cols≠len (row-by-vector dot products) |
1209
+ | Rows & cols | `get_row(r) -> EastVector` ❗bounds (contiguous copy) · `get_col(c) -> EastVector` ❗bounds (contiguous copy) |
1210
+ | Per-row (east-c) / per-element (DEPRECATED) | `map_rows(fn(row: EastVector[, i]) -> EastVector, out=None)` (the native MatrixMapRows — a body like every callback) · `map_elements(fn(el), out=None)` runs a python callback per element; it warns and will go — use the arithmetic surface or `to_numpy()` |
1211
+ | Convert | `to_vector() -> EastVector` (row-major flatten) · `to_array() -> EastArray` (`Array<Array<el>>`, one inner array per row) · `to_rows() -> EastArray` (`Array<Vector<el>>`, one `EastVector` per row) |
1212
+ | NumPy / torch | `to_numpy(dtype=None, copy=False) -> ndarray` (2-D; read-only view by default) · `to_torch(dtype=None) -> torch.Tensor` (writeable copy) · `np.asarray(m)` via `__array__` · props `.dtype` / `.element_type` |
1213
+
1214
+ ### EastBlob (a `bytes` subclass)
1215
+
1216
+ `EastBlob(b"...")` constructs like `bytes` and carries the **full `bytes` API** plus East methods:
1217
+
1218
+ | Signature | Description |
1219
+ |-----------|-------------|
1220
+ | `size() -> int` | Byte length (== `len(blob)`) |
1221
+ | `get_uint8(i) -> int` | Unsigned byte at `i` (0–255) |
1222
+ | `.data -> bytes` | Raw payload |
1223
+ | `decode_utf8() -> str` / `decode_utf16() -> str` | Text decode |
1224
+ | `EastBlob.encode_beast2(value) -> EastBlob` *(static)* | Serialize an East value to BEAST2 (type inferred via `type_of`); `East.Blob.encode_beast(value, "v2")` is the builtin twin |
1225
+ | `decode_beast(typ, version="v1") -> value` | The east-c BlobDecodeBeast / BlobDecodeBeast2 builtins (TS `decodeBeast`): `"v1"` is the original BEAST format, `"v2"` the beast2 family (any container version) |
1226
+ | `decode_beast2(typ) -> value` | Decode BEAST2 as `typ` (the serialization-layer form) |
1227
+ | `open_beast(typ) -> value` | The east-c BlobOpenBeast2 builtin (TS `openBeast`): an indexed beast2 `Array`/`Set`/`Dict` blob as a FROZEN paged proxy — the value a task's collection input opens as — whose `len`, keyed reads and iteration decode one segment; an index-less blob (what `encode_beast` writes) or a Ref-/function-bearing element shape decodes whole, frozen; mutation raises, `.copy()` gives a mutable value; a header of another type raises `cannot open a blob of type <wire> as <typ>` before any decode |
1228
+ | `decode_csv(element_type, config=None, **options) -> EastArray` | Decode CSV rows into `Array<element_type>` (east-c decoder). Build `config` with `east.serialization.csv.csv_parse_config(...)`: by default **no field text is null** (empty field == empty string); opt in with `null_strings=[""]` (`none` for Option columns, error for required); `defaults={"qty": "0.0"}` gives per-column fallbacks for unparseable fields and constant-fill for absent columns; `skip_short_rows=True` drops ragged rows instead of erroring |
1229
+
1230
+ ### EastStruct / EastVariant / EastRef
1231
+
1232
+ - **`EastStruct`** — frozen record; read fields by name: `s["price"]` or as an
1233
+ attribute, `s.price` (methods shadow same-named fields — item access always
1234
+ works). Build/transform with `struct({...}, StructType)`.
1235
+ - **`EastVariant`** — frozen tagged value; `.type` is the case name, `.value` the payload.
1236
+ Build with `variant(case, value, T)` / `some` / `none`. Dispatch with the
1237
+ `.match({case: handler}, default=None)` method (handlers are bodies, `handler(b, payload)`;
1238
+ `default` is a `default(b)` body for the other cases — TS's partial match — or a plain
1239
+ value; the module-level `match(v, cases, default)` is equivalent) or
1240
+ `.match_tag(tag, handler, default)`. Also `get_tag()`, `has_tag(tag)`, and
1241
+ `unwrap(tag="some", on_other=None)` ❗ValueError on a different case unless the
1242
+ `on_other(b)` body answers — mirroring the TS variant expression surface, and the same
1243
+ shapes build inside a body. An option reads with the **same accessors as the
1244
+ expression surface** — `is_some()`, `is_none()`, `unwrap_or(default)` — so a
1245
+ decoded option needs no `opt.type == "some"` branch.
1246
+ - **`EastRef`** — mutable cell: `get()` · `update(value)` (TS `RefUpdate`; `set` is the
1247
+ deprecated spelling, and the read-modify-write `update(fn(b, current))` is deprecated too —
1248
+ write `ref.update(f(ref.get()))`) · `merge(patch, combine(b, current, patch))`. Inside a
1249
+ body the expression twin is `East.ref(v)` with the same `get`/`update`/`merge`.
1250
+
1251
+ ### Container generators (classmethods)
1252
+
1253
+ Eager factories are **classmethods** on the container classes (snake_case); their dual-mode
1254
+ twins — usable inside a body — live on the `East.Array`/`East.Set`/`East.Dict`/
1255
+ `East.Vector`/`East.Matrix` namespaces (`East.Array.range`, `East.Vector.zeros(T, n)`,
1256
+ `East.Matrix.from_rows(rows)`, …). The namespace `generate`s take the **TypeScript argument
1257
+ order** — `East.Array.generate(size, T, fn)`, `East.Set.generate(size, T, fn, on_conflict=None)`,
1258
+ `East.Dict.generate(size, K, V, key_fn, value_fn, on_conflict=None)` — and, like TypeScript, a
1259
+ key generated twice is a runtime error `Duplicate key <k> in set/dict` unless an `on_conflict`
1260
+ handler is given (the pre-TS python order still works with a DeprecationWarning). Do not confuse them with
1261
+ `East.new_array`/`new_set`/`new_dict`, which are the CONTROL-FLOW constructors: a fresh
1262
+ loop-local collection built per evaluation inside a function.
1263
+
1264
+ | Signature | Example |
1265
+ |-----------|---------|
1266
+ | `EastArray.range(start, end, step=1)` | `EastArray.range(0, 5, 2)` → `[0, 2, 4]` |
1267
+ | `EastArray.linspace(start, end, count)` | `EastArray.linspace(0., 1., 3)` → `[0.0, 0.5, 1.0]` |
1268
+ | `EastArray.generate(count, fn(b, i), element_type=None)` | `EastArray.generate(3, lambda b, i: i*i, IntegerType)` |
1269
+ | `EastSet.generate(n, fn(b, i), element_type=None, on_conflict=None)` | `EastSet.generate(4, lambda b, i: East.Integer.remainder(i, 2), IntegerType, lambda b, k: None)` — a duplicate ERRORS without `on_conflict` (TS) |
1270
+ | `EastDict.generate(n, key_fn(b, i), value_fn(b, i), combine(b, x, y, k), key_type, value_type)` | `EastDict.generate(3, lambda b, i: i, lambda b, i: i*10, lambda b, x, y, k: x + y, IntegerType, IntegerType)` — `combine=None` makes a duplicate key an error (TS) |
1271
+ | `EastVector.zeros/ones(element_type, length)` · `fill(element_type, length, value)` · `from_array(element_type, items)` · `from_numpy(array, element_type=None)` · `from_torch(tensor, element_type=None)` | `EastVector.zeros(FloatType, 3)` |
1272
+ | `EastMatrix.zeros/ones(element_type, rows, cols)` · `fill(…, value)` · `from_array/from_rows(element_type, rows)` · `from_numpy(array, element_type=None)` · `from_torch(tensor, element_type=None)` | `EastMatrix.from_array(FloatType, [[1.,2.],[3.,4.]])` |
1273
+
1274
+ ### East.<Type> namespaces — the builtins and the standard library
1275
+
1276
+ Scalars are plain Python, so their builtins are namespace functions — **complete** lists below.
1277
+ Every one delegates to east-c, and every one is dual-mode: on plain values it runs now, on
1278
+ expressions it emits IR. The `stdlib:` rows are the TypeScript standard library
1279
+ (see [the stdlib](#the-standard-library--the-typescript-easttype-functions)).
1280
+
1281
+ **`East.Float`** (f64)
1282
+
1283
+ | Signature | Notes |
1284
+ |-----------|-------|
1285
+ | `add(a,b)` · `subtract(a,b)` · `multiply(a,b)` · `divide(a,b)` · `remainder(a,b)` · `pow(base,exp)` | arithmetic |
1286
+ | `negate(x)` · `abs(x)` · `sign(x)` · `sqrt(x)` · `exp(x)` · `log(x)` | unary / powers |
1287
+ | `sin(x)` · `cos(x)` · `tan(x)` | trig |
1288
+ | `to_integer(x) -> int` | raises on a non-integer float (e.g. `3.9`) |
1289
+ | stdlib: `approx_equal(x, y, epsilon) -> bool` · `round_floor/round_ceil/round_half/round_trunc(x) -> int` | `round_half` ties away from zero |
1290
+ | stdlib: `round_nearest/round_up/round_down/round_truncate(x, step)` · `round_to_decimals(x, decimals)` | step 0.0 returns `x`; NaN/±Infinity raise `Cannot round …` |
1291
+ | stdlib: `print_fixed(x, decimals)` · `print_comma_seperated(x, decimals)` · `print_currency(x)` · `print_compact(x)` · `print_percentage(x, decimals)` | `"3.14"` · `"1,234.57"` · `"-$42.50"` · `"1.5M"` · `"12.34%"`; NaN/±Infinity raise `Cannot format …` |
1292
+
1293
+ **`East.Integer`** (i64)
1294
+
1295
+ | Signature | Notes |
1296
+ |-----------|-------|
1297
+ | `add(a,b)` · `subtract(a,b)` · `multiply(a,b)` · `divide(a,b)` · `remainder(a,b)` · `pow(base,exp)` | arithmetic (`divide` truncates) |
1298
+ | `negate(x)` · `abs(x)` · `sign(x)` · `log(x, base)` | unary |
1299
+ | `to_float(x) -> float` | widen to f64 |
1300
+ | stdlib: `print_comma_seperated(x)` · `print_currency(x)` · `print_compact(x)` · `print_compact_si(x)` · `print_compact_computing(x)` · `print_ordinal(x)` · `print_percentage(x)` · `digit_count(x)` | `"1,234,567"` · `"$1,234"` · `"1.23M"` (K/M/B/T/Q) · `"1.23M"` (k/M/G/T/P) · `"1.17Mi"` (base 1024) · `"3rd"` · `"25%"` · `3` |
1301
+ | stdlib: `round_nearest/round_up/round_down/round_truncate(x, step)` | step 0 returns `x`; `round_nearest(127, 10)` is `130` |
1302
+
1303
+ **`East.String`**
1304
+
1305
+ | Signature | Notes |
1306
+ |-----------|-------|
1307
+ | `concat(a, b)` · `repeat(s, n)` · `substring(s, start, end)` · `length(s) -> int` | build / measure |
1308
+ | `upper_case(s)` · `lower_case(s)` · `trim(s)` · `trim_start(s)` · `trim_end(s)` | case / whitespace |
1309
+ | `replace(s, find, replacement)` · `split(s, separator) -> Array<String>` | edit / tokenize |
1310
+ | `contains(s, substring)` · `starts_with(s, prefix)` · `ends_with(s, suffix)` · `index_of(s, substring) -> int` | search (`-> bool`/`int`) |
1311
+ | `regex_contains(s, pattern, flags="")` · `regex_index_of(s, pattern, flags="")` · `regex_replace(s, pattern, replacement, flags="")` | regex |
1312
+ | `parse(typ, s)` ❗ · `print(typ, value) -> str` (the root `East.print(value[, typ])` is the same builtin, value first) | East **text** format; `parse` is a **strict whole-string** parser — trailing or leading junk raises (`"598-"`, `"$5"`, `"1.2.3"` all raise; in a body use `.try_parse(T)` for the optional form) |
1313
+ | `parse_json(typ, s)` · `print_json(typ, value) -> str` / `print_json(value)` | East **JSON** (`Integer` encodes as a JSON *string*: `print_json(ArrayType(IntegerType), [1,2,3]) == '["1","2","3"]'`; an `Option` is `null` for `none` and the payload itself for `some` wherever the payload cannot be null — `print_json(OptionType(IntegerType), some(7)) == '"7"'` — and only `Option<Null>` / `Option<Option<T>>` keep the `{"type": …, "value": …}` object; a `DateTime` prints as `YYYY-MM-DDTHH:MM:SS.sss+00:00` and parses from any RFC 3339 date-time, as UTC); the one-argument form (TS `printJson(value)`) takes the value's own type |
1314
+ | stdlib: `print_error(message, stack) -> str` | `"Error: <message>"` + one `[i] file line:column` per `{filename, line, column}` frame (TS `printError`) |
1315
+
1316
+ **`East.Blob`**
1317
+
1318
+ | Signature | Notes |
1319
+ |-----------|-------|
1320
+ | `encode_beast(value, version="v1", *, typ=None) -> EastBlob` | TS `East.Blob.encodeBeast`: `"v1"` the original BEAST format, `"v2"` the beast2 family; the type is the expression's declared type / `type_of(value)`, or `typ` to encode a plain value under a wider type |
1321
+
1322
+ **`East`** root: `East.str(*parts)` (TS `East.str` — the parts concatenated, non-String parts printed
1323
+ in East text format: `East.str("n=", 5, "!")`), `East.print(value[, typ])` (TS `East.print` — the East
1324
+ text format under the value's own type, or `typ`), `East.min(a, b)` / `East.max(a, b)` (`least`/`greatest`
1325
+ under East's total order), `East.clamp(value, lo, hi)` and `East.jq(input, program, result_type)`
1326
+ ([Queries](#queries--eastjq)) — all dual-mode.
1327
+
1328
+ **`East.DateTime`** (see [DateTime format codes](#datetime-format-codes))
1329
+
1330
+ | Signature | Notes |
1331
+ |-----------|-------|
1332
+ | `from_components(year, month=1, day=1, hour=0, minute=0, second=0, millisecond=0)` | construct (TS defaults: the trailing components are the first instant). An out-of-range component NORMALISES into the next one, it never raises: `(2024, 2, 31)` is `2024-03-02`, `(2024, 13, 1)` is `2025-01-01`, `(2023, 2, 29)` is `2023-03-01`. Range-checking the inputs does not catch it (31 February is all in range) — build the date and check `dt.get_month() == month` |
1333
+ | `from_epoch_milliseconds(millis)` · `to_epoch_milliseconds(dt) -> int` | epoch round-trip |
1334
+ | `get_year/get_month/get_day_of_month/get_day_of_week(dt) -> int` | `get_day_of_week`: Monday == 1 |
1335
+ | `get_hour/get_minute/get_second/get_millisecond(dt) -> int` | components |
1336
+ | `add_milliseconds(dt, millis)` · `subtract_milliseconds(dt, millis)` · `duration_milliseconds(a, b) -> int` | the raw builtin (`subtract` negates the amount, TS `subtractMilliseconds`): `duration` returns **a − b** (the expression METHOD `a.duration_milliseconds(b)` is the TS method, `b − a`) |
1337
+ | `add_/subtract_{seconds,minutes,hours,days,weeks}(dt, n)` | unit sugar over `add_milliseconds` (an int or float `n`; an expression `n` scales inside the body, a Float after scaling) |
1338
+ | `duration_{seconds,minutes,hours,days,weeks}(a, b) -> float` | unit sugar over `duration_milliseconds` |
1339
+ | `print_formatted(dt, fmt) -> str` · `parse_formatted(s, fmt) -> datetime` | Day.js-style tokens (TS names; `print_format`/`parse_format` are deprecated spellings) |
1340
+ | stdlib: `round_down_/round_up_/round_nearest_{millisecond,second,minute,hour,day,week}(dt, step)` | `step` units of the name; weeks align to Mondays (the reference Monday 1969-12-29) |
1341
+ | stdlib: `round_down_month(dt, step)` · `round_down_year(dt, step)` | the first instant of the `step`-aligned month / year |
1342
+
1343
+ **`East.Boolean`**
1344
+
1345
+ | Signature | Notes |
1346
+ |-----------|-------|
1347
+ | `not_(x)` · `bit_and(a, b)` · `bit_or(a, b)` · `bit_xor(a, b)` | the BooleanNot / BooleanAnd / BooleanOr / BooleanXor builtins under the TypeScript names (`not`, `bitAnd`, `bitOr`, `bitXor`; `and_`/`or_`/`xor` are deprecated spellings) — both operands are values here, so there is nothing to short-circuit; the expression twins are `.not_()`, `.bit_and`, `.bit_or`, `.bit_xor`, and the short-circuit `.and_(fn(b))`/`.or_(fn(b))` take bodies |
1348
+
1349
+ **`East`** comparisons (East total order; element type `T` first): `compare(T, a, b) -> int`,
1350
+ `equal/not_equal/less/less_equal/greater/greater_equal(T, a, b) -> bool`, and `is_(T, a, b) -> bool` (East `Is`: identity for a mutable container, value otherwise).
1351
+
1352
+ **`East`** structural diff/patch (any East type `T`; a patch is a value of `PatchType(T)`; every
1353
+ function takes `T` explicitly — a type sampled from one value cannot describe both sides of a
1354
+ variant diff):
1355
+
1356
+ | Signature | Notes |
1357
+ |-----------|-------|
1358
+ | `diff(T, before, after) -> patch` | The patch turning `before` into `after` |
1359
+ | `apply_patch(T, value, patch) -> value` | Apply; `apply_patch(T, v, diff(T, v, w)) == w` |
1360
+ | `compose_patch(T, first, second) -> patch` | One patch equal to applying `first` then `second` |
1361
+ | `invert_patch(T, patch) -> patch` | The undo: applying patch then its inverse round-trips |
1362
+
1363
+ ### DateTime format codes
1364
+
1365
+ `print_formatted(dt, fmt)` / `parse_formatted(s, fmt)` take a Day.js-style string; tokens match
1366
+ greedily (longest-first), anything else is a literal, and `\` escapes the next char.
1367
+ Examples for `2025-03-05 14:09:07.123` (a Wednesday):
1368
+
1369
+ | Code | Meaning | Ex | Code | Meaning | Ex |
1370
+ |------|---------|----|------|---------|----|
1371
+ | `YYYY` | 4-digit year | `2025` | `YY` | 2-digit year | `25` |
1372
+ | `MMMM` | full month | `March` | `MMM` | short month | `Mar` |
1373
+ | `MM` | month 01-12 | `03` | `M` | month 1-12 | `3` |
1374
+ | `DD` | day 01-31 | `05` | `D` | day 1-31 | `5` |
1375
+ | `dddd` | full weekday | `Wednesday` | `ddd` | short weekday | `Wed` |
1376
+ | `dd` | min weekday | `We` | `HH` | hour 00-23 | `14` |
1377
+ | `H` | hour 0-23 | `14` | `hh` | hour 01-12 | `02` |
1378
+ | `h` | hour 1-12 | `2` | `mm` | minute 00-59 | `09` |
1379
+ | `m` | minute 0-59 | `9` | `ss` | second 00-59 | `07` |
1380
+ | `s` | second 0-59 | `7` | `SSS` | millisecond | `123` |
1381
+ | `A` | AM/PM | `PM` | `a` | am/pm | `pm` |
1382
+
1383
+ ```python
1384
+ East.DateTime.print_formatted(dt, "YYYY-MM-DD HH:mm:ss.SSS") # '2025-03-05 14:09:07.123'
1385
+ East.DateTime.print_formatted(dt, "dddd, MMMM D, YYYY h:mm A") # 'Wednesday, March 5, 2025 2:09 PM'
1386
+ ```
1387
+
1388
+ ### Beast2 streaming — bounded-memory collections (`from east.serialization.beast2 import ...`)
1389
+
1390
+ Beast2 v5 encodes a large Array/Set/Dict as an append-only stream of
1391
+ independently decodable segments: writer memory is one segment, never the whole
1392
+ collection, and the decoders read v4 and v5 through the same entry points. The
1393
+ canonical writers — the managed file writer, `Beast2ElementWriter`,
1394
+ `encode_beast2_paged_for` — cut segments by one content-defined rule, the same in
1395
+ TypeScript and east-c, so a value's bytes are its own whoever writes them;
1396
+ `Beast2Writer` makes each batch you give it a segment instead.
1397
+
1398
+ **Managed files — start here (`open_beast2_file` / `write_beast2_file`).** Path
1399
+ in, East values out. The file is self-describing, so a read needs no declared
1400
+ type (a write does; a type declared on a read is validated at open). The file
1401
+ object owns the fd and the mmap (closed on `with`-exit), east-c does every byte,
1402
+ and the segments are the canonical ones — no buffers, iterators or batch sizes in
1403
+ your code. The read flavour mirrors the root collection's read surface name for
1404
+ name.
1405
+
1406
+ | Signature | Description |
1407
+ |-----------|-------------|
1408
+ | `write_beast2_file(path, T, value, *, codec="deflate")` | One call writes a collection of any size as one indexed v5 file — the value's canonical blob: segments where the content-defined cut rule places them, bounded in elements and bytes (wide rows never pile into one), key-disjoint for a Dict/Set. The bytes TypeScript and east-c write, and what `encode_beast2_paged_for(T)(value)` returns |
1409
+ | `open_beast2_file(path, T, mode="w", *, codec=)` | The streaming writer: `.write()` takes East collections or python `list`/`dict`/`set` batches of any size, which add up to one canonical blob (a Set/Dict batch continues strictly ascending from the last); `.segments` counts the segments closed so far |
1410
+ | `open_beast2_file(path, T=None, *, project=None)` | The read: `Beast2ArrayFile` / `Beast2DictFile` / `Beast2SetFile`, a READ-ONLY East collection VALUE subclassing `EastArray`/`EastDict`/`EastSet` — `isinstance`/`type_of` answer, every eager method works (streamed, below), mutation raises, and it binds into functions or passes into compiled calls by reference, where keyed reads answer from the pager, one frame per hit or miss, through a byte-budgeted segment cache (`EAST_PAGED_CACHE_BYTES`); `close()` defers while a bind holds it. `T` is optional (the header supplies it, also as `f.wire_type`); a declared `T` is checked against the header at open, never decoded as garbage |
1411
+ | `read_beast2_type(source) -> EastType` | The root type of any beast2 blob (v4 or v5), from a path or a buffer, nothing decoded — for a file you know nothing about |
1412
+ | `write_beast2_file_parallel(path, T, partitions, produce, *, processes=, strategy="auto", codec=, keep_shards=False, verify=False)` | N workers, ONE file: `produce(partition)` returns that partition's batches (or one collection); each worker writes a shard cut canonically from its start, and the shards splice **in partition order** as they finish. `strategy="auto"` forks on Linux/macOS — what `produce` closes over is inherited copy-on-write, so build the expensive context before the call (and before starting threads) — and runs inline on Windows, byte-identically. Any worker failure fails the call with the worker's traceback and leaves nothing behind |
1413
+ | `splice_beast2_files(path, T, sources, *, verify=False) -> (segments, elements)` | Merges indexed v5 files by **byte copy** — east-c parses the geometry, `os.sendfile` moves the frames, nothing re-encodes. `sources` may be a lazy generator (spliced as they complete, in row order). Each must be v5, indexed and self-contained with an identical type section; a refusal names the path and leaves no destination. `verify=True` re-walks the result with east-c's strict reader |
1414
+ | `f.load()` | The whole collection, decoded in east-c off the mmap — input memory stays one segment at any size; also the mutable escape hatch, like `f.copy()` |
1415
+ | `f.segments()` | DEPRECATED — the file IS its collection value; still works (with a warning) for per-batch migration code |
1416
+ | `len(f)` · `f.segment_count` · `f.self_contained` · `f.indexed` | O(1) from the trailing index, exact for every root kind |
1417
+ | Array: `f[i]` / `f[a:b]` · `f.get(i)` ❗bounds · `f.get_or_default(i, d)` · `f.try_get(i)` · `f.has(i)` · `f.slice(a, b)` · `f.get_keys(rows)` | `EastArray`'s names, signatures and errors; a point read decodes only the owning segment, `get_keys` each owning segment once |
1418
+ | Dict: `f[k]` ❗KeyError · `f.get(k[, default \| fn(k)])` · `f.get_or_default(k, d)` · `f.try_get(k)` · `f.has(k)` / `k in f` · `f.get_keys(keys, fill)` · `f.items()` / `values()` / iteration (streaming) · `f.keys()` (the Set) · `f.size()` — Set: `x in f` / `f.has(x)` | east-c binary-searches the segment *fences* (each segment's first key, from a bounded probe, cached), then decodes ONLY the owning segment (a small LRU keeps hot ones); `get_keys` merges the sorted keys against the fences and calls `fill` per missing key. The first keyed read verifies the fences: a corrupt or pre-contract blob raises `segments are not disjoint ascending key ranges` instead of reporting false misses |
1419
+ | Array sorted search: `f.find_sorted_first/last(target)` → a global index · `f.find_sorted_range(target)` → `{start, end}` | `EastArray`'s contract over the whole file, only the boundary segment decoded; no `key=` projection (the file pages in element order) — pair with `f.slice(start, end)` |
1420
+ | Compute: `map` `filter` `filter_map` `first_map` `reduce` `scan` `map_reduce` `sum` `mean` `maximum` `minimum` `every` `some` `find_*` `is_sorted` `to_set` `unique` `to_dict` `to_array` `to_columns` `map_batches` `string_join` `flat_map` (Array) / `flatten_to_array` (Set, Dict) `flatten_to_set` `flatten_to_dict` `for_each`, the whole `group_*` family, Set algebra | Each segment runs the ordinary eager method, and partials combine in east-c in stream order: order-dependent folds thread ONE accumulator and grouped folds seed each segment from the running per-group accumulators, so results equal `load()` exactly, float order included. Array `(el, idx)` callbacks, and the indices `find_*` / `group_find_*` report, are GLOBAL; `first_map`/`some`/`every`/`is_superset_of` stop decoding at the answer. Dict/Set compute streams disjointness-verified segments. A re-keyed collision (`to_dict`, `flatten_to_dict`, `group_to_dicts`) combines left-associatively in stream order — use an associative `combine`. `sort`/`reverse`/`copy`/`concat`/`union` materialize the collection — `load()` first |
1421
+ | Column projection — INFERRED on the compute family; EXPLICIT with `open_beast2_file(path, project=NARROW)` | The compute family builds its callbacks FIRST and decodes each segment to just the struct fields their IR reads (the rest are hopped over, never built — building values, not walking bytes, is what decoding costs). Fields subset by name at any depth; a subtree used any other way stays whole, so results never change; Dict keys and Set elements never narrow. Runner-opened task inputs infer the same from the body's loop IR. What cannot be inferred decodes whole and is COUNTED in `eager_stats()` (`beast2_segments_projected/whole`, `beast2_projection_declined_*` by reason) — a projection that silently stopped applying would be an invisible cliff. The explicit form serves the subset to EVERY read (point reads, keyed gets, `load()`); `project` must be a subset of the wire type (a missing field raises `ValueError` naming it), a declared `T` keeps its meaning, and `find_sorted_*` refuse under it. A segment decoded under one mask is never served to a read needing more. No wire change |
1422
+ | Degraded blobs | v4 → refused (`decode_beast2_with_header_for` still decodes it whole); index-less v5 → `load()` works, random access refuses; not self-contained → point reads refuse |
1423
+ | **Manifest directories** — one object per segment plus a manifest naming them: how e3 stores a collection, and stages a task input |
1424
+ | `Beast2ManifestWriter(T, path, *, codec="deflate")` | The canonical writer (a context manager): `.add(el)` / `.add_all(batch)`, as `Beast2ElementWriter` takes them; each segment goes to `<path>.segments/<sha256>.beast2`, a standalone blob under the shared header, and `.close()` writes the manifest to `path` — the directory TypeScript's `Beast2ManifestWriter` and east-c write. A writer left by an exception writes no manifest; `.segments` counts them; east-c does every byte, hashing included |
1425
+ | `load_beast2_manifest(path, T=None)` ❗ | The whole collection, segment by segment in east-c — the value its segments spliced into one blob decode to. `ValueError` when `path` holds no manifest, the types differ, or a segment is missing or malformed |
1426
+ | `read_beast2_manifest(source)` | The manifest — `kind`, `level`, `type`, `rule`, `header`, `entries` (each `{hash, fence, count, bytes}`) — or `None`; a blob holding a value is read no further than its type section |
1427
+
1428
+ ```python
1429
+ from east.serialization.beast2 import open_beast2_file, write_beast2_file
1430
+
1431
+ write_beast2_file(path, rows_t, rows) # any size, one call
1432
+
1433
+ with open_beast2_file(path) as f: # self-describing: type from the
1434
+ row = f[1_234_567] # header (f.wire_type); pass rows_t
1435
+ totals = f.group_sum(lambda b, r: r["sku"], # instead to VALIDATE it at open
1436
+ lambda b, r: r["qty"]) # whole-file compute: segment folds,
1437
+ top = f.maximum(by=lambda b, r: r["qty"]) # never materialized, == load() exactly
1438
+ table = f.load() # whole table when you truly need it
1439
+
1440
+ # The file IS a collection value: bind it into a function and the
1441
+ # compiled body's keyed reads answer from the pager — one frame per lookup.
1442
+ with open_beast2_file("table.beast2") as t: # Dict<String, Float>
1443
+ lookup = East.function([StringType, DictType(StringType, FloatType)], FloatType,
1444
+ lambda b, k, d: d.get_or_default(k, 0.0)).bind(t)
1445
+ joined = rows.map(lambda b, r: r.v + lookup(r.k)) # loop + callee + pager: all east-c
1446
+ ```
1447
+
1448
+ **Buffer level — you hold bytes, not a path. `_for` is NOT `_with_header_for`:**
1449
+
1450
+ | Signature | Description |
1451
+ |-----------|-------------|
1452
+ | `encode_beast2_with_header_for(T, *, version=None)` / `decode_beast2_with_header_for(T)` | **The one you want**: the self-describing container — magic, type schema, value. Encode writes the current default container (v5; `version=4` only for a reader that predates it); decode reads v4 and v5, whole (Set/Dict segments must hold the canonical value — sorted, disjoint — or the blob is rejected as corrupt). ❗The header's type must be `T`, or a subtype whose variant cases line up with `T`'s (a `none` reads as any `Option`), else `ValueError: beast2: cannot decode a blob of type … as …` before anything decodes — the words TypeScript and east-c use |
1453
+ | `encode_beast2_for(T)` / `decode_beast2_for(T)` | **Headerless** type-directed bytes: the reader must know `T` exactly, and mutable containers (Array/Set/Dict/Ref) are refused. In TypeScript `encodeBeast2For` is the FULL container — do not port a call site by name |
1454
+ | `Beast2ElementWriter(T, stream, *, codec="deflate", parallel=False)` | The canonical writer (a context manager): `.add(el)` — an Array/Set element or a Dict `(key, value)` pair — and `.add_all(batch)` (the loop in east-c); Set elements and Dict keys strictly ascending in East order. Segments fall where the cut rule places them — the bytes the managed writer, TypeScript and east-c write; memory is one open segment; `.close()` writes the last segment, the terminator and the index; `.segments` counts the closed ones. An element that does not ascend, or fails to encode, raises and leaves the writer as it was |
1455
+ | `encode_beast2_paged_for(T, *, codec="deflate") -> (value) -> bytes` | A whole collection through the canonical writer — the write side of `open_beast2_pages_for` |
1456
+ | `Beast2RunSorter(T, open_run, *, merge=None, union=False, codec="deflate", parallel=False)` | A Set's or Dict's elements in ANY order in, sorted canonical runs out: `.add(el)` encodes at once; at `RUN_MAX_COUNT` elements or `RUN_MAX_BYTES` it sorts the run — stably, so a key's values keep the order they came in — folds a repeated key (`merge`, a compiled `(K, V, V) -> V` East function, for a Dict; `union=True` for a Set; neither: it raises) and writes the run to `open_run(run)` (anything with `write(bytes)` and `close()`) as its value's canonical blob. `.finish()` writes the last run; `.runs` counts them. The runs are the ones east-c and TypeScript write; a sink's exception comes back out of the call that wrote the run; a key repeated across runs is the merge's to fold |
1457
+ | `Beast2Writer(T, stream, *, codec="deflate", self_contained=True, index=True, parallel=False)` | Segments of your choosing (a context manager): `.write(batch)` appends one segment per non-empty batch; `.close()` writes the terminator and the index; `.segments` counts. Set/Dict batches must ascend strictly across the stream. **Keep both defaults** — `index` and `self_contained` are what `open_beast2_pages_for` needs, and turning either off silently forfeits random access. `codec="none"` for already-compressed payloads or raw write throughput |
1458
+ | `encode_beast2_segments_for(T, **opts) -> (batches) -> bytes` | In-memory `Beast2Writer`: one segment per non-empty batch |
1459
+ | `encode_beast2_v5_for(T, *, codec="deflate", index=False) -> (value) -> bytes` | A whole value of any root type as v5; decode with `decode_beast2_with_header_for` |
1460
+ | `iter_beast2_segments_for(T) -> (source) -> iterator` | One decoded collection per segment, O(segment) memory; `source` is bytes, an `mmap` or a binary stream |
1461
+ | `read_beast2_index(T, blob) -> (segments, elements) \| None` | O(1) totals from a v5 blob's trailing index |
1462
+ | `open_beast2_pages_for(T) -> (source) -> Beast2Pages` | Random access: `.segment_count` `.element_count` `.self_contained` `.counts`, `.segment(i)`, `.element(row)` (also `len()` / `[]`), each decoding ONE segment. ❗Needs `index=True` and `self_contained=True` (the defaults); `.element()` is for Array roots. ❗Borrows the buffer — keep it (and an mmap) alive for the pages' lifetime, or use `open_beast2_file`, which owns it |
1463
+
1464
+ **Batch size (`Beast2Writer` only — the canonical writers cut by the rule).** A
1465
+ batch is at once your memory ceiling, one segment, one compression window and
1466
+ the granularity of random access. ~1000 rows is a good default: one row per
1467
+ batch costs about 4× the bytes, and the curve is flat past ~100. `write()` takes
1468
+ a batch, never a row — accumulate and flush yourself, or hand rows to
1469
+ `Beast2ElementWriter`, which needs no batch at all.
1470
+
1471
+ ```python
1472
+ import mmap
1473
+ from east.serialization.beast2 import (
1474
+ Beast2Writer, iter_beast2_segments_for, open_beast2_pages_for,
1475
+ )
1476
+ rows_t = ArrayType(row_type)
1477
+
1478
+ with open(path, "wb") as f, Beast2Writer(rows_t, f) as w:
1479
+ for batch in produce_batches(): # each an EastArray(row_type, ...)
1480
+ w.write(batch) # O(batch) memory, one segment each
1481
+
1482
+ # Sequential re-read. mmap, NOT .read() — .read() pulls the whole file in and
1483
+ # throws away the bounded-memory property you just paid for.
1484
+ with open(path, "rb") as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
1485
+ for batch in iter_beast2_segments_for(rows_t)(mm):
1486
+ consume(batch) # O(segment) decoded at a time
1487
+
1488
+ # Or jump straight to a row — decodes only the segment that owns it.
1489
+ pages = open_beast2_pages_for(rows_t)(mm)
1490
+ row = pages.element(1_234_567)
1491
+ ```
1492
+
1493
+ ### Columnar escape hatches — when the logic must stay python
1494
+
1495
+ When per-element logic genuinely needs python (numpy, a model, an external
1496
+ library), don't touch rows one at a time — cross the boundary **once per
1497
+ column** instead of once per row × field:
1498
+
1499
+ ```python
1500
+ cols = rows.to_columns() # {"price": np.float64[...], "qty": np.int64[...],
1501
+ # "sku": [str, ...] (interned), ...} — one crossing/column
1502
+ amount = cols["price"] * cols["qty"].astype(np.float64) # vectorised numpy
1503
+ out = EastArray.from_columns(Out, {"sku": cols["sku"], "amount": amount})
1504
+
1505
+ result = rows.map_batches(f, out=Out, batch_size=50_000) # f sees columnar chunks;
1506
+ # batches may shrink (filter-like)
1507
+
1508
+ acc = EastDict(StringType, FloatType) # dicts as accumulators:
1509
+ acc.update_many(keys, values, combine=lambda b, cur, new: cur + new) # one crossing,
1510
+ # combine builds -> collisions in C
1511
+ arr.extend(np_array) # bulk push, one crossing
1512
+
1513
+ solver_input = coerce_to( # struct fields take 1-D numpy
1514
+ {"start_nodes": np_i64, "capacities": np_i64, ...}, # columns directly: each
1515
+ MinCostFlowInputType) # Array<Int/Float/Bool> fills C-side
1516
+ ```
1517
+
1518
+ `Float`/`Integer`/`Boolean` columns move through numpy buffers filled in C
1519
+ (`Option<Float>` ↔ float64 with NaN for `none`); `String` columns box once
1520
+ through a bounded intern table (repeated categories/ids come back as the
1521
+ same python object); other field types fall back to boxed lists. The same
1522
+ contract governs the INPUT direction: `from_columns`/`coerce_to` fill C-side
1523
+ only when a numeric column arrives as a numpy array of the matching dtype
1524
+ (float64/int64/bool; float64-with-NaN for `Option<Float>`) — a plain python
1525
+ list converts per cell, so `np.asarray(col)` first when the source is a
1526
+ list. Composition rule: **East functions for East-expressible transforms,
1527
+ columns/batches for the genuinely-python remainder.**
1528
+
1529
+ **Put the logic in the platform function, not a pure-python shim.** A
1530
+ `@East.platform_function` is just a typed, validated python function; its one
1531
+ added cost, validating the declared output, is a feature. A separate helper layer
1532
+ over `list`/`dict` that the platform function converts into and out of costs
1533
+ **testability** (the typed `inputs`/`output` is the contract you test; untyped
1534
+ helpers turn bugs into silent corruption instead of a named `EastTypeError`),
1535
+ **portability** (a platform function over East values moves to an e3 task,
1536
+ another runtime or a TypeScript mirror unchanged), a **forced sandwich**
1537
+ (East→python on the way in, back on the way out) and **purity** (a platform
1538
+ function pure in its East inputs is memoisable). Small reusable functions are
1539
+ fine — have them take and return East values, pay output validation only at the
1540
+ real East↔python edge, and never call a platform function per element in a loop.
1541
+
1542
+ ### Platform functions
1543
+
1544
+ | Signature | Description |
1545
+ |-----------|-------------|
1546
+ | `@East.platform_function(*, inputs, output, name=None, is_async=None, validate_output=True, validate_input=False)` | Register a Python fn; infers sync/async from the def (`is_async=` declares it when they differ — a function East calls asynchronously on every runtime whose python is a plain `def`); validates output against `output`. DUAL-MODE: called with values it runs the python (an async def's coroutine, to await); called inside an `East.function` / `East.asyncFunction` body it emits the `Platform` node with this signature — an async one inside a sync body is the build-time error. Paired at compile by name (the def's, or `name=`). Also importable bare: `from east import platform_function` |
1547
+ | `@East.generic_platform_function(*, type_parameters, name=None, is_async=False, inputs=None, output=None, type_erased=False)` | Type-parameterized factory: the decorated fn is `fn(platform, *type_params) -> impl`; `is_async` is **explicit** (not inferred). With `inputs=`/`output=` (placeholders allowed anywhere: `output="T"`, `output=ArrayType("T")`) it is dual-mode too: `fs_open_beast(DictType(K, V), path)` in a body — the type arguments first, as TypeScript reads — emits the generic node; without them a body call is a build error naming what to declare. `type_erased=True` when the implementation ignores the type arguments (it reads the values): the decorated fn IS the implementation, python calls it directly (`causal_experiment(rows, config)`) and the factory the runtime binds is derived |
1548
+ | a platform function RETURNING a lazy value | A `@platform_function` may return a paged hold — an `open_beast2_file(...)` value, `open_paged_file(T, path)` (a C-owned mapping), `fs_open_beast(...)`'s result — and the compiled body keeps its O(segment) cost model: the C value crosses the return seam by pointer (checked against the declared output type, never re-marshalled through python), and nothing python-side has to outlive it: the mapping is the value's own. The mirror of a paged ARGUMENT crossing in |
1549
+ | `East.platform_functions(module) -> list` | Collects every decorated fn in `module` (pass `__name__`). Two consumers: `East.compile()` for in-process use, and a package's top-level `platform` list that `east-py run -p <module>` (and the e3 `{ custom }` runner) loads |
1550
+ | `@memoize` / `@memoize(salt="…")` / `memoized = memoize(fn, salt="…")` | Content-addressed memo over ONE platform function. Apply **above** `@East.platform_function` (or inline on an imported one). Key = sha256(name + salts + per-input digests of the with-header BEAST2 encodings via the declared input types); value = with-header BEAST2 of the output, decoded via the declared output type. Inert by default |
1551
+ | `configure_memo(directory, salt="")` | Activate (`None` deactivates) memoization for `@memoize` functions; overrides `EAST_MEMO_DIR` / `EAST_MEMO_SALT` env vars. Bump `salt` to invalidate after code edits — input-derived keys can't see them |
1552
+
1553
+ ### What is NOT east-c
1554
+
1555
+ Nearly everything above delegates to native builtins. What still runs python,
1556
+ so you can reason about cost and semantics:
1557
+
1558
+ - **`EastMatrix.map_elements`** — DEPRECATED: a python callback per element,
1559
+ the one shape the strict surface removes; it warns and will go. Use the
1560
+ arithmetic / reduction / mask / sparse methods, `map_rows`, or
1561
+ `to_numpy()`/`to_torch()`. (`Vector.map`/`reduce` and `Matrix.map_rows` are the
1562
+ native VectorMap / VectorFold / MatrixMapRows builtins.)
1563
+ - **`Dict.get_or_insert`** composes membership and get python-side so `fn` runs
1564
+ only on a miss (lazier than East's strict default). The other singles
1565
+ (`insert`/`update`/`swap`/`delete`/`try_delete`/`insert_or_update`) are the
1566
+ native builtins, error messages included.
1567
+ - **Reduction sugar** — `mean`, `group_mean`, `group_size`, …: several native
1568
+ passes, zero python per element, but not one fused builtin (as in TypeScript).
1569
+ - **Boundary utilities** — `coerce_to`/`assert_value_of`/`type_of`, `variant`/
1570
+ `struct` validation, `match()` dispatch, `compare_for`/`make_east_key`: python
1571
+ walkers, by design — they are the python↔East edge.
1572
+ - **Iteration** — `for x in arr` / `list(arr)` boxes per element (lazily, under
1573
+ East's iteration lock, so mutating during the loop raises `Cannot modify …
1574
+ during iteration`, as an East for-loop does); cross once with
1575
+ `to_columns`/`to_numpy` instead.
1576
+
1577
+ ## The east-py CLI
1578
+
1579
+ `east-py` (the `east-py-cli` package) runs East IR with python platform
1580
+ functions — the runner e3 drives for `runtime: 'east-py'` — and carries the
1581
+ authoring tools:
1582
+
1583
+ ```bash
1584
+ east-py run prog.beast2 -p east-py-std -i a.beast2 -i b.json -o out.beast2 # -v: timings, peak memory
1585
+ east-py run prog.beast2 -p east-py-std --profile # time per East and platform function
1586
+ east-py exec unit.beast2 # one unit: the runner protocol e3 speaks
1587
+ east-py convert value.beast2 # a beast2 value as East text (-o value.east to write it)
1588
+ east-py transpile prog.beast2 -o prog.py # IR → a python module that rebuilds it
1589
+ east-py export-functions pricing -o pricing.functions.beast2 -p east-py-std
1590
+ east-py lint src/ ; east-py check src/ ; east-py lsp
1591
+ east-py version -p east-py-std # the runner's version, and each package's
1592
+ ```
1593
+
1594
+ | Command | Does |
1595
+ |---|---|
1596
+ | `run <ir> [-p PACKAGE]… [-i FILE]… [-o FILE] [--decode lazy\|whole] [-v] [--profile]` | Compiles an IR file (`.beast2`, `.beast`, `.east` or `.json`) and calls it with one `-i` file per parameter, in order, each decoded by its extension to the parameter's type and FROZEN (a task input is immutable). An indexed beast2 collection input opens lazily, whatever it weighs — mapped, one segment decoded at a time, and decoded whole, once, when an operation the pager cannot serve first needs it — and a manifest file opens as the collection it names; `--decode whole` (east-c's and east-node's flag too) decodes every input before the program runs instead. `-o` writes the result in its extension's format (a collection as canonical indexed beast2); without it the result prints as East text. `-v` adds timings, peak memory and how each input was read: a lazy input's segment reads, or the resident memory a whole decode added |
1597
+ | `exec <unit> [-v] [--profile]` | The runner protocol: one unit file says what to run — a program over its inputs, read as its `decode` says (as `run --decode`), or a merge of an output's parts — and where; the output is written by its kind and a result recorded. Exit 0 when it succeeded, 1 when it failed (the message and locations also go to stderr), 2 when the unit cannot be read or the result written. Relative paths in the unit are relative to its directory |
1598
+ | `convert <file> [-o FILE.east] [-v]` | Decodes a beast2 value (its header names the type) and prints it as East text, or writes it to a `.east` file |
1599
+ | `transpile <ir> [-o FILE] [--name NAME] [-p PACKAGE]…` | IR as a python module rebuilding it (`--name` binds it, default `main`; each `-p` prints its calls as that package's own functions) — [IR ↔ python](#ir--python-east-py-transpile-and-the-east-c-ir-toolbox) |
1600
+ | `export-functions <module> -o FILE [-p PACKAGE]… [--name NAME] [--package-version V] [--only NAME]…` | A module's (a dotted name, or a `.py` path) `east_functions` as a function manifest; every platform call must be provided by a `-p` package; `--name` defaults to the module's top-level name, the version to the installed distribution's (else `0.0.0`); `--only` exports just those — [Cross-language functions](#cross-language-functions-east-py-export-functions-and-eastimport_function) |
1601
+ | `lint [paths…] [--format text\|json] [--disable RULE]… [--exclude DIR]… [--list-rules]` | The rules (default path `.`); exit 1 on any finding, a warning included, 2 on a usage error — [Diagnostics](#diagnostics-at-edit-time--east-py-lint-east-py-check-flake8-east-py-lsp) |
1602
+ | `check <targets…> [--format text\|json] [--only-if-enabled]` | The build's own errors; exit 1 on any |
1603
+ | `lsp [--probe]` | The Language Server over stdio (needs `pygls`); `--probe` reports whether it can start here |
1604
+ | `version [-p PACKAGE]…` | The `east-py-cli` and `east-py` versions (and the Cython extensions loaded), and each package's version and function count |
1605
+
1606
+ A `-p` package is a python module name (`east-py-std` imports `east_py_std`)
1607
+ exporting a top-level `platform` list; a project's own module is loaded the same
1608
+ way (see [Project-owned platform module](#project-owned-platform-module-calling-your-python-from-e3)).
1609
+ `run` and `exec` take `--exit-with-parent`: exit once stdin reaches end of file,
1610
+ for a parent that holds a stdin pipe it never writes to.
1611
+
1612
+ **Profiling.** `run` and `exec` take `--profile`, east-c's profiler and report:
1613
+ after the work (and after a failure's error), every East function and platform
1614
+ function it called, by self time, with its calls, its total time and where it
1615
+ is — the `file:line:column` it was written at and, for a helper called inside
1616
+ another body, where it was called. A platform function (torch, a solver) is an
1617
+ entry of its own, so its time is not its caller's, and so is each input read
1618
+ lazily, named as `-v` names it (`input 1`): the segments its reads decode, a
1619
+ call each — keyed reads at random beyond the segments the runner keeps, or a
1620
+ scan repeated, show there, not in the reading function's self time. `EAST_PROFILE=1` profiles
1621
+ without the flag — e3 hands its environment to the runner, so a task's report
1622
+ lands in its log (`e3 task logs`) — and `EAST_PROFILE_INTERVAL=N` prints the
1623
+ profile so far every N seconds while the work runs. A python function linked
1624
+ into a TypeScript task carries its locations through the manifest, so the
1625
+ report names its python lines.
1626
+
1627
+ ## Key Patterns
1628
+
1629
+ ### Python values vs East expressions
1630
+
1631
+ The `east` skill's first rule, in python: a body's PARAMETERS are expressions;
1632
+ a python value used inside a body must lift. Scalars lift on their own; a
1633
+ collection must be an explicit choice.
1634
+
1635
+ ```python
1636
+ RATES = EastDict(StringType, FloatType, {"AUD": 0.65}) # python-side East value
1637
+ CUTOFF = 100
1638
+
1639
+ @East.function([Row], FloatType)
1640
+ def usd(b, r):
1641
+ return East.if_else(r.amount > CUTOFF, # a scalar constant lifts (TS East.value)
1642
+ r.amount * RATES.get_or_default(r.ccy, 1.0), # an EXPLICIT build snapshots the dict
1643
+ 0.0)
1644
+
1645
+ rows.map(lambda b, r: r.amount * RATES.get_or_default(r.ccy, 1.0)) # ❌ an eager callback REFUSES
1646
+ # the collection capture:
1647
+ conv = East.function([Row, DictType(StringType, FloatType)], FloatType,
1648
+ lambda b, r, t: r.amount * t.get_or_default(r.ccy, 1.0))
1649
+ rows.map(conv.bind(RATES)) # ✅ bind it: live, zero-copy
1650
+ ```
1651
+
1652
+ ### The canonical platform function
1653
+
1654
+ ```python
1655
+ from east import East, FloatType, StringType, StructType, ArrayType, struct
1656
+
1657
+ LineItem = StructType([("name", StringType), ("price", FloatType)]) # Struct<String, Float>
1658
+
1659
+ @East.platform_function(inputs=[FloatType, ArrayType(LineItem)], output=ArrayType(LineItem))
1660
+ def convert_prices(fx_rate, items):
1661
+ # items: an east-c-backed array with eager methods; row["price"] is a plain float
1662
+ return items.map(lambda b, row: struct(
1663
+ {"name": row["name"], "price": row["price"] * fx_rate}, # plain f64 * f64
1664
+ LineItem, # tag + validate the result
1665
+ ))
1666
+ ```
1667
+ `.map` builds the lambda into an East function — the loop and the row construction run
1668
+ in east-c, with `fx_rate` baked in as a constant; `struct(..., LineItem)` is dual-mode, so
1669
+ the same spelling builds the row here and on plain values; the decorator validates the
1670
+ `Array<LineItem>` result — a named `EastTypeError` instead of silent corruption.
1671
+
1672
+ ### Project-owned platform module (calling your Python from e3)
1673
+
1674
+ To call a Python platform function from an **e3 task**, package it so `east-py
1675
+ run -p <module>` can load it: each module ends with `<name>_impl =
1676
+ East.platform_functions(__name__)`, and the package `__init__.py` aggregates them into
1677
+ a top-level `platform` list (the same shape as east-py-std / east-py-datascience).
1678
+
1679
+ ```python
1680
+ # platform_module/forecast.py
1681
+ @East.platform_function(inputs=[ArrayType(FloatType)], output=FloatType,
1682
+ name="my_project.forecast") # dotted "<project>.<fn>"; MUST byte-match
1683
+ def forecast(history): # the TS East.platform(...) declaration
1684
+ return sum(history) / len(history) if history else 0.0
1685
+ forecast_impl = East.platform_functions(__name__)
1686
+
1687
+ # platform_module/__init__.py
1688
+ from .forecast import forecast_impl
1689
+ platform = [*forecast_impl] # what `east-py run -p platform_module` loads
1690
+ ```
1691
+
1692
+ The e3 task wires it with `{ runtime: "east-py", platforms: [{ custom:
1693
+ "platform_module" }, "east-py-std"] }`. East code needs a TS `East.platform(
1694
+ "my_project.forecast", [...], ...)` **declaration** with the identical name (no
1695
+ codegen). Add native deps (numpy, …) to `pyproject.toml` and run `uv sync`. See
1696
+ **east-project** for the full scaffold (`--platform`) and the setuptools
1697
+ packaging; **e3** for the runner.
1698
+
1699
+ ### Memoize expensive pure stages (dev/test harnesses)
1700
+
1701
+ ```python
1702
+ from east import East, configure_memo, memoize
1703
+
1704
+ @memoize # eligibility — the author asserts purity
1705
+ @East.platform_function(inputs=[ArrayType(RowType), ConfigType], output=ModelType)
1706
+ def train_model(rows, config): ...
1707
+
1708
+ # A test harness flips the whole package on with one call (or EAST_MEMO_DIR):
1709
+ configure_memo("/tmp/my_project", salt="v1") # None deactivates
1710
+ loaded = memoize(other_pkg.load_csv, salt=file_digest) # inline, per-call salt
1711
+ ```
1712
+
1713
+ The platform-function boundary is the cache boundary, e3-style: hit = decode the
1714
+ stored BEAST2 blob via the declared output type (skipping the body entirely),
1715
+ miss = compute + atomic save. Only mark functions **pure in their East inputs** —
1716
+ a hit on a function that writes files/rows/requests silently drops those effects.
1717
+ Stochastic fits memoize their first realization. Code edits don't change keys —
1718
+ bump the salt. Inert with no directory configured: under e3, the dataflow's
1719
+ content-addressed task cache is the real memo and this stays off.
1720
+
1721
+ ### Fork-parallel export of a huge table (one file out)
1722
+
1723
+ The shape for "N CPUs on one table": build the expensive shared context once,
1724
+ let forked workers inherit it copy-on-write, and ship a single indexed file —
1725
+ consumers never learn the export was parallel. (Windows runs the same contract
1726
+ inline, sequentially; the output is byte-identical either way.)
1727
+
1728
+ ```python
1729
+ from east.serialization.beast2 import open_beast2_file, write_beast2_file_parallel
1730
+
1731
+ ROW = StructType([("order_id", StringType), ("qty", IntegerType), ("total", FloatType)])
1732
+
1733
+ lookups = load_reference_tables() # multi-GB keyed dicts: built ONCE, pre-fork,
1734
+ # inherited COW — never pickled, never rebuilt
1735
+
1736
+ def produce(span): # runs in the worker process
1737
+ start, count = span # yield batches of any size — the managed
1738
+ for chunk in chunk_ranges(start, count, 8192): # writer cuts the canonical segments
1739
+ yield compute_rows(lookups, chunk) # eager methods + East functions: native
1740
+
1741
+ write_beast2_file_parallel(
1742
+ "orders.beast2", ArrayType(ROW),
1743
+ partitions=row_ranges(total_records, shards=13), # partition order = row order
1744
+ produce=produce,
1745
+ processes=13, # a worker failure kills the rest, cleans up, re-raises
1746
+ )
1747
+
1748
+ # Consumers — e.g. the table's @East.platform_function loader — see ONE file, no
1749
+ # catalog, no shard names, and read it at one segment of memory:
1750
+ @East.platform_function(inputs=[StringType], output=ArrayType(ROW))
1751
+ def load_orders(path):
1752
+ with open_beast2_file(path, ArrayType(ROW)) as f:
1753
+ return f.load() # or compute on f directly, never holding the table
1754
+ ```
1755
+
1756
+ Shards produced by your own process topology (or on another machine)? Merge
1757
+ them directly — a pure byte copy, nothing re-encoded:
1758
+
1759
+ ```python
1760
+ splice_beast2_files("orders.beast2", ArrayType(ROW), sorted(shard_paths), verify=True)
1761
+ ```
1762
+
1763
+ ### Keyed lookups against a file too big to load
1764
+
1765
+ A reference table exported as a Dict file answers point reads without ever
1766
+ being held in memory: east-c fence-searches the segment index and decodes
1767
+ one segment per lookup, so a sparse join against a multi-GB file stays
1768
+ cheap. (A *dense* pass over most keys is still better as whole-file compute
1769
+ — `f.group_sum(...)`, `f.filter(...)`, any read method directly on the file.)
1770
+
1771
+ ```python
1772
+ from east.serialization.beast2 import open_beast2_file
1773
+
1774
+ with open_beast2_file("orders.beast2") as orders: # Dict<String, Order>; Order = Struct{id, total}
1775
+ order = orders[order_id] # fence search → ONE segment decode
1776
+ hot = orders.get_keys(wanted_ids, # each owning segment decodes once;
1777
+ lambda b, k: {"id": k, "total": 0.0}) # `fill` is a body (a python dict
1778
+ if candidate_id in orders: # captured from outside is refused)
1779
+ ...
1780
+
1781
+ # A sorted Array file answers range queries the same way — global insertion
1782
+ # indices, then slice exactly the covered rows:
1783
+ with open_beast2_file("events.beast2") as events: # Array<Event> sorted by timestamp
1784
+ span = events.find_sorted_range(cutoff_event)
1785
+ window = events.slice(span["start"], span["end"])
1786
+ ```
1787
+
1788
+ ### Sequential logic that stays in east-c (worklist / replay)
1789
+
1790
+ When the next step depends on the last — a worklist, a BFS, a fixpoint, a
1791
+ topological replay — no `map`/`filter`/`group_reduce` expresses it, and a
1792
+ python loop over decoded rows is the whole job's cost. The statement surface
1793
+ threads the state through the loop natively, so the whole thing is ONE
1794
+ compiled function. Two rules carry most of the weight. **Accumulate in
1795
+ place** — a collection threaded through a state is rebuilt every iteration,
1796
+ so `order.concat(...)` is O(n²) over the loop where `order.push_last(...)` is
1797
+ O(1). And **a mutation is a statement** — `b.do(acc.push_last(x))` in a
1798
+ statement body, `East.block(acc.push_last(x), result)` in an expression body;
1799
+ a bare `acc.push_last(x)` line is evaluated at build time and thrown away
1800
+ (the build raises rather than compile a loop that silently does nothing).
1801
+
1802
+ ```python
1803
+ from east import ArrayType, DictType, East, IntegerType, StringType
1804
+
1805
+ Node, Edges = StringType, DictType(StringType, ArrayType(StringType))
1806
+ Indeg = DictType(StringType, IntegerType)
1807
+
1808
+ # Kahn's algorithm — ONE compiled function, the TypeScript `$` shape: a
1809
+ # worklist the loop appends to, a cursor into it, and per-node successors
1810
+ # decremented in place.
1811
+ @East.function([ArrayType(Node), Edges, Indeg], ArrayType(Node))
1812
+ def topo_order(b, roots, succ, indeg):
1813
+ ready = b.const(roots.copy()) # task inputs arrive frozen: copy
1814
+ remaining = b.const(indeg.copy())
1815
+ order = b.const(East.new_array(Node))
1816
+ i = b.let(0)
1817
+
1818
+ def step(b, label):
1819
+ node = b.const(ready.get(i))
1820
+ b.do(order.push_last(node))
1821
+
1822
+ def visit(b, v):
1823
+ b.do(remaining.insert_or_update(v, -1, lambda _b, old, d: old + d))
1824
+ b.if_(remaining.get(v) == 0, lambda b: b.do(ready.push_last(v)))
1825
+
1826
+ b.for_(succ.get_or_default(node, East.new_array(Node)), visit)
1827
+ b.assign(i, i + 1)
1828
+
1829
+ b.while_(i < ready.size(), step)
1830
+ return order
1831
+
1832
+ @East.platform_function(inputs=[ArrayType(Node), Edges, Indeg], output=ArrayType(Node))
1833
+ def replay_order(roots, succ, indeg):
1834
+ return topo_order(roots, succ, indeg)
1835
+ ```
1836
+
1837
+ The same algorithm as EXPRESSION forms — a state struct threaded through
1838
+ `East.while_`/`East.for_`, the shape that fits inside a one-expression
1839
+ callback:
1840
+
1841
+ ```python
1842
+ topo_order = East.function(
1843
+ [ArrayType(Node), Edges, Indeg], ArrayType(Node),
1844
+ lambda b, roots, succ, indeg: East.while_(
1845
+ {"ready": roots.copy(), "indeg": indeg.copy(),
1846
+ "order": East.new_array(Node), "i": 0},
1847
+ cond=lambda b, s: s.i < s.ready.size(),
1848
+ body=lambda b, s: East.let(s.ready.get(s.i), lambda b, node: East.block(
1849
+ s.order.push_last(node),
1850
+ East.for_(
1851
+ succ.get_or_default(node, East.new_array(Node)),
1852
+ {**s, "i": s.i + 1}, # inner loop's state
1853
+ lambda b, t, v: East.block(
1854
+ t.indeg.insert_or_update(v, -1, lambda b, old, d: old + d),
1855
+ East.if_else(t.indeg.get(v) == 0, # newly ready?
1856
+ East.block(t.ready.push_last(v), t),
1857
+ t))))), # else: unchanged
1858
+ ).order)
1859
+
1860
+ # any carried state — a forward fill is a `for_` whose state remembers the last value
1861
+ forward_fill = East.function(
1862
+ [ArrayType(StringType)], ArrayType(StringType),
1863
+ lambda b, cells: East.for_(
1864
+ cells, {"last": "", "out": East.new_array(StringType)},
1865
+ lambda b, s, cell: East.let(
1866
+ East.if_else(cell == "", s.last, cell),
1867
+ lambda b, v: East.block(s.out.push_last(v), {"last": v, "out": s.out}))).out)
1868
+ ```
1869
+
1870
+ To stop early, put `East.break_(state)` in an `if_else` arm — the state
1871
+ argument commits before the jump, so the answer survives; `East.label(...)` on
1872
+ an outer loop lets an inner one break all the way out. Reach for all of this
1873
+ only when the work is genuinely sequential: a `group_reduce` or a `reduce` is
1874
+ both shorter and faster when it fits.
1875
+
1876
+ ### Trees from flat rows, and recursion
1877
+
1878
+ `to_tree` builds nested data — any depth, your own node type — from rows that
1879
+ name their key and their parent's key. `build` runs once per row, children
1880
+ first; the result is the roots in source order. The walk is iterative, in
1881
+ east-c, on values and in a body alike:
1882
+
1883
+ ```python
1884
+ from east import ArrayType, OptionType, StringType, StructType, recursive_type
1885
+
1886
+ Row = StructType([("id", StringType), ("parent", OptionType(StringType)), ("name", StringType)])
1887
+ Node = recursive_type(lambda self: StructType([("name", StringType), ("children", ArrayType(self))]))
1888
+
1889
+ roots = rows.to_tree(Node,
1890
+ lambda b, r: r.id,
1891
+ lambda b, r: r.parent,
1892
+ lambda b, r, _i, children: {"name": r.name, "children": children})
1893
+ ```
1894
+
1895
+ - A row whose parent is `none` is a root, and so is an orphan (its parent key
1896
+ is not in the rows). A repeated key ❗ `toTree: duplicate key <key>`; a cycle
1897
+ ❗ `toTree: cycle through key <key>` — both before any `build`.
1898
+ - A non-recursive `Node` folds bottom-up: `IntegerType` with
1899
+ `children.sum() + 1` is each root's subtree size. A Set's element and a
1900
+ Dict's key are their own keys, so those take no `key`.
1901
+ - **Recursion** — a function calling itself through a captured variable — is
1902
+ for shallow depths, in the hundreds. Deeper, a call ❗ `call stack exhausted:
1903
+ East calls nested too deeply`, a catchable `EastError` on every runtime. Walk
1904
+ deep data with `b.while_` or `to_tree`; a recursive VALUE's depth is still
1905
+ bounded, at tens of thousands of levels, by east-c's recursive collect and
1906
+ free.
1907
+
1908
+ ### Sort uses East's total order
1909
+
1910
+ ```python
1911
+ # WRONG — Python's default ordering (incorrect for floats/NaN, mixed, type-specific)
1912
+ sorted(list(arr))
1913
+
1914
+ # CORRECT — East total order, in east-c (the TypeScript names)
1915
+ arr.sort() # new array
1916
+ arr.sort_in_place() # in place
1917
+ arr.sort(lambda b, r: r["x"]) # by a projection, still East-ordered
1918
+ ```
1919
+
1920
+ ### Scalars: use the `East.<Type>` utilities for consistency — above all String & DateTime
1921
+
1922
+ East scalars *are* Python scalars, so Python operators run on them — but the
1923
+ `East.<Type>` namespaces (and `East.less`/`compare`/`equal`) are the standardised
1924
+ ops that give the **same** answer in Python, C, TS, and cached e3 tasks. Python's
1925
+ `str` methods, `re`, `datetime`, `strftime`, `//`, and `<` drift by
1926
+ engine / locale / timezone; the East functions do not. Use them for anything with
1927
+ divergent semantics — and **always** for strings and dates.
1928
+
1929
+ ```python
1930
+ # String — East.String.*, not Python str/re: split edge cases, the regex engine,
1931
+ # trim's whitespace set, and JSON encoding all differ from Python
1932
+ East.String.split("a,b,c", ",") # East semantics, not str.split
1933
+ East.String.regex_replace(s, r"\d+", "#") # East regex engine, not Python re
1934
+ East.String.trim(s) # East's whitespace set
1935
+ East.String.print_json(IntegerType, 5) # '"5"' — Integer is a JSON *string* in East
1936
+
1937
+ # DateTime — East.DateTime.* only. East DateTime is UTC; tokens are Day.js, not strftime
1938
+ East.DateTime.add_milliseconds(dt, 86_400_000) # +1 day, not dt + timedelta(days=1)
1939
+ East.DateTime.duration_milliseconds(a, b) # a − b (ms), standardised
1940
+ East.DateTime.get_day_of_week(dt) # Monday == 1, not dt.weekday() (== 0)
1941
+ East.DateTime.print_formatted(dt, "YYYY-MM-DD HH:mm:ss") # Day.js tokens, not dt.strftime("%Y-…")
1942
+ East.Float.print_currency(total) # "$1,234.57" — the stdlib, same in TS
1943
+
1944
+ # Ordering / equality — East total order (NaN-correct, mixed types, variants-by-name)
1945
+ East.less(FloatType, a, b) # not a < b / sorted(...)
1946
+ # Integer is i64 — divide truncates; Python's unbounded int can overflow i64
1947
+ East.Integer.divide(7, 2) # 3
1948
+ # (you also can't method-call a Python float/str — the ops live on the namespace)
1949
+ East.Float.sqrt(2.0) # not (2.0).sqrt()
1950
+ ```
1951
+
1952
+ ### torch interop (in a torch-having package, inline — no helpers)
1953
+
1954
+ ```python
1955
+ import torch, numpy as np
1956
+ t = mat.to_torch() # writeable torch copy of the buffer
1957
+ out = EastMatrix(FloatType, model(t).detach().cpu().numpy()) # bridge canonicalizes dtype
1958
+ ```
1959
+
1960
+ ## Sharp edges
1961
+
1962
+ - **Every body takes the block first** — `lambda b, x: …` / `def f(b, x)`, never
1963
+ `lambda x: …` — an `East.function` body, a builtin's callback, a branch, a
1964
+ loop, a `.match` handler alike. A function VALUE (a compiled `East.function`, a
1965
+ `.bind` result) takes none. Name the block `_b` when unused.
1966
+ - **Type constructors take PAIRS, not a dict** (unlike TypeScript):
1967
+ `StructType([("name", StringType), ("price", FloatType)])`,
1968
+ `VariantType([("ok", T), ("err", E)])`.
1969
+ - **A `@East.platform_function` must return an East value.** Plain python (a
1970
+ `dict`, a `list` of dicts) fails output validation — build it with
1971
+ `array`/`struct`/`variant`, or `coerce_to(raw, OutputType)` at the return.
1972
+ - **One name, two surfaces.** A body holds EXPRESSIONS; what a platform function
1973
+ is handed, or iteration yields, is the eager surface — the same vocabulary,
1974
+ options included: read a decoded option with `.unwrap_or(d)` / `.match({…})` /
1975
+ `.is_some()`, never a hand-rolled `opt.type == "some"` branch.
1976
+ - **The two `duration` spellings differ in sign**: the METHOD `a.duration_days(b)`
1977
+ is `b − a` (the TypeScript method); `East.DateTime.duration_days(a, b)` is the
1978
+ raw builtin, `a − b`.
1979
+ - **`EastDict.get(k)` errors on a missing key**, like `d[k]` and TypeScript;
1980
+ `get(k, default)` is the python convenience, `get(k, lambda b, k: …)`
1981
+ TypeScript's `onMissing` body.
1982
+ - **`append` takes an ARRAY**; one element is `push_last`. A Set or Dict spells
1983
+ the array flattening `flatten_to_array`; only an Array has `flat_map`.
1984
+ - **Task inputs arrive frozen**: mutating one raises `cannot mutate a frozen
1985
+ value (task inputs are immutable) — copy first`; `.copy()` gives a mutable
1986
+ value. Keyed gets and iteration on a paged input stay O(segment) through the
1987
+ proxy, and frozen collections compare by value under `Is`. The same frozen
1988
+ paged value comes from `blob.open_beast(T)` and from `fs_open_beast` in
1989
+ **east-py-std**; an input e3 stages as a manifest directory opens the same
1990
+ way, reading only the segment files a read lands in.
1991
+ - **A callback East cannot express RAISES**, eager paths included, naming the
1992
+ binding — a body reaching for python (`random.…`, `len`, a mutable capture,
1993
+ `nonlocal`) or one that only looks East-native (an f-string, which would
1994
+ constant-fold the proxy; an off-surface method). Build strings with
1995
+ `East.str(...)` or `+`; genuine python work is an explicit loop outside.
1996
+ - **Read a captured East collection with `.get(expr)` /
1997
+ `.get_or_default(expr, d)` / `.try_get(expr)`** — `[expr]` on a captured
1998
+ constant does not build IR (python coerces the index via `__index__`).
1999
+ - **Genuinely-python loops cross the boundary once** — `to_columns()` /
2000
+ `EastArray.from_columns` / `map_batches`, never a platform call or a decode per
2001
+ element.
2002
+
2003
+ ## Related skills
2004
+
2005
+ - **east** — the TypeScript `East.function` DSL, this surface's twin name for
2006
+ name (`$` is `b`, camelCase is snake_case): one type system and one IR, a
2007
+ program prints from one into the other (`east-py transpile` / `east-node
2008
+ transpile`), and a function exported from one is imported by the other
2009
+ (`east-py export-functions` ↔ `East.importFunction`).
2010
+ - **east-py-datascience** — python platform functions for ML and optimisation
2011
+ (XGBoost, LightGBM, Optuna, MADS, PyMC, SHAP, Torch, GoogleOR, Simulation): the
2012
+ home once a platform-function POC needs a real model or solver.
2013
+ - **east-py-std** / **east-py-io** — the platform functions of the python runtime
2014
+ (Console/FileSystem/Fetch/Crypto/Time/Random and the strict large-JSON reader;
2015
+ SQL/NoSQL/S3/FTP/SFTP/XLSX/XML/compression), each exported under its own name
2016
+ and callable with East values or inside a body. Their TypeScript siblings are
2017
+ **east-node-std** / **east-node-io**.
2018
+ - **e3** — run East functions as durable, content-addressed dataflow tasks; wire
2019
+ a project's python module with a `{ custom: 'platform_module' }` platform.
2020
+ - **east-project** — scaffold (`--platform`) and package a project-owned
2021
+ platform module; **e3-create** — a dedicated python platform package
2022
+ (`--python-packages=<name>`, a uv workspace member with its own derived e3
2023
+ environment); **east-design** — start there with a goal but no architecture.