@elaraai/east-codex-plugin 0.0.0-stage → 1.0.90

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,301 @@
1
+ ---
2
+ name: east-py-std
3
+ description: "Standard platform functions for the East language on the Python runtime - console, environment variables, filesystem, HTTP fetch, crypto, time, path, random, large-JSON reading, testing. Use when writing Python (not the TypeScript DSL) that calls or registers these platform functions. Triggers for: (1) Calling east_py_std functions (env_get, fs_read_file, fetch_request, crypto_uuid, random_normal, ...) from python or inside an East.function body — the same object does both, (2) Registering the east_py_std platform list with compile() so East programs can use Console/Env/FileSystem/Fetch/Crypto/Time/Path/Random/Json/Test on the Python runtime, (3) Building FetchRequestConfigType requests in Python, (4) Deterministic random streams with random_seed, (5) Reading a JSON document too large to decode whole — ingesting a multi-gigabyte payload from another system — with json_open / json_more / json_next, strictly against a published… See the detailed scope below."
4
+ ---
5
+
6
+ ## Detailed skill scope
7
+
8
+ Standard platform functions for the East language on the Python runtime - console, environment variables, filesystem, HTTP fetch, crypto, time, path, random, large-JSON reading, testing. Use when writing Python (not the TypeScript DSL) that calls or registers these platform functions. Triggers for: (1) Calling east_py_std functions (env_get, fs_read_file, fetch_request, crypto_uuid, random_normal, ...) from python or inside an East.function body — the same object does both, (2) Registering the east_py_std platform list with compile() so East programs can use Console/Env/FileSystem/Fetch/Crypto/Time/Path/Random/Json/Test on the Python runtime, (3) Building FetchRequestConfigType requests in Python, (4) Deterministic random streams with random_seed, (5) Reading a JSON document too large to decode whole — ingesting a multi-gigabyte payload from another system — with json_open / json_more / json_next, strictly against a published contract. For authoring East programs in TypeScript against these functions, use the east-node-std skill instead.
9
+
10
+ # East.py Standard Platform Functions
11
+
12
+ `east_py_std` is the Python implementation of the East standard platform:
13
+ console, environment variables, filesystem, HTTP fetch, crypto, time, path,
14
+ random, large-JSON reading and testing. Every function is exported under its
15
+ platform name (`fs_read_file`, `fetch_get`, …) and is **dual-mode**: a plain
16
+ python callable over East values — call it from your own
17
+ `@East.platform_function` — and, the same object, callable inside an
18
+ `East.function` body, where the call IS the `Platform` node with the
19
+ function's declared signature. Nothing restates it: register the `platform`
20
+ list at `East.compile` (or let the runner) and it runs.
21
+
22
+ This skill is for **Python**. To author East programs in TypeScript against
23
+ these functions (`Console.log`, `FileSystem.readFile`, …), load
24
+ **east-node-std** — the same functions, under the same platform names.
25
+
26
+ ## Before writing code — search the example index
27
+
28
+ Every East API has a tested example in the plugin's index — the index IS the
29
+ API reference, printed from each example's IR in TypeScript or python. Before
30
+ writing or changing East code:
31
+
32
+ 1. Call `search_east_examples` for each capability you
33
+ are about to use — `language: "python"` for east-py, `"typescript"`
34
+ otherwise. Summaries come back first: id, signature, the inputs and the
35
+ expected result, a few hundred bytes each.
36
+ 2. Fetch the one or two that match with `get_east_example`
37
+ and pattern your code on them.
38
+ 3. Do not read `node_modules/@elaraai/**` or `*.examples.ts` files wholesale,
39
+ and do not reason from `.d.ts` signatures: the index holds the same
40
+ programs, exact and far cheaper, and the signatures omit the runtime rules
41
+ that make East code correct.
42
+
43
+ Nothing is injected for you; the search is the step.
44
+
45
+ ## Quick Start
46
+
47
+ ```python
48
+ from east import ArrayType, East, IntegerType, StringType
49
+ from east_py_std import fs_read_directory, fs_read_file, fs_read_file_bytes, platform
50
+
51
+ # Direct calls — East values in, East values out, no IR round trip
52
+ @East.platform_function(inputs=[StringType], output=IntegerType)
53
+ def file_count(directory):
54
+ return fs_read_directory(directory).size()
55
+
56
+ # The same functions inside an East body: the call is the Platform node, nothing
57
+ # restates the signature — compile with the package's list
58
+ first_lines = East.function([StringType], ArrayType(StringType), lambda b, directory:
59
+ fs_read_directory(directory).map(
60
+ lambda b, name: fs_read_file(East.str(directory, "/", name)).split("\n").get(0)))
61
+ East.compile(first_lines, platform=platform)("reports")
62
+
63
+ size = East.function([StringType], IntegerType, lambda b, path: fs_read_file_bytes(path).size())
64
+ East.compile(size, platform=platform)("data.bin")
65
+ ```
66
+
67
+ An EAGER callback — `names.map(lambda b, name: fs_read_file(…))` on a value
68
+ in python — cannot call a platform function: it compiles with no platform, so
69
+ the capture refuses it (`it references fs_read_file, which has no East form`).
70
+ Put the loop in an East function compiled with `platform`, as above.
71
+
72
+ ## Decision Tree: What Do You Need?
73
+
74
+ ```
75
+ Task → What do you need?
76
+ │
77
+ ├─ Console → console_log(msg) · console_error(msg) · console_write(msg) (no newline)
78
+ │
79
+ ├─ Environment (runtime credentials and config — never literals in source)
80
+ │ └─ env_get(name) -> Option<String> (some when set, none when not)
81
+ │
82
+ ├─ Filesystem
83
+ │ ├─ Text → fs_read_file(path) · fs_write_file(path, text) · fs_append_file(path, text)
84
+ │ ├─ Bytes → fs_read_file_bytes(path) -> Blob · fs_write_file_bytes(path, blob)
85
+ │ ├─ A huge beast2 collection file → fs_open_beast(T, path) in a body (FileSystem.openBeast's twin, the type FIRST):
86
+ │ │ a FROZEN paged value over a mapping — size / keyed reads / iteration decode one segment
87
+ │ ├─ Inspect → fs_exists · fs_is_file · fs_is_directory · fs_read_directory(path) -> Array<String>
88
+ │ └─ Manage → fs_create_directory · fs_delete_file
89
+ │
90
+ ├─ JSON too large to decode whole
91
+ │ ├─ Open → json_open(path, pointer) · json_open_text(text, pointer) -> a handle (a String)
92
+ │ │ (pointer is RFC 6901: "" for the whole document, "/data" for an envelope's array)
93
+ │ ├─ Iterate → json_more(handle) then json_next(T, handle) in a body — the type FIRST;
94
+ │ │ from python the factory: json_next(None, T)(handle)
95
+ │ ├─ One subtree → json_value(T, path, pointer) — the small members beside a huge array
96
+ │ └─ Release → json_close(handle)
97
+ │
98
+ ├─ HTTP (ASYNC: await a direct call; in a body, East.asyncFunction + East.compileAsync)
99
+ │ ├─ Convenience → fetch_get(url) -> String · fetch_get_bytes(url) -> Blob · fetch_post(url, body) -> String
100
+ │ └─ Full control → fetch_request(FetchRequestConfigType) -> FetchResponseType
101
+ │
102
+ ├─ Crypto → crypto_uuid() · crypto_random_bytes(n) -> Blob · crypto_hash_sha256(text) -> hex String ·
103
+ │ crypto_hash_sha256_bytes(blob) -> Blob (the 32-byte digest)
104
+ │
105
+ ├─ Time → time_now() -> Integer (Unix milliseconds) · time_sleep(ms) (async) ·
106
+ │ time_get_timezone_offset(dt, zone) -> Integer (minutes ahead of UTC for an IANA zone at that instant)
107
+ │
108
+ ├─ Path → path_join(parts: Array<String>) · path_resolve(path) (absolute) · path_dirname(path) ·
109
+ │ path_basename(path) · path_extname(path)
110
+ │
111
+ ├─ Random (one stream; seed it for reproducibility)
112
+ │ ├─ Seed → random_seed(seed)
113
+ │ ├─ Uniform → random_uniform() in [0, 1) · random_range(min, max) -> Integer in [min, max]
114
+ │ ├─ Continuous → random_normal() (N(0, 1)) · random_log_normal(mu, sigma) · random_exponential(rate) ·
115
+ │ │ random_weibull(shape) · random_pareto(alpha) · random_bates(n) · random_irwin_hall(n)
116
+ │ └─ Discrete (-> Integer) → random_bernoulli(p) · random_binomial(n, p) · random_geometric(p) · random_poisson(rate)
117
+ │
118
+ └─ Testing (the harness the compliance runner overrides with its own) → the platform functions
119
+ testPass / testFail(msg) / test(name, body) / describe(name, body); in python test_pass · test_fail ·
120
+ test_impl_fn · describe, with reset_counters()
121
+ ```
122
+
123
+ `platform` is every function; each family's list is exported too
124
+ (`console_impl`, `env_impl`, `fs_impl`, `fetch_impl`, `crypto_impl`,
125
+ `time_impl`, `path_impl`, `random_impl`, `json_impl`, `test_impl`) when a
126
+ program should get only some of them.
127
+
128
+ ## East Type Definitions
129
+
130
+ | Type | Shape |
131
+ |------|-------|
132
+ | `FetchMethodType` | `Variant<GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS>` (Null payloads) |
133
+ | `FetchRequestConfigType` | `Struct{url: String, method: FetchMethodType, headers: Dict<String, String>, body: Option<String>}` |
134
+ | `FetchResponseType` | `Struct{status: Integer, statusText: String, headers: Dict<String, String>, body: String, ok: Boolean}` |
135
+
136
+ All three are importable from `east_py_std`. Build a request with
137
+ `coerce_to({...}, FetchRequestConfigType)` — the method with
138
+ `variant("POST", None, FetchMethodType)`, the body with `some(text)` or `none`
139
+ (a plain string is not an Option).
140
+
141
+ ## Key Patterns
142
+
143
+ ### Full-control HTTP request
144
+
145
+ ```python
146
+ import asyncio
147
+ from east import coerce_to, some, variant
148
+ from east_py_std import FetchMethodType, FetchRequestConfigType, fetch_request
149
+
150
+ request = coerce_to({
151
+ "url": "https://api.example.com/items",
152
+ "method": variant("POST", None, FetchMethodType),
153
+ "headers": {"content-type": "application/json"},
154
+ "body": some('{"name": "widget"}'),
155
+ }, FetchRequestConfigType)
156
+ response = asyncio.run(fetch_request(request)) # a direct call returns the coroutine
157
+ response["status"], response["body"] # plain int / str
158
+ ```
159
+
160
+ Inside a body the fetch functions need `East.asyncFunction` and
161
+ `East.compileAsync` (a sync body refuses them at build time). Run such a
162
+ program through the runner — `east-py run`, an e3 task: the bridge runs async
163
+ implementations on its own event loop, so the compiled coroutine cannot be
164
+ awaited under `asyncio.run` once it calls one.
165
+
166
+ ### Deterministic random streams
167
+
168
+ ```python
169
+ from east_py_std import random_normal, random_range, random_seed
170
+
171
+ random_seed(42) # same seed -> same draws
172
+ noise = random_normal() # N(0, 1); scale and shift it yourself
173
+ die = random_range(1, 6)
174
+ ```
175
+
176
+ ### Open a huge beast2 collection file lazily
177
+
178
+ `fs_open_beast` is the std family's one generic platform function — the
179
+ implementation behind `FileSystem.openBeast(T, path)` on every runtime. In a
180
+ body it reads as TypeScript does, the type argument first. The value is a
181
+ frozen paged proxy (the value a large task input opens as): size, keyed reads
182
+ and iteration decode one segment from a mapping of the file, mutation raises
183
+ `cannot mutate a frozen value`, and a file whose header carries another type
184
+ raises `Failed to open beast file <path>: beast2: cannot open a blob of type
185
+ <wire> as <T>`.
186
+
187
+ ```python
188
+ from east import DictType, East, IntegerType, StringType, StructType
189
+ from east_py_std import fs_open_beast, platform
190
+
191
+ Table = DictType(IntegerType, StructType([("id", IntegerType), ("name", StringType)]))
192
+
193
+ name_of = East.function([StringType], StringType,
194
+ lambda b, path: fs_open_beast(Table, path).get(7).name)
195
+ East.compile(name_of, platform=platform)("rows.beast2")
196
+ ```
197
+
198
+ From python it is the factory — `fs_open_beast(None, Table)` returns the
199
+ opener, and `opener(path)` a HOLD, not a python-readable value: pass it into a
200
+ compiled call, `.bind` it, or return it from your own
201
+ `@East.platform_function` declared with that type, and it crosses by pointer.
202
+ To read a file in python, use `open_beast2_file` (the **east-py** skill), whose
203
+ value has the whole read surface. An index-less file (what
204
+ `East.Blob.encode_beast` writes) decodes whole, frozen; for bytes in hand use
205
+ `blob.open_beast(T)`. The value keeps its mapping for as long as it lives:
206
+ don't hold thousands open, and don't truncate or rewrite a file under one.
207
+
208
+ ### Read a JSON document too large to decode whole
209
+
210
+ `json_open` positions a reader on the array or object an RFC 6901 pointer
211
+ names; `json_next` reads ONE element against a type. `json_next` and
212
+ `json_value` are generic, so they read as `fs_open_beast` does — the type
213
+ first in a body, the factory from python.
214
+
215
+ ```python
216
+ from east import East, IntegerType, StringType, StructType
217
+ from east_py_std import json_close, json_more, json_next, json_open_text, platform
218
+
219
+ Row = StructType([("id", IntegerType)])
220
+
221
+ # From python — the factory: (platform list, T) -> read(handle)
222
+ handle = json_open_text('[{"id":"1"},{"id":"2"}]', "")
223
+ read = json_next(None, Row)
224
+ rows = []
225
+ while json_more(handle):
226
+ rows.append(read(handle))
227
+ json_close(handle) # [{'id': 1}, {'id': 2}]
228
+
229
+ # In a body — the call itself, the type argument first
230
+ def body(b, text):
231
+ h = b.let(json_open_text(text, ""))
232
+ acc = b.let(0)
233
+ b.while_(json_more(h), lambda b: b.assign(acc, acc + json_next(Row, h)["id"]))
234
+ b.do(json_close(h))
235
+ return acc
236
+
237
+ summed = East.function([StringType], IntegerType, body)
238
+ East.compile(summed, platform=platform)('[{"id":"10"},{"id":"20"}]') # 30
239
+ ```
240
+
241
+ - **`{"meta": {…}, "data": [10M rows]}` is the ordinary shape.** Never type
242
+ the whole document as one value — a `Struct` holding the array materialises
243
+ it however good the reader is. Point at the array, and read the envelope
244
+ with `json_value(MetaType, path, "/meta")`; a member AFTER the array costs a
245
+ scan, not a parse.
246
+ - **It is strict: it accepts exactly what `json_schema_for(T)` describes**, so
247
+ a producer validating against the published schema cannot send what is then
248
+ rejected. An `Integer` is a quoted i64 decimal — not `"0x10"`, `"0b101"`,
249
+ `" 7 "`, `"007"`, `"-0"`, nor a bare number. A `DateTime` is any RFC 3339
250
+ date-time (`Z` or any offset, read as the UTC instant; `t`/`z` in either case;
251
+ any fractional digits, past the millisecond dropped; a leap second as the
252
+ Unix time its fields add up to); a day its month lacks (`2026-02-30`) or a
253
+ year outside 0001–9999 is refused, and `parse_json` reads DateTimes through
254
+ the same parser. A `Blob` is lowercase hex (`parse_json` takes either case).
255
+ - **An `Option<T>` is `null` or `T`'s own encoding** wherever `T` can never
256
+ encode as `null` (`"note": null` → none, `"note": "x"` → some), and the tagged
257
+ `{"type": "some", "value": "x"}` there is refused (`expected a String, got an
258
+ object`). Only `Option<Null>` and `Option<Option<T>>` keep the tagged object,
259
+ so a bare `null` there is refused (`expected an object, got null`). The same
260
+ on every runtime, nothing to configure; `json_schema_for` describes a flat
261
+ Option as `oneOf [null, T]`.
262
+ - **Errors name the offending node** by RFC 6901 pointer (`~` and `/` escaped
263
+ as `~0` / `~1`): `json_next: /1/id: "not-an-integer" is not a 64-bit integer
264
+ in East JSON's form` — the text `east-node-std` and `east-c-std` produce,
265
+ pinned by the shared compliance corpus; a quoted value is clipped at 200
266
+ characters.
267
+ - **`json_more` is a predicate, `json_next` advances** — they need not
268
+ alternate. **A JSON object iterates as entries**: a `Struct` of exactly `key`
269
+ (a `String`) and `value`, in either order, which a `Dict` output needs; the
270
+ entry type is checked before anything is consumed, so a refused call leaves
271
+ the reader where it was. Handles are held until closed, like a database
272
+ connection; `json_open` maps the file rather than reading it, Windows
273
+ included.
274
+ - **The reader is east-c's**, reached through the Cython bridge, so the accepted
275
+ forms, the 2048 nesting bound, UTF-8 validation (a malformed sequence is
276
+ `invalid UTF-8 in string`, never repaired), surrogate-pair joining and the
277
+ error text are shared with `east-c-std`. What is skipped is still JSON: a
278
+ fault before the pointer target or inside an unmodelled field is refused at
279
+ open with a read's text. A `Float` reads the same under any locale.
280
+ - **What the schema cannot say**: a `DateTime` instant must fall in years
281
+ 0001–9999 and loses digits past the millisecond; a `Ref` the encoder wrote as
282
+ `{"$ref": ...}` for a repeated target is not readable; a `Dict` whose entries
283
+ repeat a key satisfies `uniqueItems` and is still refused; a `Variant` must
284
+ carry `"type"` before `"value"` (struct fields may come in any order).
285
+
286
+ Scalars cross as plain python (`str`/`int`/`float`/`bool`/`datetime`); a
287
+ `Blob` is an `EastBlob`, an `Array<String>` an `EastArray` with eager methods —
288
+ the **east-py** skill has the value API.
289
+
290
+ ## Related skills
291
+
292
+ - **east-py** — the python runtime: East values as plain data, eager methods,
293
+ `coerce_to`, `open_beast2_file`, and the `@East.platform_function` on-ramp
294
+ these calls live inside (and the dual-mode rule that makes them callable in a
295
+ body).
296
+ - **east-py-io** — the I/O sibling: SQL/NoSQL databases, S3, FTP/SFTP,
297
+ XLSX/XML, compression.
298
+ - **east-py-datascience** — ML and optimisation platform functions.
299
+ - **east-node-std** — the TypeScript authoring surface for the same functions.
300
+ - **e3** — the execution engine whose python runner registers this list for
301
+ dataflow tasks.