@elaraai/east-codex-plugin 0.0.0-stage → 1.0.89

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,280 @@
1
+ ---
2
+ name: east-py-datascience
3
+ description: "Data science and machine learning platform functions for the East language (TypeScript types + directly-callable Python implementations). Use when writing East programs that need optimization (MADS, Optuna, SimAnneal, ALNS, Scipy, Optimization, GoogleOr), machine learning (XGBoost, LightGBM, NGBoost, Torch MLP, Lightning, GP), Bayesian inference (PyMC), causal inference (Causal: DoWhy, EconML DML, ALE), simulation (Simulation DES), ML utilities (Sklearn preprocessing, metrics, splits), conformal prediction (MAPIE), or model explainability (SHAP). Triggers for: (1) Writing East programs with @elaraai/east-py-datascience, (2) Derivative-free optimization with MADS, (3) Bayesian optimization with Optuna, (4) Discrete/combinatorial optimization with SimAnneal or ALNS, (5) Gradient boosting with XGBoost or LightGBM, (6) Probabilistic predictions with NGBoost or GP, (7) Neural networks with Torch MLP or Lightning, (8) Data… See the detailed scope below."
4
+ ---
5
+
6
+ ## Detailed skill scope
7
+
8
+ Data science and machine learning platform functions for the East language (TypeScript types + directly-callable Python implementations). Use when writing East programs that need optimization (MADS, Optuna, SimAnneal, ALNS, Scipy, Optimization, GoogleOr), machine learning (XGBoost, LightGBM, NGBoost, Torch MLP, Lightning, GP), Bayesian inference (PyMC), causal inference (Causal: DoWhy, EconML DML, ALE), simulation (Simulation DES), ML utilities (Sklearn preprocessing, metrics, splits), conformal prediction (MAPIE), or model explainability (SHAP). Triggers for: (1) Writing East programs with @elaraai/east-py-datascience, (2) Derivative-free optimization with MADS, (3) Bayesian optimization with Optuna, (4) Discrete/combinatorial optimization with SimAnneal or ALNS, (5) Gradient boosting with XGBoost or LightGBM, (6) Probabilistic predictions with NGBoost or GP, (7) Neural networks with Torch MLP or Lightning, (8) Data preprocessing and metrics with Sklearn, (9) Conformal prediction intervals with MAPIE, (10) Model explainability with Shap, (11) Iterative coordinate descent with Optimization, (12) Constraint programming, vehicle routing, LP/MIP, or graph algorithms with GoogleOr, (13) Bayesian regression, hierarchical models, and multi-layer estimation with PyMC, (14) Economic ontology simulation via discrete event simulation with Simulation, (15) One declarative causal experiment — naive vs adjusted effect, overlap, robustness, and an honesty verdict — with Causal.experiment, (16) Calling the east_py_datascience functions (xgboost_train_regressor, mads_optimize, …) from a project's own Python @East.platform_function, or inside an East.function body — the same object does both.
9
+
10
+ # East Data Science
11
+
12
+ Data science and machine learning platform functions for East: optimisation,
13
+ ML models, Bayesian and causal inference, simulation, preprocessing and
14
+ explainability. This is the platform's **Reason** layer — a model earns its
15
+ place by improving a decision (a forecast or optimisation that feeds a
16
+ `decision`), so wire its outputs toward one. The functions run on the python
17
+ runtime only: an e3 task needs a python runner (see Related skills).
18
+
19
+ ## Before writing code — search the example index
20
+
21
+ Every East API has a tested example in the plugin's index — the index IS the
22
+ API reference, printed from each example's IR in TypeScript or python. Before
23
+ writing or changing East code:
24
+
25
+ 1. Call `search_east_examples` for each capability you
26
+ are about to use — `language: "python"` for east-py, `"typescript"`
27
+ otherwise. Summaries come back first: id, signature, the inputs and the
28
+ expected result, a few hundred bytes each.
29
+ 2. Fetch the one or two that match with `get_east_example`
30
+ and pattern your code on them.
31
+ 3. Do not read `node_modules/@elaraai/**` or `*.examples.ts` files wholesale,
32
+ and do not reason from `.d.ts` signatures: the index holds the same
33
+ programs, exact and far cheaper, and the signatures omit the runtime rules
34
+ that make East code correct.
35
+
36
+ Nothing is injected for you; the search is the step.
37
+
38
+ ## Quick Start
39
+
40
+ ```typescript
41
+ import { East, FloatType, VectorType, none, some } from "@elaraai/east";
42
+ import { MADS } from "@elaraai/east-py-datascience";
43
+
44
+ // Tune feed rate and temperature to minimise a defect rate
45
+ export const tune = East.function([], MADS.Types.ResultType, $ => {
46
+ const objective = $.const(East.function([VectorType(FloatType)], FloatType, ($, x) => {
47
+ const feed = $.let(x.get(0n).subtract(5.0));
48
+ const temp = $.let(x.get(1n).subtract(180.0));
49
+ $.return(feed.multiply(feed).add(temp.multiply(temp).multiply(0.01)));
50
+ }));
51
+ const x0 = $.let(East.Vector.fromArray([8.0, 200.0]));
52
+ const bounds = $.let({ lower: East.Vector.fromArray([1.0, 100.0]), upper: East.Vector.fromArray([10.0, 250.0]) });
53
+ const config = $.let({
54
+ max_bb_eval: some(100n), display_degree: some(0n), direction_type: none,
55
+ initial_mesh_size: none, min_mesh_size: none, seed: some(42n),
56
+ });
57
+ $.return(MADS.optimize(objective, x0, bounds, none, config)); // none: no constraints
58
+ });
59
+ ```
60
+
61
+ ## Decision Tree: Which Module to Use
62
+
63
+ ```
64
+ Task → What do you need?
65
+ │
66
+ ├─ MADS (derivative-free continuous optimisation) → .optimize(objective, x0, bounds, constraints: Option, config)
67
+ ├─ Optuna (Bayesian hyperparameter tuning) → .optimize()
68
+ ├─ SimAnneal (discrete / combinatorial) → .optimize(), .optimizePermutation(), .optimizeSubset()
69
+ ├─ ALNS (adaptive large neighbourhood search) → .optimize([SolutionType], initial, objective, destroys, repairs, config)
70
+ │ — generic over the solution type S: define your own struct
71
+ ├─ Optimization (iterative coordinate descent over integer vectors)
72
+ │ ├─ .iterative(objective, paramSpaces, config) — the whole objective per move
73
+ │ ├─ .iterativeIncremental(elementObjective, paramSpaces, config) — a per-element contribution, recomputed only
74
+ │ │ where a move changed it (coordinate or swap mode)
75
+ │ └─ .iterativeGrouped(groupObjective, paramSpaces, config) — contributions grouped by VALUE (employees, bins,
76
+ │ vehicles): a move from A to B recomputes groups A and B
77
+ ├─ GoogleOr (OR-Tools)
78
+ │ ├─ CP-SAT → .cpsatSolve(), .cpsatSolveAll()
79
+ │ ├─ Routing → .routingSolve() (TSP, CVRP, VRPTW, VRPPD)
80
+ │ ├─ Linear → .linearSolve() (LP, MIP)
81
+ │ └─ Graph → .minCostFlow(), .maxFlow(), .assignment() (dense cost matrix),
82
+ │ .minCostAssignment() (sparse arcs, task capacity, opt-out penalty)
83
+ ├─ Scipy
84
+ │ ├─ Optimisation → .optimizeMinimize(), .optimizeMinimizeQuadratic(), .optimizeDualAnnealing()
85
+ │ ├─ Statistics → .statsDescribe(), .statsPearsonr(), .statsSpearmanr(), .statsPercentile(), .statsPercentileOfScore(),
86
+ │ │ .statsIqr(), .statsMedian(), .statsMad(), .statsRobust()
87
+ │ ├─ Histogram / KDE → .histogram(), .kdeFit(), .kdeEvaluate()
88
+ │ ├─ Curve fitting → .curveFit()
89
+ │ └─ Interpolation → .interpolate1dFit(), .interpolate1dPredict()
90
+ ├─ XGBoost → train .trainRegressor(), .trainClassifier(), .trainQuantile() · predict .predict(), .predictClass(),
91
+ │ .predictProba(), .predictQuantile()
92
+ ├─ LightGBM → train .trainRegressor(), .trainClassifier() · predict .predict(), .predictClass(), .predictProba()
93
+ ├─ NGBoost (probabilistic boosting) → .trainRegressor() · .predict(), .predictDist()
94
+ ├─ Torch (MLP) → train .mlpTrain(), .mlpTrainMulti() · predict .mlpPredict(), .mlpPredictMulti() · embeddings
95
+ │ .mlpEncode(), .mlpDecode()
96
+ ├─ Lightning (PyTorch Lightning)
97
+ │ ├─ .train(X, y, config, masks, group_weights, conditions) · .predict(model, X, masks, conditions)
98
+ │ ├─ Embeddings (autoencoders) → .encode(), .decode(), .decodeConditional()
99
+ │ ├─ Generation (sequential models) → .generateSequence(model, prefix, condition: Option, config{n_steps,
100
+ │ │ temperature (0 = argmax), return_probs}) — the generated steps, not the prefix
101
+ │ ├─ Architectures → mlp · autoencoder · conv1d (temporal) · sequential (LSTM/GRU, temporal) · transformer (temporal)
102
+ │ ├─ Output modes → regression (MSE) · binary (BCE; per-position pos_weights, masks) ·
103
+ │ │ multi_head (N CE heads; per-head class_weights, masks)
104
+ │ └─ condition_dim for conditional generation in the temporal architectures; early stopping, gradient clipping,
105
+ │ epoch callbacks, group_weights
106
+ ├─ GP (Gaussian process regression) → .train() · .predict(), .predictStd()
107
+ ├─ MAPIE (conformal prediction)
108
+ │ ├─ Regression → .trainConformalRegressor(), .trainCQR() · classification → .trainConformalClassifier()
109
+ │ ├─ Predict → .predictInterval(), .predictSet()
110
+ │ └─ For SHAP → .uncertaintyPredictorRegressor(), .uncertaintyPredictorClassifier()
111
+ ├─ Sklearn
112
+ │ ├─ Splitting → .split() (N-way, stratify, overlap, multi_overlap) · overlap filtering → .overlap()
113
+ │ ├─ Scaling → .standardScalerFit/Transform(), .minMaxScalerFit/Transform(), .robustScalerFit/Transform()
114
+ │ ├─ Encoding → .labelEncoderFit/Transform/InverseTransform(), .ordinalEncoderFit/Transform()
115
+ │ ├─ Class weights → .computeClassWeight()
116
+ │ ├─ Metrics → .computeMetrics(), .computeMetricsMulti() · .computeClassificationMetrics(),
117
+ │ │ .computeClassificationMetricsMulti() · .rocAucScore(), .logLoss(), .confusionMatrix()
118
+ │ ├─ Multi-target → .regressorChainTrain(), .regressorChainPredict()
119
+ │ └─ GMM clustering → .gmmFit(), .gmmPredict(), .gmmPredictProba(), .gmmScoreSamples(), .gmmSample(), .gmmBic(),
120
+ │ .gmmAic() · .silhouetteScore()
121
+ ├─ PyMC (Bayesian inference)
122
+ │ ├─ Train → .trainRegression(), .trainHierarchical(), .trainMultiLayer() · predict → .predict(), .predictDistribution()
123
+ │ └─ Posterior → .posteriorSummary(), .posteriorSamples() · diagnostics → .diagnostics(), .posteriorPredictiveCheck()
124
+ ├─ Simulation (economic-ontology simulation by DES) → .run([R, E], initialState, initialEvents, process, config)
125
+ ├─ Causal (one declarative experiment and an honesty verdict) — generic over the row struct: data is an
126
+ │ Array<Struct> (fields = columns), the row type a type argument, config naming columns by field
127
+ │ ├─ Did X change Y, and can I trust it? → .experiment([Row], data, config)
128
+ │ │ (binary treatment; naive vs adjusted effect, confounder balance, propensity overlap, a placebo / E-value
129
+ │ │ robustness check, and a verdict: causal / modest / adjustment_insufficient / non_identifiable_positivity /
130
+ │ │ not_estimable; `adjusted` is none when the engine refuses. DoWhy / EconML / PyALE are internal.)
131
+ │ └─ What real trial would confirm it? → .designValidation([Row], data, config, result, designConfig)
132
+ │ (a randomised-trial recipe: sample size, split options, match-on categories, power curve, rationale)
133
+ │ Honesty caveats: `causal` means robust to the OBSERVED backdoor set and the refuters that ran — NOT correctly
134
+ │ signed, free of reverse causation or of unobserved confounding. overlap.support_strength (refused / thin /
135
+ │ strong vs config.strong_overlap, default 0.55) tempers thin support to modest; opt-in config.evalue_floor folds
136
+ │ a weak E-value (risk-ratio scale) into modest; opt-in config.expected_sign flags an implausibly signed effect
137
+ │ (refutation.expected_sign_ok = some(false), verdict adjustment_insufficient). Clustered designs
138
+ │ (bootstrap.cluster_column) cluster the placebo and the naive CI too.
139
+ └─ Shap (explainability)
140
+ ├─ Create → .treeExplainerCreate() (XGBoost only), .kernelExplainerCreate() (any model)
141
+ ├─ Compute → .computeValues(), .featureImportance()
142
+ └─ KernelExplainer takes XGBoost, LightGBM, NGBoost, GP, Torch, RegressorChain and MAPIE models
143
+ ```
144
+
145
+ ## Data and Types
146
+
147
+ Data is East tensors: features `MatrixType(FloatType)` (rows × features),
148
+ targets `VectorType(FloatType)`, class labels `VectorType(IntegerType)`; the
149
+ predictions come back as vectors. Build them with
150
+ `East.Matrix.fromArray([[…], […]])` and `East.Vector.fromArray([…])` in
151
+ TypeScript, `EastMatrix` / `EastVector` (numpy) in python. The package
152
+ re-exports East's constructors as `VectorType` / `SharedVectorType` and
153
+ `SharedMatrixType`.
154
+
155
+ Each module's own types are under `Module.Types.*` — and at the top level
156
+ under a prefixed name (`MADSConfigType`, `XGBoostModelBlobType`, …):
157
+
158
+ ```typescript
159
+ import { MADS, Optuna, ALNS, Sklearn, XGBoost, ModelBlobType } from "@elaraai/east-py-datascience";
160
+
161
+ MADS.Types.BoundsType // { lower, upper } vectors
162
+ MADS.Types.ConfigType // every field an Option: some(v) / none
163
+ MADS.Types.ConstraintType // variant eb / pb over a constraint function
164
+ MADS.Types.ResultType // { x_best, f_best, success, … }
165
+ Optuna.Types.ParamSpaceType // one parameter's search space
166
+ Optuna.Types.StudyResultType
167
+ ALNS.Types.ConfigType // and ALNS.Types.ResultType, over the solution type
168
+ Sklearn.Types.SplitConfigType
169
+ XGBoost.Types.ModelBlobType // a trained model
170
+ ModelBlobType // the union of every library's model blob, for an arbitrary trained model
171
+ // (AnyModelBlobType is the SHAP explainer's input union)
172
+ ```
173
+
174
+ A model blob round-trips: what a train function returns is exactly what the
175
+ predict and explain functions take — store it in a dataset, pass it between
176
+ tasks, or hand it to `Shap`.
177
+
178
+ ## Common Patterns
179
+
180
+ ### Train and predict
181
+
182
+ ```typescript
183
+ const fit = East.function([], VectorType(FloatType), $ => {
184
+ const X = $.let(East.Matrix.fromArray([[1.0, 2.0], [2.0, 1.0], [3.0, 4.0], [4.0, 3.0]]));
185
+ const y = $.let(East.Vector.fromArray([3.0, 3.0, 7.0, 7.0]));
186
+ const config = $.let({ // every field an Option: some(value) / none
187
+ n_estimators: some(100n), max_depth: some(3n), learning_rate: none, min_child_weight: none,
188
+ subsample: none, colsample_bytree: none, reg_alpha: none, reg_lambda: none, gamma: none,
189
+ random_state: some(42n), n_jobs: none, sample_weight: none, categorical_features: none,
190
+ categorical_n: none, max_cat_to_onehot: none, max_cat_threshold: none, scale_pos_weight: none,
191
+ }, XGBoost.Types.XGBoostConfigType);
192
+ const model = $.let(XGBoost.trainRegressor(X, y, config));
193
+ $.return(XGBoost.predict(model, X));
194
+ });
195
+ ```
196
+
197
+ ### Optimisation
198
+
199
+ The objective is an East function of the decision vector —
200
+ `East.function([VectorType(FloatType)], FloatType, …)` — bound with `$.const`
201
+ and passed with the start point, the bounds and the config; the result
202
+ carries `x_best` and `f_best` (see the Quick Start).
203
+
204
+ ## Calling from Python (the functions directly)
205
+
206
+ The platform functions are exported from `east_py_datascience` as **plain
207
+ callables over East values** — no IR, no compile — and, the same objects,
208
+ callable inside an `East.function` body, where the call is the `Platform`
209
+ node with the declared signature. A project's own `@East.platform_function`
210
+ imports and calls them (the preferred way to use lightning / torch / xgboost
211
+ / sklearn from project python); a python East body calls them the same way and
212
+ compiles against the package's `platform` list:
213
+
214
+ ```python
215
+ from east import East, FloatType, MatrixType, VectorType, coerce_to, none, some
216
+ from east_py_datascience import xgboost_predict, xgboost_train_regressor
217
+ from east_py_datascience.xgboost import XGBoostConfigType
218
+
219
+ @East.platform_function(inputs=[MatrixType(FloatType), VectorType(FloatType), MatrixType(FloatType)],
220
+ output=VectorType(FloatType))
221
+ def forecast(X_train, y_train, X_new):
222
+ config = coerce_to({
223
+ "n_estimators": some(200), "max_depth": some(4), "learning_rate": some(0.05),
224
+ "min_child_weight": none, "subsample": none, "colsample_bytree": none,
225
+ "reg_alpha": none, "reg_lambda": none, "gamma": none,
226
+ "random_state": some(42), "n_jobs": none, "sample_weight": none,
227
+ "categorical_features": none, "categorical_n": none,
228
+ "max_cat_to_onehot": none, "max_cat_threshold": none,
229
+ "scale_pos_weight": none, # the binary class-imbalance weight (classifiers)
230
+ }, XGBoostConfigType)
231
+ model = xgboost_train_regressor(X_train, y_train, config) # a model blob, East in and out
232
+ return xgboost_predict(model, X_new)
233
+ ```
234
+
235
+ Rules:
236
+
237
+ - **Inputs are East values, not numpy**: `EastMatrix` / `EastVector` for data,
238
+ an `EastStruct` config built with `coerce_to(dict, ConfigType)` — every
239
+ `Option` field spelled `some(value)` or `none` (a plain value or `None` is
240
+ refused) — and `variant(case, value, Type)` for a variant.
241
+ - **Config and blob types live in each submodule**
242
+ (`east_py_datascience.xgboost.XGBoostConfigType`, `…XGBoostModelBlobType`),
243
+ mirroring the TypeScript `Module.Types.*`.
244
+ - **Names are the platform names** (`xgboost_train_regressor`,
245
+ `mads_optimize`, …), except GoogleOr's, exported without the `google_or_`
246
+ prefix: `cpsat_solve`, `cpsat_solve_all`, `routing_solve`, `linear_solve`,
247
+ `min_cost_flow`, `max_flow`, `assignment`, `min_cost_assignment`.
248
+ - **Inside an East body**: `East.function([...], ..., lambda b, X, y, cfg:
249
+ xgboost_predict(xgboost_train_regressor(X, y, cfg), X))` — the calls are
250
+ `Platform` nodes; `East.compile(fn, platform=east_py_datascience.platform)`
251
+ runs it (the list the e3 python runner registers).
252
+ - **The generic ones take the type argument FIRST in a body**, as TypeScript
253
+ reads: `causal_experiment(RowType, rows, config)`,
254
+ `alns_optimize(SolutionType, initial, objective, destroy, repair, config)`.
255
+ From python they take the values alone (`causal_experiment(rows, config)`):
256
+ they read the values, not the type.
257
+ - **`simulation_run` and `optimization_iterative*` run in C**, so there is no
258
+ python to call: the name exports the declaration, and a body calls it the
259
+ same way — `simulation_run(Resources, Events, state, events, process,
260
+ config)`.
261
+ - Optional dependencies gate at call time: a function raises
262
+ `NotImplementedError` naming the missing extra (`east-py-datascience[causal]`).
263
+
264
+ For the East-value API (eager methods, `coerce_to`, `to_numpy` / `to_torch`,
265
+ `@East.platform_function`), load the **east-py** skill.
266
+
267
+ ## Related skills
268
+
269
+ - **e3** — **required to run these**: they need the python runtime, so wrap each
270
+ call in an `e3.task` with a python runner (`{ runner: { runtime: 'east-py',
271
+ platforms: ['east-py-datascience'] } }` — the typed runner resolves east-py
272
+ from the project's `.venv`, no `uv run` wrapper needed). They do not run on
273
+ the Node or C runtimes.
274
+ - **east** — the language for objective functions, configs and results.
275
+ - **east-py** — the python runtime: East values as plain python data, eager
276
+ methods, and the `@East.platform_function` on-ramp the direct calls live in.
277
+ - **east-ontology** — the decisions these models improve are the `decision`
278
+ nodes of the business's economic ontology.
279
+ - **east-design** — place the forecast or optimisation in a decision-oriented
280
+ architecture.
@@ -0,0 +1,186 @@
1
+ ---
2
+ name: east-py-io
3
+ description: "I/O platform functions for the East language on the Python runtime - SQL databases (SQLite, PostgreSQL, MySQL, Access), NoSQL (Redis, MongoDB), S3 storage, file transfers (FTP, SFTP), file formats (XLSX, XML), and compression (Gzip, Zip, Tar). Use when writing Python (not the TypeScript DSL) that calls or registers these platform functions. Triggers for: (1) Calling east_py_io functions (sqlite_select, mongo_find, s3_put_object, ftp_get, xlsx_read, gzip_compress, ...) from a project's own Python @East.platform_function, or inside an East.function body — the same object does both, (2) Registering the east_py_io platform list with compile() so East programs can use SQL/NoSQL/Storage/Transfer/Format/Compression on the Python runtime, (3) Building connection configs (SqliteConfigType, PostgresConfigType, RedisConfigType, S3ConfigType, FtpConfigType, ...) in Python, (4) The connect -) handle -) operate -) close connection… See the detailed scope below."
4
+ ---
5
+
6
+ ## Detailed skill scope
7
+
8
+ I/O platform functions for the East language on the Python runtime - SQL databases (SQLite, PostgreSQL, MySQL, Access), NoSQL (Redis, MongoDB), S3 storage, file transfers (FTP, SFTP), file formats (XLSX, XML), and compression (Gzip, Zip, Tar). Use when writing Python (not the TypeScript DSL) that calls or registers these platform functions. Triggers for: (1) Calling east_py_io functions (sqlite_select, mongo_find, s3_put_object, ftp_get, xlsx_read, gzip_compress, ...) from a project's own Python @East.platform_function, or inside an East.function body — the same object does both, (2) Registering the east_py_io platform list with compile() so East programs can use SQL/NoSQL/Storage/Transfer/Format/Compression on the Python runtime, (3) Building connection configs (SqliteConfigType, PostgresConfigType, RedisConfigType, S3ConfigType, FtpConfigType, ...) in Python, (4) The connect -> handle -> operate -> close connection lifecycle. For authoring East programs in TypeScript against these functions, use the east-node-io skill instead.
9
+
10
+ # East.py I/O Platform Functions
11
+
12
+ `east_py_io` is the Python implementation of the East I/O platform: SQL and
13
+ NoSQL databases, S3 object storage, FTP/SFTP transfers, XLSX/XML formats and
14
+ compression. Each function is **dual-mode**: a plain python callable over East
15
+ values — call it from your own `@East.platform_function` — and, the same
16
+ object, callable inside an `East.function` / `East.asyncFunction` body, where
17
+ the call IS the `Platform` node with the function's declared signature.
18
+ Nothing restates the signature: register the `platform` list at
19
+ `East.compile` / `East.compileAsync` (or let the runner) and it runs.
20
+
21
+ Every function but `xlsx_*`, `xml_*`, `zip_*` and `ftp_close` is **async**: a
22
+ direct call returns a coroutine to `await`, and a body calling one is an
23
+ `East.asyncFunction`. Exported names are the platform names, except MongoDB's,
24
+ exported as `mongo_*` (`mongo_find` is the platform's `mongodb_find_many`).
25
+
26
+ This skill is for **Python**. To author East programs in TypeScript against
27
+ these functions (`SQL.SQLite.query`, `Storage.S3.putObject`, …), load
28
+ **east-node-io** — the same functions, under the same platform names.
29
+
30
+ ## Before writing code — search the example index
31
+
32
+ Every East API has a tested example in the plugin's index — the index IS the
33
+ API reference, printed from each example's IR in TypeScript or python. Before
34
+ writing or changing East code:
35
+
36
+ 1. Call `search_east_examples` for each capability you
37
+ are about to use — `language: "python"` for east-py, `"typescript"`
38
+ otherwise. Summaries come back first: id, signature, the inputs and the
39
+ expected result, a few hundred bytes each.
40
+ 2. Fetch the one or two that match with `get_east_example`
41
+ and pattern your code on them.
42
+ 3. Do not read `node_modules/@elaraai/**` or `*.examples.ts` files wholesale,
43
+ and do not reason from `.d.ts` signatures: the index holds the same
44
+ programs, exact and far cheaper, and the signatures omit the runtime rules
45
+ that make East code correct.
46
+
47
+ Nothing is injected for you; the search is the step.
48
+
49
+ ## Quick Start
50
+
51
+ ```python
52
+ from east import ArrayType, BlobType, East, FloatType, StringType, StructType, coerce_to, none, some
53
+ from east_py_io import (GzipOptionsType, SqliteConfigType, gzip_compress, platform,
54
+ sqlite_close, sqlite_connect, sqlite_select)
55
+
56
+ OrderRow = StructType([("sku", StringType), ("qty", FloatType)])
57
+
58
+ @East.platform_function(inputs=[StringType], output=ArrayType(OrderRow))
59
+ async def load_orders(db_path):
60
+ # An Option field takes some(x) / none — a plain value (or None) is refused
61
+ config = coerce_to({"path": db_path, "readOnly": some(True), "memory": none}, SqliteConfigType)
62
+ handle = await sqlite_connect(config) # a connection handle: String
63
+ try:
64
+ select = sqlite_select(None, OrderRow) # the typed read — rows decode to YOUR struct
65
+ return await select(handle, "SELECT sku, qty FROM orders", [])
66
+ finally:
67
+ await sqlite_close(handle)
68
+
69
+ # The same functions inside an East body — the call is the Platform node, the
70
+ # package's own named types spell the options; an async implementation needs an
71
+ # East.asyncFunction body, compiled with East.compileAsync and the package's list
72
+ packed = East.asyncFunction([BlobType], BlobType,
73
+ lambda b, data: gzip_compress(data, East.value({"level": some(9)}, GzipOptionsType)))
74
+ East.compileAsync(packed, platform=platform)
75
+ ```
76
+
77
+ Run a compiled async body through the runner (`east-py run`, an e3 task): the
78
+ bridge runs async implementations on its own event loop, so the compiled
79
+ coroutine cannot be awaited under `asyncio.run` once it calls one.
80
+
81
+ ## Connection Lifecycle
82
+
83
+ Databases, transfers and caches share one pattern: `*_connect` (Access:
84
+ `access_open`) takes a config and returns an opaque **connection handle** (a
85
+ `String`, valid for the process lifetime), every operation takes the handle,
86
+ and `*_close(handle)` / `*_close_all()` release it — always close, since
87
+ handles are pooled per process. S3 has no connection: each call takes the
88
+ `S3ConfigType`.
89
+
90
+ ## Decision Tree: What Do You Need?
91
+
92
+ ```
93
+ Task → What do you need?
94
+ │
95
+ ├─ SQL — sqlite_* (apsw) · postgres_* (asyncpg) · mysql_* (aiomysql): connect / query / select / close / close_all
96
+ │ ├─ Typed rows → *_select, generic: in a body sqlite_select(Row, handle, sql, params) (the row type FIRST, as
97
+ │ │ TypeScript reads); from python the factory sqlite_select(None, Row)(handle, sql, params)
98
+ │ ├─ Anything else → *_query(handle, sql, params) -> SqlResultType, a variant by statement kind:
99
+ │ │ select {rows: each a Dict<String, LiteralValue>} · insert {rowsAffected, lastInsertId: Option} ·
100
+ │ │ update / delete {rowsAffected}
101
+ │ ├─ params → an Array of LiteralValueType (Variant<Blob, Boolean, DateTime, Float, Integer, Null, String>)
102
+ │ └─ MS Access, read-only (access-parser) → access_open(config) · access_tables(handle) -> {tables} ·
103
+ │ access_query(Row, handle, AccessQueryOptionsType{table, columns, rowOffset, rowLimit: Options}) — a table
104
+ │ read, generic like *_select (the factory from python) · access_close / access_close_all
105
+ │
106
+ ├─ NoSQL
107
+ │ ├─ MongoDB (motor; python names mongo_*) → mongo_connect(MongoConfigType{uri, database, collection}) ·
108
+ │ │ mongo_insert_one(h, doc) -> id String · mongo_find_one(h, filter) -> Option<doc> ·
109
+ │ │ mongo_find(h, filter, MongoFindOptionsType{limit, skip}) -> Array<doc> · mongo_update_one(h, filter, update) ·
110
+ │ │ mongo_delete_one / mongo_delete_many(h, filter) -> count · mongo_close / mongo_close_all
111
+ │ │ (a document is a Dict<String, BsonValueType>; _id comes back as a String)
112
+ │ └─ Redis (redis) → redis_connect · redis_get(h, key) -> Option<String> · redis_set(h, key, value) ·
113
+ │ redis_setex(h, key, value, seconds) · redis_del(h, key) -> count · redis_close / redis_close_all
114
+ │
115
+ ├─ Object storage (boto3; each call takes the S3ConfigType first)
116
+ │ └─ s3_put_object(cfg, key, blob) · s3_get_object(cfg, key) -> Blob · s3_head_object(cfg, key) ·
117
+ │ s3_delete_object(cfg, key) · s3_list_objects(cfg, prefix, max_keys, continuation_token: Option) -> S3ListResultType ·
118
+ │ s3_presign_url(cfg, key, expires_in_seconds) -> String
119
+ │
120
+ ├─ File transfer — ftp_* (aioftp) · sftp_* (asyncssh)
121
+ │ └─ connect · list(h, path) -> FileListType · get(h, path) -> Blob · put(h, path, blob) · delete(h, path) · close / close_all
122
+ │
123
+ ├─ File formats (sync)
124
+ │ ├─ XLSX (openpyxl) → xlsx_read(blob, XlsxReadOptionsType{sheetName: Option}) -> Array<Array<LiteralValue>>, the
125
+ │ │ sheet as a grid of cells · xlsx_write(grid, options) -> Blob · xlsx_info(blob) -> {sheets: [{name, rowCount, columnCount}]}
126
+ │ └─ XML (defusedxml) → xml_parse(blob, XmlParseConfigType) -> XmlNodeType (recursive) · xml_serialize(node, XmlSerializeConfigType) -> Blob
127
+ │
128
+ └─ Compression (core — the standard library, no extra)
129
+ ├─ Gzip → gzip_compress(blob, GzipOptionsType{level: Option}) · gzip_decompress(blob) (async)
130
+ ├─ Zip → zip_compress(ZipEntriesType, ZipOptionsType) -> Blob · zip_decompress(blob) -> Dict<name, Blob> (sync)
131
+ └─ Tar → tar_create(TarEntriesType) -> Blob · tar_extract(blob) -> Dict<name, Blob> (async)
132
+ (entries: Array<{name, data: Blob}>)
133
+ ```
134
+
135
+ ## Optional Dependencies
136
+
137
+ Each driver is an extra named after its module; a function whose extra is
138
+ missing raises `NotImplementedError` naming it (`SQLite support requires the
139
+ 'sqlite' extra. Add east-py-io[sqlite] to your pyproject.toml dependencies.`):
140
+
141
+ | Extra | Driver | Extra | Driver |
142
+ |-------|--------|-------|--------|
143
+ | `sqlite` | apsw | `redis` | redis |
144
+ | `postgres` | asyncpg | `mongodb` | motor |
145
+ | `mysql` | aiomysql | `xlsx` | openpyxl |
146
+ | `access` | access-parser | `xml` | defusedxml |
147
+ | `s3` | boto3 | `ftp` / `sftp` | aioftp / asyncssh |
148
+
149
+ Convenience groups: `sql`, `nosql`, `all`. Compression is core (the standard
150
+ library).
151
+
152
+ ## East Type Definitions
153
+
154
+ Every config and result type is importable from `east_py_io`. The ones you
155
+ build:
156
+
157
+ | Type | Fields |
158
+ |------|--------|
159
+ | `SqliteConfigType` | `path`, `readOnly: Option<Boolean>`, `memory: Option<Boolean>` |
160
+ | `PostgresConfigType` / `MySqlConfigType` | `host`, `port`, `database`, `user`, `password`, `ssl: Option<Boolean>`, `maxConnections: Option<Integer>` |
161
+ | `AccessConfigType` | `path`, `password: Option<String>` |
162
+ | `RedisConfigType` | `host`, `port`, `password: Option<String>`, `db: Option<Integer>`, `keyPrefix: Option<String>` |
163
+ | `MongoConfigType` | `uri`, `database`, `collection` |
164
+ | `S3ConfigType` | `region`, `bucket`, `accessKeyId: Option<String>`, `secretAccessKey: Option<String>`, … |
165
+ | `FtpConfigType` / `SftpConfigType` | `host`, `port`, `user`, `password`, `secure: Boolean` / `host`, `port`, `username`, `password: Option<String>`, `privateKey: Option<String>` |
166
+ | `XlsxReadOptionsType` · `XmlParseConfigType` · `XmlSerializeConfigType` | `sheetName: Option<String>` · `preserveWhitespace`, `decodeEntities` · `indent: Option<String>`, `includeXmlDeclaration`, `encodeEntities`, `selfClosingTags` |
167
+ | `GzipOptionsType` / `ZipOptionsType` · `ZipEntriesType` / `TarEntriesType` | `level: Option<Integer>` · `Array<{name, data: Blob}>` |
168
+
169
+ Results come back as the documented types — `SqlResultType`, `BsonValueType`
170
+ documents, `S3ListResultType` (`objects`, `isTruncated`,
171
+ `continuationToken`), `FileListType` (`name`, `path`, `size`, `isDirectory`,
172
+ `modifiedTime`), `XmlNodeType`, `LiteralValueType` cells. Build a config with
173
+ `coerce_to({...}, ConfigType)`: an `Option` field takes `some(value)` or
174
+ `none`, never a plain value or `None`.
175
+
176
+ ## Related skills
177
+
178
+ - **east-py** — the python runtime: East values as plain data, eager methods,
179
+ `coerce_to`, and the `@East.platform_function` on-ramp these calls live
180
+ inside (and the dual-mode rule that makes them callable in a body).
181
+ - **east-py-std** — the standard-library sibling: console, filesystem, fetch,
182
+ crypto, time, path, random, large JSON.
183
+ - **east-py-datascience** — ML and optimisation platform functions.
184
+ - **east-node-io** — the TypeScript authoring surface for the same functions.
185
+ - **e3** — the execution engine whose python runner registers this list for
186
+ dataflow tasks.