decision-gem 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 tumi
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,366 @@
1
+ Metadata-Version: 2.4
2
+ Name: decision-gem
3
+ Version: 0.1.0
4
+ Summary: Jev-style decision model (noul / choice / score) on a local Gemma, MLX; prompts too
5
+ Author: tumi
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/tumikosha/decision-gem
8
+ Project-URL: Changelog, https://github.com/tumikosha/decision-gem/blob/main/CHANGELOG.md
9
+ Keywords: llm,mlx,gemma,classification,decision-model,jev,apple-silicon
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Intended Audience :: Science/Research
13
+ Classifier: Operating System :: MacOS
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
16
+ Requires-Python: >=3.10
17
+ Description-Content-Type: text/markdown
18
+ License-File: LICENSE
19
+ Requires-Dist: mlx>=0.32
20
+ Requires-Dist: mlx-lm>=0.32
21
+ Requires-Dist: huggingface_hub>=0.30
22
+ Provides-Extra: test
23
+ Requires-Dist: pytest; extra == "test"
24
+ Dynamic: license-file
25
+
26
+ # decision-gem
27
+
28
+ A decision model in the style of [TypeSafe Jev](https://developers.cloudflare.com/ai/models/typesafe/jev/) on a local
29
+ Gemma (MLX). It answers three kinds of questions about a text (the "state"): yes / no, a choice among options, a score
30
+ on a scale. No text is generated: the answer is read from the probabilities of the options' letters. Runs offline.
31
+ The same model also answers ordinary prompts by generating (`generate`, see "Prompts").
32
+
33
+ Russian version: [README.md](https://github.com/tumikosha/decision-gem/blob/main/README.md).
34
+
35
+ ## What it does
36
+
37
+ Request: `{"state": text or JSON object, "questions": {id: question}}`.
38
+
39
+ | Type | Question | Answer |
40
+ |---|---|---|
41
+ | `noul` | `instructions`, `criteria` `{"true": …, "false": …}` (optional) | `noul` = P(true) |
42
+ | `choice` | `instructions`, `criteria` `{option: description}` or `[option, …]`, 2..26 | `choice`, `confidence`, `probabilities` |
43
+ | `score` | `instructions`, `criteria` `{value: description}` or `[description, …]` (levels 1..N), 2..10 | `score` = Σ p·value, `confidence`, `legend`, `probabilities` |
44
+
45
+ Beyond Jev, every answer has `orders_agree`: whether the choice is the same in every option order (below).
46
+ `false` means an unsure answer, worth checking. The whole response: `{"model", "answers": {id: answer},
47
+ "usage": {"input_tokens", "output_tokens": 0}}`.
48
+
49
+ How it works:
50
+ 1. The state is read once into a KV cache.
51
+ 2. Each question is appended to a copy of the cache, its options under the letters A, B, C …
52
+ 3. The logits of those letters at the first answer position are read.
53
+ 4. Each question is read in several option orders. Each option's log-probabilities are averaged, so a
54
+ preference for a letter or a position cancels out.
55
+
56
+ ### Option orders
57
+
58
+ `DecisionGem(orders=...)`, per request `decide(..., orders=...)`, on the CLI `--orders`:
59
+
60
+ | `orders` | Passes per question | What cancels |
61
+ |---|---|---|
62
+ | `"reverse"` (default) | 2: in order and reversed | with 2 options, everything; with 3+ the middle stays in the middle (of 3 options, B is B both times), so a pull to or from the middle letters stays |
63
+ | `"cyclic"` | n: every rotation | each option once under every letter: position cancels fully |
64
+
65
+ For noul (2 options) both are the same two passes.
66
+
67
+ Under `cyclic`, `orders_agree` means all n passes agree, stricter than two; its `false` rate is not comparable
68
+ across modes. Calibration `T` was fitted on `reverse`; fit it again for `cyclic`.
69
+
70
+ ### Calibration
71
+
72
+ Raw LLM probabilities are overconfident: `noul` 1.0 does not mean "yes" 100% of the time. Calibration remaps them so
73
+ that 0.8 means "yes" in about 80% of such answers. Each question has its own bias, so the coefficients are kept per
74
+ question id: `[a, b]` for noul (`p = sigmoid(a·logit + b)`, `logit` being the log-probability of true minus that of
75
+ false), `T` for choice / score (`softmax(logprob / T)`, `T > 1` softens).
76
+
77
+ ```json
78
+ {"spam": [0.35, -0.8], "kind": 2.1}
79
+ ```
80
+
81
+ Pass it as `DecisionGem(calibration=...)` or `--calibration cal.json`. To fit it:
82
+
83
+ 1. Label 100–300 of your own texts by hand: the answer to every question.
84
+ 2. Get answers with the raw log-probabilities: `decide(..., raw=True)`, `{"raw": true}` in a request or `--raw` in the
85
+ CLI. Every answer gets `logprobs` {option: log-probability before calibration}.
86
+ 3. `fit_calibration(answers, labels)` returns the calibration.
87
+
88
+ ```python
89
+ from decision_gem import DecisionGem, fit_calibration
90
+
91
+ with DecisionGem() as gem:
92
+ answers = [gem.decide(text, qs, raw=True)["answers"] for text in texts]
93
+ cal = fit_calibration(answers, labels) # labels: [{"spam": True, "kind": "phishing"}, ...] in the same order
94
+ ```
95
+
96
+ A label is `True` / `False` for noul, the option for choice, the level for score; a question with no label in a text is
97
+ skipped. noul is fitted by Platt scaling (logistic regression on `logit`), `T` by the least log-loss. Both noul labels
98
+ and some model mistakes must be in the set: where the model is always right, `T` goes to 0.01 and `a` grows.
99
+
100
+
101
+ ## Differences from Jev
102
+
103
+ The request and the response are Jev's (noul / choice / score). Everything under the contract differs. Facts about Jev
104
+ come from the Cloudflare and TypeSafe docs (`docs.typesafe.ai/models.md`), November 2026; they do not disclose how Jev
105
+ works inside.
106
+
107
+ | | Jev (TypeSafe) | decision-gem |
108
+ |---|---|---|
109
+ | Where it runs | a cloud API (Cloudflare Workers AI, TypeSafe `POST /v1/systemone`), $0.042 per 1M input tokens | locally on Apple silicon (MLX), offline, data stays on the machine |
110
+ | Model | its own, trained to return calibrated decisions (RLCD), architecture not disclosed | any instruct LLM in mlx-lm format, see "Other models" |
111
+ | How it computes | not disclosed | the logits of the option letters on a copy of the state's KV cache, averaged over option orders |
112
+ | Calibration | built in by training | fitted on your own labels: `fit_calibration`, see "Calibration" |
113
+ | Question types | noul, choice, score, Box (from Jev 1.14) | noul, choice, score |
114
+ | Images (`assets`) | yes (beta) | no: text or JSON |
115
+ | Limits | 64k tokens per request, 32k per question (262k with a header) | the model's context and memory; choice 2..26 options, score 2..10 levels |
116
+
117
+ Only in decision-gem:
118
+
119
+ - `orders_agree` in every answer and `orders="cyclic"` (see "Option orders");
120
+ - `raw=True`: the raw option `logprobs`, and `fit_calibration` on them;
121
+ - keyed states: `prefill`, `decide(key=...)`, `forget`, `save` / `load`;
122
+ - open states and `extend`: text is read into the cache as it grows;
123
+ - `generate`: the same model answers ordinary prompts;
124
+ - score `criteria` also as `{value: description}`, choice `criteria` also as a list.
125
+
126
+ ## Keyed states
127
+
128
+ `decide(state, questions)` reads the state, answers and drops its cache. To ask one email many questions at different
129
+ times, or to switch between emails, read the state once and keep it under a key:
130
+
131
+ | Method | What it does |
132
+ |---|---|
133
+ | `prefill(key, state)` | reads the state into a KV cache and keeps it under `key`; returns the number of tokens |
134
+ | `decide(key=key, questions=qs)` | questions on a kept state; exactly one of `state` / `key`, else `ValueError` |
135
+ | `gem({"key": key, "questions": qs})` | the same as a Jev request |
136
+ | `forget(key)` | frees the state's memory |
137
+ | `save(key, path)` / `load(key, path)` | the state to a `.safetensors` file and back; `load` takes only a file saved with the same model |
138
+
139
+ Each question runs on its own copy of the cache, so the kept state does not change: questions are independent and the
140
+ model does not remember earlier questions (it is not a chat). An answer by key is byte-identical to the answer of
141
+ `decide(state, ...)` — checked on 3 calls in reverse order and after `save` + `load`. Copying the cache is almost
142
+ free: MLX arrays are immutable, the data is shared until a question is appended to the copy.
143
+
144
+ A state file is the email's text in another form: keep it only where the text itself may be kept.
145
+
146
+ ### Open states
147
+
148
+ `prefill(key, state, open=True)` stops the cache right after the state text; the closing `</state>` is not read yet.
149
+ `extend(key, text)` reads more text into that cache for good (it returns the tokens added; `ValueError` on a closed
150
+ state), so a growing thread can be kept open and new emails read into it. `decide` on an open state puts `</state>` and the question after the
151
+ cache on every copy and never writes the kept cache. `save` / `load` keep the open flag. `split_open(prompt)` is the
152
+ pure helper that cuts a rendered prompt before the closing tag.
153
+
154
+ Seam rule: `extend` equals one read only when the new text starts at a token boundary. Put the leading space or newline
155
+ in the new text, never split a word, and never end a chunk in whitespace (`extend` raises `ValueError`): `"hello. "` +
156
+ `"Operator"` tokenizes differently from `"hello. Operator"`. `prefill` strips the state, so it never ends in whitespace.
157
+
158
+ Precision: the gem computes in float32 by default (`float32=True`), see "The bf16 bug" below. `extend` equals one
159
+ read only in float32.
160
+
161
+ ## Prompts
162
+
163
+ `generate(prompt, state=None, *, key=None, max_tokens=512, temperature=0.0)` runs the model as an ordinary chat model:
164
+ the answer to the prompt is generated (`mlx_lm.stream_generate`) by the model already loaded.
165
+
166
+ | Call | What the model sees |
167
+ |---|---|
168
+ | `generate(prompt)` | the prompt only |
169
+ | `generate(prompt, state)` | the state, then the prompt where a question would go |
170
+ | `generate(prompt, key=key)` | the kept state (`prefill` / `load`), not read again; generated on a copy of its cache |
171
+ | `gem({"prompt": …, "state" or "key", "max_tokens", "temperature"})` | the same as a request; the CLI takes this JSON too |
172
+
173
+ Response: `{"model", "text", "finish_reason": "stop" | "length", "usage": {"input_tokens", "output_tokens"}}`.
174
+ `temperature` 0 is greedy: the same prompt gives the same text. Generating leaves a kept state as it was:
175
+ `decide(key=…)` before and after `generate(key=…)` gives byte-identical answers (the `-m models` test); the answer on a
176
+ key equals the answer on the same `state`.
177
+
178
+ Limits: the answer is not constrained to a JSON schema (mlx-lm has no structured output) — check JSON and ask again
179
+ on an error; do not run beside Ollama with the same Gemma (memory).
180
+
181
+ ## The bf16 bug: noisy answers on long states (found 2026-10-19)
182
+
183
+ **Symptom.** The same text gave a different answer depending on how it was read: whole, in chunks with `extend`,
184
+ or with `</state>` in another place. Up to ~460 tokens the answers agreed, beyond that they drifted:
185
+
186
+ | State tokens | Whole | In chunks (`extend`) |
187
+ |---|---|---|
188
+ | ≤ 462 | same | same |
189
+ | 692 | 0.75 | 0.37 |
190
+ | ~1400 | 0.45 | 0.22 |
191
+
192
+ Open vs closed state on one long text: 0.45 vs 0.61. One 1590-token text read in different ways gave `noul` from
193
+ 0.14 to 0.65: at a 0.5 threshold the yes / no answer depended on the way of reading, not on the text.
194
+
195
+ **Cause.** bf16 compute flips the expert choice of the MoE router. bf16 rounding depends on how many tokens one
196
+ read holds. Each layer's router makes a hard top-8 expert pick (`argpartition`). When the 8th and 9th expert scores
197
+ are nearly equal (at layer 8 the gap was 2.8e-5), rounding flips the pick, and the difference grows through the
198
+ later layers. The norms and the quantization scales and biases were bf16; the weights themselves are 4-bit.
199
+
200
+ **Fix.** `DecisionGem(..., float32=True)` by default. After loading, every bf16 parameter
201
+ (quantization scales and biases, norms, embeddings, router) is cast to float32; the packed 4/8-bit weights stay as
202
+ they are. In float32 the spread is 0.568..0.574 and `extend` matches one read to within 0.006 (not bit-exact).
203
+ `float32=False` keeps the old behaviour.
204
+
205
+ Cost: +1.5 GiB for the model (13.2 → 14.7 GiB), twice the KV cache (≈0.43 MiB per token), reads about 1.3x slower,
206
+ a short +1.5 GiB at load. The "Resources" numbers were measured in bf16.
207
+
208
+ **Saved states.** `save` records the compute dtype (`dtype`), `load` raises `ValueError` when it
209
+ differs from the gem's. A file without the field counts as bf16 (saved before the fix). Without the check a bf16
210
+ state loaded silently into a float32 gem and gave a mixed cache with the same noise.
211
+
212
+ **What it touched.**
213
+ - Everything computed before 2026-10-19 (before version 0.1.0) ran in bf16.
214
+ - Operator / client roles on 145 calls (state = first 20 lines): in float32 no call changed its operator, accuracy
215
+ is the same; 163 s vs 127 s. On short states the noise does not show.
216
+ - Answers on long calls (the checklist, the whole call in the state) may have depended on the noise. Calibrations
217
+ fitted on bf16 answers need refitting.
218
+
219
+ **TF32 on M5 (found 2026-10-19).** Cause: on GPUs with neural accelerators (M5) MLX multiplies float32 matrices in
220
+ TF32 (relative error ~5e-4 instead of ~1e-6) from 128 rows up (dense: from 2), with another kernel below that. So the
221
+ same tokens get different rounding depending on how many one read holds, and the router's near-ties flip as in bf16.
222
+ Symptom on an M5 Max: 0.347 in chunks vs 0.546 whole. `DecisionGem(float32=True)` sets `MLX_ENABLE_TF32=0` and
223
+ checks a float32 matmul against the CPU (`float32_matmuls`); MLX reads the variable once, at the process's first
224
+ matmul, so if one already ran with TF32 on the gem raises `RuntimeError`. With it: 0.5739 in chunks vs 0.5741 whole.
225
+ Cost: reads 2x slower than in TF32 (1594 tokens 1.6 s vs 0.8 s, a question 0.30 s vs 0.18 s).
226
+
227
+ **Check.** `tests/test_open_model.py` (with the model): a long state in chunks equals one read, open equals closed,
228
+ tolerance 0.02. In bf16 these two tests fail (0.2306 and 0.1544). Model-free tests in `tests/test_core.py` cover the
229
+ cast and `dtype` in `save` / `load`.
230
+
231
+ ## Where it was used
232
+
233
+ Speech analytics of call-centre calls, model `mlx-community/gemma-4-26b-a4b-it-4bit`.
234
+
235
+ - Operator / client roles: the right operator in 99 of the 100 calls of the roles reference.
236
+ - Checklist item statuses, 199 calls, 2250 items: 7.1 s per call, 9 of 9 manual rulings matched, the option orders
237
+ agree on 2094 of 2250.
238
+ - The full checklist with no analysis prompt: scenario, outcome, call flags, then the items; 199 calls, 15.5 s per
239
+ call, 9 of 9 manual rulings.
240
+
241
+ ## Resources
242
+
243
+ Measured on the longest call (3051 tokens of state) and on 199 calls, 2026-10-13, an Apple silicon Mac with 36 GB:
244
+
245
+ | What | Time / memory |
246
+ |---|---|
247
+ | model load | about 6 s, 13.2 GiB |
248
+ | reading a state (`prefill` or the start of `decide`) | 4.2 s for 3051 tokens (0.8 s for 882) |
249
+ | a state's KV cache | ≈0.2 MiB per token: 656 MiB for 3051 tokens; 3 calls of 882–2430 tokens — 1.11 GiB |
250
+ | cache copy per question | 0.1 ms |
251
+ | one question (two passes, options in order and reversed) | 221 ms |
252
+ | generation (`generate`) | 86–97 tokens/s; a call's summary (653 state tokens, 71 answer tokens) in 0.7 s (16.10) |
253
+ | `save` / `load` of a state | 0.34 s / 0.02 s, file 656 MiB |
254
+ | peak memory on long calls | about 17 GiB (27.8 GiB with the MLX buffer cache) |
255
+ | a call in the runs above | 2–30 s, depends on the number of questions |
256
+
257
+ On a 36 GB Mac do not run it next to another large model. The GPU gets about 27 GiB: besides the model, about 15
258
+ kept long calls fit in memory; more through `save` / `load`.
259
+
260
+ ## Memory
261
+
262
+ - `cache_limit` (default 0) — bytes of freed buffers MLX keeps for reuse (`mx.set_cache_limit`); `None` — leave the
263
+ MLX limit as it is. With the buffers the peak was 27.8 GiB instead of 17. The limit is process-wide: `unload()`
264
+ puts back the previous one (the MLX default is 34.2 GiB on this Mac).
265
+ - `gem.unload()` frees the model and all kept states (13.2 GiB → 0) without ending the process; `decide()` after it
266
+ raises `RuntimeError`. Or `with DecisionGem() as gem: ...` — unloaded on exit.
267
+ - `forget(key)` — one state's memory (3 calls: 1.11 GiB, after `forget` of one 0.59 GiB).
268
+
269
+ ## Other models
270
+
271
+ The model is set by `DecisionGem(repo=...)` or `--model`: an HF repo id or a local folder in mlx-lm format.
272
+
273
+ ```python
274
+ gem = DecisionGem("mlx-community/Qwen3-14B-4bit")
275
+ ```
276
+
277
+ ```bash
278
+ decision-gem request.json --model mlx-community/Qwen3-14B-4bit
279
+ ```
280
+
281
+ What a model needs:
282
+
283
+ - mlx-lm format (`mlx_lm.load` loads it), Apple silicon;
284
+ - instruction-tuned, with a chat template;
285
+ - every letter A..Z after the answer template is exactly one token. Checked at load (`letter_ids`): otherwise
286
+ `ValueError` before the first question;
287
+ - the first answer token is a letter, not the start of reasoning. The template gets `enable_thinking=False`. A model
288
+ that always reasons is read wrong, and the check does not catch it.
289
+
290
+ **Small models.** Models up to ~8B parameters, with no fine-tuning and no calibration, answer poorly. In JevBench
291
+ v1.6.0 (Benchmark Heaven, 2026-10-05) raw Qwen3 with a logit readout scored a Capability of 34.6 (4B Instruct),
292
+ 30.1 (8B), 8.3 (1.7B) and 8.2 (0.6B); frozen Gemma-4-12B-it and Gemma-4-31B-it scored 70.9 and 77.6, Jev 1.13.0 76.5.
293
+ This covers `gemma-4-e2b` and `gemma-4-e4b` in "Model comparison" too: the tests on them check the package, not the
294
+ quality of the answers. Take a small model only with a calibration fitted on your own labels and its accuracy checked
295
+ on them.
296
+
297
+ Calibration (`[a, b]`, `T`) is per model. The numbers in this README were measured on `gemma-4-26b-a4b-it-4bit`.
298
+
299
+ ### Model comparison
300
+
301
+ Measured on 2026-11-13, an Apple silicon Mac with 36 GB, `float32=True`, `orders="reverse"`. Set: 3 short calls,
302
+ 6 questions each — 3 noul, 2 choice, 1 score, 18 in all. Tests: `DECISION_GEM_TEST_MODEL=<repo> pytest -m models`.
303
+
304
+ | Model | `-m models` tests | Load | 18 questions |
305
+ |---|---|---|---|
306
+ | `mlx-community/gemma-4-e2b-it-4bit` | 6 of 6 | 1.4 s | 1.7 s |
307
+ | `mlx-community/gemma-4-e4b-it-4bit` | 6 of 6 | 1.8 s | 3.1 s |
308
+ | `mlx-community/Qwen3-14B-4bit` | 6 of 6 | 3.1 s | 9.2 s |
309
+ | `mlx-community/gemma-4-26b-a4b-it-4bit` (default) | 6 of 6 | 4.6 s | 6.7 s |
310
+
311
+ Gemma 26B is an MoE (~4B active parameters), so it answers faster than the dense Qwen3-14B.
312
+
313
+ ## Install and run
314
+
315
+ Apple silicon only (MLX). The model must already be in the HF cache, or run with `--online` / `offline=False`.
316
+
317
+ ```bash
318
+ pip install decision-gem # or from source: pip install -e ".[test]"
319
+ python -m decision_gem request.json # or: decision-gem request.json; - reads stdin
320
+ python -m decision_gem request.json --calibration cal.json --model <repo or folder>
321
+ pytest # tests without the model; pytest -m models — on the model
322
+ ```
323
+
324
+ ```python
325
+ from decision_gem import DecisionGem
326
+
327
+ email = {"from": '"Secure Bank" <security@secure-bank-verify.top>', # the state: text or a JSON object
328
+ "subject": "Your account has been limited!",
329
+ "body": "Dear customer, we noticed unusual activity. Verify your identity within 24 hours: "
330
+ "http://secure-bank-verify.top/login or your account will be permanently suspended."}
331
+ qs = {"spam": {"type": "noul", "instructions": "Is this email spam or phishing?"},
332
+ "kind": {"type": "choice", "instructions": "What kind of email is it?",
333
+ "criteria": {"phishing": "tries to steal logins or payment data", "ads": "unsolicited advertising",
334
+ "scam": "promises money or prizes", "normal": "an ordinary email"}},
335
+ "urgency": {"type": "score", "instructions": "How hard does the email push the reader to act right away?",
336
+ "criteria": ["no pressure", "some pressure", "strong pressure"]}}
337
+
338
+ gem = DecisionGem() # mlx-community/gemma-4-26b-a4b-it-4bit from the local HF cache
339
+ gem({"state": email, "questions": qs})
340
+ ```
341
+
342
+ The answer (Gemma 26B, 474 input tokens; `legend` and `probabilities` cut):
343
+
344
+ ```json
345
+ {"spam": {"type": "noul", "noul": 1.0, "orders_agree": true},
346
+ "kind": {"type": "choice", "choice": "phishing", "confidence": 1.0, "orders_agree": true},
347
+ "urgency": {"type": "score", "score": 3.0, "confidence": 1.0, "orders_agree": true}}
348
+ ```
349
+
350
+ On an ordinary email from a colleague about meeting notes the same questions give `spam` 0.0, `kind` `normal`,
351
+ `urgency` 1.01.
352
+
353
+ Several states at once (see "Keyed states"):
354
+
355
+ ```python
356
+ with DecisionGem() as gem:
357
+ gem.prefill("mail-1", email1) # number of tokens; ≈0.2 MiB of memory per token until forget / unload
358
+ gem.prefill("mail-2", email2)
359
+ gem.decide(key="mail-2", questions=qs) # the same answer as decide(email2, qs), without reading again
360
+ gem({"key": "mail-1", "questions": qs}) # a Jev request with key instead of state
361
+ gem.save("mail-1", "mail-1.safetensors") # to disk; load() reads it back in ~0.02 s
362
+ gem.forget("mail-1")
363
+ gem.load("mail-1", "mail-1.safetensors") # only a file saved with the same model
364
+ ```
365
+
366
+ The full contract is also in the docstring of `decision_gem/core.py`.