simthinkd 0.2.0__tar.gz → 0.3.0__tar.gz

{simthinkd-0.2.0/src/simthinkd.egg-info → simthinkd-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: simthinkd
-Version: 0.2.0
+Version: 0.3.0
 Summary: A 265k-parameter decision model that picks one action in about 2 ms on one CPU core, inside real-time loops.
 Author: Myeongseongsimjae AX Institute
 License: Apache-2.0
@@ -28,7 +28,9 @@ Requires-Dist: onnx>=1.15; extra == "onnx"
 Requires-Dist: onnxruntime>=1.17; extra == "onnx"
 Dynamic: license-file
 
-<p align="center"><img src="assets/banner.png" alt="SimThink D: a tiny decision model that runs on one CPU core" width="100%"></p>
+<p align="center"><b>English</b> | <a href="README.ko.md">한국어</a> | <a href="README.ja.md">日本語</a></p>
+
+<p align="center"><img src="assets/banner_v2.png" alt="SimThink D: a local backup for cloud decisions" width="100%"></p>
 
 <p align="center">
   <a href="https://pypi.org/project/simthinkd/"><img src="https://img.shields.io/pypi/v/simthinkd" alt="PyPI"></a>
@@ -39,27 +41,23 @@ Dynamic: license-file
   <a href="https://doi.org/10.5281/zenodo.23111615"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.23111615.svg" alt="DOI"></a>
 </p>
 
-<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/"><b>Project page, live demo and video →</b></a></p>
-
-**A tiny decision model that runs on one CPU core.** It has 265,665 parameters. It picks one action in about 2 ms. That is fast enough to decide inside every tick of a game or a control loop. No GPU is needed, not even for training.
-
-<p align="center">
-  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
-</p>
-
-<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+<p align="center"><b>Network down. Decisions stay local.</b></p>
 
-<div align="center">
+SimThink D is a small CPU model for offline backup decisions.
+It has 265,665 parameters and takes about 2 ms per decision on one CPU core.
 
-[Open in Colab](https://colab.research.google.com/github/MSSJ-AI-ORG/simthinkd/blob/main/notebooks/quickstart.ipynb) · [Try it in your browser](https://mssj-ai-org.github.io/simthinkd/#demo) · [Gradio demo](space/) · [Factory twin](examples/factory_twin/) · [Paper](docs/PAPER.md) · [Reproduce the paper](docs/REPRODUCE.md) · [Figures](docs/FIGURES.md) · [Protocol](docs/PROTOCOL.md)
+In the factory simulation, the internet was cut for 15 seconds.
+The local backup got 59 of 63 parts right, with no late decisions.
 
-</div>
+<p align="center"><a href="assets/factory_fallback.mp4"><img src="assets/factory_twin.gif" alt="Factory simulation: the internet is cut for 15 seconds and SimThink D on the factory PC fills the gap" width="100%"></a></p>
 
-### Video: the internet goes down, the line keeps going
+<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/">Project page</a> · <a href="https://mssj-ai-org.github.io/simthinkd/#demo">Try in your browser</a> · <a href="assets/factory_fallback.mp4">Watch the video (72 s)</a> · <a href="examples/factory_twin/">Factory code</a> · <a href="docs/PAPER.md">Paper</a></p>
 
-<a href="assets/factory_fallback.mp4"><img src="assets/factory_fallback_poster.jpg" alt="Internet down. The line kept going." width="100%"></a>
+```bash
+pip install simthinkd
+```
 
-A cloud decision service runs a simulated inspection line, with SimThink D on the factory PC as its backup. When an answer does not come back in time, or the network is cut, SimThink D makes the decision. [Watch the video](assets/factory_fallback.mp4) (72 s) or see it [on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/). The simulator is in [examples/factory_twin](examples/factory_twin/).
+What makes the next decision when your network goes down?
 
 ## Words used here
 
@@ -94,6 +92,23 @@ One call is one decision. A short sentence goes in. One action comes out, with i
 
 Two deciders come with the package: `doom-defend` (stand in the middle and fight) and `doom-corridor` (fight your way down a corridor).
 
+## Many decisions at once
+
+One call can carry several questions about the same situation, and one batch can carry many situations. The answers are the same as asking one by one.
+
+```python
+from simthinkd import Decider
+
+d = Decider("doom-defend")
+states = ["seen: Demon left a30 d5 | enemies 1 | sway left | gun ready | ammo25",
+          "seen: Demon right a30 d5 | enemies 1 | sway right | gun ready | ammo25"]
+answers = d.predict_batch([d.request(s) for s in states])   # many situations in one pass
+print([a["operation"]["choice"] for a in answers])
+# ['TURN_LEFT', 'TURN_RIGHT']
+```
+
+`simthinkd bench --batch` shows where the time goes on your machine. Details and limits: [docs/PARALLEL.md](docs/PARALLEL.md).
+
 ## Train your own in seconds
 
 A new task needs two things: a perception step and a teacher. The `toy` module below is a small made-up task, an inspection station on a factory belt. Swap its examples and actions for your own.
@@ -181,6 +196,12 @@ The second line measures any server that accepts the [decision request](docs/PRO
 
 ## Same CPU, same states
 
+<p align="center">
+  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
+</p>
+
+<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+
 **Read this first.** Laya is a general model and was not trained on this game. Its published speed, about 33 ms per question, is on a GPU. We only had a CPU. So this table compares time inside a real-time loop. It does not compare overall quality.
 
 We used one workstation CPU (6 threads) and 1,050 Doom states, then 10 live games on the same seeds. "Missed ticks" are ticks that passed before the decider answered.
@@ -199,7 +220,9 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 | Any language, any engine | `simthinkd serve doom-defend --port 11890`, then POST the [decision request](docs/PROTOCOL.md) to `/v1/systemone` |
 | Unity / C# | [docs/INTEGRATION_UNITY.md](docs/INTEGRATION_UNITY.md): a client loop that keeps the game running while it waits |
 | Browser | [web/](web/): the same model in plain JavaScript, no server |
+| Java / JVM games and engines | [java/](java/): the same model in plain Java 8, inside your process; export weights with `tools/export_java_weights.py`; a test checks it chooses exactly what Python chooses |
 | A factory line (simulator) | [examples/factory_twin/](examples/factory_twin/): an inspection conveyor with a 400 ms deadline per part |
+| Gradio | [space/](space/): a small web demo you can run locally or on Hugging Face Spaces |
 | MCP (Claude Desktop, Cursor and others) | `pip install "simthinkd[mcp]"`, then `python -m simthinkd.integrations.mcp_server` |
 | LangChain / LangGraph | `from simthinkd.integrations.langchain_tool import simthinkd_tool` |
 
@@ -218,6 +241,13 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 - Probabilities are calibrated for the decider's own task only.
 - A score model returns one number. It gives no probability or error bar with it.
 
+## More
+
+- [Figures from the paper](docs/FIGURES.md)
+- [What you can reproduce](docs/REPRODUCE.md)
+- [Decision request format](docs/PROTOCOL.md)
+- [The factory video on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/)
+
 ## Citation
 
 If you use SimThink D, please cite it with [CITATION.cff](CITATION.cff). GitHub shows a "Cite this repository" button for it.

{simthinkd-0.2.0 → simthinkd-0.3.0}/README.md

@@ -1,4 +1,6 @@
-<p align="center"><img src="assets/banner.png" alt="SimThink D: a tiny decision model that runs on one CPU core" width="100%"></p>
+<p align="center"><b>English</b> | <a href="README.ko.md">한국어</a> | <a href="README.ja.md">日本語</a></p>
+
+<p align="center"><img src="assets/banner_v2.png" alt="SimThink D: a local backup for cloud decisions" width="100%"></p>
 
 <p align="center">
   <a href="https://pypi.org/project/simthinkd/"><img src="https://img.shields.io/pypi/v/simthinkd" alt="PyPI"></a>
@@ -9,27 +11,23 @@
   <a href="https://doi.org/10.5281/zenodo.23111615"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.23111615.svg" alt="DOI"></a>
 </p>
 
-<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/"><b>Project page, live demo and video →</b></a></p>
-
-**A tiny decision model that runs on one CPU core.** It has 265,665 parameters. It picks one action in about 2 ms. That is fast enough to decide inside every tick of a game or a control loop. No GPU is needed, not even for training.
-
-<p align="center">
-  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
-</p>
-
-<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+<p align="center"><b>Network down. Decisions stay local.</b></p>
 
-<div align="center">
+SimThink D is a small CPU model for offline backup decisions.
+It has 265,665 parameters and takes about 2 ms per decision on one CPU core.
 
-[Open in Colab](https://colab.research.google.com/github/MSSJ-AI-ORG/simthinkd/blob/main/notebooks/quickstart.ipynb) · [Try it in your browser](https://mssj-ai-org.github.io/simthinkd/#demo) · [Gradio demo](space/) · [Factory twin](examples/factory_twin/) · [Paper](docs/PAPER.md) · [Reproduce the paper](docs/REPRODUCE.md) · [Figures](docs/FIGURES.md) · [Protocol](docs/PROTOCOL.md)
+In the factory simulation, the internet was cut for 15 seconds.
+The local backup got 59 of 63 parts right, with no late decisions.
 
-</div>
+<p align="center"><a href="assets/factory_fallback.mp4"><img src="assets/factory_twin.gif" alt="Factory simulation: the internet is cut for 15 seconds and SimThink D on the factory PC fills the gap" width="100%"></a></p>
 
-### Video: the internet goes down, the line keeps going
+<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/">Project page</a> · <a href="https://mssj-ai-org.github.io/simthinkd/#demo">Try in your browser</a> · <a href="assets/factory_fallback.mp4">Watch the video (72 s)</a> · <a href="examples/factory_twin/">Factory code</a> · <a href="docs/PAPER.md">Paper</a></p>
 
-<a href="assets/factory_fallback.mp4"><img src="assets/factory_fallback_poster.jpg" alt="Internet down. The line kept going." width="100%"></a>
+```bash
+pip install simthinkd
+```
 
-A cloud decision service runs a simulated inspection line, with SimThink D on the factory PC as its backup. When an answer does not come back in time, or the network is cut, SimThink D makes the decision. [Watch the video](assets/factory_fallback.mp4) (72 s) or see it [on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/). The simulator is in [examples/factory_twin](examples/factory_twin/).
+What makes the next decision when your network goes down?
 
 ## Words used here
 
@@ -64,6 +62,23 @@ One call is one decision. A short sentence goes in. One action comes out, with i
 
 Two deciders come with the package: `doom-defend` (stand in the middle and fight) and `doom-corridor` (fight your way down a corridor).
 
+## Many decisions at once
+
+One call can carry several questions about the same situation, and one batch can carry many situations. The answers are the same as asking one by one.
+
+```python
+from simthinkd import Decider
+
+d = Decider("doom-defend")
+states = ["seen: Demon left a30 d5 | enemies 1 | sway left | gun ready | ammo25",
+          "seen: Demon right a30 d5 | enemies 1 | sway right | gun ready | ammo25"]
+answers = d.predict_batch([d.request(s) for s in states])   # many situations in one pass
+print([a["operation"]["choice"] for a in answers])
+# ['TURN_LEFT', 'TURN_RIGHT']
+```
+
+`simthinkd bench --batch` shows where the time goes on your machine. Details and limits: [docs/PARALLEL.md](docs/PARALLEL.md).
+
 ## Train your own in seconds
 
 A new task needs two things: a perception step and a teacher. The `toy` module below is a small made-up task, an inspection station on a factory belt. Swap its examples and actions for your own.
@@ -151,6 +166,12 @@ The second line measures any server that accepts the [decision request](docs/PRO
 
 ## Same CPU, same states
 
+<p align="center">
+  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
+</p>
+
+<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+
 **Read this first.** Laya is a general model and was not trained on this game. Its published speed, about 33 ms per question, is on a GPU. We only had a CPU. So this table compares time inside a real-time loop. It does not compare overall quality.
 
 We used one workstation CPU (6 threads) and 1,050 Doom states, then 10 live games on the same seeds. "Missed ticks" are ticks that passed before the decider answered.
@@ -169,7 +190,9 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 | Any language, any engine | `simthinkd serve doom-defend --port 11890`, then POST the [decision request](docs/PROTOCOL.md) to `/v1/systemone` |
 | Unity / C# | [docs/INTEGRATION_UNITY.md](docs/INTEGRATION_UNITY.md): a client loop that keeps the game running while it waits |
 | Browser | [web/](web/): the same model in plain JavaScript, no server |
+| Java / JVM games and engines | [java/](java/): the same model in plain Java 8, inside your process; export weights with `tools/export_java_weights.py`; a test checks it chooses exactly what Python chooses |
 | A factory line (simulator) | [examples/factory_twin/](examples/factory_twin/): an inspection conveyor with a 400 ms deadline per part |
+| Gradio | [space/](space/): a small web demo you can run locally or on Hugging Face Spaces |
 | MCP (Claude Desktop, Cursor and others) | `pip install "simthinkd[mcp]"`, then `python -m simthinkd.integrations.mcp_server` |
 | LangChain / LangGraph | `from simthinkd.integrations.langchain_tool import simthinkd_tool` |
 
@@ -188,6 +211,13 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 - Probabilities are calibrated for the decider's own task only.
 - A score model returns one number. It gives no probability or error bar with it.
 
+## More
+
+- [Figures from the paper](docs/FIGURES.md)
+- [What you can reproduce](docs/REPRODUCE.md)
+- [Decision request format](docs/PROTOCOL.md)
+- [The factory video on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/)
+
 ## Citation
 
 If you use SimThink D, please cite it with [CITATION.cff](CITATION.cff). GitHub shows a "Cite this repository" button for it.

{simthinkd-0.2.0 → simthinkd-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "simthinkd"
-version = "0.2.0"
+version = "0.3.0"
 description = "A 265k-parameter decision model that picks one action in about 2 ms on one CPU core, inside real-time loops."
 readme = "README.md"
 requires-python = ">=3.10"

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd/__init__.py

@@ -6,7 +6,7 @@
 from .core import Decider, Decision, available, build_request
 from .score import Score, Scorer
 
-__version__ = '0.2.0'
+__version__ = '0.3.0'
 __all__ = ['Decider', 'Decision', 'Score', 'Scorer', 'available', 'build_request', 'fit', 'fit_score', '__version__']
 
 

simthinkd-0.3.0/src/simthinkd/bench.py

@@ -0,0 +1,103 @@
+"""Does your decider fit inside one game tick? Measures decision time and teacher agreement on recorded states.
+
+    simthinkd bench                                   # the bundled decider, in-process, bundled Doom states
+    simthinkd bench --url http://127.0.0.1:8000/v1/systemone --name my-model
+    simthinkd bench --states my_states.json --tick-hz 60
+
+Any server that accepts the decision request (PROTOCOL.md) and answers {"answers": {"operation": {"choice": ...}}}
+can be measured. States file: {"rows": [{"body": <request>, "teacher": <action>}, ...]}.
+One request at a time; the time is the full round trip seen by the caller.
+"""
+import json
+import statistics
+import time
+import urllib.request
+from pathlib import Path
+
+from .core import Decider
+
+BUNDLED = Path(__file__).resolve().parent / 'data' / 'doom_defend_states.json'
+
+
+def _post(url, body):
+    req = urllib.request.Request(url, data=json.dumps(body).encode(), headers={'Content-Type': 'application/json'})
+    with urllib.request.urlopen(req, timeout=300) as reply:
+        return json.load(reply)['answers']['operation']['choice']
+
+
+def run(states=None, url=None, decider='doom-defend', name=None, tick_hz=35.0, warmup=20, limit=0):
+    rows = json.loads(Path(states or BUNDLED).read_text(encoding='utf8'))['rows']
+    rows = rows[:limit] if limit else rows
+    if url:
+        call, label = (lambda body: _post(url, body)), name or url
+    else:
+        d = decider if isinstance(decider, Decider) else Decider(decider)
+        call, label = (lambda body: d.predict(body)['operation']['choice']), name or f'simthink-d:{d.preset["name"]}'
+    for row in rows[:warmup]:
+        call(row['body'])
+    times, agree = [], 0
+    for row in rows:
+        start = time.perf_counter()
+        choice = call(row['body'])
+        times.append((time.perf_counter() - start) * 1000)
+        agree += choice == row['teacher']
+    q = sorted(times)
+    tick_ms = 1000.0 / tick_hz
+    return {'name': label, 'states': len(rows), 'tick_ms': round(tick_ms, 2),
+            'p50_ms': round(statistics.median(q), 3), 'p95_ms': round(q[max(0, int(0.95 * len(q)) - 1)], 3),
+            'within_one_tick': round(sum(t <= tick_ms for t in times) / len(times), 4),
+            'teacher_agreement': round(agree / len(rows), 4)}
+
+
+def throughput(states=None, decider='doom-defend', sizes=(1, 10, 100, 1000), limit=1000, repeat=3):
+    """Decisions per second, one request at a time (predict) vs batches (predict_batch), both end to end with the same
+    answers; then, in a separate instrumented pass, where batch time goes: encoding text (per request), the network
+    (Policy.scores_batch, one pass per batch) and turning scores into answers. In-process only; best of `repeat` runs."""
+    from .core import answer_from
+    from .policy import encode
+    d = decider if isinstance(decider, Decider) else Decider(decider)
+    rows = json.loads(Path(states or BUNDLED).read_text(encoding='utf8'))['rows']
+    bodies = [rows[i % len(rows)]['body'] for i in range(limit)]
+    d.predict_batch(bodies[:20])
+
+    def best(run):
+        times = []
+        for _ in range(repeat):
+            start = time.perf_counter()
+            run()
+            times.append(time.perf_counter() - start)
+        return min(times)
+
+    out = {'name': f'simthink-d:{d.preset["name"]}', 'decisions': len(bodies), 'sizes': {}}
+    out['one_at_a_time_per_s'] = round(len(bodies) / best(lambda: [d.predict(b) for b in bodies]), 1)
+    for size in sizes:
+        chunks = [bodies[i:i + size] for i in range(0, len(bodies), size)]
+        total = best(lambda: [d.predict_batch(c) for c in chunks])
+        enc = net = dec = 0.0
+        for c in chunks:
+            t0 = time.perf_counter()
+            encoded = [encode(b) for b in c]
+            t1 = time.perf_counter()
+            scores = d.policy.scores_batch([e[0] for e in encoded])
+            t2 = time.perf_counter()
+            [answer_from(d.policy.decode(e, sc)) for e, sc in zip(encoded, scores)]
+            enc, net, dec = enc + t1 - t0, net + t2 - t1, dec + time.perf_counter() - t2
+        part = enc + net + dec
+        out['sizes'][size] = {'per_s': round(len(bodies) / total, 1), 'network_only_per_s': round(len(bodies) / net, 1),
+                              'encode_share': round(enc / part, 3), 'network_share': round(net / part, 3),
+                              'answer_share': round(dec / part, 3)}
+    return out
+
+
+def throughput_report(r):
+    lines = [f"{r['name']}: {r['decisions']} decisions, one at a time {r['one_at_a_time_per_s']}/s"]
+    for size, v in r['sizes'].items():
+        lines.append(f"  batch {size:>5}: {v['per_s']}/s end to end; network alone {v['network_only_per_s']}/s; time: "
+                     f"{v['encode_share']:.0%} encoding text, {v['network_share']:.0%} network, {v['answer_share']:.0%} building answers")
+    return '\n'.join(lines)
+
+
+def report(result):
+    return (f"{result['name']}: median {result['p50_ms']} ms, p95 {result['p95_ms']} ms, "
+            f"{result['within_one_tick']:.1%} of decisions within one {result['tick_ms']} ms tick, "
+            f"teacher agreement {result['teacher_agreement']:.1%} on {result['states']} states")

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd/cli.py

@@ -23,6 +23,7 @@ def main(argv=None):
     s.add_argument('--states')
     s.add_argument('--tick-hz', type=float, default=35.0)
     s.add_argument('--limit', type=int, default=0)
+    s.add_argument('--batch', action='store_true', help='decisions per second, one at a time vs batches of 1, 10, 100, 1000')
     s.add_argument('--json', action='store_true')
     s = sub.add_parser('train', help='train from a folder of protocol rows (needs simthinkd[train])')
     s.add_argument('--data', required=True)
@@ -46,8 +47,12 @@ def main(argv=None):
         serve(a.decider, a.host, a.port, a.delay_ms)
     elif a.cmd == 'bench':
         from . import bench
-        result = bench.run(a.states, a.url, a.decider, a.name, a.tick_hz, limit=a.limit)
-        print(json.dumps(result) if a.json else bench.report(result))
+        if a.batch:
+            result = bench.throughput(a.states, a.decider, limit=a.limit or 1000)
+            print(json.dumps(result) if a.json else bench.throughput_report(result))
+        else:
+            result = bench.run(a.states, a.url, a.decider, a.name, a.tick_hz, limit=a.limit)
+            print(json.dumps(result) if a.json else bench.report(result))
     elif a.cmd == 'train':
         from .train import fit_dir
         d = fit_dir(a.data, a.out, a.steps, a.seed)

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd/core.py

@@ -13,7 +13,7 @@ import time
 from dataclasses import dataclass, field
 from pathlib import Path
 
-from .policy import Policy
+from .policy import Policy, encode
 
 WEIGHTS = Path(__file__).resolve().parent / 'weights'
 
@@ -76,7 +76,11 @@ def build_request(state, actions, goal, *, url='sim://custom', title='', model='
 
 def answer(policy, body):
     """Reply `answers` block for a protocol request (operation, and `<op>_target` when targets were offered)."""
-    chosen, op, conditional, joint, rows, operations = policy.predict(body)
+    return answer_from(policy.predict(body))
+
+
+def answer_from(predicted):
+    chosen, op, conditional, joint, rows, operations = predicted
     op_probs = {key: float(op[i]) for i, key in enumerate(operations)}
     top = max(op_probs, key=op_probs.get)
     answers = {'operation': {'choice': top, 'probabilities': op_probs, 'confidence': op_probs[top]}}
@@ -90,6 +94,44 @@ def answer(policy, body):
     return answers
 
 
+def _question_body(body, question):
+    """One general question as an operation-style request on the same state: each criterion becomes one row."""
+    kind, crit = question.get('type', 'choice'), question.get('criteria')
+    if kind == 'score':
+        levels = list(crit.values()) if isinstance(crit, dict) else list(crit or [])
+        if not 2 <= len(levels) <= 10:
+            raise ValueError('a score question needs 2 to 10 levels')
+        crit = {str(i + 1): level for i, level in enumerate(levels)}
+    elif kind == 'noul':
+        crit = crit or {'true': 'Yes.', 'false': 'No.'}
+        if set(crit) != {'true', 'false'}:
+            raise ValueError('a noul question has exactly the criteria true and false')
+    elif kind != 'choice' or not crit:
+        raise ValueError(f'question type must be choice, score or noul with criteria (got {kind!r})')
+    return {'state': body['state'],
+            'questions': {'operation': {'type': 'choice', 'criteria': crit, 'instructions': question.get('instructions', {})}}}
+
+
+def _general_answer(kind, predicted):
+    op, keys = predicted[1], predicted[5]
+    probs = {k: float(op[i]) for i, k in enumerate(keys)}
+    best = max(probs, key=probs.get)
+    if kind == 'noul':
+        return {'type': 'noul', 'noul': probs['true'], 'probabilities': probs}
+    if kind == 'score':
+        return {'type': 'score', 'score': int(best), 'probabilities': probs, 'confidence': probs[best]}
+    return {'type': 'choice', 'choice': best, 'probabilities': probs, 'confidence': probs[best]}
+
+
+def _jobs(body):
+    """(name, type) per block: the operation question with its `<op>_target` companions is one coupled block (name None);
+    every other question is its own block."""
+    qs = body.get('questions', {})
+    coupled = {'operation'} | {op.lower() + '_target' for op in qs.get('operation', {}).get('criteria', {})}
+    jobs = [(None, None)] if 'operation' in qs else []
+    return jobs + [(name, q.get('type', 'choice')) for name, q in qs.items() if name not in coupled]
+
+
 @dataclass(frozen=True)
 class Decision:
     choice: str
@@ -141,6 +183,34 @@ class Decider:
         """Protocol-level call: request dict in, `answers` dict out (same as the HTTP server)."""
         return answer(self.policy, body)
 
+    def predict_batch(self, bodies):
+        """predict() for many requests at once; the network runs once over all of them."""
+        return [answer_from(p) for p in self.policy.predict_batch(bodies)]
+
+    def ask(self, body):
+        """Several questions about one situation in one call (choice, score, noul). See docs/PARALLEL.md."""
+        return self.ask_batch([body])[0]
+
+    def ask_batch(self, bodies):
+        """ask() for many requests; every question of every request goes through one padded network pass."""
+        bodies = list(bodies)
+        if not bodies:
+            return []
+        jobs, encoded = [], []
+        for i, body in enumerate(bodies):
+            body_jobs = _jobs(body)
+            if not body_jobs:  # checked per request, so a neighbour in the batch never changes the outcome
+                raise ValueError(f'request {i} has no question to answer')
+            for name, kind in body_jobs:
+                jobs.append((i, name, kind))
+                encoded.append(encode(body if name is None else _question_body(body, body['questions'][name])))
+        out = [{'answers': {}, 'meta': {'questions': 0}} for _ in bodies]
+        for (i, name, kind), e, s in zip(jobs, encoded, self.policy.scores_batch([e[0] for e in encoded])):
+            predicted = self.policy.decode(e, s)
+            out[i]['answers'].update(answer_from(predicted) if name is None else {name: _general_answer(kind, predicted)})
+            out[i]['meta']['questions'] += 1
+        return out
+
     def decide(self, state, actions=None, goal=None, tick=0, recent_actions=None):
         """One decision for one situation sentence. Returns a Decision (choice, confidence, probabilities, ms)."""
         body = self.request(state, actions, goal, tick, recent_actions)

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd/policy.py

@@ -173,9 +173,37 @@ class Policy:
         return (z @ w['score.weight'].T + w['score.bias'])[:, 0]
 
     def predict(self, body):
-        x, groups, rows, operations = encode(body)
-        op, conditional, joint = distributions(self.scores(x), groups, self.temperature)
+        return self.decode(encode(body), self.scores)
+
+    def decode(self, encoded, scores):
+        x, groups, rows, operations = encoded
+        op, conditional, joint = distributions(scores if not callable(scores) else scores(x), groups, self.temperature)
         chosen_group = int(op.argmax())
         indices = np.flatnonzero(groups == chosen_group)
         chosen = rows[int(indices[conditional[chosen_group].argmax()])]
         return chosen, op, conditional, joint, rows, operations
+
+    def scores_batch(self, blocks):
+        """Scores for many encoded blocks in one padded pass. A block is the row matrix of one question;
+        context pooling (mean, max) stays inside its block, so blocks never influence each other."""
+        if not len(blocks):
+            return []
+        if any(len(b) == 0 for b in blocks):
+            raise ValueError('every block needs at least one row')
+        sizes = np.array([len(b) for b in blocks])
+        starts = np.concatenate([[0], np.cumsum(sizes)[:-1]])
+        w = self.w
+        # All rows of all blocks go through each layer as one matrix product; only the pooling is per block.
+        h = np.maximum(0, np.concatenate(blocks) @ w['local.weight'].T + w['local.bias'])
+        context = np.concatenate([np.add.reduceat(h, starts) / sizes[:, None].astype(h.dtype), np.maximum.reduceat(h, starts)], axis=1)
+        z = np.concatenate([h, np.repeat(context, sizes, axis=0)], axis=1)
+        z = np.maximum(0, z @ w['context.weight'].T + w['context.bias'])
+        s = (z @ w['score.weight'].T + w['score.bias'])[:, 0]
+        return np.split(s, np.cumsum(sizes)[:-1])
+
+    def predict_batch(self, bodies):
+        """predict() for many requests: encoding is per request, the network runs once over all of them."""
+        encoded = [encode(b) for b in bodies]
+        if not encoded:
+            return []
+        return [self.decode(e, s) for e, s in zip(encoded, self.scores_batch([e[0] for e in encoded]))]

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd/server.py

@@ -5,6 +5,8 @@
 Binds to 127.0.0.1 by default. `--delay-ms` adds a fixed wait after inference (latency-injection experiments).
 """
 import json
+import socket
+import threading
 import time
 from datetime import datetime, timezone
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
@@ -13,6 +15,12 @@ from .core import Decider
 
 
 def make_handler(decider, name, delay_ms):
+    # One inference at a time per process. The threaded server runs one thread per connection, and NumPy's BLAS keeps its
+    # own worker threads; on 2026-10-04 a server fed by four parallel StarCraft games died after ~30 min with
+    # "malloc(): unaligned tcache chunk detected". Inference takes about 1 ms, so serialising it costs little; for more
+    # throughput run several server processes and spread clients over them.
+    infer_lock = threading.Lock()
+
     class Handler(BaseHTTPRequestHandler):
         protocol_version = 'HTTP/1.1'
 
@@ -37,9 +45,10 @@ def make_handler(decider, name, delay_ms):
                 return self._send(404, {'error': 'not found'})
             try:
                 body = json.loads(self.rfile.read(int(self.headers.get('Content-Length', 0))) or b'{}')
-                start = time.perf_counter()
-                answers = decider.predict(body)
-                infer_ms = (time.perf_counter() - start) * 1000
+                with infer_lock:
+                    start = time.perf_counter()
+                    answers = decider.predict(body)
+                    infer_ms = (time.perf_counter() - start) * 1000
             except (ValueError, KeyError, TypeError) as error:
                 return self._send(400, {'error': f'invalid request: {error}'})
             if delay_ms:
@@ -52,9 +61,23 @@ def make_handler(decider, name, delay_ms):
     return Handler
 
 
+class NoDelayHTTPServer(ThreadingHTTPServer):
+    """Turns off Nagle's algorithm on every connection.
+
+    The handler writes the headers and the body separately. On a kept-alive connection the second small write waits
+    for the client's delayed ACK, so every answer took about 40 ms (measured 2026-10-03: median 42 ms kept-alive vs
+    1.4 ms with TCP_NODELAY). Clients that reuse connections (Java HttpURLConnection does) missed 30 ms deadlines.
+    """
+
+    def get_request(self):
+        sock, addr = super().get_request()
+        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+        return sock, addr
+
+
 def serve(decider='doom-defend', host='127.0.0.1', port=11890, delay_ms=0.0, name=None):
     decider = decider if isinstance(decider, Decider) else Decider(decider)
     name = name or f'simthink-d:{decider.preset["name"]}'
-    server = ThreadingHTTPServer((host, port), make_handler(decider, name, delay_ms))
+    server = NoDelayHTTPServer((host, port), make_handler(decider, name, delay_ms))
     print(json.dumps({'listening': f'{host}:{port}', 'weights_sha256': decider.sha256, 'model': name}), flush=True)
     server.serve_forever()

{simthinkd-0.2.0 → simthinkd-0.3.0/src/simthinkd.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: simthinkd
-Version: 0.2.0
+Version: 0.3.0
 Summary: A 265k-parameter decision model that picks one action in about 2 ms on one CPU core, inside real-time loops.
 Author: Myeongseongsimjae AX Institute
 License: Apache-2.0
@@ -28,7 +28,9 @@ Requires-Dist: onnx>=1.15; extra == "onnx"
 Requires-Dist: onnxruntime>=1.17; extra == "onnx"
 Dynamic: license-file
 
-<p align="center"><img src="assets/banner.png" alt="SimThink D: a tiny decision model that runs on one CPU core" width="100%"></p>
+<p align="center"><b>English</b> | <a href="README.ko.md">한국어</a> | <a href="README.ja.md">日本語</a></p>
+
+<p align="center"><img src="assets/banner_v2.png" alt="SimThink D: a local backup for cloud decisions" width="100%"></p>
 
 <p align="center">
   <a href="https://pypi.org/project/simthinkd/"><img src="https://img.shields.io/pypi/v/simthinkd" alt="PyPI"></a>
@@ -39,27 +41,23 @@ Dynamic: license-file
   <a href="https://doi.org/10.5281/zenodo.23111615"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.23111615.svg" alt="DOI"></a>
 </p>
 
-<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/"><b>Project page, live demo and video →</b></a></p>
-
-**A tiny decision model that runs on one CPU core.** It has 265,665 parameters. It picks one action in about 2 ms. That is fast enough to decide inside every tick of a game or a control loop. No GPU is needed, not even for training.
-
-<p align="center">
-  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
-</p>
-
-<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+<p align="center"><b>Network down. Decisions stay local.</b></p>
 
-<div align="center">
+SimThink D is a small CPU model for offline backup decisions.
+It has 265,665 parameters and takes about 2 ms per decision on one CPU core.
 
-[Open in Colab](https://colab.research.google.com/github/MSSJ-AI-ORG/simthinkd/blob/main/notebooks/quickstart.ipynb) · [Try it in your browser](https://mssj-ai-org.github.io/simthinkd/#demo) · [Gradio demo](space/) · [Factory twin](examples/factory_twin/) · [Paper](docs/PAPER.md) · [Reproduce the paper](docs/REPRODUCE.md) · [Figures](docs/FIGURES.md) · [Protocol](docs/PROTOCOL.md)
+In the factory simulation, the internet was cut for 15 seconds.
+The local backup got 59 of 63 parts right, with no late decisions.
 
-</div>
+<p align="center"><a href="assets/factory_fallback.mp4"><img src="assets/factory_twin.gif" alt="Factory simulation: the internet is cut for 15 seconds and SimThink D on the factory PC fills the gap" width="100%"></a></p>
 
-### Video: the internet goes down, the line keeps going
+<p align="center"><a href="https://mssj-ai-org.github.io/simthinkd/">Project page</a> · <a href="https://mssj-ai-org.github.io/simthinkd/#demo">Try in your browser</a> · <a href="assets/factory_fallback.mp4">Watch the video (72 s)</a> · <a href="examples/factory_twin/">Factory code</a> · <a href="docs/PAPER.md">Paper</a></p>
 
-<a href="assets/factory_fallback.mp4"><img src="assets/factory_fallback_poster.jpg" alt="Internet down. The line kept going." width="100%"></a>
+```bash
+pip install simthinkd
+```
 
-A cloud decision service runs a simulated inspection line, with SimThink D on the factory PC as its backup. When an answer does not come back in time, or the network is cut, SimThink D makes the decision. [Watch the video](assets/factory_fallback.mp4) (72 s) or see it [on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/). The simulator is in [examples/factory_twin](examples/factory_twin/).
+What makes the next decision when your network goes down?
 
 ## Words used here
 
@@ -94,6 +92,23 @@ One call is one decision. A short sentence goes in. One action comes out, with i
 
 Two deciders come with the package: `doom-defend` (stand in the middle and fight) and `doom-corridor` (fight your way down a corridor).
 
+## Many decisions at once
+
+One call can carry several questions about the same situation, and one batch can carry many situations. The answers are the same as asking one by one.
+
+```python
+from simthinkd import Decider
+
+d = Decider("doom-defend")
+states = ["seen: Demon left a30 d5 | enemies 1 | sway left | gun ready | ammo25",
+          "seen: Demon right a30 d5 | enemies 1 | sway right | gun ready | ammo25"]
+answers = d.predict_batch([d.request(s) for s in states])   # many situations in one pass
+print([a["operation"]["choice"] for a in answers])
+# ['TURN_LEFT', 'TURN_RIGHT']
+```
+
+`simthinkd bench --batch` shows where the time goes on your machine. Details and limits: [docs/PARALLEL.md](docs/PARALLEL.md).
+
 ## Train your own in seconds
 
 A new task needs two things: a perception step and a teacher. The `toy` module below is a small made-up task, an inspection station on a factory belt. Swap its examples and actions for your own.
@@ -181,6 +196,12 @@ The second line measures any server that accepts the [decision request](docs/PRO
 
 ## Same CPU, same states
 
+<p align="center">
+  <img src="assets/side_by_side.gif" alt="Same Doom game, same seed, same CPU. Left: SimThink D answers every tick. Right: a 421M-parameter general decision model, used as published, misses most ticks while it thinks." width="100%" />
+</p>
+
+<p align="center"><sub>Same game, same seed, same 6-core CPU. Left: SimThink D, 1.9 ms per decision, 1 of 420 ticks missed. Right: Laya, a 421M-parameter open decision model, used as published without training on this game. It takes about 360 ms per decision and misses 390 of 420 ticks. A dark frame means the game moved on before the decider answered.</sub></p>
+
 **Read this first.** Laya is a general model and was not trained on this game. Its published speed, about 33 ms per question, is on a GPU. We only had a CPU. So this table compares time inside a real-time loop. It does not compare overall quality.
 
 We used one workstation CPU (6 threads) and 1,050 Doom states, then 10 live games on the same seeds. "Missed ticks" are ticks that passed before the decider answered.
@@ -199,7 +220,9 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 | Any language, any engine | `simthinkd serve doom-defend --port 11890`, then POST the [decision request](docs/PROTOCOL.md) to `/v1/systemone` |
 | Unity / C# | [docs/INTEGRATION_UNITY.md](docs/INTEGRATION_UNITY.md): a client loop that keeps the game running while it waits |
 | Browser | [web/](web/): the same model in plain JavaScript, no server |
+| Java / JVM games and engines | [java/](java/): the same model in plain Java 8, inside your process; export weights with `tools/export_java_weights.py`; a test checks it chooses exactly what Python chooses |
 | A factory line (simulator) | [examples/factory_twin/](examples/factory_twin/): an inspection conveyor with a 400 ms deadline per part |
+| Gradio | [space/](space/): a small web demo you can run locally or on Hugging Face Spaces |
 | MCP (Claude Desktop, Cursor and others) | `pip install "simthinkd[mcp]"`, then `python -m simthinkd.integrations.mcp_server` |
 | LangChain / LangGraph | `from simthinkd.integrations.langchain_tool import simthinkd_tool` |
 
@@ -218,6 +241,13 @@ SimThink D only knows what its teacher knows. It does not reason, read long text
 - Probabilities are calibrated for the decider's own task only.
 - A score model returns one number. It gives no probability or error bar with it.
 
+## More
+
+- [Figures from the paper](docs/FIGURES.md)
+- [What you can reproduce](docs/REPRODUCE.md)
+- [Decision request format](docs/PROTOCOL.md)
+- [The factory video on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7510142949981057024/)
+
 ## Citation
 
 If you use SimThink D, please cite it with [CITATION.cff](CITATION.cff). GitHub shows a "Cite this repository" button for it.

{simthinkd-0.2.0 → simthinkd-0.3.0}/src/simthinkd.egg-info/SOURCES.txt

@@ -25,9 +25,13 @@ src/simthinkd/weights/SHA256SUMS
 src/simthinkd/weights/doom-corridor.npz
 src/simthinkd/weights/doom-defend.npz
 tests/test_factory_twin.py
+tests/test_java_parity.py
 tests/test_notebook.py
 tests/test_package.py
+tests/test_parallel.py
 tests/test_score.py
+tests/test_server_concurrency.py
+tests/test_server_latency.py
 tests/test_space.py
 tests/test_web_page.py
 tests/test_web_parity.py

simthinkd-0.3.0/tests/test_java_parity.py

@@ -0,0 +1,81 @@
+"""Java runtime parity: java/src/main/java/simthinkd must choose exactly what the Python decider chooses.
+
+    python -X utf8 tests/test_java_parity.py <decider folder> <requests.jsonl> [--synthetic N]
+Compiles the Java runtime (javac --release 8), exports the weights (tools/export_java_weights.py), feeds every request to
+both, and compares the operation choice and, when offered, the target choice. Adds synthetic requests (Korean, emoji,
+punctuation, long texts, recent actions) built from the same decider. Exit 0 only if 100% of choices agree; reports the
+largest probability difference.
+"""
+import json
+import random
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / "src"))
+from simthinkd import Decider  # noqa: E402
+
+WORDS = ["적", "해병", "zealot", "dist", "12", "hp", "40", "near", "멀리", "😀", "—", "a.b", "x_y", "Ⅻ", "fi", "Ünïcode", "\t", "  "]
+
+
+def synthetic(base, rng, n):
+    out = []
+    for _ in range(n):
+        b = json.loads(json.dumps(base))
+        crit = b["questions"].get("attack_target", {}).get("criteria", {})
+        for k in list(crit):
+            crit[k]["description"] = " ".join(rng.choice(WORDS) for _ in range(rng.randint(1, 25)))
+        b["state"]["page"]["text"] = " ".join(rng.choice(WORDS) for _ in range(rng.randint(0, 40)))
+        if rng.random() < 0.5:
+            b["state"]["recent_actions"] = [{"action": rng.choice(["ATTACK", "e0", "e1", "WAIT"]), "page_changed": rng.random() < 0.5}
+                                            for _ in range(rng.randint(1, 6))]
+        out.append(b)
+    return out
+
+
+def main():
+    model, reqs = Path(sys.argv[1]), Path(sys.argv[2])
+    n_syn = int(sys.argv[sys.argv.index("--synthetic") + 1]) if "--synthetic" in sys.argv else 500
+    bodies = [json.loads(l) for l in reqs.read_text(encoding="utf-8").splitlines() if l.strip()]
+    bodies += synthetic(bodies[0], random.Random(5), n_syn)
+    with tempfile.TemporaryDirectory() as t:
+        t = Path(t)
+        smtd = t / "w.smtd"
+        subprocess.run([sys.executable, str(ROOT / "tools" / "export_java_weights.py"), str(model), str(smtd)], check=True, capture_output=True)
+        src = sorted(str(p) for p in (ROOT / "java" / "src" / "main" / "java" / "simthinkd").glob("*.java"))
+        subprocess.run(["javac", "--release", "8", "-encoding", "UTF-8", "-d", str(t / "cls"), *src], check=True)
+        inp = "\n".join(json.dumps(b, ensure_ascii=False) for b in bodies) + "\n"
+        r = subprocess.run(["java", "-cp", str(t / "cls"), "simthinkd.SimThinkD", str(smtd)], input=inp.encode("utf-8"), capture_output=True, check=True)
+        jans = [json.loads(l) for l in r.stdout.decode("utf-8").splitlines() if l.strip()]
+    d = Decider(str(model))
+    assert len(jans) == len(bodies), (len(jans), len(bodies))
+    agree_op = agree_t = n_t = 0
+    max_diff = 0.0
+    bad = []
+    for i, (b, ja) in enumerate(zip(bodies, jans)):
+        pa = d.predict(b)
+        agree_op += pa["operation"]["choice"] == ja["operation"]["choice"]
+        for k, v in pa["operation"]["probabilities"].items():
+            max_diff = max(max_diff, abs(v - ja["operation"]["probabilities"][k]))
+        heads = [h for h in pa if h.endswith("_target")]
+        for h in heads:
+            n_t += 1
+            ok = h in ja and pa[h]["choice"] == ja[h]["choice"]
+            agree_t += ok
+            if h in ja:
+                for k, v in pa[h]["probabilities"].items():
+                    max_diff = max(max_diff, abs(v - ja[h]["probabilities"].get(k, -1)))
+            if not ok and len(bad) < 5:
+                bad.append({"i": i, "py": pa[h]["choice"], "java": ja.get(h, {}).get("choice"),
+                            "py_p": sorted(pa[h]["probabilities"].values())[-2:]})
+    ms = sorted(j["_ms"] for j in jans)
+    rep = {"requests": len(bodies), "operation_agree": agree_op, "target_questions": n_t, "target_agree": agree_t,
+           "max_prob_diff": max_diff, "java_ms_p50": ms[len(ms) // 2], "java_ms_p95": ms[int(len(ms) * 0.95)], "mismatches": bad}
+    print(json.dumps(rep, ensure_ascii=False))
+    sys.exit(0 if agree_op == len(bodies) and agree_t == n_t else 1)
+
+
+if __name__ == "__main__":
+    main()

simthinkd-0.3.0/tests/test_parallel.py

@@ -0,0 +1,194 @@
+"""Parallel questions (Decider.ask) and batch inference (predict_batch, ask_batch): answers must not change."""
+import copy
+import json
+from pathlib import Path
+
+try:
+    import pytest
+except ImportError:  # CI runs this file as a script without pytest
+    pytest = None
+
+from simthinkd import Decider
+
+STATES = Path(__file__).resolve().parent.parent / 'src' / 'simthinkd' / 'data' / 'doom_defend_states.json'
+parametrize = pytest.mark.parametrize if pytest else (lambda *a, **k: (lambda f: f))
+TOL = 1e-6  # float32 network: a padded batch may round differently in the last bits, never in the choice
+
+
+def bodies(n=300):
+    return [r['body'] for r in json.loads(STATES.read_text(encoding='utf8'))['rows'][:n]]
+
+
+def assert_same(a, b):
+    assert a.keys() == b.keys()
+    for q in a:
+        assert a[q]['choice'] == b[q]['choice']
+        assert a[q]['probabilities'].keys() == b[q]['probabilities'].keys()
+        for k, p in a[q]['probabilities'].items():
+            assert abs(p - b[q]['probabilities'][k]) <= TOL
+
+
+def with_question(body, name, question):
+    body = copy.deepcopy(body)
+    body['questions'][name] = question
+    return body
+
+
+DANGER = {'type': 'choice', 'instructions': {'goal': 'Which side is more dangerous?'},
+          'criteria': {'left': 'More enemies on the left.', 'right': 'More enemies on the right.', 'none': 'No enemy close.'}}
+URGENCY = {'type': 'score', 'instructions': {'goal': 'How urgent is the situation?'},
+           'criteria': ['calm', 'some pressure', 'under attack', 'about to die']}
+RELOAD = {'type': 'noul', 'instructions': {'goal': 'Should we save ammunition now?'}}
+
+
+@parametrize('preset', ['doom-defend', 'doom-corridor'])
+def test_predict_batch_equals_one_at_a_time(preset):
+    d = Decider(preset)
+    xs = bodies() if preset == 'doom-defend' else [d.request(s) for s in (
+        'enemies: ShotgunGuy left near offset15 | goal: armor ahead distant | body healthy | ammo stocked',
+        'enemies: none | goal: armor ahead near | body hurt | ammo low',
+        'enemies: Zombieman right near offset5 ; ShotgunGuy left far offset30 | goal: armor ahead distant | body healthy | ammo stocked')]
+    for single, batched in zip([d.predict(b) for b in xs], d.predict_batch(xs)):
+        assert_same(single, batched)
+
+
+def test_ask_on_a_plain_request_equals_predict():
+    d = Decider('doom-defend')
+    for body in bodies(100):
+        assert_same(d.ask(body)['answers'], d.predict(body))
+
+
+def test_extra_questions_do_not_change_the_operation_answer():
+    d = Decider('doom-defend')
+    for body in bodies(100):
+        plain = d.predict(body)
+        multi = d.ask(with_question(with_question(with_question(body, 'danger', DANGER), 'urgency', URGENCY), 'reload', RELOAD))
+        assert_same({k: multi['answers'][k] for k in plain}, plain)
+        assert multi['meta']['questions'] == 4
+
+
+def test_questions_stay_independent_of_each_other():
+    d = Decider('doom-defend')
+    other = {'type': 'choice', 'instructions': {'goal': 'Pick a colour.'},
+             'criteria': {f'c{i}': f'colour number {i}' for i in range(40)}}  # many rows: changes the padded shape
+    for body in bodies(50):
+        alone = d.ask(with_question(body, 'danger', DANGER))['answers']['danger']
+        mixed = d.ask(with_question(with_question(body, 'danger', DANGER), 'colour', other))['answers']['danger']
+        assert alone['choice'] == mixed['choice']
+        assert all(abs(alone['probabilities'][k] - mixed['probabilities'][k]) <= TOL for k in alone['probabilities'])
+
+
+def test_score_and_noul_answers_have_their_shape():
+    d = Decider('doom-defend')
+    a = d.ask(with_question(with_question(bodies(1)[0], 'urgency', URGENCY), 'reload', RELOAD))['answers']
+    assert a['urgency']['type'] == 'score' and 1 <= a['urgency']['score'] <= 4
+    assert abs(sum(a['urgency']['probabilities'].values()) - 1) <= TOL
+    assert a['reload']['type'] == 'noul' and 0 <= a['reload']['noul'] <= 1
+    assert set(a['reload']['probabilities']) == {'true', 'false'}
+
+
+def test_ask_batch_equals_ask():
+    d = Decider('doom-defend')
+    xs = [with_question(b, 'danger', DANGER) if i % 2 else b for i, b in enumerate(bodies(60))]
+    for one, many in zip([d.ask(b) for b in xs], d.ask_batch(xs)):
+        assert one['meta'] == many['meta']
+        assert_same({k: v for k, v in one['answers'].items() if 'choice' in v},
+                    {k: v for k, v in many['answers'].items() if 'choice' in v})
+
+
+def test_question_only_request_without_operation():
+    d = Decider('doom-defend')
+    body = {'state': bodies(1)[0]['state'], 'questions': {'danger': DANGER, 'reload': RELOAD}}
+    a = d.ask(body)
+    assert set(a['answers']) == {'danger', 'reload'} and a['meta']['questions'] == 2
+
+
+@parametrize('bad', [{'type': 'score', 'criteria': ['only one']},
+                                 {'type': 'noul', 'criteria': {'yes': 'a', 'no': 'b'}},
+                                 {'type': 'rank', 'criteria': {'a': 'a'}},
+                                 {'type': 'choice'}])
+def test_bad_questions_are_refused(bad):
+    try:
+        Decider('doom-defend').ask(with_question(bodies(1)[0], 'bad', bad))
+    except ValueError:
+        return
+    raise AssertionError(f'bad question accepted: {bad}')
+
+
+def target_body(n, seed):
+    """An operation request with an ATTACK target question of n enemies and a WAIT operation without targets."""
+    enemies = {f'e{i}': {'element': f'[enemy] e{i}', 'description': f'Zergling dist {10 + 7 * ((i + seed) % 9)} hp {20 + 5 * i}'}
+               for i in range(n)}
+    return {'state': {'page': {'url': 'sim://custom', 'title': '', 'text': f'me: Marine | enemies {n}'}, 'elements': [], 'recent_actions': []},
+            'questions': {'operation': {'type': 'choice', 'instructions': {'goal': 'Win the fight.'},
+                                        'criteria': {'ATTACK': 'Attack one enemy.', 'WAIT': 'Hold position.'}},
+                          'attack_target': {'type': 'choice', 'criteria': enemies}}}
+
+
+def test_targets_with_unequal_counts_route_and_batch_correctly():
+    d = Decider('doom-defend')
+    xs = [target_body(n, s) for s in range(3) for n in (1, 2, 5, 9, 17)]
+    singles = [d.predict(b) for b in xs]
+    for single, batched, asked in zip(singles, d.predict_batch(xs), d.ask_batch(xs)):
+        assert_same(single, batched)
+        assert_same(single, asked['answers'])
+
+
+def test_scores_batch_equals_scores_block_by_block():
+    from simthinkd.policy import encode
+    d = Decider('doom-defend')
+    blocks = [encode(b)[0] for b in bodies(20)] + [encode(target_body(9, 1))[0]]
+    blocks += [blocks[0][:1], blocks[-1][:3]]  # single-row and cut blocks, mixed sizes
+    for block, s in zip(blocks, d.policy.scores_batch(blocks)):
+        single = d.policy.scores(block)
+        assert s.dtype == single.dtype and s.shape == single.shape
+        # raw scores are float32 logits: compare relative to their size (probabilities are checked at 1e-6 elsewhere)
+        assert all(abs(float(a) - float(b)) <= 1e-5 * max(1.0, abs(float(b))) for a, b in zip(s, single))
+
+
+def test_empty_inputs():
+    d = Decider('doom-defend')
+    assert d.predict_batch([]) == [] and d.ask_batch([]) == [] and d.policy.scores_batch([]) == []
+    assert len(d.ask_batch(iter(bodies(2)))) == 2  # any iterable, read once
+    from simthinkd.policy import encode
+    x = encode(bodies(1)[0])[0]
+    empty = {'state': bodies(1)[0]['state'], 'questions': {}}
+    for batch in ([empty], [bodies(1)[0], empty], [empty, bodies(1)[0]]):  # refused alone and beside a valid request
+        try:
+            d.ask_batch(batch)
+        except ValueError:
+            continue
+        raise AssertionError('a request without questions was accepted')
+    for blocks in ([x, x[:0]], [x[:0], x]):
+        try:
+            d.policy.scores_batch(blocks)
+        except ValueError:
+            continue
+        raise AssertionError('an empty block was accepted')
+
+
+def test_score_and_noul_values_match_their_probabilities_in_batches():
+    d = Decider('doom-defend')
+    xs = [with_question(with_question(b, 'urgency', URGENCY), 'reload', RELOAD) for b in bodies(40)]
+    for one, many in zip([d.ask(b) for b in xs], d.ask_batch(xs)):
+        for a in (one['answers'], many['answers']):
+            u, r = a['urgency'], a['reload']
+            assert u['score'] == int(max(u['probabilities'], key=u['probabilities'].get))
+            assert r['noul'] == r['probabilities']['true']
+        assert one['answers']['urgency']['score'] == many['answers']['urgency']['score']
+        assert abs(one['answers']['reload']['noul'] - many['answers']['reload']['noul']) <= TOL
+
+
+if __name__ == '__main__':  # CI runs test files as scripts
+    cases = [(test_predict_batch_equals_one_at_a_time, p) for p in ('doom-defend', 'doom-corridor')]
+    cases += [(f, None) for f in (test_ask_on_a_plain_request_equals_predict, test_extra_questions_do_not_change_the_operation_answer,
+                                  test_questions_stay_independent_of_each_other, test_score_and_noul_answers_have_their_shape,
+                                  test_ask_batch_equals_ask, test_question_only_request_without_operation,
+                                  test_targets_with_unequal_counts_route_and_batch_correctly, test_scores_batch_equals_scores_block_by_block,
+                                  test_empty_inputs, test_score_and_noul_values_match_their_probabilities_in_batches)]
+    for f, arg in cases:
+        f(arg) if arg else f()
+    for bad in [{'type': 'score', 'criteria': ['only one']}, {'type': 'noul', 'criteria': {'yes': 'a', 'no': 'b'}},
+                {'type': 'rank', 'criteria': {'a': 'a'}}, {'type': 'choice'}]:
+        test_bad_questions_are_refused(bad)
+    print(f'parallel tests: {len(cases) + 4} passed')

simthinkd-0.3.0/tests/test_server_concurrency.py

@@ -0,0 +1,75 @@
+"""Concurrent clients must get the same answers as sequential ones, and inference must never overlap inside one process
+(regression: a server fed by four parallel games died with a malloc error on 2026-10-04; inference is now serialised)."""
+import http.client
+import json
+import threading
+
+from simthinkd.core import Decider
+from simthinkd.server import NoDelayHTTPServer, make_handler
+
+
+class CountingDecider:
+    """Wraps a Decider and records the largest number of predict calls running at the same time."""
+
+    def __init__(self, inner):
+        self.inner, self.sha256, self.preset = inner, inner.sha256, inner.preset
+        self.active = self.peak = 0
+        self.lock = threading.Lock()
+
+    def predict(self, body):
+        with self.lock:
+            self.active += 1
+            self.peak = max(self.peak, self.active)
+        try:
+            return self.inner.predict(body)
+        finally:
+            with self.lock:
+                self.active -= 1
+
+
+def _post(port, body):
+    conn = http.client.HTTPConnection('127.0.0.1', port, timeout=10)
+    conn.request('POST', '/v1/systemone', json.dumps(body), {'Content-Type': 'application/json'})
+    return json.loads(conn.getresponse().read())['answers']
+
+
+def test_concurrent_answers_match_and_never_overlap():
+    inner = Decider('doom-defend')
+    body = inner.example_request() if hasattr(inner, 'example_request') else None
+    decider = CountingDecider(inner)
+    server = NoDelayHTTPServer(('127.0.0.1', 0), make_handler(decider, 'test', 0.0))
+    threading.Thread(target=server.serve_forever, daemon=True).start()
+    port = server.server_address[1]
+    try:
+        if body is None:
+            body = {'state': {'page': {'url': 'sim://custom', 'title': '', 'text': 'enemy left'},
+                              'elements': [{'id': 'A', 'role': 'button', 'label': 'A'}, {'id': 'B', 'role': 'button', 'label': 'B'}],
+                              'recent_actions': []},
+                    'questions': {'operation': {'type': 'choice', 'criteria': {'A': {'element': '[button] A', 'description': 'go left'},
+                                                                                'B': {'element': '[button] B', 'description': 'go right'}}}}}
+        expected = _post(port, body)
+        results, errors = [], []
+
+        def worker():
+            try:
+                for _ in range(20):
+                    results.append(_post(port, body))
+            except Exception as error:  # noqa: BLE001 - collected and asserted below
+                errors.append(repr(error))
+
+        threads = [threading.Thread(target=worker) for _ in range(16)]
+        for t in threads:
+            t.start()
+        for t in threads:
+            t.join()
+        assert not errors, errors[:3]
+        assert len(results) == 320
+        assert all(r == expected for r in results)
+        assert decider.peak == 1, decider.peak
+    finally:
+        server.shutdown()
+
+
+if __name__ == "__main__":
+    test_concurrent_answers_match_and_never_overlap()
+    print("server concurrency test passed")

simthinkd-0.3.0/tests/test_server_latency.py

@@ -0,0 +1,30 @@
+"""The HTTP server must answer fast on a kept-alive connection (regression: ~40 ms Nagle/delayed-ACK stall in 0.2.0)."""
+import http.client
+import statistics
+import threading
+import time
+
+from simthinkd.core import Decider
+from simthinkd.server import NoDelayHTTPServer, make_handler
+
+
+def test_keep_alive_answers_fast():
+    decider = Decider('doom-defend')
+    server = NoDelayHTTPServer(('127.0.0.1', 0), make_handler(decider, 'test', 0.0))
+    threading.Thread(target=server.serve_forever, daemon=True).start()
+    try:
+        conn = http.client.HTTPConnection('127.0.0.1', server.server_address[1], timeout=2)
+        times = []
+        for _ in range(30):
+            start = time.perf_counter()
+            conn.request('GET', '/health')
+            conn.getresponse().read()
+            times.append((time.perf_counter() - start) * 1000)
+        assert statistics.median(times) < 20, times
+    finally:
+        server.shutdown()
+
+
+if __name__ == "__main__":
+    test_keep_alive_answers_fast()
+    print("server latency test passed")

simthinkd-0.2.0/src/simthinkd/bench.py

@@ -1,55 +0,0 @@
-"""Does your decider fit inside one game tick? Measures decision time and teacher agreement on recorded states.
-
-    simthinkd bench                                   # the bundled decider, in-process, bundled Doom states
-    simthinkd bench --url http://127.0.0.1:8000/v1/systemone --name my-model
-    simthinkd bench --states my_states.json --tick-hz 60
-
-Any server that accepts the decision request (PROTOCOL.md) and answers {"answers": {"operation": {"choice": ...}}}
-can be measured. States file: {"rows": [{"body": <request>, "teacher": <action>}, ...]}.
-One request at a time; the time is the full round trip seen by the caller.
-"""
-import json
-import statistics
-import time
-import urllib.request
-from pathlib import Path
-
-from .core import Decider
-
-BUNDLED = Path(__file__).resolve().parent / 'data' / 'doom_defend_states.json'
-
-
-def _post(url, body):
-    req = urllib.request.Request(url, data=json.dumps(body).encode(), headers={'Content-Type': 'application/json'})
-    with urllib.request.urlopen(req, timeout=300) as reply:
-        return json.load(reply)['answers']['operation']['choice']
-
-
-def run(states=None, url=None, decider='doom-defend', name=None, tick_hz=35.0, warmup=20, limit=0):
-    rows = json.loads(Path(states or BUNDLED).read_text(encoding='utf8'))['rows']
-    rows = rows[:limit] if limit else rows
-    if url:
-        call, label = (lambda body: _post(url, body)), name or url
-    else:
-        d = decider if isinstance(decider, Decider) else Decider(decider)
-        call, label = (lambda body: d.predict(body)['operation']['choice']), name or f'simthink-d:{d.preset["name"]}'
-    for row in rows[:warmup]:
-        call(row['body'])
-    times, agree = [], 0
-    for row in rows:
-        start = time.perf_counter()
-        choice = call(row['body'])
-        times.append((time.perf_counter() - start) * 1000)
-        agree += choice == row['teacher']
-    q = sorted(times)
-    tick_ms = 1000.0 / tick_hz
-    return {'name': label, 'states': len(rows), 'tick_ms': round(tick_ms, 2),
-            'p50_ms': round(statistics.median(q), 3), 'p95_ms': round(q[max(0, int(0.95 * len(q)) - 1)], 3),
-            'within_one_tick': round(sum(t <= tick_ms for t in times) / len(times), 4),
-            'teacher_agreement': round(agree / len(rows), 4)}
-
-
-def report(result):
-    return (f"{result['name']}: median {result['p50_ms']} ms, p95 {result['p95_ms']} ms, "
-            f"{result['within_one_tick']:.1%} of decisions within one {result['tick_ms']} ms tick, "
-            f"teacher agreement {result['teacher_agreement']:.1%} on {result['states']} states")