tinybaselines 0.1.0a4__tar.gz → 0.1.0a5__tar.gz

tinybaselines-0.1.0a5/MANIFEST.in

@@ -0,0 +1 @@
+recursive-include examples *.py *.txt *.json *.jsonl *.c *.png README.md

{tinybaselines-0.1.0a4/tinybaselines.egg-info → tinybaselines-0.1.0a5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tinybaselines
-Version: 0.1.0a4
+Version: 0.1.0a5
 Summary: PPO in tinygrad.
 License-Expression: MIT
 Requires-Python: >=3.11
@@ -21,7 +21,7 @@ Use Python 3.11 and clang. The library install does not include Gymnasium.
 ```sh
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install tinybaselines==0.1.0a4
+python -m pip install tinybaselines==0.1.0a5
 ```
 
 Install Gymnasium and Box2D to run this example:
@@ -79,6 +79,10 @@ Each `learn` call starts fresh episodes and a new transition counter.
 
 ## Repository examples
 
+Example READMEs and screenshots: [LunarLander](examples/lunar_lander/README.md),
+[CartPole](examples/cartpole/README.md), [Pendulum](examples/pendulum/README.md),
+and [Squared Continuous](examples/puffer/README.md).
+
 From a checkout, run the full LunarLander trainer:
 
 ```sh

{tinybaselines-0.1.0a4 → tinybaselines-0.1.0a5}/README.md

@@ -9,7 +9,7 @@ Use Python 3.11 and clang. The library install does not include Gymnasium.
 ```sh
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install tinybaselines==0.1.0a4
+python -m pip install tinybaselines==0.1.0a5
 ```
 
 Install Gymnasium and Box2D to run this example:
@@ -67,6 +67,10 @@ Each `learn` call starts fresh episodes and a new transition counter.
 
 ## Repository examples
 
+Example READMEs and screenshots: [LunarLander](examples/lunar_lander/README.md),
+[CartPole](examples/cartpole/README.md), [Pendulum](examples/pendulum/README.md),
+and [Squared Continuous](examples/puffer/README.md).
+
 From a checkout, run the full LunarLander trainer:
 
 ```sh

tinybaselines-0.1.0a5/examples/cartpole/README.md

@@ -0,0 +1,5 @@
+# CartPole
+
+![The pole upright over the cart](checkpoint.png)
+
+Step 450 of a 500-step sampled episode from a reloaded PPO checkpoint.

tinybaselines-0.1.0a5/examples/lunar_lander/README.md

@@ -0,0 +1,6 @@
+# LunarLander
+
+![The lander resting between the landing flags](checkpoint.png)
+
+Safe landing in a sampled episode from a reloaded PPO checkpoint.
+The episode return was 219.8.

tinybaselines-0.1.0a5/examples/pendulum/README.md

@@ -0,0 +1,6 @@
+# Pendulum
+
+![The pendulum balancing near upright](checkpoint.png)
+
+Balancing near upright at step 150. The reloaded PPO policy scored −123.9
+in this sampled episode.

tinybaselines-0.1.0a5/examples/puffer/README.md

@@ -0,0 +1,57 @@
+# PufferLib Squared Continuous
+
+![The cyan agent one diagonal move from the red target](checkpoint.png)
+
+Live board one move before the reloaded PPO policy earned +1. The next step
+reached the target and reset the board.
+
+Native CPU smoke test on macOS. It calls PufferLib's reset and step functions
+directly and opens no window. Run from the tinybaselines checkout:
+
+```sh
+puffer_smoke=$(mktemp -d)
+git init "$puffer_smoke/pufferlib"
+git -C "$puffer_smoke/pufferlib" fetch --depth=1 https://github.com/PufferAI/PufferLib.git ea7f989371270d2948f96728bda8aec5e7634a4c
+git -C "$puffer_smoke/pufferlib" checkout --detach FETCH_HEAD
+curl -fL https://github.com/raysan5/raylib/releases/download/5.5/raylib-5.5_macos.tar.gz \
+  -o "$puffer_smoke/raylib.tar.gz"
+tar -xzf "$puffer_smoke/raylib.tar.gz" -C "$puffer_smoke"
+clang -O2 -I"$puffer_smoke/pufferlib" -I"$puffer_smoke/pufferlib/src" \
+  -I"$puffer_smoke/raylib-5.5_macos/include" examples/puffer/squared_smoke.c \
+  "$puffer_smoke/raylib-5.5_macos/lib/libraylib.a" \
+  -framework Cocoa -framework IOKit -framework CoreVideo -framework OpenGL \
+  -o "$puffer_smoke/squared_smoke"
+"$puffer_smoke/squared_smoke"
+```
+
+With tinybaselines installed, build the Python adapter's shared library and run
+its reset checks and one PPO update:
+
+```sh
+clang -O2 -dynamiclib -I"$puffer_smoke/pufferlib" -I"$puffer_smoke/pufferlib/src" \
+  -I"$puffer_smoke/raylib-5.5_macos/include" examples/puffer/squared_bridge.c \
+  "$puffer_smoke/raylib-5.5_macos/lib/libraylib.a" \
+  -framework Cocoa -framework IOKit -framework CoreVideo -framework OpenGL \
+  -o "$puffer_smoke/libsquared.dylib"
+DEV=CPU BEAM=0 python -m examples.puffer.smoke "$puffer_smoke/libsquared.dylib"
+```
+
+Train for 204,800 transitions across four environments:
+
+```sh
+DEV=CPU BEAM=0 python -m examples.puffer.train "$puffer_smoke/libsquared.dylib" \
+  --output runs/puffer-squared-ppo
+```
+
+The run saves `ppo.safetensors` and records goals over 100 sampled episodes
+before training and after checkpoint reload.
+
+The default observation is the centered agent and target coordinates, divided by
+5. Use `--observations board` for the flat grid or `--observations relative` for
+the agent coordinates and signed target offset. Use the same observation mode
+when evaluating a saved policy.
+
+At the same budget, three runs per mode reached 56/300 goals with the board,
+299/300 with absolute coordinates, and 296/300 with relative coordinates. Each
+score uses 100 fresh sampled episodes per reloaded checkpoint.
+[Settings and episode returns](results/observations.jsonl).

tinybaselines-0.1.0a5/examples/puffer/env.py

@@ -0,0 +1,83 @@
+from ctypes import CDLL, c_uint, c_void_p
+
+import numpy as np
+
+
+class SquaredVectorEnv:
+    """Headless native environments. Observations and rewards reuse their buffers."""
+
+    metadata = {'autoreset_mode': 'SameStep'}
+
+    def __init__(self, library, num_envs=4):
+        if num_envs <= 0:
+            raise ValueError('num_envs must be positive')
+        self.num_envs = num_envs
+        self._lib = CDLL(str(library))
+        self._lib.squared_create.argtypes = [c_void_p] * 4 + [c_uint]
+        self._lib.squared_create.restype = c_void_p
+        for name in ('puf_reset', 'puf_step', 'squared_destroy'):
+            function = getattr(self._lib, name)
+            function.argtypes = [c_void_p]
+            function.restype = None
+
+        self._observations = np.empty((num_envs, 121), dtype=np.uint8)
+        self._actions = np.zeros((num_envs, 2), dtype=np.float32)
+        self._rewards = np.zeros(num_envs, dtype=np.float32)
+        self._terminals = np.zeros(num_envs, dtype=np.float32)
+        self._envs = []
+        try:
+            for i, seed in enumerate(np.random.SeedSequence().generate_state(num_envs)):
+                handle = self._lib.squared_create(
+                    self._observations[i].ctypes.data, self._actions[i].ctypes.data,
+                    self._rewards[i:i+1].ctypes.data, self._terminals[i:i+1].ctypes.data, int(seed))
+                if not handle:
+                    raise MemoryError('could not allocate Squared Continuous')
+                self._envs.append(handle)
+        except BaseException:
+            self.close()
+            raise
+
+    def reset(self):
+        if not self._envs:
+            raise RuntimeError('environment is closed')
+        for handle in self._envs:
+            self._lib.puf_reset(handle)
+        self._rewards.fill(0)
+        self._terminals.fill(0)
+        return self._observations, {}
+
+    def step(self, actions):
+        if not self._envs:
+            raise RuntimeError('environment is closed')
+        self._actions[:] = actions
+        for handle in self._envs:
+            self._lib.puf_step(handle)
+        # The native timeout is a terminal loss, with an immediate reset.
+        return self._observations, self._rewards, self._terminals.astype(bool), np.zeros(self.num_envs, bool), {}
+
+    def close(self):
+        for handle in self._envs:
+            self._lib.squared_destroy(handle)
+        self._envs.clear()
+
+
+class CoordinateSquaredVectorEnv(SquaredVectorEnv):
+    def __init__(self, library, num_envs=4, *, relative):
+        super().__init__(library, num_envs)
+        self.relative = relative
+
+    def _observe(self, board):
+        def position(tile):
+            return np.column_stack(np.divmod((board == tile).argmax(axis=1), 11))
+
+        agent, target = position(1), position(2)
+        target_features = target - agent if self.relative else target - 5
+        return np.concatenate((agent - 5, target_features), axis=1).astype(np.float32) / 5
+
+    def reset(self):
+        board, info = super().reset()
+        return self._observe(board), info
+
+    def step(self, actions):
+        board, *transition = super().step(actions)
+        return self._observe(board), *transition

tinybaselines-0.1.0a5/examples/puffer/results/observations.jsonl

@@ -0,0 +1,10 @@
+{"event":"config","platform":"macOS-27.0.1-arm64-arm-64bit","python":"3.11.14","versions":{"tinygrad":"0.14.0","numpy":"2.4.6"},"library_sha256":"c83b0740db881ebd616282528e4b224a456d92074b7666b156edaff8f744c0c1","device":"CPU","num_envs":4,"requested_timesteps":204800,"rollout_steps":2048,"deterministic":false,"policy":{"action_dim":2,"hidden_dim":64,"actor_layers":2,"critic_layers":2},"ppo":{"pi_lr":0.0003000000142492354,"vf_lr":0.0010000000474974513,"gamma":0.99,"lamda":0.95,"K_epochs":4,"eps_clip":0.2,"batch_size":256,"vf_coef":0.5,"entropy_coef":0.001},"modes":{"board":{"source_commit":"5629a259ba3d9a6f8e1ea93e7b72bc41913eee64","state_dim":121},"relative":{"source_commit":"5629a259ba3d9a6f8e1ea93e7b72bc41913eee64","state_dim":4},"absolute":{"source_commit":"5629a259ba3d9a6f8e1ea93e7b72bc41913eee64","state_dim":4}},"evaluation_episodes":100,"run_order":["board-1","relative-1","relative-2","board-2","board-3","relative-3","absolute-1","absolute-2","absolute-3"]}
+{"event":"result","observations":"board","trial":1,"training_seconds":67.8307276670239,"initial":{"goals":10,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0]},"final":{"goals":25,"episodes":100,"episode_returns":[-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,-1.0]}}
+{"event":"result","observations":"board","trial":2,"training_seconds":65.84727108303923,"initial":{"goals":4,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0]},"final":{"goals":18,"episodes":100,"episode_returns":[1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0]}}
+{"event":"result","observations":"board","trial":3,"training_seconds":67.56618995795725,"initial":{"goals":4,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0]},"final":{"goals":13,"episodes":100,"episode_returns":[-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0]}}
+{"event":"result","observations":"relative","trial":1,"training_seconds":61.09397924999939,"initial":{"goals":16,"episodes":100,"episode_returns":[-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0]},"final":{"goals":99,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,-1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}
+{"event":"result","observations":"relative","trial":2,"training_seconds":60.07157345901942,"initial":{"goals":16,"episodes":100,"episode_returns":[1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0]},"final":{"goals":100,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}
+{"event":"result","observations":"relative","trial":3,"training_seconds":61.71542754204711,"initial":{"goals":10,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0]},"final":{"goals":97,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,-1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,-1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,-1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}
+{"event":"result","observations":"absolute","trial":1,"training_seconds":61.760505666956306,"initial":{"goals":13,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0]},"final":{"goals":99,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,-1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}
+{"event":"result","observations":"absolute","trial":2,"training_seconds":59.79912408301607,"initial":{"goals":16,"episodes":100,"episode_returns":[1.0,1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0]},"final":{"goals":100,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}
+{"event":"result","observations":"absolute","trial":3,"training_seconds":60.650998707977124,"initial":{"goals":7,"episodes":100,"episode_returns":[-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0]},"final":{"goals":100,"episodes":100,"episode_returns":[1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]}}

tinybaselines-0.1.0a5/examples/puffer/smoke.py

@@ -0,0 +1,43 @@
+import sys
+from contextlib import closing
+
+import numpy as np
+
+from tinybaselines import GaussianActorCritic, PPO, learn
+from .env import SquaredVectorEnv
+
+
+def main(library):
+    with closing(SquaredVectorEnv(library)) as env:
+        observations, _ = env.reset()
+        observations[observations == 2] = 0
+        observations[:, 61] = 2  # One tile right of the centered agent.
+        actions = np.zeros((4, 2), dtype=np.float32)
+        actions[:2, 1] = 1
+        observations, rewards, terminated, truncated, _ = env.step(actions)
+        np.testing.assert_array_equal(rewards, [1, 1, 0, 0])
+        np.testing.assert_array_equal(terminated, [True, True, False, False])
+        assert not truncated.any() and np.all(observations[2:, 61] == 2)
+        assert np.all(observations[:, 60] == 1)
+        env.reset()
+        actions.fill(0)
+        for _ in range(33):
+            _, rewards, terminated, truncated, _ = env.step(actions)
+            assert np.all(rewards == 0) and not terminated.any() and not truncated.any()
+        observations, rewards, terminated, truncated, _ = env.step(actions)
+        assert np.all(rewards == -1) and terminated.all() and not truncated.any()
+        assert np.all(observations[:, 60] == 1)
+        print('Goal and timeout rewards and terminal flags survive same-step resets.')
+
+        policy = GaussianActorCritic(121, 2, 16, 1, 1)
+        ppo = PPO(policy, pi_lr=1e-3, vf_lr=1e-3, gamma=0.99, lamda=0.95, K_epochs=1,
+                  eps_clip=0.2, batch_size=256, vf_coef=0.5, entropy_coef=0.01)
+        before = policy.log_std.numpy().copy()
+        step, = learn(env, ppo, total_timesteps=256, rollout_steps=256)
+        assert step.timesteps == 256 and step.episode_returns
+        assert np.isfinite(policy.log_std.numpy()).all() and not np.array_equal(policy.log_std.numpy(), before)
+        print(f'One PPO update on {step.timesteps} transitions, {len(step.episode_returns)} finished episodes.')
+
+
+if __name__ == '__main__':
+    main(sys.argv[1])

tinybaselines-0.1.0a5/examples/puffer/squared_bridge.c

@@ -0,0 +1,17 @@
+#include "ocean/squared_continuous/squared_continuous.h"
+
+Squared* squared_create(obs_t* observations, float* actions, float* reward, float* terminal, unsigned int seed) {
+    Squared* env = calloc(1, sizeof(*env));
+    if (!env) return NULL;
+    DictItem size = {.key = "size", .value = 11};
+    Dict options = {.items = &size, .size = 1, .cap = 1};
+    env->rng = seed;
+    env->agents[0] = (Agent){.observations = observations, .actions = actions,
+                             .rewards = reward, .terminals = terminal};
+    puf_init(env, &options);
+    return env;
+}
+
+void squared_destroy(Squared* env) {
+    free(env);
+}

tinybaselines-0.1.0a5/examples/puffer/squared_smoke.c

@@ -0,0 +1,39 @@
+#include <assert.h>
+
+#include "ocean/squared_continuous/squared_continuous.h"
+
+int main(void) {
+    obs_t observations[OBS_SIZE] = {0};
+    float actions[NUM_ATNS] = {0}, reward = 0, terminal = 0;
+    DictItem size = {.key = "size", .value = 11};
+    Dict options = {.items = &size, .size = 1, .cap = 1};
+    Squared env = {.rng = 1};
+    env.agents[0] = (Agent){.observations = observations, .actions = actions,
+                            .rewards = &reward, .terminals = &terminal};
+    puf_init(&env, &options);
+    puf_reset(&env);
+
+    const int center = OBS_SIZE / 2;
+    assert(env.num_agents == 1 && observations[center] == AGENT && !IsWindowReady());
+    memset(observations, EMPTY, sizeof(observations));
+    observations[center] = AGENT;
+    observations[center + 1] = TARGET;
+
+    actions[1] = 0.2f;
+    puf_step(&env);
+    assert(reward == 0 && terminal == 0 && observations[center] == AGENT);
+    actions[1] = 0.3f;
+    puf_step(&env);
+    assert(reward == 1 && terminal == 1 && env.tick == 0 && observations[center] == AGENT);
+
+    actions[1] = 0;
+    for (int i = 0; i < 3 * env.size; i++) {
+        puf_step(&env);
+        assert(reward == 0 && terminal == 0);
+    }
+    puf_step(&env);
+    assert(reward == -1 && terminal == 1 && env.tick == 0 && observations[center] == AGENT);
+    assert(!IsWindowReady());
+    puf_close(&env);
+    puts("Squared Continuous: movement threshold, goal, timeout, and same-step reset passed.");
+}

tinybaselines-0.1.0a5/examples/puffer/train.py

@@ -0,0 +1,66 @@
+import argparse
+from contextlib import closing
+from hashlib import sha256
+from importlib.metadata import version
+import json
+from pathlib import Path
+import platform
+import subprocess
+from time import perf_counter
+
+from tinybaselines import GaussianActorCritic, PPO, evaluate, learn
+from .env import CoordinateSquaredVectorEnv, SquaredVectorEnv
+
+
+def run(library, output, total_timesteps, observations='absolute'):
+    output.mkdir(parents=True, exist_ok=False)
+    def make_env():
+        if observations == 'board':
+            return SquaredVectorEnv(library, num_envs=4)
+        return CoordinateSquaredVectorEnv(library, num_envs=4, relative=observations == 'relative')
+
+    state_dim = 121 if observations == 'board' else 4
+    policy = GaussianActorCritic(state_dim, 2, 64, 2, 2)
+    ppo = PPO(policy, pi_lr=3e-4, vf_lr=1e-3, gamma=0.99, lamda=0.95, K_epochs=4,
+              eps_clip=0.2, batch_size=256, vf_coef=0.5, entropy_coef=0.001)
+
+    with (output / 'metrics.jsonl').open('x') as log:
+        def record(event, **fields):
+            line = json.dumps(dict(event=event, **fields), allow_nan=False)
+            print(line, file=log, flush=True)
+            print(line, flush=True)
+
+        def score(event, actor):
+            with closing(make_env()) as env:
+                returns = list(map(float, evaluate(env, actor, num_episodes=100, deterministic=False)))
+            assert len(returns) == 100 and all(value in (-1, 1) for value in returns)
+            record(event, goals=returns.count(1), episodes=len(returns), episode_returns=returns)
+
+        commit = subprocess.run(['git', 'rev-parse', 'HEAD'], capture_output=True, text=True).stdout.strip()
+        record('config', source_commit=commit or None, platform=platform.platform(), python=platform.python_version(),
+               versions={name: version(name) for name in ('tinygrad', 'numpy')},
+               library_sha256=sha256(library.read_bytes()).hexdigest(), device=ppo.device,
+               num_envs=4, requested_timesteps=total_timesteps, rollout_steps=2048,
+               deterministic=False, observations=observations, policy=policy.config, ppo=ppo.config)
+        score('initial', policy)
+        timesteps = 0
+        with closing(make_env()) as env:
+            started = perf_counter()
+            for step in learn(env, ppo, total_timesteps=total_timesteps, rollout_steps=2048):
+                timesteps = step.timesteps
+                record('update', **step._asdict())
+            training_seconds = perf_counter() - started
+        checkpoint = output / 'ppo.safetensors'
+        ppo.save(checkpoint)
+        record('checkpoint', timesteps=timesteps, training_seconds=training_seconds, path=checkpoint.name)
+        score('final', PPO.load(checkpoint).actor_critic)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description='Train PPO on headless Squared Continuous.')
+    parser.add_argument('library', type=Path)
+    parser.add_argument('--output', required=True, type=Path)
+    parser.add_argument('--timesteps', type=int, default=204_800)
+    parser.add_argument('--observations', choices=('board', 'absolute', 'relative'), default='absolute')
+    args = parser.parse_args()
+    run(args.library, args.output, args.timesteps, args.observations)

{tinybaselines-0.1.0a4 → tinybaselines-0.1.0a5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "tinybaselines"
-version = "0.1.0a4"
+version = "0.1.0a5"
 description = "PPO in tinygrad."
 readme = "README.md"
 license = "MIT"

{tinybaselines-0.1.0a4 → tinybaselines-0.1.0a5/tinybaselines.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tinybaselines
-Version: 0.1.0a4
+Version: 0.1.0a5
 Summary: PPO in tinygrad.
 License-Expression: MIT
 Requires-Python: >=3.11
@@ -21,7 +21,7 @@ Use Python 3.11 and clang. The library install does not include Gymnasium.
 ```sh
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install tinybaselines==0.1.0a4
+python -m pip install tinybaselines==0.1.0a5
 ```
 
 Install Gymnasium and Box2D to run this example:
@@ -79,6 +79,10 @@ Each `learn` call starts fresh episodes and a new transition counter.
 
 ## Repository examples
 
+Example READMEs and screenshots: [LunarLander](examples/lunar_lander/README.md),
+[CartPole](examples/cartpole/README.md), [Pendulum](examples/pendulum/README.md),
+and [Squared Continuous](examples/puffer/README.md).
+
 From a checkout, run the full LunarLander trainer:
 
 ```sh

{tinybaselines-0.1.0a4 → tinybaselines-0.1.0a5}/tinybaselines.egg-info/SOURCES.txt

@@ -4,9 +4,13 @@ README.md
 pyproject.toml
 requirements.txt
 examples/__init__.py
+examples/cartpole/README.md
+examples/cartpole/checkpoint.png
 examples/cartpole/train.py
 examples/cartpole/results/cpu.jsonl
+examples/lunar_lander/README.md
 examples/lunar_lander/__init__.py
+examples/lunar_lander/checkpoint.png
 examples/lunar_lander/config.py
 examples/lunar_lander/reference.py
 examples/lunar_lander/requirements-reference.txt
@@ -23,13 +27,23 @@ examples/lunar_lander/results/2026-10-26/phase-torch-thread4.jsonl
 examples/lunar_lander/results/2026-10-26/protocol.json
 examples/lunar_lander/results/2026-10-26/tinygrad-cpu.jsonl
 examples/lunar_lander/results/2026-10-26/torch-cpu.jsonl
+examples/pendulum/README.md
 examples/pendulum/__init__.py
+examples/pendulum/checkpoint.png
 examples/pendulum/env.py
 examples/pendulum/requirements.txt
 examples/pendulum/train.py
 examples/pendulum/results/cpu-2.jsonl
 examples/pendulum/results/cpu-3.jsonl
 examples/pendulum/results/cpu.jsonl
+examples/puffer/README.md
+examples/puffer/checkpoint.png
+examples/puffer/env.py
+examples/puffer/smoke.py
+examples/puffer/squared_bridge.c
+examples/puffer/squared_smoke.c
+examples/puffer/train.py
+examples/puffer/results/observations.jsonl
 tests/test_cartpole.py
 tests/test_checkpoint.py
 tests/test_core.py

tinybaselines-0.1.0a4/MANIFEST.in

@@ -1 +0,0 @@
-recursive-include examples *.py *.txt *.json *.jsonl