weightspace 0.2.0__tar.gz → 0.3.0__tar.gz

weightspace-0.3.0/CHANGELOG.md

@@ -0,0 +1,57 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.3.0] - 2026-11-03
+
+### Added
+- Run events and callbacks (`callbacks=`): `LoggingCallback`, `EventLog`, `TranscriptWriter`, `ConsoleLogger`, and `weightspace.otel.OpenTelemetryCallback` with GenAI semantic-convention spans (`weightspace[otel]`).
+- `Usage` (tokens normalized across providers, incl. cache reads/writes and reasoning) and `Pricing`; `Result.tokens`, `cost`, `stop_reason`, `run_id`, `seconds`, `timings`, `error`, `ok`.
+- `token_budget`, `cost_budget` and `time_budget`: the LLM gets a last turn to submit when one runs out.
+- Ctrl-C returns a partial `Result`; API failures raise `SessionError` carrying the partial result.
+- Read-only tool calls of one turn run in parallel (`parallel_tools=`, `wsp.tool(..., parallel=True)`).
+- `ScriptedLLM.from_transcript` to replay logged runs offline; CLI `--replay`.
+- OpenAI `prompt_cache_key` per run on api.openai.com; `max_retries` / `timeout` on all backends; Anthropic streaming for long requests.
+- CLI: `--probe FILE.py:FN`, `--forward-budget`, `--price`, `--max-cost`, `--max-tokens`, `--max-time`, `--events`, `--json`, `--version`; exit code 2 when a run does not finish cleanly.
+- `py.typed` marker.
+- Exp. 8 results in the README: base, the paper's checkpoint and the Opus and Astra edits, with thinking on and off.
+- `experiments/exp8`: `evaluate.py --no-thinking` and `table.py`.
+
+### Changed
+- `ws.diff()` and `ws.patch()` run in blocks (about 6x faster and 50x less memory on large tensors); `describe_tensor` is cached and uses a Gram-matrix spectrum for tall/wide matrices.
+- The sandbox timeout works off the main thread and without SIGALRM; sandbox code runs under `torch.no_grad()`.
+- `agent.py` split into `tools.py`, `result.py`, `events.py` and `usage.py` (same public API).
+- Faster hot paths, measured with the new `benchmarks/profile_hotpaths.py`: block-parallel `diff`, `patch`, snapshots and restores; exact histogram-based quantiles; sparse checkpoints; content-keyed `describe` cache; cheaper spectrum for large matrices. `describe` on a 151936 x 1024 embedding goes from 6.5 s and 2.4 GB to 1.2 s and 132 MB, and a 30-step agent loop from 9.2 s to 0.4 s. See docs/production.md.
+
+### Fixed
+- `apply_patch` silently did nothing on tensors stored as non-contiguous arrays.
+- `diff` and `patch` counted untouched NaN weights as edits.
+
+## [0.2.0] - 2026-11-03
+
+### Added
+- `openresponses:<model>` backend (`OpenResponses`) for the [Open Responses](https://www.openresponses.org) spec (OpenAI Responses API), replaying encrypted reasoning between turns. `openai-responses:` is kept as an alias.
+- `speed=` on `Session`, `wsp.read/fix/write/improve` and backends (CLI `--speed` / `--fast`) for provider fast mode: Anthropic `speed`, OpenAI `service_tier`. Saved in `result.json`.
+- `usage.thinking_tokens` is recorded for Anthropic models.
+- `llms.txt` with a short usage guide for coding agents.
+- Exp. 8 replication harness for Qwen3.5-0.8B in `experiments/exp8`.
+- Open-source project files, docs and a wider CI matrix.
+
+### Changed
+- Anthropic `thinking=False` falls back to `effort="low"` on models that can't disable thinking (Opus 5.5).
+
+## [0.1.0] - 2026-11-03
+
+### Added
+- `WeightSpace` with NumPy, PyTorch, Hugging Face, SafeTensors and `.pt` stores; tracked edits, diffs, checkpoints and sparse patches.
+- `read`, `fix`, `write` and `improve` sessions with a persistent Python sandbox, budgeted probes and text mode.
+- OpenAI-compatible (OpenAI, OpenRouter, vLLM, SGLang, Ollama) and Anthropic backends.
+- `thinking` switch (off / low / medium / high / xhigh / max), with adaptive thinking and prompt caching on Anthropic.
+- `weightspace` CLI and examples.
+
+[Unreleased]: https://github.com/leofmarciano/weightspace/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/leofmarciano/weightspace/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/leofmarciano/weightspace/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/leofmarciano/weightspace/releases/tag/v0.1.0

{weightspace-0.2.0 → weightspace-0.3.0}/CITATION.cff

@@ -8,7 +8,7 @@ authors:
 license: Apache-2.0
 repository-code: "https://github.com/leofmarciano/weightspace"
 date-released: "2026-11-03"
-version: "0.2.0"
+version: "0.3.0"
 keywords:
   - weight-space-programming
   - interpretability

{weightspace-0.2.0 → weightspace-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: weightspace
-Version: 0.2.0
+Version: 0.3.0
 Summary: Weight-space programming: let an LLM read and write neural network weights directly.
 Project-URL: Homepage, https://marcianotheory.com
 Project-URL: Repository, https://github.com/leofmarciano/weightspace
@@ -27,6 +27,7 @@ Provides-Extra: anthropic
 Requires-Dist: anthropic>=0.40; extra == 'anthropic'
 Provides-Extra: dev
 Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: opentelemetry-sdk>=1.20; extra == 'dev'
 Requires-Dist: pre-commit>=3; extra == 'dev'
 Requires-Dist: pytest>=8; extra == 'dev'
 Requires-Dist: ruff>=0.5; extra == 'dev'
@@ -36,6 +37,8 @@ Requires-Dist: torch>=2.1; extra == 'hf'
 Requires-Dist: transformers>=4.40; extra == 'hf'
 Provides-Extra: openai
 Requires-Dist: openai>=1.40; extra == 'openai'
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api>=1.20; extra == 'otel'
 Provides-Extra: torch
 Requires-Dist: torch>=2.1; extra == 'torch'
 Description-Content-Type: text/markdown
@@ -182,7 +185,23 @@ res = s.run("...")
 ```
 
 Custom tools: `wsp.tool("name", "description", x={"type": "number"})(fn)`.
-Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script.
+Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script, and
+`wsp.ScriptedLLM.from_transcript("runs/1/transcript.jsonl")` replays a logged run.
+
+## Production
+
+Budgets (`token_budget`, `cost_budget`, `time_budget`), cost tracking (`pricing=`), `stop_reason`, partial results
+on errors and Ctrl-C, OpenTelemetry tracing, parallel read-only tool calls and replay are covered in
+[docs/production.md](docs/production.md).
+
+```python
+from weightspace.otel import OpenTelemetryCallback
+
+res = wsp.fix(
+    model, task, llm="anthropic:claude-opus-5-5", pricing=(5, 25, 0.5, 6.25), cost_budget=5.0, callbacks=[OpenTelemetryCallback()]
+)
+print(res.stop_reason, res.tokens, res.cost)
+```
 
 ## CLI
 
@@ -190,8 +209,43 @@ Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script.
 weightspace inspect model.safetensors --describe
 weightspace read model.safetensors "What was this trained to do?" --llm openai:gpt-5 --context @notes.md
 weightspace fix model.safetensors "..." --llm anthropic:claude-sonnet-4-5 --save-weights fixed.safetensors --out runs/1
+weightspace fix model.safetensors "..." --llm openai:gpt-5 --probe probes.py:rollout --forward-budget 10 \
+    --price 1.25,10 --max-cost 2 --events events.jsonl --json
+weightspace fix model.safetensors "..." --replay runs/1/transcript.jsonl   # no tokens spent
 ```
 
+## Results: Exp. 8 replication
+
+We reran the paper's Exp. 8 with this harness: fix Qwen3.5-0.8B's math by editing its weights, no training. Test sets are MATH-500 and GSM8K test, one sample per problem, last `\boxed{}` graded (code in [`experiments/exp8`](experiments/exp8)). ± is one standard error; "cut" is the share of answers that hit the token limit.
+
+**Thinking on** (paper protocol, 32k tokens, first 250 problems of each set):
+
+| Edit | Editing LLM | Forward passes | Weights changed | MATH-500 | cut | GSM8K | cut |
+|---|---|---|---|---|---|---|---|
+| none (base) | | | 0 | 8.4 ± 1.8 | 92% | 12.4 ± 2.1 | 89% |
+| paper's edit ([checkpoint](https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming)) | gpt-6-astra | 50 | 131,072 | 52.4 ± 3.2 | 34% | 61.6 ± 3.1 | 10% |
+| 4 MLP neurons ×6 | Opus 5.5, thinking high | 7 of 12 | 4,096 | 40.0 ± 3.1 | 60% | 53.2 ± 3.2 | 42% |
+| 4 attention heads ×1.5 | Opus 5.5, thinking off, fast | 7 of 12 | 1,048,576 | 17.2 ± 2.4 | 82% | 34.0 ± 3.0 | 59% |
+| 1 MLP neuron ×0.5 (unfinished run) | gpt-6-astra, thinking high | 12 budget, run lost | 1,024 | not measured yet | | | |
+
+**Thinking off** (model card sampling, 8k tokens, full sets: 500 MATH-500, 500 GSM8K):
+
+| Edit | MATH-500 | cut | GSM8K | cut |
+|---|---|---|---|---|
+| none (base) | 46.4 ± 2.2 | 31% | 56.6 ± 2.2 | 6% |
+| paper's edit | 42.0 ± 2.2 | 29% | 57.2 ± 2.2 | 3% |
+| Opus, thinking high | 41.8 ± 2.2 | 23% | 54.4 ± 2.2 | 2% |
+| Opus, thinking off | 33.2 ± 2.1 | 20% | 42.0 ± 2.2 | 3% |
+| Astra's neuron (first 250 of each set) | 46.0 ± 3.2 | 28% | 56.4 ± 3.1 | 6% |
+
+What this says:
+
+- Editing works. Opus found 4 neurons that push down "Wait" and amplified them; 4,096 weights out of 750M took the thinking model from 8.4% / 12.4% to 40.0% / 53.2%.
+- It does not beat the paper. The paper's edit, with 50 forward passes against Opus's 7, is ahead by 8 to 12 points. Our run of the paper's checkpoint (52.4% / 61.6%) is close to what the paper reports (48.1% / 67.1%, with more samples per problem), and our base (8.4% / 12.4%) matches theirs (7.9% / 13.8%).
+- Turning thinking off is a strong baseline. The base model with thinking off scores 46.4% / 56.6%, above Opus's edit, and none of the edits help in non-thinking mode. The edits mostly cut the thinking loops; they don't add math skill. Compare any edit against the thinking-off base.
+- A second Opus session (up to 25 forward passes in total) is not in the table. It ran out of steps without submitting, its best variant (36.6% on practice problems against 30.5% for the ×6 edit) is within noise, and its layer-23 change wasn't saved before the GPU pod died.
+- Training-set gains overstate test gains: Opus measured 8.6% → 50.8% on held-out practice problems, but a fresh practice set gave 30.5%.
+
 ## Examples
 
 - `examples/fix_quadrants.py`: fix a sabotaged classifier (runs offline with a scripted LLM if you omit `--llm`)

{weightspace-0.2.0 → weightspace-0.3.0}/README.md

@@ -140,7 +140,23 @@ res = s.run("...")
 ```
 
 Custom tools: `wsp.tool("name", "description", x={"type": "number"})(fn)`.
-Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script.
+Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script, and
+`wsp.ScriptedLLM.from_transcript("runs/1/transcript.jsonl")` replays a logged run.
+
+## Production
+
+Budgets (`token_budget`, `cost_budget`, `time_budget`), cost tracking (`pricing=`), `stop_reason`, partial results
+on errors and Ctrl-C, OpenTelemetry tracing, parallel read-only tool calls and replay are covered in
+[docs/production.md](docs/production.md).
+
+```python
+from weightspace.otel import OpenTelemetryCallback
+
+res = wsp.fix(
+    model, task, llm="anthropic:claude-opus-5-5", pricing=(5, 25, 0.5, 6.25), cost_budget=5.0, callbacks=[OpenTelemetryCallback()]
+)
+print(res.stop_reason, res.tokens, res.cost)
+```
 
 ## CLI
 
@@ -148,8 +164,43 @@ Offline runs and tests: `wsp.ScriptedLLM([...])` replays a fixed script.
 weightspace inspect model.safetensors --describe
 weightspace read model.safetensors "What was this trained to do?" --llm openai:gpt-5 --context @notes.md
 weightspace fix model.safetensors "..." --llm anthropic:claude-sonnet-4-5 --save-weights fixed.safetensors --out runs/1
+weightspace fix model.safetensors "..." --llm openai:gpt-5 --probe probes.py:rollout --forward-budget 10 \
+    --price 1.25,10 --max-cost 2 --events events.jsonl --json
+weightspace fix model.safetensors "..." --replay runs/1/transcript.jsonl   # no tokens spent
 ```
 
+## Results: Exp. 8 replication
+
+We reran the paper's Exp. 8 with this harness: fix Qwen3.5-0.8B's math by editing its weights, no training. Test sets are MATH-500 and GSM8K test, one sample per problem, last `\boxed{}` graded (code in [`experiments/exp8`](experiments/exp8)). ± is one standard error; "cut" is the share of answers that hit the token limit.
+
+**Thinking on** (paper protocol, 32k tokens, first 250 problems of each set):
+
+| Edit | Editing LLM | Forward passes | Weights changed | MATH-500 | cut | GSM8K | cut |
+|---|---|---|---|---|---|---|---|
+| none (base) | | | 0 | 8.4 ± 1.8 | 92% | 12.4 ± 2.1 | 89% |
+| paper's edit ([checkpoint](https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming)) | gpt-6-astra | 50 | 131,072 | 52.4 ± 3.2 | 34% | 61.6 ± 3.1 | 10% |
+| 4 MLP neurons ×6 | Opus 5.5, thinking high | 7 of 12 | 4,096 | 40.0 ± 3.1 | 60% | 53.2 ± 3.2 | 42% |
+| 4 attention heads ×1.5 | Opus 5.5, thinking off, fast | 7 of 12 | 1,048,576 | 17.2 ± 2.4 | 82% | 34.0 ± 3.0 | 59% |
+| 1 MLP neuron ×0.5 (unfinished run) | gpt-6-astra, thinking high | 12 budget, run lost | 1,024 | not measured yet | | | |
+
+**Thinking off** (model card sampling, 8k tokens, full sets: 500 MATH-500, 500 GSM8K):
+
+| Edit | MATH-500 | cut | GSM8K | cut |
+|---|---|---|---|---|
+| none (base) | 46.4 ± 2.2 | 31% | 56.6 ± 2.2 | 6% |
+| paper's edit | 42.0 ± 2.2 | 29% | 57.2 ± 2.2 | 3% |
+| Opus, thinking high | 41.8 ± 2.2 | 23% | 54.4 ± 2.2 | 2% |
+| Opus, thinking off | 33.2 ± 2.1 | 20% | 42.0 ± 2.2 | 3% |
+| Astra's neuron (first 250 of each set) | 46.0 ± 3.2 | 28% | 56.4 ± 3.1 | 6% |
+
+What this says:
+
+- Editing works. Opus found 4 neurons that push down "Wait" and amplified them; 4,096 weights out of 750M took the thinking model from 8.4% / 12.4% to 40.0% / 53.2%.
+- It does not beat the paper. The paper's edit, with 50 forward passes against Opus's 7, is ahead by 8 to 12 points. Our run of the paper's checkpoint (52.4% / 61.6%) is close to what the paper reports (48.1% / 67.1%, with more samples per problem), and our base (8.4% / 12.4%) matches theirs (7.9% / 13.8%).
+- Turning thinking off is a strong baseline. The base model with thinking off scores 46.4% / 56.6%, above Opus's edit, and none of the edits help in non-thinking mode. The edits mostly cut the thinking loops; they don't add math skill. Compare any edit against the thinking-off base.
+- A second Opus session (up to 25 forward passes in total) is not in the table. It ran out of steps without submitting, its best variant (36.6% on practice problems against 30.5% for the ×6 edit) is within noise, and its layer-23 change wasn't saved before the GPU pod died.
+- Training-set gains overstate test gains: Opus measured 8.6% → 50.8% on held-out practice problems, but a fresh practice set gave 30.5%.
+
 ## Examples
 
 - `examples/fix_quadrants.py`: fix a sabotaged classifier (runs offline with a scripted LLM if you omit `--llm`)

weightspace-0.3.0/benchmarks/profile_hotpaths.py

@@ -0,0 +1,111 @@
+"""Time and peak memory of weightspace hot paths on a Qwen-sized synthetic model (CPU).
+
+python benchmarks/profile_hotpaths.py [--small] [--json out.json]
+"""
+
+from __future__ import annotations
+
+import argparse
+import gc
+import json
+import time
+import tracemalloc
+
+import numpy as np
+import torch
+
+import weightspace as wsp
+from weightspace.llm import ScriptedLLM, call, reply
+
+
+def model(vocab: int, hidden: int, layers: int, inter: int) -> torch.nn.Module:
+    m = torch.nn.Module()
+    m.embed = torch.nn.Embedding(vocab, hidden)
+    for i in range(layers):
+        blk = torch.nn.Module()
+        for n, shape in {
+            "q": (2 * hidden, hidden),
+            "k": (hidden // 2, hidden),
+            "v": (hidden // 2, hidden),
+            "o": (hidden, 2 * hidden),
+            "gate": (inter, hidden),
+            "up": (inter, hidden),
+            "down": (hidden, inter),
+        }.items():
+            setattr(blk, n, torch.nn.Linear(shape[1], shape[0], bias=False))
+        blk.norm = torch.nn.LayerNorm(hidden)
+        setattr(m, f"layer{i}", blk)
+    return m.to(torch.bfloat16)
+
+
+RESULTS: dict[str, dict] = {}
+
+
+def bench(label: str, fn, repeat: int = 3):
+    gc.collect()
+    tracemalloc.start()
+    best = float("inf")
+    for _ in range(repeat):
+        t0 = time.perf_counter()
+        out = fn()
+        best = min(best, time.perf_counter() - t0)
+    peak = tracemalloc.get_traced_memory()[1]
+    tracemalloc.stop()
+    RESULTS[label] = {"ms": round(best * 1e3, 2), "peak_mb": round(peak / 2**20, 1)}
+    print(f"{label:48s} {best * 1e3:10.2f} ms  {peak / 2**20:9.1f} MB peak", flush=True)
+    return out
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--small", action="store_true", help="tiny model, for a quick smoke run")
+    ap.add_argument("--json")
+    a = ap.parse_args()
+    torch.manual_seed(0)
+    dims = (4096, 256, 2, 896) if a.small else (151936, 1024, 24, 3584)
+    m = model(*dims)
+    print(f"model: {sum(p.numel() for p in m.parameters()) / 1e6:.0f}M params bf16, torch threads={torch.get_num_threads()}")
+
+    ws = wsp.WeightSpace.from_torch(m)
+    emb, mlp, attn = "embed.weight", "layer0.gate.weight", "layer0.q.weight"
+    for n in ws.names():
+        ws[n]  # materialise float32 mirrors once, outside the timings
+    s = wsp.Session(ws, ScriptedLLM([]), mode="fix", max_steps=1)
+    t = s.tools
+
+    bench("list_tensors", lambda: t["list_tensors"].fn(), 20)
+    bench("intro prompt", lambda: s._intro("task"), 20)
+    bench("describe mlp (cold)", lambda: (ws._describe_cache.clear(), ws.describe(mlp)), 3)
+    bench("describe attn (cold)", lambda: (ws._describe_cache.clear(), ws.describe(attn)), 3)
+    bench("describe embedding (cold)", lambda: (ws._describe_cache.clear(), ws.describe(emb)), 2)
+    bench("read_tensor 64x64", lambda: t["read_tensor"].fn(name=mlp, index="0:64, 0:64"), 10)
+    bench("first edit (snapshot) embedding", lambda: (ws._original.pop(emb, None), ws.edit(emb)), 3)
+    for n in ws.names("layer*.weight")[:40]:
+        ws.edit(n)
+    print(f"touched tensors: {len(ws.touched())}, {sum(ws.numel(n) for n in ws.touched()) / 1e6:.0f}M weights")
+    bench("write_tensor 8 values (+diff)", lambda: t["write_tensor"].fn(name=emb, values=[0.5] * 8, index="7, 0:8"), 5)
+    bench("diff (one change)", ws.diff, 5)
+    bench("patch", ws.patch, 3)
+    bench("sync (bf16 flush)", ws.sync, 3)
+    bench("checkpoint", lambda: ws.checkpoint("c"), 2)
+    bench("restore + diff", lambda: t["restore"].fn(label="c"), 2)
+    sb = s.sandbox
+    bench("sandbox: W[emb][0,0] += 1", lambda: sb.run(f"W['{emb}'][0, 0] += 1"), 10)
+    bench("sandbox: trivial expression", lambda: sb.run("1 + 1"), 50)
+
+    small = wsp.WeightSpace.from_numpy({"w": np.random.randn(64, 64).astype(np.float32)})
+    bench("render_space 4k weights", lambda: wsp.render_space(small), 10)
+    text = wsp.render_space(small)
+    bench("parse_weights 4k weights", lambda: wsp.parse_weights(text), 10)
+
+    calls = [
+        reply(None, call("read_tensor", name=mlp, index=f"{i}, 0:16"), call("describe_tensor", name=attn)) for i in range(30)
+    ] + [reply(None, call("submit", answer="ok"))]
+    bench("agent loop: 30 steps, 60 tool calls", lambda: wsp.read(ws, "q", llm=ScriptedLLM(list(calls)), max_steps=40), 3)
+    if a.json:
+        with open(a.json, "w") as f:
+            json.dump(RESULTS, f, indent=1)
+
+
+if __name__ == "__main__":
+    main()

{weightspace-0.2.0 → weightspace-0.3.0}/docs/README.md

@@ -3,6 +3,7 @@
 - [Concepts](concepts.md): `WeightSpace`, sessions, modes, the sandbox and what the LLM sees.
 - [Probes and budgets](probes.md): how to let the LLM run your network, and how much.
 - [Providers and thinking](providers.md): OpenAI-compatible and Anthropic backends, thinking levels, custom LLMs.
+- [Production](production.md): budgets, cost, stop reasons, events and OpenTelemetry, parallel tools, replay.
 - [Experiments](../experiments/exp8): the Exp. 8 replication on Qwen3.5-0.8B (math with thinking on).
 
 The [README](../README.md) has a quick start and the CLI reference.

weightspace-0.3.0/docs/production.md

@@ -0,0 +1,118 @@
+# Running in production
+
+Everything here is optional. A plain `wsp.fix(model, task, llm=...)` still works the same way.
+
+## Results you can act on
+
+`Result` tells you why a run stopped and what it cost:
+
+```python
+res = wsp.fix(model, task, llm="anthropic:claude-opus-5-5", pricing=(5, 25, 0.5, 6.25))
+res.stop_reason  # submitted | answered | max_steps | token_budget | cost_budget | time_budget | interrupted | error
+res.ok  # True for submitted / answered
+res.tokens  # Usage(input_tokens, output_tokens, cached_input_tokens, cache_write_tokens, reasoning_tokens, requests)
+res.cost  # USD, when pricing= is given
+res.timings  # {"llm": {"calls", "seconds"}, "tool:python": {...}, ...}
+res.run_id  # also sent as the OpenAI prompt_cache_key and the trace conversation id
+```
+
+`res.usage` still holds the provider's raw counters. `tokens` is the same shape for every provider.
+
+## Budgets
+
+```python
+wsp.improve(model, goal, llm=..., token_budget=2_000_000, cost_budget=10.0, time_budget=3600, pricing=(1.25, 10))
+```
+
+When a budget runs out the LLM gets one last turn to submit, so you still get an answer and the edits so far.
+`pricing` is USD per million tokens: `(input, output[, cached_input, cache_write])`, a dict, or `wsp.Pricing`.
+
+## Failures and Ctrl-C
+
+- Ctrl-C returns a `Result` with `stop_reason="interrupted"`: transcript, edits and usage up to that point.
+- If the LLM request fails after the SDK's own retries, `wsp.SessionError` is raised. `err.result` has the
+  partial result, so nothing is lost.
+- Backends take `max_retries` (default 4) and `timeout` (default 900 s), passed to the official SDKs.
+- Anthropic streams long requests (`stream="auto"`: when `max_tokens` > 21333), which long Opus turns need.
+
+## Observability
+
+Pass `callbacks=[...]`; each gets every run event (`run_start`, `step_start`, `llm_start`, `llm_end`, `tool_start`,
+`tool_end`, `message`, `run_end`; see `weightspace.events`). A callback is a function or an object with `on_event`.
+
+```python
+from weightspace.events import EventLog, LoggingCallback
+from weightspace.otel import OpenTelemetryCallback  # pip install "weightspace[otel]"
+
+wsp.fix(model, task, llm=..., callbacks=[OpenTelemetryCallback(), LoggingCallback(), EventLog("events.jsonl")])
+```
+
+- `OpenTelemetryCallback` emits GenAI semantic-convention spans: `invoke_agent weightspace` per run, `chat <model>`
+  per request (token usage, cache reads/writes, finish reason) and `execute_tool <name>` per tool call. Tool
+  arguments are only recorded with `capture_content=True`.
+- `LoggingCallback` logs to the `weightspace` logger (milestones at INFO, the rest at DEBUG).
+- `EventLog` / `TranscriptWriter` append JSONL as the run goes, so a crash keeps the log. `log_path=` uses
+  `TranscriptWriter`; `verbose=True` uses `ConsoleLogger`.
+
+## Parallel tool calls
+
+Read-only tools (`list_tensors`, `describe_tensor`, `read_tensor`, and custom tools declared with
+`wsp.tool(..., parallel=True)`) that the LLM asks for in the same turn run on a thread pool
+(`parallel_tools=8` workers). Anything that writes weights, runs Python or spends forward passes runs alone and
+in order, so edits never race. Results always come back in the order they were asked for.
+
+## OpenAI
+
+- `prompt_cache_key` is set to the run id on api.openai.com, so every step of a run lands on the same cache.
+  Pass `prompt_cache_key="..."` to pick your own, or `False` to turn it off.
+- Any other request field goes straight through: `get_llm("openai:gpt-5", parallel_tool_calls=True,
+  safety_identifier="user-123")`.
+- `openresponses:<model>` runs the Responses API stateless (`store=False`) and replays encrypted reasoning.
+
+## Replay
+
+Every run can be replayed offline, with no tokens spent:
+
+```python
+llm = wsp.ScriptedLLM.from_transcript("runs/1/transcript.jsonl")
+wsp.fix(model, task, llm=llm, probes=[...], forward_budget=10)
+```
+
+Tool outputs are recomputed, so replaying on other weights shows how the same program behaves there.
+CLI: `weightspace fix model.safetensors "..." --replay runs/1/transcript.jsonl`.
+
+## Sandbox
+
+- Code runs under `torch.no_grad()` (`no_grad=True`), so autograd-based training of the weights fails.
+- The timeout works on any thread: SIGALRM on the main thread, a watchdog elsewhere (for example inside a web
+  server worker). A single long C call, like a huge matmul, finishes before the timeout fires.
+- It is still not a security boundary. See [SECURITY.md](../SECURITY.md).
+
+## Performance
+
+Large tensors are processed in 1M-element blocks on a small shared thread pool (NumPy releases the GIL), so
+nothing makes a full float64 copy of a tensor:
+
+- `ws.diff()`, `ws.patch()` and checkpoints compare weights bitwise (a NaN that was not edited is not a change).
+  The original norm used by `rel_delta` is computed once per tensor.
+- Checkpoints store a sparse delta from the originals, so checkpointing a small edit to an embedding costs
+  megabytes, not gigabytes.
+- `describe_tensor` gets exact quantiles from one histogram pass plus a partition of the few bins that hold the
+  wanted ranks, instead of sorting the whole tensor. Its text is cached by content, so describing a touched but
+  unchanged tensor is free.
+- The spectrum uses an SVD for small matrices and the Gram matrix in the input precision (solved in float64)
+  above 256 rows or columns.
+
+`python benchmarks/profile_hotpaths.py` times these paths on a 546M-parameter bf16 model (CPU, 8 threads):
+
+| | 0.3.0 | now | |
+|---|---|---|---|
+| `describe` embedding 151936 x 1024 | 6,475 ms, 2,376 MB | 1,241 ms, 132 MB | 5.2x |
+| `describe` MLP 3584 x 1024 | 341 ms | 130 ms | 2.6x |
+| `write_tensor` (+ diff over 237M touched weights) | 371 ms | 80 ms | 4.6x |
+| `ws.diff()` | 381 ms | 81 ms | 4.7x |
+| `ws.patch()` | 170 ms | 83 ms | 2.1x |
+| `ws.checkpoint()` | 245 ms, 1,807 MB | 88 ms, 8 MB | 2.8x |
+| `ws.restore()` + diff | 460 ms | 147 ms | 3.1x |
+| first edit of the embedding (snapshot) | 152 ms | 29 ms | 5.3x |
+| agent loop, 30 steps, 60 tool calls | 9,242 ms | 373 ms | 24.8x |

{weightspace-0.2.0 → weightspace-0.3.0}/llms.txt

@@ -19,7 +19,8 @@ Things to know:
 - The LLM never gets the model object. It sees tensors through tools (list, describe, read slices, write) and a Python sandbox with `R` (read-only) and `W` (writable) dicts of numpy arrays, plus checkpoint / restore / revert.
 - The only way it can run the network is through probes: functions decorated with `@wsp.probe(cost=1)` whose docstring and arguments become the tool schema. `forward_budget` caps the total cost.
 - Lower level: `wsp.WeightSpace.from_torch / from_hf / from_file / from_numpy`, `ws.diff()`, `ws.save_patch()`, `ws.apply_patch()`, and `wsp.Session(ws, llm, mode=..., probes=..., namespace=..., max_steps=...).run(task)`.
-- `wsp.ScriptedLLM([...])` replays a fixed script for offline tests.
+- `wsp.ScriptedLLM([...])` replays a fixed script for offline tests; `wsp.ScriptedLLM.from_transcript(path)` replays a logged run.
+- Production: `pricing=(in, out)`, `token_budget` / `cost_budget` / `time_budget`, `res.stop_reason`, `res.tokens`, `res.cost`, `callbacks=[...]` (events, logging, `weightspace.otel.OpenTelemetryCallback`). Ctrl-C returns a partial result; API errors raise `wsp.SessionError` with `.result`.
 - The sandbox runs in your process and is not a security boundary.
 
 ## Docs
@@ -27,6 +28,7 @@ Things to know:
 - [README](https://raw.githubusercontent.com/leofmarciano/weightspace/main/README.md): quick start, thinking table, CLI
 - [Concepts](https://raw.githubusercontent.com/leofmarciano/weightspace/main/docs/concepts.md): WeightSpace, sessions, modes, sandbox, what the LLM sees
 - [Probes and budgets](https://raw.githubusercontent.com/leofmarciano/weightspace/main/docs/probes.md): letting the LLM run your network, and how much
+- [Production](https://raw.githubusercontent.com/leofmarciano/weightspace/main/docs/production.md): budgets, cost, events, OpenTelemetry, parallel tools, replay
 - [Providers and thinking](https://raw.githubusercontent.com/leofmarciano/weightspace/main/docs/providers.md): OpenAI, Open Responses, OpenRouter, vLLM, Anthropic, custom LLMs
 
 ## Examples

{weightspace-0.2.0 → weightspace-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "weightspace"
-version = "0.2.0"
+version = "0.3.0"
 description = "Weight-space programming: let an LLM read and write neural network weights directly."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -30,8 +30,9 @@ torch = ["torch>=2.1"]
 hf = ["torch>=2.1", "transformers>=4.40", "safetensors>=0.4"]
 openai = ["openai>=1.40"]
 anthropic = ["anthropic>=0.40"]
+otel = ["opentelemetry-api>=1.20"]
 all = ["torch>=2.1", "transformers>=4.40", "safetensors>=0.4", "openai>=1.40", "anthropic>=0.40"]
-dev = ["pytest>=8", "ruff>=0.5", "mypy>=1.10", "pre-commit>=3"]
+dev = ["opentelemetry-sdk>=1.20", "pytest>=8", "ruff>=0.5", "mypy>=1.10", "pre-commit>=3"]
 
 [project.scripts]
 weightspace = "weightspace.cli:main"

{weightspace-0.2.0 → weightspace-0.3.0}/src/weightspace/__init__.py

@@ -7,14 +7,16 @@ print(res)                  # answer + how many weights changed
 res.save("runs/fix-1")      # transcript, result.json, sparse patch
 """
 
-from .agent import ForwardCall, Result, Session, Tool, tool
+from .agent import ForwardCall, Result, Session, SessionError, Tool, tool
+from .events import ConsoleLogger, Event, EventLog, LoggingCallback, TranscriptWriter
 from .llm import LLM, LLMResponse, ScriptedLLM, ToolCall, get_llm
 from .probes import Probe, probe
 from .render import describe, fmt_array, parse_weights, render_space
 from .space import DiffSummary, NumpyStore, Store, WeightSpace
 from .tasks import as_weightspace, fix, improve, read, write
+from .usage import Pricing, Usage
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 
 __all__ = [
     "WeightSpace",
@@ -22,7 +24,15 @@ __all__ = [
     "NumpyStore",
     "DiffSummary",
     "Session",
+    "SessionError",
     "Result",
+    "Event",
+    "ConsoleLogger",
+    "LoggingCallback",
+    "TranscriptWriter",
+    "EventLog",
+    "Usage",
+    "Pricing",
     "ForwardCall",
     "Tool",
     "tool",

weightspace-0.3.0/src/weightspace/_chunks.py

@@ -0,0 +1,101 @@
+"""Block-wise passes over large arrays. NumPy releases the GIL inside its kernels, so big tensors are split
+into blocks that run on a small shared thread pool; small ones stay on the calling thread."""
+
+from __future__ import annotations
+
+import os
+import threading
+import zlib
+from collections.abc import Callable
+from concurrent.futures import ThreadPoolExecutor
+from typing import TypeVar
+
+import numpy as np
+
+T = TypeVar("T")
+
+CHUNK = 1 << 20  # elements per block
+PARALLEL_MIN = 1 << 22  # below this many elements, one thread is faster than the pool overhead
+WORKERS = max(1, min(8, os.cpu_count() or 1))
+
+_pool: ThreadPoolExecutor | None = None
+_pool_lock = threading.Lock()
+
+
+def _executor() -> ThreadPoolExecutor:
+    global _pool
+    if _pool is None:
+        with _pool_lock:
+            if _pool is None:
+                _pool = ThreadPoolExecutor(WORKERS, thread_name_prefix="weightspace-block")
+    return _pool
+
+
+def map_blocks(fn: Callable[[int, int], T], n: int, block: int | None = None, *, elements: int | None = None) -> list[T]:
+    """`fn(start, stop)` over `[0, n)` in blocks of `block` (default CHUNK), results in order.
+    `elements` is the total work size used to decide on threading (defaults to `n`)."""
+    block = max(1, block or CHUNK)
+    spans = [(i, min(i + block, n)) for i in range(0, n, block)]
+    if len(spans) < 2 or WORKERS < 2 or (elements if elements is not None else n) < PARALLEL_MIN:
+        return [fn(a, b) for a, b in spans]
+    return list(_executor().map(lambda s: fn(*s), spans))
+
+
+_BITS = {2: np.int16, 4: np.int32, 8: np.int64}
+
+
+def bits(a: np.ndarray) -> np.ndarray:
+    """Integer view of a float array, so equality is bitwise (NaN == NaN, a sign flip of 0 counts as an edit)."""
+    t = _BITS.get(a.dtype.itemsize) if a.dtype.kind == "f" else None
+    return a.view(t) if t is not None else a
+
+
+def flat(a: np.ndarray) -> np.ndarray:
+    return a.reshape(-1)
+
+
+def copy(src: np.ndarray) -> np.ndarray:
+    """`src.copy()`, done in parallel blocks for large contiguous arrays."""
+    if src.size < PARALLEL_MIN or not src.flags.c_contiguous:
+        return src.copy()
+    out = np.empty_like(src)
+    copy_into(out, src)
+    return out
+
+
+def copy_into(dst: np.ndarray, src: np.ndarray) -> None:
+    """`dst[...] = src` for same-shape arrays, in parallel blocks when both are large and contiguous."""
+    if dst.size < PARALLEL_MIN or not (dst.flags.c_contiguous and src.flags.c_contiguous) or dst.shape != src.shape:
+        dst[...] = src
+        return
+    d, s = flat(dst), flat(src)
+    map_blocks(lambda i, j: np.copyto(d[i:j], s[i:j]), d.size)
+
+
+def changed_indices(cur: np.ndarray, orig: np.ndarray) -> np.ndarray:
+    """Flat int64 indices where `cur` and `orig` differ bitwise."""
+    a, b = bits(flat(cur)), bits(flat(orig))
+    parts = map_blocks(lambda i, j: np.flatnonzero(a[i:j] != b[i:j]) + i, a.size)
+    return np.concatenate(parts).astype(np.int64, copy=False) if parts else np.empty(0, np.int64)
+
+
+def equal(cur: np.ndarray, orig: np.ndarray) -> bool:
+    a, b = bits(flat(cur)), bits(flat(orig))
+    return all(map_blocks(lambda i, j: bool(np.array_equal(a[i:j], b[i:j])), a.size))
+
+
+def sum_squares(a: np.ndarray) -> float:
+    x = flat(a)
+
+    def block(i: int, j: int) -> float:
+        c = x[i:j].astype(np.float64)
+        return float(np.dot(c, c))
+
+    return float(sum(map_blocks(block, x.size)))
+
+
+def fingerprint(a: np.ndarray) -> tuple:
+    """Cheap content key: shape, dtype and a CRC32 per block."""
+    x = flat(a)
+    crcs = map_blocks(lambda i, j: zlib.crc32(np.ascontiguousarray(x[i:j]).data), x.size)
+    return (a.shape, a.dtype.str, *crcs)