silentbug 0.1.0__tar.gz

silentbug-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shushant Kumar Choudhary
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

silentbug-0.1.0/PKG-INFO

@@ -0,0 +1,183 @@
+Metadata-Version: 2.4
+Name: silentbug
+Version: 0.1.0
+Summary: Find silent PyTorch training bugs: accidental broadcasting, parameters that never learn, and the line that caused them.
+Author: Shushant Kumar Choudhary
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Shushant17711/silentbug
+Project-URL: Issues, https://github.com/Shushant17711/silentbug/issues
+Project-URL: Changelog, https://github.com/Shushant17711/silentbug/blob/main/CHANGELOG.md
+Keywords: pytorch,debugging,broadcasting,gradients,training
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Debuggers
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.1
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
+
+# silentbug
+
+Your model trains, the loss goes down a little, and nothing errors. But one tensor is
+broadcast against another, half your parameters never get a gradient, or softmax runs
+twice. silentbug watches a few training steps and tells you **which** silent bug it is,
+**on which line**, **why** it matters and **how to fix it**.
+
+```
+$ silentbug run train.py
+silentbug: checked 3 backward pass(es), 3 optimizer step(s) on Net
+  2 likely bug(s), 0 thing(s) to check
+
+[BUG  ] 1. mse_loss got input (100, 1) and target (100,)
+    at train.py:13: loss = crit(yhat, y)
+    why: they broadcast to (100, 100), so each prediction is compared with many targets
+         instead of its own. The loss still goes down a little, which hides the bug.
+    fix: reshape one side so input and target have the same shape (often `.squeeze(-1)`)
+
+[BUG  ] 2. 2 parameters never get a gradient: cut by detach
+    at train.py:21: g = self.gate(h).detach()
+    parameters: gate.bias, gate.weight
+    fix: remove `.detach()` if these parameters should learn
+```
+
+## Install
+
+```bash
+pip install git+https://github.com/Shushant17711/silentbug@v0.1.0   # Python 3.10+, PyTorch 2.1+
+```
+
+From a clone, for development:
+
+```bash
+git clone https://github.com/Shushant17711/silentbug && cd silentbug
+pip install -e ".[test]" && pytest tests
+```
+
+## Use it
+
+On a script, with no code changes. It runs the first N optimizer steps (default 3), then stops:
+
+```bash
+silentbug run train.py --lr 3e-4         # your script's own arguments go after it
+silentbug run --steps 10 train.py
+silentbug run --json - train.py          # JSON report on stdout
+silentbug run -m mypackage.train         # like python -m
+```
+
+Exit codes: 0 nothing found, 1 likely bugs found, 2 only things to check.
+
+Or around a block of code, for example in a test:
+
+```python
+import silentbug
+
+with silentbug.check() as report:
+    for step in range(3):
+        loss = loss_fn(model(x), y)
+        loss.backward()
+        opt.step(); opt.zero_grad()
+
+assert not report.bugs, str(report)
+```
+
+Models and optimizers are found on their own. Pass them in (`silentbug.check(model,
+optimizer=opt)`) only when the model is never called inside the block.
+
+## What it finds
+
+| Kind | Example | Level |
+|---|---|---|
+| `broadcast` | `(N,1) - (N,)` gives an `(N,N)` loss, or `mse_loss(input (B,1), target (B,))` | BUG |
+| `dead-param` | parameters with no gradient, with the cause: `.detach()`, `.data`, `.item()`/NumPy round trip, `torch.no_grad()`, argmax/topk indices, a module never called (LoRA bypassed), an output that never reaches the loss | BUG |
+| `zero-grad` | gradients that exist but are always 0 (`round`, `sign`, `floor`, ...) | BUG |
+| `not-in-optimizer` | layers created after the optimizer was built | BUG |
+| `stale-optimizer` | the optimizer still updates parameters that were replaced | BUG |
+| `double-activation` | softmax fed to `cross_entropy`/`nll_loss`, sigmoid fed to `BCEWithLogits`, softmax then `log_softmax` | BUG |
+| `eval-mode` | the model is still in `eval()` while it trains | BUG |
+| `grad-not-cleared` | `optimizer.step()` uses gradients left over from the previous step | BUG |
+| `batchnorm-batch1` | batch norm in training mode with a batch of 1 | CHECK |
+
+BUG means almost certainly wrong. CHECK means often wrong but sometimes intended (for example an
+outer product written as a broadcast).
+
+How it works: a `TorchFunctionMode` sees every torch call. It checks the shapes going into
+broadcasting ops and losses, and tags each tensor with a bitmask of the trainable parameters it
+depends on. When a tensor built from parameters comes out of an op without a gradient, that line is
+recorded as a cut. Gradient hooks on every parameter record which ones received a gradient, and
+global module and optimizer hooks find the models and optimizers. After the run, each parameter
+that never got a gradient is matched to the cut or missing call that explains it.
+
+## Tested on real bugs
+
+`benchmarks/realworld/` has 15 reproducers of silent bugs from public issues (PyTorch, PEFT,
+torchtitan, kornia, DeepTab, poolformer, Lightning Bolts, simpletransformers and others). Each one
+is a short script that keeps the buggy code pattern from the issue. Each also has a fixed version
+(`f*.py`) that must produce no findings.
+
+```bash
+cd benchmarks/realworld && python score.py $(which silentbug)
+```
+
+| Case | Issue | Bug | Found at the right line |
+|---|---|---|---|
+| r01 | pytorch/pytorch#16045 | `MSELoss` on `(N,1)` vs `(N,)` | yes |
+| r02 | OpenTabular/DeepTab#441 | `(n,1)` target unsqueezed to `(B,1,1)` | yes |
+| r03 | kornia/kornia#6157 | SSIM loss broadcasts a batch-1 target | yes |
+| r04 | trnsci/trnsparse#67 | `y` broadcasts the spmv output to `(12,12)` | yes |
+| r05 | huggingface/peft#1612 | weights read directly, LoRA wrapper never runs | yes |
+| r06 | sail-sg/poolformer#16 | stage-0 output dropped, `norm0` never trains | yes |
+| r07 | pytorch/torchtitan#4802 | top-k indices cut the indexer's gradient | yes |
+| r08 | pytorch/pytorch#14467 | GRU created after the optimizer | yes |
+| r09 | RAI-SCC/jigsaw#224 | reshard replaces parameters, optimizer goes stale | yes |
+| r10 | gitabcworld/MatchingNetworks#5 | softmax, then `cross_entropy` | yes |
+| r11 | cloudtrainerwork/chesstrader#49 | softmax output fed to `CrossEntropyLoss` | yes |
+| r12 | Lightning-Universe/lightning-bolts#654 | `log_softmax(softmax(x))` | yes |
+| r13 | YuanGongND/ssast#2 | sigmoid fed to `BCEWithLogitsLoss` | yes |
+| r14 | ThilinaRajapakse/simpletransformers#941 | `model.eval()` never undone | yes |
+| r15 | juweipku/CoCo#1 | no `optimizer.zero_grad()` | yes |
+
+Result: 15 of 15 found at the right line, 15 of 15 fixed versions report nothing.
+
+## Overhead
+
+6-layer MLP, width 1024, batch 256, AdamW, CPU: 30.9 ms/step plain, 33.8 ms/step under
+silentbug (1.09x). It is meant to run for a few steps, not for a whole training run.
+
+## Limits
+
+## Tested setups
+
+* PyTorch 2.15 on CPU and PyTorch 2.14 with CUDA 13.0 on an RTX 4060: unit tests and all 15 benchmark
+  cases plus their 15 fixed versions, on both devices.
+* Mixed precision (`torch.autocast` with fp16 + `GradScaler`, and bf16): bugs found, clean code stays quiet.
+* `torch.compile` (a compiled model, and a compiled loss function): findings name the original module.
+* DDP with 2 processes (`torchrun`, gloo): DDP's own internal detach is not reported. Only rank 0 prints.
+* Real code, unchanged: karpathy/nanoGPT (CUDA, with and without `compile`, with gradient accumulation)
+  and pytorch/examples `word_language_model` (manual SGD with no optimizer). Neither gave a false alarm.
+  When `model.zero_grad()` was deleted from the second one, silentbug reported it at the update line.
+
+## Limits
+
+* Not tested with FSDP, tensor or pipeline parallelism, or on multiple GPUs at once.
+* `torch.compile`: while silentbug watches, the compiler stops at silentbug's hooks and runs the
+  compiled code op by op. That is why findings inside a compiled function point at the exact line
+  there. It also means bugs that only happen in compiled graphs (compiler bugs) are not seen, and
+  the watched steps run at eager speed. Compilation goes back to normal once the check ends.
+* It only sees what runs during the steps it watches. A branch that runs only every 100 steps is
+  not checked.
+* `broadcast` checks the first three tensor arguments of each op.
+
+## Related tools
+
+* [nantrace](https://github.com/Optimus2007/nantrace) finds where NaN and Inf start. silentbug
+  finds the bugs that never produce a NaN.
+* [jaxtyping](https://github.com/patrick-kidger/jaxtyping) checks shapes if you annotate them.
+  silentbug needs no annotations.
+* `TORCH_DISTRIBUTED_DEBUG=DETAIL` names unused parameters under DDP, but not why they are unused.
+
+## License
+
+MIT. See [LICENSE](LICENSE).

silentbug-0.1.0/README.md

@@ -0,0 +1,162 @@
+# silentbug
+
+Your model trains, the loss goes down a little, and nothing errors. But one tensor is
+broadcast against another, half your parameters never get a gradient, or softmax runs
+twice. silentbug watches a few training steps and tells you **which** silent bug it is,
+**on which line**, **why** it matters and **how to fix it**.
+
+```
+$ silentbug run train.py
+silentbug: checked 3 backward pass(es), 3 optimizer step(s) on Net
+  2 likely bug(s), 0 thing(s) to check
+
+[BUG  ] 1. mse_loss got input (100, 1) and target (100,)
+    at train.py:13: loss = crit(yhat, y)
+    why: they broadcast to (100, 100), so each prediction is compared with many targets
+         instead of its own. The loss still goes down a little, which hides the bug.
+    fix: reshape one side so input and target have the same shape (often `.squeeze(-1)`)
+
+[BUG  ] 2. 2 parameters never get a gradient: cut by detach
+    at train.py:21: g = self.gate(h).detach()
+    parameters: gate.bias, gate.weight
+    fix: remove `.detach()` if these parameters should learn
+```
+
+## Install
+
+```bash
+pip install git+https://github.com/Shushant17711/silentbug@v0.1.0   # Python 3.10+, PyTorch 2.1+
+```
+
+From a clone, for development:
+
+```bash
+git clone https://github.com/Shushant17711/silentbug && cd silentbug
+pip install -e ".[test]" && pytest tests
+```
+
+## Use it
+
+On a script, with no code changes. It runs the first N optimizer steps (default 3), then stops:
+
+```bash
+silentbug run train.py --lr 3e-4         # your script's own arguments go after it
+silentbug run --steps 10 train.py
+silentbug run --json - train.py          # JSON report on stdout
+silentbug run -m mypackage.train         # like python -m
+```
+
+Exit codes: 0 nothing found, 1 likely bugs found, 2 only things to check.
+
+Or around a block of code, for example in a test:
+
+```python
+import silentbug
+
+with silentbug.check() as report:
+    for step in range(3):
+        loss = loss_fn(model(x), y)
+        loss.backward()
+        opt.step(); opt.zero_grad()
+
+assert not report.bugs, str(report)
+```
+
+Models and optimizers are found on their own. Pass them in (`silentbug.check(model,
+optimizer=opt)`) only when the model is never called inside the block.
+
+## What it finds
+
+| Kind | Example | Level |
+|---|---|---|
+| `broadcast` | `(N,1) - (N,)` gives an `(N,N)` loss, or `mse_loss(input (B,1), target (B,))` | BUG |
+| `dead-param` | parameters with no gradient, with the cause: `.detach()`, `.data`, `.item()`/NumPy round trip, `torch.no_grad()`, argmax/topk indices, a module never called (LoRA bypassed), an output that never reaches the loss | BUG |
+| `zero-grad` | gradients that exist but are always 0 (`round`, `sign`, `floor`, ...) | BUG |
+| `not-in-optimizer` | layers created after the optimizer was built | BUG |
+| `stale-optimizer` | the optimizer still updates parameters that were replaced | BUG |
+| `double-activation` | softmax fed to `cross_entropy`/`nll_loss`, sigmoid fed to `BCEWithLogits`, softmax then `log_softmax` | BUG |
+| `eval-mode` | the model is still in `eval()` while it trains | BUG |
+| `grad-not-cleared` | `optimizer.step()` uses gradients left over from the previous step | BUG |
+| `batchnorm-batch1` | batch norm in training mode with a batch of 1 | CHECK |
+
+BUG means almost certainly wrong. CHECK means often wrong but sometimes intended (for example an
+outer product written as a broadcast).
+
+How it works: a `TorchFunctionMode` sees every torch call. It checks the shapes going into
+broadcasting ops and losses, and tags each tensor with a bitmask of the trainable parameters it
+depends on. When a tensor built from parameters comes out of an op without a gradient, that line is
+recorded as a cut. Gradient hooks on every parameter record which ones received a gradient, and
+global module and optimizer hooks find the models and optimizers. After the run, each parameter
+that never got a gradient is matched to the cut or missing call that explains it.
+
+## Tested on real bugs
+
+`benchmarks/realworld/` has 15 reproducers of silent bugs from public issues (PyTorch, PEFT,
+torchtitan, kornia, DeepTab, poolformer, Lightning Bolts, simpletransformers and others). Each one
+is a short script that keeps the buggy code pattern from the issue. Each also has a fixed version
+(`f*.py`) that must produce no findings.
+
+```bash
+cd benchmarks/realworld && python score.py $(which silentbug)
+```
+
+| Case | Issue | Bug | Found at the right line |
+|---|---|---|---|
+| r01 | pytorch/pytorch#16045 | `MSELoss` on `(N,1)` vs `(N,)` | yes |
+| r02 | OpenTabular/DeepTab#441 | `(n,1)` target unsqueezed to `(B,1,1)` | yes |
+| r03 | kornia/kornia#6157 | SSIM loss broadcasts a batch-1 target | yes |
+| r04 | trnsci/trnsparse#67 | `y` broadcasts the spmv output to `(12,12)` | yes |
+| r05 | huggingface/peft#1612 | weights read directly, LoRA wrapper never runs | yes |
+| r06 | sail-sg/poolformer#16 | stage-0 output dropped, `norm0` never trains | yes |
+| r07 | pytorch/torchtitan#4802 | top-k indices cut the indexer's gradient | yes |
+| r08 | pytorch/pytorch#14467 | GRU created after the optimizer | yes |
+| r09 | RAI-SCC/jigsaw#224 | reshard replaces parameters, optimizer goes stale | yes |
+| r10 | gitabcworld/MatchingNetworks#5 | softmax, then `cross_entropy` | yes |
+| r11 | cloudtrainerwork/chesstrader#49 | softmax output fed to `CrossEntropyLoss` | yes |
+| r12 | Lightning-Universe/lightning-bolts#654 | `log_softmax(softmax(x))` | yes |
+| r13 | YuanGongND/ssast#2 | sigmoid fed to `BCEWithLogitsLoss` | yes |
+| r14 | ThilinaRajapakse/simpletransformers#941 | `model.eval()` never undone | yes |
+| r15 | juweipku/CoCo#1 | no `optimizer.zero_grad()` | yes |
+
+Result: 15 of 15 found at the right line, 15 of 15 fixed versions report nothing.
+
+## Overhead
+
+6-layer MLP, width 1024, batch 256, AdamW, CPU: 30.9 ms/step plain, 33.8 ms/step under
+silentbug (1.09x). It is meant to run for a few steps, not for a whole training run.
+
+## Limits
+
+## Tested setups
+
+* PyTorch 2.15 on CPU and PyTorch 2.14 with CUDA 13.0 on an RTX 4060: unit tests and all 15 benchmark
+  cases plus their 15 fixed versions, on both devices.
+* Mixed precision (`torch.autocast` with fp16 + `GradScaler`, and bf16): bugs found, clean code stays quiet.
+* `torch.compile` (a compiled model, and a compiled loss function): findings name the original module.
+* DDP with 2 processes (`torchrun`, gloo): DDP's own internal detach is not reported. Only rank 0 prints.
+* Real code, unchanged: karpathy/nanoGPT (CUDA, with and without `compile`, with gradient accumulation)
+  and pytorch/examples `word_language_model` (manual SGD with no optimizer). Neither gave a false alarm.
+  When `model.zero_grad()` was deleted from the second one, silentbug reported it at the update line.
+
+## Limits
+
+* Not tested with FSDP, tensor or pipeline parallelism, or on multiple GPUs at once.
+* `torch.compile`: while silentbug watches, the compiler stops at silentbug's hooks and runs the
+  compiled code op by op. That is why findings inside a compiled function point at the exact line
+  there. It also means bugs that only happen in compiled graphs (compiler bugs) are not seen, and
+  the watched steps run at eager speed. Compilation goes back to normal once the check ends.
+* It only sees what runs during the steps it watches. A branch that runs only every 100 steps is
+  not checked.
+* `broadcast` checks the first three tensor arguments of each op.
+
+## Related tools
+
+* [nantrace](https://github.com/Optimus2007/nantrace) finds where NaN and Inf start. silentbug
+  finds the bugs that never produce a NaN.
+* [jaxtyping](https://github.com/patrick-kidger/jaxtyping) checks shapes if you annotate them.
+  silentbug needs no annotations.
+* `TORCH_DISTRIBUTED_DEBUG=DETAIL` names unused parameters under DDP, but not why they are unused.
+
+## License
+
+MIT. See [LICENSE](LICENSE).

silentbug-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "silentbug"
+version = "0.1.0"
+description = "Find silent PyTorch training bugs: accidental broadcasting, parameters that never learn, and the line that caused them."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Shushant Kumar Choudhary" }]
+requires-python = ">=3.10"
+dependencies = ["torch>=2.1"]
+keywords = ["pytorch", "debugging", "broadcasting", "gradients", "training"]
+classifiers = [
+  "Programming Language :: Python :: 3",
+  "Topic :: Scientific/Engineering :: Artificial Intelligence",
+  "Topic :: Software Development :: Debuggers",
+]
+
+[project.urls]
+Homepage = "https://github.com/Shushant17711/silentbug"
+Issues = "https://github.com/Shushant17711/silentbug/issues"
+Changelog = "https://github.com/Shushant17711/silentbug/blob/main/CHANGELOG.md"
+
+[project.optional-dependencies]
+test = ["pytest"]
+
+[project.scripts]
+silentbug = "silentbug.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

silentbug-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

silentbug-0.1.0/src/silentbug/__init__.py

@@ -0,0 +1,7 @@
+"""silentbug: find silent PyTorch training bugs and the line that caused them."""
+
+from .report import BUG, CHECK, Finding, Report
+from .session import Session, check
+
+__all__ = ["check", "Session", "Report", "Finding", "BUG", "CHECK"]
+__version__ = "0.1.0"

silentbug-0.1.0/src/silentbug/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+raise SystemExit(main())

silentbug-0.1.0/src/silentbug/_where.py

@@ -0,0 +1,100 @@
+"""Find the line of user code that called the current torch op."""
+
+from __future__ import annotations
+
+import linecache
+import os
+import sys
+import sysconfig
+from dataclasses import dataclass
+
+import torch
+
+_TORCH_DIR = os.path.dirname(os.path.realpath(torch.__file__)) + os.sep
+_OWN_DIR = os.path.dirname(os.path.realpath(__file__)) + os.sep
+_LIB_DIRS = tuple(
+    {os.path.realpath(p) + os.sep for p in (sysconfig.get_paths().get("purelib"), sysconfig.get_paths().get("platlib")) if p}
+)
+_STDLIB = os.path.realpath(sysconfig.get_paths()["stdlib"]) + os.sep
+
+_real_cache: dict[str, str] = {}
+
+
+def _real(path: str) -> str:
+    r = _real_cache.get(path)
+    if r is None:
+        r = os.path.realpath(path) if not path.startswith("<") else path
+        _real_cache[path] = r
+    return r
+
+
+def _skip(path: str) -> bool:
+    if path.startswith("<"):
+        return True
+    p = _real(path)
+    return p.startswith(_TORCH_DIR) or p.startswith(_OWN_DIR) or p.startswith(_STDLIB) and not any(
+        p.startswith(d) for d in _LIB_DIRS
+    )
+
+
+def _is_lib(path: str) -> bool:
+    p = _real(path)
+    return any(p.startswith(d) for d in _LIB_DIRS)
+
+
+@dataclass(frozen=True)
+class Where:
+    file: str
+    line: int
+    func: str
+    code: str
+    lib: bool = False  # True if the line is inside an installed package
+
+    def short(self) -> str:
+        f = self.file
+        try:
+            rel = os.path.relpath(f)
+            if not rel.startswith(".."):
+                f = rel
+        except ValueError:
+            pass
+        return f"{f}:{self.line}"
+
+    def to_dict(self) -> dict:
+        return {"file": self.file, "line": self.line, "function": self.func, "code": self.code, "library": self.lib}
+
+
+# torch packages that copy, sync or update parameters as bookkeeping, not as part of the model
+_INFRA = tuple(_TORCH_DIR + d + os.sep for d in (
+    "distributed", os.path.join("nn", "parallel"), "optim", os.path.join("nn", "utils"),
+    "amp", os.path.join("cuda", "amp")))
+
+
+def torch_caller(skip: int = 2) -> str | None:
+    """'infra' or 'torch' when the nearest frame outside silentbug is PyTorch's own code."""
+    f = sys._getframe(skip)
+    while f is not None:
+        p = _real(f.f_code.co_filename)
+        if not p.startswith(_OWN_DIR):
+            if p.startswith(_INFRA) or p.startswith(_TORCH_DIR + "_tensor_str"):
+                return "infra"
+            return "torch" if p.startswith(_TORCH_DIR) else None
+        f = f.f_back
+    return None
+
+
+def where(skip: int = 2) -> tuple[Where | None, Where | None]:
+    """Return (first frame outside torch, first frame outside installed packages)."""
+    f = sys._getframe(skip)
+    first = None
+    while f is not None:
+        path = f.f_code.co_filename
+        if not _skip(path):
+            lib = _is_lib(path)
+            w = Where(path, f.f_lineno, f.f_code.co_name, linecache.getline(path, f.f_lineno).strip(), lib)
+            if first is None:
+                first = w
+            if not lib:
+                return first, (w if w != first else None)
+        f = f.f_back
+    return first, None

silentbug-0.1.0/src/silentbug/cli.py

@@ -0,0 +1,80 @@
+"""Command line: `silentbug run train.py [args...]`."""
+
+from __future__ import annotations
+
+import argparse
+import os
+import runpy
+import sys
+
+from .session import Session, _Done
+
+USAGE = """silentbug run [--steps N] [--keep-going] [--json FILE] (script.py | -m module) [args...]
+
+Runs your training script with silentbug watching the first N optimizer steps
+(default 3), prints what it found, and stops the script. Exit code: 0 nothing
+found, 1 likely bugs found, 2 only things to check."""
+
+
+def _parse(argv: list[str]):
+    ap = argparse.ArgumentParser(prog="silentbug run", usage=USAGE)
+    ap.add_argument("--steps", type=int, default=3)
+    ap.add_argument("--keep-going", action="store_true", help="let the script keep running after the report")
+    ap.add_argument("--json", metavar="FILE", help="also write the report as JSON ('-' for stdout only JSON)")
+    ap.add_argument("-m", dest="module", action="store_true", help="run a module, like python -m")
+    ap.add_argument("target")
+    ap.add_argument("args", nargs=argparse.REMAINDER)
+    return ap.parse_args(argv)
+
+
+def run(argv: list[str]) -> int:
+    a = _parse(argv)
+    s = Session(max_steps=a.steps, stop_at_limit=not a.keep_going)
+    old_argv = sys.argv[:]
+    sys.argv = [a.target, *a.args]
+    if not a.module:
+        sys.path.insert(0, os.path.dirname(os.path.abspath(a.target)))
+    error = None
+    s.start()
+    try:
+        if a.module:
+            runpy.run_module(a.target, run_name="__main__", alter_sys=True)
+        else:
+            runpy.run_path(a.target, run_name="__main__")
+    except _Done:
+        pass
+    except SystemExit:
+        pass
+    except BaseException as e:  # the script crashed: still report what we saw
+        error = e
+    finally:
+        sys.argv = old_argv
+        rep = s.stop()
+    rank = os.environ.get("RANK", "0")  # under torchrun every rank sees the same code
+    if rank != "0":
+        pass
+    elif a.json == "-":
+        print(rep.to_json())
+    else:
+        print(rep, file=sys.stderr if a.keep_going else sys.stdout)
+        if a.json:
+            with open(a.json, "w") as f:
+                f.write(rep.to_json())
+    if error is not None:
+        print(f"\n(your script raised {type(error).__name__}: {error})", file=sys.stderr)
+    return 1 if rep.bugs else (2 if rep.findings else 0)
+
+
+def main(argv: list[str] | None = None) -> int:
+    argv = sys.argv[1:] if argv is None else argv
+    if not argv or argv[0] in ("-h", "--help"):
+        print(USAGE)
+        return 0
+    if argv[0] == "run":
+        return run(argv[1:])
+    if argv[0] in ("--version", "version"):
+        from . import __version__
+        print(__version__)
+        return 0
+    print(f"unknown command {argv[0]!r}\n\n{USAGE}", file=sys.stderr)
+    return 64