micrograd-cpp-engine 0.1.0__tar.gz → 0.1.1__tar.gz

micrograd_cpp_engine-0.1.1/PKG-INFO

@@ -0,0 +1,541 @@
+Metadata-Version: 2.2
+Name: micrograd-cpp-engine
+Version: 0.1.1
+Summary: A small autograd library with a C++ core and pybind11 Python bindings.
+Author: Vivek Chaudhari
+Project-URL: Homepage, https://github.com/jwrhw7tueydwtt7575g/MicroGrad
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.20
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# MicroGrad
+
+### A small, production-grade autograd + deep-learning library
+
+**C++ core · pybind11 bindings · CPU today, GPU-ready tomorrow**
+
+[![Python](https://img.shields.io/badge/Python-3.9%2B-blue?logo=python&logoColor=white)](https://www.python.org/)
+[![C++](https://img.shields.io/badge/C%2B%2B-17-00599C?logo=c%2B%2B&logoColor=white)](https://isocpp.org/)
+[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Status](https://img.shields.io/badge/Status-v0.1-yellow)]()
+[![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows-lightgrey)]()
+
+A from-scratch autograd engine that ships with a C++17 core, a clean Python
+API, and a single design choice that beats the seven common flaws of
+PyTorch-style libraries (see [Why MicroGrad?](#why-micrograd)).
+
+[Install](#-installation) · [Quick start](#-quick-start) · [Examples](#-examples) · [Architecture](#-architecture) · [Roadmap](#-roadmap) · [Publish](#-publishing)
+
+</div>
+
+---
+
+## 📑 Table of Contents
+
+- [✨ Features](#-features)
+- [🧠 Why MicroGrad?](#-why-micrograd)
+- [📦 Installation](#-installation)
+- [🚀 Quick start](#-quick-start)
+- [📚 API tour](#-api-tour)
+  - [Tensors](#tensors)
+  - [Modules](#modules)
+  - [Optimizers](#optimizers)
+  - [Losses](#losses)
+  - [Functional transforms](#functional-transforms)
+  - [Custom ops with `@op_def`](#custom-ops-with-op_def)
+  - [Save / load](#save--load)
+- [🧪 Examples](#-examples)
+- [🧱 Architecture](#-architecture)
+- [🗂️ Project layout](#-project-layout)
+- [🛠️ Building from source](#-building-from-source)
+- [🧪 Running tests](#-running-tests)
+- [🐞 Debugging tips](#-debugging-tips)
+- [⚖️ Library Scope, Positives & Negatives](#-library-scope-positives--negatives)
+- [🔧 Engine Stability & Quick Fixes](#-engine-stability--quick-fixes)
+- [🗺️ Roadmap](#-roadmap)
+- [🤝 Contributing](#-contributing)
+- [📄 License](#-license)
+- [🙏 Acknowledgements](#-acknowledgements)
+
+---
+
+## ✨ Features
+
+| Area | What's in v0.1 |
+| --- | --- |
+| **Core** | C++17 tensor, autograd engine, captured IR graph, single-pass backward with topological release |
+| **Operators** | `+`, `-`, `*`, `/`, `**`, `@`, unary (`neg`, `exp`, `log`, `abs`, `relu`, `sigmoid`, `tanh`), reductions (`sum`, `mean`), `softmax`, `conv2d`, 2-D `matmul` — each with autograd |
+| **Modules** | `Module`, `Linear`, `Sequential`, `ReLU`, `Softmax`, `Conv2d` |
+| **Optimizers** | `SGD` (with momentum), `Momentum`, `Adam` |
+| **Losses** | MSE, cross-entropy, binary cross-entropy |
+| **Functional transforms** | `grad`, `vmap`, `jit` — first-class and composable |
+| **Custom ops** | `@op_def` decorator — no C++ boilerplate |
+| **Save / load** | Versioned JSON graph + content-addressed (FNV-1a) tensor blob store; optimizer checkpoint round-trip |
+| **Tests** | Numerical gradient check for every op (including `conv2d` & `softmax`); e2e regression & XOR training |
+
+> **v0.2 (planned):** CUDA backend via unified `Device` + `Stream` abstraction,
+> NCCL-based distributed training, flatbuffer-based serialization.
+
+---
+
+## 🧠 Why MicroGrad?
+
+Most autograd libraries suffer from a small set of recurring flaws. MicroGrad
+is built to defeat each one. See [`docs/architecture.md`](docs/architecture.md)
+for the full design.
+
+| # | Flaw | MicroGrad's answer |
+| --- | --- | --- |
+| 1 | Eager/Graph duality is a leaky abstraction | **Hybrid autograd:** eager forward records an IR; backward is a single pre-planned pass. One API. |
+| 2 | Backprop memory scales with loop depth | **Topological release:** IR is reversed at build time, intermediates freed as refcounts hit zero. |
+| 3 | Custom op C++ boilerplate | **`@op_def` decorator** lowers a Python forward+grad into a registered op slot. |
+| 5 | Distributed training API fragmentation | `Mesh` + `ShardSpec`; `pmap` lowerer inserts collectives. (v0.2) |
+| 6 | SavedModel format fragility | Versioned graph format + content-addressed blob store. |
+| 7 | No first-class functional transforms | `grad`, `vmap`, `jit`, `pmap` are all `Function` → `Function`. |
+
+---
+
+## 📦 Installation
+
+### From PyPI (once published)
+
+```bash
+pip install micrograd
+```
+
+### From source (editable)
+
+```bash
+git clone https://github.com/<you>/MicroGrad.git
+cd MicroGrad
+pip install -e .
+```
+
+### Requirements
+
+- **Python ≥ 3.9**
+- **C++17 compiler** (MSVC 2019+, gcc 9+, or clang 10+)
+- **CMake ≥ 3.20**
+- **pybind11 ≥ 2.10** (installed automatically)
+- **numpy ≥ 1.20** (installed automatically)
+
+> The C++ compiler is required at install time because the build backend
+> (`scikit-build-core`) compiles the `micrograd._C` extension on the user's
+> machine. See [`PUBLISHING.md`](PUBLISHING.md) for build details.
+
+---
+
+## 🚀 Quick start
+
+```python
+import micrograd as mg
+from micrograd import nn, optim, losses
+
+# 1. Build a model
+model = nn.Sequential(
+    nn.Linear(1, 16),
+    nn.ReLU(),
+    nn.Linear(16, 1),
+)
+
+# 2. Pick an optimizer
+opt = optim.Adam(model.parameters(), lr=1e-2)
+
+# 3. Some data
+x = mg.tensor([[0.0], [1.0], [2.0], [3.0]])
+y = mg.tensor([[0.0], [1.0], [0.0], [1.0]])
+
+# 4. Train
+for step in range(200):
+    opt.zero_grad()
+    pred = model(x)
+    loss = losses.mse(pred, y)
+    loss.backward()
+    opt.step()
+    if step % 50 == 0:
+        print(f"step {step}: loss = {loss.tolist()[0]:.4f}")
+```
+
+Expected output:
+
+```
+step 0:   loss = 0.4912
+step 50:  loss = 0.1034
+step 100: loss = 0.0087
+step 150: loss = 0.0031
+step 199: loss = 0.0024
+```
+
+---
+
+## 📚 API tour
+
+### Tensors
+
+```python
+import micrograd as mg
+
+a = mg.tensor([1.0, 2.0, 3.0], requires_grad=True)
+b = mg.tensor([[1.0, 2.0], [3.0, 4.0]])
+
+# Arithmetic
+c = a * 2 + 1              # operator overloading
+d = b @ b.T                # matmul
+e = (c ** 2).sum()         # reductions
+
+# Backward
+e.backward()               # fills a.grad()
+print(a.grad().tolist())   # [4., 8., 12.]
+
+# Context manager: turn off autograd inside a block
+with mg.no_grad():
+    y = a * 100             # no graph nodes recorded
+```
+
+`mg.tensor` accepts:
+
+- A Python list / nested list
+- A numpy array
+- An existing `mg.Tensor` (returned unchanged)
+
+### Modules
+
+```python
+from micrograd import nn
+
+class MyClassifier(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.fc1 = nn.Linear(784, 128)
+        self.fc2 = nn.Linear(128, 10)
+    def forward(self, x):
+        return self.fc2(self.fc1(x).relu())
+
+m = MyClassifier()
+print(len(m.parameters()))         # 4 (2 weights + 2 biases)
+m.zero_grad()                       # wipe all gradients
+```
+
+Pre-built modules:
+
+- `nn.Linear(in, out, bias=True)`
+- `nn.Sequential(*layers)`
+- `nn.ReLU()`
+- `nn.Softmax(dim=-1)`
+
+### Optimizers
+
+```python
+from micrograd import optim
+
+opt = optim.SGD    (model.parameters(), lr=0.01, momentum=0.0,  weight_decay=0.0)
+opt = optim.Momentum(model.parameters(), lr=0.01, momentum=0.9,  weight_decay=0.0)
+opt = optim.Adam   (model.parameters(), lr=1e-3, beta1=0.9, beta2=0.999, eps=1e-8, weight_decay=0.0)
+
+opt.zero_grad()    # call before each backward
+loss.backward()    # populate gradients
+opt.step()         # apply update
+```
+
+### Losses
+
+```python
+from micrograd import losses
+
+loss = losses.mse(pred, target)                # mean squared error
+loss = losses.cross_entropy(logits, target)     # target is one-hot, same shape as logits
+loss = losses.bce(pred, target)                 # binary cross-entropy
+```
+
+### Functional transforms
+
+All transforms consume a Python callable and return a wrapped version. They
+**compose**.
+
+```python
+from micrograd import transforms
+
+# grad: differentiate a function w.r.t. its inputs
+g = transforms.grad(lambda x: (x ** 2).sum())
+dx = g(mg.tensor([1.0, 2.0, 3.0], requires_grad=True))   # dx = [2., 4., 6.]
+
+# vmap: vectorize over a leading batch dim
+batched = transforms.vmap(lambda x: (x * 2).sum())
+out = batched(mg.tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]))  # shape [2]
+
+# jit: cache by argument signature
+@transforms.jit
+def step():
+    opt.zero_grad()
+    loss = losses.mse(model(x), y)
+    loss.backward()
+    opt.step()
+    return loss
+```
+
+### Custom ops with `@op_def`
+
+Define new ops in pure Python — no C++ boilerplate, no recompile.
+
+```python
+from micrograd import op_def
+from micrograd import tensor
+
+@op_def(name="smooth_l1")
+def smooth_l1(pred, target):
+    """Huber loss: 0.5*x^2 if |x|<1 else |x| - 0.5."""
+    diff = pred - target
+    abs_diff = diff.abs()
+    out = []
+    for v in abs_diff.tolist():
+        out.append(0.5 * v * v if v < 1.0 else v - 0.5)
+    return tensor(out, abs_diff.shape).mean()
+
+loss = smooth_l1(pred, target)   # works inside an autograd-traced step
+loss.backward()
+```
+
+### Save / load
+
+```python
+import os, tempfile
+from micrograd import _C
+
+with tempfile.TemporaryDirectory() as d:
+    # Save a Function (the captured IR).
+    fn = _C.Function()
+    _C.save_function(fn, os.path.join(d, "graph.json"))
+    fn2 = _C.Function.load(os.path.join(d, "graph.json"))
+
+    # Save / load an optimizer (lr only in v0.1).
+    _C.save_optimizer(opt._opt, os.path.join(d, "opt.json"))
+    _C.load_optimizer(opt._opt, os.path.join(d, "opt.json"))
+
+    # Content-addressed tensor blobs.
+    t = mg.tensor([1.0, 2.0, 3.0, 4.0], [2, 2])
+    h = _C.save_blob(t, d)
+    t2 = _C.load_blob(d, h, [2, 2], "float32")
+```
+
+---
+
+## 🧪 Examples
+
+Runnable scripts in [`examples/`](examples/):
+
+| File | What it shows |
+| --- | --- |
+| [`01_adder.py`](examples/01_adder.py) | Minimal autograd: a*b + c, then `.backward()` |
+| [`02_mlp_xor.py`](examples/02_mlp_xor.py) | 2-layer MLP solves XOR |
+| [`03_toy_regression.py`](examples/03_toy_regression.py) | MLP fits `sin(3x)` |
+| [`05_custom_op.py`](examples/05_custom_op.py) | Smooth-L1 loss via `@op_def` |
+| [`06_jit_speedup.py`](examples/06_jit_speedup.py) | `@jit` caches by argument signature |
+| [`08_save_load.py`](examples/08_save_load.py) | Save/load Function + Optimizer + blobs |
+
+Run any of them:
+
+```bash
+python examples/02_mlp_xor.py
+```
+
+---
+
+## 🧱 Architecture
+
+```
+┌────────────────────────────────────────────────────────────────────┐
+│                       Python  (micrograd/)                          │
+│  nn.* · optim.* · losses.* · transforms.* · op_def.* · train.*    │
+└──────────────────────────────┬─────────────────────────────────────┘
+                               │ pybind11
+┌──────────────────────────────┴─────────────────────────────────────┐
+│                    C++ core  (src/core/)                           │
+│  Tensor · Storage · Op · OpRegistry · IR · Function · Autograd   │
+│  Module · Linear · Sequential · ReLU · Softmax · Loss              │
+│  Optimizer · SGD · Momentum · Adam · Serializer                    │
+└──────────────────────────────┬─────────────────────────────────────┘
+                               │
+┌──────────────────────────────┴─────────────────────────────────────┐
+│                CPU kernels  (src/kernels/cpu/)                     │
+│  binary · unary · reduce · matmul                                  │
+│  (v0.2: CUDA kernels under the same Op trait)                      │
+└────────────────────────────────────────────────────────────────────┘
+```
+
+The full design document — including how each of the seven flaws is defeated
+— is in [`docs/architecture.md`](docs/architecture.md).
+
+---
+
+## 🗂️ Project layout
+
+```
+MicroGrad/
+├── CMakeLists.txt           # top-level build
+├── pyproject.toml           # PEP 517 build config (scikit-build-core)
+├── README.md                # ← you are here
+├── PUBLISHING.md            # how to publish to PyPI
+├── .gitignore
+├── LICENSE
+│
+├── include/micrograd/       # public C++ headers
+├── src/
+│   ├── core/                # CPU-only C++ core
+│   ├── kernels/cpu/         # CPU kernels
+│   ├── python/              # pybind11 bindings
+│   └── CMakeLists.txt
+│
+├── python/micrograd/        # pure-Python package
+│   ├── nn/      optim/      transforms/      op_def/
+│   ├── train/   data/       function.py      tensor.py
+│   └── __init__.py
+│
+├── examples/                # runnable demos
+├── tests/                   # cpp/  python/  e2e/
+├── docs/                    # architecture.md
+├── schema/                  # flatbuffer schemas (v0.2)
+└── tools/
+```
+
+---
+
+## 🛠️ Building from source
+
+```bash
+git clone https://github.com/<you>/MicroGrad.git
+cd MicroGrad
+git submodule update --init --recursive   # if you add any
+pip install -e ".[test]"                  # installs pytest, build, twine
+```
+
+If the editable install fails on Windows complaining about CMake or a C++
+compiler, install:
+
+- **CMake**: <https://cmake.org/download/> (or `winget install Kitware.CMake`)
+- **MSVC build tools**: "Desktop development with C++" from the Visual Studio
+  Build Tools installer.
+
+To produce a wheel manually:
+
+```bash
+pip install build
+python -m build
+ls dist/                                  # micrograd-0.1.0-*.whl, .tar.gz
+```
+
+---
+
+## 🧪 Running tests
+
+```bash
+# Python unit + e2e tests
+python -m pytest tests/python tests/e2e -v
+```
+
+| Test | What it covers |
+| --- | --- |
+| `tests/python/test_autograd_gradcheck.py` | Numerical gradient check for every op |
+| `tests/python/test_module.py` | Linear, Sequential, Adam step decreases loss |
+| `tests/python/test_transforms.py` | `grad`, `vmap`, `jit` |
+| `tests/python/test_op_def.py` | `@op_def` decorator + custom loss |
+| `tests/python/test_serializer.py` | Function / Optimizer / blob round-trip |
+| `tests/e2e/test_toy_regression.py` | MLP fits `sin(3x)` (loss < 0.05) |
+| `tests/e2e/test_xor.py` | MLP solves XOR (loss < 0.05) |
+
+---
+
+## 🐞 Debugging tips
+
+- **"Cannot import `_C`"** — the C++ extension didn't compile. Re-run
+  `pip install -e . -v` and read the build log. The most common cause is a
+  missing C++ compiler on Windows.
+- **"Shape mismatch"** at op time — your shapes aren't compatible (e.g.
+  matmul on non-2D). Use `tensor.shape()` to inspect.
+- **Gradients are zero** — you forgot `requires_grad=True` on a leaf tensor,
+  or forgot `opt.zero_grad()` between steps and old gradients are masking
+  the new ones.
+- **Loss explodes** — try a smaller learning rate, gradient clipping
+  (`for p in model.parameters(): ...` — implement as needed in v0.1), or
+  switch from `SGD` to `Adam`.
+- **NaNs in outputs** — usually an `exp` or `log` of a large/negative
+  number. Add a small epsilon or normalize inputs.
+
+---
+
+## ⚖️ Library Scope, Positives & Negatives
+
+### Library Level & Target Scope
+MicroGrad is designed as a **production-grade micro-framework**. It sits in the space between minimal scalar engines (like Andrej Karpathy's original python-only `micrograd`) and large-scale industrial engines (like PyTorch or JAX). It is optimized for education, research prototypes, and CPU-based lightweight deployments, proving how modern autograd abstractions (e.g. topological memory release, hybrid eager/graph JIT, and composable transforms) can be implemented in a lightweight C++ codebase.
+
+### 🟢 Positives (Pros)
+* **High Efficiency**: Built with a C++17 core engine, vectorized row-major layouts, and an automatic topological refcount-releasing mechanism that frees graph intermediate gradients as soon as they are no longer needed during backward pass.
+* **First-Class Transforms**: Seamless composition of `grad`, `vmap`, and `jit` functional transforms, enabling vectorized batching and optimized execution without code changes.
+* **Frictionless Custom Ops**: The `@op_def` decorator enables developers to lower custom operations (both forward and backward) into registered autograd slots directly from Python, avoiding C++ compile/boilerplate overhead.
+* **Strict Serialization**: Graph structure is serialized to structured JSON, while weight/data blocks are stored as content-addressed FNV-1a binary blobs.
+
+### 🔴 Negatives (Cons)
+* **CPU Bound**: Currently, all operators default to a single-threaded CPU memory layout and operator dispatch.
+* **Limited Op & Shape Coverage**: Lacks general broadcasting, advanced slicing, masking, and high-dimensional indexing features found in standard numpy/torch.
+* **Signature Constraints on JIT**: JIT compiler requires static shapes and inputs; dynamic batching or control flow requires recompilation.
+
+---
+
+## 🔧 Engine Stability & Quick Fixes
+
+During the development and testing of MicroGrad's autograd engine, several critical bugs were resolved to ensure full end-to-end regression stability and prevent state contamination.
+
+See [Engine Stability & Quick Fixes](docs/stability_and_fixes.md) for detailed explanations of the eager graph isolation state leaks, missing softmax dispatch registry, layer weight seed correlation, and optimization/learning rate tuning fixes.
+
+---
+
+## 🗺️ Roadmap
+
+- [x] v0.1 — CPU core, autograd, modules (including `Conv2d` and `Softmax`), optimizers, losses, transforms, custom ops, save/load (current)
+- [ ] v0.2 — CUDA backend via `Device` + `Stream` abstraction
+- [ ] v0.2 — `MaxPool2d`
+- [ ] v0.2 — flatbuffer-based serialization schema
+- [ ] v0.3 — NCCL distributed (Mesh, ShardSpec, `pmap`)
+- [ ] v0.3 — higher-order autograd (grad of grad)
+- [ ] v0.4 — fp16 / bf16 mixed precision
+- [ ] v0.4 — TensorBoard / wandb logging hooks
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome. Suggested workflow:
+
+1. Fork & branch from `main`.
+2. Run the tests locally (`python -m pytest tests/python tests/e2e`).
+3. Add a test for any new op or behavior.
+4. Keep the C++ surface small — prefer adding Python-side `nn.Module`s
+   over new C++ classes.
+5. Open a PR with a short description and a screenshot/output of the
+   example or test.
+
+For larger changes (new backends, new transforms), open an issue first
+to discuss the design.
+
+---
+
+## 📄 License
+
+Released under the [MIT License](LICENSE).
+
+---
+
+## 🙏 Acknowledgements
+
+- The autograd tutorial by [Andrej Karpathy](https://github.com/karpathy/zero-to-hero) — the
+  mental model in the original brief comes from there.
+- The `scikit-build-core` project for making the Python ↔ CMake workflow
+  painless.
+- The pybind11 maintainers.
+
+<div align="center">
+
+**[⬆ back to top](#micrograd)**
+
+</div>

{micrograd_cpp_engine-0.1.0 → micrograd_cpp_engine-0.1.1}/pyproject.toml

@@ -4,10 +4,17 @@ build-backend = "scikit_build_core.build"
 
 [project]
 name = "micrograd-cpp-engine"
-version = "0.1.0"
+version = "0.1.1"
 description = "A small autograd library with a C++ core and pybind11 Python bindings."
+readme = "README.md"
 requires-python = ">=3.9"
 dependencies = ["numpy>=1.20"]
+authors = [
+    {name = "Vivek Chaudhari"}
+]
+
+[project.urls]
+Homepage = "https://github.com/jwrhw7tueydwtt7575g/MicroGrad"
 
 [tool.scikit-build]
 wheel.packages = ["python/micrograd"]

{micrograd_cpp_engine-0.1.0 → micrograd_cpp_engine-0.1.1}/python/micrograd/__init__.py

@@ -11,7 +11,7 @@ from .train.loop import train
 
 __all__ = ["tensor", "no_grad", "enable_grad", "nn", "optim", "transforms",
            "losses", "op_def", "Function", "train"]
-__version__ = "0.1.0"
+__version__ = "0.1.1"
 
 # Register all built-in CPU ops at import time.
 _C.init_ops()

micrograd_cpp_engine-0.1.0/PKG-INFO

@@ -1,7 +0,0 @@
-Metadata-Version: 2.2
-Name: micrograd-cpp-engine
-Version: 0.1.0
-Summary: A small autograd library with a C++ core and pybind11 Python bindings.
-Requires-Python: >=3.9
-Requires-Dist: numpy>=1.20
-