dnnlpy 2026.6.11__tar.gz

dnnlpy-2026.6.11/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: dnnlpy
+Version: 2026.6.11
+Summary: A utility library for deep learning notes, providing common functions and tools to support the main content of the deep learning notes collection.
+Keywords: deep learning notes,library,utilities
+Author: Yunjie Lin
+Author-email: Yunjie Lin <jshn9510@gmail.com>
+License-Expression: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: numpy>=2.4.0,<2.5.0
+Requires-Dist: matplotlib>=3.11.0,<3.12.0
+Requires-Dist: torch>=2.12.0,<2.13.0
+Requires-Dist: pytest>=9.0.0,<9.1.0 ; extra == 'test'
+Requires-Python: >=3.12, <3.15
+Project-URL: Homepage, https://github.com/jshn9515/deep-learning-notes
+Provides-Extra: test
+Description-Content-Type: text/markdown
+
+# dnnlpy
+
+**dnnlpy** is the companion Python package for **Deep Learning Notes Library**.
+
+It provides code examples, helper functions, and small utilities used throughout the tutorial, similar in spirit to the `d2l` package for _Dive into Deep Learning_.
+
+The package structure is similar to PyTorch, but keeps a clear boundary between reusable neural network building blocks and complete model implementations:
+
+- `dnnlpy.nn` contains general neural network modules, such as attention layers, positional encodings, and other reusable components.
+- `dnnlpy.nn.functional` contains stateless helper functions, such as functional attention implementations.
+- `dnnlpy.models` contains higher-level model architectures or model-specific components, such as ViT, DDPM, or other models introduced in the notes.
+
+The APIs are designed to feel close to their PyTorch counterparts where practical, while still keeping the code lightweight and easy to read for tutorial purposes.
+
+This package is intended as a lightweight code supplement rather than a general-purpose deep learning framework. Its goal is to make the examples in the notes easier to run, reuse, and extend.
+
+## What is this package for?
+
+The `dnnlpy` package is designed to support the code in the **Deep Learning Notes Library** tutorial.
+
+It can be used to:
+
+- Organize example code from the notes
+- Provide reusable utility functions
+- Reduce repeated boilerplate in notebooks and scripts
+- Make tutorial examples easier to reproduce
+
+In short, this package serves as the code companion to the tutorial.
+
+## Requirements
+
+- Python 3.14 or newer
+- PyTorch 3.10 or newer
+
+## Installation
+
+This project uses [uv](https://docs.astral.sh/uv/) for package management.
+
+```bash
+git clone https://github.com/jshn9515/deep-learning-notes.git
+cd dnnlpy
+uv pip install .
+```
+
+If you want to modify the package while working through the notes, editable installation is recommended:
+
+```bash
+uv pip install -e .
+```
+
+This way, changes to the source code take effect immediately without reinstalling the package each time.
+
+## Examples
+
+After installation, you can import reusable neural network modules from `dnnlpy.nn`:
+
+```python
+import torch
+import dnnlpy.nn as dnn
+
+attn = dnn.MultiheadAttention(embed_dim=16, num_heads=4)
+
+query = torch.randn(2, 8, 16)
+key = torch.randn(2, 8, 16)
+value = torch.randn(2, 8, 16)
+
+output = attn(query, key, value)
+```
+
+You can also import stateless functions from `dnnlpy.nn.functional`:
+
+```python
+import torch
+import dnnlpy.nn.functional as dF
+
+query = torch.randn(2, 4, 8, 16)
+key = torch.randn(2, 4, 8, 16)
+value = torch.randn(2, 4, 8, 16)
+
+output, weights = dF.scaled_dot_product_attention(
+    query,
+    key,
+    value,
+    need_weights=True,
+)
+```
+
+Higher-level model architectures live under `dnnlpy.models`:
+
+```python
+import torch
+from dnnlpy.models.vit import ViTForImageClassification
+
+model = ViTForImageClassification(
+    image_size=224,
+    patch_size=16,
+    in_channels=3,
+    num_classes=1000,
+    embed_dim=768,
+    num_heads=12,
+    num_layers=12,
+)
+
+images = torch.randn(2, 3, 224, 224)
+logits = model(images)
+```
+
+The `dnnlpy.models.mlp` package contains small NumPy modules for teaching manual
+forward and backward passes:
+
+```python
+import dnnlpy.models.mlp as mlp
+import numpy as np
+
+model = mlp.MLP(input_dim=4, hidden_dim=8, num_classes=3)
+loss_fn = mlp.CrossEntropyLoss()
+optimizer = mlp.SGD(model.parameters(), lr=0.1)
+
+x = np.random.randn(2, 4)
+targets = np.array([0, 2])
+
+logits = model(x)
+loss = loss_fn(logits, targets)
+model.backward(loss_fn.backward())
+
+optimizer.step()
+optimizer.zero_grad()
+```
+
+A simple rule of thumb is:
+
+- Use `dnnlpy.nn` when a component is reusable across many models.
+- Use `dnnlpy.models` when the code represents a complete architecture or is tightly coupled to one model family.
+
+## License
+
+This project is licensed under the **MIT License**.

dnnlpy-2026.6.11/README.md

@@ -0,0 +1,137 @@
+# dnnlpy
+
+**dnnlpy** is the companion Python package for **Deep Learning Notes Library**.
+
+It provides code examples, helper functions, and small utilities used throughout the tutorial, similar in spirit to the `d2l` package for _Dive into Deep Learning_.
+
+The package structure is similar to PyTorch, but keeps a clear boundary between reusable neural network building blocks and complete model implementations:
+
+- `dnnlpy.nn` contains general neural network modules, such as attention layers, positional encodings, and other reusable components.
+- `dnnlpy.nn.functional` contains stateless helper functions, such as functional attention implementations.
+- `dnnlpy.models` contains higher-level model architectures or model-specific components, such as ViT, DDPM, or other models introduced in the notes.
+
+The APIs are designed to feel close to their PyTorch counterparts where practical, while still keeping the code lightweight and easy to read for tutorial purposes.
+
+This package is intended as a lightweight code supplement rather than a general-purpose deep learning framework. Its goal is to make the examples in the notes easier to run, reuse, and extend.
+
+## What is this package for?
+
+The `dnnlpy` package is designed to support the code in the **Deep Learning Notes Library** tutorial.
+
+It can be used to:
+
+- Organize example code from the notes
+- Provide reusable utility functions
+- Reduce repeated boilerplate in notebooks and scripts
+- Make tutorial examples easier to reproduce
+
+In short, this package serves as the code companion to the tutorial.
+
+## Requirements
+
+- Python 3.14 or newer
+- PyTorch 3.10 or newer
+
+## Installation
+
+This project uses [uv](https://docs.astral.sh/uv/) for package management.
+
+```bash
+git clone https://github.com/jshn9515/deep-learning-notes.git
+cd dnnlpy
+uv pip install .
+```
+
+If you want to modify the package while working through the notes, editable installation is recommended:
+
+```bash
+uv pip install -e .
+```
+
+This way, changes to the source code take effect immediately without reinstalling the package each time.
+
+## Examples
+
+After installation, you can import reusable neural network modules from `dnnlpy.nn`:
+
+```python
+import torch
+import dnnlpy.nn as dnn
+
+attn = dnn.MultiheadAttention(embed_dim=16, num_heads=4)
+
+query = torch.randn(2, 8, 16)
+key = torch.randn(2, 8, 16)
+value = torch.randn(2, 8, 16)
+
+output = attn(query, key, value)
+```
+
+You can also import stateless functions from `dnnlpy.nn.functional`:
+
+```python
+import torch
+import dnnlpy.nn.functional as dF
+
+query = torch.randn(2, 4, 8, 16)
+key = torch.randn(2, 4, 8, 16)
+value = torch.randn(2, 4, 8, 16)
+
+output, weights = dF.scaled_dot_product_attention(
+    query,
+    key,
+    value,
+    need_weights=True,
+)
+```
+
+Higher-level model architectures live under `dnnlpy.models`:
+
+```python
+import torch
+from dnnlpy.models.vit import ViTForImageClassification
+
+model = ViTForImageClassification(
+    image_size=224,
+    patch_size=16,
+    in_channels=3,
+    num_classes=1000,
+    embed_dim=768,
+    num_heads=12,
+    num_layers=12,
+)
+
+images = torch.randn(2, 3, 224, 224)
+logits = model(images)
+```
+
+The `dnnlpy.models.mlp` package contains small NumPy modules for teaching manual
+forward and backward passes:
+
+```python
+import dnnlpy.models.mlp as mlp
+import numpy as np
+
+model = mlp.MLP(input_dim=4, hidden_dim=8, num_classes=3)
+loss_fn = mlp.CrossEntropyLoss()
+optimizer = mlp.SGD(model.parameters(), lr=0.1)
+
+x = np.random.randn(2, 4)
+targets = np.array([0, 2])
+
+logits = model(x)
+loss = loss_fn(logits, targets)
+model.backward(loss_fn.backward())
+
+optimizer.step()
+optimizer.zero_grad()
+```
+
+A simple rule of thumb is:
+
+- Use `dnnlpy.nn` when a component is reusable across many models.
+- Use `dnnlpy.models` when the code represents a complete architecture or is tightly coupled to one model family.
+
+## License
+
+This project is licensed under the **MIT License**.

dnnlpy-2026.6.11/pyproject.toml

@@ -0,0 +1,44 @@
+[project]
+name = "dnnlpy"
+version = "2026.06.11"
+keywords = ["deep learning notes", "library", "utilities"]
+description = "A utility library for deep learning notes, providing common functions and tools to support the main content of the deep learning notes collection."
+authors = [{ name = "Yunjie Lin", email = "jshn9510@gmail.com" }]
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.12,<3.15"
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Education",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Mathematics",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+]
+dependencies = [
+    'numpy>=2.4.0,<2.5.0',
+    'matplotlib>=3.11.0,<3.12.0',
+    'torch>=2.12.0,<2.13.0',
+]
+
+[project.optional-dependencies]
+test = ['pytest>=9.0.0,<9.1.0']
+
+[project.urls]
+Homepage = "https://github.com/jshn9515/deep-learning-notes"
+
+[tool.uv.sources]
+torch = { index = "pytorch" }
+
+[[tool.uv.index]]
+name = "pytorch"
+url = "https://download.pytorch.org/whl/cpu"
+explicit = true
+
+[build-system]
+requires = ["uv_build>=0.11.0,<0.12.0"]
+build-backend = "uv_build"

dnnlpy-2026.6.11/src/dnnlpy/__init__.py

@@ -0,0 +1,7 @@
+from . import models as models
+from . import nn as nn
+from . import optim as optim
+from .configtools import get_data_root as get_data_root
+from .configtools import get_default_device as get_default_device
+from .configtools import set_seed as set_seed
+from .pylabtools import set_matplotlib_format as set_matplotlib_format

dnnlpy-2026.6.11/src/dnnlpy/configtools.py

@@ -0,0 +1,64 @@
+import os
+import random
+
+import numpy as np
+import torch
+import torch.accelerator as accl
+
+__all__ = [
+    'set_seed',
+    'get_default_device',
+    'get_data_root',
+]
+
+
+def set_seed(
+    seed: int = 42,
+    *,
+    deterministic: bool = False,
+    benchmark: bool = False,
+    warn_only: bool = True,
+) -> torch.Generator:
+    """Seed Python, NumPy, and PyTorch random number generators.
+
+    Args:
+        seed (int, default: 42): Seed value to apply to all supported random
+            number generators.
+        deterministic (bool, default: False): Whether to request deterministic
+            PyTorch algorithms.
+        benchmark (bool, default: False): Whether to enable cuDNN benchmark mode.
+        warn_only (bool, default: True): Whether nondeterministic PyTorch
+            operations should warn instead of raising an error.
+
+    Returns:
+        Generator: The PyTorch generator returned by ``torch.manual_seed``.
+    """
+    random.seed(seed)
+    np.random.seed(seed)
+    torch_rng = torch.manual_seed(seed)
+
+    torch.use_deterministic_algorithms(deterministic, warn_only=warn_only)
+    torch.backends.cudnn.deterministic = deterministic
+    torch.backends.cudnn.benchmark = benchmark
+
+    return torch_rng
+
+
+def get_default_device() -> torch.device:
+    """Return the current accelerator device, or CPU when none is available."""
+    device = accl.current_accelerator(check_available=True)
+    if device is not None:
+        return device
+    return torch.device('cpu')
+
+
+def get_data_root() -> str:
+    """Return the dataset root directory, creating it when necessary.
+
+    The ``DNNL_DATA_ROOT`` environment variable overrides the default
+    ``~/datasets`` location.
+    """
+    root = os.getenv('DNNL_DATA_ROOT', os.path.expanduser('~/datasets'))
+    if not os.path.exists(root):
+        os.mkdir(root)
+    return root

dnnlpy-2026.6.11/src/dnnlpy/models/__init__.py

@@ -0,0 +1,5 @@
+from . import ddpm as ddpm
+from . import mlp as mlp
+from . import seq2seq as seq2seq
+from . import vae as vae
+from . import vit as vit

dnnlpy-2026.6.11/src/dnnlpy/models/ddpm/__init__.py

@@ -0,0 +1,5 @@
+from .ddpm import DDPMScheduler as DDPMScheduler
+from .embedding import SinusoidalTimestepEmbedding as SinusoidalTimestepEmbedding
+from .unet import UNet2DModel as UNet2DModel
+from .utils import add_noise as add_noise
+from .utils import denoise as denoise

dnnlpy-2026.6.11/src/dnnlpy/models/ddpm/ddpm.py

@@ -0,0 +1,154 @@
+import torch
+from torch import Tensor
+from torch.types import Device
+
+__all__ = ['DDPMScheduler']
+
+
+class DDPMScheduler:
+    """Noise schedule and reverse-step helper for DDPM sampling."""
+
+    def __init__(
+        self,
+        num_train_timesteps: int = 1000,
+        beta_start: float = 0.0001,
+        beta_end: float = 0.02,
+    ):
+        """Scheduler for the Denoising Diffusion Probabilistic Models (DDPM) that defines
+        the noise schedule and provides a method to add noise to the original samples based
+        on the time steps.
+
+        Args:
+            num_train_timesteps (int): The total number of time steps used during training,
+                which determines the length of the noise schedule.
+            beta_start (float): The starting value of the noise variance (beta) at time step 0.
+            beta_end (float): The ending value of the noise variance (beta) at the final time step.
+        """
+        self.num_train_timesteps = num_train_timesteps
+        self.beta_start = beta_start
+        self.beta_end = beta_end
+
+        self.betas = torch.linspace(beta_start, beta_end, num_train_timesteps)
+        self.alphas = 1.0 - self.betas
+        self.alphas_cumprod = self.alphas.cumprod(dim=0)
+
+        self.num_inference_steps = num_train_timesteps
+        self.timesteps = torch.arange(num_train_timesteps - 1, -1, -1, dtype=torch.long)
+
+    def add_noise(
+        self,
+        original_samples: Tensor,
+        noise: Tensor,
+        timesteps: Tensor,
+    ) -> Tensor:
+        """Add noise to clean samples at the requested timesteps.
+
+        Args:
+            original_samples (Tensor): Clean samples ``x_0``.
+            noise (Tensor): Gaussian noise with the same shape as ``original_samples``.
+            timesteps (Tensor): 1D tensor of timestep indices, one per batch item.
+
+        Returns:
+            Noisy samples ``x_t``.
+        """
+        if original_samples.shape != noise.shape:
+            raise AssertionError(
+                '`original_samples` and `noise` must have the same shape.'
+            )
+
+        if timesteps.ndim != 1:
+            raise AssertionError(
+                '`timesteps` must be a 1D tensor of shape (batch_size,).'
+            )
+
+        self.alphas_cumprod = self.alphas_cumprod.to(original_samples.device)
+        sqrt_alpha_bar = self.alphas_cumprod[timesteps].sqrt()
+        sqrt_alpha_bar = sqrt_alpha_bar.view(-1, 1, 1, 1)
+
+        sqrt_one_minus_alpha_bar = (1.0 - self.alphas_cumprod)[timesteps].sqrt()
+        sqrt_one_minus_alpha_bar = sqrt_one_minus_alpha_bar.view(-1, 1, 1, 1)
+
+        noisy_samples = (
+            sqrt_alpha_bar * original_samples + sqrt_one_minus_alpha_bar * noise
+        )
+        return noisy_samples
+
+    def set_timesteps(
+        self,
+        num_inference_steps: int,
+        device: Device = 'cpu',
+    ):
+        """Set the inference timestep schedule.
+
+        Args:
+            num_inference_steps (int): Number of reverse diffusion steps to run.
+            device (Device, default: 'cpu'): Device where the timestep tensor should live.
+        """
+        if num_inference_steps > self.num_train_timesteps:
+            raise AssertionError(
+                f'num_inference_steps must be in the range (0, {self.num_train_timesteps}].'
+            )
+
+        self.num_inference_steps = num_inference_steps
+        self.timesteps = torch.linspace(
+            self.num_train_timesteps - 1,
+            0,
+            num_inference_steps,
+            dtype=torch.long,
+            device=device,
+        )
+
+    def previous_timestep(self, timestep: int) -> int:
+        """Return the previous inference timestep for the current schedule."""
+        if self.num_inference_steps != self.num_train_timesteps:
+            index = (self.timesteps == timestep).float().argmax()
+            if index == len(self.timesteps) - 1:
+                prev = -1
+            else:
+                prev = int(self.timesteps[index + 1])
+        else:
+            prev = timestep - 1
+        return prev
+
+    def step(self, model_output: Tensor, timestep: int, sample: Tensor) -> Tensor:
+        """Perform a single reverse diffusion step to compute the previous sample given the
+        model's output, the current time step, and the current sample.
+
+        Args:
+            model_output (Tensor): The output from the diffusion model, which is typically
+                the predicted noise component at the current time step.
+            timestep (int): The current time step in the reverse diffusion process.
+            sample (Tensor): The current noisy sample at the given time step.
+        """
+        t = timestep
+        prev_t = self.previous_timestep(t)
+
+        alpha_t = self.alphas[t]
+        alpha_bar_t = self.alphas_cumprod[t]
+        beta_t = self.betas[t]
+
+        if prev_t >= 0:
+            alpha_bar_prev = self.alphas_cumprod[prev_t]
+        else:
+            alpha_bar_prev = torch.tensor(1.0, device=sample.device)
+
+        pred_original_sample = (
+            sample - (1 - alpha_bar_t).sqrt() * model_output
+        ) / alpha_bar_t.sqrt()
+
+        param1 = alpha_bar_prev.sqrt() * beta_t / (1 - alpha_bar_t)
+        param2 = alpha_t.sqrt() * (1 - alpha_bar_prev) / (1 - alpha_bar_t)
+        mean = param1 * pred_original_sample + param2 * sample
+
+        if prev_t >= 0:
+            variance = (1 - alpha_bar_prev) / (1 - alpha_bar_t) * beta_t
+        else:
+            variance = torch.tensor(0.0, device=sample.device)
+
+        if timestep > 0:
+            noise = torch.randn_like(sample)
+            prev_sample = mean + variance.sqrt() * noise
+        else:
+            prev_sample = mean
+
+        return prev_sample

dnnlpy-2026.6.11/src/dnnlpy/models/ddpm/embedding.py

@@ -0,0 +1,44 @@
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch import Tensor
+
+__all__ = ['SinusoidalTimestepEmbedding']
+
+
+class SinusoidalTimestepEmbedding(nn.Module):
+    """Create sinusoidal embeddings for diffusion timesteps."""
+
+    def __init__(self, embedding_dim: int, max_period: int = 10000):
+        """Initialize timestep embedding parameters.
+
+        Args:
+            embedding_dim (int): Size of each timestep embedding.
+            max_period (int, default: 10000): Controls the minimum sinusoidal frequency.
+        """
+        super().__init__()
+        self.embedding_dim = embedding_dim
+        self.max_period = max_period
+
+    def forward(self, timesteps: Tensor) -> Tensor:
+        """Embed a 1D tensor of timesteps."""
+        half_dim = self.embedding_dim // 2
+        if half_dim == 0:
+            return torch.zeros(
+                timesteps.size(0),
+                self.embedding_dim,
+                device=timesteps.device,
+                dtype=torch.float32,
+            )
+
+        scale = -math.log(self.max_period) / max(half_dim - 1, 1)
+        emb = torch.arange(half_dim, device=timesteps.device) * scale
+        emb = timesteps.unsqueeze(1) * emb.exp().unsqueeze(0)
+        emb = torch.concat([emb.sin(), emb.cos()], dim=-1)
+
+        if self.embedding_dim % 2 == 1:
+            emb = F.pad(emb, (0, 1))
+
+        return emb