acids-cc 3.0__py3-none-any.whl

acids_cc-3.0.dist-info/METADATA

@@ -0,0 +1,220 @@
+Metadata-Version: 2.4
+Name: acids-cc
+Version: 3.0
+Summary: Cached convolution layers for real-time streaming inference with PyTorch (upgrade of Caillon's cached_conv 2021 package)
+Author: Axel Chemla--Romeu-Santos, Nils Demerlé
+Author-email: ACIDS <acids.paris@gmail.com>
+License: MIT
+Project-URL: Documentation, https://cached-conv.readthedocs.io
+Project-URL: Repository, https://github.com/acids-ircam/acids-cc
+Keywords: deep learning,pytorch,convolution,streaming,real-time
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: torch>1.8
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: sphinx>=7; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+Requires-Dist: myst-parser; extra == "docs"
+Requires-Dist: sphinx-copybutton; extra == "docs"
+
+# cached_conv
+
+[![Documentation Status](https://readthedocs.org/projects/cached-conv/badge/?version=latest)](https://acids-cc.readthedocs.io/en/latest/)
+[![docs](https://github.com/acids-ircam/acids-cc/actions/workflows/docs.yml/badge.svg)](https://github.com/acids-ircam/acids-cc/actions/workflows/docs.yml)
+
+**Cached convolutions for real-time streaming inference with PyTorch.**
+
+`cached_conv` provides drop-in replacements for PyTorch convolutional layers
+that process a signal **block by block** and give the same result as
+processing it all at once. Each layer keeps the end of the previous block in
+an internal cache, used instead of zero padding at block boundaries.
+
+The same model definition can be built in two modes:
+
+- **eager** (default): regular layers, for training on full signals;
+- **cached**: streaming layers, for real-time inference, e.g. once exported
+  with TorchScript.
+
+## Features
+
+- 1D, 2D and 3D convolutions and transposed convolutions. Time is the last
+  axis, the only one that is streamed.
+- `"same"`, `"causal"` and `"anticausal"` padding, computed from the kernel
+  size and dilation.
+- Strides, dilation and groups.
+- Latency tracking: every layer exposes its `cumulative_delay`, which
+  accumulates across a whole model.
+- Windowed convolutions: the kernel is multiplied by a window before
+  convolving.
+- StyleGAN 2-modulated convolutions. The style can change from one block to
+  the next.
+- Streamable upsampling (nearest and linear), group normalization and
+  alignment of parallel branches.
+- TorchScript export, on CPU, CUDA and MPS.
+
+## Installation
+
+```bash
+pip install git+https://github.com/acids-ircam/acids-cc.git
+```
+
+Requires Python ≥ 3.9 and PyTorch ≥ 1.9.
+
+## Quick start
+
+```python
+import torch
+import cached_conv as cc
+
+
+def build_model():
+    conv1 = cc.Conv1d(1, 16, 5, padding="same")
+    conv2 = cc.Conv1d(16, 1, 5, padding="same",
+                      cumulative_delay=conv1.cumulative_delay)
+    return cc.CachedSequential(conv1, conv2)
+
+
+# 1. train in eager mode (the default)
+model = build_model()
+
+# 2. build the same model with cached layers, and load the trained weights
+cc.use_cached_conv(True)
+streaming_model = build_model()
+streaming_model.load_state_dict(model.state_dict())
+
+# 3. process a stream block by block
+stream = torch.randn(1, 1, 8192)
+y = torch.cat([streaming_model(block) for block in stream.split(512, -1)], -1)
+
+# the output is delayed by the model's latency, in samples. The edges differ:
+# before the signal starts, eager layers see zero padding, cached layers see
+# the previous layer's response to silence
+delay = streaming_model.cumulative_delay
+offline = model(stream)
+assert torch.allclose(y[..., 2 * delay:], offline[..., delay:-delay],
+                      atol=1e-5)
+```
+
+Cached layers can't look into the future: centered convolutions delay their
+output instead. Pass each layer's `cumulative_delay` to the next one so that
+the total latency is tracked correctly. With `padding="causal"` everywhere,
+the latency only comes from strides.
+
+## Layers
+
+The layers exposed by the package (`cc.Conv1d`, `cc.Upsample2d`...) are
+factories: they return a regular or a cached layer, depending on
+`cc.use_cached_conv` at construction time.
+
+| | 1D | 2D | 3D |
+| --- | --- | --- | --- |
+| Convolution | `Conv1d` | `Conv2d` | `Conv3d` |
+| Transposed convolution | `ConvTranspose1d` | `ConvTranspose2d` | `ConvTranspose3d` |
+| Windowed convolution | `WindowedConv1d` | `WindowedConv2d` | `WindowedConv3d` |
+| Modulated convolution | `ModulatedConv1d` | `ModulatedConv2d` | `ModulatedConv3d` |
+| Upsampling | `Upsample1d` | `Upsample2d` | `Upsample3d` |
+| Branch alignment | `AlignBranches1d` | `AlignBranches2d` | `AlignBranches3d` |
+| Group normalization | `CachedGroupNorm1d` | `CachedGroupNorm2d` | `CachedGroupNorm3d` |
+
+Inputs are shaped `(batch, channels, time)`,
+`(batch, channels, height, time)` and `(batch, channels, depth, height, time)`.
+In 2D and 3D, `padding` sets the mode along time; the other axes are always
+padded "same".
+
+```python
+# encoder / decoder; the transposed kernel size along time must be
+# stride + 2 * padding
+down = cc.Conv1d(16, 16, 4, stride=2, padding="same")
+up = cc.ConvTranspose1d(16, 16, 4, stride=2, padding=1,
+                        cumulative_delay=down.cumulative_delay)
+
+# spectrogram convolution (frequency, time), windowed along time only
+conv = cc.WindowedConv2d(1, 16, (3, 33), window=(None, "hann"))
+
+# StyleGAN 2 modulation
+mod = cc.ModulatedConv1d(16, 32, 3, padding="same", style_dim=64)
+y = mod(torch.randn(2, 16, 512), torch.randn(2, 64))
+
+# upsampling by an integer factor along time
+upsample = cc.Upsample1d(2, mode="linear",
+                         cumulative_delay=down.cumulative_delay)
+
+# residual connection: delays the identity branch to match the convolution
+conv = cc.Conv1d(16, 16, 3, padding="same")
+residual = cc.AlignBranches1d(conv, torch.nn.Identity(),
+                              delays=[conv.cumulative_delay, 0])
+y_conv, y_identity = residual(torch.randn(1, 16, 512))
+```
+
+## Exporting with TorchScript
+
+Caches are allocated on the first call. Run the cached model once before
+scripting it:
+
+```python
+cc.use_cached_conv(True)
+model = build_model().eval()
+
+with torch.no_grad():
+    model(torch.zeros(1, 1, 512))
+
+torch.jit.script(model).save("model.ts")
+```
+
+Caches have room for `max_batch_size` streams (a layer argument). Block
+lengths must be divisible by the model's total stride.
+
+## Testing
+
+`cc.test_equal` checks that a model gives the same output when streamed as
+when processed offline:
+
+```python
+assert cc.test_equal(build_model, torch.randn(1, 1, 4096))
+```
+
+To run the library's test suite, on all available devices:
+
+```bash
+pip install -e ".[dev]"
+pytest tests
+```
+
+To run it against several Python / PyTorch versions with
+[uv](https://docs.astral.sh/uv/):
+
+```bash
+./test_versions.sh -p "3.9 3.12" -t "1.13 2.0 latest"
+./test_versions.sh --lowest          # oldest torch for each Python version
+```
+
+`tests/test_audio.py` also writes audio examples to `tests/audio_outputs/`
+(waveforms in 1D, spectrograms in 2D / 3D), in eager and streaming modes.
+
+## Documentation
+
+The full documentation is on
+[cached-conv.readthedocs.io](https://cached-conv.readthedocs.io). To build it
+locally:
+
+```bash
+pip install -r docs/requirements.txt
+sphinx-build -b html docs docs/_build/html
+```
+
+## Credits
+
+Developed at [ACIDS](https://acids.ircam.fr) (IRCAM) by Axel
+Chemla--Romeu-Santos and Nils Demerlé. Upgrade of the original
+[cached_conv](https://github.com/acids-ircam/cached_conv) package by Antoine
+Caillon (2021).
+
+## License
+
+MIT

acids_cc-3.0.dist-info/RECORD

@@ -0,0 +1,11 @@
+cached_conv/__init__.py,sha256=ipXUyiMTOkIFXC1WZzAag2PDHM79hayQbW58ueiguR8,5548
+cached_conv/conv_1d.py,sha256=cD1crJmtqDoJEiG3kR6_5naah8GUX22VTuAbyoAyGYg,9670
+cached_conv/conv_2d.py,sha256=E5-7imoOXdGYMd_gDNaH3NycL_lA6g5_TWCh7k4kaDQ,6960
+cached_conv/conv_3d.py,sha256=qJyfYpOyQGDpT1C5BkEGPBQClCNC-5rmSHZn0P4qQs8,6662
+cached_conv/upsample.py,sha256=Wh6Ici3Rl16HdDq9rfW7Iw9ZVOBCR4Sh2N7JPpGkBdQ,5913
+cached_conv/utils.py,sha256=IF_8yLADQJUem4609sr6DlQqbTeRxLGmwlfW25B0Vqk,3696
+cached_conv/variants.py,sha256=13hhgctHuv7JLfABQZoZewcVGBSG5cu4VJQ-gt0KqQQ,6126
+acids_cc-3.0.dist-info/METADATA,sha256=sT3H5X7sj2MCzdL1RwDR2Uzl0NtrtfXiEHSus1jD06Q,7641
+acids_cc-3.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
+acids_cc-3.0.dist-info/top_level.txt,sha256=eL4WC83WC3L5w7KL96_vqaOwAKWeoW3RgUgi7UhlRos,12
+acids_cc-3.0.dist-info/RECORD,,

acids_cc-3.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.2.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

acids_cc-3.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+cached_conv

cached_conv/__init__.py

@@ -0,0 +1,200 @@
+import torch
+
+USE_CACHED_CONV = False
+MAX_BATCH_SIZE = 64 # Warning! added for retro-compatibility
+
+def use_cached_conv(state: bool):
+    global USE_CACHED_CONV
+    USE_CACHED_CONV = state
+
+
+def is_cached():
+    global USE_CACHED_CONV
+    return USE_CACHED_CONV
+
+
+from .utils import Branches
+from .conv_1d import (get_padding, CachedSequential, Sequential,
+                      CachedPadding, CachedConv1d, CachedConvTranspose1d,
+                      CachedGroupNorm, CachedGroupNorm1d)
+from .conv_1d import Conv1d as _Conv1d
+from .conv_1d import ConvTranspose1d as _ConvTranspose1d
+from .conv_1d import AlignBranches1d as _AlignBranches1d
+from .conv_2d import CachedConv2d, CachedConvTranspose2d, CachedGroupNorm2d
+from .conv_2d import Conv2d as _Conv2d
+from .conv_2d import ConvTranspose2d as _ConvTranspose2d
+from .conv_2d import AlignBranches2d as _AlignBranches2d
+from .conv_3d import CachedConv3d, CachedConvTranspose3d, CachedGroupNorm3d
+from .conv_3d import Conv3d as _Conv3d
+from .conv_3d import ConvTranspose3d as _ConvTranspose3d
+from .conv_3d import AlignBranches3d as _AlignBranches3d
+from .variants import (get_window, CachedWindowedConv1d,
+                       CachedWindowedConv2d, CachedWindowedConv3d,
+                       CachedModulatedConv1d, CachedModulatedConv2d,
+                       CachedModulatedConv3d)
+from . import variants as _variants
+from .upsample import CachedUpsample1d, CachedUpsample2d, CachedUpsample3d
+from . import upsample as _upsample
+
+
+def chunk_process(f, x, N):
+    x = torch.split(x, x.shape[-1] // N, -1)
+    y = torch.cat([f(_x) for _x in x], -1)
+    return y
+
+
+def test_equal(model_constructor, input_tensor, crop=True):
+    initial_state = USE_CACHED_CONV
+
+    use_cached_conv(False)
+    model = model_constructor().to(input_tensor.device)
+    use_cached_conv(True)
+    cmodel = model_constructor().to(input_tensor.device)
+
+    for p1, p2 in zip(model.parameters(), cmodel.parameters()):
+        p2.data.copy_(p1.data)
+
+    cd = cmodel.cumulative_delay
+    if cd > 0:
+        y = model(input_tensor)[..., :-cd]
+    else:
+        y = model(input_tensor)
+    cy = chunk_process(lambda x: cmodel(x), input_tensor, 4)[..., cd:]
+
+    if crop and cd > 0:
+        y = y[..., cd:-cd]
+        cy = cy[..., cd:-cd]
+
+    use_cached_conv(initial_state)
+    return torch.allclose(y, cy, 1e-4, 1e-4)
+
+
+def Conv1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConv1d(*args, **kwargs)
+    else:
+        return _Conv1d(*args, **kwargs)
+
+
+def ConvTranspose1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConvTranspose1d(*args, **kwargs)
+    else:
+        return _ConvTranspose1d(*args, **kwargs)
+
+
+def AlignBranches1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return _AlignBranches1d(*args, **kwargs)
+    else:
+        return Branches(*args, **kwargs)
+
+
+# backward-compatible alias
+def AlignBranches(*args, **kwargs):
+    return AlignBranches1d(*args, **kwargs)
+
+
+def Conv2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConv2d(*args, **kwargs)
+    else:
+        return _Conv2d(*args, **kwargs)
+
+
+def ConvTranspose2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConvTranspose2d(*args, **kwargs)
+    else:
+        return _ConvTranspose2d(*args, **kwargs)
+
+
+def AlignBranches2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return _AlignBranches2d(*args, **kwargs)
+    else:
+        return Branches(*args, **kwargs)
+
+
+def Conv3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConv3d(*args, **kwargs)
+    else:
+        return _Conv3d(*args, **kwargs)
+
+
+def ConvTranspose3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedConvTranspose3d(*args, **kwargs)
+    else:
+        return _ConvTranspose3d(*args, **kwargs)
+
+
+def AlignBranches3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return _AlignBranches3d(*args, **kwargs)
+    else:
+        return Branches(*args, **kwargs)
+
+
+def WindowedConv1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedWindowedConv1d(*args, **kwargs)
+    else:
+        return _variants.WindowedConv1d(*args, **kwargs)
+
+
+def WindowedConv2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedWindowedConv2d(*args, **kwargs)
+    else:
+        return _variants.WindowedConv2d(*args, **kwargs)
+
+
+def WindowedConv3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedWindowedConv3d(*args, **kwargs)
+    else:
+        return _variants.WindowedConv3d(*args, **kwargs)
+
+
+def ModulatedConv1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedModulatedConv1d(*args, **kwargs)
+    else:
+        return _variants.ModulatedConv1d(*args, **kwargs)
+
+
+def ModulatedConv2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedModulatedConv2d(*args, **kwargs)
+    else:
+        return _variants.ModulatedConv2d(*args, **kwargs)
+
+
+def ModulatedConv3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedModulatedConv3d(*args, **kwargs)
+    else:
+        return _variants.ModulatedConv3d(*args, **kwargs)
+
+
+def Upsample1d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedUpsample1d(*args, **kwargs)
+    else:
+        return _upsample.Upsample1d(*args, **kwargs)
+
+
+def Upsample2d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedUpsample2d(*args, **kwargs)
+    else:
+        return _upsample.Upsample2d(*args, **kwargs)
+
+
+def Upsample3d(*args, **kwargs):
+    if USE_CACHED_CONV:
+        return CachedUpsample3d(*args, **kwargs)
+    else:
+        return _upsample.Upsample3d(*args, **kwargs)