commkit 2.0.0__tar.gz → 2.2.0__tar.gz

{commkit-2.0.0 → commkit-2.2.0}/.github/workflows/ci.yml

@@ -110,7 +110,7 @@ jobs:
       - name: Run kernel tests with JIT disabled
         env:
           NUMBA_DISABLE_JIT: "1"
-        run: uv run pytest tests/equalization tests/recovery tests/test_frequency.py --device=cpu --cov=commkit --cov-report=
+        run: uv run pytest tests/equalization tests/recovery tests/test_frequency.py tests/test_fiber.py --device=cpu --cov=commkit --cov-report=
 
       - name: Kernel coverage report
         run: >-

{commkit-2.0.0 → commkit-2.2.0}/.gitignore

@@ -8,6 +8,11 @@ wheels/
 *.egg-info
 .ruff_cache
 .benchmarks
+.coverage
+.coverage.*
+coverage.xml
+coverage.json
+htmlcov/
 
 # Virtual environments
 .venv
@@ -17,6 +22,14 @@ wheels/
 
 .vscode/
 
+# Jupyter (uv run jupyter lab examples)
+.ipynb_checkpoints/
+
+# Nsight profiles (benchmarks/benchutils.nvtx_range)
+*.nsys-rep
+*.qdrep
+*.ncu-rep
+
 tmp/
 
 # DSP captures / large binaries

commkit-2.2.0/AGENTS.md

@@ -0,0 +1,295 @@
+# AGENTS.md
+
+Guide for anyone, human or coding agent, changing the **CommKit** repository.
+
+The rules below describe the code as it is. New code follows them; a change that breaks one changes this file in the same commit.
+
+---
+
+## 1. What CommKit is
+
+A Python library for digital-communications research: simulation, receiver DSP, metrics, plotting and experimental captures, on NumPy (CPU) and CuPy (GPU) data. It is **not differentiable**. A separate JAX library, *commax*, will hold the blocks that must sit inside a gradient. commax depends on commkit, never the reverse, and takes its conventions (`Constellation`, bit labels, pulse taps, frame layouts) from commkit.
+
+Users are researchers and engineers. Design for them:
+
+- **Algorithms are plain functions.** Data is a `Signal` or an array. Things that *describe* a modulation, a pulse or an embedded sub-algorithm are small frozen value objects written inline in the call:
+
+  ```python
+  res = lms(rx, num_taps=21, step_size=1e-3, cpr=BPS(test_phases=64))
+  ```
+
+- **Everything is expressed in Python.** No configuration files, registries, global switches or multi-step setup.
+- **We deliberately do not build** pipeline or receiver objects, DSP methods on `Signal`, string shorthands for constellations, or public "config objects per algorithm".
+- **Scope is the reference model and its inverse**, not a catalogue of research variants: the fiber channel and standard digital backpropagation (`correct_fiber_propagation`, with `steps_per_span` and `nonlinear_scale`), like chromatic dispersion and its compensation. Reduced-complexity nonlinearity compensation (filtered or enhanced split step, perturbation-based) is left to user code built on these, and learned DBP, which needs gradients, belongs in commax.
+
+---
+
+## 2. Commands
+
+Always use `uv`. Python 3.12+.
+
+```bash
+uv sync --all-extras                     # environment incl. GPU packages
+uv run nbstripout --install              # once per clone: strip notebook outputs on commit
+uv run pytest                            # CPU + GPU tests (default --device=all)
+uv run pytest --device=cpu               # what CI runs
+uv run pytest tests/equalization/ -k lms # subset
+uv run pytest --cov=commkit              # coverage
+NUMBA_DISABLE_JIT=1 uv run pytest tests/equalization tests/recovery \
+    tests/test_frequency.py tests/test_fiber.py --device=cpu --cov=commkit   # real kernel coverage
+uv run pytest --device=cpu -p tests.plugins.precision_trace \
+    --ignore=tests/test_examples.py      # precision audit (see Numerics); the suite passes under it
+```
+
+**Commit gate.** CI fails on any of these, so run all of them before every commit, with `ruff format` last:
+
+```bash
+uv run ruff check . --fix
+uv run mypy commkit/
+uv run pytest
+uv run ruff format .
+```
+
+**Benchmarks** are explicit-only; `uv run pytest` never collects them. A commit that changes a public signature also smoke-runs them once, without timing, so their call sites cannot rot unnoticed:
+
+```bash
+uv run pytest benchmarks/ --benchmark-disable --device=all
+```
+
+Timed runs:
+
+```bash
+uv run pytest benchmarks/bench_bps.py --benchmark-only --device=all
+uv run pytest benchmarks/ --benchmark-only --device=all \
+    --benchmark-compare=0003 --benchmark-storage=file://benchmarks/baselines
+uv run python benchmarks/record_baseline.py NAME   # record a new baseline
+```
+
+**Release:**
+
+1. On `main`: the CPU and GPU suites pass with no failures or skips (`uv run pytest -rs`), `uv build` succeeds and `uv run --with twine twine check dist/*` passes.
+2. `uv run bump-my-version bump {patch|minor|major}`. This commits the new version and tags it `vX.Y.Z`.
+3. `git push origin main --tags`.
+4. Publish a GitHub Release for the tag (`gh release create vX.Y.Z --generate-notes`). Publishing the release runs `.github/workflows/publish.yml`, which builds and uploads to PyPI through Trusted Publishing; pushing a tag alone publishes nothing.
+
+---
+
+## 3. Architecture
+
+Dependencies point downward only:
+
+| Layer | Modules | May import |
+| --- | --- | --- |
+| Infrastructure | `backend`, `logger`, `_cuda` | NumPy, CuPy |
+| Array and math helpers | `_array` (shape and validation), `math` (power, normalization, dB, linear trend) | infrastructure |
+| Value objects | `mapping.Constellation`, pulse classes, `recovery` CPR configs | layers above |
+| Core | `core` (`Signal`, frames, generation), `io` | layers above |
+| DSP | `mapping`, `filtering`, `multirate`, `spectral`, `smoothing`, `impairments`, `timing`, `frequency`, `recovery`, `equalization`, `metrics`, `analysis` | layers above, and lower DSP modules |
+| Presentation | `plotting` | everything; **nothing imports plotting** |
+
+- **Responsibilities:**
+  - `filtering` holds signal-chain filters with a real frequency response.
+  - `smoothing` holds display and estimation smoothers only.
+  - `analysis` holds laser and phase characterization.
+  - `metrics` is the reporting layer.
+- **Splitting modules:** promote a flat module to a package only when it exceeds about 1000 lines *and* has two or more separable concerns. The package `__init__` re-exports the public names. Never create a `utils/` package or one-function files.
+- **Placeholder modules are kept on purpose.** `commkit.coding` is a reminder of planned work. Never delete it or treat it as dead code. It is excluded from coverage and from the contract registry, and the README labels it "planned, not implemented".
+- **Shared physics lives in one private module**, used by the channel and by its compensation: `_dispersion` (β₂, β₃, the dispersion response) and `_fiber` (`Fiber` and the split-step solver behind `apply_fiber_propagation` and `correct_fiber_propagation`).
+
+---
+
+## 4. API rules
+
+### Signal
+
+- `Signal` holds: samples (NumPy or CuPy, `(N,)` or `(C, N)`, time on the last axis), the **facts** `sampling_rate`, `symbol_rate` and `center_frequency`, the **description** `constellation` and `pulse`, and the ground truth `reference` (transmitted symbols and bits) plus the `frame` layout.
+- It never holds pipeline results.
+- Construction does no hidden work: no mapping, no normalization, no device move, no shape guessing. Factories (`generate`, frames) do that work.
+- `sig.to("gpu")` and `sig.replace(...)` return new Signals. Arrays are shared, not copied; use `clone()` for an independent copy.
+- **Alignment invariant:** reference symbol *k* corresponds to symbol period *k* of the samples. Any function that drops or shifts symbol periods slices the reference to match. Metrics raise on mismatch; they never guess.
+
+### Function signatures
+
+- `f(data, other_data, *, params)`: everything after the data arguments is keyword-only. A function without data arguments may keep its single primary parameter positional (`barker_sequence(13)`, `gray_code(5)`); with several parameters, all are keyword-only (`rrc_taps(sps=2, rolloff=0.1, span=10)`). The data-argument count is recorded per function in the contract registry.
+- **Signal awareness.** A function whose main argument is waveform data (or a field a Signal really carries) accepts an array or a Signal. It unwraps once with `adapt_signal()` (in `core/_signal_adapter.py`), works on arrays, and returns through `wrap_samples()`. Never re-enter the public function recursively.
+  - Functions on *derived* quantities (phase trajectories, correlations, taps, PSD bins, results) and synthesis primitives stay array-only.
+- **Metadata rule: facts versus choices.**
+  - Facts (`sampling_rate`, `symbol_rate`, and so `sps`) come from the Signal. A conflicting argument raises `ValueError`; an equal one is fine.
+  - Choices (decision `constellation`, `noise_var`, `pulse`) default to the Signal's value, and an explicit argument wins.
+  - A missing fact for array input raises. Neither case warns.
+- Integer-SPS algorithms validate with `require_integer_sps()` before any cast. Never truncate 1.5 SPS to 1.
+- **Value objects are frozen dataclasses** with read-only NumPy arrays:
+  - `Constellation`: `.qam(M)`, `.psk(M)`, `.pam(M, unipolar=)`, or arbitrary `points`; `.shaped(nu=|entropy=)` keeps unit power. Functions accept the object only, never strings.
+  - Pulses: `RRC`, `RC`, `Gaussian`, `Rect`, `SmoothRect`. A raw taps array is accepted wherever a pulse is.
+  - Algorithm objects, one per estimation method (`BPS`, `PLL`, `ViterbiViterbi`, `MthPower`, `CycleSlip`, ...). The same object is used standalone (`correct_carrier_phase(y, BPS())`) and nested (`lms(..., cpr=BPS())`). It lives next to its kernel and validates its parameters on construction. Class names are unique across the package.
+  - Equalizer continuation uses `state=result.state`.
+- **Randomness:** `rng: int | np.random.Generator | None` (SciPy SPEC 7).
+  - What is transmitted (bits, symbols, pilots): draw on the host with the Generator, then transfer. A seed gives the same payload on CPU and GPU.
+  - What the channel adds (AWGN, phase noise, ASE): draw on the data's device through `_random.standard_normal` / `complex_normal`. Realizations differ between devices; statistics agree, and tests assert statistics.
+  - Random parameters of a realization (the PMD waveplates of `Fiber`) are few and drawn on the host, so a seed gives the same fiber on every device.
+  - Never use a global RNG.
+- **Types:** a transform is written `def f(samples: S, ...) -> S` with the TypeVar `S` from `core/_signal_adapter.py` (bound to `np.ndarray | Signal`), so a Signal gives a Signal and an array an array. Do not reassign the `samples` parameter; name the unwrapped array `x`.
+- **Device follows the data.** There are no `backend=` arguments and no global switches. Unsupported array types (JAX, PyTorch) raise `TypeError`.
+  - Only factories with no input data to follow (`generate`, `generate_phase_noise`, `Preamble`/`SingleCarrierFrame.to_signal`, `load_npz`) take `device: str = "cpu"`, validated with `require_device()`. They draw randomness as the rule above says and do the vectorized work (mapping, shaping) on `device`. The default is never "GPU if available".
+- **No `debug_plot`.** Numerical code never imports plotting. Plot functions consume results or recompute through public compute functions.
+- **Design and apply are separate functions**, for example `rrc_taps` and `fir_filter`, or `butterworth_sos` and `iir_filter`.
+
+### Returns
+
+- **Transforms:** array in gives array out on the same device; Signal in gives a new Signal out (update `sampling_rate` when the rate changes).
+- **Estimators** reduce over the time axis by input rank: `(N,)` gives 0-d and `(C, N)` gives `(C,)`. Results stay on the input device.
+- **Analysis summaries** (`estimate_linewidth`, `allan_deviation`, the `frequency_drift` statistics) are reporting-layer fits on plot-sized reductions. Like the metrics, they return host floats or `(C,)` arrays. Sample-rate outputs (trajectories, PSDs, `dsh_phase`) stay on the input device.
+- **Metrics** (`evm`, `snr`, `ber`, `ser`, `gmi`, `mi`) return host values: a `float` for 1-D input and `np.ndarray (C,)` for 2-D input. They **raise** on empty selections; they never return `None` or `0.0` for "nothing measured". Metrics never silently align, rotate or permute against the reference.
+- **More than one value** is returned as a frozen dataclass with named fields. No dicts, and no tuples longer than two.
+- **Equalizers** return `EqualizerResult`. `y_hat` is always an array, `signal` holds the 1-SPS Signal for Signal input, `state` is the continuation state, and diagnostics are returned as data.
+
+### Naming
+
+| Prefix | Meaning |
+| --- | --- |
+| `apply_*` | add an impairment or apply a model |
+| `estimate_<quantity>(x, method)` | measure, never change the data; returns a frozen `<Quantity>Estimate` (value plus diagnostics) |
+| `correct_<quantity>(x, how)` | remove the impairment; returns only the corrected data. `how` is an estimate or an algorithm object (which estimates first) |
+| `resolve_*` | pick among discrete candidates (π/2 rotation, channel permutation); nothing else |
+| `generate` | synthesis from parameters, takes `rng` |
+| `plot_*` | every public plotting function (only `apply_default_theme` is exempt) |
+
+- The algorithm is chosen by an object, never by the function name or a string: `estimate_carrier_phase(y, BPS())`, not `recover_carrier_phase_bps(y)`. The verb function dispatches on the object's type through a private table in its module. There is no `recover_*` or `compensate_*`.
+- Computations use plain nouns (`allan_deviation`).
+- **Glossary**, one name each:
+  - `num_taps`, `step_size`, `sps`, `constellation`, `pulse`, `rng`, `num_symbols`, `num_channels`, `sampling_rate`, `symbol_rate`.
+  - Units are in the docstring. Use `_db` and `_hz` suffixes where the unit is ambiguous.
+
+### Errors and logging
+
+- Wrong input raises, with a message naming the argument.
+- `logger` is for diagnostics only. Per-channel logging that needs a device-to-host transfer is wrapped in `if logger.isEnabledFor(logging.INFO):`.
+- The library never configures handlers, Matplotlib or warning filters at import time.
+
+---
+
+## 5. Numerics
+
+- **Storage dtypes:** `complex64` for IQ samples and `float32` for real signals; `generate`, frames and `load_npz` produce them, so the GPU's single-precision path is the default.
+- **Precision contract.** Every function computes in a predictable precision, and none promotes single-precision data on its own:
+  - **Transforms keep the input's precision**: complex64 in, complex64 out and computed throughout (filters, impairments, the fiber solver and DBP, resampling, delays, carrier-phase correction); complex128 in, complex128 out.
+  - **Fixed-precision blocks** compute in single precision whatever the input: the equalizers in complex64 (with complex128 accumulation inside, below), `compute_llr` / `Constellation.llr` in float32. Double input to them emits `commkit.PrecisionWarning` (`_array.warn_if_double`, attributed to the caller's line; Python shows it once per call site). Users who want strictness turn it into an error with `warnings.simplefilter("error", commkit.PrecisionWarning)`; the test suite does (`pytest_configure` in `tests/conftest.py`; never in pyproject's `filterwarnings`, which imports commkit before coverage starts and drops every module-level line from the report), so a library path that feeds double data to these blocks fails CI. Training and reference symbols are ideal values and are not checked.
+  - **Deliberate float64**, documented where it happens, always on phases, small arrays or accumulators, never on sample-rate data that is then cast back: phases before `unwrap`, phase trajectories and phase ramps (wrapped in float64, rotated in the data's precision), frequency grids and estimates, the recursions' state (PLL, Tikhonov, RLS `P`), the IIR filter's sections (poles near z = 1), and sums: the metrics compute per element in the input's precision and accumulate in float64 (`gmi` / `mi` in single precision moved by under 4e-8 bit, far below the Monte Carlo error).
+  - **Audit with `tests/plugins/precision_trace.py`** after touching numerics: it lists every float64/complex128 array of sample-rate size that a NumPy path of the library creates from single-precision data or from scratch, with its source line. Every reported site must be a deliberate use from the list above (or a test passing complex128 on purpose). Output dtypes alone do not show this: a function that computes in complex128 and casts back returns complex64.
+  - complex128 pays only where single-precision rounding accumulates over many FFT passes, as in the fiber solver (6e-4 after 10^4 steps in complex64, 1e-12 in complex128); there the GPU runs its plain CuPy stages, not the fused complex64 kernels.
+  - **The NEP 50 trap**: under NumPy 2 a NumPy scalar is not weak, so `np.float64(a) * x64` (any `np.sqrt`, `np.pi * ...` result, or a float64 array such as `xp.fft.fftfreq`) promotes complex64 data to complex128. Combine single-precision data only with Python scalars or with arrays cast to its precision; rotate with `math._derotate` (float64 phase wrap, single-precision phasor, one fused kernel on the GPU).
+- **Accumulation precision:**
+  - LMS/CMA/RDE: inputs and weights are `complex64`, but dot products and gradient updates in the hot loop are accumulated in `complex128`.
+  - RLS keeps `P`, the gain `k` and the regressor buffers in `complex128` throughout. Single precision loses the Hermitian positive-definite property and the filter diverges.
+  - Carrier phase: promote angles to `float64` before `xp.unwrap()`, because float32 rounding causes spurious quadrant slips. Tikhonov/Kalman block transitions also run in `float64`.
+- **Normalization invariant:** at `sps` samples per symbol, `E[|x|²] = 1/sps`; at symbol rate, `E[|x|²] = 1`. Any rate-changing block applies the exact gain correction.
+- **Shapes:** SISO is `(N,)` and MIMO is `(C, N)`. Use `as_2d`, `restore_1d`, `broadcast_channels` and `require_channels`; never hand-roll the promote/squeeze idiom.
+- **Reproducibility:** a seed is reproducible within a library version only. Tests and baselines assert statistics, never exact noise realizations.
+
+---
+
+## 6. Performance
+
+| Algorithm structure | Implementation | Device |
+| --- | --- | --- |
+| Vectorized: filters, resampling, spectra, impairments, metrics, LLR, frequency-domain equalizers | NumPy/CuPy via `x, xp, sp = dispatch(samples)` | follows the input |
+| Sequential recursions: LMS/RLS/CMA/RDE, PLL, Tikhonov, cycle slips | Numba `@njit(cache=True, fastmath=True, nogil=True)` | CPU; GPU input does one round trip to the host and back |
+| Hot spots limited by intermediates, launches or round trips | CUDA C++ via CuPy `RawModule` (`commkit/_cuda`) | GPU |
+| Elementwise chains with many temporaries, where a paired A/B shows the gain end to end (the PMD stage of `_fiber`: 0.61x on 4 cores, but not on the reference machine; see the roadmap) | Numba `@njit(parallel=True)` in one pass, real arithmetic (a `1j` literal promotes complex64 to complex128) | CPU |
+
+**Host-sync hygiene:**
+
+- Never extract scalars from a possibly-GPU array inside a loop (`float(x[c])`, `.item()`). Compute the vector on the device, transfer it once with `to_device(v, "cpu")`, then loop on the host.
+- Prefer on-device gathers (`xp.take_along_axis`).
+- Every FFT goes through `commkit._fft` (`fft`, `ifft`, `rfft`, `irfft`), never `np.fft` or `xp.fft` directly: on the CPU it runs `scipy.fft` on every core and keeps complex64 in single precision (NumPy 2 computes it in double, about 3x slower per core); on the GPU it is `cupy.fft`. Frequency grids (`xp.fft.fftfreq`) are exempt.
+- Frequency responses (dispersion, delays, Jones matrices) are computed at float64 phase accuracy and cast to the spectrum's precision before the multiply, so complex64 data is never promoted.
+- Chunk `(N, M)` intermediates over N with an on-device accumulator.
+- Reduce on the device before transferring, including for plotting.
+- Pass whole records to compiled code. There are no Python loops over symbols or small blocks, except inside a CUDA graph.
+- Repeated GPU work with fixed buffers (the block equalizers' blocks, the fiber solver's spans) is replayed from CUDA graphs through `_cuda.graphs.GraphRunner`, the one driver: warm once, capture on the second run, replay, fall back to eager on a failed capture. Its stream is reused per device and thread, because CuPy's pool caches freed blocks per stream and a fresh stream per call grows the pool every call.
+
+**Kernel policy:**
+
+- A `.cu` kernel needs a benchmark showing that CuPy is limited by intermediates, launch count or round trips.
+- It optimizes a CPU reference (Numba or NumPy) and is tested against it with `--device=all`.
+- Wrappers check dtype, contiguity, shape and size before launch.
+- `float32` with explicit literals.
+- `--use_fast_math` only per kernel (through `options`), after an accuracy check; the compiler default does not pass it.
+- Short elementwise chains use `ElementwiseKernel` or `cupy.fuse`.
+- Sequential recursions over a whole record are never ported to the GPU; one record has no parallel work per step. A short per-block scan may be, as one launch, when it replaces a host round trip per block (`cs_block`, `bps_anchor`); keep its parallel part parallel (float64 is 1/64 of float32 throughput on consumer GPUs).
+
+---
+
+## 7. Tests
+
+- **Fixtures:** use `backend_device`, `xp` and `xpt` from `tests/conftest.py`. Assert with `xpt.assert_allclose(...)`, because `numpy.testing` raises on CuPy arrays. Cast expectations with `xp.asarray`, and reductions with `float(...)`.
+- **Layout mirrors the source:** `tests/<subpackage>/test_<module>.py`, and `tests/test_<module>.py` for flat modules. Test basenames must be globally unique (pytest prepend import mode), hence names like `channel/test_channel_linear.py`. Split a large test file by concern.
+- **Shared builders** live in `tests/common/` and use the public API.
+- **Contract registry** (`tests/test_api_contracts.py`): every public function has a row. Generic checks run per row: array in/out, Signal in/out, device preserved, input not mutated, keyword-only parameters, `TypeError` on unsupported arrays, fact-conflict errors, return rules.
+  - Do not duplicate these checks in per-module tests.
+- **Oracles:** sequential kernels are tested against plain-Python reference implementations in `tests/common/reference_impl.py`. CUDA kernels are tested against their CPU reference.
+- **Mathematical meaning:** test documented definitions, such as a pulse width, a dispersion sign, or noise-variance scaling, with independently derived expectations, not only shapes and energies.
+- **API changes never change expected values.** A commit that changes a signature only changes how tests call the code. Numerical changes land in their own commit with independent validation.
+- **Examples** are Jupyter notebooks in `examples/`, committed without outputs: `.gitattributes` routes them through the `nbstripout` filter, which each clone enables once with `uv run nbstripout --install` (git does not ship filter configuration). `tests/test_examples.py` executes each one, so an API change updates the notebooks in the same commit.
+  - They form a course: one topic per notebook, named `NN-lowercase-words.ipynb`, numbered in teaching order (signals and DSP, channel and metrics, receiver algorithms, fiber systems, characterization). Each opens with what it covers, links the notebooks it builds on and ends with a link to the next. `examples/README.md` indexes them; a new notebook takes its place in the order and updates the index and the neighbours' links.
+  - Every claim in the text matches what the cells print: run the notebook and read its numbers before committing it. Keep each under about 20 s on the CPU.
+  - Plots of hand-computed arrays go through `to_device(x, "cpu")`, so the notebooks run unchanged on the GPU.
+- **Coverage** must not drop below the CI floor. Numba kernel coverage is measured by a separate `NUMBA_DISABLE_JIT=1` job.
+
+---
+
+## 8. Benchmarks
+
+`benchmarks/` tracks the GPU-relevant hot paths. Baselines are committed under `benchmarks/baselines/`. The current reference is `0004` (`fiber_dbp`) on the reference machine: RTX 4070 Ti, Ryzen 7 7800X3D, WSL2. It adds the fiber and DBP benchmarks to `0003` (`v2_0`, the 2.0 state); `0002` (`pre_v2`) is the 1.x state. `0002` ran the CPR equalizer and Viterbi-Viterbi benchmarks on workloads that did not converge, so compare those only from `0003` on.
+
+- **Workloads must converge.** A workload the algorithm cannot handle times a failure mode (slip storms, divergence), not the operating point. Every equalizer benchmark asserts its symbol error rate with `benchutils.assert_converged`.
+- **Baselines** are recorded with `benchmarks/record_baseline.py`: each file in its own process, best of three passes. In one full-suite process, small GPU benchmarks after the large equalizer workloads ran 2-12x slower than alone, so never record a baseline from a single `pytest benchmarks/` run.
+
+- **IDs** are `[cpu]` or `[gpu]`, meaning the input device.
+- **Timing:**
+  - Timed bodies end with the `sync` fixture; otherwise GPU timings measure launches, not execution.
+  - Each benchmark has one warmup round, so compilation and pool growth are excluded.
+- **Workloads** come from `benchmarks/workloads.py` with fixed seeds; never generate data inline.
+- **Tools:** `benchmarks/benchutils.py` provides `CudaEventTimer` and `nvtx_range` for `nsys` profiling.
+- **Logging** is set to WARNING in `benchmarks/conftest.py`.
+- **Trust deltas, not single runs.** A ±20-40% swing has been seen under load. Re-run a suspected regression in isolation, or do a controlled A/B with at least 7 repetitions, alternating old and new per repetition and comparing the paired ratios. When even unchanged benchmarks move, time the changed part itself with `CudaEventTimer` and quote it against the step it sits in.
+- A commit touching a hot path quotes its delta against `0004`. Tolerance is 5% on CPU and 10% on GPU.
+
+---
+
+## 9. Roadmap and open items
+
+What is not done yet, in suggested order. Unlike the sections above, this is a to-do list, not a description of the code: remove an item in the commit that resolves it, and add one when a known gap is left open.
+
+### Planned modules
+
+1. **Channel coding** (`coding`, a placeholder package). Resolve the "Before channel coding" items below first, so the decoder is built on soft demapping that is already at its final speed on both devices. Start with one soft decoder (LDPC) that consumes `compute_llr` output (positive LLR = bit 0) and reports post-FEC BER next to `gmi`, then add encoders. The package stays out of `commkit/__init__.py` until a real encode/decode entry point exists. Decoders are iterative message passing over many codewords: batch codewords on the GPU from the start.
+
+### Nonlinear fiber channel: future features
+
+Each is an addition that leaves the current API unchanged.
+
+- **Batched realizations**: an extra leading axis `(B, 2, N)` (launch-power sweeps, Monte Carlo over seeds) so cuFFT batches them and short records fill the GPU.
+- **Adaptive step rules**: a `step_rule=` object (maximum nonlinear phase per step, local-error method) next to the equal `steps_per_span`, which measured 3x more accurate than logarithmic steps on a 32 GBd, 80 km span.
+- **WDM helpers**: combining frequency-shifted channels into one wideband field and selecting one after propagation; the solver already handles the wide field (with `dispersion_slope_ps_nm2_km` set).
+- **Distributed (Raman) amplification and per-span fiber types** in one call (chaining calls already works: every call speaks launch-relative units).
+- **cuFFT callbacks** fusing the linear stage into the transforms (CuPy's support is experimental and compiles with nvcc at run time).
+- **Self-steepening and Raman scattering** for ultra-wideband fields.
+
+### Before channel coding: mapping and soft demapping
+
+The decoder's input path. The GPU items need a CUDA device to write, test (`--device=all`, against the Numba reference) and time.
+
+- **GPU LLRs of separable QAM.** `Constellation.llr` takes the per-axis path (2·sqrt(M) metrics instead of M) only on the CPU; on the GPU, 256-QAM evaluates 256 metrics per symbol instead of 32. Port `llr_separable` to a CUDA kernel.
+- **One exponential per point in the CUDA exact LLR.** `llr.cu` evaluates every metric twice and calls `expf` once per point and bit (M·k per symbol, 2048 for 256-QAM). The Numba kernel scales by one peak per symbol and needs M (with the underflow fallback of `_EXACT_FLOOR`). Together with the item above: 32 exponentials per 256-QAM symbol instead of 2048.
+- **CUDA LLR stores and per-call tables.** Each thread writes its k LLRs at stride k (uncoalesced), and every call rebuilds and copies the points, packed labels and log prior to the device; cache them per constellation and device.
+- **Non-square hard decisions on the GPU.** `gray._nearest_index` (used by `Constellation.demap`, `evm(blind=True)` and `ser` for PSK, cross QAM and arbitrary points) loops in Python over 4096-symbol chunks: about 250 iterations of several launches per 10^6 symbols. On the CPU that chunk and `abs` are the fastest NumPy form (measured); on the GPU, time it against a larger element-budget chunk or the nearest-point mode of `bps_min_d2`.
+- **`Constellation.map` intermediates.** It casts the bits to int32 groups and multiplies and sums them: two `(N, k)` int32 temporaries for `N·k` input bits, four times the bits' own size in int8. Encoders hand it long bit records; pack the labels in one pass.
+- **Closed-form max-log LLRs of Gray QAM** (optional, CPU and GPU): per bit, the nearest point of each half is found by snapping, O(k) instead of O(sqrt(M)). Expected gain under 2x on the CPU; measure before adopting.
+
+### Performance
+
+- **Block equalizers at small `block_size` on the GPU**: a fixed cost per block; at 256 the GPU only ties the CPU without carrier recovery.
+- **The fiber solver's Numba passes lose end to end on the reference machine.** On 4 cores (where they were tuned) the hybrid Kerr stage won from `_fiber._HYBRID_KERR_MIN = 2^18` and the trig-free Jones (PMD) pass won end to end (0.61x). On the reference machine (8 cores, 16 threads), whole propagations with NumPy stages and multi-core `scipy.fft` are faster: hybrid Kerr 1.74x slower at 2^18, 1.32x at 2^19, 1.12x at 2^20; Numba Jones 1.35x at 2^14, 2.19x at 2^16, 1.54x at 2^18, even at 2^20. The cause is contention: Numba's OpenMP workers spin after each pass and take the cores from the FFT workers (`OMP_WAIT_POLICY=passive` cuts the Kerr loss at 2^18 to 1.09x; one Numba thread makes Jones even with NumPy). The stage timed alone hides this. Options: choose the passes by core count, run them single-threaded, or keep NumPy stages only; decide by end-to-end A/Bs on both machines.
+- **A fully fused Numba Kerr stage** may beat the hybrid one where Numba has SVML (vector sin/cos); neither machine measured so far has it (`numba.core.config.USING_SVML` is False), so it is untimed.
+- **`cs_block` on phases with frequent slip decisions**: one speculation pass per changed decision (about 1 pass per block on a realistic link, more on pathological input). Outside a CUDA graph the Python wrapper (validation, scratch allocation) costs about 18 us per launch.

{commkit-2.0.0 → commkit-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: commkit
-Version: 2.0.0
+Version: 2.2.0
 Summary: A Python library for high-performance digital communications research.
 Project-URL: Homepage, https://github.com/lokgar/commkit
 Project-URL: Repository, https://github.com/lokgar/commkit
@@ -43,50 +43,20 @@ Description-Content-Type: text/markdown
 ![License](https://img.shields.io/badge/license-MIT-green)
 ![CUDA](https://img.shields.io/badge/CUDA-13.x-76B900?logo=nvidia)
 
-CommKit covers the receiver chain of coherent and IM/DD links (waveform
-generation, channel impairments, synchronization, carrier recovery, adaptive
-equalization, metrics, laser characterization and plotting) on NumPy or CuPy
-arrays. Computation runs on the device the data lives on.
+CommKit covers the receiver chain of coherent and IM/DD links (waveform generation, channel impairments, synchronization, carrier recovery, adaptive equalization, metrics, laser characterization and plotting) on NumPy or CuPy arrays. Computation runs on the device the data lives on.
 
 ---
 
 ## The model
 
-```text
-plain arrays (NumPy / CuPy)          value objects (frozen, written inline)
-        \                            Constellation, RRC / RC / Gaussian / Rect,
-         \                           BPS / PLL / CycleSlip, MthPower, ...
-          v                                   |
-       Signal  = samples + facts + description + reference
-          |
-          v
-   functions: generate, apply_*, estimate_*, correct_*, resolve_*, ...
-          |                 \
-          v                  v
-   Signal or array      typed results (EqualizerResult, *Estimate, ...)
-                             |
-                             v
-                   plotting (draws results, never computes them)
-```
+Data is a NumPy or CuPy array, or a `Signal` that wraps the samples together with what is known about them. Algorithms are plain functions that take the data and return a new `Signal` or array, or a typed result such as `EqualizerResult` or an `*Estimate`. Plotting functions draw those results; they never compute them.
 
-- **Algorithms are plain functions.** Data is a `Signal` or an array.
-- **Things that describe** a modulation, a pulse or a sub-algorithm are small
-  frozen objects written in the call: `lms(rx, cpr=BPS(test_phases=64))`.
-  The same object works standalone: `correct_carrier_phase(y, BPS())`.
-- **A `Signal` carries facts and ground truth.** Facts are `sampling_rate`,
-  `symbol_rate` and `center_frequency`. The description is `constellation`
-  and `pulse`. The ground truth is `reference` (transmitted symbols and
-  bits). Functions take facts from the Signal and raise on a conflicting
-  argument. Choices such as the decision constellation default to the
-  Signal's and can be overridden.
-- **Results with several values are frozen dataclasses** with named fields.
-  Metrics return host floats: one per channel for `(C, N)` input.
-- **The device follows the data.** Processing functions have no `backend=`
-  or `device=` argument. Only factories, which have no input data to
-  follow, take `device=`: `generate(..., device="gpu")`. Move existing data
-  with `sig.to("gpu")`.
-- There is no pipeline object, receiver class or configuration file: the
-  orchestration stays in your script.
+- **Things that describe** a modulation, a pulse or a sub-algorithm are small frozen objects written in the call: `lms(rx, cpr=BPS(test_phases=64))`. The same object works standalone: `correct_carrier_phase(y, BPS())`.
+- **A `Signal` carries facts and ground truth.** Facts are `sampling_rate`, `symbol_rate` and `center_frequency`. The description is `constellation` and `pulse`. The ground truth is `reference` (transmitted symbols and bits). Functions take facts from the Signal and raise on a conflicting argument. Choices such as the decision constellation default to the Signal's and can be overridden.
+- **Results with several values are frozen dataclasses** with named fields. Metrics return host floats: one per channel for `(C, N)` input.
+- **The device follows the data.** Processing functions have no `backend=` or `device=` argument. Only factories, which have no input data to follow, take `device=`: `generate(..., device="gpu")`. Move existing data with `sig.to("gpu")`.
+- **Plots draw results.** `plot_*` functions take a Signal, an array or a result object; numerical code never imports plotting.
+- There is no pipeline object, receiver class or configuration file: the orchestration stays in your script.
 
 ## Quickstart
 
@@ -109,30 +79,20 @@ res = ck.equalization.lms(
 y = res.signal                    # 1-SPS Signal, reference aligned
 print(f"EVM {ck.metrics.evm(y, num_skip_symbols=2000):.1f} %, "
       f"BER {ck.metrics.ber(y, num_skip_symbols=2000):.1e}")
-# EVM 12.8 %, BER 2.5e-04
+# EVM 12.7 %, BER 2.4e-04 (on the CPU; the GPU draws other noise)
 
 # The next record continues from the converged taps and CPR state.
 res2 = ck.equalization.lms(rx, num_taps=21, step_size=1e-3, cpr=cpr, state=res.state)
 ```
 
-Without CuPy, pass `device="cpu"` (the default): the same code runs on the
-CPU.
+Without CuPy, pass `device="cpu"` (the default): the same code runs on the CPU.
 
 ## CPU and GPU
 
-CommKit never picks a device for you: data stays where you put it, and each
-function runs on the device of its input. A few habits get the speed out of
-a GPU:
-
-- **Put data on the GPU once, at the source.** Build it there with
-  `generate(..., device="gpu")`, `frame.to_signal(..., device="gpu")` or
-  `load_npz(path, device="gpu")`; move a capture with `sig.to("gpu")`. Do not
-  move data back and forth between stages. Building on the GPU is faster
-  than `generate(...).to("gpu")`: about 26 ms instead of 370 ms for 4M
-  symbols, because mapping and pulse shaping run there.
-- **You do not need to move results back.** Metrics (`evm`, `ber`, ...) and
-  analysis summaries return host floats, and plots reduce on the device and
-  transfer only what they draw.
+CommKit never picks a device for you: data stays where you put it, and each function runs on the device of its input. A few habits get the speed out of a GPU:
+
+- **Put data on the GPU once, at the source.** Build it there with `generate(..., device="gpu")`, `frame.to_signal(..., device="gpu")` or `load_npz(path, device="gpu")`; move a capture with `sig.to("gpu")`. Do not move data back and forth between stages. Building on the GPU is faster than `generate(...).to("gpu")`: about 26 ms instead of 370 ms for 4M symbols, because mapping and pulse shaping run there.
+- **You do not need to move results back.** Metrics (`evm`, `ber`, ...) and analysis summaries return host floats, and plots reduce on the device and transfer only what they draw.
 - **Know what runs where.**
 
   | Work | GPU input |
@@ -141,25 +101,12 @@ a GPU:
   | Frequency-domain equalizers `block_lms`, `block_cma`, `block_rde` | runs on the GPU (CUDA graphs) |
   | Sequential equalizers `lms`, `rls`, `cma`, `rde`; `PLL`, `Tikhonov`, cycle-slip correction | runs on the CPU (Numba): one copy to the host and back, no speedup |
 
-  A sample-by-sample recursion has no parallel work per step. For long
-  records on the GPU, prefer the block equalizers with `block_size=1024` or
-  more. Each block costs a fixed overhead: at 256 the GPU only ties the CPU
-  without carrier recovery, at 2048 it is 6-9x faster. A larger block also
-  lowers the stable `step_size`.
-- **Short records do not pay off.** Launch overhead dominates below roughly
-  10⁴-10⁵ samples, where the CPU is as fast.
-- **Batch channels and records.** Pass `(C, N)` arrays rather than looping
-  over channels in Python: one call does the work of C.
-- **Keep scalars on the device inside loops.** `float(x)`, `x.item()` and
-  `if x > 0:` on a CuPy array wait for the GPU and copy. Collect values in an
-  array and transfer once.
-- **The first call compiles.** Numba and CUDA kernels compile on first use
-  and are cached on disk. Warm up once before timing anything.
-- **Seeds and devices.** The same `rng` gives the same bits and symbols on
-  both devices. Channel noise (AWGN, phase noise) is drawn on the device, so
-  its realization differs between CPU and GPU while its statistics match.
-  For the exact same phase-noise trajectory on both, draw it on the CPU with
-  `generate_phase_noise(...)` and move it.
+  A sample-by-sample recursion has no parallel work per step. For long records on the GPU, prefer the block equalizers with `block_size=1024` or more. Each block costs a fixed overhead: at 256 the GPU only ties the CPU without carrier recovery, at 2048 it is 6-9x faster. A larger block also lowers the stable `step_size`.
+- **Short records do not pay off.** Launch overhead dominates below roughly 10⁴-10⁵ samples, where the CPU is as fast.
+- **Batch channels and records.** Pass `(C, N)` arrays rather than looping over channels in Python: one call does the work of C.
+- **Keep scalars on the device inside loops.** `float(x)`, `x.item()` and `if x > 0:` on a CuPy array wait for the GPU and copy. Collect values in an array and transfer once.
+- **The first call compiles.** Numba and CUDA kernels compile on first use and are cached on disk. Warm up once before timing anything.
+- **Seeds and devices.** The same `rng` gives the same bits and symbols on both devices. Channel noise (AWGN, phase noise) is drawn on the device, so its realization differs between CPU and GPU while its statistics match. For the exact same phase-noise trajectory on both, draw it on the CPU with `generate_phase_noise(...)` and move it.
 
 ---
 
@@ -169,10 +116,10 @@ a GPU:
 | --- | --- |
 | [`commkit.core`](https://github.com/lokgar/commkit/tree/main/commkit/core) | `Signal`, `Reference`, `generate`, `SingleCarrierFrame` and `Preamble` (pilots, guard intervals, MIMO streams), `extract_payload`. |
 | [`commkit.mapping`](https://github.com/lokgar/commkit/tree/main/commkit/mapping) | `Constellation` (QAM, PSK, PAM, arbitrary points; Gray labels; probabilistic shaping with `.shaped()`), bit mapping, hard demapping, max-log and exact LLRs. |
-| [`commkit.filtering`](https://github.com/lokgar/commkit/blob/main/commkit/filtering.py) | Pulses (`RRC`, `RC`, `Gaussian`, `Rect`, `SmoothRect`), FIR and IIR design (Butterworth, Chebyshev I/II, elliptic, Bessel), `fir_filter`, `iir_filter`, `matched_filter`, overlap-save, chromatic-dispersion compensation. |
+| [`commkit.filtering`](https://github.com/lokgar/commkit/blob/main/commkit/filtering.py) | Pulses (`RRC`, `RC`, `Gaussian`, `Rect`, `SmoothRect`), FIR and IIR design (Butterworth, Chebyshev I/II, elliptic, Bessel), `fir_filter`, `iir_filter`, `matched_filter`, overlap-save, chromatic-dispersion compensation and digital backpropagation (`correct_fiber_propagation`). |
 | [`commkit.multirate`](https://github.com/lokgar/commkit/blob/main/commkit/multirate.py) | `resample` (fractional), `decimate`, `upsample`, `decimate_to_symbol_rate`. |
 | [`commkit.spectral`](https://github.com/lokgar/commkit/blob/main/commkit/spectral.py) | Welch PSD, spectrograms, frequency shifting, pilot tones. |
-| [`commkit.impairments`](https://github.com/lokgar/commkit/tree/main/commkit/impairments) | AWGN, phase noise, IQ imbalance (with Löwdin and Gram-Schmidt correction), chromatic dispersion, PMD and polarization mixing. Nonlinear channel models: **planned, not implemented**. |
+| [`commkit.impairments`](https://github.com/lokgar/commkit/tree/main/commkit/impairments) | AWGN, phase noise, IQ imbalance (with Löwdin and Gram-Schmidt correction), chromatic dispersion, PMD and polarization mixing, and nonlinear fiber propagation over amplified spans (split-step Kerr effect with dispersion, loss, PMD and ASE) of a `Fiber`. |
 | [`commkit.timing`](https://github.com/lokgar/commkit/blob/main/commkit/timing.py) | Barker and Zadoff-Chu sequences, `estimate_timing` / `correct_timing`, fractional delay estimation and correction. |
 | [`commkit.frequency`](https://github.com/lokgar/commkit/blob/main/commkit/frequency.py) | `estimate_frequency_offset` / `correct_frequency_offset` with `MthPower`, `MengaliMorelli`, `PilotSymbols` and `BiasTone`; static and blockwise. |
 | [`commkit.recovery`](https://github.com/lokgar/commkit/tree/main/commkit/recovery) | `estimate_carrier_phase` / `correct_carrier_phase` with `ViterbiViterbi`, `BPS`, `PLL`, `Tikhonov`, `DataAided`, `PilotAided`, `PilotTone(s)`; cycle-slip correction; `resolve_phase_ambiguity` and `resolve_channel_permutation`. |
@@ -185,8 +132,7 @@ a GPU:
 | [`commkit.plotting`](https://github.com/lokgar/commkit/tree/main/commkit/plotting) | Constellations, eye diagrams, spectra, filter responses, equalizer convergence, and synchronization and laser diagnostics that draw the estimates. Imported on first use. |
 | [`commkit.coding`](https://github.com/lokgar/commkit/tree/main/commkit/coding) | Channel coding and FEC: **planned, not implemented**. |
 
-Importing `commkit` has no side effects: it configures no logging, Matplotlib
-or warning filters, and does not touch the GPU.
+Importing `commkit` has no side effects: it configures no logging, Matplotlib or warning filters, and does not touch the GPU.
 
 ---
 
@@ -201,37 +147,28 @@ pip install "commkit[notebook]"     # to run the example notebooks
 pip install "commkit[full]"         # everything
 ```
 
-With [`uv`](https://github.com/astral-sh/uv), use `uv pip install` in place of
-`pip install`. Extras combine, e.g. `commkit[gpu,notebook]`.
+With [`uv`](https://github.com/astral-sh/uv), use `uv pip install` in place of `pip install`. Extras combine, e.g. `commkit[gpu,notebook]`.
 
 > [!NOTE]
-> **WSL2 and CUDA.** With NVIDIA drivers and CUDA installed on Windows, there
-> is no need to install CUDA inside WSL2. To let the Python CUDA packages find
-> the bundled NVIDIA libraries, add to `~/.bashrc`:
+> **WSL2 and CUDA.** With NVIDIA drivers and CUDA installed on Windows, there is no need to install CUDA inside WSL2. To let the Python CUDA packages find the bundled NVIDIA libraries, add to `~/.bashrc`:
 >
 > ```bash
 > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(echo $HOME/commkit/.venv/lib/python3.*/site-packages/nvidia/cu13/lib)
 > ```
 >
-> *(This assumes the repository is cloned to `$HOME/commkit`; adjust the path
-> otherwise.)*
+> *(This assumes the repository is cloned to `$HOME/commkit`; adjust the path otherwise.)*
 
 ## Examples
 
-Jupyter notebooks in [`examples/`](https://github.com/lokgar/commkit/tree/main/examples) (install the `notebook` extra
-and run `jupyter lab examples`):
+[`examples/`](https://github.com/lokgar/commkit/tree/main/examples) is a course in 26 Jupyter notebooks, one topic each, in teaching order (install the `notebook` extra and run `jupyter lab examples`):
 
-- [`qam_receiver_quickstart`](https://github.com/lokgar/commkit/blob/main/examples/qam_receiver_quickstart.ipynb) - the
-  quickstart above, cell by cell, with the constellation, spectrum and
-  equalizer plots;
-- [`carrier_phase_analysis`](https://github.com/lokgar/commkit/blob/main/examples/carrier_phase_analysis.ipynb) - drift,
-  linewidth and Allan deviation of a recovered carrier phase;
-- [`laser_linewidth_dsh`](https://github.com/lokgar/commkit/blob/main/examples/laser_linewidth_dsh.ipynb) and
-  [`laser_linewidth_homodyne_iq`](https://github.com/lokgar/commkit/blob/main/examples/laser_linewidth_homodyne_iq.ipynb)
-  - laser linewidth from delayed self-heterodyne and homodyne IQ captures;
-- `measurement_laser_linewidth_*` - lean templates for real captures.
+- **00-05, signals and DSP**: the `Signal`, constellations and bit mapping, spectra, filters, pulse shaping and matched filtering, resampling;
+- **06-08, channel and performance**: AWGN and the metrics (SNR, BER, LLRs, GMI), laser phase noise, front-end impairments;
+- **09-16, receiver algorithms**: frame synchronization, frequency-offset estimation, blind and pilot-aided carrier recovery, linear, adaptive and blind equalization, polarization demultiplexing;
+- **17-19, optical fiber systems**: chromatic dispersion, the nonlinear fiber with digital backpropagation, and an end-to-end coherent transceiver;
+- **20-25, characterization**: carrier phase and laser linewidth measurements, with templates for real captures.
 
-The notebooks are committed without outputs and run in CI.
+[`examples/README.md`](https://github.com/lokgar/commkit/blob/main/examples/README.md) lists every notebook with its topics.  The quickstart above appears cell by cell in `00-getting-started` and `14-adaptive-equalization`.  The notebooks are committed without outputs and run in CI.
 
 ## Development
 
@@ -246,9 +183,8 @@ uv run pytest --device=cpu       # what CI runs
 uv run ruff check . && uv run mypy commkit/
 ```
 
-Contributor and coding-agent guidance (architecture, API rules, numerics,
-performance and test conventions) is in [AGENTS.md](https://github.com/lokgar/commkit/blob/main/AGENTS.md).
+Contributor and coding-agent guidance (architecture, API rules, numerics, performance and test conventions) is in [AGENTS.md](https://github.com/lokgar/commkit/blob/main/AGENTS.md).
 
 ## License
 
-[MIT](https://github.com/lokgar/commkit/blob/main/LICENSE).
+[MIT](https://github.com/lokgar/commkit/blob/main/LICENSE)