commkit 3.1.3__tar.gz → 3.2.1__tar.gz

{commkit-3.1.3 → commkit-3.2.1}/AGENTS.md

@@ -33,6 +33,7 @@ uv sync --all-extras                     # environment incl. GPU packages
 uv run nbstripout --install              # once per clone: strip notebook outputs on commit
 uv run pytest                            # CPU + GPU tests (default --device=all)
 uv run pytest --device=cpu               # what CI runs
+uv run pytest --run-slow                 # also the long statistical validation (marker slow)
 uv run pytest tests/equalization/ -k lms # subset
 uv run pytest --cov=commkit              # coverage
 NUMBA_DISABLE_JIT=1 uv run pytest tests/equalization tests/recovery \
@@ -111,7 +112,7 @@ uv run python benchmarks/record_baseline.py NAME   # record a new baseline
 
 **Release:**
 
-1. On `main`: the CPU and GPU suites pass with no failures or skips (`uv run pytest -rs`), `uv build` succeeds and `uv run --with twine twine check dist/*` passes.
+1. On `main`: the CPU and GPU suites pass with no failures or skips, slow validation included (`uv run pytest -rs --run-slow`; strict xfails, if any, are listed in §9), `uv build` succeeds and `uv run --with twine twine check dist/*` passes.
 2. `CHANGELOG.md`: a `## [X.Y.Z] - YYYY-MM-DD` section for the release, written for users. Changes that may need code updates come first, with how to migrate, then Added, Performance (measured, as a ratio of new to old time), Fixed and Development. Add its compare link at the bottom, and commit it.
 3. `uv run bump-my-version bump {patch|minor|major}`. This commits the new version and tags it `vX.Y.Z`.
 4. `git push origin main --tags`. The ruleset on `main` requires pull requests; repository admins bypass it for these two commits.
@@ -175,7 +176,7 @@ Dependencies point downward only:
   - Never use a global RNG.
 - **Types:** a transform is written `def f(samples: S, ...) -> S` with the TypeVar `S` from `core/_signal_adapter.py` (bound to `np.ndarray | Signal`), so a Signal gives a Signal and an array an array. Do not reassign the `samples` parameter; name the unwrapped array `x`.
 - **Device follows the data.** There are no `backend=` arguments and no global switches. Unsupported array types (JAX, PyTorch) raise `TypeError`.
-  - Only factories with no input data to follow (`generate`, `generate_phase_noise`, `Preamble`/`SingleCarrierFrame.to_signal`, `load_npz`) take `device: str = "cpu"`, validated with `require_device()`. They draw randomness as the rule above says and do the vectorized work (mapping, shaping) on `device`. The default is never "GPU if available".
+  - Only factories with no input data to follow (`generate`, `generate_phase_noise`, `Preamble`/`SingleCarrierFrame.to_signal`, `load_npz`, and the Monte Carlo EXIT curves `decoder_exit_curve` and `demapper_exit_curve`) take `device: str = "cpu"`, validated with `require_device()` (in `coding`, which cannot import `core`, by the same check in `coding.exit`). They draw randomness as the rule above says and do the vectorized work (mapping, shaping) on `device`. The default is never "GPU if available".
 - **No `debug_plot`.** Numerical code never imports plotting. Plot functions consume results or recompute through public compute functions.
 - **Design and apply are separate functions**, for example `RRC(...).taps(sps)` and `fir_filter`, or `butterworth_sos` and `iir_filter`.
 
@@ -227,6 +228,7 @@ Docstrings describe the code as it is; a change in behavior updates them in the
 - Wrong input raises, with a message naming the argument.
 - **No fallbacks for a broken environment.** Code never catches a failure of CuPy, a CUDA kernel (compile, load, launch), a CUDA-graph capture or any other dependency in order to run a slower or different implementation; the failure raises to the caller. The kernels and graphs are tested on a GPU before every release (§2), a broken CuPy already fails when data moves to the GPU, and a path kept "just in case" is code that no test or user runs. Dispatch on the *input* is different and allowed: a kernel's documented limits (dtype, number of points, phases or channels) may route to the general implementation where that implementation exists anyway (the CPU's), and the dispatch says so where it happens. A GPU-only path that would serve only inputs beyond a kernel's limit is not written; such input raises (`compute_llr` above 16 bits per symbol).
 - `logger` is for diagnostics only. Per-channel logging that needs a device-to-host transfer is wrapped in `if logger.isEnabledFor(logging.INFO):`.
+- **Levels:** `DEBUG` traces what a call is about to do (method, parameters, sizes). `INFO` is one summary per public call of what a user would print next: the values a metric or estimator found, the parameters an impairment applied, an equalizer's exit MSE, a decoder's failed checks and iterations, what `generate`, a frame or `io` produced. `WARNING` flags a result that is returned but suspect (no convergence, a fit outside its valid range, a near-zero power, a code with 4-cycles). Value objects and pure helpers (taps, tables, code construction) do not log. `warnings.warn` is used only for `PrecisionWarning`, which callers filter.
 - The library never configures handlers, Matplotlib or warning filters at import time.
 
 ---
@@ -263,6 +265,7 @@ Docstrings describe the code as it is; a change in behavior updates them in the
 | Binary linear algebra of codes: encoding, syndromes, the correlations of maximum-likelihood decoding, the syndromes of algebraic decoding, Chase's test patterns through a hard decoder | one float32 GEMM per chunk (`coding._gf2.matmul_mod2`: BLAS, cuBLAS), exact below an inner dimension of 2^24, its parity taken in place on the CPU and by one elementwise kernel writing int8 on the GPU; small tables (generators, coset leaders) cached per code object and device (`backend._device_tables`) | follows the input |
 | Algebraic decoding from the syndromes (BCH, Reed-Solomon): Berlekamp-Massey, a Chien search that stops at the locator's degree, Forney at the roots | CPU: Numba `parallel=True` over codewords. GPU: CUDA kernel (`algebraic_correct`), one thread per codeword, the same steps in integer table arithmetic, so both agree exactly (a CuPy version running Berlekamp-Massey in lockstep over the codewords, about 450 launches per decode, was 12x slower on KP4) | follows the input |
 | Iterative decoding of many codewords: layered LDPC belief propagation (min-sum, sum-product) | CPU: Numba `parallel=True` over codewords, all layers and iterations inside, without fast-math. GPU: CUDA kernel (`ldpc_layered`), one thread block per codeword, the rows of a layer in parallel, early stop per codeword inside the kernel; the same rows in the same order on both, so min-sum agrees bit for bit (a CuPy version, about 12 launches per layer, was 7-23x slower) | follows the input |
+| Analysis by Monte Carlo (EXIT curves) | the work is the existing compiled decoders and vectorized NumPy/CuPy (the demapper's distance table in chunks, histograms on the device); the closed-form parts (the `J` table, LDPC node curves, the chart's trajectory) are host NumPy on grids of a few thousand points, about 0.1 s. No kernel of its own: a whole curve takes 0.05-0.3 s on the GPU | follows `device` |
 | Elementwise chains between multi-core FFTs (the fiber solver's Kerr and PMD stages) | NumPy: a Numba `parallel=True` pass there competes with `scipy.fft`'s threads (the solver's Numba passes measured 1.1-1.8x slower end to end, 2^14-2^20); adopt one only on a paired end-to-end A/B | CPU |
 
 **Numba kernels live at module level**, in a private module that a getter imports on first use (`equalization/_numba_loops.py`, `coding/_numba_loops.py`, `mapping/_numba_llr.py`), so `import commkit` does not load Numba and the disk cache (`cache=True`) keeps them across processes. A kernel defined inside a function that calls another kernel defined there captures that dispatcher in its closure, which Numba's cache key does not reproduce in a new process: it compiled again in every process (3 s for `compute_llr`, 6 s for polar list decoding). A getter may build a kernel in a closure that captures only constants (`_get_numba_sos`).
@@ -296,13 +299,24 @@ Docstrings describe the code as it is; a change in behavior updates them in the
 
 ## 7. Tests
 
-- **Fixtures:** use `backend_device`, `xp` and `xpt` from `tests/conftest.py`. Assert with `xpt.assert_allclose(...)`, because `numpy.testing` raises on CuPy arrays. Cast expectations with `xp.asarray`, and reductions with `float(...)`.
+- **Fixtures:** use `backend_device`, `xp` and `xpt` from `tests/conftest.py`. The GPU legs skip when CuPy is not installed, except under an explicit `--device=gpu`, which fails; an installed CuPy that does not work fails under any setting. That is the only skip: a test that does not belong to the selected run is deselected at collection (`pytest_collection_modifyitems`: `gpu_only` under `--device=cpu`, `cpu_only` under `--device=gpu`, `slow` without `--run-slow`), so a run with CuPy reports no skips, and a new test never calls `pytest.skip` for a configuration. Assert with `xpt.assert_allclose(...)`, because `numpy.testing` raises on CuPy arrays. Cast expectations with `xp.asarray`, and reductions with `float(...)`.
 - **Layout mirrors the source:** `tests/<subpackage>/test_<module>.py`, and `tests/test_<module>.py` for flat modules. Test basenames must be globally unique (pytest prepend import mode), hence names like `channel/test_channel_linear.py`. Split a large test file by concern.
 - **Shared builders** live in `tests/common/` and use the public API.
 - **Contract registry** (`tests/test_api_contracts.py`): every public function has a row. Generic checks run per row: array in/out, Signal in/out, device preserved, input not mutated, keyword-only parameters, `TypeError` on unsupported arrays, fact-conflict errors, return rules.
   - Do not duplicate these checks in per-module tests.
 - **Oracles:** sequential kernels are tested against plain-Python reference implementations in `tests/common/reference_impl.py`. CUDA kernels are tested against their CPU reference. Kernel tests skip only on the CPU parametrization (`skip_unless_gpu`), never because a kernel is unavailable, so a kernel that fails to compile fails its tests.
 - **Mathematical meaning:** test documented definitions, such as a pulse width, a dispersion sign, or noise-variance scaling, with independently derived expectations, not only shapes and energies.
+- **A test of what an algorithm achieves** (an equalizer converges, carrier recovery tracks, a decoder corrects, an estimator is calibrated) must fail on a plausibly wrong implementation:
+  - **Score against the truth**: the transmitted reference, the true phase, delay or offset, never the receiver's own decisions or another estimate from the same family. Constellation symmetry hides a wrong quadrant, a slip or a duplicated stream from a distance to the nearest point. `tests/common/scoring.py` has the helpers (`ser`, `blind_ser`, `phase_rms`, `block_rotations`, `stream_correlation`).
+  - **Resolve only the legitimate ambiguities**, each once over the whole scored interval: one rotation per stream (by the constellation's symmetry, or any angle for a phase-blind equalizer), one bijective channel permutation for blind MIMO, a documented fixed delay. Never per segment.
+  - **A baseline must fail**: the same test runs without the block it credits (step size 0, `cpr=None`, the corrector off, no compensation) and asserts that this fails.
+  - **Tolerances are derived**, from AWGN theory, a Cramér–Rao bound, a Wiener or loop-noise prediction or a binomial interval, and the docstring states the derivation and the measured value. `tests/common/theory.py` holds the closed forms, written with NumPy and SciPy only.
+  - **Real channels**: convergence tests use `make_channel_signal` (RRC pulses, ISI, a fractional delay, polarization rotation, DGD, AWGN). The repeat-upsampled builders have no channel and serve plumbing tests only.
+  - **Check the test by breaking the code**: while writing it, monkeypatch a plausible wrong result (zeros, a factor of two, the feature disabled) and confirm the test fails.
+  - A strengthened test that exposes a library fault is marked `pytest.mark.xfail(strict=True, reason=...)` with the measured numbers and listed in §9; the fix lands in its own commit, which removes the mark.
+- **Monte Carlo tests** state what their sample bounds (0 failures in 12 frames bounds the FER below 0.22 at 95 %) and check the operating point (the pre-FEC BER against theory). The default run keeps a cheap smoke size; a `slow` parametrization (deselected unless `--run-slow`) repeats it with enough trials to mean something.
+- **End-to-end chains** in `tests/test_receiver_chain.py` run several impairments and their receiver at once, scored against the source bits or AWGN theory, each with one correction left out as a failing baseline.
+- **Warnings are failures**: a WARNING from the `commkit` logger during a test fails it, unless the test is marked `allow_warnings` (it provokes a suspect result on purpose) or takes `caplog` to assert the log. There is no list of tolerated messages: a WARNING that reports what the caller asked for, not a suspect result, is a library log-level bug, fixed in the library.
 - **API changes never change expected values.** A commit that changes a signature only changes how tests call the code. Numerical changes land in their own commit with independent validation.
 - **Examples** are Jupyter notebooks in `examples/`, committed without outputs: `.gitattributes` routes them through the `nbstripout` filter, which each clone enables once with `uv run nbstripout --install` (git does not ship filter configuration). `tests/test_examples.py` executes each one, so an API change updates the notebooks in the same commit.
   - They form a course: one topic per notebook, named `NN-lowercase-words.ipynb`, numbered in teaching order (signals and DSP, channel and metrics, receiver algorithms, fiber systems, characterization). Each opens with what it covers, links the notebooks it builds on and ends with a link to the next. `examples/README.md` indexes them; a new notebook takes its place in the order and updates the index and the neighbors' links.
@@ -368,6 +382,10 @@ Steps, once CuPy 14.3 (with cp315 wheels for `cupy-cuda13x`) and a fixed `jupyte
 1. **oFEC** (OpenROADM, ITU-T G.709.3) is not implemented: its block structure is specified in the OpenROADM MSA documents, which were not available in a form to implement from. It would build on `BCHCode` (extended BCH(256, 239)), `Chase` with soft output and the staircase decoder's iteration over a window.
 2. **NR rate matching for polar codes** (sub-block interleaving, the circular buffer, puncturing and shortening with their frozen-set rules, parity-check bits) and **LTE rate matching for turbo codes** are not included: `PolarCode` and `TurboCode` are the mother codes. `Punctured` and `Shortened` cover generic rate matching.
 
+### Test suite: open items
+
+- **Standard tables against the standards.** The NR polar reliability sequence (TS 38.212 Table 5.3.1.2-1), the LTE QPP interleaver parameters (TS 36.212 Table 5.1.3-3) and the DVB-S2 and PON parity-check address tables are tested against commkit's own tables. They should be compared, in full, with fixtures under `tests/data/standards/` transcribed from the specification documents, each with a provenance header (document, version, table). ETSI's download pages refuse scripted access; the documents have to be supplied.
+
 ### Nonlinear fiber channel: future features
 
 Each is an addition that leaves the current API unchanged.

{commkit-3.1.3 → commkit-3.2.1}/CHANGELOG.md

@@ -2,7 +2,45 @@
 
 Notable changes to CommKit, newest first. Performance figures are from the reference machine (RTX 4070 Ti, Ryzen 7 7800X3D, WSL2) unless stated otherwise, as a ratio of new to old time (below 1 is faster).
 
-## [Unreleased]
+## [3.2.1] - 2026-11-02
+
+Fixes found by a rework of the test suite, which now checks what each algorithm achieves against the transmitted truth and theory. Five numerical defaults or models were wrong and change results (PLL cycle-slip repair, Tikhonov smoothing, `block_cma`'s step, `ols_fir_filter`'s alignment of even taps, `MthPower` near 0 Hz); no signature changes. Plots take their sizes from the theme.
+
+### Changes that alter results
+
+- **`Tikhonov` uses the actual variance of a Viterbi-Viterbi block phase** as its observation variance, computed from the constellation and `snr_db`: the noise through the M-th power plus the data self-noise of QAM. It used `1/(M²·SNR·N)`, too small by `M²/2` for PSK and by three to four orders of magnitude for QAM, so the smoother followed the raw block phases: on 16-QAM it now lowers the Viterbi-Viterbi error (static phase, 30 dB, 16-symbol blocks: 0.101 rad before and after smoothing, now 0.048 rad).
+- **`PLL(cycle_slip=CycleSlip())` defaults to a history of `8/μ` symbols** (twice the loop memory, between 10 and 640), standalone and inline in `lms` / `rls`; it used 640 symbols, as the block methods do. A PLL trajectory is smoothed by the loop, and the long linear fit lagged the phase noise enough to declare false slips: 16-QAM, `2π·Δν·T = 2π·1e-4`, 16 dB, `B·T = 2e-2`, four records of 2·10⁴ symbols: 10 rotation changes against the transmitted symbols with the repair, 3 without; now 3 with it. The repair does not undo the loop's own slips, which drift over about `1/(B·T)` symbols; the `CycleSlip` docstring says so. An explicit `history` is unchanged.
+- **`block_cma` defaults to `step_size=min(1e-3, 0.5 / block_size)`**, as `block_rde` does; it used 2e-4. On QPSK, where CMA and RDE are the same algorithm, the old default left the worst channel at 2.3 times the AWGN symbol error rate over the second half of 8·10⁴ symbols (dual polarization through rotation and DGD, Es/N0 10 dB; now 1.3 times) and at 16 times on 2·10⁴ symbols (now 2 times); on 16-QAM at 18 dB the SER of 2·10⁴-symbol records falls from 0.13 to 0.008. Above 2500-symbol blocks the new default is the smaller step, inside the stability ceiling that 2e-4 crossed (16-QAM, 4096-symbol blocks: SER 0.02 before, 0.004 now). Pass `step_size=2e-4` for the previous behavior.
+- **`ols_fir_filter` centres even-length taps on tap `(L - 1) // 2`**, as `fir_filter` (SciPy's `mode="same"`) does and as its docstring promised; it used `L // 2`, one sample later. Odd-length taps and `causal=True` are unchanged. The docstrings of both now state the centre tap; they said `L // 2`.
+
+- **Plots take line widths, marker sizes and fonts from the theme.** The plot functions no longer set their own line widths and marker sizes, so `apply_default_theme` and the user's rcParams control them; the theme's text is larger (16 pt, legends 14 pt, titles 18 pt) with longer ticks. SI-prefix tick labels show the prefix only (`10 k`, not `10 kHz`), as the axis labels already carry the unit.
+
+### Fixed
+
+- **`MthPower` estimates offsets near 0 Hz.** It zeroed bin 0 of the M-th power spectrum and, for constellations without a constant envelope, subtracted the mean of `x^M`, which for QAM is the tone itself: an offset with `|M Δf|` under half a bin was never found (QPSK, 4096 symbols at 1 MS/s: 0 Hz read as -82 kHz, 10 Hz as 30.5 Hz). Both steps are removed; at other offsets the estimates do not change, also with a DC offset of -20 dB in the input.
+- Messages that report what the caller asked for log at INFO, not WARNING, so a WARNING from commkit always means a suspect result: the block equalizers' advice to move CPU data to the GPU, RLS's tail trim, `correct_timing(mode="slice")` shortening the record and the plotting theme's font fallback. The advice to use more taps for fractionally spaced equalization is no longer logged at `sps=1`, where it does not apply.
+- `rls()`: `weights_history` has one row per returned symbol, so it matches `y_hat` after the tail trim (it kept the `num_taps // 2` trimmed rows). A run whose inverse correlation matrix loses positive definiteness (a forgetting factor far below 1) raises the documented `RuntimeError` instead of a bare `ZeroDivisionError` from the kernel, and the message now advises raising the forgetting factor toward 1, not lowering it.
+
+### Development
+
+- The test suite checks what the code achieves, not how it is written. Equalizers converge on RRC records through ISI, fractional delays, polarization rotation and DGD, scored against the transmitted symbols with only the legitimate rotation and permutation removed, and each "improves" test also runs a baseline that must fail; carrier recovery is scored against the true phase; estimators and metrics against Cramér–Rao bounds and AWGN theory; Chase and shortened-code decoding against brute force. New end-to-end receiver chains (coded single carrier, dual polarization, 80 km of fiber) each show that leaving out one correction breaks them. Monte Carlo tests state what their frame counts bound and gain `--run-slow` validation runs. A WARNING log from commkit fails the test. Tests that only repeated the contract registry's shape and device checks are removed.
+- The library faults these tests found are fixed above: the PLL's cycle-slip repair adding slips, Tikhonov's noise model on QAM, the dead zone of `MthPower` around 0 Hz, the alignment of even-length taps (`ols_fir_filter` against `fir_filter`) and RLS's `weights_history` length.
+- `benchutils.assert_converged` checks the symbol error rate of the worst channel (it compared bit errors before), and the 2048-tap block RDE benchmark uses a workload that converges.
+
+## [3.2.0] - 2026-10-30
+
+EXIT analysis and coding diagnostics. EXIT charts for any soft-in soft-out component, measured through the existing decoders or the demapper, or in closed form for LDPC codes with their threshold; plots of LLR distributions, parity-check matrices and EXIT charts; and INFO logging in the modules where it was missing. Nothing changes for existing code.
+
+### Added
+
+- **EXIT analysis** in `commkit.coding.exit`. `ExitCurve` is the transfer curve of one soft-in soft-out component and `exit_chart(first, second)` the `ExitChart` of two, with the decoding trajectory, whether the tunnel is open and where it closes. Curves come from `decoder_exit_curve` (measured through `decode` for any code whose decoder returns a-posteriori LLRs: LDPC, BCJR, Chase), `demapper_exit_curve` (measured, for iterative demapping on AWGN) and `ldpc_exit_curves` (the variable and check nodes of an LDPC code in closed form), with `ldpc_threshold`, the belief-propagation threshold in the Gaussian approximation (0.498 for the DVB-S2 rate-1/2 short code, the NGMI at which simulated QPSK frames start to decode). Measured curves run on the chosen device; the docstrings list the limitations (Gaussian a-priori LLRs, long interleavers, no curve for decoders without soft output such as turbo and polar, early stopping).
+- **Coding plots**: `plot_llr_distribution(llrs, bits)` draws the LLR densities given the transmitted bit and their consistency, `log p(L | 0) / p(L | 1)` against `L`, which shows LLRs from a wrong `noise_var` or demapper model; `plot_parity_check(code)` draws the nonzero entries and node degrees of a parity-check matrix; `plot_exit_chart(chart)` draws an `ExitChart` and marks where a closed tunnel stalls.
+- INFO logging where it was missing: `decode` reports the blocks whose own check failed and the iterations used, `estimate_iq_imbalance` its estimate, and `SingleCarrierFrame.to_signal`, `save_npz` and `load_npz` what they built, saved or loaded. Notebook 00 shows how to turn it on.
+- Notebook 06 checks LLR consistency; notebook 07 predicts the LDPC threshold with an EXIT chart and charts iterative demapping with Gray and natural labels.
+
+### Development
+
+- AGENTS.md states the logging levels: DEBUG for what a call is about to do, INFO for one summary per public call, WARNING for a returned but suspect result; value objects do not log, and `warnings.warn` is kept for `PrecisionWarning`.
 
 ## [3.1.3] - 2026-10-30
 
@@ -303,7 +341,12 @@ See the [GitHub release](https://github.com/lokgar/commkit/releases/tag/v1.1.0)
 
 First release. See the [GitHub release](https://github.com/lokgar/commkit/releases/tag/v1.0.0).
 
-[Unreleased]: https://github.com/lokgar/commkit/compare/v3.1.0...HEAD
+[Unreleased]: https://github.com/lokgar/commkit/compare/v3.2.1...HEAD
+[3.2.1]: https://github.com/lokgar/commkit/compare/v3.2.0...v3.2.1
+[3.2.0]: https://github.com/lokgar/commkit/compare/v3.1.3...v3.2.0
+[3.1.3]: https://github.com/lokgar/commkit/compare/v3.1.2...v3.1.3
+[3.1.2]: https://github.com/lokgar/commkit/compare/v3.1.1...v3.1.2
+[3.1.1]: https://github.com/lokgar/commkit/compare/v3.1.0...v3.1.1
 [3.1.0]: https://github.com/lokgar/commkit/compare/v3.0.0...v3.1.0
 [3.0.0]: https://github.com/lokgar/commkit/compare/v2.3.0...v3.0.0
 [2.3.0]: https://github.com/lokgar/commkit/compare/v2.2.0...v2.3.0

{commkit-3.1.3 → commkit-3.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: commkit
-Version: 3.1.3
+Version: 3.2.1
 Summary: A Python library for high-performance digital communications research.
 Project-URL: Homepage, https://github.com/lokgar/commkit
 Project-URL: Repository, https://github.com/lokgar/commkit
@@ -129,8 +129,8 @@ CommKit never picks a device for you: data stays where you put it, and each func
 | [`commkit.math`](https://github.com/lokgar/commkit/blob/main/commkit/math.py) | `rms`, `normalize`, dB conversions. |
 | [`commkit.smoothing`](https://github.com/lokgar/commkit/blob/main/commkit/smoothing.py) | Display and estimation smoothers (moving average, Savitzky-Golay, 2-D density). Signal-chain filters are in `filtering`. |
 | [`commkit.io`](https://github.com/lokgar/commkit/blob/main/commkit/io.py) | `save_npz` / `load_npz` for Signals. |
-| [`commkit.plotting`](https://github.com/lokgar/commkit/tree/main/commkit/plotting) | Constellations, eye diagrams, spectra, filter responses, equalizer convergence, and synchronization and laser diagnostics that draw the estimates. Imported on first use. |
-| [`commkit.coding`](https://github.com/lokgar/commkit/tree/main/commkit/coding) | Channel coding: `encode` / `decode` dispatched on a code object, `LinearBlockCode` (Hamming, extended and shortened), syndrome and maximum-likelihood decoding; `LDPCCode` (DVB-S2, the 802.3ca / 50G-PON code, any parity-check or quasi-cyclic matrix) with layered min-sum and sum-product decoding on the CPU (Numba) and GPU (CUDA); `ReedSolomonCode` (G.709, KR4, KP4, 10G-EPON, XGS-PON) and `BCHCode` with Berlekamp-Massey decoding; Chase-Pyndiah soft decoding; the G.709.2 `StaircaseCode` and `Concatenated` codes (400ZR C-FEC with its exact Hamming(128,119), 802.3dj); `PolarCode` (5G NR information set) with SC and CRC-aided SCL decoding; `ConvolutionalCode` (K = 7, terminated or tail-biting) with Viterbi and BCJR, `TurboCode` (LTE); `CRC` (NR, LTE, Ethernet), `Interleaver`, `Punctured` and `Shortened`; distribution matching (`CCDM`) and probabilistic amplitude shaping (`PAS`). Coded payloads through `generate(..., code=)`. |
+| [`commkit.plotting`](https://github.com/lokgar/commkit/tree/main/commkit/plotting) | Constellations, eye diagrams, spectra, filter responses, equalizer convergence, synchronization and laser diagnostics that draw the estimates, and coding diagnostics: LLR distributions and their consistency, parity-check matrices, EXIT charts. Imported on first use. |
+| [`commkit.coding`](https://github.com/lokgar/commkit/tree/main/commkit/coding) | Channel coding: `encode` / `decode` dispatched on a code object, `LinearBlockCode` (Hamming, extended and shortened), syndrome and maximum-likelihood decoding; `LDPCCode` (DVB-S2, the 802.3ca / 50G-PON code, any parity-check or quasi-cyclic matrix) with layered min-sum and sum-product decoding on the CPU (Numba) and GPU (CUDA); `ReedSolomonCode` (G.709, KR4, KP4, 10G-EPON, XGS-PON) and `BCHCode` with Berlekamp-Massey decoding; Chase-Pyndiah soft decoding; the G.709.2 `StaircaseCode` and `Concatenated` codes (400ZR C-FEC with its exact Hamming(128,119), 802.3dj); `PolarCode` (5G NR information set) with SC and CRC-aided SCL decoding; `ConvolutionalCode` (K = 7, terminated or tail-biting) with Viterbi and BCJR, `TurboCode` (LTE); `CRC` (NR, LTE, Ethernet), `Interleaver`, `Punctured` and `Shortened`; distribution matching (`CCDM`) and probabilistic amplitude shaping (`PAS`); EXIT charts of measured decoder and demapper curves, and of LDPC codes in closed form with their threshold. Coded payloads through `generate(..., code=)`. |
 
 Importing `commkit` has no side effects: it configures no logging, Matplotlib or warning filters, and does not touch the GPU.
 

{commkit-3.1.3 → commkit-3.2.1}/README.md

@@ -93,8 +93,8 @@ CommKit never picks a device for you: data stays where you put it, and each func
 | [`commkit.math`](https://github.com/lokgar/commkit/blob/main/commkit/math.py) | `rms`, `normalize`, dB conversions. |
 | [`commkit.smoothing`](https://github.com/lokgar/commkit/blob/main/commkit/smoothing.py) | Display and estimation smoothers (moving average, Savitzky-Golay, 2-D density). Signal-chain filters are in `filtering`. |
 | [`commkit.io`](https://github.com/lokgar/commkit/blob/main/commkit/io.py) | `save_npz` / `load_npz` for Signals. |
-| [`commkit.plotting`](https://github.com/lokgar/commkit/tree/main/commkit/plotting) | Constellations, eye diagrams, spectra, filter responses, equalizer convergence, and synchronization and laser diagnostics that draw the estimates. Imported on first use. |
-| [`commkit.coding`](https://github.com/lokgar/commkit/tree/main/commkit/coding) | Channel coding: `encode` / `decode` dispatched on a code object, `LinearBlockCode` (Hamming, extended and shortened), syndrome and maximum-likelihood decoding; `LDPCCode` (DVB-S2, the 802.3ca / 50G-PON code, any parity-check or quasi-cyclic matrix) with layered min-sum and sum-product decoding on the CPU (Numba) and GPU (CUDA); `ReedSolomonCode` (G.709, KR4, KP4, 10G-EPON, XGS-PON) and `BCHCode` with Berlekamp-Massey decoding; Chase-Pyndiah soft decoding; the G.709.2 `StaircaseCode` and `Concatenated` codes (400ZR C-FEC with its exact Hamming(128,119), 802.3dj); `PolarCode` (5G NR information set) with SC and CRC-aided SCL decoding; `ConvolutionalCode` (K = 7, terminated or tail-biting) with Viterbi and BCJR, `TurboCode` (LTE); `CRC` (NR, LTE, Ethernet), `Interleaver`, `Punctured` and `Shortened`; distribution matching (`CCDM`) and probabilistic amplitude shaping (`PAS`). Coded payloads through `generate(..., code=)`. |
+| [`commkit.plotting`](https://github.com/lokgar/commkit/tree/main/commkit/plotting) | Constellations, eye diagrams, spectra, filter responses, equalizer convergence, synchronization and laser diagnostics that draw the estimates, and coding diagnostics: LLR distributions and their consistency, parity-check matrices, EXIT charts. Imported on first use. |
+| [`commkit.coding`](https://github.com/lokgar/commkit/tree/main/commkit/coding) | Channel coding: `encode` / `decode` dispatched on a code object, `LinearBlockCode` (Hamming, extended and shortened), syndrome and maximum-likelihood decoding; `LDPCCode` (DVB-S2, the 802.3ca / 50G-PON code, any parity-check or quasi-cyclic matrix) with layered min-sum and sum-product decoding on the CPU (Numba) and GPU (CUDA); `ReedSolomonCode` (G.709, KR4, KP4, 10G-EPON, XGS-PON) and `BCHCode` with Berlekamp-Massey decoding; Chase-Pyndiah soft decoding; the G.709.2 `StaircaseCode` and `Concatenated` codes (400ZR C-FEC with its exact Hamming(128,119), 802.3dj); `PolarCode` (5G NR information set) with SC and CRC-aided SCL decoding; `ConvolutionalCode` (K = 7, terminated or tail-biting) with Viterbi and BCJR, `TurboCode` (LTE); `CRC` (NR, LTE, Ethernet), `Interleaver`, `Punctured` and `Shortened`; distribution matching (`CCDM`) and probabilistic amplitude shaping (`PAS`); EXIT charts of measured decoder and demapper curves, and of LDPC codes in closed form with their threshold. Coded payloads through `generate(..., code=)`. |
 
 Importing `commkit` has no side effects: it configures no logging, Matplotlib or warning filters, and does not touch the GPU.
 

{commkit-3.1.3 → commkit-3.2.1}/benchmarks/bench_block_blind.py

@@ -31,16 +31,18 @@ ROUNDS = dict(rounds=3, warmup_rounds=1, iterations=1)
 N_SYM = 100_000
 
 # Converging step sizes per (equalizer, block_size).  RDE on 16-QAM from a
-# cold start (no CMA pre-convergence) converges slowly and is still settling
-# after the skipped symbols at 2048, hence its looser SER bound; its cost does
-# not depend on the data.
+# cold start (no CMA pre-convergence) converges slowly at 2048: at 4e-4 it is
+# error-free from about symbol 70 000, so its SER is scored after that (7e-4
+# never settled, still 2-5 % SER at the end); its cost does not depend on
+# the data.
 STEP_SIZE = {
     ("cma", 256): 2e-4,
     ("cma", 2048): 3e-4,
     ("rde", 256): 3e-3,
-    ("rde", 2048): 7e-4,
+    ("rde", 2048): 4e-4,
 }
-MAX_SER = {"cma": 1e-2, "rde": 5e-2}
+SKIP = {("rde", 2048): 70_000}
+MAX_SER = {"cma": 1e-2, "rde": 1e-2}
 
 EQUALIZERS = [
     ("cma", block_cma),
@@ -66,7 +68,12 @@ def _bench_block_blind(benchmark, xp, sync, label, eq_fn, block_size):
 
     r = benchmark.pedantic(run, **ROUNDS)
     assert_converged(
-        r, syms, Constellation.qam(16), skip=20_000, max_ser=MAX_SER[label], blind=True
+        r,
+        syms,
+        Constellation.qam(16),
+        skip=SKIP.get((label, block_size), 20_000),
+        max_ser=MAX_SER[label],
+        blind=True,
     )
 
 

{commkit-3.1.3 → commkit-3.2.1}/benchmarks/bench_fiber.py

@@ -26,16 +26,18 @@ LINK = dict(
 )
 
 
+# The plain/fused split exists on the GPU only; its CPU items are deselected.
+PATHS = ["auto", pytest.param("plain", marks=pytest.mark.gpu_only)]
+
+
 def _stages(backend_device, monkeypatch, path):
     if path == "plain":
-        if backend_device != "gpu":
-            pytest.skip("the plain/fused split exists on the GPU only")
         monkeypatch.setattr(
             _fiber, "_make_ops", lambda x, xp, *args: _fiber._Ops(xp, x.dtype, *args)
         )
 
 
-@pytest.mark.parametrize("path", ["auto", "plain"])
+@pytest.mark.parametrize("path", PATHS)
 @pytest.mark.parametrize("n", [2**16, 2**20], ids=["N64k", "N1M"])
 def bench_fiber_propagation(benchmark, backend_device, xp, sync, monkeypatch, n, path):
     _stages(backend_device, monkeypatch, path)
@@ -49,7 +51,7 @@ def bench_fiber_propagation(benchmark, backend_device, xp, sync, monkeypatch, n,
     benchmark.pedantic(run, **ROUNDS)
 
 
-@pytest.mark.parametrize("path", ["auto", "plain"])
+@pytest.mark.parametrize("path", PATHS)
 @pytest.mark.parametrize("steps", [1, 10])
 def bench_dbp(benchmark, backend_device, xp, sync, monkeypatch, steps, path):
     _stages(backend_device, monkeypatch, path)

{commkit-3.1.3 → commkit-3.2.1}/benchmarks/benchutils.py

@@ -64,14 +64,22 @@ def assert_converged(result, syms, constellation, *, skip, max_ser, blind=False)
     storms, divergence) instead of the operating point, so every equalizer
     benchmark checks its symbol error rate after ``skip`` symbols.  Blind
     equalizers leave a phase ambiguity, which is resolved against the
-    reference first.
+    reference first, after the channel permutation of a butterfly.
     """
     from commkit.backend import to_device
-    from commkit.recovery import resolve_phase_ambiguity
+    from commkit.recovery import resolve_channel_permutation, resolve_phase_ambiguity
 
     y = to_device(result.y_hat, "cpu")[..., skip:]
     ref = to_device(syms, "cpu")[..., skip : skip + y.shape[-1]]
     if blind:
+        if y.ndim == 2:
+            y = resolve_channel_permutation(y, ref)
         y = resolve_phase_ambiguity(y, ref, constellation=constellation)
-    ser = float((constellation.demap(y) != constellation.demap(ref)).mean())
+    # Symbol decisions, not bits (one wrong symbol is one error), and the
+    # worst channel, so one failed stream cannot hide behind a good one.
+    wrong = constellation._nearest(y.reshape(-1)) != constellation._nearest(
+        ref.reshape(-1)
+    )
+    per_channel = to_device(wrong, "cpu").reshape(y.shape).mean(axis=-1)
+    ser = float(per_channel.max())
     assert ser <= max_ser, f"benchmark workload did not converge: SER {ser:.3g}"

{commkit-3.1.3 → commkit-3.2.1}/benchmarks/conftest.py

@@ -56,6 +56,20 @@ def pytest_generate_tests(metafunc):
         metafunc.parametrize("backend_device", params, indirect=True)
 
 
+def pytest_collection_modifyitems(config, items):
+    """Deselect, not skip, the CPU items of ``gpu_only`` parameters."""
+    cpu_only_gpu = [
+        item
+        for item in items
+        if item.get_closest_marker("gpu_only")
+        and getattr(item, "callspec", None) is not None
+        and item.callspec.params.get("backend_device") == "cpu"
+    ]
+    if cpu_only_gpu:
+        items[:] = [item for item in items if item not in cpu_only_gpu]
+        config.hook.pytest_deselected(items=cpu_only_gpu)
+
+
 @numba.njit(cache=True, parallel=True)
 def _spin(x):
     total = 0.0

{commkit-3.1.3 → commkit-3.2.1}/commkit/__init__.py

@@ -31,7 +31,7 @@ Matplotlib, change warning filters, or touch the GPU.
 import importlib
 from typing import Any
 
-__version__ = "3.1.3"
+__version__ = "3.2.1"
 
 from .backend import PrecisionWarning
 from .core import (

{commkit-3.1.3 → commkit-3.2.1}/commkit/coding/__init__.py

@@ -23,13 +23,26 @@ Codes: :class:`LinearBlockCode` (Hamming), :class:`LDPCCode` (DVB-S2, PON),
 :class:`Stages`, :class:`SC`, :class:`SCL`, :class:`Viterbi`, :class:`BCJR`,
 :class:`TurboIterations`.  Bit
 permutations: :func:`interleave` and :func:`deinterleave` with an
-:class:`Interleaver`.
+:class:`Interleaver`.  EXIT analysis: :func:`exit_chart` of two
+:class:`ExitCurve` objects, from :func:`decoder_exit_curve` (any soft-output
+decoder), :func:`demapper_exit_curve` (iterative demapping) or
+:func:`ldpc_exit_curves` (LDPC nodes, closed form, with
+:func:`ldpc_threshold`).
 """
 
 from .algebraic import BCHCode, BerlekampMassey, ReedSolomonCode
 from .base import DecodeResult, decode, encode
 from .convolutional import BCJR, ConvolutionalCode, Viterbi
 from .crc import CRC
+from .exit import (
+    ExitChart,
+    ExitCurve,
+    decoder_exit_curve,
+    demapper_exit_curve,
+    exit_chart,
+    ldpc_exit_curves,
+    ldpc_threshold,
+)
 from .interleaving import Interleaver, deinterleave, interleave
 from .ldpc import LDPCCode, MinSum, SumProduct
 from .linear import Chase, LinearBlockCode, MaximumLikelihood, Syndrome
@@ -49,6 +62,8 @@ __all__ = [
     "Concatenated",
     "ConvolutionalCode",
     "DecodeResult",
+    "ExitChart",
+    "ExitCurve",
     "HardIterations",
     "Interleaver",
     "LDPCCode",
@@ -72,5 +87,10 @@ __all__ = [
     "decode",
     "deinterleave",
     "encode",
+    "decoder_exit_curve",
+    "demapper_exit_curve",
+    "exit_chart",
     "interleave",
+    "ldpc_exit_curves",
+    "ldpc_threshold",
 ]

{commkit-3.1.3 → commkit-3.2.1}/commkit/coding/base.py

@@ -17,6 +17,7 @@ along the last axis; a block never spans channels.
 from __future__ import annotations
 
 import dataclasses
+import logging
 from collections.abc import Callable
 from dataclasses import dataclass
 from typing import Any, NamedTuple
@@ -24,7 +25,7 @@ from typing import Any, NamedTuple
 import numpy as np
 
 from .._array import as_2d, restore_1d, validate_array, warn_if_double
-from ..backend import ArrayType, get_array_module
+from ..backend import ArrayType, get_array_module, to_device
 from ..logger import logger
 
 __all__ = ["DecodeResult", "decode", "encode"]
@@ -272,6 +273,26 @@ def encode(bits: ArrayType, *, code: Any) -> ArrayType:
     return restore_1d(was_1d, out.reshape(x2.shape[0], num_blocks * code.n))
 
 
+def _log_decode(res: _Decoded, code: Any, method: Any) -> None:
+    """Log the decoder's own success check and iterations (INFO level)."""
+    # Two (R,) transfers, made only when INFO is enabled.
+    if not logger.isEnabledFor(logging.INFO) or res.valid.size == 0:
+        return
+    valid = to_device(res.valid, "cpu")
+    iterations = to_device(res.iterations, "cpu")
+    failed = int(valid.size - np.count_nonzero(valid))
+    most = int(iterations.max())
+    used = f", iterations mean {float(iterations.mean()):.1f}, max {most}"
+    logger.info(
+        "Decoded %d block(s) of %r with %r: %d failed the decoder's check%s.",
+        valid.size,
+        code,
+        method,
+        failed,
+        used if most > 1 else "",
+    )
+
+
 def decode(llrs: ArrayType, *, code: Any, method: Any = None) -> DecodeResult:
     """Decode LLRs (or hard decisions) of a code's transmitted bits.
 
@@ -325,6 +346,7 @@ def decode(llrs: ArrayType, *, code: Any, method: Any = None) -> DecodeResult:
             return None
         return restore_1d(was_1d, a.reshape(x2.shape[0], num_blocks * size))
 
+    _log_decode(res, code, method)
     return DecodeResult(
         bits=shaped(res.bits, code.k),
         llrs=shaped(res.llrs, code.n),