solvephase 0.1.0__tar.gz → 0.2.0__tar.gz

{solvephase-0.1.0 → solvephase-0.2.0}/AGENTS.md

@@ -50,12 +50,14 @@ src/solvephase/
   unwrap.py         re-exports aocore.unwrap (unwrap_phase, wrap)
   metrics.py        re-exports aocore.metrics (rms, wavefront_error, strehl_from_rms)
   result.py         Result returned by every focal-plane solver
+  _kernels.py       fused CuPy kernels for the GPU hot paths (same arithmetic as the xp code)
   algorithms/       one module per algorithm family (gerchberg_saxton, cdi, ...)
   api.py            retrieve(): the one-call high-level entry point
 tests/              pytest; mirrors the module names
 benchmarks/         speed suite (run.py), head-to-head baselines (compare.py), method
                     comparison feeding docs/choosing.md (methods.py), JSON artifacts
-validation/         physics/statistics evidence (Cramer-Rao, independent references)
+validation/         physics/statistics evidence (Cramer-Rao, independent references);
+                    nirc2.py validates on real Keck/NIRC2 data (data not in the repo)
 docs/               mkdocs-material site
 examples/           headless, deterministic scripts
 ```
@@ -214,3 +216,30 @@ note the GPU and CuPy version when you report GPU results.
   is not a package).
 - GPU timings on a shared device (or CPU timings under load) vary 2-5x;
   record the load and hardware with any performance claim.
+- Small GPU problems are bound by the host: each CuPy operation costs
+  15-30 us of Python and launch overhead on the Arm bench, whatever the
+  array size. Fuse elementwise chains in `_kernels.py`, keep the exact
+  expression order, and compile with `--fmad=false` so results match the
+  unfused CuPy chain to the bit. Products of two complex arrays are the
+  exception: write them as thrust complex products compiled with default
+  flags (as CuPy's own multiply kernel is). Inlining a conjugate into a
+  product (`a * conj(b)`) or summing with a custom `ReductionKernel` changes
+  the rounding; keep `cupy.sum` for sums. `cupy.vdot` of real vectors
+  rounds exactly like `(a * b).sum()`, which has less overhead.
+- CuPy refuses cuBLAS calls (any `matmul`) during CUDA-graph stream capture,
+  so only matmul-free bodies (the Fast & Furious step) are replayed from
+  graphs; capture under a private memory pool and warm FFT plans first.
+- CPU micro-optimizations of elementwise code on MB-sized temporaries are
+  dominated by allocation and page-fault patterns; a rewrite 5x faster in
+  isolation made `jvp` slower in context. Measure inside the solver.
+- CI has no GPU and gates coverage at 85%: mark GPU-only blocks
+  `# pragma: no cover - GPU only` and test them with `--run-gpu`
+  (`tests/test_kernels.py` compares them with the plain expressions).
+- `AOCORE_FFT_WORKERS` fixes the FFT thread count for every transform,
+  including tiny ones the default heuristic would run on one thread.
+- Real data (validation/nirc2.py): the Keck daytime bench pupil is a full
+  circle; a segmented Keck pupil predicts six-fold spikes the images don't
+  have. Crop each defocused frame around its own centroid (the image walks
+  with focus) and leave tip, tilt and focus out of run-to-run comparisons.
+  Beyond ~36 Zernikes the likelihood on these data has flat directions:
+  cold- and warm-started fits differ by ~300 nm RMS at nearly equal chi2.

{solvephase-0.1.0 → solvephase-0.2.0}/CHANGELOG.md

@@ -5,6 +5,68 @@ All notable changes to `solvephase` are documented here. The project follows
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-10-07
+
+### Performance
+
+Small GPU problems were bound by host-side launch overhead (15-30 us per CuPy
+operation on an Arm host), not by the GPU. Results are unchanged: bitwise
+identical to 0.1.0 on CPU and GPU on the reference machine (Ampere
+Neoverse-N1, 12 pinned cores, NumPy 2.5.3/SciPy 1.18.1; RTX 4060 with CuPy
+14.2.0). Speed-ups below are medians of 0.1.0 and 0.2.0 run alternately,
+three times each (the Benchmarks page, `docs/benchmarks.md`, has the tables).
+
+- **Fast & Furious on the GPU replays each step from a CUDA graph**: one copy
+  in, one graph launch and one copy out per frame, running the same kernels.
+  A step takes 0.11 ms at 64² (was 1.41 ms, 12x) and 0.27 ms at 256² (5.8x),
+  so the GPU now beats the CPU on every size.
+- **Fused GPU kernels** (`solvephase._kernels`) for the focal-plane model,
+  its reverse- and forward-mode derivatives, the Poisson, Gaussian and
+  amplitude losses, the Gerchberg-Saxton projection, the Fast & Furious step
+  and the phase-diversity metric. On the RTX 4060: objective + gradient
+  1.5-1.8x, Levenberg-Marquardt 1.4-1.7x, LIFT 1.8x, phase diversity
+  1.4-1.5x, `retrieve()` 1.2x, Gerchberg-Saxton 1.1-1.2x.
+- **L-BFGS** reuses the scalar products it already has, fuses its vector
+  updates and evaluates real scalar products on the GPU with less overhead:
+  coded-diffraction L-BFGS 1.2-1.3x on the GPU.
+- **`FocalPlaneProblem`** keeps device copies of its flux, background and
+  scaling vectors (no host-to-device copy per evaluation), caches the
+  Gauss-Newton direction maps and assembles gradients and Jacobian blocks
+  without per-channel loops.
+- **CPU**: unit phasors are built from `cos`/`sin` instead of a complex
+  `exp` (same values, about 1.4x faster), Gerchberg-Saxton computes its
+  diversity phasors once instead of every iteration, and the Poisson and
+  amplitude losses skip their Taylor terms when no pixel is below the floor.
+  Gerchberg-Saxton 1.2x, focal-plane objective and LM up to 1.1x on 12
+  Neoverse-N1 cores.
+
+### Fixed
+
+- **The CDI CPU/GPU parity test failed on aarch64.** After 50 iterations it
+  compared the objects to `atol=1e-9`, but RAAR and DM are chaotic and amplify
+  the FFT libraries' last-bit differences (about 1e5x between iterations 10
+  and 40 on an Arm Neoverse-N1 with an RTX 4060). The test now compares the
+  objects after a short run (`atol=1e-10`) and the error histories after the
+  long one (`rtol=1e-6`). The implementations already agreed.
+
+### Added
+
+- **Validation on real Keck/NIRC2 data** (`validation/nirc2.py`). Daytime
+  focus-diversity calibrations of the Keck AO bench, with known coma and
+  trefoil patterns injected on the Xinetics DM, are retrieved with
+  `FocalPlaneProblem`.
+  - The injected patterns are recovered with correlations of 0.945–0.965.
+    One DM gain fits all six runs (3 % scatter), and strength ratios come
+    out within 1 % of the commanded values.
+  - The IDL sharpening correction is predicted with no free parameter
+    (correlation 0.90, amplitude within 1 %).
+  - The fitted DM gain (652 nm OPD/V) and pupil size (9.75 actuator pitches)
+    agree with the Keck AO software's Xinetics parameters (600 nm/V; a
+    10.0-pitch control aperture).
+  - The data are not distributed; the script reads them from `--data` or
+    `SOLVEPHASE_NIRC2_DATA`. The report and figures are in
+    `validation/artifacts`.
+
 ## [0.1.0] - 2026-10-07
 
 First release.

{solvephase-0.1.0 → solvephase-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: solvephase
-Version: 0.1.0
+Version: 0.2.0
 Summary: Fast, GPU-optional phase retrieval for adaptive optics, optical metrology and coherent imaging.
 Project-URL: Homepage, https://github.com/jacotay7/solvephase
 Project-URL: Documentation, https://jacotay7.github.io/solvephase/

solvephase-0.2.0/benchmarks/artifacts/v0.1.0-compare-neoverse-n1-rtx4060.json

@@ -0,0 +1,97 @@
+{
+  "environment": {
+    "solvephase": "0.1.0",
+    "python": "3.13.15",
+    "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39",
+    "processor": "aarch64",
+    "cpu_count": 80,
+    "numpy": "2.5.3",
+    "timestamp": "2026-10-07T06:59:07+00:00",
+    "scipy": "1.18.1",
+    "gpu": "NVIDIA GeForce RTX 4060",
+    "cupy": "14.2.0",
+    "cuda_runtime": 12090,
+    "host": "cfl-test-bench: 80-core Ampere Neoverse-N1 (aarch64), shared",
+    "cpu_pinning": "taskset -c 16-31 (16 cores); OMP/OpenBLAS/FFT threads = 16",
+    "gpus": "NVIDIA GeForce RTX 4060 (8 GB), NVIDIA RTX A400 (4 GB); driver 580.173.02"
+  },
+  "rows": [
+    {
+      "seconds": 0.6354510160163045,
+      "error_nm": 0.12939041062796763,
+      "iterations": 6,
+      "problem": "focal 128x128, 36 modes, 2 images",
+      "method": "solvephase LM (cpu)"
+    },
+    {
+      "seconds": 0.09766029001912102,
+      "error_nm": 0.12939044804752545,
+      "iterations": 9,
+      "problem": "focal 128x128, 36 modes, 2 images",
+      "method": "solvephase LM (gpu)"
+    },
+    {
+      "seconds": 2.9365519550046884,
+      "error_nm": 0.1911658513284471,
+      "evaluations": 8,
+      "problem": "focal 128x128, 36 modes, 2 images",
+      "method": "HCIPy + SciPy least_squares"
+    },
+    {
+      "seconds": 3.0905606810119934,
+      "error_nm": 0.12076580233938293,
+      "iterations": 12,
+      "evaluations": 518,
+      "problem": "focal 128x128, 36 modes, 2 images",
+      "method": "HCIPy + SciPy L-BFGS-B"
+    },
+    {
+      "seconds": 1.0338798309967387,
+      "iterations_per_s": 96.7230397594587,
+      "problem": "Misell/GS 128x128, 2 images (iterations/s)",
+      "method": "NumPy textbook loop"
+    },
+    {
+      "seconds": 0.5860432240006048,
+      "iterations_per_s": 170.63587787493435,
+      "problem": "Misell/GS 128x128, 2 images (iterations/s)",
+      "method": "solvephase (cpu)"
+    },
+    {
+      "seconds": 0.08268631101236679,
+      "iterations_per_s": 1209.3900281153399,
+      "problem": "Misell/GS 128x128, 2 images (iterations/s)",
+      "method": "solvephase (gpu)"
+    },
+    {
+      "seconds": 0.6534776719927322,
+      "iterations_per_s": 153.02741667524361,
+      "problem": "CDI HIO 256x256 (iterations/s)",
+      "method": "NumPy textbook loop"
+    },
+    {
+      "seconds": 0.24491209498955868,
+      "iterations_per_s": 408.3097652007072,
+      "problem": "CDI HIO 256x256 (iterations/s)",
+      "method": "solvephase (cpu)"
+    },
+    {
+      "seconds": 1.5308567469764967,
+      "iterations_per_s": 522.5831885184764,
+      "problem": "CDI HIO 256x256 (iterations/s)",
+      "method": "solvephase (cpu, 16 starts, per start)"
+    },
+    {
+      "seconds": 0.03796361497370526,
+      "iterations_per_s": 2634.1011009953345,
+      "problem": "CDI HIO 256x256 (iterations/s)",
+      "method": "solvephase (gpu)"
+    },
+    {
+      "seconds": 0.029988599999342114,
+      "iterations_per_s": 26676.803852715708,
+      "problem": "CDI HIO 256x256 (iterations/s)",
+      "method": "solvephase (gpu, 16 starts, per start)"
+    }
+  ]
+}

solvephase-0.2.0/benchmarks/artifacts/v0.1.0-methods-neoverse-n1-rtx4060.json

@@ -0,0 +1,288 @@
+{
+  "environment": {
+    "solvephase": "0.1.0",
+    "python": "3.13.15",
+    "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39",
+    "processor": "aarch64",
+    "cpu_count": 80,
+    "numpy": "2.5.3",
+    "timestamp": "2026-10-07T07:08:36+00:00",
+    "scipy": "1.18.1",
+    "gpu": "NVIDIA GeForce RTX 4060",
+    "cupy": "14.2.0",
+    "cuda_runtime": 12090,
+    "host": "cfl-test-bench: 80-core Ampere Neoverse-N1 (aarch64), shared",
+    "cpu_pinning": "taskset -c 16-31 (16 cores); OMP/OpenBLAS/FFT threads = 16",
+    "gpus": "NVIDIA GeForce RTX 4060 (8 GB), NVIDIA RTX A400 (4 GB); driver 580.173.02"
+  },
+  "capture_levels": [
+    0.05,
+    0.1,
+    0.2,
+    0.3,
+    0.4,
+    0.5,
+    0.7
+  ],
+  "methods": [
+    {
+      "key": "misell",
+      "label": "Gerchberg-Saxton / Misell",
+      "family": "focal-plane",
+      "data": "2+ images with known diversity",
+      "requires": "known pupil",
+      "unknowns": "zonal phase, unwrapped and fitted to a basis",
+      "seconds": 0.6054978969914373,
+      "seconds_gpu": 0.32186949701281264,
+      "error": 0.010525870689563108,
+      "error_abs": 1.6839799555150254,
+      "error_unit": "nm",
+      "iterations": 300,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 1.0,
+        "0.2": 1.0,
+        "0.3": 0.16666666666666666,
+        "0.4": 0.3333333333333333,
+        "0.5": 0.16666666666666666,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "lm",
+      "label": "Nonlinear ML, modal (LM)",
+      "family": "focal-plane",
+      "data": "1+ images; 2+ with diversity to fix the sign",
+      "requires": "known pupil",
+      "unknowns": "modal coefficients + flux, background, registration",
+      "seconds": 0.18697032099589705,
+      "seconds_gpu": 0.09825141800683923,
+      "error": 0.005106228168692953,
+      "error_abs": 0.8169192020277456,
+      "error_unit": "nm",
+      "iterations": 6,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 1.0,
+        "0.2": 1.0,
+        "0.3": 0.3333333333333333,
+        "0.4": 0.16666666666666666,
+        "0.5": 0.0,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "lbfgs",
+      "label": "Nonlinear ML, zonal (L-BFGS)",
+      "family": "focal-plane",
+      "data": "2+ images with known diversity",
+      "requires": "known pupil",
+      "unknowns": "phase per pixel (+ amplitude)",
+      "seconds": 0.6653045339917298,
+      "seconds_gpu": 1.8744644250255078,
+      "error": 0.1589518492913621,
+      "error_abs": 25.429889459321217,
+      "error_unit": "nm",
+      "iterations": 300,
+      "capture": {
+        "0.05": 0.0,
+        "0.1": 1.0,
+        "0.2": 0.3333333333333333,
+        "0.3": 0.0,
+        "0.4": 0.0,
+        "0.5": 0.0,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "retrieve",
+      "label": "retrieve() (robust default)",
+      "family": "focal-plane",
+      "data": "2+ images with known diversity",
+      "requires": "known pupil",
+      "unknowns": "modal (+ optional zonal refinement)",
+      "seconds": 0.8604175919899717,
+      "seconds_gpu": 0.454890371998772,
+      "error": 0.005106226280309669,
+      "error_abs": 0.8169188999150089,
+      "error_unit": "nm",
+      "iterations": 3,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 1.0,
+        "0.2": 1.0,
+        "0.3": 1.0,
+        "0.4": 0.8333333333333334,
+        "0.5": 0.5,
+        "0.7": 0.5
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "pd",
+      "label": "Phase diversity, extended object",
+      "family": "focal-plane",
+      "data": "2+ images of the same scene with known diversity",
+      "requires": "known pupil; compact scene",
+      "unknowns": "modal coefficients + the object",
+      "seconds": 0.29611618898343295,
+      "seconds_gpu": 0.6530910670117009,
+      "error": 0.06562706370545356,
+      "error_abs": 10.499336641943728,
+      "error_unit": "nm",
+      "iterations": 119,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 0.8333333333333334,
+        "0.2": 1.0,
+        "0.3": 0.6666666666666666,
+        "0.4": 0.3333333333333333,
+        "0.5": 0.3333333333333333,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "lift",
+      "label": "LIFT",
+      "family": "focal-plane",
+      "data": "1 image with a known astigmatism bias",
+      "requires": "known pupil",
+      "unknowns": "~10 low-order modes",
+      "seconds": 0.0662676340143662,
+      "seconds_gpu": 0.10143513602088206,
+      "error": 0.2686901758718495,
+      "error_abs": 42.986360345529,
+      "error_unit": "nm",
+      "iterations": 12,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 1.0,
+        "0.2": 1.0,
+        "0.3": 0.6666666666666666,
+        "0.4": 0.16666666666666666,
+        "0.5": 0.0,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {
+        "in_model_error_nm": 6.756301510929042,
+        "n_modes": 10
+      }
+    },
+    {
+      "key": "ff",
+      "label": "Fast & Furious",
+      "family": "focal-plane",
+      "data": "1 image per step, in closed loop",
+      "requires": "a DM; small residuals; symmetric pupil",
+      "unknowns": "zonal phase (sequential)",
+      "seconds": 0.022502428153529763,
+      "seconds_gpu": 0.035019537986954674,
+      "error": 0.11888414940395173,
+      "error_abs": 19.01966407617134,
+      "error_unit": "nm",
+      "iterations": 30,
+      "capture": {
+        "0.05": 1.0,
+        "0.1": 1.0,
+        "0.2": 0.0,
+        "0.3": 0.0,
+        "0.4": 0.0,
+        "0.5": 0.0,
+        "0.7": 0.0
+      },
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "cdi",
+      "label": "CDI: HIO\u2192ER + shrinkwrap, 8 starts",
+      "family": "imaging",
+      "data": "1 oversampled diffraction pattern",
+      "requires": "isolated object (oversampling \u2265 2)",
+      "unknowns": "complex object",
+      "seconds": 1.581951920001302,
+      "seconds_gpu": 0.29780680901603773,
+      "error": 0.0009197853412670216,
+      "error_abs": 0.0009197853412670216,
+      "error_unit": "relative",
+      "iterations": 700,
+      "capture": {},
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "generic",
+      "label": "Generic: L-BFGS (coded diffraction, 6 masks)",
+      "family": "generic",
+      "data": "6 coded diffraction patterns",
+      "requires": "known random masks (m/n \u2273 4)",
+      "unknowns": "complex vector",
+      "seconds": 0.22192049998557195,
+      "seconds_gpu": 0.2058028259780258,
+      "error": 1.876543333502858e-08,
+      "error_abs": 1.876543333502858e-08,
+      "error_unit": "relative",
+      "iterations": 40,
+      "capture": {},
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    },
+    {
+      "key": "tie",
+      "label": "TIE (non-uniform intensity, DCT)",
+      "family": "near-field",
+      "data": "3 intensities at \u00b1dz",
+      "requires": "weak defocus; sample inside the field",
+      "unknowns": "phase map",
+      "seconds": 0.004973458999302238,
+      "seconds_gpu": 0.004885978007223457,
+      "error": 0.044832407811043105,
+      "error_abs": 0.044832407811043105,
+      "error_unit": "relative",
+      "iterations": 1,
+      "capture": {},
+      "trace": {
+        "times": [],
+        "errors": []
+      },
+      "extra": {}
+    }
+  ]
+}