dct-toolkit 0.5.0__tar.gz → 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/.gitignore +5 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/AGENTS.md +60 -25
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/CHANGELOG.md +108 -1
- dct_toolkit-1.0.0/PKG-INFO +316 -0
- dct_toolkit-1.0.0/README.md +270 -0
- dct_toolkit-1.0.0/dct_toolkit/__init__.py +125 -0
- dct_toolkit-1.0.0/dct_toolkit/_geometry.py +560 -0
- dct_toolkit-1.0.0/dct_toolkit/_kernels.py +31 -0
- dct_toolkit-1.0.0/dct_toolkit/cartesian.py +87 -0
- dct_toolkit-1.0.0/dct_toolkit/polar.py +428 -0
- dct_toolkit-1.0.0/dct_toolkit/stats.py +891 -0
- dct_toolkit-1.0.0/docs/API_REFERENCE.md +472 -0
- dct_toolkit-1.0.0/docs/MATHEMATICAL_BASIS.md +359 -0
- dct_toolkit-1.0.0/docs/RELEASING.md +127 -0
- dct_toolkit-1.0.0/examples/nd_polar.py +96 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/pyproject.toml +3 -3
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/tests/test_api.py +28 -0
- dct_toolkit-1.0.0/tests/test_cartesian.py +195 -0
- dct_toolkit-1.0.0/tests/test_geometry_precision.py +282 -0
- dct_toolkit-1.0.0/tests/test_polar_kernels.py +244 -0
- dct_toolkit-1.0.0/tests/test_polar_nd.py +610 -0
- dct_toolkit-1.0.0/tests/test_polar_nd_fallback.py +341 -0
- dct_toolkit-1.0.0/tests/test_polar_nd_stats.py +552 -0
- dct_toolkit-1.0.0/tests/test_polar_robustness.py +157 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/tests/test_stats.py +214 -0
- dct_toolkit-1.0.0/tests/test_stats_nd.py +116 -0
- dct_toolkit-0.5.0/PKG-INFO +0 -161
- dct_toolkit-0.5.0/README.md +0 -115
- dct_toolkit-0.5.0/dct_toolkit/__init__.py +0 -100
- dct_toolkit-0.5.0/dct_toolkit/cartesian.py +0 -56
- dct_toolkit-0.5.0/dct_toolkit/polar.py +0 -275
- dct_toolkit-0.5.0/dct_toolkit/stats.py +0 -559
- dct_toolkit-0.5.0/docs/API_REFERENCE.md +0 -203
- dct_toolkit-0.5.0/docs/MATHEMATICAL_BASIS.md +0 -77
- dct_toolkit-0.5.0/tests/test_cartesian.py +0 -104
- dct_toolkit-0.5.0/tests/test_stats_nd.py +0 -37
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/LICENSE +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/dct_toolkit/_widths.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/dct_toolkit/core.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/dct_toolkit/gap_filling.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/examples/basic_polar.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/examples/comprehensive_demo.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/tests/test_core.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/tests/test_gap_filling.py +0 -0
- {dct_toolkit-0.5.0 → dct_toolkit-1.0.0}/tests/test_polar.py +0 -0
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
# AGENTS.md - DCT Toolkit Agent Guidelines
|
|
2
2
|
|
|
3
3
|
**Project**: dct-toolkit
|
|
4
|
-
**Version**: 0.
|
|
5
|
-
|
|
6
|
-
**
|
|
4
|
+
**Version**: 1.0.0
|
|
5
|
+
|
|
6
|
+
**Status**: Geometry-aware N-D polar release track
|
|
7
|
+
|
|
8
|
+
**Last Updated**: 2026-10-26
|
|
7
9
|
|
|
8
10
|
---
|
|
9
11
|
|
|
@@ -45,7 +47,16 @@ conda activate dct-publish
|
|
|
45
47
|
- xarray, matplotlib, cmweather (radar colormaps)
|
|
46
48
|
- open-radar-data (sample radar datasets)
|
|
47
49
|
- build, twine (for PyPI publishing)
|
|
48
|
-
- dct-toolkit
|
|
50
|
+
- dct-toolkit editable install (verify the runtime and installed-distribution versions)
|
|
51
|
+
|
|
52
|
+
After changing the package version, refresh editable installation metadata:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
conda run -n dct-publish python -m pip install --no-deps -e .
|
|
56
|
+
conda run -n dct-publish python -c 'from importlib.metadata import version; import dct_toolkit as dct; assert dct.__version__ == version("dct-toolkit")'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
An editable source import alone does not update installed distribution metadata.
|
|
49
60
|
|
|
50
61
|
### PYTHONPATH
|
|
51
62
|
When running tests or examples from project root:
|
|
@@ -158,8 +169,9 @@ dct_toolkit/ # PROJECT ROOT
|
|
|
158
169
|
├── dct_toolkit/ # INSTALLABLE PACKAGE ⚠️ CODE GOES HERE
|
|
159
170
|
│ ├── __init__.py # Public API exports
|
|
160
171
|
│ ├── core.py # Transfer functions & 1D primitives
|
|
161
|
-
│ ├── cartesian.py #
|
|
162
|
-
│ ├── polar.py # 2D
|
|
172
|
+
│ ├── cartesian.py # N-D Cartesian separable smoothing
|
|
173
|
+
│ ├── polar.py # Legacy 2D and explicit geometry-aware N-D smoothing
|
|
174
|
+
│ ├── _geometry.py # Shared axis/coordinate/width validation and support
|
|
163
175
|
│ ├── stats.py # Statistical ops (Normalized Convolution)
|
|
164
176
|
│ └── gap_filling.py # Gap-filling methods (deferred for public stats-first release)
|
|
165
177
|
│
|
|
@@ -189,7 +201,7 @@ dct_toolkit/ # PROJECT ROOT
|
|
|
189
201
|
├── docs/ # DOCUMENTATION
|
|
190
202
|
│ ├── API_REFERENCE.md
|
|
191
203
|
│ ├── MATHEMATICAL_BASIS.md
|
|
192
|
-
│ └──
|
|
204
|
+
│ └── RELEASING.md # Local build, artifact checks, and publication handoff
|
|
193
205
|
│
|
|
194
206
|
├── README.md # Main documentation
|
|
195
207
|
├── CHANGELOG.md # Version history
|
|
@@ -215,7 +227,8 @@ dct_toolkit/ # PROJECT ROOT
|
|
|
215
227
|
|
|
216
228
|
3. **Review Documentation**
|
|
217
229
|
- Check `docs/MATHEMATICAL_BASIS.md` for theory
|
|
218
|
-
-
|
|
230
|
+
- Gap filling outside the exported statistics API remains research-internal;
|
|
231
|
+
inspect `dct_toolkit/gap_filling.py` only when that scope is requested
|
|
219
232
|
- Check `docs/API_REFERENCE.md` for existing API
|
|
220
233
|
|
|
221
234
|
### Making Changes
|
|
@@ -245,6 +258,10 @@ python -m pytest /path/to/dct_toolkit/tests/ -v
|
|
|
245
258
|
|
|
246
259
|
**Expected Output**: all tests passing
|
|
247
260
|
|
|
261
|
+
CI additionally runs the installed package outside the checkout, checks runtime
|
|
262
|
+
and distribution versions agree, and exercises the N-D example. Its minimum
|
|
263
|
+
dependency job uses Python 3.8, NumPy 1.20.0, and SciPy 1.8.0.
|
|
264
|
+
|
|
248
265
|
**If Tests Fail**:
|
|
249
266
|
- Fix the code, not the tests
|
|
250
267
|
- Tests represent ground truth requirements
|
|
@@ -302,8 +319,19 @@ Before declaring task complete:
|
|
|
302
319
|
- [x] Tag `v0.4.0` release
|
|
303
320
|
- [x] Create `research/gap-filling` branch for long-term research
|
|
304
321
|
|
|
305
|
-
###
|
|
306
|
-
|
|
322
|
+
### v1.0.0 Development Scope
|
|
323
|
+
|
|
324
|
+
- Geometry-aware N-D polar arrays with selectable azimuth, elevation, range,
|
|
325
|
+
time, linear, and batch roles in arbitrary array order.
|
|
326
|
+
- Per-axis kernels, boundaries, grid/physical width units, and filtering selection.
|
|
327
|
+
- Complete propagation through normalized moments, support, prefill, and fallback.
|
|
328
|
+
- Unfiltered axes remain isolated domains, including all-missing batches.
|
|
329
|
+
- Keep existing 2D calls compatible; explicit physical-coordinate mode is opt-in.
|
|
330
|
+
- New public APIs need independent numerical tests and an independent code/docs review.
|
|
331
|
+
- Do not modify manuscripts or review trackers as part of this software task.
|
|
332
|
+
|
|
333
|
+
### Future Beyond v1.0.0
|
|
334
|
+
|
|
307
335
|
- [ ] Additional kernels (Savitzky-Golay, Hanning)
|
|
308
336
|
- [ ] Performance optimizations (numba?)
|
|
309
337
|
- [ ] Jupyter notebook examples
|
|
@@ -322,12 +350,22 @@ Before declaring task complete:
|
|
|
322
350
|
### Normalized Convolution
|
|
323
351
|
Formula for robust mean with gaps:
|
|
324
352
|
```
|
|
325
|
-
μ = smooth(
|
|
353
|
+
μ = smooth(zero_filled_valid_data) / smooth(valid_mask)
|
|
326
354
|
```
|
|
355
|
+
Apply the complete smoothing operator to each input before normalizing once.
|
|
356
|
+
The denominator must be strictly greater than the floor (default `1e-3`);
|
|
357
|
+
unsupported locations use the documented selective prefill fallback. This is
|
|
358
|
+
not additive-epsilon regularization. `dct_count` reports nominal weighted
|
|
359
|
+
support, not statistical effective sample size.
|
|
327
360
|
|
|
328
361
|
### Polar Coordinates
|
|
329
|
-
- **Data Layout**: (n_azimuth, n_range)
|
|
330
|
-
- **
|
|
362
|
+
- **Legacy Data Layout**: (n_azimuth, n_range), retaining index-based width units.
|
|
363
|
+
- **Explicit N-D Mode**: axis roles, coordinates, and units define the geometry.
|
|
364
|
+
- **Physical Angular Widths**: azimuth scales with `1 / (r * cos(elevation) * dθ)`;
|
|
365
|
+
elevation scales with `1 / (r * dφ)`. Use actual coordinates, not gate indices.
|
|
366
|
+
- **Filter Order**: azimuth, elevation, range, then independent time/linear axes;
|
|
367
|
+
this is an ordered nonstationary operator, not one global separable transform.
|
|
368
|
+
- **Fallback Domains**: never borrow support across unfiltered/batch axes.
|
|
331
369
|
- **Boundary Conditions**:
|
|
332
370
|
- Reflective: DCT-II (even symmetry)
|
|
333
371
|
- Periodic: Real FFT (circular convolution)
|
|
@@ -412,20 +450,17 @@ git merge feature/hanning-window
|
|
|
412
450
|
```
|
|
413
451
|
|
|
414
452
|
### Repository Status
|
|
415
|
-
- GitHub:
|
|
416
|
-
- PyPI: **Public**
|
|
417
|
-
- conda-forge: **
|
|
418
|
-
|
|
419
|
-
**Check conda-forge PR Status**:
|
|
420
|
-
```bash
|
|
421
|
-
gh pr view 32930 --repo conda-forge/staged-recipes
|
|
422
|
-
# Or visit: https://github.com/conda-forge/staged-recipes/pull/32930
|
|
423
|
-
```
|
|
453
|
+
- GitHub: **Public** — `JValdivia23/dct-toolkit`
|
|
454
|
+
- PyPI: **Public** distribution channel; local version changes do not publish a release
|
|
455
|
+
- conda-forge: **Approved & Merged** — Feedstock live at `conda-forge/dct-toolkit-feedstock`
|
|
424
456
|
|
|
425
457
|
### Current Version
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
-
|
|
458
|
+
|
|
459
|
+
- Version: `1.0.0`, adding explicit geometry-aware N-D polar processing.
|
|
460
|
+
- Preserve the historical `v0.6.0` and older release records.
|
|
461
|
+
- Do not tag, push, upload to PyPI/conda-forge, or create a public archival record
|
|
462
|
+
without approval.
|
|
463
|
+
- See `docs/RELEASING.md` for local artifact verification and release handoff.
|
|
429
464
|
|
|
430
465
|
### Gap-Filling Status
|
|
431
466
|
- Code exists in `dct_toolkit/gap_filling.py` (available for import directly)
|
|
@@ -5,7 +5,73 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
-
## [
|
|
8
|
+
## [1.0.0] - 2026-10-26
|
|
9
|
+
|
|
10
|
+
Geometry-aware N-D polar release, preserving the v0.6.0 legacy 2D interface.
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- Explicit N-D polar geometry through keyword-only `axis_types`, `axis_coords`,
|
|
14
|
+
`boundary`, `width_units`, `angle_units`, and `filter_axes` in `smooth_polar`,
|
|
15
|
+
forwarded by the smoothing wrapper and statistics with `coordinates='polar'`.
|
|
16
|
+
- Named azimuth/elevation/range axes in arbitrary array order, plus time, linear
|
|
17
|
+
(`cartesian` alias), and independent batch axes; per-axis widths, kernels,
|
|
18
|
+
boundaries, and physical/grid units follow array order.
|
|
19
|
+
- Physical angular widths use actual range and elevation geometry:
|
|
20
|
+
`L_az / (R*cos(elevation)*delta_az)` and `L_el / (R*delta_el)` in grid cells.
|
|
21
|
+
Regular coordinate validation rejects singular origins/poles and incomplete
|
|
22
|
+
non-singleton physical periodic azimuth grids rather than capping widths.
|
|
23
|
+
- Synthetic, plotting-free `examples/nd_polar.py` demonstrates a
|
|
24
|
+
`(time, elevation, azimuth, range)` field, missing data, mixed kernels/boundaries,
|
|
25
|
+
axis permutation, and isolation of unfiltered time slices.
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
- Explicit geometry uses canonical azimuth/elevation/range/time/linear filter
|
|
29
|
+
order independently of array storage order. The generally nonseparable,
|
|
30
|
+
noncommuting operator is not an exact isotropic or cell-volume convolution.
|
|
31
|
+
- Batch and unfiltered dimensions remain independent through smoothing,
|
|
32
|
+
normalized statistics, iterative filling, and every fallback. They do not
|
|
33
|
+
contribute nominal support area; wholly missing independent domains stay NaN.
|
|
34
|
+
- Documented `dct_count` as nominal weighted support, distinct from literal
|
|
35
|
+
neighbor counts and squared-weight statistical ESS, including repeated samples
|
|
36
|
+
at reflective/periodic boundaries.
|
|
37
|
+
|
|
38
|
+
### Documentation corrections
|
|
39
|
+
- Removed the additive-epsilon description: the normal public mean ratio uses
|
|
40
|
+
the full operator once on data and mask and requires a denominator strictly
|
|
41
|
+
greater than `1e-3`. Variance estimates both moments independently, including
|
|
42
|
+
separate fallback prefills of `x` and `x**2`, then clamps negative results.
|
|
43
|
+
- Qualified nonnegative-weight statistical guarantees: arbitrary fractional
|
|
44
|
+
spectral kernels need not have nonnegative spatial weights.
|
|
45
|
+
- Replaced the broken mathematical-guide link to the removed gap-filling basis
|
|
46
|
+
with research-internal module documentation, without adding a public export.
|
|
47
|
+
|
|
48
|
+
### Fixed
|
|
49
|
+
- Validate integer coordinate spacing before floating-point conversion, including
|
|
50
|
+
large time origins and overflow-safe signed/unsigned integer differences.
|
|
51
|
+
- Promote low-precision floating coordinates before spacing validation, avoiding
|
|
52
|
+
subtraction overflow and concealed irregular grids without downcasting wider types.
|
|
53
|
+
- Compute nominal support lazily with exponent scaling. An unrepresentable count
|
|
54
|
+
factor no longer prevents valid smoothing or normalized-moment computations;
|
|
55
|
+
individual effective widths remain validated.
|
|
56
|
+
- Remove array-layout bias from repeated time/linear-role fallback through joint
|
|
57
|
+
mixed-boundary nearest completion and symmetric averaging of exact global ties.
|
|
58
|
+
Legacy 2D axis-wise tie rules remain unchanged.
|
|
59
|
+
- Preserve zero-dimensional array outputs during count, variance, and standard
|
|
60
|
+
deviation mask restoration, consistently with smoothing and mean operations.
|
|
61
|
+
- Check installed runtime/distribution version alignment and run the suite against
|
|
62
|
+
the installed package in CI, including Python 3.8 with NumPy 1.20.0/SciPy 1.8.0.
|
|
63
|
+
- Document editable metadata refresh and the coordinated PyPI/conda-forge release
|
|
64
|
+
process, including checksums of the exact published source archive.
|
|
65
|
+
|
|
66
|
+
### Compatibility
|
|
67
|
+
- Existing `smooth_polar` positional arguments, the `width_pixels` name, and
|
|
68
|
+
positional `width` in high-level functions are unchanged.
|
|
69
|
+
- `axis_types=None` preserves the legacy 2D `(azimuth, range)` path, index-based
|
|
70
|
+
range convention, scalar/pair widths and kernels, and existing boundary rules.
|
|
71
|
+
Nondefault new geometry options require explicit `axis_types`; old widths are
|
|
72
|
+
not silently reinterpreted as physical distances.
|
|
73
|
+
|
|
74
|
+
## [0.6.0] - 2026-09-21
|
|
9
75
|
|
|
10
76
|
### Changed
|
|
11
77
|
- Added anisotropic width support while keeping scalar width as the default path:
|
|
@@ -14,10 +80,51 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
14
80
|
`(width_azimuth, width_range)`.
|
|
15
81
|
- `dct_count` now uses anisotropic window area (`prod(widths)` in Cartesian,
|
|
16
82
|
`width_range * width_azimuth/(r*dtheta)` in polar).
|
|
83
|
+
- `dct_prefill` now gates per-iteration mean estimates by the local valid-sample
|
|
84
|
+
density (`smooth(mask)`) with `min_effective_density=0.35` by default. This
|
|
85
|
+
stabilizes iterative gap-filling in low-support regions and uses the same
|
|
86
|
+
density signal that `dct_count / area` represents internally. The gate is
|
|
87
|
+
applied uniformly across Cartesian and polar coordinates and can be disabled
|
|
88
|
+
by passing `min_effective_density=None`.
|
|
17
89
|
|
|
18
90
|
### Added
|
|
91
|
+
- Added mixed polar kernels in `(azimuth, range)` order for `smooth_polar`,
|
|
92
|
+
both polar transfer-function entry points, and high-level smoothing,
|
|
93
|
+
statistics, and prefill. A single string still applies to both dimensions.
|
|
94
|
+
Both azimuth boundary modes retain range-adaptive widths and apply azimuth
|
|
95
|
+
smoothing before reflective range smoothing.
|
|
96
|
+
- Added independent polar reference tests covering all mixed kernel pairs,
|
|
97
|
+
circular discrete averaging, odd/even azimuth lengths, singleton axes,
|
|
98
|
+
input validation, normalized statistics, prefill, and NaN handling.
|
|
99
|
+
- Added per-axis Cartesian kernel types: `kernel_type` accepts a string or a
|
|
100
|
+
length-`ndim` sequence of `'boxcar'`, `'boxcar_discrete'`, and `'gaussian'`.
|
|
101
|
+
Mixed kernels work with scalar/per-axis widths in `smooth_cartesian`,
|
|
102
|
+
`dct_smooth`, and Cartesian statistics/prefill. Existing single-string calls
|
|
103
|
+
retain their behavior; the backend still uses one forward/inverse N-D DCT pair.
|
|
104
|
+
- Added mixed-kernel reference, validation, and NaN-handling tests, plus a 3D
|
|
105
|
+
usage example and documentation of array axis order and Gaussian width units.
|
|
19
106
|
- Added parity and artifact-regression tests for scalar-vs-vector isotropic widths,
|
|
20
107
|
plus anisotropic Cartesian/polar smoke coverage.
|
|
108
|
+
- Added `min_effective_density` keyword argument to `dct_prefill` (default
|
|
109
|
+
`0.35`) and `dct_mean` (default `None`, opt-in) for count-based stability
|
|
110
|
+
gating. When set, `dct_mean` returns NaN wherever the local valid-sample
|
|
111
|
+
density is below the threshold. New tests in `tests/test_stats.py` cover
|
|
112
|
+
the default, the disabled, the custom threshold, and the polar-coordinates
|
|
113
|
+
paths.
|
|
114
|
+
|
|
115
|
+
### Fixed
|
|
116
|
+
- Polar nearest-neighbor prefill now respects periodic azimuth in both axis-wise
|
|
117
|
+
propagation and the global fallback, correcting seam-dependent results on
|
|
118
|
+
sparse sweeps. Range propagation remains nonperiodic. Added sparse-sweep
|
|
119
|
+
rotation, targeted global fallback, and singleton-axis regression tests.
|
|
120
|
+
- Polar smoothing, statistics, and prefill now require a finite, positive real
|
|
121
|
+
scalar `az_res_deg`, including calls that return early without smoothing.
|
|
122
|
+
Shared spacing validation prevents negative counts and invalid width arithmetic;
|
|
123
|
+
tests cover invalid inputs and valid fractional/NumPy scalar spacings.
|
|
124
|
+
- Periodic azimuth with `'boxcar_discrete'` now uses an actual centered discrete
|
|
125
|
+
boxcar with an odd integer window, matching the reflective width convention.
|
|
126
|
+
It previously substituted a Gaussian, so results for this option intentionally
|
|
127
|
+
change for both single-string and mixed-kernel calls.
|
|
21
128
|
|
|
22
129
|
## [0.5.0] - 2026-04-14
|
|
23
130
|
|
|
@@ -0,0 +1,316 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dct-toolkit
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: DCT/FFT-based convolutional statistics for Cartesian and polar data
|
|
5
|
+
Author-email: "Jairo M. Valdivia" <Jairo.ValdiviaPrado@colorado.edu>
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 Jairo M. Valdivia
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
Keywords: convolution,dct,fft,polar,radar,smoothing,statistics
|
|
29
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
30
|
+
Classifier: Intended Audience :: Developers
|
|
31
|
+
Classifier: Intended Audience :: Science/Research
|
|
32
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
33
|
+
Classifier: Programming Language :: Python :: 3
|
|
34
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
35
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
36
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
37
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
38
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
39
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
40
|
+
Classifier: Topic :: Scientific/Engineering
|
|
41
|
+
Classifier: Topic :: Scientific/Engineering :: Mathematics
|
|
42
|
+
Requires-Python: >=3.8
|
|
43
|
+
Requires-Dist: numpy>=1.20
|
|
44
|
+
Requires-Dist: scipy>=1.8
|
|
45
|
+
Description-Content-Type: text/markdown
|
|
46
|
+
|
|
47
|
+
# DCT-Toolkit: Convolutional Statistics
|
|
48
|
+
|
|
49
|
+
[](https://www.python.org/downloads/)
|
|
50
|
+
[](LICENSE)
|
|
51
|
+
|
|
52
|
+
`dct_toolkit` provides DCT/FFT-based smoothing and normalized-convolution statistics
|
|
53
|
+
for data with gaps. Runtime dependencies are NumPy and SciPy only.
|
|
54
|
+
|
|
55
|
+
**Version: v1.0.0.** Geometry-aware N-D polar processing requires version 1.0.0
|
|
56
|
+
or newer. Existing 2D polar calls retain their index-based width conventions.
|
|
57
|
+
|
|
58
|
+
## Scope
|
|
59
|
+
|
|
60
|
+
- DCT-based smoothing for 1D/N-D Cartesian data.
|
|
61
|
+
- Per-axis Cartesian widths and kernel types, including mixed boxcar/Gaussian smoothing.
|
|
62
|
+
- Geometry-aware N-D polar/spherical smoothing with named axes in any array order,
|
|
63
|
+
physical or grid widths, independent kernels/boundaries, and batch isolation.
|
|
64
|
+
- Backward-compatible 2D polar smoothing with index-based adaptive azimuth widths.
|
|
65
|
+
- Reflective boundaries via DCT and periodic boundaries via real FFT.
|
|
66
|
+
- Robust local statistics (`dct_mean`, `dct_variance`, `dct_std`, `dct_count`) with NaN support.
|
|
67
|
+
|
|
68
|
+
## Installation
|
|
69
|
+
|
|
70
|
+
### pip
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
python -m pip install dct-toolkit
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### conda
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
conda install -c conda-forge dct-toolkit
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
To update an existing installation:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
python -m pip install --upgrade dct-toolkit
|
|
86
|
+
# Or, in an environment managed by conda:
|
|
87
|
+
conda update -c conda-forge dct-toolkit
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
PyPI and conda-forge publish separately; the conda update becomes available after
|
|
91
|
+
the feedstock update has passed its builds and been merged.
|
|
92
|
+
|
|
93
|
+
For development, install from this checkout:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
python -m pip install -e .
|
|
97
|
+
python examples/nd_polar.py
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
The example uses synthetic data and needs no radar files or plotting packages.
|
|
101
|
+
Install the package first, then run the example from the repository root.
|
|
102
|
+
|
|
103
|
+
## Quick Start
|
|
104
|
+
|
|
105
|
+
### 1) Robust Mean with Missing Data
|
|
106
|
+
|
|
107
|
+
```python
|
|
108
|
+
import numpy as np
|
|
109
|
+
import dct_toolkit as dct
|
|
110
|
+
|
|
111
|
+
data = np.sin(np.linspace(0, 10, 200))
|
|
112
|
+
data[70:110] = np.nan
|
|
113
|
+
|
|
114
|
+
mu = dct.dct_mean(data, width=10.0)
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### 1b) Prefill and Smoothing
|
|
118
|
+
|
|
119
|
+
```python
|
|
120
|
+
# Automatic prefill, then smoothing; the original NaN mask is restored.
|
|
121
|
+
smoothed = dct.dct_smooth(data, width=10.0)
|
|
122
|
+
|
|
123
|
+
# To obtain a filled field instead, explicitly prefill first.
|
|
124
|
+
prefilled = dct.dct_prefill(data, width=10.0)
|
|
125
|
+
full_field = dct.dct_smooth(prefilled, width=10.0)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
### 2) Legacy 2D Polar Smoothing with Periodic Azimuth
|
|
129
|
+
|
|
130
|
+
For data in **(azimuth, range)** order, combine a boxcar along azimuth with a
|
|
131
|
+
Gaussian along range:
|
|
132
|
+
|
|
133
|
+
```python
|
|
134
|
+
radar_data = np.random.default_rng(42).standard_normal((360, 100))
|
|
135
|
+
smoothed = dct.dct_smooth(
|
|
136
|
+
radar_data,
|
|
137
|
+
width=(5.0, 3.0),
|
|
138
|
+
coordinates="polar",
|
|
139
|
+
kernel_type=("boxcar", "gaussian"),
|
|
140
|
+
az_boundary="periodic",
|
|
141
|
+
az_res_deg=1.0,
|
|
142
|
+
)
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
With `axis_types=None` (the default), both sequences follow `(azimuth, range)`
|
|
146
|
+
order. Azimuth width adapts with range:
|
|
147
|
+
`width_azimuth / (r * dtheta)` beams, where `r` starts at 1 and `dtheta` is the
|
|
148
|
+
azimuth spacing in radians. Range width is in range gates. Gaussian
|
|
149
|
+
`sigma = effective_width / sqrt(12)` on the selected axis. Filtering applies
|
|
150
|
+
azimuth first, then range; both periodic and reflective azimuth support all
|
|
151
|
+
three kernel types, including `'boxcar_discrete'`.
|
|
152
|
+
`az_res_deg` must be a finite, positive scalar in degrees. For gapped data,
|
|
153
|
+
periodic azimuth also wraps the nearest-neighbor prefill fallback across the
|
|
154
|
+
sweep seam; range distances remain nonperiodic.
|
|
155
|
+
|
|
156
|
+
This compatibility path does **not** infer actual range coordinates or elevation.
|
|
157
|
+
Its widths are not reinterpreted as physical distances in v1.0.0. Nondefault new
|
|
158
|
+
geometry arguments require explicit `axis_types`.
|
|
159
|
+
|
|
160
|
+
### 3) Local Variability Estimates
|
|
161
|
+
|
|
162
|
+
```python
|
|
163
|
+
var = dct.dct_variance(data, width=10.0)
|
|
164
|
+
std = dct.dct_std(data, width=10.0)
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
### 4) Mixed Kernels for a 3D Volume
|
|
168
|
+
|
|
169
|
+
For a volume stored in **(z, y, x)** order, apply Gaussian smoothing vertically
|
|
170
|
+
and boxcar smoothing horizontally:
|
|
171
|
+
|
|
172
|
+
```python
|
|
173
|
+
rng = np.random.default_rng(42)
|
|
174
|
+
volume = rng.standard_normal((16, 32, 32)) # (z, y, x)
|
|
175
|
+
|
|
176
|
+
smoothed = dct.dct_smooth(
|
|
177
|
+
volume,
|
|
178
|
+
width=(3.0, 5.0, 5.0),
|
|
179
|
+
kernel_type=("gaussian", "boxcar", "boxcar"),
|
|
180
|
+
)
|
|
181
|
+
|
|
182
|
+
# The same kernels work with normalized statistics on data containing NaNs.
|
|
183
|
+
volume[5:7, 12:15, 12:15] = np.nan
|
|
184
|
+
mean = dct.dct_mean(
|
|
185
|
+
volume,
|
|
186
|
+
width=(3.0, 5.0, 5.0),
|
|
187
|
+
kernel_type=("gaussian", "boxcar", "boxcar"),
|
|
188
|
+
)
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Kernel and width sequences follow NumPy axis order and must have one entry per
|
|
192
|
+
dimension. Widths are in grid cells along each axis; Gaussian
|
|
193
|
+
`sigma = width / sqrt(12)`. Convert physical widths using the corresponding
|
|
194
|
+
grid spacing before calling. A scalar width applies equal widths, and a single
|
|
195
|
+
kernel string applies that kernel to every axis. Equal widths with different
|
|
196
|
+
kernel types can still produce anisotropic smoothing. Legacy polar smoothing
|
|
197
|
+
accepts a kernel pair, using the `(azimuth, range)` convention shown above.
|
|
198
|
+
|
|
199
|
+
### 5) N-D Polar Geometry with Actual Coordinates (v1.0.0)
|
|
200
|
+
|
|
201
|
+
For a regular array in **(time, elevation, azimuth, range)** order:
|
|
202
|
+
|
|
203
|
+
```python
|
|
204
|
+
time = np.arange(3) * 30.0 # seconds
|
|
205
|
+
elevation = np.array([2.0, 4.0, 6.0]) # degrees
|
|
206
|
+
azimuth = np.arange(0.0, 360.0, 10.0) # full circle, no duplicate endpoint
|
|
207
|
+
ranges = 2.0 + 0.25 * np.arange(24) # km; first gate is really 2 km
|
|
208
|
+
data = np.random.default_rng(42).standard_normal((3, 3, 36, 24))
|
|
209
|
+
data[0, 1, :3, 8:12] = np.nan
|
|
210
|
+
|
|
211
|
+
mean = dct.dct_mean(
|
|
212
|
+
data,
|
|
213
|
+
(60.0, 0.3, 0.9, 0.75), # seconds, km, km, km
|
|
214
|
+
coordinates="polar",
|
|
215
|
+
axis_types=("time", "elevation", "azimuth", "range"),
|
|
216
|
+
axis_coords=(time, elevation, azimuth, ranges),
|
|
217
|
+
width_units="physical",
|
|
218
|
+
angle_units="degrees",
|
|
219
|
+
kernel_type=("gaussian", "boxcar_discrete", "boxcar_discrete", "gaussian"),
|
|
220
|
+
boundary=("reflective", "reflective", "periodic", "reflective"),
|
|
221
|
+
)
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
- Every per-axis sequence follows **array order**, not a fixed spherical order.
|
|
225
|
+
Reorder `axis_types`, coordinates, widths, kernels, boundaries, and per-axis
|
|
226
|
+
width units together when transposing the data.
|
|
227
|
+
- `axis_types` supports `azimuth`, `elevation`, `range`, `time`, `linear`,
|
|
228
|
+
`cartesian` (alias of `linear`), and `batch`; spherical roles cannot repeat.
|
|
229
|
+
- Explicit geometry defaults to `width_units="physical"`. Time/linear/range
|
|
230
|
+
widths are divided by coordinate spacing. Angular widths are **distances in
|
|
231
|
+
the range coordinate's units**, not degrees: in grid cells,
|
|
232
|
+
`w_az = L_az / (R * cos(elevation) * delta_az)` and
|
|
233
|
+
`w_el = L_el / (R * delta_el)`, using radians internally. Without an elevation
|
|
234
|
+
axis, the azimuth cosine factor is 1. `width_units="grid"` means literal cells;
|
|
235
|
+
a length-`ndim` sequence can mix physical and grid units.
|
|
236
|
+
- Supply actual coordinates where physical conversion needs them. Coordinates
|
|
237
|
+
must be finite real 1D arrays, strictly increasing and uniformly spaced.
|
|
238
|
+
Physical angular filtering requires positive actual ranges and the relevant
|
|
239
|
+
angular coordinates; singular origins/poles are rejected, not capped.
|
|
240
|
+
Non-singleton physical periodic azimuth filtering requires a full 360-degree
|
|
241
|
+
or `2*pi`-radian grid without repeating the endpoint.
|
|
242
|
+
- `filter_axes=None` smooths all non-batch axes. For independent time slices,
|
|
243
|
+
pass `filter_axes=(1, 2, 3)` in this example. Unselected and `batch` axes never
|
|
244
|
+
exchange values, even during prefill or low-support fallback. A wholly missing
|
|
245
|
+
independent domain stays missing. Batch axes cannot be selected for filtering.
|
|
246
|
+
Singleton axes are identities with nominal support width 1.
|
|
247
|
+
|
|
248
|
+
Filtering uses canonical role order: **azimuth, elevation, range, time, linear**,
|
|
249
|
+
regardless of storage order. Geometry-dependent widths make this operator
|
|
250
|
+
generally nonseparable and noncommuting. It is not an exact isotropic Cartesian
|
|
251
|
+
or geodesic convolution, nor a cell-volume-weighted average. Only regular
|
|
252
|
+
coordinate grids are supported, not irregular concatenated multi-sweep radar
|
|
253
|
+
acquisitions. See [the complete example](examples/nd_polar.py) for missing-data,
|
|
254
|
+
axis-permutation, and independent-domain demonstrations.
|
|
255
|
+
|
|
256
|
+
## Statistics and Support
|
|
257
|
+
|
|
258
|
+
Let `S` be the complete smoothing operator and `m` the finite-data validity mask.
|
|
259
|
+
The mean uses `S(m*x) / S(m)`, dividing **once after all filters**, with invalid
|
|
260
|
+
data zero-filled before filtering. The normal ratio path requires a finite
|
|
261
|
+
denominator **strictly greater than `1e-3`**; it does not add an epsilon.
|
|
262
|
+
Low-support locations use the existing prefill-and-smooth fallback when their
|
|
263
|
+
independent domain has support. Variance independently estimates the first and
|
|
264
|
+
second moments, including separate prefill of `x` and `x**2` in fallback, then
|
|
265
|
+
clamps `E[x**2] - E[x]**2` to zero from below.
|
|
266
|
+
|
|
267
|
+
`dct_count` keeps its API name but reports **nominal weighted support**:
|
|
268
|
+
`clip(S(m), 0, 1)` times the product of local grid widths on filtered axes.
|
|
269
|
+
Batch/unfiltered axes contribute no support factor. This is not a literal number
|
|
270
|
+
of distinct neighbors, a cell volume, or squared-weight statistical effective
|
|
271
|
+
sample size (ESS). Reflection/wrapping can revisit the same input sample.
|
|
272
|
+
Nonnegative-weight variance guarantees require actual nonnegative spatial
|
|
273
|
+
weights; arbitrary fractional spectral kernels are not automatically positive.
|
|
274
|
+
|
|
275
|
+
## Quick Cheat Sheet
|
|
276
|
+
|
|
277
|
+
| Function | Description |
|
|
278
|
+
|----------|-------------|
|
|
279
|
+
| `get_dct_transfer_function` | Build 1D spectral kernel `H[k]` from width and kernel type. |
|
|
280
|
+
| `dct_convolve_1d` | Apply one 1D DCT convolution along one axis using a precomputed `H`. |
|
|
281
|
+
| `smooth_cartesian` | Low-level separable smoothing for N-D Cartesian arrays. |
|
|
282
|
+
| `smooth_polar` | Low-level N-D geometry-aware or legacy 2D polar smoothing. |
|
|
283
|
+
| `dct_smooth` | Top-level "just smooth this" wrapper (auto-pre-fills NaNs, then restores NaN mask). |
|
|
284
|
+
| `dct_mean` | NaN-robust local mean via normalized convolution. |
|
|
285
|
+
| `dct_count` | Nominal weighted support from a validity mask (not statistical ESS). |
|
|
286
|
+
| `dct_prefill` | Iterative normalized-convolution gap fill (often used before full-field smoothing). |
|
|
287
|
+
| `dct_variance` | NaN-robust local variance from normalized-convolution moments. |
|
|
288
|
+
| `dct_std` | NaN-robust local standard deviation (`sqrt(dct_variance)`). |
|
|
289
|
+
|
|
290
|
+
## Pre-filling (`dct_prefill`)
|
|
291
|
+
|
|
292
|
+
`dct_prefill` provides an **iterative normalized-convolution gap fill** based on `dct_mean`:
|
|
293
|
+
|
|
294
|
+
- Fills gaps (NaNs or a specified mask) using iterative local averaging.
|
|
295
|
+
- Preserves non-target values exactly.
|
|
296
|
+
- Intended as a **pre-processing step** before full-field smoothing.
|
|
297
|
+
- `max_iter=3` by default; `max_iter=None` iterates until convergence or a safety
|
|
298
|
+
cap of 20 iterations. Remaining targets use nearest-neighbor fallback only
|
|
299
|
+
within their filtered domain; domains with no finite support remain missing.
|
|
300
|
+
- `min_effective_density=0.35` gates iterative estimates using the smoothed
|
|
301
|
+
validity mask, not statistical ESS. Set it to `None` to disable the gate.
|
|
302
|
+
|
|
303
|
+
See the Quick Start section above for usage examples.
|
|
304
|
+
|
|
305
|
+
Research-internal inpainting/diffusion algorithms are documented in the
|
|
306
|
+
[gap-filling module](dct_toolkit/gap_filling.py); they are not exported by the
|
|
307
|
+
top-level public API and do not inherit the new N-D geometry contract.
|
|
308
|
+
|
|
309
|
+
## Documentation
|
|
310
|
+
|
|
311
|
+
- [Mathematical basis](docs/MATHEMATICAL_BASIS.md) — Operators, moments, and support
|
|
312
|
+
- [API reference](docs/API_REFERENCE.md) — Parameters, compatibility, and limitations
|
|
313
|
+
|
|
314
|
+
## License
|
|
315
|
+
|
|
316
|
+
MIT. See `LICENSE`.
|