@pikaa-ai/pikaa 0.3.1 → 0.3.4

package/skills/fluidsim/references/advanced_features.md

@@ -1,347 +0,0 @@
-# Forcing, operators, MPI, extensions, and migrations
-
-## Forcing architecture
-
-FluidSim assembles forcing classes through the selected solver's registry.
-NS2D 0.9 advertises:
-
-- `in_script`
-- `in_script_coarse`
-- `pseudo_spectral`
-- `proportional`
-- `tcrandom`
-- `tcrandom_anisotropic`
-
-Availability and default forced variable are solver-specific.
-
-Base fields:
-
-```python
-params.forcing.enable = True
-params.forcing.type = "tcrandom"
-params.forcing.forcing_rate = 1.0
-params.forcing.key_forced = None
-params.forcing.nkmin_forcing = 4
-params.forcing.nkmax_forcing = 5
-params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"
-```
-
-Current source converts `nkmin_forcing`/`nkmax_forcing` to dimensional
-wave-number bounds using the operator's wave-number spacing. Inspect the
-resulting forced region; the integers are not necessarily physical wave
-numbers.
-
-### Normalization
-
-Current normalized-forcing fields include:
-
-```python
-params.forcing.normalized.constant_rate_of = None
-params.forcing.normalized.type = "2nd_degree_eq"
-params.forcing.normalized.which_root = "minabs"
-```
-
-The implementation can solve a quadratic normalization so the time-step-mean
-injection of a quadratic quantity matches `forcing_rate`. The exact quadratic
-quantity and key depend on the solver/forced field. Therefore:
-
-- State the intended injected invariant and units.
-- Confirm `key_forced`.
-- Measure forcing power in output.
-- Check the global/spectral budget using the same convention.
-- Test time-step sensitivity of the measured injection.
-
-Do not call `forcing_rate` “energy input” without verifying the selected class.
-
-### Time-correlated random forcing
-
-The 0.9 field is:
-
-```python
-params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"
-```
-
-or a finite time value. Current source derives the default period as a power of
-the forcing rate and stores two random seeds plus the last-change time in state
-parameters. FluidSim 0.9.0 writes these state parameters into restart files;
-0.8.6 fixed a time-correlated forcing restart bug.
-
-For reproducibility, preserve:
-
-- Initial random seed strategy.
-- Saved forcing state parameters.
-- MPI rank count/decomposition and package versions.
-- Correlation-time setting and measured autocorrelation.
-- Restart boundary diagnostics.
-
-### In-script forcing
-
-Use the solver's registered `InScriptForcing*` interface and documented
-`compute_forcing_fft_each_time` or coarse equivalent. Do not monkey-patch a
-method with a lambda copied from an old example:
-
-- State keys have changed in some solvers.
-- Local spectral layout depends on FFT/MPI backend.
-- Hermitian/reality constraints and normalization must be preserved.
-- A literal global Fourier index is not portable across decompositions.
-
-Implement a reviewed subclass/extension with unit tests on tiny sequential and
-MPI layouts. Validate zero-net/target injection, symmetry, and budget effects.
-
-## Operators and array ownership
-
-`sim.oper` provides solver-selected grids, FFT/IFFT, differentiation, vector
-calculus, projections, spectra, dealiasing, and distributed-array helpers.
-Method names and array layouts depend on operator class.
-
-Never assume:
-
-- Axis order from `nx`, `ny`, `nz`.
-- Full global arrays on every rank.
-- A Fourier mode has the same local index under another backend/rank count.
-- All FFT backends use the same spectral shape.
-- A gathered array fits rank-0 memory.
-- Direct NumPy sums have the same normalization as
-  `oper.sum_wavenumbers`.
-
-Use documented operator methods and inspect:
-
-```python
-print(type(sim.oper))
-print(sim.oper.axes)
-print(sim.params.oper)
-```
-
-For custom diagnostics, test sequential and distributed shapes and compare
-against analytical transforms at tiny resolution.
-
-## Dealiasing
-
-The common Cartesian field is:
-
-```python
-params.oper.coef_dealiasing = 2 / 3
-params.oper.truncation_shape = "cubic"
-```
-
-FluidSim also implements phase-shift time schemes. A coefficient or scheme name
-does not prove alias removal for a custom nonlinearity. Verify:
-
-- Polynomial/nonlinear form and expected alias interactions.
-- Where dealiasing is applied.
-- Truncation geometry.
-- Spectral tails and invariant transfer.
-- Results under stricter truncation or exact phase-shift method.
-
-Do not combine an aggressive cutoff and high-order dissipation merely to obtain
-a visually smooth spectrum.
-
-## Time schemes
-
-Documented pseudospectral names:
-
-- `Euler`
-- `Euler_phaseshift`
-- `Euler_phaseshift_random`
-- `RK2`
-- `RK2_trapezoid`
-- `RK2_phaseshift`
-- `RK2_phaseshift_random`
-- `RK2_phaseshift_random_split`
-- `RK2_phaseshift_exact`
-- `RK4`
-
-The implementation treats linear terms with exact coefficients in its
-pseudospectral stepper and evaluates nonlinear tendencies according to the
-named scheme. Verify source and solver coupling before making an order/stability
-claim.
-
-Always check:
-
-- Advective CFL.
-- Wave frequency limits (stratification, rotation, shallow-water waves).
-- Diffusive/hyperdiffusive limits.
-- Forcing correlation and output cadence relative to `deltat`.
-- Smaller `cfl_coef`/`deltat_max` comparison.
-
-`USE_CFL=True` only activates the solver's CFL logic; it does not guarantee all
-accuracy/stability constraints are resolved.
-
-## Custom initial conditions
-
-`in_script` gives direct control, but use current state keys:
-
-1. Construct `Simul` with `init_fields.type = "in_script"`.
-2. Inspect the solver's state documentation/keys.
-3. Fill canonical physical or spectral variables.
-4. Call the documented conversion in the correct direction.
-5. Apply projection/dealiasing/constraints as required.
-6. Save an initialization checkpoint and verify budgets before stepping.
-
-Old examples that fill `vx`/`vy` and then call a
-spectral-to-physical conversion can overwrite the intended state. NS2D 0.9
-documentation shows physical keys including `ux`, `uy`, and `rot`; use the
-selected solver's actual keys.
-
-## Extending a solver
-
-FluidSim's `InfoSolver`/class registry supports extensions. For a research
-extension:
-
-- Pin FluidSim/FluidSim Core source and version.
-- Subclass the closest solver and extend default parameters through the current
-  class mechanism.
-- Register state variables, operators, initialization, forcing, outputs, and
-  restart state explicitly.
-- Define nonlinear tendencies with documented sign and normalization.
-- Add unit/manufactured-solution tests and budget identities.
-- Test serialization/restart and old/new parameter merging.
-- Benchmark only after correctness tests.
-
-Avoid private-method snippets from old versions without source review.
-
-## FluidFFT backend selection
-
-Installed methods are entry points. Discover them:
-
-```python
-from fluidfft import get_methods
-
-print(sorted(get_methods(ndim=2)))
-print(sorted(get_methods(ndim=3)))
-```
-
-Set per run:
-
-```python
-params.oper.type_fft = "fft2d.with_pyfftw"
-```
-
-or use `FLUIDSIM_TYPE_FFT2D`/`FLUIDSIM_TYPE_FFT3D` before process start.
-Record actual method and plugin distribution.
-
-FluidFFT's 2019 primary paper demonstrates:
-
-- Unified C++/Python APIs for multiple FFT libraries.
-- One-dimensional and pencil/two-dimensional MPI decompositions.
-- Hardware/shape/process-count-dependent fastest methods.
-- Scaling beyond the limits of slab decomposition in the tested cases.
-
-Do not transfer its fastest-method or wall-time numbers to current hardware.
-Benchmark a bounded representative shape in the target environment.
-
-## MPI planning and safety
-
-Never call `mpirun`, `mpiexec`, `srun`, `qsub`, `sbatch`, OAR tools, or a
-FluidDyn cluster submitter automatically.
-
-Required preflight:
-
-- Written resource estimate and output estimate.
-- Approved allocation and partition/account.
-- Exact MPI implementation/ABI and launcher.
-- FFT plugin/native library compatibility.
-- Rank/thread placement and oversubscription check.
-- Per-rank local shapes and no zero-sized unsupported decomposition.
-- Memory/rank and rank-0 gather/output risk.
-- Wall-time signal/checkpoint behavior.
-- Filesystem quota, inode count, stripe policy, and cleanup.
-- Tiny serial then two-rank smoke.
-- Restart plan with immutable parent state.
-
-Output behavior can differ with MPI-enabled h5py. Standard h5py usually causes
-rank 0 to write assembled state; MPI h5py can use an `mpio` driver. Verify the
-locked h5py build and output path with a tiny test.
-
-## Parametric studies
-
-Do not loop over simulations and start them directly in one script by default.
-Instead:
-
-1. Materialize one strict config per case.
-2. Assign a stable case ID and seed.
-3. Validate/estimate each case.
-4. Sum aggregate CPU, memory concurrency, disk, files, and wall time.
-5. Generate scripts only.
-6. Review sampling design and avoid changing multiple factors ambiguously.
-7. Submit through an approved external workflow.
-8. Track failures/missing cases without silently resampling.
-
-Analyze observables with refinement and stochastic uncertainty, not only final
-values.
-
-## Checkpoint and restart
-
-Physical-state saving is checkpoint creation:
-
-```python
-params.output.periods_save.phys_fields = 1.0
-```
-
-But checkpoint usability requires:
-
-- Complete `/state_phys` datasets.
-- `/info_simul/params` and solver metadata.
-- 0.9 state parameters where needed.
-- Matching solver/grid/domain/state.
-- SHA-256 and parent lineage.
-- Enough disk for parent and child.
-
-Use the bundled compatibility checker before every continuation. A mechanically
-compatible state can still be scientifically invalid after changed viscosity,
-forcing, timestep, backend, or resolution.
-
-## Migration notes to 0.9.0
-
-### 0.9.0 (release notes dated 2025-12-03)
-
-- Restart files store state parameters.
-- Added basic physical-field utilities.
-- Fixed restart filenames.
-- Improved post-initialization information and profile analysis.
-
-### 0.8.6 (2025-11-23)
-
-- h5netcdf 1.7 compatibility.
-- Fixed incorrect restart for time-correlated forcing.
-
-### 0.8.5 (2025-10-23)
-
-- Python 3.14 support.
-
-### 0.8.2 (2024-08-17)
-
-- Python 3.12, NumPy 2.0, and mpi4py 4.0 compatibility.
-
-### 0.8.0 (2024-01-31)
-
-- Meson/meson-python build system.
-
-Practical migrations from the previous skill:
-
-- Python baseline: `>=3.11`, not `>=3.9` for 0.9.0.
-- Use exact `fluidsim==0.9.0` and lock dependencies.
-- Pseudospectral defaults need the `fft` extra.
-- Use `time_stepping.cfl_coef`, not `CFL`.
-- Use `forcing.tcrandom.time_correlation`, not a flat field.
-- Use `plate2d`, not `fvk`.
-- Physical states default to `.nc`, not `.h5`; spectra remain `.h5`.
-- Use `spect_energy_budg.h5`, not a timestamped budget glob.
-- Prefer `load_for_restart`/`fluidsim-restart --only-check`; preserve state
-  parameters and hashes.
-- Do not advertise ParaView direct compatibility without an explicit tested
-  conversion/plugin.
-
-## Sources (verified 2026-07-23)
-
-- [FluidSim forcing base source](https://github.com/fluiddyn/fluidsim/blob/branch/default/fluidsim/base/forcing/base.py).
-- [Specific forcing source](https://github.com/fluiddyn/fluidsim/blob/branch/default/fluidsim/base/forcing/specific.py).
-- [Pseudospectral time-step API](https://fluidsim.readthedocs.io/en/latest/generated/fluidsim.base.time_stepping.pseudo_spect.html).
-- [FluidSim development tutorial](https://fluidsim.readthedocs.io/en/latest/ipynb/tuto_dev.html).
-- [FluidSim release notes](https://fluidsim.readthedocs.io/en/latest/changes.html).
-- [FluidFFT plugins](https://fluidfft.readthedocs.io/en/latest/plugins.html).
-- [FluidFFT supported libraries](https://fluidfft.readthedocs.io/en/latest/install/fft_libs.html).
-- Mohanan et al., [FluidFFT primary paper](https://doi.org/10.5334/jors.238),
-  published 2019-04-01.
-- Mohanan et al., [FluidSim primary paper](https://doi.org/10.5334/jors.239),
-  published 2019-04-26.

package/skills/fluidsim/references/installation.md

@@ -1,263 +0,0 @@
-# Installation, native dependencies, and backends
-
-## Supported baseline
-
-Verified 2026-07-23:
-
-- `fluidsim==0.9.0`, released on PyPI 2025-12-04.
-- FluidSim package metadata requires Python `>=3.11` and classifies Python
-  3.11–3.14.
-- `fluidsim-core==0.9.0`, released 2025-12-03.
-- `fluidfft==0.4.5`, released 2025-10-13, requires Python `>=3.11`.
-- `pyFFTW==0.15.1`, released 2025-10-22, requires Python `>=3.11`.
-- `mpi4py==4.1.2`, released 2026-05-16, requires Python `>=3.8`.
-
-The FluidSim installation page still says Python `>=3.9`; current PyPI and
-`pyproject.toml` metadata say `>=3.11`. Use the package metadata for 0.9.0.
-
-FluidSim 0.9.0 declares:
-
-- Core: `fluidsim-core>=0.8.6,<0.9.1`, `h5py`, `h5netcdf`,
-  `transonic>=0.6.2`, `xarray`, `rich`, `matplotlib>=3.3`, and `scipy`.
-- `fft`: `pyfftw>=0.10.4`, `fluidfft>=0.4.0`.
-- `mpi`: `mpi4py`.
-- Other extras: `test`, `test-mpi`, and `pulp`.
-
-The broad upstream constraints are compatibility ranges, not a reproducible
-environment. Record the generated lock and artifact hashes.
-
-## Reproducible uv environment
-
-Preferred project workflow:
-
-```bash
-uv init --python 3.11
-uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
-uv lock
-uv sync --frozen
-```
-
-Check the lock into the study repository. Record:
-
-- `uv.lock` SHA-256 and target platform.
-- Python implementation/build.
-- FluidSim, FluidSim Core, FluidDyn, FluidFFT, Transonic, Pythran, NumPy,
-  SciPy, h5py, h5netcdf, xarray, and pyFFTW versions.
-- Wheel/sdist hashes and package index.
-- Compiler and native-library versions if any package builds locally.
-
-For an isolated smoke environment:
-
-```bash
-uv venv --python 3.11
-uv pip install "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
-```
-
-This pins direct dependencies but does not replace a lock for transitive
-reproducibility.
-
-Bare `fluidsim==0.9.0` supports parts of the framework and analysis stack, but a
-verified local smoke test found that importing NS2D succeeded while
-`Simul.create_default_params()` failed without `fluidfft`. Install the `fft`
-extra for pseudospectral solvers.
-
-## Sequential FFT choices
-
-The non-compiling path is:
-
-```bash
-uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
-```
-
-FluidFFT 0.4.5 registers:
-
-- `fft2d.with_pyfftw`
-- `fft3d.with_pyfftw`
-- `fft2d.with_dask` when Dask is installed
-
-The native FFTW plugin is separately versioned:
-
-```bash
-uv add "fluidfft-fftw==0.0.1"
-```
-
-The `0.0.1` plugin versions are stable PyPI releases from February 2024 and are
-versioned independently from FluidFFT 0.4.5; they are not proof of compatibility
-with a particular native stack. Before installation, verify that each PyPI
-project links to the official `fluiddyn/fluidfft` monorepo, review the plugin
-source, resolve through `uv.lock`, retain artifact hashes, and use
-`uv sync --frozen`. Do not trust a familiar distribution name alone.
-
-It provides:
-
-- `fft2d.with_fftw1d`
-- `fft2d.with_fftw2d`
-- `fft3d.with_fftw3d`
-
-It requires discoverable FFTW headers/libraries and a working native build
-toolchain. pyFFTW wheels bundle supported binaries on many 64-bit platforms;
-source builds require FFTW `>=3.3`, Cython, and a compiler.
-
-Discover only methods actually installed on the current host:
-
-```bash
-fluidfft-get-methods
-```
-
-Do not copy a method name from documentation and assume its plugin or ABI is
-usable. Run a tiny transform/FluidSim pilot and record the selected method.
-
-## MPI and distributed FFT
-
-Nothing in this skill launches MPI or submits a scheduler job. First identify:
-
-- Site MPI implementation and version: Open MPI, MPICH derivative, Intel MPI,
-  Cray MPICH, or another vendor stack.
-- Compiler wrappers and ABI.
-- FFTW and `fftw3_mpi` versions/build options.
-- Scheduler, process placement, cores/rank, threads/rank, memory/rank, wall time,
-  filesystem, and module/container environment.
-
-Then lock Python packages:
-
-```bash
-uv add "mpi4py==4.1.2" \
-  "fluidfft-mpi-with-fftw==0.0.1" \
-  "fluidfft-fftwmpi==0.0.1"
-uv lock
-```
-
-Plugin methods:
-
-- `fluidfft-mpi-with-fftw==0.0.1`:
-  `fft2d.mpi_with_fftw1d`, `fft3d.mpi_with_fftw1d`.
-- `fluidfft-fftwmpi==0.0.1`:
-  `fft2d.mpi_with_fftwmpi2d`, `fft3d.mpi_with_fftwmpi3d`.
-- `fluidfft-p3dfft==0.0.1`:
-  `fft3d.mpi_with_p3dfft`, requiring P3DFFT.
-- FluidFFT also declares `pfft` and `p3dfft` extras. Both require separately
-  installed native MPI FFT libraries.
-
-The package names use hyphens on PyPI; FluidFFT's optional dependency keys map
-to distributions such as `fluidfft-mpi_with_fftw`. Let the lock resolve the
-canonical distribution and preserve it.
-
-`mpi4py` wheels still need a compatible MPI runtime. Convenience MPI wheels can
-lack GPU awareness or site fabric support; mpi4py recommends system/vendor MPI
-for production. Never mix an `mpi4py` build from one implementation with a
-different launcher/runtime. Verify import and rank identity inside a manually
-allocated tiny job before FluidSim.
-
-The primary FluidFFT paper shows that the fastest backend depends on array
-shape, machine, and process count; one-dimensional decomposition can be useful
-at low rank count, while pencil/two-dimensional decomposition is needed to
-avoid decomposition limits at high rank count. These are benchmark-context
-claims, not universal backend recommendations.
-
-## GPU status
-
-The 2019 FluidFFT paper describes a cuFFT path, and the repository README still
-lists cuFFT. However:
-
-- FluidFFT 0.4.5 `pyproject.toml` declares no CUDA dependency/extra or cuFFT
-  plugin entry point.
-- The current supported-library page's CUDA section is an unfinished TODO.
-- FluidSim 0.9.0 declares no GPU extra.
-
-Therefore there is no supported one-line GPU installation in this skill. Do not
-install `nvidia-cufft-*` and claim FluidSim acceleration: a runtime library alone
-does not provide a registered FluidFFT method. A GPU experiment must pin CUDA,
-driver, compiler, plugin source revision, Python packages, precision, hardware,
-and validation tests separately.
-
-## Native build prerequisites
-
-Depending on selected plugins:
-
-- C/C++11 and sometimes Fortran compilers.
-- Meson/meson-python, Ninja, Pythran, Transonic, Cython, and development headers.
-- FFTW3, threaded FFTW, and/or FFTW MPI.
-- MPI compiler wrappers and runtime.
-- PFFT or P3DFFT headers/libraries.
-- BLAS configuration used by NumPy/Pythran.
-- `CPATH`, `LIBRARY_PATH`, and runtime loader paths where site modules do not
-  provide them.
-
-The P3DFFT plugin also recognizes `P3DFFT_DIR`, or
-`P3DFFT_LIB_DIR`/`P3DFFT_INCLUDE_DIR`. Record values but never alter global shell
-startup files automatically.
-
-## HDF5 and netCDF4
-
-FluidSim 0.9 physical-state files default to netCDF4/HDF5 `.nc`; spectra remain
-HDF5 `.h5`. Standard h5py wheels are usually non-MPI, which is normally
-appropriate because output is coordinated by FluidSim. Parallel HDF5 is a
-separate native build requiring:
-
-- MPI-enabled HDF5.
-- `h5py` built from source against the same MPI.
-- Matching compiler wrappers and runtime libraries.
-
-Do not build MPI-enabled h5py merely because the simulation uses MPI. Confirm
-the intended I/O path and test a tiny file first.
-
-## Runtime paths and backend selection
-
-Official variables:
-
-```bash
-export FLUIDSIM_PATH="/approved/bounded/results-root"
-export FLUIDDYN_PATH_SCRATCH="/approved/bounded/scratch-root"
-export FLUIDSIM_TYPE_FFT2D="fft2d.with_pyfftw"
-export FLUIDSIM_TYPE_FFT3D="fft3d.with_pyfftw"
-```
-
-Set only after checking:
-
-- Paths exist or will be created in an approved parent.
-- No symlink redirects outside the allocation.
-- Quota and inode limits cover the estimate.
-- The method appears in `fluidfft-get-methods`.
-- Scratch retention and purge policy are recorded.
-
-Prefer `params.oper.type_fft` for an explicit per-run choice. Environment
-variables affect process-wide behavior and must be captured in provenance.
-
-FluidFFT is also sensitive to `TRANSONIC_BACKEND`; changing it changes generated
-code/performance and belongs in provenance.
-
-## Verification ladder
-
-Run in this order:
-
-1. Dependency-free bundled CLI helps.
-2. Import/version/solver parameter smoke in an isolated pinned environment.
-3. Tiny serial `8x8` or `16x16` no-output initialization and one step.
-4. Tiny serial output round-trip and restart.
-5. Backend-specific FFT test.
-6. Manually allocated two-rank smoke, only if MPI is required.
-7. Representative bounded pilot with resource monitoring.
-
-Do not run the full upstream test suite or MPI tests on a login node without
-approval; they can compile, spawn processes, and consume resources.
-
-## Sources (verified 2026-07-23)
-
-- [FluidSim PyPI](https://pypi.org/project/fluidsim/) — 0.9.0 metadata and
-  2025-12-04 release.
-- [FluidSim 0.9 source metadata](https://github.com/fluiddyn/fluidsim/blob/branch/default/pyproject.toml)
-  — dependencies, extras, entry points, Python requirement.
-- [Install and configure](https://fluidsim.readthedocs.io/en/latest/install.html)
-  — extras, native plugins, MPI/HDF5, and environment variables.
-- [FluidFFT 0.4.5 source metadata](https://github.com/fluiddyn/fluidfft/blob/branch/default/pyproject.toml)
-  — plugin extras and methods.
-- [Official FluidFFT plugin source tree](https://github.com/fluiddyn/fluidfft/tree/branch/default/plugins)
-  — provenance for separately distributed native plugins.
-- [FluidFFT plugins](https://fluidfft.readthedocs.io/en/latest/plugins.html) and
-  [installation](https://fluidfft.readthedocs.io/en/latest/install.html).
-- [pyFFTW PyPI](https://pypi.org/project/pyFFTW/) — 0.15.1 metadata and build
-  requirements.
-- [mpi4py PyPI](https://pypi.org/project/mpi4py/) — 4.1.2 metadata and MPI ABI
-  guidance.
-- [FluidFFT primary paper](https://doi.org/10.5334/jors.238), published
-  2019-04-01 — architecture and scoped backend/scaling benchmarks.