stochasticforceinference 2.0.3__tar.gz → 2.1.0__tar.gz

{stochasticforceinference-2.0.3 → stochasticforceinference-2.1.0}/AGENTS.md

@@ -4,6 +4,10 @@
 > agents (GitHub Copilot, Claude Code, Cursor, …) on the Stochastic
 > Force Inference (SFI) repository. Human contributors should read
 > [`CONTRIBUTING.md`](CONTRIBUTING.md).
+>
+> **Using SFI as a library** (analysing data, not modifying SFI)? Read
+> [`SFI/AGENT_GUIDE.md`](SFI/AGENT_GUIDE.md) instead: it ships inside the
+> installed package (`python -m SFI guide`).
 
 ---
 
@@ -45,6 +49,8 @@ SFI has mature, well-tested abstractions for every common operation.
 | Wrap a user function as a state function | `SFI.statefunc.make_basis` / `make_sf` / `make_psf`     |
 | Run force / diffusion inference          | `SFI.OverdampedLangevinInference` / `UnderdampedLangevinInference` |
 | Select a sparse model                    | `inf.sparsify_force(criterion="PASTIS", p=0.1)`         |
+| Select a sparse diffusion model          | `inf.sparsify_diffusion(criterion="PASTIS", p=0.05)`    |
+| Keep terms in every model / select grouped terms together | `inf.sparsify_force(protected=[...], groups={...})` (indices or `basis.labels`; also `sparsify_diffusion`) |
 | Compare to ground truth                  | `inf.compare_to_exact(model_exact=...)`                 |
 | Plot in the SFI style                    | `SFI.utils.plotting.SFI_COLORS`, `dark_fig`, `phase2d`  |
 | Save / load a fitted model               | `SFI.inference.save_model` / `load_model`               |
@@ -78,6 +84,37 @@ from SFI import (
 The dynamics-order classifier (`classify_dynamics`, returning a
 `DynamicsOrderReport`) decides overdamped vs underdamped from raw
 positions; see `SFI.diagnostics` and `examples/gallery/dynamics_order_demo.py`.
+`SFI.diagnostics.finite_dt_scale(inf)` reports the per-step drift-gradient
+scale Δt‖∇F̂‖ of a fitted force (the size of the linear estimators' O(Δt)
+biases) with a `fine` / `coarse` / `unreliable` verdict; `assess(inf)` includes
+it and flags the coarse regimes.  For underdamped linear fits with the default
+constant diffusion, `assess(inf)` also compares it with the drift-corrected one
+(`report.diffusion`: difference, z, chi-squared p-value; one pass with the fitted
+force), the check `compute_diffusion_constant(method="drift-corrected")` runs and
+stores when it replaces the default. Both linear engines also record the force
+Gram condition number (`metadata["force_gram_cond"]`) and warn when the
+dictionary is ill-conditioned on the data; the underdamped linear
+estimators also warn on non-uniform frame intervals
+(`metadata["dt_mismatch_rms"]`), which they do not support.  After an
+underdamped linear fit, `compute_force_error()` also checks the basis's
+path resolution (`metadata["force_path_resolution"]`: features that vary
+along the motion within a stencil, their attenuation `kappa` and predicted
+bias in error bars; it warns when a bias exceeds its error bar).  Under
+strong localisation noise at fine sampling (`metadata["force_noise_ratio"]`
+>= 1, warned), fit with `infer_force_linear(B, stride="auto")` (experimental)
+(`preset="model-free"` / `"constant-noise"`): it picks the frame stride
+(`metadata["force_stride_scan"]`).  Estimate the diffusion constant the same
+way, `compute_diffusion_constant(stride="auto")`
+(`metadata["diffusion_constant_stride_scan"]`): at stride 1 its sign is
+random once the noise ratio exceeds about sqrt(rows), and the force fit
+needs a positive definite constant.  The diffusion field likewise:
+`infer_diffusion_linear(B_D, stride="auto")`
+(`metadata["diffusion_stride_scan"]`).  A selected stride above 1 warns with
+its reason.
+`SFI.diagnostics.psd_fraction(D_sf, coll)` reports the fraction of data
+points where a diffusion field is not positive semi-definite;
+`simulate_bootstrapped_trajectory` runs it on the fitted diffusion and warns
+(the simulator clips negative eigenvalues silently).
 
 ### 4.2 `SFI.statefunc` — composable state functions
 
@@ -89,7 +126,7 @@ positions; see `SFI.diagnostics` and `examples/gallery/dynamics_order_demo.py`.
 | `make_sf(func, dim, rank, ...)`  | Wrap a Python callable as an `SF`         |
 | `make_psf(func, dim, rank, ...)` | Wrap a callable with parameters as `PSF`  |
 | `make_basis(func, dim, ...)`     | Wrap a vector-valued callable as `Basis`  |
-| `Interactor`, `make_interactor`  | Multi-scale interaction graphs            |
+| `Interactor`, `make_interactor`  | Multi-scale interaction graphs (pairs, hyperedges; `per_slot=True` for many-body forces) |
 
 ### 4.3 `SFI.inference` — inference engines
 
@@ -134,7 +171,7 @@ agreement is itself a diagnostic.
   | `inf.print_report()` / `inf.report_dict()` | Human / machine summary |
   | `inf.compare_to_exact(model_exact=...)` | Validate against a known model |
   | `inf.holdout_score(test)` | Held-out NMSE on `coll.split_time(...)` test data (data-abundant scenarios only) |
-  | `inf.compute_entropy_production()` | Dissipation of the observed dynamics (after a linear force fit; both engines): rate ± error + fluctuation bias and the AIC-debiased rate (`Sdot_debiased`), in k_B.  Underdamped: positions-only via the even/odd split; `coefficients=` enables cross-fitting |
+  | `inf.compute_entropy_production()` | Dissipation of the observed dynamics (after a linear force fit; both engines): rate ± error + fluctuation bias and the AIC-debiased rate (`Sdot_debiased`), in k_B.  Underdamped: positions-only via the even/odd split; `coefficients=` enables cross-fitting, `jackknife=True` the delete-half jackknife (`Sdot_jackknife`, free of the plug-in bias) |
   | `inf.time_reversal_split()` | Underdamped only: the fitted field's reversible/irreversible parts `(F_even, F_odd)` as callables |
 
 * **Parametric estimators (robust, flexible; compute-intensive).**
@@ -179,14 +216,14 @@ agreement is itself a diagnostic.
 
 ```python
 inf = OverdampedLangevinInference(collection)
-inf.compute_diffusion_constant(method="auto")        # 1. constant D ("noisy"/"WeakNoise"/"MSD")
-inf.infer_force_linear(basis, preset="auto")         # 2. force ("auto"/"robust"/"clean"/"KM"/legacy)
-inf.infer_diffusion_linear()                         # 3. optional: constant sym D (default basis)
-inf.compute_force_error()                            # 4. error analysis
-inf.sparsify_force(criterion="PASTIS", p=0.1)        # 5. optional: sparsity
-inf.compute_entropy_production()                     # 6. optional: dissipation (OD & UD)
-inf.print_report()                                   # 7. summary
-inf.compare_to_exact(model_exact=process)            # 8. optional: validate
+inf.infer_force_linear(basis)                        # 1. force (overdamped preset "instrument" default/"legacy-robust"/"legacy-clean"/"legacy-KM"/"legacy-noisy-v1.0", "auto" retired: fit it FIRST, it sets the drift-corrected constant D-bar and Lambda; underdamped "model-free" default/"constant-noise"/"legacy-robust"/"legacy-clean"/legacy v1.0, after compute_diffusion_constant())
+                                                     #    constant D otherwise: overdamped compute_diffusion_constant() (default "noisy"; "WeakNoise"/"MSD" by name) only to keep a stencil D (a prior call keeps it), method="drift-corrected" after step 1; underdamped compute_diffusion_constant() before step 1 (default "model-free", legacy stencils by name, "drift-corrected" after step 1)
+inf.infer_diffusion_linear()                         # 2. optional: diffusion field (overdamped: basis optional, default = constant sym D; default "drift-corrected" = regressed on the force's residuals, needs step 1; underdamped: basis required, default "model-free", "drift-corrected" after step 1; M_mode= selects the legacy fit in both)
+inf.compute_force_error()                            # 3. error analysis
+inf.sparsify_force(criterion="PASTIS", p=0.1)        # 4. optional: sparsity
+inf.compute_entropy_production()                     # 5. optional: dissipation (OD & UD)
+inf.print_report()                                   # 6. summary
+inf.compare_to_exact(model_exact=process)            # 7. optional: validate
 ```
 
 **Parametric-estimator sequence:**
@@ -200,10 +237,19 @@ inf.compute_force_error()
 
 **Sparsity.** PASTIS is the canonical criterion in SFI. Alternative
 criteria (`AIC`, `BIC`, `SIC`) and strategies (greedy stepwise,
-STLSQ, LASSO) are available — see
+STLSQ, LASSO, `method="exhaustive"` for the exact front of small
+candidate sets) are available — see
 [`_project_index/api_map.md`](_project_index/api_map.md#sfiinferencesparse)
 for the full inventory. Precision/recall and held-out NMSE helpers
 live in `SFI.inference.sparse.overlap_metrics` and `predictive_nmse`.
+`protected=` keeps functions in every model (not candidates: nuisance
+blocks, a required friction) and `groups=` makes functions enter and
+leave together (a harmonic's `(cos, sin)` pair, per-dataset copies); the
+search then runs over units (`UnitScorer`), `k`/`max_k` count units, and
+PASTIS charges a group of `m` parameters `pastis_unit_penalty(m, n0, p)`
+(the exact chi-squared quantile at a single term's false-inclusion level;
+`ln(n0/p)` for `m = 1`).  `SparsityResult.selected()` returns the chosen
+model with its information, parameter count and unit names.
 
 ### 4.4 `SFI.trajectory` — data containers
 
@@ -354,6 +400,21 @@ Deprecated / soon-to-be-removed symbols (do not use in new code):
   deprecated tuning knobs, accepted but **inert** (a
   `DeprecationWarning` fires when set).  The banded likelihood has no
   window truncation to tune.
+- Underdamped `infer_force_linear(preset="auto" | "robust" | "clean")` —
+  retired names, accepted with a `FutureWarning`: `"auto"` runs the
+  default `"model-free"` (combined with explicit `M_mode` / `G_mode` /
+  `diffusion_method` / `cross_particle` it raises: name a `legacy-` preset); `"robust"` / `"clean"` are
+  now `"legacy-robust"` / `"legacy-clean"`.  Mode overrides apply to the
+  `legacy-` presets only; without a preset they warn and will raise.
+- Underdamped `compute_diffusion_constant(method="auto")` — retired, accepted
+  with a `FutureWarning`: runs the default `"model-free"` (the mean of the
+  model-free force design's per-frame diffusion).  The former choice
+  (`"noisy"` when `Tr Lambda > 0`, else `"WeakNoise"`) is available by name.
+- Overdamped `infer_force_linear(preset="auto")` — retired, accepted with a
+  `FutureWarning`: runs the default `"instrument"`.  Its former choice
+  (`"robust"` when `Tr Lambda > 0`, else `"clean"`) is available by name;
+  mode arguments without a preset are still completed that way
+  (`SharedNoiseOLI` keeps that choice as its default).
 
 ## 6. Testing, building, environment
 
@@ -363,6 +424,7 @@ Full details: [`docs/source/dev_notes.rst`](docs/source/dev_notes.rst).
 source .venv/bin/activate
 pytest tests/ -v                             # full suite (CPU-only via conftest)
 pytest tests/ -x --ff                        # quick smoke
+pytest tests/inference -m sparsity           # when changing sparse selection (deselected by default)
 cd docs && make clean && make html           # fast docs build
 cd docs && SFI_DOCS_RUN_GALLERY=1 make html  # full build (slow)
 ```

{stochasticforceinference-2.0.3 → stochasticforceinference-2.1.0}/MANIFEST.in

@@ -4,6 +4,7 @@ recursive-include examples *.py
 # Top-level docs we want shipped
 include README.md LICENSE pyproject.toml setup.cfg
 include AGENTS.md CONTRIBUTING.md GALLERY_STYLE_GUIDE.md
+include SFI/AGENT_GUIDE.md
 
 # Exclude common junk files
 global-exclude *.csv *.mp4 *.pkl *.npz *.npy *~ *.bak

{stochasticforceinference-2.0.3 → stochasticforceinference-2.1.0}/PKG-INFO

@@ -1,15 +1,17 @@
 Metadata-Version: 2.4
 Name: StochasticForceInference
-Version: 2.0.3
-Summary: Stochastic Force Inference for Langevin SDEs
+Version: 2.1.0
+Summary: Learn stochastic differential equations (Langevin dynamics: drift/force and diffusion) from time series and trajectory data
 Author-email: Pierre Ronceray <pierre.ronceray@univ-amu.fr>
 License: MIT
 Project-URL: Homepage, https://github.com/ronceray/StochasticForceInference
 Project-URL: Documentation, https://stochasticforceinference.readthedocs.io
+Project-URL: Gallery, https://stochasticforceinference.readthedocs.io/en/stable/gallery/index.html
+Project-URL: Guide for AI agents, https://github.com/ronceray/StochasticForceInference/blob/main/SFI/AGENT_GUIDE.md
 Project-URL: Repository, https://github.com/ronceray/StochasticForceInference
 Project-URL: Issues, https://github.com/ronceray/StochasticForceInference/issues
 Project-URL: Changelog, https://github.com/ronceray/StochasticForceInference/blob/main/CHANGELOG.md
-Keywords: stochastic force inference,Langevin dynamics,stochastic differential equations,diffusion,inference,sparse model selection,active matter,biophysics,JAX
+Keywords: stochastic force inference,Langevin dynamics,stochastic differential equations,SDE,diffusion,inference,drift estimation,diffusion estimation,Kramers-Moyal,Fokker-Planck,time series,trajectory analysis,system identification,equation discovery,sparse model selection,particle tracking,single-particle tracking,cell migration,population dynamics,gene expression,entropy production,Ornstein-Uhlenbeck,active matter,biophysics,JAX
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: MIT License
@@ -20,13 +22,14 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Scientific/Engineering :: Physics
 Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: scipy
-Requires-Dist: jax>=0.10
-Requires-Dist: jaxlib
+Requires-Dist: jax!=0.10.2,!=0.11.0,>=0.10.1
+Requires-Dist: jaxlib!=0.10.2,!=0.11.0,>=0.10.1
 Requires-Dist: pandas
 Requires-Dist: matplotlib
 Requires-Dist: pyyaml
@@ -42,6 +45,7 @@ Requires-Dist: sphinxcontrib-video; extra == "docs"
 Requires-Dist: sphinxcontrib-mermaid; extra == "docs"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-xdist; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
 Requires-Dist: pyright; extra == "dev"
 Requires-Dist: StochasticForceInference[docs]; extra == "dev"
@@ -57,14 +61,38 @@ Dynamic: license-file
 [![Documentation](https://readthedocs.org/projects/stochasticforceinference/badge/?version=latest)](https://stochasticforceinference.readthedocs.io/en/latest/)
 [![PyPI](https://img.shields.io/pypi/v/StochasticForceInference.svg)](https://pypi.org/project/StochasticForceInference/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/ronceray/StochasticForceInference/blob/main/LICENSE)
-
-**Infer force and diffusion fields from stochastic trajectory data.**
-
-SFI is a JAX-based Python package for learning the drift (force) and diffusion
-of Langevin stochastic differential equations from time-series observations.
-It handles both overdamped and underdamped dynamics, supports interacting
-particles and spatially-extended (SPDE) systems, and provides built-in
-diagnostics, sparse model selection, and bootstrapped validation.
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21532842.svg)](https://doi.org/10.5281/zenodo.21532842)
+
+**Learn the stochastic equation behind your time series.**
+
+SFI is a JAX-based Python package that infers the drift (force) and
+diffusion of Langevin stochastic differential equations from trajectories
+and other continuous time series: tracked particles, cells and animals,
+population abundances, gene-expression and other molecular levels,
+concentration fields. It returns an interpretable
+equation with error bars, selects the terms the data supports, and
+simulates the fitted model. It handles overdamped and underdamped
+dynamics, interacting particles and spatially extended (SPDE) systems.
+
+## Is SFI for my data?
+
+| You want to… | SFI does it with |
+| --- | --- |
+| estimate drift and diffusion (first and second Kramers–Moyal coefficients, a Fokker–Planck model) | `OverdampedLangevinInference`: `infer_force_linear`, `infer_diffusion_linear` |
+| discover the equation of a noisy stochastic system (sparse regression, "stochastic SINDy") | a candidate basis + `sparsify_force(criterion="PASTIS")` |
+| map forces and diffusivity from single-particle tracking, with localization error | the overdamped engine; the defaults correct white localization noise |
+| model cell migration, animal movement or active particles (persistent velocities) | `UnderdampedLangevinInference`, from positions alone |
+| measure interaction forces between particles, collective motion | pair bases (`SFI.bases.pairs`) |
+| model population dynamics: microbial or ecological abundances, growth rates, species interactions | the overdamped engine on log-abundances + `sparsify_force` ([Lotka–Volterra example](https://stochasticforceinference.readthedocs.io/en/stable/gallery/lotka_volterra_demo.html)) |
+| model gene-expression, protein or metabolite levels, whose noise scales with the level | the overdamped engine with a diffusion field `D(x)` (`infer_diffusion_linear`) |
+| fit noisy oscillators and chaotic signals (genetic or circadian clocks, hair bundles) | a nonlinear basis; `UnderdampedLangevinInference` when one coordinate of an inertial oscillator is observed |
+| quantify irreversibility: entropy production, broken detailed balance | `compute_entropy_production()` |
+| decide whether the dynamics are overdamped or underdamped | `SFI.classify_dynamics` |
+
+SFI is not the right tool for categorical data, small counts or events, for
+forecasting without an equation, for snapshots without time correlation,
+or for processes with jumps. More routes:
+[Start here](https://stochasticforceinference.readthedocs.io/en/stable/start_here.html).
 
 > **Designed for experimental data.**  SFI is built for real experimental
 > trajectories (tracked particles, cells, organisms, …) where *no dynamical
@@ -78,10 +106,11 @@ diagnostics, sparse model selection, and bootstrapped validation.
 ## Key features
 
 - **Two estimator families** — closed-form linear estimators
-  (`infer_force_linear`, seconds even on large datasets) and parametric
-  estimators (`infer_force` / `infer_diffusion`: noise-aware likelihood
-  fit, robust to localization error and coarse sampling, accepts
-  nonlinear models such as neural-network drifts).
+  (`infer_force_linear`, seconds even on large datasets; the defaults
+  correct localization noise and the leading finite-time-step bias) and
+  parametric estimators (`infer_force` / `infer_diffusion`: noise-aware
+  likelihood fit, robust to localization error and coarse sampling,
+  accepts nonlinear models such as neural-network drifts).
 - **Overdamped & underdamped inference** — works with position-only data;
   velocities are reconstructed automatically for underdamped systems.
 - **Composable state functions** — build force/diffusion models from
@@ -89,18 +118,21 @@ diagnostics, sparse model selection, and bootstrapped validation.
   parametric families, all with automatic differentiation via JAX.
 - **Sparse model selection** — Pareto-front beam search with AIC / BIC /
   PASTIS information criteria.
-- **Simulation** — simulate Langevin SDEs (Euler–Maruyama) from the same
-  model objects used for inference.
+- **Entropy production** — dissipation rate with error bar, for
+  overdamped and underdamped dynamics.
+- **Simulation** — simulate Langevin SDEs from the same model objects
+  used for inference, including bootstrapped trajectories of a fitted model.
 - **Trajectory toolkit** — mask-aware increments, synthetic degradation
   (noise, downsampling, data loss, motion blur), streaming for large
-  datasets, I/O (CSV / Parquet / HDF5).
-- **Diagnostics** — compare inferred fields to exact models, compute
-  normalized errors, generate bootstrapped trajectories.
+  datasets, I/O (CSV / Parquet / HDF5, tracking tables).
+- **Diagnostics** — residual tests with action hints, an
+  overdamped-vs-underdamped classifier, predicted errors, comparison to
+  exact models.
 
 ## Installation
 
 ```bash
-pip install StochasticForceInference
+pip install StochasticForceInference      # import name: SFI
 ```
 
 For development (editable install with test/doc dependencies):
@@ -129,12 +161,13 @@ coll = TrajectoryCollection.from_arrays(X=positions, dt=0.01)
 # Infer using a polynomial basis — no model needed
 B = monomials_up_to(order=3, dim=2, rank="vector")
 inf = OverdampedLangevinInference(coll)
-inf.compute_diffusion_constant()
-inf.infer_force_linear(B)
+inf.infer_force_linear(B)               # also estimates D and the measurement noise
 inf.sparsify_force(criterion="PASTIS")  # optional: find the minimal model
-inf.print_report()
+inf.compute_force_error()
+inf.print_report()                      # coefficients with error bars
+inf.diagnose().print_summary()          # residual tests, with hints if one fails
 
-# Noisy or coarsely-sampled recordings? Use the parametric estimator
+# Coarsely sampled or very noisy recordings? Use the parametric estimator
 # instead: inf.infer_force(B) — it models the measurement noise natively.
 ```
 
@@ -157,7 +190,6 @@ coll = proc.simulate(dt=0.01, Nsteps=10_000, key=random.PRNGKey(0))
 # 2. Infer from the trajectory using a generic polynomial basis
 B = monomials_up_to(order=2, dim=2, rank="vector")
 inf = OverdampedLangevinInference(coll)
-inf.compute_diffusion_constant()
 inf.infer_force_linear(B)
 inf.compute_force_error()
 
@@ -166,11 +198,35 @@ inf.print_report()
 inf.compare_to_exact(model_exact=proc)
 ```
 
+## Using SFI with an AI agent
+
+SFI ships a workflow guide for AI coding agents (Claude Code, Codex,
+Cursor, …) inside the package, matched to the installed version:
+
+```bash
+python -m SFI guide            # Markdown: workflow, decisions, pitfalls, API names
+
+# Install it as an Agent Skill (path shown for Claude Code; other tools that support skills use their own folder)
+mkdir -p ~/.claude/skills/stochastic-force-inference
+python -m SFI guide --skill > ~/.claude/skills/stochastic-force-inference/SKILL.md
+```
+
+The documentation has an [`llms.txt`](https://stochasticforceinference.readthedocs.io/llms.txt)
+index. Agents that modify SFI itself should read
+[`AGENTS.md`](https://github.com/ronceray/StochasticForceInference/blob/main/AGENTS.md).
+
+**Coming from the original SFI scripts** (`StochasticForceInference.py`,
+`StochasticTrajectoryData`, or the separate UnderdampedLangevinInference
+package)? SFI 2 is a rewrite with a new API; the
+[guide](https://github.com/ronceray/StochasticForceInference/blob/main/SFI/AGENT_GUIDE.md#9-coming-from-the-original-sfi-scripts-or-older-example-code)
+maps the old calls to the new ones.
+
 ## Documentation
 
 Full documentation is available at
 [stochasticforceinference.readthedocs.io](https://stochasticforceinference.readthedocs.io), including:
 
+- **[Start here](https://stochasticforceinference.readthedocs.io/en/latest/start_here.html)** — is SFI right for my data, and which route to take
 - **[Tutorial](https://stochasticforceinference.readthedocs.io/en/latest/gallery/ou_demo.html)** — end-to-end walkthrough
 - **[Examples gallery](https://stochasticforceinference.readthedocs.io/en/latest/gallery/index.html)** — worked examples covering sparsity, multi-particle, and SPDE inference
 - **[Model building](https://stochasticforceinference.readthedocs.io/en/latest/statefunc/user_guide.html)** — composable state functions
@@ -180,25 +236,28 @@ Full documentation is available at
 
 ## Package structure
 
-| Subpackage       | Purpose                                              |
-| ---------------- | ---------------------------------------------------- |
-| `SFI.statefunc`  | Composable state functions: `Basis`, `PSF`, `SF`     |
-| `SFI.inference`  | Overdamped & underdamped inference engines            |
-| `SFI.trajectory` | `TrajectoryCollection` / `TrajectoryDataset`          |
-| `SFI.langevin`   | Langevin simulators (`OverdampedProcess`, …)          |
-| `SFI.bases`      | Ready-made basis builders (monomials, constants, …)  |
-| `SFI.integrate`  | Time-averaging integration engine                    |
-| `SFI.utils`      | Math helpers, formatting, plotting                   |
-
-For contributors and AI coding agents: see [`AGENTS.md`](https://github.com/ronceray/StochasticForceInference/blob/main/AGENTS.md) for
-the canonical imports table, task playbooks, and "do not re-implement"
-guidance.
+| Subpackage        | Purpose                                              |
+| ----------------- | ---------------------------------------------------- |
+| `SFI.statefunc`   | Composable state functions: `Basis`, `PSF`, `SF`     |
+| `SFI.inference`   | Overdamped & underdamped inference engines            |
+| `SFI.trajectory`  | `TrajectoryCollection` / `TrajectoryDataset`          |
+| `SFI.langevin`    | Langevin simulators (`OverdampedProcess`, …)          |
+| `SFI.bases`       | Ready-made basis builders (monomials, pairs, SPDE, …) |
+| `SFI.diagnostics` | Residual tests, dynamics classifier, error checks     |
+| `SFI.integrate`   | Time-averaging integration engine                    |
+| `SFI.utils`       | Math helpers, formatting, plotting                   |
 
 ## Citation
 
-If you use SFI, please cite the software (see [`CITATION.cff`](CITATION.cff))
-together with the method paper(s) relevant to the features you use. See
-**[How to cite](https://stochasticforceinference.readthedocs.io/en/latest/#how-to-cite)** in the
+If you use SFI, please cite the software (see [`CITATION.cff`](https://github.com/ronceray/StochasticForceInference/blob/main/CITATION.cff))
+together with the method paper(s) relevant to the features you use.
+
+The software is archived on Zenodo. Cite the concept DOI
+[10.5281/zenodo.21532842](https://doi.org/10.5281/zenodo.21532842), which always
+resolves to the latest release; each version also has its own DOI (v2.0.4:
+[10.5281/zenodo.23498743](https://doi.org/10.5281/zenodo.23498743)).
+
+See **[How to cite](https://stochasticforceinference.readthedocs.io/en/latest/#how-to-cite)** in the
 documentation for the full, feature-specific list (overdamped, underdamped/ULI,
 trapeze, and PASTIS references).
 

stochasticforceinference-2.1.0/README.md

@@ -0,0 +1,218 @@
+# Stochastic Force Inference (SFI)
+
+[![Documentation](https://readthedocs.org/projects/stochasticforceinference/badge/?version=latest)](https://stochasticforceinference.readthedocs.io/en/latest/)
+[![PyPI](https://img.shields.io/pypi/v/StochasticForceInference.svg)](https://pypi.org/project/StochasticForceInference/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/ronceray/StochasticForceInference/blob/main/LICENSE)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21532842.svg)](https://doi.org/10.5281/zenodo.21532842)
+
+**Learn the stochastic equation behind your time series.**
+
+SFI is a JAX-based Python package that infers the drift (force) and
+diffusion of Langevin stochastic differential equations from trajectories
+and other continuous time series: tracked particles, cells and animals,
+population abundances, gene-expression and other molecular levels,
+concentration fields. It returns an interpretable
+equation with error bars, selects the terms the data supports, and
+simulates the fitted model. It handles overdamped and underdamped
+dynamics, interacting particles and spatially extended (SPDE) systems.
+
+## Is SFI for my data?
+
+| You want to… | SFI does it with |
+| --- | --- |
+| estimate drift and diffusion (first and second Kramers–Moyal coefficients, a Fokker–Planck model) | `OverdampedLangevinInference`: `infer_force_linear`, `infer_diffusion_linear` |
+| discover the equation of a noisy stochastic system (sparse regression, "stochastic SINDy") | a candidate basis + `sparsify_force(criterion="PASTIS")` |
+| map forces and diffusivity from single-particle tracking, with localization error | the overdamped engine; the defaults correct white localization noise |
+| model cell migration, animal movement or active particles (persistent velocities) | `UnderdampedLangevinInference`, from positions alone |
+| measure interaction forces between particles, collective motion | pair bases (`SFI.bases.pairs`) |
+| model population dynamics: microbial or ecological abundances, growth rates, species interactions | the overdamped engine on log-abundances + `sparsify_force` ([Lotka–Volterra example](https://stochasticforceinference.readthedocs.io/en/stable/gallery/lotka_volterra_demo.html)) |
+| model gene-expression, protein or metabolite levels, whose noise scales with the level | the overdamped engine with a diffusion field `D(x)` (`infer_diffusion_linear`) |
+| fit noisy oscillators and chaotic signals (genetic or circadian clocks, hair bundles) | a nonlinear basis; `UnderdampedLangevinInference` when one coordinate of an inertial oscillator is observed |
+| quantify irreversibility: entropy production, broken detailed balance | `compute_entropy_production()` |
+| decide whether the dynamics are overdamped or underdamped | `SFI.classify_dynamics` |
+
+SFI is not the right tool for categorical data, small counts or events, for
+forecasting without an equation, for snapshots without time correlation,
+or for processes with jumps. More routes:
+[Start here](https://stochasticforceinference.readthedocs.io/en/stable/start_here.html).
+
+> **Designed for experimental data.**  SFI is built for real experimental
+> trajectories (tracked particles, cells, organisms, …) where *no dynamical
+> model pre-exists*.  Two first-class estimator families share one API:
+> fast closed-form **linear estimators** that require no initial guess, and
+> **parametric estimators** that model measurement noise and finite sampling
+> explicitly — robust where real data is hard.  The PASTIS information
+> criterion rigorously identifies which terms the data actually supports.
+> Synthetic examples in the gallery serve as runnable demonstrations.
+
+## Key features
+
+- **Two estimator families** — closed-form linear estimators
+  (`infer_force_linear`, seconds even on large datasets; the defaults
+  correct localization noise and the leading finite-time-step bias) and
+  parametric estimators (`infer_force` / `infer_diffusion`: noise-aware
+  likelihood fit, robust to localization error and coarse sampling,
+  accepts nonlinear models such as neural-network drifts).
+- **Overdamped & underdamped inference** — works with position-only data;
+  velocities are reconstructed automatically for underdamped systems.
+- **Composable state functions** — build force/diffusion models from
+  monomials, custom basis functions, pair interactions, or arbitrary
+  parametric families, all with automatic differentiation via JAX.
+- **Sparse model selection** — Pareto-front beam search with AIC / BIC /
+  PASTIS information criteria.
+- **Entropy production** — dissipation rate with error bar, for
+  overdamped and underdamped dynamics.
+- **Simulation** — simulate Langevin SDEs from the same model objects
+  used for inference, including bootstrapped trajectories of a fitted model.
+- **Trajectory toolkit** — mask-aware increments, synthetic degradation
+  (noise, downsampling, data loss, motion blur), streaming for large
+  datasets, I/O (CSV / Parquet / HDF5, tracking tables).
+- **Diagnostics** — residual tests with action hints, an
+  overdamped-vs-underdamped classifier, predicted errors, comparison to
+  exact models.
+
+## Installation
+
+```bash
+pip install StochasticForceInference      # import name: SFI
+```
+
+For development (editable install with test/doc dependencies):
+
+```bash
+git clone https://github.com/ronceray/StochasticForceInference.git
+cd StochasticForceInference
+pip install -e ".[dev,io]"
+```
+
+> **Note:** SFI requires Python ≥ 3.11 and JAX ≥ 0.10.
+
+## Quick start
+
+For **experimental data**, load your trajectories and infer directly:
+
+```python
+import numpy as np
+from SFI import OverdampedLangevinInference, TrajectoryCollection
+from SFI.bases import monomials_up_to
+
+# Load your tracked data (positions: T×d array, dt: time step)
+positions = np.load("my_experiment.npz")["positions"]  # shape (T, d)
+coll = TrajectoryCollection.from_arrays(X=positions, dt=0.01)
+
+# Infer using a polynomial basis — no model needed
+B = monomials_up_to(order=3, dim=2, rank="vector")
+inf = OverdampedLangevinInference(coll)
+inf.infer_force_linear(B)               # also estimates D and the measurement noise
+inf.sparsify_force(criterion="PASTIS")  # optional: find the minimal model
+inf.compute_force_error()
+inf.print_report()                      # coefficients with error bars
+inf.diagnose().print_summary()          # residual tests, with hints if one fails
+
+# Coarsely sampled or very noisy recordings? Use the parametric estimator
+# instead: inf.infer_force(B) — it models the measurement noise natively.
+```
+
+For a **synthetic example** (useful for validation):
+
+```python
+import jax.numpy as jnp
+from jax import random
+
+from SFI import OverdampedLangevinInference
+from SFI.bases import X, monomials_up_to
+from SFI.langevin import OverdampedProcess
+
+# 1. Build a force model (Ornstein–Uhlenbeck: F = -k x) and simulate a trajectory.
+#    X(dim) is the identity basis x ↦ x; theta_F holds its coefficient, so F(x) = -x.
+proc = OverdampedProcess(F=X(dim=2), D=jnp.eye(2) * 0.5, theta_F=jnp.array([-1.0]))
+proc.initialize(jnp.zeros(2))
+coll = proc.simulate(dt=0.01, Nsteps=10_000, key=random.PRNGKey(0))
+
+# 2. Infer from the trajectory using a generic polynomial basis
+B = monomials_up_to(order=2, dim=2, rank="vector")
+inf = OverdampedLangevinInference(coll)
+inf.infer_force_linear(B)
+inf.compute_force_error()
+
+# 3. Inspect results and validate against the known model
+inf.print_report()
+inf.compare_to_exact(model_exact=proc)
+```
+
+## Using SFI with an AI agent
+
+SFI ships a workflow guide for AI coding agents (Claude Code, Codex,
+Cursor, …) inside the package, matched to the installed version:
+
+```bash
+python -m SFI guide            # Markdown: workflow, decisions, pitfalls, API names
+
+# Install it as an Agent Skill (path shown for Claude Code; other tools that support skills use their own folder)
+mkdir -p ~/.claude/skills/stochastic-force-inference
+python -m SFI guide --skill > ~/.claude/skills/stochastic-force-inference/SKILL.md
+```
+
+The documentation has an [`llms.txt`](https://stochasticforceinference.readthedocs.io/llms.txt)
+index. Agents that modify SFI itself should read
+[`AGENTS.md`](https://github.com/ronceray/StochasticForceInference/blob/main/AGENTS.md).
+
+**Coming from the original SFI scripts** (`StochasticForceInference.py`,
+`StochasticTrajectoryData`, or the separate UnderdampedLangevinInference
+package)? SFI 2 is a rewrite with a new API; the
+[guide](https://github.com/ronceray/StochasticForceInference/blob/main/SFI/AGENT_GUIDE.md#9-coming-from-the-original-sfi-scripts-or-older-example-code)
+maps the old calls to the new ones.
+
+## Documentation
+
+Full documentation is available at
+[stochasticforceinference.readthedocs.io](https://stochasticforceinference.readthedocs.io), including:
+
+- **[Start here](https://stochasticforceinference.readthedocs.io/en/latest/start_here.html)** — is SFI right for my data, and which route to take
+- **[Tutorial](https://stochasticforceinference.readthedocs.io/en/latest/gallery/ou_demo.html)** — end-to-end walkthrough
+- **[Examples gallery](https://stochasticforceinference.readthedocs.io/en/latest/gallery/index.html)** — worked examples covering sparsity, multi-particle, and SPDE inference
+- **[Model building](https://stochasticforceinference.readthedocs.io/en/latest/statefunc/user_guide.html)** — composable state functions
+- **[Trajectory handling](https://stochasticforceinference.readthedocs.io/en/latest/trajectory/user_guide.html)** — data ingestion, masking, degradation
+- **[Running inference](https://stochasticforceinference.readthedocs.io/en/latest/inference/user_guide.html)** — estimator choice, overdamped/underdamped engines
+- **[API reference](https://stochasticforceinference.readthedocs.io/en/latest/api_frontend.html)** — full autodoc
+
+## Package structure
+
+| Subpackage        | Purpose                                              |
+| ----------------- | ---------------------------------------------------- |
+| `SFI.statefunc`   | Composable state functions: `Basis`, `PSF`, `SF`     |
+| `SFI.inference`   | Overdamped & underdamped inference engines            |
+| `SFI.trajectory`  | `TrajectoryCollection` / `TrajectoryDataset`          |
+| `SFI.langevin`    | Langevin simulators (`OverdampedProcess`, …)          |
+| `SFI.bases`       | Ready-made basis builders (monomials, pairs, SPDE, …) |
+| `SFI.diagnostics` | Residual tests, dynamics classifier, error checks     |
+| `SFI.integrate`   | Time-averaging integration engine                    |
+| `SFI.utils`       | Math helpers, formatting, plotting                   |
+
+## Citation
+
+If you use SFI, please cite the software (see [`CITATION.cff`](https://github.com/ronceray/StochasticForceInference/blob/main/CITATION.cff))
+together with the method paper(s) relevant to the features you use.
+
+The software is archived on Zenodo. Cite the concept DOI
+[10.5281/zenodo.21532842](https://doi.org/10.5281/zenodo.21532842), which always
+resolves to the latest release; each version also has its own DOI (v2.0.4:
+[10.5281/zenodo.23498743](https://doi.org/10.5281/zenodo.23498743)).
+
+See **[How to cite](https://stochasticforceinference.readthedocs.io/en/latest/#how-to-cite)** in the
+documentation for the full, feature-specific list (overdamped, underdamped/ULI,
+trapeze, and PASTIS references).
+
+## Funding
+
+Funded by the European Union (ERC, SuperStoc, 101117322). Views and opinions
+expressed are however those of the author(s) only and do not necessarily
+reflect those of the European Union or the European Research Council Executive
+Agency. Neither the European Union nor the granting authority can be held
+responsible for them. P. R. also acknowledges funding from the French National
+Research Agency (ANR-16-CONV-0001) and from the Excellence Initiative of
+Aix-Marseille University A\*MIDEX.
+
+## License
+
+MIT — see [LICENSE](https://github.com/ronceray/StochasticForceInference/blob/main/LICENSE) for details.