electropycal 0.9.0__tar.gz

electropycal-0.9.0/.gitignore

@@ -0,0 +1,43 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+env/
+.python-version
+
+# Testing / coverage
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+.mypy_cache/
+.ruff_cache/
+
+# Notebooks
+.ipynb_checkpoints/
+
+# OS / editor
+.DS_Store
+*.swp
+.idea/
+.vscode/
+
+# Run outputs. `discover`, `freeze` and `deploy` write here by default.
+outputs/
+
+# The featureset the docs build in their worked example.
+featureset_extracted.parquet
+
+# Artifacts the notebooks write back into the data root when you run them: the
+# hash-keyed featureset caches and the QC-stats companion. Regenerated on demand.
+# (electropycal_analysis_config.json is NOT here: it is shipped demo content.)
+*_featureset__*.parquet
+electropycal_qc_stats.json
+
+# NOTE: demo/ is SHIPPED content (the synthetic demo dataset built by
+# scripts/build_demo_dataset.py) and is deliberately NOT ignored.

electropycal-0.9.0/CHANGELOG.md

@@ -0,0 +1,56 @@
+# Changelog
+
+This project follows [semantic versioning](https://semver.org/). It is pre-1.0, which here
+means what semver says it means: **the public API may change in a minor release.** Pin a
+version if you depend on it. 1.0.0 will be a deliberate act, once the surface has stopped
+moving, not a scheduled follow-up.
+
+## 0.9.0
+
+First public release.
+
+### What ships
+
+- **Ingestion.** `data.pstrace` parses PSTrace UTF-16 exports. `features.extract` walks a raw
+  directory, derives each device's timepoints as elapsed days since its own first session,
+  quality-gates, and builds a featureset.
+- **Discovery.** EIS/FSCV feature extraction, D0/Z normalization, forward-chained nested CV
+  across three tracks, five PLSR architectures, six feature selectors, and a decision-gated
+  task-queue scheduler with optional fold-parallelism.
+- **Deployment.** Freeze a model on the full in-vitro set, then recalibrate new in-vivo data
+  with frozen robust scalers and a CORAL domain-shift confidence flag. Bundles are portable
+  `.npz` plus a manifest, with no pickle anywhere.
+- **Diagnostics.** Variance partitioning and measurement-reliability estimates.
+- **A synthetic demo dataset** under `demo/`, so every command and all seven notebooks run
+  before you have data of your own.
+- **Seven notebooks**, each runnable as-is, which synthesize a demo tree when none is
+  reachable (as on Colab).
+
+`discovery` and `deployment` are included rather than held back for a later release. They are
+import-load-bearing for the CLI, three of the seven notebooks depend on them, and holding them
+back would have meant shipping a library that could not do the thing it exists to do.
+
+### Deliberate design choices worth knowing
+
+- **`extract_dataset` has no default for `band`** and raises without one. `"auto"` is a
+  percentile over whatever corpus is present, so it cannot be reproduced over a subset. The
+  CLI does default to `--band auto` and prints the band it resolved, so the choice is never
+  silent.
+- **Dependencies are bounded at both ends, and so is Python** (`>=3.11,<3.14`). A scikit-learn
+  minor release can shift PLS numerics enough to move a reported RMSEP, and on 3.10 the
+  declared ranges resolved to a stack years apart from the reference one.
+- **Results are hypothesis-generating.** Every ranking carries a bootstrap interval, and the
+  baseline queue is a one-factor-at-a-time design rather than a grid, because selecting among
+  many conditions on few folds overfits the selection itself.
+- **Extraction is reproducible through a pin.** `run_config.json` records every derived
+  run-wide parameter, so a later extraction over a staged subset either reproduces the
+  original rows or raises, rather than silently producing different ones.
+
+### Known limits
+
+- Plotting displays inline in notebooks but **writes no files**: there are no `savefig` calls,
+  so running the review modules from a script or in CI completes and produces nothing.
+- `Kd` is unidentifiable on non-saturating data, so the saturating targets (`sat_*`, `hill_*`)
+  are off by default.
+- Kernel, multi-block and multi-level PLSR, and the mRMR and permutation-`t_max` selectors,
+  are registered extension points that raise on use.

electropycal-0.9.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 thirteenbillion
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

electropycal-0.9.0/PKG-INFO

@@ -0,0 +1,156 @@
+Metadata-Version: 2.5
+Name: electropycal
+Version: 0.9.0
+Summary: Data processing and recalibration for electrochemical sensors (EIS/FSCV).
+Project-URL: Homepage, https://github.com/thirteenbillion/electropycal
+Project-URL: Repository, https://github.com/thirteenbillion/electropycal
+Project-URL: Documentation, https://github.com/thirteenbillion/electropycal/blob/main/docs/USAGE.md
+Project-URL: Changelog, https://github.com/thirteenbillion/electropycal/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/thirteenbillion/electropycal/issues
+Author: thirteenbillion
+License-Expression: MIT
+License-File: LICENSE
+Keywords: EIS,FSCV,PLSR,electrochemistry,sensor recalibration
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Chemistry
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Typing :: Typed
+Requires-Python: <3.14,>=3.11
+Requires-Dist: joblib<2,>=1.3
+Requires-Dist: numpy<3,>=1.24
+Requires-Dist: pandas<4,>=2.0
+Requires-Dist: pyarrow>=12.0
+Requires-Dist: scikit-learn<1.10,>=1.3
+Requires-Dist: scipy<2,>=1.10
+Provides-Extra: dev
+Requires-Dist: cycler; extra == 'dev'
+Requires-Dist: matplotlib; extra == 'dev'
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: plotly; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest>=7.4; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Provides-Extra: notebooks
+Requires-Dist: cycler; extra == 'notebooks'
+Requires-Dist: jupyter; extra == 'notebooks'
+Requires-Dist: matplotlib; extra == 'notebooks'
+Requires-Dist: plotly; extra == 'notebooks'
+Requires-Dist: pyarrow; extra == 'notebooks'
+Requires-Dist: seaborn; extra == 'notebooks'
+Description-Content-Type: text/markdown
+
+# ElectroPyCal
+
+Data processing and recalibration for electrochemical sensors (EIS / FSCV).
+
+```bash
+pip install electropycal
+```
+
+**ElectroPyCal** discovers interpretable PLSR recalibration models for drifting
+electrochemical sensors under leakage-free, forward-chained nested cross-validation, and
+deploys a frozen model to new data while quantifying in-vitro to in-vivo domain shift
+(CORAL). It originates from a neurostring dopamine-sensor recalibration study but is
+designed to generalize to electrochemical-sensor data processing broadly.
+
+Pre-1.0: the public API is still moving. Pin a version if you depend on it.
+
+## What it does
+
+**Ingestion.** `data.pstrace` parses PSTrace UTF-16 exports. `features.extract` walks a raw
+directory (`<YYYYMMDD>_<devicetype>_signal/<deviceid>_<signaltype>_<dose>.csv`, where `dose`
+is a concentration or a protocol token such as `stabilization`), derives each device's
+timepoints as elapsed days since its own first session, quality-gates, and builds a
+featureset.
+
+**Discovery.** EIS/FSCV feature extraction, D0/Z normalization, forward-chained nested CV
+(Tracks 1/2/3), five architectures (linear, log, weighted, orthogonal, nonlinear PLSR), six
+selectors (CARS, VIP, SR, sMC, MI, ICC), a decision-gated task-queue scheduler with optional
+joblib fold-parallelism, and a checkpointed output layout (per-fold bundles,
+`summary.parquet`, a ranked `report/`).
+
+**Deployment.** Freeze a chosen model on 100% of in-vitro data, then recalibrate new in-vivo
+data with frozen robust (median/IQR) scalers and a CORAL domain-shift confidence flag.
+In-vivo raw ingestion (`paired` / `baseline` / `live` exports) feeds per-session
+recalibration through `recalibrate_invivo` or `deploy --raw`. No pickle; bundles are portable
+`.npz` plus a manifest.
+
+**Diagnostics.** Variance partitioning across structural, drift and dose components, and
+measurement-reliability estimates for the response.
+
+Kernel, multi-block and multi-level PLSR, and the mRMR and permutation-`t_max` selectors, are
+registered extension points that raise on use. See `docs/DESIGN.md`.
+
+## Quick start
+
+```bash
+pip install electropycal
+
+# Build a featureset from a raw PSTrace export directory.
+electropycal extract --raw <pstrace_dir> --out featureset.parquet
+
+# Rank candidate recalibration models.
+electropycal discover --data featureset.parquet --profile full
+
+# Recalibrate new in-vivo data with a frozen model.
+electropycal deploy --model outputs/frozen_model --data invivo.parquet
+```
+
+A synthetic demo dataset ships with the package under `demo/`, so every command and notebook
+runs before you have data of your own:
+
+```bash
+electropycal extract --raw demo/in_vitro/input --out featureset.parquet
+```
+
+### Two things worth knowing up front
+
+The CLI's EIS analysis `--band` defaults to `auto`, a data-driven upper bound just below the
+tightest inductive onset, and the resolved band is printed so it is never a silent choice. The
+**Python API deliberately has no `band` default**: `extract_dataset(root)` raises. `auto` is a
+percentile over whatever corpus is present, so it cannot be reproduced over a staged subset.
+Pass a `(lo, hi)` tuple, `band="auto"`, or `pin=` to reuse a previous run's band.
+
+`discover` and `run_discovery` cap `min_train_times` to what the number of timepoints can
+support, so a short series yields usable folds rather than silently empty ones.
+
+## Notebooks
+
+In `notebooks/`, descriptive names, runnable in any order. Each one runs as-is against the
+shipped `demo/` tree, and synthesizes an equivalent tree if none is present (Colab).
+
+| Notebook | Purpose |
+|---|---|
+| `raw_spectra_review` | inspect raw EIS/FSCV per device; recommends the analysis band. Run first. |
+| `quality_filtering_dashboard` | per-device quality gating and channel-timepoint validity |
+| `stabilization_review` | confirm FSCV stabilized, from the cycle-to-cycle drift |
+| `diagnostics_review` | variance hierarchy, response reliability, QC-gate impact. Run between QC and discovery. |
+| `discovery_checkpointed` | run discovery batch by batch with CI-ranked review between batches |
+| `discovery_results_review` | review a completed run: ranking, CIs, per-fold spread, feature ranking |
+| `deployment_domain_shift` | freeze a model, recalibrate drifted data, CORAL drift monitor |
+
+## Docs
+
+- **`docs/USAGE.md`**: the user guide. Installation and a full discovery walkthrough, start to finish.
+- **`docs/REFERENCE.md`**: the lookup tables. Every default, the feature dictionary, directory layouts, and the API.
+- **`docs/DESIGN.md`**: architecture, assumptions, and the reasoning behind the defaults.
+- **`docs/RUNNING_AT_SCALE.md`**: extract once to parquet, then cluster workflows for large raw data.
+
+## Layout
+
+```
+src/electropycal/{data,features,models,selection,evaluation,discovery,deployment,diagnostics}
+notebooks/   # the seven review and workflow notebooks listed above
+demo/        # synthetic in-vitro and in-vivo trees, input and output
+docs/        # USAGE.md, REFERENCE.md, DESIGN.md, RUNNING_AT_SCALE.md
+```
+
+## License
+
+MIT. See `LICENSE`.

electropycal-0.9.0/README.md

@@ -0,0 +1,109 @@
+# ElectroPyCal
+
+Data processing and recalibration for electrochemical sensors (EIS / FSCV).
+
+```bash
+pip install electropycal
+```
+
+**ElectroPyCal** discovers interpretable PLSR recalibration models for drifting
+electrochemical sensors under leakage-free, forward-chained nested cross-validation, and
+deploys a frozen model to new data while quantifying in-vitro to in-vivo domain shift
+(CORAL). It originates from a neurostring dopamine-sensor recalibration study but is
+designed to generalize to electrochemical-sensor data processing broadly.
+
+Pre-1.0: the public API is still moving. Pin a version if you depend on it.
+
+## What it does
+
+**Ingestion.** `data.pstrace` parses PSTrace UTF-16 exports. `features.extract` walks a raw
+directory (`<YYYYMMDD>_<devicetype>_signal/<deviceid>_<signaltype>_<dose>.csv`, where `dose`
+is a concentration or a protocol token such as `stabilization`), derives each device's
+timepoints as elapsed days since its own first session, quality-gates, and builds a
+featureset.
+
+**Discovery.** EIS/FSCV feature extraction, D0/Z normalization, forward-chained nested CV
+(Tracks 1/2/3), five architectures (linear, log, weighted, orthogonal, nonlinear PLSR), six
+selectors (CARS, VIP, SR, sMC, MI, ICC), a decision-gated task-queue scheduler with optional
+joblib fold-parallelism, and a checkpointed output layout (per-fold bundles,
+`summary.parquet`, a ranked `report/`).
+
+**Deployment.** Freeze a chosen model on 100% of in-vitro data, then recalibrate new in-vivo
+data with frozen robust (median/IQR) scalers and a CORAL domain-shift confidence flag.
+In-vivo raw ingestion (`paired` / `baseline` / `live` exports) feeds per-session
+recalibration through `recalibrate_invivo` or `deploy --raw`. No pickle; bundles are portable
+`.npz` plus a manifest.
+
+**Diagnostics.** Variance partitioning across structural, drift and dose components, and
+measurement-reliability estimates for the response.
+
+Kernel, multi-block and multi-level PLSR, and the mRMR and permutation-`t_max` selectors, are
+registered extension points that raise on use. See `docs/DESIGN.md`.
+
+## Quick start
+
+```bash
+pip install electropycal
+
+# Build a featureset from a raw PSTrace export directory.
+electropycal extract --raw <pstrace_dir> --out featureset.parquet
+
+# Rank candidate recalibration models.
+electropycal discover --data featureset.parquet --profile full
+
+# Recalibrate new in-vivo data with a frozen model.
+electropycal deploy --model outputs/frozen_model --data invivo.parquet
+```
+
+A synthetic demo dataset ships with the package under `demo/`, so every command and notebook
+runs before you have data of your own:
+
+```bash
+electropycal extract --raw demo/in_vitro/input --out featureset.parquet
+```
+
+### Two things worth knowing up front
+
+The CLI's EIS analysis `--band` defaults to `auto`, a data-driven upper bound just below the
+tightest inductive onset, and the resolved band is printed so it is never a silent choice. The
+**Python API deliberately has no `band` default**: `extract_dataset(root)` raises. `auto` is a
+percentile over whatever corpus is present, so it cannot be reproduced over a staged subset.
+Pass a `(lo, hi)` tuple, `band="auto"`, or `pin=` to reuse a previous run's band.
+
+`discover` and `run_discovery` cap `min_train_times` to what the number of timepoints can
+support, so a short series yields usable folds rather than silently empty ones.
+
+## Notebooks
+
+In `notebooks/`, descriptive names, runnable in any order. Each one runs as-is against the
+shipped `demo/` tree, and synthesizes an equivalent tree if none is present (Colab).
+
+| Notebook | Purpose |
+|---|---|
+| `raw_spectra_review` | inspect raw EIS/FSCV per device; recommends the analysis band. Run first. |
+| `quality_filtering_dashboard` | per-device quality gating and channel-timepoint validity |
+| `stabilization_review` | confirm FSCV stabilized, from the cycle-to-cycle drift |
+| `diagnostics_review` | variance hierarchy, response reliability, QC-gate impact. Run between QC and discovery. |
+| `discovery_checkpointed` | run discovery batch by batch with CI-ranked review between batches |
+| `discovery_results_review` | review a completed run: ranking, CIs, per-fold spread, feature ranking |
+| `deployment_domain_shift` | freeze a model, recalibrate drifted data, CORAL drift monitor |
+
+## Docs
+
+- **`docs/USAGE.md`**: the user guide. Installation and a full discovery walkthrough, start to finish.
+- **`docs/REFERENCE.md`**: the lookup tables. Every default, the feature dictionary, directory layouts, and the API.
+- **`docs/DESIGN.md`**: architecture, assumptions, and the reasoning behind the defaults.
+- **`docs/RUNNING_AT_SCALE.md`**: extract once to parquet, then cluster workflows for large raw data.
+
+## Layout
+
+```
+src/electropycal/{data,features,models,selection,evaluation,discovery,deployment,diagnostics}
+notebooks/   # the seven review and workflow notebooks listed above
+demo/        # synthetic in-vitro and in-vivo trees, input and output
+docs/        # USAGE.md, REFERENCE.md, DESIGN.md, RUNNING_AT_SCALE.md
+```
+
+## License
+
+MIT. See `LICENSE`.

electropycal-0.9.0/demo/README.md

@@ -0,0 +1,24 @@
+# Synthetic demo dataset
+
+Generated by `python scripts/build_demo_dataset.py`. Everything here is synthetic; there are no
+real measurements. The generator is calibrated to statistics measured from real in-vitro
+neurostring sessions, so the traces and extracted features sit in realistic ranges (peak V_ox
+around 0.8 V, background around 30 uA, NormIpeak around 0.02 rising sub-linearly with dose,
+noise_floor around 0.015).
+
+| Path | What |
+|------|------|
+| `in_vitro/input/` | raw PSTrace exports: `*_channeltest` (0 nM screen) and `*_signal` (EIS, dose series, stabilization) folders, one per timepoint |
+| `in_vitro/output/` | what the pipeline produces from that input: the extracted featureset, the per-sensor calibration targets, the analysis config, and a manifest |
+| `in_vivo/input/` | paired FSCV+EIS exports per session (`_paired_baseline`, `_paired_live`) for the deployment and domain-shift demo |
+| `in_vivo/output/` | the in-vivo featureset extracted from it |
+
+The notebooks find this tree on their own: leave `ROOT` at `None` and run top to bottom. To
+point at it explicitly, use `demo/in_vitro/input`, or `demo/in_vivo/input` for
+`deployment_domain_shift`.
+
+Stabilization files use the `-full` spelling (6 rounds of 6 cycles), matching what current
+sessions produce. The generator can also emit the older `-start` and `-end` pair; see
+`write_synthetic_pstrace_dir(stab_phases=...)`.
+
+Regenerate at any time. The generator is seeded, so the output is reproducible.

electropycal-0.9.0/demo/in_vitro/input/electropycal_analysis_config.json

@@ -0,0 +1,19 @@
+{
+  "band": [
+    2.0,
+    2000.0
+  ],
+  "peak_method": "direct",
+  "acceptance": "monotonic",
+  "mono_tol": 0.1,
+  "min_norm_snr": 3.0,
+  "monotonic_r_min": 0.6,
+  "mono_method": "pearson",
+  "max_reps": 3,
+  "gate_on": {
+    "eis": true,
+    "monotonic": true,
+    "snr_all": false,
+    "peak_in_window": false
+  }
+}