dfc-kit 1.0.0__tar.gz

dfc_kit-1.0.0/CHANGELOG.md

@@ -0,0 +1,22 @@
+# Changelog
+
+## 1.0.0 - 2026-08-24
+
+- Moved MI/CMI estimators and fixed-window orchestration into
+  `dfckit.information`.
+- Unified random sampling and frozen schedules through one fixed-window
+  estimation path.
+- Simplified state-count selection to subject-disjoint held-out score
+  comparison with compact JSON output.
+- Removed cryptographic data/model identities, workflow-directory loaders,
+  nested checkpointing, and legacy compatibility layers from the development
+  codebase.
+- Kept model, prediction, alignment, and information artifacts pickle-free and
+  validated by explicit shapes, parameters, feature keys, and participant
+  metadata.
+- ETS and MTD share the same segment-safe instantaneous-edge framework while
+  retaining their distinct sample generators.
+- Paired NBS, HC3 inference, information estimates, and all censor-gap-safe
+  connectivity methods retain their numerical APIs.
+- FeatureStore summaries support streaming acquisition-level mean, population
+  variance, standard deviation, minimum, and maximum endpoints.

dfc_kit-1.0.0/CITATION.cff

@@ -0,0 +1,9 @@
+cff-version: 1.2.0
+message: "If you use dfc-kit, cite this software and the original papers for the methods you apply."
+title: "dfc-kit: Censor-aware dynamic functional connectivity tools"
+type: software
+authors:
+  - name: "dfc-kit contributors"
+version: 1.0.0
+date-released: 2026-08-24
+license: BSD-3-Clause

dfc_kit-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,31 @@
+# Contributing
+
+Contributions should preserve the package boundary: production inputs begin at
+XCP-D parcellated derivatives, while cohort rules, clinical semantics, figures,
+and manuscript workflows remain in downstream applications.
+
+## Development setup
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install -U pip
+python -m pip install -e '.[all,dev,docs]'
+python -m unittest discover
+ruff check src tests
+mkdocs build --strict
+```
+
+## Change requirements
+
+- Temporal methods must prove that no operation crosses a censor gap.
+- Learned methods must expose fit-subject identity and reject held-out overlap.
+- Randomized methods must require or record a reproducible seed.
+- Inferential APIs must identify the statistical unit, tail, and correction scope.
+- New numerical kernels need known-value tests and, when migrated from a research
+  implementation, a small redistributable regression fixture.
+- Public examples must not contain private participant data or project-specific
+  clinical semantics.
+
+Use focused commits and explain any change to a mathematical or statistical
+contract in both tests and user documentation.

dfc_kit-1.0.0/ENGINEERING_PLAN.md

@@ -0,0 +1,42 @@
+# Engineering plan
+
+`dfc-kit` is a numerical library, not an experiment scheduler. The design is
+organized around a short path:
+
+```text
+XCP-D derivatives
+    -> censor-safe feature estimator
+    -> FeatureStore or in-memory result
+    -> state model / information estimator
+    -> explicit score, summary, or inference result
+```
+
+## Design rules
+
+1. Preserve original frame indices and segment identities.
+2. Never calculate a temporal derivative, window, phase feature, or state
+   transition across a retained-frame gap.
+3. Keep fit and held-out subjects explicit in function arguments.
+4. Validate shapes, feature keys, finite values, model parameters, and duplicate
+   acquisition identities at the point of use.
+5. Prefer one direct computation path over resumable intermediate state.
+6. Store model parameters and human-readable metadata; do not add cryptographic
+   identities to numerical results.
+7. Keep MI/CMI in `dfckit.information`, separate from connectivity kernels.
+8. Keep the agent package separate from this toolkit.
+
+## State-count workflow
+
+For each candidate K and subject-disjoint fold, fit on the training subjects and
+score on the held-out subjects. Average repeated seeds inside each participant,
+then average participants inside each fold. The validation module returns the
+candidate table, best K, and one-standard-error K. A final refit is explicit and
+does not require a workflow directory.
+
+## Verification
+
+```bash
+python -m unittest discover
+ruff check src tests
+mkdocs build --strict
+```

dfc_kit-1.0.0/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, dfc-kit contributors
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice,
+   this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

dfc_kit-1.0.0/MANIFEST.in

@@ -0,0 +1,12 @@
+include CHANGELOG.md
+include CITATION.cff
+include CONTRIBUTING.md
+include ENGINEERING_PLAN.md
+include LICENSE
+include README.md
+include ROADMAP.md
+include SECURITY.md
+include mkdocs.yml
+recursive-include benchmarks *.md *.py
+recursive-include docs *.md
+recursive-include tests *.json *.py

dfc_kit-1.0.0/PKG-INFO

@@ -0,0 +1,184 @@
+Metadata-Version: 2.4
+Name: dfc-kit
+Version: 1.0.0
+Summary: Censor-aware tools for dynamic functional connectivity analysis
+License-Expression: BSD-3-Clause
+Project-URL: Homepage, https://github.com/yidao9518/dfc-kit
+Project-URL: Repository, https://github.com/yidao9518/dfc-kit
+Project-URL: Issues, https://github.com/yidao9518/dfc-kit/issues
+Keywords: functional connectivity,dynamic functional connectivity,fMRI,neuroimaging,XCP-D
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.23
+Provides-Extra: states
+Requires-Dist: scikit-learn>=1.2; extra == "states"
+Provides-Extra: phase
+Requires-Dist: scipy>=1.10; extra == "phase"
+Provides-Extra: inference
+Requires-Dist: scipy>=1.10; extra == "inference"
+Provides-Extra: hmm
+Requires-Dist: hmmlearn>=0.3; extra == "hmm"
+Requires-Dist: scikit-learn>=1.2; extra == "hmm"
+Provides-Extra: information
+Requires-Dist: scipy>=1.10; extra == "information"
+Provides-Extra: all
+Requires-Dist: hmmlearn>=0.3; extra == "all"
+Requires-Dist: scikit-learn>=1.2; extra == "all"
+Requires-Dist: scipy>=1.10; extra == "all"
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
+Dynamic: license-file
+
+# dfc-kit
+
+`dfc-kit` is an open-source Python toolkit for dynamic functional connectivity
+analysis of XCP-D parcellated derivatives. It provides composable estimators,
+state models, network summaries, statistical inference, and command-line
+workflows for reproducible neuroimaging analysis.
+
+```text
+BIDS -> fMRIPrep -> XCP-D -> dfc-kit
+```
+
+## Why dfc-kit
+
+`dfc-kit` provides a unified workflow for estimating time-varying functional
+connectivity, identifying recurring brain states, and testing paired or
+between-group differences. Sliding-window FC, instantaneous edges generated
+from ETS or MTD samples, LEiDA, CAP, KMeans, and Gaussian HMM analyses share
+the same data structures and output conventions, making it easier to compare
+methods without rebuilding data loading, state summaries, and statistical
+inference for every analysis.
+
+The toolkit supports both direct in-memory analysis and chunked feature stores
+for larger datasets, with matching Python and command-line interfaces.
+
+## Features
+
+- **Input and topology:** XCP-D discovery and validation, multi-atlas ROI
+  loading, acquisition identity, and censor-bounded sequences. Censored time
+  points retain their original frame indices, and temporal operations are
+  evaluated separately within contiguous retained segments.
+- **Connectivity:** weighted sliding-window FC, instantaneous ETS/MTD edges,
+  LEiDA, low-rank covariance geometry, and fixed-length MI/CMI.
+- **Connectivity and state analysis:** partition-based graph metrics, CAP, KMeans,
+  Gaussian HMMs, state alignment, occupancy/dwell/transition summaries, and
+  selection of the number of states using held-out participants.
+- **Inference:** paired sign-flips, bootstrap intervals, HC3 models,
+  declared-family FDR, generic paired endpoint inference, paired NBS, and
+  within-subject motion matching.
+- **Large-dataset workflows:** chunked, memory-mapped FeatureStores and
+  batch-wise fitting for MiniBatch KMeans and Incremental PCA.
+- **Portable results:** models and held-out predictions stored as JSON and
+  NumPy arrays with explicit feature, subject, and parameter metadata.
+
+The [method inventory](docs/method_inventory.md) maps each method family to its
+public API and guide. Public data, connectivity, state, reference, and
+inference objects are covered by the package test suite and documented
+contracts.
+
+## Scope
+
+The supported input boundary is XCP-D output. `dfc-kit` does not reimplement
+fMRIPrep-to-XCP-D denoising, filtering, censoring, interpolation, or
+parcellation. Callers provide ROI definitions, cohort labels, clinical
+variables, and manuscript-specific analyses around the library's numerical
+interfaces. The array API is also available for equivalently preprocessed ROI
+time series that are not stored as XCP-D derivatives.
+
+## Installation
+
+```bash
+python -m pip install dfc-kit
+```
+
+Install only the optional method families required by an analysis:
+
+```bash
+python -m pip install 'dfc-kit[phase,states,hmm,information,inference]'
+```
+
+Python 3.10 or newer is required. See [Getting started](docs/getting_started.md)
+for development installation and dependency details.
+
+## Quick start
+
+```python
+from dfckit.connectivity import SlidingWindowFC
+from dfckit.io import load_xcpd_run
+
+loaded = load_xcpd_run(
+    "/path/to/xcp_d",
+    subject="sub-001",
+    session="01",
+    task="rest",
+    atlases=("Schaefer200",),
+    space="MNI152NLin2009cAsym",
+    minimum_coverage=0.5,
+    tr=0.8,
+)
+
+result = SlidingWindowFC(length=60, step=10, taper="hamming").transform(loaded.run)
+print(result.features.shape)
+print(result.start_frames, result.end_frames, result.segment_ids)
+```
+
+The result contains Fisher-z upper-triangle edges and the original-frame bounds
+of every valid window. For a complete path from XCP-D discovery through state
+fitting, see the [XCP-D-to-state tutorial](docs/tutorial_xcpd_to_states.md).
+
+## Command line
+
+The `dfc-kit` command exposes XCP-D inspection, FeatureStore construction,
+state fitting, held-out prediction, scoring, alignment, and state-count
+validation. Start with:
+
+```bash
+dfc-kit --help
+dfc-kit inspect-xcpd --help
+dfc-kit build-store --help
+dfc-kit fixed-information --help
+dfc-kit describe-states --help
+dfc-kit infer-state-metrics --help
+dfc-kit summarize-store --help
+dfc-kit summarize-information --help
+dfc-kit infer-paired-endpoints --help
+```
+
+See [Command-line workflows](docs/cli.md) for complete examples and arguments,
+including fixed-length MI/CMI artifacts and frozen-window replay.
+
+## Documentation
+
+- [Documentation home](docs/index.md)
+- [XCP-D input contract](docs/xcpd_input.md)
+- [Connectivity methods](docs/correlation.md)
+- [State models and validation](docs/states.md)
+- [API map](docs/api.md)
+- [Release process](docs/release.md)
+
+## Development
+
+```bash
+python -m pip install -e '.[all,dev,docs]'
+python -m unittest discover
+ruff check src tests
+mkdocs build --strict
+```
+
+`dfc-kit` is distributed under the BSD-3-Clause license. See `LICENSE` and
+`CITATION.cff` for licensing and citation information.

dfc_kit-1.0.0/README.md

@@ -0,0 +1,139 @@
+# dfc-kit
+
+`dfc-kit` is an open-source Python toolkit for dynamic functional connectivity
+analysis of XCP-D parcellated derivatives. It provides composable estimators,
+state models, network summaries, statistical inference, and command-line
+workflows for reproducible neuroimaging analysis.
+
+```text
+BIDS -> fMRIPrep -> XCP-D -> dfc-kit
+```
+
+## Why dfc-kit
+
+`dfc-kit` provides a unified workflow for estimating time-varying functional
+connectivity, identifying recurring brain states, and testing paired or
+between-group differences. Sliding-window FC, instantaneous edges generated
+from ETS or MTD samples, LEiDA, CAP, KMeans, and Gaussian HMM analyses share
+the same data structures and output conventions, making it easier to compare
+methods without rebuilding data loading, state summaries, and statistical
+inference for every analysis.
+
+The toolkit supports both direct in-memory analysis and chunked feature stores
+for larger datasets, with matching Python and command-line interfaces.
+
+## Features
+
+- **Input and topology:** XCP-D discovery and validation, multi-atlas ROI
+  loading, acquisition identity, and censor-bounded sequences. Censored time
+  points retain their original frame indices, and temporal operations are
+  evaluated separately within contiguous retained segments.
+- **Connectivity:** weighted sliding-window FC, instantaneous ETS/MTD edges,
+  LEiDA, low-rank covariance geometry, and fixed-length MI/CMI.
+- **Connectivity and state analysis:** partition-based graph metrics, CAP, KMeans,
+  Gaussian HMMs, state alignment, occupancy/dwell/transition summaries, and
+  selection of the number of states using held-out participants.
+- **Inference:** paired sign-flips, bootstrap intervals, HC3 models,
+  declared-family FDR, generic paired endpoint inference, paired NBS, and
+  within-subject motion matching.
+- **Large-dataset workflows:** chunked, memory-mapped FeatureStores and
+  batch-wise fitting for MiniBatch KMeans and Incremental PCA.
+- **Portable results:** models and held-out predictions stored as JSON and
+  NumPy arrays with explicit feature, subject, and parameter metadata.
+
+The [method inventory](docs/method_inventory.md) maps each method family to its
+public API and guide. Public data, connectivity, state, reference, and
+inference objects are covered by the package test suite and documented
+contracts.
+
+## Scope
+
+The supported input boundary is XCP-D output. `dfc-kit` does not reimplement
+fMRIPrep-to-XCP-D denoising, filtering, censoring, interpolation, or
+parcellation. Callers provide ROI definitions, cohort labels, clinical
+variables, and manuscript-specific analyses around the library's numerical
+interfaces. The array API is also available for equivalently preprocessed ROI
+time series that are not stored as XCP-D derivatives.
+
+## Installation
+
+```bash
+python -m pip install dfc-kit
+```
+
+Install only the optional method families required by an analysis:
+
+```bash
+python -m pip install 'dfc-kit[phase,states,hmm,information,inference]'
+```
+
+Python 3.10 or newer is required. See [Getting started](docs/getting_started.md)
+for development installation and dependency details.
+
+## Quick start
+
+```python
+from dfckit.connectivity import SlidingWindowFC
+from dfckit.io import load_xcpd_run
+
+loaded = load_xcpd_run(
+    "/path/to/xcp_d",
+    subject="sub-001",
+    session="01",
+    task="rest",
+    atlases=("Schaefer200",),
+    space="MNI152NLin2009cAsym",
+    minimum_coverage=0.5,
+    tr=0.8,
+)
+
+result = SlidingWindowFC(length=60, step=10, taper="hamming").transform(loaded.run)
+print(result.features.shape)
+print(result.start_frames, result.end_frames, result.segment_ids)
+```
+
+The result contains Fisher-z upper-triangle edges and the original-frame bounds
+of every valid window. For a complete path from XCP-D discovery through state
+fitting, see the [XCP-D-to-state tutorial](docs/tutorial_xcpd_to_states.md).
+
+## Command line
+
+The `dfc-kit` command exposes XCP-D inspection, FeatureStore construction,
+state fitting, held-out prediction, scoring, alignment, and state-count
+validation. Start with:
+
+```bash
+dfc-kit --help
+dfc-kit inspect-xcpd --help
+dfc-kit build-store --help
+dfc-kit fixed-information --help
+dfc-kit describe-states --help
+dfc-kit infer-state-metrics --help
+dfc-kit summarize-store --help
+dfc-kit summarize-information --help
+dfc-kit infer-paired-endpoints --help
+```
+
+See [Command-line workflows](docs/cli.md) for complete examples and arguments,
+including fixed-length MI/CMI artifacts and frozen-window replay.
+
+## Documentation
+
+- [Documentation home](docs/index.md)
+- [XCP-D input contract](docs/xcpd_input.md)
+- [Connectivity methods](docs/correlation.md)
+- [State models and validation](docs/states.md)
+- [API map](docs/api.md)
+- [Release process](docs/release.md)
+
+## Development
+
+```bash
+python -m pip install -e '.[all,dev,docs]'
+python -m unittest discover
+ruff check src tests
+mkdocs build --strict
+```
+
+`dfc-kit` is distributed under the BSD-3-Clause license. See `LICENSE` and
+`CITATION.cff` for licensing and citation information.

dfc_kit-1.0.0/ROADMAP.md

@@ -0,0 +1,26 @@
+# Roadmap
+
+## Current development target
+
+- Keep the public API small and composable.
+- Treat XCP-D derivatives and ROI definitions as the input boundary.
+- Keep all temporal operations inside contiguous retained-frame segments.
+- Keep KMeans/HMM fitting, decoding, held-out scoring, and stability summaries
+  independent from manuscript-specific workflows.
+- Select K with subject-disjoint held-out scores and a compact in-memory
+  comparison.
+- Maintain pickle-free JSON/NumPy artifacts with explicit numerical metadata.
+- Expand numerical regression tests for MI/CMI, ETS/MTD, low-rank methods,
+  state alignment, and inference.
+
+## Deliberately out of scope
+
+- Resumable workflow directories and nested state-count orchestration.
+- Cryptographic identities for arrays, models, or result files.
+- Parkinson-specific ROI definitions, clinical variables, figures, and
+  manuscript automation.
+
+## Release boundary
+
+The independent `dfc-agent` project may orchestrate this toolkit, but remains
+in a separate repository and package.

dfc_kit-1.0.0/SECURITY.md

@@ -0,0 +1,12 @@
+# Security policy
+
+`dfc-kit` processes local derivative files and does not require network access.
+Do not include participant data, access credentials, private server paths, or
+authenticated URLs in bug reports or test fixtures.
+
+For a suspected vulnerability, contact the repository maintainer privately
+through the hosting platform rather than opening a public issue. Include the
+affected version, a minimal synthetic reproducer, and the expected impact.
+
+Only the latest minor release receives security fixes while the project remains
+pre-1.0.

dfc_kit-1.0.0/benchmarks/README.md

@@ -0,0 +1,69 @@
+# Synthetic performance benchmarks
+
+Run each method in a separate process so `ru_maxrss` has one clear scope:
+
+```bash
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method window-fc --frames 600 --rois 100 --length 60 --step 5
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method ets-full --frames 1000 --rois 200
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method ets-rss --frames 1000 --rois 200
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method ets-store --frames 1000 --rois 200
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method window-kmeans-memory --frames 1200 --rois 200 \
+  --length 60 --step 5 --states 4 --n-init 3 --max-iter 3 --batch-size 64
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method window-kmeans-store --frames 1200 --rois 200 \
+  --length 60 --step 5 --states 4 --n-init 3 --max-iter 3 --batch-size 64
+
+PYTHONPATH=src python benchmarks/profile_core.py \
+  --method window-hmm-store --frames 1200 --rois 200 \
+  --length 60 --step 5 --states 4 --n-init 1 --max-iter 20 \
+  --batch-size 64 --pca-components 10
+```
+
+The input is deterministic synthetic Gaussian data with a small shared latent
+component and one explicit censor gap. The script reports elapsed wall time,
+current RSS before/after the transform on Linux, process-lifetime peak RSS,
+transform-scoped `tracemalloc` peak, and the size of arrays retained in the
+result object. It is a developer benchmark, not a scientific result or a CI
+pass/fail threshold.
+
+`ETS().rss()` is expected to use substantially less memory than materializing
+the complete frame-by-edge ETS matrix. High-dimensional sliding-window FC and
+full ETS are the main targets for chunked feature output.
+
+The two `window-kmeans-*` cases precompute identical sliding-window FC before
+starting the measured region. They therefore compare only state fitting: the
+in-memory path pools a `FeatureSequenceDataset`, while the store path estimates
+the scaler, updates MiniBatchKMeans, scores all initializations, and emits labels
+by traversing memory-mapped chunks. Feature construction time is intentionally
+excluded from both measurements.
+
+Reference run (`1200` frames, `200` ROIs, `60`-frame windows, step `5`, `4`
+states, `3` initializations, `3` passes, batch size `64`) produced:
+
+| State-fitting path | Peak RSS | Traced peak | Time |
+|---|---:|---:|---:|
+| In-memory `FeatureSequenceDataset` | 497.28 MiB | 279.89 MiB | 8.11 s |
+| Memory-mapped `FeatureStore` | 294.03 MiB | 135.57 MiB | 8.44 s |
+
+These are synthetic engineering measurements, not scientific results. The
+stored path's files occupied `34.02 MiB`; its advantage is bounded memory, not
+an assumption that disk I/O is free.
+
+The `window-hmm-store` command above (`1` initialization, `20` HMM iterations,
+and `10` PCA components) produced `218` windows with `19,900` original FC
+features. The measured fit took `1.94 s`, reached `292.15 MiB` process-lifetime
+peak RSS and `127.22 MiB` transform-scoped traced peak, and retained `2.60 MiB`
+of result arrays. The memory-mapped feature files occupied `34.02 MiB`. The
+compact fitted model does not retain a dense
+`states x original_features x original_features` covariance tensor; an
+original-space covariance is reconstructed explicitly for one requested state.