DARTsort 0.5.2__tar.gz → 0.5.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dartsort-0.5.2 → dartsort-0.5.3}/.github/workflows/ci.yml +1 -1
- dartsort-0.5.3/.github/workflows/docs.yml +68 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/.gitignore +4 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/PKG-INFO +44 -32
- {dartsort-0.5.2 → dartsort-0.5.3}/README.md +39 -31
- dartsort-0.5.3/docs/index.md +175 -0
- dartsort-0.5.3/docs/main_api.md +47 -0
- dartsort-0.5.3/docs/toolkit_api.md +90 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/environment.yml +1 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/pyproject.toml +2 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/cluster_util.py +3 -3
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/density.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/merge.py +3 -3
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/mixture.py +60 -31
- dartsort-0.5.3/src/dartsort/config.py +358 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/detect/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/detect/detect.py +2 -2
- dartsort-0.5.3/src/dartsort/evaluate/__init__.py +2 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/hybrid_util.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/simlib.py +2 -2
- dartsort-0.5.3/src/dartsort/localize/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/localize/localize_torch.py +4 -4
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/main.py +25 -23
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/__init__.py +3 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/grab.py +1 -0
- dartsort-0.5.3/src/dartsort/peel/matching_util/__init__.py +15 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching_util/drifty.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching_util/matching_base.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/peel_base.py +3 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/reduction_template.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/subtract.py +2 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/threshold.py +1 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/get_templates.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/template_util.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/__init__.py +2 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/_multichan_denoiser_kit.py +1 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/all_transformers.py +1 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/amortized_localization.py +3 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/amplitudes.py +3 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/decollider.py +2 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/interp.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/localize.py +3 -3
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/mixture_classifier.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/pipeline.py +9 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/single_channel_denoiser.py +2 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/supervised_denoiser.py +2 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/temporal_pca.py +4 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/transform_base.py +7 -1
- dartsort-0.5.3/src/dartsort/util/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/cli_util.py +108 -28
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/data_util.py +11 -5
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/drift_util.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/internal_config.py +4 -4
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/interpolation_util.py +9 -9
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/motion.py +11 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/noise_util.py +10 -10
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/py_util.py +0 -54
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/spiketorch.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/testing_util/mixture_testing_util.py +8 -8
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/torch_optimization_util.py +2 -2
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/__init__.py +1 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/colors.py +1 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/scatterplots.py +3 -3
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/unit.py +0 -1
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/PKG-INFO +44 -32
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/SOURCES.txt +5 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/requires.txt +5 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_transform.py +4 -4
- dartsort-0.5.3/zensical.toml +389 -0
- dartsort-0.5.2/src/dartsort/config.py +0 -392
- dartsort-0.5.2/src/dartsort/evaluate/__init__.py +0 -1
- dartsort-0.5.2/src/dartsort/peel/matching_util/__init__.py +0 -3
- dartsort-0.5.2/src/dartsort/util/__init__.py +0 -0
- dartsort-0.5.2/src/dartsort/util/testing_util/__init__.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/.github/workflows/ci_windows.yml +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/.github/workflows/deploy.yml +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/.keep +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/basic_integration_test.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/ci diag.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/dartsort_dredge_demo.ipynb +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/dartsort_dredge_demo.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/data test.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/dead reckoning.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/denoise np1.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/denoise np2.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/exploratory.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/geom debug.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/hdbscan_clustering_5min.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/hybrid 10.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/hybrid cluster.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/hybrid_11.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/initial training.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/make hybrid.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/mgh.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/np1 feats.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/np2 processing version c.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/paperfigs.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/pdists.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/postcheck.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/ptp vae.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/re clust.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/reg fig loop.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/reg_sim_exp.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/relocating.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/relocating_all.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/restats +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/sortStinger1.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/sortStinger3.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/sortStinger4.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/sortStinger5.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/sortStinger6.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/spike farm.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/ss_ensemble_sim.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/ss_ensemble_sim_drift.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/ss_sim.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/stan diagnostics.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/storytelling c.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/storytelling.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/subtract_with_phaseshift_denoiser.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/debug_matching.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_agreement.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_amplitude_scaling.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_batch_optim.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_ibme.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_mm_initialization.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_normxcorr1d.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_reassignment.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/tests/test_relocation.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/notebook/thomas.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/setup.cfg +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/cli.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/__init__.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/agglomerate.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/clustering.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/clustering_features.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/forward_backward.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/kmeans.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/modes.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/clustering/refine_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/detect/detect_filters.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/analysis.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/comparison.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/noise_recording_tools.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/sim_template_tools.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/evaluate/simkit.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/localize/localize_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching_util/compressed_upsampled.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching_util/pairwise.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/matching_util/pairwise_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/peel_lib.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/peel/shaver.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/default_temporal_kernel.npy +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/detect_np1.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/detect_np2.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/glasbey1024.npz +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/offset_trained_denoiser.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/offset_trained_denoiser_added_temp.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/offset_trained_denoiser_added_temp_boost.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/original_single_chan_denoiser.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/pretrained/single_chan_denoiser.pt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/postprocess_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/realignment.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/templates.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/templates/templib.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/denoising_scorer.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/enforce_decrease.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/fixed_prop.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/matching_denoiser.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/reduction.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/transform/whiten.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/job_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/logging_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/main_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/more_operators.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/multiprocessing_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/nn_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/peel_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/preprocess_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/registration_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/residual_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/sparse_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/spikeio.py +0 -0
- {dartsort-0.5.2/src/dartsort/localize → dartsort-0.5.3/src/dartsort/util/testing_util}/__init__.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/testing_util/matching_debug_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/torch_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/util/waveform_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/analysis_plots.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/gt.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/layout.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/metrics_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/mixture.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/recanim.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/sorting.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/unit_comparison.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/versus.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/vismain.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort/vis/waveforms.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/dependency_links.txt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/entry_points.txt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/src/dartsort.egg-info/top_level.txt +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/conftest.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_alignment.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_clustering.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_config.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_dartsort.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_data_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_detect.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_drift_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_drifty_matching.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_enforce_decrease.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_grab_and_featurize.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_kmeans.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_linalg.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_localize_torch.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_matching.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_mixture_classifier.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_modeling.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_simkit.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_spiketorch.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_subtract.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_templates.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_threshold.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_torch_optimization_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_util.py +0 -0
- {dartsort-0.5.2 → dartsort-0.5.3}/tests/test_waveform_util.py +0 -0
|
@@ -26,7 +26,7 @@ jobs:
|
|
|
26
26
|
- name: Fetch tags and describe
|
|
27
27
|
run: pushd dartsort/ && git fetch --prune --unshallow --tags && git describe && popd
|
|
28
28
|
|
|
29
|
-
- uses: actions/setup-python@
|
|
29
|
+
- uses: actions/setup-python@v6
|
|
30
30
|
with:
|
|
31
31
|
python-version: "3.x"
|
|
32
32
|
cache: "pip" # caching pip dependencies
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
name: Documentation
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
release:
|
|
5
|
+
types: [published]
|
|
6
|
+
workflow_dispatch:
|
|
7
|
+
|
|
8
|
+
permissions:
|
|
9
|
+
contents: read
|
|
10
|
+
pages: write
|
|
11
|
+
id-token: write
|
|
12
|
+
jobs:
|
|
13
|
+
deploy:
|
|
14
|
+
runs-on: ubuntu-latest
|
|
15
|
+
steps:
|
|
16
|
+
- name: Checkout dartsort repo
|
|
17
|
+
uses: actions/checkout@v6
|
|
18
|
+
with:
|
|
19
|
+
path: dartsort
|
|
20
|
+
|
|
21
|
+
- name: Fetch tags and describe
|
|
22
|
+
run: pushd dartsort && git fetch --prune --unshallow --tags && git describe && popd
|
|
23
|
+
|
|
24
|
+
- name: Checkout dartsort.github.io repo
|
|
25
|
+
uses: actions/checkout@v6
|
|
26
|
+
with:
|
|
27
|
+
repository: dartsort/dartsort.github.io
|
|
28
|
+
path: io
|
|
29
|
+
# this is a "fine-grained token"
|
|
30
|
+
token: ${{ secrets.DARTSORT_GITHUB_IO }}
|
|
31
|
+
|
|
32
|
+
- name: Setup Python
|
|
33
|
+
uses: actions/setup-python@v6
|
|
34
|
+
with:
|
|
35
|
+
python-version: 3.x
|
|
36
|
+
cache: "pip" # caching pip dependencies
|
|
37
|
+
|
|
38
|
+
- name: pip install stuff
|
|
39
|
+
run: python3 -m pip install -e dartsort/[doc]
|
|
40
|
+
|
|
41
|
+
- name: Zensical build
|
|
42
|
+
run: pushd dartsort/ && zensical build --clean && popd
|
|
43
|
+
|
|
44
|
+
- name: overwrite site
|
|
45
|
+
shell: bash
|
|
46
|
+
run: |
|
|
47
|
+
cp -r dartsort/site/* io/
|
|
48
|
+
|
|
49
|
+
# shouts out to some-natalie.dev/blog/multi-repo-actions/
|
|
50
|
+
- name: Push to io
|
|
51
|
+
shell: bash
|
|
52
|
+
env:
|
|
53
|
+
CI_COMMIT_MESSAGE: update dartsort.github.io
|
|
54
|
+
CI_COMMIT_AUTHOR: github-actions[bot]
|
|
55
|
+
CI_COMMIT_EMAIL: username@users.noreply.github.com
|
|
56
|
+
run: |
|
|
57
|
+
cd io/
|
|
58
|
+
git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
|
|
59
|
+
git config --global user.email "${{ env.CI_COMMIT_EMAIL }}"
|
|
60
|
+
|
|
61
|
+
if [[ `git status --porcelain` ]]; then
|
|
62
|
+
git add .
|
|
63
|
+
git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
|
|
64
|
+
git push
|
|
65
|
+
else
|
|
66
|
+
echo "no changes"
|
|
67
|
+
exit 0
|
|
68
|
+
fi
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: dartsort
|
|
3
|
-
Version: 0.5.
|
|
3
|
+
Version: 0.5.3
|
|
4
4
|
Summary: DARTsort
|
|
5
5
|
Author-email: Charlie Windolf <ciw2048@gmail.com>
|
|
6
6
|
Project-URL: Homepage, https://github.com/cwindolf/dartsort
|
|
@@ -29,6 +29,10 @@ Requires-Dist: spikeinterface>=0.104.0
|
|
|
29
29
|
Requires-Dist: sympy
|
|
30
30
|
Requires-Dist: torch>=2.0
|
|
31
31
|
Requires-Dist: tqdm
|
|
32
|
+
Requires-Dist: typing-extensions
|
|
33
|
+
Provides-Extra: doc
|
|
34
|
+
Requires-Dist: zensical; extra == "doc"
|
|
35
|
+
Requires-Dist: mkdocstrings-python; extra == "doc"
|
|
32
36
|
Provides-Extra: test
|
|
33
37
|
Requires-Dist: pytest; extra == "test"
|
|
34
38
|
Requires-Dist: pytest-cov; extra == "test"
|
|
@@ -38,6 +42,9 @@ Requires-Dist: seaborn; extra == "vis"
|
|
|
38
42
|
Requires-Dist: matplotlib-venn; extra == "vis"
|
|
39
43
|
|
|
40
44
|
[](https://github.com/cwindolf/dartsort/actions/)
|
|
45
|
+
[](https://github.com/cwindolf/dartsort/)
|
|
46
|
+
[](https://dartsort.github.io)
|
|
47
|
+
[](https://github.com/dartsort/dartsort.github.io/actions/workflows/pages/pages-build-deployment)
|
|
41
48
|
[](https://coveralls.io/github/cwindolf/dartsort)
|
|
42
49
|
[](https://doi.org/10.5281/zenodo.16943074)
|
|
43
50
|
[](https://pypi.org/p/dartsort)
|
|
@@ -47,7 +54,7 @@ Requires-Dist: matplotlib-venn; extra == "vis"
|
|
|
47
54
|
|
|
48
55
|
|
|
49
56
|
*dartsort* is a modular spike sorter built around a statistical clustering model and a new approach to probe motion.
|
|
50
|
-
It is also a toolkit of modules for building spike sorters
|
|
57
|
+
It is also a toolkit of modules for building spike sorters and other analyses of electrophysiology data.
|
|
51
58
|
|
|
52
59
|
|
|
53
60
|
## :warning: work in progress :warning:
|
|
@@ -60,7 +67,7 @@ Please feel free to open an issue or a discussion if you run into problems.
|
|
|
60
67
|
|
|
61
68
|
### Installing into an existing environment
|
|
62
69
|
|
|
63
|
-
If you already have a Python environment with PyTorch working and you just want to
|
|
70
|
+
If you already have a Python environment with PyTorch working and you just want to install *dartsort* there, use
|
|
64
71
|
|
|
65
72
|
```sh
|
|
66
73
|
$ pip install dartsort
|
|
@@ -68,35 +75,32 @@ $ pip install dartsort
|
|
|
68
75
|
|
|
69
76
|
If you want to run the test suite or use `dartsort.vis`, you can install the optional dependencies with `pip install dartsort[test,vis]`.
|
|
70
77
|
|
|
71
|
-
|
|
78
|
+
## Setting up a Python environment
|
|
72
79
|
|
|
73
|
-
|
|
80
|
+
If you need to set up Python or PyTorch, I find that a [`conda-forge`](https://conda-forge.org/)-based distribution is the most reliable at installing the GPU dependencies which PyTorch needs (note: `conda-forge` is different from the non-free Anaconda).
|
|
74
81
|
|
|
75
|
-
|
|
82
|
+
You can use `conda-forge` to install Python, `dartsort`, and its dependencies as follows:
|
|
76
83
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
$ mamba activate dartsort
|
|
89
|
-
```
|
|
90
|
-
- Install `dartsort` and the rest of its dependencies by running the pip command [above](#installing-into-an-existing-environment)
|
|
91
|
-
|
|
92
|
-
</details>
|
|
84
|
+
- Follow the `conda-forge` installation instructions for your platform at [https://conda-forge.org/download/](https://conda-forge.org/download/)
|
|
85
|
+
- Create an environment with
|
|
86
|
+
```sh
|
|
87
|
+
$ mamba env create -f environment.yml
|
|
88
|
+
```
|
|
89
|
+
This will create an environment called `dartsort`, but you can change the name by adding `-n othername`.
|
|
90
|
+
- Activate the environment:
|
|
91
|
+
```sh
|
|
92
|
+
$ mamba activate dartsort
|
|
93
|
+
```
|
|
94
|
+
- Install `dartsort` and the rest of its dependencies by running the pip command [above](#installing-into-an-existing-environment)
|
|
93
95
|
|
|
94
96
|
|
|
95
97
|
## Usage
|
|
96
98
|
|
|
99
|
+
For more detailed documentation of the main function parameters and configuration options, see the [main API documentation](main_api.md).
|
|
100
|
+
|
|
97
101
|
### As a Python function
|
|
98
102
|
|
|
99
|
-
*dartsort* can be run
|
|
103
|
+
*dartsort* can be run in Python with:
|
|
100
104
|
|
|
101
105
|
```python
|
|
102
106
|
import dartsort
|
|
@@ -115,13 +119,13 @@ dartsort_result = dartsort.dartsort(
|
|
|
115
119
|
)
|
|
116
120
|
```
|
|
117
121
|
|
|
118
|
-
**Please** read the [important configuration details section](#important-configuration-details) below
|
|
119
|
-
Some of them, like preprocessing
|
|
120
|
-
|
|
121
|
-
Here, `recording` is a [SpikeInterface](SpikeInterface) recording object.
|
|
122
|
-
SpikeInterface can read every electrophysiology data format that I've encountered and many I haven't; see
|
|
122
|
+
**Please** read the [important configuration details section](#important-configuration-details) below for information on the parameters of the `DARTsortUserConfig`.
|
|
123
|
+
Some of them, like `preprocessing`, are not set by default and need your input!
|
|
124
|
+
(This will probably change.)
|
|
123
125
|
|
|
126
|
+
Here, `recording` is a [SpikeInterface][SpikeInterface] recording object (see their tutorial on [reading various recording formats](https://spikeinterface.readthedocs.io/en/stable/how_to/read_various_formats.html)).
|
|
124
127
|
`output_dir` is the folder where *dartsort* will save its output.
|
|
128
|
+
There are more details on these and the rest of the arguments in the [main API documentation](main_api.md).
|
|
125
129
|
|
|
126
130
|
Once you've run *dartsort*, you might want to check out [the outputs and exporting](#outputs-and-exporting) section below.
|
|
127
131
|
|
|
@@ -129,19 +133,20 @@ Once you've run *dartsort*, you might want to check out [the outputs and exporti
|
|
|
129
133
|
|
|
130
134
|
Before running *dartsort*, please be aware of the following important configuration options.
|
|
131
135
|
|
|
132
|
-
- `preprocessing`: *dartsort* won't touch your data by default (`preprocessing="none"`), leaving you free to implement your own preprocessing in SpikeInterface or otherwise, and therefore
|
|
133
|
-
- For a cheap but sensible default, try `preprocessing="ibllikecmr"`, which applies a pipeline similar to that of [the IBL]
|
|
134
|
-
- `do_motion_estimation=True` by default,
|
|
136
|
+
- `preprocessing`: *dartsort* won't touch your data by default (`preprocessing="none"`), leaving you free to implement your own preprocessing in SpikeInterface or otherwise. However, *dartsort* expects standardized input, and therefore it will explode if you don't set this flag and your data is in its original state (for instance, the raw `int16` data off the probe).
|
|
137
|
+
- For a cheap but sensible default, try `preprocessing="ibllikecmr"`, which applies a pipeline similar to that of [the IBL][iblsorting] but with global median common referencing instead of their spatial highpass filter. `preprocessing="ibllike"` will use their spatial highpass filter.
|
|
138
|
+
- `do_motion_estimation=True` by default, but you may like to disable it if you know that there is (say) less than 5 microns of total drift in your recording, or if you have handled this in your own preprocessing.
|
|
135
139
|
- `work_in_tmpdir` and `copy_recording_to_tmpdir` can be helpful in some cases where slow network drives are involved.
|
|
136
140
|
|
|
137
141
|
### Outputs and exporting
|
|
138
142
|
|
|
139
|
-
The `dartsort_result = dartsort(...)` function returns a dictionary `dartsort_result` containing a `DARTsortSorting` object
|
|
143
|
+
The `dartsort_result = dartsort(...)` function returns a dictionary `dartsort_result` containing a `DARTsortSorting` object: `sorting = dartsort_result["sorting"]`.
|
|
140
144
|
This object has all the spike train data attached (as arrays under property names `.times_samples` and `.times_seconds` for spike times in samples and seconds, `.labels` for unit labels, and many others; `print(sorting)` to see some more).
|
|
141
145
|
|
|
142
146
|
If you already ran *dartsort* and want to load the output spike trains, use `dartsort.load(output_dir)` to get the `DARTsortSorting` object.
|
|
143
147
|
|
|
144
148
|
This object can also export itself to other formats:
|
|
149
|
+
|
|
145
150
|
- For a SpikeInterface `NumpySorting` object, use `sorting.to_numpy_sorting()`
|
|
146
151
|
- For a [Pynapple][Pynapple] `TsGroup`, use `sorting.to_tsgroup()`
|
|
147
152
|
- To export to Phy, we currently suggest bridging through SpikeInterface. Start with `sorting.to_numpy_sorting()` and follow the instructions [in SpikeInterface's documentation](https://spikeinterface.readthedocs.io/en/stable/modules/exporters.html#exporting-to-phy) for first creating a `SortingAnalyzer` and then exporting that to Phy.
|
|
@@ -199,8 +204,15 @@ Please let us know if you run into any issues.
|
|
|
199
204
|
If you feel that the issue is a software bug, feel free to open an issue or a discussion on GitHub.
|
|
200
205
|
If it's more of a data-related or methodology thing, feel free to use the email on my GitHub [profile](https://github.com/cwindolf).
|
|
201
206
|
|
|
207
|
+
## References
|
|
208
|
+
|
|
209
|
+
- [SpikeInterface][SpikeInterface]
|
|
210
|
+
- DREDge [on PyPI][dredge-pypi] and the DREDge [paper][dredge-nm] ([PDF link][dredge-rdcb]).
|
|
202
211
|
|
|
203
212
|
|
|
204
213
|
[SpikeInterface]: https://spikeinterface.readthedocs.io
|
|
205
214
|
[iblsorting]: https://figshare.com/articles/online_resource/Spike_sorting_pipeline_for_the_International_Brain_Laboratory/19705522
|
|
206
215
|
[Pynapple]: https://pynapple.org/
|
|
216
|
+
[dredge-pypi]: https://pypi.org/p/dredge-ephys/
|
|
217
|
+
[dredge-nm]: https://www.nature.com/articles/s41592-025-02614-5
|
|
218
|
+
[dredge-rdcb]: https://rdcu.be/ecqZf
|
|
@@ -1,4 +1,7 @@
|
|
|
1
1
|
[](https://github.com/cwindolf/dartsort/actions/)
|
|
2
|
+
[](https://github.com/cwindolf/dartsort/)
|
|
3
|
+
[](https://dartsort.github.io)
|
|
4
|
+
[](https://github.com/dartsort/dartsort.github.io/actions/workflows/pages/pages-build-deployment)
|
|
2
5
|
[](https://coveralls.io/github/cwindolf/dartsort)
|
|
3
6
|
[](https://doi.org/10.5281/zenodo.16943074)
|
|
4
7
|
[](https://pypi.org/p/dartsort)
|
|
@@ -8,7 +11,7 @@
|
|
|
8
11
|
|
|
9
12
|
|
|
10
13
|
*dartsort* is a modular spike sorter built around a statistical clustering model and a new approach to probe motion.
|
|
11
|
-
It is also a toolkit of modules for building spike sorters
|
|
14
|
+
It is also a toolkit of modules for building spike sorters and other analyses of electrophysiology data.
|
|
12
15
|
|
|
13
16
|
|
|
14
17
|
## :warning: work in progress :warning:
|
|
@@ -21,7 +24,7 @@ Please feel free to open an issue or a discussion if you run into problems.
|
|
|
21
24
|
|
|
22
25
|
### Installing into an existing environment
|
|
23
26
|
|
|
24
|
-
If you already have a Python environment with PyTorch working and you just want to
|
|
27
|
+
If you already have a Python environment with PyTorch working and you just want to install *dartsort* there, use
|
|
25
28
|
|
|
26
29
|
```sh
|
|
27
30
|
$ pip install dartsort
|
|
@@ -29,35 +32,32 @@ $ pip install dartsort
|
|
|
29
32
|
|
|
30
33
|
If you want to run the test suite or use `dartsort.vis`, you can install the optional dependencies with `pip install dartsort[test,vis]`.
|
|
31
34
|
|
|
32
|
-
|
|
35
|
+
## Setting up a Python environment
|
|
33
36
|
|
|
34
|
-
|
|
37
|
+
If you need to set up Python or PyTorch, I find that a [`conda-forge`](https://conda-forge.org/)-based distribution is the most reliable at installing the GPU dependencies which PyTorch needs (note: `conda-forge` is different from the non-free Anaconda).
|
|
35
38
|
|
|
36
|
-
|
|
39
|
+
You can use `conda-forge` to install Python, `dartsort`, and its dependencies as follows:
|
|
37
40
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
$ mamba activate dartsort
|
|
50
|
-
```
|
|
51
|
-
- Install `dartsort` and the rest of its dependencies by running the pip command [above](#installing-into-an-existing-environment)
|
|
52
|
-
|
|
53
|
-
</details>
|
|
41
|
+
- Follow the `conda-forge` installation instructions for your platform at [https://conda-forge.org/download/](https://conda-forge.org/download/)
|
|
42
|
+
- Create an environment with
|
|
43
|
+
```sh
|
|
44
|
+
$ mamba env create -f environment.yml
|
|
45
|
+
```
|
|
46
|
+
This will create an environment called `dartsort`, but you can change the name by adding `-n othername`.
|
|
47
|
+
- Activate the environment:
|
|
48
|
+
```sh
|
|
49
|
+
$ mamba activate dartsort
|
|
50
|
+
```
|
|
51
|
+
- Install `dartsort` and the rest of its dependencies by running the pip command [above](#installing-into-an-existing-environment)
|
|
54
52
|
|
|
55
53
|
|
|
56
54
|
## Usage
|
|
57
55
|
|
|
56
|
+
For more detailed documentation of the main function parameters and configuration options, see the [main API documentation](main_api.md).
|
|
57
|
+
|
|
58
58
|
### As a Python function
|
|
59
59
|
|
|
60
|
-
*dartsort* can be run
|
|
60
|
+
*dartsort* can be run in Python with:
|
|
61
61
|
|
|
62
62
|
```python
|
|
63
63
|
import dartsort
|
|
@@ -76,13 +76,13 @@ dartsort_result = dartsort.dartsort(
|
|
|
76
76
|
)
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
-
**Please** read the [important configuration details section](#important-configuration-details) below
|
|
80
|
-
Some of them, like preprocessing
|
|
81
|
-
|
|
82
|
-
Here, `recording` is a [SpikeInterface](SpikeInterface) recording object.
|
|
83
|
-
SpikeInterface can read every electrophysiology data format that I've encountered and many I haven't; see
|
|
79
|
+
**Please** read the [important configuration details section](#important-configuration-details) below for information on the parameters of the `DARTsortUserConfig`.
|
|
80
|
+
Some of them, like `preprocessing`, are not set by default and need your input!
|
|
81
|
+
(This will probably change.)
|
|
84
82
|
|
|
83
|
+
Here, `recording` is a [SpikeInterface][SpikeInterface] recording object (see their tutorial on [reading various recording formats](https://spikeinterface.readthedocs.io/en/stable/how_to/read_various_formats.html)).
|
|
85
84
|
`output_dir` is the folder where *dartsort* will save its output.
|
|
85
|
+
There are more details on these and the rest of the arguments in the [main API documentation](main_api.md).
|
|
86
86
|
|
|
87
87
|
Once you've run *dartsort*, you might want to check out [the outputs and exporting](#outputs-and-exporting) section below.
|
|
88
88
|
|
|
@@ -90,19 +90,20 @@ Once you've run *dartsort*, you might want to check out [the outputs and exporti
|
|
|
90
90
|
|
|
91
91
|
Before running *dartsort*, please be aware of the following important configuration options.
|
|
92
92
|
|
|
93
|
-
- `preprocessing`: *dartsort* won't touch your data by default (`preprocessing="none"`), leaving you free to implement your own preprocessing in SpikeInterface or otherwise, and therefore
|
|
94
|
-
- For a cheap but sensible default, try `preprocessing="ibllikecmr"`, which applies a pipeline similar to that of [the IBL]
|
|
95
|
-
- `do_motion_estimation=True` by default,
|
|
93
|
+
- `preprocessing`: *dartsort* won't touch your data by default (`preprocessing="none"`), leaving you free to implement your own preprocessing in SpikeInterface or otherwise. However, *dartsort* expects standardized input, and therefore it will explode if you don't set this flag and your data is in its original state (for instance, the raw `int16` data off the probe).
|
|
94
|
+
- For a cheap but sensible default, try `preprocessing="ibllikecmr"`, which applies a pipeline similar to that of [the IBL][iblsorting] but with global median common referencing instead of their spatial highpass filter. `preprocessing="ibllike"` will use their spatial highpass filter.
|
|
95
|
+
- `do_motion_estimation=True` by default, but you may like to disable it if you know that there is (say) less than 5 microns of total drift in your recording, or if you have handled this in your own preprocessing.
|
|
96
96
|
- `work_in_tmpdir` and `copy_recording_to_tmpdir` can be helpful in some cases where slow network drives are involved.
|
|
97
97
|
|
|
98
98
|
### Outputs and exporting
|
|
99
99
|
|
|
100
|
-
The `dartsort_result = dartsort(...)` function returns a dictionary `dartsort_result` containing a `DARTsortSorting` object
|
|
100
|
+
The `dartsort_result = dartsort(...)` function returns a dictionary `dartsort_result` containing a `DARTsortSorting` object: `sorting = dartsort_result["sorting"]`.
|
|
101
101
|
This object has all the spike train data attached (as arrays under property names `.times_samples` and `.times_seconds` for spike times in samples and seconds, `.labels` for unit labels, and many others; `print(sorting)` to see some more).
|
|
102
102
|
|
|
103
103
|
If you already ran *dartsort* and want to load the output spike trains, use `dartsort.load(output_dir)` to get the `DARTsortSorting` object.
|
|
104
104
|
|
|
105
105
|
This object can also export itself to other formats:
|
|
106
|
+
|
|
106
107
|
- For a SpikeInterface `NumpySorting` object, use `sorting.to_numpy_sorting()`
|
|
107
108
|
- For a [Pynapple][Pynapple] `TsGroup`, use `sorting.to_tsgroup()`
|
|
108
109
|
- To export to Phy, we currently suggest bridging through SpikeInterface. Start with `sorting.to_numpy_sorting()` and follow the instructions [in SpikeInterface's documentation](https://spikeinterface.readthedocs.io/en/stable/modules/exporters.html#exporting-to-phy) for first creating a `SortingAnalyzer` and then exporting that to Phy.
|
|
@@ -160,8 +161,15 @@ Please let us know if you run into any issues.
|
|
|
160
161
|
If you feel that the issue is a software bug, feel free to open an issue or a discussion on GitHub.
|
|
161
162
|
If it's more of a data-related or methodology thing, feel free to use the email on my GitHub [profile](https://github.com/cwindolf).
|
|
162
163
|
|
|
164
|
+
## References
|
|
165
|
+
|
|
166
|
+
- [SpikeInterface][SpikeInterface]
|
|
167
|
+
- DREDge [on PyPI][dredge-pypi] and the DREDge [paper][dredge-nm] ([PDF link][dredge-rdcb]).
|
|
163
168
|
|
|
164
169
|
|
|
165
170
|
[SpikeInterface]: https://spikeinterface.readthedocs.io
|
|
166
171
|
[iblsorting]: https://figshare.com/articles/online_resource/Spike_sorting_pipeline_for_the_International_Brain_Laboratory/19705522
|
|
167
172
|
[Pynapple]: https://pynapple.org/
|
|
173
|
+
[dredge-pypi]: https://pypi.org/p/dredge-ephys/
|
|
174
|
+
[dredge-nm]: https://www.nature.com/articles/s41592-025-02614-5
|
|
175
|
+
[dredge-rdcb]: https://rdcu.be/ecqZf
|
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
[](https://github.com/cwindolf/dartsort/actions/)
|
|
2
|
+
[](https://github.com/cwindolf/dartsort/)
|
|
3
|
+
[](https://dartsort.github.io)
|
|
4
|
+
[](https://github.com/dartsort/dartsort.github.io/actions/workflows/pages/pages-build-deployment)
|
|
5
|
+
[](https://coveralls.io/github/cwindolf/dartsort)
|
|
6
|
+
[](https://doi.org/10.5281/zenodo.16943074)
|
|
7
|
+
[](https://pypi.org/p/dartsort)
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
# dartsort
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
*dartsort* is a modular spike sorter built around a statistical clustering model and a new approach to probe motion.
|
|
14
|
+
It is also a toolkit of modules for building spike sorters and other analyses of electrophysiology data.
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
## :warning: work in progress :warning:
|
|
18
|
+
|
|
19
|
+
We do not currently recommend DARTsort for production spike sorting purposes.
|
|
20
|
+
Please feel free to open an issue or a discussion if you run into problems.
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
## Installation
|
|
24
|
+
|
|
25
|
+
### Installing into an existing environment
|
|
26
|
+
|
|
27
|
+
If you already have a Python environment with PyTorch working and you just want to install *dartsort* there, use
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
$ pip install dartsort
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
If you want to run the test suite or use `dartsort.vis`, you can install the optional dependencies with `pip install dartsort[test,vis]`.
|
|
34
|
+
|
|
35
|
+
## Setting up a Python environment
|
|
36
|
+
|
|
37
|
+
If you need to set up Python or PyTorch, I find that a [`conda-forge`](https://conda-forge.org/)-based distribution is the most reliable at installing the GPU dependencies which PyTorch needs (note: `conda-forge` is different from the non-free Anaconda).
|
|
38
|
+
|
|
39
|
+
You can use `conda-forge` to install Python, `dartsort`, and its dependencies as follows:
|
|
40
|
+
|
|
41
|
+
- Follow the `conda-forge` installation instructions for your platform at [https://conda-forge.org/download/](https://conda-forge.org/download/)
|
|
42
|
+
- Create an environment with
|
|
43
|
+
```sh
|
|
44
|
+
$ mamba env create -f environment.yml
|
|
45
|
+
```
|
|
46
|
+
This will create an environment called `dartsort`, but you can change the name by adding `-n othername`.
|
|
47
|
+
- Activate the environment:
|
|
48
|
+
```sh
|
|
49
|
+
$ mamba activate dartsort
|
|
50
|
+
```
|
|
51
|
+
- Install `dartsort` and the rest of its dependencies by running the pip command [above](#installing-into-an-existing-environment)
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
## Usage
|
|
55
|
+
|
|
56
|
+
For more detailed documentation of the main function parameters and configuration options, see the [main API documentation](main_api.md).
|
|
57
|
+
|
|
58
|
+
### As a Python function
|
|
59
|
+
|
|
60
|
+
*dartsort* can be run in Python with:
|
|
61
|
+
|
|
62
|
+
```python
|
|
63
|
+
import dartsort
|
|
64
|
+
|
|
65
|
+
dartsort.dartsort(recording, output_dir)
|
|
66
|
+
|
|
67
|
+
# or, to set configuration options, something like...
|
|
68
|
+
dartsort_result = dartsort.dartsort(
|
|
69
|
+
recording,
|
|
70
|
+
output_dir,
|
|
71
|
+
cfg=dartsort.DARTsortUserConfig(
|
|
72
|
+
preprocessing="ibllike",
|
|
73
|
+
work_in_tmpdir=True,
|
|
74
|
+
copy_recording_to_tmpdir=True,
|
|
75
|
+
),
|
|
76
|
+
)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Please** read the [important configuration details section](#important-configuration-details) below for information on the parameters of the `DARTsortUserConfig`.
|
|
80
|
+
Some of them, like `preprocessing`, are not set by default and need your input!
|
|
81
|
+
(This will probably change.)
|
|
82
|
+
|
|
83
|
+
Here, `recording` is a [SpikeInterface][SpikeInterface] recording object (see their tutorial on [reading various recording formats](https://spikeinterface.readthedocs.io/en/stable/how_to/read_various_formats.html)).
|
|
84
|
+
`output_dir` is the folder where *dartsort* will save its output.
|
|
85
|
+
There are more details on these and the rest of the arguments in the [main API documentation](main_api.md).
|
|
86
|
+
|
|
87
|
+
Once you've run *dartsort*, you might want to check out [the outputs and exporting](#outputs-and-exporting) section below.
|
|
88
|
+
|
|
89
|
+
### Important configuration details
|
|
90
|
+
|
|
91
|
+
Before running *dartsort*, please be aware of the following important configuration options.
|
|
92
|
+
|
|
93
|
+
- `preprocessing`: *dartsort* won't touch your data by default (`preprocessing="none"`), leaving you free to implement your own preprocessing in SpikeInterface or otherwise. However, *dartsort* expects standardized input, and therefore it will explode if you don't set this flag and your data is in its original state (for instance, the raw `int16` data off the probe).
|
|
94
|
+
- For a cheap but sensible default, try `preprocessing="ibllikecmr"`, which applies a pipeline similar to that of [the IBL][iblsorting] but with global median common referencing instead of their spatial highpass filter. `preprocessing="ibllike"` will use their spatial highpass filter.
|
|
95
|
+
- `do_motion_estimation=True` by default, but you may like to disable it if you know that there is (say) less than 5 microns of total drift in your recording, or if you have handled this in your own preprocessing.
|
|
96
|
+
- `work_in_tmpdir` and `copy_recording_to_tmpdir` can be helpful in some cases where slow network drives are involved.
|
|
97
|
+
|
|
98
|
+
### Outputs and exporting
|
|
99
|
+
|
|
100
|
+
The `dartsort_result = dartsort(...)` function returns a dictionary `dartsort_result` containing a `DARTsortSorting` object: `sorting = dartsort_result["sorting"]`.
|
|
101
|
+
This object has all the spike train data attached (as arrays under property names `.times_samples` and `.times_seconds` for spike times in samples and seconds, `.labels` for unit labels, and many others; `print(sorting)` to see some more).
|
|
102
|
+
|
|
103
|
+
If you already ran *dartsort* and want to load the output spike trains, use `dartsort.load(output_dir)` to get the `DARTsortSorting` object.
|
|
104
|
+
|
|
105
|
+
This object can also export itself to other formats:
|
|
106
|
+
|
|
107
|
+
- For a SpikeInterface `NumpySorting` object, use `sorting.to_numpy_sorting()`
|
|
108
|
+
- For a [Pynapple][Pynapple] `TsGroup`, use `sorting.to_tsgroup()`
|
|
109
|
+
- To export to Phy, we currently suggest bridging through SpikeInterface. Start with `sorting.to_numpy_sorting()` and follow the instructions [in SpikeInterface's documentation](https://spikeinterface.readthedocs.io/en/stable/modules/exporters.html#exporting-to-phy) for first creating a `SortingAnalyzer` and then exporting that to Phy.
|
|
110
|
+
- For a simple dictionary of numpy arrays, use `dict = sorting.spike_feature_dict`.
|
|
111
|
+
- For pandas, use `sorting.to_pandas()`.
|
|
112
|
+
|
|
113
|
+
*dartsort* also saves motion information, returned as `dartsort_result["motion"]` or loaded after the fact as `dartsort.try_load_motion_info(output_dir)`.
|
|
114
|
+
|
|
115
|
+
The data is saved to `output_dir` in the following files:
|
|
116
|
+
- `dartsort_sorting.npz`: This NPZ file contains the final spike train under the keys `times_samples`, `channels`, and `labels`.
|
|
117
|
+
- `matching1.h5`: This HDF5 file contains spike features and other data from the last matching step. Amplitudes, localizations, and other features live in here; use `h5ls` on the command line to see what's in there. Be aware that the `labels` dataset in this HDF5 is not the same as what's saved in the `dartsort_sorting.npz`.
|
|
118
|
+
- `motion_info.pkl` is a pickled `MotionInfo` object.
|
|
119
|
+
- There may be a models/ folder containing PyTorch weights files with modeling quantities (for instance, featurization SVD bases, localization neural nets, Gaussian mixture model parameters). If you want to load these up, feel free to reach out for help.
|
|
120
|
+
|
|
121
|
+
### Visualization
|
|
122
|
+
|
|
123
|
+
To make some basic visualizations of the sorting result with matplotlib, try:
|
|
124
|
+
|
|
125
|
+
```python
|
|
126
|
+
import dartsort.vis as dartvis
|
|
127
|
+
|
|
128
|
+
# gather outputs from dartsort
|
|
129
|
+
dartsort_result = dartsort(recording, output_dir, ...)
|
|
130
|
+
sorting = dartsort_result["sorting"]
|
|
131
|
+
motion = dartsort_result["motion"]
|
|
132
|
+
|
|
133
|
+
# or, if you already ran it
|
|
134
|
+
sorting = dartsort.load(output_dir)
|
|
135
|
+
motion = dartsort.try_load_motion_info(output_dir)
|
|
136
|
+
|
|
137
|
+
dartvis.visualize_sorting(
|
|
138
|
+
recording,
|
|
139
|
+
sorting,
|
|
140
|
+
vis_save_dir,
|
|
141
|
+
motion=motion,
|
|
142
|
+
make_unit_summaries=False,
|
|
143
|
+
)
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Set `make_unit_summaries=True` to create a summary plot for each unit.
|
|
147
|
+
|
|
148
|
+
### Command-line interface
|
|
149
|
+
|
|
150
|
+
Try running
|
|
151
|
+
|
|
152
|
+
```sh
|
|
153
|
+
$ dartsort -h
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
on your command line to see usage instructions; parameters can be configured on the command line or read from a TOML file.
|
|
157
|
+
|
|
158
|
+
## Troubleshooting
|
|
159
|
+
|
|
160
|
+
Please let us know if you run into any issues.
|
|
161
|
+
If you feel that the issue is a software bug, feel free to open an issue or a discussion on GitHub.
|
|
162
|
+
If it's more of a data-related or methodology thing, feel free to use the email on my GitHub [profile](https://github.com/cwindolf).
|
|
163
|
+
|
|
164
|
+
## References
|
|
165
|
+
|
|
166
|
+
- [SpikeInterface][SpikeInterface]
|
|
167
|
+
- DREDge [on PyPI][dredge-pypi] and the DREDge [paper][dredge-nm] ([PDF link][dredge-rdcb]).
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+
[SpikeInterface]: https://spikeinterface.readthedocs.io
|
|
171
|
+
[iblsorting]: https://figshare.com/articles/online_resource/Spike_sorting_pipeline_for_the_International_Brain_Laboratory/19705522
|
|
172
|
+
[Pynapple]: https://pynapple.org/
|
|
173
|
+
[dredge-pypi]: https://pypi.org/p/dredge-ephys/
|
|
174
|
+
[dredge-nm]: https://www.nature.com/articles/s41592-025-02614-5
|
|
175
|
+
[dredge-rdcb]: https://rdcu.be/ecqZf
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
toc_depth: 3
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Main API: `dartsort()` and the `DARTsortUserConfig`
|
|
6
|
+
|
|
7
|
+
This page shows the main functions and objects that you'd run into when spike sorting with *dartsort*.
|
|
8
|
+
|
|
9
|
+
For details on parameters you should think about before running the sorter, see the [important configuration details note](/#important-configuration-details).
|
|
10
|
+
|
|
11
|
+
## Main function: `dartsort()`
|
|
12
|
+
|
|
13
|
+
::: dartsort.dartsort
|
|
14
|
+
options:
|
|
15
|
+
handler: python
|
|
16
|
+
show_signature: true
|
|
17
|
+
separate_signature: true
|
|
18
|
+
|
|
19
|
+
The return value from the `dartsort()` function is a DARTsortReturn object, which is a dictionary containing spike trains and motion information:
|
|
20
|
+
|
|
21
|
+
::: dartsort.DARTsortReturn
|
|
22
|
+
options:
|
|
23
|
+
show_bases: false
|
|
24
|
+
|
|
25
|
+
The spike trains and motion info are stored in some internal objects, described in the [last section on this page](#data-objects). Up next, we'll discuss how to adjust *dartsort*'s parameters.
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
## Configuration
|
|
29
|
+
|
|
30
|
+
For details on parameters you should know about before running the sorter, see the [important configuration details note](/#important-configuration-details).
|
|
31
|
+
Here, we'll show a reference for all of the configuration options.
|
|
32
|
+
Some of the important ones not mentioned in that note include: the [voltage threshold](#dartsort.DARTsortUserConfig.voltage_threshold) for initial spike detection and the "energy" thresholds for spike detection in the [initial](#dartsort.DARTsortUserConfig.initial_threshold) and [template matching](#dartsort.DARTsortUserConfig.matching_threshold) passes.
|
|
33
|
+
You may also want to take a look at the paramters for
|
|
34
|
+
|
|
35
|
+
::: dartsort.DARTsortUserConfig
|
|
36
|
+
options:
|
|
37
|
+
show_if_no_docstring: true
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
## Data objects
|
|
42
|
+
|
|
43
|
+
*dartsort* uses some internal classes to represent spike trains and other data. Spike trains and motion are represented in the following objects.
|
|
44
|
+
|
|
45
|
+
::: dartsort.DARTsortSorting
|
|
46
|
+
|
|
47
|
+
::: dartsort.MotionInfo
|