moseq2-app 1.4.0__tar.gz

moseq2_app-1.4.0/LICENSE.md

@@ -0,0 +1,49 @@
+# Non-Commercial Research and Academic Use Software License and Terms of Use
+
+
+MoSeq is a software package that includes original code created by the Harvard researchers listed below (the “Software”), and third-party code that may be obtained by End Users separately. The Software is designed to label behaviors identified from 3D pose dynamics data through the use of computational modeling and fitting approaches. The Software was developed by Alexander Wiltschko, Ph.D., Sandeep Robert Datta, Ph.D., and Matthew J. Johnson, Ph.D. at Harvard University. It is distributed for free academic and non-commercial research use by the President and Fellows of Harvard College (“Harvard”).
+
+Using the Software indicates your agreement to be bound by the terms of this Software Use Agreement (“Agreement”). Absent your agreement to the terms below, you (the “End User”) have no rights to hold or use the Software whatsoever.
+
+Harvard agrees to grant hereunder a limited non-exclusive license to End User for the use of the Software in the performance of End User’s internal, non-commercial research and academic use at End User’s academic or not-for-profit research institution (“Institution”) on the following terms and conditions:
+
+1. **NO COMMERCIAL USE.** End User shall not use the Software for Commercial use and any such use of the Software is expressly prohibited. “Commercial use” includes, but is not limited to, (i) use of the Software in fee-for-service arrangements, (ii) use of the Software by core facilities or laboratories to provide research services to (or in collaboration with) for-profit third parties for a fee, and (iii) use of the Software in industry-sponsored and/or collaborative research projects in which any commercial rights are granted to the sponsor or collaborator. If End User wishes to use the Software for Commercial use, End User must execute a separate license agreement with Harvard.
+
+_Requests for use of the Software for Commercial use, please contact:_
+
+Office of Technology Development
+Harvard University
+Smith Campus Center, Suite 727E
+1350 Massachusetts Avenue Cambridge, MA 02138 USA Telephone: (617) 495-3067
+Facsimile: (617) 495-9568
+E-mail: otd@harvard.edu
+
+2. **OWNERSHIP AND COPYRIGHT NOTICE.** Harvard owns all intellectual property in the Software. End User shall gain no ownership to the Software.  End User shall not remove or delete, and shall retain in the Software (including in any modifications to the Software and in any Derivative Works), the copyright, trademark, or other notices pertaining to Software as provided with the Software.
+
+3. **DERIVATIVE WORKS.** End User may create use and distribute Derivative Works, as such term is defined under U.S. copyright laws, provided that any such Derivative Works shall be restricted to non-commercial research and academic use. End User may not distribute Derivative Works to any for-profit third parties for Commercial use.
+
+4. **FEEDBACK.** In order to improve the Software, comments from End Users may be useful. End User agrees to provide Harvard with feedback on the End User’s use of the Software (e.g., any bugs in the Software, the user experience, etc.).  Harvard is permitted to use such information provided by End User in making changes and improvements to the Software without compensation or accounting to End User.
+
+5. **NON ASSERT.** End User acknowledges that Harvard may develop modifications to the Software that may be based on the feedback provided by End User under Section 5 above. Harvard shall not be restricted in any way by End User regarding its use of such information.  End User acknowledges the right of Harvard to prepare, publish, display, reproduce, transmit and or use modifications to the Software that may be substantially similar or functionally equivalent to End User’s modifications and/or improvements if any.  In the event that End User obtains patent protection for any modification or improvement to Software, End User agrees not to allege or enjoin infringement of End User’s patent against Harvard, or any of the researchers, medical or research staff, officers, directors and employees of those institutions.
+
+6. **PUBLICATION & ATTRIBUTION.** End User has the right to publish, present, or share results from the use of the Software.  In accordance with customary academic practice, End User will acknowledge Harvard as the provider of the Software and may cite the relevant reference(s) from the following list of publications:
+
+Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JE, Pashkovski SL, Abraira VE, Adams RP and Datta SR. (2015). Mapping sub-second Structure in Behavior. Neuron, 88:1121.
+
+Markowitz JE, Gillis WF, Beron CC, Neufeld SQ, Robertson K, Bhagat ND, Peterson RE, Peterson E, Hyun M, Linderman SW, Sabatini BL and Datta SR. (2018). The striatum organizes 3D behavior via moment-to-moment action selection. Cell, in press.
+
+7.** NO WARRANTIES.** THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST EXTENT PERMITTED BY LAW, HARVARD HEREBY DISCLAIMS ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT.  HARVARD MAKES NO WARRANTY ABOUT THE ACCURACY, RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE SOFTWARE.  HARVARD DOES NOT WARRANT THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION.
+
+8. **LIMITATIONS OF LIABILITY AND REMEDIES.** USE OF THE SOFTWARE IS AT END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, ITS EXCLUSIVE REMEDY IS TO STOP USING IT.  IN NO EVENT SHALL HARVARD BE LIABLE TO END USER OR ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH THE SOFTWARE, EVEN IF HARVARD IS NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD IS ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+9. **INDEMNIFICATION.** To the extent permitted by law, End User shall indemnify, defend and hold harmless Harvard, their corporate affiliates, current or future directors, trustees, officers, faculty, medical and professional staff, employees, students and agents and their respective successors, heirs and assigns (the "Indemnitees"), against any liability, damage, loss or expense (including reasonable attorney's fees and expenses of litigation) incurred by or imposed upon the Indemnitees or any one of them in connection with any claims, suits, actions, demands or judgments arising from End User’s breach of this Agreement or its Institution’s use of the Software except to the extent caused by the gross negligence or willful misconduct of Harvard. This indemnification provision shall survive expiration or termination of this Agreement.
+
+10. **GOVERNING LAW.** This Agreement shall be construed and governed by the laws of the Commonwealth of Massachusetts regardless of otherwise applicable choice of law standards.
+
+11. **NON-USE OF NAME.**  Nothing in this License and Terms of Use shall be construed as granting End Users or their Institutions any rights or licenses to use any trademarks, service marks or logos associated with the Software.  End User may not use the terms “Harvard” (or a substantially similar term) in any way that is inconsistent with the permitted uses described herein. End Users may not use any name or emblem of Harvard or any of its schools or subdivisions for any purpose, or to falsely suggest any relationship between End User (or its Institution) and Harvard, or in any manner that would infringe or violate any of its rights.
+
+12. End User represents and warrants that it has the legal authority to enter into this License and Terms of Use on behalf of itself and its Institution.
+
+                                                                  ***      
+
+

moseq2_app-1.4.0/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.4
+Name: moseq2-app
+Version: 1.4.0
+Summary: Interactive data exploration tools for the MoSeq2 pipeline.
+Author: Datta Lab
+License-Expression: LicenseRef-Harvard-Academic-Non-Commercial
+License-File: LICENSE.md
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.10
+Requires-Dist: bokeh>=2.4.2
+Requires-Dist: fastparquet>=0.4.1
+Requires-Dist: h5py>=2.10.0
+Requires-Dist: holoviews>=1.14.7
+Requires-Dist: ipython>=7.14.0
+Requires-Dist: ipywidgets>=7.5.1
+Requires-Dist: jinja2>=3.0.1
+Requires-Dist: joblib>=0.15.1
+Requires-Dist: jupyter-bokeh>=2.0.3
+Requires-Dist: jupyter>=1.0.0
+Requires-Dist: matplotlib>=3.1.2
+Requires-Dist: numpy>=1.18.3
+Requires-Dist: opencv-python>=4.1.2.30
+Requires-Dist: pandas>=1.0.5
+Requires-Dist: panel>=0.12.6
+Requires-Dist: plotly>=4.14.3
+Requires-Dist: qgridnext>=2.0.4
+Requires-Dist: ruamel.yaml>=0.16.5
+Requires-Dist: scikit-image>=0.16.2
+Requires-Dist: scikit-learn>=0.20.3
+Requires-Dist: scipy>=1.3.2
+Requires-Dist: seaborn>=0.11.0
+Requires-Dist: moseq2-extract<2,>=1.4.0
+Requires-Dist: moseq2-model<2,>=1.4.0
+Requires-Dist: moseq2-pca<2,>=1.4.0
+Requires-Dist: moseq2-viz<2,>=1.4.0
+Requires-Dist: Cython>=3.0.11
+Requires-Dist: requests>=2.32.3
+Provides-Extra: tests
+Requires-Dist: pytest>=8.3.2; extra == "tests"
+Requires-Dist: pytest-cov>=5.0.0; extra == "tests"
+Requires-Dist: pytest-regressions>=2.5.0; extra == "tests"
+Requires-Dist: pooch>=1.8.2; extra == "tests"
+Description-Content-Type: text/markdown
+
+<!-- [![Build Status](https://app.travis-ci.com/dattalab/moseq2-app.svg?token=ptXpSa3Fp9PKqkxJDkDr&branch=dev)](https://app.travis-ci.com/dattalab/moseq2-app) -->
+<p align="center">
+  <img src="https://drive.google.com/uc?export=view&id=1oc0_0mlN0VTZEPMQTg_hnYAC87Lb58MI" />
+</p>
+
+Welcome to the `moseq2-app` repository. Motion Sequencing (MoSeq) is an unsupervised machine learning method used to describe mouse behavior and `moseq2-app` is the starting point to MoSeq2 package suite.
+
+ To get started, install it with `pip install moseq2-app` and head over to the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Installation) for the full procedure and documentation for the MoSeq2 package suite.
+
+**Quick Links:**
+[![MoSeq Slack Channel](https://img.shields.io/badge/slack-MoSeq-blue.svg?logo=slack)](https://moseqworkspace.slack.com)
+[![Documentation](https://img.shields.io/badge/docs-GitHub%20Wiki-blue)](https://github.com/dattalab/moseq2-app/wiki)
+[![DOI](https://zenodo.org/badge/204555901.svg)](https://zenodo.org/badge/latestdoi/204555901)
+
+
+# Overview
+
+<!---Adapted from 
+Datta, Sandeep Robert, David J. Anderson, Kristin Branson, Pietro Perona, and Andrew Leifer. 2019. “Computational Neuroethology: A Call to Action.” Neuron 104 (1): 11–24.
+-->
+**MoSeq takes 3D depth videos as input (obtained using commercially-available sensors) and then uses statistical learning techniques to identify the components of mouse body language.** This is achieved by fitting an autoregressive hidden Markov model that parses behavior into a set of sub-second motifs called syllables. This segmentation naturally yields boundaries between syllables, and therefore also reveals the structure that governs the interconnections between syllables over time, which we refer to as behavioral grammar (see [Wiltschko et al., 2015](https://www.sciencedirect.com/science/article/pii/S0896627315010375) for the first description of MoSeq). 
+
+Because MoSeq relies on unsupervised machine learning, MoSeq discovers the set of syllables and grammar expressed in any given experiment. By combining MoSeq with electrophysiology, multi-color photometry, and miniscope methods, neural correlates for 3D behavioral syllables have recently been identified in the dorsolateral striatum (DLS) ([Markowitz et al., 2018](https://www.sciencedirect.com/science/article/pii/S0092867418305129)). Furthermore, MoSeq has been combined with optogenetic stimulation to reveal the differential consequences of activating the motor cortex, the dorsal striatum, and the ventral striatum ([Pisanello et al.,2017](https://www.nature.com/articles/nn.4591); [Wiltschko et al., 2015](https://www.sciencedirect.com/science/article/pii/S0896627315010375)). These results are consistent with similar results recently obtained using marker-based approaches to explore the relationship between 3D posture and activity in the posterior parietal cortex ([Mimica et al., 2018](https://www.science.org/doi/10.1126/science.aau2013)).
+
+There are two basic steps to MoSeq, and this GitHub repository and the wiki supports both. First, 3D data need to be acquired. Here you will find instructions for assembling a standard MoSeq data acquisition platform, and code to acquire 3D videos of mice as they freely behave. Second, these 3D data need to be modeled. We provide several different methods for modeling the data using MoSeq, which are compared below. We continue development of MoSeq and plan to incorporate additional features in the near future. 
+
+## MoSeq2 package suite
+The MoSeq2 toolkit enables users to model mouse behavior across different experimental groups, and measure the differences between their behavior usages, durations, transition patterns. etc.
+
+This package contains functionalities that can be used interactively in jupyter notebooks. 
+We provide a series of Jupyter Notebooks that cover the entire MoSeq pipeline to process their depth videos of mice, and segment their behavior into what is denoted as "syllables". In addition to the Jupyter notebooks, MoSeq has a Command Line Interface. Consult the wiki page for more detailed documentation of the MoSeq pipeline [here](https://github.com/dattalab/moseq2-app/wiki).
+
+# Getting Started
+
+```bash
+pip install moseq2-app
+```
+
+Python 3.10 or newer. Verified on Linux; macOS and Windows support is being
+verified. Installing `moseq2-app` brings the rest of the suite with it:
+`moseq2-extract`, `moseq2-pca`, `moseq2-model` and `moseq2-viz`, along with
+their command line interfaces.
+
+**ffmpeg is a separate requirement.** `moseq2-extract` reads and writes depth
+video by calling `ffmpeg` and `ffprobe` and expects to find them on your
+`PATH`. No Python install provides them, so install them yourself with conda,
+apt or Homebrew.
+
+**The notebooks are not part of the package.** Download the two that make up
+the pipeline and put them in the parent directory of your data, which is what
+lets the interactive video widgets load:
+
+- [MoSeq2-Extract-Modeling-Notebook.ipynb](https://raw.githubusercontent.com/dattalab/moseq2-app/release/notebooks/MoSeq2-Extract-Modeling-Notebook.ipynb)
+- [MoSeq2-Analysis-Visualization-Notebook.ipynb](https://raw.githubusercontent.com/dattalab/moseq2-app/release/notebooks/MoSeq2-Analysis-Visualization-Notebook.ipynb)
+
+### **[The installation guide](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Installation) carries the whole procedure, including how to launch Jupyter so the notebooks' video widgets work.**
+
+You can find more information about the MoSeq2 package suite, the acquisition and analysis pipeline, documentation for Command Line Interface (CLI), tutorials, etc. in the [wiki](https://github.com/dattalab/moseq2-app/wiki). If you want to explore MoSeq functionalities, check out the guide for [downloading test data](https://github.com/dattalab/moseq2-app/wiki/Download-Test-Data).
+
+# Community Support and Contributing
+- Please join [![MoSeq Slack Channel](https://img.shields.io/badge/slack-MoSeq-blue.svg?logo=slack)](https://moseqworkspace.slack.com) to post questions and interactive with MoSeq developers and users.
+- If you encounter bugs, errors or issues, please submit a Bug report [here](https://github.com/dattalab/moseq2-app/issues/new/choose). We encourage you to check out the [troubleshooting and tips section](https://github.com/dattalab/moseq2-app/wiki/Troubleshooting-and-Tips) and search your issues in [the existing issues](https://github.com/dattalab/moseq2-app/issues) first.   
+- If you want to see certain features in MoSeq or you have new ideas, please submit a Feature request [here](https://github.com/dattalab/moseq2-app/issues/new/choose).
+- If you want to contribute to our codebases, please check out our [Developer Guidelines](https://github.com/dattalab/moseq2-app/wiki/MoSeq-Developer-Guidelines).
+- Please tell us what you think by filling out [this user survey](https://forms.gle/S88jptAEs41mQjff7).
+
+# Versions
+- [Changelog](https://github.com/dattalab/moseq2-app/wiki/Changelog)
+
+# Publications
+- [Mapping Sub-Second Structure in Mouse Behavior](http://datta.hms.harvard.edu/wp-content/uploads/2018/01/pub_23.pdf)
+- [The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection](http://datta.hms.harvard.edu/wp-content/uploads/2019/06/Markowitz.final_.pdf)
+- [Q&A: Understanding the composition of behavior](http://datta.hms.harvard.edu/wp-content/uploads/2019/06/Datta-QA.pdf)
+- [Computational Neuroethology: A Call to Action]("http://datta.hms.harvard.edu/wp-content/uploads/2020/01/AC2A.pdf")
+- [Revealing the structure of pharmacobehavioral space through motion sequencing](https://www.nature.com/articles/s41593-020-00706-3)
+
+# License
+
+MoSeq is freely available for academic use under a license provided by Harvard University. Please refer to the license file for details. If you are interested in using MoSeq for commercial purposes please contact Bob Datta directly at srdatta@hms.harvard.edu, who will put you in touch with the appropriate people in the Harvard Technology Transfer office.

moseq2_app-1.4.0/README.md

@@ -0,0 +1,79 @@
+<!-- [![Build Status](https://app.travis-ci.com/dattalab/moseq2-app.svg?token=ptXpSa3Fp9PKqkxJDkDr&branch=dev)](https://app.travis-ci.com/dattalab/moseq2-app) -->
+<p align="center">
+  <img src="https://drive.google.com/uc?export=view&id=1oc0_0mlN0VTZEPMQTg_hnYAC87Lb58MI" />
+</p>
+
+Welcome to the `moseq2-app` repository. Motion Sequencing (MoSeq) is an unsupervised machine learning method used to describe mouse behavior and `moseq2-app` is the starting point to MoSeq2 package suite.
+
+ To get started, install it with `pip install moseq2-app` and head over to the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Installation) for the full procedure and documentation for the MoSeq2 package suite.
+
+**Quick Links:**
+[![MoSeq Slack Channel](https://img.shields.io/badge/slack-MoSeq-blue.svg?logo=slack)](https://moseqworkspace.slack.com)
+[![Documentation](https://img.shields.io/badge/docs-GitHub%20Wiki-blue)](https://github.com/dattalab/moseq2-app/wiki)
+[![DOI](https://zenodo.org/badge/204555901.svg)](https://zenodo.org/badge/latestdoi/204555901)
+
+
+# Overview
+
+<!---Adapted from 
+Datta, Sandeep Robert, David J. Anderson, Kristin Branson, Pietro Perona, and Andrew Leifer. 2019. “Computational Neuroethology: A Call to Action.” Neuron 104 (1): 11–24.
+-->
+**MoSeq takes 3D depth videos as input (obtained using commercially-available sensors) and then uses statistical learning techniques to identify the components of mouse body language.** This is achieved by fitting an autoregressive hidden Markov model that parses behavior into a set of sub-second motifs called syllables. This segmentation naturally yields boundaries between syllables, and therefore also reveals the structure that governs the interconnections between syllables over time, which we refer to as behavioral grammar (see [Wiltschko et al., 2015](https://www.sciencedirect.com/science/article/pii/S0896627315010375) for the first description of MoSeq). 
+
+Because MoSeq relies on unsupervised machine learning, MoSeq discovers the set of syllables and grammar expressed in any given experiment. By combining MoSeq with electrophysiology, multi-color photometry, and miniscope methods, neural correlates for 3D behavioral syllables have recently been identified in the dorsolateral striatum (DLS) ([Markowitz et al., 2018](https://www.sciencedirect.com/science/article/pii/S0092867418305129)). Furthermore, MoSeq has been combined with optogenetic stimulation to reveal the differential consequences of activating the motor cortex, the dorsal striatum, and the ventral striatum ([Pisanello et al.,2017](https://www.nature.com/articles/nn.4591); [Wiltschko et al., 2015](https://www.sciencedirect.com/science/article/pii/S0896627315010375)). These results are consistent with similar results recently obtained using marker-based approaches to explore the relationship between 3D posture and activity in the posterior parietal cortex ([Mimica et al., 2018](https://www.science.org/doi/10.1126/science.aau2013)).
+
+There are two basic steps to MoSeq, and this GitHub repository and the wiki supports both. First, 3D data need to be acquired. Here you will find instructions for assembling a standard MoSeq data acquisition platform, and code to acquire 3D videos of mice as they freely behave. Second, these 3D data need to be modeled. We provide several different methods for modeling the data using MoSeq, which are compared below. We continue development of MoSeq and plan to incorporate additional features in the near future. 
+
+## MoSeq2 package suite
+The MoSeq2 toolkit enables users to model mouse behavior across different experimental groups, and measure the differences between their behavior usages, durations, transition patterns. etc.
+
+This package contains functionalities that can be used interactively in jupyter notebooks. 
+We provide a series of Jupyter Notebooks that cover the entire MoSeq pipeline to process their depth videos of mice, and segment their behavior into what is denoted as "syllables". In addition to the Jupyter notebooks, MoSeq has a Command Line Interface. Consult the wiki page for more detailed documentation of the MoSeq pipeline [here](https://github.com/dattalab/moseq2-app/wiki).
+
+# Getting Started
+
+```bash
+pip install moseq2-app
+```
+
+Python 3.10 or newer. Verified on Linux; macOS and Windows support is being
+verified. Installing `moseq2-app` brings the rest of the suite with it:
+`moseq2-extract`, `moseq2-pca`, `moseq2-model` and `moseq2-viz`, along with
+their command line interfaces.
+
+**ffmpeg is a separate requirement.** `moseq2-extract` reads and writes depth
+video by calling `ffmpeg` and `ffprobe` and expects to find them on your
+`PATH`. No Python install provides them, so install them yourself with conda,
+apt or Homebrew.
+
+**The notebooks are not part of the package.** Download the two that make up
+the pipeline and put them in the parent directory of your data, which is what
+lets the interactive video widgets load:
+
+- [MoSeq2-Extract-Modeling-Notebook.ipynb](https://raw.githubusercontent.com/dattalab/moseq2-app/release/notebooks/MoSeq2-Extract-Modeling-Notebook.ipynb)
+- [MoSeq2-Analysis-Visualization-Notebook.ipynb](https://raw.githubusercontent.com/dattalab/moseq2-app/release/notebooks/MoSeq2-Analysis-Visualization-Notebook.ipynb)
+
+### **[The installation guide](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Installation) carries the whole procedure, including how to launch Jupyter so the notebooks' video widgets work.**
+
+You can find more information about the MoSeq2 package suite, the acquisition and analysis pipeline, documentation for Command Line Interface (CLI), tutorials, etc. in the [wiki](https://github.com/dattalab/moseq2-app/wiki). If you want to explore MoSeq functionalities, check out the guide for [downloading test data](https://github.com/dattalab/moseq2-app/wiki/Download-Test-Data).
+
+# Community Support and Contributing
+- Please join [![MoSeq Slack Channel](https://img.shields.io/badge/slack-MoSeq-blue.svg?logo=slack)](https://moseqworkspace.slack.com) to post questions and interactive with MoSeq developers and users.
+- If you encounter bugs, errors or issues, please submit a Bug report [here](https://github.com/dattalab/moseq2-app/issues/new/choose). We encourage you to check out the [troubleshooting and tips section](https://github.com/dattalab/moseq2-app/wiki/Troubleshooting-and-Tips) and search your issues in [the existing issues](https://github.com/dattalab/moseq2-app/issues) first.   
+- If you want to see certain features in MoSeq or you have new ideas, please submit a Feature request [here](https://github.com/dattalab/moseq2-app/issues/new/choose).
+- If you want to contribute to our codebases, please check out our [Developer Guidelines](https://github.com/dattalab/moseq2-app/wiki/MoSeq-Developer-Guidelines).
+- Please tell us what you think by filling out [this user survey](https://forms.gle/S88jptAEs41mQjff7).
+
+# Versions
+- [Changelog](https://github.com/dattalab/moseq2-app/wiki/Changelog)
+
+# Publications
+- [Mapping Sub-Second Structure in Mouse Behavior](http://datta.hms.harvard.edu/wp-content/uploads/2018/01/pub_23.pdf)
+- [The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection](http://datta.hms.harvard.edu/wp-content/uploads/2019/06/Markowitz.final_.pdf)
+- [Q&A: Understanding the composition of behavior](http://datta.hms.harvard.edu/wp-content/uploads/2019/06/Datta-QA.pdf)
+- [Computational Neuroethology: A Call to Action]("http://datta.hms.harvard.edu/wp-content/uploads/2020/01/AC2A.pdf")
+- [Revealing the structure of pharmacobehavioral space through motion sequencing](https://www.nature.com/articles/s41593-020-00706-3)
+
+# License
+
+MoSeq is freely available for academic use under a license provided by Harvard University. Please refer to the license file for details. If you are interested in using MoSeq for commercial purposes please contact Bob Datta directly at srdatta@hms.harvard.edu, who will put you in touch with the appropriate people in the Harvard Technology Transfer office.

moseq2_app-1.4.0/moseq2_app/__init__.py

@@ -0,0 +1 @@
+__version__ = "1.4.0"

moseq2_app-1.4.0/moseq2_app/_yaml.py

@@ -0,0 +1,73 @@
+"""YAML reading and writing, in one place.
+
+``ruamel.yaml`` removed the module-level ``safe_load``/``safe_dump`` functions in
+0.18 in favour of instantiating a parser. Routing every call through here keeps
+that detail in one file rather than at each of the call sites.
+
+The parser stays ``ruamel``, deliberately. ``ruamel``'s safe loader is YAML 1.2;
+PyYAML is YAML 1.1 and reads unquoted ``yes``/``no``/``on``/``off`` as booleans
+and ``06:00:00`` as a sexagesimal integer. moseq2 config and index files contain
+both shapes, so switching parsers would quietly change how existing user files
+are read.
+"""
+
+import io
+
+from ruamel.yaml import YAML
+
+
+def _parser():
+    """A YAML parser configured the way ``safe_dump`` was.
+
+    A new one per call, because a ``YAML`` instance carries a serializer across
+    calls: if a value cannot be represented, the failed call leaves that
+    serializer open and every later dump writes nothing at all, silently.
+    Three of the four callers here move their output over a file the user
+    already has, so a dumper that quietly produces nothing destroys data.
+
+    ``pure=True`` selects the Python implementation, matching what the
+    module-level functions used.
+
+    The two assignments restore emitter behaviour that ``YAML()`` does not
+    share with ``safe_dump``. ``allow_unicode`` escapes non-ASCII rather than
+    writing it literally, which also keeps a write from depending on the
+    locale encoding of the file handle. ``width`` is 80 in both, but the two
+    versions measure the column at which a flow sequence may break two
+    characters apart, so 82 here reproduces where ``safe_dump`` put its line
+    breaks -- both for the two-element ``path: [x.h5, x.yaml]`` entries that
+    fill an index file and for sequences long enough to wrap.
+
+    One emitter difference is left, and no configuration reaches it: where a
+    flow sequence does wrap, this version writes a space before the break. The
+    breaks land in the same places, so the files moseq2 writes -- the session
+    index, the project and per-session configs -- come out byte for byte as
+    they did.
+    """
+    parser = YAML(typ="safe", pure=True)
+    parser.allow_unicode = False
+    parser.width = 82
+    return parser
+
+
+def read_yaml(path_or_stream):
+    """Parse a YAML file, given a path or an open stream."""
+    if hasattr(path_or_stream, "read"):
+        return _parser().load(path_or_stream)
+    with open(path_or_stream, "r") as handle:
+        return _parser().load(handle)
+
+
+def write_yaml(data, path_or_stream):
+    """Write ``data`` as YAML, given a path or an open stream."""
+    if hasattr(path_or_stream, "write"):
+        _parser().dump(data, path_or_stream)
+        return
+    with open(path_or_stream, "w") as handle:
+        _parser().dump(data, handle)
+
+
+def dump_yaml(data):
+    """Serialize ``data`` to a YAML string."""
+    stream = io.StringIO()
+    _parser().dump(data, stream)
+    return stream.getvalue()

moseq2_app-1.4.0/moseq2_app/flip/train.py

@@ -0,0 +1,139 @@
+import cv2
+import h5py
+import pickle
+import joblib
+import numpy as np
+from pathlib import Path
+from sklearn.svm import SVC
+from dataclasses import dataclass
+from typing import Optional, Tuple
+from sklearn.decomposition import PCA
+from sklearn.pipeline import make_pipeline
+from sklearn.ensemble import RandomForestClassifier
+from moseq2_extract.extract.proc import clean_frames
+from sklearn.model_selection import cross_val_score, KFold
+from sklearn.preprocessing import StandardScaler, FunctionTransformer
+
+
+@dataclass
+class CleanParameters:
+    prefilter_space: tuple = (3,)
+    prefilter_time: Optional[tuple] = None
+    strel_tail: Tuple[int, int] = (9, 9)
+    strel_min: Tuple[int, int] = (5, 5)
+    iters_tail: Optional[int] = 1
+    iters_min: Optional[int] = None
+    height_threshold: int = 10
+
+
+def create_training_dataset(
+    data_index_path: str, clean_parameters: Optional[CleanParameters] = None
+) -> str:
+    np.random.seed(0)
+    data_index_path = Path(data_index_path)
+    out_path = data_index_path.with_name("training_data.npz")
+    if clean_parameters is None:
+        clean_parameters = CleanParameters()
+
+    # load trainingdata index
+    with open(data_index_path, "rb") as f:
+        session_paths, data_index = pickle.load(f)
+
+    # load frames
+    frames = []
+    for k, v in data_index.items():
+        with h5py.File(session_paths[k], "r") as h5f:
+            for left, _slice in v:
+                frames_subset = h5f["frames"][_slice]
+                if left:
+                    frames_subset = np.rot90(frames_subset, 2, axes=(1, 2))
+                frames.append(frames_subset)
+    frames = np.concatenate(frames, axis=0)
+    # rotate frames
+    frames = np.concatenate((frames, np.rot90(frames, 2, axes=(1, 2))), axis=0)
+
+    flipped = np.zeros((len(frames),), dtype=np.uint8)
+    flipped[len(frames) // 2 :] = 1
+
+    # add some randomly shifted frames
+    shifts = np.random.randint(-5, 5, size=(len(frames), 2))
+    shifted_frames = np.array([np.roll(f, tuple(s), axis=(0, 1)) for f, s in zip(frames, shifts)]).astype(np.uint8)
+
+    # remove noise from frames
+    cleaned_frames = clean_frames(
+        np.where(frames > clean_parameters.height_threshold, frames, 0),
+        clean_parameters.prefilter_space,
+        clean_parameters.prefilter_time,
+        strel_tail=cv2.getStructuringElement(
+            cv2.MORPH_ELLIPSE, clean_parameters.strel_tail
+        ),
+        iters_tail=clean_parameters.iters_tail,
+        strel_min=cv2.getStructuringElement(cv2.MORPH_RECT, clean_parameters.strel_min),
+        iters_min=clean_parameters.iters_min,
+    )
+    frames = np.concatenate((frames, shifted_frames, cleaned_frames), axis=0)
+    print(
+        f"Training data shape: {frames.shape}; memory usage: {frames.nbytes / 1e9 * 4:0.2f} GB"
+    )
+
+    flipped = np.concatenate((flipped, flipped, flipped), axis=0)
+
+    np.savez(out_path, frames=frames, flipped=flipped)
+    return out_path
+
+
+def flatten(array: np.ndarray) -> np.ndarray:
+    return array.reshape(len(array), -1)
+
+
+def batch_apply_pca(frames: np.ndarray, pca: PCA, batch_size: int = 1000) -> np.ndarray:
+    output = []
+    if len(frames) < batch_size:
+        return pca.transform(flatten(frames)).astype(np.float32)
+
+    for arr in np.array_split(frames, len(frames) // batch_size):
+        output.append(pca.transform(flatten(arr)).astype(np.float32))
+    return np.concatenate(output, axis=0).astype(np.float32)
+
+
+def train_classifier(
+    data_path: str,
+    classifier: str = "SVM",
+    n_components: int = 20,
+):
+    """Train a classifier to predict the orientation of a mouse.
+    Parameters:
+        data_path (str): Path to the training data numpy file.
+        classifier (str): Classifier to use. Either 'SVM' or 'RF'.
+        n_components (int): Number of components to keep in PCA."""
+    data = np.load(data_path)
+    frames = data["frames"]
+    flipped = data["flipped"]
+
+    print("Fitting PCA")
+    pca = PCA(n_components=n_components)
+    pca.fit(flatten(frames[-len(frames) // 3:]))
+
+    pipeline = make_pipeline(
+        FunctionTransformer(batch_apply_pca, kw_args={"pca": pca}, validate=False),
+        StandardScaler(),
+        (
+            RandomForestClassifier(n_estimators=150)
+            if classifier == "RF"
+            else SVC(probability=True)
+        ),
+    )
+
+    print("Running cross-validation")
+    accuracy = cross_val_score(
+        pipeline, frames, flipped, cv=KFold(n_splits=4, shuffle=True, random_state=0)
+    )
+    print(f"Held-out model accuracy: {accuracy.mean()}")
+
+    print("Final fitting step")
+    return pipeline.fit(frames, flipped)
+
+
+def save_classifier(clf_pipeline, out_path: str):
+    joblib.dump(clf_pipeline, out_path)
+    print(f"Classifier saved to {out_path}")