slypi 1.0.0__tar.gz

slypi-1.0.0/LICENSE

@@ -0,0 +1,28 @@
+The following license applies to the Slycat software:
+
+  Copyright 2013, 2018 National Technology & Engineering Solutions of 
+  Sandia, LLC (NTESS).  Under the terms of Contract DE-NA0003525 with NTESS, 
+  the U.S. Government retains certain rights in this software.
+
+  Redistribution and use in source and binary forms, with or without
+  modification, are permitted provided that the following conditions are met:
+  * Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+  * Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in the
+  documentation and/or other materials provided with the distribution.
+  * Neither the name of Sandia Corporation nor the
+  names of its contributors may be used to endorse or promote products
+  derived from this software without specific prior written permission.
+
+  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+  ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+  DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION BE LIABLE FOR ANY
+  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+  

slypi-1.0.0/PKG-INFO

@@ -0,0 +1,233 @@
+Metadata-Version: 2.1
+Name: slypi
+Version: 1.0.0
+Summary: Slycat python interface utilties for interacting with the Slycat data analysis and visualization server.
+Home-page: https://github.com/sandialabs/slycat
+Author: Shawn Martin
+Author-email: smartin@sandia.gov
+License: Sandia
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests
+Requires-Dist: requests-kerberos
+Requires-Dist: numpy
+Requires-Dist: scikit-learn
+Requires-Dist: pandas
+Requires-Dist: meshio
+Requires-Dist: imageio[ffmpeg]
+Requires-Dist: matplotlib
+Requires-Dist: ipyparallel
+Requires-Dist: torch
+Requires-Dist: umap-learn
+Requires-Dist: pymks
+
+## SlyPI
+Python Interface for Slycat.
+
+## Description
+Slycat is a web application for interactive visualization of ensemble data (see https://github.com/sandialabs/slycat and https://slycat.readthedocs.io/en/latest).  The SlyPI project contains code that enables you to interact with the Slycat web server through a Python interface.  You can query the Slycat server, create Slycat models, and use the code to integrate your own algorithms into the Slycat pipeline.
+
+## Documentation
+
+Full documentation can be found in the docs folder and can be compiled using Sphinx using
+
+```sh
+$ make html
+```
+
+The resulting documention can be accessed under build from the file index.html.
+
+## Installation
+This code is available at https://pypi.org/projects/slypi and can be installed using
+
+```sh
+$ pip install slypi
+```
+
+If you are working behind a proxy, you might also need, e.g.
+
+```sh
+pip install slypi --proxy your-proxy:your-port
+```
+
+If you are getting SSL certificate errors, you can use:
+
+```sh
+pip install slypi --trusted-host pypi.org --trusted-host files.pythonhosted.org
+```
+
+Be aware that the last option is insecure.  The better approach is to 
+fix your SSL certificate and/or point Python to a copy of the certificate.
+This can be done using:
+
+```sh
+pip config set global.cert path-to-your-certificate
+```
+
+You can also install locally from this repository using:
+
+```sh
+pip install -e slypi .
+```
+
+Note: that for SlyPI to work, you must have a Slycat server running.  
+See https://slycat.readthedocs.io/en/latest/ for details on setting up a server.
+
+## Basic Use
+
+SlyPI can be imported from within a Python file using
+
+    import slypi
+
+Some examples using SlyPI can be found in the SlyPI
+source directory.  These can be run using, e.g.
+
+```sh
+$ python -m slypi.util.list_markings
+```
+
+In addition, there are two main entry points defined, one for the Slycat Dial-A-Cluster model
+
+```sh
+$ dac_upload_gen
+```
+and one for the parameter space model
+
+```sh
+$ ps_upload_csv
+```
+
+## Kerberos
+
+The --kerberos option relies on a working Kerberos installation on your system.  Sometimes
+this will fail.  If you get an error related to Kerberos credentials (e.g. "Couldn't find
+Kerberos ticket," or "User not Kerberos authenticated"), try:
+
+```sh
+$ kinit
+```
+
+Then re-run the original command.
+
+## Proxies/Certificates
+
+If you are separated from the Slycat server by a proxy, or have not set up a security
+certificate, you will have to use the SlyPI proxy settings.  The proxy
+settings are available using the flags:
+
+* --http-proxy
+* --https-proxy
+* --verify
+* --no-verify
+
+The proxy flags are by default set to "no proxy".  If you have proxies set in the
+environment variables, they will be ignored.  The proxy flags are used as follows
+(for example):
+
+```sh
+$ python -m slypi.list_markings.py --http-proxy http://your.http.proxy --https-proxy https://your.https.proxy
+```
+
+The verify flag can be used to pass a security certificate as a command line argument and
+the --no-verify flag can be used to ignore the security certificates altogether.
+
+## General Utilities
+
+The simplest examples of interacting with the Slycat server issue
+requests for markings and projects, e.g.
+
+```sh
+$ python -m slypi.util.list_markings.py
+$ python -m slypi.util.list_projects.py
+```
+
+To examine a particular model or project, use
+
+```sh
+$ python -m slypi.util.get_model.py mid
+$ python -m slypi.util.get_project.py pid
+```
+
+where mid and pid are the hash identifiers for a Slycat model
+or project residing on the Slycat server.  These IDs can be extracted
+from the URL in the Slycat web browser client, or by using
+Info -> Model Details from the browser.
+
+## Creating Models
+
+The SlyPI module provides a command line option for creating Slycat
+models.  For example, to create a sample CCA model using random data, use:
+
+```sh
+$ python -m slypi.cca.upload_random.py
+```
+
+To create a sample CCA model from a CSV file, use:
+
+```sh
+$ python -m slypi.cca.upload_csv.py slycat-data/cars.csv --input Cylinders Displacement Weight Year --output MPG Horsepower Acceleration --project-name "CCA Models"
+```
+
+where "slycat-data/cars.csv" is from the slycat-data git repository at
+https://github.com/sandialabs/slycat-data.
+
+Note that when a model is created, the URL is given in the console and
+can be copied into a web browser to display the model.  The model ID
+can also be extracted from this URL (it is the hash at the end of the URL).
+
+## Parameter Space Models
+
+SlyPI also provides a programmatic interface for creating models.  To create
+a Parameter Space model from a python script, use the SlyPI module as follows:
+
+```{python}
+import slypi.ps.upload_csv as ps_upload_csv
+
+# parameter space file
+CARS_FILE = ['../slycat-data/cars.csv']
+
+# input/output columns for cars data file
+CARS_INPUT = ['--input-columns', 'Model', 'Cylinders', 'Displacement', 'Weight', 'Year']
+CARS_OUTPUT = ['--output-columns', 'MPG', 'Horsepower', 'Acceleration']
+
+# create PS model from cars file
+ps_parser = ps_upload_csv.parser()
+arguments = ps_parser.parse_args(CARS_FILE + CARS_INPUT + CARS_OUTPUT)
+ps_upload_csv.create_model(arguments, ps_upload_csv.log)     
+
+```
+
+A Parameter Space model can also  be created from .csv file using the 
+ps_csv script.  From the command line, use:
+
+```sh
+$ python -m slypi.ps.upload_csv slycat-data/cars.csv --input-columns Cylinders Displacement Weight Year --output-columns MPG Horsepower Acceleration --project-name "PS Models"
+```
+
+## Dial-A-Cluster (DAC) Models
+
+Dial-A-Cluster models can be loaded using different formats.  The first
+format is the generic dial-a-cluster format, described more fully in
+the Slycat user manual.
+
+To upload a DAC generic .zip file, use
+
+```sh
+$ dac_upload_gen slycat-data/dial-a-clsuter/weather-dac-gen.zip --project-name "DAC Models"
+```
+
+This will create a model from a single .zip file containing the appropriate
+folders with the pre-computed distance or PCA matrices.
+
+## Other Models
+
+Currently, SlyPI supports Slycat CCA and Videoswarm models.  The only Slycat model type
+still unsupported is Time Series.  For examples using CCA and Videoswarm, see the full documentation.
+
+## Contact
+Shawn Martin -- smartin@sandia.gov
+
+## License
+Distributed under the Sandia license. See LICENSE file for more information.

slypi-1.0.0/README.md

@@ -0,0 +1,208 @@
+## SlyPI
+Python Interface for Slycat.
+
+## Description
+Slycat is a web application for interactive visualization of ensemble data (see https://github.com/sandialabs/slycat and https://slycat.readthedocs.io/en/latest).  The SlyPI project contains code that enables you to interact with the Slycat web server through a Python interface.  You can query the Slycat server, create Slycat models, and use the code to integrate your own algorithms into the Slycat pipeline.
+
+## Documentation
+
+Full documentation can be found in the docs folder and can be compiled using Sphinx using
+
+```sh
+$ make html
+```
+
+The resulting documention can be accessed under build from the file index.html.
+
+## Installation
+This code is available at https://pypi.org/projects/slypi and can be installed using
+
+```sh
+$ pip install slypi
+```
+
+If you are working behind a proxy, you might also need, e.g.
+
+```sh
+pip install slypi --proxy your-proxy:your-port
+```
+
+If you are getting SSL certificate errors, you can use:
+
+```sh
+pip install slypi --trusted-host pypi.org --trusted-host files.pythonhosted.org
+```
+
+Be aware that the last option is insecure.  The better approach is to 
+fix your SSL certificate and/or point Python to a copy of the certificate.
+This can be done using:
+
+```sh
+pip config set global.cert path-to-your-certificate
+```
+
+You can also install locally from this repository using:
+
+```sh
+pip install -e slypi .
+```
+
+Note: that for SlyPI to work, you must have a Slycat server running.  
+See https://slycat.readthedocs.io/en/latest/ for details on setting up a server.
+
+## Basic Use
+
+SlyPI can be imported from within a Python file using
+
+    import slypi
+
+Some examples using SlyPI can be found in the SlyPI
+source directory.  These can be run using, e.g.
+
+```sh
+$ python -m slypi.util.list_markings
+```
+
+In addition, there are two main entry points defined, one for the Slycat Dial-A-Cluster model
+
+```sh
+$ dac_upload_gen
+```
+and one for the parameter space model
+
+```sh
+$ ps_upload_csv
+```
+
+## Kerberos
+
+The --kerberos option relies on a working Kerberos installation on your system.  Sometimes
+this will fail.  If you get an error related to Kerberos credentials (e.g. "Couldn't find
+Kerberos ticket," or "User not Kerberos authenticated"), try:
+
+```sh
+$ kinit
+```
+
+Then re-run the original command.
+
+## Proxies/Certificates
+
+If you are separated from the Slycat server by a proxy, or have not set up a security
+certificate, you will have to use the SlyPI proxy settings.  The proxy
+settings are available using the flags:
+
+* --http-proxy
+* --https-proxy
+* --verify
+* --no-verify
+
+The proxy flags are by default set to "no proxy".  If you have proxies set in the
+environment variables, they will be ignored.  The proxy flags are used as follows
+(for example):
+
+```sh
+$ python -m slypi.list_markings.py --http-proxy http://your.http.proxy --https-proxy https://your.https.proxy
+```
+
+The verify flag can be used to pass a security certificate as a command line argument and
+the --no-verify flag can be used to ignore the security certificates altogether.
+
+## General Utilities
+
+The simplest examples of interacting with the Slycat server issue
+requests for markings and projects, e.g.
+
+```sh
+$ python -m slypi.util.list_markings.py
+$ python -m slypi.util.list_projects.py
+```
+
+To examine a particular model or project, use
+
+```sh
+$ python -m slypi.util.get_model.py mid
+$ python -m slypi.util.get_project.py pid
+```
+
+where mid and pid are the hash identifiers for a Slycat model
+or project residing on the Slycat server.  These IDs can be extracted
+from the URL in the Slycat web browser client, or by using
+Info -> Model Details from the browser.
+
+## Creating Models
+
+The SlyPI module provides a command line option for creating Slycat
+models.  For example, to create a sample CCA model using random data, use:
+
+```sh
+$ python -m slypi.cca.upload_random.py
+```
+
+To create a sample CCA model from a CSV file, use:
+
+```sh
+$ python -m slypi.cca.upload_csv.py slycat-data/cars.csv --input Cylinders Displacement Weight Year --output MPG Horsepower Acceleration --project-name "CCA Models"
+```
+
+where "slycat-data/cars.csv" is from the slycat-data git repository at
+https://github.com/sandialabs/slycat-data.
+
+Note that when a model is created, the URL is given in the console and
+can be copied into a web browser to display the model.  The model ID
+can also be extracted from this URL (it is the hash at the end of the URL).
+
+## Parameter Space Models
+
+SlyPI also provides a programmatic interface for creating models.  To create
+a Parameter Space model from a python script, use the SlyPI module as follows:
+
+```{python}
+import slypi.ps.upload_csv as ps_upload_csv
+
+# parameter space file
+CARS_FILE = ['../slycat-data/cars.csv']
+
+# input/output columns for cars data file
+CARS_INPUT = ['--input-columns', 'Model', 'Cylinders', 'Displacement', 'Weight', 'Year']
+CARS_OUTPUT = ['--output-columns', 'MPG', 'Horsepower', 'Acceleration']
+
+# create PS model from cars file
+ps_parser = ps_upload_csv.parser()
+arguments = ps_parser.parse_args(CARS_FILE + CARS_INPUT + CARS_OUTPUT)
+ps_upload_csv.create_model(arguments, ps_upload_csv.log)     
+
+```
+
+A Parameter Space model can also  be created from .csv file using the 
+ps_csv script.  From the command line, use:
+
+```sh
+$ python -m slypi.ps.upload_csv slycat-data/cars.csv --input-columns Cylinders Displacement Weight Year --output-columns MPG Horsepower Acceleration --project-name "PS Models"
+```
+
+## Dial-A-Cluster (DAC) Models
+
+Dial-A-Cluster models can be loaded using different formats.  The first
+format is the generic dial-a-cluster format, described more fully in
+the Slycat user manual.
+
+To upload a DAC generic .zip file, use
+
+```sh
+$ dac_upload_gen slycat-data/dial-a-clsuter/weather-dac-gen.zip --project-name "DAC Models"
+```
+
+This will create a model from a single .zip file containing the appropriate
+folders with the pre-computed distance or PCA matrices.
+
+## Other Models
+
+Currently, SlyPI supports Slycat CCA and Videoswarm models.  The only Slycat model type
+still unsupported is Time Series.  For examples using CCA and Videoswarm, see the full documentation.
+
+## Contact
+Shawn Martin -- smartin@sandia.gov
+
+## License
+Distributed under the Sandia license. See LICENSE file for more information.

slypi-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

slypi-1.0.0/setup.py

@@ -0,0 +1,94 @@
+# Copyright (c) 2013, 2018 National Technology and Engineering Solutions of Sandia, LLC. 
+# Under the terms of Contract DE-NA0003525 with National Technology and Engineering 
+# Solutions of Sandia, LLC, the U.S. Government retains certain rights in this software.
+
+# This script creates a Python distribution wheel for the Slycat python interface, 
+# now called slypi, formerly called slycat-web-client.  When run, it copies the latest 
+# version of the slycat/web/client files (from the slycat repo) into slypi directory 
+# (see note below) before it constructs the pip installable module.
+#
+# NOTE: You need to have the slycat directory (from github) available as ../slycat
+# in order to run the setup.py file (see additional note below).
+#
+# S. Martin
+# 11/9/2023
+
+# To publish to PyPi, perform the following steps:
+#
+# $ rm -rf dist
+# $ python setup.py sdist bdist_wheel
+# $ twine upload dist/*
+#
+# To publish to testpypi, use:
+#
+# $ twine upload --repository-url https://test.pypi.org/legacy/ dist/*
+#
+# The first step builds the distribution, and the second step
+# uploads to PyPi.  To install the package from another computer use:
+#
+# $ pip install slpyi
+#
+# To install from testpypi, use:
+#
+# $ pip install --extra-index-url https://testpypi.python.org/pypi slpyi
+#   --upgrade --trusted-host testpypi.python.org --proxy wwwproxy.sandia.gov:80
+#
+# To install locally, use (from this directory):
+#
+# $ pip install -e .
+
+from shutil import copyfile
+
+# copy slycat.darray and slycat csv parser code into slypi directory. This
+# makes slypi a Python package without other Slycat dependencies.
+copyfile('../slycat/packages/slycat/darray.py', 'slypi/darray.py')
+copyfile('../slycat/packages/slycat/pandas_util.py', 'slypi/pandas_util.py')
+copyfile('../slycat/web-server/plugins/slycat-video-swarm/vs-parse-files.py', 'slypi/vs/vs_parse_files.py')
+
+import slypi
+VERSION = slypi.__version__
+
+# get README.md
+import pathlib
+
+# directory containing this file
+HERE = pathlib.Path(__file__).parent
+
+# text of the web-client-readme.txt file
+README = (HERE / "README.md").read_text()
+
+# create distribution
+import setuptools
+
+# create Python distribution wheel
+from setuptools import setup
+
+setup(
+    name="slypi",
+    version=VERSION,
+    description="Slycat python interface utilties for interacting with the Slycat " +
+                "data analysis and visualization server.",
+    long_description=README,
+    long_description_content_type="text/markdown",
+    url="https://github.com/sandialabs/slycat",
+    author="Shawn Martin",
+    author_email="smartin@sandia.gov",
+    license="Sandia",
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.11",
+    ],
+    packages=setuptools.find_packages(),
+    include_package_data=True,
+    install_requires=["requests", "requests-kerberos",
+                      "numpy", "scikit-learn", 
+                      "pandas", "meshio", "imageio[ffmpeg]",
+                      "matplotlib", "ipyparallel",
+                      "torch", "umap-learn", "pymks"],
+    entry_points={
+        "console_scripts": [
+            "ps_upload_csv=slypi.ps.upload_csv:main",
+            "dac_upload_gen=slycat.dac.upload_gen:main"
+        ]
+    },
+)