tts-data-utils 0.7.3__tar.gz

tts_data_utils-0.7.3/.gitignore

@@ -0,0 +1,180 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+env
+venv
+
+# IDEs and editors
+.idea/
+.vscode/
+*.sublime-project
+*.sublime-workspace
+
+# Jupyter notebooks
+.ipynb_checkpoints/
+
+# Jupyter lab workspace
+.jupyter/
+
+# Miscellaneous
+.DS_Store
+Thumbs.db
+._*

tts_data_utils-0.7.3/PKG-INFO

@@ -0,0 +1,81 @@
+Metadata-Version: 2.4
+Name: tts-data-utils
+Version: 0.7.3
+Summary: Utilities for working with 2D data.
+Author: muszynsk
+Author-email: muszynsk <matt.muszynski@jpl.nasa.gov>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/NASA-JPL-Teamtools-Studio/tts_data_utils
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: Apache Software License
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+Requires-Dist: docxtpl
+Requires-Dist: pandas
+Requires-Dist: pytest
+Requires-Dist: tabulate
+Requires-Dist: jinja2
+Requires-Dist: plotly
+Requires-Dist: jpl_time
+Requires-Dist: openpyxl
+Requires-Dist: datetime
+Requires-Dist: jpl-time
+Requires-Dist: tabulate
+Requires-Dist: pandas
+Requires-Dist: tts-utilities<1.0.0,>=0.2.1
+Requires-Dist: tts-html-utils<1.0.0,>=0.4.2
+Dynamic: author
+Dynamic: requires-python
+
+# Teamtools Data Utilities
+
+![Project logo](https://github.com/NASA-JPL-Teamtools-Studio/teamtools_documentation/blob/main/docs/images/tts_image_artifacts/data_utils.png)
+
+## About Teamtools Studio
+
+Teamtools Studio Utilities is part of JPL's Teamtools Studio (TTS).
+
+TTS is an effort originated in JPL's Planning and Execution section to centralize shared repositories across missions. This benefits JPL by reducing cost through reducing duplicated code, collaborating across missions, and unifying standards for development and design across JPL.
+
+Although Planning and Execution is primarily concerned with flight operations, the TTS suite has been generalized and atomized to the point where many of these tools are applicable during other mission phases and even in non-spaceflight contexts. Through our work flying space missions, we hope to provide tools to the open source community that have utility in data analysis or planning for any complex system where failure is not an option.
+
+For more infomation on how to contribute, and how these libraries form a complete ecosystem for high reliability data analysis, see the [Full TTS Documentation](https://nasa-jpl-teamtools-studio.github.io/teamtools_documentation/).
+
+## What is Data Utilities?
+
+### Overview
+
+Data Utilties is one of the central building blocks of the Teamtools Studio. It is primarily a means of organizing
+and tracking the changes in 2D data (think spreadsheets). It is used as the primary data structure in Tower and Dexter,
+and is used liberally elsewhere throughout the TTS project.
+
+It is a reality that much of Systems Engineering is executed via spreadsheets and tables. They are the media in which 
+the most SEs are the most comfortable, so data utils provides methods to read, write, manipulate, convert, and mange tabular
+data. It goes beyond a library like Pandas by providing a core DataContainer class and examples of extensions of it to
+provide the custom behavior needed for each bespoke data source.
+
+For cases where data is too complex for a single 2D table, Data Utilities also handles the nesting of tables where each
+row in a table can be associated with a subtable.
+
+Data Utilities also provides significant quality of life functionality for diffing containers (including visual diff) and
+visualizing them in custom ways in with HTML, Excel, and DOCX output styles.
+
+Additionally, Data Utilities provides a lorem ipsum data generation feature that allows you to quickly create containers with semi-realistic dummy data for prototyping and testing purposes.
+
+### Projects Currently Supported
+
+* Europa Clipper
+* Mars 2020/Perseverance
+* Mars Sample Return/Sample Retrieval Lander
+* Mars Science Laboratory/Curiosity
+* Mars Reconnaisance Orbiter
+* NISAR
+* Orbiting Carbon Observatory 2 (OCO-2)
+
+## Architecture
+
+### TTS dependencies
+
+* Teamtools Studio Utilities
+* HTML Utilites
+

tts_data_utils-0.7.3/README.md

@@ -0,0 +1,52 @@
+# Teamtools Data Utilities
+
+![Project logo](https://github.com/NASA-JPL-Teamtools-Studio/teamtools_documentation/blob/main/docs/images/tts_image_artifacts/data_utils.png)
+
+## About Teamtools Studio
+
+Teamtools Studio Utilities is part of JPL's Teamtools Studio (TTS).
+
+TTS is an effort originated in JPL's Planning and Execution section to centralize shared repositories across missions. This benefits JPL by reducing cost through reducing duplicated code, collaborating across missions, and unifying standards for development and design across JPL.
+
+Although Planning and Execution is primarily concerned with flight operations, the TTS suite has been generalized and atomized to the point where many of these tools are applicable during other mission phases and even in non-spaceflight contexts. Through our work flying space missions, we hope to provide tools to the open source community that have utility in data analysis or planning for any complex system where failure is not an option.
+
+For more infomation on how to contribute, and how these libraries form a complete ecosystem for high reliability data analysis, see the [Full TTS Documentation](https://nasa-jpl-teamtools-studio.github.io/teamtools_documentation/).
+
+## What is Data Utilities?
+
+### Overview
+
+Data Utilties is one of the central building blocks of the Teamtools Studio. It is primarily a means of organizing
+and tracking the changes in 2D data (think spreadsheets). It is used as the primary data structure in Tower and Dexter,
+and is used liberally elsewhere throughout the TTS project.
+
+It is a reality that much of Systems Engineering is executed via spreadsheets and tables. They are the media in which 
+the most SEs are the most comfortable, so data utils provides methods to read, write, manipulate, convert, and mange tabular
+data. It goes beyond a library like Pandas by providing a core DataContainer class and examples of extensions of it to
+provide the custom behavior needed for each bespoke data source.
+
+For cases where data is too complex for a single 2D table, Data Utilities also handles the nesting of tables where each
+row in a table can be associated with a subtable.
+
+Data Utilities also provides significant quality of life functionality for diffing containers (including visual diff) and
+visualizing them in custom ways in with HTML, Excel, and DOCX output styles.
+
+Additionally, Data Utilities provides a lorem ipsum data generation feature that allows you to quickly create containers with semi-realistic dummy data for prototyping and testing purposes.
+
+### Projects Currently Supported
+
+* Europa Clipper
+* Mars 2020/Perseverance
+* Mars Sample Return/Sample Retrieval Lander
+* Mars Science Laboratory/Curiosity
+* Mars Reconnaisance Orbiter
+* NISAR
+* Orbiting Carbon Observatory 2 (OCO-2)
+
+## Architecture
+
+### TTS dependencies
+
+* Teamtools Studio Utilities
+* HTML Utilites
+

tts_data_utils-0.7.3/ci/sphinx_ignore_tags.txt

@@ -0,0 +1,23 @@
+#When tts-build-sphinx-docs runs, it will collect all tags and branches and attempt to build
+#docs for each in the gh-pages branch. This is a config file that allows developers to 
+#specifically exclude any branches or tags they need. Each line is the name of a tag or 
+#branch.
+#
+#This file makes no attemtpt to tell whether a line is a tag or a branch as TTS assumes that
+#the naming conventions will always be different, and that tags should be tied to releases,
+#so they should always have semvar names, whereas branches should be more descriptive using
+#typical agile branch naming conventions.
+#
+#The most typical use case is to exclude tags that were created before you added the docs
+#prerequisites, or to exclude the gh-pages branch itself since no code is expected to live
+#there. All TTS repos are expected to have a gh-pages branch, but it is expected that it
+#will always be managed by tts-build-sphinx-docs. You do not need to manually create
+#a gh-pages branch because tts-build-sphinx-docs will create one if it does not exist.
+#
+#tts-build-sphinx-docs will also always remove all files from gh-pages and rebuild all docs
+#from scratch
+0.0.1
+0.0.2
+12.34.56v789
+98.76.54rc321
+gh-pages

tts_data_utils-0.7.3/docs/_static/bugfix.css

@@ -0,0 +1,12 @@
+/*
+Resolves a bug where the text in the header
+"[package name] [package version] documentation"
+was getting stacked up on top of itself when the variable portions were too long. Something with floats
+and elements not taking up the whole width of their contents.
+
+Needed to add it because setuptools-scm version names can be very long, especially for versions that
+are build off of a commit with no tag
+*/
+.navbar-header-items__start {
+    width: auto;
+}

tts_data_utils-0.7.3/docs/conf.py

@@ -0,0 +1,71 @@
+# TTS-specifc add ons
+from pathlib import Path
+from datetime import datetime
+from setuptools_scm import get_version
+import toml
+
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+
+import os
+import sys
+import importlib.metadata
+sys.path.insert(0, os.path.abspath('..'))
+
+# -- Project information -----------------------------------------------------
+
+pyproject = toml.load(Path(__file__).parent.parent.joinpath("pyproject.toml"))
+project = pyproject["project"]["name"]
+release = get_version(root='..', relative_to=__file__)
+version = '.'.join(release.split('.')[:2])
+
+copyright = f'{datetime.now().strftime('%Y')} JPL'
+author = 'JPL Teamtools Studio'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ['sphinx.ext.autodoc']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# The master toctree document.
+master_doc = 'index'
+
+pygments_style = 'sphinx'
+# html_theme = 'nature'
+# html_theme = 'sphinx_material'
+html_theme = 'pydata_sphinx_theme'
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+html_css_files = ['bugfix.css']
+html_copy_source = False
+html_show_sourcelink = False

tts_data_utils-0.7.3/docs/core/container_history.rst

@@ -0,0 +1,5 @@
+Container History
+===================
+
+.. automodule:: data_utils.core.container_history
+   :members:

tts_data_utils-0.7.3/docs/core/data_container.rst

@@ -0,0 +1,6 @@
+Data Container
+=========================
+
+.. automodule:: data_utils.core.data_container
+   :members:
+   :special-members: __str__, __repr__, __getitem__

tts_data_utils-0.7.3/docs/core/data_item.rst

@@ -0,0 +1,7 @@
+Data Item
+===============
+
+.. automodule:: data_utils.core.data_item
+   :members:
+   :private-members:
+   :special-members: __str__, __repr__, __getitem__, __setitem__

tts_data_utils-0.7.3/docs/core/diff.rst

@@ -0,0 +1,5 @@
+Diff Container
+===============
+
+.. automodule:: data_utils.core.diff
+   :members:

tts_data_utils-0.7.3/docs/core/generic.rst

@@ -0,0 +1,5 @@
+Generic Container
+==================
+
+.. automodule:: data_utils.core.generic
+   :members:

tts_data_utils-0.7.3/docs/index.rst

@@ -0,0 +1,46 @@
+Welcome to the Teamtools Starter Template documentation!
+=============================================================
+
+Core Modules
+==================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   core/data_container
+   core/data_item
+   core/generic
+   core/diff
+   core/container_history
+
+Multimission Data Containers
+===============================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   multimission/eha
+   multimission/evr
+   multimission/evr_gaps
+   multimission/expected_lad
+   multimission/planning_rule
+
+Invulnerable Data Manager
+===============================
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   invulnerable_data_manager/invulnerable_data_manager
+   invulnerable_data_manager/idm_utilities
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

tts_data_utils-0.7.3/docs/invulnerable_data_manager/idm_utilities.rst

@@ -0,0 +1,5 @@
+Invulnerable Data Manager Utilities
+====================================
+
+.. automodule:: data_utils.invulnerable_data_manager.utilities
+   :members:

tts_data_utils-0.7.3/docs/invulnerable_data_manager/invulnerable_data_manager.rst

@@ -0,0 +1,5 @@
+Invulnerable Data Manager 
+==========================
+
+.. automodule:: data_utils.invulnerable_data_manager.invulnerable_data_manager
+   :members:

tts_data_utils-0.7.3/docs/lorem_ipsum_feature.md

@@ -0,0 +1,69 @@
+# Lorem Ipsum Data Generation
+
+Data Utilities now includes a powerful feature for generating realistic dummy data for any DataContainer type. This is useful for prototyping, testing, and documentation purposes.
+
+## Basic Usage
+
+To generate lorem ipsum data for any DataContainer, simply pass an integer to the `lorem` parameter when creating the container:
+
+```python
+from data_utils.multimission.evr import EvrContainer
+
+# Create an EVR container with 10 dummy records
+evr_container = EvrContainer(lorem=10)
+
+# Print the container to see the generated data
+print(evr_container)
+```
+
+## How It Works
+
+The lorem ipsum generator analyzes the `DICT_VALID_KEYS` of the container's `DATA_ITEM_CLS` to understand the expected structure and data types. It then generates appropriate random values for each field, taking into account:
+
+- Field names (e.g., fields containing "id", "name", "message", etc.)
+- Data types (strings, integers, floats, datetimes, etc.)
+- Allowed null values
+
+## Smart Value Generation
+
+The generator tries to create meaningful data based on field names:
+
+- ID fields get numeric or formatted ID values
+- Name fields get short lorem ipsum phrases
+- Message fields get longer lorem ipsum text
+- Status fields get values like "pending", "completed", etc.
+- Time fields get random datetimes within the last year
+- Boolean fields get random True/False values
+
+## Example
+
+```python
+from data_utils.multimission.evr import EvrContainer
+from data_utils.multimission.eha import EhaContainer
+from data_utils.core.generic import GenericContainer
+
+# Create containers with different numbers of records
+evr_container = EvrContainer(lorem=5)
+eha_container = EhaContainer(lorem=3)
+generic_container = GenericContainer(lorem=4, name="My Lorem Data")
+
+# Use them like any other container
+print(f"EVR Container has {len(evr_container)} records")
+print(f"First EVR message: {evr_container[0].message}")
+
+# You can also convert to other formats
+html_table = evr_container.power_table()
+```
+
+## Use Cases
+
+- **Prototyping**: Quickly create realistic data structures without writing boilerplate
+- **Testing**: Generate test data with predictable structure but random values
+- **Documentation**: Create examples with realistic data
+- **UI Development**: Populate interfaces with dummy data during development
+
+## Customization
+
+The lorem ipsum generator is designed to work with any DataContainer subclass, including custom ones you create. As long as your class follows the DataContainer pattern with a properly defined DATA_ITEM_CLS and DICT_VALID_KEYS, the generator will produce appropriate data.
+
+For more examples, see the demo script at `data_utils/demo/lorem_demo.py`.

tts_data_utils-0.7.3/docs/multimission/eha.rst

@@ -0,0 +1,5 @@
+EHA Container
+===============
+
+.. automodule:: data_utils.multimission.eha
+   :members:

tts_data_utils-0.7.3/docs/multimission/evr.rst

@@ -0,0 +1,5 @@
+EVR Container
+===============
+
+.. automodule:: data_utils.multimission.evr
+   :members:

tts_data_utils-0.7.3/docs/multimission/evr_gaps.rst

@@ -0,0 +1,5 @@
+EVR Gaps Container
+===================
+
+.. automodule:: data_utils.multimission.evr_gaps
+   :members:

tts_data_utils-0.7.3/docs/multimission/expected_lad.rst

@@ -0,0 +1,5 @@
+Expected LAD Container
+======================
+
+.. automodule:: data_utils.multimission.expected_lad
+   :members:

tts_data_utils-0.7.3/docs/multimission/planning_rule.rst

@@ -0,0 +1,5 @@
+Planning Rule Container
+=======================
+
+.. automodule:: data_utils.multimission.planning_rule
+   :members: