rhapsody-py 0.1.0__tar.gz

rhapsody_py-0.1.0/LICENSE.md

@@ -0,0 +1,24 @@
+
+rhapsody is licensed under the MIT License (MIT)
+-----------------------------------------------------
+
+    Copyright (C) 2023 by the RADICAL-Cybertools Team
+    (info@radical-cybertools.org)
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy
+    of this software and associated documentation files (the "Software"), to
+    deal in the Software without restriction, including without limitation the
+    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+    sell copies of the Software, and to permit persons to whom the Software is
+    furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in
+    all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+    IN THE SOFTWARE.

rhapsody_py-0.1.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include README.md
+include LICENSE
+recursive-include src/rhapsody *.py
+recursive-include tests *.py
+recursive-include docs *.md *.rst
+recursive-include bin *.md

rhapsody_py-0.1.0/PKG-INFO

@@ -0,0 +1,207 @@
+Metadata-Version: 2.4
+Name: rhapsody-py
+Version: 0.1.0
+Summary: Runtime system for executing heterogeneous HPC-AI workflows with dynamic task graphs on high-performance computing infrastructures.
+Author-email: RADICAL Research Team <info@radical.org>
+License: MIT
+Project-URL: Homepage, https://github.com/stride-research/rhapsody
+Project-URL: Repository, https://github.com/stride-research/rhapsody
+Project-URL: Issues, https://github.com/stride-research/rhapsody/issues
+Keywords: hpc,ai,workflow,task-orchestration,distributed-computing,high-performance-computing,scientific-computing
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: System :: Logging
+Classifier: Topic :: System :: Monitoring
+Classifier: Intended Audience :: Developers
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: typeguard>=4.0.0
+Requires-Dist: requests>=2.25.0
+Provides-Extra: dask
+Requires-Dist: dask[distributed]>=2023.0.0; extra == "dask"
+Provides-Extra: radical-pilot
+Requires-Dist: radical.pilot>=1.30.0; extra == "radical-pilot"
+Provides-Extra: dragon
+Requires-Dist: dragonhpc; extra == "dragon"
+Provides-Extra: vllm-dragon
+Requires-Dist: aiohttp>=3.8.0; extra == "vllm-dragon"
+Requires-Dist: pyyaml>=6.0; extra == "vllm-dragon"
+Provides-Extra: dev
+Requires-Dist: ruff>=0.8.0; extra == "dev"
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: pytest-xdist>=3.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: types-requests>=2.31.0; extra == "dev"
+Requires-Dist: types-aiofiles>=23.0.0; extra == "dev"
+Requires-Dist: pre-commit>=4.3.0; extra == "dev"
+Requires-Dist: docformatter>=1.7.0; extra == "dev"
+Requires-Dist: tomli>=2.2.0; extra == "dev"
+Requires-Dist: detect-secrets>=1.5.0; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs>=1.5.0; extra == "docs"
+Requires-Dist: mkdocs-material>=9.0.0; extra == "docs"
+Requires-Dist: mkdocstrings[python]>=0.24.0; extra == "docs"
+Requires-Dist: mkdocs-gen-files>=0.5.0; extra == "docs"
+Requires-Dist: mkdocs-literate-nav>=0.6.0; extra == "docs"
+Requires-Dist: mkdocs-section-index>=0.3.0; extra == "docs"
+Provides-Extra: ci
+Requires-Dist: tox>=4.0.0; extra == "ci"
+Requires-Dist: tox-gh-actions>=3.0.0; extra == "ci"
+Requires-Dist: build>=0.10.0; extra == "ci"
+Requires-Dist: twine>=4.0.0; extra == "ci"
+Requires-Dist: psutil>=5.9.0; extra == "ci"
+Provides-Extra: examples
+Requires-Dist: rhapsody-py[dask]; extra == "examples"
+Provides-Extra: backends
+Requires-Dist: rhapsody-py[dask,dragon,radical_pilot,vllm-dragon]; extra == "backends"
+Provides-Extra: all
+Requires-Dist: rhapsody-py[backends,ci,dev,docs,examples]; extra == "all"
+Dynamic: license-file
+
+# RHAPSODY
+
+[![Build Status](https://github.com/radical-cybertools/rhapsody/actions/workflows/ci.yml/badge.svg)](https://github.com/radical-cybertools/rhapsody/actions/workflows/ci.yml)
+[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://radical-cybertools.github.io/rhapsody/)
+[![Python Version](https://img.shields.io/pypi/pyversions/rhapsody-py.svg)](https://pypi.org/project/rhapsody-py/)
+[![PyPI Version](https://img.shields.io/pypi/v/rhapsody-py.svg)](https://pypi.org/project/rhapsody-py/)
+[![License](https://img.shields.io/pypi/l/rhapsody-py.svg)](https://github.com/radical-cybertools/rhapsody/blob/main/LICENSE.md)
+
+**RHAPSODY** – **R**untime for **H**eterogeneous **AP**plications, **S**ervice **O**rchestration and **DY**namism
+
+A unified runtime for executing **AI and HPC workloads** on supercomputing infrastructures. RHAPSODY seamlessly integrates traditional scientific computing with AI inference, enabling complex workflows that combine simulation, analysis, and machine learning.
+
+## What RHAPSODY Offers
+
+- **Unified AI-HPC API**: Single interface for compute tasks and AI inference
+- **Multi-Backend Execution**: Run on local machines, HPC clusters ([Dragon](https://dragonhpc.github.io/dragon/doc/_build/html/index.html)), or distributed systems ([Dask](https://docs.dask.org/en/stable/))
+- **Async-First Design**: Native asyncio integration for efficient task orchestration
+- **Integratable Design**: RHAPSODY is designed to be integratable with existing workflows and tools such as [AsyncFlow](https://github.com/radical-cybertools/asyncflow) and [LangGraph/FlowGentic](https://github.com/stride-research/flowgentic).
+- **Scale-Ready**: Scale your workload and workflows to thousands of tasks and nodes.
+
+## Quick Example: AI-HPC Workflow
+
+```python
+import asyncio
+from rhapsody.api import Session, ComputeTask, AITask
+from rhapsody.backends import DragonExecutionBackendV3, DragonVllmInferenceBackend
+
+async def main():
+    # Initialize backends
+    hpc_backend = await DragonExecutionBackendV3(name="hpc", num_workers=128)
+    ai_backend = await DragonVllmInferenceBackend(name="vllm", model="Qwen2.5-7B")
+
+    # Create session with multiple backends
+    async with Session(backends=[hpc_backend, ai_backend]) as session:
+
+        # HPC simulation task
+        simulation = ComputeTask(
+            executable="./simulate",
+            arguments=["--config", "params.yaml"],
+            backend=hpc_backend.name
+        )
+
+        # AI analysis task
+        analysis = AITask(
+            prompt="Analyze the simulation results and identify key patterns...",
+            backend=ai_backend.name
+        )
+
+        # Submit and execute
+        await session.submit_tasks([simulation, analysis])
+
+        # Wait for completion (tasks are awaitable!)
+        sim_result = await simulation
+        ai_result = await analysis
+
+        print(f"Simulation: {sim_result['state']}")
+        print(f"AI Analysis: {ai_result['output']}")
+
+asyncio.run(main())
+```
+
+## Installation
+
+```bash
+# Basic installation
+pip install rhapsody-py
+
+# With specific backends
+pip install rhapsody-py[dask]           # Dask distributed computing
+pip install rhapsody-py[dragon]         # Dragon runtime (Python 3.10-3.12)
+
+# Development
+pip install rhapsody-py[dev]
+```
+
+
+## Documentation
+
+- **Full Documentation**: https://radical-cybertools.github.io/rhapsody/
+- **API Reference**: https://radical-cybertools.github.io/rhapsody/api/
+- **Examples**: See `examples/` directory
+
+## Contributing
+
+We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+### Development Workflow
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Make your changes
+4. Add tests for new functionality
+5. Ensure all tests pass (`make test-regular`)
+6. Run code quality checks (`pre-commit run --all-files`)
+7. Commit your changes (`git commit -m 'Add amazing feature'`)
+8. Push to the branch (`git push origin feature/amazing-feature`)
+9. Open a Pull Request
+
+### Reporting Issues
+
+Please use the [GitHub issue tracker](https://github.com/radical-cybertools/rhapsody/issues) to report bugs or request features.
+
+## License
+
+RHAPSODY is licensed under the [MIT License](LICENSE.md).
+
+## Acknowledgments
+
+RHAPSODY is developed by the [RADICAL Research Group](http://radical.rutgers.edu/) at Rutgers University.
+
+### Related Projects
+
+- [AsyncFlow](https://github.com/radical-cybertools/asyncflow): Asynchronous workflow management
+
+## NSF-Funded Project
+
+RHAPSODY is supported by the National Science Foundation (NSF) under Award ID [2103986](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2103986). This collaborative project aims to advance the state-of-the-art in heterogeneous workflow execution for scientific computing.
+
+### Citations
+
+If you use RHAPSODY in your research, please cite:
+
+```bibtex
+@software{rhapsody2024,
+  title={RHAPSODY: Runtime for Heterogeneous Applications, Service Orchestration and Dynamism},
+  author={RADICAL Research Team},
+  year={2024},
+  url={https://github.com/radical-cybertools/rhapsody},
+  version={0.1.0}
+}
+```
+
+## Support
+
+- **Documentation**: https://radical-cybertools.github.io/rhapsody/
+- **Issues**: https://github.com/radical-cybertools/rhapsody/issues
+- **Discussions**: https://github.com/radical-cybertools/rhapsody/discussions

rhapsody_py-0.1.0/README.md

@@ -0,0 +1,137 @@
+# RHAPSODY
+
+[![Build Status](https://github.com/radical-cybertools/rhapsody/actions/workflows/ci.yml/badge.svg)](https://github.com/radical-cybertools/rhapsody/actions/workflows/ci.yml)
+[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://radical-cybertools.github.io/rhapsody/)
+[![Python Version](https://img.shields.io/pypi/pyversions/rhapsody-py.svg)](https://pypi.org/project/rhapsody-py/)
+[![PyPI Version](https://img.shields.io/pypi/v/rhapsody-py.svg)](https://pypi.org/project/rhapsody-py/)
+[![License](https://img.shields.io/pypi/l/rhapsody-py.svg)](https://github.com/radical-cybertools/rhapsody/blob/main/LICENSE.md)
+
+**RHAPSODY** – **R**untime for **H**eterogeneous **AP**plications, **S**ervice **O**rchestration and **DY**namism
+
+A unified runtime for executing **AI and HPC workloads** on supercomputing infrastructures. RHAPSODY seamlessly integrates traditional scientific computing with AI inference, enabling complex workflows that combine simulation, analysis, and machine learning.
+
+## What RHAPSODY Offers
+
+- **Unified AI-HPC API**: Single interface for compute tasks and AI inference
+- **Multi-Backend Execution**: Run on local machines, HPC clusters ([Dragon](https://dragonhpc.github.io/dragon/doc/_build/html/index.html)), or distributed systems ([Dask](https://docs.dask.org/en/stable/))
+- **Async-First Design**: Native asyncio integration for efficient task orchestration
+- **Integratable Design**: RHAPSODY is designed to be integratable with existing workflows and tools such as [AsyncFlow](https://github.com/radical-cybertools/asyncflow) and [LangGraph/FlowGentic](https://github.com/stride-research/flowgentic).
+- **Scale-Ready**: Scale your workload and workflows to thousands of tasks and nodes.
+
+## Quick Example: AI-HPC Workflow
+
+```python
+import asyncio
+from rhapsody.api import Session, ComputeTask, AITask
+from rhapsody.backends import DragonExecutionBackendV3, DragonVllmInferenceBackend
+
+async def main():
+    # Initialize backends
+    hpc_backend = await DragonExecutionBackendV3(name="hpc", num_workers=128)
+    ai_backend = await DragonVllmInferenceBackend(name="vllm", model="Qwen2.5-7B")
+
+    # Create session with multiple backends
+    async with Session(backends=[hpc_backend, ai_backend]) as session:
+
+        # HPC simulation task
+        simulation = ComputeTask(
+            executable="./simulate",
+            arguments=["--config", "params.yaml"],
+            backend=hpc_backend.name
+        )
+
+        # AI analysis task
+        analysis = AITask(
+            prompt="Analyze the simulation results and identify key patterns...",
+            backend=ai_backend.name
+        )
+
+        # Submit and execute
+        await session.submit_tasks([simulation, analysis])
+
+        # Wait for completion (tasks are awaitable!)
+        sim_result = await simulation
+        ai_result = await analysis
+
+        print(f"Simulation: {sim_result['state']}")
+        print(f"AI Analysis: {ai_result['output']}")
+
+asyncio.run(main())
+```
+
+## Installation
+
+```bash
+# Basic installation
+pip install rhapsody-py
+
+# With specific backends
+pip install rhapsody-py[dask]           # Dask distributed computing
+pip install rhapsody-py[dragon]         # Dragon runtime (Python 3.10-3.12)
+
+# Development
+pip install rhapsody-py[dev]
+```
+
+
+## Documentation
+
+- **Full Documentation**: https://radical-cybertools.github.io/rhapsody/
+- **API Reference**: https://radical-cybertools.github.io/rhapsody/api/
+- **Examples**: See `examples/` directory
+
+## Contributing
+
+We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+### Development Workflow
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Make your changes
+4. Add tests for new functionality
+5. Ensure all tests pass (`make test-regular`)
+6. Run code quality checks (`pre-commit run --all-files`)
+7. Commit your changes (`git commit -m 'Add amazing feature'`)
+8. Push to the branch (`git push origin feature/amazing-feature`)
+9. Open a Pull Request
+
+### Reporting Issues
+
+Please use the [GitHub issue tracker](https://github.com/radical-cybertools/rhapsody/issues) to report bugs or request features.
+
+## License
+
+RHAPSODY is licensed under the [MIT License](LICENSE.md).
+
+## Acknowledgments
+
+RHAPSODY is developed by the [RADICAL Research Group](http://radical.rutgers.edu/) at Rutgers University.
+
+### Related Projects
+
+- [AsyncFlow](https://github.com/radical-cybertools/asyncflow): Asynchronous workflow management
+
+## NSF-Funded Project
+
+RHAPSODY is supported by the National Science Foundation (NSF) under Award ID [2103986](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2103986). This collaborative project aims to advance the state-of-the-art in heterogeneous workflow execution for scientific computing.
+
+### Citations
+
+If you use RHAPSODY in your research, please cite:
+
+```bibtex
+@software{rhapsody2024,
+  title={RHAPSODY: Runtime for Heterogeneous Applications, Service Orchestration and Dynamism},
+  author={RADICAL Research Team},
+  year={2024},
+  url={https://github.com/radical-cybertools/rhapsody},
+  version={0.1.0}
+}
+```
+
+## Support
+
+- **Documentation**: https://radical-cybertools.github.io/rhapsody/
+- **Issues**: https://github.com/radical-cybertools/rhapsody/issues
+- **Discussions**: https://github.com/radical-cybertools/rhapsody/discussions

rhapsody_py-0.1.0/docs/LICENSE.md

@@ -0,0 +1,24 @@
+
+rhapsody is licensed under the MIT License (MIT)
+-----------------------------------------------------
+
+    Copyright (C) 2023 by the RADICAL-Cybertools Team
+    (info@radical-cybertools.org)
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy
+    of this software and associated documentation files (the "Software"), to
+    deal in the Software without restriction, including without limitation the
+    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+    sell copies of the Software, and to permit persons to whom the Software is
+    furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in
+    all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+    IN THE SOFTWARE.

rhapsody_py-0.1.0/docs/about.md

@@ -0,0 +1,108 @@
+# About RHAPSODY
+
+RHAPSODY (<b>R</b>untime for <b>H</b>eterogeneous <b>AP</b>plications, <b>S</b>ervice <b>O</b>rchestration and <b>DY</b>namism) is a cutting-edge runtime system designed to address the evolving challenges of modern scientific computing. As computational workflows become increasingly complex, combining traditional HPC simulations with AI/ML workloads, there is a critical need for systems that can efficiently orchestrate heterogeneous tasks across diverse computing resources.
+
+## Vision
+
+Our vision is to create a unified runtime that seamlessly bridges the gap between traditional high-performance computing and emerging AI/ML paradigms, enabling scientists to focus on their research rather than the complexities of workflow orchestration.
+
+## Core Philosophy
+
+RHAPSODY is built on several key principles:
+
+### Heterogeneity as a First-Class Citizen
+
+Modern scientific workflows are inherently heterogeneous, combining:
+
+- Traditional HPC simulations (CPU-intensive)
+- Machine learning training and inference (GPU-intensive)
+- Data processing and analysis tasks
+- I/O-intensive operations
+
+RHAPSODY treats this heterogeneity as a fundamental characteristic rather than an afterthought.
+
+### Dynamic Adaptation
+
+Scientific workflows are rarely static. RHAPSODY supports:
+
+- Runtime modification of task graphs based on intermediate results
+- Adaptive resource allocation
+- Dynamic load balancing across heterogeneous resources
+
+### Platform Abstraction
+
+Scientists should not need to be experts in every computing platform. RHAPSODY provides:
+
+- Unified interfaces across different HPC systems
+- Abstraction of resource manager specifics
+- Portable workflow descriptions
+
+## Technical Innovation
+
+RHAPSODY introduces several technical innovations:
+
+### Pluggable Backend Architecture
+
+The system supports multiple execution backends, each optimized for different scenarios:
+
+- **Concurrent Backend**: For shared-memory parallel execution
+- **Dask Backend**: For distributed Python workloads
+- **RADICAL-Pilot Backend**: For large-scale HPC execution
+
+### AsyncFlow Integration
+
+Full compatibility with the [AsyncFlow](https://github.com/radical-cybertools/radical.asyncflow) workflow management system, providing:
+
+- Standardized workflow descriptions
+- Tool interoperability
+- Community ecosystem benefits
+
+### State-of-the-Art Monitoring
+
+Comprehensive monitoring and introspection capabilities for:
+
+- Real-time task execution tracking
+- Performance analysis and optimization
+- Fault detection and recovery
+
+## Research Impact
+
+RHAPSODY enables breakthrough research in numerous domains:
+
+- **Climate Science**: Coupling atmospheric simulations with ML-based post-processing
+- **Materials Science**: Integrating quantum simulations with ML property prediction
+- **Computational Biology**: Combining molecular dynamics with deep learning analysis
+
+## Project Timeline
+
+RHAPSODY development follows a structured timeline aligned with NSF project milestones:
+
+- **Phase 1** (2021-2022): Core architecture and backend development
+- **Phase 2** (2022-2024): Integration with existing workflow systems
+- **Phase 3** (2024-2026): Production deployment and optimization
+
+## Team and Collaboration
+
+RHAPSODY is developed by the [RADICAL Research Team](https://radical.rutgers.edu/) at Rutgers University, in collaboration with:
+
+- DOE National laboratories
+- Campus supercomputing centers
+- Academic research institutions
+
+## Open Science Commitment
+
+We are committed to open science principles:
+
+- **Open Source**: All code is publicly available under MIT license
+- **Open Data**: Benchmarks and performance data are shared publicly
+- **Open Standards**: We contribute to community standards and protocols
+- **Open Collaboration**: We welcome contributions from the broader community
+
+## Future Directions
+
+Looking ahead, RHAPSODY will continue to evolve to meet emerging needs:
+
+- Enhanced support for quantum computing integration
+- Advanced AI/ML workflow optimization
+- Edge computing and IoT integration for scientific workflows
+- Expanded ecosystem of compatible tools and frameworks

rhapsody_py-0.1.0/docs/api/index.md

@@ -0,0 +1,5 @@
+# API Reference
+
+Welcome to the RHAPSODY API reference documentation.
+
+Browse the modules in the sidebar to explore the API.

rhapsody_py-0.1.0/docs/changelogs.md

@@ -0,0 +1 @@
+--8<-- "CHANGELOG.md"