libinephany 1.1.22__tar.gz

libinephany-1.1.22/CODE_VERSION.cfg

@@ -0,0 +1 @@
+1.1.22

libinephany-1.1.22/LICENSE

@@ -0,0 +1,180 @@
+Copyright 2025 Inephany Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction,
+and distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by
+the copyright owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all
+other entities that control, are controlled by, or are under common
+control with that entity. For the purposes of this definition,
+"control" means (i) the power, direct or indirect, to cause the
+direction or management of such entity, whether by contract or
+otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity
+exercising permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications,
+including but not limited to software source code, documentation
+source, and configuration files.
+
+"Object" form shall mean any form resulting from mechanical
+transformation or translation of a Source form, including but
+not limited to compiled object code, generated documentation,
+and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or
+Object form, made available under the License, as indicated by a
+copyright notice that is included in or attached to the work
+(an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object
+form, that is based on (or derived from) the Work and for which the
+editorial revisions, annotations, elaborations, or other modifications
+represent, as a whole, an original work of authorship. For the purposes
+of this License, Derivative Works shall not include works that remain
+separable from, or merely link (or bind by name) to the interfaces of,
+the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including
+the original version of the Work and any modifications or additions
+to that Work or Derivative Works thereof, that is intentionally
+submitted to Licensor for inclusion in the Work by the copyright owner
+or by an individual or Legal Entity authorized to submit on behalf of
+the copyright owner. For the purposes of this definition, "submitted"
+means any form of electronic, verbal, or written communication sent
+to the Licensor or its representatives, including but not limited to
+communication on electronic mailing lists, source code control systems,
+and issue tracking systems that are managed by, or on behalf of, the
+Licensor for the purpose of discussing and improving the Work, but
+excluding communication that is conspicuously marked or otherwise
+designated in writing by the copyright owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity
+on behalf of whom a Contribution has been received by Licensor and
+subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+this License, each Contributor hereby grants to You a perpetual,
+worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+copyright license to reproduce, prepare Derivative Works of,
+publicly display, publicly perform, sublicense, and distribute the
+Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+this License, each Contributor hereby grants to You a perpetual,
+worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+(except as stated in this section) patent license to make, have made,
+use, offer to sell, sell, import, and otherwise transfer the Work,
+where such license applies only to those patent claims licensable
+by such Contributor that are necessarily infringed by their
+Contribution(s) alone or by combination of their Contribution(s)
+with the Work to which such Contribution(s) was submitted. If You
+institute patent litigation against any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Work
+or a Contribution incorporated within the Work constitutes direct
+or contributory patent infringement, then any patent licenses
+granted to You under this License for that Work shall terminate
+as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+Work or Derivative Works thereof in any medium, with or without
+modifications, and in Source or Object form, provided that You
+meet the following conditions:
+
+(a) You must give any other recipients of the Work or
+Derivative Works a copy of this License; and
+
+(b) You must cause any modified files to carry prominent notices
+stating that You changed the files; and
+
+(c) You must retain, in the Source form of any Derivative Works
+that You distribute, all copyright, patent, trademark, and
+attribution notices from the Source form of the Work,
+excluding those notices that do not pertain to any part of
+the Derivative Works; and
+
+(d) If the Work includes a "NOTICE" text file as part of its
+distribution, then any Derivative Works that You distribute must
+include a readable copy of the attribution notices contained
+within such NOTICE file, excluding those notices that do not
+pertain to any part of the Derivative Works, in at least one
+of the following places: within a NOTICE text file distributed
+as part of the Derivative Works; within the Source form or
+documentation, if provided along with the Derivative Works; or,
+within a display generated by the Derivative Works, if and
+wherever such third-party notices normally appear. The contents
+of the NOTICE file are for informational purposes only and
+do not modify the License. You may add Your own attribution
+notices within Derivative Works that You distribute, alongside
+or as an addendum to the NOTICE text from the Work, provided
+that such additional attribution notices cannot be construed
+as modifying the License.
+
+You may add Your own copyright statement to Your modifications and
+may provide additional or different license terms and conditions
+for use, reproduction, or distribution of Your modifications, or
+for any such Derivative Works as a whole, provided Your use,
+reproduction, and distribution of the Work otherwise complies with
+the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+any Contribution intentionally submitted for inclusion in the Work
+by You to the Licensor shall be under the terms and conditions of
+this License, without any additional terms or conditions.
+Notwithstanding the above, nothing herein shall supersede or modify
+the terms of any separate license agreement you may have executed
+with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+names, trademarks, service marks, or product names of the Licensor,
+except as required for reasonable and customary use in describing the
+origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+agreed to in writing, Licensor provides the Work (and each
+Contributor provides its Contributions) on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+implied, including, without limitation, any warranties or conditions
+of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+PARTICULAR PURPOSE. You are solely responsible for determining the
+appropriateness of using or redistributing the Work and assume any
+risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+whether in tort (including negligence), contract, or otherwise,
+unless required by applicable law (such as deliberate and grossly
+negligent acts) or agreed to in writing, shall any Contributor be
+liable to You for damages, including any direct, indirect, special,
+incidental, or consequential damages of any character arising as a
+result of this License or out of the use or inability to use the
+Work (including but not limited to damages for loss of goodwill,
+work stoppage, computer failure or malfunction, or any and all
+other commercial damages or losses), even if such Contributor
+has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+the Work or Derivative Works thereof, You may choose to offer,
+and charge a fee for, acceptance of support, warranty, indemnity,
+or other liability obligations and/or rights consistent with this
+License. However, in accepting such obligations, You may act only
+on Your own behalf and on Your sole responsibility, not on behalf
+of any other Contributor, and only if You agree to indemnify,
+defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason
+of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS

libinephany-1.1.22/MANIFEST.in

@@ -0,0 +1,7 @@
+include CODE_VERSION.cfg
+include README.md
+include LICENSE
+include pyproject.toml
+recursive-include libinephany *.py
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]

libinephany-1.1.22/PKG-INFO

@@ -0,0 +1,274 @@
+Metadata-Version: 2.4
+Name: libinephany
+Version: 1.1.22
+Summary: Inephany library containing code commonly used by multiple subpackages.
+Author-email: Inephany <info@inephany.com>
+License: Apache 2.0
+Keywords: libinephany,library,utilities
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic<3.0.0,>=2.5.0
+Requires-Dist: loguru<0.8.0,>=0.7.0
+Requires-Dist: requests<3.0.0,>=2.28.0
+Requires-Dist: numpy<2.0.0,>=1.24.0
+Requires-Dist: scipy<2.0.0,>=1.10.0
+Requires-Dist: slack-sdk<4.0.0,>=3.20.0
+Requires-Dist: boto3<2.0.0,>=1.26.0
+Requires-Dist: fastapi<0.116.0,>=0.100.0
+Requires-Dist: aiohttp<4.0.0,>=3.8.0
+Requires-Dist: torch<2.9.0,>=2.1.0
+Requires-Dist: transformers<4.48.3,>=4.30.0
+Requires-Dist: pandas<3.0.0,>=2.0.0
+Requires-Dist: accelerate<2.0.0,>=0.20.0
+Requires-Dist: gymnasium<2.0.0,>=0.29.0
+Provides-Extra: dev
+Requires-Dist: pytest<9.0.0,>=7.0.0; extra == "dev"
+Requires-Dist: pytest-mock<4.0.0,>=3.10.0; extra == "dev"
+Requires-Dist: pytest-asyncio<0.26.0,>=0.21.0; extra == "dev"
+Requires-Dist: bump-my-version==0.11.0; extra == "dev"
+Requires-Dist: black==24.4.2; extra == "dev"
+Requires-Dist: isort==5.9.3; extra == "dev"
+Requires-Dist: flake8==7.1.0; extra == "dev"
+Requires-Dist: pre-commit==4.0.1; extra == "dev"
+Requires-Dist: mypy==1.13.0; extra == "dev"
+Requires-Dist: types-PyYAML==6.0.12.20240808; extra == "dev"
+Requires-Dist: typeguard==4.3.0; extra == "dev"
+Dynamic: license-file
+
+# Inephany Common Library
+
+The Inephany Common Library (`libinephany`) is a core utility package that provides shared functionality, data models, and utilities used across multiple Inephany packages. It contains essential components for hyperparameter optimization, model observation, data serialization, and common utilities.
+
+## Features
+
+- **Pydantic Data Models**: Comprehensive schemas for hyperparameters, observations, and API communications
+- **Utility Functions**: Common utilities for PyTorch, optimization, transforms, and more
+- **Observation System**: Tools for collecting and managing model statistics and observations
+- **Constants and Enums**: Standardized constants and enumerations for agent types, model families, and module types
+- **AWS Integration**: Utilities for AWS services integration
+- **Web Application Utilities**: Common web app functionality and endpoints
+
+## Installation
+
+### Prerequisites
+
+- Python 3.10+
+- Make (for build automation)
+
+#### Ubuntu / Debian
+```bash
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt update
+sudo apt install python3.12
+```
+
+#### MacOS with brew
+```bash
+brew install python@3.12
+```
+
+### For Developers (Monorepo)
+
+If you're working within the Inephany monorepo, the package is already available and will be installed automatically when you run the installation commands in dependent packages.
+
+### For Clients (Standalone Installation)
+
+`libinephany` is available on PyPI and can be installed directly:
+
+```bash
+pip install libinephany
+```
+
+For development installations with additional dependencies:
+
+```bash
+pip install libinephany[dev]
+```
+
+## Key Components
+
+### Pydantic Models
+
+The package provides comprehensive data models for:
+
+- **Hyperparameter Configurations**: `HParamConfig`, `HParamConfigs`
+- **Observation Models**: `ObservationInputs`, tensor statistics
+- **API Schemas**: Request/response models for client-server communication
+- **State Management**: Hyperparameter states and update callbacks
+
+### Utility Functions
+
+#### Agent Utilities (`agent_utils.py`)
+- Agent ID generation and parsing
+- Hyperparameter group management
+- Agent type validation
+
+#### Constants (`constants.py`)
+- Hyperparameter type constants (learning_rate, weight_decay, etc.)
+- Agent prefixes and suffixes
+- API key headers and timestamp formats
+
+#### Enums (`enums.py`)
+- `AgentTypes`: Learning rate, weight decay, dropout, etc.
+- `ModelFamilies`: GPT, BERT, OLMo
+- `ModuleTypes`: Convolutional, attention, linear, embedding
+
+#### Optimization Utilities (`optim_utils.py`)
+- PyTorch optimizer utilities
+- Parameter group management
+- Learning rate scheduler utilities
+
+#### PyTorch Utilities (`torch_utils.py`)
+- Tensor operations
+- Model utilities
+- Distributed training helpers
+
+### Observation System
+
+The observation system provides tools for collecting and managing model statistics:
+
+- **StatisticManager**: Centralized statistics collection and management
+- **ObserverPipeline**: Configurable observation pipelines
+- **PipelineCoordinator**: Coordinates multiple observers
+- **StatisticTrackers**: Specialized trackers for different metric types
+
+## Usage Examples
+
+### Basic Import Examples
+
+```python
+# Import common constants
+from libinephany.utils.constants import LEARNING_RATE, WEIGHT_DECAY, AGENT_PREFIX_LR
+
+# Import enums
+from libinephany.utils.enums import AgentTypes, ModelFamilies, ModuleTypes
+
+# Import utility functions
+from libinephany.utils import agent_utils, optim_utils, torch_utils
+
+# Import data models
+from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
+from libinephany.pydantic_models.schemas.response_schemas import ClientPolicySchemaResponse
+```
+
+### Working with Agent Types
+
+```python
+from libinephany.utils.enums import AgentTypes
+
+# Check if an agent type is valid
+agent_type = "learning_rate"
+if agent_type in [agent.value for agent in AgentTypes]:
+    print(f"{agent_type} is a valid agent type")
+
+# Get agent type by index
+lr_agent = AgentTypes.get_from_index(0)  # LearningRateAgent
+```
+
+### Using Constants
+
+```python
+from libinephany.utils.constants import AGENT_PREFIX_LR, LEARNING_RATE
+
+# Generate agent ID
+agent_id = f"{AGENT_PREFIX_LR}_agent_001"
+hyperparam_type = LEARNING_RATE
+```
+
+### Working with Pydantic Models
+
+```python
+from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
+
+# Create a hyperparameter configuration
+config = HParamConfig(
+    name="learning_rate",
+    value=0.001,
+    min_value=1e-6,
+    max_value=1.0
+)
+```
+
+## Development
+
+### Running Tests
+```bash
+make execute-unit-tests
+```
+
+### Code Quality
+```bash
+make lint          # Run all linters
+make fix-black     # Fix formatting
+make fix-isort     # Fix imports
+```
+
+### Version Management
+```bash
+make increment-patch-version    # Increment patch version
+make increment-minor-version    # Increment minor version
+make increment-major-version    # Increment major version
+make increment-pre-release-version # Increment pre-release version
+```
+
+## Dependencies
+
+### Core Dependencies
+- `pydantic==2.8.2` - Data validation and serialization
+- `torch==2.7.1` - PyTorch for tensor operations
+- `numpy==1.26.4` - Numerical computing
+- `requests==2.32.4` - HTTP client
+- `loguru==0.7.2` - Logging
+
+### Optional Dependencies
+- `boto3<=1.38.44` - AWS SDK
+- `fastapi==0.115.11` - Web framework
+- `slack-sdk==3.35.0` - Slack integration
+- `transformers==4.52.4` - Hugging Face transformers
+- `accelerate==1.4.0` - Hugging Face accelerate
+- `gymnasium==1.0.0` - RL environments
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Import Errors**: Ensure you're in the virtual environment and have installed the package correctly.
+
+2. **Version Conflicts**: If you encounter dependency conflicts, try installing in a fresh virtual environment:
+   ```bash
+   python -m venv fresh_env
+   source fresh_env/bin/activate
+   make install-dev
+   ```
+
+3. **Make Command Not Found**: Ensure you have `make` installed on your system.
+
+4. **Python Version Issues**: This package requires Python 3.12+. Ensure you're using the correct version.
+
+### Getting Help
+
+- Check the example scripts in the repository
+- Review the test files for usage examples
+- Ensure all dependencies are installed correctly
+- Verify your Python version is 3.12+
+
+## Contributing
+
+When contributing to `libinephany`:
+
+1. Follow the existing code style (Black, isort, flake8)
+2. Add appropriate type hints
+3. Include unit tests for new functionality
+4. Update documentation for new features
+5. Ensure all tests pass before submitting
+
+## License
+
+This package is licensed under the Apache License, Version 2.0. See the LICENSE file for details.

libinephany-1.1.22/README.md

@@ -0,0 +1,230 @@
+# Inephany Common Library
+
+The Inephany Common Library (`libinephany`) is a core utility package that provides shared functionality, data models, and utilities used across multiple Inephany packages. It contains essential components for hyperparameter optimization, model observation, data serialization, and common utilities.
+
+## Features
+
+- **Pydantic Data Models**: Comprehensive schemas for hyperparameters, observations, and API communications
+- **Utility Functions**: Common utilities for PyTorch, optimization, transforms, and more
+- **Observation System**: Tools for collecting and managing model statistics and observations
+- **Constants and Enums**: Standardized constants and enumerations for agent types, model families, and module types
+- **AWS Integration**: Utilities for AWS services integration
+- **Web Application Utilities**: Common web app functionality and endpoints
+
+## Installation
+
+### Prerequisites
+
+- Python 3.10+
+- Make (for build automation)
+
+#### Ubuntu / Debian
+```bash
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt update
+sudo apt install python3.12
+```
+
+#### MacOS with brew
+```bash
+brew install python@3.12
+```
+
+### For Developers (Monorepo)
+
+If you're working within the Inephany monorepo, the package is already available and will be installed automatically when you run the installation commands in dependent packages.
+
+### For Clients (Standalone Installation)
+
+`libinephany` is available on PyPI and can be installed directly:
+
+```bash
+pip install libinephany
+```
+
+For development installations with additional dependencies:
+
+```bash
+pip install libinephany[dev]
+```
+
+## Key Components
+
+### Pydantic Models
+
+The package provides comprehensive data models for:
+
+- **Hyperparameter Configurations**: `HParamConfig`, `HParamConfigs`
+- **Observation Models**: `ObservationInputs`, tensor statistics
+- **API Schemas**: Request/response models for client-server communication
+- **State Management**: Hyperparameter states and update callbacks
+
+### Utility Functions
+
+#### Agent Utilities (`agent_utils.py`)
+- Agent ID generation and parsing
+- Hyperparameter group management
+- Agent type validation
+
+#### Constants (`constants.py`)
+- Hyperparameter type constants (learning_rate, weight_decay, etc.)
+- Agent prefixes and suffixes
+- API key headers and timestamp formats
+
+#### Enums (`enums.py`)
+- `AgentTypes`: Learning rate, weight decay, dropout, etc.
+- `ModelFamilies`: GPT, BERT, OLMo
+- `ModuleTypes`: Convolutional, attention, linear, embedding
+
+#### Optimization Utilities (`optim_utils.py`)
+- PyTorch optimizer utilities
+- Parameter group management
+- Learning rate scheduler utilities
+
+#### PyTorch Utilities (`torch_utils.py`)
+- Tensor operations
+- Model utilities
+- Distributed training helpers
+
+### Observation System
+
+The observation system provides tools for collecting and managing model statistics:
+
+- **StatisticManager**: Centralized statistics collection and management
+- **ObserverPipeline**: Configurable observation pipelines
+- **PipelineCoordinator**: Coordinates multiple observers
+- **StatisticTrackers**: Specialized trackers for different metric types
+
+## Usage Examples
+
+### Basic Import Examples
+
+```python
+# Import common constants
+from libinephany.utils.constants import LEARNING_RATE, WEIGHT_DECAY, AGENT_PREFIX_LR
+
+# Import enums
+from libinephany.utils.enums import AgentTypes, ModelFamilies, ModuleTypes
+
+# Import utility functions
+from libinephany.utils import agent_utils, optim_utils, torch_utils
+
+# Import data models
+from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
+from libinephany.pydantic_models.schemas.response_schemas import ClientPolicySchemaResponse
+```
+
+### Working with Agent Types
+
+```python
+from libinephany.utils.enums import AgentTypes
+
+# Check if an agent type is valid
+agent_type = "learning_rate"
+if agent_type in [agent.value for agent in AgentTypes]:
+    print(f"{agent_type} is a valid agent type")
+
+# Get agent type by index
+lr_agent = AgentTypes.get_from_index(0)  # LearningRateAgent
+```
+
+### Using Constants
+
+```python
+from libinephany.utils.constants import AGENT_PREFIX_LR, LEARNING_RATE
+
+# Generate agent ID
+agent_id = f"{AGENT_PREFIX_LR}_agent_001"
+hyperparam_type = LEARNING_RATE
+```
+
+### Working with Pydantic Models
+
+```python
+from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
+
+# Create a hyperparameter configuration
+config = HParamConfig(
+    name="learning_rate",
+    value=0.001,
+    min_value=1e-6,
+    max_value=1.0
+)
+```
+
+## Development
+
+### Running Tests
+```bash
+make execute-unit-tests
+```
+
+### Code Quality
+```bash
+make lint          # Run all linters
+make fix-black     # Fix formatting
+make fix-isort     # Fix imports
+```
+
+### Version Management
+```bash
+make increment-patch-version    # Increment patch version
+make increment-minor-version    # Increment minor version
+make increment-major-version    # Increment major version
+make increment-pre-release-version # Increment pre-release version
+```
+
+## Dependencies
+
+### Core Dependencies
+- `pydantic==2.8.2` - Data validation and serialization
+- `torch==2.7.1` - PyTorch for tensor operations
+- `numpy==1.26.4` - Numerical computing
+- `requests==2.32.4` - HTTP client
+- `loguru==0.7.2` - Logging
+
+### Optional Dependencies
+- `boto3<=1.38.44` - AWS SDK
+- `fastapi==0.115.11` - Web framework
+- `slack-sdk==3.35.0` - Slack integration
+- `transformers==4.52.4` - Hugging Face transformers
+- `accelerate==1.4.0` - Hugging Face accelerate
+- `gymnasium==1.0.0` - RL environments
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Import Errors**: Ensure you're in the virtual environment and have installed the package correctly.
+
+2. **Version Conflicts**: If you encounter dependency conflicts, try installing in a fresh virtual environment:
+   ```bash
+   python -m venv fresh_env
+   source fresh_env/bin/activate
+   make install-dev
+   ```
+
+3. **Make Command Not Found**: Ensure you have `make` installed on your system.
+
+4. **Python Version Issues**: This package requires Python 3.12+. Ensure you're using the correct version.
+
+### Getting Help
+
+- Check the example scripts in the repository
+- Review the test files for usage examples
+- Ensure all dependencies are installed correctly
+- Verify your Python version is 3.12+
+
+## Contributing
+
+When contributing to `libinephany`:
+
+1. Follow the existing code style (Black, isort, flake8)
+2. Add appropriate type hints
+3. Include unit tests for new functionality
+4. Update documentation for new features
+5. Ensure all tests pass before submitting
+
+## License
+
+This package is licensed under the Apache License, Version 2.0. See the LICENSE file for details.