silver-adapters 0.1.0__tar.gz

silver_adapters-0.1.0/CHANGELOG.md

@@ -0,0 +1,33 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2024-08-04
+
+### Added
+- Initial release of silver-adapters
+- PyTorch bridge utilities for checkpoint management
+- TensorFlow/Keras bridge utilities for model saving/loading
+- JSONL protocol encoding and decoding for event streaming
+- Remote training client with async HTTP support
+- Jupyter notebook template generation
+- Data import plans for pandas and Kaggle
+- Framework command bridge creation
+- Comprehensive test suite with mocking for optional dependencies
+- Support for Python 3.8-3.12
+
+### Features
+- `PyTorchBridge` - PyTorch checkpoint utilities
+- `TensorFlowBridge` - TensorFlow/Keras model utilities
+- `RemoteTrainingClient` - HTTP client for remote training
+- `encode_jsonl()` / `decode_jsonl()` - JSONL protocol handling
+- `notebook_document()` / `silver_notebook_cells()` - Jupyter utilities
+- `python_framework()` - Framework command bridge creation
+- `pandas_import()` / `kaggle_import()` - Data import planning
+- Optional dependencies for specific framework support
+- Async HTTP operations for remote training
+
+## [Unreleased]

silver_adapters-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,140 @@
+# Contributing to silver-adapters
+
+Thank you for your interest in contributing to silver-adapters! This document provides guidelines and instructions for contributing to the project.
+
+## Development Setup
+
+### Prerequisites
+- Python 3.8 or higher
+- Git
+- Virtual environment (recommended)
+
+### Setting Up Development Environment
+
+1. **Clone the repository**
+   ```bash
+   git clone https://github.com/adfgdartec/silver-adapters.git
+   cd silver-adapters
+   ```
+
+2. **Create a virtual environment**
+   ```bash
+   python -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   ```
+
+3. **Install development dependencies**
+   ```bash
+   pip install -e ".[dev]"
+   ```
+
+4. **Install optional dependencies for testing**
+   ```bash
+   pip install -e ".[all]"  # Or specific: [pytorch], [tensorflow], etc.
+   ```
+
+5. **Run tests**
+   ```bash
+   pytest
+   ```
+
+6. **Run tests with coverage**
+   ```bash
+   pytest --cov=silver_adapters --cov-report=html
+   ```
+
+## Code Style
+
+We use the following tools to maintain code quality:
+
+- **flake8** for linting
+- **mypy** for type checking
+- **pytest** for testing
+
+Run all quality checks:
+```bash
+flake8 src/ tests/
+mypy src/
+pytest
+```
+
+## Making Changes
+
+1. **Create a branch**
+   ```bash
+   git checkout -b feature/your-feature-name
+   ```
+
+2. **Make your changes**
+   - Write clear, descriptive commit messages
+   - Add tests for new functionality
+   - Update documentation as needed
+   - Handle optional dependencies gracefully
+
+3. **Run tests**
+   ```bash
+   pytest
+   ```
+
+4. **Submit a pull request**
+   - Describe your changes clearly
+   - Reference any related issues
+   - Ensure all tests pass
+   - Note any optional dependencies required
+
+## Testing
+
+We aim for high test coverage. When adding new features:
+
+- Write unit tests for new adapter functions
+- Mock optional dependencies (torch, tensorflow, etc.)
+- Test error handling for missing dependencies
+- Ensure existing tests still pass
+
+### Test Structure
+```
+tests/
+├── __init__.py
+└── test_adapters.py
+```
+
+## Optional Dependencies
+
+This package has optional dependencies for different ML frameworks:
+
+- `pytorch` - PyTorch support
+- `tensorflow` - TensorFlow support
+- `keras` - Keras support
+- `kaggle` - Kaggle API support
+- `jupyter` - Jupyter notebook support
+
+When adding new framework-specific features:
+
+1. Add the dependency to `pyproject.toml` under `project.optional-dependencies`
+2. Import the dependency inside the function (not at module level)
+3. Provide clear error messages when the dependency is missing
+4. Add tests that mock the dependency
+
+## Documentation
+
+- Update docstrings for any modified functions
+- Add examples for new adapters
+- Update README.md if user-facing changes are made
+- Document any new optional dependencies
+
+## Release Process
+
+Releases are managed by maintainers:
+
+1. Update version in `pyproject.toml`
+2. Update `CHANGELOG.md`
+3. Create a GitHub release
+4. Package will be automatically published to PyPI
+
+## Questions?
+
+Feel free to open an issue for questions or discussions about contributions.
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the Apache-2.0 License.

silver_adapters-0.1.0/LICENSE

@@ -0,0 +1,12 @@
+Apache License
+Version 2.0, January 2004
+
+Copyright 2026 Silver Contributors
+
+Licensed under the Apache License, Version 2.0. You may obtain a copy of the
+License at https://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed
+under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.

silver_adapters-0.1.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include README.md
+include LICENSE
+include CHANGELOG.md
+include CONTRIBUTING.md
+recursive-include src *.py
+recursive-include tests *.py

silver_adapters-0.1.0/PKG-INFO

@@ -0,0 +1,404 @@
+Metadata-Version: 2.4
+Name: silver-adapters
+Version: 0.1.0
+Summary: Backend, dataset, notebook, and remote-training bridge protocols for Silver.
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/adfgdartec/silver-adapters
+Project-URL: Repository, https://github.com/adfgdartec/silver-adapters
+Project-URL: Issues, https://github.com/adfgdartec/silver-adapters/issues
+Keywords: pytorch,keras,tensorflow,pandas,kaggle,jupyter,machine-learning,python
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: pytorch
+Requires-Dist: torch>=1.9.0; extra == "pytorch"
+Provides-Extra: tensorflow
+Requires-Dist: tensorflow>=2.6.0; extra == "tensorflow"
+Provides-Extra: keras
+Requires-Dist: tensorflow>=2.6.0; extra == "keras"
+Provides-Extra: kaggle
+Requires-Dist: kaggle>=1.5.0; extra == "kaggle"
+Provides-Extra: jupyter
+Requires-Dist: jupyter>=1.0.0; extra == "jupyter"
+Requires-Dist: nbformat>=5.0.0; extra == "jupyter"
+Provides-Extra: pandas
+Requires-Dist: pandas>=1.0.0; extra == "pandas"
+Provides-Extra: remote
+Requires-Dist: requests>=2.25.0; extra == "remote"
+Provides-Extra: all
+Requires-Dist: torch>=1.9.0; extra == "all"
+Requires-Dist: tensorflow>=2.6.0; extra == "all"
+Requires-Dist: kaggle>=1.5.0; extra == "all"
+Requires-Dist: jupyter>=1.0.0; extra == "all"
+Requires-Dist: nbformat>=5.0.0; extra == "all"
+Requires-Dist: pandas>=1.0.0; extra == "all"
+Requires-Dist: requests>=2.25.0; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: flake8>=6.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: build>=0.10.0; extra == "dev"
+Requires-Dist: twine>=4.0.0; extra == "dev"
+Dynamic: license-file
+
+# silver-adapters
+
+[![Python Version](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org/downloads/)
+[![License](https://img.shields.io/badge/license-Apache%202.0-green.svg)](LICENSE)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](tests/)
+[![Code Style](https://img.shields.io/badge/code%20style-flake8-blue.svg)](https://flake8.pycqa.org/)
+
+Backend, dataset, notebook, and remote-training bridge protocols for Silver. A Python package designed for ML researchers who need seamless integration between different ML frameworks and tools.
+
+The core package has no mandatory ML framework, pandas, requests, or Jupyter
+dependency. Install only the extras you need:
+
+```bash
+pip install silver-adapters
+pip install 'silver-adapters[pytorch]'
+pip install 'silver-adapters[remote]'
+pip install 'silver-adapters[pandas]'
+pip install 'silver-adapters[jupyter]'
+```
+
+## Installation
+
+```bash
+# Core package
+pip install silver-adapters
+
+# With specific framework support
+pip install silver-adapters[pytorch]
+pip install silver-adapters[tensorflow]
+pip install silver-adapters[kaggle]
+pip install silver-adapters[jupyter]
+
+# With all optional dependencies
+pip install silver-adapters[all]
+```
+
+## Quick Start
+
+### Framework Bridges
+
+```python
+from silver_adapters import python_framework, PyTorchBridge, TensorFlowBridge
+
+# Create a PyTorch training bridge
+bridge = python_framework(
+    framework="pytorch",
+    script="train.py",
+    args=["--config", "config.yaml"]
+)
+
+# PyTorch checkpoint utilities
+checkpoint = PyTorchBridge.from_checkpoint("model.pt")
+PyTorchBridge.to_checkpoint(model, optimizer, epoch=10, path="model.pt")
+
+# TensorFlow/Keras checkpoint utilities
+model = TensorFlowBridge.from_checkpoint("model.h5")
+TensorFlowBridge.to_checkpoint(model, "model.h5")
+```
+
+### Data Import
+
+```python
+from silver_adapters import pandas_import, kaggle_import
+
+# Import from pandas
+plan = pandas_import("data/train.csv")
+
+# Import from Kaggle
+plan = kaggle_import("competiton/dataset-name")
+```
+
+### Remote Training
+
+```python
+import asyncio
+from silver_adapters import RemoteTrainingClient
+
+async def main():
+    client = RemoteTrainingClient("https://api.example.com")
+    
+    # Start training
+    run = await client.start({"config": {...}})
+    
+    # Get events
+    events = await client.events(run["run_id"])
+    
+    # Stop training
+    await client.stop(run["run_id"], "completed successfully")
+
+asyncio.run(main())
+```
+
+### JSONL Protocol
+
+```python
+from silver_adapters import encode_jsonl, decode_jsonl, SilverJsonlEvent
+
+# Encode events
+events = [
+    SilverJsonlEvent(kind="epoch", data={"epoch": 1, "loss": 0.5}),
+    SilverJsonlEvent(kind="metrics", data={"accuracy": 0.9})
+]
+jsonl = encode_jsonl(events)
+
+# Decode events
+decoded = decode_jsonl(jsonl)
+```
+
+## Features
+
+- **Framework Bridges**: Seamless integration with PyTorch, TensorFlow, and Keras
+- **Data Import Plans**: Structured data import from pandas and Kaggle
+- **Remote Training**: Async HTTP client for distributed training
+- **JSONL Protocol**: Event streaming protocol for training observability
+- **Jupyter Integration**: Notebook utilities and template generation
+- **Checkpoint Management**: Unified checkpoint handling across frameworks
+- **Type Safety**: Full type hints for better IDE support and fewer bugs
+
+## Use Cases
+
+### Multi-Framework Training
+
+```python
+from silver_adapters import PyTorchBridge, TensorFlowBridge
+import torch
+import tensorflow as tf
+
+# Train in PyTorch
+pytorch_model = torch.nn.Linear(10, 2)
+optimizer = torch.optim.Adam(pytorch_model.parameters())
+
+# Save PyTorch checkpoint
+PyTorchBridge.to_checkpoint(pytorch_model, optimizer, epoch=10, path="pytorch_model.pt")
+
+# Load in TensorFlow for inference
+tf_model = tf.keras.Sequential([tf.keras.layers.Dense(2, input_shape=(10,))])
+# Convert weights (framework-specific conversion needed)
+TensorFlowBridge.to_checkpoint(tf_model, "tf_model.h5")
+```
+
+### Distributed Training Setup
+
+```python
+import asyncio
+from silver_adapters import RemoteTrainingClient
+
+async def run_distributed_training():
+    # Connect to remote training server
+    client = RemoteTrainingClient("https://training-server.example.com")
+    
+    # Start training on remote GPU cluster
+    config = {
+        "model": "resnet50",
+        "dataset": "imagenet",
+        "batch_size": 32,
+        "epochs": 100
+    }
+    
+    run = await client.start(config)
+    print(f"Started training run: {run['run_id']}")
+    
+    # Monitor training progress
+    while True:
+        events = await client.events(run['run_id'])
+        latest_events = events[-5:]  # Get last 5 events
+        
+        for event in latest_events:
+            if event['kind'] == 'epoch':
+                print(f"Epoch {event['epoch']}: loss={event.get('loss', 'N/A')}")
+            elif event['kind'] == 'completed':
+                print("Training completed!")
+                return
+        
+        await asyncio.sleep(10)  # Check every 10 seconds
+
+asyncio.run(run_distributed_training())
+```
+
+### Data Pipeline Integration
+
+```python
+from silver_adapters import pandas_import, kaggle_import, encode_jsonl
+import pandas as pd
+
+# Create data import plans
+csv_plan = pandas_import("data/train.csv", {"sep": ",", "encoding": "utf-8"})
+kaggle_plan = kaggle_import("competiton/titanic", {"unzip": True})
+
+# Use plans in your data pipeline
+def execute_import_plan(plan):
+    """Execute a data import plan"""
+    if plan.source == "pandas":
+        df = pd.read_csv(plan.args[1], **plan.options)
+        return df
+    elif plan.source == "kaggle":
+        # Execute Kaggle download command
+        import subprocess
+        subprocess.run([plan.command] + plan.args)
+        return pd.read_csv("downloaded_file.csv")
+
+# Create training events from pandas DataFrame
+def create_training_events(df):
+    from silver_adapters import SilverJsonlEvent
+    events = []
+    
+    for epoch in range(10):
+        # Simulate training metrics
+        events.append(SilverJsonlEvent(
+            kind="epoch",
+            data={"epoch": epoch, "loss": 0.5 - epoch * 0.05}
+        ))
+    
+    return encode_jsonl(events)
+```
+
+### Jupyter Notebook Integration
+
+```python
+from silver_adapters import notebook_document, silver_notebook_cells
+
+# Create a Silver training notebook
+cells = silver_notebook_cells()
+notebook = notebook_document(cells)
+
+# Save as Jupyter notebook
+import json
+with open("silver_training.ipynb", "w") as f:
+    json.dump({
+        "cells": [
+            {
+                "cell_type": cell.cell_type,
+                "metadata": cell.metadata,
+                "source": cell.source,
+                "outputs": cell.outputs or [],
+                "execution_count": cell.execution_count
+            } for cell in notebook.cells
+        ],
+        "metadata": notebook.metadata,
+        "nbformat": notebook.nbformat,
+        "nbformat_minor": notebook.nbformat_minor
+    }, f, indent=2)
+```
+
+## Advanced Usage
+
+### Custom Training Backend
+
+```python
+from silver_adapters import python_framework, PythonFramework
+import subprocess
+
+class CustomTrainingBackend:
+    def __init__(self, framework, script_path):
+        self.bridge = python_framework(
+            framework=framework,
+            script=script_path,
+            environment={"CUDA_VISIBLE_DEVICES": "0"}
+        )
+    
+    def launch_training(self, args):
+        """Launch training with given arguments"""
+        cmd = [self.bridge.command] + self.bridge.args + args
+        env = {**self.bridge.environment}
+        
+        process = subprocess.Popen(
+            cmd,
+            env={**subprocess.os.environ, **env}
+        )
+        return process
+
+# Usage
+backend = CustomTrainingBackend(
+    PythonFramework.PYTORCH,
+    "train.py"
+)
+process = backend.launch_training(["--epochs", "100", "--batch-size", "32"])
+```
+
+### Event Streaming
+
+```python
+from silver_adapters import encode_jsonl, decode_jsonl, SilverJsonlEvent
+import asyncio
+
+async def stream_training_events(writer):
+    """Stream training events to a writer"""
+    for epoch in range(10):
+        event = SilverJsonlEvent(
+            kind="epoch",
+            data={"epoch": epoch, "loss": 0.5 - epoch * 0.05}
+        )
+        writer.write(encode_jsonl([event]))
+        await asyncio.sleep(0.1)
+
+async def consume_training_events(reader):
+    """Consume training events from a reader"""
+    buffer = ""
+    async for chunk in reader:
+        buffer += chunk
+        events = decode_jsonl(buffer)
+        for event in events:
+            print(f"Received: {event.kind} - {event.data}")
+        buffer = ""
+```
+
+## Requirements
+
+- Python 3.8+
+- pandas>=1.0.0
+- requests>=2.25.0
+
+### Optional Dependencies
+- torch>=1.9.0 (for PyTorch support)
+- tensorflow>=2.6.0 (for TensorFlow/Keras support)
+- kaggle>=1.5.0 (for Kaggle integration)
+- jupyter>=1.0.0 (for notebook utilities)
+
+## Development
+
+```bash
+# Install development dependencies
+pip install -e ".[dev]"
+
+# Install all optional dependencies for testing
+pip install -e ".[all]"
+
+# Run tests
+pytest
+
+# Run tests with coverage
+pytest --cov=silver_adapters --cov-report=html
+
+# Run linting
+flake8 src/ tests/
+mypy src/
+```
+
+## Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+## License
+
+Apache-2.0 - see [LICENSE](LICENSE) file for details.
+
+## Related Packages
+
+- [silver-data](https://github.com/adfgdartec/silver-data) - Dataset handling
+- [silver-run](https://github.com/adfgdartec/silver-run) - Training lifecycle
+- [silver-diagnostics](https://github.com/adfgdartec/silver-diagnostics) - ML diagnostics