tensor-optix 0.1.0__tar.gz

tensor_optix-0.1.0/.gitignore

@@ -0,0 +1,53 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+*.egg
+*.egg-info/
+dist/
+build/
+eggs/
+parts/
+var/
+sdist/
+develop-eggs/
+.installed.cfg
+lib/
+lib64/
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+env/
+
+# uv lock (libraries don't pin lock files)
+uv.lock
+
+# Testing
+.pytest_cache/
+.coverage
+coverage.xml
+htmlcov/
+.tox/
+
+# Type checking
+.mypy_cache/
+.dmypy.json
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Project-specific
+tensor_optix_checkpoints/
+*.keras
+*.h5

tensor_optix-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 sup3rus3r
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tensor_optix-0.1.0/PKG-INFO

@@ -0,0 +1,299 @@
+Metadata-Version: 2.4
+Name: tensor-optix
+Version: 0.1.0
+Summary: Autonomous continuous learning loop for TensorFlow RL agents
+Author: sup3rus3r
+License: MIT License
+        
+        Copyright (c) 2026 sup3rus3r
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: gymnasium>=1.0.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: tensorflow>=2.18.0
+Provides-Extra: dev
+Requires-Dist: black; extra == 'dev'
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Requires-Dist: twine>=5.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# tensor-optix
+
+Autonomous continuous learning loop for TensorFlow RL agents.
+
+> **We own the loop. You own the model.**
+
+tensor-optix wraps your TensorFlow model and Gymnasium environment and takes full ownership of the training loop — stepping continuously, evaluating performance windows, tuning hyperparameters, checkpointing, and adapting over time without manual intervention.
+
+**No fixed episodes.** Training runs as a continuous stream of steps. The loop determines when training ends — not the environment's `done` flag.
+
+---
+
+## Install
+
+```bash
+pip install tensor-optix
+```
+
+**Requirements:** Python >= 3.11, TensorFlow >= 2.18, Gymnasium >= 1.0
+
+---
+
+## Quick Start
+
+```python
+import tensorflow as tf
+import gymnasium as gym
+from tensor_optix import RLOptimizer, TFAgent, BatchPipeline, HyperparamSet
+
+# Build your model normally
+model = tf.keras.Sequential([
+    tf.keras.layers.Input(shape=(4,)),
+    tf.keras.layers.Dense(64, activation="relu"),
+    tf.keras.layers.Dense(64, activation="relu"),
+    tf.keras.layers.Dense(2),
+])
+optimizer = tf.keras.optimizers.Adam(learning_rate=3e-4)
+
+agent = TFAgent(
+    model=model,
+    optimizer=optimizer,
+    hyperparams=HyperparamSet(
+        params={"learning_rate": 3e-4, "gamma": 0.99},
+        episode_id=0,
+    ),
+)
+
+# Continuous stepping — windows of 200 steps, no forced resets
+env = gym.make("CartPole-v1")
+pipeline = BatchPipeline(env=env, agent=agent, window_size=200)
+
+opt = RLOptimizer(agent=agent, pipeline=pipeline)
+opt.run()  # runs until DORMANT (plateau) or max_episodes
+```
+
+---
+
+## How It Works
+
+tensor-optix runs an autonomous improvement loop with four states:
+
+```
+ACTIVE   → aggressive tuning, evaluates every window
+COOLING  → recent improvement, exponential backoff on eval frequency
+DORMANT  → plateau reached — model is trained, minimal intervention
+WATCHDOG → monitoring for degradation
+```
+
+**DORMANT = trained.** The backoff determines when the model can no longer improve, not a fixed episode count.
+
+The loop:
+1. Steps continuously through the environment in fixed-size windows
+2. Evaluates each window via `primary_score`
+3. If improved: saves checkpoint, resets backoff
+4. If plateau: backs off evaluation, eventually reaches DORMANT
+5. If degraded: optionally rolls back to best checkpoint, re-activates
+6. Tunes hyperparameters using two-phase finite difference
+
+---
+
+## Optimizer — Two-Phase Finite Difference
+
+`BackoffOptimizer` uses staggered two-phase finite difference per param:
+
+```
+Phase 1 (probe):  apply θᵢ + δᵢ, run one window
+Phase 2 (commit): gradient = (score_after - score_before) / δᵢ
+                  if gradient > 0: keep θᵢ + δᵢ
+                  if gradient < 0: apply θᵢ - δᵢ  (reverse)
+```
+
+Params are cycled round-robin. Each param is probed and committed independently. Step size adapts on improvement and plateau.
+
+```python
+from tensor_optix import BackoffOptimizer
+
+opt = RLOptimizer(
+    agent=agent,
+    pipeline=pipeline,
+    optimizer=BackoffOptimizer(
+        param_bounds={
+            "learning_rate": (1e-5, 1e-2),
+            "gamma": (0.9, 0.999),
+        },
+        perturbation_scale=0.05,
+    ),
+)
+```
+
+### PBTOptimizer
+
+Pseudo population-based training. Maintains a history of `(hyperparams, score)` pairs. Exploits top performers when in the bottom 20%, explores otherwise.
+
+```python
+from tensor_optix import PBTOptimizer
+
+opt = RLOptimizer(
+    agent=agent,
+    pipeline=pipeline,
+    optimizer=PBTOptimizer(
+        param_bounds={"learning_rate": (1e-5, 1e-2)},
+        history_size=50,
+    ),
+)
+```
+
+---
+
+## Custom Evaluator
+
+```python
+from tensor_optix import BaseEvaluator, EpisodeData, EvalMetrics
+
+class TotalRewardEvaluator(BaseEvaluator):
+    def score(self, episode_data: EpisodeData, train_diagnostics: dict) -> EvalMetrics:
+        total = sum(episode_data.rewards)
+        return EvalMetrics(
+            primary_score=total,
+            metrics={"total_reward": total},
+            episode_id=episode_data.episode_id,
+        )
+
+opt = RLOptimizer(agent=agent, pipeline=pipeline, evaluator=TotalRewardEvaluator())
+```
+
+---
+
+## Custom Agent (Algorithm-Specific Learning)
+
+`TFAgent` provides a REINFORCE baseline. Subclass and override `learn()` for PPO, SAC, DQN, etc.:
+
+```python
+from tensor_optix import TFAgent
+from tensor_optix.core.types import EpisodeData
+import tensorflow as tf
+
+class PPOAgent(TFAgent):
+    def learn(self, episode_data: EpisodeData) -> dict:
+        clip_ratio = self._hyperparams.params.get("clip_ratio", 0.2)
+        # ... PPO update logic ...
+        return {"loss": loss_value, "entropy": entropy_value}
+```
+
+---
+
+## Live Pipeline
+
+For real-time data sources (trading, robotics, online environments):
+
+```python
+from tensor_optix import LivePipeline
+
+class MyFeed:
+    def stream(self):
+        while True:
+            yield obs, reward, terminated, truncated, info
+
+pipeline = LivePipeline(
+    data_source=MyFeed(),
+    agent=agent,
+    episode_boundary_fn=LivePipeline.every_n_seconds(300),
+)
+```
+
+---
+
+## Callbacks
+
+```python
+from tensor_optix import LoopCallback
+
+class MyLogger(LoopCallback):
+    def on_improvement(self, snapshot):
+        print(f"New best: {snapshot.eval_metrics.primary_score:.4f}")
+
+    def on_dormant(self, window_id):
+        print(f"Training complete at window {window_id}")
+
+opt = RLOptimizer(agent=agent, pipeline=pipeline, callbacks=[MyLogger()])
+```
+
+Available hooks: `on_loop_start`, `on_loop_stop`, `on_episode_end`, `on_improvement`, `on_plateau`, `on_dormant`, `on_degradation`, `on_hyperparam_update`.
+
+---
+
+## Full Configuration
+
+```python
+opt = RLOptimizer(
+    agent=agent,
+    pipeline=pipeline,
+    evaluator=None,                     # default: TFEvaluator
+    optimizer=None,                     # default: BackoffOptimizer
+    checkpoint_dir="./checkpoints",
+    max_snapshots=10,
+    rollback_on_degradation=False,
+    improvement_margin=0.0,
+    max_episodes=None,                  # None = run until DORMANT
+    base_interval=1,
+    backoff_factor=2.0,
+    max_interval_episodes=100,
+    plateau_threshold=5,
+    dormant_threshold=20,
+    degradation_threshold=0.95,
+    callbacks=[],
+)
+```
+
+---
+
+## Architecture
+
+```
+tensor_optix/
+├── core/
+│   ├── types.py                # EpisodeData, EvalMetrics, HyperparamSet, LoopState
+│   ├── base_agent.py           # BaseAgent — 6-method contract
+│   ├── base_evaluator.py
+│   ├── base_optimizer.py
+│   ├── base_pipeline.py
+│   ├── loop_controller.py      # State machine + main loop
+│   ├── checkpoint_registry.py
+│   └── backoff_scheduler.py
+├── adapters/tensorflow/
+│   ├── tf_agent.py             # TFAgent — Keras model wrapper
+│   └── tf_evaluator.py         # TFEvaluator — default scorer
+├── pipeline/
+│   ├── batch_pipeline.py       # Continuous stepping, fixed windows
+│   └── live_pipeline.py        # Real-time streaming
+└── optimizers/
+    ├── backoff_optimizer.py    # Two-phase finite difference
+    └── pbt_optimizer.py        # Pseudo population-based training
+```
+
+---
+
+## License
+
+MIT — Copyright (c) 2026 sup3rus3r

tensor_optix-0.1.0/PLAN.md

@@ -0,0 +1,268 @@
+# tensor-optix — Living Implementation Plan
+
+> This document is the single source of truth for building tensor-optix.
+> Update it as decisions are made, issues are found, and tasks complete.
+
+---
+
+## Project Identity
+
+- **Package name:** `tensor-optix`
+- **Import name:** `tensor_optix`
+- **Root directory:** `d:\development\AugData\tensor-optix\`
+- **Python:** `>=3.11`
+- **Framework:** TensorFlow `>=2.18.0` (TF only, no framework abstraction)
+- **Environment API:** Gymnasium `>=1.0.0` (modern API: `terminated | truncated`, not `done`)
+
+---
+
+## What This Is
+
+A PyPI-distributable Python library that replaces the conventional RL training loop with an autonomous, continuously-learning optimization system. The user builds their TF model and Gymnasium environment. The library owns the training loop, evaluation, hyperparameter tuning, checkpointing, and adaptation lifecycle.
+
+**Core philosophy:** We own the loop. The user owns the model.
+
+---
+
+## Architecture Summary
+
+```
+RLOptimizer (main entry point)
+    └── LoopController (state machine + loop orchestration)
+            ├── BaseAgent          ← user implements this
+            ├── BaseEvaluator      ← user implements or use TFEvaluator default
+            ├── BaseOptimizer      ← BackoffOptimizer or PBTOptimizer
+            ├── BasePipeline       ← BatchPipeline or LivePipeline
+            ├── CheckpointRegistry ← snapshot storage
+            └── BackoffScheduler   ← interval + state management
+```
+
+### Loop States
+| State | Behavior |
+|-------|----------|
+| ACTIVE | Aggressive tuning, eval every episode |
+| COOLING | Recent improvement, exponential backoff |
+| DORMANT | Plateau, minimal intervention |
+| WATCHDOG | Monitoring for degradation |
+
+---
+
+## Repository Structure
+
+```
+tensor-optix/
+├── PLAN.md                            ← this file
+├── pyproject.toml
+├── README.md
+├── LICENSE
+│
+├── tensor_optix/
+│   ├── __init__.py                    # Public API surface
+│   │
+│   ├── core/
+│   │   ├── __init__.py
+│   │   ├── types.py                   # EpisodeData, EvalMetrics, HyperparamSet, PolicySnapshot, LoopState
+│   │   ├── base_agent.py              # Abstract BaseAgent
+│   │   ├── base_evaluator.py          # Abstract BaseEvaluator
+│   │   ├── base_optimizer.py          # Abstract BaseOptimizer
+│   │   ├── base_pipeline.py           # Abstract BasePipeline + EpisodeBoundaryFn
+│   │   ├── loop_controller.py         # LoopController + LoopCallback
+│   │   ├── checkpoint_registry.py     # CheckpointRegistry
+│   │   └── backoff_scheduler.py       # BackoffScheduler
+│   │
+│   ├── adapters/
+│   │   ├── __init__.py
+│   │   └── tensorflow/
+│   │       ├── __init__.py
+│   │       ├── tf_agent.py            # TFAgent(BaseAgent)
+│   │       └── tf_evaluator.py        # TFEvaluator(BaseEvaluator)
+│   │
+│   ├── pipeline/
+│   │   ├── __init__.py
+│   │   ├── batch_pipeline.py          # BatchPipeline — Gymnasium env, static/episodic
+│   │   └── live_pipeline.py           # LivePipeline — real-time streaming source
+│   │
+│   └── optimizers/
+│       ├── __init__.py
+│       ├── backoff_optimizer.py       # BackoffOptimizer (default, perturbation-based)
+│       └── pbt_optimizer.py           # PBTOptimizer (pseudo population-based training)
+│
+└── tests/
+    ├── conftest.py
+    ├── test_core/
+    │   ├── test_types.py
+    │   ├── test_backoff_scheduler.py
+    │   ├── test_checkpoint_registry.py
+    │   └── test_loop_controller.py
+    ├── test_adapters/
+    │   ├── test_tf_agent.py
+    │   └── test_tf_evaluator.py
+    ├── test_pipeline/
+    │   ├── test_batch_pipeline.py
+    │   └── test_live_pipeline.py
+    ├── test_optimizers/
+    │   ├── test_backoff_optimizer.py
+    │   └── test_pbt_optimizer.py
+    └── test_integration/
+        └── test_end_to_end.py
+```
+
+---
+
+## Critical Rules (never violate)
+
+1. **Gymnasium API only.** `env.reset()` → `(obs, info)`. `env.step()` → `(obs, reward, terminated, truncated, info)`. Never use legacy `done` flag internally — merge `terminated | truncated` at the pipeline boundary.
+2. **`BaseAgent` is the only contract.** `LoopController` calls only: `act()`, `learn()`, `get_hyperparams()`, `set_hyperparams()`, `save_weights()`, `load_weights()`.
+3. **`HyperparamSet.params` is an open dict.** Core never reads specific key names. Opaque blob passed between optimizer and agent.
+4. **`EpisodeData` carries raw interaction data only.** No algorithm-specific fields.
+5. **No algorithm-specific code in `core/` or `loop_controller.py`.** PPO, DQN, SAC, etc. are never referenced there.
+6. **`LoopController` is algorithm-blind.** run episode → get score → compare → tune → repeat.
+
+---
+
+## Implementation Tasks
+
+### Phase 1 — Core Foundation
+- [ ] `pyproject.toml`
+- [ ] `tensor_optix/core/types.py`
+- [ ] `tensor_optix/core/base_agent.py`
+- [ ] `tensor_optix/core/base_evaluator.py`
+- [ ] `tensor_optix/core/base_optimizer.py`
+- [ ] `tensor_optix/core/base_pipeline.py`
+- [ ] `tensor_optix/core/backoff_scheduler.py`
+- [ ] `tensor_optix/core/checkpoint_registry.py`
+- [ ] `tensor_optix/core/loop_controller.py`
+
+### Phase 2 — TensorFlow Adapter
+- [ ] `tensor_optix/adapters/tensorflow/tf_agent.py`
+- [ ] `tensor_optix/adapters/tensorflow/tf_evaluator.py`
+
+### Phase 3 — Pipelines
+- [ ] `tensor_optix/pipeline/batch_pipeline.py`
+- [ ] `tensor_optix/pipeline/live_pipeline.py`
+
+### Phase 4 — Optimizers
+- [ ] `tensor_optix/optimizers/backoff_optimizer.py`
+- [ ] `tensor_optix/optimizers/pbt_optimizer.py`
+
+### Phase 5 — Wiring
+- [ ] `tensor_optix/optimizer.py` (RLOptimizer entry point)
+- [ ] `tensor_optix/__init__.py` (public API surface)
+- [ ] All `core/__init__.py`, `adapters/__init__.py`, `pipeline/__init__.py`, `optimizers/__init__.py`
+
+### Phase 6 — Tests
+- [ ] `tests/conftest.py`
+- [ ] `tests/test_core/test_types.py`
+- [ ] `tests/test_core/test_backoff_scheduler.py`
+- [ ] `tests/test_core/test_checkpoint_registry.py`
+- [ ] `tests/test_core/test_loop_controller.py`
+- [ ] `tests/test_adapters/test_tf_agent.py`
+- [ ] `tests/test_adapters/test_tf_evaluator.py`
+- [ ] `tests/test_pipeline/test_batch_pipeline.py`
+- [ ] `tests/test_pipeline/test_live_pipeline.py`
+- [ ] `tests/test_optimizers/test_backoff_optimizer.py`
+- [ ] `tests/test_optimizers/test_pbt_optimizer.py`
+- [ ] `tests/test_integration/test_end_to_end.py`
+
+---
+
+## Known Issues / Decisions Log
+
+| Date | Issue | Decision |
+|------|-------|----------|
+| 2026-03-27 | Blueprint said "framework-agnostic" | Corrected: TensorFlow only |
+| 2026-03-27 | Blueprint used legacy gym API | Corrected: Gymnasium >=1.0.0 |
+| 2026-03-27 | Blueprint hardcoded TF as required dep in a "framework-agnostic" core | N/A — TF-only removes the contradiction |
+| 2026-03-27 | Degradation check `score < best * threshold` breaks for negative scores | Fixed: use `score < best - abs(best) * (1 - threshold)` |
+
+---
+
+## Notes
+
+- `BatchPipeline` wraps a Gymnasium-compatible env for episodic/batch training. Not a static dataset loader.
+- `LivePipeline` wraps a streaming data source (e.g. websocket feed). User provides a `stream()` generator.
+- `TFAgent.learn()` provides a generic gradient update baseline. Users subclass and override for specific algorithms (PPO clipping, SAC entropy tuning, etc.).
+- `PBTOptimizer` approximates population-based training for single-agent use via a virtual population from history.
+
+---
+
+## Optimizer Math — BackoffOptimizer (Running Finite Difference)
+
+### Core Idea
+Estimate the gradient of `primary_score` w.r.t. each hyperparam using finite differences accumulated across episodes. Step in the direction that increases score.
+
+### Per-param gradient estimate
+```
+∂score/∂θᵢ ≈ (score_avg_after - score_avg_before) / Δθᵢ
+```
+Where `score_avg` is a rolling mean over the last N episodes (noise reduction).
+
+### Update rule
+```
+θᵢ_new = clip(θᵢ + α * ∂score/∂θᵢ, low_bound, high_bound)
+```
+
+### Step size α (adaptive)
+```
+α = base_lr / (1 + β * score_variance)
+```
+High variance in recent scores → smaller steps. Low variance → larger steps.
+
+### Perturbation size δ (per param)
+- Multiplicative: `δᵢ = perturbation_scale * |θᵢ|` (scale-invariant)
+- Clamped: `δᵢ = max(δᵢ, min_delta)` to avoid zero delta on small params
+
+### Directional memory
+- Track last direction moved per param (`+1` or `-1`)
+- Track whether that move improved score
+- If improvement: continue in same direction (momentum)
+- If no improvement: reverse direction, halve step size
+
+### Score buffer
+- Rolling window of last `score_window` (default: 5) primary scores
+- Use mean of buffer as the stable score signal for gradient estimation
+- Do not update params until buffer has at least `min_samples` entries
+
+### Bounds enforcement
+- User provides `param_bounds: dict[str, tuple[float, float]]`
+- Params not in bounds are left unchanged
+- All updates clipped to `[low, high]` after step
+
+### Variance-gated updates
+- If `score_variance > high_variance_threshold`: skip update this cycle (too noisy to trust)
+- Log skipped updates for observability
+
+---
+
+## Optimizer Math — PBTOptimizer (Pseudo Population-Based Training)
+
+### Core Idea
+Maintain a history of `(HyperparamSet, primary_score)` pairs as a virtual population. Use exploit/explore logic from PBT without parallel workers.
+
+### Exploit condition
+```
+if current_score < percentile(history_scores, 20):
+    # bottom 20% — exploit top 20%
+    best_params = params from top 20% of history (by score)
+    new_params = perturb(best_params, scale=small)
+```
+
+### Explore condition
+```
+else:
+    # not bottom 20% — explore
+    new_params = perturb(current_params, scale=medium)
+```
+
+### Perturbation function (shared with BackoffOptimizer)
+```
+perturb(θ, scale) → for each param:
+    δ = scale * (high - low)          # fraction of param range
+    new_val = θ + uniform(-δ, +δ)
+    new_val = clip(new_val, low, high)
+```
+
+### History management
+- Keep last `history_size` (default: 50) `(params, score)` pairs
+- FIFO eviction
+- Percentile computed over this window only