carl-studio 0.3.0__tar.gz

carl_studio-0.3.0/.claude/skills/terminals/carl.md

@@ -0,0 +1,132 @@
+---
+name: carl
+description: |
+  This skill should be used when the user asks to "train a model", "run CARL", "check coherence",
+  "submit a training job", "monitor training", "evaluate a checkpoint", "deploy the agent",
+  "carl train", "carl eval", "carl observe", or works with CARL-trained models, coherence metrics,
+  phase transitions, or the zero-rl-pipeline. The CARL training companion for carl-studio.
+version: 0.3.0
+---
+
+# CARL Training Companion
+
+Manage the full lifecycle of training VLMs with coherence-aware reward signals via carl-studio.
+
+## Conservation Law
+
+| Constant | Value | Meaning |
+|----------|-------|---------|
+| kappa | 64/3 | Conservation constant |
+| sigma | 3/16 | Semantic quantum |
+| kappa * sigma | 4 | Bits per embedding dimension |
+| T* | kappa * d | Decompression boundary |
+| Defect threshold | \|delta_Phi\| > 0.03 | Discontinuity detection |
+
+Triadic dimensions (d = 3*2^k) give exact binary T*. Non-triadic incur ~33% context tax.
+
+## Pipeline Architecture
+
+```
+Phase 1' (COMPLETE):  VLM grounding — 94.6% click accuracy, 100% format compliance
+Phase 2' (ACTIVE):    Environment GRPO — tool-calling through real sandbox interaction
+Phase 3  (BUILT):     TTT — SLOT optimizer + LoRA micro-update for inference-time adaptation
+```
+
+### Phase 2' — Environment GRPO (Current)
+
+The model enters a real coding sandbox (`CodingSandboxEnv`) and learns tool-calling through interaction. Four tools: `read_file`, `write_file`, `execute_code`, `run_shell`.
+
+**Reward architecture (4 functions, 3 roles):**
+1. `tool_engagement` (w=1.0) — dense: "did the model try?"
+2. `task_completion` (w=3.0) — sparse binary: "did the code run?"
+3. `gated_carl` (w=1.0) — quality: "was the reasoning clean?" (Stage B only)
+4. `gr3_length_penalty` (w=2.0) — structural: shorter correct solutions score higher
+
+**Cascade gating:** CARL activates when `task_completion` shows sustained capability. Self-calibrating — no hardcoded threshold.
+
+## Training Operations
+
+### Submit a Job
+
+```python
+from huggingface_hub import HfApi
+api = HfApi(token=HF_TOKEN)
+job = api.run_uv_job(
+    script='https://huggingface.co/datasets/wheattoast11/zero-rl-tool-calling-data/resolve/main/train_env_grpo.py',
+    flavor='a100-large',
+    timeout='2d',
+    secrets={'HF_TOKEN': HF_TOKEN},
+)
+```
+
+### Monitor
+
+```python
+j = api.inspect_job(job_id=JOB_ID)         # Status
+logs = list(api.fetch_job_logs(job_id=JOB_ID))  # Logs (generator)
+```
+
+Watch Trackio dashboard for smoothed trends. Never judge from individual log entries.
+
+### Key Health Signals
+
+- **Length phase transition:** Rise-then-drop in mean_terminated_length = model learning. Monotonic growth = death.
+- **Tool call frequency:** Should increase as model learns. 0 → 1-4 by step 30+.
+- **Cascade gate fire:** Stage A → B when task_completion sustains above running distribution median.
+- **Clipped ratio:** High (>50%) means completions hitting max_completion_length. Normal early, should decrease.
+
+## Phase Transition Gate
+
+`PhaseTransitionGate` monitors `mean_token_accuracy` during SFT. Windowed check: 3 of last 5 steps above 0.99 = crystallized. Robust to batch-level noise.
+
+Structural signature: entropy spike (1.0 → 9.3 → 0.12) over ~40 steps. Token accuracy jumps from 3% to 99% during collapse. Consistent with Kuramoto synchronization.
+
+## Coherence Trap
+
+CARL rewards high Phi (coherence). Maximum Phi = delta function = mode collapse. Without cascade gating, CARL dominates sparse task signal and collapses tool-calling at step ~26 (observed in v1). Fix: `CascadeRewardManager` gates CARL behind task performance.
+
+If GRPO produces identical completions (zero within-group variance): temperature >= 2.0 + top_k=50.
+
+## Compute
+
+| Phase | Flavor | VRAM | Price |
+|-------|--------|------|-------|
+| Phase 1' SFT/GRPO | l40sx1 | 48 GB | $1.80/hr |
+| Phase 2' Env GRPO | a100-large | 80 GB | $2.50/hr |
+| Eval | l4x1 | 24 GB | $0.80/hr |
+
+**Required env var:** `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` — set in the training script. Prevents CUDA fragmentation OOM during multi-turn generation.
+
+## CLI Reference
+
+```
+carl train         Train with CARL rewards
+carl train --send-it   Full autonomous pipeline
+carl eval          Run eval gates
+carl observe       Live coherence monitoring
+carl align         Realign a drifted model
+carl learn         Ingest new knowledge
+carl bench         Coherence meta-benchmarks
+carl status <id>   Job status
+carl push          Push checkpoint to Hub
+carl bundle        Generate self-contained script
+carl mcp           Start MCP server
+```
+
+## File Layout
+
+```
+phase1/         # CARL infrastructure (ModelSpec, rewards, tool profiles)
+phase2/         # Phase 2' environment GRPO (ACTIVE)
+phase3/         # TTT infrastructure (SLOT, LoRA, continuous RL)
+scripts/        # CLI tools, submission, eval, diagnostics
+data/           # Training/eval data
+legacy/         # Archived scripts (v1-v7 versions preserved)
+```
+
+## Hub Models
+
+| Model | Phase | Status |
+|-------|-------|--------|
+| wheattoast11/OmniCoder-9B-Zero-Phase2 | Phase 1' | PASS — 94.6% click accuracy |
+| wheattoast11/OmniCoder-9B-Zero-Phase2Prime | Phase 2' | Training (v7) |

carl_studio-0.3.0/.github/workflows/publish.yml

@@ -0,0 +1,18 @@
+name: Publish to PyPI
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install build
+      - run: python -m build
+      - uses: pypa/gh-action-pypi-publish@release/v1

carl_studio-0.3.0/.gitignore

@@ -0,0 +1,16 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+dist/
+build/
+.eggs/
+*.whl
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+*.egg
+.env
+.venv/
+venv/
+env/

carl_studio-0.3.0/AGENTS.md

@@ -0,0 +1,51 @@
+# carl-studio
+
+## Setup
+```bash
+uv pip install -e ".[dev]"
+```
+
+## Test
+```bash
+pytest tests/ -q --tb=short
+```
+
+## Type Check
+```bash
+pyright --strict src/carl_studio/types/
+```
+
+## Code Style
+- Pydantic BaseModel for all configs and data structures
+- Type hints on every function signature
+- Constants are module-level in `primitives/constants.py`, never parameters
+- `from __future__ import annotations` in every file
+- Lazy imports for optional dependencies (torch, anthropic, runpod, etc.)
+
+## Key Files
+- `src/carl_studio/primitives/constants.py` — κ, σ, threshold (NEVER modify)
+- `src/carl_studio/primitives/coherence_probe.py` — THE source of truth for coherence math
+- `src/carl_studio/primitives/math.py` — shared compute_phi() (single source, used by all reward components)
+- `src/carl_studio/primitives/coherence_observer.py` — Claude-in-the-loop diagnostics
+- `src/carl_studio/types/config.py` — all config types (TrainingConfig is the root)
+- `src/carl_studio/training/trainer.py` — CARLTrainer: async dispatch to remote or local
+- `src/carl_studio/training/rewards/composite.py` — CARLReward + make_carl_reward factory
+- `src/carl_studio/training/rewards/vlm.py` — VLM rewards (click_accuracy, coordinate_format, precision)
+- `src/carl_studio/compute/protocol.py` — ComputeBackend interface
+- `src/carl_studio/bundler.py` — self-contained HF Jobs script generator
+- `src/carl_studio/mcp/server.py` — 9 MCP tools for AI agent consumption
+- `paper/carl-paper.md` — formal research paper
+
+## Boundaries
+- Never modify constants (KAPPA, SIGMA, DEFECT_THRESHOLD)
+- Bundled scripts (from bundler.py) must be self-contained — no carl_studio imports
+- Backend implementations use lazy imports — don't require all SDKs installed
+- `import carl_studio` must work with only pydantic + numpy + typer + anyio + pyyaml
+- Crystal/coherence math uses numpy, never torch (convert at boundary)
+- Qwen3.5 thinking mode must be disabled for short-output tasks (coordinates, etc.)
+- GRPO num_generations=8 minimum — 4 causes zero-advantage zero-gradient
+
+## Naming
+- Public API: "coherence" (CoherenceProbe, CoherenceSnapshot, discontinuity)
+- Internal math: physics names preserved (Φ, κ, σ, entropy, defect)
+- The mapping: Crystal→Coherence, Defect→Discontinuity, Crystallization→Commitment, Melting→Dissolution

carl_studio-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Intuition Labs LLC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

carl_studio-0.3.0/PKG-INFO

@@ -0,0 +1,291 @@
+Metadata-Version: 2.4
+Name: carl-studio
+Version: 0.3.0
+Summary: CARL (Coherence-Aware Reinforcement Learning) — information-theoretic reward signals for LLM training via token-level probability distributions
+Project-URL: Homepage, https://github.com/wheattoast11/carl
+Project-URL: Documentation, https://github.com/wheattoast11/carl#readme
+Project-URL: Repository, https://github.com/wheattoast11/carl
+Project-URL: Issues, https://github.com/wheattoast11/carl/issues
+Author-email: Tej Desai <tej@terminals.tech>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: coherence,grpo,information-theory,llm,reinforcement-learning,reward-shaping,rl,training
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: GPU :: NVIDIA CUDA
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: anyio>=4.0
+Requires-Dist: numpy
+Requires-Dist: pydantic-settings>=2.0
+Requires-Dist: pydantic>=2.9
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=14.0
+Requires-Dist: typer>=0.12
+Provides-Extra: all
+Requires-Dist: anthropic>=0.40; extra == 'all'
+Requires-Dist: bitsandbytes; extra == 'all'
+Requires-Dist: datasets>=3.0; extra == 'all'
+Requires-Dist: huggingface-hub>=0.25; extra == 'all'
+Requires-Dist: mcp>=1.0; extra == 'all'
+Requires-Dist: peft>=0.13; extra == 'all'
+Requires-Dist: pyright; extra == 'all'
+Requires-Dist: pytest; extra == 'all'
+Requires-Dist: ruff; extra == 'all'
+Requires-Dist: runpod>=1.7; extra == 'all'
+Requires-Dist: textual-plotext>=0.3; extra == 'all'
+Requires-Dist: textual>=1.0; extra == 'all'
+Requires-Dist: tinker>=0.16; extra == 'all'
+Requires-Dist: torch>=2.4; extra == 'all'
+Requires-Dist: trackio; extra == 'all'
+Requires-Dist: transformers>=5.0; extra == 'all'
+Requires-Dist: trl>=0.12; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: pyright; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Provides-Extra: hf
+Requires-Dist: huggingface-hub>=0.25; extra == 'hf'
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.0; extra == 'mcp'
+Provides-Extra: observe
+Requires-Dist: anthropic>=0.40; extra == 'observe'
+Provides-Extra: runpod
+Requires-Dist: runpod>=1.7; extra == 'runpod'
+Provides-Extra: tinker
+Requires-Dist: tinker>=0.16; extra == 'tinker'
+Provides-Extra: training
+Requires-Dist: bitsandbytes; extra == 'training'
+Requires-Dist: datasets>=3.0; extra == 'training'
+Requires-Dist: peft>=0.13; extra == 'training'
+Requires-Dist: torch>=2.4; extra == 'training'
+Requires-Dist: trackio; extra == 'training'
+Requires-Dist: transformers>=5.0; extra == 'training'
+Requires-Dist: trl>=0.12; extra == 'training'
+Provides-Extra: tui
+Requires-Dist: textual-plotext>=0.3; extra == 'tui'
+Requires-Dist: textual>=1.0; extra == 'tui'
+Description-Content-Type: text/markdown
+
+# CARL Studio
+
+**Coherence-Aware Reinforcement Learning**
+
+> Models don't learn gradually -- they crystallize.
+
+![CARL Phase Transition](assets/carl-paradigm.gif)
+
+CARL adds information-theoretic reward signals to RL training that measure *how* a model generates, not just *what* it generates. One conservation law. Three reward components. Model-agnostic.
+
+## Install
+
+```bash
+pip install carl-studio
+```
+
+## Quick Start
+
+### Observe (zero friction -- point at any existing run)
+
+```bash
+carl observe --trackio https://your-space.hf.space
+```
+
+No training, no config, no GPU. See your model's learning geometry instantly.
+
+### Train
+
+```python
+from carl_studio import CARLTrainer, TrainingConfig
+
+trainer = CARLTrainer(TrainingConfig(
+    run_name="my-first-carl",
+    base_model="Qwen/Qwen3.5-9B",
+    output_repo="your-username/my-model",
+    method="grpo",
+    dataset_repo="trl-lib/Capybara",
+    compute_target="l4x1",
+))
+run = await trainer.train()
+```
+
+Or from the CLI:
+
+```bash
+carl train --model Qwen/Qwen3.5-9B --method grpo --compute l4x1
+carl train --send-it                  # full pipeline: SFT -> gate -> GRPO -> eval -> push
+```
+
+## Architecture
+
+```
+Layer 4  MCP Server       9 tools for AI agent consumption
+         ──────────────────────────────────────────────────
+Layer 3  CLI              carl train | eval | observe | align | learn | bench | mcp
+         ──────────────────────────────────────────────────
+Layer 2  Training         CARLTrainer, CascadeRewardManager, environments
+         ──────────────────────────────────────────────────
+Layer 1  SDK              ModelSpec, TrainSpec, VRAMBudget, CoherenceProbe
+         ──────────────────────────────────────────────────
+Layer 0  Primitives       compute_phi, kappa, sigma, PhaseTransitionGate
+```
+
+### The Reward
+
+```
+R_CARL = 0.50 * R_coherence + 0.30 * R_cloud + 0.20 * R_discontinuity
+```
+
+| Component | Measures |
+|-----------|----------|
+| Multiscale coherence | Phi consistency across dyadic block scales |
+| Cloud quality | P(selected) * Phi -- confident AND correct |
+| Discontinuity targeting | Sharp Phi transitions at structurally appropriate locations |
+
+### Cascade Gating
+
+CARL is length-biased -- a verbose, confident model scores high. Without gating, it dominates sparse task signal and causes mode collapse. The cascade solves this:
+
+```
+Stage A (early):   task rewards only         -- "learn to use tools"
+Stage B (gated):   task + CARL rewards       -- "now do it coherently"
+```
+
+The gate self-calibrates from the task metric's running distribution. No hardcoded threshold.
+
+### Order Parameter
+
+```
+Phi = 1 - H(P) / log|V|
+```
+
+0 = uniform (maximum uncertainty). 1 = delta (complete certainty).
+
+## The Conservation Law
+
+| Constant | Value | Meaning |
+|----------|-------|---------|
+| kappa | 64/3 | Conservation constant |
+| sigma | 3/16 | Semantic quantum |
+| kappa * sigma | **4** | Bits per embedding dimension |
+| T* | kappa * d | Decompression boundary |
+
+Derived in [Bounded Informational Time Crystals](https://doi.org/10.5281/zenodo.18906944). Validated across 6,244 trials in [Material Reality](https://doi.org/10.5281/zenodo.18992029). Formally proved in [Semantic Realizability](https://doi.org/10.5281/zenodo.18992031).
+
+## Key Finding: Phase Transitions
+
+During VLM SFT, the model exhibits a first-order phase transition:
+
+| Steps | Phase | Accuracy | Entropy | What happens |
+|-------|-------|----------|---------|--------------|
+| 0-10 | Baseline | 3% | 1.0 | Pre-training distribution intact |
+| 10-20 | Melting | 8% | **9.3** | Distribution destabilizes completely |
+| 20-25 | **Transition** | **65%** | 4.1 | Accuracy jumps 57 points in 5 steps |
+| 25-35 | Crystallization | 99% | 0.4 | Rapid convergence |
+| 35-46 | Converged | **99.3%** | 0.12 | Fully crystallized |
+
+Entropy spikes to near-maximum, then accuracy discontinuously jumps once the system passes the critical coupling threshold. Consistent with Kuramoto synchronization in coupled oscillator systems.
+
+## CLI
+
+**Core triad:**
+```
+carl observe       See learning geometry on any run (no GPU required)
+carl eval          Pass/fail gate on a checkpoint
+carl train         Train with CARL rewards (SFT, GRPO, DPO, KTO, ORPO)
+carl train --send-it   Full autonomous pipeline: SFT -> gate -> GRPO -> eval -> push
+```
+
+**Operations:**
+```
+carl status <id>   Job status
+carl logs <id>     Job logs
+carl stop <id>     Cancel a job
+carl push          Push checkpoint to Hub
+carl bundle        Generate self-contained training script
+carl compute       List GPU flavors and pricing
+carl setup         First-time setup
+```
+
+**Advanced** (experimental):
+```
+carl align         Realign a drifted model
+carl learn         Ingest knowledge, generate data, train
+carl bench         Coherence meta-benchmarks
+carl mcp           Start MCP server (9 tools for AI agents)
+carl dev           Development utilities
+```
+
+## Model-Agnostic
+
+`ModelSpec.from_pretrained()` auto-detects architecture, modality, thinking mode, quantization constraints, and LoRA targets from any HuggingFace config.json. No per-model branches.
+
+| Model | Status |
+|-------|--------|
+| Qwen 3.5 9B VLM | Primary -- 94.6% click accuracy |
+| Gemma 4 E4B | Planned |
+| Gemma 4 31B | Planned (multi-GPU) |
+
+## Compute Backends
+
+| Backend | Flag |
+|---------|------|
+| HuggingFace Jobs | `--compute l4x1` / `a100-large` / `h200` |
+| RunPod | `--compute runpod` |
+| Tinker | `--compute tinker` |
+| Prime Intellect | `--compute prime` |
+| SSH | `--compute ssh` |
+| Local | `--compute local` |
+
+## Test-Time Training
+
+CARL includes TTT mechanisms for post-deployment adaptation:
+
+- **SLOT** -- hidden delta injection (8 Adam steps, architecture-agnostic)
+- **LoRA micro-update** -- rank-1 online adaptation
+
+## IP Boundaries
+
+CARL Studio is MIT-licensed. The mathematics -- conservation law, order parameter, reward components -- are independently derivable from the three published papers (CC-BY-4.0).
+
+This package is the *open training framework*. It does **not** include the runtime dynamics or autonomous orchestration:
+
+| What | Where | License |
+|------|-------|---------|
+| Conservation law, Phi, rewards | **CARL Studio** (this repo) | MIT |
+| Observe, eval, train CLI | **CARL Studio** (this repo) | MIT |
+| Resonance LR modulation | **terminals-runtime** | BUSL-1.1 |
+| SLOT / LoRA micro-update (TTT) | **terminals-runtime** | BUSL-1.1 |
+| Kuramoto oscillator dynamics | **terminals-runtime** | BUSL-1.1 |
+| Coherence diagnosis methodology | **terminals-runtime** | BUSL-1.1 |
+| Audio coherence (CHORD) | Terminals Platform | BUSL-1.1 |
+| Cross-substrate isomorphisms | Terminals Platform | BUSL-1.1 |
+| Interactive Research Environment | Terminals Platform | BUSL-1.1 |
+| Material Reality datasets | Zenodo | CC-BY-4.0 |
+
+The bifurcation is deliberate: CARL Studio provides the full training loop (observe, eval, train) using published mathematics. The `terminals-runtime` package adds autonomous features (resonance-aware LR, test-time training, Claude-powered diagnosis) behind BUSL-1.1. Same conservation law, different sides of the boundary.
+
+## Citation
+
+```bibtex
+@article{desai2026carl,
+  title   = {Coherence-Aware Reinforcement Learning},
+  author  = {Desai, Tej},
+  year    = {2026},
+  url     = {https://github.com/wheattoast11/carl},
+  note    = {Intuition Labs LLC}
+}
+```
+
+## License
+
+MIT -- Intuition Labs LLC