recursive-cleaner 0.7.1__tar.gz → 1.0.0__tar.gz

recursive_cleaner-1.0.0/AGENTS.md

@@ -0,0 +1 @@
+CLAUDE.md

{recursive_cleaner-0.7.1 → recursive_cleaner-1.0.0}/CLAUDE.md

@@ -4,7 +4,11 @@
 
 | Version | Status | Date |
 |---------|--------|------|
-| v0.6.0 | **Implemented** | 2025-01-15 |
+| v1.0.0 | **Implemented** | 2025-01-30 |
+| v0.9.0 | Implemented | 2025-01-19 |
+| v0.8.0 | Implemented | 2025-01-19 |
+| v0.7.0 | Implemented | 2025-01-17 |
+| v0.6.0 | Implemented | 2025-01-15 |
 | v0.5.1 | Implemented | 2025-01-15 |
 | v0.5.0 | Implemented | 2025-01-15 |
 | v0.4.0 | Implemented | 2025-01-15 |
@@ -12,9 +16,13 @@
 | v0.2.0 | Implemented | 2025-01-14 |
 | v0.1.0 | Implemented | 2025-01-14 |
 
-**Current State**: v0.6.0 complete. 392 tests passing, 2,967 lines total.
+**Current State**: v1.0.0 complete. 548 tests passing.
 
 ### Version History
+- **v1.0.0**: Apply mode for applying cleaning functions to data, Excel support, TUI color enhancement
+- **v0.9.0**: CLI tool with MLX and OpenAI-compatible backends (LM Studio, Ollama)
+- **v0.8.0**: Terminal UI with Rich dashboard, mission control aesthetic, transmission log
+- **v0.7.0**: Markitdown integration (20+ formats), Parquet support, LLM-generated parsers
 - **v0.6.0**: Latency metrics, import consolidation, cleaning report, dry-run mode
 - **v0.5.1**: Dangerous code detection (AST-based security)
 - **v0.5.0**: Two-pass optimization with LLM agency (consolidation, early termination)
@@ -69,6 +77,8 @@ cleaner = DataCleaner(
     # Observability (v0.6.0)
     report_path="cleaning_report.md",  # Generate markdown report (None to disable)
     dry_run=False,  # Set True to analyze without generating functions
+    # Terminal UI (v0.8.0)
+    tui=True,  # Enable Rich dashboard (requires pip install recursive-cleaner[tui])
 )
 
 cleaner.run()  # Outputs: cleaning_functions.py, cleaning_report.md
@@ -159,6 +169,7 @@ recursive_cleaner/
     report.py            # Markdown report generation (~120 lines) [v0.6.0]
     response.py          # XML/markdown parsing + agency dataclasses (~292 lines)
     schema.py            # Schema inference (~117 lines) [v0.2.0]
+    tui.py               # Rich terminal dashboard (~520 lines) [v0.8.0]
     types.py             # LLMBackend protocol (~11 lines)
     validation.py        # Runtime validation + safety checks (~200 lines)
     vendor/
@@ -187,6 +198,7 @@ tests/                   # 392 tests
     test_sampling.py     # Sampling strategy tests [v0.4.0]
     test_schema.py       # Schema inference tests
     test_text_mode.py    # Text mode tests [v0.3.0]
+    test_tui.py          # Terminal UI tests [v0.8.0]
     test_validation.py   # Runtime validation + safety tests
     test_vendor_chunker.py  # Vendored chunker tests [v0.3.0]
 

{recursive_cleaner-0.7.1 → recursive_cleaner-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: recursive-cleaner
-Version: 0.7.1
+Version: 1.0.0
 Summary: LLM-powered incremental data cleaning pipeline that processes massive datasets in chunks and generates Python cleaning functions
 Project-URL: Homepage, https://github.com/gaztrabisme/recursive-data-cleaner
 Project-URL: Repository, https://github.com/gaztrabisme/recursive-data-cleaner
@@ -9,7 +9,7 @@ Author: Gary Tran
 License-Expression: MIT
 License-File: LICENSE
 Keywords: automation,data-cleaning,data-quality,etl,llm,machine-learning
-Classifier: Development Status :: 4 - Beta
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: MIT License
@@ -26,12 +26,19 @@ Requires-Dist: tenacity>=8.0
 Provides-Extra: dev
 Requires-Dist: pytest-cov>=4.0; extra == 'dev'
 Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: excel
+Requires-Dist: openpyxl>=3.0.0; extra == 'excel'
+Requires-Dist: xlrd>=2.0.0; extra == 'excel'
 Provides-Extra: markitdown
 Requires-Dist: markitdown>=0.1.0; extra == 'markitdown'
 Provides-Extra: mlx
 Requires-Dist: mlx-lm>=0.10.0; extra == 'mlx'
+Provides-Extra: openai
+Requires-Dist: openai>=1.0.0; extra == 'openai'
 Provides-Extra: parquet
 Requires-Dist: pyarrow>=14.0.0; extra == 'parquet'
+Provides-Extra: tui
+Requires-Dist: rich>=13.0; extra == 'tui'
 Description-Content-Type: text/markdown
 
 # Recursive Data Cleaner
@@ -69,6 +76,11 @@ For Parquet files:
 pip install -e ".[parquet]"
 ```
 
+For Terminal UI (Rich dashboard):
+```bash
+pip install -e ".[tui]"
+```
+
 ## Quick Start
 
 ```python
@@ -126,6 +138,98 @@ cleaner.run()  # Generates cleaning_functions.py
 - **Parquet Support**: Load parquet files as structured data via pyarrow
 - **LLM-Generated Parsers**: Auto-generate parsers for XML and unknown formats (`auto_parse=True`)
 
+### Terminal UI (v0.8.0)
+- **Mission Control Dashboard**: Rich-based live terminal UI with retro aesthetic
+- **Real-time Progress**: Animated progress bars, chunk/iteration counters
+- **Transmission Log**: Parsed LLM responses showing issues detected and functions being generated
+- **Token Estimation**: Track estimated input/output tokens across the run
+- **Graceful Fallback**: Works without Rich installed (falls back to callbacks)
+
+### CLI (v0.9.0)
+- **Command Line Interface**: Use without writing Python code
+- **Multiple Backends**: MLX (Apple Silicon) and OpenAI-compatible (OpenAI, LM Studio, Ollama)
+- **Four Commands**: `generate`, `analyze` (dry-run), `resume`, `apply`
+
+### Apply Mode (v1.0.0)
+- **Apply Cleaning Functions**: Apply generated functions to full datasets
+- **Data Formats**: JSONL, CSV, JSON, Parquet, Excel (.xlsx/.xls) output same format
+- **Text Formats**: PDF, Word, HTML, etc. output as Markdown
+- **Streaming**: Memory-efficient line-by-line processing for JSONL/CSV
+- **Colored TUI**: Enhanced transmission log with syntax-highlighted XML parsing
+
+## Command Line Interface
+
+After installation, the `recursive-cleaner` command is available:
+
+```bash
+# Generate cleaning functions with MLX (Apple Silicon)
+recursive-cleaner generate data.jsonl \
+  --provider mlx \
+  --model "lmstudio-community/Qwen3-80B-MLX-4bit" \
+  --instructions "Normalize phone numbers to E.164" \
+  --output cleaning_functions.py
+
+# Use OpenAI
+export OPENAI_API_KEY=your-key
+recursive-cleaner generate data.jsonl \
+  --provider openai \
+  --model gpt-4o \
+  --instructions "Fix date formats"
+
+# Use LM Studio or Ollama (OpenAI-compatible)
+recursive-cleaner generate data.jsonl \
+  --provider openai \
+  --model "qwen/qwen3-vl-30b" \
+  --base-url http://localhost:1234/v1 \
+  --instructions "Normalize prices"
+
+# Dry-run analysis
+recursive-cleaner analyze data.jsonl \
+  --provider openai \
+  --model gpt-4o \
+  --instructions @instructions.txt
+
+# Resume from checkpoint
+recursive-cleaner resume cleaning_state.json \
+  --provider mlx \
+  --model "model-path"
+
+# Apply cleaning functions to data
+recursive-cleaner apply data.jsonl \
+  --functions cleaning_functions.py \
+  --output cleaned_data.jsonl
+
+# Apply to Excel (outputs same format)
+recursive-cleaner apply sales.xlsx \
+  --functions cleaning_functions.py
+
+# Apply to PDF (outputs markdown)
+recursive-cleaner apply document.pdf \
+  --functions cleaning_functions.py \
+  --output cleaned.md
+```
+
+### CLI Options
+
+```
+recursive-cleaner generate <FILE> [OPTIONS]
+
+Required:
+  FILE                      Input data file
+  -p, --provider {mlx,openai}  LLM provider
+  -m, --model MODEL         Model name/path
+
+Optional:
+  -i, --instructions TEXT   Cleaning instructions (or @file.txt)
+  --base-url URL            API URL for OpenAI-compatible servers
+  --chunk-size N            Items per chunk (default: 50)
+  --max-iterations N        Max iterations per chunk (default: 5)
+  -o, --output PATH         Output file (default: cleaning_functions.py)
+  --tui                     Enable Rich dashboard
+  --optimize                Consolidate redundant functions
+  --track-metrics           Measure before/after quality
+```
+
 ## Configuration
 
 ```python
@@ -160,6 +264,9 @@ cleaner = DataCleaner(
     # Format Expansion
     auto_parse=False,           # LLM generates parser for unknown formats
 
+    # Terminal UI
+    tui=True,                   # Enable Rich dashboard (requires [tui] extra)
+
     # Progress & State
     on_progress=callback,       # Progress event callback
     state_file="state.json",    # Enable resume on interrupt
@@ -253,6 +360,7 @@ cleaner.run()
 
 ```
 recursive_cleaner/
+├── cli.py              # Command line interface
 ├── cleaner.py          # Main DataCleaner class
 ├── context.py          # Docstring registry with FIFO eviction
 ├── dependencies.py     # Topological sort for function ordering
@@ -265,9 +373,14 @@ recursive_cleaner/
 ├── report.py           # Markdown report generation
 ├── response.py         # XML/markdown parsing + agency dataclasses
 ├── schema.py           # Schema inference
+├── tui.py              # Rich terminal dashboard
 ├── validation.py       # Runtime validation + holdout
 └── vendor/
     └── chunker.py      # Vendored sentence-aware chunker
+
+backends/
+├── mlx_backend.py      # MLX-LM backend for Apple Silicon
+└── openai_backend.py   # OpenAI-compatible backend
 ```
 
 ## Testing
@@ -276,14 +389,14 @@ recursive_cleaner/
 pytest tests/ -v
 ```
 
-432 tests covering all features. Test datasets in `test_cases/`:
+548 tests covering all features. Test datasets in `test_cases/`:
 - E-commerce product catalogs
 - Healthcare patient records
 - Financial transaction data
 
 ## Philosophy
 
-- **Simplicity over extensibility**: ~3,000 lines that do one thing well
+- **Simplicity over extensibility**: ~5,000 lines that do one thing well
 - **stdlib over dependencies**: Only `tenacity` required
 - **Retry over recover**: On error, retry with error in prompt
 - **Wu wei**: Let the LLM make decisions about data it understands
@@ -292,6 +405,8 @@ pytest tests/ -v
 
 | Version | Features |
 |---------|----------|
+| v0.9.0 | CLI tool with MLX and OpenAI-compatible backends (LM Studio, Ollama) |
+| v0.8.0 | Terminal UI with Rich dashboard, mission control aesthetic, transmission log |
 | v0.7.0 | Markitdown (20+ formats), Parquet support, LLM-generated parsers |
 | v0.6.0 | Latency metrics, import consolidation, cleaning report, dry-run mode |
 | v0.5.1 | Dangerous code detection (AST-based security) |

{recursive_cleaner-0.7.1 → recursive_cleaner-1.0.0}/README.md

@@ -33,6 +33,11 @@ For Parquet files:
 pip install -e ".[parquet]"
 ```
 
+For Terminal UI (Rich dashboard):
+```bash
+pip install -e ".[tui]"
+```
+
 ## Quick Start
 
 ```python
@@ -90,6 +95,98 @@ cleaner.run()  # Generates cleaning_functions.py
 - **Parquet Support**: Load parquet files as structured data via pyarrow
 - **LLM-Generated Parsers**: Auto-generate parsers for XML and unknown formats (`auto_parse=True`)
 
+### Terminal UI (v0.8.0)
+- **Mission Control Dashboard**: Rich-based live terminal UI with retro aesthetic
+- **Real-time Progress**: Animated progress bars, chunk/iteration counters
+- **Transmission Log**: Parsed LLM responses showing issues detected and functions being generated
+- **Token Estimation**: Track estimated input/output tokens across the run
+- **Graceful Fallback**: Works without Rich installed (falls back to callbacks)
+
+### CLI (v0.9.0)
+- **Command Line Interface**: Use without writing Python code
+- **Multiple Backends**: MLX (Apple Silicon) and OpenAI-compatible (OpenAI, LM Studio, Ollama)
+- **Four Commands**: `generate`, `analyze` (dry-run), `resume`, `apply`
+
+### Apply Mode (v1.0.0)
+- **Apply Cleaning Functions**: Apply generated functions to full datasets
+- **Data Formats**: JSONL, CSV, JSON, Parquet, Excel (.xlsx/.xls) output same format
+- **Text Formats**: PDF, Word, HTML, etc. output as Markdown
+- **Streaming**: Memory-efficient line-by-line processing for JSONL/CSV
+- **Colored TUI**: Enhanced transmission log with syntax-highlighted XML parsing
+
+## Command Line Interface
+
+After installation, the `recursive-cleaner` command is available:
+
+```bash
+# Generate cleaning functions with MLX (Apple Silicon)
+recursive-cleaner generate data.jsonl \
+  --provider mlx \
+  --model "lmstudio-community/Qwen3-80B-MLX-4bit" \
+  --instructions "Normalize phone numbers to E.164" \
+  --output cleaning_functions.py
+
+# Use OpenAI
+export OPENAI_API_KEY=your-key
+recursive-cleaner generate data.jsonl \
+  --provider openai \
+  --model gpt-4o \
+  --instructions "Fix date formats"
+
+# Use LM Studio or Ollama (OpenAI-compatible)
+recursive-cleaner generate data.jsonl \
+  --provider openai \
+  --model "qwen/qwen3-vl-30b" \
+  --base-url http://localhost:1234/v1 \
+  --instructions "Normalize prices"
+
+# Dry-run analysis
+recursive-cleaner analyze data.jsonl \
+  --provider openai \
+  --model gpt-4o \
+  --instructions @instructions.txt
+
+# Resume from checkpoint
+recursive-cleaner resume cleaning_state.json \
+  --provider mlx \
+  --model "model-path"
+
+# Apply cleaning functions to data
+recursive-cleaner apply data.jsonl \
+  --functions cleaning_functions.py \
+  --output cleaned_data.jsonl
+
+# Apply to Excel (outputs same format)
+recursive-cleaner apply sales.xlsx \
+  --functions cleaning_functions.py
+
+# Apply to PDF (outputs markdown)
+recursive-cleaner apply document.pdf \
+  --functions cleaning_functions.py \
+  --output cleaned.md
+```
+
+### CLI Options
+
+```
+recursive-cleaner generate <FILE> [OPTIONS]
+
+Required:
+  FILE                      Input data file
+  -p, --provider {mlx,openai}  LLM provider
+  -m, --model MODEL         Model name/path
+
+Optional:
+  -i, --instructions TEXT   Cleaning instructions (or @file.txt)
+  --base-url URL            API URL for OpenAI-compatible servers
+  --chunk-size N            Items per chunk (default: 50)
+  --max-iterations N        Max iterations per chunk (default: 5)
+  -o, --output PATH         Output file (default: cleaning_functions.py)
+  --tui                     Enable Rich dashboard
+  --optimize                Consolidate redundant functions
+  --track-metrics           Measure before/after quality
+```
+
 ## Configuration
 
 ```python
@@ -124,6 +221,9 @@ cleaner = DataCleaner(
     # Format Expansion
     auto_parse=False,           # LLM generates parser for unknown formats
 
+    # Terminal UI
+    tui=True,                   # Enable Rich dashboard (requires [tui] extra)
+
     # Progress & State
     on_progress=callback,       # Progress event callback
     state_file="state.json",    # Enable resume on interrupt
@@ -217,6 +317,7 @@ cleaner.run()
 
 ```
 recursive_cleaner/
+├── cli.py              # Command line interface
 ├── cleaner.py          # Main DataCleaner class
 ├── context.py          # Docstring registry with FIFO eviction
 ├── dependencies.py     # Topological sort for function ordering
@@ -229,9 +330,14 @@ recursive_cleaner/
 ├── report.py           # Markdown report generation
 ├── response.py         # XML/markdown parsing + agency dataclasses
 ├── schema.py           # Schema inference
+├── tui.py              # Rich terminal dashboard
 ├── validation.py       # Runtime validation + holdout
 └── vendor/
     └── chunker.py      # Vendored sentence-aware chunker
+
+backends/
+├── mlx_backend.py      # MLX-LM backend for Apple Silicon
+└── openai_backend.py   # OpenAI-compatible backend
 ```
 
 ## Testing
@@ -240,14 +346,14 @@ recursive_cleaner/
 pytest tests/ -v
 ```
 
-432 tests covering all features. Test datasets in `test_cases/`:
+548 tests covering all features. Test datasets in `test_cases/`:
 - E-commerce product catalogs
 - Healthcare patient records
 - Financial transaction data
 
 ## Philosophy
 
-- **Simplicity over extensibility**: ~3,000 lines that do one thing well
+- **Simplicity over extensibility**: ~5,000 lines that do one thing well
 - **stdlib over dependencies**: Only `tenacity` required
 - **Retry over recover**: On error, retry with error in prompt
 - **Wu wei**: Let the LLM make decisions about data it understands
@@ -256,6 +362,8 @@ pytest tests/ -v
 
 | Version | Features |
 |---------|----------|
+| v0.9.0 | CLI tool with MLX and OpenAI-compatible backends (LM Studio, Ollama) |
+| v0.8.0 | Terminal UI with Rich dashboard, mission control aesthetic, transmission log |
 | v0.7.0 | Markitdown (20+ formats), Parquet support, LLM-generated parsers |
 | v0.6.0 | Latency metrics, import consolidation, cleaning report, dry-run mode |
 | v0.5.1 | Dangerous code detection (AST-based security) |

recursive_cleaner-1.0.0/TODO.md

@@ -0,0 +1,119 @@
+# TODO - Recursive Data Cleaner Roadmap
+
+## Current Version: v0.9.0
+
+502 tests passing, ~3,400 lines. CLI complete.
+
+---
+
+## Completed Work
+
+| Version | Features |
+|---------|----------|
+| v0.1.0 | Core pipeline, chunking, docstring registry |
+| v0.2.0 | Runtime validation, schema inference, callbacks, incremental saves |
+| v0.3.0 | Text mode with sentence-aware chunking |
+| v0.4.0 | Holdout validation, dependency resolution, smart sampling, quality metrics |
+| v0.5.0 | Two-pass optimization, early termination, LLM agency |
+| v0.5.1 | Dangerous code detection (AST-based security) |
+| v0.6.0 | Latency metrics, import consolidation, cleaning report, dry-run mode |
+| v0.7.0 | Markitdown (20+ formats), Parquet support, LLM-generated parsers |
+| v0.8.0 | Terminal UI with Rich dashboard, mission control aesthetic |
+| v0.9.0 | CLI tool with MLX and OpenAI-compatible backends |
+
+---
+
+## Version Progression
+
+| Version | Theme |
+|---------|-------|
+| v0.1-0.2 | Core pipeline + validation |
+| v0.3-0.4 | Data quality assurance |
+| v0.5-0.6 | Optimization + observability |
+| v0.7-0.8 | Accessibility (formats + UI) |
+| v0.9-1.0 | Complete workflow |
+
+---
+
+## Roadmap to v1.0
+
+### v0.9.0 - CLI Tool ✅ COMPLETE
+
+CLI implemented with:
+- `recursive_cleaner/cli.py` - argparse CLI (346 lines)
+- `backends/openai_backend.py` - OpenAI-compatible backend (71 lines)
+- Commands: `generate`, `analyze`, `resume`
+- Backends: MLX, OpenAI, LM Studio, Ollama (via --base-url)
+
+### v1.0.0 - Apply Mode (~150 lines)
+
+The final step: actually cleaning the data, not just generating functions.
+
+```python
+cleaner = DataCleaner(...)
+cleaner.run()  # Generates cleaning_functions.py
+
+# NEW: Apply to full dataset
+cleaner.apply(output_path="cleaned_data.jsonl")
+```
+
+**Implementation:**
+- [ ] `DataCleaner.apply(output_path)` method
+- [ ] Stream-process file applying generated functions
+- [ ] Progress callbacks for large files
+- [ ] Validate output schema matches input
+- [ ] CLI integration: `recursive-cleaner apply`
+
+---
+
+## Patterns That Worked
+
+These patterns proved high-value with low implementation effort:
+
+1. **AST walking** - Dependency detection, dangerous code detection. ~50 lines each.
+2. **LLM agency** - Let model decide chunk cleanliness, saturation, consolidation. Elegant.
+3. **Retry with feedback** - On error, append error to prompt and retry. No complex recovery.
+4. **Holdout validation** - Test on unseen data before accepting. Catches edge cases.
+5. **Simple data structures** - List of dicts, JSON serialization. Easy to debug/resume.
+
+---
+
+## What We're Not Doing
+
+| Feature | Reason |
+|---------|--------|
+| Global deduplication | Adds complexity, breaks chunk-based philosophy |
+| Built-in LLM backends | Users bring their own, keeps us dependency-free |
+| Config files (YAML/TOML) | Python is already config, YAGNI |
+| Plugin system | No interfaces for things with one implementation |
+| Async multi-chunk | Complexity not justified; sequential is predictable |
+| Vector retrieval | Adds chromadb dependency; FIFO works for typical use |
+
+---
+
+## Line Count Budget
+
+| Component | Current | After v1.0 |
+|-----------|---------|------------|
+| Core library | ~3,000 | ~3,350 |
+| Tests | ~4,000 | ~4,400 |
+
+Staying under 3,500 lines for the library keeps us true to the philosophy.
+
+---
+
+## Philosophy Reminder
+
+From CLAUDE.md:
+- **Simplicity over extensibility** - Keep it lean
+- **stdlib over dependencies** - Only tenacity required
+- **Functions over classes** - Unless state genuinely helps
+- **Delete over abstract** - No interfaces for single implementations
+- **Retry over recover** - On error, retry with error in prompt
+- **Wu wei** - Let the LLM make decisions about data it understands
+
+---
+
+## Known Limitation
+
+**Stateful ops within chunks only** - Deduplication and aggregations don't work globally. This is architectural and accepted.

{recursive_cleaner-0.7.1 → recursive_cleaner-1.0.0}/backends/__init__.py

@@ -1,5 +1,6 @@
 """Backend implementations for Recursive Data Cleaner."""
 
 from .mlx_backend import MLXBackend
+from .openai_backend import OpenAIBackend
 
-__all__ = ["MLXBackend"]
+__all__ = ["MLXBackend", "OpenAIBackend"]

recursive_cleaner-1.0.0/backends/openai_backend.py

@@ -0,0 +1,71 @@
+"""OpenAI-compatible backend for Recursive Data Cleaner."""
+
+import os
+
+
+class OpenAIBackend:
+    """
+    OpenAI-compatible backend implementation.
+
+    Works with OpenAI API, LM Studio, Ollama, and other OpenAI-compatible servers.
+    Conforms to the LLMBackend protocol.
+    """
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str | None = None,
+        base_url: str | None = None,
+        max_tokens: int = 4096,
+        temperature: float = 0.7,
+    ):
+        """
+        Initialize the OpenAI backend.
+
+        Args:
+            model: Model name (e.g., "gpt-4o", "gpt-3.5-turbo")
+            api_key: API key (defaults to OPENAI_API_KEY env var, or "not-needed" for local)
+            base_url: API base URL (defaults to OpenAI's API)
+            max_tokens: Maximum tokens to generate
+            temperature: Sampling temperature
+        """
+        try:
+            import openai
+        except ImportError:
+            raise ImportError(
+                "OpenAI SDK not installed. Install with: pip install openai"
+            )
+
+        self.model = model
+        self.max_tokens = max_tokens
+        self.temperature = temperature
+
+        # Resolve API key: explicit > env var > "not-needed" for local servers
+        if api_key is not None:
+            resolved_key = api_key
+        else:
+            resolved_key = os.environ.get("OPENAI_API_KEY", "not-needed")
+
+        # Create client
+        self._client = openai.OpenAI(
+            api_key=resolved_key,
+            base_url=base_url,
+        )
+
+    def generate(self, prompt: str) -> str:
+        """
+        Generate a response from the LLM.
+
+        Args:
+            prompt: The input prompt
+
+        Returns:
+            The generated text response
+        """
+        response = self._client.chat.completions.create(
+            model=self.model,
+            messages=[{"role": "user", "content": prompt}],
+            max_tokens=self.max_tokens,
+            temperature=self.temperature,
+        )
+        return response.choices[0].message.content or ""

recursive_cleaner-1.0.0/demo_tui.py

@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+"""
+Demo script to showcase the Rich TUI with real MLX backend.
+
+Run with:
+    python demo_tui.py
+
+Requirements:
+    pip install recursive-cleaner[mlx,tui]
+"""
+
+from backends import MLXBackend
+from recursive_cleaner import DataCleaner
+
+# Use a smaller/faster model for demo (change to your preferred model)
+MODEL = "lmstudio-community/Qwen3-Next-80B-A3B-Instruct-MLX-4bit"
+
+print("=" * 60)
+print("  RECURSIVE DATA CLEANER - TUI DEMO")
+print("=" * 60)
+print(f"\nLoading model: {MODEL}")
+print("This may take a moment on first run...\n")
+
+llm = MLXBackend(
+    model_path=MODEL,
+    max_tokens=2048,
+    temperature=0.3,  # Lower for more consistent output
+    verbose=False,  # Disable token streaming to avoid interfering with TUI
+)
+
+cleaner = DataCleaner(
+    llm_backend=llm,
+    file_path="test_cases/ecommerce_products.jsonl",
+    chunk_size=5,  # Small chunks for demo
+    max_iterations=3,  # Limit iterations per chunk
+    instructions="""
+    E-commerce product data cleaning:
+    - Normalize prices to float (remove $ symbols)
+    - Fix category typos and normalize to Title Case
+    - Convert weights to kg as float
+    - Ensure stock_quantity is non-negative integer
+    """,
+    tui=True,  # Enable the Rich dashboard!
+    track_metrics=True,
+)
+
+print("\nStarting cleaner with TUI enabled...")
+print("Watch the dashboard below!\n")
+
+cleaner.run()
+
+print("\n" + "=" * 60)
+print("Demo complete! Check cleaning_functions.py for output.")
+print("=" * 60)