oh-my-opencode 4.19.3 → 4.19.4

package/dist/skills/data-scientist/references/common-scenarios.md

@@ -0,0 +1,176 @@
+# Common Scenarios with Tool Selection
+
+## Scenario 1: Simple Calculation
+
+**Decision: Python (always)**
+
+```python
+uv run --with numpy python -c "
+import numpy as np
+result = np.sum([1, 2, 3, 4, 5])
+print(f'Result: {result}')
+"
+```
+
+## Scenario 2: CSV Quick Analysis
+
+**Decision: DuckDB (direct query, no memory load)**
+
+```python
+uv run --with numpy --with duckdb python -c "
+import duckdb
+result = duckdb.sql('''
+    SELECT * FROM 'data.csv'
+    LIMIT 10
+''').pl()
+print(result)
+"
+```
+
+## Scenario 3: Filter + Sort on Large Dataset
+
+**Decision: Polars (128x faster filtering, 12x faster sorting)**
+
+```python
+uv run --with numpy --with polars python -c "
+import polars as pl
+result = (
+    pl.scan_csv('large.csv')
+    .filter(pl.col('value') > 1000)
+    .sort('value', descending=True)
+    .head(100)
+    .collect()
+)
+print(result)
+"
+```
+
+## Scenario 4: Multi-Table Join + Aggregation
+
+**Decision: DuckDB (best for joins and aggregations)**
+
+```python
+uv run --with numpy --with duckdb python -c "
+import duckdb
+result = duckdb.sql('''
+    SELECT
+        a.category,
+        COUNT(*) as count,
+        SUM(b.amount) as total
+    FROM 'table1.csv' a
+    JOIN 'table2.csv' b ON a.id = b.id
+    GROUP BY a.category
+    ORDER BY total DESC
+''').pl()
+print(result)
+"
+```
+
+## Scenario 5: Data Exploration
+
+**Decision: DuckDB for quick exploration**
+
+```python
+uv run --with numpy --with duckdb python -c "
+import duckdb
+
+# Show first few rows
+print('**Sample Data**')
+print(duckdb.sql('SELECT * FROM \"data.csv\" LIMIT 5').pl())
+
+# Show summary statistics
+print('\\n**Summary Statistics**')
+print(duckdb.sql('DESCRIBE SELECT * FROM \"data.csv\"').pl())
+
+# Show row count
+print('\\n**Row Count**')
+print(duckdb.sql('SELECT COUNT(*) as total_rows FROM \"data.csv\"').pl())
+"
+```
+
+## Scenario 6: Time-Series Analysis
+
+**Decision: DuckDB for aggregation + matplotlib for visualization**
+
+```python
+uv run --with numpy --with duckdb --with pyarrow --with matplotlib python -c "
+import duckdb
+import matplotlib.pyplot as plt
+
+# Aggregate by date
+result = duckdb.sql('''
+    SELECT
+        DATE_TRUNC('day', timestamp) as date,
+        COUNT(*) as count,
+        AVG(value) as avg_value
+    FROM 'timeseries.csv'
+    GROUP BY date
+    ORDER BY date
+''').pl()
+
+# Plot
+plt.figure(figsize=(12, 6))
+plt.subplot(2, 1, 1)
+plt.plot(result['date'], result['count'])
+plt.title('Daily Count')
+
+plt.subplot(2, 1, 2)
+plt.plot(result['date'], result['avg_value'])
+plt.title('Daily Average Value')
+
+plt.tight_layout()
+plt.savefig('timeseries.png')
+print('Saved to timeseries.png')
+"
+```
+
+## Scenario 7: Complex Transformation
+
+**Decision: Polars for efficient transformations**
+
+```python
+uv run --with numpy --with polars python -c "
+import polars as pl
+
+result = (
+    pl.scan_csv('data.csv')
+    .with_columns([
+        # Create new calculated columns
+        (pl.col('price') * pl.col('quantity')).alias('total'),
+        pl.col('date').str.strptime(pl.Date, '%Y-%m-%d').alias('parsed_date'),
+        pl.col('name').str.to_uppercase().alias('upper_name'),
+    ])
+    .filter(pl.col('total') > 100)
+    .select(['parsed_date', 'upper_name', 'total'])
+    .collect()
+)
+
+print(result)
+"
+```
+
+## Scenario 8: Large File Processing
+
+**Decision: Polars streaming mode**
+
+```python
+uv run --with numpy --with polars python -c "
+import polars as pl
+
+# Process file larger than RAM
+result = (
+    pl.scan_csv('huge_file.csv')
+    .filter(pl.col('active') == True)
+    .groupby('category')
+    .agg([
+        pl.count().alias('count'),
+        pl.sum('amount').alias('total'),
+        pl.mean('amount').alias('average'),
+    ])
+    .collect(streaming=True)  # Streaming mode
+)
+
+print(result)
+print(f'\\nProcessed {result[\"count\"].sum():,} rows')
+"
+```

package/dist/skills/data-scientist/references/execution-templates.md

@@ -0,0 +1,197 @@
+# Execution Templates
+
+## Template 1: DuckDB for Simple/Complex SQL Queries
+
+**Use when:**
+- Simple aggregation on single file
+- Complex multi-table joins
+- Heavy GROUP BY operations
+- Window functions with complex SQL logic
+- Ad-hoc exploration queries
+
+```python
+uv run --with numpy --with duckdb python -c "
+import duckdb
+
+# Simple query - direct file access
+result = duckdb.sql('''
+    SELECT
+        category,
+        COUNT(*) as count,
+        AVG(amount) as avg_amount,
+        SUM(amount) as total
+    FROM 'data.csv'
+    WHERE date >= '2024-01-01'
+    GROUP BY category
+    ORDER BY total DESC
+    LIMIT 10
+''').pl()
+
+print('**Results**')
+print(result)
+print(f'\\nProcessed {len(result)} categories')
+"
+```
+
+## Template 2: Polars for Filtering & Sorting
+
+**Use when:**
+- Primary operation is filtering large dataset
+- Sorting required
+- Chain transformations
+- Memory-efficient processing needed
+
+```python
+uv run --with numpy --with polars python -c "
+import polars as pl
+
+# Lazy evaluation for optimal performance
+result = (
+    pl.scan_csv('data.csv')  # Lazy scan
+    .filter(
+        (pl.col('amount') > 1000) &
+        (pl.col('status') == 'active') &
+        (pl.col('date') >= '2024-01-01')
+    )
+    .sort('amount', descending=True)
+    .head(100)
+    .collect()  # Execute optimized plan
+)
+
+print('**Filtered and Sorted Results**')
+print(result)
+print(f'\\nFound {len(result)} matching rows')
+"
+```
+
+## Template 3: Hybrid Approach (Best of Both)
+
+**Use when:**
+- Need joins AND heavy filtering
+- Complex SQL followed by transformations
+- Optimize different operation stages
+
+```python
+uv run --with numpy --with duckdb --with polars --with pyarrow python -c "
+import duckdb
+import polars as pl
+
+print('Phase 1: DuckDB for complex join (3x faster)')
+# DuckDB excels at joins
+joined = duckdb.sql('''
+    SELECT
+        o.order_id,
+        o.amount,
+        c.customer_id,
+        c.region,
+        p.category
+    FROM 'orders.csv' o
+    JOIN 'customers.csv' c ON o.customer_id = c.customer_id
+    JOIN 'products.csv' p ON o.product_id = p.product_id
+    WHERE o.date >= '2024-01-01'
+''').pl()  # Convert to Polars
+
+print(f'Joined {len(joined):,} rows')
+
+print('\\nPhase 2: Polars for ultra-fast filtering (128x faster)')
+# Polars excels at filtering
+filtered = (
+    joined
+    .filter(
+        (pl.col('amount') > 100) &
+        (pl.col('region').is_in(['North', 'South', 'East']))
+    )
+    .with_columns([
+        (pl.col('amount') * 1.1).alias('amount_with_tax')
+    ])
+)
+
+print(f'Filtered to {len(filtered):,} rows')
+
+print('\\nPhase 3: DuckDB for final aggregation (4x faster)')
+# Back to DuckDB for aggregation
+duckdb.register('filtered_data', filtered)
+final = duckdb.sql('''
+    SELECT
+        region,
+        category,
+        COUNT(DISTINCT customer_id) as customers,
+        SUM(amount_with_tax) as total_revenue,
+        AVG(amount_with_tax) as avg_transaction
+    FROM filtered_data
+    GROUP BY region, category
+    HAVING total_revenue > 10000
+    ORDER BY total_revenue DESC
+''').pl()
+
+print('\\n**Final Results**')
+print(final)
+"
+```
+
+## Template 4: Polars Streaming for Large Files
+
+**Use when:**
+- Dataset larger than available RAM
+- Need to process data in batches
+- Memory constraints
+
+```python
+uv run --with numpy --with polars python -c "
+import polars as pl
+
+# Streaming mode - processes data in chunks
+result = (
+    pl.scan_csv('huge_file.csv')
+    .filter(pl.col('status') == 'active')
+    .with_columns([
+        (pl.col('amount') * 1.1).alias('adjusted_amount')
+    ])
+    .groupby('category')
+    .agg([
+        pl.sum('adjusted_amount').alias('total'),
+        pl.count().alias('count')
+    ])
+    .collect(streaming=True)  # Streaming mode for large data
+)
+
+print('**Streaming Results**')
+print(result)
+print(f'\\nProcessed {result[\"count\"].sum():,} total rows')
+"
+```
+
+## Template 5: Visualization with Matplotlib
+
+**Use when:**
+- User requests charts, graphs, or plots
+- Exploratory data analysis (EDA)
+- Time-series or distribution analysis
+
+```python
+uv run --with numpy --with duckdb --with polars --with pyarrow --with matplotlib python -c "
+import duckdb
+import matplotlib.pyplot as plt
+
+# Query data
+result = duckdb.sql('''
+    SELECT
+        date,
+        SUM(amount) as total
+    FROM 'data.csv'
+    GROUP BY date
+    ORDER BY date
+''').pl()
+
+# Create visualization
+plt.figure(figsize=(10, 6))
+plt.plot(result['date'], result['total'])
+plt.xlabel('Date')
+plt.ylabel('Total Amount')
+plt.title('Daily Total Trends')
+plt.xticks(rotation=45)
+plt.tight_layout()
+plt.savefig('output.png')
+print('Chart saved to output.png')
+"
+```

package/dist/skills/data-scientist/references/integration-patterns.md

@@ -0,0 +1,153 @@
+# Integration Patterns: DuckDB ↔ Polars
+
+## Zero-Copy Conversions (FASTEST)
+
+### DuckDB → Polars (Recommended)
+
+```python
+import duckdb
+import polars as pl
+
+# Direct conversion with .pl() - zero-copy via Arrow
+df_polars = duckdb.sql("""
+    SELECT * FROM 'data.parquet'
+    WHERE amount > 100
+""").pl()  # Returns Polars DataFrame directly
+
+# Lazy version for large datasets
+lazy_df = duckdb.sql("SELECT * FROM 'data.parquet'").pl(lazy=True)
+result = lazy_df.filter(pl.col('status') == 'active').collect()
+```
+
+### Polars → DuckDB (Direct Reference)
+
+```python
+import duckdb
+import polars as pl
+
+# DuckDB can query Polars DataFrames directly by name
+df = pl.read_parquet('data.parquet')
+
+result = duckdb.sql("""
+    SELECT category, SUM(amount) as total
+    FROM df
+    GROUP BY category
+    ORDER BY total DESC
+""").pl()  # Query df directly, return as Polars
+```
+
+### Via Arrow (When Needed)
+
+```python
+# Polars → Arrow → DuckDB
+df_polars = pl.read_csv('data.csv')
+duckdb.register('my_table', df_polars.to_arrow())
+
+# DuckDB → Arrow → Polars
+arrow_table = duckdb.sql("SELECT * FROM data").arrow()
+df_polars = pl.from_arrow(arrow_table)
+```
+
+## WRONG vs RIGHT Patterns
+
+### ❌ NEVER - Using Pandas
+
+```python
+# FORBIDDEN - decisively slower than Polars/DuckDB on every operation!
+import pandas as pd
+df = pd.read_csv('data.csv')
+result = df.groupby('category')['amount'].sum()
+```
+
+### ✅ CORRECT - DuckDB for simple aggregation query
+
+```python
+import duckdb
+# Direct file query - no memory loading!
+result = duckdb.sql("""
+    SELECT category, SUM(amount) as total
+    FROM 'data.csv'
+    GROUP BY category
+""").pl()  # Fast, memory-efficient
+```
+
+### ❌ NEVER - Loading file before DuckDB query
+
+```python
+# WRONG - Unnecessary memory usage
+import polars as pl
+import duckdb
+df = pl.read_csv('data.csv')  # Loads entire file
+result = duckdb.sql("SELECT * FROM df WHERE amount > 100").pl()
+```
+
+### ✅ CORRECT - Let DuckDB query directly
+
+```python
+import duckdb
+# DuckDB queries file directly - much faster!
+result = duckdb.sql("""
+    SELECT * FROM 'data.csv'
+    WHERE amount > 100
+""").pl()
+```
+
+### ❌ NEVER - Eager evaluation in Polars
+
+```python
+# WRONG - Loads everything immediately
+import polars as pl
+df = pl.read_csv('large_data.csv')  # Eager load
+filtered = df.filter(pl.col('value') > 100)
+```
+
+### ✅ CORRECT - Lazy evaluation
+
+```python
+import polars as pl
+# Lazy - builds query plan, optimizes, executes once
+df = pl.scan_csv('large_data.csv')  # Lazy
+result = (
+    df
+    .filter(pl.col('value') > 100)
+    .groupby('category')
+    .agg(pl.sum('value'))
+    .collect()  # Execute optimized plan
+)
+```
+
+### ❌ NEVER - Unnecessary Conversions
+
+```python
+# WASTEFUL (DuckDB → Pandas → Polars)
+import duckdb, pandas as pd, polars as pl
+df_pd = duckdb.sql("SELECT * FROM 'data.csv'").df()  # requires pandas - the skill never ships it
+df_pl = pl.from_pandas(df_pd)
+```
+
+### ✅ CORRECT - Direct conversion
+
+```python
+# DIRECT (DuckDB → Polars via Arrow)
+import duckdb
+df_pl = duckdb.sql("SELECT * FROM 'data.csv'").pl()
+```
+
+### ❌ NEVER - Wrong tool for heavy filtering
+
+```python
+# SLOW (DuckDB not optimal for filtering)
+import duckdb
+result = duckdb.sql("""
+    SELECT * FROM 'huge.csv'
+    WHERE complex_filter = true
+""").pl()
+```
+
+### ✅ CORRECT - Use Polars for filtering
+
+```python
+# FAST (Polars 128x faster for filtering)
+import polars as pl
+result = pl.scan_csv('huge.csv').filter(pl.col('complex_filter')).collect()
+```

package/dist/skills/data-scientist/references/performance-benchmarks.md

@@ -0,0 +1,37 @@
+# Performance Benchmarks
+
+Routing heuristics distilled from public 2024-2025 results in the [db-benchmark suite](https://github.com/h2oai/db-benchmark) (rendered at [h2oai.github.io/db-benchmark](https://h2oai.github.io/db-benchmark/)) and from the vendors' own documentation ([duckdb.org](https://duckdb.org/), [pola.rs](https://pola.rs/)).
+
+**Read these as routing guidance, not guarantees.** Multipliers vary with dataset size, cardinality, data types, and hardware. When the choice materially matters, measure on the actual data.
+
+## Operation Performance Comparison
+
+| Operation Type      | Winner      | Indicative advantage | When to Use                                    |
+|---------------------|-------------|----------------------|------------------------------------------------|
+| **Filtering**       | **Polars**  | often the fastest by a wide margin (SIMD, predicate pushdown) | Single-table filters |
+| **Sorting**         | **Polars**  | typically the fastest | ORDER BY operations, ranking                   |
+| **Joins**           | **DuckDB**  | typically faster; richer join types | Multi-table joins, especially complex joins    |
+| **Aggregations**    | **DuckDB**  | typically faster on large datasets | GROUP BY, complex aggregations                 |
+| **Window Funcs**    | **Polars**  | typically faster | RANK, LAG/LEAD, running totals                 |
+| **Transformations** | **Polars**  | typically faster | Pivot, melt, string operations                 |
+| **Direct Query**    | **DuckDB**  | avoids loading to memory entirely | Ad-hoc exploration without loading to memory   |
+| **Streaming**       | **Polars**  | handles datasets larger than RAM | Datasets larger than RAM                       |
+
+Both tools are decisively faster than pandas on 1M+ row workloads, which is why pandas is banned outright in this skill.
+
+## Operation Detection Keywords
+
+Automatically detect operation types from user requests:
+
+- **Filter**: "where", "filter", "condition", "select rows", "find"
+- **Sort**: "sort", "order", "top", "bottom", "rank"
+- **Join**: "join", "merge", "combine tables"
+- **Aggregate**: "group", "sum", "avg", "count", "mean", "total", "aggregate"
+- **Transform**: "pivot", "melt", "reshape", "string operations", "clean", "transform"
+- **Window**: "running", "cumulative", "lag", "lead", "rank", "partition"
+
+## Benchmarking Notes
+
+- Public suites exercise typical analytical workloads (1M-100M rows).
+- Performance advantages are approximate and vary by dataset characteristics.
+- Real-world performance depends on data types, cardinality, and hardware.

package/dist/skills/data-scientist/references/uv-setup.md

@@ -0,0 +1,78 @@
+# uv Setup — Per-Platform
+
+This skill runs every data operation through `uv run --with ...`. If `uv --version` fails, set uv up with the automated scripts or the manual commands below, then verify.
+
+## Automated (recommended)
+
+| Platform | Command |
+|---|---|
+| macOS / Linux / WSL / Git Bash | `bash scripts/setup-uv.sh` |
+| Windows (PowerShell) | `powershell -ExecutionPolicy Bypass -File scripts/setup-uv.ps1` |
+
+Both scripts: detect OS + architecture → install uv to the latest release when missing → upgrade it when present (`uv self update`) → make it resolvable for the current shell → verify with `uv --version`. They are idempotent — safe to re-run any time.
+
+## Manual install per platform
+
+### macOS
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh        # official installer → ~/.local/bin/uv
+# or, with Homebrew:
+brew install uv
+```
+
+### Linux (x86_64 / aarch64)
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh        # official installer → ~/.local/bin/uv
+```
+
+The installer detects glibc vs musl and downloads the right static binary. On minimal containers, ensure `curl` (or `wget`) exists; `wget -qO- https://astral.sh/uv/install.sh | sh` is the fallback.
+
+### Windows (native)
+
+```powershell
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+# or, with winget:
+winget install --id=astral-sh.uv -e
+```
+
+### Windows Subsystem for Linux / Git Bash
+
+Use the Linux/macOS installer inside the Unix shell, not the PowerShell installer:
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+### CI
+
+```yaml
+# GitHub Actions
+- uses: astral-sh/setup-uv@v5
+# or plain shell anywhere:
+- run: curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+## PATH notes
+
+- The official installers put the binary in `~/.local/bin` (Unix) or `%USERPROFILE%\.local\bin` (Windows). New shells get it automatically on most setups; an already-open shell needs `export PATH="$HOME/.local/bin:$PATH"` (Unix) or `$env:Path = "$env:USERPROFILE\.local\bin;$env:Path"` (PowerShell) once.
+- Homebrew and winget install into their own prefixes that are already on PATH.
+
+## Upgrade to latest
+
+```bash
+uv self update
+```
+
+`uv self update` only works for official-installer binaries; Homebrew/winget installs upgrade through their package manager (`brew upgrade uv` / `winget upgrade astral-sh.uv`). The setup scripts handle this automatically.
+
+## Verify
+
+```bash
+uv --version
+```
+
+## Offline / air-gapped
+
+Download the matching archive from the uv GitHub releases page, extract it, and put the `uv` binary anywhere on PATH. `uv run --with <pkg>` still needs network for first-time package resolution unless a mirror is configured via `UV_INDEX_URL`.