oh-my-opencode 5.0.0-beta.3 → 5.0.0-beta.31

package/packages/shared-skills/skills/data-scientist/references/common-scenarios.md

@@ -1,176 +0,0 @@
-# Common Scenarios with Tool Selection
-
-## Scenario 1: Simple Calculation
-
-**Decision: Python (always)**
-
-```python
-uv run --with numpy python -c "
-import numpy as np
-result = np.sum([1, 2, 3, 4, 5])
-print(f'Result: {result}')
-"
-```
-
-## Scenario 2: CSV Quick Analysis
-
-**Decision: DuckDB (direct query, no memory load)**
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-result = duckdb.sql('''
-    SELECT * FROM 'data.csv'
-    LIMIT 10
-''').pl()
-print(result)
-"
-```
-
-## Scenario 3: Filter + Sort on Large Dataset
-
-**Decision: Polars (128x faster filtering, 12x faster sorting)**
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-result = (
-    pl.scan_csv('large.csv')
-    .filter(pl.col('value') > 1000)
-    .sort('value', descending=True)
-    .head(100)
-    .collect()
-)
-print(result)
-"
-```
-
-## Scenario 4: Multi-Table Join + Aggregation
-
-**Decision: DuckDB (best for joins and aggregations)**
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-result = duckdb.sql('''
-    SELECT
-        a.category,
-        COUNT(*) as count,
-        SUM(b.amount) as total
-    FROM 'table1.csv' a
-    JOIN 'table2.csv' b ON a.id = b.id
-    GROUP BY a.category
-    ORDER BY total DESC
-''').pl()
-print(result)
-"
-```
-
-## Scenario 5: Data Exploration
-
-**Decision: DuckDB for quick exploration**
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-
-# Show first few rows
-print('**Sample Data**')
-print(duckdb.sql('SELECT * FROM \"data.csv\" LIMIT 5').pl())
-
-# Show summary statistics
-print('\\n**Summary Statistics**')
-print(duckdb.sql('DESCRIBE SELECT * FROM \"data.csv\"').pl())
-
-# Show row count
-print('\\n**Row Count**')
-print(duckdb.sql('SELECT COUNT(*) as total_rows FROM \"data.csv\"').pl())
-"
-```
-
-## Scenario 6: Time-Series Analysis
-
-**Decision: DuckDB for aggregation + matplotlib for visualization**
-
-```python
-uv run --with numpy --with duckdb --with pyarrow --with matplotlib python -c "
-import duckdb
-import matplotlib.pyplot as plt
-
-# Aggregate by date
-result = duckdb.sql('''
-    SELECT
-        DATE_TRUNC('day', timestamp) as date,
-        COUNT(*) as count,
-        AVG(value) as avg_value
-    FROM 'timeseries.csv'
-    GROUP BY date
-    ORDER BY date
-''').pl()
-
-# Plot
-plt.figure(figsize=(12, 6))
-plt.subplot(2, 1, 1)
-plt.plot(result['date'], result['count'])
-plt.title('Daily Count')
-
-plt.subplot(2, 1, 2)
-plt.plot(result['date'], result['avg_value'])
-plt.title('Daily Average Value')
-
-plt.tight_layout()
-plt.savefig('timeseries.png')
-print('Saved to timeseries.png')
-"
-```
-
-## Scenario 7: Complex Transformation
-
-**Decision: Polars for efficient transformations**
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-result = (
-    pl.scan_csv('data.csv')
-    .with_columns([
-        # Create new calculated columns
-        (pl.col('price') * pl.col('quantity')).alias('total'),
-        pl.col('date').str.strptime(pl.Date, '%Y-%m-%d').alias('parsed_date'),
-        pl.col('name').str.to_uppercase().alias('upper_name'),
-    ])
-    .filter(pl.col('total') > 100)
-    .select(['parsed_date', 'upper_name', 'total'])
-    .collect()
-)
-
-print(result)
-"
-```
-
-## Scenario 8: Large File Processing
-
-**Decision: Polars streaming mode**
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Process file larger than RAM
-result = (
-    pl.scan_csv('huge_file.csv')
-    .filter(pl.col('active') == True)
-    .groupby('category')
-    .agg([
-        pl.count().alias('count'),
-        pl.sum('amount').alias('total'),
-        pl.mean('amount').alias('average'),
-    ])
-    .collect(streaming=True)  # Streaming mode
-)
-
-print(result)
-print(f'\\nProcessed {result[\"count\"].sum():,} rows')
-"
-```

package/packages/shared-skills/skills/data-scientist/references/execution-templates.md

@@ -1,197 +0,0 @@
-# Execution Templates
-
-## Template 1: DuckDB for Simple/Complex SQL Queries
-
-**Use when:**
-- Simple aggregation on single file
-- Complex multi-table joins
-- Heavy GROUP BY operations
-- Window functions with complex SQL logic
-- Ad-hoc exploration queries
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-
-# Simple query - direct file access
-result = duckdb.sql('''
-    SELECT
-        category,
-        COUNT(*) as count,
-        AVG(amount) as avg_amount,
-        SUM(amount) as total
-    FROM 'data.csv'
-    WHERE date >= '2024-01-01'
-    GROUP BY category
-    ORDER BY total DESC
-    LIMIT 10
-''').pl()
-
-print('**Results**')
-print(result)
-print(f'\\nProcessed {len(result)} categories')
-"
-```
-
-## Template 2: Polars for Filtering & Sorting
-
-**Use when:**
-- Primary operation is filtering large dataset
-- Sorting required
-- Chain transformations
-- Memory-efficient processing needed
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Lazy evaluation for optimal performance
-result = (
-    pl.scan_csv('data.csv')  # Lazy scan
-    .filter(
-        (pl.col('amount') > 1000) &
-        (pl.col('status') == 'active') &
-        (pl.col('date') >= '2024-01-01')
-    )
-    .sort('amount', descending=True)
-    .head(100)
-    .collect()  # Execute optimized plan
-)
-
-print('**Filtered and Sorted Results**')
-print(result)
-print(f'\\nFound {len(result)} matching rows')
-"
-```
-
-## Template 3: Hybrid Approach (Best of Both)
-
-**Use when:**
-- Need joins AND heavy filtering
-- Complex SQL followed by transformations
-- Optimize different operation stages
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow python -c "
-import duckdb
-import polars as pl
-
-print('Phase 1: DuckDB for complex join (3x faster)')
-# DuckDB excels at joins
-joined = duckdb.sql('''
-    SELECT
-        o.order_id,
-        o.amount,
-        c.customer_id,
-        c.region,
-        p.category
-    FROM 'orders.csv' o
-    JOIN 'customers.csv' c ON o.customer_id = c.customer_id
-    JOIN 'products.csv' p ON o.product_id = p.product_id
-    WHERE o.date >= '2024-01-01'
-''').pl()  # Convert to Polars
-
-print(f'Joined {len(joined):,} rows')
-
-print('\\nPhase 2: Polars for ultra-fast filtering (128x faster)')
-# Polars excels at filtering
-filtered = (
-    joined
-    .filter(
-        (pl.col('amount') > 100) &
-        (pl.col('region').is_in(['North', 'South', 'East']))
-    )
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('amount_with_tax')
-    ])
-)
-
-print(f'Filtered to {len(filtered):,} rows')
-
-print('\\nPhase 3: DuckDB for final aggregation (4x faster)')
-# Back to DuckDB for aggregation
-duckdb.register('filtered_data', filtered)
-final = duckdb.sql('''
-    SELECT
-        region,
-        category,
-        COUNT(DISTINCT customer_id) as customers,
-        SUM(amount_with_tax) as total_revenue,
-        AVG(amount_with_tax) as avg_transaction
-    FROM filtered_data
-    GROUP BY region, category
-    HAVING total_revenue > 10000
-    ORDER BY total_revenue DESC
-''').pl()
-
-print('\\n**Final Results**')
-print(final)
-"
-```
-
-## Template 4: Polars Streaming for Large Files
-
-**Use when:**
-- Dataset larger than available RAM
-- Need to process data in batches
-- Memory constraints
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Streaming mode - processes data in chunks
-result = (
-    pl.scan_csv('huge_file.csv')
-    .filter(pl.col('status') == 'active')
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('adjusted_amount')
-    ])
-    .groupby('category')
-    .agg([
-        pl.sum('adjusted_amount').alias('total'),
-        pl.count().alias('count')
-    ])
-    .collect(streaming=True)  # Streaming mode for large data
-)
-
-print('**Streaming Results**')
-print(result)
-print(f'\\nProcessed {result[\"count\"].sum():,} total rows')
-"
-```
-
-## Template 5: Visualization with Matplotlib
-
-**Use when:**
-- User requests charts, graphs, or plots
-- Exploratory data analysis (EDA)
-- Time-series or distribution analysis
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow --with matplotlib python -c "
-import duckdb
-import matplotlib.pyplot as plt
-
-# Query data
-result = duckdb.sql('''
-    SELECT
-        date,
-        SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY date
-    ORDER BY date
-''').pl()
-
-# Create visualization
-plt.figure(figsize=(10, 6))
-plt.plot(result['date'], result['total'])
-plt.xlabel('Date')
-plt.ylabel('Total Amount')
-plt.title('Daily Total Trends')
-plt.xticks(rotation=45)
-plt.tight_layout()
-plt.savefig('output.png')
-print('Chart saved to output.png')
-"
-```

package/packages/shared-skills/skills/data-scientist/references/integration-patterns.md

@@ -1,153 +0,0 @@
-# Integration Patterns: DuckDB ↔ Polars
-
-## Zero-Copy Conversions (FASTEST)
-
-### DuckDB → Polars (Recommended)
-
-```python
-import duckdb
-import polars as pl
-
-# Direct conversion with .pl() - zero-copy via Arrow
-df_polars = duckdb.sql("""
-    SELECT * FROM 'data.parquet'
-    WHERE amount > 100
-""").pl()  # Returns Polars DataFrame directly
-
-# Lazy version for large datasets
-lazy_df = duckdb.sql("SELECT * FROM 'data.parquet'").pl(lazy=True)
-result = lazy_df.filter(pl.col('status') == 'active').collect()
-```
-
-### Polars → DuckDB (Direct Reference)
-
-```python
-import duckdb
-import polars as pl
-
-# DuckDB can query Polars DataFrames directly by name
-df = pl.read_parquet('data.parquet')
-
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM df
-    GROUP BY category
-    ORDER BY total DESC
-""").pl()  # Query df directly, return as Polars
-```
-
-### Via Arrow (When Needed)
-
-```python
-# Polars → Arrow → DuckDB
-df_polars = pl.read_csv('data.csv')
-duckdb.register('my_table', df_polars.to_arrow())
-
-# DuckDB → Arrow → Polars
-arrow_table = duckdb.sql("SELECT * FROM data").arrow()
-df_polars = pl.from_arrow(arrow_table)
-```
-
-## WRONG vs RIGHT Patterns
-
-### ❌ NEVER - Using Pandas
-
-```python
-# FORBIDDEN - decisively slower than Polars/DuckDB on every operation!
-import pandas as pd
-df = pd.read_csv('data.csv')
-result = df.groupby('category')['amount'].sum()
-```
-
-### ✅ CORRECT - DuckDB for simple aggregation query
-
-```python
-import duckdb
-# Direct file query - no memory loading!
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY category
-""").pl()  # Fast, memory-efficient
-```
-
-### ❌ NEVER - Loading file before DuckDB query
-
-```python
-# WRONG - Unnecessary memory usage
-import polars as pl
-import duckdb
-df = pl.read_csv('data.csv')  # Loads entire file
-result = duckdb.sql("SELECT * FROM df WHERE amount > 100").pl()
-```
-
-### ✅ CORRECT - Let DuckDB query directly
-
-```python
-import duckdb
-# DuckDB queries file directly - much faster!
-result = duckdb.sql("""
-    SELECT * FROM 'data.csv'
-    WHERE amount > 100
-""").pl()
-```
-
-### ❌ NEVER - Eager evaluation in Polars
-
-```python
-# WRONG - Loads everything immediately
-import polars as pl
-df = pl.read_csv('large_data.csv')  # Eager load
-filtered = df.filter(pl.col('value') > 100)
-```
-
-### ✅ CORRECT - Lazy evaluation
-
-```python
-import polars as pl
-# Lazy - builds query plan, optimizes, executes once
-df = pl.scan_csv('large_data.csv')  # Lazy
-result = (
-    df
-    .filter(pl.col('value') > 100)
-    .groupby('category')
-    .agg(pl.sum('value'))
-    .collect()  # Execute optimized plan
-)
-```
-
-### ❌ NEVER - Unnecessary Conversions
-
-```python
-# WASTEFUL (DuckDB → Pandas → Polars)
-import duckdb, pandas as pd, polars as pl
-df_pd = duckdb.sql("SELECT * FROM 'data.csv'").df()  # requires pandas - the skill never ships it
-df_pl = pl.from_pandas(df_pd)
-```
-
-### ✅ CORRECT - Direct conversion
-
-```python
-# DIRECT (DuckDB → Polars via Arrow)
-import duckdb
-df_pl = duckdb.sql("SELECT * FROM 'data.csv'").pl()
-```
-
-### ❌ NEVER - Wrong tool for heavy filtering
-
-```python
-# SLOW (DuckDB not optimal for filtering)
-import duckdb
-result = duckdb.sql("""
-    SELECT * FROM 'huge.csv'
-    WHERE complex_filter = true
-""").pl()
-```
-
-### ✅ CORRECT - Use Polars for filtering
-
-```python
-# FAST (Polars 128x faster for filtering)
-import polars as pl
-result = pl.scan_csv('huge.csv').filter(pl.col('complex_filter')).collect()
-```

package/packages/shared-skills/skills/data-scientist/references/performance-benchmarks.md

@@ -1,37 +0,0 @@
-# Performance Benchmarks
-
-Routing heuristics distilled from public 2024-2025 results in the [db-benchmark suite](https://github.com/h2oai/db-benchmark) (rendered at [h2oai.github.io/db-benchmark](https://h2oai.github.io/db-benchmark/)) and from the vendors' own documentation ([duckdb.org](https://duckdb.org/), [pola.rs](https://pola.rs/)).
-
-**Read these as routing guidance, not guarantees.** Multipliers vary with dataset size, cardinality, data types, and hardware. When the choice materially matters, measure on the actual data.
-
-## Operation Performance Comparison
-
-| Operation Type      | Winner      | Indicative advantage | When to Use                                    |
-|---------------------|-------------|----------------------|------------------------------------------------|
-| **Filtering**       | **Polars**  | often the fastest by a wide margin (SIMD, predicate pushdown) | Single-table filters |
-| **Sorting**         | **Polars**  | typically the fastest | ORDER BY operations, ranking                   |
-| **Joins**           | **DuckDB**  | typically faster; richer join types | Multi-table joins, especially complex joins    |
-| **Aggregations**    | **DuckDB**  | typically faster on large datasets | GROUP BY, complex aggregations                 |
-| **Window Funcs**    | **Polars**  | typically faster | RANK, LAG/LEAD, running totals                 |
-| **Transformations** | **Polars**  | typically faster | Pivot, melt, string operations                 |
-| **Direct Query**    | **DuckDB**  | avoids loading to memory entirely | Ad-hoc exploration without loading to memory   |
-| **Streaming**       | **Polars**  | handles datasets larger than RAM | Datasets larger than RAM                       |
-
-Both tools are decisively faster than pandas on 1M+ row workloads, which is why pandas is banned outright in this skill.
-
-## Operation Detection Keywords
-
-Automatically detect operation types from user requests:
-
-- **Filter**: "where", "filter", "condition", "select rows", "find"
-- **Sort**: "sort", "order", "top", "bottom", "rank"
-- **Join**: "join", "merge", "combine tables"
-- **Aggregate**: "group", "sum", "avg", "count", "mean", "total", "aggregate"
-- **Transform**: "pivot", "melt", "reshape", "string operations", "clean", "transform"
-- **Window**: "running", "cumulative", "lag", "lead", "rank", "partition"
-
-## Benchmarking Notes
-
-- Public suites exercise typical analytical workloads (1M-100M rows).
-- Performance advantages are approximate and vary by dataset characteristics.
-- Real-world performance depends on data types, cardinality, and hardware.