lazycodex-ai 5.0.0-beta.8 → 5.0.0-beta.81

package/packages/shared-skills/skills/data-scientist/references/execution-templates.md

@@ -1,197 +0,0 @@
-# Execution Templates
-
-## Template 1: DuckDB for Simple/Complex SQL Queries
-
-**Use when:**
-- Simple aggregation on single file
-- Complex multi-table joins
-- Heavy GROUP BY operations
-- Window functions with complex SQL logic
-- Ad-hoc exploration queries
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-
-# Simple query - direct file access
-result = duckdb.sql('''
-    SELECT
-        category,
-        COUNT(*) as count,
-        AVG(amount) as avg_amount,
-        SUM(amount) as total
-    FROM 'data.csv'
-    WHERE date >= '2024-01-01'
-    GROUP BY category
-    ORDER BY total DESC
-    LIMIT 10
-''').pl()
-
-print('**Results**')
-print(result)
-print(f'\\nProcessed {len(result)} categories')
-"
-```
-
-## Template 2: Polars for Filtering & Sorting
-
-**Use when:**
-- Primary operation is filtering large dataset
-- Sorting required
-- Chain transformations
-- Memory-efficient processing needed
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Lazy evaluation for optimal performance
-result = (
-    pl.scan_csv('data.csv')  # Lazy scan
-    .filter(
-        (pl.col('amount') > 1000) &
-        (pl.col('status') == 'active') &
-        (pl.col('date') >= '2024-01-01')
-    )
-    .sort('amount', descending=True)
-    .head(100)
-    .collect()  # Execute optimized plan
-)
-
-print('**Filtered and Sorted Results**')
-print(result)
-print(f'\\nFound {len(result)} matching rows')
-"
-```
-
-## Template 3: Hybrid Approach (Best of Both)
-
-**Use when:**
-- Need joins AND heavy filtering
-- Complex SQL followed by transformations
-- Optimize different operation stages
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow python -c "
-import duckdb
-import polars as pl
-
-print('Phase 1: DuckDB for complex join (3x faster)')
-# DuckDB excels at joins
-joined = duckdb.sql('''
-    SELECT
-        o.order_id,
-        o.amount,
-        c.customer_id,
-        c.region,
-        p.category
-    FROM 'orders.csv' o
-    JOIN 'customers.csv' c ON o.customer_id = c.customer_id
-    JOIN 'products.csv' p ON o.product_id = p.product_id
-    WHERE o.date >= '2024-01-01'
-''').pl()  # Convert to Polars
-
-print(f'Joined {len(joined):,} rows')
-
-print('\\nPhase 2: Polars for ultra-fast filtering (128x faster)')
-# Polars excels at filtering
-filtered = (
-    joined
-    .filter(
-        (pl.col('amount') > 100) &
-        (pl.col('region').is_in(['North', 'South', 'East']))
-    )
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('amount_with_tax')
-    ])
-)
-
-print(f'Filtered to {len(filtered):,} rows')
-
-print('\\nPhase 3: DuckDB for final aggregation (4x faster)')
-# Back to DuckDB for aggregation
-duckdb.register('filtered_data', filtered)
-final = duckdb.sql('''
-    SELECT
-        region,
-        category,
-        COUNT(DISTINCT customer_id) as customers,
-        SUM(amount_with_tax) as total_revenue,
-        AVG(amount_with_tax) as avg_transaction
-    FROM filtered_data
-    GROUP BY region, category
-    HAVING total_revenue > 10000
-    ORDER BY total_revenue DESC
-''').pl()
-
-print('\\n**Final Results**')
-print(final)
-"
-```
-
-## Template 4: Polars Streaming for Large Files
-
-**Use when:**
-- Dataset larger than available RAM
-- Need to process data in batches
-- Memory constraints
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Streaming mode - processes data in chunks
-result = (
-    pl.scan_csv('huge_file.csv')
-    .filter(pl.col('status') == 'active')
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('adjusted_amount')
-    ])
-    .groupby('category')
-    .agg([
-        pl.sum('adjusted_amount').alias('total'),
-        pl.count().alias('count')
-    ])
-    .collect(streaming=True)  # Streaming mode for large data
-)
-
-print('**Streaming Results**')
-print(result)
-print(f'\\nProcessed {result[\"count\"].sum():,} total rows')
-"
-```
-
-## Template 5: Visualization with Matplotlib
-
-**Use when:**
-- User requests charts, graphs, or plots
-- Exploratory data analysis (EDA)
-- Time-series or distribution analysis
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow --with matplotlib python -c "
-import duckdb
-import matplotlib.pyplot as plt
-
-# Query data
-result = duckdb.sql('''
-    SELECT
-        date,
-        SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY date
-    ORDER BY date
-''').pl()
-
-# Create visualization
-plt.figure(figsize=(10, 6))
-plt.plot(result['date'], result['total'])
-plt.xlabel('Date')
-plt.ylabel('Total Amount')
-plt.title('Daily Total Trends')
-plt.xticks(rotation=45)
-plt.tight_layout()
-plt.savefig('output.png')
-print('Chart saved to output.png')
-"
-```

package/packages/shared-skills/skills/data-scientist/references/integration-patterns.md

@@ -1,153 +0,0 @@
-# Integration Patterns: DuckDB ↔ Polars
-
-## Zero-Copy Conversions (FASTEST)
-
-### DuckDB → Polars (Recommended)
-
-```python
-import duckdb
-import polars as pl
-
-# Direct conversion with .pl() - zero-copy via Arrow
-df_polars = duckdb.sql("""
-    SELECT * FROM 'data.parquet'
-    WHERE amount > 100
-""").pl()  # Returns Polars DataFrame directly
-
-# Lazy version for large datasets
-lazy_df = duckdb.sql("SELECT * FROM 'data.parquet'").pl(lazy=True)
-result = lazy_df.filter(pl.col('status') == 'active').collect()
-```
-
-### Polars → DuckDB (Direct Reference)
-
-```python
-import duckdb
-import polars as pl
-
-# DuckDB can query Polars DataFrames directly by name
-df = pl.read_parquet('data.parquet')
-
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM df
-    GROUP BY category
-    ORDER BY total DESC
-""").pl()  # Query df directly, return as Polars
-```
-
-### Via Arrow (When Needed)
-
-```python
-# Polars → Arrow → DuckDB
-df_polars = pl.read_csv('data.csv')
-duckdb.register('my_table', df_polars.to_arrow())
-
-# DuckDB → Arrow → Polars
-arrow_table = duckdb.sql("SELECT * FROM data").arrow()
-df_polars = pl.from_arrow(arrow_table)
-```
-
-## WRONG vs RIGHT Patterns
-
-### ❌ NEVER - Using Pandas
-
-```python
-# FORBIDDEN - decisively slower than Polars/DuckDB on every operation!
-import pandas as pd
-df = pd.read_csv('data.csv')
-result = df.groupby('category')['amount'].sum()
-```
-
-### ✅ CORRECT - DuckDB for simple aggregation query
-
-```python
-import duckdb
-# Direct file query - no memory loading!
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY category
-""").pl()  # Fast, memory-efficient
-```
-
-### ❌ NEVER - Loading file before DuckDB query
-
-```python
-# WRONG - Unnecessary memory usage
-import polars as pl
-import duckdb
-df = pl.read_csv('data.csv')  # Loads entire file
-result = duckdb.sql("SELECT * FROM df WHERE amount > 100").pl()
-```
-
-### ✅ CORRECT - Let DuckDB query directly
-
-```python
-import duckdb
-# DuckDB queries file directly - much faster!
-result = duckdb.sql("""
-    SELECT * FROM 'data.csv'
-    WHERE amount > 100
-""").pl()
-```
-
-### ❌ NEVER - Eager evaluation in Polars
-
-```python
-# WRONG - Loads everything immediately
-import polars as pl
-df = pl.read_csv('large_data.csv')  # Eager load
-filtered = df.filter(pl.col('value') > 100)
-```
-
-### ✅ CORRECT - Lazy evaluation
-
-```python
-import polars as pl
-# Lazy - builds query plan, optimizes, executes once
-df = pl.scan_csv('large_data.csv')  # Lazy
-result = (
-    df
-    .filter(pl.col('value') > 100)
-    .groupby('category')
-    .agg(pl.sum('value'))
-    .collect()  # Execute optimized plan
-)
-```
-
-### ❌ NEVER - Unnecessary Conversions
-
-```python
-# WASTEFUL (DuckDB → Pandas → Polars)
-import duckdb, pandas as pd, polars as pl
-df_pd = duckdb.sql("SELECT * FROM 'data.csv'").df()  # requires pandas - the skill never ships it
-df_pl = pl.from_pandas(df_pd)
-```
-
-### ✅ CORRECT - Direct conversion
-
-```python
-# DIRECT (DuckDB → Polars via Arrow)
-import duckdb
-df_pl = duckdb.sql("SELECT * FROM 'data.csv'").pl()
-```
-
-### ❌ NEVER - Wrong tool for heavy filtering
-
-```python
-# SLOW (DuckDB not optimal for filtering)
-import duckdb
-result = duckdb.sql("""
-    SELECT * FROM 'huge.csv'
-    WHERE complex_filter = true
-""").pl()
-```
-
-### ✅ CORRECT - Use Polars for filtering
-
-```python
-# FAST (Polars 128x faster for filtering)
-import polars as pl
-result = pl.scan_csv('huge.csv').filter(pl.col('complex_filter')).collect()
-```

package/packages/shared-skills/skills/data-scientist/references/performance-benchmarks.md

@@ -1,37 +0,0 @@
-# Performance Benchmarks
-
-Routing heuristics distilled from public 2024-2025 results in the [db-benchmark suite](https://github.com/h2oai/db-benchmark) (rendered at [h2oai.github.io/db-benchmark](https://h2oai.github.io/db-benchmark/)) and from the vendors' own documentation ([duckdb.org](https://duckdb.org/), [pola.rs](https://pola.rs/)).
-
-**Read these as routing guidance, not guarantees.** Multipliers vary with dataset size, cardinality, data types, and hardware. When the choice materially matters, measure on the actual data.
-
-## Operation Performance Comparison
-
-| Operation Type      | Winner      | Indicative advantage | When to Use                                    |
-|---------------------|-------------|----------------------|------------------------------------------------|
-| **Filtering**       | **Polars**  | often the fastest by a wide margin (SIMD, predicate pushdown) | Single-table filters |
-| **Sorting**         | **Polars**  | typically the fastest | ORDER BY operations, ranking                   |
-| **Joins**           | **DuckDB**  | typically faster; richer join types | Multi-table joins, especially complex joins    |
-| **Aggregations**    | **DuckDB**  | typically faster on large datasets | GROUP BY, complex aggregations                 |
-| **Window Funcs**    | **Polars**  | typically faster | RANK, LAG/LEAD, running totals                 |
-| **Transformations** | **Polars**  | typically faster | Pivot, melt, string operations                 |
-| **Direct Query**    | **DuckDB**  | avoids loading to memory entirely | Ad-hoc exploration without loading to memory   |
-| **Streaming**       | **Polars**  | handles datasets larger than RAM | Datasets larger than RAM                       |
-
-Both tools are decisively faster than pandas on 1M+ row workloads, which is why pandas is banned outright in this skill.
-
-## Operation Detection Keywords
-
-Automatically detect operation types from user requests:
-
-- **Filter**: "where", "filter", "condition", "select rows", "find"
-- **Sort**: "sort", "order", "top", "bottom", "rank"
-- **Join**: "join", "merge", "combine tables"
-- **Aggregate**: "group", "sum", "avg", "count", "mean", "total", "aggregate"
-- **Transform**: "pivot", "melt", "reshape", "string operations", "clean", "transform"
-- **Window**: "running", "cumulative", "lag", "lead", "rank", "partition"
-
-## Benchmarking Notes
-
-- Public suites exercise typical analytical workloads (1M-100M rows).
-- Performance advantages are approximate and vary by dataset characteristics.
-- Real-world performance depends on data types, cardinality, and hardware.

package/packages/shared-skills/skills/visual-qa/references/agent-browser-setup.md

@@ -1,45 +0,0 @@
-# agent-browser setup (Web capture fallback)
-
-Use this when the project has **no** browser tooling configured or available for the Web
-capture path in Step 2 (no playwright / dev-browser skill, no usable headless browser).
-[agent-browser](https://github.com/vercel-labs/agent-browser) is a standalone CLI that
-drives a real Chromium and can screenshot a page at a fixed viewport — exactly what the
-`image-diff` evidence step needs.
-
-Repo: https://github.com/vercel-labs/agent-browser
-
-## Install
-
-```
-npm install -g agent-browser && agent-browser install
-```
-
-`npm install -g agent-browser` installs the CLI globally; `agent-browser install` downloads the
-managed browser it drives. If Bun is already available, `bun add -g agent-browser && agent-browser install`
-is equivalent, but npm is the default because LazyCodex/Codex installations do not require Bun.
-
-Confirm it is ready and discover the current flags:
-
-```
-agent-browser --help
-```
-
-## Capture a screenshot at a fixed viewport
-
-Set the viewport first, then screenshot — this guarantees the ACTUAL capture matches the
-REFERENCE viewport so `image-diff` compares like-for-like:
-
-```
-agent-browser set viewport 1280 720      # width height (add a third arg, e.g. 2, for retina scale)
-agent-browser screenshot actual.png      # add --full for full-page, --screenshot-dir ./shots for a custom dir
-```
-
-Then feed the PNG into the diff exactly as in Step 2:
-
-```
-node "$SKILL_DIR/scripts/visual-qa.mjs" image-diff <reference.png> actual.png
-```
-
-Match the viewport numbers to whatever the reference/mock was captured at; mismatched
-dimensions make `dimensionsMatch` false and inflate `diffRatio`. Run `agent-browser --help`
-for the full command set (navigation, batch, annotate, format/quality flags).