lazycodex-ai 5.0.0-beta.9 → 5.0.0

package/packages/omo-codex/plugin/skills/data-scientist/references/execution-templates.md

@@ -1,197 +0,0 @@
-# Execution Templates
-
-## Template 1: DuckDB for Simple/Complex SQL Queries
-
-**Use when:**
-- Simple aggregation on single file
-- Complex multi-table joins
-- Heavy GROUP BY operations
-- Window functions with complex SQL logic
-- Ad-hoc exploration queries
-
-```python
-uv run --with numpy --with duckdb python -c "
-import duckdb
-
-# Simple query - direct file access
-result = duckdb.sql('''
-    SELECT
-        category,
-        COUNT(*) as count,
-        AVG(amount) as avg_amount,
-        SUM(amount) as total
-    FROM 'data.csv'
-    WHERE date >= '2024-01-01'
-    GROUP BY category
-    ORDER BY total DESC
-    LIMIT 10
-''').pl()
-
-print('**Results**')
-print(result)
-print(f'\\nProcessed {len(result)} categories')
-"
-```
-
-## Template 2: Polars for Filtering & Sorting
-
-**Use when:**
-- Primary operation is filtering large dataset
-- Sorting required
-- Chain transformations
-- Memory-efficient processing needed
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Lazy evaluation for optimal performance
-result = (
-    pl.scan_csv('data.csv')  # Lazy scan
-    .filter(
-        (pl.col('amount') > 1000) &
-        (pl.col('status') == 'active') &
-        (pl.col('date') >= '2024-01-01')
-    )
-    .sort('amount', descending=True)
-    .head(100)
-    .collect()  # Execute optimized plan
-)
-
-print('**Filtered and Sorted Results**')
-print(result)
-print(f'\\nFound {len(result)} matching rows')
-"
-```
-
-## Template 3: Hybrid Approach (Best of Both)
-
-**Use when:**
-- Need joins AND heavy filtering
-- Complex SQL followed by transformations
-- Optimize different operation stages
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow python -c "
-import duckdb
-import polars as pl
-
-print('Phase 1: DuckDB for complex join (3x faster)')
-# DuckDB excels at joins
-joined = duckdb.sql('''
-    SELECT
-        o.order_id,
-        o.amount,
-        c.customer_id,
-        c.region,
-        p.category
-    FROM 'orders.csv' o
-    JOIN 'customers.csv' c ON o.customer_id = c.customer_id
-    JOIN 'products.csv' p ON o.product_id = p.product_id
-    WHERE o.date >= '2024-01-01'
-''').pl()  # Convert to Polars
-
-print(f'Joined {len(joined):,} rows')
-
-print('\\nPhase 2: Polars for ultra-fast filtering (128x faster)')
-# Polars excels at filtering
-filtered = (
-    joined
-    .filter(
-        (pl.col('amount') > 100) &
-        (pl.col('region').is_in(['North', 'South', 'East']))
-    )
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('amount_with_tax')
-    ])
-)
-
-print(f'Filtered to {len(filtered):,} rows')
-
-print('\\nPhase 3: DuckDB for final aggregation (4x faster)')
-# Back to DuckDB for aggregation
-duckdb.register('filtered_data', filtered)
-final = duckdb.sql('''
-    SELECT
-        region,
-        category,
-        COUNT(DISTINCT customer_id) as customers,
-        SUM(amount_with_tax) as total_revenue,
-        AVG(amount_with_tax) as avg_transaction
-    FROM filtered_data
-    GROUP BY region, category
-    HAVING total_revenue > 10000
-    ORDER BY total_revenue DESC
-''').pl()
-
-print('\\n**Final Results**')
-print(final)
-"
-```
-
-## Template 4: Polars Streaming for Large Files
-
-**Use when:**
-- Dataset larger than available RAM
-- Need to process data in batches
-- Memory constraints
-
-```python
-uv run --with numpy --with polars python -c "
-import polars as pl
-
-# Streaming mode - processes data in chunks
-result = (
-    pl.scan_csv('huge_file.csv')
-    .filter(pl.col('status') == 'active')
-    .with_columns([
-        (pl.col('amount') * 1.1).alias('adjusted_amount')
-    ])
-    .groupby('category')
-    .agg([
-        pl.sum('adjusted_amount').alias('total'),
-        pl.count().alias('count')
-    ])
-    .collect(streaming=True)  # Streaming mode for large data
-)
-
-print('**Streaming Results**')
-print(result)
-print(f'\\nProcessed {result[\"count\"].sum():,} total rows')
-"
-```
-
-## Template 5: Visualization with Matplotlib
-
-**Use when:**
-- User requests charts, graphs, or plots
-- Exploratory data analysis (EDA)
-- Time-series or distribution analysis
-
-```python
-uv run --with numpy --with duckdb --with polars --with pyarrow --with matplotlib python -c "
-import duckdb
-import matplotlib.pyplot as plt
-
-# Query data
-result = duckdb.sql('''
-    SELECT
-        date,
-        SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY date
-    ORDER BY date
-''').pl()
-
-# Create visualization
-plt.figure(figsize=(10, 6))
-plt.plot(result['date'], result['total'])
-plt.xlabel('Date')
-plt.ylabel('Total Amount')
-plt.title('Daily Total Trends')
-plt.xticks(rotation=45)
-plt.tight_layout()
-plt.savefig('output.png')
-print('Chart saved to output.png')
-"
-```

package/packages/omo-codex/plugin/skills/data-scientist/references/integration-patterns.md

@@ -1,153 +0,0 @@
-# Integration Patterns: DuckDB ↔ Polars
-
-## Zero-Copy Conversions (FASTEST)
-
-### DuckDB → Polars (Recommended)
-
-```python
-import duckdb
-import polars as pl
-
-# Direct conversion with .pl() - zero-copy via Arrow
-df_polars = duckdb.sql("""
-    SELECT * FROM 'data.parquet'
-    WHERE amount > 100
-""").pl()  # Returns Polars DataFrame directly
-
-# Lazy version for large datasets
-lazy_df = duckdb.sql("SELECT * FROM 'data.parquet'").pl(lazy=True)
-result = lazy_df.filter(pl.col('status') == 'active').collect()
-```
-
-### Polars → DuckDB (Direct Reference)
-
-```python
-import duckdb
-import polars as pl
-
-# DuckDB can query Polars DataFrames directly by name
-df = pl.read_parquet('data.parquet')
-
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM df
-    GROUP BY category
-    ORDER BY total DESC
-""").pl()  # Query df directly, return as Polars
-```
-
-### Via Arrow (When Needed)
-
-```python
-# Polars → Arrow → DuckDB
-df_polars = pl.read_csv('data.csv')
-duckdb.register('my_table', df_polars.to_arrow())
-
-# DuckDB → Arrow → Polars
-arrow_table = duckdb.sql("SELECT * FROM data").arrow()
-df_polars = pl.from_arrow(arrow_table)
-```
-
-## WRONG vs RIGHT Patterns
-
-### ❌ NEVER - Using Pandas
-
-```python
-# FORBIDDEN - decisively slower than Polars/DuckDB on every operation!
-import pandas as pd
-df = pd.read_csv('data.csv')
-result = df.groupby('category')['amount'].sum()
-```
-
-### ✅ CORRECT - DuckDB for simple aggregation query
-
-```python
-import duckdb
-# Direct file query - no memory loading!
-result = duckdb.sql("""
-    SELECT category, SUM(amount) as total
-    FROM 'data.csv'
-    GROUP BY category
-""").pl()  # Fast, memory-efficient
-```
-
-### ❌ NEVER - Loading file before DuckDB query
-
-```python
-# WRONG - Unnecessary memory usage
-import polars as pl
-import duckdb
-df = pl.read_csv('data.csv')  # Loads entire file
-result = duckdb.sql("SELECT * FROM df WHERE amount > 100").pl()
-```
-
-### ✅ CORRECT - Let DuckDB query directly
-
-```python
-import duckdb
-# DuckDB queries file directly - much faster!
-result = duckdb.sql("""
-    SELECT * FROM 'data.csv'
-    WHERE amount > 100
-""").pl()
-```
-
-### ❌ NEVER - Eager evaluation in Polars
-
-```python
-# WRONG - Loads everything immediately
-import polars as pl
-df = pl.read_csv('large_data.csv')  # Eager load
-filtered = df.filter(pl.col('value') > 100)
-```
-
-### ✅ CORRECT - Lazy evaluation
-
-```python
-import polars as pl
-# Lazy - builds query plan, optimizes, executes once
-df = pl.scan_csv('large_data.csv')  # Lazy
-result = (
-    df
-    .filter(pl.col('value') > 100)
-    .groupby('category')
-    .agg(pl.sum('value'))
-    .collect()  # Execute optimized plan
-)
-```
-
-### ❌ NEVER - Unnecessary Conversions
-
-```python
-# WASTEFUL (DuckDB → Pandas → Polars)
-import duckdb, pandas as pd, polars as pl
-df_pd = duckdb.sql("SELECT * FROM 'data.csv'").df()  # requires pandas - the skill never ships it
-df_pl = pl.from_pandas(df_pd)
-```
-
-### ✅ CORRECT - Direct conversion
-
-```python
-# DIRECT (DuckDB → Polars via Arrow)
-import duckdb
-df_pl = duckdb.sql("SELECT * FROM 'data.csv'").pl()
-```
-
-### ❌ NEVER - Wrong tool for heavy filtering
-
-```python
-# SLOW (DuckDB not optimal for filtering)
-import duckdb
-result = duckdb.sql("""
-    SELECT * FROM 'huge.csv'
-    WHERE complex_filter = true
-""").pl()
-```
-
-### ✅ CORRECT - Use Polars for filtering
-
-```python
-# FAST (Polars 128x faster for filtering)
-import polars as pl
-result = pl.scan_csv('huge.csv').filter(pl.col('complex_filter')).collect()
-```

package/packages/omo-codex/plugin/skills/data-scientist/references/performance-benchmarks.md

@@ -1,37 +0,0 @@
-# Performance Benchmarks
-
-Routing heuristics distilled from public 2024-2025 results in the [db-benchmark suite](https://github.com/h2oai/db-benchmark) (rendered at [h2oai.github.io/db-benchmark](https://h2oai.github.io/db-benchmark/)) and from the vendors' own documentation ([duckdb.org](https://duckdb.org/), [pola.rs](https://pola.rs/)).
-
-**Read these as routing guidance, not guarantees.** Multipliers vary with dataset size, cardinality, data types, and hardware. When the choice materially matters, measure on the actual data.
-
-## Operation Performance Comparison
-
-| Operation Type      | Winner      | Indicative advantage | When to Use                                    |
-|---------------------|-------------|----------------------|------------------------------------------------|
-| **Filtering**       | **Polars**  | often the fastest by a wide margin (SIMD, predicate pushdown) | Single-table filters |
-| **Sorting**         | **Polars**  | typically the fastest | ORDER BY operations, ranking                   |
-| **Joins**           | **DuckDB**  | typically faster; richer join types | Multi-table joins, especially complex joins    |
-| **Aggregations**    | **DuckDB**  | typically faster on large datasets | GROUP BY, complex aggregations                 |
-| **Window Funcs**    | **Polars**  | typically faster | RANK, LAG/LEAD, running totals                 |
-| **Transformations** | **Polars**  | typically faster | Pivot, melt, string operations                 |
-| **Direct Query**    | **DuckDB**  | avoids loading to memory entirely | Ad-hoc exploration without loading to memory   |
-| **Streaming**       | **Polars**  | handles datasets larger than RAM | Datasets larger than RAM                       |
-
-Both tools are decisively faster than pandas on 1M+ row workloads, which is why pandas is banned outright in this skill.
-
-## Operation Detection Keywords
-
-Automatically detect operation types from user requests:
-
-- **Filter**: "where", "filter", "condition", "select rows", "find"
-- **Sort**: "sort", "order", "top", "bottom", "rank"
-- **Join**: "join", "merge", "combine tables"
-- **Aggregate**: "group", "sum", "avg", "count", "mean", "total", "aggregate"
-- **Transform**: "pivot", "melt", "reshape", "string operations", "clean", "transform"
-- **Window**: "running", "cumulative", "lag", "lead", "rank", "partition"
-
-## Benchmarking Notes
-
-- Public suites exercise typical analytical workloads (1M-100M rows).
-- Performance advantages are approximate and vary by dataset characteristics.
-- Real-world performance depends on data types, cardinality, and hardware.

package/packages/omo-codex/plugin/skills/debugging/references/tools/playwright-cli.md

@@ -1,194 +0,0 @@
-# Playwright CLI — Browser QA That Actually Drives a Browser
-
-**https://playwright.dev/ · https://github.com/microsoft/playwright**
-
-For any browser-served web UI bug, this is the correct tool. Not curl. Not imagination. Not a headless HTTP library. A real browser with a real rendering engine, real JS execution, real cookies, real service workers, real viewport.
-
-**In Phase 8 Manual QA for browser products, using Playwright is not optional.** Curl cannot catch: CSS that breaks at specific viewport widths, hydration mismatches, client-side router bugs, cookie/session interactions, service-worker caching, JS-triggered navigations. All of those are common bug classes. Drive a browser.
-
-> Note: `microsoft/playwright-cli` is the legacy repo; the current tooling lives in `@playwright/test` (npm) and `playwright` (pip), which include the `playwright` CLI. Use those — the legacy `playwright-cli` package is deprecated.
-
----
-
-## When to reach for Playwright
-
-| Bug symptom | Use Playwright? |
-|---|---|
-| Form submit produces wrong result | ✅ — Playwright drives the form exactly as a user does |
-| Page blank in prod, fine locally | ✅ — hydration/env differences need a real browser |
-| CSS looks wrong at a specific width | ✅ — use `--viewport-size` |
-| Click doesn't fire / wrong handler | ✅ — Playwright fires real DOM events |
-| Flash of unstyled content / loading glitch | ✅ — use trace viewer to see frames |
-| API returns wrong data | ❌ — use curl, this isn't a browser bug |
-| Backend returns wrong status code | ❌ — use curl |
-| Client hits a URL that returns 500 | ✅ but also ❌ — Playwright shows the call + response + failure effect on UI |
-
----
-
-## Install (per-project)
-
-Playwright installs browser binaries separately from the npm package.
-
-```bash
-# In the project
-npm init playwright@latest             # interactive; picks TS/JS + browsers + config
-# Or if Playwright is already a dep:
-npx playwright install                 # downloads browsers
-npx playwright install chromium        # just chromium
-npx playwright install --with-deps     # also installs OS deps (Linux)
-```
-
-Python:
-```bash
-pip install playwright
-playwright install
-```
-
----
-
-## The four things you'll actually use
-
-### 1. `codegen` — record a session, generate the script
-
-The fastest way to create a repro. Opens a real browser; your clicks / typing become a Playwright script you can paste into a test.
-
-```bash
-npx playwright codegen https://your-app.local
-npx playwright codegen --viewport-size=375,667 https://your-app.local    # iPhone SE size
-npx playwright codegen --device="iPhone 14" https://your-app.local
-```
-
-Click / type / navigate in the browser; watch the script build in the side panel. Copy the generated script into your journal as the repro for Phase 8.
-
-### 2. A one-shot Playwright script — reproduce + capture
-
-Usually the Phase 8 QA artifact. Save to `/tmp/debug-repro.spec.ts` (journal it):
-
-```ts
-// /tmp/debug-repro.spec.ts
-import { test, expect } from '@playwright/test';
-
-test('refinement chat shows non-empty response when env var set', async ({ page }) => {
-  await page.goto('http://localhost:3000/chat');
-  await page.fill('textarea[name="message"]', 'Add a logging step');
-  await page.click('button[type=submit]');
-
-  // Wait for the response to appear (not just the spinner to disappear)
-  const response = page.locator('[data-testid="assistant-reply"]');
-  await expect(response).toBeVisible({ timeout: 30_000 });
-  await expect(response).not.toBeEmpty();
-
-  // Capture evidence
-  await page.screenshot({ path: '/tmp/debug-after-fix.png', fullPage: true });
-  console.log(await response.textContent());
-});
-```
-
-Run it with tracing enabled for rich post-mortem:
-
-```bash
-npx playwright test /tmp/debug-repro.spec.ts --trace on --headed
-```
-
-### 3. `PWDEBUG=1` — step through the script with Playwright Inspector
-
-```bash
-PWDEBUG=1 npx playwright test /tmp/debug-repro.spec.ts
-```
-
-Opens the Playwright Inspector alongside the browser. You can step through Playwright actions, see the DOM state at each step, and edit selectors on the fly.
-
-Use this when the script doesn't reproduce cleanly and you need to watch it run.
-
-### 4. `show-trace` — post-mortem on a failed run
-
-```bash
-npx playwright show-trace trace.zip
-# or from the test-results dir:
-npx playwright show-trace test-results/<test-name>/trace.zip
-```
-
-Scrubs through a recorded session: timeline, DOM snapshot at each action, network, console, source. When a test failed on CI but passed locally, this is the single best artifact.
-
----
-
-## Headless vs headed during debugging
-
-Always add `--headed` when debugging. Headless browsers sometimes behave subtly differently (font rendering, viewport, media permissions). For QA evidence, run headed and screenshot.
-
-```bash
-npx playwright test --headed
-npx playwright test --headed --project=chromium     # pin the browser
-```
-
----
-
-## Catching the silent-failure patterns Playwright is good at
-
-```ts
-// Toast that flashes and disappears
-page.on('console', msg => console.log('[browser console]', msg.type(), msg.text()));
-
-// Unhandled page errors (uncaught exceptions in the page JS)
-page.on('pageerror', err => console.error('[page error]', err));
-
-// Network failures — e.g., backend returned 500 but UI shows nothing
-page.on('response', async resp => {
-  if (!resp.ok()) {
-    console.warn(`[network ${resp.status()}] ${resp.url()} — ${await resp.text()}`);
-  }
-});
-
-// Request that never came back
-page.on('requestfailed', req => {
-  console.error('[request failed]', req.url(), req.failure()?.errorText);
-});
-```
-
-Add these listeners to the top of the debug script. They surface a lot of the "UI showed nothing" class of bug.
-
----
-
-## Viewport and device emulation
-
-CSS bugs that only appear at specific sizes, or layout bugs on mobile:
-
-```ts
-// At test level
-test.use({ viewport: { width: 375, height: 667 } });
-
-// Per-page
-await page.setViewportSize({ width: 375, height: 667 });
-
-// Predefined devices
-import { devices } from '@playwright/test';
-test.use({ ...devices['iPhone 14'] });
-```
-
----
-
-## Gotchas
-
-- **Wait for state, not for time.** `await page.waitForTimeout(2000)` is flaky. Use `await expect(locator).toBeVisible()` or `page.waitForResponse(urlPattern)`.
-- **Stale selectors re-resolve.** Playwright's locators re-find the element on each action, unlike Puppeteer's handles. Don't over-think it.
-- **Service workers persist across test runs in headed mode.** If you see cached behavior from a previous run, add `await context.clearCookies()` + clear storage before the test.
-- **Installing on CI requires `--with-deps`** on Linux images that lack the browser's shared-library deps.
-- **Parallel tests share a browser process by default**; if one test polls a debugger port, others may interfere. Use `workers: 1` for debugging.
-
----
-
-## Phase 9 cleanup specifics
-
-```bash
-# Remove trace files from debug runs
-rm -rf playwright-report/ test-results/ trace.zip
-
-# Remove debug spec files from /tmp
-rm -f /tmp/debug-*.spec.ts
-
-# Remove screenshot captures
-rm -f /tmp/debug-*.png
-
-# If you installed browsers just for this session (rare):
-# Don't remove them — they're useful for future sessions. They live in ~/Library/Caches/ms-playwright (macOS) or ~/.cache/ms-playwright (Linux).
-```