ima-claude 2.9.0

package/plugins/ima-claude/skills/py-fp/references/core-principles.md

@@ -0,0 +1,381 @@
+# Python FP Core Principles (Deep Dive)
+
+Comprehensive FP philosophy and patterns for Python. Load this when explaining WHY, making architectural decisions, or need detailed pattern guidance.
+
+## The Pythonic FP Philosophy
+
+Python's creator Guido van Rossum has been explicit: Python is not a functional language. It borrows FP concepts selectively. The key insight is working WITH this design, not against it.
+
+**Pythonic FP means**: Use FP concepts (purity, immutability, composition, higher-order functions) but express them in Pythonic syntax. Don't import Haskell wholesale.
+
+### What to Adopt
+
+| Pattern | Python Expression |
+|---------|-------------------|
+| Pure functions | Regular functions with no side effects |
+| Immutable records | `@dataclass(frozen=True)` or `NamedTuple` |
+| Composition | Direct function calls, `pipe()` for pandas |
+| Higher-order functions | Comprehensions, `functools`, `itertools` |
+| Lazy evaluation | Generators and generator expressions |
+| Partial application | `functools.partial` |
+| Memoization | `@lru_cache` / `@cache` |
+| Pattern matching | `match`/`case` (3.10+) |
+
+### What to Avoid
+
+| Pattern | Why Not in Python |
+|---------|-------------------|
+| Custom pipe/compose | Adds indirection, not Pythonic |
+| Custom curry | `partial` covers the need |
+| Custom monads | Fight the language, hard to read |
+| Point-free style | Python doesn't support it well |
+| Deep recursion | No TCO, 1000-call limit |
+| Overloading `__or__` for pipes | Clever but confusing |
+
+## Functional Core, Imperative Shell
+
+The single highest-leverage FP technique for Python. Originated from Gary Bernhardt's "Boundaries" talk.
+
+### Architecture
+
+```
+┌─────────────────────────────────────┐
+│          Imperative Shell           │
+│  (I/O, side effects, orchestration) │
+│                                     │
+│  ┌─────────────────────────────┐    │
+│  │      Functional Core        │    │
+│  │  (pure business logic)      │    │
+│  │  - calculations             │    │
+│  │  - validations              │    │
+│  │  - transformations          │    │
+│  │  - business rules           │    │
+│  └─────────────────────────────┘    │
+│                                     │
+│  Database, APIs, files, logging     │
+└─────────────────────────────────────┘
+```
+
+### Example
+
+```python
+# === PURE CORE ===
+
+def calculate_order_total(items: list[dict], tax_rate: float) -> float:
+    subtotal = sum(item["price"] * item["quantity"] for item in items)
+    return round(subtotal * (1 + tax_rate), 2)
+
+def validate_order(order: dict) -> dict:
+    if not order.get("items"):
+        return {"success": False, "error": "Order must have items"}
+    if any(item["quantity"] <= 0 for item in order["items"]):
+        return {"success": False, "error": "Quantities must be positive"}
+    return {"success": True, "data": order}
+
+def apply_discount(total: float, discount_code: str, discount_table: dict) -> float:
+    rate = discount_table.get(discount_code, 0.0)
+    return round(total * (1 - rate), 2)
+
+
+# === IMPERATIVE SHELL ===
+
+def process_order(order: dict, db, payment_gateway, logger):
+    """Shell: orchestrates I/O around pure core."""
+    validated = validate_order(order)
+    if not validated["success"]:
+        return validated
+
+    # Pure calculations
+    tax_rate = db.get_tax_rate(order["region"])
+    discount_table = db.get_discount_table()
+    total = calculate_order_total(order["items"], tax_rate)
+    final = apply_discount(total, order.get("discount_code", ""), discount_table)
+
+    # Side effects
+    charge_result = payment_gateway.charge(order["customer_id"], final)
+    if not charge_result["success"]:
+        return charge_result
+
+    db.save_order({**order, "total": final, "status": "paid"})
+    logger.info(f"Order processed: {final}")
+    return {"success": True, "data": {"total": final}}
+```
+
+**The ratio to aim for**: 80% pure core, 20% impure shell.
+
+## Immutability Deep Dive
+
+### The Immutability Spectrum in Python
+
+From most to least immutable:
+
+1. **Truly immutable**: `int`, `float`, `str`, `bytes`, `tuple`, `frozenset`
+2. **Shallow frozen**: `@dataclass(frozen=True)`, `NamedTuple`
+3. **Read-only views**: `types.MappingProxyType`
+4. **Convention-only**: Regular objects you choose not to mutate
+5. **Deeply immutable (library)**: `pyrsistent.PMap`, `pyrsistent.PVector`
+
+### Frozen Dataclasses — The Default Choice
+
+```python
+from dataclasses import dataclass, field, replace
+
+@dataclass(frozen=True)
+class Order:
+    customer_id: int
+    items: tuple[dict, ...]  # Use tuple, not list, for deep immutability
+    status: str = "pending"
+    metadata: dict = field(default_factory=dict)
+
+# Create
+order = Order(customer_id=1, items=({"sku": "A", "qty": 2},))
+
+# "Update" — returns new instance
+paid_order = replace(order, status="paid")
+
+# Original is untouched
+assert order.status == "pending"
+assert paid_order.status == "paid"
+```
+
+### When to Use pyrsistent
+
+Only when you need efficient structural sharing for large nested data:
+
+```python
+from pyrsistent import pmap, pvector, freeze, thaw
+
+# Convert mutable to immutable
+config = freeze({"db": {"host": "localhost", "port": 5432}, "debug": True})
+
+# Efficient "updates" via structural sharing
+new_config = config.set("debug", False)
+# config is unchanged, new_config shares the "db" subtree
+
+# Convert back when needed for I/O
+mutable = thaw(new_config)
+```
+
+**Don't reach for pyrsistent** for simple cases — `frozen=True` dataclasses with `replace()` cover 90% of needs.
+
+## Result Type Patterns
+
+### The Standard Shape
+
+```python
+from typing import TypeVar, Any
+
+T = TypeVar("T")
+
+def success(data: Any = None) -> dict:
+    return {"success": True, "data": data}
+
+def failure(error: str) -> dict:
+    return {"success": False, "error": error}
+```
+
+### Chaining Results
+
+```python
+def chain_results(*steps):
+    """Apply steps sequentially, short-circuiting on failure."""
+    def run(data):
+        current = data
+        for step in steps:
+            result = step(current)
+            if not result["success"]:
+                return result
+            current = result["data"]
+        return success(current)
+    return run
+
+# Usage
+process = chain_results(validate, transform, enrich)
+result = process(raw_data)
+```
+
+### Typed Results (For Larger Codebases)
+
+```python
+from dataclasses import dataclass
+from typing import TypeVar, Generic, Union
+
+T = TypeVar("T")
+
+@dataclass(frozen=True)
+class Success(Generic[T]):
+    data: T
+
+@dataclass(frozen=True)
+class Failure:
+    error: str
+
+Result = Union[Success[T], Failure]
+
+def divide(a: float, b: float) -> Result[float]:
+    if b == 0:
+        return Failure("Division by zero")
+    return Success(a / b)
+
+# Pattern matching with results (3.10+)
+match divide(10, 3):
+    case Success(data=value):
+        print(f"Result: {value}")
+    case Failure(error=msg):
+        print(f"Error: {msg}")
+```
+
+## Higher-Order Functions Done Right
+
+### Comprehensions Are Your map/filter
+
+```python
+# These are equivalent, but comprehensions are Pythonic:
+list(map(str.upper, names))        # Haskell-ish
+[name.upper() for name in names]   # Pythonic
+
+list(filter(lambda x: x > 0, nums))   # Haskell-ish
+[x for x in nums if x > 0]            # Pythonic
+
+# Nested transformations
+{
+    dept: [e["name"] for e in employees if e["active"]]
+    for dept, employees in grouped.items()
+}
+```
+
+### When map/filter ARE Appropriate
+
+```python
+# When you have a named function already — no lambda needed
+clean_lines = list(map(str.strip, lines))
+valid_emails = list(filter(is_valid_email, emails))
+
+# With operator module — cleaner than lambda
+from operator import itemgetter
+sorted_users = sorted(users, key=itemgetter("last_name", "first_name"))
+```
+
+### functools.reduce — Use Sparingly
+
+```python
+from functools import reduce
+from operator import mul
+
+# Good: simple accumulation with named operator
+factorial = reduce(mul, range(1, n + 1), 1)
+
+# Bad: complex reduce is hard to read
+# Use a loop or comprehension instead
+result = reduce(
+    lambda acc, x: {**acc, x["key"]: x["value"]},  # Unreadable
+    items,
+    {}
+)
+
+# Better: dict comprehension
+result = {item["key"]: item["value"] for item in items}
+```
+
+## Error Handling Philosophy
+
+### Exceptions vs Result Types
+
+**Use exceptions for**:
+- Truly exceptional conditions (out of memory, disk full)
+- Programming errors (wrong type, missing key)
+- Framework integration (Django, FastAPI expect exceptions)
+
+**Use result types for**:
+- Expected failures (validation errors, not found, business rule violations)
+- Operations that callers should handle explicitly
+- Pipeline/chain processing where failures are data
+
+```python
+# Exceptions for programming errors
+def process(data: list[dict]) -> list[dict]:
+    if not isinstance(data, list):
+        raise TypeError(f"Expected list, got {type(data)}")
+    return [transform(item) for item in data]
+
+# Result types for business logic
+def withdraw(account: dict, amount: float) -> dict:
+    if amount <= 0:
+        return failure("Amount must be positive")
+    if amount > account["balance"]:
+        return failure("Insufficient funds")
+    return success({**account, "balance": account["balance"] - amount})
+```
+
+## Established Libraries Reference
+
+### When to Pull In a Library
+
+| Need | Library | Justification |
+|------|---------|---------------|
+| Heavy function composition | `toolz` / `cytoolz` | Battle-tested, Cython performance |
+| Persistent data structures | `pyrsistent` | Structural sharing for large data |
+| Extended iterators | `more-itertools` | Hundreds of useful utilities |
+| Typed error handling | `returns` (dry-python) | Only if team commits to it |
+| F#-style pipelines | `expression` | Lighter than `returns` |
+
+### toolz Quick Reference
+
+```python
+from toolz import pipe, groupby, valmap, merge
+
+# pipe is acceptable FROM toolz — don't build your own
+result = pipe(
+    data,
+    clean,
+    validate,
+    transform,
+)
+
+# Useful dict operations
+grouped = groupby("category", items)
+totals = valmap(lambda items: sum(i["price"] for i in items), grouped)
+merged = merge(defaults, overrides)
+```
+
+**Rule**: Using `toolz.pipe` is fine. Building your own `pipe` is not. The distinction is between using established, tested tools and reinventing them.
+
+## Concurrency and Parallelism
+
+### Pure Functions Enable Easy Parallelism
+
+```python
+from multiprocessing import Pool
+from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
+
+# CPU-bound: multiprocessing (pure functions serialize safely)
+def compute_score(item: dict) -> dict:
+    return {**item, "score": heavy_computation(item["data"])}
+
+with ProcessPoolExecutor(max_workers=4) as executor:
+    results = list(executor.map(compute_score, items))
+
+# I/O-bound: threading (even with GIL, I/O releases it)
+def fetch_data(url: str) -> dict:
+    response = requests.get(url)
+    return {"url": url, "data": response.json()}
+
+with ThreadPoolExecutor(max_workers=10) as executor:
+    results = list(executor.map(fetch_data, urls))
+```
+
+### async/await — The Shell Layer
+
+```python
+import asyncio
+
+# Pure core — same as synchronous
+def process_response(data: dict) -> dict:
+    return {**data, "processed": True, "score": compute_score(data)}
+
+# Async shell — handles I/O
+async def fetch_and_process(session, url: str) -> dict:
+    async with session.get(url) as response:
+        data = await response.json()
+    return process_response(data)  # Pure function, no await needed
+```

package/plugins/ima-claude/skills/py-fp/references/testing-patterns.md

@@ -0,0 +1,283 @@
+# Python FP Testing Patterns (Deep Dive)
+
+Comprehensive testing strategies for functional Python code using pytest. Load this when building test suites, improving coverage, or need testing guidance.
+
+## The FP Testing Advantage
+
+Pure functions are trivially testable:
+- No setup/teardown ceremony
+- No mocking infrastructure
+- All edge cases enumerable
+- Parametrized testing is natural
+
+## pytest Fundamentals for FP
+
+### Parametrized Tests — The Core Pattern
+
+```python
+import pytest
+
+@pytest.mark.parametrize("input_val,expected", [
+    ("hello@example.com", True),
+    ("invalid", False),
+    ("", False),
+    ("a@b.c", True),
+    ("@no-local.com", False),
+    ("no-domain@", False),
+])
+def test_validate_email(input_val, expected):
+    assert validate_email(input_val) == expected
+```
+
+### Grouped Parametrize for Complex Cases
+
+```python
+@pytest.mark.parametrize("items,tax_rate,expected", [
+    # Happy path
+    ([{"price": 10, "qty": 2}, {"price": 5, "qty": 1}], 0.1, 27.50),
+    # Empty order
+    ([], 0.1, 0.0),
+    # Zero tax
+    ([{"price": 100, "qty": 1}], 0.0, 100.0),
+    # High precision
+    ([{"price": 9.99, "qty": 3}], 0.0825, 32.44),
+])
+def test_calculate_order_total(items, tax_rate, expected):
+    assert calculate_order_total(items, tax_rate) == expected
+```
+
+### Testing Result Types
+
+```python
+def test_withdraw_success():
+    account = {"id": 1, "balance": 100.0}
+    result = withdraw(account, 30.0)
+    assert result["success"] is True
+    assert result["data"]["balance"] == 70.0
+
+def test_withdraw_insufficient_funds():
+    account = {"id": 1, "balance": 10.0}
+    result = withdraw(account, 50.0)
+    assert result["success"] is False
+    assert "insufficient" in result["error"].lower()
+
+@pytest.mark.parametrize("amount,should_succeed", [
+    (50.0, True),
+    (100.0, True),    # Exact balance
+    (100.01, False),  # Just over
+    (0.0, False),     # Zero
+    (-10.0, False),   # Negative
+])
+def test_withdraw_boundary_cases(amount, should_succeed):
+    account = {"id": 1, "balance": 100.0}
+    result = withdraw(account, amount)
+    assert result["success"] is should_succeed
+```
+
+## Testing Immutability
+
+```python
+from dataclasses import FrozenInstanceError
+
+def test_frozen_dataclass_prevents_mutation():
+    user = User(name="Alice", email="alice@test.com")
+    with pytest.raises(FrozenInstanceError):
+        user.name = "Bob"
+
+def test_replace_returns_new_instance():
+    original = User(name="Alice", email="alice@test.com")
+    updated = replace(original, email="new@test.com")
+
+    assert updated.email == "new@test.com"
+    assert original.email == "alice@test.com"  # Unchanged
+    assert original is not updated  # Different objects
+
+def test_dict_immutability_pattern():
+    original = {"name": "Alice", "age": 30}
+    updated = {**original, "age": 31}
+
+    assert updated["age"] == 31
+    assert original["age"] == 30  # Unchanged
+```
+
+## Testing Generators and Lazy Pipelines
+
+```python
+def test_generator_pipeline():
+    data = ["  Alice  ", "", "  Bob  ", "  ", "  Carol  "]
+
+    def clean(lines):
+        for line in lines:
+            stripped = line.strip()
+            if stripped:
+                yield stripped
+
+    def upper(lines):
+        for line in lines:
+            yield line.upper()
+
+    result = list(upper(clean(data)))
+    assert result == ["ALICE", "BOB", "CAROL"]
+
+def test_generator_is_lazy():
+    call_count = 0
+
+    def counting_transform(items):
+        nonlocal call_count
+        for item in items:
+            call_count += 1
+            yield item * 2
+
+    gen = counting_transform([1, 2, 3, 4, 5])
+    assert call_count == 0  # Nothing executed yet
+
+    first = next(gen)
+    assert first == 2
+    assert call_count == 1  # Only one item processed
+```
+
+## Testing Data Pipelines (pandas)
+
+```python
+import pandas as pd
+import pytest
+
+@pytest.fixture
+def sample_df():
+    return pd.DataFrame({
+        "name": ["  Alice  ", "Bob", None, "Carol"],
+        "age": [25, 30, 35, 40],
+        "email": ["a@test.com", None, "c@test.com", "d@test.com"],
+    })
+
+def test_clean_nulls(sample_df):
+    result = clean_nulls(sample_df)
+    assert len(result) == 2  # Only rows with both name and email
+    assert result["name"].isna().sum() == 0
+    assert result["email"].isna().sum() == 0
+
+def test_normalize_names(sample_df):
+    result = normalize_names(sample_df.dropna(subset=["name"]))
+    assert list(result["name"]) == ["Alice", "Bob", "Carol"]
+
+def test_pipeline_composition(sample_df):
+    """Test the full pipeline produces expected shape."""
+    result = (
+        sample_df
+        .pipe(clean_nulls)
+        .pipe(normalize_names)
+    )
+    assert "name" in result.columns
+    assert result["name"].str.strip().equals(result["name"])  # No whitespace
+```
+
+## Property-Based Testing with Hypothesis
+
+```python
+from hypothesis import given, strategies as st
+
+# Pure functions enable property-based testing naturally
+@given(st.lists(st.floats(min_value=0, max_value=10000, allow_nan=False)))
+def test_calculate_total_non_negative(prices):
+    items = [{"price": p} for p in prices]
+    total = calculate_total(items)
+    assert total >= 0
+
+@given(st.text(), st.text())
+def test_concat_length(a, b):
+    result = concat(a, b)
+    assert len(result) == len(a) + len(b)
+
+@given(
+    st.dictionaries(st.text(min_size=1), st.integers()),
+    st.dictionaries(st.text(min_size=1), st.integers()),
+)
+def test_merge_dicts_contains_all_keys(d1, d2):
+    result = merge_dicts(d1, d2)
+    assert all(k in result for k in d1)
+    assert all(k in result for k in d2)
+
+# Roundtrip properties
+@given(st.builds(User, name=st.text(min_size=1), email=st.emails()))
+def test_user_serialization_roundtrip(user):
+    serialized = user_to_dict(user)
+    deserialized = user_from_dict(serialized)
+    assert deserialized == user
+```
+
+## Testing Side Effect Isolation
+
+```python
+def test_pure_core_without_mocking():
+    """Pure functions need no mocks — just pass data in, check data out."""
+    order = {"items": [{"price": 10, "qty": 2}], "discount_code": "SAVE10"}
+    discount_table = {"SAVE10": 0.10}
+
+    total = calculate_order_total(order["items"], tax_rate=0.0)
+    final = apply_discount(total, order["discount_code"], discount_table)
+
+    assert final == 18.0  # 20 - 10%
+
+def test_shell_with_minimal_mocking():
+    """Shell tests need mocks, but they're thin."""
+    mock_db = Mock()
+    mock_db.get_tax_rate.return_value = 0.0
+    mock_db.get_discount_table.return_value = {}
+    mock_payment = Mock()
+    mock_payment.charge.return_value = {"success": True}
+    mock_logger = Mock()
+
+    result = process_order(
+        {"items": [{"price": 10, "qty": 1}], "region": "US", "customer_id": 1},
+        db=mock_db,
+        payment_gateway=mock_payment,
+        logger=mock_logger,
+    )
+
+    assert result["success"]
+    mock_db.save_order.assert_called_once()
+```
+
+## Fixtures for FP Testing
+
+```python
+@pytest.fixture
+def make_user():
+    """Factory fixture — returns a function for creating test users."""
+    def _make(name="Test User", email="test@example.com", **overrides):
+        return User(name=name, email=email, **overrides)
+    return _make
+
+def test_update_email(make_user):
+    user = make_user(email="old@test.com")
+    updated = update_email(user, "new@test.com")
+    assert updated.email == "new@test.com"
+
+@pytest.fixture
+def sample_items():
+    """Reusable test data — immutable tuple."""
+    return (
+        {"id": 1, "name": "Widget", "price": 10.0, "category": "A"},
+        {"id": 2, "name": "Gadget", "price": 25.0, "category": "B"},
+        {"id": 3, "name": "Doohickey", "price": 5.0, "category": "A"},
+    )
+```
+
+## Test Organization
+
+```
+tests/
+  test_core/          # Pure function tests (no mocks)
+    test_calculations.py
+    test_validations.py
+    test_transformations.py
+  test_shell/         # Integration tests (minimal mocks)
+    test_order_service.py
+    test_api_handlers.py
+  test_pipelines/     # Data pipeline tests
+    test_etl.py
+    test_features.py
+  conftest.py         # Shared fixtures
+```
+
+**Key insight**: The pure core tests should be the majority (80%+) and run fast with no I/O. Shell tests are fewer and may need mocks or test databases.