cli-jaw 0.1.0

package/skills_ref/property-based-testing/skills/property-based-testing/references/libraries.md

@@ -0,0 +1,130 @@
+# PBT Libraries by Language
+
+## Quick Reference
+
+| Language | Library | Import/Setup |
+|----------|---------|--------------|
+| Python | Hypothesis | `from hypothesis import given, strategies as st` |
+| JavaScript/TypeScript | fast-check | `import fc from 'fast-check'` |
+| Rust | proptest | `use proptest::prelude::*` |
+| Go | rapid | `import "pgregory.net/rapid"` |
+| Java | jqwik | `@Property` annotations, `import net.jqwik.api.*` |
+| Scala | ScalaCheck | `import org.scalacheck._` |
+| C# | FsCheck | `using FsCheck; using FsCheck.Xunit;` |
+| Elixir | StreamData | `use ExUnitProperties` |
+| Haskell | QuickCheck | `import Test.QuickCheck` |
+| Clojure | test.check | `[clojure.test.check :as tc]` |
+| Ruby | PropCheck | `require 'prop_check'` |
+| Kotlin | Kotest | `io.kotest.property.*` |
+| Swift | SwiftCheck | `import SwiftCheck` ⚠️ unmaintained |
+| C++ | RapidCheck | `#include <rapidcheck.h>` |
+
+### Alternatives
+
+| Language | Alternative | Notes |
+|----------|-------------|-------|
+| Haskell | Hedgehog | Integrated shrinking, no type classes |
+| Rust | quickcheck | Simpler API, per-type shrinking |
+| Go | gopter | ScalaCheck-style, more explicit |
+
+## Smart Contract Testing (EVM/Solidity)
+
+| Tool | Type | Description |
+|------|------|-------------|
+| Echidna | Fuzzer | Property-based fuzzer for EVM contracts |
+| Medusa | Fuzzer | Next-gen fuzzer with parallel execution |
+
+```solidity
+// Echidna property example
+function echidna_balance_invariant() public returns (bool) {
+    return address(this).balance >= 0;
+}
+```
+
+**Installation**:
+```bash
+# Echidna (via crytic toolchain)
+pip install crytic-compile
+# Download binary from https://github.com/crytic/echidna
+
+# Medusa
+go install github.com/crytic/medusa@latest
+```
+
+See [secure-contracts.com](https://secure-contracts.com) for tutorials.
+
+## Installation
+
+**Python**:
+```bash
+pip install hypothesis
+```
+
+**JavaScript/TypeScript**:
+```bash
+npm install fast-check
+```
+
+**Rust** (add to Cargo.toml):
+```toml
+[dev-dependencies]
+proptest = "1.0"
+# or for quickcheck:
+quickcheck = "1.0"
+```
+
+**Go**:
+```bash
+go get pgregory.net/rapid
+# or for gopter:
+go get github.com/leanovate/gopter
+```
+
+**Java** (Maven):
+```xml
+<dependency>
+  <groupId>net.jqwik</groupId>
+  <artifactId>jqwik</artifactId>
+  <version>1.9.3</version>
+  <scope>test</scope>
+</dependency>
+```
+
+**Clojure** (deps.edn):
+```clojure
+{:deps {org.clojure/test.check {:mvn/version "1.1.2"}}}
+```
+
+**Haskell**:
+```bash
+cabal install QuickCheck
+# or for Hedgehog:
+cabal install hedgehog
+```
+
+## Detecting Existing Usage
+
+Search for PBT library imports in the codebase:
+
+```bash
+# Python
+rg "from hypothesis import" --type py
+
+# JavaScript/TypeScript
+rg "from 'fast-check'" --type js --type ts
+
+# Rust
+rg "use proptest" --type rust
+
+# Go
+rg "pgregory.net/rapid" --type go
+
+# Java
+rg "@Property" --type java
+
+# Clojure
+rg "test.check" --type clojure
+
+# Solidity (Echidna)
+rg "echidna_" --glob "*.sol"
+```

package/skills_ref/property-based-testing/skills/property-based-testing/references/refactoring.md

@@ -0,0 +1,181 @@
+# Refactoring for Property-Based Testing
+
+Identify code that could be refactored to enable or improve property-based testing.
+
+## Quick Reference
+
+| Pattern | Problem | Solution | Properties Enabled |
+|---------|---------|----------|-------------------|
+| I/O mixed with logic | Can't test without mocks | Extract pure core | Multiple |
+| Encode without decode | No roundtrip possible | Add inverse operation | Roundtrip |
+| Hardcoded config | Can't test edge cases | Inject dependencies | Full coverage |
+| In-place mutation | Hard to verify before/after | Return new value | Comparison properties |
+| String building | Can't verify structure | Structured + render | Roundtrip |
+| Implicit invariants | Can't test constraints | Make explicit with validation | Invariant |
+
+## Refactoring Patterns
+
+### 1. Extract Pure Core from Impure Functions (High Impact)
+
+**Pattern**: Functions that mix I/O with logic
+
+```python
+# BEFORE - hard to test
+def process_order(order_id: str) -> None:
+    order = db.fetch(order_id)           # I/O
+    discount = calculate_discount(order)  # Pure logic
+    total = apply_discount(order, discount)  # Pure logic
+    db.save(order_id, total)             # I/O
+
+# AFTER - pure core extracted
+def calculate_order_total(order: Order, rules: DiscountRules) -> Decimal:
+    """Pure function - easy to property test."""
+    discount = calculate_discount(order, rules)
+    return apply_discount(order, discount)
+
+def process_order(order_id: str) -> None:
+    """Thin I/O wrapper."""
+    order = db.fetch(order_id)
+    total = calculate_order_total(order, get_discount_rules())
+    db.save(order_id, total)
+```
+
+**Detection**: `rg "def \w+\(" -A 20 | grep -E "(open\(|db\.|requests\.|fetch|save)"`
+
+### 2. Add Missing Inverse Operations (High Impact)
+
+**Pattern**: One-way operations that should have pairs
+
+```python
+# BEFORE - only encode
+def encode_message(msg: dict) -> bytes:
+    return msgpack.packb(msg)
+
+# AFTER - add decode for roundtrip testing
+def encode_message(msg: dict) -> bytes:
+    return msgpack.packb(msg)
+
+def decode_message(data: bytes) -> dict:
+    return msgpack.unpackb(data)
+```
+
+**Detection**: Find encode without decode, serialize without deserialize
+
+### 3. Replace Hardcoded Dependencies (Medium Impact)
+
+**Pattern**: Functions using globals or hardcoded config
+
+```python
+# BEFORE
+def validate_input(data: str) -> bool:
+    return len(data) <= CONFIG.max_length
+
+# AFTER - dependencies injected
+def validate_input(data: str, max_length: int) -> bool:
+    return len(data) <= max_length
+```
+
+**Detection**: `rg "(CONFIG\.|SETTINGS\.|os\.environ)"`
+
+### 4. Return Values Instead of Mutating (Medium Impact)
+
+**Pattern**: Methods that mutate in place
+
+```python
+# BEFORE
+def sort_tasks(tasks: list[Task]) -> None:
+    tasks.sort(key=lambda t: t.priority)
+
+# AFTER - returns new list
+def sorted_tasks(tasks: list[Task]) -> list[Task]:
+    return sorted(tasks, key=lambda t: t.priority)
+```
+
+**Detection**: `rg "-> None:" -A 10 | grep -E "\.(sort|append|extend)"`
+
+### 5. Convert String Building to Structured + Render (Medium Impact)
+
+**Pattern**: Manual string concatenation
+
+```python
+# BEFORE
+def build_query(table: str, filters: dict) -> str:
+    q = f"SELECT * FROM {table}"
+    if filters:
+        q += " WHERE " + " AND ".join(...)
+    return q
+
+# AFTER - structured representation
+@dataclass
+class Query:
+    table: str
+    filters: dict
+
+def render_query(q: Query) -> str: ...
+def parse_query(sql: str) -> Query: ...  # Add inverse!
+```
+
+### 6. Add Validators/Generators for Predicates (Lower Impact)
+
+**Pattern**: `is_valid()` exists but no way to generate valid inputs
+
+```python
+# BEFORE
+def is_valid_email(s: str) -> bool:
+    return EMAIL_REGEX.match(s) is not None
+
+# AFTER - add generator
+@st.composite
+def valid_emails(draw):
+    local = draw(st.from_regex(r'[a-z][a-z0-9]{1,20}'))
+    domain = draw(st.sampled_from(['gmail.com', 'example.com']))
+    return f"{local}@{domain}"
+```
+
+**Detection**: `rg "def is_\w+\(" --type py`
+
+### 7. Make Implicit Invariants Explicit (Lower Impact)
+
+**Pattern**: Constraints in comments but not enforced
+
+```python
+# BEFORE - constraint only in docstring
+def allocate_buffer(size: int) -> bytes:
+    """Size must be positive and <= 1MB."""
+    return bytes(size)
+
+# AFTER - enforced
+MAX_BUFFER_SIZE = 1024 * 1024
+
+def allocate_buffer(size: int) -> bytes:
+    if not (0 < size <= MAX_BUFFER_SIZE):
+        raise ValueError(f"size must be in (0, {MAX_BUFFER_SIZE}]")
+    return bytes(size)
+```
+
+**Detection**: `rg "(must be|should be|always|never)" --type py`
+
+## Evaluation Criteria
+
+For each refactoring opportunity:
+
+| Factor | Questions |
+|--------|-----------|
+| Properties enabled | What tests become possible? Roundtrip > Idempotence > No crash |
+| Effort | Low/Medium/High - how much code change? |
+| Risk | Breaking changes? API impact? |
+| Backwards compatibility | Can old callers still work? |
+
+## Prioritization
+
+1. Strength of properties enabled (roundtrip > idempotence > no crash)
+2. Effort required (prefer low-effort wins)
+3. Risk level (prefer safe changes)
+
+## Red Flags
+
+- **Breaking the API without warning**: Flag breaking changes clearly and offer backwards-compatible alternatives
+- **Over-engineering**: Not every function needs to be perfectly testable - prioritize high-value code
+- **Ignoring existing tests**: Run existing tests after refactoring to verify behavior unchanged
+- **Missing the forest for the trees**: If a module needs wholesale restructuring, say so rather than suggesting 20 small changes
+- **Not considering effort vs value**: A complex refactoring enabling only "no crash" isn't worth it

package/skills_ref/property-based-testing/skills/property-based-testing/references/reviewing.md

@@ -0,0 +1,209 @@
+# Reviewing Property-Based Tests
+
+Evaluate quality of existing property-based tests and suggest improvements.
+
+## Quick Reference
+
+| Issue | Severity | Detection | Fix |
+|-------|----------|-----------|-----|
+| Tautological | CRITICAL | Assertion compares same expression | Rewrite with actual property |
+| Vacuous | CRITICAL | Contradictory `assume()` calls | Remove or fix filters |
+| Weak (no assertion) | HIGH | Test body has no assert | Add meaningful assertion |
+| Reimplementation | HIGH | Assertion mirrors function logic | Use algebraic property instead |
+| Over-filtered | MEDIUM | Many `assume()` calls | Redesign strategy |
+| Missing edge cases | MEDIUM | No `@example` decorators | Add explicit edge cases |
+| Poor settings | LOW | Missing or bad `@settings` | Add appropriate settings |
+
+## Quality Issues
+
+### Issue: Tautological Properties (CRITICAL)
+
+Properties that are always true regardless of implementation.
+
+```python
+# BAD - compares function to itself
+@given(st.lists(st.integers()))
+def test_sort_tautology(xs):
+    assert sorted(xs) == sorted(xs)  # Always true!
+
+# BAD - tests nothing about the function
+@given(st.integers())
+def test_useless(x):
+    result = compute(x)
+    assert result == result  # Always true!
+```
+
+**Detection**: Assertions comparing same expression, or not using function result meaningfully.
+
+### Issue: Vacuous Tests (CRITICAL)
+
+Tests where assumptions filter out most/all inputs.
+
+```python
+# VACUOUS - impossible condition
+@given(st.integers())
+def test_vacuous(x):
+    assume(x > 100)
+    assume(x < 50)  # Impossible!
+    assert compute(x) > 0
+
+# VACUOUS - overly restrictive
+@given(st.integers())
+def test_too_filtered(x):
+    assume(x == 42)  # Only tests one value!
+    assert compute(x) == expected
+```
+
+**Detection**: Multiple `assume()` calls, `assume` with very narrow conditions.
+
+### Issue: Weak Properties (HIGH)
+
+Properties that only test minimal guarantees.
+
+```python
+# WEAK - only tests no crash
+@given(st.text())
+def test_only_no_crash(s):
+    process(s)  # No assertion at all
+
+# WEAK - only tests type
+@given(st.integers())
+def test_only_type(x):
+    assert isinstance(compute(x), int)
+```
+
+**Detection**: Tests without assertions, or only `isinstance`/type checks.
+
+### Issue: Reimplementing the Function (HIGH)
+
+```python
+# BAD - just reimplements the logic
+@given(st.integers(), st.integers())
+def test_reimplements(a, b):
+    assert add(a, b) == a + b  # Tests nothing if add() is just a + b
+```
+
+**Detection**: Test assertion contains same logic as function under test.
+
+### Issue: Poor Input Coverage (MEDIUM)
+
+```python
+# NARROW - misses edge cases
+@given(st.integers(min_value=1, max_value=10))
+def test_narrow_range(x):
+    assert compute(x) >= 0  # What about 0? Negatives? Large values?
+
+# MISSING - no edge case examples
+@given(st.lists(st.integers()))
+def test_no_explicit_edges(xs):
+    # Should include @example([]) @example([1]) etc.
+    assert len(sort(xs)) == len(xs)
+```
+
+### Issue: Missing Stronger Properties (MEDIUM)
+
+```python
+# EXISTS - but could be stronger
+@given(st.lists(st.integers()))
+def test_sort_length(xs):
+    assert len(sort(xs)) == len(xs)
+# MISSING: ordering property, element preservation
+```
+
+### Issue: Poor Settings (LOW)
+
+```python
+# TOO FEW - may miss bugs
+@settings(max_examples=5)
+def test_few_examples(x): ...
+
+# NO DEADLINE - may hang in CI
+@given(expensive_strategy())
+def test_no_deadline(x): ...  # Could timeout
+```
+
+## Review Process
+
+### 1. Locate Property-Based Tests
+
+Search using library-specific patterns:
+
+**Python/Hypothesis:**
+```bash
+rg "@given\(" --type py
+rg "from hypothesis import" --type py
+```
+
+**JavaScript/fast-check:**
+```bash
+rg "fc\.(assert|property)" --type js --type ts
+```
+
+**Rust/proptest:**
+```bash
+rg "proptest!" --type rust
+```
+
+### 2. Analyze Each Test
+
+Check for issues above, starting with critical then high severity.
+
+### 3. Evaluate Shrinking Quality
+
+Will tests shrink to minimal counterexamples? Complex strategies may produce hard-to-debug failures.
+
+### 4. Check for Flakiness Potential
+
+- Non-determinism in code under test
+- Time-dependent assertions
+- Global state dependencies
+- Floating point comparisons without tolerance
+
+### 5. Suggest Stronger Properties
+
+Compare against property catalog - are stronger properties available but not tested?
+
+## Test Health Score
+
+| Category | Score | What to Check |
+|----------|-------|---------------|
+| Property Strength | X/5 | Roundtrip > Idempotence > Type > No crash |
+| Input Coverage | X/5 | Edge cases, strategy breadth |
+| Assertions | X/5 | Meaningful, not tautological |
+| Settings | X/5 | Appropriate for context |
+
+## Mutation Testing Verification
+
+Suggest specific mutations to verify tests catch bugs:
+
+```
+To verify test_sort catches bugs:
+
+1. Return input unchanged: `return xs`
+   - Should fail: test_ordering
+
+2. Drop last element: `return sorted(xs)[:-1]`
+   - Should fail: test_length_preserved
+
+3. Reverse order: `return sorted(xs, reverse=True)`
+   - Should fail: test_ordering
+```
+
+## Quality Checklist
+
+For each test, verify:
+- [ ] Not tautological (assertion doesn't compare same expression)
+- [ ] Strong assertion (not just "no crash")
+- [ ] Not vacuous (inputs not over-filtered)
+- [ ] Good coverage (edge cases via `@example`)
+- [ ] No reimplementation of function logic
+- [ ] Appropriate settings for context
+- [ ] Good shrinking potential
+- [ ] Deterministic (no flakiness risk)
+
+## Red Flags
+
+- **Marking tautologies as "fine"**: `assert x == x` is NEVER a valid test
+- **Accepting "no crash" as sufficient**: Always push for stronger properties
+- **Ignoring vacuous tests**: Tests with contradictory `assume()` provide false confidence
+- **Not checking for reimplementation**: `assert add(a,b) == a + b` tests nothing if that's how `add` is implemented

package/skills_ref/property-based-testing/skills/property-based-testing/references/strategies.md

@@ -0,0 +1,124 @@
+# Input Strategy Reference
+
+## Python/Hypothesis
+
+| Type | Strategy |
+|------|----------|
+| `int` | `st.integers()` |
+| `float` | `st.floats(allow_nan=False)` |
+| `str` | `st.text()` |
+| `bytes` | `st.binary()` |
+| `bool` | `st.booleans()` |
+| `list[T]` | `st.lists(strategy_for_T)` |
+| `dict[K, V]` | `st.dictionaries(key_strategy, value_strategy)` |
+| `set[T]` | `st.frozensets(strategy_for_T)` |
+| `tuple[T, ...]` | `st.tuples(strategy_for_T, ...)` |
+| `Optional[T]` | `st.none() \| strategy_for_T` |
+| `Union[A, B]` | `st.one_of(strategy_a, strategy_b)` |
+| Custom class | `st.builds(ClassName, field1=..., field2=...)` |
+| Enum | `st.sampled_from(EnumClass)` |
+| Constrained int | `st.integers(min_value=0, max_value=100)` |
+| Email | `st.emails()` |
+| UUID | `st.uuids()` |
+| DateTime | `st.datetimes()` |
+| Regex match | `st.from_regex(r"pattern")` |
+
+### Composite Strategies
+
+For complex types, use `@st.composite`:
+
+```python
+@st.composite
+def valid_users(draw):
+    name = draw(st.text(min_size=1, max_size=50))
+    age = draw(st.integers(min_value=0, max_value=150))
+    email = draw(st.emails())
+    return User(name=name, age=age, email=email)
+```
+
+## JavaScript/fast-check
+
+| Type | Strategy |
+|------|----------|
+| number | `fc.integer()` or `fc.float()` |
+| string | `fc.string()` |
+| boolean | `fc.boolean()` |
+| array | `fc.array(itemArb)` |
+| object | `fc.record({...})` |
+| optional | `fc.option(arb)` |
+
+### Example
+
+```typescript
+const userArb = fc.record({
+  name: fc.string({ minLength: 1, maxLength: 50 }),
+  age: fc.integer({ min: 0, max: 150 }),
+  email: fc.emailAddress(),
+});
+```
+
+## Rust/proptest
+
+| Type | Strategy |
+|------|----------|
+| i32, u64, etc | `any::<i32>()` |
+| String | `any::<String>()` or `"[a-z]+"` (regex) |
+| Vec<T> | `prop::collection::vec(strategy, size)` |
+| Option<T> | `prop::option::of(strategy)` |
+
+### Example
+
+```rust
+proptest! {
+    #[test]
+    fn test_roundtrip(s in "[a-z]{1,20}") {
+        let encoded = encode(&s);
+        let decoded = decode(&encoded)?;
+        prop_assert_eq!(s, decoded);
+    }
+}
+```
+
+## Go/rapid
+
+```go
+rapid.Check(t, func(t *rapid.T) {
+    s := rapid.String().Draw(t, "s")
+    n := rapid.IntRange(0, 100).Draw(t, "n")
+    // test with s and n
+})
+```
+
+## Best Practices
+
+1. **Constrain early**: Build constraints into strategy, not `assume()`
+   ```python
+   # GOOD
+   st.integers(min_value=1, max_value=100)
+
+   # BAD
+   st.integers().filter(lambda x: 1 <= x <= 100)
+   ```
+
+2. **Size limits**: Use `max_size` to prevent slow tests
+   ```python
+   st.lists(st.integers(), max_size=100)
+   st.text(max_size=1000)
+   ```
+
+3. **Realistic data**: Make strategies match real-world constraints
+   ```python
+   # Real user ages, not arbitrary integers
+   st.integers(min_value=0, max_value=150)
+   ```
+
+4. **Reuse strategies**: Define once, use across tests
+   ```python
+   valid_users = st.builds(User, ...)
+
+   @given(valid_users)
+   def test_one(user): ...
+
+   @given(valid_users)
+   def test_two(user): ...
+   ```