cli-jaw 0.1.6 → 0.1.8

package/skills_ref/property-based-testing/skills/property-based-testing/references/design.md

@@ -1,191 +0,0 @@
-# Property-Driven Development
-
-Design features by defining properties upfront as executable specifications, before implementation.
-
-## When to Use
-
-- Designing a new feature from scratch
-- Building something with clear algebraic properties (serialization, validation, transformations)
-- Complex domain where edge cases are likely
-- User wants to think through requirements rigorously before coding
-
-## Process
-
-### Phase 1: Understand the Feature
-
-Gather information:
-- **Purpose**: What problem does this solve?
-- **Inputs**: What data does it accept? What makes inputs valid?
-- **Outputs**: What does it produce? What guarantees?
-- **Constraints**: What must always be true?
-- **Edge cases**: Boundary conditions?
-- **Relationships**: Inverse operations? Compositions?
-
-### Phase 2: Identify Candidate Properties
-
-Work through these discovery questions:
-
-| Question | Property Type | Example |
-|----------|---------------|---------|
-| Does it have an inverse operation? | Roundtrip | `decode(encode(x)) == x` |
-| Is applying it twice the same as once? | Idempotence | `f(f(x)) == f(x)` |
-| What quantities are preserved? | Invariants | Length, sum, count |
-| Is order of arguments irrelevant? | Commutativity | `f(a, b) == f(b, a)` |
-| Can operations be regrouped? | Associativity | `f(f(a,b), c) == f(a, f(b,c))` |
-| Is there a neutral element? | Identity | `f(x, 0) == x` |
-| Is there an oracle/reference impl? | Oracle | `new(x) == old(x)` |
-| Can output be easily verified? | Hard/Easy | `is_sorted(sort(x))` |
-
-### Phase 3: Define Input Domain
-
-Specify valid inputs as strategies. The strategy IS the specification.
-
-**Key principle**: Build constraints INTO the strategy, not via `assume()`.
-
-```python
-@st.composite
-def valid_registration_requests(draw):
-    """Generate valid registration requests - this documents the domain."""
-    username = draw(st.text(
-        min_size=3,
-        max_size=20,
-        alphabet=st.characters(whitelist_categories=('L', 'N'))
-    ))
-    email = draw(st.emails())
-    password = draw(st.text(min_size=8, max_size=100))
-    age = draw(st.integers(min_value=13, max_value=150))
-
-    return RegistrationRequest(
-        username=username,
-        email=email,
-        password=password,
-        age=age
-    )
-```
-
-### Phase 4: Write Property Tests (Before Implementation)
-
-Create tests that will fail initially:
-
-```python
-class TestFeatureSpec:
-    """Property-based specification - should FAIL until implemented."""
-
-    @given(valid_inputs())
-    def test_core_property(self, x):
-        """[What this guarantees]."""
-        result = feature(x)
-        assert property_holds(result)
-```
-
-### Phase 5: Iterate on Design
-
-Properties reveal design questions:
-- "What about deleted users?"
-- "Case-sensitive?"
-- "Which algorithm?"
-- "Stable sort or not?"
-
-Surface these questions early, before implementation.
-
-## Property Strength Hierarchy
-
-Build properties incrementally from weak to strong:
-
-### Level 1: Basic (Weak)
-```python
-@given(valid_inputs())
-def test_no_crash(x):
-    process(x)  # Just don't crash
-```
-
-### Level 2: Type Preservation
-```python
-@given(valid_inputs())
-def test_returns_type(x):
-    assert isinstance(process(x), ExpectedType)
-```
-
-### Level 3: Invariants
-```python
-@given(valid_inputs())
-def test_invariant(x):
-    result = process(x)
-    assert invariant_holds(result)
-```
-
-### Level 4: Full Specification (Strong)
-```python
-@given(valid_inputs())
-def test_complete(x):
-    result = process(x)
-    assert satisfies_all_requirements(result)
-```
-
-## Strategy Design Principles
-
-### 1. Build Constraints Into Strategy
-```python
-# GOOD - constraints in strategy
-@given(st.integers(min_value=1, max_value=100))
-def test_with_valid_range(x): ...
-
-# BAD - constraints via assume
-@given(st.integers())
-def test_with_assume(x):
-    assume(1 <= x <= 100)  # High rejection rate
-```
-
-### 2. Match Real-World Constraints
-```python
-valid_users = st.builds(
-    User,
-    name=st.text(min_size=1, max_size=100),
-    age=st.integers(min_value=0, max_value=150),
-    email=st.emails(),
-)
-```
-
-### 3. Include Edge Cases Explicitly
-```python
-@given(valid_lists())
-@example([])           # Empty
-@example([1])          # Single element
-@example([1, 1, 1])    # Duplicates
-def test_with_edges(xs): ...
-```
-
-## Common Design Questions Raised
-
-Properties often reveal design gaps:
-
-| Property Attempt | Question Raised |
-|------------------|-----------------|
-| Roundtrip for users | What about deleted/deactivated users? |
-| Duplicate rejection | Case-sensitive? Unicode normalization? |
-| Password storage | Which algorithm? Salted? Configurable? |
-| Ordering guarantee | Stable sort? Tie-breaking rules? |
-
-## Red Flags
-
-- **Writing tautological properties**: Don't reimplement the function logic in the test
-  ```python
-  # BAD - tests nothing
-  assert add(a, b) == a + b
-
-  # GOOD - tests algebraic properties
-  assert add(a, 0) == a  # identity
-  assert add(a, b) == add(b, a)  # commutativity
-  ```
-- **Starting too strong**: Build from weak to strong properties
-- **Ignoring design questions**: Properties that feel awkward often reveal design gaps
-- **Overly complex strategies**: If your input strategy is 50 lines, the domain model might need simplification
-- **Not involving the user**: Design questions should be discussed, not assumed
-
-## Checklist
-
-- [ ] Properties are not tautological
-- [ ] At least one strong property defined
-- [ ] Input strategy documents valid inputs
-- [ ] Design questions have been surfaced
-- [ ] Tests will actually FAIL without implementation

package/skills_ref/property-based-testing/skills/property-based-testing/references/generating.md

@@ -1,204 +0,0 @@
-# Generating Property-Based Tests
-
-How to create complete, runnable property-based tests.
-
-## Process
-
-### 1. Analyze Target Function
-
-- Read function signature, types, and docstrings
-- Understand input types and constraints
-- Identify output type and expected behavior
-- Note preconditions or invariants
-- Check existing example-based tests as hints
-
-### 2. Design Input Strategies
-
-Create appropriate generator strategies for each input parameter.
-
-**Principles**:
-- Build constraints INTO the strategy, not via `assume()`
-- Use realistic size limits to prevent slow tests
-- Match real-world constraints
-
-### 3. Identify Applicable Properties
-
-| Property | When to Use | Test Pattern |
-|----------|-------------|--------------|
-| Roundtrip | encode/decode pairs | `assert decode(encode(x)) == x` |
-| Idempotence | normalization, sorting | `assert f(f(x)) == f(x)` |
-| Invariant | any transformation | `assert invariant(f(x))` |
-| No exception | all functions (weak) | Function completes without raising |
-| Type preservation | typed functions | `assert isinstance(f(x), ExpectedType)` |
-| Length preservation | collections | `assert len(f(xs)) == len(xs)` |
-| Element preservation | sorting, shuffling | `assert set(f(xs)) == set(xs)` |
-| Ordering | sorting | `assert all(f(xs)[i] <= f(xs)[i+1] ...)` |
-| Oracle | when reference exists | `assert f(x) == reference_impl(x)` |
-| Commutativity | binary ops | `assert f(a, b) == f(b, a)` |
-
-### 4. Generate Test Code
-
-Create test functions with:
-- Clear docstrings explaining what each property verifies
-- Appropriate `@settings` for the context
-- `@example` decorators for critical edge cases
-
-### 5. Include Edge Cases
-
-Always add explicit examples:
-```python
-@example([])           # Empty
-@example([1])          # Single element
-@example([1, 1, 1])    # Duplicates
-@example("")           # Empty string
-@example(0)            # Zero
-@example(-1)           # Negative
-```
-
-## Settings Recommendations
-
-```python
-# Development (fast feedback)
-@settings(max_examples=10)
-
-# CI (thorough)
-@settings(max_examples=200)
-
-# Nightly/Release (exhaustive)
-@settings(max_examples=1000, deadline=None)
-```
-
-## Example Test Patterns
-
-### Roundtrip (Encode/Decode)
-
-```python
-@given(valid_messages())
-def test_roundtrip(msg):
-    """Encoding then decoding returns original."""
-    assert decode(encode(msg)) == msg
-```
-
-### Idempotence
-
-```python
-@given(st.text())
-def test_normalize_idempotent(s):
-    """Normalizing twice equals normalizing once."""
-    assert normalize(normalize(s)) == normalize(s)
-```
-
-### Sorting Properties
-
-```python
-@given(st.lists(st.integers()))
-@example([])
-@example([1])
-@example([1, 1, 1])
-def test_sort(xs):
-    result = sort(xs)
-    # Length preserved
-    assert len(result) == len(xs)
-    # Elements preserved
-    assert sorted(result) == sorted(xs)
-    # Ordered
-    assert all(result[i] <= result[i+1] for i in range(len(result)-1))
-    # Idempotent
-    assert sort(result) == result
-```
-
-### Validator + Normalizer
-
-```python
-@given(valid_inputs())
-def test_normalized_is_valid(x):
-    """Normalized inputs pass validation."""
-    assert is_valid(normalize(x))
-```
-
-## Complete Example (Python/Hypothesis)
-
-```python
-"""Property-based tests for message_codec module."""
-from hypothesis import given, strategies as st, settings, example
-import pytest
-
-from myapp.codec import encode_message, decode_message, Message, DecodeError
-
-# Custom strategy for Message objects
-messages = st.builds(
-    Message,
-    id=st.uuids(),
-    content=st.text(max_size=1000),
-    priority=st.integers(min_value=1, max_value=10),
-    tags=st.lists(st.text(max_size=50), max_size=20),
-)
-
-
-class TestMessageCodecProperties:
-    """Property-based tests for message encoding/decoding."""
-
-    @given(messages)
-    def test_roundtrip(self, msg: Message):
-        """Encoding then decoding returns the original message."""
-        encoded = encode_message(msg)
-        decoded = decode_message(encoded)
-        assert decoded == msg
-
-    @given(messages)
-    def test_encode_deterministic(self, msg: Message):
-        """Same message always encodes to same bytes."""
-        assert encode_message(msg) == encode_message(msg)
-
-    @given(messages)
-    def test_encoded_is_bytes(self, msg: Message):
-        """Encoding produces bytes."""
-        assert isinstance(encode_message(msg), bytes)
-
-    @given(st.binary())
-    def test_decode_invalid_raises_or_succeeds(self, data: bytes):
-        """Random bytes either decode or raise DecodeError."""
-        try:
-            decode_message(data)
-        except DecodeError:
-            pass  # Expected for invalid input
-```
-
-## Running Tests
-
-```bash
-# Run all property tests
-pytest test_file.py -v
-
-# Run with more examples (CI)
-pytest test_file.py --hypothesis-seed=0 -v
-
-# Run with statistics
-pytest test_file.py --hypothesis-show-statistics
-```
-
-## Checklist Before Finishing
-
-- [ ] Tests are not tautological (don't reimplement the function)
-- [ ] At least one strong property (not just "no crash")
-- [ ] Edge cases covered with `@example` decorators
-- [ ] Strategy constraints are realistic, not over-filtered
-- [ ] Settings appropriate for context (dev vs CI)
-- [ ] Docstrings explain what each property verifies
-- [ ] Tests actually run and pass (or fail for expected reasons)
-
-## Red Flags
-
-- **Reimplementing the function**: If your assertion contains the same logic as the function under test, you've written a tautology
-  ```python
-  # BAD - this tests nothing
-  assert add(a, b) == a + b
-  ```
-- **Only testing "no crash"**: This is the weakest property - always look for stronger ones first
-- **Overly constrained strategies**: If you're using multiple `assume()` calls, redesign the strategy instead
-- **Missing edge cases**: No `@example` decorators for empty, single-element, or boundary values
-- **No settings**: Missing `@settings` for CI - tests may be too slow or not thorough enough
-
-## When Tests Fail
-
-See [{baseDir}/references/interpreting-failures.md]({baseDir}/references/interpreting-failures.md) for how to interpret failures and determine if they represent genuine bugs vs test errors vs ambiguous specifications.

package/skills_ref/property-based-testing/skills/property-based-testing/references/interpreting-failures.md

@@ -1,239 +0,0 @@
-# Interpreting Property-Based Test Failures
-
-How to analyze failures and determine if they represent genuine bugs.
-
-## The Self-Reflection Problem
-
-Property-based testing generates many failing examples. Not all failures are bugs:
-- **Test bugs**: Property is wrong, strategy generates invalid inputs
-- **Ambiguous specs**: Behavior undefined for edge cases
-- **Genuine bugs**: Code violates documented guarantees
-
-Before reporting a bug, **validate the failure** through systematic analysis.
-
-## Failure Analysis Workflow
-
-### 1. Reproduce with Minimal Example
-
-Start with the shrunk failing input from the test output.
-
-```python
-# Hypothesis provides the minimal failing case
-# Falsifying example: test_normalize(s='\x00')
-
-# Create standalone reproducer
-def test_reproduce():
-    s = '\x00'
-    result = normalize(normalize(s))
-    assert result == normalize(s)  # Fails
-```
-
-Verify the failure is consistent, not flaky.
-
-### 2. Ground the Property
-
-Before assuming a bug, verify your property against authoritative sources:
-
-| Source | What It Tells You |
-|--------|-------------------|
-| **Type annotations** | Return type constraints, nullability |
-| **Docstrings** | Explicit guarantees, preconditions |
-| **Function name** | Semantic expectations (e.g., `sort` implies ordering) |
-| **Error handling** | What inputs should raise vs handle |
-| **Existing unit tests** | Implicit contracts maintainers expect |
-| **External docs/specs** | Protocol specs, format definitions |
-
-**Example grounding check:**
-```python
-def normalize(s: str) -> str:
-    """Normalize a string to NFC form.
-
-    Args:
-        s: Input string (any unicode)
-
-    Returns:
-        NFC-normalized string
-    """
-```
-
-The docstring says "any unicode" - so null bytes should be valid input. The property is correctly grounded.
-
-### 3. Check Strategy Realism
-
-Does the strategy generate inputs the function should actually handle?
-
-**Red flags:**
-- Generating inputs outside documented domain
-- Missing constraints that real callers would have
-- Overly aggressive size/complexity
-
-**Questions to ask:**
-- Would real code pass this input?
-- Does the docstring exclude this case?
-- Is this a precondition violation, not a bug?
-
-### 4. Classify the Failure
-
-| Symptom | Likely Cause | Action |
-|---------|--------------|--------|
-| Fails on edge case not mentioned in spec | Ambiguous specification | Clarify with maintainer before reporting |
-| Fails on input that violates documented preconditions | Over-constrained strategy | Fix the strategy |
-| Property contradicts docstring or type hints | Wrong property | Fix the property |
-| Clear violation of documented guarantee | Genuine bug | Report with evidence |
-| Behavior differs from similar functions | Possible inconsistency | Investigate further |
-
-### 5. Decide Action
-
-- **Test bug** → Fix the property or strategy, don't report
-- **Ambiguous spec** → Open discussion issue, not bug report
-- **Genuine bug** → Report with minimal reproducer and evidence
-
-## Property Grounding Checklist
-
-Before reporting a failure as a bug, verify:
-
-- [ ] Property matches documented return type
-- [ ] Property matches docstring guarantees
-- [ ] Input is within documented domain (preconditions met)
-- [ ] No `assume()` filtering out the failing case inappropriately
-- [ ] Checked existing tests don't contradict your property
-- [ ] Behavior contradicts docs, not just expectations
-
-## Bug Report Template
-
-When confident the failure is a genuine bug:
-
-```markdown
-## Summary
-[One-line description of the bug]
-
-## Minimal Reproducing Example
-```python
-# Shrunk by Hypothesis
-from mylib import affected_function
-
-def test_bug():
-    # Minimal failing input
-    result = affected_function('\x00')
-    # Expected vs actual
-    assert result >= 0  # Fails: got -1
-```
-
-## Expected Behavior
-According to [docstring/spec/docs], the function should:
-- [Specific guarantee that was violated]
-
-## Actual Behavior
-- [What actually happened]
-
-## Evidence
-- Docstring states: "[relevant quote]"
-- Type signature promises: `-> PositiveInt`
-
-## Environment
-- Library version: X.Y.Z
-- Python version: 3.X
-- Platform: [OS]
-```
-
-## Real-World Failure Patterns
-
-### Numerical Instability
-
-**Symptom**: Distribution function returns negative probability.
-
-```python
-@given(st.floats(min_value=0, max_value=1e308))
-def test_probability_non_negative(x):
-    prob = compute_probability(x)
-    assert prob >= 0  # Fails for x=1e-320
-```
-
-**Grounding check**: Docstring says "returns probability in [0, 1]".
-
-**Classification**: Genuine bug - documented guarantee violated.
-
-### Iterator Off-by-One
-
-**Symptom**: Iterator skips elements or yields extra.
-
-```python
-@given(st.lists(st.integers()))
-def test_iterator_yields_all(xs):
-    result = list(custom_iterator(xs))
-    assert result == xs  # Fails: missing last element
-```
-
-**Grounding check**: Iterator should yield all elements based on name/docs.
-
-**Classification**: Genuine bug if documented to iterate fully.
-
-### Hash/Equality Inconsistency
-
-**Symptom**: Equal objects have different hashes.
-
-```python
-@given(valid_objects())
-def test_hash_equality(obj):
-    obj2 = create_equal_copy(obj)
-    assert obj == obj2
-    assert hash(obj) == hash(obj2)  # Fails
-```
-
-**Grounding check**: Python requires `a == b` implies `hash(a) == hash(b)`.
-
-**Classification**: Genuine bug - violates language contract.
-
-### Roundtrip Failure on Edge Cases
-
-**Symptom**: Encode/decode doesn't preserve input.
-
-```python
-@given(st.text())
-def test_roundtrip(s):
-    assert decode(encode(s)) == s  # Fails for s='\uD800'
-```
-
-**Grounding check**: Is `'\uD800'` (lone surrogate) valid input?
-
-**Classification**:
-- If docs say "valid UTF-8 only" → Strategy bug, fix filter
-- If docs say "any string" → Genuine bug, report it
-
-### Format String Errors
-
-**Symptom**: String formatting crashes on certain inputs.
-
-```python
-@given(st.text())
-def test_format_safe(template):
-    format_message(template)  # Raises on '{unclosed'
-```
-
-**Grounding check**: Does function claim to handle arbitrary strings?
-
-**Classification**:
-- If user-facing, should handle gracefully → Genuine bug
-- If internal API with preconditions → Check preconditions met
-
-## When NOT to Report
-
-Do not report as bugs:
-
-1. **Precondition violations**: If docs say "positive integers only" and you passed -1
-2. **Undefined behavior**: Spec explicitly says behavior is undefined
-3. **Implementation details**: Relying on undocumented internal behavior
-4. **Platform-specific**: Bug only on unusual platform/version
-5. **Test artifact**: Failure disappears with realistic constraints
-
-## Confidence Threshold
-
-Report only when you can answer YES to all:
-
-1. Did you reproduce with a minimal example?
-2. Did you verify the property against docs/types/docstrings?
-3. Can you point to a specific documented guarantee that's violated?
-4. Is the failing input within the documented domain?
-5. Have you ruled out test bugs and ambiguous specs?
-
-If uncertain on any point, open a discussion first, not a bug report.