@elizaos/skills 2.0.0-alpha.3

package/skills/security-property-based-testing/skills/property-based-testing/README.md

@@ -0,0 +1,88 @@
+# Property-Based Testing Skill
+
+A Claude Code skill that provides guidance for property-based testing (PBT) across multiple programming languages and smart contract development.
+
+## What This Skill Does
+
+When activated, this skill helps Claude:
+
+- **Detect PBT opportunities** - Recognizes patterns like encode/decode pairs, validators, normalizers, pure functions, and smart contract invariants
+- **Generate property-based tests** - Creates tests with appropriate strategies, properties, and edge cases
+- **Review existing PBT tests** - Identifies issues like tautological properties, vacuous tests, and weak assertions
+- **Design with properties** - Uses Property-Driven Development to define specifications before implementation
+- **Refactor for testability** - Suggests code changes that enable stronger property testing
+
+## Supported Languages
+
+| Language | Library | Notes |
+|----------|---------|-------|
+| Python | Hypothesis | |
+| JavaScript/TypeScript | fast-check | |
+| Rust | proptest | Also: quickcheck |
+| Go | rapid | Also: gopter |
+| Java | jqwik | |
+| Scala | ScalaCheck | |
+| C# | FsCheck | |
+| Elixir | StreamData | |
+| Haskell | QuickCheck | Also: Hedgehog |
+| Clojure | test.check | |
+| Ruby | PropCheck | |
+| Kotlin | Kotest | |
+| Swift | SwiftCheck | Unmaintained |
+| C++ | RapidCheck | |
+
+### Smart Contract Testing
+
+| Tool | Platform | Description |
+|------|----------|-------------|
+| Echidna | EVM/Solidity | Property-based fuzzer |
+| Medusa | EVM/Solidity | Next-gen parallel fuzzer |
+
+See [secure-contracts.com](https://secure-contracts.com) for tutorials.
+
+## File Structure
+
+```
+property-based-testing/
+├── SKILL.md           # Entry point - detection patterns and routing
+├── README.md          # This file
+└── references/
+    ├── generating.md  # How to write property-based tests
+    ├── reviewing.md   # How to evaluate test quality
+    ├── strategies.md  # Input generation reference
+    ├── design.md      # Property-Driven Development workflow
+    ├── refactoring.md # Making code more testable
+    └── libraries.md   # PBT library reference by language
+```
+
+## Usage
+
+The skill activates automatically when Claude detects relevant patterns:
+
+- Serialization pairs (`encode`/`decode`, `serialize`/`deserialize`)
+- Validators and normalizers
+- Pure functions with clear input/output types
+- Data structure operations
+- Smart contracts (Solidity/Vyper)
+
+You can also invoke it explicitly by asking Claude to use property-based testing.
+
+### Example Prompts
+
+```
+"Write property-based tests for this JSON serializer"
+"Review this Hypothesis test for quality issues"
+"Help me design this feature using properties first"
+"This function is hard to test - how can I refactor it?"
+"Write Echidna invariants for this token contract"
+```
+
+## Property Quick Reference
+
+| Property | Pattern | Use Case |
+|----------|---------|----------|
+| Roundtrip | `decode(encode(x)) == x` | Serialization |
+| Idempotence | `f(f(x)) == f(x)` | Normalization |
+| Invariant | `property(f(x))` holds | Any transformation, smart contracts |
+| Commutativity | `f(a,b) == f(b,a)` | Binary operations |
+| Oracle | `new(x) == reference(x)` | Refactoring |

package/skills/security-property-based-testing/skills/property-based-testing/SKILL.md

@@ -0,0 +1,109 @@
+---
+name: property-based-testing
+description: Provides guidance for property-based testing across multiple languages and smart contracts. Use when writing tests, reviewing code with serialization/validation/parsing patterns, designing features, or when property-based testing would provide stronger coverage than example-based tests.
+---
+
+# Property-Based Testing Guide
+
+Use this skill proactively during development when you encounter patterns where PBT provides stronger coverage than example-based tests.
+
+## When to Invoke (Automatic Detection)
+
+**Invoke this skill when you detect:**
+
+- **Serialization pairs**: `encode`/`decode`, `serialize`/`deserialize`, `toJSON`/`fromJSON`, `pack`/`unpack`
+- **Parsers**: URL parsing, config parsing, protocol parsing, string-to-structured-data
+- **Normalization**: `normalize`, `sanitize`, `clean`, `canonicalize`, `format`
+- **Validators**: `is_valid`, `validate`, `check_*` (especially with normalizers)
+- **Data structures**: Custom collections with `add`/`remove`/`get` operations
+- **Mathematical/algorithmic**: Pure functions, sorting, ordering, comparators
+- **Smart contracts**: Solidity/Vyper contracts, token operations, state invariants, access control
+
+**Priority by pattern:**
+
+| Pattern | Property | Priority |
+|---------|----------|----------|
+| encode/decode pair | Roundtrip | HIGH |
+| Pure function | Multiple | HIGH |
+| Validator | Valid after normalize | MEDIUM |
+| Sorting/ordering | Idempotence + ordering | MEDIUM |
+| Normalization | Idempotence | MEDIUM |
+| Builder/factory | Output invariants | LOW |
+| Smart contract | State invariants | HIGH |
+
+## When NOT to Use
+
+Do NOT use this skill for:
+- Simple CRUD operations without transformation logic
+- One-off scripts or throwaway code
+- Code with side effects that cannot be isolated (network calls, database writes)
+- Tests where specific example cases are sufficient and edge cases are well-understood
+- Integration or end-to-end testing (PBT is best for unit/component testing)
+
+## Property Catalog (Quick Reference)
+
+| Property | Formula | When to Use |
+|----------|---------|-------------|
+| **Roundtrip** | `decode(encode(x)) == x` | Serialization, conversion pairs |
+| **Idempotence** | `f(f(x)) == f(x)` | Normalization, formatting, sorting |
+| **Invariant** | Property holds before/after | Any transformation |
+| **Commutativity** | `f(a, b) == f(b, a)` | Binary/set operations |
+| **Associativity** | `f(f(a,b), c) == f(a, f(b,c))` | Combining operations |
+| **Identity** | `f(x, identity) == x` | Operations with neutral element |
+| **Inverse** | `f(g(x)) == x` | encrypt/decrypt, compress/decompress |
+| **Oracle** | `new_impl(x) == reference(x)` | Optimization, refactoring |
+| **Easy to Verify** | `is_sorted(sort(x))` | Complex algorithms |
+| **No Exception** | No crash on valid input | Baseline property |
+
+**Strength hierarchy** (weakest to strongest):
+No Exception → Type Preservation → Invariant → Idempotence → Roundtrip
+
+## Decision Tree
+
+Based on the current task, read the appropriate section:
+
+```
+TASK: Writing new tests
+  → Read [{baseDir}/references/generating.md]({baseDir}/references/generating.md) (test generation patterns and examples)
+  → Then [{baseDir}/references/strategies.md]({baseDir}/references/strategies.md) if input generation is complex
+
+TASK: Designing a new feature
+  → Read [{baseDir}/references/design.md]({baseDir}/references/design.md) (Property-Driven Development approach)
+
+TASK: Code is difficult to test (mixed I/O, missing inverses)
+  → Read [{baseDir}/references/refactoring.md]({baseDir}/references/refactoring.md) (refactoring patterns for testability)
+
+TASK: Reviewing existing PBT tests
+  → Read [{baseDir}/references/reviewing.md]({baseDir}/references/reviewing.md) (quality checklist and anti-patterns)
+
+TASK: Need library reference
+  → Read [{baseDir}/references/libraries.md]({baseDir}/references/libraries.md) (PBT libraries by language, includes smart contract tools)
+```
+
+## How to Suggest PBT
+
+When you detect a high-value pattern while writing tests, **offer PBT as an option**:
+
+> "I notice `encode_message`/`decode_message` is a serialization pair. Property-based testing with a roundtrip property would provide stronger coverage than example tests. Want me to use that approach?"
+
+**If codebase already uses a PBT library** (Hypothesis, fast-check, proptest, Echidna), be more direct:
+
+> "This codebase uses Hypothesis. I'll write property-based tests for this serialization pair using a roundtrip property."
+
+**If user declines**, write good example-based tests without further prompting.
+
+## When NOT to Use PBT
+
+- Simple CRUD without complex validation
+- UI/presentation logic
+- Integration tests requiring complex external setup
+- Prototyping where requirements are fluid
+- User explicitly requests example-based tests only
+
+## Red Flags
+
+- Recommending trivial getters/setters
+- Missing paired operations (encode without decode)
+- Ignoring type hints (well-typed = easier to test)
+- Overwhelming user with candidates (limit to top 5-10)
+- Being pushy after user declines

package/skills/security-property-based-testing/skills/property-based-testing/references/design.md

@@ -0,0 +1,191 @@
+# Property-Driven Development
+
+Design features by defining properties upfront as executable specifications, before implementation.
+
+## When to Use
+
+- Designing a new feature from scratch
+- Building something with clear algebraic properties (serialization, validation, transformations)
+- Complex domain where edge cases are likely
+- User wants to think through requirements rigorously before coding
+
+## Process
+
+### Phase 1: Understand the Feature
+
+Gather information:
+- **Purpose**: What problem does this solve?
+- **Inputs**: What data does it accept? What makes inputs valid?
+- **Outputs**: What does it produce? What guarantees?
+- **Constraints**: What must always be true?
+- **Edge cases**: Boundary conditions?
+- **Relationships**: Inverse operations? Compositions?
+
+### Phase 2: Identify Candidate Properties
+
+Work through these discovery questions:
+
+| Question | Property Type | Example |
+|----------|---------------|---------|
+| Does it have an inverse operation? | Roundtrip | `decode(encode(x)) == x` |
+| Is applying it twice the same as once? | Idempotence | `f(f(x)) == f(x)` |
+| What quantities are preserved? | Invariants | Length, sum, count |
+| Is order of arguments irrelevant? | Commutativity | `f(a, b) == f(b, a)` |
+| Can operations be regrouped? | Associativity | `f(f(a,b), c) == f(a, f(b,c))` |
+| Is there a neutral element? | Identity | `f(x, 0) == x` |
+| Is there an oracle/reference impl? | Oracle | `new(x) == old(x)` |
+| Can output be easily verified? | Hard/Easy | `is_sorted(sort(x))` |
+
+### Phase 3: Define Input Domain
+
+Specify valid inputs as strategies. The strategy IS the specification.
+
+**Key principle**: Build constraints INTO the strategy, not via `assume()`.
+
+```python
+@st.composite
+def valid_registration_requests(draw):
+    """Generate valid registration requests - this documents the domain."""
+    username = draw(st.text(
+        min_size=3,
+        max_size=20,
+        alphabet=st.characters(whitelist_categories=('L', 'N'))
+    ))
+    email = draw(st.emails())
+    password = draw(st.text(min_size=8, max_size=100))
+    age = draw(st.integers(min_value=13, max_value=150))
+
+    return RegistrationRequest(
+        username=username,
+        email=email,
+        password=password,
+        age=age
+    )
+```
+
+### Phase 4: Write Property Tests (Before Implementation)
+
+Create tests that will fail initially:
+
+```python
+class TestFeatureSpec:
+    """Property-based specification - should FAIL until implemented."""
+
+    @given(valid_inputs())
+    def test_core_property(self, x):
+        """[What this guarantees]."""
+        result = feature(x)
+        assert property_holds(result)
+```
+
+### Phase 5: Iterate on Design
+
+Properties reveal design questions:
+- "What about deleted users?"
+- "Case-sensitive?"
+- "Which algorithm?"
+- "Stable sort or not?"
+
+Surface these questions early, before implementation.
+
+## Property Strength Hierarchy
+
+Build properties incrementally from weak to strong:
+
+### Level 1: Basic (Weak)
+```python
+@given(valid_inputs())
+def test_no_crash(x):
+    process(x)  # Just don't crash
+```
+
+### Level 2: Type Preservation
+```python
+@given(valid_inputs())
+def test_returns_type(x):
+    assert isinstance(process(x), ExpectedType)
+```
+
+### Level 3: Invariants
+```python
+@given(valid_inputs())
+def test_invariant(x):
+    result = process(x)
+    assert invariant_holds(result)
+```
+
+### Level 4: Full Specification (Strong)
+```python
+@given(valid_inputs())
+def test_complete(x):
+    result = process(x)
+    assert satisfies_all_requirements(result)
+```
+
+## Strategy Design Principles
+
+### 1. Build Constraints Into Strategy
+```python
+# GOOD - constraints in strategy
+@given(st.integers(min_value=1, max_value=100))
+def test_with_valid_range(x): ...
+
+# BAD - constraints via assume
+@given(st.integers())
+def test_with_assume(x):
+    assume(1 <= x <= 100)  # High rejection rate
+```
+
+### 2. Match Real-World Constraints
+```python
+valid_users = st.builds(
+    User,
+    name=st.text(min_size=1, max_size=100),
+    age=st.integers(min_value=0, max_value=150),
+    email=st.emails(),
+)
+```
+
+### 3. Include Edge Cases Explicitly
+```python
+@given(valid_lists())
+@example([])           # Empty
+@example([1])          # Single element
+@example([1, 1, 1])    # Duplicates
+def test_with_edges(xs): ...
+```
+
+## Common Design Questions Raised
+
+Properties often reveal design gaps:
+
+| Property Attempt | Question Raised |
+|------------------|-----------------|
+| Roundtrip for users | What about deleted/deactivated users? |
+| Duplicate rejection | Case-sensitive? Unicode normalization? |
+| Password storage | Which algorithm? Salted? Configurable? |
+| Ordering guarantee | Stable sort? Tie-breaking rules? |
+
+## Red Flags
+
+- **Writing tautological properties**: Don't reimplement the function logic in the test
+  ```python
+  # BAD - tests nothing
+  assert add(a, b) == a + b
+
+  # GOOD - tests algebraic properties
+  assert add(a, 0) == a  # identity
+  assert add(a, b) == add(b, a)  # commutativity
+  ```
+- **Starting too strong**: Build from weak to strong properties
+- **Ignoring design questions**: Properties that feel awkward often reveal design gaps
+- **Overly complex strategies**: If your input strategy is 50 lines, the domain model might need simplification
+- **Not involving the user**: Design questions should be discussed, not assumed
+
+## Checklist
+
+- [ ] Properties are not tautological
+- [ ] At least one strong property defined
+- [ ] Input strategy documents valid inputs
+- [ ] Design questions have been surfaced
+- [ ] Tests will actually FAIL without implementation

package/skills/security-property-based-testing/skills/property-based-testing/references/generating.md

@@ -0,0 +1,200 @@
+# Generating Property-Based Tests
+
+How to create complete, runnable property-based tests.
+
+## Process
+
+### 1. Analyze Target Function
+
+- Read function signature, types, and docstrings
+- Understand input types and constraints
+- Identify output type and expected behavior
+- Note preconditions or invariants
+- Check existing example-based tests as hints
+
+### 2. Design Input Strategies
+
+Create appropriate generator strategies for each input parameter.
+
+**Principles**:
+- Build constraints INTO the strategy, not via `assume()`
+- Use realistic size limits to prevent slow tests
+- Match real-world constraints
+
+### 3. Identify Applicable Properties
+
+| Property | When to Use | Test Pattern |
+|----------|-------------|--------------|
+| Roundtrip | encode/decode pairs | `assert decode(encode(x)) == x` |
+| Idempotence | normalization, sorting | `assert f(f(x)) == f(x)` |
+| Invariant | any transformation | `assert invariant(f(x))` |
+| No exception | all functions (weak) | Function completes without raising |
+| Type preservation | typed functions | `assert isinstance(f(x), ExpectedType)` |
+| Length preservation | collections | `assert len(f(xs)) == len(xs)` |
+| Element preservation | sorting, shuffling | `assert set(f(xs)) == set(xs)` |
+| Ordering | sorting | `assert all(f(xs)[i] <= f(xs)[i+1] ...)` |
+| Oracle | when reference exists | `assert f(x) == reference_impl(x)` |
+| Commutativity | binary ops | `assert f(a, b) == f(b, a)` |
+
+### 4. Generate Test Code
+
+Create test functions with:
+- Clear docstrings explaining what each property verifies
+- Appropriate `@settings` for the context
+- `@example` decorators for critical edge cases
+
+### 5. Include Edge Cases
+
+Always add explicit examples:
+```python
+@example([])           # Empty
+@example([1])          # Single element
+@example([1, 1, 1])    # Duplicates
+@example("")           # Empty string
+@example(0)            # Zero
+@example(-1)           # Negative
+```
+
+## Settings Recommendations
+
+```python
+# Development (fast feedback)
+@settings(max_examples=10)
+
+# CI (thorough)
+@settings(max_examples=200)
+
+# Nightly/Release (exhaustive)
+@settings(max_examples=1000, deadline=None)
+```
+
+## Example Test Patterns
+
+### Roundtrip (Encode/Decode)
+
+```python
+@given(valid_messages())
+def test_roundtrip(msg):
+    """Encoding then decoding returns original."""
+    assert decode(encode(msg)) == msg
+```
+
+### Idempotence
+
+```python
+@given(st.text())
+def test_normalize_idempotent(s):
+    """Normalizing twice equals normalizing once."""
+    assert normalize(normalize(s)) == normalize(s)
+```
+
+### Sorting Properties
+
+```python
+@given(st.lists(st.integers()))
+@example([])
+@example([1])
+@example([1, 1, 1])
+def test_sort(xs):
+    result = sort(xs)
+    # Length preserved
+    assert len(result) == len(xs)
+    # Elements preserved
+    assert sorted(result) == sorted(xs)
+    # Ordered
+    assert all(result[i] <= result[i+1] for i in range(len(result)-1))
+    # Idempotent
+    assert sort(result) == result
+```
+
+### Validator + Normalizer
+
+```python
+@given(valid_inputs())
+def test_normalized_is_valid(x):
+    """Normalized inputs pass validation."""
+    assert is_valid(normalize(x))
+```
+
+## Complete Example (Python/Hypothesis)
+
+```python
+"""Property-based tests for message_codec module."""
+from hypothesis import given, strategies as st, settings, example
+import pytest
+
+from myapp.codec import encode_message, decode_message, Message, DecodeError
+
+# Custom strategy for Message objects
+messages = st.builds(
+    Message,
+    id=st.uuids(),
+    content=st.text(max_size=1000),
+    priority=st.integers(min_value=1, max_value=10),
+    tags=st.lists(st.text(max_size=50), max_size=20),
+)
+
+
+class TestMessageCodecProperties:
+    """Property-based tests for message encoding/decoding."""
+
+    @given(messages)
+    def test_roundtrip(self, msg: Message):
+        """Encoding then decoding returns the original message."""
+        encoded = encode_message(msg)
+        decoded = decode_message(encoded)
+        assert decoded == msg
+
+    @given(messages)
+    def test_encode_deterministic(self, msg: Message):
+        """Same message always encodes to same bytes."""
+        assert encode_message(msg) == encode_message(msg)
+
+    @given(messages)
+    def test_encoded_is_bytes(self, msg: Message):
+        """Encoding produces bytes."""
+        assert isinstance(encode_message(msg), bytes)
+
+    @given(st.binary())
+    def test_decode_invalid_raises_or_succeeds(self, data: bytes):
+        """Random bytes either decode or raise DecodeError."""
+        try:
+            decode_message(data)
+        except DecodeError:
+            pass  # Expected for invalid input
+```
+
+## Running Tests
+
+```bash
+# Run all property tests
+pytest test_file.py -v
+
+# Run with more examples (CI)
+pytest test_file.py --hypothesis-seed=0 -v
+
+# Run with statistics
+pytest test_file.py --hypothesis-show-statistics
+```
+
+## Checklist Before Finishing
+
+- [ ] Tests are not tautological (don't reimplement the function)
+- [ ] At least one strong property (not just "no crash")
+- [ ] Edge cases covered with `@example` decorators
+- [ ] Strategy constraints are realistic, not over-filtered
+- [ ] Settings appropriate for context (dev vs CI)
+- [ ] Docstrings explain what each property verifies
+- [ ] Tests actually run and pass (or fail for expected reasons)
+
+## Red Flags
+
+- **Reimplementing the function**: If your assertion contains the same logic as the function under test, you've written a tautology
+  ```python
+  # BAD - this tests nothing
+  assert add(a, b) == a + b
+  ```
+- **Only testing "no crash"**: This is the weakest property - always look for stronger ones first
+- **Overly constrained strategies**: If you're using multiple `assume()` calls, redesign the strategy instead
+- **Missing edge cases**: No `@example` decorators for empty, single-element, or boundary values
+- **No settings**: Missing `@settings` for CI - tests may be too slow or not thorough enough