chaos_to_the_rescue 0.1.0

data/FEATURES.md

@@ -0,0 +1,306 @@
+# Method Guidance and Verification Features
+
+## Overview
+
+This document describes the method guidance and verification features added to ChaosToTheRescue. These features help ensure that LLM-generated methods handle edge cases correctly and provide a way to verify and automatically fix incorrect implementations.
+
+## Three Main Features
+
+### 1. chaos_guidance - Provide Guidance Before Generation
+
+Add guidance when defining a class to help the LLM generate correct implementations:
+
+```ruby
+class Calculator
+  include ChaosToTheRescue::ChaosRescue
+  chaos_rescue_enabled!
+
+  chaos_guidance :pow, <<~GUIDANCE
+    Handle negative bases correctly:
+    - (-5)^2 = 25 (negative base, even exponent = positive)
+    - (-5)^3 = -125 (negative base, odd exponent = negative)
+    Common mistake: Using base.abs which always returns positive
+  GUIDANCE
+end
+```
+
+When `pow` is called and doesn't exist, the LLM will see this guidance and generate a correct implementation.
+
+### 2. additional_chaos_guidance - Add Guidance to Existing Methods
+
+Add guidance to methods that are already defined:
+
+```ruby
+class Calculator
+  def pow(base, exponent)
+    base ** exponent  # Already defined
+  end
+
+  # Add guidance for verification purposes
+  additional_chaos_guidance :pow, "Should handle negative bases correctly"
+end
+```
+
+This is an alias for `chaos_guidance` and is useful for documenting expected behavior of existing methods.
+
+### 3. Verification - Verify and Auto-Fix Method Outputs
+
+Verify that method outputs are correct:
+
+#### Class Method Verification
+
+```ruby
+# Simple verification
+result = Calculator.verify_chaos(:pow, -5, 2)
+
+if result.failed?
+  puts result.diagnosis
+  puts result.suggestion
+  puts result.fixed_code
+end
+
+# With auto-fix
+Calculator.verify_chaos(:pow, -5, 2, auto_apply: true)
+```
+
+#### Block-based Verification
+
+```ruby
+# Simple verification
+result = ChaosToTheRescue.verify do
+  Calculator.new.pow(-5, 2)
+end
+
+# With auto-fix
+result = ChaosToTheRescue.verify(auto_apply: true) do
+  Calculator.new.pow(-5, 2)
+end
+```
+
+## How It Works
+
+### Guidance Flow
+
+1. **Class-level guidance** is stored in a class variable when you call `chaos_guidance`
+2. **Global guidance** can be set in configuration: `config.method_guidance[:method_name] = "guidance"`
+3. When generating a method, the LLM receives the guidance in its prompt
+4. The LLM uses the guidance to avoid common mistakes and handle edge cases
+
+### Verification Flow
+
+1. Call the method with specific arguments
+2. Capture the actual result
+3. Send method name, arguments, result, and guidance to the LLM
+4. LLM analyzes whether the result is correct
+5. If incorrect, LLM provides:
+   - Diagnosis of what went wrong
+   - Suggestion for correct result
+   - Fixed code (if method source is available)
+6. If `auto_apply: true`, the fixed code is evaluated and replaces the method
+
+## Use Cases
+
+### 1. Handling Edge Cases in Generated Methods
+
+**Problem:** `Calculator.new.pow(-5, 2)` returns 25 (incorrect) because the implementation used `base.abs**exponent`
+
+**Solution:**
+```ruby
+class Calculator
+  include ChaosToTheRescue::ChaosRescue
+  chaos_rescue_enabled!
+
+  chaos_guidance :pow, "Handle negative bases: (-5)^2 = 25, (-5)^3 = -125"
+end
+
+# Now when pow is generated, it will handle negatives correctly
+Calculator.new.pow(-5, 2)  # Returns 25 (correct!)
+Calculator.new.pow(-5, 3)  # Returns -125 (correct!)
+```
+
+### 2. Verifying Existing Implementations
+
+**Problem:** You have a method but aren't sure if it handles all cases
+
+**Solution:**
+```ruby
+class Calculator
+  def pow(base, exponent)
+    base.abs ** exponent  # Bug!
+  end
+
+  additional_chaos_guidance :pow, "Should handle negative bases correctly"
+end
+
+# Verify the implementation
+result = Calculator.verify_chaos(:pow, -5, 2)
+puts result.diagnosis if result.failed?
+# Output: "The implementation uses .abs which always returns positive results..."
+```
+
+### 3. Auto-fixing Incorrect Methods
+
+**Problem:** A method has a bug and you want it automatically fixed
+
+**Solution:**
+```ruby
+# This will verify and auto-apply a fix if needed
+Calculator.verify_chaos(:pow, -5, 2, auto_apply: true)
+
+# Method is now fixed!
+Calculator.new.pow(-5, 3)  # Returns -125 (correct!)
+```
+
+## API Reference
+
+### Class Methods
+
+#### `chaos_guidance(method_name, guidance)`
+
+Add guidance for a method to help LLM generation.
+
+**Parameters:**
+- `method_name` (Symbol): The method name
+- `guidance` (String): Guidance text for the LLM
+
+**Example:**
+```ruby
+chaos_guidance :divide, "Handle division by zero by returning Float::INFINITY"
+```
+
+#### `additional_chaos_guidance(method_name, guidance)`
+
+Alias for `chaos_guidance`. Use for existing methods.
+
+#### `verify_chaos(method_name, *args, auto_apply: false, **kwargs)`
+
+Verify a method call on a new instance of the class.
+
+**Parameters:**
+- `method_name` (Symbol): The method to verify
+- `*args`: Arguments to pass to the method
+- `auto_apply` (Boolean): Whether to automatically apply fixes (default: false)
+- `**kwargs`: Keyword arguments to pass to the method
+
+**Returns:** `Verifier::VerificationResult`
+
+**Example:**
+```ruby
+result = Calculator.verify_chaos(:pow, -5, 2)
+result = Calculator.verify_chaos(:divide, 10, 0, auto_apply: true)
+```
+
+#### `chaos_method_guidance`
+
+Get the guidance hash for this class.
+
+**Returns:** Hash of method name => guidance string
+
+### Module Methods
+
+#### `ChaosToTheRescue.verify(auto_apply: false, &block)`
+
+Verify a method call using a block.
+
+**Parameters:**
+- `auto_apply` (Boolean): Whether to automatically apply fixes (default: false)
+- `&block`: Block containing the method call to verify
+
+**Returns:** `Verifier::VerificationResult`
+
+**Example:**
+```ruby
+result = ChaosToTheRescue.verify { Calculator.new.pow(-5, 2) }
+result = ChaosToTheRescue.verify(auto_apply: true) { Calculator.new.divide(10, 0) }
+```
+
+### Configuration
+
+#### Global Method Guidance
+
+Set guidance that applies to all classes:
+
+```ruby
+ChaosToTheRescue.configure do |config|
+  config.method_guidance[:pow] = "Handle negative bases correctly"
+  config.method_guidance[:divide] = "Handle division by zero"
+end
+```
+
+## VerificationResult Object
+
+The result of a verification has the following attributes:
+
+- `success` (Boolean): Whether the verification passed
+- `diagnosis` (String): Explanation of the result
+- `suggestion` (String): Suggested correct result (if failed)
+- `fixed_code` (String): Fixed code (if available and failed)
+- `actual_result` (Object): The actual result from calling the method
+
+**Helper Methods:**
+- `passed?` - Returns true if verification succeeded
+- `failed?` - Returns true if verification failed
+
+## Examples
+
+See `examples/guidance_and_verification.rb` for comprehensive examples covering:
+- Providing guidance when defining methods
+- Adding guidance to existing methods
+- Simple verification
+- Verification with auto-fix
+- Block-based verification
+- Testing edge cases
+- Comprehensive workflows
+
+## Safety Considerations
+
+### Auto-fix Safety
+
+When using `auto_apply: true`:
+- The LLM-generated fix code is evaluated using `eval`
+- Only use in development/test environments
+- Review the `fixed_code` in the verification result before trusting it
+- Set `log_level: :debug` to see the generated code
+
+### Guidance Best Practices
+
+1. **Be specific**: Include concrete examples of expected behavior
+2. **Mention common mistakes**: Help the LLM avoid known pitfalls
+3. **Cover edge cases**: Negative numbers, zero, empty strings, nil, etc.
+4. **Keep it concise**: Focus on the most important requirements
+
+### Verification Best Practices
+
+1. **Test edge cases**: Verify with unusual or boundary inputs
+2. **Review diagnoses**: Read what the LLM thinks went wrong
+3. **Manual review**: Check `fixed_code` before applying manually
+4. **Iterative fixing**: Verify again after auto-fix to ensure correctness
+
+## Implementation Details
+
+### Files Added
+
+- `lib/chaos_to_the_rescue/verifier.rb` - Verification logic
+- `spec/verifier_spec.rb` - Verifier tests
+- `spec/chaos_guidance_spec.rb` - Guidance feature tests
+- `examples/guidance_and_verification.rb` - Comprehensive examples
+- `FEATURES.md` - This document
+
+### Files Modified
+
+- `lib/chaos_to_the_rescue.rb` - Added verify method and verifier require
+- `lib/chaos_to_the_rescue/chaos_rescue.rb` - Added guidance methods
+- `lib/chaos_to_the_rescue/configuration.rb` - Added method_guidance attribute
+- `lib/chaos_to_the_rescue/llm_client.rb` - Added guidance to prompts
+- `README.md` - Added documentation for new features
+
+## Future Enhancements
+
+Possible improvements:
+- Verification result caching
+- Batch verification of multiple methods
+- Verification history tracking
+- Web UI for reviewing verification results
+- Integration with CI/CD for automated verification
+- Support for expected result specifications
+- Verification against test suites

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Valentino Stoll
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.