roast-ai 0.2.0 → 0.2.2

data/examples/direct_coerce_syntax/README.md

@@ -0,0 +1,32 @@
+# Direct Coerce Syntax
+
+This example demonstrates the simplified syntax for specifying `coerce_to` and other configuration options directly on iteration steps.
+
+## Direct Syntax
+
+Configuration options are specified directly on the step:
+
+```yaml
+- repeat:
+    until: "condition"
+    coerce_to: boolean
+    print_response: true
+    model: "claude-3-haiku"
+    steps: [...]
+```
+
+## Benefits
+
+1. **Cleaner YAML** - No unnecessary nesting
+2. **More intuitive** - Configuration options are at the same level as other step properties
+3. **Consistent** - Matches how other step properties are specified
+
+## Supported Options
+
+All step configuration options can be specified directly:
+- `coerce_to` - Type coercion (boolean, llm_boolean, iterable)
+- `print_response` - Whether to print LLM responses
+- `loop` - Auto-loop behavior
+- `json` - JSON response mode
+- `params` - Additional parameters
+- `model` - Model override

data/examples/direct_coerce_syntax/workflow.yml

@@ -0,0 +1,36 @@
+name: Direct Coerce Syntax Demo
+description: Demonstrates the simplified coerce_to syntax without config blocks
+
+steps:
+  # Example 1: Direct coerce_to on repeat
+  - repeat:
+      until: "check_api_ready"
+      coerce_to: boolean  # Direct syntax - no config block needed
+      max_iterations: 5
+      steps:
+        - check_api_ready:
+            prompt: "Check if the API endpoint returns a 200 status"
+        - wait: 2
+  
+  # Example 2: Direct coerce_to on each  
+  - get_data_sources:
+      prompt: "List available data sources, one per line"
+  
+  - each: "get_data_sources"
+    as: "source"
+    coerce_to: iterable  # Direct syntax
+    steps:
+      - validate_source: "Validating {{source}}..."
+      - process_source:
+          prompt: "Process data from {{source}}"
+  
+  # Example 3: Multiple configuration options
+  - repeat:
+      until: "all_tests_pass"
+      coerce_to: llm_boolean  # Override default
+      print_response: true     # Other options work too
+      max_iterations: 10
+      steps:
+        - run_tests: "$(rake test)"
+        - all_tests_pass:
+            prompt: "Did all tests pass successfully?"

data/examples/grading/generate_recommendations/output.txt

@@ -1,16 +1,16 @@
 ========== TEST RECOMMENDATIONS ==========
-<%- if response["recommendations"].empty? -%>
+<%- if response.recommendations.empty? -%>
 No recommendations found.
 <%- else -%>
-<%- response["recommendations"].each_with_index do |rec, index| -%>
+<%- response.recommendations.each_with_index do |rec, index| -%>
 Recommendation #<%= index + 1 %>:
-Description: <%= rec["description"] %>
-Impact: <%= rec["impact"] %>
-Priority: <%= rec["priority"] %>
+Description: <%= rec.description %>
+Impact: <%= rec.impact %>
+Priority: <%= rec.priority %>
 
 Code Suggestion:
 
-<%= rec["code_suggestion"] %>
+<%= rec.code_suggestion %>
 
 <%- end -%>
 <%- end -%>

data/examples/grading/workflow.yml

@@ -1,5 +1,7 @@
 name: Test Grading
-model: anthropic:claude-opus-4
+api_token: $(echo $OPENAI_API_KEY)
+# model: anthropic:claude-opus-4
+model: gpt-4.1-mini
 
 tools:
   - Roast::Tools::Grep
@@ -23,16 +25,16 @@ steps:
 
 # set non-default attributes for steps below
 analyze_coverage:
-  model: gpt-4.1-mini
+#  model: gpt-4.1-mini
   auto_loop: false
   json: true
   
 generate_grades:
-  model: o3
+#  model: o3
   json: true
 
 generate_recommendations:
-  model: o3
+#  model: o3
   auto_loop: false
   json: true
   params:

data/examples/json_handling/README.md

@@ -0,0 +1,32 @@
+# JSON Handling Example
+
+This example demonstrates how Roast handles JSON responses from LLM steps.
+
+## Key Features
+
+1. **JSON Arrays**: When a step has `json: true` and returns an array, the result is `flatten.first` - the first element after flattening the array. This is useful when the LLM returns an array with a single object.
+
+2. **JSON Objects**: Hash/object responses are maintained as proper Ruby hashes in the workflow output.
+
+3. **Replay Support**: JSON data structures are properly serialized and deserialized when saving/loading workflow state for replay functionality.
+
+## Running the Example
+
+```bash
+bin/roast examples/json_handling/workflow.yml
+```
+
+## How It Works
+
+1. The `fetch_users` step generates a JSON array of user objects
+2. The `fetch_metadata` step generates a JSON object with metadata
+3. The `process_data` step can access the structured data using `{{output.fetch_users}}` and `{{output.fetch_metadata}}`
+4. The structured data is preserved through the workflow, including during replay scenarios
+
+## Implementation Details
+
+When `json: true` is set on a step:
+- Array responses return `flatten.first` - the first element after flattening
+- Object responses are preserved as hashes
+- Non-JSON array responses (when `json: false`) are joined with newlines
+- The data maintains its structure when saved to and loaded from workflow state files

data/examples/json_handling/workflow.yml

@@ -0,0 +1,52 @@
+name: JSON Data Handling Example
+api_provider: openai
+model: gpt-4
+
+tools:
+  - type: write_file
+    allowed_paths:
+      - examples/json_handling/
+
+steps:
+  - name: fetch_users
+    prompt: |
+      Generate a JSON array of 3 user objects. Each user should have:
+      - id (number)
+      - name (string)
+      - email (string)
+      - active (boolean)
+    json: true
+
+  - name: fetch_metadata
+    prompt: |
+      Generate a JSON object with metadata about a dataset:
+      - total_records (number)
+      - last_updated (ISO date string)
+      - categories (array of strings)
+      - filters (object with status and sort fields)
+    json: true
+
+  - name: process_data
+    prompt: |
+      Based on the users data: {{output.fetch_users}}
+      And metadata: {{output.fetch_metadata}}
+      
+      Create a summary report describing:
+      1. How many active users there are
+      2. The categories available
+      3. When the data was last updated
+
+  - name: save_report
+    tool: write_file
+    path: examples/json_handling/report.txt
+    content: |
+      # JSON Data Processing Report
+      
+      ## Users Data
+      {{output.fetch_users}}
+      
+      ## Metadata
+      {{output.fetch_metadata}}
+      
+      ## Summary
+      {{output.process_data}}

data/examples/pre_post_processing/README.md

@@ -0,0 +1,111 @@
+# Pre/Post Processing Example: Test Suite Optimization
+
+This example demonstrates how to use Roast's pre/post processing framework to optimize an entire test suite across multiple files.
+
+## Overview
+
+The workflow processes multiple test files, but performs setup and aggregation tasks only once:
+
+- **Pre-processing**: Runs once before any test files are processed
+  - Gathers baseline metrics for comparison
+  - Sets up the test environment
+  
+- **Main workflow**: Runs for each test file matching the target pattern
+  - Analyzes test quality and coverage
+  - Improves test coverage
+  - Optimizes test performance
+  - Validates changes
+  
+- **Post-processing**: Runs once after all test files have been processed
+  - Aggregates metrics from all files
+  - Generates a comprehensive report
+  - Cleans up the environment
+
+## Workflow Structure
+
+```yaml
+name: test_optimization
+model: gpt-4o
+target: "test/**/*_test.rb"
+
+pre_processing:
+  - gather_baseline_metrics
+  - setup_test_environment
+
+steps:
+  - analyze_test_file
+  - improve_test_coverage
+  - optimize_test_performance
+  - validate_changes
+
+post_processing:
+  - aggregate_metrics
+  - generate_summary_report
+  - cleanup_environment
+```
+
+## Directory Structure
+
+```
+pre_post_processing/
+├── workflow.yml
+├── pre_processing/
+│   ├── gather_baseline_metrics/
+│   │   └── prompt.md
+│   └── setup_test_environment/
+│       └── prompt.md
+├── analyze_test_file/
+│   └── prompt.md
+├── improve_test_coverage/
+│   └── prompt.md
+├── optimize_test_performance/
+│   └── prompt.md
+├── validate_changes/
+│   └── prompt.md
+└── post_processing/
+    ├── aggregate_metrics/
+    │   └── prompt.md
+    ├── generate_summary_report/
+    │   └── prompt.md
+    └── cleanup_environment/
+        └── prompt.md
+```
+
+## Key Features Demonstrated
+
+1. **Shared State**: Pre-processing results are available to all subsequent steps
+2. **Result Aggregation**: Post-processing has access to results from all workflow executions
+3. **One-time Operations**: Setup and cleanup happen only once, regardless of target count
+4. **Metrics Collection**: Each file's results are stored and aggregated for reporting
+
+## Running the Example
+
+```bash
+cd examples/pre_post_processing
+roast workflow.yml
+```
+
+This will:
+1. Run pre-processing steps once
+2. Process each test file matching `test/**/*_test.rb`
+3. Run post-processing steps once with access to all results
+4. Generate a comprehensive optimization report
+
+## Use Cases
+
+This pattern is ideal for:
+- **Code migrations**: Setup migration tools, process files, generate migration report
+- **Performance audits**: Baseline metrics, analyze files, aggregate improvements
+- **Documentation generation**: Analyze codebase, generate docs per file, create index
+- **Dependency updates**: Check current versions, update files, verify compatibility
+- **Security scanning**: Setup scanners, check each file, generate security report
+
+## Customization
+
+To adapt this example for your use case:
+
+1. Update the `target` pattern to match your files
+2. Modify pre-processing steps for your setup needs
+3. Adjust main workflow steps for your processing logic
+4. Customize post-processing for your reporting requirements
+5. Use appropriate AI models for each step type

data/examples/pre_post_processing/analyze_test_file/prompt.md

@@ -0,0 +1,23 @@
+# Analyze Test File
+
+Current test file: {{file}}
+
+Please analyze this test file and identify:
+
+1. **Test Structure**: Number of test cases, test suites, and overall organization
+2. **Coverage Gaps**: Areas of the code that aren't adequately tested
+3. **Test Quality Issues**:
+   - Tests that are too brittle or implementation-dependent
+   - Missing edge cases
+   - Unclear test descriptions
+   - Excessive mocking that reduces test value
+4. **Performance Issues**:
+   - Slow setup/teardown methods
+   - Inefficient test data generation
+   - Unnecessary database operations
+5. **Opportunities for Improvement**:
+   - Tests that could be parameterized
+   - Common patterns that could be extracted to helpers
+   - Better use of test fixtures or factories
+
+Provide specific, actionable recommendations for each issue found.

data/examples/pre_post_processing/improve_test_coverage/prompt.md

@@ -0,0 +1,17 @@
+# Improve Test Coverage
+
+Based on the analysis of {{file}}, implement the following improvements:
+
+1. **Add Missing Test Cases**: Write tests for uncovered code paths, edge cases, and error conditions
+2. **Improve Test Descriptions**: Make test names more descriptive and follow consistent naming conventions
+3. **Enhance Assertions**: Add more specific assertions to catch regressions
+4. **Test Data**: Use more realistic test data that better represents production scenarios
+5. **Remove Redundancy**: Eliminate duplicate tests or merge similar ones
+
+Generate the improved test code and explain the rationale for each change.
+
+Remember to:
+- Maintain backward compatibility with existing test interfaces
+- Follow the project's testing conventions and style guide
+- Ensure new tests are fast and deterministic
+- Add appropriate comments for complex test scenarios

data/examples/pre_post_processing/optimize_test_performance/prompt.md

@@ -0,0 +1,25 @@
+# Optimize Test Performance
+
+Optimize the performance of {{file}} by:
+
+1. **Reduce Setup Overhead**:
+   - Move expensive operations out of individual test setup
+   - Use shared fixtures where appropriate
+   - Lazy-load test data only when needed
+
+2. **Optimize Database Operations**:
+   - Use transactions for test isolation instead of truncation
+   - Minimize database queries in tests
+   - Use in-memory databases where possible
+
+3. **Improve Test Isolation**:
+   - Remove unnecessary dependencies between tests
+   - Clean up resources properly to avoid test pollution
+   - Use proper test doubles instead of hitting external services
+
+4. **Parallelize When Possible**:
+   - Identify tests that can run in parallel
+   - Remove shared state that prevents parallelization
+   - Group related tests for better cache utilization
+
+Generate the optimized test code and provide before/after performance metrics estimates.

data/examples/pre_post_processing/post_processing/aggregate_metrics/prompt.md

@@ -0,0 +1,31 @@
+# Aggregate Metrics
+
+Aggregate all the metrics collected during the workflow execution:
+
+Available data:
+- Pre-processing baseline metrics: {{pre_processing.gather_baseline_metrics}}
+- Results from all processed test files: {{output.targets}}
+
+Please calculate and provide:
+
+1. **Overall Coverage Improvement**:
+   - Total coverage before and after
+   - Percentage improvement
+   - Files with biggest improvements
+
+2. **Performance Gains**:
+   - Total execution time saved
+   - Average performance improvement per file
+   - Files with best optimization results
+
+3. **Test Quality Metrics**:
+   - Number of new tests added
+   - Number of tests optimized
+   - Reduction in flaky/brittle tests
+
+4. **Summary Statistics**:
+   - Total files processed
+   - Success rate
+   - Any files that had issues
+
+Output a comprehensive metrics summary that can be used in the final report.

data/examples/pre_post_processing/post_processing/cleanup_environment/prompt.md

@@ -0,0 +1,28 @@
+# Cleanup Environment
+
+Perform post-optimization cleanup tasks:
+
+1. **Commit Changes**: Create a commit with all the test improvements
+   - Use a descriptive commit message summarizing the optimization results
+   - Include key metrics in the commit description
+
+2. **Update Documentation**:
+   - Update test documentation if structure changed significantly
+   - Add notes about any new test helpers or patterns introduced
+
+3. **Clean Temporary Files**:
+   - Remove any temporary files created during optimization
+   - Clear test caches that were used for benchmarking
+
+4. **Final Verification**:
+   - Run the full test suite one more time to ensure everything works
+   - Verify CI/CD pipelines will work with the changes
+
+5. **Create PR Description**:
+   - Generate a pull request description template with:
+     - Summary of changes
+     - Key metrics improvements
+     - Any breaking changes or considerations
+     - Review checklist
+
+Output a summary of cleanup actions performed and any final notes for the team.

data/examples/pre_post_processing/post_processing/generate_summary_report/prompt.md

@@ -0,0 +1,32 @@
+# Generate Summary Report
+
+Generate a comprehensive test optimization report based on all the collected data:
+
+## Test Suite Optimization Report
+
+### Executive Summary
+Provide a high-level overview of the optimization results, key achievements, and any issues encountered.
+
+### Metrics Summary
+Include the aggregated metrics from the previous step:
+{{aggregate_metrics}}
+
+### Detailed Results by File
+For each processed test file, include:
+- File name and path
+- Coverage improvement
+- Performance improvement
+- Number of tests added/modified
+- Key changes made
+
+### Recommendations
+Based on the optimization results, provide:
+1. Further optimization opportunities
+2. Best practices observed that should be adopted project-wide
+3. Common patterns that could be extracted into shared utilities
+4. Testing strategy improvements
+
+### Next Steps
+Suggest follow-up actions to maintain and build upon these improvements.
+
+Format the report in Markdown for easy sharing and include visual indicators (✅ ❌ ⚠️) for quick scanning.

data/examples/pre_post_processing/post_processing/output.txt

@@ -0,0 +1,24 @@
+=== Test Optimization Summary Report ===
+Generated at: <%= Time.now.strftime("%Y-%m-%d %H:%M:%S") %>
+
+## Baseline Metrics
+<%= pre_processing.gather_baseline_metrics %>
+
+## Files Processed
+Total files: <%= targets.size %>
+
+<% targets.each do |file, target| %>
+### <%= file %>
+Analysis: <%= target.output.analyze_test_file %>
+Coverage improvements: <%= target.output.improve_test_coverage %>
+Performance optimizations: <%= target.output.optimize_test_performance %>
+<% end %>
+
+## Post-Processing Results
+### Aggregated Metrics
+<%= output.aggregate_metrics %>
+
+### Summary Report
+<%= output.generate_summary_report %>
+
+=== End of Report ===

data/examples/pre_post_processing/pre_processing/gather_baseline_metrics/prompt.md

@@ -0,0 +1,26 @@
+# Gather Baseline Metrics
+
+Analyze the current test suite and gather baseline metrics for comparison. Please provide:
+
+1. Total number of test files to be processed
+2. Current overall test coverage percentage
+3. Average test execution time across all files
+4. Number of tests by type (unit, integration, system)
+5. Any test files that are particularly slow (> 5 seconds)
+
+Store these metrics in the workflow state for later comparison in post-processing.
+
+Output format:
+```json
+{
+  "total_test_files": 0,
+  "overall_coverage": 0.0,
+  "average_execution_time": 0.0,
+  "test_counts": {
+    "unit": 0,
+    "integration": 0,
+    "system": 0
+  },
+  "slow_tests": []
+}
+```

data/examples/pre_post_processing/pre_processing/setup_test_environment/prompt.md

@@ -0,0 +1,11 @@
+# Setup Test Environment
+
+Prepare the test environment for optimization. Please:
+
+1. Ensure all test dependencies are installed
+2. Create a backup branch for safety: `test-optimization-backup-{{timestamp}}`
+3. Set up any necessary test databases or fixtures
+4. Configure test parallelization settings if available
+5. Clear any test caches that might affect benchmarking
+
+Return a summary of the setup steps completed and any warnings or issues encountered.

data/examples/pre_post_processing/validate_changes/prompt.md

@@ -0,0 +1,24 @@
+# Validate Changes
+
+Validate the changes made to {{file}}:
+
+1. **Run the updated tests** and ensure they all pass
+2. **Check coverage metrics** to verify improvements
+3. **Measure execution time** to confirm performance gains
+4. **Verify no regressions** were introduced
+5. **Ensure code style** follows project conventions
+
+Store the validation results in the workflow state:
+```json
+{
+  "file": "{{file}}",
+  "tests_passed": true,
+  "coverage_before": 0.0,
+  "coverage_after": 0.0,
+  "execution_time_before": 0.0,
+  "execution_time_after": 0.0,
+  "issues_found": []
+}
+```
+
+If any issues are found, provide recommendations for fixing them.

data/examples/pre_post_processing/workflow.yml

@@ -0,0 +1,21 @@
+name: test_optimization
+model: gpt-4o
+target: "test/**/*_test.rb"
+
+# Pre-processing steps run once before any test files are processed
+pre_processing:
+  - gather_baseline_metrics
+  - setup_test_environment
+
+# Main workflow steps run for each test file
+steps:
+  - analyze_test_file
+  - improve_test_coverage
+  - optimize_test_performance
+  - validate_changes
+
+# Post-processing steps run once after all test files have been processed
+post_processing:
+  - aggregate_metrics
+  - generate_summary_report
+  - cleanup_environment

data/examples/single_target_prepost/README.md

@@ -0,0 +1,36 @@
+# Single Target with Pre/Post Processing Example
+
+This example demonstrates how pre/post processing works with single-target workflows. Even when analyzing just one file, you can use pre-processing to gather context and post-processing to format results.
+
+## Features Demonstrated
+
+1. **Pre-processing for context gathering** - Analyze dependencies before the main workflow
+2. **Single-target analysis** - Focus on one specific file
+3. **Post-processing with output template** - Custom formatting of the final report
+
+## Running the Example
+
+```bash
+roast workflow.yml
+```
+
+This will:
+1. Run pre-processing to gather dependencies and context
+2. Analyze the single target file (src/main.rb)
+3. Apply the post-processing template to format the output
+
+## Output Template
+
+The `post_processing/output.txt` template demonstrates how to:
+- Access pre-processing results with `<%= pre_processing[:step_name] %>`
+- Iterate over target results (even for single targets)
+- Include post-processing step outputs
+- Format everything into a professional report
+
+## Use Cases
+
+This pattern is ideal for:
+- Deep analysis of critical files
+- Security audits of specific components
+- Performance profiling of key modules
+- Generating documentation for important classes

data/examples/single_target_prepost/post_processing/output.txt

@@ -0,0 +1,27 @@
+=== Code Analysis Report ===
+Generated: <%= Time.now.strftime("%Y-%m-%d %H:%M:%S") %>
+
+## Dependencies & Context
+<%= pre_processing.gather_dependencies %>
+
+## Target File Analysis
+<% targets.each do |file, target| %>
+File: <%= file %>
+
+### Code Quality
+<%= target.output.analyze_code_quality %>
+
+### Identified Improvements
+<%= target.output.identify_improvements %>
+
+### Recommendations
+<%= target.output.generate_recommendations %>
+<% end %>
+
+## Summary
+<%= output.summarize_findings %>
+
+## Action Items
+<%= output.create_action_items %>
+
+=== End of Report ===

data/examples/single_target_prepost/pre_processing/gather_dependencies/prompt.md

@@ -0,0 +1,11 @@
+# Gather Dependencies
+
+Analyze the project structure to understand the dependencies and context for <%= file %>.
+
+Tasks:
+1. List all files that import or depend on the target file
+2. Identify the key modules and classes used
+3. Note any external dependencies or libraries
+4. Summarize the architectural context
+
+Provide a structured summary that will help with the main analysis.