roast-ai 0.2.1 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: db58f659180113d7a2bb1bface8da2898be6717033107247156a464b4d14159d
-  data.tar.gz: f14c2caae89fab3353b66ae3eb4457c56975e65c464037a2d401cfca30840d81
+  metadata.gz: 64b0bfad5bc7ce9abd2d750ff52695ba997a750556a403e3ab4fc449dfb28946
+  data.tar.gz: 526beebc0ca0697df5ce2f871468fff246440615634ae516439b4c0740f8ce90
 SHA512:
-  metadata.gz: 88c3f83651c535077bdfe29408a19b874f66f19d8b7e743dd4933922f82fd7aee53ee215968e3d10c0207e92d4c53018a0b190475ba7ac4b70d15cda07a634ff
-  data.tar.gz: 5ebae3dcc5d7d775a3f19ab79ead81d7c11f53fc31b5b20f1db4781a950032e693a2d331fdc4509eb0661c04072f421defcd56d15b9d6ce74f7bfc12357ee1b0
+  metadata.gz: 0600537a7242638e683a884a0b22b4d5f7fd58bb02d0c7a04240c8720d197a1f4f5e2d5f44c1a7f0335506f7822b51b6c245fd1c5f25627c8a580319d63b1250
+  data.tar.gz: e368630d036c6c3ac6efe00d102ddbf89d438332916ed1adb5aa54fe6cb5f9b34f9adf637c6e36c5baab07015c6f0fb848f675476d4233accd005a98b9630642

data/.github/workflows/ci.yaml

@@ -17,10 +17,14 @@ jobs:
       matrix:
         os: [ubuntu]
         ruby: ['3.2', '3.3', '3.4']
+        gemfile: ['activesupport7', 'activesupport8']
     runs-on: ${{ matrix.os }}-latest
-    continue-on-error: ${{ matrix.ruby == 'ruby-head' }}
+    env:
+      BUNDLE_GEMFILE: ${{ github.workspace }}/gemfiles/${{ matrix.gemfile }}.gemfile
     steps:
       - uses: actions/checkout@v4
+      - name: Install ripgrep
+        run: sudo apt-get update && sudo apt-get install -y ripgrep
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: ${{ matrix.ruby }}

data/.gitignore

@@ -11,4 +11,32 @@
 **/.claude/settings.local.json
 
 **/CLAUDE.local.md
-.roast/
+.roast/
+
+bin/_guard-core
+bin/bundle
+bin/coderay
+bin/dotenv
+bin/erb
+bin/guard
+bin/htmldiff
+bin/irb
+bin/ldiff
+bin/listen
+bin/pry
+bin/racc
+bin/rake
+bin/rbs
+bin/rdbg
+bin/rdoc
+bin/ri
+bin/rubocop
+bin/ruby-lsp
+bin/ruby-lsp-check
+bin/ruby-lsp-launcher
+bin/ruby-lsp-test-exec
+bin/ruby-parse
+bin/ruby-rewrite
+bin/thor
+
+gemfiles/*.lock

data/CHANGELOG.md

@@ -5,6 +5,33 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.2] - 2025-05-29
+
+### Added
+- Pre/post processing framework for workflows with `pre_processing` and `post_processing` sections (#86)
+  - Support for `output.txt` ERB templates in post-processing phase for custom output formatting
+  - Pre/post processing support for single-target workflows (not just multi-target)
+  - Simplified access to pre-processing data in target workflows (removed `output` intermediary level)
+- Verbose mode improvements for better debugging experience (#98)
+  - Command outputs are now displayed when using the `--verbose` flag
+  - Commands executed within conditional branches also show output in verbose mode
+- User-friendly error reporting for workflow failures (#98)
+  - Clear ❌ indicators when commands or steps fail
+  - Command failures show exit status and output (no verbose needed for failures)
+  - Step failures provide helpful context about what might be wrong
+  - Exit handler displays actionable suggestions for resolving issues
+- Automatic workflow discovery by name (#97)
+  - Can now run workflows by name without full path: `roast execute my_workflow`
+  - Automatically looks for `roast/my_workflow/workflow.yml` in current directory
+- Configurable base URI for API endpoints (#83)
+
+### Fixed
+- Search file tool now correctly prefixes paths when searching (#92)
+- Support for Ruby projects using ActiveSupport 7.0+ (#95)
+
+### Changed
+- ActiveSupport dependency relaxed to >= 7.0 for broader compatibility
+
 ## [0.2.1]
 
 ### Added

data/CLAUDE.md

@@ -50,6 +50,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
   - IterationExecutor handles iterations (each, repeat)
   - ConditionalExecutor handles conditionals (if, unless)
   - Don't combine different responsibilities in one class
+- **Do not implement prompts "inline" using a prompt: attribute nested under step names, that violates the primary design architecture of Roast**
+
+## Guidance and Expectations
+
+- Do not decide unilaterally to leave code for the sake of "backwards compatibility"... always run those decisions by me first.
 
 ## Git Workflow Practices
 

data/Gemfile.lock

@@ -1,8 +1,8 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.2.1)
-      activesupport (~> 8.0)
+    roast-ai (0.2.2)
+      activesupport (>= 7.0)
       cli-ui
       diff-lcs (~> 1.5)
       faraday-retry
@@ -13,7 +13,7 @@ PATH
 GEM
   remote: https://rubygems.org/
   specs:
-    activesupport (8.0.2)
+    activesupport (7.2.2.1)
       base64
       benchmark (>= 0.3)
       bigdecimal
@@ -25,7 +25,6 @@ GEM
       minitest (>= 5.1)
       securerandom (>= 0.3)
       tzinfo (~> 2.0, >= 2.0.5)
-      uri (>= 0.13.1)
     addressable (2.8.7)
       public_suffix (>= 2.0.2, < 7.0)
     ast (2.4.3)

data/README.md

@@ -124,6 +124,9 @@ roast execute workflow.yml target_file.rb
 
 # Or for a targetless workflow (API calls, data generation, etc.)
 roast execute workflow.yml
+
+# Roast will automatically search in `project_root/roast/workflow_name` if the path is incomplete.
+roast execute my_cool_workflow # Equivalent to `roast execute roast/my_cool_workflow/workflow.yml
 ```
 
 ### Understanding Workflows
@@ -476,9 +479,9 @@ Benefits of using OpenRouter:
 
 When using OpenRouter, specify fully qualified model names including the provider prefix (e.g., `anthropic/claude-3-opus-20240229`).
 
-#### Dynamic API Tokens
+#### Dynamic API Tokens and URIs
 
-Roast allows you to dynamically fetch API tokens using shell commands directly in your workflow configuration:
+Roast allows you to dynamically fetch attributes such as API token and URI base (to use with a proxy) via shell commands directly in your workflow configuration:
 
 ```yaml
 # This will execute the shell command and use the result as the API token
@@ -490,8 +493,13 @@ api_token: $(echo $OPENAI_API_KEY)
 # For OpenRouter (requires api_provider setting)
 api_provider: openrouter
 api_token: $(echo $OPENROUTER_API_KEY)
-```
 
+# Static Proxy URI
+uri_base: https://proxy.example.com/v1
+
+# Dynamic Proxy URI
+uri_base: $(echo $AI_PROXY_URI_BASE)
+```
 
 This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines. Alternatively, Roast will fall back to `OPENROUTER_API_KEY` or `OPENAI_API_KEY` environment variables based on the specified provider.
 
@@ -746,6 +754,157 @@ your-project/
   └── ...
 ```
 
+### Pre/Post Processing Framework
+
+Roast supports pre-processing and post-processing phases for workflows. This enables powerful workflows that need setup/teardown or result aggregation across all processed files.
+
+#### Overview
+
+- **Pre-processing**: Steps executed once before any targets are processed
+- **Post-processing**: Steps executed once after all targets have been processed
+- **Shared state**: Pre-processing results are available to all subsequent steps
+- **Result aggregation**: Post-processing has access to all workflow execution results
+- **Single-target support**: Pre/post processing works with single-target workflows too
+- **Output templates**: Post-processing supports `output.txt` templates for custom formatting
+
+#### Configuration
+
+```yaml
+name: optimize_tests
+model: gpt-4o
+target: "test/**/*_test.rb"
+
+# Pre-processing steps run once before any test files
+pre_processing:
+  - gather_baseline_metrics
+  - setup_test_environment
+
+# Main workflow steps run for each test file
+steps:
+  - analyze_test
+  - improve_coverage
+  - optimize_performance
+
+# Post-processing steps run once after all test files
+post_processing:
+  - aggregate_results
+  - generate_report
+  - cleanup_environment
+```
+
+#### Directory Structure
+
+Pre and post-processing steps follow the same conventions as regular steps but are organized in their own directories:
+
+```
+workflow.yml
+pre_processing/
+  ├── gather_baseline_metrics/
+  │   └── prompt.md
+  └── setup_test_environment/
+      └── prompt.md
+analyze_test/
+  └── prompt.md
+improve_coverage/
+  └── prompt.md
+optimize_performance/
+  └── prompt.md
+post_processing/
+  ├── output.txt
+  ├── aggregate_results/
+  │   └── prompt.md
+  ├── generate_report/
+  │   └── prompt.md
+  └── cleanup_environment/
+      └── prompt.md
+```
+
+#### Data Access
+
+**Pre-processing results in target workflows:**
+
+Target workflows have access to pre-processing results through the `pre_processing_data` variable with dot notation:
+
+```erb
+# In a target workflow step prompt
+The baseline metrics from pre-processing:
+<%= pre_processing_data.gather_baseline_metrics %>
+
+Environment setup details:
+<%= pre_processing_data.setup_test_environment %>
+```
+
+**Post-processing data access:**
+
+Post-processing steps have access to:
+
+- `pre_processing`: Direct access to pre-processing results with dot notation
+- `targets`: Hash of all target workflow results, keyed by file paths
+
+Example post-processing prompt:
+```markdown
+# Generate Summary Report
+
+Based on the baseline metrics:
+<%= pre_processing.gather_baseline_metrics %>
+
+Environment configuration:
+<%= pre_processing.setup_test_environment %>
+
+And the results from processing all files:
+<% targets.each do |file, target| %>
+File: <%= file %>
+Analysis results: <%= target.output.analyze_test %>
+Coverage improvements: <%= target.output.improve_coverage %>
+Performance optimizations: <%= target.output.optimize_performance %>
+<% end %>
+
+Please generate a comprehensive summary report showing:
+1. Overall improvements achieved
+2. Files with the most significant changes
+3. Recommendations for further optimization
+```
+
+#### Output Templates
+
+Post-processing supports custom output formatting using ERB templates. Create an `output.txt` file in your `post_processing` directory to format the final workflow output:
+
+```erb
+# post_processing/output.txt
+=== Workflow Summary Report ===
+Generated at: <%= Time.now.strftime("%Y-%m-%d %H:%M:%S") %>
+
+Environment: <%= pre_processing.setup_test_environment %>
+
+Files Processed: <%= targets.size %>
+
+<% targets.each do |file, target| %>
+- <%= file %>: <%= target.output.analyze_test %>
+<% end %>
+
+<%= output.generate_report %>
+===============================
+```
+
+The template has access to:
+- `pre_processing`: All pre-processing step outputs with dot notation
+- `targets`: Hash of all target workflow results with dot notation (each target has `.output` and `.final_output`)
+- `output`: Post-processing step outputs with dot notation
+
+#### Use Cases
+
+This pattern is ideal for:
+
+- **Code migrations**: Setup migration tools, process files, generate migration report
+- **Test optimization**: Baseline metrics, optimize tests, aggregate improvements
+- **Documentation generation**: Analyze codebase, generate docs per module, create index
+- **Dependency updates**: Check versions, update files, verify compatibility
+- **Security audits**: Setup scanners, check each file, generate security report
+- **Performance analysis**: Establish baselines, analyze components, summarize findings
+
+See the [pre/post processing example](examples/pre_post_processing) for a complete working demonstration.
+
+
 ## Development
 
 After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake` to run the tests and linter. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/examples/grading/generate_recommendations/output.txt

@@ -1,16 +1,16 @@
 ========== TEST RECOMMENDATIONS ==========
-<%- if response["recommendations"].empty? -%>
+<%- if response.recommendations.empty? -%>
 No recommendations found.
 <%- else -%>
-<%- response["recommendations"].each_with_index do |rec, index| -%>
+<%- response.recommendations.each_with_index do |rec, index| -%>
 Recommendation #<%= index + 1 %>:
-Description: <%= rec["description"] %>
-Impact: <%= rec["impact"] %>
-Priority: <%= rec["priority"] %>
+Description: <%= rec.description %>
+Impact: <%= rec.impact %>
+Priority: <%= rec.priority %>
 
 Code Suggestion:
 
-<%= rec["code_suggestion"] %>
+<%= rec.code_suggestion %>
 
 <%- end -%>
 <%- end -%>

data/examples/pre_post_processing/README.md

@@ -0,0 +1,111 @@
+# Pre/Post Processing Example: Test Suite Optimization
+
+This example demonstrates how to use Roast's pre/post processing framework to optimize an entire test suite across multiple files.
+
+## Overview
+
+The workflow processes multiple test files, but performs setup and aggregation tasks only once:
+
+- **Pre-processing**: Runs once before any test files are processed
+  - Gathers baseline metrics for comparison
+  - Sets up the test environment
+  
+- **Main workflow**: Runs for each test file matching the target pattern
+  - Analyzes test quality and coverage
+  - Improves test coverage
+  - Optimizes test performance
+  - Validates changes
+  
+- **Post-processing**: Runs once after all test files have been processed
+  - Aggregates metrics from all files
+  - Generates a comprehensive report
+  - Cleans up the environment
+
+## Workflow Structure
+
+```yaml
+name: test_optimization
+model: gpt-4o
+target: "test/**/*_test.rb"
+
+pre_processing:
+  - gather_baseline_metrics
+  - setup_test_environment
+
+steps:
+  - analyze_test_file
+  - improve_test_coverage
+  - optimize_test_performance
+  - validate_changes
+
+post_processing:
+  - aggregate_metrics
+  - generate_summary_report
+  - cleanup_environment
+```
+
+## Directory Structure
+
+```
+pre_post_processing/
+├── workflow.yml
+├── pre_processing/
+│   ├── gather_baseline_metrics/
+│   │   └── prompt.md
+│   └── setup_test_environment/
+│       └── prompt.md
+├── analyze_test_file/
+│   └── prompt.md
+├── improve_test_coverage/
+│   └── prompt.md
+├── optimize_test_performance/
+│   └── prompt.md
+├── validate_changes/
+│   └── prompt.md
+└── post_processing/
+    ├── aggregate_metrics/
+    │   └── prompt.md
+    ├── generate_summary_report/
+    │   └── prompt.md
+    └── cleanup_environment/
+        └── prompt.md
+```
+
+## Key Features Demonstrated
+
+1. **Shared State**: Pre-processing results are available to all subsequent steps
+2. **Result Aggregation**: Post-processing has access to results from all workflow executions
+3. **One-time Operations**: Setup and cleanup happen only once, regardless of target count
+4. **Metrics Collection**: Each file's results are stored and aggregated for reporting
+
+## Running the Example
+
+```bash
+cd examples/pre_post_processing
+roast workflow.yml
+```
+
+This will:
+1. Run pre-processing steps once
+2. Process each test file matching `test/**/*_test.rb`
+3. Run post-processing steps once with access to all results
+4. Generate a comprehensive optimization report
+
+## Use Cases
+
+This pattern is ideal for:
+- **Code migrations**: Setup migration tools, process files, generate migration report
+- **Performance audits**: Baseline metrics, analyze files, aggregate improvements
+- **Documentation generation**: Analyze codebase, generate docs per file, create index
+- **Dependency updates**: Check current versions, update files, verify compatibility
+- **Security scanning**: Setup scanners, check each file, generate security report
+
+## Customization
+
+To adapt this example for your use case:
+
+1. Update the `target` pattern to match your files
+2. Modify pre-processing steps for your setup needs
+3. Adjust main workflow steps for your processing logic
+4. Customize post-processing for your reporting requirements
+5. Use appropriate AI models for each step type

data/examples/pre_post_processing/analyze_test_file/prompt.md

@@ -0,0 +1,23 @@
+# Analyze Test File
+
+Current test file: {{file}}
+
+Please analyze this test file and identify:
+
+1. **Test Structure**: Number of test cases, test suites, and overall organization
+2. **Coverage Gaps**: Areas of the code that aren't adequately tested
+3. **Test Quality Issues**:
+   - Tests that are too brittle or implementation-dependent
+   - Missing edge cases
+   - Unclear test descriptions
+   - Excessive mocking that reduces test value
+4. **Performance Issues**:
+   - Slow setup/teardown methods
+   - Inefficient test data generation
+   - Unnecessary database operations
+5. **Opportunities for Improvement**:
+   - Tests that could be parameterized
+   - Common patterns that could be extracted to helpers
+   - Better use of test fixtures or factories
+
+Provide specific, actionable recommendations for each issue found.

data/examples/pre_post_processing/improve_test_coverage/prompt.md

@@ -0,0 +1,17 @@
+# Improve Test Coverage
+
+Based on the analysis of {{file}}, implement the following improvements:
+
+1. **Add Missing Test Cases**: Write tests for uncovered code paths, edge cases, and error conditions
+2. **Improve Test Descriptions**: Make test names more descriptive and follow consistent naming conventions
+3. **Enhance Assertions**: Add more specific assertions to catch regressions
+4. **Test Data**: Use more realistic test data that better represents production scenarios
+5. **Remove Redundancy**: Eliminate duplicate tests or merge similar ones
+
+Generate the improved test code and explain the rationale for each change.
+
+Remember to:
+- Maintain backward compatibility with existing test interfaces
+- Follow the project's testing conventions and style guide
+- Ensure new tests are fast and deterministic
+- Add appropriate comments for complex test scenarios

data/examples/pre_post_processing/optimize_test_performance/prompt.md

@@ -0,0 +1,25 @@
+# Optimize Test Performance
+
+Optimize the performance of {{file}} by:
+
+1. **Reduce Setup Overhead**:
+   - Move expensive operations out of individual test setup
+   - Use shared fixtures where appropriate
+   - Lazy-load test data only when needed
+
+2. **Optimize Database Operations**:
+   - Use transactions for test isolation instead of truncation
+   - Minimize database queries in tests
+   - Use in-memory databases where possible
+
+3. **Improve Test Isolation**:
+   - Remove unnecessary dependencies between tests
+   - Clean up resources properly to avoid test pollution
+   - Use proper test doubles instead of hitting external services
+
+4. **Parallelize When Possible**:
+   - Identify tests that can run in parallel
+   - Remove shared state that prevents parallelization
+   - Group related tests for better cache utilization
+
+Generate the optimized test code and provide before/after performance metrics estimates.

data/examples/pre_post_processing/post_processing/aggregate_metrics/prompt.md

@@ -0,0 +1,31 @@
+# Aggregate Metrics
+
+Aggregate all the metrics collected during the workflow execution:
+
+Available data:
+- Pre-processing baseline metrics: {{pre_processing.gather_baseline_metrics}}
+- Results from all processed test files: {{output.targets}}
+
+Please calculate and provide:
+
+1. **Overall Coverage Improvement**:
+   - Total coverage before and after
+   - Percentage improvement
+   - Files with biggest improvements
+
+2. **Performance Gains**:
+   - Total execution time saved
+   - Average performance improvement per file
+   - Files with best optimization results
+
+3. **Test Quality Metrics**:
+   - Number of new tests added
+   - Number of tests optimized
+   - Reduction in flaky/brittle tests
+
+4. **Summary Statistics**:
+   - Total files processed
+   - Success rate
+   - Any files that had issues
+
+Output a comprehensive metrics summary that can be used in the final report.

data/examples/pre_post_processing/post_processing/cleanup_environment/prompt.md

@@ -0,0 +1,28 @@
+# Cleanup Environment
+
+Perform post-optimization cleanup tasks:
+
+1. **Commit Changes**: Create a commit with all the test improvements
+   - Use a descriptive commit message summarizing the optimization results
+   - Include key metrics in the commit description
+
+2. **Update Documentation**:
+   - Update test documentation if structure changed significantly
+   - Add notes about any new test helpers or patterns introduced
+
+3. **Clean Temporary Files**:
+   - Remove any temporary files created during optimization
+   - Clear test caches that were used for benchmarking
+
+4. **Final Verification**:
+   - Run the full test suite one more time to ensure everything works
+   - Verify CI/CD pipelines will work with the changes
+
+5. **Create PR Description**:
+   - Generate a pull request description template with:
+     - Summary of changes
+     - Key metrics improvements
+     - Any breaking changes or considerations
+     - Review checklist
+
+Output a summary of cleanup actions performed and any final notes for the team.

data/examples/pre_post_processing/post_processing/generate_summary_report/prompt.md

@@ -0,0 +1,32 @@
+# Generate Summary Report
+
+Generate a comprehensive test optimization report based on all the collected data:
+
+## Test Suite Optimization Report
+
+### Executive Summary
+Provide a high-level overview of the optimization results, key achievements, and any issues encountered.
+
+### Metrics Summary
+Include the aggregated metrics from the previous step:
+{{aggregate_metrics}}
+
+### Detailed Results by File
+For each processed test file, include:
+- File name and path
+- Coverage improvement
+- Performance improvement
+- Number of tests added/modified
+- Key changes made
+
+### Recommendations
+Based on the optimization results, provide:
+1. Further optimization opportunities
+2. Best practices observed that should be adopted project-wide
+3. Common patterns that could be extracted into shared utilities
+4. Testing strategy improvements
+
+### Next Steps
+Suggest follow-up actions to maintain and build upon these improvements.
+
+Format the report in Markdown for easy sharing and include visual indicators (✅ ❌ ⚠️) for quick scanning.

data/examples/pre_post_processing/post_processing/output.txt

@@ -0,0 +1,24 @@
+=== Test Optimization Summary Report ===
+Generated at: <%= Time.now.strftime("%Y-%m-%d %H:%M:%S") %>
+
+## Baseline Metrics
+<%= pre_processing.gather_baseline_metrics %>
+
+## Files Processed
+Total files: <%= targets.size %>
+
+<% targets.each do |file, target| %>
+### <%= file %>
+Analysis: <%= target.output.analyze_test_file %>
+Coverage improvements: <%= target.output.improve_test_coverage %>
+Performance optimizations: <%= target.output.optimize_test_performance %>
+<% end %>
+
+## Post-Processing Results
+### Aggregated Metrics
+<%= output.aggregate_metrics %>
+
+### Summary Report
+<%= output.generate_summary_report %>
+
+=== End of Report ===

data/examples/pre_post_processing/pre_processing/gather_baseline_metrics/prompt.md

@@ -0,0 +1,26 @@
+# Gather Baseline Metrics
+
+Analyze the current test suite and gather baseline metrics for comparison. Please provide:
+
+1. Total number of test files to be processed
+2. Current overall test coverage percentage
+3. Average test execution time across all files
+4. Number of tests by type (unit, integration, system)
+5. Any test files that are particularly slow (> 5 seconds)
+
+Store these metrics in the workflow state for later comparison in post-processing.
+
+Output format:
+```json
+{
+  "total_test_files": 0,
+  "overall_coverage": 0.0,
+  "average_execution_time": 0.0,
+  "test_counts": {
+    "unit": 0,
+    "integration": 0,
+    "system": 0
+  },
+  "slow_tests": []
+}
+```