roast-ai 0.1.6 → 0.2.0

data/docs/ITERATION_SYNTAX.md

@@ -0,0 +1,119 @@
+# Using Iteration with Standardized Syntax
+
+## Overview
+
+Roast supports powerful iteration constructs with the `repeat` and `each` workflow steps. These features now support a standardized approach to evaluating expressions using the double-curly braces syntax (`{{...}}`).
+
+## Syntax Options for Iteration Inputs
+
+Both `until` conditions (in `repeat`) and collection expressions (in `each`) accept the following formats:
+
+### 1. Ruby Expressions with `{{...}}` Syntax
+
+For evaluating Ruby code in the workflow context:
+
+```yaml
+# Repeat until a condition is met
+- repeat:
+    steps:
+      - process_item
+    until: "{{output['counter'] >= 5}}"
+    max_iterations: 10
+
+# Iterate over a collection
+- each: "{{output['items'].filter { |item| item.active? }}}"
+  as: "current_item"
+  steps:
+    - process_item
+```
+
+### 2. Bash Commands with `$(...)` Syntax
+
+For executing shell commands and using their results:
+
+```yaml
+# Repeat until a command succeeds
+- repeat:
+    steps:
+      - check_service
+    until: "$(curl -s -o /dev/null -w '%{http_code}' http://service.local/ | grep -q 200)"
+    max_iterations: 20
+
+# Iterate over files returned by a command
+- each: "$(find . -name '*.rb' -type f)"
+  as: "current_file"
+  steps:
+    - process_file
+```
+
+### 3. Step Names (as strings)
+
+For using the result of another step:
+
+```yaml
+# Repeat until a step returns a truthy value
+- repeat:
+    steps:
+      - process_batch
+    until: "check_completion"
+    max_iterations: 100
+
+# Iterate over items returned by a step
+- each: "get_pending_items"
+  as: "pending_item"
+  steps:
+    - process_pending_item
+```
+
+### 4. Prompt Content
+
+For defining prompts directly in the workflow:
+
+```yaml
+# Using a prompt to determine continuation
+- repeat:
+    steps:
+      - process_content
+    until:
+      prompt: prompts/check_completion.md
+      model: claude-3-haiku
+    max_iterations: 10
+
+# Using a prompt to generate a collection
+- each:
+    prompt: prompts/generate_test_cases.md
+    model: claude-3-haiku
+  as: "test_case"
+  steps:
+    - run_test
+```
+
+## Type Coercion
+
+Values are automatically coerced to the appropriate types:
+
+- For `until` conditions: Values are coerced to booleans (using Ruby's truthiness rules)
+- For `each` collections: Values are coerced to iterables (converted to arrays of lines if needed)
+
+## Migrating Existing Workflows
+
+If you're updating existing workflows:
+
+1. For Ruby expressions, wrap them in `{{...}}`:
+   ```yaml
+   # Old
+   until: "output['counter'] >= 5"
+   
+   # New
+   until: "{{output['counter'] >= 5}}"
+   ```
+
+2. Bash commands, step names, and prompts can remain unchanged.
+
+## Best Practices
+
+- Use `{{...}}` for all Ruby expressions to make them explicit
+- For complex conditions, consider creating a dedicated step that returns a boolean
+- For collections, ensure they return iterable objects (arrays, hashes, etc.)
+- Always set reasonable `max_iterations` limits on repeat loops
+- Use meaningful variable names in `each` loops

data/examples/conditional/README.md

@@ -0,0 +1,161 @@
+# Conditional Execution in Roast Workflows
+
+This example demonstrates how to use conditional execution (`if` and `unless`) in Roast workflows.
+
+## Overview
+
+Conditional execution allows workflows to execute different steps based on runtime conditions. This feature supports:
+
+- `if` conditions - execute steps when a condition is true
+- `unless` conditions - execute steps when a condition is false  
+- `then` branches - steps to execute when the condition matches
+- `else` branches - steps to execute when the condition doesn't match (optional, only for `if`)
+
+## Syntax
+
+### If Statement
+
+```yaml
+- if: "{{expression}}"
+  then:
+    - step1
+    - step2
+  else:
+    - step3
+    - step4
+```
+
+### Unless Statement
+
+```yaml
+- unless: "{{expression}}"
+  then:
+    - step1
+    - step2
+```
+
+## Condition Types
+
+Conditions can be:
+
+1. **Ruby Expressions** - Wrapped in `{{...}}`
+   ```yaml
+   - if: "{{output.previous_step.success == true}}"
+   ```
+
+2. **Bash Commands** - Wrapped in `$(...)`
+   ```yaml
+   - if: "$(test -f /path/to/file && echo true || echo false)"
+   ```
+
+3. **Step References** - Reference to previous step output
+   ```yaml
+   - if: "check_condition"  # References a previous step
+   ```
+
+4. **File Checks**
+   ```yaml
+   - if: "{{File.exist?('/tmp/myfile.txt')}}"
+   ```
+
+## Examples
+
+### Basic Example
+
+```yaml
+name: Conditional Example
+tools:
+  - Roast::Tools::Cmd
+
+steps:
+  - check_status: "echo 'success'"
+  
+  - if: "{{output.check_status.strip == 'success'}}"
+    then:
+      - success_action: "echo 'Operation succeeded!'"
+    else:
+      - failure_action: "echo 'Operation failed!'"
+```
+
+### Unless Example
+
+```yaml
+name: Unless Example
+tools: []
+
+steps:
+  - check_file: "test -f /tmp/important.txt && echo exists || echo missing"
+  
+  - unless: "{{output.check_file.strip == 'exists'}}"
+    then:
+      - create_file: "touch /tmp/important.txt"
+      - notify: "echo 'Created missing file'"
+```
+
+### Nested Conditionals
+
+```yaml
+name: Nested Conditionals
+tools: []
+
+steps:
+  - outer_check: "echo 'true'"
+  - inner_check: "echo 'false'"
+  
+  - if: "{{output.outer_check.strip == 'true'}}"
+    then:
+      - if: "{{output.inner_check.strip == 'true'}}"
+        then:
+          - both_true: "echo 'Both conditions are true'"
+        else:
+          - only_outer: "echo 'Only outer condition is true'"
+    else:
+      - outer_false: "echo 'Outer condition is false'"
+```
+
+### Platform-Specific Actions
+
+```yaml
+name: Platform Detection
+tools:
+  - Roast::Tools::Cmd
+
+steps:
+  - detect_os: "uname -s"
+  
+  - if: "{{output.detect_os.strip == 'Darwin'}}"
+    then:
+      - mac_setup: "brew --version || echo 'Homebrew not installed'"
+    else:
+      - if: "{{output.detect_os.strip == 'Linux'}}"
+        then:
+          - linux_setup: "apt-get --version || yum --version"
+        else:
+          - unknown_os: "echo 'Unknown operating system'"
+```
+
+## Best Practices
+
+1. **Use Clear Conditions**: Make your conditions explicit and easy to understand
+2. **Handle Edge Cases**: Always consider what happens when conditions fail
+3. **Test Both Branches**: Ensure both `then` and `else` branches work correctly
+4. **Avoid Deep Nesting**: Keep conditional logic simple and readable
+5. **Use Unless Sparingly**: `unless` can be less intuitive than `if` with negation
+
+## Debugging
+
+To debug conditional execution:
+
+1. Check the workflow output to see which branch was executed
+2. Look for keys like `if_condition_name` or `unless_condition_name` in the output
+3. These keys contain information about the condition evaluation and branch taken
+
+## Running the Example
+
+```bash
+# Run the simple conditional example
+roast execute examples/conditional/simple_workflow.yml
+
+# Run the full conditional example (requires API configuration)
+roast execute examples/conditional/workflow.yml
+```

data/examples/conditional/check_condition/prompt.md

@@ -0,0 +1 @@
+Check if the OS is macOS or Linux and return true if it's macOS, false otherwise.

data/examples/conditional/simple_workflow.yml

@@ -0,0 +1,15 @@
+name: Simple Conditional Test
+tools: []
+
+steps:
+  - set_value: "$(echo 'true')"
+  
+  - if: "{{output.set_value.strip == 'true'}}"
+    then:
+      - success: "$(echo 'If condition worked!')"
+    else:
+      - failure: "$(echo 'If condition failed!')"
+  
+  - unless: "{{output.set_value.strip == 'false'}}"
+    then:
+      - unless_success: "$(echo 'Unless condition worked!')"

data/examples/conditional/workflow.yml

@@ -0,0 +1,23 @@
+name: Conditional Execution Example
+tools:
+  - Roast::Tools::Cmd
+
+steps:
+  - check_os: "$(uname -s)"
+  
+  - if: "{{output.check_os.strip == 'Darwin'}}"
+    then:
+      - mac_message: "$(echo 'Running on macOS!')"
+      - mac_info: "$(sw_vers)"
+    else:
+      - linux_message: "$(echo 'Running on Linux!')"
+      - linux_info: "$(lsb_release -a)"
+  
+  - check_file: "$(test -f /tmp/roast_test.txt && echo exists || echo missing)"
+  
+  - unless: "{{output.check_file.strip == 'exists'}}"
+    then:
+      - create_file: "$(touch /tmp/roast_test.txt && echo 'File created')"
+  
+  - verify_file: "$(ls -la /tmp/roast_test.txt)"
+  - cleanup: "$(rm -f /tmp/roast_test.txt)"

data/examples/dot_notation/README.md

@@ -0,0 +1,37 @@
+# Dot Notation Access Example
+
+This example demonstrates the new dot notation access feature for workflow outputs.
+
+## Usage
+
+With the new dot notation feature, you can access output values using Ruby's method syntax instead of hash syntax:
+
+### Before (hash syntax):
+```yaml
+until: "output[:update_fix_count][:fixes_applied] >= 5 || output[:select_next_issue][:no_issues_left] == true"
+```
+
+### After (dot notation):
+```yaml
+until: "output.update_fix_count.fixes_applied >= 5 || output.select_next_issue.no_issues_left?"
+```
+
+### Even cleaner (omitting output prefix):
+```yaml
+until: "update_fix_count.fixes_applied >= 5 || select_next_issue.no_issues_left?"
+```
+
+## Features
+
+1. **Nested access**: `output.step_name.nested.value`
+2. **Boolean predicates**: `output.step_name.is_complete?` returns false for nil/false values
+3. **Direct access**: Omit the `output.` prefix for cleaner syntax
+4. **Backward compatible**: Hash syntax still works (`output[:step_name][:value]`)
+
+## Example Workflow
+
+See `workflow.yml` for a complete example that demonstrates:
+- Setting values in output
+- Using dot notation in conditions
+- Boolean predicate methods
+- Nested value access

data/examples/dot_notation/workflow.yml

@@ -0,0 +1,44 @@
+name: dot_notation_example
+description: Example demonstrating dot notation access for workflow outputs
+
+steps:
+  initialize:
+    prompt: |
+      Initialize the workflow with some sample data.
+      
+      Set output.counter to 0
+      Set output.config.max_iterations to 5
+      Set output.config.enabled to true
+
+  process_items:
+    repeat:
+      # Using dot notation in conditions
+      until: "counter >= config.max_iterations || !config.enabled?"
+      steps:
+        - increment_counter
+        - check_status
+
+  increment_counter:
+    prompt: |
+      Increment the counter by 1.
+      Current counter value: {{counter}}
+      
+      Set output.counter to {{counter}} + 1
+
+  check_status:
+    prompt: |
+      Check if we should continue processing.
+      
+      Current counter: {{counter}}
+      Max iterations: {{config.max_iterations}}
+      
+      If counter is 3, set output.config.enabled to false
+      Set output.status.last_checked to current counter value
+
+  summarize:
+    prompt: |
+      Summarize the results:
+      - Total iterations: {{counter}}
+      - Last checked at: {{status.last_checked}}
+      - Was enabled: {{config.enabled}}
+      - Hit max iterations: {{counter >= config.max_iterations ? "Yes" : "No"}}

data/examples/exit_on_error/README.md

@@ -0,0 +1,50 @@
+# Exit on Error Example
+
+This example demonstrates how to use the `exit_on_error` configuration option to continue workflow execution even when a command fails.
+
+## Use Case
+
+When running a linter like RuboCop on a file with syntax errors or style violations, the command will exit with a non-zero status. By default, this would halt the workflow. However, we often want to:
+
+1. Capture the linter output (including errors)
+2. Analyze what went wrong
+3. Apply fixes based on the analysis
+
+## Configuration
+
+The key configuration is in the step configuration section:
+
+```yaml
+lint_check:
+  exit_on_error: false
+```
+
+This tells Roast to:
+- Continue workflow execution even if the command fails
+- Capture the full output (stdout and stderr)
+- Append the exit status to the output
+
+## Output Format
+
+When a command fails with `exit_on_error: false`, the output will look like:
+
+```
+lib/example.rb:5:3: C: Style/StringLiterals: Prefer double-quoted strings
+  'hello'
+  ^^^^^^^
+[Exit status: 1]
+```
+
+This allows subsequent steps to process both the error output and the exit status.
+
+## Running the Example
+
+```bash
+roast execute workflow.yml path/to/file.rb
+```
+
+The workflow will:
+1. Run RuboCop on the file
+2. Continue even if RuboCop finds issues
+3. Analyze the linter output
+4. Apply fixes based on the analysis

data/examples/exit_on_error/analyze_lint_output/prompt.md

@@ -0,0 +1,9 @@
+The linter output from the previous step shows issues with the code.
+Please analyze the output and identify the specific problems that need to be fixed.
+
+Focus on:
+- Syntax errors
+- Style violations
+- Best practice violations
+
+Provide a structured list of issues found.

data/examples/exit_on_error/apply_fixes/prompt.md

@@ -0,0 +1,2 @@
+Based on the analysis, please generate the fixes for the identified issues.
+Use the CodingAgent tool to apply the necessary changes to the file.

data/examples/exit_on_error/workflow.yml

@@ -0,0 +1,19 @@
+name: Linting with Error Recovery
+tools:
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::CodingAgent
+
+steps:
+  # Run linter on the file - may fail if there are syntax errors
+  - lint_check: $(rubocop {{file}})
+  
+  # Analyze linter output and fix issues even if linter failed
+  - analyze_lint_output
+  
+  # Apply fixes based on the analysis
+  - apply_fixes
+
+# Step configuration
+lint_check:
+  exit_on_error: false  # Continue even if rubocop exits with non-zero status

data/examples/grading/workflow.yml

@@ -1,10 +1,12 @@
-name: Grading current test changes
+name: Test Grading
+model: anthropic:claude-opus-4
 
 tools:
   - Roast::Tools::Grep
   - Roast::Tools::ReadFile
   - Roast::Tools::SearchFile
 
+# Uncomment this to run the workflow on modified tests automatically
 # each: '% cd $(git rev-parse --show-toplevel) && git status --porcelain | grep "_test\.rb" | cut -c4- | xargs realpath'
 
 steps:
@@ -35,3 +37,5 @@ generate_recommendations:
   json: true
   params:
     max_completion_tokens: 5_000
+
+

data/examples/iteration/IMPLEMENTATION.md

@@ -0,0 +1,88 @@
+# Iteration Mechanisms Implementation
+
+This document provides an overview of how the iteration mechanisms are implemented in Roast.
+
+## Core Components
+
+### 1. Schema Extensions
+
+The workflow schema has been extended to support two new iteration constructs:
+
+- **Repeat** - For conditional repetition until a condition is met
+- **Each** - For iterating over collections with a variable binding
+
+These schema extensions define the structure and validation rules for the iteration YAML syntax.
+
+### 2. Step Classes
+
+Two new step classes handle the actual iteration logic:
+
+- **RepeatStep** - Executes steps repeatedly until a condition is met or a maximum iteration count is reached
+- **EachStep** - Iterates over a collection, binding each item to a variable, and executes steps for each item
+
+Both inherit from a common `BaseIterationStep` class that provides shared functionality.
+
+### 3. Pattern Matching
+
+The `WorkflowExecutor` has been enhanced with pattern matching to recognize the `repeat` and `each` keywords and dispatch to the appropriate step classes:
+
+```ruby
+case name
+when "repeat"
+  execute_repeat_step(command)
+when "each"
+  # Handle 'each' step with its special format
+  execute_each_step(step)
+else
+  # Handle regular steps
+end
+```
+
+### 4. State Management
+
+Both iteration types include state management to:
+
+- Track current iteration number
+- Save state after each iteration
+- Support resumption after failures
+- Provide safety limits against infinite loops
+
+## Iteration Flow
+
+### RepeatStep Flow
+
+1. Start with iteration count = 0
+2. Execute the nested steps in sequence
+3. Evaluate the until condition
+4. If condition is true or max_iterations is reached, stop
+5. Otherwise, increment iteration count and go back to step 2
+6. Return the results of all iterations
+
+### EachStep Flow
+
+1. Resolve the collection expression
+2. For each item in the collection:
+   a. Set the named variable (accessible in steps through a getter method)
+   b. Execute the nested steps
+   c. Save state
+3. Return the results from all iterations
+
+## Safety Mechanisms
+
+- **max_iterations** parameter prevents infinite loops
+- State is saved after each iteration for resumption capability
+- Robust error handling during condition evaluation and step execution
+- Collection type checking ensures iterable objects
+
+## Usage Examples
+
+The workflow.yml and step files in this directory demonstrate practical applications of these iteration mechanisms for code quality analysis.
+
+## Integration with Existing Workflow Engine
+
+The iteration mechanism integrates seamlessly with the existing workflow engine:
+
+- Uses the same state persistence mechanisms
+- Follows the same execution models
+- Maintains compatibility with all existing steps
+- Supports interpolation within iteration constructs

data/examples/iteration/README.md

@@ -0,0 +1,68 @@
+# Code Quality Analysis Workflow
+
+This example demonstrates the use of Roast's iteration features to analyze and improve code quality across a codebase.
+
+## What it does
+
+1. Collects Ruby files from the codebase for analysis
+2. Analyzes each file for complexity, code smells, and potential improvements
+3. Generates recommendations for each file
+4. Prioritizes the identified issues by impact and difficulty
+5. Automatically implements fixes for the highest-priority issues
+6. Verifies each fix before moving to the next one
+7. Continues until either 5 fixes have been applied or all issues are addressed
+8. Generates a summary report of changes made
+
+## Iteration Features Demonstrated
+
+### Collection Iteration with `each`
+
+The workflow uses the `each` construct to iterate through Ruby files:
+
+```yaml
+- each: "output['get_files_to_analyze'].split('\n')"
+  as: "current_file"
+  steps:
+    - read_file
+    - analyze_complexity
+    - generate_recommendations
+```
+
+This makes the current file available as `current_file` in each step, allowing the analysis steps to process each file individually.
+
+### Conditional Repetition with `repeat`
+
+The workflow uses the `repeat` construct to iteratively fix issues until a condition is met:
+
+```yaml
+- repeat:
+    steps:
+      - select_next_issue
+      - implement_fix
+      - verify_fix
+      - update_fix_count
+    until: "output['update_fix_count']['fixes_applied'] >= 5 || output['select_next_issue']['no_issues_left'] == true"
+    max_iterations: 10
+```
+
+This continues applying fixes until either:
+- 5 fixes have been successfully applied
+- No more issues remain to be fixed
+- The maximum of 10 iterations is reached (safety limit)
+
+## Running the Example
+
+To run this workflow:
+
+```bash
+roast run examples/iteration/workflow.yml --target=/path/to/your/project
+```
+
+The workflow will analyze the Ruby files in your project, suggest improvements, and apply the highest-priority fixes.
+
+## Customizing
+
+- Adjust the file selection criteria in `get_files_to_analyze`
+- Modify the analysis criteria in `analyze_complexity`
+- Change the fix limit in the `until` condition
+- Set a different `max_iterations` value to control the maximum number of fixes

data/examples/iteration/analyze_complexity/prompt.md

@@ -0,0 +1,22 @@
+I'll analyze the code complexity of the file {{current_file}}, which I've read in the previous step.
+
+I'll examine the following aspects:
+1. Cyclomatic complexity (number of decision paths)
+2. Method length and complexity
+3. Class size and responsibilities
+4. Code smells (long parameter lists, deeply nested blocks, etc.)
+5. Potential performance bottlenecks
+6. Opportunities for refactoring
+
+```ruby
+{{output.read_file}}
+```
+
+Based on this analysis, I'll provide a structured assessment of the code quality issues found.
+
+For each issue identified, I'll assign:
+- Severity (high, medium, low)
+- Type (complexity, maintainability, performance, style)
+- Location (line numbers, method/class names)
+- Brief description of the issue
+- Suggested improvement