roast-ai 0.1.7 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 40bb03c596dcaa3dd49685c9b5f804ca6c6db00cf239d5e52748e6fa485f9c3b
-  data.tar.gz: c56399cca493aa9d925bb4d0550dc7ad0c0ed03b74b4f7894a3205ba72e66d1e
+  metadata.gz: 8ab27704514fc6ddfa09003464d2aa324b852379b0548619e2f0023bebe793ae
+  data.tar.gz: 18c0936f2b5dc2b54ef420be2ac089b8252f64a02227ab33d98db9d2e6249136
 SHA512:
-  metadata.gz: 2560cd52f21eb2aa0c954d1759ac762c12f2840a1533cc41017003a022f002212e9fb8d6e6b3d11fc1245ff413a86bd68d3aad7a142de9a30d4c534277a80f51
-  data.tar.gz: 371d573330873f1e76aec168e72ed8e2c53d0a127af83683205fae7d060dfa758aeb2352133b4f8b263803b5384b77be76b91a564d945cfd8966a682959a0d13
+  metadata.gz: 50552385cdb280592a1e331c4a3ec5de2924fa05636af5deb742b96f2fde229e3448e61e6516e570c02a8dc0f416a3b9a3c3d884a800eb89987c90c856a0258d
+  data.tar.gz: 8feeb510d87e6e6256c46cbca3e13fbc65701ceb28b049452e5bea2f44815f7770e688f8422487b431a85c5f459beffadcce590e7b860cde8a1ab4ee939ff50e

data/.github/workflows/ci.yaml

@@ -16,7 +16,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu]
-        ruby: ['3.4']
+        ruby: ['3.2', '3.3', '3.4']
     runs-on: ${{ matrix.os }}-latest
     continue-on-error: ${{ matrix.ruby == 'ruby-head' }}
     steps:

data/CHANGELOG.md

@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2025-05-26
+
+### Added
+- Conditional execution support with `if` and `unless` clauses for workflow steps
+- Iteration mechanisms for workflows with `repeat` and `each` constructs (resolving issue #48)
+- Support for conditional repetition with `until` condition and safety limits
+- Collection iteration with variable binding for processing lists of items
+- State persistence for loop iterations to enable resumption after failure
+- Standardized evaluation of Ruby expressions in iteration constructs using `{{}}` syntax
+- Support for using bash commands, step names, and Ruby expressions in iteration conditions and collections
+- Intelligent LLM response to boolean conversion with pattern-based recognition for natural language responses
+- `exit_on_error` configuration option for command steps to continue workflow on failure (resolving issue #71)
+- Dot notation access for workflow outputs (e.g., `workflow.output.step.field`)
+- `--pause` flag for stepping through workflow execution interactively
+
+### Fixed
+- Automatically add `.gitignore` file to cache directory when created (completing issue #22)
+- Load initializers before trying to load tools in case custom tools are defined in initializers (thanks @palkan)
+- Fix loading of targetless workflows (thanks @palkan)
+- Fix OpenRouter support (thanks @xrendan)
+- API authentication error handling and model access issues
+- Conditional step transcript replay regression
+- DotAccessHash serialization for AI prompts
+
+### Improved
+- Enhanced search file tool logging to show full expanded paths instead of relative paths
+- Major refactoring to eliminate circular dependencies and improve architecture
+- Extracted command execution logic into dedicated CommandExecutor class
+- Separated conditional execution from iteration logic for better SOLID compliance
+- Enhanced error messages for API authentication failures
+- Replaced all `require_relative` with `require` statements for consistency
+
+### Changed
+- Refactored god objects to improve code organization and maintainability
+- Improved separation of concerns between workflow components
+
+[0.2.0]: https://github.com/Shopify/roast/compare/v0.1.7...v0.2.0
+
 ## [0.1.7] - 2024-05-16
 
 ### Added
@@ -13,6 +51,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Comprehensive documentation for all built-in tools
 - Enhanced README with detailed tool usage examples
 
+``[0.1.7]: https://github.com/Shopify/roast/compare/v0.1.6...v0.1.7
+
 ## [0.1.6] - 2024-05-15
 
 ### Added
@@ -33,7 +73,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Improved initializer loading and error handling
 - Fixed tests for nested .roast folders
 
-[0.1.7]: https://github.com/Shopify/roast/compare/v0.1.6...v0.1.7
 [0.1.6]: https://github.com/Shopify/roast/compare/v0.1.5...v0.1.6
 
 ## [0.1.5] - 2024-05-13

data/CLAUDE.md

@@ -12,10 +12,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - Run single test: `bundle exec ruby -Itest test/path/to/test_file.rb`
 - Lint: `bundle exec rubocop`
 - Lint (with autocorrect, preferred): `bundle exec rubocop -A`
+- Whenever you want to run the whole test suite just run `bundle exec rake` to also run linting, and note the linting errors too (most will auto correct but not all)
+- **Run roast locally**: Use `bin/roast` (not `bundle exec roast` which may use the installed gem)
+- Alternative: `bundle exec exe/roast`
 
 ## Tech stack
 - `thor` and `cli-ui` for the CLI tool
 - Testing: Use Minitest, VCR for HTTP mocking, test files named with `_test.rb` suffix
+- Prefer using the more literate `test "this is a test description" do` type of testing that we get from extending ActiveSupport::TestCase over the primitive XUnit-style def test_description headings for tests
 
 ## Code Style Guidelines
 
@@ -30,6 +34,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - Add runtime dependencies to `roast.gemspec`.
 - Add development dependencies to `Gemfile`.
 - Don't ever test private methods directly. Specs should test behavior, not implementation.
+- I do not like test-specific code embedded in production code, don't ever do that
+- **Do not use require_relative**
+- Require statements should always be in alphabetical order
+- Always leave a blank line after module includes and before the rest of the class
+
+## Architecture Guidelines
+
+- **Maintain proper separation of concerns**: Don't mix unrelated concepts in the same class or module
+  - Example: Conditional execution (if/unless) should NOT be mixed with iteration execution (each/repeat)
+  - Each concept should have its own executor class and be handled separately
+- **Use appropriate inheritance**: Only inherit from a base class if the child truly "is-a" type of the parent
+  - Don't inherit just to reuse some methods - use composition instead
+- **Follow Single Responsibility Principle**: Each class should have one reason to change
+  - IterationExecutor handles iterations (each, repeat)
+  - ConditionalExecutor handles conditionals (if, unless)
+  - Don't combine different responsibilities in one class
 
 ## Git Workflow Practices
 

data/Gemfile

@@ -8,6 +8,7 @@ git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
 gemspec
 
 gem "cgi"
+gem "cli-ui"
 gem "dotenv"
 gem "guard"
 gem "guard-minitest"

data/Gemfile.lock

@@ -1,12 +1,13 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.1.7)
+    roast-ai (0.2.0)
       activesupport (~> 8.0)
+      cli-ui
       diff-lcs (~> 1.5)
       faraday-retry
       json-schema
-      raix (~> 0.8.4)
+      raix (~> 0.8.6)
       thor (~> 1.3)
 
 GEM
@@ -32,9 +33,10 @@ GEM
     benchmark (0.4.0)
     bigdecimal (3.1.9)
     cgi (0.4.2)
+    cli-ui (2.3.1)
     coderay (1.1.3)
     concurrent-ruby (1.3.5)
-    connection_pool (2.5.1)
+    connection_pool (2.5.3)
     crack (1.0.0)
       bigdecimal
       rexml
@@ -42,7 +44,7 @@ GEM
     dotenv (3.1.8)
     drb (2.2.1)
     event_stream_parser (1.0.0)
-    faraday (2.13.0)
+    faraday (2.13.1)
       faraday-net_http (>= 2.0, < 3.5)
       json
       logger
@@ -74,7 +76,7 @@ GEM
     hashdiff (1.1.2)
     i18n (1.14.7)
       concurrent-ruby (~> 1.0)
-    json (2.10.2)
+    json (2.12.0)
     json-schema (5.1.1)
       addressable (~> 2.8)
       bigdecimal (~> 3.1)
@@ -113,7 +115,7 @@ GEM
     public_suffix (6.0.1)
     racc (1.8.1)
     rainbow (3.1.1)
-    raix (0.8.4)
+    raix (0.8.6)
       activesupport (>= 6.0)
       faraday-retry (~> 2.0)
       open_router (~> 0.2)
@@ -168,6 +170,7 @@ PLATFORMS
 
 DEPENDENCIES
   cgi
+  cli-ui
   dotenv
   guard
   guard-minitest

data/README.md

@@ -46,7 +46,7 @@ Each step can have its own prompt file (e.g., `analyze_coverage/prompt.md`) and
 ```yaml
 steps:
   - prepare_data
-  - 
+  -
     - analyze_code_quality
     - check_test_coverage
     - verify_documentation
@@ -86,21 +86,21 @@ In Roast, workflows maintain a single conversation with the AI model throughout
 
 Roast supports several types of steps:
 
-1. **Standard step**: References a directory containing at least a `prompt.md` and optional `output.txt` template. This is the most common type of step. 
+1. **Standard step**: References a directory containing at least a `prompt.md` and optional `output.txt` template. This is the most common type of step.
   ```yaml
   steps:
     - analyze_code
   ```
 
   As an alternative to a directory, you can also implement a custom step as a Ruby class, optionally extending `Roast::Workflow::BaseStep`.
-  
+
   In the example given above, the script would live at `workflow/analyze_code.rb` and should contain a class named `AnalyzeCode` with an initializer that takes a workflow object as context, and a `call` method that will be invoked to run the step. The result of the `call` method will be stored in the `workflow.output` hash.
 
 
 2. **Parallel steps**: Groups of steps executed concurrently
    ```yaml
    steps:
-     - 
+     -
        - analyze_code_quality
        - check_test_coverage
    ```
@@ -113,7 +113,72 @@ Roast supports several types of steps:
    ```
    This will execute the command and store the result in the workflow output hash. Explicit key name is optional (`rubocop` in the second line of the example).
 
-4. **Raw prompt step**: Simple text prompts for the model without tools
+   By default, commands that exit with non-zero status will halt the workflow. You can configure steps to continue on error:
+   ```yaml
+   steps:
+     - lint_check: $(rubocop {{file}})
+     - fix_issues
+   
+   # Step configuration
+   lint_check:
+     exit_on_error: false  # Continue workflow even if command fails
+   ```
+   When `exit_on_error: false`, the command output will include the exit status, allowing subsequent steps to process error information.
+
+4. **Conditional steps**: Execute different steps based on conditions using `if/unless`
+   ```yaml
+   steps:
+     - check_environment:
+         if: "{{ENV['RAILS_ENV'] == 'production'}}"
+         then:
+           - run_production_checks
+           - notify_team
+         else:
+           - run_development_setup
+     
+     - verify_dependencies:
+         unless: "$(bundle check)"
+         then:
+           - bundle_install: "$(bundle install)"
+   ```
+   
+   Conditions can be:
+   - Ruby expressions: `if: "{{output['count'] > 5}}"`
+   - Bash commands: `if: "$(test -f config.yml && echo true)"` (exit code 0 = true)
+   - Step references: `if: "previous_step_name"` (uses the step's output)
+   - Direct values: `if: "true"` or `if: "false"`
+
+5. **Iteration steps**: Loop over collections or repeat steps with conditions
+   ```yaml
+   steps:
+     # Loop over a collection
+     - process_files:
+         each: "{{Dir.glob('**/*.rb')}}"
+         as: current_file
+         steps:
+           - analyze_file
+           - Generate a report for {{current_file}}
+     
+     # Repeat until a condition is met
+     - improve_code:
+         repeat:
+           until: "{{output['test_pass'] == true}}"
+           max_iterations: 5
+           steps:
+             - run_tests
+             - fix_issues
+   ```
+   
+   Each loops support:
+   - Collections from Ruby expressions: `each: "{{[1, 2, 3]}}"`
+   - Command output: `each: "$(ls *.rb)"`
+   - Step references: `each: "file_list"`
+   
+   Repeat loops support:
+   - Until conditions: `until: "{{condition}}"`
+   - Maximum iterations: `max_iterations: 10`
+
+6. **Raw prompt step**: Simple text prompts for the model without tools
    ```yaml
    steps:
      - Summarize the changes made to the codebase.
@@ -139,7 +204,7 @@ Roast handles data flow between steps in three primary ways:
      - Generate a summary for {{file}}
      - result_for_{{file}}: store_results
    ```
-   
+
    Interpolation supports:
    - Simple variable access: `{{file}}`, `{{resource.target}}`
    - Access to step outputs: `{{output['previous_step']}}`
@@ -150,7 +215,7 @@ For typical AI workflows, the continuous conversation history provides seamless
 ### Command Line Options
 
 #### Basic Options
-- `-o, --output FILE`: Save results to a file instead of outputting to STDOUT 
+- `-o, --output FILE`: Save results to a file instead of outputting to STDOUT
 - `-c, --concise`: Use concise output templates (exposed as a boolean flag on `workflow`)
 - `-v, --verbose`: Show output from all steps as they execute
 - `-r, --replay STEP_NAME`: Resume a workflow from a specific step, optionally with a specific session timestamp
@@ -241,7 +306,7 @@ roast execute workflow.yml -t "$(find . -name '*.rb' -mtime -1)"
 # Process changed test files
 roast execute workflow.yml -t "$(git diff --name-only HEAD | grep _test.rb)"
 
-# Process staged files  
+# Process staged files
 roast execute workflow.yml -t "$(git diff --cached --name-only)"
 ```
 
@@ -255,7 +320,7 @@ name: API Integration Workflow
 tools:
   - Roast::Tools::ReadFile
   - Roast::Tools::WriteFile
-  
+
 # Dynamic API token using shell command
 api_token: $(cat ~/.my_token)
 
@@ -349,7 +414,9 @@ api_provider: openrouter
 api_token: $(echo $OPENROUTER_API_KEY)
 ```
 
-This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines.
+
+This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines. Alternatively, Roast will fall back to `OPENROUTER_API_KEY` or `OPENAI_API_KEY` environment variables based on the specified provider.
+
 
 ### Template Output with ERB
 
@@ -372,7 +439,7 @@ This is an example of where the `workflow.output` hash is useful - formatting ou
 
 Available in templates:
 - `response`: The AI's response for this step
-- `workflow`: Access to the workflow object 
+- `workflow`: Access to the workflow object
 - `workflow.output`: The shared hash containing results from all steps when you need programmatic access
 - `workflow.file`: Current file being processed (or `nil` for targetless workflows)
 - All workflow configuration options
@@ -439,7 +506,7 @@ update_files(
     +new line
      line2
      line3
-    
+
     --- a/file2.txt
     +++ b/file2.txt
     @@ -5,7 +5,7 @@
@@ -556,11 +623,11 @@ module MyProject
 
       def call(commit_sha, include_diff = false)
         Roast::Helpers::Logger.info("🔍 Analyzing commit: #{commit_sha}\n")
-        
+
         # Your implementation here
         commit_info = `git show #{commit_sha} --stat`
         commit_info += "\n\n" + `git show #{commit_sha}` if include_diff
-        
+
         commit_info
       rescue StandardError => e
         "Error analyzing commit: #{e.message}".tap do |error_message|

data/bin/roast

@@ -0,0 +1,27 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'roast' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../.ruby-lsp/Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300).include?("This file was generated by Bundler")
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("roast-ai", "roast")

data/docs/ITERATION_SYNTAX.md

@@ -0,0 +1,119 @@
+# Using Iteration with Standardized Syntax
+
+## Overview
+
+Roast supports powerful iteration constructs with the `repeat` and `each` workflow steps. These features now support a standardized approach to evaluating expressions using the double-curly braces syntax (`{{...}}`).
+
+## Syntax Options for Iteration Inputs
+
+Both `until` conditions (in `repeat`) and collection expressions (in `each`) accept the following formats:
+
+### 1. Ruby Expressions with `{{...}}` Syntax
+
+For evaluating Ruby code in the workflow context:
+
+```yaml
+# Repeat until a condition is met
+- repeat:
+    steps:
+      - process_item
+    until: "{{output['counter'] >= 5}}"
+    max_iterations: 10
+
+# Iterate over a collection
+- each: "{{output['items'].filter { |item| item.active? }}}"
+  as: "current_item"
+  steps:
+    - process_item
+```
+
+### 2. Bash Commands with `$(...)` Syntax
+
+For executing shell commands and using their results:
+
+```yaml
+# Repeat until a command succeeds
+- repeat:
+    steps:
+      - check_service
+    until: "$(curl -s -o /dev/null -w '%{http_code}' http://service.local/ | grep -q 200)"
+    max_iterations: 20
+
+# Iterate over files returned by a command
+- each: "$(find . -name '*.rb' -type f)"
+  as: "current_file"
+  steps:
+    - process_file
+```
+
+### 3. Step Names (as strings)
+
+For using the result of another step:
+
+```yaml
+# Repeat until a step returns a truthy value
+- repeat:
+    steps:
+      - process_batch
+    until: "check_completion"
+    max_iterations: 100
+
+# Iterate over items returned by a step
+- each: "get_pending_items"
+  as: "pending_item"
+  steps:
+    - process_pending_item
+```
+
+### 4. Prompt Content
+
+For defining prompts directly in the workflow:
+
+```yaml
+# Using a prompt to determine continuation
+- repeat:
+    steps:
+      - process_content
+    until:
+      prompt: prompts/check_completion.md
+      model: claude-3-haiku
+    max_iterations: 10
+
+# Using a prompt to generate a collection
+- each:
+    prompt: prompts/generate_test_cases.md
+    model: claude-3-haiku
+  as: "test_case"
+  steps:
+    - run_test
+```
+
+## Type Coercion
+
+Values are automatically coerced to the appropriate types:
+
+- For `until` conditions: Values are coerced to booleans (using Ruby's truthiness rules)
+- For `each` collections: Values are coerced to iterables (converted to arrays of lines if needed)
+
+## Migrating Existing Workflows
+
+If you're updating existing workflows:
+
+1. For Ruby expressions, wrap them in `{{...}}`:
+   ```yaml
+   # Old
+   until: "output['counter'] >= 5"
+   
+   # New
+   until: "{{output['counter'] >= 5}}"
+   ```
+
+2. Bash commands, step names, and prompts can remain unchanged.
+
+## Best Practices
+
+- Use `{{...}}` for all Ruby expressions to make them explicit
+- For complex conditions, consider creating a dedicated step that returns a boolean
+- For collections, ensure they return iterable objects (arrays, hashes, etc.)
+- Always set reasonable `max_iterations` limits on repeat loops
+- Use meaningful variable names in `each` loops

data/examples/conditional/README.md

@@ -0,0 +1,161 @@
+# Conditional Execution in Roast Workflows
+
+This example demonstrates how to use conditional execution (`if` and `unless`) in Roast workflows.
+
+## Overview
+
+Conditional execution allows workflows to execute different steps based on runtime conditions. This feature supports:
+
+- `if` conditions - execute steps when a condition is true
+- `unless` conditions - execute steps when a condition is false  
+- `then` branches - steps to execute when the condition matches
+- `else` branches - steps to execute when the condition doesn't match (optional, only for `if`)
+
+## Syntax
+
+### If Statement
+
+```yaml
+- if: "{{expression}}"
+  then:
+    - step1
+    - step2
+  else:
+    - step3
+    - step4
+```
+
+### Unless Statement
+
+```yaml
+- unless: "{{expression}}"
+  then:
+    - step1
+    - step2
+```
+
+## Condition Types
+
+Conditions can be:
+
+1. **Ruby Expressions** - Wrapped in `{{...}}`
+   ```yaml
+   - if: "{{output.previous_step.success == true}}"
+   ```
+
+2. **Bash Commands** - Wrapped in `$(...)`
+   ```yaml
+   - if: "$(test -f /path/to/file && echo true || echo false)"
+   ```
+
+3. **Step References** - Reference to previous step output
+   ```yaml
+   - if: "check_condition"  # References a previous step
+   ```
+
+4. **File Checks**
+   ```yaml
+   - if: "{{File.exist?('/tmp/myfile.txt')}}"
+   ```
+
+## Examples
+
+### Basic Example
+
+```yaml
+name: Conditional Example
+tools:
+  - Roast::Tools::Cmd
+
+steps:
+  - check_status: "echo 'success'"
+  
+  - if: "{{output.check_status.strip == 'success'}}"
+    then:
+      - success_action: "echo 'Operation succeeded!'"
+    else:
+      - failure_action: "echo 'Operation failed!'"
+```
+
+### Unless Example
+
+```yaml
+name: Unless Example
+tools: []
+
+steps:
+  - check_file: "test -f /tmp/important.txt && echo exists || echo missing"
+  
+  - unless: "{{output.check_file.strip == 'exists'}}"
+    then:
+      - create_file: "touch /tmp/important.txt"
+      - notify: "echo 'Created missing file'"
+```
+
+### Nested Conditionals
+
+```yaml
+name: Nested Conditionals
+tools: []
+
+steps:
+  - outer_check: "echo 'true'"
+  - inner_check: "echo 'false'"
+  
+  - if: "{{output.outer_check.strip == 'true'}}"
+    then:
+      - if: "{{output.inner_check.strip == 'true'}}"
+        then:
+          - both_true: "echo 'Both conditions are true'"
+        else:
+          - only_outer: "echo 'Only outer condition is true'"
+    else:
+      - outer_false: "echo 'Outer condition is false'"
+```
+
+### Platform-Specific Actions
+
+```yaml
+name: Platform Detection
+tools:
+  - Roast::Tools::Cmd
+
+steps:
+  - detect_os: "uname -s"
+  
+  - if: "{{output.detect_os.strip == 'Darwin'}}"
+    then:
+      - mac_setup: "brew --version || echo 'Homebrew not installed'"
+    else:
+      - if: "{{output.detect_os.strip == 'Linux'}}"
+        then:
+          - linux_setup: "apt-get --version || yum --version"
+        else:
+          - unknown_os: "echo 'Unknown operating system'"
+```
+
+## Best Practices
+
+1. **Use Clear Conditions**: Make your conditions explicit and easy to understand
+2. **Handle Edge Cases**: Always consider what happens when conditions fail
+3. **Test Both Branches**: Ensure both `then` and `else` branches work correctly
+4. **Avoid Deep Nesting**: Keep conditional logic simple and readable
+5. **Use Unless Sparingly**: `unless` can be less intuitive than `if` with negation
+
+## Debugging
+
+To debug conditional execution:
+
+1. Check the workflow output to see which branch was executed
+2. Look for keys like `if_condition_name` or `unless_condition_name` in the output
+3. These keys contain information about the condition evaluation and branch taken
+
+## Running the Example
+
+```bash
+# Run the simple conditional example
+roast execute examples/conditional/simple_workflow.yml
+
+# Run the full conditional example (requires API configuration)
+roast execute examples/conditional/workflow.yml
+```

data/examples/conditional/check_condition/prompt.md

@@ -0,0 +1 @@
+Check if the OS is macOS or Linux and return true if it's macOS, false otherwise.