roast-ai 0.1.6 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9bdaada020320af4c7fead86184a7cd0b1c06a89e88bc15ac424b31f4b449fb6
-  data.tar.gz: 3f83f410a1f21992de59b19b8e2f0ea402b9fe50b314b82514723cf20f9a599f
+  metadata.gz: 8ab27704514fc6ddfa09003464d2aa324b852379b0548619e2f0023bebe793ae
+  data.tar.gz: 18c0936f2b5dc2b54ef420be2ac089b8252f64a02227ab33d98db9d2e6249136
 SHA512:
-  metadata.gz: c6dfdd57104e363735a6e24bf5beb43a042340f582f94d8126c3f8a0d30dd92b031c5201a7d3d136bfcd3b7095a51c49329b08c07c5c19544240b18e20ad9948
-  data.tar.gz: 02aec8d620c3fe0c57d0d4e06790a7513a4a1aa44f68462721b6aa7da3f22f377598b4c00453e4ec825b0adcac3c335d470f945599d39571edb67e8a380597ac
+  metadata.gz: 50552385cdb280592a1e331c4a3ec5de2924fa05636af5deb742b96f2fde229e3448e61e6516e570c02a8dc0f416a3b9a3c3d884a800eb89987c90c856a0258d
+  data.tar.gz: 8feeb510d87e6e6256c46cbca3e13fbc65701ceb28b049452e5bea2f44815f7770e688f8422487b431a85c5f459beffadcce590e7b860cde8a1ab4ee939ff50e

data/.github/workflows/ci.yaml

@@ -16,7 +16,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu]
-        ruby: ['3.4']
+        ruby: ['3.2', '3.3', '3.4']
     runs-on: ${{ matrix.os }}-latest
     continue-on-error: ${{ matrix.ruby == 'ruby-head' }}
     steps:

data/CHANGELOG.md

@@ -5,6 +5,54 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2025-05-26
+
+### Added
+- Conditional execution support with `if` and `unless` clauses for workflow steps
+- Iteration mechanisms for workflows with `repeat` and `each` constructs (resolving issue #48)
+- Support for conditional repetition with `until` condition and safety limits
+- Collection iteration with variable binding for processing lists of items
+- State persistence for loop iterations to enable resumption after failure
+- Standardized evaluation of Ruby expressions in iteration constructs using `{{}}` syntax
+- Support for using bash commands, step names, and Ruby expressions in iteration conditions and collections
+- Intelligent LLM response to boolean conversion with pattern-based recognition for natural language responses
+- `exit_on_error` configuration option for command steps to continue workflow on failure (resolving issue #71)
+- Dot notation access for workflow outputs (e.g., `workflow.output.step.field`)
+- `--pause` flag for stepping through workflow execution interactively
+
+### Fixed
+- Automatically add `.gitignore` file to cache directory when created (completing issue #22)
+- Load initializers before trying to load tools in case custom tools are defined in initializers (thanks @palkan)
+- Fix loading of targetless workflows (thanks @palkan)
+- Fix OpenRouter support (thanks @xrendan)
+- API authentication error handling and model access issues
+- Conditional step transcript replay regression
+- DotAccessHash serialization for AI prompts
+
+### Improved
+- Enhanced search file tool logging to show full expanded paths instead of relative paths
+- Major refactoring to eliminate circular dependencies and improve architecture
+- Extracted command execution logic into dedicated CommandExecutor class
+- Separated conditional execution from iteration logic for better SOLID compliance
+- Enhanced error messages for API authentication failures
+- Replaced all `require_relative` with `require` statements for consistency
+
+### Changed
+- Refactored god objects to improve code organization and maintainability
+- Improved separation of concerns between workflow components
+
+[0.2.0]: https://github.com/Shopify/roast/compare/v0.1.7...v0.2.0
+
+## [0.1.7] - 2024-05-16
+
+### Added
+- `UpdateFiles` tool for applying diffs/patches to multiple files at once
+- Support for atomic file updates with rollback capability
+- Comprehensive documentation for all built-in tools
+- Enhanced README with detailed tool usage examples
+
+``[0.1.7]: https://github.com/Shopify/roast/compare/v0.1.6...v0.1.7
+
 ## [0.1.6] - 2024-05-15
 
 ### Added

data/CLAUDE.md

@@ -12,10 +12,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - Run single test: `bundle exec ruby -Itest test/path/to/test_file.rb`
 - Lint: `bundle exec rubocop`
 - Lint (with autocorrect, preferred): `bundle exec rubocop -A`
+- Whenever you want to run the whole test suite just run `bundle exec rake` to also run linting, and note the linting errors too (most will auto correct but not all)
+- **Run roast locally**: Use `bin/roast` (not `bundle exec roast` which may use the installed gem)
+- Alternative: `bundle exec exe/roast`
 
 ## Tech stack
 - `thor` and `cli-ui` for the CLI tool
 - Testing: Use Minitest, VCR for HTTP mocking, test files named with `_test.rb` suffix
+- Prefer using the more literate `test "this is a test description" do` type of testing that we get from extending ActiveSupport::TestCase over the primitive XUnit-style def test_description headings for tests
 
 ## Code Style Guidelines
 
@@ -30,6 +34,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - Add runtime dependencies to `roast.gemspec`.
 - Add development dependencies to `Gemfile`.
 - Don't ever test private methods directly. Specs should test behavior, not implementation.
+- I do not like test-specific code embedded in production code, don't ever do that
+- **Do not use require_relative**
+- Require statements should always be in alphabetical order
+- Always leave a blank line after module includes and before the rest of the class
+
+## Architecture Guidelines
+
+- **Maintain proper separation of concerns**: Don't mix unrelated concepts in the same class or module
+  - Example: Conditional execution (if/unless) should NOT be mixed with iteration execution (each/repeat)
+  - Each concept should have its own executor class and be handled separately
+- **Use appropriate inheritance**: Only inherit from a base class if the child truly "is-a" type of the parent
+  - Don't inherit just to reuse some methods - use composition instead
+- **Follow Single Responsibility Principle**: Each class should have one reason to change
+  - IterationExecutor handles iterations (each, repeat)
+  - ConditionalExecutor handles conditionals (if, unless)
+  - Don't combine different responsibilities in one class
 
 ## Git Workflow Practices
 

data/Gemfile

@@ -8,6 +8,7 @@ git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
 gemspec
 
 gem "cgi"
+gem "cli-ui"
 gem "dotenv"
 gem "guard"
 gem "guard-minitest"

data/Gemfile.lock

@@ -1,11 +1,13 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.1.6)
+    roast-ai (0.2.0)
       activesupport (~> 8.0)
+      cli-ui
+      diff-lcs (~> 1.5)
       faraday-retry
       json-schema
-      raix (~> 0.8.4)
+      raix (~> 0.8.6)
       thor (~> 1.3)
 
 GEM
@@ -31,16 +33,18 @@ GEM
     benchmark (0.4.0)
     bigdecimal (3.1.9)
     cgi (0.4.2)
+    cli-ui (2.3.1)
     coderay (1.1.3)
     concurrent-ruby (1.3.5)
-    connection_pool (2.5.1)
+    connection_pool (2.5.3)
     crack (1.0.0)
       bigdecimal
       rexml
+    diff-lcs (1.6.1)
     dotenv (3.1.8)
     drb (2.2.1)
     event_stream_parser (1.0.0)
-    faraday (2.13.0)
+    faraday (2.13.1)
       faraday-net_http (>= 2.0, < 3.5)
       json
       logger
@@ -72,7 +76,7 @@ GEM
     hashdiff (1.1.2)
     i18n (1.14.7)
       concurrent-ruby (~> 1.0)
-    json (2.10.2)
+    json (2.12.0)
     json-schema (5.1.1)
       addressable (~> 2.8)
       bigdecimal (~> 3.1)
@@ -111,7 +115,7 @@ GEM
     public_suffix (6.0.1)
     racc (1.8.1)
     rainbow (3.1.1)
-    raix (0.8.4)
+    raix (0.8.6)
       activesupport (>= 6.0)
       faraday-retry (~> 2.0)
       open_router (~> 0.2)
@@ -166,6 +170,7 @@ PLATFORMS
 
 DEPENDENCIES
   cgi
+  cli-ui
   dotenv
   guard
   guard-minitest

data/README.md

@@ -46,7 +46,7 @@ Each step can have its own prompt file (e.g., `analyze_coverage/prompt.md`) and
 ```yaml
 steps:
   - prepare_data
-  - 
+  -
     - analyze_code_quality
     - check_test_coverage
     - verify_documentation
@@ -86,21 +86,21 @@ In Roast, workflows maintain a single conversation with the AI model throughout
 
 Roast supports several types of steps:
 
-1. **Standard step**: References a directory containing at least a `prompt.md` and optional `output.txt` template. This is the most common type of step. 
+1. **Standard step**: References a directory containing at least a `prompt.md` and optional `output.txt` template. This is the most common type of step.
   ```yaml
   steps:
     - analyze_code
   ```
 
   As an alternative to a directory, you can also implement a custom step as a Ruby class, optionally extending `Roast::Workflow::BaseStep`.
-  
+
   In the example given above, the script would live at `workflow/analyze_code.rb` and should contain a class named `AnalyzeCode` with an initializer that takes a workflow object as context, and a `call` method that will be invoked to run the step. The result of the `call` method will be stored in the `workflow.output` hash.
 
 
 2. **Parallel steps**: Groups of steps executed concurrently
    ```yaml
    steps:
-     - 
+     -
        - analyze_code_quality
        - check_test_coverage
    ```
@@ -113,7 +113,72 @@ Roast supports several types of steps:
    ```
    This will execute the command and store the result in the workflow output hash. Explicit key name is optional (`rubocop` in the second line of the example).
 
-4. **Raw prompt step**: Simple text prompts for the model without tools
+   By default, commands that exit with non-zero status will halt the workflow. You can configure steps to continue on error:
+   ```yaml
+   steps:
+     - lint_check: $(rubocop {{file}})
+     - fix_issues
+   
+   # Step configuration
+   lint_check:
+     exit_on_error: false  # Continue workflow even if command fails
+   ```
+   When `exit_on_error: false`, the command output will include the exit status, allowing subsequent steps to process error information.
+
+4. **Conditional steps**: Execute different steps based on conditions using `if/unless`
+   ```yaml
+   steps:
+     - check_environment:
+         if: "{{ENV['RAILS_ENV'] == 'production'}}"
+         then:
+           - run_production_checks
+           - notify_team
+         else:
+           - run_development_setup
+     
+     - verify_dependencies:
+         unless: "$(bundle check)"
+         then:
+           - bundle_install: "$(bundle install)"
+   ```
+   
+   Conditions can be:
+   - Ruby expressions: `if: "{{output['count'] > 5}}"`
+   - Bash commands: `if: "$(test -f config.yml && echo true)"` (exit code 0 = true)
+   - Step references: `if: "previous_step_name"` (uses the step's output)
+   - Direct values: `if: "true"` or `if: "false"`
+
+5. **Iteration steps**: Loop over collections or repeat steps with conditions
+   ```yaml
+   steps:
+     # Loop over a collection
+     - process_files:
+         each: "{{Dir.glob('**/*.rb')}}"
+         as: current_file
+         steps:
+           - analyze_file
+           - Generate a report for {{current_file}}
+     
+     # Repeat until a condition is met
+     - improve_code:
+         repeat:
+           until: "{{output['test_pass'] == true}}"
+           max_iterations: 5
+           steps:
+             - run_tests
+             - fix_issues
+   ```
+   
+   Each loops support:
+   - Collections from Ruby expressions: `each: "{{[1, 2, 3]}}"`
+   - Command output: `each: "$(ls *.rb)"`
+   - Step references: `each: "file_list"`
+   
+   Repeat loops support:
+   - Until conditions: `until: "{{condition}}"`
+   - Maximum iterations: `max_iterations: 10`
+
+6. **Raw prompt step**: Simple text prompts for the model without tools
    ```yaml
    steps:
      - Summarize the changes made to the codebase.
@@ -139,7 +204,7 @@ Roast handles data flow between steps in three primary ways:
      - Generate a summary for {{file}}
      - result_for_{{file}}: store_results
    ```
-   
+
    Interpolation supports:
    - Simple variable access: `{{file}}`, `{{resource.target}}`
    - Access to step outputs: `{{output['previous_step']}}`
@@ -150,7 +215,7 @@ For typical AI workflows, the continuous conversation history provides seamless
 ### Command Line Options
 
 #### Basic Options
-- `-o, --output FILE`: Save results to a file instead of outputting to STDOUT 
+- `-o, --output FILE`: Save results to a file instead of outputting to STDOUT
 - `-c, --concise`: Use concise output templates (exposed as a boolean flag on `workflow`)
 - `-v, --verbose`: Show output from all steps as they execute
 - `-r, --replay STEP_NAME`: Resume a workflow from a specific step, optionally with a specific session timestamp
@@ -241,7 +306,7 @@ roast execute workflow.yml -t "$(find . -name '*.rb' -mtime -1)"
 # Process changed test files
 roast execute workflow.yml -t "$(git diff --name-only HEAD | grep _test.rb)"
 
-# Process staged files  
+# Process staged files
 roast execute workflow.yml -t "$(git diff --cached --name-only)"
 ```
 
@@ -255,7 +320,7 @@ name: API Integration Workflow
 tools:
   - Roast::Tools::ReadFile
   - Roast::Tools::WriteFile
-  
+
 # Dynamic API token using shell command
 api_token: $(cat ~/.my_token)
 
@@ -349,7 +414,9 @@ api_provider: openrouter
 api_token: $(echo $OPENROUTER_API_KEY)
 ```
 
-This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines.
+
+This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines. Alternatively, Roast will fall back to `OPENROUTER_API_KEY` or `OPENAI_API_KEY` environment variables based on the specified provider.
+
 
 ### Template Output with ERB
 
@@ -372,7 +439,7 @@ This is an example of where the `workflow.output` hash is useful - formatting ou
 
 Available in templates:
 - `response`: The AI's response for this step
-- `workflow`: Access to the workflow object 
+- `workflow`: Access to the workflow object
 - `workflow.output`: The shared hash containing results from all steps when you need programmatic access
 - `workflow.file`: Current file being processed (or `nil` for targetless workflows)
 - All workflow configuration options
@@ -385,6 +452,151 @@ For most workflows, you'll mainly use `response` to access the current step's re
 
 Roast provides extensive instrumentation capabilities using ActiveSupport::Notifications. You can monitor workflow execution, track AI model usage, measure performance, and integrate with external monitoring systems. [Read the full instrumentation documentation](docs/INSTRUMENTATION.md).
 
+### Built-in Tools
+
+Roast provides several built-in tools that you can use in your workflows:
+
+#### ReadFile
+
+Reads the contents of a file from the filesystem.
+
+```ruby
+# Basic usage
+read_file(path: "path/to/file.txt")
+
+# Reading a specific portion of a file
+read_file(path: "path/to/large_file.txt", offset: 100, limit: 50)
+```
+
+- The `path` can be absolute or relative to the current working directory
+- Use `offset` and `limit` for large files to read specific sections (line numbers)
+- Returns the file content as a string
+
+#### WriteFile
+
+Writes content to a file, creating the file if it doesn't exist or overwriting it if it does.
+
+```ruby
+# Basic usage
+write_file(path: "output.txt", content: "This is the file content")
+
+# With path restriction for security
+write_file(
+  path: "output.txt",
+  content: "Restricted content",
+  restrict: "/safe/directory" # Only allows writing to files under this path
+)
+```
+
+- Creates missing directories automatically
+- Can restrict file operations to specific directories for security
+- Returns a success message with the number of lines written
+
+#### UpdateFiles
+
+Applies a unified diff/patch to one or more files. Changes are applied atomically when possible.
+
+```ruby
+update_files(
+  diff: <<~DIFF,
+    --- a/file1.txt
+    +++ b/file1.txt
+    @@ -1,3 +1,4 @@
+     line1
+    +new line
+     line2
+     line3
+
+    --- a/file2.txt
+    +++ b/file2.txt
+    @@ -5,7 +5,7 @@
+     line5
+     line6
+    -old line7
+    +updated line7
+     line8
+  DIFF
+  base_path: "/path/to/project", # Optional, defaults to current working directory
+  restrict_path: "/path/to/allowed", # Optional, restricts where files can be modified
+  create_files: true, # Optional, defaults to true
+)
+```
+
+- Accepts standard unified diff format from tools like `git diff`
+- Supports multiple file changes in a single operation
+- Handles file creation, deletion, and modification
+- Performs atomic operations with rollback on failure
+- Includes fuzzy matching to handle minor context differences
+- This tool is especially useful for making targeted changes to multiple files at once
+
+#### Grep
+
+Searches file contents for a specific pattern using regular expressions.
+
+```ruby
+# Basic usage
+grep(pattern: "function\\s+myFunction")
+
+# With file filtering
+grep(pattern: "class\\s+User", include: "*.rb")
+
+# With directory scope
+grep(pattern: "TODO:", path: "src/components")
+```
+
+- Uses regular expressions for powerful pattern matching
+- Can filter by file types using the `include` parameter
+- Can scope searches to specific directories with the `path` parameter
+- Returns a list of files containing matches
+
+#### SearchFile
+
+Provides advanced file search capabilities beyond basic pattern matching.
+
+```ruby
+search_file(query: "class User", file_path: "app/models")
+```
+
+- Combines pattern matching with contextual search
+- Useful for finding specific code structures or patterns
+- Returns matched lines with context
+
+#### Cmd
+
+Executes shell commands and returns their output.
+
+```ruby
+# Execute a simple command
+cmd(command: "ls -la")
+
+# With working directory specified
+cmd(command: "npm list", cwd: "/path/to/project")
+
+# With environment variables
+cmd(command: "deploy", env: { "NODE_ENV" => "production" })
+```
+
+- Provides access to shell commands for more complex operations
+- Can specify working directory and environment variables
+- Captures and returns command output
+- Useful for integrating with existing tools and scripts
+
+#### CodingAgent
+
+Creates a specialized agent for complex coding tasks or long-running operations.
+
+```ruby
+coding_agent(
+  task: "Refactor the authentication module to use JWT tokens",
+  language: "ruby",
+  files: ["app/models/user.rb", "app/controllers/auth_controller.rb"]
+)
+```
+
+- Delegates complex tasks to a specialized coding agent
+- Useful for tasks that require deep code understanding or multi-step changes
+- Can work across multiple files and languages
+
 ### Custom Tools
 
 You can create your own tools using the [Raix function dispatch pattern](https://github.com/OlympiaAI/raix-rails?tab=readme-ov-file#use-of-toolsfunctions). Custom tools should be placed in `.roast/initializers/` (subdirectories are supported):
@@ -411,11 +623,11 @@ module MyProject
 
       def call(commit_sha, include_diff = false)
         Roast::Helpers::Logger.info("🔍 Analyzing commit: #{commit_sha}\n")
-        
+
         # Your implementation here
         commit_info = `git show #{commit_sha} --stat`
         commit_info += "\n\n" + `git show #{commit_sha}` if include_diff
-        
+
         commit_info
       rescue StandardError => e
         "Error analyzing commit: #{e.message}".tap do |error_message|

data/bin/roast

@@ -0,0 +1,27 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'roast' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../.ruby-lsp/Gemfile", __dir__)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300).include?("This file was generated by Bundler")
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("roast-ai", "roast")

data/docs/INSTRUMENTATION.md

@@ -199,4 +199,45 @@ Then run:
 roast execute test_instrumentation.yml some_file.rb
 ```
 
-Your instrumentation should capture the workflow start, step execution, and workflow completion events.
+Your instrumentation should capture the workflow start, step execution, and workflow completion events.
+
+## Available Tools
+
+Roast provides several built-in tools that you can use in your workflows:
+
+### WriteFile Tool
+
+Writes content to a file, creating the file if it doesn't exist or overwriting it if it does.
+
+```ruby
+# Example usage in a prompt
+write_file(path: "output.txt", content: "This is the file content")
+```
+
+### UpdateFiles Tool
+
+Applies a unified diff/patch to one or more files. Changes are applied atomically when possible.
+
+```ruby
+# Example usage in a prompt
+update_files(diff: <<~DIFF, base_path: "/path/to/project", create_files: true)
+  --- a/file1.txt
+  +++ b/file1.txt
+  @@ -1,3 +1,4 @@
+   line1
+  +new line
+   line2
+   line3
+  
+  --- a/file2.txt
+  +++ b/file2.txt
+  @@ -5,7 +5,7 @@
+   line5
+   line6
+  -old line7
+  +updated line7
+   line8
+DIFF
+```
+
+This tool is especially useful for making targeted changes to multiple files at once, without having to replace entire file contents.