roast-ai 0.1.5 → 0.1.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bc0fd34418eef0f9546d55ab30407287745c6f96664b57b29050eeecfd0751e9
-  data.tar.gz: ee09de492428972903a8d69a768b9b661e8668c1aebe51fa5d156e01d41beb41
+  metadata.gz: 40bb03c596dcaa3dd49685c9b5f804ca6c6db00cf239d5e52748e6fa485f9c3b
+  data.tar.gz: c56399cca493aa9d925bb4d0550dc7ad0c0ed03b74b4f7894a3205ba72e66d1e
 SHA512:
-  metadata.gz: c8458190fd1fe4b21e80f1fe038f104c55f40a9bfcd0cef5ef1d66aacf25d783b7ff4c82940adeda90cc60914906e03ef3f83373c97e3880e87e62a7e5da0f17
-  data.tar.gz: 34f621a2ebc7f5074d161271e5babf1c2f20e5492c483e8b45a2be217c241d577e46251db19356e59c818d54c1e999b35225e1793f6380d639a04358b94e36eb
+  metadata.gz: 2560cd52f21eb2aa0c954d1759ac762c12f2840a1533cc41017003a022f002212e9fb8d6e6b3d11fc1245ff413a86bd68d3aad7a142de9a30d4c534277a80f51
+  data.tar.gz: 371d573330873f1e76aec168e72ed8e2c53d0a127af83683205fae7d060dfa758aeb2352133b4f8b263803b5384b77be76b91a564d945cfd8966a682959a0d13

data/.github/workflows/ci.yaml

@@ -25,5 +25,5 @@ jobs:
         with:
           ruby-version: ${{ matrix.ruby }}
           bundler-cache: true
-      - run: bundle exec rake
+      - run: bundle exec rake ci
 

data/CHANGELOG.md

@@ -5,6 +5,37 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.7] - 2024-05-16
+
+### Added
+- `UpdateFiles` tool for applying diffs/patches to multiple files at once
+- Support for atomic file updates with rollback capability
+- Comprehensive documentation for all built-in tools
+- Enhanced README with detailed tool usage examples
+
+## [0.1.6] - 2024-05-15
+
+### Added
+- Support for OpenRouter as an API provider
+- `api_provider` configuration option allowing choice between OpenAI and OpenRouter
+- Added separate CI rake task for improved build pipeline
+- Version command to check current Roast version
+- Walking up to home folder for config root
+- Improved initializer support for better project configuration
+
+### Changed
+- Enhanced search tool to work with globs for more flexible searches
+- Improved error handling in configuration and initializers
+- Fixed and simplified interpolation examples
+
+### Fixed
+- Better error messages for search file tool
+- Improved initializer loading and error handling
+- Fixed tests for nested .roast folders
+
+[0.1.7]: https://github.com/Shopify/roast/compare/v0.1.6...v0.1.7
+[0.1.6]: https://github.com/Shopify/roast/compare/v0.1.5...v0.1.6
+
 ## [0.1.5] - 2024-05-13
 
 ### Added

data/CLAUDE.md

@@ -7,16 +7,15 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Commands
 
-- Build: `bundle exec rake build`
-- Test all: `bundle exec rspec`
-- Run single test: `bundle exec rspec spec/path/to/test_file.rb`
-- Lint: `bundle exec rubocop -A`
 - Default (tests + lint): `bundle exec rake`
-- Run all tests plus rubocop: `bundle exec rake`
+- Test all: `bundle exec test`
+- Run single test: `bundle exec ruby -Itest test/path/to/test_file.rb`
+- Lint: `bundle exec rubocop`
+- Lint (with autocorrect, preferred): `bundle exec rubocop -A`
 
 ## Tech stack
-- `cli-kit` and `cli-ui` for the CLI tool
-- Testing: Use Rspec, VCR for HTTP mocking, test files named with `_spec.rb` suffix
+- `thor` and `cli-ui` for the CLI tool
+- Testing: Use Minitest, VCR for HTTP mocking, test files named with `_test.rb` suffix
 
 ## Code Style Guidelines
 
@@ -30,4 +29,115 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - Define class methods inside `class << self; end` declarations.
 - Add runtime dependencies to `roast.gemspec`.
 - Add development dependencies to `Gemfile`.
-- Don't ever test private methods directly. Specs should test behavior, not implementation.
+- Don't ever test private methods directly. Specs should test behavior, not implementation.
+
+## Git Workflow Practices
+
+1. **Amending Commits**:
+   - Use `git commit --amend --no-edit` to add staged changes to the last commit without changing the commit message
+   - This is useful for incorporating small fixes or changes that belong with the previous commit
+   - Be careful when amending commits that have already been pushed, as it will require a force push
+
+2. **Force Pushing Safety**:
+   - Always use `git push --force-with-lease` rather than `git push --force` when pushing amended commits
+   - This prevents accidentally overwriting remote changes made by others that you haven't pulled yet
+   - It's a safer alternative that respects collaborative work environments
+
+4. **PR Management**:
+   - Pay attention to linting results before pushing to avoid CI failures
+
+## GitHub API Commands
+To get comments from a Pull Request using the GitHub CLI:
+
+```bash
+# Get review comments from a PR
+gh api repos/Shopify/roast/pulls/{pr_number}/comments
+
+# Get issue-style comments
+gh api repos/Shopify/roast/issues/{pr_number}/comments
+
+# Filter comments from a specific user using jq
+gh api repos/Shopify/roast/pulls/{pr_number}/comments | jq '.[] | select(.user.login == "username")'
+
+# Get only the comment content
+gh api repos/Shopify/roast/pulls/{pr_number}/comments | jq '.[].body'
+```
+
+### Creating and Managing Issues via API
+
+```bash
+# Create a new issue
+gh api repos/Shopify/roast/issues -X POST -F title="Issue Title" -F body="Issue description"
+
+# Update an existing issue
+gh api repos/Shopify/roast/issues/{issue_number} -X PATCH -F body="Updated description"
+
+# Add a comment to an issue
+gh api repos/Shopify/roast/issues/{issue_number}/comments -X POST -F body="Comment text"
+```
+
+### Creating and Managing Pull Requests
+
+```bash
+# Create a new PR with a detailed description using heredoc
+gh pr create --title "PR Title" --body "$(cat <<'EOF'
+## Summary
+
+Detailed description here...
+
+## Testing
+
+Testing instructions here...
+EOF
+)"
+
+# Update an existing PR description
+gh pr edit {pr_number} --body "$(cat <<'EOF'
+Updated PR description...
+EOF
+)"
+
+# Check PR details
+gh pr view {pr_number}
+
+# View PR diff
+gh pr diff {pr_number}
+```
+
+#### Formatting Tips for GitHub API
+1. Use literal newlines in the body text instead of `\n` escape sequences
+2. When formatting is stripped (like backticks), use alternatives:
+   - **Bold text** instead of `code formatting`
+   - Add a follow-up comment with proper formatting
+3. For complex issues, create the basic issue first, then enhance with formatted comments
+4. Always verify the formatting in the created content
+5. Use raw JSON for complex formatting requirements:
+   ```bash
+   gh api repos/Shopify/roast/issues -X POST --raw-field '{"title":"Issue Title","body":"Complex **formatting** with `code` and lists:\n\n1. Item one\n2. Item two"}'
+   ```
+
+## PR Review Best Practices
+1. **Always provide your honest opinion about the PR** - be candid about both strengths and concerns
+2. Give a clear assessment of risks, architectural implications, and potential future issues
+3. Don't be afraid to point out potential problems even in otherwise good PRs
+4. When reviewing feature flag removal PRs, carefully inspect control flow changes, not just code branch removals
+5. Pay special attention to control flow modifiers like `next`, `return`, and `break` which affect iteration behavior
+6. Look for variable scope issues, especially for variables that persist across loop iterations
+7. Analyze how code behavior changes in all cases, not just how code structure changes
+8. Be skeptical of seemingly simple changes that simply remove conditional branches
+9. When CI checks fail, look for subtle logic inversions or control flow changes
+10. Examine every file changed in a PR with local code for context, focusing on both what's removed and what remains
+11. Verify variable initialization, modification, and usage patterns remain consistent after refactoring
+12. **Never try to directly check out PR branches** - instead, compare PR changes against the existing local codebase
+13. Understand the broader system architecture to identify potential impacts beyond the changed files
+14. Look at both the "before" and "after" state of the code when evaluating changes, not just the diff itself
+15. Consider how the changes will interact with other components that depend on the modified code
+16. Run searches or examine related files even if they're not directly modified by the PR
+17. Look for optimization opportunities, especially in frequently-called methods:
+    - Unnecessary object creation in loops 
+    - Redundant collection transformations
+    - Inefficient filtering methods that create temporary collections
+18. Prioritize code readability while encouraging performance optimizations:
+    - Avoid premature optimization outside of hot paths
+    - Consider the tradeoff between readability and performance
+    - Suggest optimizations that improve both clarity and performance

data/Gemfile

@@ -9,10 +9,9 @@ gemspec
 
 gem "cgi"
 gem "dotenv"
-gem "guard-rspec"
 gem "guard"
+gem "guard-minitest"
 gem "mocha"
-gem "pry"
 gem "rake", require: false
 gem "rubocop-shopify", require: false
 gem "vcr", require: false

data/Gemfile.lock

@@ -1,8 +1,9 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.1.5)
+    roast-ai (0.1.7)
       activesupport (~> 8.0)
+      diff-lcs (~> 1.5)
       faraday-retry
       json-schema
       raix (~> 0.8.4)
@@ -53,6 +54,7 @@ GEM
       faraday (~> 2.0)
     ffi (1.17.2)
     ffi (1.17.2-arm64-darwin)
+    ffi (1.17.2-x86_64-linux-gnu)
     formatador (1.1.0)
     guard (2.19.1)
       formatador (>= 0.2.4)
@@ -66,10 +68,9 @@ GEM
       shellany (~> 0.0)
       thor (>= 0.18.1)
     guard-compat (1.2.1)
-    guard-rspec (4.7.3)
-      guard (~> 2.1)
-      guard-compat (~> 1.1)
-      rspec (>= 2.99.0, < 4.0)
+    guard-minitest (2.4.6)
+      guard-compat (~> 1.2)
+      minitest (>= 3.0)
     hashdiff (1.1.2)
     i18n (1.14.7)
       concurrent-ruby (~> 1.0)
@@ -123,19 +124,6 @@ GEM
       ffi (~> 1.0)
     regexp_parser (2.10.0)
     rexml (3.4.1)
-    rspec (3.13.0)
-      rspec-core (~> 3.13.0)
-      rspec-expectations (~> 3.13.0)
-      rspec-mocks (~> 3.13.0)
-    rspec-core (3.13.3)
-      rspec-support (~> 3.13.0)
-    rspec-expectations (3.13.3)
-      diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.13.0)
-    rspec-mocks (3.13.2)
-      diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.13.0)
-    rspec-support (3.13.2)
     rubocop (1.75.3)
       json (~> 2.3)
       language_server-protocol (~> 3.17.0.2)
@@ -176,15 +164,14 @@ GEM
 
 PLATFORMS
   arm64-darwin-23
-  ruby
+  x86_64-linux
 
 DEPENDENCIES
   cgi
   dotenv
   guard
-  guard-rspec
+  guard-minitest
   mocha
-  pry
   rake
   roast-ai!
   rubocop-shopify

data/README.md

@@ -312,9 +312,27 @@ Individual steps can override this setting with their own model parameter:
 
 ```yaml
 analyze_data:
-  model: anthropic:claude-3-haiku  # Takes precedence over the global model
+  model: anthropic/claude-3-haiku  # Takes precedence over the global model
 ```
 
+#### API Provider Configuration
+
+Roast supports both OpenAI and OpenRouter as API providers. By default, Roast uses OpenAI, but you can specify OpenRouter:
+
+```yaml
+name: My Workflow
+api_provider: openrouter
+api_token: $(echo $OPENROUTER_API_KEY)
+model: anthropic/claude-3-opus-20240229
+```
+
+Benefits of using OpenRouter:
+- Access to multiple model providers through a single API
+- Support for models from Anthropic, Meta, Mistral, and more
+- Consistent API interface across different model providers
+
+When using OpenRouter, specify fully qualified model names including the provider prefix (e.g., `anthropic/claude-3-opus-20240229`).
+
 #### Dynamic API Tokens
 
 Roast allows you to dynamically fetch API tokens using shell commands directly in your workflow configuration:
@@ -323,8 +341,12 @@ Roast allows you to dynamically fetch API tokens using shell commands directly i
 # This will execute the shell command and use the result as the API token
 api_token: $(print-token --key)
 
-# Or a simpler example for demonstration:
+# For OpenAI (default)
 api_token: $(echo $OPENAI_API_KEY)
+
+# For OpenRouter (requires api_provider setting)
+api_provider: openrouter
+api_token: $(echo $OPENROUTER_API_KEY)
 ```
 
 This makes it easy to use environment-specific tokens without hardcoding credentials, especially useful in development environments or CI/CD pipelines.
@@ -363,6 +385,151 @@ For most workflows, you'll mainly use `response` to access the current step's re
 
 Roast provides extensive instrumentation capabilities using ActiveSupport::Notifications. You can monitor workflow execution, track AI model usage, measure performance, and integrate with external monitoring systems. [Read the full instrumentation documentation](docs/INSTRUMENTATION.md).
 
+### Built-in Tools
+
+Roast provides several built-in tools that you can use in your workflows:
+
+#### ReadFile
+
+Reads the contents of a file from the filesystem.
+
+```ruby
+# Basic usage
+read_file(path: "path/to/file.txt")
+
+# Reading a specific portion of a file
+read_file(path: "path/to/large_file.txt", offset: 100, limit: 50)
+```
+
+- The `path` can be absolute or relative to the current working directory
+- Use `offset` and `limit` for large files to read specific sections (line numbers)
+- Returns the file content as a string
+
+#### WriteFile
+
+Writes content to a file, creating the file if it doesn't exist or overwriting it if it does.
+
+```ruby
+# Basic usage
+write_file(path: "output.txt", content: "This is the file content")
+
+# With path restriction for security
+write_file(
+  path: "output.txt",
+  content: "Restricted content",
+  restrict: "/safe/directory" # Only allows writing to files under this path
+)
+```
+
+- Creates missing directories automatically
+- Can restrict file operations to specific directories for security
+- Returns a success message with the number of lines written
+
+#### UpdateFiles
+
+Applies a unified diff/patch to one or more files. Changes are applied atomically when possible.
+
+```ruby
+update_files(
+  diff: <<~DIFF,
+    --- a/file1.txt
+    +++ b/file1.txt
+    @@ -1,3 +1,4 @@
+     line1
+    +new line
+     line2
+     line3
+    
+    --- a/file2.txt
+    +++ b/file2.txt
+    @@ -5,7 +5,7 @@
+     line5
+     line6
+    -old line7
+    +updated line7
+     line8
+  DIFF
+  base_path: "/path/to/project", # Optional, defaults to current working directory
+  restrict_path: "/path/to/allowed", # Optional, restricts where files can be modified
+  create_files: true, # Optional, defaults to true
+)
+```
+
+- Accepts standard unified diff format from tools like `git diff`
+- Supports multiple file changes in a single operation
+- Handles file creation, deletion, and modification
+- Performs atomic operations with rollback on failure
+- Includes fuzzy matching to handle minor context differences
+- This tool is especially useful for making targeted changes to multiple files at once
+
+#### Grep
+
+Searches file contents for a specific pattern using regular expressions.
+
+```ruby
+# Basic usage
+grep(pattern: "function\\s+myFunction")
+
+# With file filtering
+grep(pattern: "class\\s+User", include: "*.rb")
+
+# With directory scope
+grep(pattern: "TODO:", path: "src/components")
+```
+
+- Uses regular expressions for powerful pattern matching
+- Can filter by file types using the `include` parameter
+- Can scope searches to specific directories with the `path` parameter
+- Returns a list of files containing matches
+
+#### SearchFile
+
+Provides advanced file search capabilities beyond basic pattern matching.
+
+```ruby
+search_file(query: "class User", file_path: "app/models")
+```
+
+- Combines pattern matching with contextual search
+- Useful for finding specific code structures or patterns
+- Returns matched lines with context
+
+#### Cmd
+
+Executes shell commands and returns their output.
+
+```ruby
+# Execute a simple command
+cmd(command: "ls -la")
+
+# With working directory specified
+cmd(command: "npm list", cwd: "/path/to/project")
+
+# With environment variables
+cmd(command: "deploy", env: { "NODE_ENV" => "production" })
+```
+
+- Provides access to shell commands for more complex operations
+- Can specify working directory and environment variables
+- Captures and returns command output
+- Useful for integrating with existing tools and scripts
+
+#### CodingAgent
+
+Creates a specialized agent for complex coding tasks or long-running operations.
+
+```ruby
+coding_agent(
+  task: "Refactor the authentication module to use JWT tokens",
+  language: "ruby",
+  files: ["app/models/user.rb", "app/controllers/auth_controller.rb"]
+)
+```
+
+- Delegates complex tasks to a specialized coding agent
+- Useful for tasks that require deep code understanding or multi-step changes
+- Can work across multiple files and languages
+
 ### Custom Tools
 
 You can create your own tools using the [Raix function dispatch pattern](https://github.com/OlympiaAI/raix-rails?tab=readme-ov-file#use-of-toolsfunctions). Custom tools should be placed in `.roast/initializers/` (subdirectories are supported):

data/Rakefile

@@ -12,7 +12,11 @@ end
 
 task test: [:minitest]
 
-RuboCop::RakeTask.new do |task|
+RuboCop::RakeTask.new(:rubocop_ci)
+
+task ci: [:test, :rubocop_ci]
+
+RuboCop::RakeTask.new(:rubocop) do |task|
   task.options = ["--autocorrect"]
 end
 

data/docs/INSTRUMENTATION.md

@@ -199,4 +199,45 @@ Then run:
 roast execute test_instrumentation.yml some_file.rb
 ```
 
-Your instrumentation should capture the workflow start, step execution, and workflow completion events.
+Your instrumentation should capture the workflow start, step execution, and workflow completion events.
+
+## Available Tools
+
+Roast provides several built-in tools that you can use in your workflows:
+
+### WriteFile Tool
+
+Writes content to a file, creating the file if it doesn't exist or overwriting it if it does.
+
+```ruby
+# Example usage in a prompt
+write_file(path: "output.txt", content: "This is the file content")
+```
+
+### UpdateFiles Tool
+
+Applies a unified diff/patch to one or more files. Changes are applied atomically when possible.
+
+```ruby
+# Example usage in a prompt
+update_files(diff: <<~DIFF, base_path: "/path/to/project", create_files: true)
+  --- a/file1.txt
+  +++ b/file1.txt
+  @@ -1,3 +1,4 @@
+   line1
+  +new line
+   line2
+   line3
+  
+  --- a/file2.txt
+  +++ b/file2.txt
+  @@ -5,7 +5,7 @@
+   line5
+   line6
+  -old line7
+  +updated line7
+   line8
+DIFF
+```
+
+This tool is especially useful for making targeted changes to multiple files at once, without having to replace entire file contents.

data/examples/interpolation/analyze_file/prompt.md

@@ -1,13 +1 @@
-# File Analysis
-
-I'm going to analyze the file at: <%= workflow.file %>
-
-First, let me gather some basic information about the file:
-
-1. File name: `<%= File.basename(workflow.file) %>`
-2. File extension: `<%= File.extname(workflow.file).sub('.', '') %>`
-3. File size: `<%= File.size(workflow.file) %> bytes`
-
----
-
-Can you read the file for me so I can analyze it?
+Analyze the file at: <%= workflow.file %>

data/examples/interpolation/generate_report_for_js/prompt.md

@@ -1,28 +1,3 @@
-# JavaScript File Report Generation
+Generate a nicely formatted report based on the following metadata:
 
-I'm generating a report for a JavaScript file:
-
-- File name: `<%= workflow.output['file_basename'] %>`
-- Lines: `<%= workflow.output['patterns_found']['lines'] %>`
-
-## JavaScript-Specific Report
-
-JavaScript files typically contain functions, classes, and import/export statements. I'll generate a report focused on JavaScript code analysis.
-
-### Recommendations for JavaScript Code
-
-1. Follow a consistent style guide (ESLint)
-2. Use modern JavaScript features (ES6+)
-3. Prefer const and let over var
-4. Use async/await for asynchronous code
-5. Write unit tests with Jest or Mocha
-
-### Summary
-
-This JavaScript file has been analyzed and basic metrics have been collected. A more detailed analysis would involve parsing the JavaScript code to extract functions, classes, and import/export statements.
-
-I'll create a report file with this information.
-
-I'll use my WriteFile tool to generate the report.
-
-Can you please write the report to `report_<%= workflow.output['file_basename'] %>.md` in the current directory?
+<%= workflow.output[:analyze_patterns] %>

data/examples/interpolation/generate_report_for_rb/prompt.md

@@ -1,3 +1,3 @@
 Generate a nicely formatted report based on the following metadata:
 
-<%= workflow.output[:extract_info] %>
+<%= workflow.output[:analyze_patterns] %>

data/examples/interpolation/workflow.yml

@@ -3,12 +3,10 @@ model: anthropic:claude-3-7-sonnet
 
 tools:
   - Roast::Tools::ReadFile
-  - Roast::Tools::WriteFile
-  - Roast::Tools::Grep
 
 steps:
   - analyze_file
-  - extract_info: analyze_patterns
+  - analyze_patterns
   - generate_report_for_{{File.extname(workflow.file).sub('.', '')}}
   - '$(echo "Processing completed for file: {{File.basename(workflow.file)}}")'
 

data/examples/openrouter_example/README.md

@@ -0,0 +1,48 @@
+# OpenRouter Example
+
+This example demonstrates how to use OpenRouter with Roast to access models from different providers through a single API.
+
+## Setup
+
+1. Sign up for an account at [OpenRouter](https://openrouter.ai/)
+2. Get your API key from the OpenRouter dashboard
+3. Set the API key as an environment variable:
+   ```bash
+   export OPENROUTER_API_KEY=your_api_key_here
+   ```
+
+## Running the Example
+
+```bash
+# Run without a specific target (general analysis)
+roast execute workflow.yml
+
+# Run with a specific file to analyze
+roast execute workflow.yml path/to/your/file.txt
+```
+
+## How it Works
+
+This example configures Roast to use OpenRouter as the API provider:
+
+```yaml
+api_provider: openrouter
+api_token: $(echo $OPENROUTER_API_KEY)
+model: anthropic/claude-3-haiku-20240307
+```
+
+The workflow consists of two steps:
+1. `analyze_input`: Analyzes the provided content (or generates general insights if no target is provided)
+2. `generate_response`: Creates a structured response based on the analysis
+
+## Available Models
+
+When using OpenRouter, you can access models from multiple providers by specifying the fully qualified model name, including the provider prefix. Some examples:
+
+- `anthropic/claude-3-opus-20240229`
+- `anthropic/claude-3-sonnet-20240229`
+- `meta/llama-3-70b-instruct`
+- `google/gemini-1.5-pro-latest`
+- `mistral/mistral-large-latest`
+
+Check the [OpenRouter documentation](https://openrouter.ai/docs) for the complete list of supported models.

data/examples/openrouter_example/analyze_input/prompt.md

@@ -0,0 +1,16 @@
+I'd like you to analyze the following input and provide your insights.
+
+<% if workflow.file && workflow.resource.content %>
+Here is the content to analyze:
+
+```
+<%= workflow.resource.content %>
+```
+<% else %>
+The workflow is running without a specific file target. Please provide general insights based on the context.
+<% end %>
+
+Please provide:
+1. A summary of the key points
+2. Any notable patterns or issues
+3. Recommendations based on your analysis

data/examples/openrouter_example/generate_response/prompt.md

@@ -0,0 +1,9 @@
+Based on the analysis from the previous step, please generate a response that addresses the findings.
+
+Your response should be well-structured and include:
+1. An introduction summarizing the analysis
+2. Detailed discussion of key points
+3. Clear recommendations for improvements
+4. A conclusion
+
+Format the response in markdown for better readability.