roast-ai 0.4.6 → 0.4.8

data/examples/grading/run_coverage.rb

@@ -1,54 +0,0 @@
-# frozen_string_literal: true
-
-class RunCoverage < Roast::Workflow::BaseStep
-  def call
-    # Run the test with coverage analysis
-    run_test_with_coverage
-  end
-
-  private
-
-  def run_test_with_coverage
-    subject_file = workflow.output["read_dependencies"]
-    subject_file = subject_file.match(%r{<sut>(.*?)</sut>})&.[](1) || subject_file
-    test_file = workflow.file
-    extension = File.extname(test_file).gsub(".", "")
-
-    # Handle JS/TS test files
-    extension = "js" if ["js", "jsx", "ts", "tsx"].include?(extension)
-
-    # Get the absolute path to the test_runner executable
-    test_runner_path = File.expand_path("#{extension}_test_runner", __dir__)
-
-    # Make sure the test_runner executable exists
-    unless File.exist?(test_runner_path)
-      Roast::Helpers::Logger.error("Test runner executable not found: #{test_runner_path}")
-      exit(1)
-    end
-
-    # Resolve paths to prevent issues when pwd differs from project root
-    resolved_subject_file = Roast::Helpers::PathResolver.resolve(subject_file)
-    unless File.exist?(resolved_subject_file)
-      Roast::Helpers::Logger.error("Subject file not found: #{resolved_subject_file}")
-      exit(1)
-    end
-
-    resolved_test_file = Roast::Helpers::PathResolver.resolve(test_file)
-    unless File.exist?(resolved_test_file)
-      Roast::Helpers::Logger.error("Test file not found: #{resolved_test_file}")
-      exit(1)
-    end
-
-    # Run the test_runner using shadowenv for environment consistency
-    command = "shadowenv exec --dir . -- #{test_runner_path} #{resolved_test_file} #{resolved_subject_file}"
-    output, status = Open3.capture2(command)
-
-    unless status.success?
-      Roast::Helpers::Logger.error("Test runner exited with non-zero status: #{status.exitstatus}")
-      Roast::Helpers::Logger.error(output)
-      exit(status.exitstatus)
-    end
-
-    output
-  end
-end

data/examples/grading/verify_mocks_and_stubs/prompt.md

@@ -1,12 +0,0 @@
-Find places in the provided test code where stubbing and mocking are used. Search for the corresponding implementation source code of those dependencies elsewhere in the codebase to validate that the stub or mock matches the implementation that it is doubling. Use the tool functions provided to find and read the dependencies.
-
-Once you've found the dependencies, verify that any mocks and stubs accurately reflect the real implementation. If there are discrepancies, list them out alphabetically with:
-
-1. The name of the mocked/stubbed method
-2. What the mock/stub expects in terms of arguments and/or return values
-3. What the actual implementation actually takes as arguments and returns
-4. Suggestions for fixing the discrepancy
-
-Note: If there are no discrepancies, do not summarize those that accurately reflect their real implementations in the codebase, just respond "All mocks and stubs verified."
-
-IMPORTANT: There's absolutely no need for you to waste time grepping for methods/functions that you know belong to testing libraries such as Mocha's `expects` and `stubs`. Only search for the implementation of things that are stubbed and/or mocked in the test to verify whether the test code matches the implementation code.

data/examples/grading/verify_test_helpers/prompt.md

@@ -1,53 +0,0 @@
-Now identify custom test helpers used in this test for the following purpose:
-
-1. Analyzing if they are used correctly
-2. Understanding test code that has had significant chunks of implementation abstracted away into helpers
-3. Fully understanding custom assertions that are not included by default in Ruby on Rails or part of your base knowledge
-
-Your grep tool function is vital for this work. It provides 4 lines of context before and after the matching line.
-
-For example, if you call `grep(string: "def assert_sql")`, the output will include:
-
-```
-.test/support/helpers/sql_assertions.rb-101-    end
-.test/support/helpers/sql_assertions.rb-102-    result
-.test/support/helpers/sql_assertions.rb-103-  end
-.test/support/helpers/sql_assertions.rb-104-
-.test/support/helpers/sql_assertions.rb:105:  def assert_sql(*patterns_to_match, **kwargs, &block)
-.test/support/helpers/sql_assertions.rb-106-    mysql_only_test!
-.test/support/helpers/sql_assertions.rb-107-
-.test/support/helpers/sql_assertions.rb-108-    result = T.let(nil, T.nilable(T::Boolean))
-.test/support/helpers/sql_assertions.rb-109-    counter = ActiveRecord::SQLCounter.new(**kwargs)
-```
-
-Unfortunately, many test helper methods are undocumented. In those cases (like the example above) the pre-context will be junk. However, there are a number of helper methods that do have very specific and narrow use cases, and those do tend to be well-documented. In those cases, you should use `read_file` to be able to read the full documentation.
-
-For example, here is the result of calling `grep(string: "def assert_sql_events")`
-
-```
-.test/support/helpers/externals_helper.rb-93-  # @example Logs events in the list that did not occur
-.test/support/helpers/externals_helper.rb-94-  #   expected_queries = { "Shop Load" => 1, "User Load" => 1 }
-.test/support/helpers/externals_helper.rb-95-  #   # Fails and reports that User Load occured 0 times instead of expected 1
-.test/support/helpers/externals_helper.rb-96-  #   assert_sql_events(expected_queries) { Shop.current_or_find(shop.id) }
-.test/support/helpers/externals_helper.rb:97:  def assert_sql_events(expected_events, &block)
-.test/support/helpers/externals_helper.rb-98-    mysql_only_test!
-.test/support/helpers/externals_helper.rb-99-
-.test/support/helpers/externals_helper.rb-100-    mysql_events = ExternalsCollector.new(&block).events
-.test/support/helpers/externals_helper.rb-101-      .select { |e| e.first == :mysql }
-```
-
-Notice that the documentation for the `assert_sql_events` method is cutoff. Use your `read_file` tool function to get the whole test helper source code and gain better understanding of how it is intended to be used, with the side benefit of also being able to see how it is implemented.
-
-Note: You will undoubtedly already be familiar with some of Minitest and RSpec's built-in helpers. There is no need to search for those, since they are packaged as gems you won't find them anyway.
-
-DO NOT FORGET TO PREPEND `def` TO YOUR QUERY TO FIND A METHOD DEFINITION INSTEAD OF USAGES, otherwise you may bring back a very large and useless result set!!!
-
-Once you are done understanding the custom test helpers used in the test file, analyze and report on whether it seems like any of the helpers are:
-
-1. Used incorrectly
-2. Used unnecessarily
-3. Any other problem related to the use of helper methods
-
-Where possible, use your best judgment to make recommendations for how to fix problems that you find, but ONLY related to test helpers.
-
-Note: You are only being used to help find problems so it is not necessary to report on correct usage of helpers or to make positive comments.

data/examples/grading/workflow.md

@@ -1,8 +0,0 @@
-As a senior software engineer and testing expert, evaluate the quality of this test file based on guidelines that will be subsequently provided.
-
-Next I will now provide the source code of the test that we will be analyzing, and then step you through a series of analysis activities, before finally asking you to provided a final report.
-
-<test>
-# <%= file %>
-<%= File.read(file) %>
-</test>

data/examples/grading/workflow.rb.md

@@ -1,6 +0,0 @@
-As a senior Ruby engineer and testing expert, evaluate the quality of this Ruby test file. Next I will now provide the source code of the test that we will be analyzing, and then step you through a series of analysis activities, before finally asking you to provided a final report.
-
-<test>
-# <%= file %>
-<%= File.read(file) %>
-</test>

data/examples/grading/workflow.ts+tsx.md

@@ -1,6 +0,0 @@
-As a senior front-end engineer and testing expert, evaluate the quality of this test file. Next I will now provide the source code of the test that we will be analyzing, and then step you through a series of analysis activities, before finally asking you to provided a final report.
-
-<test>
-# <%= file %>
-<%= File.read(file) %>
-</test>

data/examples/grading/workflow.yml

@@ -1,41 +0,0 @@
-name: Test Grading
-api_token: $(echo $OPENAI_API_KEY)
-# model: anthropic:claude-opus-4
-model: gpt-4.1-mini
-
-tools:
-  - Roast::Tools::Grep
-  - Roast::Tools::ReadFile
-  - Roast::Tools::SearchFile
-
-# Uncomment this to run the workflow on modified tests automatically
-# each: '% cd $(git rev-parse --show-toplevel) && git status --porcelain | grep "_test\.rb" | cut -c4- | xargs realpath'
-
-steps:
-  - read_dependencies
-  - run_coverage
-  -
-    - analyze_coverage
-    - verify_test_helpers
-    - verify_mocks_and_stubs
-  - generate_grades
-  - calculate_final_grade
-  - format_result
-  - generate_recommendations
-
-# set non-default attributes for steps below
-analyze_coverage:
-#  model: gpt-4.1-mini
-  json: true
-  
-generate_grades:
-#  model: o3
-  json: true
-
-generate_recommendations:
-#  model: o3
-  json: true
-  params:
-    max_completion_tokens: 5_000
-
-

data/examples/instrumentation.rb

@@ -1,76 +0,0 @@
-# frozen_string_literal: true
-
-# Demonstration of how to use the Roast instrumentation hooks
-# This file would typically be placed in PROJECT_ROOT/.roast/initializers/
-# for automatic loading during workflow execution
-
-# Example: Log all workflow and step events
-ActiveSupport::Notifications.subscribe(/roast\.workflow\./) do |name, start, finish, _id, payload|
-  duration = finish - start
-
-  case name
-  when "roast.workflow.start"
-    puts "\n🚀 Workflow starting: #{payload[:name]}"
-    puts "   Path: #{payload[:workflow_path]}"
-    puts "   Options: #{payload[:options]}"
-  when "roast.workflow.complete"
-    status = payload[:success] ? "✅ Successfully" : "❌ With errors"
-    puts "\n#{status} completed workflow in #{duration.round(2)} seconds"
-  end
-end
-
-# Example: Track step execution times
-ActiveSupport::Notifications.subscribe(/roast\.step\./) do |name, start, finish, _id, payload|
-  duration = finish - start
-
-  case name
-  when "roast.step.start"
-    puts "\n  ▶️ Step starting: #{payload[:step_name]}"
-  when "roast.step.complete"
-    puts "  ✅ Step completed: #{payload[:step_name]} (#{duration.round(3)}s)"
-  when "roast.step.error"
-    puts "  ❌ Step failed: #{payload[:step_name]}"
-    puts "     Error: #{payload[:error]} - #{payload[:message]}"
-  end
-end
-
-# Example: Monitor AI interactions
-ActiveSupport::Notifications.subscribe(/roast\.chat_completion\./) do |name, start, finish, _id, payload|
-  case name
-  when "roast.chat_completion.start"
-    puts "\n  🤖 AI request starting (model: #{payload[:model]})"
-    puts "     Parameters: #{payload[:parameters].inspect}" if payload[:parameters].any?
-  when "roast.chat_completion.complete"
-    duration = finish - start
-    puts "  🤖 AI request completed in #{duration.round(2)}s (execution time: #{payload[:execution_time].round(2)}s)"
-    puts "     Response size: #{payload[:response_size]} characters"
-  when "roast.chat_completion.error"
-    puts "  🤖 AI request failed: #{payload[:error]} - #{payload[:message]}"
-    puts "     Execution time: #{payload[:execution_time].round(2)}s"
-  end
-end
-
-# Example: Track tool executions
-ActiveSupport::Notifications.subscribe(/roast\.tool\./) do |name, _start, _finish, _id, payload|
-  case name
-  when "roast.tool.execute"
-    puts "\n  🔧 Executing tool: #{payload[:function_name]}"
-  when "roast.tool.complete"
-    puts "  🔧 Tool completed: #{payload[:function_name]} (#{payload[:execution_time].round(3)}s)"
-    puts "     Cache enabled: #{payload[:cache_enabled]}"
-  when "roast.tool.error"
-    puts "  🔧 Tool failed: #{payload[:function_name]}"
-    puts "     Error: #{payload[:error]} - #{payload[:message]}"
-    puts "     Execution time: #{payload[:execution_time].round(3)}s"
-  end
-end
-
-# In a Shopify-specific initializer (e.g., .roast/initializers/monorail.rb),
-# you could replace these logging examples with actual Monorail tracking:
-#
-# ActiveSupport::Notifications.subscribe("roast.workflow.start") do |name, start, finish, id, payload|
-#   Roast::Support::Monorail.track_command("run", {
-#     "workflow_path" => payload[:workflow_path],
-#     "name" => payload[:name]
-#   })
-# end

data/examples/interpolation/README.md

@@ -1,50 +0,0 @@
-# Interpolation Example
-
-This example demonstrates how to use Roast's interpolation feature to create dynamic workflows.
-
-## Overview
-
-The workflow in this example:
-1. Analyzes a file and extracts its metadata
-2. Extracts patterns based on the file type
-3. Dynamically selects a report generation step based on the file extension
-4. Outputs a completion message using the file's basename
-
-## Interpolation Examples
-
-The workflow demonstrates several types of interpolation:
-
-- `{{ }}` syntax for embedding dynamic values
-- Access to file metadata via expressions like `{{file_basename}}` and `{{file_ext}}`
-- Dynamic step selection with `generate_report_for_{{file_ext}}`
-- Shell command interpolation with `$(echo "Processing completed for file: {{file_basename}}")`
-
-## Running the Example
-
-To run this example with a Ruby file:
-
-```bash
-roast execute workflow.yml /path/to/some_file.rb
-```
-
-Or with a JavaScript file:
-
-```bash
-roast execute workflow.yml /path/to/some_file.js
-```
-
-The workflow will:
-1. Extract the file's basename and extension
-2. Store these in the workflow context
-3. Dynamically choose a report generator based on file extension
-4. Create a markdown report file
-5. Output a completion message with the filename
-
-## How Interpolation Works
-
-1. When Roast processes a step name or shell command, it looks for `{{ }}` patterns
-2. Expressions inside `{{ }}` are evaluated in the workflow's context using Ruby's `instance_eval`
-3. This allows access to the workflow's variables, methods, and output hash
-4. The evaluated expressions replace the `{{ }}` patterns in the step name or command
-
-This makes workflows dynamic and able to respond to different inputs without code changes.

data/examples/interpolation/analyze_file/prompt.md

@@ -1 +0,0 @@
-Analyze the file at: <%= workflow.file %>

data/examples/interpolation/analyze_patterns/prompt.md

@@ -1,27 +0,0 @@
-Extract some patterns about this file and return in json format like this:
-
-<json>
-{
-  "code_patterns": {
-    "class_structure": {
-      "name": "Calculator",
-      "instance_variables": ["@memory"],
-      "method_count": 7,
-      "method_types": {
-        "constructor": ["initialize"],
-        "operations": ["add", "subtract", "multiply", "divide"],
-        "accessors": ["memory"],
-        "utility": ["clear"]
-      }
-    },
-    "error_handling": {
-      "techniques": ["conditional raise", "zero check"],
-      "examples": ["raise \"Division by zero!\" if number.zero?"]
-    },
-    "design_patterns": {
-      "state": "Uses instance variable to maintain calculator state",
-      "command": "Each operation method modifies the internal state"
-    }
-  }
-}
-</json>

data/examples/interpolation/generate_report_for_js/prompt.md

@@ -1,3 +0,0 @@
-Generate a nicely formatted report based on the following metadata:
-
-<%= workflow.output[:analyze_patterns] %>

data/examples/interpolation/generate_report_for_rb/prompt.md

@@ -1,3 +0,0 @@
-Generate a nicely formatted report based on the following metadata:
-
-<%= workflow.output[:analyze_patterns] %>

data/examples/interpolation/sample.js

@@ -1,48 +0,0 @@
-// Sample JavaScript file for testing interpolation in workflows
-
-class Calculator {
-  constructor() {
-    this.memory = 0;
-  }
-
-  add(number) {
-    this.memory += number;
-    return this.memory;
-  }
-
-  subtract(number) {
-    this.memory -= number;
-    return this.memory;
-  }
-
-  multiply(number) {
-    this.memory *= number;
-    return this.memory;
-  }
-
-  divide(number) {
-    if (number === 0) {
-      throw new Error("Division by zero!");
-    }
-    this.memory /= number;
-    return this.memory;
-  }
-
-  getMemory() {
-    return this.memory;
-  }
-
-  clear() {
-    this.memory = 0;
-    return this.memory;
-  }
-}
-
-// Example usage
-if (require.main === module) {
-  const calc = new Calculator();
-  calc.add(10);
-  calc.multiply(2);
-  calc.subtract(5);
-  console.log(`Result: ${calc.getMemory()}`);
-}

data/examples/interpolation/sample.rb

@@ -1,42 +0,0 @@
-# frozen_string_literal: true
-
-# Sample Ruby file for testing interpolation in workflows
-
-class Calculator
-  def initialize
-    @memory = 0
-  end
-
-  def add(number)
-    @memory += number
-  end
-
-  def subtract(number)
-    @memory -= number
-  end
-
-  def multiply(number)
-    @memory *= number
-  end
-
-  def divide(number)
-    raise "Division by zero!" if number.zero?
-
-    @memory /= number
-  end
-
-  attr_reader :memory
-
-  def clear
-    @memory = 0
-  end
-end
-
-# Example usage
-if __FILE__ == $PROGRAM_NAME
-  calc = Calculator.new
-  calc.add(10)
-  calc.multiply(2)
-  calc.subtract(5)
-  puts "Result: #{calc.memory}"
-end

data/examples/interpolation/workflow.md

@@ -1 +0,0 @@
-You are a good robot.

data/examples/interpolation/workflow.yml

@@ -1,21 +0,0 @@
-name: interpolation_example
-model: gpt-4o-mini
-
-tools:
-  - Roast::Tools::ReadFile
-
-steps:
-  - analyze_file
-  - analyze_patterns
-  - generate_report_for_{{File.extname(workflow.file).sub('.', '')}}
-  - '$(echo "Processing completed for file: {{File.basename(workflow.file)}}")'
-
-analyze_patterns:
-  json: true
-
-generate_report_for_rb:
-  print_response: true
-
-generate_report_for_md:
-  print_response: true
-  

data/examples/iteration/IMPLEMENTATION.md

@@ -1,88 +0,0 @@
-# Iteration Mechanisms Implementation
-
-This document provides an overview of how the iteration mechanisms are implemented in Roast.
-
-## Core Components
-
-### 1. Schema Extensions
-
-The workflow schema has been extended to support two new iteration constructs:
-
-- **Repeat** - For conditional repetition until a condition is met
-- **Each** - For iterating over collections with a variable binding
-
-These schema extensions define the structure and validation rules for the iteration YAML syntax.
-
-### 2. Step Classes
-
-Two new step classes handle the actual iteration logic:
-
-- **RepeatStep** - Executes steps repeatedly until a condition is met or a maximum iteration count is reached
-- **EachStep** - Iterates over a collection, binding each item to a variable, and executes steps for each item
-
-Both inherit from a common `BaseIterationStep` class that provides shared functionality.
-
-### 3. Pattern Matching
-
-The `WorkflowExecutor` has been enhanced with pattern matching to recognize the `repeat` and `each` keywords and dispatch to the appropriate step classes:
-
-```ruby
-case name
-when "repeat"
-  execute_repeat_step(command)
-when "each"
-  # Handle 'each' step with its special format
-  execute_each_step(step)
-else
-  # Handle regular steps
-end
-```
-
-### 4. State Management
-
-Both iteration types include state management to:
-
-- Track current iteration number
-- Save state after each iteration
-- Support resumption after failures
-- Provide safety limits against infinite loops
-
-## Iteration Flow
-
-### RepeatStep Flow
-
-1. Start with iteration count = 0
-2. Execute the nested steps in sequence
-3. Evaluate the until condition
-4. If condition is true or max_iterations is reached, stop
-5. Otherwise, increment iteration count and go back to step 2
-6. Return the results of all iterations
-
-### EachStep Flow
-
-1. Resolve the collection expression
-2. For each item in the collection:
-   a. Set the named variable (accessible in steps through a getter method)
-   b. Execute the nested steps
-   c. Save state
-3. Return the results from all iterations
-
-## Safety Mechanisms
-
-- **max_iterations** parameter prevents infinite loops
-- State is saved after each iteration for resumption capability
-- Robust error handling during condition evaluation and step execution
-- Collection type checking ensures iterable objects
-
-## Usage Examples
-
-The workflow.yml and step files in this directory demonstrate practical applications of these iteration mechanisms for code quality analysis.
-
-## Integration with Existing Workflow Engine
-
-The iteration mechanism integrates seamlessly with the existing workflow engine:
-
-- Uses the same state persistence mechanisms
-- Follows the same execution models
-- Maintains compatibility with all existing steps
-- Supports interpolation within iteration constructs

data/examples/iteration/README.md

@@ -1,68 +0,0 @@
-# Code Quality Analysis Workflow
-
-This example demonstrates the use of Roast's iteration features to analyze and improve code quality across a codebase.
-
-## What it does
-
-1. Collects Ruby files from the codebase for analysis
-2. Analyzes each file for complexity, code smells, and potential improvements
-3. Generates recommendations for each file
-4. Prioritizes the identified issues by impact and difficulty
-5. Automatically implements fixes for the highest-priority issues
-6. Verifies each fix before moving to the next one
-7. Continues until either 5 fixes have been applied or all issues are addressed
-8. Generates a summary report of changes made
-
-## Iteration Features Demonstrated
-
-### Collection Iteration with `each`
-
-The workflow uses the `each` construct to iterate through Ruby files:
-
-```yaml
-- each: "output['get_files_to_analyze'].split('\n')"
-  as: "current_file"
-  steps:
-    - read_file
-    - analyze_complexity
-    - generate_recommendations
-```
-
-This makes the current file available as `current_file` in each step, allowing the analysis steps to process each file individually.
-
-### Conditional Repetition with `repeat`
-
-The workflow uses the `repeat` construct to iteratively fix issues until a condition is met:
-
-```yaml
-- repeat:
-    steps:
-      - select_next_issue
-      - implement_fix
-      - verify_fix
-      - update_fix_count
-    until: "output['update_fix_count']['fixes_applied'] >= 5 || output['select_next_issue']['no_issues_left'] == true"
-    max_iterations: 10
-```
-
-This continues applying fixes until either:
-- 5 fixes have been successfully applied
-- No more issues remain to be fixed
-- The maximum of 10 iterations is reached (safety limit)
-
-## Running the Example
-
-To run this workflow:
-
-```bash
-roast run examples/iteration/workflow.yml --target=/path/to/your/project
-```
-
-The workflow will analyze the Ruby files in your project, suggest improvements, and apply the highest-priority fixes.
-
-## Customizing
-
-- Adjust the file selection criteria in `get_files_to_analyze`
-- Modify the analysis criteria in `analyze_complexity`
-- Change the fix limit in the `until` condition
-- Set a different `max_iterations` value to control the maximum number of fixes

data/examples/iteration/analyze_complexity/prompt.md

@@ -1,22 +0,0 @@
-I'll analyze the code complexity of the file <%= current_file %>, which I've read in the previous step.
-
-I'll examine the following aspects:
-1. Cyclomatic complexity (number of decision paths)
-2. Method length and complexity
-3. Class size and responsibilities
-4. Code smells (long parameter lists, deeply nested blocks, etc.)
-5. Potential performance bottlenecks
-6. Opportunities for refactoring
-
-```ruby
-<%= output.read_file %>
-```
-
-Based on this analysis, I'll provide a structured assessment of the code quality issues found.
-
-For each issue identified, I'll assign:
-- Severity (high, medium, low)
-- Type (complexity, maintainability, performance, style)
-- Location (line numbers, method/class names)
-- Brief description of the issue
-- Suggested improvement