roast-ai 0.2.3 → 0.3.1

data/examples/step_configuration/README.md

@@ -10,7 +10,6 @@ This example demonstrates how to configure various step types in Roast workflows
 All step types support the following configuration options:
 
 - `model`: The AI model to use (e.g., "gpt-4o", "claude-3-opus")
-- `loop`: Whether to enable auto-looping (true/false)
 - `print_response`: Whether to print the response to stdout (true/false)
 - `json`: Whether to expect a JSON response (true/false)
 - `params`: Additional parameters to pass to the model (e.g., temperature, max_tokens)
@@ -30,7 +29,6 @@ Inline prompts can be configured in two ways:
 ```yaml
 analyze the code:
   model: gpt-4o
-  loop: false
   print_response: true
 ```
 
@@ -52,7 +50,6 @@ each:
   each: "{{files}}"
   as: file
   model: gpt-3.5-turbo
-  loop: false
   steps:
     - process {{file}}
 

data/examples/step_configuration/workflow.yml

@@ -3,12 +3,10 @@ description: Demonstrates how to configure various step types including inline p
 
 # Global configuration that applies to all steps
 model: openai/gpt-4o-mini
-loop: true
 
 # Configuration for specific steps
 summarize the code:
   model: claude-3-opus  # Override global model
-  loop: false          # Disable auto-loop for this step
   print_response: true # Print the response
   json: false          # Don't expect JSON response
   params:
@@ -25,7 +23,6 @@ each:
   each: "{{files}}"
   as: file
   model: gpt-3.5-turbo # Use a faster model for iteration
-  loop: false          # Don't loop on each iteration
   steps:
     - process {{file}}
 

data/examples/tool_config_example/README.md

@@ -0,0 +1,109 @@
+# Tool Configuration Example
+
+This example demonstrates how to configure tools with specific settings in Roast workflows.
+
+## Overview
+
+Starting with this update, Roast supports configuring tools with specific settings directly in the workflow YAML file. This is particularly useful for tools like `Roast::Tools::Cmd` where you might want to restrict which commands can be executed.
+
+## Configuration Syntax
+
+Tools can be configured in two ways:
+
+### 1. Simple String Format (No Configuration)
+```yaml
+tools:
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::Grep
+```
+
+### 2. Hash Format (With Configuration)
+```yaml
+tools:
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - ls
+        - pwd
+        - echo
+```
+
+### 3. Mixed Format
+You can mix both formats in the same workflow:
+
+```yaml
+tools:
+  - Roast::Tools::ReadFile
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - ls
+        - pwd
+        - ruby
+        - sed
+  - Roast::Tools::WriteFile
+  - Roast::Tools::SearchFile
+```
+
+## Example: Configuring Allowed Commands
+
+The `Roast::Tools::Cmd` tool now supports an `allowed_commands` configuration that restricts which commands can be executed:
+
+```yaml
+tools:
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - ls
+        - pwd
+        - echo
+        - cat
+        - ruby
+        - rake
+```
+
+### Enhanced Command Configuration with Descriptions
+
+You can also provide custom descriptions for commands to help the LLM understand their purpose:
+
+```yaml
+tools:
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - ls
+        - pwd
+        - name: echo
+          description: "echo command - output text to stdout, supports > for file redirection"
+        - name: cat
+          description: "cat command - display file contents, concatenate files, works with pipes"
+```
+
+This mixed format allows you to:
+- Use simple strings for commands with good default descriptions
+- Provide custom descriptions for commands that need more context
+- Help the LLM make better decisions about which command to use
+
+With this configuration:
+- ✅ `ls -la` will work
+- ✅ `echo "Hello World"` will work
+- ❌ `rm file.txt` will be rejected (not in allowed list)
+- ❌ `git status` will be rejected (not in allowed list)
+
+## Default Behavior
+
+If no configuration is provided for `Roast::Tools::Cmd`, it uses the default allowed commands:
+- pwd
+- find
+- ls
+- rake
+- ruby
+- dev
+- mkdir
+
+## Running the Example
+
+To run this example workflow:
+
+```bash
+bin/roast execute examples/tool_config_example/workflow.yml
+```
+
+The workflow will validate the tool configuration by executing various commands and demonstrating which ones are allowed and which are rejected based on the configuration.

data/examples/tool_config_example/example_step/prompt.md

@@ -0,0 +1,42 @@
+# Tool Configuration Validation
+
+Execute the following commands using the cmd tool:
+
+1. `ls -la`
+2. `pwd`
+3. `echo "Hello from configured commands!"`
+4. `git status`
+
+RESPONSE FORMAT
+You must respond in JSON format within <json> XML tags.
+
+<json>
+{
+  "commands": [
+    {
+      "command": "ls -la",
+      "exit_status": 0,
+      "output": "total 208\ndrwxr-xr-x@ 31 user  staff...",
+      "success": true
+    },
+    {
+      "command": "pwd",
+      "exit_status": 0,
+      "output": "/Users/user/project",
+      "success": true
+    },
+    {
+      "command": "echo \"Hello from configured commands!\"",
+      "exit_status": 0,
+      "output": "Hello from configured commands!",
+      "success": true
+    },
+    {
+      "command": "git status",
+      "exit_status": null,
+      "output": "Error: Command not allowed. Only commands starting with ls, pwd, echo, cat are permitted.",
+      "success": false
+    }
+  ]
+}
+</json>

data/examples/tool_config_example/workflow.yml

@@ -0,0 +1,17 @@
+name: Tool Configuration Example
+model: default
+tools:
+  - Roast::Tools::ReadFile
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - ls
+        - pwd
+        - name: echo
+          description: "echo command - output text to stdout, supports > for file redirection"
+        - name: cat
+          description: "cat command - display file contents, concatenate files, works with pipes"
+  - Roast::Tools::WriteFile
+
+steps:
+  - example_step
+

data/examples/workflow_generator/workflow.yml

@@ -23,7 +23,6 @@ steps:
 get_user_input:
   print_response: false
   json: true
-  auto_loop: false
 
 analyze_user_request:
   print_response: true

data/lib/roast/helpers/function_caching_interceptor.rb

@@ -1,9 +1,5 @@
 # frozen_string_literal: true
 
-require "active_support"
-require "active_support/isolated_execution_state"
-require "active_support/cache"
-require "active_support/notifications"
 require "roast/helpers/logger"
 
 module Roast

data/lib/roast/helpers/prompt_loader.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-require "active_support/core_ext/string"
 require "erb"
 
 module Roast

data/lib/roast/tools/bash.rb

@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+require "English"
+require "roast/helpers/logger"
+
+module Roast
+  module Tools
+    module Bash
+      extend self
+
+      class << self
+        def included(base)
+          base.class_eval do
+            function(
+              :bash,
+              "Execute any bash command without restrictions. ⚠️ WARNING: Use only in trusted environments!",
+              command: { type: "string", description: "The bash command to execute" },
+            ) do |params|
+              Roast::Tools::Bash.call(params[:command])
+            end
+          end
+        end
+      end
+
+      def call(command)
+        Roast::Helpers::Logger.info("🚀 Executing bash command: #{command}\n")
+
+        # Show warning unless explicitly disabled
+        if ENV["ROAST_BASH_WARNINGS"] != "false"
+          Roast::Helpers::Logger.warn("⚠️  WARNING: Unrestricted bash execution - use with caution!\n")
+        end
+
+        # Execute the command without any restrictions
+        result = ""
+        IO.popen("#{command} 2>&1", chdir: Dir.pwd) do |io|
+          result = io.read
+        end
+
+        exit_status = $CHILD_STATUS.exitstatus
+
+        format_output(command, result, exit_status)
+      rescue StandardError => e
+        handle_error(e)
+      end
+
+      private
+
+      def format_output(command, result, exit_status)
+        "Command: #{command}\n" \
+          "Exit status: #{exit_status}\n" \
+          "Output:\n#{result}"
+      end
+
+      def handle_error(error)
+        error_message = "Error running command: #{error.message}"
+        Roast::Helpers::Logger.error("#{error_message}\n")
+        Roast::Helpers::Logger.debug("#{error.backtrace.join("\n")}\n") if ENV["DEBUG"]
+        error_message
+      end
+    end
+  end
+end

data/lib/roast/tools/cmd.rb

@@ -8,64 +8,151 @@ module Roast
     module Cmd
       extend self
 
+      DEFAULT_ALLOWED_COMMANDS = [
+        { "name" => "pwd", "description" => "pwd command - print current working directory path" },
+        { "name" => "find", "description" => "find command - search for files/directories using patterns like -name '*.rb'" },
+        { "name" => "ls", "description" => "ls command - list directory contents with options like -la, -R" },
+        { "name" => "rake", "description" => "rake command - run Ruby tasks defined in Rakefile" },
+        { "name" => "ruby", "description" => "ruby command - execute Ruby code or scripts, supports -e for inline code" },
+        { "name" => "dev", "description" => "Shopify dev CLI - development environment tool with subcommands" },
+        { "name" => "mkdir", "description" => "mkdir command - create directories, supports -p for parent directories" },
+      ].freeze
+
+      CONFIG_ALLOWED_COMMANDS = "allowed_commands"
+      private_constant :DEFAULT_ALLOWED_COMMANDS, :CONFIG_ALLOWED_COMMANDS
+
       class << self
         # Add this method to be included in other classes
         def included(base)
+          @base_class = base
+        end
+
+        # Called after configuration is loaded
+        def post_configuration_setup(base, config = {})
+          allowed_commands = config[CONFIG_ALLOWED_COMMANDS] || DEFAULT_ALLOWED_COMMANDS
+
+          allowed_commands.each do |command_entry|
+            case command_entry
+            when String
+              register_command_function(base, command_entry, nil)
+            when Hash
+              command_name = command_entry["name"] || command_entry[:name]
+              description = command_entry["description"] || command_entry[:description]
+
+              if command_name.nil?
+                raise ArgumentError, "Command configuration must include 'name' field"
+              end
+
+              register_command_function(base, command_name, description)
+            else
+              raise ArgumentError, "Invalid command configuration format: #{command_entry.inspect}"
+            end
+          end
+        end
+
+        private
+
+        def register_command_function(base, command, custom_description = nil)
+          function_name = command.to_sym
+          description = custom_description || generate_command_description(command)
+
           base.class_eval do
             function(
-              :cmd,
-              'Run a command in the current working directory (e.g. "ls", "rake", "ruby"). ' \
-                "You may use this tool to execute tests and verify if they pass.",
-              command: { type: "string", description: "The command to run in a bash shell." },
+              function_name,
+              description,
+              args: {
+                type: "string",
+                description: "Arguments to pass to the #{command} command",
+                required: false,
+              },
             ) do |params|
-              Roast::Tools::Cmd.call(params[:command])
+              full_command = if params[:args].nil? || params[:args].empty?
+                command
+              else
+                "#{command} #{params[:args]}"
+              end
+
+              Roast::Tools::Cmd.execute_allowed_command(full_command, command)
             end
           end
         end
+
+        def generate_command_description(command)
+          default_cmd = DEFAULT_ALLOWED_COMMANDS.find { |cmd| cmd["name"] == command }
+          default_cmd ? default_cmd["description"] : "Execute the #{command} command"
+        end
+      end
+
+      def execute_allowed_command(full_command, command_prefix)
+        Roast::Helpers::Logger.info("🔧 Running command: #{full_command}\n")
+        execute_command(full_command, command_prefix)
+      rescue StandardError => e
+        handle_error(e)
       end
 
-      def call(command)
+      # Legacy method for backward compatibility
+      def call(command, config = {})
         Roast::Helpers::Logger.info("🔧 Running command: #{command}\n")
 
-        # Validate the command starts with one of the allowed prefixes
-        allowed_prefixes = ["pwd", "find", "ls", "rake", "ruby", "dev", "mkdir"]
+        allowed_commands = config[CONFIG_ALLOWED_COMMANDS] || DEFAULT_ALLOWED_COMMANDS
+        validation_result = validate_command(command, allowed_commands)
+        return validation_result unless validation_result.nil?
+
         command_prefix = command.split(" ").first
+        execute_command(command, command_prefix)
+      rescue StandardError => e
+        handle_error(e)
+      end
 
-        err = "Error: Command not allowed. Only commands starting with #{allowed_prefixes.join(", ")} are permitted."
-        return err unless allowed_prefixes.any? do |prefix|
-          command_prefix == prefix
-        end
+      private
+
+      def validate_command(command, allowed_commands)
+        command_prefix = command.split(" ").first
+
+        # Extract command names from the allowed_commands array
+        allowed_command_names = allowed_commands.map do |cmd_entry|
+          case cmd_entry
+          when String
+            cmd_entry
+          when Hash
+            cmd_entry["name"] || cmd_entry[:name]
+          end
+        end.compact
 
-        # Execute the command in the current working directory
-        result = ""
+        return if allowed_command_names.include?(command_prefix)
+
+        "Error: Command not allowed. Only commands starting with #{allowed_command_names.join(", ")} are permitted."
+      end
 
-        # Use a full shell environment for commands, especially for 'dev'
-        if command_prefix == "dev"
+      def extract_tool_config
+        return {} unless respond_to?(:configuration)
+
+        configuration&.tool_config("Roast::Tools::Cmd") || {}
+      end
+
+      def execute_command(command, command_prefix)
+        result = if command_prefix == "dev"
           # Use bash -l -c to ensure we get a login shell with all environment variables
           full_command = "bash -l -c '#{command.gsub("'", "\\'")}'"
-          IO.popen(full_command, chdir: Dir.pwd) do |io|
-            result = io.read
-          end
+          IO.popen(full_command, chdir: Dir.pwd, &:read)
         else
-          # For other commands, use the original approach
-          IO.popen(command, chdir: Dir.pwd) do |io|
-            result = io.read
-          end
+          IO.popen(command, chdir: Dir.pwd, &:read)
         end
 
-        exit_status = $CHILD_STATUS.exitstatus
+        format_output(command, result, $CHILD_STATUS.exitstatus)
+      end
 
-        # Return the command output along with exit status information
-        output = "Command: #{command}\n"
-        output += "Exit status: #{exit_status}\n"
-        output += "Output:\n#{result}"
+      def format_output(command, result, exit_status)
+        "Command: #{command}\n" \
+          "Exit status: #{exit_status}\n" \
+          "Output:\n#{result}"
+      end
 
-        output
-      rescue StandardError => e
-        "Error running command: #{e.message}".tap do |error_message|
-          Roast::Helpers::Logger.error(error_message + "\n")
-          Roast::Helpers::Logger.debug(e.backtrace.join("\n") + "\n") if ENV["DEBUG"]
-        end
+      def handle_error(error)
+        error_message = "Error running command: #{error.message}"
+        Roast::Helpers::Logger.error("#{error_message}\n")
+        Roast::Helpers::Logger.debug("#{error.backtrace.join("\n")}\n") if ENV["DEBUG"]
+        error_message
       end
     end
   end

data/lib/roast/tools/coding_agent.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 require "roast/helpers/logger"
+require "roast/tools/helpers/coding_agent_message_formatter"
+require "json"
 require "open3"
 require "tempfile"
 require "securerandom"
@@ -10,7 +12,16 @@ module Roast
     module CodingAgent
       extend self
 
+      class CodingAgentError < StandardError; end
+
+      CONFIG_CODING_AGENT_COMMAND = "coding_agent_command"
+      private_constant :CONFIG_CODING_AGENT_COMMAND
+
+      @configured_command = nil
+
       class << self
+        attr_accessor :configured_command
+
         def included(base)
           base.class_eval do
             function(
@@ -22,6 +33,11 @@ module Roast
             end
           end
         end
+
+        # Called after configuration is loaded
+        def post_configuration_setup(base, config = {})
+          self.configured_command = config[CONFIG_CODING_AGENT_COMMAND]
+        end
       end
 
       def call(prompt)
@@ -50,20 +66,83 @@ module Roast
           temp_file.write(prompt)
           temp_file.close
 
-          # Run Claude Code CLI using the temp file as input
-          claude_code_command = ENV.fetch("CLAUDE_CODE_COMMAND", "claude -p")
-          stdout, stderr, status = Open3.capture3("cat #{temp_file.path} | #{claude_code_command}")
+          # Run Claude Code CLI using the temp file as input with streaming output
+          expect_json_output = claude_code_command.include?("--output-format stream-json") ||
+            claude_code_command.include?("--output-format json")
+          command = "cat #{temp_file.path} | #{claude_code_command}"
+          result = ""
 
-          if status.success?
-            stdout
-          else
-            "Error running ClaudeCode: #{stderr}"
+          Open3.popen3(command) do |stdin, stdout, stderr, wait_thread|
+            stdin.close
+            if expect_json_output
+              stdout.each_line do |line|
+                json = parse_json(line)
+                next unless json
+
+                handle_intermediate_message(json)
+                result += handle_result(json) || ""
+              end
+            else
+              result = stdout.read
+            end
+
+            status = wait_thread.value
+            if status.success?
+              return result
+            else
+              error_output = stderr.read
+              return "Error running CodingAgent: #{error_output}"
+            end
           end
         ensure
           # Always clean up the temp file
           temp_file.unlink
         end
       end
+
+      def parse_json(line)
+        JSON.parse(line)
+      rescue JSON::ParserError => e
+        Roast::Helpers::Logger.warn("🤖 Error parsing JSON response: #{e}\n")
+        nil
+      end
+
+      def handle_intermediate_message(json)
+        case json["type"]
+        when "assistant", "user"
+          CodingAgentMessageFormatter.format_messages(json).each(&method(:log_message))
+        when "result", "system"
+          # Ignore these message types
+        else
+          Roast::Helpers::Logger.debug("🤖 Encountered unexpected message type: #{json["type"]}\n")
+        end
+      end
+
+      def handle_result(json)
+        if json["type"] == "result"
+          # NOTE: the format of an error response is { "subtype": "success", "is_error": true }
+          if json["is_error"]
+            raise CodingAgentError, json["result"]
+          elsif json["subtype"] == "success"
+            json["result"]
+          else
+            raise CodingAgentError, "CodingAgent did not complete successfully: #{line}"
+          end
+        end
+      end
+
+      def log_message(text)
+        return if text.blank?
+
+        text = text.lines.map do |line|
+          "\t#{line}"
+        end.join
+        Roast::Helpers::Logger.info("• " + text.chomp + "\n")
+      end
+
+      def claude_code_command
+        CodingAgent.configured_command || ENV["CLAUDE_CODE_COMMAND"] || "claude -p --verbose --output-format stream-json"
+      end
     end
   end
 end