roast-ai 0.3.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 28055339ac18d63a1abd7151943558a114df2e48930c63aa63c65536d193061b
-  data.tar.gz: c5be17b70caa32f73dca316ab4816d0b92c0c7121747b3e5492f58124e815f87
+  metadata.gz: 4f94565fdc94547f9ab2fa57b401930d189c2e63f5f31897165a07cf6828b10d
+  data.tar.gz: '068450417abc06ed7fb98af7c36ec972d6671b23e0e85cfdadde8a934c327fdc'
 SHA512:
-  metadata.gz: d5b300fd001d8cc55ecdb5751bc891840732f63be8e951e2c4333a178fde6c2197a2b0063f3add45b48e85251e9b2ee28119d237322bb8f89af86c2208345c1f
-  data.tar.gz: c579010aedef591cd1a72fb6cd34c15db38fc225550aa17a841397e5f8849c09c5f87cf81da3f04f2300c5dcac704ab2d8a0f78fe8fd443fabfe62e67780a015
+  metadata.gz: 07e5a581c469a37d3f0371627c92ef0b465cfeb5e977a90221a3cdb7783193de7a1c6600df3b1235705ef65e0e76f566bf88730621172314d3bc0ba70adc5dc4
+  data.tar.gz: 12e7ae70eee847f11eac8c4e593e11b5a94c0574634254a559d41d006e8fac01dc7b7c26fa2ba1a80fc13de0172ac810ff390e8bda2555e96489c0385f815537

data/CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.1] - 2025-06-05
+
+### Added
+- Default `print_response: true` for the last step in a workflow (#100)
+  - The last step now automatically prints its response unless explicitly configured otherwise
+  - Helps newcomers who would otherwise see no output from their workflows
+  - Works with all step types: string steps, hash steps with variable assignment, and conditional steps
+  - Parallel steps and iteration steps are intelligently handled (no automatic output since there's no single "last" step)
+
+### Fixed
+- PromptStep now properly passes `print_response`, `json`, and `params` parameters to chat_completion
+
 ## [0.3.0] - 2025-06-04
 
 ### Changed

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.3.0)
+    roast-ai (0.3.1)
       activesupport (>= 7.0)
       cli-ui
       diff-lcs (~> 1.5)

data/README.md

@@ -270,6 +270,31 @@ Roast supports several types of steps:
    ```
    This creates a simple prompt-response interaction without tool calls or looping. It's detected by the presence of spaces in the step name and is useful for summarization or simple questions at the end of a workflow.
 
+#### Step Configuration
+
+Steps can be configured with various options to control their behavior:
+
+```yaml
+steps:
+  - analyze_code           # Simple step reference
+  - generate_report:       # Step with configuration
+      model: gpt-4o        # Override the global model for this step
+      print_response: true # Explicitly control output printing
+      json: true           # Request JSON-formatted response
+      params:              # Additional parameters for the API call
+        temperature: 0.8
+```
+
+**Configuration options:**
+- `model`: Override the workflow's default model for this specific step
+- `print_response`: Control whether the step's response is included in the final output (default: `false`, except for the last step which defaults to `true` as of v0.3.1)
+- `json`: Request a JSON-formatted response from the model
+- `params`: Additional parameters passed to the model API (temperature, max_tokens, etc.)
+- `path`: Custom directory path for the step's prompt files
+- `coerce_to`: Type coercion for the step result (`:boolean`, `:llm_boolean`, `:iterable`)
+
+**Automatic Last Step Output**: As of version 0.3.1, the last step in a workflow automatically has `print_response: true` unless explicitly configured otherwise. This ensures that newcomers to Roast see output from their workflows by default.
+
 #### Shared Configuration
 
 Roast supports sharing common configuration and steps across multiple workflows using a `shared.yml` file.

data/examples/grading/README.md

@@ -0,0 +1,71 @@
+# Test Grading Workflow
+
+This workflow acts as a senior software engineer and testing expert to evaluate the quality of test files based on best practices and guidelines.
+
+## Prerequisites
+
+This example uses `shadowenv` for environment management, which is specific to Shopify's development environment. If you're not using shadowenv, you'll need to adapt the commands to your own setup.
+
+### If you're using shadowenv:
+```bash
+brew install shadowenv
+```
+
+### If you're NOT using shadowenv:
+You'll need to modify the `run_coverage.rb` file to remove the shadowenv commands. Look for lines like:
+```ruby
+command = "shadowenv exec -- bundle exec ruby ..."
+```
+
+And change them to match your environment:
+```ruby
+# For standard Ruby/Bundler setup:
+command = "bundle exec ruby ..."
+
+# Or if you're using rbenv/rvm:
+command = "ruby ..."
+```
+
+## Usage
+
+```bash
+# Run the grading workflow on a test file
+roast execute examples/grading/workflow.yml path/to/your_test.rb
+```
+
+## How it Works
+
+1. **read_dependencies**: Analyzes the test file and its dependencies
+2. **run_coverage**: Executes the test with coverage tracking
+3. **generate_grades**: Evaluates test quality across multiple dimensions
+4. **verify_test_helpers**: Checks for proper test helper usage
+5. **verify_mocks_and_stubs**: Ensures appropriate use of test doubles
+6. **analyze_coverage**: Reviews code coverage metrics
+7. **generate_recommendations**: Provides improvement suggestions
+8. **calculate_final_grade**: Computes an overall grade (A-F scale)
+9. **format_result**: Formats the final output
+
+## Customization
+
+Feel free to adapt this workflow to your testing environment:
+
+- **Different test frameworks**: Modify `run_coverage.rb` to work with RSpec, Jest, pytest, etc.
+- **Coverage tools**: Replace the coverage command with your preferred tool (SimpleCov, Istanbul, Coverage.py)
+- **Grading criteria**: Adjust the prompts in each step to match your team's standards
+- **Environment setup**: Remove or replace shadowenv with your environment management tool
+
+## Example Output
+
+```
+========== TEST GRADE REPORT ==========
+Test file: test/example_test.rb
+
+FINAL GRADE:
+  Score: 85/100
+  Letter Grade: B
+
+RECOMMENDATIONS:
+- Add edge case testing for error conditions
+- Improve test descriptions for clarity
+- Consider extracting common setup to helper methods
+```

data/lib/roast/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Roast
-  VERSION = "0.3.0"
+  VERSION = "0.3.1"
 end

data/lib/roast/workflow/configuration.rb

@@ -45,6 +45,8 @@ module Roast
         # Process target and resource
         @target = ConfigurationLoader.extract_target(@config_hash, options)
         process_resource
+
+        mark_last_step_for_output
       end
 
       def context_path
@@ -102,6 +104,60 @@ module Roast
           @target = @resource.value if has_target?
         end
       end
+
+      def mark_last_step_for_output
+        return if @steps.empty?
+
+        last_step = find_last_executable_step(@steps.last)
+        return unless last_step
+
+        # Get the step name/key
+        step_key = extract_step_key(last_step)
+        return unless step_key
+
+        # Ensure config exists for this step
+        @config_hash[step_key] ||= {}
+
+        # Only set print_response if not already explicitly configured
+        @config_hash[step_key]["print_response"] = true unless @config_hash[step_key].key?("print_response")
+      end
+
+      def find_last_executable_step(step)
+        case step
+        when String
+          step
+        when Hash
+          # Check if it's a special step type (if, unless, each, repeat, case)
+          if step.key?("if") || step.key?("unless")
+            # For conditional steps, try to find the last step in the "then" branch
+            then_steps = step["then"] || step["steps"]
+            find_last_executable_step(then_steps.last) if then_steps&.any?
+          elsif step.key?("each") || step.key?("repeat")
+            # For iteration steps, we can't reliably determine the last step
+            nil
+          elsif step.key?("case")
+            # For case steps, we can't reliably determine the last step
+            nil
+          elsif step.size == 1
+            # Regular hash step with variable assignment
+            step
+          end
+        when Array
+          # For parallel steps, we can't determine a single "last" step
+          nil
+        else
+          step
+        end
+      end
+
+      def extract_step_key(step)
+        case step
+        when String
+          step
+        when Hash
+          step.keys.first
+        end
+      end
     end
   end
 end

data/lib/roast/workflow/prompt_step.rb

@@ -9,7 +9,7 @@ module Roast
 
       def call
         prompt(name)
-        result = chat_completion
+        result = chat_completion(print_response:, json:, params:)
 
         # Apply coercion if configured
         apply_coercion(result)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: roast-ai
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.3.1
 platform: ruby
 authors:
 - Shopify
@@ -190,6 +190,7 @@ files:
 - examples/exit_on_error/analyze_lint_output/prompt.md
 - examples/exit_on_error/apply_fixes/prompt.md
 - examples/exit_on_error/workflow.yml
+- examples/grading/README.md
 - examples/grading/analyze_coverage/prompt.md
 - examples/grading/calculate_final_grade.rb
 - examples/grading/format_result.rb