roast-ai 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8ab27704514fc6ddfa09003464d2aa324b852379b0548619e2f0023bebe793ae
-  data.tar.gz: 18c0936f2b5dc2b54ef420be2ac089b8252f64a02227ab33d98db9d2e6249136
+  metadata.gz: db58f659180113d7a2bb1bface8da2898be6717033107247156a464b4d14159d
+  data.tar.gz: f14c2caae89fab3353b66ae3eb4457c56975e65c464037a2d401cfca30840d81
 SHA512:
-  metadata.gz: 50552385cdb280592a1e331c4a3ec5de2924fa05636af5deb742b96f2fde229e3448e61e6516e570c02a8dc0f416a3b9a3c3d884a800eb89987c90c856a0258d
-  data.tar.gz: 8feeb510d87e6e6256c46cbca3e13fbc65701ceb28b049452e5bea2f44815f7770e688f8422487b431a85c5f459beffadcce590e7b860cde8a1ab4ee939ff50e
+  metadata.gz: 88c3f83651c535077bdfe29408a19b874f66f19d8b7e743dd4933922f82fd7aee53ee215968e3d10c0207e92d4c53018a0b190475ba7ac4b70d15cda07a634ff
+  data.tar.gz: 5ebae3dcc5d7d775a3f19ab79ead81d7c11f53fc31b5b20f1db4781a950032e693a2d331fdc4509eb0661c04072f421defcd56d15b9d6ce74f7bfc12357ee1b0

data/CHANGELOG.md

@@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.1]
+
+### Added
+- Smart coercion defaults for boolean expressions based on step type
+  - Ruby expressions (`{{expr}}`) default to regular boolean coercion
+  - Bash commands (`$(cmd)`) default to exit code interpretation
+  - Inline prompts and regular steps default to "smart" LLM-powered interpretation (looks for truthy or falsy language)
+- Direct syntax for step configuration - `coerce_to` and other options are now specified directly on iteration steps
+
 ## [0.2.0] - 2025-05-26
 
 ### Added

data/CLAUDE_NOTES.md

@@ -0,0 +1,68 @@
+# Important Notes for Claude
+
+## Roast Workflow Syntax
+
+### IMPORTANT: No inline configuration for steps!
+
+Roast workflows DO NOT support inline configuration syntax like this:
+```yaml
+# WRONG - This syntax does NOT exist:
+steps:
+  - step_name:
+      prompt: "some prompt"
+      output: variable_name
+```
+
+The correct syntax is:
+```yaml
+# CORRECT - Steps are just names or hash assignments:
+steps:
+  - step_name
+  - variable_name: step_name
+  - variable_name: $(command)
+```
+
+### Control flow structures
+
+Only control flow structures (if/unless, case/when/else, each, repeat) support nested configuration:
+
+```yaml
+steps:
+  # Simple step - just the name
+  - detect_language
+  
+  # Variable assignment
+  - my_var: detect_language
+  
+  # Command execution with assignment
+  - env_type: $(echo $ENVIRONMENT)
+  
+  # Control flow - these DO have nested structure
+  - if: "{{ condition }}"
+    then:
+      - step1
+      - step2
+      
+  - case: "{{ expression }}"
+    when:
+      value1:
+        - step1
+      value2:
+        - step2
+    else:
+      - step3
+```
+
+### Step Configuration
+
+Step configuration (prompts, outputs, etc.) is handled through:
+1. File naming conventions (step_name/prompt.md, step_name/output.txt)
+2. The workflow configuration file itself (tools, target, etc.)
+
+NOT through inline step configuration in the steps array.
+
+## Remember:
+- Steps array contains step names, not step configurations
+- Only control flow structures have nested configuration
+- Variable assignment uses hash syntax: `var_name: step_name`
+- Commands use $() syntax: `var_name: $(command)`

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    roast-ai (0.2.0)
+    roast-ai (0.2.1)
       activesupport (~> 8.0)
       cli-ui
       diff-lcs (~> 1.5)

data/README.md

@@ -4,6 +4,18 @@
 
 A convention-oriented framework for creating structured AI workflows, maintained by the Augmented Engineering team at Shopify.
 
+## Installation
+
+```bash
+$ gem install roast-ai
+```
+
+Or add to your Gemfile:
+
+```ruby
+gem 'roast-ai'
+```
+
 ## Why you should use Roast
 
 Roast provides a structured, declarative approach to building AI workflows with:
@@ -64,6 +76,42 @@ steps:
   - Summarize the changes made to {{File.basename(file)}}.
 ```
 
+## Try it
+
+If you don’t have one already, get an OpenAI key from [here](https://platform.openai.com/settings/organization/api-keys). You will need an account with a credit card, make sure that a basic completion works.
+
+```bash
+export OPENAI_API_KEY=sk-proj-....
+
+curl -H "Content-Type: application/json" \
+    -H "Authorization: Bearer $API_TOKEN" \
+    -d '{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"What is 1+1?"}]}' \
+    https://api.openai.com/v1/chat/completions
+```
+
+The [test grading workflow](examples/grading/workflow.md) in this repository is a senior software engineer and testing expert that evaluates the quality of a test based on guidelines.
+
+Try the workflow.
+
+```bash
+./exe/roast execute examples/grading/workflow.yml test/roast/resources_test.rb
+
+🔥🔥🔥 Everyone loves a good roast 🔥🔥🔥
+...
+```
+
+This will output a test grade.
+
+```
+========== TEST GRADE REPORT ==========
+Test file: test/roast/resources_test.rb
+
+FINAL GRADE:
+  Score: 80/100
+  Letter Grade: B
+```
+Note that you may also need `shadowenv` and `rg`, on MacOS run `brew install shadowenv` and `brew install rg`.
+
 ## How to use Roast
 
 1. Create a workflow YAML file defining your steps and tools
@@ -178,7 +226,37 @@ Roast supports several types of steps:
    - Until conditions: `until: "{{condition}}"`
    - Maximum iterations: `max_iterations: 10`
 
-6. **Raw prompt step**: Simple text prompts for the model without tools
+6. **Case/when/else steps**: Select different execution paths based on a value (similar to Ruby's case statement)
+   ```yaml
+   steps:
+     - detect_language
+     
+     - case: "{{ workflow.output.detect_language }}"
+       when:
+         ruby:
+           - lint_with_rubocop
+           - test_with_rspec
+         javascript:
+           - lint_with_eslint
+           - test_with_jest
+         python:
+           - lint_with_pylint
+           - test_with_pytest
+       else:
+         - analyze_generic
+         - generate_basic_report
+   ```
+   
+   Case expressions can be:
+   - Workflow outputs: `case: "{{ workflow.output.variable }}"`
+   - Ruby expressions: `case: "{{ count > 10 ? 'high' : 'low' }}"`
+   - Bash commands: `case: "$(echo $ENVIRONMENT)"`
+   - Direct values: `case: "production"`
+   
+   The value is compared against each key in the `when` clause, and matching steps are executed.
+   If no match is found, the `else` steps are executed (if provided).
+
+7. **Raw prompt step**: Simple text prompts for the model without tools
    ```yaml
    steps:
      - Summarize the changes made to the codebase.
@@ -668,18 +746,6 @@ your-project/
   └── ...
 ```
 
-## Installation
-
-```bash
-$ gem install roast-ai
-```
-
-Or add to your Gemfile:
-
-```ruby
-gem 'roast-ai'
-```
-
 ## Development
 
 After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake` to run the tests and linter. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/docs/ITERATION_SYNTAX.md

@@ -90,10 +90,38 @@ For defining prompts directly in the workflow:
 
 ## Type Coercion
 
-Values are automatically coerced to the appropriate types:
+### Smart Defaults
 
-- For `until` conditions: Values are coerced to booleans (using Ruby's truthiness rules)
-- For `each` collections: Values are coerced to iterables (converted to arrays of lines if needed)
+Roast applies intelligent defaults for boolean coercion based on the type of expression:
+
+- **Ruby expressions** (`{{expr}}`) → Regular boolean coercion (`!!value`)
+- **Bash commands** (`$(cmd)`) → Exit code interpretation (0 = true, non-zero = false)
+- **Inline prompts/step names** → LLM boolean interpretation (analyzes yes/no intent)
+
+### Manual Coercion
+
+You can override the smart defaults by specifying `coerce_to` directly in the step:
+
+```yaml
+# Override prompt to use regular boolean instead of LLM boolean
+- repeat:
+    until: "check_condition"
+    coerce_to: boolean
+    steps:
+      - process_item
+
+# Force a step result to be treated as iterable
+- each: "get_items"
+  as: "item"
+  coerce_to: iterable
+  steps:
+    - process: "{{item}}"
+```
+
+Available coercion types:
+- `boolean` - Standard Ruby truthiness (`!!` operator)
+- `llm_boolean` - Natural language yes/no interpretation
+- `iterable` - Convert to array (splits strings on newlines)
 
 ## Migrating Existing Workflows
 

data/examples/case_when/README.md

@@ -0,0 +1,58 @@
+# Case/When/Else Example
+
+This example demonstrates the use of `case/when/else` control flow in Roast workflows.
+
+## Overview
+
+The `case/when/else` construct allows you to execute different steps based on the value of an expression, similar to Ruby's case statement or switch statements in other languages.
+
+## Syntax
+
+```yaml
+- case: <expression>
+  when:
+    <value1>:
+      - <steps>
+    <value2>:
+      - <steps>
+  else:
+    - <steps>
+```
+
+## Features Demonstrated
+
+1. **Basic case/when/else**: Detect file language and execute language-specific analysis
+2. **Bash command evaluation**: Use environment variables to determine deployment strategy
+3. **Complex expressions**: Use Ruby expressions to categorize numeric values
+
+## Expression Types
+
+The `case` expression can be:
+- A simple string value
+- An interpolated workflow output: `{{ workflow.output.variable }}`
+- A Ruby expression: `{{ workflow.output.count > 10 ? 'high' : 'low' }}`
+- A bash command: `$(echo $ENVIRONMENT)`
+- A reference to a previous step's output
+
+## How It Works
+
+1. The `case` expression is evaluated to produce a value
+2. The value is compared against each key in the `when` clause
+3. If a match is found, the steps under that key are executed
+4. If no match is found and an `else` clause exists, those steps are executed
+5. If no match is found and no `else` clause exists, execution continues
+
+## Running the Example
+
+```bash
+roast execute examples/case_when/workflow.yml
+```
+
+This will process all Ruby, JavaScript, Python, and Go files in the current directory, detecting their language and running appropriate analysis steps.
+
+## Use Cases
+
+- **Multi-language projects**: Different linting/testing for different file types
+- **Environment-specific workflows**: Different deployment steps for prod/staging/dev
+- **Conditional processing**: Different handling based on file size, complexity, or other metrics
+- **Error handling**: Different recovery strategies based on error types

data/examples/case_when/detect_language/prompt.md

@@ -0,0 +1,16 @@
+# Detect Programming Language
+
+Based on the file extension and content, determine the primary programming language of this file:
+- If it's a `.rb` file, return "ruby"
+- If it's a `.js` file, return "javascript"
+- If it's a `.py` file, return "python"
+- If it's a `.go` file, return "go"
+- Otherwise, return "unknown"
+
+Return ONLY the language name in lowercase, nothing else.
+
+File: {{ context.resource_uri }}
+Content:
+```
+{{ context.resource }}
+```

data/examples/case_when/workflow.yml

@@ -0,0 +1,58 @@
+name: "Case/When/Else Example"
+
+tools:
+  - Roast::Tools::Cmd
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+
+target: "**/*.{rb,js,py,go}"
+
+steps:
+  - detect_language
+
+  - case: "{{ workflow.output.detect_language }}"
+    when:
+      ruby:
+        - analyze_ruby
+        - generate_ruby_report
+      javascript:
+        - analyze_javascript
+        - generate_js_report
+      python:
+        - analyze_python
+        - generate_python_report
+      go:
+        - analyze_go
+        - generate_go_report
+    else:
+      - analyze_generic
+      - generate_generic_report
+
+  # Another example using bash command for case expression
+  - get_environment: $(echo $ENVIRONMENT || echo "development")
+
+  - case: "{{ workflow.output.get_environment }}"
+    when:
+      production:
+        - production_checks
+        - deploy_production
+      staging:
+        - staging_checks
+        - deploy_staging
+      development:
+        - run_tests
+        - local_deploy
+    else:
+      - unknown_environment
+
+  # Example with numeric case values
+  - count_issues
+
+  - case: "{{ workflow.output.count_issues.to_i > 10 ? 'high' : workflow.output.count_issues.to_i > 5 ? 'medium' : 'low' }}"
+    when:
+      high:
+        - high_priority_alert
+      medium:
+        - medium_priority_notice
+      low:
+        - low_priority_info

data/examples/direct_coerce_syntax/README.md

@@ -0,0 +1,32 @@
+# Direct Coerce Syntax
+
+This example demonstrates the simplified syntax for specifying `coerce_to` and other configuration options directly on iteration steps.
+
+## Direct Syntax
+
+Configuration options are specified directly on the step:
+
+```yaml
+- repeat:
+    until: "condition"
+    coerce_to: boolean
+    print_response: true
+    model: "claude-3-haiku"
+    steps: [...]
+```
+
+## Benefits
+
+1. **Cleaner YAML** - No unnecessary nesting
+2. **More intuitive** - Configuration options are at the same level as other step properties
+3. **Consistent** - Matches how other step properties are specified
+
+## Supported Options
+
+All step configuration options can be specified directly:
+- `coerce_to` - Type coercion (boolean, llm_boolean, iterable)
+- `print_response` - Whether to print LLM responses
+- `loop` - Auto-loop behavior
+- `json` - JSON response mode
+- `params` - Additional parameters
+- `model` - Model override

data/examples/direct_coerce_syntax/workflow.yml

@@ -0,0 +1,36 @@
+name: Direct Coerce Syntax Demo
+description: Demonstrates the simplified coerce_to syntax without config blocks
+
+steps:
+  # Example 1: Direct coerce_to on repeat
+  - repeat:
+      until: "check_api_ready"
+      coerce_to: boolean  # Direct syntax - no config block needed
+      max_iterations: 5
+      steps:
+        - check_api_ready:
+            prompt: "Check if the API endpoint returns a 200 status"
+        - wait: 2
+  
+  # Example 2: Direct coerce_to on each  
+  - get_data_sources:
+      prompt: "List available data sources, one per line"
+  
+  - each: "get_data_sources"
+    as: "source"
+    coerce_to: iterable  # Direct syntax
+    steps:
+      - validate_source: "Validating {{source}}..."
+      - process_source:
+          prompt: "Process data from {{source}}"
+  
+  # Example 3: Multiple configuration options
+  - repeat:
+      until: "all_tests_pass"
+      coerce_to: llm_boolean  # Override default
+      print_response: true     # Other options work too
+      max_iterations: 10
+      steps:
+        - run_tests: "$(rake test)"
+        - all_tests_pass:
+            prompt: "Did all tests pass successfully?"

data/examples/grading/workflow.yml

@@ -1,5 +1,7 @@
 name: Test Grading
-model: anthropic:claude-opus-4
+api_token: $(echo $OPENAI_API_KEY)
+# model: anthropic:claude-opus-4
+model: gpt-4.1-mini
 
 tools:
   - Roast::Tools::Grep
@@ -23,16 +25,16 @@ steps:
 
 # set non-default attributes for steps below
 analyze_coverage:
-  model: gpt-4.1-mini
+#  model: gpt-4.1-mini
   auto_loop: false
   json: true
   
 generate_grades:
-  model: o3
+#  model: o3
   json: true
 
 generate_recommendations:
-  model: o3
+#  model: o3
   auto_loop: false
   json: true
   params:

data/examples/json_handling/README.md

@@ -0,0 +1,32 @@
+# JSON Handling Example
+
+This example demonstrates how Roast handles JSON responses from LLM steps.
+
+## Key Features
+
+1. **JSON Arrays**: When a step has `json: true` and returns an array, the result is `flatten.first` - the first element after flattening the array. This is useful when the LLM returns an array with a single object.
+
+2. **JSON Objects**: Hash/object responses are maintained as proper Ruby hashes in the workflow output.
+
+3. **Replay Support**: JSON data structures are properly serialized and deserialized when saving/loading workflow state for replay functionality.
+
+## Running the Example
+
+```bash
+bin/roast examples/json_handling/workflow.yml
+```
+
+## How It Works
+
+1. The `fetch_users` step generates a JSON array of user objects
+2. The `fetch_metadata` step generates a JSON object with metadata
+3. The `process_data` step can access the structured data using `{{output.fetch_users}}` and `{{output.fetch_metadata}}`
+4. The structured data is preserved through the workflow, including during replay scenarios
+
+## Implementation Details
+
+When `json: true` is set on a step:
+- Array responses return `flatten.first` - the first element after flattening
+- Object responses are preserved as hashes
+- Non-JSON array responses (when `json: false`) are joined with newlines
+- The data maintains its structure when saved to and loaded from workflow state files

data/examples/json_handling/workflow.yml

@@ -0,0 +1,52 @@
+name: JSON Data Handling Example
+api_provider: openai
+model: gpt-4
+
+tools:
+  - type: write_file
+    allowed_paths:
+      - examples/json_handling/
+
+steps:
+  - name: fetch_users
+    prompt: |
+      Generate a JSON array of 3 user objects. Each user should have:
+      - id (number)
+      - name (string)
+      - email (string)
+      - active (boolean)
+    json: true
+
+  - name: fetch_metadata
+    prompt: |
+      Generate a JSON object with metadata about a dataset:
+      - total_records (number)
+      - last_updated (ISO date string)
+      - categories (array of strings)
+      - filters (object with status and sort fields)
+    json: true
+
+  - name: process_data
+    prompt: |
+      Based on the users data: {{output.fetch_users}}
+      And metadata: {{output.fetch_metadata}}
+      
+      Create a summary report describing:
+      1. How many active users there are
+      2. The categories available
+      3. When the data was last updated
+
+  - name: save_report
+    tool: write_file
+    path: examples/json_handling/report.txt
+    content: |
+      # JSON Data Processing Report
+      
+      ## Users Data
+      {{output.fetch_users}}
+      
+      ## Metadata
+      {{output.fetch_metadata}}
+      
+      ## Summary
+      {{output.process_data}}

data/examples/smart_coercion_defaults/README.md

@@ -0,0 +1,65 @@
+# Smart Coercion Defaults
+
+This example demonstrates how Roast applies intelligent defaults for boolean coercion based on the type of expression being evaluated.
+
+## Default Coercion Rules
+
+When a step is used in a boolean context (like `if`, `unless`, or `until` conditions) and no explicit `coerce_to` is specified, Roast applies these smart defaults:
+
+1. **Ruby Expressions** (`{{expression}}`) → Regular boolean coercion (`!!value`)
+   - `nil` and `false` are falsy
+   - Everything else is truthy (including 0, empty arrays, etc.)
+
+2. **Bash Commands** (`$(command)`) → Exit code interpretation
+   - Exit code 0 = true (success)
+   - Non-zero exit code = false (failure)
+
+3. **Prompt/Step Names** → LLM boolean interpretation
+   - Analyzes natural language responses for yes/no intent
+   - "Yes", "True", "Affirmative" → true
+   - "No", "False", "Negative" → false
+
+4. **Non-string Values** → Regular boolean coercion
+
+## Examples
+
+### Ruby Expression (Regular Boolean)
+```yaml
+- repeat:
+    until: "{{counter >= 5}}"  # Uses !! coercion
+    steps:
+      - increment: counter
+```
+
+### Bash Command (Exit Code)
+```yaml
+- repeat:
+    until: "$(test -f /tmp/done)"  # True when file exists (exit 0)
+    steps:
+      - wait: 1
+```
+
+### Prompt Response (LLM Boolean)
+```yaml
+- if: "Should we continue?"  # Interprets "Yes, let's continue" as true
+  then:
+    - proceed: "Continuing..."
+```
+
+## Overriding Defaults
+
+You can always override the default coercion by specifying `coerce_to` directly in the step:
+
+```yaml
+- each: "get_items"
+  as: "item"
+  coerce_to: iterable  # Override default to split into array
+  steps:
+    - process: "{{item}}"
+```
+
+## Supported Coercion Types
+
+- `boolean` - Standard Ruby truthiness (!! operator)
+- `llm_boolean` - Natural language yes/no interpretation
+- `iterable` - Convert to array (splits strings on newlines)