roast-ai 0.4.10 → 0.5.0

data/dsl/tutorial/03_targets_and_params/README.md

@@ -0,0 +1,230 @@
+# Chapter 3: Targets and Parameters
+
+In the previous chapters, you learned how to create workflows and chain cogs together. Now you'll learn how to make your
+workflows more flexible by accepting targets and custom parameters from the command line.
+
+## What You'll Learn
+
+- How to pass targets (files, URLs, etc.) to workflows
+- How to use `target!` for single-target workflows
+- How to access multiple targets with `targets`
+- How to pass custom arguments with `args`
+- How to pass key-value parameters with `kwargs`
+- How to check for the presence of arguments and parameters
+
+## Workflow Targets
+
+Targets are inputs that you want your workflow to process—files, URLs, or any other data. They're specified on the
+command line after your workflow file path:
+
+```bash
+bin/roast execute --executor=dsl workflow.rb https://example.com
+bin/roast execute --executor=dsl workflow.rb README.md
+bin/roast execute --executor=dsl workflow.rb src/*.rb
+bin/roast execute --executor=dsl workflow.rb Gemfile Gemfile.lock
+```
+
+Shell globs are expanded automatically, so `src/*.rb` will pass all Ruby files in the `src/` directory.
+
+### Accessing a Single Target
+
+Use `target!` when your workflow expects exactly one target:
+
+```ruby
+execute do
+  cmd(:fetch) do
+    "curl -sL #{target!}"
+  end
+
+  ruby do
+    puts "Target: #{target!}"
+    puts "Content length: #{cmd!(:fetch).out.length} bytes"
+  end
+end
+```
+
+Run it with:
+```bash
+bin/roast execute --executor=dsl workflow.rb https://example.com
+```
+
+**Important:** `target!` raises an error if zero or multiple targets are provided. Use it when your workflow is
+designed to process exactly one target.
+
+### Accessing Multiple Targets
+
+Use `targets` (plural) when your workflow can handle any number of files:
+
+```ruby
+execute do
+  ruby do
+    if targets.empty?
+      puts "No files provided"
+    else
+      puts "Processing #{targets.length} files:"
+      targets.each { |file| puts "  - #{file}" }
+    end
+  end
+end
+```
+
+The `targets` method always returns an array, which will be empty if the workflow is invoked with no targets specified.
+
+## Custom Arguments
+
+Custom arguments let you pass additional data to your workflows. They come after `--` on the command line:
+
+```bash
+bin/roast execute --executor=dsl workflow.rb -- hello world
+```
+
+### Simple Arguments (args)
+
+Simple word tokens become arguments, accessible as symbols:
+
+```ruby
+execute do
+  ruby do
+    puts "Arguments: #{args.inspect}"  # [:hello, :world]
+
+    # Check if a specific argument is present
+    if arg?(:save_data)
+      puts "Will save incremental data files"
+    end
+  end
+end
+```
+
+Run it with:
+```bash
+bin/roast execute --executor=dsl workflow.rb -- hello world
+```
+Or:
+```bash
+bin/roast execute --executor=dsl workflow.rb -- save_data something_else
+```
+
+### Key-Value Arguments (kwargs)
+
+Use `key=value` format for key-value parameters:
+
+```ruby
+execute do
+  ruby do
+    # Check if a kwarg is present
+    if kwarg?(:format)
+      puts "Format: #{kwarg(:format)}"
+    end
+
+    # Access all kwargs
+    puts "All kwargs: #{kwargs.inspect}"
+
+    # Access a specific kwarg (returns nil if not present)
+    name = kwarg(:name)
+    puts "Name: #{name}"
+
+    # Access with error if missing
+    email = kwarg!(:email)  # raises error if 'email' not provided
+    puts "Email: #{email}"
+  end
+end
+```
+
+Run it with:
+```bash
+bin/roast execute --executor=dsl workflow.rb -- name=Alice format=json
+```
+Or:
+```bash
+bin/roast execute --executor=dsl workflow.rb -- name=Alice email=alice@example.net
+```
+
+**Notes:**
+- Kwarg keys are parsed as symbols
+- Kwarg values are always strings
+- Simple words (without `=`) become args, not kwargs
+
+## Combining Targets and Arguments
+
+You can use targets and custom arguments together:
+
+```bash
+bin/roast execute --executor=dsl workflow.rb file1.rb file2.rb -- save_data format=summary
+```
+
+In your workflow:
+
+```ruby
+execute do
+  ruby do
+    puts "Processing #{targets.length} files"
+    puts "Save data: #{arg?(:save_data)}"
+    puts "Format: #{kwarg(:format) || "default"}"
+  end
+end
+```
+
+## Accessing Parameters from Any Cog
+
+All parameter accessors are available in any cog's input block:
+
+```ruby
+execute do
+  chat(:analyze) do
+    "Analyze this file: #{target!}"
+  end
+
+  agent(:process) do
+    files = targets.join("\n")
+    format = kwarg(:format) || "detailed"
+    "Process these files and provide a #{format} report:\n#{files}) "
+  end
+
+  cmd(:grep_pattern) do
+    pattern = kwarg!(:pattern)  # Error if not provided
+    "grep '#{pattern}' #{targets.join(" ")}"
+  end
+end
+```
+
+## Running the Workflows
+
+To run the examples in this chapter:
+
+```bash
+# Single target workflow (with a URL)
+bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/single_target.rb https://example.com
+```
+
+```bash
+# Multiple targets with arguments
+bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/multiple_targets.rb \
+  Gemfile Gemfile.lock -- save_data format=summary
+```
+
+## Try It Yourself
+
+1. **Single target processing** - Run the single_target workflow with different URLs
+2. **Multiple files** - Use shell globs to pass multiple files at once
+3. **Add arguments** - Try different combinations of args and kwargs
+4. **Error handling** - See what happens when you use `target!` with multiple files
+5. **Mix and match** - Combine targets with different argument combinations
+
+## Key Takeaways
+
+- Use `target!` when your workflow expects exactly one file/url/etc. (errors otherwise)
+- Use `targets` to access an array of all targets (can be empty)
+- Custom arguments come after `--` on the command line
+- Simple words become `args` (as symbols)
+- `key=value` pairs become `kwargs` (keys as symbols, values as strings)
+- Use `arg?(:name)` to check if an argument is present
+- Use `kwarg(:name)` to get a kwarg value (returns nil if missing)
+- Use `kwarg!(:name)` to require a kwarg (errors if missing)
+- All accessors work in any cog's input block
+
+## What's Next?
+
+In the next chapter, you'll learn about configuration options: how to fine-tune cog behavior, control what gets
+displayed, and set up different models for different tasks.
+
+But first, experiment with targets and parameters to see how they make your workflows more flexible and reusable!

data/dsl/tutorial/03_targets_and_params/multiple_targets.rb

@@ -0,0 +1,65 @@
+# typed: true
+# frozen_string_literal: true
+
+#: self as Roast::DSL::Workflow
+
+# This workflow demonstrates processing multiple targets with custom arguments.
+# It accepts any number of files and optional arguments for controlling output.
+#
+# Run it with:
+#   bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/multiple_targets.rb \
+#     Gemfile Gemfile.lock
+#
+# Or try:
+#   bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/multiple_targets.rb \
+#     dsl/**/*.rb -- count format=detailed
+#
+# Or get a little crazy:
+#   bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/multiple_targets.rb \
+#     dsl/**/*.md -- format=json
+
+config do
+  cmd { display! }
+end
+
+execute do
+  # Show what we're processing
+  ruby do
+    puts "Processing #{targets.length} file(s):\n  - #{targets.join("\n  - ")}\n"
+    puts "Counting file lines" if arg?(:count)
+    puts "Arguments: #{args.inspect}" if args.any?
+    puts "Keyword arguments: #{kwargs.inspect}" if kwargs.any?
+  end
+
+  # Analyze each file
+  chat(:analyze_files) do
+    format = kwarg(:format) || "summary"
+    <<~PROMPT
+      Please analyze these files and provide an #{format} overview:
+
+      #{targets.map { |f| "- #{f}" }.join("\n")}
+
+      Based on the filenames, what can you infer about this project?
+      What do you think these files are used for?
+
+      #{format == "detailed" ? "Provide detailed insights for each file." : "Keep it brief (2-3 sentences)."}
+    PROMPT
+  end
+
+  # Count total lines across all files
+  cmd(:total_lines) do |my|
+    my.stdin = targets.join("\n")
+    "xargs wc -l | awk '/total$/{print $1}'"
+  end
+
+  # Display results
+  ruby do
+    puts "\n" + "=" * 60
+    puts "ANALYSIS RESULTS"
+    puts "=" * 60
+    puts "Total lines: #{arg?(:count) ? cmd!(:total_lines).out : "---"}"
+    puts "Format: #{kwarg(:format)}" if kwarg?(:format)
+    puts chat!(:analyze_files).text
+    puts "=" * 60
+  end
+end

data/dsl/tutorial/03_targets_and_params/single_target.rb

@@ -0,0 +1,65 @@
+# typed: true
+# frozen_string_literal: true
+
+#: self as Roast::DSL::Workflow
+
+# This workflow demonstrates using target! to process a single URL.
+# It expects exactly one URL to be provided on the command line.
+#
+# Run it with:
+#   bin/roast execute --executor=dsl dsl/tutorial/03_targets_and_params/single_target.rb \
+#     https://example.com
+#
+# Try running it with no URLs or multiple URLs to see how target! validates input.
+
+config do
+  chat { no_show_response! }
+end
+
+execute do
+  # Fetch the target URL
+  cmd(:fetch_url) do
+    "curl -sL #{target!}"
+  end
+
+  # Count words and lines in the content
+  cmd(:count_words) do
+    content = cmd!(:fetch_url).out
+    "echo #{content.shellescape} | wc -w"
+  end
+
+  cmd(:count_lines) do |my|
+    content = cmd!(:fetch_url).out
+    my.stdin = content
+    "wc -l"
+  end
+
+  # Analyze the content with an LLM
+  chat(:analyze) do
+    content = cmd!(:fetch_url).out
+    <<~PROMPT
+      Please analyze this web page content and provide a brief summary (2-3 sentences):
+
+      URL: #{target!}
+
+      Content:
+      #{content}
+
+      Focus on what the page contains and its purpose.
+    PROMPT
+  end
+
+  # Display the results
+  ruby do
+    puts "\n" + "=" * 60
+    puts "WEB PAGE ANALYSIS"
+    puts "=" * 60
+    puts "URL: #{target!}"
+    puts "Words: #{cmd!(:count_words).out.strip}"
+    puts "Lines: #{cmd!(:count_lines).out.strip}"
+    puts
+    puts "Summary:"
+    puts chat!(:analyze).response
+    puts "=" * 60
+  end
+end

data/dsl/tutorial/04_configuration_options/README.md

@@ -0,0 +1,209 @@
+# Chapter 4: Configuration Options
+
+In previous chapters, you learned the basics of creating workflows and chaining cogs together. Now you'll learn how to
+configure cogs to control their behavior, use different models for different steps, and manage what gets displayed
+during execution.
+
+## What You'll Learn
+
+- How to configure models and providers
+- The difference between global and per-step configuration
+- How to use different models for different steps
+- How to control what gets displayed during execution
+- Common model parameters like temperature
+
+## Global Configuration
+
+The `config` block at the top of your workflow sets defaults for all cogs of a given type:
+
+```ruby
+config do
+  chat do
+    model "gpt-4o-mini"
+    provider :openai
+  end
+end
+
+execute do
+  # All chat cogs will use gpt-4o-mini unless overridden
+  chat(:first) { "Analyze this data..." }
+  chat(:second) { "Summarize that analysis..." }
+end
+```
+
+Both chat cogs will use `gpt-4o-mini` from OpenAI because that's the global default you specified in the `config` block.
+
+### Configuring Multiple Cog Types
+
+You can set defaults for different cog types in the same config block:
+
+```ruby
+config do
+  chat do
+    model "gpt-4o-mini"
+    provider :openai
+  end
+
+  agent do
+    model "claude-3-5-haiku-20241022"
+    provider :claude
+  end
+end
+```
+
+Now all `chat` cogs use "gpt-4o-mini" and all `agent` cogs use Claude Code with the "haiku" model (unless you override
+them individually).
+
+## Per-Step Configuration
+
+Override global configuration for specific named cogs by configuring them explicitly in the `config` block:
+
+```ruby
+config do
+  # Global default for all chat cogs
+  chat do
+    model "gpt-4o-mini"
+    provider :openai
+  end
+
+  # Override for a specific named cog
+  chat(:complex_task) do
+    model "gpt-5"
+    # any options you don't set here (we're not specifying 'provider' again, for instance) will be inherited from
+    # the global config, or will use Roast's default values.
+  end
+end
+
+execute do
+  # Uses global config (gpt-4o-mini)
+  chat(:simple_task) { "Categorize this: #{data}" }
+
+  # Uses the specific override (gpt-5)
+  chat(:complex_task) { "Perform deep analysis of this data: #{data}" }
+end
+```
+
+The `:simple_task` cog uses the configured global default, but `:complex_task` has its own configuration that overrides.
+
+### Pattern-Based Configuration
+
+You can also configure multiple cogs at once using a regex pattern:
+
+```ruby
+config do
+  chat do
+    model "gpt-4o-mini"
+    provider :openai
+  end
+
+  # All cogs with names matching this pattern use gpt-4o
+  chat(/analyze_/) do
+    model "gpt-4o"
+  end
+end
+
+execute do
+  chat(:extract_data) { "..." } # Uses gpt-4o-mini
+  chat(:analyze_deep) { "..." } # Uses gpt-4o (matches pattern)
+  chat(:analyze_trends) { "..." } # Uses gpt-4o (matches pattern)
+end
+```
+
+This is useful when you have multiple similar cogs that need the same configuration.
+
+## Display Options
+
+Control what gets printed during workflow execution using display methods in your `config` block:
+
+Different cogs have different display options. For the `chat` and `agent` cogs, you can control the display of
+the prompt, response, usage statistics, and (for the agent cog) incremental progress messages.
+
+- `show_prompt!` / `no_show_prompt!` - Control prompt display
+- `show_response!` / `no_show_response!` - Control response display
+- `show_stats!` / `no_show_stats!` - Control statistics display
+
+For the `cmd` cog, you can control the display of standard output and standard error.
+
+- `show_stdout!` / `no_show_stdout!` - Control stdout display
+- `show_stderr!` / `no_show_stderr!` - Control stdout display
+
+And for all cog types, you can quickly apply some typical configurations
+
+- `display!` - Show everything (prompt, response, stats / standard output and error / etc.)
+- `no_display!` / `quiet!` - Hide everything
+
+```ruby
+config do
+  chat do
+    no_show_response! # Hide all responses by default
+  end
+
+  chat(:summarize) do
+    show_response! # But show the final summary
+  end
+end
+```
+
+This is useful for hiding intermediate steps while showing only the final output. See `control_display_and_temperature.rb` for more examples.
+
+## Model Parameters
+
+Fine-tune model behavior with additional parameters:
+
+### Temperature
+
+Controls randomness (0.0-1.0, where 0.0 = deterministic, 1.0 = more random):
+
+```ruby
+config do
+  chat(:creative_writing) do
+    temperature(0.9)
+  end
+
+  chat(:data_extraction) do
+    temperature(0.0)
+  end
+end
+
+execute do
+  chat(:creative_writing) { "Write a creative story about: #{topic}" }
+  chat(:data_extraction) { "Extract structured data from: #{text}" }
+end
+```
+
+## Running the Workflows
+
+To run the examples in this chapter:
+
+```bash
+# Simple configuration example
+bin/roast execute --executor=dsl dsl/tutorial/04_configuration_options/simple_config.rb
+```
+
+```bash
+# Configuring multiple parameters for multiple steps
+bin/roast execute --executor=dsl dsl/tutorial/04_configuration_options/control_display_and_temperature.rb
+```
+
+## Try It Yourself
+
+1. **Experiment with models** - Try different models for the same prompt and compare results
+2. **Adjust temperature** - See how temperature affects creative vs factual outputs
+3. **Control display** - Hide intermediate steps and only show final output
+4. **Mix providers** - Use OpenAI for some steps and Anthropic for others
+5. **Use patterns** - Try pattern-based configuration to configure multiple cogs at once
+
+## Key Takeaways
+
+- Use `config` blocks to set global defaults for cog types
+- Override global config by naming specific cogs in the `config` block: `chat(:name) do ... end`
+- Use pattern-based config with regex: `chat(/pattern/) do ... end`
+- Use display methods like `show_prompt!` and `no_show_response!` to control what gets printed
+- Different cog types can use different default configurations
+
+## What's Next?
+
+In the next chapter, you'll learn about control flow: how to conditionally skip steps, handle failures, and create
+dynamic workflows that adapt based on intermediate results.
+
+But first, experiment with different configurations to understand how they affect workflow behavior!

data/dsl/tutorial/04_configuration_options/control_display_and_temperature.rb

@@ -0,0 +1,104 @@
+# typed: true
+# frozen_string_literal: true
+
+#: self as Roast::DSL::Workflow
+
+# This workflow demonstrates controlling what gets displayed during execution.
+# Shows how to use show_prompt!, no_show_prompt!, show_response!, no_show_response!, and temperature.
+
+config do
+  chat do
+    model "gpt-4o-mini"
+    show_prompt! # Show all prompts by default
+    no_show_response! # But hide responses by default
+  end
+
+  # Configure specific cogs with custom settings
+  chat(:generate_ideas) do
+    temperature(0.9) # Higher temperature for creative brainstorming
+  end
+
+  chat(:extract_names) do
+    temperature(0.0) # Low temperature for reliable extraction
+  end
+
+  chat(:evaluate_names) do
+    no_show_prompt! # Hide prompt for this step
+  end
+
+  chat(:pick_best) do
+    show_prompt! # Explicit show (redundant but clear)
+    show_response! # Show the final decision
+    temperature(0.3) # Low-medium for consistent evaluation
+  end
+end
+
+execute do
+  # Step 1: Generate creative content (high temperature configured above)
+  # The prompt is shown, response is hidden (our configured global default)
+  chat(:generate_ideas) do
+    <<~PROMPT
+      Brainstorm 5 creative names for a new project management tool focused on AI workflows.
+
+      Make them memorable, pronounceable, and available as .com domains (don't verify, just guess).
+    PROMPT
+  end
+
+  # Step 2: Extract structured data (low temperature configured above)
+  # The prompt is shown, response is hidden (our configured global default)
+  chat(:extract_names) do
+    ideas = chat!(:generate_ideas).response
+    <<~PROMPT
+      Extract just the names from this brainstorming output, format as JSON: `{ names: [...] }`
+
+      #{ideas}
+    PROMPT
+  end
+
+  # Step 3: Evaluate each name
+  # Prompt hidden (configured above), response hidden (global default)
+  chat(:evaluate_names) do
+    names = chat!(:extract_names).json![:names]
+    <<~PROMPT
+      Evaluate these product names for:
+      1. Memorability (1-10)
+      2. Professionalism (1-10)
+      3. Relevance to AI workflows (1-10)
+
+      Names:
+      - #{names.join("\n- ")}
+    PROMPT
+  end
+
+  # Step 4: Pick the best name
+  # Both prompt and response shown (configured above)
+  chat(:pick_best) do
+    <<~PROMPT
+      Based on these evaluations, pick the single best name and explain why in 2-3 sentences:
+
+      #{chat!(:evaluate_names).text}
+    PROMPT
+  end
+
+  # Summary of what was displayed:
+  ruby(:display_summary) do
+    puts "\n" + "=" * 70
+    puts "DISPLAY CONTROL DEMONSTRATION"
+    puts "=" * 70
+    puts <<~SUMMARY
+
+      What you saw:
+      - ✓ Prompts for steps 1-2 (global show_prompt!)
+      - ✗ Responses for steps 1-3 (global no_show_response!)
+      - ✗ Prompt for step 3 (configured with no_show_prompt!)
+      - ✓ Prompt and response for step 4 (configured with show_response!)
+      - ✓ LLM stats for every step (unconfigured; Roast default)
+
+      Temperature settings (all configured in config block):
+      - Step 1: 0.9 (higher, for creative brainstorming)
+      - Step 2: 0.0 (low, for reliable extraction)
+      - Step 4: 0.3 (low-medium, for consistent evaluation)
+    SUMMARY
+    puts "=" * 70 + "\n"
+  end
+end