roast-ai 0.2.3 → 0.3.0

data/examples/mcp/README.md

@@ -0,0 +1,223 @@
+# MCP (Model Context Protocol) Tools
+
+This example demonstrates how to use MCP tools in Roast workflows. MCP is an open standard that enables seamless integration between AI applications and external data sources and tools.
+
+## What are MCP Tools?
+
+MCP tools allow your Roast workflows to connect to external services and tools through a standardized protocol. This enables:
+
+- Access to external APIs and services
+- Integration with databases and file systems
+- Connection to specialized tools and platforms
+- Real-time data access during workflow execution
+
+## Configuration
+
+MCP tools are configured in the `tools` section of your workflow YAML file. Roast supports two types of MCP connections:
+
+### 1. SSE (Server-Sent Events) MCP Tools
+
+Connect to HTTP endpoints that implement the MCP protocol:
+
+```yaml
+tools:
+  - Tool Name:
+      url: https://example.com/mcp-endpoint
+      env:
+        - "Authorization: Bearer {{resource.api_token}}"
+      only:
+        - allowed_function_1
+        - allowed_function_2
+```
+
+### 2. Stdio MCP Tools
+
+Connect to local processes that implement the MCP protocol:
+
+```yaml
+tools:
+  - Tool Name:
+      command: docker
+      args:
+        - run
+        - -i
+        - --rm
+        - ghcr.io/example/mcp-server
+      env:
+        API_KEY: "{{env.API_KEY}}"
+      except:
+        - dangerous_function
+```
+
+## Parameters
+
+- **`url`** (SSE only): The HTTP(S) endpoint URL for the MCP server
+- **`command`** (stdio only): The command to execute
+- **`args`** (stdio only): Array of command-line arguments
+- **`env`**: Headers (SSE) or environment variables (stdio)
+- **`only`**: Whitelist of functions to include
+- **`except`**: Blacklist of functions to exclude
+
+## Example Workflow
+
+The `workflow.yml` in this directory shows a simple example using GitMCP to read documentation:
+
+```yaml
+name: MCP Tools Example
+model: gpt-4o-mini
+tools:
+  - Roast Docs:
+      url: https://gitmcp.io/Shopify/roast/docs
+
+steps:
+  - get_doc: Read the Roast docs, and tell me how to use MCP tools.
+  - summarize
+
+summarize:
+  print_response: true
+```
+
+## Running the Examples
+
+### Simple Example (no authentication required)
+
+```bash
+# Run the basic MCP example
+roast execute examples/mcp/workflow.yml
+
+# Run the multi-tool example
+roast execute examples/mcp/multi_mcp_workflow.yml
+```
+
+### GitHub Example (requires GitHub token)
+
+```bash
+# Set your GitHub token first
+export GITHUB_TOKEN="your-github-personal-access-token"
+
+# Run the GitHub workflow
+roast execute examples/mcp/github_workflow.yml
+```
+
+### Filesystem Example
+
+```bash
+# This uses the filesystem MCP server to safely browse /tmp
+roast execute examples/mcp/filesystem_demo/workflow.yml
+```
+
+## More Examples
+
+### GitHub Integration
+
+```yaml
+tools:
+  - GitHub:
+      command: npx
+      args: ["-y", "@modelcontextprotocol/server-github"]
+      env:
+        GITHUB_PERSONAL_ACCESS_TOKEN: "{{env.GITHUB_TOKEN}}"
+      only:
+        - search_repositories
+        - get_issue
+        - create_issue
+```
+
+### Database Access
+
+```yaml
+tools:
+  - Database:
+      command: npx
+      args: ["-y", "@modelcontextprotocol/server-postgres"]
+      env:
+        DATABASE_URL: "{{env.DATABASE_URL}}"
+      only:
+        - query
+        - list_tables
+```
+
+### Using Multiple MCP Tools
+
+You can combine MCP tools with traditional Roast tools:
+
+```yaml
+tools:
+  # Traditional Roast tools
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  
+  # MCP tools
+  - External API:
+      url: https://api.example.com/mcp
+  - Local Tool:
+      command: ./my-mcp-server
+```
+
+## Installing and Using MCP Servers
+
+MCP servers can be run in different ways:
+
+### Using npx (recommended for testing)
+
+Most official MCP servers can be run directly with npx without installation:
+
+```bash
+# These are automatically downloaded and run when used in workflows
+npx -y @modelcontextprotocol/server-github
+npx -y @modelcontextprotocol/server-filesystem /path/to/allow
+npx -y @modelcontextprotocol/server-sqlite /path/to/database.db
+```
+
+### Installing globally
+
+For production use, you might want to install servers globally:
+
+```bash
+npm install -g @modelcontextprotocol/server-github
+npm install -g @modelcontextprotocol/server-filesystem
+```
+
+### Available MCP Servers
+
+Popular MCP servers you can use:
+
+- **@modelcontextprotocol/server-filesystem**: Safe filesystem access
+- **@modelcontextprotocol/server-github**: GitHub API integration (requires token)
+- **@modelcontextprotocol/server-gitlab**: GitLab API integration
+- **@modelcontextprotocol/server-google-drive**: Google Drive access
+- **@modelcontextprotocol/server-slack**: Slack integration
+- **@modelcontextprotocol/server-postgres**: PostgreSQL database access
+- **@modelcontextprotocol/server-sqlite**: SQLite database access
+
+For more MCP servers, visit: https://github.com/modelcontextprotocol/servers
+
+## Troubleshooting
+
+### Common Issues
+
+1. **"No such file or directory" error**
+   - Make sure `npx` is installed: `which npx`
+   - For local commands, ensure they're in your PATH
+
+2. **"Bad credentials" or authentication errors**
+   - Check that your environment variables are set correctly
+   - Use `{{ENV['VAR_NAME']}}` syntax for interpolation
+   - Test with: `echo $GITHUB_TOKEN` to verify it's set
+
+3. **MCP server doesn't start**
+   - Try running the command manually first: `npx -y @modelcontextprotocol/server-filesystem /tmp`
+   - Check for any npm/node errors
+
+4. **"Step not found" errors**
+   - Inline prompts in the workflow use the syntax: `step_name: prompt text`
+   - For multi-line prompts use: `step_name: |` followed by indented text
+   - For separate prompt files: create `step_name/prompt.md` in the workflow directory
+
+## Security Notes
+
+- Never hardcode credentials - use environment variables
+- Use `only` to limit function access when possible
+- Be cautious with stdio tools as they execute local processes
+- Review MCP server documentation for security best practices
+- The filesystem server only allows access to specified directories

data/examples/mcp/analyze_changes/prompt.md

@@ -0,0 +1,8 @@
+For the file {{resource.target}}:
+1. Read the file contents using read_file
+2. Use the Linter tool to analyze code quality
+3. Check for common issues:
+   - Code style violations
+   - Potential bugs
+   - Performance concerns
+   - Security issues

data/examples/mcp/analyze_issues/prompt.md

@@ -0,0 +1,4 @@
+Based on the issues found:
+1. Identify common patterns or themes
+2. Categorize the bugs by type (UI, performance, functionality, etc.)
+3. Note any critical issues that need immediate attention

data/examples/mcp/analyze_schema/prompt.md

@@ -0,0 +1,4 @@
+Using the database tool:
+1. List all tables in the database
+2. For each major table, describe its structure
+3. Identify relationships between tables

data/examples/mcp/check_data_quality/prompt.md

@@ -0,0 +1,5 @@
+Run queries to check for:
+1. Tables with no data
+2. Potential duplicate records
+3. Missing required fields (NULL values in NOT NULL columns)
+4. Data consistency across related tables

data/examples/mcp/check_documentation/prompt.md

@@ -0,0 +1,4 @@
+Using the GitDocs tool:
+1. Find relevant documentation for the changed code
+2. Determine if documentation updates are needed
+3. Suggest specific documentation changes if required

data/examples/mcp/create_recommendations/prompt.md

@@ -0,0 +1,5 @@
+Create a final report with:
+1. Database health summary
+2. Critical issues that need immediate attention
+3. Performance optimization recommendations
+4. Data quality improvement suggestions

data/examples/mcp/database_workflow.yml

@@ -0,0 +1,29 @@
+# MCP Database Integration Example
+# This workflow demonstrates using a PostgreSQL MCP server for database operations
+
+name: Database Analysis Workflow
+model: gpt-4o-mini
+
+tools:
+  # PostgreSQL MCP tool
+  - Database:
+      command: npx
+      args:
+        - "-y"
+        - "@modelcontextprotocol/server-postgres"
+      env:
+        DATABASE_URL: "{{env.DATABASE_URL}}"
+      only:
+        - query
+        - list_tables
+        - describe_table
+        - get_schema
+
+steps:
+  - analyze_schema
+  - check_data_quality
+  - generate_insights
+  - create_recommendations
+
+create_recommendations:
+  print_response: true

data/examples/mcp/env_demo/workflow.yml

@@ -0,0 +1,34 @@
+# Environment Variable Interpolation Demo
+# This shows how ENV variables are interpolated in MCP tool configurations
+
+name: ENV Interpolation Demo
+model: o4-mini
+
+tools:
+  - Roast::Tools::ReadFile
+  
+  # Filesystem MCP with interpolated path
+  # The user's home directory will be resolved from ENV['HOME']
+  - UserFiles:
+      command: npx
+      args: ["-y", "@modelcontextprotocol/server-filesystem", "{{ENV['HOME'] || '/tmp'}}"]
+      only:
+        - list_directory
+        - get_file_info
+
+steps:
+  - show_env: |
+      First, tell me what user is running this workflow. The current user is: {{ENV['USER'] || 'unknown'}}
+      And their home directory is: {{ENV['HOME'] || 'not set'}}
+      
+  - list_home: |
+      Now use the UserFiles MCP tool's list_directory function to list the user's home directory.
+      Just show the first 5-10 items you find.
+
+
+show_env:
+  print_response: true
+
+list_home:
+  model: gpt-4o-mini
+  print_response: true

data/examples/mcp/fetch_pr_context/prompt.md

@@ -0,0 +1,4 @@
+Using the GitHub tool, fetch details about PR #{{env.PR_NUMBER}}:
+1. Get the PR description and title
+2. List all changed files
+3. Identify the type of changes (feature, bugfix, refactor)

data/examples/mcp/filesystem_demo/create_test_file/prompt.md

@@ -0,0 +1,2 @@
+First, use the read_file Roast tool to read this workflow file itself (examples/mcp/filesystem_demo/workflow.yml).
+This is just to verify traditional tools work and show the content of the workflow.

data/examples/mcp/filesystem_demo/list_files/prompt.md

@@ -0,0 +1,6 @@
+Use the FileSystem MCP tool's list_directory function to list the contents of the /tmp directory.
+
+The function expects these parameters:
+- path: "/tmp"
+
+Show me what files and directories are in /tmp.

data/examples/mcp/filesystem_demo/read_with_mcp/prompt.md

@@ -0,0 +1,7 @@
+If you found any .txt, .md, or .log files in /tmp from the previous step, pick one and read it using the 
+FileSystem MCP tool's read_file function. 
+
+The function expects these parameters:
+- path: (the full path to the file you want to read)
+
+If no suitable text files were found, just say "No text files found in /tmp to read".

data/examples/mcp/filesystem_demo/workflow.yml

@@ -0,0 +1,38 @@
+# Filesystem MCP Example
+# This uses the filesystem MCP server which provides safe file operations
+
+name: Filesystem MCP Example
+model: gpt-4o-mini
+
+tools:
+  # Traditional Roast tools
+  - Roast::Tools::ReadFile
+  
+  # Filesystem MCP server
+  - FileSystem:
+      command: npx
+      args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
+      only:
+        - read_file
+        - list_directory
+        - get_file_info
+
+steps:
+  - create_test_file
+  - list_files
+  - read_with_mcp
+
+create_test_file:
+  prompt: |
+    First, use the read_file Roast tool to read this workflow file itself (examples/mcp/filesystem_demo/workflow.yml).
+    This is just to verify traditional tools work.
+
+list_files:
+  prompt: |
+    Use the FileSystem MCP tool's list_directory function to list the contents of the /tmp directory.
+    The function expects a "path" parameter with value "/tmp".
+
+read_with_mcp:
+  prompt: |
+    If you see any .txt or .md files in /tmp from the previous step, try reading one with the 
+    FileSystem MCP tool's read_file function. Otherwise, just say "No text files found in /tmp".

data/examples/mcp/generate_insights/prompt.md

@@ -0,0 +1,4 @@
+Based on the schema and data quality analysis:
+1. Identify the most important tables for business operations
+2. Find opportunities for optimization (indexes, denormalization, etc.)
+3. Suggest data cleanup tasks if needed

data/examples/mcp/generate_report/prompt.md

@@ -0,0 +1,6 @@
+Generate a comprehensive bug analysis report that includes:
+- Total number of open bug issues
+- Categorization of bugs by type
+- List of critical issues
+- Recommendations for prioritization
+- Suggested next steps for the development team

data/examples/mcp/generate_review/prompt.md

@@ -0,0 +1,16 @@
+Generate a structured code review with:
+{
+  "summary": "Overall assessment",
+  "approval_status": "approve|request_changes|comment",
+  "issues": [
+    {
+      "severity": "critical|major|minor",
+      "file": "filename",
+      "line": 123,
+      "description": "Issue description",
+      "suggestion": "How to fix"
+    }
+  ],
+  "positive_feedback": ["What was done well"],
+  "documentation_updates": ["Required doc changes"]
+}

data/examples/mcp/github_workflow.yml

@@ -0,0 +1,32 @@
+# MCP GitHub Integration Example
+# This workflow demonstrates using the GitHub MCP server to interact with repositories
+
+name: GitHub Issue Analyzer
+model: gpt-4o-mini
+
+tools:
+  # Traditional Roast tool for writing output
+  - Roast::Tools::WriteFile
+  
+  # GitHub MCP tool - using npx to run the server
+  - GitHub:
+      command: npx
+      args:
+        - "-y"
+        - "@modelcontextprotocol/server-github"
+      env:
+        GITHUB_PERSONAL_ACCESS_TOKEN: "{{env.GITHUB_TOKEN}}"
+      only:
+        - search_repositories
+        - get_repository
+        - get_issue
+        - list_issues
+        - create_issue
+        - create_pull_request
+        - search_code
+
+steps:
+  - search_issues
+  - analyze_issues
+  - generate_report
+  - save_report

data/examples/mcp/multi_mcp_workflow.yml

@@ -0,0 +1,58 @@
+# Multi-MCP Integration Example
+# This workflow combines multiple MCP tools with traditional Roast tools
+
+name: Code Review Assistant
+model: gpt-4o-mini
+target: "README.md"  # Use a static file for testing
+
+tools:
+  # Traditional Roast tools
+  - Roast::Tools::ReadFile
+  - Roast::Tools::Grep
+  - Roast::Tools::WriteFile
+  
+  # Multiple MCP tools
+  # - GitDocs:
+  #     url: https://gitmcp.io/{{ENV['REPO_OWNER']}}/{{ENV['REPO_NAME']}}/docs
+  #     env:
+  #       Accept: application/json
+  
+  # GitHub MCP (requires GITHUB_TOKEN environment variable)
+  # Uncomment to use:
+  # - GitHub:
+  #     command: npx
+  #     args: ["-y", "@modelcontextprotocol/server-github"]
+  #     env:
+  #       GITHUB_PERSONAL_ACCESS_TOKEN: "{{ENV['GITHUB_TOKEN']}}"
+  #     only:
+  #       - get_pull_request
+  #       - create_pull_request_comment
+  #       - get_file_content
+  
+  # Filesystem MCP (no auth required)
+  - FileSystem:
+      command: npx
+      args: ["-y", "@modelcontextprotocol/server-filesystem", "."]
+      only:
+        - read_file
+        - list_directory
+  
+  # - Linter:
+  #     command: ./custom-linter-mcp
+  #     env:
+  #       CONFIG_PATH: "{{ENV['LINTER_CONFIG']}}"
+  #     only:
+  #       - analyze_code
+  #       - suggest_fixes
+
+steps:
+  - test_read
+  - test_filesystem
+
+test_read:
+  prompt: Use the ReadFile tool to read the README.md file
+
+test_filesystem:
+  prompt: |
+    Use the FileSystem MCP tool's list_directory function to list the current directory.
+    Then pick an interesting file and read it with the read_file function.

data/examples/mcp/post_review/prompt.md

@@ -0,0 +1,3 @@
+Using the GitHub tool, post the code review as a comment on PR #{{env.PR_NUMBER}}.
+Format the review in a friendly, constructive manner.
+Include specific line-by-line suggestions where appropriate.

data/examples/mcp/save_report/prompt.md

@@ -0,0 +1,6 @@
+Using the write_file tool, save the bug analysis report to a file named "bug_analysis_report.md".
+
+The report content should be:
+<%= output.generate_report %>
+
+Save this to the file and confirm when complete.

data/examples/mcp/search_issues/prompt.md

@@ -0,0 +1,2 @@
+Using the GitHub tool, search for open issues with the "bug" label in the Shopify/roast repository.
+List the issue titles, numbers, and created dates.

data/examples/mcp/summarize/prompt.md

@@ -0,0 +1 @@
+Generate a brief summary, with one code example, in Markdown format.

data/examples/mcp/test_filesystem/prompt.md

@@ -0,0 +1,6 @@
+Use the FileSystem MCP tool to explore the current directory:
+
+1. First, use the list_directory function with path "." to see what files are here
+2. Then pick one interesting file (like a .yml or .md file) and read it using the read_file function
+
+This demonstrates using MCP tools for filesystem access.

data/examples/mcp/test_github/prompt.md

@@ -0,0 +1,8 @@
+Use the GitHub MCP tool's get_file_content function to fetch the README.md from the Shopify/roast repository.
+
+The function should be called with these parameters:
+- owner: "Shopify" 
+- repo: "roast"
+- path: "README.md"
+
+Then summarize what the README contains.

data/examples/mcp/test_read/prompt.md

@@ -0,0 +1 @@
+Use the read_file function to read the README.md file in the current directory and summarize what it contains.

data/examples/mcp/workflow.yml

@@ -0,0 +1,35 @@
+# usage: roast execute examples/mcp/workflow.yml -o output.md
+
+name: MCP Tools Example
+model: gpt-4o-mini
+tools:
+  # SSE MCP:
+  - Roast Docs:
+      url: https://gitmcp.io/Shopify/roast/docs
+      # Can pass headers for authentication
+      # env:
+      #   - "Authorization: Bearer {{resource.api_token}}"
+  # stdio MCPs are also supported
+  # - GitHub:
+  #     command: "docker",
+  #     args:
+  #       - "run"
+  #       - "-i"
+  #       - "--rm"
+  #       - "-e"
+  #       - "GITHUB_PERSONAL_ACCESS_TOKEN"
+  #       - "ghcr.io/github/github-mcp-server"
+  #     env:
+  #       GITHUB_PERSONAL_ACCESS_TOKEN: "<YOUR_TOKEN>"
+  #     only:
+  #       - get_issue
+  #       - get_issue_comments
+  #     except:
+  #       - create_issue
+
+steps:
+  - get_doc: Read the Roast docs, and tell me how to use MCP tools.
+  - summarize
+
+summarize:
+  print_response: true

data/examples/shared_config/README.md

@@ -0,0 +1,52 @@
+# Shared Configuration Example
+
+This example demonstrates how to use `shared.yml` to define common configuration anchors that can be referenced across multiple workflow files.
+
+## Structure
+
+```
+├── shared.yml                      # Common configuration anchors
+└── example_with_shared_config/
+    └── workflow.yml               # Workflow using shared anchors
+```
+
+## How it works
+
+1. When loading a workflow file, Roast checks if `shared.yml` exists one level above the workflow directory
+2. If found, it loads `shared.yml` first, then the workflow file
+3. This allows YAML anchors defined in `shared.yml` to be referenced in workflow files
+
+## Example Usage
+
+In `shared.yml`:
+```yaml
+standard_tools: &standard_tools
+  - Roast::Tools::Grep
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::SearchFile
+
+mirage: &mirage '$(echo "Oh my god. Its a mirage")'
+```
+
+In your workflow:
+```yaml
+name: Example with shared config
+tools: *standard_tools  # Reference the standard tools from shared.yml
+
+steps:
+  - *mirage  # Reference a shared step definition
+  - sabotage: '$(echo "Im tellin yall, its sabotage")'
+```
+
+## Running the Example
+
+```bash
+# Run the workflow that uses shared configuration
+roast execute examples/shared_config/example_with_shared_config/workflow.yml
+```
+
+The workflow will:
+1. Load the shared configuration from `shared.yml`
+2. Apply the referenced tools and steps
+3. Execute the workflow with the merged configuration

data/examples/shared_config/example_with_shared_config/workflow.yml

@@ -0,0 +1,6 @@
+name: Example with shared config
+tools: *standard_tools
+
+steps:
+  - *mirage
+  - sabotage: '$(echo "Im tellin yall, its sabotage")'

data/examples/shared_config/shared.yml

@@ -0,0 +1,7 @@
+standard_tools: &standard_tools
+  - Roast::Tools::Grep
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::SearchFile
+
+mirage: &mirage '$(echo "Oh my god. Its a mirage")'