omni_agent 0.1.10 → 0.2.0

data/docs/tools.md

@@ -0,0 +1,171 @@
+<!-- Generated from omniagent-docs/docs/tools.mdx by `bundle exec rake docs:sync`. Do not edit by hand. -->
+<!-- Online version: https://omniagent.activetaskers.com/docs/tools -->
+
+# Tools
+
+Tools are how agents take action beyond generating text. Every tool is a small class that declares an input schema and implements `#execute`.
+
+## Defining a Tool
+
+Tools generated for an agent live under `app/agents/<agent_name>/tools/` and are namespaced under `<AgentClass>::Tools`:
+
+```ruby
+module ResearchAgent::Tools
+  class GetWeather < OmniAgent::Tool
+    description "Get current weather for a city"
+    tags :weather
+    metadata category: :utility
+
+    input do
+      string :city, description: "City name"
+    end
+
+    def execute(city:)
+      "Sunny in #{city}"
+    end
+  end
+end
+```
+
+* **`description`**: Sent to the provider so the model knows when to call the tool.
+* **`tags`**: Symbols used for filtering tools available to an agent.
+* **`metadata`**: Free-form hash for your own bookkeeping (not sent to the provider).
+* **`input`**: Declares the JSON-schema-style parameters the model must supply.
+* **`execute`**: Runs when the model calls the tool. Receives the parsed arguments as keyword args.
+
+## MCP Title and Annotations
+
+When a tool is exposed through an [MCP server](mcp/defining-servers.md), you can add a display title and behavior hints for MCP clients:
+
+```ruby
+class GetWeather < OmniAgent::Tool
+  title "Get weather"
+  annotations read_only: true, destructive: false, idempotent: true, open_world: true
+end
+```
+
+These are sent only to MCP clients, not to LLM providers. See [Tool Annotations](mcp/defining-servers.md#tool-annotations).
+
+## Input Schema DSL
+
+Use `input do ... end` to declare typed parameters:
+
+```ruby
+input do
+  string :city, description: "City name"
+  integer :days, description: "Forecast length", required: false
+  number :confidence, min: 0.0, max: 1.0, required: false
+  boolean :metric, required: false
+  array :tags, items_type: "string"
+  hash :coordinates do
+    string :lat
+    string :lng
+  end
+  enum :unit, values: [ "celsius", "fahrenheit" ], description: "Temperature unit"
+end
+```
+
+Supported field types:
+
+* **`string`**
+* **`integer`**
+* **`number`** — floating-point values
+* **`boolean`**
+* **`array`** — pass `items_type:` for primitive items, or a block for an array of objects
+* **`hash`** — pass a block to declare nested properties, or omit it for a free-form object
+* **`enum`** — pass `values:` as a non-empty array of same-typed values (all strings, all integers, all floats, or all booleans). Mixed-type or empty value sets raise `ArgumentError` at schema-definition time.
+* **`polymorphic`** — see [Polymorphic References](#polymorphic-references) below.
+
+All fields are `required: true` by default. Pass `required: false` to make a field optional.
+
+`invoke` validates arguments against the schema before calling `#execute`: a missing required field, an enum value outside `values:`, a constraint violation, or a failing custom validator all raise `ArgumentError` rather than reaching your tool code.
+
+## Validations
+
+`string`, `integer`, `number`, and `array` accept optional constraint kwargs. These are emitted as standard JSON Schema keywords (so the model sees them too) and enforced again at runtime by `invoke`:
+
+```ruby
+input do
+  string  :name,  min_length: 3, max_length: 40, pattern: /\A[a-z]+\z/i, format: "email"
+  integer :level, min: 1, max: 5
+  number  :ratio, min: 0.0, max: 1.0
+  array   :tags,  items_type: "string", min_items: 1, max_items: 10
+end
+```
+
+| Kwarg | Applies to | JSON Schema keyword |
+|---|---|---|
+| `min_length:` / `max_length:` | `string` | `minLength` / `maxLength` |
+| `pattern:` (`Regexp` or `String`) | `string` | `pattern` |
+| `format:` | `string` | `format` |
+| `min:` / `max:` | `integer`, `number` | `minimum` / `maximum` |
+| `min_items:` / `max_items:` | `array` | `minItems` / `maxItems` |
+
+Every field type also accepts `validate:`, a `Proc` for arbitrary runtime rules. It is never serialized into the schema sent to the provider — it only runs locally in `invoke`. Return `false` to raise a generic `ArgumentError`, or raise your own `ArgumentError` for a custom message:
+
+```ruby
+input do
+  string :slug, validate: ->(v) { v == v.downcase or raise ArgumentError, "slug must be lowercase" }
+end
+```
+
+## Polymorphic References
+
+`polymorphic :name` declares a Rails-style polymorphic reference. It expands into two ordinary schema fields, `name_type` and `name_id`:
+
+```ruby
+input do
+  polymorphic :actor,
+    types: [ "User", "Admin" ],   # allowed actor_type values -> becomes an enum
+    description: "The actor performing the action",
+    id_type: :integer,            # actor_id schema type, default "string"
+    resolve: true                 # fetch the actual record instead of raw type/id
+end
+```
+
+Individual descriptions for each sub-field can be set with the block form instead of a shared `description:`:
+
+```ruby
+polymorphic :actor, types: [ "User", "Admin" ], resolve: true do
+  type values: [ "User", "Admin" ], description: "The actor's class name"
+  id   type: :integer,             description: "The actor's primary key"
+end
+```
+
+Two runtime modes:
+
+* **`resolve: false` (default)** — `#execute` receives `actor_type:` and `actor_id:` as separate keyword arguments. `actor_type` is validated against `types:` like any other enum.
+* **`resolve: true`** — requires `types:`. At runtime, `invoke` looks up the class named by `actor_type` (restricted to the `types:` whitelist — never an arbitrary string) via `Object.const_get`, calls `.find(actor_id)` on it, and passes the result as a single `actor:` keyword argument instead. A record that isn't found raises `ArgumentError`, which the agent loop surfaces back to the model as a retryable tool error, just like an invalid enum value.
+
+`resolve: true` without `types:` raises `ArgumentError` at schema-definition time.
+
+## Stopping Generation Early
+
+Sometimes a tool call should end the agent's run instead of looping back to the model — for example, a tool that hands off to a human or returns a final answer directly.
+
+Declare it at the class level:
+
+```ruby
+class Escalate < OmniAgent::Tool
+  stops_generation
+
+  def execute(reason:)
+    "Escalated: #{reason}"
+  end
+end
+```
+
+Or trigger it conditionally from inside `#execute`:
+
+```ruby
+def execute(reason:)
+  stop_generation! if reason == "urgent"
+  "Logged: #{reason}"
+end
+```
+
+## How Tools Are Discovered
+
+`OmniAgent::Agent#available_tools` looks up the `<AgentClass>::Tools` namespace and collects every constant that is a subclass of `OmniAgent::Tool`. There's no manual registration step — defining a tool class in the right namespace is enough for the agent to pick it up.
+
+The class method `AgentClass.tool_classes` returns the same list without creating an agent instance (and therefore without a provider). [`tools_from`](mcp/defining-servers.md) in MCP servers uses it.

data/exe/omni_agent

@@ -3,13 +3,24 @@
 
 require "optparse"
 
-options = { fresh: false }
-parser = OptionParser.new do |opts|
-  opts.banner = <<~BANNER
-    Usage: omni_agent eval [pattern] [options]
-           omni_agent mcp <ServerClass>
-  BANNER
-  opts.on("--fresh", "Clear the eval cache before running") { options[:fresh] = true }
+BANNER = <<~BANNER
+  Usage: omni_agent eval [pattern] [--fresh]
+         omni_agent mcp <ServerClass>
+         omni_agent docs [page] [--path]
+BANNER
+
+# Each command gets its own parser, so a flag meant for another command is
+# rejected instead of silently ignored.
+def parse_options(options)
+  OptionParser.new do |opts|
+    opts.banner = BANNER
+    yield opts, options
+  end.parse!(ARGV)
+  options
+rescue OptionParser::ParseError => e
+  warn e.message
+  warn BANNER
+  exit 1
 end
 
 def require_rails_environment(command)
@@ -24,7 +35,9 @@ command = ARGV.shift
 
 case command
 when "eval"
-  parser.parse!(ARGV)
+  options = parse_options(fresh: false) do |opts, parsed|
+    opts.on("--fresh", "Clear the eval cache before running") { parsed[:fresh] = true }
+  end
   pattern = ARGV.shift
 
   require_rails_environment("eval")
@@ -48,11 +61,36 @@ when "mcp"
 
   require_rails_environment("mcp")
   OmniAgent::MCP::Stdio.start(server)
+when "docs"
+  options = parse_options(path: false) do |opts, parsed|
+    opts.on("--path", "Print the path of the bundled docs instead of their contents") { parsed[:path] = true }
+  end
+  require_relative "../lib/omni_agent/docs"
+
+  # docs/ is generated when the gem is built, so a gem installed from git
+  # (`gem "omni_agent", github: ...`) doesn't have it.
+  unless File.directory?(OmniAgent::Docs.root)
+    abort "The bundled docs are missing (they're generated by `rake build`). Read them online at https://omniagent.activetaskers.com/docs/intro"
+  end
+
+  page = ARGV.shift
+  file = OmniAgent::Docs.path(page || "index")
+  unless file
+    warn "Unknown docs page: #{page.inspect}. Available pages:"
+    warn OmniAgent::Docs.pages.map { |name| "  #{name}" }
+    exit 1
+  end
+
+  if options[:path]
+    puts page ? file : OmniAgent::Docs.root
+  else
+    puts File.read(file)
+  end
 when nil
-  warn parser.banner
+  warn BANNER
   exit 1
 else
   warn "Unknown command: #{command.inspect}"
-  warn parser.banner
+  warn BANNER
   exit 1
 end

data/lib/generators/omni_agent/chunks/chunks_generator.rb

@@ -0,0 +1,130 @@
+require "rails/generators/named_base"
+require_relative "../vector_store"
+
+module OmniAgent
+  module Generators
+    class ChunksGenerator < Rails::Generators::NamedBase
+      include VectorStore
+
+      desc "Creates a chunk model for a document model and declares has_chunks on it"
+
+      argument :source, type: :string, banner: "source", desc: "Attachment, text column or rich text to chunk"
+
+      class_option :as, type: :string, desc: "Loader to enforce (pdf, html, markdown, text). Default: detected per file"
+      class_option :chunker, type: :string, enum: OmniAgent::Chunkers.registry.keys.map(&:to_s), desc: "Default: recursive"
+      class_option :size, type: :numeric, desc: "Chunk size in characters. Default: 1000"
+      class_option :overlap, type: :numeric, desc: "Characters repeated between chunks. Default: 200"
+      class_option :inline, type: :boolean, default: false, desc: "Ingest during save instead of in a background job"
+      class_option :dimensions, type: :numeric, desc: "Vector size (default: detected by calling the embedding model once)"
+      class_option :backend, type: :string, enum: BACKENDS, desc: "Default: detected from the database adapter"
+
+      def check_backend_and_dimensions
+        raise Thor::Error, "#{model_path} not found. Generate the model first." unless File.exist?(File.join(destination_root, model_path))
+
+        backend
+        dimensions
+      end
+
+      def create_migration_file
+        migration_file("Create#{chunk_table.camelize}", [
+          "create_table :#{chunk_table} do |t|",
+          "  t.references :#{reference}, null: false, foreign_key: { to_table: :#{table_name} }",
+          "  t.integer :position, null: false",
+          "  t.text :content, null: false",
+          "  t.text :context",
+          "  t.json :metadata",
+          "end",
+          "add_index :#{chunk_table}, [ :#{reference}_id, :position ]",
+          *vector_column_lines(chunk_table, "embedding")
+        ])
+      end
+
+      def create_vector_store_initializer
+        create_sqlite_initializer
+      end
+
+      def create_chunk_model
+        lines = [ "class #{chunk_class} < ApplicationRecord" ]
+        lines << "  belongs_to :#{reference}#{", class_name: \"#{class_name}\"" if reference.camelize != class_name}"
+        lines << "  # Ruby backend, small tables only: every search loads and ranks all rows in Ruby." if backend == "ruby"
+        lines << "  has_embedding #{[ ':content', *vector_store_options ].join(', ')}"
+        lines << "end"
+        create_file File.join("app", "models", "#{file_path}_chunk.rb"), lines.join("\n") + "\n"
+      end
+
+      def add_has_chunks
+        lines = []
+        lines << "  has_one_attached :#{source}\n" if attachment_needed?
+        lines << "  has_chunks #{chunk_arguments.join(', ')}\n"
+        inject_into_class model_path, class_name, lines.join
+      end
+
+      def show_next_steps
+        say ""
+        say_vector_store_setup
+        say "To read PDFs, also:"
+        say "  bundle add pdf-reader"
+        say "Then migrate and ingest existing records:"
+        say "  bin/rails active_storage:install # if #{source} is an upload and you haven't" if attachment?
+        say "  bin/rails db:migrate"
+        say "  bin/rails \"omni_agent:chunks:ingest[#{class_name}]\""
+        say ""
+        say "Give an agent a search tool with:"
+        say "  searches #{class_name}"
+      end
+
+      private
+
+      def model_path
+        File.join("app", "models", "#{file_path}.rb")
+      end
+
+      def chunk_class
+        "#{class_name}Chunk"
+      end
+
+      def chunk_table
+        "#{singular_table_name}_chunks"
+      end
+
+      def reference
+        class_name.demodulize.underscore
+      end
+
+      def chunk_arguments
+        arguments = [ ":#{source}" ]
+        arguments << "as: :#{options[:as]}" if options[:as]
+        arguments << "chunker: :#{options[:chunker]}" if options[:chunker]
+        arguments << "size: #{options[:size]}" if options[:size]
+        arguments << "overlap: #{options[:overlap]}" if options[:overlap]
+        arguments << "later: false" if options[:inline]
+        arguments
+      end
+
+      def attachment?
+        attachment_needed? || model_source.match?(/has_(one|many)_attached\s+:#{source}\b/)
+      end
+
+      # A source the model doesn't declare and the table doesn't have is an upload.
+      def attachment_needed?
+        return @attachment_needed if defined?(@attachment_needed)
+
+        @attachment_needed = !model_source.match?(/(has_\w+|def|attribute|alias_attribute)\s+:?#{source}\b/) && !source_column?
+      end
+
+      def model_source
+        File.read(File.join(destination_root, model_path))
+      end
+
+      # In the database, or in a migration that may not have run yet.
+      def source_column?
+        migrations = Dir[File.join(destination_root, "db", "migrate", "*.rb")].map { |path| File.read(path) }
+        return true if migrations.any? { |migration| migration.include?(":#{table_name}") && migration.match?(/[:"]#{source}\b/) }
+
+        ActiveRecord::Base.with_connection { |connection| connection.table_exists?(table_name) && connection.column_exists?(table_name, source) }
+      rescue ActiveRecord::ActiveRecordError
+        false
+      end
+    end
+  end
+end

data/lib/generators/omni_agent/embeddings/embeddings_generator.rb

@@ -0,0 +1,72 @@
+require "rails/generators/named_base"
+require_relative "../vector_store"
+
+module OmniAgent
+  module Generators
+    class EmbeddingsGenerator < Rails::Generators::NamedBase
+      include VectorStore
+
+      desc "Adds an embedding column to a model and declares has_embedding on it"
+
+      argument :sources, type: :array, banner: "attribute [attribute ...]", desc: "Attributes to embed"
+
+      class_option :column, type: :string, default: "embedding", desc: "Column that stores the vector"
+      class_option :dimensions, type: :numeric, desc: "Vector size (default: detected by calling the embedding model once)"
+      class_option :backend, type: :string, enum: BACKENDS, desc: "Default: detected from the database adapter"
+      class_option :later, type: :boolean, default: false, desc: "Embed in a background job after commit"
+
+      def check_backend_and_dimensions
+        backend
+        dimensions
+      end
+
+      def create_migration_file
+        migration_file(migration_name, vector_column_lines(table_name, column))
+      end
+
+      def create_vector_store_initializer
+        create_sqlite_initializer
+      end
+
+      def add_has_embedding
+        model_path = File.join("app", "models", "#{file_path}.rb")
+        raise Thor::Error, "#{model_path} not found. Generate the model first." unless File.exist?(File.join(destination_root, model_path))
+
+        inject_into_class model_path, class_name, model_lines
+      end
+
+      def show_next_steps
+        say ""
+        say_vector_store_setup
+        say "Then migrate and embed existing records:"
+        say "  bin/rails db:migrate"
+        say "  bin/rails \"omni_agent:embeddings:backfill[#{class_name}]\""
+        say ""
+        say "Give an agent a search tool with:"
+        say "  searches #{class_name}"
+      end
+
+      private
+
+      def column
+        options[:column]
+      end
+
+      def migration_name
+        "Add#{column.camelize}To#{table_name.camelize}"
+      end
+
+      def model_lines
+        arguments = sources.map { |source| ":#{source}" }
+        arguments << "column: :#{column}" unless column == "embedding"
+        arguments.concat(vector_store_options)
+        arguments << "later: true" if options[:later]
+
+        lines = []
+        lines << "  # Ruby backend, small tables only: every search loads and ranks all rows in Ruby.\n" if backend == "ruby"
+        lines << "  has_embedding #{arguments.join(', ')}\n"
+        lines.join
+      end
+    end
+  end
+end

data/lib/generators/omni_agent/install/install_generator.rb

@@ -4,7 +4,55 @@ require "fileutils"
 module OmniAgent
   module Generators
     class InstallGenerator < Rails::Generators::Base
-      desc "Creates OmniAgent base directories in app/agents"
+      desc "Creates OmniAgent base directories in app/agents and config/initializers/omni_agent.rb"
+
+      INITIALIZER = <<~RUBY
+        OmniAgent.configure do |config|
+          # Every setting below shows its default. Uncomment to change it.
+
+          # Agents
+          # config.default_provider = :openai          # :openai, :ollama, :mock
+          # config.default_model = "gpt-4o-mini"
+          # config.max_tool_iterations = 10
+          # config.max_delegation_depth = 5
+
+          # Provider retries (rate limits, timeouts, 5xx), with exponential backoff
+          # config.max_retries = 3
+          # config.retry_base_delay = 0.5
+
+          # Embeddings and RAG (has_embedding, searches)
+          # config.embedding_provider = nil            # nil uses default_provider; use :mock in tests
+          # config.embedding_model = nil               # nil uses the provider's default embedding model
+          # config.embedding_job_options = {}          # for has_embedding later: true, e.g. { queue: :embeddings, priority: 10 }
+          # config.document_loaders = {}               # extra has_chunks loaders, e.g. { docx: DocxLoader }
+
+          # Evals
+          # config.eval_judge_provider = nil           # nil uses the evaluated agent's provider
+          # config.eval_judge_model = nil
+          # config.eval_cache_enabled = true
+          # config.eval_cache_path = "tmp/omni_agent_eval_cache.json"
+
+          # MCP servers
+          # config.mcp_allowed_origins = []
+          # config.mcp_max_request_bytes = 1_048_576
+        end
+      RUBY
+
+      class_option :skip_agents_md, type: :boolean, default: false,
+                                    desc: "Don't point AGENTS.md (and CLAUDE.md, if present) at the bundled OmniAgent docs"
+
+      AGENTS_MD_MARKER = "<!-- omni_agent:docs -->".freeze
+      AGENTS_MD_SECTION = <<~MARKDOWN.freeze
+        #{AGENTS_MD_MARKER}
+        ## OmniAgent
+
+        This app builds AI agents with the `omni_agent` gem. Its documentation ships
+        inside the gem: run `bundle exec omni_agent docs` for the index (conventions,
+        typical flows and every page), `bundle exec omni_agent docs <page>` to read a
+        page, or `bundle exec omni_agent docs --path` for the directory. Read it before
+        adding or changing agents (`app/agents/`), tools, MCP servers (`app/mcp_servers/`)
+        or evals (`evals/`), and prefer the `rails generate omni_agent:*` generators.
+      MARKDOWN
 
       def create_agents_root
         agents_root = File.join(destination_root, "app", "agents")
@@ -12,10 +60,39 @@ module OmniAgent
         say_status :create, agents_root
       end
 
+      def create_initializer
+        path = File.join("config", "initializers", "omni_agent.rb")
+        return say_status(:skip, path, :yellow) if File.exist?(File.join(destination_root, path))
+
+        create_file path, INITIALIZER
+      end
+
+      def point_coding_agents_at_docs
+        return if options[:skip_agents_md]
+
+        add_docs_section("AGENTS.md", create: true)
+        add_docs_section("CLAUDE.md", create: false)
+      end
+
       def show_next_steps
         say "OmniAgent install complete."
         say "Next: rails generate omni_agent:agent SupportAgent --with-tools Tool1 Tool2"
         say "Expose tools to MCP clients: rails generate omni_agent:mcp_server Support --agents SupportAgent"
+        say "Read the bundled docs: bundle exec omni_agent docs"
+      end
+
+      private
+
+      def add_docs_section(file, create:)
+        path = File.join(destination_root, file)
+
+        if File.exist?(path)
+          return say_status(:identical, file) if File.read(path).include?(AGENTS_MD_MARKER)
+
+          append_to_file file, "\n#{AGENTS_MD_SECTION}"
+        elsif create
+          create_file file, AGENTS_MD_SECTION
+        end
       end
     end
   end

data/lib/generators/omni_agent/mcp_server/mcp_server_generator.rb

@@ -101,6 +101,10 @@ module OmniAgent
 
             # Comma-separated list of accepted bearer tokens.
             authenticate :bearer, tokens: -> { ENV.fetch("#{token_env_var}", "").split(",") }
+            # Or OAuth 2.1 with tokens from your authorization server (add `gem "jwt"`):
+            # authenticate :oauth, resource: "https://app.example.com/mcp/...",
+            #                      authorization_servers: "https://auth.example.com",
+            #                      jwks_uri: "https://auth.example.com/.well-known/jwks.json"
 
             # Hide tools per principal:
             # authorize_tool { |tool_class, principal| true }

data/lib/generators/omni_agent/stock_agent/stock_agent_generator.rb

@@ -0,0 +1,73 @@
+require "rails/generators/named_base"
+require "fileutils"
+
+module OmniAgent
+  module Generators
+    class StockAgentGenerator < Rails::Generators::NamedBase
+      FIELD_TYPES = %w[string integer number boolean].freeze
+
+      class_option :from,
+                   type: :string,
+                   required: true,
+                   desc: "Stock agent to build on (e.g. ocr)"
+      class_option :fields,
+                   type: :array,
+                   default: [],
+                   banner: "name:type name:type ...",
+                   desc: "Output fields to declare (types: #{FIELD_TYPES.join(', ')})"
+
+      def create_agent_structure
+        agent_source = [
+          "class #{class_name} < #{stock_agent_class.name}",
+          "  output do",
+          *output_field_lines.map { |line| "    #{line}" },
+          "  end",
+          "end"
+        ].join("\n")
+
+        FileUtils.mkdir_p(agent_directory)
+        create_file(agent_file_path, "#{agent_source}\n")
+
+        create_file(prompt_file_path, <<~ERB)
+          <%# Optional: extra instructions appended after the #{stock_agent_class.stock_name} stock agent prompt. %>
+        ERB
+      end
+
+      private
+
+      def stock_agent_class
+        @stock_agent_class ||= OmniAgent::StockAgents.find(options[:from])
+      rescue OmniAgent::UnknownStockAgentError => e
+        raise Thor::Error, e.message
+      end
+
+      def output_field_lines
+        fields = Array(options[:fields]).map(&:strip).reject(&:empty?)
+        return [ 'string :example_field, description: "Describe the value the agent must return"' ] if fields.empty?
+
+        fields.map do |field|
+          name, type = field.split(":", 2)
+          type = (type || "string").downcase
+
+          unless FIELD_TYPES.include?(type)
+            raise Thor::Error, "Unsupported field type #{type.inspect} for #{name}. Use one of: #{FIELD_TYPES.join(', ')}"
+          end
+
+          "#{type} :#{name.underscore}"
+        end
+      end
+
+      def agent_directory
+        File.join(destination_root, "app", "agents", file_name)
+      end
+
+      def agent_file_path
+        File.join(destination_root, "app", "agents", "#{file_name}.rb")
+      end
+
+      def prompt_file_path
+        File.join(agent_directory, "prompt.md.erb")
+      end
+    end
+  end
+end