feather-ai 0.3.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9f2195b84828584d6764f60823b99fbbbe9961c1c0ebee3d26679b6450a0dd53
-  data.tar.gz: '008c505376ab82635e2a8fdae0475a3adab609c5b9bef588f6a48573c238036c'
+  metadata.gz: fa4e06ced9d8d48c533d41f97148fba4c26f3ffa478a603c4562960d50e6594f
+  data.tar.gz: 6f04ec946376b964fbf0a049a7837f20bae6eab45690e802b22e4f03b3789bc3
 SHA512:
-  metadata.gz: be14496f9c58080371192aa146f521178f563433e9bb28e4ecd720c3f802e4264e3a5efef7281813e6a1f18d2e03998eb69e4279dee5c156052879cb2510ed2b
-  data.tar.gz: 14f4968e7363d889e2d0590b675141ec4868385cd58138fbb4de4910acabe61385a14b079136ec0c10d93ba709b2a656de76e2e39bc50016b9faa98672134627
+  metadata.gz: 943e45ed67690b5791d953ff443e3d94ba9b1866f1143373ca08dc2ef59826720cbdd4a34eeb6d91cf8d2a626822738e068a9cae4750f92fbe8a4aa2a91b2cb7
+  data.tar.gz: 903b3b7968b1ebebc411e5e8c518649ca8b5cf617461cb630725b861a3dfaba8eed7e914b01803ff5b65c05d92d1806ac6607e4cc5ce235db5f74a9ed59d0d9e

data/CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+## 0.5.0 - 2026-10-02
+
+- **Breaking:** requires `ruby_llm` 2.x (fixes the ReDoS advisory in 1.x). Dev dependencies updated to clear all `bundle-audit` findings (rubocop no longer pulls in `mcp`).
+- `Result#cost` now comes from RubyLLM's per-model pricing registry instead of a hardcoded Anthropic-only table, so Haiku and non-Anthropic models are priced correctly; `nil` when the registry has no price.
+- `config.provider` is now a no-op kept for compatibility; RubyLLM 2 resolves the provider from the model.
+- Fix: `media_resolution` (a Gemini `generationConfig` field) is only sent to Gemini models. It was previously sent to every provider, which Anthropic rejects.
+- Fix: audio transcripts are now interpolated as text; previously the `RubyLLM::Transcription` object was interpolated.
+
+## 0.4.0 - 2026-07-05
+
+- Identification now returns ranked `candidates` (up to 3, each `{common_name:, species:, score:}`) on every `Result`, populated straight from the structured-output schema.
+- New `tools:` keyword on `FeatherAi.identify` (and `FeatherAi.configure { |c| c.tools = [...] }`) forwards RubyLLM tools to the chat, so identification can ground itself in real data (e.g. a species/region lookup backed by your app's database).
+- **Breaking:** consensus disagreement `candidates` changed from an array of `Result` objects to the same ranked-hash shape, scored by vote share.
+- `acts_as_sighting`'s `identify!` persists `candidates` when the table has a `candidates` column (new migrations add it as `jsonb`); existing tables without the column are unaffected.
+- Default models bumped to `claude-sonnet-4-5` / `claude-haiku-4-5` (Anthropic structured outputs require Sonnet 4.5+; the old `claude-haiku-4` id no longer resolves).
+
+Note: on Gemini models, combining a response schema with tools is rejected by the API on many models — the `tools:` option is tested against the Anthropic defaults.

data/CLAUDE.md

@@ -42,35 +42,51 @@ bundle exec rake release
 
 ## Architecture
 
-The gem's only runtime dependency is `ruby_llm`. All classes live under the `FeatherAi` module in `lib/feather_ai/`.
+The gem's only runtime dependency is `ruby_llm` 2.x (which pulls in `schematist` for the schema DSL). All classes live under the `FeatherAi` module in `lib/feather_ai/`.
 
 ### Data Flow
 
-`FeatherAi.identify(image, audio, location:, consensus:)` is the top-level entry point defined in `lib/feather_ai.rb`. It delegates to:
+`FeatherAi.identify(image, audio, location:, consensus:)` is the top-level entry point defined in `lib/feather_ai.rb`. `image` accepts a single path or an array of paths (multiple angles of the same bird). It delegates to:
 
-1. **Identifier** (`lib/feather_ai/identifier.rb`) — Core identification logic. Uses RubyLLM's vision for images and `RubyLLM.transcribe` for audio. When both inputs are provided, they're combined into a single multi-modal prompt. Uses `RubyLLM::Schema` for structured output so results are always clean `Result` objects, not raw LLM prose.
+1. **Identifier** (`lib/feather_ai/identifier.rb`) — Core identification logic. Uses RubyLLM's vision for images and `RubyLLM.transcribe` for audio. When both inputs are provided, they're combined into a single multi-modal prompt. Uses `Schematist::Schema` for structured output (including a `reasoning` field that forces step-by-step visual analysis before the identification); the parsed Hash is read via `response.parsed`, tokens via `response.tokens`, and USD cost via `response.cost.total` (RubyLLM's pricing registry). `config.media_resolution` is only sent to Gemini models.
 
-2. **Consensus** (`lib/feather_ai/consensus.rb`) — When `consensus: true`, runs identification through two configurable models independently. If they agree on species, returns `confident: true`. If they disagree, returns both as `candidates` with uncertain confidence.
+2. **Consensus** (`lib/feather_ai/consensus.rb`) — When `consensus: true`, runs identification through the configured `consensus_models` in parallel threads. Agreement is compared on normalized species name. If they agree, returns `confident: true`; if they disagree, returns both as `candidates` with low confidence. Token counts, cost, and duration are summed across models.
 
-3. **Result** (`lib/feather_ai/result.rb`) — Immutable value object wrapping all identification output. Exposes `common_name`, `species`, `family`, `confidence` (`:high`/`:medium`/`:low`), `confident?`, `region_native?`, `candidates`, `photography_tips`, and `to_h`. Photography tips are lazy-loaded via a second cheap LLM call only when accessed.
+3. **Result** (`lib/feather_ai/result.rb`) — Immutable value object wrapping all identification output. Identification fields: `common_name`, `species`, `family`, `confidence` (`:high`/`:medium`/`:low`), `confident?`, `region_native?`, `reasoning`, `candidates`, `photography_tips`, `to_h`. Observability fields from the LLM call: `model_id`, `input_tokens`, `output_tokens`, `cost`, `duration_ms`, `source` (`:vision`/`:audio`/`:multimodal`), `consensus_models`. Photography tips are lazy-loaded via a second cheap LLM call only when accessed.
 
-4. **PhotographyTips** (`lib/feather_ai/photography_tips.rb`) — Separate LLM call (small model) returning structured shooting advice for the identified species. Only invoked when `result.photography_tips` is called.
+4. **PhotographyTips** (`lib/feather_ai/photography_tips.rb`) — Separate LLM call (uses `tips_model`, a small model) returning structured shooting advice for the identified species. Only invoked when `result.photography_tips` is called.
+
+### Instrumentation
+
+`Instrumentation.instrument` (`lib/feather_ai/instrumentation.rb`) wraps identification in `ActiveSupport::Notifications` when ActiveSupport is loaded, and is a plain `yield` otherwise. Events: `identify.feather_ai` (Identifier) and `consensus.feather_ai` (Consensus); the `Result` is added to the payload after the call completes.
+
+### Errors
+
+All errors inherit from `FeatherAi::Error`: `ConfigurationError` (e.g. no image or audio provided) and `IdentificationError` (LLM call failure).
 
 ### Configuration
 
 ```ruby
 FeatherAi.configure do |c|
-  c.provider  = :anthropic
-  c.location  = "Perth, Western Australia"  # biases results to local species
-  c.model     = "claude-sonnet-4"
+  c.provider         = :anthropic                             # default
+  c.model            = "claude-sonnet-4"                      # default
+  c.location         = "Perth, Western Australia"             # biases results to local species
+  c.consensus_models = ["claude-sonnet-4", "claude-haiku-4"]  # default
+  c.tips_model       = "claude-haiku-4"                       # default
+  c.media_resolution = :high                                  # default; Gemini-only image resolution
 end
 ```
 
-Location can be set globally or per-call via `location:` keyword. It's injected into the system prompt to reduce false positives.
+`FeatherAi.configuration` is a lazily-initialized process singleton; `FeatherAi.reset!` clears it (used before every spec). Location can be set globally or per-call via the `location:` keyword. It's injected into the system prompt to reduce false positives.
 
 ### Rails Integration
 
-`lib/feather_ai/rails/` contains a Railtie and `acts_as_sighting` mixin. When included in an ActiveRecord model, it expects `photo` (ActiveStorage) and `location` (string) attributes, and adds an `identify!` method that populates species fields on the record. A generator (`bin/rails generate feather_ai:install`) scaffolds the migration.
+`lib/feather_ai/rails/` contains a Railtie and `acts_as_sighting` mixin. When included in an ActiveRecord model, it expects `photo` (ActiveStorage) and `location` (string) attributes, and adds:
+
+- `identify!` — downloads the photo, runs identification, persists species fields, returns the `Result`.
+- `correct!(attrs)` / `corrected?` / `correction_delta` — human corrections of AI identifications, stored in `corrected_*` columns with a `corrected_at` timestamp. `correction_delta` always diffs against the original AI values.
+
+Generators: `rails generate feather_ai:install [model_name]` scaffolds the identification-columns migration; `rails generate feather_ai:add_corrections [model_name]` adds the correction columns. Templates live in `lib/generators/feather_ai/templates/`.
 
 ## Code Style
 
@@ -81,7 +97,8 @@ Location can be set globally or per-call via `location:` keyword. It's injected
 
 ## Testing
 
-- Tests use VCR + WebMock to record/replay real LLM responses — no API keys needed in CI
-- Sample bird images (WA birds) live in `spec/support/fixtures/`
+- Specs stub RubyLLM directly with `instance_double`s (e.g. `allow(RubyLLM).to receive(:chat)`) — no API keys needed. VCR + WebMock are configured in `spec_helper.rb` but mainly serve to block real HTTP; there are no recorded cassettes.
+- `spec/support/helpers.rb` provides `build_result(overrides)` for constructing `FeatherAi::Result` test objects.
+- `FeatherAi.reset!` runs before every example, so config is always at defaults unless the spec configures it.
 - SimpleCov for coverage reporting
 - Dev test dependencies go in the `Gemfile`, not the gemspec

data/README.md

@@ -22,14 +22,17 @@ gem install feather-ai
 
 ```ruby
 FeatherAi.configure do |c|
-  c.provider  = :anthropic           # Default: :anthropic
-  c.model     = "claude-sonnet-4"    # Default: "claude-sonnet-4"
-  c.location  = "Perth, WA"          # Optional: biases results to local species
-  c.consensus_models = ["claude-sonnet-4", "claude-haiku-4"]  # Models used in consensus mode
+  c.provider         = :anthropic           # Deprecated no-op; RubyLLM 2 resolves the provider from the model
+  c.model            = "claude-sonnet-4-5"  # Default: "claude-sonnet-4-5"
+  c.location         = "Perth, WA"          # Optional: biases results to local species
+  c.consensus_models = ["claude-sonnet-4-5", "claude-haiku-4-5"]  # Models used in consensus mode
+  c.tips_model       = "claude-haiku-4-5"   # Model for photography tips (default)
+  c.media_resolution = :high                # Gemini only (generationConfig.mediaResolution); ignored elsewhere
+  c.tools            = []                   # RubyLLM tools available to every identification (default: none)
 end
 ```
 
-RubyLLM must be configured with your provider credentials before using FeatherAi. See the [RubyLLM docs](https://github.com/coelacanth/ruby_llm) for setup.
+Requires RubyLLM 2.x. RubyLLM must be configured with your provider credentials before using FeatherAi. See the [RubyLLM docs](https://rubyllm.com) for setup.
 
 ## Usage
 
@@ -54,6 +57,12 @@ Identify from audio:
 result = FeatherAi.identify(nil, "path/to/bird_call.mp3")
 ```
 
+Identify from multiple images at once:
+
+```ruby
+result = FeatherAi.identify(["front.jpg", "side.jpg"])
+```
+
 Identify from both image and audio:
 
 ```ruby
@@ -72,9 +81,40 @@ result.region_native? # => true/false based on species range
 
 A default location can also be set globally in configuration.
 
+### Ranked Candidates
+
+Every identification returns up to three candidate species ranked by likelihood, with the top pick first:
+
+```ruby
+result = FeatherAi.identify("path/to/bird.jpg")
+
+result.candidates
+# => [{ common_name: "Splendid Fairywren", species: "Malurus splendens", score: 0.9 },
+#     { common_name: "Superb Fairywren",   species: "Malurus cyaneus",   score: 0.1 }]
+```
+
+### Grounded Identification (Tools)
+
+Pass RubyLLM tools so the model can verify its identification against real data — for example a species/region lookup backed by your own database:
+
+```ruby
+class SpeciesLookupTool < RubyLLM::Tool
+  description "Looks up whether a species occurs in a region"
+  parameter :species, description: "Scientific species name"
+
+  def execute(species:)
+    Species.find_by(scientific_name: species)&.slice(:regions, :description) || { found: false }
+  end
+end
+
+result = FeatherAi.identify("path/to/bird.jpg", location: "Perth, WA", tools: [SpeciesLookupTool])
+```
+
+Tools can also be set globally via `c.tools` in configuration. Note: Gemini rejects structured-output schemas combined with tools on many models — the `tools:` option is tested against the Anthropic defaults.
+
 ### Consensus Mode
 
-Run identification through two models independently. When both agree on species, you get high confidence. When they disagree, you get the candidates:
+Run identification through two models independently. When both agree on species, you get high confidence. When they disagree, the top-vote candidate (ties break in `consensus_models` order) is still returned as the primary identification with `:low` confidence, and `candidates` carries the full ranked list:
 
 ```ruby
 result = FeatherAi.identify("path/to/bird.jpg", consensus: true)
@@ -82,8 +122,8 @@ result = FeatherAi.identify("path/to/bird.jpg", consensus: true)
 if result.confident?
   puts "Both models agree: #{result.species}"
 else
-  puts "Models disagree:"
-  result.candidates.each { |c| puts "  #{c.common_name} (#{c.species})" }
+  puts "Models disagree — best guess: #{result.species}"
+  result.candidates.each { |c| puts "  #{c[:common_name]} (#{c[:species]}) — #{c[:score]}" }
 end
 ```
 
@@ -91,7 +131,7 @@ Consensus models are configurable:
 
 ```ruby
 FeatherAi.configure do |c|
-  c.consensus_models = ["claude-sonnet-4", "claude-haiku-4"]
+  c.consensus_models = ["claude-sonnet-4-5", "claude-haiku-4-5"]
 end
 ```
 
@@ -120,10 +160,23 @@ All identification calls return a `FeatherAi::Result`:
 | `confidence` | Symbol | `:high`, `:medium`, or `:low` |
 | `confident?` | Boolean | `true` when confidence is `:high` |
 | `region_native?` | Boolean | Whether species is native to the given region |
-| `candidates` | Array | Alternative results when consensus disagrees |
+| `candidates` | Array | Ranked candidate hashes (`common_name`, `species`, `score`); vote-share ranked on consensus disagreement |
 | `photography_tips` | Hash | Lazy-loaded shooting advice |
 | `to_h` | Hash | All fields as a plain hash |
 
+Every result also carries observability data from the LLM call:
+
+| Method | Type | Description |
+|---|---|---|
+| `reasoning` | String | Step-by-step visual analysis the model performed |
+| `model_id` | String | Model that produced the identification |
+| `input_tokens` | Integer | Tokens sent to the model |
+| `output_tokens` | Integer | Tokens received from the model |
+| `cost` | Float | USD cost from RubyLLM's model pricing registry, or `nil` when unknown |
+| `duration_ms` | Integer | Wall-clock time of the LLM call in milliseconds |
+| `source` | Symbol | `:vision`, `:audio`, or `:multimodal` |
+| `consensus_models` | Array | Models used when consensus mode was enabled |
+
 ## Rails Integration
 
 ### Setup
@@ -153,7 +206,7 @@ class Sighting < ApplicationRecord
 end
 ```
 
-The generator adds these columns to the model's table: `common_name`, `species`, `family`, `confidence`, `region_native`.
+The generator adds these columns to the model's table: `common_name`, `species`, `family`, `confidence`, `region_native`, `candidates` (jsonb). On existing tables, `identify!` persists ranked candidates only when a `candidates` column is present — add one in your own migration or skip it.
 
 ### Identifying Records
 
@@ -170,6 +223,61 @@ sighting.confident?   # => true (delegated through result)
 
 `identify!` downloads the attached photo, calls `FeatherAi.identify`, updates the record's identification columns, and returns the `FeatherAi::Result`.
 
+### Corrections
+
+Users or moderators can correct AI identifications. First, run the corrections generator to add the necessary columns:
+
+```bash
+rails generate feather_ai:add_corrections
+# or with a custom model name:
+rails generate feather_ai:add_corrections observation
+```
+
+Then apply corrections to a record:
+
+```ruby
+sighting.correct!(common_name: "Australian Magpie", species: "Gymnorhina tibicen dorsalis")
+
+sighting.corrected?      # => true
+sighting.corrected_at    # => 2026-03-25 12:00:00 UTC
+
+sighting.correction_delta
+# => { common_name: { from: "Western Magpie", to: "Australian Magpie" },
+#      species:     { from: "Gymnorhina tibicen", to: "Gymnorhina tibicen dorsalis" } }
+```
+
+Correctable fields: `common_name`, `species`, `family`, `confidence`, `region_native`.
+
+## Instrumentation
+
+When `ActiveSupport::Notifications` is available (e.g. in Rails), every identification emits an `identify.feather_ai` event. Without ActiveSupport the instrumentation is a no-op.
+
+```ruby
+ActiveSupport::Notifications.subscribe("identify.feather_ai") do |_name, _start, _finish, _id, payload|
+  Rails.logger.info "Identified #{payload[:result].common_name} " \
+                     "with model=#{payload[:model]} in #{payload[:result].duration_ms}ms"
+end
+```
+
+Payload keys: `model`, `location`, `has_image`, `image_count`, `has_audio`, and `result` (the `FeatherAi::Result`).
+
+## Error Handling
+
+FeatherAi raises specific error classes, all inheriting from `FeatherAi::Error`:
+
+- `FeatherAi::ConfigurationError` — invalid or missing configuration (e.g. no image or audio provided)
+- `FeatherAi::IdentificationError` — failure during the LLM identification call
+
+```ruby
+begin
+  FeatherAi.identify("path/to/bird.jpg")
+rescue FeatherAi::ConfigurationError => e
+  # handle bad config
+rescue FeatherAi::IdentificationError => e
+  # handle LLM failure
+end
+```
+
 ## Development
 
 ```bash
@@ -182,6 +290,12 @@ bin/console         # Interactive console with gem loaded
 
 Tests use VCR + WebMock to record and replay LLM responses — no API keys are required to run the test suite.
 
+Use `FeatherAi.reset!` to clear configuration between test examples:
+
+```ruby
+after { FeatherAi.reset! }
+```
+
 ## Thread Safety
 
 `FeatherAi.configuration` is a process-level singleton initialised lazily with `||=`. Under MRI Ruby, the Global VM Lock (GVL) makes this safe in practice. If you use JRuby or Ractors, initialise configuration eagerly at boot time before spawning threads:
@@ -189,8 +303,7 @@ Tests use VCR + WebMock to record and replay LLM responses — no API keys are r
 ```ruby
 # In an initialiser or boot file — before any threads are created
 FeatherAi.configure do |c|
-  c.provider = :anthropic
-  c.model    = "claude-sonnet-4"
+  c.model = "claude-sonnet-4-5"
 end
 ```
 

data/lib/feather_ai/configuration.rb

@@ -3,20 +3,24 @@
 module FeatherAi
   # Configuration object for FeatherAi gem settings.
   class Configuration
-    attr_accessor :provider, :model, :location, :consensus_models, :tips_model, :media_resolution
+    # provider: kept for backwards compatibility; RubyLLM 2 resolves the provider from the model.
+    # media_resolution: Gemini-only (generationConfig.mediaResolution); ignored for other providers.
+    attr_accessor :provider, :model, :location, :consensus_models, :tips_model, :media_resolution, :tools
 
     def initialize
       @provider = :anthropic
-      @model = "claude-sonnet-4"
+      @model = "claude-sonnet-4-5"
       @location = nil
-      @consensus_models = %w[claude-sonnet-4 claude-haiku-4]
-      @tips_model = "claude-haiku-4"
+      @consensus_models = %w[claude-sonnet-4-5 claude-haiku-4-5]
+      @tips_model = "claude-haiku-4-5"
       @media_resolution = :high
+      @tools = []
     end
 
     def initialize_copy(source)
       super
       @consensus_models = source.consensus_models.dup
+      @tools = source.tools.dup
     end
   end
 end

data/lib/feather_ai/consensus.rb

@@ -2,17 +2,18 @@
 
 module FeatherAi
   # Multi-model consensus identification to improve accuracy.
+  # rubocop:disable-next Metrics/ClassLength
   class Consensus
     def initialize(config: FeatherAi.configuration)
       @config = config
       @models = config.consensus_models
     end
 
-    def identify(image = nil, audio = nil, location: nil)
+    def identify(image = nil, audio = nil, location: nil, tools: nil)
       payload = { models: @models, location: location || @config.location }
 
       Instrumentation.instrument("consensus.feather_ai", payload) do
-        results = fetch_results_from_models(image, audio, location)
+        results = fetch_results_from_models(image, audio, location, tools)
         shared_attrs = aggregate_metrics(results)
         result = build_consensus_result(results, shared_attrs)
 
@@ -24,10 +25,12 @@ module FeatherAi
 
     private
 
-    def fetch_results_from_models(image, audio, location)
+    def fetch_results_from_models(image, audio, location, tools)
       @models.map do |model|
         config_for_model = config_with_model(model)
-        Thread.new { Identifier.new(config: config_for_model).identify(image, audio, location: location) }
+        Thread.new do
+          Identifier.new(config: config_for_model).identify(image, audio, location: location, tools: tools)
+        end
       end.map(&:value)
     end
 
@@ -66,19 +69,42 @@ module FeatherAi
         confidence: :high,
         region_native: primary.region_native?,
         model_id: primary.model_id,
+        candidates: primary.candidates,
         photography_tips_loader: tips_loader_for(primary)
       }
     end
 
+    # On disagreement the top-ranked candidate is promoted to the primary
+    # identification (still :low confidence) so consumers always get a usable
+    # name — the full ranked list stays in candidates.
     def disagreed_result_attrs(results)
+      candidates = ranked_candidates(results)
       {
-        common_name: nil,
-        species: nil,
+        common_name: candidates.first[:common_name],
+        species: candidates.first[:species],
         family: calculate_agreed_family(results),
         confidence: :low,
         region_native: false,
         model_id: nil,
-        candidates: results
+        candidates: candidates
+      }
+    end
+
+    # Rank disagreeing identifications by vote share across the consensus
+    # models. Ties break by consensus_models order (sort_by isn't stable).
+    def ranked_candidates(results)
+      results.group_by { |r| r.species&.strip&.downcase }
+             .values
+             .map { |group| vote_candidate(group, results.size) }
+             .sort_by.with_index { |candidate, index| [-candidate[:score], index] }
+    end
+
+    def vote_candidate(group, total)
+      primary = group.first
+      {
+        common_name: primary.common_name,
+        species: primary.species,
+        score: group.size.to_f / total
       }
     end
 

data/lib/feather_ai/identifier.rb

@@ -2,9 +2,9 @@
 
 module FeatherAi
   # Core bird identification using LLM vision and audio transcription.
-  # rubocop:disable Metrics/ClassLength
+  # rubocop:disable-next Metrics/ClassLength
   class Identifier
-    SCHEMA = RubyLLM::Schema.create do
+    SCHEMA = Schematist::Schema.create do
       string :reasoning,
              description: "Step-by-step visual analysis: describe body size, bill shape, " \
                           "plumage, markings, and rule out similar species before identifying"
@@ -13,24 +13,28 @@ module FeatherAi
       string :family, description: "Bird family name"
       string :confidence, description: "Identification confidence: high, medium, or low"
       boolean :region_native, description: "Whether this species is native to the given region"
+      array :candidates, min_items: 1, max_items: 3,
+                         description: "Candidate species ranked most to least likely; " \
+                                      "the first entry must match your identification" do
+        object do
+          string :common_name, description: "Common name of the candidate"
+          string :species, description: "Scientific species name (Genus species)"
+          number :score, minimum: 0, maximum: 1, description: "Relative likelihood of this candidate"
+        end
+      end
     end
 
-    # Approximate mid-2025 rates (USD per 1M tokens).
-    # Use your provider's dashboard for billing accuracy — these are estimates.
-    PROVIDER_RATES = {
-      anthropic: { input: 3.00, output: 15.00 }
-    }.freeze
-
     def initialize(config: FeatherAi.configuration)
       @config = config
     end
 
     # @param image [String, Array<String>, nil] path(s) to image file(s)
     # @param audio [String, nil] path to audio file
-    def identify(image = nil, audio = nil, location: nil)
+    # @param tools [Array, nil] RubyLLM tools (classes or instances) the model may call for grounding
+    def identify(image = nil, audio = nil, location: nil, tools: nil)
       images = normalize_images(image)
       validate_inputs!(images, audio)
-      run_identification(images, audio, location || @config.location)
+      run_identification(images, audio, location || @config.location, Array(tools || @config.tools))
     end
 
     private
@@ -44,12 +48,12 @@ module FeatherAi
       end
     end
 
-    def run_identification(images, audio, effective_location)
+    def run_identification(images, audio, effective_location, tools)
       source = derive_source(images, audio)
       payload = instrumentation_payload(effective_location, images, audio)
 
       Instrumentation.instrument("identify.feather_ai", payload) do
-        response, duration_ms = perform_identification(images, audio, effective_location)
+        response, duration_ms = perform_identification(images, audio, effective_location, tools)
         result = build_result(response, duration_ms, source)
         payload[:result] = result
         result
@@ -72,8 +76,8 @@ module FeatherAi
       }
     end
 
-    def perform_identification(images, audio, location)
-      chat = configure_chat(location)
+    def perform_identification(images, audio, location, tools)
+      chat = configure_chat(location, tools)
       prompt = build_text_prompt(images, audio)
       attachments = images.any? ? images : nil
 
@@ -84,25 +88,25 @@ module FeatherAi
       [response, duration_ms]
     end
 
-    def configure_chat(location)
+    def configure_chat(location, tools)
       chat = RubyLLM.chat(model: @config.model)
-      chat.with_instructions(system_prompt(location))
+      chat.with_instructions(system_prompt(location, tools))
       chat.with_schema(SCHEMA)
-      chat.with_params(**generation_params) if generation_params.any?
+      chat.with_tools(*tools) if tools.any?
+      apply_media_resolution(chat)
       chat
     end
 
-    def generation_params
-      params = {}
-      if @config.media_resolution
-        resolution = "MEDIA_RESOLUTION_#{@config.media_resolution.to_s.upcase}"
-        params[:generationConfig] = { mediaResolution: resolution }
-      end
-      params
+    # generationConfig.mediaResolution is a Gemini request field; other providers reject unknown keys.
+    def apply_media_resolution(chat)
+      return unless @config.media_resolution && chat.provider.slug == "gemini"
+
+      resolution = "MEDIA_RESOLUTION_#{@config.media_resolution.to_s.upcase}"
+      chat.with_provider_options(generationConfig: { mediaResolution: resolution })
     end
 
     def build_result(response, duration_ms, source)
-      parsed = response.content
+      parsed = response.parsed
       Result.new(
         **parsed_identification_attrs(parsed),
         **response_observability_attrs(response, duration_ms, source)
@@ -117,16 +121,27 @@ module FeatherAi
         family: parsed["family"],
         confidence: parsed["confidence"],
         region_native: parsed["region_native"],
+        candidates: normalize_candidates(parsed["candidates"]),
         photography_tips_loader: tips_loader(parsed)
       }
     end
 
+    def normalize_candidates(candidates)
+      Array(candidates).map do |candidate|
+        {
+          common_name: candidate["common_name"],
+          species: candidate["species"],
+          score: candidate["score"]
+        }
+      end
+    end
+
     def response_observability_attrs(response, duration_ms, source)
       {
-        model_id: response.model_id,
-        input_tokens: response.input_tokens,
-        output_tokens: response.output_tokens,
-        cost: compute_cost(response.input_tokens, response.output_tokens),
+        model_id: response.model,
+        input_tokens: response.tokens.input,
+        output_tokens: response.tokens.output,
+        cost: response.cost.total,
         duration_ms: duration_ms,
         source: source
       }
@@ -152,23 +167,17 @@ module FeatherAi
       end
     end
 
-    # Returns a USD cost estimate based on token counts, or nil when the count
-    # is unavailable or the configured provider has no rate table defined here.
-    def compute_cost(input_tokens, output_tokens)
-      return nil if input_tokens.nil? || output_tokens.nil?
-
-      rates = PROVIDER_RATES[@config.provider]
-      return nil if rates.nil?
-
-      ((input_tokens * rates[:input]) + (output_tokens * rates[:output])) / 1_000_000.0
-    end
-
-    def system_prompt(location)
-      base = base_system_prompt
-      return base unless location
-
-      "#{base} The observer is located in #{location} — " \
-        "prioritise species native to that region and consider regional plumage variations."
+    def system_prompt(location, tools = [])
+      prompt = base_system_prompt
+      if location
+        prompt += " The observer is located in #{location} — " \
+                  "prioritise species native to that region and consider regional plumage variations."
+      end
+      if tools.any?
+        prompt += " Use the provided lookup tools to verify species occurrence " \
+                  "for the observer's region before committing."
+      end
+      prompt
     end
 
     def base_system_prompt
@@ -187,7 +196,7 @@ module FeatherAi
     def build_text_prompt(images, audio)
       parts = []
       if audio
-        transcript = RubyLLM.transcribe(audio)
+        transcript = RubyLLM.transcribe(audio).text
         parts << "Bird call/song transcript: #{transcript}"
       end
       parts << identification_prompt(images.size, has_audio: !audio.nil?)
@@ -204,5 +213,4 @@ module FeatherAi
       end
     end
   end
-  # rubocop:enable Metrics/ClassLength
 end

data/lib/feather_ai/photography_tips.rb

@@ -3,7 +3,7 @@
 module FeatherAi
   # Generates photography tips for identified bird species.
   class PhotographyTips
-    SCHEMA = RubyLLM::Schema.create do
+    SCHEMA = Schematist::Schema.create do
       string :time_of_day, description: "Best time of day to photograph this species"
       string :approach, description: "How to approach without disturbing the bird"
       string :settings, description: "Recommended camera settings (shutter speed, aperture, ISO)"
@@ -32,7 +32,7 @@ module FeatherAi
     def fetch_from_llm
       chat = RubyLLM.chat(model: @config.tips_model)
       chat.with_schema(SCHEMA)
-      chat.ask(prompt).content
+      chat.ask(prompt).parsed
     end
 
     def build_tips_hash(parsed)

data/lib/feather_ai/rails/acts_as_sighting.rb

@@ -33,13 +33,15 @@ module FeatherAi
         private
 
         def update_from_result!(result)
-          update!(
+          attrs = {
             common_name: result.common_name,
             species: result.species,
             family: result.family,
             confidence: result.confidence.to_s,
             region_native: result.region_native?
-          )
+          }
+          attrs[:candidates] = result.candidates if has_attribute?("candidates")
+          update!(attrs)
         end
 
         def close_photo_file(photo_file)

data/lib/feather_ai/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module FeatherAi
-  VERSION = "0.3.1"
+  VERSION = "0.5.0"
 end

data/lib/feather_ai.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require "ruby_llm"
-require "ruby_llm/schema"
 
 require_relative "feather_ai/version"
 require_relative "feather_ai/configuration"
@@ -33,11 +32,12 @@ module FeatherAi
     # Identify a bird from image(s) and/or audio.
     # @param image [String, Array<String>, nil] path(s) to image file(s)
     # @param audio [String, nil] path to audio file
-    def identify(image = nil, audio = nil, location: nil, consensus: false)
+    # @param tools [Array, nil] RubyLLM tools (classes or instances) the model may call for grounding
+    def identify(image = nil, audio = nil, location: nil, consensus: false, tools: nil)
       if consensus
-        Consensus.new.identify(image, audio, location: location)
+        Consensus.new.identify(image, audio, location: location, tools: tools)
       else
-        Identifier.new.identify(image, audio, location: location)
+        Identifier.new.identify(image, audio, location: location, tools: tools)
       end
     end
   end

data/lib/generators/feather_ai/templates/migration.rb.tt

@@ -5,5 +5,6 @@ class AddFeatherAiFieldsTo<%= model_name.camelize.pluralize %> < ActiveRecord::M
     add_column :<%= model_name.underscore.pluralize %>, :family, :string
     add_column :<%= model_name.underscore.pluralize %>, :confidence, :string
     add_column :<%= model_name.underscore.pluralize %>, :region_native, :boolean
+    add_column :<%= model_name.underscore.pluralize %>, :candidates, :jsonb
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: feather-ai
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.5.0
 platform: ruby
 authors:
 - Brandyn Britton
@@ -15,14 +15,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '2.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '2.0'
 description: A Ruby gem for identifying birds from photos and audio using RubyLLM.
   Adds multi-modal identification, location-aware results, multi-model consensus,
   and a Rails integration.
@@ -40,6 +40,7 @@ files:
 - ".rspec"
 - ".rubocop.yml"
 - ".ruby-version"
+- CHANGELOG.md
 - CLAUDE.md
 - CODE_OF_CONDUCT.md
 - LICENSE.txt
@@ -83,7 +84,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.8
+rubygems_version: 4.0.10
 specification_version: 4
 summary: Identify birds from photos and audio using LLMs
 test_files: []