bitfab 0.51.9 → 0.57.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f468d991f207bab17dbf93e1d8336e293aa2d759a83a31a8eee7631f1e93daa8
-  data.tar.gz: 4ab69ab42011952d9611a3708ed9f4308760aeb839eaa109ba413a798d5c59d4
+  metadata.gz: 1cd6fe2aaa2d03dea82e1d6a14b63e5a60eec3169c33489ce54795840ab6db8c
+  data.tar.gz: 1637726e4aa1cfa7940ecb65e1b10684b0d1236885a3c1ddd0d3748c8c7a874f
 SHA512:
-  metadata.gz: fcd27044d500aa10fb278940b3665cd54e27c2732e011e828bdd778dc598807960ae84686968957d48fd279f26dc93e2ff72cc1c8ccfa8a91824235548f0b394
-  data.tar.gz: 5c3c831e7c6e30e5ea9f8aaa2a5d63f55f561c831e57caa87504a86b7d73cfc62a98260c943f4024b664c9d6a419bb0b2727e83952dc10f0f62ba4bdb3b496ad
+  metadata.gz: 391ca0dbc155bf7826f8ff4245d305d6d426cd8cd330a2c92564de8eac72ee229d928faad6d74f72f3d7163cf8a950fab1c6dfc96c5ddefdbd215ba0167382a1
+  data.tar.gz: c8a4c5172ecb6f78b309303062167073384bbdd7079c5fc5618200b877526a44bd495f85b2e6120b9a602542f883b80d2872ba2519f81109d3e17d83977b301f

data/README.md

@@ -235,13 +235,13 @@ class OrderPipeline
 end
 ```
 
-Capture uses `TracePoint` only during the root call. It skips Ruby/runtime code, dependency gems, Bitfab SDK internals, blocks, lambdas, and forwarding wrappers. When the root belongs to an installed application gem, that gem is the first-party boundary; neighboring dependency gems remain excluded. It follows recursion, modules, inheritance, `define_method`, and same-thread child Fibers; aliases are recorded and excluded by the name that was invoked. Child Fibers created during capture inherit node policies, trace identity, parenting, and depth limits while retaining separate call stacks. Pre-existing Fibers, `Fiber.new(storage: nil)`, and child threads do not inherit subtree context. When an inner capture ends, child Fibers retain any still-active outer capture; inherited context expires once all enclosing captures end. While a root remains active, hitting `max_spans` stops new capture but allows already-open spans in every Fiber to finish. `max_depth:`, `max_spans:`, `exclude:`, and `include_wrappers:` control the subtree. Automatically captured calls are not replay mock targets. Declare a call with `bitfab_span` when replay must substitute it: its explicit span records once, and automatic descendants continue beneath it.
+Capture uses `TracePoint` only during the root call. It skips Ruby/runtime code, dependency gems, Bitfab SDK internals, blocks, lambdas, and forwarding wrappers. When the root belongs to an installed application gem, that gem is the first-party boundary; neighboring dependency gems remain excluded. It follows recursion, modules, inheritance, `define_method`, and same-thread child Fibers; aliases are recorded and excluded by the name that was invoked. Child Fibers created during capture inherit node policies, trace identity, parenting, and depth limits while retaining separate call stacks. Pre-existing Fibers, `Fiber.new(storage: nil)`, and child threads do not inherit subtree context. When an inner capture ends, child Fibers retain any still-active outer capture; inherited context expires once all enclosing captures end. While a root remains active, hitting a span limit stops new capture but allows already-open spans in every Fiber to finish. `max_depth:` (default `30`), `max_captured_subtree_spans:` (default `512`), `max_spans:` (default `2048`), `exclude:`, and `include_wrappers:` control the subtree. `max_spans:` is a strict cap on the total number of spans one trace root sends. It counts the root span itself, discovered calls, `bitfab_node` calls, nested trace roots, and spans whose capture is on or off, including spans sent without inputs and outputs because the sim plan could not be read. Nothing past it is sent. It must be at least 1, and a lower value raises `ArgumentError`. `max_captured_subtree_spans:` caps discovered calls sent with their inputs and outputs. It does not count `bitfab_node` calls, nested trace roots, or spans the loaded sim plan turned capture off for, so those keep recording past it up to `max_spans:`. Spans sent without inputs and outputs only because the sim plan could not be read still count toward it. Past it, further discovered calls with capture on record nothing. It is never higher than `max_spans:`. Once the plan has loaded, the SDK never builds the inputs or outputs of a span whose capture it turned off. The client starts reading the sim plan when it is built. A `bitfab_trace` root that starts while that first read is still running waits for it, for at most five seconds. Once the first read has finished, whether it succeeded or failed, no later root waits. Nested roots and the calls beneath a root never wait, and nothing waits when the plan is disabled. If that first read fails, spans other than each trace's root span are sent without inputs and outputs until a read succeeds, and they still count toward both limits. Automatically captured calls are not replay mock targets. Declare a call with `bitfab_node` when replay must substitute it. The configured node records once in each enclosing trace. Mixing explicit spans with subtree tracing raises `Bitfab::MixedTracingError`.
 
 When capture ends, already-open automatic calls in suspended Fibers are emitted once with a nil output and `Subtree capture ended before the call returned` as the error, unless a real error was already observed. Their inputs and parenting are retained. The SDK does not resume Fibers, and later resumption cannot add spans to the ended capture. If hot reload moves the root to an unresolved source, the last valid first-party boundary and its exclusions remain in use.
 
 Returning an `Enumerator` keeps capture pending until iteration finishes or raises, including calls already paused in child Fibers. Capture is inactive between the initial return and consumption. Automatic spans record lazy return values as placeholders without iterating them; the traced root records the yielded values.
 
-A nested `bitfab_trace` with a different key starts an independent trace while the outer trace continues recording the overlapping subtree. Both traces have distinct trace and span IDs, including during replay, and the complete structure each root would record alone, so two active roots double span volume in their shared region. A recursive call or nested annotation using the same key stays in the active trace instead of creating another root.
+Each nested `bitfab_trace` starts an independent trace during ordinary capture, including recursion and calls using the same key. Every enclosing trace retains its own recording of the shared subtree. The enclosing span and nested root carry reciprocal trace and span links. Each trace applies its own limits and exclusions. During replay and seed execution, nested annotations remain in the managed trace.
 
 Configure a discovered method with `bitfab_node`. A node never creates a span or trace by itself; the active `bitfab_trace` owns its key, limits, parent, and lifecycle:
 
@@ -258,7 +258,7 @@ end
 
 Nodes support `name:`, `type:`, `capture:`, `test_run_id:`, `mock_on_replay:`, and `finalize:`. `capture: false` omits the method while attaching captured descendants to its nearest visible parent. `finalize:` changes only the recorded output, never the return value. A finalizer failure is recorded on the node without crashing the caller. The before-`def`, inline, and after-`def` forms all preserve method visibility. For external code, use `Bitfab.client.node(MyClass, :method)`.
 
-`bitfab_trace` prepares its project boundary and exclusions when the method is declared, refreshes the boundary if the root definition moves during hot reload, caches canonical source paths, and shares one `TracePoint` dispatcher across active roots on the calling thread. After `max_spans` is exhausted, it stops collecting metadata and values immediately, then stops processing events for that session once its captured calls finish. Each active root still captures and serializes its own copy of every overlapping call, so this remains a discovery tool rather than the production default. Prefer explicit `bitfab_span` declarations on hot paths. Run `mise exec -- ruby -Ilib benchmark/subtree.rb` from this package to measure the no-network caller path on your Ruby build.
+`bitfab_trace` prepares its project boundary and exclusions when the method is declared, refreshes the boundary if the root definition moves during hot reload, caches canonical source paths, and shares one `TracePoint` dispatcher across active roots on the calling thread. After a span limit is reached, it stops reading inputs and outputs immediately, then stops processing events for that session once its captured calls finish. Each active root still captures and serializes its own copy of every overlapping call, so this remains a discovery tool rather than the production default. Prefer explicit `bitfab_span` declarations on hot paths. Run `mise exec -- ruby -Ilib benchmark/subtree.rb` from this package to measure the no-network caller path on your Ruby build.
 
 ### Input/Output Capture
 
@@ -463,7 +463,7 @@ Bitfab.reset!
 ## Thread Safety
 
 - Each thread has its own span context stack (using `Thread.current`)
-- Nested spans only work within the same thread
+- Set `trace_across_threads: true` to propagate explicit spans and replay mocks to submitted threads and optional `concurrent-ruby` thread-pool jobs
 - Background span sending is thread-safe
 
 ## Examples

data/exe/bitfab-seed

@@ -0,0 +1,13 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bitfab"
+
+begin
+  path, args = Bitfab::ReplayCommand.parse(ARGV)
+  registry = Bitfab::ReplayCommand.load_registry(path)
+  Bitfab::SeedCli.run(registry, argv: args)
+rescue => error
+  warn error.message
+  exit 1
+end

data/lib/bitfab/assertion_categories.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require "uri"
+
+module Bitfab
+  class AssertionCategories
+    PATH = "/api/sdk/assertionCategories"
+    JUSTIFICATION_UNSET = Object.new.freeze
+    private_constant :PATH
+    private_constant :JUSTIFICATION_UNSET
+
+    def initialize(http_client)
+      @http_client = http_client
+    end
+
+    def save(title, id: nil, description: nil, justification: JUSTIFICATION_UNSET)
+      payload = {"title" => title}
+      payload["id"] = id unless id.nil?
+      payload["description"] = description unless description.nil?
+      payload["justification"] = justification unless justification.equal?(JUSTIFICATION_UNSET)
+      @http_client.request(PATH, payload)["category"]
+    end
+
+    def get(id)
+      @http_client.get("#{PATH}/#{encode(id)}")["category"]
+    end
+
+    def list
+      @http_client.get(PATH)["categories"]
+    end
+
+    def delete(id)
+      @http_client.request("#{PATH}/#{encode(id)}", {}, method: "DELETE")["category"]
+    end
+
+    private
+
+    def encode(value)
+      URI.encode_www_form_component(value).gsub("+", "%20")
+    end
+  end
+end

data/lib/bitfab/assertions.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+require "uri"
+
+module Bitfab
+  class Traces
+    def initialize(http_client)
+      @http_client = http_client
+    end
+
+    def get_assertions(trace_id)
+      @http_client.get(assertions_path(trace_id))
+    end
+
+    def save_assertions(trace_id, assertions, source: "agent")
+      save_assertions_all([{"traceId" => trace_id, "assertions" => assertions}], source:)
+    end
+
+    def save_assertions_all(updates, source: "agent")
+      return [] if updates.empty?
+
+      @http_client.request("/api/sdk/traces/assertions", {"updates" => updates, "source" => source})["assertions"]
+    end
+
+    def archive_assertions(trace_id, assertion_ids)
+      @http_client.request(assertions_path(trace_id, "/archive"), {"assertionIds" => assertion_ids})["archived"]
+    end
+
+    private
+
+    def assertions_path(trace_id, suffix = "")
+      "/api/sdk/traces/#{URI.encode_www_form_component(trace_id).gsub("+", "%20")}/assertions#{suffix}"
+    end
+  end
+end

data/lib/bitfab/baml.rb

@@ -0,0 +1,208 @@
+# frozen_string_literal: true
+
+require "json"
+
+module Bitfab
+  module BAML
+    ALLOWED_ENV_KEYS = %w[OPENAI_API_KEY].freeze
+    COLLECTOR_FIELDS = %i[logs last usage function_name log_type timing calls selected_call tags
+      selected http_request http_response provider client_name input_tokens output_tokens
+      cached_input_tokens start_time_utc_ms duration_ms url method headers body status].freeze
+
+    module_function
+
+    def load_runtime
+      require "baml" unless defined?(::Baml::DynamicStruct)
+      ::Baml
+    rescue LoadError
+      raise LoadError, "BAML is required for local execution. Install the baml gem matching your generated client."
+    end
+
+    def parameters(source)
+      signature = source.match(/function\s+\w+\s*\(([^)]*)\)\s*->/)&.[](1)
+      return {} unless signature
+
+      depth = 0
+      parts = [+""]
+      signature.each_char do |char|
+        depth += 1 if char == "<"
+        depth -= 1 if char == ">"
+        if char == "," && depth.zero?
+          parts << +""
+        else
+          parts.last << char
+        end
+      end
+      parts.filter_map do |part|
+        match = part.strip.match(/\A(\w+)\s*:\s*(.+)\z/)
+        [match[1], match[2].strip.delete_suffix("?")] if match
+      end.to_h
+    end
+
+    def coerce_inputs(inputs, types)
+      inputs.to_h do |key, value|
+        type = types[key.to_s]
+        [key, (value.is_a?(String) && type) ? coerce(value, type) : value]
+      end
+    end
+
+    def coerce(value, type)
+      case type
+      when "string" then value
+      when "int" then Integer(value, 10)
+      when "float" then Float(value)
+      when "bool"
+        return true if value.downcase == "true"
+        return false if value.downcase == "false"
+        value
+      else
+        parsed = JSON.parse(value)
+        (type.end_with?("[]") && !parsed.is_a?(Array)) ? value : parsed
+      end
+    rescue ArgumentError, JSON::ParserError
+      value
+    end
+
+    def client_name(provider, model)
+      prefix = {"openai" => "OpenAI", "anthropic" => "Anthropic", "google" => "Google"}[provider] || provider.capitalize
+      "#{prefix}_#{model.sub(/^gpt-/, "GPT").tr(".-", "__")}"
+    end
+
+    def prepare_source(source, providers)
+      return source if source.include?("client<llm> OpenAI_")
+
+      clients = providers.flat_map do |provider|
+        provider.fetch("models").map do |model|
+          name = model.fetch("model")
+          temperature = (provider["provider"] == "openai" && name.start_with?("gpt-4.1", "gpt-4o")) ? "\n    temperature 0" : ""
+          "client<llm> #{client_name(provider.fetch("provider"), name)} {\n  provider #{provider.fetch("provider")}\n  options {\n    model #{name.to_json}\n    api_key env.#{provider.fetch("apiKeyEnv")}#{temperature}\n  }\n}"
+        end
+      end
+      "#{clients.join("\n\n")}\n\n#{source}"
+    end
+
+    def run(source, inputs, providers, env_vars)
+      name = source.match(/function\s+(\w+)\s*\(/)&.[](1)
+      raise ArgumentError, "No function found in BAML source" unless name
+
+      baml = load_runtime
+      env = env_vars.select { |key, value| ALLOWED_ENV_KEYS.include?(key) && value.is_a?(String) }
+      runtime = baml::Ffi::BamlRuntime.from_files("/tmp/bitfab_baml_runtime", {"source.baml" => prepare_source(source, providers)}, env)
+      collector = baml::Collector.new(name: "bitfab-collector")
+      response = runtime.call_function(name, coerce_inputs(inputs, parameters(source)), runtime.create_context_manager, nil, nil, [collector], env)
+      types = Module.new
+      result = response.parsed_using_types(types, types, false)
+      [result, serialize_collector(collector)]
+    end
+
+    def serialize_collector(value, depth = 0)
+      return nil if depth > 8
+      return value if value.nil? || value == true || value == false || value.is_a?(Numeric) || value.is_a?(String)
+      return value.map { |item| serialize_collector(item, depth + 1) } if value.is_a?(Array)
+      return value.transform_values { |item| serialize_collector(item, depth + 1) } if value.is_a?(Hash)
+      return serialize_collector(value.json, depth + 1) if value.respond_to?(:json)
+
+      COLLECTOR_FIELDS.each_with_object({}) do |field, result|
+        result[field.to_s] = serialize_collector(value.public_send(field), depth + 1) if value.respond_to?(field)
+      rescue
+        next
+      end
+    rescue
+      nil
+    end
+
+    def enrich_span(collector)
+      log = collector.last
+      calls = log&.calls || []
+      call = calls.find(&:selected) || calls.first
+      body = call&.http_request&.body&.json
+      body = {} unless body.is_a?(Hash)
+      messages = body["messages"]
+      if messages.is_a?(Array)
+        rendered = messages.filter_map do |message|
+          next unless message.is_a?(Hash) && message["role"].is_a?(String)
+          {role: message["role"], content: message["content"].is_a?(String) ? message["content"] : message["content"].to_json}
+        end
+        Bitfab.current_span.set_prompt(rendered.to_json) unless rendered.empty?
+      end
+      usage = collector.usage
+      metadata = {
+        provider: call&.provider,
+        model: body["model"] || call&.http_request&.url&.match(%r{/models/([^/:]+)})&.[](1),
+        inputTokens: usage&.input_tokens || call&.usage&.input_tokens,
+        outputTokens: usage&.output_tokens || call&.usage&.output_tokens,
+        durationMs: log&.timing&.duration_ms
+      }.compact
+      Bitfab.current_span.add_context(metadata) unless metadata.empty?
+    rescue
+      nil
+    end
+
+    class Wrapper
+      attr_reader :collector
+
+      def initialize(client, method_name, on_collector)
+        @client = client
+        @method_name = method_name
+        @on_collector = on_collector
+      end
+
+      def call(*args, **kwargs, &block)
+        @collector = nil
+        begin
+          baml = BAML.load_runtime
+        rescue LoadError
+          return @client.public_send(@method_name, *args, **kwargs, &block)
+        end
+        collector = baml::Collector.new(name: "bitfab-baml-tracing")
+        tracked = @client.with_options(collector:)
+        result = tracked.public_send(@method_name, *args, **kwargs, &block)
+        @collector = collector
+        BAML.enrich_span(collector)
+        begin
+          @on_collector&.call(collector)
+        rescue
+          nil
+        end
+        result
+      end
+    end
+  end
+
+  class Client
+    # Execute a stored BAML prompt locally and return its parsed output.
+    def call(method_name, **inputs)
+      version = @http_client.request("/api/sdk/functions/lookup", {"name" => method_name})
+      raise ArgumentError, "Function #{method_name.inspect} not found. Create it at #{@service_url}/functions" unless version["id"]
+      raise ArgumentError, "Function #{method_name.inspect} has no prompt configured." if version["prompt"].to_s.empty?
+
+      result, collector = BAML.run(version["prompt"], inputs, version["providers"] || [], @env_vars)
+      output = begin
+        result.is_a?(String) ? result : result.to_json
+      rescue
+        result.to_s
+      end
+      @http_client.send_internal_trace(version["id"], {"result" => output, "inputs" => inputs, "rawCollector" => collector, "source" => "ruby-sdk"})
+      result
+    end
+
+    # Add rendered prompt and token metadata to the active span around a BAML method.
+    def wrap_baml(method_or_client, method = nil, on_collector: nil)
+      target = method ? method_or_client : @baml_client
+      actual = method || method_or_client
+      target ||= actual.receiver if actual.is_a?(Method)
+      raise ArgumentError, "baml_client is required for wrap_baml." unless target
+      name = actual.is_a?(Method) ? actual.name : actual
+      raise ArgumentError, "wrap_baml requires a named BAML method." unless name.is_a?(String) || name.is_a?(Symbol)
+
+      BAML::Wrapper.new(target, name, on_collector)
+    end
+  end
+
+  class BitfabFunction
+    # BAML collector wrappers enrich the currently active span under any key.
+    def wrap_baml(*args, **kwargs)
+      @client.wrap_baml(*args, **kwargs)
+    end
+  end
+end