agent-harness 0.38.0 → 0.40.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 07b7223207e544b8728f23b9800ddc774b0c70d16cdd899ccac23550e4a10270
-  data.tar.gz: d004705a37d7d83fa6e27f16f2b3593ffb81a06f30980e4aa699a92bb9f06237
+  metadata.gz: ee00d0e61838facc87483e6bdcac764ed26dc5bb227a15075b4e8f0daa8983d4
+  data.tar.gz: 6034f1733639711ad2b7df6bf9c26da839d45c26d1ddb6b0b4e5ba413854fc8a
 SHA512:
-  metadata.gz: 9d59ea2adf8708b5eb6177fa2314d01e6eb83cb5b8d7748ccc19b7bc07394b1edea040015071df52d53ffb39b1684920092d393a94eeb690058356e05a4eafe5
-  data.tar.gz: d9851948407b188ed95b18413b1eb647245cfe76ea0770e1952972c2a3f83c2a8bd979c90161cc04d4af78241d7dcc83f197451523af233314c28b8d853f9f12
+  metadata.gz: 130a2897522e8b1aa73b2a9cf6094bd24b3b529696445e84356a78faa02540e0c4c2677602e306fca60e1f1a3040e48c04b3ff0a9ca28f144a24de6fc3fa5887
+  data.tar.gz: 647ab39ee27728d28a3441ccbbd84edd3521e7e434a120ac236e411849b22aa431ae7fc6dbe499ec8c525aee6159c9fdba42a5ee8ff7ec329281dd7ea3a4ad33

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.38.0"
+  ".": "0.40.0"
 }

data/CHANGELOG.md

@@ -5,6 +5,20 @@
 * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
 * **auth:** add provider-owned PKCE code-exchange API for Claude OAuth (`AgentHarness::Authentication.exchange_code`). Takes an authorization code plus PKCE verifier (and `redirect_uri`/`client_id`), posts an `authorization_code` grant to the Claude token endpoint, and persists the resulting access/refresh tokens in the native `claudeAiOauth` shape. Adds `exchange_code_supported?` and a `code_exchange` key to `auth_capabilities` ([#266](https://github.com/viamin/agent-harness/issues/266)).
 
+## [0.40.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.39.0...agent-harness/v0.40.0) (2026-09-25)
+
+
+### Features
+
+* Provide Native Embedding Support (RDR-072) ([#439](https://github.com/viamin/agent-harness/issues/439)) ([551378d](https://github.com/viamin/agent-harness/commit/551378d11627a5227059032f0b7418dc166f54e9))
+
+## [0.39.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.38.0...agent-harness/v0.39.0) (2026-09-25)
+
+
+### Features
+
+* design: Define the Provider-Neutral API Execution Contract (RDR-072) ([#438](https://github.com/viamin/agent-harness/issues/438)) ([ae2fc6b](https://github.com/viamin/agent-harness/commit/ae2fc6bafad54ac35bb9d30a32d253c68a7396f2))
+
 ## [0.38.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.37.5...agent-harness/v0.38.0) (2026-09-24)
 
 

data/README.md

@@ -39,6 +39,71 @@ puts response.output
 response = AgentHarness.send_message("Explain this code", provider: :cursor)
 ```
 
+## Native Embeddings
+
+`AgentHarness.embed` sends a whole input batch through RubyLLM and returns one
+vector per input, in the same order. Credentials, endpoint, headers, timeout,
+and retry limits are request-local; they do not change `RubyLLM.config` or the
+CLI/subscription provider configuration.
+
+```ruby
+result = AgentHarness.embed(
+  inputs: ["first document", "second document"],
+  model: "text-embedding-3-small",
+  dimensions: 512,
+  endpoint: "https://api.openai.com/v1", # optional OpenAI-compatible base URL
+  credentials: {api_key: ENV.fetch("EMBEDDING_API_KEY")},
+  headers: {"X-Tenant-ID" => tenant.external_id},
+  timeout: 30,
+  max_attempts: 3,
+  cancellation: -> { request_cancelled? }
+)
+
+result.vectors # one vector for each input
+result.usage   # { input_tokens: 42 }
+```
+
+`credentials` may also be the API key string. Extra headers cannot replace the
+`Authorization` header; change credentials explicitly instead. `max_attempts`
+includes the initial request. Agent Harness performs the only retry loop: 429,
+timeout/connection, and transient 5xx failures are retried up to that bound,
+while 401 and 403 responses fail immediately. The harness honors `Retry-After`
+within its bounded retry policy. A cancellation callable is checked immediately
+before every physical HTTP attempt.
+
+Usage is the provider-reported total for the complete batch. When the provider
+omits usage, `result.usage[:input_tokens]` remains `nil`. The harness does not
+estimate usage or allocate a batch total across vectors, so
+`result.per_vector_usage` is always `nil`.
+
+Authentication failures raise `AgentHarness::AuthenticationError`, exhausted
+rate limits raise `AgentHarness::RateLimitError`, timeouts raise
+`AgentHarness::TimeoutError`, transient provider failures raise
+`AgentHarness::ProviderError`, cancellations raise
+`AgentHarness::CancelledError`, and incomplete or invalid vector batches raise
+`AgentHarness::MalformedEmbeddingError`. Empty input returns an empty result
+without contacting the provider.
+
+### Migrating from Paid transport patches
+
+This operation replaces downstream host/container embedding transport
+extensions for OpenAI-compatible direct and proxy endpoints. After adopting an
+agent-harness release containing this capability:
+
+1. Run the downstream embedding contract suite against both the direct provider
+   and proxy endpoint, including tenant-specific credentials and headers.
+2. Verify the released gem artifact includes `AgentHarness.embed` and record the
+   passing artifact version or digest. Issue closure or a Git tag alone is not
+   release evidence.
+3. Switch only the embedding call site to `AgentHarness.embed`; leave unrelated
+   chat, schema, CLI, and subscription paths unchanged.
+4. Remove the downstream embedding request/parser/retry patch so Agent Harness
+   owns the single bounded retry loop. Keep durable workflow recovery and
+   accounting in the downstream application.
+
+The runtime dependency is Ruby 3.2 or newer and RubyLLM 2.x. No Rails database
+or RubyLLM persistence tables are required for this plain-Ruby operation.
+
 ## Configuration
 
 ### Ruby DSL
@@ -738,6 +803,12 @@ Health checks run five steps per provider: registration, CLI availability, authe
 
 ## Development
 
+The proposed provider-neutral API execution boundary for embeddings, chat,
+structured output, usage, and optional persistence is documented in the
+[Provider-Neutral API Execution Contract](docs/provider-neutral-api-execution-contract.md).
+It is a docs-only design boundary; capability support requires the release
+evidence described there.
+
 ```bash
 # Install dependencies
 bin/setup

data/docs/provider-neutral-api-execution-contract.md

@@ -0,0 +1,532 @@
+# Provider-Neutral API Execution Contract
+
+This document records the design investigation for
+[RDR-072](https://github.com/viamin/paid/blob/01672a9992853e89bcbd272b720ac9f1da73cba0/docs/rdrs/RDR-072-api-conversation-delegation.md).
+It is the upstream contract for incremental embedding, chat, structured-output,
+usage, and optional conversation-persistence work.
+
+## Status and rollout boundary
+
+This is a design contract, not a claim that the described API is implemented.
+RDR-072's rollout guard is **docs-only now**, so this change adds no runtime
+behavior or dependency. Each capability below needs its own failing-first
+contract tests, implementation, release evidence, and downstream adoption
+evidence before a caller enables it.
+
+Existing CLI and subscription behavior remains the default. Existing
+`TextTransport`, `OpenAICompatibleTransport`, `Conversation`, and `Response`
+interfaces remain available, but they do not yet satisfy this contract. A
+caller must not infer support from a gem version or issue closure.
+
+Normative words such as MUST and MUST NOT describe the future public boundary.
+
+## Ownership boundary
+
+AgentHarness owns protocol translation and one bounded provider request:
+
+- capability discovery and explicit unsupported outcomes;
+- provider-specific request and response translation;
+- request retries within caller-supplied limits;
+- normalized stream events, results, errors, and attempt reports; and
+- stable attempt and tool-call identities at its public boundary.
+
+The caller owns application authority and durable policy:
+
+- tenant and actor identity, authorization, and visible tools;
+- candidate order, credentials, allowed models, and fallback notices;
+- budgets, cancellation initiation, runner changes, and workflow recovery;
+- durable usage attribution and idempotent ingestion of attempt reports;
+- tool confirmation and side-effect reconciliation; and
+- application conversation/message identity and audit records.
+
+The harness MUST NOT select an unlisted fallback, borrow global credentials,
+change authentication mode, execute a tool, or replay a completed tool result.
+Keeping the caller's chat loop over this normalized transport is an acceptable
+final architecture.
+
+## Common execution request
+
+All operations use an immutable, request-local configuration. The eventual
+Ruby API MAY use value objects rather than hashes, but it MUST expose these
+semantics:
+
+```ruby
+request = {
+  request_id: "paid-generation-018f...", # caller-generated, stable on redelivery
+  operation: :chat,                       # :chat, :embedding, or :schema
+  candidates: [                           # ordered; first entry is initial
+    {
+      provider: :anthropic,
+      model: "claude-sonnet-4-5",
+      protocol: :messages,                # optional verified protocol
+      authentication_mode: :api_key,
+      endpoint: "https://llm-proxy.example/v1",
+      headers: {"X-Tenant-Route" => "tenant-123"},
+      credentials: {api_key: anthropic_secret}
+    },
+    {
+      provider: :openai,
+      model: "gpt-5",
+      protocol: :responses,
+      authentication_mode: :api_key,
+      endpoint: "https://api.openai.com/v1",
+      headers: {},
+      credentials: {api_key: openai_secret}
+    }
+  ],
+  fallback: {on_error_categories: [:transient]},
+  timeout: {connect_seconds: 5, read_seconds: 60},
+  retry: {max_attempts: 3, base_delay_seconds: 0.25, max_delay_seconds: 2},
+  cancellation: cancellation_token,
+  metadata: {tenant_id: "tenant-123", workflow_id: "workflow-456"}
+}
+```
+
+`candidates` MUST contain at least one complete candidate record. Its first
+entry is the initial candidate; later entries are the only authorized fallback
+order. A request that does not authorize fallback supplies a one-entry array.
+`fallback.on_error_categories` is the caller-selected allowlist of error
+categories that permit advancing to the next entry; an absent or empty
+allowlist disables fallback. Candidate records, their order, and the allowlist
+are immutable for the lifetime of the request. The observer described below is
+a request-local callback or equivalent API argument because callbacks are not
+part of a serializable request document.
+
+`request_id` identifies one logical operation. Every physical outbound request
+gets a distinct `attempt_id`. Redelivering an already reported attempt retains
+its `attempt_id`; initiating another outbound request does not.
+
+Credentials, endpoint, headers, timeouts, retry limits, and cancellation are
+request-local. Implementations MUST prevent concurrent requests from observing
+one another's credentials or headers. They MUST reject reserved header
+overrides that would conflict with the selected protocol's authentication.
+Logs and errors MUST NOT contain credentials, authorization headers, message
+bodies, tool arguments, or full provider responses.
+
+The selected candidate's provider, model, protocol, endpoint, and authentication
+mode MUST be the values actually used. The harness MUST return a configuration
+or unsupported outcome instead of silently substituting any of them.
+
+## Capability discovery and unsupported outcomes
+
+Support is declared per operation, provider, protocol, model when known, and
+authentication mode. It is not a single provider-wide Boolean. The discovery
+result has one of three states:
+
+- `supported`: the requested combination is verified;
+- `unsupported`: it is known not to work, with a stable reason; or
+- `unknown`: it has not been verified and MUST NOT be routed as supported.
+
+Stable unsupported reasons initially include:
+
+- `operation_not_supported`;
+- `protocol_not_supported`;
+- `authentication_mode_not_supported`;
+- `custom_endpoint_not_supported`;
+- `custom_headers_not_supported`;
+- `streaming_not_supported`;
+- `tools_not_supported`;
+- `structured_output_not_supported`; and
+- `state_restoration_not_supported`.
+
+An unsupported result is a normal, non-retryable outcome carrying the requested
+scope and reason. It is not a generic provider failure. Callers MAY choose
+another candidate only from their own ordered list.
+
+## Normalized chat contract
+
+### Messages
+
+The input is an ordered array. Each message has a stable caller-owned `id`, one
+of `system`, `user`, `assistant`, or `tool` as its `role`, and an ordered
+`content` array. Content parts initially support `text`; unimplemented media
+parts produce an explicit unsupported outcome. Assistant messages MAY contain
+tool calls. Tool messages MUST reference exactly one `tool_call_id`.
+
+Provider-specific wire roles, system-message placement, and content block
+shapes stay behind the harness boundary. The harness MUST preserve ordering,
+empty assistant text accompanying tool calls, and unknown versus absent data.
+
+### Tools
+
+Tool definitions contain a caller-owned name, description, and JSON Schema
+input definition. A normalized tool call contains:
+
+```ruby
+{
+  id: "toolcall_018f...",       # stable harness ID
+  provider_id: "call_abc",      # optional provider correlation only
+  name: "search_documents",
+  arguments_json: "{\"query\":\"Ruby\"}"
+}
+```
+
+The stable `id` is generated before the call is exposed or persisted and MUST
+survive export/import. `provider_id` is not a durable application identifier.
+Arguments remain JSON text until parsing succeeds; malformed arguments produce
+a classified parse failure and are never executed by the harness. Tool results
+reference the stable ID. Redelivery or restoration MUST skip a tool call when a
+completed result for that ID exists.
+
+The transport never decides approval and never executes application tools.
+
+### Streaming
+
+The stream is ordered and contains these normalized event types:
+
+- `response_started` with request and attempt identity;
+- `text_delta` with appended text;
+- `tool_call_started`, `tool_call_delta`, and `tool_call_completed`;
+- `usage_updated` when defensible cumulative or delta usage is available;
+- `response_completed` with the final normalized result; and
+- `response_failed` or `response_cancelled` with the partial result.
+
+Every event carries `request_id`, `attempt_id`, and a monotonically increasing
+`sequence`. A terminal event occurs exactly once per attempt. Callback failure
+requests cancellation and surfaces as a caller error; it MUST NOT be converted
+to provider success.
+
+If a stream ends after emitting content, the terminal result has
+`status: :partial`, retains received text and completed tool calls, and carries
+the classified error. Incomplete tool calls are marked incomplete and MUST NOT
+be executed. A partial attempt is never transparently retried: already emitted
+content cannot be withdrawn. The caller must explicitly decide whether a new
+attempt is safe and how to represent the abandoned partial message.
+
+### Result
+
+A completed chat or schema result contains:
+
+```ruby
+{
+  request_id: "paid-generation-018f...",
+  status: :succeeded, # :succeeded, :partial, :failed, or :cancelled
+  provider: :anthropic,
+  model: "claude-sonnet-4-5",
+  authentication_mode: :api_key,
+  content: "...",
+  parsed: nil,
+  tool_calls: [],
+  finish_reason: :stop,
+  attempts: [],
+  usage: {input_tokens: 12, output_tokens: 8, total_tokens: 20},
+  provider_request_id: nil,
+  error: nil
+}
+```
+
+`usage` aggregates only the attached attempt reports. Missing provider values
+remain `nil`; they are not coerced to zero. `provider_request_id` is optional
+diagnostic correlation and is not the accounting identity.
+
+## Structured-output contract
+
+A schema operation adds a JSON Schema and optional schema name to the chat
+request. The capability check MUST cover the selected provider/model/protocol
+and requested schema mode. JSON-only mode is not equivalent to schema-enforced
+output and MUST be reported separately.
+
+On success, `content` preserves provider JSON text and `parsed` contains the
+parsed Ruby value. Invalid JSON or schema mismatch is a classified,
+non-transient result with the original text retained. The harness MUST NOT
+silently strip fences, repair content, issue a second model call, or switch to
+a different credential to make parsing succeed.
+
+## Embedding contract
+
+An embedding request supplies one string or an ordered array of strings,
+provider/model candidate, and optional dimensions. Its result preserves input
+order and contains dense vectors, the model actually used, per-attempt usage,
+and optional provider request correlation. One input returns one vector;
+multiple inputs return an equal-length vector array.
+
+Empty batches, unsupported dimensions, unsupported input media, and batch-size
+limits are explicit configuration or unsupported outcomes. The first release
+need not cover multimodal or provider-side batch APIs. Vector persistence and
+similarity search remain caller responsibilities.
+
+## Attempts, usage, retries, and cancellation
+
+Each physical provider request produces an attempt report, including failed and
+cancelled attempts:
+
+```ruby
+{
+  attempt_id: "attempt_018f...",
+  request_id: "paid-generation-018f...",
+  number: 2,
+  provider: :anthropic,
+  model: "claude-sonnet-4-5",
+  status: :failed,
+  started_at: "2026-09-25T12:00:00Z",
+  finished_at: "2026-09-25T12:00:01Z",
+  usage: {input_tokens: nil, output_tokens: nil, total_tokens: nil},
+  cost: nil,
+  provider_reported: false,
+  error: {category: :transient, code: :service_unavailable}
+}
+```
+
+Attempt reports are delivered both with the final result and through an
+observer so durable accounting can persist an attempt even when no message is
+created. Callers deduplicate on `attempt_id`. Cost identifies its source as
+provider-reported or harness-estimated; unknown cost remains `nil`.
+
+Only errors classified `transient` are eligible for bounded request retry:
+connection failure, timeout before a partial stream, rate limit, server error,
+service unavailable, and overload. Authentication, authorization, billing,
+invalid request, unsupported capability, invalid schema, context length,
+configuration, and cancellation are non-retryable.
+
+The supplied `max_attempts` includes the first outbound request and is the total
+physical-attempt budget across all candidates; changing candidates does not
+reset it. It MUST be a positive integer; the harness rejects zero, negative, or
+non-integer values as configuration errors before making an outbound request.
+Delay and provider `retry-after` handling remain within the supplied bounds.
+Cancellation is checked before an attempt, during backoff, while reading a
+stream, and before returning success. It stops further attempts and returns a
+cancelled terminal outcome with any partial usage. Cancellation does not prove
+that the provider stopped processing or billing the request.
+
+The harness owns the complete attempt sequence; there is exactly one retry
+owner. RubyLLM's Faraday retry middleware resends the current candidate inside
+the transport before a classified error can return to the harness, so
+middleware-driven retries cannot honor the fallback-first order below and would
+spend the shared budget on one candidate invisibly to the observer. An adapter
+backed by RubyLLM MUST therefore disable its retry middleware with
+`max_retries: 0`, overriding the default three retries, so that one adapter
+call performs exactly one physical outbound request. The harness then issues
+one adapter call per attempt — initial, retry, or fallback — and applies the
+shared `max_attempts` budget, cancellation checks, backoff, and provider
+`retry-after` between calls. No component may retry beneath the harness, and
+the harness MUST NOT delegate this sequencing to middleware or another nested
+loop. Existing conductor retry/provider switching is not part of an API
+request's internal retry budget and MUST be disabled or bypassed for a
+migrated scope.
+
+## Caller-controlled fallback
+
+Fallback candidates are the entries after the first entry in the request's
+ordered `candidates` array. Each is a complete request-local candidate record,
+including credentials and endpoint/header overrides. A candidate change is a
+new caller-authorized selection, not a hidden retry. Before advancing, the
+harness invokes the request-local observer with the current and next candidate
+identities and the classified error, so the caller can cancel the change or
+issue a notice. The harness reports every candidate attempt.
+
+Fallback is allowed only for caller-selected error categories. It never occurs
+after a partial stream without a new explicit caller decision, never crosses
+authentication modes implicitly, and never replays completed tools. The
+harness MUST NOT use RubyLLM global fallback or global credential configuration
+where it could exceed this list or leak request-local configuration.
+
+Fallback takes precedence over retry when both are eligible. After a failed
+attempt, the harness MUST apply this order:
+
+1. Stop if cancellation was requested or any partial stream was exposed.
+2. If the budget has capacity, the error category is in
+   `fallback.on_error_categories`, and another candidate remains, notify the
+   observer and advance to that candidate.
+3. Otherwise, retry the current candidate only when the error is retryable and
+   the shared `max_attempts` budget has capacity.
+4. Otherwise, return the terminal error.
+
+The outbound request after either advancing or retrying consumes one physical
+attempt from the same budget. Candidates are never revisited, and a failed
+observer notification or observer cancellation stops before that request.
+Consequently, the example request attempts Anthropic once and then OpenAI after
+an eligible `transient` failure; only OpenAI can consume the remaining
+same-candidate retry budget.
+
+## Error contract
+
+Every terminal error exposes a stable category and code, retry eligibility,
+provider/model identity, optional sanitized status and request ID, and partial
+result/usage where available. Categories are:
+
+| Category | Examples | Request retry |
+| --- | --- | --- |
+| `transient` | timeout, connection failure, 429, 5xx, overload | bounded |
+| `authentication` | missing or rejected credential | never |
+| `authorization` | credential lacks access | never |
+| `billing` | inactive billing account or payment required | never |
+| `configuration` | invalid endpoint/header/model or local configuration | never |
+| `invalid_request` | malformed or semantically invalid provider request | never |
+| `context_length` | request exceeds the model context window | never |
+| `unsupported` | operation or option not implemented | never |
+| `invalid_response` | malformed JSON, schema/tool parse failure | never |
+| `cancelled` | caller cancellation | never |
+| `caller` | callback or local input failure | never |
+| `unknown` | provider failure that cannot be classified confidently | never |
+
+Adapters MUST use the following category and code mappings. A provider-specific
+status or error name may be retained as sanitized metadata, but MUST NOT replace
+these values or change fallback eligibility.
+
+| Failure | Category | Code |
+| --- | --- | --- |
+| Connection failure | `transient` | `connection_failed` |
+| Timeout before a partial stream | `transient` | `timeout` |
+| Rate limit | `transient` | `rate_limited` |
+| Provider server error | `transient` | `server_error` |
+| Service unavailable | `transient` | `service_unavailable` |
+| Provider overload | `transient` | `overloaded` |
+| Missing or rejected credential | `authentication` | `invalid_credential` |
+| Credential lacks access | `authorization` | `permission_denied` |
+| Billing account inactive or payment required | `billing` | `billing_unavailable` |
+| Invalid provider request | `invalid_request` | `invalid_request` |
+| Unsupported capability or option | `unsupported` | `unsupported_capability` |
+| Structured output violates its schema | `invalid_response` | `invalid_schema` |
+| Request exceeds the model context window | `context_length` | `context_length_exceeded` |
+| Invalid endpoint, header, model, or local request configuration | `configuration` | `invalid_configuration` |
+| Caller cancellation | `cancelled` | `cancelled` |
+
+If a failure cannot be mapped confidently, the adapter MUST return `unknown` /
+`unclassified_provider_error`, which is non-retryable, rather than guessing a
+retryable category. A timeout or other error after a partial stream retains its
+mapped category and code, but its per-error retry eligibility is `false` and the
+fallback rule above prohibits automatic replay.
+
+Provider error classes stay internal. Existing `ProviderError` is too broad to
+drive this policy and existing `ErrorTaxonomy` was designed for CLI provider
+switching, so neither is the future API retry contract without an explicit
+mapping and contract tests.
+
+## State export, import, and restart safety
+
+Plain Ruby state export is a versioned data document, not a serialized Ruby
+object. It contains normalized messages, stable message/tool-call IDs, completed
+tool results, pending tool calls and decisions, request IDs, and attempt reports.
+It never contains credentials, callbacks, open streams, tool implementations,
+authorization decisions, or mutable provider client objects.
+
+Import validates the version and structure and reconstructs the normalized
+conversation. The caller must re-supply tools, authorization, candidates,
+credentials, observers, retry limits, and cancellation. Unknown versions fail
+explicitly. Export/import round-trip tests must prove that completed tool IDs
+are not re-executed and unknown usage remains unknown.
+
+The restart guarantee is limited to checkpoints. A saved provider response or
+tool result is skipped after restoration. A process crash after an external
+tool side effect but before its result is durably saved can execute the tool
+again. No transcript format can close that window. Tools therefore need a
+caller-owned idempotency key derived from the stable tool-call ID, or explicit
+reconciliation before replay. Provider requests have the same ambiguity when a
+response is lost after the provider accepted it.
+
+## RubyLLM 2.0 mapping and alternatives
+
+The investigation used RubyLLM 2.0.0. RubyLLM remains an implementation detail;
+the harness public API MUST NOT expose its classes or private persistence
+records.
+
+| Contract area | RubyLLM 2.0 mapping | Decision or gap |
+| --- | --- | --- |
+| Chat/protocols | `RubyLLM.chat`, messages, tools, stream callbacks | Candidate adapter; normalize all values and errors |
+| Schema | `with_schema`, `response.parsed`, model capability registry | Candidate; distinguish enforced schema from JSON mode |
+| Embeddings | `RubyLLM.embed` and normalized vectors/usage | Candidate first capability; persistence remains in Paid |
+| Custom headers | `with_headers` | Candidate; contract-test merging and secret redaction |
+| Endpoint/credentials | provider configuration | Global mutable configuration is unsuitable; require request-local isolation or an upstream-supported client boundary |
+| Retries | Faraday retry middleware, default three retries | Disable with `max_retries: 0` so one call is one physical attempt; the harness sequences bounded retry and fallback itself; never nest |
+| Fallback | `with_fallbacks` and callbacks | Harness sequences candidates per call; `with_fallbacks` usable only if exact candidates, credentials, and per-advance observer control are preserved |
+| Cancellation | chat cancellation and `CancelledError` | Adapt to the common token and retain partial stream state |
+| Attempt usage | `usage.ruby_llm` per physical attempt | Useful facts, but the public payload has no stable attempt ID; harness must add one |
+| Plain Ruby resume | transcript can be reconstructed manually | No documented state export/import API; implement normalized export/import outside RubyLLM |
+| Rails resume | `acts_as_chat` transcript plus supporting records | Technically restart-safe at checkpoints; at-least-once side effects remain |
+
+### Optional Rails supporting tables
+
+RubyLLM 2.0 owns `ruby_llm_models`, `ruby_llm_tool_calls`,
+`ruby_llm_usages`, and `ruby_llm_batches`. Only the middle two are candidates
+for early chat adoption; batches are out of scope, while the model registry is
+needed only if capability/pricing lookup uses it.
+
+`ruby_llm_tool_calls.tool_call_id` is unique and can preserve the provider tool
+request/result link. It is acceptable only after tests prove that the harness's
+stable tool ID maps without collision and Paid tenant scoping is enforced
+through the owning application message/chat. The supporting table has no
+standalone tenant column, so every read/write path must join through a
+tenant-owned record and cross-tenant tests are mandatory.
+
+`ruby_llm_usages` records one physical attempt and links polymorphically to the
+application chat and optionally a message. It preserves failed/cancelled usage,
+but its row ID and record class are private and its public instrumentation has
+no stable attempt ID. Paid cannot use that row ID as the cross-system accounting
+key. The harness-generated `attempt_id` must be persisted alongside Paid's
+ledger, with an upstream-supported metadata column or mapping required before
+adopting this table as the authoritative delivery source.
+
+RubyLLM has no application-owned `Model`, `ToolCall`, `Usage`, or `Batch` model;
+its record classes are explicitly implementation details. Paid keeps its chat
+and message domain records, tenant ownership, message links, audit actor, and
+authorization. A Rails adoption requires generated-migration review, backup,
+production-snapshot rehearsal, backfill verification, rollback rehearsal, and
+historical/pending-conversation tests. Reverting the gem is not a data rollback.
+
+### Persistence alternatives
+
+1. **Retain Paid persistence and loop over normalized transport.** Lowest
+   migration risk and the current recommendation. Implement plain Ruby
+   export/import and stable IDs in the harness.
+2. **Use RubyLLM tool-call and usage tables selectively.** Potentially removes
+   bookkeeping, but only after stable-attempt mapping, tenant-scoped access,
+   audit, and migration tests are complete.
+3. **Delegate the full loop and all supporting tables.** Not recommended now.
+   It does not yet demonstrate reduced maintenance across both repositories and
+   increases migration and recovery coupling.
+
+The state investigation is therefore positive for checkpoint-based Rails
+restoration and normalized plain Ruby reconstruction, and negative for
+exactly-once recovery or an off-the-shelf plain Ruby export/import mechanism.
+
+## Current harness gaps and incremental delivery
+
+The current transports already normalize basic text, tool calls, token totals,
+and some HTTP errors. They do not provide request-local custom headers,
+per-request timeout/retry/cancellation, structured output, embeddings, stable
+attempt IDs, per-attempt usage, partial terminal results, state export/import,
+or explicit capability outcomes. `Conversation` stores in-memory history and
+provider formatters but has no serialization contract.
+
+Ship capabilities independently in this order:
+
+1. common values, capability discovery, classified errors, and attempt reports;
+2. embeddings for verified operation/provider/custom-endpoint scopes;
+3. normalized non-streaming and streaming chat transport;
+4. structured output for verified model/protocol combinations;
+5. plain Ruby state round trips; and
+6. optional Rails persistence evaluation, then loop evaluation.
+
+Each implementation issue starts with failing contract tests for request-local
+credential isolation, custom endpoints/headers, unsupported outcomes,
+classified errors, bounded non-nested retries, cancellation, partial streams,
+stable IDs, and unknown usage where applicable. Provider-specific fixtures and
+types stay behind the harness boundary.
+
+## Compatibility and release evidence
+
+AgentHarness currently supports Ruby 3.2 and later and must remain usable as a
+plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later, but adding it
+would introduce Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk
+runtime dependencies. A capability issue must measure and publish the resolved
+dependency set and test the harness minimum Ruby version before adoption.
+
+Rails and Active Record remain optional. Requiring `agent_harness` in a process
+without Rails MUST NOT load Active Record, connect to a database, or require
+supporting tables. Rails integration belongs behind an optional require and
+adapter.
+
+For every capability, release evidence must name:
+
+- the first published, installable agent-harness version containing it;
+- supported operation/provider/protocol/authentication combinations;
+- upstream contract and integration test runs on the minimum Ruby version;
+- verification of request-local secrets and absence of secret-bearing logs;
+- retry/cancellation and attempt-accounting test evidence;
+- the exact Paid and agent-image versions that consume the release; and
+- retained paths and follow-up issues for combinations not migrated.
+
+Paid issue `viamin/paid#4014` should receive this compatibility result, the
+state-restoration conclusion, the stable-ID gap, and the per-capability release
+evidence. Downstream adoption cannot proceed from this design issue closing or
+from a Git tag alone.

data/lib/agent_harness/embedding_adapter.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+require "faraday"
+require "faraday/net_http"
+require "json"
+
+module AgentHarness
+  # Builds a request-local Faraday adapter for headers and cancellation.
+  module EmbeddingAdapter
+    module_function
+
+    def build(headers:, cancellation: nil)
+      Class.new(Faraday::Adapter::NetHttp) do
+        define_method(:call) do |env|
+          raise CancelledError, "Embedding request cancelled" if cancellation&.call
+
+          env.request_headers.update(headers)
+          super(env).on_complete { |response| EmbeddingAdapter.order_rows(response) }
+        end
+      end
+    end
+
+    def order_rows(response)
+      payload = JSON.parse(response.body)
+      rows = payload["data"]
+      return unless rows.is_a?(Array)
+
+      indices = rows.map { |row| row["index"] if row.is_a?(Hash) }
+      unless indices.all?(Integer) && indices.sort == (0...rows.length).to_a
+        raise MalformedEmbeddingError, "Provider returned invalid embedding indices"
+      end
+
+      payload["data"] = rows.sort_by { |row| row.fetch("index") }
+      response.body = JSON.generate(payload)
+    rescue JSON::ParserError
+      nil
+    end
+  end
+end

data/lib/agent_harness/embedding_result.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module AgentHarness
+  # Normalized result returned by AgentHarness.embed.
+  class EmbeddingResult
+    attr_reader :vectors, :model, :usage, :attempts
+
+    def initialize(vectors:, model:, input_tokens: nil, attempts: [])
+      @vectors = vectors
+      @model = model
+      @usage = {input_tokens: input_tokens}.freeze
+      @attempts = attempts.freeze
+    end
+
+    # Batch usage is never guessed or divided among individual vectors.
+    def per_vector_usage
+      nil
+    end
+  end
+end

data/lib/agent_harness/embeddings.rb

@@ -0,0 +1,257 @@
+# frozen_string_literal: true
+
+require "ruby_llm"
+require "securerandom"
+require "time"
+
+module AgentHarness
+  # Provider-neutral embedding execution backed by RubyLLM.
+  class Embeddings
+    DEFAULT_TIMEOUT = 300
+    DEFAULT_MAX_ATTEMPTS = 3
+    RETRY_BASE_DELAY = 0.25
+    RETRY_MAX_DELAY = 2.0
+    CANCELLATION_POLL_INTERVAL = 0.05
+    TRANSIENT_ERRORS = [RateLimitError, TimeoutError].freeze
+
+    def initialize(model:, credentials:, endpoint: nil, headers: {}, timeout: DEFAULT_TIMEOUT,
+      max_attempts: DEFAULT_MAX_ATTEMPTS, cancellation: nil, observer: nil, request_id: nil)
+      @model = model
+      @api_key = credential(credentials, :api_key)
+      @endpoint = endpoint
+      @headers = headers.to_h.transform_keys(&:to_s).freeze
+      @timeout = timeout
+      @max_attempts = max_attempts
+      @cancellation = cancellation
+      @observer = observer
+      @request_id = request_id || SecureRandom.uuid
+      validate!
+    end
+
+    def call(inputs:, dimensions: nil)
+      inputs = Array(inputs)
+      return EmbeddingResult.new(vectors: [], model: @model) if inputs.empty?
+
+      execute(inputs, dimensions)
+    end
+
+    private
+
+    def execute(inputs, dimensions)
+      attempts = []
+
+      1.upto(@max_attempts) do |number|
+        check_cancellation!
+        result = attempt(inputs, dimensions, number, attempts)
+        return result
+      rescue RateLimitError, TimeoutError, ProviderError => e
+        raise unless retryable?(e) && number < @max_attempts
+
+        wait_before_retry(e, number)
+      end
+    end
+
+    def attempt(inputs, dimensions, number, attempts)
+      started_at = Time.now.utc
+      embedding = request_embedding(inputs, dimensions)
+      validate_result!(embedding.vectors, inputs.length)
+      report = success_report(embedding, number, started_at)
+    rescue CancelledError => e
+      record_attempt(attempts, failure_report(e, number, started_at, :cancelled, :cancelled))
+      raise
+    rescue AuthenticationError, AuthorizationError, RateLimitError, TimeoutError, ProviderError => e
+      record_attempt(attempts, failure_report(e, number, started_at, *error_classification(e)))
+      raise
+    rescue NoMethodError, TypeError => e
+      error = MalformedEmbeddingError.new("Malformed embedding response", original_error: e)
+      record_attempt(attempts, failure_report(error, number, started_at, *error_classification(error)))
+      raise error
+    else
+      record_attempt(attempts, report)
+      EmbeddingResult.new(
+        vectors: embedding.vectors, model: embedding.model,
+        input_tokens: embedding.tokens.input, attempts: attempts
+      )
+    end
+
+    def request_embedding(inputs, dimensions)
+      context.embed(
+        inputs,
+        model: @model,
+        provider: :openai,
+        assume_model_exists: true,
+        dimensions: dimensions
+      )
+    rescue RubyLLM::UnauthorizedError => e
+      raise AuthenticationError.new(e.message, provider: :openai, original_error: e)
+    rescue RubyLLM::ForbiddenError => e
+      raise AuthorizationError.new(e.message, provider: :openai, original_error: e)
+    rescue RubyLLM::RateLimitError => e
+      raise RateLimitError.new(e.message, provider: :openai, reset_time: retry_after(e), original_error: e)
+    rescue Faraday::TimeoutError, Timeout::Error => e
+      raise TimeoutError.new(e.message, original_error: e)
+    rescue Faraday::ConnectionFailed => e
+      raise connection_error(e)
+    rescue RubyLLM::ServerError, RubyLLM::ServiceUnavailableError, RubyLLM::OverloadedError => e
+      raise ProviderError.new(e.message, original_error: e)
+    rescue Faraday::ParsingError, NoMethodError, TypeError => e
+      raise MalformedEmbeddingError.new("Malformed embedding response", original_error: e)
+    rescue RubyLLM::Error => e
+      raise ProviderError.new(e.message, original_error: e)
+    end
+
+    def context
+      RubyLLM.context do |config|
+        config.openai_api_key = @api_key
+        config.openai_api_base = @endpoint if @endpoint
+        config.request_timeout = @timeout
+        config.max_retries = 0
+        config.retry_interval_randomness = 0
+        config.faraday_adapter = EmbeddingAdapter.build(headers: @headers, cancellation: @cancellation)
+      end
+    end
+
+    def success_report(embedding, number, started_at)
+      attempt_report(number, started_at).merge(
+        status: :succeeded,
+        model: embedding.model,
+        usage: usage(embedding.tokens.input),
+        provider_reported: !embedding.tokens.input.nil?,
+        error: nil
+      ).freeze
+    end
+
+    def failure_report(error, number, started_at, category, code)
+      attempt_report(number, started_at).merge(
+        status: (category == :cancelled) ? :cancelled : :failed,
+        usage: usage(nil),
+        provider_reported: false,
+        error: {category: category, code: code}.freeze
+      ).freeze
+    end
+
+    def attempt_report(number, started_at)
+      {
+        attempt_id: "attempt_#{SecureRandom.uuid}", request_id: @request_id,
+        number: number, provider: :openai, model: @model,
+        started_at: started_at.iso8601(6), finished_at: Time.now.utc.iso8601(6),
+        cost: nil, provider_request_id: nil
+      }
+    end
+
+    def usage(input_tokens)
+      {input_tokens: input_tokens, output_tokens: nil, total_tokens: input_tokens}.freeze
+    end
+
+    def record_attempt(attempts, report)
+      attempts << report
+      return unless @observer
+
+      @observer.respond_to?(:on_attempt) ? @observer.on_attempt(report) : @observer.call(report)
+    end
+
+    def error_classification(error)
+      return [error.error_category, error.error_code] if error.is_a?(AuthenticationError) || error.is_a?(AuthorizationError)
+      return [:transient, :rate_limited] if error.is_a?(RateLimitError)
+      return [:transient, :timeout] if error.is_a?(TimeoutError)
+      return transient_provider_classification(error) if transient_provider_error?(error)
+      return [:invalid_response, :malformed_response] if error.is_a?(MalformedEmbeddingError)
+
+      [:unknown, :unclassified_provider_error]
+    end
+
+    def transient_provider_classification(error)
+      original = error.original_error
+      return [:transient, :connection_failed] if original.is_a?(Faraday::ConnectionFailed)
+      return [:transient, :service_unavailable] if original.is_a?(RubyLLM::ServiceUnavailableError)
+      return [:transient, :overloaded] if original.is_a?(RubyLLM::OverloadedError)
+
+      [:transient, :server_error]
+    end
+
+    def retryable?(error)
+      TRANSIENT_ERRORS.any? { |klass| error.is_a?(klass) } || transient_provider_error?(error)
+    end
+
+    def transient_provider_error?(error)
+      original = error.original_error
+      original.is_a?(Faraday::ConnectionFailed) || original.is_a?(RubyLLM::ServerError) ||
+        original.is_a?(RubyLLM::ServiceUnavailableError) || original.is_a?(RubyLLM::OverloadedError)
+    end
+
+    def wait_before_retry(error, attempt_number)
+      deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + retry_delay(error, attempt_number)
+      loop do
+        check_cancellation!
+        remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        break unless remaining.positive?
+
+        sleep([remaining, CANCELLATION_POLL_INTERVAL].min)
+      end
+    end
+
+    def retry_delay(error, attempt_number)
+      retry_after_delay = error.reset_time - Time.now if error.is_a?(RateLimitError) && error.reset_time
+      return retry_after_delay if retry_after_delay&.positive?
+
+      [RETRY_BASE_DELAY * (2**(attempt_number - 1)), RETRY_MAX_DELAY].min
+    end
+
+    def check_cancellation!
+      raise CancelledError, "Embedding request cancelled" if @cancellation&.call
+    end
+
+    def validate_result!(vectors, expected_count)
+      valid = vectors.is_a?(Array) && vectors.length == expected_count
+      valid &&= vectors.all? { |vector| valid_vector?(vector) }
+      raise MalformedEmbeddingError, "Provider returned an invalid embedding batch" unless valid
+    end
+
+    def valid_vector?(vector)
+      vector.is_a?(Array) && !vector.empty? && vector.all? { |value| value.is_a?(Numeric) && value.finite? }
+    end
+
+    def credential(credentials, key)
+      return credentials if credentials.is_a?(String)
+
+      credentials&.[](key) || credentials&.[](key.to_s)
+    end
+
+    def validate!
+      raise ArgumentError, "model must be a non-empty string" unless @model.is_a?(String) && !@model.empty?
+      raise ArgumentError, "credentials must include api_key" unless @api_key.is_a?(String) && !@api_key.empty?
+      if @headers.keys.any? { |header| header.casecmp?("authorization") }
+        raise ArgumentError, "headers cannot override Authorization; use credentials"
+      end
+      raise ArgumentError, "timeout must be positive" unless @timeout.is_a?(Numeric) && @timeout.positive?
+      unless @max_attempts.is_a?(Integer) && @max_attempts.positive?
+        raise ArgumentError, "max_attempts must be a positive integer"
+      end
+      unless @request_id.is_a?(String) && !@request_id.empty?
+        raise ArgumentError, "request_id must be a non-empty string"
+      end
+      unless @observer.nil? || @observer.respond_to?(:on_attempt) || @observer.respond_to?(:call)
+        raise ArgumentError, "observer must respond to on_attempt or call"
+      end
+    end
+
+    def retry_after(error)
+      value = error.response&.response_headers&.[]("retry-after")
+      return unless value
+
+      seconds = Float(value, exception: false)
+      seconds ? Time.now + seconds : Time.httpdate(value)
+    rescue ArgumentError, TypeError
+      nil
+    end
+
+    def connection_error(error)
+      wrapped = error.wrapped_exception
+      if wrapped.is_a?(Timeout::Error)
+        TimeoutError.new(error.message, original_error: error)
+      else
+        ProviderError.new(error.message, original_error: error)
+      end
+    end
+  end
+end

data/lib/agent_harness/errors.rb

@@ -15,6 +15,9 @@ module AgentHarness
   # Provider-related errors
   class ProviderError < Error; end
 
+  # Raised when an embedding provider response cannot satisfy the batch contract.
+  class MalformedEmbeddingError < ProviderError; end
+
   class ProviderInstallationError < ProviderError
     attr_reader :provider, :error_category
 
@@ -39,6 +42,9 @@ module AgentHarness
 
   class CommandExecutionError < Error; end
 
+  # Raised when a caller cancels a request before a transport attempt.
+  class CancelledError < Error; end
+
   # Rate limiting and circuit breaker errors
   class RateLimitError < Error
     attr_reader :reset_time, :provider, :error_category
@@ -62,10 +68,24 @@ module AgentHarness
 
   # Authentication errors
   class AuthenticationError < Error
-    attr_reader :provider
+    attr_reader :provider, :error_category, :error_code
 
-    def initialize(message = nil, provider: nil, **kwargs)
+    def initialize(message = nil, provider: nil, error_category: :authentication, error_code: :invalid_credential, **kwargs)
+      @provider = provider
+      @error_category = error_category
+      @error_code = error_code
+      super(message, **kwargs)
+    end
+  end
+
+  # Raised when valid credentials do not grant access to the requested resource.
+  class AuthorizationError < Error
+    attr_reader :provider, :error_category, :error_code
+
+    def initialize(message = nil, provider: nil, error_category: :authorization, error_code: :permission_denied, **kwargs)
       @provider = provider
+      @error_category = error_category
+      @error_code = error_code
       super(message, **kwargs)
     end
   end

data/lib/agent_harness/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AgentHarness
-  VERSION = "0.38.0"
+  VERSION = "0.40.0"
 end

data/lib/agent_harness.rb

@@ -87,6 +87,12 @@ module AgentHarness
       conductor.send_message(prompt, provider: provider, executor: executor, **options)
     end
 
+    # Generate embeddings for a batch of strings through a request-local RubyLLM context.
+    # @return [EmbeddingResult] vectors in input order and provider-reported batch usage
+    def embed(inputs:, model:, credentials:, dimensions: nil, **options)
+      Embeddings.new(model: model, credentials: credentials, **options).call(inputs: inputs, dimensions: dimensions)
+    end
+
     # Resolve a canonical extension definition by name or inline object.
     #
     # @param reference [Symbol, String, Extensions::Base]
@@ -456,6 +462,9 @@ require_relative "agent_harness/configuration"
 require_relative "agent_harness/command_executor"
 require_relative "agent_harness/docker_command_executor"
 require_relative "agent_harness/response"
+require_relative "agent_harness/embedding_result"
+require_relative "agent_harness/embedding_adapter"
+require_relative "agent_harness/embeddings"
 require_relative "agent_harness/token_tracker"
 require_relative "agent_harness/token_usage_tracker"
 require_relative "agent_harness/error_taxonomy"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: agent-harness
 version: !ruby/object:Gem::Version
-  version: 0.38.0
+  version: 0.40.0
 platform: ruby
 authors:
 - Bart Agapinan
@@ -29,6 +29,26 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: ruby_llm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -71,6 +91,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 description: |
   AgentHarness provides a unified interface for CLI-based AI coding agents like
   Claude Code, Cursor, Gemini CLI, and others. It offers full orchestration with
@@ -97,6 +131,7 @@ files:
 - Rakefile
 - bin/console
 - bin/setup
+- docs/provider-neutral-api-execution-contract.md
 - json-2.18.1.gem
 - lib/agent-harness.rb
 - lib/agent_harness.rb
@@ -106,6 +141,9 @@ files:
 - lib/agent_harness/conversation.rb
 - lib/agent_harness/dependency_updater.rb
 - lib/agent_harness/docker_command_executor.rb
+- lib/agent_harness/embedding_adapter.rb
+- lib/agent_harness/embedding_result.rb
+- lib/agent_harness/embeddings.rb
 - lib/agent_harness/error_taxonomy.rb
 - lib/agent_harness/errors.rb
 - lib/agent_harness/execution_preparation.rb