smith-agents 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +901 -0
- data/README.md +636 -20
- data/UPSTREAM_PROPOSAL.md +1 -0
- data/docs/CONFIGURATION.md +100 -14
- data/docs/PATTERNS.md +88 -12
- data/docs/PERSISTENCE.md +738 -6
- data/docs/TOOLS_AND_GUARDRAILS.md +24 -3
- data/lib/generators/smith/install/install_generator.rb +63 -2
- data/lib/generators/smith/install/templates/create_workflow_states.rb.tt +31 -0
- data/lib/generators/smith/install/templates/smith.rb.tt +39 -8
- data/lib/generators/smith/install/templates/workflow_state.rb.tt +19 -0
- data/lib/smith/agent/completion.rb +32 -6
- data/lib/smith/agent/completion_usage_recording.rb +23 -14
- data/lib/smith/agent/content_bytes.rb +98 -0
- data/lib/smith/agent/content_parts.rb +149 -0
- data/lib/smith/agent/fallback_configuration.rb +26 -3
- data/lib/smith/agent/invocation_preparation.rb +37 -14
- data/lib/smith/agent/lifecycle.rb +16 -5
- data/lib/smith/agent/observed_model.rb +29 -0
- data/lib/smith/agent/output_contract_check.rb +53 -0
- data/lib/smith/agent/output_integrity.rb +137 -0
- data/lib/smith/agent/output_policy.rb +92 -0
- data/lib/smith/agent/output_recovery.rb +129 -0
- data/lib/smith/agent/provider_attempt.rb +17 -4
- data/lib/smith/agent/provider_call_timing.rb +71 -7
- data/lib/smith/agent/provider_completion.rb +41 -38
- data/lib/smith/agent/provider_failure_handling.rb +5 -1
- data/lib/smith/agent/provider_request.rb +36 -0
- data/lib/smith/agent/provider_streaming.rb +155 -0
- data/lib/smith/agent/provider_usage.rb +75 -2
- data/lib/smith/agent/usage_entry_recording.rb +4 -2
- data/lib/smith/agent/usage_traces.rb +9 -2
- data/lib/smith/agent/usage_tracking.rb +64 -48
- data/lib/smith/agent.rb +14 -1
- data/lib/smith/artifacts/active_record_store.rb +79 -0
- data/lib/smith/artifacts/file.rb +45 -18
- data/lib/smith/artifacts/memory.rb +47 -36
- data/lib/smith/artifacts/s3.rb +126 -0
- data/lib/smith/artifacts/scoped_store.rb +55 -7
- data/lib/smith/artifacts/store_rules.rb +49 -0
- data/lib/smith/artifacts.rb +98 -1
- data/lib/smith/batch/message_shape.rb +36 -0
- data/lib/smith/batch.rb +166 -0
- data/lib/smith/budget/amount.rb +25 -0
- data/lib/smith/budget/amount_representation.rb +20 -0
- data/lib/smith/budget/continuation.rb +153 -0
- data/lib/smith/budget/host_ledger.rb +137 -0
- data/lib/smith/budget/ledger.rb +10 -4
- data/lib/smith/budget/ledger_state_transition.rb +7 -5
- data/lib/smith/budget/request_hold.rb +211 -0
- data/lib/smith/budget/spent_reservation.rb +18 -0
- data/lib/smith/child_run.rb +41 -0
- data/lib/smith/child_runs_failed.rb +56 -0
- data/lib/smith/content.rb +96 -0
- data/lib/smith/context/session.rb +14 -2
- data/lib/smith/context/state_injection.rb +21 -6
- data/lib/smith/context.rb +28 -0
- data/lib/smith/diagnostic_text.rb +10 -1
- data/lib/smith/doctor/checks/artifacts.rb +103 -0
- data/lib/smith/doctor/checks/configuration.rb +0 -11
- data/lib/smith/doctor/checks/durability.rb +35 -17
- data/lib/smith/doctor/checks/persistence_capabilities.rb +13 -7
- data/lib/smith/doctor/checks/state_model.rb +63 -0
- data/lib/smith/doctor/installer.rb +9 -2
- data/lib/smith/doctor.rb +4 -0
- data/lib/smith/errors.rb +304 -6
- data/lib/smith/idempotency_key/canonical.rb +74 -0
- data/lib/smith/idempotency_key.rb +145 -0
- data/lib/smith/launchers/inline.rb +46 -0
- data/lib/smith/message.rb +76 -0
- data/lib/smith/models/tool_routing.rb +47 -44
- data/lib/smith/output_failure.rb +130 -0
- data/lib/smith/persistence_adapters/active_record_exact_store.rb +51 -3
- data/lib/smith/persistence_adapters/active_record_store.rb +4 -7
- data/lib/smith/persistence_adapters.rb +10 -0
- data/lib/smith/pricing/catalog_validation.rb +91 -0
- data/lib/smith/pricing.rb +95 -64
- data/lib/smith/provider_permanent_failure.rb +44 -0
- data/lib/smith/providers/anthropic/batches.rb +116 -0
- data/lib/smith/providers/batch_exchange.rb +77 -0
- data/lib/smith/providers/openai/responses.rb +45 -20
- data/lib/smith/providers/output_limit.rb +68 -0
- data/lib/smith/providers/request_holds.rb +139 -0
- data/lib/smith/providers/response_facts.rb +108 -0
- data/lib/smith/providers/schema_constraint_check.rb +141 -0
- data/lib/smith/providers/schema_constraints.rb +87 -0
- data/lib/smith/providers/schema_rules.rb +66 -0
- data/lib/smith/providers/stop_reason.rb +39 -0
- data/lib/smith/providers/stream_facts.rb +192 -0
- data/lib/smith/response_memo.rb +152 -0
- data/lib/smith/setting_constructors.rb +47 -0
- data/lib/smith/streaming/json_field.rb +136 -0
- data/lib/smith/streaming/json_string.rb +91 -0
- data/lib/smith/streaming/session.rb +77 -0
- data/lib/smith/streaming/think_tags.rb +67 -0
- data/lib/smith/streaming.rb +64 -0
- data/lib/smith/testing/checks.rb +58 -0
- data/lib/smith/testing/hooks.rb +85 -0
- data/lib/smith/testing/minitest.rb +37 -0
- data/lib/smith/testing/rspec.rb +52 -0
- data/lib/smith/testing/stub.rb +110 -0
- data/lib/smith/testing.rb +158 -0
- data/lib/smith/tool/scoped_context.rb +9 -12
- data/lib/smith/tool.rb +14 -0
- data/lib/smith/trace/logger.rb +1 -1
- data/lib/smith/trace/memory.rb +1 -1
- data/lib/smith/trace/open_telemetry.rb +1 -1
- data/lib/smith/trace.rb +11 -4
- data/lib/smith/untrusted.rb +74 -0
- data/lib/smith/version.rb +2 -2
- data/lib/smith/workflow/active_clock.rb +61 -0
- data/lib/smith/workflow/agent_result.rb +37 -0
- data/lib/smith/workflow/artifact_integration.rb +57 -7
- data/lib/smith/workflow/await_step.rb +42 -0
- data/lib/smith/workflow/batch_charges.rb +74 -0
- data/lib/smith/workflow/batch_execution.rb +149 -0
- data/lib/smith/workflow/batch_holds.rb +73 -0
- data/lib/smith/workflow/batch_wait.rb +134 -0
- data/lib/smith/workflow/budget_integration.rb +14 -8
- data/lib/smith/workflow/budget_pause.rb +51 -0
- data/lib/smith/workflow/child_run_declarations.rb +127 -0
- data/lib/smith/workflow/child_run_execution.rb +150 -0
- data/lib/smith/workflow/child_run_results.rb +55 -0
- data/lib/smith/workflow/composite/effects.rb +37 -6
- data/lib/smith/workflow/composite/error.rb +1 -1
- data/lib/smith/workflow/composite/error_evidence.rb +9 -9
- data/lib/smith/workflow/composite/input.rb +11 -4
- data/lib/smith/workflow/context_keys.rb +77 -0
- data/lib/smith/workflow/deadline_enforcement.rb +1 -1
- data/lib/smith/workflow/deterministic_execution.rb +62 -11
- data/lib/smith/workflow/deterministic_step.rb +56 -7
- data/lib/smith/workflow/dsl.rb +74 -5
- data/lib/smith/workflow/durability.rb +140 -31
- data/lib/smith/workflow/evaluator_optimizer.rb +116 -12
- data/lib/smith/workflow/event_integration.rb +9 -0
- data/lib/smith/workflow/execution.rb +32 -23
- data/lib/smith/workflow/execution_binding_resolution.rb +13 -0
- data/lib/smith/workflow/failure_reconstructor.rb +41 -2
- data/lib/smith/workflow/failure_record.rb +12 -12
- data/lib/smith/workflow/failure_record_validator.rb +32 -5
- data/lib/smith/workflow/fanout_execution.rb +5 -8
- data/lib/smith/workflow/graph/batch_diagnostic.rb +83 -0
- data/lib/smith/workflow/graph/branch_limit_diagnostic.rb +41 -0
- data/lib/smith/workflow/graph/budget_ledger_diagnostic.rb +126 -0
- data/lib/smith/workflow/graph/durable_retry_diagnostic.rb +50 -0
- data/lib/smith/workflow/graph/execution_successors.rb +16 -2
- data/lib/smith/workflow/graph/inherited_untrusted_diagnostics.rb +77 -0
- data/lib/smith/workflow/graph/isolated_input_diagnostic.rb +50 -0
- data/lib/smith/workflow/graph/nested_readiness_diagnostics.rb +43 -4
- data/lib/smith/workflow/graph/output_key_diagnostic.rb +53 -0
- data/lib/smith/workflow/graph/output_schema_diagnostic.rb +49 -0
- data/lib/smith/workflow/graph/retry_policy_diagnostic.rb +1 -0
- data/lib/smith/workflow/graph/run_launcher_diagnostic.rb +30 -0
- data/lib/smith/workflow/graph/runtime_binding_diagnostic_builder.rb +9 -3
- data/lib/smith/workflow/graph/runtime_binding_diagnostics.rb +7 -3
- data/lib/smith/workflow/graph/runtime_readiness_metrics.rb +8 -1
- data/lib/smith/workflow/graph/runtime_readiness_report_builder.rb +9 -2
- data/lib/smith/workflow/graph/runtime_readiness_traversal.rb +7 -4
- data/lib/smith/workflow/graph/stream_diagnostic.rb +95 -0
- data/lib/smith/workflow/graph/transition_contract.rb +19 -0
- data/lib/smith/workflow/graph/transition_contract_attributes.rb +25 -2
- data/lib/smith/workflow/graph/transition_contract_configurations.rb +24 -3
- data/lib/smith/workflow/graph/transition_diagnostics.rb +11 -24
- data/lib/smith/workflow/graph/transition_snapshot.rb +40 -39
- data/lib/smith/workflow/graph/untrusted_input_diagnostic.rb +57 -0
- data/lib/smith/workflow/graph/waiting_child_diagnostic.rb +24 -0
- data/lib/smith/workflow/graph.rb +12 -0
- data/lib/smith/workflow/guarded_step_execution.rb +48 -3
- data/lib/smith/workflow/guardrail_integration.rb +2 -0
- data/lib/smith/workflow/message_batch.rb +6 -0
- data/lib/smith/workflow/message_roles.rb +25 -0
- data/lib/smith/workflow/message_value_normalizer.rb +2 -1
- data/lib/smith/workflow/nested_execution.rb +47 -4
- data/lib/smith/workflow/optimization_state.rb +2 -2
- data/lib/smith/workflow/orchestrator_worker.rb +9 -3
- data/lib/smith/workflow/parallel_execution.rb +10 -0
- data/lib/smith/workflow/persistence.rb +162 -22
- data/lib/smith/workflow/remembered_calls.rb +92 -0
- data/lib/smith/workflow/retry_execution.rb +71 -8
- data/lib/smith/workflow/retry_wait.rb +134 -0
- data/lib/smith/workflow/run_result.rb +10 -0
- data/lib/smith/workflow/split_step_persistence/checkpoint.rb +3 -0
- data/lib/smith/workflow/split_step_persistence/checkpoint_state.rb +7 -0
- data/lib/smith/workflow/split_step_persistence/composite_branch_execution.rb +4 -1
- data/lib/smith/workflow/split_step_persistence/composite_preparation.rb +3 -1
- data/lib/smith/workflow/split_step_persistence/dispatch_claim.rb +1 -0
- data/lib/smith/workflow/split_step_persistence/preparation.rb +5 -0
- data/lib/smith/workflow/split_step_persistence/state_snapshot.rb +1 -0
- data/lib/smith/workflow/step_completion.rb +40 -7
- data/lib/smith/workflow/step_context.rb +21 -0
- data/lib/smith/workflow/step_io.rb +132 -0
- data/lib/smith/workflow/step_stream.rb +52 -0
- data/lib/smith/workflow/thread_context_snapshot.rb +2 -0
- data/lib/smith/workflow/transition.rb +267 -13
- data/lib/smith/workflow/transition_actionability.rb +2 -0
- data/lib/smith/workflow/usage_entry.rb +24 -3
- data/lib/smith/workflow/wait_completion.rb +131 -0
- data/lib/smith/workflow/wait_execution.rb +180 -0
- data/lib/smith/workflow/wait_record.rb +157 -0
- data/lib/smith/workflow.rb +117 -19
- data/lib/smith.rb +103 -47
- metadata +88 -1
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,907 @@ All notable changes to Smith are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
|
|
6
6
|
|
|
7
|
+
## [0.12.0] - 2026-10-11
|
|
8
|
+
|
|
9
|
+
### Upgrade notes
|
|
10
|
+
|
|
11
|
+
- `execute` reads `mode:` as the call mode (`:sync`, `:batch`, or a callable)
|
|
12
|
+
and `stream:` as a stream name; neither stays in `agent_opts`, and any other
|
|
13
|
+
`mode:` value raises `Smith::WorkflowError` at declaration.
|
|
14
|
+
|
|
15
|
+
- The artifact store must be configured: outside `test_mode`, storing or
|
|
16
|
+
reading an artifact with no `artifact_store` raises
|
|
17
|
+
`Smith::ArtifactStoreMissing` instead of using a store in process memory
|
|
18
|
+
(a run that keeps no artifacts needs none). Set `config.artifact_store` to
|
|
19
|
+
`:s3`, `:active_record`, `:file` or `:memory` (with
|
|
20
|
+
`config.artifact_options`), or a store; `smith doctor` reports a missing,
|
|
21
|
+
unusable or process-local store, and the install templates no longer write
|
|
22
|
+
`Smith::Artifacts::Memory.new`.
|
|
23
|
+
- Artifact stores take bytes: `store` refuses anything but a String with a
|
|
24
|
+
`TypeError` (Memory used to keep any object, and File wrote its `inspect`),
|
|
25
|
+
so serialize first (`JSON.generate`). A store class now gets built with
|
|
26
|
+
`artifact_options` unless the class itself stores and fetches. A tenant
|
|
27
|
+
`namespace:` must be letters, digits, `-` and `_`, and
|
|
28
|
+
`artifact_tenant_isolation` and `artifact_retention` now apply to every
|
|
29
|
+
built-in store. `Memory#delete` drops one run's claim when given its
|
|
30
|
+
`execution_namespace:`; the File store's refs are random UUIDs, as before.
|
|
31
|
+
|
|
32
|
+
- `Smith::EXECUTION_SEMANTICS_VERSION` is now `"7"`: routing, context writes,
|
|
33
|
+
restore admission, nested failure and retry semantics changed, and durable
|
|
34
|
+
composite plans may carry content parts, which an earlier build would send
|
|
35
|
+
as JSON text. Hosts that
|
|
36
|
+
fold it into definition digests or cache keys see new values, and durable
|
|
37
|
+
composite plans prepared under 0.11.0 are refused by their integrity check.
|
|
38
|
+
Drain or clear in-flight persisted runs before upgrading.
|
|
39
|
+
- Token and cost accounting changed too (see Changed): cache tokens count in
|
|
40
|
+
totals and budgets, cached input on the OpenAI Responses path is priced at
|
|
41
|
+
the cache-read rate, and completed tool-loop rounds before a failure settle
|
|
42
|
+
into budgets. A run in flight across the upgrade mixes the two accountings.
|
|
43
|
+
- OpenAI Responses calls cost less on upgrade unless the catalog sets
|
|
44
|
+
`cache_read_cost_per_token`: their cached input was priced at the full input
|
|
45
|
+
rate and is now priced at the cache-read rate, 0 when unset.
|
|
46
|
+
- `append_session_messages!` now admits only the roles `user`, `assistant` and
|
|
47
|
+
`tool`. A host that admitted system notes should put that text in
|
|
48
|
+
`seed_messages`, the agent's instructions or an `inject_state` formatter. A
|
|
49
|
+
restored payload must hold only `system`, `user`, `assistant` and `tool`
|
|
50
|
+
messages; one written earlier may hold a system message the host admitted,
|
|
51
|
+
which still reaches agents, so rebuild such sessions to rely on the new rule.
|
|
52
|
+
- An agent with no model and no fallback models now fails its step with a
|
|
53
|
+
`Smith::WorkflowError` instead of returning nil, and runtime readiness reports
|
|
54
|
+
`:agent_without_model` as an error. Give placeholder agents a model (and a
|
|
55
|
+
stubbed chat in tests). An agent with fallback models but no primary model
|
|
56
|
+
used to be skipped the same way and now runs on its fallbacks.
|
|
57
|
+
- A response cut off at its output limit, a refused response, and a structured
|
|
58
|
+
output that does not parse to a JSON object now fail the step
|
|
59
|
+
(`Smith::OutputTruncated`, `Smith::OutputRefused`, `Smith::OutputInvalid`)
|
|
60
|
+
instead of passing on as the output; `on_truncation :accept` keeps the old
|
|
61
|
+
behavior for cut-off responses. These errors are not `AgentError`s, so a
|
|
62
|
+
`rescue Smith::AgentError` no longer catches such a failure. An agent's
|
|
63
|
+
`after_completion` now runs before the structured-output checks, and does not
|
|
64
|
+
run for a refused or cut-off response.
|
|
65
|
+
|
|
66
|
+
### Added
|
|
67
|
+
|
|
68
|
+
- Durable artifact stores, configured like workflow state:
|
|
69
|
+
`config.artifact_store = :s3` (AWS S3, Cloudflare R2 and other
|
|
70
|
+
S3-compatible buckets, through the host's `aws-sdk-s3` or `client:`) or
|
|
71
|
+
`:active_record` (a host table), with `config.artifact_options`; the store
|
|
72
|
+
may also be a store class built with the options. Every built-in store
|
|
73
|
+
records which run stored each ref (`stored_by?`) and can `delete`, and
|
|
74
|
+
`smith doctor` checks the store.
|
|
75
|
+
- Content parts. `Smith::Message.user`, `.assistant` and `.system` take a String
|
|
76
|
+
or `Smith::Content` parts (`text`, and `document` and `image`, which name an
|
|
77
|
+
artifact of the run and a media type) in a step's `input:`; seed messages
|
|
78
|
+
take text parts, and only user messages take documents and images.
|
|
79
|
+
Anthropic gets native blocks with `cache_control` on parts marked
|
|
80
|
+
`cache: true` (at most 4 breakpoints per request, the chat's own included);
|
|
81
|
+
other providers get the parts in order as text with the files attached where
|
|
82
|
+
they stand, and a model that takes no such content passes the step to the
|
|
83
|
+
next in the chain. Documents and images are read once per call, only from a
|
|
84
|
+
store that records which run stored them, and checked against their media
|
|
85
|
+
type, a 24 MiB document and 5 MiB image bound, and a 32 MiB request bound;
|
|
86
|
+
any failure raises `Smith::ContentUnavailable` before the call. A message
|
|
87
|
+
with parts carries `"parts" => true`, which no agent output or
|
|
88
|
+
host-admitted message can carry, and a restored session holds parts only in
|
|
89
|
+
its seed messages. A child run may read its parent's artifacts and those of
|
|
90
|
+
the parent's nearest ancestors (`Smith::ChildRun#artifact_namespaces`, which
|
|
91
|
+
a queueing launcher passes to the job like `untrusted_keys`; persisted with
|
|
92
|
+
the child as `inherited_artifact_namespaces`).
|
|
93
|
+
- Durable retry. `retry_on ..., durable: true` waits out each backoff as a
|
|
94
|
+
wait of the run (`awaiting.kind == :retry`, due when the backoff ends)
|
|
95
|
+
instead of sleeping in the worker. `expire_wait!` past the deadline ends the
|
|
96
|
+
wait without a step, and the host continues the run, which makes the next
|
|
97
|
+
attempt under the recorded attempt number and key generation; the pending
|
|
98
|
+
retry persists (`pending_retry`, shown as `awaiting.attempt` and
|
|
99
|
+
`awaiting.error`) until the step ends without a wait, so a crash makes the
|
|
100
|
+
same attempt again under the same keys (in strict mode, the restore of a run
|
|
101
|
+
caught in it raises `Smith::StepInProgressOnRestore`). No retry wait is
|
|
102
|
+
entered past the deadline, after a tool ran, or for a backoff that does not
|
|
103
|
+
end within the persistence TTL, which runtime readiness reports
|
|
104
|
+
(`:durable_retry_outlasts_ttl`). `durable:` needs 2 attempts, a positive
|
|
105
|
+
backoff and `on_failure` (so a last failure settles the run rather than
|
|
106
|
+
being made again by a job that runs again), and is refused on await, delay,
|
|
107
|
+
parallel, fan_out, optimize and orchestrate steps; graph inspection shows it, and runtime readiness reports
|
|
108
|
+
an in-process child with a durable retry (a nested workflow, or a child run
|
|
109
|
+
the launcher runs inline) as `:nested_workflow_waits`. Split-step
|
|
110
|
+
persistence refuses a step with a durable retry. Traces gain the outcomes
|
|
111
|
+
`:retrying` and `:retry_due`.
|
|
112
|
+
- Streaming steps. `execute Agent, stream: :name` sends the reply's text, as
|
|
113
|
+
the model writes it, to a sink the host registers in
|
|
114
|
+
`Smith.config.stream_sinks`: a callable that takes the step's info
|
|
115
|
+
(`stream`, a frozen copy of the `execution_key`, `transition`) when the step
|
|
116
|
+
starts and returns that step's sink, or nil to stream nothing, so concurrent
|
|
117
|
+
runs never share a sink. A sink defines any of `start(info)` per provider
|
|
118
|
+
call (with the attempt, model and provider), `delta(text)` per text chunk
|
|
119
|
+
(not tool calls, thinking or `<think>` text, which `Smith::Streaming::ThinkTags`
|
|
120
|
+
keeps out as RubyLLM keeps it out of the message), `reset(reason)` when
|
|
121
|
+
Smith discards the text sent (`:retry`, `:fallback`, `:truncation`,
|
|
122
|
+
`:repair`; and `:retry_wait`, always, when a durable retry waits),
|
|
123
|
+
`finish(output)` with the accepted output (after guardrails, the output
|
|
124
|
+
contract and `output_to:`, and in a stored run once `persist!` has stored
|
|
125
|
+
the step; a store that fails gives `fail` with its error) and
|
|
126
|
+
`fail(error_class)`. A sink or sink callable that raises (a `ScriptError`
|
|
127
|
+
included) is logged by error class and the step goes on; an unregistered
|
|
128
|
+
name fails the step before its call (`Smith::StreamSinkMissing`). The text is
|
|
129
|
+
a draft until `finish`. `Smith::Streaming::JsonField#push` returns the text
|
|
130
|
+
each delta adds to one top-level string field of streamed JSON (keys read as
|
|
131
|
+
JSON reads them), in linear time. Stop reasons and message ids are read from
|
|
132
|
+
the stream itself (`Smith::Providers::StreamFacts`, prepended on RubyLLM's
|
|
133
|
+
Anthropic, OpenAI, Gemini, OpenRouter and Bedrock chunk builders with each
|
|
134
|
+
one's visibility kept, inherited by their subclasses, and on
|
|
135
|
+
`RubyLLM::Provider#complete` to tell a tool loop's requests apart), from the
|
|
136
|
+
step's own provider only (a tool's own RubyLLM call is not read), per
|
|
137
|
+
request, in fiber storage; each usage entry keeps its response's facts and
|
|
138
|
+
`mode: :stream`, so cut-off and refused replies are handled and recorded as
|
|
139
|
+
without streaming. A streamed reply with no stop reason counts as cut off:
|
|
140
|
+
`output_limit_reached` when its output tokens reached the request's limit,
|
|
141
|
+
else `stream_incomplete`. Streaming changes nothing durable, and a replayed
|
|
142
|
+
step streams again. `stream:` is refused on a parallel step; graph
|
|
143
|
+
inspection shows it, and runtime readiness reports a missing sink
|
|
144
|
+
(`:stream_sink_missing`) and an agent whose calls go to the Responses API,
|
|
145
|
+
which does not stream (`:stream_whole_replies`, decided as Smith's tool
|
|
146
|
+
routing decides at runtime: `Smith::Models::ToolRouting.responses?`).
|
|
147
|
+
- Batch mode. `execute Agent, mode: :batch` (or `:sync`, or a callable of the
|
|
148
|
+
context and the step attempt returning one), and the same on `fan_out`,
|
|
149
|
+
sends the step's requests through a provider's batch API. Smith prepares
|
|
150
|
+
each call as a synchronous one, admits it against the budget, captures the
|
|
151
|
+
exact request at RubyLLM's `sync_response` (`Smith::Providers::BatchExchange`,
|
|
152
|
+
bound to the call's own provider object; the connection's credentials never
|
|
153
|
+
leave it), and hands it to `Smith.config.batch_queue.enqueue(custom_id:,
|
|
154
|
+
await_token:, request:, execution_key:)`; the run waits on `:batch_result`
|
|
155
|
+
(kind `:batch`, 25 hours) with its `pending_batch` recorded. The host
|
|
156
|
+
submits, polls and collects with `Smith::Providers::Anthropic::Batches`
|
|
157
|
+
(`submit`, `status`, `results`, `cancel`; no connection retries, so a batch
|
|
158
|
+
is never submitted twice; the connection's own headers win over a request's)
|
|
159
|
+
and resumes the run once with `{custom_id => result}`, each result's message
|
|
160
|
+
checked against the provider's reply shape. The requests are queued only once
|
|
161
|
+
the step's attempt has ended without error; `pending_batch` keeps each
|
|
162
|
+
request's digest. A call that renders another request by the time its
|
|
163
|
+
result comes (a date in its prompt, data read afresh) does what its step's
|
|
164
|
+
`on_request_change:` says: `:keep` (the default) takes the result, its
|
|
165
|
+
`:provider_call` trace saying `request_changed: true`; `:resend` sends the
|
|
166
|
+
request as rendered now over the wire; `:fail` raises
|
|
167
|
+
`Smith::BatchRequestChanged` (an `AgentError`, which `retry_on` may retry).
|
|
168
|
+
A result set aside is charged as a `:failed_attempt`. The step runs again at its attempt, each call
|
|
169
|
+
of that attempt (only) taking its result through RubyLLM's own parser, and a
|
|
170
|
+
succeeded result no call took (a cancelled fan-out branch), priced under the id RubyLLM
|
|
171
|
+
resolves its agent's model to, is charged as a
|
|
172
|
+
`:failed_attempt`; in strict mode the completion runs as a marked step. A
|
|
173
|
+
batch step in an in-process child fails before capture. Usage with
|
|
174
|
+
`mode: :batch` priced
|
|
175
|
+
with `batch_discount` and charged even past a budget
|
|
176
|
+
(`Budget::Ledger#reserve_spent!`), output checks, follow-up calls over the
|
|
177
|
+
wire, guardrails and `output_to:` as after a synchronous call. A refused
|
|
178
|
+
request raises `Smith::BatchRequestInvalid`; a failed, canceled, expired or
|
|
179
|
+
missing result raises `Smith::BatchResultFailed`, which `retry_on` sends
|
|
180
|
+
again. A missing queue raises `Smith::BatchQueueMissing`, and a provider
|
|
181
|
+
without a batch interface in Smith (only Anthropic has one) or an agent with
|
|
182
|
+
tools `Smith::BatchModeUnavailable`, before anything is sent. A batch step
|
|
183
|
+
needs `on_failure` and cannot stream or be `parallel:`; graph inspection
|
|
184
|
+
shows the mode, and runtime readiness reports `:batch_queue_missing`,
|
|
185
|
+
`:batch_unavailable` and an in-process child with a batch step.
|
|
186
|
+
- A step that runs inside another run's step (an inline child run) scopes its
|
|
187
|
+
artifacts over the configured store itself, not over the outer run's
|
|
188
|
+
scoped view, through which it could read nothing of its own.
|
|
189
|
+
- A run's scoped artifact reads check, with a store that records it
|
|
190
|
+
(`stored_by?`, added to `Smith::Artifacts::Memory` and
|
|
191
|
+
`Smith::Artifacts::File`), that one of the runs it may read stored the ref,
|
|
192
|
+
not only the ref's prefix. `Smith::Artifacts::File` no longer reads a ref
|
|
193
|
+
outside its directory or another tenant's.
|
|
194
|
+
- `Smith::Workflow::Wait` gains `attempt` and `error` (nil except for a retry
|
|
195
|
+
wait). A wait as long as the persistence TTL is now refused too (it was
|
|
196
|
+
refused only when longer), since the stored run would expire as it ends.
|
|
197
|
+
- Child runs. `map items, workflow:, max_items:, input:, timeout:, into:,
|
|
198
|
+
collect:, on_item_failure:` launches a run of its own per item and waits,
|
|
199
|
+
bounded by `timeout:`, for the host to resume the parent with
|
|
200
|
+
`resume!(key, event: :children, token:, payload:)` and one
|
|
201
|
+
`Smith::ChildRun.result` entry per child; `call workflow, input:, timeout:,
|
|
202
|
+
into:` is a map of one whose `into:` gets the child's output; `spawn items,
|
|
203
|
+
workflow:, max_items:, input:` launches and continues. Runs start through
|
|
204
|
+
`Smith.config.run_launcher.launch(children:, parent_key:, await_token:)`,
|
|
205
|
+
which must be idempotent on `child.key` (the step's token and the child's
|
|
206
|
+
ref, the index of its item), so a replayed step launches the same children
|
|
207
|
+
and two steps never share a key.
|
|
208
|
+
`Smith::Launchers::Inline` (`inline?` true, the only kind of launcher whose
|
|
209
|
+
return value is taken as results) runs them in process. More than
|
|
210
|
+
`max_items` fails the step before anything launches; results that do not
|
|
211
|
+
match the number of children launched raise to the host and leave the run
|
|
212
|
+
waiting; `collect:` writes outputs, states or counts; `on_item_failure:
|
|
213
|
+
:fail` fails the step with `Smith::ChildRunsFailed` (failure family
|
|
214
|
+
`child_runs_failed`) and needs `on_failure`. No map, call or spawn takes
|
|
215
|
+
`retry_on`, and split-step persistence does not prepare them. A child gets
|
|
216
|
+
only what `input:` returns and keeps any of its parent's untrusted keys
|
|
217
|
+
passed under the same name, persisted with it as `inherited_untrusted_keys`.
|
|
218
|
+
Graph inspection shows `kind: :map`, `:call` or `:spawn` with the child and
|
|
219
|
+
its bounds; runtime readiness walks child workflows, reports
|
|
220
|
+
`:run_launcher_missing`, flags an in-process nested workflow that reaches a
|
|
221
|
+
map, call or spawn, and warns of a child agent that reads a key the parent
|
|
222
|
+
marks untrusted.
|
|
223
|
+
- Output integrity. Smith reads each response's stop reason (OpenAI's
|
|
224
|
+
message-level refusals and Gemini's prompt blocks included) as cut off or
|
|
225
|
+
refused. A cut-off response raises `Smith::OutputTruncated` unless the agent
|
|
226
|
+
declares `on_truncation :accept` or `on_truncation retry_with:
|
|
227
|
+
{ max_tokens: n }` (one more call on the same model, with the limit mapped to
|
|
228
|
+
the provider's own parameter, made only when it is larger than the request
|
|
229
|
+
had and the response did not run out of context); a refused one raises
|
|
230
|
+
`Smith::OutputRefused` unless the agent declares
|
|
231
|
+
`on_refusal fallback: "provider/model"` (one call on that model). A
|
|
232
|
+
structured output (`output_schema`, or a static RubyLLM `schema`) that is not
|
|
233
|
+
a JSON object, breaks a schema constraint, or fails `output_contract` (whose
|
|
234
|
+
`call(hash)` returns true, false, or a result answering `success?` and
|
|
235
|
+
`errors`) goes back to the model with its problems up to `output_repairs`
|
|
236
|
+
times (default 1), then raises `Smith::OutputInvalid` with at most 20 problems
|
|
237
|
+
as "path: problem" lines. No follow-up call is made once a tool has run in
|
|
238
|
+
the step or the step's reserved budget is spent, and a follow-up that fails
|
|
239
|
+
raises the output error with the provider's error as its cause. For
|
|
240
|
+
Anthropic, Smith sends the schema without the numeric, string-length and
|
|
241
|
+
array-size constraints it rejects, with `additionalProperties: false` on
|
|
242
|
+
every object, refuses an object that allows other properties or declares
|
|
243
|
+
none (runtime readiness reports `:output_schema_unsupported`), and checks
|
|
244
|
+
those constraints on the parsed output for every provider. The three errors
|
|
245
|
+
share the failure family `output_failure` (in durable composite branches
|
|
246
|
+
too), persist and restore with their details, and are not retryable. Every
|
|
247
|
+
response is billed and settles into budgets, and a rejected one leaves an
|
|
248
|
+
`:output_rejected` trace.
|
|
249
|
+
- Usage carries prompt-cache, thinking and response facts. Each provider
|
|
250
|
+
response's usage (`Smith::Agent::ProviderUsage`), each `UsageEntry`, each
|
|
251
|
+
`AgentResult`, and the `:provider_call` trace gain `cache_read_tokens`,
|
|
252
|
+
`cache_write_tokens`, `thinking_tokens`, `stop_reason`, and
|
|
253
|
+
`provider_message_id`; usage entries also gain `mode` (`:sync` for every call
|
|
254
|
+
today; streaming and batch steps will record theirs). `input_tokens` keeps
|
|
255
|
+
meaning uncached input, as RubyLLM reports every provider; the cache counts
|
|
256
|
+
sit beside it, and `thinking_tokens` are already inside `output_tokens`.
|
|
257
|
+
`stop_reason` and `provider_message_id` are read from the raw response body
|
|
258
|
+
of Anthropic Messages, OpenAI Chat Completions, OpenAI Responses, Gemini and
|
|
259
|
+
Bedrock Converse, each recognized by its own marker (for example
|
|
260
|
+
`"object" => "response"`); any other body gives neither, and a body that
|
|
261
|
+
cannot be read never fails the call. They are bounded to 256 bytes and are
|
|
262
|
+
those of the last response of a tool loop, even when that response reported
|
|
263
|
+
no usage. Each is omitted when the provider did not report it (RubyLLM's
|
|
264
|
+
OpenAI adapters report a cache write count of 0): `UsageEntry#to_h` and
|
|
265
|
+
`ProviderUsage#to_h` leave nil optional members out, so entries written by
|
|
266
|
+
earlier versions re-serialize byte-identically. Usage entries also gain
|
|
267
|
+
`mode`, set only for streamed and batch calls (later in this release), so an
|
|
268
|
+
ordinary call's entry keeps its earlier shape. `:token_usage` and `:cost`
|
|
269
|
+
traces carry the cache counts too, and `:cost` adds `cache_unpriced: true`
|
|
270
|
+
when the invocation's cache tokens were priced at 0 for lack of a rate. Hosts
|
|
271
|
+
with a `trace_fields` allowlist must add the new keys to receive them.
|
|
272
|
+
- Pricing catalogs accept `cache_read_cost_per_token` and
|
|
273
|
+
`cache_write_cost_per_token` on an entry or a tier (a tier without its own
|
|
274
|
+
uses the entry's), and `batch_discount` (at least 0 and below 1) on an entry,
|
|
275
|
+
applied to the whole cost of a call made in batch mode. They are configured
|
|
276
|
+
per catalog and count as 0 when absent, so a call is never left unpriced for
|
|
277
|
+
lack of them. `Smith::Pricing.compute_cost` takes `cache_read_tokens:`,
|
|
278
|
+
`cache_write_tokens:` and `mode:` (defaults 0, 0, `:sync`) and treats a count
|
|
279
|
+
that is not a non-negative Integer as 0, so it still never raises. Batch mode
|
|
280
|
+
is priced only through `compute_cost(mode: :batch)` until batch steps ship.
|
|
281
|
+
- Step isolation. `input: ->(ctx) { [...] }` on `execute`, `route`,
|
|
282
|
+
`fan_out`, `optimize` and `orchestrate` gives a step its own messages (at
|
|
283
|
+
most 100, user and assistant roles only, with content, bounded like session
|
|
284
|
+
messages) in place of the session transcript. `output_to: :key` writes the
|
|
285
|
+
step's output to the context instead of the transcript, JSON-shaped, once the
|
|
286
|
+
step completes; a key an explicit persist list leaves out is refused before
|
|
287
|
+
the step runs, and validation reports `:output_to_not_persisted` (and
|
|
288
|
+
`:await_into_not_persisted` for an await's `into:`). A compute step's
|
|
289
|
+
`last_output` is the latest step output wherever it went, and the next
|
|
290
|
+
shared agent step still gets the continuation turn after the transcript's
|
|
291
|
+
last output. `session :isolated` on a workflow keeps no transcript for
|
|
292
|
+
agents: every agent step must declare `input:`, outputs are not appended,
|
|
293
|
+
and `inject_state` never runs. Graph inspection shows `input` and
|
|
294
|
+
`output_to`; validation reports `:isolated_step_without_input` and warns
|
|
295
|
+
with `:isolated_workflow_nests_shared_session`.
|
|
296
|
+
- `untrusted :key` on a `Smith::Context` keeps a key from `inject_state`
|
|
297
|
+
formatters and the `seed_messages` builder, and refuses an agent that names
|
|
298
|
+
it in `inputs`, in the workflow and in any workflow it nests (runtime
|
|
299
|
+
readiness reports `:untrusted_agent_input` and
|
|
300
|
+
`:nested_untrusted_agent_input`); reserved input names are not checked, and
|
|
301
|
+
`untrusted :auto` is refused. `Smith::Untrusted.wrap(text, source:)`
|
|
302
|
+
delimits outside content with a boundary the text cannot guess, derived
|
|
303
|
+
inside a step so a re-prepared step input is identical, and marks role
|
|
304
|
+
labels after any line break, and look-alike untrusted tags, as quoted text.
|
|
305
|
+
- Durable, bounded waits. `await event, into:, timeout:` waits for an event the
|
|
306
|
+
host delivers with `Workflow.resume!(key, event:, token:, payload:)`, which
|
|
307
|
+
writes the payload to the context key `into:` and completes the step;
|
|
308
|
+
`delay seconds` waits for the time alone. `timeout:` is required (seconds,
|
|
309
|
+
or a callable of the context), and an await must declare `on_timeout` or
|
|
310
|
+
`on_failure`; `expire_wait!(key, token:, now:)` past the deadline takes
|
|
311
|
+
`on_timeout` (with a `StepCompleted` ending in the await's own state) or
|
|
312
|
+
`on_failure` with the new `Smith::WaitTimedOut` (failure family
|
|
313
|
+
`wait_timed_out`, restored with its details). An await's optional block runs
|
|
314
|
+
on entry with a `Smith::Workflow::AwaitStep` (`await_token`, `read_context`,
|
|
315
|
+
`idempotency_key`, `resolve!`). Entering a wait is not a step: `advance!`
|
|
316
|
+
returns nil, `run!` and `run_persisted!` stop, the run is never terminal,
|
|
317
|
+
and step counts and `max_transitions` do not move. `waiting?` and `awaiting`
|
|
318
|
+
(a `Smith::Workflow::Wait`) let the host index waits and schedule expiry;
|
|
319
|
+
Smith never polls or wakes a run. The class methods store the run with the
|
|
320
|
+
version check, require an adapter with `store_versioned`, and return
|
|
321
|
+
`:resumed`, `:failed`, `:expired`, `:not_due`, `:missing` or `:stale`. A
|
|
322
|
+
reply that arrives before its wait is stored completes the await at once
|
|
323
|
+
when the token is the one the run is entering it with, and the entering
|
|
324
|
+
process loses its next store. One completion at a time claims a wait on an
|
|
325
|
+
instance. A wait longer than the persistence TTL is refused on entry, and
|
|
326
|
+
`stuck_for?` treats a waiting run as stuck only past its deadline. Graph
|
|
327
|
+
inspection shows `kind: :await` or `:delay`, the wait and
|
|
328
|
+
`timeout_transition`, and runtime readiness reports a nested workflow that
|
|
329
|
+
reaches a wait (`:nested_workflow_waits`). Split-step persistence refuses
|
|
330
|
+
wait steps, and an in-process nested workflow that reaches a wait fails its
|
|
331
|
+
parent's step. `Workflow.transition_named` finds a transition by the String
|
|
332
|
+
form of its name.
|
|
333
|
+
- Idempotency keys for requests a step sends outside the system.
|
|
334
|
+
`step.idempotency_key(scope = nil)` in `compute` and `run` steps,
|
|
335
|
+
`idempotency_key(scope)` inside a `Smith::Tool`'s `perform`, and
|
|
336
|
+
`Smith::IdempotencyKey.current(scope = nil)` for host code called from a
|
|
337
|
+
running step return 64 lowercase hex characters (a SHA-256). A key is the
|
|
338
|
+
same on every attempt of the step and after a restore, and unique to the
|
|
339
|
+
run, the step's position (step number and transition, then any nested
|
|
340
|
+
workflow step, parallel or fan-out branch, durable composite branch,
|
|
341
|
+
optimizer round and role, or orchestrator round and worker task, a task
|
|
342
|
+
being named by its content), the tool's `name` and the scope. Scopes are
|
|
343
|
+
built from Strings, Symbols, numbers (a whole Float counts as the
|
|
344
|
+
Integer), booleans, `nil`, Arrays and Hashes, within 32 levels and 100,000
|
|
345
|
+
values; NaN, Infinity and other objects raise `ArgumentError`. Outside a
|
|
346
|
+
running step, and in event handlers and `on_step` callbacks, they raise
|
|
347
|
+
`Smith::WorkflowError`. Keys survive a crash replay when the run was
|
|
348
|
+
checkpointed before the step, as `advance_persisted!` and `run_persisted!`
|
|
349
|
+
do.
|
|
350
|
+
- `retry_on ..., new_idempotency_key_on: [ErrorClass]` and
|
|
351
|
+
`step.fail!(..., new_idempotency_key: true)` declare that a failed attempt's
|
|
352
|
+
own requests took no effect, so the attempts after it use new keys; every
|
|
353
|
+
other retry keeps them. `fail!`'s flag counts only for the step that raised
|
|
354
|
+
it, never for an enclosing workflow's retry. `new_idempotency_key_on:` must
|
|
355
|
+
name classes the policy retries, each once, refuses retry-forbidden classes,
|
|
356
|
+
is refused on nested workflow, parallel, fan-out, optimize and orchestrate
|
|
357
|
+
transitions, and shows in graph inspection's retry policy when declared.
|
|
358
|
+
`DeterministicStepFailure#new_idempotency_key` and `#idempotency_scope`, and
|
|
359
|
+
`DeterministicStep#attempt` (the attempt under way, from 1).
|
|
360
|
+
- `UsageEntry#total_tokens`, `AgentResult#total_tokens`,
|
|
361
|
+
`AgentResult#billable_tokens` (every known count, which budgets settle) and
|
|
362
|
+
`ProviderUsage#total_input_tokens`. `Agent::Completion#usage` (the whole
|
|
363
|
+
completion's `ProviderUsage`), `AgentResult.usage_details`,
|
|
364
|
+
`AgentResult::USAGE_DETAIL_MEMBERS` and `UsageEntry::OPTIONAL_MEMBERS` are
|
|
365
|
+
public.
|
|
366
|
+
- Response memo: `Smith.config.response_memo` (`fetch(key)`,
|
|
367
|
+
`store(key, record)`) keeps what a step's provider calls got, so a step
|
|
368
|
+
replayed after its process died reuses it instead of paying again. Each
|
|
369
|
+
call is keyed by its position's idempotency scope, the attempts it runs
|
|
370
|
+
under (enclosing runs' too), its attempt index, its place among the
|
|
371
|
+
attempt's calls and a digest of its request (a changed request misses).
|
|
372
|
+
Its record (version 1) holds the messages it added (replies, structured
|
|
373
|
+
ones too, with their usage and facts, a stream's included; tool calls and
|
|
374
|
+
results), stored after each round's tool results and when it ends. A
|
|
375
|
+
replayed call that ended sends nothing (its `:provider_call` trace says
|
|
376
|
+
`memo: true`, its usage is recorded once); one that had not continues from
|
|
377
|
+
its tools' results. Kept tool results count as tools the step ran. A memo
|
|
378
|
+
never fails a step: a record Smith does not keep is ignored, errors are
|
|
379
|
+
logged by class.
|
|
380
|
+
- Budget ledgers: `budget_ledger(*scope_keys) { |scope| ... }` names the
|
|
381
|
+
host's ledger for a run (an account's spend across runs, say). Its scope is
|
|
382
|
+
the values the keys have in the run's trusted context when it is created,
|
|
383
|
+
kept with the run (`budget_ledger_scope`); Smith builds the ledger at the
|
|
384
|
+
run's first request, never persists it, and never needs it to restore a
|
|
385
|
+
run. A call's first request over the wire (each fallback model, each
|
|
386
|
+
follow-up, streamed or not, OpenAI's Responses route included) is held on
|
|
387
|
+
it before it is sent: `reserve!(amount, key:, expires_at:)`, admitted only
|
|
388
|
+
by `true`, then `reconcile!(key, amount)` with what it spent (its hold, when
|
|
389
|
+
it answered without usage or its stream failed part way), or
|
|
390
|
+
`release!(key)` when it failed before anything came back. A hold
|
|
391
|
+
(`Smith::Budget::Amount`: `input_tokens`, `output_tokens`, `cost`,
|
|
392
|
+
`tokens`) is the request's output cap and an estimate of its input (images
|
|
393
|
+
and PDFs counted as such, in attachments and raw content blocks), priced
|
|
394
|
+
under the model's resolved id. A refusal raises `Smith::BudgetUnavailable`,
|
|
395
|
+
a `BudgetExceeded`, before anything is sent; a ledger that cannot be built
|
|
396
|
+
or answer `reserve!` raises `Smith::BudgetLedgerError`; an error from
|
|
397
|
+
`reconcile!` or `release!` is logged by class and the step goes on. Keys
|
|
398
|
+
are `"<attempt_id>:<round>"`, and a batch request's `custom_id`: a batch
|
|
399
|
+
request is held when captured, at the batch discount, and settled when its
|
|
400
|
+
result comes. Every hold carries `expires_at`,
|
|
401
|
+
`Smith.config.budget_hold_ttl` (3600 seconds by default) from now, and a
|
|
402
|
+
batch request's wait longer, so a ledger drops a hold whose process died.
|
|
403
|
+
`Smith.config.budget_hold_output_tokens` caps a hold's output part (an
|
|
404
|
+
agent with no `max_tokens` is otherwise held for the model's maximum).
|
|
405
|
+
A tool loop's later rounds are held only by a step whose `retry_on` names
|
|
406
|
+
`BudgetUnavailable`: a refused round pauses the loop, its tool calls and
|
|
407
|
+
results kept with the step's retry (`pending_retry` `continuation`), and
|
|
408
|
+
the retry continues from them, running none of its tools again; any other
|
|
409
|
+
step's later rounds go on unheld and are reconciled. A nested workflow
|
|
410
|
+
without its own ledger charges the enclosing run's; a child run, inline or
|
|
411
|
+
not, its own class's (readiness warns of a child workflow that declares
|
|
412
|
+
none, `:child_run_unbudgeted`). Runtime readiness warns of an agent whose
|
|
413
|
+
model has no price wherever the ledger is charged
|
|
414
|
+
(`:budget_ledger_unpriced`), and its metrics name the scope keys.
|
|
415
|
+
- Testing kit: `require "smith/testing"` (and `smith/testing/minitest`,
|
|
416
|
+
`smith/testing/rspec`). `Smith::Testing.stub_agent(agent, outputs:,
|
|
417
|
+
usage:, stop_reason:, stream:)` answers an agent's requests where they
|
|
418
|
+
would go over the wire, so the tool loop, budget holds, the response
|
|
419
|
+
memo, streaming and output checks run as for a provider's reply. Outputs
|
|
420
|
+
are taken in turn: a String, a Hash as JSON text, or
|
|
421
|
+
`tool_call(ToolClass, **arguments)`, which runs the agent's own tool (a
|
|
422
|
+
tool the agent lacks fails the call). Only the stubbed call's own
|
|
423
|
+
requests are answered, and a batch step's capture and delivery are left
|
|
424
|
+
to batch mode; `Smith::Testing.unstubbed = :refuse` fails an agent's call
|
|
425
|
+
that has no stub (`UnstubbedCall`) instead of sending it.
|
|
426
|
+
`run_until_waiting`, `resume_with` and `expire` drive a
|
|
427
|
+
run through its waits (a durable retry's backoff ends at once);
|
|
428
|
+
`batch_result` and `batch_payload` build a batch delivery. Minitest
|
|
429
|
+
assertions (`assert_waiting_for`, `assert_transitioned_through`,
|
|
430
|
+
`assert_usage_recorded`) and RSpec matchers (`be_waiting_for`,
|
|
431
|
+
`have_transitioned_through`, `have_recorded_usage`) drop the stubs after
|
|
432
|
+
each test.
|
|
433
|
+
- `Smith.config.persistence_max_bytes`: a stored workflow state's bound (nil
|
|
434
|
+
by default). A stored run's step whose result would take it past the
|
|
435
|
+
bound fails with `Smith::PersistedStateTooLarge` (a `WorkflowError`,
|
|
436
|
+
restored as itself) before the result is kept: its usage is recorded, its
|
|
437
|
+
`on_failure` taken, and the run stays storable, strict mode included. A
|
|
438
|
+
tool loop refused for budget does not pause into a state past the bound
|
|
439
|
+
(it goes on unheld). Any other state past it is refused before anything
|
|
440
|
+
is written, before a split-step boundary counts a write as begun (a
|
|
441
|
+
refused checkpoint may be made again).
|
|
442
|
+
- `rails g smith:install` also writes a workflow-state migration and model
|
|
443
|
+
for the `:active_record` adapter (a unique string key, a text payload
|
|
444
|
+
limited so MySQL picks a type that holds it, a `lock_version`, a
|
|
445
|
+
payload-size CHECK a third above `persistence_max_bytes`) and configures
|
|
446
|
+
them; options `--state-model` (a namespaced model gets its table name
|
|
447
|
+
set), `--max-state-bytes` (a whole number, `0` for no bound) and
|
|
448
|
+
`--skip-state-model`.
|
|
449
|
+
- `smith doctor` checks an `:active_record` adapter's model on every run
|
|
450
|
+
(`persistence.state_model`: its `lock_version`, a text payload, a unique
|
|
451
|
+
string key at least 64 characters wide, through
|
|
452
|
+
`ActiveRecordStore#check_schema!`), skipping when the database cannot be
|
|
453
|
+
reached and warning of an encrypted payload, which `replace_exact` now
|
|
454
|
+
refuses with a clear error (it compares payloads in the database).
|
|
455
|
+
|
|
456
|
+
### Changed
|
|
457
|
+
|
|
458
|
+
- The doctor's durability probe writes as runs write (`Workflow#persist!`:
|
|
459
|
+
`store_versioned` where the adapter has it, and `persistence_max_bytes`),
|
|
460
|
+
so it now fails where runs would, an `:active_record` adapter with a
|
|
461
|
+
`persistence_ttl` included.
|
|
462
|
+
|
|
463
|
+
- Smith's OpenAI Responses path (`openai_api_mode: :auto`) given a streaming
|
|
464
|
+
block no longer raises `NotImplementedError`: it completes, then passes the
|
|
465
|
+
block the reply's whole text as one chunk.
|
|
466
|
+
- **Breaking:** workflow context has one shape everywhere.
|
|
467
|
+
- Top-level keys must be Symbols, the rule `write_context` already applied.
|
|
468
|
+
`Workflow.new`, and the class-level helpers that derive a persistence key
|
|
469
|
+
from a context (`restore_or_initialize`, `persisted_state_exists?`,
|
|
470
|
+
`restorable_billing_state?`, `run_persisted!`), raise
|
|
471
|
+
`Smith::WorkflowError` naming the first other key. Before, String keys were
|
|
472
|
+
accepted live and became Symbols only on restore, and a `persistence_key`
|
|
473
|
+
builder saw the caller's context on the class-level path but the
|
|
474
|
+
workflow's own on the instance path.
|
|
475
|
+
- Every Hash key inside a context value becomes the Symbol of its String
|
|
476
|
+
form, at every depth: in `Workflow.new`, at each `write_context` call (so
|
|
477
|
+
`read_context` in the same step already sees it), after hooks that write
|
|
478
|
+
the live context, and on restore. Before, a restore symbolized only
|
|
479
|
+
top-level keys, so `context.dig(:fields, :title)` returned the value live
|
|
480
|
+
and `nil` after a JSON round trip. Integer and other keys become Symbols
|
|
481
|
+
too (`101` becomes `:"101"`, as JSON would make it). Hash subclasses such as
|
|
482
|
+
`HashWithIndifferentAccess` become plain Hashes, and default procs and
|
|
483
|
+
`compare_by_identity` are not kept.
|
|
484
|
+
- Two keys of one Hash that become the same Symbol, a value that contains
|
|
485
|
+
itself, and nesting deeper than 100 levels (JSON's own limit) raise
|
|
486
|
+
`Smith::WorkflowError` instead of keeping whichever key came last or
|
|
487
|
+
overflowing the stack.
|
|
488
|
+
- Values are not converted: Symbols, Times and other non-JSON values still
|
|
489
|
+
come back as Strings after a restore.
|
|
490
|
+
- `Workflow.new` holds a normalized copy, not the caller's Hash.
|
|
491
|
+
- Normalization is `O(N)` in the entries of the value being normalized.
|
|
492
|
+
- **Breaking:** token totals count prompt-cache input. `RunResult#total_tokens`,
|
|
493
|
+
the workflow's token totals, token budgets (`total_tokens`, `token_limit`),
|
|
494
|
+
composite effect totals and `ProviderUsage#total_tokens` add
|
|
495
|
+
`cache_read_tokens` and `cache_write_tokens` to input and output. Before,
|
|
496
|
+
cached input was not counted at all, so a workflow that cached heavily stayed
|
|
497
|
+
far under a token budget it was in fact spending.
|
|
498
|
+
- **Breaking:** `Workflow::AgentResult` gains `cache_read_tokens`,
|
|
499
|
+
`cache_write_tokens`, `thinking_tokens`, `stop_reason`,
|
|
500
|
+
`provider_message_id` and `mode` (the last set only by streaming and batch
|
|
501
|
+
steps), so its `to_h`, `to_a`, `==` and positional deconstruction carry six
|
|
502
|
+
more members.
|
|
503
|
+
- **Breaking:** `Smith::Pricing.validate_catalog!` refuses catalogs 0.11.0
|
|
504
|
+
accepted: cache rates that are not non-negative Integers or Floats, a
|
|
505
|
+
`batch_discount` outside 0 (inclusive) to 1 (exclusive) or not an Integer or
|
|
506
|
+
Float, and a `batch_discount` inside a tier, where it would never apply.
|
|
507
|
+
- **Breaking:** a tiered pricing entry is chosen on a call's total input
|
|
508
|
+
(uncached plus cache read plus cache write). Before, it was chosen on
|
|
509
|
+
uncached input alone, which could select a cheaper tier than the provider
|
|
510
|
+
bills.
|
|
511
|
+
- **Breaking:** Smith's OpenAI Responses adapter reports `input_tokens` with
|
|
512
|
+
cached and cache-written input taken out, as RubyLLM reports every other
|
|
513
|
+
provider, keeping the cache counts separate. Before, its `input_tokens`
|
|
514
|
+
included cached input, which Smith priced at the full input rate; that input
|
|
515
|
+
is now priced at `cache_read_cost_per_token`, 0 when the catalog has none, so
|
|
516
|
+
Responses-path costs fall unless the catalog sets it. Cache counts that are
|
|
517
|
+
not Integers are ignored, and counts larger than the reported input are
|
|
518
|
+
capped to it, so the reported input total is kept.
|
|
519
|
+
- **Breaking:** `append_session_messages!` admits only `user`, `assistant`
|
|
520
|
+
and `tool` messages (see Upgrade notes), and a restored session message or a
|
|
521
|
+
durable composite step's prepared message must have one of those roles or
|
|
522
|
+
`system`. Only Smith and the host's `seed_messages` builder put system
|
|
523
|
+
messages in a session, so the injected-state marker is now recognized only on
|
|
524
|
+
a system message: a user, assistant or tool message that starts with
|
|
525
|
+
`[smith:injected-state]` is left alone instead of being replaced or removed
|
|
526
|
+
as Smith's state.
|
|
527
|
+
- The parallel branch limit check moved into its own graph diagnostic class;
|
|
528
|
+
its code and output are unchanged.
|
|
529
|
+
- **Breaking:** a workflow's `wall_clock` budget bounds the time its steps
|
|
530
|
+
spend running, not the time since `created_at`. Each step gets what is left
|
|
531
|
+
of the budget after the run's persisted `active_seconds`, and adds its own
|
|
532
|
+
running time when it ends. A step that raised, or whose process died, is
|
|
533
|
+
charged on the next restore the time since it began (recorded in the
|
|
534
|
+
checkpoint written before it), at most what is left of the budget. Time spent
|
|
535
|
+
waiting, or between steps that a host advances from a queue, no longer
|
|
536
|
+
counts, so a run that used to fail its deadline while idle now continues.
|
|
537
|
+
Durable composite branches each start from what is left of the budget, and
|
|
538
|
+
their running time is not added back. Payloads gain `awaiting`,
|
|
539
|
+
`active_seconds` and `active_step_started_at`; a payload without them
|
|
540
|
+
restores as not waiting with no active time spent.
|
|
541
|
+
- Rollback: an earlier 0.12 build, and 0.11.0, refuse a persisted failure whose
|
|
542
|
+
family is `wait_timed_out`, `output_failure` or `child_runs_failed`, and a
|
|
543
|
+
waiting run whose wait is a map's or call's (`kind: "children"`); clear or
|
|
544
|
+
settle such runs before rolling back. An earlier build ignores
|
|
545
|
+
`inherited_untrusted_keys`, so a child run restored there loses its parent's
|
|
546
|
+
untrusted marks. It refuses a run waiting on a durable retry
|
|
547
|
+
(`kind: "retry"`) and ignores `pending_retry`, so a run due for a retry
|
|
548
|
+
would start the step over at its first attempt and key generation; settle
|
|
549
|
+
such runs first. It cannot load a class that declares `retry_on ...,
|
|
550
|
+
durable: true` (an unknown keyword), so remove the option first; without it
|
|
551
|
+
the retry sleeps its backoff, up to `max_delay`, in the worker. It ignores
|
|
552
|
+
`inherited_artifact_namespaces`, so a child run restored there cannot read
|
|
553
|
+
its parent's documents, and it sends a seed message with content parts to
|
|
554
|
+
providers as JSON text.
|
|
555
|
+
- A workflow's execution namespace (the persisted UUID that scopes its
|
|
556
|
+
artifacts and now its idempotency keys) is assigned when the workflow is
|
|
557
|
+
built instead of on first artifact use, so `to_state` always carries it,
|
|
558
|
+
even before the first step. Runs restored from copies of one payload
|
|
559
|
+
therefore share artifact scope and idempotency keys; build each run with
|
|
560
|
+
`new`. A payload written without a namespace gets one derived from its
|
|
561
|
+
class, persistence key and creation time, the same on every restore, and a
|
|
562
|
+
stored namespace that is not a UUID raises `Smith::SerializationError`.
|
|
563
|
+
- **Breaking:** Smith's per-step tool context (`Smith::Tool::ScopedContext`)
|
|
564
|
+
holds one more private value, `current_idempotency_scope`.
|
|
565
|
+
`ScopedContext.around` requires the complete context, so a host that builds
|
|
566
|
+
that Hash by hand rather than from `ScopedContext.capture` must include it.
|
|
567
|
+
- **Breaking:** an explicit `persist :a, :b` list is an allow-list for the
|
|
568
|
+
context in every run, persisted or not and nested or not.
|
|
569
|
+
- `write_context` of any other key raises `Smith::UndeclaredContextKey`
|
|
570
|
+
(a `WorkflowError`, with `key` and `persisted_keys`). Before, the write
|
|
571
|
+
took effect in the live run and was dropped at the next checkpoint,
|
|
572
|
+
contrary to the docs.
|
|
573
|
+
- A rejected write fails the step even when the step body rescues the
|
|
574
|
+
error, so it can never become a silently dropped write.
|
|
575
|
+
- Hooks that receive the live context (`before_eval`, `after_completion`,
|
|
576
|
+
guardrails) are checked at the end of their step. A new top-level key must
|
|
577
|
+
be a Symbol the list allows; otherwise the step fails and the context's
|
|
578
|
+
top-level values go back to how they were when the step started. New or
|
|
579
|
+
replaced values take the context's key shape.
|
|
580
|
+
- `persist` refuses keys that are not Symbols with `WorkflowError` at
|
|
581
|
+
definition time. Before, String keys made every write fail on restore.
|
|
582
|
+
- `persist :auto` and workflows with no `persist` declaration accept any
|
|
583
|
+
Symbol key.
|
|
584
|
+
- **Breaking:** restoring a payload whose position the class does not know
|
|
585
|
+
raises instead of restoring a run that reads as terminal.
|
|
586
|
+
- An unknown state, or no state while the class has an initial state,
|
|
587
|
+
raises `Smith::UnknownStateOnRestore` (`workflow`, `persistence_key`,
|
|
588
|
+
`state`). An undeclared pending transition raises
|
|
589
|
+
`Smith::UnknownTransitionOnRestore` (`workflow`, `persistence_key`,
|
|
590
|
+
`transition`). Both are `Smith::Error`s that are neither retryable nor
|
|
591
|
+
retry-forbidden, with bounded messages.
|
|
592
|
+
- A state is known when it is declared with `state` or `initial_state`, or
|
|
593
|
+
named by any transition's `from:` or `to:`. It is matched by its String
|
|
594
|
+
form (`Workflow.known_state?`, `Workflow.resolve_state_name`, `O(S + T)`),
|
|
595
|
+
so String-named states restore as the Strings the class declares. Before,
|
|
596
|
+
a String-named state reached only through transitions restored as an
|
|
597
|
+
undeclared Symbol and read as terminal.
|
|
598
|
+
- The check runs after `migrate_from` and after the `definition_digest`
|
|
599
|
+
check, so a digest mismatch is still reported as one. A migration that
|
|
600
|
+
maps an old name needs a `persistence_schema_version` bump, since
|
|
601
|
+
migrations run only when the stored version is behind; the error message
|
|
602
|
+
says so.
|
|
603
|
+
- Finished runs whose terminal state was renamed are refused as well, by
|
|
604
|
+
`restore`, `restore_or_initialize`, `restorable_billing_state?` and the
|
|
605
|
+
class-level `run_persisted!`. Migrate them, or read the stored JSON
|
|
606
|
+
directly.
|
|
607
|
+
- `stuck_for?` reports a stored run in an unknown state as stuck. Before,
|
|
608
|
+
it reported it as terminal.
|
|
609
|
+
- **Breaking:** a `compute` or `run` step that declares `routes:` must route.
|
|
610
|
+
- If its target state has an outgoing transition, finishing without
|
|
611
|
+
`step.route_to` raises `Smith::UnroutedDeterministicStep` (a
|
|
612
|
+
`WorkflowError`, with `transition` and `routes`), so `on_failure` applies
|
|
613
|
+
and none of the step's writes are kept. The exceptions are a transition
|
|
614
|
+
that declares `on_success`, and one that opts into
|
|
615
|
+
`compute(routes: [...], fallthrough: true)`.
|
|
616
|
+
- A step whose target state has no outgoing transition may finish unrouted,
|
|
617
|
+
and the run ends there, so `routes: [:fail]` can still name only an
|
|
618
|
+
exceptional route.
|
|
619
|
+
- Before, an unrouted step silently took the first transition declared from
|
|
620
|
+
its target state.
|
|
621
|
+
- `fallthrough:` must be `true` or `false`. `fallthrough: true` without
|
|
622
|
+
`routes:` raises `WorkflowError` at definition time.
|
|
623
|
+
- Graph inspection follows the runtime rule, so a transition reachable only
|
|
624
|
+
through the old fall-through is now reported as an `unreachable_transition`
|
|
625
|
+
warning; declare `fallthrough: true` to keep it.
|
|
626
|
+
- Transition snapshots of steps with `routes:` carry
|
|
627
|
+
`deterministic_fallthrough`; other snapshots are unchanged.
|
|
628
|
+
- A class-less `retry_on` does not retry the new error. `retry_on
|
|
629
|
+
Smith::WorkflowError` does, until its attempts run out.
|
|
630
|
+
- **Breaking:** nested workflow failures.
|
|
631
|
+
- A child that raises passes its error to the parent step unchanged,
|
|
632
|
+
whatever its class, so the parent's `retry_on` and
|
|
633
|
+
`Smith::Errors.retryable?` see the child's own error at any depth, and the
|
|
634
|
+
child's billable usage is rolled into the parent on every attempt. Before,
|
|
635
|
+
a raised `Smith::Error` became a plain non-retryable `WorkflowError`, and
|
|
636
|
+
the child's usage was not rolled up.
|
|
637
|
+
- A child that routes its failure through its own `on_failure` has handled
|
|
638
|
+
it. The parent still fails with a non-retryable `WorkflowError` (`nested
|
|
639
|
+
workflow failed: ...`, now bounded to 1 KiB), exactly as before.
|
|
640
|
+
- **Breaking:** tool evidence travels up through nesting. A step that runs
|
|
641
|
+
inside another step's execution (a nested workflow's step) marks the
|
|
642
|
+
enclosing step's tool execution tracker when a tool ran in it. So when a
|
|
643
|
+
tool ran in any child step, a failure of the parent step becomes
|
|
644
|
+
`Smith::ToolOutcomeUncertain` and is never retried, exactly as when the tool
|
|
645
|
+
runs in the parent step itself. Before, a parent could retry a child whose
|
|
646
|
+
tools had already run.
|
|
647
|
+
- **Breaking:** `retry_on` starts no attempt once the workflow's effective
|
|
648
|
+
wall-clock deadline has passed. Before, a step that hit its deadline could
|
|
649
|
+
sleep through every backoff and fail again each time.
|
|
650
|
+
- **Breaking:** `:deterministic_step` traces report `outcome` (`:started`,
|
|
651
|
+
`:success`, `:routed`, `:failed`) instead of `result`. `result` is a
|
|
652
|
+
reserved content key that the default content policy strips, and `outcome`
|
|
653
|
+
matches the `:transition` and `:provider_call` vocabulary. A failure carries
|
|
654
|
+
`error_class`, `error_family`, and `retryable` when known, instead of the
|
|
655
|
+
raw exception message, which could hold content. Hosts that read `result` or
|
|
656
|
+
`error` from these traces, or that name them in a `trace_fields` allowlist
|
|
657
|
+
for `:deterministic_step`, must switch to `outcome` and `error_class`;
|
|
658
|
+
otherwise they receive neither.
|
|
659
|
+
- **Breaking:** `trace_content :redacted` masks content only, and masks it
|
|
660
|
+
completely.
|
|
661
|
+
- The built-in Memory, Logger and OpenTelemetry adapters mask the content
|
|
662
|
+
fields `content`, `prompt`, `response`, `args`, and `result`, and leave
|
|
663
|
+
every other field readable. That includes identifiers such as `model`,
|
|
664
|
+
`tool`, `attempt_id`, `error_class`, and `execution_key`, which defaults
|
|
665
|
+
to the persistence key and may embed host identifiers. Before, they
|
|
666
|
+
masked every String value.
|
|
667
|
+
- Masking is recursive, in the central policy too (`args` and `result`):
|
|
668
|
+
every String and Symbol at any depth of Hashes and Arrays becomes
|
|
669
|
+
`"[REDACTED]"`, so nested content no longer leaks. Numbers, booleans and
|
|
670
|
+
nil are kept.
|
|
671
|
+
- Hosts that relied on `:redacted` to hide identifiers, or fields a custom
|
|
672
|
+
trace adds, must drop them with `trace_fields`.
|
|
673
|
+
|
|
674
|
+
### Fixed
|
|
675
|
+
|
|
676
|
+
- A step's failed attempts no longer outlive it on its thread. Smith added
|
|
677
|
+
them to the thread's list in place, and the step's thread context put that
|
|
678
|
+
same list back when the step ended, so a thread that had made a call outside
|
|
679
|
+
any step carried one step's failed attempts into the next step's budget
|
|
680
|
+
settlement (counted twice, possibly in another run).
|
|
681
|
+
- Budgets settle every billed token. The completed rounds of a tool loop whose
|
|
682
|
+
attempt then failed, and the metered responses of a tool loop in which some
|
|
683
|
+
response reported no usage, were recorded in usage entries and run totals but
|
|
684
|
+
never reached the budget ledger. A failed attempt's own reported cache tokens
|
|
685
|
+
were in its `:provider_call` trace but not in its usage entry, cost, totals or
|
|
686
|
+
budget. Budgets may now be reached sooner than in 0.11.0.
|
|
687
|
+
- Flat pricing entries with String keys (`"input_cost_per_token"`), as a
|
|
688
|
+
catalog loaded from JSON or YAML has, are priced. They were silently
|
|
689
|
+
unpriced, while tiered entries already read String keys; `total_cost` budgets
|
|
690
|
+
that never consumed such calls now do.
|
|
691
|
+
- `docs/CONFIGURATION.md` said a provider-qualified call with only a
|
|
692
|
+
model-only rate raises `Smith::PricingConfigurationError`; `compute_cost`
|
|
693
|
+
never raises and leaves such usage unpriced. The docs, the README example and
|
|
694
|
+
the generator's pricing example now use provider-qualified keys through
|
|
695
|
+
`Smith::Pricing.validate_catalog!`.
|
|
696
|
+
- A failure whose `details` raises an error whose own `message` also raises no
|
|
697
|
+
longer replaces the step's error (`FailureRecord` reads that inner message
|
|
698
|
+
through its guarded reader). This was present in 0.11.0.
|
|
699
|
+
- Classifying a deterministic step's failure for its trace can never replace
|
|
700
|
+
the step's error, and is skipped when no trace adapter is configured.
|
|
701
|
+
- The `compute` example in `docs/PATTERNS.md` called `route_to` twice on one
|
|
702
|
+
path, which raises. It now routes once on each branch and declares its
|
|
703
|
+
routes.
|
|
704
|
+
|
|
705
|
+
## [0.11.0] - 2026-10-07
|
|
706
|
+
|
|
707
|
+
### Added
|
|
708
|
+
|
|
709
|
+
- `fallback_models` accepts a context block, matching the dynamic
|
|
710
|
+
`model { |context| ... }`. `fallback_models { |context| [...] }` is evaluated
|
|
711
|
+
once per agent invocation with the workflow context, and its entries pass
|
|
712
|
+
exactly the static form's validation: every entry needs an explicit
|
|
713
|
+
provider. An unqualified or invalid entry fails the step with
|
|
714
|
+
`Smith::WorkflowError` before any provider attempt, and that error is not
|
|
715
|
+
retried: `Smith::Errors.retryable?` is false for it, so a `retry_on` without
|
|
716
|
+
error classes, `retry_on Smith::AgentError`, and host retries built on
|
|
717
|
+
`Smith::Errors.retryable_classes` all leave it failed, unlike an invalid
|
|
718
|
+
`model` block result, which raises the retryable `Smith::AgentError`. The
|
|
719
|
+
chain keeps its de-duplication. Declaring entries and a block together raises
|
|
720
|
+
`ArgumentError`; a later static declaration clears the block and a later
|
|
721
|
+
block clears the static list; subclasses inherit the block. Graph
|
|
722
|
+
inspection, runtime readiness, and the doctor model check never call the
|
|
723
|
+
block: `fallback_models` returns `nil` for the block form and
|
|
724
|
+
`fallback_models_block` exposes the block. `resolve_fallback_models(context)`
|
|
725
|
+
evaluates the block, or returns the static qualified list (empty when none
|
|
726
|
+
is declared) when no block is configured. Resolution is `O(F)` time and
|
|
727
|
+
space for `F` returned entries. The static form is unchanged.
|
|
728
|
+
- Every `:provider_call` trace carries `agent_name`, the registered name of
|
|
729
|
+
the agent that made the attempt (the value its usage entries carry), so a
|
|
730
|
+
generator and an evaluator sharing one `optimize` round, and so one
|
|
731
|
+
transition and round, stay apart; it is omitted for an unregistered agent.
|
|
732
|
+
When the provider reported usage, the trace also carries `input_tokens` and
|
|
733
|
+
`output_tokens`: the totals of the attempt's usage entries, covering a
|
|
734
|
+
successful completion, a failed attempt's completed tool-loop prefix, and
|
|
735
|
+
the usage a failed attempt's error reported. Both are omitted when the
|
|
736
|
+
provider reported none. A trace adapter can therefore record a complete
|
|
737
|
+
attempt when it ends instead of waiting for a step checkpoint. Token counts
|
|
738
|
+
are metadata, never content: `trace_content false` does not hide them.
|
|
739
|
+
Hosts with a `trace_fields` allowlist for `:provider_call` must add these
|
|
740
|
+
keys to receive them. The fields are documented in `docs/CONFIGURATION.md`.
|
|
741
|
+
- `optimize`'s `before_eval` may reject a candidate. A returned Hash whose
|
|
742
|
+
`accept` (Symbol or String key) is `false` is that round's evaluation: it is
|
|
743
|
+
normalized and validated exactly as evaluator output (`feedback` required,
|
|
744
|
+
a numeric `score` when `improvement_threshold` is set), the evaluator is not
|
|
745
|
+
called for the round, and the loop continues as a rejection, so the
|
|
746
|
+
feedback reaches the generator's refinement turn and `on_exhaustion` applies
|
|
747
|
+
when rounds run out. Any other return value, including `nil` and a Hash
|
|
748
|
+
with `accept: true`, is ignored exactly as before and the evaluator judges.
|
|
749
|
+
A rejected round records only the generator's usage and `:provider_call`.
|
|
750
|
+
Documented in the Evaluator-Optimizer section of `docs/PATTERNS.md`.
|
|
751
|
+
- `optimize`'s verdicts are an output. Each valid round's evaluation is
|
|
752
|
+
recorded as `{ round:, source:, verdict: }`: the optimizer round (the one
|
|
753
|
+
usage entries and traces carry), `:evaluator` or `:before_eval` for the one
|
|
754
|
+
that gave it, and the evaluation as normalized, every field the evaluator's
|
|
755
|
+
schema declares included, as a frozen copy. Once its loop has run, the
|
|
756
|
+
step's record carries its verdicts under `:evaluations` whether the step
|
|
757
|
+
completed or failed (a completion that fails after the loop, such as a split
|
|
758
|
+
step's snapshot, included), the verdict that ends the loop included, each
|
|
759
|
+
also naming its `attempt`, the retry policy's attempt that ran the loop: a
|
|
760
|
+
step `retry_on` runs again starts its loop afresh with rounds from 0, and the
|
|
761
|
+
earlier attempt's verdicts stay before the new ones. `RunResult#evaluations`
|
|
762
|
+
lists every optimize step's verdicts in step, attempt and round order as
|
|
763
|
+
frozen records with the step's `transition`, and
|
|
764
|
+
`OptimizationState#evaluations` gives a callable a frozen copy of its own
|
|
765
|
+
loop's verdicts so far. A step that failed before its loop, and every other
|
|
766
|
+
step, gains no key; a nested workflow's verdicts stay on its own run.
|
|
767
|
+
Verdicts are content, so no trace carries them, and like `steps` they
|
|
768
|
+
belong to the run that executed the step: nothing is persisted and a
|
|
769
|
+
terminal restore has none. Before this, a rejection's feedback lived only in
|
|
770
|
+
the loop and an accepted verdict was discarded, so a host could not see why
|
|
771
|
+
a candidate was sent back. Documented in the Evaluator-Optimizer section of
|
|
772
|
+
`docs/PATTERNS.md`.
|
|
773
|
+
- `Smith::StepInProgressOnRestore` gains `state`, the persisted state the
|
|
774
|
+
interrupted step started from (a Symbol), and `transition`, the next
|
|
775
|
+
transition the payload records (such as a routed one), else `nil`, so a
|
|
776
|
+
host can tell which step died without reading Smith's payload. Both are set
|
|
777
|
+
wherever restore raises it and default to `nil` for any other construction;
|
|
778
|
+
the constructor stays backward compatible and the message gains only
|
|
779
|
+
` state=...` when the state is known.
|
|
780
|
+
|
|
781
|
+
### Changed
|
|
782
|
+
|
|
783
|
+
- Contract tightening: an `optimize` round's verdict, now recorded as an
|
|
784
|
+
output, must hold JSON values only (Hashes with String or Symbol keys,
|
|
785
|
+
Arrays, Strings, Symbols, Integers, finite Floats, true, false, nil), and a
|
|
786
|
+
step's verdicts together must fit the limits a split step's record is held
|
|
787
|
+
to (`ExecutionResultSnapshot`'s depth, node and byte caps). A round whose
|
|
788
|
+
verdict breaks either, most likely a rejection `before_eval` returned
|
|
789
|
+
holding an object or a very large feedback, now fails the step with
|
|
790
|
+
`Smith::WorkflowError` at that round, in every run mode, keeping the
|
|
791
|
+
verdicts recorded before it. Before, such a verdict was never recorded and
|
|
792
|
+
the loop went on.
|
|
793
|
+
- **Breaking:** a deterministic step's `write_outcome` is deep-symbolized
|
|
794
|
+
when written, matching restore. A live `RunResult#outcome_payload` (and
|
|
795
|
+
`RunResult#outcome`) now has Symbol keys at every depth, so a host reading
|
|
796
|
+
String keys from a live payload must switch to Symbol keys. Restored
|
|
797
|
+
payloads were already symbolized, so an outcome now reads identically live
|
|
798
|
+
and after a JSON restore. Persisted JSON is unchanged.
|
|
799
|
+
- Contract tightening: `ActiveRecordStore` refuses to write with a non-nil
|
|
800
|
+
TTL instead of silently ignoring it, since it has no expiry column.
|
|
801
|
+
`store`, `store_versioned`, and `replace_exact` raise `ArgumentError`
|
|
802
|
+
before touching the database. Reads and deletes take no TTL, and resolving
|
|
803
|
+
the adapter never refuses, so `restore`, `persisted_state_exists?`,
|
|
804
|
+
`clear_persisted!`, `stuck_for?`, and `heartbeat_age` keep working while a
|
|
805
|
+
TTL is configured. A global or per-workflow `persistence_ttl` with this
|
|
806
|
+
adapter is refused by the initial checkpoint that `run_persisted!` and
|
|
807
|
+
`advance_persisted!` write before their first step, so no step runs before
|
|
808
|
+
the refusal. A nil TTL (the default) is unaffected. `smith doctor
|
|
809
|
+
--durability` fails a new `durability.ttl` check when the configured
|
|
810
|
+
adapter is an `ActiveRecordStore` and `Smith.config.persistence_ttl` is
|
|
811
|
+
set. The README, `docs/CONFIGURATION.md`, and the `persistence_ttl` DSL
|
|
812
|
+
comment no longer claim this adapter honours TTL.
|
|
813
|
+
- Contract tightening: workflow and agent `budget` declarations raise
|
|
814
|
+
`ArgumentError`, naming the accepted keys, when given a key Smith does not
|
|
815
|
+
read; previously such a key was accepted and ignored. Workflow budgets
|
|
816
|
+
accept `total_tokens`, `token_limit`, `total_cost`, `tool_calls`, and
|
|
817
|
+
`wall_clock`; agent budgets accept `token_limit`, `cost`, `wall_clock`,
|
|
818
|
+
`tool_calls`, `total_tokens`, and `total_cost`. `wall_clock` is in seconds:
|
|
819
|
+
the README budget example used the ignored `wall_clock_ms: 30_000` and now
|
|
820
|
+
uses `wall_clock: 30`. The check is `O(K)` time and space for `K` declared
|
|
821
|
+
keys. State written by 0.10.0 keeps restoring after the host removes such a
|
|
822
|
+
key: 0.10.0 reserved every declared key at each agent call, so it could
|
|
823
|
+
persist a consumed entry like `"wall_clock_ms" => 0`, and restore now drops
|
|
824
|
+
consumed entries (String or Symbol keys) for dimensions the workflow no
|
|
825
|
+
longer declares. An undeclared dimension has no limit, so dropping it is
|
|
826
|
+
safe. The filter is `O(D + C)` for `D` declared and `C` consumed keys.
|
|
827
|
+
|
|
828
|
+
### Fixed
|
|
829
|
+
|
|
830
|
+
- `DeterministicStep#last_output` returned `nil` after a JSON restore,
|
|
831
|
+
because a restored session message carries a Symbol key with a String role
|
|
832
|
+
value. `last_output` now accepts a Symbol or String role under either key
|
|
833
|
+
form. Restored messages keep the shape 0.10.0 gives them (Symbol keys, JSON
|
|
834
|
+
values such as `role: "assistant"`).
|
|
835
|
+
- Restored failures keep their type. `ProviderPermanentFailure`,
|
|
836
|
+
`BudgetExceeded`, and `GuardrailFailed` get their own failure families
|
|
837
|
+
(`provider_permanent_failure`, `budget_exceeded`, `guardrail_failed`) where
|
|
838
|
+
they were `other`, and restore as their own classes where they restored as
|
|
839
|
+
`RuntimeError`. `BlankAgentOutputError` keeps the `agent_error` family and
|
|
840
|
+
restores as itself where it restored as a plain `AgentError`.
|
|
841
|
+
`ProviderPermanentFailure` and `BlankAgentOutputError` gain `details` (each
|
|
842
|
+
value bounded to 512 bytes) and `from_details`, so `provider`, `model_id`,
|
|
843
|
+
`source_error_class`, `agent_name`, and `model_used` survive a restore.
|
|
844
|
+
Details stay JSON-normalized, and `FailureRecordValidator` rejects malformed
|
|
845
|
+
details, or one of these classes claiming another family, with
|
|
846
|
+
`Smith::PersistedFailureInvalid`. Records written by earlier versions (these
|
|
847
|
+
classes under `other`, or `BlankAgentOutputError` without details) keep
|
|
848
|
+
restoring exactly as before. The new families also appear as `error_family`
|
|
849
|
+
on failed `:transition` traces and `StepFailed` events. Composite branch
|
|
850
|
+
failure evidence (`Workflow::Composite::ErrorEvidence`) still classifies
|
|
851
|
+
these three errors as `other`. Rollback: 0.10.0 checks `error_family` and
|
|
852
|
+
`error_cause_family` against its own family list, so any persisted payload
|
|
853
|
+
whose `last_failed_step`, or its cause, carries one of the new families
|
|
854
|
+
fails to restore under 0.10.0, whether the run is terminal or not; clear or
|
|
855
|
+
settle such runs before rolling back.
|
|
856
|
+
- A failed or aborted attempt's `:provider_call` trace now carries
|
|
857
|
+
`error_class`: the exception's class name, bounded to 512 bytes by the
|
|
858
|
+
diagnostic text helper (`anonymous_error` for an anonymous class). It also
|
|
859
|
+
carries `error_cause_class`, the class name of the error's direct cause
|
|
860
|
+
(`Exception#cause`, else the error's `wrapped_exception`), captured the same
|
|
861
|
+
way and omitted when there is no cause: a `Faraday::ConnectionFailed`
|
|
862
|
+
wraps both never-connected causes (`Errno::ECONNREFUSED`, `SocketError`,
|
|
863
|
+
`Net::OpenTimeout`) and lost-after-sending ones (`EOFError`,
|
|
864
|
+
`Errno::ECONNRESET`), and a host deciding whether an attempt may have been
|
|
865
|
+
billed needs to tell them apart. Messages never ride the trace. Hosts with a
|
|
866
|
+
`trace_fields` allowlist for `:provider_call` must add `:error_class` and
|
|
867
|
+
`:error_cause_class` to receive them.
|
|
868
|
+
- `inject_state` injects only non-blank text. A formatter returning `nil` or
|
|
869
|
+
whitespace-only text no longer adds a bare `[smith:injected-state]` system
|
|
870
|
+
message, and when a formatter that returned text now returns blank, the
|
|
871
|
+
stale injected-state message is removed from session history. Injection
|
|
872
|
+
remains one linear scan over the session.
|
|
873
|
+
- Restore and timestamp asymmetries. The `provider` in
|
|
874
|
+
`last_agent_execution`, read by `DeterministicStep#last_agent_provider`,
|
|
875
|
+
restores as the Symbol a live run holds where it restored as a String.
|
|
876
|
+
`created_at` is now written with microsecond precision (`iso8601(6)`), so a
|
|
877
|
+
`wall_clock` deadline no longer fires up to a second early; the early firing
|
|
878
|
+
affected live runs too, since the deadline is always computed from the
|
|
879
|
+
stored string. `updated_at` is written with the same precision at every
|
|
880
|
+
step, so it never reads earlier than `created_at`. Payloads with
|
|
881
|
+
whole-second timestamps still restore.
|
|
882
|
+
- `optimize` sends a structured candidate (a Hash or Array from an
|
|
883
|
+
`output_schema` generator) to the evaluator, and replays it as the
|
|
884
|
+
refinement round's assistant turn, as JSON, using the serialization Smith
|
|
885
|
+
applies at the provider boundary, where it sent Ruby `inspect` notation.
|
|
886
|
+
String candidates are unchanged.
|
|
887
|
+
- A budget reservation of the reported remaining amount no longer raises
|
|
888
|
+
`BudgetExceeded` far below the limit. `Budget::Ledger#remaining` returned
|
|
889
|
+
the Float nearest the exact remaining amount, and reserving it re-reads its
|
|
890
|
+
shortest decimal form, which after Float-priced costs sits above the exact
|
|
891
|
+
amount about half the time: a serial step on a $0.50 `total_cost` budget
|
|
892
|
+
failed its third call at $0.0707. `remaining` now returns a value that
|
|
893
|
+
never reads back above the exact amount, so it may read one ulp lower than
|
|
894
|
+
before. `Budget::Ledger#remaining_share(key, parts)` is a new public method
|
|
895
|
+
that returns one of `parts` equal shares that all fit, which fan-out branch
|
|
896
|
+
estimates now use (a `total_cost: 0.2` budget split across three branches
|
|
897
|
+
failed with nothing spent). Integers keep floor division; a Float steps
|
|
898
|
+
down in `O(1)` ulp steps. Amounts stay Integer or Float in and JSON-safe
|
|
899
|
+
numerics out.
|
|
900
|
+
- `smith doctor`'s `persistence.capabilities` check reports an adapter that
|
|
901
|
+
fails to resolve as a failed "Persistence adapter configuration is invalid"
|
|
902
|
+
check carrying the resolution error, where it reported "No persistence
|
|
903
|
+
adapter configured".
|
|
904
|
+
- A swallowed `on_step:` callback error is logged with its class after the
|
|
905
|
+
message (`Smith::Workflow on_step callback error: <message> (<class>)`),
|
|
906
|
+
where only the message was logged. It is still swallowed.
|
|
907
|
+
|
|
7
908
|
## [0.10.0] - 2026-08-24
|
|
8
909
|
|
|
9
910
|
### Added
|