smith-agents 0.11.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +698 -0
- data/README.md +621 -15
- data/UPSTREAM_PROPOSAL.md +1 -0
- data/docs/CONFIGURATION.md +80 -14
- data/docs/PATTERNS.md +30 -13
- data/docs/PERSISTENCE.md +733 -5
- data/docs/TOOLS_AND_GUARDRAILS.md +24 -3
- data/lib/generators/smith/install/install_generator.rb +63 -2
- data/lib/generators/smith/install/templates/create_workflow_states.rb.tt +31 -0
- data/lib/generators/smith/install/templates/smith.rb.tt +39 -8
- data/lib/generators/smith/install/templates/workflow_state.rb.tt +19 -0
- data/lib/smith/agent/completion.rb +32 -6
- data/lib/smith/agent/completion_usage_recording.rb +23 -14
- data/lib/smith/agent/content_bytes.rb +98 -0
- data/lib/smith/agent/content_parts.rb +149 -0
- data/lib/smith/agent/invocation_preparation.rb +37 -14
- data/lib/smith/agent/lifecycle.rb +16 -5
- data/lib/smith/agent/observed_model.rb +29 -0
- data/lib/smith/agent/output_contract_check.rb +53 -0
- data/lib/smith/agent/output_integrity.rb +137 -0
- data/lib/smith/agent/output_policy.rb +92 -0
- data/lib/smith/agent/output_recovery.rb +129 -0
- data/lib/smith/agent/provider_attempt.rb +6 -0
- data/lib/smith/agent/provider_call_timing.rb +13 -4
- data/lib/smith/agent/provider_completion.rb +35 -32
- data/lib/smith/agent/provider_failure_handling.rb +5 -1
- data/lib/smith/agent/provider_request.rb +36 -0
- data/lib/smith/agent/provider_streaming.rb +155 -0
- data/lib/smith/agent/provider_usage.rb +70 -3
- data/lib/smith/agent/usage_entry_recording.rb +4 -2
- data/lib/smith/agent/usage_traces.rb +9 -2
- data/lib/smith/agent/usage_tracking.rb +64 -51
- data/lib/smith/agent.rb +5 -0
- data/lib/smith/artifacts/active_record_store.rb +79 -0
- data/lib/smith/artifacts/file.rb +45 -18
- data/lib/smith/artifacts/memory.rb +47 -36
- data/lib/smith/artifacts/s3.rb +126 -0
- data/lib/smith/artifacts/scoped_store.rb +55 -7
- data/lib/smith/artifacts/store_rules.rb +49 -0
- data/lib/smith/artifacts.rb +98 -1
- data/lib/smith/batch/message_shape.rb +36 -0
- data/lib/smith/batch.rb +166 -0
- data/lib/smith/budget/amount.rb +25 -0
- data/lib/smith/budget/continuation.rb +153 -0
- data/lib/smith/budget/host_ledger.rb +137 -0
- data/lib/smith/budget/ledger.rb +4 -2
- data/lib/smith/budget/ledger_state_transition.rb +7 -5
- data/lib/smith/budget/request_hold.rb +211 -0
- data/lib/smith/budget/spent_reservation.rb +18 -0
- data/lib/smith/child_run.rb +41 -0
- data/lib/smith/child_runs_failed.rb +56 -0
- data/lib/smith/content.rb +96 -0
- data/lib/smith/context/session.rb +14 -2
- data/lib/smith/context/state_injection.rb +10 -5
- data/lib/smith/context.rb +28 -0
- data/lib/smith/doctor/checks/artifacts.rb +103 -0
- data/lib/smith/doctor/checks/configuration.rb +0 -11
- data/lib/smith/doctor/checks/durability.rb +20 -18
- data/lib/smith/doctor/checks/state_model.rb +63 -0
- data/lib/smith/doctor/installer.rb +9 -2
- data/lib/smith/doctor.rb +4 -0
- data/lib/smith/errors.rb +249 -2
- data/lib/smith/idempotency_key/canonical.rb +74 -0
- data/lib/smith/idempotency_key.rb +145 -0
- data/lib/smith/launchers/inline.rb +46 -0
- data/lib/smith/message.rb +76 -0
- data/lib/smith/models/tool_routing.rb +47 -44
- data/lib/smith/output_failure.rb +130 -0
- data/lib/smith/persistence_adapters/active_record_exact_store.rb +43 -0
- data/lib/smith/pricing/catalog_validation.rb +91 -0
- data/lib/smith/pricing.rb +95 -64
- data/lib/smith/providers/anthropic/batches.rb +116 -0
- data/lib/smith/providers/batch_exchange.rb +77 -0
- data/lib/smith/providers/openai/responses.rb +45 -20
- data/lib/smith/providers/output_limit.rb +68 -0
- data/lib/smith/providers/request_holds.rb +139 -0
- data/lib/smith/providers/response_facts.rb +108 -0
- data/lib/smith/providers/schema_constraint_check.rb +141 -0
- data/lib/smith/providers/schema_constraints.rb +87 -0
- data/lib/smith/providers/schema_rules.rb +66 -0
- data/lib/smith/providers/stop_reason.rb +39 -0
- data/lib/smith/providers/stream_facts.rb +192 -0
- data/lib/smith/response_memo.rb +152 -0
- data/lib/smith/setting_constructors.rb +47 -0
- data/lib/smith/streaming/json_field.rb +136 -0
- data/lib/smith/streaming/json_string.rb +91 -0
- data/lib/smith/streaming/session.rb +77 -0
- data/lib/smith/streaming/think_tags.rb +67 -0
- data/lib/smith/streaming.rb +64 -0
- data/lib/smith/testing/checks.rb +58 -0
- data/lib/smith/testing/hooks.rb +85 -0
- data/lib/smith/testing/minitest.rb +37 -0
- data/lib/smith/testing/rspec.rb +52 -0
- data/lib/smith/testing/stub.rb +110 -0
- data/lib/smith/testing.rb +158 -0
- data/lib/smith/tool/scoped_context.rb +9 -12
- data/lib/smith/tool.rb +14 -0
- data/lib/smith/trace/logger.rb +1 -1
- data/lib/smith/trace/memory.rb +1 -1
- data/lib/smith/trace/open_telemetry.rb +1 -1
- data/lib/smith/trace.rb +11 -4
- data/lib/smith/untrusted.rb +74 -0
- data/lib/smith/version.rb +2 -2
- data/lib/smith/workflow/active_clock.rb +61 -0
- data/lib/smith/workflow/agent_result.rb +37 -0
- data/lib/smith/workflow/artifact_integration.rb +57 -7
- data/lib/smith/workflow/await_step.rb +42 -0
- data/lib/smith/workflow/batch_charges.rb +74 -0
- data/lib/smith/workflow/batch_execution.rb +149 -0
- data/lib/smith/workflow/batch_holds.rb +73 -0
- data/lib/smith/workflow/batch_wait.rb +134 -0
- data/lib/smith/workflow/budget_integration.rb +11 -5
- data/lib/smith/workflow/budget_pause.rb +51 -0
- data/lib/smith/workflow/child_run_declarations.rb +127 -0
- data/lib/smith/workflow/child_run_execution.rb +150 -0
- data/lib/smith/workflow/child_run_results.rb +55 -0
- data/lib/smith/workflow/composite/effects.rb +37 -6
- data/lib/smith/workflow/composite/error.rb +1 -1
- data/lib/smith/workflow/composite/error_evidence.rb +9 -9
- data/lib/smith/workflow/composite/input.rb +11 -4
- data/lib/smith/workflow/context_keys.rb +77 -0
- data/lib/smith/workflow/deadline_enforcement.rb +1 -1
- data/lib/smith/workflow/deterministic_execution.rb +61 -10
- data/lib/smith/workflow/deterministic_step.rb +55 -6
- data/lib/smith/workflow/dsl.rb +59 -0
- data/lib/smith/workflow/durability.rb +139 -30
- data/lib/smith/workflow/evaluator_optimizer.rb +12 -6
- data/lib/smith/workflow/event_integration.rb +9 -0
- data/lib/smith/workflow/execution.rb +30 -23
- data/lib/smith/workflow/execution_binding_resolution.rb +13 -0
- data/lib/smith/workflow/failure_reconstructor.rb +27 -4
- data/lib/smith/workflow/failure_record.rb +6 -1
- data/lib/smith/workflow/failure_record_validator.rb +11 -4
- data/lib/smith/workflow/fanout_execution.rb +5 -8
- data/lib/smith/workflow/graph/batch_diagnostic.rb +83 -0
- data/lib/smith/workflow/graph/branch_limit_diagnostic.rb +41 -0
- data/lib/smith/workflow/graph/budget_ledger_diagnostic.rb +126 -0
- data/lib/smith/workflow/graph/durable_retry_diagnostic.rb +50 -0
- data/lib/smith/workflow/graph/execution_successors.rb +16 -2
- data/lib/smith/workflow/graph/inherited_untrusted_diagnostics.rb +77 -0
- data/lib/smith/workflow/graph/isolated_input_diagnostic.rb +50 -0
- data/lib/smith/workflow/graph/nested_readiness_diagnostics.rb +43 -4
- data/lib/smith/workflow/graph/output_key_diagnostic.rb +53 -0
- data/lib/smith/workflow/graph/output_schema_diagnostic.rb +49 -0
- data/lib/smith/workflow/graph/retry_policy_diagnostic.rb +1 -0
- data/lib/smith/workflow/graph/run_launcher_diagnostic.rb +30 -0
- data/lib/smith/workflow/graph/runtime_binding_diagnostic_builder.rb +9 -3
- data/lib/smith/workflow/graph/runtime_binding_diagnostics.rb +7 -3
- data/lib/smith/workflow/graph/runtime_readiness_metrics.rb +8 -1
- data/lib/smith/workflow/graph/runtime_readiness_report_builder.rb +9 -2
- data/lib/smith/workflow/graph/runtime_readiness_traversal.rb +7 -4
- data/lib/smith/workflow/graph/stream_diagnostic.rb +95 -0
- data/lib/smith/workflow/graph/transition_contract.rb +19 -0
- data/lib/smith/workflow/graph/transition_contract_attributes.rb +25 -2
- data/lib/smith/workflow/graph/transition_contract_configurations.rb +24 -3
- data/lib/smith/workflow/graph/transition_diagnostics.rb +11 -24
- data/lib/smith/workflow/graph/transition_snapshot.rb +40 -39
- data/lib/smith/workflow/graph/untrusted_input_diagnostic.rb +57 -0
- data/lib/smith/workflow/graph/waiting_child_diagnostic.rb +24 -0
- data/lib/smith/workflow/graph.rb +12 -0
- data/lib/smith/workflow/guarded_step_execution.rb +48 -3
- data/lib/smith/workflow/message_batch.rb +6 -0
- data/lib/smith/workflow/message_roles.rb +25 -0
- data/lib/smith/workflow/message_value_normalizer.rb +2 -1
- data/lib/smith/workflow/nested_execution.rb +47 -4
- data/lib/smith/workflow/orchestrator_worker.rb +9 -3
- data/lib/smith/workflow/parallel_execution.rb +10 -0
- data/lib/smith/workflow/persistence.rb +136 -13
- data/lib/smith/workflow/remembered_calls.rb +92 -0
- data/lib/smith/workflow/retry_execution.rb +68 -9
- data/lib/smith/workflow/retry_wait.rb +134 -0
- data/lib/smith/workflow/split_step_persistence/checkpoint.rb +3 -0
- data/lib/smith/workflow/split_step_persistence/checkpoint_state.rb +7 -0
- data/lib/smith/workflow/split_step_persistence/composite_branch_execution.rb +4 -1
- data/lib/smith/workflow/split_step_persistence/composite_preparation.rb +3 -1
- data/lib/smith/workflow/split_step_persistence/dispatch_claim.rb +1 -0
- data/lib/smith/workflow/split_step_persistence/preparation.rb +5 -0
- data/lib/smith/workflow/split_step_persistence/state_snapshot.rb +1 -0
- data/lib/smith/workflow/step_completion.rb +36 -7
- data/lib/smith/workflow/step_context.rb +21 -0
- data/lib/smith/workflow/step_io.rb +132 -0
- data/lib/smith/workflow/step_stream.rb +52 -0
- data/lib/smith/workflow/thread_context_snapshot.rb +2 -0
- data/lib/smith/workflow/transition.rb +267 -13
- data/lib/smith/workflow/transition_actionability.rb +2 -0
- data/lib/smith/workflow/usage_entry.rb +24 -3
- data/lib/smith/workflow/wait_completion.rb +131 -0
- data/lib/smith/workflow/wait_execution.rb +180 -0
- data/lib/smith/workflow/wait_record.rb +157 -0
- data/lib/smith/workflow.rb +113 -17
- data/lib/smith.rb +101 -45
- metadata +88 -1
data/UPSTREAM_PROPOSAL.md
CHANGED
|
@@ -138,6 +138,7 @@ What stays Smith-owned (orchestration concerns, not provider-API concerns):
|
|
|
138
138
|
## Tracking
|
|
139
139
|
|
|
140
140
|
- RubyLLM PR #770 (OpenAI `/v1/responses` support) is the related upstream track. Smith's vendored `Smith::Providers::OpenAI::Responses` retires when #770 merges.
|
|
141
|
+
- One divergence from #770 to carry upstream or keep on retirement: the vendored parser reports `input_tokens` with cached and cache-written input taken out (the SMITH-AUTHORED `response_usage`), as RubyLLM reports every other provider. #770 passes the Responses API's `input_tokens`, which includes cached input, while also reporting `cached_tokens`; Smith totals and prices both, so retiring the vendored file without this would count cached input twice. `spec/smith/workflow/cache_usage_spec.rb` parses a Responses body end to end to catch that.
|
|
141
142
|
- This proposal (capability profiles + before_complete) is a separate, additive RubyLLM RFC. Once accepted, Smith files a migration PR to consume it.
|
|
142
143
|
|
|
143
144
|
## Current RubyLLM Docs Check
|
data/docs/CONFIGURATION.md
CHANGED
|
@@ -9,6 +9,7 @@ Use this for shared runtime services:
|
|
|
9
9
|
- artifact backend
|
|
10
10
|
- tracing
|
|
11
11
|
- pricing catalog
|
|
12
|
+
- run launcher (`run_launcher`, for child runs)
|
|
12
13
|
- logger
|
|
13
14
|
|
|
14
15
|
### 2. Agent configuration: `Smith::Agent`
|
|
@@ -23,10 +24,14 @@ Use agent classes for invocation behavior:
|
|
|
23
24
|
- `budget`
|
|
24
25
|
- `guardrails`
|
|
25
26
|
- `output_schema`
|
|
27
|
+
- `output_contract`, `output_repairs`
|
|
28
|
+
- `on_truncation`, `on_refusal`
|
|
26
29
|
- `data_volume`
|
|
27
30
|
- `fallback_models`
|
|
28
31
|
- `register_as`
|
|
29
32
|
|
|
33
|
+
The output macros are described in the README, Agent outputs.
|
|
34
|
+
|
|
30
35
|
### 3. Workflow configuration: `Smith::Workflow`
|
|
31
36
|
|
|
32
37
|
Use workflow classes for orchestration behavior:
|
|
@@ -40,6 +45,9 @@ Use workflow classes for orchestration behavior:
|
|
|
40
45
|
- `guardrails`
|
|
41
46
|
- `context_manager`
|
|
42
47
|
|
|
48
|
+
Transition steps that start runs of their own (`map`, `call`, `spawn`) need
|
|
49
|
+
`Smith.config.run_launcher`; see the PERSISTENCE guide, Child Runs.
|
|
50
|
+
|
|
43
51
|
### If You Are Unsure Where Something Goes
|
|
44
52
|
|
|
45
53
|
- "Which model should this agent use?" -> agent class
|
|
@@ -68,8 +76,14 @@ closed during agent configuration.
|
|
|
68
76
|
|
|
69
77
|
```ruby
|
|
70
78
|
Smith.configure do |config|
|
|
71
|
-
config.artifact_store =
|
|
72
|
-
config.
|
|
79
|
+
config.artifact_store = :s3 # S3 or R2; or :active_record, :file, :memory, a store
|
|
80
|
+
config.artifact_options = {
|
|
81
|
+
bucket: "smith-artifacts",
|
|
82
|
+
endpoint: "https://<account>.r2.cloudflarestorage.com", region: "auto",
|
|
83
|
+
access_key_id: ENV.fetch("R2_ACCESS_KEY_ID"), secret_access_key: ENV.fetch("R2_SECRET_ACCESS_KEY"),
|
|
84
|
+
request_checksum_calculation: "when_required", response_checksum_validation: "when_required" # R2
|
|
85
|
+
}
|
|
86
|
+
config.artifact_retention = 30 * 86_400
|
|
73
87
|
config.artifact_encryption = :none
|
|
74
88
|
config.artifact_tenant_isolation = false
|
|
75
89
|
|
|
@@ -86,12 +100,23 @@ Smith.configure do |config|
|
|
|
86
100
|
config.trace_provider_calls = true
|
|
87
101
|
config.trace_attribution = true
|
|
88
102
|
|
|
89
|
-
config.pricing =
|
|
103
|
+
config.pricing = Smith::Pricing.validate_catalog!(
|
|
90
104
|
%w[openai gpt-4.1-nano] => {
|
|
91
105
|
input_cost_per_token: 0.0000001,
|
|
92
|
-
output_cost_per_token: 0.0000004
|
|
106
|
+
output_cost_per_token: 0.0000004,
|
|
107
|
+
cache_read_cost_per_token: 0.000000025
|
|
93
108
|
}
|
|
94
|
-
|
|
109
|
+
)
|
|
110
|
+
|
|
111
|
+
# Streamed steps (`execute Agent, stream: :answer`): a callable that takes the
|
|
112
|
+
# step's info and returns its sink, or nil to stream nothing.
|
|
113
|
+
config.stream_sinks = { answer: ->(info) { AnswerStream.new(info[:execution_key]) } }
|
|
114
|
+
|
|
115
|
+
# Batch steps (`execute Agent, mode: :batch`): the host's queue of requests.
|
|
116
|
+
config.batch_queue = HostBatchQueue.new
|
|
117
|
+
|
|
118
|
+
# How long a request's hold on a workflow's budget_ledger lasts unsettled.
|
|
119
|
+
config.budget_hold_ttl = 3600
|
|
95
120
|
|
|
96
121
|
config.logger = Logger.new($stdout)
|
|
97
122
|
end
|
|
@@ -99,16 +124,37 @@ end
|
|
|
99
124
|
|
|
100
125
|
Provider-qualified model calls require provider-qualified pricing keys. Smith
|
|
101
126
|
also accepts the string form `"openai/gpt-4.1-nano"`. Legacy model-only keys
|
|
102
|
-
remain available only for calls whose provider is genuinely unknown;
|
|
103
|
-
|
|
104
|
-
|
|
127
|
+
remain available only for calls whose provider is genuinely unknown; a
|
|
128
|
+
provider-qualified call never reads a model-only rate, and usage with no
|
|
129
|
+
applicable rate stays unpriced (a nil cost), without raising.
|
|
130
|
+
`Smith::Pricing.validate_catalog!` is the raising check: it rejects model-only
|
|
131
|
+
keys, keys that collide, non-Hash entries, optional rates that are not
|
|
132
|
+
non-negative Integers or Floats, and a `batch_discount` that is out of range or
|
|
133
|
+
sits inside a tier.
|
|
134
|
+
|
|
135
|
+
Each entry, or each tier of a tiered entry, prices `input_cost_per_token` and
|
|
136
|
+
`output_cost_per_token`, and optionally `cache_read_cost_per_token` and
|
|
137
|
+
`cache_write_cost_per_token` for input read from or written to a prompt cache,
|
|
138
|
+
which providers report apart from uncached input. A tier without its own cache
|
|
139
|
+
rate uses the entry's. An entry may also carry `batch_discount`, the fraction
|
|
140
|
+
taken off the cost of a call made in batch mode (`0.5` for half price). Cache
|
|
141
|
+
rates and the batch discount are configured per catalog and count as `0` when
|
|
142
|
+
absent: cache tokens then add nothing to a cost (the `:cost` trace says so with
|
|
143
|
+
`cache_unpriced: true`), and a batch call costs the full rate. Tiers are chosen
|
|
144
|
+
on a call's total input, uncached plus cache read plus cache write, so a cached
|
|
145
|
+
call can pass every bounded tier and stay unpriced when the last tier is
|
|
146
|
+
bounded. One cache write rate prices every cache lifetime; a provider that bills
|
|
147
|
+
longer-lived cache writes higher (Anthropic's one-hour cache) needs the rate of
|
|
148
|
+
the lifetime the host uses. Keys may be Symbols or Strings, as a catalog loaded
|
|
149
|
+
from JSON or YAML has.
|
|
105
150
|
|
|
106
151
|
### What Each `Smith.configure` Setting Is For
|
|
107
152
|
|
|
108
153
|
| Setting | What it controls | Typical first use |
|
|
109
154
|
| --- | --- | --- |
|
|
110
|
-
| `artifact_store` | Where large handoff payloads are stored |
|
|
111
|
-
| `
|
|
155
|
+
| `artifact_store` | Where large handoff payloads and documents are stored: `:s3` (S3, R2), `:active_record`, `:file`, `:memory`, a store or a store class | Required outside `test_mode`; a shared, durable store whenever runs wait, retry or move between processes |
|
|
156
|
+
| `artifact_options` | Options the named store is built with (`:s3`: `bucket:`, `prefix:`, `client:` or `region:`/`endpoint:`/credentials; `:active_record`: `model:`; `:file`: `dir:`; any: `namespace:`) | Set with `artifact_store` |
|
|
157
|
+
| `artifact_retention` | Default retention window for artifact expiry checks | Longer than the longest wait of any run |
|
|
112
158
|
| `artifact_encryption` | Metadata-level encryption policy flag | Leave at default until you wire a real backend |
|
|
113
159
|
| `artifact_tenant_isolation` | Require namespaced artifact writes | Enable in multi-tenant systems |
|
|
114
160
|
| `trace_adapter` | Where structural traces go | Use `Smith::Trace::Memory` or `Smith::Trace::Logger` first |
|
|
@@ -118,20 +164,26 @@ observes a provider but finds only a model-only rate, it raises
|
|
|
118
164
|
| `trace_provider_calls` | Emit one timed `:provider_call` trace per provider attempt | Usually leave on |
|
|
119
165
|
| `trace_cost` | Emit one `:cost` trace per completed agent invocation | Emits only once pricing is configured |
|
|
120
166
|
| `trace_fields` | Allowlist structural trace fields | Use when you want tighter trace output |
|
|
121
|
-
| `trace_content` | Whether content appears in traces | Leave `false` first |
|
|
167
|
+
| `trace_content` | Whether content appears in traces: `false` drops content fields, `:redacted` replaces every String and Symbol inside them, at any depth, with `"[REDACTED]"`, and `true` keeps them. The built-in adapters treat `content`, `prompt`, `response`, `args`, and `result` as content; the central policy, which runs first, masks `args` and `result`. Every other field passes unmasked, including identifiers such as model, execution key (which defaults to the persistence key), tool name, and error class, and any field a custom trace adds: drop those with `trace_fields` if they must not leave the process | Leave `false` first |
|
|
122
168
|
| `trace_attribution` | Merge ambient execution attribution into trace payloads | Usually leave on |
|
|
123
169
|
| `pricing` | Best-known model-call cost catalog | Add once you care about `total_cost` |
|
|
124
170
|
| `logger` | Smith's runtime logger | Usually the first setting to add |
|
|
171
|
+
| `batch_queue` | Takes the requests `mode: :batch` steps send: `enqueue(custom_id:, await_token:, request:, execution_key:)`, idempotent on `custom_id`; the host submits them (`Smith::Providers::Anthropic::Batches`) and resumes each run with its results | Set before any workflow batches; readiness reports a missing one |
|
|
172
|
+
| `response_memo` | Keeps what a step's provider calls got (`fetch(key)`, `store(key, record)`), so a step replayed after its process died reuses it instead of paying again (README, "Response memo") | Set it where replays happen: lax idempotency mode, or a host that runs a crashed step again |
|
|
173
|
+
| `budget_hold_ttl` | Seconds a provider request's hold on a workflow's `budget_ledger` lasts unsettled (default 3600): each hold's `expires_at` is this from now, and a batch request's is its 25-hour wait longer. Past it the request's process is taken to have died | Raise it above the longest request (its retries and stream included) |
|
|
174
|
+
| `budget_hold_output_tokens` | The most output tokens a request's hold on a workflow's `budget_ledger` counts (default nil: the request's own `max_tokens`, else the model's maximum). Only the hold changes, not the request; a longer reply is reconciled at what it cost | Set it when agents leave `max_tokens` unset and a small budget would refuse calls on their huge holds |
|
|
175
|
+
| `stream_sinks` | The sinks `execute Agent, stream: :name` steps send text to, by name: callables that take a step's info (`stream`, `execution_key`, `transition`) and return that step's sink, or nil to stream nothing | Register each name a workflow streams to; readiness reports a missing one |
|
|
125
176
|
| `persistence_adapter` | Adapter for durable workflow state | `:redis`, `:rails_cache`, `:active_record`, `:memory`, or a custom object |
|
|
126
177
|
| `persistence_options` | Per-adapter options (client, namespace, model, columns) | See "Built-In Persistence Adapters" |
|
|
127
178
|
| `persistence_ttl` | Global TTL for persisted state (Integer/Float seconds; nil = no expiry). `:active_record` cannot expire state, so its writes refuse a non-nil TTL (reads and deletes still work, and `smith doctor --durability` flags the combination) | Set when long-tail abandoned workflows accumulate in storage |
|
|
179
|
+
| `persistence_max_bytes` | The most bytes a stored workflow state may take (nil, the default: no bound). A stored run's step whose result would take it past the bound fails with `Smith::PersistedStateTooLarge` (its `key`, `bytes` and `limit`) before the result is kept, as with any error: its usage is recorded and its `on_failure` taken, so the run stays storable, strict mode included. A tool loop refused for budget does not pause into a state past it (it goes on unheld). Any other state past it is refused before anything is written. The answer that did not fit is not kept. `rails g smith:install` sets it to 1 MiB, with a CHECK a third above it on the payload column | Set it above the largest output a step keeps, with a database CHECK on the payload, so an oversized state fails as a typed step error instead of a database one |
|
|
128
180
|
| `persistence_retry_policy` | Exponential-backoff policy for transient adapter I/O failures | Defaults to `{ attempts: 3, base_delay: 0.1, max_delay: 1.0 }` |
|
|
129
181
|
| `retry_attempt_limit` | Maximum attempts accepted by workflow and persistence retry policies | Defaults to `100` |
|
|
130
182
|
| `parallel_branch_limit` | Maximum branches accepted by one homogeneous or heterogeneous fan-out | Defaults to `1_000` |
|
|
131
183
|
| `parallel_concurrency` | Maximum active branch threads within one parallel execution | Defaults to `8` |
|
|
132
184
|
| `parallel_nesting_limit` | Maximum re-entrant fan-out depth; hard-capped at `256` | Defaults to `64` |
|
|
133
|
-
| `test_mode` | Auto-select `:memory` adapter when `persistence_adapter` is nil | Enable in `spec_helper.rb` to skip Redis/cache wiring in tests |
|
|
134
|
-
| `openai_api_mode` | `:auto` routes (gpt-5 family + tools + thinking) via `/v1/responses` using Smith's vendored Responses adapter (
|
|
185
|
+
| `test_mode` | Auto-select the in-process `:memory` adapter when `persistence_adapter` is nil, and an in-process artifact store when `artifact_store` is nil | Enable in `spec_helper.rb` to skip Redis/cache/bucket wiring in tests |
|
|
186
|
+
| `openai_api_mode` | `:auto` routes (gpt-5 family + tools + thinking) via `/v1/responses` using Smith's vendored Responses adapter, which does not stream (a streamed step gets each reply as one delta); `:off` drops incompatible tools instead | Leave `:auto` (default) unless you need token streaming with the (gpt-5 + tools + thinking) combo, in which case set `:off` for graceful tool-dropping |
|
|
135
187
|
| `trace_normalizer` | Emit `:normalizer_decision` trace events from `Smith::Models::Normalizer` | Useful when debugging cross-provider request shaping |
|
|
136
188
|
| `ruby_llm_model_registry` | `:database` to require an AR-backed RubyLLM model registry; `:bundled` for the JSON fallback | Leave at default unless you've migrated to DB-backed |
|
|
137
189
|
|
|
@@ -146,7 +198,21 @@ of the agent that made the attempt, so a generator and an evaluator sharing an
|
|
|
146
198
|
optimize round stay apart. `input_tokens` and `output_tokens` total the
|
|
147
199
|
provider-reported usage that the attempt's usage entries record (a completed
|
|
148
200
|
tool-loop prefix, and a failed attempt's own reported usage); both are omitted
|
|
149
|
-
when the provider reported none.
|
|
201
|
+
when the provider reported none. `input_tokens` counts uncached input only;
|
|
202
|
+
`cache_read_tokens` and `cache_write_tokens` are the prompt-cache input beside
|
|
203
|
+
it, and `thinking_tokens` are already inside `output_tokens`. `stop_reason` and
|
|
204
|
+
`provider_message_id` are those of the attempt's last provider response, read
|
|
205
|
+
from its raw body when the body is one Smith recognizes (Anthropic, OpenAI Chat
|
|
206
|
+
Completions and Responses, Gemini, Bedrock Converse), or from its stream when
|
|
207
|
+
the step streamed. Each of these is omitted when the provider did not report
|
|
208
|
+
it. `mode` is `:stream` for a streamed attempt, `:batch` for one answered by a
|
|
209
|
+
batch result, and omitted for a synchronous one; `request_changed: true` marks
|
|
210
|
+
a batch result whose call rendered a request other than the one sent (a date in
|
|
211
|
+
its prompt, data read afresh) and took the result all the same
|
|
212
|
+
(`on_request_change: :keep`, the default). `:token_usage` and `:cost` traces carry
|
|
213
|
+
`cache_read_tokens` and `cache_write_tokens` too, and `:cost` adds
|
|
214
|
+
`cache_unpriced: true` when cache tokens were priced at 0 for lack of a rate. A
|
|
215
|
+
failed or aborted attempt adds `error_class`,
|
|
150
216
|
and `error_cause_class` for the error's direct cause (`Exception#cause`, else a
|
|
151
217
|
wrapping error's `wrapped_exception`, such as the `EOFError` inside a
|
|
152
218
|
`Faraday::ConnectionFailed`), omitted when there is none. Class names and token
|
data/docs/PATTERNS.md
CHANGED
|
@@ -325,6 +325,16 @@ end
|
|
|
325
325
|
What you get:
|
|
326
326
|
|
|
327
327
|
- the child workflow's final output becomes the parent step output
|
|
328
|
+
- a child that raises its failure passes the error to the parent step
|
|
329
|
+
unchanged, after its billable usage is rolled into the parent, so the
|
|
330
|
+
parent's `retry_on` and `Smith::Errors.retryable?` treat it as the child's
|
|
331
|
+
own error, at any nesting depth
|
|
332
|
+
- a child that routes its failure through its own `on_failure` has handled it:
|
|
333
|
+
the parent step fails with a non-retryable `Smith::WorkflowError`
|
|
334
|
+
(`nested workflow failed: <child message>`, bounded to 1 KiB)
|
|
335
|
+
- tool evidence travels up: when a tool ran in any of the child's steps, a
|
|
336
|
+
failure of the parent step becomes `Smith::ToolOutcomeUncertain` and is never
|
|
337
|
+
retried, exactly as when the tool runs in the parent step itself
|
|
328
338
|
- parent step count stays parent-scoped
|
|
329
339
|
- parent and child share the outer budget ledger
|
|
330
340
|
- nested best-known token/cost totals roll up into the parent result
|
|
@@ -379,7 +389,8 @@ end
|
|
|
379
389
|
|
|
380
390
|
`before_eval: ->(state, context) { ... }` runs after each candidate is
|
|
381
391
|
generated and before the evaluator is called, and may write to the workflow
|
|
382
|
-
context
|
|
392
|
+
context (a new top-level key must be a Symbol that the context's `persist` list
|
|
393
|
+
allows; see Deterministic Steps, Context writes). When it returns a Hash whose `accept` (Symbol or String key) is
|
|
383
394
|
`false`, that Hash is the round's evaluation: Smith validates it exactly as
|
|
384
395
|
evaluator output (`feedback` is required, and a numeric `score` when
|
|
385
396
|
`improvement_threshold` is set), skips the evaluator for that round, and
|
|
@@ -456,11 +467,11 @@ Today that means:
|
|
|
456
467
|
|
|
457
468
|
| Loop kind | Executable? | Smith primitive | Boundary |
|
|
458
469
|
|---|---|---|---|
|
|
459
|
-
| Retry | Yes | `retry_on` | Local transition retry only. Durable scheduling
|
|
470
|
+
| Retry | Yes | `retry_on` | Local transition retry only. Durable scheduling stays with the host. A step that passes its idempotency keys (`docs/PERSISTENCE.md`, Replay Safety) sends every attempt's requests under the same keys, so a receiver that honours them can drop repeats. |
|
|
460
471
|
| Evaluator-Optimizer | Yes | `optimize` | Bounded refinement inside one transition using structured evaluator output. |
|
|
461
472
|
| Deterministic repair | Not native | Handwritten `compute` / `run` when exact semantics are owned by the workflow author. Deterministic route targets can be declared with `routes: [...]`. | No first-class repair contract or persisted repair counts yet. |
|
|
462
473
|
| Guarded state re-entry | Not native | Handwritten `compute` / `run` may declare and route to named transitions with `routes: [...]`. | No Smith-owned entry-count ledger, re-entry guard contract, or mutation policy yet. |
|
|
463
|
-
|
|
|
474
|
+
| Wait for an event or a time | Yes | `await`, `delay` | Bounded by a required timeout; Smith records the wait and its deadline. Delivery (`resume!`), expiry scheduling (`expire_wait!`) and wake-up stay with the host. Smith must not model durable polling with sleeps, busy-waits, or `max_transitions` cycling. |
|
|
464
475
|
|
|
465
476
|
Do not hide a durable wait or unbounded repair policy inside `compute` / `run`.
|
|
466
477
|
Those primitives are synchronous deterministic steps inside the current workflow
|
|
@@ -468,7 +479,7 @@ runner. They are appropriate for local verification, normalization, failure
|
|
|
468
479
|
classification, explicit routing, and bounded retry-adjacent checks; they are
|
|
469
480
|
not a durable scheduler.
|
|
470
481
|
|
|
471
|
-
If a host or compiler wants deterministic repair, polling
|
|
482
|
+
If a host or compiler wants deterministic repair, polling, or guarded
|
|
472
483
|
state re-entry as reusable graph contracts, the contract must be added explicitly
|
|
473
484
|
before code generation claims executability. Deterministic `routes: [...]`
|
|
474
485
|
annotations make handwritten route targets inspectable, but they do not by
|
|
@@ -578,17 +589,17 @@ Use `compute` for steps that check prior output and decide what happens next.
|
|
|
578
589
|
|
|
579
590
|
```ruby
|
|
580
591
|
transition :verify_research, from: :gathered, to: :verified do
|
|
581
|
-
compute do |step|
|
|
592
|
+
compute(routes: %i[structure finish_terminal_failure]) do |step|
|
|
582
593
|
if step.tool_results.any? { |t| t[:captured]&.dig(:retryable) }
|
|
583
594
|
step.fail!("research temporarily unavailable", retryable: true)
|
|
584
595
|
end
|
|
585
596
|
|
|
586
|
-
|
|
597
|
+
if step.last_output
|
|
598
|
+
step.route_to(:structure)
|
|
599
|
+
else
|
|
587
600
|
step.write_outcome(kind: :terminal_failure, payload: { message: "no usable research output" })
|
|
588
601
|
step.route_to(:finish_terminal_failure)
|
|
589
602
|
end
|
|
590
|
-
|
|
591
|
-
step.route_to(:structure)
|
|
592
603
|
end
|
|
593
604
|
|
|
594
605
|
on_failure :fail
|
|
@@ -617,7 +628,9 @@ The yielded step object exposes a narrow, read-heavy surface:
|
|
|
617
628
|
| `step.context` | `step.write_context(key, value)` |
|
|
618
629
|
| `step.read_context(key)` | `step.write_outcome(kind:, payload:)` |
|
|
619
630
|
| `step.last_output` / `step.output` | `step.route_to(:transition_name)` |
|
|
620
|
-
| | `step.fail!(msg, retryable:, kind:, details:)` |
|
|
631
|
+
| | `step.fail!(msg, retryable:, kind:, details:, new_idempotency_key:)` |
|
|
632
|
+
| `step.idempotency_key(scope = nil)` | |
|
|
633
|
+
| `step.attempt` | |
|
|
621
634
|
| `step.tool_results` | |
|
|
622
635
|
| `step.session_messages` | |
|
|
623
636
|
| `step.current_state` | |
|
|
@@ -625,13 +638,17 @@ The yielded step object exposes a narrow, read-heavy surface:
|
|
|
625
638
|
|
|
626
639
|
### Behavior
|
|
627
640
|
|
|
628
|
-
- **Routing**: `step.route_to` overrides `on_success
|
|
629
|
-
- **
|
|
641
|
+
- **Routing**: `step.route_to` overrides `on_success`, and can be called once per step. If neither is set, normal state-based resolution applies. Named transitions that do not exist fail loudly with `WorkflowError`. When a deterministic transition declares `compute(routes: [...])` or `run(routes: [...])`, `step.route_to` must target one of those names; graph inspection validates the annotated targets before runtime.
|
|
642
|
+
- **Declared routes must be taken**: a step that declares `routes:` and finishes without `step.route_to` raises `Smith::UnroutedDeterministicStep` (a `WorkflowError`, so `on_failure` applies and no write is kept) when its target state has an outgoing transition, unless the transition declares `on_success` or `compute(routes: [...], fallthrough: true)`. A step whose target state has no outgoing transition may finish unrouted: the run ends there, as declared, so `compute(routes: [:fail])` can name only its exceptional route. Graph inspection follows the same rule: such a step reaches its routes and its `on_success`, and the first transition from its target state only with `fallthrough: true`; a transition reachable only that way is reported unreachable until the step declares `fallthrough: true`. `fallthrough: true` without `routes:` raises `WorkflowError` at definition time. A class-less `retry_on` does not retry `UnroutedDeterministicStep`, but `retry_on Smith::WorkflowError` does, until its attempts run out.
|
|
643
|
+
- **Failure**: `step.fail!` raises `Smith::DeterministicStepFailure` (extends `WorkflowError`) with `retryable`, `kind`, `details` and `new_idempotency_key` metadata. Routes through `on_failure` like any other step failure. `new_idempotency_key: true` declares that this step's own requests in the attempt took no effect, so this step's `retry_on` sends them under new keys; it never moves an enclosing workflow's retry.
|
|
644
|
+
- **Idempotency keys**: `step.idempotency_key(scope = nil)` is a key for a request the step sends outside the system, the same on every attempt of the step and after a restore, unique to the run, the step and the scope. `step.attempt` is the attempt under way, from 1. See `docs/PERSISTENCE.md`, Replay Safety.
|
|
630
645
|
- **Outcome**: `step.write_outcome(kind:, payload:)` stores a workflow-owned terminal payload without smuggling it through context. The payload is persisted with the workflow and surfaced on `RunResult.outcome`, `RunResult.outcome_kind`, and `RunResult.outcome_payload`.
|
|
646
|
+
- **Context writes**: `step.write_context(key, value)` takes a Symbol key. The value is stored at the call in the context's key shape: every Hash key at every depth becomes the Symbol of its String form, which is what a restore produces, so `step.read_context` later in the same step, the next step, and a restored run all read the same keys. Values are not converted (see `docs/PERSISTENCE.md`). Two keys that become the same Symbol, a value that contains itself, or nesting deeper than 100 levels fail the write with `Smith::WorkflowError`.
|
|
647
|
+
- **Persist list as an allow-list**: when the workflow's `Smith::Context` declares `persist :a, :b`, writing any other key raises `Smith::UndeclaredContextKey`, in every run, persisted or not and nested or not, so the list is the context's declared shape rather than a filter applied at checkpoints. A rejected write fails the step even when the step body rescues the error. With `persist :auto`, or no `persist` declaration, any Symbol key may be written. Hooks that receive the live context (`before_eval`, `after_completion`, guardrails) are held to the same rules at the end of their step: a new top-level key must be a Symbol the list allows, a violation fails the step and puts the context's top-level values back as they were when the step started, and new or replaced values take the context's key shape.
|
|
631
648
|
- **Context reads**: `step.context` returns an isolated snapshot of the workflow context at step start. Mutating that snapshot does not mutate workflow state. `step.read_context(key)` returns a merged view — pending `write_context` values override the snapshot. Use `read_context` when you need read-after-write coherence within the same step.
|
|
632
649
|
- **No output**: Deterministic steps produce no session message output. `last_output` continues to mean the last agent output.
|
|
633
650
|
- **No budget**: No tokens or cost consumed.
|
|
634
|
-
- **Persistence**: Context writes and written outcomes survive `to_state`/`from_state`. The block itself (a Proc) lives on the class-level Transition and is never serialized.
|
|
635
|
-
- **Trace**: Emits `:deterministic_step` traces
|
|
651
|
+
- **Persistence**: Context writes (always to a persisted key, see above) and written outcomes survive `to_state`/`from_state`. The block itself (a Proc) lives on the class-level Transition and is never serialized.
|
|
652
|
+
- **Trace**: Emits `:deterministic_step` traces whose `outcome` is `:started`, `:success`, `:routed`, or `:failed` (`outcome`, not `result`, which the content policy strips). A failure carries `error_class`, `error_family`, and `retryable` when known, never the error message. When a step writes an outcome, the trace includes `outcome_kind`.
|
|
636
653
|
- **Mutual exclusivity**: `compute` and `run` cannot be combined with `execute`, `route`, `workflow`, `optimize`, or `orchestrate`. A transition declares exactly one primary execution body.
|
|
637
654
|
- **No hidden scheduler**: `compute` and `run` execute synchronously inside the current runner. Use them for bounded deterministic logic, not for durable polling or wake-up loops.
|