belay 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +19 -1
- data/app/controllers/belay/stats_controller.rb +2 -0
- data/app/views/belay/stats/_budget_banner.html.erb +6 -0
- data/app/views/belay/stats/_table.html.erb +2 -0
- data/app/views/belay/stats/index.html.erb +4 -0
- data/lib/belay/configuration.rb +58 -0
- data/lib/belay/daily_stats.rb +32 -5
- data/lib/belay/engine.rb +9 -0
- data/lib/belay/executor.rb +83 -4
- data/lib/belay/gemini_provider.rb +158 -0
- data/lib/belay/openai_provider.rb +246 -0
- data/lib/belay/run.rb +44 -8
- data/lib/belay/version.rb +1 -1
- data/lib/belay/webhook_job.rb +110 -0
- data/lib/belay/webhooks.rb +42 -0
- data/lib/belay.rb +4 -0
- data/lib/generators/belay/install/install_generator.rb +11 -0
- data/lib/generators/belay/install/templates/add_budget_alert_sent_at_to_belay_runs.rb.tt +9 -0
- metadata +8 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 16138d86493d56b1c9baef13c6b5039519a01a2382679f134dc7db31f721ad39
|
|
4
|
+
data.tar.gz: c044de960b0983f6bbf0d5965983d89bf1b3d6b2923decebc2e2f71907196e91
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 77143cfb6dd262d9b763a0c7c3166dec5d8fc5366fa1e613575f3ea83a1f289c448f578a279ddaf44c45fccba17d89d1fc4e81894be89450bbb3f071460f7d3e
|
|
7
|
+
data.tar.gz: 806fae91af2d064e8458a94f47aea3ea9562dc3b2dc00f454ee0fad2ad14b4b7d540e76348f65e04a8cf3b98906bf653fecd1a6c4ad9de634567c301ad35db6c
|
data/README.md
CHANGED
|
@@ -132,7 +132,7 @@ ctx.llm(model:, prompt:, schema: nil, cacheable: false)
|
|
|
132
132
|
|
|
133
133
|
Calls `Belay.config.provider`, validates the response against `schema:` provider-agnostically, and returns `response.content` directly as the step's own output.
|
|
134
134
|
|
|
135
|
-
**→ [docs/providers.md](docs/providers.md)** -
|
|
135
|
+
**→ [docs/providers.md](docs/providers.md)** - ships `AnthropicProvider`, `OpenAIProvider`, and `GeminiProvider`; bring your own for anything else.
|
|
136
136
|
**→ [docs/caching.md](docs/caching.md)** - `cacheable: true` and when it's actually safe to use.
|
|
137
137
|
|
|
138
138
|
## Dashboard
|
|
@@ -146,6 +146,24 @@ A runs index, a run detail page with the full step timeline, a replay diff view,
|
|
|
146
146
|
|
|
147
147
|
**→ [docs/dashboard.md](docs/dashboard.md)** for mounting, wiring in your own auth (Devise/Pundit/CanCanCan/anything), and what each page shows.
|
|
148
148
|
|
|
149
|
+
## Instrumentation
|
|
150
|
+
|
|
151
|
+
Belay publishes `ActiveSupport::Notifications` events for the run/step/gate lifecycle (`belay.run.started`, `belay.step.completed`, `belay.gate.paused`, and others) - the same mechanism Rails itself uses for `sql.active_record`, so you can pipe what Belay is doing into your own logs/APM without polling the dashboard.
|
|
152
|
+
|
|
153
|
+
**→ [docs/instrumentation.md](docs/instrumentation.md)** for the full event list and payloads.
|
|
154
|
+
|
|
155
|
+
## Webhooks
|
|
156
|
+
|
|
157
|
+
Built on top of instrumentation: set `config.webhook_url` and a gate pausing, a step getting flagged `needs_review`, or a run completing each POST a JSON body to it - retried automatically, optionally HMAC-signed, no polling or a subscriber block of your own required.
|
|
158
|
+
|
|
159
|
+
**→ [docs/webhooks.md](docs/webhooks.md)** for configuration, signature verification, and retry behavior.
|
|
160
|
+
|
|
161
|
+
## Cost budgets
|
|
162
|
+
|
|
163
|
+
`config.run_cost_budget_cents` fires `belay.run.budget_exceeded` the first time a single run's total cost crosses it; `config.daily_cost_budget_cents` shows a banner on the dashboard's Stats page once today's spend does. Both nil (off) by default.
|
|
164
|
+
|
|
165
|
+
**→ [docs/cost-budgets.md](docs/cost-budgets.md)** for both thresholds and why they're separate.
|
|
166
|
+
|
|
149
167
|
## Roadmap
|
|
150
168
|
|
|
151
169
|
See [ROADMAP.md](ROADMAP.md).
|
|
@@ -9,6 +9,8 @@ module Belay
|
|
|
9
9
|
class StatsController < ApplicationController
|
|
10
10
|
def index
|
|
11
11
|
@daily_stats = Belay::DailyStats.for_range
|
|
12
|
+
@daily_cost_budget_cents = Belay.config.daily_cost_budget_cents
|
|
13
|
+
@today_cost_cents = Belay::DailyStats.today_cost_cents
|
|
12
14
|
end
|
|
13
15
|
end
|
|
14
16
|
end
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
<div class="mb-6 rounded-lg border border-amber-500/25 bg-amber-500/5 p-4" role="alert">
|
|
2
|
+
<p class="text-sm text-amber-200">
|
|
3
|
+
Today's spend (<%= number_to_currency(today_cost_cents / 100.0) %>) has crossed the configured daily budget of
|
|
4
|
+
<%= number_to_currency(daily_cost_budget_cents / 100.0) %>.
|
|
5
|
+
</p>
|
|
6
|
+
</div>
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
<th class="px-4 py-2 text-left text-xs font-medium uppercase tracking-wide text-slate-500">Runs</th>
|
|
7
7
|
<th class="px-4 py-2 text-left text-xs font-medium uppercase tracking-wide text-slate-500">Real API calls</th>
|
|
8
8
|
<th class="px-4 py-2 text-left text-xs font-medium uppercase tracking-wide text-slate-500">Per 100 runs</th>
|
|
9
|
+
<th class="px-4 py-2 text-left text-xs font-medium uppercase tracking-wide text-slate-500">Spend</th>
|
|
9
10
|
</tr>
|
|
10
11
|
</thead>
|
|
11
12
|
<tbody class="divide-y divide-slate-800">
|
|
@@ -15,6 +16,7 @@
|
|
|
15
16
|
<td class="stat-runs px-4 py-2 font-mono text-xs text-slate-500"><%= day.runs %></td>
|
|
16
17
|
<td class="stat-calls px-4 py-2 font-mono text-xs text-slate-500"><%= day.real_calls %></td>
|
|
17
18
|
<td class="stat-rate px-4 py-2 font-mono text-xs font-medium text-slate-100"><%= day.calls_per_100_runs %></td>
|
|
19
|
+
<td class="stat-cost px-4 py-2 font-mono text-xs text-slate-300"><%= number_to_currency(day.cost_cents / 100.0) %></td>
|
|
18
20
|
</tr>
|
|
19
21
|
<% end %>
|
|
20
22
|
</tbody>
|
|
@@ -6,6 +6,10 @@
|
|
|
6
6
|
</p>
|
|
7
7
|
</div>
|
|
8
8
|
|
|
9
|
+
<% if @daily_cost_budget_cents && @today_cost_cents > @daily_cost_budget_cents %>
|
|
10
|
+
<%= render "budget_banner", today_cost_cents: @today_cost_cents, daily_cost_budget_cents: @daily_cost_budget_cents %>
|
|
11
|
+
<% end %>
|
|
12
|
+
|
|
9
13
|
<% if @daily_stats.any? %>
|
|
10
14
|
<%= render "chart", daily_stats: @daily_stats %>
|
|
11
15
|
<%= render "table", daily_stats: @daily_stats %>
|
data/lib/belay/configuration.rb
CHANGED
|
@@ -47,7 +47,57 @@ module Belay
|
|
|
47
47
|
# own comment), so the host app owns wherever this string actually comes from.
|
|
48
48
|
attr_accessor :custom_css
|
|
49
49
|
|
|
50
|
+
# Where Belay::WebhookJob POSTs a JSON body ({event:, payload:}) for every event
|
|
51
|
+
# in webhook_events below. nil by default - opt-in, same reasoning as provider/
|
|
52
|
+
# authenticate_with: a host app that never sets this gets no webhook behavior at
|
|
53
|
+
# all, not a silently-failing POST to nowhere. Read at delivery time (inside the
|
|
54
|
+
# subscriber block and inside WebhookJob#perform), not captured when the
|
|
55
|
+
# subscription is installed - so it can be set in config/initializers/belay.rb
|
|
56
|
+
# regardless of initializer load order relative to Belay::Engine's own.
|
|
57
|
+
attr_accessor :webhook_url
|
|
58
|
+
|
|
59
|
+
# Optional HMAC-SHA256 secret. When set, every webhook request carries an
|
|
60
|
+
# X-Belay-Signature header (hex digest of the raw JSON body) so the receiving
|
|
61
|
+
# endpoint can verify the request actually came from this app, the same pattern
|
|
62
|
+
# Stripe/GitHub webhooks use. nil by default - signing is extra integration work
|
|
63
|
+
# for a receiver that may not need it (e.g. a private endpoint on a trusted
|
|
64
|
+
# network); unsigned is still a reasonable default, matching webhook_url's own
|
|
65
|
+
# opt-in-everything posture.
|
|
66
|
+
attr_accessor :webhook_secret
|
|
67
|
+
|
|
68
|
+
# Which belay.* events actually get POSTed. A plain Array a host app can
|
|
69
|
+
# overwrite wholesale (push onto it, or replace it entirely) - not every event
|
|
70
|
+
# Belay fires is something worth waking up a human/on-call for; this starts
|
|
71
|
+
# narrow (gate pause, needs_review, run completion - exactly what
|
|
72
|
+
# FUTURE_ROADMAP.md's webhooks item asks for) rather than defaulting to "every
|
|
73
|
+
# belay.* event," which would surprise anyone who just wants "tell me when a
|
|
74
|
+
# human is needed or a run finishes," not every auto-approved gate too.
|
|
75
|
+
attr_accessor :webhook_events
|
|
76
|
+
|
|
77
|
+
# Cents. nil by default - opt-in, same posture as every other threshold here. Once
|
|
78
|
+
# a single run's total cost_cents (every step's, summed - see Run#cost_cents)
|
|
79
|
+
# crosses this, Executor fires belay.run.budget_exceeded exactly once for that
|
|
80
|
+
# run (not once per step that happens to still be over budget afterward). Not in
|
|
81
|
+
# webhook_events by default even once set - a host app that wants this pushed to
|
|
82
|
+
# its webhook adds "belay.run.budget_exceeded" itself; setting a budget purely
|
|
83
|
+
# for the dashboard banner below shouldn't silently start firing webhooks nobody
|
|
84
|
+
# asked for.
|
|
85
|
+
attr_accessor :run_cost_budget_cents
|
|
86
|
+
|
|
87
|
+
# Cents. nil by default. Purely a dashboard concern (unlike run_cost_budget_cents
|
|
88
|
+
# above, this fires no event) - Belay::StatsController compares it against
|
|
89
|
+
# today's total cost_cents (Belay::DailyStats.for_range) and the Stats page shows
|
|
90
|
+
# a banner once today's total crosses it.
|
|
91
|
+
attr_accessor :daily_cost_budget_cents
|
|
92
|
+
|
|
50
93
|
def initialize
|
|
94
|
+
assign_execution_defaults!
|
|
95
|
+
assign_webhook_and_budget_defaults!
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
private
|
|
99
|
+
|
|
100
|
+
def assign_execution_defaults!
|
|
51
101
|
@lease_duration = 300
|
|
52
102
|
@heartbeat_interval = 30
|
|
53
103
|
@provider = nil
|
|
@@ -56,5 +106,13 @@ module Belay
|
|
|
56
106
|
@rule_approval_threshold = 3
|
|
57
107
|
@custom_css = nil
|
|
58
108
|
end
|
|
109
|
+
|
|
110
|
+
def assign_webhook_and_budget_defaults!
|
|
111
|
+
@webhook_url = nil
|
|
112
|
+
@webhook_secret = nil
|
|
113
|
+
@webhook_events = %w[belay.gate.paused belay.step.needs_review belay.run.completed]
|
|
114
|
+
@run_cost_budget_cents = nil
|
|
115
|
+
@daily_cost_budget_cents = nil
|
|
116
|
+
end
|
|
59
117
|
end
|
|
60
118
|
end
|
data/lib/belay/daily_stats.rb
CHANGED
|
@@ -7,7 +7,7 @@ module Belay
|
|
|
7
7
|
# ActiveRecord model of its own: there's no belay_daily_stats table, this is
|
|
8
8
|
# a read-only view over belay_runs/belay_steps' own created_at timestamps.
|
|
9
9
|
class DailyStats
|
|
10
|
-
Row = Struct.new(:date, :runs, :real_calls, :calls_per_100_runs, keyword_init: true)
|
|
10
|
+
Row = Struct.new(:date, :runs, :real_calls, :calls_per_100_runs, :cost_cents, keyword_init: true)
|
|
11
11
|
|
|
12
12
|
# Calls are grouped by the STEP's own day, not the owning run's - a run
|
|
13
13
|
# can pause on a gate for a real, unbounded amount of time (nothing times
|
|
@@ -25,9 +25,12 @@ module Belay
|
|
|
25
25
|
def self.for_range
|
|
26
26
|
calls_by_day = calls_by_day_counts
|
|
27
27
|
runs_by_day = runs_by_day_counts
|
|
28
|
+
cost_by_day = cost_cents_by_day
|
|
28
29
|
|
|
29
|
-
days = (calls_by_day.keys + runs_by_day.keys).uniq.sort
|
|
30
|
-
days.map
|
|
30
|
+
days = (calls_by_day.keys + runs_by_day.keys + cost_by_day.keys).uniq.sort
|
|
31
|
+
days.map do |date|
|
|
32
|
+
build_row(date, runs_by_day.fetch(date, 0), calls_by_day.fetch(date, 0), cost_by_day.fetch(date, 0))
|
|
33
|
+
end
|
|
31
34
|
end
|
|
32
35
|
|
|
33
36
|
def self.calls_by_day_counts
|
|
@@ -40,6 +43,19 @@ module Belay
|
|
|
40
43
|
end
|
|
41
44
|
private_class_method :runs_by_day_counts
|
|
42
45
|
|
|
46
|
+
# Grouped by the step's own day, same reasoning calls_by_day_counts' comment
|
|
47
|
+
# above already gives for calls - a step's cost is incurred when the call it
|
|
48
|
+
# paid for actually happened, not whenever its owning run started. Not scoped to
|
|
49
|
+
# llm_call/cache_hit: false the way calls_by_day_counts is - cost_cents is only
|
|
50
|
+
# ever non-nil on a real, billed llm_call anyway (a cache hit's own cost_cents is
|
|
51
|
+
# explicitly 0, per Context#fetch_cached_llm_response's own comment, not nil), so
|
|
52
|
+
# a plain SUM across every step's cost_cents already excludes everything that
|
|
53
|
+
# shouldn't count without a separate WHERE.
|
|
54
|
+
def self.cost_cents_by_day
|
|
55
|
+
normalized_counts(Step.group("DATE(created_at)").sum(:cost_cents))
|
|
56
|
+
end
|
|
57
|
+
private_class_method :cost_cents_by_day
|
|
58
|
+
|
|
43
59
|
# .group("DATE(...)").count's hash keys come back as plain Strings on some
|
|
44
60
|
# adapter/Rails version combinations, real Date objects on others - a raw
|
|
45
61
|
# SQL group-by fragment (not an actual typed column) gives ActiveRecord
|
|
@@ -51,8 +67,19 @@ module Belay
|
|
|
51
67
|
end
|
|
52
68
|
private_class_method :normalized_counts
|
|
53
69
|
|
|
54
|
-
|
|
55
|
-
|
|
70
|
+
# Deliberately NOT @daily_stats.find { |row| row.date == Date.current } at the
|
|
71
|
+
# call site - every date above comes from Postgres's own DATE(created_at), not
|
|
72
|
+
# Ruby's Date.current, and the two can disagree for part of the day whenever the
|
|
73
|
+
# app's Time.zone differs from the DB session's own timezone setting. Asking
|
|
74
|
+
# Postgres's own CURRENT_DATE here instead of Ruby's Date.current means both
|
|
75
|
+
# sides of "is this today" are computed the same way, in the same session - they
|
|
76
|
+
# can never disagree, regardless of which timezone either happens to be in.
|
|
77
|
+
def self.today_cost_cents
|
|
78
|
+
Step.where("DATE(created_at) = CURRENT_DATE").sum(:cost_cents)
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def self.build_row(date, runs, calls, cost_cents)
|
|
82
|
+
Row.new(date: date, runs: runs, real_calls: calls, cost_cents: cost_cents,
|
|
56
83
|
calls_per_100_runs: runs.zero? ? 0.0 : (calls.to_f / runs * 100).round(1))
|
|
57
84
|
end
|
|
58
85
|
private_class_method :build_row
|
data/lib/belay/engine.rb
CHANGED
|
@@ -20,6 +20,15 @@ module Belay
|
|
|
20
20
|
class Engine < ::Rails::Engine
|
|
21
21
|
isolate_namespace Belay
|
|
22
22
|
|
|
23
|
+
# Installed once per process, at boot - not per-request. Belay::Webhooks.deliver
|
|
24
|
+
# (the callback this installs) checks Belay.config.webhook_url/webhook_events
|
|
25
|
+
# itself on every call, so it's a safe no-op in any app that never sets
|
|
26
|
+
# webhook_url, and initializer load order relative to a host app's own
|
|
27
|
+
# config/initializers/belay.rb never matters (see Webhooks' own comment).
|
|
28
|
+
initializer "belay.webhooks" do
|
|
29
|
+
Belay::Webhooks.subscribe!
|
|
30
|
+
end
|
|
31
|
+
|
|
23
32
|
# The dashboard's CSS, inlined into the layout via <style> rather than
|
|
24
33
|
# served through a <link> and whichever asset pipeline (Sprockets,
|
|
25
34
|
# Propshaft, none at all) the host app happens to have configured - that's
|
data/lib/belay/executor.rb
CHANGED
|
@@ -186,6 +186,8 @@ module Belay
|
|
|
186
186
|
# human. Run#pause! already exists (running -> paused) for exactly this shape
|
|
187
187
|
# of "stopped, not failed, waiting on something external."
|
|
188
188
|
@run.pause!
|
|
189
|
+
ActiveSupport::Notifications.instrument("belay.step.needs_review", run_id: @run.id, step_id: existing.id,
|
|
190
|
+
name: existing.name, reason: reason)
|
|
189
191
|
raise Error, "#{reason} - flagged needs_review; the run is paused until a human resolves it"
|
|
190
192
|
end
|
|
191
193
|
|
|
@@ -240,12 +242,85 @@ module Belay
|
|
|
240
242
|
output = run_block(step_run, context, step)
|
|
241
243
|
return [nil, nil] if output.equal?(AUTO_SETTLED) # auto_settle_gate! already wrote this row itself
|
|
242
244
|
|
|
243
|
-
|
|
244
|
-
[output, nil]
|
|
245
|
+
settle_succeeded!(step_run, step, context, output)
|
|
245
246
|
rescue StandardError => e
|
|
246
|
-
|
|
247
|
+
settle_failed!(step_run, step, context, e)
|
|
248
|
+
ensure
|
|
249
|
+
# One call site, not one in each settle_*! below, covering success, failure,
|
|
250
|
+
# AND (harmlessly) the AUTO_SETTLED early return above - check_run_cost_budget!
|
|
251
|
+
# itself no-ops immediately for anything that isn't a step with a real,
|
|
252
|
+
# just-persisted cost_cents (a gate never has one), so running it unconditionally
|
|
253
|
+
# here is equivalent to calling it explicitly from every exit path, without the
|
|
254
|
+
# duplication (and the risk of a future settle_*! forgetting to call it).
|
|
255
|
+
check_run_cost_budget!(step)
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
def settle_succeeded!(step_run, step, context, output)
|
|
259
|
+
status = settled_status(step_run)
|
|
260
|
+
step.update!(status: status, output: output, **extra_step_attrs(step_run, context))
|
|
261
|
+
instrument_step_completed(step, status: status)
|
|
262
|
+
[output, nil]
|
|
263
|
+
end
|
|
264
|
+
|
|
265
|
+
def settle_failed!(step_run, step, context, error)
|
|
266
|
+
step.update!(status: "failed", error: { "class" => error.class.name, "message" => error.message },
|
|
247
267
|
**extra_step_attrs(step_run, context))
|
|
248
|
-
|
|
268
|
+
instrument_step_completed(step, status: "failed")
|
|
269
|
+
[nil, error]
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
# One event per settled attempt (success or failure alike), not separate
|
|
273
|
+
# started/completed events - the same consolidation ActionController's own
|
|
274
|
+
# process_action.action_controller notification makes (one event, status: in
|
|
275
|
+
# the payload), rather than one event per outcome. A step's own row already
|
|
276
|
+
# records attempt: - a retried step fires this once per attempt, same as it
|
|
277
|
+
# gets its own Step row per attempt.
|
|
278
|
+
def instrument_step_completed(step, status:)
|
|
279
|
+
ActiveSupport::Notifications.instrument(
|
|
280
|
+
"belay.step.completed",
|
|
281
|
+
run_id: @run.id, step_id: step.id, position: step.position, name: step.name,
|
|
282
|
+
kind: step.kind, status: status, attempt: step.attempt
|
|
283
|
+
)
|
|
284
|
+
end
|
|
285
|
+
|
|
286
|
+
# Fires belay.run.budget_exceeded the FIRST time this run's total cost_cents
|
|
287
|
+
# (every step's, summed - Run#cost_cents) crosses config.run_cost_budget_cents,
|
|
288
|
+
# never again for the same run - a run that's already over budget doesn't need
|
|
289
|
+
# the same alert repeated on every subsequent step that's merely still over it.
|
|
290
|
+
# step.cost_cents.nil? short-circuits before the budget/alert_sent_at checks
|
|
291
|
+
# below even run: a step with no recorded cost (no pricing: configured, or not
|
|
292
|
+
# an llm_call at all) can never be the step that newly crosses a budget, so
|
|
293
|
+
# there's nothing to check.
|
|
294
|
+
def check_run_cost_budget!(step)
|
|
295
|
+
return if step.cost_cents.nil?
|
|
296
|
+
|
|
297
|
+
budget = Belay.config.run_cost_budget_cents
|
|
298
|
+
return if budget.nil? || @run.budget_alert_sent_at
|
|
299
|
+
|
|
300
|
+
total = @run.cost_cents
|
|
301
|
+
return if total <= budget
|
|
302
|
+
|
|
303
|
+
claim_budget_alert!(total, budget)
|
|
304
|
+
end
|
|
305
|
+
|
|
306
|
+
# A conditional UPDATE, not read-then-write - same reasoning as Run#acquire_
|
|
307
|
+
# lease!'s own comment: nothing here takes the same kind of lease
|
|
308
|
+
# run_attempt's own caller assumes holds for the whole run (Workflow.start's
|
|
309
|
+
# inline path takes none at all; flag_needs_review!'s own comment already
|
|
310
|
+
# flags that no lease mechanism exists to tell two workers apart), so two
|
|
311
|
+
# Executor instances racing on the same run could otherwise both read
|
|
312
|
+
# budget_alert_sent_at as nil and both fire this event. update_all's return
|
|
313
|
+
# value (rows actually matched) is the only thing trusted to decide that;
|
|
314
|
+
# @run's in-memory attribute is only set after a successful claim, never used
|
|
315
|
+
# to decide one.
|
|
316
|
+
def claim_budget_alert!(total, budget)
|
|
317
|
+
claimed = Run.where(id: @run.id, budget_alert_sent_at: nil).update_all(budget_alert_sent_at: Time.current)
|
|
318
|
+
return if claimed.zero?
|
|
319
|
+
|
|
320
|
+
@run.budget_alert_sent_at = Time.current
|
|
321
|
+
ActiveSupport::Notifications.instrument(
|
|
322
|
+
"belay.run.budget_exceeded", run_id: @run.id, total_cost_cents: total, budget_cents: budget
|
|
323
|
+
)
|
|
249
324
|
end
|
|
250
325
|
|
|
251
326
|
# Wrapped in @run.heartbeat_while, not called bare - this is the one call in
|
|
@@ -311,6 +386,8 @@ module Belay
|
|
|
311
386
|
def auto_settle_gate!(step, context_key, rule)
|
|
312
387
|
step.update!(status: "succeeded", output: nil, input: { "context_key" => context_key },
|
|
313
388
|
rule_applied_id: rule.id)
|
|
389
|
+
ActiveSupport::Notifications.instrument("belay.gate.auto_approved", run_id: @run.id, step_id: step.id,
|
|
390
|
+
name: step.name, rule_id: rule.id)
|
|
314
391
|
AUTO_SETTLED
|
|
315
392
|
end
|
|
316
393
|
|
|
@@ -347,6 +424,8 @@ module Belay
|
|
|
347
424
|
@run.pause!
|
|
348
425
|
@run.release_lease!
|
|
349
426
|
end
|
|
427
|
+
ActiveSupport::Notifications.instrument("belay.gate.paused", run_id: @run.id, step_id: step.id,
|
|
428
|
+
name: step.name)
|
|
350
429
|
throw :gate_paused
|
|
351
430
|
end
|
|
352
431
|
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "net/http"
|
|
4
|
+
require "json"
|
|
5
|
+
require "uri"
|
|
6
|
+
|
|
7
|
+
module Belay
|
|
8
|
+
# Calls Google's Gemini API directly over Net::HTTP (stdlib) - same
|
|
9
|
+
# reasoning as AnthropicProvider/OpenAIProvider: no SDK gem, so no second
|
|
10
|
+
# gem's own version matrix for this gem to track for a provider this
|
|
11
|
+
# central.
|
|
12
|
+
#
|
|
13
|
+
# Endpoint and field names verified against Google's own REST reference
|
|
14
|
+
# (ai.google.dev/api/generate-content) as of 2026-10-28.
|
|
15
|
+
#
|
|
16
|
+
# The API key goes in the request URL's own ?key= query param, not a
|
|
17
|
+
# header - that's Gemini's own convention, unlike Anthropic's x-api-key or
|
|
18
|
+
# OpenAI's Authorization: Bearer. URI.encode_www_form escapes it rather
|
|
19
|
+
# than interpolating it directly into the URL string, purely so a key
|
|
20
|
+
# containing a URL-meaningful character (unlikely, but API keys are
|
|
21
|
+
# opaque strings this gem didn't generate) can't corrupt the request line.
|
|
22
|
+
#
|
|
23
|
+
# schema: is passed through to generationConfig.responseSchema completely
|
|
24
|
+
# unvalidated and untranslated - Gemini's schema dialect uses its own,
|
|
25
|
+
# UPPERCASE type names ("OBJECT", "STRING", "INTEGER", ...), not standard
|
|
26
|
+
# JSON Schema's lowercase ones ("object", "string"), and doesn't support
|
|
27
|
+
# every JSON Schema keyword. A schema written for AnthropicProvider or
|
|
28
|
+
# OpenAIProvider will not work here unchanged - same spirit as
|
|
29
|
+
# AnthropicProvider's additionalProperties: false requirement, just a
|
|
30
|
+
# different provider-specific dialect. See docs/providers.md.
|
|
31
|
+
#
|
|
32
|
+
# finishReason is mapped to the same StructuredOutputError reason: values
|
|
33
|
+
# the other two providers use, so a caller can branch on reason: the same
|
|
34
|
+
# way regardless of which provider is configured: "SAFETY"/"RECITATION"
|
|
35
|
+
# (the model declined to produce matching content) -> "refusal";
|
|
36
|
+
# "MAX_TOKENS" -> "max_tokens". Like AnthropicProvider/OpenAIProvider, this
|
|
37
|
+
# check only runs when schema: was given - see OpenAIProvider's own comment
|
|
38
|
+
# for why a refusal/truncation on a plain-text call isn't this gem's call
|
|
39
|
+
# to second-guess.
|
|
40
|
+
class GeminiProvider < Provider
|
|
41
|
+
API_BASE = "https://generativelanguage.googleapis.com/v1beta/models"
|
|
42
|
+
|
|
43
|
+
# Raised on any non-2xx response. Gemini's own error body (JSON, usually
|
|
44
|
+
# {"error": {"code":, "message":, "status":}}) is preserved verbatim in
|
|
45
|
+
# the message, same reasoning as AnthropicProvider::RequestError.
|
|
46
|
+
class RequestError < Error
|
|
47
|
+
attr_reader :status
|
|
48
|
+
|
|
49
|
+
def initialize(status:, body:)
|
|
50
|
+
@status = status
|
|
51
|
+
super("Gemini API request failed with status #{status}: #{body}")
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# Raised when schema: was given but the response can't be trusted to
|
|
56
|
+
# contain valid, complete schema-matching JSON. reason: is "refusal" or
|
|
57
|
+
# "max_tokens" - see the class comment above for which finishReason maps
|
|
58
|
+
# to each.
|
|
59
|
+
class StructuredOutputError < Error
|
|
60
|
+
attr_reader :reason, :response
|
|
61
|
+
|
|
62
|
+
def initialize(reason:, response:)
|
|
63
|
+
@reason = reason
|
|
64
|
+
@response = response
|
|
65
|
+
super("Gemini did not return schema-matching output (reason: #{reason.inspect}): #{response.inspect}")
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
# pricing: {"gemini-..." => { input_per_million: <USD Float>, output_per_million: <USD Float> }, ...}
|
|
70
|
+
# Deliberately not pre-filled with Google's current rates - same
|
|
71
|
+
# reasoning as AnthropicProvider's own pricing: comment.
|
|
72
|
+
def initialize(api_key: ENV.fetch("GEMINI_API_KEY", nil), max_output_tokens: 4096, pricing: {})
|
|
73
|
+
super()
|
|
74
|
+
raise ArgumentError, "#{self.class} requires api_key: (or GEMINI_API_KEY)" if api_key.nil?
|
|
75
|
+
|
|
76
|
+
@api_key = api_key
|
|
77
|
+
@max_output_tokens = max_output_tokens
|
|
78
|
+
@pricing = pricing
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def call(model:, prompt:, schema: nil)
|
|
82
|
+
started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
83
|
+
status, body = perform_request(model, request_body(prompt: prompt, schema: schema))
|
|
84
|
+
latency_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round
|
|
85
|
+
|
|
86
|
+
raise RequestError.new(status: status, body: body) unless status.between?(200, 299)
|
|
87
|
+
|
|
88
|
+
build_response(JSON.parse(body), model: model, schema: schema, latency_ms: latency_ms)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
private
|
|
92
|
+
|
|
93
|
+
def request_body(prompt:, schema:)
|
|
94
|
+
body = {
|
|
95
|
+
contents: [{ parts: [{ text: prompt }] }],
|
|
96
|
+
generationConfig: { maxOutputTokens: @max_output_tokens }
|
|
97
|
+
}
|
|
98
|
+
return body if schema.nil?
|
|
99
|
+
|
|
100
|
+
body[:generationConfig] = body[:generationConfig].merge(responseMimeType: "application/json",
|
|
101
|
+
responseSchema: schema)
|
|
102
|
+
body
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
def perform_request(model, body)
|
|
106
|
+
uri = URI("#{API_BASE}/#{model}:generateContent")
|
|
107
|
+
uri.query = URI.encode_www_form(key: @api_key)
|
|
108
|
+
response = Net::HTTP.post(uri, JSON.generate(body), { "content-type" => "application/json" })
|
|
109
|
+
[response.code.to_i, response.body]
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
def build_response(response, model:, schema:, latency_ms:)
|
|
113
|
+
usage = response["usageMetadata"] || {}
|
|
114
|
+
input_tokens, output_tokens = usage.values_at("promptTokenCount", "candidatesTokenCount")
|
|
115
|
+
|
|
116
|
+
ProviderResponse.new(
|
|
117
|
+
content: extract_content(response, schema: schema),
|
|
118
|
+
model: response["modelVersion"] || model,
|
|
119
|
+
input_tokens: input_tokens,
|
|
120
|
+
output_tokens: output_tokens,
|
|
121
|
+
cost_cents: cost_cents(model: model, input_tokens: input_tokens, output_tokens: output_tokens),
|
|
122
|
+
latency_ms: latency_ms
|
|
123
|
+
)
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def extract_content(response, schema:)
|
|
127
|
+
schema ? extract_structured_output(response) : extract_text(response)
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
def extract_structured_output(response)
|
|
131
|
+
finish_reason = response.dig("candidates", 0, "finishReason")
|
|
132
|
+
if %w[SAFETY RECITATION].include?(finish_reason)
|
|
133
|
+
raise StructuredOutputError.new(reason: "refusal", response: response)
|
|
134
|
+
end
|
|
135
|
+
raise StructuredOutputError.new(reason: "max_tokens", response: response) if finish_reason == "MAX_TOKENS"
|
|
136
|
+
|
|
137
|
+
JSON.parse(extract_text(response))
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def extract_text(response)
|
|
141
|
+
part = response.dig("candidates", 0, "content", "parts", 0)
|
|
142
|
+
raise Error, "Gemini response had no text content part" if part.nil? || part["text"].nil?
|
|
143
|
+
|
|
144
|
+
part["text"]
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
def cost_cents(model:, input_tokens:, output_tokens:)
|
|
148
|
+
return nil if input_tokens.nil? || output_tokens.nil?
|
|
149
|
+
|
|
150
|
+
rates = @pricing[model]
|
|
151
|
+
return nil if rates.nil?
|
|
152
|
+
|
|
153
|
+
input_cost = input_tokens * rates.fetch(:input_per_million)
|
|
154
|
+
output_cost = output_tokens * rates.fetch(:output_per_million)
|
|
155
|
+
((input_cost + output_cost) / 1_000_000.0 * 100).round
|
|
156
|
+
end
|
|
157
|
+
end
|
|
158
|
+
end
|
|
@@ -0,0 +1,246 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "net/http"
|
|
4
|
+
require "json"
|
|
5
|
+
require "uri"
|
|
6
|
+
|
|
7
|
+
module Belay
|
|
8
|
+
# Calls OpenAI's API directly over Net::HTTP (stdlib) - same reasoning as
|
|
9
|
+
# AnthropicProvider: no SDK gem, so no second gem's own version matrix for
|
|
10
|
+
# this gem to track for a provider this central. POST a JSON body, read a
|
|
11
|
+
# JSON body back is small enough that stdlib Net::HTTP + JSON covers it.
|
|
12
|
+
#
|
|
13
|
+
# api: picks which of OpenAI's two current text-generation APIs to call -
|
|
14
|
+
# :responses (default, OpenAI's own forward-looking API, POST /v1/responses)
|
|
15
|
+
# or :chat_completions (the older, still fully supported POST
|
|
16
|
+
# /v1/chat/completions). Both are real, current APIs, not a deprecated/
|
|
17
|
+
# replacement pair, and their request/response shapes differ enough
|
|
18
|
+
# (structured-output config lives at text.format vs response_format; usage
|
|
19
|
+
# keys are input_tokens/output_tokens vs prompt_tokens/completion_tokens;
|
|
20
|
+
# truncation and refusal surface completely differently - see
|
|
21
|
+
# extract_responses_content/extract_chat_completions_content below) that
|
|
22
|
+
# this class branches internally on @api rather than guessing one and
|
|
23
|
+
# forcing callers who need the other to go write their own provider. The
|
|
24
|
+
# public interface (#call) and everything around it (errors, cost, latency)
|
|
25
|
+
# is identical either way.
|
|
26
|
+
#
|
|
27
|
+
# Endpoint and field names verified against OpenAI's own public OpenAPI spec
|
|
28
|
+
# (github.com/openai/openai-openapi) as of 2026-10-28.
|
|
29
|
+
#
|
|
30
|
+
# schema: on :responses sets text.format: {type: "json_schema", name:,
|
|
31
|
+
# schema:, strict: true}; on :chat_completions it's response_format:
|
|
32
|
+
# {type: "json_schema", json_schema: {name:, schema:, strict: true}}.
|
|
33
|
+
# Unlike Anthropic, OpenAI requires a name: on the schema either way - a
|
|
34
|
+
# fixed, gem-chosen value (SCHEMA_NAME) is used throughout, since nothing
|
|
35
|
+
# about the schema format itself needs that name to be caller-visible.
|
|
36
|
+
# strict: true is always passed (not left to OpenAI's own default: false) -
|
|
37
|
+
# a non-strict schema is advisory only, and this gem's own
|
|
38
|
+
# StructuredOutputError handling below assumes a schema that was actually
|
|
39
|
+
# enforced.
|
|
40
|
+
#
|
|
41
|
+
# refusal/truncation are each mapped to the same StructuredOutputError
|
|
42
|
+
# reason: values AnthropicProvider uses ("refusal"/"max_tokens"), so a
|
|
43
|
+
# caller can branch on reason: the same way regardless of which provider is
|
|
44
|
+
# configured:
|
|
45
|
+
# :responses - a "refusal" content item on the output message ->
|
|
46
|
+
# "refusal"; status: "incomplete" with
|
|
47
|
+
# incomplete_details.reason == "max_output_tokens" ->
|
|
48
|
+
# "max_tokens"
|
|
49
|
+
# :chat_completions - message.refusal present -> "refusal";
|
|
50
|
+
# finish_reason == "length" -> "max_tokens"
|
|
51
|
+
# Both checks, like AnthropicProvider's own, only run when schema: was
|
|
52
|
+
# given - a refusal or truncation on a plain-text call isn't this gem's
|
|
53
|
+
# problem to second-guess; it surfaces as whatever text (or lack of it)
|
|
54
|
+
# the author's own code then has to deal with, same as any other provider.
|
|
55
|
+
# Genuinely two providers' worth of request/response handling behind one #call
|
|
56
|
+
# interface (see api: above), not one class that grew unchecked - same
|
|
57
|
+
# ClassLength exemption reasoning as Executor::StepRunner (executor.rb).
|
|
58
|
+
class OpenAIProvider < Provider # rubocop:disable Metrics/ClassLength
|
|
59
|
+
API_URLS = {
|
|
60
|
+
responses: URI("https://api.openai.com/v1/responses"),
|
|
61
|
+
chat_completions: URI("https://api.openai.com/v1/chat/completions")
|
|
62
|
+
}.freeze
|
|
63
|
+
|
|
64
|
+
SCHEMA_NAME = "belay_response"
|
|
65
|
+
|
|
66
|
+
# Raised on any non-2xx response. OpenAI's own error body (JSON, usually
|
|
67
|
+
# {"error": {"type":, "message":}}) is preserved verbatim in the message,
|
|
68
|
+
# same reasoning as AnthropicProvider::RequestError.
|
|
69
|
+
class RequestError < Error
|
|
70
|
+
attr_reader :status
|
|
71
|
+
|
|
72
|
+
def initialize(status:, body:)
|
|
73
|
+
@status = status
|
|
74
|
+
super("OpenAI API request failed with status #{status}: #{body}")
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
# Raised when schema: was given but the response can't be trusted to
|
|
79
|
+
# contain valid, complete schema-matching JSON. reason: is "refusal" or
|
|
80
|
+
# "max_tokens" - see the class comment above for exactly what maps to
|
|
81
|
+
# each, per api:.
|
|
82
|
+
class StructuredOutputError < Error
|
|
83
|
+
attr_reader :reason, :response
|
|
84
|
+
|
|
85
|
+
def initialize(reason:, response:)
|
|
86
|
+
@reason = reason
|
|
87
|
+
@response = response
|
|
88
|
+
super("OpenAI did not return schema-matching output (reason: #{reason.inspect}): #{response.inspect}")
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# pricing: {"gpt-..." => { input_per_million: <USD Float>, output_per_million: <USD Float> }, ...}
|
|
93
|
+
# Deliberately not pre-filled with OpenAI's current rates - same reasoning
|
|
94
|
+
# as AnthropicProvider's own pricing: comment.
|
|
95
|
+
def initialize(api_key: ENV.fetch("OPENAI_API_KEY", nil), api: :responses, max_tokens: 4096, pricing: {})
|
|
96
|
+
super()
|
|
97
|
+
raise ArgumentError, "#{self.class} requires api_key: (or OPENAI_API_KEY)" if api_key.nil?
|
|
98
|
+
raise ArgumentError, "api: must be one of #{API_URLS.keys.inspect}, got #{api.inspect}" unless API_URLS.key?(api)
|
|
99
|
+
|
|
100
|
+
@api_key = api_key
|
|
101
|
+
@api = api
|
|
102
|
+
@max_tokens = max_tokens
|
|
103
|
+
@pricing = pricing
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def call(model:, prompt:, schema: nil)
|
|
107
|
+
started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
108
|
+
status, body = perform_request(request_body(model: model, prompt: prompt, schema: schema))
|
|
109
|
+
latency_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round
|
|
110
|
+
|
|
111
|
+
raise RequestError.new(status: status, body: body) unless status.between?(200, 299)
|
|
112
|
+
|
|
113
|
+
build_response(JSON.parse(body), model: model, schema: schema, latency_ms: latency_ms)
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
private
|
|
117
|
+
|
|
118
|
+
def request_body(model:, prompt:, schema:)
|
|
119
|
+
return responses_request_body(model, prompt, schema) if @api == :responses
|
|
120
|
+
|
|
121
|
+
chat_completions_request_body(model, prompt, schema)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def responses_request_body(model, prompt, schema)
|
|
125
|
+
body = { model: model, input: prompt, max_output_tokens: @max_tokens }
|
|
126
|
+
return body if schema.nil?
|
|
127
|
+
|
|
128
|
+
body.merge(text: { format: { type: "json_schema", name: SCHEMA_NAME, schema: schema, strict: true } })
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
def chat_completions_request_body(model, prompt, schema)
|
|
132
|
+
body = { model: model, max_completion_tokens: @max_tokens, messages: [{ role: "user", content: prompt }] }
|
|
133
|
+
return body if schema.nil?
|
|
134
|
+
|
|
135
|
+
body.merge(response_format: { type: "json_schema",
|
|
136
|
+
json_schema: { name: SCHEMA_NAME, schema: schema, strict: true } })
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
def perform_request(body)
|
|
140
|
+
response = Net::HTTP.post(API_URLS.fetch(@api), JSON.generate(body), request_headers)
|
|
141
|
+
[response.code.to_i, response.body]
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
def request_headers
|
|
145
|
+
{ "Authorization" => "Bearer #{@api_key}", "content-type" => "application/json" }
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
def build_response(response, model:, schema:, latency_ms:)
|
|
149
|
+
usage = response["usage"] || {}
|
|
150
|
+
input_tokens, output_tokens = usage_tokens(usage)
|
|
151
|
+
|
|
152
|
+
ProviderResponse.new(
|
|
153
|
+
content: extract_content(response, schema: schema),
|
|
154
|
+
model: response["model"] || model,
|
|
155
|
+
input_tokens: input_tokens,
|
|
156
|
+
output_tokens: output_tokens,
|
|
157
|
+
cost_cents: cost_cents(model: model, input_tokens: input_tokens, output_tokens: output_tokens),
|
|
158
|
+
latency_ms: latency_ms
|
|
159
|
+
)
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
def usage_tokens(usage)
|
|
163
|
+
keys = @api == :responses ? %w[input_tokens output_tokens] : %w[prompt_tokens completion_tokens]
|
|
164
|
+
usage.values_at(*keys)
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
def extract_content(response, schema:)
|
|
168
|
+
return extract_responses_content(response, schema: schema) if @api == :responses
|
|
169
|
+
|
|
170
|
+
extract_chat_completions_content(response, schema: schema)
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
def extract_responses_content(response, schema:)
|
|
174
|
+
message = find_output_message(response)
|
|
175
|
+
return extract_responses_structured_output(response, message) if schema
|
|
176
|
+
|
|
177
|
+
text = output_text(message)
|
|
178
|
+
raise Error, "OpenAI response had no output_text content" if text.nil?
|
|
179
|
+
|
|
180
|
+
text
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
def extract_responses_structured_output(response, message)
|
|
184
|
+
raise StructuredOutputError.new(reason: "refusal", response: response) if refusal?(message)
|
|
185
|
+
raise StructuredOutputError.new(reason: "max_tokens", response: response) if responses_truncated?(response)
|
|
186
|
+
|
|
187
|
+
text = output_text(message)
|
|
188
|
+
raise Error, "OpenAI response had no output_text content" if text.nil?
|
|
189
|
+
|
|
190
|
+
JSON.parse(text)
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
def find_output_message(response)
|
|
194
|
+
message = (response["output"] || []).find { |item| item["type"] == "message" }
|
|
195
|
+
raise Error, "OpenAI response had no output message" if message.nil?
|
|
196
|
+
|
|
197
|
+
message
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def output_text(message)
|
|
201
|
+
text_block = (message["content"] || []).find { |c| c["type"] == "output_text" }
|
|
202
|
+
text_block && text_block["text"]
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
def refusal?(message)
|
|
206
|
+
(message["content"] || []).any? { |c| c["type"] == "refusal" }
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
def responses_truncated?(response)
|
|
210
|
+
response["status"] == "incomplete" && response.dig("incomplete_details", "reason") == "max_output_tokens"
|
|
211
|
+
end
|
|
212
|
+
|
|
213
|
+
def extract_chat_completions_content(response, schema:)
|
|
214
|
+
return extract_chat_completions_structured_output(response) if schema
|
|
215
|
+
|
|
216
|
+
content = response.dig("choices", 0, "message", "content")
|
|
217
|
+
raise Error, "OpenAI response had no message content" if content.nil?
|
|
218
|
+
|
|
219
|
+
content
|
|
220
|
+
end
|
|
221
|
+
|
|
222
|
+
def extract_chat_completions_structured_output(response)
|
|
223
|
+
message = response.dig("choices", 0, "message") || {}
|
|
224
|
+
raise StructuredOutputError.new(reason: "refusal", response: response) if message["refusal"]
|
|
225
|
+
|
|
226
|
+
finish_reason = response.dig("choices", 0, "finish_reason")
|
|
227
|
+
raise StructuredOutputError.new(reason: "max_tokens", response: response) if finish_reason == "length"
|
|
228
|
+
|
|
229
|
+
content = message["content"]
|
|
230
|
+
raise Error, "OpenAI response had no message content" if content.nil?
|
|
231
|
+
|
|
232
|
+
JSON.parse(content)
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
def cost_cents(model:, input_tokens:, output_tokens:)
|
|
236
|
+
return nil if input_tokens.nil? || output_tokens.nil?
|
|
237
|
+
|
|
238
|
+
rates = @pricing[model]
|
|
239
|
+
return nil if rates.nil?
|
|
240
|
+
|
|
241
|
+
input_cost = input_tokens * rates.fetch(:input_per_million)
|
|
242
|
+
output_cost = output_tokens * rates.fetch(:output_per_million)
|
|
243
|
+
((input_cost + output_cost) / 1_000_000.0 * 100).round
|
|
244
|
+
end
|
|
245
|
+
end
|
|
246
|
+
end
|
data/lib/belay/run.rb
CHANGED
|
@@ -36,15 +36,15 @@ module Belay
|
|
|
36
36
|
validates :workflow_version, presence: true
|
|
37
37
|
|
|
38
38
|
def start!
|
|
39
|
-
transition!(from: "pending", to: "running", started_at: Time.current)
|
|
39
|
+
transition!(from: "pending", to: "running", event: "started", started_at: Time.current)
|
|
40
40
|
end
|
|
41
41
|
|
|
42
42
|
def pause!
|
|
43
|
-
transition!(from: "running", to: "paused")
|
|
43
|
+
transition!(from: "running", to: "paused", event: "paused")
|
|
44
44
|
end
|
|
45
45
|
|
|
46
46
|
def resume!
|
|
47
|
-
transition!(from: "paused", to: "running")
|
|
47
|
+
transition!(from: "paused", to: "running", event: "resumed")
|
|
48
48
|
end
|
|
49
49
|
|
|
50
50
|
# None of complete!/fail!/cancel! below clear lease_owner/lease_expires_at - a
|
|
@@ -53,15 +53,16 @@ module Belay
|
|
|
53
53
|
# first, so a stale lease on a terminal run is never actually read. Would need
|
|
54
54
|
# fixing if anything ever queries leases without that status filter.
|
|
55
55
|
def complete!
|
|
56
|
-
transition!(from: "running", to: "completed", finished_at: Time.current)
|
|
56
|
+
transition!(from: "running", to: "completed", event: "completed", finished_at: Time.current)
|
|
57
57
|
end
|
|
58
58
|
|
|
59
59
|
def fail!(error: nil)
|
|
60
|
-
transition!(from: %w[pending running paused], to: "failed",
|
|
60
|
+
transition!(from: %w[pending running paused], to: "failed", event: "failed",
|
|
61
|
+
finished_at: Time.current, error: error)
|
|
61
62
|
end
|
|
62
63
|
|
|
63
64
|
def cancel!
|
|
64
|
-
transition!(from: %w[pending running paused], to: "cancelled", finished_at: Time.current)
|
|
65
|
+
transition!(from: %w[pending running paused], to: "cancelled", event: "cancelled", finished_at: Time.current)
|
|
65
66
|
end
|
|
66
67
|
|
|
67
68
|
# At most one, by construction: Executor stops at the FIRST gate whose when:
|
|
@@ -81,6 +82,19 @@ module Belay
|
|
|
81
82
|
steps.needs_review.first
|
|
82
83
|
end
|
|
83
84
|
|
|
85
|
+
# SQL SUM, not steps.sum(&:cost_cents) in Ruby - a plain SUM already treats every
|
|
86
|
+
# NULL row (every non-llm_call step, and any llm_call step whose provider had no
|
|
87
|
+
# pricing: configured for that model) as 0 and returns 0, never nil, for a run
|
|
88
|
+
# with no costed steps at all - exactly the "we don't know the cost" vs. "the
|
|
89
|
+
# cost is zero" distinction Provider#cost_cents' own nil already draws at the
|
|
90
|
+
# single-step level, just rolled up: a run where NOTHING is priced costs "0
|
|
91
|
+
# cents we know about," the same honest answer a single unpriced step's own nil
|
|
92
|
+
# would give if summed the Ruby way (0, via Array#sum's +0 default) anyway - so
|
|
93
|
+
# there's no real information lost by using SQL's NULL-safe SUM here instead.
|
|
94
|
+
def cost_cents
|
|
95
|
+
steps.sum(:cost_cents)
|
|
96
|
+
end
|
|
97
|
+
|
|
84
98
|
# DESIGN_DECISIONS #5: "a worker claims a run with a conditional UPDATE on
|
|
85
99
|
# lease_owner/lease_expires_at." The condition - no owner yet, or the existing
|
|
86
100
|
# lease has expired - is evaluated by Postgres in the same statement that claims
|
|
@@ -251,6 +265,13 @@ module Belay
|
|
|
251
265
|
# answer wasn't quite right, which is the opposite of "safe to auto-settle
|
|
252
266
|
# identically next time."
|
|
253
267
|
record_rule_decision!(gate, approved: true) if edited_payload.nil?
|
|
268
|
+
# Last, same ordering principle as ResumeRunJob/record_rule_decision! above: a
|
|
269
|
+
# subscriber that raises (ActiveSupport::Notifications runs subscribers
|
|
270
|
+
# synchronously, in-process - a slow or broken one is this gem's problem the
|
|
271
|
+
# moment it's subscribed) must never be able to leave the gate settled with no
|
|
272
|
+
# job enqueued, which placing this any earlier would risk.
|
|
273
|
+
ActiveSupport::Notifications.instrument("belay.gate.approved", run_id: id, step_id: gate.id, name: gate.name,
|
|
274
|
+
by: by, edited: !edited_payload.nil?)
|
|
254
275
|
end
|
|
255
276
|
|
|
256
277
|
# Rejects this run's pending gate: records a rejected Belay::Approval, settles
|
|
@@ -273,6 +294,8 @@ module Belay
|
|
|
273
294
|
end
|
|
274
295
|
|
|
275
296
|
record_rule_decision!(gate, approved: false)
|
|
297
|
+
ActiveSupport::Notifications.instrument("belay.gate.rejected", run_id: id, step_id: gate.id, name: gate.name,
|
|
298
|
+
by: by)
|
|
276
299
|
end
|
|
277
300
|
|
|
278
301
|
# The two outcomes a human can give a needs_review step (flag_needs_review!,
|
|
@@ -615,7 +638,19 @@ module Belay
|
|
|
615
638
|
# No concurrency guard: this reads status in memory, then does a plain update!. Two
|
|
616
639
|
# processes racing on the same run (stale in-memory copies) can both pass the check.
|
|
617
640
|
# Deferred to the leases work (Phase 1f) rather than solved here.
|
|
618
|
-
|
|
641
|
+
#
|
|
642
|
+
# event: is a separate argument from to:, not derived from it (e.g. "belay.run.#{to}") -
|
|
643
|
+
# to: "running" is reached by both start! and resume!, and those are two meaningfully
|
|
644
|
+
# different events for anything listening (ActiveSupport::Notifications.instrument
|
|
645
|
+
# below; see docs/instrumentation.md), the same way the comment above this method
|
|
646
|
+
# already explains why dispatch can't collapse them into one target-keyed graph either.
|
|
647
|
+
#
|
|
648
|
+
# Wrapping update! in .instrument, not instrumenting after the fact, is deliberate: a
|
|
649
|
+
# subscriber gets the real duration of the UPDATE for free (same convention
|
|
650
|
+
# ActiveRecord's own sql.active_record notification uses), and a failed update! never
|
|
651
|
+
# fires the event at all - nothing here is misreported as a transition that didn't
|
|
652
|
+
# actually happen.
|
|
653
|
+
def transition!(from:, to:, event:, **attrs)
|
|
619
654
|
allowed_from = Array(from)
|
|
620
655
|
unless allowed_from.include?(status)
|
|
621
656
|
raise InvalidTransitionError,
|
|
@@ -623,7 +658,8 @@ module Belay
|
|
|
623
658
|
"(expected current status to be #{allowed_from.inspect})"
|
|
624
659
|
end
|
|
625
660
|
|
|
626
|
-
|
|
661
|
+
payload = { run_id: id, workflow: workflow, workflow_version: workflow_version, from: status, to: to }
|
|
662
|
+
ActiveSupport::Notifications.instrument("belay.run.#{event}", payload) { update!(**attrs, status: to) }
|
|
627
663
|
end
|
|
628
664
|
end
|
|
629
665
|
end
|
data/lib/belay/version.rb
CHANGED
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_job"
|
|
4
|
+
require "net/http"
|
|
5
|
+
require "json"
|
|
6
|
+
require "uri"
|
|
7
|
+
require "openssl"
|
|
8
|
+
|
|
9
|
+
module Belay
|
|
10
|
+
# POSTs a single belay.* event to Belay.config.webhook_url, enqueued by
|
|
11
|
+
# Belay::Webhooks.deliver (never called directly by application code). A separate
|
|
12
|
+
# job, not an inline HTTP call from inside the instrument block that fired the
|
|
13
|
+
# event, so a slow or down webhook receiver can never add latency to - or fail - a
|
|
14
|
+
# gate pausing, a step getting flagged needs_review, or a run completing. Same
|
|
15
|
+
# reasoning as ResumeRunJob/JanitorJob already being jobs rather than inline calls.
|
|
16
|
+
#
|
|
17
|
+
# event/payload, not the Run/Step records themselves: ActiveJob serializes a plain
|
|
18
|
+
# Hash (with its original String/Symbol keys preserved) without needing GlobalID,
|
|
19
|
+
# and every value already in an instrumentation payload (ids, strings, booleans) is
|
|
20
|
+
# plain-JSON-safe as-is - no serializer concern to design around. (Webhooks.deliver
|
|
21
|
+
# itself is responsible for never handing this job a payload containing the raw
|
|
22
|
+
# :exception_object ActiveSupport::Notifications::Instrumenter adds when a wrapped
|
|
23
|
+
# block raises - see that module's own comment - so this job never has to guard
|
|
24
|
+
# against an unserializable argument.)
|
|
25
|
+
#
|
|
26
|
+
# webhook_url/webhook_secret are both read here, at perform time, not captured at
|
|
27
|
+
# enqueue time - so clearing webhook_url between enqueue and perform (an app
|
|
28
|
+
# deliberately turning webhooks off mid-flight) makes this job a clean no-op rather
|
|
29
|
+
# than delivering to a URL the app just unset.
|
|
30
|
+
#
|
|
31
|
+
# OPEN_TIMEOUT/READ_TIMEOUT are set explicitly (unlike AnthropicProvider/
|
|
32
|
+
# OpenAIProvider/GeminiProvider's own Net::HTTP calls, which set neither): those
|
|
33
|
+
# three call a trusted vendor's own API; webhook_url is an arbitrary URL the host
|
|
34
|
+
# app configured, which could point at a receiver that hangs indefinitely without
|
|
35
|
+
# ever closing the connection. Net::HTTP's stdlib default (60s/60s) would otherwise
|
|
36
|
+
# let one hung receiver tie up a job worker for minutes across all 5 retry_on
|
|
37
|
+
# attempts; a short, fixed timeout here caps that per attempt.
|
|
38
|
+
#
|
|
39
|
+
# retry_on, not discard_on: a webhook receiver having a bad minute is exactly the
|
|
40
|
+
# case retrying helps with, and letting it eventually raise once attempts are
|
|
41
|
+
# exhausted surfaces the failure in whatever the host app's own job backend already
|
|
42
|
+
# does with a failed job (Sidekiq's dead set, GoodJob's errors, etc.) - silently
|
|
43
|
+
# swallowing a permanently-failing webhook forever would hide a real integration
|
|
44
|
+
# break from the one app that could actually fix it. A malformed webhook_url is
|
|
45
|
+
# deliberately NOT in this list, below - retrying the exact same unparseable string
|
|
46
|
+
# five times, with increasing delay, can never succeed; #perform raises a plain
|
|
47
|
+
# Belay::Error for that case instead, immediately, once.
|
|
48
|
+
class WebhookJob < ActiveJob::Base
|
|
49
|
+
OPEN_TIMEOUT = 5
|
|
50
|
+
READ_TIMEOUT = 5
|
|
51
|
+
|
|
52
|
+
# Raised on any non-2xx response. The receiving endpoint's own response body is
|
|
53
|
+
# preserved verbatim in the message, same reasoning as every Provider's own
|
|
54
|
+
# RequestError.
|
|
55
|
+
class RequestError < Error
|
|
56
|
+
attr_reader :status
|
|
57
|
+
|
|
58
|
+
def initialize(status:, body:)
|
|
59
|
+
@status = status
|
|
60
|
+
super("Belay webhook POST failed with status #{status}: #{body}")
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
retry_on RequestError, wait: :polynomially_longer, attempts: 5
|
|
65
|
+
retry_on Net::OpenTimeout, Net::ReadTimeout, SocketError, wait: :polynomially_longer, attempts: 5
|
|
66
|
+
|
|
67
|
+
def perform(event:, payload:)
|
|
68
|
+
url = Belay.config.webhook_url
|
|
69
|
+
return if url.nil?
|
|
70
|
+
|
|
71
|
+
body = JSON.generate(event: event, payload: payload)
|
|
72
|
+
response = post(parse_webhook_url!(url), body)
|
|
73
|
+
status = response.code.to_i
|
|
74
|
+
raise RequestError.new(status: status, body: response.body) unless status.between?(200, 299)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
private
|
|
78
|
+
|
|
79
|
+
# URI() alone doesn't catch every malformed URL - a scheme-less string like
|
|
80
|
+
# "example.com/hook" parses successfully into a URI::Generic with host: nil
|
|
81
|
+
# (it's treated as a relative reference), rather than raising. Checking host
|
|
82
|
+
# explicitly catches that case too, so it fails the same clear, immediate,
|
|
83
|
+
# non-retried way as a string URI() itself rejects, instead of surfacing later
|
|
84
|
+
# as a cryptic Net::HTTP connection error against a nil host.
|
|
85
|
+
def parse_webhook_url!(url)
|
|
86
|
+
uri = URI(url)
|
|
87
|
+
raise URI::InvalidURIError, "no host" if uri.host.nil?
|
|
88
|
+
|
|
89
|
+
uri
|
|
90
|
+
rescue URI::InvalidURIError => e
|
|
91
|
+
raise Error, "Belay.config.webhook_url (#{url.inspect}) is not a valid URL: #{e.message}"
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def post(uri, body)
|
|
95
|
+
Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https",
|
|
96
|
+
open_timeout: OPEN_TIMEOUT, read_timeout: READ_TIMEOUT) do |http|
|
|
97
|
+
request = Net::HTTP::Post.new(uri, request_headers(body))
|
|
98
|
+
request.body = body
|
|
99
|
+
http.request(request)
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def request_headers(body)
|
|
104
|
+
headers = { "content-type" => "application/json" }
|
|
105
|
+
secret = Belay.config.webhook_secret
|
|
106
|
+
headers["X-Belay-Signature"] = OpenSSL::HMAC.hexdigest("SHA256", secret, body) if secret
|
|
107
|
+
headers
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
end
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Belay
|
|
4
|
+
# Bridges ActiveSupport::Notifications (docs/instrumentation.md) to
|
|
5
|
+
# Belay::WebhookJob. .subscribe! is called exactly once, from Belay::Engine's own
|
|
6
|
+
# initializer, at process boot - not per-request, not re-subscribed if config
|
|
7
|
+
# changes later.
|
|
8
|
+
#
|
|
9
|
+
# Subscribes to every belay.* event unconditionally, then filters on
|
|
10
|
+
# Belay.config.webhook_events inside the single callback, rather than
|
|
11
|
+
# subscribing/unsubscribing per event as config.webhook_events changes - config can
|
|
12
|
+
# be (re)assigned any time after boot (a test's Belay.reset_config!, a console
|
|
13
|
+
# session), and re-deriving the subscription from a config value that might not
|
|
14
|
+
# exist yet at initializer time would make this fragile to initializer ordering.
|
|
15
|
+
# Checking config inside the callback instead means order never matters: by the
|
|
16
|
+
# time any belay.* event actually fires, config has long since been set.
|
|
17
|
+
module Webhooks
|
|
18
|
+
def self.subscribe!
|
|
19
|
+
ActiveSupport::Notifications.subscribe(/^belay\./) do |name, _start, _finish, _id, payload|
|
|
20
|
+
deliver(name, payload)
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def self.deliver(name, payload)
|
|
25
|
+
return if Belay.config.webhook_url.nil?
|
|
26
|
+
return unless Belay.config.webhook_events.include?(name)
|
|
27
|
+
# Run#transition! wraps update! in ActiveSupport::Notifications.instrument(...) {
|
|
28
|
+
# }; if that block raises (e.g. a validation/locking error), Instrumenter still
|
|
29
|
+
# notifies every subscriber (the `ensure` in its own #instrument), but first
|
|
30
|
+
# mutates payload in place to add :exception (a 2-element String Array - fine)
|
|
31
|
+
# AND :exception_object (the raw exception instance - not ActiveJob-serializable,
|
|
32
|
+
# would raise ActiveJob::SerializationError from inside this very call). Skip it
|
|
33
|
+
# entirely rather than strip just the unserializable key: the transition this
|
|
34
|
+
# payload describes didn't actually happen, so forwarding it at all - "run
|
|
35
|
+
# completed," when update! raised - would be reporting something false, not
|
|
36
|
+
# just a serialization hazard.
|
|
37
|
+
return if payload.key?(:exception)
|
|
38
|
+
|
|
39
|
+
WebhookJob.perform_later(event: name, payload: payload)
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
data/lib/belay.rb
CHANGED
|
@@ -20,9 +20,13 @@ module Belay
|
|
|
20
20
|
require_relative "belay/provider"
|
|
21
21
|
require_relative "belay/fake_provider"
|
|
22
22
|
require_relative "belay/anthropic_provider"
|
|
23
|
+
require_relative "belay/openai_provider"
|
|
24
|
+
require_relative "belay/gemini_provider"
|
|
23
25
|
require_relative "belay/executor"
|
|
24
26
|
require_relative "belay/resume_run_job"
|
|
25
27
|
require_relative "belay/janitor_job"
|
|
28
|
+
require_relative "belay/webhook_job"
|
|
29
|
+
require_relative "belay/webhooks"
|
|
26
30
|
require_relative "belay/engine"
|
|
27
31
|
|
|
28
32
|
# Raised by Run#start!/pause!/resume!/complete!/fail!/cancel! when the current
|
|
@@ -43,6 +43,17 @@ module Belay
|
|
|
43
43
|
def create_belay_rules_migration
|
|
44
44
|
migration_template "create_belay_rules.rb.tt", "db/migrate/create_belay_rules.rb"
|
|
45
45
|
end
|
|
46
|
+
|
|
47
|
+
# Added after 0.2.0 shipped, as a new migration - never editing
|
|
48
|
+
# create_belay_runs.rb.tt above once it's shipped (see that file's own
|
|
49
|
+
# history). A host app upgrading the gem and re-running `rails generate
|
|
50
|
+
# belay:install` only gets this one copied in: migration_template skips
|
|
51
|
+
# every migration whose class already exists in db/migrate, so the five
|
|
52
|
+
# above are untouched no-ops on a re-run.
|
|
53
|
+
def add_budget_alert_sent_at_to_belay_runs_migration
|
|
54
|
+
migration_template "add_budget_alert_sent_at_to_belay_runs.rb.tt",
|
|
55
|
+
"db/migrate/add_budget_alert_sent_at_to_belay_runs.rb"
|
|
56
|
+
end
|
|
46
57
|
end
|
|
47
58
|
end
|
|
48
59
|
end
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
class AddBudgetAlertSentAtToBelayRuns < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
|
|
2
|
+
def change
|
|
3
|
+
# Set the one time Executor fires belay.run.budget_exceeded for this run (once
|
|
4
|
+
# its total cost_cents crosses config.run_cost_budget_cents) - nil forever for
|
|
5
|
+
# a run that never has a budget configured, or never crosses it. Guards against
|
|
6
|
+
# firing the same event again on every later step that's still over budget.
|
|
7
|
+
add_column :belay_runs, :budget_alert_sent_at, :datetime
|
|
8
|
+
end
|
|
9
|
+
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: belay
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Swakhar Dey
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-11-07 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: actionpack
|
|
@@ -164,6 +164,7 @@ files:
|
|
|
164
164
|
- app/views/belay/runs/show.html.erb
|
|
165
165
|
- app/views/belay/shared/_flash.html.erb
|
|
166
166
|
- app/views/belay/shared/_sidebar.html.erb
|
|
167
|
+
- app/views/belay/stats/_budget_banner.html.erb
|
|
167
168
|
- app/views/belay/stats/_chart.html.erb
|
|
168
169
|
- app/views/belay/stats/_table.html.erb
|
|
169
170
|
- app/views/belay/stats/index.html.erb
|
|
@@ -180,7 +181,9 @@ files:
|
|
|
180
181
|
- lib/belay/engine.rb
|
|
181
182
|
- lib/belay/executor.rb
|
|
182
183
|
- lib/belay/fake_provider.rb
|
|
184
|
+
- lib/belay/gemini_provider.rb
|
|
183
185
|
- lib/belay/janitor_job.rb
|
|
186
|
+
- lib/belay/openai_provider.rb
|
|
184
187
|
- lib/belay/provider.rb
|
|
185
188
|
- lib/belay/provider_response.rb
|
|
186
189
|
- lib/belay/resume_run_job.rb
|
|
@@ -188,8 +191,11 @@ files:
|
|
|
188
191
|
- lib/belay/run.rb
|
|
189
192
|
- lib/belay/step.rb
|
|
190
193
|
- lib/belay/version.rb
|
|
194
|
+
- lib/belay/webhook_job.rb
|
|
195
|
+
- lib/belay/webhooks.rb
|
|
191
196
|
- lib/belay/workflow.rb
|
|
192
197
|
- lib/generators/belay/install/install_generator.rb
|
|
198
|
+
- lib/generators/belay/install/templates/add_budget_alert_sent_at_to_belay_runs.rb.tt
|
|
193
199
|
- lib/generators/belay/install/templates/create_belay_approvals.rb.tt
|
|
194
200
|
- lib/generators/belay/install/templates/create_belay_cache_entries.rb.tt
|
|
195
201
|
- lib/generators/belay/install/templates/create_belay_rules.rb.tt
|