coolhand 0.4.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +51 -0
- data/README.md +70 -149
- data/SECURITY.md +20 -0
- data/docs/configuration.md +59 -0
- data/docs/feedback.md +79 -0
- data/docs/openai.md +60 -0
- data/docs/vertex.md +54 -0
- data/lib/coolhand/api_service.rb +12 -1
- data/lib/coolhand/base_interceptor.rb +35 -107
- data/lib/coolhand/configuration.rb +2 -2
- data/lib/coolhand/logger_service.rb +3 -2
- data/lib/coolhand/net_http_interceptor.rb +28 -5
- data/lib/coolhand/open_ai/batch_result_processor.rb +2 -0
- data/lib/coolhand/open_ai/webhook_validator.rb +6 -2
- data/lib/coolhand/version.rb +1 -1
- data/lib/coolhand/vertex/batch_result_processor.rb +97 -40
- data/lib/coolhand/webhook_interceptor.rb +11 -0
- data/lib/coolhand.rb +1 -1
- metadata +15 -8
- data/.idea/coolhand-ruby.iml +0 -6
- data/CLAUDE.md +0 -34
data/docs/vertex.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Google Vertex AI Batch Result Logging
|
|
2
|
+
|
|
3
|
+
Log completed Google Vertex AI batch prediction job results the same way regular synchronous calls are logged. Unlike the [OpenAI batch handler](openai.md), this is not webhook-driven — you call `Coolhand::Vertex::BatchResultProcessor` directly from your own batch-job callback/polling code.
|
|
4
|
+
|
|
5
|
+
For monitoring regular (non-batch) Vertex AI calls, no extra setup is required beyond `Coolhand.configure` — `aiplatform.googleapis.com` is intercepted by default.
|
|
6
|
+
|
|
7
|
+
Requires Rails — `Coolhand::Vertex::BatchResultProcessor` logs via `Rails.logger` internally. `config.capture = false` and `Coolhand.without_capture` do not suppress these logs: unlike the passive Net::HTTP interceptor, calling this processor is an explicit, deliberate act, so it always sends.
|
|
8
|
+
|
|
9
|
+
## Usage
|
|
10
|
+
|
|
11
|
+
```ruby
|
|
12
|
+
require "coolhand/vertex/batch_result_processor"
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
- Call the `Coolhand::Vertex::BatchResultProcessor` service with `batch_info` and the downloaded batch results.
|
|
16
|
+
- Optionally pass `model:` to populate the logged `model` field. It falls back to a `"model"` key in `batch_info` if present, and is omitted from the payload entirely when neither is available. If the value (from either source) looks like a Vertex model resource path (`publishers/<publisher>/models/<id>` or `projects/<project>/locations/<location>/models/<id>`), it's normalized down to just the trailing id/version before being logged; any other string is sent as-is.
|
|
17
|
+
- `batch_info["name"]` must be the exact Vertex job resource name (`projects/<project>/locations/<location>/batchPredictionJobs/<job-id>`), and `batch_info["startTime"]`/`["endTime"]` must be valid ISO 8601 timestamps — a batch with a missing or malformed resource name or timestamps is skipped entirely (logged as an error) rather than sent with a broken URL.
|
|
18
|
+
- If `endTime` precedes `startTime` (e.g. clock skew), the batch is still logged — `duration_ms` is clamped to `0` and a warning is logged — rather than dropping the results.
|
|
19
|
+
- Each element of the `batch_results` array passed to `.call` must be a `Hash` with a `"request"` and/or `"response"` key (the request/response body to log for that item — either key alone is accepted). Any other shape is treated as malformed and skipped, with the skip counted in a `Rails.logger.warn` summary rather than raised.
|
|
20
|
+
|
|
21
|
+
## Minimal example
|
|
22
|
+
|
|
23
|
+
Only the key lines are shown — wire this into your own batch callback service.
|
|
24
|
+
|
|
25
|
+
```ruby
|
|
26
|
+
class Vertex::BatchCallbackProcessor < BaseService
|
|
27
|
+
option :batch_request, model: BatchApiRequest
|
|
28
|
+
option :batch_info
|
|
29
|
+
|
|
30
|
+
def call
|
|
31
|
+
case batch_info["state"]
|
|
32
|
+
when "JOB_STATE_PENDING"
|
|
33
|
+
nil
|
|
34
|
+
when "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
|
|
35
|
+
batch_request.update!(status: "processing")
|
|
36
|
+
|
|
37
|
+
Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call
|
|
38
|
+
when "JOB_STATE_SUCCEEDED"
|
|
39
|
+
output_file_id = batch_info["outputInfo"]["gcsOutputDirectory"]
|
|
40
|
+
results = download_batch_results(output_file_id)
|
|
41
|
+
results.each { |batch_item| process_batch_result(batch_item) }
|
|
42
|
+
|
|
43
|
+
batch_request.update!(status: "completed", completed_at: Time.current, output_file_id:)
|
|
44
|
+
|
|
45
|
+
Coolhand::Vertex::BatchResultProcessor.new(batch_info:, model: batch_request.llm_model).call(results)
|
|
46
|
+
|
|
47
|
+
# Clean up GCS files after successful processing
|
|
48
|
+
cleanup_gcs_files(output_file_id)
|
|
49
|
+
when "JOB_STATE_FAILED"
|
|
50
|
+
handle_failed_batch(batch_info["error"]["message"])
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
```
|
data/lib/coolhand/api_service.rb
CHANGED
|
@@ -70,6 +70,11 @@ module Coolhand
|
|
|
70
70
|
uri = URI.parse(@api_endpoint)
|
|
71
71
|
http = Net::HTTP.new(uri.host, uri.port)
|
|
72
72
|
http.use_ssl = (uri.scheme == "https")
|
|
73
|
+
# Bound worst-case latency: this call happens inline in the intercepted
|
|
74
|
+
# request's path, so a slow/unreachable Coolhand backend must not hang
|
|
75
|
+
# the host app's real LLM call for Ruby's ~60s Net::HTTP defaults.
|
|
76
|
+
http.open_timeout = 5
|
|
77
|
+
http.read_timeout = 5
|
|
73
78
|
|
|
74
79
|
request = Net::HTTP::Post.new(uri.request_uri)
|
|
75
80
|
headers = create_request_options(payload)
|
|
@@ -87,7 +92,13 @@ module Coolhand
|
|
|
87
92
|
request.body = json_body.force_encoding("UTF-8")
|
|
88
93
|
|
|
89
94
|
begin
|
|
90
|
-
|
|
95
|
+
# This request goes through the same patched Net::HTTP as the host
|
|
96
|
+
# app's real LLM calls. Without this, a base_url/intercept_addresses
|
|
97
|
+
# configuration that also matches Coolhand's own API host would
|
|
98
|
+
# re-intercept this log-shipping call, generating a second log
|
|
99
|
+
# request that itself gets intercepted, and so on — unbounded
|
|
100
|
+
# request amplification.
|
|
101
|
+
response = Coolhand.without_capture { http.request(request) }
|
|
91
102
|
|
|
92
103
|
if response.is_a?(Net::HTTPSuccess)
|
|
93
104
|
result = JSON.parse(response.body, symbolize_names: true)
|
|
@@ -5,93 +5,13 @@ module Coolhand
|
|
|
5
5
|
module BaseInterceptor
|
|
6
6
|
module_function
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
# Handle streaming responses - these are often enumerator objects
|
|
16
|
-
# that can't be serialized directly
|
|
17
|
-
if response.class.name.include?("Stream") || response.respond_to?(:each)
|
|
18
|
-
{
|
|
19
|
-
response_type: "streaming",
|
|
20
|
-
class: response.class.name,
|
|
21
|
-
note: "Streaming response - content captured during enumeration"
|
|
22
|
-
}
|
|
23
|
-
elsif response.respond_to?(:to_h)
|
|
24
|
-
begin
|
|
25
|
-
response.to_h
|
|
26
|
-
rescue StandardError => e
|
|
27
|
-
{
|
|
28
|
-
serialization_error: e.message,
|
|
29
|
-
class: response.class.name,
|
|
30
|
-
raw_response: response.to_s
|
|
31
|
-
}
|
|
32
|
-
end
|
|
33
|
-
else
|
|
34
|
-
# Extract content and token usage information
|
|
35
|
-
response_data = {}
|
|
36
|
-
|
|
37
|
-
# Get content
|
|
38
|
-
response_data[:content] = response.content if response.respond_to?(:content)
|
|
39
|
-
|
|
40
|
-
# Extract token usage information
|
|
41
|
-
response_data[:usage] = extract_usage_metadata(response.usage) if response.respond_to?(:usage)
|
|
42
|
-
|
|
43
|
-
# Extract model information
|
|
44
|
-
response_data[:model] = response.model if response.respond_to?(:model)
|
|
45
|
-
|
|
46
|
-
# Extract role information
|
|
47
|
-
response_data[:role] = response.role if response.respond_to?(:role)
|
|
48
|
-
|
|
49
|
-
# Extract ID if available
|
|
50
|
-
response_data[:id] = response.id if response.respond_to?(:id)
|
|
51
|
-
|
|
52
|
-
# Extract stop reason if available
|
|
53
|
-
response_data[:stop_reason] = response.stop_reason if response.respond_to?(:stop_reason)
|
|
54
|
-
|
|
55
|
-
# Add class info for debugging
|
|
56
|
-
response_data[:class] = response.class.name
|
|
57
|
-
|
|
58
|
-
response_data.empty? ? { raw_response: response.to_s, class: response.class.name } : response_data
|
|
59
|
-
end
|
|
60
|
-
end
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
def extract_usage_metadata(usage)
|
|
64
|
-
if usage.respond_to?(:to_h)
|
|
65
|
-
usage.to_h
|
|
66
|
-
elsif usage.is_a?(Hash)
|
|
67
|
-
usage
|
|
68
|
-
else
|
|
69
|
-
# Extract individual usage fields
|
|
70
|
-
usage_data = {}
|
|
71
|
-
usage_data[:input_tokens] = usage.input_tokens if usage.respond_to?(:input_tokens)
|
|
72
|
-
usage_data[:output_tokens] = usage.output_tokens if usage.respond_to?(:output_tokens)
|
|
73
|
-
usage_data[:total_tokens] = usage_data[:input_tokens].to_i + usage_data[:output_tokens].to_i
|
|
74
|
-
usage_data
|
|
75
|
-
end
|
|
76
|
-
end
|
|
77
|
-
|
|
78
|
-
def clean_request_headers(headers)
|
|
79
|
-
cleaned = headers.dup
|
|
80
|
-
|
|
81
|
-
# Remove sensitive headers
|
|
82
|
-
cleaned.delete("Authorization")
|
|
83
|
-
cleaned.delete("authorization")
|
|
84
|
-
cleaned.delete("x-api-key")
|
|
85
|
-
cleaned.delete("X-API-Key")
|
|
86
|
-
|
|
87
|
-
cleaned
|
|
88
|
-
end
|
|
89
|
-
|
|
90
|
-
def clean_response_headers(headers)
|
|
91
|
-
# Response headers typically don't contain sensitive data
|
|
92
|
-
# but we can filter if needed
|
|
93
|
-
headers.dup
|
|
94
|
-
end
|
|
8
|
+
# Matches any header whose *name* signals sensitive content, regardless of
|
|
9
|
+
# provider — covers known keys (x-api-key, x-goog-api-key, openai-api-key),
|
|
10
|
+
# AWS SigV4 session tokens (x-amz-security-token), session/CSRF cookies
|
|
11
|
+
# (cookie, set-cookie), and future/unknown providers using a
|
|
12
|
+
# similarly-named header. Shared with LoggerService so the two logging
|
|
13
|
+
# paths (interceptor + webhook forwarding) stay consistent.
|
|
14
|
+
SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization|cookie/i
|
|
95
15
|
|
|
96
16
|
def sanitize_headers(headers)
|
|
97
17
|
return {} if headers.nil?
|
|
@@ -122,7 +42,6 @@ module Coolhand
|
|
|
122
42
|
|
|
123
43
|
sanitized = raw.dup
|
|
124
44
|
|
|
125
|
-
sanitized_keys = %w[openai-api-key api-key x-api-key x-goog-api-key]
|
|
126
45
|
sanitized.each do |k, v|
|
|
127
46
|
next if v.nil?
|
|
128
47
|
|
|
@@ -134,7 +53,7 @@ module Coolhand
|
|
|
134
53
|
else
|
|
135
54
|
"[REDACTED]"
|
|
136
55
|
end
|
|
137
|
-
elsif
|
|
56
|
+
elsif key_down.match?(SENSITIVE_HEADER_PATTERN)
|
|
138
57
|
sanitized[k] = "[REDACTED]"
|
|
139
58
|
end
|
|
140
59
|
end
|
|
@@ -151,15 +70,22 @@ module Coolhand
|
|
|
151
70
|
end
|
|
152
71
|
end
|
|
153
72
|
|
|
73
|
+
# Matches query param *names* that signal sensitive content, the same way
|
|
74
|
+
# SENSITIVE_HEADER_PATTERN does for headers — covers exact params this
|
|
75
|
+
# gem already knew about (key/token/secret) as well as presigned-URL
|
|
76
|
+
# credential params used by AWS (X-Amz-Signature, X-Amz-Credential,
|
|
77
|
+
# X-Amz-Security-Token) and Google Cloud (X-Goog-Signature,
|
|
78
|
+
# X-Goog-Credential) storage APIs.
|
|
79
|
+
SENSITIVE_QUERY_PARAM_PATTERN = /key|token|secret|sig|credential|password|auth/i
|
|
80
|
+
|
|
154
81
|
def sanitize_url(url)
|
|
155
82
|
uri = URI.parse(url)
|
|
156
83
|
return url unless uri.query
|
|
157
84
|
|
|
158
|
-
sensitive = %w[key api_key apikey token access_token secret]
|
|
159
85
|
params = URI.decode_www_form(uri.query)
|
|
160
86
|
redacted = false
|
|
161
87
|
params.map! do |n, v|
|
|
162
|
-
if
|
|
88
|
+
if n.match?(SENSITIVE_QUERY_PARAM_PATTERN)
|
|
163
89
|
redacted = true
|
|
164
90
|
[n, "[REDACTED]"]
|
|
165
91
|
else
|
|
@@ -178,23 +104,25 @@ module Coolhand
|
|
|
178
104
|
end
|
|
179
105
|
|
|
180
106
|
def send_complete_request_log(request_id:, method:, url:, request_headers:, request_body:, response_headers:,
|
|
181
|
-
response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:)
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
is_streaming: is_streaming
|
|
196
|
-
}
|
|
107
|
+
response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:, source_api: nil, model: nil)
|
|
108
|
+
raw_request = {
|
|
109
|
+
id: request_id,
|
|
110
|
+
timestamp: start_time.iso8601,
|
|
111
|
+
method: method.to_s.downcase,
|
|
112
|
+
url: sanitize_url(url),
|
|
113
|
+
headers: sanitize_headers(request_headers),
|
|
114
|
+
request_body: request_body,
|
|
115
|
+
response_headers: sanitize_headers(response_headers),
|
|
116
|
+
response_body: response_body,
|
|
117
|
+
status_code: status_code,
|
|
118
|
+
duration_ms: duration_ms,
|
|
119
|
+
completed_at: end_time.iso8601,
|
|
120
|
+
is_streaming: is_streaming
|
|
197
121
|
}
|
|
122
|
+
raw_request[:source_api] = source_api if Coolhand.required_field?(source_api)
|
|
123
|
+
raw_request[:model] = model if Coolhand.required_field?(model)
|
|
124
|
+
|
|
125
|
+
request_data = { raw_request: raw_request }
|
|
198
126
|
|
|
199
127
|
api_service = Coolhand::ApiService.new
|
|
200
128
|
api_service.send_llm_request_log(request_data)
|
|
@@ -6,11 +6,11 @@ require "uri"
|
|
|
6
6
|
module Coolhand
|
|
7
7
|
# Handles all configuration settings for the gem.
|
|
8
8
|
class Configuration
|
|
9
|
-
DEFAULT_EXCLUDE_API_PATTERNS = YAML.
|
|
9
|
+
DEFAULT_EXCLUDE_API_PATTERNS = YAML.safe_load_file(
|
|
10
10
|
File.join(__dir__, "default_exclude_api_patterns.yml")
|
|
11
11
|
).freeze
|
|
12
12
|
|
|
13
|
-
DEFAULT_INTERCEPT_ADDRESSES = YAML.
|
|
13
|
+
DEFAULT_INTERCEPT_ADDRESSES = YAML.safe_load_file(
|
|
14
14
|
File.join(__dir__, "default_intercept_addresses.yml")
|
|
15
15
|
).freeze
|
|
16
16
|
|
|
@@ -73,8 +73,9 @@ module Coolhand
|
|
|
73
73
|
# Convert Rails HTTP_ prefix headers
|
|
74
74
|
clean_key = key.to_s.gsub(/^HTTP_/, "").tr("_", "-").downcase
|
|
75
75
|
|
|
76
|
-
# Redact sensitive headers
|
|
77
|
-
|
|
76
|
+
# Redact sensitive headers (shared with BaseInterceptor so both logging
|
|
77
|
+
# paths treat the same header names as sensitive)
|
|
78
|
+
clean_value = clean_key.match?(BaseInterceptor::SENSITIVE_HEADER_PATTERN) ? "[REDACTED]" : value.to_s
|
|
78
79
|
clean_headers[clean_key] = clean_value
|
|
79
80
|
end
|
|
80
81
|
clean_headers
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "stringio"
|
|
4
|
+
|
|
3
5
|
module Coolhand
|
|
4
6
|
module NetHttpInterceptor
|
|
5
7
|
include BaseInterceptor
|
|
@@ -7,7 +9,12 @@ module Coolhand
|
|
|
7
9
|
# Response streaming interceptor nested under NetHttpInterceptor
|
|
8
10
|
module ResponseInterceptor
|
|
9
11
|
def read_body(dest = nil, &block)
|
|
10
|
-
|
|
12
|
+
# Only buffer while a #request call below is actively capturing —
|
|
13
|
+
# otherwise every block-form read_body in the process (including
|
|
14
|
+
# ones on responses this gem never intercepted) accumulates into a
|
|
15
|
+
# thread-local that's never freed, which is an unbounded memory
|
|
16
|
+
# leak on any thread that streams large non-LLM responses.
|
|
17
|
+
return super unless block && Thread.current[:coolhand_capturing_stream]
|
|
11
18
|
|
|
12
19
|
super do |chunk|
|
|
13
20
|
Thread.current[:coolhand_stream_buffer] ||= +""
|
|
@@ -49,8 +56,16 @@ module Coolhand
|
|
|
49
56
|
return super unless should_capture?
|
|
50
57
|
|
|
51
58
|
# Capture body before setting the guard — if this raises we skip logging cleanly
|
|
52
|
-
# and the guard is never set, so there is no leak.
|
|
53
|
-
|
|
59
|
+
# and the guard is never set, so there is no leak. A failure here (e.g. an
|
|
60
|
+
# already-consumed body_stream) must never prevent the real request below
|
|
61
|
+
# from being attempted — this gem must never be the reason the host
|
|
62
|
+
# app's actual LLM call doesn't happen.
|
|
63
|
+
captured_body = begin
|
|
64
|
+
capture_request_body(req, body)
|
|
65
|
+
rescue StandardError => e
|
|
66
|
+
Coolhand.log "❌ Error capturing request body: #{e.message}"
|
|
67
|
+
nil
|
|
68
|
+
end
|
|
54
69
|
|
|
55
70
|
active[self] = true
|
|
56
71
|
start_time = Time.now
|
|
@@ -59,7 +74,14 @@ module Coolhand
|
|
|
59
74
|
status_code = nil
|
|
60
75
|
response_body = nil
|
|
61
76
|
|
|
77
|
+
# Save/restore rather than just nil-ing: a request made from inside
|
|
78
|
+
# this request's own streaming block (nested interception) would
|
|
79
|
+
# otherwise clobber this request's in-progress buffer with its own
|
|
80
|
+
# chunks, mixing one request's content into another's log.
|
|
81
|
+
previous_stream_buffer = Thread.current[:coolhand_stream_buffer]
|
|
82
|
+
previous_capturing_stream = Thread.current[:coolhand_capturing_stream]
|
|
62
83
|
Thread.current[:coolhand_stream_buffer] = nil
|
|
84
|
+
Thread.current[:coolhand_capturing_stream] = true
|
|
63
85
|
|
|
64
86
|
begin
|
|
65
87
|
response = super
|
|
@@ -73,7 +95,8 @@ module Coolhand
|
|
|
73
95
|
raise
|
|
74
96
|
ensure
|
|
75
97
|
active.delete(self)
|
|
76
|
-
Thread.current[:coolhand_stream_buffer] =
|
|
98
|
+
Thread.current[:coolhand_stream_buffer] = previous_stream_buffer
|
|
99
|
+
Thread.current[:coolhand_capturing_stream] = previous_capturing_stream
|
|
77
100
|
end_time = Time.now
|
|
78
101
|
duration_ms = ((end_time - start_time) * 1000).round(2)
|
|
79
102
|
|
|
@@ -141,7 +164,7 @@ module Coolhand
|
|
|
141
164
|
|
|
142
165
|
matched = patterns.find { |pattern| url.include?(pattern) }
|
|
143
166
|
if matched && Coolhand.configuration.debug_mode
|
|
144
|
-
Coolhand.log "🚫 Skipping capture for #{url} (matched exclude_api_pattern: \"#{matched}\")"
|
|
167
|
+
Coolhand.log "🚫 Skipping capture for #{sanitize_url(url)} (matched exclude_api_pattern: \"#{matched}\")"
|
|
145
168
|
end
|
|
146
169
|
!!matched
|
|
147
170
|
end
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "openssl"
|
|
4
|
+
|
|
3
5
|
module Coolhand
|
|
4
6
|
module OpenAi
|
|
5
7
|
class WebhookValidator
|
|
6
|
-
attr_reader :request, :errors, :payload
|
|
8
|
+
attr_reader :request, :errors, :payload
|
|
7
9
|
|
|
8
10
|
def initialize(request, webhook_secret)
|
|
9
11
|
@request = request
|
|
@@ -16,7 +18,7 @@ module Coolhand
|
|
|
16
18
|
@payload = request.raw_post || request.body.read
|
|
17
19
|
|
|
18
20
|
return false unless payload_valid?
|
|
19
|
-
return validate_in_non_production_env unless webhook_secret
|
|
21
|
+
return validate_in_non_production_env unless Coolhand.required_field?(webhook_secret)
|
|
20
22
|
|
|
21
23
|
secret_bytes = extract_secret_bytes
|
|
22
24
|
webhook_signature, webhook_timestamp, webhook_id = extract_webhook_headers
|
|
@@ -32,6 +34,8 @@ module Coolhand
|
|
|
32
34
|
|
|
33
35
|
private
|
|
34
36
|
|
|
37
|
+
attr_reader :webhook_secret
|
|
38
|
+
|
|
35
39
|
def payload_valid?
|
|
36
40
|
return true if @payload
|
|
37
41
|
|
data/lib/coolhand/version.rb
CHANGED
|
@@ -1,12 +1,27 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "../../coolhand"
|
|
4
|
+
|
|
3
5
|
module Coolhand
|
|
4
6
|
module Vertex
|
|
5
7
|
class BatchResultProcessor
|
|
6
|
-
|
|
8
|
+
# Global endpoint, not the job's region-specific one — matches the
|
|
9
|
+
# "aiplatform.googleapis.com" entry in default_intercept_addresses.yml
|
|
10
|
+
# so backend URL-shape classification stays consistent with the rest
|
|
11
|
+
# of the gem's Vertex traffic. The URL this produces always contains
|
|
12
|
+
# "/batchPredictionJobs/", which is also the default client-side
|
|
13
|
+
# exclude_api_patterns entry — that's harmless here since this class
|
|
14
|
+
# sends via BaseInterceptor/ApiService directly and never goes through
|
|
15
|
+
# NetHttpInterceptor's intercept/exclude filtering.
|
|
16
|
+
VERTEX_API_BASE_URL = "https://aiplatform.googleapis.com/v1/"
|
|
17
|
+
SOURCE_API = "vertex"
|
|
18
|
+
VALID_NAME_PATTERN = %r{\Aprojects/[^/?#\s]+/locations/[^/?#\s]+/batchPredictionJobs/[^/?#\s]+\z}
|
|
19
|
+
|
|
20
|
+
attr_reader :batch_info, :model
|
|
7
21
|
|
|
8
|
-
def initialize(batch_info:)
|
|
22
|
+
def initialize(batch_info:, model: nil)
|
|
9
23
|
@batch_info = batch_info
|
|
24
|
+
@model = model
|
|
10
25
|
end
|
|
11
26
|
|
|
12
27
|
def call(batch_results = [])
|
|
@@ -16,7 +31,7 @@ module Coolhand
|
|
|
16
31
|
when "JOB_STATE_PENDING", "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
|
|
17
32
|
Rails.logger.info("[Interceptor] Vertex batch #{batch_info} still processing")
|
|
18
33
|
when "JOB_STATE_SUCCEEDED"
|
|
19
|
-
|
|
34
|
+
process_completed_batch(batch_results)
|
|
20
35
|
when "JOB_STATE_FAILED"
|
|
21
36
|
handle_failed_batch
|
|
22
37
|
else
|
|
@@ -28,56 +43,98 @@ module Coolhand
|
|
|
28
43
|
|
|
29
44
|
private
|
|
30
45
|
|
|
31
|
-
def process_completed_batch(
|
|
32
|
-
|
|
46
|
+
def process_completed_batch(batch_results)
|
|
47
|
+
name = batch_info["name"].to_s.sub(%r{\A/+}, "")
|
|
48
|
+
unless name.match?(VALID_NAME_PATTERN)
|
|
49
|
+
Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a missing or " \
|
|
50
|
+
"invalid job resource name (#{batch_info['name'].inspect}); skipping request log")
|
|
51
|
+
return
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
begin
|
|
55
|
+
start_time = Time.iso8601(batch_info["startTime"])
|
|
56
|
+
end_time = Time.iso8601(batch_info["endTime"])
|
|
57
|
+
rescue TypeError, ArgumentError
|
|
58
|
+
Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a missing or " \
|
|
59
|
+
"invalid startTime/endTime; skipping request log")
|
|
60
|
+
return
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
if end_time < start_time
|
|
64
|
+
Rails.logger.warn("[Interceptor] Vertex batch #{batch_info['displayName']} has an endTime before " \
|
|
65
|
+
"its startTime; sending request log(s) with duration_ms clamped to 0")
|
|
66
|
+
end_time = start_time
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
duration_ms = ((end_time - start_time) * 1000).to_i
|
|
70
|
+
url = "#{VERTEX_API_BASE_URL}#{name}"
|
|
71
|
+
resolved = resolved_model
|
|
72
|
+
|
|
73
|
+
sent = batch_results.count do |batch_item|
|
|
74
|
+
send_item_log(batch_item, url: url, model: resolved, start_time: start_time, end_time: end_time,
|
|
75
|
+
duration_ms: duration_ms)
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
# "Sent" here means dispatched to BaseInterceptor without a local
|
|
79
|
+
# error (e.g. a malformed batch_item) — BaseInterceptor swallows any
|
|
80
|
+
# downstream delivery failure itself, so this can't confirm the
|
|
81
|
+
# Coolhand API actually received the log.
|
|
82
|
+
if sent == batch_results.size
|
|
83
|
+
Rails.logger.info("[Interceptor] Dispatched #{sent} result(s) for Vertex batch " \
|
|
84
|
+
"#{batch_info['displayName']} for logging")
|
|
85
|
+
else
|
|
86
|
+
Rails.logger.warn("[Interceptor] Dispatched #{sent}/#{batch_results.size} result(s) for Vertex " \
|
|
87
|
+
"batch #{batch_info['displayName']} for logging — " \
|
|
88
|
+
"#{batch_results.size - sent} item(s) were malformed and skipped")
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
def send_item_log(batch_item, url:, model:, start_time:, end_time:, duration_ms:)
|
|
93
|
+
unless batch_item.is_a?(Hash) && (batch_item.key?("request") || batch_item.key?("response"))
|
|
94
|
+
Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a malformed result " \
|
|
95
|
+
"item (#{batch_item.class}); skipping request log")
|
|
96
|
+
return false
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
BaseInterceptor.send_complete_request_log(
|
|
100
|
+
request_id: SecureRandom.hex(16),
|
|
33
101
|
method: "POST",
|
|
34
|
-
url:
|
|
102
|
+
url: url,
|
|
103
|
+
source_api: SOURCE_API,
|
|
104
|
+
model: model,
|
|
105
|
+
request_headers: {},
|
|
35
106
|
request_body: batch_item["request"],
|
|
107
|
+
response_headers: {},
|
|
36
108
|
response_body: batch_item["response"],
|
|
37
109
|
status_code: 200,
|
|
38
|
-
start_time:
|
|
39
|
-
end_time:
|
|
40
|
-
|
|
41
|
-
|
|
110
|
+
start_time: start_time,
|
|
111
|
+
end_time: end_time,
|
|
112
|
+
duration_ms: duration_ms,
|
|
113
|
+
is_streaming: false
|
|
114
|
+
)
|
|
115
|
+
true
|
|
42
116
|
rescue StandardError => e
|
|
43
117
|
Rails.logger.error("[Interceptor] Failed to send request log: #{e.message}")
|
|
118
|
+
false
|
|
44
119
|
end
|
|
45
120
|
|
|
46
|
-
def
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
end_time = Time.iso8601(end_time)
|
|
50
|
-
duration_ms = ((end_time - start_time) * 1000).to_i
|
|
121
|
+
def resolved_model
|
|
122
|
+
candidate = [model, batch_info["model"]].find { |c| Coolhand.required_field?(c) }&.to_s&.strip
|
|
123
|
+
return if candidate.nil?
|
|
51
124
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
request_body: request_body,
|
|
60
|
-
response_headers: {},
|
|
61
|
-
response_body: response_body,
|
|
62
|
-
status_code: status_code,
|
|
63
|
-
duration_ms: duration_ms,
|
|
64
|
-
completed_at: end_time,
|
|
65
|
-
is_streaming: false
|
|
66
|
-
}
|
|
67
|
-
}
|
|
68
|
-
|
|
69
|
-
api_service = Coolhand::ApiService.new
|
|
70
|
-
api_service.send_llm_request_log(request_data)
|
|
71
|
-
|
|
72
|
-
Coolhand.log "📤 Sent complete request/response log for #{request_id} (duration: #{duration_ms}ms)"
|
|
73
|
-
rescue StandardError => e
|
|
74
|
-
Coolhand.log "❌ Error sending complete request log: #{e.message}"
|
|
125
|
+
# Only normalize genuine Vertex model resource paths (e.g.
|
|
126
|
+
# "publishers/google/models/gemini-2.0-flash" or
|
|
127
|
+
# "projects/P/locations/L/models/M@1") down to their bare id. A
|
|
128
|
+
# caller-supplied `model:` might itself be a provider-qualified slug
|
|
129
|
+
# that happens to contain a slash (e.g. "meta-llama/Llama-3") —
|
|
130
|
+
# leave anything that isn't a resource path untouched.
|
|
131
|
+
candidate.match?(%r{\A(publishers|projects)/}) ? candidate.split("/").last : candidate
|
|
75
132
|
end
|
|
76
133
|
|
|
77
134
|
# TODO: implement API to handle failed batch results and display errors on dashboard page
|
|
78
135
|
def handle_failed_batch
|
|
79
|
-
|
|
80
|
-
|
|
136
|
+
message = batch_info["error"].is_a?(Hash) ? batch_info["error"]["message"] : batch_info["error"]
|
|
137
|
+
Rails.logger.error("[Interceptor] Vertex batch for #{batch_info['displayName']} failed: #{message}")
|
|
81
138
|
end
|
|
82
139
|
end
|
|
83
140
|
end
|
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "open_ai/webhook_validator"
|
|
4
|
+
require_relative "open_ai/batch_result_processor"
|
|
5
|
+
|
|
3
6
|
module Coolhand
|
|
4
7
|
module WebhookInterceptor
|
|
5
8
|
def intercept_batch_request
|
|
@@ -14,10 +17,18 @@ module Coolhand
|
|
|
14
17
|
end
|
|
15
18
|
|
|
16
19
|
payload = JSON.parse(@validator.payload)
|
|
20
|
+
raise TypeError, "webhook payload must be a JSON object, got #{payload.class}" unless payload.is_a?(Hash)
|
|
17
21
|
|
|
18
22
|
process_event(payload)
|
|
19
23
|
rescue StandardError => e
|
|
24
|
+
# Fail closed: any error here (malformed payload, a bug in
|
|
25
|
+
# process_event/BatchResultProcessor, etc.) must still halt the
|
|
26
|
+
# before_action chain. Falling through without calling `head` would
|
|
27
|
+
# let the controller action run for a request whose webhook
|
|
28
|
+
# signature was never confirmed valid.
|
|
20
29
|
Rails.logger.error("[Interceptor] Failed to intercept batch request: #{e.message}")
|
|
30
|
+
head :unauthorized
|
|
31
|
+
false
|
|
21
32
|
end
|
|
22
33
|
|
|
23
34
|
def webhook_secret
|
data/lib/coolhand.rb
CHANGED