recapi 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. checksums.yaml +7 -0
  2. data/LICENSE +21 -0
  3. data/README.md +104 -0
  4. data/lib/recapi.rb +1192 -0
  5. metadata +46 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: d12730fb7a1d92afff5bc3b2b2383830df795b24fada7104fd8e66c3ac84dc9a
4
+ data.tar.gz: 5accff282fae359cb6fe664dbebf5957b351df0d155199ce04fa145049ffe3d4
5
+ SHA512:
6
+ metadata.gz: eb5e428dd2fe97d7bdc35da5838b84bbcf68b1ece0ee137b8bbdd0dd4b75f24d0b3f30f01d392416a2e4034efa94ac8adc1aaceb570366b46fbf8b9012ea5646
7
+ data.tar.gz: 71cc4af3ea8dd8b424c54e4cf69b94ec50c288fc0dbb7370797064d67cd9d40d06b56c46f94c8d92dd87a237653ef8a808ece2a0c523950ee994d14af2f2ba8b
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 recapi
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,104 @@
1
+ # recapi for Ruby
2
+
3
+ Ruby client for the recapi recommendation API, for Rails, Solidus, Sinatra
4
+ and background jobs. Ruby 3.1 or newer (tested on 3.1 and 4.0), standard
5
+ library only (`net/http`, `json`), one file, no dependency: it requires only
6
+ default gems, so it loads in a Bundler app whose Gemfile lists nothing else.
7
+
8
+ Like the [Python](../python/README.md) and [Go](../go/README.md) libraries,
9
+ it keeps each user's decayed **history**
10
+ ([docs/history-spec.md](../../docs/history-spec.md)) and sends it with every
11
+ recommendation request, **sends events in the background** in batches of up
12
+ to 1,000 with **retries**, **falls back to popular items** when recapi is
13
+ slow or failing, and **attributes** events on recommended items to the
14
+ recommendation that showed them (the dashboard's clicks and conversions). It
15
+ passes the shared vectors in [`spec/`](../../spec).
16
+
17
+ ## Install
18
+
19
+ Until the library is published as a gem, recapi serves it as one file (the
20
+ dashboard's quick start shows the line with your address). In a Rails app,
21
+ save it as `lib/recapi.rb`:
22
+
23
+ ```sh
24
+ curl -fLo lib/recapi.rb https://api.<your-domain>/sdk/ruby/recapi-0.1.3.rb
25
+ ```
26
+
27
+ Once it is published on RubyGems (production releases publish it:
28
+ [docs/deployment.md](../../docs/deployment.md#publishing-the-ruby-library)),
29
+ `gem "recapi"` in your Gemfile, or `gem install recapi`, installs the same
30
+ file as a gem, and `require "recapi"` loads it.
31
+
32
+ ## Quick start
33
+
34
+ ```ruby
35
+ # config/initializers/recapi.rb: one client per process, shared by its threads.
36
+ require Rails.root.join("lib/recapi").to_s
37
+
38
+ # A secret key (rk_sec_…): it stays on your server.
39
+ RECAPI = Recapi::Client.new(ENV.fetch("RECAPI_KEY"), "https://api.example.com",
40
+ store: Rails.cache, logger: Rails.logger)
41
+ ```
42
+
43
+ ```ruby
44
+ # Queued, sent in the background. Levels: :view, :cart, :purchase, :reject or -10..10.
45
+ RECAPI.track(product.id, :view, user_id: current_user&.id, anon_id: cookies[:visitor])
46
+
47
+ recs = RECAPI.recommend(user_id: current_user.id, count: 4)
48
+ recs.items.each do |item|
49
+ item.item_id # a String, as every id recapi returns
50
+ item.metadata # the item's stored object (a Hash), or nil
51
+ # why it is listed ("for_you", "similar", "trending"...), safe to show anyone: map the codes you
52
+ # want to your own label (docs/api.md, "Reason codes"), and show none for a code you don't map.
53
+ label = LABELS[item.why]
54
+ end
55
+ ```
56
+
57
+ Ids may be strings or integers (an ActiveRecord id): they travel as strings.
58
+
59
+ ## Calls
60
+
61
+ A refused argument (an invalid id or level) raises `ArgumentError`, and an
62
+ API failure `Recapi::Error` (`code`, `status`, `retry_after`, `retryable?`,
63
+ `outage?`).
64
+
65
+ | Call | What it does |
66
+ |---|---|
67
+ | `track(item_id, level, user_id:, anon_id:, recommendation_id:)` | Records an interaction: `level` -10 to 10 or `:view` (3), `:cart` (5), `:purchase` (10), `:reject` (-10: never recommend the item to this user again until a positive event on it). The history changes at once; a background thread sends the event within 2 s (at once when a batch of 1,000 is full). |
68
+ | `flush` | Sends the queued events now, with retries (end of a job). A network error, 429 or 5xx is retried 3 times (0.5 s, 1 s, 2 s with jitter, or the `Retry-After`, at most 8 s), then the events stay queued and the sender backs off up to a minute; any other error drops them (logged). |
69
+ | `close` | Stops the sender and sends what is queued, one try per batch. It runs by itself at exit. |
70
+ | `recommend(user_id:, anon_id:, count:, placement:, tags_any:, tags_all:, exclude:, context_item_id:, discovery:)` | A `Recapi::Recommendations` (`recommendation_id`, `fallback?`, `items`). At most 1.5 s; on a network error, timeout or 5xx it returns popular items (`fallback?`, no id), or no item when recapi is down: hide the slot. A 4xx or 429 raises. |
71
+ | `identify(anon_id, user_id)` | At login: the visitor's history and shown items become the user's. The user's first restore from recapi then leaves that visitor's browsing out, so it doesn't count twice. |
72
+ | `popular(count:, tags_any:, tags_all:)` | Popular items, for pages without a user. |
73
+ | `upsert_items(items)` | Creates or replaces items (hashes of `item_id`, `title`, `tags`, `metadata`…), 1,000 per request, retried like events. Returns how many were stored. |
74
+ | `delete_user(user_id)` | Erases a user's events in recapi and their state in the store, retried like events. |
75
+
76
+ `Recapi::Client.new(api_key, base_url, store:, timeout:, flush_interval:,
77
+ max_batch:, logger:)` sets the options. The default `Recapi::MemoryStore`
78
+ keeps the 10,000 most recent users in the process; pass `Rails.cache` (it
79
+ has `read`/`write`/`delete`) or a Redis client (`get`/`set`/`del`) when
80
+ several processes serve the same users. It stores the same JSON as the
81
+ Python library, so both can share one store. Puma's threads share one
82
+ client; a forked worker (Puma cluster, Unicorn, Passenger) starts its own
83
+ queue and sender at its first call. Proxies come from `HTTPS_PROXY`,
84
+ `HTTP_PROXY` and `NO_PROXY`, as `Net::HTTP` reads them. Failures are logged
85
+ to `logger:`, anything with `warn` and `error` (`Rails.logger`, a
86
+ `Logger`); without one they go to `$stderr` as `recapi: …` lines.
87
+
88
+ ## Development
89
+
90
+ `../../scripts/ruby.sh -w test/run.rb [filter]` runs the tests (`make
91
+ test-sdk` does) against a fake API on a local socket. The autopilot
92
+ containers have no Ruby: `scripts/ruby.sh` then fetches Debian's ruby3.1
93
+ once and runs it without RubyGems, so the tests have their own small runner
94
+ (`test/test_case.rb`), no minitest; `test/gem_test.rb` loads RubyGems itself
95
+ to build the gem from `recapi.gemspec` (`lib/recapi.rb`, this README and the
96
+ licence). The API embeds `lib/recapi.rb` and
97
+ serves it (`server/internal/rubysdk`): any change to it needs a new
98
+ `Recapi::VERSION` and the line `go test ./internal/rubysdk` (in `server/`)
99
+ prints for `published.json`.
100
+
101
+ `RECAPI_KEY=rk_sec_… ../../scripts/ruby.sh sample.rb [item ids]` runs
102
+ [`sample.rb`](sample.rb) against staging (`RECAPI_URL` picks another API):
103
+ it tracks views for a new visitor, then prints their recommendations and the
104
+ popular items.
data/lib/recapi.rb ADDED
@@ -0,0 +1,1192 @@
1
+ # frozen_string_literal: true
2
+
3
+ # The recapi client for Ruby servers (Rails, Solidus, Sinatra, jobs): history,
4
+ # batching, retries and fallback. A port of sdk/python, which behaves like the
5
+ # JS SDK; spec/client_vectors.json and spec/history_vectors.json pin what they
6
+ # must agree on. Standard library only: it adds no gem to an app.
7
+ #
8
+ # require_relative "recapi" # or require "recapi"
9
+ # RECAPI = Recapi::Client.new(ENV.fetch("RECAPI_KEY"), "https://api.example.com")
10
+ # RECAPI.track("sku-1", :view, user_id: current_user.id)
11
+ # RECAPI.recommend(user_id: current_user.id, count: 4).items
12
+
13
+ require "json"
14
+ require "net/http"
15
+ require "openssl"
16
+ require "securerandom"
17
+ require "uri"
18
+ require "digest"
19
+ require "weakref"
20
+ begin
21
+ # Net::HTTP inflates gzip replies with it, when the Ruby has it.
22
+ require "zlib"
23
+ rescue LoadError
24
+ nil
25
+ end
26
+
27
+ module Recapi
28
+ VERSION = "0.1.3"
29
+
30
+ # Decayed user history, exactly as docs/history-spec.md (times in unix ms).
31
+ # A port of sdk/python's history.py, snapshot merging included.
32
+ class History
33
+ HALF_LIFE_SECONDS = 604_800
34
+ TOP_K = 50
35
+ RECENT_SIZE = 20
36
+ REJECTED_SIZE = 200
37
+ MAX_ID_BYTES = 256
38
+ RATE = Math.log(2) / HALF_LIFE_SECONDS
39
+ # Go's unicode.IsControl, which the API applies to every id: C0, DEL and C1.
40
+ CONTROL = /[\u0000-\u001f\u007f-\u009f]/.freeze
41
+
42
+ # Whether the API accepts value as an id: 1-256 bytes of UTF-8, no controls.
43
+ def self.id?(value)
44
+ return false unless value.is_a?(String) && !value.empty? && value.bytesize <= MAX_ID_BYTES
45
+
46
+ utf8 = value.encoding == Encoding::UTF_8 ? value : value.dup.force_encoding(Encoding::UTF_8)
47
+ utf8.valid_encoding? && !CONTROL.match?(utf8)
48
+ end
49
+
50
+ def self.decay(dt_ms)
51
+ Math.exp(-RATE * (dt_ms / 1000.0))
52
+ end
53
+
54
+ def self.time?(value)
55
+ (value.is_a?(Integer) || value.is_a?(Float)) && value.finite?
56
+ end
57
+
58
+ def self.weight?(value)
59
+ time?(value) && value >= 0
60
+ end
61
+
62
+ # top: item => [score, t]; recent: [[item, t]] oldest first; rejected:
63
+ # item => time of the newest rejection; last_t: the newest event applied
64
+ # to the recent list (nil when unknown).
65
+ attr_accessor :top, :recent, :rejected, :last_t
66
+
67
+ def initialize
68
+ @top = {}
69
+ @recent = []
70
+ @rejected = {}
71
+ @last_t = nil
72
+ end
73
+
74
+ # Nothing to send, so the server uses its stored history. Rejections alone
75
+ # don't count: the server still has the user's positive events.
76
+ def empty?
77
+ @top.empty? && @recent.empty?
78
+ end
79
+
80
+ # Applies one event: level 0 changes nothing, a negative level is a rejection.
81
+ def apply(item, level, t)
82
+ return if level.zero?
83
+ return reject(item, t) if level.negative?
84
+
85
+ mark = @rejected[item]
86
+ unless mark.nil?
87
+ return if t < mark
88
+
89
+ @rejected.delete(item)
90
+ end
91
+ current = @top[item]
92
+ @top[item] =
93
+ if current.nil?
94
+ [level.to_f, t]
95
+ elsif t >= current[1]
96
+ [current[0] * History.decay(t - current[1]) + level, t]
97
+ else
98
+ [current[0] + level * History.decay(current[1] - t), current[1]]
99
+ end
100
+ if @last_t.nil? || t >= @last_t
101
+ @recent = @recent.reject { |entry| entry[0] == item }
102
+ @recent << [item, t]
103
+ @recent = @recent.last(RECENT_SIZE)
104
+ @last_t = t
105
+ end
106
+ prune(t)
107
+ end
108
+
109
+ # [item, weight at now, t] by weight desc, ties by item id asc.
110
+ def ranked(now)
111
+ entries = @top.map { |item, (s, t)| [item, s * History.decay(now - t), t] }
112
+ entries.sort_by { |item, w, _| [-w, item] }
113
+ end
114
+
115
+ # The rejected items, oldest mark first, ties by item id.
116
+ def rejected_order
117
+ @rejected.keys.sort_by { |item| [@rejected[item], item] }
118
+ end
119
+
120
+ # The request's history and recent, and the rejected items. Weights are
121
+ # decayed to now and rounded half up to 4 decimals like JS Math.round
122
+ # (Ruby's round rounds half away from zero); ages are whole seconds, never negative.
123
+ def wire(now)
124
+ history = ranked(now).first(TOP_K).map do |item, weight, t|
125
+ [item, (weight * 1e4 + 0.5).floor / 1e4, [0, ((now - t) / 1000.0).floor].max]
126
+ end
127
+ [history, @recent.last(RECENT_SIZE).map(&:first), rejected_order]
128
+ end
129
+
130
+ # A history from the server's snapshot, skipping invalid entries. Each top
131
+ # entry is a score at as_of, each rejected item a mark at as_of. last_t
132
+ # stays unknown: as_of is the server clock, and a local clock behind it
133
+ # would see every new event as out of order.
134
+ def self.from_snapshot(snapshot)
135
+ state = new
136
+ return state unless snapshot.is_a?(Hash) && time?(snapshot["as_of"])
137
+
138
+ as_of = snapshot["as_of"]
139
+ list(snapshot["top"]).each do |e|
140
+ state.top[e[0]] = [e[1].to_f, as_of] if e.is_a?(Array) && e.size >= 2 && id?(e[0]) && weight?(e[1])
141
+ end
142
+ list(snapshot["recent"]).each do |item|
143
+ state.recent << [item, as_of] if id?(item) && state.recent.none? { |entry| entry[0] == item }
144
+ end
145
+ state.recent = state.recent.last(RECENT_SIZE)
146
+ list(snapshot["rejected"]).each { |item| state.rejected[item] = as_of if id?(item) }
147
+ state.send(:prune, as_of)
148
+ state.send(:prune_rejected)
149
+ state
150
+ end
151
+
152
+ # The union of two histories built from disjoint events. Top scores add
153
+ # up, expressed at the later of the two times; the recent lists are
154
+ # interleaved by time (self first on ties); rejection marks are united, and
155
+ # a mark older than the item's merged entry is dropped, otherwise the item
156
+ # leaves the top and the recent list.
157
+ def merge(other, now)
158
+ out = History.new
159
+ out.top = @top.dup
160
+ other.top.each do |item, (s, t)|
161
+ mine = out.top[item]
162
+ if mine.nil?
163
+ out.top[item] = [s, t]
164
+ next
165
+ end
166
+ later = [t, mine[1]].max
167
+ out.top[item] = [s * History.decay(later - t) + mine[0] * History.decay(later - mine[1]), later]
168
+ end
169
+
170
+ out.rejected = @rejected.dup
171
+ other.rejected.each { |item, t| out.rejected[item] = [t, out.rejected.fetch(item, t)].max }
172
+ out.rejected.to_a.each do |item, t|
173
+ entry = out.top[item]
174
+ if !entry.nil? && entry[1] > t
175
+ out.rejected.delete(item)
176
+ else
177
+ out.top.delete(item)
178
+ end
179
+ end
180
+
181
+ by_item = {}
182
+ (@recent + other.recent).each_with_index do |(item, t), order|
183
+ existing = by_item[item]
184
+ by_item[item] = [t, order] if existing.nil? || t > existing[0]
185
+ end
186
+ # Rejected items go before the trim, so they don't cost the list its places.
187
+ kept = by_item.reject { |item, _| out.rejected.key?(item) }.map { |item, (t, order)| [t, order, item] }.sort
188
+ out.recent = kept.last(RECENT_SIZE).map { |t, _, item| [item, t] }
189
+
190
+ times = [@last_t, other.last_t].compact
191
+ out.last_t = times.max
192
+ out.send(:prune, now)
193
+ out.send(:prune_rejected)
194
+ out
195
+ end
196
+
197
+ # Merges a server snapshot, keeping the items rejected here rejected: a
198
+ # local rejection is usually one the server hasn't stored yet, so it wins
199
+ # over the snapshot's score.
200
+ def merge_snapshot(snapshot, now)
201
+ trimmed = snapshot.dup
202
+ trimmed.top = snapshot.top.reject { |item, _| @rejected.key?(item) }
203
+ merge(trimmed, now)
204
+ end
205
+
206
+ # The JSON-safe stored form (raw scores), shaped like sdk/python's.
207
+ def encode
208
+ {
209
+ "top" => @top.map { |item, (s, t)| [item, s, t] },
210
+ "recent" => @recent.map { |item, t| [item, t] },
211
+ "rejected" => @rejected.map { |item, t| [item, t] },
212
+ "lastT" => @last_t
213
+ }
214
+ end
215
+
216
+ # Inverse of encode; nil for any malformed field. A state stored before
217
+ # rejections existed has no "rejected".
218
+ def self.decode(raw)
219
+ return nil unless valid_encoding?(raw)
220
+
221
+ state = new
222
+ raw["top"].each { |item, s, t| state.top[item] = [s.to_f, t] }
223
+ state.recent = raw["recent"].map { |item, t| [item, t] }
224
+ raw.fetch("rejected", []).each { |item, t| state.rejected[item] = t }
225
+ state.last_t = raw["lastT"]
226
+ state.send(:prune_rejected)
227
+ state
228
+ end
229
+
230
+ def self.valid_encoding?(raw)
231
+ return false unless raw.is_a?(Hash) && raw.key?("lastT")
232
+
233
+ top = raw["top"]
234
+ recent = raw["recent"]
235
+ rejected = raw.fetch("rejected", [])
236
+ return false unless [top, recent, rejected].all? { |v| v.is_a?(Array) }
237
+
238
+ (raw["lastT"].nil? || time?(raw["lastT"])) &&
239
+ top.all? { |e| pair?(e, 3) && weight?(e[1]) && time?(e[2]) } &&
240
+ (recent + rejected).all? { |e| pair?(e, 2) && time?(e[1]) }
241
+ end
242
+
243
+ def self.pair?(entry, size)
244
+ entry.is_a?(Array) && entry.size == size && id?(entry[0])
245
+ end
246
+
247
+ def self.list(value)
248
+ value.is_a?(Array) ? value : []
249
+ end
250
+
251
+ private_class_method :valid_encoding?, :pair?, :list
252
+
253
+ private
254
+
255
+ def reject(item, t)
256
+ # Dropped, not lowered: a lowered score would decay back towards 0.
257
+ current = @top[item]
258
+ return if !current.nil? && current[1] > t
259
+
260
+ @top.delete(item)
261
+ @recent = @recent.reject { |entry| entry[0] == item }
262
+ @rejected[item] = [t, @rejected.fetch(item, t)].max
263
+ prune_rejected
264
+ end
265
+
266
+ def prune(now)
267
+ return unless @top.size > TOP_K
268
+
269
+ @top = ranked(now).first(TOP_K).to_h { |item, _, _| [item, @top[item]] }
270
+ end
271
+
272
+ def prune_rejected
273
+ return unless @rejected.size > REJECTED_SIZE
274
+
275
+ @rejected = rejected_order.last(REJECTED_SIZE).to_h { |item| [item, @rejected[item]] }
276
+ end
277
+ end
278
+
279
+ # A failed call. status is 0 when no HTTP response was received. code is the
280
+ # API's error code (invalid_request, rate_limited, quota_exceeded...) or a
281
+ # client one: network, timeout, invalid_response.
282
+ class Error < StandardError
283
+ attr_reader :code, :status, :retry_after
284
+
285
+ def initialize(code, message, status: 0, retry_after: nil)
286
+ super(message)
287
+ @code = code
288
+ @status = status
289
+ @retry_after = retry_after
290
+ end
291
+
292
+ # Whether trying again later can succeed (quota_exceeded can't until it resets).
293
+ def retryable?
294
+ return true if %w[network timeout].include?(@code)
295
+ return @code != "quota_exceeded" if @status == 429
296
+
297
+ @status >= 500
298
+ end
299
+
300
+ # Whether recapi couldn't answer in time or failed on its side. A 429 is
301
+ # the tenant's own limit and a 4xx an integration bug: a fallback would hide both.
302
+ def outage?
303
+ %w[network timeout].include?(@code) || @status >= 500
304
+ end
305
+ end
306
+
307
+ # One recommended or popular item. metadata is the object stored with the
308
+ # item (name, picture, link, price...), nil when it has none.
309
+ #
310
+ # why is the reason code to turn into a label for visitors: "for_you",
311
+ # "similar", "next", "bought_together", "trending", "popular", "new",
312
+ # "explore" or "more" (docs/api.md "Reason codes"), nil when the API sent
313
+ # none. New codes may be added: show nothing for an unknown one.
314
+ Item = Struct.new(:item_id, :score, :slot, :metadata, :why, keyword_init: true)
315
+
316
+ # A recommendation list. fallback lists are popular items with no id.
317
+ Recommendations = Struct.new(:recommendation_id, :fallback, :items, keyword_init: true) do
318
+ alias_method :fallback?, :fallback
319
+ end
320
+
321
+ # In-process store keeping the most recently used max_users users. Pass a
322
+ # shared one (Redis, Rails.cache) when several processes serve the same users.
323
+ class MemoryStore
324
+ def initialize(max_users: 10_000)
325
+ @max = [1, max_users].max
326
+ @data = {}
327
+ @lock = Mutex.new
328
+ end
329
+
330
+ def get(key)
331
+ @lock.synchronize do
332
+ value = @data.delete(key)
333
+ @data[key] = value unless value.nil?
334
+ value
335
+ end
336
+ end
337
+
338
+ def set(key, value)
339
+ @lock.synchronize do
340
+ @data.delete(key)
341
+ @data[key] = value
342
+ @data.delete(@data.first[0]) while @data.size > @max
343
+ end
344
+ end
345
+
346
+ def delete(key)
347
+ @lock.synchronize { @data.delete(key) }
348
+ end
349
+ end
350
+
351
+ # The default logger: warnings and errors as "recapi: ..." lines on $stderr.
352
+ # Not the standard Logger: Ruby 4.0 made logger a bundled gem, which a
353
+ # Bundler app without it in its Gemfile can't require.
354
+ class StderrLogger
355
+ def warn(message)
356
+ $stderr.write("#{message}\n")
357
+ end
358
+
359
+ def error(message)
360
+ $stderr.write("#{message}\n")
361
+ end
362
+ end
363
+
364
+ # A recapi client, safe to share between threads (Puma); create one per
365
+ # process. In a forked worker (Unicorn, Puma cluster) it starts afresh.
366
+ class Client
367
+ # API limits (docs/api.md).
368
+ MAX_EVENTS_PER_REQUEST = 1000
369
+ MAX_ITEMS_PER_REQUEST = 1000
370
+ MAX_EXCLUDE = 500
371
+ DEFAULT_COUNT = 10
372
+ MAX_COUNT = 100
373
+ # A widget still empty after 1.5 s has lost the user: recommend() gives up
374
+ # then and falls back, whatever the general timeout (which suits events).
375
+ RECOMMEND_TIMEOUT = 1.5
376
+ # The recommended level scale (docs/api.md "Interaction levels"), by name.
377
+ LEVEL_NAMES = { "view" => 3, "cart" => 5, "purchase" => 10, "reject" => -10 }.freeze
378
+ # Events waiting for a send: a minute-long outage at 150 events/s fits.
379
+ MAX_QUEUE = 10_000
380
+ MAX_FLUSH_BACKOFF = 60.0
381
+ RETRIES = 3
382
+ RETRY_BASE = 0.5
383
+ RETRY_MAX = 8.0
384
+ # Attribution windows, as in the JS SDK: an event on a shown item within
385
+ # 30 minutes of the display is the click; after a click, that item's
386
+ # events are attributed for 24 hours.
387
+ DISPLAY_WINDOW_MS = 30 * 60 * 1000
388
+ CLICK_WINDOW_MS = 24 * 60 * 60 * 1000
389
+ # Shown items remembered per user.
390
+ MAX_SHOWN_ITEMS = 100
391
+ PUBLIC_KEY_PREFIX = "rk_pub_"
392
+ # Open connections kept between calls: one per call in flight at the busiest moment.
393
+ MAX_IDLE_CONNECTIONS = 16
394
+ CODE_BY_STATUS = {
395
+ 400 => "invalid_request", 401 => "unauthorized", 403 => "forbidden",
396
+ 404 => "not_found", 413 => "too_large", 429 => "rate_limited"
397
+ }.freeze
398
+ # Failures before any usable reply: the request may not have reached the API.
399
+ NETWORK_ERRORS = [
400
+ IOError, EOFError, SocketError, SystemCallError, OpenSSL::SSL::SSLError,
401
+ Net::HTTPBadResponse, Net::ProtocolError, *(Zlib::Error if defined?(Zlib::Error))
402
+ ].freeze
403
+ TIMEOUT_ERRORS = [Net::OpenTimeout, Net::ReadTimeout, Net::WriteTimeout].freeze
404
+ # A connection the server closed while idle fails this way, before any reply.
405
+ STALE_ERRORS = [EOFError, Errno::ECONNRESET, Errno::EPIPE, Errno::ECONNABORTED].freeze
406
+ # Made here, not in initialize, whose block would hold the client through
407
+ # self: the weak reference lets a dropped client be collected.
408
+ CLOSE_AT_EXIT = lambda do |ref|
409
+ at_exit do
410
+ ref.__getobj__.close
411
+ rescue WeakRef::RefError
412
+ nil
413
+ end
414
+ end
415
+ private_constant :CLOSE_AT_EXIT
416
+
417
+ # api_key: secret key (rk_sec_...): a server can then restore a user's
418
+ # history from recapi when the store lacks it. base_url: your recapi API
419
+ # origin. store: per-user state, anything with get/set/delete (a Redis
420
+ # client) or read/write/delete (Rails.cache); defaults to a MemoryStore.
421
+ # timeout: per-request seconds (recommend() uses at most 1.5).
422
+ # flush_interval: seconds between background sends. max_batch: events per
423
+ # request, at most the API's 1,000. logger: where failures are logged,
424
+ # anything with warn and error (a Logger, Rails.logger); $stderr by default.
425
+ def initialize(api_key, base_url, store: nil, timeout: 5.0, flush_interval: 2.0,
426
+ max_batch: MAX_EVENTS_PER_REQUEST, logger: nil)
427
+ raise ArgumentError, "api_key is required" unless api_key.is_a?(String) && !api_key.empty?
428
+ raise ArgumentError, "base_url is required" unless base_url.is_a?(String) && !base_url.empty?
429
+
430
+ @api_key = api_key
431
+ @base = base_url.chomp("/")
432
+ @url = URI.parse(@base)
433
+ unless @url.is_a?(URI::HTTP) && !@url.host.to_s.empty?
434
+ raise ArgumentError, "base_url must be an http(s) URL"
435
+ end
436
+
437
+ @store = store || MemoryStore.new
438
+ @timeout = timeout.to_f
439
+ @interval = flush_interval.to_f
440
+ @max_batch = max_batch.to_i.clamp(1, MAX_EVENTS_PER_REQUEST)
441
+ @logger = logger || StderrLogger.new
442
+ # Two environments' keys may share one Redis: their users must not mix.
443
+ @prefix = "recapi:#{Digest::SHA256.hexdigest(api_key)[0, 12]}:"
444
+ # Indirections the tests replace (clock, backoff sleep, jitter).
445
+ @clock = -> { Process.clock_gettime(Process::CLOCK_REALTIME, :millisecond) }
446
+ @sleeper = method(:wait)
447
+ @random = -> { rand }
448
+ init_threading
449
+ CLOSE_AT_EXIT.call(WeakRef.new(self))
450
+ end
451
+
452
+ # Records an interaction: the user's history changes at once, the event
453
+ # is queued. level is an integer -10 to 10 or :view (3), :cart (5),
454
+ # :purchase (10), :reject (-10: never recommend the item to this user
455
+ # again until a positive event on it). recommendation_id overrides the
456
+ # automatic attribution. Ids may be strings or integers.
457
+ def track(item_id, level, user_id: nil, anon_id: nil, recommendation_id: nil)
458
+ level = level_value(level)
459
+ item_id = id_string(item_id)
460
+ raise ArgumentError, "item_id #{ID_RULE}" unless History.id?(item_id)
461
+
462
+ user_id, anon_id, recommendation_id = check_ids(user_id: user_id, anon_id: anon_id,
463
+ recommendation_id: recommendation_id)
464
+ t = now_ms
465
+ attributed = recommendation_id
466
+ key = key_for(user_id, anon_id)
467
+ unless key.nil?
468
+ with_locks(key) do
469
+ state = load(key)
470
+ state.history.apply(item_id, level, t)
471
+ entry = state.shown[item_id]
472
+ # An explicit id of the recommendation that showed the item is
473
+ # still the click on it; another id leaves the entry alone.
474
+ if recommendation_id.nil? || (entry && entry[0] == recommendation_id)
475
+ attributed = attribute(entry, level, t) || recommendation_id
476
+ end
477
+ save(key, state, t)
478
+ end
479
+ end
480
+ enqueue(compact(
481
+ "item_id" => item_id, "level" => level, "user_id" => user_id, "anon_id" => anon_id,
482
+ "recommendation_id" => attributed, "timestamp" => iso(t), "event_id" => SecureRandom.uuid
483
+ ))
484
+ nil
485
+ end
486
+
487
+ # Sends every queued event now, with retries (end of a job or a request in
488
+ # a worker). Never raises: failures are logged, retryable batches stay queued.
489
+ def flush
490
+ drain(retry_errors: true)
491
+ nil
492
+ end
493
+
494
+ # Stops the sender and sends what is queued, one try per batch (runs at
495
+ # exit). No retries, so a deploy never waits on a recapi outage.
496
+ def close
497
+ # In a forked worker, the parent's queue and closed state are not this one's.
498
+ forked
499
+ closing = false
500
+ @mutex.synchronize do
501
+ closing = !@closed
502
+ # Also ends a send's retry sleep, so close waits for at most one try.
503
+ @closed = true
504
+ @cond.broadcast
505
+ end
506
+ drain(retry_errors: false) if closing
507
+ # Also on a later close (the one at exit): calls made after the first
508
+ # may have opened connections again.
509
+ idle = @idle_lock.synchronize { @idle.slice!(0..) }
510
+ idle.each { |conn| finish(conn) }
511
+ nil
512
+ end
513
+
514
+ # Personalised recommendations for a user (or a visitor's anon_id). Sends
515
+ # the history from the store; without one, recapi uses its stored history
516
+ # for user_id and returns it for the store. A history the store started
517
+ # without it (a track() or identify() came first) gets recapi's fetched
518
+ # once, within the same deadline, and merged in. No retry, at most 1.5 s:
519
+ # on a network error, timeout or 5xx it returns popular items (fallback,
520
+ # no id) and logs the error. If those fail too (recapi is down), the list
521
+ # is empty: hide the slot. A 4xx or 429 raises: a fallback would hide an
522
+ # integration bug or a plan limit.
523
+ def recommend(user_id: nil, anon_id: nil, count: 0, placement: nil, tags_any: nil, tags_all: nil,
524
+ exclude: nil, context_item_id: nil, discovery: nil)
525
+ deadline = monotonic + recommend_timeout
526
+ user_id, anon_id = check_ids(user_id: user_id, anon_id: anon_id)
527
+ context_item_id = id_string(context_item_id)
528
+ exclude = exclude&.map { |item| id_string(item) }
529
+ key = key_for(user_id, anon_id)
530
+ t = now_ms
531
+ state = key.nil? ? UserState.new : load(key)
532
+ if user_id && !state.restored && !state.history.empty? && !@api_key.start_with?(PUBLIC_KEY_PREFIX)
533
+ state = restore(key, user_id, state, deadline, t)
534
+ end
535
+ history, recent, rejected = state.history.wire(t)
536
+ excluded = with_rejected(exclude, rejected)
537
+ begin
538
+ response = request(
539
+ "POST", "/v1/recommendations",
540
+ timeout: deadline - monotonic,
541
+ body: compact(
542
+ "user_id" => user_id, "anon_id" => anon_id, "placement" => placement,
543
+ "count" => count.to_i.zero? ? nil : count, "tags_any" => non_empty(tags_any),
544
+ "tags_all" => non_empty(tags_all), "exclude" => excluded, "context_item_id" => context_item_id,
545
+ "history" => history, "recent" => recent, "discovery" => discovery
546
+ )
547
+ )
548
+ rec_id, items = parse_recommendations(response)
549
+ rescue Error => e
550
+ raise unless e.outage?
551
+
552
+ return fallback(e, count, excluded, context_item_id, tags_any, tags_all)
553
+ end
554
+
555
+ unless key.nil?
556
+ with_locks(key) do
557
+ state = load(key)
558
+ items.each do |item|
559
+ state.shown.delete(item.item_id)
560
+ state.shown[item.item_id] = [rec_id, t]
561
+ end
562
+ state.shown.delete(state.shown.first[0]) while state.shown.size > MAX_SHOWN_ITEMS
563
+ # Only trusted when we sent nothing; merged, never replacing what
564
+ # was tracked while the request was in flight, and only once: a
565
+ # concurrent first call may have merged it already.
566
+ snapshot = response["history"]
567
+ done = state.restored && !state.history.empty?
568
+ if history.empty? && recent.empty? && snapshot.is_a?(Hash) && !done
569
+ state.history = state.history.merge_snapshot(History.from_snapshot(snapshot), t)
570
+ state.restored = true
571
+ end
572
+ save(key, state, t)
573
+ end
574
+ end
575
+ Recommendations.new(recommendation_id: rec_id, fallback: false, items: items)
576
+ end
577
+
578
+ # Call at login: the visitor's history and shown items become the user's.
579
+ # Merged into what the store has for user_id (scores add up, as in the
580
+ # history spec), then the visitor's entry is deleted. Without it, a
581
+ # visitor's clicks stop counting once they log in.
582
+ def identify(anon_id, user_id)
583
+ user_id, anon_id = check_ids(user_id: user_id, anon_id: anon_id)
584
+ raise ArgumentError, "anon_id and user_id are required" if anon_id.nil? || user_id.nil?
585
+
586
+ visitor = key_for(nil, anon_id)
587
+ user = key_for(user_id, nil)
588
+ t = now_ms
589
+ with_locks(visitor, user) do
590
+ guest = load(visitor)
591
+ next if guest.history.empty? && guest.history.rejected.empty? && guest.shown.empty?
592
+
593
+ state = load(user)
594
+ state.history = state.history.merge(guest.history, t)
595
+ # recapi's snapshot, fetched later, holds this visitor's browsing too.
596
+ state.anon_id = anon_id unless state.restored
597
+ # Oldest display first, as recommend() keeps them; an item's later display wins.
598
+ shown = (state.shown.to_a + guest.shown.to_a).each_with_index.sort_by { |(_, e), i| [e[1], i] }
599
+ state.shown = {}
600
+ shown.each do |(item, entry), _|
601
+ state.shown.delete(item)
602
+ state.shown[item] = entry
603
+ end
604
+ save(user, state, t)
605
+ begin
606
+ store_call(:delete, visitor)
607
+ rescue StandardError => e
608
+ @logger.error("recapi: could not delete the visitor's stored state: #{e.message}")
609
+ end
610
+ end
611
+ nil
612
+ end
613
+
614
+ # Non-personalised popular items, for pages without a user (no retry).
615
+ def popular(count: DEFAULT_COUNT, tags_any: nil, tags_all: nil)
616
+ fetch_popular(count, tags_any, tags_all, nil)
617
+ end
618
+
619
+ # Creates or replaces items (hashes of the API's item fields: item_id,
620
+ # tags, metadata...), 1,000 per request. Retried like events. Returns how
621
+ # many were stored; an invalid item raises, and nothing of its batch of
622
+ # 1,000 is stored.
623
+ def upsert_items(items)
624
+ upserted = 0
625
+ items.each_slice(MAX_ITEMS_PER_REQUEST) do |batch|
626
+ body = { "items" => batch.map { |item| item.to_h.transform_keys(&:to_s) } }
627
+ response = with_retry { request("POST", "/v1/items:batch", body: body) }
628
+ upserted += response.is_a?(Hash) ? response.fetch("upserted", 0).to_i : 0
629
+ end
630
+ upserted
631
+ end
632
+
633
+ # Erases a user's events in recapi (GDPR) and their state in the store.
634
+ def delete_user(user_id)
635
+ user_id = id_string(user_id)
636
+ raise ArgumentError, "user_id #{ID_RULE}" unless History.id?(user_id)
637
+
638
+ with_retry { request("DELETE", "/v1/users/#{escape(user_id)}") }
639
+ begin
640
+ store_call(:delete, key_for(user_id, nil))
641
+ rescue StandardError => e
642
+ @logger.error("recapi: could not delete the user's stored state: #{e.message}")
643
+ end
644
+ nil
645
+ end
646
+
647
+ ID_RULE = "must be 1-256 bytes of UTF-8 without control characters"
648
+
649
+ # What the store keeps per user: the history and the shown items. shown
650
+ # maps item => [recommendation id, shown at, clicked at?], oldest first.
651
+ # restored is set once the history holds recapi's stored one. anon_id is
652
+ # the visitor identify merged in before that: recapi's snapshot also holds
653
+ # that visitor's browsing, so the restore asks it to leave it out.
654
+ class UserState
655
+ attr_accessor :history, :shown, :restored, :anon_id
656
+
657
+ def initialize(history = nil, shown = nil, restored = false, anon_id = nil)
658
+ @history = history || History.new
659
+ @shown = shown || {}
660
+ @restored = restored
661
+ @anon_id = anon_id
662
+ end
663
+
664
+ def dump(now)
665
+ shown = @shown.select { |_, entry| Client.attributable?(entry, now) }.map { |item, entry| [item, *entry] }
666
+ state = { "history" => @history.encode, "shown" => shown.last(MAX_SHOWN_ITEMS) }
667
+ state["restored"] = true if @restored
668
+ state["anon_id"] = @anon_id if @anon_id && !@anon_id.empty?
669
+ JSON.generate(state)
670
+ end
671
+
672
+ # nil when raw is malformed.
673
+ def self.parse(raw)
674
+ data = JSON.parse(raw)
675
+ return nil unless data.is_a?(Hash) && data["shown"].is_a?(Array)
676
+
677
+ history = History.decode(data["history"])
678
+ return nil if history.nil?
679
+
680
+ shown = {}
681
+ data["shown"].each do |e|
682
+ shown[e[0]] = e[1..] if e.is_a?(Array) && e.size.between?(3, 4) && History.id?(e[0])
683
+ end
684
+ anon_id = data["anon_id"]
685
+ new(history, shown, data["restored"] == true, History.id?(anon_id) ? anon_id : nil)
686
+ rescue JSON::ParserError, TypeError
687
+ nil
688
+ end
689
+ end
690
+
691
+ def self.attributable?(entry, t)
692
+ return t - entry[1] <= DISPLAY_WINDOW_MS if entry.size < 3
693
+
694
+ t - entry[2] <= CLICK_WINDOW_MS
695
+ end
696
+
697
+ private
698
+
699
+ def init_threading
700
+ # Also runs in a forked worker: the parent's queue is the parent's to
701
+ # send, its sender thread did not survive the fork, and its connections
702
+ # must not be shared.
703
+ @pid = Process.pid
704
+ @user_locks = Array.new(64) { Mutex.new }
705
+ @queue = []
706
+ @mutex = Mutex.new
707
+ @cond = ConditionVariable.new
708
+ @send_lock = Mutex.new
709
+ @thread = nil
710
+ @failures = 0
711
+ @closed = false
712
+ @idle = []
713
+ @idle_lock = Mutex.new
714
+ end
715
+
716
+ def forked
717
+ init_threading if @pid != Process.pid
718
+ end
719
+
720
+ def now_ms
721
+ @clock.call
722
+ end
723
+
724
+ def monotonic
725
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
726
+ end
727
+
728
+ def recommend_timeout
729
+ [@timeout, RECOMMEND_TIMEOUT].min
730
+ end
731
+
732
+ # Waits seconds; returns true, at once, when interruptible and the client
733
+ # is or becomes closed.
734
+ def wait(seconds, interruptible)
735
+ return sleep(seconds) && false unless interruptible
736
+
737
+ deadline = monotonic + seconds
738
+ @mutex.synchronize do
739
+ until @closed
740
+ left = deadline - monotonic
741
+ return false if left <= 0
742
+
743
+ @cond.wait(@mutex, left)
744
+ end
745
+ true
746
+ end
747
+ end
748
+
749
+ # --- events ---------------------------------------------------------------
750
+
751
+ def enqueue(event)
752
+ forked
753
+ closed = @mutex.synchronize do
754
+ if @thread.nil? && !@closed
755
+ @thread = Thread.new { run }
756
+ @thread.name = "recapi-sender"
757
+ @thread.report_on_exception = false
758
+ end
759
+ @queue << event
760
+ drop_overflow if @queue.size > MAX_QUEUE
761
+ @cond.broadcast if @queue.size >= @max_batch
762
+ @closed
763
+ end
764
+ drain(retry_errors: false) if closed
765
+ end
766
+
767
+ def drop_overflow
768
+ dropped = @queue.size - MAX_QUEUE
769
+ @queue.shift(dropped)
770
+ @logger.warn("recapi: event queue full, dropped the #{dropped} oldest events")
771
+ end
772
+
773
+ def run
774
+ loop do
775
+ @mutex.synchronize do
776
+ deadline = monotonic + [@interval * 2**@failures, MAX_FLUSH_BACKOFF].min
777
+ # A full batch goes at once, except after a failure: then the
778
+ # backoff holds, or a down API would be hammered.
779
+ until @closed || (@failures.zero? && @queue.size >= @max_batch)
780
+ left = deadline - monotonic
781
+ break if left <= 0
782
+
783
+ @cond.wait(@mutex, left)
784
+ end
785
+ return if @closed
786
+ end
787
+ drain(retry_errors: true)
788
+ end
789
+ end
790
+
791
+ def drain(retry_errors:)
792
+ forked
793
+ @send_lock.synchronize do
794
+ loop do
795
+ batch = @mutex.synchronize { @queue.shift(@max_batch) }
796
+ return if batch.empty?
797
+
798
+ begin
799
+ send_events(batch, retry_errors)
800
+ rescue StandardError => e
801
+ if e.is_a?(Error) && !e.retryable?
802
+ @logger.error("recapi: dropped #{batch.size} events: #{e.message}")
803
+ next
804
+ end
805
+ # Anything else keeps the batch, a bug of ours included: event ids
806
+ # make a resend harmless, and a dead sender would lose every event.
807
+ @logger.error("recapi: unexpected error sending events: #{e.class}: #{e.message}") unless e.is_a?(Error)
808
+ @mutex.synchronize do
809
+ @queue.unshift(*batch)
810
+ drop_overflow if @queue.size > MAX_QUEUE
811
+ @failures += 1
812
+ end
813
+ @logger.warn("recapi: sending #{batch.size} events failed, kept: #{e.message}")
814
+ return
815
+ end
816
+ @mutex.synchronize { @failures = 0 }
817
+ end
818
+ end
819
+ end
820
+
821
+ def send_events(events, retry_errors)
822
+ send = -> { request("POST", "/v1/events", body: { "events" => events }) }
823
+ begin
824
+ response = retry_errors ? with_retry(interruptible: true, &send) : send.call
825
+ rescue Error => e
826
+ # Over the API's body limit (ids mostly quotes or backslashes, which
827
+ # JSON escapes): halves fit where the whole didn't. If the second half
828
+ # fails, the batch is kept whole; event ids make the resend harmless.
829
+ raise if e.status != 413 || events.size < 2
830
+
831
+ half = events.size / 2
832
+ send_events(events[0, half], retry_errors)
833
+ send_events(events[half..], retry_errors)
834
+ return
835
+ end
836
+ rejected = response.is_a?(Hash) ? response["rejected"] : nil
837
+ return unless rejected.is_a?(Array) && !rejected.empty?
838
+
839
+ # Invalid events are reported, never retried: they would fail again.
840
+ detail = rejected.first(5).map do |r|
841
+ r.is_a?(Hash) && r["index"].is_a?(Integer) ? r.merge("event" => events[r["index"]]) : r
842
+ end
843
+ @logger.error("recapi: #{rejected.size} event(s) rejected: #{JSON.generate(detail)}")
844
+ end
845
+
846
+ # Calls the block, retrying retryable errors with jittered exponential
847
+ # backoff. Retry-After wins over the backoff, and both are capped at 8 s.
848
+ # Once the client closes (interruptible), the last error is raised instead.
849
+ def with_retry(interruptible: false)
850
+ attempt = 0
851
+ begin
852
+ yield
853
+ rescue Error => e
854
+ raise if attempt >= RETRIES || !e.retryable?
855
+
856
+ backoff = RETRY_BASE * 2**attempt * (0.5 + @random.call / 2.0)
857
+ attempt += 1
858
+ raise if @sleeper.call([RETRY_MAX, e.retry_after || backoff].min, interruptible)
859
+
860
+ retry
861
+ end
862
+ end
863
+
864
+ # --- recommendations ------------------------------------------------------
865
+
866
+ def fallback(error, count, excluded, context_item_id, tags_any, tags_all)
867
+ # The requested length without the excluded items, so a few more are
868
+ # asked for. 0 is the recommendations default; popular refuses it.
869
+ n = count.to_i.zero? ? DEFAULT_COUNT : count.to_i
870
+ skip = (excluded || []).to_h { |item| [item, true] }
871
+ skip[context_item_id] = true unless context_item_id.nil?
872
+ begin
873
+ popular = fetch_popular([n + skip.size, MAX_COUNT].min, tags_any, tags_all, recommend_timeout)
874
+ rescue Error => e
875
+ # recapi is down altogether: an empty list the page can hide beats an
876
+ # exception that fails the whole page.
877
+ @logger.warn("recapi: recommendations failed (#{error.message}), popular too, served no items: #{e.message}")
878
+ return Recommendations.new(recommendation_id: nil, fallback: true, items: [])
879
+ end
880
+ @logger.warn("recapi: recommendations failed, served popular items: #{error.message}")
881
+ items = popular.reject { |item| skip.key?(item.item_id) }.first(n)
882
+ Recommendations.new(recommendation_id: nil, fallback: true, items: items)
883
+ end
884
+
885
+ # Merges recapi's stored history into a local one that lacks it (a fresh
886
+ # store). The recommendation's reply carries it only when the request
887
+ # sends no history, which would drop this one. So it is asked for apart,
888
+ # once: a failure leaves the state as it was, to be tried on the next call.
889
+ def restore(key, user_id, state, deadline, t)
890
+ begin
891
+ # Half the time left, so the recommendation itself still fits.
892
+ query = state.anon_id ? { "anon_id" => state.anon_id } : nil
893
+ path = "/v1/users/#{escape(user_id)}/history"
894
+ snapshot = request("GET", path, query: query, timeout: (deadline - monotonic) / 2)
895
+ raise Error.new("invalid_response", "malformed history response") unless snapshot.is_a?(Hash)
896
+ rescue Error => e
897
+ @logger.warn("recapi: could not restore the user's stored history: #{e.message}")
898
+ return state
899
+ end
900
+ with_locks(key) do
901
+ state = load(key)
902
+ # A concurrent first call may have merged it already.
903
+ unless state.restored
904
+ state.history = state.history.merge_snapshot(History.from_snapshot(snapshot), t)
905
+ state.restored = true
906
+ state.anon_id = nil
907
+ save(key, state, t)
908
+ end
909
+ end
910
+ state
911
+ end
912
+
913
+ def fetch_popular(count, tags_any, tags_all, timeout)
914
+ query = {}
915
+ query["tags_any"] = tags_any.join(",") if tags_any && !tags_any.empty?
916
+ query["tags_all"] = tags_all.join(",") if tags_all && !tags_all.empty?
917
+ query["count"] = count.to_s
918
+ response = request("GET", "/v1/popular", timeout: timeout, query: query)
919
+ raw = response.is_a?(Hash) ? response["items"] : nil
920
+ raise Error.new("invalid_response", "malformed popular response") unless raw.is_a?(Array)
921
+
922
+ raw.map { |entry| parse_item(entry, "backfill") }
923
+ end
924
+
925
+ def parse_recommendations(response)
926
+ unless response.is_a?(Hash) && response["recommendation_id"].is_a?(String) && response["items"].is_a?(Array)
927
+ raise Error.new("invalid_response", "malformed recommendations response")
928
+ end
929
+
930
+ [response["recommendation_id"], response["items"].map { |entry| parse_item(entry, "exploit") }]
931
+ end
932
+
933
+ def parse_item(entry, slot)
934
+ unless entry.is_a?(Hash) && entry["item_id"].is_a?(String)
935
+ raise Error.new("invalid_response", "malformed item in response")
936
+ end
937
+
938
+ metadata = entry["metadata"]
939
+ why = entry["why"]
940
+ score = entry["score"]
941
+ Item.new(
942
+ item_id: entry["item_id"],
943
+ score: score.is_a?(Numeric) ? score.to_f : 0.0,
944
+ slot: entry["slot"].is_a?(String) && !entry["slot"].empty? ? entry["slot"] : slot,
945
+ metadata: metadata.is_a?(Hash) && !metadata.empty? ? metadata : nil,
946
+ why: why.is_a?(String) && !why.empty? ? why : nil
947
+ )
948
+ end
949
+
950
+ # The recommendation an event at t belongs to; a first click is recorded.
951
+ def attribute(entry, level, t)
952
+ return nil if entry.nil? || !Client.attributable?(entry, t)
953
+
954
+ entry << t if level.positive? && entry.size < 3
955
+ entry[0]
956
+ end
957
+
958
+ # The caller's exclude plus the rejected items, newest first, up to the API limit.
959
+ def with_rejected(exclude, rejected)
960
+ return exclude&.dup if rejected.empty?
961
+
962
+ out = (exclude || []).uniq
963
+ seen = out.to_h { |item| [item, true] }
964
+ rejected.reverse_each do |item|
965
+ break if out.size >= MAX_EXCLUDE
966
+ next if seen.key?(item)
967
+
968
+ seen[item] = true
969
+ out << item
970
+ end
971
+ out
972
+ end
973
+
974
+ # --- state ----------------------------------------------------------------
975
+
976
+ def key_for(user_id, anon_id)
977
+ return "#{@prefix}u:#{user_id}" if user_id
978
+ return "#{@prefix}a:#{anon_id}" if anon_id
979
+
980
+ nil
981
+ end
982
+
983
+ # Always in the same order, or two logins could each hold the other's lock.
984
+ def with_locks(*keys, &block)
985
+ locks = keys.map { |key| key.hash % @user_locks.size }.uniq.sort.map { |i| @user_locks[i] }
986
+ locks.reverse.reduce(block) { |inner, lock| -> { lock.synchronize(&inner) } }.call
987
+ end
988
+
989
+ # The store is a cache: when it fails, recapi's stored history is still
990
+ # there, so the call goes on without it.
991
+ def load(key)
992
+ raw = begin
993
+ store_call(:get, key)
994
+ rescue StandardError => e
995
+ @logger.error("recapi: store read failed: #{e.class}: #{e.message}")
996
+ return UserState.new
997
+ end
998
+ return UserState.new if raw.nil? || raw.empty?
999
+
1000
+ state = UserState.parse(raw)
1001
+ return state unless state.nil?
1002
+
1003
+ @logger.warn("recapi: discarding a malformed stored user state")
1004
+ UserState.new
1005
+ end
1006
+
1007
+ def save(key, state, now)
1008
+ store_call(:set, key, state.dump(now))
1009
+ rescue StandardError => e
1010
+ @logger.error("recapi: store write failed: #{e.class}: #{e.message}")
1011
+ end
1012
+
1013
+ # Redis clients have get/set/del, Rails.cache read/write/delete.
1014
+ def store_call(action, key, value = nil)
1015
+ case action
1016
+ when :get then @store.respond_to?(:get) ? @store.get(key) : @store.read(key)
1017
+ when :set then @store.respond_to?(:set) ? @store.set(key, value) : @store.write(key, value)
1018
+ else @store.respond_to?(:delete) ? @store.delete(key) : @store.del(key)
1019
+ end
1020
+ end
1021
+
1022
+ # --- HTTP -----------------------------------------------------------------
1023
+
1024
+ def request(method, path, body: nil, query: nil, timeout: nil)
1025
+ params = (query || {}).dup
1026
+ headers = { "Accept" => "application/json", "User-Agent" => "recapi-ruby/#{VERSION}" }
1027
+ # A public key travels in the URL only on GET /v1/popular (CDN caching);
1028
+ # the API refuses a secret key there.
1029
+ if method == "GET" && @api_key.start_with?(PUBLIC_KEY_PREFIX)
1030
+ params["key"] = @api_key
1031
+ else
1032
+ headers["Authorization"] = "Bearer #{@api_key}"
1033
+ end
1034
+ target = @url.path + path
1035
+ target += "?#{URI.encode_www_form(params)}" unless params.empty?
1036
+ data = nil
1037
+ unless body.nil?
1038
+ # Raw UTF-8: escaped, non-Latin ids would take up to 3 times the bytes.
1039
+ data = JSON.generate(body)
1040
+ headers["Content-Type"] = "application/json"
1041
+ end
1042
+ limit = timeout.nil? ? @timeout : timeout
1043
+ raise Error.new("timeout", "the deadline passed before the request was sent") if limit <= 0
1044
+
1045
+ begin
1046
+ response = exchange(method, target, data, headers, limit)
1047
+ rescue *TIMEOUT_ERRORS
1048
+ raise Error.new("timeout", "request timed out after #{format('%.3g', limit)} s")
1049
+ rescue *NETWORK_ERRORS => e
1050
+ raise Error.new("network", "#{e.class}: #{e.message}")
1051
+ end
1052
+ status = response.code.to_i
1053
+ # Net::HTTP follows no redirect: a 3xx (an http:// base_url sent on to
1054
+ # https) fails like a 4xx, or a DELETE would look done.
1055
+ raise http_error(status, response) unless (200..299).cover?(status)
1056
+
1057
+ parse_json(response.body)
1058
+ end
1059
+
1060
+ # Sends one request on an idle connection, or a new one. A connection the
1061
+ # server closed while idle fails before any reply comes: it is replaced
1062
+ # once. A failure once the reply has started is never resent: the API has
1063
+ # the request (a recommendation would count twice).
1064
+ def exchange(method, target, data, headers, limit)
1065
+ forked
1066
+ start = monotonic
1067
+ # The most recently used: the least likely to have timed out.
1068
+ conn = @idle_lock.synchronize { @idle.pop }
1069
+ 2.times do |attempt|
1070
+ remaining = limit - (monotonic - start)
1071
+ raise Net::OpenTimeout, "deadline" if remaining <= 0
1072
+
1073
+ reused = !conn.nil?
1074
+ conn ||= connection
1075
+ conn.open_timeout = conn.read_timeout = conn.write_timeout = remaining
1076
+ req = Net::HTTPGenericRequest.new(method, !data.nil?, method != "HEAD", target, headers)
1077
+ req.body = data unless data.nil?
1078
+ begin
1079
+ conn.start unless conn.started?
1080
+ response = conn.request(req)
1081
+ rescue *STALE_ERRORS
1082
+ finish(conn)
1083
+ raise if !reused || attempt.positive?
1084
+
1085
+ conn = nil
1086
+ next
1087
+ rescue Exception # rubocop:disable Lint/RescueException
1088
+ finish(conn)
1089
+ raise
1090
+ end
1091
+ # A body cut short of its Content-Length reads as whole here.
1092
+ length = response["Content-Length"]
1093
+ if length && response.body.to_s.bytesize < length.to_i && response["Content-Encoding"].nil?
1094
+ finish(conn)
1095
+ raise EOFError, "the reply was cut short"
1096
+ end
1097
+ keep(conn, response)
1098
+ return response
1099
+ end
1100
+ end
1101
+
1102
+ def connection
1103
+ # Net::HTTP honours HTTPS_PROXY, HTTP_PROXY and NO_PROXY itself.
1104
+ conn = Net::HTTP.new(@url.host, @url.port)
1105
+ conn.use_ssl = @url.scheme == "https"
1106
+ # No silent resend: a recommendation would count twice.
1107
+ conn.max_retries = 0
1108
+ # Reused until the server closes it (Net::HTTP's 2 s would handshake again).
1109
+ conn.keep_alive_timeout = 60
1110
+ conn
1111
+ end
1112
+
1113
+ # Returns a connection whose last reply was read whole to the idle ones.
1114
+ def keep(conn, response)
1115
+ return finish(conn) if response["Connection"].to_s.downcase == "close"
1116
+
1117
+ kept = @idle_lock.synchronize { @idle.size < MAX_IDLE_CONNECTIONS && @idle.push(conn) }
1118
+ finish(conn) unless kept
1119
+ end
1120
+
1121
+ def finish(conn)
1122
+ conn.finish if conn.started?
1123
+ rescue IOError, SystemCallError
1124
+ nil
1125
+ end
1126
+
1127
+ def http_error(status, response)
1128
+ payload = parse_json(response.body)
1129
+ body = payload.is_a?(Hash) && payload["error"].is_a?(Hash) ? payload["error"] : {}
1130
+ code = body["code"]
1131
+ code = CODE_BY_STATUS[status] || (status >= 500 ? "internal" : "http_error") if code.nil? || code == ""
1132
+ message = body["message"]
1133
+ message = "HTTP #{status}" if message.nil? || message == ""
1134
+ Error.new(code.to_s, message.to_s, status: status, retry_after: retry_after(response["Retry-After"]))
1135
+ end
1136
+
1137
+ def retry_after(header)
1138
+ seconds = Float(header.to_s.strip)
1139
+ seconds.finite? && seconds >= 0 ? seconds : nil
1140
+ rescue ArgumentError
1141
+ nil
1142
+ end
1143
+
1144
+ def parse_json(raw)
1145
+ return nil if raw.nil? || raw.empty?
1146
+
1147
+ JSON.parse(raw)
1148
+ rescue JSON::ParserError
1149
+ nil
1150
+ end
1151
+
1152
+ # --- arguments ------------------------------------------------------------
1153
+
1154
+ def level_value(level)
1155
+ name = level.is_a?(Symbol) ? level.to_s : level
1156
+ return LEVEL_NAMES[name] if name.is_a?(String) && LEVEL_NAMES.key?(name)
1157
+ return level if level.is_a?(Integer) && level.between?(-10, 10)
1158
+
1159
+ raise ArgumentError, "level must be an integer from -10 to 10 or one of #{LEVEL_NAMES.keys.join(', ')}"
1160
+ end
1161
+
1162
+ # Rails ids are integers: they travel as strings, which the API requires.
1163
+ def id_string(value)
1164
+ value.is_a?(Integer) ? value.to_s : value
1165
+ end
1166
+
1167
+ def check_ids(**ids)
1168
+ ids.map do |name, value|
1169
+ value = id_string(value)
1170
+ raise ArgumentError, "#{name} #{ID_RULE}" unless value.nil? || History.id?(value)
1171
+
1172
+ value
1173
+ end
1174
+ end
1175
+
1176
+ def escape(id)
1177
+ URI.encode_www_form_component(id).gsub("+", "%20")
1178
+ end
1179
+
1180
+ def non_empty(values)
1181
+ values.nil? || values.empty? ? nil : values.to_a
1182
+ end
1183
+
1184
+ def compact(fields)
1185
+ fields.compact
1186
+ end
1187
+
1188
+ def iso(t_ms)
1189
+ Time.at(t_ms / 1000, t_ms % 1000, :millisecond).utc.strftime("%Y-%m-%dT%H:%M:%S.%LZ")
1190
+ end
1191
+ end
1192
+ end
metadata ADDED
@@ -0,0 +1,46 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: recapi
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.3
5
+ platform: ruby
6
+ authors:
7
+ - recapi
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2026-11-02 00:00:00.000000000 Z
12
+ dependencies: []
13
+ description: 'Ruby client for the recapi recommendation API: per-user history, event
14
+ batching, retries, popular fallback. Standard library only.'
15
+ email:
16
+ executables: []
17
+ extensions: []
18
+ extra_rdoc_files: []
19
+ files:
20
+ - LICENSE
21
+ - README.md
22
+ - lib/recapi.rb
23
+ homepage: https://recapi.io
24
+ licenses:
25
+ - MIT
26
+ metadata: {}
27
+ post_install_message:
28
+ rdoc_options: []
29
+ require_paths:
30
+ - lib
31
+ required_ruby_version: !ruby/object:Gem::Requirement
32
+ requirements:
33
+ - - ">="
34
+ - !ruby/object:Gem::Version
35
+ version: '3.1'
36
+ required_rubygems_version: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ requirements: []
42
+ rubygems_version: 3.3.27
43
+ signing_key:
44
+ specification_version: 4
45
+ summary: Ruby client for the recapi recommendation API
46
+ test_files: []