recapi 0.1.7 → 0.1.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f8edb463693be71a5cac3a101441d5fe6c045e5e58f4003025291995ec5c9e44
-  data.tar.gz: 818ea20b257819a76f15c832af44b83694681fc36995f08b91987ac2b9e8afdf
+  metadata.gz: 45a27aa9c5eae2fa2088d58247ebd703b60df834bc22e508737d89f17e90a8a2
+  data.tar.gz: 7443f75901c5e2600ea181f3a61277bc9b2e5b2820aad2c999dba4b8353521b6
 SHA512:
-  metadata.gz: cb54199fc5b2ff480e94be20d594598a2cbda5fb2b1b8ffb854b3dd4ac0dd3d9d2858b05e67cc79b264d93f41f4e5132b57c50812aa5c626582583d369763df7
-  data.tar.gz: 33fbcbfc1cca21caf75849d106e9500236d6adf2ac897c74b49e9731eb79b6680cb1054e637677babbd0ebc52fc853aa7a999ff10a1e11911da5ff01bb538c41
+  metadata.gz: 856de23aa6d4936301ac4d9a2dbd2de72c4ffea40e44ed41459f7b310ff36dc232103f9e8e6cd11d265790e6b95eda80cea2d6c40087d68df252d2861c5f2956
+  data.tar.gz: dfd97019465b529c84fb4d315876acac76a414432fe3d53a901ec037f3b03c5490c2ce76ed2f3000651ff904f37115c9da9d046c59c99286924abcfea1ded8fc

data/README.md

@@ -30,7 +30,7 @@ line with your address): save it as `lib/recapi.rb` and
 `require Rails.root.join("lib/recapi").to_s` instead.
 
 ```sh
-curl -fLo lib/recapi.rb https://api.<your-domain>/sdk/ruby/recapi-0.1.7.rb
+curl -fLo lib/recapi.rb https://api.<your-domain>/sdk/ruby/recapi-0.1.8.rb
 ```
 
 ## Quick start
@@ -64,16 +64,19 @@ Ids may be strings or integers (an ActiveRecord id): they travel as strings.
 
 A refused argument (an invalid id or level) raises `ArgumentError`, and an
 API failure `Recapi::Error` (`code`, `status`, `retry_after`, `retryable?`,
-`outage?`).
+`outage?`). `recommend` and `popular` still raise on a bad request (400, 404,
+413, other 3xx/4xx) or a reply that isn't the API's; an outage, a limit or a
+refused key can start on a live site with no deploy, so they answer with a list
+and a log line.
 
 | Call | What it does |
 |---|---|
 | `track(item_id, level, user_id:, anon_id:, recommendation_id:)` | Records an interaction: `level` -10 to 10 or `:view` (3), `:cart` (5), `:purchase` (10), `:reject` (-10: never recommend the item to this user again until a positive event on it). The history changes at once; a background thread sends the event within 2 s (at once when a batch of 1,000 is full). |
 | `flush` | Sends the queued events now, with retries (end of a job). A network error, 429 or 5xx is retried 3 times (0.5 s, 1 s, 2 s with jitter, or the `Retry-After`, at most 8 s), then the events stay queued and the sender backs off up to a minute; any other error drops them (logged). |
 | `close` | Stops the sender and sends what is queued, one try per batch. It runs by itself at exit. |
-| `recommend(user_id:, anon_id:, count:, placement:, tags_any:, tags_all:, exclude:, context_item_id:, discovery:)` | A `Recapi::Recommendations` (`recommendation_id`, `fallback?`, `items`). At most 1.5 s; on a network error, timeout or 5xx it returns popular items (`fallback?`, no id), or no item when recapi is down: hide the slot. A 4xx or 429 raises. |
+| `recommend(user_id:, anon_id:, count:, placement:, tags_any:, tags_all:, exclude:, context_item_id:, discovery:)` | A `Recapi::Recommendations` (`recommendation_id`, `fallback?`, `items`). At most 1.5 s; on a network error, timeout, 5xx or 429 it returns popular items (`fallback?`, no id), or no item when those fail too: hide the slot; on a 401 or 403 no item at once. Any other failure (a bad request, a reply that isn't the API's) raises. |
 | `identify(anon_id, user_id)` | At login: the visitor's history and shown items become the user's. The user's first restore from recapi then leaves that visitor's browsing out, so it doesn't count twice. |
-| `popular(count:, tags_any:, tags_all:)` | Popular items, for pages without a user. |
+| `popular(count:, tags_any:, tags_all:)` | Popular items, for pages without a user. On a network error, timeout, 5xx, 429, 401 or 403: an empty list and a log line. Other failures as in `recommend`. |
 | `upsert_items(items)` | Creates or replaces items (hashes of `item_id`, `title`, `tags`, `metadata`…), 1,000 per request, retried like events. Returns how many were stored. |
 | `delete_user(user_id)` | Erases a user's events in recapi and their state in the store, retried like events. |
 
@@ -95,8 +98,9 @@ queue and sender at its first call. Proxies come from `HTTPS_PROXY`,
 to `logger:`, anything with `warn` and `error` (`Rails.logger`, a
 `Logger`); without one they go to `$stderr` as `recapi: …` lines, with lost
 events (a refused batch, rejected events) at most one line a minute for each
-error code and for rejected events, so a revoked key on a busy server doesn't
-flood the log, and a full queue one line until a send works. A failed write to
+error code and for rejected events, and `recommend`'s and `popular`'s failures
+at most one line a minute for each error code, so a revoked key on a busy server
+doesn't flood the log, and a full queue one line until a send works. A failed write to
 `$stderr` (a full disk, a closed stream) is ignored. A logger you give gets
 every line, and if it raises, that reaches your code.
 

data/lib/recapi.rb

@@ -25,7 +25,7 @@ rescue LoadError
 end
 
 module Recapi
-  VERSION = "0.1.7"
+  VERSION = "0.1.8"
 
   # Decayed user history, exactly as docs/history-spec.md (times in unix ms).
   # A port of sdk/python's history.py, snapshot merging included.
@@ -297,8 +297,7 @@ module Recapi
       @status >= 500
     end
 
-    # Whether recapi couldn't answer in time or failed on its side. A 429 is
-    # the tenant's own limit and a 4xx an integration bug: a fallback would hide both.
+    # Whether recapi couldn't answer in time or failed on its side.
     def outage?
       %w[network timeout].include?(@code) || @status >= 500
     end
@@ -551,10 +550,12 @@ module Recapi
     # for user_id and returns it for the store. A history the store started
     # without it (a track() or identify() came first) gets recapi's fetched
     # once, within the same deadline, and merged in. No retry, at most 1.5 s:
-    # on a network error, timeout or 5xx it returns popular items (fallback,
-    # no id) and logs the error. If those fail too (recapi is down), the list
-    # is empty: hide the slot. A 4xx or 429 raises: a fallback would hide an
-    # integration bug or a plan limit.
+    # on a network error, timeout, 5xx or 429 it returns popular items (fallback,
+    # no id) and logs the error. If those fail too (recapi is down, or the plan's
+    # limit covers them), the list is empty: hide the slot. On a 401 or 403 (a
+    # revoked key, a suspended shop) it is empty at once, as popular would be
+    # refused too. Any other failure raises: a bad request or a reply that isn't
+    # the API's can't start on a live site without a deploy.
     def recommend(user_id: nil, anon_id: nil, count: 0, placement: nil, tags_any: nil, tags_all: nil,
                   exclude: nil, context_item_id: nil, discovery: nil)
       deadline = monotonic + recommend_timeout
@@ -582,7 +583,7 @@ module Recapi
         )
         rec_id, items = parse_recommendations(response)
       rescue Error => e
-        raise unless e.outage?
+        raise unless spare_the_page?(e)
 
         return fallback(e, count, excluded, context_item_id, tags_any, tags_all)
       end
@@ -646,9 +647,16 @@ module Recapi
       nil
     end
 
-    # Non-personalised popular items, for pages without a user (no retry).
+    # Non-personalised popular items, for pages without a user (no retry). On a
+    # network error, timeout, 5xx, 429, 401 or 403 it logs the error and returns
+    # an empty list: hide the slot. Any other failure raises, as in recommend.
     def popular(count: DEFAULT_COUNT, tags_any: nil, tags_all: nil)
       fetch_popular(count, tags_any, tags_all, nil)
+    rescue Error => e
+      raise unless spare_the_page?(e)
+
+      log_list(e, "recapi: popular items failed, served no items: #{e.code}: #{e.message}")
+      []
     end
 
     # Creates or replaces items (hashes of the API's item fields: item_id,
@@ -892,17 +900,32 @@ module Recapi
     # A revoked key refuses every batch and the sender runs on with no backoff,
     # so $stderr gets one line a minute per kind (error code, or rejected
     # events): one line per outage would have aged out of the log by the time
-    # someone looks. A logger given gets them all. Runs under @send_lock.
+    # someone looks. A logger given gets them all.
     def log_lost(kind, message)
-      if @limit_lost_lines
-        now = @clock.call
+      @logger.error(message) if line_due?(kind)
+    end
+
+    # A list call's failure, one per page view through a revoked key or an
+    # outage: limited like lost events on $stderr, one line a minute per code
+    # whichever call hit it. Its own kind, or a list line would mute the line
+    # saying events are being lost.
+    def log_list(error, message)
+      @logger.warn(message) if line_due?("list #{error.code}")
+    end
+
+    # Any app thread can get here, next to the sender: not while holding @mutex.
+    def line_due?(kind)
+      return true unless @limit_lost_lines
+
+      now = @clock.call
+      @mutex.synchronize do
         last = @lost_printed[kind]
         # abs: a clock set back must not mute the kind until it catches up.
-        return if last && (now - last).abs < LOST_LINE_MS
+        next false if last && (now - last).abs < LOST_LINE_MS
 
         @lost_printed[kind] = now
+        true
       end
-      @logger.error(message)
     end
 
     # Calls the block, retrying retryable errors with jittered exponential
@@ -925,7 +948,25 @@ module Recapi
 
     # --- recommendations ------------------------------------------------------
 
+    # Whether recommend and popular answer this failure with a list instead of
+    # raising it. An outage, a limit or a revoked key can start on a live site
+    # with no deploy, and a page must not pay for it: the log and the dashboard
+    # show it. What's left (a bad request, a reply that isn't the API's) can't
+    # start without a deploy, and a developer sees it on the first run.
+    def spare_the_page?(error)
+      error.outage? || error.status == 429 || key_refused?(error)
+    end
+
+    # A key or shop recapi won't serve: any other call fails too.
+    def key_refused?(error)
+      [401, 403].include?(error.status)
+    end
+
     def fallback(error, count, excluded, context_item_id, tags_any, tags_all)
+      if key_refused?(error)
+        log_list(error, "recapi: recommendations refused, served no items: #{error.code}: #{error.message}")
+        return Recommendations.new(recommendation_id: nil, fallback: true, items: [])
+      end
       # The requested length without the excluded items, so a few more are
       # asked for. 0 is the recommendations default; popular refuses it.
       n = count.to_i.zero? ? DEFAULT_COUNT : count.to_i
@@ -934,12 +975,13 @@ module Recapi
       begin
         popular = fetch_popular([n + skip.size, MAX_COUNT].min, tags_any, tags_all, recommend_timeout)
       rescue Error => e
-        # recapi is down altogether: an empty list the page can hide beats an
-        # exception that fails the whole page.
-        @logger.warn("recapi: recommendations failed (#{error.message}), popular too, served no items: #{e.message}")
+        # recapi is down or the limit covers popular too: an empty list the page
+        # can hide beats an exception that fails the whole page.
+        log_list(error, "recapi: recommendations failed (#{error.code}: #{error.message}), popular too, " \
+                        "served no items: #{e.code}: #{e.message}")
         return Recommendations.new(recommendation_id: nil, fallback: true, items: [])
       end
-      @logger.warn("recapi: recommendations failed, served popular items: #{error.message}")
+      log_list(error, "recapi: recommendations failed, served popular items: #{error.code}: #{error.message}")
       items = popular.reject { |item| skip.key?(item.item_id) }.first(n)
       Recommendations.new(recommendation_id: nil, fallback: true, items: items)
     end
@@ -956,7 +998,9 @@ module Recapi
         snapshot = request("GET", path, query: query, timeout: (deadline - monotonic) / 2)
         raise Error.new("invalid_response", "malformed history response") unless snapshot.is_a?(Hash)
       rescue Error => e
-        @logger.warn("recapi: could not restore the user's stored history: #{e.message}")
+        # The same limit as the list's own line: a revoked key fails this first
+        # on every recommend for a user with a local history.
+        log_list(e, "recapi: could not restore the user's stored history: #{e.code}: #{e.message}")
         return state
       end
       with_locks(key) do

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: recapi
 version: !ruby/object:Gem::Version
-  version: 0.1.7
+  version: 0.1.8
 platform: ruby
 authors:
 - recapi