devbench 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: be38d75135b0a120747e5abf3889afadb0997377f8d58510abb5a3baadf41d64
-  data.tar.gz: d2791c38e21e79d204e71a22aa3337793b9641976827aeb0f5e6dd54a7f811dc
+  metadata.gz: dd716a7741353f88f381775e58d7d613d7a5d50ff69287feb65f86f25c38e13d
+  data.tar.gz: '03358a6c9d097c8b4adc3daacc8c032699aba2a81814284caa0287254134f95c'
 SHA512:
-  metadata.gz: 3f323a0cd9aa89ed80cee8fa78c8598c992f8ac15200bc7bf98ee5cc211079dc1458237c782479c150643a944683b27fe3b295222d1f7b2e8ec5a3bf0bacab5c
-  data.tar.gz: ec896fbc0a0d4d65d6259a83186f81df942cc09d0f437995cc34f80a4ab66c0864b90c948b944054c8404fa1f16047b002f7f5d2c56f84a9500dbaf304bc4d25
+  metadata.gz: bdc43a37a987bcc745a17bc957d704bbf93835a6ef475c66e7228acb9c64edcad1ee2ab4acd9b68020f21435d16f6cf263e378db615cd49242e20407115e6d9a
+  data.tar.gz: b0c9a3d9884106853d587c2944cb8436bd333ed197fd4630ae17a713e4f959068e96faaf40ea54bfe1cc87f21532f6b482686dd6451ee39d022d9d683e6229d5

data/README.md

@@ -1,24 +1,27 @@
 # Dev Bench for Ruby and Rails
 
 Everything Sentry's Rails SDK captures by default, plus who was affected,
-trace propagation and the handled-failure header — so an app can turn
-Sentry off. Zero runtime dependencies; loads in plain Ruby, hooks itself
+the server log lines of a failing request, the browser sensor, trace
+propagation and the handled-failure header — so an app can turn Sentry
+off. Zero runtime dependencies; loads in plain Ruby, hooks itself
 into Rails when Rails is there.
 
 ## Install (Rails)
 
-```ruby
-# Gemfile
-gem 'devbench'
+```sh
+bundle add devbench
+bin/rails generate devbench
 ```
 
+Then set **one** value in the app's environment — the DSN `adt dsn create`
+printed for this environment:
+
 ```sh
-# The DSN Dev Bench gave you for this environment
-DEVBENCH_DSN=https://<key>@adt-ingest.onrender.com
+DEVBENCH_DSN=https://<public>:<secret>@adt-ingest.onrender.com
 ```
 
-That is the whole install. No middleware line, no initializer, no process
-to run next to the app. Check it:
+That is the whole install: no browser key, no sidecar, no service names,
+no initializer, no middleware line. Check it:
 
 ```sh
 bin/rails devbench:test
@@ -30,16 +33,51 @@ It prints the HTTP result, or says plainly what is wrong (`DEVBENCH_DSN is
 not set`, `HTTP 401: the key in DEVBENCH_DSN was rejected`, `could not reach
 …`) and exits non-zero. Outside Rake: `Devbench.test!`.
 
+What the generator does, and nothing else (run it again and nothing
+changes):
+
+- puts `<%= devbench_script_tag %>` in `app/views/layouts/application.html.erb`
+  just before `</head>` (`.haml` / `.slim`: `= devbench_script_tag` as the
+  last line of `head`). Without that layout it prints what to add where.
+- if `config/initializers/content_security_policy.rb` defines a policy,
+  adds `https://unpkg.com` to `script_src`, and the ingest host and
+  `https://*.storage.supabase.co` (evidence uploads) to `connect_src`.
+
+### The browser tag
+
+`devbench_script_tag` renders the browser sensor, version-pinned to this
+gem (upgrading the gem upgrades the sensor), with **only the public part**
+of `DEVBENCH_DSN`:
+
+```html
+<script src="https://unpkg.com/devbench@0.7.0/dist/devbench.min.js"
+        data-dsn="https://<public>@adt-ingest.onrender.com" data-release="<release>" defer></script>
+```
+
+The secret never reaches a page. Without a DSN (or with a 0.5 single-key
+DSN, or `DEVBENCH_ENABLED=false`) it renders nothing at all, so a
+development or test environment without a DSN serves no tag. If the app
+uses CSP nonces (`content_security_policy_nonce_generator`), the tag
+carries the request's nonce.
+
+**CSP:** if your policy is defined somewhere other than the standard
+initializer, add these yourself:
+
+```ruby
+policy.script_src  ..., "https://unpkg.com"
+policy.connect_src ..., "https://adt-ingest.onrender.com", "https://*.storage.supabase.co"
+```
+
 ## Configuration
 
 All optional except the DSN.
 
 | Variable | Default | |
 |---|---|---|
-| `DEVBENCH_DSN` | — (falls back to `ADT_DSN`) | `https://<key>@<host>[:port]`. Plain `http://` is accepted only for localhost. |
-| `DEVBENCH_SERVICE` | your app's module, underscored (`AcmeShop` → `acme_shop`); `app` outside Rails | Groups this app's issues. |
-| `DEVBENCH_RELEASE` | `GIT_SHA`, `SOURCE_VERSION`, `RENDER_GIT_COMMIT`, else empty | Which deploy an occurrence came from. |
-| `DEVBENCH_ENABLED` | on | `false` turns everything off: no middleware, no hooks, nothing sent. |
+| `DEVBENCH_DSN` | — (falls back to `ADT_DSN`) | `https://<public>:<secret>@<host>[:port]` from `adt dsn create`. The server authenticates with the secret; the page gets the public part. A 0.5 `https://<key>@<host>` still works (no browser tag, see below). Plain `http://` is accepted only for localhost. |
+| `DEVBENCH_SERVICE` | your app's module, underscored (`AcmeShop` → `acme_shop`); `app` outside Rails; **`<that>-sidekiq` in a Sidekiq process** | Groups this app's issues. Set it and it wins everywhere. |
+| `DEVBENCH_RELEASE` | the first found of: a `REVISION` file in `Rails.root` (Capistrano writes one), `HEROKU_SLUG_COMMIT`, `KAMAL_VERSION`, `GITHUB_SHA`, `GIT_SHA`, `SOURCE_VERSION`, `RENDER_GIT_COMMIT`; else empty | Which deploy an occurrence came from. `c.release` in code wins over all of these. |
+| `DEVBENCH_ENABLED` | on | `false` turns everything off: no middleware, no hooks, no tag, nothing sent. |
 
 Or from code, e.g. `config/initializers/devbench.rb`:
 
@@ -51,7 +89,7 @@ end
 ```
 
 A DSN that does not parse logs one `[devbench]` warning and reports
-nothing; it never raises into the app.
+nothing; it never raises into the app. Neither key is ever printed.
 
 ## What is captured automatically
 
@@ -94,8 +132,9 @@ Like the browser sensor, the gem does not send one request per error:
   Dev Bench sensor). Repeats cost a counter increment.
 - **Once a minute** a background thread sends the counts — fingerprint,
   how many, first/last seen, and up to 20 affected users each — in one
-  small request. Nothing is sent for a quiet minute, and what is left is
-  sent when the process exits (waiting at most 2 seconds).
+  small request. Nothing is sent for a quiet minute (except a small poll
+  while the process holds log lines, below), and what is left is sent
+  when the process exits (waiting at most 2 seconds).
 - **Detail only on request.** When Dev Bench sees a fingerprint for the
   first time it asks for evidence, and the gem uploads the stack and one
   example message — **templated and scrubbed first** (emails, numbers,
@@ -107,6 +146,33 @@ Like the browser sensor, the gem does not send one request per error:
   counted). Safe under forking servers (Puma, Unicorn, Sidekiq): each
   worker reports on its own.
 
+## Server log lines
+
+Triage reads the server's log lines for the user action that failed. With
+`DEVBENCH_DSN` set, the gem keeps them itself — no sidecar:
+
+- **What:** every line written to `Rails.logger` and `Sidekiq.logger`
+  while a request or job carries a trace (`x-adt-trace` from the browser
+  sensor; a job inherits the trace of the request that enqueued it).
+  Untraced lines are not kept. Rails 7.1+: a capturing logger joins
+  `Rails.logger`'s broadcast (it follows the app's level and `silence`,
+  and writes nothing); older Rails and `Sidekiq.logger`: a tee after the
+  logger's own write. **What your logger writes is unchanged.**
+- **Bounded:** per process, the newest 10,000 lines or 4 MiB, nothing older
+  than 15 minutes, 4 KiB per line. A logging call is never blocked on I/O
+  and never raises because of this.
+- **Sent only when asked:** when Dev Bench asks for a trace, the flush
+  thread sends that trace's lines from this process — the newest ≤ 200
+  lines / 64 KiB — **redacted first** with the same strict rules as
+  evidence (quoted values, numbers, emails, cards, tokens, credentials,
+  IPs and two-word names removed), plus any redaction rules Dev Bench has
+  learned for your app. A process holding lines it was not asked about
+  sends nothing but a small poll once a minute while it holds them.
+- **Forking servers:** each Puma worker and Sidekiq process keeps and
+  answers for its own lines; Dev Bench assembles them.
+
+A Rack app without Rails: `Devbench.capture_logs(logger)`.
+
 ## Who was affected
 
 Set it once per request, typically in `ApplicationController`:
@@ -198,11 +264,12 @@ browser cannot read `x-adt-handled` unless the server lists it in
 than overwrites. **A CORS layer that replaces the response header set will
 strip it** and detection will silently never fire; check the ordering.
 
-## Optional: the sidecar (server logs)
+## Optional: the sidecar
 
 The Dev Bench sidecar is a separate process that reads your app's log
 stream on the host and answers triage's requests for the log lines of one
-user action. It is optional; nothing above needs it.
+user action. It is optional: with a DSN the gem keeps those lines itself
+(above). It remains for setups that cannot load the gem.
 
 Without a DSN, the gem reports to the sidecar's unix socket instead
 (`$ADT_SIDECAR_SOCKET`, default `/tmp/adt-sidecar.sock`), exactly as 0.4
@@ -212,6 +279,26 @@ gem does not also write to its socket, so nothing is counted twice.
 Fingerprints are identical in both modes, so moving between them keeps
 every issue.
 
+## Upgrading from 0.5
+
+Nothing is required: a 0.5 single-key `DEVBENCH_DSN` keeps reporting
+exactly as before. To get server log lines and the browser tag from the
+same value, mint a pair and replace the DSN:
+
+```sh
+adt dsn create --tenant <tenant> --environment production
+# https://<public>:<secret>@adt-ingest.onrender.com
+```
+
+then run `bin/rails generate devbench` for the tag and CSP. A page that
+loaded the browser sensor with its own DSN can drop that and use the tag.
+
+One default changed: a Sidekiq process without `DEVBENCH_SERVICE` now
+reports as `<app>-sidekiq` (it was `<app>`), so job failures are grouped
+apart from web failures — and an existing job issue reappears once under
+the new name. Set `DEVBENCH_SERVICE` in the Sidekiq process to keep the
+old name.
+
 ## Upgrading from `adt` (0.4)
 
 - Change the Gemfile line to `gem 'devbench'`. `require 'adt'` and every
@@ -221,5 +308,5 @@ every issue.
 - `config.middleware.insert_before 0, ADT::Middleware` can stay or go: the
   Railtie sees it and does not insert a second one.
 - Running the sidecar and want to keep it that way? Change nothing else.
-  To report directly instead, set `DEVBENCH_DSN` (a key minted with
-  `--source server`); the sidecar then only serves logs.
+  To report directly instead, set `DEVBENCH_DSN` (from `adt dsn create`);
+  see "Upgrading from 0.5".

data/lib/devbench/config.rb

@@ -4,23 +4,31 @@ require 'uri'
 
 module Devbench
   # Where reports go, parsed from DEVBENCH_DSN (docs/SERVER_SDK_SPEC.md, "Dev
-  # Bench naming and configuration"):
+  # Bench naming and configuration"; DECISIONS #161):
   #
-  #   https://<ingest key>@<host>[:port]
+  #   https://<public>:<secret>@<host>[:port]    from `adt dsn create`
+  #   https://<key>@<host>[:port]                0.5 and earlier: one key
   #
-  # scheme://host[:port] is the ingest base and the userinfo is the key. A
-  # path or query, if present, is ignored: the key alone identifies tenant,
-  # environment and source.
+  # scheme://host[:port] is the ingest base. The secret (a server key) is
+  # what this process authenticates with; the public part (a browser key) is
+  # only ever handed to the page, via #browser_dsn. A single-key DSN keeps
+  # working with its one key and has no browser part — that key is a server
+  # secret and must never reach a page. A path or query, if present, is
+  # ignored: the keys alone identify tenant, environment and source.
   class DSN
     class Invalid < StandardError; end
 
     LOOPBACK = /\A(?:localhost|127(?:\.\d{1,3}){3}|::1|\[::1\])\z|\.localhost\z/i
+    SHAPE = 'https://<public>:<secret>@<host>'
 
-    attr_reader :base, :key
+    attr_reader :base, :key, :public_key
 
-    def initialize(base, key)
+    # key: what the server sends as X-ADT-Key (the secret, or a 0.5 DSN's
+    # single key). public_key: the browser key, or nil.
+    def initialize(base, key, public_key = nil)
       @base = base.freeze
       @key = key.freeze
+      @public_key = public_key&.freeze
       freeze
     end
 
@@ -28,8 +36,26 @@ module Devbench
       "#{@base}/v1/flush"
     end
 
-    # The DSN without its key, for messages and logs. The key is a secret
-    # on a server: it must never be printed.
+    # The self-test endpoint: validates the key, stores nothing.
+    def check_url
+      "#{@base}/v1/check"
+    end
+
+    def logs_url
+      "#{@base}/v1/logs"
+    end
+
+    # The DSN a browser may hold: https://<public>@host. Nil when there is
+    # no public part.
+    def browser_dsn
+      return nil if @public_key.nil?
+
+      scheme, rest = @base.split('://', 2)
+      "#{scheme}://#{URI.encode_www_form_component(@public_key)}@#{rest}"
+    end
+
+    # The DSN without its keys, for messages and logs. The secret must never
+    # be printed.
     def to_s
       @base
     end
@@ -37,7 +63,7 @@ module Devbench
     alias inspect to_s
 
     # Raises DSN::Invalid with a message that says what to fix. Never echoes
-    # the key back.
+    # either key back.
     def self.parse(raw)
       text = raw.to_s.strip
       raise Invalid, 'is empty' if text.empty?
@@ -45,28 +71,32 @@ module Devbench
       uri = begin
         URI.parse(text)
       rescue URI::Error
-        raise Invalid, 'is not a URL (expected https://<key>@<host>)'
+        raise Invalid, "is not a URL (expected #{SHAPE})"
       end
 
       scheme = uri.scheme.to_s.downcase
-      raise Invalid, 'must start with https:// (expected https://<key>@<host>)' unless %w[http https].include?(scheme)
+      raise Invalid, "must start with https:// (expected #{SHAPE})" unless %w[http https].include?(scheme)
 
       host = uri.host.to_s
-      raise Invalid, 'has no host (expected https://<key>@<host>)' if host.empty?
+      raise Invalid, "has no host (expected #{SHAPE})" if host.empty?
 
-      key = URI.decode_www_form_component(uri.user.to_s)
-      raise Invalid, 'has no key (expected https://<key>@<host>)' if key.strip.empty?
+      user = URI.decode_www_form_component(uri.user.to_s).strip
+      secret = URI.decode_www_form_component(uri.password.to_s).strip
+      raise Invalid, "has no key (expected #{SHAPE})" if user.empty? && secret.empty?
 
-      # The key is a real secret on a server. Over plain http it would cross
-      # the network readable by anything on the path, so http is accepted
-      # only for a loopback ingest (local development and tests).
+      # The secret is a real secret on a server. Over plain http it would
+      # cross the network readable by anything on the path, so http is
+      # accepted only for a loopback ingest (local development and tests).
       if scheme == 'http' && !LOOPBACK.match?(host)
         raise Invalid, "uses http:// for #{host}: the key would cross the network in plaintext; use https://"
       end
 
       port = uri.port && uri.port != uri.default_port ? ":#{uri.port}" : ''
       host = "[#{host}]" if host.include?(':') && !host.start_with?('[')
-      new("#{scheme}://#{host}#{port}", key.strip)
+      base = "#{scheme}://#{host}#{port}"
+      return new(base, user) if secret.empty?
+
+      new(base, secret, user.empty? ? nil : user)
     end
   end
 
@@ -77,12 +107,22 @@ module Devbench
   #                      sidecar's socket, as before 0.5.
   #   DEVBENCH_SERVICE   this app's name. Default: the Rails application's
   #                      module, underscored (AcmeShop -> acme_shop);
-  #                      else "app".
-  #   DEVBENCH_RELEASE   the deployed version. Default: GIT_SHA,
-  #                      SOURCE_VERSION, RENDER_GIT_COMMIT, else "".
+  #                      else "app"; with "-sidekiq" appended in a
+  #                      Sidekiq process.
+  #   DEVBENCH_RELEASE   the deployed version. Default, first found: the
+  #                      REVISION file in the app root (Capistrano),
+  #                      HEROKU_SLUG_COMMIT, KAMAL_VERSION, GITHUB_SHA,
+  #                      GIT_SHA, SOURCE_VERSION, RENDER_GIT_COMMIT,
+  #                      else "".
   #   DEVBENCH_ENABLED   "false" turns everything off.
   class Configuration
-    RELEASE_FALLBACKS = %w[GIT_SHA SOURCE_VERSION RENDER_GIT_COMMIT].freeze
+    # Tried in order after the REVISION file when no release is configured.
+    # Common deploy tools and platforms only; never an app's own variable.
+    RELEASE_FALLBACKS = %w[HEROKU_SLUG_COMMIT KAMAL_VERSION GITHUB_SHA GIT_SHA SOURCE_VERSION RENDER_GIT_COMMIT].freeze
+    # Capistrano writes the deployed commit here, in the release's root.
+    REVISION_FILE = 'REVISION'
+    # A commit or tag, not a document: read no more than this.
+    REVISION_MAX_BYTES = 255
     OFF = %w[false 0 no off].freeze
 
     # Strings, or nil for "not set".
@@ -94,8 +134,8 @@ module Devbench
     def initialize(env = ENV)
       @dsn = presence(env['DEVBENCH_DSN']) || presence(env['ADT_DSN'])
       @service = presence(env['DEVBENCH_SERVICE'])
-      @release = presence(env['DEVBENCH_RELEASE']) ||
-                 RELEASE_FALLBACKS.lazy.map { |name| presence(env[name]) }.find(&:itself)
+      @release = presence(env['DEVBENCH_RELEASE'])
+      @release_env = RELEASE_FALLBACKS.lazy.map { |name| presence(env[name]) }.find(&:itself)
       @enabled = !OFF.include?(env['DEVBENCH_ENABLED'].to_s.strip.downcase)
       @flush_interval = 60
     end
@@ -106,16 +146,58 @@ module Devbench
 
     # The service name sent with every flush and folded into every
     # fingerprint. Resolved when reporting starts, after Rails has booted.
+    #
+    # A Sidekiq process (Sidekiq.server?) defaults to "<app>-sidekiq", so a
+    # typical app's web and job processes are told apart with no
+    # DEVBENCH_SERVICE at all. An explicit service always wins.
     def resolved_service
-      presence(@service) || rails_service || 'app'
+      explicit = presence(@service)
+      return explicit if explicit
+
+      base = rails_service || 'app'
+      sidekiq_server? ? "#{base}-sidekiq" : base
     end
 
+    # The release sent with every flush and on the browser tag. Explicit
+    # (configure or DEVBENCH_RELEASE) wins; else the REVISION file, read
+    # once reporting starts (after Rails has booted, so Rails.root is
+    # known); else the deploy variables in RELEASE_FALLBACKS order; else "".
     def resolved_release
-      @release.to_s
+      explicit = presence(@release)
+      return explicit if explicit
+
+      @detected_release ||= (revision_file_release || @release_env).to_s
     end
 
     private
 
+    # The first line of <app root>/REVISION, or nil when there is none or it
+    # cannot be read. The app root is Rails.root, else the working directory.
+    def revision_file_release
+      path = File.join(app_root, REVISION_FILE)
+      return nil unless File.file?(path)
+
+      presence(File.open(path, 'rb') { |f| f.read(REVISION_MAX_BYTES) }.to_s.lines.first)
+    rescue StandardError
+      nil
+    end
+
+    def app_root
+      if defined?(::Rails) && ::Rails.respond_to?(:root) && ::Rails.root
+        ::Rails.root.to_s
+      else
+        Dir.pwd
+      end
+    rescue StandardError
+      Dir.pwd
+    end
+
+    def sidekiq_server?
+      defined?(::Sidekiq) && ::Sidekiq.respond_to?(:server?) && ::Sidekiq.server? ? true : false
+    rescue StandardError, SystemStackError
+      false
+    end
+
     def presence(value)
       text = value.to_s.strip
       text.empty? ? nil : text