sixty 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7b90bc82ca6cbc6175f936bbe3fae61559f2ed25496038a0606113c6d07d337c
+  data.tar.gz: 76a91e28604d7f832e9470cc1232a00eb4a918aeb603ee2d59cbcd8869f6bc58
+SHA512:
+  metadata.gz: 1714f467d461bb3dfea330ea8cbebd48b849e5203c859f2d1062a306f620e08b593aa56d4b1f579064eb69f99c433a3d89abf856842182f87d6e306a83baceda
+  data.tar.gz: '048abc678da88c8279c84b229918c1130b8cbdbbaa76606b00976e18ef80b41d8dc9574845a96a23418b3ca9f14f5796aa30bf72c88b55eb554a612235a94ee3'

data/README.md

@@ -0,0 +1,326 @@
+# sixty (Ruby)
+
+Zero-configuration performance drift detection for Ruby and Rails services.
+
+```ruby
+# Gemfile
+gem 'sixty'
+```
+
+```bash
+export SIXTY_API_KEY=sixty_sk_…
+export SIXTY_SERVICE=checkout-api
+export SIXTY_RELEASE=$(git rev-parse --short HEAD)
+```
+
+That is the install. In a Rails application the railtie adds the middleware,
+subscribes to `sql.active_record` and wraps controller actions on boot — there
+is no initializer to write and nothing to call.
+
+Outside Rails:
+
+```ruby
+require 'sixty'
+Sixty.init
+
+use Sixty::Instrument::Rack   # config.ru
+```
+
+## What it reports
+
+Not latency alone. **Shape** — the numbers that barely move on a warm
+development database and take production down a week later:
+
+| signal | the question it answers |
+|---|---|
+| rows per call | did this query start returning 30,000 rows instead of 30? |
+| queries per call | did this method start issuing 20 queries instead of 1? |
+| round trips per call | did this read start fetching in three hundred batches? |
+| self time | is *my* code slower, or is something I call slower? |
+| query plan | did this statement stop using its index? |
+| bytes | did the payload go from 4KB to 2MB? |
+
+Each is recorded per operation, per release. The collector compares one release
+against the release before it — which is why `SIXTY_RELEASE` matters more than
+any other setting here. Without it there is no "before".
+
+## Which database you use
+
+Postgres, MySQL and MongoDB, and nothing to configure for any of them.
+
+| client | how it is picked up | rows means | plans |
+|---|---|---|---|
+| ActiveRecord (any adapter) | automatic | `row_count` from the adapter | Postgres only |
+| `pg` | automatic, outside Rails | `ntuples`, or `cmd_tuples` for a write | no |
+| `mysql2` / `trilogy` | automatic, outside Rails | result count, or `affected_rows` | no |
+| `mongo` (and Mongoid) | automatic | documents returned, or affected | no |
+
+**ActiveRecord excludes the raw drivers, deliberately.** `sql.active_record` and
+a patched `PG::Connection` see the same query — the adapter runs it through the
+driver — so installing both would count every query in a Rails app twice. A
+doubled `db_calls` is not a visible error; it is a plausible number that makes
+every fanout finding wrong by a factor of two. The subscriber wins wherever
+ActiveRecord is in the process, and the driver patches cover everything else:
+Sinatra, Sequel, ROM, workers, scripts. `SIXTY_INSTRUMENT=active_record,pg`
+overrides that if your app really does issue queries down both paths.
+
+Prepared statements are covered on both `pg` and `mysql2`: the text is
+remembered where it is prepared and looked up where it is executed, because
+`statement.execute(id)` carries no SQL and an application that uses them would
+otherwise report no database calls at all.
+
+Installation is retried on every flush, because a driver is a constant that may
+not exist yet: `Sixty.init` before `require 'mysql2'` would otherwise report no
+queries for the life of the process, silently.
+
+### What is not covered
+
+Stated plainly, because a blind spot nobody wrote down is the failure this
+project exists to prevent:
+
+- **`pg`'s asynchronous API** (`send_query` + `get_result`). The synchronous
+  family — `exec`, `exec_params`, `exec_prepared`, `async_exec` — is patched;
+  pairing a send with a later get is stateful and is not attempted.
+- **Fiber schedulers** (Falcon, the `async` gem). Context is fiber-local, so a
+  span opened in one fiber is not visible in a fiber spawned from it: counts and
+  durations stay correct, parent/child edges are lost. Thread-per-request
+  servers — Puma, Unicorn, Sidekiq — are unaffected.
+- **Query plans outside ActiveRecord**, which needs a connection pool to borrow
+  from. See below.
+- **`Sixty.instrument(Klass, :method)`** wraps the methods you name and does not
+  follow ones defined later.
+
+### MySQL is read by different rules, not the same ones
+
+`"alice@example.com"` is a quoted *identifier* in Postgres — schema, kept, and
+keeping it is what makes `select "userId" from t` readable in the feed — and in
+MySQL's default `sql_mode` the same bytes are a string *literal*, which is
+exactly the PII this agent exists never to transmit. Backticks are the mirror
+image. So the dialect is asked of the adapter rather than guessed from the text,
+and it decides which lexer runs.
+
+### MongoDB has no statement, so identity is the shape
+
+There is nothing to strip literals from: a filter is a tree where the values sit
+beside the keys. So the identity is *built* from keys only, and a value has no
+path into it at all:
+
+```
+find orders {filter{user_id},limit,sort{created_at}}
+aggregate orders [$match{status}][$lookup{from}][$unwind]
+insert orders {documents{email,total,user_id}}
+```
+
+`{_id: {$in: [...]}}` with three ids and with three thousand is one operation —
+the same fold `in (?, ?, ?) → in (?)` performs for SQL. A pipeline is the
+opposite case: every stage counts, and in order, because collapsing it would
+hide the `$lookup` that turned one query into an N+1.
+
+**A cursor is one operation, however many batches it took.** A `find` returning
+thirty thousand documents at the default batch size is the initial command plus
+roughly three hundred `getMore` round trips, each a network wait. Recording each
+of those as its own operation would tell you your method makes three hundred
+database calls — the signature of an N+1 your code does not contain — and send
+you looking for a loop when the fix is `batch_size`. So the span stays open
+until the cursor is drained: `db_calls` says one call, `rows` says thirty
+thousand, and `round_trips` says three hundred, which is its own finding with
+its own fix.
+
+Batch size is deliberately *not* part of the identity, for the same reason:
+setting it is the change the round-trips signal exists to report, and an
+identity that moved with it would leave the detector with nothing to compare.
+`limit` is part of the identity, because a limit changes what you asked for.
+
+The instrumentation is the driver's own command-monitoring API rather than a
+monkey patch, which is only possible because the Ruby driver is synchronous:
+events are published on the calling thread, so a query still knows which method
+issued it. (`@sixty-sh/node` cannot do this — in Node those events fire after the
+caller's async context is gone, so it patches the collection instead.)
+
+### Why MySQL and MongoDB get no query plans
+
+An index that stopped being used is the highest-value finding this agent could
+produce, and it is still refused for both. Postgres has `EXPLAIN (GENERIC_PLAN)`,
+which plans a statement *without binding a parameter* — there is no step at
+which a value could enter it. MySQL and MongoDB can only explain a query that
+still has its values in it, so capturing a plan there would mean retaining
+somebody's data in order to compose a command out of it. The same refusal, for
+the same reason, in all three agents.
+
+Postgres plans are also refused for a statement that arrived carrying literals —
+what an app with prepared statements disabled produces — because this agent
+explains on the flush thread rather than in front of a user, and queueing such a
+statement would mean holding those values until the next flush.
+
+## Your own code
+
+Controllers and queries are instrumented automatically. Service objects, query
+objects and jobs — the layer where an N+1 is actually born — are one line:
+
+```ruby
+class OrdersQuery
+  include Sixty::Instrumented
+
+  def for_user(id) = Order.where(user_id: id).limit(30).to_a
+  def enrich(orders) = ...
+end
+```
+
+Every public instance method becomes an operation, with its queries and rows
+attributed to it. Private methods are left alone, accessors are skipped, and
+arguments, keyword arguments, blocks, return values, raised exceptions and
+method visibility pass through unchanged — there is a test for each of those,
+because an instrumentation layer that changes program behaviour is unshippable.
+
+For a class you do not own:
+
+```ruby
+Sixty.instrument(Stripe::Charge, :create)
+```
+
+And for anything else:
+
+```ruby
+Sixty.trace('nightly-reconciliation') { ... }
+Sixty.annotate(:rows, results.length)
+```
+
+## What it costs
+
+Measured, not asserted:
+
+| | overhead |
+|---|---|
+| a traced method (`Sixty::Instrumented`) | **~3.5µs** per call |
+| a query, recorded and rolled up | **~4µs** |
+| `pg` query, end to end against a real server | **+12–16µs** |
+| `mysql2` / `trilogy` query | **+13–14µs** |
+| `mongo` command | **+27–30µs** |
+
+The per-call number is asserted by `test/safety_test.rb`, which fails if it
+becomes milliseconds. The driver numbers are paired interleaved A/B samples
+against live servers — every measurement is (one query without the agent, one
+with, back to back), reported as the median difference, with an A/A control run
+to prove the harness has no bias of its own. Against a local query that takes
+300–500µs, that is a few percent; against a Rails request it was not
+measurable — 731 rps without the agent and 797 with it, i.e. inside the noise.
+
+Two of those numbers started five times worse, which is the reason the
+benchmark exists at all: naming an operation (`select:orders`) rebuilt regular
+expressions on every query, and estimating a result's size rendered five rows to
+strings. Both are now computed once per statement and read from libpq
+respectively.
+
+Nothing else happens on the request path. Spans go into an in-process rollup
+capped at 2,000 operations; a background thread posts a window every 15 seconds.
+A stack trace is captured once per query, ever, because a call site is a
+property of the statement rather than of the call. `EXPLAIN` runs on the flush
+thread on a connection of its own, never in front of a user, and never
+`EXPLAIN ANALYZE`.
+
+## What happens when the collector is down
+
+Nothing, to your application:
+
+- the request path never opens a socket to the collector
+- a failed flush is swallowed, logged at most once a minute, and backs off
+  exponentially to a five-minute ceiling
+- the undeliverable window is **dropped**, so the agent's memory is bounded by
+  your application's shape rather than by somebody else's uptime
+- shutdown waits at most two seconds for a final flush
+- an exception anywhere inside the agent is caught before it reaches your code —
+  the Rack middleware is written so that the only unguarded line in it is
+  `@app.call(env)`
+
+`test/safety_test.rb` asserts all of the above, including that a deliberately
+broken agent still returns 200.
+
+## What never leaves the process
+
+Raw SQL does not, and neither does a Mongo filter's contents. Statements are reduced to their literal-free shape before
+anything is recorded (`select * from users where email = ?`), which is also what
+keeps `where id = 1` and `where id = 99` from becoming two operations. Query
+plans come from `EXPLAIN (GENERIC_PLAN)`, which plans a parameterised statement
+without ever binding a parameter, so there is no step at which a value could
+enter one. Errors carry their class and message, never their arguments or
+backtrace.
+
+Every one of those claims is asserted against a live server rather than a
+fixture: `test/integration` runs the same checks against Postgres, MySQL and
+MongoDB, including "no value from a real query reaches the payload".
+
+## Configuration
+
+Every setting is an environment variable, and `DRIFT_*` still answers everywhere
+`SIXTY_*` does.
+
+| variable | default |
+|---|---|
+| `SIXTY_API_KEY` | — (without it the agent stays inactive and says so) |
+| `SIXTY_ENDPOINT` | `http://localhost:4319` |
+| `SIXTY_SERVICE` | the Rails application's name |
+| `SIXTY_ENV` | `Rails.env` |
+| `SIXTY_RELEASE` | a git SHA from the platform's own variables, if it sets one |
+| `SIXTY_FLUSH_MS` | `15000` |
+| `SIXTY_SAMPLE_RATE` | `0.05` (retained exemplar traces, not measurement) |
+| `SIXTY_SLOW_TRACE_MS` | `1000` |
+| `SIXTY_CAPTURE_PLANS` | on; `0` disables `EXPLAIN` |
+| `SIXTY_DEBUG` | `1` prints what the agent decided at boot |
+
+In code, through the railtie:
+
+```ruby
+config.sixty.sample_rate = 0.2
+config.sixty.ignore_paths = [%r{\A/internal/}]
+```
+
+## Compatibility
+
+Ruby 2.7+. Rails 6.1+ for the automatic install; Rack alone is enough for the
+manual one. Postgres via `pg`, MySQL via `mysql2` or `trilogy`, MongoDB via the
+`mongo` driver (which is what Mongoid uses). No runtime dependencies — this gem is loaded into other people's
+production processes, and every dependency it took would be a version conflict
+it could cause in an application that has nothing to do with observability.
+
+## Tests
+
+```bash
+bundle install
+rake test
+```
+
+Integration tests skip when a database is not running; `docker compose` in the
+repository root starts Postgres, and MySQL and MongoDB need one container each:
+
+```bash
+docker run -d -p 3307:3306 -e MYSQL_ROOT_PASSWORD=drift -e MYSQL_DATABASE=sixty_test \
+  -e MYSQL_USER=drift -e MYSQL_PASSWORD=drift mysql:8
+docker run -d -p 27018:27017 mongo:7
+```
+
+CI starts all three as services and fails if any test reports a skip, so
+"skipped" can never quietly become "never run".
+
+The sketch and SQL suites assert against fixtures generated by the JavaScript
+agent (`node test/fixtures/generate.mjs`) rather than against themselves. A
+codec bug that is consistent between a Ruby writer and a Ruby reader passes
+every roundtrip test there is; the collector decodes these bytes with the
+JavaScript implementation, so that is what the expectations come from.
+
+A running example lives in [`apps/demo-rails`](../../apps/demo-rails).
+
+## Releasing
+
+Bump `Sixty::VERSION` in `lib/sixty/version.rb` and merge to main. CI publishes
+the version to RubyGems if it is not already there, so a merge that does not
+change the version is a no-op rather than a failed build.
+
+Authentication is [trusted publishing][tp] rather than an API key: the gemspec
+sets `rubygems_mfa_required`, which is incompatible with an unattended `gem
+push` — deliberately, for a package that loads into other people's production
+processes. GitHub mints a short-lived OIDC token for this repository and this
+workflow instead, so there is no secret to leak or rotate. It is configured once
+per gem at `rubygems.org/gems/sixty/trusted_publishers`, and until it is, the
+publish job fails on the credentials step.
+
+[tp]: https://guides.rubygems.org/trusted-publishing/

data/lib/sixty/aggregator.rb

@@ -0,0 +1,257 @@
+# frozen_string_literal: true
+
+require_relative 'sketch'
+require_relative 'tracer'
+
+module Sixty
+  # In-process rollup.
+  #
+  # Instead of shipping one record per call, the agent keeps sketches keyed by
+  # operation and flushes them on an interval. A service doing 50k req/s across
+  # 400 distinct operations sends 400 rows per flush, not 50k per second.
+  #
+  # Cardinality is capped here as well as on the collector. Once MAX_OPERATIONS
+  # distinct operations are seen in a window, further new ones collapse into a
+  # single `__overflow__` bucket and a warning is emitted once. Silently
+  # dropping them would make the numbers quietly wrong; unbounded growth would
+  # take the host down. Overflow is visible and bounded.
+  #
+  # ── Why this is locked and the JavaScript one is not ──────────────────────
+  #
+  # Node's agent runs on one thread. A Rails app under Puma runs the request
+  # path on many, and every one of them records into this object — so a mutex is
+  # not defensive programming here, it is the difference between a correct
+  # counter and a torn one. It is held for the duration of a hash update and a
+  # sketch insertion, both of which are microseconds; the flush swaps the maps
+  # under the same lock and serializes outside it.
+  class Aggregator
+    MAX_OPERATIONS = 2000
+    MAX_EDGES = 5000
+
+    def initialize(alpha: 0.01, on_warn: ->(_msg) {})
+      @alpha = alpha
+      @on_warn = on_warn
+      @ops = {}
+      @edges = {}
+      @overflowed = false
+      @window_start = now_ms
+      @mutex = Mutex.new
+    end
+
+    def record(span)
+      @mutex.synchronize { record_locked(span) }
+    end
+
+    # Serialize and reset. Returns nil when there is nothing to send.
+    def drain
+      ops, edges, window_start = @mutex.synchronize do
+        return nil if @ops.empty?
+
+        taken = [@ops, @edges, @window_start]
+        @ops = {}
+        @edges = {}
+        @overflowed = false
+        @window_start = now_ms
+        taken
+      end
+
+      {
+        windowStart: window_start,
+        windowEnd: now_ms,
+        operations: ops.map { |key, op| serialize_operation(key, op) },
+        edges: edges.map { |key, edge| serialize_edge(key, edge) }
+      }
+    end
+
+    # Stable identity for an operation within one agent window.
+    #
+    # A db operation is identified by its statement, never by its label: the
+    # label is presentation and will keep improving, and folding it into the key
+    # would re-identify the operation — orphaning its history — every time
+    # somebody improves the naming.
+    def self.span_key(span)
+      if span.kind == Tracer::KIND_DB
+        "db\x01#{span.attrs[:normalized_sql] || span.name}"
+      else
+        "#{span.kind}\x01#{span.name}"
+      end
+    end
+
+    private
+
+    def record_locked(span)
+      key = self.class.span_key(span)
+      op = operation_for(key, span)
+
+      op[:count] += 1
+      op[:duration].add(span.duration)
+      op[:self].add(Tracer.self_time(span))
+
+      if span.error
+        op[:errors] += 1
+        signature = "#{span.error[:type]}: #{span.error[:message]}"[0, 200]
+        op[:error_samples][signature] = (op[:error_samples][signature] || 0) + 1
+      end
+
+      # rows: on a db span it is what the query returned; on anything else it is
+      # everything its descendants pulled back.
+      rows = span.kind == Tracer::KIND_DB ? span.attrs[:rows] : span.db_rows
+      if rows.is_a?(Numeric)
+        op[:rows].add(rows)
+        op[:rows_sum] += rows
+      end
+
+      unless span.kind == Tracer::KIND_DB
+        op[:db_calls_sum] += span.db_calls
+        op[:db_calls_max] = span.db_calls if span.db_calls > op[:db_calls_max]
+      end
+
+      if span.attrs[:bytes].is_a?(Numeric)
+        op[:bytes].add(span.attrs[:bytes])
+        op[:bytes_sum] += span.attrs[:bytes]
+      end
+
+      # Network round trips, reported only by a client that can make more than
+      # one per operation — today that is MongoDB, where a cursor fetches in
+      # batches. A sum rather than a sketch, like db_calls: what matters is
+      # round trips *per call*, and a distribution of a number that is 1 almost
+      # everywhere would cost a blob per operation to say so.
+      #
+      # Absent on every other client, which is what keeps this from claiming a
+      # Postgres query makes one round trip — it may well not, and this agent
+      # cannot see it.
+      op[:round_trips_sum] += span.attrs[:round_trips] if span.attrs[:round_trips].is_a?(Numeric)
+
+      # A plan that arrives after the operation was first recorded still belongs
+      # to it, so it is accepted whenever it turns up rather than only on
+      # creation. EXPLAIN runs after the span was emitted (see plans.rb), so the
+      # first execution of a query never carries one.
+      op[:plan] ||= plan_for(span)
+
+      record_edge(span.parent, span) if span.parent
+    end
+
+    def operation_for(key, span)
+      existing = @ops[key]
+      return existing if existing
+
+      if @ops.size >= MAX_OPERATIONS
+        unless @overflowed
+          @overflowed = true
+          @on_warn.call(
+            "sixty: operation cardinality cap (#{MAX_OPERATIONS}) reached; further " \
+            'operations are grouped as __overflow__. This usually means a route or ' \
+            'query is not being normalized.'
+          )
+        end
+        key = '__overflow__'
+        existing = @ops[key]
+        return existing if existing
+
+        return @ops[key] = blank_operation(kind: Tracer::KIND_FUNCTION, name: '__overflow__')
+      end
+
+      @ops[key] = blank_operation(
+        kind: span.kind,
+        name: span.name,
+        normalized_sql: span.attrs[:normalized_sql],
+        source_file: span.attrs[:file],
+        source_line: span.attrs[:line],
+        # Captured on the first sighting only, so later spans carry nothing.
+        frames: span.attrs[:frames],
+        plan: plan_for(span)
+      )
+    end
+
+    def blank_operation(kind:, name:, normalized_sql: nil, source_file: nil,
+                        source_line: nil, frames: nil, plan: nil)
+      {
+        kind: kind,
+        name: name,
+        normalized_sql: normalized_sql,
+        source_file: source_file,
+        source_line: source_line,
+        frames: frames,
+        plan: plan,
+        count: 0,
+        errors: 0,
+        duration: Sketch.new(@alpha),
+        self: Sketch.new(@alpha),
+        rows: Sketch.new(@alpha),
+        rows_sum: 0,
+        db_calls_sum: 0,
+        db_calls_max: 0,
+        bytes: Sketch.new(@alpha),
+        bytes_sum: 0,
+        round_trips_sum: 0,
+        error_samples: {}
+      }
+    end
+
+    def record_edge(parent, child)
+      key = "#{self.class.span_key(parent)}\x00#{self.class.span_key(child)}"
+      edge = @edges[key]
+      unless edge
+        return if @edges.size >= MAX_EDGES
+
+        edge = { count: 0, duration: Sketch.new(@alpha), rows_sum: 0 }
+        @edges[key] = edge
+      end
+      edge[:count] += 1
+      edge[:duration].add(child.duration)
+      edge[:rows_sum] += child.attrs[:rows] if child.attrs[:rows].is_a?(Numeric)
+    end
+
+    def plan_for(span)
+      return nil unless span.kind == Tracer::KIND_DB
+
+      key = span.attrs[:normalized_sql]
+      key ? Sixty::Plans.get(key) : nil
+    end
+
+    def serialize_operation(key, op)
+      {
+        key: key,
+        kind: op[:kind],
+        name: op[:name],
+        normalizedSql: op[:normalized_sql],
+        sourceFile: op[:source_file],
+        sourceLine: op[:source_line],
+        frames: op[:frames],
+        plan: op[:plan],
+        count: op[:count],
+        errors: op[:errors],
+        durationSketch: op[:duration].to_base64,
+        selfSketch: op[:self].to_base64,
+        # Null rather than an empty sketch: a counter has no distribution, and
+        # an empty one would be indistinguishable from "measured, all zero".
+        rowsSketch: op[:rows].count.positive? ? op[:rows].to_base64 : nil,
+        rowsSum: op[:rows_sum],
+        dbCallsSum: op[:db_calls_sum],
+        dbCallsMax: op[:db_calls_max],
+        bytesSketch: op[:bytes].count.positive? ? op[:bytes].to_base64 : nil,
+        bytesSum: op[:bytes_sum],
+        roundTripsSum: op[:round_trips_sum],
+        errorSamples: op[:error_samples]
+          .sort_by { |_message, count| -count }
+          .first(5)
+          .map { |message, count| { message: message, count: count } }
+      }
+    end
+
+    def serialize_edge(key, edge)
+      parent_key, child_key = key.split("\x00")
+      {
+        parentKey: parent_key,
+        childKey: child_key,
+        count: edge[:count],
+        durationSketch: edge[:duration].to_base64,
+        rowsSum: edge[:rows_sum]
+      }
+    end
+
+    def now_ms
+      (Time.now.to_f * 1000).round
+    end
+  end
+end