wide_events 0.1.0 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 60fd90a80072a4f375595529b63b5c739751de76d92d7e29ff1d8581ae6b5a07
-  data.tar.gz: 6656514b71043fa80493946446f19582d1a32fe63799f4f7a04e9f67b2cd595f
+  metadata.gz: 2c53fd3a4d33b817af708d0675bfa0b8047e2487a354d62cfb9e3c915b9c3472
+  data.tar.gz: 747298c60ca63d3a90db33abbf69d430e20a67d5037f79dcbf6d018ab5c86cb4
 SHA512:
-  metadata.gz: 43721569390e303660bde87aae8317fe320cd3c9722f33ec0a49e6bebc180258c5536e1596181653bc3302b4010fba56153b70c0129c3c5f458e89fbb055d6e2
-  data.tar.gz: 587ec94b148be767437f9d0d126c0ebd321ae5f91db31ef4236ff75cad1d39c5572f8e44d3791a410420b48c7304143260d51049e43607afb2d0573d220ab82b
+  metadata.gz: d54b65bd07d6294109efb49020299da91aeb040000c42841a175f1b7af3e7fb5a6989971fee3fae5a57f8c0d2c5692572cd08bd4fb978c5798cb7ce9ce98783f
+  data.tar.gz: e2a3a0d2d6400b1c1cd951c3f6c71c059ecc0cd2b67342beba43901a8ca30929b98ecafe5cab47994e0c3a5153b9773f186670a73ef7eb13894fff40bd9055f3

data/CHANGELOG.md

@@ -1,5 +1,26 @@
 # Changelog
 
+## v0.1.2 (2026-08-03)
+
+- README "In production" section and `docs/production-example.md`: an
+  anonymized composite event, instrumentation patterns, and standing
+  queries from the production app the gem was extracted from.
+
+## v0.1.1 (2026-08-03)
+
+- The install generator wires `WideEvent::TestHelper` into
+  `test/test_helper.rb` (or prints manual instructions when the file
+  isn't recognized).
+- Development boot logs a one-time notice when wide events are disabled,
+  pointing at the `:log` sink quickstart.
+- `assert_registered_wide_event_attributes` resets the violation tally
+  after each check, so it can run as a global `teardown` and works with
+  parallel test workers.
+- README rewritten around first-run experience: 60-second `:log` sink
+  quickstart, OpenTelemetry setup guide, enablement modes, end-to-end
+  registry workflow, log sink operations, request-vs-job event semantics.
+- New `docs/clickhouse-hyperdx.md`: local quickstart and query cookbook.
+
 ## v0.1.0 (2026-08-03)
 
 First release, extracted from a production Rails app.

data/README.md

@@ -1,5 +1,8 @@
 # Wide Events
 
+[![Gem Version](https://img.shields.io/gem/v/wide_events)](https://rubygems.org/gems/wide_events)
+[![CI](https://github.com/adammiribyan/wide_events/actions/workflows/ci.yml/badge.svg)](https://github.com/adammiribyan/wide_events/actions/workflows/ci.yml)
+
 One wide telemetry event per Rails request or job execution, in a database you own.
 
 Wide Events collects everything your app knows about each unit of work (route, user, account, build SHA, query counts, cache hits, feature flags, phase timings, errors) into one flat, high-cardinality event on the OpenTelemetry root span you already export. Every request becomes one row. Every question becomes one query against storage you run: ClickHouse with HyperDX on top is a proven pairing (both open source), any OTLP backend works, and a log-line sink emits one JSON event per request if you'd rather not run tracing at all.
@@ -11,18 +14,70 @@ That format matters more now that agents build with you. Telemetry stops being s
 ```ruby
 # Gemfile
 gem "wide_events"
+gem "useragent"  # optional: parses user_agent.browser/os/platform from the raw UA string
 ```
 
 ```bash
-bin/rails generate wide_events:install   # initializer + attribute registry + AGENTS.md section
+bin/rails generate wide_events:install   # initializer + attribute registry + AGENTS.md section + test helper wiring
 bin/rails generate wide_events:skills    # agent skills into .claude/skills/
 ```
 
-Wide events turn on wherever `OTEL_EXPORTER_OTLP_ENDPOINT` is set. Dev and test pay nothing.
+## See an event in 60 seconds
 
-## What you get per event
+Wide events are off unless `OTEL_EXPORTER_OTLP_ENDPOINT` is set (the development log tells you so at boot). To see one immediately, no tracing required, enable the log sink in `config/initializers/wide_events.rb`:
+
+```ruby
+WideEvent.configure do |config|
+  config.enabled = true
+  config.sink = :log
+end
+```
+
+Hit any route, then:
+
+```bash
+grep '"main":true' log/development.log
+```
+
+One JSON object per request: route, status, timings, query counts, everything.
+
+## Production setup with OpenTelemetry
+
+The default sink writes the event onto the current OTel root span, which is the span OTel's Rack instrumentation opens at the top of the middleware stack. That means production needs an OpenTelemetry SDK configured; the gem detects it and wires itself after your initializers run. Minimal setup:
+
+```ruby
+# Gemfile
+gem "opentelemetry-sdk"
+gem "opentelemetry-exporter-otlp"
+gem "opentelemetry-instrumentation-rails"
+gem "opentelemetry-instrumentation-rack"
+gem "opentelemetry-instrumentation-pg"        # enables stats.postgres_query_*
+gem "opentelemetry-instrumentation-net_http"  # enables stats.http_call_*
+```
+
+```ruby
+# config/initializers/opentelemetry.rb
+return if ENV["OTEL_EXPORTER_OTLP_ENDPOINT"].to_s.empty?
+
+require "opentelemetry/sdk"
+require "opentelemetry/exporter/otlp"
+require "opentelemetry/instrumentation/rails"
+require "opentelemetry/instrumentation/rack"
+require "opentelemetry/instrumentation/pg"
+require "opentelemetry/instrumentation/net/http"
+
+OpenTelemetry::SDK.configure(&:use_all)
+```
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318   # your collector or backend
+OTEL_SERVICE_NAME=myapp
+OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
+```
 
-The gem records the generic attributes on every request and job:
+With the endpoint set, wide events enable themselves: the railtie inserts the middleware directly below `ActionDispatch::Executor` (the executor clears the per-request store, so placement matters), instruments `ActiveJob::Base`, and registers the span counter and notification subscribers via `WideEvent.install!` after your initializers run. Without Rails, call `WideEvent.install!` yourself after configuring the SDK.
+
+## What you get per event
 
 - `http.request.id`, `http.response.status_code`, `http.route.controller`, request body size, parsed `user_agent.*`
 - `db.duration_ms` and `view.duration_ms` (Rails' own measurements)
@@ -31,21 +86,32 @@ The gem records the generic attributes on every request and job:
 - `job.class`, `job.queue`, `job.queue_latency_ms`, `job.executions`, `job.scheduled` (Solid Queue recurring detection built in, detector pluggable)
 - `error`, `exception.type`, `exception.message`, `uptime_sec`, `main: true`
 
-Your code adds the parts only it knows:
+A request and the jobs it enqueues are separate units of work: the request's wide event records the enqueue inside its timings, and each job execution emits its own event carrying `job.queue_latency_ms` (time spent waiting in the queue), so queue pressure is visible per job, not smeared into request latency.
+
+## Adding your own attributes
 
 ```ruby
 WideEvent.set("report.id" => report.id, "report.format" => "pdf")
 WideEvent.phase("pdf_render") { render_pdf }         # -> pdf_render.duration_ms
-WideEvent.error!(slug: "err-export-source-missing", exception: e, expected: true)
 ```
 
 Every call is a safe no-op outside a unit of work and never raises into app code. A telemetry bug cannot fail a request or a job.
 
-"Something is slow" turns into "the reports route is slow for account 4218 on build f3a91c, and it's running 742 queries": one query in HyperDX for you, one SQL call for your agent.
+Errors come in two shapes, and the difference is the point:
+
+```ruby
+# Handled: you rescued it, you name it
+rescue Vendor::ApiError => e
+  WideEvent.error!(slug: "err-vendor-sync-failed", exception: e, expected: true)
+
+# Unhandled: any exception that escapes gets error: true with NO slug, automatically
+```
+
+That makes `error = true AND exception.slug IS NULL` a standing query: every row is a failure nobody wrote a rescue for, which is a permanent, prioritized to-do list of rescues worth instrumenting.
 
 ## The registry is a schema, not a wiki page
 
-Every attribute is declared in `config/wide_event/registry.yml`:
+Every attribute is declared in `config/wide_event/registry.yml` (the gem's own attributes are pre-registered; globs like `feature_flag.*` cover dynamic families):
 
 ```yaml
 report.format:
@@ -55,15 +121,30 @@ report.format:
   notes: pdf or csv
 ```
 
-The gem registers its own attributes by default; globs (`feature_flag.*`) cover dynamic families. With `config.strict = true` in the test environment, the suite records every undeclared attribute it sees, and `assert_registered_wide_event_attributes` fails on them. `rake wide_events:registry:check` validates the file; `rake wide_events:registry:docs` generates the human-readable registry doc from it, so documentation can't drift from reality.
+The workflow, end to end:
+
+1. Set the attribute in code, declare it in `registry.yml`, same change.
+2. Assert it in a test with `assert_wide_event`.
+3. Strict mode (`config.strict = Rails.env.test?`, the generated default) records any undeclared attribute the suite sees; `assert_registered_wide_event_attributes` fails on them.
+4. CI runs `bin/rails wide_events:registry:check` to validate the file.
+5. `bin/rails wide_events:registry:docs` generates `docs/wide-events-registry.md` from it, so documentation can't drift from reality.
+
+```yaml
+# .github/workflows/ci.yml (host app)
+- run: bin/rails wide_events:registry:check
+- run: bin/rails test
+```
 
 ## Testing
 
+The install generator wires this into `test/test_helper.rb` (or tells you how):
+
 ```ruby
-# test_helper.rb
 require "wide_event/test_helper"
+
 class ActiveSupport::TestCase
   include WideEvent::TestHelper
+  teardown { assert_registered_wide_event_attributes }
 end
 ```
 
@@ -80,14 +161,26 @@ test "job emits one wide event" do
 end
 ```
 
+The teardown pattern makes the registry check work with parallel test workers: violation tracking is per process, and checking (then resetting) after every test means the test that set an undeclared attribute is the one that fails, in whichever worker it ran.
+
+## When events are emitted
+
+| Environment | Typical setup | Result |
+|---|---|---|
+| Production / staging | `OTEL_EXPORTER_OTLP_ENDPOINT` set | Enabled automatically, `:otel` sink, events on root spans |
+| Development | `config.enabled = true`, `config.sink = :log` | One JSON line per request in the log |
+| Test | Enabled via the test helper's capture, or strict mode for registry tracking | `capture_wide_events` collects them |
+
+`WideEvent.install!` runs once at boot, only if enabled at that point. Flipping `config.enabled` at runtime gates the middleware and job hook (they check per request), but the span counter and subscribers only register at boot, so treat enablement as a boot-time decision.
+
 ## Built for the agent loop
 
 `bin/rails generate wide_events:skills` installs two agent skills:
 
 - **instrumenting-wide-events**: the write path. Naming conventions, when to use `set` vs `phase` vs `error!`, the registry workflow, PII rules, test assertions.
-- **debugging-with-wide-events**: the read path. Symptom-to-query workflow against ClickHouse/HyperDX or JSON logs, plus the standing queries worth running: `error = true AND exception.slug IS NULL` is a permanent, queryable to-do list of rescues nobody instrumented.
+- **debugging-with-wide-events**: the read path. Symptom-to-query workflow against ClickHouse/HyperDX or JSON logs, plus the standing queries worth running.
 
-The install generator also appends a wide-events section to `AGENTS.md`, so every future session knows the instrumentation exists and how to extend it.
+Skills land in `.claude/skills/`, which Claude Code and Claude-compatible agents read. Other agents (Cursor, etc.) read the pointer the install generator appends to `AGENTS.md`; the SKILL.md files are plain markdown, so copy or symlink them wherever your tooling looks (`.cursor/rules/`, docs, a system prompt).
 
 ## Configuration
 
@@ -95,6 +188,7 @@ The install generator also appends a wide-events section to `AGENTS.md`, so ever
 WideEvent.configure do |config|
   config.enabled = ENV["OTEL_EXPORTER_OTLP_ENDPOINT"].present?  # default
   config.sink = :otel               # :otel, :log, or any object responding to flush(attrs)
+  config.logger = nil               # :log sink target; defaults to Rails.logger
   config.strict = Rails.env.test?   # track attributes against the registry
   config.max_cache_attrs = 10
   config.span_scopes = {            # OTel instrumentation scope -> stats.* name
@@ -106,14 +200,30 @@ WideEvent.configure do |config|
 end
 ```
 
-The railtie inserts the middleware directly below `ActionDispatch::Executor` (the executor clears the per-request store, so placement matters), instruments `ActiveJob::Base`, and wires the span counter and notification subscribers after your initializers run. Without Rails, call `WideEvent.install!` yourself after configuring the OpenTelemetry SDK.
+Log sink notes: events go to `config.logger` (or `Rails.logger`) at info level as single-line JSON. Point it at a dedicated logger (`config.logger = Logger.new("log/wide_events.log")`) to keep them out of your main log, and leave the sink off in the test environment unless you want one JSON line per test request; `capture_wide_events` is the intended test-side tap.
+
+## Querying: ClickHouse + HyperDX
+
+[docs/clickhouse-hyperdx.md](docs/clickhouse-hyperdx.md) has a local quickstart (one container) and a query cookbook. The flavor:
+
+```sql
+SELECT SpanAttributes['http.route.controller'] AS controller,
+       count() AS requests,
+       round(quantile(0.5)(Duration/1e6)) AS p50_ms,
+       round(avg(toFloat64OrZero(SpanAttributes['stats.postgres_query_count']))) AS avg_queries
+FROM otel_traces
+WHERE SpanAttributes['main'] = 'true'
+GROUP BY controller ORDER BY p50_ms DESC LIMIT 15
+```
+
+## In production
+
+Wide Events was extracted from a multi-tenant Rails app subject to healthcare privacy rules, where it runs in staging and production today: one row per request and job execution, exported over OTLP into self-hosted ClickStack. A request that runs a hybrid search and drafts a reply with an LLM still lands as one event, carrying the account, the feature flags evaluated, search quality (semantic hits kept, top cosine similarity), the model, tokens, latency, and cost, the Postgres query count, and an error slug for anything rescued along the way. [docs/production-example.md](docs/production-example.md) shows a full anonymized event, the instrumentation patterns behind it, and two standing queries taken from that deployment.
 
 ## Conventions
 
 Flat keys, dot namespaces, snake_case leaves. Durations end in `_duration_ms`, counts in `_count`, booleans read as assertions, timestamps serialize to RFC 3339. Opaque ids are fine; names, emails, and request params are not, and anything that could quote user input is flagged `pii: review` in the registry.
 
-Unhandled exceptions get `error: true` with no slug, deliberately: the missing slug marks the rescue you haven't instrumented yet.
-
 ## Development
 
 ```bash

data/lib/generators/wide_events/install/install_generator.rb

@@ -21,6 +21,23 @@ module WideEvents
           create_file "AGENTS.md", section
         end
       end
+
+      def wire_test_helper
+        path = File.join(destination_root, "test/test_helper.rb")
+        content = File.exist?(path) ? File.read(path) : ""
+        if content.match?(%r{require "rails/test_help"\n}) && content.match?(/class TestCase\n/)
+          inject_into_file "test/test_helper.rb", "require \"wide_event/test_helper\"\n",
+            after: %r{require "rails/test_help"\n}
+          inject_into_file "test/test_helper.rb", "    include WideEvent::TestHelper\n",
+            after: /class TestCase\n/
+        else
+          say_status :skip, "test/test_helper.rb (not found or unrecognized) - wire the test helper manually:", :yellow
+          say <<~MSG
+                require "wide_event/test_helper"   # after rails/test_help
+                include WideEvent::TestHelper      # inside ActiveSupport::TestCase
+          MSG
+        end
+      end
     end
   end
 end

data/lib/wide_event/railtie.rb

@@ -22,7 +22,15 @@ module WideEvent
     # After the app's own initializers, so the host's OpenTelemetry::SDK
     # configuration (and any WideEvent.configure block) has already run.
     config.after_initialize do
-      WideEvent.install! if WideEvent.enabled?
+      if WideEvent.enabled?
+        WideEvent.install!
+      elsif Rails.env.development?
+        Rails.logger&.info(
+          "wide_events: installed but disabled (OTEL_EXPORTER_OTLP_ENDPOINT is not set). " \
+          "To see events locally, set `config.enabled = true` and `config.sink = :log` " \
+          "in config/initializers/wide_events.rb."
+        )
+      end
     end
 
     rake_tasks do

data/lib/wide_event/registry.rb

@@ -49,6 +49,11 @@ module WideEvent
       @violations.to_a
     end
 
+    def clear_violations!
+      @mutex.synchronize { @violations.clear }
+      nil
+    end
+
     def validate
       entries.filter_map do |key, entry|
         unless entry.is_a?(Hash) && entry["type"]

data/lib/wide_event/test_helper.rb

@@ -43,11 +43,18 @@ module WideEvent
     end
 
     # Fails if strict mode saw any attribute that isn't declared in the
-    # registry. Call from a dedicated test (or a teardown hook) with
-    # config.strict = true to make the registry an enforced schema.
+    # registry since the last check, then resets the tally. The reset is what
+    # makes a global teardown hook practical: the test that set the
+    # undeclared attribute fails; later tests stay green.
+    #
+    #   class ActiveSupport::TestCase
+    #     include WideEvent::TestHelper
+    #     teardown { assert_registered_wide_event_attributes }
+    #   end
     def assert_registered_wide_event_attributes
       registry = WideEvent.config.registry
       violations = registry ? registry.violations : []
+      registry&.clear_violations!
       assert violations.empty?,
         "unregistered wide-event attributes #{violations.sort.inspect} - declare them in #{WideEvent.config.registry_path}"
     end

data/lib/wide_event/version.rb

@@ -1,3 +1,3 @@
 module WideEvent
-  VERSION = "0.1.0"
+  VERSION = "0.1.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: wide_events
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.2
 platform: ruby
 authors:
 - Adam Miribyan
@@ -81,6 +81,8 @@ metadata:
   homepage_uri: https://github.com/adammiribyan/wide_events
   source_code_uri: https://github.com/adammiribyan/wide_events
   changelog_uri: https://github.com/adammiribyan/wide_events/blob/main/CHANGELOG.md
+  bug_tracker_uri: https://github.com/adammiribyan/wide_events/issues
+  documentation_uri: https://rubydoc.info/gems/wide_events
   rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths: