mbuzz 0.8.2 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 58bb33f56016e563027651d3fe6e0d5a8bd55a5b51e9920693a74b380380612a
-  data.tar.gz: 169f9818380dad8e239311ed1bd9c3501eb96b24d3f650a90679af1ad91630d0
+  metadata.gz: 81970794a962b276527d5aa05efb59703e1e3507598218c66f3eb3c238a96b6a
+  data.tar.gz: 391e3993a7aa60f3ad5fe50dfeff1394c9f3282538d3074de12f7a7a284d8e15
 SHA512:
-  metadata.gz: 684d91a340148e2a0762a476c401b9c8dc21f0bbff61882c8740394cc00ff72ea7ed65264f0d37880bac055e8add2b62548d4b6b474b427fe3fb8d03544d126e
-  data.tar.gz: 446e948eb88580966f4aaa9e5e0757510e7569f84fecf6bd1967c8ac364745b445e71484584807ae525f298b2b6e939bdeff6357982b47d1bf3cea3c79145808
+  metadata.gz: 2187c1959502e1dcc183e6f3fd2d77b46b90e87427bb79078c04cac35715891ec89cf274514cf8e8f327de2d8176e0d381875c961b9f61b37fee18f313c8ff9e
+  data.tar.gz: eb34d5989f34c7e83d5315c57000dff733af12a26494255817a414002d0448a1d8ee8f71143a94348f1b5118df743acf97cdeffd2347584e0cbe561b73b396e3

data/CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.9.0] - 2026-09-02
+
+### Added
+
+- **A dropped event or conversion now says so.** `Mbuzz.event` and `Mbuzz.conversion` return `false` when there is no `visitor_id` and no `user_id` — previously with no request, no log and no signal of any kind. That silence is what made the caching bug above invisible on a live account for a full day. They now warn at `Rails.logger.warn` (or stderr outside Rails), naming the dropped call and pointing at the session endpoint. Deliberately **not** gated behind `debug`: the people who hit this are precisely the ones not running in debug.
+- **Attribution now survives a full-page cache.** A cached page is answered without entering the Rack stack, so the tracking middleware never ran, no visitor cookie was set, and every later event was dropped for having no one to attribute it to — silently, while the page rendered perfectly. `Mbuzz::Middleware::SessionEndpoint` answers `POST /_mbuzz/session`, a path caches don't store, and the server sets the cookie on that response. Mounted automatically in Rails; add it ahead of `Tracking` in Rack/Sinatra apps. See "Full-page caching" in the README for the one-time page snippet.
+
+### Notes
+
+- The visitor id is still never created or read in JavaScript. It stays `HttpOnly` and server-set, preserving its full two-year life — a cookie written by `document.cookie` is capped at 7 days under Safari's ITP, and 24 hours after an ad click.
+- Inline the page snippet rather than serving it as a file: asset optimisers delay external scripts until first interaction, which would miss any visitor who lands and converts without clicking first.
+
 ## [0.8.2] - 2026-03-15
 
 ### Changed

data/README.md

@@ -188,10 +188,47 @@ require 'mbuzz'
 
 Mbuzz.init(api_key: ENV['MBUZZ_API_KEY'])
 
+use Mbuzz::Middleware::SessionEndpoint   # see "Full-page caching" below
 use Mbuzz::Middleware::Tracking
 run MyApp
 ```
 
+## Full-page caching
+
+If pages are served from a full-page cache (Cloudflare, Varnish, nginx, Rack::Cache, a CDN),
+the cache answers the request **without entering the Rack stack**, so the tracking middleware
+never runs, no visitor cookie is set, and every later event is dropped for having no one to
+attribute it to. The page renders perfectly and nothing is logged — the failure is silent.
+
+`Mbuzz::Middleware::SessionEndpoint` fixes this. It answers `POST /_mbuzz/session`, a path
+caches don't store, and the **server** sets the cookie on that response. Rails mounts it for
+you; Rack and Sinatra apps add it ahead of `Tracking` as shown above.
+
+Then call it once per page, from your layout:
+
+```html
+<script>
+  fetch('/_mbuzz/session', {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ url: location.href, referrer: document.referrer || '' }),
+    credentials: 'same-origin',
+    keepalive: true
+  }).catch(function () {});
+</script>
+```
+
+Two things to keep as they are:
+
+- **Inline the script, don't enqueue a file.** Asset optimisers (WP Rocket, LiteSpeed, and the
+  Rails equivalents) delay external scripts until the visitor first interacts. A visitor who
+  lands and converts without clicking anything first would never be established.
+- **`credentials: 'same-origin'` is required**, or the cookie never comes back.
+
+The visitor id is never created or read in JavaScript. It stays `HttpOnly` and server-set, which
+is what preserves its full two-year life — a cookie written by `document.cookie` is capped at
+7 days under Safari's ITP, and 24 hours after an ad click.
+
 ## Configuration Options
 
 ```ruby

data/lib/mbuzz/client/conversion_request.rb

@@ -19,7 +19,7 @@ module Mbuzz
       end
 
       def call
-        return false unless input_valid?
+        return warn_dropped unless input_valid?
         return proxy_result if proxy_accepted?
         return false unless conversion_id
 
@@ -28,6 +28,11 @@ module Mbuzz
 
       private
 
+      def warn_dropped
+        DroppedCall.warn_missing_identity("conversion", @conversion_type) unless has_identifier?
+        false
+      end
+
       def input_valid?
         has_identifier? && present?(@conversion_type) && hash?(@properties)
       end

data/lib/mbuzz/client/track_request.rb

@@ -14,7 +14,7 @@ module Mbuzz
       end
 
       def call
-        return false unless input_valid?
+        return warn_dropped unless input_valid?
         return proxy_result if proxy_accepted?
         return false unless event
 
@@ -24,6 +24,13 @@ module Mbuzz
 
       private
 
+      # A drop with no request and no log is how this failure stayed invisible
+      # on a live account for a full day. Say it out loud.
+      def warn_dropped
+        DroppedCall.warn_missing_identity("event", @event_type) unless @user_id || @visitor_id
+        false
+      end
+
       def input_valid?
         present?(@event_type) && hash?(@properties) && (@user_id || @visitor_id)
       end

data/lib/mbuzz/dropped_call.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Mbuzz
+  # Says out loud when a call is dropped for having nothing to attribute it to.
+  #
+  # Every SDK guarded its send with a bare `return false unless valid?`: no
+  # request, no log, nothing. Behind a full-page cache that is the whole failure
+  # — the cookie is never minted, so every later event fails this guard and
+  # vanishes. It cost a full day on a live account precisely because the silence
+  # was total from both sides.
+  #
+  # Deliberately NOT behind config.debug. The customers who hit this are exactly
+  # the ones not running in debug, so a debug-gated warning would be silent for
+  # everyone who needs it.
+  module DroppedCall
+    # Built lazily: SESSION_ENDPOINT_PATH is defined in mbuzz.rb *after* this
+    # file is required, so interpolating it at load time would not resolve.
+    def self.missing_identity
+      "no visitor_id and no user_id. If your pages are served from a full-page " \
+        "cache, mount Mbuzz::Middleware::SessionEndpoint and call POST " \
+        "#{SESSION_ENDPOINT_PATH} from the page — see the README's " \
+        "\"Full-page caching\" section."
+    end
+
+    def self.warn_missing_identity(kind, name)
+      emit("dropped #{kind} #{name.inspect}: #{missing_identity}")
+    end
+
+    def self.warn_invalid(kind, name, reason)
+      emit("dropped #{kind} #{name.inspect}: #{reason}.")
+    end
+
+    def self.emit(message)
+      text = "[mbuzz] #{message}"
+      return Rails.logger.warn(text) if defined?(Rails) && Rails.logger
+
+      Kernel.warn(text)
+    end
+    private_class_method :emit
+  end
+end

data/lib/mbuzz/middleware/session_endpoint.rb

@@ -0,0 +1,135 @@
+# frozen_string_literal: true
+
+require "rack"
+require "json"
+require "digest"
+require "securerandom"
+
+module Mbuzz
+  module Middleware
+    # Establishes the visitor from a page a cache served.
+    #
+    # A cached page never enters the Rack stack, so Tracking never runs and no
+    # visitor cookie is set — every later event is then rejected for having no
+    # one to attribute it to, silently, while the page renders perfectly.
+    #
+    # This endpoint is the one request on such a page that always reaches the
+    # app. A small script on the page POSTs here; the SERVER mints the cookie on
+    # the response. The id is never created or read in JS, so it stays HttpOnly
+    # and keeps its full two-year life — a cookie written by document.cookie is
+    # capped at 7 days under Safari's ITP, and 24 hours after an ad click.
+    #
+    # Mount ahead of Tracking:
+    #
+    #   config.middleware.insert_before Mbuzz::Middleware::Tracking,
+    #                                   Mbuzz::Middleware::SessionEndpoint
+    class SessionEndpoint
+      # Nothing to return: the response exists for its Set-Cookie header.
+      NO_CONTENT_STATUS = 204
+
+      def initialize(app)
+        @app = app
+      end
+
+      def call(env)
+        return @app.call(env) unless session_request?(env)
+
+        mint(Rack::Request.new(env))
+      end
+
+      private
+
+      # POST only: a GET is cacheable by an intermediary, which would reintroduce
+      # the very bug this endpoint exists to fix.
+      def session_request?(env)
+        env["REQUEST_METHOD"] == "POST" &&
+          env["PATH_INFO"].to_s == SESSION_ENDPOINT_PATH
+      end
+
+      def mint(request)
+        context = build_context(request)
+        create_session_async(context)
+
+        [NO_CONTENT_STATUS, response_headers(context, request), []]
+      end
+
+      def build_context(request)
+        payload    = parse_body(request)
+        ip         = extract_ip(request)
+        user_agent = request.user_agent.to_s
+
+        {
+          visitor_id: resolve_visitor_id(request),
+          session_id: SecureRandom.uuid,
+          # The page's URL, not ours — a script on the page called us, so our own
+          # path would attribute every session to this endpoint.
+          url: payload["url"],
+          referrer: payload["referrer"],
+          ip: ip,
+          user_agent: user_agent,
+          device_fingerprint: Digest::SHA256.hexdigest("#{ip}|#{user_agent}")[0, 32]
+        }.freeze
+      end
+
+      def parse_body(request)
+        body = request.body&.read.to_s
+        return {} if body.empty?
+
+        parsed = JSON.parse(body)
+        parsed.is_a?(Hash) ? parsed : {}
+      rescue JSON::ParserError
+        {}
+      end
+
+      def resolve_visitor_id(request)
+        request.cookies[VISITOR_COOKIE_NAME] || Visitor::Identifier.generate
+      end
+
+      def create_session_async(context)
+        Thread.new do
+          Client.session(
+            visitor_id: context[:visitor_id],
+            session_id: context[:session_id],
+            url: context[:url],
+            referrer: context[:referrer],
+            device_fingerprint: context[:device_fingerprint],
+            user_agent: context[:user_agent]
+          )
+        rescue StandardError => e
+          log_error("Session creation failed: #{e.message}") if Mbuzz.config.debug
+        end
+      end
+
+      def response_headers(context, request)
+        headers = { "cache-control" => "no-store, no-cache, must-revalidate, private" }
+        Rack::Utils.set_cookie_header!(headers, VISITOR_COOKIE_NAME, cookie_options(context, request))
+        headers
+      end
+
+      def cookie_options(context, request)
+        options = {
+          value: context[:visitor_id],
+          max_age: VISITOR_COOKIE_MAX_AGE,
+          path: VISITOR_COOKIE_PATH,
+          httponly: true,
+          same_site: VISITOR_COOKIE_SAME_SITE
+        }
+        options[:secure] = true if request.ssl?
+        options
+      end
+
+      def extract_ip(request)
+        forwarded = request.env["HTTP_X_FORWARDED_FOR"]
+        return forwarded.split(",").first.strip if forwarded
+
+        request.ip
+      end
+
+      def log_error(message)
+        return unless defined?(Rails) && Rails.logger
+
+        Rails.logger.error("[Mbuzz] #{message}")
+      end
+    end
+  end
+end

data/lib/mbuzz/railtie.rb

@@ -3,6 +3,9 @@
 module Mbuzz
   class Railtie < Rails::Railtie
     initializer "mbuzz.configure_rails" do |app|
+      # The endpoint must sit ahead of Tracking: it answers its own path and
+      # never falls through, so Tracking should not also process it.
+      app.middleware.use Mbuzz::Middleware::SessionEndpoint
       app.middleware.use Mbuzz::Middleware::Tracking
 
       ActiveSupport.on_load(:action_controller) do

data/lib/mbuzz/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Mbuzz
-  VERSION = "0.8.2"
+  VERSION = "0.9.0"
 end

data/lib/mbuzz.rb

@@ -2,11 +2,13 @@
 
 require_relative "mbuzz/version"
 require_relative "mbuzz/configuration"
+require_relative "mbuzz/dropped_call"
 require_relative "mbuzz/visitor/identifier"
 require_relative "mbuzz/request_context"
 require_relative "mbuzz/api"
 require_relative "mbuzz/client"
 require_relative "mbuzz/middleware/tracking"
+require_relative "mbuzz/middleware/session_endpoint"
 require_relative "mbuzz/controller_helpers"
 
 # CurrentAttributes for automatic background job context propagation (Rails only)
@@ -27,6 +29,10 @@ module Mbuzz
   VISITOR_COOKIE_PATH = "/"
   VISITOR_COOKIE_SAME_SITE = "Lax"
 
+  # The one request a cached page always sends to the app. Kept off /api and
+  # /assets so host-app routing and cache rules don't shadow it.
+  SESSION_ENDPOINT_PATH = "/_mbuzz/session"
+
   SESSION_USER_ID_KEY = "user_id"
   ENV_USER_ID_KEY = "mbuzz.user_id"
   ENV_VISITOR_ID_KEY = "mbuzz.visitor_id"
@@ -110,8 +116,13 @@ module Mbuzz
     resolved_visitor_id = visitor_id || self.visitor_id
     resolved_user_id = user_id
 
-    # Must have at least one identifier
-    return false unless resolved_visitor_id || resolved_user_id
+    # Must have at least one identifier. Warn rather than drop in silence: with
+    # no visitor and no user there is nothing to attribute this to, and behind a
+    # full-page cache that is the normal case, not an edge one.
+    unless resolved_visitor_id || resolved_user_id
+      DroppedCall.warn_missing_identity("event", event_type)
+      return false
+    end
 
     Client.track(
       visitor_id: resolved_visitor_id,
@@ -161,8 +172,12 @@ module Mbuzz
     resolved_visitor_id = visitor_id || self.visitor_id
     resolved_user_id = user_id || self.user_id
 
-    # Must have at least one identifier (visitor_id or user_id)
-    return false unless resolved_visitor_id || resolved_user_id
+    # Must have at least one identifier (visitor_id or user_id). A conversion
+    # dropped here is lost revenue attribution, so it is never silent.
+    unless resolved_visitor_id || resolved_user_id
+      DroppedCall.warn_missing_identity("conversion", conversion_type)
+      return false
+    end
 
     Client.conversion(
       visitor_id: resolved_visitor_id,

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mbuzz
 version: !ruby/object:Gem::Version
-  version: 0.8.2
+  version: 0.9.0
 platform: ruby
 authors:
 - mbuzz team
@@ -32,13 +32,10 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".DS_Store"
 - CHANGELOG.md
-- CHECK_BUG.md
 - LICENSE.txt
 - README.md
 - Rakefile
-- lib/.DS_Store
 - lib/mbuzz.rb
 - lib/mbuzz/api.rb
 - lib/mbuzz/client.rb
@@ -49,18 +46,13 @@ files:
 - lib/mbuzz/configuration.rb
 - lib/mbuzz/controller_helpers.rb
 - lib/mbuzz/current.rb
+- lib/mbuzz/dropped_call.rb
+- lib/mbuzz/middleware/session_endpoint.rb
 - lib/mbuzz/middleware/tracking.rb
 - lib/mbuzz/railtie.rb
 - lib/mbuzz/request_context.rb
 - lib/mbuzz/version.rb
 - lib/mbuzz/visitor/identifier.rb
-- lib/specs/old/SPECIFICATION.md
-- lib/specs/old/conversions.md
-- lib/specs/old/event_ids_response.md
-- lib/specs/old/v0.2.0_breaking_changes.md
-- lib/specs/old/v2.0.0_sessions_upgrade.md
-- lib/specs/v0.5.0_four_call_model.md
-- lib/specs/v0.7.0_deterministic_sessions.md
 - sig/mbuzz.rbs
 homepage: https://mbuzz.co
 licenses:

data/CHECK_BUG.md

@@ -1,168 +0,0 @@
-# Thread-Safety Bug Fix Verification
-
-## Bug Summary
-
-**Issue**: Middleware used instance variables (`@session_id`, `@visitor_id`, `@request`) shared across concurrent requests in multi-threaded servers (Puma). This caused race conditions where session/visitor IDs leaked between requests.
-
-**Impact**: Pet Resorts Australia had **178,428 sessions** for only **172,000 visitors**. Some visitors had 1,500+ sessions because cookies were set with wrong session_ids under concurrent load.
-
-**Root Cause**: Rack middleware is instantiated once and shared across all requests. Instance variables are not thread-safe.
-
-## Fix Details
-
-| Field | Value |
-|-------|-------|
-| **Fixed in version** | 0.6.3 |
-| **Commit** | `bdf4c64` |
-| **Fix deployed** | 2025-12-22 ~21:30 UTC (2025-12-23 ~08:30 AEDT) |
-| **Gem published** | 2025-12-23 |
-
-## Verification Checklist
-
-### After 24-48 hours (by 2025-12-25):
-
-Run these queries in production Rails console:
-
-```ruby
-# 1. Check session creation rate AFTER fix
-# Should see dramatically fewer sessions per hour
-cutoff = Time.parse("2025-12-23 08:30:00 UTC")  # Adjust to actual deploy time
-
-puts "Sessions BEFORE fix (last 24h before deploy):"
-before_sessions = Session.where(created_at: (cutoff - 24.hours)..cutoff).count
-puts "  Count: #{before_sessions}"
-
-puts "\nSessions AFTER fix (24h after deploy):"
-after_sessions = Session.where(created_at: cutoff..(cutoff + 24.hours)).count
-puts "  Count: #{after_sessions}"
-
-puts "\nReduction: #{((before_sessions - after_sessions).to_f / before_sessions * 100).round(1)}%"
-```
-
-```ruby
-# 2. Check sessions per visitor ratio
-# Should be close to 1.0-1.5 for new visitors (was 1.05 overall but outliers had 1500+)
-cutoff = Time.parse("2025-12-23 08:30:00 UTC")
-
-new_visitors = Visitor.where("created_at > ?", cutoff)
-new_visitor_ids = new_visitors.pluck(:id)
-
-sessions_for_new = Session.where(visitor_id: new_visitor_ids).count
-puts "New visitors since fix: #{new_visitors.count}"
-puts "Sessions for new visitors: #{sessions_for_new}"
-puts "Ratio: #{(sessions_for_new.to_f / new_visitors.count).round(2)}"
-```
-
-```ruby
-# 3. Check for any new outliers (visitors with 10+ sessions in 24h)
-cutoff = Time.parse("2025-12-23 08:30:00 UTC")
-
-outliers = Session.where("created_at > ?", cutoff)
-  .group(:visitor_id)
-  .having("count(*) > 10")
-  .count
-
-puts "Visitors with 10+ sessions since fix: #{outliers.count}"
-outliers.sort_by { |_, v| -v }.first(5).each do |vid, count|
-  puts "  Visitor #{vid}: #{count} sessions"
-end
-```
-
-### Expected Results After Fix:
-
-- [ ] Session creation rate drops by 90%+
-- [ ] Sessions per new visitor ratio < 2.0
-- [ ] No new outliers with 100+ sessions
-- [ ] Cookie session_id matches env session_id (verified by tests)
-
----
-
-## Other SDKs to Review
-
-**CRITICAL**: Check all other SDKs for the same thread-safety bug!
-
-### SDK Review Checklist:
-
-| SDK | Location | Status | Reviewed By | Date |
-|-----|----------|--------|-------------|------|
-| mbuzz-ruby | `/Users/vlad/code/m/mbuzz-ruby` | FIXED | Claude | 2025-12-22 |
-| mbuzz-python | `/Users/vlad/code/m/mbuzz-python` | SAFE | Claude | 2025-12-22 |
-| mbuzz-php | `/Users/vlad/code/m/mbuzz-php` | SAFE | Claude | 2025-12-22 |
-| mbuzz-node | `/Users/vlad/code/m/mbuzz-node` | SAFE | Claude | 2025-12-22 |
-
-### Review Results:
-
-**mbuzz-python**: SAFE
-- Uses `contextvars.ContextVar` for thread-safe context storage
-- Uses Flask's `g` object for request-scoped storage
-- Local variables used throughout middleware
-- Async session creation captures values in local variables before spawning thread
-
-**mbuzz-php**: SAFE
-- PHP is single-process per request by default
-- No shared state between requests
-- Each request gets fresh instance of everything
-
-**mbuzz-node**: SAFE
-- Uses `AsyncLocalStorage` from `node:async_hooks` for async request isolation
-- Express middleware uses local variables (`visitor`, `session`, `secure`)
-- Attaches data to request-scoped `req.mbuzz` object
-- `createSessionAsync` captures values as function parameters before `setImmediate`
-- Node.js is single-threaded, so race conditions are inherently less likely
-
-### What to Look For:
-
-1. **Middleware/Handler using instance variables or class variables for request-specific data**
-   - BAD: `self.session_id = ...` or `@session_id = ...`
-   - GOOD: Local variables passed through function calls
-
-2. **Mutable shared state**
-   - BAD: Global or class-level dicts/hashes storing request data
-   - GOOD: Request-scoped context objects or local variables
-
-3. **Thread-local storage without proper cleanup**
-   - Check that thread-local data is cleared after each request
-
-### Python-specific concerns:
-- Check for module-level variables
-- Check Flask/Django middleware for shared state
-- WSGI apps can have similar issues with global state
-
-### PHP-specific concerns:
-- PHP is typically single-threaded per request, so likely SAFE
-- But check for any persistent worker modes (Swoole, RoadRunner, FrankenPHP)
-
-### Node.js-specific concerns:
-- Node is single-threaded, so likely SAFE
-- But check for any shared state in closures or module scope
-
----
-
-## Data Cleanup (Optional)
-
-After verifying the fix works, consider cleaning up the bad data:
-
-```ruby
-# Find sessions with no events (likely created by the bug)
-# BE CAREFUL - only run after thorough analysis
-
-# Count empty sessions by account
-Account.find_each do |account|
-  session_ids_with_events = account.events.distinct.pluck(:session_id)
-  empty_sessions = account.sessions.where.not(session_id: session_ids_with_events).count
-  total_sessions = account.sessions.count
-
-  next if empty_sessions == 0
-
-  puts "#{account.name}: #{empty_sessions}/#{total_sessions} empty sessions (#{(empty_sessions.to_f/total_sessions*100).round(1)}%)"
-end
-```
-
----
-
-## Notes
-
-- Bug was discovered via dashboard metrics investigation (avg visits showing 28.5 with 0.6 avg days)
-- Traced to Pet Resorts Australia account (PetPro360)
-- Logs showed session creation every few seconds with different session_ids
-- Test added: `test_race_condition_with_slow_app` - 49/50 failures before fix, 0 after