railwatch 0.5.1 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f70fd1ff6ba5db7cfb11bff9fd076b480d6ac899c0a65694b9fbb7ce4753a10
-  data.tar.gz: da7db6bc23129be7120f38405d522b1be212b76dd7d30435088fbd462e795028
+  metadata.gz: 6da9b1e629335b1623cfc24e8b18c5a5d17bed076630768b9d947ff2d0b1a5c2
+  data.tar.gz: 827e8cc13fa4e1373bd6656c851e7fabbdfe2af60c37432c9feeb10801972367
 SHA512:
-  metadata.gz: 7c042d82399b79a4aeefaefd0b6a9d9af3d6868af5b4656d215115b46d84c233bbbe185c6e4692170c36c057431328f66d01ad2b4885d8b470e3cc534dbc17b1
-  data.tar.gz: 8eb054b07c1e559362647bd125e030add13ee7b4d84d03bdd72dd1bfe3b89cdb26d98f961d01c20c682cf320570700abd6d61baca1c7263c5b3b44004d71926b
+  metadata.gz: 741964c31992572e63aeabb9a601482d4a4facb6f895ac1c3fe362c9e0f493f7e7b7636b070bc0909b764e38514495f838e8dbda9de5fef8423cd978fc833281
+  data.tar.gz: 2eb78e494a27b97451023876f6e72e1cb73bf4aebacb002af148d674841a3f62348c2992cd131d16f7cc75a6308203d68da2738b1431791c4a15fd134b7d3fc4

data/AGENTS.md

@@ -14,10 +14,14 @@ render, exception, and span is a child of one, linked by
 
 ```sh
 bundle add railwatch
-bin/rails generate railwatch:install --prompt-token --kamal-secrets
+bin/rails generate railwatch:install                                 # embedded: dashboard at /railwatch
+bin/rails generate railwatch:install --prompt-token --kamal-secrets  # or: Railwatch Cloud
 ```
 
-The generator writes `config/initializers/railwatch.rb`, mounts `Railwatch::Engine`
+With no flags the install is embedded (see [Embedded mode](docs/embedded.md)):
+telemetry stays in two SQLite databases the app owns and the dashboard is served
+at `/railwatch`. `--cloud`, or any of `--prompt-token`, `--token-stdin`, `--url`,
+`--kamal-secrets`, sends it to Railwatch Cloud instead. For the cloud install the generator writes `config/initializers/railwatch.rb`, mounts `Railwatch::Engine`
 at `/railwatch`, adds the Kamal `post-deploy` hook and the Inertia browser client
 where the app has them, requires `railwatch/rspec` (or `railwatch/minitest`) in the
 test helper, and then runs `railwatch:doctor`. A prompted/stdin/environment token

data/CHANGELOG.md

@@ -1,5 +1,124 @@
 # Changelog
 
+## Unreleased
+
+<!-- Pull requests add their entry here. The version number and the date are
+     filled in by the release commit, which is also the only commit that
+     touches lib/railwatch/version.rb and Gemfile.lock. See CONTRIBUTING.md. -->
+
+## 0.6.0 (2026-09-22)
+
+- Embedded is now the installer's default. `bin/rails generate
+  railwatch:install` with no flags writes what `--local` used to: the
+  `railwatch` and `railwatch_telemetry` SQLite databases, `plugin
+  :railwatch` in `config/puma.rb`, and `c.transport = :local`, so two
+  commands give a working dashboard at `/railwatch` with no account and no
+  token. The cloud install is `--cloud`, and any option that only means
+  something there (`--prompt-token`, `--token-stdin`, `--url`,
+  `--kamal-secrets`) implies it, so existing cloud instructions keep
+  working. An exported `RAILWATCH_TOKEN` on its own does not pick the
+  cloud: a token in the shell is not a decision about where data goes.
+  `--local` is gone (it is the default); the gem's runtime default when no
+  initializer sets a transport is unchanged. The embedded next steps now
+  end with how to mirror to Railwatch Cloud (`c.export_enabled`).
+
+- The embedded dashboard is open in development when HTTP Basic has no
+  credentials, so the first run needs no password step. Every other
+  environment is unchanged: closed, 401, until credentials exist. That
+  case now also logs a boot warning outside development and test (it was
+  only in the doctor), since a 401 on a deployed dashboard otherwise looks
+  like a broken install. Configured credentials apply in development too.
+
+- Bound how long Puma waits for the embedded writer to stop. The plugin sent
+  the writer TERM and then called `Process.wait` on it, which has no timeout:
+  a writer that did not exit -- stuck in a SQLite write, on a full disk --
+  held Puma's shutdown open for as long as it stayed stuck. Measured against
+  a child that ignores TERM, the stop never returned (the harness gave up at
+  10s with the child still alive). Puma now waits `shutdown_timeout` (2s),
+  the same allowance it gives its own reporter, then KILLs the writer and
+  reaps it, so a wedged writer costs a bounded 2s and never leaves a zombie.
+  A writer killed mid-batch loses nothing -- the batch is retried by id
+  against the next writer -- which is why the bound is not the writer's own
+  worst-case drain (13s at the defaults): waiting for it would buy no data,
+  only exit time, and that time counts against the container's stop grace.
+  A writer that exits on TERM is let go the moment it does (measured
+  ~50ms), and a pid the cluster has already reaped is still treated as gone.
+
+- Make one HTTP attempt per delivery. `Transport::Http#deliver` retried a
+  raised error or a 5xx once on its own, inside a reporter that already owns
+  a retry ladder of eight attempts, so each rung cost two socket timeouts
+  and the effective attempt count was about sixteen. Against a receiver that
+  accepts connections and never answers, one delivery cost 6.0s; it now
+  costs 3.0s (one `read_timeout`), and one attempt, like `deliver_encoded`.
+  Response classification and `Retry-After` are unchanged: a 5xx or a raised
+  error still comes back retryable and the reporter still schedules it.
+
+- Correct `docs/configuration.md` and `docs/troubleshooting.md`, which told
+  users to raise `buffer_size` under pressure. At the defaults the byte
+  ceiling (`buffer_bytes`, 16 MiB) fills at roughly 5,000 records on a
+  realistic mix, so the 10,000-record count is never reached and raising it
+  changes nothing. The setting stays; the advice now points at `buffer_bytes`.
+
+- Say so when records are lost. `Railwatch.on_unrecoverable` fell back to
+  the debug log, so with no callback registered and `RAILWATCH_DEBUG` unset
+  a batch dropped after its retry ladder, one the receiver permanently
+  refused, or the records still unsent when `at_exit`'s bounded shutdown
+  ran out of time all vanished without a word. Since 0.3.7 that shutdown is
+  the only delivery a rake task or `rails runner` gets, so a cron job whose
+  exception never reached the platform looked exactly like one that had
+  nothing to report. Confirmed against a receiver that accepts and never
+  answers: the process left inside `shutdown_timeout` carrying three unsent
+  records and printed nothing.
+
+  A `Reporter::DeliveryError` -- raised only once the records are already
+  gone -- now prints one `[railwatch]` stderr line, and `warn_on_data_loss`
+  (`RAILWATCH_WARN_ON_DATA_LOSS`) defaults to **on**. Silence was the wrong
+  default: telemetry that disappears without a word looks exactly like
+  having nothing to report, which is the one failure an operator cannot
+  diagnose from the platform side, because the evidence is what went
+  missing. One line a deploy is the whole cost, and it only ever appears
+  when something was actually lost.
+
+  Both ways out are named in the line itself, so nobody has to find this
+  entry to stop it: a registered `on_unrecoverable` always wins, which is
+  how an app routes the loss somewhere better (`Rails.error.report`), and
+  `warn_on_data_loss = false` restores silence. Recovered internal errors (a
+  subscriber that raised, a flush that will be retried) stay debug-only
+  either way: the gem carried on and there is nothing for an operator to do.
+
+- Report a lost batch outside the flush lock too. The fix above moved the
+  callback out of `@mutex`, the inner lock -- but `Reporter#flush` holds
+  `@flush_mutex` around the whole of `deliver_buffer`, and the give-up path,
+  the permanent-rejection path and the rescue all report from inside it. A
+  callback that asks this same reporter to flush (`Railwatch.flush` is public
+  and documented) hit the same non-reentrant `Mutex` one level out: the same
+  `ThreadError: deadlock; recursive locking`, rescued and hidden by
+  `notify_unrecoverable`, so the callback ran halfway and reported nothing.
+  The locked path now collects what it needs to report and `flush` hands it
+  over once the lock is released. Found by CodeRabbit on this pull request.
+
+- Report a given-up batch after releasing the reporter lock, not under it.
+  `Reporter#retain` called `on_unrecoverable` inside `@mutex.synchronize`.
+  The documented callback is `Rails.error.report`, whose subscriber records
+  the error as an exception -- a write back into the same reporter, which
+  takes `@mutex` to arm its thread or request a flush. That was
+  `ThreadError: deadlock; recursive locking`, rescued and hidden by
+  `notify_unrecoverable`, so the callback died halfway and the loss it was
+  reporting was never seen. A callback that merely blocked held every
+  request thread's `write_now` and `shutdown` itself behind it for the
+  duration. `notify_unsent` and `delivery_rejected` already called out
+  unlocked; this was the one that did not.
+
+- Investigated and left alone: `Reporter#shutdown` after `thread.join`.
+  The bookkeeping that follows the join (`pending_delivery`, the
+  once-only notify latch) takes `@mutex` for microseconds and never does
+  I/O -- measured 0.2ms over `shutdown_timeout` against a transport wedged
+  forever. The only thing that can extend it is the operator's own
+  callback, which runs once and is theirs to bound, as any `at_exit`
+  handler is. Wrapping it in `Timeout` would trade a visible cost for a
+  killed thread.
+
+
 ## 0.5.1 (2026-09-21)
 
 Three small seams for a host that runs these models on its own routes and

data/README.md

@@ -6,25 +6,31 @@ outgoing HTTP, storage, views, and logs, and links them into one trace
 per execution, for about half a millisecond per request plus tens of
 microseconds per query, with zero writes to your database.
 
-Send that to Railwatch Cloud, or keep all of it inside the app:
-[embedded mode](docs/embedded.md) serves the same dashboard at
+By default all of it stays inside the app:
+[embedded mode](docs/embedded.md) serves the full dashboard at
 `/railwatch` out of two SQLite files your app owns, with no token, no
-Node, no Redis and no job worker.
+Node, no Redis and no job worker. Railwatch Cloud is optional, for
+alerts that arrive when the app is down, MCP for your AI assistant, and
+many apps and servers in one place.
 
 ## Install
 
 ```sh
-bundle add railwatch            # 1. add the public gem
-bin/rails generate railwatch:install --prompt-token  # 2. hidden token input plus app wiring
-bin/rails railwatch:doctor                    # 3. check every piece is wired up after restart
+bundle add railwatch                  # 1. add the public gem
+bin/rails generate railwatch:install  # 2. embedded: databases, Puma writer, dashboard at /railwatch
 ```
 
-Or keep everything inside your app, with the full dashboard at
-`/railwatch` and no token ([Embedded mode](docs/embedded.md)):
+Restart and open `/railwatch`. It is open in development; before you
+deploy, give it a password with
+`RAILS_ENV=production bin/rails railwatch:authentication:configure`
+(it answers 401 in production until you do).
+
+Or send everything to Railwatch Cloud instead:
 
 ```sh
 bundle add railwatch
-bin/rails generate railwatch:install --local
+bin/rails generate railwatch:install --prompt-token  # hidden token input plus app wiring
+bin/rails railwatch:doctor                          # check every piece is wired up after restart
 ```
 
 Getting the token, the generator's flags, and deploying with Kamal,
@@ -73,7 +79,7 @@ expect { get "/widgets" }.not_to have_railwatch_n_plus_one
 
 ## Embedded mode
 
-`--local` keeps everything inside the application. Telemetry goes to two
+The default install keeps everything inside the application. Telemetry goes to two
 SQLite files it owns -- `railwatch` for issues, comments and saved views,
 `railwatch_telemetry` for what the app reports -- and the dashboard is
 served at `/railwatch` from a bundle shipped inside the gem. Nothing
@@ -86,9 +92,11 @@ runs the maintenance clock too, so exception grouping, rollups, retention
 and threshold scans happen without a queue.
 
 That dashboard reads every query, log line and exception the app
-produced, so it is closed by default the way Mission Control Jobs is:
-HTTP Basic is on with no credentials, and every request is 401 until you
-set them with `bin/rails railwatch:authentication:configure`. Apps that
+produced, so outside development it is closed by default the way
+Mission Control Jobs is: HTTP Basic is on with no credentials, and every
+request is 401 until you set them with
+`bin/rails railwatch:authentication:configure`. In development, with no
+credentials set, it is open. Apps that
 would rather use their own session hand it a `dashboard_user` resolver
 instead. [Embedded mode](docs/embedded.md) covers all of it, including
 upgrades and what it costs to store.

data/app/controllers/railwatch/dashboard_controller.rb

@@ -47,6 +47,7 @@ module Railwatch
     def authenticate_by_http_basic
       config = Railwatch.config
       return unless config.http_basic_auth_enabled
+      return if config.http_basic_auth_waived?
 
       if config.http_basic_auth_configured?
         http_basic_authenticate_or_request_with(name: config.http_basic_auth_user, password: config.http_basic_auth_password,

data/app/models/railwatch/application_record.rb

@@ -15,7 +15,7 @@ module Railwatch
       # No `railwatch` entry in this environment's database.yml, or an entry
       # whose adapter gem is not in the bundle yet (LoadError). A cloud-transport
       # app has none and still eager-loads this class in production, and so
-      # does the --local installer's own boot, before it has written the
+      # does the installer's own boot, before it has written the
       # entry -- so loading must not raise. Using it must, though: without
       # connects_to this class would inherit ActiveRecord::Base's PRIMARY
       # connection, and its tables are unprefixed, so a query would read and
@@ -25,7 +25,7 @@ module Railwatch
       def self.connection_pool
         raise Railwatch::DatabaseNotConfigured,
               "the `railwatch` database (issues, comments, saved views and deploys) is not configured for the " \
-              "#{Rails.env} environment; run `bin/rails generate railwatch:install --local` " \
+              "#{Rails.env} environment; run `bin/rails generate railwatch:install` " \
               "or add it to config/database.yml (docs/embedded.md)"
       end
     end

data/app/models/railwatch/telemetry_record.rb

@@ -18,7 +18,7 @@ module Railwatch
       # No `railwatch_telemetry` entry in this environment's database.yml, or an entry
       # whose adapter gem is not in the bundle yet (LoadError). A cloud-transport
       # app has none and still eager-loads this class in production, and so
-      # does the --local installer's own boot, before it has written the
+      # does the installer's own boot, before it has written the
       # entry -- so loading must not raise. Using it must, though: without
       # connects_to this class would inherit ActiveRecord::Base's PRIMARY
       # connection, and its tables are unprefixed, so a query would read and
@@ -28,7 +28,7 @@ module Railwatch
       def self.connection_pool
         raise Railwatch::DatabaseNotConfigured,
               "the `railwatch_telemetry` database (everything the app reports) is not configured for the " \
-              "#{Rails.env} environment; run `bin/rails generate railwatch:install --local` " \
+              "#{Rails.env} environment; run `bin/rails generate railwatch:install` " \
               "or add it to config/database.yml (docs/embedded.md)"
       end
     end

data/docs/configuration.md

@@ -385,7 +385,7 @@ the app database.
 
 | Attribute | Env var | Default | Meaning |
 |---|---|---|---|
-| `buffer_size` | `RAILWATCH_BUFFER_SIZE` | `10000` | Max buffered records (`Railwatch::Buffer`). Oldest is dropped (and counted) when full — never blocks the request thread. Keep it at or above `Execution::MAX_RECORDS` (10,000): a kept execution's whole tree is written here at once when it ends, and a queue smaller than the tree drops the tree's own oldest records first. |
+| `buffer_size` | `RAILWATCH_BUFFER_SIZE` | `10000` | Max buffered records (`Railwatch::Buffer`). Oldest is dropped (and counted) when full — never blocks the request thread. In practice `buffer_bytes` fills first: on a realistic mix of records, 16 MiB holds about 5,000 of them, so this count is never reached and raising it changes nothing. Tune `buffer_bytes` instead. Do not lower this below `Execution::MAX_RECORDS` (10,000): a kept execution's whole tree is written here at once when it ends, and a queue smaller than the tree drops the tree's own oldest records first. |
 | `buffer_bytes` | `RAILWATCH_BUFFER_BYTES` | `16777216` (16 MiB) | Estimated payload memory the reporter queue may hold. A record count alone does not bound memory: 10,000 records is a few megabytes of ordinary telemetry, or a gigabyte of captured attachments. Oldest records are dropped (and counted) under byte pressure, same as under count pressure. |
 | `execution_buffer_bytes` | `RAILWATCH_EXECUTION_BUFFER_BYTES` | `8388608` (8 MiB) | The same ceiling for one execution's buffered tree, before it finishes. A normal execution keeps its earliest records; a failure-context ring keeps its latest. |
 | `batch_bytes` | `RAILWATCH_BATCH_BYTES` | `8388608` (8 MiB) | Uncompressed NDJSON bytes in one ingest request. A queue holding more than this is delivered as several batches — the tail is kept for the next flush, not dropped. |
@@ -399,13 +399,13 @@ the app database.
 
 Delivery is `Railwatch::Transport::Http`, in
 `lib/railwatch/transport/http.rb`: a gzip NDJSON POST to
-`{ingest_url}/ingest`, with one retry on a raised error or a 5xx within
-each delivery attempt. If that still fails, or ingest returns 402, 408,
-or 429, the immutable batch and its prior drop count are retained for
-retry. Every newly formed batch gets an `X-Railwatch-Batch-Id` UUID. It is
-reused for the immediate HTTP retry and every later reporter retry, so
-the platform can return the first committed result without inserting the
-payload twice. Records written while a request is in flight collect in a
+`{ingest_url}/ingest`, one attempt per delivery. If it raises, or ingest
+returns 402, 408, 429, or a 5xx, the immutable batch and its prior drop
+count are retained for retry; the transport does not retry on its own,
+so each rung of the reporter's ladder costs one timeout, not two. Every
+newly formed batch gets an `X-Railwatch-Batch-Id` UUID. It is reused for
+every reporter retry, so the platform can return the first committed
+result without inserting the payload twice. Records written while a request is in flight collect in a
 separate bounded buffer, so they never change the retained request's
 identity. At most one retained batch plus one live buffer are held in
 memory. The reporter retries with jittered exponential backoff, from one
@@ -827,9 +827,25 @@ Railwatch.on_unrecoverable { |error| Rails.error.report(error, handled: true) }
 Called whenever Railwatch rescues one of its own internal errors, ingest
 permanently rejects a batch, or shutdown expires with retained records
 that could not be sent. Retryable delivery failures stay buffered and do
-not fire the callback on every attempt. With no callback registered, this
-falls back to `Railwatch.debug`. That goes to stderr, gated on
-`RAILWATCH_DEBUG`, never `Rails.logger`, so gem-internal failures can
+not fire the callback on every attempt. With no callback registered, a
+recovered internal error falls back to `Railwatch.debug`, gated on
+`RAILWATCH_DEBUG` — the gem carried on and there is nothing to do about it.
+
+**Lost records are different, and are reported by default.** A batch
+dropped after its retry ladder, one the receiver permanently refused, or
+records still unsent when the bounded shutdown ran out of time each print
+one `[railwatch]` stderr line. Telemetry that vanishes silently looks
+exactly like having nothing to report, and a short-lived process — a rake
+task, a `rails runner`, a cron job — gets one bounded shutdown and no
+second chance to mention it.
+
+Two ways out, and the line names both. A registered `on_unrecoverable`
+always wins, which is how an app routes the loss somewhere better
+(`Rails.error.report`). Or set `warn_on_data_loss = false`
+(`RAILWATCH_WARN_ON_DATA_LOSS=false`) and the gem goes back to saying
+nothing.
+
+Either way it is stderr, never `Rails.logger`, so gem-internal failures can
 never themselves become `log` records.
 
 ## Faraday
@@ -872,7 +888,7 @@ c.transport = :local        # RAILWATCH_TRANSPORT; default "http"
 c.issue_prefix = "SHOP"     # RAILWATCH_ISSUE_PREFIX; default from the app name
 c.repository_url = "..."    # RAILWATCH_REPOSITORY_URL
 c.retention_days = 7        # RAILWATCH_RETENTION_DAYS
-c.http_basic_auth_enabled = true      # RAILWATCH_HTTP_BASIC_AUTH_ENABLED; on and closed until credentials exist
+c.http_basic_auth_enabled = true      # RAILWATCH_HTTP_BASIC_AUTH_ENABLED; on, and closed outside development until credentials exist
 c.http_basic_auth_user = "ops"        # RAILWATCH_HTTP_BASIC_AUTH_USER, or credentials railwatch.http_basic_auth_user
 c.http_basic_auth_password = "..."    # RAILWATCH_HTTP_BASIC_AUTH_PASSWORD, or credentials railwatch.http_basic_auth_password
 c.base_controller_class = "AdminController"  # RAILWATCH_BASE_CONTROLLER_CLASS; default ActionController::Base
@@ -884,7 +900,7 @@ With `transport = :local` the reporter writes each batch into the app's
 own `railwatch_telemetry` database instead of POSTing it, and the engine
 serves the dashboard at its mount. `enabled?` no longer needs a token.
 The others only matter in that mode. The dashboard is behind HTTP Basic
-by default and answers 401 until `bin/rails
+by default and, outside development, answers 401 until `bin/rails
 railwatch:authentication:configure` has written credentials; a host with
 its own admin auth turns Basic off and sets `base_controller_class` or a
 routes constraint. Full walkthrough: [Embedded mode](embedded.md).

data/docs/embedded.md

@@ -1,7 +1,7 @@
 # Embedded mode: the dashboard inside your app
 
-Railwatch can keep every record in your own application and serve the
-full dashboard at `/railwatch`, with no token and no cloud. The gem's
+This is the default. Railwatch keeps every record in your own application
+and serves the full dashboard at `/railwatch`, with no token and no cloud. The gem's
 reporter, buffer and sampling are the same; the only difference is
 where a batch ends up. In embedded mode it is written straight into a
 SQLite database your app owns, and the dashboard reads it back from
@@ -25,8 +25,9 @@ up:
 | **Cloud** | Railwatch Cloud | the hosted one |
 | **Both** | your app's files, *and* Railwatch Cloud | either |
 
-Embedded is `c.transport = :local`, which is what `--local` writes.
-Cloud is the default. "Both" is embedded plus one more line:
+Embedded is `c.transport = :local`, which is what the installer writes
+unless you ask it for the cloud (`--cloud`, or any token or URL option).
+Cloud is the gem's default when no initializer says otherwise. "Both" is embedded plus one more line:
 
 ```ruby
 c.export_enabled = true   # or RAILWATCH_EXPORT_ENABLED=true
@@ -59,10 +60,11 @@ documents: `id`, `slug`, `name`, `with_telemetry`, and the display attributes.
 
 ```sh
 bundle add railwatch
-bin/rails generate railwatch:install --local
+bin/rails generate railwatch:install
 ```
 
-Restart the app and open `/railwatch`. Then `bin/rails railwatch:doctor`
+Restart the app and open `/railwatch`; in development it is open with no
+password (see [Authentication](#authentication) for production). Then `bin/rails railwatch:doctor`
 checks the wiring. The generator creates and migrates both databases
 itself; `bin/rails db:prepare`, which a deploy already runs, migrates
 them after every gem update.
@@ -83,7 +85,7 @@ On a PostgreSQL or MySQL app that means the install is two commands
 rather than one, because SQLite's adapter gem will not be in your bundle:
 
 ```sh
-bin/rails generate railwatch:install --local   # adds gem "sqlite3", writes the config
+bin/rails generate railwatch:install   # adds gem "sqlite3", writes the config
 bundle install
 bin/rails db:prepare                           # creates the two SQLite files
 ```
@@ -92,7 +94,7 @@ Verified end to end on both. On a PostgreSQL app and on a MySQL app, the
 application's own four databases stay where they were, Railwatch's two are
 files under `storage/`, and neither server gains a single Railwatch table.
 
-What `--local` writes, on top of the usual install:
+What the embedded install writes, on top of what every install writes:
 
 - `config/initializers/railwatch.rb` with `c.transport = :local` and the
   dashboard's own paths excluded from request capture.
@@ -140,15 +142,22 @@ and the MCP server.
 The dashboard shows every query, log line and exception your app
 produced, so it works the way Mission Control Jobs does: **HTTP Basic
 authentication is on and closed by default**. With no credentials
-configured every dashboard request is 401, and `railwatch:doctor` says
-so. Set them with
+configured every dashboard request is 401, the app logs a warning at
+boot, and `railwatch:doctor` says so. Set them with
 
 ```sh
 bin/rails railwatch:authentication:configure
 RAILS_ENV=production bin/rails railwatch:authentication:configure
 ```
 
-which writes them to that environment's Rails credentials:
+The one exception is development. There, with Basic on and no
+credentials set, the dashboard is open, so a first run is the install and
+a page rather than a password step first; Rails already shows full error
+pages in development for the same reason. Set credentials there too and
+development asks for them like everywhere else. Test, staging and
+production are closed until you do.
+
+`railwatch:authentication:configure` writes them to that environment's Rails credentials:
 
 ```yml
 railwatch:
@@ -262,7 +271,7 @@ authorisation rule as well as a label.
 ## The writer process
 
 Puma forks one Railwatch writer from its master when `config/puma.rb`
-carries the plugin (`--local` adds it):
+carries the plugin (the embedded install adds it):
 
 ```ruby
 plugin :railwatch if defined?(Railwatch)
@@ -319,6 +328,17 @@ through `Railwatch.on_unrecoverable`, so loss is never silent. A Puma
 phased restart stops the writer and starts a fresh one once the new
 workers are up.
 
+Stopping the writer is bounded. Puma sends it TERM and waits up to
+`c.shutdown_timeout` (2 seconds) for it to exit, the same allowance it
+gives its own reporter, then kills it. A writer killed mid-batch loses
+nothing: the transaction rolls back and the worker retries that batch by
+id against the next writer, so waiting longer for its drain would buy no
+data. The bound is what keeps Puma's exit short when the writer is wedged
+in a SQLite write or on a full disk. It counts against the container's
+stop grace (Docker's default is 10 seconds; Kamal's `stop_timeout` sets
+it), and `RAILWATCH_SHUTDOWN_TIMEOUT` raises it for an app whose grace
+allows more.
+
 ## Maintenance
 
 Railwatch needs no job worker and nothing in `config/recurring.yml`.

data/docs/faq.md

@@ -182,10 +182,10 @@ being silent.
 A queue holding more than one batch is delivered as several batches: the
 tail is put back for the next flush rather than dropped.
 
-A background thread drains the buffer and POSTs. Each POST retries one
-raised network error or 5xx immediately. If delivery still fails, the batch
-and its drop counter go back into the bounded buffer; **402**, **408**,
-**429**, and all **5xx** responses are retained the same way. So is a **2xx
+A background thread drains the buffer and POSTs, one attempt per delivery.
+If it fails, the batch and its drop counter go back into the bounded
+buffer; a raised network error, **402**, **408**, **429**, and all **5xx**
+responses are retained the same way. So is a **2xx
 that cannot acknowledge the batch** — a proxy's HTML sign-in page, malformed
 JSON, or `accepted`/`rejected` counts that do not cover what was sent — which
 would otherwise be a silent drop. The reporter
@@ -217,7 +217,8 @@ already appear on its own ingest batch. Those are visible under
 On shutdown, `at_exit` gives the thread `c.shutdown_timeout` (2 seconds) to
 attempt retained records immediately and retry within the remaining time.
 If the deadline expires, the records stay retained and their count is sent
-to `Railwatch.on_unrecoverable` (or stderr under `RAILWATCH_DEBUG=1`). This is an
+to `Railwatch.on_unrecoverable` (or, with no callback, to `Railwatch.debug`, and
+to stderr by default, which `warn_on_data_loss = false` turns off). This is an
 in-memory buffer, not an on-disk spool: a hard kill, or exiting after that
 deadline, cannot carry those records into the next process. Railwatch never
 uses `Rails.logger` for its own failures, which would turn them into `log`

data/docs/getting-started.md

@@ -4,10 +4,10 @@ Five minutes from `bundle add` to a request on the dashboard, on a
 Rails 8 app. Everything below is the gem's own generator and rake tasks;
 nothing else has to be wired by hand.
 
-Running one server and want the dashboard inside the app itself, with
-no token and no cloud? That is `bin/rails generate railwatch:install
---local`; see [Embedded mode](embedded.md). The rest of this page is
-the cloud install.
+The installer's default is embedded: the dashboard inside the app at
+`/railwatch`, no token and no cloud; see [Embedded mode](embedded.md).
+This page is the cloud install, which is the same generator with
+`--cloud` or any of the token options below.
 
 ## 1. Add the gem
 
@@ -21,7 +21,7 @@ The gem, its Ruby namespace, and its require path share one name:
 ## 2. Run the installer
 
 ```sh
-bin/rails generate railwatch:install
+bin/rails generate railwatch:install --cloud
 ```
 
 With the token already in hand, let the generator read it without placing the
@@ -36,6 +36,10 @@ bin/rails generate railwatch:install \
 
 - `--prompt-token` reads without echo. `--token-stdin` is available for a
   secret-manager pipe; an already exported `RAILWATCH_TOKEN` is also detected.
+- Any of `--prompt-token`, `--token-stdin`, `--url=` and `--kamal-secrets`
+  means the cloud, so none of them needs `--cloud` as well. An exported
+  `RAILWATCH_TOKEN` on its own does not: without one of these flags the
+  install is embedded.
 - A token is written to `.env` only when Git confirms that `.env` is ignored.
   A tracked or non-ignored dotenv file is refused; use Rails credentials, a
   deployment secret manager, or add `.env` to `.gitignore` first. Token values

data/docs/troubleshooting.md

@@ -215,14 +215,18 @@ can't be made until it ends.
   (10,000). Past that, records are dropped and counted. The count is
   added to the reporter's drop counter so the loss is visible on the
   platform rather than silent.
-- `c.buffer_size` (default 10,000, the same as `MAX_RECORDS`) caps the
-  process-wide queue between the app and the reporter thread.
-  Oldest-dropped-first, also counted. Do not set it below `MAX_RECORDS`.
-  An execution's tree is written to the queue in one go when it ends, so
-  a tree larger than the queue loses its own first records. Typically
+- The process-wide queue between the app and the reporter thread is
+  bounded by `c.buffer_bytes` (default 16 MiB) and by `c.buffer_size`
+  (default 10,000, the same as `MAX_RECORDS`). Oldest-dropped-first, also
+  counted. The byte ceiling is the one that fills: on a realistic mix of
+  records, 16 MiB holds about 5,000 of them, so raising `buffer_size`
+  changes nothing. Do not set it below `MAX_RECORDS`, though: an
+  execution's tree is written to the queue in one go when it ends, so a
+  tree larger than the queue loses its own first records. Typically
   those are the outgoing requests a long job made before it started
   writing. Keeping far more executions than before means far more
-  records arriving at this queue. Raise it, or lower what you keep.
+  records arriving at this queue. Raise `buffer_bytes`, or lower what
+  you keep.
 - `c.profile_slow_ms` compounds it. It profiles every tail-buffering
   execution from its first line and throws away the fast ones, so the
   profiler's stack table is held alongside the record buffer.