logstruct 0.1.12 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3aec4782f7bcf89ea8cba8792210e1e5f04165221250178ed283196d7ac89c9d
-  data.tar.gz: 4b1d1ffc34a814f4131a43074a598fca7a14acb4ac4cc1ae41cb51f3f35125c2
+  metadata.gz: 30339bbe0c17b5baf55abe19e911758af000864097ec3777d133572da938c9b4
+  data.tar.gz: 2963dd6d8680361a664fdea12e6137176a065e18a5bb2b7332556b648a09ef4d
 SHA512:
-  metadata.gz: ab3a907bc346e681caefe09e8a5abda86b4724221722460a6376f13601c2c10dd6d066f5e523b7b1d6088afbb6db60b6a506d57c25b2c762aacb9e7a602ca2cc
-  data.tar.gz: 64812fee369aa713104846f69dea5ac17b93c12258ef619ff63c7c9726f51b79fc6d449e45e026d7cd46cc24441ed6a820386346be4df6655ae73974742c2bb9
+  metadata.gz: 3a6bb597673b7fffc1dac97c78b184b2277ec363ab14f061bf00e1d58931259a41ec82c67cafa32c101389201f3c58a0aee079087513ae02b430f5158ea11316
+  data.tar.gz: 14dc4bb646c20a8a9e917e451e4e6a43cddd49cc419ee120d9e7182e735be595cd50f7a397c98f59abef04469d1629496b37f72cb84701d34fb30df4cd60a9c1

data/CHANGELOG.md

@@ -5,10 +5,142 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.2.0] - 2026-10-18
 
 ### Changed
 
+- **Breaking**: A Hash passed to the logger (e.g. `Rails.logger.info({evt: "signup", user_id: 1})`) is written with its keys at the top level of the line, instead of nested under `payload`. A Hash can set its own `src` and `evt`; `ts` and `lvl` are reserved and are reported as an error and dropped
+- **Breaking**: `Log::Error`'s `error_class` is any `Exception` class, not only `StandardError` classes, since ActiveJob and GoodJob pass any `Exception` that a job lets through
+- **Breaking**: LogStruct depends on `railties`, `actionpack`, `activesupport`, `globalid` and `lograge` instead of `rails`, so apps no longer install every Rails framework to use it (#9, thanks @langalex)
+- **Rails**: Integrations are set up after `config/initializers` run, so `config.integrations.*` settings and filters added in `config/initializers/logstruct.rb` take effect. `enabled` and `enabled_environments` can be set in an initializer; `LOGSTRUCT_ENABLED` overrides both
+- **Rails**: Because integrations are set up after the app's initializers, LogStruct's loggers and handlers replace ones an initializer set (for example a custom Sidekiq logger or lograge settings). Configure them through LogStruct instead
+- **Sidekiq**: A failed job is logged once, as a structured warn line with its `jid`, class, queue and retry count, instead of Sidekiq's default handler's three lines with the job's arguments in a JSON string. ActiveJob jobs' failures are left to the ActiveJob integration
+- **Puma**: Request errors Puma handles itself (its "Rack app (...)" lines) are structured error lines with the method and path, without the query string
+- **Breaking**: Production (any environment outside `config.local_environments`) logs to STDOUT only. LogStruct added a `log/<env>.log` file appender, which nothing rotated, unless `RAILS_LOG_TO_STDOUT` was set, and it overrode `config.logger = STDOUT`. Development logs to STDOUT and its log file, test to its log file; with `RAILS_LOG_TO_STDOUT` set, only to STDOUT
+- **Filtering**: Rails' `filter_parameters` are matched the way Rails matches them: a Symbol or String filters every key that contains it (`:secret` filters `clientSecret`), case-insensitively, and Regexps and blocks are applied as Rails applies them. Rails' partial matches don't filter LogStruct's own top-level fields (Rails' default `:_key` would filter GoodJob's `cron_key`); exact names in the list and LogStruct's own filters still do
+- **Filtering**: Blocks in `filter_parameters` get copies of the key and value, so a block that changes its value in place doesn't change the app's data
+- **ActiveJob**: For `ActionMailer::MailDeliveryJob` (every `deliver_later`) and its subclasses, job lines log only the mailer, the mail and the delivery method as `arguments`, not the mail's own arguments. Devise passes its confirmation, password reset and invitation tokens there as plain strings, which key-based filtering can't find, so apps using Devise with `deliver_later` logged live tokens. Other jobs' arguments are logged as before (when the job's `log_arguments` is on). With the ActiveJob integration off, Rails' own job logs still include them
+- **Breaking**: Hashed values (email addresses, `filter_keys_with_hashes`) are an HMAC-SHA256 keyed with `hash_salt`, or by default with a key derived from the app's `secret_key_base`, and are 16 characters long. They were a SHA256 with a salt shared by every install, 12 characters long, which a list of candidate emails could reverse. Every hash value changes; set `hash_salt` to keep hashes the same across apps or after a `secret_key_base` change. Hashes are pseudonyms, not anonymous data
+- **Breaking**: Sensitive keys are matched by their words, whatever their case or separators, so `apiKey`, `X-Api-Key`, `HTTP_AUTHORIZATION`, `Set-Cookie` and plurals like `passwords` are filtered by default, and so are `cookie`, `session`, `jwt`, `bearer`, `otp` and `passphrase`. `filter_keys` entries match by their words too. Whole words keep `author`, `keyboard` and `signature_field_id`
+- **Breaking**: A filtered String is logged as `{_filtered: {_class: "String"}}`, without its length (`_bytes`)
+- **Breaking**: `Log::Sidekiq` is now `Log::Sidekiq::Log`. Sidekiq's job error line (`Log::Sidekiq::Error`), Puma's request error line (`Log::Puma::Error`) and ActiveJob's error line (`Log::ActiveJob::Error`) have typed fields instead of `additional_data`, with a new `error_context` field for what Sidekiq or Puma was doing. Error lines have the request's `path`, `method`, `source_ip`, `user_agent` and `referer` as fields
+- **Breaking**: `error_handling_modes.type_checking_errors` defaults to `ReportProduction`: Sorbet type errors in the app and its gems are raised in development and test, and logged and reported in production. They were only logged
+- **Breaking**: ActiveStorage `url` lines don't have the `url` field: the signed URL (S3 signatures, Disk tokens) lets anyone read the file until it expires. `url` and `exist` lines are debug lines, as Rails logs them
+- **Breaking**: Lines have size limits (`config.limits`): Hashes write their first 100 keys and a `_more_keys` count, backtrace-like Arrays (an error line's `backtrace` too) 20 lines, other Arrays 10 items, and strings are cut at 32 KB after they're scrubbed (a URL password or `Basic` credentials that the cut runs through are replaced too). Error lines' backtraces were cut to 5 lines
+- **Breaking**: The IP spoofing check no longer runs on every request and answers 403. Rails raises `IpSpoofAttackError` only where `remote_ip` is read, and responds as it would without LogStruct; LogStruct logs the error there (and reports it, in the `security_errors` mode)
+- **Errors**: A message logged with an exception (`logger.error("Import failed", e)`) is an error line with the message in `msg` and the exception's in the new `error_message` field
+- **Errors**: A failure building an integration's own line (ActiveModelSerializers, Dotenv, Puma, Ahoy, host authorization, GoodJob) is a LogStruct error (`logstruct_errors`: logged in production, raised in development and test), not one of the app's (`standard_errors`, raised everywhere), so it can't fail a render, a request or the boot in production
+- **Host authorization**: Blocked hosts get the app's own `response_app` if it set one (LogStruct's 403 page otherwise). LogStruct replaced it
+- **Puma**: The TERM and INT traps log the shutdown and then run the handler they replaced (or the signal's default action). The Puma patches, the boot banner replacement and the Puma lifecycle lines are off when LogStruct is disabled
+- **Ahoy**: The tracking wrapper passes Ahoy's arguments on as given, so Ahoy's default properties still apply
+- **Breaking**: LogStruct needs Ruby 3.3 or newer, the oldest version its CI tests
+- **Breaking**: Keys in `filter_keys_with_hashes` (default `email` and `email_address`) are filtered, with a `_hash` of the value so lines about the same address can be found. They were only hashed when another filter matched them too
+- **Breaking**: An email address has one hash however it's capitalized, and addresses in any script (`müller@example.de`) are scrubbed
+- **Breaking**: Every time in a struct line (`scheduled_at`, `started_at`, `finished_at`, ...) is written with milliseconds, like `ts`
+- **Breaking**: Removed fields that were never written: ActiveJob's `retries` on enqueue lines, the blocked host line's `allowed_hosts` and `allow_ip_hosts`, Shrine's `metadata` and `location` on metadata lines and `exist` on exists lines (Shrine's events don't carry them), and ActiveStorage's `filename`, `mime_type`, `size` and `metadata` on upload and download lines (Rails' events have only the blob's key). ActiveStorage metadata lines have the `mime_type` instead of `metadata`
+- **Breaking**: `Log::ActiveStorage::Stream` and the `stream` event are removed: Rails sends no `service_stream` event. Streaming downloads are `download` lines
+- **Requests**: Request lines have the client's IP (`source_ip`) and the `host`, as Rails' own request line does. A `lograge_custom_options` that added them can drop them
+- **Tags**: Tags are written: `logger.tagged("x")` and `config.log_tags` give a `tags` array, and named tags go in `ctx` (`request_id` is written as `req_id`)
+- **ActionMailer**: Mailers keep their logger, so `logger.info` in a mailer is written. LogStruct detaches Rails' ActionMailer log subscriber instead, which wrote the text lines its own mail lines replace
+- **Breaking**: **ActionMailer**: `log_and_report_error` always reports the error and never raises it, as its name says. It went through `error_handling_modes.standard_errors`, so in `Raise` mode (the default) it raised the error without reporting it, and in the `Log` and `Ignore` modes it didn't report it. `log_and_reraise_error` still raises every error, and reports it only in the `Report` modes
+- **ActionMailer**: The error line says "Email delivery failed" instead of "Email delivery error, will retry" (nothing retries a mail by default), and names the mail's recipients (scrubbed) when the error doesn't say which recipients failed
+- **ActiveJob**: A job the backend couldn't enqueue (`ActiveJob::EnqueueError`, or another exception) writes an `error` line instead of an `enqueue` line, and a job a `before_enqueue` callback stopped writes no line. A job a `before_perform` callback stopped has no `finish` line
+- **ActiveJob**: A finish line's `ts` is when the job finished, not when it started
+- **CarrierWave**: An `upload` line is written when an uploader stores a new file, with its duration, and not when a record is saved with files that are already stored. Loading a stored file (`retrieve_from_store!`) logs nothing
+- **Sidekiq**: Lines logged outside a job have no `ctx` (it was `{}`)
+- **Logger settings**: `prefer_json_in_development`, `enable_color_output`, `color_map`, `filter_noisy_loggers` and `local_environments` set in `config/initializers` apply: LogStruct adds its appenders again after the initializers run. Appenders the app added to SemanticLogger there are kept
+- **Development**: `color_map` changes only the colors it names; the others keep their defaults
+
+### Added
+
+- **ActiveJob**: `retry`, `retry_stopped` and `discard` lines, with the exception that `retry_on` or `discard_on` caught (and the wait before the retry). The exception was in no line. `perform_all_later` writes a line per job; with an adapter that reports only how many jobs it enqueued (Sidekiq's), a line per job when it enqueued them all, one warn line with the counts when it didn't, and one error line when it raised
+- **ActiveStorage**: Deleting files by prefix (a blob's variants, when an image is purged) is a `delete` line with `prefix`. It raised a type error inside LogStruct, and wrote no line
+- **CarrierWave**: `download` lines (a remote file fetched with `download!`) and `delete` lines (`remove!`)
+- **Shrine**: Upload lines have the file's `mime_type` and `size`, and metadata lines how long extracting the metadata took. They leave out the file's original name, which can name a person, and the file names in a `content_disposition` upload option
+- **Security**: Blocked host lines have `blocked_hosts`, every host Rails blocked (the Host header, and X-Forwarded-Host)
+- **Filtering**: `Bearer` tokens and `Basic` credentials in strings are scrubbed (`filters.authorization_credentials`, on by default)
+- **Filtering**: The default sensitive key words include `passcode`, `apikey`, `social_security`, `credit_card` and `card_number`
+- **Filtering**: US phone numbers are scrubbed in their other common formats (`(555) 123-4567`, `555.123.4567`, with or without `+1`). The opt-in scrubbers also match MAC addresses written with dashes (`filters.mac_addresses`) and IPv6 addresses (`filters.ip_addresses`)
+- **ActionMailer**: Where LogStruct is off (development, consoles, Rake tasks, job runners it doesn't recognize), mailers still have `log_and_ignore_error`, `log_and_report_error` and `log_and_reraise_error`, which log a text line and report or raise. A `rescue_from` naming one of them raised `NameError` there
+- **Docs**: Pages for each job, mail, SQL and security integration, every log field, when LogStruct runs, upgrading to 0.2.0 and troubleshooting
+- **Error handling**: `LogStruct.handle_exception` takes `log:` (default `true`). Pass `log: false` when your own structured line already logged the error, so that it's only reported (or raised) according to the mode. If your tests stub `handle_exception` with a lambda, accept `log:` too
+- **Sidekiq**: Every line logged during a Sidekiq job includes the job's `jid` and class in `ctx`. A Hash logged with its own `ctx` is merged with the job's
+- **Filtering**: `FilterMatcher` takes `matches_logstruct_fields:` (default `true`), and `ParamFilters.should_filter_key?` takes `logstruct_field:` and `path:`, for matchers that shouldn't apply to LogStruct's own field names and for nested keys
+- **Filtering**: Rails' dotted `filter_parameters` (`"card.number"`, `/\Acard\.number\z/`) match nested keys by their path, and three-argument filter blocks get the parameters around the key, as in Rails. New `filters.path_matchers` setting (a matcher that doesn't read the parameters sets `uses_root: false`, so they aren't copied for it)
+- **Filtering**: Sensitive query params in URLs are filtered (`?token=`, `?apiKey=`, `user[password]=`, OAuth's `?code=`), checked like keys. New `filters.url_query_params` setting, on by default
+- **Config**: `config.limits` (`max_hash_keys`, `max_array_items`, `max_backtrace_lines`, `max_string_bytes`)
+- **Forking servers**: SemanticLogger's appenders are reopened in every forked process (a `Process._fork` hook), so Unicorn, Passenger and Pitchfork workers, Resque jobs and other forked children write their lines. It replaces the Puma cluster worker patch
+- **Repository**: `SECURITY.md` (report vulnerabilities privately) and weekly Dependabot updates
+
+### Removed
+
+- **GoodJob**: `LogStruct::Integrations::GoodJob::LogSubscriber`, which listened for events GoodJob doesn't send (`enqueue`, `start`, `finish`, `error`, `schedule`), so it never logged anything. GoodJob's jobs are logged by the ActiveJob integration
+- **GoodJob**: the `Log::GoodJob::Enqueue`, `Start`, `Finish` and `Schedule` log types, which only that subscriber built. The Terraform provider's catalog lists only `error` and `log` for GoodJob; ActiveJob's job lines have the same `src` and `evt` as the removed types
+- **ActionMailer**: The mailer metadata that was meant to add the current tags and job ID to a mailer's lines. It asked for methods neither TaggedLogging nor ActiveJob has, so it never added anything
+- **Config**: `config.integrations.enable_semantic_logger`, which did nothing: LogStruct always writes through SemanticLogger. Setting it raises `NoMethodError`
+- **Rack**: `LogStruct::Integrations::Rack`, which nothing loaded and which required a file that doesn't exist
+
+### Fixed
+
+- **Lograge**: The Hash a `lograge_custom_options` proc returns is merged into the request line, as Lograge does
+- **Errors**: An error the app didn't handle (Rails' 500 page, and errors Rails turns into a 404 or 422) is an error line with `src` `rails`, the error's class and message, the trace Rails logs, the request's path, method and IP, the response's `status`, and the error's cause (`cause_class`, `cause_message`), instead of Rails' block of text. It's written at `config.action_dispatch.debug_exception_log_level`, and `config.action_dispatch.log_rescued_responses = false` still leaves out the 404s and 422s. A CSRF or IP spoofing error has only its security line
+- **Sorbet**: LogStruct installs Sorbet's four runtime error handlers once. It also installed one that the next replaced
+- **Security**: LogStruct no longer empties Rails' `filter_parameters` or loads ActiveRecord and the other frameworks before the app's initializers run, so `filter_attributes` and Rails' own parameter filtering keep working (GHSA-cpp6-p4hf-383g)
+- **Security**: Values nested deeper than the depth limit are summarized instead of logged raw, skipping filtering and scrubbing (#15)
+- **Security**: When Rails can't parse a request's body (malformed JSON), its debug line gives the body's size and type. Rails logged the whole body there, so a password in it was logged unfiltered
+- **Security**: The error line for params Rails can't parse (malformed JSON, a bad %-encoding) leaves out the input the parser's message quotes: it has no `cause_message`, and Rails' "Invalid request parameters: ..." stops before the input. So do the lines for such an error that the app logs itself (`LogStruct.handle_exception`, `Rails.logger.error(error)`, in JSON and in color). A password where the parser stopped was logged unfiltered
+- **Security**: Structs, `ActionController::Parameters` and other objects in a log line are filtered and scrubbed like Hashes, instead of being serialized unfiltered
+- **Fix**: A value that refers back to itself (a Hash or Array containing its own parent) is logged once instead of being expanded up to the depth limit, which took seconds and produced very large lines
+- **ActiveRecord**: Verbose query logs (the "↳ caller" lines) are turned off as intended. The integration set them on `ActiveRecord::Base`, which loaded it early and never matched, because the setting is on the `ActiveRecord` module
+- **Sidekiq**: Lines logged during a Sidekiq job are written once instead of twice (#13)
+- **Sidekiq**: The logger logs a copy of `Sidekiq::Context` instead of the live hash, which could mark successful jobs as failed (#14)
+- **Development**: Colored lines filter named tags as JSON lines filter `ctx`: `logger.tagged(password: "...")` wrote the password. Colored Hash and LogStruct lines are the JSON line, pretty-printed, with the message, tags and named tags it has, and without a blank line after them
+- **Errors**: An `error_reporting_handler` that raises (a timeout calling its service) no longer changes what happens to the error it was reporting: `handle_exception` in the Report modes and the mailers' handlers carry on (or raise the original error), and the handler's error is a `logstruct` error line
+- **Logger settings**: `filter_noisy_loggers` leaves out ActionView's, routing errors' and schema migrations' lines. It kept only those lines and left out every other one
+- **GoodJob**: The GoodJob integration now sets GoodJob's logger and logs GoodJob's thread errors. It referred to its own module instead of `::GoodJob`, so neither was set up before
+- **GoodJob**: LogStruct turns itself on in GoodJob's executable (`good_job start`), like Puma and Sidekiq, and stops GoodJob adding a plain text STDOUT logger beside it, so GoodJob workers log JSON, each line once
+- **GoodJob**: Thread errors are logged whenever the app sets its own handler (`config.good_job.on_thread_error`, which GoodJob applies after LogStruct's setup, or `GoodJob.on_thread_error =` at any time). LogStruct logs the error, then calls the app's handler, which keeps reporting it as before. This needs GoodJob 3.0 or later
+- **GoodJob**: A failed job is logged once. GoodJob passes every unhandled job exception to `on_thread_error`, and the ActiveJob integration has already logged it with the job's ID and class, so the thread error line skips it. Exceptions that aren't `StandardError`s (GoodJob's `handled_exceptions` can let them through) are logged and passed on too; outside production they raised a `TypeError` and skipped the app's handler
+- **GoodJob**: GoodJob's own lines during a job have the job's ID, class, queue, execution count, scheduled time and priority. The logger read them from a thread variable GoodJob never sets
+- **GoodJob**: A thread error is logged once, as LogStruct's structured line. GoodJob's own log subscriber also wrote it as a plain text "GoodJob error: ..." line with the whole backtrace. GoodJob still logs the notifier's connection errors, which it passes to `on_thread_error` only once they repeat, without the backtrace. Loggers an app adds to `GoodJob::LogSubscriber.loggers` no longer get job errors; they're in `GoodJob.logger`'s structured line
+- **GoodJob**: LogStruct's GoodJob logger logs LogStruct log entries and Hashes as they are. It only took strings, so the thread error handler's entry raised a `TypeError`
+- **Output**: STDOUT is written without buffering. Without it, a process whose STDOUT is a pipe (a container) held its lines back until the buffer filled; GoodJob's executable sets `$stdout.sync` only for its own logger, which LogStruct turns off
+- **Sidekiq**: Sidekiq's logger is replaced as soon as Rails' logger is, so lines Sidekiq logs while the app's initializers run (connecting to Redis, setting up a fetcher) are JSON too
+- **Puma**: Removed a debug line (`[logstruct] puma lifecycle init`) that Puma servers wrote to STDERR at boot
+- **ActionMailer**: `log_and_reraise_error`, `log_and_ignore_error` and `log_and_report_error` also work when `ActionMailer::MailDeliveryJob` passes an error to the mailer class. Every email that failed to send in a delivery job raised `NameError: undefined method 'log_and_reraise_error' for class` instead of the real error
+- **ActionMailer**: The mailer logs a failed delivery once. It logged it twice, and a third time when the error reached the mailer class (in `Raise` mode, the default for standard errors, `handle_exception` raised before the error was marked as logged); in `Log` and `Report` modes `handle_exception` logged it again too. A delivery job also has the ActiveJob integration's job error line
+- **ActionMailer**: In `Raise` mode, a `deliver_later` email whose mailer handles the error with `log_and_report_error` fails its job, as `deliver_now` raises. The job succeeded and the error was never reported
+- **Shrine**: Shrine's own log subscriber no longer logs the events LogStruct logs
+- **ActiveModelSerializers**: AMS's own text log subscriber is detached, so renders are logged once
+- **ActiveJob**: Integer `provider_job_id` values (SolidQueue) are logged as strings instead of raising a TypeError (#7, thanks @justinko)
+- **Security**: Strings and Hash keys are written as valid UTF-8. A byte like `\xFF` in a header (Puma passes headers as binary strings) stopped the whole line from being written as JSON and SemanticLogger dropped it, so anyone could hide their request line and the IP spoofing and CSRF lines. A line that still can't be written is replaced by one that says so
+- **Security**: The Referer is logged without its query and fragment (request, blocked host, CSRF and IP spoofing lines, and the context sent to error reporters), which can carry password reset, confirmation and magic link tokens or OAuth codes
+- **Security**: Error reporters (Sentry, Bugsnag, Rollbar, Honeybadger and custom handlers) get the context filtered and scrubbed like a log line
+- **Security**: The request ID, program name and Sidekiq job context are filtered and scrubbed like the rest of the line. They were added after filtering, so an `X-Request-Id` header could carry a card number into the logs
+- **Security**: With `sql_log_bind_params` on, bind values are filtered by their column names, like request params. Only values that looked like tokens were filtered, so a `password` bind was logged, and values were filtered for containing a word like `key`, `auth` or `password` (`monkey`, `Author`, "Forgot password"). Values shaped like tokens still are. Binary values are logged as their size
+- **Security**: `Sidekiq.logger` passes Hashes on as structured data, so their keys are filtered. It turned them into Strings first, so `{password: "hunter2"}` was logged as it was
+- **Security**: Card numbers of 13 to 19 digits are scrubbed (written together, in fours, or in American Express and Diners Club groups), and numbers that fail the Luhn check are left alone. URL passwords are scrubbed whole (`user:pass:word`, `redis://:pw@host`, lowercase `%3a`/`%40`). Hash keys that aren't plain identifiers (an email address as a key), a filtered Hash's `_keys`, Symbol values and GlobalIDs are scrubbed
+- **Security**: The color formatter (development, opt-in) and loggers added with `broadcast_to` get scrubbed messages
+- **Security**: Formatting is bounded (see `config.limits`): one huge value held up the thread that writes every line, and memory grew
+- **CSRF**: CSRF lines are logged. They never appeared: the middleware sat outside ShowExceptions, which turns `InvalidAuthenticityToken` into a 422 first. LogStruct registers a DebugExceptions interceptor instead, and Rails responds as before
+- **Errors**: `logger.error(exception)` writes an error line with the class, message and backtrace (it wrote an empty line), and the message logged beside a Hash (`logger.info("Saved", id: 1)`) is kept. The log methods take SemanticLogger's exception argument, which raised an `ArgumentError`
+- **Errors**: An object whose `as_json` refers back to itself is summarized instead of raising `SystemStackError` from the formatter
+- **Sidekiq**: An ActiveJob job's error is logged by the Sidekiq integration unless the ActiveJob integration logged it. An error raised before ActiveJob ran the job (a job class a deploy removed) was in no line
+- **Puma**: Requiring LogStruct with `server` in `ARGV` and no Puma installed raised `LoadError`, so the app didn't boot
+- **GoodJob**: The app's `on_thread_error` handler runs even if LogStruct's line fails
+- **Ahoy**: Symbol event names and properties that aren't a Hash (they can come from the browser) are logged without raising; a String of properties raised a type error and failed the request in production
+- **TypeScript**: The generated types give `error_class` as a string, not `any`
+- **Gem**: The gem packages only files committed to git
+- **Logging**: `rails server` in development printed every message twice: Rails' console logger wrote it as text beside LogStruct's line. `broadcast_to` ignores a logger that writes to STDOUT or STDERR when SemanticLogger already writes there
+- **Logging**: Loggers added with `broadcast_to` got each `logger << "..."` line twice
+- **Notifications**: `log.logstruct` subscribers get an entry after its line is queued to be written, so a subscriber that raises no longer stops the line being written. Its error is handled like the app's other errors (`standard_errors`, which raises it from the app's log call by default)
+- **ActionMailer**: The entry a mailer's own `rescue_from` handler logs has level `error`, like its line. It said `info`, which `log.logstruct` subscribers saw
+- **ActiveJob**: With the inline adapter, a failed `perform_later` writes one error line. The error that raised out of `perform_later` was logged again
+- **Sorbet**: Type error lines and reports have a backtrace that starts at the call that failed. Method signature errors had none, and the others started inside sorbet-runtime
+- **Rails 8.2**: Apps on Rails main boot with LogStruct. Rails' ActiveStorage, ActionMailer and ActiveRecord log subscribers read `Rails.event` there, and detaching them raised `NoMethodError`
+- **Rails 8.2**: Requests on Rails main log only the request line, without Rails' "Processing by" and "Completed 200 OK" text lines, which lograge detaches only on earlier Rails versions
+- **Filtering**: US phone numbers with a leading `1` (`1-555-123-4567`) are scrubbed whole, SSN-shaped parts of longer numbers (`123-45-6789-0`) are left alone, and an IPv6 address's zone (`fe80::1%en0`) is scrubbed with it
+
 ## [0.1.12] - 2026-01-30
 
 ### Added

data/README.md

@@ -1,95 +1,103 @@
 # LogStruct
 
-Adds secure JSON structured logging to any Rails app (>= 7.1). Simply add the gem to your Gemfile and add an initializer to configure it. By default, your Rails app prints JSON logs to STDOUT (or to the configured destination when `RAILS_LOG_TO_STDOUT` is set). They're easy to search and filter, you can turn them into metrics and alerts, and they're great for building dashboards in CloudWatch, Grafana, or Datadog.
+JSON structured logging for Rails apps (Rails 7.1 or newer, Ruby 3.3 or newer). Add the gem, and in production your
+app writes one JSON line per log entry to STDOUT: requests, jobs, mail, errors, security events and your own logs. The
+lines are easy to search, and you can turn them into metrics, alerts and dashboards in CloudWatch, Grafana, Datadog or
+any other log platform. Sensitive values are filtered and scrubbed before a line is written.
 
-We support all your other favorite gems too, like Sidekiq, Sentry, and Shrine. (And if not, please open a PR!)
+```json
+{"src":"rails","evt":"request","lvl":"info","ts":"2026-10-17T03:09:30.839+00:00","path":"/orders","method":"GET","source_ip":"203.0.113.7","user_agent":"Mozilla/5.0","referer":"https://app.example.com/","format":"html","controller":"OrdersController","action":"index","status":200,"duration_ms":42.1,"view":30.2,"db":8.4,"params":{"page":"2"},"host":"app.example.com","accept":"text/html","prog":"Rails","req_id":"6f1c2b7e-0d4a-4c1e-9a55-3b8e2f7d9c10"}
+```
 
 ## Features
 
-- JSON logging enabled by default for server processes in production and test environments (automatically disabled for console and other Rake tasks)
-- ActionMailer integration for email delivery logging
-- ActiveJob integration for job execution logging
-- Sidekiq integration for background job logging
-- Shrine integration for file upload logging
-- Error handling and reporting
-- Metadata collection for rich context
-- Lograge integration for structured request logging
-- Sensitive data scrubbing for strings (inspired by the Logstop gem)
-- Host authorization logging for security violations
-- Rack middleware for enhanced error logging
-- Type checking with Sorbet and RBS annotations
+- **On where it matters.** In production LogStruct switches on for the processes it recognizes: Puma, Unicorn,
+  Thin, Falcon, `rails server`, a Sidekiq server and `good_job start`. In test it's on for every process except the
+  console. It's off in development, in consoles and in production Rake tasks, unless you set `LOGSTRUCT_ENABLED=true`.
+  See [When LogStruct runs](https://logstruct.com/docs/when-logstruct-runs).
+- **Integrations** for Lograge (request lines), ActionMailer, ActiveJob, Sidekiq, GoodJob, Puma, ActiveStorage,
+  Shrine, CarrierWave, Ahoy, ActiveModelSerializers, Dotenv, host authorization, CSRF and IP spoofing errors, and
+  Sorbet's runtime type errors. SQL query logging is there too, off by default.
+- **Filtering by key name.** Values under keys like `password`, `token`, `api_key`, `authorization` and `cookie` are
+  replaced by a summary of the value, in request params, job arguments and anything else you log. Keys are matched by
+  their words, so `apiKey`, `X-Api-Key` and `HTTP_AUTHORIZATION` are filtered too. The keys in Rails'
+  `config.filter_parameters` are filtered the way Rails filters them.
+- **Scrubbing inside strings** (inspired by [Logstop](https://github.com/ankane/logstop)): email addresses become a
+  keyed hash (`[EMAIL:5f0c1e2a9b3d4e6f]`), so you can follow one person through your logs without logging the
+  address. Card numbers that pass the Luhn check, US phone numbers, SSNs written as 123-45-6789, URL passwords,
+  `Bearer` and `Basic` credentials and sensitive URL query params (`?token=`) are replaced too. IP and MAC address
+  scrubbing are opt-in.
+- **Limits.** Huge values are cut down (100 keys per Hash, 10 items per Array, 20 lines per backtrace, 32 KB per
+  String by default), so one large request body can't slow down your logging.
+- **Error handling.** A mode for each kind of error decides whether it's raised, logged or reported, and whether
+  that differs between development and production. Reports go to Sentry, Bugsnag, Rollbar, Honeybadger or your own
+  reporter.
+- **Sorbet types.** Every built-in line is a typed struct, and you can define your own typed log lines.
+- **STDOUT only in production**, with no log file to rotate or fill the disk.
 
 ## Installation
 
 Add this line to your application's Gemfile:
 
 ```ruby
-gem 'logstruct'
-```
-
-And then execute:
-
-```bash
-$ bundle install
+gem "logstruct"
 ```
 
-Or install it yourself as:
+And then run:
 
 ```bash
-$ gem install logstruct
-```
-
-## Configuration and Initialization
-
-LogStruct is designed to be highly opinionated and work out of the box with minimal configuration.
-
-Please see the [documentation](https://logstruct.com/docs/configuration/) for configuration options.
-
-### Important Notes on Integration
-
-Once initialized (and enabled), the gem automatically includes its modules into the appropriate base classes:
-
-- `ActiveSupport::TaggedLogging` is patched to support both Hashes and Strings (only when LogStruct is enabled)
-- `ActionMailer::Base` includes error handling and event logging modules
-- We configure `Lograge` for request logging
-- Lograge request logs include request metadata (request_id, source_ip, user_agent, referer, host, content_type, accept) and custom fields from `lograge_custom_options`
-- A Rack middleware is inserted to catch and log errors, including security violations (IP spoofing, CSRF, blocked hosts, etc.)
-- Structured logging is set up for ActiveJob, Sidekiq, Shrine, etc.
-- Rails `config.filter_parameters` are merged into LogStruct's filters and then cleared (to avoid double filtering). Configure sensitive keys via `LogStruct.config.filters`.
-- When `RAILS_LOG_TO_STDOUT` is set, we log to STDOUT only. Otherwise, we log to STDOUT by default without adding a file appender to avoid duplicate logs.
-
-### Default behavior by process type
-
-- **Server processes** (`rails server`): JSON logging is enabled by default in production and test environments
-- **Test runs** (`rails test`): JSON logging is enabled by default in test environment to ensure tests catch production bugs
-- **Console** (`rails console`): JSON logging is disabled by default in all environments, providing human-readable logs instead
-- **Other Rake tasks** (`rake db:migrate`, etc.): JSON logging is disabled by default in production, providing human-readable logs instead
-- **Development environment**: Disabled by default for all process types. Enable explicitly via `LOGSTRUCT_ENABLED=true` or `LogStruct.configure { |c| c.enabled = true }`.
-
-When enabled in development, LogStruct defaults to production‑style JSON output so you can preview exactly what will be shipped in prod. You can opt back into the colorful human formatter with:
-
-```ruby
-LogStruct.configure do |c|
-  c.prefer_json_in_development = false
-  c.enable_color_output = true
-end
+bundle install
 ```
 
-To force JSON logs in console or other Rake tasks (e.g., for debugging), set `LOGSTRUCT_ENABLED=true` in your environment.
+That's all you need: there's nothing to generate. To configure LogStruct, add an initializer
+(`config/initializers/logstruct.rb`). The [configuration docs](https://logstruct.com/docs/configuration) list every
+setting.
+
+To see the JSON lines in development, start your server with `LOGSTRUCT_ENABLED=true rails server`.
+
+## What LogStruct changes in your app
+
+When LogStruct is on in a process:
+
+- `Rails.logger` and the framework loggers (ActiveRecord, ActionController, ActionMailer, ActiveJob, ActionView,
+  ActionCable) are replaced by LogStruct's SemanticLogger logger. A logger you set with `config.logger` only applies
+  while LogStruct is off.
+- Lograge is turned on and writes the request lines. Request lines include `req_id`, `source_ip`, `host`,
+  `user_agent`, `referer` (without its query string), `content_type`, `accept` and the fields your
+  `lograge_custom_options` proc adds.
+- Every line logged during a request gets its `req_id`. Tags from `logger.tagged(...)` and `config.log_tags` are
+  written in `tags`.
+- Rails' own text lines for requests, mail and jobs are replaced by LogStruct's JSON lines. An error your app didn't
+  handle is one error line with the request's fields and the response's status, instead of Rails' block of text.
+- Mailers get a `rescue_from StandardError` that logs a failed delivery before re-raising it, and the
+  `log_and_ignore_error`, `log_and_report_error` and `log_and_reraise_error` methods for your own `rescue_from`
+  handlers (those methods exist where LogStruct is off too).
+- CSRF and IP spoofing errors are logged as security lines and reported (`error_handling_modes.security_errors`), and
+  Rails still responds to them as usual. Blocked hosts are logged too.
+- Rails' `filter_parameters` list is left in place, so `request.filtered_parameters`,
+  `ActiveRecord::Base.filter_attributes` and other Rails features keep filtering. Values Rails has already replaced
+  are logged as `[FILTERED]`.
+- Loggers you build yourself with `ActiveSupport::TaggedLogging.new(...)` write LogStruct's JSON lines too.
+- In production LogStruct logs to STDOUT only. In development it logs to STDOUT and Rails' log file, and in test to
+  the log file only. With `RAILS_LOG_TO_STDOUT` set, both log to STDOUT only.
 
 ## Documentation
 
-Please see the [documentation](https://logstruct.com/docs) for more details. (All code examples are type-checked and tested, and it's harder to keep a README up to date.)
-
-### Custom Typed Logs
+The full documentation is at [logstruct.com/docs](https://logstruct.com/docs). Its Ruby examples are tested in this
+repository's test suite.
 
-In addition to the built‑in, strictly typed log structures (Request, Error, SQL, etc.), you can define your own app‑specific typed logs while still using the public `LogStruct.info/error/...` methods.
+- [Getting started](https://logstruct.com/docs/getting-started)
+- [Configuration](https://logstruct.com/docs/configuration)
+- [Filtering sensitive data](https://logstruct.com/docs/filtering-sensitive-data)
+- [Log fields](https://logstruct.com/docs/log-fields)
+- [Custom typed logs](https://logstruct.com/docs/sorbet-types#custom-log-classes)
+- [Upgrading to 0.2.0](https://logstruct.com/docs/upgrading) (many breaking changes)
+- [Troubleshooting](https://logstruct.com/docs/troubleshooting)
+- [Changelog](CHANGELOG.md)
 
-- Compose the public interfaces: include `LogStruct::Log::Interfaces::PublicCommonFields` and the helpers `SerializeCommonPublic` + `MergeAdditionalDataFields` in your `T::Struct`.
-- Fix your `source` to a constant (e.g., return the string `"payments"`), and restrict `event` with a `T::Enum` (e.g., `processed|failed|refunded`).
-- The `LogStruct.info` signature accepts either the internal `CommonFields` (for built‑ins) or your public custom type, so you keep type safety at the call site.
+## Security
 
-See the docs page for a complete example: [Sorbet Types → Custom Typed Logs](https://logstruct.com/docs/sorbet-types#custom-log-classes).
+To report a vulnerability, see [SECURITY.md](SECURITY.md).
 
 ## License