data_redactor 0.11.0-x86_64-linux → 0.14.0-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 79d2dc2a7a24145626e06741bdec238660b21d2cc2268e824878238e62d8331b
-  data.tar.gz: 8e15d8faddbf76cf4d339efe14b88d401e91b4f67ed911682a7dad75b29ca384
+  metadata.gz: 86ed390986b76cb2bcb48a80c322be42b8bd91dcd5b9afb8299a6ab4c0dd9419
+  data.tar.gz: ca7260eff77b03d15392b0c763e56ee7b5bf146eb7f0b7597f78f41d2a6dfe08
 SHA512:
-  metadata.gz: 37103c8e31cc7271ba2bc0903ceb90fc09d3e2f39441264249e3adb16ff40ed3ffcc713f31f78134fd2f7df3f1a635bfc01e625d52c0a978b405793ec3d18082
-  data.tar.gz: 4aa60db835020332e65f1a24d93b584143a203695469a9a94b9c3974f12803146d8778dde4e62ce0d020f740417255c3385da120ccec64aa489cce7be21c5c34
+  metadata.gz: a683277146fbb1e19caefffe829a9de090d48b181f0d15cc98a625509f47da1a7cf4f7f69df165f4b077a321f2ff1ee125ef4881e5746949b62f6db87269e022
+  data.tar.gz: ecb6d0acfcf1a9b9c2c79f1d0e016ae4ad13d0c938e31c9d97f0b3bdc8f323ac7c7c1d1a2828067b3a5b9f33895e0e9bc3bb6a69bd9c7f324e2055cdeb082ee8

data/CHANGELOG.md

@@ -7,6 +7,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.14.0] - 2026-06-17
+
+### Added
+- **Key-name-anchored secret redaction** (`:credentials`). A new pattern tier
+  redacts a secret by the *name of the field it is assigned to*, for values with
+  no distinctive shape of their own — the primary case being an `.env` file or
+  config blob passed through the redactor. Anchored on the key words `password`,
+  `passwd`, `pwd`, `secret`, `token`, `api_key`, `apikey`, `access_key`, and
+  `client_secret` (case-insensitive), followed by `=` or `:` (dotenv and YAML
+  styles), with quoted (`"..."`/`'...'`) or unquoted (≥6 chars) values. Only the
+  **value** is redacted; the key is kept so logs stay greppable
+  (`PASSWORD=[REDACTED]`). Compound key names match whether the secret word is a
+  prefix or suffix segment (`POSTGRES_DB_PASSWORD=`, `PASSWORD_POSTGRES=`).
+  Requires the assignment separator, so the word in prose ("reset your password")
+  is not a false positive.
+- `examples/` directory with runnable, copy-pasteable usage scripts for every
+  feature (core redaction, scan/dry-run, custom patterns, deep/JSON traversal,
+  and the Logger / Rack / Rails / LLM integrations). Repo-only — not packaged in
+  the gem. Linked from the README.
+
+## [0.13.0] - 2026-06-13
+
+### Changed
+- **Custom-pattern registration is now thread-safe.** `add_pattern`,
+  `remove_pattern`, and `clear_custom_patterns!` are guarded by a mutex shared
+  with the `redact`/`scan` custom-pattern loop, so patterns may be registered,
+  removed, or cleared from any thread at any time — including at runtime from a
+  request handler — without coordinating with in-flight redactions. The previous
+  "register custom patterns at boot only" caveat is lifted. (The C extension now
+  links `-lpthread` on glibc; no-op on musl and macOS where pthread is in libc.)
+- **`redact` releases the GVL for large inputs.** The v19 engine's per-scan
+  mutable state (NFA scratch and the lazy DFA cache) moved into per-thread
+  storage, making the engine re-entrant. `redact` now releases the GVL
+  (`rb_thread_call_without_gvl`) around the built-in scan for inputs above a few
+  KB, so a large redaction on one thread no longer blocks other Ruby threads.
+  Small inputs keep the GVL. No public API change; output is byte-for-byte
+  identical (verified by a differential gate over ~6000 inputs). The per-thread
+  DFA cache's allocation floor was tuned so this adds ~0.86 MB per scanning
+  thread (down from a naive ~3.2 MB), with no throughput change. Per-thread scan
+  state is freed at thread exit (via a `pthread_key` destructor), so processes
+  that churn many short-lived scanning threads do not accumulate dead caches —
+  RSS stays flat across thousands of threads.
+
 ## [0.11.0] - 2026-06-10
 
 ### Added
@@ -232,7 +275,9 @@ features as 0.7.1 plus the pipeline fix.
 - `DataRedactor.redact(text)` module function returning the input with every match replaced by `[REDACTED]`.
 - RSpec suite with one example per pattern.
 
-[Unreleased]: https://github.com/danielefrisanco/data_redactor/compare/v0.11.0...HEAD
+[Unreleased]: https://github.com/danielefrisanco/data_redactor/compare/v0.14.0...HEAD
+[0.14.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.13.0...v0.14.0
+[0.13.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.11.0...v0.13.0
 [0.11.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.10.1...v0.11.0
 [0.10.1]: https://github.com/danielefrisanco/data_redactor/compare/v0.10.0...v0.10.1
 [0.10.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.9.0...v0.10.0

data/README.md

@@ -19,7 +19,7 @@ It ships **88 built-in patterns** across 15+ countries, grouped into tags
 (`:credentials`, `:financial`, `:contact`, ...) so you can redact only what you
 care about. Beyond plain strings it can walk nested Hashes, Arrays, and JSON,
 audit a payload without mutating it (`scan`), and plug into Logger, Rails, and
-Rack. You can also register your own patterns at boot.
+Rack. You can also register your own patterns — at boot or at runtime from any thread.
 
 ### Use cases
 
@@ -46,6 +46,12 @@ DataRedactor.redact(text)
 # => "User CF is [REDACTED] and key is [REDACTED]"
 ```
 
+Prefer runnable code? The [`examples/`](examples/) directory has self-contained,
+copy-pasteable scripts for every feature below — core redaction, scan/dry-run,
+custom patterns, deep/JSON traversal, and the Logger / Rack / Rails / LLM
+integrations. Run any of them with `bundle exec ruby examples/<name>.rb` (see
+[examples/README.md](examples/README.md)).
+
 ### Filtering by tag or pattern name
 
 `only:` and `except:` both accept a single value or an Array, mixing **Symbols** (tag names) and **Strings** (specific pattern names).
@@ -161,7 +167,7 @@ DataRedactor.redact_json("not json")  # => JSON::ParserError
 
 ### Custom patterns
 
-Teams often have internal IDs that the gem can't ship. Register them at boot:
+Teams often have internal IDs that the gem can't ship. Register them at boot — or at runtime from any thread (registration is thread-safe, see [Thread safety](#thread-safety)):
 
 ```ruby
 # String (POSIX ERE) or Regexp — both accepted
@@ -415,6 +421,16 @@ redactor/
 │       └── tags.h                # TAG_* bit constants
 ├── spec/
 │   └── data_redactor_spec.rb     # RSpec tests — at least one example per pattern, plus filter / placeholder / custom-pattern coverage
+├── examples/                     # Repo-only runnable usage scripts (not packaged in the gem)
+│   ├── README.md                 # Index + how to run
+│   ├── basic_redact.rb           # redact, tag filters, placeholder modes
+│   ├── scan_report.rb            # scan dry-run with byte offsets
+│   ├── custom_pattern.rb         # add_pattern + name_pattern
+│   ├── deep_and_json.rb          # redact_deep / redact_json
+│   ├── logger.rb                 # Logger::Formatter integration
+│   ├── rack_middleware.rb        # Rack middleware (body + headers)
+│   ├── rails_filter.rb           # filter_parameters adapter
+│   └── llm_payload.rb            # Claude / OpenAI message + response redaction
 ├── benchmark/                    # Repo-only perf scripts (not packaged in the gem)
 │   ├── README.md                 # How to run, what each script measures
 │   ├── support/corpus.rb         # Shared payload builders + pure-Ruby baseline redactor
@@ -571,9 +587,9 @@ All C-side buffers are heap-allocated with `malloc`/`strdup` and freed before th
 
 ## Thread safety
 
-`DataRedactor.redact` and `DataRedactor.scan` are safe to call concurrently from multiple threads. The v19 engine holds MRI's GVL for the duration of each call (no `rb_thread_call_without_gvl`), so concurrent calls are serialised by the GVL. Each call allocates its own working buffers; built-in engine state is read-only after `mm_init()` at load time.
+`DataRedactor.redact` and `DataRedactor.scan` are safe to call concurrently from multiple threads. The v19 engine keeps its compiled patterns immutable and shared (read-only after `mm_init()` at load time) and all per-scan mutable state — NFA scratch and the lazy DFA cache — in per-thread storage, so concurrent scans never touch each other's state. For inputs above a few KB, `redact` **releases the GVL** (`rb_thread_call_without_gvl`) around the built-in scan, so a large redaction on one thread no longer blocks other Ruby threads from running. Small inputs keep the GVL (the release bookkeeping would cost more than the scan). Each call allocates its own working buffers. A thread's per-thread state is freed automatically when the thread exits, so processes that spawn many short-lived scanning threads do not accumulate memory.
 
-`DataRedactor.add_pattern`, `remove_pattern`, and `clear_custom_patterns!` mutate a shared dynamic array and are **not** thread-safe. Register custom patterns once at boot — before spawning worker threads or forking — and they will be visible (read-only) to every subsequent `redact`/`scan` call.
+`DataRedactor.add_pattern`, `remove_pattern`, and `clear_custom_patterns!` are also thread-safe: the shared custom-pattern array is guarded by a mutex that writers take around the mutation and `redact`/`scan` take around their custom-pattern loop. You can register, remove, or clear custom patterns from any thread at any time — including from request handlers in a running server — without coordinating with in-flight redactions. (Registration is still a rare operation; the lock is uncontended in practice.)
 
 ## Versioning
 

data/lib/data_redactor/version.rb

@@ -1,4 +1,4 @@
 module DataRedactor
   # Current gem version. Follows {https://semver.org Semantic Versioning 2.0.0}.
-  VERSION = "0.11.0"
+  VERSION = "0.14.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: data_redactor
 version: !ruby/object:Gem::Version
-  version: 0.11.0
+  version: 0.14.0
 platform: x86_64-linux
 authors:
 - Daniele Frisanco