data_redactor 0.7.2-x86_64-linux

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 308a3387249fae55cdff4392a2655072450589df3a07409a3623156eb9c1c4fd
+  data.tar.gz: 2cbdf4a55c7648ae74245b11c9ee298eef10116e154f2fb9e27926546c909182
+SHA512:
+  metadata.gz: 2e4a8c4ce276ccbd236e023eca96640b293a644ed3b0c4c2a8d844f0151f0b9befb18bdbd2221ea4ca4524d8dc28e376651b742eb9d6488fcdbb71a267672f95
+  data.tar.gz: 7079f9aeb48c18264a2d74b4d5c50383f3b38c0227b39cb336fab94dbac28db4147bb8963d7d6f2d5a1e04813c5e7ef71167ee906f0bd3eaad0613be39b3d882

data/CHANGELOG.md

@@ -0,0 +1,171 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.7.2] - 2026-05-09
+
+**Supersedes 0.7.1, which has been yanked from RubyGems.**
+
+0.7.1 had a release pipeline bug: the source gem and the precompiled native
+gems were published by two independent workflows, with no gating between
+them. When the native-binary builds failed (`oxidize-rb/actions/cross-gem`
+couldn't pull `rbsys/aarch64-linux:0.9.128` from Docker Hub), the source
+gem still published — leaving users with release notes that promised
+precompiled binaries that didn't exist on RubyGems. 0.7.2 ships the same
+features as 0.7.1 plus the pipeline fix.
+
+### Changed
+- **Atomic release pipeline.** Source-gem publishing moved out of `ci.yml`
+  and into `release-binaries.yml`, alongside the native-gem builds. The
+  publish job now `needs: [build-source, build-native]`; if any native
+  platform fails to build, **nothing publishes**. This guarantees the
+  RubyGems release matches what the GitHub release notes promise.
+- **Direct `rake-compiler-dock` invocation in CI** instead of the
+  `oxidize-rb/actions/cross-gem` action. Same code path as `rake gem:all`
+  locally and the existing PR-time smoke test in `ci.yml`. Uses
+  `ghcr.io/rake-compiler/*` images (no Docker Hub rate limits).
+
+### Fixed
+- All 6 precompiled native gems now actually publish on release — the
+  `aarch64-linux` variant in particular was previously failing.
+
+### Documentation
+- README installation section rewritten around the user's question
+  ("what changes for me?"). Adds explicit Docker / Alpine guidance and a
+  heads-up about `bundle lock --add-platform` for cross-platform deploys.
+
+## [0.7.1] - 2026-05-09 [YANKED]
+
+### Added
+- **Precompiled native gems** for the most common platforms — installing
+  `data_redactor` no longer requires a C toolchain on these targets:
+  - `x86_64-linux`, `aarch64-linux` (glibc)
+  - `x86_64-linux-musl`, `aarch64-linux-musl` (Alpine)
+  - `x86_64-darwin`, `arm64-darwin` (macOS Intel + Apple Silicon)
+  Each native gem ships compiled `.so` files for Ruby 3.1, 3.2, 3.3, and 3.4.
+  Bundler/RubyGems automatically picks the right gem for the host; users on
+  any other platform fall back to the source gem and compile as before.
+- `rake gem:all` task — builds every native gem locally via `rake-compiler-dock`
+  (requires Docker). Single command to regenerate the full release matrix.
+- `.github/workflows/release-binaries.yml` — builds & publishes all native
+  gems on every GitHub release. Also exposes `workflow_dispatch` so a
+  maintainer can rebuild any past release without cutting a new tag.
+
+### Changed
+- CI test matrix now includes Ruby 3.4 in addition to 3.1, 3.2, 3.3.
+- Gemspec: added `rake-compiler-dock` as a development dependency. Source-only
+  gem size is unchanged — native gems strip `ext/` and the `extconf.rb`
+  extension hook so they only carry the prebuilt `.so` files.
+
+## [0.7.0] - 2026-05-08
+
+### Added
+- **Rails / Rack / Logger integrations** under `lib/data_redactor/integrations/`. Soft-required — none are loaded by default; the gem still has zero runtime dependencies in the gemspec.
+  - `DataRedactor::Integrations::Logger` — drop-in `Logger::Formatter` that scrubs every emitted line, wraps an inner formatter (default `Logger::Formatter`), and preserves exception cause chains.
+  - `DataRedactor::Integrations::Rails.filter(...)` — returns a `(key, value)` proc for `Rails.application.config.filter_parameters`. Mutates String values in place via `String#replace`.
+  - `DataRedactor::Integrations::Rack` — middleware with selectable surfaces. `scrub:` accepts any subset of `[:body, :headers]` (default both). `:body` buffers the response and drops `Content-Length`; `:headers` scrubs sensitive response headers (`Set-Cookie`, `Authorization`, `X-Api-Key`, ...) and request headers in the env hash. Unknown surfaces raise `ArgumentError`.
+- All three integrations forward `only:`, `except:`, `placeholder:` to `DataRedactor.redact`.
+
+### Changed
+- Gemspec: added `rack` as a development dependency. No new runtime dependencies.
+
+## [0.6.1] - 2026-05-08
+
+### Added
+- Six new distinctive-prefix API key patterns under the `:credentials` tag, exposed via `DataRedactor.pattern_names`:
+  - `anthropic_api_key` — `sk-ant-apiNN-...`
+  - `openai_project_api_key` — `sk-proj-...`
+  - `gitlab_pat` — `glpat-...`
+  - `digitalocean_pat` — `dop_v1_...`
+  - `databricks_api_token` — `dapi...`
+  - `sentry_dsn` — `https://KEY@oNNN.ingest.sentry.io/PID` (also matches the legacy `KEY:SECRET@` form)
+
+### Changed
+- `NUM_PATTERNS` is now 85 (was 79). Built-in pattern indices in C have shifted accordingly; the public Ruby API and pattern names are stable.
+
+## [0.6.0] - 2026-05-08
+
+### Added
+- **Per-pattern allow / deny via `only:` / `except:`.** Both kwargs now accept a mix of Symbols (tags) and Strings (pattern names from `DataRedactor.pattern_names`). They can be combined: `only: :contact, except: ["email"]` redacts every contact pattern except email. Mixed-list shapes like `only: [:credentials, "iban_de"]` also work. Precedence: `except:` always wins when the two overlap.
+- `DataRedactor.pattern_names` — array of every known pattern name (built-ins + currently registered custom).
+- `DataRedactor::BUILTIN_PATTERN_NAMES` and `DataRedactor::BUILTIN_PATTERN_TAG_BITS` constants (frozen) exposing the compiled-in pattern roster.
+- `DataRedactor::UnknownPatternError` raised when a String passed to `only:`/`except:` does not match any known pattern.
+- YARD docs deploy job in `.github/workflows/ci.yml` publishes `bundle exec yard doc` output to GitHub Pages on every push to `main`.
+
+### Changed
+- **C entry-point signatures.** `_redact(text, ph_mode, ph_str, enable_bits)` and `_scan(text, enable_bits)` now take a per-pattern enable bit array (built by the Ruby wrapper from `only:`/`except:`) instead of a tag bitmask. The public `DataRedactor.redact` / `.scan` API is fully backward compatible — only the underscore-prefixed C boundary changed. Single-pass: filtering happens in C, no second pass through `_scan`.
+- `only:` and `except:` may now be combined (previously raised `ArgumentError` if both were passed).
+- **Internal: C extension split into focused modules.** `ext/data_redactor/data_redactor.c` was a single ~1000-line file; it is now a 60-line entry point plus `patterns.{c,h}`, `placeholder.{c,h}`, `redact.{c,h}`, `scan.{c,h}`, `custom_patterns.{c,h}`, and `tags.h`. `extconf.rb` now globs every `.c` in the extension directory via `$srcs`, so adding a new module needs no Makefile edits.
+- **YARD inline docs** — every public method on `DataRedactor` now has `@param`/`@return`/`@raise` annotations (100% coverage); `.yardopts` configures markdown rendering with the README as the front page.
+
+### Documentation
+- README: gem version / CI / license badges; new "Thread safety" section clarifying that `redact`/`scan` are thread-safe but `add_pattern`/`remove_pattern`/`clear_custom_patterns!` are not (register custom patterns once at boot).
+
+## [0.5.0] - 2026-05-02
+
+### Added
+- `DataRedactor.scan(text, only:, except:)` — returns `{ redacted: String, matches: Array<Hash> }` where each match contains `:tag` (Symbol), `:name` (pattern name String), `:value` (matched text), `:start` (byte offset into original), `:length` (byte length). Accepts the same `only:`/`except:` tag filters as `redact`. Includes both built-in and custom pattern matches.
+- `pattern_names[]` array in the C extension mapping each built-in pattern index to a stable snake_case name string (e.g. `"aws_access_key_id"`, `"email"`, `"iban_de"`).
+
+## [0.4.0] - 2026-05-02
+
+### Added
+- `placeholder:` keyword argument on `DataRedactor.redact`.
+  - Plain string (default `"[REDACTED]"`): `placeholder: "***"`
+  - Tagged: `placeholder: :tagged` → `[REDACTED:CONTACT]`, `[REDACTED:CREDENTIALS]`, etc.
+  - Deterministic hash: `placeholder: :hash` → `[CONTACT_a3f9]` (4-hex djb2 suffix, same value always produces the same token — useful for correlating redactions across log lines).
+- `PH_MODE_PLAIN`, `PH_MODE_TAGGED`, `PH_MODE_HASH` integer constants exposed from C.
+- `DataRedactor::PLACEHOLDER_DEFAULT` constant (`"[REDACTED]"`).
+
+### Changed
+- `DataRedactor._redact` now takes 4 arguments: `(text, mask, ph_mode, ph_str)`. The public `DataRedactor.redact` API is fully backward compatible.
+
+## [0.3.0] - 2026-05-02
+
+### Added
+- User-supplied custom patterns via `DataRedactor.add_pattern(name:, regex:, tag: :custom, boundary: false)`.
+- `DataRedactor.remove_pattern(name)` — remove a named custom pattern (returns `true`/`false`).
+- `DataRedactor.custom_patterns` — list all registered custom patterns as an array of hashes.
+- `DataRedactor.clear_custom_patterns!` — remove all custom patterns (useful in test suites).
+- New `:custom` tag and `TAG_CUSTOM` bitmask constant for custom patterns. Works with `only:`/`except:`.
+- `DataRedactor::InvalidPatternError` raised when a pattern fails `regcomp` or uses unsupported Ruby-only syntax (`\d`, `\s`, `\w`, `\b`, lookaround, non-greedy quantifiers, named groups).
+- Capture groups rejected at registration when `boundary: true` (group indices would shift).
+- Name collisions replace the existing pattern (the old compiled `regex_t` is freed).
+
+## [0.2.0] - 2026-05-02
+
+### Added
+- Tag system: every pattern now belongs to one of 8 tags (`:credentials`, `:financial`, `:tax_id`, `:national_id`, `:contact`, `:network`, `:travel`, `:other`).
+- `DataRedactor.redact(text, only: [...])` to redact only patterns in the given tags.
+- `DataRedactor.redact(text, except: [...])` to redact every tag except the given ones.
+- `DataRedactor.tags` returning the list of supported tags.
+- `DataRedactor::TAGS` constant mapping tag symbols to bitmask values, plus `TAG_*` integer constants exposed from C for advanced use.
+- `DataRedactor::UnknownTagError` raised when an unknown tag symbol is passed.
+
+### Changed
+- The C-level entry point is now `DataRedactor._redact(text, mask)` (two-arg, mask is an integer bitmask). The public API is the Ruby wrapper `DataRedactor.redact`, which remains backward compatible: `redact(text)` with no keyword arguments runs every pattern exactly as before.
+
+## [0.1.0] - 2026-05-02
+
+### Added
+- Initial release.
+- C extension (`ext/data_redactor/data_redactor.c`) using POSIX `regex.h` for high-throughput scanning.
+- 79 redaction patterns across cloud secrets, API keys, IBANs, national IDs, and PII for 15+ countries.
+- Patterns ordered most-specific to most-generic to prevent shorter patterns from consuming parts of longer matches.
+- Boundary-wrapping mechanism for generic digit/alphanum sequences so they only match at word boundaries.
+- `DataRedactor.redact(text)` module function returning the input with every match replaced by `[REDACTED]`.
+- RSpec suite with one example per pattern.
+
+[Unreleased]: https://github.com/danielefrisanco/data_redactor/compare/v0.7.2...HEAD
+[0.7.2]: https://github.com/danielefrisanco/data_redactor/compare/v0.7.1...v0.7.2
+[0.7.1]: https://github.com/danielefrisanco/data_redactor/compare/v0.7.0...v0.7.1
+[0.7.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.6.1...v0.7.0
+[0.6.1]: https://github.com/danielefrisanco/data_redactor/compare/v0.6.0...v0.6.1
+[0.6.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.5.0...v0.6.0
+[0.2.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/danielefrisanco/data_redactor/releases/tag/v0.1.0

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Daniele Frisanco
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/lib/data_redactor/integrations/logger.rb

@@ -0,0 +1,42 @@
+require "logger"
+require "data_redactor"
+
+module DataRedactor
+  module Integrations
+    # Logger formatter that runs every log message through {DataRedactor.redact}
+    # before delegating to an inner formatter.
+    #
+    # @example Drop-in replacement for Ruby's default formatter
+    #   logger = Logger.new($stdout)
+    #   logger.formatter = DataRedactor::Integrations::Logger.new
+    #   logger.info("Auth failed for user alice@example.com")
+    #   # => "I, [...] -- : Auth failed for user [REDACTED]"
+    #
+    # @example Wrapping an existing formatter (e.g. Rails JSON logger)
+    #   logger.formatter = DataRedactor::Integrations::Logger.new(
+    #     inner: Rails.logger.formatter,
+    #     only:  [:credentials, :contact]
+    #   )
+    class Logger
+      # @param inner [#call, nil] formatter to wrap. Defaults to {::Logger::Formatter}.
+      # @param only [Symbol, String, Array, nil] forwarded to {DataRedactor.redact}.
+      # @param except [Symbol, String, Array, nil] forwarded to {DataRedactor.redact}.
+      # @param placeholder forwarded to {DataRedactor.redact}.
+      def initialize(inner: ::Logger::Formatter.new, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        @inner = inner
+        @only = only
+        @except = except
+        @placeholder = placeholder
+      end
+
+      # Formatter contract — called by Logger for every emitted line.
+      # Lets the inner formatter render whatever it likes (string, exception,
+      # arbitrary object) and scrubs the resulting line in one pass. Keeps the
+      # exception cause chain intact so downstream formatters still see it.
+      def call(severity, time, progname, msg)
+        line = @inner.call(severity, time, progname, msg)
+        DataRedactor.redact(line.to_s, only: @only, except: @except, placeholder: @placeholder)
+      end
+    end
+  end
+end

data/lib/data_redactor/integrations/rack.rb

@@ -0,0 +1,121 @@
+require "data_redactor"
+
+module DataRedactor
+  module Integrations
+    # Rack middleware that scrubs sensitive data from selectable surfaces of
+    # the response (and request headers, for downstream loggers to see scrubbed
+    # values).
+    #
+    # @example Both surfaces (default)
+    #   use DataRedactor::Integrations::Rack, scrub: [:body, :headers]
+    #
+    # @example Headers only — leave the response body untouched
+    #   use DataRedactor::Integrations::Rack, scrub: [:headers]
+    #
+    # ### Surfaces
+    #
+    # - `:body` — wraps the response body so emitted bytes pass through
+    #   {DataRedactor.redact} before reaching the client. Drops the
+    #   `Content-Length` header (the redacted body may have a different
+    #   byte length, and recomputing requires buffering).
+    # - `:headers` — scrubs response headers in place. Sensitive request
+    #   headers (`Authorization`, `Cookie`, `X-Api-Key`, etc.) are redacted in
+    #   the env hash so any downstream middleware that logs them sees scrubbed
+    #   values.
+    class Rack
+      DEFAULT_SCRUB = [:body, :headers].freeze
+
+      SENSITIVE_REQUEST_HEADERS = %w[
+        HTTP_AUTHORIZATION
+        HTTP_PROXY_AUTHORIZATION
+        HTTP_COOKIE
+        HTTP_X_API_KEY
+        HTTP_X_AUTH_TOKEN
+        HTTP_X_ACCESS_TOKEN
+      ].freeze
+
+      SENSITIVE_RESPONSE_HEADERS = %w[
+        Set-Cookie
+        Authorization
+        X-Api-Key
+        X-Auth-Token
+        X-Access-Token
+      ].freeze
+
+      # @param app [#call] the Rack app
+      # @param scrub [Array<Symbol>] which surfaces to redact. Subset of
+      #   `[:body, :headers]`. Defaults to `[:body, :headers]`.
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      def initialize(app, scrub: DEFAULT_SCRUB, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        @app = app
+        @scrub = Array(scrub).map(&:to_sym)
+        unknown = @scrub - [:body, :headers]
+        unless unknown.empty?
+          raise ArgumentError, "unknown scrub surface(s) #{unknown.inspect}; valid: [:body, :headers]"
+        end
+        @only = only
+        @except = except
+        @placeholder = placeholder
+      end
+
+      def call(env)
+        scrub_request_headers(env) if @scrub.include?(:headers)
+        status, headers, body = @app.call(env)
+        headers = scrub_response_headers(headers) if @scrub.include?(:headers)
+        if @scrub.include?(:body)
+          body, headers = wrap_body(body, headers)
+        end
+        [status, headers, body]
+      end
+
+      private
+
+      def redact(s)
+        DataRedactor.redact(s, only: @only, except: @except, placeholder: @placeholder)
+      end
+
+      def scrub_request_headers(env)
+        SENSITIVE_REQUEST_HEADERS.each do |key|
+          value = env[key]
+          env[key] = redact(value) if value.is_a?(String) && !value.empty?
+        end
+      end
+
+      def scrub_response_headers(headers)
+        # Rack 3 uses lower-case header names; Rack 2 uses Capitalized.
+        # Match case-insensitively against our known list.
+        sensitive_lc = SENSITIVE_RESPONSE_HEADERS.map(&:downcase)
+        headers.each_with_object({}) do |(key, value), out|
+          if sensitive_lc.include?(key.to_s.downcase)
+            out[key] = scrub_header_value(value)
+          else
+            out[key] = value
+          end
+        end
+      end
+
+      def scrub_header_value(value)
+        case value
+        when String then redact(value)
+        when Array  then value.map { |v| v.is_a?(String) ? redact(v) : v }
+        else value
+        end
+      end
+
+      def wrap_body(body, headers)
+        # Buffer the body, redact, return as a single-element array.
+        # Stripping Content-Length because the redacted body may differ in
+        # byte length; downstream servers will recompute or chunk-encode.
+        buffered = +""
+        body.each { |chunk| buffered << chunk.to_s }
+        body.close if body.respond_to?(:close)
+
+        scrubbed = redact(buffered)
+        new_headers = headers.reject { |k, _| k.to_s.downcase == "content-length" }
+        [[scrubbed], new_headers]
+      end
+    end
+  end
+end

data/lib/data_redactor/integrations/rails.rb

@@ -0,0 +1,38 @@
+require "data_redactor"
+
+module DataRedactor
+  module Integrations
+    # Rails `config.filter_parameters` adapter. Returns a `Proc` that Rails
+    # invokes with `(key, value)` for every leaf in the params tree; we redact
+    # the value in place when it is a String.
+    #
+    # @example
+    #   # config/initializers/filter_parameter_logging.rb
+    #   require "data_redactor/integrations/rails"
+    #   Rails.application.config.filter_parameters += [
+    #     DataRedactor::Integrations::Rails.filter
+    #   ]
+    #
+    # @example Restricting to specific tags
+    #   Rails.application.config.filter_parameters += [
+    #     DataRedactor::Integrations::Rails.filter(only: [:credentials, :financial])
+    #   ]
+    module Rails
+      module_function
+
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      # @return [Proc] a `(key, value)` proc compatible with `config.filter_parameters`
+      def filter(only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        lambda do |_key, value|
+          next unless value.is_a?(String)
+          # Rails' Parameter Filter mutates the value in place. We can't
+          # reassign `value` here, so use String#replace.
+          redacted = DataRedactor.redact(value, only: only, except: except, placeholder: placeholder)
+          value.replace(redacted) if redacted != value
+        end
+      end
+    end
+  end
+end

data/lib/data_redactor/version.rb

@@ -0,0 +1,4 @@
+module DataRedactor
+  # Current gem version. Follows {https://semver.org Semantic Versioning 2.0.0}.
+  VERSION = "0.7.2"
+end

data/lib/data_redactor.rb

@@ -0,0 +1,347 @@
+require "set"
+require_relative "data_redactor/version"
+require_relative "data_redactor/data_redactor" # loads the compiled .so
+
+# High-performance regex-based redactor for sensitive data.
+#
+# DataRedactor scans text for sensitive patterns (API keys, IBANs, national
+# IDs, emails, phone numbers, etc.) and replaces matches with a configurable
+# placeholder. The matching is done by a C extension backed by POSIX
+# +regex.h+, so it is fast enough to run inline on large payloads.
+#
+# @example Basic redaction
+#   DataRedactor.redact("key is AKIAIOSFODNN7EXAMPLE")
+#   # => "key is [REDACTED]"
+#
+# @example Filter by tag or pattern name
+#   DataRedactor.redact(text, only: :credentials)
+#   DataRedactor.redact(text, except: [:contact, :network])
+#   DataRedactor.redact(text, only: :contact, except: ["email"])
+#   DataRedactor.redact(text, only: ["aws_access_key_id"])
+#
+# @example Custom placeholder
+#   DataRedactor.redact(text, placeholder: "***")
+#   DataRedactor.redact(text, placeholder: :tagged) # => "[REDACTED:CONTACT]"
+#   DataRedactor.redact(text, placeholder: :hash)   # => "[CONTACT_a3f9]"
+#
+# @example Audit / dry-run
+#   DataRedactor.scan(text)
+#   # => { redacted: "...", matches: [{tag:, name:, value:, start:, length:}, ...] }
+#
+# @example Custom pattern
+#   DataRedactor.add_pattern(name: "employee_id", regex: "EMP-[0-9]{6}")
+module DataRedactor
+  # Map of tag symbol to the integer bit used by the C layer.
+  #
+  # The keys of this hash are the canonical list of supported tags; pass any
+  # of them to {redact} or {scan} via +only:+ / +except:+.
+  #
+  # @return [Hash{Symbol => Integer}] frozen tag-to-bit map
+  TAGS = {
+    credentials: TAG_CREDENTIALS,
+    financial:   TAG_FINANCIAL,
+    tax_id:      TAG_TAX_ID,
+    national_id: TAG_NATIONAL_ID,
+    contact:     TAG_CONTACT,
+    network:     TAG_NETWORK,
+    travel:      TAG_TRAVEL,
+    other:       TAG_OTHER,
+    custom:      TAG_CUSTOM
+  }.freeze
+
+  # Raised when a tag symbol passed to +only:+ / +except:+ / +tag:+ is not in {TAGS}.
+  class UnknownTagError     < ArgumentError; end
+
+  # Raised when a String passed via +only:+ / +except:+ does not match any
+  # registered pattern name. See {pattern_names}.
+  class UnknownPatternError < ArgumentError; end
+
+  # Raised by {add_pattern} when the supplied regex is not valid POSIX ERE,
+  # uses Ruby-only syntax (+\d+, +\s+, lookaround, non-greedy, etc.), or
+  # contains capture groups while +boundary: true+ is requested.
+  class InvalidPatternError < ArgumentError; end
+
+  # @api private
+  # Capture groups break boundary-wrapper group index assumptions ([1],[2],[3] shift).
+  CAPTURE_GROUP_RE = /(?<!\\)\((?!\?:)/.freeze
+
+  # @api private
+  # Ruby regex syntax that has no POSIX ERE equivalent.
+  RUBY_ONLY_SYNTAX_RE = /\\[dDwWsShHbB]|\(\?[<!=]|\(\?<[a-zA-Z]|\(\?[imx]|[*+?]\?/.freeze
+
+  # Default placeholder used when +placeholder:+ is not given to {redact}.
+  PLACEHOLDER_DEFAULT = "[REDACTED]"
+
+  module_function
+
+  # List of supported tag symbols.
+  #
+  # @return [Array<Symbol>] every key from {TAGS}
+  def tags
+    TAGS.keys
+  end
+
+  # List of every pattern name the redactor knows about.
+  #
+  # Includes the {BUILTIN_PATTERN_NAMES} plus any names registered via
+  # {add_pattern}. Useful for discovering what String values +only:+ /
+  # +except:+ accept, and for filtering / debugging.
+  #
+  # @return [Array<String>] built-in names first (in execution order),
+  #   then custom names in registration order.
+  def pattern_names
+    BUILTIN_PATTERN_NAMES + _custom_patterns.map { |h| h[:name] }
+  end
+
+  # Redact every match of the configured patterns in +text+.
+  #
+  # +only:+ and +except:+ both accept a single value or an Array, mixing:
+  # - **Symbols** — tag names from {TAGS} (e.g. +:contact+, +:credentials+).
+  # - **Strings** — specific pattern names from {pattern_names} (e.g. +"email"+).
+  #
+  # They can be combined: +only: :contact, except: ["email"]+ means
+  # "redact every contact pattern except email." Symbols give you tag-level
+  # control; Strings give you per-pattern precision.
+  #
+  # **Precedence:** a pattern is redacted iff
+  # +(only is nil OR pattern matches only:)+ AND +(pattern does not match except:)+.
+  # +except:+ always wins over +only:+ when they overlap — e.g.
+  # +only: :contact, except: :contact+ produces an empty redaction (no-op),
+  # and +only: ["email"], except: ["email"]+ likewise skips email entirely.
+  #
+  # @param text [String] input string. Returned unchanged if no patterns match.
+  # @param only [Symbol, String, Array, nil] include only the given tag(s)
+  #   and/or pattern name(s).
+  # @param except [Symbol, String, Array, nil] exclude the given tag(s)
+  #   and/or pattern name(s). May be combined with +only:+.
+  # @param placeholder [String, :tagged, :hash] replacement strategy.
+  #   A String is used verbatim. +:tagged+ produces +[REDACTED:TAGNAME]+.
+  #   +:hash+ produces a deterministic +[TAGNAME_xxxx]+ token (4-hex djb2)
+  #   so the same input value always maps to the same token.
+  # @return [String] a new string with every match replaced.
+  # @raise [ArgumentError] if +placeholder:+ is not a String/:tagged/:hash.
+  # @raise [UnknownTagError] if any Symbol in +only:+/+except:+ is not in {TAGS}.
+  # @raise [UnknownPatternError] if any String in +only:+/+except:+ is not in {pattern_names}.
+  #
+  # @example
+  #   DataRedactor.redact("token sk_live_abc123", only: :credentials)
+  #   DataRedactor.redact(text, only: [:contact, "aws_access_key_id"])
+  #   DataRedactor.redact(text, only: :contact, except: ["email"])
+  def redact(text, only: nil, except: nil, placeholder: PLACEHOLDER_DEFAULT)
+    enable_bits = build_enable_bits(only, except)
+    ph_mode, ph_str = resolve_placeholder(placeholder)
+    _redact(text, ph_mode, ph_str, enable_bits)
+  end
+
+  # Scan +text+ and return both the redacted string and per-match metadata.
+  #
+  # Useful for auditing, false-positive tuning, and compliance pipelines.
+  # +:start+ and +:length+ are byte offsets into the *original* string, so
+  # +text.byteslice(m[:start], m[:length]) == m[:value]+.
+  #
+  # @param text [String] input string.
+  # @param only [Symbol, String, Array, nil] same semantics as {redact}.
+  # @param except [Symbol, String, Array, nil] same semantics as {redact}.
+  # @return [Hash{Symbol => Object}] +{ redacted: String, matches:
+  #   Array<Hash> }+. Each match hash has +:tag+ (Symbol), +:name+ (String),
+  #   +:value+ (String), +:start+ (Integer byte offset), +:length+ (Integer).
+  # @raise [UnknownTagError] if any Symbol in +only:+/+except:+ is not in {TAGS}.
+  # @raise [UnknownPatternError] if any String in +only:+/+except:+ is not in {pattern_names}.
+  #
+  # @example
+  #   DataRedactor.scan("user@example.com")
+  #   # => { redacted: "[REDACTED]",
+  #   #      matches: [{tag: :contact, name: "email",
+  #   #                 value: "user@example.com", start: 0, length: 16}] }
+  def scan(text, only: nil, except: nil)
+    enable_bits = build_enable_bits(only, except)
+    result = _scan(text, enable_bits)
+    # Normalise: convert tag string from C (uppercase) back to the Symbol used in TAGS
+    result[:matches].each { |m| m[:tag] = m[:tag].to_s.downcase.to_sym }
+    result
+  end
+
+  # Register a custom redaction pattern.
+  #
+  # Patterns must be valid POSIX ERE. Ruby-only syntax (+\d+, +\s+, +\w+,
+  # +\b+, lookaround, non-greedy quantifiers, named groups) is rejected
+  # at registration time, never at redaction time.
+  #
+  # If a pattern with the same +name+ is already registered, it is replaced
+  # (the old compiled +regex_t+ is freed).
+  #
+  # @param name [String] unique identifier for this pattern. Used by {remove_pattern}.
+  # @param regex [String, Regexp] POSIX ERE source. A Regexp is accepted
+  #   for convenience but only its +.source+ is used; flags are ignored.
+  # @param tag [Symbol] one of {TAGS} keys. Defaults to +:custom+.
+  # @param boundary [Boolean] when true, the pattern is wrapped with
+  #   +(^|[^0-9A-Za-z])(...)([^0-9A-Za-z]|$)+ so it only matches when not
+  #   embedded in a longer alphanumeric token. Incompatible with patterns
+  #   that contain capture groups.
+  # @return [Boolean] +true+ on success.
+  # @raise [ArgumentError] if +name+ is not a non-empty String, or +regex+
+  #   is neither a String nor a Regexp.
+  # @raise [InvalidPatternError] if the pattern uses Ruby-only syntax,
+  #   contains capture groups while +boundary: true+, or fails +regcomp+.
+  # @raise [UnknownTagError] if +tag+ is not in {TAGS}.
+  #
+  # @example
+  #   DataRedactor.add_pattern(name: "employee_id", regex: "EMP-[0-9]{6}")
+  #   DataRedactor.add_pattern(name: "internal_key",
+  #                            regex: /INT-[A-Z]{3}/,
+  #                            tag: :credentials,
+  #                            boundary: true)
+  def add_pattern(name:, regex:, tag: :custom, boundary: false)
+    raise ArgumentError, "name must be a non-empty String" \
+      unless name.is_a?(String) && !name.empty?
+
+    source = case regex
+             when String then regex
+             when Regexp then regex.source
+             else raise ArgumentError, "regex must be a String or Regexp, got #{regex.class}"
+             end
+
+    if source =~ RUBY_ONLY_SYNTAX_RE
+      raise InvalidPatternError,
+        "pattern #{name.inspect} uses Ruby-only syntax (#{$&.inspect}); " \
+        "use POSIX ERE — no \\d, \\s, \\w, \\b, lookaround, non-greedy, or named groups"
+    end
+
+    if boundary && source =~ CAPTURE_GROUP_RE
+      raise InvalidPatternError,
+        "pattern #{name.inspect} has capture groups and cannot use boundary: true"
+    end
+
+    tag_bit = TAGS[tag] or raise UnknownTagError,
+      "unknown tag #{tag.inspect}; valid tags: #{TAGS.keys.inspect}"
+
+    _add_pattern(name, source, tag_bit, boundary ? 1 : 0)
+  end
+
+  # Remove a previously registered custom pattern.
+  #
+  # @param name [String, Symbol] the +name+ used in {add_pattern}.
+  # @return [Boolean] +true+ if a pattern was removed, +false+ if no
+  #   pattern with that name was registered.
+  def remove_pattern(name)
+    _remove_pattern(name.to_s)
+  end
+
+  # List every currently registered custom pattern.
+  #
+  # @return [Array<Hash{Symbol => Object}>] one hash per pattern with keys
+  #   +:name+ (String), +:source+ (String — the POSIX ERE source),
+  #   +:tag+ (Symbol), +:boundary+ (Boolean).
+  def custom_patterns
+    _custom_patterns.map do |h|
+      { name: h[:name], source: h[:source], tag: TAGS.key(h[:tag_bit]) || :custom,
+        boundary: h[:boundary] }
+    end
+  end
+
+  # Remove every registered custom pattern.
+  #
+  # Mostly useful in test suites that need a clean slate between examples.
+  #
+  # @return [nil]
+  def clear_custom_patterns!
+    _clear_custom_patterns
+  end
+
+  # @api private
+  # Split a mixed Symbol/String filter list into +(tag_bitmask, name_set)+.
+  #
+  # @param entries [nil, Symbol, String, Array]
+  # @return [Array(Integer, Set<String>)] tag bits OR-ed together; set of
+  #   pattern-name Strings.
+  # @raise [UnknownTagError] for unknown Symbols.
+  # @raise [UnknownPatternError] for unknown Strings.
+  def split_filter(entries)
+    bits = 0
+    names = Set.new
+    return [bits, names] if entries.nil?
+    Array(entries).each do |e|
+      case e
+      when Symbol
+        bit = TAGS[e] or raise UnknownTagError,
+          "unknown tag #{e.inspect}; valid tags: #{TAGS.keys.inspect}"
+        bits |= bit
+      when String
+        unless pattern_names.include?(e)
+          raise UnknownPatternError,
+            "unknown pattern name #{e.inspect}; see DataRedactor.pattern_names"
+        end
+        names << e
+      else
+        raise ArgumentError,
+          "only:/except: entries must be a Symbol (tag) or String (pattern name), got #{e.inspect}"
+      end
+    end
+    [bits, names]
+  end
+
+  # @api private
+  # Build the per-pattern enable bit-list passed to the C layer.
+  #
+  # The list has one Integer (0 or 1) per pattern in execution order:
+  # built-ins first (NUM_PATTERNS entries), then currently registered custom
+  # patterns in registration order. C iterates by index and skips zeros.
+  #
+  # Semantics of +only:+ / +except:+ — both accept a mix of Symbols (tags)
+  # and Strings (pattern names):
+  #   enabled(p) iff
+  #     (only is nil OR p.tag ∈ only_tags OR p.name ∈ only_names)
+  #     AND p.tag ∉ except_tags AND p.name ∉ except_names
+  #
+  # @return [Array<Integer>] same length as built-ins + customs.
+  def build_enable_bits(only, except)
+    only_bits,   only_names   = split_filter(only)
+    except_bits, except_names = split_filter(except)
+    only_present = !only.nil?
+
+    bits = Array.new(BUILTIN_PATTERN_NAMES.length + _custom_patterns.length, 0)
+
+    BUILTIN_PATTERN_NAMES.each_with_index do |name, i|
+      tag_bit = BUILTIN_PATTERN_TAG_BITS[i]
+      bits[i] = 1 if pattern_enabled?(name, tag_bit, only_present,
+                                      only_bits, only_names,
+                                      except_bits, except_names)
+    end
+
+    _custom_patterns.each_with_index do |h, i|
+      bits[BUILTIN_PATTERN_NAMES.length + i] = 1 if pattern_enabled?(
+        h[:name], h[:tag_bit], only_present,
+        only_bits, only_names, except_bits, except_names)
+    end
+
+    bits
+  end
+
+  # @api private
+  def pattern_enabled?(name, tag_bit, only_present, only_bits, only_names,
+                       except_bits, except_names)
+    return false if (tag_bit & except_bits) != 0
+    return false if except_names.include?(name)
+    return true  unless only_present
+    return true  if (tag_bit & only_bits) != 0
+    only_names.include?(name)
+  end
+
+  # @api private
+  # Translate the user-facing +placeholder:+ value into the +(mode_int, str)+
+  # pair the C layer expects.
+  #
+  # @param placeholder [String, :tagged, :hash]
+  # @return [Array(Integer, String)]
+  # @raise [ArgumentError] if +placeholder+ is none of the accepted values.
+  def resolve_placeholder(placeholder)
+    case placeholder
+    when :tagged then [PH_MODE_TAGGED, ""]
+    when :hash   then [PH_MODE_HASH,   ""]
+    when String  then [PH_MODE_PLAIN,  placeholder]
+    else
+      raise ArgumentError,
+        "placeholder must be a String, :tagged, or :hash — got #{placeholder.inspect}"
+    end
+  end
+end

data/readme.md

@@ -0,0 +1,395 @@
+# DataRedactor
+
+[![Gem Version](https://badge.fury.io/rb/data_redactor.svg)](https://rubygems.org/gems/data_redactor)
+[![CI](https://github.com/danielefrisanco/data_redactor/actions/workflows/ci.yml/badge.svg)](https://github.com/danielefrisanco/data_redactor/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+A Ruby gem with a C extension for high-performance regex-based redaction of sensitive data from strings.
+
+## What it does
+
+DataRedactor scans text for sensitive patterns and replaces matches with `[REDACTED]`. It uses a C extension backed by POSIX `regex.h` so the heavy lifting happens outside the Ruby VM, making it fast enough for large payloads.
+
+## Usage
+
+```ruby
+require "data_redactor"
+
+text = "User CF is RSSMRA85M01H501Z and key is AKIAIOSFODNN7EXAMPLE"
+DataRedactor.redact(text)
+# => "User CF is [REDACTED] and key is [REDACTED]"
+```
+
+### Filtering by tag or pattern name
+
+`only:` and `except:` both accept a single value or an Array, mixing **Symbols** (tag names) and **Strings** (specific pattern names).
+
+```ruby
+DataRedactor.tags
+# => [:credentials, :financial, :tax_id, :national_id, :contact, :network, :travel, :other, :custom]
+
+DataRedactor.pattern_names
+# => ["aws_s3_presigned_url", "aws_access_key_id", "email", "phone_e164", "ipv4", ...]
+
+# Tag-level filtering
+DataRedactor.redact(text, only: [:credentials])
+DataRedactor.redact(text, except: :contact)
+
+# Single specific pattern
+DataRedactor.redact(text, only: ["aws_access_key_id"])
+
+# Mix — every credentials pattern PLUS aws_access_key_id (even if it lived in another tag)
+DataRedactor.redact(text, only: [:credentials, "aws_access_key_id"])
+
+# Combine — every contact pattern EXCEPT email
+DataRedactor.redact(text, only: :contact, except: ["email"])
+```
+
+**Precedence:** a pattern is redacted iff `(only is nil OR matches only:)` AND `(does not match except:)`. `except:` always wins when the two overlap, so `only: :contact, except: :contact` produces a no-op (everything is excluded).
+
+**Errors:** an unknown tag Symbol raises `DataRedactor::UnknownTagError`; an unknown pattern name String raises `DataRedactor::UnknownPatternError`.
+
+### Configurable placeholder
+
+By default every match is replaced with `[REDACTED]`. Use the `placeholder:` keyword to change this:
+
+```ruby
+# Plain string — any replacement text
+DataRedactor.redact(text, placeholder: "***")
+DataRedactor.redact(text, placeholder: "")
+
+# Tagged — embeds the pattern's tag name so you know what was redacted
+DataRedactor.redact(text, placeholder: :tagged)
+# "user@example.com"  → "[REDACTED:CONTACT]"
+# "AKIAIOSFODNN7EXAMPLE" → "[REDACTED:CREDENTIALS]"
+# "DE89370400440532013000" → "[REDACTED:FINANCIAL]"
+
+# Hash — deterministic 4-hex suffix of the matched value
+# Same value always produces the same token — useful for correlating
+# redactions across log lines without leaking the original.
+DataRedactor.redact(text, placeholder: :hash)
+# "user@example.com"  → "[CONTACT_3d7a]"
+# "user@example.com"  → "[CONTACT_3d7a]"  (same every time)
+# "other@example.com" → "[CONTACT_91fc]"  (different value, different hash)
+```
+
+All three modes compose with `only:` and `except:`:
+
+```ruby
+DataRedactor.redact(text, only: :contact, placeholder: :tagged)
+```
+
+### Scan / dry-run mode
+
+`DataRedactor.scan` returns every match alongside the redacted string — useful for auditing, tuning false positives, and compliance pipelines:
+
+```ruby
+result = DataRedactor.scan("User AKIAIOSFODNN7EXAMPLE logged in from 192.168.1.1")
+# => {
+#   redacted: "User [REDACTED] logged in from [REDACTED]",
+#   matches: [
+#     { tag: :credentials, name: "aws_access_key_id", value: "AKIAIOSFODNN7EXAMPLE", start: 5,  length: 20 },
+#     { tag: :network,     name: "ipv4",              value: "192.168.1.1",          start: 35, length: 11 }
+#   ]
+# }
+
+# :start and :length are byte offsets into the original string
+m = result[:matches].first
+original_text.byteslice(m[:start], m[:length])  # => "AKIAIOSFODNN7EXAMPLE"
+
+# Accepts the same filters as redact (tags + specific pattern names)
+DataRedactor.scan(text, only: :credentials)
+DataRedactor.scan(text, except: :network)
+DataRedactor.scan(text, only: :contact, except: ["email"])
+```
+
+### Custom patterns
+
+Teams often have internal IDs that the gem can't ship. Register them at boot:
+
+```ruby
+# String (POSIX ERE) or Regexp — both accepted
+DataRedactor.add_pattern(name: "employee_id", regex: "EMP-[0-9]{6}")
+DataRedactor.add_pattern(name: "ticket_ref",  regex: /TICKET-[A-Z]{2}[0-9]{4}/, boundary: true)
+
+# Custom patterns are tagged :custom by default; pass any built-in tag to group differently
+DataRedactor.add_pattern(name: "internal_key", regex: "INT-[A-Z]{3}", tag: :credentials)
+
+DataRedactor.redact(text)                         # runs all patterns including custom
+DataRedactor.redact(text, only: [:custom])         # only user patterns
+DataRedactor.redact(text, only: [:custom, :credentials]) # mix
+
+DataRedactor.custom_patterns   # => [{name:, source:, tag:, boundary:}, ...]
+DataRedactor.remove_pattern("employee_id")
+DataRedactor.clear_custom_patterns!               # mostly for test suites
+```
+
+**Regex rules** — patterns must be POSIX ERE (the same engine used for built-ins). Not supported: `\d`, `\s`, `\w`, `\b`, lookahead/lookbehind, non-greedy quantifiers, named groups. Violations raise `DataRedactor::InvalidPatternError` at registration time, never at redaction time. Use `[0-9]` instead of `\d`, `[[:space:]]` instead of `\s`, etc.
+
+**`boundary: true`** — wraps the pattern with `(^|[^0-9A-Za-z])(PATTERN)([^0-9A-Za-z]|$)` so it only fires when the token is not embedded in a longer alphanumeric string. Incompatible with patterns that contain capture groups.
+
+## Integrations
+
+Optional adapters for Logger, Rails, and Rack. None are loaded automatically — `require` only what you use, and the gem adds zero runtime dependencies in the gemspec.
+
+### Logger formatter
+
+Drop-in `Logger::Formatter` replacement that scrubs every emitted line:
+
+```ruby
+require "data_redactor/integrations/logger"
+
+logger = Logger.new($stdout)
+logger.formatter = DataRedactor::Integrations::Logger.new
+logger.info("Auth failed for alice@example.com")
+# => I, [...] -- : Auth failed for [REDACTED]
+```
+
+Wraps an inner formatter (defaults to `Logger::Formatter`), so it composes with structured loggers. Forwards `only:`, `except:`, `placeholder:` to `DataRedactor.redact`. Exception messages and arbitrary objects are scrubbed too — the wrapped object is passed unchanged to the inner formatter so the exception cause chain is preserved; only the rendered string is redacted.
+
+### Rails `filter_parameters` adapter
+
+```ruby
+# config/initializers/filter_parameter_logging.rb
+require "data_redactor/integrations/rails"
+
+Rails.application.config.filter_parameters += [
+  DataRedactor::Integrations::Rails.filter
+]
+```
+
+Returns a `(key, value)` proc compatible with Rails' parameter filter. String values are mutated in place via `String#replace` so Rails sees the redacted value. Non-strings are left alone. Accepts the same `only:`/`except:`/`placeholder:` kwargs.
+
+### Rack middleware
+
+```ruby
+# config.ru
+require "data_redactor/integrations/rack"
+
+use DataRedactor::Integrations::Rack, scrub: [:body, :headers]
+run MyApp
+```
+
+`scrub:` selects which surfaces to redact (default `[:body, :headers]`):
+
+- **`:body`** — buffers the response body, runs `DataRedactor.redact` over it, returns it as a single chunk. Drops the `Content-Length` header so the server recomputes (the redacted body may differ in byte length).
+- **`:headers`** — scrubs sensitive **response** headers (`Set-Cookie`, `Authorization`, `X-Api-Key`, `X-Auth-Token`, `X-Access-Token`) in place, and sensitive **request** headers (`HTTP_AUTHORIZATION`, `HTTP_PROXY_AUTHORIZATION`, `HTTP_COOKIE`, `HTTP_X_API_KEY`, `HTTP_X_AUTH_TOKEN`, `HTTP_X_ACCESS_TOKEN`) in the env hash so any downstream middleware that logs them sees redacted values.
+
+Pass an empty subset (e.g. `scrub: [:headers]`) to opt out of body wrapping. Forwards `only:`/`except:`/`placeholder:` to `DataRedactor.redact`. Unknown surfaces raise `ArgumentError` at boot.
+
+> **Body wrapping is buffering.** The middleware reads the entire response body into memory before scanning. For streaming endpoints (SSE, large file downloads, Rack::Hijack) use `scrub: [:headers]` and rely on the Logger formatter for application logs instead.
+
+## Detected patterns (85 total)
+
+The table below is a representative sample. Use `DataRedactor.pattern_names` for the canonical, machine-readable list — it stays in sync with the C extension automatically.
+
+### Cloud & API secrets
+
+| # | Pattern | Example |
+|---|---|---|
+| — | AWS Access Key ID | `AKIAIOSFODNN7EXAMPLE` |
+| — | AWS Secret Access Key | 40-character base64 string |
+| — | Google API Key | `AIzaSyXXXX...` |
+| — | GitHub Personal Access Token | `github_pat_XXXX...` |
+| — | GitHub Classic PAT / OAuth | `ghp_XXXX...` / `gho_XXXX...` |
+| — | Slack Webhook URL | `https://hooks.slack.com/services/T.../B.../...` |
+| — | Stripe Secret Key | `sk_live_XXXX...` |
+| — | Anthropic API Key | `sk-ant-api03-XXXX...` |
+| — | OpenAI Project API Key | `sk-proj-XXXX...` |
+| — | GitLab Personal Access Token | `glpat-XXXX...` |
+| — | DigitalOcean PAT | `dop_v1_XXXX...` |
+| — | Databricks API Token | `dapiXXXX...` |
+| — | Sentry DSN | `https://KEY@oNNN.ingest.sentry.io/PID` |
+| — | PEM Private Key header | `-----BEGIN RSA PRIVATE KEY-----` |
+| — | Scaleway Access Key | `SCW12345ABCDE6789FGHIJ` |
+| — | UUID v4 / Scaleway Secret Key | `550e8400-e29b-41d4-a716-446655440000` |
+
+### Travel documents
+
+| # | Pattern | Example |
+|---|---|---|
+| 2 | Italian Codice Fiscale (basic) | `RSSMRA85M01H501Z` |
+| 3 | Passport — letter prefix + digits | `AB1234567` |
+| 4 | Passport — 9 consecutive digits ¹ | `123456789` |
+| 22 | Italian Codice Fiscale (omocodia) | `RSSMRALPMNLH5LMZ` |
+
+### Payment & network
+
+| # | Pattern | Example |
+|---|---|---|
+| 11 | Credit card — Visa, Mastercard, Amex, Discover, JCB | `4111111111111111` |
+| 12 | IPv4 address | `192.168.1.100` |
+
+### IBANs
+
+| # | Country | Example |
+|---|---|---|
+| 10 | Italy | `IT60X0542811101000000123456` |
+| 15 | France | `FR7630006000011234567890189` |
+| 16 | Germany | `DE89370400440532013000` |
+| 17 | Spain | `ES9121000418450200051332` |
+| 18 | Netherlands | `NL91ABNA0417164300` |
+| 19 | Belgium | `BE68539007547034` |
+| 20 | Portugal | `PT50000201231234567890154` |
+| 21 | Ireland | `IE29AIBK93115212345678` |
+| 28 | Sweden | `SE4550000000058398257466` |
+| 29 | Denmark | `DK5000400440116243` |
+| 30 | Norway | `NO9386011117947` |
+| 31 | Finland | `FI2112345600000785` |
+| 37 | Poland | `PL61109010140000071219812874` |
+| 38 | Austria | `AT611904300234573201` |
+| 39 | Switzerland | `CH9300762011623852957` |
+| 40 | Czechia | `CZ6508000000192000145399` |
+| 41 | Hungary | `HU42117730161111101800000000` |
+| 42 | Romania | `RO49AAAA1B31007593840000` |
+
+### National personal identifiers
+
+| # | Country | Type | Example |
+|---|---|---|---|
+| 23 | France | NIR / Social Security ¹ | `185126203450342` |
+| 24 | Spain | DNI ¹ | `12345678Z` |
+| 25 | Spain | NIE | `X1234567L` |
+| 26 | Netherlands | BSN ¹ | `123456789` |
+| 27 | Poland | PESEL ¹ | `85121612345` |
+| 32 | Belgium | National Number ¹ | `85121612345` |
+| 33 | Sweden | Personnummer ¹ | `850101-1234` |
+| 34 | Denmark | CPR Number ¹ | `010185-1234` |
+| 35 | Norway | Fødselsnummer ¹ | `01018512345` |
+| 36 | Finland | HETU ¹ | `010185-123A` |
+| 43 | Poland | PESEL (alt slot) ¹ | `90010112345` |
+| 44 | Austria | Abgabenkontonummer ¹ | `123456789` |
+| 45 | Switzerland | AHV Number ¹ | `756.1234.5678.90` |
+| 46 | Czechia | Rodné číslo ¹ | `856121/1234` |
+| 47 | Hungary | Tax ID ¹ | `8012345678` |
+| 48 | Romania | CNP ¹ | `1850101123456` |
+
+> ¹ **Word-boundary protected** — these patterns are wrapped with `(^|[^0-9A-Za-z])(PATTERN)([^0-9A-Za-z]|$)` at compile time so they do not fire when the digit sequence appears inside a longer alphanumeric token.
+
+## Directory structure
+
+```
+redactor/
+├── data_redactor.gemspec
+├── Gemfile
+├── Rakefile
+├── lib/
+│   ├── data_redactor.rb          # Ruby entry point, loads the .so
+│   └── data_redactor/
+│       └── version.rb
+├── ext/
+│   └── data_redactor/
+│       ├── extconf.rb            # Checks for C headers, generates Makefile (globs *.c)
+│       ├── data_redactor.c       # Entry point: Init_data_redactor only
+│       ├── patterns.{c,h}        # Built-in pattern table + compiled regex_t array
+│       ├── placeholder.{c,h}     # write_placeholder, djb2 hash, tag_name_for_bit
+│       ├── redact.{c,h}          # _redact + replace_all_matches + wrap_boundary
+│       ├── scan.{c,h}            # _scan + byte-offset replacement-log macros
+│       ├── custom_patterns.{c,h} # Dynamic registry: add/remove/clear/list
+│       └── tags.h                # TAG_* bit constants
+└── spec/
+    └── data_redactor_spec.rb     # RSpec tests — at least one example per pattern, plus filter / placeholder / custom-pattern coverage
+```
+
+## Requirements
+
+- Ruby >= 2.7
+- A C compiler (`gcc` or `clang`) — only required when installing the source gem
+- POSIX `regex.h` — only required when installing the source gem (standard on Linux and macOS)
+
+## Installation
+
+```ruby
+# Gemfile
+gem "data_redactor"
+```
+
+```bash
+bundle install
+```
+
+That's it — there is nothing extra to configure for precompiled binaries. Bundler/RubyGems looks at your platform and Ruby version and picks the right gem automatically.
+
+### What you'll see
+
+- **On a supported platform** (Linux glibc/musl, macOS Intel/ARM): bundler downloads a precompiled gem with the C extension already built. Install is near-instant — **no compiler, no `make`, no `regex.h` headers needed**. Especially valuable in slim Docker images (`ruby:3.x-alpine`, `ruby:3.x-slim`) that don't ship `gcc`.
+- **On any other platform** (FreeBSD, OpenBSD, etc.): bundler downloads the source gem and compiles the C extension on install — the same behavior as before 0.7.1. You'll need a C compiler and POSIX `regex.h` available.
+
+### Supported precompiled targets
+
+Each precompiled gem ships compiled binaries for Ruby 3.1, 3.2, 3.3, and 3.4.
+
+| Platform | Targets |
+|---|---|
+| Linux (glibc) | `x86_64-linux`, `aarch64-linux` |
+| Linux (musl / Alpine) | `x86_64-linux-musl`, `aarch64-linux-musl` |
+| macOS | `x86_64-darwin` (Intel), `arm64-darwin` (Apple Silicon) |
+
+### Bundler-locked deploys
+
+If your `Gemfile.lock` was generated on one platform but you deploy to another, run `bundle lock --add-platform <target>` so bundler resolves the right native gem at deploy time. Example for Alpine deploys built from a glibc dev box:
+
+```bash
+bundle lock --add-platform x86_64-linux-musl aarch64-linux-musl
+```
+
+## Compile the C extension (source / development install only)
+
+```bash
+bundle exec rake compile
+```
+
+This runs `extconf.rb` via `rake-compiler`, which generates a `Makefile` and compiles `data_redactor.c` into a `.so` shared library placed under `lib/data_redactor/`.
+
+## Building precompiled gems locally
+
+Maintainers can rebuild the full set of native gems with one command (requires Docker):
+
+```bash
+bundle exec rake gem:all
+```
+
+This invokes `rake-compiler-dock` to cross-compile every supported (platform × Ruby ABI) combination. Output lands in `pkg/`.
+
+## Run the tests
+
+```bash
+bundle exec rake spec
+```
+
+Or compile and test in one step:
+
+```bash
+bundle exec rake
+```
+
+## How it works
+
+1. At load time, `Init_data_redactor` compiles all 85 regex patterns once using `regcomp` (POSIX ERE) and stores them as static `regex_t` structs. Patterns marked as boundary-wrapped are expanded with `wrap_boundary()` before compilation.
+2. `DataRedactor.redact(text)` receives a Ruby `String`, converts it to a C `char*` via `StringValueCStr`, and runs each compiled pattern in sequence on a working buffer.
+3. For each pattern, `replace_all_matches` iterates using `regexec`, copies non-matching segments to a fresh output buffer, and inserts `[REDACTED]` in place of each match. For boundary-wrapped patterns, `regexec` is called with `nmatch=4` and sub-match groups `[1]`/`[3]` identify the boundary characters so they are preserved verbatim.
+4. The output buffer is grown with `realloc` as needed. After all patterns are applied the result is returned as a Ruby `String` via `rb_str_new_cstr`. All intermediate `malloc`/`strdup` allocations are explicitly `free`d.
+
+## Memory management
+
+All C-side buffers are heap-allocated with `malloc`/`strdup` and freed before the function returns. The only Ruby-managed allocation is the final return value from `rb_str_new_cstr`. No Ruby objects are created mid-processing, so GC cannot collect anything out from under the C code.
+
+## Thread safety
+
+`DataRedactor.redact` and `DataRedactor.scan` are safe to call concurrently from multiple threads. Built-in patterns are compiled into a static `regex_t` array at load time and never mutated afterward, and each call allocates its own working buffers. POSIX `regexec` is documented as thread-safe.
+
+`DataRedactor.add_pattern`, `remove_pattern`, and `clear_custom_patterns!` mutate a shared dynamic array and are **not** thread-safe. Register custom patterns once at boot — before spawning worker threads or forking — and they will be visible (read-only) to every subsequent `redact`/`scan` call.
+
+## Versioning
+
+This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html). Until `1.0.0`, minor versions may introduce breaking changes; from `1.0.0` onward, breaking changes will only land in major versions. See [CHANGELOG.md](CHANGELOG.md) for the release history.
+
+## License
+
+Released under the [MIT License](LICENSE).
+
+## Known limitations
+
+- **Pattern ordering matters** — patterns run sequentially. An early broad pattern (e.g. the 9-digit passport) may consume digits that a later pattern (e.g. credit card) depends on. Boundary wrapping mitigates this for pure-digit patterns.
+- **AWS Secret Key (pattern 1)** — 40 consecutive base64 characters is a broad match. It can produce false positives in base64-encoded content such as embedded images or binary blobs.
+- **Duplicate digit patterns** — several national ID formats share the same digit-length (11 digits: PESEL, Norwegian Fødselsnummer, Belgian National Number). They are kept as separate slots for clarity but the practical effect is that any 11-digit boundary-delimited number will be redacted.

metadata

@@ -0,0 +1,122 @@
+--- !ruby/object:Gem::Specification
+name: data_redactor
+version: !ruby/object:Gem::Version
+  version: 0.7.2
+platform: x86_64-linux
+authors:
+- Daniele Frisanco
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rake-compiler-dock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
+- !ruby/object:Gem::Dependency
+  name: rack
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.0'
+description: A Ruby gem with a C extension for high-performance scanning and redaction
+  of 85 sensitive patterns — API keys, tokens, credentials, IBANs, national IDs, emails,
+  phone numbers, and PII from 15+ countries. Optional Logger formatter, Rails filter_parameters
+  adapter, and Rack middleware. Designed to sanitize text before sending to LLMs,
+  logging systems, or any public/third-party API.
+email:
+- daniele.frisanco@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE
+- lib/data_redactor.rb
+- lib/data_redactor/3.0/data_redactor.so
+- lib/data_redactor/3.1/data_redactor.so
+- lib/data_redactor/3.2/data_redactor.so
+- lib/data_redactor/3.3/data_redactor.so
+- lib/data_redactor/3.4/data_redactor.so
+- lib/data_redactor/4.0/data_redactor.so
+- lib/data_redactor/integrations/logger.rb
+- lib/data_redactor/integrations/rack.rb
+- lib/data_redactor/integrations/rails.rb
+- lib/data_redactor/version.rb
+- readme.md
+homepage: https://github.com/danielefrisanco/data_redactor
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/danielefrisanco/data_redactor
+  source_code_uri: https://github.com/danielefrisanco/data_redactor
+  changelog_uri: https://github.com/danielefrisanco/data_redactor/blob/main/CHANGELOG.md
+  bug_tracker_uri: https://github.com/danielefrisanco/data_redactor/issues
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+  - - "<"
+    - !ruby/object:Gem::Version
+      version: 4.1.dev
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.6
+specification_version: 4
+summary: Redact PII and secrets from strings before sending to AI or external services
+test_files: []