data_redactor 0.10.1-x86_64-darwin → 0.11.0-x86_64-darwin

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 85bd57c818650ea452e6787af3596c4b868d54d9b4d0caaf66b4729bd3618ddb
-  data.tar.gz: 61eba720403deacbce30fea57626629885cf4b0d68bc8c1446e31f79e66569d6
+  metadata.gz: a96a7eca713bd504ed219651ad47b4feeef9ae577ab2b5c2787316a00630ede9
+  data.tar.gz: 5d47a193b079d6d09176b204abd4f1d488da88c135fe8b3a83d533ca40b6cc2c
 SHA512:
-  metadata.gz: 937c343346177564cd9688b85028817fac5918846e96a03bf2376cb276c54e6df6b33238271a17473106b823cebc54ac2c2ff98d6459b34825941b0162c885e2
-  data.tar.gz: a44bee564f7ab53c85b1cc244ae14f2893b39761d59b4043b81acf19c053315731ce4756e5cf54540a38ca1867f723c60c11312510772a3496f076ac11c98753
+  metadata.gz: 63520c587722bef51d5fc24df838935f2311b2b5c2e3f55776e1230eb670c787e32cc0b264f9d571b0f8b1a6909364202a3e76c81f8b1de35045dfe7c079c1cc
+  data.tar.gz: a9e7ffb2a7b043ddb9e87e5fb96e7d79bd2da90a94de88dd511fc63208e245be16914487f4c46c9d663c461b24026527cc2f1c84b9a2a288aa2414901ba6e14e

data/CHANGELOG.md

@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.11.0] - 2026-06-10
+
+### Added
+- **Claude / OpenAI LLM integrations** — two new soft-required adapters that
+  scrub PII and secrets from LLM payloads before they leave the process and
+  from responses before they're logged:
+  - `DataRedactor::Integrations::Claude` — `.redact_messages` (handles the
+    `messages` array plus a top-level `system:` prompt; String or
+    array-of-content-block content) and `.redact_response` (Messages API
+    `content` text blocks).
+  - `DataRedactor::Integrations::OpenAI` — `.redact_messages` (Chat
+    Completions `messages`, including a `system` message and array-of-parts
+    content) and `.redact_response` (`choices[].message.content`).
+  Both operate on plain Ruby Hashes/Arrays with String or Symbol keys (no
+  runtime dependency on the `anthropic`/`openai` gems), return a deep copy
+  (never mutate the caller's input), pass non-text content blocks through
+  untouched, and forward `only:`/`except:`/`placeholder:` to
+  `DataRedactor.redact`.
+
 ## [0.10.1] - 2026-06-10
 
 ### Fixed
@@ -213,7 +232,8 @@ features as 0.7.1 plus the pipeline fix.
 - `DataRedactor.redact(text)` module function returning the input with every match replaced by `[REDACTED]`.
 - RSpec suite with one example per pattern.
 
-[Unreleased]: https://github.com/danielefrisanco/data_redactor/compare/v0.10.1...HEAD
+[Unreleased]: https://github.com/danielefrisanco/data_redactor/compare/v0.11.0...HEAD
+[0.11.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.10.1...v0.11.0
 [0.10.1]: https://github.com/danielefrisanco/data_redactor/compare/v0.10.0...v0.10.1
 [0.10.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.9.0...v0.10.0
 [0.9.0]: https://github.com/danielefrisanco/data_redactor/compare/v0.8.0...v0.9.0

data/README.md

@@ -275,6 +275,34 @@ Pass an empty subset (e.g. `scrub: [:headers]`) to opt out of body wrapping. For
 
 > **Body wrapping is buffering.** The middleware reads the entire response body into memory before scanning. For streaming endpoints (SSE, large file downloads, Rack::Hijack) use `scrub: [:headers]` and rely on the Logger formatter for application logs instead.
 
+### Claude / OpenAI LLM payloads
+
+Sanitize LLM message payloads before they leave the process, and scrub responses before they're logged or stored. Both adapters operate on plain Ruby Hashes/Arrays (String **or** Symbol keys), so they work with the `anthropic`/`openai` gems, a raw HTTP client, or parsed JSON — no runtime dependency on any SDK. They **return a deep copy and never mutate your input**, and forward `only:`/`except:`/`placeholder:` to `DataRedactor.redact`.
+
+```ruby
+require "data_redactor/integrations/claude"
+
+# Redact a messages array before sending to Claude
+safe_messages = DataRedactor::Integrations::Claude.redact_messages(messages)
+client.messages.create(model: "claude-opus-4-8", max_tokens: 1024, messages: safe_messages)
+
+# Redact the response (assistant content blocks) before logging
+safe_response = DataRedactor::Integrations::Claude.redact_response(response)
+```
+
+```ruby
+require "data_redactor/integrations/openai"
+
+# Redact a messages array before sending to OpenAI
+safe_messages = DataRedactor::Integrations::OpenAI.redact_messages(messages)
+client.chat(parameters: { model: "gpt-4o", messages: safe_messages })
+
+# Redact the response (choices[].message.content) before logging
+safe_response = DataRedactor::Integrations::OpenAI.redact_response(response)
+```
+
+`content` may be a plain String or an array of content blocks/parts (`{ type: "text", text: "..." }`) — only the `text` of `text` blocks is redacted; image and other block types pass through untouched. For Claude, a top-level `system:` String is also redacted; for OpenAI, a `{ role: "system" }` message in the array is redacted like any other. Pass a bare `messages` array or the whole request Hash (with a `messages` key) — either works.
+
 ## Detected patterns (88 total)
 
 The table below is a representative sample. Use `DataRedactor.pattern_names` for the canonical, machine-readable list — it stays in sync with the C extension automatically.
@@ -510,6 +538,26 @@ All payload sizes pass a correctness check (redaction count matches pure-Ruby `g
 The previous engine (per-pattern `regexec`) was **4.25× slower** than pure Ruby on the
 1 MB payload — a ~9× swing. Old numbers are in git history (`CHANGELOG.md` [0.9.0]).
 
+#### Linear scaling
+
+Throughput stays flat as input grows — the single-pass engine is O(N), so a 10×
+larger payload takes ~10× longer and MB/s holds steady. The old per-pattern
+`regexec` engine was O(N²) and fell off a cliff on large inputs (a 10 MB log took
+tens of seconds); v19 redacts the same 10 MB in ~1.4 s.
+
+| Size   | Time     | MB/s    |
+|--------|----------|---------|
+| 1 KB   | 0.14 ms  | 7.1     |
+| 100 KB | 13.4 ms  | 7.3     |
+| 1 MB   | 142 ms   | 7.0     |
+| 10 MB  | 1.42 s   | 7.0     |
+| 50 MB  | 7.14 s   | 7.0     |
+
+No published benchmarks exist for comparable Ruby PII-redaction gems, so the
+numbers above are absolute (vs pure-Ruby `gsub`), not a head-to-head against
+another gem. Run `benchmark/scaling.rb` on your own hardware — absolute MB/s is
+machine-dependent, but the flat curve is not.
+
 ## How it works
 
 1. At load time, `Init_data_redactor` compiles all 85 regex patterns once using `regcomp` (POSIX ERE) and stores them as static `regex_t` structs. Patterns marked as boundary-wrapped are expanded with `wrap_boundary()` before compilation.

data/lib/data_redactor/integrations/claude.rb

@@ -0,0 +1,113 @@
+require "data_redactor"
+require "data_redactor/integrations/llm_support"
+
+module DataRedactor
+  module Integrations
+    # Adapter for Anthropic Claude (Messages API) payloads. Scrubs PII and
+    # secrets from a request's `messages` (and top-level `system:` prompt)
+    # before they leave the process, and from a response's `content` blocks
+    # before they're logged or stored.
+    #
+    # Operates on plain Ruby Hashes/Arrays with either String or Symbol keys,
+    # so it works with the `anthropic` gem, a raw HTTP client, or parsed JSON —
+    # no runtime dependency on any SDK. Inputs are never mutated; a deep copy
+    # is returned.
+    #
+    # @example Scrub a request before sending
+    #   require "data_redactor/integrations/claude"
+    #
+    #   messages = [{ role: "user", content: "my email is alice@example.com" }]
+    #   safe = DataRedactor::Integrations::Claude.redact_messages(messages)
+    #   client.messages.create(model: "claude-opus-4-8", max_tokens: 1024,
+    #                          messages: safe)
+    #
+    # @example Scrub a response before logging
+    #   resp = client.messages.create(...)
+    #   logger.info DataRedactor::Integrations::Claude.redact_response(resp)
+    module Claude
+      module_function
+
+      # Redact a Claude `messages` array (and an optional top-level `system:`
+      # String) before sending the request. Returns a deep copy; the input is
+      # not mutated.
+      #
+      # Each message's `content` may be a String or an array of content blocks
+      # (`{ type: "text", text: "..." }`); only the `text` field of `text`
+      # blocks is redacted. Non-text blocks (e.g. `image`) pass through
+      # untouched.
+      #
+      # @param messages [Array<Hash>, Hash] either a bare array of
+      #   `{ role:, content: }` hashes, or a request Hash containing a
+      #   `messages` key and an optional `system` key. Keys may be String or
+      #   Symbol.
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      # @return [Array<Hash>, Hash] a deep copy of the input with text leaves
+      #   redacted; an Array if an Array was given, a Hash if a Hash was given.
+      # @example
+      #   Claude.redact_messages([{ role: "user", content: "ssn 123-45-6789" }])
+      #   #=> [{ role: "user", content: "ssn [REDACTED]" }]
+      def redact_messages(messages, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        redact = ->(s) { DataRedactor.redact(s, only: only, except: except, placeholder: placeholder) }
+
+        if messages.is_a?(Hash)
+          out = LLMSupport.deep_copy(messages)
+          sys = LLMSupport.fetch(out, :system)
+          LLMSupport.put(out, :system, redact.call(sys)) if sys.is_a?(String)
+          list = LLMSupport.fetch(out, :messages)
+          LLMSupport.put(out, :messages, redact_message_list(list, redact)) if list.is_a?(Array)
+          out
+        else
+          redact_message_list(LLMSupport.deep_copy(messages), redact)
+        end
+      end
+
+      # Redact a Claude Messages API response before logging or storing it.
+      # Returns a deep copy; the input is not mutated.
+      #
+      # Walks the response's `content` array and redacts the `text` field of
+      # each `text` block, leaving the rest of the response (id, role, usage,
+      # non-text blocks) intact.
+      #
+      # @param response [Hash] a Claude response Hash with a `content` array of
+      #   blocks. Keys may be String or Symbol.
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      # @return [Hash] a deep copy of the response with text blocks redacted.
+      # @example
+      #   Claude.redact_response(
+      #     "content" => [{ "type" => "text", "text" => "card 4111111111111111" }]
+      #   )
+      #   #=> {"content"=>[{"type"=>"text", "text"=>"card [REDACTED]"}]}
+      def redact_response(response, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        redact = ->(s) { DataRedactor.redact(s, only: only, except: except, placeholder: placeholder) }
+        out = LLMSupport.deep_copy(response)
+        return out unless out.is_a?(Hash)
+
+        content = LLMSupport.fetch(out, :content)
+        LLMSupport.put(out, :content, LLMSupport.redact_text_blocks(content, redact)) if content.is_a?(Array)
+        out
+      end
+
+      # @!visibility private
+      # Redact each message's content (String or array of blocks) in place.
+      # Expects an already-deep-copied list.
+      def redact_message_list(messages, redact)
+        messages.map do |msg|
+          next msg unless msg.is_a?(Hash)
+
+          content = LLMSupport.fetch(msg, :content)
+          case content
+          when String
+            LLMSupport.put(msg, :content, redact.call(content))
+          when Array
+            LLMSupport.put(msg, :content, LLMSupport.redact_text_blocks(content, redact))
+          end
+          msg
+        end
+      end
+    end
+  end
+end

data/lib/data_redactor/integrations/llm_support.rb

@@ -0,0 +1,63 @@
+require "data_redactor"
+
+module DataRedactor
+  module Integrations
+    # Shared helpers for the LLM payload adapters ({Claude}, {OpenAI}).
+    #
+    # Both adapters operate on plain Ruby Hashes/Arrays whose keys may be
+    # String or Symbol, never mutate the caller's input, and redact only the
+    # `text` field of text content blocks. This module holds that common,
+    # non-trivial logic; the per-provider modules keep only their own
+    # provider-specific shape walking.
+    #
+    # @!visibility private
+    module LLMSupport
+      module_function
+
+      # Read a key from a Hash that may use String or Symbol keys.
+      # @return the value under the Symbol or String form of `key`, or nil.
+      def fetch(hash, key)
+        hash.key?(key) ? hash[key] : hash[key.to_s]
+      end
+
+      # Write a value back under whichever key form (String/Symbol) the Hash
+      # already uses for `key`, defaulting to the Symbol form.
+      # @return [void]
+      def put(hash, key, value)
+        if hash.key?(key.to_s)
+          hash[key.to_s] = value
+        else
+          hash[key] = value
+        end
+      end
+
+      # Recursively copy a Hash/Array/String structure so the original is never
+      # mutated. Non-container, non-String leaves are returned as-is.
+      # @return a deep copy of `obj`.
+      def deep_copy(obj)
+        case obj
+        when Hash   then obj.each_with_object({}) { |(k, v), o| o[k] = deep_copy(v) }
+        when Array  then obj.map { |v| deep_copy(v) }
+        when String then obj.dup
+        else obj
+        end
+      end
+
+      # Redact the `text` field of each `text` content block in `blocks`,
+      # passing non-text blocks (e.g. images) through untouched. Mutates the
+      # blocks in `blocks` (call on an already-copied structure).
+      # @param blocks [Array] content blocks.
+      # @param redact [#call] a String -> String redaction lambda.
+      # @return [Array] the same `blocks` array, with text blocks redacted.
+      def redact_text_blocks(blocks, redact)
+        blocks.map do |block|
+          next block unless block.is_a?(Hash)
+
+          text = fetch(block, :text)
+          put(block, :text, redact.call(text)) if text.is_a?(String)
+          block
+        end
+      end
+    end
+  end
+end

data/lib/data_redactor/integrations/openai.rb

@@ -0,0 +1,118 @@
+require "data_redactor"
+require "data_redactor/integrations/llm_support"
+
+module DataRedactor
+  module Integrations
+    # Adapter for OpenAI Chat Completions payloads. Scrubs PII and secrets from
+    # a request's `messages` before they leave the process, and from a
+    # response's `choices[].message.content` before they're logged or stored.
+    #
+    # Operates on plain Ruby Hashes/Arrays with either String or Symbol keys,
+    # so it works with the `openai` gem, a raw HTTP client, or parsed JSON — no
+    # runtime dependency on any SDK. Inputs are never mutated; a deep copy is
+    # returned.
+    #
+    # @example Scrub a request before sending
+    #   require "data_redactor/integrations/openai"
+    #
+    #   messages = [{ role: "user", content: "my email is alice@example.com" }]
+    #   safe = DataRedactor::Integrations::OpenAI.redact_messages(messages)
+    #   client.chat(parameters: { model: "gpt-4o", messages: safe })
+    #
+    # @example Scrub a response before logging
+    #   resp = client.chat(parameters: { ... })
+    #   logger.info DataRedactor::Integrations::OpenAI.redact_response(resp)
+    module OpenAI
+      module_function
+
+      # Redact an OpenAI `messages` array before sending the request. Returns a
+      # deep copy; the input is not mutated.
+      #
+      # Each message's `content` may be a String or an array of parts
+      # (`{ type: "text", text: "..." }`); only the `text` field of `text`
+      # parts is redacted. Non-text parts (e.g. `image_url`) pass through
+      # untouched. A `{ role: "system", content: ... }` entry is redacted like
+      # any other message (OpenAI carries the system prompt in the array).
+      #
+      # @param messages [Array<Hash>, Hash] either a bare array of
+      #   `{ role:, content: }` hashes, or a request Hash containing a
+      #   `messages` key. Keys may be String or Symbol.
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      # @return [Array<Hash>, Hash] a deep copy of the input with text leaves
+      #   redacted; an Array if an Array was given, a Hash if a Hash was given.
+      # @example
+      #   OpenAI.redact_messages([{ role: "user", content: "ssn 123-45-6789" }])
+      #   #=> [{ role: "user", content: "ssn [REDACTED]" }]
+      def redact_messages(messages, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        redact = ->(s) { DataRedactor.redact(s, only: only, except: except, placeholder: placeholder) }
+
+        if messages.is_a?(Hash)
+          out = LLMSupport.deep_copy(messages)
+          list = LLMSupport.fetch(out, :messages)
+          LLMSupport.put(out, :messages, redact_message_list(list, redact)) if list.is_a?(Array)
+          out
+        else
+          redact_message_list(LLMSupport.deep_copy(messages), redact)
+        end
+      end
+
+      # Redact an OpenAI Chat Completions response before logging or storing it.
+      # Returns a deep copy; the input is not mutated.
+      #
+      # Walks `choices[].message.content` and redacts each (String content),
+      # leaving the rest of the response (id, usage, finish_reason) intact.
+      #
+      # @param response [Hash] a response Hash with a `choices` array, each
+      #   choice carrying a `message` Hash with a `content` String. Keys may be
+      #   String or Symbol.
+      # @param only forwarded to {DataRedactor.redact}
+      # @param except forwarded to {DataRedactor.redact}
+      # @param placeholder forwarded to {DataRedactor.redact}
+      # @return [Hash] a deep copy of the response with message content redacted.
+      # @example
+      #   OpenAI.redact_response(
+      #     "choices" => [{ "message" => { "content" => "card 4111111111111111" } }]
+      #   )
+      #   #=> {"choices"=>[{"message"=>{"content"=>"card [REDACTED]"}}]}
+      def redact_response(response, only: nil, except: nil, placeholder: DataRedactor::PLACEHOLDER_DEFAULT)
+        redact = ->(s) { DataRedactor.redact(s, only: only, except: except, placeholder: placeholder) }
+        out = LLMSupport.deep_copy(response)
+        return out unless out.is_a?(Hash)
+
+        choices = LLMSupport.fetch(out, :choices)
+        return out unless choices.is_a?(Array)
+
+        choices.each do |choice|
+          next unless choice.is_a?(Hash)
+
+          message = LLMSupport.fetch(choice, :message)
+          next unless message.is_a?(Hash)
+
+          content = LLMSupport.fetch(message, :content)
+          LLMSupport.put(message, :content, redact.call(content)) if content.is_a?(String)
+        end
+        out
+      end
+
+      # @!visibility private
+      # Redact each message's content (String or array of parts) in place.
+      # Expects an already-deep-copied list.
+      def redact_message_list(messages, redact)
+        messages.map do |msg|
+          next msg unless msg.is_a?(Hash)
+
+          content = LLMSupport.fetch(msg, :content)
+          case content
+          when String
+            LLMSupport.put(msg, :content, redact.call(content))
+          when Array
+            LLMSupport.put(msg, :content, LLMSupport.redact_text_blocks(content, redact))
+          end
+          msg
+        end
+      end
+    end
+  end
+end

data/lib/data_redactor/integrations/rack.rb

@@ -2,7 +2,8 @@ require "data_redactor"
 
 module DataRedactor
   # Namespace for the optional framework adapters under
-  # +lib/data_redactor/integrations/+ ({Logger}, +Rails+, {Rack}).
+  # +lib/data_redactor/integrations/+ ({Logger}, +Rails+, {Rack},
+    # {Claude}, {OpenAI}).
   #
   # Each adapter is soft-required — none load with +require "data_redactor"+;
   # +require+ only the one you need. They add no runtime gem dependencies and

data/lib/data_redactor/version.rb

@@ -1,4 +1,4 @@
 module DataRedactor
   # Current gem version. Follows {https://semver.org Semantic Versioning 2.0.0}.
-  VERSION = "0.10.1"
+  VERSION = "0.11.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: data_redactor
 version: !ruby/object:Gem::Version
-  version: 0.10.1
+  version: 0.11.0
 platform: x86_64-darwin
 authors:
 - Daniele Frisanco
@@ -114,7 +114,10 @@ files:
 - lib/data_redactor/3.3/data_redactor.bundle
 - lib/data_redactor/3.4/data_redactor.bundle
 - lib/data_redactor/4.0/data_redactor.bundle
+- lib/data_redactor/integrations/claude.rb
+- lib/data_redactor/integrations/llm_support.rb
 - lib/data_redactor/integrations/logger.rb
+- lib/data_redactor/integrations/openai.rb
 - lib/data_redactor/integrations/rack.rb
 - lib/data_redactor/integrations/rails.rb
 - lib/data_redactor/name_pattern.rb