RubyGems - brotli_splice - Versions diffs - 0.1.1 - Mend

brotli_splice 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (9) hide show

checksums.yaml +7 -0
data/LICENSE.md +21 -0
data/README.md +174 -0
data/brotli_splice.gemspec +40 -0
data/ext/brotli_splice/brotli_splice.c +197 -0
data/ext/brotli_splice/extconf.rb +14 -0
data/lib/brotli_splice/version.rb +5 -0
data/lib/brotli_splice.rb +13 -0
metadata +60 -0

checksums.yaml ADDED Viewed

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: '0810be973e8e1ad5dd1b28a4733754bd2ec87f6e5d5c5c3e6a29d44589e505ab'
+  data.tar.gz: cc459d36cbf640e4fb47d426dbce7dcb68f4458cf6daf49080e7fa9d67d5e0bd
+SHA512:
+  metadata.gz: 8af6e37050b29d440cf25ae1f97ea502d2dffa513516e8592216fd16664a3a68c8796121704dde5f10618eea4353ba39a9a117820f35de34a281a24775913b72
+  data.tar.gz: 5742e2a3183a99e2965d8dd2e8dbe3e3f93c8c7ea1cbe6e2bcfb065bf3ab82c5eb88b7f32590a4f36b98a6b18bf17025fa14600260e2334b2b797e1c0ece604f

data/LICENSE.md ADDED Viewed

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+Copyright 2026-present, Shopify Inc.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md ADDED Viewed

@@ -0,0 +1,174 @@
+# BrotliSplice
+A Ruby C extension that produces Brotli-compressed streams with a fixed-length
+**spliceable slot** — a region whose raw bytes can be overwritten without
+decompressing or re-encoding the rest of the stream.
+## Use case
+You have a large HTML document with a small secret (token, nonce, personalized
+data) embedded at a known position. You want to:
+1. Brotli-compress the document **once** at build/deploy time
+2. Swap in a different secret **per request** with a simple `memcpy`
+## How it works
+The stream is split into three chunks:
+```
+[chunk1: compressed part1]  [chunk2: uncompressed slot]  [chunk3: compressed part3]
+         (FLUSH)               (raw bytes)              (STREAM_OFFSET + FINISH)
+```
+- **chunk1** — standard Brotli compression of everything before the secret
+- **chunk2** — an uncompressed meta-block (3-byte header + raw data), whose
+  bytes sit at a known offset in the stream
+- **chunk3** — standard Brotli compression of everything after the secret,
+  produced by a second encoder with `BROTLI_PARAM_STREAM_OFFSET` so that
+  distance calculations, the ring buffer, and the WBITS header are all correct
+The last 2 bytes of the secret slot are reserved as a fixed **context suffix**
+(`\r\n`). These are baked into chunk3 and cannot be changed — the Brotli encoder
+uses the preceding 2 bytes as literal context when compressing part3. The
+replaceable portion is therefore `secret_length - 2` bytes.
+Compression overhead vs standard Brotli is small — typically 100–200 bytes
+for realistic HTML documents. Two sources of overhead:
+1. **FLUSH boundary** — the encoder can't carry pending matches from part1
+   across the slot. A very small part1 may show proportionally higher
+   overhead since this cost isn't amortized.
+2. **Part3 can't back-reference part1** — `STREAM_OFFSET` starts a fresh
+   encoder with an empty hash table, so part3 compresses in isolation.
+   If part1 and part3 share significant repeated content (e.g. identical
+   nav markup in header and footer), this gap widens. For typical HTML
+   where each section has plenty of self-repetition, the loss is small.
+**Recommendation:** place the secret early in the document (e.g. in a
+`<script>` tag right after `<head>`) so that part1 is small. This keeps
+the FLUSH cost negligible and gives part3 nearly the entire document to
+compress against itself.
+## Installation
+Requires the Brotli C library (`libbrotlienc`, `libbrotlidec`, `libbrotlicommon`).
+Add the gem to your Gemfile:
+```ruby
+gem "brotli_splice"
+```
+```sh
+# macOS (Homebrew)
+brew install brotli
+# Build and install the gem from this checkout
+gem build brotli_splice.gemspec
+gem install ./brotli_splice-*.gem
+```
+## Releasing
+This gem is configured for public release on RubyGems.org.
+After RubyGems Trusted Publishing is configured for this repository, publishing
+a GitHub release runs the `Release` workflow and publishes the gem.
+```sh
+gem build brotli_splice.gemspec
+gem push brotli_splice-*.gem
+```
+Alternatively, use Bundler's release task after tagging the version:
+```sh
+bundle exec rake release
+```
+For local extension development without installing the gem:
+```sh
+cd ext/brotli_splice
+ruby extconf.rb
+make
+cd ../..
+ruby -Ilib test_ext.rb
+```
+## API
+### `BrotliSplice.encode(html, secret_offset, secret_length, quality: 11) → Hash`
+Compress `html` with a spliceable slot at the given byte position.
+**Parameters:**
+- `html` — the full HTML string (binary encoding)
+- `secret_offset` — byte offset where the replaceable section starts
+- `secret_length` — total byte length of the replaceable section (must be > 2)
+- `quality:` — Brotli quality level, 0–11 (default: 11)
+**Returns** a Hash:
+```ruby
+{
+  data:           String,   # the Brotli-compressed stream
+  secret_offset:  Integer,  # byte offset of the replaceable data within the compressed stream
+  secret_length:  Integer,  # replaceable byte count (= secret_length - 2)
+  context_suffix: String,   # the 2 fixed context bytes ("\r\n")
+}
+```
+### `BrotliSplice.replace(compressed_data, new_secret, secret_offset, secret_length) → String`
+Replace the secret in a compressed stream. Returns a new string with the
+swapped bytes. This is a pure `memcpy` — no compression or decompression.
+**Parameters:**
+- `compressed_data` — the Brotli stream from `encode`
+- `new_secret` — replacement content, must be exactly `secret_length` bytes
+- `secret_offset` — from the `encode` result
+- `secret_length` — from the `encode` result
+## Example
+```ruby
+require "brotli_splice"
+require "brotli" # for verification
+html = "<html>...TOKEN_PLACEHOLDER_HERE...</html>"
+token_offset = html.index("TOKEN_PLACEHOLDER_HERE")
+token_length = "TOKEN_PLACEHOLDER_HERE".bytesize + 2  # +2 for context suffix
+# Compress once
+result = BrotliSplice.encode(html, token_offset, token_length, quality: 11)
+# Per-request: swap in the real token (must be result[:secret_length] bytes)
+real_token = "user-specific-secret".ljust(result[:secret_length], "\0")
+response = BrotliSplice.replace(result[:data], real_token,
+                                 result[:secret_offset], result[:secret_length])
+# The response is a valid Brotli stream ready to send
+decoded = Brotli.inflate(response)
+# => "<html>...user-specific-secret\0\0\r\n...</html>"
+```
+## Decoded output
+The decoded stream is the original HTML with the secret region replaced by:
+```
+[replaceable bytes (secret_length - 2)] [context suffix "\r\n"]
+```
+The 2-byte context suffix replaces the last 2 bytes of the original secret
+region. Design your placeholder to account for this (e.g. make it 2 bytes
+longer, or place the suffix where a line break is natural).
+## Limitations
+- The replaceable slot must be ≤ 65534 bytes (64KB minus the 2-byte context)
+- The replacement must be **exactly** `secret_length` bytes (no length changes)
+- The last 2 bytes of the original secret region become `\r\n` in the output
+- Only one spliceable slot per stream (encode with multiple slots would require
+  chaining encoders)

data/brotli_splice.gemspec ADDED Viewed

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+require_relative "lib/brotli_splice/version"
+Gem::Specification.new do |spec|
+  spec.name = "brotli_splice"
+  spec.version = BrotliSplice::VERSION
+  spec.summary = "Create Brotli streams with fixed-length spliceable slots"
+  spec.description = <<~DESC
+    BrotliSplice is a Ruby C extension that creates Brotli-compressed streams
+    containing a fixed-length uncompressed slot. The slot can be overwritten
+    with a simple byte copy, making it useful for injecting fixed-size secrets
+    or tokens into pre-compressed HTML responses.
+  DESC
+  spec.authors = ["Shopify"]
+  spec.email = ["gems@shopify.com"]
+  spec.homepage = "https://github.com/Shopify/brotli_splice"
+  spec.license = "MIT"
+  spec.required_ruby_version = ">= 3.1"
+  spec.require_paths = ["lib"]
+  spec.extensions = ["ext/brotli_splice/extconf.rb"]
+  spec.metadata["allowed_push_host"] = "https://rubygems.org"
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = "#{spec.homepage}/tree/main"
+  spec.metadata["changelog_uri"] = "#{spec.homepage}/releases"
+  spec.metadata["rubygems_mfa_required"] = "true"
+  spec.files = Dir.chdir(__dir__) do
+    Dir[
+      "lib/**/*.rb",
+      "ext/brotli_splice/**/*.{c,h,rb}",
+      "README.md",
+      "LICENSE.md",
+      "brotli_splice.gemspec",
+    ].reject { |file| File.directory?(file) }
+  end
+end

data/ext/brotli_splice/brotli_splice.c ADDED Viewed

@@ -0,0 +1,197 @@
+/*
+ * BrotliSplice — Ruby C extension for spliceable Brotli encoding.
+ *
+ * Produces a Brotli stream with a fixed-length uncompressed slot whose
+ * raw bytes can be overwritten without re-encoding the rest.
+ *
+ * Uses the standard Brotli streaming API with BROTLI_PARAM_STREAM_OFFSET.
+ */
+#include <ruby.h>
+#include <ruby/encoding.h>
+#include <brotli/encode.h>
+#include <brotli/decode.h>
+#include <string.h>
+static VALUE mBrotliSplice;
+static VALUE eBrotliSpliceError;
+#define CTX_LEN 2
+static const uint8_t CTX_BYTES[CTX_LEN] = { '\r', '\n' };
+/* ── Streaming encoder helper ─────────────────────────────────── */
+static size_t encoder_feed(BrotliEncoderState *s, BrotliEncoderOperation op,
+                           const uint8_t *in, size_t in_len,
+                           uint8_t *out, size_t out_cap) {
+    size_t avail_in = in_len, avail_out = out_cap, total = 0;
+    const uint8_t *next_in = in;
+    uint8_t *next_out = out;
+    if (!BrotliEncoderCompressStream(s, op,
+            &avail_in, &next_in, &avail_out, &next_out, &total)) {
+        return 0;
+    }
+    return out_cap - avail_out;
+}
+/* ── Build uncompressed meta-block (ISLAST=0) ─────────────────── */
+static size_t make_uncompressed_block(uint8_t *out,
+                                       const uint8_t *data, size_t len) {
+    if (len == 0 || len > 65536) return 0;
+    uint32_t r = (uint32_t)(len - 1);
+    uint32_t nibbles = 0;
+    if (len > (1u << 16)) nibbles = (len > (1u << 20)) ? 2 : 1;
+    uint32_t bits = (nibbles << 1) | (r << 3) | (1u << (19 + 4 * nibbles));
+    out[0] = (uint8_t)bits;
+    out[1] = (uint8_t)(bits >> 8);
+    out[2] = (uint8_t)(bits >> 16);
+    size_t hdr = 3;
+    if (nibbles == 2) { out[3] = (uint8_t)(bits >> 24); hdr = 4; }
+    memcpy(out + hdr, data, len);
+    return hdr + len;
+}
+/*
+ * call-seq:
+ *   BrotliSplice.encode(html, secret_offset, secret_length, quality: 11) -> Hash
+ *
+ * Brotli-encode +html+ with a spliceable slot at the given position.
+ *
+ * The slot's last 2 bytes are reserved as a fixed context suffix ("\r\n")
+ * and cannot be replaced. The replaceable portion is +secret_length - 2+
+ * bytes.
+ *
+ * Returns:
+ *   { data:           String,   # the Brotli-encoded stream
+ *     secret_offset:  Integer,  # byte offset of the replaceable data in the stream
+ *     secret_length:  Integer,  # replaceable byte count (secret_length - 2)
+ *     context_suffix: String }  # the 2 fixed context bytes ("\r\n")
+ */
+static VALUE rb_brotli_splice_encode(int argc, VALUE *argv, VALUE self) {
+    VALUE rb_html, rb_sec_off, rb_sec_len, rb_opts;
+    rb_scan_args(argc, argv, "3:", &rb_html, &rb_sec_off, &rb_sec_len, &rb_opts);
+    Check_Type(rb_html, T_STRING);
+    const uint8_t *html = (const uint8_t *)RSTRING_PTR(rb_html);
+    size_t html_len = RSTRING_LEN(rb_html);
+    size_t sec_off = NUM2SIZET(rb_sec_off);
+    size_t sec_total = NUM2SIZET(rb_sec_len);
+    if (sec_total <= CTX_LEN)
+        rb_raise(eBrotliSpliceError, "secret_length must be > %d", CTX_LEN);
+    if (sec_off + sec_total > html_len)
+        rb_raise(eBrotliSpliceError, "secret_offset + secret_length exceeds html size");
+    int quality = 11;
+    if (!NIL_P(rb_opts)) {
+        VALUE q = rb_hash_aref(rb_opts, ID2SYM(rb_intern("quality")));
+        if (!NIL_P(q)) quality = NUM2INT(q);
+    }
+    /* Split into part1 / secret_body / context / part3 */
+    const uint8_t *part1 = html;
+    size_t p1_len = sec_off;
+    size_t sec_body = sec_total - CTX_LEN;
+    const uint8_t *secret = html + sec_off;          /* secret_body bytes */
+    const uint8_t *part3 = html + sec_off + sec_total;
+    size_t p3_len = html_len - sec_off - sec_total;
+    /* Allocate output buffer */
+    size_t out_cap = html_len + 65536;
+    uint8_t *out = xmalloc(out_cap);
+    size_t pos = 0;
+    /* ── Chunk 1: compress part1 with FLUSH ──────────────────────── */
+    BrotliEncoderState *enc1 = BrotliEncoderCreateInstance(NULL, NULL, NULL);
+    if (!enc1) { xfree(out); rb_raise(eBrotliSpliceError, "encoder creation failed"); }
+    BrotliEncoderSetParameter(enc1, BROTLI_PARAM_QUALITY, quality);
+    BrotliEncoderSetParameter(enc1, BROTLI_PARAM_LGWIN, 22);
+    size_t c1 = encoder_feed(enc1, BROTLI_OPERATION_FLUSH,
+                              part1, p1_len, out + pos, out_cap - pos);
+    BrotliEncoderDestroyInstance(enc1);
+    if (c1 == 0 && p1_len > 0) {
+        xfree(out);
+        rb_raise(eBrotliSpliceError, "chunk1 encoding failed");
+    }
+    pos += c1;
+    /* ── Chunk 2: uncompressed meta-block for secret body ────────── */
+    size_t sec_data_offset = pos + 3;  /* 3-byte header for ≤64KB */
+    size_t c2 = make_uncompressed_block(out + pos, secret, sec_body);
+    if (c2 == 0) { xfree(out); rb_raise(eBrotliSpliceError, "chunk2 failed"); }
+    pos += c2;
+    /* ── Chunk 3: context bytes + part3, STREAM_OFFSET ───────────── */
+    BrotliEncoderState *enc2 = BrotliEncoderCreateInstance(NULL, NULL, NULL);
+    if (!enc2) { xfree(out); rb_raise(eBrotliSpliceError, "encoder2 creation failed"); }
+    BrotliEncoderSetParameter(enc2, BROTLI_PARAM_QUALITY, quality);
+    BrotliEncoderSetParameter(enc2, BROTLI_PARAM_LGWIN, 22);
+    BrotliEncoderSetParameter(enc2, BROTLI_PARAM_STREAM_OFFSET,
+                               (uint32_t)(p1_len + sec_body));
+    size_t c3_in_len = CTX_LEN + p3_len;
+    uint8_t *c3_in = xmalloc(c3_in_len);
+    memcpy(c3_in, CTX_BYTES, CTX_LEN);
+    memcpy(c3_in + CTX_LEN, part3, p3_len);
+    size_t c3 = encoder_feed(enc2, BROTLI_OPERATION_FINISH,
+                              c3_in, c3_in_len, out + pos, out_cap - pos);
+    BrotliEncoderDestroyInstance(enc2);
+    xfree(c3_in);
+    if (c3 == 0) { xfree(out); rb_raise(eBrotliSpliceError, "chunk3 encoding failed"); }
+    pos += c3;
+    /* Build result */
+    VALUE data = rb_str_new((char *)out, pos);
+    rb_enc_associate(data, rb_ascii8bit_encoding());
+    xfree(out);
+    VALUE result = rb_hash_new();
+    rb_hash_aset(result, ID2SYM(rb_intern("data")), data);
+    rb_hash_aset(result, ID2SYM(rb_intern("secret_offset")), SIZET2NUM(sec_data_offset));
+    rb_hash_aset(result, ID2SYM(rb_intern("secret_length")), SIZET2NUM(sec_body));
+    rb_hash_aset(result, ID2SYM(rb_intern("context_suffix")),
+                 rb_str_new((const char *)CTX_BYTES, CTX_LEN));
+    return result;
+}
+/*
+ * call-seq:
+ *   BrotliSplice.replace(compressed_data, new_secret, secret_offset, secret_length) -> String
+ *
+ * Replace the secret in a compressed stream. +new_secret+ must be
+ * exactly +secret_length+ bytes. Returns a new string.
+ */
+static VALUE rb_brotli_splice_replace(VALUE self,
+                                       VALUE rb_data, VALUE rb_secret,
+                                       VALUE rb_offset, VALUE rb_length) {
+    Check_Type(rb_data, T_STRING);
+    Check_Type(rb_secret, T_STRING);
+    size_t offset = NUM2SIZET(rb_offset);
+    size_t length = NUM2SIZET(rb_length);
+    if ((size_t)RSTRING_LEN(rb_secret) != length)
+        rb_raise(eBrotliSpliceError,
+                 "secret length %ld != expected %zu",
+                 RSTRING_LEN(rb_secret), length);
+    if (offset + length > (size_t)RSTRING_LEN(rb_data))
+        rb_raise(eBrotliSpliceError, "offset+length exceeds data size");
+    VALUE result = rb_str_dup(rb_data);
+    rb_str_modify(result);
+    memcpy(RSTRING_PTR(result) + offset, RSTRING_PTR(rb_secret), length);
+    return result;
+}
+void Init_brotli_splice(void) {
+    mBrotliSplice = rb_define_module("BrotliSplice");
+    eBrotliSpliceError = rb_define_class_under(mBrotliSplice, "Error", rb_eRuntimeError);
+    rb_define_singleton_method(mBrotliSplice, "encode",
+                               rb_brotli_splice_encode, -1);
+    rb_define_singleton_method(mBrotliSplice, "replace",
+                               rb_brotli_splice_replace, 4);
+}

data/ext/brotli_splice/extconf.rb ADDED Viewed

@@ -0,0 +1,14 @@
+require "mkmf"
+# Find brotli headers and libraries (Homebrew or system)
+dir_config("brotli",
+  ["/opt/homebrew/include", "/usr/local/include"],
+  ["/opt/homebrew/lib", "/usr/local/lib"])
+abort "missing brotli/encode.h" unless have_header("brotli/encode.h")
+abort "missing brotli/decode.h" unless have_header("brotli/decode.h")
+abort "missing libbrotlienc"    unless have_library("brotlienc")
+abort "missing libbrotlidec"    unless have_library("brotlidec")
+abort "missing libbrotlicommon" unless have_library("brotlicommon")
+create_makefile("brotli_splice/brotli_splice")

data/lib/brotli_splice/version.rb ADDED Viewed

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+module BrotliSplice
+  VERSION = "0.1.1"
+end

data/lib/brotli_splice.rb ADDED Viewed

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+require_relative "brotli_splice/version"
+begin
+  require "brotli_splice/brotli_splice"
+rescue LoadError => error
+  begin
+    require_relative "../ext/brotli_splice/brotli_splice"
+  rescue LoadError
+    raise error
+  end
+end

metadata ADDED Viewed

@@ -0,0 +1,60 @@
+--- !ruby/object:Gem::Specification
+name: brotli_splice
+version: !ruby/object:Gem::Version
+  version: 0.1.1
+platform: ruby
+authors:
+- Shopify
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2026-06-25 00:00:00.000000000 Z
+dependencies: []
+description: |
+  BrotliSplice is a Ruby C extension that creates Brotli-compressed streams
+  containing a fixed-length uncompressed slot. The slot can be overwritten
+  with a simple byte copy, making it useful for injecting fixed-size secrets
+  or tokens into pre-compressed HTML responses.
+email:
+- gems@shopify.com
+executables: []
+extensions:
+- ext/brotli_splice/extconf.rb
+extra_rdoc_files: []
+files:
+- LICENSE.md
+- README.md
+- brotli_splice.gemspec
+- ext/brotli_splice/brotli_splice.c
+- ext/brotli_splice/extconf.rb
+- lib/brotli_splice.rb
+- lib/brotli_splice/version.rb
+homepage: https://github.com/Shopify/brotli_splice
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+  homepage_uri: https://github.com/Shopify/brotli_splice
+  source_code_uri: https://github.com/Shopify/brotli_splice/tree/main
+  changelog_uri: https://github.com/Shopify/brotli_splice/releases
+  rubygems_mfa_required: 'true'
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.22
+signing_key:
+specification_version: 4
+summary: Create Brotli streams with fixed-length spliceable slots
+test_files: []