RubyGems - findxpand - Versions diffs - 0.1.0 - Mend

findxpand 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (15) hide show

checksums.yaml +7 -0
data/README.md +325 -0
data/exe/findxpand +20 -0
data/findxpand.gemspec +71 -0
data/lib/findxpand/admin.rb +157 -0
data/lib/findxpand/auto.rb +607 -0
data/lib/findxpand/cli.rb +214 -0
data/lib/findxpand/encoding.rb +160 -0
data/lib/findxpand/middleware.rb +597 -0
data/lib/findxpand/railtie.rb +72 -0
data/lib/findxpand/rewrite.rb +628 -0
data/lib/findxpand/signature.rb +127 -0
data/lib/findxpand/store.rb +657 -0
data/lib/findxpand.rb +255 -0
metadata +62 -0

data/lib/findxpand/rewrite.rb ADDED Viewed

@@ -0,0 +1,628 @@
+# frozen_string_literal: true
+# The rewrite, and nothing else. No I/O, no framework, no state.
+#
+# Pure, because this is the half that has to agree byte for byte with the Node
+# middleware, the Python middleware and the Cloudflare Worker.
+# `test/conformance_test.rb` runs the engine's shared fixture — the same JSON
+# file the other packages run — so a change made here and not there is a test
+# failure rather than a page with two titles.
+#
+# **Only the head is rewritten, and that is what makes string surgery safe.** A
+# pattern applied to a whole document will happily rewrite a `<title>` inside
+# inline SVG or a code sample, which is a real way to corrupt a page and the
+# reason the edge worker uses a streaming parser instead. Bounding every head
+# edit to the slice between `<head>` and `</head>` removes that class of mistake
+# without asking a client to install a parser. The two body fields, `h1` and
+# `image_alt`, are matched by element rather than by text for the same reason.
+#
+# Standard library only, deliberately — in fact this file requires nothing at
+# all. It runs inside somebody else's application, and a middleware that drags
+# in a dependency tree is a middleware that can break their build.
+#
+# ## Ruby's own ways to get this wrong
+#
+# A port of a working Python file into Ruby is not a translation exercise; four
+# of Ruby's most ordinary idioms fail *silently* here, and each one passes the
+# shared fixture in at least one direction. They are named at the line that
+# avoids them, and collected once so a reader knows what to look for:
+#
+# * **`sub!` and `gsub!` return `nil` when nothing matched.** `out = out.sub!(re)
+#   { ... }` therefore assigns `nil` and *deletes the whole document* the first
+#   time an element is absent. Nothing below ever assigns from a bang method,
+#   and nothing below uses one at all. See `replace_inner`.
+# * **`sub`/`gsub` with a String replacement expand `\0 \1 \& \` \' \k<name>`.**
+#   A customer's title reading `Beans \& Brew` would splice the whole matched
+#   tag into the page. Every substitution here takes the block form, which
+#   interprets nothing, and `esc` uses a Hash replacement, which interprets
+#   nothing either.
+# * **`.` does not cross a newline without `/m`.** Real pages are pretty-printed
+#   and the naive `/(<title[^>]*>)(.*?)(<\/title>)/i` passes every single-line
+#   fixture case, then rewrites nothing on a real page — and worse, the
+#   *presence* test answers false, so a second `<title>` is appended to a page
+#   that already has one. Every multi-character wildcard here is `[\s\S]`, which
+#   needs no flag and cannot be lost in a refactor the way `/m` can.
+# * **`downcase` is Unicode-aware and changes length.** `'İ'.downcase` (U+0130,
+#   the Turkish capital I) is two characters, so an offset computed on a
+#   downcased copy is off by one against the original. Nothing here downcases
+#   anything: every lookup is a case-insensitive `Regexp`, which answers the
+#   same question over the string actually being sliced.
+#
+# ## One unit throughout: characters
+#
+# `Regexp#match(str, pos)`, `MatchData#begin`/`#end` and `String#[]` all count in
+# **characters** for a String. `bytesize` and `byteslice` count in bytes and
+# appear nowhere in this file. Mixing the two is not a rounding error: the head
+# of the Arabic document in the fixture is 124 characters and 141 bytes, so a
+# body offset taken in bytes starts the body slice 17 characters late, past
+# `<body><h1>`, and the h1 rule stops applying with nothing raising and nothing
+# counted (§14 — Arabic is the market, not an edge case). The HTTP layer is the
+# opposite: `content-length` is a byte count and is taken with `bytesize` in
+# `middleware.rb`. Each unit belongs to exactly one layer.
+#
+# Frozen string literals are on. Every accumulator below is created with `+''`,
+# which is an unfrozen copy; nothing appends to a literal, which would raise
+# `FrozenError` at the worst possible moment — inside somebody's request.
+module Findxpand
+  module Rewrite
+    # One field, and exactly how it is written into a head.
+    #
+    # `match_attribute` is what the element is *found* by and is not what the
+    # value is written to: a description is located by `name="description"` and
+    # written at `content`. Collapsing those two is the bug that produces
+    # `name="Roasted in Dubai"`.
+    HeadField = Struct.new(:key, :tag, :match_attribute, :match_value, :attribute)
+    # Mirrors `engine.fix.origin.HEAD_FIELDS`, in append order. The conformance
+    # fixture carries the same table and the test asserts they are identical,
+    # so this cannot drift silently.
+    HEAD_FIELDS = [
+      HeadField.new('title', 'title', '', '', '').freeze,
+      HeadField.new('meta_description', 'meta', 'name', 'description', 'content').freeze,
+      HeadField.new('canonical', 'link', 'rel', 'canonical', 'href').freeze,
+      HeadField.new('meta_robots', 'meta', 'name', 'robots', 'content').freeze,
+      HeadField.new('og_title', 'meta', 'property', 'og:title', 'content').freeze,
+      HeadField.new('og_description', 'meta', 'property', 'og:description', 'content').freeze,
+      HeadField.new('og_image', 'meta', 'property', 'og:image', 'content').freeze,
+      HeadField.new('twitter_title', 'meta', 'name', 'twitter:title', 'content').freeze,
+      HeadField.new('twitter_description', 'meta', 'name', 'twitter:description', 'content').freeze
+    ].freeze
+    # A run of attributes, in which a quoted value is skipped **whole**.
+    #
+    # This replaces the `[^>]*` that every implementation started with, and the
+    # difference is a corrupted page rather than a missed one. `[^>]*` stops at
+    # the first `>` in the tag whether or not it is inside quotes, so measured on
+    # `<meta name="description" content="Beans > 5 AED">` all three shipped
+    # packages emit `<meta name="description" content="New"> 5 AED">` — half an
+    # element replaced and the other half left on the page as text. `Home > Shop`
+    # is a breadcrumb and `Beans > 5 AED` is a price; neither is exotic, and the
+    # `<img>` form of the same bug emits two `alt=` attributes in one tag.
+    #
+    # The alternation is unambiguous at every position — a `"` can only enter the
+    # first branch, a `'` only the second, anything else only the third — so it
+    # does not backtrack and stays linear on a page-sized document. That matters:
+    # the fixture carries a ~9.7 KB case precisely because a pattern that
+    # backtracks super-linearly is invisible at 168 bytes and a stalled worker at
+    # 100 KB.
+    ATTRS = %q{(?:"[^"]*"|'[^']*'|[^>"'])*}
+    # Where a tag name ends. `` is not this test: it is ASCII in JavaScript
+    # and PCRE and Unicode here and in Python, so `<titleé>` was rewritten as a
+    # title by Node and PHP and refused by this file and the reference - two
+    # implementations writing into an element that does not exist. A tag name
+    # ends at whitespace, `/` or `>`, which is the same sentence in every
+    # language. Spelled the same way in all four since 2 Sep 2026: this file
+    # already gave the right answer, and a right answer reached by a different
+    # spelling is the next divergence waiting to happen (§20.1 rule 2).
+    NAME_END = '(?=[\s/>])'
+    # An HTML comment, matched so its interior can be blanked before the head is
+    # located. Non-greedy; the replacement preserves the character length.
+    COMMENT = /<!--[\s\S]*?-->/
+    # Where an attribute name is allowed to begin: at the start of the tag or
+    # after whitespace, never after a hyphen, an underscore, a colon or another
+    # word character.
+    #
+    # `\b` is not that test. It puts the boundary *after* the hyphen, so `\balt`
+    # matched `data-alt`, `\bsrc` matched `data-src` and `\bname` matched
+    # `data-name`. All three were measured on markup real pages ship:
+    #
+    # * `<img src="/a.jpg" data-alt="lazy">` had our text written into the
+    #   *placeholder*. The image still went out with no `alt`, the accessibility
+    #   finding the fix exists for was untouched, and the fix reported as applied.
+    # * `<img data-src="/a.jpg" src="/spacer.gif">` was keyed by the lazy URL, so
+    #   the description of a photograph nobody has loaded yet was written onto
+    #   the visible spacer GIF — the wrong caption on the wrong image, and on the
+    #   one image a reader's screen reader will actually reach.
+    # * `<meta data-name="description" content="Old">` was read as the page's own
+    #   description and *replaced*, deleting both the attribute and the content
+    #   while the real description stayed missing.
+    #
+    # Lazy-loading markup is on most pages we will ever rewrite. This is the
+    # ordinary case, not the exotic one.
+    # Where an attribute name may begin - stated positively since 2 Sep 2026.
+    # The negative form contained `\w`, which matches `é` and Arabic letters
+    # here and in Python and does not in JavaScript or PCRE without `/u`.
+    # Measured: `<img éalt="placeholder" src="/a.jpg">` had the caption written
+    # into `éalt` by Node and PHP while this file refused. Same answer, said in
+    # a way that cannot be read four ways.
+    ATTR_START = '(?<=[\s"\'])'
+    # `<head ...>`: any casing, any attributes. `<head profile=…>` is what every
+    # WordPress theme descended from the 2009 starter kit emits and
+    # `<head prefix="og: …">` is what an Open Graph plugin adds, so a finder
+    # matching the literal `<head>` rewrites nothing on precisely the sites we
+    # sell Open Graph fixes to.
+    HEAD_OPEN = /<head#{NAME_END}#{ATTRS}>/i
+    # The head's own closing tag, searched for **in the original string**.
+    #
+    # Never `out.downcase.index('</head>')`: that is an offset into a string
+    # which is not the one being sliced. `'İ'.downcase` (U+0130, the Turkish
+    # capital I with a dot) is two characters in Ruby — as it is in Python and in
+    # JavaScript — so one such letter in a title shifts every later offset by one
+    # and an appended tag lands as `<title>İstanbul</title><<meta …>/head>`:
+    # markup corrupted by a Turkish word. A GCC/Arabic product (§14) meets
+    # non-ASCII heads as routine, so this was never an exotic input.
+    #
+    # `\s*` because `</head >` closes a head too, and a literal lookup that
+    # misses it does not merely skip the append — `body_start` stays at zero and
+    # every body edit then runs over the head as well.
+    HEAD_CLOSE = %r{</head\s*>}i
+    IMG = /<img\b#{ATTRS}>/i
+    SRC = /#{ATTR_START}src\s*=\s*["']([^"']+)["']/i
+    ALT = /#{ATTR_START}alt\s*=\s*["'][^"']*["']/i
+    # `&` first, always. Escaping it last turns the `&lt;` just written into
+    # `&amp;lt;`.
+    #
+    # One pass over a character class rather than four chained `gsub`s, and the
+    # two are equivalent: the chain is order-dependent only because a later pass
+    # can re-read what an earlier one wrote, and a single pass reads each
+    # original character exactly once and never revisits its replacement. It is
+    # also the form that cannot be got wrong by a later edit reordering the
+    # chain.
+    #
+    # **A Hash replacement interprets nothing.** `gsub(re, '\&')` would splice
+    # the match back in; `gsub(re, hash)` has no replacement grammar at all,
+    # which is the same guarantee the block form gives everywhere else here.
+    ESCAPES = { '&' => '&amp;', '<' => '&lt;', '>' => '&gt;', '"' => '&quot;' }.freeze
+    ESCAPABLE = /[&<>"]/.freeze
+    # Escapes for both attribute and text context. Identical to the Worker's `esc`.
+    #
+    # An apostrophe is deliberately left alone: escaping it is visible to a
+    # reader — `Dubai&#39;s` in a SERP snippet is a bug a client reports — and it
+    # is safe unescaped only because `render_tag` writes double quotes.
+    def self.esc(value)
+      value.to_s.gsub(ESCAPABLE, ESCAPES)
+    end
+    # The pattern that finds one field's element, built once at load.
+    #
+    # Per-request `Regexp.new` on nine fields is a compile per field per page.
+    # Built here instead, keyed by the field's own key, because a table and a
+    # cache that can disagree about which pattern belongs to which field is
+    # §20.1 rule 2 in miniature.
+    # Both branches build through `Regexp.new` rather than one of them through a
+    # `/…/` literal. `return /<#{tag}…/i if cond` is legal and reads as a
+    # division to about half the people who meet it, and the two patterns are
+    # easier to compare when they are written the same way.
+    def self.element_pattern(field)
+      if field.match_attribute.empty?
+        return Regexp.new("<#{field.tag}\\b#{ATTRS}>[\\s\\S]*?<\\/#{field.tag}>",
+                          Regexp::IGNORECASE)
+      end
+      Regexp.new(
+        "<#{field.tag}#{NAME_END}#{ATTRS}#{ATTR_START}#{Regexp.escape(field.match_attribute)}" \
+        "\\s*=\\s*[\"']#{Regexp.escape(field.match_value)}[\"']#{ATTRS}>",
+        Regexp::IGNORECASE
+      )
+    end
+    ELEMENT_PATTERNS = HEAD_FIELDS.to_h { |field| [field.key, element_pattern(field)] }.freeze
+    # `(<tag …>)(text)(</tag>)`. `[\s\S]*?` and never `(.*?)`: `.` stops at a
+    # newline and a template engine indents.
+    def self.inner_pattern(tag)
+      Regexp.new("(<#{tag}\\b#{ATTRS}>)([\\s\\S]*?)(<\\/#{tag}>)", Regexp::IGNORECASE)
+    end
+    INNER_PATTERNS = { 'title' => inner_pattern('title'), 'h1' => inner_pattern('h1') }.freeze
+    # Double quotes around every value, always. Single-quoted attributes parse
+    # just as well, and nothing but this decides which of the two we emit — a
+    # value carrying an apostrophe is safe only because the quote is a double
+    # one, which is why `esc` leaves apostrophes alone.
+    def self.render_tag(field, value)
+      return "<#{field.tag}>#{esc(value)}</#{field.tag}>" if field.match_attribute.empty?
+      found_by = %(#{field.match_attribute}="#{esc(field.match_value)}")
+      written = %(#{field.attribute}="#{esc(value)}")
+      "<#{field.tag} #{found_by} #{written}>"
+    end
+    # Replace an element's inner text, keeping the opening tag's attributes.
+    #
+    # **Returns the input unchanged when the element is absent, and that is the
+    # single most dangerous line in this file to write in Ruby.** The idiomatic
+    # `out.sub!(pattern) { ... }` returns `nil` on a miss, so `out = out.sub!(…)`
+    # serves an empty page the first time a rule names an `h1` a page does not
+    # have — a page with no heading is not rare, it is most pages. PHP's
+    # `preg_replace` returns the subject untouched in the same situation and is
+    # fine, which is exactly why this has to be pinned rather than assumed.
+    #
+    # Sliced rather than substituted so that the *first* match is the one
+    # replaced and the value is never read as a replacement pattern.
+    def self.replace_inner(html, tag, value)
+      pattern = INNER_PATTERNS[tag] || inner_pattern(tag)
+      match = pattern.match(html)
+      return html if match.nil?
+      # `match.end(0)` can equal the length, and `String#[]` answers `""` there
+      # and `nil` past it. `.to_s` costs nothing on a String — it returns self —
+      # and removes a class of nil that Ruby reports as `undefined method for
+      # nil` three frames away from the cause.
+      html[0, match.begin(0)].to_s + match[1] + esc(value) + match[3] +
+        html[match.end(0)..-1].to_s
+    end
+    # The document with every comment's interior replaced by spaces.
+    #
+    # **Used only to *locate* the head, never to produce output.** The string
+    # returned is the same character length as the one passed in, so an index
+    # found in it is valid in the original - which is the whole technique, and
+    # the same discipline the rest of this file follows by never indexing one
+    # string with an offset found in another.
+    #
+    # A live defect in all four implementations until 2 Sep 2026: `head_start`
+    # landed on the first `<head` anywhere in the document, and a conditional
+    # comment or a commented-out block puts one there. Measured on
+    # `<!-- <head><title>fake</title></head> --><html><head><title>Real</title>`:
+    # the fix was written into the COMMENT and the real title left alone. What
+    # makes it worse than an ordinary miss is that §22.3 verification re-fetches,
+    # greps the raw HTML, finds the new string sitting in the comment, and
+    # passes - so the change reports as deployed and no crawler will ever see it.
+    # §3 rule 4 defeated by its own evidence.
+    #
+    # `gsub` with a BLOCK, never a replacement string: a replacement string
+    # expands `` and `\&`, and the one thing this method must not do is let a
+    # document's own bytes change its length.
+    def self.without_comments(html)
+      return html unless html.include?('<!--')
+      html.gsub(COMMENT) { |m| "<!--#{' ' * (m.length - 7)}-->" }
+    end
+    def self.rewrite_head(head, rule)
+      out = head
+      appended = +''
+      HEAD_FIELDS.each do |field|
+        value = rule[field.key]
+        # A value that is not a non-empty String is not a value. An empty string
+        # is "no fix", never "empty the field" — the difference between a fix
+        # that did not apply and a title we deleted — and a number is a
+        # malformed manifest, which §3 rule 7 fails closed rather than writing
+        # `<title>2026</title>` onto somebody's homepage.
+        next unless value.is_a?(String) && !value.empty?
+        # Presence is always asked of the *original* head. Asking the partially
+        # rewritten copy would let one field's inserted tag answer the next
+        # field's question, which is how a page ends up with two descriptions.
+        pattern = ELEMENT_PATTERNS.fetch(field.key)
+        present = pattern.match?(head)
+        if field.match_attribute.empty?
+          # The title keeps whatever attributes it had; only its text is ours.
+          if present
+            out = replace_inner(out, field.tag, value)
+          else
+            appended << render_tag(field, value)
+          end
+          next
+        end
+        tag = render_tag(field, value)
+        if present
+          # Block form. A String replacement would read `\&` and `\1` in a
+          # customer's description as backreferences and splice the matched tag
+          # into the page.
+          out = out.sub(pattern) { tag }
+        else
+          appended << tag
+        end
+      end
+      appended << hreflang_tags(head, rule['hreflang'])
+      # Additive, always (§9). Never bound to an existing ld+json element and
+      # never replacing one — overwriting somebody's Product block is a price we
+      # invented and their liability. Never escaped either: a JSON-LD block is
+      # script content, not markup, and `&amp;` inside it is a parse error
+      # rather than a safety measure. The two emitters either side of this line
+      # have opposite rules on purpose.
+      json_ld = rule['json_ld_additive']
+      if json_ld.is_a?(String) && !json_ld.empty?
+        appended << %(<script type="application/ld+json">#{json_ld}</script>)
+      end
+      out + appended
+    end
+    # The alternates worth emitting, as markup.
+    #
+    # Every guard here skips the *entry* and keeps the rest, and that is the
+    # whole point: a raise is caught upstream, counted as `rewrite_raised` and
+    # the page served untouched — so one malformed entry in one page's rule
+    # would turn off every rewrite on the site while `/status` named no rule.
+    # Measured, the Node package throws `value.replace is not a function` on a
+    # non-string `lang`; Python does not raise and is not therefore right, since
+    # it emits `hreflang="7"`, an invalid language tag. §3 rule 7, entry by
+    # entry and value by value.
+    def self.hreflang_tags(head, alternates)
+      out = +''
+      return out unless alternates.is_a?(Array)
+      alternates.each do |alternate|
+        next unless alternate.is_a?(Hash)
+        lang = alternate['lang']
+        href = alternate['href']
+        next unless lang.is_a?(String) && !lang.empty?
+        next unless href.is_a?(String) && !href.empty?
+        # A multilingual plugin may already have emitted this language. A second
+        # tag for one language is worse than none, so skip rather than
+        # duplicate — and case-insensitively, because BCP 47 is: `AR-ae` and
+        # `ar-AE` are the same language to every consumer of the tag.
+        # `ATTR_START` here for the reason it is everywhere else, and this one
+        # was measured propagating: Node carried the guard, the Python reference
+        # did not, and this port inherited the defect from the copy that never
+        # learned it - `<link data-hreflang="ar-AE">` suppressed the alternate
+        # here and not there (§20.1 rule 2, caught mid-propagation).
+        next if /#{ATTR_START}hreflang\s*=\s*["']#{Regexp.escape(lang)}["']/i.match?(head)
+        out << %(<link rel="alternate" hreflang="#{esc(lang)}" href="#{esc(href)}">)
+      end
+      out
+    end
+    def self.rewrite_body(body, rule)
+      out = body
+      heading = rule['h1']
+      out = replace_inner(out, 'h1', heading) if heading.is_a?(String) && !heading.empty?
+      alts = rule['image_alt']
+      return out unless alts.is_a?(Hash)
+      out.gsub(IMG) do |tag|
+        src = SRC.match(tag)
+        # `MatchData` is `nil`, not `false` and not an exception. An `<img>` with
+        # no `src` — a template placeholder, a `<img srcset>` — is left alone.
+        next tag if src.nil?
+        # **An exact key, never `src.include?(key)`.** `/a.jpg` is a suffix of
+        # `/images/a.jpg`, so a substring lookup captions the roastery counter as
+        # a single bean; and the key is the attribute value byte for byte, so an
+        # implementation that helpfully strips a resize query or resolves a CDN
+        # host looks the rule up under a key the engine never wrote. `Hash#[]`
+        # answers `nil` for an unkeyed image, which is left exactly as it was —
+        # falling back to the first value in the table would write a confident
+        # caption about a different picture.
+        alt = alts[src[1]]
+        next tag unless alt.is_a?(String) && !alt.empty?
+        if ALT.match?(tag)
+          tag.sub(ALT) { %(alt="#{esc(alt)}") }
+        elsif tag.end_with?('/>')
+          # XHTML. Rails, Blade and Twig emit these constantly, and appending
+          # before the slash produces `<img src="/b.jpg" / alt="…">`, which no
+          # parser reads as an image at all. The space before `/>` is inserted
+          # rather than preserved, so `"/c.jpg"/>` comes back well formed too.
+          %(#{tag[0..-3].rstrip} alt="#{esc(alt)}" />)
+        else
+          %(#{tag[0..-2].rstrip} alt="#{esc(alt)}">)
+        end
+      end
+    end
+    # Apply one page's rules to one HTML document.
+    #
+    # Returns the input unchanged when there is nothing to do — an empty rule
+    # must never read as "blank the field", which is the difference between a fix
+    # that did not apply and a page we emptied.
+    #
+    # The guard is "a non-empty Hash", not "not nil". A manifest arrives over
+    # HTTP, so the value here is whatever was sent: `[]` is falsy in the other
+    # implementations and is not a mapping, and a guard written `rule.nil?`
+    # reaches `rule['title']` on an Array, which raises `TypeError: no implicit
+    # conversion of String into Integer` — caught upstream, counted as
+    # `rewrite_raised`, and every page on the site served untouched over one bad
+    # rule.
+    def self.transform(html, rule)
+      return html unless rule.is_a?(Hash) && !rule.empty?
+      out = html
+      # **One lookup for `</head>`, anchored at the head's own open tag**, and
+      # one index deciding both slices. There were two — one from the open tag
+      # to cut the head, one from position zero to decide where the body begins
+      # — and they disagreed the moment the string `</head>` appeared earlier in
+      # the document, in a comment or a conditional comment. The body slice then
+      # began *before* the head, so every body edit ran over the head as well:
+      # measured, `<!-- </head> --><html><head><noscript><img src="/a.jpg">…`
+      # had the head's own image rewritten, undoing the rule that leaves images
+      # in a head alone. §20.1 rule 2 in miniature — one question, two
+      # implementations, and only one of them was ever taught where a head ends.
+      body_start = 0
+      # Located against the masked copy, sliced out of the real one. Same
+      # character length, so the offsets carry over - see `without_comments`.
+      masked = without_comments(out)
+      open_tag = HEAD_OPEN.match(masked)
+      unless open_tag.nil?
+        head_start = open_tag.end(0)
+        closing = HEAD_CLOSE.match(masked, head_start)
+        unless closing.nil?
+          head = rewrite_head(out[head_start...closing.begin(0)].to_s, rule)
+          out = out[0, head_start].to_s + head + out[closing.begin(0)..-1].to_s
+          # The rewritten head is a different length from the one that was
+          # measured, so the body slice cannot reuse `closing.begin(0)`. It is
+          # the same character: the new head ends exactly where it now sits.
+          # Counted in characters, like every other offset in this file.
+          body_start = head_start + head.length
+        end
+      end
+      # Body edits still run on a document with no head, and on one whose head is
+      # never closed: a fragment rendered by a partial still has an h1 worth
+      # correcting, and the head branch above simply had nothing to attach to.
+      out[0, body_start].to_s + rewrite_body(out[body_start..-1].to_s, rule)
+    end
+    # Trimmed from both ends of a URL before it is read as a key: U+0000-U+0020,
+    # which is every C0 control plus the space.
+    #
+    # **Trim the ends, preserve the middle**, and the asymmetry is the whole
+    # rule. A space or a tab wrapped around a path is somebody's typo in a
+    # hand-written manifest and never part of the path, so removing it files the
+    # rule where the author meant it. A control character *inside* a path is the
+    # opposite case: no HTTP request target may carry a raw tab, CR or LF, so a
+    # path with one is a key no request can ever produce — and removing the
+    # newline would file that rule under `/beans`, rewriting a real page from a
+    # manifest entry nobody approved for it. Left alone it matches nothing,
+    # which is the direction §3 rule 7 asks a write to fail in.
+    #
+    # `String#strip` is **not** this set. It removes NUL, tab, newline, vertical
+    # tab, form feed, carriage return and space, and leaves U+0001-U+0008 and
+    # U+000E-U+001F where they are — so a path wrapped in a stray U+0001 would
+    # key differently here than in the engine, which is the one thing this
+    # function exists not to do.
+    TRIM_ENDS = /\A[\x00-\x20]+|[\x00-\x20]+\z/
+    def self.trim_ends(value)
+      value.gsub(TRIM_ENDS) { '' }
+    end
+    # The same string, in an encoding the lookup and the patterns can use.
+    #
+    # **A manifest key is UTF-8 and a request path is bytes, and Ruby will not
+    # call those equal.** `JSON.parse` tags every key UTF-8; the Rack SPEC has
+    # request values arrive as ASCII-8BIT. For an ASCII path the two compare
+    # equal and nothing is wrong. For `/café` — or any Arabic path, which is our
+    # market (§14) — `"/café".b.eql?("/café")` is **false**, so `pages[key]`
+    # misses, the page is served untouched, no counter moves and `/status`
+    # reports health. That is the `normalise_path` disagreement all over again,
+    # in one language rather than four, and it would have been invisible for the
+    # same reason.
+    #
+    # Bytes that are not valid UTF-8 are left as bytes rather than re-tagged:
+    # they cannot equal a manifest key whatever we do, and a UTF-8 string
+    # holding invalid bytes makes every `Regexp` below raise
+    # `ArgumentError: invalid byte sequence` — on the read path, which is a 500
+    # on the customer's homepage over a rule key.
+    def self.readable(url)
+      string = url.to_s
+      return string if string.encoding == ::Encoding::UTF_8 && string.valid_encoding?
+      candidate = string.dup.force_encoding(::Encoding::UTF_8)
+      candidate.valid_encoding? ? candidate : string.dup.force_encoding(::Encoding::BINARY)
+    end
+    # The rules table's key for a URL or a request path: the path, and nothing
+    # else.
+    #
+    # No scheme, no host, no query, no fragment, no trailing slash; `""` and
+    # `"/"` are both `"/"`; the result always begins with `/`. Percent-escapes
+    # are neither decoded nor encoded — `/caf%C3%A9` and `/café` are different
+    # keys to a request router, and translating one into the other files a rule
+    # under a path no request will ever produce.
+    #
+    # **This is the one function four implementations have to compute
+    # identically, and until 1 Sep 2026 they did not.** The engine files keys
+    # from an absolute URL, because that is what a `Fix` carries, while the Node
+    # and Python middlewares stripped the query, the fragment and a trailing
+    # slash and never touched the scheme and host — under a comment that said
+    # "no host". So the engine filed `/beans` and a middleware asked for
+    # `https://shop.example/beans`. Seventeen inputs are pinned in
+    # `middleware/conformance/cases.json` under `paths` and every suite runs
+    # them; nine of the seventeen disagreed.
+    #
+    # **The failure this produces has no symptom, which is why it survived.** A
+    # key that does not match is not an error anywhere: `Store#rule_for` returns
+    # nil, the response is passed through untouched, nothing calls
+    # `Store#record`, so `considered` stays at zero — and `degraded_reason`
+    # reports a problem only when `considered > 0`. A site with a full manifest
+    # and not one rule landing therefore reports *healthy* (§20.1 rule 3:
+    # nothing examined read as nothing wrong).
+    #
+    # Hand-split rather than handed to `URI.parse`, which is the Ruby equivalent
+    # of the trap the Node comment describes for `new URL()`: `URI.parse` raises
+    # `URI::InvalidURIError` on inputs a stranger can send (`//[`, a raw space, a
+    # `|`), and this runs on the read path before the application is reached, so
+    # a raise here is a 500 on the customer's site over a rule key. It also
+    # re-encodes and resolves `.`/`..`, and a rule key must be what the engine
+    # saw rather than what it could reduce.
+    def self.normalise_path(url)
+      # Trimmed once, at the top, so every branch below — the refusal
+      # included — answers about the same string.
+      raw = trim_ends(readable(url))
+      rest = raw
+      # Fragment first, then query: a fragment may itself contain a `?`
+      # (`/beans#a?b`), and splitting on `?` first leaves `a` behind in the key.
+      #
+      # `sub` and not `split`: `''.split('#')` is `[]`, so `[0]` is `nil` rather
+      # than `''`, and `""` is one of the pinned inputs. `[\s\S]` and not `.`
+      # for the same reason as everywhere else in this file.
+      rest = rest.sub(/#[\s\S]*\z/) { '' }
+      rest = rest.sub(/\?[\s\S]*\z/) { '' }
+      # A scheme is one letter followed by letters, digits, `+`, `-` or `.`, then
+      # a colon (RFC 3986 §3.1). Nothing else before a colon qualifies, so
+      # `/beans:roast` keeps its colon and stays an ordinary path.
+      # Case-insensitive because `HTTPS://SHOP.EXAMPLE/Beans` is a shape
+      # manifests really carry, and its key must come back `/Beans` with the
+      # path's own casing intact — a scheme and a host are case-insensitive, a
+      # path is not.
+      rest = rest.sub(/\A[a-zA-Z][a-zA-Z0-9+.\-]*:/) { '' }
+      if rest.start_with?('//')
+        # What is left of an absolute URL once its scheme is gone, or a
+        # protocol-relative URL written that way. The authority ends at the
+        # first `/` after it; the query and fragment that could otherwise end it
+        # are already gone. `String#index` answers `nil`, never `-1`.
+        authority = rest[2..-1].to_s.split('/', 2).first.to_s
+        # **A key we cannot compute must never come out as `/`.** Square
+        # brackets have to be balanced and the first `]` has to follow the first
+        # `[`; `//[` and `//[::1` have no closing bracket and `//]x[` has them
+        # the wrong way round. Under the surgery below all three end at "no `/`
+        # after the authority", which produces `""` and then `/` — the
+        # homepage's rule handed to a malformed request any stranger can send.
+        # Refused by returning the trimmed input instead: a string no rule is
+        # ever filed under and no request router will match, so it matches
+        # nothing and looks like nothing. `//[::1]/beans` is a real IPv6
+        # authority and parses normally.
+        if authority.count('[') != authority.count(']') ||
+           (authority.include?(']') && authority.index(']').to_i < authority.index('[').to_i)
+          return raw
+        end
+        slash = rest.index('/', 2)
+        rest = slash.nil? ? '' : rest[slash..-1].to_s
+      end
+      trimmed = rest.sub(/\/+\z/) { '' }
+      return '/' if trimmed.empty?
+      # A key is a path, and a path starts at the root. `beans` used to key as
+      # `/` in the engine — a fake origin prefix swallowed it as a hostname — so
+      # a relative path collapsed onto the homepage and a fix for one page
+      # rewrote another. Normalised rather than filed under a key no request can
+      # produce.
+      trimmed.start_with?('/') ? trimmed : "/#{trimmed}"
+    end
+  end
+end