vicary 0.2.4 → 0.2.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 585c77607d6c331c3cebe14a5da44463a0231f7c3a15e5c1ba68f79d8b9f588b
-  data.tar.gz: 883c07a66721ce9c67fb0cfe1f1ea16ce486decd6be62164b1ebbbf53fec9bf0
+  metadata.gz: 537ad53f4299fd2a3cdfc7bceab94b4678bb3a7da01c2e73f5f0c1883d1dad2e
+  data.tar.gz: e51c1775be63158b1724739c94ddecb48b406c20a339d1b4b8a0acba1d753e5d
 SHA512:
-  metadata.gz: 060ff017a7a0de4ede9652d3e87ddd14083022725ce225739758ce0c7882fe953d6bfaf27ccf48f4fa8b73b480fce98f623dfaf02406944e19e841763b42c022
-  data.tar.gz: 0b80b3df83f7399b5118d67abbf2e8c1da589b0ec902d7f2cc04a60be8498e41df9ebdbc569af9c57b487025994bfd5715a58b0d0c2a9a6994e06271fa2e2dcd
+  metadata.gz: d406808fe2dfc71599bb85516237fe3272876bd2302e3d794c12fb2d296fb2e4f65ea9906be0a8fe1f7deaaf1afab7cb3e6fa3c15dbe5f77f68c861b5f70ad39
+  data.tar.gz: 70e6fbef81ab903425190feda6683d3343604b980da6033f0ad7471816f4c2887eaf8408637dc1092e394e60162b877a1e5de6aa1a91516ba4c5d161bcd0b116

data/README.md

@@ -23,8 +23,21 @@ Vicary.redact("My cousin Terrence Okonkwo came over that summer.", identity)
 
 masked, n, restore_map = Vicary.redact_with_report(essay, identity)
 Vicary.restore(masked, restore_map) == essay   # => true
+
+# Where each replacement came from, for a host that draws on the essay: a
+# placeholder is not the width of the name it replaced, so an offset measured
+# against `masked` is displaced against the composition the student holds.
+spans = Vicary::Spans.derive(masked, restore_map)
+Vicary::Spans.to_original(offset, spans)   # masked coordinates -> the student's
+Vicary::Spans.to_redacted(offset, spans)   # and back
+Vicary::Spans.original(masked, restore_map) == essay   # => true
 ```
 
+An offset landing inside a placeholder resolves to the start of the span it
+replaced. A map that is absent, or that covers only some of the placeholders in
+the text, yields **no** spans and translates as the identity — a caller can
+handle "no spans" and cannot detect a wrong offset.
+
 ## Checking it
 
 Three layers, because each catches what the one above it cannot.
@@ -33,7 +46,7 @@ Three layers, because each catches what the one above it cannot.
 |---|---|
 | `rake conformance` | the scoreboard against the 54 frames — the final bar, and a coarse first one |
 | `rake gates` | the nine gates, all nine measured from what the repository ships |
-| `rake test` | the unit suites, including `primitives_test.rb`: forty-odd primitives over the shared corpus, which says *which brick* is crooked |
+| `rake test` | the unit suites, including `primitives_test.rb` (forty-odd primitives over the shared corpus, which says *which brick* is crooked) and `spans_test.rb` (the offset arithmetic over the shared `spans.json`) |
 | `rake parity` | gazetteer verdicts, name by name, against the Python reference |
 | `rake redaction_parity` | masked bytes against the Python reference, on prose no fixture contains |
 
@@ -47,15 +60,23 @@ zip reader and a binary read parsed as CSV yields zero rows — a *lower* exposu
 than the truth, and the wrong direction to fail in silently. The shipped table is
 gzip, which `zlib` reads, so that hazard does not arise on the default path.
 
-The last three need an essay corpus no package here ships, and this gem measures
-them once `VICARY_EVAL_CORPUS_TSV` points at one: 100% carrier recall (29/29
-held-out REDACT spans) and 0.60 over-fired spans per essay (15 across 25),
-identical to Python and TypeScript, plus its own latency p95 of 8.8–9.9 ms. **That
-last one is worth knowing about**: the bar is ≤ 10 ms, and this port runs nearest
-it of the three — roughly 4× Python — so the latency gate is a live constraint
-here rather than a formality. The carrier essays are built from offsets recorded
-in `conformance/carrier.json` rather than from a reimplementation of Python's
-RNG, and the suite asserts their sha256.
+Two of the last three read the corpus the repository now ships in
+`conformance/corpora/`, so they measure on a bare checkout with no environment
+set: 100% carrier recall and 8.150 over-fired spans per essay against a ≤ 8.15
+bar, identical to Python and TypeScript. `VICARY_EVAL_CORPUS_TSV` is an override
+for a different corpus, not a requirement.
+
+The ninth is latency, and **this port's absolute figure no longer constrains it**.
+The gate was a ≤ 10 ms bar, which this port ran nearest of the three; it is now a
+ratio against the last release timed on the same machine, held to ≤ +8%. That
+change matters most here: across three CPU models this port's absolute median
+spreads **31.8%**, the same axis that made the old bar a coin flip, while its
+ratio spreads 0.36 pp. Measured, the ratio holds σ 0.46% — the *widest* margin of
+the three ports, where the absolute figure gave it the narrowest.
+
+The carrier essays are built from offsets recorded in `conformance/carrier.json`
+rather than from a reimplementation of Python's RNG, and the suite asserts their
+sha256.
 
 The last two need the reference interpreter — run `just py-setup` from the
 repository root first.

data/lib/vicary/conformance.rb

@@ -155,6 +155,32 @@ module Vicary
         raw
       end
 
+      # The offset-translation spec — {Vicary::Spans}' cases and their answers.
+      #
+      # A third layer, checking a third thing. `frames.json` says what is masked
+      # and `primitives.json` says which rule decided it; neither constrains
+      # WHERE a replacement came from, and a port can reproduce every byte of
+      # both while being unable to tell a host where to draw a highlight on the
+      # student's own essay.
+      #
+      # Absent is an error rather than an empty document, for the same reason
+      # `just _conformance-check` refuses to run without a spec: a port that
+      # finds no cases and reports success has checked its arithmetic against
+      # nothing.
+      def load_spans(dir = nil)
+        dir = Pathname.new(dir || directory)
+        path = dir.join("spans.json")
+        unless path.file?
+          raise SpecError,
+                "no spans.json at #{path}. The ports would check their offset " \
+                "translation against nothing."
+        end
+
+        raw = JSON.parse(path.read)
+        require_version(raw["document_version"], "spans.json")
+        raw
+      end
+
       # Score an implementation against every frame.
       #
       # The block receives (sentence, identity) — the same input every Python arm

data/lib/vicary/corpus.rb

@@ -445,13 +445,23 @@ module Vicary
         over_fire = 0
         rewrites = 0
 
-        # Load the gazetteer before the clock starts. It is a one-time ~207 ms
-        # cost in this port, and whichever essay happens to be first pays all of
-        # it: at n=25 that single sample lands at or above p95 and sets the
-        # gate's answer by itself — 14.3 ms cold against 7.6 ms warm, on a 10 ms
-        # bar. The number the gate claims is essay-length redaction latency, not
-        # process startup. Excluded in all three ports alike.
-        yield(cases.first.base[0, 200], identity) unless cases.empty?
+        # Warm up before the clock starts, over the WHOLE corpus rather than one
+        # 200-char call. Two costs are being excluded, and the second one is why
+        # this grew.
+        #
+        # The gazetteer load is a one-time ~207 ms cost in this port, and
+        # whichever essay happens to be first pays all of it — 14.3 ms cold
+        # against 7.6 ms warm.
+        #
+        # The second belongs to TypeScript, where V8 tiers the redaction path up
+        # over roughly the first four essays and runs them at about twice their
+        # steady-state cost. This port barely moves under a full warmup and does
+        # it anyway: the three ports measure identically or the gate is three
+        # different gates.
+        cases.each do |kase|
+          yield(kase.text, identity)
+          yield(kase.base, identity)
+        end
 
         cases.each do |kase|
           # The median of LATENCY_REPEATS, not one sample — see that constant.

data/lib/vicary/latency_baseline.rb

@@ -6,37 +6,77 @@ require "pathname"
 module Vicary
   # Is this build slower than the last release, and is that a fair question here?
   #
-  # The latency gate used to hold an absolute number — 10 ms — which is a claim
-  # about the machine as much as about the code. It passed on a laptop and failed
-  # on the CI runner enforcing it, so v0.2.3 published to PyPI and npm and was
-  # refused by RubyGems on the same commit. This gem is the one that caught it.
+  # The gate has asked this three ways. The first two are worth keeping in view,
+  # because each looked correct until it decided a release.
   #
-  # What replaced it asks a relative question: is this port slower than it was at
-  # the last release, by more than the tolerance. That only means something
-  # between measurements taken on comparable hardware, so this module's real work
-  # is REFUSING to compare when they are not — a machine difference reported as a
-  # code regression is worse than no gate, because it trains the reader to ignore
-  # it.
+  # **An absolute bar — 10 ms.** A claim about the machine as much as about the
+  # code. It passed on a laptop and failed on the CI runner enforcing it, so
+  # v0.2.3 published to PyPI and npm and was refused by RubyGems on the same
+  # commit. This gem is the one that caught it.
   #
-  # This port reaches its own verdict from the shared file. It does not read
+  # **A stored baseline** — record each release's number and compare the next run
+  # against it, refusing unless the run claims the profile the baseline was
+  # recorded on. Better, and still wrong, for a reason no estimator fixes: the
+  # profile `github-ubuntu-latest` is not a machine. Thirty-six processes across
+  # six runners per port, on identical code, spread 67% in THIS port — 6.53 ms on
+  # an Intel Xeon 6973P-C against 10.63 ms on an EPYC 7763 — 26% in Python and
+  # 21% in TypeScript, against an 8% bar. One probe run drew five CPU models from
+  # that one label, and two runners of the same model still differed by 26%.
+  #
+  # **A pair, measured here.** The previous release's code and this checkout,
+  # measured on the SAME machine, interleaved and counterbalanced, by
+  # `tools/latency_pair.py`. Every property of the machine is common to both
+  # sides and cancels; what is left is within-process noise, 1.7% in this port.
+  #
+  # Which leaves this module the job it has always had: REFUSING to compare when
+  # the two sides would not be like for like. What changed is that the refusals
+  # are about the pair record — is there one, is it this port's, was it measured
+  # on these essays, was it measured for this commit — rather than about the
+  # profile of a machine somewhere else.
+  #
+  # This port reaches its own verdict from the shared record. It does not read
   # Python's answer.
   module LatencyBaseline
-    BASELINE_FILENAME = "latency_baseline.json"
-
-    # Set by CI on the one matrix entry whose language version matches the
-    # recorded profile. Absent everywhere else on purpose: a developer's laptop
-    # measures the same commit two to three times faster than the runner, and
-    # comparing that against a runner baseline reports a phantom improvement.
-    PROFILE_ENV_VAR = "VICARY_LATENCY_PROFILE"
+    # The tolerance and the protocol, in the repository. Not a measurement:
+    # nothing is recorded at release time any more, because the comparison point
+    # is the previous release's *code*, which the repository already has.
+    SPEC_FILENAME = "latency_baseline.json"
+
+    # Where `tools/latency_pair.py` left the paired measurement. Set by CI in the
+    # same job, seconds before the gate runs. Absent on a laptop unless the
+    # harness was run there by hand, and that absence is a refusal to compare
+    # rather than a pass — measuring one side of a comparison is not a gate.
+    PAIR_ENV_VAR = "VICARY_LATENCY_PAIR"
+
+    # What this reader understands. A record from a future shape is refused
+    # rather than half-read: a partly-understood record still yields a number,
+    # and a number is exactly what must not be invented here.
+    PAIR_DOCUMENT_VERSION = 1
 
     IMPLEMENTATION = "ruby"
 
+    # The bar, chosen rather than derived — 8% is what a reviewer is willing to
+    # call a regression. What the noise decides is whether the bar is USABLE,
+    # and it is: the gate statistic holds **sigma 0.46%** in this port (95% CI
+    # 0.34-0.72%, sixteen runs across eight CI runners, fixed head and tag),
+    # putting 8% at 17.2 sigma — the widest margin of the three. It was about a
+    # third of a sigma under the stored baseline, which is how that one red-lit
+    # `main` on unchanged code, and how it refused this port's 0.2.3 while the
+    # other two took the same commit.
+    #
+    # This is also the port where the pairing earns the most. Across three CPU
+    # models its absolute figure spreads **31.8%** — the same axis, and nearly
+    # the same size, as the 67% that killed the stored baseline — while its ratio
+    # spreads 0.36 pp. Same runs, same data. See `tools/latency_pair.py`.
+    #
+    # It does not catch drift: +5% a release passes every time and compounds.
+    # That is deliberate — this gate is for the step change, not the trend.
     DEFAULT_TOLERANCE_PCT = 8.0
 
     # The gate's answer, and — when it declines — why.
     Comparison = Struct.new(
-      :measured_ms, :baseline_ms, :regression_pct, :tolerance_pct,
-      :comparable, :reason,
+      :measured_ms, :previous_ms, :current_ms, :regression_pct, :tolerance_pct,
+      :against, :comparable, :reason,
       keyword_init: true
     ) do
       def holds?
@@ -47,97 +87,114 @@ module Vicary
     end
 
     class << self
-      def baseline_path(dir = nil)
+      def spec_path(dir = nil)
         root = dir || Conformance.directory
         return nil if root.nil?
 
-        path = Pathname.new(root).join(BASELINE_FILENAME)
+        path = Pathname.new(root).join(SPEC_FILENAME)
         path.exist? ? path : nil
       end
 
       def load(dir = nil)
-        path = baseline_path(dir)
+        path = spec_path(dir)
         return nil if path.nil?
 
         JSON.parse(path.read)
       end
 
-      # `major.minor` of the running Ruby, matching how the profile records it.
-      def language_version
-        RUBY_VERSION.split(".").first(2).join(".")
+      # The paired measurement, or why there is none to read.
+      #
+      # An unreadable file and an absent one stay distinguishable: the first is a
+      # broken harness and the second is an ordinary laptop, and they should not
+      # report the same thing.
+      def load_pair(path = nil)
+        given = (path || ENV[PAIR_ENV_VAR] || "").strip
+        if given.empty?
+          return [nil,
+                  "#{PAIR_ENV_VAR} is unset, so no paired measurement was taken on " \
+                  "this machine; the gate compares this build against the last " \
+                  "release measured HERE, and one side of a comparison is not a gate"]
+        end
+        return [nil, "#{PAIR_ENV_VAR}=#{given.inspect} does not exist"] unless File.exist?(given)
+
+        begin
+          [JSON.parse(File.read(given)), nil]
+        rescue StandardError => e
+          [nil, "the pair record at #{given} could not be read: #{e.message}"]
+        end
       end
 
-      # Compare +measured_ms+ against the recorded baseline for this port.
+      # Compare the pair measured on this machine, for this port.
       #
-      # Every reason below is a refusal to compare, not a failure to measure: the
-      # number was measured either way and is reported either way. What is
-      # withheld is the verdict, because the two sides would not be like for like.
+      # +measured_ms+ is this process's own figure. It is reported either way and
+      # it is never the verdict: the verdict comes from the two numbers in the
+      # pair record, taken back to back on one machine. Mixing this process's
+      # measurement with the pair's other side would reintroduce exactly the
+      # machine difference the pair exists to cancel.
       def compare(measured_ms, corpus_id, dir: nil, implementation: IMPLEMENTATION,
-                  observed_language_version: nil, profile_env: nil)
-        doc = load(dir)
-        tolerance = (doc && doc["tolerance_pct"] || DEFAULT_TOLERANCE_PCT).to_f
-        lang = observed_language_version || language_version
-
-        declined = lambda do |reason, baseline_ms = nil|
-          Comparison.new(measured_ms: measured_ms, baseline_ms: baseline_ms,
-                         regression_pct: nil, tolerance_pct: tolerance,
+                  pair_path: nil, building_sha: nil)
+        doc = load(dir) || {}
+        tolerance = (doc["tolerance_pct"] || DEFAULT_TOLERANCE_PCT).to_f
+
+        declined = lambda do |reason|
+          Comparison.new(measured_ms: measured_ms, previous_ms: nil, current_ms: nil,
+                         regression_pct: nil, tolerance_pct: tolerance, against: nil,
                          comparable: false, reason: reason)
         end
 
-        return declined.call("no #{BASELINE_FILENAME} in this checkout") if doc.nil?
+        record, why = load_pair(pair_path)
+        return declined.call(why || "no paired measurement") if record.nil?
 
-        profile = doc["profile"] || {}
-        want_profile = profile["id"]
-        have_profile = (profile_env || ENV[PROFILE_ENV_VAR] || "").strip
-        if have_profile.empty?
+        unless record["document_version"] == PAIR_DOCUMENT_VERSION
           return declined.call(
-            "#{PROFILE_ENV_VAR} is unset, so this machine does not claim to be " \
-            "#{want_profile.inspect}; the baseline was recorded there"
+            "the pair record is document_version #{record['document_version']} " \
+            "and this reader knows #{PAIR_DOCUMENT_VERSION}"
           )
         end
-        unless have_profile == want_profile
+        unless record["implementation"] == implementation
           return declined.call(
-            "#{PROFILE_ENV_VAR}=#{have_profile.inspect} but the baseline was " \
-            "recorded on #{want_profile.inspect}"
+            "the pair record measures #{record['implementation'].inspect}, " \
+            "not #{implementation.inspect}"
           )
         end
-
-        want_lang = (profile["language_versions"] || {})[implementation]
-        if !want_lang.nil? && want_lang.to_s != lang
+        unless record["corpus"] == corpus_id
           return declined.call(
-            "#{implementation} #{lang} is not the #{want_lang} the baseline was " \
-            "recorded on; interpreter versions differ by more than the bar"
+            "the pair was measured on corpus #{record['corpus'].inspect} and this " \
+            "run is #{corpus_id.inspect}; latency scales with essay length"
           )
         end
 
-        want_corpus = doc["corpus"]
-        if !want_corpus.nil? && want_corpus != corpus_id
+        # Only where there is something to check against. `GITHUB_SHA` names the
+        # commit the job is building, so a record left over from an earlier
+        # commit is caught here rather than being read as this build's verdict.
+        # Locally there is no such witness and no such risk: the harness is run
+        # by hand, minutes before, on the tree in front of you.
+        building = (building_sha || ENV["GITHUB_SHA"] || "").strip
+        head = record["head_sha"].to_s
+        if !building.empty? && !head.empty? && building != head
           return declined.call(
-            "corpus #{corpus_id.inspect} is not the #{want_corpus.inspect} the " \
-            "baseline was recorded on; latency scales with essay length"
+            "the pair was measured for commit #{head[0, 12]} and this job is " \
+            "building #{building[0, 12]}; the record is stale"
           )
         end
 
-        entry = (doc["implementations"] || {})[implementation] || {}
-        recorded = entry["pooled_median_ms"]
-        if recorded.nil?
-          return declined.call(
-            "no baseline recorded for #{implementation} yet — the next release " \
-            "records one"
-          )
+        previous = record["previous_ms"]
+        current = record["current_ms"]
+        unless previous.is_a?(Numeric) && current.is_a?(Numeric)
+          return declined.call("the pair record carries no pair of measurements")
         end
-
-        recorded = recorded.to_f
-        if recorded <= 0
+        if previous <= 0
           return declined.call(
-            "recorded baseline for #{implementation} is not positive", recorded
+            "the previous release measured #{previous} ms, which is not positive"
           )
         end
 
         Comparison.new(
-          measured_ms: measured_ms, baseline_ms: recorded,
-          regression_pct: (measured_ms / recorded - 1.0) * 100.0,
-          tolerance_pct: tolerance, comparable: true, reason: nil
+          measured_ms: measured_ms, previous_ms: previous.to_f,
+          current_ms: current.to_f,
+          regression_pct: (current.to_f / previous.to_f - 1.0) * 100.0,
+          tolerance_pct: tolerance, against: (record["against"] || {})["ref"],
+          comparable: true, reason: nil
         )
       end
 
@@ -149,9 +206,10 @@ module Vicary
         end
 
         sign = c.regression_pct >= 0 ? "+" : ""
-        format("latency %.3f ms vs %.3f ms at the last release — %s%.2f%% " \
-               "against a %d%% bar",
-               c.measured_ms, c.baseline_ms, sign, c.regression_pct, c.tolerance_pct)
+        format("latency %.3f ms here; paired on this machine, %.3f ms against " \
+               "%s's %.3f ms — %s%.2f%% against a %d%% bar",
+               c.measured_ms, c.current_ms, c.against || "the last release",
+               c.previous_ms, sign, c.regression_pct, c.tolerance_pct)
       end
 
       # The keyword arguments Gates.measure wants. Returns the *detail* rather

data/lib/vicary/spans.rb

@@ -0,0 +1,129 @@
+# frozen_string_literal: true
+
+module Vicary
+  # One replacement, located in both the original and the masked text.
+  #
+  # `orig_start...orig_end` is what was removed; `new_start...new_end` is the
+  # placeholder standing in its place. The two widths differ — that difference
+  # is the whole reason this type exists.
+  RedactionSpan = Struct.new(:orig_start, :orig_end, :new_start, :new_end) do
+    # How much this replacement moved everything after it.
+    def delta
+      (new_end - new_start) - (orig_end - orig_start)
+    end
+
+    def to_h
+      { orig_start: orig_start, orig_end: orig_end,
+        new_start: new_start, new_end: new_end }
+    end
+  end
+
+  # Offset translation between the masked text and the composition the student
+  # holds — the Ruby port of `python/src/vicary/redaction.py`'s span layer.
+  #
+  # Why a host needs it. `redact_with_report` hands back masked bytes and a
+  # restore map, and between them they still say nothing about WHERE anything
+  # sits. A host rendering a highlight works in the student's coordinates, the
+  # detector produced its text in masked coordinates, and `{NAME_1}` is not the
+  # width of the name it replaced: every offset after the first replacement is
+  # displaced by the cumulative delta. Measured on a 56-paper corpus, redaction
+  # fired on 43 essays and not one recorded offset pair on those 43 landed on
+  # the original text.
+  #
+  # **Nothing in the masker records anything for this to work.** That is the
+  # point, and it is why this is a pure module rather than instrumentation:
+  # {Vicary::Candidates.mask_candidates} is many passes over a string each pass
+  # mutates, so a span recorded inside one pass is in that pass's intermediate
+  # coordinates and would have to be composed forward through every later one.
+  # It is unnecessary — the finished text still *contains* every placeholder, so
+  # each replacement's new span is where its placeholder sits, and the restore
+  # map gives the original, so its width is a `length`.
+  #
+  # Checked against `conformance/spans.json`, generated from the Python
+  # reference, by `test/spans_test.rb`. The arithmetic is shared with the other
+  # two ports and pinned there rather than described here.
+  module Spans
+    # A placeholder token as it appears in masked text, e.g. `{NAME_1}`.
+    PLACEHOLDER = /\{[A-Z_]+(?:_\d+)?\}/.freeze
+
+    class << self
+      # The span map, derived from the masked text and the restore map alone.
+      #
+      # Replacement preserves order, so one left-to-right walk accumulating the
+      # running delta recovers the original offsets exactly.
+      #
+      # Returns `[]` when `restore_map` is empty or does not cover a placeholder
+      # present in the text. A partial map is refused rather than answered for
+      # the half it can place, because a translation that is right for some
+      # offsets and silently wrong for others is worse than one that declines:
+      # the caller can handle "no map" and cannot detect "wrong offset".
+      def derive(masked, restore_map)
+        return [] if restore_map.nil? || restore_map.empty?
+
+        spans = []
+        delta = 0
+        masked.to_s.enum_for(:scan, PLACEHOLDER).each do
+          match = Regexp.last_match
+          original = restore_map[match[0]]
+          return [] if original.nil?
+
+          orig_start = match.begin(0) - delta
+          span = RedactionSpan.new(orig_start, orig_start + original.length,
+                                   match.begin(0), match.end(0))
+          spans << span
+          delta += span.delta
+        end
+        spans
+      end
+
+      # A redacted-text offset in original coordinates.
+      #
+      # An offset landing INSIDE a placeholder maps to the start of the span it
+      # replaced: the placeholder's interior has no counterpart in the original,
+      # so any position within it is the same position in original terms.
+      def to_original(offset, spans)
+        result = offset
+        spans.each do |span|
+          break if offset < span.new_start
+          return span.orig_start if offset < span.new_end
+
+          result = span.orig_end + (offset - span.new_end)
+        end
+        result
+      end
+
+      # An original-text offset in redacted coordinates.
+      #
+      # An offset inside a replaced span maps to the start of its placeholder,
+      # for the mirror-image reason.
+      def to_redacted(offset, spans)
+        result = offset
+        spans.each do |span|
+          break if offset < span.orig_start
+          return span.new_start if offset < span.orig_end
+
+          result = span.new_end + (offset - span.orig_end)
+        end
+        result
+      end
+
+      # Reconstruct the pre-redaction text from the masked bytes and the map.
+      #
+      # Exact where a span map exists, and the masked text unchanged where one
+      # does not — which is the honest answer rather than a partial restoration,
+      # for the reason {derive} declines.
+      def original(masked, restore_map)
+        masked = masked.to_s
+        out = +""
+        prev = 0
+        derive(masked, restore_map).each do |span|
+          out << masked[prev...span.new_start]
+          out << restore_map[masked[span.new_start...span.new_end]]
+          prev = span.new_end
+        end
+        out << masked[prev..].to_s
+        out
+      end
+    end
+  end
+end

data/lib/vicary/version.rb

@@ -6,5 +6,5 @@ module Vicary
   # Shared across all three front doors on purpose: one detector, one number. A
   # gem 0.3.0 that corresponds to nothing on PyPI cannot be reasoned about, and
   # the parity claim is between *versions*, not between package names.
-  VERSION = "0.2.4"
+  VERSION = "0.2.6"
 end

data/lib/vicary.rb

@@ -43,3 +43,4 @@ require_relative "vicary/census"
 require_relative "vicary/corpus"
 require_relative "vicary/latency_baseline"
 require_relative "vicary/redact"
+require_relative "vicary/spans"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: vicary
 version: !ruby/object:Gem::Version
-  version: 0.2.4
+  version: 0.2.6
 platform: ruby
 authors:
 - Blake Thomas
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-08-13 00:00:00.000000000 Z
+date: 2026-09-09 00:00:00.000000000 Z
 dependencies: []
 description: |
   Finds the names a student writes about — classmates, teachers, relatives — and
@@ -38,6 +38,7 @@ files:
 - lib/vicary/lexicon.rb
 - lib/vicary/minter.rb
 - lib/vicary/redact.rb
+- lib/vicary/spans.rb
 - lib/vicary/structured.rb
 - lib/vicary/version.rb
 homepage: https://github.com/bwthomas/vicary