ucode 0.3.3 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9d21572adc317e05b3a29573cf799cc8cedd61c56949339d844022c9ccf92b0e
-  data.tar.gz: 41ff686beb95370ff0927002fc0cc9350aa448221b8dbf12c87712d005059e98
+  metadata.gz: 0d2232b7f9bb393115f100e6eeb7e81757743ba215953149071a6a38d46b9ddd
+  data.tar.gz: a57ee93b28d31266b8f6e53595d68066b724b735aa50ddfa551ff87f896bf15b
 SHA512:
-  metadata.gz: 2de58e0bab78c0be15ef4e99ab699333198702012832985958eb86910c0151f47b4147b07f958c2ba881d9f9750d7336586151a264ad835339a54064412907e6
-  data.tar.gz: f3a90c63779e0eec7d93fd221063f80cf8d64df16707185a3f9c201abf8d2e1376ece32a0da2af1b5513d0190fdb0a785b5dbcccf65fbb427ff31ae745c22ca7
+  metadata.gz: a2b3b88ff4b4cae4549435999951c62b6ff663f692505f3518c21cbc1b0f8be0d4d90635b8c94cd23aa51c02eeedead1aa8eef2bde1ea040b26bc0ecea95bd98
+  data.tar.gz: 4eda9506f93e4feff2d24f2348a2b5e629df089c90e99db1c52b6c43a53332c16e5885c0268e3e785e89b7840ff7e2115b797c8dce769d694510ed3dfc248c96

data/README.md

@@ -149,6 +149,30 @@ Pillar 1 → Pillar 2 → Pillar 3). No new extraction logic — the
 `ucode code-chart` subcommand is a thin CLI wrapper over `Ucode::CodeChart::Writer`,
 which orchestrates `Ucode::Glyphs::Resolver` for each codepoint.
 
+### Which blocks work
+
+Most Unicode 17 blocks extract cleanly via pillar 1 (PDF-embedded font
+with a trustworthy `/ToUnicode` CMap). Three known failure shapes
+require pillar 2 (positional correlation) to take over:
+
+- **No `/ToUnicode` at all** — older subset practice (Garay, Ol Onal,
+  Kana Extended-A/B, Small Kana Extension). The CMap is missing; pillar
+  2 attributes glyphs via chart-grid geometry.
+- **`/ToUnicode` encodes the wrong codepoints** — the Enclosed
+  Ideographic Supplement class (U+1F200..U+1F2FF). The font's CMap
+  encodes the *composing* ideographs shown in chart annotations (中, 新,
+  三) rather than the specimens themselves (🈀, 🈁, 🈂). The orchestrator
+  auto-detects this via block-scope awareness: when the ToUnicode result
+  has zero in-block intersection, pillar 2 takes over.
+- **Partial overlap** — the CMap covers some in-block codepoints but
+  misses others. Caller can force positional attribution via
+  `force_positional_for_font_ids:` on `Ucode::Glyphs::EmbeddedFonts::Catalog`.
+
+For the full strategy-chain semantics (positional? predicate,
+partition, range-aware bail, positional-precedence merge) see
+[docs/architecture.md → The 4-tier glyph sourcing
+strategy](docs/architecture.md#the-4-tier-glyph-sourcing-strategy).
+
 ## Glyph extraction (4-tier pipeline)
 
 The `ucode glyphs` command and the `--include-glyphs` flag on `ucode build`
@@ -446,18 +470,33 @@ Pillar 1 handles only the fonts where correlation is unambiguous:
   skipped. These codepoints fall through to **pillar 2** (content-stream
   positional correlation), and from there to **pillar 3** (Last Resort)
   if pillar 2 cannot resolve them either.
+- **Type0 fonts whose `/ToUnicode` CMap encodes the wrong
+  codepoints** — the Enclosed Ideographic Supplement failure mode.
+  Some Code Charts PDFs embed a CID font whose CMap maps the font's
+  CIDs to the *composing* ideographs used in chart annotations (e.g.
+  U+4E2D 中, U+65B0 新) rather than to the block's actual specimens
+  (e.g. U+1F200 🈀, U+1F201 🈁). Without block-scope awareness,
+  pillar 1 "succeeds" with the wrong codepoints and the positional
+  fallback never runs. The `CodepointMapper` orchestrator now
+  auto-detects this: when the caller threads a `block_range:` and the
+  ToUnicode result has zero in-block intersection, the intrinsic
+  strategy is dropped and positional strategies take over. A
+  `force_positional_for_font_ids:` override handles the
+  partial-overlap case (some in-block, some out-of-block) where
+  auto-detection cannot tell that positional is still needed.
 - **Stream-form `/CIDToGIDMap`** — a binary lookup table. Treated as
   unsupported; the font is skipped.
 - **Bare CFF streams fontisan does not yet recognize** — a separate
   fontisan-side issue; flagged for investigation.
 
 Code Charts cells not covered by pillar 1 are exactly the cells whose
-character is not drawn from an embedded subsetted font with
-`/ToUnicode` — either a label, a glyph in a font without `/ToUnicode`,
-or a placeholder. **Pillar 2** (content-stream positional correlation)
-handles the no-`/ToUnicode` case, and **pillar 3** (Last Resort UFO)
-handles the placeholder case; the small remainder are correctly absent
-from the dataset.
+character is not drawn from an embedded subsetted font with a
+trustworthy `/ToUnicode` — either a label, a glyph in a font without
+`/ToUnicode`, a font whose CMap encodes the wrong codepoints, or a
+placeholder. **Pillar 2** (content-stream positional correlation)
+handles the no-`/ToUnicode` and wrong-`/ToUnicode` cases, and
+**Pillar 3** (Last Resort UFO) handles the placeholder case; the
+small remainder are correctly absent from the dataset.
 
 ## System dependencies
 

data/config/unicode17_universal_glyph_set.yml

@@ -1,6 +1,6 @@
 ---
 unicode_version: 17.0.0
-ucode_version: 0.3.2
+ucode_version: 0.4.0
 generated_at: '2026-06-28T00:00:00Z'
 default_sources:
 - kind: fontist

data/lib/ucode/cli.rb

@@ -150,49 +150,44 @@ module Ucode
 
       desc "extract --block BLOCK --to DIR [VERSION]",
            "Extract per-codepoint SVG + provenance sidecars from a Code Charts PDF"
-      option :block, type: :string, required: true,
-                     desc: "Block identifier (e.g. Sidetic)"
+      option :block, type: :string, required: false,
+                     desc: "Block identifier (required unless --missing-from is given)"
       option :to, type: :string, required: true,
                   desc: "Output directory (will contain <block_id>/<U+XXXX>.svg + .json)"
+      option :verify, type: :boolean, default: false,
+                      desc: "Run pixel-diff verification on every extracted glyph"
+      option :missing_from, type: :string, default: nil,
+                            desc: "Path to essenfont manifest; extracts only the gaps"
       def extract(version = nil)
         with_codechart_errors do
           version_str = VersionResolver.resolve(version)
-          block = resolve_block!(options[:block], version_str)
-          block_first_cp = block.range_first
-
-          # Download (idempotent — re-runs skip when the PDF is cached).
-          Commands::FetchCommand.new.fetch_charts(version_str, block_first_cps: [block_first_cp])
-
-          pdf = Ucode::Glyphs::PdfFetcher.new(version_str)
-            .fetch(block_first_cp: block_first_cp)
-          unless pdf
-            raise Ucode::CodeChartNotFoundError.new(
-              "Code Charts PDF unavailable for block #{block.id.inspect}",
-              context: { block_id: block.id, version: version_str },
-            )
-          end
 
-          writer = Ucode::CodeChart::Writer.new(
-            output_root: Pathname.new(options[:to]),
-            pdf_path: pdf,
-            ucd_version: version_str,
-          )
-          summary = writer.write(block)
-          puts JSON.pretty_generate(summary.to_h.compact)
+          if options[:missing_from]
+            run_batch_extract(version: version_str,
+                              manifest_path: options[:missing_from],
+                              to: options[:to], verify: options[:verify])
+          else
+            raise Thor::Error, "--block is required when --missing-from is not set" unless options[:block]
+
+            run_single_block_extract(version: version_str,
+                                     block_id: options[:block],
+                                     to: options[:to], verify: options[:verify])
+          end
         end
       end
 
       desc "list", "List cached Code Charts PDFs under the version's cache"
+      option :coverage_gap_only, type: :boolean, default: false,
+                                 desc: "Only list blocks with OFL coverage gaps"
+      option :coverage, type: :string, default: nil,
+                        desc: "Path to a coverage YAML (required with --coverage-gap-only)"
       def list
-        version = VersionResolver.resolve(nil)
-        pdfs_dir = Ucode::Cache.pdfs_dir(version)
-        files = pdfs_dir.exist? ? pdfs_dir.children.sort : []
-        if files.empty?
-          puts "(no cached Code Charts PDFs)"
-          return
-        end
-        files.each do |f|
-          puts f.basename
+        with_codechart_errors do
+          if options[:coverage_gap_only]
+            run_coverage_gap_list
+          else
+            run_simple_list
+          end
         end
       end
 
@@ -209,6 +204,180 @@ module Ucode
         resolve_block!(block_id, version).range_first
       end
 
+      def run_single_block_extract(version:, block_id:, to:, verify:)
+        block = resolve_block!(block_id, version)
+        fetcher = Ucode::CodeChart::Fetcher.new(version: version)
+        pdf_path = fetcher.fetch(block: block)
+
+        writer = Ucode::CodeChart::Writer.new(
+          output_root: Pathname.new(to),
+          pdf_path: pdf_path,
+          ucd_version: version,
+        )
+        summary = writer.write(block)
+
+        result = { **summary.to_h.compact }
+        if verify
+          result[:verification] = verify_block(block: block, pdf_path: pdf_path,
+                                               output_root: Pathname.new(to))
+        end
+        puts JSON.pretty_generate(result)
+      end
+
+      def run_batch_extract(version:, manifest_path:, to:, verify:)
+        manifest = Ucode::CodeChart::GapAnalyzer::EssenfontManifest.new(manifest_path)
+        analyzer = Ucode::CodeChart::GapAnalyzer::Analyzer.new(
+          manifest: manifest,
+          blocks: load_all_blocks(version),
+        )
+        fetcher = Ucode::CodeChart::Fetcher.new(version: version)
+        runner = Ucode::CodeChart::BatchRunner.new(
+          output_root: Pathname.new(to), ucd_version: version,
+          fetcher: fetcher,
+        )
+        aggregate = runner.run(gap_analyzer: analyzer,
+                               blocks: load_all_blocks(version))
+        result = aggregate.to_h
+        result[:verification] = verify_aggregate(output_root: Pathname.new(to)) if verify
+        puts JSON.pretty_generate(result.compact)
+      end
+
+      # Load all blocks from the cached Blocks.txt as a {block_id => Block} map.
+      def load_all_blocks(version)
+        blocks_txt = Ucode::Cache.ucd_dir(version).join("Blocks.txt")
+        Ucode::Parsers::Blocks.each_record(blocks_txt).to_h do |b|
+          [b.id, b]
+        end
+      end
+
+      # Pixel-diff verification after single-block extract. Returns
+      # a tally of Pass/Fail/Skipped counts + artifact paths for Fail.
+      def verify_block(block:, pdf_path:, output_root:)
+        verifier = Ucode::CodeChart::Verifier.new(diff_dir: output_root.join(".verify"))
+        return { status: "skipped", reason: "no verifier strategy" } unless verifier.available?
+
+        block_dir = output_root.join(block.id)
+        tallies = { passed: 0, failed: 0, skipped: 0, failures: [] }
+        Dir.glob(block_dir.join("U+*.svg")).each do |svg_path|
+          cp = parse_cp_from_path(svg_path)
+          next unless cp
+
+          extractor_result = build_extractor_result_for_verification(block_dir, svg_path, cp)
+          result = verifier.verify(extractor_result, pdf_path: pdf_path)
+          update_tally(tallies, result)
+        end
+        tallies
+      end
+
+      def verify_aggregate(output_root:, **_kwargs)
+        verifier = Ucode::CodeChart::Verifier.new(diff_dir: output_root.join(".verify"))
+        return { status: "skipped", reason: "no verifier strategy" } unless verifier.available?
+
+        tallies = { passed: 0, failed: 0, skipped: 0, failures: [] }
+        Dir.glob(output_root.join("*", "U+*.svg")).each do |svg_path|
+          cp = parse_cp_from_path(svg_path)
+          next unless cp
+
+          block_id = Pathname.new(svg_path).dirname.basename.to_s
+          sidecar = Pathname.new(svg_path.sub_ext(".json").to_s)
+          next unless sidecar.exist?
+
+          data = JSON.parse(sidecar.read)
+          pdf_path = Pathname.new(Ucode::Cache.pdfs_dir(data["ucd_version"])
+                                  .join("U#{block_first_cp_hex(block_id)}.pdf"))
+          next unless pdf_path.exist?
+
+          extractor_result = build_extractor_result_for_verification(
+            Pathname.new(svg_path).dirname, svg_path, cp
+          )
+          result = verifier.verify(extractor_result, pdf_path: pdf_path)
+          update_tally(tallies, result)
+        end
+        tallies
+      end
+
+      def parse_cp_from_path(svg_path)
+        basename = File.basename(svg_path, ".svg")
+        return nil unless basename.start_with?("U+")
+
+        basename.sub("U+", "").to_i(16)
+      end
+
+      def block_first_cp_hex(block_id)
+        # Block IDs use underscored names but PDFs are keyed by first
+        # codepoint hex. Resolve via Blocks.txt on disk.
+        blocks_txt = Ucode::Cache.ucd_dir(VersionResolver.resolve(nil)).join("Blocks.txt")
+        block = Ucode::Parsers::Blocks.find_by_id!(blocks_txt, block_id)
+        block.range_first.to_s(16).upcase.rjust(4, "0")
+      end
+
+      def build_extractor_result_for_verification(_block_dir, svg_path, codepoint)
+        sidecar = Pathname.new(svg_path.sub_ext(".json").to_s)
+        data = sidecar.exist? ? JSON.parse(sidecar.read) : {}
+        Ucode::CodeChart::Extractor::Result.new(
+          codepoint: codepoint,
+          svg: File.read(svg_path),
+          tier: :pillar1,
+          provenance: data["extractor_version"] || "verify",
+          base_font: data["base_font"],
+          gid: data["gid"],
+          source_page: data["source_page"],
+          source_cell: data["source_cell"],
+        )
+      end
+
+      def update_tally(tally, result)
+        case result
+        when Ucode::CodeChart::Verifier::Result::Pass
+          tally[:passed] += 1
+        when Ucode::CodeChart::Verifier::Result::Fail
+          tally[:failed] += 1
+          tally[:failures] << { codepoint: result.codepoint,
+                                percent: result.percent,
+                                diff_path: result.diff_path.to_s }
+        when Ucode::CodeChart::Verifier::Result::Skipped
+          tally[:skipped] += 1
+        end
+      end
+
+      def run_simple_list
+        version = VersionResolver.resolve(nil)
+        pdfs_dir = Ucode::Cache.pdfs_dir(version)
+        files = pdfs_dir.exist? ? pdfs_dir.children.sort : []
+        if files.empty?
+          puts "(no cached Code Charts PDFs)"
+          return
+        end
+        files.each do |f|
+          puts f.basename
+        end
+      end
+
+      def run_coverage_gap_list
+        coverage_path = options[:coverage]
+        raise Thor::Error, "--coverage <path.yml> is required with --coverage-gap-only" unless coverage_path
+
+        version = VersionResolver.resolve(nil)
+        blocks = load_all_blocks(version)
+        index = Ucode::CodeChart::CoverageGapIndex.from_yaml(coverage_path, blocks: blocks)
+        gaps = index.gap_blocks
+        if gaps.empty?
+          puts "(no coverage gaps)"
+          return
+        end
+        printf "%<block_id>-40s %<assigned>10s %<covered>10s %<missing>10s\n",
+               block_id: "block_id", assigned: "assigned",
+               covered: "covered", missing: "missing"
+        gaps.each do |gap|
+          block = blocks[gap.block_id]
+          assigned = block.range_last - block.range_first + 1
+          covered = assigned - gap.size
+          printf "%<block_id>-40s %<assigned>10d %<covered>10d %<missing>10d\n",
+                 block_id: gap.block_id, assigned: assigned,
+                 covered: covered, missing: gap.size
+        end
+      end
+
       # Convert semantic Ucode errors into Thor errors so Thor's
       # dispatch prints the message cleanly instead of a stack trace.
       # Thor's `start` rescues only `Thor::Error`; without this bridge,

data/lib/ucode/code_chart/batch_runner.rb

@@ -0,0 +1,184 @@
+# frozen_string_literal: true
+
+require "pathname"
+require "json"
+
+module Ucode
+  module CodeChart
+    # Multi-block orchestration: drives one {Writer} per gap block,
+    # enforces per-block idempotency, isolates per-block failures.
+    #
+    # Composes four orthogonal concerns:
+    #
+    #   * {Fetcher}      — owns PDF download + cache + integrity check
+    #   * {Writer}       — owns one block's extraction + disk write
+    #   * {GapAnalyzer}  — owns "which codepoints to extract per block"
+    #   * {SkipChecker}  — owns "should this block be skipped" (this file)
+    #
+    # Each concern is replaceable; BatchRunner itself is a pure
+    # iterator + aggregator. Adding a new "skip if X" rule = one
+    # method on {SkipChecker}, one entry in the predicate chain.
+    class BatchRunner
+      # Per-block run summary.
+      BlockSummary = Struct.new(
+        :block_id, :codepoints_extracted, :svgs_written,
+        :svgs_skipped, :pdf_sha256, :skipped, :error,
+        keyword_init: true,
+      )
+
+      # Aggregate across the whole batch.
+      Aggregate = Struct.new(
+        :blocks_processed, :blocks_skipped, :blocks_failed,
+        :svgs_written, :svgs_skipped, :total_codepoints,
+        keyword_init: true,
+      ) do
+        def initialize(*)
+          super
+          self.blocks_processed ||= 0
+          self.blocks_skipped ||= 0
+          self.blocks_failed ||= 0
+          self.svgs_written ||= 0
+          self.svgs_skipped ||= 0
+          self.total_codepoints ||= 0
+        end
+      end
+
+      # @param output_root [Pathname, String] root for per-block dirs
+      # @param ucd_version [String]
+      # @param fetcher [Fetcher]
+      # @param writer_class [Class] injectable for tests
+      # @param skip_checker_class [Class] injectable for tests
+      def initialize(output_root:, ucd_version:, fetcher:,
+                     writer_class: Writer, skip_checker_class: SkipChecker)
+        @output_root = Pathname.new(output_root)
+        @ucd_version = ucd_version
+        @fetcher = fetcher
+        @writer_class = writer_class
+        @skip_checker_class = skip_checker_class
+      end
+
+      # @param gap_analyzer [GapAnalyzer::Analyzer]
+      # @param blocks [Hash{String=>Ucode::Models::Block}] block_id → Block
+      # @yieldparam summary [BlockSummary]
+      # @return [Aggregate]
+      def run(gap_analyzer:, blocks:)
+        summaries = []
+        gap_analyzer.each_block_gap do |gap|
+          summary = process_gap(gap, blocks.fetch(gap.block_id))
+          summaries << summary
+          yield summary if block_given?
+        end
+        build_aggregate(summaries)
+      end
+
+      private
+
+      def process_gap(gap, block)
+        pdf_path = @fetcher.fetch(block: block)
+
+        if skip?(gap, block, pdf_path)
+          return BlockSummary.new(
+            block_id: block.id, codepoints_extracted: 0,
+            svgs_written: 0, svgs_skipped: gap.size,
+            pdf_sha256: Ucode::CodeChart::Provenance.sha256_of(pdf_path),
+            skipped: true, error: nil,
+          )
+        end
+
+        writer = @writer_class.new(
+          output_root: @output_root, pdf_path: pdf_path,
+          ucd_version: @ucd_version, assigned_only: true,
+          codepoints: gap.missing_codepoints,
+        )
+        writer_summary = writer.write(block)
+        BlockSummary.new(
+          block_id: writer_summary.block,
+          codepoints_extracted: writer_summary.codepoints_extracted,
+          svgs_written: writer_summary.svgs_written,
+          svgs_skipped: 0,
+          pdf_sha256: writer_summary.pdf_sha256,
+          skipped: false, error: nil,
+        )
+      rescue Ucode::Error => e
+        BlockSummary.new(block_id: block.id, codepoints_extracted: 0,
+                         svgs_written: 0, svgs_skipped: 0,
+                         pdf_sha256: "", skipped: false,
+                         error: { class: e.class.name, message: e.message })
+      end
+
+      def skip?(gap, block, pdf_path)
+        @skip_checker_class
+          .new(output_root: @output_root, ucd_version: @ucd_version)
+          .skip?(gap: gap, block: block, pdf_path: pdf_path)
+      end
+
+      def build_aggregate(summaries)
+        agg = Aggregate.new
+        summaries.each do |s|
+          if s.error
+            agg.blocks_failed += 1
+          elsif s.skipped
+            agg.blocks_skipped += 1
+          else
+            agg.blocks_processed += 1
+          end
+          agg.svgs_written += s.svgs_written
+          agg.svgs_skipped += s.svgs_skipped
+          agg.total_codepoints += s.codepoints_extracted
+        end
+        agg
+      end
+
+      # Per-block idempotency check. A block is skippable iff:
+      #   1. Its output dir exists.
+      #   2. Every codepoint in the {BlockGap} has an `.svg` AND a `.json`.
+      #   3. Each sidecar's `source_pdf_sha256` matches the current PDF's sha256.
+      #   4. Each sidecar's `ucd_version` matches the current UCD version.
+      #
+      # Adding a new predicate = one method + one entry in #skip?.
+      # No change to {BatchRunner}.
+      class SkipChecker
+        # @param output_root [Pathname, String]
+        # @param ucd_version [String]
+        def initialize(output_root:, ucd_version:)
+          @output_root = Pathname.new(output_root)
+          @ucd_version = ucd_version
+        end
+
+        def skip?(gap:, block:, pdf_path:)
+          return false unless block_dir(block).exist?
+
+          current_sha = Provenance.sha256_of(pdf_path)
+          gap.missing_codepoints.all? do |cp|
+            sidecar_matches?(block, cp, current_sha)
+          end
+        end
+
+        private
+
+        def block_dir(block)
+          @output_root.join(block.id)
+        end
+
+        def sidecar_path(block, codepoint)
+          block_dir(block).join("#{format_cp(codepoint)}.json")
+        end
+
+        def sidecar_matches?(block, codepoint, current_sha)
+          path = sidecar_path(block, codepoint)
+          return false unless path.exist?
+
+          data = JSON.parse(path.read)
+          data["source_pdf_sha256"] == current_sha &&
+            data["ucd_version"] == @ucd_version
+        rescue JSON::ParserError
+          false
+        end
+
+        def format_cp(codepoint)
+          "U+#{codepoint.to_s(16).upcase.rjust(4, '0')}"
+        end
+      end
+    end
+  end
+end

data/lib/ucode/code_chart/block_index.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+module Ucode
+  module CodeChart
+    # Owns assigned-codepoint membership for one Unicode block.
+    #
+    # The single place in the CodeChart namespace that knows "what
+    # codepoints belong to this block". {Extractor}, {GapAnalyzer},
+    # {BatchRunner}, and the CLI's `list --coverage-gap-only` all
+    # consume this API — none of them re-derive block membership.
+    #
+    # ## Definition of "assigned"
+    #
+    # Two iteration modes, both supported:
+    #
+    #   * `#each_codepoint_in_range` — every Integer in
+    #     `block.range_first..block.range_last`. Fast, no I/O.
+    #   * `#each_assigned_codepoint` — same enumeration today; this
+    #     is the place to swap in a precise UCD-backed filter (walk
+    #     `UnicodeData.txt` for the block range) without changing
+    #     callers. The approximation matches the audit pipeline's
+    #     existing definition ({Ucode::Audit::UcdOnlyReference}).
+    #
+    # ## OCP
+    #
+    # Adding a new mode (e.g. `:reserved_only`, `:noncharacter_only`)
+    # = one new method. Adding a new precision level = override of
+    # `#each_assigned_codepoint`. Callers never change.
+    class BlockIndex
+      # @param block [Ucode::Models::Block]
+      def initialize(block:)
+        @block = block
+      end
+
+      # @return [Ucode::Models::Block]
+      attr_reader :block
+
+      # Yields every codepoint in the block's range, ascending.
+      # Includes reserved/unassigned slots — callers that need only
+      # assigned codepoints should use {#each_assigned_codepoint}
+      # (or filter the result through whatever resolver they
+      # compose).
+      #
+      # @yieldparam codepoint [Integer]
+      # @return [Enumerator, void]
+      def each_codepoint_in_range(&)
+        return enum_for(:each_codepoint_in_range) unless block_given?
+
+        (@block.range_first..@block.range_last).each(&)
+      end
+
+      # Yields assigned codepoints only. Today this is equivalent
+      # to {#each_codepoint_in_range} — the audit pipeline's
+      # existing definition of "assigned" is range-based. A future
+      # enhancement can swap in a UCD-backed precise filter without
+      # changing any caller.
+      #
+      # @yieldparam codepoint [Integer]
+      # @return [Enumerator, void]
+      def each_assigned_codepoint(&)
+        return enum_for(:each_assigned_codepoint) unless block_given?
+
+        each_codepoint_in_range(&)
+      end
+
+      # @return [Array<Integer>] materialized view of
+      #   {#each_assigned_codepoint}, sorted ascending
+      def assigned_codepoints
+        each_assigned_codepoint.to_a
+      end
+
+      # @return [Set<Integer>] frozen membership set; built once
+      def assigned_set
+        @assigned_set ||= assigned_codepoints.to_set.freeze
+      end
+
+      # @param codepoint [Integer]
+      # @return [Boolean]
+      def assigned?(codepoint)
+        assigned_set.include?(codepoint)
+      end
+
+      # @return [Integer] count of assigned codepoints
+      def size
+        assigned_codepoints.size
+      end
+    end
+  end
+end