ucode 0.3.3 → 0.5.0

data/TODO.extract-code-chart/03-codechart-namespace.md

@@ -1,82 +0,0 @@
-# TODO 03 — CodeChart namespace
-
-## Status
-
-Pending. Depends on TODO 02 (block name resolver) so the Extractor
-can consume it; depends on TODO 01 (error class) so the namespace
-can declare typed errors.
-
-## Goal
-
-Establish the `Ucode::CodeChart` namespace as the home for the
-Code Charts per-codepoint extraction feature. The REQ calls this
-namespace `Ucode::CodeChart::*`; we follow the REQ.
-
-This is the autoload-hub file plus the autoload declaration in
-`lib/ucode.rb`.
-
-## Files
-
-- `lib/ucode/code_chart.rb` — new autoload hub (defines
-  `Ucode::CodeChart` and declares child autoloads).
-- `lib/ucode.rb` — add `autoload :CodeChart, "ucode/code_chart"` in
-  the namespace-hubs block.
-
-## Design
-
-### Autoload hub shape
-
-```ruby
-# lib/ucode/code_chart.rb
-module Ucode
-  module CodeChart
-    autoload :Extractor, "ucode/code_chart/extractor"
-    autoload :Provenance, "ucode/code_chart/provenance"
-    autoload :Sidecar, "ucode/code_chart/sidecar"
-    autoload :Writer, "ucode/code_chart/writer"
-  end
-end
-```
-
-Per the global rule (`~/.claude/CLAUDE.md`): declare autoloads in the
-immediate parent namespace's file. `Ucode::CodeChart` is the immediate
-parent of `Extractor`, `Provenance`, `Sidecar`, `Writer`; this file
-is the immediate parent's file.
-
-`Ucode` is the immediate parent of `CodeChart`; the autoload
-declaration `autoload :CodeChart, "ucode/code_chart"` goes in
-`lib/ucode.rb`.
-
-### Why a new namespace (not under `Ucode::Glyphs`)
-
-`Ucode::Glyphs::*` is the existing 4-tier sourcing pipeline
-(`EmbeddedFonts`, `RealFonts`, `LastResort`, `Writer`). The REQ's
-`CodeChart::*` is a feature-facing namespace that orchestrates the
-glyphs pipeline for one specific use case (extracting from a per-block
-PDF for the essenfont donor pipeline). Keeping the feature-facing
-namespace separate from the implementation namespace:
-
-- Lets callers say `Ucode::CodeChart.extract(block: "Sidetic")`
-  without first knowing about `Glyphs::EmbeddedFonts`.
-- Makes it easy to swap the implementation later (different
-  resolution strategy, alternative PDF parser) without breaking the
-  public API.
-- Keeps `Glyphs::` focused on tier mechanics, free of feature
-  ergonomics.
-
-The REQ's namespace name is what we use.
-
-## Acceptance
-
-- `lib/ucode/code_chart.rb` exists with the autoload declarations.
-- `lib/ucode.rb` has the new `autoload :CodeChart, "ucode/code_chart"`
-  in the namespace-hubs block.
-- `Ucode::CodeChart` resolves to a module without loading any of its
-  children.
-
-## Out of scope
-
-- `Ucode::CodeChart::Command` (Thin wrapper). The CLI lives in
-  `lib/ucode/cli.rb` per the existing pattern; no separate
-  `Commands::CodeChartCommand` is introduced (single source of
-  truth for CLI dispatch).

data/TODO.extract-code-chart/04-codechart-extractor.md

@@ -1,154 +0,0 @@
-# TODO 04 — CodeChart::Extractor
-
-## Status
-
-Pending. Depends on TODO 01 (error class), TODO 02 (block name
-resolver), TODO 03 (namespace).
-
-## Goal
-
-`Ucode::CodeChart::Extractor` is the single entry point for
-"extract every assigned codepoint in block X as a standalone SVG."
-
-It orchestrates the existing 4-tier resolver (one source of truth for
-"how do I get the SVG for a given codepoint") and returns a list of
-extraction results — one per codepoint — that the downstream Writer
-serializes to disk.
-
-This is *not* a new extraction pipeline; it is the existing
-`Ucode::Glyphs::Resolver` with per-block inputs pre-configured.
-
-## Files
-
-- `lib/ucode/code_chart/extractor.rb` — `Ucode::CodeChart::Extractor`
-  class.
-- `spec/ucode/code_chart/extractor_spec.rb` — model/value-object
-  specs (constructor invariants, Resolver wiring) plus an integration
-  test against the fixture `spec/fixtures/pdfs/basic_latin.pdf`.
-
-## Design
-
-### Class shape
-
-```ruby
-class Ucode::CodeChart::Extractor
-  Result = Struct.new(:codepoint, :svg, :tier, :provenance, :base_font,
-                      :gid, keyword_init: true)
-
-  def initialize(block:, blocks_txt:, pdf_fetcher: nil,
-                 font_cache_dir: nil, last_resort_root: nil)
-    @block = block                      # Models::Block
-    @blocks_txt = blocks_txt            # Pathname
-    @pdf_fetcher = pdf_fetcher          # optional injectable
-    @font_cache_dir = font_cache_dir    # default: data/pdf-fonts/
-    @last_resort_root = last_resort_root
-  end
-
-  # Walks every assigned codepoint in @block and returns one Result
-  # per codepoint. Codepoints with no glyph from any tier are
-  # silently skipped (no Result yielded) — the REQ's "skip
-  # unassigned codepoints with a warning" is satisfied by the
-  # Resolver returning nil for them.
-  #
-  # @return [Array<Result>]
-  def extract
-  end
-end
-```
-
-### Wiring (single source of truth)
-
-The Extractor does NOT implement tier selection. It builds a
-`Ucode::Glyphs::Resolver` and calls `resolver.resolve(codepoint)` for
-each cp. The Resolver's tier order is preserved (Pillar 1 → 2 → 3
-for this feature; no Tier 1 because we're starting from the Code
-Charts PDF, not a real-font source).
-
-```ruby
-def build_resolver
-  pdf = fetch_pdf!
-  embedded_source = Glyphs::EmbeddedFonts::Source.new(
-    pdf: pdf, cache_dir: @font_cache_dir,
-  )
-  catalog = Glyphs::EmbeddedFonts::Catalog.new(embedded_source)
-  pillar1 = Glyphs::Sources::Pillar1EmbeddedTounicode.new(
-    renderer: Glyphs::EmbeddedFonts::Renderer.new(catalog),
-  )
-  pillar3 = Glyphs::Sources::Pillar3LastResort.new(
-    renderer: Glyphs::LastResort::Renderer.new(
-      Glyphs::LastResort::Source.new(root: @last_resort_root),
-    ),
-  )
-  Glyphs::Resolver.new(
-    sources: [pillar1, pillar3],
-    order: %i[pillar1 pillar3],
-  )
-end
-```
-
-The tier ordering is documented inline: we skip Pillar 2 because for
-the CodeChart use case the catalog's ToUnicode is the dominant path
-and Pillar 2 (positional correlation) is reserved for fonts where
-Pillar 1 fails. If a future use case needs Pillar 2, add it without
-changing this constructor — that's the OCP payoff of consuming the
-Resolver.
-
-### Why no Tier 1
-
-Tier 1 (real-font cmap) needs a configured `SourceConfig` mapping
-block → font. The CodeChart use case is for blocks where no real
-font exists (Sidetic, Egyptian Ext-B). Tier 1 wouldn't contribute
-anything. The Extractor accepts a Tier 1 source in the future by
-having callers pass a fully-built Resolver instead of constructing
-one internally.
-
-### Why PDF fetch is delegated to `PdfFetcher`
-
-`Ucode::Glyphs::PdfFetcher` is the existing seam for resolving a
-block to its PDF on disk (per-block cache + monolith fallback). It
-already handles `force:` and the cache directory. The Extractor
-constructs a `PdfFetcher` per call (cheap — it's just a path
-resolver) and reuses it across codepoints.
-
-### Per-codepoint loop
-
-```ruby
-def extract
-  resolver = build_resolver
-  @block.codepoint_ids.flat_map do |cp_id|
-    cp = Integer(cp_id.delete_prefix("U+"), 16)
-    resolver_result = resolver.resolve(cp)
-    next nil unless resolver_result&.svg
-
-    Result.new(
-      codepoint: cp,
-      svg: resolver_result.svg,
-      tier: resolver_result.tier,
-      provenance: resolver_result.provenance,
-    )
-  end.compact
-end
-```
-
-The Resolver returns a `Sources::Result` (tier + codepoint + svg +
-provenance). We adapt that to the Extractor's `Result` (with
-codepoint + svg + tier + provenance), stripping the resolver-specific
-shape at the boundary.
-
-## Acceptance
-
-- `Extractor.new(block: ..., blocks_txt: ...)` constructs without
-  raising when the block and PDF are present.
-- `#extract` returns one Result per codepoint that any tier
-  produced a glyph for.
-- `#extract` skips codepoints no tier could produce (returns no
-  Result, not a Result-with-nil).
-- Integration test: against the fixture PDF, at least one codepoint's
-  Result has tier `:pillar1` and provenance `"pillar-1:embedded-tounicode"`.
-
-## Out of scope
-
-- Writing files — that's `CodeChart::Writer` (TODO 06).
-- Provenance JSON — that's `CodeChart::Sidecar` (TODO 05).
-- Tier 1 (real-font) source injection — not needed for the REQ's
-  blocks. Future extension point if a real-font fallback is desired.

data/TODO.extract-code-chart/05-provenance-and-sidecar.md

@@ -1,147 +0,0 @@
-# TODO 05 — CodeChart::Provenance and CodeChart::Sidecar
-
-## Status
-
-Pending. Depends on TODO 03 (namespace). Depends on TODO 04 (Extractor)
-because Sidecar consumes the Extractor's Result.
-
-## Goal
-
-`Provenance` is the value object carrying the metadata the REQ (R5)
-requires for each extracted SVG's sidecar JSON. `Sidecar` is the
-writer that serializes one Provenance to disk next to its SVG.
-
-## Files
-
-- `lib/ucode/code_chart/provenance.rb` — `Ucode::CodeChart::Provenance`
-  Struct with all REQ R5 fields plus the construction helper.
-- `lib/ucode/code_chart/sidecar.rb` — `Ucode::CodeChart::Sidecar`
-  class (writes a sidecar JSON next to an SVG, idempotent via the
-  existing `Ucode::Repo::AtomicWrites`).
-- `spec/ucode/code_chart/provenance_spec.rb`
-- `spec/ucode/code_chart/sidecar_spec.rb`
-
-## Design
-
-### Provenance value object
-
-The REQ (R5) lists these fields:
-
-```json
-{
-  "codepoint": "U+10920",
-  "block": "Sidetic",
-  "source_pdf_url": "https://www.unicode.org/charts/PDF/U-10920.pdf",
-  "source_pdf_sha256": "...",
-  "ucd_version": "17.0.0",
-  "extracted_at": "2026-06-30T12:00:00Z",
-  "extractor_version": "0.1.0"
-}
-```
-
-`Struct` is the right tool — single source of truth for the schema,
-keyword-init for clarity, immutable-by-convention. Mirror the
-existing `Ucode::Repo::BuildReportAccumulator` pattern.
-
-```ruby
-Provenance = Struct.new(
-  :codepoint,           # String "U+10920"
-  :block,               # String "Sidetic"
-  :source_pdf_url,      # String
-  :source_pdf_sha256,   # String (hex digest)
-  :ucd_version,         # String "17.0.0"
-  :extracted_at,        # String ISO8601 UTC
-  :extractor_version,   # String "0.2.0"
-  keyword_init: true,
-)
-```
-
-`extractor_version` reads from `Ucode::VERSION` so it stays in sync
-with the gem — single source of truth.
-
-`extracted_at` is set at construction (not at file write) so the
-field describes the extraction event, not the serialization event.
-
-### Provenance → Hash serialization
-
-`Provenance#to_h` returns the hash form. NO hand-rolled `to_json` /
-`from_json` per the global rule — `Provenance` is a value object, but
-its schema is simple enough that `to_h` + `JSON.pretty_generate` is
-the framework-driven approach (lutaml-model is overkill for a flat
-struct).
-
-Wait — re-reading the global rule: "ALL (de)serialization goes through
-the framework. In Coradoc and any project using `lutaml-model`." So
-the rule is for projects using lutaml-model, and ucode uses lutaml-model
-for UCD models. This Provenance struct is not a UCD model — it's a
-feature-local value object. JSON via `JSON.pretty_generate(provenance.to_h)`
-is acceptable and avoids ceremony.
-
-`to_h` produces a hash with the Struct's keyword keys. No
-indirection, no lutaml-model mapping for what is effectively a
-record.
-
-### Sidecar writer
-
-```ruby
-class Sidecar
-  include Ucode::Repo::AtomicWrites
-
-  def initialize(output_root:)
-    @output_root = Pathname.new(output_root)
-  end
-
-  # Writes <output_root>/<cp_id>.json next to the corresponding SVG.
-  # Idempotent: re-writing the same content is a no-op (byte-stable).
-  #
-  # @param provenance [Ucode::CodeChart::Provenance]
-  # @return [Pathname] the written path
-  def write(provenance)
-    path = path_for(provenance)
-    payload = JSON.pretty_generate(provenance.to_h)
-    write_atomic(path, payload + "\n")
-    path
-  end
-
-  private
-
-  def path_for(provenance)
-    @output_root.join("#{provenance.codepoint}.json")
-  end
-end
-```
-
-`Repo::AtomicWrites#write_atomic` is the project's single source of
-truth for idempotent file writes — bytes-identical re-writes are
-no-ops (the temp-file rename is skipped when content matches).
-Reuse, don't reimplement.
-
-### Why a separate Sidecar class
-
-A `Provenance.to_disk(path)` method would couple the value object
-to I/O. Keeping the writer separate lets:
-- Tests assert `Provenance#to_h` without touching disk.
-- The Writer (TODO 06) compose `Extractor` + `Sidecar` with explicit
-  dependency injection (seam for testing).
-- Future formats (e.g. a different sidecar schema) replace Sidecar
-  without touching Provenance.
-
-This is MECE: Provenance is data; Sidecar is I/O; Writer is
-orchestration.
-
-## Acceptance
-
-- `Provenance.new(codepoint: "U+10920", block: "Sidetic", ...)`
-  constructs without raising.
-- `Provenance#to_h` returns a Hash with exactly the REQ's fields.
-- `Sidecar#write(provenance)` writes `<codepoint>.json` next to
-  where the SVG lives; the JSON content matches `Provenance#to_h`.
-- Re-writing the same Provenance is a no-op (file unchanged).
-- Specs cover all five REQ fields plus the idempotency guarantee.
-
-## Out of scope
-
-- License attribution text — the REQ mentions `LICENSE-SOURCES.md`
-  obligations, but that's a fontist-side concern (downstream
-  essenfont build). Ucode emits provenance; the consumer stitches
-  attribution.

data/TODO.extract-code-chart/06-codechart-writer.md

@@ -1,134 +0,0 @@
-# TODO 06 — CodeChart::Writer
-
-## Status
-
-Pending. Depends on TODO 04 (Extractor) and TODO 05 (Provenance +
-Sidecar).
-
-## Goal
-
-`Ucode::CodeChart::Writer` is the single entry point for
-"extract every codepoint in block X and write SVG + sidecar JSON
-files under output_dir." It's the orchestration layer the CLI
-calls and the only thing that touches disk.
-
-## Files
-
-- `lib/ucode/code_chart/writer.rb` — `Ucode::CodeChart::Writer` class.
-- `spec/ucode/code_chart/writer_spec.rb`
-
-## Design
-
-### Class shape
-
-```ruby
-class Ucode::CodeChart::Writer
-  Summary = Struct.new(:block, :codepoints_total, :svgs_written,
-                       :sidecars_written, :pdf_sha256, keyword_init: true)
-
-  def initialize(output_root:, pdf_path:, cache_dir: nil,
-                 last_resort_root: nil, blocks_txt: nil)
-    @output_root = Pathname.new(output_root)
-    @pdf_path = Pathname.new(pdf_path)
-    @cache_dir = cache_dir
-    @last_resort_root = last_resort_root
-    @blocks_txt = blocks_txt || Ucode::Cache.ucd_dir(Ucode::VersionResolver.resolve(nil)).join("Blocks.txt")
-    @sidecar = Sidecar.new(output_root: @output_root)
-  end
-
-  # Extracts every codepoint in @block (a Models::Block) and writes
-  # SVG + sidecar JSON under @output_root. Returns a Summary.
-  #
-  # @param block [Ucode::Models::Block]
-  # @return [Summary]
-  def write(block)
-  end
-end
-```
-
-### Per-codepoint flow (single source of truth)
-
-```ruby
-def write(block)
-  output_root_for(block).mkpath
-  pdf_sha = sha256(@pdf_path)
-
-  extractor = Extractor.new(
-    block: block,
-    blocks_txt: @blocks_txt,
-    pdf_path: @pdf_path,
-    cache_dir: @cache_dir,
-    last_resort_root: @last_resort_root,
-  )
-  results = extractor.extract
-
-  svgs = 0
-  sidecars = 0
-  results.each do |result|
-    svg_path = output_root_for(block).join("#{cp_id(result.codepoint)}.svg")
-    File.write(svg_path, result.svg) unless svg_path.exist? && File.read(svg_path) == result.svg
-    svgs += 1 if svg_path.exist?
-
-    provenance = build_provenance(block, result.codepoint, pdf_sha)
-    @sidecar.write(provenance)
-    sidecars += 1
-  end
-
-  Summary.new(
-    block: block.id,
-    codepoints_total: results.size,
-    svgs_written: svgs,
-    sidecars_written: sidecars,
-    pdf_sha256: pdf_sha,
-  )
-end
-```
-
-### Why not use `Repo::AtomicWrites` for the SVGs
-
-The Sidecar uses `Repo::AtomicWrites` because JSON has a stable
-canonical form. SVG output from `EmbeddedFonts::Svg#to_s` is also
-byte-stable, but the writer pattern is simpler with `File.write` —
-the byte-equality check above guarantees idempotency at the I/O
-layer. Both paths reach the same outcome.
-
-If future output formats gain non-stable serialization (timestamps,
-random IDs), the SVG path will need `Repo::AtomicWrites` too. Until
-then, simpler is better.
-
-### Why compute `pdf_sha256` once
-
-Every Provenance in this block carries the same `source_pdf_sha256`.
-Computing it once avoids 32+ disk reads of the PDF per block
-extraction. The Writer is the single place that knows "one block,
-one PDF, one hash" — pushing the calculation into Sidecar or
-Provenance would require either (a) repeated computation per
-Provenance or (b) a parameter-passing thread through Extractor. Both
-violate locality.
-
-### Output layout
-
-`<output_root>/<block_id>/<U+XXXX>.svg` and `<U+XXXX>.json`.
-
-One folder per block keeps the `Writer`'s output self-contained and
-discoverable — a downstream consumer (fontisan) can iterate a block's
-folder without scanning the whole tree. This mirrors the existing
-`Ucode::Repo::Writers::BlocksWriter` output convention (one folder
-per block, index.json inside).
-
-## Acceptance
-
-- `Writer#write(block)` creates `<output_root>/<block_id>/` and
-  fills it with `<U+XXXX>.svg` + `<U+XXXX>.json` for every
-  extracted codepoint.
-- Re-running `Writer#write(block)` with no changes produces
-  byte-identical files (no rewrites).
-- `Summary#svgs_written` equals the number of extracted codepoints.
-- Specs cover the full lifecycle including idempotency.
-
-## Out of scope
-
-- Per-block write isolation — concurrent `Writer#write` calls for
-  different blocks are safe (different folders), but the Writer is
-  not thread-safe within a single block. That's a parallel-extraction
-  concern, not a per-block concern.

data/TODO.extract-code-chart/07-codechart-cli.md

@@ -1,135 +0,0 @@
-# TODO 07 — CodeChart CLI
-
-## Status
-
-Pending. Depends on TODO 06 (Writer), TODO 04 (Extractor),
-TODO 02 (block name resolver).
-
-## Goal
-
-`ucode code-chart fetch | extract | list` — the REQ (R4) commands.
-Thin Thor wrappers that delegate to the existing `CodeChart::*`
-modules. No orchestration logic in the CLI; every command is a
-single delegation.
-
-## Files
-
-- `lib/ucode/cli.rb` — add the `CodeChartCmd` Thor subcommand class.
-- `spec/ucode/cli_spec.rb` (extend existing) — verify the new
-  subcommand wires up.
-
-## Design
-
-### Subcommand shape
-
-```ruby
-class Cli < Thor
-  # …existing commands…
-
-  class CodeChartCmd < Thor
-    desc "fetch --block BLOCK", "Download the Code Charts PDF for a block"
-    option :block, type: :string, required: true,
-                  desc: "Block identifier (e.g. Sidetic)"
-    def fetch
-      puts JSON.pretty_generate(
-        Commands::FetchCommand.new.fetch_charts(
-          VersionResolver.resolve(nil),
-          block_first_cps: [block_first_cp!(options[:block])],
-        ),
-      )
-    end
-
-    desc "extract --block BLOCK --to DIR", "Extract per-codepoint SVG + provenance sidecars"
-    option :block, type: :string, required: true,
-                  desc: "Block identifier (e.g. Sidetic)"
-    option :to, type: :string, required: true,
-                desc: "Output directory"
-    def extract
-      # ...
-    end
-
-    desc "list", "List blocks that have Code Charts PDFs available locally"
-    def list
-      # ...
-    end
-  end
-
-  desc "code-chart", "Extract per-codepoint SVG glyphs from Unicode Code Charts PDFs"
-  subcommand "code-chart", CodeChartCmd
-end
-```
-
-### `extract` flow
-
-```ruby
-def extract
-  Ucode::Commands::FetchCommand.new.fetch_charts(
-    VersionResolver.resolve(nil),
-    block_first_cps: [block_first_cp!(options[:block])],
-  )
-  blocks_txt = Ucode::Cache.ucd_dir(VersionResolver.resolve(nil)).join("Blocks.txt")
-  block = Parsers::Blocks.find_by_name(blocks_txt, options[:block]) or
-    raise Thor::Error, "Unknown block: #{options[:block].inspect}"
-  pdf = Ucode::Glyphs::PdfFetcher.new(
-    VersionResolver.resolve(nil),
-    monolith_path: nil,
-    blocks: [block],
-  ).fetch(block_first_cp: block.range_first, force: false) or
-    raise Thor::Error, "PDF unavailable for block #{options[:block]}"
-
-  writer = Ucode::CodeChart::Writer.new(
-    output_root: Pathname.new(options[:to]),
-    pdf_path: pdf,
-    blocks_txt: blocks_txt,
-  )
-  summary = writer.write(block)
-  puts JSON.pretty_generate(summary.to_h.compact)
-end
-
-def block_first_cp!(block_id)
-  cache = Ucode::Cache.ucd_dir(VersionResolver.resolve(nil))
-  block = Ucode::Parsers::Blocks.find_by_name(cache.join("Blocks.txt"), block_id)
-  raise Thor::Error, "Unknown block: #{block_id.inspect}" unless block
-  block.range_first
-end
-```
-
-### Why `Ucode::Commands::FetchCommand.new.fetch_charts` for fetch
-
-`fetch_charts` is the existing CLI hook for "download a Code Charts
-PDF for these block first-cps." We just call it with the block's
-first cp. Reuse, don't reimplement.
-
-### Why resolve version once at the top of `extract`
-
-Per Candidate 4 of the architecture review (`refactor/build-context-resolve-version-once`,
-merged): every CLI method resolves the version once and threads it
-through. This CLI method does the same — one call per invocation.
-
-### Why no separate `Commands::CodeChartCommand` class
-
-Following the existing pattern (e.g. `Cli::Audit` calls
-`Commands::Audit::*Command` *only* when the logic is non-trivial).
-The CodeChart commands are trivial delegations — a one-liner each.
-The CLI methods call `CodeChart::Writer` and `CodeChart::Extractor`
-directly. Adding a `Commands::CodeChartCommand` class would be
-indirection without a payoff.
-
-If the extract logic grows (e.g. progress reporting, partial
-extraction), extract it into a Command class at that point.
-
-## Acceptance
-
-- `ucode code-chart fetch --block Sidetic` downloads the PDF.
-- `ucode code-chart extract --block Sidetic --to /tmp/s/` extracts
-  to the given directory.
-- `ucode code-chart list` prints available blocks.
-- Unknown block names produce a clean error, not a stack trace.
-- The CLI matches the REQ's signature exactly.
-
-## Out of scope
-
-- `--version` flag (the REQ doesn't specify, and existing commands
-  default to the configured version).
-- `--format svg|glif` (the REQ specifies SVG; `.glif` output is a
-  future extension).