ucode 0.2.0 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/Gemfile +10 -2
  3. data/README.md +66 -20
  4. data/Rakefile +19 -8
  5. data/TODO.extract-code-chart/01-pdf-fetch-validation.md +80 -0
  6. data/TODO.extract-code-chart/02-block-name-resolver.md +68 -0
  7. data/TODO.extract-code-chart/03-codechart-namespace.md +82 -0
  8. data/TODO.extract-code-chart/04-codechart-extractor.md +154 -0
  9. data/TODO.extract-code-chart/05-provenance-and-sidecar.md +147 -0
  10. data/TODO.extract-code-chart/06-codechart-writer.md +134 -0
  11. data/TODO.extract-code-chart/07-codechart-cli.md +135 -0
  12. data/TODO.extract-code-chart/08-specs.md +87 -0
  13. data/config/unicode17_universal_glyph_set.yml +1 -1
  14. data/lib/ucode/audit/reference_factory.rb +1 -1
  15. data/lib/ucode/cli.rb +101 -0
  16. data/lib/ucode/code_chart/extractor.rb +120 -0
  17. data/lib/ucode/code_chart/provenance.rb +82 -0
  18. data/lib/ucode/code_chart/sidecar.rb +52 -0
  19. data/lib/ucode/code_chart/writer.rb +128 -0
  20. data/lib/ucode/code_chart.rb +39 -0
  21. data/lib/ucode/commands/fetch.rb +1 -1
  22. data/lib/ucode/commands/glyphs.rb +1 -1
  23. data/lib/ucode/commands/lookup.rb +1 -1
  24. data/lib/ucode/commands/parse.rb +1 -1
  25. data/lib/ucode/coordinator/indices.rb +2 -2
  26. data/lib/ucode/error.rb +11 -0
  27. data/lib/ucode/fetch/code_charts.rb +3 -4
  28. data/lib/ucode/fetch/http.rb +75 -7
  29. data/lib/ucode/glyphs/page_renderer.rb +15 -2
  30. data/lib/ucode/glyphs/pipeline.rb +1 -2
  31. data/lib/ucode/parsers/blocks.rb +34 -0
  32. data/lib/ucode/repo/aggregate_writer.rb +1 -1
  33. data/lib/ucode/repo/writers/blocks_writer.rb +13 -13
  34. data/lib/ucode/repo/writers/enums_writer.rb +2 -2
  35. data/lib/ucode/repo/writers/indexes_writer.rb +4 -4
  36. data/lib/ucode/repo/writers/manifest_writer.rb +4 -4
  37. data/lib/ucode/repo/writers/named_sequences_writer.rb +1 -1
  38. data/lib/ucode/repo/writers/planes_writer.rb +17 -17
  39. data/lib/ucode/repo/writers/relationships_writer.rb +1 -1
  40. data/lib/ucode/repo/writers/scripts_writer.rb +6 -6
  41. data/lib/ucode/repo/writers.rb +1 -1
  42. data/lib/ucode/version.rb +1 -1
  43. data/lib/ucode.rb +3 -0
  44. data/ucode.gemspec +6 -1
  45. metadata +19 -6
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d9d5a29a388017c0338a8d2723f97dfd4ac8985324b6ab7fc9b8921a5f4b0b82
4
- data.tar.gz: f6988cf20f74efc1b94ebfedbd7b9507680de1c0880ffd28ac9a4a00851195a8
3
+ metadata.gz: e2e60f073662cc78885f8ee6a0333a2307a8b894ddb9789b1e206f6ae63d25e3
4
+ data.tar.gz: 50b3984de26589d0aab193250e9c8dd3f56fd0d10a4a77a21ffdcb236c38c737
5
5
  SHA512:
6
- metadata.gz: 890fec1b40ed6016b269982d32d28b15eb2683dfa7d4e9fc32e5a6633c46fb6f8a94b0d004d1bbd4381fb5bbe629faf00984729036005c12e1558c65d70afe4f
7
- data.tar.gz: 792b58d5eb3434b05008740457b1c432f5810bd45b2d0d80936bd0c539445098b835c1d242093487cf0a4a4754efba2fe725c500b2d3f1a23f4906b4a17809dc
6
+ metadata.gz: 88d9ef3df0f99af9b3897cd429666092d48a373d9c60d3c83a6e06a59404e0523c736a8d971fbb6d6d29c9b4b80d9610cad54a64cda439326e19a80670d67ba9
7
+ data.tar.gz: 8146f6984defddb5b45204fe7a8852cd3051bf1474584eddc7489e5ec9fd4900d48452ac5462a1c6c2ac9b8213823ace68cebe6c5b5176dd0c25c922f8f1d4ba
data/Gemfile CHANGED
@@ -4,8 +4,13 @@ source "https://rubygems.org"
4
4
 
5
5
  gemspec
6
6
 
7
+ # Rake must be in the default group (not :development) because the
8
+ # GHA release workflow runs `bundle exec rake release` to publish the
9
+ # gem. The release runner installs with `--without development`, so
10
+ # gems in the :development group are excluded.
11
+ gem "rake"
12
+
7
13
  group :development do
8
- gem "rake"
9
14
  gem "rspec"
10
15
  gem "rubocop"
11
16
  gem "rubocop-performance"
@@ -19,4 +24,7 @@ end
19
24
  # local sibling checkout, set FONTISAN_PATH before running bundle.
20
25
  # FONTISAN_PATH=../fontisan bundle install
21
26
  gem "fontisan", path: ENV["FONTISAN_PATH"] if ENV["FONTISAN_PATH"]
22
- gem "fontisan", "~> 0.2" unless ENV["FONTISAN_PATH"]
27
+ # Pin fontisan to 0.2.22 0.2.23+ removed
28
+ # `Fontisan::Commands::AuditCommand` and 0.4.x removed the Audit
29
+ # subsystem entirely. See ucode.gemspec for the rationale.
30
+ gem "fontisan", "= 0.2.22" unless ENV["FONTISAN_PATH"]
data/README.md CHANGED
@@ -2,14 +2,16 @@
2
2
 
3
3
  `ucode` is a Ruby toolkit for the Unicode Character Database (UCD). It turns the
4
4
  official UCD text files into a structured, browsable dataset: one JSON document
5
- per assigned codepoint, plus a Vitepress site for navigation.
5
+ per assigned codepoint, plus a Vitepress site for navigation. It also extracts
6
+ per-codepoint SVG glyphs from the Unicode Code Charts PDFs and audits font
7
+ coverage against the Unicode baseline.
6
8
 
7
- > **Status (v0.1).** The JSON dataset, lookup index, and Vitepress site are
8
- > production-ready. **SVG glyph extraction from the Code Charts PDFs is
9
- > experimental and deferred to v0.2** see
10
- > [Glyph extraction (experimental)](#glyph-extraction-experimental) below.
9
+ > **Status (v0.2.1).** The JSON dataset, lookup index, Vitepress site, and
10
+ > 4-tier glyph extraction pipeline are production-ready. The `ucode code-chart`
11
+ > subcommand extracts standalone SVGs + provenance sidecars from per-block
12
+ > Code Charts PDFs. Font coverage auditing (`ucode audit`) is production-ready.
11
13
 
12
- ## What you get (v0.1)
14
+ ## What you get (v0.2)
13
15
 
14
16
  - **Per-codepoint JSON** at `output/blocks/<BLOCK>/<U+XXXX>/index.json` with
15
17
  full UCD properties, the human-curated relationships from `NamesList.txt`
@@ -21,6 +23,14 @@ per assigned codepoint, plus a Vitepress site for navigation.
21
23
  relationships, named sequences, manifest.
22
24
  - **SQLite lookup index** for fast codepoint → block/script/char queries.
23
25
  - **Vitepress site** at `site/` for browsing Plane → Block → Character.
26
+ - **4-tier glyph extraction** — per-codepoint `glyph.svg` sourced from real
27
+ fonts (Tier 1), PDF-embedded fonts (Pillars 1+2), or Last Resort UFO
28
+ (Pillar 3).
29
+ - **Per-block Code Chart extraction** — `ucode code-chart extract` produces
30
+ standalone SVG + provenance JSON for every codepoint in a block.
31
+ - **Font coverage audit** — `ucode audit` compares a font's cmap against the
32
+ Unicode baseline and reports per-block coverage, missing codepoints, and
33
+ optional HTML browsers.
24
34
 
25
35
  ## Install
26
36
 
@@ -31,7 +41,7 @@ gem install ucode
31
41
  Or in a Gemfile:
32
42
 
33
43
  ```ruby
34
- gem "ucode", "~> 0.1"
44
+ gem "ucode", "~> 0.2"
35
45
  ```
36
46
 
37
47
  ## Quick start
@@ -107,7 +117,39 @@ Then:
107
117
  cd site && npm install && npm run dev
108
118
  ```
109
119
 
110
- ## Glyph extraction (experimental in v0.1; concrete plan for v0.2)
120
+ ## Code Chart extraction
121
+
122
+ Extract per-codepoint SVG glyphs from a Unicode Code Charts PDF, with
123
+ provenance sidecar JSON:
124
+
125
+ ```sh
126
+ # Download the Code Charts PDF for a block
127
+ ucode code-chart fetch --block Sidetic
128
+
129
+ # Extract every codepoint as SVG + provenance JSON
130
+ ucode code-chart extract --block Sidetic --to /tmp/sidetic/
131
+
132
+ # List cached Code Charts PDFs
133
+ ucode code-chart list
134
+ ```
135
+
136
+ Output layout:
137
+
138
+ ```
139
+ /tmp/sidetic/Sidetic/
140
+ U+10920.svg # vector glyph outline
141
+ U+10920.json # provenance: source PDF, sha256, version, timestamp
142
+ U+10921.svg
143
+ U+10921.json
144
+ ...
145
+ ```
146
+
147
+ The extractor uses the existing 4-tier glyph sourcing pipeline (Tier 1 →
148
+ Pillar 1 → Pillar 2 → Pillar 3). No new extraction logic — the
149
+ `ucode code-chart` subcommand is a thin CLI wrapper over `Ucode::CodeChart::Writer`,
150
+ which orchestrates `Ucode::Glyphs::Resolver` for each codepoint.
151
+
152
+ ## Glyph extraction (4-tier pipeline)
111
153
 
112
154
  The `ucode glyphs` command and the `--include-glyphs` flag on `ucode build`
113
155
  are **opt-in and experimental in v0.1**. They emit per-codepoint `glyph.svg`
@@ -436,29 +478,33 @@ from the dataset.
436
478
 
437
479
  ## Architecture
438
480
 
439
- Five concerns, each isolated:
481
+ Seven concerns, each isolated:
440
482
 
441
483
  1. **`Ucode::Models`** — `lutaml-model` classes for every UCD aggregate.
442
484
  2. **`Ucode::Parsers`** — one streaming parser per UCD text file.
443
485
  3. **`Ucode::Coordinator`** — single-pass enrichment that merges indices
444
486
  into each `CodePoint` as it streams.
445
- 4. **`Ucode::Repo`** — atomic, idempotent writers for the output tree.
446
- 5. **`Ucode::Glyphs`** vector glyph extraction from Code Charts PDFs
447
- (experimental in v0.1).
448
- 6. **`Ucode::Site`** Vitepress scaffold + config/page generator.
487
+ 4. **`Ucode::Repo`** — atomic, idempotent writers for the output tree
488
+ (per-concern writers under `Repo::Writers::*`).
489
+ 5. **`Ucode::Glyphs`** — 4-tier vector glyph extraction from Code Charts PDFs
490
+ (RealFonts, EmbeddedFonts, LastResort + Resolver).
491
+ 6. **`Ucode::CodeChart`** — per-block SVG extraction + provenance sidecar
492
+ for the "Code Chart donor" use case.
493
+ 7. **`Ucode::Site`** — Vitepress scaffold + config/page generator.
449
494
 
450
495
  CLI is thin Thor dispatch over `Ucode::Commands::*`. Each command class
451
- is a pure, in-process testable unit.
496
+ is a pure, in-process testable unit. Version resolution happens once
497
+ per CLI invocation and threads through to all sub-commands.
452
498
 
453
- See `CLAUDE.md` for the full architecture notes. See
454
- `docs/FONTISAN_MIGRATION.md` for the fontisan integration plan.
499
+ See `docs/architecture.md` for the canonical architecture reference and
500
+ `docs/adr/` for Architecture Decision Records.
455
501
 
456
502
  ## Authoritative source
457
503
 
458
- ucode parses the **UCD text files** (per UAX #44). The
459
- `ucd.all.flat.xml` shipped with the repo is reference-onlyit omits
460
- the human-curated relationship data in `NamesList.txt` and has partial
461
- Unihan coverage. We never parse it.
504
+ ucode parses the **UCD text files** (`UnicodeData.txt`, `NamesList.txt`,
505
+ `Blocks.txt`, etc.) per UAX #44. Never uses the flat XML dumpthe text
506
+ files carry the human-curated relationship data that makes this project
507
+ valuable.
462
508
 
463
509
  ## License
464
510
 
data/Rakefile CHANGED
@@ -4,15 +4,26 @@ require "rubygems"
4
4
  require "rake"
5
5
  require "bundler/gem_tasks"
6
6
 
7
- require "rspec/core/rake_task"
8
- RSpec::Core::RakeTask.new(:spec)
7
+ begin
8
+ require "rspec/core/rake_task"
9
+ RSpec::Core::RakeTask.new(:spec)
10
+ rescue LoadError
11
+ # rspec is in the :development group; not available in the
12
+ # release runner (`bundle install --without development`).
13
+ end
9
14
 
10
- require "rubocop/rake_task"
11
- RuboCop::RakeTask.new
15
+ begin
16
+ require "rubocop/rake_task"
17
+ RuboCop::RakeTask.new
18
+ rescue LoadError
19
+ end
12
20
 
13
- require "yard"
14
- YARD::Rake::YardocTask.new do |t|
15
- t.options = ["--output-dir", "docs/api"]
21
+ begin
22
+ require "yard"
23
+ YARD::Rake::YardocTask.new do |t|
24
+ t.options = ["--output-dir", "docs/api"]
25
+ end
26
+ rescue LoadError
16
27
  end
17
28
 
18
- task default: %i[spec rubocop]
29
+ task default: %i[spec rubocop]
@@ -0,0 +1,80 @@
1
+ # TODO 01 — PDF fetch validation
2
+
3
+ ## Status
4
+
5
+ Pending.
6
+
7
+ ## Goal
8
+
9
+ Raise a typed `Ucode::CodeChartNotFoundError` when a Unicode Code
10
+ Charts PDF cannot be downloaded or fails content validation. The REQ
11
+ (R1) requires:
12
+
13
+ - HTTP 4xx / 5xx → `CodeChartNotFoundError`
14
+ - `Content-Type: application/pdf`
15
+ - First 4 bytes are `%PDF`
16
+
17
+ ## Files
18
+
19
+ - `lib/ucode/error.rb` — add `Ucode::CodeChartNotFoundError` under the
20
+ `GlyphError` subtree.
21
+ - `lib/ucode.rb` — add an `autoload` for the new class so any rescue
22
+ clause triggers one load of `error.rb`.
23
+ - `lib/ucode/fetch/http.rb` — extend `Http.get` with an optional
24
+ `validate:` keyword. When `validate: :pdf`, after a successful
25
+ download, verify the `Content-Type` response header starts with
26
+ `application/pdf` and the first 4 bytes of the body are `%PDF`.
27
+ - `lib/ucode/fetch/code_charts.rb` — pass `validate: :pdf` to `Http.get`
28
+ for every chart PDF download.
29
+ - `spec/ucode/fetch/code_charts_spec.rb` (new) — cover the happy path
30
+ and the validation failure modes.
31
+
32
+ ## Design
33
+
34
+ ### Why a new error class
35
+
36
+ `FetchError` already covers transport failures, but it doesn't carry
37
+ "this URL produced an HTML error page / 404 / non-PDF body" semantics.
38
+ Splitting the type keeps existing `rescue Ucode::FetchError` callers
39
+ from accidentally swallowing the typed signal that "we expected a PDF
40
+ and didn't get one" — which is a different problem class from "the
41
+ network was down."
42
+
43
+ `CodeChartNotFoundError < Ucode::Error` (under `GlyphError`) reflects
44
+ the REQ's framing: the chart for the requested block is not
45
+ obtainable.
46
+
47
+ ### Why `validate:` is optional on `Http.get`
48
+
49
+ `Http` is the single network boundary (per the comment at the top of
50
+ `http.rb`). All callers funnel through it. Adding an optional
51
+ keyword keeps the MECE pattern intact: non-PDF callers (UCD zip,
52
+ Unihan zip, font zip) pass nothing; the single PDF caller passes
53
+ `validate: :pdf`. No second network boundary is needed.
54
+
55
+ ### Why no separate "magic bytes" check class
56
+
57
+ Magic-byte verification is 4 lines of code; extracting it into a
58
+ class would be ceremony. Inline check after `write_body`, raising
59
+ the typed error with the offending content-type or magic bytes in
60
+ the context payload.
61
+
62
+ ## Acceptance
63
+
64
+ - `Http.get(url, dest:, validate: :pdf)` raises
65
+ `CodeChartNotFoundError` (a) when the response Content-Type is not
66
+ `application/pdf`, (b) when the first 4 bytes are not `%PDF`.
67
+ - `Fetch::CodeCharts.call(version, block_first_cps: [...])` raises
68
+ `CodeChartNotFoundError` when the unicode.org endpoint returns
69
+ 4xx/5xx or non-PDF content.
70
+ - Existing callers of `Http.get` that don't pass `validate:` are
71
+ unchanged.
72
+ - Spec coverage: happy path, HTTP 404, wrong content-type, truncated
73
+ body missing the `%PDF` magic.
74
+
75
+ ## Out of scope
76
+
77
+ - SHA-256 verification of the PDF — that's a downstream concern (the
78
+ Code Charts are not versioned by hash on unicode.org).
79
+ - Resumable / partial downloads — the existing `Http` writes a
80
+ `.part` then renames; that's sufficient.
@@ -0,0 +1,68 @@
1
+ # TODO 02 — Block name resolver
2
+
3
+ ## Status
4
+
5
+ Pending. Depends on nothing.
6
+
7
+ ## Goal
8
+
9
+ Add a class method `Ucode::Parsers::Blocks.find_by_name(name)` that
10
+ resolves a Unicode block identifier (e.g. `"Sidetic"`,
11
+ `"Egyptian_Hieroglyphs_Extended-B"`) to the `Ucode::Models::Block`
12
+ instance in a given version's cached `Blocks.txt`.
13
+
14
+ This is the CLI ergonomics glue: the REQ's `ucode code-chart extract
15
+ --block Sidetic` flow takes a human-readable name and needs to know
16
+ the block's range to know which `U+XXXX` codepoints to iterate.
17
+
18
+ ## Files
19
+
20
+ - `lib/ucode/parsers/blocks.rb` — add `Blocks.find_by_name(path, name)`.
21
+ - `spec/ucode/parsers/blocks_spec.rb` — cover name lookup, missing-name,
22
+ case-sensitivity.
23
+
24
+ ## Design
25
+
26
+ ### Method shape
27
+
28
+ ```ruby
29
+ # @param path [Pathname, String] path to a Blocks.txt
30
+ # @param name [String] block identifier (matches Models::Block#id)
31
+ # @return [Models::Block, nil] nil when no block matches
32
+ def find_by_name(path, name)
33
+ ```
34
+
35
+ Returns nil for "not found" — callers (CLI, Extractor) decide whether
36
+ to raise. This matches `Models::Block` consumers that already expect
37
+ nilable lookups.
38
+
39
+ ### Name matching rule
40
+
41
+ `Blocks.txt` uses `name` with whitespace collapsed to underscores
42
+ into `id`. `find_by_name` matches against `id` (the underscored
43
+ form). The REQ's example `--block Sidetic` shows that the caller
44
+ provides the underscored form already. This is consistent with the
45
+ existing `Parsers::Blocks` build logic (`name.gsub(/\s+/, "_")`).
46
+
47
+ ### Why a separate method
48
+
49
+ `each_record` streams every block — the caller doesn't want to walk
50
+ ~340 blocks for every name lookup. `find_by_name` short-circuits on
51
+ first match.
52
+
53
+ ## Acceptance
54
+
55
+ - `find_by_name(path, "Basic_Latin")` returns the Basic Latin block.
56
+ - `find_by_name(path, "Nonexistent")` returns nil.
57
+ - Streaming still works for callers that need every block.
58
+
59
+ ## Out of scope
60
+
61
+ - Fuzzy matching — exact match only. Callers validate the user's
62
+ input against `Parsers::Blocks.each_record(path).map(&:id)` to
63
+ surface "did you mean …?" suggestions if we ever want that; for
64
+ now, a clean `UnknownBlockError` at the call site is enough.
65
+ - Database-backed lookup — `Ucode::Database#block_ranges_by_name` is
66
+ a different concern (full UCD index). `find_by_name` operates on
67
+ the cached `Blocks.txt` directly because the CodeChart extractor
68
+ is meant to be runnable without a built database.
@@ -0,0 +1,82 @@
1
+ # TODO 03 — CodeChart namespace
2
+
3
+ ## Status
4
+
5
+ Pending. Depends on TODO 02 (block name resolver) so the Extractor
6
+ can consume it; depends on TODO 01 (error class) so the namespace
7
+ can declare typed errors.
8
+
9
+ ## Goal
10
+
11
+ Establish the `Ucode::CodeChart` namespace as the home for the
12
+ Code Charts per-codepoint extraction feature. The REQ calls this
13
+ namespace `Ucode::CodeChart::*`; we follow the REQ.
14
+
15
+ This is the autoload-hub file plus the autoload declaration in
16
+ `lib/ucode.rb`.
17
+
18
+ ## Files
19
+
20
+ - `lib/ucode/code_chart.rb` — new autoload hub (defines
21
+ `Ucode::CodeChart` and declares child autoloads).
22
+ - `lib/ucode.rb` — add `autoload :CodeChart, "ucode/code_chart"` in
23
+ the namespace-hubs block.
24
+
25
+ ## Design
26
+
27
+ ### Autoload hub shape
28
+
29
+ ```ruby
30
+ # lib/ucode/code_chart.rb
31
+ module Ucode
32
+ module CodeChart
33
+ autoload :Extractor, "ucode/code_chart/extractor"
34
+ autoload :Provenance, "ucode/code_chart/provenance"
35
+ autoload :Sidecar, "ucode/code_chart/sidecar"
36
+ autoload :Writer, "ucode/code_chart/writer"
37
+ end
38
+ end
39
+ ```
40
+
41
+ Per the global rule (`~/.claude/CLAUDE.md`): declare autoloads in the
42
+ immediate parent namespace's file. `Ucode::CodeChart` is the immediate
43
+ parent of `Extractor`, `Provenance`, `Sidecar`, `Writer`; this file
44
+ is the immediate parent's file.
45
+
46
+ `Ucode` is the immediate parent of `CodeChart`; the autoload
47
+ declaration `autoload :CodeChart, "ucode/code_chart"` goes in
48
+ `lib/ucode.rb`.
49
+
50
+ ### Why a new namespace (not under `Ucode::Glyphs`)
51
+
52
+ `Ucode::Glyphs::*` is the existing 4-tier sourcing pipeline
53
+ (`EmbeddedFonts`, `RealFonts`, `LastResort`, `Writer`). The REQ's
54
+ `CodeChart::*` is a feature-facing namespace that orchestrates the
55
+ glyphs pipeline for one specific use case (extracting from a per-block
56
+ PDF for the essenfont donor pipeline). Keeping the feature-facing
57
+ namespace separate from the implementation namespace:
58
+
59
+ - Lets callers say `Ucode::CodeChart.extract(block: "Sidetic")`
60
+ without first knowing about `Glyphs::EmbeddedFonts`.
61
+ - Makes it easy to swap the implementation later (different
62
+ resolution strategy, alternative PDF parser) without breaking the
63
+ public API.
64
+ - Keeps `Glyphs::` focused on tier mechanics, free of feature
65
+ ergonomics.
66
+
67
+ The REQ's namespace name is what we use.
68
+
69
+ ## Acceptance
70
+
71
+ - `lib/ucode/code_chart.rb` exists with the autoload declarations.
72
+ - `lib/ucode.rb` has the new `autoload :CodeChart, "ucode/code_chart"`
73
+ in the namespace-hubs block.
74
+ - `Ucode::CodeChart` resolves to a module without loading any of its
75
+ children.
76
+
77
+ ## Out of scope
78
+
79
+ - `Ucode::CodeChart::Command` (Thin wrapper). The CLI lives in
80
+ `lib/ucode/cli.rb` per the existing pattern; no separate
81
+ `Commands::CodeChartCommand` is introduced (single source of
82
+ truth for CLI dispatch).
@@ -0,0 +1,154 @@
1
+ # TODO 04 — CodeChart::Extractor
2
+
3
+ ## Status
4
+
5
+ Pending. Depends on TODO 01 (error class), TODO 02 (block name
6
+ resolver), TODO 03 (namespace).
7
+
8
+ ## Goal
9
+
10
+ `Ucode::CodeChart::Extractor` is the single entry point for
11
+ "extract every assigned codepoint in block X as a standalone SVG."
12
+
13
+ It orchestrates the existing 4-tier resolver (one source of truth for
14
+ "how do I get the SVG for a given codepoint") and returns a list of
15
+ extraction results — one per codepoint — that the downstream Writer
16
+ serializes to disk.
17
+
18
+ This is *not* a new extraction pipeline; it is the existing
19
+ `Ucode::Glyphs::Resolver` with per-block inputs pre-configured.
20
+
21
+ ## Files
22
+
23
+ - `lib/ucode/code_chart/extractor.rb` — `Ucode::CodeChart::Extractor`
24
+ class.
25
+ - `spec/ucode/code_chart/extractor_spec.rb` — model/value-object
26
+ specs (constructor invariants, Resolver wiring) plus an integration
27
+ test against the fixture `spec/fixtures/pdfs/basic_latin.pdf`.
28
+
29
+ ## Design
30
+
31
+ ### Class shape
32
+
33
+ ```ruby
34
+ class Ucode::CodeChart::Extractor
35
+ Result = Struct.new(:codepoint, :svg, :tier, :provenance, :base_font,
36
+ :gid, keyword_init: true)
37
+
38
+ def initialize(block:, blocks_txt:, pdf_fetcher: nil,
39
+ font_cache_dir: nil, last_resort_root: nil)
40
+ @block = block # Models::Block
41
+ @blocks_txt = blocks_txt # Pathname
42
+ @pdf_fetcher = pdf_fetcher # optional injectable
43
+ @font_cache_dir = font_cache_dir # default: data/pdf-fonts/
44
+ @last_resort_root = last_resort_root
45
+ end
46
+
47
+ # Walks every assigned codepoint in @block and returns one Result
48
+ # per codepoint. Codepoints with no glyph from any tier are
49
+ # silently skipped (no Result yielded) — the REQ's "skip
50
+ # unassigned codepoints with a warning" is satisfied by the
51
+ # Resolver returning nil for them.
52
+ #
53
+ # @return [Array<Result>]
54
+ def extract
55
+ end
56
+ end
57
+ ```
58
+
59
+ ### Wiring (single source of truth)
60
+
61
+ The Extractor does NOT implement tier selection. It builds a
62
+ `Ucode::Glyphs::Resolver` and calls `resolver.resolve(codepoint)` for
63
+ each cp. The Resolver's tier order is preserved (Pillar 1 → 2 → 3
64
+ for this feature; no Tier 1 because we're starting from the Code
65
+ Charts PDF, not a real-font source).
66
+
67
+ ```ruby
68
+ def build_resolver
69
+ pdf = fetch_pdf!
70
+ embedded_source = Glyphs::EmbeddedFonts::Source.new(
71
+ pdf: pdf, cache_dir: @font_cache_dir,
72
+ )
73
+ catalog = Glyphs::EmbeddedFonts::Catalog.new(embedded_source)
74
+ pillar1 = Glyphs::Sources::Pillar1EmbeddedTounicode.new(
75
+ renderer: Glyphs::EmbeddedFonts::Renderer.new(catalog),
76
+ )
77
+ pillar3 = Glyphs::Sources::Pillar3LastResort.new(
78
+ renderer: Glyphs::LastResort::Renderer.new(
79
+ Glyphs::LastResort::Source.new(root: @last_resort_root),
80
+ ),
81
+ )
82
+ Glyphs::Resolver.new(
83
+ sources: [pillar1, pillar3],
84
+ order: %i[pillar1 pillar3],
85
+ )
86
+ end
87
+ ```
88
+
89
+ The tier ordering is documented inline: we skip Pillar 2 because for
90
+ the CodeChart use case the catalog's ToUnicode is the dominant path
91
+ and Pillar 2 (positional correlation) is reserved for fonts where
92
+ Pillar 1 fails. If a future use case needs Pillar 2, add it without
93
+ changing this constructor — that's the OCP payoff of consuming the
94
+ Resolver.
95
+
96
+ ### Why no Tier 1
97
+
98
+ Tier 1 (real-font cmap) needs a configured `SourceConfig` mapping
99
+ block → font. The CodeChart use case is for blocks where no real
100
+ font exists (Sidetic, Egyptian Ext-B). Tier 1 wouldn't contribute
101
+ anything. The Extractor accepts a Tier 1 source in the future by
102
+ having callers pass a fully-built Resolver instead of constructing
103
+ one internally.
104
+
105
+ ### Why PDF fetch is delegated to `PdfFetcher`
106
+
107
+ `Ucode::Glyphs::PdfFetcher` is the existing seam for resolving a
108
+ block to its PDF on disk (per-block cache + monolith fallback). It
109
+ already handles `force:` and the cache directory. The Extractor
110
+ constructs a `PdfFetcher` per call (cheap — it's just a path
111
+ resolver) and reuses it across codepoints.
112
+
113
+ ### Per-codepoint loop
114
+
115
+ ```ruby
116
+ def extract
117
+ resolver = build_resolver
118
+ @block.codepoint_ids.flat_map do |cp_id|
119
+ cp = Integer(cp_id.delete_prefix("U+"), 16)
120
+ resolver_result = resolver.resolve(cp)
121
+ next nil unless resolver_result&.svg
122
+
123
+ Result.new(
124
+ codepoint: cp,
125
+ svg: resolver_result.svg,
126
+ tier: resolver_result.tier,
127
+ provenance: resolver_result.provenance,
128
+ )
129
+ end.compact
130
+ end
131
+ ```
132
+
133
+ The Resolver returns a `Sources::Result` (tier + codepoint + svg +
134
+ provenance). We adapt that to the Extractor's `Result` (with
135
+ codepoint + svg + tier + provenance), stripping the resolver-specific
136
+ shape at the boundary.
137
+
138
+ ## Acceptance
139
+
140
+ - `Extractor.new(block: ..., blocks_txt: ...)` constructs without
141
+ raising when the block and PDF are present.
142
+ - `#extract` returns one Result per codepoint that any tier
143
+ produced a glyph for.
144
+ - `#extract` skips codepoints no tier could produce (returns no
145
+ Result, not a Result-with-nil).
146
+ - Integration test: against the fixture PDF, at least one codepoint's
147
+ Result has tier `:pillar1` and provenance `"pillar-1:embedded-tounicode"`.
148
+
149
+ ## Out of scope
150
+
151
+ - Writing files — that's `CodeChart::Writer` (TODO 06).
152
+ - Provenance JSON — that's `CodeChart::Sidecar` (TODO 05).
153
+ - Tier 1 (real-font) source injection — not needed for the REQ's
154
+ blocks. Future extension point if a real-font fallback is desired.