ucode 0.3.3 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (98) hide show
  1. checksums.yaml +4 -4
  2. data/config/unicode17_universal_glyph_set.yml +1 -1
  3. data/lib/ucode/code_chart/extractor.rb +7 -3
  4. data/lib/ucode/code_chart/writer.rb +1 -1
  5. data/lib/ucode/error.rb +5 -0
  6. data/lib/ucode/glyphs/embedded_fonts/catalog.rb +20 -3
  7. data/lib/ucode/glyphs/embedded_fonts/codepoint_mapper/correlator_strategy.rb +46 -0
  8. data/lib/ucode/glyphs/embedded_fonts/codepoint_mapper/strategy.rb +54 -0
  9. data/lib/ucode/glyphs/embedded_fonts/codepoint_mapper/tounicode_strategy.rb +41 -0
  10. data/lib/ucode/glyphs/embedded_fonts/codepoint_mapper/trace_strategy.rb +64 -0
  11. data/lib/ucode/glyphs/embedded_fonts/codepoint_mapper.rb +143 -90
  12. data/lib/ucode/glyphs/embedded_fonts/mutool/draw.rb +26 -0
  13. data/lib/ucode/glyphs/embedded_fonts/mutool/info.rb +23 -0
  14. data/lib/ucode/glyphs/embedded_fonts/mutool/show.rb +36 -0
  15. data/lib/ucode/glyphs/embedded_fonts/mutool/trace.rb +26 -0
  16. data/lib/ucode/glyphs/embedded_fonts/mutool.rb +30 -0
  17. data/lib/ucode/glyphs/embedded_fonts/page_trace_cache.rb +90 -0
  18. data/lib/ucode/glyphs/embedded_fonts/pdf_indexer.rb +50 -56
  19. data/lib/ucode/glyphs/embedded_fonts/{pdf_location.rb → pdf_source.rb} +1 -1
  20. data/lib/ucode/glyphs/embedded_fonts/trace_runner.rb +12 -28
  21. data/lib/ucode/glyphs/embedded_fonts.rb +3 -1
  22. data/lib/ucode/glyphs/real_fonts/coverage_auditor.rb +0 -4
  23. data/lib/ucode/glyphs/real_fonts/font_coverage_report.rb +0 -2
  24. data/lib/ucode/glyphs/real_fonts/writer.rb +0 -2
  25. data/lib/ucode/glyphs/sources/pillar1_embedded_tounicode.rb +1 -1
  26. data/lib/ucode/version.rb +1 -1
  27. metadata +13 -74
  28. data/TODO.extract-code-chart/01-pdf-fetch-validation.md +0 -80
  29. data/TODO.extract-code-chart/02-block-name-resolver.md +0 -68
  30. data/TODO.extract-code-chart/03-codechart-namespace.md +0 -82
  31. data/TODO.extract-code-chart/04-codechart-extractor.md +0 -154
  32. data/TODO.extract-code-chart/05-provenance-and-sidecar.md +0 -147
  33. data/TODO.extract-code-chart/06-codechart-writer.md +0 -134
  34. data/TODO.extract-code-chart/07-codechart-cli.md +0 -135
  35. data/TODO.extract-code-chart/08-specs.md +0 -87
  36. data/TODO.full/00-README.md +0 -116
  37. data/TODO.full/01-panglyph-vision.md +0 -112
  38. data/TODO.full/02-panglyph-repo-bootstrap.md +0 -184
  39. data/TODO.full/03-panglyph-font-builder.md +0 -201
  40. data/TODO.full/04-panglyph-publish-pipeline.md +0 -126
  41. data/TODO.full/05-ucode-0-1-1-release.md +0 -139
  42. data/TODO.full/06-fontisan-remove-audit.md +0 -142
  43. data/TODO.full/07-fontisan-remove-ucd.md +0 -125
  44. data/TODO.full/08-archive-private-bin-build.md +0 -143
  45. data/TODO.full/09-archive-public-structure.md +0 -164
  46. data/TODO.full/10-fontist-org-woff-glyphs.md +0 -131
  47. data/TODO.full/11-fontist-org-audit-coverage.md +0 -140
  48. data/TODO.full/12-implementation-order.md +0 -216
  49. data/TODO.full/13-fontisan-font-writer-api.md +0 -189
  50. data/TODO.full/14-fontisan-table-writers.md +0 -66
  51. data/TODO.full/15-panglyph-builder-real.md +0 -82
  52. data/TODO.full/16-archive-public-sync-workflows.md +0 -167
  53. data/TODO.full/17-fontist-org-font-picker.md +0 -73
  54. data/TODO.full/18-comprehensive-spec-coverage.md +0 -64
  55. data/TODO.full/19-ucode-0-1-2-patch.md +0 -32
  56. data/TODO.full/20-fontisan-0-2-23-release.md +0 -52
  57. data/TODO.new/00-README.md +0 -96
  58. data/TODO.new/01-pillar-terminology-alignment.md +0 -69
  59. data/TODO.new/02-audit-schema-design.md +0 -255
  60. data/TODO.new/03-directory-output-spec.md +0 -203
  61. data/TODO.new/04-fontist-org-contract.md +0 -173
  62. data/TODO.new/05-baseline-unicode17-coverage-audit.md +0 -144
  63. data/TODO.new/06-audit-namespace-skeleton.md +0 -105
  64. data/TODO.new/07-audit-models-port.md +0 -132
  65. data/TODO.new/08-extractors-cheap-port.md +0 -113
  66. data/TODO.new/09-extractors-expensive-port.md +0 -99
  67. data/TODO.new/10-aggregations-ucd-rewrite.md +0 -168
  68. data/TODO.new/11-differ-and-library-auditor-port.md +0 -102
  69. data/TODO.new/12-formatters-port.md +0 -115
  70. data/TODO.new/13-directory-emitter.md +0 -147
  71. data/TODO.new/14-html-face-browser.md +0 -144
  72. data/TODO.new/15-html-library-browser.md +0 -102
  73. data/TODO.new/16-cli-audit-subcommands.md +0 -142
  74. data/TODO.new/17-fontisan-cleanup-audit.md +0 -147
  75. data/TODO.new/18-fontisan-cleanup-ucd.md +0 -156
  76. data/TODO.new/19-fontisan-docs-update.md +0 -155
  77. data/TODO.new/20-canonical-resolver-4-tier.md +0 -182
  78. data/TODO.new/21-canonical-unicode17-build.md +0 -148
  79. data/TODO.new/22-implementation-order.md +0 -176
  80. data/TODO.new/23-universal-glyph-set-source-map.md +0 -312
  81. data/TODO.new/24-universal-glyph-set-build.md +0 -189
  82. data/TODO.new/25-font-audit-against-universal-set.md +0 -195
  83. data/TODO.new/26-missing-glyph-reporter.md +0 -189
  84. data/TODO.new/27-fontist-org-consumer-integration.md +0 -200
  85. data/TODO.new/28-implementation-order-update.md +0 -187
  86. data/TODO.new/29-universal-set-curation-uc17.md +0 -312
  87. data/TODO.new/30-tier1-font-acquisition.md +0 -241
  88. data/TODO.new/31-universal-set-production-build.md +0 -205
  89. data/TODO.new/32-uc17-coverage-matrix.md +0 -165
  90. data/TODO.new/33-specialist-font-acquisition-refresh.md +0 -138
  91. data/TODO.new/34-pillar2-content-stream-correlator.md +0 -147
  92. data/TODO.new/35-universal-set-production-run.md +0 -160
  93. data/TODO.new/36-per-font-coverage-audit.md +0 -145
  94. data/TODO.new/37-coverage-highlight-reporter.md +0 -125
  95. data/TODO.new/38-fontist-org-glyph-consumer.md +0 -141
  96. data/TODO.new/39-implementation-order-update-32-38.md +0 -258
  97. data/TODO.new/40-archive-private-uses-ucode-audit.md +0 -124
  98. data/TODO.new/41-ucode-unicode-archive-bridge.md +0 -160
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pathname"
4
+ require "tempfile"
5
+
6
+ require "ucode/glyphs/embedded_fonts/mutool"
7
+
8
+ module Ucode
9
+ module Glyphs
10
+ module EmbeddedFonts
11
+ module Mutool
12
+ class Show
13
+ def initialize(runner: SystemRunner.new)
14
+ @runner = runner
15
+ end
16
+
17
+ def grep(pdf, *obj_ids)
18
+ return "" if obj_ids.empty?
19
+
20
+ @runner.run("mutool", "show", "-g", pdf.to_s,
21
+ *obj_ids.map(&:to_s))
22
+ end
23
+
24
+ def stream(pdf, obj_id)
25
+ Tempfile.create("mutool-stream") do |tmp|
26
+ tmp.close
27
+ @runner.run("mutool", "show", "-o", tmp.path, "-b",
28
+ pdf.to_s, obj_id.to_s)
29
+ File.binread(tmp.path).force_encoding("UTF-8")
30
+ end
31
+ end
32
+ end
33
+ end
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,26 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pathname"
4
+
5
+ require "ucode/glyphs/embedded_fonts/mutool"
6
+
7
+ module Ucode
8
+ module Glyphs
9
+ module EmbeddedFonts
10
+ module Mutool
11
+ class Trace
12
+ def initialize(runner: SystemRunner.new)
13
+ @runner = runner
14
+ end
15
+
16
+ def call(pdf, *pages)
17
+ return "" if pages.empty?
18
+
19
+ @runner.run("mutool", "trace", pdf.to_s,
20
+ *pages.map(&:to_s))
21
+ end
22
+ end
23
+ end
24
+ end
25
+ end
26
+ end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "open3"
4
+
5
+ require "ucode/error"
6
+
7
+ module Ucode
8
+ module Glyphs
9
+ module EmbeddedFonts
10
+ module Mutool
11
+ autoload :Info, "ucode/glyphs/embedded_fonts/mutool/info"
12
+ autoload :Show, "ucode/glyphs/embedded_fonts/mutool/show"
13
+ autoload :Draw, "ucode/glyphs/embedded_fonts/mutool/draw"
14
+ autoload :Trace, "ucode/glyphs/embedded_fonts/mutool/trace"
15
+
16
+ class SystemRunner
17
+ def run(*argv)
18
+ out, err, status = Open3.capture3(*argv)
19
+ return out + err if status.success?
20
+
21
+ raise Ucode::MutoolError.new(
22
+ "mutool failed (exit #{status.exitstatus}): #{err.strip}",
23
+ context: { argv: argv },
24
+ )
25
+ end
26
+ end
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,90 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pathname"
4
+
5
+ require "ucode/glyphs/embedded_fonts/mutool"
6
+ require "ucode/glyphs/embedded_fonts/trace_parser"
7
+
8
+ module Ucode
9
+ module Glyphs
10
+ module EmbeddedFonts
11
+ # Per-PDF trace cache. Runs `mutool trace` once per page
12
+ # (1..page_count), parses each result, and stores glyphs
13
+ # grouped by page so each TraceStrategy can correlate
14
+ # positionally without re-tracing.
15
+ #
16
+ # Replaces the O(F × P) explosion in the pre-cache TraceStrategy,
17
+ # which spawned mutool trace once per page per CID font. For
18
+ # Code Charts PDFs with F trace-needing fonts and P pages, that
19
+ # was F×P subprocess invocations; this cache makes it exactly P.
20
+ #
21
+ # Why per-page grouping: TraceCorrelator's algorithm clusters
22
+ # label glyphs by Y bucket, and Y positions are page-local.
23
+ #
24
+ # Lazy: glyphs are only fetched when a strategy first asks for
25
+ # them. PDFs where every font has /ToUnicode never trigger the
26
+ # trace at all.
27
+ class PageTraceCache
28
+ # @param pdf [Pathname, String]
29
+ # @param page_count [Integer] total pages in the PDF
30
+ # @param mutool [Mutool::Trace]
31
+ def initialize(pdf:, page_count:, mutool: Mutool::Trace.new)
32
+ @pdf = Pathname.new(pdf)
33
+ @page_count = page_count
34
+ @mutool = mutool
35
+ end
36
+
37
+ # @return [Array<Array<TraceGlyph>>] one Array per page,
38
+ # 1-indexed (index 0 is unused). Each inner array holds
39
+ # every glyph emitted by `mutool trace` on that page.
40
+ def glyphs_by_page
41
+ @glyphs_by_page ||= fetch_glyphs_by_page
42
+ end
43
+
44
+ # @param base_font [String] specimen font BaseFont name
45
+ # @yieldparam page [Integer] 1-based page number
46
+ # @yieldparam glyphs [Array<TraceGlyph>] every glyph on that
47
+ # page (all fonts — TraceCorrelator filters internally)
48
+ # @return [Boolean] true if at least one page references the
49
+ # font; false otherwise
50
+ # @return [Enumerator] if no block given
51
+ def each_page_for(base_font)
52
+ return enum_for(:each_page_for, base_font) unless block_given?
53
+
54
+ present_in_any = false
55
+ glyphs_by_page.each_with_index do |glyphs, idx|
56
+ next if idx.zero?
57
+
58
+ present = glyphs.any? { |g| g.font_name == base_font }
59
+ next unless present
60
+
61
+ present_in_any = true
62
+ yield idx, glyphs
63
+ end
64
+ present_in_any
65
+ end
66
+
67
+ # @param base_font [String]
68
+ # @return [Boolean] true if any page references this font
69
+ def references_font?(base_font)
70
+ glyphs_by_page.any? do |page_glyphs|
71
+ page_glyphs.any? { |g| g.font_name == base_font }
72
+ end
73
+ end
74
+
75
+ private
76
+
77
+ def fetch_glyphs_by_page
78
+ result = [[]] # index 0 unused (1-based pages)
79
+ return result unless @page_count.positive?
80
+
81
+ (1..@page_count).each do |page|
82
+ xml = @mutool.call(@pdf, page)
83
+ result << TraceParser.parse(xml)
84
+ end
85
+ result
86
+ end
87
+ end
88
+ end
89
+ end
90
+ end
@@ -1,23 +1,23 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "open3"
4
3
  require "pathname"
5
4
 
5
+ require "ucode/glyphs/embedded_fonts/mutool"
6
+
6
7
  module Ucode
7
8
  module Glyphs
8
9
  module EmbeddedFonts
9
- # Walks the Code Charts PDF once via `mutool info` + `mutool show`
10
- # and builds an Array of {RawFontDescriptor} — one per Type0 font
11
- # that has the required descendant CIDFont, FontDescriptor, and
12
- # FontFile2/3 + Identity CIDToGIDMap.
13
- #
14
- # Pure subprocess + parsing concern. Does NOT resolve codepoint →
15
- # GID (that's {CodepointMapper}'s job). The descriptor carries
16
- # every ref the mapper needs to do its work.
17
10
  class PdfIndexer
18
- # @param source [PdfLocation]
19
- def initialize(source:)
11
+ Type0Entry = Struct.new(:font_obj_id, :base_font, keyword_init: true)
12
+
13
+ # @param source [PdfSource]
14
+ # @param mutool_info [Mutool::Info] injectable for tests
15
+ # @param mutool_show [Mutool::Show] injectable for tests
16
+ def initialize(source:, mutool_info: Mutool::Info.new,
17
+ mutool_show: Mutool::Show.new)
20
18
  @source = source
19
+ @mutool_info = mutool_info
20
+ @mutool_show = mutool_show
21
21
  end
22
22
 
23
23
  # @return [Array<RawFontDescriptor>]
@@ -68,11 +68,45 @@ module Ucode
68
68
  # @param base_font [String] e.g. "GPJAHB+WolofGaraySansSerif"
69
69
  # @return [Boolean] true if this font appears on any page
70
70
  def font_appears?(base_font)
71
- font_entries_cache.key?(base_font)
71
+ type0_base_fonts.include?(base_font)
72
72
  end
73
73
 
74
74
  private
75
75
 
76
+ # Single source of truth for the Type0 font list. Both
77
+ # `discover_type0_fonts` (keyed by obj_id) and
78
+ # `font_appears?` (membership check by name) derive their
79
+ # answer from this parse, so changes to `mutool info`'s output
80
+ # format break in exactly one place.
81
+ #
82
+ # @return [Array<Type0Entry>]
83
+ def type0_entries
84
+ @type0_entries ||= parse_type0_entries
85
+ end
86
+
87
+ # @return [Set<String>] every Type0 BaseFont name seen
88
+ def type0_base_fonts
89
+ @type0_base_fonts ||= type0_entries.each_with_object(Set.new) do |e, s|
90
+ s << e.base_font
91
+ end
92
+ end
93
+
94
+ def parse_type0_entries
95
+ seen = Set.new
96
+ mutool_info_text.each_line.filter_map do |line|
97
+ next unless line.include?("Type0")
98
+
99
+ m = line.match(/Type0\s+'([^']+)'\s+\S+\s+\((\d+)\s+0\s+R\)/)
100
+ next unless m
101
+
102
+ font_obj_id = m[2].to_i
103
+ next if seen.include?(font_obj_id)
104
+
105
+ seen << font_obj_id
106
+ Type0Entry.new(font_obj_id: font_obj_id, base_font: m[1])
107
+ end
108
+ end
109
+
76
110
  def build_descriptors(type0_refs, type0_dicts, descendant_dicts, fontdesc_dicts)
77
111
  type0_refs.filter_map do |font_obj_id, base_font|
78
112
  build_descriptor(
@@ -133,35 +167,15 @@ module Ucode
133
167
  # ---- mutool subprocess + dict parsing ----------------------------
134
168
 
135
169
  def discover_type0_fonts
136
- text = mutool_info_text
137
- result = {}
138
- seen = Set.new
139
- text.each_line do |line|
140
- next unless line.include?("Type0")
141
-
142
- m = line.match(/Type0\s+'([^']+)'\s+\S+\s+\((\d+)\s+0\s+R\)/)
143
- next unless m
144
-
145
- font_obj_id = m[2].to_i
146
- next if seen.include?(font_obj_id)
147
-
148
- seen << font_obj_id
149
- result[font_obj_id] = m[1]
170
+ type0_entries.to_h do |e|
171
+ [e.font_obj_id, e.base_font]
150
172
  end
151
- result
152
173
  end
153
174
 
154
175
  def fetch_objects(obj_ids)
155
176
  return {} if obj_ids.empty?
156
177
 
157
- args = ["mutool", "show", "-g",
158
- @source.pdf_to_s].concat(obj_ids.map(&:to_s))
159
- out, err, status = Open3.capture3(*args)
160
- unless status.success?
161
- raise Ucode::EmbeddedFontsMissingError,
162
- "mutool show failed: #{err.strip}"
163
- end
164
-
178
+ out = @mutool_show.grep(@source.pdf_to_s, *obj_ids)
165
179
  parse_grep_output(out)
166
180
  end
167
181
 
@@ -208,27 +222,7 @@ module Ucode
208
222
  end
209
223
 
210
224
  def mutool_info_text
211
- @mutool_info_text ||= run_mutool_info
212
- end
213
-
214
- def run_mutool_info
215
- out, err, status = Open3.capture3("mutool", "info", @source.pdf_to_s)
216
- status.success? ? out + err : ""
217
- end
218
-
219
- def font_entries_cache
220
- @font_entries_cache ||= begin
221
- result = {}
222
- mutool_info_text.each_line do |line|
223
- next unless line.include?("Type0")
224
-
225
- font_match = line.match(/Type0\s+'([^']+)'/)
226
- next unless font_match
227
-
228
- result[font_match[1]] = true
229
- end
230
- result
231
- end
225
+ @mutool_info_text ||= @mutool_info.call(@source.pdf_to_s)
232
226
  end
233
227
  end
234
228
  end
@@ -27,7 +27,7 @@ module Ucode
27
27
  # the BaseFont (e.g. `CIAIIP+Uni2000Generalpunctuation.ttf`).
28
28
  # Re-runs skip extraction when the cached file is newer than the
29
29
  # PDF.
30
- class PdfLocation
30
+ class PdfSource
31
31
  attr_reader :pdf_path, :cache_dir
32
32
 
33
33
  # @param pdf [String, Pathname, nil] path to a Code Charts PDF
@@ -1,51 +1,35 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "open3"
4
3
  require "pathname"
5
4
 
6
- require "ucode/error"
5
+ require "ucode/glyphs/embedded_fonts/mutool"
6
+ require "ucode/glyphs/embedded_fonts/trace_parser"
7
7
 
8
8
  module Ucode
9
9
  module Glyphs
10
10
  module EmbeddedFonts
11
11
  # Thin I/O wrapper around `mutool trace <pdf> <page>`.
12
12
  #
13
- # Runs mutool on the given pages, captures the XML output,
14
- # delegates parsing to {TraceParser}, and returns a flat
15
- # `Array<TraceGlyph>` across all pages.
16
- #
17
- # The only class in the trace pipeline that touches the
18
- # filesystem / spawns subprocesses. Everything upstream
19
- # (parser, correlator) is pure.
13
+ # Delegates the actual subprocess to {Mutool::Trace} and the
14
+ # XML parsing to {TraceParser}. Kept for backward compatibility
15
+ # with callers that pre-date the Mutool seam.
20
16
  class TraceRunner
21
17
  # @param pdf_path [Pathname, String]
22
- def initialize(pdf_path)
18
+ # @param mutool [Mutool::Trace] injectable for tests
19
+ def initialize(pdf_path, mutool: Mutool::Trace.new)
23
20
  @pdf_path = Pathname.new(pdf_path)
21
+ @mutool = mutool
24
22
  end
25
23
 
26
24
  # @param page_numbers [Array<Integer>] 1-based PDF page numbers
27
25
  # @return [Array<TraceGlyph>]
28
26
  def trace(page_numbers)
29
- page_numbers.flat_map { |page| trace_page(page) }
30
- end
31
-
32
- private
27
+ return [] if page_numbers.empty?
33
28
 
34
- def trace_page(page)
35
- xml = run_mutool(page)
36
- TraceParser.parse(xml)
37
- end
38
-
39
- def run_mutool(page)
40
- out, err, status = Open3.capture3(
41
- "mutool", "trace", @pdf_path.to_s, page.to_s,
42
- )
43
- unless status.success?
44
- raise Ucode::EmbeddedFontsMissingError,
45
- "mutool trace failed: #{(out + err).strip}"
29
+ page_numbers.flat_map do |page|
30
+ xml = @mutool.call(@pdf_path, page)
31
+ TraceParser.parse(xml)
46
32
  end
47
-
48
- out + err
49
33
  end
50
34
  end
51
35
  end
@@ -36,7 +36,7 @@ module Ucode
36
36
  # `mutool info` (font enumeration) and `mutool show -b -o` (raw
37
37
  # stream extraction).
38
38
  module EmbeddedFonts
39
- autoload :PdfLocation, "ucode/glyphs/embedded_fonts/pdf_location"
39
+ autoload :PdfSource, "ucode/glyphs/embedded_fonts/pdf_source"
40
40
  autoload :ToUnicode, "ucode/glyphs/embedded_fonts/tounicode"
41
41
  autoload :FontEntry, "ucode/glyphs/embedded_fonts/font_entry"
42
42
  autoload :RawFontDescriptor, "ucode/glyphs/embedded_fonts/raw_font_descriptor"
@@ -50,6 +50,8 @@ module Ucode
50
50
  autoload :TraceParser, "ucode/glyphs/embedded_fonts/trace_parser"
51
51
  autoload :TraceCorrelator, "ucode/glyphs/embedded_fonts/trace_correlator"
52
52
  autoload :TraceRunner, "ucode/glyphs/embedded_fonts/trace_runner"
53
+ autoload :Mutool, "ucode/glyphs/embedded_fonts/mutool"
54
+ autoload :PageTraceCache, "ucode/glyphs/embedded_fonts/page_trace_cache"
53
55
  autoload :Svg, "ucode/glyphs/embedded_fonts/svg"
54
56
  autoload :Renderer, "ucode/glyphs/embedded_fonts/renderer"
55
57
  autoload :Writer, "ucode/glyphs/embedded_fonts/writer"
@@ -5,10 +5,6 @@ require "time"
5
5
 
6
6
  require "fontisan"
7
7
 
8
- require_relative "block_coverage"
9
- require_relative "font_coverage_report"
10
- require_relative "unicode_17_blocks"
11
-
12
8
  module Ucode
13
9
  module Glyphs
14
10
  module RealFonts
@@ -2,8 +2,6 @@
2
2
 
3
3
  require "lutaml/model"
4
4
 
5
- require_relative "block_coverage"
6
-
7
5
  module Ucode
8
6
  module Glyphs
9
7
  module RealFonts
@@ -3,8 +3,6 @@
3
3
  require "fileutils"
4
4
  require "pathname"
5
5
 
6
- require_relative "font_coverage_report"
7
-
8
6
  module Ucode
9
7
  module Glyphs
10
8
  module RealFonts
@@ -31,7 +31,7 @@ module Ucode
31
31
  # @param renderer [EmbeddedFonts::Renderer] the renderer to
32
32
  # delegate to. Callers typically construct it with the
33
33
  # {EmbeddedFonts::Catalog} built from the resolved Code
34
- # Charts {EmbeddedFonts::PdfLocation}. To enable pillar-2
34
+ # Charts {EmbeddedFonts::PdfSource}. To enable pillar-2
35
35
  # fallback, that Catalog must be constructed with
36
36
  # +correlator_configs:+.
37
37
  def initialize(renderer:)
data/lib/ucode/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Ucode
4
- VERSION = "0.3.3"
4
+ VERSION = "0.4.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ucode
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.3
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-07-03 00:00:00.000000000 Z
11
+ date: 2026-07-08 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: base64
@@ -162,77 +162,6 @@ files:
162
162
  - Gemfile
163
163
  - README.md
164
164
  - Rakefile
165
- - TODO.extract-code-chart/01-pdf-fetch-validation.md
166
- - TODO.extract-code-chart/02-block-name-resolver.md
167
- - TODO.extract-code-chart/03-codechart-namespace.md
168
- - TODO.extract-code-chart/04-codechart-extractor.md
169
- - TODO.extract-code-chart/05-provenance-and-sidecar.md
170
- - TODO.extract-code-chart/06-codechart-writer.md
171
- - TODO.extract-code-chart/07-codechart-cli.md
172
- - TODO.extract-code-chart/08-specs.md
173
- - TODO.full/00-README.md
174
- - TODO.full/01-panglyph-vision.md
175
- - TODO.full/02-panglyph-repo-bootstrap.md
176
- - TODO.full/03-panglyph-font-builder.md
177
- - TODO.full/04-panglyph-publish-pipeline.md
178
- - TODO.full/05-ucode-0-1-1-release.md
179
- - TODO.full/06-fontisan-remove-audit.md
180
- - TODO.full/07-fontisan-remove-ucd.md
181
- - TODO.full/08-archive-private-bin-build.md
182
- - TODO.full/09-archive-public-structure.md
183
- - TODO.full/10-fontist-org-woff-glyphs.md
184
- - TODO.full/11-fontist-org-audit-coverage.md
185
- - TODO.full/12-implementation-order.md
186
- - TODO.full/13-fontisan-font-writer-api.md
187
- - TODO.full/14-fontisan-table-writers.md
188
- - TODO.full/15-panglyph-builder-real.md
189
- - TODO.full/16-archive-public-sync-workflows.md
190
- - TODO.full/17-fontist-org-font-picker.md
191
- - TODO.full/18-comprehensive-spec-coverage.md
192
- - TODO.full/19-ucode-0-1-2-patch.md
193
- - TODO.full/20-fontisan-0-2-23-release.md
194
- - TODO.new/00-README.md
195
- - TODO.new/01-pillar-terminology-alignment.md
196
- - TODO.new/02-audit-schema-design.md
197
- - TODO.new/03-directory-output-spec.md
198
- - TODO.new/04-fontist-org-contract.md
199
- - TODO.new/05-baseline-unicode17-coverage-audit.md
200
- - TODO.new/06-audit-namespace-skeleton.md
201
- - TODO.new/07-audit-models-port.md
202
- - TODO.new/08-extractors-cheap-port.md
203
- - TODO.new/09-extractors-expensive-port.md
204
- - TODO.new/10-aggregations-ucd-rewrite.md
205
- - TODO.new/11-differ-and-library-auditor-port.md
206
- - TODO.new/12-formatters-port.md
207
- - TODO.new/13-directory-emitter.md
208
- - TODO.new/14-html-face-browser.md
209
- - TODO.new/15-html-library-browser.md
210
- - TODO.new/16-cli-audit-subcommands.md
211
- - TODO.new/17-fontisan-cleanup-audit.md
212
- - TODO.new/18-fontisan-cleanup-ucd.md
213
- - TODO.new/19-fontisan-docs-update.md
214
- - TODO.new/20-canonical-resolver-4-tier.md
215
- - TODO.new/21-canonical-unicode17-build.md
216
- - TODO.new/22-implementation-order.md
217
- - TODO.new/23-universal-glyph-set-source-map.md
218
- - TODO.new/24-universal-glyph-set-build.md
219
- - TODO.new/25-font-audit-against-universal-set.md
220
- - TODO.new/26-missing-glyph-reporter.md
221
- - TODO.new/27-fontist-org-consumer-integration.md
222
- - TODO.new/28-implementation-order-update.md
223
- - TODO.new/29-universal-set-curation-uc17.md
224
- - TODO.new/30-tier1-font-acquisition.md
225
- - TODO.new/31-universal-set-production-build.md
226
- - TODO.new/32-uc17-coverage-matrix.md
227
- - TODO.new/33-specialist-font-acquisition-refresh.md
228
- - TODO.new/34-pillar2-content-stream-correlator.md
229
- - TODO.new/35-universal-set-production-run.md
230
- - TODO.new/36-per-font-coverage-audit.md
231
- - TODO.new/37-coverage-highlight-reporter.md
232
- - TODO.new/38-fontist-org-glyph-consumer.md
233
- - TODO.new/39-implementation-order-update-32-38.md
234
- - TODO.new/40-archive-private-uses-ucode-audit.md
235
- - TODO.new/41-ucode-unicode-archive-bridge.md
236
165
  - UCODE_CHANGELOG.md
237
166
  - config/specialist_fonts.yml
238
167
  - config/unicode17_tier1_fonts.yml
@@ -363,10 +292,20 @@ files:
363
292
  - lib/ucode/glyphs/embedded_fonts.rb
364
293
  - lib/ucode/glyphs/embedded_fonts/catalog.rb
365
294
  - lib/ucode/glyphs/embedded_fonts/codepoint_mapper.rb
295
+ - lib/ucode/glyphs/embedded_fonts/codepoint_mapper/correlator_strategy.rb
296
+ - lib/ucode/glyphs/embedded_fonts/codepoint_mapper/strategy.rb
297
+ - lib/ucode/glyphs/embedded_fonts/codepoint_mapper/tounicode_strategy.rb
298
+ - lib/ucode/glyphs/embedded_fonts/codepoint_mapper/trace_strategy.rb
366
299
  - lib/ucode/glyphs/embedded_fonts/content_stream_correlator.rb
367
300
  - lib/ucode/glyphs/embedded_fonts/font_entry.rb
301
+ - lib/ucode/glyphs/embedded_fonts/mutool.rb
302
+ - lib/ucode/glyphs/embedded_fonts/mutool/draw.rb
303
+ - lib/ucode/glyphs/embedded_fonts/mutool/info.rb
304
+ - lib/ucode/glyphs/embedded_fonts/mutool/show.rb
305
+ - lib/ucode/glyphs/embedded_fonts/mutool/trace.rb
306
+ - lib/ucode/glyphs/embedded_fonts/page_trace_cache.rb
368
307
  - lib/ucode/glyphs/embedded_fonts/pdf_indexer.rb
369
- - lib/ucode/glyphs/embedded_fonts/pdf_location.rb
308
+ - lib/ucode/glyphs/embedded_fonts/pdf_source.rb
370
309
  - lib/ucode/glyphs/embedded_fonts/positional_matcher.rb
371
310
  - lib/ucode/glyphs/embedded_fonts/raw_font_descriptor.rb
372
311
  - lib/ucode/glyphs/embedded_fonts/renderer.rb
@@ -1,80 +0,0 @@
1
- # TODO 01 — PDF fetch validation
2
-
3
- ## Status
4
-
5
- Pending.
6
-
7
- ## Goal
8
-
9
- Raise a typed `Ucode::CodeChartNotFoundError` when a Unicode Code
10
- Charts PDF cannot be downloaded or fails content validation. The REQ
11
- (R1) requires:
12
-
13
- - HTTP 4xx / 5xx → `CodeChartNotFoundError`
14
- - `Content-Type: application/pdf`
15
- - First 4 bytes are `%PDF`
16
-
17
- ## Files
18
-
19
- - `lib/ucode/error.rb` — add `Ucode::CodeChartNotFoundError` under the
20
- `GlyphError` subtree.
21
- - `lib/ucode.rb` — add an `autoload` for the new class so any rescue
22
- clause triggers one load of `error.rb`.
23
- - `lib/ucode/fetch/http.rb` — extend `Http.get` with an optional
24
- `validate:` keyword. When `validate: :pdf`, after a successful
25
- download, verify the `Content-Type` response header starts with
26
- `application/pdf` and the first 4 bytes of the body are `%PDF`.
27
- - `lib/ucode/fetch/code_charts.rb` — pass `validate: :pdf` to `Http.get`
28
- for every chart PDF download.
29
- - `spec/ucode/fetch/code_charts_spec.rb` (new) — cover the happy path
30
- and the validation failure modes.
31
-
32
- ## Design
33
-
34
- ### Why a new error class
35
-
36
- `FetchError` already covers transport failures, but it doesn't carry
37
- "this URL produced an HTML error page / 404 / non-PDF body" semantics.
38
- Splitting the type keeps existing `rescue Ucode::FetchError` callers
39
- from accidentally swallowing the typed signal that "we expected a PDF
40
- and didn't get one" — which is a different problem class from "the
41
- network was down."
42
-
43
- `CodeChartNotFoundError < Ucode::Error` (under `GlyphError`) reflects
44
- the REQ's framing: the chart for the requested block is not
45
- obtainable.
46
-
47
- ### Why `validate:` is optional on `Http.get`
48
-
49
- `Http` is the single network boundary (per the comment at the top of
50
- `http.rb`). All callers funnel through it. Adding an optional
51
- keyword keeps the MECE pattern intact: non-PDF callers (UCD zip,
52
- Unihan zip, font zip) pass nothing; the single PDF caller passes
53
- `validate: :pdf`. No second network boundary is needed.
54
-
55
- ### Why no separate "magic bytes" check class
56
-
57
- Magic-byte verification is 4 lines of code; extracting it into a
58
- class would be ceremony. Inline check after `write_body`, raising
59
- the typed error with the offending content-type or magic bytes in
60
- the context payload.
61
-
62
- ## Acceptance
63
-
64
- - `Http.get(url, dest:, validate: :pdf)` raises
65
- `CodeChartNotFoundError` (a) when the response Content-Type is not
66
- `application/pdf`, (b) when the first 4 bytes are not `%PDF`.
67
- - `Fetch::CodeCharts.call(version, block_first_cps: [...])` raises
68
- `CodeChartNotFoundError` when the unicode.org endpoint returns
69
- 4xx/5xx or non-PDF content.
70
- - Existing callers of `Http.get` that don't pass `validate:` are
71
- unchanged.
72
- - Spec coverage: happy path, HTTP 404, wrong content-type, truncated
73
- body missing the `%PDF` magic.
74
-
75
- ## Out of scope
76
-
77
- - SHA-256 verification of the PDF — that's a downstream concern (the
78
- Code Charts are not versioned by hash on unicode.org).
79
- - Resumable / partial downloads — the existing `Http` writes a
80
- `.part` then renames; that's sufficient.