ucode 0.3.3 → 0.4.0

data/TODO.full/16-archive-public-sync-workflows.md

@@ -1,167 +0,0 @@
-# 16 — archive-public: three sync workflows (GHA)
-
-## Goal
-
-Add the three GitHub Actions workflows that sync `fontist-archive-public`
-from its three upstream sources:
-
-1. `sync-private.yml` — pulls from `fontist-archive-private` on push
-2. `sync-ucode.yml` — pulls from `fontist/ucode` publish workflow
-3. `sync-panglyph.yml` — pulls from `fontist/panglyph` tag pushes
-
-Each writes to a disjoint directory (`coverage/`, `unicode/`, `panglyph/`),
-so concurrent runs don't conflict.
-
-## Why this is a separate TODO
-
-The directory placeholders exist (committed in the
-`audit/add-unicode-and-panglyph-dirs` branch). The sync workflows are
-the actual machinery. Without them, the directories stay empty.
-
-## Scope
-
-### sync-private.yml
-
-```yaml
-name: Sync from fontist-archive-private
-
-on:
-  repository_dispatch:
-    types: [archive-private-updated]
-  workflow_dispatch:
-
-jobs:
-  sync:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Clone private (read-only deploy key)
-        env:
-          SSH_KEY: ${{ secrets.ARCHIVE_PRIVATE_DEPLOY_KEY }}
-        run: |
-          mkdir -p ~/.ssh && echo "$SSH_KEY" > ~/.ssh/id_ed25519 && chmod 600 ~/.ssh/id_ed25519
-          git clone --depth 1 git@github.com:fontist/fontist-archive-private.git /tmp/private
-      - name: Sync coverage/
-        run: |
-          rm -rf coverage
-          cp -r /tmp/private/coverage .
-      - name: Sync woff/ (open-license only)
-        run: |
-          rm -rf woff
-          mkdir -p woff
-          for d in /tmp/private/woff/*; do
-            base=$(basename "$d")
-            [ "$base" != "macos" ] && cp -r "$d" "woff/"
-          done
-      - name: Sync fonts.json + font-metadata.json
-        run: cp /tmp/private/fonts.json /tmp/private/font-metadata.json . || true
-      - name: Commit
-        run: |
-          git config user.email "archive-bot@fontist.org"
-          git config user.name "archive-bot"
-          git add -A
-          git diff --quiet HEAD || git commit -m "Sync from fontist-archive-private @ $(date -u +%Y-%m-%dT%H:%M:%SZ)"
-          git push
-```
-
-### sync-ucode.yml
-
-```yaml
-name: Sync from fontist/ucode
-
-on:
-  workflow_run:
-    workflows: [publish-unicode-archive]
-    types: [completed]
-    branches: [main]
-  workflow_dispatch:
-
-jobs:
-  sync:
-    runs-on: ubuntu-latest
-    if: github.event.workflow_run.conclusion == 'success'
-    steps:
-      - uses: actions/checkout@v4
-      - name: Download ucode artifacts
-        uses: actions/github-script@v7
-        with:
-          script: |
-            const runId = context.payload.workflow_run.id
-            const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
-              owner: 'fontist', repo: 'ucode', run_id: runId
-            })
-            // download unicode-block-feed.tar.gz + universal-glyph-set.tar.gz
-            // ...
-      - name: Sync unicode/
-        run: |
-          rm -rf unicode/block-feed unicode/universal-glyph-set
-          mkdir -p unicode/block-feed unicode/universal-glyph-set
-          tar xzf /tmp/block-feed.tar.gz -C unicode/block-feed
-          tar xzf /tmp/universal-glyph-set.tar.gz -C unicode/universal-glyph-set
-      - name: Commit
-        run: |
-          git config user.email "archive-bot@fontist.org"
-          git config user.name "archive-bot"
-          git add -A unicode/
-          git diff --quiet HEAD -- unicode/ || git commit -m "Sync unicode/ from fontist/ucode"
-          git push
-```
-
-### sync-panglyph.yml
-
-```yaml
-name: Sync from fontist/panglyph
-
-on:
-  repository_dispatch:
-    types: [panglyph-released]
-  workflow_dispatch:
-    inputs:
-      version:
-        description: 'panglyph version (e.g. 17.0.0)'
-        required: true
-
-jobs:
-  sync:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Download panglyph release
-        env:
-          GH_TOKEN: ${{ secrets.ARCHIVE_PUBLIC_BOT_TOKEN }}
-        run: |
-          VERSION="${{ inputs.version || github.event.client_payload.version }}"
-          gh release download "v${VERSION}" \
-            --repo fontist/panglyph \
-            --pattern "panglyph-unicode*.{ttf,woff2,otf}" \
-            --pattern "coverage-report.json" \
-            --pattern "source-manifest.json" \
-            --dir "/tmp/v${VERSION}"
-      - name: Sync panglyph/
-        run: |
-          mkdir -p "panglyph/v${VERSION}"
-          cp /tmp/v${VERSION}/* "panglyph/v${VERSION}/"
-          # Update top-level manifest.json (script in bin/update-panglyph-manifest.rb)
-          ruby bin/update-panglyph-manifest.rb "${VERSION}"
-      - name: Commit
-        run: |
-          git config user.email "archive-bot@fontist.org"
-          git config user.name "archive-bot"
-          git add -A panglyph/
-          git diff --quiet HEAD -- panglyph/ || git commit -m "Sync panglyph v${VERSION}"
-          git push
-```
-
-## Acceptance
-
-- [ ] All three workflows exist + are syntactically valid YAML
-- [ ] Each writes to a disjoint directory
-- [ ] Manual dispatch works (`workflow_dispatch`)
-- [ ] Trigger-based dispatch works (`repository_dispatch` / `workflow_run`)
-- [ ] Bot commits identify as `archive-bot@fontist.org`
-
-## References
-
-- [TODO.full/09](09-archive-public-structure.md) — target structure
-- [TODO.full/04](04-panglyph-publish-pipeline.md) — panglyph publish trigger
-- [TODO.new/41](../TODO.new/41-ucode-unicode-archive-bridge.md) — ucode publish trigger

data/TODO.full/17-fontist-org-font-picker.md

@@ -1,73 +0,0 @@
-# 17 — fontist.org: font picker on the universal Unicode browser
-
-## Goal
-
-Add a font picker to the standalone `/unicode` page so users can browse
-Unicode with a specific font active. Today the universal browser uses
-system fallback fonts; selecting a font from `public/fonts/*.woff2`
-injects `@font-face` and renders every glyph in that font.
-
-Pairs with the existing `/fonts/{slug}/unicode` page (FontUnicodePage),
-which already does this — but only for one font at a time. The universal
-browser should let users switch fonts without leaving `/unicode`.
-
-## Why this is a separate TODO
-
-TODO.full/10 covered the rendering primitives (useFontFace composable,
-FontPicker component, etc.). The integration work — wiring the picker
-into `/unicode`, persisting selection, surfacing on the block + char
-pages — is separate.
-
-## Scope
-
-### Phase A — Reuse the existing FontPicker
-
-1. The `FontStyleUnicodePage` already injects `@font-face` via
-   `injectFontFace(slug, 'fonts/${slug}.woff2', true)`. Extract this
-   into a global composable that any page can call.
-
-2. Add a `<FontPicker>` component to `/unicode` (top of page, sticky).
-
-3. On font selection: set a global Pinia store (or provide/inject)
-   so all child routes inherit the active font.
-
-### Phase B — Coverage overlay
-
-4. When a font is active on `/unicode`:
-   - Each block row shows: `[coverage bar: 128/128 = 100%]` next to the name
-   - Color-coded: green ≥95%, yellow 50-95%, red <50%
-   - Click → navigates to `/unicode/block/<slug>?font=<active-slug>`
-
-5. On `/unicode/block/<slug>`:
-   - If `?font=X` is set, inject X's WOFF + show coverage overlay on each cell
-   - Cells with chars the font covers: render in WOFF
-   - Cells with chars the font misses: gray out + tooltip "Not in X"
-
-### Phase C — Persistence
-
-6. Active font persists across page navigation via `localStorage`.
-
-7. URL parameter `?font=X` overrides localStorage. Useful for sharing
-   links like "view this block in Noto Sans".
-
-### Phase D — "Best font per block" page
-
-8. New route `/unicode/best-fonts/{block-slug}`: lists fonts sorted by
-   fill ratio for that block. Top result is the canonical Tier 1 font
-   from ucode's universal-set manifest.
-
-## Acceptance
-
-- [ ] `/unicode` shows FontPicker at top
-- [ ] Selecting a font injects `@font-face` and re-renders the page
-- [ ] Block list shows coverage bars when a font is active
-- [ ] Block detail page respects `?font=X`
-- [ ] Active font persists in localStorage
-- [ ] `/unicode/best-fonts/{block}` exists
-
-## References
-
-- [TODO.full/10](10-fontist-org-woff-glyphs.md) — rendering primitives
-- [TODO.full/11](11-fontist-org-audit-coverage.md) — coverage data layer
-- `src/pages/FontUnicodePage.vue` — existing per-font page (pattern source)
-- `src/composables/useFontFace.ts` — existing composable

data/TODO.full/18-comprehensive-spec-coverage.md

@@ -1,64 +0,0 @@
-# 18 — Comprehensive spec coverage for new code
-
-## Goal
-
-Every class added in TODO.full/13–17 must have specs covering:
-- Public API behavior
-- Edge cases (empty input, malformed input, boundary values)
-- Round-trip serialization (write → read = same data)
-- Performance (no obvious pathological cases for 299k codepoints)
-
-## Why a separate TODO
-
-Quality bar per global rules: "Good specs throughout. Every public
-method should have specs. Every behavioral edge case should be covered.
-Specs use real model instances — never doubles."
-
-The new code (FontWriter, panglyph Builder, font picker integration)
-touches core paths. Without specs, regressions slip silently.
-
-## Scope
-
-### fontisan FontWriter (TODO.full/13, 14)
-
-- `FontWriter#set_cmap` — empty, single entry, full BMP, supplementary
-- `FontWriter#add_glyph` — simple outline, composite outline, no instructions
-- `FontWriter#set_name_records` — all 6 standard name_ids
-- `FontWriter#write_to` — round-trip: write → reopen via Fontisan::Font.open
-- Per-table specs:
-  - `Tables::Head` — byte-exact comparison with a known fixture
-  - `Tables::Cmap` — format 4 BMP, format 12 supplementary, both
-  - `Tables::Glyf` — simple glyph, composite glyph, empty glyph (.notdef)
-  - etc.
-
-### panglyph (TODO.full/15)
-
-- `Builder#call` — full pipeline with a 5-codepoint fixture
-- `OutlineExtractor#extract_many` — real font, 3 codepoints
-- `FontAssembler#assemble` — output TTF opens via Fontisan::Font.open
-- `CoverageReport#validate_font` — known gap → reported in missing_codepoints
-- `Publisher#call` — atomic sync to temp git repo fixture
-
-### fontist.org (TODO.full/17)
-
-- `FontPicker.vue` — emits selection event
-- `useActiveFont` composable — persists to localStorage
-- `/unicode/best-fonts/{block}` — sorts by fill_ratio desc
-
-### Integration specs
-
-- End-to-end: ucode parse → universal-set build → panglyph build → archive sync → fontist.org fetch
-- Smoke test: built panglyph TTF renders U+0041 in a headless browser
-
-## Acceptance
-
-- [ ] Every new public class has a corresponding spec file
-- [ ] Every public method has at least one happy-path + one edge-case spec
-- [ ] Round-trip specs exist for any serialization path
-- [ ] `bundle exec rspec` passes on all repos
-- [ ] No `double()` usage (per global rule)
-
-## References
-
-- Global rule: "Good specs throughout"
-- `spec/support/model_round_trip.rb` — shared example for round-trip models

data/TODO.full/19-ucode-0-1-2-patch.md

@@ -1,32 +0,0 @@
-# 19 — ucode 0.1.2 patch: include FontWriter + universal-set fixes
-
-## Goal
-
-After TODO.full/13 (FontWriter) lands in fontisan, bump ucode to 0.1.2
-with whatever fixes accumulated. Keep the gem install path stable for
-fontist-archive-private CI.
-
-## Why a separate TODO
-
-ucode 0.1.1 (TODO.full/05) ships the audit subsystem + BlockFeedEmitter.
-0.1.2 is a routine patch — no SemVer drama, just iteration. Bundles
-whatever real-data fixes surface when CI actually runs against all
-formulas.
-
-## Scope
-
-- Bump `lib/ucode/version.rb` to `0.1.2`
-- Add CHANGELOG entry (typically: 1-2 fixes)
-- Run full test suite + rubocop
-- Open PR + tag + release per standard release process
-
-## Acceptance
-
-- [ ] Version bumped
-- [ ] CHANGELOG documents 0.1.2
-- [ ] Specs green
-- [ ] Tag v0.1.2 + `rake release` (with explicit user authorization)
-
-## References
-
-- [TODO.full/05](05-ucode-0-1-1-release.md) — prior release process

data/TODO.full/20-fontisan-0-2-23-release.md

@@ -1,52 +0,0 @@
-# 20 — fontisan 0.2.23: ship FontWriter
-
-## Goal
-
-Once TODO.full/13 (FontWriter API) + TODO.full/14 (table writers) are
-merged to main, ship fontisan 0.2.23 with the new capability. Stays
-on 0.2.x because the removal of audit + UCD was framed as dead-code
-cleanup, not a SemVer event.
-
-## Why this is a separate TODO
-
-panglyph's gemspec depends on `fontisan, "~> 0.3"` today (the bootstrap
-commit assumed the 0.3.0 bump). After your correction (no bump), the
-constraint needs to relax to `~> 0.2` to pick up FontWriter in 0.2.23.
-
-## Scope
-
-1. Verify TODO.full/13 + 14 are merged to main.
-2. Bump `lib/fontisan/version.rb` to `0.2.23`.
-3. Add CHANGELOG entry:
-   ```markdown
-   ## [0.2.23] — 2026-XX-XX
-
-   ### Added
-
-   - `Fontisan::FontWriter` — new API for assembling fonts from scratch
-     (cmap + glyf + name + metrics). Pairs with the existing font-reading
-     API. Enables downstream consumers (panglyph) to build new fonts
-     from extracted outlines.
-   - `Fontisan::FontWriter::Tables::*` — one writer class per OpenType
-     table (Head, Cmap, Hmtx, Hhea, Glyf, Loca, Maxp, Os2, Post, Name).
-   - Typed structs: `Outline`, `Point`, `NameRecord`, `Metrics`,
-     `GlyphEntry`, `FontModel`.
-   ```
-4. Update panglyph gemspec: `gem "fontisan", "~> 0.2"` (was `~> 0.3`).
-5. Run full test suite + rubocop.
-6. Open PR + tag + release.
-
-## Acceptance
-
-- [ ] Version bumped to 0.2.23
-- [ ] CHANGELOG entry documents FontWriter addition
-- [ ] panglyph.gemspec constraint relaxed to `~> 0.2`
-- [ ] `bundle exec panglyph version` works against the new fontisan
-- [ ] Tag v0.2.23 + `rake release` (with explicit user authorization)
-
-## References
-
-- [TODO.full/06](06-fontisan-remove-audit.md) — why we stayed on 0.2.x
-- [TODO.full/07](07-fontisan-remove-ucd.md) — same reasoning
-- [TODO.full/13](13-fontisan-font-writer-api.md) — what 0.2.23 ships
-- [TODO.full/14](14-fontisan-table-writers.md) — companion

data/TODO.new/00-README.md

@@ -1,96 +0,0 @@
-# TODO.new — audit migration + Mode 2 work
-
-Work tracks for the fontisan audit → ucode audit migration, the
-per-font-audit output spec, and the Mode 1 canonical-dataset alignment.
-The full architecture reference is `docs/architecture.md` — read that
-first; these TODOs reference sections of it.
-
-## Tracks
-
-### Alignment & contract (lock these before any code moves)
-
-- [01 — Pillar terminology alignment](01-pillar-terminology-alignment.md)
-- [02 — Audit schema design](02-audit-schema-design.md)
-- [03 — Directory output spec](03-directory-output-spec.md)
-- [04 — fontist.org contract](04-fontist-org-contract.md)
-
-### Baseline measurement (know where we are)
-
-- [05 — Unicode 17 baseline coverage audit](05-baseline-unicode17-coverage-audit.md)
-
-### Audit migration (the big work)
-
-- [06 — Audit namespace skeleton](06-audit-namespace-skeleton.md)
-- [07 — Models::Audit port](07-audit-models-port.md)
-- [08 — Cheap extractors port](08-extractors-cheap-port.md)
-- [09 — Expensive extractors port](09-extractors-expensive-port.md)
-- [10 — Aggregations rewrite on ucode UCD](10-aggregations-ucd-rewrite.md)
-- [11 — Differ + library auditor port](11-differ-and-library-auditor-port.md)
-- [12 — Formatters port](12-formatters-port.md)
-
-### Output + browser
-
-- [13 — Directory emitter](13-directory-emitter.md)
-- [14 — HTML face browser](14-html-face-browser.md)
-- [15 — HTML library browser](15-html-library-browser.md)
-- [16 — CLI audit subcommands](16-cli-audit-subcommands.md)
-
-### Fontisan cleanup (after ucode audit ships)
-
-- [17 — Fontisan: delete audit subsystem](17-fontisan-cleanup-audit.md)
-- [18 — Fontisan: delete UCD subsystem](18-fontisan-cleanup-ucd.md)
-- [19 — Fontisan: docs and shim update](19-fontisan-docs-update.md)
-
-### Canonical Mode 1 alignment
-
-- [20 — Canonical 4-tier resolver](20-canonical-resolver-4-tier.md)
-- [21 — Canonical Unicode 17 dataset build](21-canonical-unicode17-build.md)
-
-### Universal glyph set + UC17 curation
-
-- [23 — Universal glyph set: Tier 1 source map](23-universal-glyph-set-source-map.md)
-- [24 — Universal glyph set build](24-universal-glyph-set-build.md)
-- [25 — Font audit against universal set](25-font-audit-against-universal-set.md)
-- [26 — Missing glyph reporter (drill-down view)](26-missing-glyph-reporter.md)
-- [27 — fontist.org consumer integration](27-fontist-org-consumer-integration.md)
-- [29 — Universal glyph set: full Unicode 17 curation (Part 1)](29-universal-set-curation-uc17.md)
-- [30 — Tier 1 font acquisition: specialist fonts](30-tier1-font-acquisition.md)
-- [31 — Universal set production build + coverage validation](31-universal-set-production-build.md)
-
-### Full UC17 coverage + per-font audit (Part 1 close + Part 2)
-
-- [32 — Universal glyph set: full UC17 coverage matrix (Part 1 master)](32-uc17-coverage-matrix.md)
-- [33 — Specialist font acquisition refresh](33-specialist-font-acquisition-refresh.md)
-- [34 — Pillar 2 ContentStreamCorrelator (generalize correlate-v4)](34-pillar2-content-stream-correlator.md)
-- [35 — Universal set production run + glyph provenance (Part 1 close)](35-universal-set-production-run.md)
-- [36 — Per-font coverage audit against universal set (Part 2 master)](36-per-font-coverage-audit.md)
-- [37 — Coverage highlight reporter (missing-glyph visualizer)](37-coverage-highlight-reporter.md)
-- [38 — fontist.org glyph consumer + provenance display](38-fontist-org-glyph-consumer.md)
-
-### Pipeline wiring (archive integration)
-
-- [40 — fontist-archive-private bin/build uses ucode audit](40-archive-private-uses-ucode-audit.md)
-- [41 — ucode Unicode artifacts → fontist-archive-public bridge](41-ucode-unicode-archive-bridge.md)
-- [30 — Tier 1 font acquisition: specialist fonts](30-tier1-font-acquisition.md)
-- [31 — Universal set production build + coverage validation](31-universal-set-production-build.md)
-
-### Sequencing
-
-- [22 — Implementation order](22-implementation-order.md)
-- [28 — Implementation order update (TODOs 23-31)](28-implementation-order-update.md)
-- [39 — Implementation order update (TODOs 32-38)](39-implementation-order-update-32-38.md)
-
-## Conventions
-
-- One concern per file. If a TODO grows past ~250 lines it should split.
-- File numbering is stable; reuse the next free number for additions.
-- Every TODO lists: Goal, Files, Scope, Acceptance, References.
-- Specs use real model instances — never `double()` (global rule).
-- All new lib files use Ruby `autoload` (declared in the immediate
-  parent namespace's file) for same-library code. No `require_relative`
-  and no `require "ucode/..."` inside the library.
-- No AI attribution in any commit, doc, or comment.
-- Branch naming: `audit/<track-slug>` (e.g. `audit/schema-design`).
-  One PR per track unless tracks are tightly coupled.
-- Land PR #1 (`tier1-cmap-audit`) before starting any track in this dir.
-  The migration builds on top of the merged RealFonts subsystem.

data/TODO.new/01-pillar-terminology-alignment.md

@@ -1,69 +0,0 @@
-# 01 — Pillar terminology alignment
-
-## Goal
-
-Fix the inconsistency between the README's "two pillars" claim and the
-actual 4-tier glyph sourcing strategy. The recent commit `24e6bfd`
-("Pillar-2 content-stream correlation fallback") was named correctly;
-the README at `docs/architecture.md`'s "4-tier strategy" section is
-authoritative.
-
-## Problem
-
-The README currently says (line ~155):
-
-> ### The v0.2 plan — two pillars
-> 1. Real character glyphs — extract the subsetted fonts from the PDF.
-> 2. Last Resort placeholders — render directly from the UFO source.
-
-This collapses Tier 1 (real-font cmap) and the three PDF-side pillars
-into "two pillars". The actual strategy (per project memory and the
-in-tree code) is four-tier:
-
-1. **Tier 1** — real-font cmap (`Ucode::Glyphs::RealFonts`).
-2. **Pillar 1** — PDF-embedded font with `/ToUnicode` (`EmbeddedFonts::Catalog`).
-3. **Pillar 2** — PDF content-stream correlation (`ContentStreamCorrelator`).
-4. **Pillar 3** — Last Resort UFO (`Ucode::Glyphs::LastResort`).
-
-The mismatch confuses anyone reading the code (where each tier is
-distinct) vs the README (which merges three of them).
-
-## Files to change
-
-- `README.md` — replace the "two pillars" section with the 4-tier table.
-  Cross-link to `docs/architecture.md` §"The 4-tier glyph sourcing
-  strategy" as the canonical reference.
-- `docs/architecture.md` — already correct; no change here.
-- `CLAUDE.md` — has a brief mention of glyph sourcing; align the
-  vocabulary with the 4-tier names.
-
-## Scope
-
-In scope:
-- README rewrite (one section, ~50 lines).
-- CLAUDE.md vocabulary tweak (one paragraph).
-- No code changes.
-
-Out of scope:
-- Renaming any code symbol. The current symbols (`RealFonts`,
-  `EmbeddedFonts::Catalog`, `ContentStreamCorrelator`, `LastResort`) are
-  fine; the names match their function. Only the prose label "tier" vs
-  "pillar" needs disambiguation.
-- Updating the commit message of `24e6bfd`. The commit was correctly
-  named; do not rewrite history.
-
-## Acceptance
-
-- `grep -ni "two pillars" README.md` returns no matches.
-- `grep -ni "pillar" README.md` returns matches that fit the 4-tier
-  vocabulary (Tier 1 + Pillars 1-3).
-- README's strategy section cross-links to `docs/architecture.md`.
-- No code changes; no spec changes; no changelog entry needed beyond
-  the commit message.
-
-## References
-
-- `docs/architecture.md` §"The 4-tier glyph sourcing strategy"
-- Commit `24e6bfd` (correctly named)
-- Commit `307fda3` (Tier-1 implementation)
-- Memory: `ucode_glyph_extraction_cell_border_bug.md`