ucode 0.1.0 → 0.1.1

data/TODO.full/16-archive-public-sync-workflows.md

@@ -0,0 +1,167 @@
+# 16 — archive-public: three sync workflows (GHA)
+
+## Goal
+
+Add the three GitHub Actions workflows that sync `fontist-archive-public`
+from its three upstream sources:
+
+1. `sync-private.yml` — pulls from `fontist-archive-private` on push
+2. `sync-ucode.yml` — pulls from `fontist/ucode` publish workflow
+3. `sync-panglyph.yml` — pulls from `fontist/panglyph` tag pushes
+
+Each writes to a disjoint directory (`coverage/`, `unicode/`, `panglyph/`),
+so concurrent runs don't conflict.
+
+## Why this is a separate TODO
+
+The directory placeholders exist (committed in the
+`audit/add-unicode-and-panglyph-dirs` branch). The sync workflows are
+the actual machinery. Without them, the directories stay empty.
+
+## Scope
+
+### sync-private.yml
+
+```yaml
+name: Sync from fontist-archive-private
+
+on:
+  repository_dispatch:
+    types: [archive-private-updated]
+  workflow_dispatch:
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Clone private (read-only deploy key)
+        env:
+          SSH_KEY: ${{ secrets.ARCHIVE_PRIVATE_DEPLOY_KEY }}
+        run: |
+          mkdir -p ~/.ssh && echo "$SSH_KEY" > ~/.ssh/id_ed25519 && chmod 600 ~/.ssh/id_ed25519
+          git clone --depth 1 git@github.com:fontist/fontist-archive-private.git /tmp/private
+      - name: Sync coverage/
+        run: |
+          rm -rf coverage
+          cp -r /tmp/private/coverage .
+      - name: Sync woff/ (open-license only)
+        run: |
+          rm -rf woff
+          mkdir -p woff
+          for d in /tmp/private/woff/*; do
+            base=$(basename "$d")
+            [ "$base" != "macos" ] && cp -r "$d" "woff/"
+          done
+      - name: Sync fonts.json + font-metadata.json
+        run: cp /tmp/private/fonts.json /tmp/private/font-metadata.json . || true
+      - name: Commit
+        run: |
+          git config user.email "archive-bot@fontist.org"
+          git config user.name "archive-bot"
+          git add -A
+          git diff --quiet HEAD || git commit -m "Sync from fontist-archive-private @ $(date -u +%Y-%m-%dT%H:%M:%SZ)"
+          git push
+```
+
+### sync-ucode.yml
+
+```yaml
+name: Sync from fontist/ucode
+
+on:
+  workflow_run:
+    workflows: [publish-unicode-archive]
+    types: [completed]
+    branches: [main]
+  workflow_dispatch:
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+    if: github.event.workflow_run.conclusion == 'success'
+    steps:
+      - uses: actions/checkout@v4
+      - name: Download ucode artifacts
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const runId = context.payload.workflow_run.id
+            const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
+              owner: 'fontist', repo: 'ucode', run_id: runId
+            })
+            // download unicode-block-feed.tar.gz + universal-glyph-set.tar.gz
+            // ...
+      - name: Sync unicode/
+        run: |
+          rm -rf unicode/block-feed unicode/universal-glyph-set
+          mkdir -p unicode/block-feed unicode/universal-glyph-set
+          tar xzf /tmp/block-feed.tar.gz -C unicode/block-feed
+          tar xzf /tmp/universal-glyph-set.tar.gz -C unicode/universal-glyph-set
+      - name: Commit
+        run: |
+          git config user.email "archive-bot@fontist.org"
+          git config user.name "archive-bot"
+          git add -A unicode/
+          git diff --quiet HEAD -- unicode/ || git commit -m "Sync unicode/ from fontist/ucode"
+          git push
+```
+
+### sync-panglyph.yml
+
+```yaml
+name: Sync from fontist/panglyph
+
+on:
+  repository_dispatch:
+    types: [panglyph-released]
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'panglyph version (e.g. 17.0.0)'
+        required: true
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Download panglyph release
+        env:
+          GH_TOKEN: ${{ secrets.ARCHIVE_PUBLIC_BOT_TOKEN }}
+        run: |
+          VERSION="${{ inputs.version || github.event.client_payload.version }}"
+          gh release download "v${VERSION}" \
+            --repo fontist/panglyph \
+            --pattern "panglyph-unicode*.{ttf,woff2,otf}" \
+            --pattern "coverage-report.json" \
+            --pattern "source-manifest.json" \
+            --dir "/tmp/v${VERSION}"
+      - name: Sync panglyph/
+        run: |
+          mkdir -p "panglyph/v${VERSION}"
+          cp /tmp/v${VERSION}/* "panglyph/v${VERSION}/"
+          # Update top-level manifest.json (script in bin/update-panglyph-manifest.rb)
+          ruby bin/update-panglyph-manifest.rb "${VERSION}"
+      - name: Commit
+        run: |
+          git config user.email "archive-bot@fontist.org"
+          git config user.name "archive-bot"
+          git add -A panglyph/
+          git diff --quiet HEAD -- panglyph/ || git commit -m "Sync panglyph v${VERSION}"
+          git push
+```
+
+## Acceptance
+
+- [ ] All three workflows exist + are syntactically valid YAML
+- [ ] Each writes to a disjoint directory
+- [ ] Manual dispatch works (`workflow_dispatch`)
+- [ ] Trigger-based dispatch works (`repository_dispatch` / `workflow_run`)
+- [ ] Bot commits identify as `archive-bot@fontist.org`
+
+## References
+
+- [TODO.full/09](09-archive-public-structure.md) — target structure
+- [TODO.full/04](04-panglyph-publish-pipeline.md) — panglyph publish trigger
+- [TODO.new/41](../TODO.new/41-ucode-unicode-archive-bridge.md) — ucode publish trigger

data/TODO.full/17-fontist-org-font-picker.md

@@ -0,0 +1,73 @@
+# 17 — fontist.org: font picker on the universal Unicode browser
+
+## Goal
+
+Add a font picker to the standalone `/unicode` page so users can browse
+Unicode with a specific font active. Today the universal browser uses
+system fallback fonts; selecting a font from `public/fonts/*.woff2`
+injects `@font-face` and renders every glyph in that font.
+
+Pairs with the existing `/fonts/{slug}/unicode` page (FontUnicodePage),
+which already does this — but only for one font at a time. The universal
+browser should let users switch fonts without leaving `/unicode`.
+
+## Why this is a separate TODO
+
+TODO.full/10 covered the rendering primitives (useFontFace composable,
+FontPicker component, etc.). The integration work — wiring the picker
+into `/unicode`, persisting selection, surfacing on the block + char
+pages — is separate.
+
+## Scope
+
+### Phase A — Reuse the existing FontPicker
+
+1. The `FontStyleUnicodePage` already injects `@font-face` via
+   `injectFontFace(slug, 'fonts/${slug}.woff2', true)`. Extract this
+   into a global composable that any page can call.
+
+2. Add a `<FontPicker>` component to `/unicode` (top of page, sticky).
+
+3. On font selection: set a global Pinia store (or provide/inject)
+   so all child routes inherit the active font.
+
+### Phase B — Coverage overlay
+
+4. When a font is active on `/unicode`:
+   - Each block row shows: `[coverage bar: 128/128 = 100%]` next to the name
+   - Color-coded: green ≥95%, yellow 50-95%, red <50%
+   - Click → navigates to `/unicode/block/<slug>?font=<active-slug>`
+
+5. On `/unicode/block/<slug>`:
+   - If `?font=X` is set, inject X's WOFF + show coverage overlay on each cell
+   - Cells with chars the font covers: render in WOFF
+   - Cells with chars the font misses: gray out + tooltip "Not in X"
+
+### Phase C — Persistence
+
+6. Active font persists across page navigation via `localStorage`.
+
+7. URL parameter `?font=X` overrides localStorage. Useful for sharing
+   links like "view this block in Noto Sans".
+
+### Phase D — "Best font per block" page
+
+8. New route `/unicode/best-fonts/{block-slug}`: lists fonts sorted by
+   fill ratio for that block. Top result is the canonical Tier 1 font
+   from ucode's universal-set manifest.
+
+## Acceptance
+
+- [ ] `/unicode` shows FontPicker at top
+- [ ] Selecting a font injects `@font-face` and re-renders the page
+- [ ] Block list shows coverage bars when a font is active
+- [ ] Block detail page respects `?font=X`
+- [ ] Active font persists in localStorage
+- [ ] `/unicode/best-fonts/{block}` exists
+
+## References
+
+- [TODO.full/10](10-fontist-org-woff-glyphs.md) — rendering primitives
+- [TODO.full/11](11-fontist-org-audit-coverage.md) — coverage data layer
+- `src/pages/FontUnicodePage.vue` — existing per-font page (pattern source)
+- `src/composables/useFontFace.ts` — existing composable

data/TODO.full/18-comprehensive-spec-coverage.md

@@ -0,0 +1,64 @@
+# 18 — Comprehensive spec coverage for new code
+
+## Goal
+
+Every class added in TODO.full/13–17 must have specs covering:
+- Public API behavior
+- Edge cases (empty input, malformed input, boundary values)
+- Round-trip serialization (write → read = same data)
+- Performance (no obvious pathological cases for 299k codepoints)
+
+## Why a separate TODO
+
+Quality bar per global rules: "Good specs throughout. Every public
+method should have specs. Every behavioral edge case should be covered.
+Specs use real model instances — never doubles."
+
+The new code (FontWriter, panglyph Builder, font picker integration)
+touches core paths. Without specs, regressions slip silently.
+
+## Scope
+
+### fontisan FontWriter (TODO.full/13, 14)
+
+- `FontWriter#set_cmap` — empty, single entry, full BMP, supplementary
+- `FontWriter#add_glyph` — simple outline, composite outline, no instructions
+- `FontWriter#set_name_records` — all 6 standard name_ids
+- `FontWriter#write_to` — round-trip: write → reopen via Fontisan::Font.open
+- Per-table specs:
+  - `Tables::Head` — byte-exact comparison with a known fixture
+  - `Tables::Cmap` — format 4 BMP, format 12 supplementary, both
+  - `Tables::Glyf` — simple glyph, composite glyph, empty glyph (.notdef)
+  - etc.
+
+### panglyph (TODO.full/15)
+
+- `Builder#call` — full pipeline with a 5-codepoint fixture
+- `OutlineExtractor#extract_many` — real font, 3 codepoints
+- `FontAssembler#assemble` — output TTF opens via Fontisan::Font.open
+- `CoverageReport#validate_font` — known gap → reported in missing_codepoints
+- `Publisher#call` — atomic sync to temp git repo fixture
+
+### fontist.org (TODO.full/17)
+
+- `FontPicker.vue` — emits selection event
+- `useActiveFont` composable — persists to localStorage
+- `/unicode/best-fonts/{block}` — sorts by fill_ratio desc
+
+### Integration specs
+
+- End-to-end: ucode parse → universal-set build → panglyph build → archive sync → fontist.org fetch
+- Smoke test: built panglyph TTF renders U+0041 in a headless browser
+
+## Acceptance
+
+- [ ] Every new public class has a corresponding spec file
+- [ ] Every public method has at least one happy-path + one edge-case spec
+- [ ] Round-trip specs exist for any serialization path
+- [ ] `bundle exec rspec` passes on all repos
+- [ ] No `double()` usage (per global rule)
+
+## References
+
+- Global rule: "Good specs throughout"
+- `spec/support/model_round_trip.rb` — shared example for round-trip models

data/TODO.full/19-ucode-0-1-2-patch.md

@@ -0,0 +1,32 @@
+# 19 — ucode 0.1.2 patch: include FontWriter + universal-set fixes
+
+## Goal
+
+After TODO.full/13 (FontWriter) lands in fontisan, bump ucode to 0.1.2
+with whatever fixes accumulated. Keep the gem install path stable for
+fontist-archive-private CI.
+
+## Why a separate TODO
+
+ucode 0.1.1 (TODO.full/05) ships the audit subsystem + BlockFeedEmitter.
+0.1.2 is a routine patch — no SemVer drama, just iteration. Bundles
+whatever real-data fixes surface when CI actually runs against all
+formulas.
+
+## Scope
+
+- Bump `lib/ucode/version.rb` to `0.1.2`
+- Add CHANGELOG entry (typically: 1-2 fixes)
+- Run full test suite + rubocop
+- Open PR + tag + release per standard release process
+
+## Acceptance
+
+- [ ] Version bumped
+- [ ] CHANGELOG documents 0.1.2
+- [ ] Specs green
+- [ ] Tag v0.1.2 + `rake release` (with explicit user authorization)
+
+## References
+
+- [TODO.full/05](05-ucode-0-1-1-release.md) — prior release process

data/TODO.full/20-fontisan-0-2-23-release.md

@@ -0,0 +1,52 @@
+# 20 — fontisan 0.2.23: ship FontWriter
+
+## Goal
+
+Once TODO.full/13 (FontWriter API) + TODO.full/14 (table writers) are
+merged to main, ship fontisan 0.2.23 with the new capability. Stays
+on 0.2.x because the removal of audit + UCD was framed as dead-code
+cleanup, not a SemVer event.
+
+## Why this is a separate TODO
+
+panglyph's gemspec depends on `fontisan, "~> 0.3"` today (the bootstrap
+commit assumed the 0.3.0 bump). After your correction (no bump), the
+constraint needs to relax to `~> 0.2` to pick up FontWriter in 0.2.23.
+
+## Scope
+
+1. Verify TODO.full/13 + 14 are merged to main.
+2. Bump `lib/fontisan/version.rb` to `0.2.23`.
+3. Add CHANGELOG entry:
+   ```markdown
+   ## [0.2.23] — 2026-XX-XX
+
+   ### Added
+
+   - `Fontisan::FontWriter` — new API for assembling fonts from scratch
+     (cmap + glyf + name + metrics). Pairs with the existing font-reading
+     API. Enables downstream consumers (panglyph) to build new fonts
+     from extracted outlines.
+   - `Fontisan::FontWriter::Tables::*` — one writer class per OpenType
+     table (Head, Cmap, Hmtx, Hhea, Glyf, Loca, Maxp, Os2, Post, Name).
+   - Typed structs: `Outline`, `Point`, `NameRecord`, `Metrics`,
+     `GlyphEntry`, `FontModel`.
+   ```
+4. Update panglyph gemspec: `gem "fontisan", "~> 0.2"` (was `~> 0.3`).
+5. Run full test suite + rubocop.
+6. Open PR + tag + release.
+
+## Acceptance
+
+- [ ] Version bumped to 0.2.23
+- [ ] CHANGELOG entry documents FontWriter addition
+- [ ] panglyph.gemspec constraint relaxed to `~> 0.2`
+- [ ] `bundle exec panglyph version` works against the new fontisan
+- [ ] Tag v0.2.23 + `rake release` (with explicit user authorization)
+
+## References
+
+- [TODO.full/06](06-fontisan-remove-audit.md) — why we stayed on 0.2.x
+- [TODO.full/07](07-fontisan-remove-ucd.md) — same reasoning
+- [TODO.full/13](13-fontisan-font-writer-api.md) — what 0.2.23 ships
+- [TODO.full/14](14-fontisan-table-writers.md) — companion

data/TODO.new/00-README.md

@@ -46,9 +46,39 @@ first; these TODOs reference sections of it.
 - [20 — Canonical 4-tier resolver](20-canonical-resolver-4-tier.md)
 - [21 — Canonical Unicode 17 dataset build](21-canonical-unicode17-build.md)
 
+### Universal glyph set + UC17 curation
+
+- [23 — Universal glyph set: Tier 1 source map](23-universal-glyph-set-source-map.md)
+- [24 — Universal glyph set build](24-universal-glyph-set-build.md)
+- [25 — Font audit against universal set](25-font-audit-against-universal-set.md)
+- [26 — Missing glyph reporter (drill-down view)](26-missing-glyph-reporter.md)
+- [27 — fontist.org consumer integration](27-fontist-org-consumer-integration.md)
+- [29 — Universal glyph set: full Unicode 17 curation (Part 1)](29-universal-set-curation-uc17.md)
+- [30 — Tier 1 font acquisition: specialist fonts](30-tier1-font-acquisition.md)
+- [31 — Universal set production build + coverage validation](31-universal-set-production-build.md)
+
+### Full UC17 coverage + per-font audit (Part 1 close + Part 2)
+
+- [32 — Universal glyph set: full UC17 coverage matrix (Part 1 master)](32-uc17-coverage-matrix.md)
+- [33 — Specialist font acquisition refresh](33-specialist-font-acquisition-refresh.md)
+- [34 — Pillar 2 ContentStreamCorrelator (generalize correlate-v4)](34-pillar2-content-stream-correlator.md)
+- [35 — Universal set production run + glyph provenance (Part 1 close)](35-universal-set-production-run.md)
+- [36 — Per-font coverage audit against universal set (Part 2 master)](36-per-font-coverage-audit.md)
+- [37 — Coverage highlight reporter (missing-glyph visualizer)](37-coverage-highlight-reporter.md)
+- [38 — fontist.org glyph consumer + provenance display](38-fontist-org-glyph-consumer.md)
+
+### Pipeline wiring (archive integration)
+
+- [40 — fontist-archive-private bin/build uses ucode audit](40-archive-private-uses-ucode-audit.md)
+- [41 — ucode Unicode artifacts → fontist-archive-public bridge](41-ucode-unicode-archive-bridge.md)
+- [30 — Tier 1 font acquisition: specialist fonts](30-tier1-font-acquisition.md)
+- [31 — Universal set production build + coverage validation](31-universal-set-production-build.md)
+
 ### Sequencing
 
 - [22 — Implementation order](22-implementation-order.md)
+- [28 — Implementation order update (TODOs 23-31)](28-implementation-order-update.md)
+- [39 — Implementation order update (TODOs 32-38)](39-implementation-order-update-32-38.md)
 
 ## Conventions