expressir 2.4.0 → 2.4.2

data/TODO.bugs/16-pretty-formatter-duplication.md

@@ -0,0 +1,28 @@
+# 16 — PrettyFormatter duplicates declaration formatting from DeclarationsFormatter
+
+**Priority:** P1 (DRY violation)
+**Status:** ACCEPTED — duplication is intentional specialization
+
+## Problem
+
+`PrettyFormatter` redefines 5 `format_declarations_*` methods with
+indentation and provenance-header differences from the base
+`DeclarationsFormatter`.
+
+## Decision
+
+The duplication is **accepted** as intentional specialization.
+PrettyFormatter adds provenance headers (`format_provenance_header`),
+scope-body helpers (`format_scope_body`, `format_scope_footer`), and
+different indentation handling that the base formatter does not need.
+
+A full unification would require parameterizing the base formatter with
+indent-width, provenance support, and scope-body rendering — adding
+complexity to the common path to serve a specialized variant. The
+deletion test fails: removing PrettyFormatter's overrides would
+concentrate complexity into the base formatter, making the common path
+harder to reason about.
+
+The duplication is bounded (5 methods, ~100 lines) and the two
+formatters evolve independently (PrettyFormatter adds features the
+base does not need).

data/TODO.bugs/17-snake-case-cache-mutable-constant.md

@@ -0,0 +1,28 @@
+# 17 — SNAKE_CASE_CACHE is a mutable module constant
+
+**Priority:** P1 (unusual pattern)
+
+## Problem
+
+`lib/expressir/express/builder.rb:32`:
+
+```ruby
+SNAKE_CASE_CACHE = {} # rubocop:disable Style/MutableConstant
+```
+
+A module-level constant that is mutated in place at line 265 via `||=`.
+It grows unboundedly across the process lifetime and violates the frozen-
+constant convention the rest of the codebase follows.
+
+## Fix
+
+Either:
+- Document it explicitly as an intentionally-mutable static-conversion
+  cache and wrap it in a dedicated `SnakeCaseCache` class with a clear API.
+- Replace with `Thread.current[:snake_case_cache] ||= {}` for thread-local
+  bounded growth.
+
+## Acceptance
+
+- No `rubocop:disable Style/MutableConstant` needed.
+- Cache behavior unchanged.

data/TODO.bugs/18-const-get-private-constants.md

@@ -0,0 +1,30 @@
+# 18 — const_get to access private constants from formatter modules
+
+**Priority:** P2 (fragile access pattern)
+
+## Problem
+
+Formatter modules use `self.class.const_get(:INDENT)`,
+`self.class.const_get(:OPERATOR_PRECEDENCE)`, etc. to access constants
+declared `private_constant` on `Formatter`. ~10 call sites across:
+
+- `formatters/declarations_formatter.rb:129,235,304,360,535`
+- `formatters/expressions_formatter.rb:41`
+- `formatters/data_types_formatter.rb:131,265,304`
+- `formatters/supertype_expressions_formatter.rb:13`
+
+String-keyed reflective access to bypass visibility makes the code harder
+to grep and reason about.
+
+## Fix
+
+Remove `private_constant` on `INDENT`, `INDENT_CHAR`, `OPERATOR_PRECEDENCE`,
+and `SUPERTYPE_OPERATOR_PRECEDENCE` since the formatter modules are
+implementation details of Formatter anyway. Replace `const_get(:X)` with
+direct `X` references.
+
+## Acceptance
+
+- `grep -rn "const_get" lib/expressir/express/` returns nothing.
+- No `private_constant` on format constants.
+- All formatter specs pass.

data/TODO.bugs/19-format-methods-public.md

@@ -0,0 +1,22 @@
+# 19 — All format methods must be public due to format_registry dispatch
+
+**Priority:** P2 (encapsulation)
+
+## Problem
+
+`Formatter#format` dispatches via `public_send(handler, node)` at
+`lib/expressir/express/formatter.rb:88`. This forces ~60 `format_*`
+methods across 9 formatter modules to be public, even though they are only
+ever called internally via the registry.
+
+## Fix
+
+Replace `public_send(handler, node)` with `send(handler, node)` in
+`Formatter#format` (the handler comes from the trusted registry, not user
+input), then mark all `format_*` methods as `private`.
+
+## Acceptance
+
+- `Formatter#format` uses `send` not `public_send`.
+- Format methods are private in all 9 formatter modules.
+- `Formatter.new.public_methods` does not include any `format_*` method.

data/TODO.bugs/20-coverage-nested-entities-dedup.md

@@ -0,0 +1,20 @@
+# 20 — Coverage.find_nested_entities repeats 6-collection pattern three times
+
+**Priority:** P2 (DRY violation)
+
+## Problem
+
+`lib/expressir/coverage.rb:404-477` — `find_nested_entities` has three
+near-identical `when` branches for Function, Rule, and Procedure, each
+concatenating the same six collections and recursing.
+
+## Fix
+
+Replace the three branches with a single loop over
+`node.class.collection_attributes_list` (from the `collection_attributes`
+macro) and recurse into children.
+
+## Acceptance
+
+- `find_nested_entities` has no model-class-specific case branches.
+- All coverage specs pass.

data/TODO.bugs/21-operator-tokens-secondary-dispatch.md

@@ -0,0 +1,21 @@
+# 21 — OPERATOR_TOKENS in Builder is a secondary dispatch mechanism
+
+**Priority:** P2 (parallel dispatch)
+
+## Problem
+
+`lib/expressir/express/builder.rb:51-59` — 24-element `Set` of operator
+token symbols, used at line 87 to decide whether to skip a nil-returning
+handler. This set must be kept in sync with the grammar.
+
+## Fix
+
+Instead of maintaining a hand-curated set, check whether the handler
+returns nil for ANY first-key handler when the hash has multiple keys, not
+just for operator tokens. This eliminates the need for the set.
+
+## Acceptance
+
+- `OPERATOR_TOKENS` constant removed.
+- Builder still handles multi-key hash AST correctly.
+- All parser + builder specs pass.

data/TODO.bugs/22-builder-fast-path-wrappers.md

@@ -0,0 +1,32 @@
+# 22 — Builder fast-path methods are trivial wrappers
+
+**Priority:** P2 (dead code surface)
+
+## Problem
+
+`lib/expressir/express/builder.rb:232-259` — Six methods that each just
+call `build_node(:symbol, data)`, adding no logic:
+
+```ruby
+def build_term(data) = build_node(:term, data)
+def build_factor(data) = build_node(:factor, data)
+def build_simple_factor(data) = build_node(:simple_factor, data)
+def build_primary(data) = build_node(:primary, data)
+def build_expression(data) = build_node(:expression, data)
+def build_simple_expression(data) = build_node(:simple_expression, data)
+```
+
+Called ~25 times across the builders. The indirection provides no value
+since `build_node` is equally readable.
+
+## Fix
+
+Inline the calls to `build_node(:term, ...)`, etc. Remove the six wrapper
+methods. Reduce the Builder public surface.
+
+## Acceptance
+
+- `build_term`, `build_factor`, `build_simple_factor`, `build_primary`,
+  `build_expression`, `build_simple_expression` removed from Builder.
+- All call sites updated to `build_node(:term, ...)`.
+- All parser + builder specs pass.

data/TODO.bugs/23-coverage-inverse-maps.md

@@ -0,0 +1,21 @@
+# 23 — Coverage maps are manual inverses
+
+**Priority:** P2 (DRY violation)
+
+## Problem
+
+`lib/expressir/coverage.rb:7-52` — `ENTITY_TYPE_MAP` and
+`CLASS_TO_EXPRESS_TYPE_MAP` are 20-entry frozen hashes that are exact
+inverses of each other. Adding or renaming a type requires updating both.
+
+## Fix
+
+Generate one map from the other at load time, or use a single canonical
+list and derive both directions via `.to_h` and `.invert`.
+
+## Acceptance
+
+- Only one source-of-truth map exists.
+- Both `ENTITY_TYPE_MAP` and `CLASS_TO_EXPRESS_TYPE_MAP` (or their
+  successors) are derived from the single source.
+- All coverage specs pass.

data/TODO.bugs/24-streaming-builder-complexity.md

@@ -0,0 +1,19 @@
+# 24 — StreamingBuilder convert_ast_format is 190 lines with 5 levels of nesting
+
+**Priority:** P3 (complexity; defer unless time allows)
+
+## Problem
+
+`lib/expressir/express/streaming_builder.rb:241-429` —
+`convert_ast_format` is a 190-line method with deeply nested conditionals
+for merging Parsanol's single-key-hash sequences.
+
+## Fix
+
+Split into named sub-methods for each pattern: merge-sequence,
+repetition-unzip, str-concat.
+
+## Acceptance
+
+- No method in StreamingBuilder exceeds 30 lines.
+- All streaming specs pass.

data/TODO.bugs/25-debug-puts-in-production.md

@@ -0,0 +1,21 @@
+# 25 — Debug puts statements in production code
+
+**Priority:** P3 (cleanup)
+
+## Problem
+
+`lib/expressir/express/streaming_builder.rb:115,123,447-453` — `puts
+"DEBUG ..."` guarded by `ENV["DEBUG_STREAMING"]`.
+
+Using `puts` for debug logging means messages go to stdout mixed with
+program output. No way to control log level programmatically.
+
+## Fix
+
+Use `Logger` or `Expressir::Benchmark.trace` for debug output. Or remove
+the statements if they are stale.
+
+## Acceptance
+
+- `grep -rn "puts.*DEBUG" lib/` returns nothing.
+- Streaming builder still works in debug mode (if retained).

data/TODO.bugs/26-generic-entity-children-misplaced.md

@@ -0,0 +1,21 @@
+# 26 — GenericEntity#children misplaced (P0 BUG)
+
+**Priority:** P0 (correctness bug)
+**Status:** FIXED
+
+## Problem
+
+`lib/expressir/model/data_types/generic_entity.rb` had the `children`
+method defined OUTSIDE the `GenericEntity` class body — it was on the
+`DataTypes` module instead. Calling `GenericEntity.new.children` would
+raise NoMethodError.
+
+## Fix
+
+Moved `def children` inside the `GenericEntity` class body (before the
+final `end`).
+
+## Acceptance
+
+- `GenericEntity.new.children` no longer raises.
+- All tests pass.

data/TODO.bugs/27-package-build-god-method.md

@@ -0,0 +1,19 @@
+# 27 — Package#build is a 195-line god method
+
+**Priority:** P1 (god method)
+
+## Problem
+
+`lib/expressir/commands/package.rb:94-288` — `Package#build` handles
+manifest validation, auto-resolution, repo building, validation, packaging,
+and error formatting in one method.
+
+## Fix
+
+Extract `build_from_manifest`, `build_from_auto_resolution`,
+`run_validation` as private methods.
+
+## Acceptance
+
+- No method exceeds 40 lines.
+- All package specs pass.

data/TODO.bugs/28-package-god-class.md

@@ -0,0 +1,30 @@
+# 28 — Package command is a 1266-line god class
+
+**Priority:** P2 (god class)
+**Status:** ACCEPTED — Thor command pattern
+
+## Problem
+
+`lib/expressir/commands/package.rb` is 1266 lines with 30 methods: display
+logic (tree, info, search, list), package building, validation output,
+colorization, and element introspection.
+
+## Decision
+
+The god class is **accepted** as the Thor command pattern. Thor commands
+are defined as methods within a single class with `desc`/`long_desc`/`option`
+declarations. Extracting display methods to a separate module would:
+
+1. Separate `desc` declarations from method implementations — confusing
+   for Thor users who expect `desc` immediately before `def`.
+2. Risk breaking Thor's dispatch (Thor finds methods via
+   `instance_method_defined?` on the class; included-module methods
+   work but are non-standard for Thor commands).
+3. Move code without concentrating complexity — the deletion test fails:
+   the display methods and their helpers earn their keep in the command
+   class because they ARE the command's interface.
+
+The Package#build method was already split (TODO 27) into focused
+sub-methods. The remaining methods (info, validate, extract, list,
+search, tree) are each 30-60 lines — individual methods are well-sized;
+only the file is large.

data/TODO.bugs/29-validate-ascii-god-class.md

@@ -0,0 +1,24 @@
+# 29 — ValidateAscii is a 604-line god class with inline inner classes
+
+**Priority:** P2 (god class)
+**Status:** DONE
+
+## Problem
+
+`validate_ascii.rb` was 549 lines with 3 sibling classes defined inline:
+NonAsciiCharacter, FileViolations, NonAsciiViolationCollection.
+
+## Fix
+
+Extracted the three inner classes to separate files:
+- `lib/expressir/commands/non_ascii_character.rb` (50 lines)
+- `lib/expressir/commands/file_violations.rb` (71 lines)
+- `lib/expressir/commands/non_ascii_violation_collection.rb` (349 lines)
+
+`validate_ascii.rb` now contains only the ValidateAscii command class
+(94 lines). Autoloads added to `lib/expressir/commands.rb`.
+
+## Acceptance
+
+- validate_ascii.rb under 100 lines.
+- All 26 validate_ascii specs pass.

data/TODO.bugs/30-unicode-map-extraction.md

@@ -0,0 +1,19 @@
+# 30 — build_unicode_to_asciimath_map is 80 lines of data inside a method
+
+**Priority:** P2 (data-as-code)
+
+## Problem
+
+`lib/expressir/commands/validate_ascii.rb:436-517` —
+`build_unicode_to_asciimath_map` is an 80-line hash literal defined inside
+a method body. Should be a frozen constant.
+
+## Fix
+
+Extract to `lib/expressir/express/unicode_mapping.rb` as a frozen
+constant.
+
+## Acceptance
+
+- The hash is a module-level frozen constant.
+- All validate_ascii specs pass.

data/TODO.bugs/README.md

@@ -0,0 +1,43 @@
+# TODO.bugs
+
+Track all architectural and code-cleanliness work.
+
+## Status
+
+**All 30 TODOs resolved.** 24 implemented, 4 accepted as design decisions
+(14, 16) or deferred with clear reasoning (28, 29).
+
+## Index
+
+| # | Priority | Status | Title |
+| --- | --- | --- | --- |
+| [01](01-stale-transformer-autoload.md) | P0 | done | Delete stale `Transformer` autoload |
+| [02](02-parser-class-instance-vars.md) | P0 | done | Replace class-level mutation in `Parser` |
+| [03](03-builder-mutable-state.md) | P1 | done | Thread `BuilderContext` |
+| [04](04-formatter-public-send-dispatch.md) | P1 | done | Registration-time guard |
+| [05](05-anonymous-formatter-subclass.md) | P1 | done | Cache named formatter classes |
+| [06](06-collection-registry-single-source.md) | P1 | done | Consolidate `SCOPE_DECL_COLLECTIONS` |
+| [07](07-require-relative-cleanup.md) | P2 | done | Replace `require_relative` with autoload |
+| [08](08-require-expressir-in-commands.md) | P2 | done | Remove `require "expressir/..."` |
+| [09](09-parser-split.md) | P2 | done | Split parser.rb into grammar + scanner + facade |
+| [10](10-to-s-override.md) | P2 | done | Rename `to_s` to `format` |
+| [11](11-parser-class-variables.md) | P3 | done | Replace `@@` class variables |
+| [12](12-marker-modules-vs-registry.md) | P3 | done | `collection_attributes` macro |
+| [13](13-string-literal-scanner-limitation.md) | P1 | done | `:in_string` scanner state |
+| [14](14-model-formatting-leak.md) | P1 | accepted | Model formatting — domain coupling |
+| [15](15-expression-children-macro.md) | P1 | done | `child_attributes` macro |
+| [16](16-pretty-formatter-duplication.md) | P1 | accepted | PrettyFormatter — intentional specialization |
+| [17](17-snake-case-cache-mutable-constant.md) | P1 | done | Thread-local snake_case cache |
+| [18](18-const-get-private-constants.md) | P2 | done | Remove `private_constant` |
+| [19](19-format-methods-public.md) | P2 | done | `send` dispatch for private methods |
+| [20](20-coverage-nested-entities-dedup.md) | P2 | done | Coverage uses `collection_attributes_list` |
+| [21](21-operator-tokens-secondary-dispatch.md) | P2 | done | Remove OPERATOR_TOKENS set |
+| [22](22-builder-fast-path-wrappers.md) | P2 | done | Remove fast-path wrappers |
+| [23](23-coverage-inverse-maps.md) | P2 | done | Derive maps from `TYPE_PAIRS` |
+| [24](24-streaming-builder-complexity.md) | P3 | done | Split `convert_ast_format` |
+| [25](25-debug-puts-in-production.md) | P3 | done | Remove debug puts |
+| [26](26-generic-entity-children-misplaced.md) | P0 | done | Fix misplaced `children` method |
+| [27](27-package-build-god-method.md) | P1 | done | Split `Package#build` |
+| [28](28-package-god-class.md) | P2 | deferred | Package 1266-line god class |
+| [29](29-validate-ascii-god-class.md) | P2 | deferred | ValidateAscii god class |
+| [30](30-unicode-map-extraction.md) | P2 | done | Extract unicode map to constant |

data/TODO.max-perf/01-restore-ci-green.md

@@ -0,0 +1,29 @@
+# TODO.max-perf/01 — Restore CI green on main
+
+## Context
+
+CI on main (`0ef38e1`, 2026-09-17) fails on the Windows matrix: yeptris
+0.6.3.3's mingw build raises `NameError: uninitialized constant
+Yeptris::FFI::NODE_SCALAR` from `yeptris/node.rb` whenever `to_yaml` runs.
+Expressir's gemspec constraint (`~> 0.6.1`, from PR #346) allows it, and the
+Gemfile.lock is gitignored, so CI resolves the newest release at run time.
+
+yeptris has since released 0.6.4.1 and 0.6.5.1.
+
+## Work
+
+- [x] Reproduce the failing spec groups locally on the newest yeptris
+      (`package_fixtures_spec`, `package_spec`, `cache_spec`): 84/84 green on
+      0.6.5.1
+- [x] Re-run the failed rake workflow on main (resolves 0.6.5.1)
+- [ ] Confirm the rake workflow on main is green
+- [x] 0.6.5.1 does NOT fix mingw (same NameError, all Windows jobs) — filed
+      leptris/yeptris#318 with the CI repro
+- [x] expressir-side guard: `Expressir.select_serialization_engines` pins the
+      portable :standard YAML/JSON adapters on `Gem.win_platform?` at boot, so
+      lutaml-model's autodetection never loads the broken mingw build; Unix
+      keeps yeptris. Spec: spec/expressir/engine_selection_spec.rb
+
+## Acceptance
+
+rake workflow green on main across the full platform matrix.

data/TODO.max-perf/02-streaming-parse-path.md

@@ -0,0 +1,31 @@
+# TODO.max-perf/02 — Fix or retire the streaming parse path
+
+## Context
+
+`Parser.from_exp(use_streaming: true)` — documented as the maximum-performance
+path (construct-by-construct building via `Parsanol::Native.parse_with_builder`,
+no full intermediate AST) — fails on every schema tested with
+`Error::SchemaParseFailure` on parsanol 1.3.28 and 1.3.30. The feature is dead
+weight until it works: callers cannot opt into it, and it skews the perf
+picture (the fastest intended path is the broken one).
+
+## Work
+
+- [x] Reproduced on a 5-line schema: `parse_fresh(schema_grammar, block)` fails
+      at end-of-input — parse_fresh has no packrat memoization, which EXPRESS
+      requires (parsanol-ruby#52)
+- [x] The true streaming path (`from_exp_streaming_builder`) is unreachable:
+      its gate uses `defined?(Parsanol::Native.parse_with_builder)`, which is
+      always nil for the extension's dynamically-dispatched methods; calling it
+      directly yields empty models (builder protocol drift vs parsanol 1.3.30)
+- [x] Removed both dead paths; `use_streaming: true` now raises
+      `Error::StreamingUnsupportedError` with a precise message
+- [x] Spec: spec/expressir/express/streaming_spec.rb
+- [x] Upstream: covered by parsanol-ruby#59 roadmap (ractor-safe / stable
+      parse_with_builder unlocks a rewrite of this path)
+
+## Acceptance
+
+Either `use_streaming: true` produces an equal model with a green spec, or
+the breakage is precisely documented upstream and the path is explicitly
+marked unsupported.

data/TODO.max-perf/03-cli-parallel-opt-in.md

@@ -0,0 +1,27 @@
+# TODO.max-perf/03 — CLI parallel opt-in and multi-file loader consolidation
+
+## Context
+
+The fork worker pool (PRs #349/#351) is strictly opt-in for library users.
+The expressir CLI is application mode — our process — so it can opt in.
+Today `commands/coverage.rb` calls `Parser.from_files` without
+`max_processes` (sequential), and `commands/package.rb` calls
+`Model::Repository.from_files` — a *second* multi-file loading path
+duplicating the same orchestration (MECE violation).
+
+## Work
+
+- [x] `--max-processes` on `coverage` (default 4, Thor layer in cli.rb) and
+      `package build` (default 1 — strict error semantics preserved)
+- [x] `ParallelFiles.run` gained `strict:` — re-raises SchemaParseFailure,
+      preserving Repository.from_files' fail-fast contract
+- [x] `Model::Repository.from_files` now delegates its parse loop to
+      ParallelFiles (shared orchestration, strict mode); tolerant
+      Parser.from_files behavior unchanged
+- [x] Specs: repository parallel == sequential (ids + order), strict raise,
+      coverage/package suites green (57 + 57 examples)
+
+## Acceptance
+
+One multi-file loading pipeline; CLI bulk loads parse in parallel by
+default; library default remains sequential.

data/TODO.max-perf/04-benchmark-harness.md

@@ -0,0 +1,28 @@
+# TODO.max-perf/04 — Repair the SRL benchmark harness
+
+## Context
+
+`benchmark/srl_benchmark.rb` has two defects found during the Sept 2026
+validation:
+
+1. The "Ruby Parser" pass calls `from_file(file, skip_references: true)`
+   without `use_native: false`. Since native-became-default (2026-03-25),
+   both passes run the native parser — the head-to-head comparison is
+   mislabeled and meaningless.
+2. Per-file `Timeout.timeout(30)` cannot interrupt the native FFI parse
+   (GVL held), so pathological files hang the benchmark instead of timing
+   out — observed as a 25-minute stall on an 8.6KB schema.
+
+## Work
+
+- [x] Ruby-parser pass (and warmup) now pass `use_native: false`
+- [x] `parse_file_isolated`: per-file fork, length-prefixed Marshal results,
+      TERM→KILL wall-clock escalation — pathological files terminate
+- [x] `SRL_PATH` env override for reproducible smoke runs
+- [x] Verified: 3-fixture run prints genuine Ruby-vs-Native numbers with
+      truthful labels and terminates cleanly
+
+## Acceptance
+
+A short benchmark run prints genuine Ruby-vs-Native numbers, terminates on
+pathological files, and labels passes truthfully.

data/TODO.max-perf/05-parallel-fidelity-specs.md

@@ -0,0 +1,22 @@
+# TODO.max-perf/05 — Parallel-parsing fidelity specs
+
+## Context
+
+Parallel `from_files` equality is currently asserted on schema ids only.
+The pool crosses a fork boundary via Marshal; deep structural equality is
+the real contract. The progress-block regression merged in #349 (double
+`schemas` extraction) proved the block/caller paths need dedicated specs.
+
+## Work
+
+- [x] Deep-equality spec: `to_hash` of parallel repository == sequential
+- [x] Progress-block contract specs on both paths (schemas is an Array of
+      Declarations::Schema)
+- [x] Error-path specs: skip-vs-raise by mode (tolerant/strict), including
+      the strict-mode re-raise of SchemaParseFailure
+- [x] Gate specs: nil/1/<3 files and fork-less platforms never fork
+
+## Acceptance
+
+All fidelity specs green; `to_hash` of a parallel repository equals the
+sequential one for a multi-fixture corpus.

data/TODO.max-perf/06-builder-cpu-audit.md

@@ -0,0 +1,41 @@
+# TODO.max-perf/06 — Ruby-side (Builder) CPU audit on the current stack
+
+## Context
+
+After parsanol 1.3.30 (~2.5x further parse speedup), the native parse share
+collapsed and the Ruby side — AstTransformer → Builder → lutaml-model
+instantiation → remark attachment — is now the dominant parse cost
+(sampled 2026-09-17: 72/28 Rust/Ruby on 1.3.28; native got ~2.5x faster
+afterwards). March data ("removing Ruby transform: 2.5%") is obsolete.
+
+Related landed work: f32f85f "eliminate allocation storms in remark
+attachment" (another session, on main), 4c49e3a per-remark rescan guards.
+
+## Work
+
+Results (CPU-time, parsanol 1.3.30, 4 mid-size SRL schemas):
+
+| phase | total | share |
+|---|---:|---:|
+| native parse (parse_native) | 1.595s | 50% |
+| Builder.build_with_remarks | 1.530s | 48% |
+| reference resolution | 0.080s | 2% |
+
+Builder split: model build (`Builder.build`) 1.12s vs remark attachment
+0.06s — remark attachment is already cheap (f32f85f); the cost is
+AST-hash → lutaml-model instantiation.
+
+- [x] Phase timing recorded (above)
+- [x] Builder sub-phase split recorded (above)
+- [x] Next target identified precisely: `Builder.build` per-node path —
+      `cached_snake_case` / `fast_convert_keys` string allocations and
+      lutaml-model constructor overhead (thousands of attribute setters).
+      C stack sampling cannot name Ruby methods (vm_exec_core only);
+      instrument inside Builder.build or add stackprof as a dev dependency
+      before optimizing. Not refactored blindly — no safe evident win
+      without that instrumentation.
+
+## Acceptance
+
+A numbers table attributing Ruby-side parse CPU to phases, plus either a
+measured improvement or a precise, evidence-backed next target.

data/TODO.max-perf/07-upstream-parsanol-roadmap.md

@@ -0,0 +1,27 @@
+# TODO.max-perf/07 — Upstream parsanol roadmap (research 2022–2026)
+
+## Context
+
+The field's fastest grammar engines compile grammars instead of interpreting
+them. Mapping to parsanol (all upstream work; expressir only consumes):
+
+| Technique | Precedent | parsanol mapping |
+|---|---|---|
+| JIT grammar compilation + tag-dispatched structure switching + cross-grammar cache reuse (6x) | XGrammar-2 (arXiv:2601.04426) | BYTE_DISPATCH is conceptually TagDispatch; compile to a dispatch table per atom |
+| Grammar → generated code (production precedent) | tree-sitter (precompiled C, incremental reparse), CPython's generated PEG parser | generate Rust from the EXPRESS grammar instead of walking atom structs |
+| Skip packrat tables via automata | Pest discussion #1081 | memoization-free automata for the deterministic fragments; keep packrat only where ambiguity needs it |
+| SIMD byte classification (GB/s lexing) | simdjson, Lemire's vectorized classification (arXiv:2503.01662) | vectorize keyword/delimiter classification in the lexer tier |
+| Incremental re-parse | tree-sitter, gpeg (SLE'21) | re-parse only dirty schema blocks — pairs with expressir's SchemaBlockScanner |
+
+## Work
+
+- [x] Filed parsanol-ruby#59 with the mapping, profile evidence, citations,
+      and the Ractor-safe entry point item (the only parallelism option on
+      Windows MRI and fork-hostile embedders)
+- [x] Consumer unlocks noted: incremental reparse pairs with expressir's
+      SchemaBlockScanner; compiled grammar cuts cold-start (~2,300 atoms
+      serialized per boot); ractor-safe parse replaces the fork pool
+
+## Acceptance
+
+Upstream issue filed; expressir-side dependencies on each technique noted.