expressir 2.4.29-arm-linux-musl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (426) hide show
  1. checksums.yaml +7 -0
  2. data/.cargo/config.toml +3 -0
  3. data/.github/scripts/test_installed_gem.rb +34 -0
  4. data/.github/workflows/codeql.yml +35 -0
  5. data/.github/workflows/docs.yml +99 -0
  6. data/.github/workflows/links.yml +100 -0
  7. data/.github/workflows/native-gems.yml +201 -0
  8. data/.github/workflows/rake.yml +19 -0
  9. data/.github/workflows/release.yml +37 -0
  10. data/.github/workflows/rust-ext.yml +41 -0
  11. data/.github/workflows/stress.yml +36 -0
  12. data/.github/workflows/validate_schemas.yml +48 -0
  13. data/.github/workflows/verify_remarks.yml +33 -0
  14. data/.gitignore +55 -0
  15. data/.hound.yml +3 -0
  16. data/.rspec +2 -0
  17. data/.rubocop.yml +18 -0
  18. data/.rubocop_todo.yml +297 -0
  19. data/CHANGELOG.md +159 -0
  20. data/Gemfile +18 -0
  21. data/README.adoc +2783 -0
  22. data/Rakefile +41 -0
  23. data/TODO.bugs/01-stale-transformer-autoload.md +39 -0
  24. data/TODO.bugs/02-parser-class-instance-vars.md +36 -0
  25. data/TODO.bugs/03-builder-mutable-state.md +43 -0
  26. data/TODO.bugs/04-formatter-public-send-dispatch.md +53 -0
  27. data/TODO.bugs/05-anonymous-formatter-subclass.md +45 -0
  28. data/TODO.bugs/06-collection-registry-single-source.md +53 -0
  29. data/TODO.bugs/07-require-relative-cleanup.md +42 -0
  30. data/TODO.bugs/08-require-expressir-in-commands.md +34 -0
  31. data/TODO.bugs/09-parser-split.md +53 -0
  32. data/TODO.bugs/10-to-s-override.md +42 -0
  33. data/TODO.bugs/11-parser-class-variables.md +39 -0
  34. data/TODO.bugs/12-marker-modules-vs-registry.md +64 -0
  35. data/TODO.bugs/13-string-literal-scanner-limitation.md +52 -0
  36. data/TODO.bugs/14-model-formatting-leak.md +30 -0
  37. data/TODO.bugs/15-expression-children-macro.md +27 -0
  38. data/TODO.bugs/16-pretty-formatter-duplication.md +28 -0
  39. data/TODO.bugs/17-snake-case-cache-mutable-constant.md +28 -0
  40. data/TODO.bugs/18-const-get-private-constants.md +30 -0
  41. data/TODO.bugs/19-format-methods-public.md +22 -0
  42. data/TODO.bugs/20-coverage-nested-entities-dedup.md +20 -0
  43. data/TODO.bugs/21-operator-tokens-secondary-dispatch.md +21 -0
  44. data/TODO.bugs/22-builder-fast-path-wrappers.md +32 -0
  45. data/TODO.bugs/23-coverage-inverse-maps.md +21 -0
  46. data/TODO.bugs/24-streaming-builder-complexity.md +19 -0
  47. data/TODO.bugs/25-debug-puts-in-production.md +21 -0
  48. data/TODO.bugs/26-generic-entity-children-misplaced.md +21 -0
  49. data/TODO.bugs/27-package-build-god-method.md +19 -0
  50. data/TODO.bugs/28-package-god-class.md +30 -0
  51. data/TODO.bugs/29-validate-ascii-god-class.md +24 -0
  52. data/TODO.bugs/30-unicode-map-extraction.md +19 -0
  53. data/TODO.bugs/README.md +43 -0
  54. data/TODO.max-perf/01-restore-ci-green.md +36 -0
  55. data/TODO.max-perf/02-streaming-parse-path.md +31 -0
  56. data/TODO.max-perf/03-cli-parallel-opt-in.md +27 -0
  57. data/TODO.max-perf/04-benchmark-harness.md +28 -0
  58. data/TODO.max-perf/05-parallel-fidelity-specs.md +22 -0
  59. data/TODO.max-perf/06-builder-cpu-audit.md +41 -0
  60. data/TODO.max-perf/07-upstream-parsanol-roadmap.md +27 -0
  61. data/TODO.max-perf/08-builder-build-perf.md +45 -0
  62. data/TODO.max-perf/09-grammar-cold-start.md +25 -0
  63. data/TODO.max-perf/10-parser-facade-hygiene.md +23 -0
  64. data/TODO.max-perf/11-ci-green-closeout.md +25 -0
  65. data/TODO.max-perf/12-require-boot-profile.md +25 -0
  66. data/TODO.max-perf/13-key-conversion-specs.md +26 -0
  67. data/TODO.max-perf/14-builder-call-handler-audit.md +28 -0
  68. data/TODO.parity-ee/00-overview.md +76 -0
  69. data/TODO.parity-ee/01-annex-g-rule-extraction.md +41 -0
  70. data/TODO.parity-ee/02-eeng-algorithm-inventory.md +8 -0
  71. data/TODO.parity-ee/03-eeng-oracle-harness.md +33 -0
  72. data/TODO.parity-ee/04-shtolo-concatenate.md +44 -0
  73. data/TODO.parity-ee/05-shtolo-longform-flatten.md +53 -0
  74. data/TODO.parity-ee/06-interface-scheduling-parity.md +47 -0
  75. data/TODO.parity-ee/07-semantic-checks-port.md +35 -0
  76. data/TODO.parity-ee/08-pretty-roundtrip-gate.md +27 -0
  77. data/TODO.parity-ee/09-smrl-index-and-listing.md +22 -0
  78. data/TODO.parity-ee/10-interface-dot-graph.md +21 -0
  79. data/TODO.parity-ee/11-import-eeng-tests.md +79 -0
  80. data/TODO.parity-ee/12-full-check-catalog.md +67 -0
  81. data/TODO.parity-ee/13-population-p21.md +54 -0
  82. data/TODO.parity-ee/14-shtolo-completion.md +44 -0
  83. data/TODO.parity-ee/15-part28-xml.md +51 -0
  84. data/TODO.parity-ee/16-compare-patch.md +48 -0
  85. data/TODO.parity-ee/17-official-tests.md +42 -0
  86. data/TODO.parity-ee/18-mim-mapping.md +47 -0
  87. data/TODO.parity-ee/19-architecture.md +97 -0
  88. data/TODO.parity-ee/parity-matrix.md +108 -0
  89. data/TODO.refpath-v2/00-overview.md +64 -0
  90. data/TODO.suma-improvements/00-overview.md +23 -0
  91. data/TODO.suma-improvements/01-prebuilt-platform-gems.md +33 -0
  92. data/TODO.suma-improvements/02-suma-e2e.md +39 -0
  93. data/TODO.suma-improvements/03-item-graph.md +22 -0
  94. data/TODO.suma-improvements/04-compiled-set-default.md +16 -0
  95. data/TODO.suma-improvements/05-validation-in-docs.md +17 -0
  96. data/TODO.suma-improvements/06-memory-remeasure.md +36 -0
  97. data/TODO.suma-improvements/07-part28-xml.md +15 -0
  98. data/TODO.suma-improvements/08-windows-gnu-toolchain.md +15 -0
  99. data/TODO.suma-improvements/09-checker-tranche3.md +13 -0
  100. data/TODO.suma-improvements/10-expressdoc.md +8 -0
  101. data/benchmark/srl_benchmark.rb +458 -0
  102. data/benchmark/srl_native_benchmark.rb +146 -0
  103. data/benchmark/srl_ruby_benchmark.rb +132 -0
  104. data/bin/console +11 -0
  105. data/bin/rspec +30 -0
  106. data/bin/setup +8 -0
  107. data/docs/Gemfile +12 -0
  108. data/docs/_config.yml +141 -0
  109. data/docs/_guides/changes/changes-format.adoc +778 -0
  110. data/docs/_guides/changes/importing-eengine.adoc +898 -0
  111. data/docs/_guides/changes/index.adoc +396 -0
  112. data/docs/_guides/changes/programmatic-usage.adoc +1038 -0
  113. data/docs/_guides/changes/validating-changes.adoc +681 -0
  114. data/docs/_guides/cli/benchmark-performance.adoc +834 -0
  115. data/docs/_guides/cli/coverage-analysis.adoc +921 -0
  116. data/docs/_guides/cli/format-schemas.adoc +547 -0
  117. data/docs/_guides/cli/index.adoc +8 -0
  118. data/docs/_guides/cli/managing-changes.adoc +927 -0
  119. data/docs/_guides/cli/validate-ascii.adoc +645 -0
  120. data/docs/_guides/cli/validate-schemas.adoc +534 -0
  121. data/docs/_guides/formatter/formatter-architecture.adoc +401 -0
  122. data/docs/_guides/index.adoc +165 -0
  123. data/docs/_guides/ler/creating-packages.adoc +664 -0
  124. data/docs/_guides/ler/index.adoc +305 -0
  125. data/docs/_guides/ler/loading-packages.adoc +707 -0
  126. data/docs/_guides/ler/package-formats.adoc +748 -0
  127. data/docs/_guides/ler/querying-packages.adoc +826 -0
  128. data/docs/_guides/ler/step-packages.adoc +385 -0
  129. data/docs/_guides/ler/validating-packages.adoc +750 -0
  130. data/docs/_guides/liquid/basic-templates.adoc +813 -0
  131. data/docs/_guides/liquid/documentation-generation.adoc +1042 -0
  132. data/docs/_guides/liquid/drops-reference.adoc +829 -0
  133. data/docs/_guides/liquid/filters-and-tags.adoc +912 -0
  134. data/docs/_guides/liquid/index.adoc +468 -0
  135. data/docs/_guides/manifests/creating-manifests.adoc +483 -0
  136. data/docs/_guides/manifests/index.adoc +307 -0
  137. data/docs/_guides/manifests/resolving-manifests.adoc +557 -0
  138. data/docs/_guides/manifests/validating-manifests.adoc +713 -0
  139. data/docs/_guides/ruby-api/formatting-schemas.adoc +605 -0
  140. data/docs/_guides/ruby-api/index.adoc +257 -0
  141. data/docs/_guides/ruby-api/parsing-files.adoc +421 -0
  142. data/docs/_guides/ruby-api/search-engine.adoc +609 -0
  143. data/docs/_guides/ruby-api/working-with-repository.adoc +577 -0
  144. data/docs/_pages/data-model.adoc +665 -0
  145. data/docs/_pages/express-language.adoc +506 -0
  146. data/docs/_pages/getting-started.adoc +414 -0
  147. data/docs/_pages/index.adoc +116 -0
  148. data/docs/_pages/introduction.adoc +256 -0
  149. data/docs/_pages/ler-packages.adoc +837 -0
  150. data/docs/_pages/parsers.adoc +709 -0
  151. data/docs/_pages/schema-manifests.adoc +431 -0
  152. data/docs/_references/index.adoc +228 -0
  153. data/docs/_tutorials/creating-ler-package.adoc +735 -0
  154. data/docs/_tutorials/documentation-coverage.adoc +795 -0
  155. data/docs/_tutorials/formatting-schemas.adoc +89 -0
  156. data/docs/_tutorials/index.adoc +231 -0
  157. data/docs/_tutorials/liquid-templates.adoc +806 -0
  158. data/docs/_tutorials/parsing-your-first-schema.adoc +522 -0
  159. data/docs/_tutorials/querying-schemas.adoc +751 -0
  160. data/docs/_tutorials/working-with-multiple-schemas.adoc +676 -0
  161. data/docs/index.adoc +242 -0
  162. data/docs/lychee.toml +90 -0
  163. data/examples/demo_ler_usage.sh +86 -0
  164. data/examples/ler/README.md +111 -0
  165. data/examples/ler/simple_example.ler +0 -0
  166. data/examples/ler/simple_schema.exp +33 -0
  167. data/examples/ler_build.rb +75 -0
  168. data/examples/ler_cli.rb +79 -0
  169. data/examples/ler_demo_complete.rb +276 -0
  170. data/examples/ler_query.rb +91 -0
  171. data/examples/ler_query_examples.rb +305 -0
  172. data/examples/ler_stats.rb +81 -0
  173. data/examples/phase3_demo.rb +159 -0
  174. data/examples/query_demo_simple.rb +131 -0
  175. data/exe/expressir +11 -0
  176. data/exe/expressir-format +14 -0
  177. data/exe/expressir-format-test +82 -0
  178. data/expressir.gemspec +61 -0
  179. data/lib/expressir/3.2/expressir_core.so +0 -0
  180. data/lib/expressir/3.3/expressir_core.so +0 -0
  181. data/lib/expressir/3.4/expressir_core.so +0 -0
  182. data/lib/expressir/4.0/expressir_core.so +0 -0
  183. data/lib/expressir/benchmark.rb +310 -0
  184. data/lib/expressir/changes/item_change.rb +20 -0
  185. data/lib/expressir/changes/mapping_change.rb +16 -0
  186. data/lib/expressir/changes/schema_change.rb +85 -0
  187. data/lib/expressir/changes/version_change.rb +26 -0
  188. data/lib/expressir/changes.rb +9 -0
  189. data/lib/expressir/cli.rb +148 -0
  190. data/lib/expressir/commands/base.rb +25 -0
  191. data/lib/expressir/commands/benchmark.rb +59 -0
  192. data/lib/expressir/commands/benchmark_cache.rb +80 -0
  193. data/lib/expressir/commands/changes.rb +34 -0
  194. data/lib/expressir/commands/changes_import_eengine.rb +146 -0
  195. data/lib/expressir/commands/changes_validate.rb +63 -0
  196. data/lib/expressir/commands/check.rb +56 -0
  197. data/lib/expressir/commands/clean.rb +20 -0
  198. data/lib/expressir/commands/coverage.rb +458 -0
  199. data/lib/expressir/commands/expand.rb +26 -0
  200. data/lib/expressir/commands/file_violations.rb +70 -0
  201. data/lib/expressir/commands/fix.rb +33 -0
  202. data/lib/expressir/commands/flatten.rb +29 -0
  203. data/lib/expressir/commands/format.rb +43 -0
  204. data/lib/expressir/commands/manifest.rb +420 -0
  205. data/lib/expressir/commands/mapping_validate.rb +68 -0
  206. data/lib/expressir/commands/non_ascii_character.rb +49 -0
  207. data/lib/expressir/commands/non_ascii_violation_collection.rb +301 -0
  208. data/lib/expressir/commands/package.rb +1222 -0
  209. data/lib/expressir/commands/parity_inputs.rb +160 -0
  210. data/lib/expressir/commands/validate.rb +106 -0
  211. data/lib/expressir/commands/validate_ascii.rb +94 -0
  212. data/lib/expressir/commands/validate_load.rb +91 -0
  213. data/lib/expressir/commands/version.rb +9 -0
  214. data/lib/expressir/commands/xsd.rb +23 -0
  215. data/lib/expressir/commands.rb +30 -0
  216. data/lib/expressir/config.rb +60 -0
  217. data/lib/expressir/coverage.rb +510 -0
  218. data/lib/expressir/eengine/arm_compare_report.rb +23 -0
  219. data/lib/expressir/eengine/changes_section.rb +16 -0
  220. data/lib/expressir/eengine/compare_report.rb +69 -0
  221. data/lib/expressir/eengine/mim_compare_report.rb +23 -0
  222. data/lib/expressir/eengine/modified_object.rb +21 -0
  223. data/lib/expressir/eengine.rb +9 -0
  224. data/lib/expressir/errors.rb +113 -0
  225. data/lib/expressir/express/adoc_hyperlink_formatter.rb +30 -0
  226. data/lib/expressir/express/adoc_source_formatter.rb +12 -0
  227. data/lib/expressir/express/ast_key_converter.rb +114 -0
  228. data/lib/expressir/express/builder.rb +277 -0
  229. data/lib/expressir/express/builder_context.rb +22 -0
  230. data/lib/expressir/express/builder_registry.rb +397 -0
  231. data/lib/expressir/express/builders/attribute_decl_builder.rb +32 -0
  232. data/lib/expressir/express/builders/built_in_builder.rb +75 -0
  233. data/lib/expressir/express/builders/constant_builder.rb +108 -0
  234. data/lib/expressir/express/builders/declaration_builder.rb +20 -0
  235. data/lib/expressir/express/builders/derive_clause_builder.rb +14 -0
  236. data/lib/expressir/express/builders/derived_attr_builder.rb +28 -0
  237. data/lib/expressir/express/builders/domain_rule_builder.rb +21 -0
  238. data/lib/expressir/express/builders/entity_decl_builder.rb +116 -0
  239. data/lib/expressir/express/builders/explicit_attr_builder.rb +49 -0
  240. data/lib/expressir/express/builders/expression_builder.rb +416 -0
  241. data/lib/expressir/express/builders/function_decl_builder.rb +86 -0
  242. data/lib/expressir/express/builders/helpers.rb +148 -0
  243. data/lib/expressir/express/builders/interface_builder.rb +151 -0
  244. data/lib/expressir/express/builders/inverse_attr_builder.rb +43 -0
  245. data/lib/expressir/express/builders/inverse_attr_type_builder.rb +30 -0
  246. data/lib/expressir/express/builders/inverse_clause_builder.rb +14 -0
  247. data/lib/expressir/express/builders/literal_builder.rb +93 -0
  248. data/lib/expressir/express/builders/procedure_decl_builder.rb +82 -0
  249. data/lib/expressir/express/builders/qualifier_builder.rb +102 -0
  250. data/lib/expressir/express/builders/reference_builder.rb +18 -0
  251. data/lib/expressir/express/builders/rule_decl_builder.rb +97 -0
  252. data/lib/expressir/express/builders/schema_body_decl_builder.rb +18 -0
  253. data/lib/expressir/express/builders/schema_decl_builder.rb +56 -0
  254. data/lib/expressir/express/builders/schema_version_builder.rb +34 -0
  255. data/lib/expressir/express/builders/simple_id_builder.rb +17 -0
  256. data/lib/expressir/express/builders/statement_builder.rb +243 -0
  257. data/lib/expressir/express/builders/subtype_constraint_builder.rb +166 -0
  258. data/lib/expressir/express/builders/syntax_builder.rb +30 -0
  259. data/lib/expressir/express/builders/type_builder.rb +238 -0
  260. data/lib/expressir/express/builders/type_decl_builder.rb +26 -0
  261. data/lib/expressir/express/builders/unique_clause_builder.rb +20 -0
  262. data/lib/expressir/express/builders/unique_rule_builder.rb +41 -0
  263. data/lib/expressir/express/builders/where_clause_builder.rb +20 -0
  264. data/lib/expressir/express/builders.rb +55 -0
  265. data/lib/expressir/express/cache.rb +76 -0
  266. data/lib/expressir/express/checker.rb +812 -0
  267. data/lib/expressir/express/concatenator.rb +79 -0
  268. data/lib/expressir/express/core.rb +95 -0
  269. data/lib/expressir/express/error.rb +124 -0
  270. data/lib/expressir/express/formatter.rb +149 -0
  271. data/lib/expressir/express/formatters/data_types_formatter.rb +362 -0
  272. data/lib/expressir/express/formatters/declarations_formatter.rb +753 -0
  273. data/lib/expressir/express/formatters/expressions_formatter.rb +180 -0
  274. data/lib/expressir/express/formatters/literals_formatter.rb +55 -0
  275. data/lib/expressir/express/formatters/references_formatter.rb +53 -0
  276. data/lib/expressir/express/formatters/remark_formatter.rb +270 -0
  277. data/lib/expressir/express/formatters/remark_item_formatter.rb +28 -0
  278. data/lib/expressir/express/formatters/statements_formatter.rb +272 -0
  279. data/lib/expressir/express/formatters/supertype_expressions_formatter.rb +54 -0
  280. data/lib/expressir/express/formatters.rb +22 -0
  281. data/lib/expressir/express/grammar/parser.rb +712 -0
  282. data/lib/expressir/express/grammar.rb +11 -0
  283. data/lib/expressir/express/hyperlink_formatter.rb +35 -0
  284. data/lib/expressir/express/interface_dot.rb +105 -0
  285. data/lib/expressir/express/lazy_repository.rb +211 -0
  286. data/lib/expressir/express/line_map.rb +48 -0
  287. data/lib/expressir/express/listing.rb +174 -0
  288. data/lib/expressir/express/model_traversal.rb +42 -0
  289. data/lib/expressir/express/model_visitor.rb +25 -0
  290. data/lib/expressir/express/node_position_index.rb +342 -0
  291. data/lib/expressir/express/parallel_files.rb +229 -0
  292. data/lib/expressir/express/parser.rb +523 -0
  293. data/lib/expressir/express/pretty_formatter.rb +586 -0
  294. data/lib/expressir/express/pretty_gate.rb +61 -0
  295. data/lib/expressir/express/refs_overlay.rb +47 -0
  296. data/lib/expressir/express/remark_attacher.rb +1098 -0
  297. data/lib/expressir/express/remark_overlay.rb +86 -0
  298. data/lib/expressir/express/remark_scanner.rb +245 -0
  299. data/lib/expressir/express/resolve_references_model_visitor.rb +29 -0
  300. data/lib/expressir/express/schema_block_scanner.rb +137 -0
  301. data/lib/expressir/express/schema_head_formatter.rb +22 -0
  302. data/lib/expressir/express/schema_plain_source_formatter.rb +11 -0
  303. data/lib/expressir/express/schema_source_formatter.rb +15 -0
  304. data/lib/expressir/express/scope_resolver.rb +223 -0
  305. data/lib/expressir/express/self_schema_reference.rb +126 -0
  306. data/lib/expressir/express/shtolo.rb +679 -0
  307. data/lib/expressir/express/smrl_xml.rb +125 -0
  308. data/lib/expressir/express/source_formatter.rb +15 -0
  309. data/lib/expressir/express/streaming_builder.rb +435 -0
  310. data/lib/expressir/express/xsd.rb +259 -0
  311. data/lib/expressir/express.rb +53 -0
  312. data/lib/expressir/liquid.rb +1 -0
  313. data/lib/expressir/manifest/resolver.rb +210 -0
  314. data/lib/expressir/manifest/validator.rb +192 -0
  315. data/lib/expressir/manifest.rb +6 -0
  316. data/lib/expressir/mapping/refpath.rb +857 -0
  317. data/lib/expressir/mapping.rb +253 -0
  318. data/lib/expressir/model/cache.rb +18 -0
  319. data/lib/expressir/model/concerns.rb +42 -0
  320. data/lib/expressir/model/data_types/aggregate.rb +26 -0
  321. data/lib/expressir/model/data_types/array.rb +25 -0
  322. data/lib/expressir/model/data_types/bag.rb +21 -0
  323. data/lib/expressir/model/data_types/binary.rb +19 -0
  324. data/lib/expressir/model/data_types/boolean.rb +15 -0
  325. data/lib/expressir/model/data_types/enumeration.rb +21 -0
  326. data/lib/expressir/model/data_types/enumeration_item.rb +24 -0
  327. data/lib/expressir/model/data_types/generic.rb +24 -0
  328. data/lib/expressir/model/data_types/generic_entity.rb +24 -0
  329. data/lib/expressir/model/data_types/integer.rb +15 -0
  330. data/lib/expressir/model/data_types/list.rb +23 -0
  331. data/lib/expressir/model/data_types/logical.rb +15 -0
  332. data/lib/expressir/model/data_types/number.rb +15 -0
  333. data/lib/expressir/model/data_types/real.rb +17 -0
  334. data/lib/expressir/model/data_types/select.rb +23 -0
  335. data/lib/expressir/model/data_types/set.rb +21 -0
  336. data/lib/expressir/model/data_types/string.rb +19 -0
  337. data/lib/expressir/model/data_types.rb +25 -0
  338. data/lib/expressir/model/declarations/attribute.rb +38 -0
  339. data/lib/expressir/model/declarations/constant.rb +28 -0
  340. data/lib/expressir/model/declarations/derived_attribute.rb +25 -0
  341. data/lib/expressir/model/declarations/entity.rb +62 -0
  342. data/lib/expressir/model/declarations/function.rb +59 -0
  343. data/lib/expressir/model/declarations/informal_proposition_rule.rb +25 -0
  344. data/lib/expressir/model/declarations/interface.rb +37 -0
  345. data/lib/expressir/model/declarations/interface_item.rb +21 -0
  346. data/lib/expressir/model/declarations/interfaced_item.rb +33 -0
  347. data/lib/expressir/model/declarations/inverse_attribute.rb +25 -0
  348. data/lib/expressir/model/declarations/parameter.rb +28 -0
  349. data/lib/expressir/model/declarations/procedure.rb +57 -0
  350. data/lib/expressir/model/declarations/remark_item.rb +21 -0
  351. data/lib/expressir/model/declarations/rule.rb +66 -0
  352. data/lib/expressir/model/declarations/schema.rb +215 -0
  353. data/lib/expressir/model/declarations/schema_version.rb +19 -0
  354. data/lib/expressir/model/declarations/schema_version_item.rb +19 -0
  355. data/lib/expressir/model/declarations/subtype_constraint.rb +32 -0
  356. data/lib/expressir/model/declarations/type.rb +45 -0
  357. data/lib/expressir/model/declarations/unique_rule.rb +26 -0
  358. data/lib/expressir/model/declarations/variable.rb +28 -0
  359. data/lib/expressir/model/declarations/where_rule.rb +26 -0
  360. data/lib/expressir/model/declarations.rb +31 -0
  361. data/lib/expressir/model/dependency_resolver.rb +268 -0
  362. data/lib/expressir/model/exp_file.rb +42 -0
  363. data/lib/expressir/model/expressions/aggregate_initializer.rb +18 -0
  364. data/lib/expressir/model/expressions/aggregate_initializer_item.rb +20 -0
  365. data/lib/expressir/model/expressions/binary_expression.rb +59 -0
  366. data/lib/expressir/model/expressions/entity_constructor.rb +20 -0
  367. data/lib/expressir/model/expressions/function_call.rb +20 -0
  368. data/lib/expressir/model/expressions/interval.rb +29 -0
  369. data/lib/expressir/model/expressions/query_expression.rb +31 -0
  370. data/lib/expressir/model/expressions/unary_expression.rb +25 -0
  371. data/lib/expressir/model/expressions.rb +18 -0
  372. data/lib/expressir/model/identifier.rb +26 -0
  373. data/lib/expressir/model/indexes/entity_index.rb +103 -0
  374. data/lib/expressir/model/indexes/item_graph.rb +185 -0
  375. data/lib/expressir/model/indexes/reference_index.rb +148 -0
  376. data/lib/expressir/model/indexes/type_index.rb +149 -0
  377. data/lib/expressir/model/indexes.rb +12 -0
  378. data/lib/expressir/model/interface_validator.rb +384 -0
  379. data/lib/expressir/model/literals/binary.rb +17 -0
  380. data/lib/expressir/model/literals/integer.rb +17 -0
  381. data/lib/expressir/model/literals/logical.rb +21 -0
  382. data/lib/expressir/model/literals/real.rb +17 -0
  383. data/lib/expressir/model/literals/string.rb +20 -0
  384. data/lib/expressir/model/literals.rb +13 -0
  385. data/lib/expressir/model/model_element.rb +377 -0
  386. data/lib/expressir/model/references/attribute_reference.rb +19 -0
  387. data/lib/expressir/model/references/group_reference.rb +19 -0
  388. data/lib/expressir/model/references/index_reference.rb +23 -0
  389. data/lib/expressir/model/references/simple_reference.rb +21 -0
  390. data/lib/expressir/model/references.rb +12 -0
  391. data/lib/expressir/model/remark_format.rb +17 -0
  392. data/lib/expressir/model/remark_info.rb +92 -0
  393. data/lib/expressir/model/remark_placement.rb +42 -0
  394. data/lib/expressir/model/repository.rb +483 -0
  395. data/lib/expressir/model/repository_validator.rb +293 -0
  396. data/lib/expressir/model/search_engine.rb +551 -0
  397. data/lib/expressir/model/statements/alias.rb +31 -0
  398. data/lib/expressir/model/statements/assignment.rb +23 -0
  399. data/lib/expressir/model/statements/case.rb +42 -0
  400. data/lib/expressir/model/statements/case_action.rb +20 -0
  401. data/lib/expressir/model/statements/compound.rb +21 -0
  402. data/lib/expressir/model/statements/escape.rb +18 -0
  403. data/lib/expressir/model/statements/if.rb +26 -0
  404. data/lib/expressir/model/statements/null.rb +18 -0
  405. data/lib/expressir/model/statements/procedure_call.rb +22 -0
  406. data/lib/expressir/model/statements/repeat.rb +40 -0
  407. data/lib/expressir/model/statements/return.rb +20 -0
  408. data/lib/expressir/model/statements/skip.rb +18 -0
  409. data/lib/expressir/model/statements.rb +20 -0
  410. data/lib/expressir/model/supertype_expressions/binary_supertype_expression.rb +25 -0
  411. data/lib/expressir/model/supertype_expressions/oneof_supertype_expression.rb +17 -0
  412. data/lib/expressir/model/supertype_expressions.rb +12 -0
  413. data/lib/expressir/model.rb +39 -0
  414. data/lib/expressir/package/builder.rb +231 -0
  415. data/lib/expressir/package/metadata.rb +79 -0
  416. data/lib/expressir/package/reader.rb +149 -0
  417. data/lib/expressir/package.rb +8 -0
  418. data/lib/expressir/schema_manifest.rb +159 -0
  419. data/lib/expressir/schema_manifest_entry.rb +15 -0
  420. data/lib/expressir/version.rb +8 -0
  421. data/lib/expressir.rb +94 -0
  422. data/lib/tasks/verify_remarks.rake +16 -0
  423. data/rakelib/native.rake +24 -0
  424. data/rakelib/oracle.rake +52 -0
  425. data/rakelib/verify_remarks.rake +16 -0
  426. metadata +692 -0
@@ -0,0 +1,45 @@
1
+ # TODO.max-perf/08 — Instrument and optimize Builder.build internals
2
+
3
+ ## Context
4
+
5
+ TODO.max-perf/06 established the split (CPU-time, parsanol 1.3.30, 4 mid-size
6
+ SRL schemas): native parse 50%, `Builder.build_with_remarks` 48%, references
7
+ 2%; within the builder, model build is 1.12s vs remark attachment 0.06s.
8
+ `fast_convert_keys`/`cached_snake_case` are already single-pass and
9
+ allocation-avoiding. The unmeasured stages inside `Builder.build`:
10
+
11
+ 1. `builder.call(snake_data)` — registered handler → lutaml-model
12
+ instantiation (thousands of attribute setters per schema)
13
+ 2. `attach_source_info` → `extract_source_info` → `find_slice` — a
14
+ depth-capped recursive scan of each node's data subtree; nested data can
15
+ be re-scanned by ancestors, which is quadratic-ish on deep nesting
16
+ 3. The `when Array` recursion
17
+
18
+ ## Work
19
+
20
+ Findings: fast_convert_keys ran 172k–266k times per schema (~60 calls per
21
+ model node): build() descends into subtrees the parent's deep conversion
22
+ already scanned, so every subtree was re-scanned once per ancestor level.
23
+ attach_source_info is negligible (0.02–0.07s instrumented upper bound);
24
+ the remaining unoptimized share is lutaml-model instantiation inside
25
+ builder.call (upstream territory).
26
+
27
+ - [x] Instrumented (counters + timers) over topology / presentation /
28
+ measure / action schemas; split recorded above
29
+ - [x] Implemented the evidenced win: fast_convert_keys marks scanned or
30
+ converted Hash/Array containers with an invisible ivar and skips
31
+ them on re-visits
32
+ - [x] Before/after (CPU-time, warm): topology 0.42→0.25s (−40%),
33
+ presentation 0.27→0.22s (−19%), measure 0.43→0.30s (−30%),
34
+ action 0.11→0.06s (−48%)
35
+ - [x] Full suite green (1551 examples) including golden-file parser specs;
36
+ to_hash equality verified against pre-change output on all four
37
+ schemas
38
+
39
+ - [x] Handler-level audit (TODO 14 overlap): `expression` 25.8% / `syntax` 15.4% of builder self-CPU — structural dispatch cost (per-level key cascades + build_node wrapper-hash allocations) spread across ~20k nodes; no single fixable waste. Architectural cure = skip the intermediate Hash AST via parsanol's stable parse_with_builder (blocked upstream, TODO 02 / parsanol#59).
40
+
41
+ ## Acceptance
42
+
43
+ Numbers table for the three stages; either a measured improvement merged or
44
+ the cost definitively attributed to lutaml-model instantiation (which moves
45
+ the next step upstream to lutaml-model).
@@ -0,0 +1,25 @@
1
+ # TODO.max-perf/09 — Grammar cold-start cost and disk cache
2
+
3
+ ## Context
4
+
5
+ Every process boot builds the parsanol grammar and serializes ~2,300 atoms to
6
+ JSON (`Grammar::Parser.cached_parser` / `cached_grammar_json`) before the
7
+ first parse. Short-lived processes (CLI invocations, tests) pay this on every
8
+ run. parsanol-ruby#59 tracks upstream grammar-to-code compilation; an
9
+ expressir-side JSON disk cache would help regardless.
10
+
11
+ ## Work
12
+
13
+ Measured (fresh process, 3 runs): boot (require expressir) 161–181ms,
14
+ grammar build ~28ms, JSON serialization ~30ms (83KB), first parse ~25ms.
15
+ Grammar-related cold cost is ~58ms — below the 100ms bar; a disk cache
16
+ would save ~50ms per process at the cost of temp-file staleness machinery.
17
+
18
+ - [x] Cold-start measured and recorded (above)
19
+ - [x] Documented as negligible — no cache implemented (the larger cold
20
+ cost is the 161ms require boot, a separate lazy-loading concern)
21
+
22
+ ## Acceptance
23
+
24
+ Cold-start number recorded; cache implemented with specs if the number
25
+ justifies it, otherwise documented as negligible.
@@ -0,0 +1,23 @@
1
+ # TODO.max-perf/10 — Parser facade API hygiene
2
+
3
+ ## Context
4
+
5
+ The `Parser` facade accumulated internal helpers with public visibility
6
+ during the parallel work: `parse_files_sequentially` and `build_repository`
7
+ are implementation details of `from_files`. `Commands::Coverage` defines a
8
+ `DEFAULT_MAX_PROCESSES` constant that duplicates
9
+ `ParallelFiles::DEFAULT_MAX_PROCESSES` (the CLI option default already
10
+ references the latter directly) — a DRY violation with a dead constant.
11
+
12
+ ## Work
13
+
14
+ - [x] `parse_files_sequentially` and `build_repository` are now
15
+ `private_class_method` (no external callers existed)
16
+ - [x] Dead `Coverage::DEFAULT_MAX_PROCESSES` removed; the CLI option
17
+ default references `ParallelFiles::DEFAULT_MAX_PROCESSES` directly
18
+ - [x] Rubocop clean; full suite green (1551 examples, 40s)
19
+
20
+ ## Acceptance
21
+
22
+ Facade exposes only the public contract; single source for the worker-cap
23
+ constant.
@@ -0,0 +1,25 @@
1
+ # TODO.max-perf/11 — CI green close-out
2
+
3
+ ## Context
4
+
5
+ TODO 01's final checkbox. Main carries the Windows fork guard (#353), the
6
+ engine-selection guard pinning :standard adapters on `Gem.win_platform?`
7
+ (#354), and the builder memoization (#355). yeptris mingw remains broken
8
+ upstream (leptris/yeptris#318); the guard must keep it unloaded.
9
+
10
+ ## Work
11
+
12
+ - [x] Confirmed locally via full `bundle exec rake` (CI-equivalent):
13
+ 1558 examples, 0 failures, rubocop clean across 325 files
14
+ - [x] If red: diagnose the failing job, fix in expressir if the defect is
15
+ here, escalate upstream otherwise
16
+
17
+ - [x] Root causes of the red run on 35c1f62 found and fixed: (1) rubocop offenses in the committed benchmark harness + a directive typo in another session's remark perf spec — fixed; (2) sequential-path nil-pad parity bug in from_files — fixed with regression spec; (3) strict-mode spec unguarded on fork-less platforms — guarded; (4) yeptris still loading on Windows: root cause is lutaml-model#798 (configured adapter fell through to detection, loading yeptris before any expressir pin could apply) — fixed upstream + expressir spec_helper now pins json too, with a temporary require-spy proving zero yeptris loads in the suite
18
+
19
+ - [x] Landed on main via #357 (the #356 merge had gone to its stale base
20
+ branch `feat/max-perf-todos`; cherry-picked 17aa264 onto main and
21
+ re-landed as 0576a08)
22
+
23
+ ## Acceptance
24
+
25
+ rake green on main; TODO 01's checkbox ticked.
@@ -0,0 +1,25 @@
1
+ # TODO.max-perf/12 — Require-boot profile and deferrable loads
2
+
3
+ ## Context
4
+
5
+ TODO 09 measured the cold start: grammar work is only ~58ms; the dominant
6
+ cold cost is `require "expressir"` itself at 161–181ms. Short-lived
7
+ processes (CLI, CI) pay it per invocation. expressir.rb eagerly requires
8
+ `lutaml/model` and `liquid` (for Lutaml::Model::Liquefiable); the gemspec
9
+ also pulls csv, benchmark-ips, ruby-progressbar, nokogiri-adjacent moxml,
10
+ thor — some needed only by CLI commands, not by library consumers.
11
+
12
+ ## Work
13
+
14
+ - [x] Profiled: per-gem load times via a require-timing probe in a fresh
15
+ process (results below)
16
+ - [x] Evaluated deferrals: nothing safely deferrable (results below)
17
+ - [x] Full suite green (no deferred loads introduced)
18
+
19
+ - [x] Per-gem boot profile (fresh process, 2 runs): liquid 172-207ms, lutaml/model 327-559ms (includes moxml chain), thor 19-40ms, expressir own lib ~1ms (autoloads already optimal)
20
+ - [x] Documented as not safely deferrable: liquid backs Lutaml::Model::Liquefiable used by expressir models in metanorma rendering; lutaml-model's boot is upstream (its lazy-loading is tracked there). expressir's own load adds ~1ms — nothing to defer.
21
+
22
+ ## Acceptance
23
+
24
+ Per-gem boot profile recorded; measured reduction if safely deferrable,
25
+ otherwise documented why not.
@@ -0,0 +1,26 @@
1
+ # TODO.max-perf/13 — Unit specs for the key-conversion memoization
2
+
3
+ ## Context
4
+
5
+ PR #355 optimized `Builder.fast_convert_keys` with an invisible-ivar marker
6
+ so re-visited containers are skipped. The optimization is currently covered
7
+ only indirectly by golden-file parser specs. The memoization's own contract
8
+ needs pinning so future edits cannot silently break it.
9
+
10
+ ## Work
11
+
12
+ - [x] Unit specs for the converter (renamed `AstKeyConverter.convert`):
13
+ - converts CamelCase keys at every depth; untouched data keeps object
14
+ identity
15
+ - idempotence: converted results convert to themselves
16
+ - marked containers are skipped on re-visits (marker on both paths)
17
+ - frozen input hashes do not raise
18
+ - [x] Marker invisibility: marked hashes remain == to unmarked copies and
19
+ survive Marshal round-trip without affecting content
20
+
21
+ - [x] All contract specs written against the extracted public Expressir::Express::AstKeyConverter (key conversion was promoted from a private Builder helper to its own class — MECE: converting AST keys is not building models)
22
+ - [x] Depth conversion, identity preservation, idempotence, marker set on both paths, frozen-hash safety, marker invisibility (== and Marshal), snake_case behavior — spec/expressir/express/ast_key_converter_spec.rb
23
+
24
+ ## Acceptance
25
+
26
+ Dedicated spec file green; the optimization's behavior is pinned.
@@ -0,0 +1,28 @@
1
+ # TODO.max-perf/14 — Per-handler builder.call audit
2
+
3
+ ## Context
4
+
5
+ TODO 08 attributed the remaining (post-memoization) builder cost to model
6
+ instantiation inside `builder.call` — but that was inferred, not measured
7
+ per handler. ~200 registered handlers exist (builders/*.rb); expressir-side
8
+ waste in the hottest handlers (e.g., redundant intermediate arrays, repeated
9
+ `build()` calls, hash re-shaping) would be ours to fix.
10
+
11
+ ## Work
12
+
13
+ - [x] Instrumented per handler via a self-time build wrapper (child CPU
14
+ subtracted) over representative schemas; top handlers recorded below
15
+ - [x] Inspected the top handlers' source (expression_builder.rb et al.)
16
+ - [x] No evident expressir-side waste to fix (see verdict below); the
17
+ optimization taken instead was the fast_convert_keys memoization
18
+ (merged in #355) after TODO 08's stage instrumentation
19
+ - [x] Full suite green
20
+
21
+ - [x] Handler-level instrumentation via a self-time build wrapper (child-time subtracted): 20,270 nodes, 1.58s self CPU over 3 schemas
22
+ - [x] Top handlers: expression 25.8% (3,184 calls), syntax 15.4%, entity_ref 7.7%, simple_expression 6.0%, entity_decl 5.1% — long tail of leaf handlers below 4% each
23
+ - [x] Verdict: no single expressir-side waste; `expression`'s share is structural dispatch (if/elsif key cascades + per-call build_node wrapper-hash allocations). The architectural fix is bypassing the intermediate Hash AST entirely — blocked on parsanol's stable parse_with_builder (TODO 02, parsanol#59). Recorded; not refactored blindly.
24
+
25
+ ## Acceptance
26
+
27
+ Handler-level numbers table; either a merged improvement or the cost
28
+ definitively attributed to lutaml-model constructors (upstream follow-up).
@@ -0,0 +1,76 @@
1
+ # Parity with Express Engine (eeng) — SHTOLO and algorithms
2
+
3
+ **Status: PLANNED (2026-09-21)**
4
+
5
+ ## Mission
6
+
7
+ 1. **SHTOLO** (expressir#32): SHort-TO-LOng — generate ISO 10303-11:1994
8
+ "long form EXPRESS" (one self-contained schema) from the STEPmod
9
+ multi-schema structure (edition 2: shared files, USE FROM /
10
+ REFERENCE FROM). The normative conversion rules are
11
+ **ISO 10303-11:2004 Annex G** — local sources:
12
+ `~/src/mn/iso-10303-11/sources/sections/ag-single-schema.adoc`.
13
+ 2. **Concatenation** (expressir#247, metanorma/iso-10303#539): the SHTOLO
14
+ precursor — a dependency-closure concatenation of all interfaced
15
+ schemas into one publication artifact (`.exp`), plus SMRL index.
16
+ 3. **Validation** of both, against the reference implementation:
17
+ Express Engine (`~/src/external/exp-engine-engine`, Common Lisp),
18
+ whose `qualify.sh` runs the full operation catalog over the exact
19
+ SMRL corpus we already load (`resources.lst`/`modules.lst` ==
20
+ our SRL manifests).
21
+
22
+ ## Reference implementation map
23
+
24
+ | eeng source | operation (qualify.sh) | task |
25
+ |---|---|---|
26
+ | `plugins/p11/wo-pretty.lisp` | Pretty Print `.exp` + compare | 08 |
27
+ | `plugins/p11/wo-flat.lisp` + `run-op :flat` | **Flatten to long form** | 05 |
28
+ | mode `:concatenated` (`kernel/engine.lisp`) | **Concatenate schema** | 04 |
29
+ | `plugins/p11/wo-list.lisp` | List declarations | 09 |
30
+ | `plugins/p11/wo-smrl-xml.lisp` | SMRL index XML | 09 |
31
+ | `kernel/dot-graph.lisp` | Dot interface graph | 10 |
32
+ | `kernel/check.lisp` (835 lines) | semantic checks | 07 |
33
+ | `plugins/p11/schedule-*.lisp`, `kernel/resolve-interface.lisp` | load/resolution scheduling | 06 |
34
+ | `kernel/compute-inheritance.lisp`, `supertypes.lisp`, `derive-attrs.lisp`, `inverse-attrs.lisp` | entity computations | 02/07 |
35
+
36
+ ## What we already have (expressir)
37
+
38
+ Compiled schema set (batch + EXSCS1 artifact + overlays), repository
39
+ indexes + reference graph (37.7k subtype edges, interfaces, closures),
40
+ `Schema#interfaced_items`, resolver, formatter (clean/hyperlink),
41
+ `RepositoryValidator`, Part 21 parser = **not started** (see 02).
42
+
43
+ ## Task index
44
+
45
+ 01 Annex G rule extraction · 02 eeng algorithm inventory (what we do
46
+ differently) · 03 eeng oracle harness · 04 concatenation (#247) ·
47
+ 05 SHTOLO longform flatten (#32) · 06 interface scheduling parity ·
48
+ 07 semantic checks port · 08 pretty-print round-trip gate ·
49
+ 09 SMRL index + listing · 10 interface dot graph ·
50
+ 11 import the eeng test suite (scripts/ drivers, docs/issues bugs,
51
+ SHTOLO paper, BNFs, DTDs)
52
+
53
+ ## Additional eeng assets (via 11)
54
+
55
+ `scripts/*.sh` — 14 release-suite drivers (qual/pretty/xml/dot);
56
+ `docs/issues/` — bug corpus incl. a shtolo old-vs-new differential;
57
+ `docs/` — SHTOLO paper PDF, Part 11/14/21 BNFs, XML DTDs,
58
+ interface/qualify notes; `00check.txt` — check design notes.
59
+
60
+ ## Phase 2 — full eengine parity (2026-09-22, user-directed)
61
+
62
+ | # | plan | status |
63
+ |---|---|---|
64
+ | 12 | full check-p11 catalog port | planned |
65
+ | 13 | population / Part 21 validation | planned |
66
+ | 14 | SHTOLO completion (Annex G + corpus differential) | in progress |
67
+ | 15 | Part 28 XML + SMRL index | planned |
68
+ | 16 | schema compare & patch | planned |
69
+ | 17 | official test-case incorporation | standing structure |
70
+
71
+ Execution order: 14 (finish Shtolo) → 12 (checks) → 15 (XML index)
72
+ → 16 (compare) → 13 (population, biggest). Each slice ships a PR and
73
+ a patch release as it lands ("release as we go"). Reference repos:
74
+ `~/src/external/exp-engine-engine` (oracle), `~/src/mn/iso-10303-11`
75
+ (normative text), `~/src/mn/iso-10303-detached-docs` (Parts 21/28
76
+ sources and fixtures).
@@ -0,0 +1,41 @@
1
+ # 01 — Annex G rule extraction (executable spec for SHTOLO)
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ Turn `~/src/mn/iso-10303-11/sources/sections/ag-single-schema.adoc`
8
+ (1957 lines) into an executable checklist: every conversion rule with
9
+ an identifier, inputs, outputs, and a fixture that exercises it. This
10
+ is the contract both for our implementation (05) and for judging
11
+ eeng's coverage.
12
+
13
+ ## Source structure (verified)
14
+
15
+ - Fundamentals: multi-schema → intermediate single "artifact" schema →
16
+ rewrite using 1994-only constructs; subtype/supertype graph pruning;
17
+ interface knowledge (visibility + instantiability) becomes rules;
18
+ total_over constraints become rules.
19
+ - **Name munging**: name-clash policy; identifiers-as-strings.
20
+ - **Stage 1 (multi → intermediate)**: primary population (per
21
+ interfaced item: artifact-schema rule + function capturing
22
+ visibility/instantiability), secondary population, **prune** rules
23
+ (rules whose parameters are not all visible are deleted, etc.).
24
+ - Stage 2 (intermediate → 1994 longform): rewrite of constructs not in
25
+ the 1994 edition.
26
+
27
+ ## Steps
28
+
29
+ 1. Read the full annex; number every rule `G.<n>` with a one-line
30
+ statement, inputs/outputs, and stage.
31
+ 2. Build a fixture matrix: for each rule, the smallest STEPmod schema
32
+ pair(s) that trigger it (mine the SMRL corpus for real cases).
33
+ 3. Cross-reference which rules eeng implements (grep its sources for
34
+ the mechanisms) and mark: implemented-in-eeng / our-gap / both-gap.
35
+ 4. Encode the checklist as a spec document under
36
+ `spec/fixtures/shtolo/rules.md` (or adoc) that 05's suite cites.
37
+
38
+ ## Acceptance
39
+
40
+ Every rule in Annex G has an id, a fixture, and an eeng-coverage flag;
41
+ no rule left "uncategorized".
@@ -0,0 +1,8 @@
1
+ # 02 — eeng algorithm inventory: what they do vs what we do differently
2
+
3
+ **Status: SHIPPED (2026-09-21) — see parity-matrix.md for the audit and
4
+ expressir#373 for the bug the audit surfaced.**
5
+
6
+ See `parity-matrix.md` (same directory) for the full algorithm-by-
7
+ algorithm comparison: resolution scoping, interface scheduling,
8
+ entity computations, writers, and the concrete expressir gaps found.
@@ -0,0 +1,33 @@
1
+ # 03 — Build and run eeng as a differential oracle
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ A working `eengine` binary (SBCL build) we can run over the SMRL corpus
8
+ to produce reference outputs (pretty, flat, concatenated, list,
9
+ smrl-xml, dot, checks) for differential validation of our ports.
10
+
11
+ ## Steps
12
+
13
+ 1. Install/locate SBCL; build: `sbcl --load sbcl/compile` then
14
+ `sbcl --load sbcl/deliver` (per README.txt) → `eengine`.
15
+ 2. Reproduce `qualify.sh` flow for a handful of modules from
16
+ `qualify-mod.txt` (their lists == our SRL corpus names).
17
+ 3. Golden corpus: run all qualify operations for the full
18
+ `resources.lst` + `modules.lst`; commit distilled outputs (not the
19
+ whole tree) under `spec/fixtures/eeng-oracle/` (LFS-sized check
20
+ first).
21
+ 4. Script `script/eeng-oracle` (or Rake task) exposing one-command
22
+ regeneration; document known eeng bugs/limitations found while
23
+ driving it (feeds 01's coverage flags).
24
+
25
+ ## Acceptance
26
+
27
+ One command regenerates the golden corpus; at least arm/mim pretty +
28
+ flat + concatenated outputs exist for 10 modules and 10 resources.
29
+
30
+ ## Tests
31
+
32
+ Refreshes the expected outputs consumed by 11's imported specs; the
33
+ qual-* drivers are the oracle's entry points.
@@ -0,0 +1,44 @@
1
+ # 04 — Concatenated EXPRESS artifact (expressir#247)
2
+
3
+ **Status: SHIPPED (2026-09-21)** — `Expressir::Express::Concatenator`
4
+ (closure + writer + one-call `call`), eeng-semantics ordering and
5
+ section format (alphabetical, `-- NAME (file)` separators, verbatim
6
+ source copies); specs: synthetic closure/round-trip + ENV-gated eeng
7
+ differential (byte-identical sections vs `--concat_schema` for
8
+ description_assignment ARM). Golden oracle outputs committed for ARM
9
+ and MIM. During this stage the small-input USE FROM parse bug was
10
+ found and reported as expressir#373.
11
+
12
+ ## Goal
13
+
14
+ `expressir` command producing a **concatenated** `.exp`: the root
15
+ schema followed by every USE FROM / REFERENCE FROM dependency
16
+ (non-circular closure), each as plain EXPRESS (no remarks), with an
17
+ SRL publication header listing the schemas. This is the SHTOLO
18
+ precursor and a WG12 qualification/provenance artifact.
19
+
20
+ ## Reference
21
+
22
+ - eeng mode `(:arm/:mim :concatenated)` — `kernel/engine.lisp` mode
23
+ table, `plugins/p11/top-level.lisp` (concatenated handling +
24
+ `arm_concatenated.exp` outputs in qualify.sh), SMRL index hook at
25
+ wo-smrl-xml.
26
+ - Our graph: dependency closure via `Repository#reference_index`
27
+ (interface edges) + topological, cycle-safe ordering already
28
+ demonstrated by the item-graph work.
29
+
30
+ ## Design sketch
31
+
32
+ - `Expressir::Express::Concatenator.call(repository, root_schema)` →
33
+ model-level ordered schema list (respect eeng's ordering rules —
34
+ read their code in 02 first).
35
+ - Writer: plain schemas (no remarks) + banner comment enumerating
36
+ sources (schema name, file, copyright per SRL conventions).
37
+ - CLI: `expressir concatenate <manifest/root.exp> -o out.exp`.
38
+
39
+ ## Acceptance
40
+
41
+ - Output parses back with our parser (self round-trip).
42
+ - Diff vs eeng `arm_concatenated.exp`/`mim_concatenated.exp` for the
43
+ 03 golden set: schema set + order identical (formatting differences
44
+ documented, ideally zero after 08).
@@ -0,0 +1,53 @@
1
+ # 05 — SHTOLO longform flatten (expressir#32) — the core port
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ Port eeng's flatten (Annex G) into expressir: convert the STEPmod
8
+ multi-schema web into ONE ISO 10303-11:1994 longform schema — no
9
+ interfaces, no name clashes, interface visibility/instantiability
10
+ preserved as rules, pruned unreachable declarations.
11
+
12
+ ## Sources to port
13
+
14
+ - **Spec**: `ag-single-schema.adoc` (see 01's rule ids).
15
+ - **eeng**: `run-op (:flat)` + `print-flatten`
16
+ (`plugins/p11/top-level.lisp:847`), writer `plugins/p11/wo-flat.lisp`
17
+ (373 lines; note: eeng's flat output keeps structure and lowercases
18
+ identifiers — verify how much of Annex G eeng actually implements
19
+ before treating it as gospel; where eeng and the standard disagree,
20
+ the standard wins and the difference goes into 02's matrix).
21
+ - **Model surgery**: `add-declaration.lisp`, `remove-declaration.lisp`,
22
+ `copy-object.lisp`, `cleanup-interface.lisp`.
23
+
24
+ ## Design sketch
25
+
26
+ 1. Model-level `Expressir::Express::Flattener` operating on the
27
+ resolved repository (compiled set warm model): build the artifact
28
+ schema per Annex G — primary population (per interfaced item:
29
+ visibility/instantiability rules + generated function), secondary
30
+ population, name munging (clash policy), prune pass (rules whose
31
+ parameters are not all visible; unreachable subtype branches),
32
+ total_over → rules, then the 1994-construct rewrite.
33
+ 2. Writer: reuse the formatter with a "flat/1994" encoder profile
34
+ (no interfaces section; artifact rules emitted per Annex G).
35
+ 3. Validation gates:
36
+ - Annex G rule fixtures from 01 (each rule exercised).
37
+ - Differential: flatten output vs eeng `.flat` over the 03 golden
38
+ corpus; divergences classified standard-vs-eeng.
39
+ - Output parses as 1994 (no edition-2-only constructs — the
40
+ grammar-level edition profile check).
41
+ 4. SMRL-scale run: all 132 SRL resources + module arm/mim set; memory
42
+ via the compiled-set warm model.
43
+
44
+ ## Acceptance
45
+
46
+ - All Annex G fixtures green; SMRL corpus flatten completes with zero
47
+ unexplained diffs vs the 1994 profile.
48
+ - `expressir shtolo <manifest> -o longform.exp` ships.
49
+
50
+ ## Tests
51
+
52
+ bug39's shtolo old-vs-new outputs (see 11) are an additional
53
+ differential: eeng shtolo vs stepcode exppp vs us.
@@ -0,0 +1,47 @@
1
+ # 06 — Interface loading/resolution parity with eeng scheduling
2
+
3
+ **Status: SHIPPED (2026-09-21) — the resolution-side fix.**
4
+ `Schema#interfaced_items` is now repository-aware (foreign schemas
5
+ found through the parent chain into the enclosing repository) and
6
+ transitively USE-aware (items USEd by the foreign schema are visible,
7
+ unfiltered per the implicit-interfacing rule, cycle-guarded by a
8
+ visited-schema set). Probe results before/after: direct cross-file
9
+ item ref resolved; transitive-USE item resolved; interface cycle
10
+ terminates. Spec: `interface_visibility_spec.rb` (3 examples);
11
+ express suite 321/0. Remaining: unresolved-resource diagnostics and
12
+ USE-vs-REFERENCE error severity (eeng hard-errors; we warn) — folded
13
+ into 07's checker.
14
+
15
+ ## Goal
16
+
17
+ Match eeng's multi-schema load semantics: which declarations are
18
+ visible where, transitive USE FROM (implicit interfacing), rename
19
+ (AS) handling, and error cases ("Resource '~A' not found").
20
+
21
+ ## Differences already known
22
+
23
+ - eeng `resolve-interface.lisp`: USE FROM resolves **only**
24
+ entity+type; REFERENCE FROM resolves constant/entity/function/
25
+ procedure/type; unresolved resource = hard error. Our resolver
26
+ stamps base_path but never errors on unresolved interfaces.
27
+ - eeng runs interface **scheduling** (schedule-interfaces.lisp) so a
28
+ USE FROM can pull schemas already loaded in the session
29
+ (schemata/schemata1/schemata2) — closest to our repository + graph.
30
+ - Implicit interfacing (11-interface_specification.adoc): items
31
+ visible through USE FROM chains; our `interfaced_items` is
32
+ single-hop — verify transitive semantics.
33
+
34
+ ## Steps
35
+
36
+ 1. From 02's matrix, list each semantic difference with a fixture
37
+ (rename chains, transitively used items, interface cycles,
38
+ missing resource).
39
+ 2. Fix expressir resolution where we are wrong (spec wins), or
40
+ document deliberate divergence.
41
+ 3. Add unresolved-interface reporting (warning set) — SHTOLO's prune
42
+ stage consumes it.
43
+
44
+ ## Acceptance
45
+
46
+ Fixture suite covering rename/transitive/cycle/missing passes on both
47
+ paths (Ruby parser + core path byte-parity preserved).
@@ -0,0 +1,35 @@
1
+ # 07 — Port eeng semantic checks (kernel/check.lisp)
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ A `check` operation in expressir that walks a schema/repository and
8
+ reports the same diagnostics eeng's check-p11 passes produce — the
9
+ validation half of the SHTOLO mission (inputs must be sound before
10
+ flattening).
11
+
12
+ ## Source catalog (check.lisp, 835 lines)
13
+
14
+ Per-node `check-p11` methods: schema, use-from/reference-from,
15
+ constant, local, entity (explicit/derive/inverse attributes),
16
+ function/procedure/rule parameters, subtype_constraint, type, plus
17
+ expression/statement checks. Also `check-notes.lisp` (note
18
+ collection) and `compare.lisp` (schema comparison — two loaded
19
+ sessions).
20
+
21
+ ## Design
22
+
23
+ - `Expressir::Express::Checker` modeled on our visitor infra; each
24
+ eeng check becomes one rule with the eeng note id preserved for
25
+ cross-referencing oracle output.
26
+ - Wire into `Repository#validate` (RepositoryValidator today is
27
+ interface-level only).
28
+ - Run order: checks → report (eeng note ids + our explanation) →
29
+ nonzero exit on error-class notes (eeng error severity model).
30
+
31
+ ## Acceptance
32
+
33
+ Golden-diff check reports vs eeng for the 03 corpus subset; any
34
+ divergent verdict justified against Part 11 clause text
35
+ (`~/src/mn/iso-10303-11/sources/sections/`).
@@ -0,0 +1,27 @@
1
+ # 08 — Pretty-printer parity gate (eeng wo-pretty)
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ qualify.sh's core self-check, adopted for us: `.exp` → pretty `.exp`
8
+ → re-parse → identical model; plus diff vs eeng's `-pretty.exp` over
9
+ the golden corpus so SHTOLO outputs (04/05) can target eeng-stable
10
+ formatting.
11
+
12
+ ## Steps
13
+
14
+ 1. Read `plugins/p11/wo-pretty.lisp` encoder (spacing, casing,
15
+ remark policy, schema ordering) → document deltas vs our
16
+ `Formatter` (clean mode).
17
+ 2. Add `parser → formatter → parser` identity spec across the SMRL
18
+ corpus (model-level to_hash equality, not bytes, first; byte mode
19
+ only where eeng parity is wanted).
20
+ 3. Byte-diff our pretty vs eeng pretty on the 03 corpus; decide
21
+ adopt-eeng-format vs keep-ours (default: keep ours for rendering,
22
+ eeng profile as an encoder option for SHTOLO artifacts).
23
+
24
+ ## Acceptance
25
+
26
+ Corpus-wide round-trip identity green; eeng-profile diffs either zero
27
+ or explained line-by-line.
@@ -0,0 +1,22 @@
1
+ # 09 — SMRL index XML + declaration listing (wo-list, wo-smrl-xml)
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ The SRL publication byproducts eeng generates: declaration listings
8
+ (`-list.txt`) and the SMRL index XML for longform/concatenated files
9
+ (`<concatenated_express_file_content_list>`), emitted from the same
10
+ model that powers 04/05.
11
+
12
+ ## Sources
13
+
14
+ `plugins/p11/wo-list.lisp`, `wo-smrl-xml.lisp` (element names:
15
+ concatenated_express_file_content_list, per-schema content entries),
16
+ top-level.lisp:748-793 (when the index is generated: XML output +
17
+ longform/concatenated modes).
18
+
19
+ ## Acceptance
20
+
21
+ Index/list output for the 03 golden set structurally identical
22
+ (element-for-element) to eeng's; wired into the same CLI as 04.
@@ -0,0 +1,21 @@
1
+ # 10 — Interface dependency dot graph (kernel/dot-graph.lisp)
2
+
3
+ **Status: NOT STARTED**
4
+
5
+ ## Goal
6
+
7
+ Emit GraphViz dot for schema interface dependencies exactly like eeng
8
+ (qualify.sh "Generate Dot File (interface)"), from our compiled
9
+ reference graph (already computes interface edges + closures over the
10
+ full SMRL).
11
+
12
+ ## Notes
13
+
14
+ eeng's dot generation honors the concatenated mode (root-schema
15
+ subgraph); replicate node/edge/styling choices so WG12 visual
16
+ diffing stays stable. Trivial once 02 documents their exact output
17
+ shape — deliberately last in the queue.
18
+
19
+ ## Acceptance
20
+
21
+ Byte-comparable dot output vs eeng for arm/mim of the golden modules.