expressir 1.2.10-aarch64-linux → 2.4.29-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.cargo/config.toml +3 -0
- data/.github/scripts/test_installed_gem.rb +34 -0
- data/.github/workflows/codeql.yml +35 -0
- data/.github/workflows/docs.yml +99 -0
- data/.github/workflows/links.yml +100 -0
- data/.github/workflows/native-gems.yml +201 -0
- data/.github/workflows/rake.yml +10 -296
- data/.github/workflows/release.yml +26 -114
- data/.github/workflows/rust-ext.yml +41 -0
- data/.github/workflows/stress.yml +36 -0
- data/.github/workflows/validate_schemas.yml +48 -0
- data/.github/workflows/verify_remarks.yml +33 -0
- data/.gitignore +33 -1
- data/.rubocop.yml +11 -11
- data/.rubocop_todo.yml +297 -0
- data/CHANGELOG.md +159 -0
- data/Gemfile +15 -1
- data/README.adoc +2675 -47
- data/Rakefile +34 -10
- data/TODO.bugs/01-stale-transformer-autoload.md +39 -0
- data/TODO.bugs/02-parser-class-instance-vars.md +36 -0
- data/TODO.bugs/03-builder-mutable-state.md +43 -0
- data/TODO.bugs/04-formatter-public-send-dispatch.md +53 -0
- data/TODO.bugs/05-anonymous-formatter-subclass.md +45 -0
- data/TODO.bugs/06-collection-registry-single-source.md +53 -0
- data/TODO.bugs/07-require-relative-cleanup.md +42 -0
- data/TODO.bugs/08-require-expressir-in-commands.md +34 -0
- data/TODO.bugs/09-parser-split.md +53 -0
- data/TODO.bugs/10-to-s-override.md +42 -0
- data/TODO.bugs/11-parser-class-variables.md +39 -0
- data/TODO.bugs/12-marker-modules-vs-registry.md +64 -0
- data/TODO.bugs/13-string-literal-scanner-limitation.md +52 -0
- data/TODO.bugs/14-model-formatting-leak.md +30 -0
- data/TODO.bugs/15-expression-children-macro.md +27 -0
- data/TODO.bugs/16-pretty-formatter-duplication.md +28 -0
- data/TODO.bugs/17-snake-case-cache-mutable-constant.md +28 -0
- data/TODO.bugs/18-const-get-private-constants.md +30 -0
- data/TODO.bugs/19-format-methods-public.md +22 -0
- data/TODO.bugs/20-coverage-nested-entities-dedup.md +20 -0
- data/TODO.bugs/21-operator-tokens-secondary-dispatch.md +21 -0
- data/TODO.bugs/22-builder-fast-path-wrappers.md +32 -0
- data/TODO.bugs/23-coverage-inverse-maps.md +21 -0
- data/TODO.bugs/24-streaming-builder-complexity.md +19 -0
- data/TODO.bugs/25-debug-puts-in-production.md +21 -0
- data/TODO.bugs/26-generic-entity-children-misplaced.md +21 -0
- data/TODO.bugs/27-package-build-god-method.md +19 -0
- data/TODO.bugs/28-package-god-class.md +30 -0
- data/TODO.bugs/29-validate-ascii-god-class.md +24 -0
- data/TODO.bugs/30-unicode-map-extraction.md +19 -0
- data/TODO.bugs/README.md +43 -0
- data/TODO.max-perf/01-restore-ci-green.md +36 -0
- data/TODO.max-perf/02-streaming-parse-path.md +31 -0
- data/TODO.max-perf/03-cli-parallel-opt-in.md +27 -0
- data/TODO.max-perf/04-benchmark-harness.md +28 -0
- data/TODO.max-perf/05-parallel-fidelity-specs.md +22 -0
- data/TODO.max-perf/06-builder-cpu-audit.md +41 -0
- data/TODO.max-perf/07-upstream-parsanol-roadmap.md +27 -0
- data/TODO.max-perf/08-builder-build-perf.md +45 -0
- data/TODO.max-perf/09-grammar-cold-start.md +25 -0
- data/TODO.max-perf/10-parser-facade-hygiene.md +23 -0
- data/TODO.max-perf/11-ci-green-closeout.md +25 -0
- data/TODO.max-perf/12-require-boot-profile.md +25 -0
- data/TODO.max-perf/13-key-conversion-specs.md +26 -0
- data/TODO.max-perf/14-builder-call-handler-audit.md +28 -0
- data/TODO.parity-ee/00-overview.md +76 -0
- data/TODO.parity-ee/01-annex-g-rule-extraction.md +41 -0
- data/TODO.parity-ee/02-eeng-algorithm-inventory.md +8 -0
- data/TODO.parity-ee/03-eeng-oracle-harness.md +33 -0
- data/TODO.parity-ee/04-shtolo-concatenate.md +44 -0
- data/TODO.parity-ee/05-shtolo-longform-flatten.md +53 -0
- data/TODO.parity-ee/06-interface-scheduling-parity.md +47 -0
- data/TODO.parity-ee/07-semantic-checks-port.md +35 -0
- data/TODO.parity-ee/08-pretty-roundtrip-gate.md +27 -0
- data/TODO.parity-ee/09-smrl-index-and-listing.md +22 -0
- data/TODO.parity-ee/10-interface-dot-graph.md +21 -0
- data/TODO.parity-ee/11-import-eeng-tests.md +79 -0
- data/TODO.parity-ee/12-full-check-catalog.md +67 -0
- data/TODO.parity-ee/13-population-p21.md +54 -0
- data/TODO.parity-ee/14-shtolo-completion.md +44 -0
- data/TODO.parity-ee/15-part28-xml.md +51 -0
- data/TODO.parity-ee/16-compare-patch.md +48 -0
- data/TODO.parity-ee/17-official-tests.md +42 -0
- data/TODO.parity-ee/18-mim-mapping.md +47 -0
- data/TODO.parity-ee/19-architecture.md +97 -0
- data/TODO.parity-ee/parity-matrix.md +108 -0
- data/TODO.refpath-v2/00-overview.md +64 -0
- data/TODO.suma-improvements/00-overview.md +23 -0
- data/TODO.suma-improvements/01-prebuilt-platform-gems.md +33 -0
- data/TODO.suma-improvements/02-suma-e2e.md +39 -0
- data/TODO.suma-improvements/03-item-graph.md +22 -0
- data/TODO.suma-improvements/04-compiled-set-default.md +16 -0
- data/TODO.suma-improvements/05-validation-in-docs.md +17 -0
- data/TODO.suma-improvements/06-memory-remeasure.md +36 -0
- data/TODO.suma-improvements/07-part28-xml.md +15 -0
- data/TODO.suma-improvements/08-windows-gnu-toolchain.md +15 -0
- data/TODO.suma-improvements/09-checker-tranche3.md +13 -0
- data/TODO.suma-improvements/10-expressdoc.md +8 -0
- data/benchmark/srl_benchmark.rb +458 -0
- data/benchmark/srl_native_benchmark.rb +146 -0
- data/benchmark/srl_ruby_benchmark.rb +132 -0
- data/bin/rspec +2 -1
- data/docs/Gemfile +12 -0
- data/docs/_config.yml +141 -0
- data/docs/_guides/changes/changes-format.adoc +778 -0
- data/docs/_guides/changes/importing-eengine.adoc +898 -0
- data/docs/_guides/changes/index.adoc +396 -0
- data/docs/_guides/changes/programmatic-usage.adoc +1038 -0
- data/docs/_guides/changes/validating-changes.adoc +681 -0
- data/docs/_guides/cli/benchmark-performance.adoc +834 -0
- data/docs/_guides/cli/coverage-analysis.adoc +921 -0
- data/docs/_guides/cli/format-schemas.adoc +547 -0
- data/docs/_guides/cli/index.adoc +8 -0
- data/docs/_guides/cli/managing-changes.adoc +927 -0
- data/docs/_guides/cli/validate-ascii.adoc +645 -0
- data/docs/_guides/cli/validate-schemas.adoc +534 -0
- data/docs/_guides/formatter/formatter-architecture.adoc +401 -0
- data/docs/_guides/index.adoc +165 -0
- data/docs/_guides/ler/creating-packages.adoc +664 -0
- data/docs/_guides/ler/index.adoc +305 -0
- data/docs/_guides/ler/loading-packages.adoc +707 -0
- data/docs/_guides/ler/package-formats.adoc +748 -0
- data/docs/_guides/ler/querying-packages.adoc +826 -0
- data/docs/_guides/ler/step-packages.adoc +385 -0
- data/docs/_guides/ler/validating-packages.adoc +750 -0
- data/docs/_guides/liquid/basic-templates.adoc +813 -0
- data/docs/_guides/liquid/documentation-generation.adoc +1042 -0
- data/docs/_guides/liquid/drops-reference.adoc +829 -0
- data/docs/_guides/liquid/filters-and-tags.adoc +912 -0
- data/docs/_guides/liquid/index.adoc +468 -0
- data/docs/_guides/manifests/creating-manifests.adoc +483 -0
- data/docs/_guides/manifests/index.adoc +307 -0
- data/docs/_guides/manifests/resolving-manifests.adoc +557 -0
- data/docs/_guides/manifests/validating-manifests.adoc +713 -0
- data/docs/_guides/ruby-api/formatting-schemas.adoc +605 -0
- data/docs/_guides/ruby-api/index.adoc +257 -0
- data/docs/_guides/ruby-api/parsing-files.adoc +421 -0
- data/docs/_guides/ruby-api/search-engine.adoc +609 -0
- data/docs/_guides/ruby-api/working-with-repository.adoc +577 -0
- data/docs/_pages/data-model.adoc +665 -0
- data/docs/_pages/express-language.adoc +506 -0
- data/docs/_pages/getting-started.adoc +414 -0
- data/docs/_pages/index.adoc +116 -0
- data/docs/_pages/introduction.adoc +256 -0
- data/docs/_pages/ler-packages.adoc +837 -0
- data/docs/_pages/parsers.adoc +709 -0
- data/docs/_pages/schema-manifests.adoc +431 -0
- data/docs/_references/index.adoc +228 -0
- data/docs/_tutorials/creating-ler-package.adoc +735 -0
- data/docs/_tutorials/documentation-coverage.adoc +795 -0
- data/docs/_tutorials/formatting-schemas.adoc +89 -0
- data/docs/_tutorials/index.adoc +231 -0
- data/docs/_tutorials/liquid-templates.adoc +806 -0
- data/docs/_tutorials/parsing-your-first-schema.adoc +522 -0
- data/docs/_tutorials/querying-schemas.adoc +751 -0
- data/docs/_tutorials/working-with-multiple-schemas.adoc +676 -0
- data/docs/index.adoc +242 -0
- data/docs/lychee.toml +90 -0
- data/examples/demo_ler_usage.sh +86 -0
- data/examples/ler/README.md +111 -0
- data/examples/ler/simple_example.ler +0 -0
- data/examples/ler/simple_schema.exp +33 -0
- data/examples/ler_build.rb +75 -0
- data/examples/ler_cli.rb +79 -0
- data/examples/ler_demo_complete.rb +276 -0
- data/examples/ler_query.rb +91 -0
- data/examples/ler_query_examples.rb +305 -0
- data/examples/ler_stats.rb +81 -0
- data/examples/phase3_demo.rb +159 -0
- data/examples/query_demo_simple.rb +131 -0
- data/exe/expressir +6 -17
- data/exe/expressir-format +14 -0
- data/exe/{format-test → expressir-format-test} +8 -7
- data/expressir.gemspec +50 -37
- data/lib/expressir/3.2/expressir_core.so +0 -0
- data/lib/expressir/3.3/expressir_core.so +0 -0
- data/lib/expressir/3.4/expressir_core.so +0 -0
- data/lib/expressir/4.0/expressir_core.so +0 -0
- data/lib/expressir/benchmark.rb +310 -0
- data/lib/expressir/changes/item_change.rb +20 -0
- data/lib/expressir/changes/mapping_change.rb +16 -0
- data/lib/expressir/changes/schema_change.rb +85 -0
- data/lib/expressir/changes/version_change.rb +26 -0
- data/lib/expressir/changes.rb +9 -0
- data/lib/expressir/cli.rb +138 -11
- data/lib/expressir/commands/base.rb +25 -0
- data/lib/expressir/commands/benchmark.rb +59 -0
- data/lib/expressir/commands/benchmark_cache.rb +80 -0
- data/lib/expressir/commands/changes.rb +34 -0
- data/lib/expressir/commands/changes_import_eengine.rb +146 -0
- data/lib/expressir/commands/changes_validate.rb +63 -0
- data/lib/expressir/commands/check.rb +56 -0
- data/lib/expressir/commands/clean.rb +20 -0
- data/lib/expressir/commands/coverage.rb +458 -0
- data/lib/expressir/commands/expand.rb +26 -0
- data/lib/expressir/commands/file_violations.rb +70 -0
- data/lib/expressir/commands/fix.rb +33 -0
- data/lib/expressir/commands/flatten.rb +29 -0
- data/lib/expressir/commands/format.rb +43 -0
- data/lib/expressir/commands/manifest.rb +420 -0
- data/lib/expressir/commands/mapping_validate.rb +68 -0
- data/lib/expressir/commands/non_ascii_character.rb +49 -0
- data/lib/expressir/commands/non_ascii_violation_collection.rb +301 -0
- data/lib/expressir/commands/package.rb +1222 -0
- data/lib/expressir/commands/parity_inputs.rb +160 -0
- data/lib/expressir/commands/validate.rb +106 -0
- data/lib/expressir/commands/validate_ascii.rb +94 -0
- data/lib/expressir/commands/validate_load.rb +91 -0
- data/lib/expressir/commands/version.rb +9 -0
- data/lib/expressir/commands/xsd.rb +23 -0
- data/lib/expressir/commands.rb +30 -0
- data/lib/expressir/config.rb +40 -3
- data/lib/expressir/coverage.rb +510 -0
- data/lib/expressir/eengine/arm_compare_report.rb +23 -0
- data/lib/expressir/eengine/changes_section.rb +16 -0
- data/lib/expressir/eengine/compare_report.rb +69 -0
- data/lib/expressir/eengine/mim_compare_report.rb +23 -0
- data/lib/expressir/eengine/modified_object.rb +21 -0
- data/lib/expressir/eengine.rb +9 -0
- data/lib/expressir/errors.rb +113 -0
- data/lib/expressir/express/adoc_hyperlink_formatter.rb +30 -0
- data/lib/expressir/express/adoc_source_formatter.rb +12 -0
- data/lib/expressir/express/ast_key_converter.rb +114 -0
- data/lib/expressir/express/builder.rb +277 -0
- data/lib/expressir/express/builder_context.rb +22 -0
- data/lib/expressir/express/builder_registry.rb +397 -0
- data/lib/expressir/express/builders/attribute_decl_builder.rb +32 -0
- data/lib/expressir/express/builders/built_in_builder.rb +75 -0
- data/lib/expressir/express/builders/constant_builder.rb +108 -0
- data/lib/expressir/express/builders/declaration_builder.rb +20 -0
- data/lib/expressir/express/builders/derive_clause_builder.rb +14 -0
- data/lib/expressir/express/builders/derived_attr_builder.rb +28 -0
- data/lib/expressir/express/builders/domain_rule_builder.rb +21 -0
- data/lib/expressir/express/builders/entity_decl_builder.rb +116 -0
- data/lib/expressir/express/builders/explicit_attr_builder.rb +49 -0
- data/lib/expressir/express/builders/expression_builder.rb +416 -0
- data/lib/expressir/express/builders/function_decl_builder.rb +86 -0
- data/lib/expressir/express/builders/helpers.rb +148 -0
- data/lib/expressir/express/builders/interface_builder.rb +151 -0
- data/lib/expressir/express/builders/inverse_attr_builder.rb +43 -0
- data/lib/expressir/express/builders/inverse_attr_type_builder.rb +30 -0
- data/lib/expressir/express/builders/inverse_clause_builder.rb +14 -0
- data/lib/expressir/express/builders/literal_builder.rb +93 -0
- data/lib/expressir/express/builders/procedure_decl_builder.rb +82 -0
- data/lib/expressir/express/builders/qualifier_builder.rb +102 -0
- data/lib/expressir/express/builders/reference_builder.rb +18 -0
- data/lib/expressir/express/builders/rule_decl_builder.rb +97 -0
- data/lib/expressir/express/builders/schema_body_decl_builder.rb +18 -0
- data/lib/expressir/express/builders/schema_decl_builder.rb +56 -0
- data/lib/expressir/express/builders/schema_version_builder.rb +34 -0
- data/lib/expressir/express/builders/simple_id_builder.rb +17 -0
- data/lib/expressir/express/builders/statement_builder.rb +243 -0
- data/lib/expressir/express/builders/subtype_constraint_builder.rb +166 -0
- data/lib/expressir/express/builders/syntax_builder.rb +30 -0
- data/lib/expressir/express/builders/type_builder.rb +238 -0
- data/lib/expressir/express/builders/type_decl_builder.rb +26 -0
- data/lib/expressir/express/builders/unique_clause_builder.rb +20 -0
- data/lib/expressir/express/builders/unique_rule_builder.rb +41 -0
- data/lib/expressir/express/builders/where_clause_builder.rb +20 -0
- data/lib/expressir/express/builders.rb +55 -0
- data/lib/expressir/express/cache.rb +43 -18
- data/lib/expressir/express/checker.rb +812 -0
- data/lib/expressir/express/concatenator.rb +79 -0
- data/lib/expressir/express/core.rb +95 -0
- data/lib/expressir/express/error.rb +124 -0
- data/lib/expressir/express/formatter.rb +89 -1548
- data/lib/expressir/express/formatters/data_types_formatter.rb +362 -0
- data/lib/expressir/express/formatters/declarations_formatter.rb +753 -0
- data/lib/expressir/express/formatters/expressions_formatter.rb +180 -0
- data/lib/expressir/express/formatters/literals_formatter.rb +55 -0
- data/lib/expressir/express/formatters/references_formatter.rb +53 -0
- data/lib/expressir/express/formatters/remark_formatter.rb +270 -0
- data/lib/expressir/express/formatters/remark_item_formatter.rb +28 -0
- data/lib/expressir/express/formatters/statements_formatter.rb +272 -0
- data/lib/expressir/express/formatters/supertype_expressions_formatter.rb +54 -0
- data/lib/expressir/express/formatters.rb +22 -0
- data/lib/expressir/express/grammar/parser.rb +712 -0
- data/lib/expressir/express/grammar.rb +11 -0
- data/lib/expressir/express/hyperlink_formatter.rb +4 -5
- data/lib/expressir/express/interface_dot.rb +105 -0
- data/lib/expressir/express/lazy_repository.rb +211 -0
- data/lib/expressir/express/line_map.rb +48 -0
- data/lib/expressir/express/listing.rb +174 -0
- data/lib/expressir/express/model_traversal.rb +42 -0
- data/lib/expressir/express/model_visitor.rb +11 -10
- data/lib/expressir/express/node_position_index.rb +342 -0
- data/lib/expressir/express/parallel_files.rb +229 -0
- data/lib/expressir/express/parser.rb +501 -62
- data/lib/expressir/express/pretty_formatter.rb +586 -0
- data/lib/expressir/express/pretty_gate.rb +61 -0
- data/lib/expressir/express/refs_overlay.rb +47 -0
- data/lib/expressir/express/remark_attacher.rb +1098 -0
- data/lib/expressir/express/remark_overlay.rb +86 -0
- data/lib/expressir/express/remark_scanner.rb +245 -0
- data/lib/expressir/express/resolve_references_model_visitor.rb +7 -9
- data/lib/expressir/express/schema_block_scanner.rb +137 -0
- data/lib/expressir/express/schema_head_formatter.rb +4 -5
- data/lib/expressir/express/schema_plain_source_formatter.rb +11 -0
- data/lib/expressir/express/schema_source_formatter.rb +15 -0
- data/lib/expressir/express/scope_resolver.rb +223 -0
- data/lib/expressir/express/self_schema_reference.rb +126 -0
- data/lib/expressir/express/shtolo.rb +679 -0
- data/lib/expressir/express/smrl_xml.rb +125 -0
- data/lib/expressir/express/source_formatter.rb +15 -0
- data/lib/expressir/express/streaming_builder.rb +435 -0
- data/lib/expressir/express/xsd.rb +259 -0
- data/lib/expressir/express.rb +53 -0
- data/lib/expressir/liquid.rb +1 -0
- data/lib/expressir/manifest/resolver.rb +210 -0
- data/lib/expressir/manifest/validator.rb +192 -0
- data/lib/expressir/manifest.rb +6 -0
- data/lib/expressir/mapping/refpath.rb +857 -0
- data/lib/expressir/mapping.rb +253 -0
- data/lib/expressir/model/cache.rb +10 -9
- data/lib/expressir/model/concerns.rb +42 -0
- data/lib/expressir/model/data_types/aggregate.rb +8 -13
- data/lib/expressir/model/data_types/array.rb +15 -21
- data/lib/expressir/model/data_types/bag.rb +11 -15
- data/lib/expressir/model/data_types/binary.rb +9 -12
- data/lib/expressir/model/data_types/boolean.rb +7 -2
- data/lib/expressir/model/data_types/enumeration.rb +11 -15
- data/lib/expressir/model/data_types/enumeration_item.rb +5 -7
- data/lib/expressir/model/data_types/generic.rb +6 -8
- data/lib/expressir/model/data_types/generic_entity.rb +11 -13
- data/lib/expressir/model/data_types/integer.rb +7 -2
- data/lib/expressir/model/data_types/list.rb +13 -18
- data/lib/expressir/model/data_types/logical.rb +7 -2
- data/lib/expressir/model/data_types/number.rb +7 -2
- data/lib/expressir/model/data_types/real.rb +7 -9
- data/lib/expressir/model/data_types/select.rb +13 -18
- data/lib/expressir/model/data_types/set.rb +11 -15
- data/lib/expressir/model/data_types/string.rb +9 -12
- data/lib/expressir/model/data_types.rb +25 -0
- data/lib/expressir/model/declarations/attribute.rb +19 -28
- data/lib/expressir/model/declarations/constant.rb +10 -16
- data/lib/expressir/model/declarations/derived_attribute.rb +25 -0
- data/lib/expressir/model/declarations/entity.rb +39 -30
- data/lib/expressir/model/declarations/function.rb +32 -40
- data/lib/expressir/model/declarations/informal_proposition_rule.rb +25 -0
- data/lib/expressir/model/declarations/interface.rb +25 -16
- data/lib/expressir/model/declarations/interface_item.rb +9 -11
- data/lib/expressir/model/declarations/interfaced_item.rb +15 -19
- data/lib/expressir/model/declarations/inverse_attribute.rb +25 -0
- data/lib/expressir/model/declarations/parameter.rb +10 -16
- data/lib/expressir/model/declarations/procedure.rb +30 -37
- data/lib/expressir/model/declarations/remark_item.rb +11 -11
- data/lib/expressir/model/declarations/rule.rb +38 -43
- data/lib/expressir/model/declarations/schema.rb +162 -64
- data/lib/expressir/model/declarations/schema_version.rb +8 -11
- data/lib/expressir/model/declarations/schema_version_item.rb +8 -11
- data/lib/expressir/model/declarations/subtype_constraint.rb +14 -22
- data/lib/expressir/model/declarations/type.rb +18 -18
- data/lib/expressir/model/declarations/unique_rule.rb +8 -13
- data/lib/expressir/model/declarations/variable.rb +10 -16
- data/lib/expressir/model/declarations/where_rule.rb +8 -13
- data/lib/expressir/model/declarations.rb +31 -0
- data/lib/expressir/model/dependency_resolver.rb +268 -0
- data/lib/expressir/model/exp_file.rb +42 -0
- data/lib/expressir/model/expressions/aggregate_initializer.rb +8 -9
- data/lib/expressir/model/expressions/aggregate_initializer_item.rb +10 -12
- data/lib/expressir/model/expressions/binary_expression.rb +42 -36
- data/lib/expressir/model/expressions/entity_constructor.rb +10 -12
- data/lib/expressir/model/expressions/function_call.rb +10 -12
- data/lib/expressir/model/expressions/interval.rb +18 -23
- data/lib/expressir/model/expressions/query_expression.rb +11 -15
- data/lib/expressir/model/expressions/unary_expression.rb +13 -15
- data/lib/expressir/model/expressions.rb +18 -0
- data/lib/expressir/model/identifier.rb +18 -26
- data/lib/expressir/model/indexes/entity_index.rb +103 -0
- data/lib/expressir/model/indexes/item_graph.rb +185 -0
- data/lib/expressir/model/indexes/reference_index.rb +148 -0
- data/lib/expressir/model/indexes/type_index.rb +149 -0
- data/lib/expressir/model/indexes.rb +12 -0
- data/lib/expressir/model/interface_validator.rb +384 -0
- data/lib/expressir/model/literals/binary.rb +7 -9
- data/lib/expressir/model/literals/integer.rb +7 -9
- data/lib/expressir/model/literals/logical.rb +10 -12
- data/lib/expressir/model/literals/real.rb +7 -9
- data/lib/expressir/model/literals/string.rb +10 -12
- data/lib/expressir/model/literals.rb +13 -0
- data/lib/expressir/model/model_element.rb +286 -117
- data/lib/expressir/model/references/attribute_reference.rb +9 -12
- data/lib/expressir/model/references/group_reference.rb +9 -12
- data/lib/expressir/model/references/index_reference.rb +11 -15
- data/lib/expressir/model/references/simple_reference.rb +11 -14
- data/lib/expressir/model/references.rb +12 -0
- data/lib/expressir/model/remark_format.rb +17 -0
- data/lib/expressir/model/remark_info.rb +92 -0
- data/lib/expressir/model/remark_placement.rb +42 -0
- data/lib/expressir/model/repository.rb +472 -12
- data/lib/expressir/model/repository_validator.rb +293 -0
- data/lib/expressir/model/search_engine.rb +551 -0
- data/lib/expressir/model/statements/alias.rb +12 -16
- data/lib/expressir/model/statements/assignment.rb +12 -11
- data/lib/expressir/model/statements/case.rb +31 -14
- data/lib/expressir/model/statements/case_action.rb +9 -11
- data/lib/expressir/model/statements/compound.rb +10 -8
- data/lib/expressir/model/statements/escape.rb +10 -2
- data/lib/expressir/model/statements/if.rb +15 -14
- data/lib/expressir/model/statements/null.rb +10 -2
- data/lib/expressir/model/statements/procedure_call.rb +11 -11
- data/lib/expressir/model/statements/repeat.rb +21 -28
- data/lib/expressir/model/statements/return.rb +9 -8
- data/lib/expressir/model/statements/skip.rb +10 -2
- data/lib/expressir/model/statements.rb +20 -0
- data/lib/expressir/model/supertype_expressions/binary_supertype_expression.rb +13 -17
- data/lib/expressir/model/supertype_expressions/oneof_supertype_expression.rb +7 -9
- data/lib/expressir/model/supertype_expressions.rb +12 -0
- data/lib/expressir/model.rb +38 -78
- data/lib/expressir/package/builder.rb +231 -0
- data/lib/expressir/package/metadata.rb +79 -0
- data/lib/expressir/package/reader.rb +149 -0
- data/lib/expressir/package.rb +8 -0
- data/lib/expressir/schema_manifest.rb +159 -0
- data/lib/expressir/schema_manifest_entry.rb +15 -0
- data/lib/expressir/version.rb +6 -1
- data/lib/expressir.rb +79 -9
- data/lib/tasks/verify_remarks.rake +16 -0
- data/rakelib/native.rake +24 -0
- data/rakelib/oracle.rake +52 -0
- data/rakelib/verify_remarks.rake +16 -0
- metadata +431 -129
- data/.cross_rubies +0 -20
- data/.gitmodules +0 -6
- data/.yardopts +0 -11
- data/docs/development.md +0 -90
- data/exe/format +0 -18
- data/exe/generate-parser +0 -51
- data/lib/expressir/cli/ui.rb +0 -36
- data/lib/expressir/express/2.7/express_parser.so +0 -0
- data/lib/expressir/express/3.0/express_parser.so +0 -0
- data/lib/expressir/express/3.1/express_parser.so +0 -0
- data/lib/expressir/express/3.2/express_parser.so +0 -0
- data/lib/expressir/express/visitor.rb +0 -2588
- data/lib/expressir/model/data_type.rb +0 -9
- data/lib/expressir/model/declaration.rb +0 -9
- data/lib/expressir/model/expression.rb +0 -9
- data/lib/expressir/model/literal.rb +0 -9
- data/lib/expressir/model/reference.rb +0 -9
- data/lib/expressir/model/statement.rb +0 -9
- data/lib/expressir/model/supertype_expression.rb +0 -9
- data/rakelib/antlr4-native.rake +0 -161
- data/rakelib/cross-ruby.rake +0 -368
- data/spec/acceptance/version_spec.rb +0 -27
- data/spec/expressir/express/cache_spec.rb +0 -89
- data/spec/expressir/express/formatter_spec.rb +0 -173
- data/spec/expressir/express/parser_spec.rb +0 -141
- data/spec/expressir/model/model_element_spec.rb +0 -318
- data/spec/spec_helper.rb +0 -24
- data/spec/support/console_helper.rb +0 -29
- data/spec/syntax/multiple.exp +0 -23
- data/spec/syntax/multiple.yaml +0 -198
- data/spec/syntax/multiple_formatted.exp +0 -71
- data/spec/syntax/multiple_hyperlink_formatted.exp +0 -71
- data/spec/syntax/multiple_schema_head_hyperlink_formatted.exp +0 -13
- data/spec/syntax/remark.exp +0 -193
- data/spec/syntax/remark.yaml +0 -471
- data/spec/syntax/remark_formatted.exp +0 -228
- data/spec/syntax/single.exp +0 -4
- data/spec/syntax/single.yaml +0 -18
- data/spec/syntax/single_formatted.exp +0 -10
- data/spec/syntax/single_formatted.yaml +0 -36
- data/spec/syntax/syntax.exp +0 -333
- data/spec/syntax/syntax.yaml +0 -3509
- data/spec/syntax/syntax_formatted.exp +0 -902
- data/spec/syntax/syntax_hyperlink_formatted.exp +0 -902
- data/spec/syntax/syntax_schema_head_formatted.exp +0 -18
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# 09 — SMRL index XML + declaration listing (wo-list, wo-smrl-xml)
|
|
2
|
+
|
|
3
|
+
**Status: NOT STARTED**
|
|
4
|
+
|
|
5
|
+
## Goal
|
|
6
|
+
|
|
7
|
+
The SRL publication byproducts eeng generates: declaration listings
|
|
8
|
+
(`-list.txt`) and the SMRL index XML for longform/concatenated files
|
|
9
|
+
(`<concatenated_express_file_content_list>`), emitted from the same
|
|
10
|
+
model that powers 04/05.
|
|
11
|
+
|
|
12
|
+
## Sources
|
|
13
|
+
|
|
14
|
+
`plugins/p11/wo-list.lisp`, `wo-smrl-xml.lisp` (element names:
|
|
15
|
+
concatenated_express_file_content_list, per-schema content entries),
|
|
16
|
+
top-level.lisp:748-793 (when the index is generated: XML output +
|
|
17
|
+
longform/concatenated modes).
|
|
18
|
+
|
|
19
|
+
## Acceptance
|
|
20
|
+
|
|
21
|
+
Index/list output for the 03 golden set structurally identical
|
|
22
|
+
(element-for-element) to eeng's; wired into the same CLI as 04.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# 10 — Interface dependency dot graph (kernel/dot-graph.lisp)
|
|
2
|
+
|
|
3
|
+
**Status: NOT STARTED**
|
|
4
|
+
|
|
5
|
+
## Goal
|
|
6
|
+
|
|
7
|
+
Emit GraphViz dot for schema interface dependencies exactly like eeng
|
|
8
|
+
(qualify.sh "Generate Dot File (interface)"), from our compiled
|
|
9
|
+
reference graph (already computes interface edges + closures over the
|
|
10
|
+
full SMRL).
|
|
11
|
+
|
|
12
|
+
## Notes
|
|
13
|
+
|
|
14
|
+
eeng's dot generation honors the concatenated mode (root-schema
|
|
15
|
+
subgraph); replicate node/edge/styling choices so WG12 visual
|
|
16
|
+
diffing stays stable. Trivial once 02 documents their exact output
|
|
17
|
+
shape — deliberately last in the queue.
|
|
18
|
+
|
|
19
|
+
## Acceptance
|
|
20
|
+
|
|
21
|
+
Byte-comparable dot output vs eeng for arm/mim of the golden modules.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# 11 — Import the Express Engine test suite
|
|
2
|
+
|
|
3
|
+
**Status: NOT STARTED**
|
|
4
|
+
|
|
5
|
+
## Goal
|
|
6
|
+
|
|
7
|
+
Fully import eeng's tests — their release-suite scripts, the bug
|
|
8
|
+
corpus, and the reference documents that define pass/fail — as
|
|
9
|
+
executable RSpec feature specs and regression fixtures in expressir.
|
|
10
|
+
Nothing in their suite should remain unrepresented.
|
|
11
|
+
|
|
12
|
+
## Inventory (verified 2026-09-21)
|
|
13
|
+
|
|
14
|
+
eeng has no formal test framework (`tests/` is a README describing an
|
|
15
|
+
unrealized Makefile suite); the suite is:
|
|
16
|
+
|
|
17
|
+
### CLI drivers — `scripts/*.sh` (14)
|
|
18
|
+
|
|
19
|
+
| script | drives | our target operation |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| `qual-arm.sh`, `qual-mod.sh` | per-module qualification loop (arm/mim × every qualify.sh op) | 04, 05, 07-10 end-to-end specs |
|
|
22
|
+
| `qual-res.sh` | same over `qualify-res.txt` resources | idem, resource side |
|
|
23
|
+
| `pretty-test.sh` | `--pretty` ap209/ap210/ap242 × {arm,mim} × {shortform,longform} | 08 formatter specs |
|
|
24
|
+
| `ap210.sh` | the 2.5MB long-form giant, pretty × 4 modes | 08 scale spec |
|
|
25
|
+
| `test-tt.sh`, `test-tt1.sh` | `--xml` arm/mim shortform + Saxon HTM conversion + regenerate-exp compare | 09 (XML chain) |
|
|
26
|
+
| `build-xml.sh` | XML builds over the whole `modules.lst` + `resources.lst` | 09 corpus spec |
|
|
27
|
+
| `test.sh` | ad-hoc resource XML/HTM driver (kinematic schema) | 09 |
|
|
28
|
+
| `scan-res.lisp` | resource corpus scan | 07 check corpus |
|
|
29
|
+
| `kevin.sh`, `veronique.sh`, `copy-mod-xml.sh` | person-named scratch drivers | triage: keep as scenarios or drop with rationale |
|
|
30
|
+
|
|
31
|
+
Each script's eeng CLI matrix (`-mode {arm,mim}_{shortform,longform,
|
|
32
|
+
concatenated}` `-stepmod` `-schema` `--pretty/--xml/--flat/--dot
|
|
33
|
+
--check`) maps 1:1 onto our planned operations — the flag matrix
|
|
34
|
+
becomes our spec matrix.
|
|
35
|
+
|
|
36
|
+
### Bug corpus — `docs/issues/`
|
|
37
|
+
|
|
38
|
+
- `bug39-Greedy-Parens.txt` — **shtolo fixture**: contains "results
|
|
39
|
+
of old shtolo and stepcode exppp" vs "results of new shtolo"
|
|
40
|
+
outputs; doubles as a differential vs stepcode exppp.
|
|
41
|
+
- `bug40-Pretty-Expressions.txt` — pretty-printer expression cases.
|
|
42
|
+
Both become regression specs with the literal inputs embedded.
|
|
43
|
+
|
|
44
|
+
### Reference documents — `docs/` (define pass/fail)
|
|
45
|
+
|
|
46
|
+
- `shtolo_converting_STEP_short_listings_to_annotated_listings_libes.pdf`
|
|
47
|
+
— the SHTOLO paper (short listings → annotated listings); required
|
|
48
|
+
reading for 01/05 semantics.
|
|
49
|
+
- BNF grammars: `iso-10303-11--2004{,-raw}.bnf`, `iso-10303-14*.bnf`,
|
|
50
|
+
**`iso-10303-21--{1994,2002}.bnf`** (Part 21!), `xml.bnf`,
|
|
51
|
+
`patch_schema.bnf` — grammar cross-checks (05's 1994 profile; the
|
|
52
|
+
Part 21 BNFs feed the future p21 parser).
|
|
53
|
+
- `express_model.dtd`, `module.dtd` — the DTDs their XML emits (09's
|
|
54
|
+
structural contract).
|
|
55
|
+
- `qualify.txt`, `interfaces.txt`, `interface-notes.pdf` —
|
|
56
|
+
qualification + interface semantics notes.
|
|
57
|
+
- `00check.txt` (repo root) — check.lisp design notes → 07.
|
|
58
|
+
|
|
59
|
+
## Steps
|
|
60
|
+
|
|
61
|
+
1. Copy the durable assets into the repo: scripts verbatim under
|
|
62
|
+
`spec/fixtures/eeng/scripts/` (provenance header), bug corpus under
|
|
63
|
+
`spec/fixtures/eeng/issues/`, DTDs + BNFs under
|
|
64
|
+
`spec/fixtures/eeng/grammar/` (license is MIT-style per eeng
|
|
65
|
+
headers — verify and record attribution).
|
|
66
|
+
2. Translate the CLI matrix into RSpec feature specs
|
|
67
|
+
(`spec/expressir/eeng_parity/`): one describe per driver family,
|
|
68
|
+
one example per (schema × mode × operation), corpus-scaled ones
|
|
69
|
+
tagged `smrl`.
|
|
70
|
+
3. Port bug39/bug40 as literal regression specs now (they need only
|
|
71
|
+
parser + formatter, both of which exist).
|
|
72
|
+
4. Extend 03's oracle harness to emit/refresh the expected outputs
|
|
73
|
+
these specs consume.
|
|
74
|
+
|
|
75
|
+
## Acceptance
|
|
76
|
+
|
|
77
|
+
Every driver row and both bugs have a corresponding spec (or a
|
|
78
|
+
documented drop rationale); `smrl`-tagged parity suite green against
|
|
79
|
+
the 03 golden corpus.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# 12 — Full check-p11 catalog port (eeng `kernel/check.lisp`)
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Bring `Expressir::Express::Checker` from the current documented subset
|
|
6
|
+
to the full eeng check-p11 catalog: every note eeng 5.1.0 emits over
|
|
7
|
+
the SRL/SMRL corpora, with note ids and severities aligned so oracle
|
|
8
|
+
differentials stay cross-referenceable.
|
|
9
|
+
|
|
10
|
+
## Current subset (shipped)
|
|
11
|
+
|
|
12
|
+
check-schema-interface-redundant, check-schema-iface-resource-duplicate,
|
|
13
|
+
check-subtype-ref, check-subtype-cycle (path-tracked DFS),
|
|
14
|
+
check-duplicate-declaration, check-where-name-pattern,
|
|
15
|
+
check-unique-name-pattern, check-unresolved-ref,
|
|
16
|
+
check-select-extended-type, check-enumeration-extended-type,
|
|
17
|
+
check-self-schema-reference (#125), parse-failure → nonzero exit.
|
|
18
|
+
|
|
19
|
+
## Sources
|
|
20
|
+
|
|
21
|
+
- `~/src/external/exp-engine-engine/kernel/check.lisp` (835 lines,
|
|
22
|
+
`check-p11` generic + per-node-type methods) and
|
|
23
|
+
`kernel/check-notes.lisp` (note catalog).
|
|
24
|
+
- ISO 10303-11:2004 clause text — `~/src/mn/iso-10303-11/sources/sections/`
|
|
25
|
+
(each ported check cites its clause).
|
|
26
|
+
- Existing plan: `07-semantic-checks-port.md`; current impl
|
|
27
|
+
`lib/expressir/express/checker.rb`.
|
|
28
|
+
|
|
29
|
+
## Design
|
|
30
|
+
|
|
31
|
+
1. **Catalog extraction**: enumerate every `post-message` site in
|
|
32
|
+
check.lisp/check-notes.lisp → (note-id, severity, node type,
|
|
33
|
+
message shape). Commit as `spec/fixtures/eeng/check-catalog.md`
|
|
34
|
+
with our coverage column; CI gate diffs the catalog against the
|
|
35
|
+
oracle so new eeng notes surface.
|
|
36
|
+
2. **Port order** (by eeng's own walk): schema-level (interfaces,
|
|
37
|
+
constants) → entity (supertype/subtype-of, attributes, derives,
|
|
38
|
+
inverses, uniques, wheres) → type (select/enumeration/aggregate
|
|
39
|
+
bounds) → function/procedure/rule bodies (expression typing lite:
|
|
40
|
+
arity/name resolution of called functions, built-ins from BUILTINS,
|
|
41
|
+
variable scoping via ScopeResolver) → attribute-level
|
|
42
|
+
redeclaration consistency (redispatches, `SELF\` qualifiers).
|
|
43
|
+
3. **Differential gate**: run eeng `--validate` and our `validate
|
|
44
|
+
check` over the SRL corpus + module arm/mim set; every corpus file
|
|
45
|
+
must produce the SAME note-id multiset (message text may differ).
|
|
46
|
+
Land as `checker_eeng_full_catalog_differential_spec`
|
|
47
|
+
(production_scale-tagged for the big corpus; small fixtures in the
|
|
48
|
+
default suite).
|
|
49
|
+
4. **Severity mapping** table: eeng `:error/:warning` → ours verbatim.
|
|
50
|
+
|
|
51
|
+
## Acceptance
|
|
52
|
+
|
|
53
|
+
- Catalog file lists 100% of eeng note ids with status columns.
|
|
54
|
+
- SRL differential: zero unexplained note-id mismatches.
|
|
55
|
+
- `validate check` docs updated (README table).
|
|
56
|
+
|
|
57
|
+
## Test cases
|
|
58
|
+
|
|
59
|
+
- eeng `docs/issues/` corpus inputs; qualify loops
|
|
60
|
+
(`scripts/qual-*.sh`) over resources.lst/modules.lst.
|
|
61
|
+
- Per-clause unit fixtures citing ISO 10303-11 clause numbers.
|
|
62
|
+
|
|
63
|
+
## Release checkpoint
|
|
64
|
+
|
|
65
|
+
PR + expressir patch release once the entity/type/schema tiers land
|
|
66
|
+
(expression-tier notes may follow in a second PR — keep the catalog
|
|
67
|
+
honest about which tier each note belongs to).
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# 13 — Population / Part 21 instance validation
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Validate STEP Part 21 instance data (.stp/.p21) against a loaded
|
|
6
|
+
schema — eeng's core strength (`--validate` with population), absent
|
|
7
|
+
from expressir today.
|
|
8
|
+
|
|
9
|
+
## Sources
|
|
10
|
+
|
|
11
|
+
- ISO 10303-21: exchange structure grammar — BNF in
|
|
12
|
+
`~/src/mn/iso-10303-detached-docs/sources/iso-10303-21*/` (and
|
|
13
|
+
eeng's copy under `plugins/p21/`, incl. its parser + validation).
|
|
14
|
+
- eeng: `plugins/p21/` (p21 reader, population), `--validate`
|
|
15
|
+
(`run-op` population path), `wo-xml.lisp` instance-side references.
|
|
16
|
+
- eeng banner: population validation "not fully supported" upstream —
|
|
17
|
+
our target is the supported subset, documented precisely.
|
|
18
|
+
|
|
19
|
+
## Design (scope ladder; each step ships)
|
|
20
|
+
|
|
21
|
+
1. **P21 reader**: parsanol grammar for the exchange structure
|
|
22
|
+
(header entities + DATA section: complex entity `(`name(...)`)`,
|
|
23
|
+
entity instances, list/set values, typed refs `#1`, enums `.T.`,
|
|
24
|
+
binary `\"..."`, redefines `/`). Emit a Population model
|
|
25
|
+
(`Model::Population`): instances with entity name + attribute
|
|
26
|
+
values keyed by explicit attribute position.
|
|
27
|
+
2. **Structural validation**: every instance maps to a known entity
|
|
28
|
+
in the schema closure (complex entities → all parts); attribute
|
|
29
|
+
arity matches; value kinds match declared types (aggregate bounds,
|
|
30
|
+
select members, string/binary widths, REAL precision) — the
|
|
31
|
+
aggregate of eeng's population notes.
|
|
32
|
+
3. **Constraint validation**: UNIQUE rules (per-entity + inherited),
|
|
33
|
+
WHERE rules — evaluation needs an EXPRESS expression interpreter;
|
|
34
|
+
reuse/extend the `benchmark`/expression-walk machinery, evaluate
|
|
35
|
+
over instance bindings (scope: no INVERSE/derives first).
|
|
36
|
+
4. **CLI**: `expressir validate population SCHEMA -p DATA.p21`,
|
|
37
|
+
JSON/plain reports aligned with Checker notes.
|
|
38
|
+
|
|
39
|
+
## Acceptance
|
|
40
|
+
|
|
41
|
+
- Official Part 21 test corpora parse + validate (see tests).
|
|
42
|
+
- Differential vs eeng `--validate` on shared fixtures: same
|
|
43
|
+
accept/reject decision and note ids.
|
|
44
|
+
|
|
45
|
+
## Test cases
|
|
46
|
+
|
|
47
|
+
- ISO 10303-21 validation suite files from detached-docs where
|
|
48
|
+
available; else eeng `tests/` + `scripts/test-tt*.sh` fixtures.
|
|
49
|
+
- Hand fixtures: arity mismatch, bad enum, UNIQUE collision, WHERE
|
|
50
|
+
failure, complex-entity mismatch.
|
|
51
|
+
|
|
52
|
+
## Release checkpoint
|
|
53
|
+
|
|
54
|
+
Steps 1–2 = first release (structural validator); step 3 second.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# 14 — SHTOLO completion (expressir#32, Annex G)
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Finish `Expressir::Express::Shtolo` to full Annex G conformance and
|
|
6
|
+
close the eeng differential over the whole corpus. Current state:
|
|
7
|
+
G.1 + G.2 stages implemented (522 lines), suite green (10 examples),
|
|
8
|
+
`extenders:` :all/:none matches the eeng-default split.
|
|
9
|
+
|
|
10
|
+
## Remaining work
|
|
11
|
+
|
|
12
|
+
1. **Rule audit**: walk `spec/fixtures/shtolo/annex-g-rules.md`
|
|
13
|
+
(G.NM.1-2, G.1.1-10, G.2.1-7) and mark every rule PASS/partial/✖
|
|
14
|
+
in the file; a fixture per rule. Known-thin areas from the audit:
|
|
15
|
+
- G.1.6 dependently-instantiable rule output (function + WHERE
|
|
16
|
+
wiring) — verify against Annex G text, not eeng (eeng ✖).
|
|
17
|
+
- name-munging policy (`_dot_` and friends) — eeng ✖; standard wins.
|
|
18
|
+
- prune pass reachability (unreachable subtype branches).
|
|
19
|
+
2. **Corpus differential**: for every module arm/mim in modules.lst,
|
|
20
|
+
compare our flatten output to `eengine --flat` (eeng default =
|
|
21
|
+
`extenders: :none` shape); classify divergences
|
|
22
|
+
standard-conformant vs bug. Feed the mismatch corpus into specs.
|
|
23
|
+
3. **Scale**: SMRL full set flatten over the compiled-set warm model;
|
|
24
|
+
memory ceiling documented.
|
|
25
|
+
4. **CLI**: `expressir shtolo MANIFEST -o longform.exp` (root schema
|
|
26
|
+
+ closure resolution reuses ParityInputs).
|
|
27
|
+
|
|
28
|
+
## Acceptance
|
|
29
|
+
|
|
30
|
+
- Every Annex G rule has a green fixture.
|
|
31
|
+
- Corpus differential: zero unexplained diffs; every explained
|
|
32
|
+
divergence documented in `parity-matrix.md`.
|
|
33
|
+
- SMRL flatten completes; output parses under the 1994 profile.
|
|
34
|
+
|
|
35
|
+
## Test cases
|
|
36
|
+
|
|
37
|
+
- annex-g-rules.md fixtures; eeng `docs/issues/bug39` shtolo outputs
|
|
38
|
+
(old-vs-new vs stepcode exppp); `scripts/qual-*.sh` loops;
|
|
39
|
+
`pretty-test.sh` ap209/210/242 × arm/mim shape checks.
|
|
40
|
+
|
|
41
|
+
## Release checkpoint
|
|
42
|
+
|
|
43
|
+
One PR per coherent slice (rule audit + fixtures; differential gate;
|
|
44
|
+
CLI), patch releases as each merges.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# 15 — EXPRESS → XML (ISO 10303-28 / eeng --xml)
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Generate XML Schema output for a schema (expressir#276): eeng's
|
|
6
|
+
`--xml` (`wo-xml-p11.lisp`) plus the SMRL index listing
|
|
7
|
+
(`wo-smrl-xml.lisp`, feeds the SMRL document set).
|
|
8
|
+
|
|
9
|
+
## Sources
|
|
10
|
+
|
|
11
|
+
- ISO 10303-28:2007 ( EXPRESS-driven XML representation) — detached
|
|
12
|
+
docs `~/src/mn/iso-10303-detached-docs/sources/iso-10303-28*/`;
|
|
13
|
+
eeng `docs/` carries `express_model.dtd` + module.dtd (legacy
|
|
14
|
+
forms).
|
|
15
|
+
- eeng: `plugins/p11/wo-xml-p11.lisp` (schema → XSD),
|
|
16
|
+
`wo-smrl-xml.lisp` (SMRL index XML), `--xml` run-op.
|
|
17
|
+
- Consumer: iso-10303 repo's `schemas/data/*.xml` SMRL pages
|
|
18
|
+
(rendered by metanorma today) — our output must be drop-in for
|
|
19
|
+
that pipeline.
|
|
20
|
+
|
|
21
|
+
## Design
|
|
22
|
+
|
|
23
|
+
1. **SMRL index XML first** (small, high value for iso-10303): the
|
|
24
|
+
index of all schemas in a repository (name, file, interfaces,
|
|
25
|
+
counts) as the repo's data-layer source — replaces the
|
|
26
|
+
hand-maintained entries.
|
|
27
|
+
2. **express2xsd**: schema → XSD via a declarative mapping table
|
|
28
|
+
(entity → element+complexType, type → simpleType/complexType,
|
|
29
|
+
select → choice, explicit attrs in order, supertype substitution
|
|
30
|
+
groups for SUBTYPE OF, aggregates → minOccurs/maxOccurs). Emit
|
|
31
|
+
through a Nokogiri-free XML writer (ROXML via lutaml-model where
|
|
32
|
+
it fits; hand-rolled tree otherwise — the mapping is the product,
|
|
33
|
+
not the serialization).
|
|
34
|
+
3. **CLI**: `expressir xml SCHEMA -o schema.xsd`;
|
|
35
|
+
`expressir xml-index REPO -o smrl.xml`.
|
|
36
|
+
|
|
37
|
+
## Acceptance
|
|
38
|
+
|
|
39
|
+
- Output validates against the Part 28 reference XSDs (detached
|
|
40
|
+
docs) for the fixture set; xsd diff vs eeng --xml on SRL corpus
|
|
41
|
+
classified standard-vs-eeng.
|
|
42
|
+
- iso-10303 SMRL pages render from our generated index unchanged.
|
|
43
|
+
|
|
44
|
+
## Test cases
|
|
45
|
+
|
|
46
|
+
- eeng `scripts/build-xml.sh` full-corpus reference outputs;
|
|
47
|
+
`test-tt{,1}.sh` (XML + Saxon HTM chain) as end-to-end smoke.
|
|
48
|
+
|
|
49
|
+
## Release checkpoint
|
|
50
|
+
|
|
51
|
+
SMRL index first (unblocks iso-10303 pipeline), express2xsd second.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# 16 — Schema compare & patch (eeng --compare / --patch)
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Structural diff of two schemata with a machine-applicable patch
|
|
6
|
+
output — eeng `--compare` (+ `--concat_compare`, `--patch`,
|
|
7
|
+
`--patch_schema_name`), used by WG12 to track schema revisions.
|
|
8
|
+
|
|
9
|
+
## Sources
|
|
10
|
+
|
|
11
|
+
- eeng `plugins/p11/wo-*.lisp` compare entry (`run-op (:compare)`
|
|
12
|
+
in top-level.lisp) and patch application paths; mode args:
|
|
13
|
+
`:trial_schema :reference_schema :xml-output :patch-output`.
|
|
14
|
+
- Semantics baseline: declaration-level matching by id within the
|
|
15
|
+
flattened/concatenated form (eeng compares post-concat shapes for
|
|
16
|
+
shortforms).
|
|
17
|
+
|
|
18
|
+
## Design
|
|
19
|
+
|
|
20
|
+
1. **Model diff** on the resolved repositories: declarations matched
|
|
21
|
+
by canonical path (`schema.item`); kinds of delta — added,
|
|
22
|
+
removed, changed (per-attribute structural diff: types,
|
|
23
|
+
attribute lists, where/unique bodies via formatted-text
|
|
24
|
+
comparison first, structural second).
|
|
25
|
+
2. **Outputs**: unified-ish text report (human), XML
|
|
26
|
+
(`--xml-output` parity), and patch script (the eeng patch
|
|
27
|
+
format: rename/redefine vocabulary it defines — ported from
|
|
28
|
+
wo-patch output shapes).
|
|
29
|
+
3. **CLI**: `expressir compare TRIAL.exp REFERENCE.exp [--xml|--patch]`;
|
|
30
|
+
`concat-compare` rides the Concatenator for shortform pairs.
|
|
31
|
+
4. **Patch application**: apply a patch file to a model (rename
|
|
32
|
+
declarations, remap interfaces) — the write-back side eeng uses
|
|
33
|
+
for round-trip patching.
|
|
34
|
+
|
|
35
|
+
## Acceptance
|
|
36
|
+
|
|
37
|
+
- SRL corpus self-compare = empty diff; mutated fixtures (rename,
|
|
38
|
+
add/remove attr, retype) produce the expected delta kinds.
|
|
39
|
+
- Differential vs eeng `--compare` XML on the same pairs.
|
|
40
|
+
|
|
41
|
+
## Test cases
|
|
42
|
+
|
|
43
|
+
- eeng `docs/issues/` pairs; hand-mutated fixture ladder; the
|
|
44
|
+
`--concat_compare` module-vs-module pairs from scripts.
|
|
45
|
+
|
|
46
|
+
## Release checkpoint
|
|
47
|
+
|
|
48
|
+
Text+XML diff first release; patch apply second.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# 17 — Official test-case incorporation
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
One place that maps every OFFICIAL/reference test corpus to our
|
|
6
|
+
specs, and CI gates that run the tractable tier on every push.
|
|
7
|
+
|
|
8
|
+
## Corpora
|
|
9
|
+
|
|
10
|
+
1. **eeng suite** (already imported, `spec/fixtures/eeng/`):
|
|
11
|
+
`scripts/*.sh` 14 drivers (qual-arm/mod/res loops, pretty-test.sh
|
|
12
|
+
ap209/210/242 × arm/mim × shortform/longform, ap210 giant file,
|
|
13
|
+
test-tt{,1}.sh XML chain, build-xml.sh), `docs/issues/` bug
|
|
14
|
+
corpus (bug39 shtolo old-vs-new vs stepcode exppp; bug40 pretty
|
|
15
|
+
expressions), `qualify.txt`, `interfaces.txt`, `00check.txt`.
|
|
16
|
+
2. **ISO 10303-11** grammar/conformance sources:
|
|
17
|
+
`~/src/mn/iso-10303-11/sources/sections/` — Annex G rules
|
|
18
|
+
(fixtures live in `spec/fixtures/shtolo/annex-g-rules.md`),
|
|
19
|
+
syntax annexes for negative/positive grammar tests.
|
|
20
|
+
3. **ISO detached docs** `~/src/mn/iso-10303-detached-docs/sources/
|
|
21
|
+
iso-10303-*`: Part 21 exchange-structure fixtures (population),
|
|
22
|
+
Part 28 reference XSDs (XML), Part 22 (SDAI vocabulary, later).
|
|
23
|
+
4. **SRL/SMRL corpora** (`resources.lst`/`modules.lst` names match
|
|
24
|
+
our checkout): the differential tier.
|
|
25
|
+
|
|
26
|
+
## Structure
|
|
27
|
+
|
|
28
|
+
- `spec/expressir/eeng_parity/` — one spec per capability matrix
|
|
29
|
+
row; each example tags its source corpus file in a comment.
|
|
30
|
+
- Tiering: `:fast` (fixtures, every push) / `:corpus`
|
|
31
|
+
(SRL differential, nightly or manual tag `production_scale`) /
|
|
32
|
+
`:smoke_eeng` (runs the local eeng oracle when the binary exists —
|
|
33
|
+
skipped otherwise).
|
|
34
|
+
- `rake parity:status` prints the capability × corpus matrix with
|
|
35
|
+
green/gray cells — the single progress view.
|
|
36
|
+
|
|
37
|
+
## Rules
|
|
38
|
+
|
|
39
|
+
- No test is added without its provenance row in the fixture dir's
|
|
40
|
+
PROVENANCE.md (established practice, keep it).
|
|
41
|
+
- Oracle outputs are regenerated, never hand-edited; the regeneration
|
|
42
|
+
command lives next to each golden.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# 18 — EXPRESS MIM mapping parser (expressir#88)
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Parse and validate MIM "reference path" mappings — the application ↔
|
|
6
|
+
interpretation mapping every STEP module carries — into a typed model,
|
|
7
|
+
with the mapping paths validated against the ARM/MIM schemas.
|
|
8
|
+
|
|
9
|
+
## Sources
|
|
10
|
+
|
|
11
|
+
- This checkout's canonical form: `schemas/modules/*/mapping.yaml` —
|
|
12
|
+
structured ae/aimelt/aa records whose values use the
|
|
13
|
+
`<<express:SCHEMA.ITEM,ITEM>>` link markup (the same markup
|
|
14
|
+
`Suma::LinkValidation` validates today). Generated upstream from
|
|
15
|
+
module.xml.
|
|
16
|
+
- Upstream SMRL module.xml: `mapping`/`ae`/`aimelt` elements with
|
|
17
|
+
`reference_path` bodies — the ISO 10303-1x mapping methodology
|
|
18
|
+
grammar: entity chains `A -> B -> C`, qualifiers `[attr => val]`,
|
|
19
|
+
`=>` derived references, `->>` subtype traversal, `<>` select
|
|
20
|
+
member. Port the grammar from the Part 1x mapping annex text
|
|
21
|
+
(detached docs) + existing module corpus as the oracle.
|
|
22
|
+
- eeng has no counterpart — this is an expressir-first capability.
|
|
23
|
+
|
|
24
|
+
## Design
|
|
25
|
+
|
|
26
|
+
1. `Expressir::Model::Mapping` — MappingDocument →
|
|
27
|
+
ApplicationElements (entity, aimelt, attributes) and
|
|
28
|
+
ReferencePath nodes (path-expression AST: steps, qualifiers,
|
|
29
|
+
subtype traversals).
|
|
30
|
+
2. `mapping.yaml` reader (lutaml-model typed) — yaml → MappingDocument.
|
|
31
|
+
3. Reference-path expression parser — parsanol grammar over the path
|
|
32
|
+
syntax; AST reuses Model::References where shapes coincide.
|
|
33
|
+
4. Validation: every path resolves against the resolved ARM+MIM
|
|
34
|
+
repository (rides SchemaIndex / ItemGraph); unknown step = error
|
|
35
|
+
note (Checker id `check-mapping-path`).
|
|
36
|
+
5. CLI: `expressir mapping validate DIR|FILE` over module dirs.
|
|
37
|
+
|
|
38
|
+
## Acceptance
|
|
39
|
+
|
|
40
|
+
- All module mapping.yaml files in the SMRL parse; path validation
|
|
41
|
+
reports zero unknown steps (or a classified, explained residue).
|
|
42
|
+
- Round-trip: MappingDocument → yaml byte-stable.
|
|
43
|
+
|
|
44
|
+
## Test cases
|
|
45
|
+
|
|
46
|
+
- module corpus (603 modules); hand fixtures for each path grammar
|
|
47
|
+
production; invalid-path fixtures.
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# 19 — End-state architecture: Expressir in Rust, bound to metanorma via lutaml-model
|
|
2
|
+
|
|
3
|
+
**Question (2026-09-23): should Expressir ultimately be in Rust, bound to
|
|
4
|
+
metanorma via lutaml-model?**
|
|
5
|
+
|
|
6
|
+
**Answer: yes as a direction, no as a rewrite.** The end state is
|
|
7
|
+
*Rust engine, Ruby presentation* — reached in three milestones, not by
|
|
8
|
+
porting everything at once. The binding to metanorma goes through
|
|
9
|
+
lutaml-model's Liquid Drop layer, which is exactly the seam that
|
|
10
|
+
already works (expressir#255, #319, #421, #426).
|
|
11
|
+
|
|
12
|
+
## Current state (v2.4.9, measured)
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
.exp ──► parsanol-rs 0.8.8 (parse, wire tree)
|
|
16
|
+
└─► expressir_core ext (batch, EXSCS1 artifact, warm start)
|
|
17
|
+
└─► instantiate → Expressir::Model (lutaml-model, Ruby)
|
|
18
|
+
└─► Checker / Shtolo / formatters (Ruby semantics)
|
|
19
|
+
└─► Liquid Drops ──► metanorma / suma
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
- Parse is Rust and fast (SRL cold 36 s CPU, SMRL 558 s CPU / 1448
|
|
23
|
+
schemas); parsanol 0.8.8's selective memoization is adopted.
|
|
24
|
+
- The measured floor is now the **wire→Ruby-object hydration**
|
|
25
|
+
(71 % of warm CPU) and the overlays (29 %). Rendering (metanorma)
|
|
26
|
+
touches a small fraction of the model.
|
|
27
|
+
- Semantics (Checker subset, SHTOLO with the 50-module eengine
|
|
28
|
+
differential, PrettyFormatter, SMRL XML) live in Ruby, eeng-parity
|
|
29
|
+
validated, with Ruby CI harnesses.
|
|
30
|
+
|
|
31
|
+
## Options considered
|
|
32
|
+
|
|
33
|
+
**A — Status quo+.** Rust keeps parse+artifact; all new semantics
|
|
34
|
+
(Checker catalog, mapping validation) land in Ruby. No rewrite risk,
|
|
35
|
+
parity harnesses stay, but the hydration floor remains and the Rust
|
|
36
|
+
surface stays private.
|
|
37
|
+
|
|
38
|
+
**B — Lazy binding (the recommended milestone).** Keep the Ruby model
|
|
39
|
+
as the canonical object model for feature work, and add a READ path
|
|
40
|
+
that serves metanorma directly from Rust-held compiled-set data:
|
|
41
|
+
expressir_core exposes accessors over the loaded set (schema by id,
|
|
42
|
+
kind listings, field reads), and a lutaml-model Drop proxies them on
|
|
43
|
+
demand — rendering never instantiates the full object graph.
|
|
44
|
+
lutaml-model keeps ownership of mappings/serialization/Drop
|
|
45
|
+
integration; the Drop becomes the literal "Expressir in Rust bound to
|
|
46
|
+
metanorma via lutaml-model".
|
|
47
|
+
|
|
48
|
+
**C — Full Rust Expressir.** Model + semantics in Rust; Ruby keeps
|
|
49
|
+
only Drops and CLI glue; publish expressir-rs on crates.io; unlock
|
|
50
|
+
non-Ruby consumers (standalone binary, WASM). Rejected for now: it
|
|
51
|
+
re-implements eeng-validated semantics with parity risk, doubles CI,
|
|
52
|
+
and metanorma still needs a Drop/FFI layer — the boundary cost moves,
|
|
53
|
+
it does not disappear.
|
|
54
|
+
|
|
55
|
+
**Decision: B now; C deferred behind explicit triggers**
|
|
56
|
+
(a non-Ruby consumer materializes; the Ruby semantic layers become a
|
|
57
|
+
measured bottleneck — they are not today; or crate publishing is
|
|
58
|
+
wanted for supply-chain reasons independent of consumers).
|
|
59
|
+
|
|
60
|
+
## Milestones
|
|
61
|
+
|
|
62
|
+
### M1 — rev-pinned core dependency (supply chain, one line)
|
|
63
|
+
The ext pins `expressir-rs = { git …, branch = "main" }`; a released
|
|
64
|
+
gem can pick up a shifted core. Pin `rev = "<sha>"` per expressir
|
|
65
|
+
release (2.4.8 ↔ today's expressir-rs main), bump per release.
|
|
66
|
+
|
|
67
|
+
### M2 — lazy Drops for the metanorma hot path
|
|
68
|
+
1. expressir-rs: read accessors over the loaded compiled set
|
|
69
|
+
(`schema(id)`, kind listings, per-declaration field reads,
|
|
70
|
+
remark/refs overlay application in place).
|
|
71
|
+
2. expressir_core: magnus wrappers returning primitives.
|
|
72
|
+
3. Ruby: `LazySchemaDrop` (lutaml-model #319 custom Drop, the same
|
|
73
|
+
seam as #255/#426) implementing id / declaration listings / source
|
|
74
|
+
faces via the accessors.
|
|
75
|
+
4. Gates: suma-style schema-doc pages render output-comparable with
|
|
76
|
+
the eager path on a fixture corpus; warm page-render benchmark;
|
|
77
|
+
SMRL warm-start CPU target ≤ 30 % of today's 128 s for the
|
|
78
|
+
read-only path.
|
|
79
|
+
|
|
80
|
+
### M3 — semantics placement rule (standing)
|
|
81
|
+
New semantic features land in Ruby **unless** they are
|
|
82
|
+
parse-adjacent or graph-scale (then Rust in expressir-rs). The
|
|
83
|
+
Checker catalog (12) and mapping validation (18) stay Ruby; SHTOLO
|
|
84
|
+
stays Ruby with its eengine differential as the gate.
|
|
85
|
+
|
|
86
|
+
### M4 — trigger review for option C
|
|
87
|
+
Reassess on any of: a non-Ruby consumer request; hydrate/overlay CPU
|
|
88
|
+
re-emerging as the top cost after M2; a crates.io publication ask.
|
|
89
|
+
M2's accessors are exactly the seed of C's public API — nothing in B
|
|
90
|
+
is throwaway.
|
|
91
|
+
|
|
92
|
+
## What lutaml-model owns in every option
|
|
93
|
+
|
|
94
|
+
The mapping DSL (yaml/json faces — incl. the #88 `mapping.yaml`
|
|
95
|
+
reader), Liquid integration (#319 Drops), artifact serialization —
|
|
96
|
+
the binding contract between the engine and metanorma. Rust never
|
|
97
|
+
duplicates it.
|