expressir 2.4.31-aarch64-linux → 2.4.35-aarch64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e49355b67f28dde08504db791f89d8da2a00bd6cf52fcdfded84d4ffc1fe2ab9
-  data.tar.gz: c2c8e67c0ec0dbe02a92e1764c165d3497825049c2d8bd9be9be5d56cf535f4b
+  metadata.gz: 03162a2402e4a51e168ed98bad8aa762fd6a7095ce944b0ac2a6dd979d81291a
+  data.tar.gz: ccceee1f9fb2bc52451f055bf9afcc7381e8f663d2cce41b6d1ebe82edafdee2
 SHA512:
-  metadata.gz: 0c85e6529d67ed3354c2f12d5dc8aa39ee61c60f911cea1816a56f8fba94be419877a30da9c5ac9501fe143e71f3d62a0d4fd487b1b1c9b08431c6d0c1333e84
-  data.tar.gz: b1413fb816b3ac223eda607d4e01d7f154fb35d6ccc484d6123b85da855887d6a137bdc147c8588f980d7c0b716233f98e545a3b4edf84f9e889e3a43514e27d
+  metadata.gz: b5324d363f1064cd9447f1ae4c3fc7161f43145bbc10729eb81bc0163f2f0efd7f58af08bdc83330b3568b2ca944306cd49256ec8bfb4390c4d3a7eecaa7e2b5
+  data.tar.gz: ed333015608ba1021fc621064b1cda0b405331667a3052c65336b832b87dda8f024010a0a37bc8b9430d0ffc3ffd6064197db697751e628f76ca9babf3ce45e2

data/TODO.eengine/00-overview.md

@@ -0,0 +1,39 @@
+# TODO.eengine — the eengine-parity program
+
+Consolidates the remaining eengine-parity tasks (formerly scattered
+across TODO.parity-ee) into one execution-ordered series. The oracle
+is eengine-5.1.0 (GIT eeng-5.1.0) at `~/src/external/exp-engine-engine`; where
+eengine and ISO 10303-11 disagree, the standard wins and the
+divergence is classified in the differential output.
+
+## Execution order and status
+
+| n | task | folds | status |
+|---|------|-------|--------|
+| 01 | resolution diagnostics + USE/REFERENCE tiering | 06-remainder, 07-part | done |
+| 02 | check-p11 catalog completion | 07, 12 | done (type-resolution tranche; generic unresolved-ref walk covers expression bodies; subtype-constraint supertype refs + SELF redeclaration resolution added; constness has no eeng rule — covered by the generic walk) |
+| 03 | entity computations (inherited/derived/inverse closures) | matrix §3, 05-prereq | done |
+| 04 | SHTOLO rule audit + corpus differential | 14-remainder | done (audit spec live; G.NM.2 + G.1.10 fixed; G.1.6 synthesis remains per annex-g-rules.md) |
+| 05 | SHTOLO longform flatten (Annex G) | 05 | done |
+| 06 | population / Part 21 reader + validation | 13 | done (parsanol reader; schema-mapping validator; Annex H corpus clean; eengine p21 differential blocked — eeng 5.1.0 aborts bare-p11 validate/*PARSE-MODE* NIL, every -MODE demands stepmod context) |
+| 07 | schema compare & patch | 16 | done (SchemaCompare canonical-path deltas + SchemaPatch.apply; oracle differential vs eeng --compare green on add/remove entity/attribute + enum-value changes) |
+| 08 | official corpus CI gate | 17 | done |
+| 09 | MIM mapping parser | 18 | done |
+
+## Sources
+
+- eengine: `~/src/external/exp-engine-engine` (kernel/*.lisp,
+  plugins/p11/*.lisp, plugins/p21/*).
+- ISO 10303-11 clause text: `~/src/mn/iso-10303-11/sources/sections/`.
+- Detached docs: `~/src/mn/iso-10303-detached-docs/sources/` (Part 21
+  grammar, Part 28 text).
+
+## Standing rules
+
+- Semantics land in Ruby; parse-adjacent and graph-scale work lands in
+  expressir-rs (TODO.parity-ee 19, option B).
+- Where eengine and the standard disagree, the standard wins; every
+  such divergence is classified in the differential output, never
+  silently copied.
+- Every ported check keeps its eengine note id for oracle
+  cross-referencing, and cites its ISO 10303-11 clause.

data/TODO.eengine/01-resolution-diagnostics.md

@@ -0,0 +1,30 @@
+# 01 — Resolution diagnostics + USE/REFERENCE tiering
+
+Folds the shipped remainder of TODO.parity-ee 06 (transitive USE and
+the cycle guard are in; `interface_visibility_spec.rb` gates them)
+with the tiering half of 07.
+
+## Gaps
+
+- USE FROM resolves **type + entity only**; REFERENCE FROM resolves
+  constant/entity/function/procedure/type (eeng
+  resolve-interface.lisp:30-66). Our resolution treats both alike.
+- Unresolved resources: eeng hard-errors `Resource '~A' not found`;
+  we warn (severity folded here from 07).
+
+## Design
+
+- `Schema#interfaced_items(kind:)` filters by interface kind.
+- `Schema#resolve_item(name, kinds:)` walks own declarations then
+  interfaces (transitively through USE for :use-from scope),
+  cycle-guarded, honoring per-kind restriction.
+- `Expressir::Express::Checker` reports unresolved references as
+  errors (`E-RES-001`, the eeng note id family) with schema+name.
+
+## Acceptance
+
+- USE-only visibility: an interface's function is NOT visible through
+  USE FROM; IS visible through REFERENCE FROM.
+- Transitive USE chains still resolve (06's shipped behavior).
+- Cycle-guarded resolution terminates.
+- The checker flags an unresolved USE FROM resource as an error.

data/TODO.eengine/02-check-catalog-completion.md

@@ -0,0 +1,53 @@
+# 02 — check-p11 catalog completion
+
+Folds TODO.parity-ee 07 + 12. Shipped subset: interface redundant,
+interface resource duplicate, subtype ref/cycle, duplicate
+declaration, where/unique name pattern, unresolved ref, select/enumeration
+extended type, self-schema reference, parse failure.
+
+## Sources
+
+- `~/src/external/exp-engine-engine/kernel/check.lisp` (835 lines,
+  per-node-type check-p11 methods) + `kernel/check-notes.lisp`.
+- ISO 10303-11:2004 clause text:
+  `~/src/mn/iso-10303-11/sources/sections/`.
+
+## Port list (diff of eeng methods vs shipped subset)
+
+Per eeng check-p11: entity attribute redeclare compatibility,
+inverse target/entity/attribute existence, derive aggregation rules,
+rule-entity reference containment, function/procedure parameter and
+return checks, subtype-constraint reflexive/rule references,
+constant expression constness. Each ported check keeps the eeng note
+id and cites its ISO 10303-11 clause.
+
+## Acceptance
+
+- eeng-differential over the SRL corpus: every eeng-emitted note of
+  the ported kinds is emitted here with matching schema/name; no
+  false positives on clean corpora (express suite green).
+
+## Status: done
+
+Shipped across three tranches:
+
+1. Original subset — interface redundant, interface resource
+   duplicate, subtype ref/cycle, duplicate declaration, where/unique
+   name pattern, unresolved ref, select/enumeration extended type,
+   self-schema reference, parse failure; inverse target/entity/
+   attribute checks; group references; aggregate bounds;
+   check-attrib-name-fun.
+2. Type-resolution tranche — constant/local/parameter/return types
+   (eeng check-constant-type, check-local-type, check-parameter-type,
+   check-fun-return-type) plus rule FOR entity refs.
+3. Remainder — subtype-constraint supertype-expression refs (eeng
+   kernel/check.lisp:426, check_supertype_ref note id) and
+   SELF\ redeclaration resolution (ISO 10303-11:2004 9.2.2.3; eeng
+   kernel/check.lisp:255 comment-only). Expression bodies are covered
+   generically by check_unresolved_refs; eengine has no separate
+   constant-expression constness rule (kernel/check.lisp only checks
+   the constant's type), so that port-list item closes as covered.
+
+Foreign-blindness rule: names that do not resolve in a single-file
+parse (USE FROM targets of unloaded schemas) are skipped, not flagged —
+both new checks verified note-free over 265 stepmod resource schemas.

data/TODO.eengine/03-entity-computations.md

@@ -0,0 +1,22 @@
+# 03 — Entity computations
+
+Matrix §3: eeng materializes inherited attributes and closures
+(kernel/compute-inheritance.lisp, derive-attrs.lisp,
+inverse-attrs.lisp, supertypes.lisp, explicit-attrs.lisp); expressir
+walks supertype chains ad hoc. Needed by Annex G.2.6 select synthesis
+(04/05) and the entity checks (02).
+
+## Design
+
+- `Entity#supertype_closure` — transitively closed supertype refs,
+  cycle-safe.
+- `Entity#inherited_explicit_attributes` — explicit attributes from
+  the full closure, redeclaration-aware (SELF\\supertype.attr
+  overrides).
+- `Entity#derived_attributes` / `Entity#inverse_attributes` —
+  accumulated from the closure.
+
+## Acceptance
+
+- chain + multiple inheritance + redeclaration fixtures; the
+  materialization matches a hand-computed table.

data/TODO.eengine/04-shtolo-rule-audit.md

@@ -0,0 +1,32 @@
+# 04 — SHTOLO rule audit + corpus differential
+
+Folds TODO.parity-ee 14's remainder.
+
+- Walk `spec/fixtures/shtolo/annex-g-rules.md` (G.NM.1-2, G.1.1-10,
+  G.2.1-7): mark every rule PASS/partial/✖ with a fixture per rule.
+- G.1.6 dependently-instantiable output verified against the Annex G
+  text (eeng ✖ there; the standard wins).
+- name-munging policy (`_dot_` and friends): eeng ✖; standard wins.
+- prune-pass reachability (unreachable subtype branches).
+- corpus differential: every modules.lst module arm/mim, flat output
+  diffed vs eeng :flat, divergences classified.
+
+## Result: DONE (2026-10-15) — the audit is a live spec
+
+`shtolo_annex_g_rule_audit_spec.rb` — 8 examples, one per previously
+uncovered rule. Audited outcome:
+
+- G.NM.2, G.1.2, G.1.5, G.1.8 (model-direct), G.2.3, G.2.4, G.2.6 —
+  PASS (G.2.4's folded extension keeps its G.NM.1-munged name —
+  correct per the name-clash rule).
+- G.1.10 — was absent; **implemented** (`add_provenance_remark!`):
+  all-or-none provenance remark on the longform schema with
+  schema + schema_version_id lines.
+- Still open (recorded in annex-g-rules.md): G.1.6
+  dependently-instantiable RULE/FUNCTION synthesis (eeng ✖; annex is
+  the contract) and the G.1.9 prune-depth notes.
+
+Fixture note: `EXTENSIBLE_ENUMERATION` as a single composite keyword
+is not in the grammar and not in the corpus — the corpus writes
+`EXTENSIBLE ENUMERATION` (302×) / `EXTENSIBLE SELECT` (546×), both
+supported.

data/TODO.eengine/05-shtolo-longform-flatten.md

@@ -0,0 +1,12 @@
+# 05 — SHTOLO longform flatten (Annex G)
+
+TODO.parity-ee 05. Convert the STEPmod multi-schema web into ONE
+ISO 10303-11:1994 longform schema: no interfaces, visibility
+preserved as rules, no name clashes, unreachable declarations pruned.
+
+Sources: `ag-single-schema.adoc` (rule ids), eeng run-op (:flat) +
+print-flatten + wo-flat.lisp (373 lines; eeng keeps structure and
+lowercases — the standard wins on divergence), add/remove-declaration
+and cleanup-interface lisps.
+
+Builds on 01 (visibility), 03 (closures), 04 (rule audit).

data/TODO.eengine/06-population-p21.md

@@ -0,0 +1,34 @@
+# 06 — Population / Part 21
+
+TODO.parity-ee 13, scope ladder (each step ships):
+
+1. P21 reader: parsanol grammar for ISO 10303-21 (HEADER entities,
+   DATA section: complex instances `(A(...)B(...))`, simple
+   `#1=TYPE(...)`, list/set/typed-ref values, enums `.T.`, binary
+   `"...`, redefines `/`); a `Model::Population` (header + instances).
+2. Population validation against a loaded schema: instance entity
+   exists (direct or complex against the supertype graph), attribute
+   names/arity match, value types check.
+3. eengine differential on the Part 21 fixtures
+   (`~/src/mn/iso-10303-detached-docs/sources/iso-10303-21*`).
+
+## Status: done
+
+1. Reader — `Expressir::P21` (parsanol grammar + PopulationBuilder):
+   HEADER FILE_* entities, DATA simple/complex instances, lists,
+   typed parameters, references, enumerations, binaries, strings
+   (doubled-quote escape), `$`/`*`, `/* */` comments.
+2. Validator — `Expressir::P21::Validator`: FILE_SCHEMA name,
+   instance keywords, complex-record supertype cliques, mapped
+   attribute arity in Part 21 order (supertype attributes first;
+   DERIVE-redeclared slots keep their position valued `*` per clause
+   12.2.6), enumeration membership, instance references,
+   typed-parameter resolution; optional short_names map for clause
+   12.3 short names.
+3. Corpus — ISO 10303-21 Annex H complete exchange structure parses
+   and validates clean (13 instances, comments, short names).
+
+eengine differential (ladder step 3) is blocked by the oracle build:
+eengine 5.1.0 --validate/--xref abort loading a bare p11 schema
+(*PARSE-MODE* NIL) and every -MODE demands a stepmod context, while
+--validate itself is flagged "not fully supported" upstream.

data/TODO.eengine/07-schema-compare-patch.md

@@ -0,0 +1,25 @@
+# 07 — Schema compare & patch
+
+TODO.parity-ee 16. Structural diff of two schemata with a
+machine-applicable patch: declarations matched by canonical path
+(`schema.item`); deltas added/removed/changed (per-attribute
+structural diff: types, optionality, aggregates); patch output
+declarable against the flattened base (eeng `--compare`/`--patch`
+semantics, plugin/p11 compare entry).
+
+## Status: done
+
+- `Expressir::Express::SchemaCompare` — deltas at `schema.item` and
+  entity-attribute `schema.item.attribute` paths; signatures are
+  lutaml-model `to_hash` trees stripped of resolution artifacts
+  (`base_path`, `ref`, `source`, remarks).
+- `Expressir::Express::SchemaPatch.apply` — rebuilds the revised
+  schema through the framework round trip; the base is never mutated;
+  item-level `:changed` nodes subsume their attribute sub-deltas.
+- Interfaces are out of scope by design: the patch target is a
+  flattened (longform) schema.
+- Oracle: `spec/expressir/express/schema_compare_oracle_spec.rb`
+  agrees with `eengine --compare -MODE RESOURCE -STEPMOD <checkout>`
+  on added/removed entities, removed attributes and enum-value type
+  changes; gated on binary + stepmod checkout presence
+  (EENGINE_BIN / STEPMOD_DIR).

data/TODO.eengine/08-official-corpus-gate.md

@@ -0,0 +1,7 @@
+# 08 — Official corpus CI gate
+
+TODO.parity-ee 17. One rake task mapping every official corpus to
+specs, with CI tiers: (a) every push — eeng bug corpus (bug39/40),
+SHTOLO rule fixtures, Part 21 grammar fixtures; (b) nightly/manual —
+SRL/SMRL full corpus differentials (pretty/flat/xml) when the eengine
+oracle is on PATH.

data/TODO.eengine/09-mim-mapping-parser.md

@@ -0,0 +1,9 @@
+# 09 — MIM mapping parser
+
+TODO.parity-ee 18 (expressir#88). Parse + validate MIM reference
+paths into a typed model: entity chains `A -> B -> C`, qualifiers
+`[attr => val]`, `=>` derived references, `->>` subtype traversal,
+`<>` select member. Sources: this checkout's
+`schemas/modules/*/mapping.yaml` (`<<express:SCHEMA.ITEM,ITEM>>` link
+markup), upstream module.xml `reference_path` bodies, the Part 1x
+mapping-annex grammar; the module corpus is the oracle.

data/TODO.parity-ee/00-overview.md

@@ -64,7 +64,7 @@ interface/qualify notes; `00check.txt` — check design notes.
 | 12 | full check-p11 catalog port | planned |
 | 13 | population / Part 21 validation | planned |
 | 14 | SHTOLO completion (Annex G + corpus differential) | in progress |
-| 15 | Part 28 XML + SMRL index | planned |
+| 15 | Part 28 XML + SMRL index | done (SMRL index + express2xsd shipped, #276; reference-XSD diff harness escalated to 17) |
 | 16 | schema compare & patch | planned |
 | 17 | official test-case incorporation | standing structure |
 

data/TODO.suma-improvements/07-part28-xml.md

@@ -1,15 +1,12 @@
-# 07 — Part 28 XML: select mapping v2 (#276)
+# 07 — Part 28 XML: select mapping v2 (#276) — DONE
 
-## Goal
-Full ISO 10303-28 EXPRESS-to-XML beyond the v1 skeleton
-(lib/expressir/express/xsd.rb): proper SELECT mapping (currently
-xs:anyType), aggregate/optional fidelity, schema-level structure.
+Shipped: the ISO 10303-28 SELECT mapping lives in
+lib/expressir/express/xsd.rb (complexType + xs:choice, entity members
+as global-element refs, defined-type members by name+type, extensible-
+empty fallback to sequence) and is covered by spec/expressir/express/
+xsd_spec.rb. The CLI exposes both halves: `expressir xsd` and
+`expressir xml-index` (#503).
 
-## Steps
-- ISO 10303-28 mapping rules for SELECT types (union/member semantics)
-- apply on top of xsd.rb's element/complexType generation
-- validate generated XSDs against the ISO 10303-28 conformance examples
-
-## Acceptance
-- a select-typed attribute renders per the standard (not xs:anyType)
-- the v1 examples still generate unchanged except where v2 improves them
+Remaining escalation (moved to parity-ee 17 scope): validating the
+generated XSDs against the Part 28 reference corpus — the detached
+docs carry the standard text only, not the reference XSDs.

data/lib/expressir/cli.rb

@@ -111,6 +111,13 @@ module Expressir
       Expressir::Commands::Xsd.new(options).run(path)
     end
 
+    desc "xml-index DIR", "Write the SMRL index XML for every EXPRESS schema under DIR"
+    method_option :output, type: :string, aliases: "-o",
+                           desc: "Output file path (defaults to stdout)"
+    def xml_index(path)
+      Expressir::Commands::SmrlXmlIndex.new(options).run(path)
+    end
+
     desc "validate SUBCOMMAND", "EXPRESS schema validation commands"
     subcommand "validate", Expressir::Commands::Validate
 

data/lib/expressir/commands/smrl_xml_index.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require "thor"
+
+module Expressir
+  module Commands
+    # `expressir xml-index` (#276): write the SMRL index XML for a
+    # repository — one <schema> element per schema, the data layer of
+    # the STEP Module Resource Library document set.
+    class SmrlXmlIndex < Base
+      def run(path)
+        repo = Expressir::Express::Parser.from_files(Dir.glob(File.join(path, "**", "*.exp")), skip_references: true)
+        raise Thor::Error, "no schemas under #{path}" if repo.schemas.empty?
+
+        xml = Expressir::Express::SmrlXml.format(repo)
+        if options[:output]
+          File.write(options[:output], xml)
+          say "written to #{options[:output]}"
+        else
+          $stdout.write(xml)
+        end
+      end
+    end
+  end
+end

data/lib/expressir/commands.rb

@@ -26,5 +26,6 @@ module Expressir
     autoload :Manifest, "#{__dir__}/commands/manifest"
     autoload :MappingValidate, "#{__dir__}/commands/mapping_validate"
     autoload :Xsd, "#{__dir__}/commands/xsd"
+    autoload :SmrlXmlIndex, "#{__dir__}/commands/smrl_xml_index"
   end
 end

data/lib/expressir/express/builder.rb

@@ -186,17 +186,29 @@ module Expressir
 
             case item
             when Array
-              item.each do |sub|
-                result << build(sub) unless sub.nil?
-              end
+              item.each { |sub| append_built(result, build(sub)) unless sub.nil? }
             else
-              built = build(item)
-              result << built if built
+              append_built(result, build(item))
             end
           end
           result
         end
 
+        # Builders for list productions (ExplicitAttrBuilder and friends)
+        # return Arrays of nodes. Splice those items into the parent list
+        # rather than nesting them — nested Arrays break every consumer
+        # that expects ModelElements (`x.kind`, tree walks, formatters).
+        def append_built(result, built)
+          case built
+          when nil
+            nil
+          when Array
+            built.each { |b| append_built(result, b) }
+          else
+            result << built
+          end
+        end
+
         # Fast build methods - call registered builder directly without hash wrapping
         # These are optimized for hot paths in expression building
         # NOTE: These assume data is already snake_case (no key conversion needed)

data/lib/expressir/express/builders/entity_decl_builder.rb

@@ -73,17 +73,17 @@ module Expressir
           if entity_body.is_a?(Hash)
             if entity_body[:explicit_attr]
               explicit_attrs = Builder.build_children(entity_body[:explicit_attr])
-              attributes.concat(explicit_attrs)
+              attributes.concat(flatten_items(explicit_attrs))
             end
 
             if entity_body[:derive_clause]
               derived = Builder.build({ derive_clause: entity_body[:derive_clause] })
-              attributes.push(derived) if derived
+              attributes.concat(flatten_items(derived))
             end
 
             if entity_body[:inverse_clause]
               inverse = Builder.build({ inverse_clause: entity_body[:inverse_clause] })
-              attributes.push(inverse) if inverse
+              attributes.concat(flatten_items(inverse))
             end
 
             if entity_body[:unique_clause]
@@ -103,13 +103,25 @@ module Expressir
             supertype_expression: supertype_expression,
             subtype_of: subtype_of,
             attributes: attributes,
-            unique_rules: [unique_rules].flatten,
-            where_rules: [where_rules].flatten,
+            unique_rules: flatten_items(unique_rules),
+            where_rules: flatten_items(where_rules),
           }
           entity_params[:abstract] = true if abstract
 
           Expressir::Model::Declarations::Entity.new(**entity_params)
         end
+
+        private
+
+        # ExplicitAttrBuilder / DeriveClauseBuilder / InverseClauseBuilder
+        # return Arrays (one node per declaration). Call sites must splice
+        # those items into the parent list — nesting them leaves
+        # `entity.attributes` as Array-of-Array, which formatters then
+        # crash on (`undefined method 'kind' for an instance of Array`).
+        # lutaml-model 0.8.107 no longer flattens collection assignments.
+        def flatten_items(value)
+          Array(value).flatten.compact
+        end
       end
     end
   end