expressir 2.4.31-x86_64-linux → 2.4.36-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 48ddbee40a47febda3a26577196114e5d64a41dbd53ead30b70a496fac792798
-  data.tar.gz: 2aa5f102ed661e63bf59768b052c9160f81f9534f9065e8b4b8d4db9dd9e5bd3
+  metadata.gz: 562234e66991852131024e0be0ba5b648391aa42c2600bf30a2b87cdcee87ed7
+  data.tar.gz: 49c531fb45fe7cb41444a377a44c8e4c86c549dd32bd8daaf3fe2852425f7941
 SHA512:
-  metadata.gz: 8636b926948dcd0a7d6e4f0d0ec18f0bb168ace9b7668574b0881597c077d5f36937f1f167e5957e1ffa9498288800465b86660e6e19b2403ce5fd9141474946
-  data.tar.gz: 55a118765bed43bda7eb42ec1d563a6cc41bb0edfafdb3f925ec670c73115eb5a1714be0279fc6309a467aeb0e840b618e1456346ae8f374fd2661e18ea7430b
+  metadata.gz: 0d9c63f577f8997f4107e644ddd0d3c3a1900fd7ef8a1285dcb1403fd356d161d12506c58480a747f68c8cf27d567d2a23983eae083b8be4794c14ffedda1fa0
+  data.tar.gz: f49f9d19130b52a44ac5486c4f0e29ad5a6549a61c3344d78681b5cf830c6af84b5d75a98acdfe16564b4b0fe5c89d8bda666a981cb7fed0052bc433ef7d0e9a

data/.github/workflows/conformance.yml

@@ -0,0 +1,34 @@
+name: conformance
+
+on:
+  schedule:
+    - cron: "0 3 * * *"
+  workflow_dispatch:
+
+jobs:
+  conformance:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/checkout@v4
+        with:
+          repository: metanorma/iso-10303-stepmod
+          path: stepmod
+          sparse-checkout: data/resources
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: "3.4"
+          bundler-cache: true
+      - name: Round-trip conformance over the stepmod corpus
+        run: bundle exec rake conformance
+        env:
+          STEPMOD_DIR: ${{ github.workspace }}/stepmod
+      - name: Trended property metrics
+        run: bundle exec rake conformance:trend
+        env:
+          STEPMOD_DIR: ${{ github.workspace }}/stepmod
+      - name: Upload the trend artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: conformance-trend
+          path: conformance-trend.json

data/.gitignore

@@ -53,3 +53,6 @@ Cargo.lock
 /lib/expressir/expressir_core.bundle
 /lib/expressir/expressir_core.so
 /lib/expressir/expressir_core.dll
+
+# nightly conformance trend artifact (rake conformance:trend)
+/conformance-trend.json

data/Gemfile

@@ -6,7 +6,7 @@ gemspec
 
 gem "canon"
 gem "irb"
-gem "lutaml-model", "0.8.76"
+gem "lutaml-model", "0.8.114"
 gem "openssl", "~> 3.0"
 gem "rake"
 gem "rake-compiler"

data/TODO.eengine/00-overview.md

@@ -0,0 +1,39 @@
+# TODO.eengine — the eengine-parity program
+
+Consolidates the remaining eengine-parity tasks (formerly scattered
+across TODO.parity-ee) into one execution-ordered series. The oracle
+is eengine-5.1.0 (GIT eeng-5.1.0) at `~/src/external/exp-engine-engine`; where
+eengine and ISO 10303-11 disagree, the standard wins and the
+divergence is classified in the differential output.
+
+## Execution order and status
+
+| n | task | folds | status |
+|---|------|-------|--------|
+| 01 | resolution diagnostics + USE/REFERENCE tiering | 06-remainder, 07-part | done |
+| 02 | check-p11 catalog completion | 07, 12 | done (type-resolution tranche; generic unresolved-ref walk covers expression bodies; subtype-constraint supertype refs + SELF redeclaration resolution added; constness has no eeng rule — covered by the generic walk) |
+| 03 | entity computations (inherited/derived/inverse closures) | matrix §3, 05-prereq | done |
+| 04 | SHTOLO rule audit + corpus differential | 14-remainder | done (audit spec live; G.NM.2 + G.1.10 fixed; G.1.6 synthesis remains per annex-g-rules.md) |
+| 05 | SHTOLO longform flatten (Annex G) | 05 | done |
+| 06 | population / Part 21 reader + validation | 13 | done (parsanol reader; schema-mapping validator; Annex H corpus clean; eengine p21 differential blocked — eeng 5.1.0 aborts bare-p11 validate/*PARSE-MODE* NIL, every -MODE demands stepmod context) |
+| 07 | schema compare & patch | 16 | done (SchemaCompare canonical-path deltas + SchemaPatch.apply; oracle differential vs eeng --compare green on add/remove entity/attribute + enum-value changes) |
+| 08 | official corpus CI gate | 17 | done |
+| 09 | MIM mapping parser | 18 | done |
+
+## Sources
+
+- eengine: `~/src/external/exp-engine-engine` (kernel/*.lisp,
+  plugins/p11/*.lisp, plugins/p21/*).
+- ISO 10303-11 clause text: `~/src/mn/iso-10303-11/sources/sections/`.
+- Detached docs: `~/src/mn/iso-10303-detached-docs/sources/` (Part 21
+  grammar, Part 28 text).
+
+## Standing rules
+
+- Semantics land in Ruby; parse-adjacent and graph-scale work lands in
+  expressir-rs (TODO.parity-ee 19, option B).
+- Where eengine and the standard disagree, the standard wins; every
+  such divergence is classified in the differential output, never
+  silently copied.
+- Every ported check keeps its eengine note id for oracle
+  cross-referencing, and cites its ISO 10303-11 clause.

data/TODO.eengine/01-resolution-diagnostics.md

@@ -0,0 +1,30 @@
+# 01 — Resolution diagnostics + USE/REFERENCE tiering
+
+Folds the shipped remainder of TODO.parity-ee 06 (transitive USE and
+the cycle guard are in; `interface_visibility_spec.rb` gates them)
+with the tiering half of 07.
+
+## Gaps
+
+- USE FROM resolves **type + entity only**; REFERENCE FROM resolves
+  constant/entity/function/procedure/type (eeng
+  resolve-interface.lisp:30-66). Our resolution treats both alike.
+- Unresolved resources: eeng hard-errors `Resource '~A' not found`;
+  we warn (severity folded here from 07).
+
+## Design
+
+- `Schema#interfaced_items(kind:)` filters by interface kind.
+- `Schema#resolve_item(name, kinds:)` walks own declarations then
+  interfaces (transitively through USE for :use-from scope),
+  cycle-guarded, honoring per-kind restriction.
+- `Expressir::Express::Checker` reports unresolved references as
+  errors (`E-RES-001`, the eeng note id family) with schema+name.
+
+## Acceptance
+
+- USE-only visibility: an interface's function is NOT visible through
+  USE FROM; IS visible through REFERENCE FROM.
+- Transitive USE chains still resolve (06's shipped behavior).
+- Cycle-guarded resolution terminates.
+- The checker flags an unresolved USE FROM resource as an error.

data/TODO.eengine/02-check-catalog-completion.md

@@ -0,0 +1,53 @@
+# 02 — check-p11 catalog completion
+
+Folds TODO.parity-ee 07 + 12. Shipped subset: interface redundant,
+interface resource duplicate, subtype ref/cycle, duplicate
+declaration, where/unique name pattern, unresolved ref, select/enumeration
+extended type, self-schema reference, parse failure.
+
+## Sources
+
+- `~/src/external/exp-engine-engine/kernel/check.lisp` (835 lines,
+  per-node-type check-p11 methods) + `kernel/check-notes.lisp`.
+- ISO 10303-11:2004 clause text:
+  `~/src/mn/iso-10303-11/sources/sections/`.
+
+## Port list (diff of eeng methods vs shipped subset)
+
+Per eeng check-p11: entity attribute redeclare compatibility,
+inverse target/entity/attribute existence, derive aggregation rules,
+rule-entity reference containment, function/procedure parameter and
+return checks, subtype-constraint reflexive/rule references,
+constant expression constness. Each ported check keeps the eeng note
+id and cites its ISO 10303-11 clause.
+
+## Acceptance
+
+- eeng-differential over the SRL corpus: every eeng-emitted note of
+  the ported kinds is emitted here with matching schema/name; no
+  false positives on clean corpora (express suite green).
+
+## Status: done
+
+Shipped across three tranches:
+
+1. Original subset — interface redundant, interface resource
+   duplicate, subtype ref/cycle, duplicate declaration, where/unique
+   name pattern, unresolved ref, select/enumeration extended type,
+   self-schema reference, parse failure; inverse target/entity/
+   attribute checks; group references; aggregate bounds;
+   check-attrib-name-fun.
+2. Type-resolution tranche — constant/local/parameter/return types
+   (eeng check-constant-type, check-local-type, check-parameter-type,
+   check-fun-return-type) plus rule FOR entity refs.
+3. Remainder — subtype-constraint supertype-expression refs (eeng
+   kernel/check.lisp:426, check_supertype_ref note id) and
+   SELF\ redeclaration resolution (ISO 10303-11:2004 9.2.2.3; eeng
+   kernel/check.lisp:255 comment-only). Expression bodies are covered
+   generically by check_unresolved_refs; eengine has no separate
+   constant-expression constness rule (kernel/check.lisp only checks
+   the constant's type), so that port-list item closes as covered.
+
+Foreign-blindness rule: names that do not resolve in a single-file
+parse (USE FROM targets of unloaded schemas) are skipped, not flagged —
+both new checks verified note-free over 265 stepmod resource schemas.

data/TODO.eengine/03-entity-computations.md

@@ -0,0 +1,22 @@
+# 03 — Entity computations
+
+Matrix §3: eeng materializes inherited attributes and closures
+(kernel/compute-inheritance.lisp, derive-attrs.lisp,
+inverse-attrs.lisp, supertypes.lisp, explicit-attrs.lisp); expressir
+walks supertype chains ad hoc. Needed by Annex G.2.6 select synthesis
+(04/05) and the entity checks (02).
+
+## Design
+
+- `Entity#supertype_closure` — transitively closed supertype refs,
+  cycle-safe.
+- `Entity#inherited_explicit_attributes` — explicit attributes from
+  the full closure, redeclaration-aware (SELF\\supertype.attr
+  overrides).
+- `Entity#derived_attributes` / `Entity#inverse_attributes` —
+  accumulated from the closure.
+
+## Acceptance
+
+- chain + multiple inheritance + redeclaration fixtures; the
+  materialization matches a hand-computed table.

data/TODO.eengine/04-shtolo-rule-audit.md

@@ -0,0 +1,32 @@
+# 04 — SHTOLO rule audit + corpus differential
+
+Folds TODO.parity-ee 14's remainder.
+
+- Walk `spec/fixtures/shtolo/annex-g-rules.md` (G.NM.1-2, G.1.1-10,
+  G.2.1-7): mark every rule PASS/partial/✖ with a fixture per rule.
+- G.1.6 dependently-instantiable output verified against the Annex G
+  text (eeng ✖ there; the standard wins).
+- name-munging policy (`_dot_` and friends): eeng ✖; standard wins.
+- prune-pass reachability (unreachable subtype branches).
+- corpus differential: every modules.lst module arm/mim, flat output
+  diffed vs eeng :flat, divergences classified.
+
+## Result: DONE (2026-10-15) — the audit is a live spec
+
+`shtolo_annex_g_rule_audit_spec.rb` — 8 examples, one per previously
+uncovered rule. Audited outcome:
+
+- G.NM.2, G.1.2, G.1.5, G.1.8 (model-direct), G.2.3, G.2.4, G.2.6 —
+  PASS (G.2.4's folded extension keeps its G.NM.1-munged name —
+  correct per the name-clash rule).
+- G.1.10 — was absent; **implemented** (`add_provenance_remark!`):
+  all-or-none provenance remark on the longform schema with
+  schema + schema_version_id lines.
+- Still open (recorded in annex-g-rules.md): G.1.6
+  dependently-instantiable RULE/FUNCTION synthesis (eeng ✖; annex is
+  the contract) and the G.1.9 prune-depth notes.
+
+Fixture note: `EXTENSIBLE_ENUMERATION` as a single composite keyword
+is not in the grammar and not in the corpus — the corpus writes
+`EXTENSIBLE ENUMERATION` (302×) / `EXTENSIBLE SELECT` (546×), both
+supported.

data/TODO.eengine/05-shtolo-longform-flatten.md

@@ -0,0 +1,12 @@
+# 05 — SHTOLO longform flatten (Annex G)
+
+TODO.parity-ee 05. Convert the STEPmod multi-schema web into ONE
+ISO 10303-11:1994 longform schema: no interfaces, visibility
+preserved as rules, no name clashes, unreachable declarations pruned.
+
+Sources: `ag-single-schema.adoc` (rule ids), eeng run-op (:flat) +
+print-flatten + wo-flat.lisp (373 lines; eeng keeps structure and
+lowercases — the standard wins on divergence), add/remove-declaration
+and cleanup-interface lisps.
+
+Builds on 01 (visibility), 03 (closures), 04 (rule audit).

data/TODO.eengine/06-population-p21.md

@@ -0,0 +1,34 @@
+# 06 — Population / Part 21
+
+TODO.parity-ee 13, scope ladder (each step ships):
+
+1. P21 reader: parsanol grammar for ISO 10303-21 (HEADER entities,
+   DATA section: complex instances `(A(...)B(...))`, simple
+   `#1=TYPE(...)`, list/set/typed-ref values, enums `.T.`, binary
+   `"...`, redefines `/`); a `Model::Population` (header + instances).
+2. Population validation against a loaded schema: instance entity
+   exists (direct or complex against the supertype graph), attribute
+   names/arity match, value types check.
+3. eengine differential on the Part 21 fixtures
+   (`~/src/mn/iso-10303-detached-docs/sources/iso-10303-21*`).
+
+## Status: done
+
+1. Reader — `Expressir::P21` (parsanol grammar + PopulationBuilder):
+   HEADER FILE_* entities, DATA simple/complex instances, lists,
+   typed parameters, references, enumerations, binaries, strings
+   (doubled-quote escape), `$`/`*`, `/* */` comments.
+2. Validator — `Expressir::P21::Validator`: FILE_SCHEMA name,
+   instance keywords, complex-record supertype cliques, mapped
+   attribute arity in Part 21 order (supertype attributes first;
+   DERIVE-redeclared slots keep their position valued `*` per clause
+   12.2.6), enumeration membership, instance references,
+   typed-parameter resolution; optional short_names map for clause
+   12.3 short names.
+3. Corpus — ISO 10303-21 Annex H complete exchange structure parses
+   and validates clean (13 instances, comments, short names).
+
+eengine differential (ladder step 3) is blocked by the oracle build:
+eengine 5.1.0 --validate/--xref abort loading a bare p11 schema
+(*PARSE-MODE* NIL) and every -MODE demands a stepmod context, while
+--validate itself is flagged "not fully supported" upstream.

data/TODO.eengine/07-schema-compare-patch.md

@@ -0,0 +1,25 @@
+# 07 — Schema compare & patch
+
+TODO.parity-ee 16. Structural diff of two schemata with a
+machine-applicable patch: declarations matched by canonical path
+(`schema.item`); deltas added/removed/changed (per-attribute
+structural diff: types, optionality, aggregates); patch output
+declarable against the flattened base (eeng `--compare`/`--patch`
+semantics, plugin/p11 compare entry).
+
+## Status: done
+
+- `Expressir::Express::SchemaCompare` — deltas at `schema.item` and
+  entity-attribute `schema.item.attribute` paths; signatures are
+  lutaml-model `to_hash` trees stripped of resolution artifacts
+  (`base_path`, `ref`, `source`, remarks).
+- `Expressir::Express::SchemaPatch.apply` — rebuilds the revised
+  schema through the framework round trip; the base is never mutated;
+  item-level `:changed` nodes subsume their attribute sub-deltas.
+- Interfaces are out of scope by design: the patch target is a
+  flattened (longform) schema.
+- Oracle: `spec/expressir/express/schema_compare_oracle_spec.rb`
+  agrees with `eengine --compare -MODE RESOURCE -STEPMOD <checkout>`
+  on added/removed entities, removed attributes and enum-value type
+  changes; gated on binary + stepmod checkout presence
+  (EENGINE_BIN / STEPMOD_DIR).

data/TODO.eengine/08-official-corpus-gate.md

@@ -0,0 +1,7 @@
+# 08 — Official corpus CI gate
+
+TODO.parity-ee 17. One rake task mapping every official corpus to
+specs, with CI tiers: (a) every push — eeng bug corpus (bug39/40),
+SHTOLO rule fixtures, Part 21 grammar fixtures; (b) nightly/manual —
+SRL/SMRL full corpus differentials (pretty/flat/xml) when the eengine
+oracle is on PATH.

data/TODO.eengine/09-mim-mapping-parser.md

@@ -0,0 +1,9 @@
+# 09 — MIM mapping parser
+
+TODO.parity-ee 18 (expressir#88). Parse + validate MIM reference
+paths into a typed model: entity chains `A -> B -> C`, qualifiers
+`[attr => val]`, `=>` derived references, `->>` subtype traversal,
+`<>` select member. Sources: this checkout's
+`schemas/modules/*/mapping.yaml` (`<<express:SCHEMA.ITEM,ITEM>>` link
+markup), upstream module.xml `reference_path` bodies, the Part 1x
+mapping-annex grammar; the module corpus is the oracle.

data/TODO.parity-ee/00-overview.md

@@ -64,7 +64,7 @@ interface/qualify notes; `00check.txt` — check design notes.
 | 12 | full check-p11 catalog port | planned |
 | 13 | population / Part 21 validation | planned |
 | 14 | SHTOLO completion (Annex G + corpus differential) | in progress |
-| 15 | Part 28 XML + SMRL index | planned |
+| 15 | Part 28 XML + SMRL index | done (SMRL index + express2xsd shipped, #276; reference-XSD diff harness escalated to 17) |
 | 16 | schema compare & patch | planned |
 | 17 | official test-case incorporation | standing structure |
 

data/TODO.sota/00-overview.md

@@ -0,0 +1,57 @@
+# TODO.sota — the SOTA program (vision)
+
+Expressir is the open compiler-infrastructure layer for ISO 10303
+information models: a faithful, programmable model of EXPRESS
+(ISO 10303-11), the standard transformations between the language's
+normative forms (Annex G longform, Part 21, Part 28/XML, XSD, MIM
+mappings), and CI-grade guarantees for the standards corpus. The
+eengine-parity program (TODO.eengine, done) made it the ecosystem's
+reference toolchain. This program takes it to SOTA — the point where
+Expressir is not just the ecosystem's parser but its *engine*, its
+developer platform, and its interoperability hub.
+
+## Execution order and status
+
+| n | task | tier | status |
+|---|------|------|--------|
+| 01 | static type checking of expressions | semantics | done (tiers 1-6; :check_expr_operand_domain/:check_expr_initializer_type/:check_expr_assignment_type/:check_expr_return/:check_expr_where_boolean/:check_expr_unique_duplicate; corpus 261-slice 0 notes) |
+| 02 | evaluator — run EXPRESS (functions, WHERE rules over populations) | execution | done (value model + ch.12 expressions, ch.13 interpreter + builtin set A, instance-aware eval over P21 populations, Rules.check with SELF; eengine validate broken — mutation-sweep differential) |
+| 03 | conformance (Part 21 conformance corpus, property-based grammar harness, G.1.7 closure) | rigor | done (P21 annex corpus + negative ladder-exclusion gate; generated round-trip harness + nightly corpus tier + rake conformance; every Annex G rule audited) |
+| 04 | LSP + incremental platform | platform | done (stdio LSP 3.17 server + `expressir-lsp` binary; Checker diagnostics; USE/alias definition, references, hover; cross-schema rename verified by SchemaCompare+Formatter; full reparse — expr-rs incremental API is the measured gap) |
+| 05 | interop exports (JSON Schema, OWL/RDF) | interop | done (JSON Schema 2020-12 + OWL/Turtle exporters, CLI commands, decision guides; Annex H population consumer round-trip; SRL corpus tier; UML/XMI optional not shipped) |
+| 06 | dialect awareness (1994 vs 2004, amendments) | semantics | done (Dialect normalize + parser/checker flag; :check_dialect_2004_only/:check_dialect_deprecated/:check_dialect_mixed; audited 2004 change list fixture with live examples; both-dialect corpus tier) |
+| 07 | reference artifacts (grammar cross-walk, published guides) | reference | done (250-rule generated cross-walk with divergence classes + sync spec; trended nightly metrics artifact; checker/p21/shtolo/evaluator/lsp guides with generated note-id catalog) |
+
+Dependency shape: 02 consumes 01's type tiers loosely (interleave by
+tier when a check needs execution semantics). 04 scaffolds
+independently; its diagnostics richness grows with 01. 03/05/06/07 are
+independent.
+
+## Sources
+
+- ISO 10303-11 text: `~/src/mn/iso-10303-11/sources/sections/`
+  (expressions ch.12, statements ch.13, data types ch.8, rules 9.x,
+  Annex A/B productions).
+- Part 21: `~/src/mn/iso-10303-detached-docs/sources/iso-10303-21/`.
+- stepmod corpus: `~/src/mn/iso-10303-stepmod/data/` (SRL/BOM/module
+  manifests; the 265-file <250KB resource slice is the fast
+  differential set).
+- eengine oracle: `~/bin/eengine` — schema ops need
+  `-MODE RESOURCE -STEPMOD ~/src/mn/iso-10303-stepmod`;
+  --validate/--xref are broken on bare p11 (never build on them).
+- leptris runs FFI-fallback on this box; no spec may assume the native
+  engine.
+
+## Standing rules
+
+- Where eengine and ISO 10303-11 disagree, the standard wins; classify
+  the divergence in the differential output, never copy silently.
+- Every ported check keeps its eengine note id (where one exists) and
+  cites its ISO 10303-11 clause.
+- Semantics land in Ruby; parse-adjacent and graph-scale hot paths may
+  land in expressir-rs (gated per task).
+- All model (de)serialization goes through lutaml-model mappings —
+  never hand-rolled to_h/from_h.
+- Specs use real model instances, never doubles; rubocop clean is a
+  merge gate; the suite runs in chunks on this box (full single-process
+  runs OOM).

data/TODO.sota/01-type-checking.md

@@ -0,0 +1,115 @@
+# 01 — Static type checking of expressions
+
+The Checker resolves references (TODO.eengine/02/03); this task gives
+it TYPES: full static checking of the expression and statement
+language per ISO 10303-11 ch.12/13/8/9.
+
+## Tiers (each ships)
+
+1. Clause map: expression grammar ↔ ISO 10303-11 12.x productions ↔
+   eeng check.lisp expression methods, with the typing rule for each
+   (operand domains, result type). Table in this file before code.
+2. Operator domains: arithmetic on the NUMBER family, logical on
+   BOOLEAN/LOGICAL, relational polymorphism, string/binary ops,
+   aggregate operators (INDEX/QUERY/SUM and friends).
+3. Aggregates: element types through indexing/slicing, bounds vs
+   literals where static, nested aggregates.
+4. SELECT/ENUM membership: assignment compatibility, IN-of-enum,
+   `x : t := expr` compatibility in CONSTANT/LOCAL/DERIVE initializers.
+5. Statement flow: assignment compatibility, IF/CASE selector types,
+   REPEAT control types, function RETURN type vs return statements.
+6. WHERE/UNIQUE structural checks: expression boolean-ness, UNIQUE
+   label uniqueness, rule FOR-entity containment for referenced
+   attributes.
+
+## Interface
+
+`Checker#check_expression_types(schema)` emitting notes with eeng ids
+where check.lisp has them, own ids (`:check_expr_*`) citing clauses
+where it does not (mirror TODO.eengine/02's SELF-redeclaration
+precedent).
+
+## Acceptance
+
+- Corpus-clean over the 265-file resource slice (foreign-blindness
+  rule applies: unresolvable names skipped, not flagged).
+- eeng differential for the expression checks eengine actually
+  implements; standard wins on divergence.
+
+## Clause map (tier 2 — operator domains)
+
+Model operator vocabulary: `BinaryExpression` constants (ADDITION,
+SUBTRACTION, MULTIPLICATION, REAL_DIVISION, INTEGER_DIVISION, MODULO,
+EXPONENTIATION, COMBINE, EQUAL, NOT_EQUAL, LESS_THAN,
+LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL, IN, LIKE,
+INSTANCE_EQUAL, INSTANCE_NOT_EQUAL, AND, OR, XOR);
+`UnaryExpression` (PLUS, MINUS, NOT); `Interval`.
+
+| operator | ISO 10303-11:2004 clause | operand domains | result |
+|----------|--------------------------|-----------------|--------|
+| unary `+` `-` (PLUS/MINUS) | 12.1 | numeric | numeric |
+| `not` (NOT) | 12.4 | logical | logical |
+| `+` (ADDITION) | 12.1 add; 12.5.2 string concat; 12.3.2 binary concat; 12.6 union (table 17 admits aggregate+element singletons on either side) | num,num; str,str; bin,bin; agg,agg; agg,elem; elem,agg | num / str / bin / agg |
+| `-` (SUBTRACTION) | 12.1; 12.6 difference (table 18: aggregate left, element singleton right) | num,num; agg,agg; agg,elem | num / agg |
+| `*` (MULTIPLICATION) | 12.1; 12.6 intersection (both operands aggregates) | num,num; agg,agg | num / agg |
+| `/` (REAL_DIVISION) | 12.1 | numeric | real |
+| `**` (EXPONENTIATION) | 12.1 | numeric | num (int if both int) |
+| `div` `mod` | 12.1 | numeric (reals truncated) | integer |
+| `and` `or` `xor` | 12.4 | logical,logical | logical |
+| `=` `<>` | 12.2.1 | comparable: numeric, logical, string, binary, enum; aggregates and entities also admitted | logical |
+| `<` `<=` `>` `>=` | 12.2.1; 12.6 subset/superset | ordering families only (numeric, logical, string, binary, non-extensible enum); if either side is aggregate, subset/superset semantics (agg,agg) | logical |
+| `:=:` `:<>>:` (INSTANCE_EQUAL/NOT_EQUAL) | 12.2.2 | instance types (entity, aggregate) | logical |
+| `in` (IN) | 12.2.3 | rhs aggregate; lhs compatible with rhs base | logical |
+| `like` (LIKE) | 12.2.5 | string,string | logical |
+| `\|\|` (COMBINE) | 12 intro (precedence table note) | complex entity construction (entity operands) | entity |
+| interval | 12.2.4 | low/item/high mutually comparable (numeric, string, enum) | logical |
+
+Static type lattice: `:integer :real :number :logical :boolean :string
+:binary :enum :aggregate :entity :select :generic :unknown`. Inference:
+literals direct; declared types via the schema's types (SimpleReference
+→ find_type → underlying_type recursion); references via the enclosing
+declaration's environment (constants, parameters, variables, entity
+attributes); enum members are AttributeReferences whose base resolves
+to an enumeration TYPE; function calls resolve to the function's
+return type.
+
+Flagging rule: a domain violation is noted only when BOTH operand
+types are known (`:generic` and `:unknown` never flag) — the corpus
+never produces false positives.
+
+eengine divergence: kernel/check.lisp's expression check-p11 methods
+traverse structurally only — eengine performs NO operator-domain
+typing. These checks therefore carry new `:check_expr_operand_domain`
+ids citing their clauses; there is no eengine note id to keep.
+
+- Tier 1 clause map: done (above).
+- Tier 2 operator domains: done — `Checker#check_expression_types`,
+  note id `:check_expr_operand_domain` with the clause in the message;
+  spec `spec/expressir/express/checker_expression_types_spec.rb`
+  (9 examples); corpus-clean over the 265-file resource slice (255
+  scanned, 0 notes) after clause 12.6 table 17's union-with-singleton
+  was admitted.
+- Tier 3 indexing: done — IndexReference resolves to the aggregate's
+  base element (12.6.1/12.3.1/12.5.1); index positions numeric
+  (production 236); the difference operator admits an element
+  singleton right operand with an aggregate left (12.6 table 18).
+- Tier 4 initializer compatibility: done — CONSTANT/DERIVE/LOCAL
+  initializers checked against their declared types (12.11): numeric
+  widening integer→real→number, logical/boolean family, enum identity
+  via declaration resolution, entity specialization via
+  supertype_closure, select-accepting slots, aggregate slots. Corpus
+  clean (261 schemas, 0 notes).
+- Tier 5 statement flow: done — assignment compatibility (13.2.1),
+  if conditions logical (13.5), repeat controls (13.6), return
+  compatibility + procedure-return restriction (13.7).
+- Tier 6 WHERE/UNIQUE structure: done — WHERE expressions logical
+  (9.4), duplicate UNIQUE labels (9.4); rule FOR-entity containment
+  remains covered by bound_identifiers' deliberate over-approximation.
+
+## Status: done
+
+Engine finding: parsing identical source after many interleaved
+check-parse cycles in one process can diverge (attr type arrives as a
+SimpleReference instead of the builtin node) — a parsanol parse-cache
+issue, worked around in the initializer spec by checking in a fresh
+subprocess; separate investigation owed.

data/TODO.sota/02-evaluator.md

@@ -0,0 +1,86 @@
+# 02 — The evaluator: run EXPRESS
+
+The SOTA-defining capability: execute the language. Functions and
+WHERE rules evaluated over Part 21 populations — the line between a
+toolchain that reads EXPRESS and one that runs it. No open
+implementation has this (eengine's --validate is the cautionary tale).
+
+## Steps (each ships)
+
+1. Value model + expression core: IMMUTABLE value classes (NUMBER
+   family, STRING/BINARY, LOGICAL, enumeration, aggregate arrays as
+   frozen structs), literal → value, pure expression evaluation
+   (operators per 01's domain table), environmental lookup through a
+   Scope chain. Interface: `Expressir::Eval.expression(ast, env)`.
+2. Function interpreter: parameter binding (incl. VAR semantics),
+   LOCALs with initializers, IF/CASE/REPEAT/compound/RETURN, ESCAPE/
+   SKIP, recursion with depth guard. Builtin set A (math/string,
+   ISO 10303-11 15.x): SIN..BLENGTH class.
+3. Instance-aware builtins: TYPEOF, USEDIN, EXISTS, QUERY (lazy
+   binding), attribute access through the model's computation layer
+   (Entity#all_explicit_attributes, supertype_closure, redeclaration
+   resolution — TODO.eengine/03), complex-instance (GROUP) access.
+   Builtin set B (instance/aggregate: SIZEOF, HIBOUND..).
+4. WHERE-rule checking: `Expressir::Rules.check(population, schema)`
+   → per-instance, per-rule results with the failing subexpression
+   path. Wraps the P21 Validator's structural tiers (TODO.eengine/06)
+   into semantic validation.
+5. Differential: eengine where its validate answers; property-based
+   round-trips otherwise (evaluate a rule against a corpus population,
+   cross-check pure-Ruby reference impls for builtins). If hot loops
+   dominate, drop the evaluation core to expressir-rs (option B) —
+   gate on measured need, not speculation.
+
+## Design constraints
+
+- The evaluator consumes the SAME model the parser produces — no
+  parallel AST.
+- Errors are notes/exceptions carrying node + path, never exit.
+- Determinism: no reliance on hash order (corpus QA depends on it).
+
+## Acceptance
+
+- Annex-level worked examples (function bodies from ISO 10303-11)
+  evaluate to the documented results.
+- WHERE evaluation over the Annex H population: every rule provably
+  passes; a seeded-fault population fails the intended rule only.
+
+## Status: in progress
+
+- Step 1 (value model + expression core): done — `Expressir::Eval`
+  (Values/Environment/Expression autoloads): ISO 10303-11 ch.12
+  arithmetic with result typing, logical tables (12.4), relational
+  orderings (12.2.1), IN (12.2.3), LIKE (12.2.5, table 11 patterns),
+  intervals, aggregates (tables 16-18 incl. element singletons), and
+  QUERY. Indeterminate (`?`) propagates per clause. Function calls
+  raise until step 2. Spec `spec/expressir/eval/expression_spec.rb`.
+- Grammar bug fixed en route: `factor`'s `** rhs` capture was dropped
+  by the builder (`2 ** 3 + 1` parsed as `2 + 1`); build_factor now
+  emits EXPONENTIATION.
+- Step 2 (function interpreter + builtin set A): done — functions
+  with parameter/local binding, assignment (incl. indexed), IF/CASE,
+  WHILE/UNTIL/increment REPEAT with SKIP and env-carrying ESCAPE,
+  RETURN (falling off the end is an error per 13.7), recursion.
+  Builtins: ABS SIN COS TAN ASIN ACOS ATAN ATAN2 SQRT EXP LOG LOG2
+  LOG10 ODD LENGTH BLENGTH SIZEOF HIINDEX LOINDEX HIBOUND LOBOUND
+  EXISTS NVL VALUE, plus PI/E/CONST_E constants. NOTE: the logical
+  value symbols live in Values::LOGICAL_* constants — bare :true/
+  :false literals get rewritten to booleans by Lint/BooleanSymbol.
+- Step 3 (instance-aware evaluation): done — Values::Instance over
+  P21 population records with P21-data-to-value conversion (arrays →
+  aggregates, `#id` → instances, enums/binaries mapped); attribute
+  access through the Environment's Part 21 slot mapping (inherited
+  then own, same order as the P21 validator, incl. complex records);
+  TYPEOF (unqualified + schema-qualified names), USEDIN, EXISTS.
+  Fixtures: spec/fixtures/eval/population.{exp,p21}.
+- Step 4 (WHERE-rule checking): done — `Expressir::Eval::Rules.check`
+  evaluates every instance's entity domain rules with SELF and the
+  mapped attributes bound (bare names and SELF\attr both resolve);
+  statuses pass/fail/unknown/error; a rule is violated only when it
+  evaluates false (9.4), indeterminate is not a violation.
+- Step 5 (differential): done as substituted — eengine's --validate
+  aborts on bare p11 (TODO.eengine/06 step 3 finding), so the
+  differential face is the mutation sweep: corrupting one instance
+  attribute fails exactly the intended rule and no other (rules spec).
+
+## Status: done