expressir 2.4.35 → 2.4.37

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2fef762a5f189a2fa5e19af4e03b508ceecb9c51f4f70a805af7b0cfaa955c66
-  data.tar.gz: f4bfae05b286248e59f3bb54c83369c1c1bba957edeb5dc1e9e0aa8e7d0bc147
+  metadata.gz: 46faf8f6bb37c2f1849a2d131b1489e35deed5511f07c8ec873cd07057f7bf79
+  data.tar.gz: e123c04c9745923412e7119140130e07805836aea196848420b11df6871cf93f
 SHA512:
-  metadata.gz: 75be497abf1fc565aa7098271d933715ab0e2ccaa2eed76eda589d37fe511cb8b52a2c08ffa537e23011cb847d14c9a9647c99e814473cdb5820577e61b31af1
-  data.tar.gz: 5420b2a0edfd9f4135515c75cf163b68dbb147e6b60fd7900528671862672f3695936884ea919692036c8e09eabb60f85c13cc939c61fdc67c9ec8ad2e1571cd
+  metadata.gz: 7531e5fe41e6c6a338d0157837cba39f9d23cecece891517963e122fec9cd90dc01d8901689b171f2e68798788da19bee80cc25204cb42b786df8c55ea1ac27c
+  data.tar.gz: c6b759368286b4562933e4871e735df7347fcda3ba548215bb28d574f416db8e00c9b8958966fd650878d538a13962be79c27ed2d002e4ddc07597bb3983c88f

data/.github/workflows/conformance.yml

@@ -0,0 +1,34 @@
+name: conformance
+
+on:
+  schedule:
+    - cron: "0 3 * * *"
+  workflow_dispatch:
+
+jobs:
+  conformance:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/checkout@v4
+        with:
+          repository: metanorma/iso-10303-stepmod
+          path: stepmod
+          sparse-checkout: data/resources
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: "3.4"
+          bundler-cache: true
+      - name: Round-trip conformance over the stepmod corpus
+        run: bundle exec rake conformance
+        env:
+          STEPMOD_DIR: ${{ github.workspace }}/stepmod
+      - name: Trended property metrics
+        run: bundle exec rake conformance:trend
+        env:
+          STEPMOD_DIR: ${{ github.workspace }}/stepmod
+      - name: Upload the trend artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: conformance-trend
+          path: conformance-trend.json

data/.gitignore

@@ -53,3 +53,6 @@ Cargo.lock
 /lib/expressir/expressir_core.bundle
 /lib/expressir/expressir_core.so
 /lib/expressir/expressir_core.dll
+
+# nightly conformance trend artifact (rake conformance:trend)
+/conformance-trend.json

data/Gemfile

@@ -6,7 +6,7 @@ gemspec
 
 gem "canon"
 gem "irb"
-gem "lutaml-model", "0.8.76"
+gem "lutaml-model", "0.8.114"
 gem "openssl", "~> 3.0"
 gem "rake"
 gem "rake-compiler"

data/TODO.sota/00-overview.md

@@ -0,0 +1,57 @@
+# TODO.sota — the SOTA program (vision)
+
+Expressir is the open compiler-infrastructure layer for ISO 10303
+information models: a faithful, programmable model of EXPRESS
+(ISO 10303-11), the standard transformations between the language's
+normative forms (Annex G longform, Part 21, Part 28/XML, XSD, MIM
+mappings), and CI-grade guarantees for the standards corpus. The
+eengine-parity program (TODO.eengine, done) made it the ecosystem's
+reference toolchain. This program takes it to SOTA — the point where
+Expressir is not just the ecosystem's parser but its *engine*, its
+developer platform, and its interoperability hub.
+
+## Execution order and status
+
+| n | task | tier | status |
+|---|------|------|--------|
+| 01 | static type checking of expressions | semantics | done (tiers 1-6; :check_expr_operand_domain/:check_expr_initializer_type/:check_expr_assignment_type/:check_expr_return/:check_expr_where_boolean/:check_expr_unique_duplicate; corpus 261-slice 0 notes) |
+| 02 | evaluator — run EXPRESS (functions, WHERE rules over populations) | execution | done (value model + ch.12 expressions, ch.13 interpreter + builtin set A, instance-aware eval over P21 populations, Rules.check with SELF; eengine validate broken — mutation-sweep differential) |
+| 03 | conformance (Part 21 conformance corpus, property-based grammar harness, G.1.7 closure) | rigor | done (P21 annex corpus + negative ladder-exclusion gate; generated round-trip harness + nightly corpus tier + rake conformance; every Annex G rule audited) |
+| 04 | LSP + incremental platform | platform | done (stdio LSP 3.17 server + `expressir-lsp` binary; Checker diagnostics; USE/alias definition, references, hover; cross-schema rename verified by SchemaCompare+Formatter; full reparse — expr-rs incremental API is the measured gap) |
+| 05 | interop exports (JSON Schema, OWL/RDF) | interop | done (JSON Schema 2020-12 + OWL/Turtle exporters, CLI commands, decision guides; Annex H population consumer round-trip; SRL corpus tier; UML/XMI optional not shipped) |
+| 06 | dialect awareness (1994 vs 2004, amendments) | semantics | done (Dialect normalize + parser/checker flag; :check_dialect_2004_only/:check_dialect_deprecated/:check_dialect_mixed; audited 2004 change list fixture with live examples; both-dialect corpus tier) |
+| 07 | reference artifacts (grammar cross-walk, published guides) | reference | done (250-rule generated cross-walk with divergence classes + sync spec; trended nightly metrics artifact; checker/p21/shtolo/evaluator/lsp guides with generated note-id catalog) |
+
+Dependency shape: 02 consumes 01's type tiers loosely (interleave by
+tier when a check needs execution semantics). 04 scaffolds
+independently; its diagnostics richness grows with 01. 03/05/06/07 are
+independent.
+
+## Sources
+
+- ISO 10303-11 text: `~/src/mn/iso-10303-11/sources/sections/`
+  (expressions ch.12, statements ch.13, data types ch.8, rules 9.x,
+  Annex A/B productions).
+- Part 21: `~/src/mn/iso-10303-detached-docs/sources/iso-10303-21/`.
+- stepmod corpus: `~/src/mn/iso-10303-stepmod/data/` (SRL/BOM/module
+  manifests; the 265-file <250KB resource slice is the fast
+  differential set).
+- eengine oracle: `~/bin/eengine` — schema ops need
+  `-MODE RESOURCE -STEPMOD ~/src/mn/iso-10303-stepmod`;
+  --validate/--xref are broken on bare p11 (never build on them).
+- leptris runs FFI-fallback on this box; no spec may assume the native
+  engine.
+
+## Standing rules
+
+- Where eengine and ISO 10303-11 disagree, the standard wins; classify
+  the divergence in the differential output, never copy silently.
+- Every ported check keeps its eengine note id (where one exists) and
+  cites its ISO 10303-11 clause.
+- Semantics land in Ruby; parse-adjacent and graph-scale hot paths may
+  land in expressir-rs (gated per task).
+- All model (de)serialization goes through lutaml-model mappings —
+  never hand-rolled to_h/from_h.
+- Specs use real model instances, never doubles; rubocop clean is a
+  merge gate; the suite runs in chunks on this box (full single-process
+  runs OOM).

data/TODO.sota/01-type-checking.md

@@ -0,0 +1,115 @@
+# 01 — Static type checking of expressions
+
+The Checker resolves references (TODO.eengine/02/03); this task gives
+it TYPES: full static checking of the expression and statement
+language per ISO 10303-11 ch.12/13/8/9.
+
+## Tiers (each ships)
+
+1. Clause map: expression grammar ↔ ISO 10303-11 12.x productions ↔
+   eeng check.lisp expression methods, with the typing rule for each
+   (operand domains, result type). Table in this file before code.
+2. Operator domains: arithmetic on the NUMBER family, logical on
+   BOOLEAN/LOGICAL, relational polymorphism, string/binary ops,
+   aggregate operators (INDEX/QUERY/SUM and friends).
+3. Aggregates: element types through indexing/slicing, bounds vs
+   literals where static, nested aggregates.
+4. SELECT/ENUM membership: assignment compatibility, IN-of-enum,
+   `x : t := expr` compatibility in CONSTANT/LOCAL/DERIVE initializers.
+5. Statement flow: assignment compatibility, IF/CASE selector types,
+   REPEAT control types, function RETURN type vs return statements.
+6. WHERE/UNIQUE structural checks: expression boolean-ness, UNIQUE
+   label uniqueness, rule FOR-entity containment for referenced
+   attributes.
+
+## Interface
+
+`Checker#check_expression_types(schema)` emitting notes with eeng ids
+where check.lisp has them, own ids (`:check_expr_*`) citing clauses
+where it does not (mirror TODO.eengine/02's SELF-redeclaration
+precedent).
+
+## Acceptance
+
+- Corpus-clean over the 265-file resource slice (foreign-blindness
+  rule applies: unresolvable names skipped, not flagged).
+- eeng differential for the expression checks eengine actually
+  implements; standard wins on divergence.
+
+## Clause map (tier 2 — operator domains)
+
+Model operator vocabulary: `BinaryExpression` constants (ADDITION,
+SUBTRACTION, MULTIPLICATION, REAL_DIVISION, INTEGER_DIVISION, MODULO,
+EXPONENTIATION, COMBINE, EQUAL, NOT_EQUAL, LESS_THAN,
+LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL, IN, LIKE,
+INSTANCE_EQUAL, INSTANCE_NOT_EQUAL, AND, OR, XOR);
+`UnaryExpression` (PLUS, MINUS, NOT); `Interval`.
+
+| operator | ISO 10303-11:2004 clause | operand domains | result |
+|----------|--------------------------|-----------------|--------|
+| unary `+` `-` (PLUS/MINUS) | 12.1 | numeric | numeric |
+| `not` (NOT) | 12.4 | logical | logical |
+| `+` (ADDITION) | 12.1 add; 12.5.2 string concat; 12.3.2 binary concat; 12.6 union (table 17 admits aggregate+element singletons on either side) | num,num; str,str; bin,bin; agg,agg; agg,elem; elem,agg | num / str / bin / agg |
+| `-` (SUBTRACTION) | 12.1; 12.6 difference (table 18: aggregate left, element singleton right) | num,num; agg,agg; agg,elem | num / agg |
+| `*` (MULTIPLICATION) | 12.1; 12.6 intersection (both operands aggregates) | num,num; agg,agg | num / agg |
+| `/` (REAL_DIVISION) | 12.1 | numeric | real |
+| `**` (EXPONENTIATION) | 12.1 | numeric | num (int if both int) |
+| `div` `mod` | 12.1 | numeric (reals truncated) | integer |
+| `and` `or` `xor` | 12.4 | logical,logical | logical |
+| `=` `<>` | 12.2.1 | comparable: numeric, logical, string, binary, enum; aggregates and entities also admitted | logical |
+| `<` `<=` `>` `>=` | 12.2.1; 12.6 subset/superset | ordering families only (numeric, logical, string, binary, non-extensible enum); if either side is aggregate, subset/superset semantics (agg,agg) | logical |
+| `:=:` `:<>>:` (INSTANCE_EQUAL/NOT_EQUAL) | 12.2.2 | instance types (entity, aggregate) | logical |
+| `in` (IN) | 12.2.3 | rhs aggregate; lhs compatible with rhs base | logical |
+| `like` (LIKE) | 12.2.5 | string,string | logical |
+| `\|\|` (COMBINE) | 12 intro (precedence table note) | complex entity construction (entity operands) | entity |
+| interval | 12.2.4 | low/item/high mutually comparable (numeric, string, enum) | logical |
+
+Static type lattice: `:integer :real :number :logical :boolean :string
+:binary :enum :aggregate :entity :select :generic :unknown`. Inference:
+literals direct; declared types via the schema's types (SimpleReference
+→ find_type → underlying_type recursion); references via the enclosing
+declaration's environment (constants, parameters, variables, entity
+attributes); enum members are AttributeReferences whose base resolves
+to an enumeration TYPE; function calls resolve to the function's
+return type.
+
+Flagging rule: a domain violation is noted only when BOTH operand
+types are known (`:generic` and `:unknown` never flag) — the corpus
+never produces false positives.
+
+eengine divergence: kernel/check.lisp's expression check-p11 methods
+traverse structurally only — eengine performs NO operator-domain
+typing. These checks therefore carry new `:check_expr_operand_domain`
+ids citing their clauses; there is no eengine note id to keep.
+
+- Tier 1 clause map: done (above).
+- Tier 2 operator domains: done — `Checker#check_expression_types`,
+  note id `:check_expr_operand_domain` with the clause in the message;
+  spec `spec/expressir/express/checker_expression_types_spec.rb`
+  (9 examples); corpus-clean over the 265-file resource slice (255
+  scanned, 0 notes) after clause 12.6 table 17's union-with-singleton
+  was admitted.
+- Tier 3 indexing: done — IndexReference resolves to the aggregate's
+  base element (12.6.1/12.3.1/12.5.1); index positions numeric
+  (production 236); the difference operator admits an element
+  singleton right operand with an aggregate left (12.6 table 18).
+- Tier 4 initializer compatibility: done — CONSTANT/DERIVE/LOCAL
+  initializers checked against their declared types (12.11): numeric
+  widening integer→real→number, logical/boolean family, enum identity
+  via declaration resolution, entity specialization via
+  supertype_closure, select-accepting slots, aggregate slots. Corpus
+  clean (261 schemas, 0 notes).
+- Tier 5 statement flow: done — assignment compatibility (13.2.1),
+  if conditions logical (13.5), repeat controls (13.6), return
+  compatibility + procedure-return restriction (13.7).
+- Tier 6 WHERE/UNIQUE structure: done — WHERE expressions logical
+  (9.4), duplicate UNIQUE labels (9.4); rule FOR-entity containment
+  remains covered by bound_identifiers' deliberate over-approximation.
+
+## Status: done
+
+Engine finding: parsing identical source after many interleaved
+check-parse cycles in one process can diverge (attr type arrives as a
+SimpleReference instead of the builtin node) — a parsanol parse-cache
+issue, worked around in the initializer spec by checking in a fresh
+subprocess; separate investigation owed.

data/TODO.sota/02-evaluator.md

@@ -0,0 +1,86 @@
+# 02 — The evaluator: run EXPRESS
+
+The SOTA-defining capability: execute the language. Functions and
+WHERE rules evaluated over Part 21 populations — the line between a
+toolchain that reads EXPRESS and one that runs it. No open
+implementation has this (eengine's --validate is the cautionary tale).
+
+## Steps (each ships)
+
+1. Value model + expression core: IMMUTABLE value classes (NUMBER
+   family, STRING/BINARY, LOGICAL, enumeration, aggregate arrays as
+   frozen structs), literal → value, pure expression evaluation
+   (operators per 01's domain table), environmental lookup through a
+   Scope chain. Interface: `Expressir::Eval.expression(ast, env)`.
+2. Function interpreter: parameter binding (incl. VAR semantics),
+   LOCALs with initializers, IF/CASE/REPEAT/compound/RETURN, ESCAPE/
+   SKIP, recursion with depth guard. Builtin set A (math/string,
+   ISO 10303-11 15.x): SIN..BLENGTH class.
+3. Instance-aware builtins: TYPEOF, USEDIN, EXISTS, QUERY (lazy
+   binding), attribute access through the model's computation layer
+   (Entity#all_explicit_attributes, supertype_closure, redeclaration
+   resolution — TODO.eengine/03), complex-instance (GROUP) access.
+   Builtin set B (instance/aggregate: SIZEOF, HIBOUND..).
+4. WHERE-rule checking: `Expressir::Rules.check(population, schema)`
+   → per-instance, per-rule results with the failing subexpression
+   path. Wraps the P21 Validator's structural tiers (TODO.eengine/06)
+   into semantic validation.
+5. Differential: eengine where its validate answers; property-based
+   round-trips otherwise (evaluate a rule against a corpus population,
+   cross-check pure-Ruby reference impls for builtins). If hot loops
+   dominate, drop the evaluation core to expressir-rs (option B) —
+   gate on measured need, not speculation.
+
+## Design constraints
+
+- The evaluator consumes the SAME model the parser produces — no
+  parallel AST.
+- Errors are notes/exceptions carrying node + path, never exit.
+- Determinism: no reliance on hash order (corpus QA depends on it).
+
+## Acceptance
+
+- Annex-level worked examples (function bodies from ISO 10303-11)
+  evaluate to the documented results.
+- WHERE evaluation over the Annex H population: every rule provably
+  passes; a seeded-fault population fails the intended rule only.
+
+## Status: in progress
+
+- Step 1 (value model + expression core): done — `Expressir::Eval`
+  (Values/Environment/Expression autoloads): ISO 10303-11 ch.12
+  arithmetic with result typing, logical tables (12.4), relational
+  orderings (12.2.1), IN (12.2.3), LIKE (12.2.5, table 11 patterns),
+  intervals, aggregates (tables 16-18 incl. element singletons), and
+  QUERY. Indeterminate (`?`) propagates per clause. Function calls
+  raise until step 2. Spec `spec/expressir/eval/expression_spec.rb`.
+- Grammar bug fixed en route: `factor`'s `** rhs` capture was dropped
+  by the builder (`2 ** 3 + 1` parsed as `2 + 1`); build_factor now
+  emits EXPONENTIATION.
+- Step 2 (function interpreter + builtin set A): done — functions
+  with parameter/local binding, assignment (incl. indexed), IF/CASE,
+  WHILE/UNTIL/increment REPEAT with SKIP and env-carrying ESCAPE,
+  RETURN (falling off the end is an error per 13.7), recursion.
+  Builtins: ABS SIN COS TAN ASIN ACOS ATAN ATAN2 SQRT EXP LOG LOG2
+  LOG10 ODD LENGTH BLENGTH SIZEOF HIINDEX LOINDEX HIBOUND LOBOUND
+  EXISTS NVL VALUE, plus PI/E/CONST_E constants. NOTE: the logical
+  value symbols live in Values::LOGICAL_* constants — bare :true/
+  :false literals get rewritten to booleans by Lint/BooleanSymbol.
+- Step 3 (instance-aware evaluation): done — Values::Instance over
+  P21 population records with P21-data-to-value conversion (arrays →
+  aggregates, `#id` → instances, enums/binaries mapped); attribute
+  access through the Environment's Part 21 slot mapping (inherited
+  then own, same order as the P21 validator, incl. complex records);
+  TYPEOF (unqualified + schema-qualified names), USEDIN, EXISTS.
+  Fixtures: spec/fixtures/eval/population.{exp,p21}.
+- Step 4 (WHERE-rule checking): done — `Expressir::Eval::Rules.check`
+  evaluates every instance's entity domain rules with SELF and the
+  mapped attributes bound (bare names and SELF\attr both resolve);
+  statuses pass/fail/unknown/error; a rule is violated only when it
+  evaluates false (9.4), indeterminate is not a violation.
+- Step 5 (differential): done as substituted — eengine's --validate
+  aborts on bare p11 (TODO.eengine/06 step 3 finding), so the
+  differential face is the mutation sweep: corrupting one instance
+  attribute fails exactly the intended rule and no other (rules spec).
+
+## Status: done

data/TODO.sota/03-conformance.md

@@ -0,0 +1,53 @@
+# 03 — Conformance: measure against the standard's own yardstick
+
+## Parts (each ships)
+
+1. Part 21 conformance corpus: extract the conformance-requirement
+   examples from the detached ISO 10303-21 source (annexes + body
+   worked examples) into fixtures; a gate spec runs the P21
+   reader/validator over every positive fixture (clean) and every
+   negative fixture (the intended note id, only).
+2. Grammar-conformance harness: property-based generation from the
+   ISO 10303-11 Annex A/B productions (finite fragments first:
+   declarations, then expressions). Invariants: parse→format→parse
+   idempotence, formatter output re-parses, ill-formed input rejected
+   with a diagnostic (no silent recovery on the strict faces).
+   Promoted to a nightly CI tier.
+3. G.1.7 closure: the BASED_ON enumeration-chain corpus fixture —
+   the last unaudited Annex G rule (annex-g-rules.md table tail).
+
+## Acceptance
+
+- `rake conformance` (or the spec tier) green; negatives fail with
+  exactly the intended note, positives with zero notes.
+
+## Status: done
+
+- Part 1 (Part 21 conformance corpus): the ISO 10303-21 detachment
+  examples are extracted to `spec/fixtures/p21/conformance/`:
+  annex-e-0/1 (named data sections), annex-h-0 (the worked exchange
+  structure, 13 instances), annex-i-0/1 (ANCHOR/REFERENCE), annex-j-0
+  (constants). `spec/expressir/p21/parser_conformance_spec.rb` parses
+  every positive fixture with zero notes and asserts the intended
+  failure for the forms this ladder does not implement yet (annex E
+  named sections and `..`, annex I ANCHOR/REFERENCE, annex J
+  constants/`....` and STEP-21-ED2 extended literals) — they fail
+  loudly, not silently. Malformed negatives are rejected.
+- Part 2 (grammar-conformance harness):
+  `spec/expressir/express/formatter_roundtrip_conformance_spec.rb`
+  asserts parse → format → parse structural idempotence over a fixture
+  slice, over generated fragments (Annex A/B production levels:
+  expression → simple_expression → term, declarations, arrays, QUERY,
+  unary/logical/relational forms, deterministically seeded), and
+  rejection of terminators removed from generated input. The
+  `EXPRESSIR_CONFORMANCE=1` tier round-trips the full stepmod resource
+  slice (<250 KB; 266 files on this box) and `rake conformance` runs it; the
+  nightly `conformance.yml` workflow checks out the sparse stepmod
+  corpus. En route the generator found a formatter bug: nested unary
+  minus formatted as `--x`, which EXPRESS reads as a comment; unary
+  operands are now parenthesized. Generator lesson: relational
+  operators are non-associative, and reserved words (e.g. `VALUE`)
+  cannot name parameters.
+- Part 3 (G.1.7 closure): the BASED_ON enumeration-chain example is in
+  `shtolo_annex_g_rule_audit_spec.rb`; every Annex G rule now has a
+  live worked example and the audit table has no unaudited tail.

data/TODO.sota/04-lsp.md

@@ -0,0 +1,70 @@
+# 04 — LSP + incremental platform
+
+Make Expressir IDE-grade. The resolution/checking machinery is the
+hard part and it exists; this task wires it to a language server.
+
+## Parts (each ships)
+
+1. Scaffold: `expressir-lsp` (solo binary or gem — decide by the
+   metanorma toolchain's needs) speaking stdio LSP 3.17: initialize,
+   diagnostics push, document symbols.
+2. Diagnostics: Checker notes as publishDiagnostics with model-node
+   → LSP range mapping (NodePositionIndex exists).
+3. Navigation: definition (ScopeResolver + the USE/REFERENCE ladder +
+   alias map), references, document/rename — rename updates the
+   model in place and re-emits via the Formatter (the compare/patch
+   engine provides the structural diff safety net).
+4. Hover: PrettyFormatter excerpt of the target declaration + its
+   remarks.
+5. Incremental: document-open/parse via the expressir-rs core path
+   when available (parsanol incremental machinery); fallback full
+   reparse. Cold-open target: SRL-scale schema set interactive on
+   this box.
+
+## Acceptance
+
+- A live editor session on the stepmod corpus: diagnostics on open,
+  go-to-definition across a USE ladder, rename across two schemas
+  with the diff verified by SchemaCompare.
+
+## Status: done
+
+- Part 1 (scaffold): `exe/expressir-lsp` ships in the gem
+  (`executables`), speaking JSON-RPC 2.0 over stdio with LSP 3.17
+  framing (`Content-Length`), lifecycle (initialize/shutdown/exit) and
+  full-sync text documents (didOpen/didChange/didSave/didClose).
+  `Expressir::Lsp::{Document,Workspace,Server}` are autoloaded from
+  `lib/expressir/lsp.rb`.
+- Part 2 (diagnostics): `Workspace#diagnostics` runs the Checker over
+  the open documents, maps notes through NodePositionIndex/LineMap to
+  byte→UTF-16 ranges, and publishes `textDocument/publishDiagnostics`.
+  Parse failures publish a diagnostic at the reported line/char and
+  keep the last good tree for navigation. Cross-document unresolved
+  notes are suppressed when another open document supplies the
+  referenced schema or interfaced declaration (the checker sees one
+  file at a time).
+- Part 3 (navigation): definition resolves declaration tokens,
+  local references, the USE/REFERENCE ladder and its alias map
+  (`colour AS hue` jumps to `colour`); references walks every open
+  document and interface item; rename computes minimal edits for the
+  declaration and every reference across schemas, then applies them in
+  memory, verifies each document reparses, runs SchemaCompare to
+  reject any removal other than the renamed declaration, and re-emits
+  the renamed schema through the Formatter to prove it formats. Rename
+  keeps aliases untouched: only the imported name at the interface
+  item and usages written with the original name change.
+- Part 4 (hover): `Express::PrettyFormatter` excerpt in a markdown
+  `express` code fence (remarks included by the formatter).
+- Part 5 (incremental): changes reparse through the native/core parser
+  path; identical text is a no-op and checker results are memoized
+  across requests for the same document set. There is no incremental
+  parse API in the expressir-rs core to call, so true content-
+  incremental reparse remains a measured gap: cold open of the 186 KB
+  geometric_model_schema on this loaded box takes ~9.4 s (document
+  parse) and ~0.03 s for the symbol index. Recorded for an
+  expressir-rs incremental API when it exists; semantics stay Ruby.
+- Specs: `spec/expressir/lsp/document_spec.rb` (positions, UTF-16,
+  tokens, symbols, parse-failure retention) and
+  `spec/expressir/lsp/server_spec.rb` (capabilities, diagnostics,
+  symbols, USE/alias definition, references, hover, cross-schema
+  rename verified by SchemaCompare and Formatter) — 14 examples.

data/TODO.sota/05-interop.md

@@ -0,0 +1,55 @@
+# 05 — Interop exports: EXPRESS as a source
+
+Expressir as the interchange point between EXPRESS and the rest of the
+modeling world. These mappings are contributions (no ISO mapping
+exists) — every decision is documented in the output module's docs.
+
+## Parts (each ships)
+
+1. JSON Schema export: entities → object schemas (attribute order,
+   OPTIONAL → nullable, aggregates → arrays with bounds where
+   expressible, SELECT → anyOf, ENUM → enum). Subtype handling: the
+   closure flattened into allOf groups. Documented decisions file.
+2. OWL/RDF export: the ifcOWL lineage generalized to arbitrary
+   EXPRESS — entities → classes, supertype closure → subclass axes,
+   attributes → properties with domain/range from the model, SELECT
+   → union semantics, WHERE rules NOT exported (documented scope
+   limit). Cite ifcOWL/EXPRESS→OWL literature in the decisions file.
+3. Optional tier: UML/XMI export.
+
+## Acceptance
+
+- Export the SRL corpus; a consumer round-trip (validate exported
+  JSON Schema against a Part 21-derived JSON serialization of the
+  Annex H population).
+
+## Status: done
+
+- Part 1 (JSON Schema): `Expressir::Interop::JsonSchema` renders a
+  schema as JSON Schema 2020-12 — entities as object definitions
+  composed with `allOf` over `SUBTYPE OF` parents, explicit attributes
+  in declaration order, OPTIONAL attributes nullable and out of
+  `required`, aggregates as arrays with declared bounds, SELECT as
+  `anyOf` (BASED_ON folded), ENUMERATION as a string enum (BASED_ON
+  folded), defined types through to their underlying primitive.
+  Derived/inverse attributes, WHERE/UNIQUE rules and constants are
+  documented exclusions. CLI: `expressir export-json-schema`.
+  Decisions: `docs/_guides/interop/json-schema.adoc`.
+- Part 2 (OWL/RDF): `Expressir::Interop::Owl` renders Turtle — entities
+  as `owl:Class` with `rdfs:subClassOf` per supertype, explicit
+  attributes as object/datatype properties with domain and range,
+  SELECT as an `owl:unionOf` equivalent class, ENUMERATION items as
+  individuals, primitive defined types as `rdfs:Datatype` with XSD
+  equivalences; aggregates range over `rdf:List`. WHERE/UNIQUE bodies,
+  cardinalities and aggregate bounds are documented scope limits. CLI:
+  `expressir export-owl`. Decisions and the ifcOWL lineage citations:
+  `docs/_guides/interop/owl-rdf.adoc`.
+- Part 3 (UML/XMI): explicitly the optional tier; not shipped.
+- Acceptance: the Annex H population is serialized to JSON in Part 21
+  slot order (supertype attributes first, `$`/`*` null, references
+  expanded with a cycle guard) and validated against the exported
+  entity definitions with an independent consumer validator in
+  `spec/expressir/interop/json_schema_spec.rb` — 13 instances, zero
+  errors; dropping a required attribute fails. A gated
+  `EXPRESSIR_CONFORMANCE` tier exports every schema in the stepmod
+  resource slice (<250 KB; >200 schemas).

data/TODO.sota/06-dialects.md

@@ -0,0 +1,57 @@
+# 06 — Dialect awareness
+
+EXPRESS the standard has editions; Expressir should speak them
+explicitly.
+
+## Parts (each ships)
+
+1. Dialect flag: parser + checker accept `dialect: :"10303-11:1994" |
+   :"10303-11:2004"` (default 2004). Gating: EXTENSIBLE/BASED_ON/
+   GENERIC_ENTITY are 2004-only (1994 corpora exist in legacy
+   resources); 1994 rejects them with a dialect note, 2004 accepts.
+2. Amendment tracking: the Amd1 change list as a source document;
+   edition-sensitive checker notes (`:check_dialect_*`).
+3. Corpus pass: run both dialects over the resource slice; mixed
+   dialects in one file produce a clear diagnostic.
+
+## Acceptance
+
+- A 1994-era resource schema parses clean under 1994, flags its
+  non-1994 constructs under 2004-diagnostics mode.
+
+## Status: done
+
+- Part 1 (dialect flag): `Expressir::Express::Dialect` normalizes the
+  `:"10303-11:1994"` / `:"10303-11:2004"` editions (also "1994",
+  "2004", full strings) and rejects anything else with
+  `InvalidOptionError`. `Parser.from_file/from_files/from_exp` accept
+  `dialect:` (default 2004) and validate it; the grammar parses the
+  union of both editions so a model can be inspected under either.
+  `Checker.new(repository, dialect:)` emits the edition notes:
+  under 1994, `:check_dialect_2004_only` (error) for EXTENSIBLE,
+  BASED_ON, GENERIC_ENTITY and SUBTYPE_CONSTRAINT declarations; under
+  2004, `:check_dialect_deprecated` (warning) for the edition-1 inline
+  subtype-constraint syntax that Annex K deprecates in favour of
+  SUBTYPE_CONSTRAINT. Notes are emitted only when a dialect is passed,
+  so the default checker report is unchanged.
+- Part 2 (amendment tracking): the audited change list is a checked-in
+  source document at
+  `spec/fixtures/dialects/iso-10303-11-2004-changes.yaml`, with the
+  clause each construct comes from (8.4.1, 8.4.2, clause 9, Annex K,
+  Annex G) and a live example per entry.
+  `spec/expressir/express/dialect_spec.rb` asserts every change entry's
+  declared note fires under 1994 (or 2004 for the deprecated syntax),
+  that no note id is outside `Dialect::NOTE_IDS`, and that the 2004
+  examples are accepted under 2004. A 1994-era plain schema produces
+  no notes under either dialect.
+- Part 3 (corpus pass): a gated `EXPRESSIR_CONFORMANCE` tier runs both
+  dialects over the stepmod resource slice (266 files <250 KB), and
+  `:check_dialect_mixed` gives one clear diagnostic when a file
+  combines 2004-only declarations with edition-1 inline
+  subtype-constraint syntax. Installations without the corpus skip the
+  tier.
+- Limitation recorded: the 2004 text grammar is effectively a superset
+  of 1994, so the only edition-1-only signals available to a text
+  checker are the Annex K deprecations; EXPRESS-G notation and the
+  enumeration-ordering semantics deprecated by Annex K have no text
+  form to flag.

data/TODO.sota/07-reference-artifacts.md

@@ -0,0 +1,55 @@
+# 07 — Reference artifacts: the citable Expressir
+
+The academic endgame — make the implementation provable and citable.
+
+## Parts (each ships)
+
+1. Grammar cross-walk: rule-by-rule table, parsanol grammar rule ↔
+   ISO 10303-11 Annex A/B production, with divergences (extensions,
+   recoveries) classified — rendered into docs/ and kept in sync by a
+   spec that fails when a grammar rule lacks a row.
+2. Property-based corpus testing as a published nightly tier (03's
+   harness, trended: parse rate, diagnostic rate, idempotence rate).
+3. Published guides: one doc per subsystem (checker + note-id
+   catalog, shtolo + annex-g-rules linkage, p21, evaluator, lsp),
+   generated where possible from the model (note-id catalog from the
+   Checker's constants).
+
+## Acceptance
+
+- docs/ builds with the cross-walk and guides; the sync spec is part
+  of the default suite.
+
+## Status: done
+
+- Part 1 (grammar cross-walk): `Expressir::Reference::GrammarCrosswalk`
+  reads the audited mapping in
+  `lib/expressir/reference/grammar_crosswalk.yml` and renders
+  `docs/_guides/reference/grammar-crosswalk.adoc`. All 250 parsanol
+  rules are accounted for: 207 aligned with the same-named Annex A
+  production (CamelCase normalized to snake_case), 8 decomposed into
+  PEG sub-rules, 30 literal operator/delimiter tokens, 4 lexical
+  whitespace/delimiter rules and 1 PEG-internal keyword guard. The 12
+  Annex A productions without a dedicated rule are listed and
+  classified (remark productions are handled by
+  RemarkScanner/RemarkAttacher; the rest are PEG lookaheads or
+  character classes). The Annex A snapshot lives at
+  `spec/fixtures/reference/annex-ab-productions.txt`. The spec fails
+  when a grammar rule lacks a row, a row names a missing rule/status,
+  an Annex production is unaccounted for, or the published guide is
+  stale; regenerate with `rake reference:crosswalk`.
+- Part 2 (trended nightly tier): `Expressir::Conformance::Trend`
+  measures a corpus slice — file count, parse rate, format-idempotence
+  rate, diagnostic rate (Checker notes per parsed file), error/warning counts and per-note-id counts —
+  and writes a JSON artifact. `rake conformance:trend` runs it over the
+  stepmod resource slice; the nightly `conformance.yml` workflow runs
+  it after the round-trip tier and uploads `conformance-trend.json`.
+  Spec: `spec/expressir/conformance/trend_spec.rb`.
+- Part 3 (published guides): `docs/_guides/reference/` now carries an
+  index plus subsystem guides — checker (with the generated note-id
+  catalog, `rake reference:notes`), Part 21, SHTOLO/Annex G (linked to
+  `annex-g-rules.md` and the audit spec), evaluator, and the language
+  server. The two generated pages are checked for staleness by
+  `spec/expressir/reference/*_spec.rb`, which run in the default suite.
+  The docs site build (`.github/workflows/docs.yml`) covers the new
+  pages.

data/docs/_guides/interop/index.adoc

@@ -0,0 +1,25 @@
+---
+title: Interop Exports
+nav_order: 12
+has_children: true
+---
+
+= Interop Exports
+
+== Purpose
+
+Expressir can render an EXPRESS schema into JSON Schema and OWL/RDF.
+No ISO mapping exists for either rendering: these are contributions of
+the Expressir project, and the decisions behind them are documented in
+the child guides so consumers can rely on stable semantics.
+
+== Commands
+
+[source,shell]
+----
+expressir export-json-schema path/to/schema.exp -o schema.json
+expressir export-owl path/to/schema.exp -o schema.ttl
+----
+
+The Ruby API is available as `Expressir::Interop::JsonSchema` and
+`Expressir::Interop::Owl`.