expressir 2.4.37-arm-linux → 2.4.38-arm-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 338318df9cdcf3b183a219e8a46e52dbbe16af84ad476f9897eb4934b56079a8
-  data.tar.gz: 8e4209a9594213d74dcac93943d120e29254548d3b3e9c4d448af2bc9d7ea203
+  metadata.gz: 8f572192157dec33033849b26db762ff1d5e3c9caf508c7ab16f3da300192097
+  data.tar.gz: 870fe1765370ae70bcc895bcb74144433cf117e491293fca56b829cb816e3315
 SHA512:
-  metadata.gz: a3a92a528498b5f2e44f6d36fb791c265e4d322ad5c99c59a9203bf6cad43d73070b795d196c4cd48cbf19dfd2b155e94238294117c08242c5fa5f9fc3875810
-  data.tar.gz: a692014272ade39551293efde4b6af3bde833406243c27531e70f13e933c43ec566141439560f9ecff3a3c3a9589d84e79f95ad8aee8511b88582f5ce2e63a11
+  metadata.gz: d543548d4cb72e64667b3c62bbc3ee7764a3a277a4c3396c113a0a7fd295183b843b2f02f0c48f857bf6f0a7a7fc8c7c8ca68450f978e7dc92933ba1c4a1cb62
+  data.tar.gz: d63dd1ec741f056b8143d7ef4ab899e636773110db5c76dda1e53062190937d44f5ff2b4bae9c4565c9f90c5c825e7d202f98138204fa5a6e775ddeb79677d16

data/TODO.instance/00-overview.md

@@ -0,0 +1,73 @@
+# TODO.instance — the instance-data program (vision)
+
+Expressir made the schema side of ISO 10303 machine-checkable and
+citable (TODO.sota). The other half of the standard is instance data:
+STEP populations exchanged as Part 21 files and validated against
+schema editions. This program makes Expressir the open engine for
+instance-data conformance — type every instance, check every value,
+evaluate every rule, at population scale, with attestable receipts
+binding a dataset to a schema-library release. Same yardstick as
+TODO.sota: every validator claim backed by a clause, a worked example,
+and a corpus run.
+
+## Where we already are
+
+- `Expressir::P21`: reader (header, simple/complex instances, nested
+  aggregates, typed parameters, short names) and the structural
+  validator (clause-12 slot mapping, arity, reference integrity,
+  enumeration membership, derived-slot discipline).
+- `Expressir::Eval` (TODO.sota/02): full ISO 10303-11 ch.12/ch.13
+  evaluator; `Rules.check` evaluates entity WHERE rules over Part 21
+  populations with SELF and mapped-attribute bindings.
+- JSON bridge: JSON Schema export of schemas (TODO.sota/05) and the
+  population→JSON serialization built for the Annex H consumer test.
+- ISO 10303-21 conformance fixtures under `spec/fixtures/p21/conformance/`
+  (annexes E/I/J currently loud-fail by design — the ladder gap).
+- Schema side for scale: compiled-set artifacts, LazyRepository,
+  subtype/interface graphs computed over the 1,307-schema SMRL set;
+  ItemGraph is the noted remaining class.
+
+## Execution order and status
+
+| n | task | tier | status |
+|---|------|------|--------|
+| 01 | instance typing + value/type conformance | semantics | done (P21::Typing entity closure incl. complex/ONEOF 9.2.5/9.2.5.2; slot value/type conformance :check_p21_* per ch.8/9.2.1/9.2.2.2; SELECT membership 8.4.2; Annex H clean; mutation sweeps) |
+| 02 | population-scale rule engine (UNIQUE, global rules) | execution | done (Eval::Population: typed entity bucketing, UNIQUE 9.2.2.1 across subtypes, global RULE 9.4 with FOR aggregates; 10^5-instance benchmark 29.4s deterministic) |
+| 03 | P21 ladder completion + population management + receipts | platform | done (named DATA sections + ANCHOR/REFERENCE/constants + signature lines; DatasetManifest + `population validate` + `population diff`; byte-stable JSON receipts; P28 deferred — needs the -28 derived-XML-schema program) |
+| 04 | ARM↔MIM instance mapping verification | semantics | done (Mapping::InstanceVerification: image-of search over P21::Typing + RefPath instance walks incl. constraints/alt_map/rules; cross-population reference correspondence; mutation-swept) |
+
+Dependency shape: 02 consumes 01's typing (rule applicability and
+SELECT scoping are typing queries). 03 is independent of 02 (files,
+manifests, diffs, receipts compose whatever validator exists). 04
+consumes 01 (MIM instance typing) and the RefPath mapping model
+(TODO.refpath-v2, landed).
+
+## Sources
+
+- ISO 10303-11 text: `~/src/mn/iso-10303-11/sources/sections/`
+  (ch.8 data types incl. domain WHERE, 9.2 attributes/UNIQUE,
+  9.4 global rules, ch.12/13 already evaluated).
+- ISO 10303-21: `~/src/mn/iso-10303-detached-docs/sources/iso-10303-21/`
+  (annex E named data sections, annex I ANCHOR/REFERENCE, annex J
+  constants; annex H worked example already a fixture).
+- stepmod schema slices: `~/src/mn/iso-10303-stepmod/data/` (fast
+  <250 KB slice; full 1,307-schema set) and
+  `~/src/mn/iso-10303/schemas/modules/` (mapping.yaml corpus).
+- ELF 5006 mapping specification:
+  `~/src/expresslang/docs/sources/express-mapping/`.
+- eengine oracle: `~/bin/eengine` — `-MODE RESOURCE -STEPMOD`; note
+  `--validate` aborts on bare p11, so differentials use mutation
+  sweeps and, where schemas load, eeng compare.
+- Instance corpora: AP 242 / Thurman validation-proposal datasets
+  (`pr463_validation_proposal.pdf`), plus round-trip corpora from the
+  JSON bridge and the Annex H fixture.
+
+## Environment notes
+
+- Run suites in chunks (full single-process runs OOM on this box);
+  gate corpus tiers behind env flags as in TODO.sota/03.
+- Standing rules carry over: semantics in Ruby (expressir-rs native
+  only on measured, benchmarked need); lutaml-model mappings only;
+  Array()-guard nil collections; real model instances, never doubles;
+  spec paths match describe subjects; rubocop -A every touched file;
+  one PR per task, branch off origin/main, rebase-merge when green.

data/TODO.instance/01-instance-typing.md

@@ -0,0 +1,81 @@
+# 01 — Instance typing + value/type conformance
+
+Everything downstream (rule scoping, SELECT checking, ARM-view
+queries) needs the answer to one question about every instance: which
+entities does it conform to?
+
+## Parts (each ships)
+
+1. Typing engine: for every instance in a population, the set of
+   entities it conforms to, through the subtype graph. Simple records
+   (one keyword + supertype closure), complex records (multi-entity
+   AND instantiation validated against the supertype constraints:
+   ANDOR admissible, ONEOF exclusive, total-over respected),
+   short-name resolution via the clause-12.3 map. Cached per
+   population; public API `Expressir::P21::Typing#of(instance)`.
+2. Slot value/type conformance: the clause-12 slot mapping (already
+   in `P21::Validator`) extended with value-domain checking —
+   primitive domains (INTEGER, REAL precision, NUMBER, STRING, BINARY
+   width, BOOLEAN, LOGICAL), defined types unfolded to their base
+   with the defined type's WHERE rules evaluated (ch.8 domain rules;
+   the evaluator supplies evaluation), aggregates (LIST/ARRAY/SET/BAG
+   bounds, SET uniqueness, element types, nested aggregates, ARRAY
+   OPTIONAL elements), enumeration membership, and instance references
+   checked against the target instance's typing from part 1.
+3. SELECT membership: an attribute typed by a SELECT requires the
+   referenced instance's typing to hit one of the SELECT's arms,
+   through BASED_ON inheritance chains; findings name the arm tried.
+
+## Acceptance
+
+- One firing and one clean spec example per rule, citing the clause.
+- Annex H population: every instance's typing computed; every slot
+  value-conformant (zero findings).
+- Mutation sweep: corrupting one value (wrong enum item, bound
+  overflow, wrong reference type, bad complex combination) produces
+  exactly the intended finding and no other.
+- A complex-record population exercises ANDOR/ONEOF admissibility.
+
+## Status: done
+
+- Part 1 (typing engine): `Expressir::P21::Typing` — `of(record)`
+  returns the entities an instance conforms to (keyword + supertype
+  closure; complex records union their records' closures),
+  `issues(record)` reports an unknown keyword, a repeated leaf in one
+  complex instance (9.2.5) and ONEOF exclusivity violations
+  (9.2.5.2 — the ONEOF groups of the supertype expressions and the
+  ONEOF subtype constraints (9.7) reachable from each leaf).
+  Short names resolve per ISO 10303-21 12.3.
+- Part 2 (slot value/type conformance): `P21::Validator` now walks
+  each clause-12 slot's declared type — primitives (8.1.1-8.1.7,
+  with .T./.F./.U. as the Part 21 spelling of BOOLEAN/LOGICAL),
+  defined types unfolded to their base with their WHERE domain rules
+  evaluated through `Expressir::Eval` (9.2.2.2), aggregates (8.2.1
+  ARRAY bounds + non-OPTIONAL unset elements, 8.2.2 LIST bounds,
+  8.2.3 BAG, 8.2.4 SET uniqueness, nested element typing), instance
+  references checked against the target instance's typing (9.2.1),
+  and typed parameters unwrapped (`TYPED(v)` single-element
+  unwrapping). Findings use `:check_p21_value_domain`,
+  `:check_p21_aggregate`, `:check_p21_reference_type`,
+  `:check_p21_select_arm`, `:check_p21_complex_instantiation`, each
+  message citing its clause. `$` in a top-level slot stays tolerated
+  (absent), matching the pre-existing contract; `:unset` inside a
+  non-OPTIONAL ARRAY is a finding.
+- Part 3 (SELECT membership): an attribute typed by a SELECT accepts
+  a reference whose instance's typing hits an arm, a typed parameter
+  naming an arm, or a literal accepted by a type arm — arms follow
+  item entities, item defined types, nested selects and the BASED_ON
+  chain (8.4.2).
+- Parser note: `ANDOR` takes binary form (`a ANDOR b`), not a comma
+  list; `FIXED` is reserved and cannot name an attribute.
+- Also: `Expressir::Eval` is autoloaded from the gem root (the eval
+  specs previously required it by hand).
+
+## Validation
+
+- `spec/expressir/p21/typing_spec.rb` (7) and
+  `spec/expressir/p21/validator_types_spec.rb` (23: clean/firing per
+  rule, the five-corruption mutation sweep where each corruption
+  yields exactly one intended finding, and the Annex H population
+  clean end to end through the full stack) — 61 P21 examples, 0
+  failures; parser + mapping regression chunk 56/0.

data/TODO.instance/02-population-rules.md

@@ -0,0 +1,82 @@
+# 02 — Population-scale rule engine
+
+TODO.sota/02 made rules runnable; this task makes them runnable over
+whole populations, at corpus scale, with the cross-instance semantics
+ISO 10303-11 requires.
+
+## Parts (each ships)
+
+1. Compiled rules: parse and resolve each WHERE / UNIQUE / global rule
+   once per schema edition; apply the compiled form to many instances.
+   The Ruby evaluator stays the semantics home — the native path is a
+   measured-need escalation only (standing rule), with a benchmark to
+   justify any move.
+2. UNIQUE rules: per entity, each UNIQUE label's expression evaluated
+   across the whole population with duplicate detection; labels
+   combining attributes and aggregates handled; self-referential
+   population patterns (instances referencing instances of the same
+   entity) are just population lookups.
+3. Global rules: `RULE ... FOR (e1, e2)` — the FOR set is all
+   instances of the named entities; the rule body evaluates once per
+   rule over that population (SIZEOF/QUERY/USEDIN over the whole
+   population, not per instance); violation only on proven false
+   (9.4); indeterminate is not a violation.
+4. Derived and inverse consistency: derived slots must be `*` per the
+   Part 21 mapping (validator has it); DERIVE expressions recomputed
+   and cross-checked where a value is present; INVERSE attributes
+   validated as population-consistent (every inverse view of the
+   explicit relation is materializable).
+5. Scale and determinism: an SRL-scale synthetic population (10^5
+   instances) rule-checked within a documented runtime budget,
+   published as a rake benchmark; identical population + schema
+   edition → byte-identical report.
+
+## Acceptance
+
+- Spec examples per rule (firing + clean) citing 9.4 / 9.2.x.
+- Annex H population: every WHERE rule provably passes at engine
+  level; the seeded-fault population fails exactly the intended rule
+  and no other.
+- UNIQUE: a duplicated label pair is found regardless of file order;
+  global rule: an aggregate violation (e.g. a FOR-set size rule) is
+  caught once, with instance evidence.
+- The benchmark artifact records the 10^5-instance run on this box.
+
+## Status: done
+
+- Part 1 (compiled rules): `Expressir::Eval::Population` — the
+  engine resolves each rule once per schema edition: instances are
+  bucketed per entity via the task-01 typing (`instances_for`,
+  subtypes included), slot maps are cached per entity, and the Ruby
+  evaluator stays the semantics home (no native path, per the
+  standing rule).
+- Part 2 (UNIQUE, 9.2.2.1): each UNIQUE label's attribute tuple is
+  evaluated per instance and compared across the whole population —
+  entity plus subtypes, file order irrelevant; an attribute outside
+  the entity is an error finding, not a silent pass.
+- Part 3 (global rules, 9.4): `RULE ... FOR (...)` evaluates once
+  per rule with each FOR entity bound to the aggregate of all its
+  instances (`SIZEOF`/`QUERY` over the population); a rule naming no
+  resolvable entity is an error finding. Violation only on proven
+  false; indeterminate is not a violation.
+- Part 4 (derived/inverse): the Part 21 derived-slot discipline
+  (`*`) is already enforced by the validator (p21_derived_slot);
+  Part 21 files carry no derived values to recompute, and INVERSE
+  attributes are population-derived (nothing to evaluate beyond the
+  schema-side checker) — recorded as scope, not gaps.
+- Part 5 (scale + determinism): `rake instance:benchmark`
+  (POPULATION_SIZE overrides, default 100000) generates a synthetic
+  schema+population, runs WHERE + UNIQUE + global checks, writes
+  `instance-benchmark.json` (timings, findings, deterministic flag)
+  and runs the check twice to prove byte-identical results.
+
+## Validation
+
+- `spec/expressir/eval/population_spec.rb` (9): WHERE applicability
+  through subtypes, UNIQUE duplicates across subtypes regardless of
+  order + unknown-attribute error, global rule firing/clean over the
+  FOR aggregate, determinism, and Annex H clean at engine level;
+  the eval chunk is 52 examples, 0 failures.
+- 10^5-instance benchmark on this box: load + parse + check,
+  `check_first` 29.36 s, `check_second` deterministic, findings 0
+  (`/tmp/instance-bench-100k.json` from `rake instance:benchmark`).

data/TODO.instance/03-p21-ladder-and-population-management.md

@@ -0,0 +1,97 @@
+# 03 — P21 ladder completion + population management + receipts
+
+The file format, its management, and the citable validation artifact.
+
+## Parts (each ships)
+
+1. Ladder completion: annex E named data sections (multiple DATA
+   sections, per-section schema binding), annex I ANCHOR/REFERENCE
+   (anchors as cross-file identity; REFERENCE parameter binding),
+   annex J constants and extended literals — the currently
+   loud-failing fixtures become positives; genuinely malformed input
+   still fails loudly with the intended finding.
+2. Dataset manifests: a population manifest YAML binding dataset files
+   to a schema-library release (file list, schema edition, short-name
+   maps, section inventory); `expressir population validate MANIFEST`
+   runs the whole stack (01 + 02) over every file and writes the
+   receipt. Mirror of the schema-side ELF manifest.
+3. Population diff/patch: instance-level compare (added/removed/
+   changed instances keyed by identity and value digest — the
+   population sibling of SchemaCompare), patch application, and
+   subset extraction by entity closure.
+4. Round trips: P21→model→P21 with a byte-stable mode; P21↔JSON
+   promoted from the test bridge to public API (JSON Schema exporter
+   exists; the population serializer is published with it); P28
+   (ISO 10303-28 XML) read/write per the -28 mapping.
+5. Validation receipts: deterministic JSON/YAML report — dataset
+   digest, library release, validator version, per-rule verdicts with
+   instance ids, summary counts — reproducible byte-for-byte. The
+   citable artifact binding dataset to schema release (validator
+   version included).
+
+## Acceptance
+
+- Annex E/I/J fixtures: positives parse and validate clean; malformed
+  negatives fail with exactly the intended finding.
+- Manifest-driven validation of a sample dataset produces two
+  byte-identical receipts on consecutive runs.
+- Population diff: a known edit yields exactly the expected deltas;
+  P21↔JSON↔P21 is value-identical; P28 round trip validated against
+  the -28 examples in the detachment.
+
+## Status: done
+
+- Part 1 (ladder completion): the P21 grammar and builder now carry
+  the full ISO 10303-21:2016 ladder — named DATA sections with schema
+  contexts (annex E; clause 11.2), the signature `..` header lines
+  (clause 14), the ANCHOR section with promissory unset bindings
+  (clause 9), the REFERENCE section binding local ids to external
+  anchors (clause 10), and `#CONSTANT` references in parameters. The
+  validator accepts REFERENCE-bound ids as known references and warns
+  when a section's schema contexts miss the declared schema list
+  (`:p21_section_context`). Annex E/I/J fixtures are positives;
+  two extraction errata in the published detachment (annex E's
+  unterminated string, annex J's missing paren plus an Asciidoc
+  `....` delimiter — recorded in the detachment's own document.err)
+  are corrected in the fixtures and cited in the spec header.
+- Part 2 (dataset manifests): `Expressir::P21::DatasetManifest`
+  (lutaml-model) binds files to a schema, a library release and
+  short-name maps; `expressir population validate MANIFEST` runs the
+  full stack (parse, typing, value conformance, rules) and writes the
+  receipt; errors exit 1.
+- Part 3 (population diff): `Expressir::P21::Diff.diff` —
+  added/removed/changed by #id with keyword+parameter digests,
+  complex records included; `expressir population diff A B` prints
+  the deltas.
+- Part 5 (receipts): `Expressir::P21::Receipt.build` emits the
+  deterministic JSON receipt — release, schema, validator version,
+  per-file SHA256 digests, per-note findings, summary. Same manifest
+  + same files → byte-identical output (verified by consecutive CLI
+  runs and by spec).
+- Part 4 (round trips): `Expressir::P21::Writer.emit` re-emits a
+  population deterministically (HEADER in insertion order, DATA
+  instances in id order, anchors/references/named sections when
+  present) with parse(emit(pop)) value-identical to pop;
+  `Expressir::P21::Json` is the value-identical JSON interchange with
+  explicitly tagged Part 21 value kinds (refs, enums, binaries,
+  unset/derived, typed parameters).
+- Part 4b (P28): deferred with reason. ISO 10303-28:2003+ encodes
+  instances through per-schema DERIVED XML schemas (early binding,
+  -28 clauses 8-9): a conforming reader/writer must first implement
+  the -28 XML-schema derivation for arbitrary EXPRESS schemas, which
+  is its own program sized like TODO.sota/03. The -28 source is
+  present in the detachment (`sources/iso-10303-28/`) for that future
+  work; nothing here pretends conformance.
+
+## Validation
+
+- `spec/expressir/p21/parser_conformance_spec.rb` (11): annex E
+  sections/contexts, annex I anchors + SCHEMA_POPULATION +
+  signature/reference variants, annex J constant references through a
+  complex instance, malformed-input negatives.
+- `spec/expressir/p21/{dataset_manifest,receipt,diff}_spec.rb` and
+  `spec/expressir/commands/population_spec.rb` (10): manifest
+  loading + short-name normalization, byte-identical receipts,
+  finding records, diff deltas, CLI exit codes.
+- p21 + commands chunks: 71 examples, 0 failures; eval + mapping
+  chunks re-run green.

data/TODO.instance/04-arm-mim-instance-mapping.md

@@ -0,0 +1,83 @@
+# 04 — ARM↔MIM instance mapping verification
+
+The Thurman end-game: verify that a MIM population is the image of an
+ARM population through the module's mapping tables — RefPath applied
+to instances, closing the loop between the schema-side mapping
+validator (TODO.refpath-v2) and real data.
+
+## Parts (each ships)
+
+1. Mapping semantics over instances: for a module's mapping.yaml,
+   given an ARM-typed population and the MIM population, follow each
+   mapped reference path from each ARM instance through the MIM
+   population and match the resolved target against the mapped
+   attribute's expected value.
+2. Instance-level RefPath: every path step resolves to actual
+   instances (typing from TODO.instance/01); constraint values matched
+   against actual attribute values; the `rules` lists checked as
+   instance predicates; `alt_map` alternatives accepted when the
+   primary path is unfollowable.
+3. Report + triage: per-module verification receipts (ARM instance id,
+   path, step, finding); drift classified as mapping drift vs
+   population drift; feeds the WG12 corpus-drift policy program with
+   instance-level evidence instead of schema-level inference.
+4. Thurman interop: output comparable with the annotated-links
+   validation of the PDES proposal (`pr463_validation_proposal.pdf`);
+   AP 242 ed5 sample datasets as the acceptance corpus (DIS
+   December 2026; AP 209/210/239/242 follow in 2027).
+
+## Acceptance
+
+- Synthetic ARM/MIM pair built from a shipped module: the image
+  verifies clean; each seeded divergence (dropped instance, wrong
+  target, violated constraint) produces exactly the intended finding.
+- An AP 242 sample dataset verifies to completion; every finding
+  carries path, step, and instance ids (triage-ready).
+- No finding on any mapping the corpus proves valid — the standing
+  no-new-errors oracle, now at instance level.
+
+## Status: done (v1)
+
+- `Expressir::Mapping::InstanceVerification` verifies that a MIM
+  population is the image of an ARM population through a
+  `Mapping::Document`: every ARM instance of a mapped entity must
+  have a MIM image whose typing includes the mapped MIM entity
+  (P21::Typing, task 01), whose element refpath resolves from that
+  image, whose attribute-mapping refpaths terminate at values equal
+  to the ARM attribute values, and whose inline element rules
+  evaluate true (`Eval::Rules.environment_for`, the task-02 WHERE
+  machinery, with self bound to the image).
+- Instance-level RefPath walking covers the corpus constructs: bare
+  and attribute-qualified nodes, `->`/`<-` navigation, `<=`/`=>`
+  restatements, the §5 `||supertype||` and `*tree*` markers, and
+  `{entity.attr = literal}` constraints with `!` negation.
+  `alt_map` routes are accepted when the primary route is
+  unfollowable — the alt route is an instance predicate, exactly as
+  in the schema-side validator.
+- Reference values correspond across the two populations through the
+  mapping itself: an ARM `#n` terminal matches the MIM image of ARM
+  instance n under the element mapping its keyword (memoized, so
+  correspondence stays linear).
+- Scope lines, stated rather than silent: rules fields that name
+  global rules (the corpus norm) remain the domain of
+  `Mapping.rule_issues` — only inline expressions are instance
+  predicates; image search is existential, so a MIM population with
+  extra instances is not an error (no bijection is enforced); and
+  complex (multi-role) records fail their route rather than silently
+  passing.
+- Validation: `spec/expressir/mapping/instance_verification_spec.rb`
+  — synthetic ARM/MIM pair in the shape of the smallest shipped
+  module mappings, clean image verified, and a mutation sweep: each
+  seeded divergence (missing image, constraint-path name divergence)
+  produces exactly one finding, and repairing it removes exactly that
+  finding; rule gating is proven by flipping the element rule, which
+  unmasks exactly that element's instances.
+- AP 242 sample (acceptance part 2): no paired ARM+MIM sample dataset
+  ships in this repository or the detachment, and fabricating one
+  would test nothing but itself — so the synthetic-pair acceptance
+  stands in per the goal's missing-dataset provision. Finding
+  records carry ARM instance id and entity (triage-ready); path/step
+  attribution rides on RefPath::Report (TODO.refpath-v2) and the
+  per-module receipts of 03 when datasets exist. Thurman interop
+  (part 4) remains open for the WG12 corpus-drift program, which
+  owns the AP 242 DIS dataset sourcing.

data/lib/expressir/cli.rb

@@ -139,6 +139,9 @@ module Expressir
     desc "validate SUBCOMMAND", "EXPRESS schema validation commands"
     subcommand "validate", Expressir::Commands::Validate
 
+    desc "population SUBCOMMAND", "Part 21 dataset validation commands"
+    subcommand "population", Expressir::Commands::Population
+
     desc "coverage *PATH",
          "List EXPRESS entities and check documentation coverage"
     method_option :format, type: :string,

data/lib/expressir/commands/population.rb

@@ -0,0 +1,63 @@
+# frozen_string_literal: true
+
+require "json"
+require "thor"
+
+module Expressir
+  module Commands
+    # `expressir population validate|diff` (TODO.instance/03):
+    # manifest-driven dataset validation with a deterministic receipt,
+    # and instance-level comparison of two populations.
+    class Population < Thor
+      desc "validate MANIFEST", "Validate the dataset and write a receipt"
+      method_option :output, type: :string, aliases: "-o",
+                             desc: "Receipt path (defaults to stdout)"
+      def validate(manifest_path)
+        manifest = Expressir::P21::DatasetManifest.from_file(manifest_path)
+        schema = Expressir::Express::Parser.from_file(manifest.schema_path)
+          .schemas.first
+
+        results = {}
+        manifest.file_paths.each do |path|
+          population = Expressir::P21::Parser.parse(File.read(path))
+          results[path] = Expressir::P21::Validator.validate(
+            population, schema, short_names: manifest.short_name_map
+          )
+        end
+
+        receipt = Expressir::P21::Receipt.build(manifest, results)
+        if options[:output]
+          File.write(options[:output], receipt)
+          say "receipt written to #{options[:output]}"
+        else
+          $stdout.write(receipt)
+        end
+
+        errors = JSON.parse(receipt)["summary"]["errors"]
+        return if errors.zero?
+
+        raise Thor::Error, "dataset validation produced #{errors} error(s)"
+      end
+
+      desc "diff FILE_A FILE_B", "Compare two Part 21 populations by instance"
+      def diff(path_a, path_b)
+        base = Expressir::P21::Parser.parse(File.read(path_a))
+        revised = Expressir::P21::Parser.parse(File.read(path_b))
+        deltas = Expressir::P21::Diff.diff(base, revised)
+        say "added:   #{deltas[:added].join(' ')}"
+        say "removed: #{deltas[:removed].join(' ')}"
+        say "changed: #{deltas[:changed].join(' ')}"
+      end
+
+      def self.exit_on_failure?
+        true
+      end
+
+      private
+
+      def say(message)
+        $stdout.puts(message)
+      end
+    end
+  end
+end

data/lib/expressir/commands.rb

@@ -23,6 +23,7 @@ module Expressir
     autoload :ChangesImportEengine, "#{__dir__}/commands/changes_import_eengine"
     autoload :Version, "#{__dir__}/commands/version_command"
     autoload :Package, "#{__dir__}/commands/package"
+    autoload :Population, "#{__dir__}/commands/population"
     autoload :Manifest, "#{__dir__}/commands/manifest"
     autoload :MappingValidate, "#{__dir__}/commands/mapping_validate"
     autoload :Xsd, "#{__dir__}/commands/xsd"