chemicalml 0.1.0 → 0.2.1

data/TODO.align/40-unit-type-dictionary-convention-coverage.md

@@ -0,0 +1,28 @@
+# 40 — UnitType-dictionary convention: full upstream constraint coverage
+
+## Why
+
+`reference-docs/conventions/unitType-dictionary.md` requires
+namespace, definition child, and id pattern. The gem ships 1 rule.
+
+## Current constraints (1)
+
+UnitTypeMustHaveIdAndName.
+
+## Missing rules
+
+- `unitTypeList` MUST have `namespace` attribute.
+- `unitType` MUST contain a single `definition` child.
+- `unitType` `id` MUST match `[A-Za-z][A-Za-z0-9._-]*`.
+- `unitTypeList` MUST contain at least one `unitType` child.
+
+## Implementation
+
+NodeConstraint subclasses. Register in
+`lib/chemicalml/convention/unit_type_dictionary.rb`.
+
+## Acceptance
+
+- 4 new constraint classes registered.
+- Per-class spec under `spec/chemicalml/convention/unit_type_dictionary/`.
+- Full suite still green.

data/TODO.align/41-expand-canonical-model-layer.md

@@ -0,0 +1,43 @@
+# 41 — Expand canonical Model layer for chemistry completeness
+
+## Why
+
+The canonical `Model::*` layer has 28 classes but the CML wire layer
+has 121. Many chemistry concepts (crystallography, spectroscopy,
+z-matrix, isotopes, samples, reaction schemes) have no canonical
+representation, forcing callers to use wire classes directly —
+breaking the "all adapters speak the canonical model" invariant.
+
+## New Model classes
+
+- `Model::Lattice` — `lattice_type`, `a/b/c/alpha/beta/gamma`,
+  children: cell_parameters, lattice_vectors
+- `Model::Crystal` — wraps lattice + scalar/array geometry
+- `Model::ZMatrix` — internal coordinates (atomRefs + lengths/angles)
+- `Model::Isotope` — number, spin, abundance, mass
+- `Model::IsotopeList` — collection
+- `Model::Sample` — sample metadata (state, amount, count)
+- `Model::Spectrum` — `id`, `title`, `dict_ref`, xaxis/yaxis,
+  spectrum_data
+- `Model::Peak` — `atomRefs`, `xValue`, `yValue`, `multiplicity`,
+  `peakShape`
+- `Model::PeakList` — collection
+- `Model::ReactionScheme` — `id`, `name`, reaction_step_list
+- `Model::ReactionStep` — `atoms`, `reactiveCentre`, `type`
+- `Model::Spectator` / `Model::SpectatorList`
+- `Model::MoleculeList` — collection (distinct from `molecule[]`)
+- `Model::Observation` — observation record
+
+## Implementation pattern
+
+Each class subclasses `Model::Node` and provides:
+- `attr_accessor` for every value
+- `initialize` with keyword args + sensible defaults
+- `children` returning child Model nodes
+- `value_attributes` returning the equality/hash snapshot
+
+## Acceptance
+
+- 14 new Model classes.
+- Per-class spec under `spec/chemicalml/model/`.
+- Full suite still green.

data/TODO.align/42-translator-wire-new-models.md

@@ -0,0 +1,31 @@
+# 42 — Wire new Model classes through the translator
+
+## Why
+
+New `Model::*` classes from TODO 41 are inert until the translator
+maps them to/from CML wire format.
+
+## Translator additions
+
+- `molecule_to_canonical`: include `crystal`, `spectra`,
+  `spectra_lists`, `z_matrix`, `isotope_list`, `sample` fields.
+- `molecule_from_canonical`: reverse of above.
+- New helpers: `crystal_to/from_canonical`,
+  `lattice_to/from_canonical`, `spectrum_to/from_canonical`,
+  `peak_to/from_canonical`, `z_matrix_to/from_canonical`,
+  `isotope_to/from_canonical`, `sample_to/from_canonical`,
+  `reaction_scheme_to/from_canonical`, `spectator_to/from_canonical`.
+- `module_to_canonical`: stop hardcoding `lists: []` — collect
+  `<list>` children into Model::List collection.
+
+## Implementation
+
+All schema-aware (take `schema:` keyword). Use `WireClassRegistry`
+for every wire class lookup. Add to existing
+`translator.rb` + `translator/value_translations.rb`.
+
+## Acceptance
+
+- Round-trip spec: parse CML with crystal/spectrum/zMatrix →
+  Model → serialize → same CML (canon-compared).
+- Full suite still green.

data/TODO.align/43-dictionary-model-no-to-h.md

@@ -0,0 +1,39 @@
+# 43 — Refactor Dictionary::Model away from hand-rolled `to_h`
+
+## Why
+
+`lib/chemicalml/dictionary/model.rb`, `entry.rb`, `enum.rb`, `link.rb`
+define `def to_h` — hand-rolled serialization. The project's coding
+standard #2 forbids this. The Loader uses `to_h` to round-trip YAML,
+but the canonical Model layer uses `value_attributes` for the same
+purpose. Align Dictionary with the rest of the codebase.
+
+## Current state
+
+```ruby
+class Model
+  def to_h
+    { namespace: ..., prefix: ..., title: ...,
+      description: ..., entries: entries.map(&:to_h) }
+  end
+end
+```
+
+Same pattern in Entry, Enum, Link.
+
+## Target
+
+- Drop `def to_h`.
+- `value_attributes` returns the same shape (used by Loader to dump
+  YAML, by `==`/`hash` for equality).
+- Loader's `dump_to_hash` calls `model.value_attributes`.
+
+If Loader needs a specific YAML shape (string keys vs symbol keys),
+add a thin `as_yaml` adapter — but the model itself only exposes
+`value_attributes`.
+
+## Acceptance
+
+- No `def to_h` / `to_hash` anywhere in `lib/chemicalml/dictionary/`.
+- Loader round-trip still works (specs pass).
+- Full suite still green.

data/TODO.align/44-schema3-only-and-schema24-module.md

@@ -0,0 +1,44 @@
+# 44 — Fix SCHEMA3_ONLY and add Schema24 Module class
+
+## Why
+
+Actual XSD diff (via `grep '<xsd:element name='`):
+
+- Schema3 has **121** elements. Our `Elements::ALL` matches.
+- Schema24 has **137** elements (a superset of Schema3 for most parts).
+- **`anyCml` is in Schema3 but NOT Schema24** → must be in
+  `SCHEMA3_ONLY`.
+- `module` is in BOTH schemas → currently listed in SCHEMA3_ONLY
+  is **wrong**.
+
+Current `SCHEMA3_ONLY = %i[Module]` is incorrect and Schema24 is
+missing its `cml_module.rb` wire class entirely.
+
+## Fix
+
+1. `lib/chemicalml/cml/elements.rb`:
+   `SCHEMA3_ONLY = %i[AnyCml].freeze`
+2. Create `lib/chemicalml/cml/schema24/cml_module.rb` mirroring
+   `schema3/cml_module.rb`.
+3. Update Schema24 autoload list to include `cml_module`.
+4. Update translator comment ("Schema 3 only — Schema 2.4 lacks
+   `<module>`") — no longer accurate.
+5. Verify Schema24 Configuration registers Module.
+
+## Schema24-only elements (not modeled yet)
+
+Schema24 has 16 elements not in Schema3: alternative, annotation,
+appinfo, arg, complexObject, enumeration, expression, float,
+floatArray, integer, integerArray, operator, relatedEntry, string,
+stringArray, tcell, trow.
+
+These are Schema-2.4-only legacy elements. They're outside the
+gem's current scope (the gem focuses on Schema 3 + intersection).
+Document them in CLAUDE.md so future contributors know.
+
+## Acceptance
+
+- SCHEMA3_ONLY = %i[AnyCml].
+- Schema24::CmlModule wire class exists, parses, serializes.
+- Translator's module_from_canonical works for schema24.
+- Full suite still green.

data/TODO.align/45-documentation-and-final-verification.md

@@ -0,0 +1,23 @@
+# 45 — Documentation cleanup and final spec verification
+
+## Why
+
+Stale comments and docs rot fast. After TODOs 36–44 land, sweep the
+codebase for outdated references and run the full suite end-to-end.
+
+## Updates
+
+- `lib/chemicalml/cml.rb`: comment says "All 36 aliases load from a
+  single file" — actually 121 now.
+- `CLAUDE.md`: refresh any counts (121 elements, ~45 constraints,
+  14 model classes added, etc.).
+- `CLAUDE.md`: note Schema24-only legacy elements not modeled.
+- Verify `bundle exec rspec` is fully green.
+- Verify `bundle exec rubocop` is clean (or autocorrect what's
+  safe).
+
+## Acceptance
+
+- All counts in docs match the actual codebase.
+- Full rspec suite passes.
+- Rubocop reports no new offenses.

data/TODO.align/46-lattice-vector-content.md

@@ -0,0 +1,24 @@
+# 46 — Fix LatticeVector content attribute
+
+## Why
+
+`<latticeVector>1 0 0</latticeVector>` carries vector components as
+element content. `lib/chemicalml/cml/base/lattice_vector.rb` declares
+only id/title/dictRef/convention/units — **no content attribute**.
+The translator's workaround maps `lv.content` ↔ wire `units` —
+semantically wrong (units ≠ vector components).
+
+## Fix
+
+1. Add `attribute :content, :string` + `map_content` to Base::LatticeVector.
+2. Update Model::LatticeVector to carry `units` and `content` as
+   independent fields.
+3. Update translator `lattice_vector_to/from_canonical` to map both.
+4. Spec round-trip.
+
+## Acceptance
+
+- `<latticeVector units="unit:nm">1 0 0</latticeVector>` round-trips
+  with units and content preserved.
+- Translator spec covers the fix.
+- Full suite green.

data/TODO.align/47-molecule-crystal-spectra-property-lists.md

@@ -0,0 +1,23 @@
+# 47 — Wire Molecule#crystal/spectra/property_lists through translator
+
+## Why
+
+`Base::Molecule` declares `crystal`, `spectra`, `property_lists`
+children, but `Model::Molecule` doesn't have those fields and
+`molecule_to_canonical/from_canonical` silently drops them. Silent
+data loss.
+
+## Fix
+
+1. Add `crystal`, `spectra`, `property_lists` accessors to
+   `Model::Molecule`.
+2. Wire through `molecule_to_canonical` (read wire, build Model) and
+   `molecule_from_canonical` (read Model, build wire).
+3. Update `Model::Molecule#children` and `#value_attributes`.
+4. Spec round-trip with a molecule containing crystal + propertyList.
+
+## Acceptance
+
+- A molecule with `<crystal>` and `<propertyList>` children round-trips
+  through canonical without loss.
+- Full suite green.

data/TODO.align/48-model-list-and-module-lists.md

@@ -0,0 +1,21 @@
+# 48 — Add Model::List and wire Module#lists
+
+## Why
+
+`Model::Module` has a `lists` accessor but `module_to_canonical`
+hardcodes `lists: []`. There is no `Model::List` canonical class, so
+wire `<list>` children of `<module>` are silently dropped.
+
+## Fix
+
+1. Add `Model::List` mirroring `Base::List` shape: id/title/dictRef/
+   convention + scalar/array/matrix/list children.
+2. Wire `module_to_canonical` to map `wire_module.lists` (drop the
+   hardcoded `[]`).
+3. Wire `module_from_canonical` reverse.
+4. Spec round-trip.
+
+## Acceptance
+
+- A `<module>` containing a `<list>` child round-trips without loss.
+- Full suite green.

data/TODO.align/49-more-chemistry-models.md

@@ -0,0 +1,34 @@
+# 49 — Add more chemistry Model classes
+
+## Why
+
+Several chemically-meaningful CML elements still lack canonical Model
+classes, forcing callers to use wire classes directly and breaking the
+"all adapters speak canonical" invariant.
+
+## New Model classes
+
+- `Model::Torsion` — `<torsion>` (4-atom dihedral)
+- `Model::Angle` — `<angle>` (3-atom angle)
+- `Model::Length` — `<length>` (2-atom distance)
+- `Model::Action` / `Model::ActionList` — `<action>` / `<actionList>`
+- `Model::Map` — `<map>` (atom mapping across reaction)
+- `Model::Fragment` / `Model::FragmentList` — substructure references
+- `Model::ReactionStepList` — list of `<reactionStep>`
+- `Model::ReactiveCentre` — `<reactiveCentre>`
+- `Model::TransitionState` — `<transitionState>`
+- `Model::PeakGroup` — `<peakGroup>`
+- `Model::SpectrumList` — `<spectrumList>`
+- `Model::SpectrumData` — `<spectrumData>`
+- `Model::Region` — `<region>`
+
+## Wire translator mappings
+
+For each new Model class, add `*_to_canonical` / `*_from_canonical`
+helpers. Most are simple (id/title/dictRef/convention).
+
+## Acceptance
+
+- Each new Model class has instantiation + value_attributes specs.
+- Translator mappings round-trip.
+- Full suite green.

data/TODO.align/50-validation-report.md

@@ -0,0 +1,29 @@
+# 50 — ValidationReport with severity filtering
+
+## Why
+
+`Convention.validate` returns a flat array of `Violation`s. Callers
+that want only errors or only warnings have to filter manually. Add
+a `ValidationReport` value object that wraps the array and exposes
+severity-based views.
+
+## API
+
+```ruby
+report = Chemicalml::Convention::ValidationReport.new(violations)
+report.errors      # [Violation] with severity == :error
+report.warnings    # [Violation] with severity == :warning
+report.ok?         # errors.empty?
+report.has_warnings? # warnings.any?
+report.violations  # raw array (backward compat)
+```
+
+Also add `Convention.validate_report(doc)` returning a
+`ValidationReport` instead of an array. Existing `validate` keeps its
+array return shape (backward compatible).
+
+## Acceptance
+
+- New spec covers all 5 helpers.
+- Existing validate specs unchanged.
+- Full suite green.

data/TODO.align/51-convention-auto-detection.md

@@ -0,0 +1,30 @@
+# 51 — Convention auto-detection
+
+## Why
+
+Currently a caller has to know the convention QName and pass it to
+`Convention::Registry.validate(doc, qname: "convention:molecular")`.
+The document itself declares its convention via the `convention`
+attribute on its root element — the gem should be able to detect and
+dispatch automatically.
+
+## API
+
+```ruby
+Chemicalml::Convention::Registry.detect_and_validate(doc)
+# → [Violation, ...] (or ValidationReport)
+```
+
+Internally:
+1. Find the root element (Document or Molecule or Module).
+2. Read its `convention` attribute.
+3. Look up in `Registry`.
+4. If found → run `convention.validate(doc)`.
+5. If not found → raise ArgumentError naming the convention.
+
+## Acceptance
+
+- Spec: a doc with `convention="convention:molecular"` auto-validates.
+- Spec: unknown convention raises ArgumentError.
+- Spec: doc without convention raises ArgumentError.
+- Full suite green.

data/TODO.align/52-schema24-legacy-elements.md

@@ -0,0 +1,28 @@
+# 52 — Schema24 legacy elements (selective)
+
+## Why
+
+Schema 2.4 XSD declares 17 elements not in Schema 3. Three of them
+are useful enough to model: `annotation` (documentation wrapper),
+`appinfo` (application info), `enumeration` (dictionary enum, used
+by entries that predate the schema3 enum attribute).
+
+The other 14 (`alternative`, `arg`, `complexObject`, `expression`,
+`float`, `floatArray`, `integer`, `integerArray`, `operator`,
+`relatedEntry`, `string`, `stringArray`, `tcell`, `trow`) are either
+redundant with Schema 3's unified `scalar`/`array`/`table` or relate
+to legacy expression machinery. Out of scope for now.
+
+## Implementation
+
+- Add `Base::Annotation`, `Base::Appinfo`, `Base::Enumeration`.
+- Add `Role::Annotation`, `Role::Appinfo`, `Role::Enumeration`.
+- Add `Schema24::Annotation`, `Schema24::Appinfo`, `Schema24::Enumeration`.
+- Add to `Elements::ALL` with new `SCHEMA24_ONLY` list.
+- Schema3 skips these; Schema24 registers them.
+
+## Acceptance
+
+- Three new wire classes registered for Schema24 only.
+- Schema3 lookup raises ArgumentError for these (not in Schema 3).
+- Full suite green.

data/TODO.align/53-round-trip-integration-specs.md

@@ -0,0 +1,30 @@
+# 53 — Round-trip integration specs
+
+## Why
+
+Per-element translator specs cover individual `*_to_canonical` /
+`*_from_canonical` calls. They miss integration issues — wire-format
+mismatches, attribute renames, namespace leaks — that only surface
+when you go XML → canonical → XML → canonical and compare.
+
+## Specs
+
+A new `spec/chemicalml/integration/round_trip_spec.rb` that, for each
+fixture under `spec/fixtures/schema3/`:
+
+1. Parse the CML XML → wire tree.
+2. Translate wire → canonical Model.
+3. Translate canonical → wire.
+4. Serialize wire → XML.
+5. Re-parse → second wire tree.
+6. Compare first and second wire trees via `value_attributes`.
+
+Uses real wire instances throughout (no doubles per project rule).
+Also includes synthetic minimal docs for: crystal-bearing molecule,
+spectrum-bearing molecule, zMatrix molecule, isotopeList molecule.
+
+## Acceptance
+
+- Every fixture round-trips.
+- Synthetic minimal docs round-trip.
+- Full suite green.

data/TODO.align/54-document-versioned-parser-shim.md

@@ -0,0 +1,38 @@
+# 54 — Document VersionedParser namespace shim
+
+## Why
+
+`lib/chemicalml/versioned_parser.rb` uses regex for two pre-processing
+steps before delegating to `lutaml-model`:
+
+1. `root_element_of(xml)` — peeks the XML for the root element local
+   name so the parser can dispatch to the right wire class.
+2. `inject_namespace(xml, uri, root)` — when the caller passes
+   `namespace_exist: false`, injects `xmlns="..."` into the root
+   start tag.
+
+Both are pre-processing shims. The actual parse goes through
+`Lutaml::Model::Serializable.from_xml` (line 45). Neither function
+serializes, deserializes, or constructs XML nodes — they're string
+pre-processing for input normalization.
+
+The user asked to confirm this. The answer:
+
+- All actual XML parsing and serialization goes through lutaml-model.
+- 242 wire classes extend `Lutaml::Model::Serializable`.
+- Zero `def to_xml`/`from_xml`/`to_h`/`from_h` on any model class.
+- Zero Nokogiri/REXML references in `lib/`.
+- The two regex helpers in `versioned_parser.rb` are pre-processing
+  for input normalization, not serialization. They are explicitly
+  documented as such.
+
+## Action
+
+- Add a clear comment block at the top of `versioned_parser.rb`
+  explaining the boundary.
+- No code change.
+
+## Acceptance
+
+- Comment added.
+- Full suite green.

data/TODO.align/55-molecule-missing-attributes.md

@@ -0,0 +1,20 @@
+# 55 — Fix Model::Molecule missing attributes (correctness)
+
+## Why
+
+`Base::Molecule` declares `spin_multiplicity`, `dict_ref`,
+`convention`, `chirality` attributes, but `Model::Molecule` doesn't
+have them and the translator drops them silently. These are
+chemically important (chirality in particular).
+
+## Fix
+
+Add `spin_multiplicity`, `dict_ref`, `convention`, `chirality` to
+`Model::Molecule`. Update translator molecule_to_canonical and
+molecule_from_canonical to map them.
+
+## Acceptance
+
+- All 4 attributes round-trip through translator.
+- Existing specs unchanged.
+- Full suite green.

data/TODO.align/56-geometry-models.md

@@ -0,0 +1,24 @@
+# 56 — Geometry Model classes
+
+## Why
+
+CML declares 6 geometry primitives (`point3`, `vector3`, `line3`,
+`plane3`, `sphere3`, `transform3`) plus `cellParameter` and
+`dimension` — all share the same shape (id/title/dictRef/convention +
+content). They're used as children of `atom`, `molecule`, `crystal`,
+etc. Without canonical Models, callers must use wire classes directly.
+
+## Implementation
+
+One Model class per element. Each carries `id`, `title`, `dict_ref`,
+`convention`, `content` (the geometry string). Identical shape, so
+specs can be parametrized.
+
+For `cellParameter`: also `cellType` attribute (per Schema 3 XSD).
+For `dimension`: also `name` attribute.
+
+## Acceptance
+
+- 8 new Model classes registered.
+- Per-class spec.
+- Full suite green.

data/TODO.align/57-compchem-models.md

@@ -0,0 +1,26 @@
+# 57 — Compchem Model classes
+
+## Why
+
+Schema 3 declares 10 compchem-specific elements not yet modeled:
+`gradient`, `eigen`, `kpoint`, `kpointList`, `band`, `bandList`,
+`basisSet`, `atomicBasisFunction`, `particle`, `electron`. All
+are common in computational chemistry output.
+
+## Implementation
+
+- `Model::Gradient` — id/title/dictRef/convention/units/content
+- `Model::Eigen` — id/title/dictRef/convention/units/type/content
+- `Model::Kpoint` / `Model::KpointList`
+- `Model::Band` / `Model::BandList`
+- `Model::BasisSet` / `Model::AtomicBasisFunction`
+- `Model::Particle` / `Model::Electron`
+
+Each follows the Node pattern: attr_accessor, initialize, children,
+value_attributes.
+
+## Acceptance
+
+- 10 new Model classes.
+- Per-class spec.
+- Full suite green.

data/TODO.align/58-remaining-models.md

@@ -0,0 +1,24 @@
+# 58 — Remaining Model classes for full element coverage
+
+## Why
+
+Several wire elements still lack canonical Models, breaking the "all
+adapters speak canonical" invariant for these element types.
+
+## New Model classes (25)
+
+- **Documentation**: Definition, Description, Documentation, Link
+- **Chemistry**: Symmetry (pointGroup/spaceGroup), System
+  (atomArray/molecules), Stmml
+- **Containers**: ArrayList, ConditionList, SubstanceList
+- **Sets**: AtomSet, BondSet
+- **Typing**: AtomType, AtomTypeList, BondType, BondTypeList
+- **Mechanism**: Mechanism, MechanismComponent, Join
+- **Other**: Object, Observation, PeakStructure, Potential,
+  PotentialForm, PotentialList
+
+## Acceptance
+
+- 25 new Model classes.
+- Per-class spec (lightweight — just instantiation + value_attributes).
+- Full suite green.

data/TODO.align/59-wire-new-models-translator.md

@@ -0,0 +1,18 @@
+# 59 — Wire new Models through translator
+
+## Why
+
+Models added in TODOs 56-58 are inert without translator mappings.
+Add `*_to_canonical` / `*_from_canonical` for each.
+
+## Implementation
+
+Most share the geometry-simple shape (id/title/dictRef/convention +
+content). Factor a shared helper to avoid copy-paste.
+
+## Acceptance
+
+- Translator spec for each new Model class.
+- Round-trip integration test covers a synthetic doc using one new
+  Model class end-to-end.
+- Full suite green.

data/TODO.align/60-docs-and-final-verification.md

@@ -0,0 +1,19 @@
+# 60 — Update CLAUDE.md and final verification
+
+## Why
+
+Counts in CLAUDE.md and project docs are stale after TODOs 36-59.
+
+## Updates
+
+- CLAUDE.md model count (was 43, now ~100).
+- Constraint count (was 17, now 60).
+- Note ValidationReport, detect_and_validate APIs.
+- Note Schema24 legacy elements.
+- Verify `bundle exec rspec` fully green.
+- Verify zero forbidden patterns.
+
+## Acceptance
+
+- Docs reflect current state.
+- Full suite green.

data/TODO.align/README.round10.md

@@ -0,0 +1,30 @@
+# TODO.align (round 10)
+
+Code cleanliness fixes.
+
+| # | Workstream | Status |
+|---|---|---|
+| 40 | Fix Base file indentation + extract Translator helpers | complete |
+
+## Issues fixed
+
+### 1. Base file indentation
+
+All 36 Base mixin files had inconsistent indentation on the
+`namespace` line inside `xml do` blocks — 12 spaces instead of 14.
+Fixed via automated `ruby -i -pe` across `lib/chemicalml/cml/base/*.rb`.
+
+### 2. Translator DRY: extracted name_to_canonical / identifier_to_canonical
+
+`molecule_to_canonical` was constructing `Model::Name` and
+`Model::Identifier` inline (two multi-argument `.new` calls in a
+single hash value) instead of calling dedicated helper methods —
+unlike every other element type which has its own `X_to_canonical`
+helper. Extracted to `name_to_canonical` / `identifier_to_canonical`
+methods for consistency with the rest of the Translator.
+
+## Final metrics
+
+- 179 examples, 0 failures
+- 0 forbidden patterns
+- 227 lib files