sts 0.5.2 → 0.5.4

data/TODO.sts-refactor/06-missing-attributes.md

@@ -0,0 +1,152 @@
+# 06: Missing Attributes Implementation
+
+**Priority**: HIGH
+**Category**: Feature Gap
+**Estimated Effort**: Medium
+**Files Affected**: Existing model files
+
+## `@id` Missing on Elements (From Guidelines Sec 3.4)
+
+The IEC/ISO Coding Guidelines ed.2.1 Section 3.4 lists elements that MUST carry `@id`. These are missing in sts-ruby:
+
+| Element | Current Model | Has `@id` | Fix |
+|---------|--------------|-----------|-----|
+| `<def-item>` | `DefItem` | No | Add `attribute :id, :string` |
+| `<graphic>` | `Graphic` | No | Add `attribute :id, :string` |
+| `<list-item>` | `ListItem` | No | Add `attribute :id, :string` |
+| `<mixed-citation>` | `MixedCitation` | No | Add `attribute :id, :string` |
+| `<std>` | `ReferenceStandard` | No | Add `attribute :id, :string` |
+| `<table-wrap-foot>` | `TableWrapFoot` | No | Add `attribute :id, :string` |
+| `<xref>` | `Xref` | No | Add `attribute :id, :string` |
+
+### Implementation Pattern
+
+```ruby
+# For each affected model, add:
+attribute :id, :string
+
+# In xml block:
+map_attribute "id", to: :id
+```
+
+## New Attributes from NISO STS 1.2
+
+### `@stage-code` on `<release-version>`
+```ruby
+attribute :stage_code, :string
+# map_attribute "stage-code", to: :stage_code
+```
+
+### `@assigning-authority` Added to Multiple Elements
+Per NISO STS 1.2 Section 7:
+- `<article-id>`, `<award-id>`, `<compound-kwd>`, `<compound-subject>`
+- `<contrib-id>`, `<institution-id>`, `<issue-id>`, `<journal-id>`
+- `<kwd>`, `<kwd-group>`, `<nested-kwd>`, `<object-id>`
+- `<resource-id>`, `<subject>`, `<subj-group>`, `<volume-id>`
+
+### `@award-id-type`, `@award-type`, XLink on `<award-id>`
+Per NISO STS 1.2 Section 7:
+```ruby
+attribute :award_id_type, :string
+attribute :award_type, :string
+attribute :xlink_href, :string
+attribute :xlink_type, :string
+attribute :xlink_role, :string
+attribute :xlink_title, :string
+attribute :xlink_show, :string
+attribute :xlink_actuate, :string
+```
+
+### `@custom-type` / `@custom-value` Solution
+Per NISO STS 1.2 Section 7, these attributes get custom-type/custom-value:
+- `@form-type`
+- `@person-group-type`
+- `@pub-id-type`
+- `@related-term-type`
+- `@ref-type`
+- `std/@type` (dated, undated, multipart)
+- `std-ref/@type`
+
+### `@dtd-version` Made Optional with Value List
+```ruby
+attribute :dtd_version, :string
+# Values: "iso-0.9", "iso-1.0", "iso-1.1", "1.0", "1.1d1", "1.2"
+```
+
+### `@degree-contribution` on `<role>`
+```ruby
+attribute :degree_contribution, :string
+```
+
+### `@hreflang` on XLink-bearing Elements
+Added to any element with `@xlink:href`:
+- `<related-article>`, `<related-object>`, and all elements with xlink
+
+### `@originator` on Multiple Elements
+Per NISO STS 1.2 Section 7:
+- `<meta-date>`, `<release-date>`, `<release-version-id>`
+- `<secretariat>`, `<self-uri>`, `<wi-number>`
+
+### `@style-detail` on `<styled-content>` and `<list>`
+```ruby
+attribute :style_detail, :string
+# Usage: <list list-type="bullet" style-detail="dash">
+```
+
+### `@use-type` on `<std>`, `<mixed-citation>`, `<element-citation>`
+```ruby
+attribute :use_type, :string
+```
+
+### `@vocab` and `@vocab-identifier`
+Added to: `<kwd>`, `<compound-kwd>`, `<nested-kwd>`, `<subject>`, `<compound-subject>`, `<inline-media>`, `<institution-id>`
+
+### `@vocab-term` and `@vocab-term-identifier` on `<inline-media>`
+
+### `@specific-use` on `<release-version>`
+
+### New `xref/@ref-type` Value: "abbreviation"
+
+### `@list-type` Added Value: "arabic"
+
+## `@xml:lang` Missing on Multilingual Elements
+
+IEC documents use multiple languages. Check and add `@xml:lang` to:
+- `<p>`, `<title>`, `<term>`, `<def>`, `<label>`, `<caption>`
+- Any element in `MetadataStd` that supports multi-language variants
+
+## `@content-type` Missing
+- `<p>` (NisoSts) — already has, verify
+- `<editing-instruction>` — missing per IEC guidelines
+
+## Implementation Order
+
+1. Fix `@id` on 7 elements (trivial, high impact)
+2. Add `@originator` to elements that need it
+3. Add `@assigning-authority` to listed elements
+4. Add `@xml:lang` to multilingual elements
+5. Add `@style-detail`, `@use-type`, `@custom-type` attributes
+6. Add XLink attributes to `<award-id>`
+7. Add remaining NISO STS 1.2 attributes
+
+## Dependencies
+
+- `04-register-versioning.md` — some attributes are version-specific
+
+## TODO Checklist
+
+- [x] Add `@id` to 7 elements listed above
+- [x] Add `@originator` to elements (Paragraph, Section, Abstract, App, DispQuote, List, NormativeNote, NonNormativeNote, DefList)
+- [x] Add `@assigning-authority` to 16 elements (XSD has it on `ext-link` only — done)
+- [x] Add `@xml:lang` to multilingual elements (Label, Caption, DefItem, DefList)
+- [x] Add NISO STS 1.2 specific attributes (vocab, vocab-identifier on IcsWrap, KwdGroup, NamedContent, Role, SubjGroup, Term)
+- [x] Add missing attributes to `<award-id>` (id, specific-use, xml:lang)
+- [x] Add missing attributes to `<ext-link>` (assigning-authority, id, specific-use, xml:lang, xlink attrs)
+- [x] Add missing attributes to `<title>` (content-type, id, specific-use)
+- [x] Add missing attributes to `<caption>` (id, specific-use, xml:lang)
+- [x] Add `@vocab`/`@vocab-identifier` to elements (IcsWrap, KwdGroup, NamedContent, Role, SubjGroup, Term)
+- [ ] Verify attribute mappings match XSD/DTD (OASIS CALS remaining)
+- [x] Add `@id` to all 161 NisoSts models that have `@id` in XSD
+- [x] Add `@id` to 32 TbxIsoTml models
+- [x] Add `@id` to 29 IsoSts models
+- [x] Run round-trip tests after each batch

data/TODO.sts-refactor/07-incomplete-models.md

@@ -0,0 +1,180 @@
+# 07: Incomplete Models Fix
+
+**Priority**: MEDIUM
+**Category**: Feature Gap
+**Estimated Effort**: Medium
+**Files Affected**: Existing model files
+
+## Models Needing Completion
+
+### 1. Permissions (Critical — IEC License Data Lost)
+
+**Current**: `copyright_statement` is `:string`, `copyright_holder` is single string
+**Needed**:
+- `copyright_statement` → mixed content (supports `<inline-graphic>`, `<br/>`)
+- `copyright_holder` → collection (multiple holders)
+- Add `<license>` child element
+- Add `<license-p>` child element (paragraphs within license)
+- Add `<license_ref>` (ALI namespace, `ali:license_ref`)
+- Add `<ext-link>` child
+
+```ruby
+# Current (WRONG)
+attribute :copyright_statement, :string
+attribute :copyright_holder, :string
+
+# Fixed
+attribute :copyright_statement, :string  # keep as mixed_content
+attribute :copyright_holder, :string, collection: true
+attribute :license, License, collection: true
+attribute :copyright_year, :string
+attribute :copyright_month, :string
+attribute :copyright_day, :string
+```
+
+### 2. EditingInstruction (IEC Amendment Data)
+
+**Current**: Missing `@content-type` and `<p>` child
+**Needed**:
+- `@content-type` attribute
+- `<p>` as repeatable child
+- Support for IEC amendment instructions
+
+### 3. TermDisplay (IEC Term Content)
+
+**Current**: Exists but missing critical children
+**Needed**:
+- `<term>` element support
+- `<def>` element support
+- `<list>` element support
+- `<xref>` element support
+- `<label>` element support
+- `<term-display-string>` (NISO STS 1.2)
+
+### 4. DefList
+
+**Current**: Missing `@list-content` attribute
+**Needed**:
+- `@list-content` attribute (formula-key, figure-key, abbreviation-key)
+- Proper relationship to DefItem children
+
+### 5. Graphic
+
+**Current**: Missing `@id`, `<label>`, `<caption>` children
+**Needed**:
+- `@id` attribute
+- `<label>` child (for sub-figure captions)
+- `<caption>` child
+- `<alt-text>` child
+
+### 6. Front (NisoSts)
+
+**Current**: Missing `<std-doc-meta>` wiring
+**Needed**:
+- `<std-doc-meta>` child element
+- Support for multiple `<std-meta>` (adoption documents)
+- Proper metadata hierarchy
+
+### 7. BoxedText
+
+**Current**: Most valid children missing
+**Needed**:
+- `<table-wrap>`, `<fig>`, `<disp-formula>`, `<list>`
+- `<p>`, `<caption>`, `<label>`
+- `<sec>` (nested sections in boxed text)
+- All emphasis and inline elements
+
+### 8. DispFormula
+
+**Current**: Missing `<caption>` (added in NISO STS 1.2)
+**Needed**:
+- `<caption>` child element
+
+### 9. Source (Citation Source)
+
+**Current**: Missing inline elements added in NISO STS 1.2
+**Needed**:
+- `<abbrev>`, `<alternatives>`, `<inline-graphic>`, `<private-char>`
+- `<chem-struct>`, `<inline-formula>`, `<mml:math>`, `<tex-math>`
+- `<named-content>`, `<styled-content>`, `<fn>`, `<target>`, `<xref>`
+
+### 10. InlineFormula
+
+**Current**: Missing accessibility elements (NISO STS 1.2)
+**Needed**:
+- `<alt-text>` child
+- `<long-desc>` child
+
+### 11. AwardGroup
+
+**Current**: Missing new NISO STS 1.2 elements
+**Needed**:
+- `<award-name>` child
+- `<award-desc>` child
+
+### 12. DataTitle
+
+**Current**: Missing `<inline-graphic>`, `<private-char>` (NISO STS 1.2)
+**Needed**:
+- `<inline-graphic>` child
+- `<private-char>` child
+
+### 13. Table Cells (td, th)
+
+**Current**: Expanded to full XSD inside-cell content model
+**Done**: disp-quote, speech, and statement are NOT in XSD inside-cell group.
+Td/Th now include: disp-formula, disp-formula-group, element-citation, mixed-citation,
+all emphasis elements, fig, fig-group, inline-graphic, private-char, normative-note,
+normative-example, non-normative-example, email, ext-link, named-content,
+alternatives, code, media, preformat, target, array
+
+### 14. StringName Usage Expansion (NISO STS 1.2)
+
+`<string-name>` added to:
+- `<contrib>`, `<person-group>`, `<name-alternatives>`
+- `<related-article>`, `<related-object>`
+- `<mixed-citation>`, `<element-citation>`
+
+### 15. SubjGroup and KwdGroup Expansion (NISO STS 1.2)
+
+`<subj-group>` and `<kwd-group>` added to:
+- `<app>`, `<boxed-text>`, `<sec>`, `<app-group>`
+- `<chem-struct-wrap>`, `<disp-formula>`, `<disp-formula-group>`
+- `<fig>`, `<fig-group>`, `<graphic>`, `<media>`, `<statement>`
+- `<supplementary-material>`, `<table-wrap>`, `<table-wrap-group>`
+
+## Implementation Order
+
+1. Permissions — highest impact, IEC documents
+2. Graphic — sub-figure support
+3. TermDisplay — terminology sections
+4. DefList — definition lists
+5. EditingInstruction — amendments
+6. Front/std-doc-meta — document structure
+7. BoxedText — content completeness
+8. NISO STS 1.2 additions (DispFormula, Source, InlineFormula, etc.)
+
+## Dependencies
+
+- `04-register-versioning.md` — version-specific additions
+- `05-missing-elements.md` — some children are new elements
+
+## TODO Checklist
+
+- [x] Fix Permissions model (mixed content, collection, license)
+- [x] Fix Graphic model (add @id, label, caption)
+- [x] Fix TermDisplay model (add term, def, list, xref, label)
+- [x] Fix DefList model (add @list-content, @originator, @continued-from, @prefix-word, @xml:lang, term_head, def_head, editing_instruction)
+- [x] Fix EditingInstruction model (add @content-type, @rid, title, p)
+- [x] Fix Front model (add std-doc-meta, reg-meta, nat-meta, ack, notes, toc)
+- [x] Fix BoxedText model (add valid children)
+- [x] Fix Back model (add index, toc, ack, notes, glossary, bio, label, title)
+- [x] Add DispFormula caption (NISO STS 1.2) — expanded DisplayFormula with caption, originator, mixed content
+- [x] Expand Source inline elements (NISO STS 1.2) — full inline element support with attributes
+- [x] Add InlineFormula accessibility (NISO STS 1.2) — alt-text, long-desc, emphasis elements
+- [x] Add AwardGroup elements (NISO STS 1.2) — rid, specific-use, xlink attrs, collection recipients
+- [x] Add DataTitle elements (NISO STS 1.2) — full inline elements with attributes
+- [x] Expand SubjGroup/KwdGroup locations (NISO STS 1.2) — SubjGroup recursive nesting, compound-subject
+- [x] Expand table cell content (NISO STS 1.2) — full XSD inside-cell content model
+- [x] Expand StringName usage (NISO STS 1.2) — added to name-alternatives
+- [x] Run round-trip tests after each model fix

data/TODO.sts-refactor/08-isoiec-profile.md

@@ -0,0 +1,169 @@
+# 08: ISO/IEC Usage Profile Layer
+
+**Priority**: MEDIUM
+**Category**: Architecture
+**Estimated Effort**: High
+**Files Affected**: New module
+
+## Context
+
+ISO/IEC NISO STS is NOT a separate schema. It is a **usage profile** — a constraint layer on top of NISO STS that specifies:
+- Which elements are required vs. optional
+- Which attribute values are allowed
+- How to use elements in ISO/IEC context
+- Additional conventions (ID schemes, committee references, etc.)
+
+This means we should NOT create separate model classes for ISO/IEC. Instead, we need a **validation/constraint layer**.
+
+## Reference Sources
+
+- `reference-docs/Guidelines for coding ed.2.1.pdf` — IEC/ISO Coding Guidelines ed.2.1
+- ISO/IEC Directives Part 2 — document structure rules
+- NISO STS Tag Library — element definitions
+
+## Architecture: Validation Layer
+
+### Option A: Dry-Validation Schema
+
+```ruby
+# lib/sts/profiles/iso_iec/validator.rb
+module Sts
+  module Profiles
+    module IsoIec
+      class Validator
+        def initialize(register = :niso_sts_v1_0)
+          @register = register
+        end
+
+        def validate(document)
+          errors = []
+          errors.concat(validate_structure(document))
+          errors.concat(validate_ids(document))
+          errors.concat(validate_metadata(document))
+          errors.concat(validate_references(document))
+          errors
+        end
+
+        private
+
+        def validate_structure(doc)
+          errors = []
+          # ISO/IEC requires <front>, <body> in <standard>
+          unless doc.front
+            errors << "ISO/IEC profile requires <front> in <standard>"
+          end
+          unless doc.body
+            errors << "ISO/IEC profile requires <body> in <standard>"
+          end
+          errors
+        end
+
+        def validate_ids(doc)
+          # ISO/IEC: all <sec> must have @id matching pattern sec_N
+          # ISO/IEC: all <fig> must have @id matching pattern fig_N
+          # etc.
+        end
+
+        def validate_metadata(doc)
+          # ISO/IEC: <std-meta> must have <std-ident> with specific children
+          # ISO/IEC: committee reference format
+        end
+
+        def validate_references(doc)
+          # ISO/IEC: <std> references must have dated/undated format
+        end
+      end
+    end
+  end
+end
+```
+
+### Option B: Profile Module with Class Methods
+
+```ruby
+# lib/sts/profiles/iso_iec/constraints.rb
+module Sts
+  module Profiles
+    module IsoIec
+      module Constraints
+        # Elements that MUST have @id
+        REQUIRED_ID_ELEMENTS = %i[
+          sec app fig table-wrap disp-formula
+          ref-list ref p term-sec
+        ].freeze
+
+        # Required children of <std-meta>
+        REQUIRED_STD_META_CHILDREN = %i[
+          std-ident doc-ref release-version
+        ].freeze
+
+        # Allowed values for @std-id-type
+        STD_ID_TYPES = %w[
+          dated undated isbn doi
+        ].freeze
+      end
+    end
+  end
+end
+```
+
+### Option C: Custom lutaml-model Validation
+
+Extend lutaml-model's validation to support profile-specific rules:
+
+```ruby
+# This would require lutaml-model changes — consider for future
+class Sts::NisoSts::Standard
+  validates :front, presence: { message: "required by ISO/IEC profile" },
+    if: -> { iso_iec_profile? }
+end
+```
+
+## Recommended Approach
+
+Start with **Option B** (constraint constants) as a documentation layer, then build **Option A** (validator) on top. This avoids coupling validation to the model classes themselves.
+
+## Profile Differences to Capture
+
+### ISO Documents
+- `<std-ident>` with `<originator>ISO</originator>`
+- Committee reference format: `TC/SC/WG`
+- Stage codes per ISO harmonized system
+
+### IEC Documents
+- `<std-ident>` with `<originator>IEC</originator>`
+- Different committee reference format
+- Copyright statement with inline-graphic
+
+### National Adoptions
+- `<std-doc-meta>` wrapper for national metadata
+- `<adopted-from>` reference to international standard
+
+## Files to Create
+
+```
+lib/sts/profiles/
+├── iso_iec/
+│   ├── constraints.rb       # Constraint constants
+│   ├── validator.rb         # Validation logic
+│   ├── id_scheme.rb         # ID naming conventions
+│   └── metadata_rules.rb    # Metadata requirements
+└── profiles.rb              # Entry point
+```
+
+## Dependencies
+
+- `04-register-versioning.md` — needs to know which schema version to validate against
+
+## TODO Checklist
+
+- [x] Extract ISO/IEC constraints from Guidelines ed.2.1
+- [ ] Extract IEC-specific constraints
+- [x] Design validator architecture
+- [x] Implement constraint constants module
+- [x] Implement basic structural validator
+- [ ] Implement ID scheme validator
+- [x] Implement metadata validator (originator, doc-type validation)
+- [ ] Add profile selection to parsing API
+- [x] Write specs for validation
+- [ ] Document usage

data/TODO.sts-refactor/09-autoload-cleanup.md

@@ -0,0 +1,88 @@
+# 09: Autoload and Require Cleanup
+
+**Priority**: LOW
+**Category**: Cleanup
+**Estimated Effort**: Medium
+**Files Affected**: `lib/sts.rb`, `lib/sts/niso_sts.rb`, `lib/sts/iso_sts.rb`
+
+## Current Issues
+
+### 1. Autoload Alphabetization
+
+The autoload entries in `lib/sts/niso_sts.rb` should be alphabetically ordered. Check for any out-of-order entries when adding new elements.
+
+### 2. Moxml Monkey-Patch
+
+`lib/sts.rb` contains a monkey-patch on Moxml:
+
+```ruby
+Moxml::Adapter::Nokogiri.singleton_class.alias_method(:_orig_children, :children)
+Moxml::Adapter::Nokogiri.define_singleton_method(:children, &:children)
+```
+
+This patches an external library's behavior. If Moxml fixes the underlying issue, this will mask it or cause conflicts.
+
+**Action**: Add a comment explaining WHY this patch exists (the Nokogiri adapter bug it works around) and a link to the upstream issue if tracked.
+
+### 3. Orphaned/Unused Files
+
+After refactoring (TODOs 01-04), check for:
+- Files in `lib/sts/niso_sts/` that are no longer referenced
+- Files in `lib/sts/iso_sts/` that are no longer referenced
+- Unused requires
+
+### 4. Require Order Dependencies
+
+Some files may have implicit load-order dependencies (class A references class B, but B hasn't been loaded yet). The autoload system should handle this, but verify.
+
+### 5. Missing Requires for Cross-Namespace Types
+
+After fixing namespace coupling (TODO 03), ensure all cross-namespace references are replaced with proper requires or base type references.
+
+## File Structure After Cleanup
+
+```
+lib/
+├── sts.rb                           # Top-level: requires, Moxml patch (documented)
+├── sts/
+│   ├── namespaces.rb                # Namespace constants
+│   ├── mathml.rb                    # MathML wrapper (after TODO 01 fix)
+│   ├── mathml/                      # MathML-specific types (if needed)
+│   ├── base/                        # Shared base types (after TODO 03)
+│   │   ├── text/                    # Bold, Italic, Sub, Sup
+│   │   ├── struct/                  # Section, Paragraph, List
+│   │   ├── meta/                    # Title, Label, Caption
+│   │   └── refs/                    # Xref, BiblioRef
+│   ├── iso_sts/                     # ISOSTS namespace
+│   │   ├── iso_sts.rb               # Autoload declarations
+│   │   ├── v1_1/                    # ISOSTS v1.1 classes
+│   │   └── content_groups/          # Content group helpers
+│   ├── niso_sts/                    # NISO STS namespace
+│   │   ├── niso_sts.rb              # Autoload declarations
+│   │   ├── v1_0/                    # NISO STS 1.0 classes
+│   │   ├── v1_2/                    # NISO STS 1.2 classes
+│   │   ├── content_groups/          # Content group helpers
+│   │   ├── mml_content/             # MathML content elements
+│   │   └── xi.rb, ali.rb            # Namespace modules
+│   ├── tbx_iso_tml/                 # TBX namespace
+│   │   ├── tbx_iso_tml.rb           # Autoload declarations
+│   │   └── ...
+│   ├── content_groups/              # Shared content group helpers
+│   │   └── helpers.rb               # After TODO 02 fix
+│   └── profiles/                    # ISO/IEC validation layer (after TODO 08)
+```
+
+## Dependencies
+
+- `01-mathml-delegation.md` — MathML wrapper location
+- `03-namespace-coupling.md` — base types directory structure
+- `04-register-versioning.md` — versioned subdirectories
+
+## TODO Checklist
+
+- [x] Document Moxml monkey-patch with rationale (removed entirely — upstream fix in moxml aa58888)
+- [x] Audit autoload entries for alphabetical order
+- [x] After refactoring: remove orphaned files (deleted content_groups dead code)
+- [ ] After refactoring: verify no implicit load-order deps
+- [x] After refactoring: clean up unused requires
+- [x] Verify `bundle exec rake` passes cleanly