sts 0.5.3 → 0.5.4

data/TODO.sts-refactor/07-incomplete-models.md

@@ -0,0 +1,180 @@
+# 07: Incomplete Models Fix
+
+**Priority**: MEDIUM
+**Category**: Feature Gap
+**Estimated Effort**: Medium
+**Files Affected**: Existing model files
+
+## Models Needing Completion
+
+### 1. Permissions (Critical — IEC License Data Lost)
+
+**Current**: `copyright_statement` is `:string`, `copyright_holder` is single string
+**Needed**:
+- `copyright_statement` → mixed content (supports `<inline-graphic>`, `<br/>`)
+- `copyright_holder` → collection (multiple holders)
+- Add `<license>` child element
+- Add `<license-p>` child element (paragraphs within license)
+- Add `<license_ref>` (ALI namespace, `ali:license_ref`)
+- Add `<ext-link>` child
+
+```ruby
+# Current (WRONG)
+attribute :copyright_statement, :string
+attribute :copyright_holder, :string
+
+# Fixed
+attribute :copyright_statement, :string  # keep as mixed_content
+attribute :copyright_holder, :string, collection: true
+attribute :license, License, collection: true
+attribute :copyright_year, :string
+attribute :copyright_month, :string
+attribute :copyright_day, :string
+```
+
+### 2. EditingInstruction (IEC Amendment Data)
+
+**Current**: Missing `@content-type` and `<p>` child
+**Needed**:
+- `@content-type` attribute
+- `<p>` as repeatable child
+- Support for IEC amendment instructions
+
+### 3. TermDisplay (IEC Term Content)
+
+**Current**: Exists but missing critical children
+**Needed**:
+- `<term>` element support
+- `<def>` element support
+- `<list>` element support
+- `<xref>` element support
+- `<label>` element support
+- `<term-display-string>` (NISO STS 1.2)
+
+### 4. DefList
+
+**Current**: Missing `@list-content` attribute
+**Needed**:
+- `@list-content` attribute (formula-key, figure-key, abbreviation-key)
+- Proper relationship to DefItem children
+
+### 5. Graphic
+
+**Current**: Missing `@id`, `<label>`, `<caption>` children
+**Needed**:
+- `@id` attribute
+- `<label>` child (for sub-figure captions)
+- `<caption>` child
+- `<alt-text>` child
+
+### 6. Front (NisoSts)
+
+**Current**: Missing `<std-doc-meta>` wiring
+**Needed**:
+- `<std-doc-meta>` child element
+- Support for multiple `<std-meta>` (adoption documents)
+- Proper metadata hierarchy
+
+### 7. BoxedText
+
+**Current**: Most valid children missing
+**Needed**:
+- `<table-wrap>`, `<fig>`, `<disp-formula>`, `<list>`
+- `<p>`, `<caption>`, `<label>`
+- `<sec>` (nested sections in boxed text)
+- All emphasis and inline elements
+
+### 8. DispFormula
+
+**Current**: Missing `<caption>` (added in NISO STS 1.2)
+**Needed**:
+- `<caption>` child element
+
+### 9. Source (Citation Source)
+
+**Current**: Missing inline elements added in NISO STS 1.2
+**Needed**:
+- `<abbrev>`, `<alternatives>`, `<inline-graphic>`, `<private-char>`
+- `<chem-struct>`, `<inline-formula>`, `<mml:math>`, `<tex-math>`
+- `<named-content>`, `<styled-content>`, `<fn>`, `<target>`, `<xref>`
+
+### 10. InlineFormula
+
+**Current**: Missing accessibility elements (NISO STS 1.2)
+**Needed**:
+- `<alt-text>` child
+- `<long-desc>` child
+
+### 11. AwardGroup
+
+**Current**: Missing new NISO STS 1.2 elements
+**Needed**:
+- `<award-name>` child
+- `<award-desc>` child
+
+### 12. DataTitle
+
+**Current**: Missing `<inline-graphic>`, `<private-char>` (NISO STS 1.2)
+**Needed**:
+- `<inline-graphic>` child
+- `<private-char>` child
+
+### 13. Table Cells (td, th)
+
+**Current**: Expanded to full XSD inside-cell content model
+**Done**: disp-quote, speech, and statement are NOT in XSD inside-cell group.
+Td/Th now include: disp-formula, disp-formula-group, element-citation, mixed-citation,
+all emphasis elements, fig, fig-group, inline-graphic, private-char, normative-note,
+normative-example, non-normative-example, email, ext-link, named-content,
+alternatives, code, media, preformat, target, array
+
+### 14. StringName Usage Expansion (NISO STS 1.2)
+
+`<string-name>` added to:
+- `<contrib>`, `<person-group>`, `<name-alternatives>`
+- `<related-article>`, `<related-object>`
+- `<mixed-citation>`, `<element-citation>`
+
+### 15. SubjGroup and KwdGroup Expansion (NISO STS 1.2)
+
+`<subj-group>` and `<kwd-group>` added to:
+- `<app>`, `<boxed-text>`, `<sec>`, `<app-group>`
+- `<chem-struct-wrap>`, `<disp-formula>`, `<disp-formula-group>`
+- `<fig>`, `<fig-group>`, `<graphic>`, `<media>`, `<statement>`
+- `<supplementary-material>`, `<table-wrap>`, `<table-wrap-group>`
+
+## Implementation Order
+
+1. Permissions — highest impact, IEC documents
+2. Graphic — sub-figure support
+3. TermDisplay — terminology sections
+4. DefList — definition lists
+5. EditingInstruction — amendments
+6. Front/std-doc-meta — document structure
+7. BoxedText — content completeness
+8. NISO STS 1.2 additions (DispFormula, Source, InlineFormula, etc.)
+
+## Dependencies
+
+- `04-register-versioning.md` — version-specific additions
+- `05-missing-elements.md` — some children are new elements
+
+## TODO Checklist
+
+- [x] Fix Permissions model (mixed content, collection, license)
+- [x] Fix Graphic model (add @id, label, caption)
+- [x] Fix TermDisplay model (add term, def, list, xref, label)
+- [x] Fix DefList model (add @list-content, @originator, @continued-from, @prefix-word, @xml:lang, term_head, def_head, editing_instruction)
+- [x] Fix EditingInstruction model (add @content-type, @rid, title, p)
+- [x] Fix Front model (add std-doc-meta, reg-meta, nat-meta, ack, notes, toc)
+- [x] Fix BoxedText model (add valid children)
+- [x] Fix Back model (add index, toc, ack, notes, glossary, bio, label, title)
+- [x] Add DispFormula caption (NISO STS 1.2) — expanded DisplayFormula with caption, originator, mixed content
+- [x] Expand Source inline elements (NISO STS 1.2) — full inline element support with attributes
+- [x] Add InlineFormula accessibility (NISO STS 1.2) — alt-text, long-desc, emphasis elements
+- [x] Add AwardGroup elements (NISO STS 1.2) — rid, specific-use, xlink attrs, collection recipients
+- [x] Add DataTitle elements (NISO STS 1.2) — full inline elements with attributes
+- [x] Expand SubjGroup/KwdGroup locations (NISO STS 1.2) — SubjGroup recursive nesting, compound-subject
+- [x] Expand table cell content (NISO STS 1.2) — full XSD inside-cell content model
+- [x] Expand StringName usage (NISO STS 1.2) — added to name-alternatives
+- [x] Run round-trip tests after each model fix

data/TODO.sts-refactor/08-isoiec-profile.md

@@ -0,0 +1,169 @@
+# 08: ISO/IEC Usage Profile Layer
+
+**Priority**: MEDIUM
+**Category**: Architecture
+**Estimated Effort**: High
+**Files Affected**: New module
+
+## Context
+
+ISO/IEC NISO STS is NOT a separate schema. It is a **usage profile** — a constraint layer on top of NISO STS that specifies:
+- Which elements are required vs. optional
+- Which attribute values are allowed
+- How to use elements in ISO/IEC context
+- Additional conventions (ID schemes, committee references, etc.)
+
+This means we should NOT create separate model classes for ISO/IEC. Instead, we need a **validation/constraint layer**.
+
+## Reference Sources
+
+- `reference-docs/Guidelines for coding ed.2.1.pdf` — IEC/ISO Coding Guidelines ed.2.1
+- ISO/IEC Directives Part 2 — document structure rules
+- NISO STS Tag Library — element definitions
+
+## Architecture: Validation Layer
+
+### Option A: Dry-Validation Schema
+
+```ruby
+# lib/sts/profiles/iso_iec/validator.rb
+module Sts
+  module Profiles
+    module IsoIec
+      class Validator
+        def initialize(register = :niso_sts_v1_0)
+          @register = register
+        end
+
+        def validate(document)
+          errors = []
+          errors.concat(validate_structure(document))
+          errors.concat(validate_ids(document))
+          errors.concat(validate_metadata(document))
+          errors.concat(validate_references(document))
+          errors
+        end
+
+        private
+
+        def validate_structure(doc)
+          errors = []
+          # ISO/IEC requires <front>, <body> in <standard>
+          unless doc.front
+            errors << "ISO/IEC profile requires <front> in <standard>"
+          end
+          unless doc.body
+            errors << "ISO/IEC profile requires <body> in <standard>"
+          end
+          errors
+        end
+
+        def validate_ids(doc)
+          # ISO/IEC: all <sec> must have @id matching pattern sec_N
+          # ISO/IEC: all <fig> must have @id matching pattern fig_N
+          # etc.
+        end
+
+        def validate_metadata(doc)
+          # ISO/IEC: <std-meta> must have <std-ident> with specific children
+          # ISO/IEC: committee reference format
+        end
+
+        def validate_references(doc)
+          # ISO/IEC: <std> references must have dated/undated format
+        end
+      end
+    end
+  end
+end
+```
+
+### Option B: Profile Module with Class Methods
+
+```ruby
+# lib/sts/profiles/iso_iec/constraints.rb
+module Sts
+  module Profiles
+    module IsoIec
+      module Constraints
+        # Elements that MUST have @id
+        REQUIRED_ID_ELEMENTS = %i[
+          sec app fig table-wrap disp-formula
+          ref-list ref p term-sec
+        ].freeze
+
+        # Required children of <std-meta>
+        REQUIRED_STD_META_CHILDREN = %i[
+          std-ident doc-ref release-version
+        ].freeze
+
+        # Allowed values for @std-id-type
+        STD_ID_TYPES = %w[
+          dated undated isbn doi
+        ].freeze
+      end
+    end
+  end
+end
+```
+
+### Option C: Custom lutaml-model Validation
+
+Extend lutaml-model's validation to support profile-specific rules:
+
+```ruby
+# This would require lutaml-model changes — consider for future
+class Sts::NisoSts::Standard
+  validates :front, presence: { message: "required by ISO/IEC profile" },
+    if: -> { iso_iec_profile? }
+end
+```
+
+## Recommended Approach
+
+Start with **Option B** (constraint constants) as a documentation layer, then build **Option A** (validator) on top. This avoids coupling validation to the model classes themselves.
+
+## Profile Differences to Capture
+
+### ISO Documents
+- `<std-ident>` with `<originator>ISO</originator>`
+- Committee reference format: `TC/SC/WG`
+- Stage codes per ISO harmonized system
+
+### IEC Documents
+- `<std-ident>` with `<originator>IEC</originator>`
+- Different committee reference format
+- Copyright statement with inline-graphic
+
+### National Adoptions
+- `<std-doc-meta>` wrapper for national metadata
+- `<adopted-from>` reference to international standard
+
+## Files to Create
+
+```
+lib/sts/profiles/
+├── iso_iec/
+│   ├── constraints.rb       # Constraint constants
+│   ├── validator.rb         # Validation logic
+│   ├── id_scheme.rb         # ID naming conventions
+│   └── metadata_rules.rb    # Metadata requirements
+└── profiles.rb              # Entry point
+```
+
+## Dependencies
+
+- `04-register-versioning.md` — needs to know which schema version to validate against
+
+## TODO Checklist
+
+- [x] Extract ISO/IEC constraints from Guidelines ed.2.1
+- [ ] Extract IEC-specific constraints
+- [x] Design validator architecture
+- [x] Implement constraint constants module
+- [x] Implement basic structural validator
+- [ ] Implement ID scheme validator
+- [x] Implement metadata validator (originator, doc-type validation)
+- [ ] Add profile selection to parsing API
+- [x] Write specs for validation
+- [ ] Document usage

data/TODO.sts-refactor/09-autoload-cleanup.md

@@ -0,0 +1,88 @@
+# 09: Autoload and Require Cleanup
+
+**Priority**: LOW
+**Category**: Cleanup
+**Estimated Effort**: Medium
+**Files Affected**: `lib/sts.rb`, `lib/sts/niso_sts.rb`, `lib/sts/iso_sts.rb`
+
+## Current Issues
+
+### 1. Autoload Alphabetization
+
+The autoload entries in `lib/sts/niso_sts.rb` should be alphabetically ordered. Check for any out-of-order entries when adding new elements.
+
+### 2. Moxml Monkey-Patch
+
+`lib/sts.rb` contains a monkey-patch on Moxml:
+
+```ruby
+Moxml::Adapter::Nokogiri.singleton_class.alias_method(:_orig_children, :children)
+Moxml::Adapter::Nokogiri.define_singleton_method(:children, &:children)
+```
+
+This patches an external library's behavior. If Moxml fixes the underlying issue, this will mask it or cause conflicts.
+
+**Action**: Add a comment explaining WHY this patch exists (the Nokogiri adapter bug it works around) and a link to the upstream issue if tracked.
+
+### 3. Orphaned/Unused Files
+
+After refactoring (TODOs 01-04), check for:
+- Files in `lib/sts/niso_sts/` that are no longer referenced
+- Files in `lib/sts/iso_sts/` that are no longer referenced
+- Unused requires
+
+### 4. Require Order Dependencies
+
+Some files may have implicit load-order dependencies (class A references class B, but B hasn't been loaded yet). The autoload system should handle this, but verify.
+
+### 5. Missing Requires for Cross-Namespace Types
+
+After fixing namespace coupling (TODO 03), ensure all cross-namespace references are replaced with proper requires or base type references.
+
+## File Structure After Cleanup
+
+```
+lib/
+├── sts.rb                           # Top-level: requires, Moxml patch (documented)
+├── sts/
+│   ├── namespaces.rb                # Namespace constants
+│   ├── mathml.rb                    # MathML wrapper (after TODO 01 fix)
+│   ├── mathml/                      # MathML-specific types (if needed)
+│   ├── base/                        # Shared base types (after TODO 03)
+│   │   ├── text/                    # Bold, Italic, Sub, Sup
+│   │   ├── struct/                  # Section, Paragraph, List
+│   │   ├── meta/                    # Title, Label, Caption
+│   │   └── refs/                    # Xref, BiblioRef
+│   ├── iso_sts/                     # ISOSTS namespace
+│   │   ├── iso_sts.rb               # Autoload declarations
+│   │   ├── v1_1/                    # ISOSTS v1.1 classes
+│   │   └── content_groups/          # Content group helpers
+│   ├── niso_sts/                    # NISO STS namespace
+│   │   ├── niso_sts.rb              # Autoload declarations
+│   │   ├── v1_0/                    # NISO STS 1.0 classes
+│   │   ├── v1_2/                    # NISO STS 1.2 classes
+│   │   ├── content_groups/          # Content group helpers
+│   │   ├── mml_content/             # MathML content elements
+│   │   └── xi.rb, ali.rb            # Namespace modules
+│   ├── tbx_iso_tml/                 # TBX namespace
+│   │   ├── tbx_iso_tml.rb           # Autoload declarations
+│   │   └── ...
+│   ├── content_groups/              # Shared content group helpers
+│   │   └── helpers.rb               # After TODO 02 fix
+│   └── profiles/                    # ISO/IEC validation layer (after TODO 08)
+```
+
+## Dependencies
+
+- `01-mathml-delegation.md` — MathML wrapper location
+- `03-namespace-coupling.md` — base types directory structure
+- `04-register-versioning.md` — versioned subdirectories
+
+## TODO Checklist
+
+- [x] Document Moxml monkey-patch with rationale (removed entirely — upstream fix in moxml aa58888)
+- [x] Audit autoload entries for alphabetical order
+- [x] After refactoring: remove orphaned files (deleted content_groups dead code)
+- [ ] After refactoring: verify no implicit load-order deps
+- [x] After refactoring: clean up unused requires
+- [x] Verify `bundle exec rake` passes cleanly

data/TODO.sts-refactor/10-test-coverage.md

@@ -0,0 +1,164 @@
+# 10: Test Coverage Improvements
+
+**Priority**: ONGOING
+**Category**: Quality
+**Estimated Effort**: Ongoing
+**Files Affected**: `spec/`
+
+## Current Test Structure
+
+```
+spec/
+├── spec_helper.rb
+├── sts_spec.rb                   # Core parsing/serialization tests
+├── round_trip/
+│   └── reference_docs_spec.rb    # Round-trip tests with ISO doc fixtures
+└── fixtures/
+    └── *.xml                     # Test XML documents
+```
+
+## Test Categories Needed
+
+### 1. Unit Tests Per Element
+
+Each model class should have unit tests covering:
+
+```ruby
+# spec/sts/niso_sts/paragraph_spec.rb
+RSpec.describe Sts::NisoSts::Paragraph do
+  describe "XML parsing" do
+    it "parses <p> with text content"
+    it "parses <p> with mixed content"
+    it "parses <p> with @id attribute"
+    it "parses <p> with @content-type attribute"
+    it "parses <p> with @xml:lang attribute"
+  end
+
+  describe "XML serialization" do
+    it "round-trips text content"
+    it "round-trips mixed content"
+    it "round-trips all attributes"
+  end
+end
+```
+
+Priority elements for unit tests:
+- `Standard`, `Front`, `Body`, `Back` — structural
+- `Section`, `Paragraph` — high frequency
+- `Fig`, `TableWrap`, `Graphic` — media
+- `Ref`, `RefList`, `Xref` — references
+- `TermSec`, `TermDisplay` — terminology
+- `Permissions` — after model fix
+
+### 2. Schema Validation Tests
+
+```ruby
+# spec/schema_validation_spec.rb
+RSpec.describe "Schema compliance" do
+  context "NISO STS 1.0" do
+    it "validates against NISO STS Extended XSD"
+    it "validates against NISO STS Interchange XSD"
+  end
+
+  context "ISOSTS v1.1" do
+    it "validates against ISOSTS DTD"
+  end
+
+  context "NISO STS 1.2" do
+    it "validates against NISO STS 1.2 XSD"
+  end
+end
+```
+
+### 3. Version-Specific Tests (After TODO 04)
+
+```ruby
+# spec/versioning/register_spec.rb
+RSpec.describe "Schema versioning" do
+  it "loads ISOSTS v1.1 register independently"
+  it "loads NISO STS 1.0 register independently"
+  it "loads NISO STS 1.2 register independently"
+  it "prevents cross-register contamination"
+end
+```
+
+### 4. Profile Validation Tests (After TODO 08)
+
+```ruby
+# spec/profiles/iso_iec_validator_spec.rb
+RSpec.describe Sts::Profiles::IsoIec::Validator do
+  it "rejects document without <front>"
+  it "accepts valid ISO document"
+  it "validates ID scheme conventions"
+  it "validates metadata requirements"
+end
+```
+
+### 5. Anti-Pattern Regression Tests
+
+```ruby
+# spec/anti_patterns_spec.rb
+RSpec.describe "No anti-patterns in codebase" do
+  it "contains no method_missing definitions"
+  it "contains no respond_to_missing definitions"
+  it "contains no send calls"
+  it "contains no Object.const_get calls"
+end
+```
+
+### 6. Performance Tests
+
+```ruby
+# spec/performance_spec.rb
+RSpec.describe "Performance" do
+  it "parses large document within time budget"
+  it "serialization doesn't regress from baseline"
+end
+```
+
+## Test Fixtures Needed
+
+| Fixture | Purpose | Priority |
+|---------|---------|----------|
+| Simple standard (minimal) | Basic round-trip | HIGH |
+| IEC standard with license | Permissions testing | HIGH |
+| Document with index | Index hierarchy | HIGH |
+| Document with TOC | TOC hierarchy | HIGH |
+| Document with code blocks | Code element testing | HIGH |
+| Document with MathML | Formula testing | MEDIUM |
+| Document with TBX | Terminology testing | MEDIUM |
+| Document with OASIS tables | CALS table testing | LOW |
+| Multi-language document | xml:lang testing | MEDIUM |
+| Amendment/corrigendum | Amendment structure | MEDIUM |
+| National adoption | std-doc-meta testing | LOW |
+
+## Coverage Metrics
+
+Track coverage with:
+```bash
+COVERAGE=true bundle exec rspec
+```
+
+Target: >80% line coverage for `lib/sts/`
+
+## Dependencies
+
+All other TODOs feed into this — add tests as features are implemented.
+
+## TODO Checklist
+
+- [x] Add anti-pattern regression test
+- [x] Add unit tests for structural elements (Standard, Front, Body, Back)
+- [x] Add unit tests for Section and Paragraph
+- [x] Add unit tests for Fig, TableWrap, Graphic
+- [x] Add unit tests for Ref, RefList, Xref
+- [x] Add unit tests for TermSec, TermDisplay
+- [x] Add unit tests for Permissions (after model fix)
+- [x] Add unit tests for new elements (ProcessingMeta, Code, Legend, InlineMedia, TableWrapGroup, DefHead)
+- [x] Add unit tests for attribute additions (Title, Caption, Label, ExtLink, AwardId, DefItem)
+- [x] Add unit tests for Index and TOC hierarchy elements
+- [ ] Add schema validation tests
+- [ ] Add versioning tests (after TODO 04)
+- [x] Add profile validation tests (after TODO 08)
+- [x] Set up coverage tracking
+- [x] Achieve >80% line coverage target (achieved 99.8%)