relaton 3.0.4 → 3.0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f6cef1665e29f9d4a6e8ab48528253675e5b901399666138306c2be69c0d1b6e
-  data.tar.gz: e4eef10cab8373db1d911a4b68342b35c1bb335c15f986d24697f286e83d9a96
+  metadata.gz: 6e8ce7b77ba7b71f2bc7232319b926e70116ca3cdadf53395ef39e1dc4437a64
+  data.tar.gz: d0f29e54fa2129e444819703dfbd9b20ced132b34613053b6944b560370f9447
 SHA512:
-  metadata.gz: 41185be7ff82e3b45667eb33380fb3eee8e548e3ea24039d8743c5ed50b2fcc1b1c252536f87a1d6b0c4821e41e17b08a45e861af1d6886a8abe9df287a800a6
-  data.tar.gz: 3235473497b828e0f5d8890a59b6548994b2ab113d665f010d24e1970c6b24eaa398b298d7f83ba3b770153c558c712059ff5147b5f39d86b61ad5e35312d930
+  metadata.gz: 158c8d47b5a66ddd647f568aa35327ec4f1ee51e5d4ece559c77a610d3ac9474bcd90aa9bcedd544dd0cdaacf7dcac74f72560f810e014c2caa86477d7e33954
+  data.tar.gz: bf49242892015cae4f95d8b63120aa494bb936db2dd98eac34ac8f90ca1f94b889f7d99fd79cc933777f5a04796cb48dc3b933a110f79e29ce6d1f283bb5b744

data/lib/relaton/bib/CLAUDE.md

@@ -61,7 +61,15 @@ Each bibliographic attribute has its own class in `lib/relaton/bib/model/`:
 **`Relaton::Bib::Sanitizer`** ([lib/relaton/bib/sanitizer.rb](lib/relaton/bib/sanitizer.rb))
 strips markup outside the basicdoc set. `LocalizedMarkedUpString#content=` calls it on
 assignment, so every `Title`, `Abstract`, `Note`, `Formattedref` and `Docidentifier` in
-every flavor passes through it. Two properties are load-bearing:
+every flavor passes through it. The set it keeps depends on the field, through
+`#sanitizer_allowed`: every field keeps `ALLOWED` (basicdoc inline markup), and `Abstract`
+keeps `ALLOWED_WITH_BLOCKS`, which adds `BLOCK` — the basicdoc lists, notes, tables, figures
+and admonitions, with the children their grammar requires (`name`, `thead`/`tbody`/`tfoot`,
+`image`). That split follows `biblio.rng`: `<abstract>` takes `BasicBlockNoId`, and a title,
+note, docidentifier or formattedref takes only `PureTextElement`, so do not widen the
+default. Before relaton#421 an abstract's list or note unwrapped to bare `<p>`s with no
+error. A consumer must therefore not assume that an abstract holds only `<p>` and inline
+markup. Two properties are load-bearing:
 
 - **It does not give up on an undeclared namespace prefix.** Third-party markup prefixes
   its elements — Crossref returns JATS as `<jats:p><jats:italic>x</jats:italic></jats:p>`,
@@ -99,7 +107,17 @@ every flavor passes through it. Two properties are load-bearing:
 ### Rendering
 
 - **`Renderer::BibtexBuilder`** - Converts `ItemData` to BibTeX format
-- **`Converter::BibXml`** - Converts `ItemData` to RFC XML format via `Relaton::Bib::Converter::BibXml.to_xml`
+- **`Converter::BibXml`** - Converts `ItemData` to an `Rfcxml::V3` model object via
+  `Relaton::Bib::Converter::BibXml.from_item`; call `to_xml` on the result (this is what
+  `ItemData#to_rfcxml` does). Two output shapes share one class hierarchy:
+  - `ToRfcxml` / `ToRfcxmlReferencegroup` (default) emit **BibXML**, the shape the Relaton
+    data repositories publish. It keeps the RFC 7991-deprecated `<format>` element, so it
+    must stay byte-compatible — every relaton data fetcher's `"bibxml"` format is
+    `entry.to_rfcxml`. Do not change its output.
+  - `ToRfcxmlV3` / `ToRfcxmlReferencegroupV3` (`v3: true`) subclass those and emit strict
+    **RFC 7991**: no `<format>`, plus `<stream>`, `ascii*` attributes and `<refcontent>`.
+    New v3-only behaviour belongs in the subclass, so the default path stays untouched by
+    construction and `spec/bib/relaton/bib/converter/bibxml_spec.rb` keeps passing unedited.
 - **lutaml-model native** - `to_xml`, `to_yaml`, `to_json` via the serialization classes
 
 ### Usage Pattern
@@ -120,11 +138,13 @@ item.to_xml                    # as <bibitem>
 item.to_xml(bibdata: true)     # as <bibdata>
 item.to_yaml
 item.to_bibtex
-item.to_rfcxml
+item.to_rfcxml                 # BibXML (what the data repos publish)
+item.to_rfcxml(v3: true)       # strict RFC 7991 / xml2rfc v3
+item.to_rfcxml(anchor: "ISO712", include_keywords: false)
 ```
 
 ## Code Style
 
 - Follows Ribose OSS Ruby style guide (inherited via `.rubocop.yml`)
-- Target Ruby version: 3.1+
+- Target Ruby version: 3.3+ (`required_ruby_version` in the gemspec, `TargetRubyVersion` in `.rubocop.yml`)
 - Uses YARD documentation comments

data/lib/relaton/bib/README.adoc

@@ -311,6 +311,70 @@ item.to_rfcxml
 </reference>
 ----
 
+=== Export bibliographic item to strict RFC XML v3
+
+`to_rfcxml` emits the BibXML shape published by the Relaton data repositories,
+which still carries the `<format>` element that RFC 7991 deprecates. Pass
+`v3: true` for output that follows the RFC 7991 (xml2rfc v3) vocabulary
+instead: no `<format>`, a `<stream>` element, `ascii*` attributes, and
+`<refcontent>` for identifiers that cannot be expressed as `<seriesInfo>`.
+
+[source,ruby]
+----
+item.to_rfcxml v3: true
+<reference anchor="ISO20483">
+  <front>
+    <title>Cereals and pulses</title>
+    <author initials="Ö." asciiInitials="O." surname="Nürk" asciiSurname="Nurk"/>
+    <author>
+      <organization abbrev="ISO" ascii="International Organization for Standardization">International Organization for Standardization</organization>
+    </author>
+    <date year="2013"/>
+  </front>
+  <refcontent>ISO 20483:2013-2014</refcontent>
+</reference>
+----
+
+Documents published by the IETF (identified by a contributing organization
+named `IETF`, `Internet Engineering Task Force` or `RFC Publisher`) express
+their identifiers as `<seriesInfo>` rather than `<refcontent>`:
+
+[source,ruby]
+----
+<reference anchor="RFC2119" target="https://www.rfc-editor.org/info/rfc2119">
+  <stream>IETF</stream>
+  <front>
+    <title>Key words for use in RFCs to Indicate Requirement Levels</title>
+    <seriesInfo name="BCP" value="14"/>
+    <seriesInfo name="RFC" value="2119"/>
+    ...
+  </front>
+</reference>
+----
+
+The v3 mode also applies the citation rules the IETF toolchain expects, which
+the BibXML mode leaves alone:
+
+* Contributors with role `author` or `editor` become `<author>`; the wider set
+  (`publisher`, `authorizer`, `translator`, …) is used only when the item names
+  no author or editor, so an RFC's publisher is not listed as its author.
+* `<title>` takes the composite `title[@type="main"]`, else joins
+  title-intro / title-main / title-part, so multipart standards keep their full
+  title. Inline markup is flattened, since v3 `<title>` is text-only.
+* Identifiers that restate the reference label, `URN` identifiers and the
+  `I-D.` anchor form are left out of `<refcontent>`; Internet-Draft
+  `<seriesInfo>` carries the full `draft-…` name, which is what xml2rfc
+  resolves against.
+* Where folding a name leaves no letters (a non-Latin script), the `ascii*`
+  attribute is omitted rather than emitted blank.
+
+Two further options are accepted by both modes:
+
+`anchor:`:: overrides the anchor otherwise derived from `docnumber`, for
+callers that carry their own identifier (for example a Metanorma Semantic XML
+`bibitem/@id`).
+`include_keywords:`:: set to `false` to suppress `<keyword>` elements.
+
 === Logging
 
 RelatonBib uses the relaton-logger gem for logging. By default, it logs to STDOUT. To change the log levels and add other loggers, read the https://github.com/relaton/relaton-logger#usage[relaton-logger] documentation.

data/lib/relaton/bib/converter/bibxml/from_rfcxml.rb

@@ -326,7 +326,18 @@ module Relaton
 
               t = Title.new(content: si.name, language: "en", script: "Latn")
               Series.new(title: [t], number: si.value, type: "main")
-            end.compact
+            end.compact + stream_series
+          end
+
+          # RFC XML v3 carries the publication stream in a <stream> element.
+          # ItemData has no dedicated slot for it, so it is kept the way
+          # relaton-ietf does: as a series marked `type="stream"`.
+          def stream_series
+            return [] unless @reference.stream
+
+            t = Title.new(content: @reference.stream, language: "en",
+                          script: "Latn")
+            [Series.new(title: [t], type: "stream")]
           end
 
           # --- Keyword ---

data/lib/relaton/bib/converter/bibxml/to_rfcxml.rb

@@ -5,14 +5,15 @@ module Relaton
     module Converter
       module BibXml
         class ToRfcxml
-          def initialize(item, include_keywords: true)
+          def initialize(item, include_keywords: true, anchor: nil)
             @item = item
             @include_keywords = include_keywords
+            @anchor = anchor
           end
 
           def transform
             model = ::Rfcxml::V3::Reference.new
-            model.anchor = @item.docnumber || derive_anchor
+            model.anchor = @anchor || @item.docnumber || derive_anchor
             model.target = create_target
             model.front = create_front
             model.format = create_format
@@ -26,14 +27,22 @@ module Relaton
             di&.content&.to_s&.gsub(" ", ".")
           end
 
+          # A <uri> may legitimately carry no type -- biblio.rng makes it
+          # optional, and hand-authored bibitems often omit it -- so never call
+          # String methods on it unguarded.
+          def target_types = %w[src doi]
+
           def create_target
-            target = @item.source.detect { |l| l.type.casecmp("src").zero? } ||
-              @item.source.detect { |l| l.type.casecmp("doi").zero? }
+            target = target_types.filter_map { |t| source_of_type(t) }.first
             return unless target
 
             target.content.to_s
           end
 
+          def source_of_type(type)
+            @item.source.detect { |l| l.type&.casecmp(type)&.zero? }
+          end
+
           def create_front # rubocop:disable Metrics/AbcSize,Metrics/MethodLength
             front = ::Rfcxml::V3::Front.new
             front.title = Rfcxml::V3::Title.new(content: @item.title[0].content) if @item.title.any?
@@ -59,9 +68,16 @@ module Relaton
             end
           end
 
+          # A `type="stream"` series is the publication stream, not a series:
+          # it has no number, so it would yield a valueless <seriesInfo>.
+          def seriesinfo_series?(ser)
+            ser.type != "stream" &&
+              ser.title.reject { |t| t.content == "DOI" }.any?
+          end
+
           def series_to_seriesinfo # rubocop:disable Metrics/AbcSize,Metrics/CyclomaticComplexity
             @item.series.select do |s|
-              s.title.reject { |t| t.content == "DOI" }.any?
+              seriesinfo_series?(s)
             end.uniq do |s|
               s.title.find { |t| t.content != "DOI" }.content
             end.each_with_object([]) do |s, si|
@@ -71,8 +87,13 @@ module Relaton
             end
           end
 
+          # Which contributors become <author>; overridden by the v3 emitter.
+          def contributors_for_authors
+            @item.contributor.reject { |c| committee_contributor?(c) }
+          end
+
           def create_authors # rubocop:disable Metrics/AbcSize,Metrics/MethodLength
-            @item.contributor.reject { |c| committee_contributor?(c) }.map do |contrib|
+            contributors_for_authors.map do |contrib|
               role = "editor" if contrib.role.detect { |r| r.type == "editor" }
               Rfcxml::V3::Author.new(
                 role: role,
@@ -284,7 +305,10 @@ module Relaton
             return unless @item.abstract.any?
 
             content = @item.abstract[0].content
-            paragraphs = content.scan(%r{<p>(.*?)</p>}m).flatten
+            # Semantic XML paragraphs carry attributes (`<p id="_349ea…">`);
+            # match those too, or the whole abstract falls through to the
+            # fallback below and the markup is escaped into the <t>.
+            paragraphs = content.scan(%r{<p\b[^>]*>(.*?)</p>}m).flatten
             paragraphs = [content] if paragraphs.empty?
             ts = paragraphs.map { |p| Rfcxml::V3::Text.new(content: CGI.unescapeHTML(p)) }
             Rfcxml::V3::Abstract.new(t: ts)
@@ -294,7 +318,7 @@ module Relaton
 
           def create_format # rubocop:disable Metrics/AbcSize
             @item.source.each_with_object([]) do |l, a|
-              next unless FORMAT_TYPES.any? { |ft| l.type.casecmp(ft).zero? }
+              next unless FORMAT_TYPES.any? { |ft| l.type&.casecmp(ft)&.zero? }
 
               a << Rfcxml::V3::Format.new(
                 type: l.type, target: l.content,

data/lib/relaton/bib/converter/bibxml/to_rfcxml_referencegroup.rb

@@ -3,9 +3,10 @@ module Relaton
     module Converter
       module BibXml
         class ToRfcxmlReferencegroup
-          def initialize(item, include_keywords: true)
+          def initialize(item, include_keywords: true, anchor: nil)
             @item = item
             @include_keywords = include_keywords
+            @anchor = anchor
           end
 
           def transform
@@ -30,6 +31,8 @@ module Relaton
           end
 
           def create_anchor
+            return @anchor if @anchor
+
             docid = @item.docidentifier.detect(&:primary) ||
               @item.docidentifier[0]
             return unless docid
@@ -38,9 +41,13 @@ module Relaton
             id.sub(/^(RFC|BCP|FYI|STD) /, '\1').sub(/^\w+\./, "")
           end
 
+          # An untyped <uri> is legal, so guard before comparing.
+          def target_types = %w[src doi]
+
           def create_target
-            target = @item.source.detect { |l| l.type.casecmp("src").zero? } ||
-              @item.source.detect { |l| l.type.casecmp("doi").zero? }
+            target = target_types.filter_map do |type|
+              @item.source.detect { |l| l.type&.casecmp(type)&.zero? }
+            end.first
             return unless target
 
             target.content.to_s

data/lib/relaton/bib/converter/bibxml/to_rfcxml_referencegroup_v3.rb

@@ -0,0 +1,43 @@
+module Relaton
+  module Bib
+    module Converter
+      module BibXml
+        #
+        # Strict RFC 7991 flavour of {ToRfcxmlReferencegroup}: members are
+        # rendered by {ToRfcxmlV3}, and every one of them gets an anchor,
+        # which the v3 grammar makes mandatory on `<reference>`.
+        #
+        class ToRfcxmlReferencegroupV3 < ToRfcxmlReferencegroup
+          private
+
+          def target_types = %w[src HTML doi]
+
+          # <referencegroup> needs an anchor just as much as its members do,
+          # so fall back past the docidentifier the parent relies on.
+          def create_anchor
+            anchor = super || @item.docnumber || @item.id
+            anchor && anchor.to_s.strip.gsub(/\s+/, ".")
+          end
+
+          def build_references
+            included_items.each_with_index.map do |item, idx|
+              ref = ToRfcxmlV3.new(item, include_keywords: @include_keywords)
+                .transform
+              ref.anchor = member_anchor(idx) if ref.anchor.to_s.empty?
+              ref
+            end
+          end
+
+          def included_items
+            @item.relation.select { |rel| rel.type == "includes" }
+              .map(&:bibitem)
+          end
+
+          def member_anchor(idx)
+            "#{create_anchor || 'reference'}_#{idx + 1}"
+          end
+        end
+      end
+    end
+  end
+end

data/lib/relaton/bib/converter/bibxml/to_rfcxml_v3.rb

@@ -0,0 +1,416 @@
+module Relaton
+  module Bib
+    module Converter
+      module BibXml
+        #
+        # Strict RFC 7991 (xml2rfc v3) flavour of {ToRfcxml}.
+        #
+        # It differs from the BibXML output its parent produces in four ways:
+        #
+        # * the deprecated `<format>` element is never emitted;
+        # * identifiers of documents published outside the IETF go into
+        #   `<refcontent>` rather than being forced into `<seriesInfo>`
+        #   (RFC 7991 2.39), as do series that carry no number;
+        # * `<stream>` and the `ascii*` attributes are populated;
+        # * `<front>` is filled out enough to satisfy the v3 grammar, which
+        #   requires a title and at least one author.
+        #
+        class ToRfcxmlV3 < ToRfcxml
+          # Organizations whose documents are "home" standards: their
+          # identifiers are expressible as <seriesInfo>.
+          HOME_ORGS = [
+            "IETF", "Internet Engineering Task Force", "RFC Publisher"
+          ].freeze
+
+          # Identifier types that are already emitted as <seriesInfo> by the
+          # parent, or that never belong in one.
+          NON_AUTHORITATIVE_TYPES = %w[DOI URI].freeze
+
+          # Identifiers metanorma-ietf drops as internal bookkeeping.
+          IGNORED_ID_RE = /\A(rfc-anchor|Internet-Draft)/
+
+          # Series that name the document itself; handled as identifiers.
+          SUBSERIES = %w[BCP STD].freeze
+
+          HOME_ID_RE = /\A(?<name>RFC|BCP|STD|FYI)\s*(?<value>\d+)\z/i
+          # xml2rfc only resolves a draft reference when seriesInfo carries the
+          # full draft name, so keep `draft-…` intact. The `I-D.` anchor form
+          # is a citation label, not the identifier (metanorma-ietf#283).
+          DRAFT_ID_RE = /\A(?<value>draft-\S+)\z/i
+          DRAFT_ANCHOR_RE = /\AI-D[.\s]\s*\S+\z/i
+
+          # Identifier types that duplicate a human-readable identifier or are
+          # bookkeeping rather than citation content (metanorma-ietf#301).
+          EXCLUDED_ID_TYPES = %w[URN].freeze
+
+          # Title parts joined when relaton supplies no composite title.
+          TITLE_PARTS = %w[title-intro title-main title-part].freeze
+
+          # Roles that may stand in for an author when the item names none,
+          # mirroring the released renderer's `creatornames_roles_allowed`.
+          FALLBACK_ROLES = %w[performer adapter translator publisher
+                              distributor authorizer].freeze
+
+          # The values RFC XML v3 allows on <stream>; "editorial" was added by
+          # RFC 9280. Keyed by the spellings the RFC Editor and relaton-ietf
+          # use, so "Independent Submission" is not silently thrown away.
+          STREAMS = {
+            "ietf" => "IETF", "iab" => "IAB", "irtf" => "IRTF",
+            "independent" => "independent",
+            "independent submission" => "independent",
+            "independent submission stream" => "independent",
+            "editorial" => "editorial", "rfc editor" => "editorial",
+            "rfc-editor" => "editorial"
+          }.freeze
+
+          # Latin letters with no NFKD decomposition, which would otherwise be
+          # dropped outright ("Sørensen" -> "Srensen").
+          TRANSLITERATIONS = {
+            "æ" => "ae", "Æ" => "AE", "œ" => "oe", "Œ" => "OE",
+            "ø" => "o", "Ø" => "O", "ß" => "ss", "ẞ" => "SS",
+            "đ" => "d", "Đ" => "D", "ð" => "d", "Ð" => "D",
+            "þ" => "th", "Þ" => "Th", "ł" => "l", "Ł" => "L",
+            "ħ" => "h", "Ħ" => "H", "ŋ" => "ng", "Ŋ" => "NG",
+            "ŧ" => "t", "Ŧ" => "T", "ı" => "i", "ĸ" => "k"
+          }.freeze
+
+          def transform
+            model = ::Rfcxml::V3::Reference.new
+            model.anchor = create_anchor
+            model.target = create_target
+            model.stream = create_stream
+            model.front = create_front
+            model.refcontent = create_refcontent
+            model
+          end
+
+          private
+
+          # The released IETF renderer takes an HTML-typed uri as the reference
+          # target when there is no src (isodoc/ietf/references.rb), and the
+          # transformer does the same; match it here.
+          def target_types = %w[src HTML doi]
+
+          # anchor is mandatory in v3 and is an XML identifier, so fall back
+          # through every source the item has and never leave whitespace in it.
+          def create_anchor
+            sanitize_anchor(@anchor || @item.docnumber || derive_anchor ||
+                            @item.id)
+          end
+
+          def sanitize_anchor(anchor)
+            return nil if anchor.nil?
+
+            anchor.to_s.strip.gsub(/\s+/, ".")
+          end
+
+          # <front> needs a title and at least one author to be valid v3.
+          def create_front
+            front = super
+            front.title = compound_title || formattedref_title
+            front.author = [unknown_author] if Array(front.author).empty?
+            front
+          end
+
+          # The base emitter takes title[0], which on a multipart standard is
+          # the intro alone ("IT Security techniques"). A citation needs the
+          # whole compound, so prefer the composite title relaton builds, then
+          # the intro-main-part join, then whatever is first.
+          def compound_title
+            content = composite_title || joined_title ||
+              @item.title.first&.content
+            return nil if content.nil?
+
+            ::Rfcxml::V3::Title.new(content: plain_text(content))
+          end
+
+          def composite_title
+            @item.title.find { |t| t.type == "main" }&.content
+          end
+
+          def joined_title
+            parts = TITLE_PARTS.filter_map do |type|
+              @item.title.find { |t| t.type == type }&.content
+            end
+            parts.empty? ? nil : parts.join(" - ")
+          end
+
+          def formattedref_title
+            content = @item.formattedref&.content or return nil
+
+            ::Rfcxml::V3::Title.new(content: plain_text(content))
+          end
+
+          # v3 <title> is text-only, so inline markup a formattedref or title
+          # carries has to be flattened rather than escaped into the output.
+          def plain_text(content)
+            text = content.to_s
+            return text unless text.include?("<")
+
+            Nokogiri::XML.fragment(text).text.squeeze(" ").strip
+          end
+
+          def unknown_author
+            ::Rfcxml::V3::Author.new(surname: "Unknown")
+          end
+
+          # One <seriesInfo> per (name, value): the same identifier can arrive
+          # from a docidentifier and a series, or from two docidentifiers that
+          # spell it differently ("RFC 2119" and "RFC2119").
+          def create_seriesinfo
+            (docidentifier_to_seriesinfo + series_to_seriesinfo +
+              identifier_to_seriesinfo).uniq { |si| [si.name, si.value] }
+          end
+
+          # Only series that carry a number can become <seriesInfo>; the rest
+          # are rendered as <refcontent>.
+          def series_to_seriesinfo
+            numbered_series.map do |ser|
+              ::Rfcxml::V3::SeriesInfo.new(name: series_title(ser).content,
+                                           value: ser.number)
+            end
+          end
+
+          def numbered_series
+            renderable_series.select(&:number)
+          end
+
+          # Series that are neither the stream marker nor a restatement of
+          # something already rendered from the document's identifiers. Only
+          # names actually emitted elsewhere are dropped: a series such as
+          # Internet-Draft still carries its number when no docidentifier
+          # supplies one.
+          def renderable_series
+            @item.series.reject { |ser| skip_series?(ser) }
+              .uniq { |ser| series_title(ser).content }
+          end
+
+          def skip_series?(ser)
+            return true if ser.type == "stream"
+
+            title = series_title(ser)
+            title.nil? || title.content.to_s == "DOI" ||
+              identifier_series_names.include?(title.content.to_s)
+          end
+
+          # Series names already covered by <seriesInfo> or <refcontent> built
+          # from the item's identifiers.
+          def identifier_series_names
+            @identifier_series_names ||=
+              identifier_to_seriesinfo.map(&:name) +
+              subseries_identifiers.map { |id| id.split.first }
+          end
+
+          def series_title(ser)
+            ser.title.find { |title| title.content.to_s != "DOI" }
+          end
+
+          # --- Identifiers ---
+
+          def identifier_to_seriesinfo
+            return [] unless home_standard?
+
+            authoritative_identifiers.filter_map { |id| id_to_seriesinfo(id) }
+          end
+
+          def id_to_seriesinfo(id)
+            name, value = split_identifier(id)
+            return nil unless name
+
+            # No `stream` attribute here: in v3 the stream is carried by the
+            # <stream> element, and duplicating it only adds noise.
+            ::Rfcxml::V3::SeriesInfo.new(name: name, value: value,
+                                         status: seriesinfo_status)
+          end
+
+          def split_identifier(id)
+            if (match = DRAFT_ID_RE.match(id))
+              ["Internet-Draft", match[:value]]
+            elsif (match = HOME_ID_RE.match(id))
+              [match[:name].upcase, match[:value]]
+            end
+          end
+
+          # An untyped docidentifier that merely restates the reference's own
+          # label ("ZELLER", "Grail") is a citation label, not a citation: keep
+          # it only when the reference would otherwise show nothing at all.
+          def label_echo?(docid)
+            return false unless docid.type.nil?
+
+            docid.content.to_s.casecmp(label_candidates.to_s).zero? &&
+              reference_has_visible_text?
+          end
+
+          def label_candidates
+            @item.docnumber || @item.id
+          end
+
+          def reference_has_visible_text?
+            @item.title.any? || @item.formattedref ||
+              @item.contributor.any? || @item.date.any?
+          end
+
+          def seriesinfo_status
+            @item.status&.stage&.content
+          end
+
+          def authoritative_identifiers
+            ids = subseries_identifiers + docidentifier_identifiers
+            ids.reject { |id| id.empty? || IGNORED_ID_RE.match?(id) }.uniq
+          end
+
+          def docidentifier_identifiers
+            @item.docidentifier.reject { |di| non_authoritative?(di) }
+              .map { |di| di.content.to_s.strip }
+          end
+
+          def non_authoritative?(docid)
+            NON_AUTHORITATIVE_TYPES.include?(docid.type) ||
+              EXCLUDED_ID_TYPES.include?(docid.type) ||
+              docid.scope == "trademark" ||
+              DRAFT_ANCHOR_RE.match?(docid.content.to_s) ||
+              label_echo?(docid)
+          end
+
+          def subseries_identifiers
+            @item.series.filter_map do |ser|
+              title = ser.title.find { |t| SUBSERIES.include?(t.content.to_s) }
+              "#{title.content} #{ser.number}" if title && ser.number
+            end
+          end
+
+          def home_standard?
+            @item.contributor.any? do |contrib|
+              org = contrib.organization or next false
+
+              ([org.abbreviation&.content] + org.name.map(&:content))
+                .compact.any? { |name| HOME_ORGS.include?(name.to_s) }
+            end
+          end
+
+          # --- refcontent ---
+
+          def create_refcontent
+            (identifier_refcontent + series_refcontent)
+              .map { |text| ::Rfcxml::V3::Refcontent.new(content: text) }
+          end
+
+          # Identifiers that cannot be expressed as <seriesInfo> — either
+          # because the document is not an IETF one, or because the identifier
+          # does not split into a series name and number.
+          # Whatever <seriesInfo> already states must not be repeated here,
+          # whichever route put it there.
+          def identifier_refcontent
+            ids = authoritative_identifiers.reject { |id| in_seriesinfo?(id) }
+            ids.empty? ? [] : [ids.join(", ")]
+          end
+
+          def in_seriesinfo?(id)
+            name, value = split_identifier(id)
+            return false if name.nil?
+
+            create_seriesinfo.any? { |si| si.name == name && si.value == value }
+          end
+
+          def series_refcontent
+            (renderable_series - numbered_series).filter_map do |ser|
+              text = ser.formattedref&.content&.to_s || series_description(ser)
+              text unless text.strip.empty?
+            end
+          end
+
+          def series_description(ser)
+            [series_title(ser)&.content, ser.run, ser.organization,
+             ser.place&.city, series_dates(ser)]
+              .map(&:to_s).reject(&:empty?).join(", ")
+          end
+
+          def series_dates(ser)
+            [ser.from, ser.to].compact.map(&:to_s).reject(&:empty?).join("-")
+          end
+
+          # --- stream ---
+
+          # Values outside the grammar (relaton-ietf also has "Legacy") have no
+          # valid representation, so they are dropped with a warning rather
+          # than emitted as invalid XML.
+          def create_stream
+            value = ext_stream || series_stream
+            return nil if value.nil?
+
+            stream = STREAMS[value.to_s.strip.downcase]
+            unless stream
+              Util.warn "Dropping stream `#{value}`: not an RFC XML v3 stream"
+            end
+            stream
+          end
+
+          def ext_stream
+            @item.ext.respond_to?(:stream) ? @item.ext.stream : nil
+          end
+
+          def series_stream
+            ser = @item.series.find { |s| s.type == "stream" }
+            ser && ser.title.first&.content
+          end
+
+          # --- Contributors ---
+
+          # Every relaton-ietf RFC record carries publisher and authorizer
+          # contributors, and published RFC XML never lists those as authors.
+          # So: authors and editors if there are any, and only otherwise fall
+          # back to the wider set, which keeps a translator-only monograph its
+          # translator (metanorma-ietf#301).
+          def contributors_for_authors
+            contribs = super
+            primary = contribs.select { |c| role?(c, %w[author editor]) }
+            primary.any? ? primary : contribs.select { |c| fallback_author?(c) }
+          end
+
+          def role?(contrib, types)
+            contrib.role.any? { |r| types.include?(r.type) }
+          end
+
+          # A contributor with no role at all is an author by default; that is
+          # how most non-IETF bibitems are written.
+          def fallback_author?(contrib)
+            contrib.role.empty? || role?(contrib, FALLBACK_ROLES)
+          end
+
+          # --- ascii folding ---
+
+          def create_authors
+            super.each do |author|
+              author.ascii_fullname = ascii(author.fullname)
+              author.ascii_surname = ascii(author.surname)
+              author.ascii_initials = ascii(author.initials)
+            end
+          end
+
+          def create_organization(contrib)
+            org = super or return nil
+
+            org.ascii = ascii(Array(org.content).join)
+            org
+          end
+
+          # Transliterate to bare ASCII: expand the letters above, then strip
+          # the combining marks NFKD leaves behind. "Nürk" -> "Nurk",
+          # "Ö." -> "O.", "Sørensen" -> "Sorensen".
+          def ascii(str)
+            return nil if str.nil?
+
+            folded = str.to_s.gsub(/[#{TRANSLITERATIONS.keys.join}]/,
+                                   TRANSLITERATIONS)
+              .unicode_normalize(:nfkd)
+              .encode("ASCII", invalid: :replace, undef: :replace, replace: "")
+            # Folding a non-Latin script leaves only the punctuation and
+            # spacing behind ("Νίκος Παπαδόπουλος" -> " "), which is worse than
+            # no attribute at all. Real transliteration is the caller's policy.
+            folded.match?(/[[:alnum:]]/) ? folded : nil
+          end
+        end
+      end
+    end
+  end
+end

data/lib/relaton/bib/converter/bibxml.rb

@@ -1,5 +1,7 @@
 require_relative "bibxml/to_rfcxml"
 require_relative "bibxml/to_rfcxml_referencegroup"
+require_relative "bibxml/to_rfcxml_v3"
+require_relative "bibxml/to_rfcxml_referencegroup_v3"
 require_relative "bibxml/from_rfcxml"
 require_relative "bibxml/from_rfcxml_referencegroup"
 
@@ -16,17 +18,27 @@ module Relaton
         RFCPREFIXES = %w[RFC BCP FYI STD].freeze
 
         # Forward: ItemData -> Rfcxml model
-        def self.from_item(item, include_keywords: true) # rubocop:disable Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity
-          if bcp?(item)
-            ToRfcxmlReferencegroup.new(
-              item, include_keywords: include_keywords
-            ).transform
-          else
-            ToRfcxml.new(
-              item, include_keywords: include_keywords
-            ).transform
+        #
+        # @param v3 [Boolean] emit strict RFC 7991 (xml2rfc v3) instead of the
+        #   BibXML shape the relaton data fetchers publish
+        # @param anchor [String, nil] anchor to use instead of the one derived
+        #   from the item's docnumber/docidentifier
+        def self.from_item(item, include_keywords: true, v3: false, anchor: nil) # rubocop:disable Naming/MethodParameterName
+          klass = if v3 then v3_converter(item)
+                  elsif bcp?(item) then ToRfcxmlReferencegroup
+                  else ToRfcxml
+                  end
+          klass.new(item, include_keywords: include_keywords,
+                          anchor: anchor).transform
+        end
+
+        def self.v3_converter(item)
+          if item.relation.any? { |rel| rel.type == "includes" }
+            ToRfcxmlReferencegroupV3
+          else ToRfcxmlV3
           end
         end
+        private_class_method :v3_converter
 
         def self.bcp?(item) # rubocop:disable Metrics/CyclomaticComplexity
           item.docnumber&.match(/^BCP/) ||

data/lib/relaton/bib/item_data.rb

@@ -19,8 +19,18 @@ module Relaton
       COLLECTION_WRITE_ONLY_ATTRIBUTES = %i[title abstract source relation].freeze
 
       ATTRIBUTES.each { |attr| attr_accessor attr }
-      COLLECTION_ATTRBUTES.each { |attr| attr_accessor attr }
-      COLLECTION_WRITE_ONLY_ATTRIBUTES.each { |attr| attr_writer attr }
+      COLLECTION_ATTRBUTES.each { |attr| attr_reader attr }
+
+      # Collections are always arrays. lutaml-model assigns every mapped
+      # attribute after building the instance, so an element absent from the
+      # XML would otherwise overwrite the [] set in #initialize with nil --
+      # `Item.from_xml` items then blew up in any consumer that iterated them,
+      # while `from_yaml` ones did not.
+      (COLLECTION_ATTRBUTES + COLLECTION_WRITE_ONLY_ATTRIBUTES).each do |attr|
+        define_method("#{attr}=") do |value|
+          instance_variable_set("@#{attr}", array(value))
+        end
+      end
 
       attr_reader :id, :docidentifier
 
@@ -28,10 +38,10 @@ module Relaton
         ATTRIBUTES.each { |attr| instance_variable_set("@#{attr}", args[attr]) }
 
         (COLLECTION_ATTRBUTES + COLLECTION_WRITE_ONLY_ATTRIBUTES).each do |attr|
-          instance_variable_set("@#{attr}", args[attr] || [])
+          instance_variable_set("@#{attr}", array(args[attr]))
         end
 
-        @docidentifier = args[:docidentifier] || []
+        @docidentifier = array(args[:docidentifier])
         self.schema_version = schema
         @id = args[:id]
         create_id
@@ -42,7 +52,7 @@ module Relaton
       end
 
       def docidentifier=(value)
-        @docidentifier = value || []
+        @docidentifier = array(value)
         create_id
       end
 
@@ -179,8 +189,10 @@ module Relaton
         Converter::Asciibib.from_item(self)
       end
 
-      def to_rfcxml
-        Converter::BibXml.from_item(self).to_xml
+      # @param opts [Hash] passed to {Converter::BibXml.from_item}; notably
+      #   `v3: true` for strict RFC 7991 output instead of BibXML.
+      def to_rfcxml(**)
+        Converter::BibXml.from_item(self, **).to_xml
       end
 
       def deep_clone

data/lib/relaton/bib/model/abstract.rb

@@ -11,6 +11,11 @@ module Relaton
       key_value do
         map "format", to: :format
       end
+
+      # biblio.rng admits basicdoc blocks in <abstract> only.
+      def sanitizer_allowed
+        Relaton::Bib::Sanitizer::ALLOWED_WITH_BLOCKS
+      end
     end
   end
 end

data/lib/relaton/bib/model/localized_string.rb

@@ -28,10 +28,15 @@ module Relaton
     class LocalizedMarkedUpString < LocalizedStringAttrs
       module ContentSanitization
         def content=(value)
-          super(Relaton::Bib::Sanitizer.sanitize(value))
+          super(Sanitizer.sanitize(value, allowed: sanitizer_allowed))
         end
       end
 
+      # The elements that content keeps; Abstract widens it to blocks.
+      def sanitizer_allowed
+        Relaton::Bib::Sanitizer::ALLOWED
+      end
+
       # Cast at the assignment layer (lutaml-model#972): a bare scalar
       # assigned to a singular custom-typed attribute (e.g. a relation's
       # description) is wrapped here, so the serializer's type validation

data/lib/relaton/bib/sanitizer.rb

@@ -24,12 +24,27 @@ module Relaton
     # markup, and must be preserved verbatim. Without the opaque-skip,
     # the recursive walk would unwrap MathML / AsciiMath elements down to
     # bare text nodes — see #116 for the round-trip-loss symptom.
+    #
+    # ALLOWED_WITH_BLOCKS adds the basicdoc block elements: lists, notes,
+    # tables, figures and admonitions, with the children that their grammar
+    # requires (a caption <name>, <thead>/<tbody>/<tfoot>, a figure's
+    # <image>). Only Abstract asks for it, because <abstract> is the one
+    # field whose grammar admits blocks; a title or a note admits only
+    # PureTextElement. Before relaton#421 an abstract's list or note
+    # unwrapped in place and came back as bare paragraphs, with no error.
     module Sanitizer
       ALLOWED = %w[
         em strong sub sup tt underline strike smallcap br stem
         p eref xref fn link
       ].freeze
 
+      BLOCK = %w[
+        ul ol li dl dt dd note admonition
+        table thead tbody tfoot tr td th figure name image
+      ].freeze
+
+      ALLOWED_WITH_BLOCKS = (ALLOWED + BLOCK).freeze
+
       # Elements whose children are non-basicdoc inline notation
       # (MathML, AsciiMath, LaTeX, …) and must be preserved verbatim
       # rather than sanitised against ALLOWED.
@@ -63,13 +78,13 @@ module Relaton
       # Reserved prefixes. XML declares both, so the content must not.
       NS_RESERVED = %w[xml xmlns].freeze
 
-      def self.sanitize(content)
+      def self.sanitize(content, allowed: ALLOWED)
         return content unless sanitizable?(content)
 
         node = parse(content)
         return content if node.nil?
 
-        sanitize_children(node)
+        sanitize_children(node, allowed)
         node.children.map { |c| c.to_xml(encoding: "UTF-8", indent: 0, expand_empty: false) }.join
       end
 
@@ -273,15 +288,15 @@ module Relaton
       end
       private_class_method :sanitizable?
 
-      def self.sanitize_children(node)
+      def self.sanitize_children(node, allowed)
         node.children.to_a.each do |child|
           next unless child.element?
 
           child.name = RENAME[child.name] if RENAME.key?(child.name)
           next if OPAQUE.include?(child.name)
 
-          sanitize_children(child)
-          unwrap(child) unless ALLOWED.include?(child.name)
+          sanitize_children(child, allowed)
+          unwrap(child) unless allowed.include?(child.name)
         end
       end
       private_class_method :sanitize_children

data/lib/relaton/ietf/bibxml_parser.rb

@@ -131,6 +131,12 @@ module Relaton
           BibXMLParser.pubid_type(id)
         end
 
+        # The base parser keeps <stream> as a `type="stream"` series, for items
+        # with no other slot. This flavor carries the stream in `ext.stream`.
+        def stream_series
+          []
+        end
+
         def contributor # rubocop:disable Metrics/MethodLength
           contribs = []
           unless @reference.anchor&.match?(/^I-D/)

data/lib/relaton/iho/CLAUDE.md

@@ -55,9 +55,11 @@ In `item.rb`, `require_relative "relation"` is placed **after** the `Item` class
 
 `Bibliography.search`/`.get` fetches YAML from the relaton-data-iho GitHub repo via `Relaton::Index`, then deserializes into model objects.
 
-**Editions.** The pubid `version` is the edition (`IHO B-12 3.0.0`; the grammar takes `a.b.c` only). A reference with no version answers with the **latest** edition: `version_key` compares the segments as integers (`2.10.0` after `2.9.0`). It used to be `min_by` on the string, which returned the oldest.
+**Editions and series.** The pubid `version` is the edition (`IHO B-12 3.0.0`; the grammar takes `a.b.c` only). A reference with no version answers with the **latest** edition: `recency_key` compares the segments as integers (`2.10.0` after `2.9.0`). It used to be `min_by` on the string, which returned the oldest.
 
-**The edition reaches the index only through the committed row.** A record prints `B-12` and keeps its edition in `edition:`, so `relaton index` cannot read a version from the docid. `MachineIndex#refine` (relaton-cli) takes the committed `index-v3.yaml` row when it adds only a `version` to the parse. Before that, the Pages index (generated 2026-10-29) had no `version` in any of its 325 rows, so every versioned reference answered not-found (relaton-data-iho#20). The suite did not see it: `spec/fixtures/index-v3.zip` is the committed index, which has versions.
+**Series are identified by year or volume, not an edition (relaton#407).** B-7, P-5 and P-6 carry a `year` (`IHO P-5 2024`), the IHR (P-1) a `volume` with an optional `issue` (`IHO P-1 Vol. 30 (2)`) — plain scalars added by pubid#635, mutually exclusive with `version` (the grammar's tail picks exactly one). The same `recency_key` orders each kind numerically, so a bare series query answers with the latest year/volume/issue. Matching (`pubid_match?`) treats a component the query omits as a wildcard: `IHO P-1 Vol. 30` matches any issue of volume 30. The comparison runs through pubid's `#exclude` — excluding `:volume` nils the `issue` with it (the pair is one scalar cluster) — the way BIPM's `stem_keys` does. The subdivisions (part/appendix/annex/supplement) stay strict: this flavor cannot use the `===` subset match, because pubid does not declare `subset_strict` for IHO's subdivisions and a nil `part` would read as "any part" (`IHO S-100` would answer with a part row).
+
+**The edition reaches the index only through the committed row.** A record prints `B-12` and keeps its edition in `edition:`, so `relaton index` cannot read a version from the docid. `MachineIndex#refine` (relaton-cli) takes the committed `index-v3.yaml` row when it adds only a key in `REFINABLE_KEYS` to the parse — `version`/`edition`/`volume`/`year`/`issue` (relaton#407 added the year/issue pair for the series rows). Before that, the Pages index (generated 2026-10-29) had no `version` in any of its 325 rows, so every versioned reference answered not-found (relaton-data-iho#20). The suite did not see it: `spec/fixtures/index-v3.zip` is the committed index, which has versions. The suite's fixture currently carries the **re-keyed** series rows (year/volume) ahead of the data repo: the placeholder `1.0.0` rows for p-1/p-1-2x/p-5/p-6/b-7 are replaced by `year`/`volume` rows — refresh the fixture from the live published zip once relaton-data-iho re-keys, and reconcile any file-name drift then.
 
 ### Test Infrastructure
 

data/lib/relaton/iho/bibliography.rb

@@ -21,7 +21,7 @@ module Relaton
           # Pubid::Iho::Identifiers::* (Relaton::Index deserialized it via the
           # `pubid_class` passed in `#index`), so `pubid_match?` compares pubids.
           row = index.search(pubid) { |r| pubid_match?(r[:id], pubid) }
-                     .max_by { |r| version_key(r[:id]) }
+                     .max_by { |r| recency_key(r[:id]) }
           unless row
             Util.info "Not found.", key: pubid.to_s
             return
@@ -96,26 +96,57 @@ module Relaton
           )
         end
 
-        # The sort key for the latest edition: segments compare as integers,
-        # so 2.10.0 is later than 2.9.0. A row with no version is the oldest.
-        def version_key(id)
-          id.version.to_s.split(".").map(&:to_i)
+        # The sort key for the latest record in a bucket (relaton#407). Each
+        # IHO identity carries exactly one trailing discriminator — a dotted
+        # `version` (an edition, `B-12 3.0.0`), a series `year` (`P-5 2024`)
+        # or a series `volume` with its `issue` (`P-1 Vol. 30 (2)`) — and the
+        # key orders by that component, numerically: version segments compare
+        # as integers (2.10.0 is later than 2.9.0), as do year, volume and
+        # issue. A row with no discriminator at all is the oldest. Rows in
+        # one bucket share the discriminator kind once the data is keyed;
+        # while kinds coexist the array compare is deterministic, and a
+        # year's bare [2024] beats a placeholder edition's [1, 0, 0].
+        def recency_key(id)
+          if id.year
+            [id.year.to_i]
+          elsif id.volume
+            [id.volume.to_i, id.issue.to_i]
+          else
+            id.version.to_s.split(".").map(&:to_i)
+          end
         end
 
+        # The components that make an IHO document the one it is beyond the
+        # series and subdivisions: the edition (`version`), or — for the
+        # series identified by year or volume instead (B-7, P-1, P-5, P-6;
+        # relaton#407) — the `year` or the `volume`/`issue` pair.
+        DISCRIMINATOR_KEYS = %i[version year volume issue].freeze
+
         # Both `row_id` and `query` are Pubid::Iho::Identifiers::* instances.
         # Class identity pins the series (S/P/M/B/C). Subdivision keys
         # (part/appendix/annex/supplement) use strict equality — a nil query
         # must match a nil row (the umbrella), not an arbitrary subdivision
-        # under the same (number, version). Only :version stays nil-tolerant: an
-        # unqualified `IHO B-11` query is expected to find the latest edition.
+        # under the same number. That is why this flavor does not use the
+        # `===` subset match: pubid reads a nil `part` there as "any part"
+        # (pubid's `subset_strict` is not declared for the IHO subdivisions),
+        # so `IHO S-100` would answer with a part row.
+        #
+        # The discriminator keys stay nil-tolerant: an unqualified `IHO B-11`
+        # finds the latest edition, an unqualified `IHO P-5` the latest year,
+        # and `IHO P-1 Vol. 30` any issue of that volume — while a stated
+        # component must match exactly. The comparison runs through pubid's
+        # `#exclude` (the primitive `matches?` is built on), so the rules the
+        # flavor declares there ride along: excluding `:volume` nils the
+        # `issue` with it. `stem_keys` in the BIPM flavor is the same shape.
         def pubid_match?(row_id, query)
           row_id.class == query.class &&
-            row_id.number == query.number &&
-            row_id.part.to_s == query.part.to_s &&
-            row_id.appendix.to_s == query.appendix.to_s &&
-            row_id.annex.to_s == query.annex.to_s &&
-            row_id.supplement.to_s == query.supplement.to_s &&
-            (query.version.nil? || row_id.version.to_s == query.version.to_s)
+            row_id.exclude(*wildcards(query)) == query.exclude(*wildcards(query))
+        end
+
+        # The discriminators the query leaves unset: wildcards for the
+        # search, significant whenever the query states one.
+        def wildcards(query)
+          DISCRIMINATOR_KEYS.select { |key| query.public_send(key).nil? }
         end
       end
     end

data/lib/relaton/version.rb

@@ -1,3 +1,3 @@
 module Relaton
-  VERSION = "3.0.4".freeze
+  VERSION = "3.0.5".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: relaton
 version: !ruby/object:Gem::Version
-  version: 3.0.4
+  version: 3.0.5
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-11-01 00:00:00.000000000 Z
+date: 2026-11-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: addressable
@@ -408,28 +408,28 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.3.114
+        version: 1.3.121
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.3.114
+        version: 1.3.121
 - !ruby/object:Gem::Dependency
   name: pubid
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 2.0.1
+        version: 2.0.3
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 2.0.1
+        version: 2.0.3
 - !ruby/object:Gem::Dependency
   name: yeptris
   requirement: !ruby/object:Gem::Requirement
@@ -566,6 +566,8 @@ files:
 - lib/relaton/bib/converter/bibxml/from_rfcxml_referencegroup.rb
 - lib/relaton/bib/converter/bibxml/to_rfcxml.rb
 - lib/relaton/bib/converter/bibxml/to_rfcxml_referencegroup.rb
+- lib/relaton/bib/converter/bibxml/to_rfcxml_referencegroup_v3.rb
+- lib/relaton/bib/converter/bibxml/to_rfcxml_v3.rb
 - lib/relaton/bib/converter/csl.rb
 - lib/relaton/bib/converter/ris.rb
 - lib/relaton/bib/converter/titles.rb