relaton-nist 1.20.4 → 2.0.0.pre.alpha.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d05e892d5c1bb5b67b8165cd69cbc431b8d6f7ea443b3acb98baf66b9edfba0
-  data.tar.gz: 419aac532181bbe1b52e5eab80b657a8fb2b645282b0f88bdcd1c7cdca337118
+  metadata.gz: c03d23fadbcef92134d86d7df992c92c185202d0385d0a788944a281e69dd2c4
+  data.tar.gz: 05ea6538b1c9383a3ce1df1636bedfc4b044115d335b09d34b72a8ca60ca858c
 SHA512:
-  metadata.gz: 28f8c667337a14db4429205bdb7514f8990609dc9dd3c52d62e25b175e4e6c0a3c9984e049f01f709f22378995c8f5a74e57d9496d36e2c67cf9f1e4c42376dc
-  data.tar.gz: 92f1e0eaeb4a82a37da3cf951a99af009a397909caef0b654b51463e741476523d8257532e4f72e80d161027695ace88539df7c6c693ad176c300753154a2738
+  metadata.gz: 7f20c86344e122030a98651429bbf43559b2d0955ebbd5697f071c20421996f2bad161b0967bc6171888251bcbd77d3b8c615803d17b93de08162734a7a79c71
+  data.tar.gz: fd93a125142aa2274f8bb0d00825cde4d21ba701529e112c998c61b3f698a36506d44f449b80072dde7270c9dfb01cb1b5fe066df0be8da3edae499abcfd36ec

data/.gitignore

@@ -15,3 +15,4 @@
 pubs-export.json
 Gemfile.lock
 .byebug_history
+.claude/

data/.rubocop.yml

@@ -7,6 +7,6 @@ require: rubocop-rails
 inherit_from:
   - https://raw.githubusercontent.com/riboseinc/oss-guides/master/ci/rubocop.yml
 AllCops:
-  TargetRubyVersion: 3.1
+  TargetRubyVersion: 3.2
 Rails:
   Enabled: false

data/CLAUDE.md

@@ -0,0 +1,96 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+relaton-nist is a Ruby gem that retrieves and manages NIST (National Institute of Standards and Technology) bibliographic metadata as part of the Relaton ecosystem. Data sources include the NIST Cybersecurity Resource Center (CSRC) and the NIST Library (MARC21/MODS XML).
+
+## Development Commands
+
+```bash
+# Install dependencies
+bundle install
+
+# Run all tests (any of these)
+rake
+rake spec
+bundle exec rspec
+
+# Run a single test file
+bundle exec rspec spec/relaton/nist/item_spec.rb
+
+# Run a specific test by line number
+bundle exec rspec spec/relaton/nist/item_spec.rb:10
+
+# Interactive console with gem loaded
+bin/console
+
+# Lint (uses Ribose OSS rubocop config)
+bundle exec rubocop
+```
+
+## Architecture
+
+### Module Structure
+
+All classes live under `Relaton::Nist` (entry point: `require 'relaton/nist'`).
+
+### Lutaml-Model Serialization
+
+All data models inherit from `Lutaml::Model::Serializable` and use declarative attribute/mapping DSL for XML and YAML serialization. The base bibliographic types come from `relaton-bib` (aliased as `Bib`).
+
+**Core model hierarchy**:
+- `Bib::Item` → `Relaton::Nist::Item` (`model ItemData`)
+  - `Relaton::Nist::Bibitem` (includes `Bib::BibitemShared`)
+  - `Relaton::Nist::Bibdata` (includes `Bib::BibdataShared`)
+- `Bib::ItemData` → `Relaton::Nist::ItemData`
+
+**NIST-specific extensions** (under `lib/relaton/nist/`):
+- `Ext` — extension block holding doctype, comment period (extends `Bib::Ext`)
+- `Date` — adds `abandoned` and `superseded` date types (extends `Bib::Date`)
+- `Relation` — adds `obsoletedBy`, `supersedes`, `supersededBy` relation types (extends `Bib::Relation`)
+- `CommentPeriod` — from/to/extended date ranges (extends `Lutaml::Model::Serializable`)
+- `Doctype` — currently only `standard` (extends `Bib::Doctype`)
+
+**Search/retrieval layer** (extends `relaton-core`):
+- `Bibliography` — class methods `search(text, year, opts)` and `get(code, year, opts)`
+- `HitCollection` < `Core::HitCollection` — searches GitHub index + CSRC JSON
+- `Hit` < `Core::Hit` — lazily resolves items via `Scraper`
+- `Processor` < `Core::Processor` — Relaton plugin interface (`get`, `fetch_data`, `from_xml`, `from_yaml`)
+
+**Data fetching/parsing**:
+- `DataFetcher` < `Core::DataFetcher` — fetches NIST Tech Pubs MODS XML from GitHub releases
+- `ModsParser` — maps MODS XML (via `loc_mods`) to `ItemData`
+- `Scraper` — fetches items from GitHub YAML or CSRC JSON
+- `PubsExport` — singleton; caches CSRC pubs-export zip with thread-safe daily updates
+
+### Serialization Round-Trip Pattern
+
+Models support `from_xml`/`to_xml` and `from_yaml`/`to_yaml`. Tests verify round-trip fidelity by parsing a fixture, re-serializing, and comparing output to input.
+
+### XML Schema Validation
+
+RNG (Relax NG) schemas in `grammars/` validate XML output. Tests use the `jing` gem:
+```ruby
+schema = Jing.new("grammars/relaton-nist-compile.rng")
+errors = schema.validate(file)
+```
+`relaton-nist-compile.rng` is the top-level schema that includes `relaton-nist.rng` and `basicdoc.rng`.
+
+### HTTP Recording
+
+Tests use VCR with WebMock. Cassettes are stored in `spec/vcr_cassettes/` and re-record every 7 days.
+
+## Key Dependencies
+
+- `relaton-bib` — base bibliographic models and shared mixins
+- `relaton-core` — base classes for Processor, HitCollection, Hit, DataFetcher
+- `loc_mods` — MODS (Metadata Object Description Schema) XML parsing
+- `pubid` — NIST publication ID parsing
+- `relaton-index` — index/search utilities
+- `mechanize` — HTTP fetching for data sources
+
+## Code Style
+
+RuboCop config inherits from the Ribose OSS style guide. Target Ruby version is 3.1.

data/README.adoc

@@ -98,7 +98,7 @@ Or install it yourself as:
 
 [source,sh]
 ----
-$ gem install relaton_nist
+$ gem install relaton-nist
 ----
 
 == Usage
@@ -120,19 +120,16 @@ https://www.nist.gov/news-events/news/2021/08/nist-technical-series-publications
 ====
 [source,ruby]
 ----
-> require 'relaton_nist'
-=>
-true
-
-> hit_collection = RelatonNist::NistBibliography.search("NISTIR 8200")
-[relaton-nist] (NIST IR 8200) Fetching from csrc.nist.gov ...
-[relaton-nist] (NIST IR 8200) Fetching from Relaton repository ...
-=>
-<RelatonNist::HitCollection:0x00000000004b28 @ref=NIST IR 8200 @fetched=false>
-
-> item = hit_collection[0].fetch
-=>
-#<RelatonNist::NistBibliographicItem:0x007fc049aa6778
+require 'relaton/nist'
+=> true
+
+hit_collection = Relaton::Nist::Bibliography.search("NISTIR 8200")
+[relaton-nist] INFO: (NIST IR 8200) Fetching from csrc.nist.gov ...
+[relaton-nist] INFO: (NIST IR 8200) Fetching from Relaton repository ...
+=> <Relaton::Nist::HitCollection:0x00000000002c10 @ref=NIST IR 8200 @fetched=false>
+
+item = hit_collection[0].item
+=> #<Relaton::Nist::ItemData:0x000000012c1d8008
 # ...
 ----
 ====
@@ -146,12 +143,11 @@ revisions or editions.
 ====
 [source,ruby]
 ----
-> RelatonNist::NistBibliography.get("NIST IR 8200", "2018")
-[relaton-nist] (NIST SP 8200:2018) Fetching from csrc.nist.gov ...
-[relaton-nist] (NIST IR 8200:2018) Fetching from Relaton repository...
-[relaton-nist] (NIST IR 8200:2018) Found: `NIST IR 8200`
-=>
-#<RelatonNist::NistBibliographicItem:0x00007fab74a572c0
+Relaton::Nist::Bibliography.get("NIST IR 8200", "2018")
+[relaton-nist] INFO: (NIST IR 8200) Fetching from csrc.nist.gov ...
+[relaton-nist] INFO: (NIST IR 8200) Fetching from Relaton repository ...
+[relaton-nist] INFO: (NIST IR 8200) Found: `NIST IR 8200`
+=> #<Relaton::Nist::ItemData:0x000000012c5e7590
 # ...
 ----
 ====
@@ -170,14 +166,14 @@ A NIST PubID can contain these optional parameters `{ptN}{vN}{verN}{rN}{/Add}`:
 ====
 [source,ruby]
 ----
-item = RelatonNist::NistBibliography.get 'NIST SP 800-67r1'
-[relaton-nist] (NIST SP 800-67r1) Fetching from csrc.nist.gov ...
-[relaton-nist] (NIST SP 800-67r1) Found: `NIST SP 800-67 Rev. 1`
-=> #<RelatonNist::NistBibliographicItem:0x00000001105acd08
+item = Relaton::Nist::Bibliography.get 'NIST SP 800-67r1'
+[relaton-nist] INFO: (NIST SP 800-67r1) Fetching from csrc.nist.gov ...
+[relaton-nist] INFO: (NIST SP 800-67r1) Found: `NIST SP 800-67r1 fpd`
+=> #<Relaton::Nist::ItemData:0x000000012cf294c8
 ...
 
-item.docidentifier.first.id
-=> "NIST SP 800-67 Rev. 1"
+item.docidentifier.first.content
+=> "NIST SP 800-67r1 fpd"
 ----
 ====
 
@@ -186,14 +182,14 @@ item.docidentifier.first.id
 ====
 [source,ruby]
 ----
-item = RelatonNist::NistBibliography.get 'NIST SP 800-38A/Add'
-[relaton-nist] (NIST SP 800-38A/Add) Fetching from csrc.nist.gov ...
-[relaton-nist] (NIST SP 800-38A/Add) Found: `NIST SP 800-38A-Add`
-=> #<RelatonNist::NistBibliographicItem:0x00000001105abf48
+item = Relaton::Nist::Bibliography.get 'NIST SP 800-38A/Add'
+[relaton-nist] INFO: (NIST SP 800-38A Add) Fetching from csrc.nist.gov ...
+[relaton-nist] INFO: (NIST SP 800-38A Add) Found: `NIST SP 800-38A fpd Add`
+=> #<Relaton::Nist::ItemData:0x0000000124864190
 ...
 
-item.docidentifier.first.id
-=> "NIST SP 800-38A-Add"
+item.docidentifier.first.content
+=> "NIST SP 800-38A fpd Add"
 ----
 ====
 
@@ -225,11 +221,10 @@ In case the state is "draft" it should be:
 ====
 [source,ruby]
 ----
-> RelatonNist::NistBibliography.get("NIST SP 800-205 (February 2019) (IPD)")
-[relaton-nist] (NIST SP 800-205) Fetching from csrc.nist.gov ...
-[relaton-nist] (NIST SP 800-205) Found: `NIST SP 800-205 (Draft)`
-=>
-#<RelatonNist::NistBibliographicItem:0x00000001105afdc8
+Relaton::Nist::Bibliography.get("NIST SP 800-205 (February 2019) (IPD)")
+[relaton-nist] INFO: (NIST SP 800-205) Fetching from csrc.nist.gov ...
+[relaton-nist] INFO: (NIST SP 800-205) Found: `NIST SP 800-205 ipd`
+=> #<Relaton::Nist::ItemData:0x000000012494fb40
 # ...
 ----
 ====
@@ -247,11 +242,10 @@ The format for FIPS publications is:
 ====
 [source,ruby]
 ----
-> RelatonNist::NistBibliography.get("NIST FIPS 140-3")
+Relaton::Nist::Bibliography.get("NIST FIPS 140-3")
 [relaton-nist] INFO: (NIST FIPS 140-3) Fetching from csrc.nist.gov ...
-[relaton-nist] INFO: (NIST FIPS 140-3) Found: `NIST FIPS 140-3`
-=>
-# #<RelatonNist::NistBibliographicItem:0x000000013b68fb50
+[relaton-nist] INFO: (NIST FIPS 140-3) Found: `NIST FIPS 140-3 2pd`
+=> #<Relaton::Nist::ItemData:0x0000000124d8dfe0
 # ...
 ----
 ====
@@ -266,12 +260,16 @@ The format for FIPS publications is:
 ====
 [source,ruby]
 ----
-> item.to_xml
-#=> "<bibitem id="NISTIR8200" type="standard" schema-version="v1.2.9">
-#      <fetched>2023-10-16</fetched>
-#      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
-#      ...
-#    <bibitem>"
+item.to_xml
+=> "<bibitem id="NISTSP80038AfpdAdd" type="standard" schema-version="v1.4.1">
+      <fetched>2026-02-26</fetched>
+      <title language="en" script="Latn">Recommendation for Block Cipher Modes of Operation - Three Variants of Ciphertext Stealing for CBC Mode</title>
+      <uri type="src">https://csrc.nist.gov/pubs/sp/800/38/a/sup/final</uri>
+      <uri type="doi">https://doi.org/10.6028/NIST.SP.800-38A-Add</uri>
+      <docidentifier type="NIST" primary="true">NIST SP 800-38A fpd Add</docidentifier>
+      <docidentifier type="DOI">NIST.SP.800-38A-Add</docidentifier>
+      ...
+    </bibitem>"
 ----
 ====
 
@@ -283,16 +281,20 @@ the `bibdata` element and adds a flavor-specific `ext` element.
 ====
 [source,ruby]
 ----
-> item.to_xml bibdata: true
-=>
-"<bibdata type="standard" schema-version="v1.2.9">
-#      <fetched>2023-10-16</fetched>
-#      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
-#      ...
-#      <ext schema-version="v1.0.0">
-#        <doctype>standard</doctype>
-#      </ext>
-#  </bibdata>"
+item.to_xml bibdata: true
+=> "<bibdata type="standard" schema-version="v1.4.1">
+      <fetched>2026-02-26</fetched>
+      <title language="en" script="Latn">Recommendation for Block Cipher Modes of Operation - Three Variants of Ciphertext Stealing for CBC Mode</title>
+      <uri type="src">https://csrc.nist.gov/pubs/sp/800/38/a/sup/final</uri>
+      <uri type="doi">https://doi.org/10.6028/NIST.SP.800-38A-Add</uri>
+      <docidentifier type="NIST" primary="true">NIST SP 800-38A fpd Add</docidentifier>
+      <docidentifier type="DOI">NIST.SP.800-38A-Add</docidentifier>
+      ...
+      <ext schema-version="v1.0.0">
+        <doctype>standard</doctype>
+        <flavor>nist</flavor>
+      </ext>
+    </bibdata>
 ----
 ====
 
@@ -318,18 +320,16 @@ They can be accessed through the following methods:
 NIST publications may have `src` and `doi` links, obtained using the `#link`
 method.
 
-.Accessing links of a NIST bibliographic item
+.Accessing source links of a NIST bibliographic item
 [example]
 ====
 [source,ruby]
 ----
-item.link
-=> [#<RelatonBib::TypedUri:0x0000000111087a98
-  @content=#<Addressable::URI:0x8c0 URI:https://csrc.nist.gov/pubs/sp/800/38/a/sup/final>,
-  @language=nil,
-  @script=nil,
-  @type="src">,
- #<RelatonBib::TypedUri:0x00000001110879a8 @content=#<Addressable::URI:0x8d4 URI:https://doi.org/10.6028/NIST.SP.800-38A-Add>, @language=nil, @script=nil, @type="doi">]
+item.source[0].type
+=> "src"
+
+item.source[0].content
+=> "https://csrc.nist.gov/pubs/sp/800/38/a/sup/final"
 ----
 ====
 
@@ -347,12 +347,8 @@ Offline Relaton YAML files can be loaded directly as bibliographic items.
 ====
 [source,ruby]
 ----
-hash = YAML.load_file 'spec/examples/nist_bib_item.yml'
-=> {"id"=>"NISTIR 8011 Vol. 3",
-...
-
-RelatonNist::NistBibliographicItem.from_hash hash
-=> #<RelatonNist::NistBibliographicItem:0x007f8b708505b8
+Relaton::Nist::Item.from_yaml File.read("spec/fixtures/item.yaml")
+=> #<Relaton::Nist::ItemData:0x0000000124d4ab50
 ...
 ----
 ====
@@ -371,7 +367,9 @@ The following method fetches all the documents from the NIST-Tech-Pubs dataset,
 ====
 [source,ruby]
 ----
-> RelatonNist::DataFetcher.fetch
+require "relaton/nist/data_fetcher"
+
+Relaton::Nist::DataFetcher.fetch
 Started at: 2021-09-01 18:01:01 +0200
 Stopped at: 2021-09-01 18:01:43 +0200
 Done in: 42 sec.
@@ -384,7 +382,7 @@ Done in: 42 sec.
 ====
 [source,ruby]
 ----
-> RelatonNist::DataFetcher.fetch(output: "data", format: "yaml")
+> Relaton::Nist::DataFetcher.fetch(output: "data", format: "yaml")
 ----
 ====