relaton 3.1.1 → 3.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. data/README.adoc +5 -3
  3. data/lib/relaton/3gpp.rb +1 -1
  4. data/lib/relaton/bib/converter/csl/from_csl.rb +195 -0
  5. data/lib/relaton/bib/converter/csl.rb +13 -1
  6. data/lib/relaton/bib/model/docidentifier.rb +66 -5
  7. data/lib/relaton/calconnect/hit_collection.rb +1 -1
  8. data/lib/relaton/cli/command.rb +3 -1
  9. data/lib/relaton/cli/frontend_assets.rb +7 -6
  10. data/lib/relaton/cli/subcommand_collection.rb +5 -1
  11. data/lib/relaton/core/request_error.rb +18 -0
  12. data/lib/relaton/db/config.rb +9 -1
  13. data/lib/relaton/db/workers_pool.rb +11 -1
  14. data/lib/relaton/db.rb +160 -58
  15. data/lib/relaton/ecma/bibliography.rb +2 -2
  16. data/lib/relaton/etsi/data_fetcher.rb +16 -6
  17. data/lib/relaton/iala/bibliography.rb +1 -1
  18. data/lib/relaton/iana.rb +1 -1
  19. data/lib/relaton/iec/data_parser.rb +41 -2
  20. data/lib/relaton/itu/README.adoc +1 -1
  21. data/lib/relaton/itu/data_crawler_r.rb +1 -1
  22. data/lib/relaton/jis/README.adoc +2 -0
  23. data/lib/relaton/jis/data_fetcher.rb +74 -13
  24. data/lib/relaton/jis/hit_collection.rb +2 -1
  25. data/lib/relaton/jis/processor.rb +1 -1
  26. data/lib/relaton/jis/scraper.rb +46 -5
  27. data/lib/relaton/oasis/bibliography.rb +1 -1
  28. data/lib/relaton/ogc/hit_collection.rb +1 -1
  29. data/lib/relaton/ogc.rb +1 -1
  30. data/lib/relaton/plateau/data_fetcher.rb +10 -1
  31. data/lib/relaton/plateau/handbook_parser.rb +38 -2
  32. data/lib/relaton/plateau/hit_collection.rb +20 -5
  33. data/lib/relaton/sae.rb +1 -1
  34. data/lib/relaton/version.rb +1 -1
  35. data/lib/relaton/w3c/bibliography.rb +1 -1
  36. data/lib/relaton/xsf/bibliography.rb +1 -1
  37. data/lib/relaton/xsf.rb +1 -1
  38. metadata +35 -48
  39. data/lib/relaton/3gpp/CLAUDE.md +0 -218
  40. data/lib/relaton/adobe/CLAUDE.md +0 -104
  41. data/lib/relaton/bib/CLAUDE.md +0 -150
  42. data/lib/relaton/bipm/CLAUDE.md +0 -187
  43. data/lib/relaton/bsi/CLAUDE.md +0 -35
  44. data/lib/relaton/calconnect/CLAUDE.md +0 -177
  45. data/lib/relaton/ccsds/CLAUDE.md +0 -74
  46. data/lib/relaton/cen/CLAUDE.md +0 -219
  47. data/lib/relaton/cie/CLAUDE.md +0 -90
  48. data/lib/relaton/core/CLAUDE.md +0 -140
  49. data/lib/relaton/db/CLAUDE.md +0 -385
  50. data/lib/relaton/doi/CLAUDE.md +0 -105
  51. data/lib/relaton/easc/CLAUDE.md +0 -85
  52. data/lib/relaton/ecma/CLAUDE.md +0 -217
  53. data/lib/relaton/etsi/CLAUDE.md +0 -219
  54. data/lib/relaton/evs/CLAUDE.md +0 -120
  55. data/lib/relaton/gb/CLAUDE.md +0 -47
  56. data/lib/relaton/gost/CLAUDE.md +0 -100
  57. data/lib/relaton/iala/CLAUDE.md +0 -97
  58. data/lib/relaton/iana/CLAUDE.md +0 -105
  59. data/lib/relaton/iec/CLAUDE.md +0 -13
  60. data/lib/relaton/ieee/CLAUDE.md +0 -202
  61. data/lib/relaton/ietf/CLAUDE.md +0 -382
  62. data/lib/relaton/iho/CLAUDE.md +0 -76
  63. data/lib/relaton/index/CLAUDE.md +0 -208
  64. data/lib/relaton/isbn/CLAUDE.md +0 -47
  65. data/lib/relaton/iso/CLAUDE.md +0 -38
  66. data/lib/relaton/itu/CLAUDE.md +0 -685
  67. data/lib/relaton/jcgm/CLAUDE.md +0 -107
  68. data/lib/relaton/jis/CLAUDE.md +0 -64
  69. data/lib/relaton/logger/CLAUDE.md +0 -37
  70. data/lib/relaton/nist/CLAUDE.md +0 -194
  71. data/lib/relaton/oasis/CLAUDE.md +0 -251
  72. data/lib/relaton/ogc/CLAUDE.md +0 -242
  73. data/lib/relaton/oiml/CLAUDE.md +0 -47
  74. data/lib/relaton/omg/CLAUDE.md +0 -137
  75. data/lib/relaton/plateau/CLAUDE.md +0 -126
  76. data/lib/relaton/sae/CLAUDE.md +0 -91
  77. data/lib/relaton/sdo/CLAUDE.md +0 -81
  78. data/lib/relaton/un/CLAUDE.md +0 -91
  79. data/lib/relaton/w3c/CLAUDE.md +0 -369
  80. data/lib/relaton/xsf/CLAUDE.md +0 -189
@@ -1,137 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-omg is a Ruby gem that searches and fetches standards from The Object Management Group (OMG) at https://www.omg.org. It is part of the larger Relaton family of bibliographic data gems (v2.0.0-alpha.1).
8
-
9
- ## Commands
10
-
11
- - **Run all tests:** `bundle exec rspec`
12
- - **Run a single test:** `bundle exec rspec spec/relaton/omg/item_spec.rb`
13
- - **Lint:** `bundle exec rubocop` (follows Ribose OSS style guide)
14
- - **Lint with autofix:** `bundle exec rubocop -a`
15
- - **Install deps:** `bundle install`
16
-
17
- ## Architecture
18
-
19
- ### Module structure
20
-
21
- All source lives under `lib/relaton/omg/`. The main entry point is `lib/relaton/omg.rb` which loads everything under the `Relaton::Omg` module namespace.
22
-
23
- ### Class hierarchy
24
-
25
- The gem extends `relaton-bib` (~> 2.0.0-alpha.1), the core Relaton bibliographic data library:
26
-
27
- - `Relaton::Omg::Item` < `Bib::Item` — base item class, uses `Omg::ItemData` model (subclass of `Bib::ItemData`)
28
- - `Relaton::Omg::Ext` < `Bib::Ext` — overrides `get_schema_version` to return the OMG model version
29
- - `Relaton::Omg::Bibitem` < `Item` — includes `Bib::BibitemShared`, for `<bibitem>` XML
30
- - `Relaton::Omg::Bibdata` < `Item` — includes `Bib::BibdataShared`, for `<bibdata>` XML
31
- - `Relaton::Omg::Docidentifier` < `Bib::Docidentifier` — the document id, backed by `Pubid::Omg`; `Item` declares it, so XML and YAML deserialization produce it too
32
- - `Relaton::Omg::Processor` < `Core::Processor` — relaton-core integration, delegates to `Bibliography`, `Bibitem`, `Item`
33
- - `Relaton::Omg::Bibliography` — fetches standards via `Scraper`
34
- - `Relaton::Omg::Scraper` — scrapes https://www.omg.org/spec for bibliographic data
35
-
36
- ### References are parsed with pubid
37
-
38
- OMG uses `Pubid::Omg` and no hand regex. The flavor historically had no index
39
- and no data repository (the BSI and CEN shape: parse the reference, and back
40
- the document id with pubid). That changed: **relaton-data-omg** now crawls
41
- www.omg.org/spec and serves the machine index on Pages (the relaton#206
42
- contract), and `Relaton::Omg::INDEXFILE = "index-v2"` names it — the gem-side
43
- consumer flip (reading the machine index in `Bibliography` the way the other
44
- flavors do) is the follow-up; retrieval still scrapes live.
45
-
46
- The grammar is `OMG <ACRONYM>[ <VERSION>][<sep><PART>]`, where `<sep>` is a
47
- space, or a slash after a version. The version can carry a beta label (`2.5 beta`,
48
- `2.0 beta 1`). The part is a volume or a format name (`Superstructure`, `PDF`).
49
-
50
- - **`get` takes a String or a `Pubid::Omg::Identifier`** (the parse that
51
- `Relaton::Db` routed with, relaton#205). `Scraper.scrape_page` parses only a
52
- String, and the log keys print `ref.to_s`.
53
- - **An unrecognized reference raises.** `Scraper.scrape_page` lets
54
- `Pubid::Errors::ParseError` propagate, so
55
- `OMG Model Driven Architecture Guide rev. 2.0` raises and does not show as
56
- "Not found". Only the transport errors are rescued, by name.
57
- - **The request URL is the acronym and the version, and nothing else.**
58
- `get_doc` builds `https://www.omg.org/spec/<ACRONYM>/<VERSION>`, with each
59
- space of the version turned into `/` (`1.0 beta 2` becomes `1.0/beta/2`). The
60
- part stays out of the URL: `fetch_link` uses it to find the part's PDF link
61
- on the version page, and `fetch_title` appends it. The cassettes replay only
62
- while the URL does not change.
63
- - **The docid version comes from the page, not from the query.** That is why
64
- `get "OMG AMI4CCM"` answers `OMG AMI4CCM 1.1`. `fetch_docid` and the History
65
- ids of `fetch_relation` are rendered by
66
- `Pubid::Omg::Identifiers::Specification`, not by parsing a joined string.
67
- - **pubid renders one separator.** `OMG DDS 1.4/PDF` renders back as
68
- `OMG DDS 1.4 PDF`. Both spellings name the same document, and one rendering
69
- keeps `==` and `matches?` true between them. If you echo a caller's
70
- reference, echo the string, not `pubid.to_s`.
71
- - **Known limit.** OMG's URL spelling `Beta2` (label and number joined) parses
72
- as a part, not a version. It round-trips, so nothing is lost.
73
- - **The acronym is the URL segment, verbatim.** 30 of the 270 acronyms in the
74
- OMG catalog carry a hyphen, a slash, a plus, or start in lower case
75
- (`DDS-XTypes`, `EDMC-FIBO/BE`, `VSIPL++`, `smartant`). pubid PR #376 accepts
76
- all 270. A slash **before** the version belongs to the acronym
77
- (`OMG EDMC-FIBO/BE 1.1` fetches `spec/EDMC-FIBO/BE/1.1`); a slash **after**
78
- the version separates the part. So `OMG UML/Superstructure` is the acronym
79
- `UML/Superstructure`; write the part behind a space. With pubid `ce2f75ef`
80
- (PR #372) those 30 raised, and the migration waited for #376. Fix a grammar
81
- gap in pubid; do not split the acronym in the flavor.
82
-
83
- `Docidentifier` parses its `content` on the data side and rescues the parse, so
84
- a value that is not an OMG identifier stays verbatim with a nil `pubid`. Its
85
- mutators:
86
-
87
- - `remove_date!` drops the **version**. OMG identifiers carry no date, and the
88
- version is the discriminator, as IALA maps `remove_date!` onto its edition.
89
- So `to_most_recent_reference` turns `OMG AMI4CCM 1.0` into `OMG AMI4CCM`.
90
- - `remove_part!` drops the part: `OMG UML 2.1.1 Superstructure` becomes
91
- `OMG UML 2.1.1`.
92
- - `to_all_parts!` is the inherited no-op. An OMG part is a name, not a numbered
93
- part, so there is no "all parts" form.
94
-
95
- The processor sets `@pubid_flavor = :Omg`. That changes nothing today:
96
- `Pubid::Omg.prefixes` is `["OMG"]`, the same as `@prefix`.
97
-
98
- ### Publication date comes from JSON-LD, not the visible text
99
-
100
- `Scraper#pub_date` reads the `publicationDate` value out of the page's
101
- `application/ld+json` block, not the `<dt>Publication Date:</dt>` text. Two
102
- reasons, both verified against cassettes:
103
-
104
- - **Locale.** The OMG server renders the month name in its own locale, and
105
- Cloudflare caches that variant. A response can carry `Content-Language: zh-CN`
106
- with an otherwise English page (`<html lang="en">`) and a `<dd>七月 2007</dd>`,
107
- which made `Date.parse` raise `Date::Error`. The request already sends
108
- `Accept-Language: en-us,en;q=0.5`; the CDN answers `Vary: accept-encoding`
109
- only, so it does not vary on language and no request header can prevent this.
110
- - **Precision.** The visible text is a lossy render of the same value:
111
- `2013-03-31` prints as "March 2013". Parsing the text rounded every OMG date
112
- down to day 01.
113
-
114
- The `<dd>` text stays as a fallback for a page with no JSON-LD, and a month name
115
- it cannot parse logs a warning instead of raising.
116
- `spec/omg/relaton/omg/scraper_spec.rb` guards all three paths with
117
- `spec/omg/fixtures/localized_date.html`. Do not "simplify" the parser back to
118
- the `<dd>` text.
119
-
120
- ### Serialization formats
121
-
122
- Items can be serialized to/from YAML and XML. Tests verify round-trip fidelity for all three classes. XML output is validated against RELAX NG schemas in `grammars/`.
123
-
124
- ### Test patterns
125
-
126
- - RSpec with `expect` syntax (monkey patching disabled)
127
- - `canon` for XML comparison
128
- - `ruby-jing` for RELAX NG schema validation against `grammars/relaton-omg-compile.rng`
129
- - VCR cassettes in `spec/vcr_cassettes/` for HTTP interaction recording
130
- - Fixtures in `spec/fixtures/` (YAML and XML reference files)
131
- - Tests follow a round-trip pattern: load fixture → parse → serialize → compare to fixture
132
-
133
- ## Style
134
-
135
- - Ruby >= 3.1.0
136
- - Rubocop inherits from Ribose OSS guide with `rubocop-rails` required but Rails cops disabled
137
- - OMG document reference format: `OMG {ACRONYM}[ {VERSION}][ {PART}]` (e.g., `OMG AMI4CCM 1.0`, `OMG UML 2.1.1 Superstructure`)
@@ -1,126 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-plateau is a Ruby gem for retrieving and managing bibliographic metadata for Project PLATEAU (Japanese 3D city model standards published by MLIT). It is part of the Relaton ecosystem of bibliographic gems.
8
-
9
- ## Commands
10
-
11
- ```bash
12
- # Install dependencies
13
- bundle install
14
-
15
- # Run all tests
16
- bundle exec rake spec
17
-
18
- # Run a single test file
19
- bundle exec rspec spec/relaton/plateau/bibitem_spec.rb
20
-
21
- # Run a specific test by line number
22
- bundle exec rspec spec/relaton/plateau/bibitem_spec.rb:15
23
-
24
- # Lint
25
- bundle exec rake rubocop
26
-
27
- # Default rake task (runs specs)
28
- bundle exec rake
29
- ```
30
-
31
- ## Architecture
32
-
33
- All code lives in `lib/relaton/plateau/`. The gem uses LutaML::Model::Serializable for data modeling with automatic XML/YAML serialization.
34
-
35
- ### Models
36
- - **Item** (`item.rb`) — base Plateau item extending `Bib::Item`, declares `model ItemData` and adds `ext` attribute
37
- - **ItemData** (`item_data.rb`) — data class extending `Bib::ItemData`, returned by `Item.from_xml`, `Item.from_yaml`, and `Bibliography.get`
38
- - **Bibitem** (`bibitem.rb`) — bibliographic item variant (includes `Bib::BibitemShared`)
39
- - **Bibdata** (`bibdata.rb`) — bibliographic data variant (includes `Bib::BibdataShared`)
40
- - **Ext** (`ext.rb`) — extension element: doctype, subdoctype, flavor, editorialgroup, stagename, filesize, etc.
41
- - **Doctype** (`doctype.rb`) — extends `Bib::Doctype`, valid values: `handbook`, `technical-report`, `annex`
42
-
43
- ### Retrieval & Data Fetching
44
- - **Bibliography** (`bibliography.rb`) — module with `get(code)` for index-based document retrieval, returns `ItemData`
45
- - **HitCollection** / **Hit** — search the relaton-data-plateau index and fetch YAML documents
46
- - **DataFetcher** (`data_fetcher.rb`) — extends `Core::DataFetcher`, scrapes MLIT JSON APIs for handbooks and technical reports
47
- - **Parser** / **HandbookParser** / **TechnicalReportParser** — parse JSON into `ItemData` objects
48
- - **Processor** (`processor.rb`) — standard Relaton processor plugin (`get`, `from_xml`, `from_yaml`, `fetch_data`, `grammar_hash`)
49
-
50
- ### Serialization Methods
51
- - `bib.to_xml` — XML (bibitem format)
52
- - `Item.to_yaml(bib)` — YAML (class method)
53
- - `bib.to_rfcxml` — BibXML/RFC XML
54
- - `Item.from_xml(xml)` — parse XML into `ItemData`
55
- - `Item.from_yaml(yaml)` — parse YAML into `ItemData`
56
-
57
- ### Data Sources
58
- - Handbooks: `https://www.mlit.go.jp/plateau/_next/data/1.3.0/libraries/handbooks.json`
59
- - Technical Reports: `https://www.mlit.go.jp/plateau/_next/data/1.3.0/libraries/technical-reports.json`
60
- - Pre-fetched index: `https://raw.githubusercontent.com/relaton/relaton-data-plateau/v2/` (`HitCollection::ENDPOINT`)
61
-
62
- ### Pubid-backed index (index-v2)
63
-
64
- The index is **pubid-structured** (`_type: pubid:plateau:*` rows), mirroring
65
- NIST/CIE. The producer and the consumer `Relaton::Index.find_or_create` call
66
- sites (`data_fetcher.rb`, `hit_collection.rb`) pass
67
- `pubid_class: ::Pubid::Plateau::Identifier`, and `INDEXFILE = "index-v2"`.
68
- `processor.rb#remove_index_file` passes `url: true` and `file:` only: the delete
69
- never reads the index (see `lib/relaton/index/CLAUDE.md`).
70
-
71
- **Ids are pubid-canonical.** The parsers emit the form `Pubid::Plateau.parse`
72
- accepts, which also matches the MLIT source:
73
- - Handbook: `PLATEAU Handbook #NN 第X.Y版` (Japanese edition label — the parser
74
- keeps the source `第X.Y版`; the Latinized `#edition` is retained only for the
75
- structured `Bib::Edition`/`ext` fields).
76
- - Technical Report: `PLATEAU Technical Report #NN` (**no edition** — pubid TRs
77
- carry none; the document still has a `1.0` `Bib::Edition`).
78
- - Sub-numbers use a hyphen (`#46-1`), not the source underscore (`#46_1`).
79
-
80
- `DataFetcher#save_document` parses the canonical id via `#pubid` (a round-trip
81
- guard: `from_hash(to_hash) == to_hash`) and stores the `Pubid::Plateau::Identifier`
82
- object; unparseable ids are warned and skipped (defensive — the parser already
83
- emits canonical, so nothing is expected to skip).
84
-
85
- **Search accepts both forms.** `HitCollection#find` parses the query via `#ref`
86
- (`Pubid::Plateau.parse`, memoized; an unparseable query **raises**
87
- `Pubid::Errors::ParseError`; `Bibliography.get` rescues transport errors only,
88
- as `Relaton::RequestError`, and `Hit#item` raises that same error on a
89
- non-200) and matches on **pubid objects**, mirroring the
90
- other pubid flavors: an exact edition uses `index.search(ref, exact: true)`
91
- (pubid `==` compares type + number + annex + edition); a family query
92
- (`ref.edition.nil?` — an edition-less handbook, or any Technical Report) uses
93
- the default subset match `ref === row`, where the omitted edition matches any
94
- value. pubid declares `annex` strict for PLATEAU (pubid#408), so
95
- `PLATEAU Handbook #03` does not reach the annex `#03-1`;
96
- `hit_collection_spec.rb` pins that against the index fixture. This keeps the
97
- family-vs-exact decision and the match on the *same* parsed identifier (no raw
98
- `@ref` regex). Canonical (`第1.0版`), edition-less (`#00`), and **legacy Latin**
99
- (`#00 1.0`) references all resolve — `Pubid::Plateau` normalizes Latin input to
100
- the canonical id (metanorma/pubid #269; needs the pubid `main` pin). No
101
- relaton-side Latin shim exists by design.
102
-
103
- `EDITION_SUFFIX` is **not** used for matching; it lives only in `#to_all_editions`,
104
- which strips the edition off the fetched document's `Bib::Docidentifier`/docnumber
105
- **strings** (relaton-bib content, not pubid) to build the family record — it
106
- matches both Japanese and Latin suffixes so it works against the current (Latin)
107
- VCR-cassette documents and future canonical ones.
108
-
109
- ### Schema Validation
110
- RNG grammar files in `grammars/` define the XML schema. Tests validate fixtures against `relaton-plateau-compile.rng` using the `ruby-jing` gem.
111
-
112
- ## Testing
113
-
114
- - **Index fixture:** `spec/fixtures/index-v2.zip` (pubid-backed, `_type: pubid:plateau:*` rows) is pre-loaded into the `Relaton::Index` pool in `before(:suite)` via a `Relaton::Index::Type` built with `::Pubid::Plateau::Identifier` (configured in `spec/support/webmock.rb`, mirroring NIST/CIE). Run `rake spec:update_index` to refresh from relaton-data-plateau.
115
- - RSpec with `expect` syntax (no monkey-patching)
116
- - VCR cassettes record HTTP interactions (in `spec/fixtures/vcr_cassettes/`)
117
- - `canon` for XML comparison assertions
118
- - `jing` matcher for RNG schema validation
119
- - SimpleCov for code coverage
120
-
121
- ## Key Conventions
122
-
123
- - Document identifiers are pubid-canonical: `PLATEAU Handbook #XX 第Y.Z版` (Japanese edition) or `PLATEAU Technical Report #XX` (no edition); sub-numbers use a hyphen (`#XX-N`). See "Pubid-backed index" above.
124
- - The gem supports multi-format serialization: XML, YAML, BibXML (RFC XML)
125
- - RuboCop follows Ribose OSS style guide; target Ruby version is 3.0
126
- - The `ext` element in XML/YAML carries PLATEAU-specific metadata (doctype, flavor, editorialgroup, etc.)
@@ -1,91 +0,0 @@
1
- # CLAUDE.md — SAE flavor
2
-
3
- `Relaton::Sae` fetches SAE International standards (relaton/relaton-sae#1).
4
-
5
- ## Development
6
-
7
- - `bundle exec rake spec:sae` — this flavor's suite (from the repo root).
8
- - No VCR: the specs stub HTTP with WebMock. `spec/sae/fixtures/crossref/`
9
- holds three Crossref cursor pages built from real records (11 works:
10
- six J3016/J3194 editions, three AMS4121 revisions, a part and a
11
- letter-pair number). `spec/sae/fixtures/index-v2.yaml` and `fixtures/data/` are the
12
- **output of `DataFetcher` over those pages**: regenerate both together by
13
- running the crawler over the page fixtures, never by hand.
14
-
15
- ## Where the data comes from
16
-
17
- **sae.org answers every script request with a CAPTCHA** (product pages and
18
- search alike), so no scraper is possible. SAE deposits every standard edition
19
- with Crossref instead: prefix `10.4271`, type `standard`, ~47,700 records. The
20
- DOI suffix is SAE's own edition code in lowercase:
21
-
22
- | DOI | SAE code | scheme |
23
- |---|---|---|
24
- | `10.4271/j3016_202104` | `SAE J3016_202104` | J-series: `_YYYYMM` edition date |
25
- | `10.4271/ams4121j` | `SAE AMS4121J` | aerospace: revision letter, no date |
26
- | `10.4271/j2337/2_201702` | `SAE J2337/2_201702` | a part |
27
-
28
- Crossref's text search does **not** find a standard by its id (it matches
29
- titles), so an undated reference cannot be answered live. The flavor is
30
- therefore index-based, the ECMA/IALA shape:
31
-
32
- - **Producer** — `DataFetcher` (`relaton fetch-data sae-crossref`) walks
33
- `api.crossref.org/prefixes/10.4271/works?filter=type:standard` with the
34
- deep-paging cursor (1000 a request, one request a second through
35
- `Core::Pacer`; 429/5xx and transport errors retried with back-off, five
36
- attempts, then `RequestError`, because
37
- a partial crawl would publish a partial index). Each work goes through
38
- `Relaton::Doi::Parser`, then gets SAE's code as its **primary**
39
- docidentifier (type `SAE`), the DOI beside it, and `ext` doctype
40
- `standard`, flavor `sae`. `fetched` is cleared: a data file carries no
41
- fetch date, or every daily crawl would rewrite and commit all ~47,700
42
- files. `Bibliography#fetch_doc` sets it when it reads a document.
43
- - **Consumer** — `Bibliography.get` reads the `relaton-data-sae` Pages index
44
- (`PAGES_URL`) and the document from `ENDPOINT` (raw GitHub).
45
-
46
- ## Three identifier forms
47
-
48
- `DoiCode` gives each record three strings:
49
-
50
- - `#native` — the code in the DOI's spelling, `SAE J3016_202104`,
51
- `SAE AMSQQA250/4B`. The output file is named after it
52
- (`data/sae-amsqqa250-4b.yaml`), so a file name never depends on pubid.
53
- - `#printed` — the code as SAE prints it (`SAE AMS-QQ-A-250/4B`): pubid's
54
- render of the native code, or the native code when pubid cannot read it.
55
- It is the primary docidentifier.
56
- - `#reference` — what the index row is parsed from: the native code when the
57
- DOI has a date, else the code with the year of the record's `approved` date
58
- (`SAE AMS4121J:2017`), so a dated reference also reaches an aerospace
59
- revision.
60
-
61
- `Pubid::Sae` (pubid >= 2.0.9, pubid#665) reads all three, the month
62
- included, so the `Db` keys the query and the fetched document with the
63
- default `cache_pubid`. A reference pubid cannot read (`SAE J3016_2021`,
64
- `SAE CPGM2016`) answers nil with a log line (relaton#235); it does not raise.
65
-
66
- ## Selection
67
-
68
- `index.search(pubid)` with no block, so the subset match `query === row`:
69
- a revision, a year or a month the query omits matches any value; the type
70
- (`J`, `AMS`, …) is compared, and a part and an amendment are `subset_strict`
71
- (`SAE J2337` does not answer with `SAE J2337/2`). Of the matches the highest
72
- revision wins (`[length, letters]`, so `AA` follows `Z`), then the latest
73
- year and month, then the file name (the index sort is not stable). A year
74
- argument goes on a **copy** of a pubid handed in by `Db`.
75
-
76
- The index key carries the month, so two editions of one year are two rows.
77
- Two DOIs that spell one code (`ams-4121j`, `ams4121j`) share a key, and
78
- `DataFetcher#add_to_index` keeps the later one (DOI date, then approval
79
- date).
80
-
81
- ## What pubid still cannot read
82
-
83
- About 960 of the ~47,700 deposits: 791 `CP*` collected-paper products (not
84
- standards) and ~170 one-off shapes (`SUP1` supplements, language tails such
85
- as `_PTBR`, `_4Q2012`, `_MMYYYY` dates). See pubid's `lib/pubid/sae/CLAUDE.md`.
86
- Such a record is written, reported in `@errors` (the crawl's GitHub issue),
87
- and not indexed: one row that fails to deserialize makes `Relaton::Index`
88
- reject the whole index.
89
-
90
- A multi-letter tail stays in `number` (`J1939DA`, and also `AMS4121AA`), so
91
- a two-letter revision is its own bucket: `SAE AMS4121` does not reach it.
@@ -1,81 +0,0 @@
1
- # CLAUDE.md
2
-
3
- Dev notes for `Relaton::Sdo` — the SDO organization & logo store.
4
-
5
- ## What this is
6
-
7
- A **non-flavor** component that answers metanorma#346 (central logo store) and
8
- its sister relaton-db#132 (map abbreviation → organisation name/translation).
9
- It is **not** a bibliographic flavor: it has **no processor**, **no
10
- `Db::Registry` entry**, and is not reachable from `Db#fetch`. It is a standalone
11
- top-level store, exposed through `Relaton.organization`.
12
-
13
- This deliberately honors the scope decision in `lib/relaton/db/CLAUDE.md`
14
- ("org metadata … kept out of the processors"): nothing here touches a processor.
15
- The user's choice was to still surface it from the `relaton` gem itself (one
16
- `require`, one entry point) as a **lazily-autoloaded** component — referencing
17
- `Relaton::Sdo` or calling `Relaton.organization` is what loads it.
18
-
19
- ## Public API
20
-
21
- ```ruby
22
- org = Relaton.organization("ISO") # case-insensitive; nil if unknown
23
- org.name # default (untranslated) name
24
- org.name("fr") # translation, or nil
25
- org.name(language: "fr") # same, via the language: keyword
26
- org.logo_query(format:, size:, style:) # all args optional → matching Logo list
27
- org.logo(format:, size:, style:) # one Logo: nil if 0, the match if 1,
28
- # raises Sdo::Error if the filter is ambiguous
29
- logo.content # raw binary (lazy HTTP GET of logo.url, memoized)
30
- logo.data_uri # "data:<mime>;base64,…"
31
- logo.save # write to a filename derived from the URL; returns path
32
- logo.save("/path.eps") # write to a custom path
33
- ```
34
-
35
- ## Data model & retrieval flow
36
-
37
- `Store` (singleton) lazily builds an index from a single **`index.yaml`**
38
- manifest served by the `relaton-data-sdo` data repo, and memoizes it (`reset!`
39
- drops the memo — used by specs). Shape:
40
-
41
- ```yaml
42
- organizations:
43
- ISO:
44
- name:
45
- - content: International Organization for Standardization # default
46
- - { language: fr, content: Organisation internationale de normalisation }
47
- logo:
48
- - { style: default, format: eps, size: 700x300, url: "https://…/iso.eps" }
49
- - { style: red, format: svg, url: "https://…/red.svg",
50
- applicability: "stage>=60" } # opaque text; the store never interprets it
51
- ```
52
-
53
- - **`style`** is the primary discriminator for multiple logos per org (e.g.
54
- `default`, `red`, `grey`, `iso_1972`, `white_paper_2022`, BIPM `si_aspect_full`).
55
- - **`applicability`** (year/stage/doctype/cover-color rules) is carried through
56
- verbatim and exposed; **selection is the consumer's job** (metanorma), not the
57
- store's.
58
-
59
- `Config` (`Relaton::Sdo.configuration`) holds the index `url`, the cache
60
- `storage_dir` (default `~/.relaton/sdo`), the `storage` backend (defaults to
61
- `Relaton::Index::FileStorage`; swappable, e.g. S3), and the `ttl` (24 h).
62
- `Fetcher.fetch_index` serves a fresh on-disk copy within the TTL, re-downloads
63
- otherwise, and falls back to a stale cache if the network fails. A URL with no
64
- scheme (or `file://`) is read straight off disk — handy for fixtures/specs. Logo
65
- binaries are fetched on demand and memoized on the `Logo` (no disk cache yet).
66
-
67
- ## Tests
68
-
69
- `spec/sdo/` is a self-contained suite (auto-discovered by `rake spec`). It uses
70
- WebMock to stub the index and logo URLs, points `storage_dir` at a fresh
71
- `Dir.mktmpdir` per example, and resets the store between examples. It
72
- `require "relaton"` because the public entry point (`Relaton.organization`)
73
- lives in the root entry file, which the shared spec_helper does not load.
74
-
75
- ## Scope / not yet done
76
-
77
- - The real `relaton-data-sdo` data repo (assets + build GHA) — see
78
- `HANDOFFS/relaton__relaton-data-sdo.md`. Until it publishes, `Config#url`
79
- points at its (future) raw `index.yaml`.
80
- - No structured interpretation of `applicability`; no on-disk cache of logo
81
- binaries; no writeback/authoring.
@@ -1,91 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-un is a Ruby gem that provides bibliographic data models for UN documents, part of the Relaton family of gems. It extends `relaton-bib` with UN-specific metadata using the LutaML-Model serialization framework. Currently on the `lutaml-integration` branch undergoing a major migration from the old `RelatonUn` namespace to `Relaton::Un`.
8
-
9
- ## Commands
10
-
11
- ```bash
12
- # Install dependencies
13
- bundle install
14
-
15
- # Run all tests (also the default rake task)
16
- bundle exec rake spec
17
-
18
- # Run a specific test file
19
- bundle exec rspec spec/relaton/un/bibitem_spec.rb
20
-
21
- # Run a specific test by line number
22
- bundle exec rspec spec/relaton/un/bibitem_spec.rb:7
23
-
24
- # Lint
25
- bundle exec rubocop
26
-
27
- # Interactive console with gem loaded
28
- bin/console
29
- ```
30
-
31
- ## Architecture
32
-
33
- ### Flavor Model Pattern
34
-
35
- The gem follows the Relaton flavor pattern: base classes from `relaton-bib` (`Bib::Item`) are extended with a flavor-specific `Ext` class containing UN-specific metadata.
36
-
37
- ```
38
- Bib::ItemData (from relaton-bib)
39
- └── Relaton::Un::ItemData # UN flavor identity, enables NamespaceHelper#namespace → Relaton::Un
40
-
41
- Bib::Item (from relaton-bib)
42
- └── Relaton::Un::Item # model ItemData; adds Ext attribute
43
- ├── Relaton::Un::Bibitem # includes Bib::BibitemShared (for <bibitem> XML)
44
- └── Relaton::Un::Bibdata # includes Bib::BibdataShared (for <bibdata> XML)
45
- ```
46
-
47
- ### Lookup
48
-
49
- `Bibliography.get` takes a String (`UN TRADE/CEFACT/2004/32`, the `UN ` token
50
- optional) or a parsed `Pubid::Un::Identifier`, which renders the document
51
- symbol without the token (`TRADE/CEFACT/2004/32`) — the form the UN data
52
- itself writes. Either becomes the symbol the search matches (`document_symbol`).
53
- `Relaton::Db` routes UN by the `UN ` prefix (`Registry#route` falls back to
54
- `class_by_ref`): `Pubid::Un` renders the symbol without the token, so a
55
- `UN …` reference is not an exact parse. A symbol without the token
56
- (`TRADE/CEFACT/2004/32`) is not routed at all — no flavor claims it — and
57
- raises `Relaton::UnknownReferenceError`. `Pubid::Un` also parses a bare DOI
58
- exactly (`10.6028/NIST.IR.8245`), but UN does not claim it (`10.` is a DOI
59
- prefix), so it does not route to UN either.
60
-
61
- ### Key Classes (all in `lib/relaton/un/`)
62
-
63
- - **ItemData** — extends `Bib::ItemData`; the UN-specific data container used by `Item` via `model ItemData`
64
- - **Item** — extends `Bib::Item`, adds `ext` attribute of type `Ext`
65
- - **Bibitem / Bibdata** — extend Item with shared mixins for two XML serialization formats
66
- - **Ext** — UN-specific metadata: `doctype`, `distribution`, `session`, `submissionlanguage`, `job_number`; extends `Bib::Ext`
67
- - **Session** — session metadata (number, date, item numbers/names, collaborator, agenda_id)
68
- - **Doctype** — restricts document types to: recommendation, plenary, addendum, communication, corrigendum, reissue, agenda, budgetary, sec-gen-notes, expert-report, resolution
69
-
70
- ### LutaML-Model Serialization
71
-
72
- All model classes use `Lutaml::Model::Serializable`. XML mappings are defined in `xml do` blocks:
73
- - `map_attribute` for XML attributes (e.g., `schema-version` → `schema_version`)
74
- - `map_element` for XML elements (e.g., `session-date` → `session_date`)
75
- - `collection: true` for array attributes
76
- - `values: [...]` for restricted string values
77
-
78
- ### Schema Validation
79
-
80
- RelaxNG grammars in `grammars/` validate XML output. The compiled schema is `grammars/relaton-un-compile.rng`. Tests use `ruby-jing` for validation.
81
-
82
- ### Testing Approach
83
-
84
- Tests are round-trip serialization tests: load a fixture, serialize to XML/YAML, compare against the original fixture. Bibitem and Bibdata specs also validate XML against the RelaxNG schema. Fixtures live in `spec/fixtures/`.
85
-
86
- ## Dependencies
87
-
88
- - **relaton-bib** (~> 2.0.0-alpha.1) — base bibliographic model with LutaML-Model integration
89
- - **faraday** — HTTP client
90
- - **ruby-jing** (dev) — RelaxNG schema validation in tests
91
- - **canon** (dev) — XML comparison in tests