relaton 3.1.0 → 3.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. checksums.yaml +4 -4
  2. data/README.adoc +5 -3
  3. data/lib/relaton/3gpp.rb +1 -1
  4. data/lib/relaton/bib/model/docidentifier.rb +66 -5
  5. data/lib/relaton/calconnect/hit_collection.rb +1 -1
  6. data/lib/relaton/cli/command.rb +6 -4
  7. data/lib/relaton/cli/frontend_assets.rb +7 -6
  8. data/lib/relaton/db.rb +31 -14
  9. data/lib/relaton/ecma/bibliography.rb +2 -2
  10. data/lib/relaton/etsi/data_fetcher.rb +1 -1
  11. data/lib/relaton/iala/bibliography.rb +1 -1
  12. data/lib/relaton/iana.rb +1 -1
  13. data/lib/relaton/iec/bibliography.rb +32 -20
  14. data/lib/relaton/iec/data_parser.rb +41 -2
  15. data/lib/relaton/itu/README.adoc +1 -1
  16. data/lib/relaton/itu/data_crawler_r.rb +1 -1
  17. data/lib/relaton/oasis/bibliography.rb +1 -1
  18. data/lib/relaton/ogc/hit_collection.rb +1 -1
  19. data/lib/relaton/ogc.rb +1 -1
  20. data/lib/relaton/plateau/handbook_parser.rb +31 -2
  21. data/lib/relaton/plateau/hit_collection.rb +1 -1
  22. data/lib/relaton/render.rb +17 -0
  23. data/lib/relaton/sae/data_fetcher.rb +5 -0
  24. data/lib/relaton/sae.rb +1 -1
  25. data/lib/relaton/version.rb +1 -1
  26. data/lib/relaton/w3c/bibliography.rb +1 -1
  27. data/lib/relaton/xsf/bibliography.rb +1 -1
  28. data/lib/relaton/xsf.rb +1 -1
  29. data/lib/relaton-render.rb +4 -10
  30. metadata +33 -46
  31. data/lib/relaton/3gpp/CLAUDE.md +0 -218
  32. data/lib/relaton/adobe/CLAUDE.md +0 -104
  33. data/lib/relaton/bib/CLAUDE.md +0 -150
  34. data/lib/relaton/bipm/CLAUDE.md +0 -187
  35. data/lib/relaton/bsi/CLAUDE.md +0 -35
  36. data/lib/relaton/calconnect/CLAUDE.md +0 -177
  37. data/lib/relaton/ccsds/CLAUDE.md +0 -74
  38. data/lib/relaton/cen/CLAUDE.md +0 -219
  39. data/lib/relaton/cie/CLAUDE.md +0 -90
  40. data/lib/relaton/core/CLAUDE.md +0 -140
  41. data/lib/relaton/db/CLAUDE.md +0 -385
  42. data/lib/relaton/doi/CLAUDE.md +0 -105
  43. data/lib/relaton/easc/CLAUDE.md +0 -85
  44. data/lib/relaton/ecma/CLAUDE.md +0 -217
  45. data/lib/relaton/etsi/CLAUDE.md +0 -219
  46. data/lib/relaton/evs/CLAUDE.md +0 -120
  47. data/lib/relaton/gb/CLAUDE.md +0 -47
  48. data/lib/relaton/gost/CLAUDE.md +0 -100
  49. data/lib/relaton/iala/CLAUDE.md +0 -97
  50. data/lib/relaton/iana/CLAUDE.md +0 -105
  51. data/lib/relaton/iec/CLAUDE.md +0 -13
  52. data/lib/relaton/ieee/CLAUDE.md +0 -202
  53. data/lib/relaton/ietf/CLAUDE.md +0 -382
  54. data/lib/relaton/iho/CLAUDE.md +0 -76
  55. data/lib/relaton/index/CLAUDE.md +0 -208
  56. data/lib/relaton/isbn/CLAUDE.md +0 -47
  57. data/lib/relaton/iso/CLAUDE.md +0 -38
  58. data/lib/relaton/itu/CLAUDE.md +0 -685
  59. data/lib/relaton/jcgm/CLAUDE.md +0 -107
  60. data/lib/relaton/jis/CLAUDE.md +0 -64
  61. data/lib/relaton/logger/CLAUDE.md +0 -37
  62. data/lib/relaton/nist/CLAUDE.md +0 -194
  63. data/lib/relaton/oasis/CLAUDE.md +0 -251
  64. data/lib/relaton/ogc/CLAUDE.md +0 -242
  65. data/lib/relaton/oiml/CLAUDE.md +0 -47
  66. data/lib/relaton/omg/CLAUDE.md +0 -137
  67. data/lib/relaton/plateau/CLAUDE.md +0 -126
  68. data/lib/relaton/sae/CLAUDE.md +0 -89
  69. data/lib/relaton/sdo/CLAUDE.md +0 -81
  70. data/lib/relaton/un/CLAUDE.md +0 -91
  71. data/lib/relaton/w3c/CLAUDE.md +0 -369
  72. data/lib/relaton/xsf/CLAUDE.md +0 -189
@@ -1,251 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-oasis is a Ruby gem for retrieving and serializing OASIS Standards bibliographic metadata. It is part of the Relaton family of gems and implements the IsoBibliographicItem model for OASIS standards. Uses the lutaml-model pattern on the `lutaml-integration` branch.
8
-
9
- ## Commands
10
-
11
- ```bash
12
- bundle exec rake spec # Run all tests (default rake task)
13
- bundle exec rake rubocop # Run linter
14
- bundle exec rspec # Run tests directly
15
- bundle exec rspec spec/relaton/oasis/bibitem_spec.rb # Run a single test file
16
- bin/console # Interactive IRB with gem loaded
17
- ```
18
-
19
- ## Architecture
20
-
21
- ### Class Hierarchy (under `Relaton::Oasis`)
22
-
23
- All model classes use the lutaml-model pattern from `relaton-bib`:
24
-
25
- - **ItemData** (`lib/relaton/oasis/item_data.rb`) — Data model class extending `Bib::ItemData`
26
- - **Item** (`lib/relaton/oasis/item.rb`) — Base class extending `Bib::Item`, uses `ItemData` model, adds `ext` attribute of type `Ext`
27
- - **Bibitem** (`lib/relaton/oasis/bibitem.rb`) — Extends `Item`, includes `Bib::BibitemShared` for individual bibliography entries
28
- - **Bibdata** (`lib/relaton/oasis/bibdata.rb`) — Extends `Item`, includes `Bib::BibdataShared` for complete bibliography records
29
- - **Ext** (`lib/relaton/oasis/ext.rb`) — Extends `Bib::Ext`, OASIS-specific metadata: `doctype`, `technology_area`, `schema_version`
30
- - **Doctype** (`lib/relaton/oasis/doctype.rb`) — Extends `Bib::Doctype`, valid values: `specification`, `memorandum`, `resolution`, `standard`
31
- - **Docidentifier** (`lib/relaton/oasis/docidentifier.rb`) — Extends
32
- `Bib::Docidentifier`, parses its `content` into a `Pubid::Oasis::Identifier`
33
- exposed as `#pubid`; see **Identifiers and the index**
34
-
35
- ### Bibliography (under `Relaton::Oasis`)
36
-
37
- - **Bibliography** (`lib/relaton/oasis/bibliography.rb`) — Module with `search`/`get` class methods for fetching OASIS standards from the relaton-data-oasis GitHub repository. Delegates to private methods: `find_index_entry`, `fetch_yaml`, `parse_item`
38
-
39
- ### Data Fetching (under `Relaton::Oasis`)
40
-
41
- These classes scrape https://www.oasis-open.org/standards/ to produce YAML/XML data files. Loaded separately via `require "relaton/oasis/data_fetcher"`.
42
-
43
- - **DataFetcher** (`lib/relaton/oasis/data_fetcher.rb`) — Extends `Core::DataFetcher`, orchestrates fetching all OASIS standards
44
- - **DataParser** (`lib/relaton/oasis/data_parser.rb`) — Parses individual OASIS document nodes into `ItemData`
45
- - **DataPartParser** (`lib/relaton/oasis/data_part_parser.rb`) — Parses multi-part OASIS document nodes into `ItemData`
46
- - **DataParserUtils** (`lib/relaton/oasis/data_parser_utils.rb`) — Shared module included by both parsers; provides contributor parsing, docid construction, retry logic, and doctype detection
47
-
48
- ### Identifiers and the index
49
-
50
- #### Pubid-backed docidentifier
51
-
52
- `Relaton::Oasis::Docidentifier` parses its `content` into a
53
- `Pubid::Oasis::Identifier` kept in `@pubid`, while the lutaml `content`
54
- attribute stays a plain **string** for serialization. Parsing is **soft**:
55
- `content=` lazily requires pubid and rescues `LoadError`/`StandardError`, so a
56
- missing gem or non-OASIS content leaves `@pubid` nil rather than raising. The
57
- stored content carries the `OASIS ` publisher token, which is
58
- `Pubid::Oasis::Identifier#to_s`'s default rendering, so a bare slug
59
- (`"amqp-core"`) is *not* an OASIS printed id and leaves `@pubid` nil.
60
-
61
- The scrapers still compose the slug by hand (`DataParserUtils#parse_docid`,
62
- `#parse_spec`, `#parse_part`, `#parse_errata`, `DataParser#title_to_docid`) —
63
- pubid parses the result, it does not replace those heuristics.
64
-
65
- **`remove_part!` / `remove_date!` / `to_all_parts!` stay the inherited no-ops,
66
- deliberately.** `Pubid::Oasis::Renderer` echoes the verbatim slug held in
67
- `original`, so clearing `part`, `version` or `stage` cannot change the printed
68
- id — the mutation would be invisible — and rewriting `original` would invent a
69
- reference form OASIS does not publish. OASIS slugs are free-form with an
70
- inconsistent internal structure, and no OASIS citation drops a part or a
71
- version; there is no "all parts" or "most recent" spelling to render. Since
72
- `dfbd26c72` those defaults are no-ops rather than `NotImplementedError` raises,
73
- so `Bib::ItemData#to_all_parts` and `#to_most_recent_reference` return the item
74
- unchanged instead of blowing up. Do not "finish" this class by adding them.
75
- (Contrast `Relaton::Ogc::Docidentifier`, whose components really do render.)
76
-
77
- #### `index-v2`, written and read
78
-
79
- `INDEXFILE = "index-v2"` is the only index this flavor touches: pubid-keyed
80
- rows (`_type: pubid:oasis:standard`), published by `relaton-data-oasis` on its
81
- `v2` branch — the same branch `Bibliography::ENDPOINT` reads.
82
-
83
- The producer and the consumer pass `pubid_class: ::Pubid::Oasis::Identifier`:
84
-
85
- | Call site | Why it needs `pubid_class:` |
86
- |---|---|
87
- | `DataFetcher#index` | `FileIO#save` calls `to_hash` only for instances of it — without it the crawl writes v1-shaped rows under a v2 name, silently |
88
- | `Bibliography#index` | without it the rows stay raw hashes, `FileIO#sorted` is false, and every lookup scans all 605 rows |
89
-
90
- `Processor#remove_index_file` passes `url: true` and `file:`, and no
91
- `pubid_class:`: the delete never reads the index (see
92
- `lib/relaton/index/CLAUDE.md`). It used to omit `url: true`, so `Db#clear` on an
93
- empty pool deleted `./index-v2.yaml` in the working directory and kept the
94
- cache.
95
-
96
- `Bibliography#index` also passes `file:`, which the pre-migration code omitted.
97
- `file:` names the cache file (`Type` falls back to `index.yaml` without it), so
98
- the consumer and `remove_index_file` must pass the same one, or `Db#clear`
99
- leaves the consumer's cache in place.
100
-
101
- **This gem never writes `index-v1`.** `relaton-data-oasis`'s crawler derives it
102
- from the v2 rows, for relaton v2 consumers only.
103
-
104
- #### Resolving a reference
105
-
106
- `Bibliography#find_index_entry` is the pubid `best_match` shape:
107
-
108
- - **`#parse_ref` supplies the publisher token.** An OASIS printed id starts
109
- with `OASIS `, and pubid's grammar requires it, but callers write bare slugs
110
- (`mqtt-v5.0`) and `Db#fetch` passes the reference through verbatim. Adding
111
- the token is normalization, not identification, so it happens in the flavor —
112
- the same place W3C normalizes a URL or a leading `TR-`.
113
- - **The parsed pubid is passed to `Type#search`, never the string.** Search
114
- narrows only for a non-`String` argument, so passing text would disable the
115
- binary search however the index was built. `pubid_class:` alone fixes
116
- nothing; both had to change together.
117
- - **Selection is pubid's asymmetric subset match** (`Pubid::SubsetMatch`):
118
- `pubid === row[:id]`, the reference on the left. A component the reference
119
- omits (`version`, `stage`, `part`, `label`) matches any value; a component it
120
- states must be equal. `number` is always stated: it is the specification name,
121
- the whole identity of an OASIS record and the key the index bsearches on.
122
- `#ignored` survives only for `#ranking_key`, which counts the components the
123
- caller did not ask for. Measured over the whole fixture index (605 rows,
124
- 6,795 comparisons): the subset match and the old
125
- `matches?(…, ignore: #ignored)` agree on every pair.
126
- - **There is no substring fallback.** After normalization the only unparseable
127
- inputs are a blank string and one over pubid's 1000-character cap, and a
128
- substring scan for `""` matches every row — worse than a miss. An
129
- unparseable reference raises `Pubid::Errors::ParseError`
130
- (`spec/oasis/relaton/oasis/bibliography_spec.rb` pins it), so a malformed
131
- identifier stays distinct from an absent document.
132
-
133
- ##### Ordering: `#ranking_key`
134
-
135
- Ignoring a component means "don't care", so a loose reference also matches the
136
- more specific rows. The key, in order: an **exact printed id**; then the
137
- **newest version**, segments compared as integers; then the **least specific**
138
- row; then the **later stage revision** (digits only — OASIS stage letters have
139
- no ranking this code may invent); then the printed id, so the result never
140
- depends on index order.
141
-
142
- The exact-match rule is not decoration. Five records are a bare specification
143
- name that also has versioned siblings — `OASIS EDXL`, `OData`, `OSLC`, `SAML`,
144
- `WSS` — and without it three of them answered with their newest sibling instead
145
- of themselves. Published ids are unique (0 duplicates in 605 rows), so at most
146
- one row can score there.
147
-
148
- Verified over the whole published index: **all 605 ids resolve to their own
149
- record**, with and without the publisher token.
150
-
151
- ##### One behaviour is deliberately lost
152
-
153
- v1 matched by substring, so `OASIS amqp` resolved to some `amqp-core` record.
154
- v2 matches identifiers, so a partial specification name no longer resolves.
155
- That is the same semantic change IANA recorded, and
156
- `spec/oasis/relaton/oasis/bibliography_spec.rb` asserts it so it stays a
157
- decision rather than a surprise.
158
-
159
- #### Unparseable ids are reported, in two places on purpose
160
-
161
- `DataFetcher#add_to_index` indexes `docid.pubid` and, when it is nil, records
162
- the reason in `@errors` and skips the row rather than indexing it unparsed:
163
- `Relaton::Index` rejects the WHOLE index if a single row fails to deserialize,
164
- and its sort calls `.root.number` on every id. The data file is still written,
165
- so a document is unindexed, never lost.
166
-
167
- `DataParserUtils#record_unparseable_id` records the **same** thing at the moment
168
- the id is built. That is not redundant: the part ids that only ever become a
169
- relation's `formattedref` (`DataParser#parse_relation`,
170
- `DataPartParser#parse_relation`) never reach `save_doc`, so an index-time hook
171
- alone cannot see them. Both entries are keyed on the **id string**, so a
172
- document that is saved reports one line — the fetcher's, which also names the
173
- output file. Keep the two keys identical.
174
-
175
- Both routes rely on `Core::DataFetcher#report_errors` treating a **String**
176
- value in `@errors` as the message, logged verbatim at `:error`, which is the
177
- level the `GhIssue` channel subscribes to. So each entry becomes one line of the
178
- crawl's "Error fetching documents" GitHub issue, and the key serves only to
179
- de-duplicate. W3C and 3GPP use exactly this `@errors` route; ISO, ITU and NIST
180
- reach the same issue through a side list plus a `report_errors` override.
181
- OASIS is the only flavor that also records from its parser, for the
182
- `formattedref` reason above.
183
-
184
- #### The index key
185
-
186
- `Index::Type#candidates_by_number` bsearches on `id.root.number.to_s`, and
187
- `FileIO` sorts by that same key. `Pubid::Oasis::Identifier#number` holds the
188
- **specification name** — "OSLC-CoreShapes", "STIX", "amqp-core" — so every
189
- version, stage and part of one specification shares a bucket, the same shape as
190
- an IETF draft slug or an IANA registry slug.
191
-
192
- This needed a pubid change: the attribute was called `spec` and `number` was
193
- never set, so every row keyed on `""` — one bucket, bsearch degraded to a
194
- linear scan, silently, exactly the IANA and W3C trap. pubid `cfe8ee84` renames
195
- it, with **no alias**, and it ships in the released `2.0.0.alpha.10` that
196
- `relaton.gemspec` pins — so this flavor needs nothing from the `Gemfile`'s
197
- temporary pubid `main` pin, unlike the nine listed there.
198
- `spec/oasis/relaton/oasis/pubid_contract_spec.rb` is the standing guard: it
199
- fails against any pubid that does not set `number`.
200
-
201
- The flavor is indifferent to pubid's parse-failure class, which moved twice
202
- during this work — `RuntimeError`, then `Parslet::ParseFailed`, and in
203
- alpha.10 `Pubid::Errors::ParseError` (a `Parslet::ParseFailed`) with
204
- `Pubid::Errors::InvalidInputError` (an `ArgumentError`) for a non-String or an
205
- over-long input. `Docidentifier#content=` and `Bibliography#parse_ref` both
206
- rescue `StandardError`, which every one of those descends from.
207
-
208
- Measured over all 605 published rows, written through
209
- `Relaton::Oasis::Docidentifier` into a `pubid_class:`-configured index and read
210
- back: 605 rows round-trip byte-identically, 0 unparseable, 0 keying on `""`,
211
- 309 buckets with the largest 25, and a `STIX` lookup narrows to 25 of 605.
212
-
213
- ### Serialization
214
-
215
- Supports XML (with RELAX NG schema validation via `grammars/`) and YAML round-trip serialization. Classes use `from_xml`/`to_xml` and `from_yaml`/`to_yaml` class methods inherited from the lutaml-model base.
216
-
217
- ### Test Conventions
218
-
219
- Tests are round-trip based: parse a fixture file, serialize back, and compare output to input. XML tests also validate against the RELAX NG schema (`../../grammar/relaton-oasis-compile.rng`) using the `ruby-jing` gem — so the suite needs a Java runtime on `PATH`. Fixtures live in `spec/oasis/fixtures/`. VCR cassettes record HTTP interactions in `spec/oasis/vcr_cassettes/`. Private methods are tested via `send(:method_name, ...)`.
220
-
221
- ## Style
222
-
223
- RuboCop config inherits from the Ribose OSS style guide. Target Ruby version: 3.4. Rails cops are required but disabled.
224
-
225
- ## Testing
226
-
227
- Run the suite with `bundle exec rake spec:oasis` from the repo root; specs live
228
- in `spec/oasis/` and run self-contained (see the root `CLAUDE.md`).
229
-
230
- - **Index fixture:** `spec/oasis/fixtures/index-v2.zip` — the whole published
231
- index (605 rows, verbatim), seeded into the `Relaton::Index` pool by
232
- `spec/oasis/support/webmock.rb` in `before(:each)`, not only
233
- `before(:suite)`: `Index::Pool#type` replaces the pooled entry whenever
234
- `actual?` says no, and `DataFetcher#index` asks for the same `:oasis` type
235
- with a `file:` and no `url:`. One producer example would otherwise evict the
236
- fixture and send every later consumer example to a real download. The pool
237
- key is `type.upcase.to_sym`, so `:oasis` is pooled as `:OASIS`.
238
- - It is built **with `pubid_class:`**, and `type.index` runs once offline to
239
- force the deserialize and sort. Without that the suite would pass while
240
- exercising something the runtime never does.
241
- - Refresh it with `bundle exec rake spec:update_index_oasis`
242
- (`tasks/index_fixture_oasis.rb`), which copies the published rows verbatim
243
- and refuses a v1 source rather than writing a fixture `Relaton::Index` would
244
- reject wholesale.
245
- - **The fixture must be written with `File.binwrite`.** The zip entry comes
246
- back `ASCII-8BIT`, and one published row carries a non-breaking space —
247
- `OpenC2-MQTT-v1.0]<NBSP>-CS01`, a malformed upstream "cite as" that pubid
248
- keeps verbatim in `original`. `File.write(..., encoding: "UTF-8")`
249
- transcodes and raises `Encoding::UndefinedConversionError` on it. Same
250
- invariant as `Index::FileStorage#write`; OGC's copy of this fixture code gets
251
- away with the transcode only because its rows are pure ASCII.
@@ -1,242 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-ogc is a Ruby gem for retrieving and managing OGC (Open Geospatial Consortium) Standards metadata. It is part of the Relaton ecosystem for bibliographic data management. The gem implements the OGC flavor on top of `relaton-iso`, which itself builds on `relaton-bib`.
8
-
9
- ## Commands
10
-
11
- ```bash
12
- # Install dependencies
13
- bundle install
14
-
15
- # Run all tests
16
- bundle exec rspec
17
-
18
- # Run a single test file
19
- bundle exec rspec spec/relaton/ogc/item_spec.rb
20
-
21
- # Run a single test by line number
22
- bundle exec rspec spec/relaton/ogc/item_spec.rb:7
23
-
24
- # Lint
25
- bundle exec rubocop
26
-
27
- # Lint with auto-correct
28
- bundle exec rubocop -A
29
-
30
- # Interactive console
31
- bin/console
32
-
33
- # Build and install gem locally
34
- bundle exec rake install
35
- ```
36
-
37
- ## Architecture
38
-
39
- ### Class Hierarchy
40
-
41
- All model classes use `Lutaml::Model::Serializable` for XML/YAML serialization with declarative attribute and mapping definitions.
42
-
43
- ```
44
- Iso::Item (from relaton-iso)
45
- └── Relaton::Ogc::Item # serializer, extends Iso::Item with `ext` and `docidentifier`
46
- ├── Relaton::Ogc::Bibitem # includes Bib::BibitemShared
47
- └── Relaton::Ogc::Bibdata # includes Bib::BibdataShared
48
-
49
- Bib::ItemData (from relaton-bib)
50
- └── Relaton::Ogc::ItemData # data model returned by Item.from_xml / Item.from_yaml
51
- ```
52
-
53
- - **Item** (`lib/relaton/ogc/item.rb`) — serializer; extends `Iso::Item`, declares `model ItemData`, adds `ext` (Ext) and `docidentifier` (Docidentifier) attributes
54
- - **ItemData** (`lib/relaton/ogc/item_data.rb`) — plain data model extending `Bib::ItemData`
55
- - **Ext** (`lib/relaton/ogc/ext.rb`) — OGC extension data: doctype, subdoctype (with constrained values)
56
- - **Doctype** (`lib/relaton/ogc/doctype.rb`) — enumerated OGC document types (standard, engineering-report, best-practice, etc.)
57
- - **Docidentifier** (`lib/relaton/ogc/docidentifier.rb`) — parses its `content`
58
- into a `Pubid::Ogc::Identifier` exposed as `#pubid`; see **Pubid-backed
59
- docidentifier**
60
- - **Bibitem** / **Bibdata** — thin wrappers mixing in shared serialization behavior from `relaton-bib`
61
-
62
- ### Search & Fetch Pipeline
63
-
64
- - **Bibliography** (`lib/relaton/ogc/bibliography.rb`) — entry point: `search(text)` and `get(code, year, opts)`
65
- - **HitCollection** (`lib/relaton/ogc/hit_collection.rb`) — fetches from the `relaton-data-ogc` repo (`v2` branch) and resolves the reference against the index; see **Index (`index-v2`, pubid-keyed)**
66
- - **Hit** (`lib/relaton/ogc/hit.rb`) — wraps a single search result; lazy-fetches `ItemData` on `#item`
67
- - **Processor** (`lib/relaton/ogc/processor.rb`) — `Core::Processor` implementation for Relaton registry integration
68
-
69
- ### Pubid-backed docidentifier
70
-
71
- `Relaton::Ogc::Docidentifier` parses its `content` into a
72
- `Pubid::Ogc::Identifier` kept in `@pubid`, while the lutaml `content` attribute
73
- stays a plain **string** for serialization. Parsing is **soft**: `content=`
74
- lazily requires pubid and rescues `LoadError`/`StandardError`, so a missing gem
75
- or non-OGC content leaves `@pubid` nil rather than raising.
76
-
77
- It was an empty subclass before, which meant it inherited
78
- `Bib::Docidentifier`'s `remove_part!` / `remove_date!` / `to_all_parts!` — each
79
- of which raises `NotImplementedError`. `Bib::ItemData` broadcasts all three to
80
- every docidentifier (`item_data.rb:74`, `:87`), so `#to_all_parts` and
81
- `#to_most_recent_reference` **raised on every OGC item**, and because
82
- `NotImplementedError` descends from `ScriptError` a caller's `rescue => e` did
83
- not catch it. The mapping is OGC-specific:
84
-
85
- - **`remove_date!` → clears `revision`.** OGC carries no date component;
86
- `revision` is its version discriminator — the same component a reference may
87
- omit, which `HitCollection#best_match` then treats as a wildcard — so clearing
88
- it yields the version-agnostic ("most recent") reference: `12-128r19` →
89
- `12-128`.
90
- - **`year` is never cleared.** It looks date-like but is half the document
91
- number (`12-128`), not a publication qualifier; dropping it would leave
92
- `-128`, which identifies nothing.
93
- - **`remove_part!` → clears the (unused) `part`/`subpart`.** A no-op for the
94
- rendered string today — OGC models no part — but implemented so it never
95
- raises, and it starts working if pubid-ogc ever adds one. (The IALA
96
- precedent.)
97
- - **`to_all_parts!` → both, then wraps `@pubid` in pubid's `AllParts`.**
98
- `content` stays the plain stripped id — OGC's own renderer has no
99
- "(all parts)" marker, and `content` is cached from the pre-wrap pubid on
100
- purpose. `#pubid` itself, read directly, now answers `all_parts? == true`
101
- and renders WITH pubid's generic marker (`#to_s`), since it's the wrapper.
102
-
103
- `refresh_content!` renders a bare `to_s`. The OGC printed form carries no
104
- publisher token and the stored `content` does not either, so — unlike 3GPP,
105
- whose `refresh_content!` must pass `with_publisher: true` — this one must not.
106
-
107
- ### Index (`index-v2`, pubid-keyed)
108
-
109
- `INDEXFILE` is the pubid-backed `index-v2`: rows are `Pubid::Ogc::Identifier`
110
- hashes (`_type: pubid:ogc:document` with `year`/`number`/`revision`). The
111
- consumer (`HitCollection#index`) and the producer (`DataFetcher#index`) pass
112
- `pubid_class: ::Pubid::Ogc::Identifier`. That is what makes `Relaton::Index`
113
- deserialize the rows into identifiers, sort them by `id.root.number`, and let
114
- `Type#search` bsearch. Omitting it on the **producer** side writes v1-shaped
115
- rows under a v2 name, silently (`FileIO#save` only calls `to_hash` when the
116
- value is an instance of `pubid_class`); omitting it on the **consumer** side
117
- leaves the rows raw hashes with `FileIO#sorted` false, so every lookup scans
118
- all ~1,258 rows. `Processor#remove_index_file` passes `url: true` and `file:`
119
- only: the delete never reads the index (see `lib/relaton/index/CLAUDE.md`).
120
-
121
- `HitCollection#best_match` follows the ETSI/W3C/IALA idiom:
122
-
123
- - **Pass the pubid, not the string.** `Type#search_candidates` narrows only
124
- when the argument is not a `String`, so passing the reference text — which
125
- this flavor used to do — disables the binary search however the index was
126
- built. `pubid_class:` alone fixes nothing; both had to change together.
127
- - **Selection is pubid's asymmetric subset match** (`Pubid::SubsetMatch`):
128
- `pubid === r[:id]`, the reference on the left. A component the reference omits
129
- matches any value, so a bare `OGC 12-128` finds the `r19` row, while
130
- `OGC 12-128r19` reaches only its own. `revision` is OGC's only optional
131
- component. Measured over the whole fixture index (1,258 rows, 19,590
132
- comparisons): the subset match and the old `matches?(…, ignore: [:revision])`
133
- agree on every pair.
134
- - **`year` is always stated.** The bsearch key is the `<nnn>` field alone, so
135
- one bucket holds every year that reused the number — `05-015`, `08-015r2`,
136
- `26-015r1` all sit in bucket `015` (179 buckets, largest 20). `year` is what
137
- tells them apart.
138
- - **An unparseable reference raises**, and the previous full-scan substring
139
- match on `id.to_s` is gone with the rescue: it answered an ambiguous
140
- reference with whichever row sorted first.
141
-
142
- #### Ordering: latest revision wins
143
-
144
- `min_by { |r| r[:id] }` compared raw strings, so a bare `12-128` returned
145
- `r10` — `"r10" < "r14"` as text. The key is now the revision, with its number
146
- compared **as an integer** (`r19` beats `r2`) and the token itself breaking the
147
- tie (`r3a` above `r3`, `r12a` above `r12`), because the index sort is not
148
- stable.
149
-
150
- Scored against each document's own `date[0].at` over the 106 multi-revision
151
- documents in the published index:
152
-
153
- | Key | Picks the newest published document |
154
- |---|---|
155
- | revision number, numeric | 105/106 — **99%** |
156
- | the old `min_by { r[:id] }` | 1/106 — **1%** |
157
-
158
- Only 106 of 1,110 documents carry more than one revision, so the change is
159
- narrow — but for those it was returning the oldest revision almost every time.
160
-
161
- #### Two ids that do not round-trip
162
-
163
- Measured over all 1,258 published rows, 1,256 render back byte-identically. The
164
- two that do not are **data defects upstream**, which pubid canonicalizes:
165
- `"20-001r2 "` (trailing space) and `"11-038R2"` (uppercase revision). Migrating
166
- fixes both — the v2 rows store the canonical form — so a lookup of either
167
- spelling now resolves. Do not "fix" them by special-casing the parser.
168
-
169
- #### Verified against the published index
170
-
171
- `relaton-data-ogc` publishes `index-v2.zip`. Loaded through
172
- `HitCollection#index`: 1,258 rows, all deserialized to identifiers, sorted, **0**
173
- keying on `""`, and a `12-128` lookup narrows to **11 of 1,258**. Live fetches
174
- resolve every shape the flavor has to handle — a bare `OGC 12-128` → `r19`
175
- (2024-02-06), an explicit `12-128r14` → 2017, the letter-suffixed `12-128r12a`,
176
- the two upstream data defects (`11-038R2`, `20-001r2`), the revision-less
177
- `16-079`, both sides of the shared `015` bucket (`05-015` and `26-015r1`), and a
178
- reference with no `OGC ` token.
179
-
180
- The published rows are byte-for-byte what this gem's `DataFetcher` writes: when
181
- the fixture was still being converted from `index-v1` locally, the converted set
182
- and the published one held the same 1,258 rows.
183
-
184
- ### Data Fetching
185
-
186
- - **DataFetcher** (`lib/relaton/ogc/data_fetcher.rb`) — bulk-fetches OGC documents from the NamingAuthority JSON endpoint
187
- - **Scraper** (`lib/relaton/ogc/scraper.rb`) — parses individual OGC document JSON into `ItemData`
188
-
189
- ### Key Dependencies
190
-
191
- - `relaton-iso` — provides `Iso::Item` base class and ISO bibliographic structures
192
- - `relaton-bib` (transitive via relaton-iso) — core types: `Bib::ItemData`, `Bib::Doctype`, `Bib::Ext`, etc.
193
- - `lutaml-model` (transitive) — serialization framework for XML/YAML mapping
194
- - `faraday` — HTTP client for fetching remote data
195
- - `relaton-index` — index-based document lookup from GitHub-hosted data repos
196
-
197
- ### Serialization Pattern
198
-
199
- Models declare attributes and XML mappings in the same class. `Item` is the serializer (handles `from_xml`, `from_yaml`, `to_xml`), `ItemData` is the plain data object returned. Round-trip fidelity (parse → serialize → parse) is the primary correctness criterion tested in specs.
200
-
201
- ### Test Setup
202
-
203
- - **RSpec** with `--format documentation`
204
- - **VCR** + **WebMock** for HTTP interaction recording (`spec/vcr_cassettes/`)
205
- - **Jing** for RelaxNG schema validation against `grammars/relaton-ogc-compile.rng`
206
- - **canon** for XML comparison in tests
207
- - Fixtures live in `spec/fixtures/` (YAML, XML)
208
- - Integration tests in `spec/relaton/ogc/integration_spec.rb` exercise the full search → fetch → serialize pipeline
209
-
210
- ### Grammars
211
-
212
- RelaxNG schemas in `grammars/` define the valid XML structure. `relaton-ogc-compile.rng` is the combined schema used for test validation.
213
-
214
- ### API Notes
215
-
216
- - `Bib::Docidentifier` uses `.content` (not `.id`) for the identifier string
217
- - `Bib::Date` uses `.at` returning a `StringDate::Value`; parse with `::Date.parse(d.at.to_s)` to extract year
218
-
219
- ## Linting
220
-
221
- RuboCop inherits from the [Ribose OSS style guide](https://github.com/riboseinc/oss-guides). Target Ruby version is 3.1. Rails cops are loaded but disabled.
222
-
223
- ## CI
224
-
225
- GitHub Actions workflows in `.github/workflows/` are auto-generated by Cimas — avoid manual edits. `rake.yml` runs tests on push/PR; `release.yml` handles gem publishing.
226
-
227
- ## Testing
228
-
229
- - Run the suite with `bundle exec rake spec:ogc` from the repo root; specs live
230
- in `spec/ogc/` and run self-contained (see the root `CLAUDE.md`).
231
- - **Index fixture:** `spec/ogc/fixtures/index-v2.zip` is seeded into the
232
- `Relaton::Index` pool by `spec/ogc/support/webmock.rb` — in `before(:each)`,
233
- not only `before(:suite)`, because `Index::Pool#type` replaces the pooled
234
- entry whenever `actual?` says no and `DataFetcher#index` asks for the same
235
- type without a `url:`. The pool key is `type.upcase.to_sym`, so the flavor's
236
- `:ogc` is pooled as `:OGC`. It is built **with `pubid_class:`**; without it
237
- the suite would pass while exercising something the runtime never does.
238
- - Refresh it with `bundle exec rake spec:update_index_ogc`. Unlike 3GPP's, this
239
- fixture is the whole published index (~1,258 rows) rather than a curated
240
- subset — small enough that keeping all of it costs nothing. Rows are copied
241
- **verbatim** from the published `index-v2.zip`; `#build` refuses a v1 source
242
- rather than writing a fixture `Relaton::Index` would reject wholesale.
@@ -1,47 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-oiml retrieves International Organization of Legal Metrology (OIML) publications using the Relaton model. It is **index-backed** (no scraping): it searches a pre-built index via `Relaton::Index` and fetches per-document YAML from the `relaton/relaton-data-oiml` GitHub repo. OIML reference parsing uses `Pubid::Oiml::Identifier`.
8
-
9
- ## Development
10
-
11
- ```bash
12
- bundle exec rake # default task → rspec
13
- bundle exec rspec spec/relaton/oiml/bibliography_spec.rb # single file
14
- bundle exec rake spec:update_index # refresh the OIML index fixture from relaton-data-oiml
15
- ```
16
-
17
- (No gem-local rubocop config; the monorepo root config applies.)
18
-
19
- ## Architecture
20
-
21
- Namespace: `Relaton::Oiml`. Retrieval flow:
22
-
23
- 1. **Bibliography** (`lib/relaton/oiml/bibliography.rb`) — `get(code, year)` / `search`; parses the reference with `Pubid::Oiml.parse`, looks it up in the index, and fetches the matching YAML. `get` suppresses the edition year for an **undated** citation (mirroring ISO): when the reference carries no year (and `opts[:keep_year]` is not set) it returns `item.to_most_recent_reference`, so `OIML B 18` renders undated, while a dated citation (`OIML B 18:2022`, or an explicit `year`) still pins that edition. This relies on `Oiml::Docidentifier#remove_date!` refreshing `content` from the dateless pubid — mutating the wrapped `@pubid` alone leaves the rendered `content` dated (issue #72).
24
- 2. **Index** — `Relaton::Index.find_or_create(:oiml, url: ..., pubid_class: Pubid::Oiml::Identifier)`. `INDEXFILE` is defined in `lib/relaton/oiml.rb`. `Bibliography#best_row` matches a **Bulletin** exactly (`index.search(query, exact: true)`), and every other type by `pubid_match?`: the year-and-language-stripped "stem" is equal, the language is equal, and the year is equal when the reference or the `year` argument names one; the latest edition wins. A Bulletin needs the exact match because its year is part of its locator: the issue and the sequence render after the year, so the stem reduced every article of one year to `OIML Bulletin`, and a lookup returned an arbitrary one of them.
25
-
26
- **OIML does not use pubid's subset match `===` yet**, unlike the other flavors that pubid#408 unblocked. Measured over the full `relaton-data-oiml` index (5,646 rows), `===` disagrees with `pubid_match?` outside the Bulletins on 192 pairs, and each one is a regression: `language` is not `subset_strict`, so a language-less Amendment, Annex or Errata reference reaches its translations (156 pairs); `subpart` is not strict, so `OIML R 137-1 (F)` reaches `OIML R 137-1-2:2012 (F)` (6); and the `year_on_base` render flag is compared, so `OIML R 102 Annex B-C` misses `OIML R 102:1995 Annex B-C` (30). A plain Recommendation escapes the first one only because its render hint `parsed_format` differs, which is luck. The hand-off `metanorma__pubid__oiml-subset-match-strict-language` asks pubid for the fix. `spec/oiml/relaton/oiml/bibliography_spec.rb` pins all of these with an in-memory index ("where the subset match would disagree"), so a later switch must pass them.
27
-
28
- **Dual-published identifiers** (relaton#180, pubid#437): OIML sometimes co-publishes a document jointly with another SDO (ISO confirmed so far), printed with both identifiers joined by a bare `|`, e.g. `ISO 4064-1:2024|OIML R 49-1:2024`. pubid models this as `Pubid::Oiml::Identifiers::DualPublished`, which wraps `first`/`second` (each a real, independently-typed `::Pubid::Identifier`, in original print order) and delegates `#root`/`#code`/`#type`/`#stage`/`#iteration`/`#publisher`/the `mr_*` hooks to whichever side `#oiml_identifier` finds — but deliberately **not** `#year`, `#date`, `#part`, or `#language`: reading or mutating those directly on the wrapper is a silent no-op (the wrapper's own copies of those attributes are simply never populated). Only the generic, immutable `#exclude(:attr)` correctly recurses into both sides, because it walks the wrapper's declared attributes and, for the two that are nested identifiers (`first`/`second`), recurses into each.
29
-
30
- Because `#root` already delegates to the OIML side, `Relaton::Index` needs no change — a dual-published row buckets under its OIML number exactly like a plain one. The gap was in `Bibliography` and `Docidentifier`, both of which read/mutate a pubid's flat attributes directly:
31
-
32
- - `Bibliography#oiml_side(pubid)` unwraps a `DualPublished` to its OIML side (a plain pubid passes through unchanged), and `stem`/`pubid_match?`/`best_row`'s edition `max_by` all route through it before touching `.year`/`.language`. Without this, a plain query's stem (`"OIML R 49-1"`) can never equal a dual-published row's stem (`"ISO 4064-1|OIML R 49-1"` at full string), and `.year`/`.language` read directly off a `DualPublished` are always nil/no-op — silently breaking edition selection and language matching. Reducing both sides to the OIML identifier also makes stem comparison print-order-independent (`OIML R 49-1|ISO 4064-1` and `ISO 4064-1|OIML R 49-1` match identically).
33
- - `Docidentifier#remove_date!`/`#remove_part!` compute the mutated string via `@pubid.exclude(:date)`/`@pubid.exclude(:part)` and reassign through `content=` (which reparses and resyncs `@pubid`), instead of the old `@pubid.date = nil`/`@pubid.part = nil` in-place mutation — the in-place form is indistinguishable from success (no exception) but silently does nothing on a `DualPublished`. `#exclude` behaves identically to the old in-place mutation for a plain (non-dual) pubid, so this is a safe rewrite, not just a dual-published-only branch.
34
-
35
- No second `docidentifier` array entry is added for the external (ISO) side — pubid's `DualPublished` already exposes both sides structurally off the one docidentifier's `pubid` (`oiml_identifier`/`external_identifier`), so a consumer that needs the ISO side reads it from there.
36
- 3. **Item / ItemData / Ext** (`item.rb`, `item_data.rb`, `ext.rb`) — `Item` extends `Bib::Item`; `Ext` carries OIML-specific fields (scope, quantity, measuring_instrument, focus_area, sustainability_framework, doi). `Item.from_yaml` deserializes the fetched document.
37
- 4. **Processor** (`lib/relaton/oiml/processor.rb`) — registry integration; `@prefix = "OIML"`, `@defaultprefix = %r{^OIML\s}`. Lazy-`require_relative`s `../oiml` in its methods, including `remove_index_file`.
38
-
39
- There are no scrapers — everything comes from the curated index + GitHub YAML.
40
-
41
- ## External dependencies
42
-
43
- `pubid ~> 2.0.0.pre.alpha.3`, `base64`, `relaton-core`, `relaton-bib`, `relaton-index`.
44
-
45
- ## Testing
46
-
47
- RSpec with WebMock + VCR. The offline index fixture `spec/fixtures/index-v2.zip` is pre-loaded into the `Relaton::Index` pool in `before(:suite)` (configured in `spec/support/webmock.rb`); per-document data requests are stubbed by WebMock and served from `spec/fixtures/data/*.yaml`. Run `rake spec:update_index` to refresh the fixture.