relaton 3.0.0.pre.alpha.17 → 3.0.0.pre.alpha.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/data/equivalence/adoption-chains.yaml +59 -0
- data/lib/relaton/3gpp/CLAUDE.md +187 -0
- data/lib/relaton/3gpp/README.adoc +149 -0
- data/lib/relaton/adobe/CLAUDE.md +104 -0
- data/lib/relaton/bib/CLAUDE.md +130 -0
- data/lib/relaton/bib/README.adoc +330 -0
- data/lib/relaton/bib/model/item_shared.rb +5 -1
- data/lib/relaton/bib/model/series.rb +1 -1
- data/lib/relaton/bipm/CLAUDE.md +173 -0
- data/lib/relaton/bipm/README.adoc +352 -0
- data/lib/relaton/bipm/data_fetcher.rb +3 -2
- data/lib/relaton/bsi/CLAUDE.md +35 -0
- data/lib/relaton/bsi/README.adoc +112 -0
- data/lib/relaton/calconnect/CLAUDE.md +177 -0
- data/lib/relaton/calconnect/README.adoc +140 -0
- data/lib/relaton/ccsds/CLAUDE.md +74 -0
- data/lib/relaton/ccsds/README.adoc +154 -0
- data/lib/relaton/cen/CLAUDE.md +219 -0
- data/lib/relaton/cen/README.adoc +155 -0
- data/lib/relaton/cie/CLAUDE.md +90 -0
- data/lib/relaton/cie/README.adoc +121 -0
- data/lib/relaton/core/CLAUDE.md +140 -0
- data/lib/relaton/core/README.adoc +64 -0
- data/lib/relaton/core/data_fetcher.rb +142 -3
- data/lib/relaton/db/CLAUDE.md +373 -0
- data/lib/relaton/db.rb +21 -1
- data/lib/relaton/doi/CLAUDE.md +105 -0
- data/lib/relaton/doi/README.adoc +89 -0
- data/lib/relaton/easc/CLAUDE.md +85 -0
- data/lib/relaton/ecma/CLAUDE.md +217 -0
- data/lib/relaton/ecma/README.adoc +178 -0
- data/lib/relaton/equivalence.rb +117 -0
- data/lib/relaton/etsi/CLAUDE.md +219 -0
- data/lib/relaton/etsi/README.adoc +137 -0
- data/lib/relaton/evs/CLAUDE.md +120 -0
- data/lib/relaton/evs/README.adoc +27 -0
- data/lib/relaton/gb/CLAUDE.md +47 -0
- data/lib/relaton/gb/README.adoc +169 -0
- data/lib/relaton/gost/CLAUDE.md +100 -0
- data/lib/relaton/iala/CLAUDE.md +97 -0
- data/lib/relaton/iana/CLAUDE.md +105 -0
- data/lib/relaton/iana/README.adoc +107 -0
- data/lib/relaton/iec/CLAUDE.md +13 -0
- data/lib/relaton/iec/README.adoc +280 -0
- data/lib/relaton/iec/data_fetcher.rb +24 -8
- data/lib/relaton/iec/project_fetcher.rb +48 -0
- data/lib/relaton/iec/project_parser.rb +111 -0
- data/lib/relaton/iec.rb +5 -0
- data/lib/relaton/ieee/CLAUDE.md +176 -0
- data/lib/relaton/ieee/README.adoc +131 -0
- data/lib/relaton/ietf/CLAUDE.md +357 -0
- data/lib/relaton/ietf/README.adoc +167 -0
- data/lib/relaton/iho/CLAUDE.md +70 -0
- data/lib/relaton/iho/README.adoc +119 -0
- data/lib/relaton/index/CLAUDE.md +207 -0
- data/lib/relaton/index/README.adoc +221 -0
- data/lib/relaton/index/dataset_lock.rb +122 -0
- data/lib/relaton/index/family.rb +63 -0
- data/lib/relaton/index/type.rb +32 -0
- data/lib/relaton/index.rb +3 -0
- data/lib/relaton/isbn/CLAUDE.md +47 -0
- data/lib/relaton/isbn/README.adoc +91 -0
- data/lib/relaton/iso/CLAUDE.md +38 -0
- data/lib/relaton/iso/README.adoc +370 -0
- data/lib/relaton/iso/data_fetcher.rb +16 -1
- data/lib/relaton/itu/CLAUDE.md +685 -0
- data/lib/relaton/itu/README.adoc +188 -0
- data/lib/relaton/jcgm/CLAUDE.md +107 -0
- data/lib/relaton/jis/CLAUDE.md +64 -0
- data/lib/relaton/jis/README.adoc +167 -0
- data/lib/relaton/logger/CLAUDE.md +37 -0
- data/lib/relaton/logger/README.adoc +120 -0
- data/lib/relaton/nist/CLAUDE.md +180 -0
- data/lib/relaton/nist/README.adoc +389 -0
- data/lib/relaton/nist/bibliography.rb +17 -1
- data/lib/relaton/nist/data_fetcher.rb +60 -4
- data/lib/relaton/nist/hit_collection.rb +121 -59
- data/lib/relaton/nist/scraper.rb +21 -7
- data/lib/relaton/oasis/CLAUDE.md +251 -0
- data/lib/relaton/oasis/README.adoc +132 -0
- data/lib/relaton/ogc/CLAUDE.md +242 -0
- data/lib/relaton/ogc/README.adoc +134 -0
- data/lib/relaton/ogc/data_fetcher.rb +9 -3
- data/lib/relaton/oiml/CLAUDE.md +47 -0
- data/lib/relaton/oiml/README.adoc +70 -0
- data/lib/relaton/omg/CLAUDE.md +137 -0
- data/lib/relaton/omg/README.adoc +101 -0
- data/lib/relaton/plateau/CLAUDE.md +126 -0
- data/lib/relaton/plateau/README.adoc +139 -0
- data/lib/relaton/sdo/CLAUDE.md +81 -0
- data/lib/relaton/un/CLAUDE.md +91 -0
- data/lib/relaton/un/README.adoc +125 -0
- data/lib/relaton/version.rb +1 -1
- data/lib/relaton/w3c/CLAUDE.md +369 -0
- data/lib/relaton/w3c/README.adoc +140 -0
- data/lib/relaton/w3c/data_fetcher.rb +1 -1
- data/lib/relaton/xsf/CLAUDE.md +189 -0
- data/lib/relaton/xsf/README.adoc +127 -0
- data/lib/relaton.rb +1 -0
- metadata +85 -4
|
@@ -0,0 +1,242 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## Project Overview
|
|
6
|
+
|
|
7
|
+
relaton-ogc is a Ruby gem for retrieving and managing OGC (Open Geospatial Consortium) Standards metadata. It is part of the Relaton ecosystem for bibliographic data management. The gem implements the OGC flavor on top of `relaton-iso`, which itself builds on `relaton-bib`.
|
|
8
|
+
|
|
9
|
+
## Commands
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
# Install dependencies
|
|
13
|
+
bundle install
|
|
14
|
+
|
|
15
|
+
# Run all tests
|
|
16
|
+
bundle exec rspec
|
|
17
|
+
|
|
18
|
+
# Run a single test file
|
|
19
|
+
bundle exec rspec spec/relaton/ogc/item_spec.rb
|
|
20
|
+
|
|
21
|
+
# Run a single test by line number
|
|
22
|
+
bundle exec rspec spec/relaton/ogc/item_spec.rb:7
|
|
23
|
+
|
|
24
|
+
# Lint
|
|
25
|
+
bundle exec rubocop
|
|
26
|
+
|
|
27
|
+
# Lint with auto-correct
|
|
28
|
+
bundle exec rubocop -A
|
|
29
|
+
|
|
30
|
+
# Interactive console
|
|
31
|
+
bin/console
|
|
32
|
+
|
|
33
|
+
# Build and install gem locally
|
|
34
|
+
bundle exec rake install
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Architecture
|
|
38
|
+
|
|
39
|
+
### Class Hierarchy
|
|
40
|
+
|
|
41
|
+
All model classes use `Lutaml::Model::Serializable` for XML/YAML serialization with declarative attribute and mapping definitions.
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
Iso::Item (from relaton-iso)
|
|
45
|
+
└── Relaton::Ogc::Item # serializer, extends Iso::Item with `ext` and `docidentifier`
|
|
46
|
+
├── Relaton::Ogc::Bibitem # includes Bib::BibitemShared
|
|
47
|
+
└── Relaton::Ogc::Bibdata # includes Bib::BibdataShared
|
|
48
|
+
|
|
49
|
+
Bib::ItemData (from relaton-bib)
|
|
50
|
+
└── Relaton::Ogc::ItemData # data model returned by Item.from_xml / Item.from_yaml
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
- **Item** (`lib/relaton/ogc/item.rb`) — serializer; extends `Iso::Item`, declares `model ItemData`, adds `ext` (Ext) and `docidentifier` (Docidentifier) attributes
|
|
54
|
+
- **ItemData** (`lib/relaton/ogc/item_data.rb`) — plain data model extending `Bib::ItemData`
|
|
55
|
+
- **Ext** (`lib/relaton/ogc/ext.rb`) — OGC extension data: doctype, subdoctype (with constrained values)
|
|
56
|
+
- **Doctype** (`lib/relaton/ogc/doctype.rb`) — enumerated OGC document types (standard, engineering-report, best-practice, etc.)
|
|
57
|
+
- **Docidentifier** (`lib/relaton/ogc/docidentifier.rb`) — parses its `content`
|
|
58
|
+
into a `Pubid::Ogc::Identifier` exposed as `#pubid`; see **Pubid-backed
|
|
59
|
+
docidentifier**
|
|
60
|
+
- **Bibitem** / **Bibdata** — thin wrappers mixing in shared serialization behavior from `relaton-bib`
|
|
61
|
+
|
|
62
|
+
### Search & Fetch Pipeline
|
|
63
|
+
|
|
64
|
+
- **Bibliography** (`lib/relaton/ogc/bibliography.rb`) — entry point: `search(text)` and `get(code, year, opts)`
|
|
65
|
+
- **HitCollection** (`lib/relaton/ogc/hit_collection.rb`) — fetches from the `relaton-data-ogc` repo (`v2` branch) and resolves the reference against the index; see **Index (`index-v2`, pubid-keyed)**
|
|
66
|
+
- **Hit** (`lib/relaton/ogc/hit.rb`) — wraps a single search result; lazy-fetches `ItemData` on `#item`
|
|
67
|
+
- **Processor** (`lib/relaton/ogc/processor.rb`) — `Core::Processor` implementation for Relaton registry integration
|
|
68
|
+
|
|
69
|
+
### Pubid-backed docidentifier
|
|
70
|
+
|
|
71
|
+
`Relaton::Ogc::Docidentifier` parses its `content` into a
|
|
72
|
+
`Pubid::Ogc::Identifier` kept in `@pubid`, while the lutaml `content` attribute
|
|
73
|
+
stays a plain **string** for serialization. Parsing is **soft**: `content=`
|
|
74
|
+
lazily requires pubid and rescues `LoadError`/`StandardError`, so a missing gem
|
|
75
|
+
or non-OGC content leaves `@pubid` nil rather than raising.
|
|
76
|
+
|
|
77
|
+
It was an empty subclass before, which meant it inherited
|
|
78
|
+
`Bib::Docidentifier`'s `remove_part!` / `remove_date!` / `to_all_parts!` — each
|
|
79
|
+
of which raises `NotImplementedError`. `Bib::ItemData` broadcasts all three to
|
|
80
|
+
every docidentifier (`item_data.rb:74`, `:87`), so `#to_all_parts` and
|
|
81
|
+
`#to_most_recent_reference` **raised on every OGC item**, and because
|
|
82
|
+
`NotImplementedError` descends from `ScriptError` a caller's `rescue => e` did
|
|
83
|
+
not catch it. The mapping is OGC-specific:
|
|
84
|
+
|
|
85
|
+
- **`remove_date!` → clears `revision`.** OGC carries no date component;
|
|
86
|
+
`revision` is its version discriminator — the same component a reference may
|
|
87
|
+
omit, which `HitCollection#best_match` then treats as a wildcard — so clearing
|
|
88
|
+
it yields the version-agnostic ("most recent") reference: `12-128r19` →
|
|
89
|
+
`12-128`.
|
|
90
|
+
- **`year` is never cleared.** It looks date-like but is half the document
|
|
91
|
+
number (`12-128`), not a publication qualifier; dropping it would leave
|
|
92
|
+
`-128`, which identifies nothing.
|
|
93
|
+
- **`remove_part!` → clears the (unused) `part`/`subpart`.** A no-op for the
|
|
94
|
+
rendered string today — OGC models no part — but implemented so it never
|
|
95
|
+
raises, and it starts working if pubid-ogc ever adds one. (The IALA
|
|
96
|
+
precedent.)
|
|
97
|
+
- **`to_all_parts!` → both, then wraps `@pubid` in pubid's `AllParts`.**
|
|
98
|
+
`content` stays the plain stripped id — OGC's own renderer has no
|
|
99
|
+
"(all parts)" marker, and `content` is cached from the pre-wrap pubid on
|
|
100
|
+
purpose. `#pubid` itself, read directly, now answers `all_parts? == true`
|
|
101
|
+
and renders WITH pubid's generic marker (`#to_s`), since it's the wrapper.
|
|
102
|
+
|
|
103
|
+
`refresh_content!` renders a bare `to_s`. The OGC printed form carries no
|
|
104
|
+
publisher token and the stored `content` does not either, so — unlike 3GPP,
|
|
105
|
+
whose `refresh_content!` must pass `with_publisher: true` — this one must not.
|
|
106
|
+
|
|
107
|
+
### Index (`index-v2`, pubid-keyed)
|
|
108
|
+
|
|
109
|
+
`INDEXFILE` is the pubid-backed `index-v2`: rows are `Pubid::Ogc::Identifier`
|
|
110
|
+
hashes (`_type: pubid:ogc:document` with `year`/`number`/`revision`). The
|
|
111
|
+
consumer (`HitCollection#index`) and the producer (`DataFetcher#index`) pass
|
|
112
|
+
`pubid_class: ::Pubid::Ogc::Identifier`. That is what makes `Relaton::Index`
|
|
113
|
+
deserialize the rows into identifiers, sort them by `id.root.number`, and let
|
|
114
|
+
`Type#search` bsearch. Omitting it on the **producer** side writes v1-shaped
|
|
115
|
+
rows under a v2 name, silently (`FileIO#save` only calls `to_hash` when the
|
|
116
|
+
value is an instance of `pubid_class`); omitting it on the **consumer** side
|
|
117
|
+
leaves the rows raw hashes with `FileIO#sorted` false, so every lookup scans
|
|
118
|
+
all ~1,258 rows. `Processor#remove_index_file` passes `url: true` and `file:`
|
|
119
|
+
only: the delete never reads the index (see `lib/relaton/index/CLAUDE.md`).
|
|
120
|
+
|
|
121
|
+
`HitCollection#best_match` follows the ETSI/W3C/IALA idiom:
|
|
122
|
+
|
|
123
|
+
- **Pass the pubid, not the string.** `Type#search_candidates` narrows only
|
|
124
|
+
when the argument is not a `String`, so passing the reference text — which
|
|
125
|
+
this flavor used to do — disables the binary search however the index was
|
|
126
|
+
built. `pubid_class:` alone fixes nothing; both had to change together.
|
|
127
|
+
- **Selection is pubid's asymmetric subset match** (`Pubid::SubsetMatch`):
|
|
128
|
+
`pubid === r[:id]`, the reference on the left. A component the reference omits
|
|
129
|
+
matches any value, so a bare `OGC 12-128` finds the `r19` row, while
|
|
130
|
+
`OGC 12-128r19` reaches only its own. `revision` is OGC's only optional
|
|
131
|
+
component. Measured over the whole fixture index (1,258 rows, 19,590
|
|
132
|
+
comparisons): the subset match and the old `matches?(…, ignore: [:revision])`
|
|
133
|
+
agree on every pair.
|
|
134
|
+
- **`year` is always stated.** The bsearch key is the `<nnn>` field alone, so
|
|
135
|
+
one bucket holds every year that reused the number — `05-015`, `08-015r2`,
|
|
136
|
+
`26-015r1` all sit in bucket `015` (179 buckets, largest 20). `year` is what
|
|
137
|
+
tells them apart.
|
|
138
|
+
- **An unparseable reference raises**, and the previous full-scan substring
|
|
139
|
+
match on `id.to_s` is gone with the rescue: it answered an ambiguous
|
|
140
|
+
reference with whichever row sorted first.
|
|
141
|
+
|
|
142
|
+
#### Ordering: latest revision wins
|
|
143
|
+
|
|
144
|
+
`min_by { |r| r[:id] }` compared raw strings, so a bare `12-128` returned
|
|
145
|
+
`r10` — `"r10" < "r14"` as text. The key is now the revision, with its number
|
|
146
|
+
compared **as an integer** (`r19` beats `r2`) and the token itself breaking the
|
|
147
|
+
tie (`r3a` above `r3`, `r12a` above `r12`), because the index sort is not
|
|
148
|
+
stable.
|
|
149
|
+
|
|
150
|
+
Scored against each document's own `date[0].at` over the 106 multi-revision
|
|
151
|
+
documents in the published index:
|
|
152
|
+
|
|
153
|
+
| Key | Picks the newest published document |
|
|
154
|
+
|---|---|
|
|
155
|
+
| revision number, numeric | 105/106 — **99%** |
|
|
156
|
+
| the old `min_by { r[:id] }` | 1/106 — **1%** |
|
|
157
|
+
|
|
158
|
+
Only 106 of 1,110 documents carry more than one revision, so the change is
|
|
159
|
+
narrow — but for those it was returning the oldest revision almost every time.
|
|
160
|
+
|
|
161
|
+
#### Two ids that do not round-trip
|
|
162
|
+
|
|
163
|
+
Measured over all 1,258 published rows, 1,256 render back byte-identically. The
|
|
164
|
+
two that do not are **data defects upstream**, which pubid canonicalizes:
|
|
165
|
+
`"20-001r2 "` (trailing space) and `"11-038R2"` (uppercase revision). Migrating
|
|
166
|
+
fixes both — the v2 rows store the canonical form — so a lookup of either
|
|
167
|
+
spelling now resolves. Do not "fix" them by special-casing the parser.
|
|
168
|
+
|
|
169
|
+
#### Verified against the published index
|
|
170
|
+
|
|
171
|
+
`relaton-data-ogc` publishes `index-v2.zip`. Loaded through
|
|
172
|
+
`HitCollection#index`: 1,258 rows, all deserialized to identifiers, sorted, **0**
|
|
173
|
+
keying on `""`, and a `12-128` lookup narrows to **11 of 1,258**. Live fetches
|
|
174
|
+
resolve every shape the flavor has to handle — a bare `OGC 12-128` → `r19`
|
|
175
|
+
(2024-02-06), an explicit `12-128r14` → 2017, the letter-suffixed `12-128r12a`,
|
|
176
|
+
the two upstream data defects (`11-038R2`, `20-001r2`), the revision-less
|
|
177
|
+
`16-079`, both sides of the shared `015` bucket (`05-015` and `26-015r1`), and a
|
|
178
|
+
reference with no `OGC ` token.
|
|
179
|
+
|
|
180
|
+
The published rows are byte-for-byte what this gem's `DataFetcher` writes: when
|
|
181
|
+
the fixture was still being converted from `index-v1` locally, the converted set
|
|
182
|
+
and the published one held the same 1,258 rows.
|
|
183
|
+
|
|
184
|
+
### Data Fetching
|
|
185
|
+
|
|
186
|
+
- **DataFetcher** (`lib/relaton/ogc/data_fetcher.rb`) — bulk-fetches OGC documents from the NamingAuthority JSON endpoint
|
|
187
|
+
- **Scraper** (`lib/relaton/ogc/scraper.rb`) — parses individual OGC document JSON into `ItemData`
|
|
188
|
+
|
|
189
|
+
### Key Dependencies
|
|
190
|
+
|
|
191
|
+
- `relaton-iso` — provides `Iso::Item` base class and ISO bibliographic structures
|
|
192
|
+
- `relaton-bib` (transitive via relaton-iso) — core types: `Bib::ItemData`, `Bib::Doctype`, `Bib::Ext`, etc.
|
|
193
|
+
- `lutaml-model` (transitive) — serialization framework for XML/YAML mapping
|
|
194
|
+
- `faraday` — HTTP client for fetching remote data
|
|
195
|
+
- `relaton-index` — index-based document lookup from GitHub-hosted data repos
|
|
196
|
+
|
|
197
|
+
### Serialization Pattern
|
|
198
|
+
|
|
199
|
+
Models declare attributes and XML mappings in the same class. `Item` is the serializer (handles `from_xml`, `from_yaml`, `to_xml`), `ItemData` is the plain data object returned. Round-trip fidelity (parse → serialize → parse) is the primary correctness criterion tested in specs.
|
|
200
|
+
|
|
201
|
+
### Test Setup
|
|
202
|
+
|
|
203
|
+
- **RSpec** with `--format documentation`
|
|
204
|
+
- **VCR** + **WebMock** for HTTP interaction recording (`spec/vcr_cassettes/`)
|
|
205
|
+
- **Jing** for RelaxNG schema validation against `grammars/relaton-ogc-compile.rng`
|
|
206
|
+
- **equivalent-xml** for XML comparison in tests
|
|
207
|
+
- Fixtures live in `spec/fixtures/` (YAML, XML)
|
|
208
|
+
- Integration tests in `spec/relaton/ogc/integration_spec.rb` exercise the full search → fetch → serialize pipeline
|
|
209
|
+
|
|
210
|
+
### Grammars
|
|
211
|
+
|
|
212
|
+
RelaxNG schemas in `grammars/` define the valid XML structure. `relaton-ogc-compile.rng` is the combined schema used for test validation.
|
|
213
|
+
|
|
214
|
+
### API Notes
|
|
215
|
+
|
|
216
|
+
- `Bib::Docidentifier` uses `.content` (not `.id`) for the identifier string
|
|
217
|
+
- `Bib::Date` uses `.at` returning a `StringDate::Value`; parse with `::Date.parse(d.at.to_s)` to extract year
|
|
218
|
+
|
|
219
|
+
## Linting
|
|
220
|
+
|
|
221
|
+
RuboCop inherits from the [Ribose OSS style guide](https://github.com/riboseinc/oss-guides). Target Ruby version is 3.1. Rails cops are loaded but disabled.
|
|
222
|
+
|
|
223
|
+
## CI
|
|
224
|
+
|
|
225
|
+
GitHub Actions workflows in `.github/workflows/` are auto-generated by Cimas — avoid manual edits. `rake.yml` runs tests on push/PR; `release.yml` handles gem publishing.
|
|
226
|
+
|
|
227
|
+
## Testing
|
|
228
|
+
|
|
229
|
+
- Run the suite with `bundle exec rake spec:ogc` from the repo root; specs live
|
|
230
|
+
in `spec/ogc/` and run self-contained (see the root `CLAUDE.md`).
|
|
231
|
+
- **Index fixture:** `spec/ogc/fixtures/index-v2.zip` is seeded into the
|
|
232
|
+
`Relaton::Index` pool by `spec/ogc/support/webmock.rb` — in `before(:each)`,
|
|
233
|
+
not only `before(:suite)`, because `Index::Pool#type` replaces the pooled
|
|
234
|
+
entry whenever `actual?` says no and `DataFetcher#index` asks for the same
|
|
235
|
+
type without a `url:`. The pool key is `type.upcase.to_sym`, so the flavor's
|
|
236
|
+
`:ogc` is pooled as `:OGC`. It is built **with `pubid_class:`**; without it
|
|
237
|
+
the suite would pass while exercising something the runtime never does.
|
|
238
|
+
- Refresh it with `bundle exec rake spec:update_index_ogc`. Unlike 3GPP's, this
|
|
239
|
+
fixture is the whole published index (~1,258 rows) rather than a curated
|
|
240
|
+
subset — small enough that keeping all of it costs nothing. Rows are copied
|
|
241
|
+
**verbatim** from the published `index-v2.zip`; `#build` refuses a v1 source
|
|
242
|
+
rather than writing a fixture `Relaton::Index` would reject wholesale.
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
= Relaton::Ogc
|
|
2
|
+
|
|
3
|
+
Relaton::Ogc is a Ruby gem that implements the https://github.com/metanorma/metanorma-model-iso#iso-bibliographic-item[IsoBibliographicItem model].
|
|
4
|
+
|
|
5
|
+
You can use it to retrieve metadata of OGC Standards from the https://github.com/relaton/relaton-data-ogc[relaton-data-ogc] dataset, and access such metadata through the `Relaton::Ogc::ItemData` object.
|
|
6
|
+
|
|
7
|
+
== Usage
|
|
8
|
+
|
|
9
|
+
=== Search for a standard using keywords
|
|
10
|
+
|
|
11
|
+
[source,ruby]
|
|
12
|
+
----
|
|
13
|
+
require 'relaton/ogc'
|
|
14
|
+
=> true
|
|
15
|
+
|
|
16
|
+
hits = Relaton::Ogc::Bibliography.search("OGC 19-025r1")
|
|
17
|
+
=> <Relaton::Ogc::HitCollection:0x00000000001230 @ref=19-025r1 @fetched=false>
|
|
18
|
+
|
|
19
|
+
item = hits[0].item
|
|
20
|
+
=> #<Relaton::Ogc::ItemData:0x0000000127b1d870
|
|
21
|
+
...
|
|
22
|
+
----
|
|
23
|
+
|
|
24
|
+
=== XML serialization
|
|
25
|
+
[source,ruby]
|
|
26
|
+
----
|
|
27
|
+
item.to_xml
|
|
28
|
+
=> "<bibitem id="19025r1" type="standard" schema-version="v1.4.1">
|
|
29
|
+
<fetched>2026-02-27</fetched>
|
|
30
|
+
<title language="en" script="Latn" type="title-main">Development of Spatial Data Infrastructures for Marine Data Management</title>
|
|
31
|
+
<title language="en" script="Latn" type="main">Development of Spatial Data Infrastructures for Marine Data Management</title>
|
|
32
|
+
<uri type="src">http://www.opengis.net/doc/per/marine-sdi</uri>
|
|
33
|
+
<uri type="pdf">https://portal.ogc.org/files/?artifact_id=88037</uri>
|
|
34
|
+
<docidentifier type="OGC" primary="true">19-025r1</docidentifier>
|
|
35
|
+
...
|
|
36
|
+
</bibitem>"
|
|
37
|
+
----
|
|
38
|
+
With argument `bibdata: true` it outputs XML wrapped by `bibdata` element and adds flavor `ext` element.
|
|
39
|
+
[source,ruby]
|
|
40
|
+
----
|
|
41
|
+
item.to_xml bibdata: true
|
|
42
|
+
=> "<bibdata type="standard" schema-version="v1.4.1">
|
|
43
|
+
<fetched>2026-02-27</fetched>
|
|
44
|
+
<title language="en" script="Latn" type="title-main">Development of Spatial Data Infrastructures for Marine Data Management</title>
|
|
45
|
+
<title language="en" script="Latn" type="main">Development of Spatial Data Infrastructures for Marine Data Management</title>
|
|
46
|
+
<uri type="src">http://www.opengis.net/doc/per/marine-sdi</uri>
|
|
47
|
+
<uri type="pdf">https://portal.ogc.org/files/?artifact_id=88037</uri>
|
|
48
|
+
<docidentifier type="OGC" primary="true">19-025r1</docidentifier>
|
|
49
|
+
...
|
|
50
|
+
<ext schema-version="v1.0.1">
|
|
51
|
+
<doctype>engineering-report</doctype>
|
|
52
|
+
<flavor>ogc</flavor>
|
|
53
|
+
</ext>
|
|
54
|
+
</bibdata>"
|
|
55
|
+
----
|
|
56
|
+
|
|
57
|
+
=== Get code, and year
|
|
58
|
+
[source,ruby]
|
|
59
|
+
----
|
|
60
|
+
Relaton::Ogc::Bibliography.get "OGC 19-025r1", "2019"
|
|
61
|
+
[relaton-ogc] INFO: (OGC 19-025r1) Fetching from Relaton repository ...
|
|
62
|
+
[relaton-ogc] INFO: (OGC 19-025r1) Found: `19-025r1`
|
|
63
|
+
=> #<Relaton::Ogc::ItemData:0x00000001283d6f48
|
|
64
|
+
...
|
|
65
|
+
----
|
|
66
|
+
|
|
67
|
+
=== Create bibliographic item from XML
|
|
68
|
+
[source,ruby]
|
|
69
|
+
----
|
|
70
|
+
Relaton::Ogc::Item.from_xml File.read('spec/fixtures/12_128r14.xml')
|
|
71
|
+
=> #<Relaton::Ogc::ItemData:0x000000012835e610
|
|
72
|
+
...
|
|
73
|
+
----
|
|
74
|
+
|
|
75
|
+
=== Typed links
|
|
76
|
+
|
|
77
|
+
Each OGC document has `scr` type source link.
|
|
78
|
+
|
|
79
|
+
[source,ruby]
|
|
80
|
+
----
|
|
81
|
+
item.source[0].type
|
|
82
|
+
=> "obp"
|
|
83
|
+
|
|
84
|
+
item.source[0].content
|
|
85
|
+
=> "http://www.opengis.net/doc/per/marine-sdi"
|
|
86
|
+
----
|
|
87
|
+
|
|
88
|
+
=== Create bibliographic item from YAML
|
|
89
|
+
[source,ruby]
|
|
90
|
+
----
|
|
91
|
+
Relaton::Ogc::Item.from_yaml File.read('spec/fixtures/item.yaml')
|
|
92
|
+
=> #<Relaton::Ogc::ItemData:0x0000000124b19d90
|
|
93
|
+
...
|
|
94
|
+
----
|
|
95
|
+
|
|
96
|
+
=== Fetch data
|
|
97
|
+
|
|
98
|
+
This gem uses the https://raw.githubusercontent.com/opengeospatial/NamingAuthority/master/definitions/docs/docs.json[OGC NamingAuthority] dataset as a data source.
|
|
99
|
+
|
|
100
|
+
The method `Relaton::Ogc::DataFetcher.fetch(output: "data", format: "yaml")` fetches all the documents from the dataset and saves them to the `./data` folder in YAML format.
|
|
101
|
+
Arguments:
|
|
102
|
+
|
|
103
|
+
- `output` - folder to save documents (default './data').
|
|
104
|
+
- `format` - the format in which the documents are saved. Possible formats are: `yaml`, `xml`, `bibxml` (default `yaml`).
|
|
105
|
+
|
|
106
|
+
[source,ruby]
|
|
107
|
+
----
|
|
108
|
+
require "relaton/ogc/data_fetcher"
|
|
109
|
+
|
|
110
|
+
Relaton::Ogc::DataFetcher.fetch
|
|
111
|
+
Started at: 2026-02-27 11:21:46 +0200
|
|
112
|
+
Stopped at: 2026-02-27 11:21:48 +0200
|
|
113
|
+
=> nil
|
|
114
|
+
----
|
|
115
|
+
|
|
116
|
+
=== Logging
|
|
117
|
+
|
|
118
|
+
Relaton::Ogc uses the relaton-logger gem for logging. By default, it logs to STDOUT. To change the log levels and add other loggers, read the https://github.com/relaton/relaton-logger#usage[relaton-logger] documentation.
|
|
119
|
+
|
|
120
|
+
== Development
|
|
121
|
+
|
|
122
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `bundle exec rspec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
123
|
+
|
|
124
|
+
To update the index test fixture (used by tests), run `rake spec:update_index`. This downloads the latest `index-v1.zip` from the https://github.com/relaton/relaton-data-ogc[relaton-data-ogc] repository.
|
|
125
|
+
|
|
126
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
127
|
+
|
|
128
|
+
== Contributing
|
|
129
|
+
|
|
130
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/relaton/relaton-ogc.
|
|
131
|
+
|
|
132
|
+
== License
|
|
133
|
+
|
|
134
|
+
The gem is available as open source under the terms of the https://opensource.org/licenses/MIT[MIT License].
|
|
@@ -46,9 +46,13 @@ module Relaton
|
|
|
46
46
|
|
|
47
47
|
def fetch_doc(hit)
|
|
48
48
|
return if hit["type"] == "CC"
|
|
49
|
+
# The list entry carries the whole record payload, so an unchanged
|
|
50
|
+
# entry skips the parse and the write (the digest ledger).
|
|
51
|
+
return if unchanged?(hit["identifier"], hit)
|
|
49
52
|
|
|
50
53
|
bib = Scraper.parse_page hit, @errors
|
|
51
|
-
write_document bib
|
|
54
|
+
file = write_document bib
|
|
55
|
+
note_digest(hit["identifier"], hit, file: file) if file
|
|
52
56
|
true
|
|
53
57
|
rescue StandardError => e
|
|
54
58
|
Util.error "Fetching document: #{hit['identifier']}\n" \
|
|
@@ -56,17 +60,19 @@ module Relaton
|
|
|
56
60
|
false
|
|
57
61
|
end
|
|
58
62
|
|
|
63
|
+
# @return [String, nil] the file written, or nil for a duplicate
|
|
59
64
|
def write_document(bib) # rubocop:disable Metrics/AbcSize
|
|
60
65
|
docid = bib.docidentifier[0].content
|
|
61
66
|
if @docids.include?(docid)
|
|
62
67
|
@dupids << docid
|
|
63
|
-
return
|
|
68
|
+
return nil
|
|
64
69
|
end
|
|
65
70
|
|
|
66
71
|
@docids << docid
|
|
67
72
|
file = file_name bib
|
|
68
73
|
index.add_or_update index_id(docid), file
|
|
69
|
-
|
|
74
|
+
force_write file, serialize(bib)
|
|
75
|
+
file
|
|
70
76
|
end
|
|
71
77
|
|
|
72
78
|
#
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## Project Overview
|
|
6
|
+
|
|
7
|
+
relaton-oiml retrieves International Organization of Legal Metrology (OIML) publications using the Relaton model. It is **index-backed** (no scraping): it searches a pre-built index via `Relaton::Index` and fetches per-document YAML from the `relaton/relaton-data-oiml` GitHub repo. OIML reference parsing uses `Pubid::Oiml::Identifier`.
|
|
8
|
+
|
|
9
|
+
## Development
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
bundle exec rake # default task → rspec
|
|
13
|
+
bundle exec rspec spec/relaton/oiml/bibliography_spec.rb # single file
|
|
14
|
+
bundle exec rake spec:update_index # refresh the OIML index fixture from relaton-data-oiml
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
(No gem-local rubocop config; the monorepo root config applies.)
|
|
18
|
+
|
|
19
|
+
## Architecture
|
|
20
|
+
|
|
21
|
+
Namespace: `Relaton::Oiml`. Retrieval flow:
|
|
22
|
+
|
|
23
|
+
1. **Bibliography** (`lib/relaton/oiml/bibliography.rb`) — `get(code, year)` / `search`; parses the reference with `Pubid::Oiml.parse`, looks it up in the index, and fetches the matching YAML. `get` suppresses the edition year for an **undated** citation (mirroring ISO): when the reference carries no year (and `opts[:keep_year]` is not set) it returns `item.to_most_recent_reference`, so `OIML B 18` renders undated, while a dated citation (`OIML B 18:2022`, or an explicit `year`) still pins that edition. This relies on `Oiml::Docidentifier#remove_date!` refreshing `content` from the dateless pubid — mutating the wrapped `@pubid` alone leaves the rendered `content` dated (issue #72).
|
|
24
|
+
2. **Index** — `Relaton::Index.find_or_create(:oiml, url: ..., pubid_class: Pubid::Oiml::Identifier)`. `INDEXFILE` is defined in `lib/relaton/oiml.rb`. `Bibliography#best_row` matches a **Bulletin** exactly (`index.search(query, exact: true)`), and every other type by `pubid_match?`: the year-and-language-stripped "stem" is equal, the language is equal, and the year is equal when the reference or the `year` argument names one; the latest edition wins. A Bulletin needs the exact match because its year is part of its locator: the issue and the sequence render after the year, so the stem reduced every article of one year to `OIML Bulletin`, and a lookup returned an arbitrary one of them.
|
|
25
|
+
|
|
26
|
+
**OIML does not use pubid's subset match `===` yet**, unlike the other flavors that pubid#408 unblocked. Measured over the full `relaton-data-oiml` index (5,646 rows), `===` disagrees with `pubid_match?` outside the Bulletins on 192 pairs, and each one is a regression: `language` is not `subset_strict`, so a language-less Amendment, Annex or Errata reference reaches its translations (156 pairs); `subpart` is not strict, so `OIML R 137-1 (F)` reaches `OIML R 137-1-2:2012 (F)` (6); and the `year_on_base` render flag is compared, so `OIML R 102 Annex B-C` misses `OIML R 102:1995 Annex B-C` (30). A plain Recommendation escapes the first one only because its render hint `parsed_format` differs, which is luck. The hand-off `metanorma__pubid__oiml-subset-match-strict-language` asks pubid for the fix. `spec/oiml/relaton/oiml/bibliography_spec.rb` pins all of these with an in-memory index ("where the subset match would disagree"), so a later switch must pass them.
|
|
27
|
+
|
|
28
|
+
**Dual-published identifiers** (relaton#180, pubid#437): OIML sometimes co-publishes a document jointly with another SDO (ISO confirmed so far), printed with both identifiers joined by a bare `|`, e.g. `ISO 4064-1:2024|OIML R 49-1:2024`. pubid models this as `Pubid::Oiml::Identifiers::DualPublished`, which wraps `first`/`second` (each a real, independently-typed `::Pubid::Identifier`, in original print order) and delegates `#root`/`#code`/`#type`/`#stage`/`#iteration`/`#publisher`/the `mr_*` hooks to whichever side `#oiml_identifier` finds — but deliberately **not** `#year`, `#date`, `#part`, or `#language`: reading or mutating those directly on the wrapper is a silent no-op (the wrapper's own copies of those attributes are simply never populated). Only the generic, immutable `#exclude(:attr)` correctly recurses into both sides, because it walks the wrapper's declared attributes and, for the two that are nested identifiers (`first`/`second`), recurses into each.
|
|
29
|
+
|
|
30
|
+
Because `#root` already delegates to the OIML side, `Relaton::Index` needs no change — a dual-published row buckets under its OIML number exactly like a plain one. The gap was in `Bibliography` and `Docidentifier`, both of which read/mutate a pubid's flat attributes directly:
|
|
31
|
+
|
|
32
|
+
- `Bibliography#oiml_side(pubid)` unwraps a `DualPublished` to its OIML side (a plain pubid passes through unchanged), and `stem`/`pubid_match?`/`best_row`'s edition `max_by` all route through it before touching `.year`/`.language`. Without this, a plain query's stem (`"OIML R 49-1"`) can never equal a dual-published row's stem (`"ISO 4064-1|OIML R 49-1"` at full string), and `.year`/`.language` read directly off a `DualPublished` are always nil/no-op — silently breaking edition selection and language matching. Reducing both sides to the OIML identifier also makes stem comparison print-order-independent (`OIML R 49-1|ISO 4064-1` and `ISO 4064-1|OIML R 49-1` match identically).
|
|
33
|
+
- `Docidentifier#remove_date!`/`#remove_part!` compute the mutated string via `@pubid.exclude(:date)`/`@pubid.exclude(:part)` and reassign through `content=` (which reparses and resyncs `@pubid`), instead of the old `@pubid.date = nil`/`@pubid.part = nil` in-place mutation — the in-place form is indistinguishable from success (no exception) but silently does nothing on a `DualPublished`. `#exclude` behaves identically to the old in-place mutation for a plain (non-dual) pubid, so this is a safe rewrite, not just a dual-published-only branch.
|
|
34
|
+
|
|
35
|
+
No second `docidentifier` array entry is added for the external (ISO) side — pubid's `DualPublished` already exposes both sides structurally off the one docidentifier's `pubid` (`oiml_identifier`/`external_identifier`), so a consumer that needs the ISO side reads it from there.
|
|
36
|
+
3. **Item / ItemData / Ext** (`item.rb`, `item_data.rb`, `ext.rb`) — `Item` extends `Bib::Item`; `Ext` carries OIML-specific fields (scope, quantity, measuring_instrument, focus_area, sustainability_framework, doi). `Item.from_yaml` deserializes the fetched document.
|
|
37
|
+
4. **Processor** (`lib/relaton/oiml/processor.rb`) — registry integration; `@prefix = "OIML"`, `@defaultprefix = %r{^OIML\s}`. Lazy-`require_relative`s `../oiml` in its methods, including `remove_index_file`.
|
|
38
|
+
|
|
39
|
+
There are no scrapers — everything comes from the curated index + GitHub YAML.
|
|
40
|
+
|
|
41
|
+
## External dependencies
|
|
42
|
+
|
|
43
|
+
`pubid ~> 2.0.0.pre.alpha.3`, `base64`, `relaton-core`, `relaton-bib`, `relaton-index`.
|
|
44
|
+
|
|
45
|
+
## Testing
|
|
46
|
+
|
|
47
|
+
RSpec with WebMock + VCR. The offline index fixture `spec/fixtures/index-v2.zip` is pre-loaded into the `Relaton::Index` pool in `before(:suite)` (configured in `spec/support/webmock.rb`); per-document data requests are stubbed by WebMock and served from `spec/fixtures/data/*.yaml`. Run `rake spec:update_index` to refresh the fixture.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
= Relaton OIML
|
|
2
|
+
|
|
3
|
+
RelatonOiml is a library to fetch bibliographic items for OIML
|
|
4
|
+
(International Organization of Legal Metrology) publications from the
|
|
5
|
+
https://github.com/relaton/relaton-data-oiml[relaton-data-oiml] repository.
|
|
6
|
+
The result is a Relaton bibliographic item that can be serialised to Relaton
|
|
7
|
+
XML or YAML.
|
|
8
|
+
|
|
9
|
+
It is a part of the https://github.com/relaton/relaton[Relaton] project.
|
|
10
|
+
|
|
11
|
+
== Supported document types
|
|
12
|
+
|
|
13
|
+
OIML publications are identified with https://github.com/metanorma/pubid[pubid]
|
|
14
|
+
(`Pubid::Oiml`). The following document types are covered:
|
|
15
|
+
|
|
16
|
+
* Recommendations (`R`)
|
|
17
|
+
* Documents (`D`)
|
|
18
|
+
* Guides (`G`)
|
|
19
|
+
* Vocabularies (`V`)
|
|
20
|
+
* Basic Publications (`B`)
|
|
21
|
+
* Expert Reports (`E`)
|
|
22
|
+
* Seminar Reports (`S`)
|
|
23
|
+
|
|
24
|
+
== Usage
|
|
25
|
+
|
|
26
|
+
=== Search for a standard using keywords
|
|
27
|
+
|
|
28
|
+
[source,ruby]
|
|
29
|
+
----
|
|
30
|
+
require "relaton/oiml"
|
|
31
|
+
|
|
32
|
+
item = Relaton::Oiml::Bibliography.get "OIML R 138"
|
|
33
|
+
# [relaton-oiml] INFO: (OIML R 138) Fetching from Relaton repository ...
|
|
34
|
+
# [relaton-oiml] INFO: (OIML R 138) Found: `OIML R 138:2007`
|
|
35
|
+
=> #<Relaton::Oiml::Item:0x...>
|
|
36
|
+
|
|
37
|
+
item.docidentifier.first.content
|
|
38
|
+
=> "OIML R 138:2007"
|
|
39
|
+
----
|
|
40
|
+
|
|
41
|
+
A specific edition and language can be requested:
|
|
42
|
+
|
|
43
|
+
[source,ruby]
|
|
44
|
+
----
|
|
45
|
+
Relaton::Oiml::Bibliography.get "OIML R 138:2007 (E)"
|
|
46
|
+
----
|
|
47
|
+
|
|
48
|
+
=== Serialization
|
|
49
|
+
|
|
50
|
+
[source,ruby]
|
|
51
|
+
----
|
|
52
|
+
item.to_xml bibdata: true
|
|
53
|
+
item.to_yaml
|
|
54
|
+
----
|
|
55
|
+
|
|
56
|
+
== OIML-specific metadata
|
|
57
|
+
|
|
58
|
+
Enhanced OIML Recommendations carry additional structured metadata exposed via
|
|
59
|
+
the bibliographic item extension (`item.ext`):
|
|
60
|
+
|
|
61
|
+
* `scope`
|
|
62
|
+
* `quantity`
|
|
63
|
+
* `measuring_instrument`
|
|
64
|
+
* `focus_area`
|
|
65
|
+
* `sustainability_framework`
|
|
66
|
+
* `doi`
|
|
67
|
+
|
|
68
|
+
== License
|
|
69
|
+
|
|
70
|
+
Copyright Ribose. BSD 2-clause licensed.
|