relaton 3.1.0 → 3.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.adoc +5 -3
- data/lib/relaton/3gpp.rb +1 -1
- data/lib/relaton/bib/model/docidentifier.rb +66 -5
- data/lib/relaton/calconnect/hit_collection.rb +1 -1
- data/lib/relaton/cli/command.rb +6 -4
- data/lib/relaton/cli/frontend_assets.rb +7 -6
- data/lib/relaton/db.rb +31 -14
- data/lib/relaton/ecma/bibliography.rb +2 -2
- data/lib/relaton/etsi/data_fetcher.rb +1 -1
- data/lib/relaton/iala/bibliography.rb +1 -1
- data/lib/relaton/iana.rb +1 -1
- data/lib/relaton/iec/bibliography.rb +32 -20
- data/lib/relaton/iec/data_parser.rb +41 -2
- data/lib/relaton/itu/README.adoc +1 -1
- data/lib/relaton/itu/data_crawler_r.rb +1 -1
- data/lib/relaton/oasis/bibliography.rb +1 -1
- data/lib/relaton/ogc/hit_collection.rb +1 -1
- data/lib/relaton/ogc.rb +1 -1
- data/lib/relaton/plateau/handbook_parser.rb +31 -2
- data/lib/relaton/plateau/hit_collection.rb +1 -1
- data/lib/relaton/render.rb +17 -0
- data/lib/relaton/sae/data_fetcher.rb +5 -0
- data/lib/relaton/sae.rb +1 -1
- data/lib/relaton/version.rb +1 -1
- data/lib/relaton/w3c/bibliography.rb +1 -1
- data/lib/relaton/xsf/bibliography.rb +1 -1
- data/lib/relaton/xsf.rb +1 -1
- data/lib/relaton-render.rb +4 -10
- metadata +33 -46
- data/lib/relaton/3gpp/CLAUDE.md +0 -218
- data/lib/relaton/adobe/CLAUDE.md +0 -104
- data/lib/relaton/bib/CLAUDE.md +0 -150
- data/lib/relaton/bipm/CLAUDE.md +0 -187
- data/lib/relaton/bsi/CLAUDE.md +0 -35
- data/lib/relaton/calconnect/CLAUDE.md +0 -177
- data/lib/relaton/ccsds/CLAUDE.md +0 -74
- data/lib/relaton/cen/CLAUDE.md +0 -219
- data/lib/relaton/cie/CLAUDE.md +0 -90
- data/lib/relaton/core/CLAUDE.md +0 -140
- data/lib/relaton/db/CLAUDE.md +0 -385
- data/lib/relaton/doi/CLAUDE.md +0 -105
- data/lib/relaton/easc/CLAUDE.md +0 -85
- data/lib/relaton/ecma/CLAUDE.md +0 -217
- data/lib/relaton/etsi/CLAUDE.md +0 -219
- data/lib/relaton/evs/CLAUDE.md +0 -120
- data/lib/relaton/gb/CLAUDE.md +0 -47
- data/lib/relaton/gost/CLAUDE.md +0 -100
- data/lib/relaton/iala/CLAUDE.md +0 -97
- data/lib/relaton/iana/CLAUDE.md +0 -105
- data/lib/relaton/iec/CLAUDE.md +0 -13
- data/lib/relaton/ieee/CLAUDE.md +0 -202
- data/lib/relaton/ietf/CLAUDE.md +0 -382
- data/lib/relaton/iho/CLAUDE.md +0 -76
- data/lib/relaton/index/CLAUDE.md +0 -208
- data/lib/relaton/isbn/CLAUDE.md +0 -47
- data/lib/relaton/iso/CLAUDE.md +0 -38
- data/lib/relaton/itu/CLAUDE.md +0 -685
- data/lib/relaton/jcgm/CLAUDE.md +0 -107
- data/lib/relaton/jis/CLAUDE.md +0 -64
- data/lib/relaton/logger/CLAUDE.md +0 -37
- data/lib/relaton/nist/CLAUDE.md +0 -194
- data/lib/relaton/oasis/CLAUDE.md +0 -251
- data/lib/relaton/ogc/CLAUDE.md +0 -242
- data/lib/relaton/oiml/CLAUDE.md +0 -47
- data/lib/relaton/omg/CLAUDE.md +0 -137
- data/lib/relaton/plateau/CLAUDE.md +0 -126
- data/lib/relaton/sae/CLAUDE.md +0 -89
- data/lib/relaton/sdo/CLAUDE.md +0 -81
- data/lib/relaton/un/CLAUDE.md +0 -91
- data/lib/relaton/w3c/CLAUDE.md +0 -369
- data/lib/relaton/xsf/CLAUDE.md +0 -189
data/lib/relaton/index/CLAUDE.md
DELETED
|
@@ -1,208 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
relaton-index is a Ruby gem that provides indexing and searching of Relaton document references. It maps document identifiers to file paths, supporting both local index creation (for publishing) and remote index consumption (downloading from URLs with 24-hour caching).
|
|
8
|
-
|
|
9
|
-
## Commands
|
|
10
|
-
|
|
11
|
-
```bash
|
|
12
|
-
# Run all tests (default rake task)
|
|
13
|
-
rake spec
|
|
14
|
-
|
|
15
|
-
# Run linting
|
|
16
|
-
rake rubocop
|
|
17
|
-
|
|
18
|
-
# Run specific test file
|
|
19
|
-
bundle exec rspec spec/relaton/type_spec.rb
|
|
20
|
-
|
|
21
|
-
# Run specific test by name
|
|
22
|
-
bundle exec rspec spec/relaton/file_io_spec.rb -e "fetch_and_save"
|
|
23
|
-
|
|
24
|
-
# Install dependencies
|
|
25
|
-
bin/setup
|
|
26
|
-
|
|
27
|
-
# Interactive console
|
|
28
|
-
bin/console
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
## Architecture
|
|
32
|
-
|
|
33
|
-
### Core Classes (all under `Relaton::Index` module in `lib/relaton/index/`)
|
|
34
|
-
|
|
35
|
-
- **`Relaton::Index`** (module, `lib/relaton/index.rb`) — Static API entry point. Delegates to Pool and Config. Main methods: `find_or_create`, `close`, `configure`.
|
|
36
|
-
|
|
37
|
-
- **Pool** — Object pool that caches Type instances by document type (`:ISO`, `:IEC`, `:IHO`, etc.). Reuses existing indexes if parameters match, recreates if they change.
|
|
38
|
-
|
|
39
|
-
- **Type** — Represents one index for a document type. Holds an array of `{id:, file:}` hashes. Provides `add_or_update`, `search` (subset match, string substring match, or block), and `save`.
|
|
40
|
-
|
|
41
|
-
- **FileIO** — Handles reading/writing/downloading index files. Three modes based on `@url`: string URL (download and cache to `~/.relaton/{type}/`), `true` (read local file from `~/.relaton/{type}/`), `nil` (read from current directory). Uses class-level Mutex for thread-safe downloads. Validates index format on load.
|
|
42
|
-
|
|
43
|
-
- **ShardSource** (`shard_source.rb`) — Reads the machine index on a data repo's Pages site (relaton#189): `index/manifest.json`, then the one `index/shard-NNNNN.json` at `Zlib.crc32(id.root.number.to_s) % shards`. Memory only, 24 h TTL, one `Mutex`. Built by `Type` when `find_or_create` gets `pages_url:`. Reuses `FileIO#deserialize_id`/`#deserialize_pubid` through a file-less `FileIO`.
|
|
44
|
-
|
|
45
|
-
- **FileStorage** — Storage abstraction module with `ctime`, `read`, `write`, `remove`. Can be replaced via `Config.storage=` for custom backends (e.g., S3).
|
|
46
|
-
|
|
47
|
-
- **Config** — Global configuration: `storage`, `storage_dir`, `filename` (default: "index.yaml").
|
|
48
|
-
|
|
49
|
-
### Data Flow
|
|
50
|
-
|
|
51
|
-
1. `Relaton::Index.find_or_create(:TYPE, url:, file:, pubid_class:, pages_url:)` → Pool looks up or creates Type (`id_keys:` is accepted for one release with a deprecation warning, then ignored)
|
|
52
|
-
2. Type lazily loads index via FileIO on first access
|
|
53
|
-
3. FileIO either reads local YAML or downloads ZIP from URL, extracts, validates format
|
|
54
|
-
4. Search matches against `:id` field (string comparison via `include?` or custom block)
|
|
55
|
-
5. `save` writes index as YAML to local file
|
|
56
|
-
|
|
57
|
-
With `pages_url:` (#189; W3C piloted it, and every indexed flavor's consumer
|
|
58
|
-
now uses it except JIS, NIST and Plateau, which still read the zip) steps 2–3 change: a non-String query
|
|
59
|
-
goes to `ShardSource#rows` and reads one shard, and `Type#index` (a String query,
|
|
60
|
-
a block-only search) is `ShardSource#whole_index`, the `<index>.zip` that the
|
|
61
|
-
manifest names, read from the Pages site. No `FileIO#read`, no disk cache.
|
|
62
|
-
|
|
63
|
-
### What `#search` matches by default
|
|
64
|
-
|
|
65
|
-
Search is narrow, then match. The **match** step, for two identifiers, is
|
|
66
|
-
pubid's asymmetric subset match `id === item[:id]` (`Pubid::SubsetMatch`): the
|
|
67
|
-
query is the reference, and a component it leaves nil or empty matches any
|
|
68
|
-
value. That is what a search by reference means, so a flavor whose optional
|
|
69
|
-
components are true wildcards passes **no block** — OASIS, W3C, XSF, IALA,
|
|
70
|
-
OGC, ECMA, 3GPP, CalConnect, GOST and Plateau each call
|
|
71
|
-
`index.search(pubid)`. A component that a flavor reads as "none" when nil is
|
|
72
|
-
declared `subset_strict` in pubid (pubid#408), so `===` compares it exactly.
|
|
73
|
-
|
|
74
|
-
A nil is a wildcard here, not "the document has none". A caller that needs
|
|
75
|
-
**exact** equality says so with `search(id, exact: true)`, which selects the
|
|
76
|
-
rows with `item[:id] == id`. The binary-search narrowing still runs first.
|
|
77
|
-
These callers do:
|
|
78
|
-
|
|
79
|
-
| caller | why |
|
|
80
|
-
|---|---|
|
|
81
|
-
| `Relaton::Ccsds::HitCollection#rows` (edition branch) | pubid's CCSDS `suffix` is a wildcard, so `CCSDS 101.0-B-4` would also reach the historical `101.0-B-4-S` — 260 such pairs in the index fixture. (The omitted language reached the translations too, until pubid#408 made `language` `subset_strict`.) |
|
|
82
|
-
| `Relaton::Ietf::Scraper#fetch_doc` | a draft slug omits the version, so `draft-foo` would match every `draft-foo-NN` and `.first` would pick one — 20,513 such pairs in the first 40k fixture rows |
|
|
83
|
-
| `Relaton::Gost::Bibliography#rows` (dated citation) | a dated citation names one edition; it replaces the old `query.matches?(row)` with no ignore list, which is `==` |
|
|
84
|
-
| `Relaton::Plateau::HitCollection#find` (reference with an edition) | a reference with an edition names that one row, as the old `row[:id] == ref` block did |
|
|
85
|
-
| `Relaton::Oiml::Bibliography#best_row` (a Bulletin) | a Bulletin's year, issue and sequence are its locator, so `OIML Bulletin 1960` is the volume and not every article of 1960 |
|
|
86
|
-
|
|
87
|
-
A String query keeps the substring match it always had, in both directions;
|
|
88
|
-
with `exact: true` it is compared with `==`. A block is a custom predicate for
|
|
89
|
-
any other rule, and `search` raises `ArgumentError` if it gets both a block and
|
|
90
|
-
`exact: true`.
|
|
91
|
-
|
|
92
|
-
### The narrowing key — one expression, its call sites
|
|
93
|
-
|
|
94
|
-
Search is two-stage: narrow, then match. Narrowing binary-searches the index for
|
|
95
|
-
the run of entries whose **base document number** equals the query's, using the key
|
|
96
|
-
`id.root.number.to_s` (`#root` walks a supplement/amendment `.base` chain, so a
|
|
97
|
-
document and its wrappers share one key and cluster together).
|
|
98
|
-
|
|
99
|
-
**Narrowing only happens for non-String queries on a `pubid_class:` index** —
|
|
100
|
-
`search_candidates` loads the raw side first (that load is what settles
|
|
101
|
-
`@file_io.sorted`), then narrows. A String query scans the whole index
|
|
102
|
-
*and* matches via `item[:id].to_s.include?(id)`, which renders every pubid in it.
|
|
103
|
-
So a flavor on a `pubid_class:` index must query with parsed identifiers; querying
|
|
104
|
-
it with strings is slower than the plain-string index it replaced.
|
|
105
|
-
|
|
106
|
-
That expression is written out in these places, and they must all agree or
|
|
107
|
-
bsearch silently returns the wrong slice:
|
|
108
|
-
|
|
109
|
-
| file | method | role |
|
|
110
|
-
|---|---|---|
|
|
111
|
-
| `type.rb` | `candidates_by_number` | the query's key |
|
|
112
|
-
| `type.rb` | `bsearch_left` / `bsearch_right` | bounds, over the **precomputed keys array** |
|
|
113
|
-
| `type.rb` | `narrowing_key` | add-time key (root-guarded for String ids) |
|
|
114
|
-
| `file_io.rb` | `deserialize_pubid` | build-time sort of the materialized rows |
|
|
115
|
-
| `file_io.rb` | `warn_unless_sorted` | sortedness check |
|
|
116
|
-
| `file_io.rb` | `sort_structured_index` / `save_raw` | save-time sort |
|
|
117
|
-
|
|
118
|
-
### Raw rows and the sidecar (relaton#242 stopgap)
|
|
119
|
-
|
|
120
|
-
`FileIO#read_raw` returns `[keys, rows]`: the rows as plain `to_hash` hashes —
|
|
121
|
-
no pubid objects — with a parallel array of precomputed root-number keys, sorted.
|
|
122
|
-
A **Marshal sidecar** (`<index>.yaml.ms`, `SIDECAR_VERSION`-stamped) holds them,
|
|
123
|
-
so a repeat load skips the YAML parse and the one-time full materialization that
|
|
124
|
-
computes the keys (objects exist only during that build; the raw hash cannot
|
|
125
|
-
walk `.base` itself). The sidecar is authoritative while it is at least as new
|
|
126
|
-
as the yaml — the yaml is not parsed until the sidecar is stale or unreadable —
|
|
127
|
-
and `save`/`remove` delete it. A parsed search materializes pubid objects only
|
|
128
|
-
for its bucket (`FileIO#materialize`); the whole graph materializes only through
|
|
129
|
-
`Type#index`, a String query, or a block. `add_or_update` stores raw rows
|
|
130
|
-
(`to_hash` when the id is a `pubid_class` instance, verbatim otherwise) with an
|
|
131
|
-
O(1) raw-side dedup by id-hash equality. Numbers: the 79,993-row ISO index is
|
|
132
|
-
~311 MB of raw rows instead of a ~675 MB object graph; only the searched bucket
|
|
133
|
-
is materialized at search time.
|
|
134
|
-
|
|
135
|
-
**Consequence for pubid flavors:** once a flavor does query with parsed ids, an
|
|
136
|
-
identifier family whose `number` is nil keys every row to `""`, so they collapse
|
|
137
|
-
into one bucket and the bsearch buys nothing. It fails *silently* — results stay
|
|
138
|
-
correct, only speed drops. IETF Internet-Drafts are the worked example (see
|
|
139
|
-
`docs/data-repository-format.adoc`, "Two obligations a pubid-backed index
|
|
140
|
-
carries"); the fix belongs upstream, giving the family a real `number`, rather than
|
|
141
|
-
special-casing the key here.
|
|
142
|
-
|
|
143
|
-
### Index Format
|
|
144
|
-
|
|
145
|
-
YAML array of hashes with `:id` (string or structured hash) and `:file` (path string). Supports backward compatibility with old string-based format and newer pubid object format. The full data-repository/index specification (schema, `:id` v1/v2 shapes, publishing + GitHub Pages contract) lives in `docs/data-repository-format.adoc` at the repo root.
|
|
146
|
-
|
|
147
|
-
### Flavor `INDEXFILE` convention
|
|
148
|
-
|
|
149
|
-
Each flavor names its published index via a single constant in its top-level
|
|
150
|
-
`lib/relaton/<flavor>.rb`:
|
|
151
|
-
|
|
152
|
-
```ruby
|
|
153
|
-
INDEXFILE = "index-vN".freeze # base name only — no extension
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
**Rules:**
|
|
157
|
-
- The constant is spelled **`INDEXFILE`** (one word, no underscore) and holds the
|
|
158
|
-
**base name without extension** — `"index-v1"`, `"index-v2"`, etc.
|
|
159
|
-
- Call sites append the extension: `"#{INDEXFILE}.yaml"` for the local file passed
|
|
160
|
-
as `file:`, `"#{INDEXFILE}.zip"` for the published artifact appended to the
|
|
161
|
-
consumer `url:`. Never embed `.yaml`/`.zip` in the constant, and never hardcode
|
|
162
|
-
`index-vN.zip` at a call site — derive it from `INDEXFILE` so a version bump is
|
|
163
|
-
a one-line change.
|
|
164
|
-
- The version number encodes the index **structure** (`v1` = plain-string `:id`,
|
|
165
|
-
`v2` = pubid-hash `:id`); bumping it lets the previous gem line keep reading the
|
|
166
|
-
old file while the new structure is published under a new name.
|
|
167
|
-
|
|
168
|
-
### What each flavor call site passes
|
|
169
|
-
|
|
170
|
-
A pubid flavor calls `find_or_create` from three places, and they do not need
|
|
171
|
-
the same arguments:
|
|
172
|
-
|
|
173
|
-
| call site | `url:` | `pubid_class:` |
|
|
174
|
-
|---|---|---|
|
|
175
|
-
| producer — `DataFetcher#index` | none (`file:` is CWD-relative) | **yes** — `FileIO#save` calls `to_hash` only for instances of it |
|
|
176
|
-
| consumer on the zip (JIS, NIST, Plateau) — `Bibliography`/`HitCollection#index` | the published `.zip` | **yes** — without it the rows stay raw hashes and `Type#search` stops narrowing |
|
|
177
|
-
| consumer on the Pages shards (all other indexed flavors) | none; **`pages_url:`** | **yes** — a shard row is deserialized with it |
|
|
178
|
-
| delete — `Processor#remove_index_file` | **`true`** | **no** |
|
|
179
|
-
|
|
180
|
-
The delete path is `Type#remove_file` → `FileIO#remove` →
|
|
181
|
-
`FileStorage.remove(file)`. `FileIO#file` is built from `url`, the type's
|
|
182
|
-
directory and `file:` only, and the delete never reads or deserializes the
|
|
183
|
-
index, so `pubid_class:` has no effect on it. `url:` does: without it `file` is
|
|
184
|
-
the bare name, so a `Db#clear` on an empty pool deletes `./index-vN.yaml` in the
|
|
185
|
-
working directory and leaves `~/.relaton/<type>/` in place. OASIS shipped that
|
|
186
|
-
bug until `url: true` was added. `spec/index/relaton/type_spec.rb` and
|
|
187
|
-
`spec/oasis/relaton/oasis/processor_spec.rb` guard both facts.
|
|
188
|
-
|
|
189
|
-
### Key Design Decisions
|
|
190
|
-
|
|
191
|
-
- **The shard path does not fall back.** A shard 404 is "not found" (`[]`), a
|
|
192
|
-
transport failure raises `Relaton::RequestError` so `Db#net_retry` retries it,
|
|
193
|
-
and an unreadable manifest/shard raises `Relaton::Index::Error`. Falling back to
|
|
194
|
-
the monolith would turn a Pages outage into a silent full download.
|
|
195
|
-
`Relaton::RequestError` is declared in `lib/relaton/core/request_error.rb`
|
|
196
|
-
(`Relaton::Core::RequestError` is an alias). `relaton/index` requires that one file,
|
|
197
|
-
not all of `relaton/core` or `relaton/bib`.
|
|
198
|
-
- **`Type#actual?` compares `pages_url`.** Without it `Pool#type` would keep
|
|
199
|
-
serving a zip-backed `Type` to a caller that asks for the shards, or the
|
|
200
|
-
reverse. A spec fixture that seeds the pool must answer `actual?` for
|
|
201
|
-
`pages_url:` as well (see `spec/w3c/support/webmock.rb`).
|
|
202
|
-
- **`Type#search` does not look at a loaded `@index` in shard mode.** A parsed
|
|
203
|
-
query always goes to its shard, so the 24 h TTL applies; `@index`, once
|
|
204
|
-
loaded, is not refreshed.
|
|
205
|
-
- Remote indexes cached for 24 hours at `~/.relaton/{type}/index.yaml`
|
|
206
|
-
- Thread safety via `@@mutex` in FileIO prevents concurrent downloads of the same file
|
|
207
|
-
- Pubid deserialization is optional — when `pubid_class` is provided, string IDs are converted to structured objects
|
|
208
|
-
- Index format validation checks for required `:id` and `:file` keys, with automatic recovery (re-download or removal) on corruption
|
data/lib/relaton/isbn/CLAUDE.md
DELETED
|
@@ -1,47 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## What this gem does
|
|
6
|
-
|
|
7
|
-
Retrieves bibliographic items from the OpenLibrary API by ISBN (10 or 13 digit) and returns `Relaton::Bib::ItemData` objects. Part of the Relaton family of gems.
|
|
8
|
-
|
|
9
|
-
## Commands
|
|
10
|
-
|
|
11
|
-
```sh
|
|
12
|
-
bundle exec rspec # run all tests
|
|
13
|
-
bundle exec rspec spec/relaton/isbn/parser_spec.rb # run single spec file
|
|
14
|
-
bundle exec rubocop # lint
|
|
15
|
-
bin/console # IRB with gem loaded
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
## Architecture
|
|
19
|
-
|
|
20
|
-
Request flow: `OpenLibrary.get(code)` → `Isbn.new(code).parse` (validates/normalizes ISBN) → `OpenLibrary.request_api(isbn)` (HTTP to openlibrary.org) → `Parser.parse(json)` (builds `ItemData`)
|
|
21
|
-
|
|
22
|
-
`get` also takes a parsed `Pubid::Isbn::Identifier`: its digits (`raw`, 10 or 13) go through the same `Isbn#parse`.
|
|
23
|
-
|
|
24
|
-
Key modules under `Relaton::Isbn`:
|
|
25
|
-
- **OpenLibrary** — API client, main entry point via `.get`
|
|
26
|
-
- **Isbn** — ISBN-10/13 validation and conversion (always normalizes to ISBN-13)
|
|
27
|
-
- **Parser** — Transforms OpenLibrary JSON into `Relaton::Bib::ItemData` with titles, contributors, dates, etc.
|
|
28
|
-
- **Processor** — Integration hook for relaton-core framework (`from_xml`, `from_yaml`, `get`)
|
|
29
|
-
- **Util** — Logging via `Relaton::Bib::Util`
|
|
30
|
-
|
|
31
|
-
## Data model (relaton-bib 2.0)
|
|
32
|
-
|
|
33
|
-
Uses `Relaton::Bib::` namespace (not the old `RelatonBib::`). Key classes: `ItemData`, `Title`, `Docidentifier` (uses `.content` not `.id`), `Uri`, `Contributor`, `Person`, `Organization`, `Date` (uses `at:` not `on:`), `Place`.
|
|
34
|
-
|
|
35
|
-
`Parser#parse` returns `Bib::ItemData` (not `Bibitem`) — `ItemData` is the plain data class; `Bibitem`/`Item` are lutaml-model serializers that wrap it.
|
|
36
|
-
|
|
37
|
-
## Testing
|
|
38
|
-
|
|
39
|
-
- **VCR** records HTTP interactions in `spec/vcr_cassettes/` (never auto-rerecorded; re-record deliberately)
|
|
40
|
-
- **WebMock** disables real HTTP by default
|
|
41
|
-
- **SimpleCov** tracks coverage
|
|
42
|
-
- Fixtures in `spec/fixtures/`
|
|
43
|
-
|
|
44
|
-
## Style
|
|
45
|
-
|
|
46
|
-
- RuboCop with Ribose OSS style guide, target Ruby 3.1
|
|
47
|
-
- `rubocop-rails` is required but Rails cops are disabled (not a Rails app)
|
data/lib/relaton/iso/CLAUDE.md
DELETED
|
@@ -1,38 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Development
|
|
6
|
-
|
|
7
|
-
- `bin/setup` — install dependencies
|
|
8
|
-
- `rake spec` — run tests
|
|
9
|
-
- `rspec spec/relaton/iso/bibliography_spec.rb` — run a single test file
|
|
10
|
-
- `rspec spec/relaton/iso/bibliography_spec.rb -e "some description"` — run a specific example
|
|
11
|
-
- `rake spec:update_index` — download latest ISO index fixture (`spec/fixtures/index-v1.zip`) from relaton-data-iso
|
|
12
|
-
- `bin/console` — interactive prompt with the gem loaded
|
|
13
|
-
- `rubocop` — lint (Ribose OSS style guide, Ruby 3.2 target)
|
|
14
|
-
|
|
15
|
-
## Architecture
|
|
16
|
-
|
|
17
|
-
relaton-iso retrieves ISO standard bibliographic data. The core retrieval flow:
|
|
18
|
-
|
|
19
|
-
1. **Bibliography** (`lib/relaton/iso/bibliography.rb`) — entry point via `search(pubid)` and `get(ref, year, opts)`. Handles year filtering, part matching, and type/stage validation. Both **propagate** `Pubid::Errors::ParseError` when the caller's reference can't be parsed by pubid (they do not swallow it — the CLI turns it into a friendly message; API consumers rescue it themselves). Note `look_up_with_any_types_stages` deliberately rescues `Pubid::Errors::ParseError` internally: it re-parses a *machine-derived* type/stage-stripped probe, so a failure there just means "no alternative-type/stage candidates" and must not abort a lookup whose original reference already parsed.
|
|
20
|
-
2. **HitCollection** (`lib/relaton/iso/hit_collection.rb`) — searches a pre-built YAML index (`index-v1.zip` from relaton-data-iso) using `Relaton::Index`. Matches ids through the parsed pubid (`reference === row`, relaton#109). Returns sorted Hit array.
|
|
21
|
-
3. **Hit** (`lib/relaton/iso/hit.rb`) — wraps an index result. The `item` attribute lazy-loads the full document from GitHub raw content (relaton-data-iso repo). `sort_weight` prioritizes published over withdrawn/deleted.
|
|
22
|
-
4. **ItemData** / **Model::Item** — ISO-specific bibliographic item extending `Relaton::Bib::ItemData`.
|
|
23
|
-
5. **Scraper** (`lib/relaton/iso/scraper.rb`) — parses individual ISO website pages. Used only by `Bibliography.get` as a fallback when an item is missing from the curated index; no longer drives bulk ingest.
|
|
24
|
-
6. **DataFetcher** (`lib/relaton/iso/data_fetcher.rb`) — streams the ISO Open Data programme JSONL feeds (`iso_deliverables_metadata.jsonl` for documents, `iso_technical_committees.jsonl` for committee titles) and writes one YAML per primary docid into `@output`. Short-circuits on upstream `Last-Modified`; falls back to a full pass when `data/` or `index-v1.yaml` is missing. Two source modes:
|
|
25
|
-
- `iso-open-data` (default) — incremental, skip when upstream is unchanged.
|
|
26
|
-
- `iso-open-data-all` — wipe `@output` and re-emit every record.
|
|
27
|
-
|
|
28
|
-
File names come from `Core::DataFetcher#unique_output_file`, not the plain `output_file`: `output_file` is not injective (`ISO/DIS 15869.3` and `ISO/DIS 15869-3` both sanitize to `iso-dis-15869-3.yaml`), so a second distinct docid gets a digest-suffixed path. `rewrite_with_same_or_newer` (edition / substage-98) therefore only ever compares two records of the **same** docid.
|
|
29
|
-
7. **DataParser** (`lib/relaton/iso/data_parser.rb`) — converts one Open Data record (`Hash`) into a `Relaton::Iso::ItemData`. Takes a `ref_index` (id → reference) for resolving `replaces`/`replacedBy` and a `tc_index` (reference → `{ "en"/"fr" => title }`) for resolving committee labels.
|
|
30
|
-
|
|
31
|
-
Key dependency: `pubid-iso` gem handles ISO publication identifier parsing and comparison.
|
|
32
|
-
|
|
33
|
-
## Testing
|
|
34
|
-
|
|
35
|
-
- **Framework:** RSpec with VCR cassettes and WebMock
|
|
36
|
-
- **Network access:** fully blocked via `WebMock.disable_net_connect!`
|
|
37
|
-
- **Index fixture:** `spec/fixtures/index-v1.zip` is pre-loaded into `Relaton::Index` pool in `before(:suite)` (configured in `spec/support/webmock.rb`). Run `rake spec:update_index` to refresh from relaton-data-iso.
|
|
38
|
-
- **VCR:** cassettes in `spec/vcr_cassettes/`, record mode `:once`, never auto-rerecorded (`re_record_interval: nil`). Index download requests are ignored by VCR (handled by fixture instead).
|