relaton 3.1.1 → 3.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.adoc +5 -3
- data/lib/relaton/3gpp.rb +1 -1
- data/lib/relaton/bib/converter/csl/from_csl.rb +195 -0
- data/lib/relaton/bib/converter/csl.rb +13 -1
- data/lib/relaton/bib/model/docidentifier.rb +66 -5
- data/lib/relaton/calconnect/hit_collection.rb +1 -1
- data/lib/relaton/cli/command.rb +3 -1
- data/lib/relaton/cli/frontend_assets.rb +7 -6
- data/lib/relaton/cli/subcommand_collection.rb +5 -1
- data/lib/relaton/core/request_error.rb +18 -0
- data/lib/relaton/db/config.rb +9 -1
- data/lib/relaton/db/workers_pool.rb +11 -1
- data/lib/relaton/db.rb +160 -58
- data/lib/relaton/ecma/bibliography.rb +2 -2
- data/lib/relaton/etsi/data_fetcher.rb +16 -6
- data/lib/relaton/iala/bibliography.rb +1 -1
- data/lib/relaton/iana.rb +1 -1
- data/lib/relaton/iec/data_parser.rb +41 -2
- data/lib/relaton/itu/README.adoc +1 -1
- data/lib/relaton/itu/data_crawler_r.rb +1 -1
- data/lib/relaton/jis/README.adoc +2 -0
- data/lib/relaton/jis/data_fetcher.rb +74 -13
- data/lib/relaton/jis/hit_collection.rb +2 -1
- data/lib/relaton/jis/processor.rb +1 -1
- data/lib/relaton/jis/scraper.rb +46 -5
- data/lib/relaton/oasis/bibliography.rb +1 -1
- data/lib/relaton/ogc/hit_collection.rb +1 -1
- data/lib/relaton/ogc.rb +1 -1
- data/lib/relaton/plateau/data_fetcher.rb +10 -1
- data/lib/relaton/plateau/handbook_parser.rb +38 -2
- data/lib/relaton/plateau/hit_collection.rb +20 -5
- data/lib/relaton/sae.rb +1 -1
- data/lib/relaton/version.rb +1 -1
- data/lib/relaton/w3c/bibliography.rb +1 -1
- data/lib/relaton/xsf/bibliography.rb +1 -1
- data/lib/relaton/xsf.rb +1 -1
- metadata +35 -48
- data/lib/relaton/3gpp/CLAUDE.md +0 -218
- data/lib/relaton/adobe/CLAUDE.md +0 -104
- data/lib/relaton/bib/CLAUDE.md +0 -150
- data/lib/relaton/bipm/CLAUDE.md +0 -187
- data/lib/relaton/bsi/CLAUDE.md +0 -35
- data/lib/relaton/calconnect/CLAUDE.md +0 -177
- data/lib/relaton/ccsds/CLAUDE.md +0 -74
- data/lib/relaton/cen/CLAUDE.md +0 -219
- data/lib/relaton/cie/CLAUDE.md +0 -90
- data/lib/relaton/core/CLAUDE.md +0 -140
- data/lib/relaton/db/CLAUDE.md +0 -385
- data/lib/relaton/doi/CLAUDE.md +0 -105
- data/lib/relaton/easc/CLAUDE.md +0 -85
- data/lib/relaton/ecma/CLAUDE.md +0 -217
- data/lib/relaton/etsi/CLAUDE.md +0 -219
- data/lib/relaton/evs/CLAUDE.md +0 -120
- data/lib/relaton/gb/CLAUDE.md +0 -47
- data/lib/relaton/gost/CLAUDE.md +0 -100
- data/lib/relaton/iala/CLAUDE.md +0 -97
- data/lib/relaton/iana/CLAUDE.md +0 -105
- data/lib/relaton/iec/CLAUDE.md +0 -13
- data/lib/relaton/ieee/CLAUDE.md +0 -202
- data/lib/relaton/ietf/CLAUDE.md +0 -382
- data/lib/relaton/iho/CLAUDE.md +0 -76
- data/lib/relaton/index/CLAUDE.md +0 -208
- data/lib/relaton/isbn/CLAUDE.md +0 -47
- data/lib/relaton/iso/CLAUDE.md +0 -38
- data/lib/relaton/itu/CLAUDE.md +0 -685
- data/lib/relaton/jcgm/CLAUDE.md +0 -107
- data/lib/relaton/jis/CLAUDE.md +0 -64
- data/lib/relaton/logger/CLAUDE.md +0 -37
- data/lib/relaton/nist/CLAUDE.md +0 -194
- data/lib/relaton/oasis/CLAUDE.md +0 -251
- data/lib/relaton/ogc/CLAUDE.md +0 -242
- data/lib/relaton/oiml/CLAUDE.md +0 -47
- data/lib/relaton/omg/CLAUDE.md +0 -137
- data/lib/relaton/plateau/CLAUDE.md +0 -126
- data/lib/relaton/sae/CLAUDE.md +0 -91
- data/lib/relaton/sdo/CLAUDE.md +0 -81
- data/lib/relaton/un/CLAUDE.md +0 -91
- data/lib/relaton/w3c/CLAUDE.md +0 -369
- data/lib/relaton/xsf/CLAUDE.md +0 -189
data/lib/relaton/easc/CLAUDE.md
DELETED
|
@@ -1,85 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with
|
|
4
|
-
the EASC flavor. It is a dev doc — excluded from the packaged gem via the
|
|
5
|
-
gemspec `files` glob.
|
|
6
|
-
|
|
7
|
-
## What this flavor is
|
|
8
|
-
|
|
9
|
-
`Relaton::Easc` retrieves **EASC** (Eurasian Economic Standards Council; the
|
|
10
|
-
Cyrillic МГС council) publications using the Relaton model. It
|
|
11
|
-
is **index-backed** (no scraping): it searches a pre-built index via
|
|
12
|
-
`Relaton::Index` and fetches per-document YAML from the
|
|
13
|
-
`relaton/relaton-data-easc` GitHub repo. Reference parsing uses
|
|
14
|
-
`Pubid::Easc::Identifier`.
|
|
15
|
-
|
|
16
|
-
Two document series, both issued by EASC and catalogued on **mgscatalog.by**:
|
|
17
|
-
|
|
18
|
-
- **ПМГ** (`pmg`) — Правила по межгосударственной стандартизации (interstate
|
|
19
|
-
standardization *rules*), e.g. `ПМГ 03-2025`, plus the defense variant
|
|
20
|
-
`ПМГ В 31-2001`.
|
|
21
|
-
- **РМГ** (`rmg`) — Рекомендации по межгосударственной стандартизации
|
|
22
|
-
(interstate standardization *recommendations*), e.g. `РМГ 151-2025`.
|
|
23
|
-
|
|
24
|
-
## Architecture
|
|
25
|
-
|
|
26
|
-
Namespace: `Relaton::Easc`. Retrieval flow:
|
|
27
|
-
|
|
28
|
-
1. **Bibliography** (`lib/relaton/easc/bibliography.rb`) — `get(code, year)` /
|
|
29
|
-
`search`; parses the reference with `Pubid::Easc.parse`, narrows the index by
|
|
30
|
-
number, fetches the matching YAML over `Net::HTTP`, and returns
|
|
31
|
-
`Item.from_yaml` (an `ItemData`), stamping `fetched`. EASC citations are
|
|
32
|
-
always dated (`ПМГ 03-2025`), so — unlike ISO/OIML — there is no
|
|
33
|
-
undated/most-recent handling; `get` just returns what `search` resolves.
|
|
34
|
-
`ENDPOINT` is the `relaton-data-easc/main` raw base; `INDEXFILE` (`index-v2`)
|
|
35
|
-
is defined in `lib/relaton/easc.rb`.
|
|
36
|
-
2. **Index keying** — `Relaton::Index.find_or_create(:easc, url: ...,
|
|
37
|
-
pubid_class: ::Pubid::Easc::Identifier)`. `Pubid::Easc` (from
|
|
38
|
-
`metanorma/pubid`) parses both the Cyrillic surface forms and the Latin
|
|
39
|
-
transliterations (`PMG`/`RMG`/`V`) and renders canonical Cyrillic; its
|
|
40
|
-
attributes are `series` (`"PMG"|"RMG"`, Latin-canonical), `variant`
|
|
41
|
-
(`"V"|nil` — the «В» defense marker), `number`, `year`, and it serializes to
|
|
42
|
-
`_type: pubid:easc:{pmg,rmg}`. `Bibliography#pubid_match?` matches on
|
|
43
|
-
series + variant + number (nil-tolerant year), so ПМГ never matches РМГ and
|
|
44
|
-
the «В» variant never matches the plain series.
|
|
45
|
-
3. **Item / ItemData / Ext** (`item.rb`, `item_data.rb`, `ext.rb`,
|
|
46
|
-
`item_base.rb`, `bibitem.rb`, `bibdata.rb`) — `Item` extends `Bib::Item`
|
|
47
|
-
with the typed `Ext` subclass; `Bibitem`/`Bibdata` alias it. `Ext` carries
|
|
48
|
-
the EASC-specific fields so they round-trip natively through both XML and
|
|
49
|
-
YAML (no per-repo merge hacks): `urn` (`urn:easc:<series>:...`), `webpage`
|
|
50
|
-
(mgscatalog.by detail URL), `session` (adopting МГС council session, e.g.
|
|
51
|
-
`67МГС`), `developer` (Разработчик), `joining_states` (collection of country
|
|
52
|
-
codes, e.g. `%w[АРМ БЕИ КАЗ]`), and `assigned_to` (Закреплен за).
|
|
53
|
-
4. **Docidentifier** (`docidentifier.rb`) — a plain `Bib::Docidentifier`
|
|
54
|
-
subclass; the `content` is the canonical Cyrillic citation (`ПМГ 03-2025`).
|
|
55
|
-
Subclassed so future `#pubid` integration can hook in without changing the
|
|
56
|
-
public interface.
|
|
57
|
-
5. **Doctype** (`doctype.rb`) — `TYPES = %w[pmg rmg]`.
|
|
58
|
-
6. **Processor** (`lib/relaton/easc/processor.rb`) — registry integration;
|
|
59
|
-
`@prefix = "EASC"`, `@idtype = "EASC"`, and `@defaultprefix =
|
|
60
|
-
%r{^(?:ПМГ|РМГ|PMG|RMG)\b}` routes **both** Cyrillic and Latin series
|
|
61
|
-
prefixes here. `get` delegates to `Bibliography.get`. Per the lazy-registry
|
|
62
|
-
invariant, each method `require_relative "../easc"` before touching a flavor
|
|
63
|
-
constant, including `remove_index_file`.
|
|
64
|
-
|
|
65
|
-
There are no scrapers — everything comes from the curated index + GitHub YAML.
|
|
66
|
-
|
|
67
|
-
## Grammar (upstream, not yet present)
|
|
68
|
-
|
|
69
|
-
The RelaxNG schemas in the repo-root `grammar/` are **copies of the upstream
|
|
70
|
-
source of truth** `metanorma/metanorma-model-iso/grammars/` (authored as `.rnc`,
|
|
71
|
-
compiled to `.rng`). There is **no EASC grammar upstream yet**, so this flavor
|
|
72
|
-
intentionally ships **no** `grammar/relaton-easc*.rng` and **no** Jing/XML-schema
|
|
73
|
-
validation spec. Adding the upstream `relaton-easc.rnc` (extending
|
|
74
|
-
`BibDataExtensionType` with the `Ext` elements above, à la the plateau pattern)
|
|
75
|
-
is tracked as a cross-project hand-off to metanorma-model-iso; once it lands,
|
|
76
|
-
copy the compiled pair into `grammar/` and add the round-trip Jing spec.
|
|
77
|
-
|
|
78
|
-
## Testing
|
|
79
|
-
|
|
80
|
-
RSpec + WebMock. `spec/easc/support/webmock.rb` builds an offline `index-v2`
|
|
81
|
-
in-memory from `Pubid::Easc` identifiers (so each row round-trips through
|
|
82
|
-
`from_hash`/`to_hash` like a published index), preloads it into the
|
|
83
|
-
`Relaton::Index` pool in `before(:suite)`, and stubs per-document requests from
|
|
84
|
-
`spec/easc/fixtures/data/*.yaml`. No test hits the network. Run with
|
|
85
|
-
`bundle exec rake spec:easc`.
|
data/lib/relaton/ecma/CLAUDE.md
DELETED
|
@@ -1,217 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
## Development
|
|
4
|
-
|
|
5
|
-
- `bundle install` — install dependencies
|
|
6
|
-
- `bundle exec rake spec:ecma` — run this flavor's suite (from the repo root)
|
|
7
|
-
- The ECMA flavor needs pubid support that is not in the released
|
|
8
|
-
`2.0.0.pre.alpha.8`, so the root `Gemfile` git-pins pubid. Pick it up with
|
|
9
|
-
**`bundle update pubid`**, never `bundle install` — see the
|
|
10
|
-
stale-pubid-lock trap in the root `CLAUDE.md`.
|
|
11
|
-
|
|
12
|
-
## Pubid-backed docidentifier
|
|
13
|
-
|
|
14
|
-
`Relaton::Ecma::Docidentifier` parses its `content` into a
|
|
15
|
-
`Pubid::Ecma::Identifier` kept in `@pubid`, while the lutaml `content` attribute
|
|
16
|
-
stays a plain **string** for serialization. Parsing is **soft**: `content=`
|
|
17
|
-
lazily requires pubid and rescues `LoadError`/`StandardError`, so a missing gem
|
|
18
|
-
or non-ECMA content leaves `@pubid` nil rather than raising. All three mutators
|
|
19
|
-
therefore no-op safely, and `Bib::ItemData`'s `#to_all_parts` /
|
|
20
|
-
`#to_most_recent_reference` never raise on ECMA items.
|
|
21
|
-
|
|
22
|
-
**The one ECMA-specific rule — `refresh_content!` renders**
|
|
23
|
-
**`to_s(with_edition: false, with_volume: false)`.**
|
|
24
|
-
|
|
25
|
-
`Pubid::Ecma::Identifier#to_s` renders the edition and the volume **by default**
|
|
26
|
-
(`"ECMA-269 ed3 vol2"`). That default is deliberate on the pubid side, because
|
|
27
|
-
the index keys on a bare `to_s` — see below. A **document's own** docidentifier
|
|
28
|
-
is the opposite: every `ECMA-269` volume file in `relaton-data-ecma` carries
|
|
29
|
-
`docidentifier: ECMA-269` and the same title, so the edition and the volume are
|
|
30
|
-
index metadata, not part of the printed id. Without the opt-out, any mutation
|
|
31
|
-
would silently promote the stored content to the index form.
|
|
32
|
-
|
|
33
|
-
This is the mirror image of `Relaton::ThreeGpp::Docidentifier`, whose
|
|
34
|
-
`refresh_content!` must pass `with_publisher: true` because *its* pubid defaults
|
|
35
|
-
to the index rendering and *its* stored content is the fuller one. The two look
|
|
36
|
-
contradictory only until you notice each re-renders what its own `content`
|
|
37
|
-
already holds.
|
|
38
|
-
|
|
39
|
-
Component mapping:
|
|
40
|
-
|
|
41
|
-
- **`remove_date!` → clears `edition`.** ECMA has no date component; `edition`
|
|
42
|
-
is its version discriminator. It is invisible in the rendered bare form, but
|
|
43
|
-
the identifier really does change and a consumer reading `#pubid` sees it.
|
|
44
|
-
- **`remove_part!` → clears `part`/`subpart`.** Real here: `ECMA-418-1` →
|
|
45
|
-
`ECMA-418`.
|
|
46
|
-
- **`to_all_parts!` → both, then wraps `@pubid` in pubid's `AllParts`.**
|
|
47
|
-
`content` stays the plain stripped id — the ECMA renderer has no
|
|
48
|
-
"(all parts)" marker, and `content` is cached from the pre-wrap pubid on
|
|
49
|
-
purpose. `#pubid` itself, read directly, now answers `all_parts? == true`
|
|
50
|
-
and renders WITH pubid's generic marker (`#to_s`), since it's the wrapper.
|
|
51
|
-
|
|
52
|
-
## Index
|
|
53
|
-
|
|
54
|
-
### The gem builds and reads `index-v2`; the data repo derives `index-v1`
|
|
55
|
-
|
|
56
|
-
`INDEXFILE` is the pubid-backed `index-v2`: rows are `Pubid::Ecma::Identifier`
|
|
57
|
-
hashes (`_type: pubid:ecma:{standard,technical-report,memento}` with
|
|
58
|
-
`number`/`part`/`edition`/`volume`). The consumer (`Bibliography#index`) and
|
|
59
|
-
the producer (`DataFetcher#index`) pass
|
|
60
|
-
`pubid_class: ::Pubid::Ecma::Identifier`. Omitting it on the **producer** writes
|
|
61
|
-
v1-shaped rows under a v2 name, silently (`FileIO#save` calls `to_hash` only for
|
|
62
|
-
instances of `pubid_class`); omitting it on the **consumer** leaves the rows raw
|
|
63
|
-
hashes with `FileIO#sorted` false, so every lookup scans all 804.
|
|
64
|
-
`Processor#remove_index_file` passes `url: true` and `file:` only: the delete
|
|
65
|
-
never reads the index (see `lib/relaton/index/CLAUDE.md`).
|
|
66
|
-
|
|
67
|
-
This gem no longer produces the legacy `index-v1`. `relaton-data-ecma`'s
|
|
68
|
-
`crawler.rb` derives it from the v2 rows, the IANA/BIPM/W3C shape.
|
|
69
|
-
|
|
70
|
-
**Why the edition must be in the key.** `Index::Type#add_or_update` keys on a
|
|
71
|
-
**bare** `id.to_s`, and 740 of the 804 published rows carry an edition. Before
|
|
72
|
-
pubid rendered it, those 804 rows collapsed onto 421 keys — `ECMA-74` 22 rows to
|
|
73
|
-
1, `ECMA-262` 18 to 1 — so a crawl would drop **383 of 804 rows and report
|
|
74
|
-
success**. Four rows also need `volume`: `ECMA-269` edition 3, volumes 1-4 share
|
|
75
|
-
one docidentifier *and* one title, so the volume is the only thing that tells
|
|
76
|
-
them apart.
|
|
77
|
-
|
|
78
|
-
**`#index_id` builds the identifier from the MODEL, not from a rendered string.**
|
|
79
|
-
It takes the docidentifier's own pubid, **duplicates** it, then sets `edition`
|
|
80
|
-
from `bib.edition.content` and `volume` from the extent locality — the same
|
|
81
|
-
three fields `#filename_id` reads. The `dup` is load-bearing: those two are
|
|
82
|
-
index metadata, so setting them on the shared object would promote the
|
|
83
|
-
document's own printed id to the index form. Filenames are unaffected either
|
|
84
|
-
way; `output_file` is fed by `#filename_id`, giving `ecma-269-3-1.yaml`.
|
|
85
|
-
|
|
86
|
-
**An unparseable docid is recorded, not just warned.** `#add_to_index` writes
|
|
87
|
-
`@errors[docid] = "Unparseable primary id … was not indexed (…)"`, which the
|
|
88
|
-
inherited `report_errors` logs and `Logger::Channels::GhIssue` turns into a
|
|
89
|
-
GitHub issue at the end of the crawl. (`Core::DataFetcher#report_errors` treats
|
|
90
|
-
a **String** value as the message; the `@errors[:key] &&= …` boolean form the
|
|
91
|
-
parsers use means "this field failed for every record" and is the wrong shape
|
|
92
|
-
here.) The row is skipped rather than indexed unparsed, because
|
|
93
|
-
`Relaton::Index` rejects the **whole** index if one row fails to deserialize and
|
|
94
|
-
its sort calls `.root.number` on every id. The data file is still written, so
|
|
95
|
-
the document is unindexed, never lost. (The 3GPP/W3C precedent.)
|
|
96
|
-
|
|
97
|
-
Verified against the live published corpus: all 804 rows build an identifier,
|
|
98
|
-
they render **804 distinct keys** (not 421), `root.number` is empty for none,
|
|
99
|
-
`from_hash(to_hash)` round-trips all 804, and the four `ECMA-269` volumes stay
|
|
100
|
-
distinct.
|
|
101
|
-
|
|
102
|
-
### Lookup: `best_match`
|
|
103
|
-
|
|
104
|
-
`Bibliography#search` follows the ETSI/W3C/OGC idiom:
|
|
105
|
-
|
|
106
|
-
- **Pass the pubid, not the string.** `Type#search_candidates` narrows only when
|
|
107
|
-
the argument is not a `String`, and a block alone never narrows — so the plain
|
|
108
|
-
string this flavor used to pass disabled the binary search however the index
|
|
109
|
-
was built. `pubid_class:` alone fixes nothing; both had to change together.
|
|
110
|
-
Measured: `ECMA-269` now narrows to 12 of 804 rows.
|
|
111
|
-
- **Select with the subset match.** `search` calls `index.search(pubid)` with
|
|
112
|
-
no block, so `Index::Type#search` selects with pubid's `pubid === row`. An
|
|
113
|
-
`edition` or `volume` that the reference omits matches any value; they are
|
|
114
|
-
index metadata that a document's own docidentifier never carries. pubid
|
|
115
|
-
declares `part` strict for ECMA (pubid#408), so a bare `ECMA-418` does not
|
|
116
|
-
match `ECMA-418-1`. `===` requires the same class, which keeps `ECMA-100`
|
|
117
|
-
and `ECMA TR/100` apart. Measured over the 804-row fixture: `===` and the
|
|
118
|
-
old `matches?(…, ignore: %i[edition volume])` agree on all 6,492 pairs.
|
|
119
|
-
- **An unparseable reference finds nothing**, with a warning. There is
|
|
120
|
-
deliberately no substring-scan fallback (OGC has one): an ECMA row renders as
|
|
121
|
-
`ECMA-262 ed17`, so a scan would answer a truncated `ECMA-26` with every
|
|
122
|
-
ECMA-26x document, and an ambiguous answer is worse than none.
|
|
123
|
-
|
|
124
|
-
**A reference must now parse WHOLE.** The old `parse_ref` regex was unanchored
|
|
125
|
-
at the end, so trailing text after a valid prefix was ignored and still
|
|
126
|
-
resolved; pubid rejects it. Measured:
|
|
127
|
-
|
|
128
|
-
| reference | old | new |
|
|
129
|
-
|---|---|---|
|
|
130
|
-
| `ECMA-6 (draft)` | `ECMA-6` | none |
|
|
131
|
-
| `ECMA-6:1991` | `ECMA-6` | none |
|
|
132
|
-
| `ECMA-6 2nd edition` | `ECMA-6` | none |
|
|
133
|
-
| `ECMA-6 ` (trailing space) | `ECMA-6` | `ECMA-6` |
|
|
134
|
-
| ` ECMA-6` (leading space) | none | `ECMA-6` — `parse_ref` strips |
|
|
135
|
-
|
|
136
|
-
That is the same trade-off as the missing fallback — a strict parse beats an
|
|
137
|
-
ambiguous match — and it is the first thing to check against a report that a
|
|
138
|
-
reference "used to resolve and now does not". `ecma-6` (lowercase) parsed under
|
|
139
|
-
neither.
|
|
140
|
-
|
|
141
|
-
#### Ordering: latest edition, then lowest volume
|
|
142
|
-
|
|
143
|
-
That is the order the bespoke `compare_edition_volume` + `min` implemented, kept
|
|
144
|
-
deliberately — but the edition is compared **segment-wise as integers**
|
|
145
|
-
(`edition_key`), and `r[:file]` breaks the tie because the index sort is not
|
|
146
|
-
stable. Comparing the rendered strings made `"9"` beat `"17"`, so 5 of the 421
|
|
147
|
-
document families returned an older document than the reference asked for.
|
|
148
|
-
Scored against each document's own published date:
|
|
149
|
-
|
|
150
|
-
| family | string compare | integer compare | dates |
|
|
151
|
-
|---|---|---|---|
|
|
152
|
-
| ECMA-262 | ed9 | **ed17** | 2018-06 → 2026-06 |
|
|
153
|
-
| ECMA-74 | ed9 | **ed22** | 2005-12 → 2025-12 |
|
|
154
|
-
| ECMA-402 | ed9 | **ed13** | → 2026-06 |
|
|
155
|
-
| ECMA-328 | ed7 | **ed10** | same direction |
|
|
156
|
-
| ECMA-109 | ed9 | **ed11** | same direction |
|
|
157
|
-
|
|
158
|
-
The integer key picks the newer document in all 5. An absent edition sorts below
|
|
159
|
-
every present one — right for the 64 edition-less rows (mementos, a few
|
|
160
|
-
reports), none of which shares a document with an edition-bearing row.
|
|
161
|
-
|
|
162
|
-
**`ECMA-262 ed5.1` is the only dotted edition in the whole published corpus.**
|
|
163
|
-
It is what forces the segment-wise compare (`5.1` must beat `5`), but since 262's
|
|
164
|
-
latest is ed17 the dotted ordering never decides a bare lookup — so it is pinned
|
|
165
|
-
by a unit assertion on `edition_key` rather than by an end-to-end example. Do not
|
|
166
|
-
"simplify" the key to `to_i`.
|
|
167
|
-
|
|
168
|
-
The old `match_ref` also matched on a **prefix** (`/^ECMA[-\s]#{id}/`), so
|
|
169
|
-
`ECMA-43` matched `ECMA-430…434`. Four families collided this way (418, 43, 35,
|
|
170
|
-
13); none of the four changed its answer, so exact matching removed a latent
|
|
171
|
-
risk rather than moving results.
|
|
172
|
-
|
|
173
|
-
#### Verified against the published index
|
|
174
|
-
|
|
175
|
-
`relaton-data-ecma` publishes `index-v2.zip`. Loaded through
|
|
176
|
-
`Bibliography#index`: **804 rows**, all deserialized to identifiers, sorted,
|
|
177
|
-
**0** keying on `""`, 804 distinct keys, and `from_hash(to_hash)` round-trips
|
|
178
|
-
every row. Live lookups resolve every shape the flavor handles, each to a file
|
|
179
|
-
that returns HTTP 200 — a bare `ECMA-269` → ed9, the space form `ECMA 269` →
|
|
180
|
-
the same, `ECMA-269 ed3` → vol1, all four `ed3 vol<N>`, `ECMA-262` → ed17
|
|
181
|
-
(2026-06), `ECMA-262 ed5.1`, `ECMA-418` vs `ECMA-418-1`, `ECMA-100` vs
|
|
182
|
-
`ECMA TR/100`, `ECMA MEM/2021`, and `ECMA-43` narrowed to number `43` alone.
|
|
183
|
-
|
|
184
|
-
## Testing
|
|
185
|
-
|
|
186
|
-
- **Framework:** RSpec with VCR cassettes and WebMock.
|
|
187
|
-
- **Index fixture:** `spec/ecma/fixtures/index-v2.zip` — the whole published
|
|
188
|
-
index (804 rows), copied verbatim, so the stored shapes are exactly what the
|
|
189
|
-
runtime deserializes. Refresh it with `bundle exec rake spec:update_index_ecma`
|
|
190
|
-
(`tasks/index_fixture_ecma.rb`); `#build` refuses a source that is not a pubid
|
|
191
|
-
index-v2 rather than writing a fixture `Relaton::Index` would reject wholesale.
|
|
192
|
-
- It is seeded into the `Relaton::Index` pool by `spec/ecma/support/webmock.rb`,
|
|
193
|
-
built **with `pubid_class:`** — without it the rows stay raw hashes,
|
|
194
|
-
`FileIO#sorted` stays false, and `Type#search` silently stops narrowing, so the
|
|
195
|
-
suite would pass while exercising something the runtime never does. Seeded in
|
|
196
|
-
`before(:each)`, not only `before(:suite)`, because `Index::Pool#type` replaces
|
|
197
|
-
the pooled entry whenever `actual?` says no and `DataFetcher#index` asks for
|
|
198
|
-
the same type with `file:` but no `url:`. With `before(:suite)` alone every
|
|
199
|
-
example after the first data-fetcher one searched a producer index, and the
|
|
200
|
-
consumer then built a third type and went to the network for real. The pool key
|
|
201
|
-
is `type.upcase.to_sym`, so the fetcher's `:ecma` and the consumer's `:ECMA`
|
|
202
|
-
share one slot. (The OGC pattern.)
|
|
203
|
-
- **VCR cassettes:** `spec/ecma/vcr_cassettes/` — index downloads are ignored by
|
|
204
|
-
VCR, matched on `INDEXFILE` so a version bump cannot let a real download
|
|
205
|
-
through.
|
|
206
|
-
- `spec/ecma/relaton/ecma/index_key_spec.rb` checks the whole fixture corpus for
|
|
207
|
-
the properties the index depends on: every row deserializes, one distinct
|
|
208
|
-
rendered key per row, no empty bsearch key, sorted, and a `to_hash` round-trip.
|
|
209
|
-
- A live check of the flavor cannot use `Bibliography.get` behind an HTTP proxy:
|
|
210
|
-
the document fetch goes through **Mechanize**, which does not read
|
|
211
|
-
`http_proxy`/`https_proxy` the way `Net::HTTP` does, so it dies with a
|
|
212
|
-
connection error on a host `curl` reaches. The index download (`Net::HTTP`)
|
|
213
|
-
works, so drive `best_match` and fetch the resolved file with `Net::HTTP`
|
|
214
|
-
instead.
|
|
215
|
-
- The `bibdata`/`bibitem` round-trip examples validate XML with Jing, which
|
|
216
|
-
needs a **JVM on PATH**. Without one they fail with `Jing::ExecutionError`;
|
|
217
|
-
that is environmental, not a code defect.
|
data/lib/relaton/etsi/CLAUDE.md
DELETED
|
@@ -1,219 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
relaton-etsi is a Ruby gem that retrieves ETSI (European Telecommunications Standards Institute) standards metadata using the Relaton bibliographic item model. It fetches data from the ETSI website and the relaton-data-etsi GitHub repository.
|
|
8
|
-
|
|
9
|
-
## Commands
|
|
10
|
-
|
|
11
|
-
```bash
|
|
12
|
-
# Install dependencies
|
|
13
|
-
bin/setup
|
|
14
|
-
|
|
15
|
-
# Run all tests
|
|
16
|
-
rake spec
|
|
17
|
-
|
|
18
|
-
# Run a specific test file
|
|
19
|
-
rspec spec/relaton/etsi/bibliography_spec.rb
|
|
20
|
-
|
|
21
|
-
# Run a specific test by line number
|
|
22
|
-
rspec spec/relaton/etsi/bibliography_spec.rb:15
|
|
23
|
-
|
|
24
|
-
# Run linter
|
|
25
|
-
rake rubocop
|
|
26
|
-
|
|
27
|
-
# Interactive console
|
|
28
|
-
bin/console
|
|
29
|
-
|
|
30
|
-
# Install gem locally
|
|
31
|
-
bundle exec rake install
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Architecture
|
|
35
|
-
|
|
36
|
-
The codebase is transitioning from `RelatonEtsi` namespace to `Relaton::Etsi` namespace.
|
|
37
|
-
|
|
38
|
-
### New Architecture (lib/relaton/etsi/)
|
|
39
|
-
|
|
40
|
-
Uses Lutaml for serialization:
|
|
41
|
-
- `Item` - base bibliographic item inheriting from `Bib::Item`
|
|
42
|
-
- `Bibitem` - for `<bibitem>` XML output (includes `Bib::BibitemShared`)
|
|
43
|
-
- `Bibdata` - for `<bibdata>` XML output (includes `Bib::BibdataShared`)
|
|
44
|
-
- `Ext` - ETSI-specific extension data (marker, frequency, mandate, custom_collection)
|
|
45
|
-
- `Doctype` - ETSI document type with abbreviations (EN, ES, GS, TS, TR, etc.)
|
|
46
|
-
|
|
47
|
-
### Legacy Architecture (lib/relaton_etsi/)
|
|
48
|
-
|
|
49
|
-
- `BibliographicItem` - extends `RelatonBib::BibliographicItem`
|
|
50
|
-
- `XMLParser` - parses XML into bibliographic items
|
|
51
|
-
- `HashConverter` - converts hashes to bibliographic items
|
|
52
|
-
- `DocumentType` - document type with type/abbreviation mapping
|
|
53
|
-
|
|
54
|
-
### Core Components
|
|
55
|
-
|
|
56
|
-
- `Bibliography` - searches and retrieves standards from relaton-data-etsi index
|
|
57
|
-
- `DataFetcher` - fetches all documents from ETSI website CSV export
|
|
58
|
-
- `DataParser` - parses CSV rows into bibliographic items
|
|
59
|
-
- `Processor` - Relaton processor for integration with the relaton ecosystem
|
|
60
|
-
|
|
61
|
-
### Data Flow
|
|
62
|
-
|
|
63
|
-
1. `Bibliography.get(ref)` searches the relaton-data-etsi index
|
|
64
|
-
2. Fetches YAML from GitHub, converts to `Item` using `from_yaml`
|
|
65
|
-
3. `DataFetcher.fetch` pulls CSV from etsi.org, parses with `DataParser`, saves to output folder
|
|
66
|
-
|
|
67
|
-
### Pubid-backed index-v2
|
|
68
|
-
|
|
69
|
-
The ETSI index is **pubid-structured** (`index-v2.yaml`/`.zip`): each row's `:id`
|
|
70
|
-
is a `Pubid::Etsi::Identifier` serialized to its `_type: pubid:etsi:{etsi-standard,
|
|
71
|
-
amendment,corrigendum}` hash (the whole published ETSI corpus round-trips on the
|
|
72
|
-
pinned pubid — see the root `CLAUDE.md` pubid-pin note). The rows use the **flat,
|
|
73
|
-
compact** shape (`type`/`number`/`version`/`year`/`month` as scalars under
|
|
74
|
-
`_type`) that the published `relaton-data-etsi` index carries — the ETSI
|
|
75
|
-
`to_hash` flattening (merged from `refactor/flatten-etsi-to-hash`) ships in the
|
|
76
|
-
released pubid `2.0.0.pre.alpha.13` that the gemspec pins. The wiring mirrors NIST/JCGM:
|
|
77
|
-
|
|
78
|
-
- **Producer** (`DataFetcher`): `index` calls `find_or_create(:etsi, file:
|
|
79
|
-
"#{INDEXFILE}.yaml", pubid_class: ::Pubid::Etsi::Identifier)`; `#save` parses the
|
|
80
|
-
docid via `#pubid` and stores the **pubid object** (`index.add_or_update pid,
|
|
81
|
-
file`) so `Relaton::Index` sorts by id number and serializes each id to its
|
|
82
|
-
`_type:` hash on save. `#pubid` returns nil (skipping the whole record) for any
|
|
83
|
-
id it can't parse **or** `to_hash`-serialize, so one malformed record can never
|
|
84
|
-
abort the crawl or corrupt the index. (No current ETSI record is skipped.)
|
|
85
|
-
- **Consumer** (`Bibliography#search`): parses the reference with
|
|
86
|
-
`::Pubid::Etsi.parse` and lets a `Pubid::Errors::ParseError` on an unrecognized ref
|
|
87
|
-
**propagate** (ISO parity — the CLI renders a friendly message, API callers
|
|
88
|
-
rescue it), then `#best_match` selects with pubid's subset match
|
|
89
|
-
`pubid === row`. The `pubid` (not a String) lets `Relaton::Index` narrow
|
|
90
|
-
candidates by number via binary search. A `version` or `date` that the ref
|
|
91
|
-
omits matches any value, so a bare `ETSI GS ZSM 012` matches every edition
|
|
92
|
-
and a fully-qualified ref matches only its edition.
|
|
93
|
-
`max_by { edition_key(row[:id]) }` returns the most recent.
|
|
94
|
-
Requires pubid `main` (partial-ref parsing, `Pubid::Errors::ParseError` on
|
|
95
|
-
failure, `subset_strict`).
|
|
96
|
-
- **A part-less reference strips the ROW's parts, it does not ask pubid for
|
|
97
|
-
"all parts".** pubid declares `parts` strict for ETSI (pubid#408), so a
|
|
98
|
-
part-less ref matches no part row on its own. `#comparable` therefore
|
|
99
|
-
matches each row without its parts (`exclude(:part, :subpart, :parts)`, a
|
|
100
|
-
copy — the cached index id is untouched), and `ETSI EN 300 175` reaches its
|
|
101
|
-
16 part rows. **Do not use `#to_all_parts` here.** Since pubid#433 "all
|
|
102
|
-
parts" is a class, and `Pubid::AllPartsIdentifier#===` compares the document
|
|
103
|
-
alone: it drops the version and the date as well, so
|
|
104
|
-
`ETSI GR ZSM 011 V1.1.1 (2023-02)` answered with V2.1.1 (2024-09) — 586,487
|
|
105
|
-
pairs over the fixture. `resolves a fully-qualified part-less reference to
|
|
106
|
-
that exact edition` in `bibliography_spec.rb` pins it.
|
|
107
|
-
- **A base reference does not reach its supplements.** `===` requires the
|
|
108
|
-
same class, so `ETSI ETR 310` does not match the corrigendum
|
|
109
|
-
`ETSI ETR 310/C1`, and a corrigendum of the base (`ETS 300 092-1/C1`) does
|
|
110
|
-
not match a corrigendum of its amendment (`092-1/A1/C1`). The
|
|
111
|
-
`matches?(…, ignore:)` it replaced did match them: over the 28,651-row
|
|
112
|
-
fixture, 298 pairs changed, all of them a supplement row that the old match
|
|
113
|
-
let in, and 218 of 94,079 references resolved to a newer amendment or
|
|
114
|
-
corrigendum instead of the document itself (`ETSI ETR 310` →
|
|
115
|
-
`ETR 310/C1 ed.1 (1996-10)`). `bibliography_spec.rb` pins that case.
|
|
116
|
-
- **`#edition_key` orders on the parsed version, never on the rendered id.** ETSI
|
|
117
|
-
versions are not zero-padded, so a String comparison of `row[:id].to_s` orders
|
|
118
|
-
`V9.0.0` above `V19.0.0` and `ed.9` above `ed.11` — it picked the wrong edition
|
|
119
|
-
for 983 of the 3007 multi-edition documents in the spec fixture, and a bare
|
|
120
|
-
`ETSI TR 155 919` resolved to the 2010 edition. `Pubid::Etsi::Identifier` is
|
|
121
|
-
**not** `Comparable` (its `<=>` returns nil), so the key is built from the
|
|
122
|
-
components: `id.version.version` holds the bare numbers (`"19.0.0"`, or `"9"`
|
|
123
|
-
for the `ed.N` form) and `id.date` renders as `yyyy-mm` for the tie-break.
|
|
124
|
-
Both delegate to `base` on a `pubid:etsi:corrigendum`/`amendment` row (162 of
|
|
125
|
-
them in the fixture), so every row shape keys the same way. The key ignores
|
|
126
|
-
`is_edition`, which puts `ed.11` above `V1.0.0` — safe because **no** document
|
|
127
|
-
mixes the two forms (measured: 0 of 28651 fixture rows). Do not put `.to_s`
|
|
128
|
-
back.
|
|
129
|
-
- **Processor** `#remove_index_file` passes `url: true` and `file:` only. It
|
|
130
|
-
needs no `pubid_class:`: the delete never reads the index (see
|
|
131
|
-
`lib/relaton/index/CLAUDE.md`).
|
|
132
|
-
|
|
133
|
-
### The crawl query keeps superseded editions (`version=1`)
|
|
134
|
-
|
|
135
|
-
`DataFetcher::SOURCEURL` is the ETSI standards-search query, and
|
|
136
|
-
`relaton-data-etsi/data/` is **exactly** what it returns — `crawler.rb` rebuilds
|
|
137
|
-
`data/` on every run, so a record the query drops cannot be reinstated in the
|
|
138
|
-
data repo. The `version` flag selects editions, not statuses: `version=0`
|
|
139
|
-
returns only the current edition of each branch, `version=1` returns the
|
|
140
|
-
superseded ones as well. The status flags (`withdrawn`, `historical`,
|
|
141
|
-
`superseded`, `onApproval`, `isCurrent`) are a **separate axis** and are already
|
|
142
|
-
all `1` — that is why the dataset carried `Historical` and `Withdrawn` documents
|
|
143
|
-
while still missing older editions (metanorma/metanorma-pdfa#95, which wanted
|
|
144
|
-
`ETSI EN 319 142-1 V1.2.1 (2024-01)` back). `version=0` also still returns
|
|
145
|
-
parallel current branches, so the response's own `superseded`/`new_versions`
|
|
146
|
-
fields, not the major version, are what `version` keys on.
|
|
147
|
-
|
|
148
|
-
The flavor uses `version=1`. Measured against the live API on 2026-08-27:
|
|
149
|
-
|
|
150
|
-
| | `version=0` | `version=1` |
|
|
151
|
-
|---|---|---|
|
|
152
|
-
| records | 28625 | 67446 (2.36x) |
|
|
153
|
-
| pages at 50/page | 573 | 1349 |
|
|
154
|
-
| `data/` on disk | 112 MB | ~264 MB |
|
|
155
|
-
| `index-v2.yaml` | 5.0 MB | ~11.8 MB |
|
|
156
|
-
|
|
157
|
-
Nothing else in the producer needed changing: the version is part of the ETSI
|
|
158
|
-
docid, so `Core::DataFetcher#unique_output_file` gives each edition its own
|
|
159
|
-
filename and `Index::Type#add_or_update` (keyed on `id.to_s`) gives each its own
|
|
160
|
-
row. `spec/etsi/relaton/etsi/data_fetcher_spec.rb` pins the query flag, and pins
|
|
161
|
-
that machinery too — it was already correct, but `version=1` is what makes
|
|
162
|
-
several editions of one document a common case rather than a rare one.
|
|
163
|
-
|
|
164
|
-
**Ordering rule for the data repo.** This is producer-side only; nothing changes
|
|
165
|
-
for users until `relaton-data-etsi` re-crawls. Do **not** deploy a `version=1`
|
|
166
|
-
crawl until the released `relaton-etsi` gem picks the newest edition. Its
|
|
167
|
-
`Bibliography#search` uses `min_by` on the rendered id, which returns the
|
|
168
|
-
**oldest** match, so a bare `ETSI EN 319 401` would resolve to the 2013 edition
|
|
169
|
-
across the whole corpus. This flavor picks the newest edition (`#best_match`
|
|
170
|
-
orders on `#edition_key`). Also check the crawl against the 6-hour GitHub
|
|
171
|
-
Actions job cap — it roughly doubles — and confirm all three EN 319 142-1
|
|
172
|
-
editions land in `data/` before merging the re-crawl.
|
|
173
|
-
|
|
174
|
-
### A bad page fails the crawl; it is never skipped
|
|
175
|
-
|
|
176
|
-
The relaton-data-etsi crawl of 2026-09-30 (run 36761804954) died at about page
|
|
177
|
-
700 with `JSON::ParserError: unexpected character: 'Array'`: data.php answered
|
|
178
|
-
once with PHP's `Array` text in place of JSON. It is transient and server side
|
|
179
|
-
— the code had not changed since the good crawl of 2026-09-28, and a full
|
|
180
|
-
re-crawl on 2026-10-02 gave valid JSON on all 1,353 pages (67,602 records,
|
|
181
|
-
`RowNum` 1..67,602 with no gap). Note also that `curl` gets an HTTP 403 page from
|
|
182
|
-
ETSI's web application firewall; Mechanize's user agent gets JSON.
|
|
183
|
-
|
|
184
|
-
`DataFetcher#fetch_page` therefore parses the body **inside**
|
|
185
|
-
`fetch_with_retry`: a body that is not a JSON array raises `BadPage`, which is
|
|
186
|
-
retried like a network error. A page still bad after its retries is deferred
|
|
187
|
-
and fetched again after the last page (`DEFERRED_DELAY`); if it is still bad,
|
|
188
|
-
the crawl raises `BadPage` with the page number and the first 200 characters of
|
|
189
|
-
the body, and `index.save` is never reached. Page 1 is not deferred (it gives
|
|
190
|
-
`total_count`). **Do not change this to skip the page:** `crawler.rb` deletes
|
|
191
|
-
`data/` before the crawl and commits what the crawl writes, so a skipped page
|
|
192
|
-
unpublishes 50 documents, while a failed crawl commits nothing and the previous
|
|
193
|
-
data stays live.
|
|
194
|
-
|
|
195
|
-
The result set is sorted by deliverable number (`sort=1`), not by date, so a
|
|
196
|
-
document that ETSI publishes during the one-hour crawl shifts the later pages
|
|
197
|
-
by one. The page count from page 1's `total_count` would then miss the last
|
|
198
|
-
record, so `fetch_remaining_pages` reads on past it while the pages stay full.
|
|
199
|
-
A record read twice overwrites its own file. Two bounds keep the loop finite
|
|
200
|
-
(data.php answers `[]` past the end today, but the loop must not depend on it):
|
|
201
|
-
a deferred page extends the range by one page only when it lies inside page
|
|
202
|
-
1's `total_count`, and a crawl still reading full pages `MAX_EXTRA_PAGES` past
|
|
203
|
-
that count raises. A deferred page that comes back `[]` inside the range also
|
|
204
|
-
fails the crawl. **Known limit:** a document *removed* during the crawl shifts
|
|
205
|
-
the later pages the other way, so one record can fall onto a page already read.
|
|
206
|
-
Nothing detects that; it is rare, because every status flag is on.
|
|
207
|
-
|
|
208
|
-
## Testing
|
|
209
|
-
|
|
210
|
-
- **Index fixture:** `spec/fixtures/index-v2.zip` (pubid `_type:` rows) is loaded
|
|
211
|
-
into the `Relaton::Index` pool in `before(:suite)` (`spec/support/webmock.rb`):
|
|
212
|
-
the YAML is written to a temp file and read through
|
|
213
|
-
`Relaton::Index::Type.new(:etsi, file:, pubid_class: ::Pubid::Etsi::Identifier)`,
|
|
214
|
-
and `type.index` forces the offline `pubid_class` deserialize before the net is
|
|
215
|
-
blocked; `actual?` is overridden to match only the remote (`url:`) lookup so the
|
|
216
|
-
producer-side `find_or_create(:etsi, file:, pubid_class:)` still gets a fresh
|
|
217
|
-
instance. Regenerate by parsing `index-v1`'s ids through `Pubid::Etsi` into a
|
|
218
|
-
`pubid_class` `Type` and re-zipping (`_type: pubid:etsi:…` rows).
|
|
219
|
-
Uses RSpec with VCR for HTTP interaction recording. VCR cassettes are in `spec/vcr_cassettes/`. When tests make new HTTP requests, VCR will record them.
|
data/lib/relaton/evs/CLAUDE.md
DELETED
|
@@ -1,120 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
relaton-evs retrieves Estonian standardisation (EVS, Eesti Standardimis- ja
|
|
8
|
-
Akrediteerimiskeskus) bibliographic data using the Relaton model. Like CEN/BSI,
|
|
9
|
-
it publishes **no index and has no data repo**: the flavor **scrapes** the EVS
|
|
10
|
-
e-shop (`www.evs.ee`) with Mechanize. Reference handling is entirely
|
|
11
|
-
`Pubid::Evs` (the `:evs` flavor, registered in pubid's registry). EVS records
|
|
12
|
-
are dominated by **adoptions**: `EVS-EN …` adopts a European document,
|
|
13
|
-
`EVS-EN ISO …` a European adoption of an international one, `EVS-ISO …` /
|
|
14
|
-
`EVS-IEC …` adopt international documents directly. The adoption is expressed
|
|
15
|
-
with `adoptedFrom` relations parsed from the page's "Base Documents" group.
|
|
16
|
-
|
|
17
|
-
## Development
|
|
18
|
-
|
|
19
|
-
```bash
|
|
20
|
-
bundle exec rake spec:evs # the EVS suite
|
|
21
|
-
cd spec/evs && bundle exec rspec -I . relaton/evs/docidentifier_spec.rb
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
(No gem-local rubocop config; the monorepo root config applies.)
|
|
25
|
-
|
|
26
|
-
## Architecture
|
|
27
|
-
|
|
28
|
-
Namespace: `Relaton::Evs`. This is the **BSI/CEN shape**: parse the query with
|
|
29
|
-
pubid, select the hit with pubid, back the `Docidentifier` with pubid. No
|
|
30
|
-
`INDEXFILE`, no `pubid_class:`, no data repo. **Don't add an index** (unless a
|
|
31
|
-
`relaton-data-evs` repo appears — relaton#164 tracks that possibility).
|
|
32
|
-
|
|
33
|
-
Retrieval flow:
|
|
34
|
-
|
|
35
|
-
1. **Bibliography** (`lib/relaton/evs/bibliography.rb`) — `get(code, year,
|
|
36
|
-
opts)` / `search` / `parse`. `parse` wraps `Pubid::Evs.parse` and **raises**
|
|
37
|
-
on an unrecognized query (root CLAUDE.md law); only the data side rescues,
|
|
38
|
-
in `Hit#pubid`. `get` keeps an early `nil` for an **empty** reference and
|
|
39
|
-
accepts the flavor's own pubid without mutating it (`Relaton::Db` hands it
|
|
40
|
-
the parse it routed). Transport errors are rescued **by name**
|
|
41
|
-
(`Net::ReadTimeout`, `Net::OpenTimeout` → `Relaton::RequestError`, which
|
|
42
|
-
`Db#net_retry` retries).
|
|
43
|
-
2. **HitCollection** (`hit_collection.rb`) — the search is a **direct slug
|
|
44
|
-
probe**, not a form submit: the catalogue URL is the printed reference
|
|
45
|
-
slugified (`EVS-EN 18216:2026` → `/en/evs-en-18216-2026`), so the product
|
|
46
|
-
page exists iff the reference denotes a published document. A miss
|
|
47
|
-
**redirects** to `/page-not-found` (HTTP 302, followed by Mechanize) — that
|
|
48
|
-
redirect target is the "no hit" signal; a hard 4xx/5xx raises
|
|
49
|
-
`Mechanize::ResponseCodeError` and is a miss (`hits` rescues to `[]`).
|
|
50
|
-
The `et`/`et-en` language codes probe the `/et/` locale, which serves the
|
|
51
|
-
Estonian-language product.
|
|
52
|
-
- **The hit's `code` keeps the language code** (only the slug derivation
|
|
53
|
-
strips it): the filter `query === hit.pubid` needs the language ON the
|
|
54
|
-
hit for a language-carrying query to match its own record.
|
|
55
|
-
- A reference **without a year** probes `…-<number>` and misses — the
|
|
56
|
-
catalogue URL always carries the year. "Not found." is honest there; do
|
|
57
|
-
not add a search-endpoint fallback without evidence the shop renders one
|
|
58
|
-
server-side (its search is client-rendered).
|
|
59
|
-
- Slug shapes verified against the live catalogue: plain `EVS 945:2023`,
|
|
60
|
-
`EVS-EN 16798-1:2019`, `EVS-EN ISO 14001:2026`, the ETSI face
|
|
61
|
-
`EVS-EN 301 406-1 V3.2.1:2026`, `EVS-EN IEC 61000-4-30:2026`.
|
|
62
|
-
3. **Hit** (`hit.rb`) — `item` lazy-loads the detail page via the scraper;
|
|
63
|
-
`pubid` memoizes the hit code parsed with pubid and is **nil** when the
|
|
64
|
-
grammar cannot read it.
|
|
65
|
-
4. **Scraper** (`scraper.rb`) — Mechanize/XPath over the product page:
|
|
66
|
-
`h1` → docidentifier, `h2.product-title` → title (`Bib::Title.from_string`
|
|
67
|
-
splits " - " into intro/main — it returns an Array, don't re-wrap), the
|
|
68
|
-
"Valid from dd.mm.yyyy" badge → `published` date, "Base Documents" →
|
|
69
|
-
`adoptedFrom` relations (a double adoption lists both bases in ONE text
|
|
70
|
-
node, `ISO 14001:2026; EN ISO 14001:2026` — split on the semicolon), the
|
|
71
|
-
scope tab (`div#tab-1`) → abstract, ICS group links → `Ext.ics` via
|
|
72
|
-
`Isoics`. Contributor/copyright: the EVS organization (publisher); place
|
|
73
|
-
Tallinn.
|
|
74
|
-
5. **ItemData / models** (`item_data.rb`, `model/`) — `Relaton::Evs::ItemData`
|
|
75
|
-
extends `Relaton::Bib::ItemData`; `Ext` adds a `StructuredIdentifier`
|
|
76
|
-
(docnumber/partnumber from the pubid **root**, agency "EVS"). There is no
|
|
77
|
-
`relaton-model-evs` gem, so `Ext` does not override `get_schema_version`
|
|
78
|
-
(inherits the nil default, like the other ten model-less flavors). The
|
|
79
|
-
**shared ext grammar requires a `doctype`** — `fetch_doctype` always emits
|
|
80
|
-
one (`standard`; `technical-specification`/`technical-report` for the
|
|
81
|
-
`EVS-TS`/`EVS-TR` faces), or Jing fails on `relaton-evs-compile.rng`.
|
|
82
|
-
|
|
83
|
-
### `#root` is the accessor that answers for every form
|
|
84
|
-
|
|
85
|
-
A `NationalAdoption` (`EVS-EN ISO 9001:2015/A1:2024`) holds the origin
|
|
86
|
-
standard's number and year behind its `base` chain (base = the CEN/CENELEC
|
|
87
|
-
document; `AdoptedEuropeanNorm#adopted` = the ISO document; `#root` recurses),
|
|
88
|
-
so the year, number and part are read from `#root`. An `InternationalAdoption`
|
|
89
|
-
(`EVS-ISO 9001:2015`) or `EstonianStandard` (`EVS 945:2023`) is its own root.
|
|
90
|
-
|
|
91
|
-
### `Docidentifier` — the year rule
|
|
92
|
-
|
|
93
|
-
`Relaton::Evs::Docidentifier` (`model/docidentifier.rb`, `< Bib::Docidentifier`)
|
|
94
|
-
parses its `content` into `@pubid` (DATA side: lazy `require "pubid"`, rescues
|
|
95
|
-
`LoadError`/`StandardError`; a foreign value stays verbatim with `@pubid` nil).
|
|
96
|
-
`Pubid#exclude` returns a **copy**, so each mutator replaces `@pubid` and
|
|
97
|
-
re-renders through the aliased `store_content`. **`remove_date!` drops the
|
|
98
|
-
LAST year**: on a supplement (`EVS-EN ISO 9001:2015/A1:2024`) the supplement's
|
|
99
|
-
year goes first (`exclude(:supplement_year)` → `…/A1`), the base year only
|
|
100
|
-
when there is none. `to_all_parts!` wraps in pubid's `AllParts` — `content`
|
|
101
|
-
stays the part-and-date-stripped string on purpose; `#pubid` renders the
|
|
102
|
-
"(all parts)" marker.
|
|
103
|
-
|
|
104
|
-
### Printed separator is identity
|
|
105
|
-
|
|
106
|
-
`EVS EN 18216:2026` (space) and `EVS-EN 18216:2026` (hyphen) parse to the same
|
|
107
|
-
class and root, but each renders back **as written** — pubid preserves the
|
|
108
|
-
input's separator. Never compare EVS identifiers by `to_s`; compare by parse
|
|
109
|
-
(`===`, `root.number`).
|
|
110
|
-
|
|
111
|
-
### Registration
|
|
112
|
-
|
|
113
|
-
`Processor` (`processor.rb`) sets `@short = :relaton_evs`, `@prefix = "EVS"`,
|
|
114
|
-
`@pubid_flavor = :Evs` (prefixes sourced from `Pubid::Evs.prefixes`) and
|
|
115
|
-
`@defaultprefix = %r{^EVS[\s-]}`. It lazy-`require_relative`s `../evs` inside
|
|
116
|
-
each method, like every other flavor (`spec/relaton/lazy_loading_spec.rb`
|
|
117
|
-
guards that). `lib/relaton.rb` autoloads `:Evs`; `Db::Registry::SUPPORTED_GEMS`
|
|
118
|
-
lists `relaton/evs`; `grammar/relaton-evs(-compile).rng` mirrors the CEN pair
|
|
119
|
-
(the shared `biblio-standoc` ext already models doctype/flavor/ics/
|
|
120
|
-
structuredidentifier, and its `DocRelationType` carries `adoptedFrom`).
|