relaton 3.1.1 → 3.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.adoc +5 -3
- data/lib/relaton/3gpp.rb +1 -1
- data/lib/relaton/bib/model/docidentifier.rb +66 -5
- data/lib/relaton/calconnect/hit_collection.rb +1 -1
- data/lib/relaton/cli/frontend_assets.rb +7 -6
- data/lib/relaton/db.rb +31 -14
- data/lib/relaton/ecma/bibliography.rb +2 -2
- data/lib/relaton/etsi/data_fetcher.rb +1 -1
- data/lib/relaton/iala/bibliography.rb +1 -1
- data/lib/relaton/iana.rb +1 -1
- data/lib/relaton/iec/data_parser.rb +41 -2
- data/lib/relaton/itu/README.adoc +1 -1
- data/lib/relaton/itu/data_crawler_r.rb +1 -1
- data/lib/relaton/oasis/bibliography.rb +1 -1
- data/lib/relaton/ogc/hit_collection.rb +1 -1
- data/lib/relaton/ogc.rb +1 -1
- data/lib/relaton/plateau/handbook_parser.rb +31 -2
- data/lib/relaton/plateau/hit_collection.rb +1 -1
- data/lib/relaton/sae.rb +1 -1
- data/lib/relaton/version.rb +1 -1
- data/lib/relaton/w3c/bibliography.rb +1 -1
- data/lib/relaton/xsf/bibliography.rb +1 -1
- data/lib/relaton/xsf.rb +1 -1
- metadata +32 -46
- data/lib/relaton/3gpp/CLAUDE.md +0 -218
- data/lib/relaton/adobe/CLAUDE.md +0 -104
- data/lib/relaton/bib/CLAUDE.md +0 -150
- data/lib/relaton/bipm/CLAUDE.md +0 -187
- data/lib/relaton/bsi/CLAUDE.md +0 -35
- data/lib/relaton/calconnect/CLAUDE.md +0 -177
- data/lib/relaton/ccsds/CLAUDE.md +0 -74
- data/lib/relaton/cen/CLAUDE.md +0 -219
- data/lib/relaton/cie/CLAUDE.md +0 -90
- data/lib/relaton/core/CLAUDE.md +0 -140
- data/lib/relaton/db/CLAUDE.md +0 -385
- data/lib/relaton/doi/CLAUDE.md +0 -105
- data/lib/relaton/easc/CLAUDE.md +0 -85
- data/lib/relaton/ecma/CLAUDE.md +0 -217
- data/lib/relaton/etsi/CLAUDE.md +0 -219
- data/lib/relaton/evs/CLAUDE.md +0 -120
- data/lib/relaton/gb/CLAUDE.md +0 -47
- data/lib/relaton/gost/CLAUDE.md +0 -100
- data/lib/relaton/iala/CLAUDE.md +0 -97
- data/lib/relaton/iana/CLAUDE.md +0 -105
- data/lib/relaton/iec/CLAUDE.md +0 -13
- data/lib/relaton/ieee/CLAUDE.md +0 -202
- data/lib/relaton/ietf/CLAUDE.md +0 -382
- data/lib/relaton/iho/CLAUDE.md +0 -76
- data/lib/relaton/index/CLAUDE.md +0 -208
- data/lib/relaton/isbn/CLAUDE.md +0 -47
- data/lib/relaton/iso/CLAUDE.md +0 -38
- data/lib/relaton/itu/CLAUDE.md +0 -685
- data/lib/relaton/jcgm/CLAUDE.md +0 -107
- data/lib/relaton/jis/CLAUDE.md +0 -64
- data/lib/relaton/logger/CLAUDE.md +0 -37
- data/lib/relaton/nist/CLAUDE.md +0 -194
- data/lib/relaton/oasis/CLAUDE.md +0 -251
- data/lib/relaton/ogc/CLAUDE.md +0 -242
- data/lib/relaton/oiml/CLAUDE.md +0 -47
- data/lib/relaton/omg/CLAUDE.md +0 -137
- data/lib/relaton/plateau/CLAUDE.md +0 -126
- data/lib/relaton/sae/CLAUDE.md +0 -91
- data/lib/relaton/sdo/CLAUDE.md +0 -81
- data/lib/relaton/un/CLAUDE.md +0 -91
- data/lib/relaton/w3c/CLAUDE.md +0 -369
- data/lib/relaton/xsf/CLAUDE.md +0 -189
data/lib/relaton/adobe/CLAUDE.md
DELETED
|
@@ -1,104 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
Dev notes for the `Relaton::Adobe` flavor. Excluded from the packaged gem via
|
|
4
|
-
the gemspec `files` glob (dev docs only). See the root `CLAUDE.md` for the
|
|
5
|
-
cross-flavor conventions.
|
|
6
|
-
|
|
7
|
-
## What Adobe is
|
|
8
|
-
|
|
9
|
-
`Relaton::Adobe` covers Adobe technical publications: **font tech notes** (the
|
|
10
|
-
`adobe-type-tools/font-tech-notes` repo, numbered `ATN<NNNN>` / "Adobe Technical
|
|
11
|
-
Note #<number>") and **named publications** cited by PDF-Association standards
|
|
12
|
-
(Adobe Glyph List, PostScript Language, the Adobe-Japan1/GB1/CNS1/Korea1/KR-9
|
|
13
|
-
character collections, etc.). Records live in the `relaton-data-adobe` data repo.
|
|
14
|
-
|
|
15
|
-
The flavor mirrors the shape of `Relaton::Iala` — a thin model layer over
|
|
16
|
-
`Relaton::Bib` plus a `Bibliography` retrieval front-end.
|
|
17
|
-
|
|
18
|
-
## Two doctypes
|
|
19
|
-
|
|
20
|
-
`Doctype::TYPES` (`doctype.rb`) is `%w[tech-note publication]`, carried in the
|
|
21
|
-
inherited `content` attribute:
|
|
22
|
-
|
|
23
|
-
- **`tech-note`** — numbered notes; `tech_note_number` + `source_repo_path` set.
|
|
24
|
-
- **`publication`** — slug-keyed named publications; `publication_slug` set.
|
|
25
|
-
|
|
26
|
-
## Typed Ext (`ext.rb`)
|
|
27
|
-
|
|
28
|
-
`Relaton::Adobe::Ext < Bib::Ext` adds Adobe-specific structured metadata so it
|
|
29
|
-
round-trips natively through **both** XML and YAML (no per-repo merge hacks):
|
|
30
|
-
`urn`, `webpage`, `tech_note_number`, `source_repo_path`, `publication_slug`.
|
|
31
|
-
Each is declared in both the `xml` and `key_value` mapping blocks. The elements
|
|
32
|
-
land directly under `<ext>` (alongside the inherited `doctype`/`flavor`); the
|
|
33
|
-
RelaxNG grammar (`grammar/relaton-adobe.rng`) extends `BibDataExtensionType`
|
|
34
|
-
(`combine="interleave"`) with the five optional elements and restricts
|
|
35
|
-
`DocumentType` to the two doctypes. `spec/adobe/relaton/adobe/xml_spec.rb`
|
|
36
|
-
Jing-validates the produced XML against it.
|
|
37
|
-
|
|
38
|
-
## Docidentifier (pubid-backed, mirrors OIML)
|
|
39
|
-
|
|
40
|
-
`Relaton::Adobe::Docidentifier < Bib::Docidentifier` **soft-parses** its `content`
|
|
41
|
-
into a `Pubid::Adobe` identifier held in `@pubid` (a `TechNote` or `Publication`),
|
|
42
|
-
exactly like `Relaton::Oiml::Docidentifier`:
|
|
43
|
-
|
|
44
|
-
- `content=` runs `::Pubid::Adobe.parse` and rescues — a non-canonical string
|
|
45
|
-
(e.g. a title-cased `"Adobe Glyph List"`, vs the canonical
|
|
46
|
-
`"Adobe Publication adobe-glyph-list"`) leaves `@pubid` nil while the plain
|
|
47
|
-
`content` string still round-trips.
|
|
48
|
-
- The base class's abstract `remove_part!` / `remove_date!` / `to_all_parts!`
|
|
49
|
-
(which otherwise raise `NotImplementedError`) are implemented as **safe no-ops**
|
|
50
|
-
over the pubid: Adobe identifiers carry no part/date/all-parts component, so
|
|
51
|
-
these keep `Bib::ItemData#to_most_recent_reference` / `#to_all_parts` from
|
|
52
|
-
raising and never change the rendered `content`.
|
|
53
|
-
|
|
54
|
-
`require "pubid"` (in `adobe.rb`) exposes `Pubid::Adobe`. pubid is already a
|
|
55
|
-
gemspec dependency (shared with iso/oiml/…), so no new dep was added.
|
|
56
|
-
|
|
57
|
-
## Retrieval flow (pubid-backed, mirrors OIML)
|
|
58
|
-
|
|
59
|
-
`Processor#get` (`processor.rb`) → `Bibliography.get` → `Bibliography.search`
|
|
60
|
-
(`bibliography.rb`), modelled on `lib/relaton/oiml/bibliography.rb`:
|
|
61
|
-
|
|
62
|
-
1. Parse the reference with `::Pubid::Adobe.parse` (`pubid_for`); an unparseable
|
|
63
|
-
ref is a **graceful miss** (logged, returns nil) rather than a raised Parslet
|
|
64
|
-
error — it only reached here because it matched the flavor prefix regex.
|
|
65
|
-
2. Look up in the `relaton-data-adobe` `index-v2` via
|
|
66
|
-
`Relaton::Index.find_or_create(:adobe, …, pubid_class: ::Pubid::Adobe::Identifier)`,
|
|
67
|
-
so each row's `:id` is a `Pubid::Adobe` identifier. **Tech notes narrow by
|
|
68
|
-
number** (binary search); **publications have no number**, so `search(nil)`
|
|
69
|
-
scans and the block filters. `pubid_match?` compares `row_id.to_s == query.to_s`
|
|
70
|
-
— the human render is a stable identity key (drops a tech note's filename
|
|
71
|
-
slug so `ATN5014` matches a slug-carrying record; includes a publication's
|
|
72
|
-
slug+version). Unlike OIML there is **no edition-year selection** — each
|
|
73
|
-
reference maps to a single record.
|
|
74
|
-
3. `Net::HTTP.get_response` for the row's `:file` under `ENDPOINT`
|
|
75
|
-
(`raw.githubusercontent.com/relaton/relaton-data-adobe/main/`).
|
|
76
|
-
4. `Relaton::Adobe::Item.from_yaml` → an `ItemData`, date-stamped via `fetched=`.
|
|
77
|
-
|
|
78
|
-
Network/HTTP errors become `Relaton::RequestError`. **Do not** delegate `get`
|
|
79
|
-
back into `Relaton::Db#fetch` — that both mis-arities (`Db#fetch` is 1..3 args)
|
|
80
|
-
and re-enters the dispatcher, so it can never resolve.
|
|
81
|
-
|
|
82
|
-
The `Processor` registers `short = :relaton_adobe`, `prefix = "Adobe"`,
|
|
83
|
-
`defaultprefix = /^(?:Adobe|ATN)/` so both `Adobe Technical Note #5014` and
|
|
84
|
-
`ATN5014` route here.
|
|
85
|
-
|
|
86
|
-
## Status / follow-ups
|
|
87
|
-
|
|
88
|
-
- `Pubid::Adobe` (identifier flavor: `TechNote` + `Publication`, prefixes
|
|
89
|
-
`["Adobe", "ATN"]`) is on pubid `main`, the version this repo pins, and is now
|
|
90
|
-
wired into both `Docidentifier` and `Bibliography`.
|
|
91
|
-
- `relaton-data-adobe` (dataset) is not yet published; retrieval is wired and
|
|
92
|
-
unit-tested offline (webmock + a stubbed pubid-class index), but a live
|
|
93
|
-
end-to-end fetch waits on the data repo. When it lands, confirm rows serialize
|
|
94
|
-
as `_type: pubid:adobe:{tech-note,publication}` (the `ADOBE_TYPE_MAP` pubid
|
|
95
|
-
deserializes via `pubid_class`).
|
|
96
|
-
|
|
97
|
-
## Testing
|
|
98
|
-
|
|
99
|
-
- `bundle exec rake spec:adobe` — run the Adobe suite.
|
|
100
|
-
- Specs live in `spec/adobe/relaton/adobe/` and run self-contained (see the root
|
|
101
|
-
`CLAUDE.md` Testing section). `xml_spec.rb` covers schema validation;
|
|
102
|
-
`docidentifier_spec.rb` covers pubid parsing + the mutation hooks;
|
|
103
|
-
`bibliography_spec.rb` covers the offline retrieval path (tech-note narrowing,
|
|
104
|
-
publication scan, ATN-alias identity, graceful miss).
|
data/lib/relaton/bib/CLAUDE.md
DELETED
|
@@ -1,150 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
RelatonBib is a Ruby gem that implements the [BibliographicItem model](https://github.com/metanorma/relaton-models#bibliography-uml-models) for bibliographic reference management. It's part of the Relaton ecosystem and serves as the base library for other Relaton gems.
|
|
8
|
-
|
|
9
|
-
## Common Commands
|
|
10
|
-
|
|
11
|
-
```bash
|
|
12
|
-
# Install dependencies
|
|
13
|
-
bundle install
|
|
14
|
-
|
|
15
|
-
# Run all tests
|
|
16
|
-
bundle exec rake spec
|
|
17
|
-
|
|
18
|
-
# Run a single test file
|
|
19
|
-
bundle exec rspec spec/relaton_bib/bibliographic_item_spec.rb
|
|
20
|
-
|
|
21
|
-
# Run a specific test by line number
|
|
22
|
-
bundle exec rspec spec/relaton_bib/bibliographic_item_spec.rb:42
|
|
23
|
-
|
|
24
|
-
# Run linting
|
|
25
|
-
bundle exec rubocop
|
|
26
|
-
|
|
27
|
-
# Auto-fix linting issues
|
|
28
|
-
bundle exec rubocop -a
|
|
29
|
-
|
|
30
|
-
# Interactive console
|
|
31
|
-
bin/console
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Architecture
|
|
35
|
-
|
|
36
|
-
### Serialization Layer (lutaml-model)
|
|
37
|
-
|
|
38
|
-
The codebase uses [lutaml-model](https://github.com/lutaml/lutaml-model) for serialization. Each model class in `lib/relaton/bib/model/` inherits from `Lutaml::Model::Serializable` and declares attributes with XML/YAML/JSON mappings.
|
|
39
|
-
|
|
40
|
-
### Core Classes
|
|
41
|
-
|
|
42
|
-
**`Relaton::Bib::Item`** ([lib/relaton/bib/model/item.rb](lib/relaton/bib/model/item.rb)) - The main serialization class defining all bibliographic attributes and their XML mappings. Uses `ItemData` as its underlying model.
|
|
43
|
-
|
|
44
|
-
**`Relaton::Bib::ItemData`** ([lib/relaton/bib/item_data.rb](lib/relaton/bib/item_data.rb)) - The data container class that holds bibliographic data and provides conversion methods (`to_xml`, `to_yaml`, `to_bibtex`, `to_rfcxml`). This separation allows the same data to be serialized differently (bibitem vs bibdata).
|
|
45
|
-
|
|
46
|
-
**`Relaton::Bib::Bibitem`** and **`Relaton::Bib::Bibdata`** - Variants of `Item` with different XML root elements and attributes (bibitem excludes `ext`, bibdata excludes `id`).
|
|
47
|
-
|
|
48
|
-
### Component Models
|
|
49
|
-
|
|
50
|
-
Each bibliographic attribute has its own class in `lib/relaton/bib/model/`:
|
|
51
|
-
|
|
52
|
-
- `Title`, `LocalizedString`, `LocalizedMarkedUpString` - text with language/script
|
|
53
|
-
- `Contributor`, `Person`, `Organization` - contributors and affiliations
|
|
54
|
-
- `Docidentifier` - document IDs (DOI, ISBN, etc.)
|
|
55
|
-
- `Date` - publication dates with custom `StringDate` type
|
|
56
|
-
- `Relation` - related documents (circular reference with Item)
|
|
57
|
-
- `Ext` - extension data (doctype, ICS codes, structured identifiers)
|
|
58
|
-
|
|
59
|
-
### Sanitizing marked-up content
|
|
60
|
-
|
|
61
|
-
**`Relaton::Bib::Sanitizer`** ([lib/relaton/bib/sanitizer.rb](lib/relaton/bib/sanitizer.rb))
|
|
62
|
-
strips markup outside the basicdoc set. `LocalizedMarkedUpString#content=` calls it on
|
|
63
|
-
assignment, so every `Title`, `Abstract`, `Note`, `Formattedref` and `Docidentifier` in
|
|
64
|
-
every flavor passes through it. The set it keeps depends on the field, through
|
|
65
|
-
`#sanitizer_allowed`: every field keeps `ALLOWED` (basicdoc inline markup), and `Abstract`
|
|
66
|
-
keeps `ALLOWED_WITH_BLOCKS`, which adds `BLOCK` — the basicdoc lists, notes, tables, figures
|
|
67
|
-
and admonitions, with the children their grammar requires (`name`, `thead`/`tbody`/`tfoot`,
|
|
68
|
-
`image`). That split follows `biblio.rng`: `<abstract>` takes `BasicBlockNoId`, and a title,
|
|
69
|
-
note, docidentifier or formattedref takes only `PureTextElement`, so do not widen the
|
|
70
|
-
default. Before relaton#421 an abstract's list or note unwrapped to bare `<p>`s with no
|
|
71
|
-
error. A consumer must therefore not assume that an abstract holds only `<p>` and inline
|
|
72
|
-
markup. Two properties are load-bearing:
|
|
73
|
-
|
|
74
|
-
- **It does not give up on an undeclared namespace prefix.** Third-party markup prefixes
|
|
75
|
-
its elements — Crossref returns JATS as `<jats:p><jats:italic>x</jats:italic></jats:p>`,
|
|
76
|
-
sometimes with an `xlink:` prefix on an attribute — and Relaton never declares the prefix.
|
|
77
|
-
Nokogiri reports an undeclared prefix as a parse error, so the old
|
|
78
|
-
`return content if fragment.errors.any?` returned exactly the markup that most needed
|
|
79
|
-
sanitizing. The undeclared prefix then reached the bibdata XML, relaton-render's
|
|
80
|
-
`sanitise_citations_input_string` returned nil, and isodoc raised
|
|
81
|
-
`NoMethodError: undefined method '[]' for nil` (metanorma-pdfa#99). `parse_with_prefixes`
|
|
82
|
-
now declares each prefix on a wrapper element, parses, and removes **only the wrapper's
|
|
83
|
-
own** declarations, matched by identity rather than by URI — `remove_namespaces!` would
|
|
84
|
-
also strip a namespace the content declares itself, such as the MathML `xmlns` inside an
|
|
85
|
-
opaque `<stem>`. `wrapper_name` renames the wrapper while the content contains it, so the
|
|
86
|
-
content cannot close it early — in **one** scan (`NS_WRAPPER_RX`), because growing the
|
|
87
|
-
name and re-testing with `include?` is quadratic, the same trap the root `CLAUDE.md`
|
|
88
|
-
records for relaton-cli's `strip_tags`. An undeclared prefix inside `<stem>` goes as well
|
|
89
|
-
(`<mml:math>` → `<math>`): an undeclared prefix in the output is the failure being fixed.
|
|
90
|
-
Un-prefixing an attribute renames it, so `drop_attribute_namespaces` **drops** a prefixed
|
|
91
|
-
attribute whose plain name the element already carries — keeping both would emit
|
|
92
|
-
`target="a" target="b"`, which Nokogiri rejects as `Attribute target redefined`, and that
|
|
93
|
-
is the very unparseable output this path exists to prevent. Test the collision with
|
|
94
|
-
`plain_attribute?`, not `Node#attribute`, which matches on the name alone and so finds the
|
|
95
|
-
prefixed attribute itself. The guard for genuinely malformed input stays — unbalanced tags,
|
|
96
|
-
a stray `<`, and text that only looks like a prefixed tag are all returned untouched.
|
|
97
|
-
- **It does not indent.** `SAVE_OPTS` drops Nokogiri's `FORMAT` default, so
|
|
98
|
-
`sanitize("<p><em>x</em></p>")` returns the input rather than `"<p>\n <em>x</em>\n</p>"`.
|
|
99
|
-
Don't reintroduce a bare `to_xml`.
|
|
100
|
-
|
|
101
|
-
### Parsing
|
|
102
|
-
|
|
103
|
-
- **`HashParserV1`** ([lib/relaton/bib/hash_parser_v1.rb](lib/relaton/bib/hash_parser_v1.rb)) - Converts legacy Hash/YAML format to `ItemData`
|
|
104
|
-
- **`Converter::BibXml`** ([lib/relaton/bib/converter/bibxml.rb](lib/relaton/bib/converter/bibxml.rb)) - Parses RFC BibXML format via `Relaton::Bib::Converter::BibXml.to_item`
|
|
105
|
-
- **lutaml-model native** - `Item.from_xml`, `Item.from_yaml`, `Item.from_json` handle current format
|
|
106
|
-
|
|
107
|
-
### Rendering
|
|
108
|
-
|
|
109
|
-
- **`Renderer::BibtexBuilder`** - Converts `ItemData` to BibTeX format
|
|
110
|
-
- **`Converter::BibXml`** - Converts `ItemData` to an `Rfcxml::V3` model object via
|
|
111
|
-
`Relaton::Bib::Converter::BibXml.from_item`; call `to_xml` on the result (this is what
|
|
112
|
-
`ItemData#to_rfcxml` does). Two output shapes share one class hierarchy:
|
|
113
|
-
- `ToRfcxml` / `ToRfcxmlReferencegroup` (default) emit **BibXML**, the shape the Relaton
|
|
114
|
-
data repositories publish. It keeps the RFC 7991-deprecated `<format>` element, so it
|
|
115
|
-
must stay byte-compatible — every relaton data fetcher's `"bibxml"` format is
|
|
116
|
-
`entry.to_rfcxml`. Do not change its output.
|
|
117
|
-
- `ToRfcxmlV3` / `ToRfcxmlReferencegroupV3` (`v3: true`) subclass those and emit strict
|
|
118
|
-
**RFC 7991**: no `<format>`, plus `<stream>`, `ascii*` attributes and `<refcontent>`.
|
|
119
|
-
New v3-only behaviour belongs in the subclass, so the default path stays untouched by
|
|
120
|
-
construction and `spec/bib/relaton/bib/converter/bibxml_spec.rb` keeps passing unedited.
|
|
121
|
-
- **lutaml-model native** - `to_xml`, `to_yaml`, `to_json` via the serialization classes
|
|
122
|
-
|
|
123
|
-
### Usage Pattern
|
|
124
|
-
|
|
125
|
-
```ruby
|
|
126
|
-
# Parse from YAML
|
|
127
|
-
item = Relaton::Bib::Item.from_yaml(yaml_string)
|
|
128
|
-
|
|
129
|
-
# Parse from XML
|
|
130
|
-
item = Relaton::Bib::Bibitem.from_xml(xml_string)
|
|
131
|
-
item = Relaton::Bib::Bibdata.from_xml(xml_string)
|
|
132
|
-
|
|
133
|
-
# Parse from RFC XML
|
|
134
|
-
item = Relaton::Bib::Converter::BibXml.to_item(xml_string)
|
|
135
|
-
|
|
136
|
-
# Convert to different formats (returns string)
|
|
137
|
-
item.to_xml # as <bibitem>
|
|
138
|
-
item.to_xml(bibdata: true) # as <bibdata>
|
|
139
|
-
item.to_yaml
|
|
140
|
-
item.to_bibtex
|
|
141
|
-
item.to_rfcxml # BibXML (what the data repos publish)
|
|
142
|
-
item.to_rfcxml(v3: true) # strict RFC 7991 / xml2rfc v3
|
|
143
|
-
item.to_rfcxml(anchor: "ISO712", include_keywords: false)
|
|
144
|
-
```
|
|
145
|
-
|
|
146
|
-
## Code Style
|
|
147
|
-
|
|
148
|
-
- Follows Ribose OSS Ruby style guide (inherited via `.rubocop.yml`)
|
|
149
|
-
- Target Ruby version: 3.3+ (`required_ruby_version` in the gemspec, `TargetRubyVersion` in `.rubocop.yml`)
|
|
150
|
-
- Uses YARD documentation comments
|
data/lib/relaton/bipm/CLAUDE.md
DELETED
|
@@ -1,187 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
relaton-bipm is a Ruby gem that retrieves BIPM (Bureau International des Poids et Mesures) standards metadata for bibliographic use. It's part of the larger Relaton family of gems that handle bibliographic data for standards organizations.
|
|
8
|
-
|
|
9
|
-
## Common Commands
|
|
10
|
-
|
|
11
|
-
- **Install dependencies:** `bundle install`
|
|
12
|
-
- **Run all tests:** `bundle exec rake spec`
|
|
13
|
-
- **Run a single test file:** `bundle exec rspec spec/relaton/bipm/bibliography_spec.rb`
|
|
14
|
-
- **Run a single test by line:** `bundle exec rspec spec/relaton/bipm/bibliography_spec.rb:42`
|
|
15
|
-
- **Interactive console:** `bin/console`
|
|
16
|
-
|
|
17
|
-
## Architecture
|
|
18
|
-
|
|
19
|
-
### Namespace & Module Structure
|
|
20
|
-
|
|
21
|
-
All code lives under `Relaton::Bipm` (in `lib/relaton/bipm/`). The gem name is `relaton-bipm`, the require path is `relaton/bipm`.
|
|
22
|
-
|
|
23
|
-
### Pubid-backed `index-v2` (the runtime index)
|
|
24
|
-
|
|
25
|
-
The flavor's runtime index is the **pubid-backed `index-v2`** (`INDEXFILE =
|
|
26
|
-
"index-v2"`), whose rows are `Pubid::Bipm` identifiers serialized as
|
|
27
|
-
`_type: pubid:bipm:{committee-document,meeting,metrologia-article,si-brochure}`
|
|
28
|
-
— mirroring the JCGM/OIML/JIS convention, not BIPM's old bespoke
|
|
29
|
-
`{group,type,number,year}` hash. The `DataFetcher` builds it (`pubid_class:
|
|
30
|
-
::Pubid::Bipm::Identifier`), and the consumer reads it.
|
|
31
|
-
|
|
32
|
-
The **legacy bespoke `index-v1`** (the old `{group,type,number,year}` hash form)
|
|
33
|
-
is no longer produced or read by this flavor. It is still published by
|
|
34
|
-
`relaton-data-bipm`'s own `crawler.rb`, which builds it with the **retained**
|
|
35
|
-
`Relaton::Bipm::Id` (this flavor keeps `Id` public for exactly that). So `Id` is
|
|
36
|
-
now used in **one place only** — the crawler's `index-v1` generation. It was
|
|
37
|
-
also this flavor's consumer query parser until pubid's grammar widened enough to
|
|
38
|
-
replace it; see **Pubid-to-pubid lookup** below.
|
|
39
|
-
|
|
40
|
-
### Key Components
|
|
41
|
-
|
|
42
|
-
- **`Bibliography`** (`bibliography.rb`) - Main entry point. Parses the user
|
|
43
|
-
reference with `::Pubid::Bipm.parse` (`#parse_ref`; an unparseable ref raises
|
|
44
|
-
`Pubid::Errors::ParseError`, while the machine-derived `#year_number_retry`
|
|
45
|
-
probe rescues it to nil) and matches the pubid `index-v2` rows as
|
|
46
|
-
pubid objects — see **Pubid-to-pubid lookup** below for the stem and the two
|
|
47
|
-
escapes under it. The latest edition wins via
|
|
48
|
-
`max_by { |r| [r[:id].year.to_i, r[:file]] }` (the file path only breaks a
|
|
49
|
-
tie — Metrologia and SI Brochure rows carry no year, so all 6206 of them
|
|
50
|
-
score 0 and the index sort is not stable).
|
|
51
|
-
- **`Id`** (`id_parser.rb`) - The flexible bespoke regex parser. Parses BIPM
|
|
52
|
-
reference strings into `{group,type,number,year,…}` hashes; the `TYPES` hash
|
|
53
|
-
maps full EN/FR type names to abbreviations, and `normalize_hash` applies
|
|
54
|
-
`CCDS→CCTF` and leading-zero stripping. **Retained solely** as the
|
|
55
|
-
`relaton-data-bipm` crawler's `index-v1` builder — it no longer parses
|
|
56
|
-
queries, and it is **not** used to write this flavor's `index-v2`.
|
|
57
|
-
`bibliography.rb` does not require it; `lib/relaton/bipm.rb` does, which is
|
|
58
|
-
what keeps `Relaton::Bipm::Id` reachable for the crawler. JCGM is not handled
|
|
59
|
-
here.
|
|
60
|
-
- **`Processor`** (`processor.rb`) - Relaton framework integration point (extends
|
|
61
|
-
`Relaton::Core::Processor`). Registers prefix `BIPM` and default prefix pattern
|
|
62
|
-
matching BIPM, CCTF, CCDS, CGPM, CIPM, JCRB (JCGM was split out into the `jcgm`
|
|
63
|
-
flavor). `remove_index_file` targets `index-v2`.
|
|
64
|
-
- **`DataFetcher`** (`data_fetcher.rb`) - Bulk fetches from three data sources:
|
|
65
|
-
`bipm-data-outcomes`, `bipm-si-brochure`, `rawdata-bipm-metrologia`. Delegates
|
|
66
|
-
to specialized parsers, which route every index write through the public,
|
|
67
|
-
**guarded** `#add_to_index(docnumber, path)` — it parses `docnumber` with
|
|
68
|
-
`::Pubid::Bipm.parse` and stores the identifier object, or **warns and skips**
|
|
69
|
-
an unparseable id rather than aborting the crawl (mirrors
|
|
70
|
-
`Relaton::Jcgm::DataFetcher#add_to_index`; `Relaton::Index` rejects a whole
|
|
71
|
-
index if one row fails to deserialize). Public so `relaton-data-bipm`'s crawler
|
|
72
|
-
can index its curated `static/` docs through the same guarded path.
|
|
73
|
-
- **`Item`** / **`ItemData`** (`model/item.rb`, `item_data.rb`) - The
|
|
74
|
-
bibliographic item model, extending `Relaton::Bib::Item`. Supports XML, YAML,
|
|
75
|
-
and JSON serialization.
|
|
76
|
-
- **`model/`** directory - Lutaml model classes (Bibdata, Bibitem, Ext, etc.) for
|
|
77
|
-
XML/YAML serialization.
|
|
78
|
-
|
|
79
|
-
### Pubid-to-pubid lookup
|
|
80
|
-
|
|
81
|
-
`#get_bipm` parses the reference with `::Pubid::Bipm.parse` and matches the
|
|
82
|
-
index rows as pubid objects — the same shape ISO, OIML and JCGM use. The
|
|
83
|
-
bespoke `Id` grammar takes **no part in a query**. That became possible with
|
|
84
|
-
pubid `a96e9f45`, which accepts the loose consumer forms this flavor once
|
|
85
|
-
needed `Id` for and *normalizes* them to BIPM's canonical spelling
|
|
86
|
-
(`CCDS Recommendation 2 (2009)` → `CCTF Recommendation 2 (2009)`,
|
|
87
|
-
`CIPM 111e Réunion (2022)` → `CIPM 111<sup>e</sup> réunion (2022)`).
|
|
88
|
-
|
|
89
|
-
**The stem.** Committee-document and meeting rows are stored language- and
|
|
90
|
-
form-neutral, while a reference may name a language (`(2009, E)`) and always
|
|
91
|
-
names a form (short `CCTF REC 2` vs long `CCTF Recommendation 2`). So
|
|
92
|
-
`#pubid_match?` compares `exclude(:language, :form)`, adding `:year` when the
|
|
93
|
-
**query** carries none — `CIPM Meeting 43` has no year, its row has 1950. That
|
|
94
|
-
last clause is what `Id#==` did with
|
|
95
|
-
`other_hash.delete(:year) unless hash[:year]`; without it the lookup regresses.
|
|
96
|
-
|
|
97
|
-
**Two escapes sit under the narrowing**, each for a case where the query's
|
|
98
|
-
bsearch key cannot equal its row's:
|
|
99
|
-
|
|
100
|
-
1. **A bare `SI Brochure` names no edition**, so it keys to `""` while its row
|
|
101
|
-
keys to `"9e"`. `#search_index` rescans the whole index. A brochure query
|
|
102
|
-
with a nil edition is a *partial* reference and deliberately matches any
|
|
103
|
-
brochure row — what the old `{group: "SI", type: "Brochure"}` projection
|
|
104
|
-
meant. `Part N` names a section inside the brochure, not a document, so it
|
|
105
|
-
stays out of the key exactly as `Id#==` kept `:part` out.
|
|
106
|
-
2. **`CCTF Recommendation 2009-02` parses as the literal number `2009-02`**,
|
|
107
|
-
while the row keys on `2`. A rescan cannot help — no row carries that number
|
|
108
|
-
— so `#year_number_retry` re-reads a trailing `YYYY-NN` as number plus year.
|
|
109
|
-
It runs **only after a miss**, because `NNNN-NN` is also a real BIPM number
|
|
110
|
-
(`CIPM 2005-06`), so the literal reading must win.
|
|
111
|
-
|
|
112
|
-
**One rule survives from `Id#==` and must not be dropped.** It treated a
|
|
113
|
-
document numbered `"1"` and a number-less one as the same document when both
|
|
114
|
-
carried a year, so `CIPM Resolution 1 (1879)` reached `CIPM RES (1879)`. pubid
|
|
115
|
-
has no such rule, and the six ordinal-less declarations
|
|
116
|
-
(`CGPM DECL (1889)`, `CIPM RES (1879)`, …) are the only rows it applies to — but
|
|
117
|
-
they are addressable *both* ways, so removing it silently broke
|
|
118
|
-
`CIPM Resolution 1 (1879)` and `CGPM Declaration 1 (1889)`. `#number_collapses?`
|
|
119
|
-
restores it by excluding `:number` from that one comparison. Both forms are in
|
|
120
|
-
the pinned baseline. Note the corpus has **no two rows** the rule could join
|
|
121
|
-
(6 number-less against 83 numbered `"1"`, no shared group/type/year), so it only
|
|
122
|
-
ever widens a query, never merges records.
|
|
123
|
-
|
|
124
|
-
**Cost.** `#exclude` copies the identifier, so reducing per candidate row is
|
|
125
|
-
expensive — roughly **165x** a plain attribute read. Two things keep the lookup
|
|
126
|
-
at or below what the `Id` grammar cost: the query is reduced **once per search**,
|
|
127
|
-
not once per row, and `#pubid_match?` rejects on `CHEAP_KEYS`
|
|
128
|
-
(`number group year issue article`) before reducing anything. Those are exactly
|
|
129
|
-
the attributes the stem never removes, so stem equality implies all of them and
|
|
130
|
-
the rejects cannot change an answer — a spec asserts that over the corpus rather
|
|
131
|
-
than trusting the argument. Without both, a bare-brochure rescan built ~7,900
|
|
132
|
-
copies and took **2.3 s**; with them the same lookup is 2.8 ms and a narrowed
|
|
133
|
-
one 0.4–0.5 ms, against 0.5–0.7 ms under `Id`.
|
|
134
|
-
|
|
135
|
-
**The regression guard.** `spec/bipm/relaton/bipm/bibliography_spec.rb` pins,
|
|
136
|
-
literally, the file each of the 29 references the suite exercises resolved to
|
|
137
|
-
under the `Id` grammar, captured before it was removed. Any change to matching
|
|
138
|
-
that moves a lookup fails there by name. That is the evidence for the swap —
|
|
139
|
-
`Id#==` was fuzzier than the stem match in ways no spec pinned (it collapsed
|
|
140
|
-
number `"1"` with nil, dropped `:part`/`:append`), so equivalence had to be
|
|
141
|
-
demonstrated rather than argued.
|
|
142
|
-
|
|
143
|
-
`Relaton::Bipm::Id` and `id_parser.rb` **stay** — `relaton-data-bipm`'s crawler
|
|
144
|
-
builds `index-v1` with them, and `id_parser_spec.rb` still covers them. Only
|
|
145
|
-
`bibliography.rb` stopped requiring the file.
|
|
146
|
-
|
|
147
|
-
### Internal identifier
|
|
148
|
-
|
|
149
|
-
`docid.to_s(form: :internal)` drops the
|
|
150
|
-
`BIPM` publisher (relaton#49): `BIPM SI Brochure Part 1` →
|
|
151
|
-
`SI Brochure Part 1`. Committee ids (`CGPM RES (1889)`, `CCTF REC 2 (2009)`)
|
|
152
|
-
have no publisher token and are the same in both forms. The rule is the `"BIPM"` row of
|
|
153
|
-
`Bib::Docidentifier::FORM_PUBID`: BIPM docidentifiers carry no pubid,
|
|
154
|
-
so the content is parsed with `Pubid::Bipm` at call time and rendered with
|
|
155
|
-
`to_s(with_publisher: false)`; unparseable content is returned unchanged. It
|
|
156
|
-
also holds for a relation's nested item. Do not move it into a `Bipm::Docidentifier`
|
|
157
|
-
subclass: typing the item's `docidentifier` collection with a subclass makes
|
|
158
|
-
YAML serialization reject the `Bib::Docidentifier` objects the parsers build
|
|
159
|
-
(`IncorrectModelError`).
|
|
160
|
-
|
|
161
|
-
### Data Sources
|
|
162
|
-
|
|
163
|
-
The gem fetches from three external datasets:
|
|
164
|
-
|
|
165
|
-
1. **bipm-data-outcomes** - CGPM/CIPM/committee resolutions, recommendations, decisions (the `jcgm` meeting dir is now harvested by the `Relaton::Jcgm::DataFetcher` instead)
|
|
166
|
-
2. **bipm-si-brochure** - SI Brochure documents
|
|
167
|
-
3. **rawdata-bipm-metrologia** - Metrologia journal articles (parsed from CrossRef-style data)
|
|
168
|
-
|
|
169
|
-
### Testing
|
|
170
|
-
|
|
171
|
-
- **Index fixture:** `spec/bipm/fixtures/index-v2.zip` (the pubid `index-v2`) is
|
|
172
|
-
deserialized via `Pubid::Bipm::Identifier` and pre-loaded into the
|
|
173
|
-
`Relaton::Index` pool in `before(:suite)` (`spec/bipm/support/webmock.rb`,
|
|
174
|
-
mirroring the JCGM/JIS setup). It was regenerated from the real
|
|
175
|
-
`relaton-data-bipm` docs' docnumbers (each parsed with `Pubid::Bipm`); the
|
|
176
|
-
handful that pubid can't parse — JCGM orphans, `CIPM 2005-06`, docs removed
|
|
177
|
-
upstream — are absent by design.
|
|
178
|
-
- **Parser specs** assert the parsers call `data_fetcher.add_to_index(docnumber,
|
|
179
|
-
path)` with the right docnumber string (the pubid parsing/guarding lives on the
|
|
180
|
-
mocked `DataFetcher`, keeping the parser tests format-agnostic).
|
|
181
|
-
- Tests use RSpec with VCR cassettes (`spec/bipm/vcr_cassettes/`) and WebMock.
|
|
182
|
-
|
|
183
|
-
### Dependencies
|
|
184
|
-
|
|
185
|
-
Key runtime dependencies: `pubid` (provides `Pubid::Bipm`), `relaton-bib` (core
|
|
186
|
-
bibliographic model), `relaton-index` (index management), `relaton-core`
|
|
187
|
-
(framework base), `parslet`, `mechanize` (HTTP), `faraday`, `rubyzip`.
|
data/lib/relaton/bsi/CLAUDE.md
DELETED
|
@@ -1,35 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
-
|
|
5
|
-
## Project Overview
|
|
6
|
-
|
|
7
|
-
relaton-bsi retrieves British Standards Institution (BSI) bibliographic data using the Relaton model. Unlike index-backed flavors, it searches the BSI Shopify storefront: an **Algolia** product index finds candidate standards, then a **GraphQL** query against the Shopify storefront API fetches full metadata (titles, dates, ISBNs, committee info). Depends on relaton-iso for its ISO-based item/extension models.
|
|
8
|
-
|
|
9
|
-
## Development
|
|
10
|
-
|
|
11
|
-
```bash
|
|
12
|
-
bundle exec rake # default task → rspec
|
|
13
|
-
bundle exec rspec spec/relaton/bsi/bibliography_spec.rb # single file
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
(No gem-local rubocop config; the monorepo root config applies.)
|
|
17
|
-
|
|
18
|
-
## Architecture
|
|
19
|
-
|
|
20
|
-
Namespace: `Relaton::Bsi`. Retrieval flow:
|
|
21
|
-
|
|
22
|
-
1. **Bibliography** (`lib/relaton/bsi/bibliography.rb`) — `get(code, year, opts)` / `search` entry points. Reference handling is **entirely `Pubid::Bsi`** (from the unified `pubid` gem) with **no hand-rolled identifier regex**: `parse` wraps `Pubid::Bsi::Identifier.parse` and **raises** on an unparseable query (only `Hit#pubid` rescues to nil — the Algolia catalogue carries codes such as `BS 381C SET:1996 (R2002)`, and one such row must not abort the search), `publication_year` reads `pubid.root.year` (not `base_document.year`: pubid holds an adopted standard such as `BS EN ISO 8848:2021` in `base`, so its `base_document.year` is nil; over pubid's 1,536 parseable BSI fixture ids `root.year` equals the old reader on 1,529 and fills in the base year on the other 7), and `same_reference?(query, hit, skip_rest:, drop_amd:)` decides whether a query and a candidate hit denote the same document. `same_reference?` uses only pubid's typed API — `#base_document` (peels supplements + Expert-commentary / Flex-version wrappers), `#matches?(other, ignore: %i[date month])` for the base-document comparison (`:month` is excluded alongside `:date` because Flex stores the month separately, so `exclude(:date)` alone leaves them unequal), and the `Amendment`/`Corrigendum` `#supplement_type`, `#number` and `#year` (amendment year optional when the query omits it). pubid removes the `#supplement_number`/`#supplement_year` aliases, but the locked pubid still has them and leaves a BSI supplement's `#number`/`#year` nil, so the private `supplement_number`/`supplement_year` helpers read the alias while it exists. Remove that fallback once pubid `main` has no alias. A canonical `rest_marker` (`:expert_commentary` / `:"flex_<edition>"`) makes an `ExComm` query match an `Expert Commentary` hit. `search` still lightly normalizes the **Algolia query text** (drops `BSI `, abbreviates the Expert-commentary suffix) — search-engine tuning, not identifier parsing. This depends on the `pubid` `feat/bsi-parse-gaps` work (see `HANDOFFS/metanorma__pubid.md`): the three parse/exclude gaps **plus** the Phase-2 API (`#base_document`, `exclude(:amendment|:supplement|:edition)`, uniform supplement accessors, `#matches?`).
|
|
23
|
-
2. **HitCollection** (`lib/relaton/bsi/hit_collection.rb`, extends `Relaton::Core::HitCollection`) — queries the Algolia index (`shopify_products`) and filters hits by parsing each hit code and calling `Bibliography.same_reference?` against the parsed query, in three passes: exact → `skip_rest` (ignore the ExComm/Flex-version suffix) → `drop_amd` (compare base documents only, and rewrite the hit id to `hit.base_document.to_s`). Algolia keys are public/client-side.
|
|
24
|
-
3. **Hit** (`lib/relaton/bsi/hit.rb`, extends `Relaton::Core::Hit`) — `item` lazy-loads the full document via the scraper; `pubid` memoizes the hit code parsed with `Pubid::Bsi` (like `Relaton::Iso::Hit#pubid`) so the three filter passes don't re-parse it, and is nil for a code pubid cannot parse. `Bibliography#results_filter` reads the year from this memoized pubid too, not from the (possibly `drop_amd`-rewritten) `hit[:code]`.
|
|
25
|
-
4. **Scraper** (`lib/relaton/bsi/scraper.rb`) — GraphQL client; runs schema-based queries against the Shopify storefront GraphQL endpoint to build a full record (titles split on em-dash into intro/main/part, dates, contributors, ICS, source URI).
|
|
26
|
-
5. **ItemData / Model::Item** (`item_data.rb`, `model/`) — `Relaton::Bsi::ItemData` extends `Relaton::Iso::ItemData`; `Ext` adds doctype, ICS, and a structured identifier. **Docidentifier** (`model/docidentifier.rb`) keeps the parsed `Pubid::Bsi::Identifier` as the **single stored value** of `content` (via the flavor-agnostic `Iso::Type::Pubid` lutaml type, which preserves the instance in and stringifies it out) — one source of truth: `#pubid` returns the instance, `#content` renders it to a string. `#remove_date!` → `pubid.exclude(:date, :month)`, `#remove_part!` → `pubid.exclude(:part, :subpart)`. `#to_all_parts!` wraps `pubid` in pubid's `AllParts` (`self.content = pubid.exclude(:supplement_year).to_all_parts`) — since BSI has no dedicated pubid `AllParts` subclass, the wrapper is the generic `Pubid::AllPartsIdentifier`, whose `#to_s` DOES print a "(all parts)" marker (unlike BSI's own renderer, which never had one). The `exclude(:supplement_year)` pre-step matters on a consolidated id (`BS 7273-4:2015+A1:2021`): pubid protects an `Amendment`/`Corrigendum`'s own date from a bare `exclude(:date)` — the same protection `#remove_date!` relies on to drop the base year while keeping the amendment's — but `to_all_parts`'s identity computation is built on that same `exclude`, so without the explicit `:supplement_year` key the amendment's year survives into the "(all parts)" form, distinguishing editions that marker is meant to collapse (`BS 7273+A1 (all parts)`, not `BS 7273+A1:2021 (all parts)`). Because `content` and `#pubid` are the same stored value here, both now show the "(all parts)" marker after `#to_all_parts!` — unlike flavors with a separately cached content string, which keep `content` marker-less on purpose. `#pubid`'s type check is widened to also accept `::Pubid::AllParts` (not just `Pubid::Bsi::Identifier`) for this reason. All three no-op when `#pubid` is nil. Non-BSI identifiers (e.g. ISBN) and anything pubid can't parse are stored verbatim as plain strings.
|
|
27
|
-
6. **Processor** (`lib/relaton/bsi/processor.rb`) — registry integration; lazy-`require_relative`s `../bsi` in its methods.
|
|
28
|
-
|
|
29
|
-
## External dependencies
|
|
30
|
-
|
|
31
|
-
`algolia ~> 2.3.0`, `graphql ~> 2.3`, `graphql-client ~> 0.23`, `faraday-net_http_persistent ~> 2.0`, `relaton-core`, `relaton-iso`.
|
|
32
|
-
|
|
33
|
-
## Testing
|
|
34
|
-
|
|
35
|
-
RSpec with WebMock + VCR (cassettes in `spec/vcr_cassettes/`, record `:once`, never auto-rerecorded). Network is blocked via WebMock. No index fixture — BSI has no curated index; `spec/fixtures/` holds XML/YAML round-trip data.
|