relaton 3.1.1 → 3.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. data/README.adoc +5 -3
  3. data/lib/relaton/3gpp.rb +1 -1
  4. data/lib/relaton/bib/converter/csl/from_csl.rb +195 -0
  5. data/lib/relaton/bib/converter/csl.rb +13 -1
  6. data/lib/relaton/bib/model/docidentifier.rb +66 -5
  7. data/lib/relaton/calconnect/hit_collection.rb +1 -1
  8. data/lib/relaton/cli/command.rb +3 -1
  9. data/lib/relaton/cli/frontend_assets.rb +7 -6
  10. data/lib/relaton/cli/subcommand_collection.rb +5 -1
  11. data/lib/relaton/core/request_error.rb +18 -0
  12. data/lib/relaton/db/config.rb +9 -1
  13. data/lib/relaton/db/workers_pool.rb +11 -1
  14. data/lib/relaton/db.rb +160 -58
  15. data/lib/relaton/ecma/bibliography.rb +2 -2
  16. data/lib/relaton/etsi/data_fetcher.rb +16 -6
  17. data/lib/relaton/iala/bibliography.rb +1 -1
  18. data/lib/relaton/iana.rb +1 -1
  19. data/lib/relaton/iec/data_parser.rb +41 -2
  20. data/lib/relaton/itu/README.adoc +1 -1
  21. data/lib/relaton/itu/data_crawler_r.rb +1 -1
  22. data/lib/relaton/jis/README.adoc +2 -0
  23. data/lib/relaton/jis/data_fetcher.rb +74 -13
  24. data/lib/relaton/jis/hit_collection.rb +2 -1
  25. data/lib/relaton/jis/processor.rb +1 -1
  26. data/lib/relaton/jis/scraper.rb +46 -5
  27. data/lib/relaton/oasis/bibliography.rb +1 -1
  28. data/lib/relaton/ogc/hit_collection.rb +1 -1
  29. data/lib/relaton/ogc.rb +1 -1
  30. data/lib/relaton/plateau/data_fetcher.rb +10 -1
  31. data/lib/relaton/plateau/handbook_parser.rb +38 -2
  32. data/lib/relaton/plateau/hit_collection.rb +20 -5
  33. data/lib/relaton/sae.rb +1 -1
  34. data/lib/relaton/version.rb +1 -1
  35. data/lib/relaton/w3c/bibliography.rb +1 -1
  36. data/lib/relaton/xsf/bibliography.rb +1 -1
  37. data/lib/relaton/xsf.rb +1 -1
  38. metadata +35 -48
  39. data/lib/relaton/3gpp/CLAUDE.md +0 -218
  40. data/lib/relaton/adobe/CLAUDE.md +0 -104
  41. data/lib/relaton/bib/CLAUDE.md +0 -150
  42. data/lib/relaton/bipm/CLAUDE.md +0 -187
  43. data/lib/relaton/bsi/CLAUDE.md +0 -35
  44. data/lib/relaton/calconnect/CLAUDE.md +0 -177
  45. data/lib/relaton/ccsds/CLAUDE.md +0 -74
  46. data/lib/relaton/cen/CLAUDE.md +0 -219
  47. data/lib/relaton/cie/CLAUDE.md +0 -90
  48. data/lib/relaton/core/CLAUDE.md +0 -140
  49. data/lib/relaton/db/CLAUDE.md +0 -385
  50. data/lib/relaton/doi/CLAUDE.md +0 -105
  51. data/lib/relaton/easc/CLAUDE.md +0 -85
  52. data/lib/relaton/ecma/CLAUDE.md +0 -217
  53. data/lib/relaton/etsi/CLAUDE.md +0 -219
  54. data/lib/relaton/evs/CLAUDE.md +0 -120
  55. data/lib/relaton/gb/CLAUDE.md +0 -47
  56. data/lib/relaton/gost/CLAUDE.md +0 -100
  57. data/lib/relaton/iala/CLAUDE.md +0 -97
  58. data/lib/relaton/iana/CLAUDE.md +0 -105
  59. data/lib/relaton/iec/CLAUDE.md +0 -13
  60. data/lib/relaton/ieee/CLAUDE.md +0 -202
  61. data/lib/relaton/ietf/CLAUDE.md +0 -382
  62. data/lib/relaton/iho/CLAUDE.md +0 -76
  63. data/lib/relaton/index/CLAUDE.md +0 -208
  64. data/lib/relaton/isbn/CLAUDE.md +0 -47
  65. data/lib/relaton/iso/CLAUDE.md +0 -38
  66. data/lib/relaton/itu/CLAUDE.md +0 -685
  67. data/lib/relaton/jcgm/CLAUDE.md +0 -107
  68. data/lib/relaton/jis/CLAUDE.md +0 -64
  69. data/lib/relaton/logger/CLAUDE.md +0 -37
  70. data/lib/relaton/nist/CLAUDE.md +0 -194
  71. data/lib/relaton/oasis/CLAUDE.md +0 -251
  72. data/lib/relaton/ogc/CLAUDE.md +0 -242
  73. data/lib/relaton/oiml/CLAUDE.md +0 -47
  74. data/lib/relaton/omg/CLAUDE.md +0 -137
  75. data/lib/relaton/plateau/CLAUDE.md +0 -126
  76. data/lib/relaton/sae/CLAUDE.md +0 -91
  77. data/lib/relaton/sdo/CLAUDE.md +0 -81
  78. data/lib/relaton/un/CLAUDE.md +0 -91
  79. data/lib/relaton/w3c/CLAUDE.md +0 -369
  80. data/lib/relaton/xsf/CLAUDE.md +0 -189
@@ -1,104 +0,0 @@
1
- # CLAUDE.md
2
-
3
- Dev notes for the `Relaton::Adobe` flavor. Excluded from the packaged gem via
4
- the gemspec `files` glob (dev docs only). See the root `CLAUDE.md` for the
5
- cross-flavor conventions.
6
-
7
- ## What Adobe is
8
-
9
- `Relaton::Adobe` covers Adobe technical publications: **font tech notes** (the
10
- `adobe-type-tools/font-tech-notes` repo, numbered `ATN<NNNN>` / "Adobe Technical
11
- Note #<number>") and **named publications** cited by PDF-Association standards
12
- (Adobe Glyph List, PostScript Language, the Adobe-Japan1/GB1/CNS1/Korea1/KR-9
13
- character collections, etc.). Records live in the `relaton-data-adobe` data repo.
14
-
15
- The flavor mirrors the shape of `Relaton::Iala` — a thin model layer over
16
- `Relaton::Bib` plus a `Bibliography` retrieval front-end.
17
-
18
- ## Two doctypes
19
-
20
- `Doctype::TYPES` (`doctype.rb`) is `%w[tech-note publication]`, carried in the
21
- inherited `content` attribute:
22
-
23
- - **`tech-note`** — numbered notes; `tech_note_number` + `source_repo_path` set.
24
- - **`publication`** — slug-keyed named publications; `publication_slug` set.
25
-
26
- ## Typed Ext (`ext.rb`)
27
-
28
- `Relaton::Adobe::Ext < Bib::Ext` adds Adobe-specific structured metadata so it
29
- round-trips natively through **both** XML and YAML (no per-repo merge hacks):
30
- `urn`, `webpage`, `tech_note_number`, `source_repo_path`, `publication_slug`.
31
- Each is declared in both the `xml` and `key_value` mapping blocks. The elements
32
- land directly under `<ext>` (alongside the inherited `doctype`/`flavor`); the
33
- RelaxNG grammar (`grammar/relaton-adobe.rng`) extends `BibDataExtensionType`
34
- (`combine="interleave"`) with the five optional elements and restricts
35
- `DocumentType` to the two doctypes. `spec/adobe/relaton/adobe/xml_spec.rb`
36
- Jing-validates the produced XML against it.
37
-
38
- ## Docidentifier (pubid-backed, mirrors OIML)
39
-
40
- `Relaton::Adobe::Docidentifier < Bib::Docidentifier` **soft-parses** its `content`
41
- into a `Pubid::Adobe` identifier held in `@pubid` (a `TechNote` or `Publication`),
42
- exactly like `Relaton::Oiml::Docidentifier`:
43
-
44
- - `content=` runs `::Pubid::Adobe.parse` and rescues — a non-canonical string
45
- (e.g. a title-cased `"Adobe Glyph List"`, vs the canonical
46
- `"Adobe Publication adobe-glyph-list"`) leaves `@pubid` nil while the plain
47
- `content` string still round-trips.
48
- - The base class's abstract `remove_part!` / `remove_date!` / `to_all_parts!`
49
- (which otherwise raise `NotImplementedError`) are implemented as **safe no-ops**
50
- over the pubid: Adobe identifiers carry no part/date/all-parts component, so
51
- these keep `Bib::ItemData#to_most_recent_reference` / `#to_all_parts` from
52
- raising and never change the rendered `content`.
53
-
54
- `require "pubid"` (in `adobe.rb`) exposes `Pubid::Adobe`. pubid is already a
55
- gemspec dependency (shared with iso/oiml/…), so no new dep was added.
56
-
57
- ## Retrieval flow (pubid-backed, mirrors OIML)
58
-
59
- `Processor#get` (`processor.rb`) → `Bibliography.get` → `Bibliography.search`
60
- (`bibliography.rb`), modelled on `lib/relaton/oiml/bibliography.rb`:
61
-
62
- 1. Parse the reference with `::Pubid::Adobe.parse` (`pubid_for`); an unparseable
63
- ref is a **graceful miss** (logged, returns nil) rather than a raised Parslet
64
- error — it only reached here because it matched the flavor prefix regex.
65
- 2. Look up in the `relaton-data-adobe` `index-v2` via
66
- `Relaton::Index.find_or_create(:adobe, …, pubid_class: ::Pubid::Adobe::Identifier)`,
67
- so each row's `:id` is a `Pubid::Adobe` identifier. **Tech notes narrow by
68
- number** (binary search); **publications have no number**, so `search(nil)`
69
- scans and the block filters. `pubid_match?` compares `row_id.to_s == query.to_s`
70
- — the human render is a stable identity key (drops a tech note's filename
71
- slug so `ATN5014` matches a slug-carrying record; includes a publication's
72
- slug+version). Unlike OIML there is **no edition-year selection** — each
73
- reference maps to a single record.
74
- 3. `Net::HTTP.get_response` for the row's `:file` under `ENDPOINT`
75
- (`raw.githubusercontent.com/relaton/relaton-data-adobe/main/`).
76
- 4. `Relaton::Adobe::Item.from_yaml` → an `ItemData`, date-stamped via `fetched=`.
77
-
78
- Network/HTTP errors become `Relaton::RequestError`. **Do not** delegate `get`
79
- back into `Relaton::Db#fetch` — that both mis-arities (`Db#fetch` is 1..3 args)
80
- and re-enters the dispatcher, so it can never resolve.
81
-
82
- The `Processor` registers `short = :relaton_adobe`, `prefix = "Adobe"`,
83
- `defaultprefix = /^(?:Adobe|ATN)/` so both `Adobe Technical Note #5014` and
84
- `ATN5014` route here.
85
-
86
- ## Status / follow-ups
87
-
88
- - `Pubid::Adobe` (identifier flavor: `TechNote` + `Publication`, prefixes
89
- `["Adobe", "ATN"]`) is on pubid `main`, the version this repo pins, and is now
90
- wired into both `Docidentifier` and `Bibliography`.
91
- - `relaton-data-adobe` (dataset) is not yet published; retrieval is wired and
92
- unit-tested offline (webmock + a stubbed pubid-class index), but a live
93
- end-to-end fetch waits on the data repo. When it lands, confirm rows serialize
94
- as `_type: pubid:adobe:{tech-note,publication}` (the `ADOBE_TYPE_MAP` pubid
95
- deserializes via `pubid_class`).
96
-
97
- ## Testing
98
-
99
- - `bundle exec rake spec:adobe` — run the Adobe suite.
100
- - Specs live in `spec/adobe/relaton/adobe/` and run self-contained (see the root
101
- `CLAUDE.md` Testing section). `xml_spec.rb` covers schema validation;
102
- `docidentifier_spec.rb` covers pubid parsing + the mutation hooks;
103
- `bibliography_spec.rb` covers the offline retrieval path (tech-note narrowing,
104
- publication scan, ATN-alias identity, graceful miss).
@@ -1,150 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- RelatonBib is a Ruby gem that implements the [BibliographicItem model](https://github.com/metanorma/relaton-models#bibliography-uml-models) for bibliographic reference management. It's part of the Relaton ecosystem and serves as the base library for other Relaton gems.
8
-
9
- ## Common Commands
10
-
11
- ```bash
12
- # Install dependencies
13
- bundle install
14
-
15
- # Run all tests
16
- bundle exec rake spec
17
-
18
- # Run a single test file
19
- bundle exec rspec spec/relaton_bib/bibliographic_item_spec.rb
20
-
21
- # Run a specific test by line number
22
- bundle exec rspec spec/relaton_bib/bibliographic_item_spec.rb:42
23
-
24
- # Run linting
25
- bundle exec rubocop
26
-
27
- # Auto-fix linting issues
28
- bundle exec rubocop -a
29
-
30
- # Interactive console
31
- bin/console
32
- ```
33
-
34
- ## Architecture
35
-
36
- ### Serialization Layer (lutaml-model)
37
-
38
- The codebase uses [lutaml-model](https://github.com/lutaml/lutaml-model) for serialization. Each model class in `lib/relaton/bib/model/` inherits from `Lutaml::Model::Serializable` and declares attributes with XML/YAML/JSON mappings.
39
-
40
- ### Core Classes
41
-
42
- **`Relaton::Bib::Item`** ([lib/relaton/bib/model/item.rb](lib/relaton/bib/model/item.rb)) - The main serialization class defining all bibliographic attributes and their XML mappings. Uses `ItemData` as its underlying model.
43
-
44
- **`Relaton::Bib::ItemData`** ([lib/relaton/bib/item_data.rb](lib/relaton/bib/item_data.rb)) - The data container class that holds bibliographic data and provides conversion methods (`to_xml`, `to_yaml`, `to_bibtex`, `to_rfcxml`). This separation allows the same data to be serialized differently (bibitem vs bibdata).
45
-
46
- **`Relaton::Bib::Bibitem`** and **`Relaton::Bib::Bibdata`** - Variants of `Item` with different XML root elements and attributes (bibitem excludes `ext`, bibdata excludes `id`).
47
-
48
- ### Component Models
49
-
50
- Each bibliographic attribute has its own class in `lib/relaton/bib/model/`:
51
-
52
- - `Title`, `LocalizedString`, `LocalizedMarkedUpString` - text with language/script
53
- - `Contributor`, `Person`, `Organization` - contributors and affiliations
54
- - `Docidentifier` - document IDs (DOI, ISBN, etc.)
55
- - `Date` - publication dates with custom `StringDate` type
56
- - `Relation` - related documents (circular reference with Item)
57
- - `Ext` - extension data (doctype, ICS codes, structured identifiers)
58
-
59
- ### Sanitizing marked-up content
60
-
61
- **`Relaton::Bib::Sanitizer`** ([lib/relaton/bib/sanitizer.rb](lib/relaton/bib/sanitizer.rb))
62
- strips markup outside the basicdoc set. `LocalizedMarkedUpString#content=` calls it on
63
- assignment, so every `Title`, `Abstract`, `Note`, `Formattedref` and `Docidentifier` in
64
- every flavor passes through it. The set it keeps depends on the field, through
65
- `#sanitizer_allowed`: every field keeps `ALLOWED` (basicdoc inline markup), and `Abstract`
66
- keeps `ALLOWED_WITH_BLOCKS`, which adds `BLOCK` — the basicdoc lists, notes, tables, figures
67
- and admonitions, with the children their grammar requires (`name`, `thead`/`tbody`/`tfoot`,
68
- `image`). That split follows `biblio.rng`: `<abstract>` takes `BasicBlockNoId`, and a title,
69
- note, docidentifier or formattedref takes only `PureTextElement`, so do not widen the
70
- default. Before relaton#421 an abstract's list or note unwrapped to bare `<p>`s with no
71
- error. A consumer must therefore not assume that an abstract holds only `<p>` and inline
72
- markup. Two properties are load-bearing:
73
-
74
- - **It does not give up on an undeclared namespace prefix.** Third-party markup prefixes
75
- its elements — Crossref returns JATS as `<jats:p><jats:italic>x</jats:italic></jats:p>`,
76
- sometimes with an `xlink:` prefix on an attribute — and Relaton never declares the prefix.
77
- Nokogiri reports an undeclared prefix as a parse error, so the old
78
- `return content if fragment.errors.any?` returned exactly the markup that most needed
79
- sanitizing. The undeclared prefix then reached the bibdata XML, relaton-render's
80
- `sanitise_citations_input_string` returned nil, and isodoc raised
81
- `NoMethodError: undefined method '[]' for nil` (metanorma-pdfa#99). `parse_with_prefixes`
82
- now declares each prefix on a wrapper element, parses, and removes **only the wrapper's
83
- own** declarations, matched by identity rather than by URI — `remove_namespaces!` would
84
- also strip a namespace the content declares itself, such as the MathML `xmlns` inside an
85
- opaque `<stem>`. `wrapper_name` renames the wrapper while the content contains it, so the
86
- content cannot close it early — in **one** scan (`NS_WRAPPER_RX`), because growing the
87
- name and re-testing with `include?` is quadratic, the same trap the root `CLAUDE.md`
88
- records for relaton-cli's `strip_tags`. An undeclared prefix inside `<stem>` goes as well
89
- (`<mml:math>` → `<math>`): an undeclared prefix in the output is the failure being fixed.
90
- Un-prefixing an attribute renames it, so `drop_attribute_namespaces` **drops** a prefixed
91
- attribute whose plain name the element already carries — keeping both would emit
92
- `target="a" target="b"`, which Nokogiri rejects as `Attribute target redefined`, and that
93
- is the very unparseable output this path exists to prevent. Test the collision with
94
- `plain_attribute?`, not `Node#attribute`, which matches on the name alone and so finds the
95
- prefixed attribute itself. The guard for genuinely malformed input stays — unbalanced tags,
96
- a stray `<`, and text that only looks like a prefixed tag are all returned untouched.
97
- - **It does not indent.** `SAVE_OPTS` drops Nokogiri's `FORMAT` default, so
98
- `sanitize("<p><em>x</em></p>")` returns the input rather than `"<p>\n <em>x</em>\n</p>"`.
99
- Don't reintroduce a bare `to_xml`.
100
-
101
- ### Parsing
102
-
103
- - **`HashParserV1`** ([lib/relaton/bib/hash_parser_v1.rb](lib/relaton/bib/hash_parser_v1.rb)) - Converts legacy Hash/YAML format to `ItemData`
104
- - **`Converter::BibXml`** ([lib/relaton/bib/converter/bibxml.rb](lib/relaton/bib/converter/bibxml.rb)) - Parses RFC BibXML format via `Relaton::Bib::Converter::BibXml.to_item`
105
- - **lutaml-model native** - `Item.from_xml`, `Item.from_yaml`, `Item.from_json` handle current format
106
-
107
- ### Rendering
108
-
109
- - **`Renderer::BibtexBuilder`** - Converts `ItemData` to BibTeX format
110
- - **`Converter::BibXml`** - Converts `ItemData` to an `Rfcxml::V3` model object via
111
- `Relaton::Bib::Converter::BibXml.from_item`; call `to_xml` on the result (this is what
112
- `ItemData#to_rfcxml` does). Two output shapes share one class hierarchy:
113
- - `ToRfcxml` / `ToRfcxmlReferencegroup` (default) emit **BibXML**, the shape the Relaton
114
- data repositories publish. It keeps the RFC 7991-deprecated `<format>` element, so it
115
- must stay byte-compatible — every relaton data fetcher's `"bibxml"` format is
116
- `entry.to_rfcxml`. Do not change its output.
117
- - `ToRfcxmlV3` / `ToRfcxmlReferencegroupV3` (`v3: true`) subclass those and emit strict
118
- **RFC 7991**: no `<format>`, plus `<stream>`, `ascii*` attributes and `<refcontent>`.
119
- New v3-only behaviour belongs in the subclass, so the default path stays untouched by
120
- construction and `spec/bib/relaton/bib/converter/bibxml_spec.rb` keeps passing unedited.
121
- - **lutaml-model native** - `to_xml`, `to_yaml`, `to_json` via the serialization classes
122
-
123
- ### Usage Pattern
124
-
125
- ```ruby
126
- # Parse from YAML
127
- item = Relaton::Bib::Item.from_yaml(yaml_string)
128
-
129
- # Parse from XML
130
- item = Relaton::Bib::Bibitem.from_xml(xml_string)
131
- item = Relaton::Bib::Bibdata.from_xml(xml_string)
132
-
133
- # Parse from RFC XML
134
- item = Relaton::Bib::Converter::BibXml.to_item(xml_string)
135
-
136
- # Convert to different formats (returns string)
137
- item.to_xml # as <bibitem>
138
- item.to_xml(bibdata: true) # as <bibdata>
139
- item.to_yaml
140
- item.to_bibtex
141
- item.to_rfcxml # BibXML (what the data repos publish)
142
- item.to_rfcxml(v3: true) # strict RFC 7991 / xml2rfc v3
143
- item.to_rfcxml(anchor: "ISO712", include_keywords: false)
144
- ```
145
-
146
- ## Code Style
147
-
148
- - Follows Ribose OSS Ruby style guide (inherited via `.rubocop.yml`)
149
- - Target Ruby version: 3.3+ (`required_ruby_version` in the gemspec, `TargetRubyVersion` in `.rubocop.yml`)
150
- - Uses YARD documentation comments
@@ -1,187 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-bipm is a Ruby gem that retrieves BIPM (Bureau International des Poids et Mesures) standards metadata for bibliographic use. It's part of the larger Relaton family of gems that handle bibliographic data for standards organizations.
8
-
9
- ## Common Commands
10
-
11
- - **Install dependencies:** `bundle install`
12
- - **Run all tests:** `bundle exec rake spec`
13
- - **Run a single test file:** `bundle exec rspec spec/relaton/bipm/bibliography_spec.rb`
14
- - **Run a single test by line:** `bundle exec rspec spec/relaton/bipm/bibliography_spec.rb:42`
15
- - **Interactive console:** `bin/console`
16
-
17
- ## Architecture
18
-
19
- ### Namespace & Module Structure
20
-
21
- All code lives under `Relaton::Bipm` (in `lib/relaton/bipm/`). The gem name is `relaton-bipm`, the require path is `relaton/bipm`.
22
-
23
- ### Pubid-backed `index-v2` (the runtime index)
24
-
25
- The flavor's runtime index is the **pubid-backed `index-v2`** (`INDEXFILE =
26
- "index-v2"`), whose rows are `Pubid::Bipm` identifiers serialized as
27
- `_type: pubid:bipm:{committee-document,meeting,metrologia-article,si-brochure}`
28
- — mirroring the JCGM/OIML/JIS convention, not BIPM's old bespoke
29
- `{group,type,number,year}` hash. The `DataFetcher` builds it (`pubid_class:
30
- ::Pubid::Bipm::Identifier`), and the consumer reads it.
31
-
32
- The **legacy bespoke `index-v1`** (the old `{group,type,number,year}` hash form)
33
- is no longer produced or read by this flavor. It is still published by
34
- `relaton-data-bipm`'s own `crawler.rb`, which builds it with the **retained**
35
- `Relaton::Bipm::Id` (this flavor keeps `Id` public for exactly that). So `Id` is
36
- now used in **one place only** — the crawler's `index-v1` generation. It was
37
- also this flavor's consumer query parser until pubid's grammar widened enough to
38
- replace it; see **Pubid-to-pubid lookup** below.
39
-
40
- ### Key Components
41
-
42
- - **`Bibliography`** (`bibliography.rb`) - Main entry point. Parses the user
43
- reference with `::Pubid::Bipm.parse` (`#parse_ref`; an unparseable ref raises
44
- `Pubid::Errors::ParseError`, while the machine-derived `#year_number_retry`
45
- probe rescues it to nil) and matches the pubid `index-v2` rows as
46
- pubid objects — see **Pubid-to-pubid lookup** below for the stem and the two
47
- escapes under it. The latest edition wins via
48
- `max_by { |r| [r[:id].year.to_i, r[:file]] }` (the file path only breaks a
49
- tie — Metrologia and SI Brochure rows carry no year, so all 6206 of them
50
- score 0 and the index sort is not stable).
51
- - **`Id`** (`id_parser.rb`) - The flexible bespoke regex parser. Parses BIPM
52
- reference strings into `{group,type,number,year,…}` hashes; the `TYPES` hash
53
- maps full EN/FR type names to abbreviations, and `normalize_hash` applies
54
- `CCDS→CCTF` and leading-zero stripping. **Retained solely** as the
55
- `relaton-data-bipm` crawler's `index-v1` builder — it no longer parses
56
- queries, and it is **not** used to write this flavor's `index-v2`.
57
- `bibliography.rb` does not require it; `lib/relaton/bipm.rb` does, which is
58
- what keeps `Relaton::Bipm::Id` reachable for the crawler. JCGM is not handled
59
- here.
60
- - **`Processor`** (`processor.rb`) - Relaton framework integration point (extends
61
- `Relaton::Core::Processor`). Registers prefix `BIPM` and default prefix pattern
62
- matching BIPM, CCTF, CCDS, CGPM, CIPM, JCRB (JCGM was split out into the `jcgm`
63
- flavor). `remove_index_file` targets `index-v2`.
64
- - **`DataFetcher`** (`data_fetcher.rb`) - Bulk fetches from three data sources:
65
- `bipm-data-outcomes`, `bipm-si-brochure`, `rawdata-bipm-metrologia`. Delegates
66
- to specialized parsers, which route every index write through the public,
67
- **guarded** `#add_to_index(docnumber, path)` — it parses `docnumber` with
68
- `::Pubid::Bipm.parse` and stores the identifier object, or **warns and skips**
69
- an unparseable id rather than aborting the crawl (mirrors
70
- `Relaton::Jcgm::DataFetcher#add_to_index`; `Relaton::Index` rejects a whole
71
- index if one row fails to deserialize). Public so `relaton-data-bipm`'s crawler
72
- can index its curated `static/` docs through the same guarded path.
73
- - **`Item`** / **`ItemData`** (`model/item.rb`, `item_data.rb`) - The
74
- bibliographic item model, extending `Relaton::Bib::Item`. Supports XML, YAML,
75
- and JSON serialization.
76
- - **`model/`** directory - Lutaml model classes (Bibdata, Bibitem, Ext, etc.) for
77
- XML/YAML serialization.
78
-
79
- ### Pubid-to-pubid lookup
80
-
81
- `#get_bipm` parses the reference with `::Pubid::Bipm.parse` and matches the
82
- index rows as pubid objects — the same shape ISO, OIML and JCGM use. The
83
- bespoke `Id` grammar takes **no part in a query**. That became possible with
84
- pubid `a96e9f45`, which accepts the loose consumer forms this flavor once
85
- needed `Id` for and *normalizes* them to BIPM's canonical spelling
86
- (`CCDS Recommendation 2 (2009)` → `CCTF Recommendation 2 (2009)`,
87
- `CIPM 111e Réunion (2022)` → `CIPM 111<sup>e</sup> réunion (2022)`).
88
-
89
- **The stem.** Committee-document and meeting rows are stored language- and
90
- form-neutral, while a reference may name a language (`(2009, E)`) and always
91
- names a form (short `CCTF REC 2` vs long `CCTF Recommendation 2`). So
92
- `#pubid_match?` compares `exclude(:language, :form)`, adding `:year` when the
93
- **query** carries none — `CIPM Meeting 43` has no year, its row has 1950. That
94
- last clause is what `Id#==` did with
95
- `other_hash.delete(:year) unless hash[:year]`; without it the lookup regresses.
96
-
97
- **Two escapes sit under the narrowing**, each for a case where the query's
98
- bsearch key cannot equal its row's:
99
-
100
- 1. **A bare `SI Brochure` names no edition**, so it keys to `""` while its row
101
- keys to `"9e"`. `#search_index` rescans the whole index. A brochure query
102
- with a nil edition is a *partial* reference and deliberately matches any
103
- brochure row — what the old `{group: "SI", type: "Brochure"}` projection
104
- meant. `Part N` names a section inside the brochure, not a document, so it
105
- stays out of the key exactly as `Id#==` kept `:part` out.
106
- 2. **`CCTF Recommendation 2009-02` parses as the literal number `2009-02`**,
107
- while the row keys on `2`. A rescan cannot help — no row carries that number
108
- — so `#year_number_retry` re-reads a trailing `YYYY-NN` as number plus year.
109
- It runs **only after a miss**, because `NNNN-NN` is also a real BIPM number
110
- (`CIPM 2005-06`), so the literal reading must win.
111
-
112
- **One rule survives from `Id#==` and must not be dropped.** It treated a
113
- document numbered `"1"` and a number-less one as the same document when both
114
- carried a year, so `CIPM Resolution 1 (1879)` reached `CIPM RES (1879)`. pubid
115
- has no such rule, and the six ordinal-less declarations
116
- (`CGPM DECL (1889)`, `CIPM RES (1879)`, …) are the only rows it applies to — but
117
- they are addressable *both* ways, so removing it silently broke
118
- `CIPM Resolution 1 (1879)` and `CGPM Declaration 1 (1889)`. `#number_collapses?`
119
- restores it by excluding `:number` from that one comparison. Both forms are in
120
- the pinned baseline. Note the corpus has **no two rows** the rule could join
121
- (6 number-less against 83 numbered `"1"`, no shared group/type/year), so it only
122
- ever widens a query, never merges records.
123
-
124
- **Cost.** `#exclude` copies the identifier, so reducing per candidate row is
125
- expensive — roughly **165x** a plain attribute read. Two things keep the lookup
126
- at or below what the `Id` grammar cost: the query is reduced **once per search**,
127
- not once per row, and `#pubid_match?` rejects on `CHEAP_KEYS`
128
- (`number group year issue article`) before reducing anything. Those are exactly
129
- the attributes the stem never removes, so stem equality implies all of them and
130
- the rejects cannot change an answer — a spec asserts that over the corpus rather
131
- than trusting the argument. Without both, a bare-brochure rescan built ~7,900
132
- copies and took **2.3 s**; with them the same lookup is 2.8 ms and a narrowed
133
- one 0.4–0.5 ms, against 0.5–0.7 ms under `Id`.
134
-
135
- **The regression guard.** `spec/bipm/relaton/bipm/bibliography_spec.rb` pins,
136
- literally, the file each of the 29 references the suite exercises resolved to
137
- under the `Id` grammar, captured before it was removed. Any change to matching
138
- that moves a lookup fails there by name. That is the evidence for the swap —
139
- `Id#==` was fuzzier than the stem match in ways no spec pinned (it collapsed
140
- number `"1"` with nil, dropped `:part`/`:append`), so equivalence had to be
141
- demonstrated rather than argued.
142
-
143
- `Relaton::Bipm::Id` and `id_parser.rb` **stay** — `relaton-data-bipm`'s crawler
144
- builds `index-v1` with them, and `id_parser_spec.rb` still covers them. Only
145
- `bibliography.rb` stopped requiring the file.
146
-
147
- ### Internal identifier
148
-
149
- `docid.to_s(form: :internal)` drops the
150
- `BIPM` publisher (relaton#49): `BIPM SI Brochure Part 1` →
151
- `SI Brochure Part 1`. Committee ids (`CGPM RES (1889)`, `CCTF REC 2 (2009)`)
152
- have no publisher token and are the same in both forms. The rule is the `"BIPM"` row of
153
- `Bib::Docidentifier::FORM_PUBID`: BIPM docidentifiers carry no pubid,
154
- so the content is parsed with `Pubid::Bipm` at call time and rendered with
155
- `to_s(with_publisher: false)`; unparseable content is returned unchanged. It
156
- also holds for a relation's nested item. Do not move it into a `Bipm::Docidentifier`
157
- subclass: typing the item's `docidentifier` collection with a subclass makes
158
- YAML serialization reject the `Bib::Docidentifier` objects the parsers build
159
- (`IncorrectModelError`).
160
-
161
- ### Data Sources
162
-
163
- The gem fetches from three external datasets:
164
-
165
- 1. **bipm-data-outcomes** - CGPM/CIPM/committee resolutions, recommendations, decisions (the `jcgm` meeting dir is now harvested by the `Relaton::Jcgm::DataFetcher` instead)
166
- 2. **bipm-si-brochure** - SI Brochure documents
167
- 3. **rawdata-bipm-metrologia** - Metrologia journal articles (parsed from CrossRef-style data)
168
-
169
- ### Testing
170
-
171
- - **Index fixture:** `spec/bipm/fixtures/index-v2.zip` (the pubid `index-v2`) is
172
- deserialized via `Pubid::Bipm::Identifier` and pre-loaded into the
173
- `Relaton::Index` pool in `before(:suite)` (`spec/bipm/support/webmock.rb`,
174
- mirroring the JCGM/JIS setup). It was regenerated from the real
175
- `relaton-data-bipm` docs' docnumbers (each parsed with `Pubid::Bipm`); the
176
- handful that pubid can't parse — JCGM orphans, `CIPM 2005-06`, docs removed
177
- upstream — are absent by design.
178
- - **Parser specs** assert the parsers call `data_fetcher.add_to_index(docnumber,
179
- path)` with the right docnumber string (the pubid parsing/guarding lives on the
180
- mocked `DataFetcher`, keeping the parser tests format-agnostic).
181
- - Tests use RSpec with VCR cassettes (`spec/bipm/vcr_cassettes/`) and WebMock.
182
-
183
- ### Dependencies
184
-
185
- Key runtime dependencies: `pubid` (provides `Pubid::Bipm`), `relaton-bib` (core
186
- bibliographic model), `relaton-index` (index management), `relaton-core`
187
- (framework base), `parslet`, `mechanize` (HTTP), `faraday`, `rubyzip`.
@@ -1,35 +0,0 @@
1
- # CLAUDE.md
2
-
3
- This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
-
5
- ## Project Overview
6
-
7
- relaton-bsi retrieves British Standards Institution (BSI) bibliographic data using the Relaton model. Unlike index-backed flavors, it searches the BSI Shopify storefront: an **Algolia** product index finds candidate standards, then a **GraphQL** query against the Shopify storefront API fetches full metadata (titles, dates, ISBNs, committee info). Depends on relaton-iso for its ISO-based item/extension models.
8
-
9
- ## Development
10
-
11
- ```bash
12
- bundle exec rake # default task → rspec
13
- bundle exec rspec spec/relaton/bsi/bibliography_spec.rb # single file
14
- ```
15
-
16
- (No gem-local rubocop config; the monorepo root config applies.)
17
-
18
- ## Architecture
19
-
20
- Namespace: `Relaton::Bsi`. Retrieval flow:
21
-
22
- 1. **Bibliography** (`lib/relaton/bsi/bibliography.rb`) — `get(code, year, opts)` / `search` entry points. Reference handling is **entirely `Pubid::Bsi`** (from the unified `pubid` gem) with **no hand-rolled identifier regex**: `parse` wraps `Pubid::Bsi::Identifier.parse` and **raises** on an unparseable query (only `Hit#pubid` rescues to nil — the Algolia catalogue carries codes such as `BS 381C SET:1996 (R2002)`, and one such row must not abort the search), `publication_year` reads `pubid.root.year` (not `base_document.year`: pubid holds an adopted standard such as `BS EN ISO 8848:2021` in `base`, so its `base_document.year` is nil; over pubid's 1,536 parseable BSI fixture ids `root.year` equals the old reader on 1,529 and fills in the base year on the other 7), and `same_reference?(query, hit, skip_rest:, drop_amd:)` decides whether a query and a candidate hit denote the same document. `same_reference?` uses only pubid's typed API — `#base_document` (peels supplements + Expert-commentary / Flex-version wrappers), `#matches?(other, ignore: %i[date month])` for the base-document comparison (`:month` is excluded alongside `:date` because Flex stores the month separately, so `exclude(:date)` alone leaves them unequal), and the `Amendment`/`Corrigendum` `#supplement_type`, `#number` and `#year` (amendment year optional when the query omits it). pubid removes the `#supplement_number`/`#supplement_year` aliases, but the locked pubid still has them and leaves a BSI supplement's `#number`/`#year` nil, so the private `supplement_number`/`supplement_year` helpers read the alias while it exists. Remove that fallback once pubid `main` has no alias. A canonical `rest_marker` (`:expert_commentary` / `:"flex_<edition>"`) makes an `ExComm` query match an `Expert Commentary` hit. `search` still lightly normalizes the **Algolia query text** (drops `BSI `, abbreviates the Expert-commentary suffix) — search-engine tuning, not identifier parsing. This depends on the `pubid` `feat/bsi-parse-gaps` work (see `HANDOFFS/metanorma__pubid.md`): the three parse/exclude gaps **plus** the Phase-2 API (`#base_document`, `exclude(:amendment|:supplement|:edition)`, uniform supplement accessors, `#matches?`).
23
- 2. **HitCollection** (`lib/relaton/bsi/hit_collection.rb`, extends `Relaton::Core::HitCollection`) — queries the Algolia index (`shopify_products`) and filters hits by parsing each hit code and calling `Bibliography.same_reference?` against the parsed query, in three passes: exact → `skip_rest` (ignore the ExComm/Flex-version suffix) → `drop_amd` (compare base documents only, and rewrite the hit id to `hit.base_document.to_s`). Algolia keys are public/client-side.
24
- 3. **Hit** (`lib/relaton/bsi/hit.rb`, extends `Relaton::Core::Hit`) — `item` lazy-loads the full document via the scraper; `pubid` memoizes the hit code parsed with `Pubid::Bsi` (like `Relaton::Iso::Hit#pubid`) so the three filter passes don't re-parse it, and is nil for a code pubid cannot parse. `Bibliography#results_filter` reads the year from this memoized pubid too, not from the (possibly `drop_amd`-rewritten) `hit[:code]`.
25
- 4. **Scraper** (`lib/relaton/bsi/scraper.rb`) — GraphQL client; runs schema-based queries against the Shopify storefront GraphQL endpoint to build a full record (titles split on em-dash into intro/main/part, dates, contributors, ICS, source URI).
26
- 5. **ItemData / Model::Item** (`item_data.rb`, `model/`) — `Relaton::Bsi::ItemData` extends `Relaton::Iso::ItemData`; `Ext` adds doctype, ICS, and a structured identifier. **Docidentifier** (`model/docidentifier.rb`) keeps the parsed `Pubid::Bsi::Identifier` as the **single stored value** of `content` (via the flavor-agnostic `Iso::Type::Pubid` lutaml type, which preserves the instance in and stringifies it out) — one source of truth: `#pubid` returns the instance, `#content` renders it to a string. `#remove_date!` → `pubid.exclude(:date, :month)`, `#remove_part!` → `pubid.exclude(:part, :subpart)`. `#to_all_parts!` wraps `pubid` in pubid's `AllParts` (`self.content = pubid.exclude(:supplement_year).to_all_parts`) — since BSI has no dedicated pubid `AllParts` subclass, the wrapper is the generic `Pubid::AllPartsIdentifier`, whose `#to_s` DOES print a "(all parts)" marker (unlike BSI's own renderer, which never had one). The `exclude(:supplement_year)` pre-step matters on a consolidated id (`BS 7273-4:2015+A1:2021`): pubid protects an `Amendment`/`Corrigendum`'s own date from a bare `exclude(:date)` — the same protection `#remove_date!` relies on to drop the base year while keeping the amendment's — but `to_all_parts`'s identity computation is built on that same `exclude`, so without the explicit `:supplement_year` key the amendment's year survives into the "(all parts)" form, distinguishing editions that marker is meant to collapse (`BS 7273+A1 (all parts)`, not `BS 7273+A1:2021 (all parts)`). Because `content` and `#pubid` are the same stored value here, both now show the "(all parts)" marker after `#to_all_parts!` — unlike flavors with a separately cached content string, which keep `content` marker-less on purpose. `#pubid`'s type check is widened to also accept `::Pubid::AllParts` (not just `Pubid::Bsi::Identifier`) for this reason. All three no-op when `#pubid` is nil. Non-BSI identifiers (e.g. ISBN) and anything pubid can't parse are stored verbatim as plain strings.
27
- 6. **Processor** (`lib/relaton/bsi/processor.rb`) — registry integration; lazy-`require_relative`s `../bsi` in its methods.
28
-
29
- ## External dependencies
30
-
31
- `algolia ~> 2.3.0`, `graphql ~> 2.3`, `graphql-client ~> 0.23`, `faraday-net_http_persistent ~> 2.0`, `relaton-core`, `relaton-iso`.
32
-
33
- ## Testing
34
-
35
- RSpec with WebMock + VCR (cassettes in `spec/vcr_cassettes/`, record `:once`, never auto-rerecorded). Network is blocked via WebMock. No index fixture — BSI has no curated index; `spec/fixtures/` holds XML/YAML round-trip data.