relaton-cli 3.0.7 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/lib/relaton-cli.rb +4 -16
- metadata +11 -123
- data/.gitignore +0 -22
- data/.hound.yml +0 -3
- data/.rspec +0 -1
- data/CLAUDE.md +0 -524
- data/Gemfile +0 -42
- data/Rakefile +0 -21
- data/bin/console +0 -11
- data/bin/rspec +0 -29
- data/bin/setup +0 -8
- data/docs/README.adoc +0 -589
- data/exe/relaton +0 -20
- data/frontend/dist/404.html +0 -18
- data/frontend/dist/app.iife.js +0 -1
- data/frontend/dist/style.css +0 -3
- data/i18n.yaml +0 -1
- data/lib/relaton/bibcollection.rb +0 -146
- data/lib/relaton/bibdata.rb +0 -92
- data/lib/relaton/cli/base_convertor.rb +0 -134
- data/lib/relaton/cli/command.rb +0 -390
- data/lib/relaton/cli/data_fetcher.rb +0 -20
- data/lib/relaton/cli/frontend_assets.rb +0 -58
- data/lib/relaton/cli/full_text_search.rb +0 -98
- data/lib/relaton/cli/index_item_normalizer.rb +0 -380
- data/lib/relaton/cli/index_site_generator.rb +0 -1008
- data/lib/relaton/cli/relaton_file.rb +0 -290
- data/lib/relaton/cli/subcommand_collection.rb +0 -263
- data/lib/relaton/cli/subcommand_db.rb +0 -81
- data/lib/relaton/cli/util.rb +0 -8
- data/lib/relaton/cli/version.rb +0 -11
- data/lib/relaton/cli/xml_convertor.rb +0 -47
- data/lib/relaton/cli/xml_to_html_renderer.rb +0 -117
- data/lib/relaton/cli/yaml_convertor.rb +0 -102
- data/lib/relaton/cli.rb +0 -113
- data/lib/relaton/element_finder.rb +0 -29
- data/relaton-cli.gemspec +0 -49
- data/templates/_document.liquid +0 -111
- data/templates/_index.liquid +0 -61
- data/templates/index/page.liquid +0 -20
- data/templates/index-style.css +0 -1017
data/docs/README.adoc
DELETED
|
@@ -1,589 +0,0 @@
|
|
|
1
|
-
= Relaton CLI: Command-line Interface for Bibliographic References
|
|
2
|
-
|
|
3
|
-
image:https://img.shields.io/gem/v/relaton-cli.svg["Gem Version", link="https://rubygems.org/gems/relaton-cli"]
|
|
4
|
-
image:https://github.com/relaton/relaton/actions/workflows/rake.yml/badge.svg["Build Status", link="https://github.com/relaton/relaton/actions/workflows/rake.yml"]
|
|
5
|
-
|
|
6
|
-
Relaton CLI is a command-line tool for fetching, converting, and managing
|
|
7
|
-
bibliographic references to standards (ISO, IEC, IETF, NIST, IEEE, and many
|
|
8
|
-
more). It is part of the https://github.com/relaton[Relaton] /
|
|
9
|
-
https://www.metanorma.org[Metanorma] ecosystem.
|
|
10
|
-
|
|
11
|
-
== Installation
|
|
12
|
-
|
|
13
|
-
Add to your `Gemfile`:
|
|
14
|
-
|
|
15
|
-
[source,ruby]
|
|
16
|
-
----
|
|
17
|
-
gem "relaton-cli"
|
|
18
|
-
----
|
|
19
|
-
|
|
20
|
-
Or install directly:
|
|
21
|
-
|
|
22
|
-
[source,console]
|
|
23
|
-
----
|
|
24
|
-
$ gem install relaton-cli
|
|
25
|
-
----
|
|
26
|
-
|
|
27
|
-
== Requirements
|
|
28
|
-
|
|
29
|
-
* Ruby >= 3.3.0
|
|
30
|
-
|
|
31
|
-
== Usage
|
|
32
|
-
|
|
33
|
-
Each command supports a `--verbose` (short form `-v`) option to include
|
|
34
|
-
warnings in the output.
|
|
35
|
-
|
|
36
|
-
=== relaton fetch
|
|
37
|
-
|
|
38
|
-
[source,console]
|
|
39
|
-
----
|
|
40
|
-
$ relaton fetch CODE -t TYPE -f FORMAT -y YEAR -r RETRIES --all-parts --keep-year --no-cache --publication-date-before DATE --publication-date-after DATE
|
|
41
|
-
----
|
|
42
|
-
|
|
43
|
-
Fetch the Relaton XML entry corresponding to the document identifier `CODE`.
|
|
44
|
-
|
|
45
|
-
[[relaton-fetch-types]]
|
|
46
|
-
* `TYPE` is optional and specifies the standards class library to use. Recognized values for `TYPE` are: `3GPP`, `BIPM`, `CCTF`, `CCDS`, `CGPM`, `CIPM`, `JCRB`, `JCGM`, `BSI`, `BS`, `PD`, `CC`, `CCSDS`, `CEN`, `EN`, `ENV`, `CWA`, `HD`, `CR`, `CIE`, `DOI`, `ECMA`, `ETSI`, `CN`, `GB`, `GB/T`, `GB/Z`, `IANA`, `IEC`, `CISPR`, `IEV`, `IEEE`, `ANSI`, `NACE`, `AIEE`, `ASA`, `IRE`, `IETF`, `RFC`, `BCP`, `FYI`, `STD`, `I-D`, `IHO`, `ISBN`, `ISO`, `ISO/IEC`, `ITU`, `JIS`, `TR`, `NIST`, `NBS`, `NISTGCR`, `ITL Bulletin`, `JPCRD`, `NISTIR`, `CSRC`, `FIPS`, `OASIS`, `OGC`, `OMG`, `UN`, `W3C`, `XEP`, `PLATEAU`.
|
|
47
|
-
* `FORMAT` is optional. Recognized values are `xml` (default), `yaml`, `bibtex`.
|
|
48
|
-
* `YEAR` is optional and specifies the year of publication.
|
|
49
|
-
* `RETRIES` is optional, number of network retries (default 1).
|
|
50
|
-
* `--all-parts` fetch all parts of the standard.
|
|
51
|
-
* `--keep-year` undated reference should return an actual reference with year.
|
|
52
|
-
* `--no-cache` do not use cache.
|
|
53
|
-
* `--publication-date-before DATE` fetch only documents published before the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`.
|
|
54
|
-
* `--publication-date-after DATE` fetch only documents published after the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`.
|
|
55
|
-
|
|
56
|
-
=== relaton fetch-data
|
|
57
|
-
|
|
58
|
-
[source,console]
|
|
59
|
-
----
|
|
60
|
-
$ relaton fetch-data DATASET -o DIR -f FORMAT
|
|
61
|
-
----
|
|
62
|
-
|
|
63
|
-
Fetch all documents from a `DATASET` source and save them to folder `DIR` in format `FORMAT`.
|
|
64
|
-
|
|
65
|
-
The following datasets are available:
|
|
66
|
-
|
|
67
|
-
* `nist-tech-pubs` - https://github.com/usnistgov/NIST-Tech-Pubs/releases/download/May2024/allrecords-MODS.xml
|
|
68
|
-
* `cie-techstreet` - https://www.techstreet.com/cie/searches/31156444
|
|
69
|
-
* `calconnect-org` - https://standards.calconnect.org/relaton/index.yaml
|
|
70
|
-
* `ogc-naming-authority` - https://raw.githubusercontent.com/opengeospatial/NamingAuthority/master/incubation/bibliography/bibliography.json
|
|
71
|
-
* `ieee-rawbib` - looks for the IEEE dataset in the local `./ieee-rawbib` directory. The dataset can be downloaded from https://github.com/ietf-ribose/ieee-rawbib
|
|
72
|
-
* `w3c-rdf` - http://www.w3.org/2002/01/tr-automation/tr.rdf
|
|
73
|
-
* `w3c-tr-archive` - looks for the W3C archive dataset in local `./w3c-tr-archive` directory. The dataset can be downloaded from https://github.com/relaton/w3c-tr-archive
|
|
74
|
-
* `iana-registries` - https://github.com/ietf-ribose/iana-registries
|
|
75
|
-
* `status-smg-3GPP` - updates previously downloaded data if a new archive is available in ftp://www.3gpp.org/Information/Databases/
|
|
76
|
-
* `status-smg-3GPP-force` - download data from the latest archive in ftp://www.3gpp.org/Information/Databases/
|
|
77
|
-
* `ietf-rfcsubseries` - https://www.rfc-editor.org/rfc-index.xml (`<bcp-entry>`, `<fyi-entry>`, `<std-entry>`)
|
|
78
|
-
* `ietf-internet-drafts` - looks for the Internet-Drafts dataset in the local `./bibxml-ids` directory. The dataset can be downloaded using `rsync -avcizxL rsync.ietf.org::bibxml-ids ./bibxml-ids`.
|
|
79
|
-
* `ietf-rfc-entries` - https://www.rfc-editor.org/rfc-index.xml (`<rfc-entry>`)
|
|
80
|
-
* `oasis-open` - https://www.oasis-open.org/standards/
|
|
81
|
-
* `bipm-data-outcomes` - looks for the BIPM dataset in the local `./bipm-data-outcomes` directory. The dataset can be downloaded from https://github.com/metanorma/bipm-data-outcomes
|
|
82
|
-
* `si-brochure` - looks for the SI-Brochure dataset in the local `./bipm-si-brochure` directory. The dataset can be downloaded from https://github.com/metanorma/bipm-si-brochure
|
|
83
|
-
* `ecma-standards` - https://www.ecma-international.org/publications/standards/
|
|
84
|
-
* `itu-t` - https://www.itu.int/mws/api/recommendations/searchRecs
|
|
85
|
-
* `itu-r` - **unavailable**: ITU decommissioned the bulk-enumeration endpoint this crawler used, so it raises rather than harvesting (see https://github.com/relaton/relaton/issues/75)
|
|
86
|
-
* `ccsds` - https://public.ccsds.org/Publications/AllPubs.aspx
|
|
87
|
-
* `etsi-csv` - https://www.etsi.org/
|
|
88
|
-
* `plateau-handbooks` - https://www.mlit.go.jp/plateau/_next/data/1.3.0/libraries/handbooks.json
|
|
89
|
-
* `plateau-technical-reports` - https://www.mlit.go.jp/plateau/_next/data/1.3.0/libraries/technical-reports.json
|
|
90
|
-
* `jis-webdesk` - https://webdesk.jsa.or.jp/books/W11M0270
|
|
91
|
-
|
|
92
|
-
Options:
|
|
93
|
-
|
|
94
|
-
* `DIR` - folder name to store documents (default `./data`).
|
|
95
|
-
* `FORMAT` - output format: `yaml` (default), `xml`, `bibxml`.
|
|
96
|
-
|
|
97
|
-
=== relaton extract
|
|
98
|
-
|
|
99
|
-
[source,console]
|
|
100
|
-
----
|
|
101
|
-
$ relaton extract Metanorma-XML-Directory Relaton-XML-Directory -x EXTENSION
|
|
102
|
-
----
|
|
103
|
-
|
|
104
|
-
Iterate through all Metanorma XML files in `Metanorma-XML-Directory` and extract the `bibdata` element from each. Save the `bibdata` element to `Relaton-XML-Directory` as the Relaton XML description for that file. If a document identifier is present in `bibdata`, it is used as the filename; otherwise, the original filename is used. The filename is suffixed with `EXTENSION` (default `.rxl`).
|
|
105
|
-
|
|
106
|
-
=== relaton concatenate
|
|
107
|
-
|
|
108
|
-
[source,console]
|
|
109
|
-
----
|
|
110
|
-
$ relaton concatenate Source-Directory Relaton-Collection-File -t TITLE -g ORGANIZATION
|
|
111
|
-
----
|
|
112
|
-
|
|
113
|
-
Iterate through all Relaton files (YAML and XML) in `Source-Directory` and concatenate them into a Relaton Collection file. The extension of the Collection file can be set via the filename (e.g., `.yaml` for YAML, `.rxl` for XML) or using the `-x` (`--extension`) option.
|
|
114
|
-
|
|
115
|
-
For each Relaton input file in `Source-Directory`, if a document file with the same base name is identified (XML, HTML, PDF, or DOC), a link to that file is inserted.
|
|
116
|
-
|
|
117
|
-
If `TITLE` or `ORGANIZATION` options are given, they are added to the output as the title and author of the collection.
|
|
118
|
-
|
|
119
|
-
=== relaton split
|
|
120
|
-
|
|
121
|
-
[source,console]
|
|
122
|
-
----
|
|
123
|
-
$ relaton split Relaton-Collection-File Relaton-File-Directory -x rxl
|
|
124
|
-
----
|
|
125
|
-
|
|
126
|
-
Split a `Relaton-Collection-File` into multiple files in `Relaton-File-Directory`. Supports the `-x` (`--extension`) option to set the file extension.
|
|
127
|
-
|
|
128
|
-
=== relaton convert
|
|
129
|
-
|
|
130
|
-
[source,console]
|
|
131
|
-
----
|
|
132
|
-
$ relaton convert XML -f FORMAT -o OUTPUT-FILE
|
|
133
|
-
----
|
|
134
|
-
|
|
135
|
-
Convert a Relaton XML document into YAML, AsciiBib, or BibTeX format. Allowed `--format` values are `yaml`, `asciibib`, `bibtex`. If `--output` is omitted, a new file is created in the same folder with the appropriate extension.
|
|
136
|
-
|
|
137
|
-
=== relaton yaml2xml
|
|
138
|
-
|
|
139
|
-
[source,console]
|
|
140
|
-
----
|
|
141
|
-
$ relaton yaml2xml YAML -o OUTPUT-DIRECTORY -x RELATON_EXTENSION -p PREFIX -r LIBRARY
|
|
142
|
-
----
|
|
143
|
-
|
|
144
|
-
Convert a Relaton YAML file into Relaton Collection XML or separate files. If the YAML file specifies multiple bibliographic items and `OUTPUT-DIRECTORY` is given, also convert into individual Relaton XML files per entry. The document identifier is used as the filename, suffixed with `RELATON_EXTENSION` (default `.rxl`) and prefixed with `PREFIX` (default empty). Libraries required for conversion are specified in `LIBRARY` as a space-delimited list.
|
|
145
|
-
|
|
146
|
-
[[relaton-yaml]]
|
|
147
|
-
A Relaton Collection YAML file contains initial metadata and a list of bibliographic entries:
|
|
148
|
-
|
|
149
|
-
[source,yaml]
|
|
150
|
-
----
|
|
151
|
-
root:
|
|
152
|
-
title: CalConnect Standards Registry
|
|
153
|
-
author: The Calendaring and Scheduling Consortium
|
|
154
|
-
items:
|
|
155
|
-
- id: CC36000
|
|
156
|
-
type: standard
|
|
157
|
-
title:
|
|
158
|
-
- content: Standardization documents -- Vocabulary
|
|
159
|
-
docidentifier:
|
|
160
|
-
- type: CC
|
|
161
|
-
content: CC 36000
|
|
162
|
-
primary: true
|
|
163
|
-
date:
|
|
164
|
-
- type: issued
|
|
165
|
-
at: '2018-10-25'
|
|
166
|
-
status:
|
|
167
|
-
stage:
|
|
168
|
-
content: proposal
|
|
169
|
-
- id: CC34000
|
|
170
|
-
type: standard
|
|
171
|
-
title:
|
|
172
|
-
- content: Date and time -- Concepts and vocabulary
|
|
173
|
-
docidentifier:
|
|
174
|
-
- type: CC
|
|
175
|
-
content: CC 34000
|
|
176
|
-
primary: true
|
|
177
|
-
date:
|
|
178
|
-
- type: issued
|
|
179
|
-
at: '2018-10-25'
|
|
180
|
-
status:
|
|
181
|
-
stage:
|
|
182
|
-
content: proposal
|
|
183
|
-
----
|
|
184
|
-
|
|
185
|
-
A Relaton YAML file describing an individual bibliographic entry is limited to metadata specific to that entry. The link:https://github.com/relaton/relaton-bib/blob/master/docs/hash.adoc#yaml[Relaton YAML] illustrates the common fields supported by all flavors.
|
|
186
|
-
|
|
187
|
-
=== relaton xml2yaml
|
|
188
|
-
|
|
189
|
-
[source,console]
|
|
190
|
-
----
|
|
191
|
-
$ relaton xml2yaml XML -o OUTPUT-DIRECTORY -x RELATON_EXTENSION -p PREFIX -r LIBRARY
|
|
192
|
-
----
|
|
193
|
-
|
|
194
|
-
Convert a Relaton XML file into Relaton YAML. If the XML file is a collection and `OUTPUT-DIRECTORY` is given, also produce individual YAML files per entry. The document identifier is used as the filename, suffixed with `RELATON_EXTENSION` (default `.yaml`) and prefixed with `PREFIX` (default empty). Libraries required for conversion are specified in `LIBRARY` as a space-delimited list.
|
|
195
|
-
|
|
196
|
-
[[relaton-xml2html]]
|
|
197
|
-
=== relaton xml2html
|
|
198
|
-
|
|
199
|
-
[source,console]
|
|
200
|
-
----
|
|
201
|
-
$ relaton xml2html <relaton-xml> [<stylesheet>] [<html-template-dir>]
|
|
202
|
-
----
|
|
203
|
-
|
|
204
|
-
Render a Relaton Collection XML as an HTML file. Used to generate an HTML index of standards.
|
|
205
|
-
|
|
206
|
-
* `relaton-xml` is the Relaton Collection XML file.
|
|
207
|
-
* `stylesheet` is the CSS stylesheet to style the output.
|
|
208
|
-
* `html-template-dir` is a directory containing HTML Liquid Template files. Two templates are required:
|
|
209
|
-
|
|
210
|
-
** Index template (`_index.liquid`) -- recognizes the following parameters:
|
|
211
|
-
*** `css`: the CSS stylesheet content
|
|
212
|
-
*** `title`: the collection title from `./relaton-collection/title`
|
|
213
|
-
*** `author`: the collection author from `./relaton-collection/contributor[role/@type = 'author']/organization/name`
|
|
214
|
-
*** `content`: the list of resources generated by the script
|
|
215
|
-
|
|
216
|
-
** Individual entry template (`_document.liquid`) -- recognizes attributes of a bibliographic entry (`document`) following the naming convention of <<relaton-yaml,Relaton YAML>>; e.g. `document.html` is the HTML URI for the document.
|
|
217
|
-
|
|
218
|
-
The default stylesheet and templates are provided in the `templates` directory.
|
|
219
|
-
|
|
220
|
-
=== relaton yaml2html
|
|
221
|
-
|
|
222
|
-
[source,console]
|
|
223
|
-
----
|
|
224
|
-
$ relaton yaml2html YAML [<stylesheet>] [<liquid-template-dir>]
|
|
225
|
-
----
|
|
226
|
-
|
|
227
|
-
Render a Relaton YAML file as an HTML file. The `stylesheet` and `liquid-template-dir` parameters are as for <<relaton-xml2html,relaton xml2html>>.
|
|
228
|
-
|
|
229
|
-
[[relaton-index]]
|
|
230
|
-
=== relaton index
|
|
231
|
-
|
|
232
|
-
[source,console]
|
|
233
|
-
----
|
|
234
|
-
$ relaton index [DATA-DIR] [options]
|
|
235
|
-
----
|
|
236
|
-
|
|
237
|
-
Build a modern, browsable HTML index site from a folder of Relaton
|
|
238
|
-
bibliographic YAML documents (default `./data`, arbitrarily nested). This is the
|
|
239
|
-
command that generates the GitHub Pages index for `relaton-data-<flavor>`
|
|
240
|
-
repositories (e.g. https://relaton.github.io/relaton-data-bipm/).
|
|
241
|
-
|
|
242
|
-
The generated `index.html` inlines a compiled Vue + Tailwind frontend (search,
|
|
243
|
-
doctype/stage filters, sort, list/grid, dark mode, copy-DocID, responsive
|
|
244
|
-
pagination). It carries **no document data of its own** — the page is a fixed
|
|
245
|
-
~110 KB regardless of whether the corpus holds 200 documents or 200,000. The
|
|
246
|
-
documents live in numbered JSON shards written alongside it, which the page
|
|
247
|
-
fetches:
|
|
248
|
-
|
|
249
|
-
[source]
|
|
250
|
-
----
|
|
251
|
-
_site/index.html branding + the inlined bundle + the shard layout
|
|
252
|
-
_site/search-0000.json summary records, --shard-size each
|
|
253
|
-
_site/search-0001.json …loaded progressively: the first shard paints,
|
|
254
|
-
the rest stream in behind it
|
|
255
|
-
_site/detail-0000.json the richer fields, --detail-shard-size each
|
|
256
|
-
_site/detail-0001.json …fetched only when a detail page is opened
|
|
257
|
-
----
|
|
258
|
-
|
|
259
|
-
This keeps page weight a function of what the reader is looking at rather than
|
|
260
|
-
of corpus size. It matters at the top of the fleet: in the previous
|
|
261
|
-
single-page design `relaton-data-3gpp` (88,464 documents) published a
|
|
262
|
-
151,688,883-byte `index.html`, and `relaton-data-ids` (166,658 documents) would
|
|
263
|
-
have published roughly twice that. The cost is not really the download —
|
|
264
|
-
GitHub Pages gzips, so that page transfers in ~6.4 MB — but the browser still
|
|
265
|
-
has to parse it and build a DOM node per document before anything appears.
|
|
266
|
-
|
|
267
|
-
Clicking a document opens a shareable **detail page** (`?doc=<id>`) showing its
|
|
268
|
-
full bibliographic details — abstract, contributors, all identifiers and dates,
|
|
269
|
-
edition, languages, keywords, publisher, and the landing/raw-YAML links. Those
|
|
270
|
-
fields come from the `detail-NNNN.json` shard holding that document, fetched
|
|
271
|
-
when the panel opens; each block renders as it arrives, and a document with no
|
|
272
|
-
such fields simply shows its summary and links.
|
|
273
|
-
|
|
274
|
-
[NOTE]
|
|
275
|
-
====
|
|
276
|
-
Because the page fetches its data, it must be served over `http(s)://`. Opening
|
|
277
|
-
`_site/index.html` straight off the filesystem shows an empty index — browsers
|
|
278
|
-
block `fetch` from `file://`. To preview a build locally:
|
|
279
|
-
|
|
280
|
-
[source,console]
|
|
281
|
-
----
|
|
282
|
-
$ cd _site && python3 -m http.server
|
|
283
|
-
----
|
|
284
|
-
|
|
285
|
-
Do **not** pre-compress the shards. GitHub Pages already serves them gzipped
|
|
286
|
-
(verified: a 32.9 MB `search.json` transfers in 4.8 MB); a stored `.json.gz`
|
|
287
|
-
would be served as `application/gzip` with no `Content-Encoding`, so the browser
|
|
288
|
-
would not transparently inflate it.
|
|
289
|
-
====
|
|
290
|
-
|
|
291
|
-
.Options
|
|
292
|
-
[cols="1,3"]
|
|
293
|
-
|===
|
|
294
|
-
| `-o, --output` | Output directory for the generated site (default `_site`).
|
|
295
|
-
| `--flavor` | Build from `relaton/relaton-data-<flavor>` (shallow-cloned) instead of a local folder.
|
|
296
|
-
| `--repo` | Build from a GitHub repo `ORG/NAME` (shallow-cloned) instead of a local folder.
|
|
297
|
-
| `--branch` | Branch to clone when `--flavor`/`--repo` is used (default: the repo's default branch).
|
|
298
|
-
| `-t, --title` | Index page title.
|
|
299
|
-
| `--description` | Index page description. Emitted as `<meta name="description">` (search-result blurb) and rendered as the header subtitle in place of the default sentence.
|
|
300
|
-
| `--favicon` | Favicon href for `<link rel="icon">`, emitted **verbatim**. Normally an absolute URL (e.g. the SDO's own logo). A site-relative path also works, but `relaton index` copies no assets — only the HTML shell and the JSON shards are written — so the caller must put that file into the deployed site itself. A known image extension (`.svg`, `.png`, `.ico`, `.gif`, `.jpg`) also sets `type`; anything else is left for the browser to sniff.
|
|
301
|
-
| `--base-url` | Base URL for the raw-YAML links (defaults to the raw GitHub URL when cloning).
|
|
302
|
-
| `--shard-size` | Summary records per `search-NNNN.json` (default `5000`, ≈1 MB per shard). Smaller shards paint sooner and cost more requests.
|
|
303
|
-
| `--detail-shard-size` | Detail records per `detail-NNNN.json` (default `500`). This is how much is fetched to open one detail page, so keep it modest.
|
|
304
|
-
| `--[no-]detail` | Emit the detail shards (default `true`). `--no-detail` roughly halves the deployed site and degrades detail pages to the summary fields plus links.
|
|
305
|
-
| `--[no-]static` | Also index a sibling `static/` folder next to `DATA-DIR` (auto-detected, default `true`). Pass `--no-static` to index `DATA-DIR` only.
|
|
306
|
-
| `--pubid-flavor` | Flavor whose pubid `Identifier` parses each docid and whose `INDEXFILE` names the published index (e.g. `iso`, `iho`, `iec`). **Required** unless `--no-machine-index`.
|
|
307
|
-
| `--[no-]machine-index` | Emit the machine-consumable index (default `true`). `--no-machine-index` builds the human site only, and is the way to run without `--pubid-flavor`.
|
|
308
|
-
| `--index-name` | Base name for the published index, overriding the flavor's `INDEXFILE` (e.g. `index-v2`). For a corpus that is not a relaton flavor. Must be a single file name — no `/` and no leading `.`.
|
|
309
|
-
| `--[no-]publish-data` | Copy the scanned corpus to `_site/data/` so document fetches share the Pages origin (default `false`). For a repo whose records are served only from Pages; it roughly doubles the deployed size, and most repos are already served by `raw.githubusercontent.com`.
|
|
310
|
-
| `-f, --overwrite` | Overwrite existing output files (default `true`).
|
|
311
|
-
|===
|
|
312
|
-
|
|
313
|
-
==== The machine index
|
|
314
|
-
|
|
315
|
-
Alongside the human site, `relaton index` publishes a machine-consumable copy of
|
|
316
|
-
the index so a Relaton client can resolve a reference from the Pages origin:
|
|
317
|
-
|
|
318
|
-
[source]
|
|
319
|
-
----
|
|
320
|
-
_site/index-v2.yaml the whole index, same shape as the repo's own
|
|
321
|
-
_site/index-v2.zip its zip sibling — what `Relaton::Index url:` fetches
|
|
322
|
-
_site/index/manifest.json the lookup rule
|
|
323
|
-
_site/index/shard-00000.json one bucket of rows
|
|
324
|
-
----
|
|
325
|
-
|
|
326
|
-
The monolith is named from the flavor's own `INDEXFILE` (hence `index-v2` for
|
|
327
|
-
ISO, `index-v3` for IHO), so it is the file that flavor's consumer already
|
|
328
|
-
fetches. Shard buckets are `crc32(pubid.root.number.to_s) % shards`, the same
|
|
329
|
-
key `Relaton::Index` narrows on, so a document family lands in one shard.
|
|
330
|
-
|
|
331
|
-
The wire contract — manifest fields, shard naming, and what the shards cannot
|
|
332
|
-
serve — is specified in `docs/data-repository-format.adoc` in the `relaton`
|
|
333
|
-
repo.
|
|
334
|
-
|
|
335
|
-
NOTE: Some `relaton-data-<flavor>` repos keep manually-curated bibliographic YAML
|
|
336
|
-
records (documents the crawler can't fetch — e.g. ISO/IEC Directives, JCGM/GUM
|
|
337
|
-
guides, NIST research-library metadata) in a `static/` folder alongside `data/`.
|
|
338
|
-
These are part of the served corpus (they appear in the repo's `index-vN.yaml`),
|
|
339
|
-
so `relaton index` includes them by default: it auto-detects a `static/` sibling
|
|
340
|
-
of `DATA-DIR` and folds its docs into the same site, with the raw-YAML links
|
|
341
|
-
resolving to `static/<path>`. If a document id appears in both folders the `data/`
|
|
342
|
-
copy wins. Use `--no-static` to suppress this.
|
|
343
|
-
|
|
344
|
-
.Examples
|
|
345
|
-
[source,console]
|
|
346
|
-
----
|
|
347
|
-
# From the checked-out data repo (as in the GitHub Action):
|
|
348
|
-
$ relaton index data --output _site --pubid-flavor bipm --base-url \
|
|
349
|
-
https://raw.githubusercontent.com/relaton/relaton-data-bipm/v2
|
|
350
|
-
|
|
351
|
-
# From a local folder of YAML docs (default ./data), human site only:
|
|
352
|
-
$ relaton index ./my-docs --title "My Standards Index" --no-machine-index
|
|
353
|
-
|
|
354
|
-
# Branded per flavor — title, search-result blurb and tab icon:
|
|
355
|
-
$ relaton index data --pubid-flavor w3c --title "W3C Index" \
|
|
356
|
-
--description "Welcome to the World Wide Web Consortium standards index site!" \
|
|
357
|
-
--favicon https://www.w3.org/assets/logos/w3c/w3c-no-bars.svg
|
|
358
|
-
|
|
359
|
-
# Ad-hoc, cloning the official data repo by flavor:
|
|
360
|
-
$ relaton index --flavor bipm --pubid-flavor bipm --output _site
|
|
361
|
-
|
|
362
|
-
# From an arbitrary GitHub repo, pinning the branch to clone:
|
|
363
|
-
$ relaton index --repo relaton/relaton-data-bipm --branch v2 \
|
|
364
|
-
--pubid-flavor bipm --output _site
|
|
365
|
-
|
|
366
|
-
# A corpus that is not a relaton flavor: name the index explicitly:
|
|
367
|
-
$ relaton index data --pubid-flavor iso --index-name index-v2 --output _site
|
|
368
|
-
|
|
369
|
-
# Tune the shard layout for a very large corpus:
|
|
370
|
-
$ relaton index data --pubid-flavor iso --shard-size 10000 \
|
|
371
|
-
--detail-shard-size 250 --output _site
|
|
372
|
-
|
|
373
|
-
# Skip the detail shards entirely (smaller site, summary-only detail pages):
|
|
374
|
-
$ relaton index data --pubid-flavor iso --no-detail --output _site
|
|
375
|
-
----
|
|
376
|
-
|
|
377
|
-
[IMPORTANT]
|
|
378
|
-
====
|
|
379
|
-
`--mode` (`embedded` / `dom` / `static-json`) has been **removed**. Those modes
|
|
380
|
-
existed to trade the crawler-indexable DOM against page weight; there is now one
|
|
381
|
-
delivery mode, so there is nothing to choose. A caller still passing `--mode`
|
|
382
|
-
fails with a usage error and a non-zero exit — deliberately loud, because
|
|
383
|
-
silently ignoring it would publish a site nobody asked for.
|
|
384
|
-
====
|
|
385
|
-
|
|
386
|
-
NOTE: When building from a git checkout of relaton-cli, the compiled frontend
|
|
387
|
-
must exist first: run `bundle exec rake build_frontend` (or
|
|
388
|
-
`cd frontend && npm install && npm run build`). The released gem already ships
|
|
389
|
-
the compiled bundle, so installed users need only Ruby.
|
|
390
|
-
|
|
391
|
-
To publish these indexes from CI (GitHub Pages), a `relaton-data-<flavor>` repo
|
|
392
|
-
calls the shared reusable workflow in `relaton/support`:
|
|
393
|
-
|
|
394
|
-
[source,yaml]
|
|
395
|
-
----
|
|
396
|
-
jobs:
|
|
397
|
-
deploy:
|
|
398
|
-
uses: relaton/support/.github/workflows/data-deploy.yml@main
|
|
399
|
-
----
|
|
400
|
-
|
|
401
|
-
That workflow installs relaton-cli, runs `relaton index`, and deploys the result
|
|
402
|
-
to GitHub Pages.
|
|
403
|
-
|
|
404
|
-
=== relaton version
|
|
405
|
-
|
|
406
|
-
[source,console]
|
|
407
|
-
----
|
|
408
|
-
$ relaton version
|
|
409
|
-
----
|
|
410
|
-
|
|
411
|
-
Display version information for the CLI and the bundled `relaton` gem (every
|
|
412
|
-
flavor now reports the single `relaton` version).
|
|
413
|
-
|
|
414
|
-
=== relaton collection
|
|
415
|
-
|
|
416
|
-
The `relaton collection` command provides subcommands for collection manipulation.
|
|
417
|
-
|
|
418
|
-
==== relaton collection create
|
|
419
|
-
|
|
420
|
-
----
|
|
421
|
-
$ relaton collection create COLLECTION -d DIRECTORY --author AUTHOR --title TITLE --doctype DOCTYPE
|
|
422
|
-
----
|
|
423
|
-
|
|
424
|
-
Create a new empty collection with the name `COLLECTION`.
|
|
425
|
-
|
|
426
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
427
|
-
* `AUTHOR`, `TITLE`, and `DOCTYPE` are optional metadata.
|
|
428
|
-
|
|
429
|
-
==== relaton collection info
|
|
430
|
-
|
|
431
|
-
----
|
|
432
|
-
$ relaton collection info COLLECTION -d DIRECTORY
|
|
433
|
-
----
|
|
434
|
-
|
|
435
|
-
Show information about `COLLECTION` (number of items, file size, last updated, name, metadata).
|
|
436
|
-
|
|
437
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
438
|
-
|
|
439
|
-
==== relaton collection list
|
|
440
|
-
|
|
441
|
-
----
|
|
442
|
-
$ relaton collection list -d DIRECTORY -e
|
|
443
|
-
----
|
|
444
|
-
|
|
445
|
-
List all collections.
|
|
446
|
-
|
|
447
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
448
|
-
* `-e` - when set, list the document ID of each entry.
|
|
449
|
-
|
|
450
|
-
==== relaton collection get
|
|
451
|
-
|
|
452
|
-
----
|
|
453
|
-
$ relaton collection get CODE -c COLLECTION -d DIRECTORY -f FORMAT -o FILE
|
|
454
|
-
----
|
|
455
|
-
|
|
456
|
-
Get a document matching `CODE` from `COLLECTION`.
|
|
457
|
-
|
|
458
|
-
* `COLLECTION` - optional name of a collection. If omitted, fetches the first match across all collections in `DIRECTORY`.
|
|
459
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
460
|
-
* `FORMAT` - optional. If omitted, output is in human-readable form. Allowed values: `abb` (AsciiBib), `xml` (XML).
|
|
461
|
-
* `FILE` - optional. Save the document with the given filename. The file extension (`abb`, `xml`) defines the output format.
|
|
462
|
-
|
|
463
|
-
==== relaton collection find
|
|
464
|
-
|
|
465
|
-
----
|
|
466
|
-
$ relaton collection find TEXT -c COLLECTION -d DIRECTORY
|
|
467
|
-
----
|
|
468
|
-
|
|
469
|
-
Full-text search through a collection or all collections.
|
|
470
|
-
|
|
471
|
-
* `COLLECTION` - optional name of a collection. If omitted, searches across all collections.
|
|
472
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
473
|
-
|
|
474
|
-
==== relaton collection fetch
|
|
475
|
-
|
|
476
|
-
----
|
|
477
|
-
$ relaton collection fetch CODE -t TYPE -y YEAR -c COLLECTION -d DIRECTORY --publication-date-before DATE --publication-date-after DATE
|
|
478
|
-
----
|
|
479
|
-
|
|
480
|
-
Fetch the Relaton entry for document identifier `CODE` and save it into `COLLECTION`.
|
|
481
|
-
|
|
482
|
-
* `TYPE` - specifies the standards class library. See <<relaton-fetch-types,recognized TYPE values>>.
|
|
483
|
-
* `YEAR` - optional year of publication.
|
|
484
|
-
* `COLLECTION` - name of a collection (required).
|
|
485
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
486
|
-
* `--publication-date-before DATE` fetch only documents published before the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`.
|
|
487
|
-
* `--publication-date-after DATE` fetch only documents published after the specified date. Accepted formats: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`.
|
|
488
|
-
|
|
489
|
-
==== relaton collection export
|
|
490
|
-
|
|
491
|
-
----
|
|
492
|
-
$ relaton collection export COLLECTION -d DIRECTORY
|
|
493
|
-
----
|
|
494
|
-
|
|
495
|
-
Export `COLLECTION` into an XML file.
|
|
496
|
-
|
|
497
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
498
|
-
|
|
499
|
-
==== relaton collection import
|
|
500
|
-
|
|
501
|
-
----
|
|
502
|
-
$ relaton collection import FILE -c COLLECTION -d DIRECTORY
|
|
503
|
-
----
|
|
504
|
-
|
|
505
|
-
Import a document or collection from XML `FILE` into `COLLECTION`.
|
|
506
|
-
|
|
507
|
-
* `COLLECTION` - optional. If the collection does not exist, it will be created.
|
|
508
|
-
* `DIRECTORY` - optional path to a directory with collections. Default: `$HOME/.relaton/collections`.
|
|
509
|
-
|
|
510
|
-
=== Database manipulation
|
|
511
|
-
|
|
512
|
-
==== relaton db create
|
|
513
|
-
|
|
514
|
-
----
|
|
515
|
-
$ relaton db create DIR
|
|
516
|
-
----
|
|
517
|
-
|
|
518
|
-
Create a new cache database in directory `DIR` (optional, default: `$HOME/.relaton/cache`). If the target directory exists, it will be used as a database.
|
|
519
|
-
|
|
520
|
-
==== relaton db mv
|
|
521
|
-
|
|
522
|
-
----
|
|
523
|
-
$ relaton db mv DIR
|
|
524
|
-
----
|
|
525
|
-
|
|
526
|
-
Move the cache database to a new directory `DIR`.
|
|
527
|
-
|
|
528
|
-
==== relaton db clear
|
|
529
|
-
|
|
530
|
-
----
|
|
531
|
-
$ relaton db clear
|
|
532
|
-
----
|
|
533
|
-
|
|
534
|
-
Delete all entries from the cache database.
|
|
535
|
-
|
|
536
|
-
==== relaton db fetch
|
|
537
|
-
|
|
538
|
-
----
|
|
539
|
-
$ relaton db fetch CODE -t TYPE -f FORMAT -y YEAR --publication-date-before DATE --publication-date-after DATE
|
|
540
|
-
----
|
|
541
|
-
|
|
542
|
-
Fetch an entry from the cache database. Arguments are the same as for `relaton fetch`.
|
|
543
|
-
|
|
544
|
-
==== relaton db fetch_all
|
|
545
|
-
|
|
546
|
-
----
|
|
547
|
-
$ relaton db fetch_all TEXT -e EDITION -y YEAR -f FORMAT
|
|
548
|
-
----
|
|
549
|
-
|
|
550
|
-
Fetch all entries from the cache database.
|
|
551
|
-
|
|
552
|
-
* `TEXT` - optional, search for a specific string.
|
|
553
|
-
* `EDITION` - optional, filter documents by edition.
|
|
554
|
-
* `YEAR` - optional, filter documents by year.
|
|
555
|
-
* `FORMAT` - optional output format: `xml` (default), `yaml`, `bibtex`.
|
|
556
|
-
|
|
557
|
-
==== relaton db doctype
|
|
558
|
-
|
|
559
|
-
----
|
|
560
|
-
$ relaton db doctype REF
|
|
561
|
-
----
|
|
562
|
-
|
|
563
|
-
Detect and display the document type from reference `REF`.
|
|
564
|
-
|
|
565
|
-
----
|
|
566
|
-
$ relaton db doctype 'CN(GB/T 1.1)'
|
|
567
|
-
Chinese Standard
|
|
568
|
-
GB/T 1.1
|
|
569
|
-
----
|
|
570
|
-
|
|
571
|
-
== Development
|
|
572
|
-
|
|
573
|
-
[source,console]
|
|
574
|
-
----
|
|
575
|
-
$ bundle install
|
|
576
|
-
$ bundle exec rspec
|
|
577
|
-
$ bundle exec rubocop
|
|
578
|
-
----
|
|
579
|
-
|
|
580
|
-
== Contributing
|
|
581
|
-
|
|
582
|
-
Open an issue or pull request at https://github.com/relaton/relaton.
|
|
583
|
-
|
|
584
|
-
This gem is developed, maintained, and funded by
|
|
585
|
-
https://www.ribose.com[Ribose Inc.]
|
|
586
|
-
|
|
587
|
-
== License
|
|
588
|
-
|
|
589
|
-
BSD 2-Clause. See link:../LICENSE[LICENSE] for details.
|
data/exe/relaton
DELETED
|
@@ -1,20 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env ruby
|
|
2
|
-
# encoding: UTF-8
|
|
3
|
-
|
|
4
|
-
# resolve bin path, ignoring symlinks
|
|
5
|
-
require "pathname"
|
|
6
|
-
bin_file = Pathname.new(__FILE__).realpath
|
|
7
|
-
|
|
8
|
-
# add self to libpath
|
|
9
|
-
$:.unshift File.expand_path("../../lib", bin_file)
|
|
10
|
-
|
|
11
|
-
# Fixes https://github.com/rubygems/rubygems/issues/1420
|
|
12
|
-
require "rubygems/specification"
|
|
13
|
-
|
|
14
|
-
class Gem::Specification
|
|
15
|
-
def this; self; end
|
|
16
|
-
end
|
|
17
|
-
|
|
18
|
-
require "relaton-cli"
|
|
19
|
-
|
|
20
|
-
Relaton::Cli.start(ARGV)
|
data/frontend/dist/404.html
DELETED
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
<!doctype html>
|
|
2
|
-
<html>
|
|
3
|
-
<head>
|
|
4
|
-
<meta charset="utf-8">
|
|
5
|
-
<title>Redirecting…</title>
|
|
6
|
-
<script>
|
|
7
|
-
// GitHub Pages has no URL rewrites, so path-based document URLs
|
|
8
|
-
// (/doc/<id>) land on this 404 fallback. Hand the id to the app as
|
|
9
|
-
// ?doc=<id>, preserving the site's base path wherever it is hosted.
|
|
10
|
-
(function () {
|
|
11
|
-
var m = location.pathname.match(/\/doc\/([^/?#]+)$/);
|
|
12
|
-
var root = m ? location.pathname.slice(0, m.index) : location.pathname.replace(/[^/]*$/, "");
|
|
13
|
-
location.replace(root + "?doc=" + (m ? m[1] : ""));
|
|
14
|
-
})();
|
|
15
|
-
</script>
|
|
16
|
-
</head>
|
|
17
|
-
<body></body>
|
|
18
|
-
</html>
|