glossarist 2.10.4 → 2.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CLAUDE.md +24 -11
- data/lib/glossarist/cli/export_command.rb +35 -9
- data/lib/glossarist/glossary_store.rb +43 -0
- data/lib/glossarist/rdf/gloss_figure.rb +9 -2
- data/lib/glossarist/rdf/gloss_figure_image.rb +43 -0
- data/lib/glossarist/rdf/namespaces.rb +3 -1
- data/lib/glossarist/rdf/v3.rb +4 -0
- data/lib/glossarist/rdf.rb +11 -2
- data/lib/glossarist/schema_migration/cli_pipeline.rb +168 -0
- data/lib/glossarist/schema_migration.rb +27 -139
- data/lib/glossarist/transforms/concept_to_gloss_transform.rb +24 -4
- data/lib/glossarist/version.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b70c846aebbbbd0af25b5ba83b94ea880631518cf60b7a8ce39911275ab44ee4
|
|
4
|
+
data.tar.gz: bada2553c608f3816aa6532313cededc476680e52f5efdf6bf0285061340ade9
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 40bd8e71cbb34e419bb1ed4d82f1474666c82d29467afbc9b6e5537f7e706a61f8d53fb6682a25a65040b1e89d7b59ecacec2f9af77e9305ec2ce5f561c6f769
|
|
7
|
+
data.tar.gz: e39ba317e19833514190bcd693538b44f75241f07ba03ca25e261ac56efaa3671acb2802a505743cfc1bc4d0e52f0a3f11d8f7765be1a64ad44ecd0d8a01f106
|
data/CLAUDE.md
CHANGED
|
@@ -61,7 +61,7 @@ Designation inheritance hierarchy (MECE):
|
|
|
61
61
|
|
|
62
62
|
### Dataset Loading
|
|
63
63
|
|
|
64
|
-
- **`GlossaryStore`** (`glossary_store.rb`) — the dataset abstraction. Backed by `Lutaml::Store::PackageStore`, handles loading/saving from directories and ZIPs, concept CRUD, metadata, bibliography, images, and stats. Format detection is model-driven via `ConceptDocument.for_version`. **All callers that need to load concepts from a dataset should use GlossaryStore.**
|
|
64
|
+
- **`GlossaryStore`** (`glossary_store.rb`) — the dataset abstraction. Backed by `Lutaml::Store::PackageStore`, handles loading/saving from directories and ZIPs, concept CRUD, metadata, bibliography, images, dataset-level non-verbal entities (`#figures`, `#tables`, `#formulas` lazy-loaded from `figures/`/`tables/`/`formulas/` subdirectories), and stats. Format detection is model-driven via `ConceptDocument.for_version`. **All callers that need to load concepts from a dataset should use GlossaryStore.**
|
|
65
65
|
- **`ConceptCollector`** (`concept_collector.rb`) — legacy scanner with hand-rolled format detection (file-system heuristics, not model-driven). Being replaced by GlossaryStore. Do not add new callers.
|
|
66
66
|
- **`ConceptManager`** (`concept_manager.rb`) — legacy file I/O used by `ManagedConceptCollection`. Being replaced by GlossaryStore. Do not add new callers.
|
|
67
67
|
- **`ManagedConceptCollection`** (`managed_concept_collection.rb`) — legacy collection that loads via ConceptManager. Being replaced by GlossaryStore.
|
|
@@ -132,7 +132,8 @@ The `exe/glossarist` executable uses Thor. Commands:
|
|
|
132
132
|
|
|
133
133
|
- **`ConceptToTbxTransform`** (`transforms/concept_to_tbx_transform.rb`) — converts ManagedConcept to TBX-XML using the tbx gem (ISO 30042:2019). Produces `Tbx::ConceptEntry` per concept or `Tbx::Document` for full export.
|
|
134
134
|
- **`ConceptToSkosTransform`** (`transforms/concept_to_skos_transform.rb`) — converts ManagedConcept to SKOS RDF using `Glossarist::Rdf::SkosConcept`. Has `transform` (single) and `transform_document` (batch, returns `SkosVocabulary`). Produces JSON-LD and Turtle via the unified `rdf` DSL.
|
|
135
|
-
-
|
|
135
|
+
- **`ConceptToGlossTransform`** (`transforms/concept_to_gloss_transform.rb`) — converts ManagedConcept and dataset-level non-verbal entities to ontology-faithful RDF using `Glossarist::Rdf::Gloss*` view classes. `transform_document(concepts, figures:, tables:, formulas:)` and `to_turtle`/`to_jsonld` accept figures/tables/formulas kwargs (backward compatible — empty defaults). Uses a `DESIGNATION_BUILDERS` registry keyed by exact Designation subclass (OCP) instead of a case/when switch.
|
|
136
|
+
- **SKOS/RDF models** (`lib/glossarist/rdf/`) — `GlossConcept`, `GlossLocalizedConcept`, `GlossDesignation` and subclasses, `GlossFigure`/`GlossTable`/`GlossFormula`/`GlossFigureImage` (dataset-level non-verbal entities per concept-model v3.1.0 K1/K2 shapes), `SkosConcept`, `SkosVocabulary`, `LocalizedLiteral`, namespace classes. `EmitsExtraTriples` is autoloaded from `lutaml_ext.rb` via the immediate parent namespace file (`rdf.rb`); `LutamlTurtleTransformExt` is prepended into `Lutaml::Turtle::Transform` so the framework asks each instance for extra triples.
|
|
136
137
|
- TBX, Turtle, JSON-LD, JSONL export all write a single document file; JSON writes per-concept files.
|
|
137
138
|
|
|
138
139
|
### Dependencies
|
|
@@ -142,14 +143,28 @@ The `exe/glossarist` executable uses Thor. Commands:
|
|
|
142
143
|
- `relaton` (>= 2.0.0, < 3) — bibliography database integration
|
|
143
144
|
- `thor` — CLI commands
|
|
144
145
|
|
|
145
|
-
##
|
|
146
|
+
## Dependencies
|
|
146
147
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
-
|
|
150
|
-
|
|
151
|
-
-
|
|
152
|
-
-
|
|
148
|
+
- `relaton` (>= 2.0.0, < 3) — bibliography database integration. Upstream
|
|
149
|
+
shipped 2.1.0 with lutaml-model 0.8 compatibility, so the historical
|
|
150
|
+
git-branch overrides are no longer needed and the Gemfile pins the
|
|
151
|
+
released gem directly.
|
|
152
|
+
- `lutaml-model` (~> 0.8.5), `lutaml-store` (~> 0.2.0) — serialization
|
|
153
|
+
framework (YAML/XML/JSON-LD/Turtle) and package store.
|
|
154
|
+
- `tbx` — ISO 30042:2019 TBX model classes.
|
|
155
|
+
- `thor` — CLI commands.
|
|
156
|
+
- `rdf-turtle` (~> 3.3), `shacl` (~> 0.4) — Turtle emission + SHACL
|
|
157
|
+
validation against vendored concept-model shapes.
|
|
158
|
+
|
|
159
|
+
## SchemaMigration module split
|
|
160
|
+
|
|
161
|
+
`SchemaMigration` is a thin facade over three single-concern modules:
|
|
162
|
+
|
|
163
|
+
- `SchemaMigration::V0ToV1` — hash-to-hash transform (IEV legacy YAML → V1 shape). Single `#migrate` entry point.
|
|
164
|
+
- `SchemaMigration::V2ToV3` — model-to-model transform (`V2::ManagedConcept` → `V3::ManagedConcept`). Class methods `migrate_concept` / `concept_version`.
|
|
165
|
+
- `SchemaMigration::CliPipeline` — file I/O + dispatch + output writing for `glossarist upgrade`. Single `#run` entry point; output can be a directory of YAML files or a `.gcr` ZIP package.
|
|
166
|
+
|
|
167
|
+
The facade preserves the historical class-method API (`SchemaMigration.new`, `.migrate_concept`, `.concept_version`, `.upgrade_directory`) so existing callers don't break. Adding V3→V4 later means adding a new module under `SchemaMigration::`, not editing these.
|
|
153
168
|
|
|
154
169
|
## Schema Version Subclasses Are NOT Duplication
|
|
155
170
|
|
|
@@ -170,8 +185,6 @@ See `TODO.improve/` for detailed plans.
|
|
|
170
185
|
|
|
171
186
|
1. **Migrate callers to GlossaryStore** — ConceptCollector (8 call sites) and ConceptManager (1 call site) duplicate format detection with file-system heuristics. GlossaryStore is model-driven, framework-aligned, and handles both directory and ZIP. All callers should go through GlossaryStore.
|
|
172
187
|
|
|
173
|
-
2. **Split SchemaMigration into focused modules** — Currently handles V0→V1 hash migration, V2→V3 model migration, and a CLI pipeline at three different abstraction levels. Each should be its own module.
|
|
174
|
-
|
|
175
188
|
### Rejected candidates (do not re-suggest)
|
|
176
189
|
|
|
177
190
|
- **Collapse V2/V3 ManagedConceptData** — Version subclasses are a real seam, not duplication. See "Schema Version Subclasses Are NOT Duplication" above.
|
|
@@ -52,13 +52,17 @@ module Glossarist
|
|
|
52
52
|
private
|
|
53
53
|
|
|
54
54
|
def load_concepts
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
55
|
+
store.concepts
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def store
|
|
59
|
+
@store ||= if @path.end_with?(".gcr")
|
|
60
|
+
pkg = GcrPackage.load(@path)
|
|
61
|
+
resolve_metadata_from_package(pkg)
|
|
62
|
+
pkg
|
|
63
|
+
else
|
|
64
|
+
GlossaryStore.new.tap { |s| s.load(@path) }
|
|
65
|
+
end
|
|
62
66
|
end
|
|
63
67
|
|
|
64
68
|
def resolve_metadata_from_package(package)
|
|
@@ -106,6 +110,22 @@ module Glossarist
|
|
|
106
110
|
@options.fetch(:validate, false)
|
|
107
111
|
end
|
|
108
112
|
|
|
113
|
+
# Dataset-level non-verbal entities come from the GlossaryStore (which
|
|
114
|
+
# lazy-loads figures/tables/formulas from the dataset directory). GCR
|
|
115
|
+
# packages don't yet expose these — empty arrays keep the call site
|
|
116
|
+
# uniform. Future: extend GcrPackage to surface them.
|
|
117
|
+
def dataset_figures
|
|
118
|
+
@store.is_a?(GlossaryStore) ? @store.figures : []
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
def dataset_tables
|
|
122
|
+
@store.is_a?(GlossaryStore) ? @store.tables : []
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
def dataset_formulas
|
|
126
|
+
@store.is_a?(GlossaryStore) ? @store.formulas : []
|
|
127
|
+
end
|
|
128
|
+
|
|
109
129
|
def per_concept_supported?(format)
|
|
110
130
|
PER_CONCEPT_FORMATS.include?(format)
|
|
111
131
|
end
|
|
@@ -136,7 +156,10 @@ module Glossarist
|
|
|
136
156
|
transform = Transforms::ConceptToGlossTransform.new(nil,
|
|
137
157
|
transform_options)
|
|
138
158
|
File.write(File.join(output_dir, "#{name}.jsonld"),
|
|
139
|
-
transform.to_jsonld(concepts
|
|
159
|
+
transform.to_jsonld(concepts,
|
|
160
|
+
figures: dataset_figures,
|
|
161
|
+
tables: dataset_tables,
|
|
162
|
+
formulas: dataset_formulas))
|
|
140
163
|
end
|
|
141
164
|
|
|
142
165
|
def export_turtle(concepts, name, output_dir)
|
|
@@ -144,7 +167,10 @@ module Glossarist
|
|
|
144
167
|
transform = Transforms::ConceptToGlossTransform.new(nil,
|
|
145
168
|
transform_options)
|
|
146
169
|
File.write(File.join(output_dir, "#{name}.ttl"),
|
|
147
|
-
transform.to_turtle(concepts
|
|
170
|
+
transform.to_turtle(concepts,
|
|
171
|
+
figures: dataset_figures,
|
|
172
|
+
tables: dataset_tables,
|
|
173
|
+
formulas: dataset_formulas))
|
|
148
174
|
end
|
|
149
175
|
|
|
150
176
|
def export_tbx(concepts, name, output_dir)
|
|
@@ -9,11 +9,16 @@ module Glossarist
|
|
|
9
9
|
@concept_document_class = V3::ConceptDocument
|
|
10
10
|
@v1_concepts = nil
|
|
11
11
|
@localized_concepts_dir_name = nil
|
|
12
|
+
@dataset_path = nil
|
|
13
|
+
@figures = nil
|
|
14
|
+
@tables = nil
|
|
15
|
+
@formulas = nil
|
|
12
16
|
end
|
|
13
17
|
|
|
14
18
|
# ── Load ──
|
|
15
19
|
|
|
16
20
|
def load_directory(path, format: nil)
|
|
21
|
+
@dataset_path = File.expand_path(path)
|
|
17
22
|
if v1_dataset?(path)
|
|
18
23
|
load_v1_fallback(path)
|
|
19
24
|
return self
|
|
@@ -44,6 +49,7 @@ module Glossarist
|
|
|
44
49
|
end
|
|
45
50
|
|
|
46
51
|
def load_zip(path, format: nil)
|
|
52
|
+
@dataset_path = File.expand_path(path)
|
|
47
53
|
metadata = load_metadata_from_zip(path)
|
|
48
54
|
@concept_document_class = resolve_concept_document_class(metadata)
|
|
49
55
|
|
|
@@ -174,6 +180,28 @@ module Glossarist
|
|
|
174
180
|
@package.asset_paths.select { |p| p.start_with?("images/") }
|
|
175
181
|
end
|
|
176
182
|
|
|
183
|
+
# ── Dataset-level non-verbal entities ──
|
|
184
|
+
#
|
|
185
|
+
# Loaded lazily from the dataset's figures/, tables/, formulas/
|
|
186
|
+
# subdirectories. Each YAML file is parsed as the corresponding model
|
|
187
|
+
# (Figure, Table, Formula). The entities are referenced from
|
|
188
|
+
# ManagedConceptData via FigureReference/TableReference/FormulaReference
|
|
189
|
+
# but live as standalone shared entities at the dataset level.
|
|
190
|
+
# Wired into ConceptToGlossTransform via the figures:/tables:/formulas:
|
|
191
|
+
# kwargs on transform_document.
|
|
192
|
+
|
|
193
|
+
def figures
|
|
194
|
+
@figures ||= load_dataset_entities("figures", Figure)
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
def tables
|
|
198
|
+
@tables ||= load_dataset_entities("tables", Table)
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
def formulas
|
|
202
|
+
@formulas ||= load_dataset_entities("formulas", Formula)
|
|
203
|
+
end
|
|
204
|
+
|
|
177
205
|
# ── Stats ──
|
|
178
206
|
|
|
179
207
|
def stats
|
|
@@ -201,6 +229,21 @@ module Glossarist
|
|
|
201
229
|
@package = Lutaml::Store::PackageStore.new(definition)
|
|
202
230
|
end
|
|
203
231
|
|
|
232
|
+
# Scans `{dataset_path}/{subdir}/*.{yaml,yml}` and parses each file as
|
|
233
|
+
# the given model class. Returns [] when the directory doesn't exist
|
|
234
|
+
# (e.g., datasets that don't carry this entity kind) — absent directory
|
|
235
|
+
# is not an error, just an empty collection.
|
|
236
|
+
def load_dataset_entities(subdir, klass)
|
|
237
|
+
return [] unless @dataset_path
|
|
238
|
+
|
|
239
|
+
dir = File.join(@dataset_path, subdir)
|
|
240
|
+
return [] unless File.directory?(dir)
|
|
241
|
+
|
|
242
|
+
Dir.glob(File.join(dir, "*.{yaml,yml}")).filter_map do |path|
|
|
243
|
+
klass.from_file(path)
|
|
244
|
+
end
|
|
245
|
+
end
|
|
246
|
+
|
|
204
247
|
def resolve_concept_document_class(metadata)
|
|
205
248
|
version = metadata&.schema_version.to_s
|
|
206
249
|
ConceptDocument.for_version(version)
|
|
@@ -4,13 +4,18 @@ require "lutaml/model"
|
|
|
4
4
|
|
|
5
5
|
module Glossarist
|
|
6
6
|
module Rdf
|
|
7
|
-
# RDF view for a dataset-level Figure entity (concept-model v3.1.0 K1).
|
|
7
|
+
# RDF view for a dataset-level Figure entity (concept-model v3.1.0 K1+K2).
|
|
8
8
|
#
|
|
9
9
|
# Emits a `gloss:Figure` subject with:
|
|
10
|
-
# - gloss:image — xsd:anyURI (the first image variant's src
|
|
10
|
+
# - gloss:image — xsd:anyURI (the first image variant's src, per K1
|
|
11
|
+
# FigureShape which constrains it to min 1)
|
|
11
12
|
# - gloss:caption — xsd:string (one language picked from the localized hash)
|
|
12
13
|
# - dcterms:description — xsd:string (one language picked)
|
|
13
14
|
#
|
|
15
|
+
# Per-variant foaf:Image subjects (K2 ImageShape) are emitted via the
|
|
16
|
+
# `images` collection — each variant becomes its own subject with
|
|
17
|
+
# dcterms:format and gloss:imageRole.
|
|
18
|
+
#
|
|
14
19
|
# The Figure domain model (Glossarist::Figure) stores caption/description
|
|
15
20
|
# as language-keyed hashes; the K1 FigureShape constrains them to single
|
|
16
21
|
# xsd:string values, so the transform picks one language at build time.
|
|
@@ -20,6 +25,7 @@ module Glossarist
|
|
|
20
25
|
attribute :image, :string
|
|
21
26
|
attribute :caption, :string
|
|
22
27
|
attribute :description, :string
|
|
28
|
+
attribute :images, GlossFigureImage, collection: true
|
|
23
29
|
|
|
24
30
|
rdf do
|
|
25
31
|
namespace Namespaces::GlossaristNamespace,
|
|
@@ -35,6 +41,7 @@ module Glossarist
|
|
|
35
41
|
to: :caption
|
|
36
42
|
predicate :description, namespace: Namespaces::DctermsNamespace,
|
|
37
43
|
to: :description
|
|
44
|
+
members :images
|
|
38
45
|
end
|
|
39
46
|
end
|
|
40
47
|
end
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "lutaml/model"
|
|
4
|
+
|
|
5
|
+
module Glossarist
|
|
6
|
+
module Rdf
|
|
7
|
+
# RDF view for one image variant within a Figure (concept-model v3.1.0 K2).
|
|
8
|
+
#
|
|
9
|
+
# Emits a `foaf:Image` subject with:
|
|
10
|
+
# - dcterms:format — required by K2 ImageShape (e.g. "svg", "png")
|
|
11
|
+
# - dcterms:language — optional (max 1) — when the variant is localized
|
|
12
|
+
# - dcat:byteSize — optional (max 1) — when the source model carries it
|
|
13
|
+
# - gloss:imageRole — the variant's role (vector/raster/dark/light/print)
|
|
14
|
+
#
|
|
15
|
+
# The FigureImage domain model carries src, format, role, width, height,
|
|
16
|
+
# scale. byte_size isn't on the model (it would require file-system
|
|
17
|
+
# inspection); we emit nil when absent, which the RDF layer skips.
|
|
18
|
+
class GlossFigureImage < Lutaml::Model::Serializable
|
|
19
|
+
attribute :src, :string
|
|
20
|
+
attribute :format, :string
|
|
21
|
+
attribute :role, :string
|
|
22
|
+
attribute :byte_size, :integer
|
|
23
|
+
|
|
24
|
+
rdf do
|
|
25
|
+
namespace Namespaces::DctermsNamespace,
|
|
26
|
+
Namespaces::FoafNamespace,
|
|
27
|
+
Namespaces::DcatNamespace,
|
|
28
|
+
Namespaces::GlossaristNamespace
|
|
29
|
+
|
|
30
|
+
subject { |i| "image/#{i.src}" }
|
|
31
|
+
|
|
32
|
+
types "foaf:Image"
|
|
33
|
+
|
|
34
|
+
predicate :format, namespace: Namespaces::DctermsNamespace,
|
|
35
|
+
to: :format
|
|
36
|
+
predicate :byteSize, namespace: Namespaces::DcatNamespace,
|
|
37
|
+
to: :byte_size
|
|
38
|
+
predicate :imageRole, namespace: Namespaces::GlossaristNamespace,
|
|
39
|
+
to: :role
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
@@ -3,7 +3,9 @@
|
|
|
3
3
|
module Glossarist
|
|
4
4
|
module Rdf
|
|
5
5
|
module Namespaces
|
|
6
|
-
autoload :
|
|
6
|
+
autoload :DcatNamespace, "#{__dir__}/namespaces/dcat_namespace"
|
|
7
|
+
autoload :DctermsNamespace, "#{__dir__}/namespaces/dcterms_namespace"
|
|
8
|
+
autoload :FoafNamespace, "#{__dir__}/namespaces/foaf_namespace"
|
|
7
9
|
autoload :GlossaristNamespace,
|
|
8
10
|
"#{__dir__}/namespaces/glossarist_namespace"
|
|
9
11
|
autoload :IsoThesNamespace, "#{__dir__}/namespaces/iso_thes_namespace"
|
data/lib/glossarist/rdf/v3.rb
CHANGED
data/lib/glossarist/rdf.rb
CHANGED
|
@@ -4,10 +4,18 @@ require "lutaml/rdf"
|
|
|
4
4
|
require "lutaml/turtle"
|
|
5
5
|
require "lutaml/jsonld"
|
|
6
6
|
|
|
7
|
-
require_relative "rdf/lutaml_ext"
|
|
8
|
-
|
|
9
7
|
module Glossarist
|
|
10
8
|
module Rdf
|
|
9
|
+
# lutaml_ext.rb defines EmitsExtraTriples (mixed into RDF model classes
|
|
10
|
+
# that want to emit extra triples) and LutamlTurtleTransformExt (prepended
|
|
11
|
+
# into Lutaml::Turtle::Transform so the framework asks each instance for
|
|
12
|
+
# its extra triples). The prepend must run before the first transform
|
|
13
|
+
# call; autoload here fires the first time EmitsExtraTriples is
|
|
14
|
+
# referenced (i.e., when GlossLocalizedConcept includes it during its
|
|
15
|
+
# own autoload), which is before any transform runs.
|
|
16
|
+
autoload :EmitsExtraTriples, "#{__dir__}/rdf/lutaml_ext"
|
|
17
|
+
autoload :LutamlTurtleTransformExt, "#{__dir__}/rdf/lutaml_ext"
|
|
18
|
+
|
|
11
19
|
autoload :Namespaces, "#{__dir__}/rdf/namespaces"
|
|
12
20
|
autoload :LocalizedLiteral, "#{__dir__}/rdf/localized_literal"
|
|
13
21
|
autoload :RelationshipPredicates, "#{__dir__}/rdf/relationship_predicates"
|
|
@@ -31,6 +39,7 @@ module Glossarist
|
|
|
31
39
|
autoload :GlossConcept, "#{__dir__}/rdf/gloss_concept"
|
|
32
40
|
autoload :GlossDocument, "#{__dir__}/rdf/gloss_concept"
|
|
33
41
|
autoload :GlossFigure, "#{__dir__}/rdf/gloss_figure"
|
|
42
|
+
autoload :GlossFigureImage, "#{__dir__}/rdf/gloss_figure_image"
|
|
34
43
|
autoload :GlossTable, "#{__dir__}/rdf/gloss_table"
|
|
35
44
|
autoload :GlossFormula, "#{__dir__}/rdf/gloss_formula"
|
|
36
45
|
autoload :V3, "#{__dir__}/rdf/v3"
|
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "fileutils"
|
|
4
|
+
|
|
5
|
+
module Glossarist
|
|
6
|
+
class SchemaMigration
|
|
7
|
+
# Orchestrator for the `glossarist upgrade` CLI command.
|
|
8
|
+
#
|
|
9
|
+
# Reads source concepts from a directory, dispatches each file to the
|
|
10
|
+
# right migrator (V0ToV1 for IEV-legacy hashes, V2ToV3 for model-level
|
|
11
|
+
# version bumps), reads register.yaml, and writes the output either as
|
|
12
|
+
# a directory of YAML files or as a `.gcr` ZIP package.
|
|
13
|
+
#
|
|
14
|
+
# This is the third concern that previously lived tangled in
|
|
15
|
+
# SchemaMigration itself (alongside V0ToV1 and V2ToV3). Splitting it
|
|
16
|
+
# out keeps each module at a single abstraction level:
|
|
17
|
+
# - V0ToV1: hash → hash transform
|
|
18
|
+
# - V2ToV3: model → model transform
|
|
19
|
+
# - CliPipeline (this file): file I/O + dispatch + output writing
|
|
20
|
+
class CliPipeline
|
|
21
|
+
attr_reader :source_dir, :output, :target_version, :cross_references,
|
|
22
|
+
:dry_run
|
|
23
|
+
|
|
24
|
+
def initialize(source_dir, output:, target_version:,
|
|
25
|
+
cross_references: nil, dry_run: false)
|
|
26
|
+
@source_dir = File.expand_path(source_dir)
|
|
27
|
+
@output = output
|
|
28
|
+
@target_version = target_version
|
|
29
|
+
@cross_references = cross_references
|
|
30
|
+
@dry_run = dry_run
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def run
|
|
34
|
+
validate_source!
|
|
35
|
+
concepts = read_and_migrate_concepts
|
|
36
|
+
register_data = read_register_yaml
|
|
37
|
+
write_output(concepts, register_data)
|
|
38
|
+
|
|
39
|
+
{
|
|
40
|
+
concepts: concepts,
|
|
41
|
+
register_data: register_data,
|
|
42
|
+
source_version: source_version,
|
|
43
|
+
target_version: target_version,
|
|
44
|
+
output: File.expand_path(output),
|
|
45
|
+
count: concepts.length,
|
|
46
|
+
}
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
private
|
|
50
|
+
|
|
51
|
+
def validate_source!
|
|
52
|
+
unless File.directory?(source_dir)
|
|
53
|
+
raise ArgumentError, "#{source_dir} is not a directory"
|
|
54
|
+
end
|
|
55
|
+
return if concepts_dir
|
|
56
|
+
|
|
57
|
+
raise ArgumentError, "No concept YAML files found in #{source_dir}"
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def concepts_dir
|
|
61
|
+
@concepts_dir ||= [
|
|
62
|
+
File.join(source_dir, "concepts"),
|
|
63
|
+
source_dir,
|
|
64
|
+
].find { |dir| Dir.glob(File.join(dir, "*.yaml")).any? }
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def source_version
|
|
68
|
+
@source_version ||= begin
|
|
69
|
+
register = V1::Register.from_file(File.join(source_dir,
|
|
70
|
+
"register.yaml"))
|
|
71
|
+
register&.schema_version || "0"
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def ref_maps
|
|
76
|
+
@ref_maps ||= begin
|
|
77
|
+
xref = V1::CrossReferences.from_file(cross_references)
|
|
78
|
+
xref ? xref.to_ref_maps : {}
|
|
79
|
+
end
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def read_and_migrate_concepts
|
|
83
|
+
files = Dir.glob(File.join(concepts_dir, "*.yaml"))
|
|
84
|
+
concepts = []
|
|
85
|
+
errors = 0
|
|
86
|
+
|
|
87
|
+
files.each do |file|
|
|
88
|
+
v1 = V1::Concept.from_file(file)
|
|
89
|
+
next unless v1
|
|
90
|
+
|
|
91
|
+
migration = V0ToV1.new(
|
|
92
|
+
v1.to_yaml_hash,
|
|
93
|
+
from_version: source_version,
|
|
94
|
+
to_version: target_version,
|
|
95
|
+
ref_maps: ref_maps,
|
|
96
|
+
)
|
|
97
|
+
concepts << migration.migrate
|
|
98
|
+
rescue Errors::Base, Psych::SyntaxError => e
|
|
99
|
+
errors += 1
|
|
100
|
+
warn " Error migrating #{File.basename(file)}: #{e.message}" if errors <= 5
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
warn " ... #{errors - 5} more errors" if errors > 5
|
|
104
|
+
concepts
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def read_register_yaml
|
|
108
|
+
register = V1::Register.from_file(File.join(source_dir,
|
|
109
|
+
"register.yaml"))
|
|
110
|
+
return nil unless register
|
|
111
|
+
|
|
112
|
+
data = register.to_h
|
|
113
|
+
data["schema_version"] = target_version
|
|
114
|
+
data
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def write_output(concepts, register_data)
|
|
118
|
+
output_path = File.expand_path(output)
|
|
119
|
+
|
|
120
|
+
if File.extname(output).downcase == ".gcr"
|
|
121
|
+
write_gcr(concepts, register_data, output_path)
|
|
122
|
+
else
|
|
123
|
+
write_directory(concepts, register_data, output_path)
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def write_gcr(concepts, register_data, output_path)
|
|
128
|
+
if dry_run
|
|
129
|
+
puts "Would package #{concepts.length} concepts into #{output_path}"
|
|
130
|
+
return
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
v1_concepts = concepts.map { |h| V1::Concept.of_yaml(h).to_managed_concept }
|
|
134
|
+
rd = register_data ? RegisterData.of_yaml(register_data) : nil
|
|
135
|
+
metadata = GcrMetadata.from_concepts(v1_concepts, register_data: rd)
|
|
136
|
+
GcrPackage.create(
|
|
137
|
+
concepts: v1_concepts,
|
|
138
|
+
metadata: metadata,
|
|
139
|
+
register_data: rd,
|
|
140
|
+
output_path: output_path,
|
|
141
|
+
)
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
def write_directory(concepts, register_data, output_path)
|
|
145
|
+
if dry_run
|
|
146
|
+
puts "Would write #{concepts.length} concepts to #{File.join(
|
|
147
|
+
output_path, 'concepts/'
|
|
148
|
+
)}"
|
|
149
|
+
return
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
concepts_out = File.join(output_path, "concepts")
|
|
153
|
+
FileUtils.mkdir_p(concepts_out)
|
|
154
|
+
|
|
155
|
+
concepts.each do |concept|
|
|
156
|
+
termid = concept["termid"]
|
|
157
|
+
mc = V1::Concept.of_yaml(concept).to_managed_concept
|
|
158
|
+
File.write(File.join(concepts_out, "#{termid}.yaml"), mc.to_yaml)
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
return unless register_data
|
|
162
|
+
|
|
163
|
+
rd = RegisterData.of_yaml(register_data)
|
|
164
|
+
File.write(File.join(output_path, "register.yaml"), rd.to_yaml)
|
|
165
|
+
end
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
end
|
|
@@ -1,14 +1,29 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
# Facade for schema migration. Each abstraction level lives in its own
|
|
4
|
+
# module under SchemaMigration:: — the facade preserves the historical
|
|
5
|
+
# class-method API so existing callers (CLI upgrade command, V2/V3
|
|
6
|
+
# namespace specs, downstream gems) don't break.
|
|
7
|
+
#
|
|
8
|
+
# Architecture (MECE):
|
|
9
|
+
# - V0ToV1: hash to hash transform (IEV legacy YAML to V1 shape).
|
|
10
|
+
# - V2ToV3: model to model transform (V2::ManagedConcept to V3::ManagedConcept).
|
|
11
|
+
# - CliPipeline: file I/O + dispatch + output writing for the
|
|
12
|
+
# `glossarist upgrade` CLI command.
|
|
13
|
+
#
|
|
14
|
+
# Each module has a single public entry point (V0ToV1#migrate,
|
|
15
|
+
# V2ToV3.migrate_concept, CliPipeline#run) and is independently testable.
|
|
16
|
+
# Adding V3ToV4 later = adding a new module, not editing these.
|
|
5
17
|
module Glossarist
|
|
6
18
|
class SchemaMigration
|
|
7
19
|
CURRENT_SCHEMA_VERSION = "1"
|
|
8
20
|
|
|
9
|
-
autoload :V0ToV1,
|
|
10
|
-
autoload :V2ToV3,
|
|
21
|
+
autoload :V0ToV1, "glossarist/schema_migration/v0_to_v1"
|
|
22
|
+
autoload :V2ToV3, "glossarist/schema_migration/v2_to_v3"
|
|
23
|
+
autoload :CliPipeline, "glossarist/schema_migration/cli_pipeline"
|
|
11
24
|
|
|
25
|
+
# Convenience: `SchemaMigration.new(input, **opts)` is shorthand for
|
|
26
|
+
# `V0ToV1.new(input, **opts)`. Preserved for backward compatibility.
|
|
12
27
|
def self.new(...)
|
|
13
28
|
V0ToV1.new(...)
|
|
14
29
|
end
|
|
@@ -21,142 +36,15 @@ module Glossarist
|
|
|
21
36
|
V2ToV3.concept_version(concept)
|
|
22
37
|
end
|
|
23
38
|
|
|
24
|
-
def self.upgrade_directory(source_dir, output:,
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
concepts_dir = find_concepts_dir(source_dir)
|
|
31
|
-
unless File.directory?(source_dir)
|
|
32
|
-
raise ArgumentError,
|
|
33
|
-
"#{source_dir} is not a directory"
|
|
34
|
-
end
|
|
35
|
-
unless concepts_dir
|
|
36
|
-
raise ArgumentError,
|
|
37
|
-
"No concept YAML files found in #{source_dir}"
|
|
38
|
-
end
|
|
39
|
-
|
|
40
|
-
source_version = detect_schema_version(source_dir)
|
|
41
|
-
ref_maps = load_ref_maps(cross_references)
|
|
42
|
-
concepts = read_and_migrate_concepts(concepts_dir, source_version,
|
|
43
|
-
target_version, ref_maps)
|
|
44
|
-
register_data = read_register_yaml(source_dir, target_version)
|
|
45
|
-
|
|
46
|
-
write_output(concepts, register_data, output, dry_run)
|
|
47
|
-
|
|
48
|
-
{
|
|
49
|
-
concepts: concepts,
|
|
50
|
-
register_data: register_data,
|
|
51
|
-
source_version: source_version,
|
|
39
|
+
def self.upgrade_directory(source_dir, output:, target_version: CURRENT_SCHEMA_VERSION,
|
|
40
|
+
cross_references: nil, dry_run: false)
|
|
41
|
+
CliPipeline.new(
|
|
42
|
+
source_dir,
|
|
43
|
+
output: output,
|
|
52
44
|
target_version: target_version,
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
end
|
|
57
|
-
|
|
58
|
-
class << self
|
|
59
|
-
private
|
|
60
|
-
|
|
61
|
-
def find_concepts_dir(source_dir)
|
|
62
|
-
candidates = [
|
|
63
|
-
File.join(source_dir, "concepts"),
|
|
64
|
-
source_dir,
|
|
65
|
-
]
|
|
66
|
-
candidates.find { |dir| Dir.glob(File.join(dir, "*.yaml")).any? }
|
|
67
|
-
end
|
|
68
|
-
|
|
69
|
-
def detect_schema_version(source_dir)
|
|
70
|
-
register = V1::Register.from_file(File.join(source_dir,
|
|
71
|
-
"register.yaml"))
|
|
72
|
-
register&.schema_version || "0"
|
|
73
|
-
end
|
|
74
|
-
|
|
75
|
-
def load_ref_maps(cross_references_path)
|
|
76
|
-
xref = V1::CrossReferences.from_file(cross_references_path)
|
|
77
|
-
xref ? xref.to_ref_maps : {}
|
|
78
|
-
end
|
|
79
|
-
|
|
80
|
-
def read_and_migrate_concepts(concepts_dir, source_version, # rubocop:disable Metrics/MethodLength
|
|
81
|
-
target_version, ref_maps)
|
|
82
|
-
files = Dir.glob(File.join(concepts_dir, "*.yaml"))
|
|
83
|
-
concepts = []
|
|
84
|
-
errors = 0
|
|
85
|
-
|
|
86
|
-
files.each do |file|
|
|
87
|
-
v1 = V1::Concept.from_file(file)
|
|
88
|
-
next unless v1
|
|
89
|
-
|
|
90
|
-
migration = V0ToV1.new(
|
|
91
|
-
v1.to_yaml_hash,
|
|
92
|
-
from_version: source_version,
|
|
93
|
-
to_version: target_version,
|
|
94
|
-
ref_maps: ref_maps,
|
|
95
|
-
)
|
|
96
|
-
concepts << migration.migrate
|
|
97
|
-
rescue Errors::Base, Psych::SyntaxError => e
|
|
98
|
-
errors += 1
|
|
99
|
-
warn " Error migrating #{File.basename(file)}: #{e.message}" if errors <= 5
|
|
100
|
-
end
|
|
101
|
-
|
|
102
|
-
warn " ... #{errors - 5} more errors" if errors > 5
|
|
103
|
-
concepts
|
|
104
|
-
end
|
|
105
|
-
|
|
106
|
-
def read_register_yaml(source_dir, target_version)
|
|
107
|
-
register = V1::Register.from_file(File.join(source_dir,
|
|
108
|
-
"register.yaml"))
|
|
109
|
-
return nil unless register
|
|
110
|
-
|
|
111
|
-
data = register.to_h
|
|
112
|
-
data["schema_version"] = target_version
|
|
113
|
-
data
|
|
114
|
-
end
|
|
115
|
-
|
|
116
|
-
def write_output(concepts, register_data, output, dry_run) # rubocop:disable Metrics/MethodLength
|
|
117
|
-
output_path = File.expand_path(output)
|
|
118
|
-
|
|
119
|
-
if File.extname(output).downcase == ".gcr"
|
|
120
|
-
if dry_run
|
|
121
|
-
puts "Would package #{concepts.length} concepts into #{output_path}"
|
|
122
|
-
return
|
|
123
|
-
end
|
|
124
|
-
|
|
125
|
-
v1_concepts = concepts.map { |h| V1::Concept.of_yaml(h).to_managed_concept }
|
|
126
|
-
rd = register_data ? RegisterData.of_yaml(register_data) : nil
|
|
127
|
-
metadata = GcrMetadata.from_concepts(v1_concepts,
|
|
128
|
-
register_data: rd)
|
|
129
|
-
GcrPackage.create(
|
|
130
|
-
concepts: v1_concepts,
|
|
131
|
-
metadata: metadata,
|
|
132
|
-
register_data: rd,
|
|
133
|
-
output_path: output_path,
|
|
134
|
-
)
|
|
135
|
-
else
|
|
136
|
-
if dry_run
|
|
137
|
-
puts "Would write #{concepts.length} concepts to #{File.join(
|
|
138
|
-
output_path, 'concepts/'
|
|
139
|
-
)}"
|
|
140
|
-
return
|
|
141
|
-
end
|
|
142
|
-
|
|
143
|
-
concepts_out = File.join(output_path, "concepts")
|
|
144
|
-
FileUtils.mkdir_p(concepts_out)
|
|
145
|
-
|
|
146
|
-
concepts.each do |concept|
|
|
147
|
-
termid = concept["termid"]
|
|
148
|
-
mc = V1::Concept.of_yaml(concept).to_managed_concept
|
|
149
|
-
File.write(File.join(concepts_out, "#{termid}.yaml"),
|
|
150
|
-
mc.to_yaml)
|
|
151
|
-
end
|
|
152
|
-
|
|
153
|
-
if register_data
|
|
154
|
-
rd = RegisterData.of_yaml(register_data)
|
|
155
|
-
File.write(File.join(output_path, "register.yaml"),
|
|
156
|
-
rd.to_yaml)
|
|
157
|
-
end
|
|
158
|
-
end
|
|
159
|
-
end
|
|
45
|
+
cross_references: cross_references,
|
|
46
|
+
dry_run: dry_run,
|
|
47
|
+
).run
|
|
160
48
|
end
|
|
161
49
|
end
|
|
162
50
|
end
|
|
@@ -63,9 +63,11 @@ module Glossarist
|
|
|
63
63
|
Rdf::GlossDocument.to_turtle(doc)
|
|
64
64
|
end
|
|
65
65
|
|
|
66
|
-
def to_turtle(concepts_or_concept = nil
|
|
66
|
+
def to_turtle(concepts_or_concept = nil, figures: [], tables: [],
|
|
67
|
+
formulas: [])
|
|
67
68
|
if concepts_or_concept.is_a?(Array)
|
|
68
|
-
build_document(concepts_or_concept
|
|
69
|
+
build_document(concepts_or_concept, figures: figures,
|
|
70
|
+
tables: tables, formulas: formulas)
|
|
69
71
|
else
|
|
70
72
|
target = concepts_or_concept || @concept
|
|
71
73
|
return "" unless target
|
|
@@ -75,12 +77,18 @@ module Glossarist
|
|
|
75
77
|
end
|
|
76
78
|
end
|
|
77
79
|
|
|
78
|
-
def to_jsonld(concepts_or_concept = nil
|
|
80
|
+
def to_jsonld(concepts_or_concept = nil, figures: [], tables: [],
|
|
81
|
+
formulas: [])
|
|
79
82
|
if concepts_or_concept.is_a?(Array)
|
|
80
83
|
gloss_concepts = concepts_or_concept.map do |c|
|
|
81
84
|
build_gloss_concept(c)
|
|
82
85
|
end
|
|
83
|
-
doc = Rdf::GlossDocument.new(
|
|
86
|
+
doc = Rdf::GlossDocument.new(
|
|
87
|
+
concepts: gloss_concepts,
|
|
88
|
+
figures: figures.map { |f| build_gloss_figure(f) },
|
|
89
|
+
tables: tables.map { |t| build_gloss_table(t) },
|
|
90
|
+
formulas: formulas.map { |f| build_gloss_formula(f) },
|
|
91
|
+
)
|
|
84
92
|
Rdf::GlossDocument.to_jsonld(doc)
|
|
85
93
|
else
|
|
86
94
|
target = concepts_or_concept || @concept
|
|
@@ -379,6 +387,18 @@ desig_index)
|
|
|
379
387
|
image: figure.images.first&.src,
|
|
380
388
|
caption: localized_pick(figure.caption),
|
|
381
389
|
description: localized_pick(figure.description),
|
|
390
|
+
images: Array(figure.images).map { |img| build_gloss_figure_image(img) },
|
|
391
|
+
)
|
|
392
|
+
end
|
|
393
|
+
|
|
394
|
+
def build_gloss_figure_image(image)
|
|
395
|
+
Rdf::GlossFigureImage.new(
|
|
396
|
+
src: image.src,
|
|
397
|
+
format: image.format,
|
|
398
|
+
role: image.role,
|
|
399
|
+
# FigureImage doesn't carry byte_size — emit nil so the RDF layer
|
|
400
|
+
# skips dcat:byteSize rather than emitting a spurious "0".
|
|
401
|
+
byte_size: nil,
|
|
382
402
|
)
|
|
383
403
|
end
|
|
384
404
|
|
data/lib/glossarist/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: glossarist
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 2.
|
|
4
|
+
version: 2.11.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ribose
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-07-
|
|
11
|
+
date: 2026-07-07 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: lutaml-model
|
|
@@ -304,6 +304,7 @@ files:
|
|
|
304
304
|
- lib/glossarist/rdf/gloss_designation.rb
|
|
305
305
|
- lib/glossarist/rdf/gloss_detailed_definition.rb
|
|
306
306
|
- lib/glossarist/rdf/gloss_figure.rb
|
|
307
|
+
- lib/glossarist/rdf/gloss_figure_image.rb
|
|
307
308
|
- lib/glossarist/rdf/gloss_formula.rb
|
|
308
309
|
- lib/glossarist/rdf/gloss_grammar_info.rb
|
|
309
310
|
- lib/glossarist/rdf/gloss_locality.rb
|
|
@@ -340,6 +341,7 @@ files:
|
|
|
340
341
|
- lib/glossarist/resolution_adapter/remote.rb
|
|
341
342
|
- lib/glossarist/resolution_adapter/route.rb
|
|
342
343
|
- lib/glossarist/schema_migration.rb
|
|
344
|
+
- lib/glossarist/schema_migration/cli_pipeline.rb
|
|
343
345
|
- lib/glossarist/schema_migration/v0_to_v1.rb
|
|
344
346
|
- lib/glossarist/schema_migration/v2_to_v3.rb
|
|
345
347
|
- lib/glossarist/section.rb
|