glossarist 2.5.0 → 2.5.2

data/README.adoc

@@ -344,30 +344,406 @@ modification:: A description of the modification to the cited definition of the
 
 == Commands
 
-`generate_latex`:: Convert Concepts to Latex format
+=== generate_latex
+
+Convert Concepts to Latex format.
 
-=== Usage:
 [,bash]
 ----
-glossarist generate_latex p, --concepts-path=CONCEPTS_PATH
+glossarist generate_latex -p PATH_TO_CONCEPTS
 ----
 
-=== Options:
+Options:
 [cols="1,1"]
 |===
 |p, --concepts-path
 |Path to yaml concepts directory
 
 |l, --latex-concepts
-|File path having list of concepts that should be converted to LATEX format. If not provided all the concepts will be converted to the latex format
+|File path having list of concepts that should be converted to LATEX format
 
 |o, --output-file
-|Output file path. By default the output will pe printed to the console
+|Output file path
 
 |e, --extra-attributes
-|List of extra attributes that are not in standard Glossarist Concept model. eg -e one two three
+|List of extra attributes that are not in standard Glossarist Concept model
+|===
+
+=== package
+
+Create a `.gcr` ZIP archive from a concept dataset.
+
+[,bash]
+----
+glossarist package DIR -o output.gcr --shortname mydataset --version 1.0.0 --uri-prefix urn:iso:std:iso:19111
+----
+
+Options:
+[cols="1,1"]
+|===
+|o, --output (required)
+|Output `.gcr` file path
+
+|--shortname (required)
+|Machine-readable dataset shortname (e.g. `iev`, `iso19111`)
+
+|--version (required)
+|Semantic version (e.g. `1.0.0`)
+
+|--title
+|Human-readable dataset title
+
+|--description
+|Dataset description
+
+|--owner
+|Dataset owner
+
+|--register-yaml
+|Path to register.yaml to include in package
+
+|--uri-prefix
+|URI namespace this dataset provides (e.g. `urn:iec:std:iec:60050`)
+
+|--tags
+|Tags for the dataset
+|===
+
+Ruby API:
+[,ruby]
+----
+GcrPackage.create_from_directory(
+  "path/to/dataset",
+  output: "output.gcr",
+  shortname: "mydataset",
+  version: "1.0.0",
+  uri_prefix: "urn:iso:std:iso:19111",
+)
+----
+
+=== validate
+
+Validate a dataset directory or `.gcr` file for schema compliance.
+
+[,bash]
+----
+glossarist validate PATH
+glossarist validate PATH --reference-path path/to/gcrs/
+----
+
+Options:
+[cols="1,1"]
+|===
+|--strict
+|Treat warnings as errors
+
+|--format
+|Output format: `text`, `json`, or `yaml`
+
+|--reference-path
+|Path to directory of `.gcr` files for cross-dataset reference validation
+|===
+
+Ruby API:
+[,ruby]
+----
+result = DatasetValidator.new.validate("path/to/dataset")
+result = DatasetValidator.new.validate("path/to/dataset", reference_path: "gcrs/")
+result.valid?   # => true/false
+result.errors   # => [...]
+result.warnings # => [...]
+----
+
+=== upgrade
+
+Upgrade a dataset to the current schema version.
+
+[,bash]
+----
+glossarist upgrade SOURCE_DIR -o OUTPUT_DIR
+----
+
+== Glossarist Concept Repository (GCR)
+
+A **GCR** (Glossarist Concept Repository) is a distributable, versioned ZIP archive containing glossary concepts and metadata. GCR packages are created from v2 datasets.
+
+=== GCR Package Format
+
+A `.gcr` file is a ZIP archive with the following structure:
+
+----
+metadata.yaml          # Package metadata
+register.yaml          # Optional register information
+concepts/              # Concept YAML files
+  102-01-01.yaml
+  200.yaml
+----
+
+=== Creating a GCR Package
+
+CLI:
+[,bash]
+----
+glossarist package path/to/v2-dataset -o mydataset-1.0.0.gcr \
+  --shortname mydataset --version 1.0.0 --uri-prefix urn:iso:std:iso:19111
+----
+
+Ruby API:
+[,ruby]
+----
+GcrPackage.create_from_directory(
+  "path/to/v2-dataset",
+  output: "mydataset-1.0.0.gcr",
+  shortname: "mydataset",
+  version: "1.0.0",
+  uri_prefix: "urn:iso:std:iso:19111",
+  title: "My Dataset",
+  description: "A terminology dataset",
+)
+----
+
+=== Loading a GCR Package
+
+[,ruby]
+----
+pkg = GcrPackage.load("mydataset-1.0.0.gcr")
+pkg.metadata     # => Hash with metadata fields
+pkg.concepts     # => Array of concept hashes
+----
+
+=== GCR Metadata
+
+Metadata fields in `metadata.yaml`:
+
+[cols="1,1"]
+|===
+|shortname
+|Machine-readable dataset identifier (e.g. `iev`)
+
+|version
+|Semantic version (e.g. `1.0.0`)
+
+|title
+|Human-readable title
+
+|description
+|Dataset description
+
+|owner
+|Dataset owner
+
+|tags
+|Array of tags
+
+|concept_count
+|Number of concepts in the package
+
+|languages
+|Array of language codes present
+
+|created_at
+|ISO 8601 timestamp of package creation
+
+|glossarist_version
+|Version of the Glossarist gem used
+
+|schema_version
+|Schema version of the package format
+
+|uri_prefix
+|URI namespace this dataset provides (e.g. `urn:iec:std:iec:60050`)
+
+|external_references
+|Array of `{uri: "..."}` for URI namespaces this dataset references
+|===
+
+=== GCR Statistics
+
+[,ruby]
+----
+stats = GcrStatistics.from_concepts(concepts)
+stats.total_concepts           # => 150
+stats.languages                # => ["eng", "fra", "deu"]
+stats.concepts_by_status       # => { "valid" => 140, "draft" => 10 }
+stats.concepts_with_definitions # => 148
+stats.concepts_with_sources    # => 130
+----
+
+== Concept Mentions
+
+Concepts can reference other concepts within the same dataset (intra-set) or in different datasets (inter-set) using inline mention syntax. All mentions use double braces `{{...}}`.
+
+=== Syntax
+
+The concept mention syntax mirrors HTML `<a href="id">display_text</a>` — the display text is independent of the target concept's canonical designation.
+
+[cols="1,2,2,2"]
+|===
+|Form |Syntax |Example |Resolution
+
+|ID only
+|`{{ID}}`
+|`{{200}}`
+|Intra-set: concept 200, auto-display
+
+|ID + display
+|`{{TEXT, ID}}`
+|`{{geodetic latitude, 200}}`
+|Intra-set: concept 200, custom display
+
+|Designation
+|`{{TEXT}}`
+|`{{geodetic latitude}}`
+|Intra-set: find by designation
+
+|URN + display
+|`{{TEXT, URN}}`
+|`{{equality, urn:iec:std:iec:60050-102-01-01}}`
+|Inter-set: resolve by URN
+
+|URN only
+|`{{URN}}`
+|`{{urn:iec:std:iec:60050-102-01-01}}`
+|Inter-set: resolve URN, auto-display
 |===
 
+=== URN Schemes
+
+IEC URN (IEV):: `urn:iec:std:iec:60050-{code}` — source is `urn:iec:std:iec:60050`, concept_id is the IEV code
+ISO URN (RFC 5141):: `urn:iso:std:iso:{std}:...:term:{id}` — source is `urn:iso:std:iso:{std}`, concept_id is the term ID
+
+=== Extracting Mentions (Ruby API)
+
+[,ruby]
+----
+extractor = ReferenceExtractor.new
+
+# From a text string
+refs = extractor.extract_from_text("See {{equality, urn:iec:std:iec:60050-102-01-01}} and {{lat, 200}}")
+# => [ConceptReference(term: "equality", concept_id: "102-01-01",
+#                      source: "urn:iec:std:iec:60050", ref_type: "urn"),
+#     ConceptReference(term: "lat", concept_id: "200",
+#                      source: nil, ref_type: "local")]
+
+# From all text fields in a localized concept
+refs = extractor.extract_from_localized(lc_hash)
+
+# From all language blocks in a concept
+refs = extractor.extract_from_concept_hash(concept_hash)
+----
+
+=== Resolving Mentions (Ruby API)
+
+Resolution uses an adapter chain: route overrides → local → package → remote.
+
+[,ruby]
+----
+resolver = ReferenceResolver.new
+
+# Register the current dataset for intra-set resolution
+resolver.register_self(concepts)
+
+# Register co-loaded GCRs with their URI prefixes
+resolver.register_package(iev_concepts, uri_prefix: "urn:iec:std:iec:60050")
+resolver.register_package(iso_concepts, uri_prefix: "urn:iso:std:iso:19111")
+
+# Add URI route overrides (e.g. author used wrong URI)
+resolver.add_route(from: "urn:iso:std:iso:19115", to: "urn:iso:std:iso:19111")
+
+# Resolve a single reference
+ref = ConceptReference.new(term: "equality", concept_id: "102-01-01",
+                           source: "urn:iec:std:iec:60050", ref_type: "urn")
+resolver.resolve(ref)  # => concept hash
+
+# Validate all references in a package
+result = resolver.validate_all(concepts)
+result.errors    # => structural errors
+result.warnings  # => unresolvable references
+----
+
+=== GCR Collection & Routing
+
+When multiple GCRs are placed together in a directory, a `collection.yaml` configures resolution:
+
+[,yaml]
+----
+# collection.yaml
+packages:
+  - file: iev-2.0.0.gcr
+  - file: iso19111-1.0.0.gcr
+
+routes:
+  - from: "urn:iso:std:iso:19115"
+    to: "urn:iso:std:iso:19111"
+
+remote:
+  - uri_prefix: "urn:iec:std:iec:60050"
+    endpoint: "https://vocabulary.example.org/api/concepts"
+----
+
+[,ruby]
+----
+resolver = ReferenceResolver.new
+resolver.load_collection("path/to/gcr_collection/")
+# Packages auto-registered with their uri_prefix from metadata
+# Route overrides applied
+# Remote endpoints registered
+----
+
+=== Resolution Adapters
+
+The resolution framework uses a chain of adapters, each implementing `resolve(reference) → concept_hash | nil`:
+
+LocalAdapter:: Resolves intra-set references by concept ID or designation lookup
+PackageAdapter:: Resolves inter-set references by matching `source` URI to a GCR's `uri_prefix`
+RouteAdapter:: Remaps incorrect source URIs before delegation
+RemoteAdapter:: Resolves via HTTP to an online GCR endpoint
+
+=== URN-to-HTTP Resolution
+
+Concept mentions rendered as hyperlinks need HTTP URLs. The `UrnResolver` converts URNs to their canonical web locations:
+
+[,ruby]
+----
+# Class-level convenience
+url = UrnResolver.resolve("urn:iec:std:iec:60050-102-01-01")
+# => "https://www.electropedia.org/iev/iev.nsf/display?openform&ievref=102-01-01"
+
+url = UrnResolver.resolve("urn:iso:std:iso:19111:ed-3:v1:en:term:3.1.32")
+# => "https://www.iso.org/obp/ui/#iso:std:iso:19111:ed-3:v1:en:term:3.1.32"
+
+# Also accepts ConceptReference objects
+ref = ConceptReference.new(term: "equality", concept_id: "102-01-01",
+                           source: "urn:iec:std:iec:60050", ref_type: "urn")
+url = UrnResolver.resolve(ref)
+# => "https://www.electropedia.org/iev/iev.nsf/display?openform&ievref=102-01-01"
+----
+
+Built-in mappings:
+
+[cols="1,1,1"]
+|===
+|URN Prefix |Target |Example URL
+
+|`urn:iec:std:iec:60050-*`
+|IEC Electropedia
+|`electropedia.org/iev/iev.nsf/display?openform&ievref=102-01-01`
+
+|`urn:iso:*`
+|ISO Online Browsing Platform
+|`iso.org/obp/ui/#iso:std:iso:19111:term:3.1.32`
+|===
+
+Register custom schemes:
+
+[,ruby]
+----
+resolver = UrnResolver.new
+resolver.register_scheme("urn:example:") do |urn|
+  "https://example.org/concepts/#{urn.sub('urn:example:', '')}"
+end
+----
+
 == Credits
 
 This gem is developed, maintained and funded by

data/TODO.integration/01-gcr-package-cli.md

@@ -0,0 +1,180 @@
+# 01 — GCR Packaging CLI with Versioning
+
+## Goal
+
+The `glossarist` Ruby gem provides the canonical way to build versioned GCR packages from concept datasets. Each glossary repo uses `glossarist package` in CI to publish GCR releases.
+
+## Current State
+
+- CLI has `package`, `upgrade`, `validate` commands (via Thor)
+- `GcrPackage.create` / `GcrPackage.load` handle ZIP I/O
+- `GcrMetadata` generates metadata with statistics
+- `SchemaMigration` handles v0→v1 harmonization
+- Missing: `shortname` and `version` fields in GcrMetadata
+- Missing: v2 format support in `PackageCommand`
+
+## Status
+
+Tasks 1–5 are **implemented** on branch `fix/gemfile-lutaml-model-substring`. Remaining: task 6 (publish gem to RubyGems).
+
+## Tasks
+
+### 1. Add `shortname` and `version` to `GcrMetadata`
+
+Edit `lib/glossarist/gcr_metadata.rb`:
+
+```ruby
+attr_accessor :shortname, :version, :title, :description, :owner, :tags,
+              :concept_count, :languages,
+              :created_at, :glossarist_version, :schema_version,
+              :statistics, :homepage, :repository, :license
+
+def initialize(attrs = {})
+  @shortname = attrs[:shortname]
+  @version = attrs[:version]
+  # ... existing fields ...
+end
+
+def self.from_concepts(concepts, register_data: nil, options: {})
+  stats = GcrStatistics.from_concepts(concepts)
+  new(
+    shortname: options[:shortname],
+    version: options[:version],
+    title: options[:title] || register_data&.dig("name"),
+    # ... existing fields ...
+  )
+end
+
+def to_h
+  h = {
+    "shortname" => shortname,
+    "version" => version,
+    "title" => title,
+    # ... existing fields ...
+  }
+  h.compact
+end
+```
+
+### 2. Add `--shortname` and `--version` CLI options
+
+Edit `lib/glossarist/cli.rb`:
+
+```ruby
+desc "package DIR", "Create a .gcr ZIP archive from a dataset"
+option :output, aliases: :o, required: true, desc: "Output .gcr file path"
+option :shortname, type: :string, required: true, desc: "Machine-readable dataset ID"
+option :version, type: :string, required: true, desc: "Semantic version (e.g. 1.0.0)"
+option :title, type: :string, desc: "Dataset title"
+option :description, type: :string, desc: "Dataset description"
+option :owner, type: :string, desc: "Dataset owner"
+option :register_yaml, type: :string, desc: "Path to register.yaml"
+option :tags, type: :array, desc: "Tags for the dataset"
+def package(dir)
+  # ...
+end
+```
+
+### 3. Add v2 format support to `PackageCommand`
+
+Edit `lib/glossarist/cli/package_command.rb`:
+
+```ruby
+def collect_concepts
+  if v1_concepts?
+    collect_v1_concepts
+  elsif v2_concepts?
+    collect_v2_concepts
+  else
+    []
+  end
+end
+
+def v1_concepts?
+  concepts_dir = File.join(@dir, "concepts")
+  File.directory?(concepts_dir) && Dir.glob(File.join(concepts_dir, "*.yaml")).any?
+end
+
+def v2_concepts?
+  File.directory?(File.join(@dir, "geolexica-v2"))
+end
+
+def collect_v2_concepts
+  collection = Glossarist::ManagedConceptCollection.new
+  manager = Glossarist::ConceptManager.new(path: File.join(@dir, "geolexica-v2"))
+  manager.load_from_files(collection: collection)
+
+  collection.map { |concept| concept_to_v1_hash(concept) }
+end
+
+def concept_to_v1_hash(concept)
+  hash = { "termid" => concept.data.id.to_s }
+  concept.localizations.each do |lang, l10n|
+    hash[lang] = localized_to_hash(l10n)
+  end
+  hash["term"] = preferred_designation(hash["eng"]&.dig("terms")) || ""
+  hash
+end
+
+def localized_to_hash(l10n)
+  h = {}
+  h["terms"] = l10n.designations.map(&:to_h) if l10n.designations.any?
+  h["definition"] = l10n.definition.map { |d| { "content" => d.content } } if l10n.definition.any?
+  h["notes"] = l10n.notes.map { |n| { "content" => n.content } } if l10n.notes.any?
+  h["examples"] = l10n.examples.map { |e| { "content" => e.content } } if l10n.examples.any?
+  h["sources"] = l10n.sources.map(&:to_h) if l10n.sources.any?
+  h["language_code"] = l10n.language_code if l10n.language_code
+  h["entry_status"] = l10n.entry_status if l10n.entry_status
+  h["dates"] = l10n.dates.map(&:to_h) if l10n.dates.any?
+  h
+end
+```
+
+### 4. Auto-derive shortname from directory name
+
+If `--shortname` is not provided, derive from:
+1. `register.yaml` → `register["shortname"]` or `register["id"]`
+2. Directory basename
+3. Raise error if none available
+
+### 5. Validate filename matches metadata
+
+In `GcrPackage.validate`, check that the filename pattern `{shortname}-{version}.gcr` matches the `shortname` and `version` in metadata.yaml.
+
+### 6. Publish gem
+
+```bash
+gem build glossarist.gemspec
+gem push glossarist-2.6.0.gem
+```
+
+## CLI Usage
+
+```bash
+# Install
+gem install glossarist
+
+# Package (v1 format)
+glossarist package ./isotc204-glossary \
+  --shortname isotc204 --version 1.0.0 \
+  -o isotc204-1.0.0.gcr \
+  --title "ISO/TC 204 ITS Vocabulary" --owner "ISO/TC 204"
+
+# Package (v2 format, auto-detected)
+glossarist package ./isotc211-glossary \
+  --shortname isotc211 --version 2.3.0 \
+  -o isotc211-2.3.0.gcr \
+  --title "ISO/TC 211 Multi-Lingual Glossary" --owner "ISO/TC 211"
+
+# Validate
+glossarist validate isotc204-1.0.0.gcr
+```
+
+## Acceptance Criteria
+
+- [x] `GcrMetadata` includes `shortname` and `version` fields
+- [x] `glossarist package --shortname X --version Y` produces `{X}-{Y}.gcr`
+- [x] `metadata.yaml` contains `shortname` and `version`
+- [x] `glossarist validate` checks metadata has required fields
+- [x] Works with both v1 (`concepts/*.yaml`) and v2 (`geolexica-v2/*.yaml`) datasets
+- [ ] Gem published to RubyGems

data/exe/glossarist

@@ -1,56 +1,4 @@
 #!/usr/bin/env ruby
 
 require_relative "../lib/glossarist"
-
-class GlossaristCommand < Thor
-  desc "generate_latex", "Convert Concepts to Latex format"
-
-  option :concepts_path, aliases: :p, required: true,
-                         desc: "Path to yaml concepts directory"
-  option :latex_concepts, aliases: :l,
-                          desc: "File path having list of concepts that should be converted to LATEX format. If not provided all the concepts will be converted to the latex format"
-  option :output_file, aliases: :o,
-                       desc: "Output file path. By default the output will pe printed to the console"
-  option :extra_attributes, aliases: :e, type: :array,
-                            desc: "List of extra attributes that are not in standard Glossarist Concept model"
-  def generate_latex
-    assets = []
-    latex_concepts_file = options[:latex_concepts]
-
-    if options[:extra_attributes]
-      Glossarist.configure do |config|
-        config.register_extension_attributes(options[:extra_attributes])
-      end
-    end
-
-    concept_set = Glossarist::ConceptSet.new(options[:concepts_path], assets)
-    latex_str = concept_set.to_latex(latex_concepts_file)
-    output_latex(latex_str)
-  end
-
-  def output_latex(latex_str)
-    output_file_path = options[:output_file]
-
-    if output_file_path
-      File.open(output_file_path, "w") { |file| file.puts latex_str }
-    else
-      puts latex_str
-    end
-  end
-
-  def method_missing(*args)
-    warn "No method found named: #{args[0]}"
-    warn "Run with `--help` or `-h` to see available options"
-    exit 1
-  end
-
-  def respond_to_missing?
-    true
-  end
-
-  def self.exit_on_failure?
-    true
-  end
-end
-
-GlossaristCommand.start(ARGV)
+Glossarist::CLI.start(ARGV)

data/glossarist.gemspec

@@ -33,5 +33,6 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency "lutaml-model", "~> 0.8"
   spec.add_dependency "relaton", ">= 2.0.0", "< 3"
+  spec.add_dependency "rubyzip", ">= 2.3", "< 3"
   spec.add_dependency "thor"
 end

data/lib/glossarist/asset.rb

@@ -2,7 +2,7 @@ module Glossarist
   class Asset < Lutaml::Model::Serializable
     attribute :path, :string
 
-    yaml do
+    key_value do
       map :path, to: :path
     end
 

data/lib/glossarist/citation.rb

@@ -34,7 +34,7 @@ module Glossarist
 
     attribute :custom_locality, CustomLocality, collection: true
 
-    yaml do
+    key_value do
       map :id, to: :id, with: { from: :id_from_yaml, to: :id_to_yaml }
       map :text, to: :text, with: { from: :text_from_yaml, to: :text_to_yaml }
       map :source, to: :source,

data/lib/glossarist/cli/package_command.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Glossarist
+  class CLI
+    class PackageCommand
+      def initialize(dir, options)
+        @dir = dir
+        @options = options
+      end
+
+      def run
+        GcrPackage.create_from_directory(
+          @dir,
+          output: @options[:output],
+          shortname: @options[:shortname],
+          version: @options[:version],
+          title: @options[:title],
+          description: @options[:description],
+          owner: @options[:owner],
+          tags: @options[:tags],
+          register_yaml: @options[:register_yaml],
+          uri_prefix: @options[:uri_prefix],
+        )
+
+        puts "Created #{@options[:output]}"
+      rescue ArgumentError => e
+        warn "Error: #{e.message}"
+        exit 1
+      end
+    end
+  end
+end