glossarist 2.6.5 → 2.6.7

data/README.adoc

@@ -101,7 +101,7 @@ related:: Array of <<related-concept,RelatedConcept>>
 status:: Enum for the normative status of the term.
 dates:: Array of <<concept-date,ConceptDate>>
 localized_concepts:: Hash of all localizations where keys are language codes and values are uuid of the localized concept.
-groups:: Array of groups in string format
+domains:: Array of <<concept-reference,ConceptReference>> — upper concepts (subject areas, concept schemes, organizing concepts) that this concept belongs to across all languages. Each domain is a typed reference (e.g. `{ concept_id: "103", ref_type: "domain" }`).
 localizations:: Hash of all localizations for this concept where keys are language codes and values are instances of <<localized-concept,LocalizedConcept>>.
 
 There are two ways to initialize and populate a managed concept
@@ -118,9 +118,8 @@ concept = Glossarist::ManagedConcept.new({
       "eng" => "<uuid>"
     },
     "localizations" => <Array of localized concepts or localized concept hashes>,
-    "groups" => [
-      "foo",
-      "bar",
+    "domains" => [
+      { "concept_id" => "103", "ref_type" => "domain" },
     ],
   },
 })
@@ -132,7 +131,9 @@ concept = Glossarist::ManagedConcept.new({
 ----
 concept = Glossarist::ManagedConcept.new
 concept.id = "123"
-concept.groups = ["foo", "bar"]
+concept.data.domains = [
+  Glossarist::ConceptReference.new(concept_id: "103", ref_type: "domain"),
+]
 concept.localizations = <Array of localized concepts or localized concept hashes>
 ----
 
@@ -146,55 +147,252 @@ Localized concept has the following fields
 id:: An optional identifier for the term, to be used in cross-references.
 uuid:: UUID for the concept
 designations:: Array of <<designation,Designations>> under which the term being defined is known. This method will also accept an array of hashes for designation and will convert them to their respective classes.
-domain:: An optional semantic domain for the term being defined, in case the term is ambiguous between several semantic domains.
+domain:: URI reference to the subject area or section concept. Can be a relative URI (e.g. `section-103-01`), a URN (e.g. `urn:iec:std:iec:60050-103-01`), or a URL (e.g. `https://www.electropedia.org/iev/iev.nsf/display?openform&ievref=103-01`). This is the per-language upper concept reference — the subject area for this specific localization. Different languages may assign the same abstract concept to different domains.
+related:: Array of <<related-concept,RelatedConcept>> — per-language concept relationships. Concept hierarchies can differ across languages (e.g. Russian distinguishes голубой/siniy as coordinate basic colors, while English unifies them under "blue"). Language-specific broader/narrower/equivalent relationships go here.
 subject:: Subject of the term.
 definition:: Array of <<detailed-definition,Detailed Definition>> of the term.
 non_verb_rep:: Array of <<non-verbal,non-verbal>> representations used to help define the term.
 notes:: Zero or more notes about the term. A note is in <<detailed-definition,Detailed Definition>> format.
 examples:: Zero or more examples of how the term is to be used in <<detailed-definition,Detailed Definition>> format.
 language_code:: The language of the localization, as an ISO-639 3-letter code.
+script:: The script of the localization, as an ISO 15924 4-letter code (e.g. `Hans` for Simplified Chinese, `Latn` for Latin, `Cyrl` for Cyrillic). Optional — when omitted, the default script for the language is assumed.
+system:: The ISO 24229 conversion system code used to produce this localization (e.g. `Var:jpn-Hrkt:Latn:Hepburn-1886` for Hepburn-romanized Japanese). Optional — only set when the localization is a romanization or transliteration.
 entry_status:: Entry status of the concept. Must be one of the following: +notValid+, +valid+, +superseded+, +retired+.
 classification:: Classification of the concept. Must be one of the following: +preferred+, +admitted+, +deprecated+.
 
 [[id,designation]]
-=== Designation::Base
+=== Designation
+
+A name under which a managed term is known. Designations follow an
+inheritance hierarchy based on ISO 10241-1 and the Metanorma concept model.
+
+==== Designation::Base (common to all types)
+
+designation:: String — the term text or symbol.
+normative_status:: Enum — one of `preferred`, `admitted`, `deprecated`, `superseded`.
+geographical_area:: String — geographic usage region (ISO 3166-1 country code).
+language:: String — language of this designation (ISO 639 code). Usually inherited from the LocalizedConcept's `language_code`, but can differ for borrowed terms.
+script:: String — script of the designation text (ISO 15924 code, e.g. `Hani` for Kanji, `Latn` for Latin, `Cyrl` for Cyrillic).
+system:: String — ISO 24229 conversion system code used to produce this designation (e.g. `Var:jpn-Hrkt:Latn:Hepburn-1886` for Hepburn romanization). Optional — only set when the designation is a romanization or transliteration.
+international:: Boolean — whether the designation is used internationally.
+absent:: Boolean — whether the designation is intentionally absent in this language.
+pronunciation:: Collection of `Pronunciation` entries — phonetic or romanized representations of the designation.
+sources:: Collection of `ConceptSource` entries — bibliographic sources for this designation (ISO 10241-1 §6.8).
+term_type:: Enum (ISO 12620) — optional classification of the designation's term type. See <<iso12620-term-types>> below.
+related:: Collection of `RelatedConcept` entries — term-level (designation-to-designation) relationships within the same concept entry. Used for linking abbreviated forms to full forms, short forms to expanded forms, etc. (TBX xref types).
++
+Each `Pronunciation` entry has:
++
+[cols="1,1,2"]
+|===
+|Attribute |Standard |Description
+
+|`content` |— |The pronunciation text
+|`language` |ISO 639 |Language/dialect being pronounced (3-letter code)
+|`script` |ISO 15924 |Script of the pronunciation text (4-letter code)
+|`country` |ISO 3166-1 |Country variant (2-letter code, optional)
+|`system` |ISO 24229 |Conversion system code or identifier (e.g. `IPA`, `Var:jpn-Hrkt:Latn:Hepburn-1886`)
+|===
++
+Example:
++
+[,yaml]
+----
+pronunciation:
+  - content: "toːkjoː"
+    language: jpn
+    script: Latn
+    system: IPA
+  - content: "Tōkyō"
+    language: jpn
+    script: Latn
+    system: "Var:jpn-Hrkt:Latn:Hepburn-1886"
+----
+
+==== Designation::Expression (text-based, inherits Base)
+
+prefix:: String — text before the designation.
+usage_info:: String — disambiguation text for the designation.
+field_of_application:: String — IEC "specific use", appears in angle brackets after the designation (e.g. "in communication theory").
+grammar_info:: Array of GrammarInfo — gender, number, part of speech.
+
+==== Designation::Abbreviation (inherits Expression)
+
+acronym:: Boolean — is this an acronym?
+initialism:: Boolean — is this an initialism?
+truncation:: Boolean — is this a truncation?
+
+==== Designation::Symbol (inherits Base)
+
+No additional attributes beyond Base.
 
-A name under which a managed term is known.
+==== Designation::LetterSymbol (inherits Symbol)
 
-Methods::
-  `from_h(options)`::: Creates a new designation instance based on the specified type.
+text:: String — the letter symbol text.
+
+==== Designation::GraphicalSymbol (inherits Symbol)
+
+text:: String — description of the symbol.
+image:: String — the graphical symbol (emoji, path, or data URL).
+
+==== Factory Method
+
+`Designation::Base.from_h(options)` creates a new designation instance based on the specified type.
 
 Parameters::
   * options (Hash) - The options for creating the designation.
-  * "type" (String) - The type of designation (expression, symbol, abbreviation, graphical_symbol, letter_symbol). Note: type key should be string and not a symbol so { type: "expression" } will not work.
+  * "type" (String) - The type of designation (`expression`, `symbol`, `abbreviation`, `graphical_symbol`, `letter_symbol`). Note: type key should be string and not a symbol so `{ type: "expression" }` will not work.
   * Additional options depend on the specific designation type.
 
 Returns::
-  Designation::{type}::: A new instance of specified type. e.g `Glossarist::Designation::Base.from_h("type" => "expression")` will return `Glossarist::Designation::Expression`
+  Designation::{type}::: A new instance of specified type.
 
 Example
 [,ruby]
 ----
-# Example usage of Designation::Base class
+# Expression with field of application
+expr = Designation::Base.from_h({
+  "type" => "expression",
+  "designation" => "information",
+  "normative_status" => "preferred",
+  "field_of_application" => "in communication theory",
+})
+
+# International abbreviation
+abbr = Designation::Base.from_h({
+  "type" => "abbreviation",
+  "designation" => "ISO",
+  "international" => true,
+  "acronym" => true,
+})
+----
+
+[[iso12620-term-types]]
+==== ISO 12620 Term Types
+
+The `term_type` attribute on `Designation::Base` classifies designations
+according to ISO 12620 (also used as TBX `termType`). This is orthogonal to
+the structural designation `type` (expression/abbreviation/symbol): the
+structural type determines how the designation is serialized, while
+`term_type` provides ISO 12620 semantic classification.
+
+[cols="1,2"]
+|===
+|Term type |Description
+
+|`abbreviation`
+|A shortened form of a word or phrase (general category)
+
+|`acronym`
+|An abbreviation pronounced as a word (e.g. NATO, laser)
 
-attributes_for_expression = { designation: "foobar", geographical_area: "abc", normative_status: "status" }
-designation_expression = Designation::Base.from_h({ "type" => "expression" }.merge(attributes_for_expression))
+|`clipped_term`
+|A term formed by clipping part of a longer term (e.g. "phone" from "telephone")
 
-attributes_for_abbreviation = { designation: "foobar", geographical_area: "abc", normative_status: "status", international: true }
-designation_abbreviation = Designation::Base.from_h({ "type" => "abbreviation" }.merge(attributes_for_abbreviation))
+|`common_name`
+|A name in common use for a concept (e.g. "water" vs H₂O)
 
+|`entry_term`
+|The headword or main term in a terminological entry
+
+|`equation`
+|A mathematical equation used as a designation
+
+|`formula`
+|A chemical or mathematical formula (e.g. H₂O, E=mc²)
+
+|`full_form`
+|The complete, unabbreviated form of a designation (e.g. "World Wide Web")
+
+|`initialism`
+|An abbreviation pronounced letter by letter (e.g. "URL", "FBI")
+
+|`internationalism`
+|A term used with the same meaning across many languages (e.g. "computer", "algorithm")
+
+|`international_scientific_term`
+|A term established by international scientific agreement (e.g. "hydrogen")
+
+|`logical_expression`
+|A logical or Boolean expression used as a designation
+
+|`part_number`
+|A part number or catalog identifier used as a designation
+
+|`phraseological_unit`
+|A multi-word expression or phrase functioning as a term (e.g. "software engineering")
+
+|`transcribed_form`
+|A designation produced by phonetic transcription from another script
+
+|`transliterated_form`
+|A designation produced by transliteration from another script (e.g. "Moskva" from "Москва")
+
+|`short_form`
+|A shortened form of a designation that is not an abbreviation (e.g. "US" for "United States")
+
+|`shortcut`
+|A keyboard shortcut or command sequence (e.g. "Ctrl+V" for paste)
+
+|`sku`
+|A stock keeping unit identifier
+
+|`standard_text`
+|A standardized text passage used as a designation
+
+|`symbol`
+|A non-verbal symbol representing the concept (e.g. Ω for ohm)
+
+|`synonym`
+|A term with the same meaning in the same language, used as an alternative designation
+
+|`synonymous_phrase`
+|A phrase that is synonymous with the preferred designation
+
+|`variant`
+|A spelling, regional, or stylistic variant of another designation
+|===
+
+==== Designation-Level Relationships (TBX xref)
+
+Designations can have intra-entry relationships — links between
+designations of the *same* concept. These correspond to TBX `xref`
+elements on term information groups (`<tig>`).
+
+[cols="1,2"]
+|===
+|Relationship type |Description
+
+|`abbreviated_form_for`
+|This designation is an abbreviated form of the target (e.g. "WWW" → "World Wide Web")
+
+|`short_form_for`
+|This designation is a short form of the target (e.g. "US" → "United States of America")
+|===
+
+Example:
+[,yaml]
+----
+terms:
+  - designation: WWW
+    type: abbreviation
+    term_type: acronym
+    related:
+      - type: abbreviated_form_for
+        content: "World Wide Web"
+  - designation: World Wide Web
+    type: expression
+    term_type: full_form
 ----
 
 [[id,related-concept]]
 === RelatedConcept
 
-A term related to the current term.
-
-Following fields are available for the Related Concept
+A concept related to the current concept with a typed relationship.
 
-type:: An enum to denote the relation of the term to the current term.
-content:: The designation of the related term.
-ref:: A <<citation, citation>> of the related term, in a Termbase.
+type:: Enum — the relationship type (see <<relationship-types,Relationship Types>> below).
+content:: String — free-text content describing the related concept.
+ref:: A <<concept-ref,ConceptRef>> reference to the related concept (`source + id`, no version or locality).
 
 There are two ways to initialize and populate a related concept
 
@@ -205,7 +403,7 @@ There are two ways to initialize and populate a related concept
 related_concept = Glossarist::RelatedConcept.new({
   content: "Test content",
   type: :supersedes,
-  ref: <concept citation>
+  ref: Glossarist::ConceptRef.new(source: "IEC", id: "102-03-02"),
 })
 ----
 
@@ -216,7 +414,209 @@ related_concept = Glossarist::RelatedConcept.new({
 related_concept = Glossarist::RelatedConcept.new
 related_concept.type = "supersedes"
 related_concept.content = "designation of the related concept"
-related_concept.ref = <Citation object>
+related_concept.ref = Glossarist::ConceptRef.new(source: "IEC", id: "102-03-02")
+----
+
+[[relationship-types]]
+==== Relationship Types
+
+Relationship types are drawn from ISO 10241-1, ISO 25964/SKOS, and ISO 12620/TBX. The table below shows each type with its provenance and cross-standard equivalents.
+
+[cols="1,1,1,1,1"]
+|===
+|Glossarist type |Category |ISO 10241-1 |ISO 25964 / SKOS |ISO 12620 / TBX
+
+|`deprecates`
+|Lifecycle
+|deprecates
+|—
+|—
+
+|`supersedes`
+|Lifecycle
+|supersedes
+|—
+|—
+
+|`superseded_by`
+|Lifecycle
+|superseded by
+|—
+|—
+
+|`broader`
+|Hierarchical
+|broader concept
+|BT (broaderTerm)
+|broaderTerm
+
+|`narrower`
+|Hierarchical
+|narrower concept
+|NT (narrowerTerm)
+|narrowerTerm
+
+|`broader_generic`
+|Hierarchical (generic)
+|—
+|BTG (broaderGeneric, is-a)
+|broaderTermGeneric
+
+|`narrower_generic`
+|Hierarchical (generic)
+|—
+|NTG (narrowerGeneric)
+|narrowerTermGeneric
+
+|`broader_partitive`
+|Hierarchical (partitive)
+|—
+|BTP (broaderPartitive, part-whole)
+|broaderTermPartitive
+
+|`narrower_partitive`
+|Hierarchical (partitive)
+|—
+|NTP (narrowerPartitive)
+|narrowerTermPartitive
+
+|`broader_instantial`
+|Hierarchical (instantial)
+|—
+|BTI (broaderInstantial, instance-of)
+|broaderTermInstantial
+
+|`narrower_instantial`
+|Hierarchical (instantial)
+|—
+|NTI (narrowerInstantial)
+|narrowerTermInstantial
+
+|`equivalent`
+|Equivalence
+|equivalent
+|exactMatch
+|—
+
+|`close_match`
+|Approx. equiv.
+|—
+|closeMatch
+|—
+
+|`broad_match`
+|Cross-vocab mapping
+|—
+|broadMatch
+|—
+
+|`narrow_match`
+|Cross-vocab mapping
+|—
+|narrowMatch
+|—
+
+|`related_match`
+|Cross-vocab mapping
+|—
+|relatedMatch
+|—
+
+|`compare`
+|Comparative
+|compare
+|—
+|—
+
+|`contrast`
+|Comparative
+|contrast
+|—
+|—
+
+|`see`
+|Associative
+|see also
+|RT (relatedTerm)
+|crossReference
+
+|`related_concept`
+|Associative
+|—
+|—
+|relatedConcept
+
+|`related_concept_broader`
+|Associative (broader)
+|—
+|—
+|relatedConceptBroader
+
+|`related_concept_narrower`
+|Associative (narrower)
+|—
+|—
+|relatedConceptNarrower
+
+|`sequentially_related_concept`
+|Associative (sequential)
+|—
+|—
+|sequentiallyRelatedConcept
+
+|`spatially_related_concept`
+|Associative (spatial)
+|—
+|—
+|spatiallyRelatedConcept
+
+|`temporally_related_concept`
+|Associative (temporal)
+|—
+|—
+|temporallyRelatedConcept
+
+|`homograph`
+|Lexical
+|—
+|—
+|homograph
+
+|`false_friend`
+|Lexical
+|—
+|—
+|falseFriend
+|===
+
+[[id,concept-reference]]
+=== ConceptReference
+
+A typed reference to another concept, either local (within the same glossary) or external (in another concept registry).
+
+term:: String — the display text for the referenced concept.
+concept_id:: String — the identifier of the target concept.
+source:: String — the registry URI prefix for external references (e.g. `urn:iec:std:iec:60050`).
+ref_type:: String — the reference type: `local`, `designation`, or `urn`.
+urn:: String — a direct URN for the target concept (e.g. `urn:iec:std:iec:60050-102-01-01`).
+
+Local references use `concept_id` without `source`. External references use `source` + `concept_id` or a direct `urn`.
+
+[,ruby]
+----
+# Local reference
+ref = Glossarist::ConceptReference.new(term: "latitude", concept_id: "200", ref_type: "local")
+
+# External reference via URN
+ref = Glossarist::ConceptReference.new(
+  term: "equality",
+  concept_id: "102-01-01",
+  source: "urn:iec:std:iec:60050",
+  ref_type: "urn",
+)
+
+ref.local?    # => false
+ref.external? # => true
 ----
 
 [[id,concept-date]]
@@ -258,7 +658,7 @@ A definition of the managed term.
 It has the following attributes:
 
 content:: The text of the definition of the managed term.
-sources:: List of Bibliographic references(<<citation,Citation>>) for this particular definition of the managed term.
+sources:: List of <<concept-source,bibliographic sources>> for this particular definition of the managed term.
 
 There are two ways to initialize and populate a detailed definition
 
@@ -284,52 +684,119 @@ detailed_definition.sources = [<list of citations>]
 [[id,citation]]
 === Citation
 
-Citation can be either structured or unstructured. A citation is structured if its reference contains one or all of the following keys `{ id: "id", source: "source", version: "version"}` and is unstructured if its reference is plain text. This also has 2 methods `structured?` and `plain?` to check if citation is structured or not.
+A bibliographic citation used in <<concept-source,ConceptSource>> entries. The
+`ref` field is always a structured hash (`Citation::Ref`), never a plain string.
 
 Citation has the following attributes.
 
-ref:: A hash or string based on type of citation. Hash if citation is structured or string if citation is plain.
-clause:: Referred clause of the document.
-link:: Link to document.
+ref:: A `Citation::Ref` object (always a hash). See below.
+locality:: A `Locality` with `type`, `reference_from`, and `reference_to`.
+link:: URL link to the source document.
+original:: Pre-parsed original citation text.
+custom_locality:: Array of `{ name:, value: }` pairs for non-standard locality types.
 
-There are two ways to initialize and populate a Citation
+`Citation::Ref` inner class:
+
+source:: String — the document series or termbase (e.g. `"ISO"`, `"IEC"`).
+id:: String — the document identifier within the series (e.g. `"1087-1:2000"`).
+version:: String — optional version of the document.
+
+`Citation#label` returns a display string combining source and id.
 
-1. Setting the fields by using a hash while initializing
-+
 [,ruby]
 ----
-# Unstructured Citation
-citation = Glossarist::Citation.new({
-  ref: "plain text reference",
-  clause: "clause",
-  link: "link",
-})
+# Create a Citation
+citation = Glossarist::Citation.new(
+  ref: Glossarist::Citation::Ref.new(source: "ISO", id: "9000"),
+  locality: Glossarist::Locality.new(type: "clause", reference_from: "3.3.3"),
+  link: "https://www.iso.org/standard/45481.html",
+)
 
-# Structured Citation
-citation = Glossarist::Citation.new({
-  ref: { id: "123", source: "source", version: "1.1" },
-  clause: "clause",
-  link: "link",
-})
+citation.label  # => "ISO 9000"
 ----
 
-2. Setting the fields after creating an object
-+
+YAML serialization (always this shape):
+
+[,yaml]
+----
+origin:
+  ref:
+    source: ISO
+    id: "1087-1:2000"
+  locality:
+    type: clause
+    reference_from: "3.2.9"
+  link: https://www.iso.org/standard/20057.html
+----
+
+=== ConceptRef
+
+A concept reference used in <<related-concept,RelatedConcept>> entries. Has
+`source` and `id` only — no version, locality, or link.
+
+[,ruby]
+----
+ref = Glossarist::ConceptRef.new(source: "IEC", id: "102-01-01")
+----
+
+=== V2 and V3 Namespaces
+
+Glossarist provides versioned model namespaces for forward and backward
+compatibility:
+
+* `Glossarist::V2::*` — loads V2 datasets where `ref` may be a plain string.
+  Maps string refs to a `text` attribute internally.
+* `Glossarist::V3::*` — canonical V3 models with structured `Citation::Ref`
+  hashes, `BibliographyEntry`, `BibliographyFile`, `ImageEntry`, `ImageFile`.
+
+The `ConceptDocument.for_version("2"|"3")` method dispatches to the correct
+namespace. `ConceptCollector.detect_schema_version(dir)` auto-detects from the
+data by checking `schema_version` in the first concept file.
+
+=== Concept Comparison
+
+Compare two concept datasets to identify additions, removals, and changes:
+
+[,bash]
+----
+glossarist compare path/to/v1 path/to/v2
+----
+
+Ruby API:
+
 [,ruby]
 ----
-citation = Glossarist::Citation.new
-citation.ref = <plain or structured ref>
-citation.clause = "some clause"
+comparator = Glossarist::ConceptComparator.new
+result = comparator.compare(old_concepts, new_concepts)
+result.matched_count    # concepts present in both
+result.new_only_count   # concepts only in new set
+result.old_only_count   # concepts only in old set
+result.diffs            # Array of ConceptDiff with similarity scores
 ----
 
 === NonVerbRep
 
-Non-verbal Representation have the following fields
+Non-verbal representations are associated resources (images, tables, formulas) used to help define a concept (ISO 10241-1 §6.5). They live outside the concept model and are referenced by URI. Resources can be shared across concepts and belong either to the dataset package (relative path) or are externally referenced (URL/URN).
 
-image:: An image used to help define a term.
-table:: A table used to help define a term.
-formula:: A formula used to help define a term.
-sources:: Bibliographic <<concept-source,concept source>> for the non-verbal representation of the term.
+type:: String — the type of representation: `image`, `table`, or `formula`.
+ref:: String — URI reference to the resource (relative path within the GCR package, URN, or URL).
+text:: String — optional text description or alt text.
+sources:: Collection of <<concept-source,ConceptSource>> entries — bibliographic sources for the representation.
+
+Example:
++
+[,yaml]
+----
+non_verbal_rep:
+  - type: image
+    ref: assets/images/figure-1.svg
+    text: Diagram showing the concept hierarchy
+  - type: formula
+    ref: urn:gcr:assets:formula-eq1
+    sources:
+      - type: authoritative
+        status: identical
+----
 
 [[id,concept-source]]
 === ConceptSource
@@ -338,7 +805,7 @@ Concept Source has the following fields
 
 status:: The status of the managed term in the present context, relative to the term as found in the bibliographic source.
 type:: The type of the managed term in the present context.
-origin:: The bibliographic <<citation,citation>> for the managed term. This is also aliased as `ref`.
+origin:: The bibliographic <<citation,citation>> for the managed term.
 modification:: A description of the modification to the cited definition of the term, if any, as it is to be applied in the present context.
 
 
@@ -863,6 +1330,15 @@ Glossarist::Validation::Rules::Registry.register(MyCustomRule)
 Custom rules are automatically picked up by `DatasetValidator`, `GcrValidator`,
 and `ConceptValidator` on the next validation run.
 
+=== compare
+
+Compare two concept datasets and report differences.
+
+[,bash]
+----
+glossarist compare PATH_OLD PATH_NEW [--format text|json|yaml]
+----
+
 === upgrade
 
 Upgrade a dataset to the current schema version.