unitsdb 2.1.0 → 2.2.1

data/README.adoc

@@ -19,6 +19,11 @@ https://github.com/unitsml/unitsdb.
 This repository contains the Ruby codebase for UnitsDB, which is used
 to access and manipulate the UnitsDB content.
 
+The UnitsDB gem ships with the UnitsDB data bundled internally. When you install
+the gem, the YAML data files are included automatically, so you do not need to
+obtain or configure a separate data source. The data is accessed via
+`Unitsdb.data_dir` or the convenience method `Unitsdb.database`.
+
 == Install
 
 [source,sh]
@@ -26,7 +31,79 @@ to access and manipulate the UnitsDB content.
 $ gem install unitsdb
 ----
 
+The UnitsDB gem ships with the UnitsDB YAML data files bundled inside the gem.
+Because the gem bundles immutable data, **the UnitsDB data must be released
+(tagged) before the gem can be released with updated data**. The gem version
+(`Unitsdb::VERSION`) and the data version (`Unitsdb::UNITS_DATA_VERSION`)
+are independent: the gem can be patched without changing data, and the data
+can be updated independently of the gem.
+
+
+
+== How the Database Works
+
+The `Unitsdb::Database` class loads all UnitsDB YAML files and provides
+methods for searching and querying the data.
+
+[source,ruby]
+----
+require 'unitsdb'
+
+# Access the pre-loaded bundled database (recommended)
+db = Unitsdb.database
+
+# Or load from a specific path
+db = Unitsdb::Database.from_db('/path/to/data')
+----
 
+=== Lazy Loading and Caching
+
+The bundled database (`Unitsdb.database`) is loaded on first access and cached
+for subsequent calls. The path to the bundled data is available via:
+
+[source,ruby]
+----
+Unitsdb.data_dir  # => Path to the data/ directory inside the gem
+----
+
+=== Database Structure
+
+The database contains collections for each entity type:
+
+[source,ruby]
+----
+db.units        # Array of Unit objects
+db.prefixes     # Array of Prefix objects
+db.dimensions   # Array of Dimension objects
+db.quantities   # Array of Quantity objects
+db.unit_systems # Array of UnitSystem objects
+db.scales       # Array of Scale objects
+----
+
+=== Searching the Database
+
+[source,ruby]
+----
+# Search by text (searches names, identifiers, descriptions)
+results = db.search(text: "meter")
+
+# Find by exact ID
+unit = db.get_by_id(id: "NISTu1")
+
+# Find by symbol
+units = db.find_by_symbol("m")
+----
+
+=== Validation
+
+[source,ruby]
+----
+# Check identifier uniqueness
+dups = db.validate_uniqueness
+
+# Validate all references exist
+invalid_refs = db.validate_references
+----
 
 == UnitsDB version support
 
@@ -38,6 +115,24 @@ The version of the YAML files are stored in the `version` field of the `*.yaml`
 files. The library checks this version when loading the database and raises an
 error if the version is not 2.0.0.
 
+The `unitsdb-ruby` gem version tracks the bundled data version via
+`Unitsdb::UNITS_DATA_VERSION`. For example, `unitsdb-ruby v2.1.2` bundles
+`unitsdb` data at `v2.0.0`. When the gem is released with an updated data
+submodule, both the gem version and `UNITS_DATA_VERSION` are updated together.
+
+The `data/` submodule is pinned to a specific tag in the
+https://github.com/unitsml/unitsdb[UnitsDB repository] (e.g. `refs/tags/v2.0.0`).
+This means:
+
+* Every bundled version of `unitsdb-ruby` ships with a known, immutable version
+  of the UnitsDB data.
+* A new `unitsdb-ruby` release can only be made after the upstream UnitsDB data
+  has been released (tagged) in its own repository.
+* To release a new gem with updated data: tag the new data in
+  `unitsml/unitsdb`, update the submodule's `branch` in `.gitmodules` to point
+  to `refs/tags/new-data-tag`, update `UNITS_DATA_VERSION`, then bump the gem
+  version and release.
+
 === UnitsDB 2.0.0 features
 
 ==== General
@@ -101,6 +196,7 @@ Syntax:
 [source,yaml]
 ----
 schema_version: 2.0.0
+version: 1.0.0  # Release version (in semantic format x.y.z)
 dimensions:
   - identifiers: [...]
     length: {...}
@@ -224,6 +320,29 @@ Options:
 
 `--database`, `-d`:: Path to UnitsDB database (required)
 
+===== QUDT references validation
+
+Validates that each QUDT reference is unique per entity type:
+
+[source,sh]
+----
+$ unitsdb validate qudt_references --database=/path/to/unitsdb/data
+----
+
+This command checks that each QUDT (Quantities, Units, Dimensions and Types) URI is referenced by at most
+one entity of each type. Multiple entities of the same type referencing the same
+QUDT URI could cause issues with mapping and conversion processes.
+
+The command reports:
+
+* Any duplicate QUDT references within each entity type
+* The entities that share the same QUDT reference
+* Their position in the database for easy location
+
+Options:
+
+`--database`, `-d`:: Path to UnitsDB database (required)
+
 === Examples of validation commands
 
 * Check identifiers for uniqueness:
@@ -269,7 +388,7 @@ Options:
 
 `--all`, `-a`:: Process all YAML files in the repository
 `--database`, `-d`:: Path to UnitsDB database (required)
-`--sort`:: Sort units by: 'short' (name, default), 'nist' (ID), 'unitsml' (ID), or 'none'
+`--sort`:: Sort units by: 'short' (name), 'nist' (ID, default), 'unitsml' (ID), or 'none'
 
 
 ==== Search
@@ -340,27 +459,27 @@ entities
 [source,sh]
 ----
 # Check all entity types and generate a report
-$ unitsdb check_si --database=spec/fixtures/unitsdb --ttl-dir=spec/fixtures/bipm-si-ttl
+$ unitsdb check_si --database=data --ttl-dir=spec/fixtures/bipm-si-ttl
 
 # Check a specific entity type (units, quantities, or prefixes)
 $ unitsdb check_si --entity-type=units \
-  --database=spec/fixtures/unitsdb \
+  --database=data \
   --ttl-dir=spec/fixtures/bipm-si-ttl
 
 # Check in a specific direction only
 $ unitsdb check_si --direction=from_si \
-  --database=spec/fixtures/unitsdb \
+  --database=data \
   --ttl-dir=spec/fixtures/bipm-si-ttl
 
 # Update references and write to output directory
 $ unitsdb check_si --output-updated-database=new_unitsdb \
-  --database=spec/fixtures/unitsdb \
+  --database=data \
   --ttl-dir=spec/fixtures/bipm-si-ttl
 
 # Include potential matches when updating references (default: false)
 $ unitsdb check_si --include-potential-matches \
   --output-updated-database=new_unitsdb \
-  --database=spec/fixtures/unitsdb \
+  --database=data \
   --ttl-dir=spec/fixtures/bipm-si-ttl
 ----
 
@@ -383,6 +502,240 @@ with added SI references
 `--include-potential-matches`, `-p`:: Include potential matches when updating
 references (default: false)
 
+
+==== Check references to UCUM
+
+Performs a comprehensive check of entities in the UCUM XML file against UnitsDB
+database entities and updates UnitsDB with UCUM references.
+
+UCUM supports the following entity types:
+
+* Base units
+* Units
+* Prefixes
+
+UCUM provides dimensions as part of their unit definitions but not as
+uniquely referencable entities.
+
+This combined command checks in both directions to ensure UnitsDB supports
+every UCUM entity:
+
+* From UCUM to UnitsDB: Ensures every UCUM entity is referenced by at least one
+UnitsDB entity
+
+* From UnitsDB to UCUM: Identifies UnitsDB entities that should reference UCUM
+entities
+
+There are two commands:
+
+* `ucum check`: Checks for matches between UnitsDB and UCUM entities and reports results
+
+* `ucum update`: Updates UnitsDB entities with references to matching UCUM entities
+
+[source,sh]
+----
+# Check all entity types and generate a report
+$ unitsdb ucum check --database=data --ucum-file=spec/fixtures/ucum/ucum-essence.xml
+
+# Check a specific entity type (units or prefixes)
+$ unitsdb ucum check --entity-type=units \
+  --database=data \
+  --ucum-file=spec/fixtures/ucum/ucum-essence.xml
+
+# Check in a specific direction only
+$ unitsdb ucum check --direction=from_ucum \
+  --database=data \
+  --ucum-file=spec/fixtures/ucum/ucum-essence.xml
+----
+
+Options:
+
+`--database`, `-d`:: Path to UnitsDB database (required)
+
+`--ucum-file`, `-u`:: Path to the UCUM essence XML file (required)
+
+`--entity-type`, `-e`:: Entity type to check (units or prefixes).
+If not specified, all types are checked.
+
+`--direction`, `-r`:: Direction to check: `to_ucum` (UnitsDB→UCUM), `from_ucum`
+(UCUM→UnitsDB), or `both` (default)
+
+`--output-updated-database`, `-o`:: Directory path to write updated YAML files
+with added UCUM references
+
+`--include-potential-matches`, `-p`:: Include potential matches when updating
+references (default: false)
+
+
+==== Update UCUM references
+
+[source,sh]
+----
+# Update all entity types with UCUM references
+$ unitsdb ucum update --database=data \
+  --ucum-file=spec/fixtures/ucum/ucum-essence.xml \
+  --output-dir=new_unitsdb
+
+# Update a specific entity type (units or prefixes)
+$ unitsdb ucum update --entity-type=units \
+  --database=data \
+  --ucum-file=spec/fixtures/ucum/ucum-essence.xml \
+  --output-dir=new_unitsdb
+
+# Include potential matches when updating references (default: false)
+$ unitsdb ucum update --include-potential-matches \
+  --database=data \
+  --ucum-file=spec/fixtures/ucum/ucum-essence.xml \
+  --output-dir=new_unitsdb
+----
+
+Options:
+
+`--database`, `-d`:: Path to UnitsDB database (required)
+
+`--ucum-file`, `-u`:: Path to the UCUM essence XML file (required)
+
+`--entity-type`, `-e`:: Entity type to update (units or prefixes).
+If not specified, all types are updated
+
+`--output-dir`, `-o`:: Directory path to write updated YAML files
+(defaults to database path)
+
+`--include-potential-matches`, `-p`:: Include potential matches when updating
+references (default: false)
+
+
+==== Check references to QUDT
+
+Performs a comprehensive check of entities in the
+https://qudt.org/3.1.2/vocab/unit[QUDT] (Quantities, Units, Dimensions and
+Types) vocabularies against UnitsDB database entities and updates UnitsDB with
+QUDT references.
+
+The support of QUDT mappings in the UnitsDB is purely informative.
+
+QUDT supports the following entity types, and they are mapped to
+UnitsDB as follows:
+
+* Units: mapped to Units in UnitsDB
+* Quantity Kinds: mapped to Quantities in UnitsDB
+* Dimension Vectors: mapped to Dimensions in UnitsDB
+* Systems of Units: mapped to Unit Systems in UnitsDB
+* Prefixes: mapped to Prefixes in UnitsDB
+* (Physical Constants: not supported in UnitsDB)
+* (Systems of Quantity Kinds: not supported in UnitsDB)
+
+The QUDT Vocabulary is very extensive and includes many entities that are not
+reflected in the UnitsDB database, with the following categories:
+
+* Many composed units in QUDT are omitted from UnitsDB for separation of
+concerns;
+
+* Some quantities are not included in UnitsDB for being less commonly used;
+(e.g. "Deaths per million")
+
+
+This combined command checks in both directions to ensure UnitsDB supports
+every QUDT entity.
+
+* From QUDT to UnitsDB: Ensures every QUDT entity is referenced by at least one
+UnitsDB entity
+
+* From UnitsDB to QUDT: Identifies UnitsDB entities that should reference QUDT
+entities
+
+There are two commands:
+
+* `qudt check`: Checks for matches between UnitsDB and QUDT entities and reports results
+
+* `qudt update`: Updates UnitsDB entities with references to matching QUDT entities
+
+[source,sh]
+----
+# Check all entity types and generate a report
+$ unitsdb qudt check --database=data
+
+# Check a specific entity type (units, quantities, dimensions, or unit_systems)
+$ unitsdb qudt check --entity-type=units \
+  --database=data
+
+# Use local TTL files instead of downloading from QUDT.org
+$ unitsdb qudt check --ttl-dir=/path/to/qudt/ttl/files \
+  --database=data
+
+# Check in a specific direction only
+$ unitsdb qudt check --direction=from_qudt \
+  --database=data
+
+# Include potential matches in the output
+$ unitsdb qudt check --include-potential-matches \
+  --database=data
+
+# Output updated database files
+$ unitsdb qudt check --output-dir=/path/to/output \
+  --database=data
+----
+
+Options:
+
+`--database`, `-d`:: Path to UnitsDB database (required)
+
+`--ttl-dir`, `-t`:: Path to the directory containing QUDT TTL files (optional,
+downloads from QUDT.org if not specified)
+
+`--entity-type`, `-e`:: Entity type to check (units, quantities, dimensions, or
+unit_systems). If not specified, all types are checked.
+
+`--direction`, `-r`:: Direction to check: `to_qudt` (UnitsDB→QUDT), `from_qudt`
+(QUDT→UnitsDB), or `both` (default)
+
+`--output-dir`, `-o`:: Directory path to write updated YAML files with added
+QUDT references
+
+`--include-potential-matches`, `-p`:: Include potential matches when updating
+references (default: false)
+
+
+==== Update QUDT references
+
+[source,sh]
+----
+# Update all entity types with QUDT references
+$ unitsdb qudt update --database=data \
+  --output-dir=new_unitsdb
+
+# Update a specific entity type (units, quantities, dimensions, or unit_systems)
+$ unitsdb qudt update --entity-type=units \
+  --database=data \
+  --output-dir=new_unitsdb
+
+# Use local TTL files instead of downloading
+$ unitsdb qudt update --ttl-dir=/path/to/qudt/ttl/files \
+  --database=data \
+  --output-dir=new_unitsdb
+
+# Include potential matches when updating references (default: false)
+$ unitsdb qudt update --include-potential-matches \
+  --database=data \
+  --output-dir=new_unitsdb
+----
+
+Options:
+
+`--database`, `-d`:: Path to UnitsDB database (required)
+
+`--ttl-dir`, `-t`:: Path to the directory containing QUDT TTL files (optional,
+downloads from QUDT.org if not specified)
+
+`--entity-type`, `-e`:: Entity type to update (units, quantities, dimensions, or
+unit_systems). If not specified, all types are updated
+
+`--output-dir`, `-o`:: Directory path to write updated YAML files (defaults to
+database path)
+
+`--include-potential-matches`, `-p`:: Include potential matches when updating
+references (default: false)
+
 ==== Release
 
 Creates release files for UnitsDB in unified formats:
@@ -400,6 +753,9 @@ $ unitsdb release --format=zip --database=/path/to/unitsdb/data
 
 # Specify output directory
 $ unitsdb release --output-dir=/path/to/output --database=/path/to/unitsdb/data
+
+# Specify a version (required)
+$ unitsdb release --version=2.1.0 --database=/path/to/unitsdb/data
 ----
 
 This command creates release files for UnitsDB in two formats:
@@ -410,7 +766,7 @@ This command creates release files for UnitsDB in two formats:
 
 The command verifies that all files have the same schema version before creating
 the release files. The output files are named with the schema version (e.g.,
-`unitsdb-2.0.0.yaml` and `unitsdb-2.0.0.zip`).
+`unitsdb-2.1.0.yaml` and `unitsdb-2.1.0.zip`).
 
 Options:
 
@@ -537,15 +893,15 @@ symbol-to-symbol and partial matches always classified as potential
 
 === Loading the database
 
-The primary way to load the UnitsDB data is through the `Database.from_db`
-method, which reads data from YAML files:
+The UnitsDB gem ships with the UnitsDB data bundled inside the gem. You can load
+the database using the convenience method:
 
 [source,ruby]
 ----
 require 'unitsdb'
 
-# Load from the UnitsDB data directory
-db = Unitsdb::Database.from_db('/path/to/unitsdb/data')
+# Load the bundled UnitsDB data (all entity types pre-loaded)
+db = Unitsdb.database
 
 # Access different collections
 units = db.units
@@ -555,6 +911,19 @@ quantities = db.quantities
 unit_systems = db.unit_systems
 ----
 
+Alternatively, you can load from a specific path using `Database.from_db`:
+
+[source,ruby]
+----
+require 'unitsdb'
+
+# Load from the bundled data directory
+db = Unitsdb::Database.from_db(Unitsdb.data_dir)
+
+# Load from an external directory
+external_db = Unitsdb::Database.from_db('/path/to/custom/unitsdb/data')
+----
+
 === Database search methods
 
 The UnitsDB Ruby gem provides several methods for searching and retrieving
@@ -715,13 +1084,17 @@ The `Quantity` class represents physical quantities that can be measured:
 
 === Database files
 
-The `Database.from_db` method reads the following YAML files:
+The UnitsDB gem bundles the following YAML files from the
+https://github.com/unitsml/unitsdb[UnitsDB repository]. They are included in the
+gem under the `data/` directory and are available immediately after installation
+without any additional setup.
 
 * `prefixes.yaml` - Contains prefix definitions (e.g., kilo-, mega-)
 * `dimensions.yaml` - Contains dimension definitions (e.g., length, mass)
 * `units.yaml` - Contains unit definitions (e.g., meter, kilogram)
 * `quantities.yaml` - Contains quantity definitions (e.g., length, mass)
 * `unit_systems.yaml` - Contains unit system definitions (e.g., SI, Imperial)
+* `scales.yaml` - Contains scale definitions