esgvoc 1.0.1__tar.gz → 1.1.1__tar.gz

{esgvoc-1.0.1 → esgvoc-1.1.1}/PKG-INFO

@@ -1,12 +1,13 @@
 Metadata-Version: 2.3
 Name: esgvoc
-Version: 1.0.1
+Version: 1.1.1
 Summary: python library and CLI to interact with WCRP CVs
 Project-URL: Repository, https://github.com/ESGF/esgf-vocab
 Author-email: Sébastien Gardoll <sebastien@gardoll.fr>, Guillaume Levavasseur <guillaume.levavasseur@ipsl.fr>, Laurent Troussellier <laurent.troussellier@ipsl.fr>
 License: CECILL-2.1
 Requires-Python: >=3.10
 Requires-Dist: idna>=3.10
+Requires-Dist: jinja2>=3.1.6
 Requires-Dist: platformdirs>=4.3.6
 Requires-Dist: pydantic>=2.9.2
 Requires-Dist: pyld>=2.0.4

{esgvoc-1.0.1 → esgvoc-1.1.1}/docs/source/api_documentation/miscellaneous.md

@@ -2,7 +2,7 @@
 
 ```{eval-rst}
 .. automodule:: esgvoc.api.search
-   :members: Item
+   :members: Item, MatchingTerm
    :imported-members:
    :member-order: groupwise
 .. autoclass:: esgvoc.api.search.ItemKind

esgvoc-1.1.1/docs/source/how_to/configuration.rst

@@ -0,0 +1,296 @@
+Configuration
+#############
+
+The configuration system allows you to manage different project setups and easily switch between them. Each configuration defines which projects are active and where their repositories and databases are stored.
+
+Understanding Configurations
+============================
+
+A configuration contains:
+
+- **Universe settings**: Global WCRP vocabulary repository settings
+- **Project settings**: Individual project repositories (CMIP6, Input4MIPs, etc.)
+
+Default Projects Available
+==========================
+
+ESGVoc comes with several pre-configured projects that can be easily added:
+
+- **cmip6**: CMIP6 Controlled Vocabularies
+- **cmip6plus**: CMIP6Plus Controlled Vocabularies  
+- **input4mip**: Input4MIPs Controlled Vocabularies
+- **obs4mip**: Obs4MIPs Controlled Vocabularies
+- **cordex-cmip6**: CORDEX-CMIP6 Controlled Vocabularies
+
+Basic Configuration Commands
+============================
+
+Viewing Available Projects
+--------------------------
+
+.. code-block:: bash
+
+   # Show all available projects and their status in current config
+   esgvoc config avail
+
+   # Check projects in a specific configuration
+   esgvoc config avail --config my_config
+
+This displays a table showing which projects are active (✓) and which are available to add (○).
+
+Creating Empty Configurations
+-----------------------------
+
+.. code-block:: bash
+
+   # Create empty configuration and switch to it
+   esgvoc config init minimal_setup
+
+   # Create empty configuration but stay on current one
+   esgvoc config init test_config --no-switch
+
+An empty configuration contains only universe settings with no projects, allowing you to add projects selectively.
+
+Adding Projects
+---------------
+
+.. code-block:: bash
+
+   # Add single project to current configuration
+   esgvoc config add input4mip
+
+   # Add multiple projects at once
+   esgvoc config add input4mip obs4mip cordex-cmip6
+
+   # Add projects to specific configuration
+   esgvoc config add cmip6 --config production
+
+When you add projects:
+
+1. The project is added to the configuration using default settings
+2. The project's controlled vocabularies are automatically downloaded
+3. Local repositories and databases are set up
+
+.. note::
+   Projects that already exist in the configuration will be skipped with a warning.
+
+Removing Projects
+-----------------
+
+.. code-block:: bash
+
+   # Remove single project (with confirmation)
+   esgvoc config rm input4mip
+
+   # Remove multiple projects at once
+   esgvoc config rm input4mip obs4mip cordex-cmip6
+
+   # Remove without confirmation prompt
+   esgvoc config rm input4mip --force
+
+   # Remove from configuration but keep local files
+   esgvoc config rm input4mip --keep-files
+
+When you remove projects:
+
+1. The project is removed from the configuration
+2. Local repository directory is deleted (unless ``--keep-files``)
+3. Database file is deleted (unless ``--keep-files``)
+
+.. warning::
+   By default, removing a project deletes all its local files. Use ``--keep-files`` if you want to preserve the downloaded data.
+
+Advanced Configuration Management
+=================================
+
+Listing Configurations
+----------------------
+
+.. code-block:: bash
+
+   # List all available configurations
+   esgvoc config list
+
+   # Show current configuration content
+   esgvoc config show
+
+   # Show specific configuration content
+   esgvoc config show my_config
+
+Creating New Configurations
+---------------------------
+
+.. code-block:: bash
+
+   # Create configuration based on default settings
+   esgvoc config create production
+
+   # Create configuration based on existing one
+   esgvoc config create test --base production
+
+   # Create and immediately switch to it
+   esgvoc config create development --switch
+
+Switching Configurations
+------------------------
+
+.. code-block:: bash
+
+   # Switch to different configuration
+   esgvoc config switch production
+
+   # Check which configuration is currently active
+   esgvoc config list
+
+Managing Project Settings
+-------------------------
+
+.. code-block:: bash
+
+   # List projects in current configuration
+   esgvoc config list-projects
+
+   # List projects in specific configuration
+   esgvoc config list-projects --config production
+
+   # Update project repository URL
+   esgvoc config update-project cmip6 --repo https://github.com/new/repo
+
+   # Update project branch
+   esgvoc config update-project cmip6 --branch new_branch
+
+   # Update multiple settings at once
+   esgvoc config update-project cmip6 --repo https://github.com/new/repo --branch main
+
+Manual Configuration Editing
+-----------------------------
+
+.. code-block:: bash
+
+   # Edit current configuration in default editor
+   esgvoc config edit
+
+   # Edit specific configuration
+   esgvoc config edit production
+
+   # Use specific editor
+   esgvoc config edit --editor nano
+
+   # Modify settings via command line
+   esgvoc config set 'universe:branch=esgvoc_dev'
+   esgvoc config set 'cmip6:github_repo=https://github.com/new/cmip6'
+
+Removing Configurations
+-----------------------
+
+.. code-block:: bash
+
+   # Remove configuration (with confirmation)
+   esgvoc config remove test_config
+
+.. note::
+   You cannot remove the "default" configuration, and removing the active configuration will automatically switch you to the default one.
+
+Configuration Workflows
+========================
+
+Setting Up a Development Environment
+------------------------------------
+
+.. code-block:: bash
+
+   # Create empty development configuration
+   esgvoc config init dev
+
+   # Add only the projects you need
+   esgvoc config add cmip6 input4mip
+
+   # Check what's active
+   esgvoc config avail
+
+Setting Up Multiple Project Environments
+----------------------------------------
+
+.. code-block:: bash
+
+   # Create minimal configuration for CMIP6 only
+   esgvoc config init cmip6_only
+   esgvoc config add cmip6
+
+   # Create full configuration with all projects
+   esgvoc config init full_setup
+   esgvoc config add cmip6 cmip6plus input4mip obs4mip cordex-cmip6
+
+   # Switch between them as needed
+   esgvoc config switch cmip6_only
+   esgvoc config switch full_setup
+
+Cleaning Up Old Projects
+------------------------
+
+.. code-block:: bash
+
+   # Check what projects are currently active
+   esgvoc config avail
+
+   # Remove projects no longer needed (keeps files)
+   esgvoc config rm obs4mip cordex-cmip6 --keep-files
+
+   # Or remove completely including files
+   esgvoc config rm obs4mip cordex-cmip6
+
+Configuration File Location
+===========================
+
+Configuration files are stored in platform-specific directories:
+
+- **Linux**: ``~/.config/esgvoc/``
+- **macOS**: ``~/Library/Application Support/esgvoc/``  
+- **Windows**: ``%APPDATA%\\esgvoc\\``
+
+The main registry file (``config_registry.toml``) tracks all configurations and which one is active.
+
+Troubleshooting
+===============
+
+Configuration Not Found
+-----------------------
+
+If you get "Configuration not found" errors:
+
+.. code-block:: bash
+
+   # List available configurations
+   esgvoc config list
+
+   # Switch to default if needed
+   esgvoc config switch default
+
+Project Already Exists
+----------------------
+
+When adding projects that already exist:
+
+.. code-block:: bash
+
+   # Check current projects
+   esgvoc config list-projects
+
+   # Check all available projects and their status
+   esgvoc config avail
+
+Reset to Clean State
+--------------------
+
+To start fresh:
+
+.. code-block:: bash
+
+   # Create new empty configuration
+   esgvoc config init fresh --no-switch
+
+   # Remove old configuration
+   esgvoc config remove old_config
+
+   # Switch to clean setup
+   esgvoc config switch fresh

{esgvoc-1.0.1 → esgvoc-1.1.1}/pyproject.toml

@@ -21,6 +21,7 @@ dependencies = [
     "toml>=0.10.2",
     "typer>=0.15.0",
     "platformdirs>=4.3.6",
+    "jinja2>=3.1.6",
 ]
 readme = "README.md"
 requires-python = ">= 3.10"
@@ -59,6 +60,8 @@ dev = [
     "pytest>=8.3.5",
     "pytest-mock>=3.14.0",
     "pytest-cov>=6.0.0",
+    "jsonschema>=4.24.0",
+    "pip>=25.2",
 ]
 
 [tool.hatch.metadata]

{esgvoc-1.0.1 → esgvoc-1.1.1}/src/esgvoc/__init__.py

@@ -1,3 +1,3 @@
 import esgvoc.core.logging_handler  # noqa
 
-__version__ = "1.0.1"
+__version__ = "1.1.1"

{esgvoc-1.0.1 → esgvoc-1.1.1}/src/esgvoc/api/__init__.py

@@ -1,8 +1,5 @@
 from esgvoc.api.project_specs import (
-                                      DrsCollection,
-                                      DrsConstant,
                                       DrsPart,
-                                      DrsPartKind,
                                       DrsSpecification,
                                       DrsType,
                                       ProjectSpecs,
@@ -51,10 +48,7 @@ from esgvoc.api.universe import (
 )
 
 __all__ = [
-    "DrsCollection",
-    "DrsConstant",
     "DrsPart",
-    "DrsPartKind",
     "DrsSpecification",
     "DrsType",
     "find_collections_in_project",

{esgvoc-1.0.1 → esgvoc-1.1.1}/src/esgvoc/api/data_descriptors/__init__.py

@@ -1,7 +1,9 @@
 from esgvoc.api.data_descriptors.activity import Activity
+from esgvoc.api.data_descriptors.archive import Archive
 from esgvoc.api.data_descriptors.area_label import AreaLabel
 from esgvoc.api.data_descriptors.branded_suffix import BrandedSuffix
 from esgvoc.api.data_descriptors.branded_variable import BrandedVariable
+from esgvoc.api.data_descriptors.citation_url import CitationUrl
 from esgvoc.api.data_descriptors.consortium import Consortium
 from esgvoc.api.data_descriptors.contact import Contact
 from esgvoc.api.data_descriptors.conventions import Convention
@@ -30,6 +32,7 @@ from esgvoc.api.data_descriptors.product import Product
 from esgvoc.api.data_descriptors.publication_status import PublicationStatus
 from esgvoc.api.data_descriptors.realisation_index import RealisationIndex
 from esgvoc.api.data_descriptors.realm import Realm
+from esgvoc.api.data_descriptors.regex import Regex
 from esgvoc.api.data_descriptors.region import Region
 from esgvoc.api.data_descriptors.resolution import Resolution
 from esgvoc.api.data_descriptors.source import Source
@@ -90,4 +93,7 @@ DATA_DESCRIPTOR_CLASS_MAPPING: dict[str, type[DataDescriptor]] = {
     "region": Region,
     "member_id": MemberId,
     "obs_type": ObsType,  # obs4Mips
+    "regex": Regex,
+    "citation_url": CitationUrl,
+    "archive": Archive,
 }

esgvoc-1.1.1/src/esgvoc/api/data_descriptors/archive.py

@@ -0,0 +1,5 @@
+from esgvoc.api.data_descriptors.data_descriptor import PlainTermDataDescriptor
+
+
+class Archive(PlainTermDataDescriptor):
+    pass

esgvoc-1.1.1/src/esgvoc/api/data_descriptors/citation_url.py

@@ -0,0 +1,5 @@
+from esgvoc.api.data_descriptors.data_descriptor import PatternTermDataDescriptor
+
+
+class CitationUrl(PatternTermDataDescriptor):
+    pass

{esgvoc-1.0.1 → esgvoc-1.1.1}/src/esgvoc/api/data_descriptors/experiment.py

@@ -20,8 +20,8 @@ class Experiment(PlainTermDataDescriptor):
     experiment: str
     required_model_components: list[str] | None
     additional_allowed_model_components: list[str] = Field(default_factory=list)
-    start_year: int | None
-    end_year: int | None
+    start_year: str | int | None
+    end_year: str | int | None
     min_number_yrs_per_sim: int | None
     parent_activity_id: list[str] | None
     parent_experiment_id: list[str] | None

esgvoc-1.1.1/src/esgvoc/api/data_descriptors/known_branded_variable.py

@@ -0,0 +1,76 @@
+from typing import Any, Dict, List, Optional
+
+from pydantic import Field
+
+from esgvoc.api.data_descriptors.data_descriptor import PlainTermDataDescriptor
+
+#
+# class KnownBrandedVariable(PlainTermDataDescriptor):
+#     """
+#     A climate-related quantity or measurement, including information about sampling.
+#
+#     The concept of a branded variable was introduced in CMIP7.
+#     A branded variable is composed of two parts.
+#     The first part is the root variable (see :py:class:`Variable`).
+#     The second is the suffix (see :py:class:`BrandedSuffix`).
+#
+#     For further details on the development of branded variables,
+#     see [this paper draft](https://docs.google.com/document/d/19jzecgymgiiEsTDzaaqeLP6pTvLT-NzCMaq-wu-QoOc/edit?pli=1&tab=t.0).
+#     """
+#
+#     description: str
+#     dimensions: list[str] = Field(default_factory=list)
+#     cell_methods: str
+#     variable: str
+#     label: str
+#
+
+
+class KnownBrandedVariable(PlainTermDataDescriptor):
+    """
+    A climate-related quantity or measurement, including information about sampling.
+
+    The concept of a branded variable was introduced in CMIP7.
+    A branded variable is composed of two parts.
+    The first part is the root variable (see :py:class:`Variable`).
+    The second is the suffix (see :py:class:`BrandedSuffix`).
+
+    For further details on the development of branded variables,
+    see [this paper draft](https://docs.google.com/document/d/19jzecgymgiiEsTDzaaqeLP6pTvLT-NzCMaq-wu-QoOc/edit?pli=1&tab=t.0).
+    """
+
+    # # ESGVoc required fields
+    # id: str = Field(description="Unique identifier, e.g., 'ta_tavg-p19-hxy-air'")
+    # type: str = Field(default="branded_variable", description="ESGVoc type identifier")
+    # drs_name: str = Field(description="DRS name, same as id")
+    # => already in PlainTermDataDescriptor
+
+    # CF Standard Name context (flattened from hierarchy)
+    cf_standard_name: str = Field(description="CF standard name, e.g., 'air_temperature'")
+    cf_units: str = Field(description="CF standard units, e.g., 'K'")
+    cf_sn_status: str = Field(description="CF standard name status, e.g., 'approved'")
+
+    # Variable Root context (flattened from hierarchy)
+    variable_root_name: str = Field(description="Variable root name, e.g., 'ta'")
+    var_def_qualifier: str = Field(default="", description="Variable definition qualifier")
+    branding_suffix_name: str = Field(description="Branding suffix, e.g., 'tavg-p19-hxy-air'")
+
+    # Variable metadata
+    description: str = Field(description="Human-readable description")
+    dimensions: List[str] = Field(description="NetCDF dimensions")
+    cell_methods: str = Field(default="", description="CF cell_methods attribute")
+    cell_measures: str = Field(default="", description="CF cell_measures attribute")
+    history: str = Field(default="", description="Processing history")
+    realm: str = Field(description="Earth system realm, e.g., 'atmos'")
+
+    # Label components (embedded, not references)
+    temporal_label: str = Field(description="Temporal label, e.g., 'tavg'")
+    vertical_label: str = Field(description="Vertical label, e.g., 'p19'")
+    horizontal_label: str = Field(description="Horizontal label, e.g., 'hxy'")
+    area_label: str = Field(description="Area label, e.g., 'air'")
+
+    # Status
+    bn_status: str = Field(description="Branded variable status, e.g., 'accepted'")
+
+    # Additional required fields from specifications
+    positive_direction: str = Field(default="", description="Positive direction for the variable")

esgvoc-1.1.1/src/esgvoc/api/data_descriptors/regex.py

@@ -0,0 +1,5 @@
+from esgvoc.api.data_descriptors.data_descriptor import PatternTermDataDescriptor
+
+
+class Regex(PatternTermDataDescriptor):
+    pass

{esgvoc-1.0.1 → esgvoc-1.1.1}/src/esgvoc/api/data_descriptors/vertical_label.py

@@ -1,7 +1,7 @@
-from esgvoc.api.data_descriptors.data_descriptor import PatternTermDataDescriptor
+from esgvoc.api.data_descriptors.data_descriptor import PlainTermDataDescriptor
 
 
-class VerticalLabel(PatternTermDataDescriptor):
+class VerticalLabel(PlainTermDataDescriptor):
     """
     Vertical label.
 

esgvoc-1.1.1/src/esgvoc/api/project_specs.py

@@ -0,0 +1,116 @@
+from enum import Enum
+
+from pydantic import BaseModel, ConfigDict
+
+
+class DrsType(str, Enum):
+    """
+    The types of DRS specification (directory, file name and dataset id).
+    """
+
+    DIRECTORY = "directory"
+    """The DRS directory specification type."""
+    FILE_NAME = "file_name"
+    """The DRS file name specification type."""
+    DATASET_ID = "dataset_id"
+    """The DRS dataset id specification type."""
+
+
+class DrsPart(BaseModel):
+    """A fragment of a DRS specification"""
+
+    source_collection: str
+    """The collection id."""
+    source_collection_term: str | None = None
+    "Specifies a specific term in the collection."
+    is_required: bool
+    """Whether the collection is required for the DRS specification or not."""
+
+    def __str__(self) -> str:
+        return self.source_collection
+
+
+class DrsSpecification(BaseModel):
+    """
+    A DRS specification.
+    """
+
+    type: DrsType
+    """The type of the specification."""
+    regex: str
+    """General pattern for simples checks"""
+    separator: str
+    """The textual separator string or character."""
+    properties: dict | None = None
+    """The other specifications (e.g., file name extension for file name DRS specification)."""
+    parts: list[DrsPart]
+    """The parts of the DRS specification."""
+
+
+class CatalogProperty(BaseModel):
+    """
+    A dataset property described in a catalog.
+    """
+
+    source_collection: str
+    "The project collection that originated the property."
+    catalog_field_value_type: str
+    "The type of the field value."
+    is_required: bool
+    "Specifies if the property must be present in the dataset properties."
+    source_collection_term: str | None = None
+    "Specifies a specific term in the collection."
+    catalog_field_name: str | None = None
+    "The name of the collection referenced in the catalog."
+    source_collection_key: str | None = None
+    "Specifies a key other than drs_name in the collection."
+
+
+class CatalogExtension(BaseModel):
+    name: str
+    """The name of the extension"""
+    version: str
+    """The version of the extension"""
+
+
+class CatalogProperties(BaseModel):
+    name: str
+    """The name of the catalog system."""
+    url_template: str
+    """The URI template of the catalog system."""
+    extensions: list[CatalogExtension]
+    """The extensions of the catalog."""
+
+
+class CatalogSpecification(BaseModel):
+    """
+    A catalog specifications.
+    """
+
+    version: str
+    """The version of the catalog."""
+
+    catalog_properties: CatalogProperties
+    """The properties of the catalog."""
+
+    dataset_properties: list[CatalogProperty]
+    "The properties of the dataset described in a catalog."
+    file_properties: list[CatalogProperty]
+    "The properties of the files described in a catalog."
+
+
+class ProjectSpecs(BaseModel):
+    """
+    A project specifications.
+    """
+
+    project_id: str
+    """The project id."""
+    description: str
+    """The description of the project."""
+    drs_specs: dict[DrsType, DrsSpecification]
+    """The DRS specifications of the project (directory, file name and dataset id)."""
+    # TODO: release = None when all projects have catalog_specs.yaml.
+    catalog_specs: CatalogSpecification | None = None
+    """The catalog specifications of the project."""
+    model_config = ConfigDict(extra="allow")