pysdmx 1.2.2__tar.gz → 1.4.0__tar.gz

pysdmx-1.4.0/PKG-INFO

@@ -0,0 +1,119 @@
+Metadata-Version: 2.3
+Name: pysdmx
+Version: 1.4.0
+Summary: Your opinionated Python SDMX library
+License: Apache-2.0
+Keywords: sdmx,data discovery,data retrieval,metadata,fmr
+Author: Xavier Sosnovsky
+Author-email: <xavier.sosnovsky@bis.org>
+Requires-Python: >=3.9
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Typing :: Typed
+Provides-Extra: all
+Provides-Extra: data
+Provides-Extra: dc
+Provides-Extra: vtl
+Provides-Extra: xml
+Requires-Dist: httpx (>=0)
+Requires-Dist: lxml (>=5.2) ; extra == "all"
+Requires-Dist: lxml (>=5.2) ; extra == "xml"
+Requires-Dist: msgspec (>=0)
+Requires-Dist: pandas (>=2.1.4) ; extra == "all"
+Requires-Dist: pandas (>=2.1.4) ; extra == "data"
+Requires-Dist: parsy (>=2.1)
+Requires-Dist: python-dateutil (>=2.8.2) ; extra == "all"
+Requires-Dist: python-dateutil (>=2.8.2) ; extra == "dc"
+Requires-Dist: sdmxschemas (>=0.2.0) ; extra == "all"
+Requires-Dist: sdmxschemas (>=0.2.0) ; extra == "xml"
+Requires-Dist: vtlengine (>=1.1,<2.0) ; extra == "all"
+Requires-Dist: vtlengine (>=1.1,<2.0) ; extra == "vtl"
+Requires-Dist: xmltodict (>=0.13) ; extra == "all"
+Requires-Dist: xmltodict (>=0.13) ; extra == "xml"
+Project-URL: Bug Tracker, https://bis-med-it.github.io/pysdmx/issues
+Project-URL: Documentation, https://bis-med-it.github.io/pysdmx
+Project-URL: Homepage, https://sdmx.io/tools/pysdmx
+Project-URL: Repository, https://github.com/bis-med-it/pysdmx
+Description-Content-Type: text/x-rst
+
+.. |pypi badge| image:: https://img.shields.io/pypi/v/pysdmx.svg
+   :target: https://pypi.org/project/pysdmx/
+
+.. |awesome badge| image:: https://awesome.re/mentioned-badge.svg
+   :target: http://www.awesomeofficialstatistics.org
+
+|pypi badge| |awesome badge|
+
+``pysdmx`` in a nutshell
+************************
+
+What is pysdmx?
+===============
+
+``pysdmx`` is a **pragmatic** and **opinionated** SDMX library written in
+**Python**. It focuses on **simplicity**, providing a subset of SDMX functionalities
+without requiring advanced knowledge of SDMX. ``pysdmx`` is developed as part of
+the `sdmx.io <http://sdmx.io/>`_ project under the **BIS Open Tech initiative**.
+
+What does it do?
+================
+
+``pysdmx`` aspires to be a versatile SDMX toolbox for Python, covering various
+use cases. Here are some highlights:
+
+SDMX information model in Python
+--------------------------------
+
+``pysdmx`` offers Python classes representing a **simplified subset of the SDMX
+information model**, enabling a domain-driven development of SDMX processes in
+Python. The model classes support serialization in formats like JSON, YAML, or
+MessagePack. This functionality relies on the 
+`msgspec library <https://jcristharif.com/msgspec/>`_.
+
+Metadata in action
+------------------
+
+SDMX metadata are very useful for documenting statistical processes. For example,
+they can define the structure we expect for a data collection process and share
+it with the organizations providing data so that they know what to send.
+
+However, metadata can do so much more than that, i.e., they can be “active” and
+**drive various types of statistical processes**, such as generating the filesystem
+layout, creating the physical data model, validating data, mapping data, and
+configuring processes. To drive such processes, ``pysdmx`` supports retrieving
+metadata from an SDMX Registry or any service compliant with the SDMX-REST 2.0.0 (or
+above) API. Use these metadata to power your own statistical processes!
+
+Reading and writing SDMX files
+------------------------------
+
+``pysdmx`` supports reading and writing SDMX data and structure messages, in various
+formats, such as SDMX-CSV, SDMX-JSON, and SDMX-ML.
+
+Data discovery and data retrieval
+---------------------------------
+
+This functionality is under development. Once ready, ``pysdmx`` will allow:
+ 
+- **Listing public SDMX services**.
+- **Discovering data** available in these services.
+- **Retrieving data** from these services.
+ 
+This functionality is based on the **SDMX Global Discovery Service initiative**.
+
+Integration with the ecosystem
+------------------------------
+
+``pysdmx`` integrates nicely with other standards, like the `Validation and
+Transformation Language (VTL) <https://sdmx.org/about-sdmx/about-vtl/>`_,
+and major Python libraries like `Pandas <https://pandas.pydata.org/>`_.
+Take a look at the ``pysdmx`` toolkit module to learn more.
+
+``pysdmx`` is available on `PyPI <https://pypi.org/>`_ and can be
+installed using options such as pip, pipx, poetry, etc.
+
+For more details, check the `project documentation 
+<https://py.sdmx.io>`_.
+

pysdmx-1.4.0/README.rst

@@ -0,0 +1,78 @@
+.. |pypi badge| image:: https://img.shields.io/pypi/v/pysdmx.svg
+   :target: https://pypi.org/project/pysdmx/
+
+.. |awesome badge| image:: https://awesome.re/mentioned-badge.svg
+   :target: http://www.awesomeofficialstatistics.org
+
+|pypi badge| |awesome badge|
+
+``pysdmx`` in a nutshell
+************************
+
+What is pysdmx?
+===============
+
+``pysdmx`` is a **pragmatic** and **opinionated** SDMX library written in
+**Python**. It focuses on **simplicity**, providing a subset of SDMX functionalities
+without requiring advanced knowledge of SDMX. ``pysdmx`` is developed as part of
+the `sdmx.io <http://sdmx.io/>`_ project under the **BIS Open Tech initiative**.
+
+What does it do?
+================
+
+``pysdmx`` aspires to be a versatile SDMX toolbox for Python, covering various
+use cases. Here are some highlights:
+
+SDMX information model in Python
+--------------------------------
+
+``pysdmx`` offers Python classes representing a **simplified subset of the SDMX
+information model**, enabling a domain-driven development of SDMX processes in
+Python. The model classes support serialization in formats like JSON, YAML, or
+MessagePack. This functionality relies on the 
+`msgspec library <https://jcristharif.com/msgspec/>`_.
+
+Metadata in action
+------------------
+
+SDMX metadata are very useful for documenting statistical processes. For example,
+they can define the structure we expect for a data collection process and share
+it with the organizations providing data so that they know what to send.
+
+However, metadata can do so much more than that, i.e., they can be “active” and
+**drive various types of statistical processes**, such as generating the filesystem
+layout, creating the physical data model, validating data, mapping data, and
+configuring processes. To drive such processes, ``pysdmx`` supports retrieving
+metadata from an SDMX Registry or any service compliant with the SDMX-REST 2.0.0 (or
+above) API. Use these metadata to power your own statistical processes!
+
+Reading and writing SDMX files
+------------------------------
+
+``pysdmx`` supports reading and writing SDMX data and structure messages, in various
+formats, such as SDMX-CSV, SDMX-JSON, and SDMX-ML.
+
+Data discovery and data retrieval
+---------------------------------
+
+This functionality is under development. Once ready, ``pysdmx`` will allow:
+ 
+- **Listing public SDMX services**.
+- **Discovering data** available in these services.
+- **Retrieving data** from these services.
+ 
+This functionality is based on the **SDMX Global Discovery Service initiative**.
+
+Integration with the ecosystem
+------------------------------
+
+``pysdmx`` integrates nicely with other standards, like the `Validation and
+Transformation Language (VTL) <https://sdmx.org/about-sdmx/about-vtl/>`_,
+and major Python libraries like `Pandas <https://pandas.pydata.org/>`_.
+Take a look at the ``pysdmx`` toolkit module to learn more.
+
+``pysdmx`` is available on `PyPI <https://pypi.org/>`_ and can be
+installed using options such as pip, pipx, poetry, etc.
+
+For more details, check the `project documentation 
+<https://py.sdmx.io>`_.

{pysdmx-1.2.2 → pysdmx-1.4.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pysdmx"
-version = "1.2.2"
+version = "1.4.0"
 description = "Your opinionated Python SDMX library"
 license = { text = "Apache-2.0" }
 readme = "README.rst"
@@ -9,7 +9,8 @@ authors = [
     { name = "Xavier Sosnovsky", email = "<xavier.sosnovsky@bis.org>" },
     { name = "Stratos Nikoloutsos", email = "<stratos.nikoloutsos@bis.org>" },
     { name = "Francisco Javier Hernandez del Caño", email = "<javier.hernandez@meaningfuldata.eu>" },
-    { name = "Mateo de Lorenzo Argelés", email = "<mateo.delorenzo@meaningfuldata.eu>" }
+    { name = "Mateo de Lorenzo Argelés", email = "<mateo.delorenzo@meaningfuldata.eu>" },
+    { name = "Jose Antonio Franco Martin", email = "<jose.franco@meaningfuldata.eu>" }
 ]
 keywords = ["sdmx", "data discovery", "data retrieval", "metadata", "fmr"]
 classifiers = [
@@ -34,9 +35,9 @@ documentation = "https://bis-med-it.github.io/pysdmx"
 [project.optional-dependencies]
 data = ["pandas>=2.1.4"]
 dc = ["python-dateutil>=2.8.2"]
-vtl = ["vtlengine>=1.1rc1"]
+vtl = ["vtlengine>=1.1,<2.0"]
 xml = ["lxml>=5.2", "xmltodict>=0.13", "sdmxschemas>=0.2.0"]
-all = ["lxml>=5.2", "xmltodict>=0.13", "sdmxschemas>=0.2.0", "pandas>=2.1.4", "python-dateutil>=2.8.2", "vtlengine>=1.1rc1"]
+all = ["lxml>=5.2", "xmltodict>=0.13", "sdmxschemas>=0.2.0", "pandas>=2.1.4", "python-dateutil>=2.8.2", "vtlengine>=1.1,<2.0"]
 
 [tool.poetry]
 requires-poetry = ">=2.0"

{pysdmx-1.2.2 → pysdmx-1.4.0}/src/pysdmx/__extras_check.py

@@ -50,7 +50,8 @@ def __check_vtl_extra() -> None:
     except ImportError:
         raise ImportError(
             ERROR_MESSAGE.format(
-                extra_name="vtlengine",
-                extra_desc="validations for VTL engine",
+                extra_name="vtl",
+                extra_desc="VTL Scripts, SDMX-VTL model validations"
+                " and prettify",
             )
         ) from None

{pysdmx-1.2.2 → pysdmx-1.4.0}/src/pysdmx/__init__.py

@@ -1,3 +1,3 @@
 """Your opinionated Python SDMX library."""
 
-__version__ = "1.2.2"
+__version__ = "1.4.0"

{pysdmx-1.2.2 → pysdmx-1.4.0}/src/pysdmx/api/fmr/__init__.py

@@ -642,7 +642,7 @@ class RegistryClient(__BaseRegistryClient):
         """
         query = super()._report_q(provider, id, version)
         out = self.__fetch(query)
-        return super()._out(out, self.deser.report)[0]
+        return super()._out(out, self.deser.report).reports[0]
 
     def get_reports(
         self,
@@ -667,7 +667,7 @@ class RegistryClient(__BaseRegistryClient):
         """
         query = super()._reports_q(artefact_type, agency, id, version)
         out = self.__fetch(query)
-        return super()._out(out, self.deser.report, True)
+        return super()._out(out, self.deser.report).reports
 
     def get_mapping(
         self,
@@ -1104,7 +1104,7 @@ class AsyncRegistryClient(__BaseRegistryClient):
         """
         query = super()._report_q(provider, id, version)
         out = await self.__fetch(query)
-        return super()._out(out, self.deser.report)[0]
+        return super()._out(out, self.deser.report).reports[0]
 
     async def get_reports(
         self,
@@ -1129,7 +1129,7 @@ class AsyncRegistryClient(__BaseRegistryClient):
         """
         query = super()._reports_q(artefact_type, agency, id, version)
         out = await self.__fetch(query)
-        return super()._out(out, self.deser.report, True)
+        return super()._out(out, self.deser.report).reports
 
     async def get_mapping(
         self,

pysdmx-1.4.0/src/pysdmx/api/gds/__init__.py

@@ -0,0 +1,328 @@
+"""API client for interacting with the GDS (Global Discovery Service) service.
+
+This module provides classes and utilities to interact with the GDS service,
+allowing retrieval of metadata such as agency information in SDMX-JSON format.
+
+Exports: GdsClient: A synchronous client for retrieving metadata from
+the GDS.
+"""
+
+from typing import Any, Literal, Optional, Sequence
+
+from msgspec.json import decode
+
+from pysdmx.api.qb.gds import GdsQuery, GdsType
+from pysdmx.api.qb.service import GdsAsyncRestService, GdsRestService
+from pysdmx.api.qb.util import REST_ALL
+from pysdmx.io.json.gds.reader import deserializers as gds_readers
+from pysdmx.io.serde import Deserializer
+from pysdmx.model import Agency
+from pysdmx.model.gds import (
+    GdsCatalog,
+    GdsSdmxApi,
+    GdsService,
+    GdsUrnResolver,
+)
+
+GDS_BASE_ENDPOINT = "https://gds.sdmx.io/"
+
+READERS = gds_readers
+
+
+class __BaseGdsClient:
+    def __init__(
+        self,
+        api_endpoint: str = GDS_BASE_ENDPOINT,
+        pem: Optional[str] = None,
+    ):
+        """Instantiate a new client against the target endpoint."""
+        if api_endpoint.endswith("/"):
+            api_endpoint = api_endpoint[0:-1]
+        self.api_endpoint = api_endpoint
+        self.reader = READERS
+
+    def _out(self, response: bytes, typ: Deserializer, *params: Any) -> Any:
+        return decode(response, type=typ).to_model(*params)
+
+    def _agencies_q(self, agency: str) -> GdsQuery:
+        return GdsQuery(artefact_type=GdsType.GDS_AGENCY, resource_id=agency)
+
+    def _catalogs_q(
+        self,
+        agency: str,
+        resource: str = REST_ALL,
+        version: str = REST_ALL,
+        resource_type: Optional[Literal["data", "metadata"]] = None,
+        message_format: Optional[Literal["json", "csv", "xml"]] = None,
+        api_version: Optional[str] = None,
+        detail: Optional[Literal["full", "raw"]] = None,
+        references: Optional[Literal["none", "children"]] = None,
+    ) -> GdsQuery:
+        return GdsQuery(
+            artefact_type=GdsType.GDS_CATALOG,
+            agency=agency,
+            resource_id=resource,
+            version=version,
+            resource_type=resource_type,
+            message_format=message_format,
+            api_version=api_version,
+            detail=detail,
+            references=references,
+        )
+
+    def _sdmx_api_q(self, id: str = REST_ALL) -> GdsQuery:
+        return GdsQuery(artefact_type=GdsType.GDS_SDMX_API, resource_id=id)
+
+    def _services_q(
+        self, agency: str, resource: str = REST_ALL, version: str = REST_ALL
+    ) -> GdsQuery:
+        return GdsQuery(
+            artefact_type=GdsType.GDS_SERVICE,
+            agency=agency,
+            resource_id=resource,
+            version=version,
+        )
+
+    def _urn_resolver_q(self, urn: str) -> GdsQuery:
+        return GdsQuery(
+            artefact_type=GdsType.GDS_URN_RESOLVER, resource_id=urn
+        )
+
+
+class GdsClient(__BaseGdsClient):
+    """A client to be used to retrieve metadata from the GDS.
+
+    With this client, metadata will be retrieved in a synchronous fashion.
+    """
+
+    def __init__(
+        self,
+        api_endpoint: str = GDS_BASE_ENDPOINT,
+        pem: Optional[str] = None,
+    ) -> None:
+        """Instantiate a new client against the target endpoint.
+
+        Args:
+            api_endpoint: The endpoint of the targeted service.
+            pem: In case the service exposed a certificate created
+                by an unknown certificate authority, you can pass
+                a pem file for this authority using this parameter.
+        """
+        super().__init__(api_endpoint, pem)
+        self.__service = GdsRestService(
+            self.api_endpoint,
+            pem=pem,
+            timeout=10.0,
+        )
+
+    def __fetch(
+        self,
+        query: GdsQuery,
+    ) -> bytes:
+        """Fetch the requested metadata from the GDS service."""
+        return self.__service.gds(query)
+
+    def get_agencies(self, agency: str) -> Sequence[Agency]:
+        """Get the list of agencies for the supplied name.
+
+        Args:
+            agency: The agency maintaining the agency scheme from
+                which sub-agencies must be returned.
+
+        Returns:
+            The requested list of agencies.
+        """
+        query = super()._agencies_q(agency)
+        out = self.__fetch(query)
+        agencies = super()._out(out, self.reader.agencies)
+        return agencies
+
+    def get_catalogs(
+        self,
+        catalog: str,
+        resource: str = REST_ALL,
+        version: str = REST_ALL,
+        resource_type: Optional[Literal["data", "metadata"]] = None,
+        message_format: Optional[Literal["json", "csv", "xml"]] = None,
+        api_version: Optional[str] = None,
+        detail: Optional[Literal["full", "raw"]] = None,
+        references: Optional[Literal["none", "children"]] = None,
+    ) -> Sequence[GdsCatalog]:
+        """Get the list of catalogs for the supplied parameters.
+
+        Args:
+            catalog: The agency maintaining the catalog.
+            resource: The resource ID(s) to query. Defaults to '*'.
+            version: The version(s) of the resource. Defaults to '*'.
+            resource_type: Filters the endpoints that support
+             the requested resource type (eg, 'data', 'metadata')
+            message_format: Filters the endpoints that support any
+              of the requested message formats.
+            api_version: Filters the endpoints that is in a
+              specific SDMX API version.
+              Multiple values separated by commas are possible.
+              By default (if nothing is sent) it returns everything.
+            detail: The amount of information to be returned.
+              If detail=full: All available information for all artefacts
+              should be returned.
+              If detail=raw: Any nested service will be referenced.
+            references: Instructs the web service to return
+              (or not) the artefacts referenced by the
+              artefact to be returned.
+              If references=none: No referenced artefacts will be returned.
+              If references=children: Returns the artefacts
+              referenced by the artefact to be returned.
+
+        Returns:
+            A list of GdsCatalog objects.
+        """
+        query = super()._catalogs_q(
+            catalog,
+            resource,
+            version,
+            resource_type,
+            message_format,
+            api_version,
+            detail,
+            references,
+        )
+        response = self.__fetch(query)
+        catalogs = super()._out(response, self.reader.catalogs)
+        return catalogs
+
+    def get_sdmx_apis(
+        self, api_version: str = REST_ALL
+    ) -> Sequence[GdsSdmxApi]:
+        """Get the list of SDMX API versions.
+
+        Args:
+            api_version: The version of the SDMX API to be returned.
+              Defaults to '*'.
+        """
+        query = super()._sdmx_api_q(api_version)
+        response = self.__fetch(query)
+        sdmx_api = super()._out(response, self.reader.sdmx_api)
+        return sdmx_api
+
+    def get_services(
+        self, service: str, resource: str = REST_ALL, version: str = REST_ALL
+    ) -> Sequence[GdsService]:
+        """Get the list of services for the supplied parameters.
+
+        Args:
+            service: The agency maintaining the service.
+            resource: The resource ID(s) to query. Defaults to '*'.
+            version: The version(s) of the resource. Defaults to '*'.
+
+        Returns:
+            A list of GdsService objects.
+        """
+        query = super()._services_q(service, resource, version)
+        response = self.__fetch(query)
+        services = super()._out(response, self.reader.services)
+        return services
+
+    def get_urn_resolver(self, urn: str) -> GdsUrnResolver:
+        """Resolve a URN to its corresponding resource.
+
+        Args:
+            urn: The URN to resolve.
+
+        Returns:
+            A GdsUrnResolver object with the resolved information.
+        """
+        query = super()._urn_resolver_q(urn)
+        response = self.__fetch(query)
+        urn_resolution = super()._out(response, self.reader.urn_resolver)
+        return urn_resolution
+
+
+class AsyncGdsClient(__BaseGdsClient):
+    """A client to be used to retrieve metadata from the GDS asynchronously."""
+
+    def __init__(
+        self,
+        api_endpoint: str = GDS_BASE_ENDPOINT,
+        pem: Optional[str] = None,
+    ) -> None:
+        """Instantiate a new client against the target endpoint.
+
+        Args:
+            api_endpoint: The endpoint of the targeted service.
+            pem: PEM file for unknown certificate authorities.
+        """
+        super().__init__(api_endpoint, pem)
+        self.__service = GdsAsyncRestService(
+            self.api_endpoint,
+            pem=pem,
+            timeout=10.0,
+        )
+
+    async def __fetch(self, query: GdsQuery) -> bytes:
+        """Fetch the requested metadata from the GDS service asynchronously."""
+        return await self.__service.gds(query)
+
+    async def get_agencies(self, agency: str) -> Sequence[Agency]:
+        """Get the list of agencies for the supplied name asynchronously.
+
+        Args:
+            agency: The agency maintaining the agency scheme.
+
+        Returns:
+            The requested list of agencies.
+        """
+        query = super(AsyncGdsClient, self)._agencies_q(agency)
+        out = await self.__fetch(query)
+        agencies = super(AsyncGdsClient, self)._out(out, self.reader.agencies)
+        return agencies
+
+    async def get_catalogs(
+        self,
+        catalog: str,
+        resource: str = REST_ALL,
+        version: str = REST_ALL,
+        resource_type: Optional[Literal["data", "metadata"]] = None,
+        message_format: Optional[Literal["json", "csv", "xml"]] = None,
+        api_version: Optional[str] = None,
+        detail: Optional[Literal["full", "raw"]] = None,
+        references: Optional[Literal["none", "children"]] = None,
+    ) -> Sequence[GdsCatalog]:
+        """Get the list of catalogs for the supplied params asynchronously."""
+        query = super()._catalogs_q(
+            catalog,
+            resource,
+            version,
+            resource_type,
+            message_format,
+            api_version,
+            detail,
+            references,
+        )
+        response = await self.__fetch(query)
+        catalogs = super()._out(response, self.reader.catalogs)
+        return catalogs
+
+    async def get_sdmx_apis(
+        self, api_version: str = REST_ALL
+    ) -> Sequence[GdsSdmxApi]:
+        """Get the list of SDMX API versions asynchronously."""
+        query = super()._sdmx_api_q(api_version)
+        response = await self.__fetch(query)
+        sdmx_api = super()._out(response, self.reader.sdmx_api)
+        return sdmx_api
+
+    async def get_services(
+        self, service: str, resource: str = REST_ALL, version: str = REST_ALL
+    ) -> Sequence[GdsService]:
+        """Get a list of services for the supplied params asynchronously."""
+        query = super()._services_q(service, resource, version)
+        response = await self.__fetch(query)
+        services = super()._out(response, self.reader.services)
+        return services
+
+    async def get_urn_resolver(self, urn: str) -> GdsUrnResolver:
+        """Resolve a URN to its corresponding resource asynchronously."""
+        query = super()._urn_resolver_q(urn)
+        response = await self.__fetch(query)
+        urn_resolution = super()._out(response, self.reader.urn_resolver)
+        return urn_resolution