pysdmx 1.0.0b3__tar.gz → 1.10.1__tar.gz

pysdmx-1.10.1/PKG-INFO

@@ -0,0 +1,123 @@
+Metadata-Version: 2.4
+Name: pysdmx
+Version: 1.10.1
+Summary: Your opinionated Python SDMX library
+License: Apache-2.0
+License-File: LICENSE
+Keywords: sdmx,data discovery,data retrieval,metadata,fmr
+Author: Xavier Sosnovsky
+Author-email: <xavier.sosnovsky@bis.org>
+Requires-Python: >=3.9
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Typing :: Typed
+Provides-Extra: all
+Provides-Extra: data
+Provides-Extra: dc
+Provides-Extra: json
+Provides-Extra: vtl
+Provides-Extra: xml
+Requires-Dist: httpx[http2] (>=0)
+Requires-Dist: jsonschema (>=4.10) ; extra == "json"
+Requires-Dist: lxml (>=5.2) ; extra == "all"
+Requires-Dist: lxml (>=5.2) ; extra == "xml"
+Requires-Dist: msgspec (>=0)
+Requires-Dist: pandas (>=2.1.4) ; extra == "all"
+Requires-Dist: pandas (>=2.1.4) ; extra == "data"
+Requires-Dist: parsy (>=2.1)
+Requires-Dist: python-dateutil (>=2.8.2) ; extra == "all"
+Requires-Dist: python-dateutil (>=2.8.2) ; extra == "dc"
+Requires-Dist: sdmxschemas (>=1.0.0) ; extra == "all"
+Requires-Dist: sdmxschemas (>=1.0.0) ; extra == "json"
+Requires-Dist: sdmxschemas (>=1.0.0) ; extra == "xml"
+Requires-Dist: vtlengine (>=1.2,<2.0) ; extra == "all"
+Requires-Dist: vtlengine (>=1.2,<2.0) ; extra == "vtl"
+Requires-Dist: xmltodict (>=0.13) ; extra == "all"
+Requires-Dist: xmltodict (>=0.13) ; extra == "xml"
+Project-URL: Bug Tracker, https://bis-med-it.github.io/pysdmx/issues
+Project-URL: Documentation, https://bis-med-it.github.io/pysdmx
+Project-URL: Homepage, https://sdmx.io/tools/pysdmx
+Project-URL: Repository, https://github.com/bis-med-it/pysdmx
+Description-Content-Type: text/x-rst
+
+.. |pypi badge| image:: https://img.shields.io/pypi/v/pysdmx.svg
+   :target: https://pypi.org/project/pysdmx/
+
+.. |awesome badge| image:: https://awesome.re/mentioned-badge.svg
+   :target: http://www.awesomeofficialstatistics.org
+
+|pypi badge| |awesome badge|
+
+``pysdmx`` in a nutshell
+************************
+
+What is pysdmx?
+===============
+
+``pysdmx`` is a **pragmatic** and **opinionated** SDMX library written in
+**Python**. It focuses on **simplicity**, providing a subset of SDMX functionalities
+without requiring advanced knowledge of SDMX. ``pysdmx`` is developed as part of
+the `sdmx.io <http://sdmx.io/>`_ project under the **BIS Open Tech initiative**.
+
+What does it do?
+================
+
+``pysdmx`` aspires to be a versatile SDMX toolbox for Python, covering various
+use cases. Here are some highlights:
+
+SDMX information model in Python
+--------------------------------
+
+``pysdmx`` offers Python classes representing a **simplified subset of the SDMX
+information model**, enabling a domain-driven development of SDMX processes in
+Python. The model classes support serialization in formats like JSON, YAML, or
+MessagePack. This functionality relies on the 
+`msgspec library <https://jcristharif.com/msgspec/>`_.
+
+Metadata in action
+------------------
+
+SDMX metadata are very useful for documenting statistical processes. For example,
+they can define the structure we expect for a data collection process and share
+it with the organizations providing data so that they know what to send.
+
+However, metadata can do so much more than that, i.e., they can be “active” and
+**drive various types of statistical processes**, such as generating the filesystem
+layout, creating the physical data model, validating data, mapping data, and
+configuring processes. To drive such processes, ``pysdmx`` supports retrieving
+metadata from an SDMX Registry or any service compliant with the SDMX-REST 2.0.0 (or
+above) API. Use these metadata to power your own statistical processes!
+
+Reading and writing SDMX files
+------------------------------
+
+``pysdmx`` supports reading and writing SDMX data and structure messages, in various
+formats, such as SDMX-CSV, SDMX-JSON, and SDMX-ML.
+
+Data discovery and data retrieval
+---------------------------------
+
+This functionality is under development. Once ready, ``pysdmx`` will allow:
+ 
+- **Listing public SDMX services**.
+- **Discovering data** available in these services.
+- **Retrieving data** from these services.
+ 
+This functionality is based on the **SDMX Global Discovery Service initiative**.
+
+Integration with the ecosystem
+------------------------------
+
+``pysdmx`` integrates nicely with other standards, like the `Validation and
+Transformation Language (VTL) <https://sdmx.org/about-sdmx/about-vtl/>`_,
+and major Python libraries like `Pandas <https://pandas.pydata.org/>`_.
+Take a look at the ``pysdmx`` toolkit module to learn more.
+
+``pysdmx`` is available on `PyPI <https://pypi.org/>`_ and can be
+installed using options such as pip, pipx, poetry, etc.
+
+For more details, check the `project documentation 
+<https://py.sdmx.io>`_.
+

pysdmx-1.10.1/README.rst

@@ -0,0 +1,78 @@
+.. |pypi badge| image:: https://img.shields.io/pypi/v/pysdmx.svg
+   :target: https://pypi.org/project/pysdmx/
+
+.. |awesome badge| image:: https://awesome.re/mentioned-badge.svg
+   :target: http://www.awesomeofficialstatistics.org
+
+|pypi badge| |awesome badge|
+
+``pysdmx`` in a nutshell
+************************
+
+What is pysdmx?
+===============
+
+``pysdmx`` is a **pragmatic** and **opinionated** SDMX library written in
+**Python**. It focuses on **simplicity**, providing a subset of SDMX functionalities
+without requiring advanced knowledge of SDMX. ``pysdmx`` is developed as part of
+the `sdmx.io <http://sdmx.io/>`_ project under the **BIS Open Tech initiative**.
+
+What does it do?
+================
+
+``pysdmx`` aspires to be a versatile SDMX toolbox for Python, covering various
+use cases. Here are some highlights:
+
+SDMX information model in Python
+--------------------------------
+
+``pysdmx`` offers Python classes representing a **simplified subset of the SDMX
+information model**, enabling a domain-driven development of SDMX processes in
+Python. The model classes support serialization in formats like JSON, YAML, or
+MessagePack. This functionality relies on the 
+`msgspec library <https://jcristharif.com/msgspec/>`_.
+
+Metadata in action
+------------------
+
+SDMX metadata are very useful for documenting statistical processes. For example,
+they can define the structure we expect for a data collection process and share
+it with the organizations providing data so that they know what to send.
+
+However, metadata can do so much more than that, i.e., they can be “active” and
+**drive various types of statistical processes**, such as generating the filesystem
+layout, creating the physical data model, validating data, mapping data, and
+configuring processes. To drive such processes, ``pysdmx`` supports retrieving
+metadata from an SDMX Registry or any service compliant with the SDMX-REST 2.0.0 (or
+above) API. Use these metadata to power your own statistical processes!
+
+Reading and writing SDMX files
+------------------------------
+
+``pysdmx`` supports reading and writing SDMX data and structure messages, in various
+formats, such as SDMX-CSV, SDMX-JSON, and SDMX-ML.
+
+Data discovery and data retrieval
+---------------------------------
+
+This functionality is under development. Once ready, ``pysdmx`` will allow:
+ 
+- **Listing public SDMX services**.
+- **Discovering data** available in these services.
+- **Retrieving data** from these services.
+ 
+This functionality is based on the **SDMX Global Discovery Service initiative**.
+
+Integration with the ecosystem
+------------------------------
+
+``pysdmx`` integrates nicely with other standards, like the `Validation and
+Transformation Language (VTL) <https://sdmx.org/about-sdmx/about-vtl/>`_,
+and major Python libraries like `Pandas <https://pandas.pydata.org/>`_.
+Take a look at the ``pysdmx`` toolkit module to learn more.
+
+``pysdmx`` is available on `PyPI <https://pypi.org/>`_ and can be
+installed using options such as pip, pipx, poetry, etc.
+
+For more details, check the `project documentation 
+<https://py.sdmx.io>`_.

pysdmx-1.10.1/pyproject.toml

@@ -0,0 +1,123 @@
+[project]
+name = "pysdmx"
+version = "1.10.1"
+description = "Your opinionated Python SDMX library"
+license = { text = "Apache-2.0" }
+readme = "README.rst"
+requires-python = ">=3.9"
+authors = [
+    { name = "Xavier Sosnovsky", email = "<xavier.sosnovsky@bis.org>" },
+    { name = "Stratos Nikoloutsos", email = "<stratos.nikoloutsos@bis.org>" },
+    { name = "Francisco Javier Hernandez del Caño", email = "<javier.hernandez@meaningfuldata.eu>" },
+    { name = "Mateo de Lorenzo Argelés", email = "<mateo.delorenzo@meaningfuldata.eu>" },
+    { name = "Jose Antonio Franco Martin", email = "<jose.franco@meaningfuldata.eu>" },
+    { name = "Mario Cortés Ruiz", email = "<mario.cortes@meaningfuldata.eu>" }
+
+]
+keywords = ["sdmx", "data discovery", "data retrieval", "metadata", "fmr"]
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: Apache Software License",
+    "Typing :: Typed"
+]
+dependencies = [
+    "httpx[http2]>=0.*",
+    "msgspec>=0.*",
+    "parsy>=2.1"
+]
+
+[project.urls]
+homepage = "https://sdmx.io/tools/pysdmx"
+repository = "https://github.com/bis-med-it/pysdmx"
+documentation = "https://bis-med-it.github.io/pysdmx"
+"Bug Tracker" = "https://bis-med-it.github.io/pysdmx/issues"
+
+[project.optional-dependencies]
+data = ["pandas>=2.1.4"]
+dc = ["python-dateutil>=2.8.2"]
+vtl = ["vtlengine>=1.2,<2.0"]
+json = ["sdmxschemas>=1.0.0", "jsonschema>=4.10"]
+xml = ["lxml>=5.2", "xmltodict>=0.13", "sdmxschemas>=1.0.0"]
+all = ["lxml>=5.2", "xmltodict>=0.13", "sdmxschemas>=1.0.0", "pandas>=2.1.4", "python-dateutil>=2.8.2", "vtlengine>=1.2,<2.0"]
+
+[tool.poetry]
+requires-poetry = ">=2.0"
+
+[tool.poetry.dependencies]
+python = ">=3.9,<4.0"
+
+[tool.poetry.group.dev.dependencies]
+ruff = ">=0.13.2"
+mypy = "^1.18.2"
+pytest = "^8.3.2"
+pytest-asyncio = "^0.21.1"
+pytest-cov = "^4.0.0"
+respx = "^0.22.0"
+pyroma = "^4.2"
+lxml-stubs = "^0.5.1"
+types-xmltodict = "^0.13.0.3"
+types-python-dateutil = "^2.9.0.20240316"
+pandas-stubs = "^2.1.4.231227"
+types-jsonschema = "^4.25.1.20251009"
+
+[tool.poetry.group.docs.dependencies]
+sphinx = "^7.2.6"
+sphinx-rtd-theme = "^1.3.0"
+sphinx-autodoc-typehints = "^1.24.0"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.ruff]
+line-length = 79
+lint.select = [
+    "ASYNC", # flake8-async: checks for async/await best practices
+    "B",     # flake8-bugbear: detect bugs and design problems
+    "C4",    # flake8-comprehensions: better list/dict/set comprehensions
+    "C90",   # mccabe: code complexity checker
+    "D",     # pydocstyle: enforce docstring conventions
+    "DTZ",   # flake8-datetimez: datetime best practices and timezone issues
+    "E",     # pycodestyle errors
+    "ERA",   # eradicate: Find commented-out code that should be removed
+    "F",     # pyflakes: detect various errors
+    "FURB",  # refurb: functional Python improvements and refactoring
+    "I",     # isort: import sorting
+    "LOG",   # flake8-logging: logging best practices
+    "PERF",  # perflint: performance optimizations
+    "PT",    # flake8-pytest-style: pytest best practices
+    "S",     # flake8-bandit: security issues
+    "SIM",   # flake8-simplify: code simplification suggestions
+    "W"      # pycodestyle warnings    
+]
+lint.ignore = ["D411", "E203", "F901"]
+lint.mccabe.max-complexity = 10
+lint.pydocstyle.convention = "google"
+
+[tool.ruff.lint.per-file-ignores]
+"tests/*" = ["C901", "DTZ", "D100", "D103", "D104", "PERF", "S101", "S311"]
+
+[tool.mypy]
+files = "src"
+disallow_untyped_defs = true
+disallow_untyped_calls = true
+ignore_errors = false
+no_implicit_optional = true
+show_column_numbers = true
+strict_equality = true
+strict_optional = true
+strict = true
+enable_error_code = [
+    "redundant-expr",
+    "truthy-bool",
+]
+warn_return_any = false
+
+[tool.coverage.run]
+branch = true
+omit = [
+    # We cannot test this properly with pytest as we always install all extras
+    "__extras_check.py"
+]

pysdmx-1.10.1/src/pysdmx/__extras_check.py

@@ -0,0 +1,71 @@
+EXTRAS_DOCS = "https://py.sdmx.io/start.html#how-can-i-get-it"
+ERROR_MESSAGE = (
+    "The '{extra_name}' extra is required to run {extra_desc}. "
+    "Please install it using 'pip install pysdmx[{extra_name}]' or "
+    "install all extras with 'pip install pysdmx[all]'. "
+    f"Check the documentation at: {EXTRAS_DOCS}"
+)
+
+
+def __check_dc_extra() -> None:
+    try:
+        import dateutil  # noqa: F401
+    except ImportError:
+        raise ImportError(
+            ERROR_MESSAGE.format(
+                extra_name="dc", extra_desc="the python parser for dates"
+            )
+        ) from None
+
+
+def __check_data_extra() -> None:
+    try:
+        import pandas  # noqa: F401
+    except ImportError:
+        raise ImportError(
+            ERROR_MESSAGE.format(
+                extra_name="data",
+                extra_desc="the reading and writing of Data Messages",
+            )
+        ) from None
+
+
+def __check_xml_extra() -> None:
+    try:
+        import lxml  # noqa: F401
+        import sdmxschemas  # noqa: F401
+        import xmltodict  # noqa: F401
+    except ImportError:
+        raise ImportError(
+            ERROR_MESSAGE.format(
+                extra_name="xml",
+                extra_desc="the reading and writing of SDMX-ML Messages",
+            )
+        ) from None
+
+
+def __check_vtl_extra() -> None:
+    try:
+        import vtlengine  # noqa: F401
+    except ImportError:
+        raise ImportError(
+            ERROR_MESSAGE.format(
+                extra_name="vtl",
+                extra_desc="VTL Scripts, SDMX-VTL model validations"
+                " and prettify",
+            )
+        ) from None
+
+
+def __check_json_extra() -> None:
+    try:
+        import jsonschema  # noqa: F401
+        import sdmxschemas  # noqa: F401
+    except ImportError:
+        raise ImportError(
+            ERROR_MESSAGE.format(
+                extra_name="json",
+                extra_desc="the validation of SDMX-JSON Structure Messages "
+                "(hint, use validate=False if you don't need validation)",
+            )
+        ) from None

{pysdmx-1.0.0b3 → pysdmx-1.10.1}/src/pysdmx/__init__.py

@@ -1,3 +1,3 @@
 """Your opinionated Python SDMX library."""
 
-__version__ = "1.0.0-beta-3"
+__version__ = "1.10.1"

pysdmx-1.10.1/src/pysdmx/api/__init__.py

@@ -0,0 +1 @@
+"""pysdmx api package."""

pysdmx-1.10.1/src/pysdmx/api/dc/__init__.py

@@ -0,0 +1,5 @@
+"""pysdmx simple data discovery and retrieval API."""
+
+from pysdmx.api.dc._api import Connector
+
+__all__ = ["Connector"]

pysdmx-1.10.1/src/pysdmx/api/dc/_api.py

@@ -0,0 +1,213 @@
+"""API to be implemented by connectors."""
+
+from datetime import datetime
+from typing import (
+    Any,
+    Generator,
+    Iterable,
+    List,
+    Literal,
+    Optional,
+    Protocol,
+    Union,
+    runtime_checkable,
+)
+
+from pysdmx.api.dc.query import (
+    BooleanFilter,
+    DateTimeFilter,
+    MultiFilter,
+    NotFilter,
+    NumberFilter,
+    SortBy,
+    TextFilter,
+)
+from pysdmx.model import (
+    Dataflow,
+    DataflowInfo,
+    DataflowRef,
+    Organisation,
+    SeriesInfo,
+)
+
+
+@runtime_checkable
+class Connector(Protocol):
+    """A Connector provides access to a data source.
+
+    Connectors support both data discovery and data retrieval.
+
+    Attributes:
+        owner: The organisation providing the Connector.
+            It is recommended to provide contact details, in case users
+            of the Connector have questions about it.
+    """
+
+    owner: Organisation
+
+    def dataflows(
+        self,
+        filter_query: Optional[str] = None,
+        provider: Optional[str] = None,
+    ) -> Union[Iterable[Dataflow], Iterable[DataflowRef]]:
+        """Get the dataflows provided by the Connector.
+
+        Args:
+            filter_query: A search term. If set, any dataflow containing the
+                term in its id, name or description will be returned.
+            provider: A data provider. If set, any dataflow for which data are
+                being provided by the supplied provider will be returned.
+
+        Returns: Iterable[DataflowRef]: A collection of dataflow references.
+
+                The references contain core information about the
+                dataflows provided by the Connector. Most importantly,
+                they contain the identifying information required to
+                retrieve more information about a specific dataflow
+                (see dataflow).
+
+                It is expected that this method, if implemented,
+                will return at least one DataflowRef object.
+        """
+
+    def providers(
+        self,
+        filter_query: Optional[str] = None,
+    ) -> Iterable[Organisation]:
+        """Get the providers of which data are available in the Connector.
+
+        Args:
+            filter_query: A search term. If set, any provider containing the
+                term in its id, name or description will be returned.
+
+        Returns: Iterable[Organisation]: A collection of data providers.
+        """
+
+    def dataflow(
+        self,
+        dataflow: Union[str, DataflowRef, Dataflow],
+        metrics: bool = False,
+    ) -> DataflowInfo:
+        """Get information about a dataflow.
+
+        Args:
+            dataflow: Either a string representing the dataflow ID or
+                a DataflowRef, as returned by calling the dataflows
+                function.
+            metrics: Metrics contain useful information such as
+                the number of observations, series, when the
+                dataflow was last updated, etc. However, it may be
+                expensive to fetch this information. In case metrics
+                are not required, you may set metrics to False, to
+                prevent them from being computed.
+
+        Returns: DataflowInfo: Information about the requested dataflow.
+
+            The information includes:
+
+            - Some basic metadata about the dataflow (such as its ID and name).
+            - Some useful metrics such as the number of observations.
+            - The expected structure of data (i.e. the data schema), including
+              the expected columns, their types, etc.
+        """
+
+    def series(
+        self,
+        dataflow: Union[str, DataflowRef, Dataflow, DataflowInfo],
+        provider: Optional[Union[str, Organisation]] = None,
+        filters: Optional[
+            Union[
+                BooleanFilter,
+                DateTimeFilter,
+                MultiFilter,
+                NotFilter,
+                NumberFilter,
+                str,
+                TextFilter,
+            ]
+        ] = None,
+        discontinued: bool = False,
+        updated_after: Optional[datetime] = None,
+    ) -> Generator[SeriesInfo, None, None]:
+        """Get the series available in the dataflow.
+
+        Args:
+            dataflow: Either a string representing the dataflow ID or
+                a DataflowRef, as returned by calling the dataflows
+                function, or a DataflowInfo, as returned by calling the
+                dataflow function.
+            provider: The organisation providing the data to be returned.
+            filters: A dictionary, where the keys are the IDs of the
+                components used for filtering and the values
+                represent the filters to be applied.
+            discontinued: Whether discontinued (i.e. old)
+                series should be returned. Defaults to `False`.
+            updated_after: Retrieve the series updated after
+                the supplied timestamp.
+
+        Returns: A generator, to iterate over the collection matching series
+            The information includes:
+
+            - Some basic metadata about the series (such as an ID or name).
+            - Some useful metrics such as the number of observations, the last
+              time it was updated, etc.
+        """
+
+    def data(
+        self,
+        dataflow: Union[str, DataflowRef, Dataflow, DataflowInfo],
+        provider: Optional[Union[str, Organisation]] = None,
+        series: Optional[Iterable[str]] = None,
+        filters: Optional[
+            Union[
+                BooleanFilter,
+                DateTimeFilter,
+                MultiFilter,
+                NotFilter,
+                NumberFilter,
+                str,
+                TextFilter,
+            ]
+        ] = None,
+        columns: Optional[Iterable[str]] = None,
+        sort: Optional[List[SortBy]] = None,
+        offset: int = 0,
+        limit: Optional[int] = None,
+        history: bool = False,
+        updated_after: Optional[datetime] = None,
+        format: Literal["dict", "pandas"] = "dict",
+    ) -> Any:
+        """Get data for the selected dataflow.
+
+        Args:
+            dataflow: Either a string representing the dataflow ID or
+                a DataflowRef, as returned by calling the dataflows
+                function, or a DataflowInfo, as returned by calling the
+                dataflow function.
+            provider: The organisation providing the data to be returned.
+            series: One or more strings identifying the series
+                for which data should be returned (e.g. D.USD.EUR.SP00).
+                wildcards can be used too (e.g. D.*.EUR.SP00).
+            filters: A dictionary, where the keys are the IDs of the
+                components used for filtering and the values
+                represent the filters to be applied.
+            columns: The fields to be returned.
+            sort: The desired sorting order of the results.
+            offset: From which row to start fetching. This is useful to
+                paginate results. Default to 0.
+            limit: The number of rows to be returned.
+            history: Whether to retrieve previous "versions" of the data or
+                only the latest one. This is useful to see how the data may
+                have evolved or have been corrected over time.
+            updated_after: Retrieve the series updated after
+                the supplied timestamp.
+            format: The desired format for the data. By default, a generator
+                of dict objects will be returned, but there are other options
+                such as pandas data frames.
+
+        Returns: A generator, to iterate over the matching data, if format is
+            set to dict (the default), or a pandas data frame.
+        """
+
+
+__all__ = ["Connector"]

pysdmx-1.10.1/src/pysdmx/api/dc/query/__init__.py

@@ -0,0 +1,27 @@
+"""Classes and parsers for data queries."""
+
+from pysdmx.api.dc.query._model import (
+    BooleanFilter,
+    DateTimeFilter,
+    LogicalOperator,
+    MultiFilter,
+    NotFilter,
+    NullFilter,
+    NumberFilter,
+    Operator,
+    SortBy,
+    TextFilter,
+)
+
+__all__ = [
+    "BooleanFilter",
+    "DateTimeFilter",
+    "LogicalOperator",
+    "MultiFilter",
+    "NotFilter",
+    "NullFilter",
+    "NumberFilter",
+    "Operator",
+    "SortBy",
+    "TextFilter",
+]