eba-xbridge 1.5.0rc3__py3-none-any.whl → 1.5.0rc5__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- eba_xbridge-1.5.0rc5.dist-info/METADATA +308 -0
- {eba_xbridge-1.5.0rc3.dist-info → eba_xbridge-1.5.0rc5.dist-info}/RECORD +10 -8
- eba_xbridge-1.5.0rc5.dist-info/entry_points.txt +3 -0
- xbridge/__init__.py +1 -1
- xbridge/__main__.py +82 -0
- xbridge/converter.py +17 -30
- xbridge/instance.py +76 -1
- xbridge/modules.py +6 -4
- eba_xbridge-1.5.0rc3.dist-info/METADATA +0 -62
- {eba_xbridge-1.5.0rc3.dist-info → eba_xbridge-1.5.0rc5.dist-info}/WHEEL +0 -0
- {eba_xbridge-1.5.0rc3.dist-info → eba_xbridge-1.5.0rc5.dist-info}/licenses/LICENSE +0 -0
|
@@ -0,0 +1,308 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: eba-xbridge
|
|
3
|
+
Version: 1.5.0rc5
|
|
4
|
+
Summary: XBRL-XML to XBRL-CSV converter for EBA Taxonomy (version 4.1)
|
|
5
|
+
License: Apache 2.0
|
|
6
|
+
License-File: LICENSE
|
|
7
|
+
Keywords: xbrl,eba,taxonomy,csv,xml
|
|
8
|
+
Author: MeaningfulData
|
|
9
|
+
Author-email: info@meaningfuldata.eu
|
|
10
|
+
Maintainer: Antonio Olleros
|
|
11
|
+
Maintainer-email: antonio.olleros@meaningfuldata.eu
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Intended Audience :: Information Technology
|
|
16
|
+
Classifier: Intended Audience :: Science/Research
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Typing :: Typed
|
|
19
|
+
Requires-Dist: lxml (>=5.2.1,<6.0)
|
|
20
|
+
Requires-Dist: numpy (>=1.23.2,<2) ; python_version < "3.13"
|
|
21
|
+
Requires-Dist: numpy (>=2.1.0) ; python_version >= "3.13"
|
|
22
|
+
Requires-Dist: pandas (>=2.1.4,<3.0)
|
|
23
|
+
Project-URL: Documentation, https://docs.xbridge.meaningfuldata.eu
|
|
24
|
+
Project-URL: IssueTracker, https://github.com/Meaningful-Data/xbridge/issues
|
|
25
|
+
Project-URL: MeaningfulData, https://www.meaningfuldata.eu/
|
|
26
|
+
Project-URL: Repository, https://github.com/Meaningful-Data/xbridge
|
|
27
|
+
Description-Content-Type: text/x-rst
|
|
28
|
+
|
|
29
|
+
XBridge (eba-xbridge)
|
|
30
|
+
#####################
|
|
31
|
+
|
|
32
|
+
.. image:: https://img.shields.io/pypi/v/eba-xbridge.svg
|
|
33
|
+
:target: https://pypi.org/project/eba-xbridge/
|
|
34
|
+
:alt: PyPI version
|
|
35
|
+
|
|
36
|
+
.. image:: https://img.shields.io/pypi/pyversions/eba-xbridge.svg
|
|
37
|
+
:target: https://pypi.org/project/eba-xbridge/
|
|
38
|
+
:alt: Python versions
|
|
39
|
+
|
|
40
|
+
.. image:: https://img.shields.io/github/license/Meaningful-Data/xbridge.svg
|
|
41
|
+
:target: https://github.com/Meaningful-Data/xbridge/blob/main/LICENSE
|
|
42
|
+
:alt: License
|
|
43
|
+
|
|
44
|
+
.. image:: https://img.shields.io/github/actions/workflow/status/Meaningful-Data/xbridge/testing.yml?branch=main
|
|
45
|
+
:target: https://github.com/Meaningful-Data/xbridge/actions
|
|
46
|
+
:alt: Build status
|
|
47
|
+
|
|
48
|
+
Overview
|
|
49
|
+
========
|
|
50
|
+
|
|
51
|
+
XBridge is a Python library for converting XBRL-XML files into XBRL-CSV files using the EBA (European Banking Authority) taxonomy. It provides a simple, reliable way to transform regulatory reporting data from XML format to CSV format.
|
|
52
|
+
|
|
53
|
+
The library currently supports **EBA Taxonomy version 4.2** and includes support for DORA (Digital Operational Resilience Act) CSV conversion. The library must be updated with each new EBA taxonomy version release.
|
|
54
|
+
|
|
55
|
+
Key Features
|
|
56
|
+
============
|
|
57
|
+
|
|
58
|
+
* **XBRL-XML to XBRL-CSV Conversion**: Seamlessly convert XBRL-XML instance files to XBRL-CSV format
|
|
59
|
+
* **Command-Line Interface**: Quick conversions without writing code using the ``xbridge`` CLI
|
|
60
|
+
* **Python API**: Programmatic conversion for integration with other tools and workflows
|
|
61
|
+
* **EBA Taxonomy 4.2 Support**: Built for the latest EBA taxonomy specification
|
|
62
|
+
* **DORA CSV Conversion**: Support for Digital Operational Resilience Act reporting
|
|
63
|
+
* **Configurable Validation**: Flexible filing indicator validation with strict or warning modes
|
|
64
|
+
* **Decimal Handling**: Intelligent decimal precision handling with configurable options
|
|
65
|
+
* **Type Safety**: Fully typed codebase with MyPy strict mode compliance
|
|
66
|
+
* **Python 3.9+**: Supports Python 3.9 through 3.13
|
|
67
|
+
|
|
68
|
+
Prerequisites
|
|
69
|
+
=============
|
|
70
|
+
|
|
71
|
+
* **Python**: 3.9 or higher
|
|
72
|
+
* **7z Command-Line Tool**: Required for loading compressed taxonomy files (7z or ZIP format)
|
|
73
|
+
|
|
74
|
+
* On Ubuntu/Debian: ``sudo apt-get install p7zip-full``
|
|
75
|
+
* On macOS: ``brew install p7zip``
|
|
76
|
+
* On Windows: Download from `7-zip.org <https://www.7-zip.org/>`_
|
|
77
|
+
|
|
78
|
+
Installation
|
|
79
|
+
============
|
|
80
|
+
|
|
81
|
+
Install XBridge from PyPI using pip:
|
|
82
|
+
|
|
83
|
+
.. code-block:: bash
|
|
84
|
+
|
|
85
|
+
pip install eba-xbridge
|
|
86
|
+
|
|
87
|
+
For development installation, see `CONTRIBUTING.md <CONTRIBUTING.md>`_.
|
|
88
|
+
|
|
89
|
+
Quick Start
|
|
90
|
+
===========
|
|
91
|
+
|
|
92
|
+
XBridge offers two ways to convert XBRL-XML files to XBRL-CSV: a command-line interface (CLI) for quick conversions, and a Python API for programmatic use.
|
|
93
|
+
|
|
94
|
+
Command-Line Interface
|
|
95
|
+
----------------------
|
|
96
|
+
|
|
97
|
+
The CLI provides a quick way to convert files without writing code:
|
|
98
|
+
|
|
99
|
+
.. code-block:: bash
|
|
100
|
+
|
|
101
|
+
# Basic conversion (output to same directory as input)
|
|
102
|
+
xbridge instance.xbrl
|
|
103
|
+
|
|
104
|
+
# Specify output directory
|
|
105
|
+
xbridge instance.xbrl --output-path ./output
|
|
106
|
+
|
|
107
|
+
# Continue with warnings instead of errors
|
|
108
|
+
xbridge instance.xbrl --no-strict-validation
|
|
109
|
+
|
|
110
|
+
# Include headers as datapoints
|
|
111
|
+
xbridge instance.xbrl --headers-as-datapoints
|
|
112
|
+
|
|
113
|
+
**CLI Options:**
|
|
114
|
+
|
|
115
|
+
* ``--output-path PATH``: Output directory (default: same as input file)
|
|
116
|
+
* ``--headers-as-datapoints``: Treat headers as datapoints (default: False)
|
|
117
|
+
* ``--strict-validation``: Raise errors on validation failures (default: True)
|
|
118
|
+
* ``--no-strict-validation``: Emit warnings instead of errors
|
|
119
|
+
|
|
120
|
+
For more CLI options, run ``xbridge --help``.
|
|
121
|
+
|
|
122
|
+
Python API - Basic Conversion
|
|
123
|
+
------------------------------
|
|
124
|
+
|
|
125
|
+
Convert an XBRL-XML instance file to XBRL-CSV using the Python API:
|
|
126
|
+
|
|
127
|
+
.. code-block:: python
|
|
128
|
+
|
|
129
|
+
from xbridge.api import convert_instance
|
|
130
|
+
|
|
131
|
+
# Basic conversion
|
|
132
|
+
input_path = "path/to/instance.xbrl"
|
|
133
|
+
output_path = "path/to/output"
|
|
134
|
+
|
|
135
|
+
convert_instance(input_path, output_path)
|
|
136
|
+
|
|
137
|
+
The converted XBRL-CSV files will be saved as a ZIP archive in the output directory.
|
|
138
|
+
|
|
139
|
+
Python API - Advanced Usage
|
|
140
|
+
----------------------------
|
|
141
|
+
|
|
142
|
+
Customize the conversion with additional parameters:
|
|
143
|
+
|
|
144
|
+
.. code-block:: python
|
|
145
|
+
|
|
146
|
+
from xbridge.api import convert_instance
|
|
147
|
+
|
|
148
|
+
# Conversion with custom options
|
|
149
|
+
convert_instance(
|
|
150
|
+
instance_path="path/to/instance.xbrl",
|
|
151
|
+
output_path="path/to/output",
|
|
152
|
+
headers_as_datapoints=True, # Treat headers as datapoints
|
|
153
|
+
validate_filing_indicators=True, # Validate filing indicators
|
|
154
|
+
strict_validation=False, # Emit warnings instead of errors for orphaned facts
|
|
155
|
+
)
|
|
156
|
+
|
|
157
|
+
Loading an Instance
|
|
158
|
+
-------------------
|
|
159
|
+
|
|
160
|
+
Load and inspect an XBRL-XML instance without converting:
|
|
161
|
+
|
|
162
|
+
.. code-block:: python
|
|
163
|
+
|
|
164
|
+
from xbridge.api import load_instance
|
|
165
|
+
|
|
166
|
+
instance = load_instance("path/to/instance.xbrl")
|
|
167
|
+
|
|
168
|
+
# Access instance properties
|
|
169
|
+
print(f"Entity: {instance.entity}")
|
|
170
|
+
print(f"Period: {instance.period}")
|
|
171
|
+
print(f"Facts count: {len(instance.facts)}")
|
|
172
|
+
|
|
173
|
+
How XBridge Works
|
|
174
|
+
=================
|
|
175
|
+
|
|
176
|
+
XBridge performs the conversion in several steps:
|
|
177
|
+
|
|
178
|
+
1. **Load the XBRL-XML instance**: Parse and extract facts, contexts, scenarios, and filing indicators
|
|
179
|
+
2. **Load the EBA taxonomy**: Access pre-processed taxonomy modules containing tables and variables
|
|
180
|
+
3. **Match and validate**: Join instance facts with taxonomy definitions
|
|
181
|
+
4. **Generate CSV files**: Create XBRL-CSV files including:
|
|
182
|
+
|
|
183
|
+
* Data tables with facts and dimensions
|
|
184
|
+
* Filing indicators showing reported tables
|
|
185
|
+
* Parameters (entity, period, base currency, decimals)
|
|
186
|
+
|
|
187
|
+
5. **Package output**: Bundle all CSV files into a ZIP archive
|
|
188
|
+
|
|
189
|
+
Output Structure
|
|
190
|
+
----------------
|
|
191
|
+
|
|
192
|
+
The output ZIP file contains:
|
|
193
|
+
|
|
194
|
+
* **META-INF/**: JSON report package metadata
|
|
195
|
+
* **reports/**: CSV files for each reported table
|
|
196
|
+
* **filing-indicators.csv**: Table reporting indicators
|
|
197
|
+
* **parameters.csv**: Report-level parameters
|
|
198
|
+
|
|
199
|
+
Documentation
|
|
200
|
+
=============
|
|
201
|
+
|
|
202
|
+
Comprehensive documentation is available at `docs.xbridge.meaningfuldata.eu <https://docs.xbridge.meaningfuldata.eu>`_.
|
|
203
|
+
|
|
204
|
+
The documentation includes:
|
|
205
|
+
|
|
206
|
+
* **API Reference**: Complete API documentation
|
|
207
|
+
* **Quickstart Guide**: Step-by-step tutorials
|
|
208
|
+
* **Technical Notes**: Architecture and design details
|
|
209
|
+
* **FAQ**: Frequently asked questions
|
|
210
|
+
|
|
211
|
+
Taxonomy Loading
|
|
212
|
+
================
|
|
213
|
+
|
|
214
|
+
If you need to work with the EBA taxonomy directly, you can load it using:
|
|
215
|
+
|
|
216
|
+
.. code-block:: bash
|
|
217
|
+
|
|
218
|
+
python -m xbridge.taxonomy_loader --input_path path/to/FullTaxonomy.7z
|
|
219
|
+
|
|
220
|
+
This generates an ``index.json`` file containing module references and pre-processed taxonomy data.
|
|
221
|
+
|
|
222
|
+
.. warning::
|
|
223
|
+
Loading the taxonomy from a 7z package may take several minutes. Ensure the ``7z`` command is available on your system.
|
|
224
|
+
|
|
225
|
+
Configuration Options
|
|
226
|
+
=====================
|
|
227
|
+
|
|
228
|
+
convert_instance Parameters
|
|
229
|
+
----------------------------
|
|
230
|
+
|
|
231
|
+
* **instance_path** (str | Path): Path to the XBRL-XML instance file
|
|
232
|
+
* **output_path** (str | Path | None): Output directory for CSV files (default: current directory)
|
|
233
|
+
* **headers_as_datapoints** (bool): Treat table headers as datapoints (default: False)
|
|
234
|
+
* **validate_filing_indicators** (bool): Validate that facts belong to reported tables (default: True)
|
|
235
|
+
* **strict_validation** (bool): Raise errors on validation failures; if False, emit warnings (default: True)
|
|
236
|
+
|
|
237
|
+
Troubleshooting
|
|
238
|
+
===============
|
|
239
|
+
|
|
240
|
+
Common Issues
|
|
241
|
+
-------------
|
|
242
|
+
|
|
243
|
+
**7z command not found**
|
|
244
|
+
Install the 7z command-line tool using your system's package manager (see Prerequisites).
|
|
245
|
+
|
|
246
|
+
**Taxonomy version mismatch**
|
|
247
|
+
Ensure you're using the correct version of XBridge for your taxonomy version. XBridge 1.5.x supports EBA Taxonomy 4.1.
|
|
248
|
+
|
|
249
|
+
**Orphaned facts warning/error**
|
|
250
|
+
Facts that don't belong to any reported table. Set ``strict_validation=False`` to continue with warnings instead of errors.
|
|
251
|
+
|
|
252
|
+
**Decimal precision issues**
|
|
253
|
+
XBridge automatically handles decimal precision from the taxonomy. Check the parameters.csv file for applied decimal settings.
|
|
254
|
+
|
|
255
|
+
For more issues, see our `FAQ <https://docs.xbridge.meaningfuldata.eu/faq.html>`_ or `open an issue <https://github.com/Meaningful-Data/xbridge/issues>`_.
|
|
256
|
+
|
|
257
|
+
Contributing
|
|
258
|
+
============
|
|
259
|
+
|
|
260
|
+
We welcome contributions! Please see `CONTRIBUTING.md <CONTRIBUTING.md>`_ for:
|
|
261
|
+
|
|
262
|
+
* Development setup instructions
|
|
263
|
+
* Code style guidelines
|
|
264
|
+
* Testing requirements
|
|
265
|
+
* Pull request process
|
|
266
|
+
|
|
267
|
+
Before contributing, please read our `Code of Conduct <CODE_OF_CONDUCT.md>`_.
|
|
268
|
+
|
|
269
|
+
Changelog
|
|
270
|
+
=========
|
|
271
|
+
|
|
272
|
+
See `CHANGELOG.md <CHANGELOG.md>`_ for a detailed history of changes.
|
|
273
|
+
|
|
274
|
+
Support
|
|
275
|
+
=======
|
|
276
|
+
|
|
277
|
+
* **Documentation**: https://docs.xbridge.meaningfuldata.eu
|
|
278
|
+
* **Issue Tracker**: https://github.com/Meaningful-Data/xbridge/issues
|
|
279
|
+
* **Email**: info@meaningfuldata.eu
|
|
280
|
+
* **Company**: https://www.meaningfuldata.eu/
|
|
281
|
+
|
|
282
|
+
Security
|
|
283
|
+
========
|
|
284
|
+
|
|
285
|
+
For security issues, please see our `Security Policy <SECURITY.md>`_.
|
|
286
|
+
|
|
287
|
+
License
|
|
288
|
+
=======
|
|
289
|
+
|
|
290
|
+
This project is licensed under the Apache License 2.0 - see the `LICENSE <LICENSE>`_ file for details.
|
|
291
|
+
|
|
292
|
+
Authors & Maintainers
|
|
293
|
+
=====================
|
|
294
|
+
|
|
295
|
+
**MeaningfulData** - https://www.meaningfuldata.eu/
|
|
296
|
+
|
|
297
|
+
Maintainers:
|
|
298
|
+
|
|
299
|
+
* Antonio Olleros (antonio.olleros@meaningfuldata.eu)
|
|
300
|
+
* Jesus Simon (jesus.simon@meaningfuldata.eu)
|
|
301
|
+
* Francisco Javier Hernandez del Caño (javier.hernandez@meaningfuldata.eu)
|
|
302
|
+
* Guillermo Garcia Martin (guillermo.garcia@meaningfuldata.eu)
|
|
303
|
+
|
|
304
|
+
Acknowledgments
|
|
305
|
+
===============
|
|
306
|
+
|
|
307
|
+
This project is designed to work with the European Banking Authority (EBA) taxonomy for regulatory reporting
|
|
308
|
+
|
|
@@ -1,7 +1,8 @@
|
|
|
1
|
-
xbridge/__init__.py,sha256=
|
|
1
|
+
xbridge/__init__.py,sha256=S6yAyrnTJ19Trbh-OrGVDIwi5lh88C2pCxDncQyX1VY,68
|
|
2
|
+
xbridge/__main__.py,sha256=trtFEv7TRJgrLL84leIapPvgC_iVTj05qLHRRS1Olts,2219
|
|
2
3
|
xbridge/api.py,sha256=NCBz7VRJWE3gID6ndgL4Awoxw0w1yMIIf_OTLRuZyyQ,1559
|
|
3
|
-
xbridge/converter.py,sha256=
|
|
4
|
-
xbridge/instance.py,sha256=
|
|
4
|
+
xbridge/converter.py,sha256=vWlIaqQ2eA8Y5b1jqnqTIcOxeubJEu1itggMrFxYnS8,24077
|
|
5
|
+
xbridge/instance.py,sha256=GA5D2p3IobCp5W4-x1dibgOQFuy_W4R8LmZnQ4H900g,32292
|
|
5
6
|
xbridge/modules/ae_ae_4.2.json,sha256=AdFvwZqX0KVP3jF1iHeQc5QSnSMvvT3GvoA2G1AgXis,460165
|
|
6
7
|
xbridge/modules/ae_con_cir-680-2014_2017-04-04.json,sha256=4n0t9dKJNU8Nb5QHpssrDs8ZLwzI-Mw75ax-ar9pLu0,363273
|
|
7
8
|
xbridge/modules/ae_con_cir-680-2014_2018-03-31.json,sha256=aVWeLLs20p39kQQUthUzqrxBGKTycqhgX9WLk1rVlNw,363538
|
|
@@ -379,10 +380,11 @@ xbridge/modules/sbpimv_ind_its-2016-svbxx_2016-02-01.json,sha256=SED-dW--UKxhHNY
|
|
|
379
380
|
xbridge/modules/sbpimv_sbp_4.2.json,sha256=Bj4z7zofZngG9EJ7-q74F-JF41O1FK_mX8RTfYdLP9I,7023
|
|
380
381
|
xbridge/modules/sepa_ipr_pay_4.1.json,sha256=awsJeBUDhMIFs5so6CWUQmlcHSDcGMd8fnLy_r_iMik,27054
|
|
381
382
|
xbridge/modules/sepa_ipr_pay_4.2.json,sha256=JLJvR02LOAJy6SWPRuhV1TT02oXQhsG83FBn176KWsA,27742
|
|
382
|
-
xbridge/modules.py,sha256=
|
|
383
|
+
xbridge/modules.py,sha256=bTvBXtp3w4Gad2DpEQE7Hb-UfuUQLlRl8gywRstQtpU,22399
|
|
383
384
|
xbridge/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
384
385
|
xbridge/taxonomy_loader.py,sha256=K0lnJVryvkKsaoK3fMis-L2JpmwLO6z3Ruq3yj9FxDY,9317
|
|
385
|
-
eba_xbridge-1.5.
|
|
386
|
-
eba_xbridge-1.5.
|
|
387
|
-
eba_xbridge-1.5.
|
|
388
|
-
eba_xbridge-1.5.
|
|
386
|
+
eba_xbridge-1.5.0rc5.dist-info/METADATA,sha256=88P20ZbVDK8cMGG0-rs1W7OihOdBrmOxujMvoeB3RQM,10430
|
|
387
|
+
eba_xbridge-1.5.0rc5.dist-info/WHEEL,sha256=zp0Cn7JsFoX2ATtOhtaFYIiE2rmFAD4OcMhtUki8W3U,88
|
|
388
|
+
eba_xbridge-1.5.0rc5.dist-info/entry_points.txt,sha256=FATct4icSewM04cegjhybtm7xcQWhaSahL-DTtuFdZw,49
|
|
389
|
+
eba_xbridge-1.5.0rc5.dist-info/licenses/LICENSE,sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ,11357
|
|
390
|
+
eba_xbridge-1.5.0rc5.dist-info/RECORD,,
|
xbridge/__init__.py
CHANGED
xbridge/__main__.py
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
"""Command-line interface for xbridge."""
|
|
2
|
+
|
|
3
|
+
import argparse
|
|
4
|
+
import sys
|
|
5
|
+
from pathlib import Path
|
|
6
|
+
|
|
7
|
+
from xbridge.api import convert_instance
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
def main() -> None:
|
|
11
|
+
"""Main CLI entry point for xbridge converter."""
|
|
12
|
+
parser = argparse.ArgumentParser(
|
|
13
|
+
description="Convert XBRL-XML instances to XBRL-CSV format",
|
|
14
|
+
prog="xbridge",
|
|
15
|
+
)
|
|
16
|
+
|
|
17
|
+
parser.add_argument(
|
|
18
|
+
"input_file",
|
|
19
|
+
type=str,
|
|
20
|
+
help="Path to the input XBRL-XML file",
|
|
21
|
+
)
|
|
22
|
+
|
|
23
|
+
parser.add_argument(
|
|
24
|
+
"--output-path",
|
|
25
|
+
type=str,
|
|
26
|
+
default=None,
|
|
27
|
+
help="Output directory path (default: same folder as input file)",
|
|
28
|
+
)
|
|
29
|
+
|
|
30
|
+
parser.add_argument(
|
|
31
|
+
"--headers-as-datapoints",
|
|
32
|
+
action="store_true",
|
|
33
|
+
default=False,
|
|
34
|
+
help="Treat headers as datapoints (default: False)",
|
|
35
|
+
)
|
|
36
|
+
|
|
37
|
+
parser.add_argument(
|
|
38
|
+
"--strict-validation",
|
|
39
|
+
action="store_true",
|
|
40
|
+
default=True,
|
|
41
|
+
help="Raise errors on validation failures (default: True)",
|
|
42
|
+
)
|
|
43
|
+
|
|
44
|
+
parser.add_argument(
|
|
45
|
+
"--no-strict-validation",
|
|
46
|
+
action="store_false",
|
|
47
|
+
dest="strict_validation",
|
|
48
|
+
help="Emit warnings instead of errors for validation failures",
|
|
49
|
+
)
|
|
50
|
+
|
|
51
|
+
args = parser.parse_args()
|
|
52
|
+
|
|
53
|
+
# Determine output path
|
|
54
|
+
input_path = Path(args.input_file)
|
|
55
|
+
if not input_path.exists():
|
|
56
|
+
print(f"Error: Input file not found: {args.input_file}", file=sys.stderr)
|
|
57
|
+
sys.exit(1)
|
|
58
|
+
|
|
59
|
+
if args.output_path is None:
|
|
60
|
+
output_path = input_path.parent
|
|
61
|
+
else:
|
|
62
|
+
output_path = Path(args.output_path)
|
|
63
|
+
if not output_path.exists():
|
|
64
|
+
print(f"Error: Output path does not exist: {args.output_path}", file=sys.stderr)
|
|
65
|
+
sys.exit(1)
|
|
66
|
+
|
|
67
|
+
try:
|
|
68
|
+
result_path = convert_instance(
|
|
69
|
+
instance_path=input_path,
|
|
70
|
+
output_path=output_path,
|
|
71
|
+
headers_as_datapoints=args.headers_as_datapoints,
|
|
72
|
+
validate_filing_indicators=True,
|
|
73
|
+
strict_validation=args.strict_validation,
|
|
74
|
+
)
|
|
75
|
+
print(f"Conversion successful: {result_path}")
|
|
76
|
+
except Exception as e:
|
|
77
|
+
print(f"Conversion failed: {e}", file=sys.stderr)
|
|
78
|
+
sys.exit(1)
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
if __name__ == "__main__":
|
|
82
|
+
main()
|
xbridge/converter.py
CHANGED
|
@@ -77,7 +77,7 @@ class Converter:
|
|
|
77
77
|
output_path: Union[str, Path],
|
|
78
78
|
headers_as_datapoints: bool = False,
|
|
79
79
|
validate_filing_indicators: bool = True,
|
|
80
|
-
strict_validation: bool =
|
|
80
|
+
strict_validation: bool = False,
|
|
81
81
|
) -> Path:
|
|
82
82
|
"""Convert the ``XML Instance`` to a CSV file or between CSV formats"""
|
|
83
83
|
if not output_path:
|
|
@@ -300,28 +300,27 @@ class Converter:
|
|
|
300
300
|
if instance_df.empty or table.variable_df is None:
|
|
301
301
|
return set()
|
|
302
302
|
|
|
303
|
+
variable_columns = set(table.variable_columns or [])
|
|
303
304
|
open_keys = set(table.open_keys)
|
|
304
305
|
|
|
305
|
-
|
|
306
|
+
# self.instance.instance_df is guaranteed to not be None due to check at line 296
|
|
307
|
+
instance_columns = set(self.instance.instance_df.columns)
|
|
306
308
|
|
|
307
|
-
|
|
308
|
-
merge_cols: list[str] = []
|
|
309
|
-
if "metric" in datapoint_df.columns and "metric" in instance_df.columns:
|
|
310
|
-
merge_cols.append("metric")
|
|
311
|
-
merge_cols.extend(
|
|
312
|
-
[key for key in open_keys if key in datapoint_df.columns and key in instance_df.columns]
|
|
313
|
-
)
|
|
309
|
+
datapoint_df = table.variable_df.copy()
|
|
314
310
|
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
311
|
+
# Handle missing columns by filtering datapoint_df (same as _variable_generator)
|
|
312
|
+
# This prevents requiring dimensions that don't exist in the instance
|
|
313
|
+
missing_cols = list(variable_columns - instance_columns)
|
|
314
|
+
if "data_type" in missing_cols:
|
|
315
|
+
missing_cols.remove("data_type")
|
|
316
|
+
if missing_cols:
|
|
317
|
+
mask = datapoint_df[missing_cols].isnull().all(axis=1)
|
|
318
|
+
datapoint_df = datapoint_df.loc[mask]
|
|
319
|
+
datapoint_df = datapoint_df.drop(columns=missing_cols)
|
|
319
320
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
if col in instance_df.columns:
|
|
324
|
-
instance_df[col] = instance_df[col].map(_strip_prefix)
|
|
321
|
+
# Match on all variable columns (dimensions) to avoid Cartesian product
|
|
322
|
+
# explosion. Consistent with _variable_generator() to prevent OOM.
|
|
323
|
+
merge_cols = list(variable_columns & instance_columns)
|
|
325
324
|
|
|
326
325
|
instance_df = instance_df.copy()
|
|
327
326
|
instance_df["_idx"] = instance_df.index
|
|
@@ -367,18 +366,6 @@ class Converter:
|
|
|
367
366
|
# Join the dataframes on the datapoint_columns
|
|
368
367
|
merge_cols = list(variable_columns & instance_columns)
|
|
369
368
|
|
|
370
|
-
def _strip_prefix(val: Any) -> Any:
|
|
371
|
-
if isinstance(val, str) and ":" in val:
|
|
372
|
-
return val.split(":", 1)[1]
|
|
373
|
-
return val
|
|
374
|
-
|
|
375
|
-
# Align merge columns by stripping any namespace prefixes from both sides
|
|
376
|
-
for col in merge_cols:
|
|
377
|
-
if col in datapoint_df.columns:
|
|
378
|
-
datapoint_df[col] = datapoint_df[col].map(_strip_prefix)
|
|
379
|
-
if col in instance_df.columns:
|
|
380
|
-
instance_df[col] = instance_df[col].map(_strip_prefix)
|
|
381
|
-
|
|
382
369
|
table_df = pd.merge(datapoint_df, instance_df, on=merge_cols, how="inner")
|
|
383
370
|
|
|
384
371
|
if "data_type" in table_df.columns and "decimals" in table_df.columns:
|
xbridge/instance.py
CHANGED
|
@@ -40,6 +40,49 @@ def _derive_csv_prefix(namespace_uri: str) -> Optional[str]:
|
|
|
40
40
|
return prefix
|
|
41
41
|
|
|
42
42
|
|
|
43
|
+
def _derive_metric_prefix(namespace_uri: str) -> Optional[str]:
|
|
44
|
+
"""
|
|
45
|
+
Derive the CSV prefix for metrics from a namespace URI.
|
|
46
|
+
|
|
47
|
+
For metrics, we preserve version suffixes in the prefix:
|
|
48
|
+
- http://www.eba.europa.eu/xbrl/crr/dict/met -> eba_met
|
|
49
|
+
- http://www.eba.europa.eu/xbrl/crr/dict/met/3.5 -> eba_met_3.5
|
|
50
|
+
- http://www.eba.europa.eu/xbrl/crr/dict/met/4.0 -> eba_met_4.0
|
|
51
|
+
"""
|
|
52
|
+
if not namespace_uri:
|
|
53
|
+
return None
|
|
54
|
+
|
|
55
|
+
cached = _namespace_prefix_cache.get(f"metric:{namespace_uri}")
|
|
56
|
+
if cached is not None:
|
|
57
|
+
return cached
|
|
58
|
+
|
|
59
|
+
cleaned = namespace_uri.rstrip("#/")
|
|
60
|
+
|
|
61
|
+
# Split the URI into path segments
|
|
62
|
+
segments = cleaned.split("/")
|
|
63
|
+
|
|
64
|
+
# Find the 'met' (metrics) segment and check if there's a version after it
|
|
65
|
+
prefix = None
|
|
66
|
+
for i, segment in enumerate(segments):
|
|
67
|
+
if segment == "met":
|
|
68
|
+
# Check if there's a version suffix (e.g., "3.5", "4.0")
|
|
69
|
+
if i + 1 < len(segments):
|
|
70
|
+
version = segments[i + 1]
|
|
71
|
+
prefix = f"eba_met_{version}"
|
|
72
|
+
else:
|
|
73
|
+
prefix = "eba_met"
|
|
74
|
+
break
|
|
75
|
+
|
|
76
|
+
# If we didn't find 'met', fall back to the standard logic
|
|
77
|
+
if prefix is None:
|
|
78
|
+
prefix = _derive_csv_prefix(namespace_uri)
|
|
79
|
+
|
|
80
|
+
if prefix:
|
|
81
|
+
_namespace_prefix_cache[f"metric:{namespace_uri}"] = prefix
|
|
82
|
+
|
|
83
|
+
return prefix
|
|
84
|
+
|
|
85
|
+
|
|
43
86
|
def _normalize_namespaced_value(
|
|
44
87
|
value: Optional[str], nsmap: Dict[Optional[str], str]
|
|
45
88
|
) -> Optional[str]:
|
|
@@ -67,6 +110,34 @@ def _normalize_namespaced_value(
|
|
|
67
110
|
return value
|
|
68
111
|
|
|
69
112
|
|
|
113
|
+
def _normalize_metric_value(
|
|
114
|
+
value: Optional[str], nsmap: Dict[Optional[str], str]
|
|
115
|
+
) -> Optional[str]:
|
|
116
|
+
"""
|
|
117
|
+
Normalize a metric namespaced value to the CSV prefix convention.
|
|
118
|
+
For metrics, we preserve version suffixes (e.g., eba_met_3.5, eba_met_4.0).
|
|
119
|
+
Returns the original value if no namespace can be resolved.
|
|
120
|
+
"""
|
|
121
|
+
if value is None:
|
|
122
|
+
return None
|
|
123
|
+
|
|
124
|
+
# Clark notation: {uri}local
|
|
125
|
+
if value.startswith("{") and "}" in value:
|
|
126
|
+
uri, local = value[1:].split("}", 1)
|
|
127
|
+
derived = _derive_metric_prefix(uri)
|
|
128
|
+
return f"{derived}:{local}" if derived else value
|
|
129
|
+
|
|
130
|
+
# Prefixed notation: prefix:local
|
|
131
|
+
if ":" in value:
|
|
132
|
+
potential_prefix, local = value.split(":", 1)
|
|
133
|
+
namespace_uri = nsmap.get(potential_prefix)
|
|
134
|
+
if namespace_uri:
|
|
135
|
+
derived = _derive_metric_prefix(namespace_uri)
|
|
136
|
+
return f"{derived}:{local}" if derived else value
|
|
137
|
+
|
|
138
|
+
return value
|
|
139
|
+
|
|
140
|
+
|
|
70
141
|
class Instance:
|
|
71
142
|
"""
|
|
72
143
|
Abstract class representing an XBRL instance file.
|
|
@@ -728,7 +799,11 @@ class Fact:
|
|
|
728
799
|
def __dict__(self) -> Dict[str, Any]: # type: ignore[override]
|
|
729
800
|
metric_clean = ""
|
|
730
801
|
if self.metric:
|
|
731
|
-
|
|
802
|
+
# Normalize metric using metric-specific logic that preserves version suffixes
|
|
803
|
+
metric_clean = _normalize_metric_value(self.metric, self.fact_xml.nsmap) or ""
|
|
804
|
+
# If still in Clark notation, extract the local name
|
|
805
|
+
if metric_clean.startswith("{") and "}" in metric_clean:
|
|
806
|
+
metric_clean = metric_clean.split("}", 1)[1]
|
|
732
807
|
|
|
733
808
|
return {
|
|
734
809
|
"metric": metric_clean,
|
xbridge/modules.py
CHANGED
|
@@ -306,9 +306,9 @@ class Table:
|
|
|
306
306
|
variable_info: dict[str, Any] = {}
|
|
307
307
|
for dim_k, dim_v in variable.dimensions.items():
|
|
308
308
|
if dim_k not in ("unit", "decimals"):
|
|
309
|
-
variable_info[dim_k] = dim_v
|
|
309
|
+
variable_info[dim_k] = dim_v
|
|
310
310
|
if "concept" in variable.dimensions:
|
|
311
|
-
variable_info["metric"] = variable.dimensions["concept"]
|
|
311
|
+
variable_info["metric"] = variable.dimensions["concept"]
|
|
312
312
|
del variable_info["concept"]
|
|
313
313
|
|
|
314
314
|
if variable.code is None:
|
|
@@ -324,9 +324,11 @@ class Table:
|
|
|
324
324
|
if "dimensions" in column:
|
|
325
325
|
for dim_k, dim_v in column["dimensions"].items():
|
|
326
326
|
if dim_k == "concept":
|
|
327
|
-
variable_info["metric"] = dim_v
|
|
327
|
+
variable_info["metric"] = dim_v
|
|
328
328
|
elif dim_k not in ("unit", "decimals"):
|
|
329
|
-
|
|
329
|
+
# Keep the full dimension key and value with prefixes
|
|
330
|
+
dim_k_clean = dim_k.split(":")[1] if ":" in dim_k else dim_k
|
|
331
|
+
variable_info[dim_k_clean] = dim_v
|
|
330
332
|
|
|
331
333
|
if "decimals" in column:
|
|
332
334
|
variable_info["data_type"] = column["decimals"]
|
|
@@ -1,62 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: eba-xbridge
|
|
3
|
-
Version: 1.5.0rc3
|
|
4
|
-
Summary: XBRL-XML to XBRL-CSV converter for EBA Taxonomy (version 4.1)
|
|
5
|
-
License: Apache 2.0
|
|
6
|
-
License-File: LICENSE
|
|
7
|
-
Keywords: xbrl,eba,taxonomy,csv,xml
|
|
8
|
-
Author: MeaningfulData
|
|
9
|
-
Author-email: info@meaningfuldata.eu
|
|
10
|
-
Maintainer: Antonio Olleros
|
|
11
|
-
Maintainer-email: antonio.olleros@meaningfuldata.eu
|
|
12
|
-
Requires-Python: >=3.9
|
|
13
|
-
Classifier: Development Status :: 5 - Production/Stable
|
|
14
|
-
Classifier: Intended Audience :: Developers
|
|
15
|
-
Classifier: Intended Audience :: Information Technology
|
|
16
|
-
Classifier: Intended Audience :: Science/Research
|
|
17
|
-
Classifier: Programming Language :: Python :: 3
|
|
18
|
-
Classifier: Typing :: Typed
|
|
19
|
-
Requires-Dist: lxml (>=5.2.1,<6.0)
|
|
20
|
-
Requires-Dist: numpy (>=1.23.2,<2) ; python_version < "3.13"
|
|
21
|
-
Requires-Dist: numpy (>=2.1.0) ; python_version >= "3.13"
|
|
22
|
-
Requires-Dist: pandas (>=2.1.4,<3.0)
|
|
23
|
-
Project-URL: Documentation, https://docs.xbridge.meaningfuldata.eu
|
|
24
|
-
Project-URL: IssueTracker, https://github.com/Meaningful-Data/xbridge/issues
|
|
25
|
-
Project-URL: MeaningfulData, https://www.meaningfuldata.eu/
|
|
26
|
-
Project-URL: Repository, https://github.com/Meaningful-Data/xbridge
|
|
27
|
-
Description-Content-Type: text/x-rst
|
|
28
|
-
|
|
29
|
-
Overview
|
|
30
|
-
============
|
|
31
|
-
XBridge is a Python library which main function is to convert XBRL-XML files into XBRL-CSV files by using EBA's taxonomy.
|
|
32
|
-
It works with EBA Taxonomy latest published version (4.1). Library must be updated on each new EBA taxonomy version.
|
|
33
|
-
|
|
34
|
-
Installation
|
|
35
|
-
============
|
|
36
|
-
|
|
37
|
-
To install the library, run the following command:
|
|
38
|
-
|
|
39
|
-
.. code:: bash
|
|
40
|
-
|
|
41
|
-
pip install eba-xbridge
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
How XBridge works:
|
|
45
|
-
=========================
|
|
46
|
-
|
|
47
|
-
Firstly, an XBRL-XML file has to be selected to convert it. Then, that XBRL-XML file is input in the following function contained in the ``API`` package:
|
|
48
|
-
|
|
49
|
-
.. code:: python
|
|
50
|
-
|
|
51
|
-
>>> from xbridge.api import convert_instance
|
|
52
|
-
|
|
53
|
-
>>> input_path = "data/input"
|
|
54
|
-
|
|
55
|
-
>>> output_path = "data/output"
|
|
56
|
-
|
|
57
|
-
>>> convert_instance(input_path, output_path)
|
|
58
|
-
|
|
59
|
-
The sources to do this process are two: The XML-instances and EBA´s taxonomy.
|
|
60
|
-
|
|
61
|
-
The output is the converted XBRL-CSV file placed in the output_path, as zip format
|
|
62
|
-
|
|
File without changes
|
|
File without changes
|