dpm-xbrl 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
dpm_xbrl-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Luuk Hopman
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,305 @@
1
+ Metadata-Version: 2.4
2
+ Name: dpm-xbrl
3
+ Version: 0.1.0
4
+ Summary: Read and write EBA-style xBRL-CSV and XBRL-XML regulatory reports (DPM taxonomies)
5
+ Keywords: EBA,XBRL,xBRL-CSV,regulatory reporting,report package
6
+ Author: Luuk Hopman
7
+ Author-email: Luuk Hopman <github@luukhopman.nl>
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Financial and Insurance Industry
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3 :: Only
14
+ Classifier: Programming Language :: Python :: 3.11
15
+ Classifier: Programming Language :: Python :: 3.12
16
+ Classifier: Programming Language :: Python :: 3.13
17
+ Classifier: Programming Language :: Python :: 3.14
18
+ Classifier: Topic :: Office/Business :: Financial
19
+ Classifier: Typing :: Typed
20
+ Requires-Dist: isal>=1.8,<2 ; extra == 'fast'
21
+ Requires-Dist: polars>=2.0,<3 ; extra == 'frames'
22
+ Requires-Python: >=3.11
23
+ Project-URL: Homepage, https://github.com/luukhopman/dpm-xbrl
24
+ Project-URL: Source, https://github.com/luukhopman/dpm-xbrl
25
+ Project-URL: Issues, https://github.com/luukhopman/dpm-xbrl/issues
26
+ Provides-Extra: fast
27
+ Provides-Extra: frames
28
+ Description-Content-Type: text/markdown
29
+
30
+ # dpm-xbrl
31
+
32
+ Read and write EBA-style regulatory XBRL reports in both serializations, xBRL-CSV
33
+ report packages and XBRL 2.1 XML instances, through one neutral fact model.
34
+ It targets reports built on the data point model (DPM) taxonomies of the
35
+ European Banking Authority and is tested on EBA reports. It is an independent
36
+ project, not affiliated with or endorsed by the EBA.
37
+ The package uses only the standard library and needs Python 3.11 or newer.
38
+ The optional `fast` extra adds `isal`, which inflates ZIP entries about twice
39
+ as fast as zlib; the bytes read are the same either way.
40
+
41
+ ```python
42
+ from dpmxbrl import read_report
43
+
44
+ report = read_report("report.zip") # or a directory, report.json, .xbrl, or bytes
45
+ print(report.entity_id, report.reference_date, len(report.facts))
46
+ for fact in report.facts_for_template("C_01.00"):
47
+ print(fact.datapoint, fact.keys, fact.value)
48
+ ```
49
+
50
+ ## Model
51
+
52
+ `dpmxbrl.model` (re-exported from `dpmxbrl`) defines two frozen, slotted
53
+ dataclasses:
54
+
55
+ - `Fact`: `value` (lexical value as written, `None` when nil), `concept`
56
+ (prefixed qname), `datapoint` (xBRL-CSV datapoint code), `template`
57
+ (xBRL-CSV table template id, `None` for XML), `dimensions` (explicit dimension
58
+ qname to member qname), `typed_dimensions` (typed dimension qname to the typed
59
+ value text), `keys` (xBRL-CSV non-fact column name to cell value, in header
60
+ order), `unit` (`iso4217:EUR`, `xbrli:pure`, `num/denom` for divide units,
61
+ `a*b` for products), `decimals` (`None` means INF or unspecified) and
62
+ `location` (`reports/c_01.00.csv:3` with the 1-based physical line, or
63
+ `instance.xbrl#f12` with the fact id, `#factN` for the Nth fact without id).
64
+ - `Report`: `format`, `entry_point`, `entity_scheme`, `entity_id`,
65
+ `reference_date`, `base_currency`, `parameters`, `decimals_parameters`,
66
+ `filing_indicators`, `facts`, `namespaces`, `warnings` and `filing` (the
67
+ EBA filing rules the report breaks, see below).
68
+
69
+ `Report.facts_for_template(code)` compares template codes case-insensitively,
70
+ so filing-indicator codes such as `C_01.00` select facts from `c_01.00.csv`.
71
+ `local_name(qname)` returns the local part of a prefixed or Clark qname.
72
+
73
+ Qnames always use prefixes declared in `Report.namespaces`. XML keeps the
74
+ instance's own prefixes; a namespace declared again deeper in the document
75
+ under another prefix is rendered with the first prefix bound to it.
76
+
77
+ ## Reading
78
+
79
+ ```python
80
+ read_report(source, *, name=None, metadata_resolver=None,
81
+ taxonomy_packages=(), limits=Limits()) -> Report
82
+ read_xbrl_csv(source, *, name=None, metadata_resolver=None,
83
+ taxonomy_packages=(), limits=Limits()) -> Report
84
+ read_xbrl_xml(source, *, name=None, limits=Limits()) -> Report
85
+ detect_format(source) -> "xbrl-csv" | "xbrl-xml"
86
+ ```
87
+
88
+ `read_report` treats ZIP input (bytes or a path), a directory and a `.json`
89
+ path as xBRL-CSV and everything else as XBRL-XML. Every failure raises
90
+ `ReportError` (a `ValueError`) naming the file and, where known, the line.
91
+
92
+ ### xBRL-CSV
93
+
94
+ - A ZIP or directory with exactly one `reports/*.json` report (normally
95
+ `reports/report.json`), optionally below one top-level folder, plus an
96
+ optional `META-INF/reportPackage.json` whose `documentType` must be
97
+ `https://xbrl.org/report-package/2023` or `.../2023/xbrl-csv`. A directory
98
+ may also hold `report.json` directly.
99
+ - `documentInfo.documentType` must be `https://xbrl.org/2021/xbrl-csv`; the
100
+ 2019-2021 PWD/CR/PR draft URIs are accepted with a warning. Duplicate JSON
101
+ keys are rejected.
102
+ - `parameters.csv` (or the metadata's `parameterURL`) must have the header
103
+ `name,value`. `entityID` such as `lei:5299...` becomes `entity_id`; its
104
+ prefix is looked up in `namespaces` for `entity_scheme`, else kept as the
105
+ scheme. `refPeriod` must be a `YYYY-MM-DD` date. Parameters named
106
+ `decimals*` populate `decimals_parameters` (`#none` and `INF` map to `None`).
107
+ Inline JSON `parameters` are merged first; the CSV file overrides them.
108
+ - `FilingIndicators.csv` must have the header `templateID,reported`, with
109
+ `reported` one of `true`, `false`, `1`, `0`.
110
+ - Every other `*.csv` under the reports folder is a table. Its template id is
111
+ the file stem unless the metadata `tables` maps the file URL to a template.
112
+
113
+ **EBA convention (no metadata).** Tables need `datapoint` and `factValue`
114
+ columns; every other column is a key column kept in `Fact.keys`. Each row with
115
+ a non-empty `factValue` is one fact. `#nil` gives a nil fact, `#empty` and
116
+ `#none` are skipped, `##` escapes a leading `#`, and other `#` values are kept
117
+ as written; each case adds a warning. `Fact.decimals` stays `None`; consumers
118
+ pick the `decimals_parameters` entry for the fact's data type. Exact duplicate
119
+ rows are reported once; conflicting duplicates (same template, datapoint and
120
+ keys, different value) are all kept. Repeated findings are aggregated into one
121
+ warning with a count and the first location.
122
+
123
+ **Resolved metadata.** `report.json` and its `extends` chain are followed
124
+ recursively (cycle-safe, diamond-safe, depth-limited), merging `namespaces`,
125
+ `tableTemplates`, `tables`, `parameters`, `dimensions`, `decimals` and
126
+ `parameterURL`, with the extending document taking precedence. Relative URLs
127
+ resolve inside the package or against the extending document's URL. Absolute
128
+ URLs are resolved by `metadata_resolver(url)` first and then through the
129
+ `META-INF/catalog.xml` `rewriteURI`/`rewriteSystem` prefixes of the given
130
+ taxonomy package ZIPs. Nothing is fetched over the network. Table `url`s and
131
+ `parameterURL` resolve against the report's own JSON.
132
+
133
+ For a table whose template is resolved, the supported xBRL-CSV subset is:
134
+ template `columns`; fact columns (with `dimensions` or `propertiesFrom`);
135
+ property-group columns (`propertyGroups`, such as EBA's `datapoint` column keyed
136
+ by `dp...`); other columns as key columns; report, template, column and
137
+ property-group `dimensions` and `decimals`, in increasing precedence;
138
+ `$column` and `$parameter` references (row cell, then table parameters, then
139
+ report parameters), `$$` escapes and `#none`; core dimensions `concept`,
140
+ `unit`, `period`, `entity` and `language`; and taxonomy-defined dimensions. A
141
+ taxonomy-defined dimension value that is a qname with a declared prefix is an
142
+ explicit member; any other value is a typed value. `Fact.datapoint` is the
143
+ property-group key and `Fact.keys` keeps the key columns. A table whose
144
+ template cannot be resolved falls back to the EBA convention, and unresolved
145
+ `extends` produce one warning.
146
+
147
+ ### XBRL-XML
148
+
149
+ - The prolog is scanned before parsing and any DOCTYPE (and with it any ENTITY
150
+ declaration) is rejected; external entities are never resolved. Parsing
151
+ streams with `iterparse` and clears each top-level element once processed.
152
+ - `link:schemaRef` gives `entry_point`. Contexts provide the entity identifier
153
+ and scheme, the reference date (instant, or the end date of a duration) and
154
+ `xbrldi:explicitMember`/`xbrldi:typedMember` from segment or scenario. A typed
155
+ member's value is the stripped text of its single child element; the child
156
+ element name is not kept.
157
+ - Units with one or more measures, or a divide, become strings. Facts are
158
+ top-level elements outside the `xbrli`, `link` and `find` namespaces with a
159
+ `contextRef`; `xsi:nil="true"` gives `value=None`; `decimals="INF"` gives
160
+ `None`. `precision` is ignored, and tuples and elements without `contextRef`
161
+ are skipped, each with a warning.
162
+ - `find:fIndicators/find:filingIndicator` provide filing indicators;
163
+ `find:filed` defaults to true.
164
+ - The report entity and reference date are the most common across facts, and
165
+ `base_currency` is the most common single-measure `iso4217` unit. More than
166
+ one entity or period adds a warning, because EBA requires one.
167
+
168
+ ### Limits
169
+
170
+ `Limits(max_zip_entries=10_000, max_total_size=2 GiB, max_file_size=512 MiB,
171
+ max_facts=20_000_000, max_extends_depth=32)`. Report packages are also rejected
172
+ for absolute or `..` entry names, duplicate entries and encrypted entries, and
173
+ package directories for links that leave the directory. The file-size limit
174
+ also applies to XML instances and to files read from taxonomy packages. CSV
175
+ rows are streamed with the `csv` module (`utf-8-sig`), including from ZIPs.
176
+ An entry whose compressed data is damaged raises `ReportError` naming the entry.
177
+
178
+ ### EBA filing rules
179
+
180
+ Both readers record the [EBA Filing Rules](https://www.eba.europa.eu/sites/default/files/2026-02/10436139-46ae-4a7d-880f-c399ba6de669/EBA%20Filing%20Rules%20v5.8_2026_02_25.pdf)
181
+ (version 5.8, February 2026) that a report breaks and that the file alone can show, in
182
+ `Report.filing`: one `FilingFinding` per rule and problem, with the rule's
183
+ number (`2.19`), its tag in the EBA document (`nilUsed`), a severity, a
184
+ message, how many times it occurs and up to five locations.
185
+ `dpmxbrl.filing.RULE_TITLES` names each rule. Severity is `error` where the
186
+ rule says MUST (EBA rejects the report) and `warning` where it says SHOULD;
187
+ file-naming findings are warnings because each supervisor confirms its own
188
+ naming.
189
+
190
+ - Both formats: one absolute taxonomy entry point (1.5, 2.2, 2.3); the
191
+ reporting subject's scheme, suffix, upper case and LEI check digits (2.8,
192
+ 3.6); and the file name against `ReportSubject_Country_
193
+ FrameworkCodeModuleVersion_Module_ReferenceDate_CreationTimestamp`,
194
+ including the reporting subject, module and reference date.
195
+ - XBRL-XML: UTF-8 and no standalone declaration (1.4, 1.13); filing
196
+ indicators present, unique and in a context without segment or scenario
197
+ (1.6, 1.6.1); no `xsi:schemaLocation`, XInclude, `xml:base`,
198
+ `link:linkbaseRef`, footnotes, segments, forever or duration periods, or
199
+ non-dimensional scenario content (1.14, 1.15, 2.1, 2.4, 2.25, 2.11, 2.13, 2.14,
200
+ 2.15); one entity and reference date, as plain dates (2.9, 2.10, 2.13); no
201
+ duplicate facts, and none differing only by unit (2.16, 2.16.1); `decimals`
202
+ rather than `precision`, and present on numeric facts (2.17, 2.18); no nil or
203
+ empty facts (2.19); unused or repeated contexts and units (2.7, 2.21, 2.22);
204
+ one reporting currency, and currency-of-denomination facts in the currency
205
+ their CUS or CUA member names (3.1); `xbrli:pure` for other numeric facts
206
+ (3.2); and namespace prefixes declared once, on the document element, and
207
+ used (3.4, 3.9, 3.10).
208
+ - xBRL-CSV: `META-INF/reportPackage.json`, `parameters.csv` with
209
+ `refPeriod`, and `FilingIndicators.csv`; the root folder named as the ZIP;
210
+ no `documentInfo.taxonomy` (1.5); a plain-date `refPeriod` (2.10); unique
211
+ filing indicators (1.6.1); no table file for a template indicated as not
212
+ reported (xBRL-CSV rule 6); no special values in fact or key columns
213
+ (`#nil`, `#empty` and any other value starting with `#`; 2.19 and rule 5);
214
+ no decimals suffix (rule 1); key values without leading or trailing spaces
215
+ (3.11).
216
+
217
+ Rules that need the taxonomy are left to the caller, which can build
218
+ `FilingFinding`s and merge them with `dpmxbrl.filing.FilingLog`: a filing
219
+ indicator for every template of the module and only valid codes (1.6, 1.6.3),
220
+ facts outside the reported templates (1.7.1), minimum decimals per data type
221
+ (2.18), fact values without leading or trailing spaces, which applies to
222
+ string data points only (3.11, `SPACES` in `dpmxbrl.filing`), an entry
223
+ point valid for the reference date (1.5), and `baseCurrency` when monetary
224
+ facts use it. Not checked: units against the XBRL unit registry (2.23),
225
+ decimal notation for rates (3.2), canonical prefixes (3.5),
226
+ generating-software information (2.26) and comments (2.5).
227
+ `read_xbrl_csv_frames` checks special fact values only; the decimals suffix,
228
+ spaces around key values and special key values are left to callers that
229
+ split values by data type, which can record them with the same messages
230
+ (`DECIMALS_SUFFIX`, `KEY_SPACES`, `SPECIAL_KEY_VALUE` in `dpmxbrl.filing`).
231
+
232
+ ## Writing
233
+
234
+ ```python
235
+ write_xbrl_csv(report, target, *, root=None) -> None
236
+ write_xbrl_xml(report, target, *, typed_domains=None) -> None
237
+ default_package_root(report) -> str
238
+ ```
239
+
240
+ `write_xbrl_csv` writes a ZIP report package in EBA convention:
241
+ `<root>/META-INF/reportPackage.json`, `<root>/reports/report.json` (extending
242
+ `entry_point` and declaring `namespaces`), `parameters.csv` (`entityID`,
243
+ `refPeriod`, `baseCurrency`, `decimals*`, then any other parameter),
244
+ `FilingIndicators.csv` and one `<template>.csv` per template with header
245
+ `datapoint,factValue,<key columns in first-seen order>`. Nil values are
246
+ written as `#nil`, empty strings as `#empty` and a leading `#` as `##`. The root
247
+ defaults to `<entity id>_<reference date>`. Output is deterministic (sorted
248
+ templates, source fact order, fixed ZIP timestamps). Facts without a template
249
+ or datapoint raise `ReportError` before anything is written.
250
+
251
+ `write_xbrl_xml` writes `schemaRef`, filing indicators, contexts deduplicated
252
+ by dimensions (`c1`, `c2`, ..., entity plus an instant at `reference_date`,
253
+ dimensions in the scenario), units (`u1`, ...) and facts. A fact with a unit
254
+ is numeric and gets `decimals` (`INF` when unset) unless it is nil. Facts
255
+ without a concept, undeclared prefixes and a missing entity or reference date
256
+ raise `ReportError`. Because the reader does not keep typed-domain element
257
+ names, typed dimensions need `typed_domains={"eba_dim:qINC": "eba_typ:qCC"}`.
258
+
259
+ Reading a written report reproduces its facts in both formats; the tests
260
+ cover these round trips.
261
+
262
+ ## CLI
263
+
264
+ ```
265
+ dpm-xbrl inspect PATH [--json] [--taxonomy-package ZIP ...]
266
+ dpm-xbrl convert IN OUT [--taxonomy-package ZIP ...]
267
+ [--typed-domain DIMENSION=ELEMENT ...] [--root NAME]
268
+ ```
269
+
270
+ `inspect` prints the format, entry point, entity, reference date, base
271
+ currency, filing indicators (reported of total), the fact count per template,
272
+ the warnings and the EBA filing rules the report breaks. `convert` chooses the output format from the suffix: `.zip`
273
+ writes xBRL-CSV, `.xbrl` or `.xml` writes XBRL-XML. Errors print `error: ...`
274
+ and exit with status 1.
275
+
276
+ ## Development
277
+
278
+ ```
279
+ uv sync
280
+ uv run pytest
281
+ uv run ruff check
282
+ uv run ruff format --check
283
+ uv run ty check
284
+ ```
285
+
286
+ ## Known gaps
287
+
288
+ - No taxonomy validation, schema lookups or calculation and formula checks.
289
+ Without a taxonomy, an explicit member and a typed value that looks like a
290
+ declared qname cannot be told apart in xBRL-CSV metadata.
291
+ - xBRL-CSV: no `rowIdColumn`, `links`, footnotes, `@start`/`@end` period
292
+ semantics, per-fact `period`/`entity`/`language` values (they are ignored),
293
+ or multiple reports per package. Taxonomy packages must be ZIPs.
294
+ - EBA-convention CSV facts carry no concept, dimensions, unit or decimals, so
295
+ they cannot be written as XML; XML facts carry no datapoint or template, so
296
+ they cannot be written as CSV. Converting between formats needs resolved
297
+ xBRL-CSV metadata.
298
+ - Empty-string facts do not survive an xBRL-CSV round trip (`#empty` is
299
+ skipped when read), and `Fact.decimals`, concepts and dimensions are not
300
+ written to EBA-convention CSV.
301
+ - XML: tuples, footnotes, `precision`, segment versus scenario placement,
302
+ forever periods and typed-domain element names are not kept. XML output puts
303
+ all dimensions in the scenario and uses one entity and instant. The XML
304
+ writer holds the instance text in memory before writing it.
305
+ - Fact locations in XML use the fact id rather than a line number.
@@ -0,0 +1,276 @@
1
+ # dpm-xbrl
2
+
3
+ Read and write EBA-style regulatory XBRL reports in both serializations, xBRL-CSV
4
+ report packages and XBRL 2.1 XML instances, through one neutral fact model.
5
+ It targets reports built on the data point model (DPM) taxonomies of the
6
+ European Banking Authority and is tested on EBA reports. It is an independent
7
+ project, not affiliated with or endorsed by the EBA.
8
+ The package uses only the standard library and needs Python 3.11 or newer.
9
+ The optional `fast` extra adds `isal`, which inflates ZIP entries about twice
10
+ as fast as zlib; the bytes read are the same either way.
11
+
12
+ ```python
13
+ from dpmxbrl import read_report
14
+
15
+ report = read_report("report.zip") # or a directory, report.json, .xbrl, or bytes
16
+ print(report.entity_id, report.reference_date, len(report.facts))
17
+ for fact in report.facts_for_template("C_01.00"):
18
+ print(fact.datapoint, fact.keys, fact.value)
19
+ ```
20
+
21
+ ## Model
22
+
23
+ `dpmxbrl.model` (re-exported from `dpmxbrl`) defines two frozen, slotted
24
+ dataclasses:
25
+
26
+ - `Fact`: `value` (lexical value as written, `None` when nil), `concept`
27
+ (prefixed qname), `datapoint` (xBRL-CSV datapoint code), `template`
28
+ (xBRL-CSV table template id, `None` for XML), `dimensions` (explicit dimension
29
+ qname to member qname), `typed_dimensions` (typed dimension qname to the typed
30
+ value text), `keys` (xBRL-CSV non-fact column name to cell value, in header
31
+ order), `unit` (`iso4217:EUR`, `xbrli:pure`, `num/denom` for divide units,
32
+ `a*b` for products), `decimals` (`None` means INF or unspecified) and
33
+ `location` (`reports/c_01.00.csv:3` with the 1-based physical line, or
34
+ `instance.xbrl#f12` with the fact id, `#factN` for the Nth fact without id).
35
+ - `Report`: `format`, `entry_point`, `entity_scheme`, `entity_id`,
36
+ `reference_date`, `base_currency`, `parameters`, `decimals_parameters`,
37
+ `filing_indicators`, `facts`, `namespaces`, `warnings` and `filing` (the
38
+ EBA filing rules the report breaks, see below).
39
+
40
+ `Report.facts_for_template(code)` compares template codes case-insensitively,
41
+ so filing-indicator codes such as `C_01.00` select facts from `c_01.00.csv`.
42
+ `local_name(qname)` returns the local part of a prefixed or Clark qname.
43
+
44
+ Qnames always use prefixes declared in `Report.namespaces`. XML keeps the
45
+ instance's own prefixes; a namespace declared again deeper in the document
46
+ under another prefix is rendered with the first prefix bound to it.
47
+
48
+ ## Reading
49
+
50
+ ```python
51
+ read_report(source, *, name=None, metadata_resolver=None,
52
+ taxonomy_packages=(), limits=Limits()) -> Report
53
+ read_xbrl_csv(source, *, name=None, metadata_resolver=None,
54
+ taxonomy_packages=(), limits=Limits()) -> Report
55
+ read_xbrl_xml(source, *, name=None, limits=Limits()) -> Report
56
+ detect_format(source) -> "xbrl-csv" | "xbrl-xml"
57
+ ```
58
+
59
+ `read_report` treats ZIP input (bytes or a path), a directory and a `.json`
60
+ path as xBRL-CSV and everything else as XBRL-XML. Every failure raises
61
+ `ReportError` (a `ValueError`) naming the file and, where known, the line.
62
+
63
+ ### xBRL-CSV
64
+
65
+ - A ZIP or directory with exactly one `reports/*.json` report (normally
66
+ `reports/report.json`), optionally below one top-level folder, plus an
67
+ optional `META-INF/reportPackage.json` whose `documentType` must be
68
+ `https://xbrl.org/report-package/2023` or `.../2023/xbrl-csv`. A directory
69
+ may also hold `report.json` directly.
70
+ - `documentInfo.documentType` must be `https://xbrl.org/2021/xbrl-csv`; the
71
+ 2019-2021 PWD/CR/PR draft URIs are accepted with a warning. Duplicate JSON
72
+ keys are rejected.
73
+ - `parameters.csv` (or the metadata's `parameterURL`) must have the header
74
+ `name,value`. `entityID` such as `lei:5299...` becomes `entity_id`; its
75
+ prefix is looked up in `namespaces` for `entity_scheme`, else kept as the
76
+ scheme. `refPeriod` must be a `YYYY-MM-DD` date. Parameters named
77
+ `decimals*` populate `decimals_parameters` (`#none` and `INF` map to `None`).
78
+ Inline JSON `parameters` are merged first; the CSV file overrides them.
79
+ - `FilingIndicators.csv` must have the header `templateID,reported`, with
80
+ `reported` one of `true`, `false`, `1`, `0`.
81
+ - Every other `*.csv` under the reports folder is a table. Its template id is
82
+ the file stem unless the metadata `tables` maps the file URL to a template.
83
+
84
+ **EBA convention (no metadata).** Tables need `datapoint` and `factValue`
85
+ columns; every other column is a key column kept in `Fact.keys`. Each row with
86
+ a non-empty `factValue` is one fact. `#nil` gives a nil fact, `#empty` and
87
+ `#none` are skipped, `##` escapes a leading `#`, and other `#` values are kept
88
+ as written; each case adds a warning. `Fact.decimals` stays `None`; consumers
89
+ pick the `decimals_parameters` entry for the fact's data type. Exact duplicate
90
+ rows are reported once; conflicting duplicates (same template, datapoint and
91
+ keys, different value) are all kept. Repeated findings are aggregated into one
92
+ warning with a count and the first location.
93
+
94
+ **Resolved metadata.** `report.json` and its `extends` chain are followed
95
+ recursively (cycle-safe, diamond-safe, depth-limited), merging `namespaces`,
96
+ `tableTemplates`, `tables`, `parameters`, `dimensions`, `decimals` and
97
+ `parameterURL`, with the extending document taking precedence. Relative URLs
98
+ resolve inside the package or against the extending document's URL. Absolute
99
+ URLs are resolved by `metadata_resolver(url)` first and then through the
100
+ `META-INF/catalog.xml` `rewriteURI`/`rewriteSystem` prefixes of the given
101
+ taxonomy package ZIPs. Nothing is fetched over the network. Table `url`s and
102
+ `parameterURL` resolve against the report's own JSON.
103
+
104
+ For a table whose template is resolved, the supported xBRL-CSV subset is:
105
+ template `columns`; fact columns (with `dimensions` or `propertiesFrom`);
106
+ property-group columns (`propertyGroups`, such as EBA's `datapoint` column keyed
107
+ by `dp...`); other columns as key columns; report, template, column and
108
+ property-group `dimensions` and `decimals`, in increasing precedence;
109
+ `$column` and `$parameter` references (row cell, then table parameters, then
110
+ report parameters), `$$` escapes and `#none`; core dimensions `concept`,
111
+ `unit`, `period`, `entity` and `language`; and taxonomy-defined dimensions. A
112
+ taxonomy-defined dimension value that is a qname with a declared prefix is an
113
+ explicit member; any other value is a typed value. `Fact.datapoint` is the
114
+ property-group key and `Fact.keys` keeps the key columns. A table whose
115
+ template cannot be resolved falls back to the EBA convention, and unresolved
116
+ `extends` produce one warning.
117
+
118
+ ### XBRL-XML
119
+
120
+ - The prolog is scanned before parsing and any DOCTYPE (and with it any ENTITY
121
+ declaration) is rejected; external entities are never resolved. Parsing
122
+ streams with `iterparse` and clears each top-level element once processed.
123
+ - `link:schemaRef` gives `entry_point`. Contexts provide the entity identifier
124
+ and scheme, the reference date (instant, or the end date of a duration) and
125
+ `xbrldi:explicitMember`/`xbrldi:typedMember` from segment or scenario. A typed
126
+ member's value is the stripped text of its single child element; the child
127
+ element name is not kept.
128
+ - Units with one or more measures, or a divide, become strings. Facts are
129
+ top-level elements outside the `xbrli`, `link` and `find` namespaces with a
130
+ `contextRef`; `xsi:nil="true"` gives `value=None`; `decimals="INF"` gives
131
+ `None`. `precision` is ignored, and tuples and elements without `contextRef`
132
+ are skipped, each with a warning.
133
+ - `find:fIndicators/find:filingIndicator` provide filing indicators;
134
+ `find:filed` defaults to true.
135
+ - The report entity and reference date are the most common across facts, and
136
+ `base_currency` is the most common single-measure `iso4217` unit. More than
137
+ one entity or period adds a warning, because EBA requires one.
138
+
139
+ ### Limits
140
+
141
+ `Limits(max_zip_entries=10_000, max_total_size=2 GiB, max_file_size=512 MiB,
142
+ max_facts=20_000_000, max_extends_depth=32)`. Report packages are also rejected
143
+ for absolute or `..` entry names, duplicate entries and encrypted entries, and
144
+ package directories for links that leave the directory. The file-size limit
145
+ also applies to XML instances and to files read from taxonomy packages. CSV
146
+ rows are streamed with the `csv` module (`utf-8-sig`), including from ZIPs.
147
+ An entry whose compressed data is damaged raises `ReportError` naming the entry.
148
+
149
+ ### EBA filing rules
150
+
151
+ Both readers record the [EBA Filing Rules](https://www.eba.europa.eu/sites/default/files/2026-02/10436139-46ae-4a7d-880f-c399ba6de669/EBA%20Filing%20Rules%20v5.8_2026_02_25.pdf)
152
+ (version 5.8, February 2026) that a report breaks and that the file alone can show, in
153
+ `Report.filing`: one `FilingFinding` per rule and problem, with the rule's
154
+ number (`2.19`), its tag in the EBA document (`nilUsed`), a severity, a
155
+ message, how many times it occurs and up to five locations.
156
+ `dpmxbrl.filing.RULE_TITLES` names each rule. Severity is `error` where the
157
+ rule says MUST (EBA rejects the report) and `warning` where it says SHOULD;
158
+ file-naming findings are warnings because each supervisor confirms its own
159
+ naming.
160
+
161
+ - Both formats: one absolute taxonomy entry point (1.5, 2.2, 2.3); the
162
+ reporting subject's scheme, suffix, upper case and LEI check digits (2.8,
163
+ 3.6); and the file name against `ReportSubject_Country_
164
+ FrameworkCodeModuleVersion_Module_ReferenceDate_CreationTimestamp`,
165
+ including the reporting subject, module and reference date.
166
+ - XBRL-XML: UTF-8 and no standalone declaration (1.4, 1.13); filing
167
+ indicators present, unique and in a context without segment or scenario
168
+ (1.6, 1.6.1); no `xsi:schemaLocation`, XInclude, `xml:base`,
169
+ `link:linkbaseRef`, footnotes, segments, forever or duration periods, or
170
+ non-dimensional scenario content (1.14, 1.15, 2.1, 2.4, 2.25, 2.11, 2.13, 2.14,
171
+ 2.15); one entity and reference date, as plain dates (2.9, 2.10, 2.13); no
172
+ duplicate facts, and none differing only by unit (2.16, 2.16.1); `decimals`
173
+ rather than `precision`, and present on numeric facts (2.17, 2.18); no nil or
174
+ empty facts (2.19); unused or repeated contexts and units (2.7, 2.21, 2.22);
175
+ one reporting currency, and currency-of-denomination facts in the currency
176
+ their CUS or CUA member names (3.1); `xbrli:pure` for other numeric facts
177
+ (3.2); and namespace prefixes declared once, on the document element, and
178
+ used (3.4, 3.9, 3.10).
179
+ - xBRL-CSV: `META-INF/reportPackage.json`, `parameters.csv` with
180
+ `refPeriod`, and `FilingIndicators.csv`; the root folder named as the ZIP;
181
+ no `documentInfo.taxonomy` (1.5); a plain-date `refPeriod` (2.10); unique
182
+ filing indicators (1.6.1); no table file for a template indicated as not
183
+ reported (xBRL-CSV rule 6); no special values in fact or key columns
184
+ (`#nil`, `#empty` and any other value starting with `#`; 2.19 and rule 5);
185
+ no decimals suffix (rule 1); key values without leading or trailing spaces
186
+ (3.11).
187
+
188
+ Rules that need the taxonomy are left to the caller, which can build
189
+ `FilingFinding`s and merge them with `dpmxbrl.filing.FilingLog`: a filing
190
+ indicator for every template of the module and only valid codes (1.6, 1.6.3),
191
+ facts outside the reported templates (1.7.1), minimum decimals per data type
192
+ (2.18), fact values without leading or trailing spaces, which applies to
193
+ string data points only (3.11, `SPACES` in `dpmxbrl.filing`), an entry
194
+ point valid for the reference date (1.5), and `baseCurrency` when monetary
195
+ facts use it. Not checked: units against the XBRL unit registry (2.23),
196
+ decimal notation for rates (3.2), canonical prefixes (3.5),
197
+ generating-software information (2.26) and comments (2.5).
198
+ `read_xbrl_csv_frames` checks special fact values only; the decimals suffix,
199
+ spaces around key values and special key values are left to callers that
200
+ split values by data type, which can record them with the same messages
201
+ (`DECIMALS_SUFFIX`, `KEY_SPACES`, `SPECIAL_KEY_VALUE` in `dpmxbrl.filing`).
202
+
203
+ ## Writing
204
+
205
+ ```python
206
+ write_xbrl_csv(report, target, *, root=None) -> None
207
+ write_xbrl_xml(report, target, *, typed_domains=None) -> None
208
+ default_package_root(report) -> str
209
+ ```
210
+
211
+ `write_xbrl_csv` writes a ZIP report package in EBA convention:
212
+ `<root>/META-INF/reportPackage.json`, `<root>/reports/report.json` (extending
213
+ `entry_point` and declaring `namespaces`), `parameters.csv` (`entityID`,
214
+ `refPeriod`, `baseCurrency`, `decimals*`, then any other parameter),
215
+ `FilingIndicators.csv` and one `<template>.csv` per template with header
216
+ `datapoint,factValue,<key columns in first-seen order>`. Nil values are
217
+ written as `#nil`, empty strings as `#empty` and a leading `#` as `##`. The root
218
+ defaults to `<entity id>_<reference date>`. Output is deterministic (sorted
219
+ templates, source fact order, fixed ZIP timestamps). Facts without a template
220
+ or datapoint raise `ReportError` before anything is written.
221
+
222
+ `write_xbrl_xml` writes `schemaRef`, filing indicators, contexts deduplicated
223
+ by dimensions (`c1`, `c2`, ..., entity plus an instant at `reference_date`,
224
+ dimensions in the scenario), units (`u1`, ...) and facts. A fact with a unit
225
+ is numeric and gets `decimals` (`INF` when unset) unless it is nil. Facts
226
+ without a concept, undeclared prefixes and a missing entity or reference date
227
+ raise `ReportError`. Because the reader does not keep typed-domain element
228
+ names, typed dimensions need `typed_domains={"eba_dim:qINC": "eba_typ:qCC"}`.
229
+
230
+ Reading a written report reproduces its facts in both formats; the tests
231
+ cover these round trips.
232
+
233
+ ## CLI
234
+
235
+ ```
236
+ dpm-xbrl inspect PATH [--json] [--taxonomy-package ZIP ...]
237
+ dpm-xbrl convert IN OUT [--taxonomy-package ZIP ...]
238
+ [--typed-domain DIMENSION=ELEMENT ...] [--root NAME]
239
+ ```
240
+
241
+ `inspect` prints the format, entry point, entity, reference date, base
242
+ currency, filing indicators (reported of total), the fact count per template,
243
+ the warnings and the EBA filing rules the report breaks. `convert` chooses the output format from the suffix: `.zip`
244
+ writes xBRL-CSV, `.xbrl` or `.xml` writes XBRL-XML. Errors print `error: ...`
245
+ and exit with status 1.
246
+
247
+ ## Development
248
+
249
+ ```
250
+ uv sync
251
+ uv run pytest
252
+ uv run ruff check
253
+ uv run ruff format --check
254
+ uv run ty check
255
+ ```
256
+
257
+ ## Known gaps
258
+
259
+ - No taxonomy validation, schema lookups or calculation and formula checks.
260
+ Without a taxonomy, an explicit member and a typed value that looks like a
261
+ declared qname cannot be told apart in xBRL-CSV metadata.
262
+ - xBRL-CSV: no `rowIdColumn`, `links`, footnotes, `@start`/`@end` period
263
+ semantics, per-fact `period`/`entity`/`language` values (they are ignored),
264
+ or multiple reports per package. Taxonomy packages must be ZIPs.
265
+ - EBA-convention CSV facts carry no concept, dimensions, unit or decimals, so
266
+ they cannot be written as XML; XML facts carry no datapoint or template, so
267
+ they cannot be written as CSV. Converting between formats needs resolved
268
+ xBRL-CSV metadata.
269
+ - Empty-string facts do not survive an xBRL-CSV round trip (`#empty` is
270
+ skipped when read), and `Fact.decimals`, concepts and dimensions are not
271
+ written to EBA-convention CSV.
272
+ - XML: tuples, footnotes, `precision`, segment versus scenario placement,
273
+ forever periods and typed-domain element names are not kept. XML output puts
274
+ all dimensions in the scenario and uses one entity and instant. The XML
275
+ writer holds the instance text in memory before writing it.
276
+ - Fact locations in XML use the fact id rather than a line number.