otb-cbso-webservice-pyclient 0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- otb_cbso_webservice_pyclient-0.1/.gitignore +14 -0
- otb_cbso_webservice_pyclient-0.1/PKG-INFO +337 -0
- otb_cbso_webservice_pyclient-0.1/README.md +311 -0
- otb_cbso_webservice_pyclient-0.1/pyproject.toml +148 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/__init__.py +108 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/api.py +518 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/archives.py +259 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/cli.py +705 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/client.py +490 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/download.py +1308 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/filenames.py +390 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/keys.py +110 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/logs.py +68 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/models.py +184 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/numbers.py +165 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/outputs.py +587 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/py.typed +0 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/references.py +466 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/s3.py +156 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/state.py +254 -0
- otb_cbso_webservice_pyclient-0.1/src/cbso_webservice_client/transport.py +400 -0
|
@@ -0,0 +1,337 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: otb-cbso-webservice-pyclient
|
|
3
|
+
Version: 0.1
|
|
4
|
+
Summary: Download annual account data from the Belgian National Bank's Central Balance Sheet Office webservices
|
|
5
|
+
Project-URL: Homepage, https://github.com/openthebox/cbso-webservice-pyclient
|
|
6
|
+
Project-URL: Issues, https://github.com/openthebox/cbso-webservice-pyclient/issues
|
|
7
|
+
Project-URL: Changelog, https://github.com/openthebox/cbso-webservice-pyclient/releases
|
|
8
|
+
Author: openthebox
|
|
9
|
+
License: Proprietary
|
|
10
|
+
Keywords: annual accounts,belgium,cbso,nbb,xbrl
|
|
11
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Typing :: Typed
|
|
17
|
+
Requires-Python: >=3.10
|
|
18
|
+
Requires-Dist: boto3>=1.34
|
|
19
|
+
Requires-Dist: urllib3>=2.0
|
|
20
|
+
Provides-Extra: dev
|
|
21
|
+
Requires-Dist: coverage[toml]>=7.4; extra == 'dev'
|
|
22
|
+
Requires-Dist: mypy>=1.9; extra == 'dev'
|
|
23
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
24
|
+
Requires-Dist: ruff>=0.4; extra == 'dev'
|
|
25
|
+
Description-Content-Type: text/markdown
|
|
26
|
+
|
|
27
|
+
# otb-cbso-webservice-pyclient
|
|
28
|
+
|
|
29
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
30
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
31
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
32
|
+

|
|
33
|
+
[](https://docs.astral.sh/ruff/)
|
|
34
|
+
[](https://mypy-lang.org/)
|
|
35
|
+
|
|
36
|
+
Download annual account data from the Belgian National Bank's Central Balance
|
|
37
|
+
Sheet Office (CBSO) [webservices][webservices], from Python or from the command
|
|
38
|
+
line.
|
|
39
|
+
|
|
40
|
+
The CBSO publishes the annual accounts that Belgian legal entities file, as
|
|
41
|
+
XBRL, JSON-XBRL and PDF, along with the corrections the bank applies to a
|
|
42
|
+
selection of them. This package speaks all five of its products, files what it
|
|
43
|
+
downloads under the enterprise that filed it, and picks up where it left off
|
|
44
|
+
when a run is repeated.
|
|
45
|
+
|
|
46
|
+
[webservices]: https://www.nbb.be/en/central-balance-sheet-office/consultation/web-services
|
|
47
|
+
|
|
48
|
+
- Two dependencies: `urllib3` for pooled, retrying HTTP, and `boto3`, loaded
|
|
49
|
+
only when an `s3://` destination is used.
|
|
50
|
+
- Typed throughout, with a `py.typed` marker.
|
|
51
|
+
- Each product is usable on its own, so one subscription is enough to start.
|
|
52
|
+
|
|
53
|
+
## Installation
|
|
54
|
+
|
|
55
|
+
```console
|
|
56
|
+
pip install otb-cbso-webservice-pyclient
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Python 3.10 or later.
|
|
60
|
+
|
|
61
|
+
## The five products
|
|
62
|
+
|
|
63
|
+
The CBSO is not one API but five, each a separate subscription on the bank's
|
|
64
|
+
developer portal with a subscription key of its own. A key issued for one will
|
|
65
|
+
not authenticate against another.
|
|
66
|
+
|
|
67
|
+
| Product | What it serves | Key |
|
|
68
|
+
|---------|----------------|-----|
|
|
69
|
+
| `authentic` | One deposit at a time, as filed | `CBSO_KEY_AUTHENTIC` |
|
|
70
|
+
| `extracts` | Daily batches of the accounts as filed | `CBSO_KEY_EXTRACTS` |
|
|
71
|
+
| `authentic-archive` | The same batches, any date in the last three years | `CBSO_KEY_AUTHENTIC_ARCHIVE` |
|
|
72
|
+
| `improved` | The bank's corrections and conversions | `CBSO_KEY_IMPROVED` |
|
|
73
|
+
| `improved-archive` | Improved batches, any date in the last three years | `CBSO_KEY_IMPROVED_ARCHIVE` |
|
|
74
|
+
|
|
75
|
+
A key is read when a product is contacted, not when a client is built. You only
|
|
76
|
+
need keys for the products you actually use, and the error naming a missing one
|
|
77
|
+
arrives at the call that needed it.
|
|
78
|
+
|
|
79
|
+
## Configuration
|
|
80
|
+
|
|
81
|
+
Set the key of each product you hold a subscription for:
|
|
82
|
+
|
|
83
|
+
```console
|
|
84
|
+
export CBSO_KEY_EXTRACTS=...
|
|
85
|
+
export CBSO_KEY_AUTHENTIC=...
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Any key can also be passed in code, which wins over the environment.
|
|
89
|
+
|
|
90
|
+
## Command line
|
|
91
|
+
|
|
92
|
+
```console
|
|
93
|
+
cbso-fetch daily --destination ./data
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Four subcommands:
|
|
97
|
+
|
|
98
|
+
- `daily` fetches recent days from the daily products
|
|
99
|
+
- `backfill` fetches a date range from the archive products
|
|
100
|
+
- `deposit` fetches the documents of one deposit
|
|
101
|
+
- `legal-entity` lists what one enterprise has filed
|
|
102
|
+
|
|
103
|
+
```console
|
|
104
|
+
# The last three days, JSON only, into a local directory
|
|
105
|
+
cbso-fetch daily --destination ./data --days 3 --formats json
|
|
106
|
+
|
|
107
|
+
# A month from the archive, into S3
|
|
108
|
+
cbso-fetch backfill --destination s3://my-bucket/cbso \
|
|
109
|
+
--from 2026-01-01 --to 2026-01-31
|
|
110
|
+
|
|
111
|
+
# See what a run would fetch, without fetching it
|
|
112
|
+
cbso-fetch daily --destination ./data --dry-run
|
|
113
|
+
|
|
114
|
+
# One deposit, printed to stdout. One form at a time: stdout has no way to
|
|
115
|
+
# say where one document ends and the next begins
|
|
116
|
+
cbso-fetch deposit 2021-00000148
|
|
117
|
+
cbso-fetch deposit 2021-00000148 --formats xbrl,pdf --destination ./data
|
|
118
|
+
|
|
119
|
+
# An improvement instead of the account as filed. Written under the name the
|
|
120
|
+
# package reads back, so `-corrected` becomes `-aa-corrected` or `-sb-corrected`
|
|
121
|
+
cbso-fetch deposit 2021-00000148 --improvement corrected --destination ./data
|
|
122
|
+
|
|
123
|
+
# What an enterprise has filed, by enterprise number or VAT number
|
|
124
|
+
cbso-fetch legal-entity 0203201340 --year 2024
|
|
125
|
+
cbso-fetch legal-entity BE0203201340
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Production is the default. `--env test`, or its shorthand `--test`, points at
|
|
129
|
+
the bank's UAT2 deployment instead, which needs its own keys. The two cannot
|
|
130
|
+
both be given. Run with `-v` to see which host was resolved.
|
|
131
|
+
|
|
132
|
+
Exit codes: `0` success, `1` other failure, `2` usage error, `3` a subscription
|
|
133
|
+
key is missing, `4` the service refused, could not be reached, or answered with
|
|
134
|
+
something unreadable — or the destination stopped answering. Only `2` means the
|
|
135
|
+
command itself was wrong, so a wrapper can tell what is worth retrying from what
|
|
136
|
+
needs a person.
|
|
137
|
+
|
|
138
|
+
## Library
|
|
139
|
+
|
|
140
|
+
Each product has a client that stands alone:
|
|
141
|
+
|
|
142
|
+
```python
|
|
143
|
+
from datetime import date
|
|
144
|
+
|
|
145
|
+
from cbso_webservice_client import AuthenticClient, ExtractsClient
|
|
146
|
+
|
|
147
|
+
# One deposit
|
|
148
|
+
accounts = AuthenticClient().accounting_data("2021-00000148")
|
|
149
|
+
|
|
150
|
+
# A whole day, as a zip archive
|
|
151
|
+
archive = ExtractsClient().accounting_data(date(2026, 8, 3))
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Keys can be passed instead of being read from the environment, and the test
|
|
155
|
+
environment is one argument away:
|
|
156
|
+
|
|
157
|
+
```python
|
|
158
|
+
from cbso_webservice_client import Environment, ExtractsClient
|
|
159
|
+
|
|
160
|
+
client = ExtractsClient(key="...", environment=Environment.TEST)
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
### Enterprise numbers
|
|
164
|
+
|
|
165
|
+
The webservices address a legal entity by its KBO/BCE enterprise number, and
|
|
166
|
+
accept nothing else: pass a VAT number and the service refuses it. This package
|
|
167
|
+
takes either, along with the dots and spaces a number is usually written with,
|
|
168
|
+
and converts to the enterprise number before making the request:
|
|
169
|
+
|
|
170
|
+
```python
|
|
171
|
+
from cbso_webservice_client import parse_number
|
|
172
|
+
|
|
173
|
+
parse_number("BE 0203.201.340").value # '0203201340'
|
|
174
|
+
parse_number("0203201340").vat # 'BE0203201340'
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
A number is checked before any request is made: ten ASCII digits, a leading `0`
|
|
178
|
+
or `1`, and check digits equal to `97 - (first eight mod 97)`. A value that
|
|
179
|
+
fails raises `InvalidNumberError` naming the rule it broke, rather than costing
|
|
180
|
+
a round trip answered with a bare 400. Digits from another script are refused
|
|
181
|
+
rather than sent: they satisfy `str.isdigit` without necessarily converting, and
|
|
182
|
+
one that does convert would still be percent-encoded into a URL the service
|
|
183
|
+
cannot answer.
|
|
184
|
+
|
|
185
|
+
`CbsoClient` gathers all five, building each on first use:
|
|
186
|
+
|
|
187
|
+
```python
|
|
188
|
+
from cbso_webservice_client import CbsoClient
|
|
189
|
+
|
|
190
|
+
client = CbsoClient()
|
|
191
|
+
references = client.authentic.legal_entity_references("0203201340")
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
### Reading references
|
|
195
|
+
|
|
196
|
+
A reference is the record tying a deposit to the enterprise that filed it:
|
|
197
|
+
|
|
198
|
+
```python
|
|
199
|
+
from cbso_webservice_client import AuthenticClient
|
|
200
|
+
|
|
201
|
+
reference = AuthenticClient().reference("2021-00000148")
|
|
202
|
+
|
|
203
|
+
reference.enterprise_number # '0203201340'
|
|
204
|
+
reference.exercise_dates # Period(start_date=..., end_date=...)
|
|
205
|
+
reference.model_type.schema_type # SchemaType.FULL
|
|
206
|
+
reference.from_pdf # whether no XBRL exists for it
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
The query operations answer in PascalCase and the JSON inside the batch
|
|
210
|
+
archives is camelCase. Both parse through the same path, so you never have to
|
|
211
|
+
know which one you are holding.
|
|
212
|
+
|
|
213
|
+
### Downloading
|
|
214
|
+
|
|
215
|
+
`Downloader` walks a range of dates and stores everything under a destination:
|
|
216
|
+
|
|
217
|
+
```python
|
|
218
|
+
from cbso_webservice_client.download import Downloader, recent_dates
|
|
219
|
+
from cbso_webservice_client.outputs import writer_for
|
|
220
|
+
|
|
221
|
+
report = Downloader(writer_for("./data")).run(recent_dates(7))
|
|
222
|
+
|
|
223
|
+
print(report.summary())
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
A destination is a local directory or an `s3://bucket/prefix` URL. Documents
|
|
227
|
+
are filed by artefact and enterprise:
|
|
228
|
+
|
|
229
|
+
```text
|
|
230
|
+
references/0203201340/2021-00000148.json
|
|
231
|
+
jsons/0203201340/2021-00000148.json
|
|
232
|
+
xbrls/0203201340/2021-00000148.xbrl
|
|
233
|
+
improved-references/0203201340/2021-00000148-aa-corrected.json
|
|
234
|
+
improved-jsons/0203201340/2021-00000148-aa-corrected.json
|
|
235
|
+
_state/extracts/2026-08-03.zip-jsonxbrl.json
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
The last of those is a marker rather than data: it is what a rerun reads to know
|
|
239
|
+
the day is done. Nothing downstream is expected to depend on this layout.
|
|
240
|
+
|
|
241
|
+
An `s3://` destination needs `s3:GetObject` and `s3:PutObject` beneath the
|
|
242
|
+
prefix. `s3:ListBucket` is not required, but granting it makes the run's
|
|
243
|
+
skipping sharper: without it S3 answers a check for an object that is not there
|
|
244
|
+
with `403` rather than `404`, which cannot be told apart from a check that was
|
|
245
|
+
refused for some other reason. A run establishes which case it is in once, by
|
|
246
|
+
asking whether it may list at all, and then treats `403` accordingly. A
|
|
247
|
+
destination that stops answering altogether stops the run rather than reading as
|
|
248
|
+
empty, since a run treats "not there" as work already done.
|
|
249
|
+
|
|
250
|
+
What a run does is controlled by `DownloadOptions`:
|
|
251
|
+
|
|
252
|
+
```python
|
|
253
|
+
from cbso_webservice_client.api import Representation
|
|
254
|
+
from cbso_webservice_client.download import DownloadOptions, Downloader, recent_dates
|
|
255
|
+
from cbso_webservice_client.outputs import writer_for
|
|
256
|
+
|
|
257
|
+
options = DownloadOptions(
|
|
258
|
+
representations=(Representation.ZIP_JSONXBRL,),
|
|
259
|
+
include_improved=False,
|
|
260
|
+
concurrency=8,
|
|
261
|
+
)
|
|
262
|
+
|
|
263
|
+
Downloader(writer_for("s3://my-bucket/cbso"), options=options).run(recent_dates(7))
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Pass `archive=True` to `run()` to ask the archive products, which reach back
|
|
267
|
+
three years, rather than the daily ones.
|
|
268
|
+
|
|
269
|
+
### Resuming
|
|
270
|
+
|
|
271
|
+
Each batch is recorded as done in the destination, per product, per date and
|
|
272
|
+
per representation. A repeated run skips what it already has, and a run that
|
|
273
|
+
stopped part-way repeats only the step that failed. Because the record lives
|
|
274
|
+
with the data, a run against S3 resumes correctly from a different machine.
|
|
275
|
+
|
|
276
|
+
A day the service had nothing for is recorded as done as well, but only once
|
|
277
|
+
its batches were due: the bank publishes a day from 05:00 Belgian time on the
|
|
278
|
+
following day, so an empty answer before then is asked for again on the next
|
|
279
|
+
run rather than written off. The report says which of the two happened.
|
|
280
|
+
|
|
281
|
+
### When a deposit cannot be attributed
|
|
282
|
+
|
|
283
|
+
Batch files are named after the deposit alone, so the enterprise comes from
|
|
284
|
+
that day's references. A document whose reference is missing has its reference
|
|
285
|
+
fetched on its own; if that finds nothing, the document is set aside under
|
|
286
|
+
`pending/` rather than being lost. It is retried once the run's remaining
|
|
287
|
+
batches have been read, since those are what may now name it, and by later runs
|
|
288
|
+
after that.
|
|
289
|
+
|
|
290
|
+
A document named after an improvement the bank has only just started publishing
|
|
291
|
+
goes there too, since its name still says which deposit it belongs to.
|
|
292
|
+
|
|
293
|
+
What no later run can place is kept under `unfiled/` instead — a reference
|
|
294
|
+
naming no enterprise, a reference that will not parse, an improvement whose kind
|
|
295
|
+
has no name here. Nothing retries those; they are kept because the day is
|
|
296
|
+
recorded as done either way, so a batch entry not written somewhere now is one
|
|
297
|
+
nothing would ever ask for again. Both roots are listed in the report's
|
|
298
|
+
`unresolved`, as the path each record was written to.
|
|
299
|
+
|
|
300
|
+
### Errors
|
|
301
|
+
|
|
302
|
+
Everything the package raises descends from a small set of types:
|
|
303
|
+
|
|
304
|
+
| Exception | Meaning |
|
|
305
|
+
|-----------|---------|
|
|
306
|
+
| `MissingKeyError` | No subscription key for the product being contacted |
|
|
307
|
+
| `NotFoundError` | The service holds nothing at that address |
|
|
308
|
+
| `HttpError` | The service answered with an error status |
|
|
309
|
+
| `UnreachableError` | The service never answered |
|
|
310
|
+
| `ReferenceParseError` | A reference document could not be read |
|
|
311
|
+
| `ArchiveError` | A batch payload could not be read as a zip archive |
|
|
312
|
+
| `InvalidNumberError` | A value is not a well-formed enterprise number |
|
|
313
|
+
| `OutputError` | The destination could not say what it holds |
|
|
314
|
+
|
|
315
|
+
`ArchiveError`, `ReferenceParseError` and `InvalidNumberError` all subclass
|
|
316
|
+
`ValueError`, which is worth knowing if you catch that. Only the last of the
|
|
317
|
+
three means the call was made wrongly; the other two mean what came back could
|
|
318
|
+
not be read, which is why `cbso-fetch` exits `4` for them. `NotFoundError` is a
|
|
319
|
+
subclass of `HttpError`, and both descend from `TransportError`. Server-side statuses and
|
|
320
|
+
connection failures are retried with doubling backoff, honouring a `Retry-After`
|
|
321
|
+
header when the service sends one, before any of them is raised; a missing key
|
|
322
|
+
never is.
|
|
323
|
+
|
|
324
|
+
A key missing for a product a run only wanted to ask about single deposits is
|
|
325
|
+
reported instead of raised: those operations belong to the authentic and improved
|
|
326
|
+
products, and a run holding only the archive keys does less rather than stopping
|
|
327
|
+
on every rerun at the same deposit. The report's `missing` names what went
|
|
328
|
+
unasked.
|
|
329
|
+
|
|
330
|
+
Connections are pooled and kept alive, which matters because filling the gaps a
|
|
331
|
+
batch left makes several requests per deposit. Raise `max_connections` on the
|
|
332
|
+
transport above the `concurrency` a run uses, or the surplus connections are
|
|
333
|
+
opened and discarded again.
|
|
334
|
+
|
|
335
|
+
## Licence
|
|
336
|
+
|
|
337
|
+
Proprietary. © openthebox.
|
|
@@ -0,0 +1,311 @@
|
|
|
1
|
+
# otb-cbso-webservice-pyclient
|
|
2
|
+
|
|
3
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
4
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
5
|
+
[](https://github.com/openthebox/cbso-webservice-pyclient/actions/workflows/ci.yml)
|
|
6
|
+

|
|
7
|
+
[](https://docs.astral.sh/ruff/)
|
|
8
|
+
[](https://mypy-lang.org/)
|
|
9
|
+
|
|
10
|
+
Download annual account data from the Belgian National Bank's Central Balance
|
|
11
|
+
Sheet Office (CBSO) [webservices][webservices], from Python or from the command
|
|
12
|
+
line.
|
|
13
|
+
|
|
14
|
+
The CBSO publishes the annual accounts that Belgian legal entities file, as
|
|
15
|
+
XBRL, JSON-XBRL and PDF, along with the corrections the bank applies to a
|
|
16
|
+
selection of them. This package speaks all five of its products, files what it
|
|
17
|
+
downloads under the enterprise that filed it, and picks up where it left off
|
|
18
|
+
when a run is repeated.
|
|
19
|
+
|
|
20
|
+
[webservices]: https://www.nbb.be/en/central-balance-sheet-office/consultation/web-services
|
|
21
|
+
|
|
22
|
+
- Two dependencies: `urllib3` for pooled, retrying HTTP, and `boto3`, loaded
|
|
23
|
+
only when an `s3://` destination is used.
|
|
24
|
+
- Typed throughout, with a `py.typed` marker.
|
|
25
|
+
- Each product is usable on its own, so one subscription is enough to start.
|
|
26
|
+
|
|
27
|
+
## Installation
|
|
28
|
+
|
|
29
|
+
```console
|
|
30
|
+
pip install otb-cbso-webservice-pyclient
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Python 3.10 or later.
|
|
34
|
+
|
|
35
|
+
## The five products
|
|
36
|
+
|
|
37
|
+
The CBSO is not one API but five, each a separate subscription on the bank's
|
|
38
|
+
developer portal with a subscription key of its own. A key issued for one will
|
|
39
|
+
not authenticate against another.
|
|
40
|
+
|
|
41
|
+
| Product | What it serves | Key |
|
|
42
|
+
|---------|----------------|-----|
|
|
43
|
+
| `authentic` | One deposit at a time, as filed | `CBSO_KEY_AUTHENTIC` |
|
|
44
|
+
| `extracts` | Daily batches of the accounts as filed | `CBSO_KEY_EXTRACTS` |
|
|
45
|
+
| `authentic-archive` | The same batches, any date in the last three years | `CBSO_KEY_AUTHENTIC_ARCHIVE` |
|
|
46
|
+
| `improved` | The bank's corrections and conversions | `CBSO_KEY_IMPROVED` |
|
|
47
|
+
| `improved-archive` | Improved batches, any date in the last three years | `CBSO_KEY_IMPROVED_ARCHIVE` |
|
|
48
|
+
|
|
49
|
+
A key is read when a product is contacted, not when a client is built. You only
|
|
50
|
+
need keys for the products you actually use, and the error naming a missing one
|
|
51
|
+
arrives at the call that needed it.
|
|
52
|
+
|
|
53
|
+
## Configuration
|
|
54
|
+
|
|
55
|
+
Set the key of each product you hold a subscription for:
|
|
56
|
+
|
|
57
|
+
```console
|
|
58
|
+
export CBSO_KEY_EXTRACTS=...
|
|
59
|
+
export CBSO_KEY_AUTHENTIC=...
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Any key can also be passed in code, which wins over the environment.
|
|
63
|
+
|
|
64
|
+
## Command line
|
|
65
|
+
|
|
66
|
+
```console
|
|
67
|
+
cbso-fetch daily --destination ./data
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Four subcommands:
|
|
71
|
+
|
|
72
|
+
- `daily` fetches recent days from the daily products
|
|
73
|
+
- `backfill` fetches a date range from the archive products
|
|
74
|
+
- `deposit` fetches the documents of one deposit
|
|
75
|
+
- `legal-entity` lists what one enterprise has filed
|
|
76
|
+
|
|
77
|
+
```console
|
|
78
|
+
# The last three days, JSON only, into a local directory
|
|
79
|
+
cbso-fetch daily --destination ./data --days 3 --formats json
|
|
80
|
+
|
|
81
|
+
# A month from the archive, into S3
|
|
82
|
+
cbso-fetch backfill --destination s3://my-bucket/cbso \
|
|
83
|
+
--from 2026-01-01 --to 2026-01-31
|
|
84
|
+
|
|
85
|
+
# See what a run would fetch, without fetching it
|
|
86
|
+
cbso-fetch daily --destination ./data --dry-run
|
|
87
|
+
|
|
88
|
+
# One deposit, printed to stdout. One form at a time: stdout has no way to
|
|
89
|
+
# say where one document ends and the next begins
|
|
90
|
+
cbso-fetch deposit 2021-00000148
|
|
91
|
+
cbso-fetch deposit 2021-00000148 --formats xbrl,pdf --destination ./data
|
|
92
|
+
|
|
93
|
+
# An improvement instead of the account as filed. Written under the name the
|
|
94
|
+
# package reads back, so `-corrected` becomes `-aa-corrected` or `-sb-corrected`
|
|
95
|
+
cbso-fetch deposit 2021-00000148 --improvement corrected --destination ./data
|
|
96
|
+
|
|
97
|
+
# What an enterprise has filed, by enterprise number or VAT number
|
|
98
|
+
cbso-fetch legal-entity 0203201340 --year 2024
|
|
99
|
+
cbso-fetch legal-entity BE0203201340
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Production is the default. `--env test`, or its shorthand `--test`, points at
|
|
103
|
+
the bank's UAT2 deployment instead, which needs its own keys. The two cannot
|
|
104
|
+
both be given. Run with `-v` to see which host was resolved.
|
|
105
|
+
|
|
106
|
+
Exit codes: `0` success, `1` other failure, `2` usage error, `3` a subscription
|
|
107
|
+
key is missing, `4` the service refused, could not be reached, or answered with
|
|
108
|
+
something unreadable — or the destination stopped answering. Only `2` means the
|
|
109
|
+
command itself was wrong, so a wrapper can tell what is worth retrying from what
|
|
110
|
+
needs a person.
|
|
111
|
+
|
|
112
|
+
## Library
|
|
113
|
+
|
|
114
|
+
Each product has a client that stands alone:
|
|
115
|
+
|
|
116
|
+
```python
|
|
117
|
+
from datetime import date
|
|
118
|
+
|
|
119
|
+
from cbso_webservice_client import AuthenticClient, ExtractsClient
|
|
120
|
+
|
|
121
|
+
# One deposit
|
|
122
|
+
accounts = AuthenticClient().accounting_data("2021-00000148")
|
|
123
|
+
|
|
124
|
+
# A whole day, as a zip archive
|
|
125
|
+
archive = ExtractsClient().accounting_data(date(2026, 8, 3))
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Keys can be passed instead of being read from the environment, and the test
|
|
129
|
+
environment is one argument away:
|
|
130
|
+
|
|
131
|
+
```python
|
|
132
|
+
from cbso_webservice_client import Environment, ExtractsClient
|
|
133
|
+
|
|
134
|
+
client = ExtractsClient(key="...", environment=Environment.TEST)
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### Enterprise numbers
|
|
138
|
+
|
|
139
|
+
The webservices address a legal entity by its KBO/BCE enterprise number, and
|
|
140
|
+
accept nothing else: pass a VAT number and the service refuses it. This package
|
|
141
|
+
takes either, along with the dots and spaces a number is usually written with,
|
|
142
|
+
and converts to the enterprise number before making the request:
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
from cbso_webservice_client import parse_number
|
|
146
|
+
|
|
147
|
+
parse_number("BE 0203.201.340").value # '0203201340'
|
|
148
|
+
parse_number("0203201340").vat # 'BE0203201340'
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
A number is checked before any request is made: ten ASCII digits, a leading `0`
|
|
152
|
+
or `1`, and check digits equal to `97 - (first eight mod 97)`. A value that
|
|
153
|
+
fails raises `InvalidNumberError` naming the rule it broke, rather than costing
|
|
154
|
+
a round trip answered with a bare 400. Digits from another script are refused
|
|
155
|
+
rather than sent: they satisfy `str.isdigit` without necessarily converting, and
|
|
156
|
+
one that does convert would still be percent-encoded into a URL the service
|
|
157
|
+
cannot answer.
|
|
158
|
+
|
|
159
|
+
`CbsoClient` gathers all five, building each on first use:
|
|
160
|
+
|
|
161
|
+
```python
|
|
162
|
+
from cbso_webservice_client import CbsoClient
|
|
163
|
+
|
|
164
|
+
client = CbsoClient()
|
|
165
|
+
references = client.authentic.legal_entity_references("0203201340")
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Reading references
|
|
169
|
+
|
|
170
|
+
A reference is the record tying a deposit to the enterprise that filed it:
|
|
171
|
+
|
|
172
|
+
```python
|
|
173
|
+
from cbso_webservice_client import AuthenticClient
|
|
174
|
+
|
|
175
|
+
reference = AuthenticClient().reference("2021-00000148")
|
|
176
|
+
|
|
177
|
+
reference.enterprise_number # '0203201340'
|
|
178
|
+
reference.exercise_dates # Period(start_date=..., end_date=...)
|
|
179
|
+
reference.model_type.schema_type # SchemaType.FULL
|
|
180
|
+
reference.from_pdf # whether no XBRL exists for it
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
The query operations answer in PascalCase and the JSON inside the batch
|
|
184
|
+
archives is camelCase. Both parse through the same path, so you never have to
|
|
185
|
+
know which one you are holding.
|
|
186
|
+
|
|
187
|
+
### Downloading
|
|
188
|
+
|
|
189
|
+
`Downloader` walks a range of dates and stores everything under a destination:
|
|
190
|
+
|
|
191
|
+
```python
|
|
192
|
+
from cbso_webservice_client.download import Downloader, recent_dates
|
|
193
|
+
from cbso_webservice_client.outputs import writer_for
|
|
194
|
+
|
|
195
|
+
report = Downloader(writer_for("./data")).run(recent_dates(7))
|
|
196
|
+
|
|
197
|
+
print(report.summary())
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
A destination is a local directory or an `s3://bucket/prefix` URL. Documents
|
|
201
|
+
are filed by artefact and enterprise:
|
|
202
|
+
|
|
203
|
+
```text
|
|
204
|
+
references/0203201340/2021-00000148.json
|
|
205
|
+
jsons/0203201340/2021-00000148.json
|
|
206
|
+
xbrls/0203201340/2021-00000148.xbrl
|
|
207
|
+
improved-references/0203201340/2021-00000148-aa-corrected.json
|
|
208
|
+
improved-jsons/0203201340/2021-00000148-aa-corrected.json
|
|
209
|
+
_state/extracts/2026-08-03.zip-jsonxbrl.json
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
The last of those is a marker rather than data: it is what a rerun reads to know
|
|
213
|
+
the day is done. Nothing downstream is expected to depend on this layout.
|
|
214
|
+
|
|
215
|
+
An `s3://` destination needs `s3:GetObject` and `s3:PutObject` beneath the
|
|
216
|
+
prefix. `s3:ListBucket` is not required, but granting it makes the run's
|
|
217
|
+
skipping sharper: without it S3 answers a check for an object that is not there
|
|
218
|
+
with `403` rather than `404`, which cannot be told apart from a check that was
|
|
219
|
+
refused for some other reason. A run establishes which case it is in once, by
|
|
220
|
+
asking whether it may list at all, and then treats `403` accordingly. A
|
|
221
|
+
destination that stops answering altogether stops the run rather than reading as
|
|
222
|
+
empty, since a run treats "not there" as work already done.
|
|
223
|
+
|
|
224
|
+
What a run does is controlled by `DownloadOptions`:
|
|
225
|
+
|
|
226
|
+
```python
|
|
227
|
+
from cbso_webservice_client.api import Representation
|
|
228
|
+
from cbso_webservice_client.download import DownloadOptions, Downloader, recent_dates
|
|
229
|
+
from cbso_webservice_client.outputs import writer_for
|
|
230
|
+
|
|
231
|
+
options = DownloadOptions(
|
|
232
|
+
representations=(Representation.ZIP_JSONXBRL,),
|
|
233
|
+
include_improved=False,
|
|
234
|
+
concurrency=8,
|
|
235
|
+
)
|
|
236
|
+
|
|
237
|
+
Downloader(writer_for("s3://my-bucket/cbso"), options=options).run(recent_dates(7))
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
Pass `archive=True` to `run()` to ask the archive products, which reach back
|
|
241
|
+
three years, rather than the daily ones.
|
|
242
|
+
|
|
243
|
+
### Resuming
|
|
244
|
+
|
|
245
|
+
Each batch is recorded as done in the destination, per product, per date and
|
|
246
|
+
per representation. A repeated run skips what it already has, and a run that
|
|
247
|
+
stopped part-way repeats only the step that failed. Because the record lives
|
|
248
|
+
with the data, a run against S3 resumes correctly from a different machine.
|
|
249
|
+
|
|
250
|
+
A day the service had nothing for is recorded as done as well, but only once
|
|
251
|
+
its batches were due: the bank publishes a day from 05:00 Belgian time on the
|
|
252
|
+
following day, so an empty answer before then is asked for again on the next
|
|
253
|
+
run rather than written off. The report says which of the two happened.
|
|
254
|
+
|
|
255
|
+
### When a deposit cannot be attributed
|
|
256
|
+
|
|
257
|
+
Batch files are named after the deposit alone, so the enterprise comes from
|
|
258
|
+
that day's references. A document whose reference is missing has its reference
|
|
259
|
+
fetched on its own; if that finds nothing, the document is set aside under
|
|
260
|
+
`pending/` rather than being lost. It is retried once the run's remaining
|
|
261
|
+
batches have been read, since those are what may now name it, and by later runs
|
|
262
|
+
after that.
|
|
263
|
+
|
|
264
|
+
A document named after an improvement the bank has only just started publishing
|
|
265
|
+
goes there too, since its name still says which deposit it belongs to.
|
|
266
|
+
|
|
267
|
+
What no later run can place is kept under `unfiled/` instead — a reference
|
|
268
|
+
naming no enterprise, a reference that will not parse, an improvement whose kind
|
|
269
|
+
has no name here. Nothing retries those; they are kept because the day is
|
|
270
|
+
recorded as done either way, so a batch entry not written somewhere now is one
|
|
271
|
+
nothing would ever ask for again. Both roots are listed in the report's
|
|
272
|
+
`unresolved`, as the path each record was written to.
|
|
273
|
+
|
|
274
|
+
### Errors
|
|
275
|
+
|
|
276
|
+
Everything the package raises descends from a small set of types:
|
|
277
|
+
|
|
278
|
+
| Exception | Meaning |
|
|
279
|
+
|-----------|---------|
|
|
280
|
+
| `MissingKeyError` | No subscription key for the product being contacted |
|
|
281
|
+
| `NotFoundError` | The service holds nothing at that address |
|
|
282
|
+
| `HttpError` | The service answered with an error status |
|
|
283
|
+
| `UnreachableError` | The service never answered |
|
|
284
|
+
| `ReferenceParseError` | A reference document could not be read |
|
|
285
|
+
| `ArchiveError` | A batch payload could not be read as a zip archive |
|
|
286
|
+
| `InvalidNumberError` | A value is not a well-formed enterprise number |
|
|
287
|
+
| `OutputError` | The destination could not say what it holds |
|
|
288
|
+
|
|
289
|
+
`ArchiveError`, `ReferenceParseError` and `InvalidNumberError` all subclass
|
|
290
|
+
`ValueError`, which is worth knowing if you catch that. Only the last of the
|
|
291
|
+
three means the call was made wrongly; the other two mean what came back could
|
|
292
|
+
not be read, which is why `cbso-fetch` exits `4` for them. `NotFoundError` is a
|
|
293
|
+
subclass of `HttpError`, and both descend from `TransportError`. Server-side statuses and
|
|
294
|
+
connection failures are retried with doubling backoff, honouring a `Retry-After`
|
|
295
|
+
header when the service sends one, before any of them is raised; a missing key
|
|
296
|
+
never is.
|
|
297
|
+
|
|
298
|
+
A key missing for a product a run only wanted to ask about single deposits is
|
|
299
|
+
reported instead of raised: those operations belong to the authentic and improved
|
|
300
|
+
products, and a run holding only the archive keys does less rather than stopping
|
|
301
|
+
on every rerun at the same deposit. The report's `missing` names what went
|
|
302
|
+
unasked.
|
|
303
|
+
|
|
304
|
+
Connections are pooled and kept alive, which matters because filling the gaps a
|
|
305
|
+
batch left makes several requests per deposit. Raise `max_connections` on the
|
|
306
|
+
transport above the `concurrency` a run uses, or the surplus connections are
|
|
307
|
+
opened and discarded again.
|
|
308
|
+
|
|
309
|
+
## Licence
|
|
310
|
+
|
|
311
|
+
Proprietary. © openthebox.
|