astro-colibri 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- astro_colibri-1.0.0/CITATION.cff +41 -0
- astro_colibri-1.0.0/COMMERCIAL-LICENSE.md +17 -0
- astro_colibri-1.0.0/LICENSE +75 -0
- astro_colibri-1.0.0/MANIFEST.in +4 -0
- astro_colibri-1.0.0/PKG-INFO +261 -0
- astro_colibri-1.0.0/README.md +227 -0
- astro_colibri-1.0.0/astro_colibri.egg-info/PKG-INFO +261 -0
- astro_colibri-1.0.0/astro_colibri.egg-info/SOURCES.txt +18 -0
- astro_colibri-1.0.0/astro_colibri.egg-info/dependency_links.txt +1 -0
- astro_colibri-1.0.0/astro_colibri.egg-info/requires.txt +8 -0
- astro_colibri-1.0.0/astro_colibri.egg-info/top_level.txt +1 -0
- astro_colibri-1.0.0/astrocolibri/__init__.py +40 -0
- astro_colibri-1.0.0/astrocolibri/_version.py +1 -0
- astro_colibri-1.0.0/astrocolibri/consumer.py +336 -0
- astro_colibri-1.0.0/astrocolibri/exceptions.py +49 -0
- astro_colibri-1.0.0/pyproject.toml +54 -0
- astro_colibri-1.0.0/setup.cfg +4 -0
- astro_colibri-1.0.0/tests/test_consumer.py +293 -0
- astro_colibri-1.0.0/tests/test_integration_broker.py +197 -0
- astro_colibri-1.0.0/tests/test_package.py +106 -0
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
cff-version: 1.2.0
|
|
2
|
+
type: software
|
|
3
|
+
message: "If you use this software, please cite it and the Astro-COLIBRI platform paper."
|
|
4
|
+
title: "astro-colibri"
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
date-released: 2026-08-30
|
|
7
|
+
abstract: "Python SDK for consuming Astro-COLIBRI real-time astronomical alerts and accessing Astro-COLIBRI services."
|
|
8
|
+
authors:
|
|
9
|
+
- name: "Astro-COLIBRI team"
|
|
10
|
+
repository-code: "https://github.com/astro-transients/astro-colibri"
|
|
11
|
+
url: "https://astro-colibri.com"
|
|
12
|
+
license: "PolyForm-Noncommercial-1.0.0"
|
|
13
|
+
keywords:
|
|
14
|
+
- "astronomy"
|
|
15
|
+
- "multi-messenger astronomy"
|
|
16
|
+
- "transient astronomy"
|
|
17
|
+
- "Kafka"
|
|
18
|
+
- "VOEvent"
|
|
19
|
+
|
|
20
|
+
preferred-citation:
|
|
21
|
+
type: article
|
|
22
|
+
title: "Astro-COLIBRI 2 - An Advanced Platform for Real-Time Multi-Messenger Discoveries"
|
|
23
|
+
authors:
|
|
24
|
+
- family-names: "Reichherzer"
|
|
25
|
+
given-names: "Patrick"
|
|
26
|
+
- family-names: "Schüssler"
|
|
27
|
+
given-names: "Fabian"
|
|
28
|
+
- family-names: "Lefranc"
|
|
29
|
+
given-names: "Valentin"
|
|
30
|
+
- family-names: "Becker Tjus"
|
|
31
|
+
given-names: "Julia"
|
|
32
|
+
- family-names: "Mourier"
|
|
33
|
+
given-names: "Jayson"
|
|
34
|
+
- family-names: "Alkan"
|
|
35
|
+
given-names: "Atilla Kaan"
|
|
36
|
+
journal: "Galaxies"
|
|
37
|
+
year: 2023
|
|
38
|
+
volume: 11
|
|
39
|
+
issue: 1
|
|
40
|
+
doi: "10.3390/galaxies11010022"
|
|
41
|
+
url: "https://doi.org/10.3390/galaxies11010022"
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Commercial Licensing
|
|
2
|
+
|
|
3
|
+
The Astro-COLIBRI Python SDK is publicly available under the PolyForm
|
|
4
|
+
Noncommercial License 1.0.0. That license does not grant rights for commercial
|
|
5
|
+
use.
|
|
6
|
+
|
|
7
|
+
To request a separate commercial license from the Astro-COLIBRI project,
|
|
8
|
+
contact:
|
|
9
|
+
|
|
10
|
+
**Astro-COLIBRI (Fabian Schüssler)**
|
|
11
|
+
<astro.colibri@gmail.com>
|
|
12
|
+
|
|
13
|
+
A request or discussion does not grant commercial-use rights. Commercial use
|
|
14
|
+
is permitted only after a separate written license has been explicitly granted
|
|
15
|
+
by the licensor.
|
|
16
|
+
|
|
17
|
+
Older releases remain governed by the license included with those releases.
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# PolyForm Noncommercial License 1.0.0
|
|
2
|
+
|
|
3
|
+
<https://polyformproject.org/licenses/noncommercial/1.0.0>
|
|
4
|
+
|
|
5
|
+
## Acceptance
|
|
6
|
+
|
|
7
|
+
In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
|
|
8
|
+
|
|
9
|
+
## Copyright License
|
|
10
|
+
|
|
11
|
+
The licensor grants you a copyright license for the software to do everything you might do with the software that would otherwise infringe the licensor's copyright in it for any permitted purpose. However, you may only distribute the software according to [Distribution License](#distribution-license) and make changes or new works based on the software according to [Changes and New Works License](#changes-and-new-works-license).
|
|
12
|
+
|
|
13
|
+
## Distribution License
|
|
14
|
+
|
|
15
|
+
The licensor grants you an additional copyright license to distribute copies of the software. Your license to distribute covers distributing the software with changes and new works permitted by [Changes and New Works License](#changes-and-new-works-license).
|
|
16
|
+
|
|
17
|
+
## Notices
|
|
18
|
+
|
|
19
|
+
You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms or the URL for them above, as well as copies of any plain-text lines beginning with `Required Notice:` that the licensor provided with the software. For example:
|
|
20
|
+
|
|
21
|
+
> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
|
|
22
|
+
|
|
23
|
+
## Changes and New Works License
|
|
24
|
+
|
|
25
|
+
The licensor grants you an additional copyright license to make changes and new works based on the software for any permitted purpose.
|
|
26
|
+
|
|
27
|
+
## Patent License
|
|
28
|
+
|
|
29
|
+
The licensor grants you a patent license for the software that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the software.
|
|
30
|
+
|
|
31
|
+
## Noncommercial Purposes
|
|
32
|
+
|
|
33
|
+
Any noncommercial purpose is a permitted purpose.
|
|
34
|
+
|
|
35
|
+
## Personal Uses
|
|
36
|
+
|
|
37
|
+
Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious observance, without any anticipated commercial application, is use for a permitted purpose.
|
|
38
|
+
|
|
39
|
+
## Noncommercial Organizations
|
|
40
|
+
|
|
41
|
+
Use by any charitable organization, educational institution, public research organization, public safety or health organization, environmental protection organization, or government institution is use for a permitted purpose regardless of the source of funding or obligations resulting from the funding.
|
|
42
|
+
|
|
43
|
+
## Fair Use
|
|
44
|
+
|
|
45
|
+
You may have "fair use" rights for the software under the law. These terms do not limit them.
|
|
46
|
+
|
|
47
|
+
## No Other Rights
|
|
48
|
+
|
|
49
|
+
These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply any other licenses.
|
|
50
|
+
|
|
51
|
+
## Patent Defense
|
|
52
|
+
|
|
53
|
+
If you make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
|
|
54
|
+
|
|
55
|
+
## Violations
|
|
56
|
+
|
|
57
|
+
The first time you are notified in writing that you have violated any of these terms, or done anything with the software not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 32 days of receiving notice. Otherwise, all your licenses end immediately.
|
|
58
|
+
|
|
59
|
+
## No Liability
|
|
60
|
+
|
|
61
|
+
***As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.***
|
|
62
|
+
|
|
63
|
+
## Definitions
|
|
64
|
+
|
|
65
|
+
The **licensor** is the individual or entity offering these terms, and the **software** is the software the licensor makes available under these terms.
|
|
66
|
+
|
|
67
|
+
**You** refers to the individual or entity agreeing to these terms.
|
|
68
|
+
|
|
69
|
+
**Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
|
|
70
|
+
|
|
71
|
+
**Your licenses** are all the licenses granted to you for the software under these terms.
|
|
72
|
+
|
|
73
|
+
**Use** means anything you do with the software requiring one of your licenses.
|
|
74
|
+
|
|
75
|
+
Required Notice: Copyright 2026 Fabian Schüssler (Astro-COLIBRI project). Commercial licensing: astro.colibri@gmail.com
|
|
@@ -0,0 +1,261 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: astro-colibri
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Python SDK for Astro-COLIBRI alerts and services
|
|
5
|
+
Author-email: Astro-Colibri Team <astro.colibri@gmail.com>
|
|
6
|
+
License-Expression: PolyForm-Noncommercial-1.0.0
|
|
7
|
+
Project-URL: Homepage, https://astro-colibri.com
|
|
8
|
+
Project-URL: Documentation, https://astro-colibri.science/brokerdoc
|
|
9
|
+
Project-URL: Repository, https://github.com/astro-transients/astro-colibri
|
|
10
|
+
Project-URL: Bug Tracker, https://github.com/astro-transients/astro-colibri/issues
|
|
11
|
+
Keywords: astronomy,alerts,kafka,broker,voevent,transients
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: Topic :: Scientific/Engineering :: Astronomy
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
+
Classifier: Operating System :: OS Independent
|
|
22
|
+
Requires-Python: >=3.9
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
License-File: COMMERCIAL-LICENSE.md
|
|
26
|
+
Requires-Dist: confluent-kafka>=2.3.0
|
|
27
|
+
Provides-Extra: dev
|
|
28
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
29
|
+
Requires-Dist: pytest>=7.4; extra == "dev"
|
|
30
|
+
Requires-Dist: pytest-cov>=4.1; extra == "dev"
|
|
31
|
+
Requires-Dist: requests>=2.31; extra == "dev"
|
|
32
|
+
Requires-Dist: twine>=5; extra == "dev"
|
|
33
|
+
Dynamic: license-file
|
|
34
|
+
|
|
35
|
+
# Astro-COLIBRI Python SDK
|
|
36
|
+
|
|
37
|
+
The `astro-colibri` distribution provides the `astrocolibri` Python package,
|
|
38
|
+
the SDK for **Astro-COLIBRI**. Its first public capability is the astronomical
|
|
39
|
+
alert-broker consumer; future modules will add supported access to the main API
|
|
40
|
+
and shared event models.
|
|
41
|
+
|
|
42
|
+
Receive multi-messenger astrophysics alerts in real time — JSON or
|
|
43
|
+
VOEvent/XML — directly from the Astro-Colibri broker.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## Installation
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pip install astro-colibri
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Requires Python 3.9+ and `confluent-kafka` (installed automatically).
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Quick start
|
|
58
|
+
|
|
59
|
+
### 1. Get your credentials
|
|
60
|
+
|
|
61
|
+
Request broker access from your account page on
|
|
62
|
+
[astro-colibri.com](https://astro-colibri.com). Your SCRAM `username` and
|
|
63
|
+
`password` are available under **Manage broker access**.
|
|
64
|
+
|
|
65
|
+
### 2. Subscribe and receive alerts
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
import json
|
|
69
|
+
from astrocolibri import Consumer
|
|
70
|
+
|
|
71
|
+
with Consumer(
|
|
72
|
+
username="your-username",
|
|
73
|
+
password="your-password",
|
|
74
|
+
) as consumer:
|
|
75
|
+
consumer.subscribe(["astrocolibri.all.JSON"])
|
|
76
|
+
|
|
77
|
+
for message in consumer.consume(timeout=30):
|
|
78
|
+
alert = json.loads(message.value())
|
|
79
|
+
print(f"Alert received: {alert['id']} — RA={alert['ra']}, Dec={alert['dec']}")
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Example script
|
|
85
|
+
|
|
86
|
+
A minimal, ready-to-run version of the snippet above is provided in
|
|
87
|
+
[`example.py`](example.py). It connects with your credentials, subscribes to
|
|
88
|
+
all topics and prints each alert as it arrives — the fastest way
|
|
89
|
+
to check that your setup works before wiring the SDK into your own pipeline.
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
python example.py
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Available topics
|
|
98
|
+
|
|
99
|
+
| Topic | Description |
|
|
100
|
+
|---|---|
|
|
101
|
+
| `astrocolibri.all.JSON` | Every alert, Astro-Colibri JSON format |
|
|
102
|
+
| `astrocolibri.all.VOEvent` | Every alert, VOEvent/XML format |
|
|
103
|
+
| `astrocolibri.important.JSON` | Important alerts, Astro-Colibri JSON format |
|
|
104
|
+
| `astrocolibri.important.VOEvent` | Important alerts, VOEvent/XML format |
|
|
105
|
+
| `astrocolibri.heartbeat` | Pipeline liveness message, JSON format |
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## API reference
|
|
110
|
+
|
|
111
|
+
### `Consumer`
|
|
112
|
+
|
|
113
|
+
```python
|
|
114
|
+
Consumer(
|
|
115
|
+
username: str,
|
|
116
|
+
password: str,
|
|
117
|
+
*,
|
|
118
|
+
broker_url: str | None = None,
|
|
119
|
+
group_id: str | None = None,
|
|
120
|
+
start_at: str = "earliest", # "earliest" | "latest"
|
|
121
|
+
security_protocol: str = "SASL_SSL",
|
|
122
|
+
config: dict | None = None, # advanced confluent-kafka options
|
|
123
|
+
)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
#### `consumer.subscribe(topics, *, on_assign=None, on_revoke=None)`
|
|
127
|
+
|
|
128
|
+
Subscribe to a list of topics.
|
|
129
|
+
|
|
130
|
+
#### `consumer.consume(num_messages=1, timeout=-1)`
|
|
131
|
+
|
|
132
|
+
Message generator.
|
|
133
|
+
|
|
134
|
+
- `timeout=-1` (default): blocks until the next message — infinite loop.
|
|
135
|
+
- `timeout=N` (seconds): returns after N seconds with no message.
|
|
136
|
+
|
|
137
|
+
```python
|
|
138
|
+
# Infinite loop
|
|
139
|
+
for message in consumer.consume():
|
|
140
|
+
handle(message.value())
|
|
141
|
+
|
|
142
|
+
# With a timeout (lets you do other work between batches)
|
|
143
|
+
while True:
|
|
144
|
+
for message in consumer.consume(timeout=5.0):
|
|
145
|
+
handle(message.value())
|
|
146
|
+
check_app_state()
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
#### `consumer.close()`
|
|
150
|
+
|
|
151
|
+
Cleanly closes the connection (called automatically by the context manager).
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Your read position
|
|
156
|
+
|
|
157
|
+
Your read position (offset) is always persisted. With no `group_id`, the client
|
|
158
|
+
joins the consumer group `<your-username>.default`, so restarting a program
|
|
159
|
+
resumes exactly where it left off: nothing is re-read, nothing is missed.
|
|
160
|
+
|
|
161
|
+
`start_at` only applies the **first** time a given consumer group connects. On
|
|
162
|
+
every later run the stored offset wins, so changing `start_at` on an existing
|
|
163
|
+
group has no effect. To deliberately re-read the retention window, use a
|
|
164
|
+
`group_id` you have never used before.
|
|
165
|
+
|
|
166
|
+
The client automatically prefixes `group_id` with your Kafka username to satisfy
|
|
167
|
+
the per-user ACL, so `group_id="my-program-v1"` becomes the Kafka group
|
|
168
|
+
`your-username.my-program-v1`.
|
|
169
|
+
|
|
170
|
+
```python
|
|
171
|
+
consumer = Consumer(
|
|
172
|
+
username="your-username",
|
|
173
|
+
password="your-password",
|
|
174
|
+
group_id="my-program-v1", # its own independent read position
|
|
175
|
+
start_at="latest", # only applies on this group's very first run
|
|
176
|
+
)
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## Running several scripts with the same credentials
|
|
182
|
+
|
|
183
|
+
One set of credentials can drive as many scripts as you like, but **give each
|
|
184
|
+
script its own `group_id`**. Consumers that share a group are treated by Kafka
|
|
185
|
+
as one logical reader and have the partitions divided between them, so each
|
|
186
|
+
script would receive only a slice of the stream rather than every alert.
|
|
187
|
+
|
|
188
|
+
```python
|
|
189
|
+
# ingest.py
|
|
190
|
+
consumer = Consumer(username="alice", password="...", group_id="ingest")
|
|
191
|
+
|
|
192
|
+
# alerting.py
|
|
193
|
+
consumer = Consumer(username="alice", password="...", group_id="alerting")
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Each group keeps its own independent read position, so the two scripts can run
|
|
197
|
+
at different speeds, restart independently, and both still see the full alert
|
|
198
|
+
stream.
|
|
199
|
+
|
|
200
|
+
Leaving `group_id` unset in more than one script is the case to avoid: they all
|
|
201
|
+
land in `<your-username>.default` and silently share the stream between them.
|
|
202
|
+
|
|
203
|
+
Running the *same* script as several replicas is the one case where sharing a
|
|
204
|
+
`group_id` is what you want: that is how you spread the load, and Kafka
|
|
205
|
+
rebalances the partitions across the replicas automatically.
|
|
206
|
+
|
|
207
|
+
---
|
|
208
|
+
|
|
209
|
+
## Testing locally against your own broker
|
|
210
|
+
|
|
211
|
+
If you're running the Astro-Colibri broker stack locally (see the
|
|
212
|
+
broker's `QUICKSTART.md`), point the client at it directly:
|
|
213
|
+
|
|
214
|
+
```python
|
|
215
|
+
consumer = Consumer(
|
|
216
|
+
username="alice",
|
|
217
|
+
password="alice-strong-password",
|
|
218
|
+
broker_url="localhost:9092",
|
|
219
|
+
security_protocol="SASL_PLAINTEXT", # local trusted broker only
|
|
220
|
+
)
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Install the package in editable mode from the repository root for
|
|
224
|
+
development:
|
|
225
|
+
|
|
226
|
+
```bash
|
|
227
|
+
cd Colibri_v2/colibri_client
|
|
228
|
+
pip install -e ".[dev]"
|
|
229
|
+
pytest tests/ -v
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
## Citation
|
|
235
|
+
|
|
236
|
+
Please cite the software release using [`CITATION.cff`](CITATION.cff) and the
|
|
237
|
+
Astro-COLIBRI platform papers:
|
|
238
|
+
|
|
239
|
+
- Reichherzer et al. (2023), *Astro-COLIBRI 2 - An Advanced Platform for
|
|
240
|
+
Real-Time Multi-Messenger Discoveries*, Galaxies 11, 22,
|
|
241
|
+
[doi:10.3390/galaxies11010022](https://doi.org/10.3390/galaxies11010022).
|
|
242
|
+
- Reichherzer et al. (2021), *Astro-COLIBRI - The COincidence LIBrary for
|
|
243
|
+
Real-time Inquiry for Multimessenger Astrophysics*, ApJS 256, 5,
|
|
244
|
+
[doi:10.3847/1538-4365/ac1517](https://doi.org/10.3847/1538-4365/ac1517).
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## License
|
|
249
|
+
|
|
250
|
+
This source-available software is licensed under the
|
|
251
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). It may be used, modified, and
|
|
252
|
+
redistributed for permitted noncommercial purposes, including use by
|
|
253
|
+
educational institutions and public research organizations.
|
|
254
|
+
|
|
255
|
+
Commercial use requires a separate written license. Contact
|
|
256
|
+
[Astro-COLIBRI (Fabian Schüssler)](mailto:astro.colibri@gmail.com) and see
|
|
257
|
+
[COMMERCIAL-LICENSE.md](COMMERCIAL-LICENSE.md).
|
|
258
|
+
|
|
259
|
+
Use of Astro-COLIBRI hosted services, including the Kafka broker and its data,
|
|
260
|
+
is governed separately by the
|
|
261
|
+
[Astro-COLIBRI Terms of Service](https://astro-colibri.science/tos).
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# Astro-COLIBRI Python SDK
|
|
2
|
+
|
|
3
|
+
The `astro-colibri` distribution provides the `astrocolibri` Python package,
|
|
4
|
+
the SDK for **Astro-COLIBRI**. Its first public capability is the astronomical
|
|
5
|
+
alert-broker consumer; future modules will add supported access to the main API
|
|
6
|
+
and shared event models.
|
|
7
|
+
|
|
8
|
+
Receive multi-messenger astrophysics alerts in real time — JSON or
|
|
9
|
+
VOEvent/XML — directly from the Astro-Colibri broker.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Installation
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
pip install astro-colibri
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Requires Python 3.9+ and `confluent-kafka` (installed automatically).
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Quick start
|
|
24
|
+
|
|
25
|
+
### 1. Get your credentials
|
|
26
|
+
|
|
27
|
+
Request broker access from your account page on
|
|
28
|
+
[astro-colibri.com](https://astro-colibri.com). Your SCRAM `username` and
|
|
29
|
+
`password` are available under **Manage broker access**.
|
|
30
|
+
|
|
31
|
+
### 2. Subscribe and receive alerts
|
|
32
|
+
|
|
33
|
+
```python
|
|
34
|
+
import json
|
|
35
|
+
from astrocolibri import Consumer
|
|
36
|
+
|
|
37
|
+
with Consumer(
|
|
38
|
+
username="your-username",
|
|
39
|
+
password="your-password",
|
|
40
|
+
) as consumer:
|
|
41
|
+
consumer.subscribe(["astrocolibri.all.JSON"])
|
|
42
|
+
|
|
43
|
+
for message in consumer.consume(timeout=30):
|
|
44
|
+
alert = json.loads(message.value())
|
|
45
|
+
print(f"Alert received: {alert['id']} — RA={alert['ra']}, Dec={alert['dec']}")
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Example script
|
|
51
|
+
|
|
52
|
+
A minimal, ready-to-run version of the snippet above is provided in
|
|
53
|
+
[`example.py`](example.py). It connects with your credentials, subscribes to
|
|
54
|
+
all topics and prints each alert as it arrives — the fastest way
|
|
55
|
+
to check that your setup works before wiring the SDK into your own pipeline.
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
python example.py
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Available topics
|
|
64
|
+
|
|
65
|
+
| Topic | Description |
|
|
66
|
+
|---|---|
|
|
67
|
+
| `astrocolibri.all.JSON` | Every alert, Astro-Colibri JSON format |
|
|
68
|
+
| `astrocolibri.all.VOEvent` | Every alert, VOEvent/XML format |
|
|
69
|
+
| `astrocolibri.important.JSON` | Important alerts, Astro-Colibri JSON format |
|
|
70
|
+
| `astrocolibri.important.VOEvent` | Important alerts, VOEvent/XML format |
|
|
71
|
+
| `astrocolibri.heartbeat` | Pipeline liveness message, JSON format |
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## API reference
|
|
76
|
+
|
|
77
|
+
### `Consumer`
|
|
78
|
+
|
|
79
|
+
```python
|
|
80
|
+
Consumer(
|
|
81
|
+
username: str,
|
|
82
|
+
password: str,
|
|
83
|
+
*,
|
|
84
|
+
broker_url: str | None = None,
|
|
85
|
+
group_id: str | None = None,
|
|
86
|
+
start_at: str = "earliest", # "earliest" | "latest"
|
|
87
|
+
security_protocol: str = "SASL_SSL",
|
|
88
|
+
config: dict | None = None, # advanced confluent-kafka options
|
|
89
|
+
)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
#### `consumer.subscribe(topics, *, on_assign=None, on_revoke=None)`
|
|
93
|
+
|
|
94
|
+
Subscribe to a list of topics.
|
|
95
|
+
|
|
96
|
+
#### `consumer.consume(num_messages=1, timeout=-1)`
|
|
97
|
+
|
|
98
|
+
Message generator.
|
|
99
|
+
|
|
100
|
+
- `timeout=-1` (default): blocks until the next message — infinite loop.
|
|
101
|
+
- `timeout=N` (seconds): returns after N seconds with no message.
|
|
102
|
+
|
|
103
|
+
```python
|
|
104
|
+
# Infinite loop
|
|
105
|
+
for message in consumer.consume():
|
|
106
|
+
handle(message.value())
|
|
107
|
+
|
|
108
|
+
# With a timeout (lets you do other work between batches)
|
|
109
|
+
while True:
|
|
110
|
+
for message in consumer.consume(timeout=5.0):
|
|
111
|
+
handle(message.value())
|
|
112
|
+
check_app_state()
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
#### `consumer.close()`
|
|
116
|
+
|
|
117
|
+
Cleanly closes the connection (called automatically by the context manager).
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Your read position
|
|
122
|
+
|
|
123
|
+
Your read position (offset) is always persisted. With no `group_id`, the client
|
|
124
|
+
joins the consumer group `<your-username>.default`, so restarting a program
|
|
125
|
+
resumes exactly where it left off: nothing is re-read, nothing is missed.
|
|
126
|
+
|
|
127
|
+
`start_at` only applies the **first** time a given consumer group connects. On
|
|
128
|
+
every later run the stored offset wins, so changing `start_at` on an existing
|
|
129
|
+
group has no effect. To deliberately re-read the retention window, use a
|
|
130
|
+
`group_id` you have never used before.
|
|
131
|
+
|
|
132
|
+
The client automatically prefixes `group_id` with your Kafka username to satisfy
|
|
133
|
+
the per-user ACL, so `group_id="my-program-v1"` becomes the Kafka group
|
|
134
|
+
`your-username.my-program-v1`.
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
consumer = Consumer(
|
|
138
|
+
username="your-username",
|
|
139
|
+
password="your-password",
|
|
140
|
+
group_id="my-program-v1", # its own independent read position
|
|
141
|
+
start_at="latest", # only applies on this group's very first run
|
|
142
|
+
)
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Running several scripts with the same credentials
|
|
148
|
+
|
|
149
|
+
One set of credentials can drive as many scripts as you like, but **give each
|
|
150
|
+
script its own `group_id`**. Consumers that share a group are treated by Kafka
|
|
151
|
+
as one logical reader and have the partitions divided between them, so each
|
|
152
|
+
script would receive only a slice of the stream rather than every alert.
|
|
153
|
+
|
|
154
|
+
```python
|
|
155
|
+
# ingest.py
|
|
156
|
+
consumer = Consumer(username="alice", password="...", group_id="ingest")
|
|
157
|
+
|
|
158
|
+
# alerting.py
|
|
159
|
+
consumer = Consumer(username="alice", password="...", group_id="alerting")
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Each group keeps its own independent read position, so the two scripts can run
|
|
163
|
+
at different speeds, restart independently, and both still see the full alert
|
|
164
|
+
stream.
|
|
165
|
+
|
|
166
|
+
Leaving `group_id` unset in more than one script is the case to avoid: they all
|
|
167
|
+
land in `<your-username>.default` and silently share the stream between them.
|
|
168
|
+
|
|
169
|
+
Running the *same* script as several replicas is the one case where sharing a
|
|
170
|
+
`group_id` is what you want: that is how you spread the load, and Kafka
|
|
171
|
+
rebalances the partitions across the replicas automatically.
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Testing locally against your own broker
|
|
176
|
+
|
|
177
|
+
If you're running the Astro-Colibri broker stack locally (see the
|
|
178
|
+
broker's `QUICKSTART.md`), point the client at it directly:
|
|
179
|
+
|
|
180
|
+
```python
|
|
181
|
+
consumer = Consumer(
|
|
182
|
+
username="alice",
|
|
183
|
+
password="alice-strong-password",
|
|
184
|
+
broker_url="localhost:9092",
|
|
185
|
+
security_protocol="SASL_PLAINTEXT", # local trusted broker only
|
|
186
|
+
)
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Install the package in editable mode from the repository root for
|
|
190
|
+
development:
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
cd Colibri_v2/colibri_client
|
|
194
|
+
pip install -e ".[dev]"
|
|
195
|
+
pytest tests/ -v
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Citation
|
|
201
|
+
|
|
202
|
+
Please cite the software release using [`CITATION.cff`](CITATION.cff) and the
|
|
203
|
+
Astro-COLIBRI platform papers:
|
|
204
|
+
|
|
205
|
+
- Reichherzer et al. (2023), *Astro-COLIBRI 2 - An Advanced Platform for
|
|
206
|
+
Real-Time Multi-Messenger Discoveries*, Galaxies 11, 22,
|
|
207
|
+
[doi:10.3390/galaxies11010022](https://doi.org/10.3390/galaxies11010022).
|
|
208
|
+
- Reichherzer et al. (2021), *Astro-COLIBRI - The COincidence LIBrary for
|
|
209
|
+
Real-time Inquiry for Multimessenger Astrophysics*, ApJS 256, 5,
|
|
210
|
+
[doi:10.3847/1538-4365/ac1517](https://doi.org/10.3847/1538-4365/ac1517).
|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
## License
|
|
215
|
+
|
|
216
|
+
This source-available software is licensed under the
|
|
217
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). It may be used, modified, and
|
|
218
|
+
redistributed for permitted noncommercial purposes, including use by
|
|
219
|
+
educational institutions and public research organizations.
|
|
220
|
+
|
|
221
|
+
Commercial use requires a separate written license. Contact
|
|
222
|
+
[Astro-COLIBRI (Fabian Schüssler)](mailto:astro.colibri@gmail.com) and see
|
|
223
|
+
[COMMERCIAL-LICENSE.md](COMMERCIAL-LICENSE.md).
|
|
224
|
+
|
|
225
|
+
Use of Astro-COLIBRI hosted services, including the Kafka broker and its data,
|
|
226
|
+
is governed separately by the
|
|
227
|
+
[Astro-COLIBRI Terms of Service](https://astro-colibri.science/tos).
|