django-gobeat 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- django_gobeat-0.1.0/LICENSE +21 -0
- django_gobeat-0.1.0/PKG-INFO +374 -0
- django_gobeat-0.1.0/README.md +347 -0
- django_gobeat-0.1.0/django_gobeat/__init__.py +20 -0
- django_gobeat-0.1.0/django_gobeat/admin.py +39 -0
- django_gobeat-0.1.0/django_gobeat/apps.py +51 -0
- django_gobeat-0.1.0/django_gobeat/cron.py +279 -0
- django_gobeat-0.1.0/django_gobeat/management/__init__.py +0 -0
- django_gobeat-0.1.0/django_gobeat/management/commands/__init__.py +0 -0
- django_gobeat-0.1.0/django_gobeat/management/commands/gobeat_sync.py +156 -0
- django_gobeat-0.1.0/django_gobeat/migrations/0001_initial.py +105 -0
- django_gobeat-0.1.0/django_gobeat/migrations/__init__.py +0 -0
- django_gobeat-0.1.0/django_gobeat/models.py +124 -0
- django_gobeat-0.1.0/django_gobeat/resolve.py +344 -0
- django_gobeat-0.1.0/django_gobeat/triggers.py +65 -0
- django_gobeat-0.1.0/django_gobeat.egg-info/PKG-INFO +374 -0
- django_gobeat-0.1.0/django_gobeat.egg-info/SOURCES.txt +20 -0
- django_gobeat-0.1.0/django_gobeat.egg-info/dependency_links.txt +1 -0
- django_gobeat-0.1.0/django_gobeat.egg-info/requires.txt +5 -0
- django_gobeat-0.1.0/django_gobeat.egg-info/top_level.txt +1 -0
- django_gobeat-0.1.0/pyproject.toml +40 -0
- django_gobeat-0.1.0/setup.cfg +4 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 BAPP
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,374 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: django-gobeat
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: cron.py periodic jobs for Django + the companion app of gobeat, a small Go replacement for the celery beat process
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Project-URL: Homepage, https://bapp-open.github.io/django-celery-beat-go/
|
|
7
|
+
Project-URL: Repository, https://github.com/bapp-open/django-celery-beat-go
|
|
8
|
+
Project-URL: Issues, https://github.com/bapp-open/django-celery-beat-go/issues
|
|
9
|
+
Keywords: django,celery,beat,cron,scheduler,periodic tasks,django-celery-beat
|
|
10
|
+
Classifier: Development Status :: 4 - Beta
|
|
11
|
+
Classifier: Framework :: Django
|
|
12
|
+
Classifier: Framework :: Django :: 5.2
|
|
13
|
+
Classifier: Framework :: Django :: 6.0
|
|
14
|
+
Classifier: Framework :: Celery
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Topic :: System :: Distributed Computing
|
|
19
|
+
Requires-Python: >=3.10
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: Django>=5.2
|
|
23
|
+
Requires-Dist: celery<5.8,>=5.7
|
|
24
|
+
Provides-Extra: django-celery-beat
|
|
25
|
+
Requires-Dist: django-celery-beat<2.10,>=2.8; extra == "django-celery-beat"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
|
|
28
|
+
# django-celery-beat-go (gobeat)
|
|
29
|
+
|
|
30
|
+
[](https://github.com/bapp-open/django-celery-beat-go/actions/workflows/ci.yml)
|
|
31
|
+
[](https://pypi.org/project/django-gobeat/)
|
|
32
|
+
[](https://github.com/bapp-open/django-celery-beat-go/pkgs/container/django-celery-beat-go)
|
|
33
|
+
|
|
34
|
+
**gobeat** is a small Go service that replaces the `celery beat` process
|
|
35
|
+
for Django projects. Workers and tasks stay exactly as they are; only the
|
|
36
|
+
scheduler process changes. It comes with **django-gobeat**, a Django app
|
|
37
|
+
that lets you declare periodic jobs in each app's `cron.py`.
|
|
38
|
+
|
|
39
|
+
```sh
|
|
40
|
+
pip install django-gobeat # the Django app
|
|
41
|
+
docker run -e DATABASE_URL=postgres://… ghcr.io/bapp-open/django-celery-beat-go # the scheduler
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## What it does
|
|
45
|
+
|
|
46
|
+
* **Schedules periodic Celery tasks** from two sources:
|
|
47
|
+
`cron.py` classes extending `CronInterface` (auto-discovered like
|
|
48
|
+
`models.py`, stored in gobeat's own table, no django-celery-beat
|
|
49
|
+
needed), and — optionally — django-celery-beat's `PeriodicTask` rows
|
|
50
|
+
and admin, with behaviour identical to its `DatabaseScheduler`.
|
|
51
|
+
* **Publishes the same messages Celery does** (protocol v2) to **Redis**
|
|
52
|
+
or **RabbitMQ**: byte-identical to kombu's, verified against real
|
|
53
|
+
Celery.
|
|
54
|
+
* **Never double-fires or silently loses a run**: every run is claimed in
|
|
55
|
+
Postgres before it is published, kept in a journal until the broker
|
|
56
|
+
confirms it, and only one replica is active at a time.
|
|
57
|
+
* **Stays small**: one static binary on an empty image (16 MB), about
|
|
58
|
+
12 MB of memory.
|
|
59
|
+
* **Shadow mode**: runs beside Python beat without writing anything, and
|
|
60
|
+
a tool compares the two dispatch logs before you switch.
|
|
61
|
+
|
|
62
|
+
## What it does not do
|
|
63
|
+
|
|
64
|
+
* **Only PostgreSQL** as the Django database; **only Redis and RabbitMQ**
|
|
65
|
+
as brokers (no SQS, no Redis Sentinel/Cluster, one broker URL).
|
|
66
|
+
* **Only the latest stable Celery** (5.7.x). It refuses to start on other
|
|
67
|
+
versions instead of guessing.
|
|
68
|
+
* **JSON task messages only**; no pickle/yaml/msgpack serializers, no
|
|
69
|
+
compression other than gzip/zlib.
|
|
70
|
+
* **No solar schedules.**
|
|
71
|
+
* **It is not a worker and does not run your code**: it never imports the
|
|
72
|
+
project, so publish signal handlers (Sentry/OpenTelemetry header
|
|
73
|
+
injection), `task-sent` events, custom `Task.apply_async` and router
|
|
74
|
+
functions that depend on call arguments are not reproduced.
|
|
75
|
+
* **No built-in result handling**; it only sends tasks.
|
|
76
|
+
|
|
77
|
+
Anything unsupported is reported with an error that names the task — it
|
|
78
|
+
is skipped, never approximated. Details:
|
|
79
|
+
[Known differences](#known-differences-from-python-beat).
|
|
80
|
+
|
|
81
|
+
Reference versions: **Celery 5.7.0, kombu 5.7.0, django-celery-beat 2.9.0**
|
|
82
|
+
— see [Version support](#version-support). Design notes and the source
|
|
83
|
+
map (which Python file each behaviour was copied from) are in
|
|
84
|
+
[docs/DESIGN.md](docs/DESIGN.md).
|
|
85
|
+
|
|
86
|
+
## How it fits together
|
|
87
|
+
|
|
88
|
+
```
|
|
89
|
+
manage.py migrate (Django project, at deploy; syncs on post_migrate)
|
|
90
|
+
│ writes one JSON row: broker, timezone, queues, per-task routing
|
|
91
|
+
▼
|
|
92
|
+
Postgres ◀── LISTEN/NOTIFY + polling ── gobeat container ──▶ Redis / RabbitMQ ──▶ workers
|
|
93
|
+
(django_celery_beat tables, settings row, dispatch journal)
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
gobeat never imports your code. Everything that needs the Celery app
|
|
97
|
+
(`task_routes`, decorator options, queue declarations, `beat_schedule`)
|
|
98
|
+
is resolved by Celery itself inside `gobeat_sync`.
|
|
99
|
+
|
|
100
|
+
## Periodic jobs in `cron.py`
|
|
101
|
+
|
|
102
|
+
```python
|
|
103
|
+
# myapp/cron.py - discovered automatically for every app in INSTALLED_APPS
|
|
104
|
+
from datetime import timedelta
|
|
105
|
+
from django_gobeat import CronInterface, crontab
|
|
106
|
+
|
|
107
|
+
class NightlyCleanup(CronInterface):
|
|
108
|
+
schedule = crontab(minute=0, hour=4) # celery.schedules.crontab, a timedelta, or seconds
|
|
109
|
+
timezone = "Europe/Bucharest" # crontab zone; default: CELERY_TIMEZONE
|
|
110
|
+
queue = "maintenance" # also: exchange, routing_key, priority, expires, headers
|
|
111
|
+
|
|
112
|
+
def run(self): # executed by a Celery worker
|
|
113
|
+
...
|
|
114
|
+
|
|
115
|
+
class Heartbeat(CronInterface):
|
|
116
|
+
schedule = timedelta(seconds=30)
|
|
117
|
+
task = "myapp.tasks.ping" # schedule an existing task instead of run()
|
|
118
|
+
kwargs = {"source": "cron"}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
* A class with `run()` is registered as a Celery task named after the job
|
|
122
|
+
(`myapp.cron.NightlyCleanup`, or `name = "..."`); workers find it
|
|
123
|
+
because discovery happens in `AppConfig.ready()`.
|
|
124
|
+
* Project conventions go into your own base class: mark it
|
|
125
|
+
`abstract = True` and override `get_schedule()`, `get_args()`,
|
|
126
|
+
`get_kwargs()`, `get_headers()`, `is_enabled()` or `run()`.
|
|
127
|
+
* After `migrate` (or with `manage.py gobeat_sync`) the discovered jobs
|
|
128
|
+
are written to `django_gobeat_job` (creating, updating and removing rows to match the
|
|
129
|
+
code; run counts and the admin's *paused* switch are kept). Jobs can
|
|
130
|
+
also be created by hand in the admin (`source = admin`).
|
|
131
|
+
* Without django-celery-beat, entries of `app.conf.beat_schedule` are
|
|
132
|
+
written to the same table.
|
|
133
|
+
* Semantics are Celery's: crontabs are evaluated exactly like
|
|
134
|
+
`celery.schedules.crontab` in the job's zone; an interval job first
|
|
135
|
+
runs one interval after it was defined and then every interval.
|
|
136
|
+
|
|
137
|
+
## Migration steps
|
|
138
|
+
|
|
139
|
+
1. `pip install django-gobeat` and add the app (`django-celery-beat` may
|
|
140
|
+
stay installed or not — gobeat uses its tables only when it is):
|
|
141
|
+
|
|
142
|
+
```python
|
|
143
|
+
INSTALLED_APPS += ["django_gobeat"]
|
|
144
|
+
# optional: GOBEAT_CELERY_APP = "proj.celery:app" (default: celery.current_app)
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
2. Run `migrate` on every deploy, as you already do:
|
|
148
|
+
|
|
149
|
+
```sh
|
|
150
|
+
python manage.py migrate
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
django_gobeat hooks `post_migrate` (which fires even when there is
|
|
154
|
+
nothing to migrate) and then writes the settings row, the `cron.py`
|
|
155
|
+
jobs and — as Python beat does on start-up — code-defined
|
|
156
|
+
`beat_schedule` entries. Warnings are printed for anything it cannot
|
|
157
|
+
resolve exactly; a failure is reported but never breaks the migration.
|
|
158
|
+
The same step is available as `python manage.py gobeat_sync`
|
|
159
|
+
(`--dry-run` prints the settings document, `--strict` fails on
|
|
160
|
+
warnings). Settings: `GOBEAT_AUTO_SYNC = False` disables the hook,
|
|
161
|
+
`GOBEAT_STORE_BROKER_URL = False` keeps the broker URL out of the row,
|
|
162
|
+
`GOBEAT_CELERY_APP = "proj.celery:app"` names the app explicitly.
|
|
163
|
+
|
|
164
|
+
3. **Shadow first.** Keep Python beat running and start gobeat beside it
|
|
165
|
+
in dry-run mode; it logs what it would dispatch and writes nothing:
|
|
166
|
+
|
|
167
|
+
```sh
|
|
168
|
+
docker run -e DATABASE_URL=postgres://… -e GOBEAT_DRY_RUN=1 ghcr.io/bapp-open/django-celery-beat-go
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
After a day, compare the two logs:
|
|
172
|
+
|
|
173
|
+
```sh
|
|
174
|
+
gobeat shadow-diff --python-log beat.log --gobeat-log gobeat.log \
|
|
175
|
+
--python-tz Europe/Bucharest --tolerance 2s
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
(`beat.log` is `celery beat --loglevel=INFO` output in Celery's default
|
|
179
|
+
format; `--python-tz` is the zone of its timestamps, normally Django's
|
|
180
|
+
`TIME_ZONE`.) Exit status 0 means every dispatch has a partner.
|
|
181
|
+
|
|
182
|
+
4. Cut over: stop Python beat, then start gobeat without `GOBEAT_DRY_RUN`.
|
|
183
|
+
Do not run both live — Python beat does not take part in the claim.
|
|
184
|
+
|
|
185
|
+
## Rollback
|
|
186
|
+
|
|
187
|
+
Stop gobeat, start `celery beat` again. gobeat keeps `last_run_at` and
|
|
188
|
+
`total_run_count` in the django-celery-beat tables up to date, so Python
|
|
189
|
+
beat continues from where gobeat stopped. Jobs that exist only in
|
|
190
|
+
`cron.py` / `django_gobeat_job` are not known to Python beat: list them in
|
|
191
|
+
`beat_schedule` if you need them during a rollback. The `django_gobeat_*`
|
|
192
|
+
tables can stay or be dropped with `manage.py migrate django_gobeat zero`.
|
|
193
|
+
|
|
194
|
+
## Configuration
|
|
195
|
+
|
|
196
|
+
One container per project. Only `DATABASE_URL` is required; everything
|
|
197
|
+
else comes from the settings row and environment variables override it.
|
|
198
|
+
|
|
199
|
+
| Variable | Default | Meaning |
|
|
200
|
+
|---|---|---|
|
|
201
|
+
| `DATABASE_URL` | – (required) | Postgres URL of the Django database |
|
|
202
|
+
| `GOBEAT_DRY_RUN` | `0` | log dispatches, write and publish nothing (shadow mode) |
|
|
203
|
+
| `GOBEAT_MISSED_RUN_POLICY` | `catch_up_once` | after downtime: one run for the gap (what Python beat does), or `skip` |
|
|
204
|
+
| `GOBEAT_MISSED_RUN_GRACE` | `60s` | lateness beyond which `skip` skips a run |
|
|
205
|
+
| `GOBEAT_PUBLISH_TIMEOUT` | `10s` | per publish attempt |
|
|
206
|
+
| `GOBEAT_PUBLISH_MAX_AGE` | `1h` | keep retrying a claimed run this long, then abandon it loudly |
|
|
207
|
+
| `GOBEAT_JOURNAL_RETENTION` | `24h` | keep published journal rows this long |
|
|
208
|
+
| `GOBEAT_HTTP_ADDR` | `:9099` | `/healthz` and `/metrics` (`off` disables) |
|
|
209
|
+
| `GOBEAT_HEALTH_MAX_AGE` | `30s` | `/healthz` fails when the loop has not ticked for this long |
|
|
210
|
+
| `GOBEAT_STANDBY_INTERVAL` | `5s` | how often a standby replica retries the lock |
|
|
211
|
+
| `GOBEAT_LOCK_KEY` | `0x676f62656174` | advisory lock key |
|
|
212
|
+
| `GOBEAT_SETTINGS_KEY` | `default` | settings row to use |
|
|
213
|
+
| `GOBEAT_LOG_LEVEL` / `GOBEAT_LOG_FORMAT` | `info` / `json` | |
|
|
214
|
+
| `GOBEAT_ALLOW_UNSUPPORTED_VERSIONS` | `0` | run against untested Celery/django-celery-beat versions |
|
|
215
|
+
| **overrides of the settings row** | | |
|
|
216
|
+
| `GOBEAT_BROKER_URL` | row | `redis://`, `rediss://`, `amqp://`, `amqps://` |
|
|
217
|
+
| `GOBEAT_BROKER_TRANSPORT_OPTIONS` | row | JSON (`priority_steps`, `sep`, `global_keyprefix`) |
|
|
218
|
+
| `GOBEAT_TIMEZONE` | row | `CELERY_TIMEZONE` |
|
|
219
|
+
| `GOBEAT_MAX_LOOP_INTERVAL` | row (5) | seconds, `beat_max_loop_interval` |
|
|
220
|
+
| `GOBEAT_CRON_STARTING_DEADLINE` | row | seconds, `beat_cron_starting_deadline` |
|
|
221
|
+
| `GOBEAT_SETTINGS_OVERRIDE` | – | JSON object merged over the settings document |
|
|
222
|
+
|
|
223
|
+
The broker URL (with its password) is part of the settings row; use
|
|
224
|
+
`GOBEAT_STORE_BROKER_URL = False` (or `gobeat_sync --no-broker-url`) and
|
|
225
|
+
`GOBEAT_BROKER_URL` to keep it out of the database.
|
|
226
|
+
|
|
227
|
+
gobeat **refuses to start**, with an explanatory error, when the settings
|
|
228
|
+
row is missing, the django-celery-beat schema is unknown, the versions are
|
|
229
|
+
unsupported or the configuration cannot be reproduced (see below). Once
|
|
230
|
+
started it never exits on database or broker failures; it retries with
|
|
231
|
+
backoff.
|
|
232
|
+
|
|
233
|
+
### Endpoints
|
|
234
|
+
|
|
235
|
+
* `GET /healthz` – 200 while the loop ticks (leader, standby or retrying
|
|
236
|
+
a dead database), 503 when it has not ticked for `GOBEAT_HEALTH_MAX_AGE`.
|
|
237
|
+
The image's `HEALTHCHECK` uses `gobeat healthcheck`.
|
|
238
|
+
* `GET /metrics` – Prometheus text: `gobeat_dispatch_total`,
|
|
239
|
+
`gobeat_dispatch_lateness_seconds` (histogram), `gobeat_errors_total{kind}`,
|
|
240
|
+
`gobeat_reloads_total{reason}`, `gobeat_claim_conflicts_total`,
|
|
241
|
+
`gobeat_skipped_runs_total`, `gobeat_abandoned_total`,
|
|
242
|
+
`gobeat_publish_retries_total`, `gobeat_pending_publishes`,
|
|
243
|
+
`gobeat_leader`, `gobeat_tasks`, `gobeat_tasks_refused`,
|
|
244
|
+
`gobeat_last_tick_timestamp_seconds`.
|
|
245
|
+
|
|
246
|
+
## Delivery guarantees
|
|
247
|
+
|
|
248
|
+
* **Claim before publish.** One SQL statement updates `last_run_at` /
|
|
249
|
+
`total_run_count` only if `last_run_at` is still what gobeat saw, and
|
|
250
|
+
inserts the rendered message into `django_gobeat_dispatch`. Only then is
|
|
251
|
+
the message published; the row is marked when the broker accepted it
|
|
252
|
+
(Redis reply / RabbitMQ publisher confirm).
|
|
253
|
+
* **No lost claimed runs.** Broker down: runs keep being claimed and wait
|
|
254
|
+
in the journal, published in order when the broker is back. Crash
|
|
255
|
+
between claim and publish: the next leader publishes the orphan first.
|
|
256
|
+
A run is given up only after `GOBEAT_PUBLISH_MAX_AGE`, with an error log
|
|
257
|
+
and `gobeat_abandoned_total`.
|
|
258
|
+
* **At-least-once, one edge.** If the process dies after the broker
|
|
259
|
+
accepted a message but before the journal row was marked, that one run
|
|
260
|
+
is published again after restart — with the same task id.
|
|
261
|
+
* **Deterministic task ids**: UUIDv5 of (periodic task id, scheduled
|
|
262
|
+
instant), so a duplicate is recognisable; the journal's unique task id
|
|
263
|
+
also stops the same slot from being claimed twice.
|
|
264
|
+
* **One leader.** A second replica stands by on the advisory lock and
|
|
265
|
+
takes over within `GOBEAT_STANDBY_INTERVAL` of the leader's connection
|
|
266
|
+
ending.
|
|
267
|
+
* **After downtime** (gobeat, database, or a forward clock jump): one
|
|
268
|
+
catch-up run per task, then the normal cadence (`catch_up_once`), or no
|
|
269
|
+
catch-up (`skip`). A clock stepping backwards delays runs; it never
|
|
270
|
+
repeats one.
|
|
271
|
+
|
|
272
|
+
## Known differences from Python beat
|
|
273
|
+
|
|
274
|
+
Deliberate:
|
|
275
|
+
|
|
276
|
+
1. **State is in the database.** Python writes `last_run_at` every three
|
|
277
|
+
minutes; gobeat writes it with every claim.
|
|
278
|
+
2. **One-off tasks are disabled right after their run**, not at the next
|
|
279
|
+
evaluation; the resulting row is the same (`enabled=False`,
|
|
280
|
+
`total_run_count=0`).
|
|
281
|
+
3. **No crontab hour window on load.** django-celery-beat 2.8 only loads
|
|
282
|
+
numeric-hour crontabs within ±2 h of the current hour. A side effect:
|
|
283
|
+
a run missed during downtime is caught up when its hour window opens
|
|
284
|
+
the next day. gobeat catches up at start-up.
|
|
285
|
+
4. **Publish failures are retried** instead of being logged and dropped,
|
|
286
|
+
and messages are published with publisher confirms and `mandatory`
|
|
287
|
+
(RabbitMQ) — an unroutable message is an error, not a silent drop.
|
|
288
|
+
5. **Refused, never guessed** (an error names the task; it is not
|
|
289
|
+
scheduled): solar schedules; non-JSON serializers; compression other
|
|
290
|
+
than gzip/zlib; `args`/`kwargs`/`headers` that are not a JSON
|
|
291
|
+
array/object or contain kombu `__type__` markers; non-positive
|
|
292
|
+
intervals; crontabs that never match (February 31st, a `ValueError`
|
|
293
|
+
in Celery too); routing that ends on an empty queue name (Python
|
|
294
|
+
publishes to a queue called `""`); fan-out exchanges on Redis.
|
|
295
|
+
Python beat *disables* a task whose JSON does not parse; gobeat only
|
|
296
|
+
skips it.
|
|
297
|
+
6. **Refused at start-up**: `USE_TZ` ≠ `DJANGO_CELERY_BEAT_TZ_AWARE`,
|
|
298
|
+
`enable_utc=False` without `CELERY_TIMEZONE`, `task_always_eager`,
|
|
299
|
+
task protocol 1, several broker URLs, other transports.
|
|
300
|
+
|
|
301
|
+
Not reproduced (things only a Python process can do):
|
|
302
|
+
|
|
303
|
+
* `before_task_publish` / `after_task_publish` signal handlers
|
|
304
|
+
(Sentry/OpenTelemetry header injection), `task-sent` events, custom
|
|
305
|
+
`Task.apply_async` / `shadow_name`, argument checking (`typing=True`).
|
|
306
|
+
* Router *functions* in `task_routes` are evaluated once per task by
|
|
307
|
+
`gobeat_sync` with empty arguments (a warning is printed); routes that
|
|
308
|
+
depend on call arguments are not reproduced.
|
|
309
|
+
* `reply_to` is a random id; no reply queue is declared for the RPC
|
|
310
|
+
result backend.
|
|
311
|
+
* gzip/zlib bodies decompress to the identical bytes but the compressed
|
|
312
|
+
stream itself is produced by Go's zlib.
|
|
313
|
+
* AMQP: content header and body are byte-identical; the `Basic.Publish`
|
|
314
|
+
frame additionally sets `mandatory`.
|
|
315
|
+
|
|
316
|
+
Inherited from Celery on purpose (the reference version does this):
|
|
317
|
+
|
|
318
|
+
* Crontabs are wall-clock in their own time zone: a time that does not
|
|
319
|
+
exist on the spring-forward day runs an hour later, a repeated time
|
|
320
|
+
runs once, and an every-minute crontab in a DST zone skips the repeated
|
|
321
|
+
hour in autumn. Intervals are elapsed time and are not affected by DST.
|
|
322
|
+
* A crontab with day-of-month *and* day-of-week needs both to match.
|
|
323
|
+
* A never-run task counts from `date_changed`; with `start_time` it first
|
|
324
|
+
runs at `start_time`.
|
|
325
|
+
|
|
326
|
+
## Version support
|
|
327
|
+
|
|
328
|
+
gobeat checks the versions recorded by `gobeat_sync` and the applied
|
|
329
|
+
django-celery-beat migrations before doing anything.
|
|
330
|
+
|
|
331
|
+
| | status |
|
|
332
|
+
|---|---|
|
|
333
|
+
| Celery 5.7.x | supported (reference: 5.7.0) |
|
|
334
|
+
| django-celery-beat 2.8.x, 2.9.x | supported: goldens regenerated with 2.8.1 and 2.9.0 are byte-identical |
|
|
335
|
+
| Celery ≤ 5.6 | refused: crontab arithmetic around DST, interval arithmetic, `saferepr`, the message headers and beat's tick differ from 5.7 |
|
|
336
|
+
| django-celery-beat ≤ 2.7 | refused: a never-run task starts from "now" instead of `date_changed` |
|
|
337
|
+
|
|
338
|
+
When a new Celery release appears: bump `tests/requirements.txt`, run
|
|
339
|
+
`make golden soak-golden` and `make ci`; every behavioural change shows up
|
|
340
|
+
as a failing golden case.
|
|
341
|
+
|
|
342
|
+
## Testing
|
|
343
|
+
|
|
344
|
+
`make ci` runs everything; the pieces:
|
|
345
|
+
|
|
346
|
+
| | command | what |
|
|
347
|
+
|---|---|---|
|
|
348
|
+
| unit | `make test` | schedule maths, crontab parsing, time zones, message encoding, config, HTTP, shadow tool — with `-race` and a goroutine-leak check in every package |
|
|
349
|
+
| differential | `make test` (replay), `make golden` (regenerate) | 70,000 schedule cases, 6,000 JSON/repr cases and 6,000 messages produced by real Celery/kombu/django-celery-beat (`tests/differential`); messages are captured from real Redis and RabbitMQ publishes |
|
|
350
|
+
| fuzz | `make fuzz` | crontab parser, crontab evaluation, JSON loader, message encoder |
|
|
351
|
+
| soak | `make test`, `make soak-golden` | real `DatabaseScheduler.tick()` on a fake clock vs gobeat's simulation over 30 simulated days: identical dispatch counts for every task (1,000 tasks, 1.24 M dispatches; and 10,000 tasks, 12.4 M dispatches, with a flat heap) |
|
|
352
|
+
| version matrix | `make matrix` | goldens regenerated under each supported version must be byte-identical |
|
|
353
|
+
| Django app | `make pytest` | `cron.py` discovery, `CronInterface`, job sync |
|
|
354
|
+
| integration | `make integration` | docker compose: Postgres, Redis, RabbitMQ, a Django project, real workers — on both brokers, plus a pass with django-celery-beat not installed |
|
|
355
|
+
| failure injection | `make integration` | database down/up, broker down/up, SIGKILL between claim and publish and between publish and journal mark, network partition, two instances with failover, missed-run policies; clock jumps on a fake clock in `internal/scheduler` |
|
|
356
|
+
| shadow | `make integration` | Python beat and gobeat dry-run side by side, logs compared with `gobeat shadow-diff` |
|
|
357
|
+
|
|
358
|
+
## Releasing
|
|
359
|
+
|
|
360
|
+
1. Set the new version in `pyproject.toml` and `django_gobeat/__init__.py`.
|
|
361
|
+
2. Tag and push: `git tag v0.2.0 && git push origin v0.2.0`.
|
|
362
|
+
|
|
363
|
+
The `release` workflow then runs the test-suite, pushes the image to
|
|
364
|
+
`ghcr.io/bapp-open/django-celery-beat-go` (`:0.2.0`, `:0.2`, `:latest`,
|
|
365
|
+
amd64 + arm64), publishes `django-gobeat` to PyPI (trusted publishing) and
|
|
366
|
+
attaches static binaries to the GitHub release.
|
|
367
|
+
|
|
368
|
+
## Development
|
|
369
|
+
|
|
370
|
+
```sh
|
|
371
|
+
make build # bin/gobeat (static)
|
|
372
|
+
make test # needs only Go; golden files are committed
|
|
373
|
+
make docker # the image
|
|
374
|
+
```
|