dbworker 0.0.6__tar.gz → 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- dbworker-0.1.0/PKG-INFO +109 -0
- dbworker-0.1.0/README.md +91 -0
- {dbworker-0.0.6 → dbworker-0.1.0}/pyproject.toml +1 -1
- {dbworker-0.0.6 → dbworker-0.1.0}/src/dbworker.py +43 -6
- dbworker-0.0.6/PKG-INFO +0 -233
- dbworker-0.0.6/README.md +0 -215
- {dbworker-0.0.6 → dbworker-0.1.0}/LICENSE +0 -0
- {dbworker-0.0.6 → dbworker-0.1.0}/poetry.lock +0 -0
dbworker-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dbworker
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Portable, single-script Python/SQLAlchemy background worker for SQLite, PostgreSQL and MySQL — no Redis, broker or enqueue
|
|
5
|
+
License: MIT
|
|
6
|
+
License-File: LICENSE
|
|
7
|
+
Author: Ziyang Song
|
|
8
|
+
Requires-Python: >=3.12,<4.0
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
14
|
+
Requires-Dist: sqlalchemy (>=2.0,<3.0)
|
|
15
|
+
Project-URL: Repository, https://github.com/zysilm/dbworker
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
|
|
18
|
+
# dbworker
|
|
19
|
+
|
|
20
|
+
**Your database rows are already the queue.**
|
|
21
|
+
|
|
22
|
+
A portable, single-script background worker for SQLAlchemy. No broker, no enqueue, no duplicate job model.
|
|
23
|
+
|
|
24
|
+
```text
|
|
25
|
+
Celery / Huey / Graphile Worker:
|
|
26
|
+
event → job → worker
|
|
27
|
+
|
|
28
|
+
dbworker:
|
|
29
|
+
row → worker
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Requires Python 3.12+. Install into your application:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
poetry add dbworker
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Usage
|
|
39
|
+
|
|
40
|
+
Register a handler for an existing SQLAlchemy model. DBWorker discovers eligible
|
|
41
|
+
rows, runs the handler in a child process, and commits its database changes with
|
|
42
|
+
the execution status.
|
|
43
|
+
|
|
44
|
+
```python
|
|
45
|
+
from sqlalchemy import select
|
|
46
|
+
from sqlalchemy.orm import Session
|
|
47
|
+
from dbworker import Coordinator, Finished
|
|
48
|
+
|
|
49
|
+
coordinator = Coordinator(session_factory, database_url=database_url)
|
|
50
|
+
|
|
51
|
+
@coordinator.transactional_worker(
|
|
52
|
+
name="process",
|
|
53
|
+
source=YourModel,
|
|
54
|
+
eligible=lambda: (
|
|
55
|
+
select(YourModel)
|
|
56
|
+
.where(YourModel.your_bool_property.is_(True))
|
|
57
|
+
.order_by(YourModel.id)
|
|
58
|
+
),
|
|
59
|
+
concurrency=4,
|
|
60
|
+
)
|
|
61
|
+
def process(source: YourModel, session: Session) -> Finished:
|
|
62
|
+
# Apply your application logic using the supplied session.
|
|
63
|
+
return Finished()
|
|
64
|
+
|
|
65
|
+
if __name__ == "__main__":
|
|
66
|
+
coordinator.create_worker_tables()
|
|
67
|
+
coordinator.start()
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
`YourModel`, `session_factory`, and `database_url` come from your application.
|
|
71
|
+
`eligible` selects which rows can be claimed and in what order.
|
|
72
|
+
It is optional: omitting it selects from the whole model. DBWorker automatically
|
|
73
|
+
excludes completed, failed, and actively claimed work.
|
|
74
|
+
Return `Finished()` when processing is complete; no enqueue call is needed.
|
|
75
|
+
|
|
76
|
+
See the [Usage Instructions](https://github.com/zysilm/dbworker/blob/main/doc/usage.md) for eligibility queries, incremental
|
|
77
|
+
processing, service startup and shutdown, status tracking, dependencies,
|
|
78
|
+
failure handling, and configuration.
|
|
79
|
+
|
|
80
|
+
## Examples
|
|
81
|
+
|
|
82
|
+
- [Image deduplication](examples/imagededup_system_dbwork/README.md): independent FastAPI and worker services, artifact building, paged comparisons, and worker dependencies.
|
|
83
|
+
- [Redis + Celery equivalent](examples/imagededup_system_redis_celery/README.md).
|
|
84
|
+
- [Benchmarks](benchmarks/imagededup_benckmark/README.md) with structured JSON results.
|
|
85
|
+
|
|
86
|
+
## License
|
|
87
|
+
|
|
88
|
+
[MIT](LICENSE) © 2026 Ziyang Song.
|
|
89
|
+
|
|
90
|
+
<!-- benchmark-results:start -->
|
|
91
|
+
## Benchmark Results
|
|
92
|
+
|
|
93
|
+
Median wall time in seconds; **bold** marks the faster backend. Each experiment runs in a fresh Docker container on its own GitHub-hosted Ubuntu VM. Fixed profile: 8 producers, a 60-second scheduled submission window and 8 total execution slots; image hash building uses one native bulk import. Three repetitions per backend. Actual schedule delays and backlog are recorded in JSON; this is one load point, not a maximum-capacity search.
|
|
94
|
+
|
|
95
|
+
| Experiment | Workload | Celery wall (s) | DBWorker wall (s) | C / D | Task P95 / peak outstanding (C; D) |
|
|
96
|
+
|---|---|---:|---:|---:|---|
|
|
97
|
+
| imagededup | Build 1,000 image hashes (one bulk import) | **8.510** | 8.904 | 0.96 | n/a; n/a |
|
|
98
|
+
| imagededup | Compare 1,000 images / 999,000 directed pairs | 150.199 | **106.599** | 1.41 | n/a; n/a |
|
|
99
|
+
| imagededup | Build + compare 1,000 images / 999,000 directed pairs | 139.493 | **133.968** | 1.04 | n/a; n/a |
|
|
100
|
+
| superset | SQL Lab: 3,000 queries / 10,000 rows | **150.445** | 165.700 | 0.91 | 42.066s / 1550; 49.179s / 1615 |
|
|
101
|
+
| saleor | Export: 1,000 x 256 products + 1,000 emails | **109.182** | 138.540 | 0.79 | 24.518s / 495; 37.793s / 757 |
|
|
102
|
+
| paperless_ngx | OCR: 200 scans, archive and index | 105.615 | **60.843** | 1.74 | 44.666s / 90; 7.350s / 22 |
|
|
103
|
+
| posthog | 2FA: 5,000 requests / 10,000 tasks | **210.518** | 330.952 | 0.64 | 0.276s / 16; 102.777s / 4451 |
|
|
104
|
+
| sentry | Email: 2,000 requests / 4,000 deliveries | 60.303 | **60.157** | 1.00 | 0.054s / 16; 1.094s / 172 |
|
|
105
|
+
|
|
106
|
+
Run: `github-37987827539-1`. [Details and scope](doc/benchmark-results.md) · [JSON results](benchmarks/results/latest/index.json).
|
|
107
|
+
Scoped application workloads; Sentry uses historical 24.1.0. Wall time includes the fixed submission window, so a ratio near 1 does not prove equal processing capacity. Task P95 starts at publication; peak outstanding counts already published tasks. Ratios above 1 favor DBWorker; no cross-project average is computed.
|
|
108
|
+
<!-- benchmark-results:end -->
|
|
109
|
+
|
dbworker-0.1.0/README.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# dbworker
|
|
2
|
+
|
|
3
|
+
**Your database rows are already the queue.**
|
|
4
|
+
|
|
5
|
+
A portable, single-script background worker for SQLAlchemy. No broker, no enqueue, no duplicate job model.
|
|
6
|
+
|
|
7
|
+
```text
|
|
8
|
+
Celery / Huey / Graphile Worker:
|
|
9
|
+
event → job → worker
|
|
10
|
+
|
|
11
|
+
dbworker:
|
|
12
|
+
row → worker
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Requires Python 3.12+. Install into your application:
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
poetry add dbworker
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Usage
|
|
22
|
+
|
|
23
|
+
Register a handler for an existing SQLAlchemy model. DBWorker discovers eligible
|
|
24
|
+
rows, runs the handler in a child process, and commits its database changes with
|
|
25
|
+
the execution status.
|
|
26
|
+
|
|
27
|
+
```python
|
|
28
|
+
from sqlalchemy import select
|
|
29
|
+
from sqlalchemy.orm import Session
|
|
30
|
+
from dbworker import Coordinator, Finished
|
|
31
|
+
|
|
32
|
+
coordinator = Coordinator(session_factory, database_url=database_url)
|
|
33
|
+
|
|
34
|
+
@coordinator.transactional_worker(
|
|
35
|
+
name="process",
|
|
36
|
+
source=YourModel,
|
|
37
|
+
eligible=lambda: (
|
|
38
|
+
select(YourModel)
|
|
39
|
+
.where(YourModel.your_bool_property.is_(True))
|
|
40
|
+
.order_by(YourModel.id)
|
|
41
|
+
),
|
|
42
|
+
concurrency=4,
|
|
43
|
+
)
|
|
44
|
+
def process(source: YourModel, session: Session) -> Finished:
|
|
45
|
+
# Apply your application logic using the supplied session.
|
|
46
|
+
return Finished()
|
|
47
|
+
|
|
48
|
+
if __name__ == "__main__":
|
|
49
|
+
coordinator.create_worker_tables()
|
|
50
|
+
coordinator.start()
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`YourModel`, `session_factory`, and `database_url` come from your application.
|
|
54
|
+
`eligible` selects which rows can be claimed and in what order.
|
|
55
|
+
It is optional: omitting it selects from the whole model. DBWorker automatically
|
|
56
|
+
excludes completed, failed, and actively claimed work.
|
|
57
|
+
Return `Finished()` when processing is complete; no enqueue call is needed.
|
|
58
|
+
|
|
59
|
+
See the [Usage Instructions](https://github.com/zysilm/dbworker/blob/main/doc/usage.md) for eligibility queries, incremental
|
|
60
|
+
processing, service startup and shutdown, status tracking, dependencies,
|
|
61
|
+
failure handling, and configuration.
|
|
62
|
+
|
|
63
|
+
## Examples
|
|
64
|
+
|
|
65
|
+
- [Image deduplication](examples/imagededup_system_dbwork/README.md): independent FastAPI and worker services, artifact building, paged comparisons, and worker dependencies.
|
|
66
|
+
- [Redis + Celery equivalent](examples/imagededup_system_redis_celery/README.md).
|
|
67
|
+
- [Benchmarks](benchmarks/imagededup_benckmark/README.md) with structured JSON results.
|
|
68
|
+
|
|
69
|
+
## License
|
|
70
|
+
|
|
71
|
+
[MIT](LICENSE) © 2026 Ziyang Song.
|
|
72
|
+
|
|
73
|
+
<!-- benchmark-results:start -->
|
|
74
|
+
## Benchmark Results
|
|
75
|
+
|
|
76
|
+
Median wall time in seconds; **bold** marks the faster backend. Each experiment runs in a fresh Docker container on its own GitHub-hosted Ubuntu VM. Fixed profile: 8 producers, a 60-second scheduled submission window and 8 total execution slots; image hash building uses one native bulk import. Three repetitions per backend. Actual schedule delays and backlog are recorded in JSON; this is one load point, not a maximum-capacity search.
|
|
77
|
+
|
|
78
|
+
| Experiment | Workload | Celery wall (s) | DBWorker wall (s) | C / D | Task P95 / peak outstanding (C; D) |
|
|
79
|
+
|---|---|---:|---:|---:|---|
|
|
80
|
+
| imagededup | Build 1,000 image hashes (one bulk import) | **8.510** | 8.904 | 0.96 | n/a; n/a |
|
|
81
|
+
| imagededup | Compare 1,000 images / 999,000 directed pairs | 150.199 | **106.599** | 1.41 | n/a; n/a |
|
|
82
|
+
| imagededup | Build + compare 1,000 images / 999,000 directed pairs | 139.493 | **133.968** | 1.04 | n/a; n/a |
|
|
83
|
+
| superset | SQL Lab: 3,000 queries / 10,000 rows | **150.445** | 165.700 | 0.91 | 42.066s / 1550; 49.179s / 1615 |
|
|
84
|
+
| saleor | Export: 1,000 x 256 products + 1,000 emails | **109.182** | 138.540 | 0.79 | 24.518s / 495; 37.793s / 757 |
|
|
85
|
+
| paperless_ngx | OCR: 200 scans, archive and index | 105.615 | **60.843** | 1.74 | 44.666s / 90; 7.350s / 22 |
|
|
86
|
+
| posthog | 2FA: 5,000 requests / 10,000 tasks | **210.518** | 330.952 | 0.64 | 0.276s / 16; 102.777s / 4451 |
|
|
87
|
+
| sentry | Email: 2,000 requests / 4,000 deliveries | 60.303 | **60.157** | 1.00 | 0.054s / 16; 1.094s / 172 |
|
|
88
|
+
|
|
89
|
+
Run: `github-37987827539-1`. [Details and scope](doc/benchmark-results.md) · [JSON results](benchmarks/results/latest/index.json).
|
|
90
|
+
Scoped application workloads; Sentry uses historical 24.1.0. Wall time includes the fixed submission window, so a ratio near 1 does not prove equal processing capacity. Task P95 starts at publication; peak outstanding counts already published tasks. Ratios above 1 favor DBWorker; no cross-project average is computed.
|
|
91
|
+
<!-- benchmark-results:end -->
|
|
@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
|
|
|
4
4
|
|
|
5
5
|
[tool.poetry]
|
|
6
6
|
name = "dbworker"
|
|
7
|
-
version = "0.0
|
|
7
|
+
version = "0.1.0"
|
|
8
8
|
description = "Portable, single-script Python/SQLAlchemy background worker for SQLite, PostgreSQL and MySQL — no Redis, broker or enqueue"
|
|
9
9
|
authors = ["Ziyang Song"]
|
|
10
10
|
license = "MIT"
|
|
@@ -18,7 +18,7 @@ from datetime import datetime, timedelta, timezone
|
|
|
18
18
|
from typing import Any, TypeAlias, TypeVar, cast
|
|
19
19
|
from multiprocessing.util import Finalize
|
|
20
20
|
|
|
21
|
-
from sqlalchemy import Column, DateTime, Enum, ForeignKey, String, Table, Text, and_, create_engine, event, exists, inspect, insert, or_, select, update
|
|
21
|
+
from sqlalchemy import Column, DateTime, Enum, ForeignKey, String, Table, Text, and_, create_engine, event, exists, inspect, insert, or_, select, tuple_, update
|
|
22
22
|
from sqlalchemy.exc import IntegrityError
|
|
23
23
|
from sqlalchemy.engine import CursorResult, RowMapping, URL, make_url
|
|
24
24
|
from sqlalchemy.orm import DeclarativeBase, Mapper, Session, sessionmaker
|
|
@@ -368,6 +368,39 @@ class Coordinator:
|
|
|
368
368
|
return self._claim_with_db_lock(worker)
|
|
369
369
|
return self._claim_with_conditional_update(worker)
|
|
370
370
|
|
|
371
|
+
def _claim_many(self, worker: _Worker, limit: int) -> list[Claim]:
|
|
372
|
+
"""Commit ownership for free execution slots before dispatching work."""
|
|
373
|
+
if limit == 1:
|
|
374
|
+
claim = self.claim(worker)
|
|
375
|
+
return [] if claim is None else [claim]
|
|
376
|
+
supports_skip_locked = self._supports_skip_locked
|
|
377
|
+
timestamp = now()
|
|
378
|
+
with self.session_factory.begin() as session:
|
|
379
|
+
connection = session.connection()
|
|
380
|
+
if (connection.dialect.name == "sqlite"
|
|
381
|
+
and not connection.connection.driver_connection.in_transaction):
|
|
382
|
+
# Legacy sqlite3 transaction control does not begin on SELECT.
|
|
383
|
+
# Without an outer transaction, releasing the first insertion
|
|
384
|
+
# savepoint would commit it even if a later claim fails.
|
|
385
|
+
connection.exec_driver_sql("BEGIN IMMEDIATE")
|
|
386
|
+
query = self._candidate(worker, timestamp).limit(limit)
|
|
387
|
+
if supports_skip_locked:
|
|
388
|
+
query = query.with_for_update(skip_locked=True, of=worker.source_table)
|
|
389
|
+
# Custom eligibility joins may return the same source more than once.
|
|
390
|
+
source_ids = dict.fromkeys(session.scalars(query))
|
|
391
|
+
claims = [claim for source_id in source_ids
|
|
392
|
+
if (claim := self._record_claim(session, worker, source_id, timestamp)) is not None]
|
|
393
|
+
if claims:
|
|
394
|
+
# Earlier records must not enter the queue with leases aged by
|
|
395
|
+
# the rest of this batch. Refresh while ownership is still locked.
|
|
396
|
+
table = worker.table
|
|
397
|
+
session.execute(update(table).where(
|
|
398
|
+
table.c.execution_status == ExecutionStatus.WORKING,
|
|
399
|
+
tuple_(table.c.source_id, table.c.claim_token).in_(
|
|
400
|
+
[(claim.source_id, claim.token) for claim in claims]),
|
|
401
|
+
).values(lease_expires_at=now() + timedelta(seconds=self.lease_seconds)))
|
|
402
|
+
return claims
|
|
403
|
+
|
|
371
404
|
def renew(self, worker: _Worker, claims: Iterable[Claim]) -> None:
|
|
372
405
|
table = worker.table
|
|
373
406
|
with self.session_factory.begin() as session:
|
|
@@ -400,8 +433,11 @@ class Coordinator:
|
|
|
400
433
|
for future in list(active):
|
|
401
434
|
if future.done():
|
|
402
435
|
claim = active.pop(future)
|
|
436
|
+
# Completion can publish eligible children after an empty query.
|
|
437
|
+
next_claim_at = 0.0
|
|
403
438
|
try:
|
|
404
439
|
future.result()
|
|
440
|
+
delay = self.poll_seconds
|
|
405
441
|
except LostClaim:
|
|
406
442
|
logger.info("Discarded stale result for %s:%s", worker.name, claim.source_id)
|
|
407
443
|
except Exception as exc:
|
|
@@ -413,16 +449,17 @@ class Coordinator:
|
|
|
413
449
|
next_renewal = time.monotonic() + self.lease_seconds / 3
|
|
414
450
|
if not self._stop.is_set() and time.monotonic() >= next_claim_at:
|
|
415
451
|
while len(active) < worker.concurrency and not self._stop.is_set():
|
|
416
|
-
|
|
417
|
-
if
|
|
452
|
+
claims = self._claim_many(worker, worker.concurrency - len(active))
|
|
453
|
+
if not claims:
|
|
418
454
|
next_claim_at = time.monotonic() + delay
|
|
419
455
|
delay = min(delay * 2, self.max_poll_seconds)
|
|
420
456
|
break
|
|
421
457
|
delay = self.poll_seconds
|
|
422
458
|
next_claim_at = 0.0
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
459
|
+
for claim in claims:
|
|
460
|
+
future = handlers.submit(_execute_in_process, claim)
|
|
461
|
+
active[future] = claim
|
|
462
|
+
future.add_done_callback(lambda completed: wakeup.set())
|
|
426
463
|
deadlines: list[float] = []
|
|
427
464
|
if active:
|
|
428
465
|
deadlines.append(next_renewal)
|
dbworker-0.0.6/PKG-INFO
DELETED
|
@@ -1,233 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: dbworker
|
|
3
|
-
Version: 0.0.6
|
|
4
|
-
Summary: Portable, single-script Python/SQLAlchemy background worker for SQLite, PostgreSQL and MySQL — no Redis, broker or enqueue
|
|
5
|
-
License: MIT
|
|
6
|
-
License-File: LICENSE
|
|
7
|
-
Author: Ziyang Song
|
|
8
|
-
Requires-Python: >=3.12,<4.0
|
|
9
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
-
Classifier: Programming Language :: Python :: 3
|
|
11
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
14
|
-
Requires-Dist: sqlalchemy (>=2.0,<3.0)
|
|
15
|
-
Project-URL: Repository, https://github.com/zysilm/dbworker
|
|
16
|
-
Description-Content-Type: text/markdown
|
|
17
|
-
|
|
18
|
-
# dbworker
|
|
19
|
-
|
|
20
|
-
**Your database rows are already the queue.**
|
|
21
|
-
|
|
22
|
-
A portable, single-script background worker for SQLAlchemy. No broker, no enqueue, no duplicate job model.
|
|
23
|
-
|
|
24
|
-
```text
|
|
25
|
-
Celery / Huey / Graphile Worker:
|
|
26
|
-
event → job → worker
|
|
27
|
-
|
|
28
|
-
dbworker:
|
|
29
|
-
row → worker
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
Requires Python 3.12+. Install into your application:
|
|
33
|
-
|
|
34
|
-
```sh
|
|
35
|
-
poetry add dbworker
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
## Quick start
|
|
39
|
-
|
|
40
|
-
Use your existing SQLAlchemy `session_factory` and database URL:
|
|
41
|
-
|
|
42
|
-
```python
|
|
43
|
-
from dbworker import Coordinator
|
|
44
|
-
|
|
45
|
-
coordinator = Coordinator(session_factory, database_url=database_url)
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
You write a **handler**: a Python function containing the work you want to run. DBWorker runs handlers in separate processes.
|
|
49
|
-
|
|
50
|
-
Before calling your handler, DBWorker chooses the next task and marks it as “being worked on” in the database. This step is called a **claim**. It lets several processes share the tasks without choosing the same task at the same time.
|
|
51
|
-
|
|
52
|
-
DBWorker keeps the claim active until your handler finishes. If the process crashes, the claim expires so another process can take over the task.
|
|
53
|
-
|
|
54
|
-
`source` is the database table that supplies input to your handler, specified as a SQLAlchemy model. It can store user requests, or any entries whose creation should automatically start a procedure. Each new entry gives DBWorker new work to run, and the selected entry is passed to your handler. Its primary key identifies that work so DBWorker can track its completion; the model must have a single primary-key column.
|
|
55
|
-
|
|
56
|
-
`eligible` controls which inputs can be claimed next. Without it, any new entry can be picked up. Add it when work should wait for a condition or run in a particular order. In the example below, `YourModel` stands for your model. The `enabled` filter and ordering by `id` illustrate a selection rule; replace them with your own conditions and ordering.
|
|
57
|
-
|
|
58
|
-
Decorate your handler like this. The comments describe where your application logic goes:
|
|
59
|
-
|
|
60
|
-
**Complete the work in one invocation:**
|
|
61
|
-
|
|
62
|
-
```python
|
|
63
|
-
from sqlalchemy import select
|
|
64
|
-
from sqlalchemy.orm import Session
|
|
65
|
-
from dbworker import Finished
|
|
66
|
-
|
|
67
|
-
@coordinator.transactional_worker(
|
|
68
|
-
name="process",
|
|
69
|
-
source=YourModel,
|
|
70
|
-
eligible=lambda: (
|
|
71
|
-
select(YourModel)
|
|
72
|
-
.where(YourModel.enabled.is_(True))
|
|
73
|
-
.order_by(YourModel.id)
|
|
74
|
-
),
|
|
75
|
-
concurrency=4,
|
|
76
|
-
)
|
|
77
|
-
def process(source: YourModel, session: Session) -> Finished:
|
|
78
|
-
# Your procedure goes here.
|
|
79
|
-
# Return Finished() when the procedure is complete.
|
|
80
|
-
return Finished()
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
- `name` identifies the worker for status queries.
|
|
84
|
-
- `concurrency=4` allows four handlers to run in child processes.
|
|
85
|
-
- `source` is the selected instance of your source model.
|
|
86
|
-
- `session` is a normal SQLAlchemy session supplied by DBWorker. The handler can leave either argument unused.
|
|
87
|
-
|
|
88
|
-
Before each claim, DBWorker calls `eligible` and uses its query to select the next item. If nothing can be claimed, it waits and checks again. If your application later enables an item, a subsequent check can select it. Finished, failed, and currently claimed items are excluded automatically; you do not need to put those checks in your query. Omitting `eligible` lets DBWorker select from all entries in `source`.
|
|
89
|
-
|
|
90
|
-
**Process part of the work, then continue in another invocation:**
|
|
91
|
-
|
|
92
|
-
```python
|
|
93
|
-
from dbworker import Finished, Outcome, Unfinished
|
|
94
|
-
|
|
95
|
-
@coordinator.transactional_worker(
|
|
96
|
-
name="process_in_steps",
|
|
97
|
-
source=YourModel,
|
|
98
|
-
eligible=lambda: (
|
|
99
|
-
select(YourModel)
|
|
100
|
-
.where(YourModel.enabled.is_(True))
|
|
101
|
-
.order_by(YourModel.id)
|
|
102
|
-
),
|
|
103
|
-
concurrency=4,
|
|
104
|
-
)
|
|
105
|
-
def process_in_steps(source: YourModel, session: Session) -> Outcome:
|
|
106
|
-
# Your procedure goes here.
|
|
107
|
-
# Return Unfinished() if it needs another invocation to continue.
|
|
108
|
-
# Return Finished() instead when it is complete.
|
|
109
|
-
return Unfinished()
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
`Finished()` means the work item is complete. `Unfinished()` means this invocation is done, but more work remains. Both save the execution status and commit any writes made through the supplied session; the handler does not have to make any database writes. After `Unfinished()`, DBWorker can invoke the handler again when eligible. Each invocation starts the function from the beginning. Your procedure decides how to continue; DBWorker does not save its position in the function. Use `Outcome` as the return annotation when a handler can return either result.
|
|
113
|
-
|
|
114
|
-
If the handler raises an exception, DBWorker rolls back its transaction and marks the work failed. If your procedure has prerequisites that can be checked in the database, `eligible` can delay claiming until they are met.
|
|
115
|
-
|
|
116
|
-
DBWorker owns the handler's commit and session cleanup. Do not commit or close this session yourself. For long computations, you can release a read transaction with `session.rollback()` first. Copy needed values before rollback, and do this before making writes you want to keep.
|
|
117
|
-
|
|
118
|
-
### Start and stop
|
|
119
|
-
|
|
120
|
-
Register handlers at module scope in an importable module. In your worker service, initialize DBWorker's tables and start the coordinator:
|
|
121
|
-
|
|
122
|
-
```python
|
|
123
|
-
if __name__ == "__main__":
|
|
124
|
-
coordinator.create_worker_tables()
|
|
125
|
-
coordinator.start()
|
|
126
|
-
```
|
|
127
|
-
|
|
128
|
-
`start()` returns while workers keep running. When a source entry matches `eligible`, DBWorker can claim its work and call the handler—there is no enqueue call.
|
|
129
|
-
|
|
130
|
-
On service shutdown, call:
|
|
131
|
-
|
|
132
|
-
```python
|
|
133
|
-
coordinator.stop()
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
`stop()` stops new claims and waits for active handlers to finish. Your API can run independently, using the same database. See the [complete example](examples/imagededup_system_dbwork/README.md) for runnable API and worker commands with shutdown handling.
|
|
137
|
-
|
|
138
|
-
## Track progress and coordinate dependent work
|
|
139
|
-
|
|
140
|
-
Your application may need to show whether work is running or complete. Another worker may also depend on that information: one procedure prepares something, and a second can begin only after preparation finishes. If preparation fails, the application may need to report that failure instead of continuing.
|
|
141
|
-
|
|
142
|
-
DBWorker tracks execution separately for each worker and source entry. Access it through the coordinator using the worker's registered name and the source entry's primary key; you do not need to manage or query DBWorker's internal tables yourself.
|
|
143
|
-
|
|
144
|
-
Use `execution_status()` to get the current status in your API or handler:
|
|
145
|
-
|
|
146
|
-
```python
|
|
147
|
-
from dbworker import ExecutionStatus
|
|
148
|
-
|
|
149
|
-
with session_factory() as session:
|
|
150
|
-
status = coordinator.execution_status(
|
|
151
|
-
session, worker="process", source_id=source_id,
|
|
152
|
-
)
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
This returns `None` before the first claim, or an `ExecutionStatus` enum: `WORKING`, `UNFINISHED`, `FINISHED`, or `FAILED`.
|
|
156
|
-
|
|
157
|
-
Sometimes you need to select inputs based on the status of their work—for example, list only inputs whose processing has finished. Calling `execution_status()` for each input would mean checking them individually.
|
|
158
|
-
|
|
159
|
-
`has_execution_status()` lets you include that check in a database query. It returns a SQLAlchemy condition meaning: **“Does this worker have one of these execution statuses for this input?”** It does not run a query or return a Python `True` or `False` when called.
|
|
160
|
-
|
|
161
|
-
```python
|
|
162
|
-
finished = coordinator.has_execution_status(
|
|
163
|
-
worker="process",
|
|
164
|
-
source_id=YourModel.id,
|
|
165
|
-
statuses=(ExecutionStatus.FINISHED,),
|
|
166
|
-
)
|
|
167
|
-
```
|
|
168
|
-
|
|
169
|
-
- `worker` names the registered worker whose status you want to check.
|
|
170
|
-
- `source_id` identifies its input. Using `YourModel.id` checks the corresponding input for each entry considered by the query.
|
|
171
|
-
- `statuses` contains the acceptable statuses. The condition matches if any one applies; an input with no execution record does not match.
|
|
172
|
-
|
|
173
|
-
Use the condition in an ordinary SQLAlchemy query:
|
|
174
|
-
|
|
175
|
-
```python
|
|
176
|
-
query = select(YourModel).where(finished)
|
|
177
|
-
|
|
178
|
-
with session_factory() as session:
|
|
179
|
-
inputs = session.scalars(query).all()
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
This returns inputs whose work under `process` is finished. The database checks their statuses as part of this query.
|
|
183
|
-
|
|
184
|
-
The same condition can be combined with other query filters, including in an `eligible` query. Register the named worker before calling `has_execution_status()`.
|
|
185
|
-
|
|
186
|
-
## Understand failures and retry work
|
|
187
|
-
|
|
188
|
-
When a handler raises an exception, DBWorker records the error and marks the work as `FAILED`. It will not automatically run that work again. Your application may need to show what went wrong and let someone retry after correcting the cause.
|
|
189
|
-
|
|
190
|
-
`state()` reads the execution details for one input, giving you more information than its status alone:
|
|
191
|
-
|
|
192
|
-
```python
|
|
193
|
-
with session_factory() as session:
|
|
194
|
-
state = coordinator.workers["process"].state(session, source_id)
|
|
195
|
-
error = state["error"] if state is not None else None
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
`"process"` is the worker's registered name, and `source_id` is the input's primary key. The result is a mapping containing `execution_status`, the recorded `error`, and `lease_expires_at` (the claim's expiration time). It returns `None` if that worker has never claimed this input. You can use the error to explain the failure in your API or logs.
|
|
199
|
-
|
|
200
|
-
After correcting the cause, use `reset_failed()` to allow another attempt:
|
|
201
|
-
|
|
202
|
-
```python
|
|
203
|
-
with session_factory.begin() as session:
|
|
204
|
-
reset = coordinator.workers["process"].reset_failed(session, source_id)
|
|
205
|
-
```
|
|
206
|
-
|
|
207
|
-
`reset_failed()` changes a failed execution to `UNFINISHED` and clears its recorded error and claim. It returns `True` if it reset a failed execution, or `False` if there was no failed execution to reset. The `begin()` block commits this change.
|
|
208
|
-
|
|
209
|
-
Resetting does not call the handler immediately. The running coordinator can claim the work again when it matches `eligible`. The handler starts from the beginning; resetting does not delete application results or progress. If the handler has effects outside the database transaction, make them safe to repeat.
|
|
210
|
-
|
|
211
|
-
## Configuration
|
|
212
|
-
|
|
213
|
-
| `Coordinator` argument | Purpose | Default |
|
|
214
|
-
|---|---|---|
|
|
215
|
-
| `session_factory` | SQLAlchemy session factory used for coordination. | Required |
|
|
216
|
-
| `database_url` | Database URL used by child processes; use the same database. | Required |
|
|
217
|
-
| `engine_options` | SQLAlchemy `create_engine()` options for child processes. | `None` |
|
|
218
|
-
| `lease_seconds` | Claim lifetime without renewal. Active claims are renewed automatically. | `30` |
|
|
219
|
-
| `poll_seconds` | Initial wait after finding no claimable work. | `0.25` |
|
|
220
|
-
| `max_poll_seconds` | Maximum wait after exponential backoff. Successful claims continue without waiting. | `10` |
|
|
221
|
-
|
|
222
|
-
For SQLite, use a file-backed database. Worker names must start with a lowercase letter and contain only lowercase letters, digits and underscores.
|
|
223
|
-
|
|
224
|
-
## Examples
|
|
225
|
-
|
|
226
|
-
- [Image deduplication](examples/imagededup_system_dbwork/README.md): independent FastAPI and worker services, artifact building, paged comparisons, and worker dependencies.
|
|
227
|
-
- [Redis + Celery equivalent](examples/imagededup_system_redis_celery/README.md).
|
|
228
|
-
- [Benchmarks](benchmarks/imagededup_benckmark/README.md) with structured JSON results.
|
|
229
|
-
|
|
230
|
-
## License
|
|
231
|
-
|
|
232
|
-
[MIT](LICENSE) © 2026 Ziyang Song.
|
|
233
|
-
|
dbworker-0.0.6/README.md
DELETED
|
@@ -1,215 +0,0 @@
|
|
|
1
|
-
# dbworker
|
|
2
|
-
|
|
3
|
-
**Your database rows are already the queue.**
|
|
4
|
-
|
|
5
|
-
A portable, single-script background worker for SQLAlchemy. No broker, no enqueue, no duplicate job model.
|
|
6
|
-
|
|
7
|
-
```text
|
|
8
|
-
Celery / Huey / Graphile Worker:
|
|
9
|
-
event → job → worker
|
|
10
|
-
|
|
11
|
-
dbworker:
|
|
12
|
-
row → worker
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
Requires Python 3.12+. Install into your application:
|
|
16
|
-
|
|
17
|
-
```sh
|
|
18
|
-
poetry add dbworker
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
## Quick start
|
|
22
|
-
|
|
23
|
-
Use your existing SQLAlchemy `session_factory` and database URL:
|
|
24
|
-
|
|
25
|
-
```python
|
|
26
|
-
from dbworker import Coordinator
|
|
27
|
-
|
|
28
|
-
coordinator = Coordinator(session_factory, database_url=database_url)
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
You write a **handler**: a Python function containing the work you want to run. DBWorker runs handlers in separate processes.
|
|
32
|
-
|
|
33
|
-
Before calling your handler, DBWorker chooses the next task and marks it as “being worked on” in the database. This step is called a **claim**. It lets several processes share the tasks without choosing the same task at the same time.
|
|
34
|
-
|
|
35
|
-
DBWorker keeps the claim active until your handler finishes. If the process crashes, the claim expires so another process can take over the task.
|
|
36
|
-
|
|
37
|
-
`source` is the database table that supplies input to your handler, specified as a SQLAlchemy model. It can store user requests, or any entries whose creation should automatically start a procedure. Each new entry gives DBWorker new work to run, and the selected entry is passed to your handler. Its primary key identifies that work so DBWorker can track its completion; the model must have a single primary-key column.
|
|
38
|
-
|
|
39
|
-
`eligible` controls which inputs can be claimed next. Without it, any new entry can be picked up. Add it when work should wait for a condition or run in a particular order. In the example below, `YourModel` stands for your model. The `enabled` filter and ordering by `id` illustrate a selection rule; replace them with your own conditions and ordering.
|
|
40
|
-
|
|
41
|
-
Decorate your handler like this. The comments describe where your application logic goes:
|
|
42
|
-
|
|
43
|
-
**Complete the work in one invocation:**
|
|
44
|
-
|
|
45
|
-
```python
|
|
46
|
-
from sqlalchemy import select
|
|
47
|
-
from sqlalchemy.orm import Session
|
|
48
|
-
from dbworker import Finished
|
|
49
|
-
|
|
50
|
-
@coordinator.transactional_worker(
|
|
51
|
-
name="process",
|
|
52
|
-
source=YourModel,
|
|
53
|
-
eligible=lambda: (
|
|
54
|
-
select(YourModel)
|
|
55
|
-
.where(YourModel.enabled.is_(True))
|
|
56
|
-
.order_by(YourModel.id)
|
|
57
|
-
),
|
|
58
|
-
concurrency=4,
|
|
59
|
-
)
|
|
60
|
-
def process(source: YourModel, session: Session) -> Finished:
|
|
61
|
-
# Your procedure goes here.
|
|
62
|
-
# Return Finished() when the procedure is complete.
|
|
63
|
-
return Finished()
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
- `name` identifies the worker for status queries.
|
|
67
|
-
- `concurrency=4` allows four handlers to run in child processes.
|
|
68
|
-
- `source` is the selected instance of your source model.
|
|
69
|
-
- `session` is a normal SQLAlchemy session supplied by DBWorker. The handler can leave either argument unused.
|
|
70
|
-
|
|
71
|
-
Before each claim, DBWorker calls `eligible` and uses its query to select the next item. If nothing can be claimed, it waits and checks again. If your application later enables an item, a subsequent check can select it. Finished, failed, and currently claimed items are excluded automatically; you do not need to put those checks in your query. Omitting `eligible` lets DBWorker select from all entries in `source`.
|
|
72
|
-
|
|
73
|
-
**Process part of the work, then continue in another invocation:**
|
|
74
|
-
|
|
75
|
-
```python
|
|
76
|
-
from dbworker import Finished, Outcome, Unfinished
|
|
77
|
-
|
|
78
|
-
@coordinator.transactional_worker(
|
|
79
|
-
name="process_in_steps",
|
|
80
|
-
source=YourModel,
|
|
81
|
-
eligible=lambda: (
|
|
82
|
-
select(YourModel)
|
|
83
|
-
.where(YourModel.enabled.is_(True))
|
|
84
|
-
.order_by(YourModel.id)
|
|
85
|
-
),
|
|
86
|
-
concurrency=4,
|
|
87
|
-
)
|
|
88
|
-
def process_in_steps(source: YourModel, session: Session) -> Outcome:
|
|
89
|
-
# Your procedure goes here.
|
|
90
|
-
# Return Unfinished() if it needs another invocation to continue.
|
|
91
|
-
# Return Finished() instead when it is complete.
|
|
92
|
-
return Unfinished()
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
`Finished()` means the work item is complete. `Unfinished()` means this invocation is done, but more work remains. Both save the execution status and commit any writes made through the supplied session; the handler does not have to make any database writes. After `Unfinished()`, DBWorker can invoke the handler again when eligible. Each invocation starts the function from the beginning. Your procedure decides how to continue; DBWorker does not save its position in the function. Use `Outcome` as the return annotation when a handler can return either result.
|
|
96
|
-
|
|
97
|
-
If the handler raises an exception, DBWorker rolls back its transaction and marks the work failed. If your procedure has prerequisites that can be checked in the database, `eligible` can delay claiming until they are met.
|
|
98
|
-
|
|
99
|
-
DBWorker owns the handler's commit and session cleanup. Do not commit or close this session yourself. For long computations, you can release a read transaction with `session.rollback()` first. Copy needed values before rollback, and do this before making writes you want to keep.
|
|
100
|
-
|
|
101
|
-
### Start and stop
|
|
102
|
-
|
|
103
|
-
Register handlers at module scope in an importable module. In your worker service, initialize DBWorker's tables and start the coordinator:
|
|
104
|
-
|
|
105
|
-
```python
|
|
106
|
-
if __name__ == "__main__":
|
|
107
|
-
coordinator.create_worker_tables()
|
|
108
|
-
coordinator.start()
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
`start()` returns while workers keep running. When a source entry matches `eligible`, DBWorker can claim its work and call the handler—there is no enqueue call.
|
|
112
|
-
|
|
113
|
-
On service shutdown, call:
|
|
114
|
-
|
|
115
|
-
```python
|
|
116
|
-
coordinator.stop()
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
`stop()` stops new claims and waits for active handlers to finish. Your API can run independently, using the same database. See the [complete example](examples/imagededup_system_dbwork/README.md) for runnable API and worker commands with shutdown handling.
|
|
120
|
-
|
|
121
|
-
## Track progress and coordinate dependent work
|
|
122
|
-
|
|
123
|
-
Your application may need to show whether work is running or complete. Another worker may also depend on that information: one procedure prepares something, and a second can begin only after preparation finishes. If preparation fails, the application may need to report that failure instead of continuing.
|
|
124
|
-
|
|
125
|
-
DBWorker tracks execution separately for each worker and source entry. Access it through the coordinator using the worker's registered name and the source entry's primary key; you do not need to manage or query DBWorker's internal tables yourself.
|
|
126
|
-
|
|
127
|
-
Use `execution_status()` to get the current status in your API or handler:
|
|
128
|
-
|
|
129
|
-
```python
|
|
130
|
-
from dbworker import ExecutionStatus
|
|
131
|
-
|
|
132
|
-
with session_factory() as session:
|
|
133
|
-
status = coordinator.execution_status(
|
|
134
|
-
session, worker="process", source_id=source_id,
|
|
135
|
-
)
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
This returns `None` before the first claim, or an `ExecutionStatus` enum: `WORKING`, `UNFINISHED`, `FINISHED`, or `FAILED`.
|
|
139
|
-
|
|
140
|
-
Sometimes you need to select inputs based on the status of their work—for example, list only inputs whose processing has finished. Calling `execution_status()` for each input would mean checking them individually.
|
|
141
|
-
|
|
142
|
-
`has_execution_status()` lets you include that check in a database query. It returns a SQLAlchemy condition meaning: **“Does this worker have one of these execution statuses for this input?”** It does not run a query or return a Python `True` or `False` when called.
|
|
143
|
-
|
|
144
|
-
```python
|
|
145
|
-
finished = coordinator.has_execution_status(
|
|
146
|
-
worker="process",
|
|
147
|
-
source_id=YourModel.id,
|
|
148
|
-
statuses=(ExecutionStatus.FINISHED,),
|
|
149
|
-
)
|
|
150
|
-
```
|
|
151
|
-
|
|
152
|
-
- `worker` names the registered worker whose status you want to check.
|
|
153
|
-
- `source_id` identifies its input. Using `YourModel.id` checks the corresponding input for each entry considered by the query.
|
|
154
|
-
- `statuses` contains the acceptable statuses. The condition matches if any one applies; an input with no execution record does not match.
|
|
155
|
-
|
|
156
|
-
Use the condition in an ordinary SQLAlchemy query:
|
|
157
|
-
|
|
158
|
-
```python
|
|
159
|
-
query = select(YourModel).where(finished)
|
|
160
|
-
|
|
161
|
-
with session_factory() as session:
|
|
162
|
-
inputs = session.scalars(query).all()
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
This returns inputs whose work under `process` is finished. The database checks their statuses as part of this query.
|
|
166
|
-
|
|
167
|
-
The same condition can be combined with other query filters, including in an `eligible` query. Register the named worker before calling `has_execution_status()`.
|
|
168
|
-
|
|
169
|
-
## Understand failures and retry work
|
|
170
|
-
|
|
171
|
-
When a handler raises an exception, DBWorker records the error and marks the work as `FAILED`. It will not automatically run that work again. Your application may need to show what went wrong and let someone retry after correcting the cause.
|
|
172
|
-
|
|
173
|
-
`state()` reads the execution details for one input, giving you more information than its status alone:
|
|
174
|
-
|
|
175
|
-
```python
|
|
176
|
-
with session_factory() as session:
|
|
177
|
-
state = coordinator.workers["process"].state(session, source_id)
|
|
178
|
-
error = state["error"] if state is not None else None
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
`"process"` is the worker's registered name, and `source_id` is the input's primary key. The result is a mapping containing `execution_status`, the recorded `error`, and `lease_expires_at` (the claim's expiration time). It returns `None` if that worker has never claimed this input. You can use the error to explain the failure in your API or logs.
|
|
182
|
-
|
|
183
|
-
After correcting the cause, use `reset_failed()` to allow another attempt:
|
|
184
|
-
|
|
185
|
-
```python
|
|
186
|
-
with session_factory.begin() as session:
|
|
187
|
-
reset = coordinator.workers["process"].reset_failed(session, source_id)
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
`reset_failed()` changes a failed execution to `UNFINISHED` and clears its recorded error and claim. It returns `True` if it reset a failed execution, or `False` if there was no failed execution to reset. The `begin()` block commits this change.
|
|
191
|
-
|
|
192
|
-
Resetting does not call the handler immediately. The running coordinator can claim the work again when it matches `eligible`. The handler starts from the beginning; resetting does not delete application results or progress. If the handler has effects outside the database transaction, make them safe to repeat.
|
|
193
|
-
|
|
194
|
-
## Configuration
|
|
195
|
-
|
|
196
|
-
| `Coordinator` argument | Purpose | Default |
|
|
197
|
-
|---|---|---|
|
|
198
|
-
| `session_factory` | SQLAlchemy session factory used for coordination. | Required |
|
|
199
|
-
| `database_url` | Database URL used by child processes; use the same database. | Required |
|
|
200
|
-
| `engine_options` | SQLAlchemy `create_engine()` options for child processes. | `None` |
|
|
201
|
-
| `lease_seconds` | Claim lifetime without renewal. Active claims are renewed automatically. | `30` |
|
|
202
|
-
| `poll_seconds` | Initial wait after finding no claimable work. | `0.25` |
|
|
203
|
-
| `max_poll_seconds` | Maximum wait after exponential backoff. Successful claims continue without waiting. | `10` |
|
|
204
|
-
|
|
205
|
-
For SQLite, use a file-backed database. Worker names must start with a lowercase letter and contain only lowercase letters, digits and underscores.
|
|
206
|
-
|
|
207
|
-
## Examples
|
|
208
|
-
|
|
209
|
-
- [Image deduplication](examples/imagededup_system_dbwork/README.md): independent FastAPI and worker services, artifact building, paged comparisons, and worker dependencies.
|
|
210
|
-
- [Redis + Celery equivalent](examples/imagededup_system_redis_celery/README.md).
|
|
211
|
-
- [Benchmarks](benchmarks/imagededup_benckmark/README.md) with structured JSON results.
|
|
212
|
-
|
|
213
|
-
## License
|
|
214
|
-
|
|
215
|
-
[MIT](LICENSE) © 2026 Ziyang Song.
|
|
File without changes
|
|
File without changes
|