outlabs-taskq 0.1.0a26__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- outlabs_taskq-0.1.0a26/.gitignore +36 -0
- outlabs_taskq-0.1.0a26/LICENSE +21 -0
- outlabs_taskq-0.1.0a26/PKG-INFO +282 -0
- outlabs_taskq-0.1.0a26/README.md +218 -0
- outlabs_taskq-0.1.0a26/docs/CLI-MIGRATION.md +40 -0
- outlabs_taskq-0.1.0a26/docs/CLI.md +249 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a19.md +67 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a20.md +48 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a21.md +49 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a22.md +66 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a23.md +49 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a24.md +35 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a25.md +51 -0
- outlabs_taskq-0.1.0a26/docs/RELEASE-0.1.0a26.md +41 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue 0.1 Function Manifest.md +1819 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Outlabs Auth Composition Remediation Specification.md +191 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Read Model Specification.md +311 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Stage 2A Typed Enqueue Specification.md +203 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Stage 2B Worker Runtime Specification.md +401 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Stage 2C Claim Loop and Worker CLI Specification.md +351 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Stage 2D Consumer Testing Specification.md +217 -0
- outlabs_taskq-0.1.0a26/docs/Task Queue Stage 3 FastAPI and Authorization Specification.md +664 -0
- outlabs_taskq-0.1.0a26/docs/TaskQ Standalone Scheduler Specification.md +254 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-001-product-boundary.md +22 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-002-fixed-schema-sql-ownership.md +21 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-003-fencing-typed-outcomes.md +22 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-004-migrations-canonical.md +22 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-007-atomic-followups-fenced-cancel.md +29 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-012-null-boundaries-byte-safe-diagnostics.md +29 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-013-effective-lease-in-claim-projection.md +26 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-014-http-worker-presence.md +57 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-015-defer-queue-profile-read.md +52 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-016-final-http-wire-normalization.md +64 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-017-final-manifest-backed-wire-corrections.md +58 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-019-safe-read-model-reactivation.md +77 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-020-supported-sql-contract-sets.md +60 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-021-read-model-conformance-repairs.md +69 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-025-followup-helper-return-shape.md +34 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-028-maintenance-schedule-http-boundary.md +59 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-030-workflow-counter-lock-order.md +57 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-036-trusted-effect-fence.md +83 -0
- outlabs_taskq-0.1.0a26/docs/adr/ADR-037-standalone-scheduler-and-target-attestation.md +152 -0
- outlabs_taskq-0.1.0a26/docs/adr/README.md +29 -0
- outlabs_taskq-0.1.0a26/docs/evidence/scheduler-0.3.0-2026-08-03.md +84 -0
- outlabs_taskq-0.1.0a26/docs/pilots/qdarte-staging.md +77 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/01-typed-enqueue-results.md +141 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/02-job-key-and-uniqueness-modes.md +172 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/03-handler-settle-results.md +151 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/04-insert-only-client.md +80 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/05-queue-profiles.md +94 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/06-notify-nudge-and-poll.md +80 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/07-dead-letter-lineage-and-redrive.md +121 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/09-retry-value-surface.md +85 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/10-test-helpers.md +114 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/12-migrate-break-channel.md +53 -0
- outlabs_taskq-0.1.0a26/docs/taskq-borrowed-features/13-sql-packaging-conventions.md +106 -0
- outlabs_taskq-0.1.0a26/docs/workflow-continuations/evidence/wfc-i00-catalog-pg16-pg18-20260728.json +41 -0
- outlabs_taskq-0.1.0a26/docs/workflow-continuations/wfc-i00-catalog-delta.json +195 -0
- outlabs_taskq-0.1.0a26/docs/workflow-continuations/wfc-i01-canonical-vectors.json +29 -0
- outlabs_taskq-0.1.0a26/examples/qdarte-staging-intake-review.yaml +18 -0
- outlabs_taskq-0.1.0a26/examples/schedules.minimal.yaml +10 -0
- outlabs_taskq-0.1.0a26/pyproject.toml +98 -0
- outlabs_taskq-0.1.0a26/src/taskq/__init__.py +258 -0
- outlabs_taskq-0.1.0a26/src/taskq/bench.py +1439 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/__init__.py +35 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/__main__.py +3 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/_runtime.py +924 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/app.py +2364 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/context.py +236 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/cursor.py +62 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/errors.py +162 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/models.py +226 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/output.py +182 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/specs.py +308 -0
- outlabs_taskq-0.1.0a26/src/taskq/cli/transport.py +356 -0
- outlabs_taskq-0.1.0a26/src/taskq/client.py +417 -0
- outlabs_taskq-0.1.0a26/src/taskq/continuations.py +394 -0
- outlabs_taskq-0.1.0a26/src/taskq/errors.py +171 -0
- outlabs_taskq-0.1.0a26/src/taskq/execution.py +310 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/__init__.py +86 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/client.py +1761 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/deps.py +226 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/facade.py +1753 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/hub.py +109 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/outlabs.py +578 -0
- outlabs_taskq-0.1.0a26/src/taskq/http/runtime.py +831 -0
- outlabs_taskq-0.1.0a26/src/taskq/protocol.py +2784 -0
- outlabs_taskq-0.1.0a26/src/taskq/registry.py +351 -0
- outlabs_taskq-0.1.0a26/src/taskq/scheduler.py +689 -0
- outlabs_taskq-0.1.0a26/src/taskq/schedules.py +167 -0
- outlabs_taskq-0.1.0a26/src/taskq/settings.py +83 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/__init__.py +1232 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/effects.py +108 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/manifest.py +706 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0001_initial.sql +2628 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0002_contract_0_1_1.sql +666 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0003_contract_0_1_2.sql +161 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0004_read_models.sql +261 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0005_read_model_conformance.sql +92 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0006_activate_ready_read_model.sql +24 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0007_admission_reservations.sql +600 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0008_followups.sql +347 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0009_workflows.sql +1415 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0010_schedules.sql +1123 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0011_finite_projections.sql +264 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0012_activate_finite_projections.sql +31 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0013_workflow_page_composite_repair.sql +103 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0014_worker_presence_projection.sql +195 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0015_activate_worker_presence.sql +38 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0016_workflow_continuations.sql +1871 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0017_activate_workflow_continuations.sql +78 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0018_trusted_effect_fence.sql +102 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0019_scheduler_target_identity.sql +389 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0020_standalone_scheduler.sql +1080 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/migrations/0021_cli_read_model.sql +533 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/notifications.py +134 -0
- outlabs_taskq-0.1.0a26/src/taskq/sql/transport.py +1851 -0
- outlabs_taskq-0.1.0a26/src/taskq/testing.py +2138 -0
- outlabs_taskq-0.1.0a26/src/taskq/transport.py +482 -0
- outlabs_taskq-0.1.0a26/src/taskq/worker.py +1753 -0
- outlabs_taskq-0.1.0a26/tests/conftest.py +418 -0
- outlabs_taskq-0.1.0a26/tests/test_bench_smoke.py +105 -0
- outlabs_taskq-0.1.0a26/tests/test_cli_v1.py +644 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_1_1.py +395 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_1_2.py +175 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_1_3.py +110 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_1_5.py +517 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_0.py +348 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_1.py +912 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_2.py +611 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_3.py +277 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_4.py +220 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_5.py +133 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_2_6.py +329 -0
- outlabs_taskq-0.1.0a26/tests/test_contract_0_3_1.py +370 -0
- outlabs_taskq-0.1.0a26/tests/test_installer_matrix.py +300 -0
- outlabs_taskq-0.1.0a26/tests/test_job_detail_decode.py +28 -0
- outlabs_taskq-0.1.0a26/tests/test_manifest_parity.py +472 -0
- outlabs_taskq-0.1.0a26/tests/test_migration_lock_recovery.py +127 -0
- outlabs_taskq-0.1.0a26/tests/test_plans.py +823 -0
- outlabs_taskq-0.1.0a26/tests/test_reserved_roles.py +144 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_client.py +303 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_registry.py +671 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_sql_transport.py +661 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_testing_audit.py +122 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_testing_fake.py +350 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_testing_helpers.py +346 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_cli.py +178 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_cli_subprocess.py +195 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_execution.py +309 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_lifecycle.py +487 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_races.py +314 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_service_audit.py +123 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_service_lifecycle.py +258 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_service_poll.py +277 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_settlement.py +436 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_sql.py +362 -0
- outlabs_taskq-0.1.0a26/tests/test_s2_worker_supervision.py +399 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_facade.py +1299 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_facade_sql.py +153 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_http_catalog.py +639 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_http_clients.py +559 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_http_worker_cli.py +102 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_notification_channels.py +115 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_outlabs.py +746 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_runtime.py +750 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_runtime_sql.py +416 -0
- outlabs_taskq-0.1.0a26/tests/test_s3_sql_http_parity.py +507 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_admission_surface.py +630 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_schedule_evaluator.py +208 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_schedule_surface.py +281 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_schedule_testing.py +119 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_trusted_effect_reporter.py +308 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_workflow_surface.py +595 -0
- outlabs_taskq-0.1.0a26/tests/test_s5_workflow_testing.py +226 -0
- outlabs_taskq-0.1.0a26/tests/test_scheduler.py +160 -0
- outlabs_taskq-0.1.0a26/tests/test_scheduler_sql.py +411 -0
- outlabs_taskq-0.1.0a26/tests/test_scheduler_target_cli.py +231 -0
- outlabs_taskq-0.1.0a26/tests/test_t1_unit.py +279 -0
- outlabs_taskq-0.1.0a26/tests/test_t2_contract.py +526 -0
- outlabs_taskq-0.1.0a26/tests/test_t3_races.py +446 -0
- outlabs_taskq-0.1.0a26/tests/test_t3_stress.py +368 -0
- outlabs_taskq-0.1.0a26/tests/test_t4_model.py +447 -0
- outlabs_taskq-0.1.0a26/tests/test_verify_manifest.py +382 -0
- outlabs_taskq-0.1.0a26/tests/test_version.py +8 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i00_catalog_delta.py +150 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i00_claim_plan.py +83 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i01_continuations.py +468 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i03_runtime.py +511 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i04_races.py +432 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i04_settlement.py +530 -0
- outlabs_taskq-0.1.0a26/tests/test_wfc_i05_activation.py +271 -0
- outlabs_taskq-0.1.0a26/tests/worker_cli_registry.py +63 -0
- outlabs_taskq-0.1.0a26/tests/worker_support.py +314 -0
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
*.egg-info/
|
|
6
|
+
.eggs/
|
|
7
|
+
dist/
|
|
8
|
+
build/
|
|
9
|
+
.venv/
|
|
10
|
+
venv/
|
|
11
|
+
.env
|
|
12
|
+
.env.*
|
|
13
|
+
!.env.example
|
|
14
|
+
|
|
15
|
+
# Test / tooling
|
|
16
|
+
.pytest_cache/
|
|
17
|
+
.ruff_cache/
|
|
18
|
+
.mypy_cache/
|
|
19
|
+
.coverage
|
|
20
|
+
htmlcov/
|
|
21
|
+
.tox/
|
|
22
|
+
.cache/
|
|
23
|
+
|
|
24
|
+
# OS / editor
|
|
25
|
+
.DS_Store
|
|
26
|
+
.idea/
|
|
27
|
+
.vscode/
|
|
28
|
+
*.swp
|
|
29
|
+
|
|
30
|
+
# Local
|
|
31
|
+
*.log
|
|
32
|
+
tmp/
|
|
33
|
+
.direnv/
|
|
34
|
+
bench/results/*.json
|
|
35
|
+
.claude/settings.local.json
|
|
36
|
+
.codex-*
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 OUTLABS LLC
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: outlabs-taskq
|
|
3
|
+
Version: 0.1.0a26
|
|
4
|
+
Summary: Postgres-native task queue: SQL-first contract, Python client, optional FastAPI facade.
|
|
5
|
+
Project-URL: Homepage, https://outlabs.io
|
|
6
|
+
Project-URL: Repository, https://github.com/outlabsio/outlabs-taskq
|
|
7
|
+
Project-URL: Documentation, https://github.com/outlabsio/outlabs-taskq/tree/main/docs
|
|
8
|
+
Author-email: OUTLABS LLC <contact@outlabs.io>
|
|
9
|
+
Maintainer-email: OUTLABS LLC <contact@outlabs.io>
|
|
10
|
+
License: MIT License
|
|
11
|
+
|
|
12
|
+
Copyright (c) 2026 OUTLABS LLC
|
|
13
|
+
|
|
14
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
15
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
16
|
+
in the Software without restriction, including without limitation the rights
|
|
17
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
18
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
19
|
+
furnished to do so, subject to the following conditions:
|
|
20
|
+
|
|
21
|
+
The above copyright notice and this permission notice shall be included in all
|
|
22
|
+
copies or substantial portions of the Software.
|
|
23
|
+
|
|
24
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
25
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
26
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
27
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
28
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
29
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
30
|
+
SOFTWARE.
|
|
31
|
+
License-File: LICENSE
|
|
32
|
+
Keywords: async,fastapi,job-queue,postgres,postgresql,pydantic,task-queue
|
|
33
|
+
Classifier: Development Status :: 2 - Pre-Alpha
|
|
34
|
+
Classifier: Framework :: FastAPI
|
|
35
|
+
Classifier: Intended Audience :: Developers
|
|
36
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
37
|
+
Classifier: Operating System :: OS Independent
|
|
38
|
+
Classifier: Programming Language :: Python :: 3
|
|
39
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
40
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
41
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
42
|
+
Requires-Python: >=3.12
|
|
43
|
+
Requires-Dist: asyncpg>=0.29.0
|
|
44
|
+
Requires-Dist: click<9.0,>=8.1.0
|
|
45
|
+
Requires-Dist: croniter>=2.0.0
|
|
46
|
+
Requires-Dist: pydantic-settings>=2.0.0
|
|
47
|
+
Requires-Dist: pydantic>=2.0.0
|
|
48
|
+
Requires-Dist: pyyaml>=6.0.0
|
|
49
|
+
Requires-Dist: sqlalchemy[asyncio]>=2.0.0
|
|
50
|
+
Provides-Extra: dev
|
|
51
|
+
Requires-Dist: hypothesis>=6.0.0; extra == 'dev'
|
|
52
|
+
Requires-Dist: psycopg[binary]<4.0,>=3.2.0; extra == 'dev'
|
|
53
|
+
Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
|
|
54
|
+
Requires-Dist: pytest>=8.0.0; extra == 'dev'
|
|
55
|
+
Requires-Dist: ruff>=0.6.0; extra == 'dev'
|
|
56
|
+
Provides-Extra: http
|
|
57
|
+
Requires-Dist: fastapi>=0.115.0; extra == 'http'
|
|
58
|
+
Requires-Dist: httpx>=0.27.0; extra == 'http'
|
|
59
|
+
Provides-Extra: outlabs
|
|
60
|
+
Requires-Dist: fastapi>=0.115.0; extra == 'outlabs'
|
|
61
|
+
Requires-Dist: httpx>=0.27.0; extra == 'outlabs'
|
|
62
|
+
Requires-Dist: outlabs-auth<0.2.0,>=0.1.0a27; extra == 'outlabs'
|
|
63
|
+
Description-Content-Type: text/markdown
|
|
64
|
+
|
|
65
|
+
# outlabs-taskq
|
|
66
|
+
|
|
67
|
+
Postgres-native durable task queue for Python services.
|
|
68
|
+
|
|
69
|
+
**Status:** alpha — **`0.1.0a26`** uses SQL contract **`0.3.1`** and Protocol revision **`1.0.16`**. The resource-oriented, non-interactive CLI is a complete operator and coding-agent surface over direct PostgreSQL and HTTP. Migration `0021` adds bounded job-event, workflow, schedule, and missing job-view projections.
|
|
70
|
+
|
|
71
|
+
SQL functions in schema `taskq` are the contract. The Python package provides the installer, typed client, worker runtime, and an optional FastAPI facade. `outlabs-auth` is an optional adapter, not a hard dependency. Queue storage may be co-resident with the host database or dedicated; the HTTP facade may use OutLabsAuth, a host-supplied/remote authorizer, or simple packaged credentials, while trusted direct-SQL deployments use PostgreSQL capability roles.
|
|
72
|
+
|
|
73
|
+
## Docs
|
|
74
|
+
|
|
75
|
+
Start here:
|
|
76
|
+
|
|
77
|
+
| Doc | What it is |
|
|
78
|
+
|---|---|
|
|
79
|
+
| [`docs/CLI.md`](docs/CLI.md) | Operator/agent CLI contract, contexts, output, safety, and command catalog |
|
|
80
|
+
| [`docs/CLI-MIGRATION.md`](docs/CLI-MIGRATION.md) | Intentional alpha grammar migration; there are no compatibility aliases |
|
|
81
|
+
| [`docs/adr/`](docs/adr/README.md) | Accepted reusable architecture decisions |
|
|
82
|
+
| [`docs/Task Queue 0.1 Function Manifest.md`](docs/Task%20Queue%200.1%20Function%20Manifest.md) | **Canonical 0.1 SQL surface** — migration 0001 derives from this |
|
|
83
|
+
| [`docs/Task Queue Stage 2A Typed Enqueue Specification.md`](docs/Task%20Queue%20Stage%202A%20Typed%20Enqueue%20Specification.md) | Typed enqueue contract |
|
|
84
|
+
| [`docs/Task Queue Stage 2B Worker Runtime Specification.md`](docs/Task%20Queue%20Stage%202B%20Worker%20Runtime%20Specification.md) | Worker runtime behavior |
|
|
85
|
+
| [`docs/Task Queue Stage 3 FastAPI and Authorization Specification.md`](docs/Task%20Queue%20Stage%203%20FastAPI%20and%20Authorization%20Specification.md) | Optional HTTP and authorization integration |
|
|
86
|
+
| [`docs/RELEASE-0.1.0a26.md`](docs/RELEASE-0.1.0a26.md) | 0.1.0a26 production-integration closeout and rollout checklist |
|
|
87
|
+
| [`docs/RELEASE-0.1.0a25.md`](docs/RELEASE-0.1.0a25.md) | 0.1.0a25 complete operator read model and CLI foundation |
|
|
88
|
+
| [`docs/RELEASE-0.1.0a24.md`](docs/RELEASE-0.1.0a24.md) | Historical unreleased a24 candidate record |
|
|
89
|
+
| [`docs/RELEASE-0.1.0a22.md`](docs/RELEASE-0.1.0a22.md) | 0.1.0a22 standalone scheduler release notes and rollout checklist |
|
|
90
|
+
| [`docs/TaskQ Standalone Scheduler Specification.md`](docs/TaskQ%20Standalone%20Scheduler%20Specification.md) | Owner-approved standalone scheduler, target-attestation, manifest, and evidence contract |
|
|
91
|
+
| [`docs/RELEASE-0.1.0a21.md`](docs/RELEASE-0.1.0a21.md) | 0.1.0a21 compatible OutLabs Auth prerelease range and checklist |
|
|
92
|
+
| [`docs/RELEASE-0.1.0a20.md`](docs/RELEASE-0.1.0a20.md) | 0.1.0a20 OutLabs Auth a27 compatibility release and checklist |
|
|
93
|
+
| [`docs/RELEASE-0.1.0a19.md`](docs/RELEASE-0.1.0a19.md) | 0.1.0a19 compatibility fix, hardening changes, and release checklist |
|
|
94
|
+
|
|
95
|
+
## Install
|
|
96
|
+
|
|
97
|
+
Install the exact published prerelease selected by the consumer lockfile:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
pip install outlabs-taskq==0.1.0a26
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Credential handling
|
|
104
|
+
|
|
105
|
+
Keep DSNs and HTTP credentials out of shell history and process arguments. Supply them through the
|
|
106
|
+
process environment using a secret manager, container secret, or service supervisor, then omit the
|
|
107
|
+
DSN from commands. A context stores only the environment-variable name:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
# TASKQ_STAGING_DSN is supplied by the deployment environment.
|
|
111
|
+
taskq --context staging db plan -o json
|
|
112
|
+
taskq --context staging db verify
|
|
113
|
+
|
|
114
|
+
# A context file is optional when the caller supplies target identity explicitly.
|
|
115
|
+
taskq --dsn-env TASKQ_STAGING_DSN \
|
|
116
|
+
--expected-environment staging --actor operator:release-agent \
|
|
117
|
+
db verify -o json
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
`taskq db migrate` requires a superuser or a managed database-owner role with
|
|
121
|
+
`CREATEROLE`. On PostgreSQL 16/18, the installer bootstraps `taskq_owner` and
|
|
122
|
+
retains owner membership on that migration role so later owner-only binding
|
|
123
|
+
and upgrades work. This must be a dedicated owner/migration credential: never
|
|
124
|
+
give it to an API, worker, or scheduler. Runtime logins receive only the
|
|
125
|
+
required TaskQ capability roles.
|
|
126
|
+
|
|
127
|
+
Migration `0019` deliberately stops at an unbound target before scheduler
|
|
128
|
+
activation. Inspect and bind the safe fingerprint, then resume migration:
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
PLAN=$(taskq --context staging db plan -o json)
|
|
132
|
+
# Review PLAN and extract data.plan_digest.
|
|
133
|
+
taskq --context staging --yes db migrate --plan-digest "$PLAN_DIGEST"
|
|
134
|
+
taskq --context staging target show -o json
|
|
135
|
+
taskq --context staging --yes target bind staging \
|
|
136
|
+
--expected-installation-id "$TASKQ_INSTALLATION_ID" \
|
|
137
|
+
--expected-binding-version 0
|
|
138
|
+
taskq --context staging db plan -o json
|
|
139
|
+
taskq --context staging --yes db migrate --plan-digest "$PLAN_DIGEST"
|
|
140
|
+
taskq --context staging db verify
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
Run the framework-neutral scheduler with static target expectations. The
|
|
144
|
+
bounded mode is first-class for platform timers and scale-to-zero databases:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
taskq --context staging scheduler run
|
|
148
|
+
taskq --context staging scheduler once -o json
|
|
149
|
+
taskq --context staging scheduler doctor -o json
|
|
150
|
+
|
|
151
|
+
taskq --context staging schedule manifest plan examples/schedules.minimal.yaml -o json
|
|
152
|
+
taskq --context staging schedule manifest apply examples/schedules.minimal.yaml \
|
|
153
|
+
--plan-digest "$PLAN_DIGEST"
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Activating or resuming an interval schedule is `from now`: the first scheduler
|
|
157
|
+
pass initializes its recurrence and the first job becomes due after one full
|
|
158
|
+
interval. Activation does not enqueue an immediate job. Size attended pilot
|
|
159
|
+
windows accordingly, or use an explicit one-shot job when an immediate canary
|
|
160
|
+
is required.
|
|
161
|
+
|
|
162
|
+
The scheduler is a clock, not a task executor. Run one supervised scheduler per
|
|
163
|
+
database/environment, but place workers according to the application's existing
|
|
164
|
+
operating model. A single host-native worker can use a combined registry and
|
|
165
|
+
subscribe to multiple queues; TaskQ does not require one container per worker,
|
|
166
|
+
queue, task, or schedule. Prefer existing local worker hosts unless a particular
|
|
167
|
+
task has a documented cloud availability, latency, or network requirement.
|
|
168
|
+
|
|
169
|
+
Every recurring source manifest must set `catchup: fire_once` or
|
|
170
|
+
`catchup: fire_all` explicitly. Do not rely on the current `skip` default: the
|
|
171
|
+
released SQL contract intentionally accepts only zero occurrences for that
|
|
172
|
+
policy, so continuous polling advances without enqueueing. A corrected
|
|
173
|
+
`skip_missed` semantic requires a new SQL migration and compatibility audit; it
|
|
174
|
+
is not being patched only in Python.
|
|
175
|
+
|
|
176
|
+
Worker services may still use their runtime settings. Operator CLI commands require an explicit
|
|
177
|
+
context or complete connection flags; they never infer a current context or endpoint. HTTP contexts
|
|
178
|
+
store a bearer-token environment-variable name, never a literal token or arbitrary header value.
|
|
179
|
+
OutLabs IAM provisioning reads `TASKQ_AUTH_DSN`.
|
|
180
|
+
|
|
181
|
+
Use `https://` for traffic that leaves a protected private network. The mounted facade intentionally
|
|
182
|
+
publishes its public wire contract at `/taskq/openapi.json`; deployments that do not want that route
|
|
183
|
+
reachable should gate it at the host application or reverse proxy.
|
|
184
|
+
|
|
185
|
+
## Package layout
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
src/taskq/
|
|
189
|
+
sql/ # migrations 0001-0021, runner/verifier, manifest, SQL transport
|
|
190
|
+
scheduler.py # standalone runtime, bounded mode, doctor, YAML plan/apply
|
|
191
|
+
protocol.py # closed command/outcome/error single-source (Tier-0 parity-tested)
|
|
192
|
+
continuations.py # pure policy compiler, negotiation, and derived identities
|
|
193
|
+
registry.py # typed Task[In, Out] registry
|
|
194
|
+
client.py # TaskQ facade: transactional typed enqueue
|
|
195
|
+
transport.py # TaskqTransport protocol
|
|
196
|
+
worker.py # supervisor + fair poll/presence/shutdown service
|
|
197
|
+
settings.py # secret-safe worker environment/CLI configuration
|
|
198
|
+
testing.py # fake client, enqueue assertions, direct work, inline and drain helpers
|
|
199
|
+
cli/ # Click registry, contexts, transports, safety, output, errors
|
|
200
|
+
http/ # optional clients, mounted facade, composable runtime/lifespan
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
## Consumer testing
|
|
204
|
+
|
|
205
|
+
Fast unit tests can replace one facade without starting a worker or database:
|
|
206
|
+
|
|
207
|
+
```python
|
|
208
|
+
from taskq.testing import FakeTaskQClient
|
|
209
|
+
|
|
210
|
+
with tq.replace_client(FakeTaskQClient()) as fake:
|
|
211
|
+
await application_call()
|
|
212
|
+
fake.assert_enqueued("mail.send", where={"payload.recipient": "me@example.test"})
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Inline mode executes registered handlers immediately; bounded drain tests queued behavior without sleeps:
|
|
216
|
+
|
|
217
|
+
```python
|
|
218
|
+
from taskq.testing import drain, inline_mode
|
|
219
|
+
|
|
220
|
+
async with inline_mode(tq) as recorder:
|
|
221
|
+
await application_call()
|
|
222
|
+
assert recorder.settled("mail.send")[0].is_complete
|
|
223
|
+
|
|
224
|
+
report = await drain(tq, queue="mail", max_jobs=100)
|
|
225
|
+
assert report.completed == 1
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
These are consumer-test conveniences, not production modes. The fake intentionally does not model PostgreSQL fencing, privileges, budgets, or transaction isolation. Use a scratch PostgreSQL transaction with `work`, `require_enqueued`, or `drain(..., connection=connection)` when those contracts matter; every helper preserves caller transaction ownership and makes runaway caps fail loudly.
|
|
229
|
+
|
|
230
|
+
## Atomic native follow-ups
|
|
231
|
+
|
|
232
|
+
A handler declares its finite child graph in the registry and returns typed children with its
|
|
233
|
+
successful result. The parent settlement and every child insert commit together; the worker never
|
|
234
|
+
receives a generic producer client.
|
|
235
|
+
|
|
236
|
+
```python
|
|
237
|
+
from taskq import Complete, Followup, FollowupTarget, Task, TaskRegistry
|
|
238
|
+
|
|
239
|
+
child = Task(
|
|
240
|
+
name="listing.enrich",
|
|
241
|
+
queue="enrichment",
|
|
242
|
+
input_model=EnrichInput,
|
|
243
|
+
output_model=EnrichOutput,
|
|
244
|
+
handler=enrich,
|
|
245
|
+
)
|
|
246
|
+
|
|
247
|
+
async def discover(payload: DiscoverInput) -> Complete:
|
|
248
|
+
return Complete(
|
|
249
|
+
result={"accepted": True},
|
|
250
|
+
followups=(
|
|
251
|
+
Followup(
|
|
252
|
+
step="enrich",
|
|
253
|
+
job_type=child.name,
|
|
254
|
+
queue=child.queue,
|
|
255
|
+
payload={"listing_id": payload.listing_id},
|
|
256
|
+
),
|
|
257
|
+
),
|
|
258
|
+
)
|
|
259
|
+
|
|
260
|
+
parent = Task(
|
|
261
|
+
name="listing.discover",
|
|
262
|
+
queue="discovery",
|
|
263
|
+
input_model=DiscoverInput,
|
|
264
|
+
output_model=DiscoverOutput,
|
|
265
|
+
followup_targets=(FollowupTarget(queue=child.queue, job_type=child.name),),
|
|
266
|
+
handler=discover,
|
|
267
|
+
)
|
|
268
|
+
|
|
269
|
+
registry = TaskRegistry((parent, child))
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
Worker construction rejects missing or queue-mismatched target declarations. HTTP completion
|
|
273
|
+
authorizes the parent queue before decoding the body, then authorizes every distinct child queue
|
|
274
|
+
before SQL; direct SQL retains the trusted runner-role boundary.
|
|
275
|
+
|
|
276
|
+
## Development gates
|
|
277
|
+
|
|
278
|
+
Protect `main` with pull requests, require branches to be current, and require the CI gates that run on pull requests: `lint`, `dependency-audit`, both `import-isolation` and `unit` Python lanes, `built-artifacts`, both PostgreSQL `sql-contract` lanes, both `fresh-cluster-security` lanes, `races`, `stage3-audit`, `migrations`, and `bench-smoke`. The scheduled/dispatchable `million-row-plans` job keeps structural plans honest without charging every pull request. Do not bypass a failed required check except through the repository's explicit break-glass process.
|
|
279
|
+
|
|
280
|
+
## License
|
|
281
|
+
|
|
282
|
+
MIT — see [LICENSE](LICENSE).
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
# outlabs-taskq
|
|
2
|
+
|
|
3
|
+
Postgres-native durable task queue for Python services.
|
|
4
|
+
|
|
5
|
+
**Status:** alpha — **`0.1.0a26`** uses SQL contract **`0.3.1`** and Protocol revision **`1.0.16`**. The resource-oriented, non-interactive CLI is a complete operator and coding-agent surface over direct PostgreSQL and HTTP. Migration `0021` adds bounded job-event, workflow, schedule, and missing job-view projections.
|
|
6
|
+
|
|
7
|
+
SQL functions in schema `taskq` are the contract. The Python package provides the installer, typed client, worker runtime, and an optional FastAPI facade. `outlabs-auth` is an optional adapter, not a hard dependency. Queue storage may be co-resident with the host database or dedicated; the HTTP facade may use OutLabsAuth, a host-supplied/remote authorizer, or simple packaged credentials, while trusted direct-SQL deployments use PostgreSQL capability roles.
|
|
8
|
+
|
|
9
|
+
## Docs
|
|
10
|
+
|
|
11
|
+
Start here:
|
|
12
|
+
|
|
13
|
+
| Doc | What it is |
|
|
14
|
+
|---|---|
|
|
15
|
+
| [`docs/CLI.md`](docs/CLI.md) | Operator/agent CLI contract, contexts, output, safety, and command catalog |
|
|
16
|
+
| [`docs/CLI-MIGRATION.md`](docs/CLI-MIGRATION.md) | Intentional alpha grammar migration; there are no compatibility aliases |
|
|
17
|
+
| [`docs/adr/`](docs/adr/README.md) | Accepted reusable architecture decisions |
|
|
18
|
+
| [`docs/Task Queue 0.1 Function Manifest.md`](docs/Task%20Queue%200.1%20Function%20Manifest.md) | **Canonical 0.1 SQL surface** — migration 0001 derives from this |
|
|
19
|
+
| [`docs/Task Queue Stage 2A Typed Enqueue Specification.md`](docs/Task%20Queue%20Stage%202A%20Typed%20Enqueue%20Specification.md) | Typed enqueue contract |
|
|
20
|
+
| [`docs/Task Queue Stage 2B Worker Runtime Specification.md`](docs/Task%20Queue%20Stage%202B%20Worker%20Runtime%20Specification.md) | Worker runtime behavior |
|
|
21
|
+
| [`docs/Task Queue Stage 3 FastAPI and Authorization Specification.md`](docs/Task%20Queue%20Stage%203%20FastAPI%20and%20Authorization%20Specification.md) | Optional HTTP and authorization integration |
|
|
22
|
+
| [`docs/RELEASE-0.1.0a26.md`](docs/RELEASE-0.1.0a26.md) | 0.1.0a26 production-integration closeout and rollout checklist |
|
|
23
|
+
| [`docs/RELEASE-0.1.0a25.md`](docs/RELEASE-0.1.0a25.md) | 0.1.0a25 complete operator read model and CLI foundation |
|
|
24
|
+
| [`docs/RELEASE-0.1.0a24.md`](docs/RELEASE-0.1.0a24.md) | Historical unreleased a24 candidate record |
|
|
25
|
+
| [`docs/RELEASE-0.1.0a22.md`](docs/RELEASE-0.1.0a22.md) | 0.1.0a22 standalone scheduler release notes and rollout checklist |
|
|
26
|
+
| [`docs/TaskQ Standalone Scheduler Specification.md`](docs/TaskQ%20Standalone%20Scheduler%20Specification.md) | Owner-approved standalone scheduler, target-attestation, manifest, and evidence contract |
|
|
27
|
+
| [`docs/RELEASE-0.1.0a21.md`](docs/RELEASE-0.1.0a21.md) | 0.1.0a21 compatible OutLabs Auth prerelease range and checklist |
|
|
28
|
+
| [`docs/RELEASE-0.1.0a20.md`](docs/RELEASE-0.1.0a20.md) | 0.1.0a20 OutLabs Auth a27 compatibility release and checklist |
|
|
29
|
+
| [`docs/RELEASE-0.1.0a19.md`](docs/RELEASE-0.1.0a19.md) | 0.1.0a19 compatibility fix, hardening changes, and release checklist |
|
|
30
|
+
|
|
31
|
+
## Install
|
|
32
|
+
|
|
33
|
+
Install the exact published prerelease selected by the consumer lockfile:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
pip install outlabs-taskq==0.1.0a26
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Credential handling
|
|
40
|
+
|
|
41
|
+
Keep DSNs and HTTP credentials out of shell history and process arguments. Supply them through the
|
|
42
|
+
process environment using a secret manager, container secret, or service supervisor, then omit the
|
|
43
|
+
DSN from commands. A context stores only the environment-variable name:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
# TASKQ_STAGING_DSN is supplied by the deployment environment.
|
|
47
|
+
taskq --context staging db plan -o json
|
|
48
|
+
taskq --context staging db verify
|
|
49
|
+
|
|
50
|
+
# A context file is optional when the caller supplies target identity explicitly.
|
|
51
|
+
taskq --dsn-env TASKQ_STAGING_DSN \
|
|
52
|
+
--expected-environment staging --actor operator:release-agent \
|
|
53
|
+
db verify -o json
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`taskq db migrate` requires a superuser or a managed database-owner role with
|
|
57
|
+
`CREATEROLE`. On PostgreSQL 16/18, the installer bootstraps `taskq_owner` and
|
|
58
|
+
retains owner membership on that migration role so later owner-only binding
|
|
59
|
+
and upgrades work. This must be a dedicated owner/migration credential: never
|
|
60
|
+
give it to an API, worker, or scheduler. Runtime logins receive only the
|
|
61
|
+
required TaskQ capability roles.
|
|
62
|
+
|
|
63
|
+
Migration `0019` deliberately stops at an unbound target before scheduler
|
|
64
|
+
activation. Inspect and bind the safe fingerprint, then resume migration:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
PLAN=$(taskq --context staging db plan -o json)
|
|
68
|
+
# Review PLAN and extract data.plan_digest.
|
|
69
|
+
taskq --context staging --yes db migrate --plan-digest "$PLAN_DIGEST"
|
|
70
|
+
taskq --context staging target show -o json
|
|
71
|
+
taskq --context staging --yes target bind staging \
|
|
72
|
+
--expected-installation-id "$TASKQ_INSTALLATION_ID" \
|
|
73
|
+
--expected-binding-version 0
|
|
74
|
+
taskq --context staging db plan -o json
|
|
75
|
+
taskq --context staging --yes db migrate --plan-digest "$PLAN_DIGEST"
|
|
76
|
+
taskq --context staging db verify
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Run the framework-neutral scheduler with static target expectations. The
|
|
80
|
+
bounded mode is first-class for platform timers and scale-to-zero databases:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
taskq --context staging scheduler run
|
|
84
|
+
taskq --context staging scheduler once -o json
|
|
85
|
+
taskq --context staging scheduler doctor -o json
|
|
86
|
+
|
|
87
|
+
taskq --context staging schedule manifest plan examples/schedules.minimal.yaml -o json
|
|
88
|
+
taskq --context staging schedule manifest apply examples/schedules.minimal.yaml \
|
|
89
|
+
--plan-digest "$PLAN_DIGEST"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Activating or resuming an interval schedule is `from now`: the first scheduler
|
|
93
|
+
pass initializes its recurrence and the first job becomes due after one full
|
|
94
|
+
interval. Activation does not enqueue an immediate job. Size attended pilot
|
|
95
|
+
windows accordingly, or use an explicit one-shot job when an immediate canary
|
|
96
|
+
is required.
|
|
97
|
+
|
|
98
|
+
The scheduler is a clock, not a task executor. Run one supervised scheduler per
|
|
99
|
+
database/environment, but place workers according to the application's existing
|
|
100
|
+
operating model. A single host-native worker can use a combined registry and
|
|
101
|
+
subscribe to multiple queues; TaskQ does not require one container per worker,
|
|
102
|
+
queue, task, or schedule. Prefer existing local worker hosts unless a particular
|
|
103
|
+
task has a documented cloud availability, latency, or network requirement.
|
|
104
|
+
|
|
105
|
+
Every recurring source manifest must set `catchup: fire_once` or
|
|
106
|
+
`catchup: fire_all` explicitly. Do not rely on the current `skip` default: the
|
|
107
|
+
released SQL contract intentionally accepts only zero occurrences for that
|
|
108
|
+
policy, so continuous polling advances without enqueueing. A corrected
|
|
109
|
+
`skip_missed` semantic requires a new SQL migration and compatibility audit; it
|
|
110
|
+
is not being patched only in Python.
|
|
111
|
+
|
|
112
|
+
Worker services may still use their runtime settings. Operator CLI commands require an explicit
|
|
113
|
+
context or complete connection flags; they never infer a current context or endpoint. HTTP contexts
|
|
114
|
+
store a bearer-token environment-variable name, never a literal token or arbitrary header value.
|
|
115
|
+
OutLabs IAM provisioning reads `TASKQ_AUTH_DSN`.
|
|
116
|
+
|
|
117
|
+
Use `https://` for traffic that leaves a protected private network. The mounted facade intentionally
|
|
118
|
+
publishes its public wire contract at `/taskq/openapi.json`; deployments that do not want that route
|
|
119
|
+
reachable should gate it at the host application or reverse proxy.
|
|
120
|
+
|
|
121
|
+
## Package layout
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
src/taskq/
|
|
125
|
+
sql/ # migrations 0001-0021, runner/verifier, manifest, SQL transport
|
|
126
|
+
scheduler.py # standalone runtime, bounded mode, doctor, YAML plan/apply
|
|
127
|
+
protocol.py # closed command/outcome/error single-source (Tier-0 parity-tested)
|
|
128
|
+
continuations.py # pure policy compiler, negotiation, and derived identities
|
|
129
|
+
registry.py # typed Task[In, Out] registry
|
|
130
|
+
client.py # TaskQ facade: transactional typed enqueue
|
|
131
|
+
transport.py # TaskqTransport protocol
|
|
132
|
+
worker.py # supervisor + fair poll/presence/shutdown service
|
|
133
|
+
settings.py # secret-safe worker environment/CLI configuration
|
|
134
|
+
testing.py # fake client, enqueue assertions, direct work, inline and drain helpers
|
|
135
|
+
cli/ # Click registry, contexts, transports, safety, output, errors
|
|
136
|
+
http/ # optional clients, mounted facade, composable runtime/lifespan
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
## Consumer testing
|
|
140
|
+
|
|
141
|
+
Fast unit tests can replace one facade without starting a worker or database:
|
|
142
|
+
|
|
143
|
+
```python
|
|
144
|
+
from taskq.testing import FakeTaskQClient
|
|
145
|
+
|
|
146
|
+
with tq.replace_client(FakeTaskQClient()) as fake:
|
|
147
|
+
await application_call()
|
|
148
|
+
fake.assert_enqueued("mail.send", where={"payload.recipient": "me@example.test"})
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Inline mode executes registered handlers immediately; bounded drain tests queued behavior without sleeps:
|
|
152
|
+
|
|
153
|
+
```python
|
|
154
|
+
from taskq.testing import drain, inline_mode
|
|
155
|
+
|
|
156
|
+
async with inline_mode(tq) as recorder:
|
|
157
|
+
await application_call()
|
|
158
|
+
assert recorder.settled("mail.send")[0].is_complete
|
|
159
|
+
|
|
160
|
+
report = await drain(tq, queue="mail", max_jobs=100)
|
|
161
|
+
assert report.completed == 1
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
These are consumer-test conveniences, not production modes. The fake intentionally does not model PostgreSQL fencing, privileges, budgets, or transaction isolation. Use a scratch PostgreSQL transaction with `work`, `require_enqueued`, or `drain(..., connection=connection)` when those contracts matter; every helper preserves caller transaction ownership and makes runaway caps fail loudly.
|
|
165
|
+
|
|
166
|
+
## Atomic native follow-ups
|
|
167
|
+
|
|
168
|
+
A handler declares its finite child graph in the registry and returns typed children with its
|
|
169
|
+
successful result. The parent settlement and every child insert commit together; the worker never
|
|
170
|
+
receives a generic producer client.
|
|
171
|
+
|
|
172
|
+
```python
|
|
173
|
+
from taskq import Complete, Followup, FollowupTarget, Task, TaskRegistry
|
|
174
|
+
|
|
175
|
+
child = Task(
|
|
176
|
+
name="listing.enrich",
|
|
177
|
+
queue="enrichment",
|
|
178
|
+
input_model=EnrichInput,
|
|
179
|
+
output_model=EnrichOutput,
|
|
180
|
+
handler=enrich,
|
|
181
|
+
)
|
|
182
|
+
|
|
183
|
+
async def discover(payload: DiscoverInput) -> Complete:
|
|
184
|
+
return Complete(
|
|
185
|
+
result={"accepted": True},
|
|
186
|
+
followups=(
|
|
187
|
+
Followup(
|
|
188
|
+
step="enrich",
|
|
189
|
+
job_type=child.name,
|
|
190
|
+
queue=child.queue,
|
|
191
|
+
payload={"listing_id": payload.listing_id},
|
|
192
|
+
),
|
|
193
|
+
),
|
|
194
|
+
)
|
|
195
|
+
|
|
196
|
+
parent = Task(
|
|
197
|
+
name="listing.discover",
|
|
198
|
+
queue="discovery",
|
|
199
|
+
input_model=DiscoverInput,
|
|
200
|
+
output_model=DiscoverOutput,
|
|
201
|
+
followup_targets=(FollowupTarget(queue=child.queue, job_type=child.name),),
|
|
202
|
+
handler=discover,
|
|
203
|
+
)
|
|
204
|
+
|
|
205
|
+
registry = TaskRegistry((parent, child))
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Worker construction rejects missing or queue-mismatched target declarations. HTTP completion
|
|
209
|
+
authorizes the parent queue before decoding the body, then authorizes every distinct child queue
|
|
210
|
+
before SQL; direct SQL retains the trusted runner-role boundary.
|
|
211
|
+
|
|
212
|
+
## Development gates
|
|
213
|
+
|
|
214
|
+
Protect `main` with pull requests, require branches to be current, and require the CI gates that run on pull requests: `lint`, `dependency-audit`, both `import-isolation` and `unit` Python lanes, `built-artifacts`, both PostgreSQL `sql-contract` lanes, both `fresh-cluster-security` lanes, `races`, `stage3-audit`, `migrations`, and `bench-smoke`. The scheduled/dispatchable `million-row-plans` job keeps structural plans honest without charging every pull request. Do not bypass a failed required check except through the repository's explicit break-glass process.
|
|
215
|
+
|
|
216
|
+
## License
|
|
217
|
+
|
|
218
|
+
MIT — see [LICENSE](LICENSE).
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Migrating to the resource-oriented TaskQ CLI
|
|
2
|
+
|
|
3
|
+
Release 0.1.0a25 intentionally replaces the alpha command grammar. There are
|
|
4
|
+
no compatibility aliases. Pinning exact prereleases prevents accidental
|
|
5
|
+
rollout; update automation and deployment examples before changing the pin.
|
|
6
|
+
|
|
7
|
+
| Before 0.1.0a25 | 0.1.0a25 and later |
|
|
8
|
+
|---|---|
|
|
9
|
+
| `taskq migrate` | `taskq db plan`, then `taskq db migrate --plan-digest … --yes` |
|
|
10
|
+
| `taskq verify` | `taskq db verify` |
|
|
11
|
+
| `taskq worker` | `taskq worker run` |
|
|
12
|
+
| `taskq scheduler` | `taskq scheduler run` |
|
|
13
|
+
| `taskq scheduler --once` | `taskq scheduler once` |
|
|
14
|
+
| `taskq scheduler doctor` | `taskq scheduler doctor` |
|
|
15
|
+
| `taskq schedule plan` | `taskq schedule manifest plan` |
|
|
16
|
+
| `taskq schedule apply` | `taskq schedule manifest apply` |
|
|
17
|
+
| `taskq schedule retire` for a manifest key | `taskq schedule manifest retire` |
|
|
18
|
+
| `taskq auth sync-permissions` | `taskq auth plan`, then `taskq auth apply` |
|
|
19
|
+
| `--json` | `-o json` |
|
|
20
|
+
|
|
21
|
+
Connected commands no longer infer an endpoint from a generic environment
|
|
22
|
+
variable. Add a secret-free context and pass `--context`, or pass complete
|
|
23
|
+
explicit connection flags. Mutating SQL commands also need an actor.
|
|
24
|
+
|
|
25
|
+
Deployment migration order:
|
|
26
|
+
|
|
27
|
+
1. Install and pin the same 0.1.0a25-or-newer artifact in every TaskQ runtime
|
|
28
|
+
consumer while the database remains on contract 0.3.0.
|
|
29
|
+
2. Replace old command invocations and parse the `taskq.cli/v1` envelope.
|
|
30
|
+
3. Add and validate contexts; put only credential environment-variable names
|
|
31
|
+
in the config file.
|
|
32
|
+
4. Run `doctor`, `target show`, and `db verify` read-only checks.
|
|
33
|
+
5. Review `db plan`, capture `data.plan_digest`, and apply with literal
|
|
34
|
+
`--yes` (plus production gates when applicable).
|
|
35
|
+
6. Apply migration 0021 only after all consumers sharing the database run an
|
|
36
|
+
artifact that supports both contract 0.3.0 and 0.3.1.
|
|
37
|
+
|
|
38
|
+
After database activation, rollback means restoring the previously pinned
|
|
39
|
+
artifact only if it supports the active contract. Migration 0021 is additive
|
|
40
|
+
and forward-only; database rollback requires restore or a forward fix.
|