schedex 0.1.0__tar.gz

schedex-0.1.0/LICENSE

@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <https://unlicense.org>

schedex-0.1.0/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: schedex
+Version: 0.1.0
+Summary: Distributed persistent extendable job scheduling library
+License-Expression: Unlicense
+License-File: LICENSE
+Keywords: schedex,async,scheduler,persistent,distributed
+Author: Dmitry Pershin
+Author-email: dapper1291@gmail.com
+Requires-Python: >=3.12,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Utilities
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Typing :: Typed
+Provides-Extra: asyncmy
+Provides-Extra: cron
+Provides-Extra: dishka
+Provides-Extra: opentelemetry
+Provides-Extra: psycopg
+Provides-Extra: sqlalchemy
+Requires-Dist: asyncmy (>=0.2.14) ; extra == "asyncmy"
+Requires-Dist: crontools (>=0.1.6) ; extra == "cron"
+Requires-Dist: dishka (>=1.10.1) ; extra == "dishka"
+Requires-Dist: msgspec (>=0.21.1)
+Requires-Dist: opentelemetry-api (>=1.44.0) ; extra == "opentelemetry"
+Requires-Dist: psycopg[pool] (>=3.3.4) ; extra == "psycopg"
+Requires-Dist: sqlalchemy[asyncio] (>=2.0.52) ; extra == "sqlalchemy"
+Project-URL: Documentation, https://github.com/dapper91/schedex
+Project-URL: Homepage, https://github.com/dapper91/schedex
+Project-URL: Repository, https://github.com/dapper91/schedex
+Description-Content-Type: text/markdown
+
+# schedex
+
+Distributed persistent extendable job scheduling library
+
+## Installation
+
+```commandline
+pip3 install schedex
+```
+
+**Optional dependencies**:
+
+- **cron**: crontab based schedules
+- **sqlalchemy**: [sqlalchemy](https://www.sqlalchemy.org/) backed scheduler
+- **psycopg**: [psycopg](https://www.psycopg.org/psycopg3/docs/) backed scheduler and event broker (through LISTEN/NOTIFY mechanism)
+- **asyncmy**: mysql([asyncmy](https://github.com/long2ice/asyncmy)) backed scheduler
+- **pymongo**: mongodb([pymongo](https://pymongo.readthedocs.io/en/stable/index.html)) backed scheduler
+- **opentelemetry**: telemetry compatible with [opentelemetry](https://opentelemetry.io/) standard
+- **dishka**: dependency injection with [dishka](https://dishka.readthedocs.io/) library
+
+## Overview
+
+`Schedex` is a modern strictly-typed distributed persistent backend agnostic job scheduling library.
+
+### Features
+
+- **Database agnostic** nature allows to use it with any relation database (PostgreSQL, MySql, MsSql, MariaDB, Percona etc.)
+or NoSql one (MongoDB, Redis, etc.) which let you pick up the best solution suitable for your requirements.
+
+- The library is aimed to real-world projects so that it supports essential features required in production like
+**transactional updates** - may be necessary if your application logic requires scheduling multiple jobs or 
+cancel one job and schedule another or modify your domain data and schedule a job atomically.
+
+- For data serialization `schedex` uses [msgspec](https://msgspec.dev/) library which make it **serialization format agnostic** (JSON, MsgPack etc.) 
+
+- **Strongly typed** codebase helps typecheckers like mypy or pyright to validate your code for type correctness 
+and fix more bugs on development stage.
+
+- The library supports **locking mechanism customization** so that you can choose the one that is more suitable 
+in your environment. It supports select-for-update locks (with relational databases), leasing locks; but you can
+implement your own one if it is necessary.
+
+### Why yet another scheduler?
+
+There are some libraries that occupy the same niche. But they have some disadvantages 
+which motivated to write `schedex` instead.
+
+#### [procrastinate](https://procrastinate.readthedocs.io)
+    
+Mature and well tested library with great documentation and support. But it is tied to PostgreSQL which 
+doesn't allow to use it with different databases. No transaction support make it useless when the business 
+logic requires strong consistency.
+
+#### [apscheduler](https://apscheduler.readthedocs.io)
+
+Mature and well tested database agnostic library, but has some disadvantages: it is not strictly typed which
+leads to many bugs in runtime; only one locking mechanism supported (leasing locks), transactional modifications
+are not supported which makes it problematic to use the library in strong consistency required applications.
+
+## Quickstart
+
+```python
+import datetime as dt
+from typing import Any, MutableMapping
+
+import psycopg_pool as ppg_pool
+import sqlalchemy.ext.asyncio as aiosa
+
+import schedex as sx
+import schedex.contrib.psycopg as sxpg
+import schedex.contrib.sqlalchemy as sxsa
+
+AppState = MutableMapping[str, Any]
+TaskLock = sxsa.SqlAlchemySfuTaskLock
+Context = sx.Context[AppState, TaskLock]
+
+tasks = sx.TaskRegistry[AppState, TaskLock]()
+
+
+@tasks.task(name="print")
+async def print_message(context: Context, message: str, /, *, end: str = "\n") -> None:
+    print(message, end=end)
+
+
+async def start_scheduler() -> None:
+    db_url = "postgresql+psycopg://user:password@localhost:5432/myapp"
+    db_engine = aiosa.create_async_engine(db_url)
+
+    async with db_engine.begin() as conn:
+        await conn.run_sync(sxsa.BaseModel.metadata.create_all)
+
+    async with ppg_pool.AsyncConnectionPool(db_url) as pg_pool:
+        event_sender = sxpg.PsycopgEventSender(pg_pool)
+
+        async with sxsa.SqlAlchemyTransactionalScheduler(
+            aiosa.async_sessionmaker(db_engine),
+            schedule_type=sx.PeriodicSchedule,
+            event_sender=event_sender,
+            graceful_timeout=0.1,
+        ) as scheduler:
+            await scheduler.add_job(
+                sx.Job(
+                    sx.PeriodicSchedule.after(dt.timedelta(seconds=0)),
+                    print_message.with_params("let's get started"),
+                )
+            )
+
+            async with scheduler.transactional() as scheduler_tx:
+                await scheduler_tx.add_job(
+                    sx.Job(
+                        sx.PeriodicSchedule.periodically(
+                            start_at=dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=0),
+                            delay=dt.timedelta(seconds=2),
+                        ).with_max_count(5),
+                        print_message.with_params("hello", end=" "),
+                    )
+                )
+                await scheduler_tx.add_job(
+                    sx.Job(
+                        sx.PeriodicSchedule.periodically(
+                            start_at=dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=1),
+                            delay=dt.timedelta(seconds=2),
+                        ).with_max_count(5),
+                        print_message.with_params("world!!!"),
+                    )
+                )
+
+
+async def start_executor() -> None:
+    db_url = "postgresql+psycopg://user:password@localhost:5432/myapp"
+    db_engine = aiosa.create_async_engine(db_url)
+    async with db_engine.begin() as conn:
+        await conn.run_sync(sxsa.BaseModel.metadata.create_all)
+
+    app_state: AppState = {}
+
+    async with ppg_pool.AsyncConnectionPool(db_url) as pg_pool:
+        executor = sx.WorkerPoolExecutor(
+            max_workers=3,
+            state=app_state,
+            polling_interval=dt.timedelta(seconds=15),
+            schedule_type=sx.PeriodicSchedule,
+            task_registry=tasks,
+            lock_manager=sxsa.SqlAlchemySfuLockManager(db_engine, sxpg.PsycopgEventSender(pg_pool)),
+            event_receiver=sxpg.PsycopgEventReceiver(pg_pool),
+        )
+
+        await executor.run()
+
+```
+

schedex-0.1.0/README.md

@@ -0,0 +1,150 @@
+# schedex
+
+Distributed persistent extendable job scheduling library
+
+## Installation
+
+```commandline
+pip3 install schedex
+```
+
+**Optional dependencies**:
+
+- **cron**: crontab based schedules
+- **sqlalchemy**: [sqlalchemy](https://www.sqlalchemy.org/) backed scheduler
+- **psycopg**: [psycopg](https://www.psycopg.org/psycopg3/docs/) backed scheduler and event broker (through LISTEN/NOTIFY mechanism)
+- **asyncmy**: mysql([asyncmy](https://github.com/long2ice/asyncmy)) backed scheduler
+- **pymongo**: mongodb([pymongo](https://pymongo.readthedocs.io/en/stable/index.html)) backed scheduler
+- **opentelemetry**: telemetry compatible with [opentelemetry](https://opentelemetry.io/) standard
+- **dishka**: dependency injection with [dishka](https://dishka.readthedocs.io/) library
+
+## Overview
+
+`Schedex` is a modern strictly-typed distributed persistent backend agnostic job scheduling library.
+
+### Features
+
+- **Database agnostic** nature allows to use it with any relation database (PostgreSQL, MySql, MsSql, MariaDB, Percona etc.)
+or NoSql one (MongoDB, Redis, etc.) which let you pick up the best solution suitable for your requirements.
+
+- The library is aimed to real-world projects so that it supports essential features required in production like
+**transactional updates** - may be necessary if your application logic requires scheduling multiple jobs or 
+cancel one job and schedule another or modify your domain data and schedule a job atomically.
+
+- For data serialization `schedex` uses [msgspec](https://msgspec.dev/) library which make it **serialization format agnostic** (JSON, MsgPack etc.) 
+
+- **Strongly typed** codebase helps typecheckers like mypy or pyright to validate your code for type correctness 
+and fix more bugs on development stage.
+
+- The library supports **locking mechanism customization** so that you can choose the one that is more suitable 
+in your environment. It supports select-for-update locks (with relational databases), leasing locks; but you can
+implement your own one if it is necessary.
+
+### Why yet another scheduler?
+
+There are some libraries that occupy the same niche. But they have some disadvantages 
+which motivated to write `schedex` instead.
+
+#### [procrastinate](https://procrastinate.readthedocs.io)
+    
+Mature and well tested library with great documentation and support. But it is tied to PostgreSQL which 
+doesn't allow to use it with different databases. No transaction support make it useless when the business 
+logic requires strong consistency.
+
+#### [apscheduler](https://apscheduler.readthedocs.io)
+
+Mature and well tested database agnostic library, but has some disadvantages: it is not strictly typed which
+leads to many bugs in runtime; only one locking mechanism supported (leasing locks), transactional modifications
+are not supported which makes it problematic to use the library in strong consistency required applications.
+
+## Quickstart
+
+```python
+import datetime as dt
+from typing import Any, MutableMapping
+
+import psycopg_pool as ppg_pool
+import sqlalchemy.ext.asyncio as aiosa
+
+import schedex as sx
+import schedex.contrib.psycopg as sxpg
+import schedex.contrib.sqlalchemy as sxsa
+
+AppState = MutableMapping[str, Any]
+TaskLock = sxsa.SqlAlchemySfuTaskLock
+Context = sx.Context[AppState, TaskLock]
+
+tasks = sx.TaskRegistry[AppState, TaskLock]()
+
+
+@tasks.task(name="print")
+async def print_message(context: Context, message: str, /, *, end: str = "\n") -> None:
+    print(message, end=end)
+
+
+async def start_scheduler() -> None:
+    db_url = "postgresql+psycopg://user:password@localhost:5432/myapp"
+    db_engine = aiosa.create_async_engine(db_url)
+
+    async with db_engine.begin() as conn:
+        await conn.run_sync(sxsa.BaseModel.metadata.create_all)
+
+    async with ppg_pool.AsyncConnectionPool(db_url) as pg_pool:
+        event_sender = sxpg.PsycopgEventSender(pg_pool)
+
+        async with sxsa.SqlAlchemyTransactionalScheduler(
+            aiosa.async_sessionmaker(db_engine),
+            schedule_type=sx.PeriodicSchedule,
+            event_sender=event_sender,
+            graceful_timeout=0.1,
+        ) as scheduler:
+            await scheduler.add_job(
+                sx.Job(
+                    sx.PeriodicSchedule.after(dt.timedelta(seconds=0)),
+                    print_message.with_params("let's get started"),
+                )
+            )
+
+            async with scheduler.transactional() as scheduler_tx:
+                await scheduler_tx.add_job(
+                    sx.Job(
+                        sx.PeriodicSchedule.periodically(
+                            start_at=dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=0),
+                            delay=dt.timedelta(seconds=2),
+                        ).with_max_count(5),
+                        print_message.with_params("hello", end=" "),
+                    )
+                )
+                await scheduler_tx.add_job(
+                    sx.Job(
+                        sx.PeriodicSchedule.periodically(
+                            start_at=dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=1),
+                            delay=dt.timedelta(seconds=2),
+                        ).with_max_count(5),
+                        print_message.with_params("world!!!"),
+                    )
+                )
+
+
+async def start_executor() -> None:
+    db_url = "postgresql+psycopg://user:password@localhost:5432/myapp"
+    db_engine = aiosa.create_async_engine(db_url)
+    async with db_engine.begin() as conn:
+        await conn.run_sync(sxsa.BaseModel.metadata.create_all)
+
+    app_state: AppState = {}
+
+    async with ppg_pool.AsyncConnectionPool(db_url) as pg_pool:
+        executor = sx.WorkerPoolExecutor(
+            max_workers=3,
+            state=app_state,
+            polling_interval=dt.timedelta(seconds=15),
+            schedule_type=sx.PeriodicSchedule,
+            task_registry=tasks,
+            lock_manager=sxsa.SqlAlchemySfuLockManager(db_engine, sxpg.PsycopgEventSender(pg_pool)),
+            event_receiver=sxpg.PsycopgEventReceiver(pg_pool),
+        )
+
+        await executor.run()
+
+```

schedex-0.1.0/pyproject.toml

@@ -0,0 +1,69 @@
+[project]
+name = "schedex"
+description = "Distributed persistent extendable job scheduling library"
+version = "0.1.0"
+readme = "README.md"
+license = "Unlicense"
+authors = [
+    {name = "Dmitry Pershin", email = "dapper1291@gmail.com"}
+]
+keywords = ["schedex", "async", "scheduler", "persistent", "distributed"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Natural Language :: English",
+    "Operating System :: OS Independent",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: Utilities",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Typing :: Typed",
+]
+
+requires-python = ">=3.12,<4.0"
+dependencies = [
+    "msgspec>=0.21.1",
+]
+
+[project.urls]
+Homepage = "https://github.com/dapper91/schedex"
+Repository = "https://github.com/dapper91/schedex"
+Documentation = "https://github.com/dapper91/schedex"
+
+
+[project.optional-dependencies]
+cron = [
+    "crontools>=0.1.6",
+]
+sqlalchemy = [
+    "sqlalchemy[asyncio]>=2.0.52",
+]
+psycopg = [
+    "psycopg[pool]>=3.3.4",
+]
+asyncmy = [
+    "asyncmy>=0.2.14",
+]
+dishka = [
+    "dishka>=1.10.1",
+]
+opentelemetry = [
+    "opentelemetry-api>=1.44.0",
+]
+
+[dependency-groups]
+dev = [
+    "mypy>=2.3.0",
+    "polyfactory>=3.3.0",
+    "pytest-asyncio>=1.4.0",
+    "pytest>=9.1.1",
+    "sqlalchemy-utils>=0.42.1",
+    "time-machine>=3.5.0",
+    "pytest-timeout>=2.4.0",
+    "pytest-lazy-fixtures>=1.4.1",
+]
+linters = [
+    "pre-commit>=4.6.2",
+]

schedex-0.1.0/schedex/__init__.py

@@ -0,0 +1,35 @@
+from . import mem
+from .context import Context, TaskInfo
+from .eventbus import BufferedEventSender, Event, EventKind, EventReceiver, EventSender
+from .executor import (
+    ExecutorMiddleware,
+    ExecutorMiddlewareWrappedFunc,
+    TaskFetcher,
+    Worker,
+    WorkerPool,
+    WorkerPoolExecutor,
+)
+from .lock import EventManagerMixin, JobLock, Lock, LockManager, TaskLock
+from .schedule import CombinedSchedule, PeriodicSchedule, Schedule
+from .scheduler import (
+    Job,
+    OwnedSchedulerTransaction,
+    Scheduler,
+    SchedulerMiddleware,
+    SchedulerMiddlewareWrappedFunc,
+    SchedulerTransaction,
+    TransactionalScheduler,
+)
+from .serializer import Serializable
+from .storage import (
+    JobManager,
+    JobStatus,
+    OwnedTransaction,
+    StoredJob,
+    StoredTask,
+    TaskManager,
+    TaskStatus,
+    Transaction,
+    TransactionManager,
+)
+from .task import BoundTask, Task, TaskRegistry

schedex-0.1.0/schedex/context.py

@@ -0,0 +1,40 @@
+import dataclasses as dc
+import datetime as dt
+
+from .metadata import Metadata
+
+
+@dc.dataclass(frozen=True)
+class TaskInfo:
+    """
+    Executing task information.
+
+    :param id: task identifier
+    :param job_id: job identifier which spawned the task
+    :param sequence_number: task sequence number (unique within a job)
+    :param created_at: task creation time
+    :param attempts: number of task execution attempts
+    :param meta: task metadata
+    """
+
+    id: str
+    job_id: str
+    sequence_number: int
+    created_at: dt.datetime
+    attempts: int
+    meta: Metadata
+
+
+@dc.dataclass(frozen=True)
+class Context[StT, LkT]:
+    """
+    Task context.
+
+    :param state: application state
+    :param task: executing task
+    :param lock: executing task lock
+    """
+
+    state: StT
+    task: TaskInfo
+    lock: LkT

schedex-0.1.0/schedex/contrib/dishka.py

@@ -0,0 +1,53 @@
+from collections import ChainMap
+from typing import Any, Callable, Concatenate, MutableMapping, Never
+
+import dishka
+from dishka.integrations.base import wrap_injection
+
+from schedex import Context, ExecutorMiddlewareWrappedFunc, Task
+
+CONTAINER_KEY = "dishka_container"
+
+AppState = MutableMapping[str, Any]
+
+
+class InjectedType:
+    def __getattribute__(self, item: Any) -> Never:
+        raise AssertionError("argument is not injected")
+
+
+Injected: Any = InjectedType()
+
+
+def inject[StT: AppState, **P, R](
+    func: Callable[Concatenate[Context[StT, Any], P], R],
+) -> Callable[Concatenate[Context[StT, Any], P], R]:
+    """
+    Function decorator injecting dependencies into the function arguments.
+    """
+
+    return wrap_injection(
+        func=func,
+        is_async=True,
+        container_getter=lambda args, kwargs: args[0].state[CONTAINER_KEY],
+    )
+
+
+async def request_container_middleware(
+    context: Context[AppState, Any],
+    task: Task[Context[AppState, Any], Any, Any],
+    /,
+    *,
+    wrapped: ExecutorMiddlewareWrappedFunc[Context[AppState, Any]],
+) -> None:
+    """
+    A middleware that initialize request-scoped di container.
+    Works only with MutableMapping application state.
+    """
+
+    container: dishka.AsyncContainer = context.state[CONTAINER_KEY]
+
+    async with container(scope=dishka.Scope.REQUEST) as request_container:
+        request_context = ChainMap({CONTAINER_KEY: request_container}, context.state)
+        context = Context(state=request_context, task=context.task, lock=context.lock)
+        return await wrapped(context, task)