fluxera 0.0.1__tar.gz

fluxera-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 JaeWang Lee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fluxera-0.0.1/PKG-INFO

@@ -0,0 +1,199 @@
+Metadata-Version: 2.4
+Name: fluxera
+Version: 0.0.1
+Summary: Async-native message processing inspired by Dramatiq.
+Project-URL: Homepage, https://github.com/JaeWangL/fluxera
+Project-URL: Repository, https://github.com/JaeWangL/fluxera
+Keywords: asyncio,dramatiq,message-queue,redis,worker
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: AsyncIO
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Distributed Computing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: redis<8,>=5
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Dynamic: license-file
+
+# Fluxera
+
+Fluxera is an async-native Python task runtime inspired by Dramatiq.
+
+It is built for workloads where a worker should keep a lot of I/O in flight without buying concurrency through large worker-thread pools, while still handling synchronous and CPU-bound work through dedicated execution lanes.
+
+## Why Fluxera
+
+- `async def` actors run as real `asyncio` tasks on the worker event loop.
+- `def` actors still work through a bounded thread lane.
+- CPU-heavy actors can be isolated in a separate process lane.
+- Redis Streams is supported as an at-least-once transport with lease renewal, stale reclaim, deduplication, and idempotency primitives.
+- Rolling deploys can hand off unstarted backlog between old and new worker revisions without rotating namespaces.
+
+## Status
+
+`0.0.1` is the first public alpha.
+
+The runtime, Redis transport v2, revision management, benchmark harnesses, and release packaging are in place, but APIs may still change as the project hardens.
+
+## Install
+
+```bash
+pip install fluxera
+```
+
+For local Redis development:
+
+```bash
+docker compose up -d
+```
+
+## Quick Start
+
+```python
+import asyncio
+
+import fluxera
+
+
+broker = fluxera.RedisBroker(
+    "redis://127.0.0.1:6379/15",
+    namespace="hello-fluxera",
+)
+
+
+@fluxera.actor(broker=broker, queue_name="default")
+async def fetch_user(user_id: str) -> None:
+    await asyncio.sleep(0.1)
+    print("fetched", user_id)
+
+
+async def main() -> None:
+    async with fluxera.Worker(
+        broker,
+        concurrency=128,
+        thread_concurrency=16,
+        process_concurrency=4,
+    ):
+        await fetch_user.send("user-123")
+        await broker.join(fetch_user.queue_name)
+
+
+asyncio.run(main())
+```
+
+## Execution Model
+
+Fluxera has three execution lanes:
+
+- `async`: default for `async def` actors
+- `thread`: default for regular `def` actors
+- `process`: opt-in for CPU-heavy actors
+
+The process lane defaults to `spawn` for safe multithreaded startup. You can still override it through `Worker(process_start_method=...)` or `FLUXERA_PROCESS_START_METHOD` when needed.
+
+Example CPU actor:
+
+```python
+import fluxera
+
+
+broker = fluxera.RedisBroker("redis://127.0.0.1:6379/15", namespace="cpu-example")
+
+
+def score_document(text: str) -> int:
+    return sum(ord(ch) for ch in text)
+
+
+score_document_actor = fluxera.actor(
+    broker=broker,
+    actor_name="score_document",
+    queue_name="cpu",
+    execution="process",
+)(score_document)
+```
+
+## Serving Revision Admin
+
+Fluxera keeps `namespace` as the broker identity boundary and uses `worker_revision` and `serving_revision` for rollout control.
+
+Read the current serving revision:
+
+```bash
+fluxera revision get \
+  --redis-url redis://127.0.0.1:6379/15 \
+  --namespace hello-fluxera \
+  --queue default
+```
+
+Promote a new serving revision with a CAS guard:
+
+```bash
+fluxera revision promote \
+  --redis-url redis://127.0.0.1:6379/15 \
+  --namespace hello-fluxera \
+  --queue default \
+  --revision 20260329153000 \
+  --expected-revision 20260329140000
+```
+
+Use `--format json` when the command is called by deployment automation.
+
+## Delivery Semantics
+
+- Transport delivery is at-least-once.
+- Deduplication is an enqueue-time admission policy, not exactly-once execution.
+- Effectively-once side effects require idempotency keys or application-level dedupe.
+- Redis workers renew leases for long-running tasks and reclaim stale pending deliveries.
+
+## Benchmark Snapshot
+
+Latest local measurements were taken on `2026-03-29` on `macOS 26.3.1`, `Python 3.12.10`, `Apple M5 Pro (15 cores)`.
+
+Benchmark label legend:
+
+- `c=`: Fluxera worker concurrency setting used by the benchmark runner
+- `t=`: Dramatiq `worker_threads`
+
+Headline results against the current local Dramatiq checkout:
+
+| Scenario | Fluxera | Dramatiq | Takeaway |
+| --- | --- | --- | --- |
+| production-shaped async fanout | `0.258s` | `0.385s (t=8)` / `0.319s (t=32)` | Fluxera is faster with `2` threads instead of `12` or `36` |
+| single-worker CPU-bound | `1.270s` | `3.893s (t=8)` / `3.704s (t=32)` | process lane still gives Fluxera a large single-worker win |
+| mixed long I/O + short work | `short_drain=0.040s` | `6.038s (t=8)` / `0.098s (t=32)` | long I/O does not starve short work |
+| Redis mixed long/short | `wall=1.527s`, `short_drain=0.081s` | `3.176s`, `1.673s (t=8)` / `1.713s`, `0.089s (t=32)` | transport advantage remains on real Redis |
+
+See [BENCHMARK.md](docs/BENCHMARK.md) for the full methodology and numbers.
+
+One nuance matters: with the safer default `spawn` process policy, cluster-scale CPU throughput is no longer universally faster than Dramatiq. Fluxera's strongest advantage is still async-heavy and mixed I/O workloads.
+
+## Verification
+
+The current release candidate was checked with:
+
+- `python3 -m unittest discover -s tests -v`
+- `python3 benchmarks/production_compare.py --profile smoke`
+- `python3 benchmarks/redis_transport_compare.py --repeat 3 --long-io-secs 1.5`
+- `/tmp/fluxera-release-venv/bin/python -m build --sdist --wheel`
+- `/tmp/fluxera-release-venv/bin/python -m twine check dist/*`
+
+## Documentation
+
+- [Getting Started](docs/GETTING_STARTED.md)
+- [Benchmark Results](docs/BENCHMARK.md)
+- [Revision Management](docs/REVISION_MANAGEMENT.md)
+- [System Design](docs/SYSTEM_DESIGN.md)
+- [Deduplication and Idempotency](docs/DEDUP_IDEMPOTENCY.md)
+- [Redis Lua Contract](docs/REDIS_LUA_CONTRACT.md)
+
+## Current Limits
+
+- public APIs may still change during the alpha period
+- result backends are not implemented yet
+- message registry garbage collection is still intentionally simple

fluxera-0.0.1/README.md

@@ -0,0 +1,176 @@
+# Fluxera
+
+Fluxera is an async-native Python task runtime inspired by Dramatiq.
+
+It is built for workloads where a worker should keep a lot of I/O in flight without buying concurrency through large worker-thread pools, while still handling synchronous and CPU-bound work through dedicated execution lanes.
+
+## Why Fluxera
+
+- `async def` actors run as real `asyncio` tasks on the worker event loop.
+- `def` actors still work through a bounded thread lane.
+- CPU-heavy actors can be isolated in a separate process lane.
+- Redis Streams is supported as an at-least-once transport with lease renewal, stale reclaim, deduplication, and idempotency primitives.
+- Rolling deploys can hand off unstarted backlog between old and new worker revisions without rotating namespaces.
+
+## Status
+
+`0.0.1` is the first public alpha.
+
+The runtime, Redis transport v2, revision management, benchmark harnesses, and release packaging are in place, but APIs may still change as the project hardens.
+
+## Install
+
+```bash
+pip install fluxera
+```
+
+For local Redis development:
+
+```bash
+docker compose up -d
+```
+
+## Quick Start
+
+```python
+import asyncio
+
+import fluxera
+
+
+broker = fluxera.RedisBroker(
+    "redis://127.0.0.1:6379/15",
+    namespace="hello-fluxera",
+)
+
+
+@fluxera.actor(broker=broker, queue_name="default")
+async def fetch_user(user_id: str) -> None:
+    await asyncio.sleep(0.1)
+    print("fetched", user_id)
+
+
+async def main() -> None:
+    async with fluxera.Worker(
+        broker,
+        concurrency=128,
+        thread_concurrency=16,
+        process_concurrency=4,
+    ):
+        await fetch_user.send("user-123")
+        await broker.join(fetch_user.queue_name)
+
+
+asyncio.run(main())
+```
+
+## Execution Model
+
+Fluxera has three execution lanes:
+
+- `async`: default for `async def` actors
+- `thread`: default for regular `def` actors
+- `process`: opt-in for CPU-heavy actors
+
+The process lane defaults to `spawn` for safe multithreaded startup. You can still override it through `Worker(process_start_method=...)` or `FLUXERA_PROCESS_START_METHOD` when needed.
+
+Example CPU actor:
+
+```python
+import fluxera
+
+
+broker = fluxera.RedisBroker("redis://127.0.0.1:6379/15", namespace="cpu-example")
+
+
+def score_document(text: str) -> int:
+    return sum(ord(ch) for ch in text)
+
+
+score_document_actor = fluxera.actor(
+    broker=broker,
+    actor_name="score_document",
+    queue_name="cpu",
+    execution="process",
+)(score_document)
+```
+
+## Serving Revision Admin
+
+Fluxera keeps `namespace` as the broker identity boundary and uses `worker_revision` and `serving_revision` for rollout control.
+
+Read the current serving revision:
+
+```bash
+fluxera revision get \
+  --redis-url redis://127.0.0.1:6379/15 \
+  --namespace hello-fluxera \
+  --queue default
+```
+
+Promote a new serving revision with a CAS guard:
+
+```bash
+fluxera revision promote \
+  --redis-url redis://127.0.0.1:6379/15 \
+  --namespace hello-fluxera \
+  --queue default \
+  --revision 20260329153000 \
+  --expected-revision 20260329140000
+```
+
+Use `--format json` when the command is called by deployment automation.
+
+## Delivery Semantics
+
+- Transport delivery is at-least-once.
+- Deduplication is an enqueue-time admission policy, not exactly-once execution.
+- Effectively-once side effects require idempotency keys or application-level dedupe.
+- Redis workers renew leases for long-running tasks and reclaim stale pending deliveries.
+
+## Benchmark Snapshot
+
+Latest local measurements were taken on `2026-03-29` on `macOS 26.3.1`, `Python 3.12.10`, `Apple M5 Pro (15 cores)`.
+
+Benchmark label legend:
+
+- `c=`: Fluxera worker concurrency setting used by the benchmark runner
+- `t=`: Dramatiq `worker_threads`
+
+Headline results against the current local Dramatiq checkout:
+
+| Scenario | Fluxera | Dramatiq | Takeaway |
+| --- | --- | --- | --- |
+| production-shaped async fanout | `0.258s` | `0.385s (t=8)` / `0.319s (t=32)` | Fluxera is faster with `2` threads instead of `12` or `36` |
+| single-worker CPU-bound | `1.270s` | `3.893s (t=8)` / `3.704s (t=32)` | process lane still gives Fluxera a large single-worker win |
+| mixed long I/O + short work | `short_drain=0.040s` | `6.038s (t=8)` / `0.098s (t=32)` | long I/O does not starve short work |
+| Redis mixed long/short | `wall=1.527s`, `short_drain=0.081s` | `3.176s`, `1.673s (t=8)` / `1.713s`, `0.089s (t=32)` | transport advantage remains on real Redis |
+
+See [BENCHMARK.md](docs/BENCHMARK.md) for the full methodology and numbers.
+
+One nuance matters: with the safer default `spawn` process policy, cluster-scale CPU throughput is no longer universally faster than Dramatiq. Fluxera's strongest advantage is still async-heavy and mixed I/O workloads.
+
+## Verification
+
+The current release candidate was checked with:
+
+- `python3 -m unittest discover -s tests -v`
+- `python3 benchmarks/production_compare.py --profile smoke`
+- `python3 benchmarks/redis_transport_compare.py --repeat 3 --long-io-secs 1.5`
+- `/tmp/fluxera-release-venv/bin/python -m build --sdist --wheel`
+- `/tmp/fluxera-release-venv/bin/python -m twine check dist/*`
+
+## Documentation
+
+- [Getting Started](docs/GETTING_STARTED.md)
+- [Benchmark Results](docs/BENCHMARK.md)
+- [Revision Management](docs/REVISION_MANAGEMENT.md)
+- [System Design](docs/SYSTEM_DESIGN.md)
+- [Deduplication and Idempotency](docs/DEDUP_IDEMPOTENCY.md)
+- [Redis Lua Contract](docs/REDIS_LUA_CONTRACT.md)
+
+## Current Limits
+
+- public APIs may still change during the alpha period
+- result backends are not implemented yet
+- message registry garbage collection is still intentionally simple

fluxera-0.0.1/fluxera/__init__.py

@@ -0,0 +1,38 @@
+from __future__ import annotations
+
+from .admin import (
+    ServingRevisionPromotion,
+    ServingRevisionStatus,
+    ensure_serving_revision,
+    get_serving_revision,
+    promote_serving_revision,
+)
+from .actor import Actor, actor
+from .broker import Broker, Consumer, Delivery, get_broker, set_broker
+from .encoder import JSONMessageEncoder, PickleMessageEncoder
+from .brokers.redis import RedisBroker
+from .brokers.stub import StubBroker
+from .message import Message
+from .runtime.worker import TaskRecord, Worker
+
+__all__ = [
+    "Actor",
+    "Broker",
+    "Consumer",
+    "Delivery",
+    "JSONMessageEncoder",
+    "Message",
+    "PickleMessageEncoder",
+    "RedisBroker",
+    "ServingRevisionPromotion",
+    "ServingRevisionStatus",
+    "StubBroker",
+    "TaskRecord",
+    "Worker",
+    "actor",
+    "ensure_serving_revision",
+    "get_broker",
+    "get_serving_revision",
+    "promote_serving_revision",
+    "set_broker",
+]

fluxera-0.0.1/fluxera/__main__.py

@@ -0,0 +1,7 @@
+from __future__ import annotations
+
+from .cli import main
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

fluxera-0.0.1/fluxera/actor.py

@@ -0,0 +1,176 @@
+from __future__ import annotations
+
+import asyncio
+from inspect import iscoroutinefunction
+from typing import TYPE_CHECKING, Any, Awaitable, Callable, Optional, Protocol, TypeVar, Union, overload
+
+from .broker import Broker, get_broker
+from .message import Message
+
+if TYPE_CHECKING:
+    from .runtime.worker import Worker
+
+
+P = TypeVar("P")
+R = TypeVar("R")
+ExecutionMode = str
+
+
+class Actor:
+    """Thin wrapper around callables that stores queueing metadata."""
+
+    def __init__(
+        self,
+        fn: Callable[..., Any],
+        *,
+        broker: Broker,
+        actor_name: str,
+        queue_name: str,
+        execution: Optional[ExecutionMode] = None,
+        options: Optional[dict[str, Any]] = None,
+    ) -> None:
+        if actor_name in broker.actors:
+            raise ValueError(f"An actor named {actor_name!r} is already registered.")
+
+        inferred_execution = "async" if iscoroutinefunction(fn) else "thread"
+        self.fn = fn
+        self.broker = broker
+        self.actor_name = actor_name
+        self.queue_name = queue_name
+        self.execution = execution or inferred_execution
+        self.options = options or {}
+
+        if self.execution == "async" and not iscoroutinefunction(fn):
+            raise TypeError("Execution mode 'async' requires an async function.")
+
+        self.broker.declare_actor(self)
+
+    def message(self, *args: Any, **kwargs: Any) -> Message:
+        return self.message_with_options(args=args, kwargs=kwargs)
+
+    def message_with_options(
+        self,
+        *,
+        args: tuple[Any, ...] = (),
+        kwargs: Optional[dict[str, Any]] = None,
+        **options: Any,
+    ) -> Message:
+        return Message(
+            queue_name=self.queue_name,
+            actor_name=self.actor_name,
+            args=args,
+            kwargs=(kwargs or {}).copy(),
+            options={**self.options, **options},
+        )
+
+    async def send(self, *args: Any, **kwargs: Any) -> Message:
+        return await self.send_with_options(args=args, kwargs=kwargs)
+
+    async def send_with_options(
+        self,
+        *,
+        args: tuple[Any, ...] = (),
+        kwargs: Optional[dict[str, Any]] = None,
+        delay: Optional[float] = None,
+        **options: Any,
+    ) -> Message:
+        return await self.broker.send(
+            self.message_with_options(args=args, kwargs=kwargs, **options),
+            delay=delay,
+        )
+
+    def send_sync(self, *args: Any, **kwargs: Any) -> Message:
+        try:
+            asyncio.get_running_loop()
+        except RuntimeError:
+            return asyncio.run(self.send(*args, **kwargs))
+
+        raise RuntimeError("send_sync cannot be called while an event loop is already running.")
+
+    def send_with_options_sync(
+        self,
+        *,
+        args: tuple[Any, ...] = (),
+        kwargs: Optional[dict[str, Any]] = None,
+        delay: Optional[float] = None,
+        **options: Any,
+    ) -> Message:
+        try:
+            asyncio.get_running_loop()
+        except RuntimeError:
+            return asyncio.run(self.send_with_options(args=args, kwargs=kwargs, delay=delay, **options))
+
+        raise RuntimeError("send_with_options_sync cannot be called while an event loop is already running.")
+
+    async def run(self, *args: Any, worker: Optional["Worker"] = None, **kwargs: Any) -> Any:
+        if self.execution == "async":
+            async_fn = self.fn
+            return await async_fn(*args, **kwargs)
+
+        if self.execution == "thread":
+            if worker is not None:
+                return await worker.run_in_thread(self.fn, *args, **kwargs)
+            return await asyncio.to_thread(self.fn, *args, **kwargs)
+
+        if self.execution == "process":
+            if worker is None:
+                raise RuntimeError("Process execution requires a worker runtime.")
+            return await worker.run_in_process(self.fn, *args, **kwargs)
+
+        raise RuntimeError(f"Unknown execution mode {self.execution!r}.")
+
+    def __call__(self, *args: Any, **kwargs: Any) -> Any:
+        return self.fn(*args, **kwargs)
+
+    def __repr__(self) -> str:
+        return (
+            "Actor(fn=%r, actor_name=%r, queue_name=%r, execution=%r)"
+            % (self.fn, self.actor_name, self.queue_name, self.execution)
+        )
+
+
+class ActorDecorator(Protocol):
+    def __call__(self, fn: Callable[..., Any]) -> Actor: ...
+
+
+@overload
+def actor(fn: Callable[..., Any]) -> Actor:
+    pass
+
+
+@overload
+def actor(
+    *,
+    broker: Optional[Broker] = None,
+    actor_name: Optional[str] = None,
+    queue_name: str = "default",
+    execution: Optional[ExecutionMode] = None,
+    **options: Any,
+) -> ActorDecorator:
+    pass
+
+
+def actor(
+    fn: Optional[Callable[..., Any]] = None,
+    *,
+    broker: Optional[Broker] = None,
+    actor_name: Optional[str] = None,
+    queue_name: str = "default",
+    execution: Optional[ExecutionMode] = None,
+    **options: Any,
+) -> Union[Actor, ActorDecorator]:
+    def decorator(inner_fn: Callable[..., Any]) -> Actor:
+        target_broker = broker or get_broker()
+        return Actor(
+            inner_fn,
+            broker=target_broker,
+            actor_name=actor_name or inner_fn.__name__,
+            queue_name=queue_name,
+            execution=execution,
+            options=options,
+        )
+
+    if fn is None:
+        return decorator
+
+    return decorator(fn)