taskiq-llm-queue 0.1.0a1__py3-none-any.whl

taskiq_llm_queue-0.1.0a1.dist-info/METADATA

@@ -0,0 +1,296 @@
+Metadata-Version: 2.5
+Name: taskiq-llm-queue
+Version: 0.1.0a1
+Summary: An LLM request client with its own durable queue: taskiq + Amazon SQS, adaptive limits, retry and fallback.
+Project-URL: Homepage, https://github.com/techiewonk/taskiq-llm-queue
+Project-URL: Documentation, https://github.com/techiewonk/taskiq-llm-queue#readme
+Project-URL: Source, https://github.com/techiewonk/taskiq-llm-queue
+Project-URL: Issues, https://github.com/techiewonk/taskiq-llm-queue/issues
+Project-URL: Changelog, https://github.com/techiewonk/taskiq-llm-queue/blob/main/CHANGELOG.md
+Author: The taskiq-llm-queue authors
+License-Expression: MIT
+License-File: LICENSE
+License-File: NOTICE
+Keywords: bedrock,langchain,langgraph,llm,openai,queue,rate-limit,sqs,taskiq
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Framework :: AsyncIO
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Distributed Computing
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: aiobotocore>=2.23
+Requires-Dist: orjson<4,>=3.10
+Requires-Dist: pydantic<3,>=2.10
+Requires-Dist: redis>=5.0.1
+Requires-Dist: structlog>=24.1
+Requires-Dist: taskiq<0.14,>=0.13
+Provides-Extra: langchain-aws
+Requires-Dist: langchain-aws<2,>=0.2.24; extra == 'langchain-aws'
+Requires-Dist: langchain-core<2,>=0.3.68; extra == 'langchain-aws'
+Provides-Extra: langchain-openai
+Requires-Dist: langchain-core<2,>=0.3.68; extra == 'langchain-openai'
+Requires-Dist: langchain-openai<2,>=0.3.26; extra == 'langchain-openai'
+Provides-Extra: metrics
+Requires-Dist: prometheus-client>=0.20; extra == 'metrics'
+Provides-Extra: openai
+Requires-Dist: httpx<1,>=0.27; extra == 'openai'
+Requires-Dist: openai<4,>=1.40; extra == 'openai'
+Provides-Extra: speedups
+Requires-Dist: hiredis>=3; extra == 'speedups'
+Description-Content-Type: text/markdown
+
+# taskiq-llm-queue
+
+> **Status:** pre-alpha (`0.1.0a1`). Built and tested: the `LLMQueue` client, the SQS broker, adaptive limiter, claims, circuit breaker, retry and fallback, JSON and S3 result backends, Bedrock and OpenAI-compatible providers, LangChain/LangGraph adapters, canary routing, failover and the CLI. 619 tests on Valkey 8.1, ElasticMQ 1.7.1 and a fake LLM server; CI on Python 3.11–3.14. Breaking changes are still possible before 0.1 (see [CHANGELOG](https://github.com/techiewonk/taskiq-llm-queue/blob/main/CHANGELOG.md)).
+
+An LLM request client with its own durable queue, built on [taskiq](https://github.com/taskiq-python/taskiq) and Amazon SQS.
+
+Your code keeps calling LangChain models (`ainvoke`, `aembed_query`) or a small client API. Each call becomes a task on an SQS queue. Workers in your own app's processes run it under a limit learned from the provider's throttling, retry and fall back when needed, and return the result through Redis/Valkey.
+
+```
+your code ──> client / LangChain adapter ──> SQS (per model: -rt queue first, base, bulk)
+                                               │
+              result <── Redis/Valkey <── worker in your pods ──> Bedrock / OpenAI-compatible
+            (per-pod dispatcher)          (adaptive limiter, retry, fallback)
+```
+
+## Why
+
+- **Spikes wait in a queue instead of failing.** Throttled calls aren't lost or retried in a storm.
+- **No quota numbers to maintain.** The limit for each model and target is learned from throttle errors and shared by all pods.
+- **Chat first.** Separate `-rt` queues and reserved capacity keep live traffic ahead of background and bulk work.
+- **No extra servers.** The same app pods send and execute; SQS is only the broker.
+- **Drop-in.** LangChain/LangGraph code doesn't change: tool calling, callbacks (Langfuse, LangSmith) and response objects are identical.
+- **Safe rollout.** `direct` mode uses the same limiter, and a live canary moves traffic per model and lane without restarts.
+
+## Install
+
+```bash
+uv add "taskiq-llm-queue[langchain-aws]"                     # Bedrock + LangChain
+uv add "taskiq-llm-queue[openai,langchain-openai]"           # OpenAI-compatible + LangChain
+uv add "taskiq-llm-queue[speedups]"                          # hiredis parser for redis-py
+```
+
+| | Packages |
+|---|---|
+| Core | `taskiq`, `aiobotocore`, `redis>=5.0.1`, `pydantic>=2.10`, `orjson`, `structlog` |
+| `openai` | `openai`, `httpx` |
+| `langchain-aws` / `langchain-openai` | LangChain **0.3 and 1.x** are both supported |
+| `metrics` | `prometheus-client` |
+| `speedups` | `hiredis` |
+
+The package declares version **ranges**; pin exact versions in your app's lockfile. `aiobotocore` pins a narrow `botocore` window, so let your resolver pick an `aiobotocore` that matches your `boto3`.
+
+## Quickstart
+
+```python
+from taskiq_llm_queue import LLMQueue, ModelConfig, QueueConfig, Target, request_context
+from taskiq_llm_queue.langchain import QueuedChatBedrockConverse
+
+llmq = LLMQueue(QueueConfig(
+    key_prefix="llmq:local",
+    redis_host="localhost", redis_port=6379, redis_password="localdev", redis_ssl=False,
+    sqs_endpoint_url="http://localhost:9324",                 # ElasticMQ locally; omit on AWS
+    models={
+        "gpt120b": ModelConfig(
+            provider="bedrock", model_id="openai.gpt-oss-120b-1:0",
+            targets={"primary": Target(region="ap-south-1"),
+                     "apac": Target(profile_id="apac.openai.gpt-oss-120b-1:0")},
+            queues={"rt": "llm-gpt120b-rt", "default": "llm-gpt120b"}),
+    },
+))                                                            # or QueueConfig.from_env()  (LLMQ_* variables)
+
+async def main():
+    await llmq.start(mode="both")                             # "send" | "work" | "both"
+    model = QueuedChatBedrockConverse(llmq=llmq, model_alias="gpt120b")
+    with request_context(lane="rt", session_id="s-1", user_id="u-1"):
+        reply = await model.ainvoke("Summarise today's market in one line.")
+    print(reply.content, reply.response_metadata["served_model"])
+    await llmq.stop()                                         # drains in-flight work
+```
+
+In a FastAPI app, call `start()` in the lifespan startup and `stop()` on shutdown. Send-only processes, such as CLIs and scripts, use `mode="send"`.
+
+## Client API
+
+Every call returns a result object with `.meta`: `served_model`, `served_target`, `fallback_used`, `attempts`, `queue_wait_ms`, `provider_ms`, `worker`.
+
+```python
+res = await llmq.converse("gpt120b", {"messages": [...], "system": [...], "toolConfig": ...},
+                          lane="rt", fallback="chain", timeout_secs=90, request_id="req-1")
+res.response                    # exact Bedrock Converse dict
+
+emb = await llmq.embed("titan", ["a", "b"], lane="bulk")
+emb.vectors                     # list[list[float]]; `emb` itself also works as a list
+
+chat = await llmq.chat("llama", {"messages": [{"role": "user", "content": "hi"}]})
+chat.response                   # OpenAI-compatible chat completion dict
+
+raw = await llmq.invoke_model("titan", {"inputText": "a"}, content_type="application/json")
+raw.json(), raw.body, raw.content_type
+
+await llmq.cancel("req-1")      # not run if not started; the caller gets LLMQueueCancelled
+```
+
+- **Timing:** `timeout_secs` (relative) or `deadline_at` (absolute epoch seconds) on every call.
+- **Model ids** are aliases, optionally with a target: `gpt120b`, `gpt120b@apac`. Unknown aliases are rejected when the config is built.
+- **Errors:** everything derives from `LLMQueueError`: `LLMQueueTimeout`, `LLMQueueCancelled`, `ProviderError(retryable, retry_after_secs)`, `QueueUnavailable`, `ConfigError`.
+- **Sync callers:** `converse_sync`, `chat_sync`, `embed_sync`, `invoke_model_sync` run `direct` under the shared limiter (counted as sync fallbacks). Set `strict_async=True` to raise instead. `close_sync()` (or `stop()`) closes the sync runner.
+- **Jobs:** `register_job(name, fn)` on workers, `send_job(name, payload, timeout_secs=...)` anywhere; jobs run on the job queues with the same claims and DLQ rules.
+
+## LangChain and LangGraph
+
+| Class | Based on | How |
+|---|---|---|
+| `QueuedChatBedrockConverse` | `langchain_aws.ChatBedrockConverse` | Builds the request and parses the reply with the installed langchain-aws's own code, so `bind_tools`, structured output, cache usage and service tier match exactly |
+| `QueuedBedrockEmbeddings` | `langchain_aws.BedrockEmbeddings` | `aembed_query` / `aembed_documents` go through the queue |
+| `QueuedChatOpenAI` | `langchain_openai.ChatOpenAI` | `http_async_client` with a queued httpx transport: chat completions and embeddings go through the queue; queue meta lands in `response_metadata`. The Responses API (`/responses`) is not supported yet |
+| `QueuedOpenAIEmbeddings` | `langchain_openai.OpenAIEmbeddings` | Same transport; `check_embedding_ctx_length=False` by default for non-OpenAI servers |
+
+- **Model names:** pass the queue alias as `model_alias=` (plus `model_id=` or `llmq=`). Passing an alias as `model=` raises a clear error.
+- **Streaming:** all queued classes force `disable_streaming=True` / `streaming=False`. `astream()` and LangGraph `stream_mode="messages"` return one complete message, so nothing bypasses the queue.
+- **Laziness:** queued classes find the running `LLMQueue` on first use and check routing and the kill switch on every call, so you can create them at import time.
+
+## Configuration
+
+`QueueConfig.from_env()` reads `LLMQ_*` variables (durations end in `_SECS`; older `_S` names still work with a one-time warning). Notable settings:
+
+| Setting | Default | Note |
+|---|---|---|
+| `aws_region` | `None` | Left to the AWS credential chain / IRSA |
+| `redis_url` or `redis_host`/`port`/`username`/`password`/`ssl` | localhost | `redis_password` is a `SecretStr` |
+| `redis_socket_timeout_secs` / `connect_timeout` / `health_check_interval` | 5 / 2 / 30 | Connection errors retried 3× with backoff |
+| `id_hash_salt` (`LLMQ_ID_HASH_SALT`) | library constant | **Set your own in production**; keys the BLAKE2b hash of session/user ids |
+| `timeout_secs` per lane | rt 90, bg 300, bulk 900 | |
+| `attempt_timeout_secs` | 180 | One provider attempt; a hung target moves to the next fallback step |
+| `provider_read_timeout_secs` / `provider_connect_timeout_secs` | 300 / 10 | Bedrock botocore and OpenAI httpx timeouts |
+| `sqs_send_timeout_secs` | 5 | SQS counts as down only when no send completed for this long; then the call runs direct |
+| `throttle_retry_attempts` | 6 | Tries per fallback step for throttles (backoff holds no slot) |
+| `valkey_wait_failover_secs` | 10 | Result reads failing this long make waiting callers run direct |
+| `redis_max_connections` | 32 | Raised automatically to worker slots + 16 |
+
+Redis keys are cluster-safe (hash-tagged per request and per model+target), so cluster-mode Valkey (e.g. ElastiCache Serverless) works.
+
+## Providers
+
+```python
+ModelConfig(provider="openai", model_id="llama-3.1-70b",
+            targets={"primary": Target(base_url="http://vllm:8000/v1")}, api_key_env="LOCAL_LLM_KEY")
+QueueConfig(allowed_providers={"bedrock"}, denied_hosts=["*.openai.azure.com"])   # enforced on sender and worker
+QueueConfig(providers={"myprov": "mypkg.providers:MyProvider"})                    # custom provider, loaded by every pod
+```
+
+A custom provider implements `async call(op, params, target) -> dict` and `classify(exc) -> ErrorInfo(kind="throttle" | "retryable" | "fatal", retry_after_secs)`. It can also be registered through the `taskiq_llm_queue.providers` entry point. SDK-level retries are off inside workers (botocore `total_max_attempts=1`, openai `max_retries=0`), so throttles reach the limiter.
+
+## Lanes and queues
+
+| Lane | Queue | Use |
+|---|---|---|
+| `rt` | `llm-<model>-rt` | Live user requests; polled first; may use the whole limit |
+| `bg` | `llm-<model>` | Background work; at most 80 % of the limit |
+| `bulk` | `llm-<model>-bulk` if configured (e.g. embeddings), else base | Ingestion/batch; lowest priority |
+
+A queued `bg`/`bulk` call waits at most 15 s for a slot, then goes back to the queue with a delay instead of holding a worker.
+
+## Results
+
+Results are stored as JSON (never pickle) for 5 minutes and read once (`GETDEL`). Each pod runs one result dispatcher: a pub/sub wake reads only the announced results, and a periodic batched sweep catches the rest, so 10K waiting calls don't mean 10K polling loops.
+
+## Fallbacks: how to check and update
+
+A fallback chain is tried **inside the worker**. Each step is retried with backoff before moving on:
+
+1. same model, primary target
+2. same model, another target (e.g. a cross-region profile `gpt120b@apac`)
+3. another model (only when the call's policy is `chain`)
+
+Embeddings never switch model. Throttles shrink the limiter but never trip the circuit breaker; only 5xx and timeouts do.
+
+```bash
+uv run llm-queue fallback get                       # whole config (JSON); starts from your app's defaults
+uv run llm-queue fallback get --model gpt120b       # resolved chain for one model
+uv run llm-queue fallback test --model gpt120b      # dry run: which target would serve now
+uv run llm-queue fallback set --model gpt120b --chain "gpt20b"
+uv run llm-queue fallback set --file fallback.json  # merges per model; --replace to overwrite all
+uv run llm-queue fallback reset                     # back to the startup default
+```
+
+Chains are validated and reloaded by every pod within 30 s with no restart. An invalid config is rejected and the last good one stays. Every change is written to an audit stream.
+
+```json
+{ "chains": {
+    "gpt120b": ["gpt20b"],
+    "gpt20b": [{"model": "gpt120b", "require_headroom": true}]
+  },
+  "skip_targets": { "gpt120b": ["apac"] } }
+```
+
+**To see fallbacks happen:** `res.meta` / `response_metadata` show `served_model`, `served_target`, `fallback_used` and `attempts`, and the `llmq_fallback_total{model,step}` metric counts them. `samples/05_fallback_on_throttle.py` shows it end to end.
+
+## Operating it
+
+```bash
+uv run llm-queue routing set --model gpt20b --lane rt --pct 10   # canary: 10 % of sessions via SQS
+uv run llm-queue routing kill on                                 # everything direct within 30 s
+uv run llm-queue routing reset
+uv run llm-queue limiter show --model gpt120b                    # learned limit, in-flight, pause
+uv run llm-queue limiter reset --model gpt120b [--target apac]   # forget the learned limit
+uv run llm-queue breaker show | reset
+uv run llm-queue queues                                          # messages per queue and DLQ
+uv run llm-queue job send ingest --payload-file payload.json --queue jobs-ingest
+```
+
+## Failure behaviour
+
+| Failure | What happens |
+|---|---|
+| Provider throttles | Limiter shrinks and pauses that model + target; brokers stop receiving it unless another target in the chain has headroom; work waits in SQS |
+| Provider down (5xx, timeouts) | Circuit breaker opens → next fallback step |
+| Worker killed mid-task | Message reappears after the visibility timeout; the claim prevents a second provider call |
+| Rolling deploy | Pods stop receiving at once, release unstarted work, finish running calls, then exit (90 s budget in the reference deployment) |
+| **SQS unreachable** | That model switches to `direct` (+ shared limiter) for 300 s on transient send errors only. An ambiguous send is claimed first, so it can't run twice |
+| **Redis/Valkey unreachable** | Workers release messages instead of losing them; senders run `direct` with a local per-pod cap (last known limit ÷ live pods) |
+
+## Logging
+
+The library logs with structlog's key-value API on top of stdlib `logging` and **never configures logging itself**. Records are named `taskiq_llm_queue.*`; the message reads `event key=value ...` and the fields are also on `record.llmq` (a dict). Prompts, responses and params are never logged.
+
+```python
+import logging
+logging.getLogger("taskiq_llm_queue").setLevel(logging.INFO)   # stdlib apps: that's all
+
+# loguru apps: the usual InterceptHandler picks these up like any stdlib logger.
+# structlog apps: use structlog.stdlib.ProcessorFormatter on your handler to render them.
+```
+
+The `llm-queue` CLI logs warnings and errors to stderr, so its stdout is always plain JSON.
+
+## Security and data
+
+- Encryption is server-side on the queues (SSE-SQS or SSE-KMS); the library needs no setting for either.
+- Prompts and responses are never logged. `user_id` / `session_id` are hashed (keyed BLAKE2b) in task labels; raw ids stay inside the message body.
+- Nothing secret goes into messages; credentials come from the AWS chain (IRSA / Pod Identity) or env.
+- Unknown task names or newer message versions are released for another pod, never silently dropped.
+
+## Run it locally (no AWS needed)
+
+```bash
+make up              # Valkey (password) + ElasticMQ + fakellm
+make test            # unit + integration against the stack
+make down
+# Windows without make:  uv run python tools/llmq_dev.py up | test | down
+```
+
+Runnable examples are in [`samples/`](https://github.com/techiewonk/taskiq-llm-queue/tree/main/samples): quickstart, chat, embeddings, lanes, fallback on throttle, send-only and worker pods, LangChain, LangGraph, jobs, cancel and timeouts, sync calls, CLI, and `13_research_assistant_flow.py`, which replays a real app's flows locally or against real Bedrock (`--target aws`, settings from `.env` or AWS Secrets Manager). The stack, capability checks and test layers are described in [docs/TESTING.md](https://github.com/techiewonk/taskiq-llm-queue/blob/main/docs/TESTING.md); delivery-safety rules in [docs/DESIGN.md](https://github.com/techiewonk/taskiq-llm-queue/blob/main/docs/DESIGN.md).
+
+## Credits
+
+Broker design and feature set build on [taskiq-aio-sqs](https://github.com/vonsteer/taskiq-aio-sqs) (MIT, © 2022-2024 Jesse Constante) and [taskiq-sqs](https://github.com/taskiq-python/taskiq-sqs) (MIT, © 2026 Mike Shultz and the Taskiq team). Licence: MIT.

taskiq_llm_queue-0.1.0a1.dist-info/RECORD

@@ -0,0 +1,65 @@
+taskiq_llm_queue/__init__.py,sha256=YbXgYe6Rnel0PYCxxMvTB18lNmT1IjRhzGlYI3ZYz-8,5061
+taskiq_llm_queue/_logging.py,sha256=10VK1aDkD7rjSCC7F1ti7iKuC5zdbX8qMXbhdGo8hdY,1704
+taskiq_llm_queue/_version.py,sha256=wJ7_Rt22BJ255pg0_dqlU1SMURFvmz5YSNLNi9dSlwg,150
+taskiq_llm_queue/cli.py,sha256=vJlTmIsD1wh8pdi3ovsa4GSA_EM0NxwwS8ASB-g3HIA,20366
+taskiq_llm_queue/client.py,sha256=hSlKCKDb1xDXKM2k-OeUrixIimjBb-7HBVVvdqZDQ4E,56771
+taskiq_llm_queue/config.py,sha256=qdd76IoGxDYor7lrSKqSW2dNos5ZmZd794OoKg6w5KY,25638
+taskiq_llm_queue/context.py,sha256=09AJL3I3vJLK7Ke2B7GeqtN8ElZk7DuN_40nG7jNhN4,1977
+taskiq_llm_queue/core.py,sha256=YdNKGtIZLhOlJdvazojwvWHTsbCIoW8jaMjl2mRSsks,35468
+taskiq_llm_queue/envelope.py,sha256=ZNLbssnB2dQWwYYPk5hMFkBxOPnTIMKUtcqTtesQMNA,4956
+taskiq_llm_queue/errors.py,sha256=z7mkXXW9E-oSmnJx38aULncZ-A0SJEqOBEibuJWVkz0,2339
+taskiq_llm_queue/failover.py,sha256=DNC32e0G2LSxeFY2UFzFT84AA6NryIZtvPXasIdqwvU,12089
+taskiq_llm_queue/http_transport.py,sha256=a9so7lIofhqRsAVDL1iWOTckZa2WI0YDcMk1BQvEolY,9609
+taskiq_llm_queue/keys.py,sha256=Me6C-feM7vOXyNyKUcudSR1OfJ85htY446ngNFI00Fg,3200
+taskiq_llm_queue/lifecycle.py,sha256=NTgsMuVjPCLpdyOXW7FpiAUiNKLuMxzQyRJFFVG_sNc,20279
+taskiq_llm_queue/metrics_names.py,sha256=IOfG6NsAGlkUR-_dIxFCyez-0GwSMnnT2s89UEfuyZQ,771
+taskiq_llm_queue/protocols.py,sha256=mWzPP0mcbYnCvMk2TiaUxk-yDznnbsIUJObjYlGclGs,4408
+taskiq_llm_queue/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+taskiq_llm_queue/routing.py,sha256=xcLM2Nmsz7gYIVqKMtwP-McKuO4Gv5NlJ24yGv9Y9Xo,7061
+taskiq_llm_queue/routing_hooks.py,sha256=s7WI-KhO7LrvEU3G7EHmfN455pw5C2vicRJ2ctxsIQQ,3357
+taskiq_llm_queue/tasks.py,sha256=B3Hd5OrXaAuPYCVAnCIS7j7DLjz5BwqHbbOKSkCeMWo,19505
+taskiq_llm_queue/broker/__init__.py,sha256=Qsiy6A6yTxLeCM_V5Kln161PCXWE-IO3jNrNtR9eOHY,2330
+taskiq_llm_queue/broker/acker.py,sha256=3N4gucLrgpbTONQH-CzqTINm_xW0TFR4WG2tQn_9opA,9288
+taskiq_llm_queue/broker/batching.py,sha256=Nw0FhHxJ8yDgn0HR_CeJfmdvrBvT3oNKUpDnnbRtRQQ,6977
+taskiq_llm_queue/broker/broker.py,sha256=oFEKVZz6T9roMGpzO0V9Z4mvak6EKmyHygVeyoRpvKs,41634
+taskiq_llm_queue/broker/delivery.py,sha256=p6p7xo55f86hBaMoT2u2DfyT92vK_HWoohaBlFvO1-g,1767
+taskiq_llm_queue/broker/errors.py,sha256=cwCSeBn9dZUPSBqUDGLdprKxOtZ9Q4IpWTjq4XJYvH8,6134
+taskiq_llm_queue/broker/events.py,sha256=GGCsXZlA7FGpF2NWDt3OKPWHCAS7LddJCn9Bxym32YI,2113
+taskiq_llm_queue/broker/filters.py,sha256=5WCl66ykcDXtFaRiBwUul3x7bRQ2Pp0MYpxd7c6bpfQ,1195
+taskiq_llm_queue/broker/heartbeat.py,sha256=YoqUfr9esLvk73Omtv2w4RMhphWJccmeG8DlInPPVQY,6387
+taskiq_llm_queue/broker/queue.py,sha256=rb4FsckJz_o5xRss44kFSgK9pQJ70rJh-LigAjhYYz0,5953
+taskiq_llm_queue/broker/s3_offload.py,sha256=jOeIEELqTZSznd8c1ekNF40P12jnpChu36v54jTLho8,14584
+taskiq_llm_queue/broker/scheduler.py,sha256=zf5u13ZzsnlOsU535znZ-_WolhVUYVvWyXV5lNN_be8,4751
+taskiq_llm_queue/broker/serializer.py,sha256=YuPMNqv8X8EFi7EJtriqNx6BxwnYyD-6XmYfJS__uys,1363
+taskiq_llm_queue/langchain/__init__.py,sha256=pleTIl0TOmC9F7wuMU61oPWI4EiYk7mIi_hsvA_QugA,2159
+taskiq_llm_queue/langchain/_client.py,sha256=h6fdA73Tgb51OZkMzQaQxh0vii-icNM7saxrIcA9r7E,4352
+taskiq_llm_queue/langchain/bedrock.py,sha256=UQmpb2bq-yFJ5aI6X9Q3ld-_IqjonxumUwiI1H8Ee6A,11616
+taskiq_llm_queue/langchain/openai.py,sha256=cFwq-EEJMR62xaBrHynKSNH1_WAJxf6dCxIoZVRlhwI,4158
+taskiq_llm_queue/metrics/__init__.py,sha256=XsUAn_NFMgrB8_qXnAFOuweF10qM0biyMy0ZyMk-xB4,1704
+taskiq_llm_queue/metrics/emf.py,sha256=PvR2htszW2Ft8iRbXQeINqloXqj2yAW8Sua3WPASc4c,3919
+taskiq_llm_queue/metrics/prometheus.py,sha256=COSeCv3yDAZCY72GOKCINcOzIK3t81R9Rcnsu7GCV10,2946
+taskiq_llm_queue/providers/__init__.py,sha256=C45nWW0R6bTTioW7G_L3iaaEk-dRptGkBORpX6VotUE,1662
+taskiq_llm_queue/providers/bedrock.py,sha256=X0ex2odFg3qouy4Q82VqtUp7-Oyj9pqGEa6Cw-JY1qs,8128
+taskiq_llm_queue/providers/classify.py,sha256=JJPeAhMDxzTWxxZtz-IEU_Qz8_QhuiTot1w5cDej58U,9667
+taskiq_llm_queue/providers/embeddings.py,sha256=7i6klmXAaau_iyhFdq__ROIhw2jVEybuI4Bbvd-JId8,3371
+taskiq_llm_queue/providers/hosts.py,sha256=aywHB5S7-zI4Cc730_y9FngkvjGwQfRnrKa2rqf8l54,1290
+taskiq_llm_queue/providers/openai_compat.py,sha256=xuz4XkZZhtRpmshJ5Tda_wVnBnHpK_X9OI9B44jxvT0,6191
+taskiq_llm_queue/providers/registry.py,sha256=GsbwqPHB6ymMZYyIRkqax6hhR6KVOp-OFFra0G5IV50,7340
+taskiq_llm_queue/reliability/__init__.py,sha256=FIUW0u-u_XnupyDxQd83VrUZvXNWd-b4OCj2q5QyizI,1502
+taskiq_llm_queue/reliability/_lua.py,sha256=IpqqM-4dn-XGf6_riC2bDcDDuLiFnvCP9ZiPTTRqhRw,18622
+taskiq_llm_queue/reliability/breaker.py,sha256=nbbEICcJmyxmjpkjLsoPKxHqcdiVxN43R-lrQon-FR0,3287
+taskiq_llm_queue/reliability/claims.py,sha256=VbVzRogoMMtxlHtnkmY-eaT51I4_ehM_NCpGYFW0RuI,9838
+taskiq_llm_queue/reliability/config_store.py,sha256=p9ceiNSMTDzqfbSNA-F7NehTq_nopWp56atSN1Z_pPY,5997
+taskiq_llm_queue/reliability/fallback.py,sha256=HaSK5minJBmbvqkT_0tMsTseAypvFNigYdznlt2-03Y,4022
+taskiq_llm_queue/reliability/limiter.py,sha256=bDwnD901gKh_FR604FIUgmkCq80EwygZ9hxSSpLB2Bo,19866
+taskiq_llm_queue/reliability/local_limiter.py,sha256=DC4JDeq-0RVFhlS3igUQpr6fxF_ml8P48jZ0y0OkS7E,5937
+taskiq_llm_queue/results/__init__.py,sha256=u-N4eMjqYHxH20kUIW2agLHrbXu-LZPElKCaQcO-XBY,719
+taskiq_llm_queue/results/backend.py,sha256=mE5i3XXfuZyrJwdejl13UHAjuIwCl1lhMBqgzMDYb6g,9899
+taskiq_llm_queue/results/dispatcher.py,sha256=GZeHiqlLwxuX6ISLhyS7Fc_8X9BtG7TZZPEkn4xmrZs,16444
+taskiq_llm_queue/results/s3_backend.py,sha256=3snDgJhRMjlTPhPNWHVwsQqh4JvdXFFwDxbNCWq0oOA,4104
+taskiq_llm_queue-0.1.0a1.dist-info/METADATA,sha256=OvWaIwuQeqUOaGyJZ0ZUnqiHMu6K_TQARUwzOyxxNX0,18269
+taskiq_llm_queue-0.1.0a1.dist-info/WHEEL,sha256=2iZrJkDIkeDh0k9SikKMqFuRP-ua6mb4Ir1AsER4BEw,87
+taskiq_llm_queue-0.1.0a1.dist-info/entry_points.txt,sha256=u9QOO0rKyinfwfcBrnyNu5LdDproymBwFJSSedoSWBg,64
+taskiq_llm_queue-0.1.0a1.dist-info/licenses/LICENSE,sha256=fyt0lY9XpMIg0h9QEgiY3twFAGBqEiyCBB_9C83IpIc,1085
+taskiq_llm_queue-0.1.0a1.dist-info/licenses/NOTICE,sha256=QBk0_Gzqpez_MheapoFec71cpSSfrPDKQMoK4x1JgYQ,794
+taskiq_llm_queue-0.1.0a1.dist-info/RECORD,,

taskiq_llm_queue-0.1.0a1.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.33.0
+Root-Is-Purelib: true
+Tag: py3-none-any

taskiq_llm_queue-0.1.0a1.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+llm-queue = taskiq_llm_queue.cli:console_main

taskiq_llm_queue-0.1.0a1.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 The taskiq-llm-queue authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

taskiq_llm_queue-0.1.0a1.dist-info/licenses/NOTICE

@@ -0,0 +1,15 @@
+taskiq-llm-queue
+Copyright (c) 2026 The taskiq-llm-queue authors. Licensed under the MIT License.
+
+The SQS broker's design and feature set build on two MIT-licensed projects. This package is
+an independent implementation; any code adapted from them keeps its original notice below.
+
+taskiq-aio-sqs — https://github.com/vonsteer/taskiq-aio-sqs
+  Copyright (c) 2022-2024 Jesse Constante. MIT License.
+  Ideas used: aiobotocore-based broker, visibility heartbeat, message metadata,
+  extended (S3) messages, batching, fair queues.
+
+taskiq-sqs — https://github.com/taskiq-python/taskiq-sqs
+  Copyright (c) 2026 Mike Shultz and Taskiq team. MIT License.
+  Ideas used: multi-queue structure, queue declaration, message expiry,
+  batch flush on shutdown, S3 offload middleware and result backend.