tempest-fastapi-sdk 0.95.0__tar.gz → 0.96.0__tar.gz

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/CHANGELOG.md

@@ -5,6 +5,33 @@ All notable changes to **tempest-fastapi-sdk** are listed below.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.96.0] — 2026-07-05
+
+### Added
+
+- **Self-hosted GenAI — capacity check (`tempest_fastapi_sdk.genai`)**,
+  the first slice of running HuggingFace models on your own hardware.
+  Before downloading gigabytes of weights, check whether the host can run
+  a model:
+  - `probe_hardware()` → `HardwareInfo` (CPU, total/available RAM, CUDA
+    GPUs with per-device VRAM, Apple MPS, free disk). Degrades gracefully
+    without `psutil` / `torch`.
+  - `can_run(...)` → `CapacityReport` (`fits`, chosen `device`, estimated
+    vs available bytes, `headroom_pct`, and a concrete `suggestion` when
+    it doesn't fit — quantize, offload, or pick a smaller model).
+  - `recommend(...)` picks the first precision (`bfloat16` → `int8` →
+    `int4`) that fits.
+  - `estimate_model_bytes` / `bytes_per_param` (the estimation math) and
+    `fetch_num_params` (reads a model's parameter count from the Hub via
+    `huggingface_hub`, without downloading weights). `ModelDtype` enum.
+  - New `[genai]` extra (transformers + torch + accelerate + safetensors +
+    huggingface-hub) and `[genai-quant]` (bitsandbytes). The capacity
+    functions import **without** the extra — `torch` is only used to probe
+    real GPUs. Import from `tempest_fastapi_sdk.genai`.
+  - Upcoming slices: `TextGenerator` (+ quantization), `Embedder`,
+    model/result caching, `BatchScheduler`, and RAG context (web search +
+    PDF reading).
+
 ## [0.95.0] — 2026-07-05
 
 ### Added

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/CLAUDE.md

@@ -129,6 +129,19 @@ The SDK currently covers (Sep 2025+, post-v0.31.x):
   `ort-vision-sdk`: lazy `Detector`/`Classifier`/`Segmenter` + prediction
   schemas + `to_detection_schemas`/`to_classification_schema`/
   `to_segmentation_schemas` mappers.
+- **GenAI self-hosted** (`[genai]` extra: transformers+torch+accelerate;
+  `[genai-quant]` = bitsandbytes) — `tempest_fastapi_sdk.genai`, delivered
+  in slices. **Shipped (v0.96):** hardware capacity check — `probe_hardware`
+  → `HardwareInfo` (CPU/RAM/CUDA-VRAM/MPS/disk, degrades without
+  psutil/torch), `can_run`/`recommend` → `CapacityReport` (fits? device,
+  estimate vs available, suggestion to quantize/offload), `estimate_model_bytes`/
+  `bytes_per_param`/`fetch_num_params` (Hub metadata, no weight download),
+  `ModelDtype`. Capacity fns import WITHOUT the extra. **Planned slices:**
+  `TextGenerator` (LLM + int4/int8 quant, lazy load + idle-unload),
+  `Embedder`, model/result cache, `BatchScheduler` (coalesce concurrent
+  calls), RAG context (pluggable web search "leviathan-style" + PDF
+  reader → chunks) to inject into LLMs. Classes-only (no bundled router).
+  Submodule import like queue/tasks/vision.
 - **SSR** (`[ssr]` extra) — `tempest_fastapi_sdk.ssr`: typed Python
   pages rendered to HTML via `tempestweb`'s `render_to_html` /
   `render_document`. `Page` (typed `Component` base — `body()` +

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tempest-fastapi-sdk
-Version: 0.95.0
+Version: 0.96.0
 Summary: Shared FastAPI building blocks: base schemas, ORM model, async repository, exceptions, pagination and settings — the conventions used across Tempest projects.
 Project-URL: Homepage, https://github.com/mauriciobenjamin700/tempest-fastapi-sdk
 Project-URL: Repository, https://github.com/mauriciobenjamin700/tempest-fastapi-sdk
@@ -69,6 +69,14 @@ Provides-Extra: email
 Requires-Dist: aiosmtplib>=4.0.0; extra == 'email'
 Requires-Dist: email-validator>=2.0.0; extra == 'email'
 Requires-Dist: jinja2>=3.1.0; extra == 'email'
+Provides-Extra: genai
+Requires-Dist: accelerate>=0.33.0; extra == 'genai'
+Requires-Dist: huggingface-hub>=0.24.0; extra == 'genai'
+Requires-Dist: safetensors>=0.4.0; extra == 'genai'
+Requires-Dist: torch>=2.2.0; extra == 'genai'
+Requires-Dist: transformers>=4.44.0; extra == 'genai'
+Provides-Extra: genai-quant
+Requires-Dist: bitsandbytes>=0.43.0; extra == 'genai-quant'
 Provides-Extra: http
 Requires-Dist: httpx>=0.28.0; extra == 'http'
 Provides-Extra: metrics
@@ -226,6 +234,7 @@ Feature-rich helpers pull in third-party dependencies that you only need when yo
 | `[prometheus]` | `prometheus-client` | `PrometheusMiddleware`, `make_prometheus_router`, `make_prometheus_registry` |
 | `[mfa]` | `pyotp` | `TOTPHelper` + MFA/2FA endpoints on the bundled auth flow |
 | `[vision]` | `ort-vision-sdk` | `Detector` / `Classifier` / `Segmenter` (ONNX) + prediction schemas |
+| `[genai]` (+ `[genai-quant]`) | `transformers`, `torch`, `accelerate`, `safetensors`, `huggingface-hub` (+ `bitsandbytes`) | Self-hosted GenAI — hardware capacity check (`probe_hardware` / `can_run` / `recommend`); model runners upcoming |
 | `[ssr]` | `tempestweb` | `Page`, `html_response`, `make_htmx_router` — typed Python pages rendered to HTML |
 | `[otel]` | `opentelemetry-sdk` + OTLP/gRPC exporter + FastAPI/SQLAlchemy/httpx instrumentors | `setup_tracing` — distributed tracing |
 | `[sqlite]` | `aiosqlite` | SQLite async driver for `sqlite+aiosqlite://` URLs (dev default) |
@@ -268,11 +277,12 @@ Since `0.7.1` every optional dependency is imported lazily at first instantiatio
 | `tempest_fastapi_sdk.utils.storage_backends` *(extra: `[upload]`)* | `UploadStorage` protocol, `LocalUploadStorage`, `MinIOUploadStorage`, `UploadResult`, `ContentValidator` |
 | `tempest_fastapi_sdk.tasks` *(extra: `[tasks]`)* | `TaskQueue` (typed TaskIQ facade — `.rabbitmq`/`.redis`/`.memory`, `@tq.task` → `Task.enqueue`/`.run`, `@tq.cron`/`@tq.interval`, `start_scheduler`), `Task`; class-based tasks (`TaskDef` + `@task_method` + `TaskQueue.register`); cron helpers (`Cron` / `CronOffset` / `Weekday` + `daily`/`weekdays`/`every_n_minutes`/… builders); `AsyncTaskBrokerManager` / `AsyncTaskScheduler` (legacy wrappers) |
 | `tempest_fastapi_sdk.vision` *(extra: `[vision]`)* | `Detector`, `Classifier`, `Segmenter` (ONNX, lazy), `DetectionSchema`/`ClassificationSchema`/`SegmentationSchema`/`BoundingBoxSchema`/`ClassProbabilitySchema`, `to_detection_schemas`/`to_classification_schema`/`to_segmentation_schemas` |
+| `tempest_fastapi_sdk.genai` *(extra: `[genai]`; capacity check imports without it)* | Self-hosted GenAI capacity check: `probe_hardware`/`HardwareInfo`, `can_run`/`recommend`/`CapacityReport`, `estimate_model_bytes`/`bytes_per_param`/`fetch_num_params`, `ModelDtype` — model runners (`TextGenerator`, `Embedder`), caching, batching and RAG context in upcoming slices |
 | `tempest_fastapi_sdk.ssr` *(extra: `[ssr]`)* | `Page` (typed component base), `html_response` (widget tree → `HTMLResponse`, full document or HTMX fragment), `make_htmx_router` (serves bundled HTMX locally, no CDN) |
 | `tempest_fastapi_sdk.utils` | `to_utc`, `utcnow`, `modify_dict`, `LogUtils`, `AttemptThrottle`/`ThrottleBackend`/`ThrottleStatus`, `generate_opaque_token`/`hash_opaque_token`/`verify_opaque_token`, `get_client_ip`/`get_client_ip_from_scope`, `PasswordUtils` *(extra: `[auth]`)*, `JWTUtils` *(extra: `[auth]`)*, `TOTPHelper` *(extra: `[mfa]`)*, `EmailUtils` *(extra: `[email]`)*, `UploadUtils`/`sniff_mime` *(extra: `[upload]`)*, `DownloadUtils`/`build_content_disposition` *(no extra)*, `FileStoreUtils` — unified upload+download+presign facade *(extra: `[upload]` local / `[minio]` MinIO)*, `MetricsUtils`/`CPUMetrics`/`MemoryMetrics`/`DiskMetrics`/`GPUMetrics`/`SystemMetrics` *(extra: `[metrics]`)*, validated field types (`PositiveIntField`, `NonNegativeIntField`, `CentsField`, `PortField`, `PositiveFloatField`, `NonNegativeFloatField`, `PercentField`, `RatioField`, `LatitudeField`, `LongitudeField`, `PriceField`, `NonEmptyStrField`, `SlugField`, `HexColorField`), BR regex helpers (`CPFField`, `CNPJField`, `CPFOrCNPJField`, `PhoneBRField`, `CEPField`, `PixKeyField` — old names without the suffix kept as deprecated aliases — `PixKeyType`/`detect_pix_key_type`/`is_valid_pix_key`, `is_valid_*`, `normalize_*`, `only_digits`, `*_PATTERN`), BR states/cities (`UF`, `Region`, `StateBR`, `CityBR`, `ChoiceBR`, `UFField`, `CityNameField`, `list_states`, `get_state`, `cities_by_uf`, `states_by_region`, `uf_choices`/`region_choices`/`city_choices`, `is_valid_uf`/`normalize_uf`, `is_valid_city`/`normalize_city`) |
 | `tempest_fastapi_sdk.cli` | `tempest` console script — `new <name>` (scaffold layered service), `lint` / `format` / `fmt-check` / `type` / `test` / `check` (run preferred quality gates), `version` / `--version` |
 
-Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
+Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision`, `tempest_fastapi_sdk.genai` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
 
 ---
 

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/README.md

@@ -117,6 +117,7 @@ Feature-rich helpers pull in third-party dependencies that you only need when yo
 | `[prometheus]` | `prometheus-client` | `PrometheusMiddleware`, `make_prometheus_router`, `make_prometheus_registry` |
 | `[mfa]` | `pyotp` | `TOTPHelper` + MFA/2FA endpoints on the bundled auth flow |
 | `[vision]` | `ort-vision-sdk` | `Detector` / `Classifier` / `Segmenter` (ONNX) + prediction schemas |
+| `[genai]` (+ `[genai-quant]`) | `transformers`, `torch`, `accelerate`, `safetensors`, `huggingface-hub` (+ `bitsandbytes`) | Self-hosted GenAI — hardware capacity check (`probe_hardware` / `can_run` / `recommend`); model runners upcoming |
 | `[ssr]` | `tempestweb` | `Page`, `html_response`, `make_htmx_router` — typed Python pages rendered to HTML |
 | `[otel]` | `opentelemetry-sdk` + OTLP/gRPC exporter + FastAPI/SQLAlchemy/httpx instrumentors | `setup_tracing` — distributed tracing |
 | `[sqlite]` | `aiosqlite` | SQLite async driver for `sqlite+aiosqlite://` URLs (dev default) |
@@ -159,11 +160,12 @@ Since `0.7.1` every optional dependency is imported lazily at first instantiatio
 | `tempest_fastapi_sdk.utils.storage_backends` *(extra: `[upload]`)* | `UploadStorage` protocol, `LocalUploadStorage`, `MinIOUploadStorage`, `UploadResult`, `ContentValidator` |
 | `tempest_fastapi_sdk.tasks` *(extra: `[tasks]`)* | `TaskQueue` (typed TaskIQ facade — `.rabbitmq`/`.redis`/`.memory`, `@tq.task` → `Task.enqueue`/`.run`, `@tq.cron`/`@tq.interval`, `start_scheduler`), `Task`; class-based tasks (`TaskDef` + `@task_method` + `TaskQueue.register`); cron helpers (`Cron` / `CronOffset` / `Weekday` + `daily`/`weekdays`/`every_n_minutes`/… builders); `AsyncTaskBrokerManager` / `AsyncTaskScheduler` (legacy wrappers) |
 | `tempest_fastapi_sdk.vision` *(extra: `[vision]`)* | `Detector`, `Classifier`, `Segmenter` (ONNX, lazy), `DetectionSchema`/`ClassificationSchema`/`SegmentationSchema`/`BoundingBoxSchema`/`ClassProbabilitySchema`, `to_detection_schemas`/`to_classification_schema`/`to_segmentation_schemas` |
+| `tempest_fastapi_sdk.genai` *(extra: `[genai]`; capacity check imports without it)* | Self-hosted GenAI capacity check: `probe_hardware`/`HardwareInfo`, `can_run`/`recommend`/`CapacityReport`, `estimate_model_bytes`/`bytes_per_param`/`fetch_num_params`, `ModelDtype` — model runners (`TextGenerator`, `Embedder`), caching, batching and RAG context in upcoming slices |
 | `tempest_fastapi_sdk.ssr` *(extra: `[ssr]`)* | `Page` (typed component base), `html_response` (widget tree → `HTMLResponse`, full document or HTMX fragment), `make_htmx_router` (serves bundled HTMX locally, no CDN) |
 | `tempest_fastapi_sdk.utils` | `to_utc`, `utcnow`, `modify_dict`, `LogUtils`, `AttemptThrottle`/`ThrottleBackend`/`ThrottleStatus`, `generate_opaque_token`/`hash_opaque_token`/`verify_opaque_token`, `get_client_ip`/`get_client_ip_from_scope`, `PasswordUtils` *(extra: `[auth]`)*, `JWTUtils` *(extra: `[auth]`)*, `TOTPHelper` *(extra: `[mfa]`)*, `EmailUtils` *(extra: `[email]`)*, `UploadUtils`/`sniff_mime` *(extra: `[upload]`)*, `DownloadUtils`/`build_content_disposition` *(no extra)*, `FileStoreUtils` — unified upload+download+presign facade *(extra: `[upload]` local / `[minio]` MinIO)*, `MetricsUtils`/`CPUMetrics`/`MemoryMetrics`/`DiskMetrics`/`GPUMetrics`/`SystemMetrics` *(extra: `[metrics]`)*, validated field types (`PositiveIntField`, `NonNegativeIntField`, `CentsField`, `PortField`, `PositiveFloatField`, `NonNegativeFloatField`, `PercentField`, `RatioField`, `LatitudeField`, `LongitudeField`, `PriceField`, `NonEmptyStrField`, `SlugField`, `HexColorField`), BR regex helpers (`CPFField`, `CNPJField`, `CPFOrCNPJField`, `PhoneBRField`, `CEPField`, `PixKeyField` — old names without the suffix kept as deprecated aliases — `PixKeyType`/`detect_pix_key_type`/`is_valid_pix_key`, `is_valid_*`, `normalize_*`, `only_digits`, `*_PATTERN`), BR states/cities (`UF`, `Region`, `StateBR`, `CityBR`, `ChoiceBR`, `UFField`, `CityNameField`, `list_states`, `get_state`, `cities_by_uf`, `states_by_region`, `uf_choices`/`region_choices`/`city_choices`, `is_valid_uf`/`normalize_uf`, `is_valid_city`/`normalize_city`) |
 | `tempest_fastapi_sdk.cli` | `tempest` console script — `new <name>` (scaffold layered service), `lint` / `format` / `fmt-check` / `type` / `test` / `check` (run preferred quality gates), `version` / `--version` |
 
-Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
+Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision`, `tempest_fastapi_sdk.genai` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
 
 ---
 

tempest_fastapi_sdk-0.96.0/docs/recipes/genai.en.md

@@ -0,0 +1,93 @@
+# Self-hosted generative AI
+
+Run HuggingFace models on **your own hardware** — no external API, no data
+leaving your servers. The `tempest_fastapi_sdk.genai` module ships in
+slices; this page covers the **first**: knowing, *before* you download
+gigabytes of weights, whether the machine can handle the model.
+
+!!! info "Module roadmap"
+    - **Now (v0.96):** `genai.hardware` — probing + `can_run` / `recommend`.
+    - **Coming:** `TextGenerator` (LLM + int4/int8 quantization),
+      `Embedder`, model/result caching, `BatchScheduler`, and RAG context
+      (web search + PDF reading) to inject into LLMs.
+
+The `[genai]` extra (transformers + torch + accelerate) is only needed to
+**run** models. The capacity functions **import without the extra** —
+`torch` is only used (when present) to read real GPU VRAM.
+
+## "Can the machine handle it?"
+
+Loading a too-large model ends in an OOM minutes into the download.
+`can_run` answers first:
+
+```python
+from tempest_fastapi_sdk.genai import can_run, ModelDtype
+
+report = can_run(model_id="Qwen/Qwen2.5-7B-Instruct", dtype=ModelDtype.BFLOAT16)
+
+if report.fits:
+    print(f"OK on {report.device} — {report.headroom_pct:.0f}% headroom")
+else:
+    print(report.reason)
+    print("Suggestion:", report.suggestion)   # e.g. "Quantize to int4 ..."
+```
+
+`CapacityReport` carries: `fits`, `device` (`cuda`/`mps`/`cpu`),
+`estimated_bytes` vs `available_bytes`, `headroom_pct`, `reason`, and a
+concrete `suggestion` when it doesn't fit (quantize, offload to CPU, or
+pick a smaller model).
+
+!!! tip "Let the SDK pick the precision"
+    `recommend(...)` tries `bfloat16` → `int8` → `int4` on the best
+    available device and returns the **first** config that fits:
+
+    ```python
+    from tempest_fastapi_sdk.genai import recommend
+
+    best = recommend(model_id="meta-llama/Llama-3.1-8B")
+    print(best.device, best.dtype, best.fits)   # e.g. cuda int8 True
+    ```
+
+## Probing the hardware
+
+```python
+from tempest_fastapi_sdk.genai import probe_hardware
+
+hw = probe_hardware()
+print(hw.cpu_cores, hw.ram_available_bytes)
+print(hw.has_cuda, [g.name for g in hw.gpus])   # per-GPU VRAM when CUDA is present
+```
+
+`HardwareInfo` reports CPU, total/available RAM, CUDA GPUs (name +
+total/free VRAM), MPS (Apple), and free disk space. Without `psutil` or
+`torch` installed, the matching fields fall back to safe defaults (`0` /
+`False` / empty list) — nothing breaks.
+
+## Estimate without downloading weights
+
+The math is `params × bytes-per-param × overhead`. Bytes per param come
+from the precision (`float32`=4, `float16`/`bfloat16`=2, `int8`=1,
+`int4`≈0.6); the overhead (×1.25) covers activations, KV cache and
+runtime context.
+
+```python
+from tempest_fastapi_sdk.genai import estimate_model_bytes, ModelDtype
+
+gb = estimate_model_bytes(7_000_000_000, ModelDtype.INT4) / 1e9
+print(f"~{gb:.1f} GB")   # 7B in int4
+```
+
+The parameter count can be passed explicitly (`num_params=`) or read from
+the Hub by `model_id` (via `huggingface_hub`, without downloading the
+weights — safetensors metadata).
+
+## Recap
+
+- **`can_run` / `recommend`** — answer whether the host runs the model
+  and what to do if not, **before** the download.
+- **`probe_hardware`** — a CPU/RAM/GPU/disk snapshot; degrades without the
+  extras.
+- **`estimate_model_bytes` / `bytes_per_param`** — the estimation math,
+  testable and reusable.
+- Everything imports without the `[genai]` extra; install it to actually
+  run models (upcoming module slices).

tempest_fastapi_sdk-0.96.0/docs/recipes/genai.md

@@ -0,0 +1,93 @@
+# IA generativa self-hosted
+
+Rodar modelos do HuggingFace no **seu próprio hardware** — sem API
+externa, sem enviar dados pra fora. O módulo `tempest_fastapi_sdk.genai`
+está sendo entregue em fatias; esta página cobre a **primeira**: saber,
+*antes* de baixar gigabytes de pesos, se a máquina aguenta o modelo.
+
+!!! info "Roadmap do módulo"
+    - **Agora (v0.96):** `genai.hardware` — sondagem + `can_run` / `recommend`.
+    - **Em breve:** `TextGenerator` (LLM + quantização int4/int8),
+      `Embedder`, cache de modelo/resultado, `BatchScheduler`, e
+      contexto RAG (busca web + leitura de PDF) pra injetar nas LLMs.
+
+O extra `[genai]` (transformers + torch + accelerate) só é necessário pra
+**rodar** modelos. As funções de capacidade **importam sem o extra** — o
+`torch` só é usado (quando presente) pra ver a VRAM real da GPU.
+
+## "A máquina aguenta?"
+
+Carregar um modelo grande demais termina num OOM minutos depois do
+download começar. `can_run` responde antes:
+
+```python
+from tempest_fastapi_sdk.genai import can_run, ModelDtype
+
+report = can_run(model_id="Qwen/Qwen2.5-7B-Instruct", dtype=ModelDtype.BFLOAT16)
+
+if report.fits:
+    print(f"OK em {report.device} — {report.headroom_pct:.0f}% de folga")
+else:
+    print(report.reason)
+    print("Sugestão:", report.suggestion)   # ex.: "Quantize to int4 ..."
+```
+
+O `CapacityReport` traz: `fits`, `device` (`cuda`/`mps`/`cpu`),
+`estimated_bytes` vs `available_bytes`, `headroom_pct`, `reason` e uma
+`suggestion` concreta quando não cabe (quantizar, offload pra CPU, ou
+trocar de modelo).
+
+!!! tip "Deixe o SDK escolher a precisão"
+    `recommend(...)` tenta `bfloat16` → `int8` → `int4` no melhor device
+    disponível e devolve a **primeira** config que cabe:
+
+    ```python
+    from tempest_fastapi_sdk.genai import recommend
+
+    best = recommend(model_id="meta-llama/Llama-3.1-8B")
+    print(best.device, best.dtype, best.fits)   # ex.: cuda int8 True
+    ```
+
+## Sondando o hardware
+
+```python
+from tempest_fastapi_sdk.genai import probe_hardware
+
+hw = probe_hardware()
+print(hw.cpu_cores, hw.ram_available_bytes)
+print(hw.has_cuda, [g.name for g in hw.gpus])   # VRAM por GPU quando há CUDA
+```
+
+`HardwareInfo` reporta CPU, RAM total/disponível, GPUs CUDA (nome +
+VRAM total/livre), MPS (Apple) e espaço livre em disco. Sem `psutil` ou
+`torch` instalados, os campos correspondentes caem pra defaults seguros
+(`0` / `False` / lista vazia) — nada quebra.
+
+## Estimativa sem baixar pesos
+
+A conta é `nº de parâmetros × bytes por parâmetro × overhead`. Os bytes
+por parâmetro vêm da precisão (`float32`=4, `float16`/`bfloat16`=2,
+`int8`=1, `int4`≈0.6); o overhead (×1.25) cobre ativações, KV-cache e
+contexto de runtime.
+
+```python
+from tempest_fastapi_sdk.genai import estimate_model_bytes, ModelDtype
+
+gb = estimate_model_bytes(7_000_000_000, ModelDtype.INT4) / 1e9
+print(f"~{gb:.1f} GB")   # 7B em int4
+```
+
+O número de parâmetros pode vir explícito (`num_params=`) ou ser lido do
+Hub por `model_id` (via `huggingface_hub`, sem baixar os pesos —
+metadados safetensors).
+
+## Recap
+
+- **`can_run` / `recommend`** — respondem se o host roda o modelo e o que
+  fazer se não rodar, **antes** do download.
+- **`probe_hardware`** — snapshot de CPU/RAM/GPU/disco; degrada sem os
+  extras.
+- **`estimate_model_bytes` / `bytes_per_param`** — a matemática da
+  estimativa, testável e reutilizável.
+- Tudo importa sem o extra `[genai]`; instale-o pra rodar modelos de fato
+  (fatias seguintes do módulo).

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/mkdocs.yml

@@ -229,6 +229,7 @@ nav:
       - Fila e Tarefas: recipes/queue-tasks.md
       - Helpers brasileiros: recipes/br-helpers.md
       - "HTTP client (saída)": recipes/http-client.md
+      - "IA generativa self-hosted": recipes/genai.md
       - Idempotência: recipes/idempotency.md
       - Logging: recipes/logging.md
       - Métricas: recipes/metrics.md

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "tempest-fastapi-sdk"
-version = "0.95.0"
+version = "0.96.0"
 description = "Shared FastAPI building blocks: base schemas, ORM model, async repository, exceptions, pagination and settings — the conventions used across Tempest projects."
 readme = "README.md"
 requires-python = ">=3.11"
@@ -91,6 +91,16 @@ mfa = [
 vision = [
     "ort-vision-sdk>=0.4.0",
 ]
+genai = [
+    "transformers>=4.44.0",
+    "torch>=2.2.0",
+    "accelerate>=0.33.0",
+    "safetensors>=0.4.0",
+    "huggingface-hub>=0.24.0",
+]
+genai-quant = [
+    "bitsandbytes>=0.43.0",
+]
 ssr = [
     "tempestweb>=0.9.0",
 ]
@@ -300,6 +310,16 @@ module = [
     "faststream.*",
     "taskiq_redis",
     "taskiq_redis.*",
+    "typer._click",
+    "typer._click.*",
+    "torch",
+    "torch.*",
+    "transformers",
+    "transformers.*",
+    "huggingface_hub",
+    "huggingface_hub.*",
+    "accelerate",
+    "accelerate.*",
     "pyotp",
     "pyotp.*",
     "ort_vision_sdk",

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.96.0}/tempest_fastapi_sdk/__init__.py

@@ -372,7 +372,7 @@ from tempest_fastapi_sdk.websockets import (
     make_websocket_router,
 )
 
-__version__: str = "0.95.0"
+__version__: str = "0.96.0"
 
 __all__: list[str] = [
     "BASE_COLUMN_ORDER",

tempest_fastapi_sdk-0.96.0/tempest_fastapi_sdk/genai/__init__.py

@@ -0,0 +1,51 @@
+"""Self-hosted generative AI — run HuggingFace models on your own hardware.
+
+The first slice ships the **capacity check**: before you download and
+load a model, ask whether the host can actually run it. Everything here
+imports without the ``[genai]`` extra; ``torch`` / ``transformers`` are
+only needed to probe real GPUs and (later) run inference.
+"""
+
+from tempest_fastapi_sdk.genai.hardware import (
+    bytes_per_param as bytes_per_param,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    can_run as can_run,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    estimate_model_bytes as estimate_model_bytes,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    fetch_num_params as fetch_num_params,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    probe_hardware as probe_hardware,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    recommend as recommend,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    CapacityReport as CapacityReport,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    GPUInfo as GPUInfo,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    HardwareInfo as HardwareInfo,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    ModelDtype as ModelDtype,
+)
+
+__all__: list[str] = [
+    "CapacityReport",
+    "GPUInfo",
+    "HardwareInfo",
+    "ModelDtype",
+    "bytes_per_param",
+    "can_run",
+    "estimate_model_bytes",
+    "fetch_num_params",
+    "probe_hardware",
+    "recommend",
+]