tempest-fastapi-sdk 0.95.0__tar.gz → 0.97.0__tar.gz

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/CHANGELOG.md

@@ -5,6 +5,55 @@ All notable changes to **tempest-fastapi-sdk** are listed below.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.97.0] — 2026-07-05
+
+### Added
+
+- **Self-hosted GenAI — RAG context (`tempest_fastapi_sdk.genai.rag`)**,
+  slice 2: feed a local LLM with web + PDF knowledge, without shipping
+  data to a third party.
+  - **Web search** — `WebSearchBackend` Protocol + `SearxngBackend` (the
+    leviathan pattern: SearXNG JSON API over an injected `httpx` client)
+    + `WebSearch` facade. Returns `SearchResult`s.
+  - **Content extraction** — `ContentExtractor` fetches a URL and pulls
+    the clean article body via `trafilatura`; failures surface as
+    `ExtractionResult(failed=True)`, never raised.
+  - **PDF reading** — `PdfReader` (PyMuPDF, detailed reading-order
+    extraction) → `Document` (text + per-page + metadata) and overlapping
+    `Chunk`s (`read` / `chunks`).
+  - **Context assembly** — `build_context(question, sources)` renders
+    `SearchResult`s and/or `Chunk`s into one prompt-ready, source-labeled
+    block (mix web + PDF, optional per-source truncation).
+  - New `[genai-rag]` extra (httpx + trafilatura + pymupdf); everything
+    imports lazily without it. Import from `tempest_fastapi_sdk.genai.rag`.
+
+## [0.96.0] — 2026-07-05
+
+### Added
+
+- **Self-hosted GenAI — capacity check (`tempest_fastapi_sdk.genai`)**,
+  the first slice of running HuggingFace models on your own hardware.
+  Before downloading gigabytes of weights, check whether the host can run
+  a model:
+  - `probe_hardware()` → `HardwareInfo` (CPU, total/available RAM, CUDA
+    GPUs with per-device VRAM, Apple MPS, free disk). Degrades gracefully
+    without `psutil` / `torch`.
+  - `can_run(...)` → `CapacityReport` (`fits`, chosen `device`, estimated
+    vs available bytes, `headroom_pct`, and a concrete `suggestion` when
+    it doesn't fit — quantize, offload, or pick a smaller model).
+  - `recommend(...)` picks the first precision (`bfloat16` → `int8` →
+    `int4`) that fits.
+  - `estimate_model_bytes` / `bytes_per_param` (the estimation math) and
+    `fetch_num_params` (reads a model's parameter count from the Hub via
+    `huggingface_hub`, without downloading weights). `ModelDtype` enum.
+  - New `[genai]` extra (transformers + torch + accelerate + safetensors +
+    huggingface-hub) and `[genai-quant]` (bitsandbytes). The capacity
+    functions import **without** the extra — `torch` is only used to probe
+    real GPUs. Import from `tempest_fastapi_sdk.genai`.
+  - Upcoming slices: `TextGenerator` (+ quantization), `Embedder`,
+    model/result caching, `BatchScheduler`, and RAG context (web search +
+    PDF reading).
+
 ## [0.95.0] — 2026-07-05
 
 ### Added

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/CLAUDE.md

@@ -129,6 +129,25 @@ The SDK currently covers (Sep 2025+, post-v0.31.x):
   `ort-vision-sdk`: lazy `Detector`/`Classifier`/`Segmenter` + prediction
   schemas + `to_detection_schemas`/`to_classification_schema`/
   `to_segmentation_schemas` mappers.
+- **GenAI self-hosted** (`[genai]` extra: transformers+torch+accelerate;
+  `[genai-quant]` = bitsandbytes) — `tempest_fastapi_sdk.genai`, delivered
+  in slices. **Shipped (v0.96):** hardware capacity check — `probe_hardware`
+  → `HardwareInfo` (CPU/RAM/CUDA-VRAM/MPS/disk, degrades without
+  psutil/torch), `can_run`/`recommend` → `CapacityReport` (fits? device,
+  estimate vs available, suggestion to quantize/offload), `estimate_model_bytes`/
+  `bytes_per_param`/`fetch_num_params` (Hub metadata, no weight download),
+  `ModelDtype`. Capacity fns import WITHOUT the extra. **Planned slices:**
+  `TextGenerator` (LLM + int4/int8 quant, lazy load + idle-unload),
+  `Embedder`, model/result cache, `BatchScheduler` (coalesce concurrent
+  calls). Classes-only (no bundled router). Submodule import like
+  queue/tasks/vision. **Shipped (v0.97) — RAG context**
+  (`tempest_fastapi_sdk.genai.rag`, `[genai-rag]` extra = httpx +
+  trafilatura + pymupdf): `WebSearchBackend` Protocol + `SearxngBackend`
+  (SearXNG JSON API, leviathan pattern) + `WebSearch`; `ContentExtractor`
+  (trafilatura, failures never raise); `PdfReader` (PyMuPDF detailed
+  extraction → `Document`/`Chunk`, `read`/`chunks` with overlap);
+  `build_context(question, sources)` → prompt block mixing web +
+  PDF. All import lazily.
 - **SSR** (`[ssr]` extra) — `tempest_fastapi_sdk.ssr`: typed Python
   pages rendered to HTML via `tempestweb`'s `render_to_html` /
   `render_document`. `Page` (typed `Component` base — `body()` +

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tempest-fastapi-sdk
-Version: 0.95.0
+Version: 0.97.0
 Summary: Shared FastAPI building blocks: base schemas, ORM model, async repository, exceptions, pagination and settings — the conventions used across Tempest projects.
 Project-URL: Homepage, https://github.com/mauriciobenjamin700/tempest-fastapi-sdk
 Project-URL: Repository, https://github.com/mauriciobenjamin700/tempest-fastapi-sdk
@@ -69,6 +69,18 @@ Provides-Extra: email
 Requires-Dist: aiosmtplib>=4.0.0; extra == 'email'
 Requires-Dist: email-validator>=2.0.0; extra == 'email'
 Requires-Dist: jinja2>=3.1.0; extra == 'email'
+Provides-Extra: genai
+Requires-Dist: accelerate>=0.33.0; extra == 'genai'
+Requires-Dist: huggingface-hub>=0.24.0; extra == 'genai'
+Requires-Dist: safetensors>=0.4.0; extra == 'genai'
+Requires-Dist: torch>=2.2.0; extra == 'genai'
+Requires-Dist: transformers>=4.44.0; extra == 'genai'
+Provides-Extra: genai-quant
+Requires-Dist: bitsandbytes>=0.43.0; extra == 'genai-quant'
+Provides-Extra: genai-rag
+Requires-Dist: httpx>=0.28.0; extra == 'genai-rag'
+Requires-Dist: pymupdf>=1.24.0; extra == 'genai-rag'
+Requires-Dist: trafilatura>=1.12.0; extra == 'genai-rag'
 Provides-Extra: http
 Requires-Dist: httpx>=0.28.0; extra == 'http'
 Provides-Extra: metrics
@@ -226,6 +238,8 @@ Feature-rich helpers pull in third-party dependencies that you only need when yo
 | `[prometheus]` | `prometheus-client` | `PrometheusMiddleware`, `make_prometheus_router`, `make_prometheus_registry` |
 | `[mfa]` | `pyotp` | `TOTPHelper` + MFA/2FA endpoints on the bundled auth flow |
 | `[vision]` | `ort-vision-sdk` | `Detector` / `Classifier` / `Segmenter` (ONNX) + prediction schemas |
+| `[genai]` (+ `[genai-quant]`) | `transformers`, `torch`, `accelerate`, `safetensors`, `huggingface-hub` (+ `bitsandbytes`) | Self-hosted GenAI — hardware capacity check (`probe_hardware` / `can_run` / `recommend`); model runners upcoming |
+| `[genai-rag]` | `httpx`, `trafilatura`, `pymupdf` | RAG context for local LLMs — SearXNG web search, page extraction, PDF reading, `build_context` |
 | `[ssr]` | `tempestweb` | `Page`, `html_response`, `make_htmx_router` — typed Python pages rendered to HTML |
 | `[otel]` | `opentelemetry-sdk` + OTLP/gRPC exporter + FastAPI/SQLAlchemy/httpx instrumentors | `setup_tracing` — distributed tracing |
 | `[sqlite]` | `aiosqlite` | SQLite async driver for `sqlite+aiosqlite://` URLs (dev default) |
@@ -268,11 +282,13 @@ Since `0.7.1` every optional dependency is imported lazily at first instantiatio
 | `tempest_fastapi_sdk.utils.storage_backends` *(extra: `[upload]`)* | `UploadStorage` protocol, `LocalUploadStorage`, `MinIOUploadStorage`, `UploadResult`, `ContentValidator` |
 | `tempest_fastapi_sdk.tasks` *(extra: `[tasks]`)* | `TaskQueue` (typed TaskIQ facade — `.rabbitmq`/`.redis`/`.memory`, `@tq.task` → `Task.enqueue`/`.run`, `@tq.cron`/`@tq.interval`, `start_scheduler`), `Task`; class-based tasks (`TaskDef` + `@task_method` + `TaskQueue.register`); cron helpers (`Cron` / `CronOffset` / `Weekday` + `daily`/`weekdays`/`every_n_minutes`/… builders); `AsyncTaskBrokerManager` / `AsyncTaskScheduler` (legacy wrappers) |
 | `tempest_fastapi_sdk.vision` *(extra: `[vision]`)* | `Detector`, `Classifier`, `Segmenter` (ONNX, lazy), `DetectionSchema`/`ClassificationSchema`/`SegmentationSchema`/`BoundingBoxSchema`/`ClassProbabilitySchema`, `to_detection_schemas`/`to_classification_schema`/`to_segmentation_schemas` |
+| `tempest_fastapi_sdk.genai` *(extra: `[genai]`; capacity check imports without it)* | Self-hosted GenAI capacity check: `probe_hardware`/`HardwareInfo`, `can_run`/`recommend`/`CapacityReport`, `estimate_model_bytes`/`bytes_per_param`/`fetch_num_params`, `ModelDtype` — model runners (`TextGenerator`, `Embedder`), caching and batching in upcoming slices |
+| `tempest_fastapi_sdk.genai.rag` *(extra: `[genai-rag]`)* | RAG context for local LLMs: `WebSearch`/`SearxngBackend`/`WebSearchBackend` + `SearchResult`, `ContentExtractor` (trafilatura), `PdfReader` (PyMuPDF) + `Document`/`Chunk`/`PdfPage`, `build_context` |
 | `tempest_fastapi_sdk.ssr` *(extra: `[ssr]`)* | `Page` (typed component base), `html_response` (widget tree → `HTMLResponse`, full document or HTMX fragment), `make_htmx_router` (serves bundled HTMX locally, no CDN) |
 | `tempest_fastapi_sdk.utils` | `to_utc`, `utcnow`, `modify_dict`, `LogUtils`, `AttemptThrottle`/`ThrottleBackend`/`ThrottleStatus`, `generate_opaque_token`/`hash_opaque_token`/`verify_opaque_token`, `get_client_ip`/`get_client_ip_from_scope`, `PasswordUtils` *(extra: `[auth]`)*, `JWTUtils` *(extra: `[auth]`)*, `TOTPHelper` *(extra: `[mfa]`)*, `EmailUtils` *(extra: `[email]`)*, `UploadUtils`/`sniff_mime` *(extra: `[upload]`)*, `DownloadUtils`/`build_content_disposition` *(no extra)*, `FileStoreUtils` — unified upload+download+presign facade *(extra: `[upload]` local / `[minio]` MinIO)*, `MetricsUtils`/`CPUMetrics`/`MemoryMetrics`/`DiskMetrics`/`GPUMetrics`/`SystemMetrics` *(extra: `[metrics]`)*, validated field types (`PositiveIntField`, `NonNegativeIntField`, `CentsField`, `PortField`, `PositiveFloatField`, `NonNegativeFloatField`, `PercentField`, `RatioField`, `LatitudeField`, `LongitudeField`, `PriceField`, `NonEmptyStrField`, `SlugField`, `HexColorField`), BR regex helpers (`CPFField`, `CNPJField`, `CPFOrCNPJField`, `PhoneBRField`, `CEPField`, `PixKeyField` — old names without the suffix kept as deprecated aliases — `PixKeyType`/`detect_pix_key_type`/`is_valid_pix_key`, `is_valid_*`, `normalize_*`, `only_digits`, `*_PATTERN`), BR states/cities (`UF`, `Region`, `StateBR`, `CityBR`, `ChoiceBR`, `UFField`, `CityNameField`, `list_states`, `get_state`, `cities_by_uf`, `states_by_region`, `uf_choices`/`region_choices`/`city_choices`, `is_valid_uf`/`normalize_uf`, `is_valid_city`/`normalize_city`) |
 | `tempest_fastapi_sdk.cli` | `tempest` console script — `new <name>` (scaffold layered service), `lint` / `format` / `fmt-check` / `type` / `test` / `check` (run preferred quality gates), `version` / `--version` |
 
-Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
+Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision`, `tempest_fastapi_sdk.genai` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
 
 ---
 

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/README.md

@@ -117,6 +117,8 @@ Feature-rich helpers pull in third-party dependencies that you only need when yo
 | `[prometheus]` | `prometheus-client` | `PrometheusMiddleware`, `make_prometheus_router`, `make_prometheus_registry` |
 | `[mfa]` | `pyotp` | `TOTPHelper` + MFA/2FA endpoints on the bundled auth flow |
 | `[vision]` | `ort-vision-sdk` | `Detector` / `Classifier` / `Segmenter` (ONNX) + prediction schemas |
+| `[genai]` (+ `[genai-quant]`) | `transformers`, `torch`, `accelerate`, `safetensors`, `huggingface-hub` (+ `bitsandbytes`) | Self-hosted GenAI — hardware capacity check (`probe_hardware` / `can_run` / `recommend`); model runners upcoming |
+| `[genai-rag]` | `httpx`, `trafilatura`, `pymupdf` | RAG context for local LLMs — SearXNG web search, page extraction, PDF reading, `build_context` |
 | `[ssr]` | `tempestweb` | `Page`, `html_response`, `make_htmx_router` — typed Python pages rendered to HTML |
 | `[otel]` | `opentelemetry-sdk` + OTLP/gRPC exporter + FastAPI/SQLAlchemy/httpx instrumentors | `setup_tracing` — distributed tracing |
 | `[sqlite]` | `aiosqlite` | SQLite async driver for `sqlite+aiosqlite://` URLs (dev default) |
@@ -159,11 +161,13 @@ Since `0.7.1` every optional dependency is imported lazily at first instantiatio
 | `tempest_fastapi_sdk.utils.storage_backends` *(extra: `[upload]`)* | `UploadStorage` protocol, `LocalUploadStorage`, `MinIOUploadStorage`, `UploadResult`, `ContentValidator` |
 | `tempest_fastapi_sdk.tasks` *(extra: `[tasks]`)* | `TaskQueue` (typed TaskIQ facade — `.rabbitmq`/`.redis`/`.memory`, `@tq.task` → `Task.enqueue`/`.run`, `@tq.cron`/`@tq.interval`, `start_scheduler`), `Task`; class-based tasks (`TaskDef` + `@task_method` + `TaskQueue.register`); cron helpers (`Cron` / `CronOffset` / `Weekday` + `daily`/`weekdays`/`every_n_minutes`/… builders); `AsyncTaskBrokerManager` / `AsyncTaskScheduler` (legacy wrappers) |
 | `tempest_fastapi_sdk.vision` *(extra: `[vision]`)* | `Detector`, `Classifier`, `Segmenter` (ONNX, lazy), `DetectionSchema`/`ClassificationSchema`/`SegmentationSchema`/`BoundingBoxSchema`/`ClassProbabilitySchema`, `to_detection_schemas`/`to_classification_schema`/`to_segmentation_schemas` |
+| `tempest_fastapi_sdk.genai` *(extra: `[genai]`; capacity check imports without it)* | Self-hosted GenAI capacity check: `probe_hardware`/`HardwareInfo`, `can_run`/`recommend`/`CapacityReport`, `estimate_model_bytes`/`bytes_per_param`/`fetch_num_params`, `ModelDtype` — model runners (`TextGenerator`, `Embedder`), caching and batching in upcoming slices |
+| `tempest_fastapi_sdk.genai.rag` *(extra: `[genai-rag]`)* | RAG context for local LLMs: `WebSearch`/`SearxngBackend`/`WebSearchBackend` + `SearchResult`, `ContentExtractor` (trafilatura), `PdfReader` (PyMuPDF) + `Document`/`Chunk`/`PdfPage`, `build_context` |
 | `tempest_fastapi_sdk.ssr` *(extra: `[ssr]`)* | `Page` (typed component base), `html_response` (widget tree → `HTMLResponse`, full document or HTMX fragment), `make_htmx_router` (serves bundled HTMX locally, no CDN) |
 | `tempest_fastapi_sdk.utils` | `to_utc`, `utcnow`, `modify_dict`, `LogUtils`, `AttemptThrottle`/`ThrottleBackend`/`ThrottleStatus`, `generate_opaque_token`/`hash_opaque_token`/`verify_opaque_token`, `get_client_ip`/`get_client_ip_from_scope`, `PasswordUtils` *(extra: `[auth]`)*, `JWTUtils` *(extra: `[auth]`)*, `TOTPHelper` *(extra: `[mfa]`)*, `EmailUtils` *(extra: `[email]`)*, `UploadUtils`/`sniff_mime` *(extra: `[upload]`)*, `DownloadUtils`/`build_content_disposition` *(no extra)*, `FileStoreUtils` — unified upload+download+presign facade *(extra: `[upload]` local / `[minio]` MinIO)*, `MetricsUtils`/`CPUMetrics`/`MemoryMetrics`/`DiskMetrics`/`GPUMetrics`/`SystemMetrics` *(extra: `[metrics]`)*, validated field types (`PositiveIntField`, `NonNegativeIntField`, `CentsField`, `PortField`, `PositiveFloatField`, `NonNegativeFloatField`, `PercentField`, `RatioField`, `LatitudeField`, `LongitudeField`, `PriceField`, `NonEmptyStrField`, `SlugField`, `HexColorField`), BR regex helpers (`CPFField`, `CNPJField`, `CPFOrCNPJField`, `PhoneBRField`, `CEPField`, `PixKeyField` — old names without the suffix kept as deprecated aliases — `PixKeyType`/`detect_pix_key_type`/`is_valid_pix_key`, `is_valid_*`, `normalize_*`, `only_digits`, `*_PATTERN`), BR states/cities (`UF`, `Region`, `StateBR`, `CityBR`, `ChoiceBR`, `UFField`, `CityNameField`, `list_states`, `get_state`, `cities_by_uf`, `states_by_region`, `uf_choices`/`region_choices`/`city_choices`, `is_valid_uf`/`normalize_uf`, `is_valid_city`/`normalize_city`) |
 | `tempest_fastapi_sdk.cli` | `tempest` console script — `new <name>` (scaffold layered service), `lint` / `format` / `fmt-check` / `type` / `test` / `check` (run preferred quality gates), `version` / `--version` |
 
-Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
+Core primitives are re-exported from `tempest_fastapi_sdk` at the top level — `from tempest_fastapi_sdk import BaseModel, BaseRepository, AppException` always works. The extras-gated managers in `tempest_fastapi_sdk.cache`, `tempest_fastapi_sdk.queue`, `tempest_fastapi_sdk.tasks`, `tempest_fastapi_sdk.vision`, `tempest_fastapi_sdk.genai` and `tempest_fastapi_sdk.ssr` must be imported from their own submodule (`from tempest_fastapi_sdk.queue import MessageBroker`).
 
 ---
 

tempest_fastapi_sdk-0.97.0/docs/recipes/genai.en.md

@@ -0,0 +1,167 @@
+# Self-hosted generative AI
+
+Run HuggingFace models on **your own hardware** — no external API, no data
+leaving your servers. The `tempest_fastapi_sdk.genai` module ships in
+slices; this page covers the **first**: knowing, *before* you download
+gigabytes of weights, whether the machine can handle the model.
+
+!!! info "Module roadmap"
+    - **v0.96:** `genai.hardware` — probing + `can_run` / `recommend`.
+    - **v0.97:** `genai.rag` — RAG context (SearXNG web search + PDF
+      reading) to inject into LLMs (this page, [RAG context](#rag-context)).
+    - **Coming:** `TextGenerator` (LLM + int4/int8 quantization),
+      `Embedder`, model/result caching, `BatchScheduler`.
+
+The `[genai]` extra (transformers + torch + accelerate) is only needed to
+**run** models. The capacity functions **import without the extra** —
+`torch` is only used (when present) to read real GPU VRAM.
+
+## "Can the machine handle it?"
+
+Loading a too-large model ends in an OOM minutes into the download.
+`can_run` answers first:
+
+```python
+from tempest_fastapi_sdk.genai import can_run, ModelDtype
+
+report = can_run(model_id="Qwen/Qwen2.5-7B-Instruct", dtype=ModelDtype.BFLOAT16)
+
+if report.fits:
+    print(f"OK on {report.device} — {report.headroom_pct:.0f}% headroom")
+else:
+    print(report.reason)
+    print("Suggestion:", report.suggestion)   # e.g. "Quantize to int4 ..."
+```
+
+`CapacityReport` carries: `fits`, `device` (`cuda`/`mps`/`cpu`),
+`estimated_bytes` vs `available_bytes`, `headroom_pct`, `reason`, and a
+concrete `suggestion` when it doesn't fit (quantize, offload to CPU, or
+pick a smaller model).
+
+!!! tip "Let the SDK pick the precision"
+    `recommend(...)` tries `bfloat16` → `int8` → `int4` on the best
+    available device and returns the **first** config that fits:
+
+    ```python
+    from tempest_fastapi_sdk.genai import recommend
+
+    best = recommend(model_id="meta-llama/Llama-3.1-8B")
+    print(best.device, best.dtype, best.fits)   # e.g. cuda int8 True
+    ```
+
+## Probing the hardware
+
+```python
+from tempest_fastapi_sdk.genai import probe_hardware
+
+hw = probe_hardware()
+print(hw.cpu_cores, hw.ram_available_bytes)
+print(hw.has_cuda, [g.name for g in hw.gpus])   # per-GPU VRAM when CUDA is present
+```
+
+`HardwareInfo` reports CPU, total/available RAM, CUDA GPUs (name +
+total/free VRAM), MPS (Apple), and free disk space. Without `psutil` or
+`torch` installed, the matching fields fall back to safe defaults (`0` /
+`False` / empty list) — nothing breaks.
+
+## Estimate without downloading weights
+
+The math is `params × bytes-per-param × overhead`. Bytes per param come
+from the precision (`float32`=4, `float16`/`bfloat16`=2, `int8`=1,
+`int4`≈0.6); the overhead (×1.25) covers activations, KV cache and
+runtime context.
+
+```python
+from tempest_fastapi_sdk.genai import estimate_model_bytes, ModelDtype
+
+gb = estimate_model_bytes(7_000_000_000, ModelDtype.INT4) / 1e9
+print(f"~{gb:.1f} GB")   # 7B in int4
+```
+
+The parameter count can be passed explicitly (`num_params=`) or read from
+the Hub by `model_id` (via `huggingface_hub`, without downloading the
+weights — safetensors metadata).
+
+## RAG context
+
+A local LLM only knows what it trained on. For current, grounded answers,
+inject context: `tempest_fastapi_sdk.genai.rag` searches the web
+(self-hosted SearXNG), extracts page bodies, reads PDFs, and assembles a
+prompt-ready block — without sending data outside. Needs the `[genai-rag]`
+extra (httpx + trafilatura + pymupdf); the pieces import lazily.
+
+### Web search (SearXNG)
+
+```python
+import httpx
+from tempest_fastapi_sdk.genai.rag import SearxngBackend, WebSearch, build_context
+
+client = httpx.AsyncClient()
+search = WebSearch(SearxngBackend("http://localhost:8080", http_client=client))
+
+results = await search.search("what is PIX?", max_results=5)   # list[SearchResult]
+context = build_context("what is PIX?", results, long_text=False, max_chars=2000)
+# -> a string ready to inject into your TextGenerator prompt
+```
+
+The backend is a `Protocol` (`WebSearchBackend`) — swap SearXNG for another
+provider without touching call sites. The `httpx.AsyncClient` is injected
+(pool reuse; wire it in the FastAPI lifespan).
+
+### Extract page bodies
+
+Search snippets are thin. To give the LLM ground truth, fetch each page
+and extract the clean text (via `trafilatura`):
+
+```python
+from tempest_fastapi_sdk.genai.rag import ContentExtractor
+
+extractor = ContentExtractor(http_client=client)
+for result in results:
+    outcome = await extractor.extract(result.url)
+    result.content = outcome.text          # "" on failure; outcome.failed marks it
+context = build_context("what is PIX?", results)   # now with full bodies
+```
+
+Failures (timeout, 4xx/5xx, empty page) **never** raise — they come back
+as `ExtractionResult(text="", failed=True)`, so no source is silently
+dropped.
+
+### Read PDFs (knowledge base)
+
+`PdfReader` (PyMuPDF — detailed, reading-order extraction) turns PDF paths
+into text and prompt/embedding-ready chunks:
+
+```python
+from tempest_fastapi_sdk.genai.rag import PdfReader, build_context
+
+reader = PdfReader()
+doc = reader.read("/kb/manual.pdf")             # Document: text + pages + metadata
+chunks = reader.chunks("/kb/manual.pdf", max_chars=2000, overlap=200)
+
+context = build_context("how to refund?", chunks)   # cites "file (page N)"
+```
+
+`chunks(..., overlap=200)` shares characters between neighbors so a fact on
+a boundary isn't cut in half; `per_page=True` (default) keeps each chunk on
+a single page and records its page number.
+
+!!! tip "Mix web + PDF in one context"
+    `build_context` accepts `SearchResult` and `Chunk` in the same list —
+    it delimits each source with `---` and labels the origin (URL or `file
+    (page N)`) so the LLM can cite. Pass `long_text=False` to truncate each
+    source to `max_chars`.
+
+## Recap
+
+- **`can_run` / `recommend`** — answer whether the host runs the model
+  and what to do if not, **before** the download.
+- **RAG** — `WebSearch`/`SearxngBackend` (search), `ContentExtractor`
+  (page bodies), `PdfReader` (PDF → text/chunks) and `build_context`
+  (prompt block), all under the `[genai-rag]` extra.
+- **`probe_hardware`** — a CPU/RAM/GPU/disk snapshot; degrades without the
+  extras.
+- **`estimate_model_bytes` / `bytes_per_param`** — the estimation math,
+  testable and reusable.
+- Everything imports without the `[genai]` extra; install it to actually
+  run models (upcoming module slices).

tempest_fastapi_sdk-0.97.0/docs/recipes/genai.md

@@ -0,0 +1,168 @@
+# IA generativa self-hosted
+
+Rodar modelos do HuggingFace no **seu próprio hardware** — sem API
+externa, sem enviar dados pra fora. O módulo `tempest_fastapi_sdk.genai`
+está sendo entregue em fatias; esta página cobre a **primeira**: saber,
+*antes* de baixar gigabytes de pesos, se a máquina aguenta o modelo.
+
+!!! info "Roadmap do módulo"
+    - **v0.96:** `genai.hardware` — sondagem + `can_run` / `recommend`.
+    - **v0.97:** `genai.rag` — contexto RAG (busca web SearXNG + leitura de
+      PDF) pra injetar nas LLMs (esta página, seção [Contexto RAG](#contexto-rag)).
+    - **Em breve:** `TextGenerator` (LLM + quantização int4/int8),
+      `Embedder`, cache de modelo/resultado, `BatchScheduler`.
+
+O extra `[genai]` (transformers + torch + accelerate) só é necessário pra
+**rodar** modelos. As funções de capacidade **importam sem o extra** — o
+`torch` só é usado (quando presente) pra ver a VRAM real da GPU.
+
+## "A máquina aguenta?"
+
+Carregar um modelo grande demais termina num OOM minutos depois do
+download começar. `can_run` responde antes:
+
+```python
+from tempest_fastapi_sdk.genai import can_run, ModelDtype
+
+report = can_run(model_id="Qwen/Qwen2.5-7B-Instruct", dtype=ModelDtype.BFLOAT16)
+
+if report.fits:
+    print(f"OK em {report.device} — {report.headroom_pct:.0f}% de folga")
+else:
+    print(report.reason)
+    print("Sugestão:", report.suggestion)   # ex.: "Quantize to int4 ..."
+```
+
+O `CapacityReport` traz: `fits`, `device` (`cuda`/`mps`/`cpu`),
+`estimated_bytes` vs `available_bytes`, `headroom_pct`, `reason` e uma
+`suggestion` concreta quando não cabe (quantizar, offload pra CPU, ou
+trocar de modelo).
+
+!!! tip "Deixe o SDK escolher a precisão"
+    `recommend(...)` tenta `bfloat16` → `int8` → `int4` no melhor device
+    disponível e devolve a **primeira** config que cabe:
+
+    ```python
+    from tempest_fastapi_sdk.genai import recommend
+
+    best = recommend(model_id="meta-llama/Llama-3.1-8B")
+    print(best.device, best.dtype, best.fits)   # ex.: cuda int8 True
+    ```
+
+## Sondando o hardware
+
+```python
+from tempest_fastapi_sdk.genai import probe_hardware
+
+hw = probe_hardware()
+print(hw.cpu_cores, hw.ram_available_bytes)
+print(hw.has_cuda, [g.name for g in hw.gpus])   # VRAM por GPU quando há CUDA
+```
+
+`HardwareInfo` reporta CPU, RAM total/disponível, GPUs CUDA (nome +
+VRAM total/livre), MPS (Apple) e espaço livre em disco. Sem `psutil` ou
+`torch` instalados, os campos correspondentes caem pra defaults seguros
+(`0` / `False` / lista vazia) — nada quebra.
+
+## Estimativa sem baixar pesos
+
+A conta é `nº de parâmetros × bytes por parâmetro × overhead`. Os bytes
+por parâmetro vêm da precisão (`float32`=4, `float16`/`bfloat16`=2,
+`int8`=1, `int4`≈0.6); o overhead (×1.25) cobre ativações, KV-cache e
+contexto de runtime.
+
+```python
+from tempest_fastapi_sdk.genai import estimate_model_bytes, ModelDtype
+
+gb = estimate_model_bytes(7_000_000_000, ModelDtype.INT4) / 1e9
+print(f"~{gb:.1f} GB")   # 7B em int4
+```
+
+O número de parâmetros pode vir explícito (`num_params=`) ou ser lido do
+Hub por `model_id` (via `huggingface_hub`, sem baixar os pesos —
+metadados safetensors).
+
+## Contexto RAG
+
+Uma LLM local só sabe o que treinou. Pra respostas atuais e fundamentadas,
+injete contexto: `tempest_fastapi_sdk.genai.rag` busca na web (SearXNG
+self-hosted), extrai o corpo das páginas, lê PDFs e monta um bloco pronto
+pro prompt — tudo sem enviar dados pra fora. Requer o extra `[genai-rag]`
+(httpx + trafilatura + pymupdf); as peças importam preguiçosamente.
+
+### Busca web (SearXNG)
+
+```python
+import httpx
+from tempest_fastapi_sdk.genai.rag import SearxngBackend, WebSearch, build_context
+
+client = httpx.AsyncClient()
+search = WebSearch(SearxngBackend("http://localhost:8080", http_client=client))
+
+results = await search.search("o que é PIX?", max_results=5)   # list[SearchResult]
+context = build_context("o que é PIX?", results, long_text=False, max_chars=2000)
+# -> string pronta pra injetar no prompt do seu TextGenerator
+```
+
+O backend é um `Protocol` (`WebSearchBackend`) — troque o SearXNG por
+outro provedor sem mexer no call site. O `httpx.AsyncClient` é injetado
+(reaproveita o pool; ligue no lifespan do FastAPI).
+
+### Extrair o corpo das páginas
+
+O snippet do buscador é raso. Pra dar "verdade" pra LLM, busque cada
+página e extraia o texto limpo (via `trafilatura`):
+
+```python
+from tempest_fastapi_sdk.genai.rag import ContentExtractor
+
+extractor = ContentExtractor(http_client=client)
+for result in results:
+    outcome = await extractor.extract(result.url)
+    result.content = outcome.text          # "" quando falha; outcome.failed marca
+context = build_context("o que é PIX?", results)   # agora com corpo completo
+```
+
+Falhas (timeout, 4xx/5xx, página sem corpo) **nunca** levantam — voltam
+como `ExtractionResult(text="", failed=True)`, então nenhuma fonte some
+silenciosamente.
+
+### Ler PDFs (base de conhecimento)
+
+`PdfReader` (PyMuPDF — extração detalhada, ordem de leitura) transforma
+caminhos de PDF em texto e em chunks prontos pra prompt ou índice de
+embeddings:
+
+```python
+from tempest_fastapi_sdk.genai.rag import PdfReader, build_context
+
+reader = PdfReader()
+doc = reader.read("/base/manual.pdf")            # Document: text + pages + metadata
+chunks = reader.chunks("/base/manual.pdf", max_chars=2000, overlap=200)
+
+context = build_context("como estornar?", chunks)   # cita "arquivo (page N)"
+```
+
+`chunks(..., overlap=200)` compartilha caracteres entre pedaços vizinhos,
+pra um fato na fronteira não ser cortado ao meio; `per_page=True` (padrão)
+mantém cada chunk numa página só, carregando o número dela.
+
+!!! tip "Misture web + PDF no mesmo contexto"
+    `build_context` aceita `SearchResult` e `Chunk` na mesma lista —
+    delimita cada fonte com `---` e rotula a origem (URL ou `arquivo
+    (page N)`), pra LLM citar. Passe `long_text=False` pra truncar cada
+    fonte a `max_chars`.
+
+## Recap
+
+- **`can_run` / `recommend`** — respondem se o host roda o modelo e o que
+  fazer se não rodar, **antes** do download.
+- **RAG** — `WebSearch`/`SearxngBackend` (busca), `ContentExtractor`
+  (corpo das páginas), `PdfReader` (PDF → texto/chunks) e `build_context`
+  (bloco pro prompt), todos sob o extra `[genai-rag]`.
+- **`probe_hardware`** — snapshot de CPU/RAM/GPU/disco; degrada sem os
+  extras.
+- **`estimate_model_bytes` / `bytes_per_param`** — a matemática da
+  estimativa, testável e reutilizável.
+- Tudo importa sem o extra `[genai]`; instale-o pra rodar modelos de fato
+  (fatias seguintes do módulo).

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/mkdocs.yml

@@ -229,6 +229,7 @@ nav:
       - Fila e Tarefas: recipes/queue-tasks.md
       - Helpers brasileiros: recipes/br-helpers.md
       - "HTTP client (saída)": recipes/http-client.md
+      - "IA generativa self-hosted": recipes/genai.md
       - Idempotência: recipes/idempotency.md
       - Logging: recipes/logging.md
       - Métricas: recipes/metrics.md

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "tempest-fastapi-sdk"
-version = "0.95.0"
+version = "0.97.0"
 description = "Shared FastAPI building blocks: base schemas, ORM model, async repository, exceptions, pagination and settings — the conventions used across Tempest projects."
 readme = "README.md"
 requires-python = ">=3.11"
@@ -91,6 +91,21 @@ mfa = [
 vision = [
     "ort-vision-sdk>=0.4.0",
 ]
+genai = [
+    "transformers>=4.44.0",
+    "torch>=2.2.0",
+    "accelerate>=0.33.0",
+    "safetensors>=0.4.0",
+    "huggingface-hub>=0.24.0",
+]
+genai-quant = [
+    "bitsandbytes>=0.43.0",
+]
+genai-rag = [
+    "httpx>=0.28.0",
+    "trafilatura>=1.12.0",
+    "pymupdf>=1.24.0",
+]
 ssr = [
     "tempestweb>=0.9.0",
 ]
@@ -195,6 +210,8 @@ dev = [
     "nvidia-ml-py>=12.560.30",
     "redis>=5.0.0",
     "ruff>=0.8.0",
+    "trafilatura>=1.12.0",
+    "pymupdf>=1.24.0",
     "faststream[rabbit]>=0.7.1",
     "taskiq>=0.11.0",
     "taskiq-aio-pika>=0.4.0",
@@ -300,6 +317,20 @@ module = [
     "faststream.*",
     "taskiq_redis",
     "taskiq_redis.*",
+    "typer._click",
+    "typer._click.*",
+    "torch",
+    "torch.*",
+    "transformers",
+    "transformers.*",
+    "huggingface_hub",
+    "huggingface_hub.*",
+    "accelerate",
+    "accelerate.*",
+    "trafilatura",
+    "trafilatura.*",
+    "pymupdf",
+    "fitz",
     "pyotp",
     "pyotp.*",
     "ort_vision_sdk",

{tempest_fastapi_sdk-0.95.0 → tempest_fastapi_sdk-0.97.0}/tempest_fastapi_sdk/__init__.py

@@ -372,7 +372,7 @@ from tempest_fastapi_sdk.websockets import (
     make_websocket_router,
 )
 
-__version__: str = "0.95.0"
+__version__: str = "0.97.0"
 
 __all__: list[str] = [
     "BASE_COLUMN_ORDER",

tempest_fastapi_sdk-0.97.0/tempest_fastapi_sdk/genai/__init__.py

@@ -0,0 +1,51 @@
+"""Self-hosted generative AI — run HuggingFace models on your own hardware.
+
+The first slice ships the **capacity check**: before you download and
+load a model, ask whether the host can actually run it. Everything here
+imports without the ``[genai]`` extra; ``torch`` / ``transformers`` are
+only needed to probe real GPUs and (later) run inference.
+"""
+
+from tempest_fastapi_sdk.genai.hardware import (
+    bytes_per_param as bytes_per_param,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    can_run as can_run,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    estimate_model_bytes as estimate_model_bytes,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    fetch_num_params as fetch_num_params,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    probe_hardware as probe_hardware,
+)
+from tempest_fastapi_sdk.genai.hardware import (
+    recommend as recommend,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    CapacityReport as CapacityReport,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    GPUInfo as GPUInfo,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    HardwareInfo as HardwareInfo,
+)
+from tempest_fastapi_sdk.genai.schemas import (
+    ModelDtype as ModelDtype,
+)
+
+__all__: list[str] = [
+    "CapacityReport",
+    "GPUInfo",
+    "HardwareInfo",
+    "ModelDtype",
+    "bytes_per_param",
+    "can_run",
+    "estimate_model_bytes",
+    "fetch_num_params",
+    "probe_hardware",
+    "recommend",
+]