ff-aitoolkit 0.2.0__tar.gz

ff_aitoolkit-0.2.0/.gitignore

@@ -0,0 +1,28 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+*.egg
+build/
+dist/
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# Local secrets / config
+.env
+
+# Tooling caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Editors / OS
+.vscode/
+.idea/
+.DS_Store

ff_aitoolkit-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Faisal Fida
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ff_aitoolkit-0.2.0/PKG-INFO

@@ -0,0 +1,159 @@
+Metadata-Version: 2.4
+Name: ff-aitoolkit
+Version: 0.2.0
+Summary: Centralized AI clients (LLM, embeddings, STT, TTS, RAG) for self-hosted OpenAI-compatible GPU services.
+Project-URL: Homepage, https://github.com/CNIT-Organization/aitoolkit
+Project-URL: Repository, https://github.com/CNIT-Organization/aitoolkit
+Project-URL: Issues, https://github.com/CNIT-Organization/aitoolkit/issues
+Author: Faisal Fida
+License-Expression: MIT
+License-File: LICENSE
+Keywords: ai,embeddings,llm,openai,rag,self-hosted,stt,tts
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.27
+Requires-Dist: loguru>=0.7
+Requires-Dist: openai>=1.50
+Requires-Dist: pydantic-settings>=2.3
+Requires-Dist: pydantic>=2.7
+Provides-Extra: all
+Requires-Dist: langchain-core>=0.3; extra == 'all'
+Requires-Dist: langchain-openai>=0.2; extra == 'all'
+Requires-Dist: qdrant-client>=1.12; extra == 'all'
+Requires-Dist: redis>=5.0; extra == 'all'
+Provides-Extra: cache
+Requires-Dist: redis>=5.0; extra == 'cache'
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.3; extra == 'langchain'
+Requires-Dist: langchain-openai>=0.2; extra == 'langchain'
+Provides-Extra: rag
+Requires-Dist: qdrant-client>=1.12; extra == 'rag'
+Description-Content-Type: text/markdown
+
+# aitoolkit
+
+Centralized AI clients — **LLM, Embeddings, Speech-to-Text, Text-to-Speech, and
+RAG** — targeting self-hosted, OpenAI-compatible services. Reusable across
+projects via a single git install.
+
+## Why
+
+Replaces scattered, provider-specific AI code (cloud-LLM wrappers, local Whisper,
+ad-hoc LangChain) with one thin, stable, dependency-light package. The core is
+**100% LangChain-free**; LangChain is an opt-in extra used only where LangGraph
+orchestration needs it.
+
+## Design principles
+
+1. **Depend on stable interfaces, hide volatile implementations.** The public API
+   is a small set of clients + plain types; provider SDKs stay internal.
+2. **Core is dependency-light and LangChain-free.** Heavy/optional things (qdrant,
+   redis, langchain) live behind extras and import lazily.
+3. **No project specifics in the package.** Hosts, model ids, voices, collection
+   names, and keywords are parameters/config — never hardcoded.
+4. **OpenAI-compatible first.** LLM, embeddings and STT use the `openai` SDK; only
+   TTS is a small custom `httpx` client.
+5. **Async-first**, with sync convenience wrappers where ergonomics demand it.
+
+## Install
+
+```bash
+# core only (LLM, embeddings, STT, TTS)
+pip install "aitoolkit @ git+https://github.com/CNIT-Organization/aitoolkit.git@v0.2.0"
+
+# with RAG (Qdrant) + caching + LangChain bridge
+pip install "aitoolkit[all] @ git+https://github.com/CNIT-Organization/aitoolkit.git@v0.2.0"
+
+# pick exactly what a service needs
+pip install "aitoolkit[rag,cache] @ git+...@v0.2.0"
+pip install "aitoolkit[rag,langchain] @ git+...@v0.2.0"
+```
+
+Extras: `rag` (qdrant-client) · `cache` (redis) · `langchain` (langchain-core +
+langchain-openai) · `all`.
+
+## Configuration
+
+All config is environment-driven (`AITOOLKIT_*`, see [.env.example](.env.example)),
+but every client also accepts explicit overrides, so no env is strictly required.
+
+## Quick start
+
+```python
+import asyncio
+from aitoolkit import get_llm_client, get_embeddings_client, get_stt_client, get_tts_client
+
+async def main():
+    llm = get_llm_client()
+    print(await llm.chat("Say hello in one short sentence."))
+
+    async for tok in llm.stream("Count to five."):
+        print(tok, end="", flush=True)
+
+    emb = get_embeddings_client()
+    vecs = await emb.aembed_documents(["first document", "second document"])
+    print("dim:", emb.dimension)
+
+    stt = get_stt_client()
+    result = await stt.transcribe("audio.wav", language="en")
+    print(result.text)
+
+    tts = get_tts_client()
+    audio = await tts.synthesize("Hello world", voice="your-voice-id")
+    open("out.wav", "wb").write(audio)
+
+    # multi-speaker: synthesize each turn with its own voice and stitch to one WAV
+    dialogue = await tts.synthesize_dialogue([
+        {"voice_id": "voice-a", "text": "Welcome to the overview."},
+        {"voice_id": "voice-b", "text": "Let's dive in."},
+    ])
+    open("dialogue.wav", "wb").write(dialogue)
+
+asyncio.run(main())
+```
+
+### Structured output
+
+```python
+from pydantic import BaseModel
+class Flashcard(BaseModel):
+    question: str
+    answer: str
+
+card = await get_llm_client().chat_structured(Flashcard, "Make a flashcard about the water cycle.")
+```
+
+### RAG (`aitoolkit[rag]`)
+
+```python
+from aitoolkit.rag import get_rag_agent
+agent = get_rag_agent(collection_name="documents")
+await agent.add_documents(["chunk 1", "chunk 2"], file_id="doc-42")
+answer = await agent.answer_question("What does the document say about safety?")
+```
+
+### LangChain bridge (`aitoolkit[langchain]`)
+
+```python
+from aitoolkit.integrations.langchain import to_chat_model, LangChainEmbeddings
+chat_model = to_chat_model(temperature=0.3)   # a LangChain BaseChatModel for LangGraph
+embeddings = LangChainEmbeddings()            # a LangChain Embeddings
+```
+
+## Testing
+
+```bash
+pip install -e ".[all]" --group dev
+pytest                       # unit tests (mocked)
+AITOOLKIT_RUN_LIVE=1 pytest  # also run live smoke tests against your endpoints
+```
+
+Live tests auto-skip when the configured endpoints are unreachable.

ff_aitoolkit-0.2.0/README.md

@@ -0,0 +1,119 @@
+# aitoolkit
+
+Centralized AI clients — **LLM, Embeddings, Speech-to-Text, Text-to-Speech, and
+RAG** — targeting self-hosted, OpenAI-compatible services. Reusable across
+projects via a single git install.
+
+## Why
+
+Replaces scattered, provider-specific AI code (cloud-LLM wrappers, local Whisper,
+ad-hoc LangChain) with one thin, stable, dependency-light package. The core is
+**100% LangChain-free**; LangChain is an opt-in extra used only where LangGraph
+orchestration needs it.
+
+## Design principles
+
+1. **Depend on stable interfaces, hide volatile implementations.** The public API
+   is a small set of clients + plain types; provider SDKs stay internal.
+2. **Core is dependency-light and LangChain-free.** Heavy/optional things (qdrant,
+   redis, langchain) live behind extras and import lazily.
+3. **No project specifics in the package.** Hosts, model ids, voices, collection
+   names, and keywords are parameters/config — never hardcoded.
+4. **OpenAI-compatible first.** LLM, embeddings and STT use the `openai` SDK; only
+   TTS is a small custom `httpx` client.
+5. **Async-first**, with sync convenience wrappers where ergonomics demand it.
+
+## Install
+
+```bash
+# core only (LLM, embeddings, STT, TTS)
+pip install "aitoolkit @ git+https://github.com/CNIT-Organization/aitoolkit.git@v0.2.0"
+
+# with RAG (Qdrant) + caching + LangChain bridge
+pip install "aitoolkit[all] @ git+https://github.com/CNIT-Organization/aitoolkit.git@v0.2.0"
+
+# pick exactly what a service needs
+pip install "aitoolkit[rag,cache] @ git+...@v0.2.0"
+pip install "aitoolkit[rag,langchain] @ git+...@v0.2.0"
+```
+
+Extras: `rag` (qdrant-client) · `cache` (redis) · `langchain` (langchain-core +
+langchain-openai) · `all`.
+
+## Configuration
+
+All config is environment-driven (`AITOOLKIT_*`, see [.env.example](.env.example)),
+but every client also accepts explicit overrides, so no env is strictly required.
+
+## Quick start
+
+```python
+import asyncio
+from aitoolkit import get_llm_client, get_embeddings_client, get_stt_client, get_tts_client
+
+async def main():
+    llm = get_llm_client()
+    print(await llm.chat("Say hello in one short sentence."))
+
+    async for tok in llm.stream("Count to five."):
+        print(tok, end="", flush=True)
+
+    emb = get_embeddings_client()
+    vecs = await emb.aembed_documents(["first document", "second document"])
+    print("dim:", emb.dimension)
+
+    stt = get_stt_client()
+    result = await stt.transcribe("audio.wav", language="en")
+    print(result.text)
+
+    tts = get_tts_client()
+    audio = await tts.synthesize("Hello world", voice="your-voice-id")
+    open("out.wav", "wb").write(audio)
+
+    # multi-speaker: synthesize each turn with its own voice and stitch to one WAV
+    dialogue = await tts.synthesize_dialogue([
+        {"voice_id": "voice-a", "text": "Welcome to the overview."},
+        {"voice_id": "voice-b", "text": "Let's dive in."},
+    ])
+    open("dialogue.wav", "wb").write(dialogue)
+
+asyncio.run(main())
+```
+
+### Structured output
+
+```python
+from pydantic import BaseModel
+class Flashcard(BaseModel):
+    question: str
+    answer: str
+
+card = await get_llm_client().chat_structured(Flashcard, "Make a flashcard about the water cycle.")
+```
+
+### RAG (`aitoolkit[rag]`)
+
+```python
+from aitoolkit.rag import get_rag_agent
+agent = get_rag_agent(collection_name="documents")
+await agent.add_documents(["chunk 1", "chunk 2"], file_id="doc-42")
+answer = await agent.answer_question("What does the document say about safety?")
+```
+
+### LangChain bridge (`aitoolkit[langchain]`)
+
+```python
+from aitoolkit.integrations.langchain import to_chat_model, LangChainEmbeddings
+chat_model = to_chat_model(temperature=0.3)   # a LangChain BaseChatModel for LangGraph
+embeddings = LangChainEmbeddings()            # a LangChain Embeddings
+```
+
+## Testing
+
+```bash
+pip install -e ".[all]" --group dev
+pytest                       # unit tests (mocked)
+AITOOLKIT_RUN_LIVE=1 pytest  # also run live smoke tests against your endpoints
+```
+
+Live tests auto-skip when the configured endpoints are unreachable.

ff_aitoolkit-0.2.0/pyproject.toml

@@ -0,0 +1,71 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "ff-aitoolkit"
+version = "0.2.0"
+description = "Centralized AI clients (LLM, embeddings, STT, TTS, RAG) for self-hosted OpenAI-compatible GPU services."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Faisal Fida" }]
+keywords = ["llm", "embeddings", "stt", "tts", "rag", "openai", "self-hosted", "ai"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Typing :: Typed",
+]
+
+# Core stays light and 100% LangChain-free. Heavy/optional deps live in extras.
+dependencies = [
+    "openai>=1.50",
+    "httpx>=0.27",
+    "pydantic>=2.7",
+    "pydantic-settings>=2.3",
+    "loguru>=0.7",
+]
+
+[project.optional-dependencies]
+# Retrieval-augmented generation (Qdrant vector store).
+rag = ["qdrant-client>=1.12"]
+# Query result caching for the retriever.
+cache = ["redis>=5.0"]
+# Return LangChain BaseChatModel / Embeddings for LangGraph consumers (e.g. notebook).
+langchain = ["langchain-core>=0.3", "langchain-openai>=0.2"]
+# Everything.
+all = ["ff-aitoolkit[rag,cache,langchain]"]
+
+[project.urls]
+Homepage = "https://github.com/CNIT-Organization/aitoolkit"
+Repository = "https://github.com/CNIT-Organization/aitoolkit"
+Issues = "https://github.com/CNIT-Organization/aitoolkit/issues"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+    "pytest-asyncio>=0.23",
+    "respx>=0.21",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/aitoolkit"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src/aitoolkit", "README.md", "LICENSE"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]
+filterwarnings = ["ignore::DeprecationWarning"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"

ff_aitoolkit-0.2.0/src/aitoolkit/__init__.py

@@ -0,0 +1,66 @@
+"""aitoolkit — centralized AI clients for self-hosted OpenAI-compatible services.
+
+Core capabilities (LLM, embeddings, STT, TTS) are always available. RAG and the
+LangChain bridge live behind extras and are imported from their own subpackages
+(``aitoolkit.rag``, ``aitoolkit.integrations.langchain``) so that importing the
+top-level package never forces an optional dependency.
+"""
+
+from __future__ import annotations
+
+from aitoolkit.config import AIToolkitSettings, configure, get_settings
+from aitoolkit.embeddings import EmbeddingsClient, get_embeddings_client
+from aitoolkit.exceptions import (
+    AIToolkitError,
+    ConfigurationError,
+    EmbeddingsError,
+    LLMError,
+    STTError,
+    TTSError,
+    VectorStoreError,
+)
+from aitoolkit.llm import LLMClient, get_llm_client
+from aitoolkit.stt import STTClient, get_stt_client
+from aitoolkit.tts import TTSClient, concat_wav, get_tts_client
+from aitoolkit.types import (
+    ChatMessage,
+    DialogueTurn,
+    RetrievedChunk,
+    TranscriptionResult,
+)
+
+__version__ = "0.2.0"
+
+__all__ = [
+    "__version__",
+    # config
+    "AIToolkitSettings",
+    "configure",
+    "get_settings",
+    # llm
+    "LLMClient",
+    "get_llm_client",
+    # embeddings
+    "EmbeddingsClient",
+    "get_embeddings_client",
+    # stt
+    "STTClient",
+    "get_stt_client",
+    # tts
+    "TTSClient",
+    "get_tts_client",
+    "concat_wav",
+    # types
+    "ChatMessage",
+    "DialogueTurn",
+    "RetrievedChunk",
+    "TranscriptionResult",
+    # exceptions
+    "AIToolkitError",
+    "ConfigurationError",
+    "LLMError",
+    "EmbeddingsError",
+    "STTError",
+    "TTSError",
+    "VectorStoreError",
+]

ff_aitoolkit-0.2.0/src/aitoolkit/config.py

@@ -0,0 +1,107 @@
+"""Central configuration for aitoolkit.
+
+All settings are read from ``AITOOLKIT_*`` environment variables (or a ``.env``
+file) but every client also accepts explicit overrides, so the package can be
+used with zero environment configuration.
+
+Defaults intentionally point at ``localhost`` — they are NOT specific to any one
+deployment. Production endpoints are supplied by the consuming application via
+environment variables (see ``.env.example``).
+"""
+
+from __future__ import annotations
+
+from functools import lru_cache
+from typing import Optional
+
+from pydantic import Field
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+# A placeholder key for OpenAI-compatible servers that perform no app-layer auth
+# (our GPU services are firewalled, not key-gated). The openai SDK requires a
+# non-empty key, so we provide one.
+_NO_AUTH = "no-auth"
+
+
+class AIToolkitSettings(BaseSettings):
+    """Runtime configuration for every aitoolkit capability."""
+
+    model_config = SettingsConfigDict(
+        env_prefix="AITOOLKIT_",
+        env_file=".env",
+        env_file_encoding="utf-8",
+        extra="ignore",
+    )
+
+    # --- LLM (vLLM, OpenAI-compatible) ---
+    llm_base_url: str = Field(default="http://localhost:8000/v1")
+    llm_api_key: str = Field(default=_NO_AUTH)
+    # No default model id — set AITOOLKIT_LLM_MODEL to your served model.
+    llm_model: str = Field(default="")
+    llm_temperature: float = Field(default=0.2)
+    llm_timeout: float = Field(default=60.0)
+    llm_max_retries: int = Field(default=2)
+
+    # --- Embeddings (TEI, OpenAI-compatible) ---
+    embeddings_base_url: str = Field(default="http://localhost:8001/v1")
+    embeddings_api_key: str = Field(default=_NO_AUTH)
+    # No default model id — set AITOOLKIT_EMBEDDINGS_MODEL to your served model.
+    embeddings_model: str = Field(default="")
+    # TEI accepts modest batches; keep conservative and configurable.
+    embeddings_batch_size: int = Field(default=32)
+    embeddings_timeout: float = Field(default=60.0)
+
+    # --- Speech-to-Text (faster-whisper, OpenAI-compatible) ---
+    stt_base_url: str = Field(default="http://localhost:8003/v1")
+    stt_api_key: str = Field(default=_NO_AUTH)
+    stt_model: str = Field(default="whisper-1")
+    stt_language: Optional[str] = Field(default=None)
+    stt_timeout: float = Field(default=120.0)
+
+    # --- Text-to-Speech (custom /api/tts) ---
+    tts_base_url: str = Field(default="http://localhost:8002")
+    tts_default_voice: Optional[str] = Field(default=None)
+    tts_timeout: float = Field(default=120.0)
+
+    # --- Vector store (Qdrant) ---
+    qdrant_url: str = Field(default="http://localhost:6333")
+    qdrant_collection: str = Field(default="documents")
+    # Optional fixed vector size. When None, it is detected from the embedding model.
+    qdrant_vector_size: Optional[int] = Field(default=None)
+    # The qdrant-client refuses to talk to a server whose minor version differs by
+    # more than one, emitting a UserWarning. Self-hosted servers often lag the
+    # client; set False to silence the check when the API surface we use is stable.
+    qdrant_check_compatibility: bool = Field(default=True)
+
+    # --- Retriever cache (Redis, optional) ---
+    redis_url: Optional[str] = Field(default=None)
+    cache_ttl: int = Field(default=3600)
+
+
+_override: Optional[AIToolkitSettings] = None
+
+
+@lru_cache(maxsize=1)
+def _env_settings() -> AIToolkitSettings:
+    """Settings built from AITOOLKIT_* env vars (cached fallback)."""
+    return AIToolkitSettings()
+
+
+def get_settings() -> AIToolkitSettings:
+    """Return the process-wide settings.
+
+    Prefers an instance installed via :func:`configure`; otherwise builds one
+    from ``AITOOLKIT_*`` environment variables.
+    """
+    return _override if _override is not None else _env_settings()
+
+
+def configure(settings: AIToolkitSettings) -> None:
+    """Install an explicit settings object as the process-wide singleton.
+
+    Lets a consuming application own configuration directly instead of relying
+    on environment variables and this package's generic defaults. Call once at
+    startup, before any client is created.
+    """
+    global _override
+    _override = settings

ff_aitoolkit-0.2.0/src/aitoolkit/embeddings/__init__.py

@@ -0,0 +1,5 @@
+"""Embeddings capability — OpenAI-compatible embeddings (e.g. TEI)."""
+
+from aitoolkit.embeddings.client import EmbeddingsClient, get_embeddings_client
+
+__all__ = ["EmbeddingsClient", "get_embeddings_client"]