kiapi 0.6.0__tar.gz → 0.8.0__tar.gz

{kiapi-0.6.0 → kiapi-0.8.0}/.gitignore

@@ -32,3 +32,6 @@ tests/assets/
 
 # Node modules (pnpm-managed dev tooling, e.g. firebase-tools)
 node_modules/
+
+# Web UI build output (`mise run web:build`)
+/src/kiapi/api/ui/static/

kiapi-0.8.0/PKG-INFO

@@ -0,0 +1,349 @@
+Metadata-Version: 2.5
+Name: kiapi
+Version: 0.8.0
+Summary: Unified local inference API server for LLM agents on Apple Silicon (MLX): chat, embedding, image, music, sound-effect, and video — one process, one global memory budget, one single-flight queue, unified jobs and files.
+Project-URL: Homepage, https://github.com/kiarina/kiapi
+Project-URL: Repository, https://github.com/kiarina/kiapi
+Project-URL: Documentation, https://kiarina.github.io/kiapi/
+Project-URL: Changelog, https://github.com/kiarina/kiapi/blob/main/CHANGELOG.md
+Author-email: kiarina <kiarinadawa@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent,api,apple-silicon,fastapi,local-inference,mlx
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: MacOS X
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: MacOS
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: <3.13,>=3.12
+Requires-Dist: click
+Requires-Dist: fastapi>=0.141
+Requires-Dist: httpx
+Requires-Dist: kiarina-utils-app>=2.3.0
+Requires-Dist: kiarina-utils-common>=2.2.0
+Requires-Dist: mflux>=0.20.0
+Requires-Dist: mlx-audiocraft>=0.1.0
+Requires-Dist: mlx-embeddings>=0.1.0
+Requires-Dist: mlx-lm>=0.31.3
+Requires-Dist: mlx-vlm==0.7.1
+Requires-Dist: numpy
+Requires-Dist: pillow
+Requires-Dist: psutil
+Requires-Dist: pydantic-settings-manager>=3.8.0
+Requires-Dist: pydantic>=2
+Requires-Dist: python-multipart
+Requires-Dist: pyyaml
+Requires-Dist: questionary
+Requires-Dist: soundfile>=0.13.1
+Requires-Dist: torch
+Requires-Dist: torchaudio
+Requires-Dist: torchvision
+Requires-Dist: uvicorn[standard]
+Description-Content-Type: text/markdown
+
+# kiapi
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
+![Python](https://img.shields.io/badge/Python-3.12-blue.svg)
+[![API Docs](https://img.shields.io/badge/API%20Docs-GitHub%20Pages-green.svg)](https://kiarina.github.io/kiapi/)
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/playground-dark.webp">
+  <img alt="The kiapi web UI running Z-Image, with the generated image next to the form" src="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/playground-light.webp">
+</picture>
+
+## Summary
+
+kiapi is an API server that uses a Mac Studio M4 Max with 128GB of memory at home to provide the following capabilities.
+
+| Capability | Support |
+| --- | --- |
+| Chat | OpenAI Chat Completions API compatible<br>text + image + audio + video input support<br>tool call + tool choice (auto, any, specific) + parallel tool calls + streaming support |
+| Embedding | text + image input support |
+| Image generation | text2image, image2image, image editing, and LoRA training support |
+| Music and sound-effect generation | text2audio, cover, repaint, and extract support |
+| Video generation | text2video, image2video, and audio2video support |
+| Web | search + fetch support |
+
+See: [API Documents](https://kiarina.github.io/kiapi/)
+
+> [!NOTE]
+> The Mac Studio M4 Max with 128GB of memory is an example.
+> Other Apple Silicon models will also work if they have sufficient unified memory.
+
+## Web UI
+
+Open `http://localhost:8000/` while kiapi runs. The web UI lets you try every
+family without writing a request, and shows what each one can do.
+
+- **Playground** for every family: forms built from the API schema, uploads or
+  stored files as inputs, and results as they finish.
+- **Write with chat** drafts prompts with a local chat model that reads the
+  family's guide.
+- **Ask about a family** from the corner button. The assistant answers from
+  the family's OpenAPI document and can fill in the form for you; you review
+  and submit it.
+- **Models** shows which models are ready and the `kiapi activate` command for
+  anything missing. **Jobs** and **Files** show progress and previews.
+
+<p>
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/overview-dark.webp">
+  <img alt="Overview of server status, jobs, and recent files" src="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/overview-light.webp" width="49%">
+</picture>
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/models-dark.webp">
+  <img alt="Model setup status with activate commands" src="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/models-light.webp" width="49%">
+</picture>
+</p>
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/assistant-dark.webp">
+  <img alt="The assistant filling in the Z-Image form from a request" src="https://raw.githubusercontent.com/kiarina/kiapi/main/docs/images/web-ui/assistant-light.webp">
+</picture>
+
+## Resources
+
+To provide its capabilities, kiapi downloads and uses selected resources from the list below.
+Each resource is governed by its upstream license.
+Always review the upstream license to confirm the terms and whether commercial use is permitted.
+
+> [!IMPORTANT] Review date: 2026-06-23
+
+| Domain | Family | Resource | Kind | Upstream license | Notes |
+|---|---|---|---|---|---|
+| chat | [chat](src/kiapi/capabilities/chat/README.md) | [mlx-community/Qwen3-Omni-30B-A3B-Instruct-4bit](https://huggingface.co/mlx-community/Qwen3-Omni-30B-A3B-Instruct-4bit) | model weights | Apache-2.0 | MLX-converted Qwen3 Omni model. |
+|  |  | [mlx-community/Qwen3.8-Flash-Next-4bit](https://huggingface.co/mlx-community/Qwen3.8-Flash-Next-4bit) | model weights | Qwen Community License 1.0 | MLX-converted Qwen3.8-Flash-Next. Commercial Model-as-a-Service or AI-work-assistant businesses need a separate license from Qwen. |
+|  |  | [mlx-community/Qwen3.8-27B-4bit](https://huggingface.co/mlx-community/Qwen3.8-27B-4bit) | model weights | Apache-2.0 | MLX-converted Qwen3.8 model. |
+| embedding | [embedding](src/kiapi/capabilities/embedding/README.md) | [mlx-community/Qwen3-Embedding-8B-mxfp8](https://huggingface.co/mlx-community/Qwen3-Embedding-8B-mxfp8) | model weights | Apache-2.0 | Text embedding model. |
+|  |  | [mlx-community/Qwen3-VL-Embedding-2B-mxfp8](https://huggingface.co/mlx-community/Qwen3-VL-Embedding-2B-mxfp8) | model weights | Apache-2.0 | Text + image embedding model. |
+| image | [zimage](src/kiapi/capabilities/zimage/README.md) | [filipstrand/Z-Image-Turbo-mflux-4bit](https://huggingface.co/filipstrand/Z-Image-Turbo-mflux-4bit) | model weights | Tongyi Qianwen License | Quantized MLX-compatible Z-Image Turbo; inherits the original Z-Image Turbo license. |
+|  |  | [Tongyi-MAI/Z-Image](https://huggingface.co/Tongyi-MAI/Z-Image) | model weights | Apache-2.0 | Base Z-Image model. |
+|  | [flux2](src/kiapi/capabilities/flux2/README.md) | [black-forest-labs/FLUX.2-klein-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-9B) | model weights | FLUX Non-Commercial License | Gated upstream model. Confirm terms before any commercial use. |
+|  |  | [black-forest-labs/FLUX.2-klein-base-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B) | model weights | Apache-2.0 | Open-weight FLUX.2 Klein Base 4B variant. |
+|  |  | [black-forest-labs/FLUX.2-klein-base-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B) | model weights | FLUX Non-Commercial License | Gated upstream model. Confirm terms before any commercial use. |
+|  | [qwen](src/kiapi/capabilities/qwen/README.md) | [Qwen/Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) | model weights | Apache-2.0 | Text-to-image model. |
+|  |  | [Qwen/Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) | model weights | Apache-2.0 | Image editing model. |
+|  |  | [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1) | model weights | Qwen Research License | Non-commercial (research and evaluation) only; commercial use needs a separate license from Alibaba. Unified generation and editing, RGBA output. Editing is pinned from the [kiarina fork](https://github.com/kiarina/mflux/tree/qwen-image-2.1-edit) of mflux until [#741](https://github.com/mflux-community/mflux/pull/741) is released. |
+|  | [ideogram4](src/kiapi/capabilities/ideogram4/README.md) | [ideogram-ai/ideogram-4-fp8](https://huggingface.co/ideogram-ai/ideogram-4-fp8) | model weights | Ideogram Non-Commercial Model Agreement | Gated upstream model. Confirm hosted-service and commercial-use terms. |
+|  | [ernie](src/kiapi/capabilities/ernie/README.md) | [baidu/ERNIE-Image-Turbo](https://huggingface.co/baidu/ERNIE-Image-Turbo) | model weights | Apache-2.0 | Turbo ERNIE-Image variant. |
+|  |  | [baidu/ERNIE-Image](https://huggingface.co/baidu/ERNIE-Image) | model weights | Apache-2.0 | Base ERNIE-Image variant. |
+|  | [seedvr2](src/kiapi/capabilities/seedvr2/README.md) | [numz/SeedVR2_comfyUI](https://huggingface.co/numz/SeedVR2_comfyUI) | model weights | Apache-2.0 | SeedVR2 3B and 7B upscaling checkpoints. |
+|  | [depthpro](src/kiapi/capabilities/depthpro/README.md) | [apple/ml-depth-pro](https://github.com/apple/ml-depth-pro) / [depth_pro.pt](https://ml-site.cdn-apple.com/models/depth-pro/depth_pro.pt) | code + model file | Apple custom license | GitHub reports `NOASSERTION`; review Apple's license text before redistribution or commercial use. |
+| audio | [acestep](src/kiapi/capabilities/acestep/README.md) | [ace-step/ACE-Step-1.5](https://github.com/ace-step/ACE-Step-1.5) | Python package | MIT | Installed into the ACE-Step dedicated venv. |
+|  |  | [ACE-Step/Ace-Step1.5](https://huggingface.co/ACE-Step/Ace-Step1.5) | shared checkpoints | MIT | Shared ACE-Step 1.5 checkpoint resources. |
+|  |  | [ACE-Step/acestep-v15-xl-base](https://huggingface.co/ACE-Step/acestep-v15-xl-base) | model weights | MIT | Extra checkpoint used by `xl-base`. |
+| audio | [audiogen](src/kiapi/capabilities/audiogen/README.md) | [facebook/audiogen-medium](https://huggingface.co/facebook/audiogen-medium) | model weights | CC-BY-NC-4.0 | Non-commercial license. |
+| video | [ltx2](src/kiapi/capabilities/ltx2/README.md) | [Blaizzy/mlx-video](https://github.com/Blaizzy/mlx-video) | Python package | MIT | Installed from a pinned Git commit for LTX-2.5 / LTX-2 inference. LTX-2.5 support is pinned from the [kiarina fork](https://github.com/kiarina/mlx-video/tree/kiapi/ltx-2.5) until [#52](https://github.com/Blaizzy/mlx-video/pull/52) is merged. |
+|  |  | [Lightricks/LTX-2.5](https://huggingface.co/Lightricks/LTX-2.5) | model weights | LTX-2.x Community License | Gated; accept the model terms on Hugging Face. Used by the default `ltx-2.5-distilled`. |
+|  |  | [Lightricks/LTX-2.5-22b-IC-LoRA-Pixel-Spatial-Upscaler](https://huggingface.co/Lightricks/LTX-2.5-22b-IC-LoRA-Pixel-Spatial-Upscaler) | model weights | LTX-2.x Community License | Gated. Detailing adapter for `pipeline="dfr"`. |
+|  |  | [mlx-community/gemma-4-e2b-it-bf16](https://huggingface.co/mlx-community/gemma-4-e2b-it-bf16) | model weights | Gemma Terms of Use | Prompt enhancer for `enhance_prompt`. |
+|  |  | [prince-canuma/LTX-2-distilled](https://huggingface.co/prince-canuma/LTX-2-distilled) | model weights | Not declared upstream | The model card has no license metadata; verify rights before use. |
+| web | [web](src/kiapi/capabilities/web/README.md) | [searxng/searxng](https://github.com/searxng/searxng) / `searxng/searxng:latest` | Docker image | AGPL-3.0 | Web search backend. AGPL obligations can matter for network services. |
+|  |  | [unclecode/crawl4ai](https://github.com/unclecode/crawl4ai) / `unclecode/crawl4ai:latest` | Docker image | Apache-2.0 | Web fetch backend. |
+
+> [!NOTE]
+> Using all resources at once requires a little under 600GB of disk space in total.
+> It also consumes a little under 50GB of memory at peak.
+> For the size and memory consumption of each resource, see the per-family links above.
+
+## Design
+
+**Reliably provide every capability:**
+
+- Queue non-administrative requests and process them one at a time
+- Manage API server memory to prevent overcommit failures
+
+**Support interactive integration with LLM agents:**
+
+- Provide LLMs with tips as well as I/O specifications through `openapi.json`
+- Run generation tasks in both sync and async modes
+- Make asynchronous task progress observable
+
+**Provide secure external access to kiapi inside a closed network:**
+
+- kiapi binds to `127.0.0.1` by default and never exposes an inbound socket by itself
+- For access from other machines, put it behind your own private network layer, for example `tailscale serve`
+
+> [!NOTE]
+> See [ARCHITECTURE.md](ARCHITECTURE.md) for details.
+
+## API
+
+| Domain | Family | Endpoint | Description |
+|---|---|---|---|
+| chat |  | `POST /v1/chat` | [Chat API details](src/kiapi/capabilities/chat/README.md) |
+| embedding |  | `POST /v1/embedding` | [Embedding API details](src/kiapi/capabilities/embedding/README.md) |
+| image | zimage | `POST /v1/image/zimage` | [Z-Image API details](src/kiapi/capabilities/zimage/README.md) |
+|  | flux2 | `POST /v1/image/flux2` | [FLUX.2 API details](src/kiapi/capabilities/flux2/README.md) |
+|  | qwen | `POST /v1/image/qwen` | [Qwen Image API details](src/kiapi/capabilities/qwen/README.md) |
+|  | ideogram4 | `POST /v1/image/ideogram4` | [Ideogram 4 API details](src/kiapi/capabilities/ideogram4/README.md) |
+|  | ernie | `POST /v1/image/ernie` | [ERNIE-Image API details](src/kiapi/capabilities/ernie/README.md) |
+|  | seedvr2 | `POST /v1/image/seedvr2` | [SeedVR2 API details](src/kiapi/capabilities/seedvr2/README.md) |
+|  | depthpro | `POST /v1/image/depthpro` | [Depth Pro API details](src/kiapi/capabilities/depthpro/README.md) |
+| audio | acestep | `POST /v1/audio/acestep` | [ACE-Step API details](src/kiapi/capabilities/acestep/README.md) |
+|  | audiogen | `POST /v1/audio/audiogen` | [AudioGen API details](src/kiapi/capabilities/audiogen/README.md) |
+| video | ltx2 | `POST /v1/video/ltx2` | [LTX-2 API details](src/kiapi/capabilities/ltx2/README.md) |
+| web |  | `POST /v1/web` | [Web API details](src/kiapi/capabilities/web/README.md) |
+| core | files | `POST /v1/files` | Upload input files, LoRA adapters, and other files, then issue a `file_id`. |
+|  |  | `GET /v1/files` | Return a list of stored files. |
+|  |  | `GET /v1/files/{file_id}` | Return file metadata. |
+|  |  | `GET /v1/files/{file_id}/download` | Download the file body. |
+|  |  | `DELETE /v1/files/{file_id}` | Delete a stored file. |
+|  | jobs | `GET /v1/jobs` | Return a list of generation jobs. |
+|  |  | `GET /v1/jobs/{job_id}` | Return job status, progress, result, and artifact `file_id`s. |
+|  |  | `DELETE /v1/jobs/{job_id}` | Remove a job from the job store. Running jobs are not interrupted. |
+|  | openapi | `GET /openapi.json` | Return the common API and each capability documentation URL. |
+|  |  | `GET /v1/{domain}/{family}/openapi.json` | Return detailed input/output specs, usage, tips, and examples for each family. |
+|  | health | `GET /health` | Return server status, warmup status, queue length, and memory usage. |
+|  | setup | `GET /v1/setup` | Return every model with its setup state and the command that activates a missing resource. |
+|  | web UI | `GET /` | Serve the web UI. |
+
+See: [kiapi API Docs](https://kiarina.github.io/kiapi/)
+
+## Requirements
+
+- macOS / Apple Silicon
+- Python `>=3.12,<3.13`
+- `uv` (optional, recommended for isolated CLI installs and faster venv/package setup in `kiapi activate`)
+- `mise` (used for development)
+- Docker (when using the Web capability)
+- Enough disk capacity for model weights and Docker images
+
+kiapi is developed mainly for personal use on a **Mac Studio M4 Max 128GB**.
+Some or all features may work on other Apple Silicon environments, but they are
+not the primary verification target.
+
+The memory budget can be specified with `KIAPI_MEMORY_LIMIT_GB`. If omitted,
+kiapi automatically uses 80% of installed memory as the effective budget on
+startup. If a model's required memory does not fit in that budget, requests
+return 503 as an insufficient memory budget error.
+
+`kiapi activate --all` uses a little under 600GB of disk capacity, including
+model weights and Docker images. At first, it is recommended to use `kiapi activate`
+to set up only the capabilities you need.
+
+## Quick Start
+
+**Set up kiapi:**
+```sh
+# Install kiapi
+python3.12 -m pip install --upgrade kiapi  # If you cannot use uv
+uv tool install --python 3.12 kiapi        # If you can use uv
+
+# Change the default host, port, or memory budget if needed
+kiapi config init  # Create the configuration file
+kiapi config edit  # Edit the configuration file in an editor
+
+# Check the current setup state
+kiapi status
+
+# Prepare model weights, Docker images, and dedicated venv environments
+kiapi activate                   # Choose targets from the interactive list
+kiapi activate --all             # Set up everything (just under 600GB)
+kiapi activate --family acestep  # Set up only the specified family
+
+# Verify the setup
+kiapi check        # Choose targets from the interactive list
+kiapi check --all  # Verify everything
+```
+
+**Use from an LLM agent:**
+```sh
+# Start the kiapi server
+kiapi run                             # Start based on the configuration file (default: 127.0.0.1:8000)
+kiapi run --host 0.0.0.0 --port 8500  # Start on a specific host and port
+
+# Example integration with an agent
+codex e "
+Please inspect http://localhost:8000/openapi.json.
+Use the music generation API to create a 20-second BGM track at ~/Downloads/bgm.wav
+with the theme 'a person walking in the rain'.
+"
+
+# Inspect the generated file
+open ~/Downloads/bgm.wav
+```
+
+**Use from a browser:**
+Open `http://localhost:8000/` while kiapi runs. See [Web UI](#web-ui).
+
+**Run as a background service:**
+```sh
+# kiapi
+kiapi service install    # Register
+kiapi service show       # Show the installed plist
+kiapi service start      # Start
+kiapi service status     # Check status and the tail of logs
+kiapi service stop       # Stop
+kiapi service uninstall  # Remove
+```
+
+**Access from other machines:**
+kiapi binds to `127.0.0.1` by default. To reach it from other machines, expose
+it over your own private network layer. For example, with
+[Tailscale](https://tailscale.com/) on the kiapi machine:
+
+```sh
+tailscale serve --bg --https=8500 8500  # TLS endpoint reachable only inside your tailnet
+```
+
+## Local Storage
+
+kiapi mainly writes to these local paths at runtime.
+
+| Purpose | Setting | Default | Notes |
+|---|---|---|---|
+| Files API uploads, generated artifacts, and URL/data URL inputs | `KIAPI_FILES_ROOT` | `/tmp/kiapi/files` | Storage referenced by `file_id`. The default may disappear after OS reboot or tmp cleanup. Use `~/.kiapi/files` or external storage for long-term retention. |
+| Temporary working directories during request processing | `KIAPI_TMP_ROOT` | `/tmp/kiapi/work` | Used for chat/embedding input expansion, generation intermediates, LoRA training work, and similar tasks. |
+| Web backend subprocess logs | `KIAPI_WEB_BACKEND_LOG_DIR` | `/tmp/kiapi/logs/web` | stdout/stderr for SearXNG / Crawl4AI Docker subprocesses. |
+| ACE-Step dedicated venv / project / checkpoints | `KIAPI_ACESTEP_PYTHON_PATH`, `KIAPI_ACESTEP_PROJECT_ROOT`, `KIAPI_ACESTEP_CHECKPOINT_DIR` | `acestep/` under the user data dir | When `python_path`, `project_root`, and `checkpoint_dir` are omitted, kiapi places the ACE-Step venv and checkpoints under a persistent ACE-Step directory. |
+
+Other model weights and library caches are managed by Hugging Face, mflux,
+Docker, or each library/tool. kiapi generally does not move them into its own
+storage location.
+
+## Security
+
+By default, `kiapi run` starts on `127.0.0.1:8000`.
+When `--host 0.0.0.0` is specified, the server may be reachable from other
+machines, so use it only on trusted networks.
+## Development
+
+```sh
+make init     # Install dependencies, download test data, and create venv environments
+make update   # Sync dependencies
+make upgrade  # Upgrade dependencies
+
+# ... implement
+
+make       # Format, type-check, and regenerate dynamic documentation
+make test  # Unit tests
+make dev   # Start the development server with auto-reload
+make web-dev  # Start the web UI dev server against `make dev` (needs Node and pnpm via mise)
+
+# GPU-backed functional and regression tests
+make verify        # Choose the capability family interactively
+make verify-fast   # Interactive choice, light tests only
+make verify-kiapi  # Run every capability non-interactively
+```
+
+## Release
+
+Releases to PyPI are automated by GitHub Actions workflows.
+
+> [!NOTE]
+> See [docs/runbooks/release.md](docs/runbooks/release.md) for the detailed release procedure.
+
+## Project Status
+
+> [!NOTE]
+> kiapi is OSS developed mainly for personal use.
+> The API, supported models, and setup instructions may change in the future.
+> Issues and pull requests are welcome, but support is best-effort because this is a personal project.