ollambda 0.1.0__tar.gz

ollambda-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ollambda contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ollambda-0.1.0/PKG-INFO

@@ -0,0 +1,83 @@
+Metadata-Version: 2.4
+Name: ollambda
+Version: 0.1.0
+Summary: Add your description here
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: flask>=3.1.3
+Requires-Dist: gunicorn>=26.0.0
+Requires-Dist: pre-commit>=4.6.0
+Requires-Dist: pytest>=9.1.1
+Requires-Dist: pytest-cov>=7.1.0
+Requires-Dist: pyyaml>=6.0.3
+Dynamic: license-file
+
+# ollambda
+
+Simple Flask based API server to proxy ollama calls to a more performant backend (such as llama.cpp)
+
+## How to Use
+
+### Installation
+
+To install ollambda, clone this repository and use `pip install .`
+
+#### venv
+
+Venv usage is encouraged, the simplest way to manage this is to use [uv](https://docs.astral.sh/uv/):
+
+```
+cd ollambda
+uv venv .venv
+source .venv/bin/activate
+uv pip install .
+```
+
+### Configuration
+
+Currently ollambda proxies requests to an upstream llama.cpp server. The address is configured via environment variables or a YAML config file at `~/.ollambda/config.yaml`:
+
+| Setting    | Env var               | Config key            | Default     |
+| ---------- | --------------------- | --------------------- | ----------- |
+| Proxy IP   | `OLLAMBDA_PROXY_IP`   | `ollambda_proxy_ip`   | `127.0.0.1` |
+| Proxy port | `OLLAMBDA_PROXY_PORT` | `ollambda_proxy_port` | `8080`      |
+
+### Running the server
+
+Start with Gunicorn (included as a dependency):
+
+```bash
+gunicorn -c gunicorn.conf.py ollambda:app
+```
+
+This spawns one worker per CPU core and binds to `0.0.0.0:5000` by default. Override settings on the command line:
+
+```bash
+gunicorn ollambda:app -w 4 -b 0.0.0.0:5000
+```
+
+## ollama API coverage
+
+| Method | Endpoint             | Description                                                 | Implemented? |
+| ------ | -------------------- | ----------------------------------------------------------- | ------------ |
+| POST   | `/api/generate`      | Generate a completion (streaming or non-streaming)          | N            |
+| POST   | `/api/chat`          | Generate a chat completion (with tool calling support)      | Y            |
+| POST   | `/api/create`        | Create a model from another model, GGUF, or safetensors     | N (no plans) |
+| GET    | `/api/tags`          | List locally available models                               | Y            |
+| POST   | `/api/show`          | Show model information (details, modelfile, template, etc.) | Y            |
+| POST   | `/api/copy`          | Copy a model to a new name                                  | N            |
+| DELETE | `/api/delete`        | Delete a model                                              | Y            |
+| POST   | `/api/pull`          | Download a model from the Ollama library                    | N            |
+| POST   | `/api/push`          | Upload a model to a model library                           | N (no plans) |
+| POST   | `/api/embed`         | Generate embeddings from a model                            | Y            |
+| GET    | `/api/ps`            | List models currently loaded in memory                      | Y            |
+| GET    | `/api/version`       | Retrieve the Ollama server version                          | Y            |
+| HEAD   | `/api/blobs/:digest` | Check if a file blob exists on the server                   | N (no plans) |
+| POST   | `/api/blobs/:digest` | Push a file blob to the server                              | N (no plans) |
+
+> [!IMPORTANT]
+> Current "implementation" may not be fully complete and API aligned.
+
+> [!NOTE]
+> Source: https://github.com/ollama/ollama/blob/main/docs/api.md

ollambda-0.1.0/README.md

@@ -0,0 +1,68 @@
+# ollambda
+
+Simple Flask based API server to proxy ollama calls to a more performant backend (such as llama.cpp)
+
+## How to Use
+
+### Installation
+
+To install ollambda, clone this repository and use `pip install .`
+
+#### venv
+
+Venv usage is encouraged, the simplest way to manage this is to use [uv](https://docs.astral.sh/uv/):
+
+```
+cd ollambda
+uv venv .venv
+source .venv/bin/activate
+uv pip install .
+```
+
+### Configuration
+
+Currently ollambda proxies requests to an upstream llama.cpp server. The address is configured via environment variables or a YAML config file at `~/.ollambda/config.yaml`:
+
+| Setting    | Env var               | Config key            | Default     |
+| ---------- | --------------------- | --------------------- | ----------- |
+| Proxy IP   | `OLLAMBDA_PROXY_IP`   | `ollambda_proxy_ip`   | `127.0.0.1` |
+| Proxy port | `OLLAMBDA_PROXY_PORT` | `ollambda_proxy_port` | `8080`      |
+
+### Running the server
+
+Start with Gunicorn (included as a dependency):
+
+```bash
+gunicorn -c gunicorn.conf.py ollambda:app
+```
+
+This spawns one worker per CPU core and binds to `0.0.0.0:5000` by default. Override settings on the command line:
+
+```bash
+gunicorn ollambda:app -w 4 -b 0.0.0.0:5000
+```
+
+## ollama API coverage
+
+| Method | Endpoint             | Description                                                 | Implemented? |
+| ------ | -------------------- | ----------------------------------------------------------- | ------------ |
+| POST   | `/api/generate`      | Generate a completion (streaming or non-streaming)          | N            |
+| POST   | `/api/chat`          | Generate a chat completion (with tool calling support)      | Y            |
+| POST   | `/api/create`        | Create a model from another model, GGUF, or safetensors     | N (no plans) |
+| GET    | `/api/tags`          | List locally available models                               | Y            |
+| POST   | `/api/show`          | Show model information (details, modelfile, template, etc.) | Y            |
+| POST   | `/api/copy`          | Copy a model to a new name                                  | N            |
+| DELETE | `/api/delete`        | Delete a model                                              | Y            |
+| POST   | `/api/pull`          | Download a model from the Ollama library                    | N            |
+| POST   | `/api/push`          | Upload a model to a model library                           | N (no plans) |
+| POST   | `/api/embed`         | Generate embeddings from a model                            | Y            |
+| GET    | `/api/ps`            | List models currently loaded in memory                      | Y            |
+| GET    | `/api/version`       | Retrieve the Ollama server version                          | Y            |
+| HEAD   | `/api/blobs/:digest` | Check if a file blob exists on the server                   | N (no plans) |
+| POST   | `/api/blobs/:digest` | Push a file blob to the server                              | N (no plans) |
+
+> [!IMPORTANT]
+> Current "implementation" may not be fully complete and API aligned.
+
+> [!NOTE]
+> Source: https://github.com/ollama/ollama/blob/main/docs/api.md

ollambda-0.1.0/ollambda/__init__.py

@@ -0,0 +1,7 @@
+__version__ = "0.0.1"
+
+from ollambda.app_factory import create_app
+
+app = create_app()
+
+__all__ = ["create_app", "app", "__version__"]

ollambda-0.1.0/ollambda/api/api.py

@@ -0,0 +1,160 @@
+import json
+
+from flask import Blueprint, Response, abort, jsonify, request
+
+from ollambda import __version__
+from ollambda.engine.chat import ChatHandler
+from ollambda.engine.llm.providers.llamacpp import ProviderLlamaCPP
+from ollambda.engine.messages import list_message_factory
+
+bp = Blueprint("api", __name__, url_prefix="/api")
+
+
+chat_handler = ChatHandler(provider=ProviderLlamaCPP())
+
+
+def _parse_request_body() -> dict:
+    """Try JSON body first, then fall back to form-encoded JSON."""
+    data = request.get_json(silent=True)
+    if data is not None:
+        return data
+
+    for key, value in request.form.items():
+        for candidate in (value, key):
+            if candidate.strip().startswith("{"):
+                try:
+                    return json.loads(candidate)
+                except json.JSONDecodeError:
+                    continue
+
+    return {}
+
+
+@bp.route("/chat", methods=("POST",))
+def chat():
+    """Generate the next message in a chat.
+
+    Parameters:
+        model (str): model name
+        messages (list[str]): list of messages in the chat
+        tools (list[str]): list of tools in json
+        think (bool): should the model think before responding?
+        format (str): optional json format for response
+        stream (bool): return response as a stream of objects if True. Default False.
+
+    `message` object fields:
+        role (str): system, user, assistant or tool
+        content (str): message content
+        thinking (str): thinking process (for thinking models)
+        images (list[str], optional): list of images
+        tool_calls (list[str], optional): list of tools in JSON that the model wants to use
+        tool_name (str, optional): name of the tool that was executed
+    """
+    data = _parse_request_body()
+
+    # pop the messages, they're very verbose when printing the data
+    model = data.get("model", None)
+    # handle remaining args/settings
+    messages = data.pop("messages", [])
+    stream = data.get("stream", False)
+
+    print("Extra args:")
+    print(data)
+
+    if model is None:
+        abort(400, "No model specified")
+
+    if model not in chat_handler.provider.models:
+        abort(400, chat_handler.provider.invalid_model_error(model))
+
+    tools = data.get("tools", None)
+
+    messages = list_message_factory(messages)
+    if stream:
+        return Response(
+            chat_handler.generate_response_stream(
+                model=model, messages=messages, tools=tools
+            ),
+            mimetype="application/x-ndjson",
+        )
+
+    abort(400, "Only streaming responses are currently supported")
+
+
+@bp.route("/tags", methods=("GET",))
+def tags():
+    """List locally available models."""
+    print("Listing models.")
+
+    models = chat_handler.provider.get_models()
+
+    return jsonify({"models": [model.info for model in models]})
+
+
+@bp.route("/show", methods=("POST",))
+def show():
+    """Show model information.
+
+    Parameters:
+        model (str): model name
+        verbose (bool, optional): returns full data for verbose response fields if True
+    """
+    data = _parse_request_body()
+
+    if "model" not in data:
+        abort(400, "No model specified")
+
+    return jsonify({})
+
+
+@bp.route("/delete", methods=("DELETE",))
+def delete():
+    """Delete a model.
+
+    Parameters:
+        model (str): the model name
+    """
+    data = _parse_request_body()
+
+    if "model" not in data:
+        abort(400, "No model specified")
+
+    model = data["model"]
+
+    abort(404, f"Model {model} not found")
+    # should return 200 on successful deletion
+
+
+@bp.route("/embed", methods=("POST",))
+def embed():
+    """Generate embeddings.
+
+    Parameters:
+        model: name of the model
+        input: text or list of text to generate embeddings for
+    """
+    data = _parse_request_body()
+
+    if "model" not in data:
+        abort(400, "No model specified")
+
+    model = data.get("model")
+
+    input = data.get("input")
+    if isinstance(input, str):
+        input = [input]
+
+    # generate embeddings
+    return jsonify({"model": model, "embeddings": ""})
+
+
+@bp.route("/ps", methods=("GET",))
+def ps():
+    """List running models."""
+    return jsonify({"models": []})
+
+
+@bp.route("/version", methods=("GET",))
+def version():
+    """Respond with version info."""
+    return jsonify({"version": __version__})

ollambda-0.1.0/ollambda/app_factory.py

@@ -0,0 +1,21 @@
+import os
+
+from flask import Flask
+
+from ollambda.api import api
+
+
+def create_app(test_config=None):
+    app = Flask(__name__, instance_relative_config=True)
+    app.config.from_mapping(SECRET_KEY="test")
+
+    if test_config is None:
+        app.config.from_pyfile("config.py", silent=True)
+    else:
+        app.config.from_mapping(test_config)
+
+    os.makedirs(app.instance_path, exist_ok=True)
+
+    app.register_blueprint(api.bp)
+
+    return app

ollambda-0.1.0/ollambda/engine/chat.py

@@ -0,0 +1,64 @@
+import json
+from typing import Generator
+
+from ollambda.engine.llm.providers.baseprovider import BaseProvider
+from ollambda.engine.messages import AnyMessage
+from ollambda.engine.messages.assistant import AIMessage
+
+
+class ChatHandler:
+    """Wrapper class for a chat instance."""
+
+    def __init__(self, provider: BaseProvider) -> None:
+        self._provider = provider
+
+        self._messages: list[AnyMessage] = []
+
+    @property
+    def messages(self) -> list[AnyMessage]:
+        return self._messages
+
+    @property
+    def provider(self) -> BaseProvider:
+        return self._provider
+
+    @provider.setter
+    def provider(self, provider: BaseProvider) -> None:
+        self._provider = provider
+
+    def generate_response_stream(
+        self,
+        model: str,
+        messages: list[AnyMessage],
+        tools: list[dict[str, str]] | None = None,
+    ) -> Generator[str]:
+        """Stream response tokens, yielding tokens as they are generated."""
+
+        print("-" * AIMessage._terminal_line_width)
+        print("Generating responses. Input context:")
+        print("-" * AIMessage._terminal_line_width)
+        for message in messages:
+            print(message)
+
+        print("-" * AIMessage._terminal_line_width)
+        print("Response:")
+
+        response = AIMessage()
+        for event in self.provider.generate_stream(
+            model=model, messages=messages, tools=tools
+        ):
+            response.add_chunk(event)
+
+            if event.is_empty:
+                continue
+
+            print(event.thinking, end="")
+            yield json.dumps(event.message_chunk) + "\n"
+
+        # by convention, once a tool call is complete, the llm stops responding
+        if response.tool_calls:
+            # extract and yield the json serialisable chunk from the first tool call
+            data = response.tool_calls[0].message_chunk
+            yield json.dumps(data) + "\n"
+
+        print(response.pretty_str(show_thinking=False))

ollambda-0.1.0/ollambda/engine/llm/model.py

@@ -0,0 +1,123 @@
+from datetime import datetime, timezone
+from typing import Any
+
+
+class Model:
+    """Represents an LLM instance.
+
+    Args:
+        name (str): "Friendly name" for the model
+        model (str): Actual model name
+        size (int, optional): Model's size on disk
+        digest (str, optional): Model's SHA256 hash
+        created_at (timestamp-like, datetime, optional): Model creation time (usually file ctime)
+        details (dict, optional): Details field from OpenAI compatible model info dump
+    """
+
+    def __init__(
+        self,
+        name: str,
+        model: str,
+        size: int = -1,
+        digest: str | None = None,
+        created_at: int | float | datetime | str | None = None,
+        details: dict[str, str | int] | None = None,
+    ) -> None:
+        self.name = name
+        self.model = model
+        self.size = size
+        self.digest = digest or "sha256:" + "0" * 64
+
+        self.details = details or {}
+
+        if created_at is None:
+            self._created_at = datetime.now(timezone.utc)
+        else:
+            self._created_at = Model._parse_datetime(created_at)
+
+    @staticmethod
+    def _parse_datetime(t: int | float | str | datetime | None) -> datetime:
+        """Parse a generic-type datetime/timestamp, returning datetime."""
+        if isinstance(t, (float, int)):
+            return datetime.fromtimestamp(t, tz=timezone.utc)
+        elif isinstance(t, datetime):
+            return t
+        elif t == "":
+            return datetime.now(timezone.utc)
+        else:
+            try:
+                return datetime.fromtimestamp(float(t), tz=timezone.utc)
+            except ValueError as ex:
+                raise ValueError(
+                    f"Could not convert created_at of type {type(t)}"
+                ) from ex
+
+    @property
+    def created_at(self) -> datetime:
+        return self._created_at
+
+    @created_at.setter
+    def created_at(self, t: int | float | datetime) -> None:
+        self._created_at = Model._parse_datetime(t)
+
+    @classmethod
+    def from_llamacpp_models_endpoint(cls, data: dict, metadata: list[dict]) -> "Model":
+        """Construct a Model handler from the /v1/models llama-server endpoint.
+
+        This endpoint provides two lists:
+        models: [
+            {"name": "...", ...}
+        ]
+        data: [
+            {"id": "..."}
+        ]
+
+        We must combine info from these lists.
+
+        Args:
+            data (dict): Info from  the `models` section of the response
+            metadata (list[dict]): Complete `data` section of the response, for parsing
+        """
+        name = data.get("name", None)
+        model = data.get("model", None)
+        size = data.get("size", -1)
+        digest = data.get("digest", None)
+
+        if not name:
+            raise ValueError("No name found in response!")
+
+        if not model:
+            raise ValueError("No model found in response!")
+
+        new = cls(name=name, model=model, size=size, digest=digest)
+
+        for meta in metadata:
+            if meta.get("id", None) != model:
+                continue
+            # data field has an "inner" metadata dict
+            inner_meta = meta.get("meta", {})
+
+            created_at = meta.get("created", None)
+            if created_at is not None:
+                new.created_at = created_at
+
+            size = inner_meta.get("size", None)
+            if size is not None:
+                new.size = size
+
+        return new
+
+    @property
+    def info(self) -> dict[Any, Any]:
+        data = {
+            "name": self.name,
+            "model": self.model,
+            "modified_at": self.created_at.isoformat(timespec="seconds"),
+            "size": self.size,
+            "digest": self.digest,
+        }
+
+        if self.details:
+            data["details"] = self.details
+
+        return data

ollambda-0.1.0/ollambda/engine/llm/providers/baseprovider.py

@@ -0,0 +1,46 @@
+from typing import Generator
+
+from ollambda.engine.llm.model import Model
+from ollambda.engine.llm.server_sent_event import ServerSentEvent
+from ollambda.engine.messages import AnyMessage
+
+
+class ModelNotFoundError(ValueError):
+    """Raised when a requested model is not available."""
+
+    pass
+
+
+class BaseProvider:
+    """Allows ollmbda to function as a "proxy", forwarding the request onwards."""
+
+    def __init__(self) -> None:
+        self._models = None
+
+    @property
+    def models(self) -> dict[str, Model]:
+        if self._models is None:
+            self._models = {m.name: m for m in self.get_models()}
+        return self._models
+
+    @models.setter
+    def models(self, models: dict[str, Model]) -> None:
+        self._models = models
+
+    def invalid_model_error(self, model: str) -> str:
+        """Return a dynamic error message for an invalid model request."""
+        return f"Model {model} is not available. Try: {[m for m in self.models]}"
+
+    def get_models(self) -> list[Model]:
+        """This method should return a list of available Model objects."""
+        raise NotImplementedError
+
+    def generate_stream(
+        self,
+        model: str,
+        messages: list[AnyMessage],
+        tools: list[dict[str, str]] | None = None,
+        **kwargs,
+    ) -> Generator[ServerSentEvent]:
+        """Yield a series of ServerSentEvent objects."""
+        raise NotImplementedError

ollambda-0.1.0/ollambda/engine/llm/providers/dummy.py

@@ -0,0 +1,41 @@
+from typing import Generator
+
+from ollambda.engine.llm.model import Model
+from ollambda.engine.llm.providers.baseprovider import BaseProvider
+from ollambda.engine.llm.server_sent_event import ServerSentEvent
+from ollambda.engine.messages import AnyMessage
+
+
+class ProviderDummy(BaseProvider):
+    """Dummy provider for testing.
+
+    Models and stream events are configured directly rather than fetched
+    from an upstream server.
+    """
+
+    def __init__(
+        self,
+        models: list[Model] | None = None,
+        events: list[ServerSentEvent] | None = None,
+    ) -> None:
+        super().__init__()
+
+        self._models_list = models or [Model(name="dummy", model="dummy:latest")]
+        self._events = events or []
+
+    def get_models(self) -> list[Model]:
+        return self._models_list
+
+    def generate_stream(
+        self,
+        model: str,
+        messages: list[AnyMessage],
+        tools: list[dict[str, str]] | None = None,
+        **kwargs,
+    ) -> Generator[ServerSentEvent]:
+        model_obj = self.models.get(model, None)
+        if model_obj is None:
+            raise ValueError(self.invalid_model_error(model))
+
+        for event in self._events:
+            yield event