petsitter 0.1.0__py3-none-any.whl

petsitter-0.1.0.dist-info/METADATA

@@ -0,0 +1,280 @@
+Metadata-Version: 2.4
+Name: petsitter
+Version: 0.1.0
+Summary: OpenAI-compatible proxy that adds functionality to models through tricks
+License-File: LICENSE.MIT
+Requires-Python: >=3.10
+Requires-Dist: click>=8.1.0
+Requires-Dist: httpx>=0.25.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: starlette>=0.34.0
+Requires-Dist: uvicorn>=0.24.0
+Provides-Extra: test
+Requires-Dist: pytest-asyncio>=0.21.0; extra == 'test'
+Requires-Dist: pytest>=7.0.0; extra == 'test'
+Description-Content-Type: text/markdown
+
+# Petsitter
+
+**Teach old models new tricks.**
+
+Petsitter is an OpenAI-compatible proxy that layers smart harnesses on top of language models, giving them capabilities they don't natively have. Smaller models can't do tool calling? Petsitter tricks them into it. Need structured JSON output? Petsitter will loop until it gets it right.
+
+But that's only the beginning. Cyclomatic complexity? Halstead metrics? Chidamber and Kemerer? Why not!
+
+## Who Is This For?
+
+- **You run local models** (Ollama, llama.cpp, vllm, sglang) and miss OpenAI's features
+- **You use small/cheap models** that lack tool calling or JSON mode
+- **You build agentic systems** that need consistent capabilities across different models
+- **You want to experiment** with prompt engineering tricks without changing your application code
+
+## What Does It Do?
+
+Petsitter sits between your application and your model, intercepting requests and responses to apply "tricks" — pluggable transformations that add functionality through:
+
+1. **Prompt engineering** — Inject instructions and tool definitions
+2. **Context manipulation** — Modify messages before/after the model sees them
+3. **Retry loops** — Call the model again if output doesn't meet requirements
+4. **Response transformation** — Convert outputs to expected formats (e.g., OpenAI tool_calls)
+
+## Why Use It?
+
+- **No model changes required** — Works with any OpenAI-compatible endpoint
+- **Pluggable architecture** — Write your own tricks in Python
+- **Transparent to your app** — Point your existing code at petsitter instead of the model
+- **Mix and match** — Combine multiple tricks for compound effects
+
+---
+
+## Installation
+
+```bash
+# Create virtual environment
+uv venv
+
+# Activate it
+source .venv/bin/activate
+
+# Install petsitter
+pip install -e .
+```
+
+## Quick Start
+
+```bash
+# Start your model backend (e.g., Ollama)
+ollama serve
+
+# Activate the virtual environment
+source .venv/bin/activate
+
+# Run petsitter with tricks
+./petsitter --model_url http://localhost:11434 \
+            --model_name llama3:8b \
+            --trick tricks/json_mode.py \
+            --trick tricks/tool_call.py \
+            --listen_on localhost:8080
+```
+
+Now point your AI applications to `http://localhost:8080/v1`.
+
+## CLI Options
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `--model_url` | Yes | Base URL of upstream model (e.g., `http://localhost:11434`) |
+| `--model_name` | No | Model name (optional for vllm, sglang, llama.cpp) |
+| `--api_key` | No | API key for upstream (if required) |
+| `--trick` | No | Path to a trick module (can be repeated) |
+| `--listen_on` | No | Host:port to listen on (default: `localhost:8080`) |
+
+## Built-in Tricks
+
+### JSON Mode (`tricks/json_mode.py`)
+
+Enforces valid JSON output by:
+- Adding formatting instructions to the system prompt
+- Retrying with feedback if response isn't valid JSON
+- Stripping markdown code blocks
+
+```bash
+./petsitter --model_url http://localhost:11434 --trick tricks/json_mode.py
+```
+
+### Tool Calling (`tricks/tool_call.py`)
+
+Enables tool calling for models without native support:
+- Injects tool definitions into prompts
+- Parses JSONRPC-style tool call responses
+- Converts to OpenAI `tool_calls` format
+
+```bash
+./petsitter --model_url http://localhost:11434 --trick tricks/tool_call.py
+```
+
+### List Files (`tricks/list_files.py`)
+
+Test trick that provides a `list_files` tool. Useful for testing tool calling functionality.
+
+## Creating Custom Tricks
+
+The `Trick` class has four hooks you can implement. Each hook is optional — only implement what you need.
+
+### `system_prompt(to_add: str) -> str`
+
+**When:** Called once per request, before any messages are sent to the model.
+
+**Purpose:** Append instructions to the system prompt. This is how you "prime" the model to behave a certain way.
+
+**Example:**
+```python
+def system_prompt(self, to_add: str) -> str:
+    return "IMPORTANT: Respond only in valid JSON. No markdown, no explanations."
+```
+
+### `pre_hook(context: list, params: dict) -> list`
+
+**When:** Called after the system prompt is set, before the model receives the messages.
+
+**Purpose:** Modify the conversation context. You can inject tool definitions, add few-shot examples, or restructure messages.
+
+**Parameters:**
+- `context`: List of message dicts (`[{"role": "user", "content": "..."}]`)
+- `params`: Request parameters including `tools`, `temperature`, etc.
+
+**Example:**
+```python
+def pre_hook(self, context: list, params: dict) -> list:
+    if "tools" in params:
+        # Inject tool definitions into system prompt
+        tools_json = json.dumps(params["tools"])
+        context[0]["content"] += f"\n\nAvailable tools: {tools_json}"
+    return context
+```
+
+### `post_hook(context: list) -> list`
+
+**When:** Called after the model responds, before the response goes back to your application.
+
+**Purpose:** Validate, transform, or retry. This is where you can:
+- Parse the response and convert it to a different format
+- Detect when the model failed and call it again with feedback
+- Extract tool calls from natural language
+
+**Example (JSON validation with retry):**
+```python
+def post_hook(self, context: list) -> list:
+    attempts = 3
+    while attempts > 0:
+        try:
+            json.loads(context[-1]["content"])
+            break  # Valid JSON, we're done
+        except json.JSONDecodeError:
+            attempts -= 1
+            if attempts == 0:
+                break
+            # Retry with feedback
+            context = callmodel(context, "That wasn't valid JSON. Try again.")
+    return context
+```
+
+**Example (Tool call detection):**
+```python
+def post_hook(self, context: list) -> list:
+    content = context[-1]["content"]
+    if self._looks_like_tool_call(content):
+        # Convert to OpenAI tool_calls format
+        context[-1]["tool_calls"] = [self._parse_tool_call(content)]
+        context[-1]["content"] = None
+    return context
+```
+
+### `info(capabilities: dict) -> dict`
+
+**When:** Called when building the response to your application.
+
+**Purpose:** Declare what capabilities this trick provides. Some frameworks check for capabilities before using certain features.
+
+**Example:**
+```python
+def info(self, capabilities: dict) -> dict:
+    capabilities["json_mode"] = True
+    capabilities["tools_support"] = True
+    return capabilities
+```
+
+## Full Trick Example
+
+Here's a trick that makes any model respond in haiku:
+
+```python
+from src.trick import Trick
+
+class HaikuTrick(Trick):
+    """Force the model to respond only in haiku."""
+
+    def system_prompt(self, to_add: str) -> str:
+        return (
+            "You must respond only in haiku (5-7-5 syllables). "
+            "No explanations, no extra text. Just haiku."
+        )
+
+    def post_hook(self, context: list) -> list:
+        # Could add syllable counting and retry here
+        return context
+
+    def info(self, capabilities: dict) -> dict:
+        capabilities["haiku_mode"] = True
+        return capabilities
+```
+
+Use it:
+```bash
+./petsitter --model_url http://localhost:11434 --trick haiku.py
+```
+
+## API Endpoints
+
+Petsitter exposes OpenAI-compatible endpoints:
+
+- `POST /v1/chat/completions` - Chat completions (proxied + transformed)
+- `GET /v1/models` - List available models (proxied)
+- `GET /health` - Health check
+
+## Running Tests
+
+```bash
+# Activate virtual environment
+source .venv/bin/activate
+
+# Install test dependencies
+pip install -e ".[test]"
+
+# Run tests
+pytest tests/
+```
+
+## Example: Using with an Agentic Framework
+
+```python
+from openai import OpenAI
+
+# Point to petsitter instead of directly to the model
+client = OpenAI(
+    base_url="http://localhost:8080/v1",
+    api_key="not-needed"
+)
+
+response = client.chat.completions.create(
+    model="any-model-name",
+    messages=[{"role": "user", "content": "List files in /tmp"}],
+    tools=[{"type": "function", "function": {"name": "list_files", ...}}]
+)
+
+# With tool_call trick, even small models can use tools!
+```
+
+## License
+
+MIT

petsitter-0.1.0.dist-info/RECORD

@@ -0,0 +1,11 @@
+src/__init__.py,sha256=aUogdBrPbcR_sN60wGtqWp8NKj62XAxpURRAGHRi838,73
+src/context.py,sha256=6fz-_CxCCzRkfKgXjemMcRCFkIlYIC_hoIwqzfM0fSM,2328
+src/loader.py,sha256=1wN00V-waodqkt1r7T3zAVGSkNrA209Oc_8QZhvk4nk,1727
+src/proxy.py,sha256=m9IP_mkjyGoj4QDHv1vceCxnXE7qnXrflmYJ0DMIMpo,5969
+src/server.py,sha256=4V9n82sMhvmwCNPBhpOY0s2i3PjXZEyUiDel2nOk6AI,6316
+src/trick.py,sha256=wn4ET7qD9T1DpFDCuWDGrKY-Y-PEh0_uNQ3DGxSgh0Q,3057
+petsitter-0.1.0.dist-info/METADATA,sha256=XObnYYC2WRLhk0g1BfrVlPUakf9uy305YP3Z4eYZOss,8481
+petsitter-0.1.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+petsitter-0.1.0.dist-info/entry_points.txt,sha256=CYTLVRD5Gm9J6u9giXkhrAewEe9xHZ-uz_wSOZZtl4E,45
+petsitter-0.1.0.dist-info/licenses/LICENSE.MIT,sha256=5T9l8YHMRH7MQGFKfOrJ0CHHUAjpH_YKmcnAQRjCABg,1071
+petsitter-0.1.0.dist-info/RECORD,,

petsitter-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

petsitter-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+petsitter = src.server:cli

petsitter-0.1.0.dist-info/licenses/LICENSE.MIT

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Chris McKenzie
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

src/__init__.py

@@ -0,0 +1,3 @@
+from src.trick import Trick, callmodel
+
+__all__ = ["Trick", "callmodel"]

src/context.py

@@ -0,0 +1,98 @@
+"""Context manipulation utilities for petsitter."""
+
+from typing import Any
+
+
+def get_system_prompt(context: list) -> str:
+    """Extract the system prompt from context if present.
+
+    Args:
+        context: List of message dicts.
+
+    Returns:
+        System prompt content or empty string.
+    """
+    if context and context[0].get("role") == "system":
+        return context[0].get("content", "")
+    return ""
+
+
+def set_system_prompt(context: list, content: str) -> list:
+    """Set or update the system prompt in context.
+
+    Args:
+        context: List of message dicts.
+        content: New system prompt content.
+
+    Returns:
+        Modified context (may be new list).
+    """
+    if not context:
+        return [{"role": "system", "content": content}]
+
+    if context[0].get("role") == "system":
+        context[0]["content"] = content
+    else:
+        context.insert(0, {"role": "system", "content": content})
+
+    return context
+
+
+def append_to_system_prompt(context: list, addition: str) -> list:
+    """Append text to the system prompt.
+
+    Args:
+        context: List of message dicts.
+        addition: Text to append.
+
+    Returns:
+        Modified context.
+    """
+    current = get_system_prompt(context)
+    if current:
+        new_content = current + "\n" + addition
+    else:
+        new_content = addition
+    return set_system_prompt(context, new_content)
+
+
+def get_last_message(context: list) -> dict | None:
+    """Get the last message in context.
+
+    Args:
+        context: List of message dicts.
+
+    Returns:
+        Last message dict or None.
+    """
+    return context[-1] if context else None
+
+
+def set_last_message_content(context: list, content: str) -> list:
+    """Replace the content of the last message.
+
+    Args:
+        context: List of message dicts.
+        content: New content.
+
+    Returns:
+        Modified context.
+    """
+    if context:
+        context[-1]["content"] = content
+    return context
+
+
+def add_message(context: list, role: str, content: str) -> list:
+    """Add a new message to the context.
+
+    Args:
+        context: List of message dicts.
+        role: Message role (user, assistant, system, tool).
+        content: Message content.
+
+    Returns:
+        Modified context.
+    """
+    context.append({"role": role, "content": content})
+    return context

src/loader.py

@@ -0,0 +1,63 @@
+"""Dynamic loading of trick modules."""
+
+import importlib.util
+import sys
+from pathlib import Path
+from typing import Type
+
+from src.trick import Trick
+
+
+def load_trick_from_path(path: str) -> Type[Trick]:
+    """Load a Trick class from a Python file path.
+
+    Args:
+        path: File path to the trick module (e.g., 'tricks/tools.py').
+
+    Returns:
+        The Trick subclass defined in the module.
+
+    Raises:
+        FileNotFoundError: If the path doesn't exist.
+        ImportError: If no Trick subclass is found.
+    """
+    trick_path = Path(path).resolve()
+    if not trick_path.exists():
+        raise FileNotFoundError(f"Trick file not found: {path}")
+
+    module_name = trick_path.stem
+    spec = importlib.util.spec_from_file_location(module_name, trick_path)
+    if spec is None or spec.loader is None:
+        raise ImportError(f"Could not load module: {path}")
+
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[module_name] = module
+    spec.loader.exec_module(module)
+
+    # Find the Trick subclass (not the base Trick itself)
+    for attr_name in dir(module):
+        attr = getattr(module, attr_name)
+        if (
+            isinstance(attr, type)
+            and issubclass(attr, Trick)
+            and attr is not Trick
+        ):
+            return attr
+
+    raise ImportError(f"No Trick subclass found in {path}")
+
+
+def load_tricks(paths: list[str]) -> list[Trick]:
+    """Load multiple tricks from file paths.
+
+    Args:
+        paths: List of file paths to trick modules.
+
+    Returns:
+        List of instantiated Trick objects.
+    """
+    tricks = []
+    for path in paths:
+        trick_class = load_trick_from_path(path)
+        tricks.append(trick_class())
+    return tricks

src/proxy.py

@@ -0,0 +1,168 @@
+"""OpenAI API proxy handling for petsitter."""
+
+import json
+import logging
+from typing import Any
+
+import httpx
+
+from src.context import append_to_system_prompt
+from src.trick import Trick, callmodel
+
+logger = logging.getLogger("petsitter")
+
+
+class ProxyHandler:
+    """Handles proxied requests to the upstream model."""
+
+    def __init__(
+        self,
+        model_url: str,
+        model_name: str | None,
+        api_key: str = "",
+        tricks: list[Trick] | None = None,
+    ):
+        self.model_url = model_url.rstrip("/")
+        self.model_name = model_name
+        self.api_key = api_key
+        self.tricks = tricks or []
+
+    def _build_headers(self) -> dict[str, str]:
+        """Build request headers."""
+        headers = {"Content-Type": "application/json"}
+        if self.api_key:
+            headers["Authorization"] = f"Bearer {self.api_key}"
+        return headers
+
+    def _apply_system_prompt_tricks(self, system_prompt: str) -> str:
+        """Apply system_prompt hooks from all tricks."""
+        result = system_prompt
+        for trick in self.tricks:
+            addition = trick.system_prompt(result)
+            if addition:
+                result = result + "\n" + addition if result else addition
+        return result
+
+    def _apply_pre_hooks(self, context: list, params: dict) -> list:
+        """Apply pre_hook from all tricks."""
+        result = context
+        for trick in self.tricks:
+            result = trick.pre_hook(result, params)
+        return result
+
+    def _apply_post_hooks(self, context: list) -> list:
+        """Apply post_hook from all tricks."""
+        result = context
+        for trick in self.tricks:
+            result = trick.post_hook(result)
+        return result
+
+    def _merge_capabilities(self) -> dict:
+        """Merge capabilities from all tricks."""
+        capabilities = {}
+        for trick in self.tricks:
+            capabilities = trick.info(capabilities)
+        return capabilities
+
+    async def chat_completions(self, payload: dict) -> dict:
+        """Handle /v1/chat/completions request.
+
+        Args:
+            payload: The incoming request body.
+
+        Returns:
+            The response from the upstream model (possibly modified).
+        """
+        # Extract messages and apply system prompt tricks
+        messages = payload.get("messages", [])
+        system_prompt = ""
+        if messages and messages[0].get("role") == "system":
+            system_prompt = messages[0].get("content", "")
+            messages = messages[1:]
+
+        # Apply system prompt tricks
+        new_system_prompt = self._apply_system_prompt_tricks(system_prompt)
+        if new_system_prompt:
+            messages = [{"role": "system", "content": new_system_prompt}] + messages
+
+        # Apply pre-hooks
+        messages = self._apply_pre_hooks(messages, payload)
+
+        # Build upstream request
+        upstream_payload = {
+            "model": self.model_name or payload.get("model", "default"),
+            "messages": messages,
+        }
+        # Pass through optional params (but force stream=false for upstream)
+        for key in ["temperature", "max_tokens"]:
+            if key in payload:
+                upstream_payload[key] = payload[key]
+        
+        # Don't pass tools/stream to upstream - we handle tool calling via tricks
+        # and always fetch full response for post-processing
+        # upstream_payload["stream"] = False
+
+        logger.info(f"Calling upstream model: {self.model_url}/v1/chat/completions")
+        logger.debug(f"Upstream payload: {json.dumps(upstream_payload, indent=2)}")
+
+        # Call upstream model
+        async with httpx.AsyncClient() as client:
+            response = await client.post(
+                f"{self.model_url}/v1/chat/completions",
+                json=upstream_payload,
+                headers=self._build_headers(),
+                timeout=120.0,
+            )
+            
+            # Log response details for debugging
+            logger.info(f"Upstream response status: {response.status_code}")
+            logger.debug(f"Upstream response headers: {dict(response.headers)}")
+            logger.debug(f"Upstream response body: {response.text[:500] if response.text else '(empty)'}")
+            
+            response.raise_for_status()
+            
+            # Check for empty response
+            if not response.content:
+                logger.error(f"Empty response from upstream. Status: {response.status_code}")
+                logger.error(f"Response headers: {dict(response.headers)}")
+                raise ValueError(f"Upstream returned empty response (status {response.status_code})")
+            
+            result = response.json()
+
+        logger.debug(f"Upstream response: {json.dumps(result, indent=2)}")
+
+        # Extract assistant message and build context for post-hooks
+        assistant_message = result["choices"][0]["message"]
+        context = messages + [assistant_message]
+
+        logger.debug(f"Context before post-hooks: {json.dumps(context, indent=2)}")
+
+        # Apply post-hooks
+        context = self._apply_post_hooks(context)
+
+        logger.debug(f"Context after post-hooks: {json.dumps(context, indent=2)}")
+
+        # Update result with potentially modified response
+        result["choices"][0]["message"] = context[-1]
+
+        # Merge capabilities into response if present
+        capabilities = self._merge_capabilities()
+        if capabilities:
+            result["capabilities"] = capabilities
+
+        return result
+
+    async def models(self) -> dict:
+        """Handle /v1/models request.
+
+        Returns:
+            Model listing response.
+        """
+        async with httpx.AsyncClient() as client:
+            response = await client.get(
+                f"{self.model_url}/v1/models",
+                headers=self._build_headers(),
+                timeout=30.0,
+            )
+            response.raise_for_status()
+            return response.json()

src/server.py

@@ -0,0 +1,203 @@
+"""HTTP server and CLI for petsitter."""
+
+import json
+import logging
+import os
+from typing import Any
+
+import click
+import uvicorn
+from starlette.applications import Starlette
+from starlette.requests import Request
+from starlette.responses import JSONResponse, Response, StreamingResponse
+
+from src.loader import load_tricks
+from src.proxy import ProxyHandler
+from src.trick import Trick
+
+
+def create_app(
+    model_url: str,
+    model_name: str | None,
+    api_key: str,
+    trick_paths: list[str],
+) -> Starlette:
+    """Create the petsitter Starlette application.
+
+    Args:
+        model_url: Base URL of the upstream model.
+        model_name: Optional model name override.
+        api_key: API key for upstream.
+        trick_paths: List of trick file paths.
+
+    Returns:
+        Configured Starlette app.
+    """
+    tricks = load_tricks(trick_paths) if trick_paths else []
+    handler = ProxyHandler(model_url, model_name, api_key, tricks)
+
+    app = Starlette()
+
+    async def stream_chat_completions(handler: ProxyHandler, payload: dict):
+        """Stream chat completions as SSE events in OpenAI format."""
+        try:
+            result = await handler.chat_completions(payload)
+            
+            # Convert to streaming format with delta instead of message
+            message = result["choices"][0]["message"]
+            
+            # Build streaming response with delta
+            stream_result = {
+                "id": result.get("id", "chatcmpl-petsitter"),
+                "object": "chat.completion.chunk",
+                "created": result.get("created", __import__("time").time()),
+                "model": result.get("model", "unknown"),
+                "choices": [{
+                    "index": 0,
+                    "delta": {
+                        "role": "assistant",
+                        "content": message.get("content"),
+                    },
+                    "finish_reason": result["choices"][0].get("finish_reason", "stop"),
+                }],
+            }
+            
+            # Add tool_calls to delta if present
+            if "tool_calls" in message:
+                stream_result["choices"][0]["delta"]["tool_calls"] = message["tool_calls"]
+            
+            yield f"data: {json.dumps(stream_result)}\n\n"
+            yield "data: [DONE]\n\n"
+        except Exception as e:
+            import traceback
+            tb = traceback.format_exc()
+            click.echo(f"ERROR in stream_chat_completions: {e}")
+            click.echo(tb)
+            error_data = {
+                "error": {"message": str(e), "type": "proxy_error"}
+            }
+            yield f"data: {json.dumps(error_data)}\n\n"
+
+    @app.route("/v1/chat/completions", methods=["POST"])
+    async def chat_completions(request: Request) -> Response:
+        """Proxy chat completions to upstream model."""
+        try:
+            payload = await request.json()
+            stream = payload.get("stream", False)
+            
+            if stream:
+                return StreamingResponse(
+                    stream_chat_completions(handler, payload),
+                    media_type="text/event-stream",
+                )
+            else:
+                result = await handler.chat_completions(payload)
+                return JSONResponse(result)
+        except Exception as e:
+            import traceback
+            tb = traceback.format_exc()
+            click.echo(f"ERROR in chat_completions: {e}")
+            click.echo(tb)
+            return JSONResponse(
+                {"error": {"message": str(e), "type": "proxy_error", "traceback": tb}},
+                status_code=500,
+            )
+
+    @app.route("/v1/models", methods=["GET"])
+    async def models(request: Request) -> Response:
+        """Proxy models listing to upstream."""
+        try:
+            result = await handler.models()
+            return JSONResponse(result)
+        except Exception as e:
+            import traceback
+            tb = traceback.format_exc()
+            click.echo(f"ERROR in models: {e}")
+            click.echo(tb)
+            return JSONResponse(
+                {"error": {"message": str(e), "type": "proxy_error", "traceback": tb}},
+                status_code=500,
+            )
+
+    @app.route("/health", methods=["GET"])
+    async def health(request: Request) -> Response:
+        """Health check endpoint."""
+        return JSONResponse({"status": "ok"})
+
+    return app
+
+
+@click.command()
+@click.option(
+    "--model_url",
+    required=True,
+    help="Base URL of the upstream model (e.g., http://localhost:11434)",
+)
+@click.option(
+    "--model_name",
+    default=None,
+    help="Model name to use (optional for some backends like vllm, sglang)",
+)
+@click.option(
+    "--api_key",
+    default="",
+    help="API key for upstream (if required)",
+)
+@click.option(
+    "--trick",
+    "tricks",
+    multiple=True,
+    help="Path to a trick module (can be specified multiple times)",
+)
+@click.option(
+    "--listen_on",
+    default="localhost:8080",
+    help="Host:port to listen on (default: localhost:8080)",
+)
+def cli(
+    model_url: str,
+    model_name: str | None,
+    api_key: str,
+    tricks: tuple[str, ...],
+    listen_on: str,
+) -> None:
+    """Petsitter - OpenAI-compatible proxy with tricks.
+
+    Example:
+
+    \b
+        petsitter --model_url http://localhost:11434 \\
+                  --model_name llama3:8b \\
+                  --trick tricks/tool_call.py \\
+                  --trick tricks/json_mode.py \\
+                  --listen_on localhost:8080
+    """
+    # Parse listen_on
+    if ":" in listen_on:
+        host, port_str = listen_on.rsplit(":", 1)
+        port = int(port_str)
+    else:
+        host = listen_on
+        port = 8080
+
+    app = create_app(model_url, model_name, api_key, list(tricks))
+
+    click.echo(f"Starting petsitter on {host}:{port}")
+    click.echo(f"Upstream: {model_url}")
+    if model_name:
+        click.echo(f"Model: {model_name}")
+    if tricks:
+        click.echo(f"Tricks: {', '.join(tricks)}")
+
+    # Configure logging from environment
+    log_level = os.getenv("LOGLEVEL", "INFO").upper()
+    logging.basicConfig(
+        level=getattr(logging, log_level, logging.INFO),
+        format="%(levelname)s: %(message)s"
+    )
+
+    uvicorn.run(app, host=host, port=port)
+
+
+if __name__ == "__main__":
+    cli()

src/trick.py

@@ -0,0 +1,114 @@
+"""Base Trick class and callmodel utility for petsitter."""
+
+import json
+from typing import Any
+
+import httpx
+
+
+class Trick:
+    """Base class for all petsitter tricks.
+
+    Subclass this and implement any of the hooks to add functionality.
+    """
+
+    def system_prompt(self, to_add: str) -> str:
+        """Add instructions to the system prompt.
+
+        Args:
+            to_add: The current system prompt content.
+
+        Returns:
+            Modified system prompt content.
+        """
+        return ""
+
+    def pre_hook(self, context: list, params: dict) -> list:
+        """Modify context before it reaches the model.
+
+        Args:
+            context: The conversation context (list of messages).
+            params: Request parameters (tools, model, etc.).
+
+        Returns:
+            Modified context.
+        """
+        return context
+
+    def post_hook(self, context: list) -> list:
+        """Modify context after model processes but before returning upstream.
+
+        Args:
+            context: The conversation context including model response.
+
+        Returns:
+            Modified context.
+        """
+        return context
+
+    def info(self, capabilities: dict) -> dict:
+        """Declare capabilities added by this trick.
+
+        Args:
+            capabilities: Current capabilities dict.
+
+        Returns:
+            Modified capabilities dict.
+        """
+        return capabilities
+
+
+async def callmodel(
+    context: list,
+    instruction: str = "",
+    model_url: str = "",
+    model_name: str = "",
+    api_key: str = "",
+) -> list:
+    """Make a follow-up call to the model.
+
+    Used by tricks that need to retry or refine model output.
+
+    Args:
+        context: Current conversation context.
+        instruction: Optional system instruction to append.
+        model_url: Base URL of the model endpoint.
+        model_name: Name of the model to use.
+        api_key: API key if required.
+
+    Returns:
+        Updated context with model response.
+    """
+    if not model_url:
+        raise ValueError("model_url is required for callmodel")
+
+    messages = context.copy()
+    if instruction:
+        # Add instruction as system message or append to existing
+        if messages and messages[0].get("role") == "system":
+            messages[0]["content"] += f"\n{instruction}"
+        else:
+            messages.insert(0, {"role": "system", "content": instruction})
+
+    payload = {
+        "model": model_name or "default",
+        "messages": messages,
+    }
+
+    headers = {"Content-Type": "application/json"}
+    if api_key:
+        headers["Authorization"] = f"Bearer {api_key}"
+
+    async with httpx.AsyncClient() as client:
+        response = await client.post(
+            f"{model_url}/v1/chat/completions",
+            json=payload,
+            headers=headers,
+            timeout=60.0,
+        )
+        response.raise_for_status()
+        result = response.json()
+
+    # Extract the assistant's response
+    assistant_message = result["choices"][0]["message"]
+    return context + [assistant_message]