stubllm 0.1.0__tar.gz

stubllm-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,70 @@
+name: CI
+
+on:
+  push:
+    branches: [main, develop]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+
+      - name: Lint with ruff
+        run: ruff check src/ tests/
+
+      - name: Type check with mypy
+        run: mypy src/mockllm --ignore-missing-imports
+        continue-on-error: true
+
+      - name: Run tests with coverage
+        run: |
+          pytest tests/ \
+            --cov=mockllm \
+            --cov-report=term-missing \
+            --cov-report=xml \
+            --cov-fail-under=80 \
+            -v
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v4
+        with:
+          file: ./coverage.xml
+          fail_ci_if_error: false
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Check package
+        run: |
+          pip install twine
+          twine check dist/*

stubllm-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 mockllm contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

stubllm-0.1.0/PKG-INFO

@@ -0,0 +1,437 @@
+Metadata-Version: 2.4
+Name: stubllm
+Version: 0.1.0
+Summary: Deterministic mock server for LLM APIs. Test your AI code without spending tokens.
+Author: mockllm contributors
+License: MIT
+License-File: LICENSE
+Keywords: anthropic,gemini,llm,mock,openai,stub,testing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Requires-Dist: anyio>=4.0.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: fastapi>=0.110.0
+Requires-Dist: httpx>=0.26.0
+Requires-Dist: pydantic>=2.5.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: uvicorn[standard]>=0.27.0
+Provides-Extra: dev
+Requires-Dist: httpx>=0.26.0; extra == 'dev'
+Requires-Dist: mypy>=1.8.0; extra == 'dev'
+Requires-Dist: openai>=1.0.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.3.0; extra == 'dev'
+Provides-Extra: pytest
+Requires-Dist: pytest-asyncio>=0.23.0; extra == 'pytest'
+Requires-Dist: pytest>=7.0; extra == 'pytest'
+Description-Content-Type: text/markdown
+
+# stubllm
+
+**Deterministic mock server for LLM APIs. Test your AI code without spending tokens.**
+
+[![CI](https://github.com/your-org/stubllm/actions/workflows/ci.yml/badge.svg)](https://github.com/your-org/stubllm/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/stubllm)](https://pypi.org/project/stubllm/)
+[![Python](https://img.shields.io/pypi/pyversions/stubllm)](https://pypi.org/project/stubllm/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+Works with: **OpenAI** · **Anthropic** · **Google Gemini**
+
+---
+
+## 30-second quickstart
+
+```bash
+# 1. Install
+pip install stubllm
+
+# 2. Create a fixture
+mkdir fixtures
+cat > fixtures/chat.yaml << 'EOF'
+fixtures:
+  - name: "greeting"
+    match:
+      provider: openai
+      messages:
+        - role: user
+          content:
+            contains: "hello"
+    response:
+      content: "Hello! How can I help you today?"
+EOF
+
+# 3. Start the server
+stubllm serve --port 8765
+
+# 4. Point your code at it
+export OPENAI_BASE_URL=http://localhost:8765/v1/
+python your_app.py  # no real API calls, no tokens spent
+```
+
+---
+
+## Why stubllm?
+
+| | stubllm | Real API | Ollama |
+|---|---|---|---|
+| Cost | Free | Paid | Free |
+| Speed | <1ms | 1-30s | 5-30s |
+| Deterministic | ✅ | ❌ | ❌ |
+| Works offline | ✅ | ❌ | ✅ |
+| No GPU needed | ✅ | ✅ | ❌ |
+| Pytest integration | ✅ | ❌ | ❌ |
+| Fixtures / record-replay | ✅ | ❌ | ❌ |
+| CI-friendly | ✅ | Slow/expensive | Heavy |
+
+---
+
+## Installation
+
+```bash
+pip install stubllm
+
+# With pytest support
+pip install "stubllm[pytest]"
+```
+
+---
+
+## Fixture format
+
+Fixtures are YAML (or JSON) files that map request patterns to responses.
+
+### Basic text response
+
+```yaml
+fixtures:
+  - name: "greeting"
+    match:
+      provider: openai          # openai | anthropic | gemini | any
+      endpoint: /v1/chat/completions   # optional
+      model: "gpt-4o"           # optional
+      messages:
+        - role: user
+          content:
+            contains: "hello"   # exact | contains | regex
+    response:
+      content: "Hello! How can I help you today?"
+      usage:
+        prompt_tokens: 10
+        completion_tokens: 12
+        total_tokens: 22
+```
+
+### Tool call response
+
+```yaml
+fixtures:
+  - name: "weather_tool"
+    match:
+      provider: openai
+      messages:
+        - role: user
+          content:
+            contains: "weather"
+      tools_present: true        # only match when tools are provided
+    response:
+      tool_calls:
+        - id: "call_abc123"
+          type: function
+          function:
+            name: "get_weather"
+            arguments: '{"location": "Amsterdam"}'
+```
+
+### Streaming with delay
+
+```yaml
+fixtures:
+  - name: "slow_story"
+    match:
+      messages:
+        - role: user
+          content:
+            contains: "story"
+    response:
+      content: "Once upon a time..."
+      stream_chunk_delay_ms: 50   # simulate realistic streaming speed
+```
+
+### Error response
+
+```yaml
+fixtures:
+  - name: "rate_limit"
+    match:
+      messages:
+        - role: user
+          content:
+            contains: "trigger_error"
+    response:
+      content: '{"error": {"message": "Rate limit exceeded", "type": "rate_limit_error"}}'
+      http_status: 429
+```
+
+### Content match strategies
+
+```yaml
+# Exact match (highest priority)
+content:
+  exact: "Hello, world!"
+
+# Substring match (case-insensitive)
+content:
+  contains: "weather"
+
+# Regular expression
+content:
+  regex: "tell me.*joke"
+```
+
+### Matching priority
+
+Higher specificity = higher priority. When multiple fixtures match, the most specific wins:
+
+1. `exact` message content (score: +10)
+2. `contains` message content (score: +5)
+3. `regex` message content (score: +4)
+4. `model` specified (score: +2)
+5. `tools_present` specified (score: +2)
+6. `provider` specified (score: +1)
+7. Fallback (no match criteria)
+
+---
+
+## Pytest plugin
+
+### Basic setup
+
+```python
+# conftest.py — nothing needed, stubllm auto-registers as a pytest plugin
+# The `stubllm_server` fixture is available automatically after installing stubllm
+```
+
+```python
+# test_my_app.py
+import openai
+from stubllm.pytest_plugin import use_fixtures
+
+@use_fixtures("fixtures/chat.yaml")
+def test_greeting(stubllm_server):
+    client = openai.OpenAI(
+        base_url=stubllm_server.openai_url,  # includes /v1/ — openai SDK needs this
+        api_key="test-key"
+    )
+    response = client.chat.completions.create(
+        model="gpt-4o",
+        messages=[{"role": "user", "content": "hello world"}]
+    )
+    assert "Hello" in response.choices[0].message.content
+    assert stubllm_server.call_count == 1
+```
+
+### Assertion helpers
+
+```python
+def test_with_assertions(stubllm_server):
+    # ... make some calls ...
+
+    # Assert specific prompt was sent
+    stubllm_server.assert_called_with_prompt("hello")
+
+    # Assert number of calls
+    assert stubllm_server.call_count == 2
+
+    # Inspect all calls
+    for call in stubllm_server.calls:
+        print(call["path"], call["body"])
+
+    # Assert last call path
+    stubllm_server.assert_last_call_path("/v1/chat/completions")
+```
+
+### Multiple fixture files
+
+```python
+@use_fixtures("fixtures/chat.yaml", "fixtures/tools.yaml")
+def test_combined(stubllm_server):
+    ...  # both fixture files are active for this test
+```
+
+---
+
+## Multi-provider support
+
+### OpenAI
+
+```python
+import openai
+
+client = openai.OpenAI(
+    base_url="http://localhost:8765/v1/",  # note: /v1/ required — the OpenAI SDK does not add it
+    api_key="test-key"
+)
+response = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": "hello"}]
+)
+```
+
+### Anthropic
+
+```python
+import anthropic
+
+client = anthropic.Anthropic(
+    base_url="http://localhost:8765",  # Anthropic SDK adds /v1/ itself
+    api_key="test-key"
+)
+message = client.messages.create(
+    model="claude-opus-4-6",
+    max_tokens=1024,
+    messages=[{"role": "user", "content": "hello"}]
+)
+```
+
+### Google Gemini
+
+```python
+import google.generativeai as genai
+
+genai.configure(
+    api_key="test-key",
+    client_options={"api_endpoint": "localhost:8765"}
+)
+model = genai.GenerativeModel("gemini-pro")
+response = model.generate_content("hello")
+```
+
+---
+
+## Streaming
+
+All providers support streaming. Fixtures work identically — streaming is controlled by the `stream: true` parameter in the request, not the fixture.
+
+```python
+# OpenAI streaming
+stream = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": "hello"}],
+    stream=True,
+)
+for chunk in stream:
+    print(chunk.choices[0].delta.content, end="", flush=True)
+```
+
+Control streaming speed in fixtures:
+```yaml
+response:
+  content: "A long streaming response..."
+  stream_chunk_delay_ms: 20   # default: 20ms between chunks
+```
+
+---
+
+## Record and replay
+
+Record real API interactions for later replay:
+
+```bash
+# Start in record mode (proxies to real OpenAI, saves fixtures)
+stubllm record \
+  --target https://api.openai.com \
+  --fixture-dir ./recorded_fixtures
+
+# Run your app against the recording proxy
+OPENAI_BASE_URL=http://localhost:8765/v1/ python your_app.py
+
+# Fixtures are saved to ./recorded_fixtures/
+ls recorded_fixtures/
+# recorded_hello_world_1706000000.yaml
+# recorded_weather_query_1706000001.yaml
+```
+
+Recorded fixtures are sanitized (API keys removed) and can be committed to your repo.
+
+---
+
+## CLI reference
+
+```bash
+# Start server (auto-loads ./fixtures/ if it exists)
+stubllm serve
+
+# Custom port and fixture directory
+stubllm serve --port 9000 --fixture-dir ./my-fixtures
+
+# Multiple fixture directories
+stubllm serve --fixture-dir ./fixtures/openai --fixture-dir ./fixtures/anthropic
+
+# Individual fixture files
+stubllm serve --fixture-file chat.yaml --fixture-file tools.yaml
+
+# Record mode
+stubllm record --target https://api.openai.com --fixture-dir ./recorded
+
+# Version
+stubllm --version
+```
+
+---
+
+## Structured output (JSON schema)
+
+When `response_format: { type: "json_schema" }` is set, stubllm validates that the fixture response is valid JSON. If it's not, it wraps the content automatically.
+
+```yaml
+fixtures:
+  - name: "structured"
+    match:
+      provider: openai
+    response:
+      content: '{"name": "Alice", "age": 30}'  # must be valid JSON
+```
+
+---
+
+## Project structure
+
+```
+stubllm/
+├── src/stubllm/
+│   ├── fixtures/       # YAML/JSON loading, Pydantic models, matching engine
+│   ├── providers/      # OpenAI, Anthropic, Gemini endpoint handlers
+│   ├── streaming/      # SSE streaming simulation
+│   ├── recorder/       # Record-and-replay proxy
+│   ├── pytest_plugin/  # pytest fixtures and @use_fixtures decorator
+│   ├── server.py       # FastAPI app factory
+│   └── cli.py          # click CLI
+├── tests/              # >80% coverage
+└── examples/           # Working examples (basic + advanced)
+```
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/your-org/stubllm
+cd stubllm
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+---
+
+## License
+
+MIT