sg-reranker 0.1.0__tar.gz

sg_reranker-0.1.0/PKG-INFO

@@ -0,0 +1,214 @@
+Metadata-Version: 2.4
+Name: sg-reranker
+Version: 0.1.0
+Summary: Lightweight client for the SG reranker service
+License: Proprietary
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+
+# BGE Reranker v2-m3 — Docker Service
+
+A production-ready CPU reranker microservice running
+[BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3)
+via ONNX Runtime (INT8 dynamic quantization).
+
+Tuned for a GCP **n2-highmem-2** VM: 2 vCPUs, 16 GB RAM.
+
+---
+
+## Quick start
+
+### 1 — Build
+
+```bash
+docker compose build
+```
+
+The build downloads the model from Hugging Face and bakes the quantized ONNX
+file into the image (≈ 5–15 min, depending on network speed and CPU).
+Subsequent rebuilds are fast because Docker caches the conversion layer as long
+as `requirements.txt` and `app/convert.py` are unchanged.
+
+### 2 — Configure (optional)
+
+```bash
+cp .env.example .env
+# Edit .env to set API_KEY, MAX_LENGTH, thread counts, etc.
+```
+
+### 3 — Run
+
+```bash
+docker compose up -d
+docker compose logs -f   # watch startup + warmup
+```
+
+The service is ready when you see **"Model ready — serving on port 8000"** in
+the logs (usually 10–30 s after container start).
+
+### 4 — Health check
+
+```bash
+curl http://localhost:8000/health
+# {"status":"ok","model":"BAAI/bge-reranker-v2-m3","max_length":512}
+```
+
+### 5 — Rerank
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query": "What is machine learning?",
+    "documents": [
+      "Machine learning enables systems to learn from data without explicit programming.",
+      "Python is a popular general-purpose programming language.",
+      "Deep learning models complex patterns using multi-layer neural networks."
+    ],
+    "top_n": 2,
+    "return_documents": true
+  }' | python3 -m json.tool
+```
+
+With bearer-token auth enabled (`API_KEY` set in `.env`):
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $API_KEY" \
+  -d '{...}'
+```
+
+### 6 — Python client
+
+```bash
+# against local container
+python client.py
+
+# against a remote VM
+RERANKER_URL=http://10.0.0.5:8000 API_KEY=secret python client.py
+```
+
+---
+
+## API reference
+
+### `GET /health`
+
+```json
+{"status": "ok", "model": "BAAI/bge-reranker-v2-m3", "max_length": 512}
+```
+
+### `POST /rerank`
+
+**Request body**
+
+| Field              | Type           | Default | Description                              |
+|--------------------|----------------|---------|------------------------------------------|
+| `query`            | string         | —       | The search query                         |
+| `documents`        | list of string | —       | Candidate documents to score             |
+| `top_n`            | int \| null    | null    | Return only the top N results            |
+| `max_length`       | int \| null    | null    | Token limit per pair (env var fallback)  |
+| `return_documents` | bool           | true    | Include document text in the response    |
+
+**Response**
+
+```json
+{
+  "results": [
+    {"index": 0, "score": 0.9821, "document": "Machine learning enables …"},
+    {"index": 2, "score": 0.7634, "document": "Deep learning models …"}
+  ],
+  "model": "BAAI/bge-reranker-v2-m3"
+}
+```
+
+Results are sorted by `score` descending. `index` refers to the original
+position in the `documents` list so you can map back to your data.
+
+---
+
+## Reaching the service from another machine
+
+There are three common approaches; choose based on your threat model.
+
+### Option A — Same-VPC internal IP (recommended for GCP)
+
+If your client runs in the same GCP VPC as the VM, use the VM's **internal
+IP** directly. No firewall rule is needed because traffic stays on the private
+network.
+
+```bash
+RERANKER_URL=http://10.128.0.X:8000 python client.py
+```
+
+Find the internal IP: `gcloud compute instances describe <VM_NAME> --format='get(networkInterfaces[0].networkIP)'`
+
+### Option B — External IP with a locked-down firewall rule
+
+Create a GCP firewall rule that allows TCP port 8000 only from specific source
+IP ranges (your office CIDR, a bastion IP, etc.):
+
+```bash
+gcloud compute firewall-rules create allow-reranker \
+  --direction=INGRESS \
+  --action=ALLOW \
+  --rules=tcp:8000 \
+  --source-ranges=<YOUR_CIDR> \
+  --target-tags=reranker-vm
+```
+
+Then tag your VM and hit it on its external IP:
+
+```bash
+RERANKER_URL=http://<EXTERNAL_IP>:8000 python client.py
+```
+
+> **Warning** — plain HTTP is unencrypted.  Anyone on the network path can
+> read requests and responses, including the documents you are reranking.
+> Use this option only within a trusted network or with TLS termination
+> (e.g. a load balancer or nginx with a certificate).
+
+### Option C — SSH tunnel (for local development)
+
+Forward a local port to the service through an encrypted SSH session.  No
+firewall rule is needed and no traffic leaves the tunnel unencrypted.
+
+```bash
+# In one terminal — keep this open
+gcloud compute ssh <VM_NAME> -- -N -L 8000:localhost:8000
+
+# In another terminal
+curl http://localhost:8000/health
+python client.py          # uses http://localhost:8000 by default
+```
+
+---
+
+## Environment variables
+
+| Variable          | Default                         | Description                              |
+|-------------------|---------------------------------|------------------------------------------|
+| `MODEL_DIR`       | `/models/onnx_reranker_quant`   | Path to the quantized ONNX model dir     |
+| `MAX_LENGTH`      | `512`                           | Default token limit per query+doc pair   |
+| `OMP_NUM_THREADS` | `2`                             | PyTorch / OpenBLAS thread count          |
+| `ORT_NUM_THREADS` | `2`                             | ONNX Runtime intra-op thread count       |
+| `API_KEY`         | _(unset = auth disabled)_       | Bearer token for `/rerank`               |
+
+---
+
+## Notes
+
+- **Single Uvicorn worker** is intentional.  Each worker loads a full copy of
+  the model.  On a 2-vCPU VM a second worker would double RAM usage and cause
+  thread contention rather than improve throughput.
+- **INT8 quantization** cuts model size by ~3× and speeds up matrix
+  multiplications on CPUs that support AVX2 or AVX-512 VNNI (most modern Intel
+  and AMD cores).  Accuracy loss on typical reranking benchmarks is < 1%.
+- **TLS** — this service speaks plain HTTP.  For production deployments on the
+  public internet, terminate TLS at a load balancer or a reverse proxy (nginx,
+  Caddy) and keep the container on an internal network.

sg_reranker-0.1.0/README.md

@@ -0,0 +1,202 @@
+# BGE Reranker v2-m3 — Docker Service
+
+A production-ready CPU reranker microservice running
+[BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3)
+via ONNX Runtime (INT8 dynamic quantization).
+
+Tuned for a GCP **n2-highmem-2** VM: 2 vCPUs, 16 GB RAM.
+
+---
+
+## Quick start
+
+### 1 — Build
+
+```bash
+docker compose build
+```
+
+The build downloads the model from Hugging Face and bakes the quantized ONNX
+file into the image (≈ 5–15 min, depending on network speed and CPU).
+Subsequent rebuilds are fast because Docker caches the conversion layer as long
+as `requirements.txt` and `app/convert.py` are unchanged.
+
+### 2 — Configure (optional)
+
+```bash
+cp .env.example .env
+# Edit .env to set API_KEY, MAX_LENGTH, thread counts, etc.
+```
+
+### 3 — Run
+
+```bash
+docker compose up -d
+docker compose logs -f   # watch startup + warmup
+```
+
+The service is ready when you see **"Model ready — serving on port 8000"** in
+the logs (usually 10–30 s after container start).
+
+### 4 — Health check
+
+```bash
+curl http://localhost:8000/health
+# {"status":"ok","model":"BAAI/bge-reranker-v2-m3","max_length":512}
+```
+
+### 5 — Rerank
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query": "What is machine learning?",
+    "documents": [
+      "Machine learning enables systems to learn from data without explicit programming.",
+      "Python is a popular general-purpose programming language.",
+      "Deep learning models complex patterns using multi-layer neural networks."
+    ],
+    "top_n": 2,
+    "return_documents": true
+  }' | python3 -m json.tool
+```
+
+With bearer-token auth enabled (`API_KEY` set in `.env`):
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $API_KEY" \
+  -d '{...}'
+```
+
+### 6 — Python client
+
+```bash
+# against local container
+python client.py
+
+# against a remote VM
+RERANKER_URL=http://10.0.0.5:8000 API_KEY=secret python client.py
+```
+
+---
+
+## API reference
+
+### `GET /health`
+
+```json
+{"status": "ok", "model": "BAAI/bge-reranker-v2-m3", "max_length": 512}
+```
+
+### `POST /rerank`
+
+**Request body**
+
+| Field              | Type           | Default | Description                              |
+|--------------------|----------------|---------|------------------------------------------|
+| `query`            | string         | —       | The search query                         |
+| `documents`        | list of string | —       | Candidate documents to score             |
+| `top_n`            | int \| null    | null    | Return only the top N results            |
+| `max_length`       | int \| null    | null    | Token limit per pair (env var fallback)  |
+| `return_documents` | bool           | true    | Include document text in the response    |
+
+**Response**
+
+```json
+{
+  "results": [
+    {"index": 0, "score": 0.9821, "document": "Machine learning enables …"},
+    {"index": 2, "score": 0.7634, "document": "Deep learning models …"}
+  ],
+  "model": "BAAI/bge-reranker-v2-m3"
+}
+```
+
+Results are sorted by `score` descending. `index` refers to the original
+position in the `documents` list so you can map back to your data.
+
+---
+
+## Reaching the service from another machine
+
+There are three common approaches; choose based on your threat model.
+
+### Option A — Same-VPC internal IP (recommended for GCP)
+
+If your client runs in the same GCP VPC as the VM, use the VM's **internal
+IP** directly. No firewall rule is needed because traffic stays on the private
+network.
+
+```bash
+RERANKER_URL=http://10.128.0.X:8000 python client.py
+```
+
+Find the internal IP: `gcloud compute instances describe <VM_NAME> --format='get(networkInterfaces[0].networkIP)'`
+
+### Option B — External IP with a locked-down firewall rule
+
+Create a GCP firewall rule that allows TCP port 8000 only from specific source
+IP ranges (your office CIDR, a bastion IP, etc.):
+
+```bash
+gcloud compute firewall-rules create allow-reranker \
+  --direction=INGRESS \
+  --action=ALLOW \
+  --rules=tcp:8000 \
+  --source-ranges=<YOUR_CIDR> \
+  --target-tags=reranker-vm
+```
+
+Then tag your VM and hit it on its external IP:
+
+```bash
+RERANKER_URL=http://<EXTERNAL_IP>:8000 python client.py
+```
+
+> **Warning** — plain HTTP is unencrypted.  Anyone on the network path can
+> read requests and responses, including the documents you are reranking.
+> Use this option only within a trusted network or with TLS termination
+> (e.g. a load balancer or nginx with a certificate).
+
+### Option C — SSH tunnel (for local development)
+
+Forward a local port to the service through an encrypted SSH session.  No
+firewall rule is needed and no traffic leaves the tunnel unencrypted.
+
+```bash
+# In one terminal — keep this open
+gcloud compute ssh <VM_NAME> -- -N -L 8000:localhost:8000
+
+# In another terminal
+curl http://localhost:8000/health
+python client.py          # uses http://localhost:8000 by default
+```
+
+---
+
+## Environment variables
+
+| Variable          | Default                         | Description                              |
+|-------------------|---------------------------------|------------------------------------------|
+| `MODEL_DIR`       | `/models/onnx_reranker_quant`   | Path to the quantized ONNX model dir     |
+| `MAX_LENGTH`      | `512`                           | Default token limit per query+doc pair   |
+| `OMP_NUM_THREADS` | `2`                             | PyTorch / OpenBLAS thread count          |
+| `ORT_NUM_THREADS` | `2`                             | ONNX Runtime intra-op thread count       |
+| `API_KEY`         | _(unset = auth disabled)_       | Bearer token for `/rerank`               |
+
+---
+
+## Notes
+
+- **Single Uvicorn worker** is intentional.  Each worker loads a full copy of
+  the model.  On a 2-vCPU VM a second worker would double RAM usage and cause
+  thread contention rather than improve throughput.
+- **INT8 quantization** cuts model size by ~3× and speeds up matrix
+  multiplications on CPUs that support AVX2 or AVX-512 VNNI (most modern Intel
+  and AMD cores).  Accuracy loss on typical reranking benchmarks is < 1%.
+- **TLS** — this service speaks plain HTTP.  For production deployments on the
+  public internet, terminate TLS at a load balancer or a reverse proxy (nginx,
+  Caddy) and keep the container on an internal network.

sg_reranker-0.1.0/pyproject.toml

@@ -0,0 +1,25 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sg-reranker"
+version = "0.1.0"
+description = "Lightweight client for the SG reranker service"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "Proprietary" }
+dependencies = ["requests>=2.28"]
+
+[project.optional-dependencies]
+dev = ["pytest>=7", "pytest-cov"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+namespaces = true   # enables implicit namespace packages (sg.reranker, sg.embeddings, ...)
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.coverage.run]
+source = ["src"]

sg_reranker-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

sg_reranker-0.1.0/src/sg/reranker/__init__.py

@@ -0,0 +1,7 @@
+from importlib.metadata import version
+
+from ._client import Reranker
+from ._models import RankedDocument, RerankResult
+
+__version__ = version("sg-reranker")
+__all__ = ["Reranker", "RankedDocument", "RerankResult", "__version__"]

sg_reranker-0.1.0/src/sg/reranker/_base.py

@@ -0,0 +1,47 @@
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from typing import TYPE_CHECKING, Optional
+
+import requests
+
+if TYPE_CHECKING:
+    from ._models import RerankResult
+
+
+class SGServiceClient(ABC):
+    """Base for all SG service clients. Inject a custom session for testing or connection pooling."""
+
+    def __init__(
+        self,
+        base_url: str,
+        api_key: Optional[str] = None,
+        session: Optional[requests.Session] = None,
+    ) -> None:
+        self._base_url = base_url.rstrip("/")
+        self._api_key = api_key
+        self._session = session or requests.Session()
+
+    def _headers(self) -> dict[str, str]:
+        h = {"Content-Type": "application/json"}
+        if self._api_key:
+            h["Authorization"] = f"Bearer {self._api_key}"
+        return h
+
+    @abstractmethod
+    def health(self) -> dict: ...
+
+
+class InferenceInterface(ABC):
+    """Contract for inference services. Implement this to add a new inference backend."""
+
+    @abstractmethod
+    def rerank(
+        self,
+        model: str,
+        query: str,
+        documents: list[str],
+        top_n: Optional[int] = None,
+        return_documents: bool = True,
+        max_length: Optional[int] = None,
+    ) -> "RerankResult": ...

sg_reranker-0.1.0/src/sg/reranker/_client.py

@@ -0,0 +1,87 @@
+from __future__ import annotations
+
+from typing import Optional
+
+import requests
+
+from ._base import InferenceInterface, SGServiceClient
+from ._models import RankedDocument, RerankResult
+
+
+class _Inference(InferenceInterface):
+    def __init__(self, client: Reranker) -> None:
+        self._client = client
+
+    def rerank(
+        self,
+        model: str,
+        query: str,
+        documents: list[str],
+        top_n: Optional[int] = None,
+        return_documents: bool = True,
+        max_length: Optional[int] = None,
+    ) -> RerankResult:
+        payload: dict = {
+            "query": query,
+            "documents": documents,
+            "return_documents": return_documents,
+        }
+        if top_n is not None:
+            payload["top_n"] = top_n
+        if max_length is not None:
+            payload["max_length"] = max_length
+
+        resp = self._client._session.post(
+            f"{self._client._base_url}/rerank",
+            json=payload,
+            headers=self._client._headers(),
+            timeout=30,
+        )
+        resp.raise_for_status()
+        body = resp.json()
+
+        return RerankResult(
+            model=body["model"],
+            data=[
+                RankedDocument(
+                    index=r["index"],
+                    score=r["score"],
+                    document={"text": r["document"]} if r.get("document") else None,
+                )
+                for r in body["results"]
+            ],
+        )
+
+
+class Reranker(SGServiceClient):
+    """
+    Client for the BGE reranker service.
+
+        client = Reranker("http://34.91.24.7:8004", api_key="secret")
+        results = client.inference.rerank(
+            model="bge-reranker-v2-m3",
+            query="What is ML?",
+            documents=[...],
+            top_n=3,
+        )
+        for r in results.data:
+            print(r.index, r.score, r.document["text"])
+    """
+
+    def __init__(
+        self,
+        base_url: str = "http://localhost:8004",
+        api_key: Optional[str] = None,
+        session: Optional[requests.Session] = None,
+    ) -> None:
+        super().__init__(base_url, api_key, session)
+        self.inference = _Inference(self)
+
+    def health(self) -> dict:
+        resp = self._session.get(
+            f"{self._base_url}/health",
+            headers=self._headers(),
+            timeout=10,
+        )
+        resp.raise_for_status()
+        return resp.json()

sg_reranker-0.1.0/src/sg/reranker/_models.py

@@ -0,0 +1,17 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Optional
+
+
+@dataclass
+class RankedDocument:
+    index: int
+    score: float
+    document: Optional[dict]  # {"text": "..."} when return_documents=True, else None
+
+
+@dataclass
+class RerankResult:
+    data: list[RankedDocument]
+    model: str

sg_reranker-0.1.0/src/sg_reranker.egg-info/PKG-INFO

@@ -0,0 +1,214 @@
+Metadata-Version: 2.4
+Name: sg-reranker
+Version: 0.1.0
+Summary: Lightweight client for the SG reranker service
+License: Proprietary
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+
+# BGE Reranker v2-m3 — Docker Service
+
+A production-ready CPU reranker microservice running
+[BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3)
+via ONNX Runtime (INT8 dynamic quantization).
+
+Tuned for a GCP **n2-highmem-2** VM: 2 vCPUs, 16 GB RAM.
+
+---
+
+## Quick start
+
+### 1 — Build
+
+```bash
+docker compose build
+```
+
+The build downloads the model from Hugging Face and bakes the quantized ONNX
+file into the image (≈ 5–15 min, depending on network speed and CPU).
+Subsequent rebuilds are fast because Docker caches the conversion layer as long
+as `requirements.txt` and `app/convert.py` are unchanged.
+
+### 2 — Configure (optional)
+
+```bash
+cp .env.example .env
+# Edit .env to set API_KEY, MAX_LENGTH, thread counts, etc.
+```
+
+### 3 — Run
+
+```bash
+docker compose up -d
+docker compose logs -f   # watch startup + warmup
+```
+
+The service is ready when you see **"Model ready — serving on port 8000"** in
+the logs (usually 10–30 s after container start).
+
+### 4 — Health check
+
+```bash
+curl http://localhost:8000/health
+# {"status":"ok","model":"BAAI/bge-reranker-v2-m3","max_length":512}
+```
+
+### 5 — Rerank
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query": "What is machine learning?",
+    "documents": [
+      "Machine learning enables systems to learn from data without explicit programming.",
+      "Python is a popular general-purpose programming language.",
+      "Deep learning models complex patterns using multi-layer neural networks."
+    ],
+    "top_n": 2,
+    "return_documents": true
+  }' | python3 -m json.tool
+```
+
+With bearer-token auth enabled (`API_KEY` set in `.env`):
+
+```bash
+curl -s -X POST http://localhost:8000/rerank \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $API_KEY" \
+  -d '{...}'
+```
+
+### 6 — Python client
+
+```bash
+# against local container
+python client.py
+
+# against a remote VM
+RERANKER_URL=http://10.0.0.5:8000 API_KEY=secret python client.py
+```
+
+---
+
+## API reference
+
+### `GET /health`
+
+```json
+{"status": "ok", "model": "BAAI/bge-reranker-v2-m3", "max_length": 512}
+```
+
+### `POST /rerank`
+
+**Request body**
+
+| Field              | Type           | Default | Description                              |
+|--------------------|----------------|---------|------------------------------------------|
+| `query`            | string         | —       | The search query                         |
+| `documents`        | list of string | —       | Candidate documents to score             |
+| `top_n`            | int \| null    | null    | Return only the top N results            |
+| `max_length`       | int \| null    | null    | Token limit per pair (env var fallback)  |
+| `return_documents` | bool           | true    | Include document text in the response    |
+
+**Response**
+
+```json
+{
+  "results": [
+    {"index": 0, "score": 0.9821, "document": "Machine learning enables …"},
+    {"index": 2, "score": 0.7634, "document": "Deep learning models …"}
+  ],
+  "model": "BAAI/bge-reranker-v2-m3"
+}
+```
+
+Results are sorted by `score` descending. `index` refers to the original
+position in the `documents` list so you can map back to your data.
+
+---
+
+## Reaching the service from another machine
+
+There are three common approaches; choose based on your threat model.
+
+### Option A — Same-VPC internal IP (recommended for GCP)
+
+If your client runs in the same GCP VPC as the VM, use the VM's **internal
+IP** directly. No firewall rule is needed because traffic stays on the private
+network.
+
+```bash
+RERANKER_URL=http://10.128.0.X:8000 python client.py
+```
+
+Find the internal IP: `gcloud compute instances describe <VM_NAME> --format='get(networkInterfaces[0].networkIP)'`
+
+### Option B — External IP with a locked-down firewall rule
+
+Create a GCP firewall rule that allows TCP port 8000 only from specific source
+IP ranges (your office CIDR, a bastion IP, etc.):
+
+```bash
+gcloud compute firewall-rules create allow-reranker \
+  --direction=INGRESS \
+  --action=ALLOW \
+  --rules=tcp:8000 \
+  --source-ranges=<YOUR_CIDR> \
+  --target-tags=reranker-vm
+```
+
+Then tag your VM and hit it on its external IP:
+
+```bash
+RERANKER_URL=http://<EXTERNAL_IP>:8000 python client.py
+```
+
+> **Warning** — plain HTTP is unencrypted.  Anyone on the network path can
+> read requests and responses, including the documents you are reranking.
+> Use this option only within a trusted network or with TLS termination
+> (e.g. a load balancer or nginx with a certificate).
+
+### Option C — SSH tunnel (for local development)
+
+Forward a local port to the service through an encrypted SSH session.  No
+firewall rule is needed and no traffic leaves the tunnel unencrypted.
+
+```bash
+# In one terminal — keep this open
+gcloud compute ssh <VM_NAME> -- -N -L 8000:localhost:8000
+
+# In another terminal
+curl http://localhost:8000/health
+python client.py          # uses http://localhost:8000 by default
+```
+
+---
+
+## Environment variables
+
+| Variable          | Default                         | Description                              |
+|-------------------|---------------------------------|------------------------------------------|
+| `MODEL_DIR`       | `/models/onnx_reranker_quant`   | Path to the quantized ONNX model dir     |
+| `MAX_LENGTH`      | `512`                           | Default token limit per query+doc pair   |
+| `OMP_NUM_THREADS` | `2`                             | PyTorch / OpenBLAS thread count          |
+| `ORT_NUM_THREADS` | `2`                             | ONNX Runtime intra-op thread count       |
+| `API_KEY`         | _(unset = auth disabled)_       | Bearer token for `/rerank`               |
+
+---
+
+## Notes
+
+- **Single Uvicorn worker** is intentional.  Each worker loads a full copy of
+  the model.  On a 2-vCPU VM a second worker would double RAM usage and cause
+  thread contention rather than improve throughput.
+- **INT8 quantization** cuts model size by ~3× and speeds up matrix
+  multiplications on CPUs that support AVX2 or AVX-512 VNNI (most modern Intel
+  and AMD cores).  Accuracy loss on typical reranking benchmarks is < 1%.
+- **TLS** — this service speaks plain HTTP.  For production deployments on the
+  public internet, terminate TLS at a load balancer or a reverse proxy (nginx,
+  Caddy) and keep the container on an internal network.

sg_reranker-0.1.0/src/sg_reranker.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+README.md
+pyproject.toml
+src/sg/reranker/__init__.py
+src/sg/reranker/_base.py
+src/sg/reranker/_client.py
+src/sg/reranker/_models.py
+src/sg_reranker.egg-info/PKG-INFO
+src/sg_reranker.egg-info/SOURCES.txt
+src/sg_reranker.egg-info/dependency_links.txt
+src/sg_reranker.egg-info/requires.txt
+src/sg_reranker.egg-info/top_level.txt
+tests/test_reranker.py

sg_reranker-0.1.0/src/sg_reranker.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

sg_reranker-0.1.0/src/sg_reranker.egg-info/requires.txt

@@ -0,0 +1,5 @@
+requests>=2.28
+
+[dev]
+pytest>=7
+pytest-cov

sg_reranker-0.1.0/src/sg_reranker.egg-info/top_level.txt

@@ -0,0 +1 @@
+sg

sg_reranker-0.1.0/tests/test_reranker.py

@@ -0,0 +1,87 @@
+from unittest.mock import patch
+import pytest
+from sg.reranker import Reranker, RankedDocument, RerankResult
+from conftest import mock_response, RERANK_RESPONSE, HEALTH_RESPONSE
+
+DOCS = [
+    "Machine learning is a subset of AI.",
+    "Paris is the capital of France.",
+    "Deep learning uses neural networks.",
+]
+QUERY = "What is machine learning?"
+
+
+class TestHealth:
+    def test_returns_status_ok(self, client):
+        with patch.object(client._session, "get", return_value=mock_response(HEALTH_RESPONSE)):
+            assert client.health()["status"] == "ok"
+
+    def test_calls_correct_endpoint(self, client):
+        with patch.object(client._session, "get", return_value=mock_response(HEALTH_RESPONSE)) as mock_get:
+            client.health()
+        mock_get.assert_called_once_with(
+            "http://test-server:8004/health",
+            headers={"Content-Type": "application/json"},
+            timeout=10,
+        )
+
+    def test_raises_on_server_error(self, client):
+        resp = mock_response({}, status_code=503)
+        resp.raise_for_status.side_effect = Exception("Service unavailable")
+        with patch.object(client._session, "get", return_value=resp):
+            with pytest.raises(Exception, match="Service unavailable"):
+                client.health()
+
+
+class TestRerank:
+    def test_returns_rerank_result(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)):
+            result = client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)
+        assert isinstance(result, RerankResult)
+        assert result.model == "BAAI/bge-reranker-v2-m3"
+
+    def test_data_contains_ranked_documents(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)):
+            result = client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)
+        assert len(result.data) == 3
+        assert all(isinstance(r, RankedDocument) for r in result.data)
+
+    def test_document_text_wrapped_in_dict(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)):
+            result = client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)
+        assert result.data[0].document == {"text": "Machine learning is a subset of AI."}
+
+    def test_top_n_sent_in_payload(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)) as mock_post:
+            client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS, top_n=2)
+        assert mock_post.call_args.kwargs["json"]["top_n"] == 2
+
+    def test_max_length_sent_in_payload(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)) as mock_post:
+            client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS, max_length=256)
+        assert mock_post.call_args.kwargs["json"]["max_length"] == 256
+
+    def test_return_documents_false_omits_text(self, client):
+        response = {"model": "BAAI/bge-reranker-v2-m3", "results": [{"index": 0, "score": 0.9, "document": None}]}
+        with patch.object(client._session, "post", return_value=mock_response(response)):
+            result = client.inference.rerank(
+                model="bge-reranker-v2-m3", query=QUERY, documents=DOCS, return_documents=False
+            )
+        assert result.data[0].document is None
+
+    def test_api_key_sent_as_bearer_token(self, auth_client):
+        with patch.object(auth_client._session, "post", return_value=mock_response(RERANK_RESPONSE)) as mock_post:
+            auth_client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)
+        assert mock_post.call_args.kwargs["headers"]["Authorization"] == "Bearer secret"
+
+    def test_no_api_key_omits_auth_header(self, client):
+        with patch.object(client._session, "post", return_value=mock_response(RERANK_RESPONSE)) as mock_post:
+            client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)
+        assert "Authorization" not in mock_post.call_args.kwargs["headers"]
+
+    def test_raises_on_401(self, client):
+        resp = mock_response({}, status_code=401)
+        resp.raise_for_status.side_effect = Exception("Unauthorized")
+        with patch.object(client._session, "post", return_value=resp):
+            with pytest.raises(Exception, match="Unauthorized"):
+                client.inference.rerank(model="bge-reranker-v2-m3", query=QUERY, documents=DOCS)