latencyops 0.1.0__tar.gz

latencyops-0.1.0/LICENSE

@@ -0,0 +1,15 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

latencyops-0.1.0/PKG-INFO

@@ -0,0 +1,346 @@
+Metadata-Version: 2.4
+Name: latencyops
+Version: 0.1.0
+Summary: Latency measurement and quality-aware inference planning toolkit
+License-Expression: Apache-2.0
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# LatencyOps
+
+LatencyOps is a provider-neutral Python toolkit for measuring AI inference latency and selecting explainable execution policies under latency and quality constraints.
+
+It is a lightweight policy and measurement layer that composes with OpenAI-compatible model endpoints and existing infrastructure. It does not contain model weights or replace OpenAI, LiteLLM, vLLM, SGLang, Hugging Face, or a production edge gateway.
+
+## Why use LatencyOps?
+
+Most inference applications send every request to one model with one fixed execution policy. LatencyOps makes the execution decision per request using:
+
+- Deadline and optional TTFT/TPOT targets
+- Quality floor and risk class
+- Prompt and expected-output token counts
+- Difficulty and context length
+- Queue and cache pressure when runtime metrics are available
+- Provider health and observed latency/quality
+
+The planner returns an explainable execution plan, provider capability enforcement makes unsupported controls visible, and the benchmark layer measures TTFT, TPOT, end-to-end latency, p50, and p95.
+
+A single planner can serve one application, a batch of any number of analyses, or many agents in a multi-agent workflow. The host application remains responsible for task decomposition, agent memory, tools, retries, and final aggregation. Detailed architecture notes are maintained separately for internal design review and are not part of the public README.
+
+## Where LatencyOps sits
+
+```text
+Your application / agent workflow
+              |
+              | Python library import or HTTP request
+              v
++-----------------------------------------------+
+| LatencyOps public policy + measurement layer  |
+| planner | capability enforcement | routing    |
+| TTFT | TPOT | end-to-end | content-free data |
++----------------------+------------------------+
+                       |
+                       v
+       OpenAI-compatible provider endpoint
+   OpenAI | LiteLLM | vLLM | SGLang | Hugging Face
+                       |
+                       v
+                 model response
+
+Optional runtime metrics from vLLM, SGLang, or
+TensorRT-LLM feed queue/cache/health signals back
+into the LatencyOps planner.
+```
+
+LatencyOps sits between the application or agent workflow and the model-serving endpoint. It can run in-process as a Python library or as an internal gateway; it is not the model server and it is not an agent framework.
+
+## Core capabilities
+
+- Request latency and quality contracts
+- Content-free request profiles and telemetry
+- Conservative model-tier and precision planning
+- Long-context, cache-pressure, speculation, and early-exit policy signals
+- Provider-neutral routing and adaptive provider selection
+- OpenAI-compatible completion and chat adapters
+- Streaming TTFT and TPOT measurement
+- Capability-aware requested-versus-enforced plans
+- Optional vLLM, SGLang, and TensorRT-LLM runtime signal normalization
+- Static-baseline comparisons and workload benchmarking
+- Optional quality outcome hooks and safe re-planning
+- Local reference gateway for Python and non-Python clients
+
+The current release is a dependency-light alpha/reference implementation suitable for development and controlled integration testing. It is not a hardened production serving platform.
+
+See also:
+
+- [`COMPARISON_WITH_EXISTING_TOOLS.md`](COMPARISON_WITH_EXISTING_TOOLS.md) for market comparison and community/commercial boundaries
+- [`SECURITY.md`](SECURITY.md) for safe operation and reporting
+- [`CONTRIBUTING.md`](CONTRIBUTING.md) for development and release verification
+- [`DEPLOYMENT.md`](DEPLOYMENT.md) for the reference gateway deployment model
+
+## Install and run
+
+LatencyOps requires Python 3.12 or newer.
+
+### From a GitHub checkout
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install .
+latencyops --help
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install .
+latencyops --help
+```
+
+For development, use an editable install.
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install -e .
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install -e .
+```
+
+The runtime uses the Python standard library. Model-serving systems and credentials remain external.
+
+### From a wheel
+
+After building or downloading a release artifact:
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install dist\latencyops-0.1.0-py3-none-any.whl
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install dist/latencyops-0.1.0-py3-none-any.whl
+```
+
+### From PyPI
+
+After the package is published to PyPI:
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install latencyops
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install latencyops
+```
+
+The package is locally verified but has not yet been published to PyPI.
+
+## Plan without a model endpoint
+
+Create `request.json`:
+
+```json
+{
+  "prompt_tokens": 100,
+  "expected_output_tokens": 32,
+  "difficulty": 0.2,
+  "queue_pressure": 0.1,
+  "cache_pressure": 0.1,
+  "quality_floor": 0.9,
+  "risk_class": "low",
+  "deadline_ms": 1000
+}
+```
+
+Run the installed planner.
+
+PowerShell (Windows):
+
+```powershell
+latencyops plan --request request.json
+```
+
+Ubuntu (bash):
+
+```bash
+latencyops plan --request request.json
+```
+
+This returns an explainable plan without calling a model endpoint.
+
+## Verified real model coverage
+
+These are the real model paths explicitly verified in the current controlled test record. They are endpoint and latency checks, not a general model-quality ranking.
+
+| Provider | Exact model ID | Route | Verified coverage | Runtime signal status |
+|---|---|---|---|---|
+| OpenAI | `gpt-4o-mini` | OpenAI-compatible OpenAI API | Model discovery and streaming smoke test with TTFT, TPOT, and end-to-end measurement | OpenAI API-level signals only |
+| Qwen | `Qwen/Qwen3.6-35B-A3B-FP8` | LiteLLM → Qwen vLLM deployment | Authenticated model discovery, streaming smoke test, proactive request path, TTFT, TPOT, and end-to-end measurement | Direct vLLM queue/KV-cache metrics were not available through the public proxy; real tests use synthetic `SystemSignals` for planning |
+
+The recorded Qwen smoke result was a successful `Hello!` response with TTFT `743.01 ms`, TPOT `8.22 ms`, and end-to-end latency `751.23 ms`. These values are workload-, region-, time-, and deployment-dependent and are not performance guarantees.
+
+### Sample streaming comparison
+
+The following sanitized aggregate is provided as an example reference. It was generated on 2026-09-03 using a synthetic single-sentence streaming workload, three runs per model, and no retained prompt or response content.
+
+| Provider | Model | Runs | Success | TTFT p50 / p95 | TPOT p50 | End-to-end p50 / p95 |
+|---|---|---:|---:|---:|---:|---:|
+| OpenAI | `gpt-4o-mini` | 3 | 100% | 935.19 / 1634.29 ms | 7.58 ms | 939.58 / 1643.24 ms |
+| OpenAI | `gpt-4.1-mini` | 3 | 100% | 753.22 / 945.96 ms | 8.82 ms | 760.25 / 954.65 ms |
+| OpenAI | `gpt-5-mini` | 3 | 100% | 1859.08 / 1901.65 ms | 8.58 ms | 1867.66 / 1913.66 ms |
+| Qwen | `Qwen/Qwen3.6-35B-A3B-FP8` | 3 | 100% | 750.39 / 809.08 ms | 6.39 ms | 758.88 / 815.68 ms |
+
+These values are illustrative observations from one client, region, workload, and deployment window. They are not model-quality scores, service-level guarantees, or a substitute for running the workload against the reader's own account and endpoint.
+
+## Configured comparison candidates
+
+These IDs are configured as comparison or example candidates. A configured model ID is not evidence of a successful test; verify availability through the authenticated `/v1/models` response first.
+
+| Provider | Model ID | Status |
+|---|---|---|
+| OpenAI | `gpt-4.1-mini` | Configured comparison candidate |
+| OpenAI | `gpt-5-mini` | Configured comparison candidate |
+| Qwen | `Qwen/Qwen3.5-9B` | Example configuration candidate |
+| Qwen | `Qwen/Qwen3.6-35B-A3B` | Example configuration candidate |
+
+## How users can use it
+
+### Python application or evaluation job
+
+Embed the planner in an existing Python service when you want direct control over the request lifecycle. Give each analysis item or agent task its own deadline, quality floor, risk class, and workload profile.
+
+### Many analyses through one system
+
+Use one planner or gateway for document analysis, research pipelines, support classification, batch evaluation, or other workloads with different request requirements. LatencyOps can select different plans per item instead of forcing every item through one global model policy.
+
+### Multi-agent workflow
+
+Use LatencyOps as a shared execution control plane for research, extraction, review, synthesis, or other agents. Each agent can have a different quality and deadline contract while the application retains responsibility for agent coordination and memory.
+
+### Optional HTTP gateway
+
+Use the gateway when multiple applications or non-Python clients need one internal policy and telemetry endpoint. Place it behind an authenticated TLS-capable edge proxy before controlled deployment.
+
+## Synthetic gateway example
+
+The example starts a local gateway with deterministic providers and no credentials.
+
+PowerShell (Windows):
+
+```powershell
+$env:PYTHONPATH = "src"
+python examples\synthetic_gateway.py
+```
+
+Ubuntu (bash):
+
+```bash
+export PYTHONPATH=src
+python3 examples/synthetic_gateway.py
+```
+
+A completion can then be sent to `http://127.0.0.1:8080/v1/completions`, and measurements are available at `http://127.0.0.1:8080/metrics`.
+
+## Optional configured gateway
+
+Copy `latencyops.example.toml` to a private TOML file, set the environment variables named by `api_key_env`, and start the gateway.
+
+PowerShell (Windows):
+
+```powershell
+latencyops gateway --config latencyops.toml
+```
+
+Ubuntu (bash):
+
+```bash
+latencyops gateway --config latencyops.toml
+```
+
+Clients can call `POST /v1/completions`; inspect `GET /metrics` for content-free measurements.
+
+## Real endpoint tests
+
+The real-endpoint probe reads an external dotenv file, prints only the endpoint, credential variable name, model IDs, and measured results, and never prints the credential. Set `LATENCYOPS_ENV_FILE` and `LATENCYOPS_RESULTS_DIR` to paths outside the repository. If unset, the examples use `~/.latencyops/.env` and `~/.latencyops/results`. Use approved credentials and synthetic or approved evaluation data.
+
+PowerShell (Windows):
+
+```powershell
+$env:LATENCYOPS_ENV_FILE = "C:\path\to\private\latencyops\.env"
+$env:LATENCYOPS_RESULTS_DIR = "C:\path\to\private\latencyops\results"
+$env:PYTHONPATH = "src"
+python examples\real_endpoint_test.py
+python examples\real_endpoint_test.py --smoke --model "approved-model-id"
+python examples\compare_real_endpoints.py
+```
+
+Ubuntu (bash):
+
+```bash
+export LATENCYOPS_ENV_FILE="/path/to/private/latencyops/.env"
+export LATENCYOPS_RESULTS_DIR="/path/to/private/latencyops/results"
+export PYTHONPATH=src
+python3 examples/real_endpoint_test.py
+python3 examples/real_endpoint_test.py --smoke --model "approved-model-id"
+python3 examples/compare_real_endpoints.py
+```
+
+The comparison defaults to `gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5-mini`, and the configured Qwen model. Override the OpenAI set without changing source files.
+
+PowerShell (Windows):
+
+```powershell
+$env:LATENCYOPS_OPENAI_MODELS = "gpt-4o-mini,gpt-4.1-mini"
+$env:LATENCYOPS_KEY_NAME = "QWEN_API_KEY"
+```
+
+Ubuntu (bash):
+
+```bash
+export LATENCYOPS_OPENAI_MODELS="gpt-4o-mini,gpt-4.1-mini"
+export LATENCYOPS_KEY_NAME="QWEN_API_KEY"
+```
+
+The public workload runner tests eight OpenAI categories: factual, extraction, JSON, summarization, code, classification, reasoning, and safety.
+
+## Local verification
+
+PowerShell (Windows):
+
+```powershell
+python -m unittest discover -s tests -v
+python -m compileall -q src tests examples
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m unittest discover -s tests -v
+python3 -m compileall -q src tests examples
+```
+
+## Security and limitations
+
+- Do not place API keys, customer data, proprietary prompts, or production configuration in this repository.
+- Do not expose the reference gateway directly to an untrusted network.
+- Use an authenticated TLS-capable edge proxy for controlled deployment.
+- Queue and cache signals are real only when a configured metrics endpoint supplies them.
+- Provider-specific controls are only applied when the provider advertises support.
+- The package does not include model weights, GPU runtimes, hosted inference, durable queues, cancellation, circuit breakers, production backpressure, billing integration, or a complete quality evaluator.
+
+See [`SECURITY.md`](SECURITY.md) and [`DEPLOYMENT.md`](DEPLOYMENT.md) before deployment.

latencyops-0.1.0/README.md

@@ -0,0 +1,336 @@
+# LatencyOps
+
+LatencyOps is a provider-neutral Python toolkit for measuring AI inference latency and selecting explainable execution policies under latency and quality constraints.
+
+It is a lightweight policy and measurement layer that composes with OpenAI-compatible model endpoints and existing infrastructure. It does not contain model weights or replace OpenAI, LiteLLM, vLLM, SGLang, Hugging Face, or a production edge gateway.
+
+## Why use LatencyOps?
+
+Most inference applications send every request to one model with one fixed execution policy. LatencyOps makes the execution decision per request using:
+
+- Deadline and optional TTFT/TPOT targets
+- Quality floor and risk class
+- Prompt and expected-output token counts
+- Difficulty and context length
+- Queue and cache pressure when runtime metrics are available
+- Provider health and observed latency/quality
+
+The planner returns an explainable execution plan, provider capability enforcement makes unsupported controls visible, and the benchmark layer measures TTFT, TPOT, end-to-end latency, p50, and p95.
+
+A single planner can serve one application, a batch of any number of analyses, or many agents in a multi-agent workflow. The host application remains responsible for task decomposition, agent memory, tools, retries, and final aggregation. Detailed architecture notes are maintained separately for internal design review and are not part of the public README.
+
+## Where LatencyOps sits
+
+```text
+Your application / agent workflow
+              |
+              | Python library import or HTTP request
+              v
++-----------------------------------------------+
+| LatencyOps public policy + measurement layer  |
+| planner | capability enforcement | routing    |
+| TTFT | TPOT | end-to-end | content-free data |
++----------------------+------------------------+
+                       |
+                       v
+       OpenAI-compatible provider endpoint
+   OpenAI | LiteLLM | vLLM | SGLang | Hugging Face
+                       |
+                       v
+                 model response
+
+Optional runtime metrics from vLLM, SGLang, or
+TensorRT-LLM feed queue/cache/health signals back
+into the LatencyOps planner.
+```
+
+LatencyOps sits between the application or agent workflow and the model-serving endpoint. It can run in-process as a Python library or as an internal gateway; it is not the model server and it is not an agent framework.
+
+## Core capabilities
+
+- Request latency and quality contracts
+- Content-free request profiles and telemetry
+- Conservative model-tier and precision planning
+- Long-context, cache-pressure, speculation, and early-exit policy signals
+- Provider-neutral routing and adaptive provider selection
+- OpenAI-compatible completion and chat adapters
+- Streaming TTFT and TPOT measurement
+- Capability-aware requested-versus-enforced plans
+- Optional vLLM, SGLang, and TensorRT-LLM runtime signal normalization
+- Static-baseline comparisons and workload benchmarking
+- Optional quality outcome hooks and safe re-planning
+- Local reference gateway for Python and non-Python clients
+
+The current release is a dependency-light alpha/reference implementation suitable for development and controlled integration testing. It is not a hardened production serving platform.
+
+See also:
+
+- [`COMPARISON_WITH_EXISTING_TOOLS.md`](COMPARISON_WITH_EXISTING_TOOLS.md) for market comparison and community/commercial boundaries
+- [`SECURITY.md`](SECURITY.md) for safe operation and reporting
+- [`CONTRIBUTING.md`](CONTRIBUTING.md) for development and release verification
+- [`DEPLOYMENT.md`](DEPLOYMENT.md) for the reference gateway deployment model
+
+## Install and run
+
+LatencyOps requires Python 3.12 or newer.
+
+### From a GitHub checkout
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install .
+latencyops --help
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install .
+latencyops --help
+```
+
+For development, use an editable install.
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install -e .
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install -e .
+```
+
+The runtime uses the Python standard library. Model-serving systems and credentials remain external.
+
+### From a wheel
+
+After building or downloading a release artifact:
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install dist\latencyops-0.1.0-py3-none-any.whl
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install dist/latencyops-0.1.0-py3-none-any.whl
+```
+
+### From PyPI
+
+After the package is published to PyPI:
+
+PowerShell (Windows):
+
+```powershell
+python -m pip install latencyops
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m pip install latencyops
+```
+
+The package is locally verified but has not yet been published to PyPI.
+
+## Plan without a model endpoint
+
+Create `request.json`:
+
+```json
+{
+  "prompt_tokens": 100,
+  "expected_output_tokens": 32,
+  "difficulty": 0.2,
+  "queue_pressure": 0.1,
+  "cache_pressure": 0.1,
+  "quality_floor": 0.9,
+  "risk_class": "low",
+  "deadline_ms": 1000
+}
+```
+
+Run the installed planner.
+
+PowerShell (Windows):
+
+```powershell
+latencyops plan --request request.json
+```
+
+Ubuntu (bash):
+
+```bash
+latencyops plan --request request.json
+```
+
+This returns an explainable plan without calling a model endpoint.
+
+## Verified real model coverage
+
+These are the real model paths explicitly verified in the current controlled test record. They are endpoint and latency checks, not a general model-quality ranking.
+
+| Provider | Exact model ID | Route | Verified coverage | Runtime signal status |
+|---|---|---|---|---|
+| OpenAI | `gpt-4o-mini` | OpenAI-compatible OpenAI API | Model discovery and streaming smoke test with TTFT, TPOT, and end-to-end measurement | OpenAI API-level signals only |
+| Qwen | `Qwen/Qwen3.6-35B-A3B-FP8` | LiteLLM → Qwen vLLM deployment | Authenticated model discovery, streaming smoke test, proactive request path, TTFT, TPOT, and end-to-end measurement | Direct vLLM queue/KV-cache metrics were not available through the public proxy; real tests use synthetic `SystemSignals` for planning |
+
+The recorded Qwen smoke result was a successful `Hello!` response with TTFT `743.01 ms`, TPOT `8.22 ms`, and end-to-end latency `751.23 ms`. These values are workload-, region-, time-, and deployment-dependent and are not performance guarantees.
+
+### Sample streaming comparison
+
+The following sanitized aggregate is provided as an example reference. It was generated on 2026-09-03 using a synthetic single-sentence streaming workload, three runs per model, and no retained prompt or response content.
+
+| Provider | Model | Runs | Success | TTFT p50 / p95 | TPOT p50 | End-to-end p50 / p95 |
+|---|---|---:|---:|---:|---:|---:|
+| OpenAI | `gpt-4o-mini` | 3 | 100% | 935.19 / 1634.29 ms | 7.58 ms | 939.58 / 1643.24 ms |
+| OpenAI | `gpt-4.1-mini` | 3 | 100% | 753.22 / 945.96 ms | 8.82 ms | 760.25 / 954.65 ms |
+| OpenAI | `gpt-5-mini` | 3 | 100% | 1859.08 / 1901.65 ms | 8.58 ms | 1867.66 / 1913.66 ms |
+| Qwen | `Qwen/Qwen3.6-35B-A3B-FP8` | 3 | 100% | 750.39 / 809.08 ms | 6.39 ms | 758.88 / 815.68 ms |
+
+These values are illustrative observations from one client, region, workload, and deployment window. They are not model-quality scores, service-level guarantees, or a substitute for running the workload against the reader's own account and endpoint.
+
+## Configured comparison candidates
+
+These IDs are configured as comparison or example candidates. A configured model ID is not evidence of a successful test; verify availability through the authenticated `/v1/models` response first.
+
+| Provider | Model ID | Status |
+|---|---|---|
+| OpenAI | `gpt-4.1-mini` | Configured comparison candidate |
+| OpenAI | `gpt-5-mini` | Configured comparison candidate |
+| Qwen | `Qwen/Qwen3.5-9B` | Example configuration candidate |
+| Qwen | `Qwen/Qwen3.6-35B-A3B` | Example configuration candidate |
+
+## How users can use it
+
+### Python application or evaluation job
+
+Embed the planner in an existing Python service when you want direct control over the request lifecycle. Give each analysis item or agent task its own deadline, quality floor, risk class, and workload profile.
+
+### Many analyses through one system
+
+Use one planner or gateway for document analysis, research pipelines, support classification, batch evaluation, or other workloads with different request requirements. LatencyOps can select different plans per item instead of forcing every item through one global model policy.
+
+### Multi-agent workflow
+
+Use LatencyOps as a shared execution control plane for research, extraction, review, synthesis, or other agents. Each agent can have a different quality and deadline contract while the application retains responsibility for agent coordination and memory.
+
+### Optional HTTP gateway
+
+Use the gateway when multiple applications or non-Python clients need one internal policy and telemetry endpoint. Place it behind an authenticated TLS-capable edge proxy before controlled deployment.
+
+## Synthetic gateway example
+
+The example starts a local gateway with deterministic providers and no credentials.
+
+PowerShell (Windows):
+
+```powershell
+$env:PYTHONPATH = "src"
+python examples\synthetic_gateway.py
+```
+
+Ubuntu (bash):
+
+```bash
+export PYTHONPATH=src
+python3 examples/synthetic_gateway.py
+```
+
+A completion can then be sent to `http://127.0.0.1:8080/v1/completions`, and measurements are available at `http://127.0.0.1:8080/metrics`.
+
+## Optional configured gateway
+
+Copy `latencyops.example.toml` to a private TOML file, set the environment variables named by `api_key_env`, and start the gateway.
+
+PowerShell (Windows):
+
+```powershell
+latencyops gateway --config latencyops.toml
+```
+
+Ubuntu (bash):
+
+```bash
+latencyops gateway --config latencyops.toml
+```
+
+Clients can call `POST /v1/completions`; inspect `GET /metrics` for content-free measurements.
+
+## Real endpoint tests
+
+The real-endpoint probe reads an external dotenv file, prints only the endpoint, credential variable name, model IDs, and measured results, and never prints the credential. Set `LATENCYOPS_ENV_FILE` and `LATENCYOPS_RESULTS_DIR` to paths outside the repository. If unset, the examples use `~/.latencyops/.env` and `~/.latencyops/results`. Use approved credentials and synthetic or approved evaluation data.
+
+PowerShell (Windows):
+
+```powershell
+$env:LATENCYOPS_ENV_FILE = "C:\path\to\private\latencyops\.env"
+$env:LATENCYOPS_RESULTS_DIR = "C:\path\to\private\latencyops\results"
+$env:PYTHONPATH = "src"
+python examples\real_endpoint_test.py
+python examples\real_endpoint_test.py --smoke --model "approved-model-id"
+python examples\compare_real_endpoints.py
+```
+
+Ubuntu (bash):
+
+```bash
+export LATENCYOPS_ENV_FILE="/path/to/private/latencyops/.env"
+export LATENCYOPS_RESULTS_DIR="/path/to/private/latencyops/results"
+export PYTHONPATH=src
+python3 examples/real_endpoint_test.py
+python3 examples/real_endpoint_test.py --smoke --model "approved-model-id"
+python3 examples/compare_real_endpoints.py
+```
+
+The comparison defaults to `gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5-mini`, and the configured Qwen model. Override the OpenAI set without changing source files.
+
+PowerShell (Windows):
+
+```powershell
+$env:LATENCYOPS_OPENAI_MODELS = "gpt-4o-mini,gpt-4.1-mini"
+$env:LATENCYOPS_KEY_NAME = "QWEN_API_KEY"
+```
+
+Ubuntu (bash):
+
+```bash
+export LATENCYOPS_OPENAI_MODELS="gpt-4o-mini,gpt-4.1-mini"
+export LATENCYOPS_KEY_NAME="QWEN_API_KEY"
+```
+
+The public workload runner tests eight OpenAI categories: factual, extraction, JSON, summarization, code, classification, reasoning, and safety.
+
+## Local verification
+
+PowerShell (Windows):
+
+```powershell
+python -m unittest discover -s tests -v
+python -m compileall -q src tests examples
+```
+
+Ubuntu (bash):
+
+```bash
+python3 -m unittest discover -s tests -v
+python3 -m compileall -q src tests examples
+```
+
+## Security and limitations
+
+- Do not place API keys, customer data, proprietary prompts, or production configuration in this repository.
+- Do not expose the reference gateway directly to an untrusted network.
+- Use an authenticated TLS-capable edge proxy for controlled deployment.
+- Queue and cache signals are real only when a configured metrics endpoint supplies them.
+- Provider-specific controls are only applied when the provider advertises support.
+- The package does not include model weights, GPU runtimes, hosted inference, durable queues, cancellation, circuit breakers, production backpressure, billing integration, or a complete quality evaluator.
+
+See [`SECURITY.md`](SECURITY.md) and [`DEPLOYMENT.md`](DEPLOYMENT.md) before deployment.