apiwells 0.1.0__tar.gz → 0.2.0__tar.gz

apiwells-0.2.0/CHANGELOG.md

@@ -0,0 +1,28 @@
+# Changelog
+
+## Unreleased — v0.2 development
+
+The package and CLI remain at 0.1.0 (v0.2 development build); v0.2.0 has not
+been released.
+
+- Add Basic connectivity/authentication/models diagnostics and explicit-model
+  Deep capability diagnostics while retaining legacy `--chat` behavior.
+- Validate Streaming/SSE and report client TTFT and total latency.
+- Parse provider-reported Usage without estimating missing token counts.
+- Validate Tool Calling through the complete local-tool/two-request round trip.
+- Validate Structured Output against JSON Schema, distinguishing strict schema
+  support from JSON-object-only behavior.
+- Separate probe execution status from capability support.
+- Add a shared error taxonomy and centralized secret redaction.
+- Add versioned structured JSON reporting alongside compatibility reports.
+- Validate live endpoint compatibility with three preserved ED-031 records;
+  results remain point-in-time observations, including partial capabilities.
+- Validate local wheel/sdist packaging and Windows clean installation, including
+  Basic and critical Deep smoke checks and 43 clean-wheel integration tests.
+- Document Local-First/no-telemetry behavior, security boundaries, test coverage
+  and remaining release gates.
+
+## 0.1.0
+- Add a single-request models endpoint check and opt-in text chat smoke check.
+- Add HTTP/network failure categories, response shape checks and sanitized JSON.
+- Disable redirects, retries and implicit environment proxy use.

apiwells-0.2.0/PKG-INFO

@@ -0,0 +1,197 @@
+Metadata-Version: 2.4
+Name: apiwells
+Version: 0.2.0
+Summary: Endpoint Doctor for OpenAI-compatible model APIs
+License-Expression: MIT
+Keywords: api,diagnostics,llm,endpoint
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: jsonschema<5,>=4.26
+Dynamic: license-file
+
+# ApiWells Endpoint Doctor
+
+A Local-First CLI for diagnosing OpenAI-compatible model API endpoints from
+your machine. Endpoint Doctor checks connectivity and protocol capabilities;
+it does not score model quality. Python 3.10+ is required.
+
+This checkout is a **v0.2 development build**. The installed package and CLI
+version remain **0.1.0**; v0.2.0 has not been released. Runtime dependencies
+include `jsonschema` for Structured Output validation.
+
+## Install
+
+Install the current released package from PyPI:
+
+```sh
+python -m pip install apiwells
+apiwells --version
+```
+
+The current repository contains v0.2 development work but remains at version
+0.1.0 until RC version synchronization. v0.2.0 has not been released to PyPI.
+
+Install this local checkout with:
+
+```sh
+python -m pip install .
+```
+
+The checkout's version command currently prints `apiwells 0.1.0`.
+
+## Basic Doctor
+
+Set `APIWELLS_API_KEY` using your shell's secret-input mechanism. Do not put a
+real key in a command, screenshot, issue or repository. Use `--api-key-env NAME`
+to select another environment variable.
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --v2-json
+```
+
+Basic is the default mode: **URL → DNS → TLS → HTTP → Authentication → /models**.
+It sends no inference request. HTTP, authentication and model-list checks share
+one GET to `BASE/models`. An empty model list can pass the response-shape check;
+a model list alone does not prove inference works.
+
+Supply the exact API base: `/v1` is not added automatically. For a gateway with
+`/openai/v1`, include that prefix. Do not supply the full `/models` or
+`/chat/completions` URL.
+
+For a local unauthenticated development server:
+
+```sh
+apiwells doctor --base-url http://127.0.0.1:3000/v1 --anonymous
+```
+
+## Deep Doctor
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID --v2-json
+```
+
+Deep requires an explicit, nonempty model ID and follows:
+**Basic → Chat → Streaming/SSE → Usage → Tool Calling → Structured Output**.
+Usage is extracted from Chat and Streaming responses, not a separate request.
+Capability probes run only after Basic passes and the requested model is found.
+
+Deep makes multiple potentially billable inference requests:
+
+- Chat validates a nonempty assistant text response.
+- Streaming validates SSE framing, output and terminal `[DONE]`; it reports
+  TTFT and total latency. TTFT is measured from request start to the first valid
+  model-output event. HTTP headers, keepalive, role-only, empty and metadata-only
+  events do not count.
+- Usage contains provider-reported token counts only. Missing values remain
+  unavailable or null; ApiWells never estimates them.
+- Tool Calling must complete a full round trip: tool call, validated arguments,
+  local diagnostic tool execution, tool result, second request and a final
+  response using that result. Merely accepting `tools` is insufficient.
+- Structured Output must return JSON that passes the requested JSON Schema.
+  An explicit rejection of `json_schema` as unsupported can trigger one
+  additional, potentially billable `json_object` sub-test. JSON-object-only
+  success does not certify strict JSON Schema support.
+
+There are no hidden billable retries or fallback models. The Tool Calling
+second request and conditional Structured Output sub-test are diagnostic steps,
+with request counts included in their results. `--max-tokens` controls applicable
+generation budgets and is not a guaranteed total cost ceiling; diagnostic
+sub-tests may use bounded request parameters. Provider support varies; not every
+endpoint supports every capability.
+
+## Legacy chat compatibility
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --chat --model YOUR-MODEL-ID --json
+```
+
+Legacy `--chat` remains a single, opt-in, potentially billable non-streaming
+request to `BASE/chat/completions`, with `Reply OK.` and default `max_tokens=8`.
+A pass requires nonempty assistant text, not exactly `OK`. Use `--max-tokens 32`
+to change the budget; some models need more output tokens. It performs no
+automatic retry or fallback model selection. `--chat` and `--deep` are mutually
+exclusive.
+
+## Results and timing
+
+Probe execution status (`PASS`, `FAIL`, `PARTIAL`) is separate from capability
+support (`SUPPORTED`, `UNSUPPORTED`, `UNKNOWN`, `NOT_APPLICABLE`). A failed or
+partial probe with unknown support does not establish that a capability is
+unsupported. Core failures make the overall result FAIL; non-passing
+Streaming, Tool Calling or Structured Output probes make it PARTIAL when core
+checks pass.
+
+The exit-code contract remains:
+
+| Code | Meaning |
+|---|---|
+| `0` | Selected check / overall diagnostics passed |
+| `1` | Endpoint diagnostics did not pass, including an overall PARTIAL |
+| `2` | Local usage or configuration error |
+
+For Basic/Deep, `--v2-json` writes a versioned JSON report with
+`schema_version: "1"`, `apiwells_version`, `overall_status` and `probes`.
+Each probe includes status, support, summary, metrics, evidence and error code.
+The report schema version is independent of the package version.
+`--json` retains the compatibility report with `ok`; legacy `--chat --json`
+retains its single-check report. Use `--json`, not `--v2-json`, for legacy chat.
+Completed checks produce one JSON object on stdout; usage/configuration errors
+go to stderr and do not produce a JSON report.
+
+Error codes distinguish configuration, DNS/TLS/connectivity/timeout, HTTP/auth,
+JSON/schema, SSE/interruption and capability validation failures. They are
+diagnostic evidence, not definitive root-cause conclusions.
+
+`--timeout 15` is a socket-operation timeout, not an overall deadline. Client
+TTFT and total latency include transport effects and are not server-only
+inference timings. Legacy `elapsed_ms` is request/response elapsed time, not
+streaming TTFT.
+
+## Security and boundaries
+
+- Local-First execution; no telemetry or hidden upload of prompts, responses or
+  usage. Requested diagnostic traffic goes to the endpoint you select.
+- API keys and Authorization header values must not be printed. Centralized
+  redaction protects console/JSON reports and error paths; raw provider bodies
+  are not dumped into reports. Review reports before sharing.
+- HTTPS certificate verification stays enabled. Remote HTTP requires
+  `--allow-http`; literal loopback addresses and localhost are allowed for
+  development.
+- All redirects, including same-host redirects, are blocked so credentials are
+  not forwarded to redirect destinations.
+- Environment proxies are ignored unless `--use-env-proxy` is supplied.
+- Test only endpoints you are authorized to test. Private/local destinations are
+  intentionally allowed; this CLI is not an unrestricted server-side URL fetcher.
+
+See [Security](SECURITY.md) for handling and reporting guidance.
+
+## Compatibility and development
+
+[Compatibility](docs/COMPATIBILITY.md) indexes three point-in-time ED-031 live
+certification records. These establish observed endpoint behavior, not
+universal provider support or model quality.
+
+[The release-facing specification](docs/V0.2_SPEC.md) summarizes the frozen
+v0.2 scope. Benchmark, scoring/ranking, recommendations, cost optimization,
+Router, Gateway, long-term Monitoring, Web Dashboard, SaaS accounts, cloud
+telemetry and complete Responses API coverage are outside that scope.
+
+For test coverage and the observed QA harness dependencies, see
+[the test plan](docs/TEST_PLAN.md). With the test harness installed, run:
+
+```sh
+python -m pytest tests -q
+```
+
+See [the release checklist](docs/RELEASE_CHECKLIST.md) for completed evidence
+and pending release gates, and [the changelog](CHANGELOG.md) for changes.
+License: MIT.

apiwells-0.2.0/README.md

@@ -0,0 +1,179 @@
+# ApiWells Endpoint Doctor
+
+A Local-First CLI for diagnosing OpenAI-compatible model API endpoints from
+your machine. Endpoint Doctor checks connectivity and protocol capabilities;
+it does not score model quality. Python 3.10+ is required.
+
+This checkout is a **v0.2 development build**. The installed package and CLI
+version remain **0.1.0**; v0.2.0 has not been released. Runtime dependencies
+include `jsonschema` for Structured Output validation.
+
+## Install
+
+Install the current released package from PyPI:
+
+```sh
+python -m pip install apiwells
+apiwells --version
+```
+
+The current repository contains v0.2 development work but remains at version
+0.1.0 until RC version synchronization. v0.2.0 has not been released to PyPI.
+
+Install this local checkout with:
+
+```sh
+python -m pip install .
+```
+
+The checkout's version command currently prints `apiwells 0.1.0`.
+
+## Basic Doctor
+
+Set `APIWELLS_API_KEY` using your shell's secret-input mechanism. Do not put a
+real key in a command, screenshot, issue or repository. Use `--api-key-env NAME`
+to select another environment variable.
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --v2-json
+```
+
+Basic is the default mode: **URL → DNS → TLS → HTTP → Authentication → /models**.
+It sends no inference request. HTTP, authentication and model-list checks share
+one GET to `BASE/models`. An empty model list can pass the response-shape check;
+a model list alone does not prove inference works.
+
+Supply the exact API base: `/v1` is not added automatically. For a gateway with
+`/openai/v1`, include that prefix. Do not supply the full `/models` or
+`/chat/completions` URL.
+
+For a local unauthenticated development server:
+
+```sh
+apiwells doctor --base-url http://127.0.0.1:3000/v1 --anonymous
+```
+
+## Deep Doctor
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID --v2-json
+```
+
+Deep requires an explicit, nonempty model ID and follows:
+**Basic → Chat → Streaming/SSE → Usage → Tool Calling → Structured Output**.
+Usage is extracted from Chat and Streaming responses, not a separate request.
+Capability probes run only after Basic passes and the requested model is found.
+
+Deep makes multiple potentially billable inference requests:
+
+- Chat validates a nonempty assistant text response.
+- Streaming validates SSE framing, output and terminal `[DONE]`; it reports
+  TTFT and total latency. TTFT is measured from request start to the first valid
+  model-output event. HTTP headers, keepalive, role-only, empty and metadata-only
+  events do not count.
+- Usage contains provider-reported token counts only. Missing values remain
+  unavailable or null; ApiWells never estimates them.
+- Tool Calling must complete a full round trip: tool call, validated arguments,
+  local diagnostic tool execution, tool result, second request and a final
+  response using that result. Merely accepting `tools` is insufficient.
+- Structured Output must return JSON that passes the requested JSON Schema.
+  An explicit rejection of `json_schema` as unsupported can trigger one
+  additional, potentially billable `json_object` sub-test. JSON-object-only
+  success does not certify strict JSON Schema support.
+
+There are no hidden billable retries or fallback models. The Tool Calling
+second request and conditional Structured Output sub-test are diagnostic steps,
+with request counts included in their results. `--max-tokens` controls applicable
+generation budgets and is not a guaranteed total cost ceiling; diagnostic
+sub-tests may use bounded request parameters. Provider support varies; not every
+endpoint supports every capability.
+
+## Legacy chat compatibility
+
+```sh
+apiwells doctor --base-url https://YOUR-API-HOST/v1 --chat --model YOUR-MODEL-ID --json
+```
+
+Legacy `--chat` remains a single, opt-in, potentially billable non-streaming
+request to `BASE/chat/completions`, with `Reply OK.` and default `max_tokens=8`.
+A pass requires nonempty assistant text, not exactly `OK`. Use `--max-tokens 32`
+to change the budget; some models need more output tokens. It performs no
+automatic retry or fallback model selection. `--chat` and `--deep` are mutually
+exclusive.
+
+## Results and timing
+
+Probe execution status (`PASS`, `FAIL`, `PARTIAL`) is separate from capability
+support (`SUPPORTED`, `UNSUPPORTED`, `UNKNOWN`, `NOT_APPLICABLE`). A failed or
+partial probe with unknown support does not establish that a capability is
+unsupported. Core failures make the overall result FAIL; non-passing
+Streaming, Tool Calling or Structured Output probes make it PARTIAL when core
+checks pass.
+
+The exit-code contract remains:
+
+| Code | Meaning |
+|---|---|
+| `0` | Selected check / overall diagnostics passed |
+| `1` | Endpoint diagnostics did not pass, including an overall PARTIAL |
+| `2` | Local usage or configuration error |
+
+For Basic/Deep, `--v2-json` writes a versioned JSON report with
+`schema_version: "1"`, `apiwells_version`, `overall_status` and `probes`.
+Each probe includes status, support, summary, metrics, evidence and error code.
+The report schema version is independent of the package version.
+`--json` retains the compatibility report with `ok`; legacy `--chat --json`
+retains its single-check report. Use `--json`, not `--v2-json`, for legacy chat.
+Completed checks produce one JSON object on stdout; usage/configuration errors
+go to stderr and do not produce a JSON report.
+
+Error codes distinguish configuration, DNS/TLS/connectivity/timeout, HTTP/auth,
+JSON/schema, SSE/interruption and capability validation failures. They are
+diagnostic evidence, not definitive root-cause conclusions.
+
+`--timeout 15` is a socket-operation timeout, not an overall deadline. Client
+TTFT and total latency include transport effects and are not server-only
+inference timings. Legacy `elapsed_ms` is request/response elapsed time, not
+streaming TTFT.
+
+## Security and boundaries
+
+- Local-First execution; no telemetry or hidden upload of prompts, responses or
+  usage. Requested diagnostic traffic goes to the endpoint you select.
+- API keys and Authorization header values must not be printed. Centralized
+  redaction protects console/JSON reports and error paths; raw provider bodies
+  are not dumped into reports. Review reports before sharing.
+- HTTPS certificate verification stays enabled. Remote HTTP requires
+  `--allow-http`; literal loopback addresses and localhost are allowed for
+  development.
+- All redirects, including same-host redirects, are blocked so credentials are
+  not forwarded to redirect destinations.
+- Environment proxies are ignored unless `--use-env-proxy` is supplied.
+- Test only endpoints you are authorized to test. Private/local destinations are
+  intentionally allowed; this CLI is not an unrestricted server-side URL fetcher.
+
+See [Security](SECURITY.md) for handling and reporting guidance.
+
+## Compatibility and development
+
+[Compatibility](docs/COMPATIBILITY.md) indexes three point-in-time ED-031 live
+certification records. These establish observed endpoint behavior, not
+universal provider support or model quality.
+
+[The release-facing specification](docs/V0.2_SPEC.md) summarizes the frozen
+v0.2 scope. Benchmark, scoring/ranking, recommendations, cost optimization,
+Router, Gateway, long-term Monitoring, Web Dashboard, SaaS accounts, cloud
+telemetry and complete Responses API coverage are outside that scope.
+
+For test coverage and the observed QA harness dependencies, see
+[the test plan](docs/TEST_PLAN.md). With the test harness installed, run:
+
+```sh
+python -m pytest tests -q
+```
+
+See [the release checklist](docs/RELEASE_CHECKLIST.md) for completed evidence
+and pending release gates, and [the changelog](CHANGELOG.md) for changes.
+License: MIT.

apiwells-0.2.0/docs/COMPATIBILITY.md

@@ -0,0 +1,34 @@
+# v0.2 Compatibility
+
+ED-031 real endpoint/provider live validation is **CLOSED / PASS**. This page is
+a release-facing index; the historical
+[compatibility matrix](compatibility/compatibility-matrix.md) and individual
+records remain the authoritative detailed evidence. An overall PARTIAL record
+does not mean that the ED-031 validation work package failed.
+
+| Record | Provider | Model | Test date | ApiWells version | Overall |
+|---|---|---|---|---|---|
+| [MODEL-0001](compatibility/model0001-deepseek-flash.md) | DeepSeek Official | `deepseek-flash` | 2026-09-24 | 0.1.0 (v0.2 development build) | PARTIAL |
+| [MODEL-0002](compatibility/model0002-alibaba-qwen37-plus.md) | Alibaba Cloud Model Studio | `qwen3.7-plus` | 2026-09-27 | 0.1.0 (v0.2 development build) | PASS |
+| [MODEL-0003](compatibility/model0003-openrouter-ling30-flash-fin.md) | OpenRouter | `inclusionai/ling-3.0-flash-fin:free` | 2026-09-27 | 0.1.0 (v0.2 development build) | PARTIAL |
+
+## Interpretation
+
+- Results are point-in-time observations for the recorded endpoint, model,
+  configuration and date, not permanent provider guarantees.
+- Probe execution status and capability support are separate concepts.
+  `FAIL / UNKNOWN` does not establish permanent lack of support;
+  `PARTIAL / UNKNOWN` does not mean unsupported.
+- Tool Calling certification requires the full round trip, and Structured
+  Output certification requires successful JSON Schema validation. Request
+  acceptance alone is insufficient.
+- Some historical records use `AVAILABLE` for Usage. This is a
+  compatibility-record-local label meaning provider-reported usage data was
+  returned and parsed. It does not add a new ProbeResult support enum; those
+  remain `SUPPORTED`, `UNSUPPORTED`, `UNKNOWN` and `NOT_APPLICABLE`.
+- Compatibility evidence is not a model-quality Benchmark, ranking or
+  recommendation. Timings are observations, not a performance guarantee.
+
+Detailed metrics and qualifications belong to the linked ED-031 records and
+are not duplicated here. ED-032 Windows clean-install smoke evidence validates
+installation and execution; its metrics do not replace ED-031 measurements.

apiwells-0.2.0/docs/RELEASE_CHECKLIST.md

@@ -0,0 +1,79 @@
+# v0.2 Release Checklist
+
+Repository release-facing scope reference: [V0.2_SPEC.md](V0.2_SPEC.md), derived
+from the project-level frozen engineering baseline maintained outside the
+repository release artifacts.
+
+A checked evidence item applies only to its recorded development build/environment,
+not automatically to a future RC or final artifact. Unverified work stays unchecked.
+
+Current baseline: branch `feature/v0.2-diagnostics`, starting commit
+`f63de41`, package/CLI **0.1.0 (v0.2 development build)**.
+v0.2.0 has not been released.
+
+## Completed development-build evidence
+
+- [x] ED-031 real endpoint/provider live validation: **CLOSED / PASS**.
+  The [compatibility index](COMPATIBILITY.md) links the authoritative matrix
+  and MODEL-0001, MODEL-0002 and MODEL-0003 records.
+- [x] ED-032 Windows clean-install test: **CLOSED / PASS**.
+- [x] Local wheel + sdist build: **PASS**.
+- [x] Windows Basic real endpoint smoke: **PASS**.
+- [x] Windows clean-wheel integration suite: **43 passed**.
+- [x] Windows critical Deep smoke: **PASS**.
+- [x] No secret leak observed in the recorded validation.
+
+The ED-032 outcomes above are carried forward from the supplied ED-033
+acceptance baseline; they are not newly executed ED-033 checks. They do not
+replace or overwrite ED-031 compatibility measurements. Historical
+[0.1.0 validation](releases/0.1.0/VALIDATION.md) remains separate evidence, not
+v0.2 RC acceptance.
+
+## Frozen Scope gates (section 46)
+
+| Gate | Required evidence | Current release disposition |
+|---|---|---|
+| A — Baseline | v0.1 regression tests, editable install, CLI smoke, clean baseline | Historical gate evidence must be included in final release review; no new PASS claimed here |
+| B — Core Architecture | Unified ProbeResult; runner independent of console; reporters do not send HTTP | Implementation described in the spec; final gate acceptance remains to be recorded |
+| C — Functional | Success, unsupported and failure paths for Models, Chat, Streaming, Tools and Structured Output | Test plan covers these paths; complete release validation evidence remains pending |
+| D — Security | Secret Leak Test passes; any complete credential leak blocks release | No leak observed in accepted validation; retain the dedicated release test evidence before final acceptance |
+| E — Real Providers | At least 3 independent providers/endpoints covering 3–5 Chinese models | PASS: closed ED-031 records |
+| F — Package | Build wheel + sdist and clean install | PASS for the development build via ED-032; RC/final artifact acceptance remains pending |
+| G — Release Documentation | README, CHANGELOG, SECURITY and the four release documents | PASS — ED-033 documentation reviewed and accepted |
+
+## Documentation acceptance (ED-033)
+
+- [x] Reconcile the release-facing specification/checklist with the supplied
+  frozen specification and current CLI compatibility behavior.
+- [x] Reviewer acceptance of README, changelog, security guidance, specification,
+  test plan, compatibility index and this checklist (Gate G): PASS.
+
+TESTENV-01: the observed Windows harness used pytest **9.1.1**,
+pytest-httpserver **1.1.5** and Werkzeug **3.1.8**. These are test dependencies,
+not ApiWells runtime dependencies. See [TEST_PLAN.md](TEST_PLAN.md).
+No new dev-dependency packaging mechanism is selected in ED-033.
+
+## Remaining formal release gates
+
+- [ ] Complete remaining required release validation under the frozen scope:
+  unit, mock integration, Secret Leak Test and any outstanding platform/Python
+  coverage; retain evidence tied to the release candidate.
+- [ ] Complete macOS and Linux installation, `--version`, `--help`, Basic smoke
+  and mock-suite validation required by section 38.
+- [ ] Verify the minimal CI checks: unit tests, mock integration, wheel/sdist build
+  and package verification; live provider tests stay separately controlled.
+- [ ] Synchronize production version declarations to **0.2.0** in both
+  `pyproject.toml` and `src/apiwells/__init__.py` before formal RC/final artifacts.
+- [ ] Build and identify the formal **RC artifact**; validate metadata and
+  preserve artifact hashes and provenance.
+- [ ] Complete **TestPyPI** publication/installation validation.
+- [ ] Complete **RC clean-install acceptance** against the actual RC artifact.
+- [ ] Review all required release evidence and approve the final release.
+- [ ] Publish the final **PyPI 0.2.0 release**.
+- [ ] Complete **post-release PyPI clean-install smoke**, including the frozen
+  onboarding target of a first Basic check within about five minutes.
+
+VERSION-01: both version declarations remain 0.1.0 in ED-033. Whether to refactor
+to a single version source is undecided. Closed ED-031/ED-032 work is not
+reopened by these future artifact-specific gates. This checklist does not
+authorize publishing.

apiwells-0.2.0/docs/TEST_PLAN.md

@@ -0,0 +1,132 @@
+# v0.2 Test Plan
+
+This plan covers the frozen Endpoint Doctor v0.2 scope summarized in
+[V0.2_SPEC.md](V0.2_SPEC.md). It describes required validation, not a claim that
+every release gate has passed. See [the release checklist](RELEASE_CHECKLIST.md)
+for evidence status. ED-031 and ED-032 remain closed; ED-033 does not rerun or
+reopen them.
+
+## 1. Unit testing
+
+Keep unit tests offline. Use the tests under `tests/` to cover:
+
+| Area | Required checks |
+|---|---|
+| URL/configuration | Exact base path, schemes, host/port, credentials/query/fragment rejection, loopback versus remote HTTP, key characters, model requirement, token/timeout bounds |
+| ProbeResult | Required fields, allowed status/support enums and error codes, ordered probe collection |
+| Status/support | Execution outcome separate from support; core failure versus capability PARTIAL aggregation; unknown support is not unsupported |
+| Redaction | Exact runtime secrets, bearer/header/query patterns, nested values, summaries, metrics and evidence |
+| Error classification | DNS, TLS, connection, timeout, redirect, authentication, permissions, quota/rate, upstream, parsing and capability errors |
+| JSON parsing | Valid and malformed JSON, invalid response shapes, non-JSON HTTP 200 responses, bounded response reads |
+| SSE parsing | LF/CRLF, multiline data, comments, UTF-8, event boundaries, malformed JSON, broken/pending events, missing DONE and interrupted streams |
+| TTFT/latency | Measure from request start to the first valid model-output event; HTTP headers, keepalive, role-only, empty and metadata-only events do not count; total latency remains distinct |
+| Tool Calling | Correct tool name/ID, arguments JSON and schema, diagnostic challenge, local execution, tool-result linkage, second request and final response using the runtime tool result |
+| Structured Output | Strict JSON Schema success, malformed JSON, wrong types/extra properties, explicit unsupported detection, json_object-only result distinct from schema support |
+| Usage | Provider-reported nonnegative integer fields, partial/missing/invalid fields, streaming merge, no estimated counts or derived missing totals |
+| Reporters | Console, versioned JSON and legacy compatibility output; schema/package versions, fields/order, aggregation, secret safety, stdout/stderr and exit codes |
+
+## 2. Mock integration testing
+
+Use `tests/integration/` with the local HTTP mock server. Validate Basic and
+Deep end to end, CLI flag combinations, legacy `--chat` (including the default
+`max_tokens=8`), console/JSON agreement and the exit-code contract.
+
+Exercise successful and failed authentication, missing requested model,
+301/302 redirects, 400/401/403/404/429/500/502/503 responses, invalid JSON/schema, broken and interrupted
+SSE, invalid tool arguments, full tool round trips and Structured Output
+outcomes. Verify that Deep stops before billable probes when Basic does not
+pass. Check redirect blocking and request counts: no hidden billable retry;
+only the documented Tool Calling second request and conditional
+`json_object` sub-test may add their respective requests.
+
+With a prepared development/test environment:
+
+```sh
+python -m pytest tests -q
+python -m pytest tests/integration -q
+```
+
+The first command includes the integration suite. The second is the focused
+integration command, including for clean-wheel acceptance when package imports
+are verified to resolve to the installed wheel.
+
+## 3. Secret Leak Test
+
+Use synthetic canary credentials, never live keys in fixtures. Inject them into
+provider echoes, errors, headers, nested evidence, malformed responses and
+unexpected exceptions. Check captured stdout, stderr, logs, tracebacks, console reports, both
+JSON paths and serialized results for credential leakage or raw traceback/body
+exposure. Exercise full Basic, Deep and legacy paths, including timeout and
+interrupted-stream failures. Search captured outputs and generated test artifacts for complete canary secrets;
+any leak blocks release. Preserve only sanitized evidence.
+
+## 4. Real Endpoint Certification
+
+The frozen minimum is three independent providers/endpoints covering three to
+five Chinese models. Record provider, exact endpoint/model, date, package/build version, environment,
+configuration, per-capability status/support, safe evidence and client timings.
+Validate Models, Chat, SSE/TTFT, provider-reported Usage, Tool Calling's full
+round trip and strict JSON Schema output. Preserve FAIL/PARTIAL/UNKNOWN
+observations without turning them into permanent provider claims.
+
+ED-031 is **CLOSED / PASS**. The three certified records and matrix linked from
+[COMPATIBILITY.md](COMPATIBILITY.md) are authoritative historical evidence.
+Do not overwrite them or replace their measurements with clean-install smoke
+timings. Provider capability outcomes need not all be PASS for a valid record.
+
+## 5. Packaging / clean-install testing
+
+Build local wheel and sdist, check distribution metadata, install the wheel
+into a clean environment, and verify imports resolve to that environment's
+site-packages rather than the checkout. Check package/CLI version agreement,
+entry points, dependency consistency, Basic real endpoint smoke, the mock
+integration suite and critical Deep smoke. Preserve artifact identity and
+sanitized command/results evidence.
+
+ED-032 Windows clean-install testing is **CLOSED / PASS**. The supplied ED-032
+acceptance evidence records local wheel + sdist build PASS, Windows Basic real
+endpoint smoke PASS, clean-wheel integration **43 passed**, critical Deep smoke
+PASS and no secret leak observed. These are prior observations, not new ED-033
+test runs.
+
+### TESTENV-01: observed Windows QA harness
+
+| Test dependency | Observed version |
+|---|---|
+| pytest | 9.1.1 |
+| pytest-httpserver | 1.1.5 |
+| Werkzeug | 3.1.8 |
+
+These are **test dependencies, not ApiWells runtime dependencies**. They describe
+the observed Windows clean-wheel integration environment, not a new supported
+dependency matrix. ED-033 neither adds them to runtime dependencies nor chooses
+a new dev-dependency packaging or management mechanism.
+
+## 6. Cross-platform testing
+
+Validate the release candidate on Windows, macOS and Linux, including shell
+entry points, environment-key handling, TLS/proxy behavior, JSON encoding and
+exit codes. macOS and Linux each require installation, `--version`, `--help`,
+Basic smoke and the mock suite. Each platform run records OS, Python version
+and the tested artifact. Python must satisfy the declared `>=3.10` requirement.
+The Frozen Scope requires the specified Windows/macOS/Linux validation, not an
+exhaustive OS × Python-version matrix.
+
+Windows development-build acceptance is supported by ED-032. Remaining required
+platform validation and RC acceptance stay open until corresponding evidence
+exists. Historical 0.1.0 validation does not certify a v0.2 RC.
+
+## 7. Release evidence
+
+Keep live provider tests separately controlled and out of ordinary CI by default;
+provide credentials through environment variables or managed CI secrets.
+Keep unit/mock/secret-test summaries, sanitized compatibility records,
+build/metadata checks, artifact hashes and clean-install/platform results tied
+to the tested commit and artifact. Record failures and limitations as well as
+passes. Before formal RC/final artifacts, synchronize both version declarations
+to 0.2.0; they remain 0.1.0 in ED-033 (VERSION-01). A single version source is
+not decided here.
+
+TestPyPI, RC artifact/clean-install acceptance, remaining required validation,
+final PyPI release and post-release PyPI clean-install smoke are future gates,
+tracked in [RELEASE_CHECKLIST.md](RELEASE_CHECKLIST.md).