technical-answer-validator 0.1.1__tar.gz → 0.1.2__tar.gz

technical_answer_validator-0.1.2/PKG-INFO

@@ -0,0 +1,113 @@
+Metadata-Version: 2.5
+Name: technical-answer-validator
+Version: 0.1.2
+Summary: MCP and REST tools for deterministic rubric-based technical answer review
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: mcp==2.2.0
+Description-Content-Type: text/markdown
+
+# Technical Answer Validator
+
+<!-- mcp-name: io.github.Christofer566/technical-answer-validator -->
+
+A tiny, deterministic answer review tool for AI agents, available over **MCP stdio** and as a REST API. Each caller supplies the concepts, accepted synonyms, numeric requirements, and answer text for a single evaluation. It does not include a question bank or answer corpus.
+
+This is an **assistive practice tool**, not an official certification exam grader. Keyword matching can miss semantically correct paraphrases and can accept misleading surface matches. Users should review the supplied rubric and every result.
+
+## Run locally
+
+Python 3.10+; the REST server uses the standard library. The MCP adapter uses the official Python SDK 2.2.0.
+
+```powershell
+$env:TAV_API_KEY = "replace-with-a-long-random-secret-at-least-24-characters"
+python -m tav_api
+```
+
+The service listens on `127.0.0.1:8080` by default. To change it, set `TAV_HOST` and `TAV_PORT`. Local single-key mode refuses to start without a `TAV_API_KEY` of at least 24 characters. For deployment, create a distinct key per caller with `python scripts/create_api_key.py CLIENT_ID` and configure `TAV_API_KEYS` as a JSON object mapping each client ID to the generated SHA-256 digest. Store the one-time raw key with that client; do not store or commit it in this repository. When `TAV_API_KEYS` is set, it takes precedence over `TAV_API_KEY`.
+
+## MCP for AI agents
+
+Install the pinned MCP SDK 2.2.0 in a virtual environment from the committed lockfile:
+
+```powershell
+uv sync --locked
+.\.venv\Scripts\Activate.ps1
+```
+
+The stdio MCP server exposes one tool: `evaluate_answer(rubric, answer)`. Configure an MCP host with the absolute path to the environment's Python and `mcp_server.py`. Example Claude Desktop configuration (replace paths):
+
+```json
+{
+  "mcpServers": {
+    "technical-answer-validator": {
+      "command": "C:\\path\\to\\technical-answer-validator\\.venv\\Scripts\\python.exe",
+      "args": ["C:\\path\\to\\technical-answer-validator\\mcp_server.py"]
+    }
+  }
+}
+```
+
+After publishing to PyPI, run with `uvx --from technical-answer-validator tav-mcp`. For local development use the `.venv` Python plus `mcp_server.py`. For Codex CLI or another MCP host, use its stdio server configuration with that command and script path. Restart the host, then ask it to list tools and call `evaluate_answer`. The stdio transport is local to the user's agent host and needs no internet endpoint or API key.
+
+The Codex TOML template is `codex-mcp-config.example.toml`; the Claude Desktop JSON template is `claude-mcp-config.example.json`. Replace both placeholder paths with absolute paths. Merge the block into the host configuration; do not overwrite other MCP servers or global settings.
+
+### Local MCP usage counts
+
+The MCP server keeps local daily aggregate counts for tool calls, successful evaluations, invalid requests, and internal errors. It never stores rubric or answer text and sends nothing over the network. Counts are retained for 90 days in `~/.technical-answer-validator/usage.sqlite3` (Windows: the user's home directory). Run `tav-mcp-stats` to print a JSON report. Set `TAV_ANALYTICS=off` in the MCP server environment to disable counting; set `TAV_ANALYTICS_DB` to choose another local database path. These counts remain on the user's computer unless the user voluntarily shares the report.
+
+To **voluntarily share anonymous usage counts** with the maintainers, enable remote telemetry in the MCP server environment only after reviewing this notice:
+
+```text
+TAV_TELEMETRY=on
+TAV_TELEMETRY_URL=https://technical-answer-validator-telemetry.chl1591204.workers.dev/v1/event
+```
+
+Remote telemetry is off by default. When enabled, each tool call sends only a random persistent installation ID, package version, and outcome (`success`, `invalid_request`, or `error`) to the configured Cloudflare Worker. The ID is random and is not derived from a user, device, or account identifier; it lets us estimate distinct installations, not distinct people. Answers, rubrics, tool outputs, IP addresses, host names, and request contents are not included in the event payload. Cloudflare necessarily receives network connection metadata such as the sender IP to deliver the request; our Worker does not log or store it. Analytics Engine retains event data for three months. Disable by setting `TAV_TELEMETRY=off` and optionally remove `~/.technical-answer-validator/telemetry-id` to reset the random ID. Telemetry failures never interrupt evaluations.
+
+The collector is deployed at the URL above and accepts events only at `/v1/event`. To read aggregate usage, create a Cloudflare API token restricted to **Account Analytics Read** for the account that owns the Worker, then run `python scripts/query_telemetry.py` from this repository. The script prompts for the token without echoing or saving it. The report groups call totals by UTC day, package version, and outcome, and estimates distinct installations; it does not identify people. Do not put the token in source control, an MCP configuration, or a command line.
+
+PyPI 0.1.1 does not include this opt-in telemetry code. It will be available to package users in 0.1.2; collection starts only after that version is installed and a user explicitly enables both environment variables above. The local-only `tav-mcp-stats` command continues to work independently.
+
+## Request
+
+`POST /v1/evaluate`
+
+```json
+{
+  "rubric": {
+    "required_concepts": ["isolation", "lockout tag"],
+    "accepted_synonyms": {"isolation": ["energy isolation"]},
+    "numeric_requirements": [],
+    "required_count": 2
+  },
+  "answer": "Apply energy isolation and attach a lockout tag."
+}
+```
+
+`accepted_synonyms` keys must exactly match a concept. `numeric_requirements` is an optional array such as `[{"value":"10","unit":"kN","tolerance":"0"}]`. Numbers in an answer are only checked when explicit requirements are provided. Concept score is matched concepts / required_count (defaults to the number of concepts), capped at 1.0; numeric failures apply a 50% score penalty. The verdict thresholds are `correct >= 0.8`, `partial >= 0.4`, otherwise `wrong`.
+
+## Response and errors
+
+Successful requests return `api_version`, `status`, `score`, `verdict`, matched/missing concepts, numeric check details, and `review_required: true`.
+
+Errors use JSON `{ "error": { "code": "...", "message": "..." } }`. Statuses include 400 (invalid request), 401 (missing/invalid key), 404, 405, 413 (body over 64 KiB), and 429 (over 60 requests/minute per client ID). The rate limit is 60 requests/minute per authenticated client ID, in memory, and resets when the process restarts. Daily request/success/client-error/rate-limited counters are persisted in SQLite without answer text and expire after 90 days. `GET /v1/usage` returns only the caller's current UTC-day counts. Retain and back up the usage volume as desired; it contains client IDs and aggregates only.
+
+## Privacy and deployment limits
+
+The stdio MCP option runs locally and sends no usage information to a central service unless the user explicitly enables remote telemetry as described above. Local aggregate call counts can be disabled separately. The REST API has separate server-side aggregates per API client.
+
+The server does not log request bodies or answers. It stores daily counts keyed by client ID and request timestamps in process memory for REST rate limiting. The **stdio MCP option runs locally inside the agent host** and sends no requests to this HTTP server. The REST API is containerized and keeps usage counters in a persistent volume. Before public service, terminate TLS at a reverse proxy, set proxy-level rate/concurrency limits, deploy from a secret manager, monitor the host, and publish a data-retention/contact policy. The app-level per-client rate limit resets on restart and is not a substitute for edge controls.
+
+## Verify
+
+```powershell
+python -m unittest discover -s tests -v
+```
+
+The OpenAPI contract is in `openapi.yaml`; the draft official MCP Registry descriptor is `server.json`, with publication steps in `PUBLISHING.md`. Run locally with Docker Compose after copying `.env.example` to `.env` and adding a private key; Compose publishes the service only on loopback, so configure an HTTPS reverse proxy separately. `compose.yaml` persists aggregate usage in a named volume and applies a read-only root filesystem, dropped Linux capabilities, and resource limits.
+
+These tests check API and grading behavior; they do not establish professional exam accuracy.
+
+

technical_answer_validator-0.1.2/PUBLISHING.md

@@ -0,0 +1,53 @@
+# Public release runbook
+
+The package contains two delivery paths: local MCP stdio for an agent that can spawn a local process, and a REST API container for hosted agents. GitHub and PyPI releases are public, and the container image is published. The REST service itself still needs a host deployment and a public HTTPS endpoint.
+
+The stdio server records local daily aggregate call counts. Version 0.1.2 adds optional remote MCP usage telemetry, disabled by default and requiring explicit user opt-in; the collector runs on Cloudflare Workers and the provider reads aggregate usage through the Analytics Engine SQL API. The REST host remains deferred.
+
+## Local MCP package
+
+1. MIT was selected and is included in `LICENSE` and the Python package metadata. Confirm that all included source/assets are eligible for MIT before public release.
+2. The public GitHub repository exists at `https://github.com/Christofer566/technical-answer-validator`; source is on `main`. Keep `.venv`, `.uv-cache`, `.env`, usage databases, and generated state out of commits.
+3. Build and inspect the wheel/sdist: `uv build`; test the wheel in a clean virtual environment; `uv run python -m unittest discover -s tests -v`.
+4. PyPI 0.1.1 is already published. The pending GitHub Actions Trusted Publisher is configured for project `technical-answer-validator`, owner `Christofer566`, repository `technical-answer-validator`, workflow `.github/workflows/publish-pypi.yml`, environment `pypi`. For 0.1.2, merge the reviewed telemetry change, build and inspect the wheel/sdist, then publish through the configured trusted-publisher workflow using tag `v0.1.2`.
+5. Confirm the built PyPI README contains the exact `mcp-name` marker and that package metadata, repository URL, license, and author are correct.
+6. Validate `server.json` against the current MCP Registry schema and verify the GitHub/PyPI namespace, package name, and version. Then publish with the official MCP Registry CLI using the owner's GitHub identity.
+7. Test installation from the published package in a clean environment with `uvx --from technical-answer-validator tav-mcp`; connect it in target agent hosts and call `evaluate_answer`.
+
+## Hosted REST API
+
+### Deployment decision
+
+Do not create the Render service yet. The stdio MCP package is already public and runs on the user's own computer. The REST API source and GHCR image are also public for self-hosting. There is no confirmed demand for an always-on API operated by us, so the estimated recurring Render cost and service operation are not justified yet.
+
+Review managed hosting 30 days after first external MCP announcement. Check installs/downloads, successful connections, support requests, and whether users specifically need an API URL operated by us. If there is no concrete hosted-API request, review once more at day 90. If there is still no evidence or willingness to pay for this convenience, defer hosting and review quarterly only when usage signals change.
+
+Consider Render only when at least one real user concretely requests an operator-managed HTTPS endpoint because local MCP/self-hosting is unsuitable, and they confirm willingness to pay or participate in a bounded trial. Before creation, set a monthly spend cap, name an operator, define key issuance/revocation and data retention, and verify actual billing. Do not buy a custom domain until demand is validated.
+
+### When the trigger is met
+
+`render.yaml` is a prepared independent Render configuration (Singapore, Starter, HTTPS, `/healthz`, and a 1 GB persistent disk at `/data`). Expected starting cost is about $7.25/month before taxes/overages; verify current pricing in Render before creating the service. This deployment is independent of Windows and the existing VPS.
+
+1. Set `TAV_API_KEYS` in the Render secret environment to `{client_id: sha256_hex}` entries generated by `python scripts/create_api_key.py CLIENT_ID`. Deliver each raw key privately to its intended agent operator; only the digest belongs in deployment configuration.
+2. Published image: `ghcr.io/christofer566/technical-answer-validator-api:0.1.1` (also `:latest`), built by the successful GitHub Actions workflow for tag `v0.1.1`. Deploy the image to the selected Render service, retaining the `/data` disk. Add platform-level request and rate controls; app-level rate buckets reset on restart.
+3. Monitor `/healthz`, per-customer `/v1/usage`, host availability, disk space for SQLite aggregates, and spend. Define key rotation/revocation by removing the corresponding digest and restarting/redeploying.
+4. Verify external HTTPS from a separate network with unauthenticated rejection, valid client isolation, 64 KiB limit, rate limits, no answer logging, and accurate OpenAPI docs before announcing a URL.
+
+## Current deployment progress and remaining prerequisites
+
+- Public source repository: `https://github.com/Christofer566/technical-answer-validator`; `server.json` points to it.
+- PyPI 0.1.1: `https://pypi.org/project/technical-answer-validator/0.1.1/`.
+- GitHub Release v0.1.1: `https://github.com/Christofer566/technical-answer-validator/releases/tag/v0.1.1`.
+- GHCR image publication and CI succeeded for v0.1.1.
+- Render hosting plan and `render.yaml` are prepared, but service creation is intentionally deferred pending demand. Review 30 days after external MCP announcement, then at day 90 if the first review finds no concrete hosted-API need.
+- This API deployment is intentionally independent of the VPS and Windows runtime.
+- MIT is selected. Public source ownership and rights to included files should remain confirmed by the owner.
+- No public API URL has been created; image publication is not a running service.
+- The tool's keyword/numeric decisions are not validated against independent subject-matter gold data. Position it as assistive agent feedback, not a trustworthy final grader.
+## Provider usage measurement (0.1.2 candidate)
+
+The opt-in MCP telemetry collector is deployed at `https://technical-answer-validator-telemetry.chl1591204.workers.dev`. Cloudflare Analytics Engine dataset `tav_mcp_usage` is bound as `TAV_USAGE`; the Worker accepts `/v1/event` and does not log request bodies. Remote telemetry is disabled by default. MCP users must install the 0.1.2 release and explicitly set `TAV_TELEMETRY=on` plus `TAV_TELEMETRY_URL` to opt in. The event contains only a random installation UUID, package version, and outcome. No answer, rubric, tool output, host name, or account identifier is sent.
+
+The provider query utility is `python scripts/query_telemetry.py`. It asks for a Cloudflare Account Analytics Read API token using hidden input and does not persist it. The query estimates unique installations, not unique people, and reports UTC daily totals by package version and outcome. Analytics Engine retains data for three months. Do not publish the provider token or put it in MCP configuration.
+
+

technical_answer_validator-0.1.2/README.md

@@ -0,0 +1,103 @@
+# Technical Answer Validator
+
+<!-- mcp-name: io.github.Christofer566/technical-answer-validator -->
+
+A tiny, deterministic answer review tool for AI agents, available over **MCP stdio** and as a REST API. Each caller supplies the concepts, accepted synonyms, numeric requirements, and answer text for a single evaluation. It does not include a question bank or answer corpus.
+
+This is an **assistive practice tool**, not an official certification exam grader. Keyword matching can miss semantically correct paraphrases and can accept misleading surface matches. Users should review the supplied rubric and every result.
+
+## Run locally
+
+Python 3.10+; the REST server uses the standard library. The MCP adapter uses the official Python SDK 2.2.0.
+
+```powershell
+$env:TAV_API_KEY = "replace-with-a-long-random-secret-at-least-24-characters"
+python -m tav_api
+```
+
+The service listens on `127.0.0.1:8080` by default. To change it, set `TAV_HOST` and `TAV_PORT`. Local single-key mode refuses to start without a `TAV_API_KEY` of at least 24 characters. For deployment, create a distinct key per caller with `python scripts/create_api_key.py CLIENT_ID` and configure `TAV_API_KEYS` as a JSON object mapping each client ID to the generated SHA-256 digest. Store the one-time raw key with that client; do not store or commit it in this repository. When `TAV_API_KEYS` is set, it takes precedence over `TAV_API_KEY`.
+
+## MCP for AI agents
+
+Install the pinned MCP SDK 2.2.0 in a virtual environment from the committed lockfile:
+
+```powershell
+uv sync --locked
+.\.venv\Scripts\Activate.ps1
+```
+
+The stdio MCP server exposes one tool: `evaluate_answer(rubric, answer)`. Configure an MCP host with the absolute path to the environment's Python and `mcp_server.py`. Example Claude Desktop configuration (replace paths):
+
+```json
+{
+  "mcpServers": {
+    "technical-answer-validator": {
+      "command": "C:\\path\\to\\technical-answer-validator\\.venv\\Scripts\\python.exe",
+      "args": ["C:\\path\\to\\technical-answer-validator\\mcp_server.py"]
+    }
+  }
+}
+```
+
+After publishing to PyPI, run with `uvx --from technical-answer-validator tav-mcp`. For local development use the `.venv` Python plus `mcp_server.py`. For Codex CLI or another MCP host, use its stdio server configuration with that command and script path. Restart the host, then ask it to list tools and call `evaluate_answer`. The stdio transport is local to the user's agent host and needs no internet endpoint or API key.
+
+The Codex TOML template is `codex-mcp-config.example.toml`; the Claude Desktop JSON template is `claude-mcp-config.example.json`. Replace both placeholder paths with absolute paths. Merge the block into the host configuration; do not overwrite other MCP servers or global settings.
+
+### Local MCP usage counts
+
+The MCP server keeps local daily aggregate counts for tool calls, successful evaluations, invalid requests, and internal errors. It never stores rubric or answer text and sends nothing over the network. Counts are retained for 90 days in `~/.technical-answer-validator/usage.sqlite3` (Windows: the user's home directory). Run `tav-mcp-stats` to print a JSON report. Set `TAV_ANALYTICS=off` in the MCP server environment to disable counting; set `TAV_ANALYTICS_DB` to choose another local database path. These counts remain on the user's computer unless the user voluntarily shares the report.
+
+To **voluntarily share anonymous usage counts** with the maintainers, enable remote telemetry in the MCP server environment only after reviewing this notice:
+
+```text
+TAV_TELEMETRY=on
+TAV_TELEMETRY_URL=https://technical-answer-validator-telemetry.chl1591204.workers.dev/v1/event
+```
+
+Remote telemetry is off by default. When enabled, each tool call sends only a random persistent installation ID, package version, and outcome (`success`, `invalid_request`, or `error`) to the configured Cloudflare Worker. The ID is random and is not derived from a user, device, or account identifier; it lets us estimate distinct installations, not distinct people. Answers, rubrics, tool outputs, IP addresses, host names, and request contents are not included in the event payload. Cloudflare necessarily receives network connection metadata such as the sender IP to deliver the request; our Worker does not log or store it. Analytics Engine retains event data for three months. Disable by setting `TAV_TELEMETRY=off` and optionally remove `~/.technical-answer-validator/telemetry-id` to reset the random ID. Telemetry failures never interrupt evaluations.
+
+The collector is deployed at the URL above and accepts events only at `/v1/event`. To read aggregate usage, create a Cloudflare API token restricted to **Account Analytics Read** for the account that owns the Worker, then run `python scripts/query_telemetry.py` from this repository. The script prompts for the token without echoing or saving it. The report groups call totals by UTC day, package version, and outcome, and estimates distinct installations; it does not identify people. Do not put the token in source control, an MCP configuration, or a command line.
+
+PyPI 0.1.1 does not include this opt-in telemetry code. It will be available to package users in 0.1.2; collection starts only after that version is installed and a user explicitly enables both environment variables above. The local-only `tav-mcp-stats` command continues to work independently.
+
+## Request
+
+`POST /v1/evaluate`
+
+```json
+{
+  "rubric": {
+    "required_concepts": ["isolation", "lockout tag"],
+    "accepted_synonyms": {"isolation": ["energy isolation"]},
+    "numeric_requirements": [],
+    "required_count": 2
+  },
+  "answer": "Apply energy isolation and attach a lockout tag."
+}
+```
+
+`accepted_synonyms` keys must exactly match a concept. `numeric_requirements` is an optional array such as `[{"value":"10","unit":"kN","tolerance":"0"}]`. Numbers in an answer are only checked when explicit requirements are provided. Concept score is matched concepts / required_count (defaults to the number of concepts), capped at 1.0; numeric failures apply a 50% score penalty. The verdict thresholds are `correct >= 0.8`, `partial >= 0.4`, otherwise `wrong`.
+
+## Response and errors
+
+Successful requests return `api_version`, `status`, `score`, `verdict`, matched/missing concepts, numeric check details, and `review_required: true`.
+
+Errors use JSON `{ "error": { "code": "...", "message": "..." } }`. Statuses include 400 (invalid request), 401 (missing/invalid key), 404, 405, 413 (body over 64 KiB), and 429 (over 60 requests/minute per client ID). The rate limit is 60 requests/minute per authenticated client ID, in memory, and resets when the process restarts. Daily request/success/client-error/rate-limited counters are persisted in SQLite without answer text and expire after 90 days. `GET /v1/usage` returns only the caller's current UTC-day counts. Retain and back up the usage volume as desired; it contains client IDs and aggregates only.
+
+## Privacy and deployment limits
+
+The stdio MCP option runs locally and sends no usage information to a central service unless the user explicitly enables remote telemetry as described above. Local aggregate call counts can be disabled separately. The REST API has separate server-side aggregates per API client.
+
+The server does not log request bodies or answers. It stores daily counts keyed by client ID and request timestamps in process memory for REST rate limiting. The **stdio MCP option runs locally inside the agent host** and sends no requests to this HTTP server. The REST API is containerized and keeps usage counters in a persistent volume. Before public service, terminate TLS at a reverse proxy, set proxy-level rate/concurrency limits, deploy from a secret manager, monitor the host, and publish a data-retention/contact policy. The app-level per-client rate limit resets on restart and is not a substitute for edge controls.
+
+## Verify
+
+```powershell
+python -m unittest discover -s tests -v
+```
+
+The OpenAPI contract is in `openapi.yaml`; the draft official MCP Registry descriptor is `server.json`, with publication steps in `PUBLISHING.md`. Run locally with Docker Compose after copying `.env.example` to `.env` and adding a private key; Compose publishes the service only on loopback, so configure an HTTPS reverse proxy separately. `compose.yaml` persists aggregate usage in a named volume and applies a read-only root filesystem, dropped Linux capabilities, and resource limits.
+
+These tests check API and grading behavior; they do not establish professional exam accuracy.
+
+

{technical_answer_validator-0.1.1 → technical_answer_validator-0.1.2}/mcp_server.py

@@ -1,46 +1,53 @@
-"""MCP stdio server exposing the answer evaluator as an agent tool."""
-
-from __future__ import annotations
-
-import json
-import logging
-from typing import Any
-
-from mcp.server import MCPServer
-
+"""MCP stdio server exposing the answer evaluator as an agent tool."""
+
+from __future__ import annotations
+
+import json
+import logging
+from typing import Any
+
+from mcp.server import MCPServer
+
 from tav_core import RequestError, evaluate
-
-logging.basicConfig(level=logging.WARNING, format="%(levelname)s %(message)s")
-mcp = MCPServer(
-    "Technical Answer Validator",
-    instructions=(
-        "Evaluate a user's technical answer against a rubric supplied for this call. "
-        "Always pass the caller's rubric and answer. Explain that results are keyword-based "
-        "assistive feedback, never official exam grades; preserve review_required in your response."
-    ),
-)
-
-
-@mcp.tool()
-def evaluate_answer(rubric: dict[str, Any], answer: str) -> dict[str, Any]:
-    """Check an answer against caller-provided concepts, synonyms, and numeric requirements.
-
-    Rubric fields: required_concepts (1-50 strings), optional accepted_synonyms keyed by
-    concept, optional numeric_requirements [{value, unit?, tolerance?}], and optional
-    required_count. No question bank is bundled. Review the returned result; it is not
-    an official exam grade.
-    """
+from tav_analytics import record_call
+
+logging.basicConfig(level=logging.WARNING, format="%(levelname)s %(message)s")
+mcp = MCPServer(
+    "Technical Answer Validator",
+    instructions=(
+        "Evaluate a user's technical answer against a rubric supplied for this call. "
+        "Always pass the caller's rubric and answer. Explain that results are keyword-based "
+        "assistive feedback, never official exam grades; preserve review_required in your response."
+    ),
+)
+
+
+@mcp.tool()
+def evaluate_answer(rubric: dict[str, Any], answer: str) -> dict[str, Any]:
+    """Check an answer against caller-provided concepts, synonyms, and numeric requirements.
+
+    Rubric fields: required_concepts (1-50 strings), optional accepted_synonyms keyed by
+    concept, optional numeric_requirements [{value, unit?, tolerance?}], and optional
+    required_count. No question bank is bundled. Review the returned result; it is not
+    an official exam grade.
+    """
     try:
-        return evaluate({"rubric": rubric, "answer": answer})
+        result = evaluate({"rubric": rubric, "answer": answer})
+        record_call("success")
+        return result
     except RequestError as exc:
-        # Return a structured error result so the agent can repair its inputs.
+        record_call("invalid_request")
+        # Return a structured error result so the agent can repair its inputs.
         return {"status": "invalid_request", "error": str(exc), "review_required": True}
-
-
-def main() -> None:
-    # FastMCP stdio transport uses stdout for protocol messages; keep diagnostics on stderr.
-    mcp.run(transport="stdio")
-
-
-if __name__ == "__main__":
-    main()
+    except Exception:
+        record_call("error")
+        raise
+
+
+def main() -> None:
+    # FastMCP stdio transport uses stdout for protocol messages; keep diagnostics on stderr.
+    mcp.run(transport="stdio")
+
+
+if __name__ == "__main__":
+    main()

{technical_answer_validator-0.1.1 → technical_answer_validator-0.1.2}/pyproject.toml

@@ -1,42 +1,40 @@
-[build-system]
-requires = ["hatchling>=1.27,<2"]
-build-backend = "hatchling.build"
-
-
-[project]
-name = "technical-answer-validator"
-version = "0.1.1"
-description = "MCP and REST tools for deterministic rubric-based technical answer review"
-readme = "README.md"
-requires-python = ">=3.10"
-license = "MIT"
-license-files = ["LICENSE"]
-dependencies = ["mcp==2.2.0"]
-
-
-[project.scripts]
-tav-mcp = "mcp_server:main"
+[build-system]
+requires = ["hatchling>=1.27,<2"]
+build-backend = "hatchling.build"
+
+[project]
+name = "technical-answer-validator"
+version = "0.1.2"
+description = "MCP and REST tools for deterministic rubric-based technical answer review"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+dependencies = ["mcp==2.2.0"]
+
+[project.scripts]
+tav-mcp = "mcp_server:main"
 tav-api = "tav_api:main"
-
-
+tav-mcp-stats = "tav_analytics:main"
+
 [tool.hatch.build.targets.wheel]
-only-include = ["mcp_server.py", "tav_core.py", "tav_api.py"]
-
-
-[tool.hatch.build.targets.wheel.force-include]
-"mcp_server.py" = "mcp_server.py"
-"tav_core.py" = "tav_core.py"
+only-include = ["mcp_server.py", "tav_core.py", "tav_api.py", "tav_analytics.py"]
+
+[tool.hatch.build.targets.wheel.force-include]
+"mcp_server.py" = "mcp_server.py"
+"tav_core.py" = "tav_core.py"
 "tav_api.py" = "tav_api.py"
-
-
-[tool.hatch.build.targets.sdist]
-only-include = [
-  ".dockerignore", ".env.example", ".gitignore", "Dockerfile", "NOTICE.md",
-  "PUBLISHING.md", "README.md", "claude-mcp-config.example.json",
-  "codex-mcp-config.example.toml", "compose.yaml", "mcp_server.py",
-  "openapi.yaml", "pyproject.toml", "requirements.txt", "server.json",
-  "tav_api.py", "tav_core.py", "uv.lock", ".github", "scripts", "tests"
+"tav_analytics.py" = "tav_analytics.py"
+
+[tool.hatch.build.targets.sdist]
+only-include = [
+  ".dockerignore", ".env.example", ".gitignore", "Dockerfile", "NOTICE.md",
+  "PUBLISHING.md", "README.md", "claude-mcp-config.example.json",
+  "codex-mcp-config.example.toml", "compose.yaml", "mcp_server.py",
+  "openapi.yaml", "pyproject.toml", "requirements.txt", "server.json",
+  "tav_api.py", "tav_core.py", "tav_analytics.py", "uv.lock", ".github", "scripts", "tests"
 ]
-
-
-[tool.pytest.ini_options]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+

technical_answer_validator-0.1.2/scripts/query_telemetry.py

@@ -0,0 +1,46 @@
+"""Query Cloudflare Analytics Engine without persisting the API token."""
+
+from __future__ import annotations
+
+import getpass
+import json
+import os
+import sys
+from urllib.request import Request, urlopen
+
+
+QUERY = """
+SELECT
+  toStartOfDay(timestamp) AS day,
+  blob1 AS version,
+  blob2 AS outcome,
+  SUM(double1 * _sample_interval) AS calls,
+  COUNT(DISTINCT index1) AS active_installations
+FROM tav_mcp_usage
+WHERE timestamp > NOW() - INTERVAL '90' DAY
+GROUP BY day, version, outcome
+ORDER BY day DESC, version, outcome
+FORMAT JSON
+""".strip()
+
+
+def main() -> int:
+    account_id = os.environ.get("CLOUDFLARE_ACCOUNT_ID", "3723bfb3f71e2120e3ed1391fa2ea566")
+    token = getpass.getpass("Cloudflare Account Analytics Read token (input hidden): ").strip()
+    if not token:
+        print("No token provided.", file=sys.stderr)
+        return 2
+    request = Request(
+        f"https://api.cloudflare.com/client/v4/accounts/{account_id}/analytics_engine/sql",
+        data=QUERY.encode("utf-8"),
+        headers={"Authorization": f"Bearer {token}", "Content-Type": "text/plain"},
+        method="POST",
+    )
+    with urlopen(request, timeout=20) as response:
+        result = json.loads(response.read().decode("utf-8"))
+    print(json.dumps(result, ensure_ascii=False, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

{technical_answer_validator-0.1.1 → technical_answer_validator-0.1.2}/server.json

@@ -1,21 +1,21 @@
-{
-  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
+{
+  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
   "name": "io.github.Christofer566/technical-answer-validator",
-  "title": "Technical Answer Validator",
-  "description": "Rubric-based technical answer feedback for AI agents. Human review required; not an official grade.",
-  "version": "0.1.1",
-  "repository": {
-    "url": "https://github.com/Christofer566/technical-answer-validator",
-    "source": "github"
-  },
-  "packages": [
-    {
-      "registryType": "pypi",
-      "registryBaseUrl": "https://pypi.org",
-      "identifier": "technical-answer-validator",
-      "version": "0.1.1",
-      "transport": { "type": "stdio" }
-    }
-  ]
-}
-
+  "title": "Technical Answer Validator",
+  "description": "Deterministic rubric-based technical answer feedback for AI agents; review required, not an official grade.",
+  "version": "0.1.2",
+  "repository": {
+    "url": "https://github.com/Christofer566/technical-answer-validator",
+    "source": "github"
+  },
+  "packages": [
+    {
+      "registryType": "pypi",
+      "registryBaseUrl": "https://pypi.org",
+      "identifier": "technical-answer-validator",
+      "version": "0.1.2",
+      "transport": { "type": "stdio" }
+    }
+  ]
+}
+

technical_answer_validator-0.1.2/tav_analytics.py

@@ -0,0 +1,138 @@
+"""Local-only aggregate usage counts for the MCP stdio server.
+
+No request text, rubric, answer, host identity, or network address is stored.
+"""
+
+from __future__ import annotations
+
+from contextlib import closing
+from datetime import datetime, timezone
+import json
+import os
+from pathlib import Path
+import sqlite3
+import threading
+import urllib.error
+import urllib.request
+import uuid
+
+PACKAGE_VERSION = "0.1.2"
+
+
+def _db_path() -> Path:
+    configured = os.environ.get("TAV_ANALYTICS_DB")
+    if configured:
+        return Path(configured)
+    return Path.home() / ".technical-answer-validator" / "usage.sqlite3"
+
+
+def _install_id_path() -> Path:
+    return Path.home() / ".technical-answer-validator" / "telemetry-id"
+
+
+def _install_id() -> str:
+    path = _install_id_path()
+    path.parent.mkdir(parents=True, exist_ok=True)
+    try:
+        value = path.read_text(encoding="ascii").strip()
+        uuid.UUID(value)
+        return value
+    except (OSError, ValueError):
+        value = str(uuid.uuid4())
+        path.write_text(value, encoding="ascii")
+        return value
+
+
+def _send_event(outcome: str) -> None:
+    """Best-effort, explicit opt-in remote event; never include tool inputs or outputs."""
+    if os.environ.get("TAV_TELEMETRY", "off").lower() not in {"on", "true", "1", "yes"}:
+        return
+    endpoint = os.environ.get("TAV_TELEMETRY_URL", "").strip()
+    if not endpoint.startswith("https://"):
+        return
+
+    def send() -> None:
+        try:
+            payload = json.dumps({
+                "installation_id": _install_id(),
+                "version": PACKAGE_VERSION,
+                "outcome": outcome,
+            }).encode("utf-8")
+            request = urllib.request.Request(
+                endpoint, data=payload,
+                headers={"Content-Type": "application/json", "User-Agent": f"technical-answer-validator/{PACKAGE_VERSION}"},
+                method="POST",
+            )
+            with urllib.request.urlopen(request, timeout=2) as response:
+                response.read(128)
+        except (OSError, urllib.error.URLError, ValueError):
+            return
+
+    threading.Thread(target=send, name="tav-telemetry", daemon=True).start()
+
+
+def record_call(outcome: str) -> None:
+    """Increment the local daily total; analytics can be disabled with TAV_ANALYTICS=off."""
+    if outcome not in {"success", "invalid_request", "error"}:
+        outcome = "error"
+    _send_event(outcome)
+    if os.environ.get("TAV_ANALYTICS", "on").lower() in {"off", "false", "0", "no"}:
+        return
+    try:
+        path = _db_path()
+        path.parent.mkdir(parents=True, exist_ok=True)
+        day = datetime.now(timezone.utc).date().isoformat()
+        with closing(sqlite3.connect(path, timeout=5)) as connection:
+            with connection:
+                connection.execute("""CREATE TABLE IF NOT EXISTS daily_mcp_usage (
+                    day TEXT PRIMARY KEY,
+                    calls INTEGER NOT NULL DEFAULT 0,
+                    successful INTEGER NOT NULL DEFAULT 0,
+                    invalid_requests INTEGER NOT NULL DEFAULT 0,
+                    errors INTEGER NOT NULL DEFAULT 0
+                )""")
+                connection.execute("DELETE FROM daily_mcp_usage WHERE day < date('now', '-89 day')")
+                connection.execute("INSERT OR IGNORE INTO daily_mcp_usage(day) VALUES(?)", (day,))
+                column = {"success": "successful", "invalid_request": "invalid_requests", "error": "errors"}[outcome]
+                connection.execute(
+                    f"UPDATE daily_mcp_usage SET calls=calls+1, {column}={column}+1 WHERE day=?", (day,)
+                )
+    except (OSError, sqlite3.Error):
+        # Local analytics must never make an otherwise valid MCP request fail.
+        return
+
+
+def report() -> dict[str, object]:
+    path = _db_path()
+    if not path.exists():
+        rows = []
+    else:
+        with closing(sqlite3.connect(path, timeout=5)) as connection:
+            rows = connection.execute(
+                "SELECT day,calls,successful,invalid_requests,errors FROM daily_mcp_usage ORDER BY day"
+            ).fetchall()
+    return {
+        "source": "local_mcp_stdio",
+        "privacy": "local aggregate only; no rubric, answer, host ID, or network address",
+        "retention_days": 90,
+        "days": [
+            {"date_utc": day, "calls": calls, "successful": successful,
+             "invalid_requests": invalid_requests, "errors": errors}
+            for day, calls, successful, invalid_requests, errors in rows
+        ],
+        "totals": {
+            "calls": sum(row[1] for row in rows),
+            "successful": sum(row[2] for row in rows),
+            "invalid_requests": sum(row[3] for row in rows),
+            "errors": sum(row[4] for row in rows),
+        },
+    }
+
+
+def main() -> None:
+    print(json.dumps(report(), ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+    main()
+

{technical_answer_validator-0.1.1 → technical_answer_validator-0.1.2}/uv.lock

@@ -863,7 +863,7 @@ wheels = [
 
 [[package]]
 name = "technical-answer-validator"
-version = "0.1.1"
+version = "0.1.2"
 source = { editable = "." }
 dependencies = [
     { name = "mcp" },

technical_answer_validator-0.1.1/PKG-INFO

@@ -1,35 +0,0 @@
-Metadata-Version: 2.5
-Name: technical-answer-validator
-Version: 0.1.1
-Summary: MCP and REST tools for deterministic rubric-based technical answer review
-License-Expression: MIT
-License-File: LICENSE
-Requires-Python: >=3.10
-Requires-Dist: mcp==2.2.0
-Description-Content-Type: text/markdown
-
-# Technical Answer Validator
-
-
-<!-- mcp-name: io.github.Christofer566/technical-answer-validator -->
-
-
-A tiny, deterministic answer review tool for AI agents, available over **MCP stdio** and as a REST API. Each caller supplies the concepts, accepted synonyms, numeric requirements, and answer text for a single evaluation. It does not include a question bank or answer corpus.
-
-
-This is an **assistive practice tool**, not an official certification exam grader. Keyword matching can miss semantically correct paraphrases and can accept misleading surface matches. Users should review the supplied rubric and every result.
-
-
-## Run locally
-
-
-Python 3.10+; the REST server uses the standard library. The MCP adapter uses the official Python SDK 2.2.0.
-
-
-```powershell
-$env:TAV_API_KEY = "replace-with-a-long-random-secret-at-least-24-characters"
-python -m tav_api
-```
-
-
-The service listens on `127.0.0.1:8080` by default. To change it, set `TAV_HOST` and `TAV_PORT`. Local single-key mode refuses to start without a `TAV_API_KEY` of at least 24 characters. For deployment, create a distinct key per caller with `python scripts/create_api_key.py CLIENT_ID` and configure `TAV_API_KEYS` as a JSON object mapping each client ID to the generated SHA-256 digest. Store the one-time raw key with that client; do not store or commit it in this repository. When `TAV_API_KEYS` is set, it takes precedence over `TAV_API_KEY`.

technical_answer_validator-0.1.1/PUBLISHING.md

@@ -1,31 +0,0 @@
-# Public release runbook
-
-The package contains two delivery paths: local MCP stdio for an agent that can spawn a local process, and a REST API container for hosted agents. No public artifact has been uploaded and no service is deployed by this preparation.
-
-## Local MCP package
-
-1. MIT was selected and is included in `LICENSE` and the Python package metadata. Confirm that all included source/assets are eligible for MIT before public release.
-2. The public GitHub repository exists at `https://github.com/Christofer566/technical-answer-validator`; source is on `main`. Keep `.venv`, `.uv-cache`, `.env`, usage databases, and generated state out of commits.
-3. Build and inspect the wheel/sdist: `uv build`; test the wheel in a clean virtual environment; `uv run python -m unittest discover -s tests -v`.
-4. PyPI JSON lookup returned HTTP 404 on 2026-10-02. In the PyPI account, register a pending GitHub Actions Trusted Publisher for project `technical-answer-validator`, owner `Christofer566`, repository `technical-answer-validator`, workflow `.github/workflows/publish-pypi.yml`, environment `pypi`. The GitHub `pypi` environment already exists. Publish version `0.1.0` by pushing a `v0.1.0` tag only after ownership/IP approval.
-5. Confirm the built PyPI README contains the exact `mcp-name` marker and that package metadata, repository URL, license, and author are correct.
-6. Validate `server.json` against the current MCP Registry schema and verify the GitHub/PyPI namespace, package name, and version. Then publish with the official MCP Registry CLI using the owner's GitHub identity.
-7. Test installation from the published package in a clean environment with `uvx --from technical-answer-validator tav-mcp`; connect it in target agent hosts and call `evaluate_answer`.
-
-## Hosted REST API
-
-1. Choose the VPS/host, public domain, TLS reverse proxy, budget ceiling, operator contact, and retention policy. These account/domain choices are not present in the workspace.
-2. Set `TAV_API_KEYS` in the host's secret manager to `{client_id: sha256_hex}` entries generated by `python scripts/create_api_key.py CLIENT_ID`. Deliver each raw key privately to its intended agent operator; only the digest belongs in deployment configuration.
-3. Push an approved `v0.1.0` tag to build and publish `ghcr.io/<owner>/<repo>-api` via `.github/workflows/publish-container.yml`. Pull that image on the host or build directly from the reviewed repository; deploy with `compose.yaml` and retain the `/data` volume. Keep the app bound to loopback behind the HTTPS proxy. Add proxy-level connection/time/body limits and rate limiting; app-level rate buckets reset on restart.
-4. Monitor `/healthz`, per-customer `/v1/usage`, host availability, disk space for SQLite aggregates, and spend. Define key rotation/revocation by removing the corresponding digest and restarting/redeploying.
-5. Verify external HTTPS from a separate network with unauthenticated rejection, valid client isolation, 64 KiB limit, rate limits, no answer logging, and accurate OpenAPI docs before announcing a URL.
-
-## Blocks that require owner-controlled decisions/actions
-
-- Public source repository exists and `server.json` points to it.
-- PyPI name was not registered at the 2026-10-02 lookup (HTTP 404). PyPI account authentication is required to register its pending publisher; no package has been uploaded.
-- MIT is selected. The owner must still confirm project ownership and third-party rights for included files before publication.
-- No production URL, host account, budget, or service operator contact has been supplied; therefore the container remains local and has not been deployed.
-- The tool's keyword/numeric decisions are not validated against independent subject-matter gold data. Position it as assistive agent feedback, not a trustworthy final grader.
-
-

technical_answer_validator-0.1.1/README.md

@@ -1,25 +0,0 @@
-# Technical Answer Validator
-
-
-<!-- mcp-name: io.github.Christofer566/technical-answer-validator -->
-
-
-A tiny, deterministic answer review tool for AI agents, available over **MCP stdio** and as a REST API. Each caller supplies the concepts, accepted synonyms, numeric requirements, and answer text for a single evaluation. It does not include a question bank or answer corpus.
-
-
-This is an **assistive practice tool**, not an official certification exam grader. Keyword matching can miss semantically correct paraphrases and can accept misleading surface matches. Users should review the supplied rubric and every result.
-
-
-## Run locally
-
-
-Python 3.10+; the REST server uses the standard library. The MCP adapter uses the official Python SDK 2.2.0.
-
-
-```powershell
-$env:TAV_API_KEY = "replace-with-a-long-random-secret-at-least-24-characters"
-python -m tav_api
-```
-
-
-The service listens on `127.0.0.1:8080` by default. To change it, set `TAV_HOST` and `TAV_PORT`. Local single-key mode refuses to start without a `TAV_API_KEY` of at least 24 characters. For deployment, create a distinct key per caller with `python scripts/create_api_key.py CLIENT_ID` and configure `TAV_API_KEYS` as a JSON object mapping each client ID to the generated SHA-256 digest. Store the one-time raw key with that client; do not store or commit it in this repository. When `TAV_API_KEYS` is set, it takes precedence over `TAV_API_KEY`.