mcp-token-saver 0.1.0__tar.gz

mcp_token_saver-0.1.0/.gitignore

@@ -0,0 +1,74 @@
+# --- Python ---
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+wheelhouse/*
+!wheelhouse/.gitkeep
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# Virtual environments
+.venv/
+venv/
+env/
+ENV/
+.python-version
+
+# Testing / coverage
+.pytest_cache/
+.coverage
+.coverage.*
+htmlcov/
+.tox/
+.nox/
+coverage.xml
+*.cover
+.cache
+.hypothesis/
+
+# Type checking
+.mypy_cache/
+.pyre/
+.pytype/
+.ruff_cache/
+
+# IDEs
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# --- Docker ---
+*.pid
+docker-compose.override.yml
+.docker/
+
+# --- Project-local ---
+.env
+.env.local
+
+# Personal dev-only scripts (not shipped)
+scripts/test-install.sh

mcp_token_saver-0.1.0/CHANGELOG.md

@@ -0,0 +1,118 @@
+# Changelog
+
+All notable changes to `mcp-token-saver` are documented here.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
+the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased] — pre-PyPI hardening
+
+### Added
+
+- **Async API.** `TokenSaver.afilter_tools(...)` and the module-level
+  `afilter_tools(...)` offload the sync routing dispatch to a worker thread via
+  `asyncio.to_thread`, so async gateways (FastAPI, Starlette, aiohttp) no
+  longer block their event loop during Haiku roundtrips.
+- **`TokenSaver.aindex_catalog(...)`** async wrapper for the catalog embedding
+  step. Prevents the ~30-50s Titan Embed loop from blocking during FastAPI
+  `lifespan` startup.
+- **Retry with exponential backoff for every Bedrock call.**
+  `_bedrock_retry.call_with_retry` wraps `invoke_model` in the Titan embedding
+  provider and the LLM router with jittered retries (3 attempts by default) on
+  transient error codes (`ThrottlingException`, `ServiceUnavailableException`,
+  `InternalServerException`, timeouts). Non-transient failures (validation,
+  auth) still fail fast.
+- **`TokenSaverConfig.provider_kind`** with values `"auto"` (default),
+  `"bedrock"` (force Bedrock, fail loud), `"hash"` (force
+  `HashEmbeddingProvider`, guaranteed zero Bedrock billing). Fixes the
+  "accidental AWS invoice from ambient `AWS_ACCESS_KEY_ID`" footgun.
+- **`TokenSaverConfig.hybrid_low_similarity_action`** with three modes:
+  `"haiku_shortlist"` (default), `"haiku_full"`, `"fallback_full"`. Controls
+  how the `hybrid` strategy handles low-confidence embeddings queries.
+- **`TokenSaver(..., routing_strategy=...)`** — injection point for a custom
+  routing strategy. Bypasses the built-in `embeddings_topk / llm_router /
+  hybrid` dispatch. Also available as
+  `TokenSaver.default(..., routing_strategy=...)`. Docs include an OpenAI
+  GPT-4o-mini example.
+- **`TOKEN_SAVER_STORE`** env var for the demo gateway to switch between
+  `numpy` (default) and `faiss` vector stores without touching code. Falls
+  back to numpy with a warning if the `[faiss]` extra is missing.
+- **`TOKEN_SAVER_ROUTER_MODEL`** env var for the demo gateway to override the
+  Haiku model id (e.g. to point at a different region's cross-region inference
+  profile).
+- **`PricingRegistry.from_dict`** and **`PricingRegistry.merge_file`** — the
+  former builds a fresh registry from the canonical `{"models": {...}}` shape,
+  the latter mutates an existing registry in place. `load_file` still returns
+  a fresh registry (classmethod).
+- **Curated third-party pricing defaults** for OpenAI (`gpt-4o`,
+  `gpt-4o-mini`, `gpt-4-turbo`, `text-embedding-3-*`) and Anthropic direct API
+  model ids so the OpenAI custom-router example works out of the box.
+- **CHANGELOG.md** (this file).
+
+### Changed
+
+- **Concise INFO tracing on every routing decision.** The gateway logs one
+  line per `filter_tools` call and per `index_catalog` boot, with strategy,
+  top similarity, tokens saved, Haiku cost, and latency. Explicit
+  `LLM router: invoking …` lines when the LLM router is used.
+- **`RoutingEvent` gets a `__post_init__`** that coerces `selected_tool_names`
+  from list/iterable to tuple, validates non-negative token counts, and
+  canonicalises float precision so JSON round-trips are lossless.
+- **Default Haiku model id** now `eu.anthropic.claude-haiku-4-5-20251001-v1:0`
+  (previously the US-only `anthropic.claude-3-5-haiku-20241022-v1:0`, which
+  Bedrock rejected in EU regions with a `ValidationException`).
+- **`RoutingEvent.latency_ms`** is now `float` instead of `int` — measurements
+  from `time.perf_counter()` were being truncated on the way in.
+- **LLM router response parser hardened.** `_extract_json_array` handles raw
+  JSON, markdown-fenced JSON, and JSON embedded in prose so future Claude
+  releases that decorate output do not crash the router.
+- **`README`** rewritten to document the three strategies, the pluggable
+  interfaces, the custom-routing example, the pricing extension mechanism,
+  and the demo tuning knobs.
+
+### Removed
+
+- **`mcp_token_saver.events.RoutingCosts`.** Duplicate of the (correct)
+  `router.base.RoutingCosts`. The `events` module now exports only
+  `RoutingEvent` and `EventBus`.
+- **`mcp_token_saver.stores.numpy_inmem.InMemoryNumpyStore`.** Despite the
+  name it was pure Python (no numpy), i.e. an unnecessary duplicate of
+  `PurePythonStore`. Removed from `__init__` exports and from the vector
+  stores table in `README`.
+- **Langfuse integration** — the sink, the docker-compose services
+  (`postgres`, `clickhouse`, `redis`, `langfuse-web`, `langfuse-worker`), the
+  `example/langfuse/` directory, all `LANGFUSE_*` env vars, and the
+  `[langfuse]` install extra. The demo now ships a minimal 4-service stack
+  (mcp-server, gateway, agent, ui). Routing metrics live in the Streamlit UI
+  and the structured gateway logs.
+
+### Fixed
+
+- **`hybrid` strategy no longer AttributeErrors** on low-similarity queries
+  (the code referenced `cfg.hybrid_low_similarity_action` but the field did
+  not exist in `TokenSaverConfig`).
+- **`llm_router` and `hybrid` were dispatched as pure labels**, not actually
+  invoked. `TokenSaver.filter_tools` now truly dispatches to
+  `LLMRouterStrategy` / `HybridStrategy` based on `strategy_name`.
+- **`RoutingCosts` import mismatch** between strategies (`events` version) and
+  the `RoutingStrategy` protocol (`router.base` version) was raising
+  `TypeError: missing 4 required positional arguments` when `llm_router` /
+  `hybrid` were selected. Both strategies now import from `router.base`.
+- **Test suite realigned with production code.** `test_api.py` and
+  `test_router_invariants.py` were written against an older constructor
+  signature (`strategy=..., pricing_registry=...`) and referenced classes that
+  no longer existed. Rewritten to match the current `TokenSaver(pricing=...,
+  routing_strategy=...)` shape and use the deduplicated `router.base`
+  imports.
+- **Demo gateway boot** — `mcp_server`, `mcp_gateway`, `agent`, and `ui`
+  `Dockerfile`s were failing on hatchling metadata due to missing
+  `README.md` files. Cleaned up.
+- **`agent` and `ui` container CMDs** referenced modules
+  (`agent.main:app`, `src/ui/app.py`) that were not on the container's
+  `PYTHONPATH`. Package layouts fixed so `uvicorn agent.main:app` and
+  `streamlit run /app/src/ui/app.py` both resolve.
+
+## [0.1.0] — initial internal draft
+
+- Initial semantic tool filtering skeleton, three routing strategies,
+  Bedrock-only demo stack, Langfuse observability sink.

mcp_token_saver-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 mcp-token-saver contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.