cacheverifier 0.4.0__tar.gz → 0.5.0__tar.gz

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/CHANGELOG.md

@@ -1,5 +1,35 @@
 # Changelog
 
+## 0.5.0
+
+New LangChain integration: `cacheverifier.integrations.langchain.VerifiedCache`
+wraps any LangChain cache (e.g. `RedisSemanticCache`) and verifies each hit
+before it is served; a rejected hit becomes a miss. Install with
+`pip install "cacheverifier[langchain]"`.
+
+`VerifyResult` gains two optional fields, both None unless the server sends
+them:
+
+- `notice`: the plain-language reason a call wasn't decided by a model (cold
+  start), which the API already returned but the client dropped.
+- `shadow`: on a cold-start call, a `ShadowVerdict` with what the shared stock
+  model would have decided (`approved`, `score`, `threshold`, `latency_ms`).
+  The call itself is still rejected, so this is a preview: your first calls
+  on a new account show what the model thinks before anything is served.
+
+## 0.4.1
+
+`verify()` and `verify_batch()` now treat a **rate-limited 429** (one with a
+`Retry-After` header) like a 5xx: they log a warning and return a `degraded`
+result (`approved=False` by default, per `fail_open`) instead of raising
+`CacheVerifierError`. A rate limit is transient load on your request path, so
+it should fall through to your LLM like an outage rather than throw. A 429
+*without* `Retry-After` means the monthly verify quota is used up and still
+raises, as do other 4xx responses (bad key, bad request).
+`CacheVerifierError` gains a `retry_after` attribute. Upgrade before the
+hosted service switches its rate-limit response from 503 to the documented
+429.
+
 ## 0.4.0
 
 `cacheverifier healthcheck --base-model` picks the model the local Health

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: cacheverifier
-Version: 0.4.0
+Version: 0.5.0
 Summary: Python client for the hosted CacheVerifier semantic-cache verification API
 Project-URL: Homepage, https://www.cacheverifier.com
 Project-URL: Documentation, https://www.cacheverifier.com/docs
@@ -19,6 +19,7 @@ Classifier: Typing :: Typed
 Requires-Python: >=3.9
 Requires-Dist: httpx>=0.24
 Provides-Extra: dev
+Requires-Dist: langchain-core>=0.3; extra == 'dev'
 Requires-Dist: mypy>=1.5; extra == 'dev'
 Requires-Dist: pytest>=7; extra == 'dev'
 Requires-Dist: ruff>=0.4; extra == 'dev'
@@ -28,6 +29,8 @@ Provides-Extra: healthcheck
 Requires-Dist: numpy<2,>=1.26; extra == 'healthcheck'
 Requires-Dist: sentence-transformers<4,>=3.0; extra == 'healthcheck'
 Requires-Dist: torch>=2.2; extra == 'healthcheck'
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.3; extra == 'langchain'
 Description-Content-Type: text/markdown
 
 # cacheverifier
@@ -94,6 +97,19 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
 | `model_version` | `"stock"`, `"v<id>"` (fine-tuned), `"cold_start_fail_closed"` / `"cold_start_auto_pending"`, or `"verify_unavailable"` |
 | `latency_ms` | server-side inference time |
 | `degraded` | `True` when the call failed and this result was synthesized client-side |
+| `notice` | why no model decided the call (cold start); `None` otherwise |
+| `shadow` | cold start only: a `ShadowVerdict` with what the shared stock model would have decided. The call is still rejected, so treat it as a preview |
+
+A new account starts in cold start: every call returns `approved=False` until a
+Health Check shows the shared model is accurate enough on your data. Look at
+`result.shadow` to see what it would have decided in the meantime:
+
+```python
+result = cv.verify("when does my flight board", "Boarding starts 40 minutes before departure.")
+if result.shadow:
+    print(result.notice)
+    print("stock model would approve:", result.shadow.approved, result.shadow.score)
+```
 
 ## On your request path
 
@@ -103,11 +119,13 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
   fine-tuning / feedback / monitoring calls. Warm verification is tens of
   milliseconds server-side; 1s covers the network round trip and a cold model
   load after a deploy without letting a stuck verifier stall your request.
-- **Fails closed.** On a timeout, connection error, or 5xx, `verify()` does not
-  raise — it logs a warning on the `cacheverifier` logger and returns a
-  `VerifyResult` with `degraded=True` and `approved=False`, so you fall through
-  to your LLM exactly as you would on a cache miss. A `4xx` (bad key, bad
-  request, rate limit) still raises `CacheVerifierError`.
+- **Fails closed.** On a timeout, connection error, 5xx, or a rate-limited 429
+  (one carrying `Retry-After`), `verify()` does not raise — it logs a warning on
+  the `cacheverifier` logger and returns a `VerifyResult` with `degraded=True`
+  and `approved=False`, so you fall through to your LLM exactly as you would on
+  a cache miss. Other `4xx` responses still raise `CacheVerifierError`: a bad
+  key, a bad request, or a 429 without `Retry-After`, which means your monthly
+  verify quota is used up.
 
 ```python
 cv = CacheVerifier(api_key="cv_...", verify_timeout=1.0, fail_open=False)  # the defaults
@@ -137,6 +155,30 @@ evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9
 
 See [`examples/gptcache_example.py`](examples/gptcache_example.py).
 
+## LangChain
+
+`VerifiedCache` wraps any LangChain cache, typically a semantic one, and only
+serves a hit after CacheVerifier approves it; a rejected hit is a miss, so
+your model is called as usual. For chat models it verifies against the last
+user message.
+
+```bash
+pip install "cacheverifier[langchain]"
+```
+
+```python
+from langchain_core.globals import set_llm_cache
+from langchain_redis import RedisSemanticCache
+from cacheverifier.integrations.langchain import VerifiedCache
+
+set_llm_cache(
+    VerifiedCache(
+        RedisSemanticCache(redis_url="redis://localhost:6379", embeddings=embeddings),
+        api_key="cv_...",
+    )
+)
+```
+
 ## Fine-tuning
 
 Once you have ~20+ feedback rows (the service found fine-tuning is often a net negative

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/README.md

@@ -62,6 +62,19 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
 | `model_version` | `"stock"`, `"v<id>"` (fine-tuned), `"cold_start_fail_closed"` / `"cold_start_auto_pending"`, or `"verify_unavailable"` |
 | `latency_ms` | server-side inference time |
 | `degraded` | `True` when the call failed and this result was synthesized client-side |
+| `notice` | why no model decided the call (cold start); `None` otherwise |
+| `shadow` | cold start only: a `ShadowVerdict` with what the shared stock model would have decided. The call is still rejected, so treat it as a preview |
+
+A new account starts in cold start: every call returns `approved=False` until a
+Health Check shows the shared model is accurate enough on your data. Look at
+`result.shadow` to see what it would have decided in the meantime:
+
+```python
+result = cv.verify("when does my flight board", "Boarding starts 40 minutes before departure.")
+if result.shadow:
+    print(result.notice)
+    print("stock model would approve:", result.shadow.approved, result.shadow.score)
+```
 
 ## On your request path
 
@@ -71,11 +84,13 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
   fine-tuning / feedback / monitoring calls. Warm verification is tens of
   milliseconds server-side; 1s covers the network round trip and a cold model
   load after a deploy without letting a stuck verifier stall your request.
-- **Fails closed.** On a timeout, connection error, or 5xx, `verify()` does not
-  raise — it logs a warning on the `cacheverifier` logger and returns a
-  `VerifyResult` with `degraded=True` and `approved=False`, so you fall through
-  to your LLM exactly as you would on a cache miss. A `4xx` (bad key, bad
-  request, rate limit) still raises `CacheVerifierError`.
+- **Fails closed.** On a timeout, connection error, 5xx, or a rate-limited 429
+  (one carrying `Retry-After`), `verify()` does not raise — it logs a warning on
+  the `cacheverifier` logger and returns a `VerifyResult` with `degraded=True`
+  and `approved=False`, so you fall through to your LLM exactly as you would on
+  a cache miss. Other `4xx` responses still raise `CacheVerifierError`: a bad
+  key, a bad request, or a 429 without `Retry-After`, which means your monthly
+  verify quota is used up.
 
 ```python
 cv = CacheVerifier(api_key="cv_...", verify_timeout=1.0, fail_open=False)  # the defaults
@@ -105,6 +120,30 @@ evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9
 
 See [`examples/gptcache_example.py`](examples/gptcache_example.py).
 
+## LangChain
+
+`VerifiedCache` wraps any LangChain cache, typically a semantic one, and only
+serves a hit after CacheVerifier approves it; a rejected hit is a miss, so
+your model is called as usual. For chat models it verifies against the last
+user message.
+
+```bash
+pip install "cacheverifier[langchain]"
+```
+
+```python
+from langchain_core.globals import set_llm_cache
+from langchain_redis import RedisSemanticCache
+from cacheverifier.integrations.langchain import VerifiedCache
+
+set_llm_cache(
+    VerifiedCache(
+        RedisSemanticCache(redis_url="redis://localhost:6379", embeddings=embeddings),
+        api_key="cv_...",
+    )
+)
+```
+
 ## Fine-tuning
 
 Once you have ~20+ feedback rows (the service found fine-tuning is often a net negative

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/cacheverifier/__init__.py

@@ -16,7 +16,7 @@ Docs: https://www.cacheverifier.com/docs
 Research behind it: https://github.com/imxinchengyou/CacheVerifier
 """
 
-from cacheverifier.client import CacheVerifier, CacheVerifierError, VerifyResult
+from cacheverifier.client import CacheVerifier, CacheVerifierError, ShadowVerdict, VerifyResult
 
-__version__ = "0.4.0"
-__all__ = ["CacheVerifier", "CacheVerifierError", "VerifyResult", "__version__"]
+__version__ = "0.5.0"
+__all__ = ["CacheVerifier", "CacheVerifierError", "ShadowVerdict", "VerifyResult", "__version__"]

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/cacheverifier/client.py

@@ -46,13 +46,41 @@ class CacheVerifierError(RuntimeError):
     """Raised for any non-2xx response from the API.
 
     `status_code` is the HTTP status; `detail` is the server's error message
-    (the JSON body's `detail` field when present, otherwise the raw text).
+    (the JSON body's `detail` field when present, otherwise the raw text);
+    `retry_after` is the response's `Retry-After` header, if it sent one.
     """
 
-    def __init__(self, status_code: int, detail: str) -> None:
+    def __init__(self, status_code: int, detail: str, retry_after: str | None = None) -> None:
         super().__init__(f"CacheVerifier API error {status_code}: {detail}")
         self.status_code = status_code
         self.detail = detail
+        self.retry_after = retry_after
+
+
+@dataclass(frozen=True)
+class ShadowVerdict:
+    """What the shared stock model would have decided for a cold-start call.
+
+    Only a preview: during cold start the call itself is always rejected
+    (`VerifyResult.approved` is False). The stock model hasn't been checked
+    against your data, so run a Health Check before relying on it.
+    """
+
+    approved: bool
+    score: float
+    threshold: float
+    latency_ms: float
+    model_version: str = "stock"
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> ShadowVerdict:
+        return cls(
+            approved=bool(d["approved"]),
+            score=float(d["score"]),
+            threshold=float(d["threshold"]),
+            latency_ms=float(d["latency_ms"]),
+            model_version=str(d.get("model_version", "stock")),
+        )
 
 
 @dataclass(frozen=True)
@@ -69,9 +97,14 @@ class VerifyResult:
       fallback result (see `degraded`).
     - `latency_ms`: server-side model inference time, not round-trip time.
     - `degraded`: True when the verify call failed (timeout / connection error /
-      5xx) and this result was synthesized by the client rather than returned by
-      the API. `approved` then reflects the client's `fail_open` setting
+      5xx / rate-limited 429) and this result was synthesized by the client
+      rather than returned by the API. `approved` then reflects the client's `fail_open` setting
       (default `False` -> `approved=False`, i.e. fall through to your LLM).
+    - `notice`: a plain-language reason when no model decided the call (cold
+      start); None otherwise.
+    - `shadow`: on a cold-start call, what the shared stock model would have
+      decided (a `ShadowVerdict`); None otherwise, and on servers that don't
+      send it.
     """
 
     approved: bool
@@ -80,15 +113,20 @@ class VerifyResult:
     model_version: str
     latency_ms: float
     degraded: bool = False
+    notice: str | None = None
+    shadow: ShadowVerdict | None = None
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> VerifyResult:
+        shadow = d.get("shadow")
         return cls(
             approved=bool(d["approved"]),
             score=float(d["score"]),
             threshold=float(d["threshold"]),
             model_version=str(d["model_version"]),
             latency_ms=float(d["latency_ms"]),
+            notice=d.get("notice"),
+            shadow=ShadowVerdict._from_json(shadow) if shadow else None,
         )
 
 
@@ -112,12 +150,14 @@ class CacheVerifier:
 
     - `verify_timeout` (default 1s, vs 10s for everything else) bounds how long
       a slow or overloaded verifier can stall your request.
-    - On a timeout, connection error, or 5xx, `verify()` does not raise -- it
-      logs a warning and returns a synthetic `VerifyResult` with
-      `degraded=True`. `fail_open=False` (the default) makes that result
-      `approved=False` so you fall through to your LLM; `fail_open=True` makes
-      it `approved=True` so you serve the cached answer anyway. 4xx responses
-      (bad key, bad request, rate limit) still raise `CacheVerifierError`.
+    - On a timeout, connection error, 5xx, or a rate-limited 429 (one with a
+      `Retry-After` header), `verify()` does not raise -- it logs a warning and
+      returns a synthetic `VerifyResult` with `degraded=True`.
+      `fail_open=False` (the default) makes that result `approved=False` so you
+      fall through to your LLM; `fail_open=True` makes it `approved=True` so
+      you serve the cached answer anyway. Other 4xx responses still raise
+      `CacheVerifierError`: a bad key, a bad request, or a 429 without
+      `Retry-After`, which means your monthly verify quota is used up.
     """
 
     def __init__(
@@ -157,7 +197,7 @@ class CacheVerifier:
     def verify(self, query: str, candidate_answer: str) -> VerifyResult:
         """Approve or reject one gray-zone cache hit. `POST /v1/verify`.
 
-        On a timeout / connection error / 5xx this returns a synthetic
+        On a timeout / connection error / 5xx / rate-limited 429 this returns a synthetic
         `VerifyResult` (`degraded=True`, `approved` per `fail_open`) instead of
         raising -- the verifier being unreachable should degrade to a normal
         cache miss, not an exception on your request path.
@@ -189,14 +229,19 @@ class CacheVerifier:
     def _verify_call(self, path: str, payload: dict[str, Any]) -> Any | None:
         """POST a verify request with the request-path timeout. Returns the
         parsed body, or None to signal "unavailable, fall back" on a
-        timeout / connection error / 5xx. 4xx still raises.
+        timeout / connection error / 5xx / 429 with `Retry-After`. Other 4xx
+        still raises: a bad key or request is a bug to fix, and a 429 without
+        `Retry-After` is the monthly quota running out, which needs a decision
+        (top up or subscribe), not a silent month of fallbacks. A rate limit is
+        transient load and should degrade like an outage instead.
         """
         try:
             return self._post(path, json=payload, timeout=self._verify_timeout)
         except httpx.TransportError as exc:
             self._warn_degraded(path, exc)
         except CacheVerifierError as exc:
-            if exc.status_code < 500:
+            rate_limited = exc.status_code == 429 and exc.retry_after is not None
+            if exc.status_code < 500 and not rate_limited:
                 raise
             self._warn_degraded(path, exc)
         return None
@@ -365,7 +410,7 @@ class CacheVerifier:
             detail = body["detail"] if isinstance(body, dict) and "detail" in body else resp.text
         except ValueError:
             detail = resp.text
-        raise CacheVerifierError(resp.status_code, str(detail))
+        raise CacheVerifierError(resp.status_code, str(detail), resp.headers.get("retry-after"))
 
 
 def _drop_none(**kwargs: Any) -> dict[str, Any]:

cacheverifier-0.5.0/cacheverifier/integrations/langchain.py

@@ -0,0 +1,140 @@
+"""LangChain integration: wrap any LangChain semantic cache so a hit is only
+served after CacheVerifier approves it.
+
+A semantic cache (`RedisSemanticCache`, `MongoDBAtlasSemanticCache`, ...)
+returns a stored answer whenever a past prompt is similar enough to the new
+one, including when the wording differs in a way that changes the answer.
+`VerifiedCache` sits in front of it: on a hit it asks `/v1/verify` whether
+the cached answer fits the new question, and turns a rejected hit into a
+miss, so your model is called as usual.
+
+    from langchain_core.globals import set_llm_cache
+    from langchain_redis import RedisSemanticCache
+    from cacheverifier.integrations.langchain import VerifiedCache
+
+    set_llm_cache(
+        VerifiedCache(
+            RedisSemanticCache(redis_url="redis://localhost:6379", embeddings=embeddings),
+            api_key="cv_...",
+        )
+    )
+
+Writes, clears and exact-match caches pass straight through. If the verify
+call times out or the service is unreachable, the hit is rejected by
+default (a verifier outage becomes a normal cache miss); pass
+`fail_open=True` to serve it instead.
+
+`langchain-core` is an optional dependency -- `pip install "cacheverifier[langchain]"`.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from typing import Any
+
+from cacheverifier.client import (
+    DEFAULT_BASE_URL,
+    DEFAULT_TIMEOUT,
+    DEFAULT_VERIFY_TIMEOUT,
+    CacheVerifier,
+)
+
+try:
+    from langchain_core.caches import RETURN_VAL_TYPE, BaseCache
+except ImportError as exc:  # pragma: no cover - exercised only without the extra
+    raise ImportError(
+        'cacheverifier.integrations.langchain needs langchain-core: pip install "cacheverifier[langchain]"'
+    ) from exc
+
+
+def _message_text(content: Any) -> str:
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):  # multimodal content blocks
+        return "\n".join(
+            block if isinstance(block, str) else str(block.get("text", ""))
+            for block in content
+            if isinstance(block, str) or (isinstance(block, dict) and block.get("type") == "text")
+        )
+    return ""
+
+
+def user_question(prompt: str) -> str:
+    """The text to verify against for a LangChain cache key.
+
+    Plain LLMs use the prompt string itself as the key. Chat models use a
+    JSON dump of the whole message list, system prompt and history
+    included; the question being asked is the last human message, so that
+    is what the verifier sees. Anything unrecognized falls back to the raw
+    prompt."""
+    if not prompt.startswith("["):
+        return prompt
+    try:
+        messages = json.loads(prompt)
+    except ValueError:
+        return prompt
+    for message in reversed(messages if isinstance(messages, list) else []):
+        kwargs = message.get("kwargs", {}) if isinstance(message, dict) else {}
+        if kwargs.get("type") == "human":
+            text = _message_text(kwargs.get("content"))
+            if text:
+                return text
+    return prompt
+
+
+class VerifiedCache(BaseCache):
+    """A LangChain cache that verifies another cache's hits before serving them."""
+
+    def __init__(
+        self,
+        cache: BaseCache,
+        api_key: str | None = None,
+        *,
+        client: CacheVerifier | None = None,
+        base_url: str = DEFAULT_BASE_URL,
+        timeout: float = DEFAULT_TIMEOUT,
+        verify_timeout: float = DEFAULT_VERIFY_TIMEOUT,
+        fail_open: bool = False,
+    ) -> None:
+        if client is None:
+            if not api_key:
+                raise ValueError("pass api_key or client")
+            client = CacheVerifier(
+                api_key=api_key,
+                base_url=base_url,
+                timeout=timeout,
+                verify_timeout=verify_timeout,
+                fail_open=fail_open,
+            )
+        self.cache = cache
+        self.client = client
+
+    def _approved(self, prompt: str, hit: RETURN_VAL_TYPE) -> bool:
+        answer = "\n".join(generation.text for generation in hit)
+        return self.client.verify(user_question(prompt), answer).approved
+
+    def lookup(self, prompt: str, llm_string: str) -> RETURN_VAL_TYPE | None:
+        hit = self.cache.lookup(prompt, llm_string)
+        if not hit:
+            return hit
+        return hit if self._approved(prompt, hit) else None
+
+    async def alookup(self, prompt: str, llm_string: str) -> RETURN_VAL_TYPE | None:
+        hit = await self.cache.alookup(prompt, llm_string)
+        if not hit:
+            return hit
+        approved = await asyncio.get_running_loop().run_in_executor(None, self._approved, prompt, hit)
+        return hit if approved else None
+
+    def update(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> None:
+        self.cache.update(prompt, llm_string, return_val)
+
+    async def aupdate(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> None:
+        await self.cache.aupdate(prompt, llm_string, return_val)
+
+    def clear(self, **kwargs: Any) -> None:
+        self.cache.clear(**kwargs)
+
+    async def aclear(self, **kwargs: Any) -> None:
+        await self.cache.aclear(**kwargs)

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "cacheverifier"
-version = "0.4.0"
+version = "0.5.0"
 description = "Python client for the hosted CacheVerifier semantic-cache verification API"
 readme = "README.md"
 requires-python = ">=3.9"
@@ -40,6 +40,7 @@ Research = "https://github.com/imxinchengyou/CacheVerifier"
 
 [project.optional-dependencies]
 gptcache = ["gptcache>=0.1.30"]
+langchain = ["langchain-core>=0.3"]
 # `cacheverifier healthcheck` -- runs the hosted Health Check Report's
 # stock-vs-fine-tuned evaluation locally. Deliberately NOT a base
 # dependency: torch + sentence-transformers dwarf the httpx-only client.
@@ -54,7 +55,7 @@ healthcheck = [
     "torch>=2.2",
     "sentence-transformers>=3.0,<4",
 ]
-dev = ["pytest>=7", "mypy>=1.5", "ruff>=0.4"]
+dev = ["pytest>=7", "mypy>=1.5", "ruff>=0.4", "langchain-core>=0.3"]
 
 [tool.hatch.build.targets.wheel]
 packages = ["cacheverifier"]

{cacheverifier-0.4.0 → cacheverifier-0.5.0}/tests/test_client.py

@@ -6,7 +6,7 @@ import json
 import httpx
 import pytest
 
-from cacheverifier import CacheVerifier, CacheVerifierError, VerifyResult
+from cacheverifier import CacheVerifier, CacheVerifierError, ShadowVerdict, VerifyResult
 
 
 def make_client(handler, **kwargs):
@@ -51,6 +51,32 @@ def test_verify_parses_result_and_sends_api_key():
     assert seen["path"] == "/v1/verify"
     assert seen["api_key"] == "cv_test"
     assert seen["body"] == {"query": "how do I cancel", "candidate_answer": "Go to Settings > Billing."}
+    assert result.notice is None
+    assert result.shadow is None
+
+
+def test_verify_parses_cold_start_notice_and_shadow_verdict():
+    def handler(request: httpx.Request) -> httpx.Response:
+        return httpx.Response(
+            200,
+            json={
+                "approved": False,
+                "score": 0.0,
+                "latency_ms": 0.0,
+                "model_version": "cold_start_auto_pending",
+                "threshold": 0.0,
+                "decision_rule": "cold_start",
+                "notice": "No verifier has been checked against your data yet.",
+                "shadow": {"approved": True, "score": 2.1, "threshold": 0.0, "latency_ms": 18.0, "model_version": "stock"},
+            },
+        )
+
+    with make_client(handler) as cv:
+        result = cv.verify("when does my flight board", "Boarding starts 40 minutes before departure.")
+
+    assert result.approved is False
+    assert result.notice.startswith("No verifier")
+    assert result.shadow == ShadowVerdict(approved=True, score=2.1, threshold=0.0, latency_ms=18.0)
 
 
 def test_verify_batch_returns_ordered_results():
@@ -135,6 +161,7 @@ def test_error_response_raises_with_detail():
 
     assert excinfo.value.status_code == 429
     assert excinfo.value.detail == "free tier limit reached"
+    assert excinfo.value.retry_after is None
 
 
 def test_error_response_without_json_body_falls_back_to_text():
@@ -204,14 +231,35 @@ def test_verify_fails_closed_on_5xx():
     assert result.approved is False
 
 
-def test_verify_still_raises_on_4xx():
+def test_verify_fails_closed_on_rate_limit_429():
     def handler(request: httpx.Request) -> httpx.Response:
-        return httpx.Response(429, json={"detail": "rate limited"})
+        return httpx.Response(429, headers={"Retry-After": "1"}, json={"detail": "rate limited"})
+
+    with make_client(handler) as cv:
+        result = cv.verify("q", "a")
+
+    assert result.degraded is True
+    assert result.approved is False
+
+
+def test_verify_batch_degrades_on_rate_limit_429():
+    def handler(request: httpx.Request) -> httpx.Response:
+        return httpx.Response(429, headers={"Retry-After": "1"}, json={"detail": "rate limited"})
+
+    with make_client(handler) as cv:
+        results = cv.verify_batch([("q1", "a1"), ("q2", "a2")])
+
+    assert all(r.degraded and not r.approved for r in results)
+
+
+def test_verify_still_raises_on_other_4xx():
+    def handler(request: httpx.Request) -> httpx.Response:
+        return httpx.Response(401, json={"detail": "invalid API key"})
 
     with make_client(handler) as cv, pytest.raises(CacheVerifierError) as excinfo:
         cv.verify("q", "a")
 
-    assert excinfo.value.status_code == 429
+    assert excinfo.value.status_code == 401
 
 
 def test_verify_batch_degrades_every_pair_on_failure():

cacheverifier-0.5.0/tests/test_langchain_integration.py

@@ -0,0 +1,90 @@
+"""VerifiedCache against a real langchain-core cache and chat model, with
+every verify call served by an httpx.MockTransport."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+
+import httpx
+import pytest
+
+pytest.importorskip("langchain_core")
+
+from langchain_core.caches import InMemoryCache
+from langchain_core.language_models.fake_chat_models import FakeListChatModel
+from langchain_core.messages import HumanMessage, SystemMessage
+from langchain_core.outputs import Generation
+
+from cacheverifier import CacheVerifier
+from cacheverifier.integrations.langchain import VerifiedCache, user_question
+
+
+def _verified(approve: bool, seen: list | None = None, inner=None) -> VerifiedCache:
+    def handler(request: httpx.Request) -> httpx.Response:
+        if seen is not None:
+            seen.append(json.loads(request.content))
+        return httpx.Response(
+            200,
+            json={"approved": approve, "score": 1.0, "latency_ms": 5.0, "model_version": "stock", "threshold": 0.0},
+        )
+
+    client = CacheVerifier(api_key="cv_test", transport=httpx.MockTransport(handler))
+    return VerifiedCache(inner or InMemoryCache(), client=client)
+
+
+def test_approved_hit_is_served_and_rejected_hit_becomes_a_miss():
+    seen: list = []
+    cache = _verified(True, seen)
+    cache.update("when does my flight board", "llm", [Generation(text="40 minutes before departure.")])
+    assert cache.lookup("when does my flight board", "llm")[0].text == "40 minutes before departure."
+    assert seen == [{"query": "when does my flight board", "candidate_answer": "40 minutes before departure."}]
+
+    rejecting = _verified(False, inner=cache.cache)
+    assert rejecting.lookup("when does my flight board", "llm") is None
+
+
+def test_a_miss_never_calls_the_verifier():
+    seen: list = []
+    cache = _verified(True, seen)
+    assert cache.lookup("never cached", "llm") is None
+    assert seen == []
+
+
+def test_chat_model_verifies_against_the_last_human_message():
+    seen: list = []
+    cache = _verified(True, seen)
+    model = FakeListChatModel(responses=["Gate B12, boarding at 9:20.", "should not be called"], cache=cache)
+    messages = [SystemMessage("You are an airline assistant."), HumanMessage("which gate is my flight")]
+
+    assert model.invoke(messages).content == "Gate B12, boarding at 9:20."  # miss: model called, then cached
+    assert model.invoke(messages).content == "Gate B12, boarding at 9:20."  # hit: verified, served
+    assert seen == [{"query": "which gate is my flight", "candidate_answer": "Gate B12, boarding at 9:20."}]
+
+
+def test_rejected_chat_hit_falls_through_to_the_model():
+    cache = _verified(False)
+    model = FakeListChatModel(responses=["first answer", "fresh answer"], cache=cache)
+    assert model.invoke("what is the baggage allowance").content == "first answer"
+    assert model.invoke("what is the baggage allowance").content == "fresh answer"
+
+
+def test_async_lookup_verifies_too():
+    cache = _verified(False)
+    asyncio.run(cache.aupdate("q", "llm", [Generation(text="a")]))
+    assert asyncio.run(cache.alookup("q", "llm")) is None
+    assert cache.cache.lookup("q", "llm")[0].text == "a"  # the inner cache still has it
+
+
+def test_user_question_falls_back_to_the_raw_prompt():
+    assert user_question("plain prompt") == "plain prompt"
+    assert user_question("[not json") == "[not json"
+    multimodal = json.dumps(
+        [{"kwargs": {"type": "human", "content": [{"type": "text", "text": "what's in this photo"}, {"type": "image_url"}]}}]
+    )
+    assert user_question(multimodal) == "what's in this photo"
+
+
+def test_needs_a_key_or_a_client():
+    with pytest.raises(ValueError):
+        VerifiedCache(InMemoryCache())