cacheverifier 0.6.0__tar.gz → 0.7.1__tar.gz

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/.github/workflows/ci.yml

@@ -7,7 +7,9 @@ on:
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+    # ubuntu-latest is 26.04, which setup-python has no 3.9 build for; 3.9
+    # stays on 24.04 until it is dropped from requires-python.
+    runs-on: ${{ matrix.python-version == '3.9' && 'ubuntu-24.04' || 'ubuntu-latest' }}
     strategy:
       fail-fast: false
       matrix:

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## 0.7.1
+
+README only, no code change, so the PyPI page matches the docs: the
+`VerifyResult` table lists `audit`, `verification_id` and `decision_rule`;
+the quickstart reports audited calls; certified thresholds are described as
+informational, valid while traffic resembles the calibration data, and
+covering only verified hits; the offline Health Check notes that the CLI
+defaults to `ms_marco` while hosted fine-tunes default to `nli`; known limits
+cover the stock model's MS MARCO training-data terms.
+
+## 0.7.0
+
+`entries_to_refresh()` lists the cache entries your feedback reported stale,
+or wrong more often than right, and `resolve_entry(key)` marks one refreshed
+(`GET /v1/monitor/entries-to-refresh`, every plan). `feedback()` (on both
+`CacheVerifier` and `LocalVerifier`) takes `cache_entry_id`, your own key for
+the entry, which the list hands back so you can refresh or drop it directly.
+
 ## 0.6.0
 
 New `LocalVerifier` (`pip install "cacheverifier[local]"`, Managed plans):

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: cacheverifier
-Version: 0.6.0
+Version: 0.7.1
 Summary: Python client for the CacheVerifier semantic-cache verification API, hosted or scored locally
 Project-URL: Homepage, https://www.cacheverifier.com
 Project-URL: Documentation, https://www.cacheverifier.com/docs
@@ -43,18 +43,44 @@ Description-Content-Type: text/markdown
 [![PyPI](https://img.shields.io/pypi/v/cacheverifier)](https://pypi.org/project/cacheverifier/)
 [![Python](https://img.shields.io/pypi/pyversions/cacheverifier)](https://pypi.org/project/cacheverifier/)
 
-Python client for **[CacheVerifier](https://www.cacheverifier.com)** — a hosted API that
-verifies semantic-cache hits. Given a query and a candidate cached answer, it approves or
-rejects serving that answer from cache, so a similarity match that is *close but wrong*
-doesn't become a silent error in your app.
+Python client for **[CacheVerifier](https://www.cacheverifier.com)**: verification and
+operations for semantic caches. It checks the cache hits a similarity threshold can't call,
+then keeps your cache right after launch.
+
+- **Verify the uncertain hits.** Given a query and a candidate cached answer, approve or
+  reject serving it in tens of milliseconds, so a match that is *close but wrong* doesn't
+  become a silent error. Fine-tuned on your own feedback.
+- **Know your real error rate.** Randomized audits of approved hits give a conservative
+  (Clopper-Pearson) upper bound on how often a served answer is wrong, measured on your
+  traffic, not a benchmark.
+- **Certified thresholds.** At fine-tune time, Conformal Risk Control can report a threshold
+  that keeps false reuses under a target you set. It is informational (`verify()` doesn't
+  apply it), holds only while traffic resembles the calibration data, and covers only the
+  hits you send to `verify()`, not the ones your cache serves on similarity alone. An
+  optional online (ACI) threshold does change decisions, following what the audits find.
+- **Catch drift and recalibrate.** When traffic moves away from what the verifier learned,
+  the dashboard flags it; on Team and Scale you also get a webhook alert and it is
+  retrained on your feedback automatically.
+- **Know which entries to fix.** `entries_to_refresh()` lists the cache entries your
+  feedback reported stale, or wrong more often than right, with your own cache keys.
+- **Run it in your stack.** On Managed plans, `LocalVerifier` scores with your fine-tuned
+  model locally: no network hop, your texts stay with you, nothing metered.
 
 CacheVerifier does **not** run your cache or do similarity search. Your cache backend does
 its own lookup first; you call `verify()` only on the candidates in the similarity "gray
 zone", where a plain threshold match might be wrong.
 
+| | answers | examples |
+|---|---|---|
+| Static-threshold caches | "how similar is it?" | GPTCache, LiteLLM, Portkey, Redis LangCache |
+| Adaptive-threshold caches | "how similar, with a learned per-entry cutoff?" (replaces your cache) | vCache |
+| **Verification layer** | "is this cached answer right for this question?" (sits on top of your cache) | **CacheVerifier** |
+
 - Docs / API reference: <https://www.cacheverifier.com/docs>
 - Why similarity ≠ correctness: <https://www.cacheverifier.com/why-similarity-fails>
-- The research behind it (paper + benchmarks): <https://github.com/imxinchengyou/CacheVerifier>
+- The research behind it (four papers, code and raw results): <https://github.com/imxinchengyou/CacheVerifier>
+- TypeScript / Node: no npm package yet; the docs have a dependency-free `fetch` client and a
+  LangChain.js cache wrapper to copy: <https://www.cacheverifier.com/docs#typescript>
 
 ## Install
 
@@ -73,25 +99,31 @@ are opt-in.
 
 ## Quickstart
 
-Get a free API key at <https://www.cacheverifier.com> (self-serve verify and fine-tuning
-are free forever, no card).
+Get a free API key at <https://www.cacheverifier.com> (verify, fine-tuning, audits and
+entries to refresh are free, no card).
 
 ```python
 from cacheverifier import CacheVerifier
 
 cv = CacheVerifier(api_key="cv_...")
 
-query = "how do I cancel my subscription"
-candidate = "Go to Settings > Billing > Pause subscription for a month."  # from your cache
+query = "can I change the delivery address after my parcel ships"
+candidate = "You can change the delivery address any time before the parcel ships."  # from your cache
 
 result = cv.verify(query, candidate)
 if result.approved:
     answer = candidate                       # verified hit — skip the LLM call
 else:
     answer = call_your_llm(query)            # not trustworthy — fall through
-
-# Later, once you know if it was actually right (thumbs-down, reopened ticket, ...):
-cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
+    if result.audit:                         # a randomized audit of an approval: report whether
+        cv.feedback(query, candidate,        # the cached answer matched the fresh one
+                    was_correct=same_answer(answer, candidate),
+                    verification_id=result.verification_id)
+
+# Later, label the cached candidate (not the fresh LLM answer): was it right for this
+# query? (a thumbs-down, a reopened ticket, a reviewer's call)
+cv.feedback(query, candidate, was_correct=candidate_was_right, similarity_score=0.86,
+            verification_id=result.verification_id)
 ```
 
 `verify()` returns a `VerifyResult`:
@@ -99,7 +131,10 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
 | field | meaning |
 |---|---|
 | `approved` | serve the cached answer (`True`) or fall through (`False`) |
-| `score` / `threshold` | `approved` is `score >= threshold` |
+| `audit` | `True` when an approval was drawn for a randomized audit, so `approved` is `False`: call your LLM, serve its answer, and report whether the cached one matched |
+| `verification_id` | set while auditing is on; pass it to `feedback()` |
+| `score` / `threshold` | with `decision_rule="threshold"`, `approved` is `score >= threshold` |
+| `decision_rule` | `"threshold"`, `"joint"` / `"joint_gap"` (a fitted rule that also uses your `similarity_score`), `"aci"` (online threshold), or `"cold_start"` |
 | `model_version` | `"stock"`, `"v<id>"` (fine-tuned), `"cold_start_fail_closed"` / `"cold_start_auto_pending"`, or `"verify_unavailable"` |
 | `latency_ms` | server-side inference time |
 | `degraded` | `True` when the call failed and this result was synthesized client-side |
@@ -144,23 +179,6 @@ cv = CacheVerifier(api_key="cv_...", fail_open=True)
 There is no formal uptime SLA yet, which is the other reason the fallback path
 is a built-in default rather than left to you.
 
-## GPTCache
-
-Drop the verifier into a GPTCache pipeline as its similarity evaluator — no fork required:
-
-```python
-from gptcache import cache
-from cacheverifier.integrations.gptcache import CacheVerifierEvaluation
-
-evaluator = CacheVerifierEvaluation(api_key="cv_...")
-cache.init(similarity_evaluation=evaluator, ...)
-
-# when you learn a served hit's real outcome:
-evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9)
-```
-
-See [`examples/gptcache_example.py`](examples/gptcache_example.py).
-
 ## LangChain
 
 `VerifiedCache` wraps any LangChain cache, typically a semantic one, and only
@@ -185,6 +203,36 @@ set_llm_cache(
 )
 ```
 
+## GPTCache
+
+Drop the verifier into a GPTCache pipeline as its similarity evaluator — no fork required:
+
+```python
+from gptcache import cache
+from cacheverifier.integrations.gptcache import CacheVerifierEvaluation
+
+evaluator = CacheVerifierEvaluation(api_key="cv_...")
+cache.init(similarity_evaluation=evaluator, ...)
+
+# when you learn a served hit's real outcome:
+evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9)
+```
+
+See [`examples/gptcache_example.py`](examples/gptcache_example.py).
+
+## Entries to refresh
+
+Send your cache's own key with feedback, and the service tells you which
+entries to fix: ones reported stale, or wrong more often than right.
+
+```python
+cv.feedback(query, cached_answer, was_correct=False, stale=True, cache_entry_id="faq:shipping")
+
+for e in cv.entries_to_refresh():
+    my_cache.delete(e["cache_entry_id"])   # or regenerate the answer
+    cv.resolve_entry(e["key"])
+```
+
 ## Fine-tuning
 
 Once you have ~20+ feedback rows (the service found fine-tuning is often a net negative
@@ -260,12 +308,14 @@ fully offline. `--emit-summary` writes an aggregate-only JSON file (AUCs, counts
 no query or answer text) that's safe to share for a human read.
 
 **Choosing the base model.** `--base-model` takes the same values as the hosted
-service's `base_model` fine-tune parameter, so local and hosted numbers stay comparable:
+service's `base_model` fine-tune parameter. The defaults differ: the CLI starts from
+`ms_marco`, hosted fine-tunes from `nli`. To compare a local run with a hosted fine-tune,
+pass `--base-model nli`.
 
 | `--base-model` | model | use for |
 |---|---|---|
-| `ms_marco` (default) | `cross-encoder/ms-marco-MiniLM-L6-v2` | English traffic |
-| `nli` | `cross-encoder/nli-MiniLM2-L6-H768` | English; entailment pretraining, stronger on negation / entity swaps |
+| `ms_marco` (CLI default) | `cross-encoder/ms-marco-MiniLM-L6-v2` | English traffic |
+| `nli` (hosted default) | `cross-encoder/nli-MiniLM2-L6-H768` | English; entailment pretraining, stronger on negation / entity swaps |
 | `multilingual` | `BAAI/bge-reranker-base` | Chinese and other non-English traffic (~6 GB RAM to fine-tune) |
 | `multilingual_small` | `cross-encoder/mmarco-mMiniLMv2-L12-H384-v1` | same, smaller (~3 GB RAM) |
 
@@ -295,6 +345,30 @@ verdict
   IMPROVED   -- fine-tuning on your own data helps this traffic
 ```
 
+## Recommended production setup
+
+- Keep randomized audits on (the default for new accounts) and report each audited call
+  with `feedback(..., verification_id=r.verification_id)`, or `result=r` on
+  `LocalVerifier`. That is what makes the measured error rate real.
+- Fine-tune with `cost_ratio=` (how many times worse a wrong reuse is than an extra LLM
+  call) instead of the default threshold. ACI (hosted only) is optional: it lets the
+  threshold follow the audits, but on the one real drifting stream we tested it gave no
+  clear advantage over a fixed threshold.
+- Send `cache_entry_id=` with feedback so `entries_to_refresh()` hands back your keys.
+- At volume, score locally with `LocalVerifier`.
+
+## Known limits
+
+- The hosted verifier is **English-only** today. Don't send Chinese or other non-English
+  traffic to it; multilingual base models are evaluated in the docs but not deployed.
+- Query + answer share a **128-token** input; longer pairs are truncated head and tail.
+- Verify endpoints allow about **20 requests/second per API key**. Use `verify_batch()` or
+  `LocalVerifier` beyond that.
+- No formal uptime SLA yet; `verify()` fails closed by default (see above).
+- The shared stock model (`cross-encoder/ms-marco-MiniLM-L6-v2`) is Apache-2.0 but trained
+  on MS MARCO, whose terms describe it as for non-commercial research. Fine-tunes start from
+  the NLI base (`cross-encoder/nli-MiniLM2-L6-H768`, Apache-2.0) by default.
+
 ## API surface
 
 | method | endpoint |
@@ -306,6 +380,7 @@ verdict
 | `get_finetune_job(id)` / `list_finetune_jobs()` | `GET /v1/finetune/jobs[/id]` |
 | `activate_model_version(id)` | `POST /v1/finetune/model-versions/{id}/activate` |
 | `drift_status()` | `GET /v1/monitor/drift-status` |
+| `entries_to_refresh()` / `resolve_entry(key)` | `GET /v1/monitor/entries-to-refresh` / `POST .../{key}/resolve` |
 | `gray_zone_threshold()` | `GET /v1/monitor/gray-zone-threshold` |
 | `usage()` / `savings()` | `GET /v1/usage/status` / `/savings` |
 | `LocalVerifier.verify(...)` / `verify_batch(...)` | local; decisions to `POST /v1/verify/decisions` |

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/README.md

@@ -4,18 +4,44 @@
 [![PyPI](https://img.shields.io/pypi/v/cacheverifier)](https://pypi.org/project/cacheverifier/)
 [![Python](https://img.shields.io/pypi/pyversions/cacheverifier)](https://pypi.org/project/cacheverifier/)
 
-Python client for **[CacheVerifier](https://www.cacheverifier.com)** — a hosted API that
-verifies semantic-cache hits. Given a query and a candidate cached answer, it approves or
-rejects serving that answer from cache, so a similarity match that is *close but wrong*
-doesn't become a silent error in your app.
+Python client for **[CacheVerifier](https://www.cacheverifier.com)**: verification and
+operations for semantic caches. It checks the cache hits a similarity threshold can't call,
+then keeps your cache right after launch.
+
+- **Verify the uncertain hits.** Given a query and a candidate cached answer, approve or
+  reject serving it in tens of milliseconds, so a match that is *close but wrong* doesn't
+  become a silent error. Fine-tuned on your own feedback.
+- **Know your real error rate.** Randomized audits of approved hits give a conservative
+  (Clopper-Pearson) upper bound on how often a served answer is wrong, measured on your
+  traffic, not a benchmark.
+- **Certified thresholds.** At fine-tune time, Conformal Risk Control can report a threshold
+  that keeps false reuses under a target you set. It is informational (`verify()` doesn't
+  apply it), holds only while traffic resembles the calibration data, and covers only the
+  hits you send to `verify()`, not the ones your cache serves on similarity alone. An
+  optional online (ACI) threshold does change decisions, following what the audits find.
+- **Catch drift and recalibrate.** When traffic moves away from what the verifier learned,
+  the dashboard flags it; on Team and Scale you also get a webhook alert and it is
+  retrained on your feedback automatically.
+- **Know which entries to fix.** `entries_to_refresh()` lists the cache entries your
+  feedback reported stale, or wrong more often than right, with your own cache keys.
+- **Run it in your stack.** On Managed plans, `LocalVerifier` scores with your fine-tuned
+  model locally: no network hop, your texts stay with you, nothing metered.
 
 CacheVerifier does **not** run your cache or do similarity search. Your cache backend does
 its own lookup first; you call `verify()` only on the candidates in the similarity "gray
 zone", where a plain threshold match might be wrong.
 
+| | answers | examples |
+|---|---|---|
+| Static-threshold caches | "how similar is it?" | GPTCache, LiteLLM, Portkey, Redis LangCache |
+| Adaptive-threshold caches | "how similar, with a learned per-entry cutoff?" (replaces your cache) | vCache |
+| **Verification layer** | "is this cached answer right for this question?" (sits on top of your cache) | **CacheVerifier** |
+
 - Docs / API reference: <https://www.cacheverifier.com/docs>
 - Why similarity ≠ correctness: <https://www.cacheverifier.com/why-similarity-fails>
-- The research behind it (paper + benchmarks): <https://github.com/imxinchengyou/CacheVerifier>
+- The research behind it (four papers, code and raw results): <https://github.com/imxinchengyou/CacheVerifier>
+- TypeScript / Node: no npm package yet; the docs have a dependency-free `fetch` client and a
+  LangChain.js cache wrapper to copy: <https://www.cacheverifier.com/docs#typescript>
 
 ## Install
 
@@ -34,25 +60,31 @@ are opt-in.
 
 ## Quickstart
 
-Get a free API key at <https://www.cacheverifier.com> (self-serve verify and fine-tuning
-are free forever, no card).
+Get a free API key at <https://www.cacheverifier.com> (verify, fine-tuning, audits and
+entries to refresh are free, no card).
 
 ```python
 from cacheverifier import CacheVerifier
 
 cv = CacheVerifier(api_key="cv_...")
 
-query = "how do I cancel my subscription"
-candidate = "Go to Settings > Billing > Pause subscription for a month."  # from your cache
+query = "can I change the delivery address after my parcel ships"
+candidate = "You can change the delivery address any time before the parcel ships."  # from your cache
 
 result = cv.verify(query, candidate)
 if result.approved:
     answer = candidate                       # verified hit — skip the LLM call
 else:
     answer = call_your_llm(query)            # not trustworthy — fall through
-
-# Later, once you know if it was actually right (thumbs-down, reopened ticket, ...):
-cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
+    if result.audit:                         # a randomized audit of an approval: report whether
+        cv.feedback(query, candidate,        # the cached answer matched the fresh one
+                    was_correct=same_answer(answer, candidate),
+                    verification_id=result.verification_id)
+
+# Later, label the cached candidate (not the fresh LLM answer): was it right for this
+# query? (a thumbs-down, a reopened ticket, a reviewer's call)
+cv.feedback(query, candidate, was_correct=candidate_was_right, similarity_score=0.86,
+            verification_id=result.verification_id)
 ```
 
 `verify()` returns a `VerifyResult`:
@@ -60,7 +92,10 @@ cv.feedback(query, answer, was_correct=True, similarity_score=0.86)
 | field | meaning |
 |---|---|
 | `approved` | serve the cached answer (`True`) or fall through (`False`) |
-| `score` / `threshold` | `approved` is `score >= threshold` |
+| `audit` | `True` when an approval was drawn for a randomized audit, so `approved` is `False`: call your LLM, serve its answer, and report whether the cached one matched |
+| `verification_id` | set while auditing is on; pass it to `feedback()` |
+| `score` / `threshold` | with `decision_rule="threshold"`, `approved` is `score >= threshold` |
+| `decision_rule` | `"threshold"`, `"joint"` / `"joint_gap"` (a fitted rule that also uses your `similarity_score`), `"aci"` (online threshold), or `"cold_start"` |
 | `model_version` | `"stock"`, `"v<id>"` (fine-tuned), `"cold_start_fail_closed"` / `"cold_start_auto_pending"`, or `"verify_unavailable"` |
 | `latency_ms` | server-side inference time |
 | `degraded` | `True` when the call failed and this result was synthesized client-side |
@@ -105,23 +140,6 @@ cv = CacheVerifier(api_key="cv_...", fail_open=True)
 There is no formal uptime SLA yet, which is the other reason the fallback path
 is a built-in default rather than left to you.
 
-## GPTCache
-
-Drop the verifier into a GPTCache pipeline as its similarity evaluator — no fork required:
-
-```python
-from gptcache import cache
-from cacheverifier.integrations.gptcache import CacheVerifierEvaluation
-
-evaluator = CacheVerifierEvaluation(api_key="cv_...")
-cache.init(similarity_evaluation=evaluator, ...)
-
-# when you learn a served hit's real outcome:
-evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9)
-```
-
-See [`examples/gptcache_example.py`](examples/gptcache_example.py).
-
 ## LangChain
 
 `VerifiedCache` wraps any LangChain cache, typically a semantic one, and only
@@ -146,6 +164,36 @@ set_llm_cache(
 )
 ```
 
+## GPTCache
+
+Drop the verifier into a GPTCache pipeline as its similarity evaluator — no fork required:
+
+```python
+from gptcache import cache
+from cacheverifier.integrations.gptcache import CacheVerifierEvaluation
+
+evaluator = CacheVerifierEvaluation(api_key="cv_...")
+cache.init(similarity_evaluation=evaluator, ...)
+
+# when you learn a served hit's real outcome:
+evaluator.report_feedback(query, answer, was_correct=False, similarity_score=0.9)
+```
+
+See [`examples/gptcache_example.py`](examples/gptcache_example.py).
+
+## Entries to refresh
+
+Send your cache's own key with feedback, and the service tells you which
+entries to fix: ones reported stale, or wrong more often than right.
+
+```python
+cv.feedback(query, cached_answer, was_correct=False, stale=True, cache_entry_id="faq:shipping")
+
+for e in cv.entries_to_refresh():
+    my_cache.delete(e["cache_entry_id"])   # or regenerate the answer
+    cv.resolve_entry(e["key"])
+```
+
 ## Fine-tuning
 
 Once you have ~20+ feedback rows (the service found fine-tuning is often a net negative
@@ -221,12 +269,14 @@ fully offline. `--emit-summary` writes an aggregate-only JSON file (AUCs, counts
 no query or answer text) that's safe to share for a human read.
 
 **Choosing the base model.** `--base-model` takes the same values as the hosted
-service's `base_model` fine-tune parameter, so local and hosted numbers stay comparable:
+service's `base_model` fine-tune parameter. The defaults differ: the CLI starts from
+`ms_marco`, hosted fine-tunes from `nli`. To compare a local run with a hosted fine-tune,
+pass `--base-model nli`.
 
 | `--base-model` | model | use for |
 |---|---|---|
-| `ms_marco` (default) | `cross-encoder/ms-marco-MiniLM-L6-v2` | English traffic |
-| `nli` | `cross-encoder/nli-MiniLM2-L6-H768` | English; entailment pretraining, stronger on negation / entity swaps |
+| `ms_marco` (CLI default) | `cross-encoder/ms-marco-MiniLM-L6-v2` | English traffic |
+| `nli` (hosted default) | `cross-encoder/nli-MiniLM2-L6-H768` | English; entailment pretraining, stronger on negation / entity swaps |
 | `multilingual` | `BAAI/bge-reranker-base` | Chinese and other non-English traffic (~6 GB RAM to fine-tune) |
 | `multilingual_small` | `cross-encoder/mmarco-mMiniLMv2-L12-H384-v1` | same, smaller (~3 GB RAM) |
 
@@ -256,6 +306,30 @@ verdict
   IMPROVED   -- fine-tuning on your own data helps this traffic
 ```
 
+## Recommended production setup
+
+- Keep randomized audits on (the default for new accounts) and report each audited call
+  with `feedback(..., verification_id=r.verification_id)`, or `result=r` on
+  `LocalVerifier`. That is what makes the measured error rate real.
+- Fine-tune with `cost_ratio=` (how many times worse a wrong reuse is than an extra LLM
+  call) instead of the default threshold. ACI (hosted only) is optional: it lets the
+  threshold follow the audits, but on the one real drifting stream we tested it gave no
+  clear advantage over a fixed threshold.
+- Send `cache_entry_id=` with feedback so `entries_to_refresh()` hands back your keys.
+- At volume, score locally with `LocalVerifier`.
+
+## Known limits
+
+- The hosted verifier is **English-only** today. Don't send Chinese or other non-English
+  traffic to it; multilingual base models are evaluated in the docs but not deployed.
+- Query + answer share a **128-token** input; longer pairs are truncated head and tail.
+- Verify endpoints allow about **20 requests/second per API key**. Use `verify_batch()` or
+  `LocalVerifier` beyond that.
+- No formal uptime SLA yet; `verify()` fails closed by default (see above).
+- The shared stock model (`cross-encoder/ms-marco-MiniLM-L6-v2`) is Apache-2.0 but trained
+  on MS MARCO, whose terms describe it as for non-commercial research. Fine-tunes start from
+  the NLI base (`cross-encoder/nli-MiniLM2-L6-H768`, Apache-2.0) by default.
+
 ## API surface
 
 | method | endpoint |
@@ -267,6 +341,7 @@ verdict
 | `get_finetune_job(id)` / `list_finetune_jobs()` | `GET /v1/finetune/jobs[/id]` |
 | `activate_model_version(id)` | `POST /v1/finetune/model-versions/{id}/activate` |
 | `drift_status()` | `GET /v1/monitor/drift-status` |
+| `entries_to_refresh()` / `resolve_entry(key)` | `GET /v1/monitor/entries-to-refresh` / `POST .../{key}/resolve` |
 | `gray_zone_threshold()` | `GET /v1/monitor/gray-zone-threshold` |
 | `usage()` / `savings()` | `GET /v1/usage/status` / `/savings` |
 | `LocalVerifier.verify(...)` / `verify_batch(...)` | local; decisions to `POST /v1/verify/decisions` |

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/cacheverifier/__init__.py

@@ -19,7 +19,7 @@ Research behind it: https://github.com/imxinchengyou/CacheVerifier
 from cacheverifier.client import CacheVerifier, CacheVerifierError, ShadowVerdict, VerifyResult
 from cacheverifier.local import LocalVerifier, ModelCheckFailed
 
-__version__ = "0.6.0"
+__version__ = "0.7.1"
 __all__ = [
     "CacheVerifier",
     "CacheVerifierError",

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/cacheverifier/client.py

@@ -152,7 +152,7 @@ class CacheVerifier:
         from cacheverifier import CacheVerifier
 
         cv = CacheVerifier(api_key="cv_...")
-        result = cv.verify("how do I cancel", "Go to Settings > Billing > Pause.")
+        result = cv.verify("can I change my flight date", "You can change your seat until check-in closes.")
         if result.approved:
             ...  # serve the cached answer
         else:
@@ -312,6 +312,7 @@ class CacheVerifier:
         similarity_score: float | None = None,
         cached_query: str | None = None,
         verification_id: int | None = None,
+        cache_entry_id: str | None = None,
         stale: bool = False,
         idempotency_key: str | None = None,
     ) -> int:
@@ -328,6 +329,8 @@ class CacheVerifier:
         - `cached_query`: the cached entry's own original query, if you have it.
         - `verification_id`: from the `VerifyResult` this feedback is about
           (always for an `audit` result) -- it ties the label to that verdict.
+        - `cache_entry_id`: your own key for the cache entry the answer came
+          from; `entries_to_refresh()` hands it back.
         - `stale`: set instead of a bare `was_correct=False` when the answer
           is wrong ONLY because a fact changed (price, date, ...), not a
           semantic mismatch. Stale rows are excluded from training and drift.
@@ -340,7 +343,12 @@ class CacheVerifier:
             "stale": stale,
         }
         payload.update(
-            _drop_none(similarity_score=similarity_score, cached_query=cached_query, verification_id=verification_id)
+            _drop_none(
+                similarity_score=similarity_score,
+                cached_query=cached_query,
+                verification_id=verification_id,
+                cache_entry_id=cache_entry_id,
+            )
         )
         headers = {"Idempotency-Key": idempotency_key} if idempotency_key else None
         return int(self._post("/v1/feedback", json=payload, headers=headers)["id"])
@@ -348,7 +356,8 @@ class CacheVerifier:
     def feedback_batch(self, items: Iterable[dict[str, Any]]) -> list[int]:
         """Bulk-upload feedback rows (max 500). Each item is a dict with
         `query`, `candidate_answer`, `was_correct`, and optionally
-        `similarity_score` / `stale`. Returns the created row ids.
+        `similarity_score` / `cached_query` / `cache_entry_id` / `stale`.
+        Returns the created row ids.
         `POST /v1/feedback/batch`.
         """
         data = self._post("/v1/feedback/batch", json={"items": list(items)})
@@ -412,6 +421,21 @@ class CacheVerifier:
         """
         return self._get("/v1/monitor/gray-zone-threshold")
 
+    def entries_to_refresh(self) -> list[dict[str, Any]]:
+        """Cache entries your feedback reported stale, or wrong more often
+        than right -- refresh or drop them, then `resolve_entry(e["key"])`.
+        Each has `key`, `cache_entry_id` (if you sent one), `reason`
+        (`"stale"` / `"often_wrong"`), the answer, report counts and sample
+        queries. `GET /v1/monitor/entries-to-refresh`.
+        """
+        return list(self._get("/v1/monitor/entries-to-refresh")["entries"])
+
+    def resolve_entry(self, key: str) -> dict[str, Any]:
+        """Mark an entry refreshed: earlier feedback stops counting toward it.
+        `POST /v1/monitor/entries-to-refresh/{key}/resolve`.
+        """
+        return self._post(f"/v1/monitor/entries-to-refresh/{key}/resolve")
+
     def usage(self) -> dict[str, Any]:
         """`GET /v1/usage/status`."""
         return self._get("/v1/usage/status")

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/cacheverifier/local.py

@@ -475,6 +475,7 @@ class LocalVerifier:
         result: VerifyResult | None = None,
         similarity_score: float | None = None,
         cached_query: str | None = None,
+        cache_entry_id: str | None = None,
         stale: bool = False,
         idempotency_key: str | None = None,
     ) -> int:
@@ -488,6 +489,7 @@ class LocalVerifier:
             similarity_score=similarity_score,
             cached_query=cached_query,
             verification_id=verification_id,
+            cache_entry_id=cache_entry_id,
             stale=stale,
             idempotency_key=idempotency_key,
         )

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/examples/gptcache_example.py

@@ -31,16 +31,16 @@ cache.init(
     similarity_evaluation=evaluator,
 )
 
-put("how do I cancel my subscription?", "Go to Settings > Billing > Cancel subscription.")
+put("how do I change my flight date?", "Open Manage booking, pick a new date, and pay any fare difference.")
 
 # Close-but-different intent. Cosine similarity would likely reuse the
 # cached answer; the verifier call decides whether that is actually safe.
-hit = get("how do I pause my subscription for a month?")
+hit = get("how do I change my seat on my flight?")
 print("cache returned:", hit)
 
 # When you later learn whether a served hit was right:
 evaluator.report_feedback(
-    "how do I pause my subscription for a month?",
+    "how do I change my seat on my flight?",
     hit or "",
     was_correct=False,
 )

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/examples/quickstart.py

@@ -16,17 +16,17 @@ from cacheverifier import CacheVerifier
 def call_your_llm(query: str) -> str:
     """Stand-in for your own LLM call. CacheVerifier doesn't do this part --
     your cache backend already has one."""
-    return "Go to Settings > Billing > Cancel subscription."
+    return "Once a parcel has shipped, contact the carrier to redirect it; the address can't be edited online."
 
 
 def main() -> None:
     api_key = os.environ["CACHEVERIFIER_API_KEY"]
 
     # A gray-zone example: your cache's similarity search found a candidate
-    # close enough to look like a match, but "cancel" vs "pause" is a real
-    # difference a plain similarity threshold can't tell apart.
-    query = "how do I cancel my subscription"
-    candidate = "Go to Settings > Billing > Pause subscription for a month."
+    # close enough to look like a match, but "after" vs "before" it ships is a
+    # real difference a plain similarity threshold can't tell apart.
+    query = "can I change the delivery address after my parcel ships"
+    candidate = "You can change the delivery address any time before the parcel ships."
 
     # verify() sits on your request path: it uses a 1s timeout and, if the
     # service is unreachable, returns a degraded result (approved=False here)

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "cacheverifier"
-version = "0.6.0"
+version = "0.7.1"
 description = "Python client for the CacheVerifier semantic-cache verification API, hosted or scored locally"
 readme = "README.md"
 requires-python = ">=3.9"

{cacheverifier-0.6.0 → cacheverifier-0.7.1}/tests/test_client.py

@@ -133,6 +133,7 @@ def test_feedback_returns_id_and_forwards_idempotency_key():
 def test_feedback_omits_optional_fields_when_unset():
     def handler(request: httpx.Request) -> httpx.Response:
         assert "similarity_score" not in json.loads(request.content)
+        assert "cache_entry_id" not in json.loads(request.content)
         assert request.headers.get("Idempotency-Key") is None
         return httpx.Response(200, json={"id": 1})
 
@@ -140,6 +141,29 @@ def test_feedback_omits_optional_fields_when_unset():
         assert cv.feedback("q", "a", was_correct=True) == 1
 
 
+def test_entries_to_refresh_round_trip():
+    seen = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        seen.append((request.method, request.url.path, request.content))
+        if request.url.path == "/v1/feedback":
+            return httpx.Response(200, json={"id": 7})
+        if request.method == "GET":
+            return httpx.Response(200, json={"entries": [{"key": "k" * 32, "cache_entry_id": "faq:shipping", "reason": "stale"}]})
+        return httpx.Response(200, json={"key": "k" * 32, "resolved_at": "2026-10-31T00:00:00Z"})
+
+    with make_client(handler) as cv:
+        cv.feedback("q", "a", was_correct=False, stale=True, cache_entry_id="faq:shipping")
+        [entry] = cv.entries_to_refresh()
+        assert entry["cache_entry_id"] == "faq:shipping"
+        assert cv.resolve_entry(entry["key"])["key"] == entry["key"]
+    assert json.loads(seen[0][2])["cache_entry_id"] == "faq:shipping"
+    assert [s[:2] for s in seen[1:]] == [
+        ("GET", "/v1/monitor/entries-to-refresh"),
+        ("POST", f"/v1/monitor/entries-to-refresh/{'k' * 32}/resolve"),
+    ]
+
+
 def test_finetune_passes_query_params():
     def handler(request: httpx.Request) -> httpx.Response:
         assert request.url.path == "/v1/finetune/jobs"