pytest-jev 0.1.0__tar.gz

pytest_jev-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,24 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+        with:
+          python-version: ${{ matrix.python }}
+      - run: uv sync --locked
+      - run: uv run ruff check .
+      - run: uv run ruff format --check .
+      # Offline: a fake Jev answers every question, so no API key is needed.
+      - run: uv run pytest

pytest_jev-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,20 @@
+name: Publish to PyPI
+
+# Runs when a GitHub release is published. PyPI trusts this workflow directly
+# (trusted publishing), so the repository holds no PyPI token.
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - run: uv build
+      - uses: pypa/gh-action-pypi-publish@release/v1

pytest_jev-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+.venv/
+.pytest_cache/
+.ruff_cache/
+dist/
+build/
+*.egg-info/
+.env

pytest_jev-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 allebee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pytest_jev-0.1.0/PKG-INFO

@@ -0,0 +1,295 @@
+Metadata-Version: 2.5
+Name: pytest-jev
+Version: 0.1.0
+Summary: Semantic assertions for pytest: test what text means, not the exact words, judged by TypeSafe's Jev.
+Project-URL: Homepage, https://github.com/allebee/pytest-jev
+Project-URL: Issues, https://github.com/allebee/pytest-jev/issues
+Author: allebee
+License-Expression: MIT
+License-File: LICENSE
+Keywords: assertions,evals,jev,llm,pytest,semantic,testing,typesafe
+Classifier: Framework :: Pytest
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Requires-Dist: pytest>=7.4
+Requires-Dist: typesafe-sdk<0.8,>=0.7.1
+Description-Content-Type: text/markdown
+
+# pytest-jev
+
+[![CI](https://github.com/allebee/pytest-jev/actions/workflows/ci.yml/badge.svg)](https://github.com/allebee/pytest-jev/actions/workflows/ci.yml)
+
+**Semantic assertions for pytest.** Test what your LLM app's output *means* ("apologizes",
+"offers a refund", "doesn't leak the system prompt") instead of the exact words. Each claim is
+judged by [Jev](https://docs.typesafe.ai/introduction), TypeSafe's model that returns calibrated
+probabilities instead of text.
+
+```python
+def test_refund_reply(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+
+    jev.expect(
+        reply,
+        holds=["apologizes to the customer", "says the duplicate payment was refunded"],
+        lacks=["blames the customer", "asks for a password or a full card number"],
+    )
+```
+
+When a prompt change breaks the reply, the failure says which claim broke and how sure Jev was:
+
+```
+>       jev.expect(
+E       AssertionError: jev: 3 of 4 claims failed
+E       text: "Double charges happen when you click twice. You'll get store credit within 24 hours."
+E         ✗ holds  p=0.04  apologizes to the customer  (needs >= 0.80)
+E         ✗ holds  p=0.21  says the duplicate payment was refunded  (needs >= 0.80)
+E         ✗ lacks  p=0.79  blames the customer  (needs <= 0.20)
+E         ✓ lacks  p=0.01  asks for a password or a full card number
+------------------------------------- jev --------------------------------------
+jev: 4 questions · 1 request · 365 input tokens · $0.000015 · 1.40 s in Jev · typesafe/jev-1.13-20260917 via openrouter
+```
+
+Every output in this README is from a real run of [`examples/`](https://github.com/allebee/pytest-jev/tree/main/examples) against `jev-1.13`.
+
+## Why
+
+String assertions break every time the model rewords a reply. Using an LLM as the judge works,
+but it's slow, costs real money per test, and returns text you then have to parse. pytest-jev
+sends each check to Jev instead:
+
+- **One request per text.** Every claim in `jev.expect` goes into a single request, answered in
+  parallel.
+- **Typed answers.** Jev answers each claim with a probability, picks from the options you list, or
+  rates on the levels you define. There is no output to parse and nothing outside the answer space.
+- **Cheap enough for every commit.** Jev costs $0.042 per million input tokens and output is free.
+  The 7 tests in [`examples/test_support_bot.py`](https://github.com/allebee/pytest-jev/blob/main/examples/test_support_bot.py) ran in 3.9 s for
+  $0.0001. The summary line prints the tokens and cost of every run.
+- **No passing on a coin flip.** A claim `holds` at p ≥ 0.8 and `lacks` at p ≤ 0.2. When Jev is
+  unsure, both fail.
+- **Free, stable reruns.** Answers are cached in `.pytest_cache`, so rerunning unchanged tests
+  makes no requests and gives the same verdicts.
+
+## Install
+
+```bash
+pip install pytest-jev
+```
+
+It needs Python 3.10+ and pytest 7.4+.
+
+Set one API key:
+
+```bash
+export TYPESAFE_API_KEY=...     # https://console.typesafe.ai (early access)
+export OPENROUTER_API_KEY=...   # or https://openrouter.ai/settings/keys (no waitlist)
+```
+
+Without a key, tests that use `jev` are skipped (see [CI](#ci-and-running-without-a-key)).
+
+## Usage
+
+The `jev` fixture has five methods. Each sends one request and returns a result that works in a
+plain `assert`.
+
+### `holds` and `lacks`: one claim
+
+```python
+def test_reply_confirms_the_refund(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+    assert jev.holds(reply, "says the duplicate payment was refunded")
+    assert jev.lacks(reply, "asks for a password or a full card number")
+```
+
+With the broken reply from above:
+
+```
+E       assert <jev holds 'says the duplicate payment was refunded': p=0.16, needs >= 0.80>
+```
+
+The result also carries the probability: `jev.holds(reply, "...").p`.
+
+### `expect`: many claims, one request
+
+```python
+jev.expect(reply, holds=["apologizes", "offers a refund"], lacks=["blames the customer"])
+```
+
+It fails with a report of every claim, as shown at the top. It returns the claims when they pass.
+
+### `context`: check the text against something else
+
+Extra state goes in `context`, such as a policy or the documents a RAG app retrieved. A claim can
+name it in backticks:
+
+```python
+def test_reply_matches_the_policy(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+    assert jev.lacks(reply, "contradicts the policy in `policy`", context={"policy": REFUND_POLICY})
+```
+
+The broken reply promises store credit in 24 hours; the policy says refunds to the card in 5
+business days:
+
+```
+E       assert <jev lacks 'contradicts the policy in `policy`': p=0.95, needs <= 0.20>
+```
+
+The text under test is always `text`, so `context` can't use that key.
+
+### `choice`: which option fits
+
+```python
+TEAMS = {
+    "billing": "Payments, charges, invoices and refunds",
+    "technical": "Bugs, errors, crashes and integrations",
+    "account": "Logins, passwords and account settings",
+    "other": "Anything that fits none of the teams above",
+}
+
+
+def test_checkout_errors_go_to_billing(jev):
+    ticket = "Your checkout page throws a 500 error when I enter my card."
+    assert jev.choice(ticket, "Which team should handle this ticket?", TEAMS) == "billing"
+```
+
+```
+E       AssertionError: assert jev chose 'technical', not 'billing'
+E         question: Which team should handle this ticket?
+E           technical  0.90  ██████████████████░░
+E           billing    0.10  ██░░░░░░░░░░░░░░░░░░
+E           account    0.00  ░░░░░░░░░░░░░░░░░░░░
+E           other      0.00  ░░░░░░░░░░░░░░░░░░░░
+E         confidence 0.87
+```
+
+Sometimes the failure means the test's expectation needs another look: a 500 error at checkout is
+arguably a bug first.
+
+Options can be a dict of label to description, or a plain list of labels. Comparing with a label
+that isn't an option (`team == "biling"`) raises an error instead of quietly failing.
+
+### `score`: rate on ordered levels
+
+```python
+POLITENESS = {
+    "rude": "Rude, dismissive or blaming the customer",
+    "neutral": "Neutral and matter-of-fact, no warmth",
+    "warm": "Warm and polite, acknowledges the customer's frustration",
+}
+
+
+def test_reply_is_warm(jev):
+    tone = jev.score(reply, "How polite is this support reply?", POLITENESS)
+    assert tone >= "warm"
+```
+
+Levels go lowest first. The comparison is probabilistic: `tone >= "warm"` passes when Jev puts at
+least 80% of its probability on "warm" or higher. `>`, `<=`, `<`, `==` and `!=` work the same way,
+with labels or level indices. The broken reply passes `tone >= "neutral"` (0.83) but not this:
+
+```
+E       AssertionError: assert jev gave P(level >= 'warm') = 0.00, needs >= 0.80
+E         question: How polite is this support reply?
+E           0 rude     0.17  ███░░░░░░░░░░░░░░░░░
+E           1 neutral  0.83  █████████████████░░░
+E           2 warm     0.00  ░░░░░░░░░░░░░░░░░░░░
+E         expected level 0.84, confidence 0.75
+```
+
+## Thresholds and models
+
+The threshold defaults to 0.8: `holds` needs p ≥ 0.8, `lacks` needs p ≤ 0.2. It must be between 0.5
+and 1. Set it per call, per test, or for the whole run:
+
+```python
+assert jev.holds(reply, "offers a refund", threshold=0.9)
+
+
+@pytest.mark.jev(threshold=0.9, model="jev-1.13")
+def test_strict(jev): ...
+```
+
+```ini
+# pytest.ini (or [tool.pytest.ini_options] in pyproject.toml)
+[pytest]
+jev_model = jev-1.13
+jev_threshold = 0.85
+```
+
+`jev-latest` changes when TypeSafe ships a new version, so pin `jev-1.13` when runs must be
+reproducible.
+
+| Option | ini | Default | |
+|---|---|---|---|
+| `--jev-model` | `jev_model` | `jev-latest` | Jev model to ask |
+| `--jev-threshold` | `jev_threshold` | `0.8` | Claim threshold |
+| `--jev-provider` | `jev_provider` | `auto` | `typesafe`, `openrouter`, or `auto` (OpenRouter if its key is set) |
+| `--jev-no-cache` | | off | Ask again instead of reusing cached answers |
+| `--jev-require` | `jev_require` | off | Fail instead of skip when no key is set |
+
+## CI and running without a key
+
+- Tests that use `jev` get the `jev` marker automatically. `pytest -m "not jev"` runs everything
+  else offline.
+- Without a key, `jev` tests are **skipped** and say why. In CI, pass `--jev-require` (or set
+  `jev_require = true`) so a missing secret fails the build instead.
+- Answers are cached by model, text, context and question. Pass `--jev-no-cache` to ask again.
+
+## Use another backend
+
+Requests go through the session-scoped `jev_client` fixture. Override it in `conftest.py` with
+anything that has the TypeSafe SDK's `system_one(state=, questions=, model=)` method, such as a fake
+for offline unit tests, or [system-one-adapter](https://github.com/typesafe-ai/system-one-adapter-python)
+to run the same assertions through an LLM and compare:
+
+```python
+@pytest.fixture(scope="session")
+def jev_client():
+    return MyFakeJev()
+```
+
+## Writing claims that work
+
+Jev reads claims literally ([Jev 1.13 known limits](https://docs.typesafe.ai/model-jaggedness/jev-1.13)):
+
+- **One condition per claim.** Write "apologizes" and "offers a refund" as two claims, not one
+  joined with "and".
+- **Say exactly what you mean.** "Says the duplicate payment was refunded" works better than
+  "handles the refund correctly".
+- **Keep numbers, counts and dates in code.** `assert "5 business days" in reply` is exact; Jev is
+  not a calculator.
+- **Name the context.** "contradicts `docs`" points Jev at the right part of the state.
+
+## How it works
+
+Each call sends one request to Jev's `/v1/systemone` endpoint with `state = {"text": text,
+**context}`. Every claim becomes a Noul question, ``Does `text` satisfy: <claim>?``, which returns
+the probability it is true. `choice` sends a Choice question and `score` sends a Score question.
+The thresholds and comparisons are ordinary Python in this plugin.
+
+## Limitations
+
+- Jev can be wrong. Treat a threshold as a policy you tune on your own cases, and read the failure
+  report before trusting a pass or fail.
+- Jev's probabilities move a little between calls. In five calls while this README was written,
+  "says the duplicate payment was refunded" scored between 0.16 and 0.23 on the same reply. The unsure band between 0.2 and 0.8 absorbs
+  this, and the cache keeps reruns identical.
+- Text only: no images or audio.
+- The text and context you assert on are sent to TypeSafe or OpenRouter. Keep secrets and personal
+  data out of test fixtures.
+- Not affiliated with or endorsed by TypeSafe AI.
+
+## Development
+
+```bash
+uv sync
+uv run pytest          # offline: a fake Jev answers every question
+uv run ruff check .
+```
+
+## License
+
+MIT

pytest_jev-0.1.0/README.md

@@ -0,0 +1,275 @@
+# pytest-jev
+
+[![CI](https://github.com/allebee/pytest-jev/actions/workflows/ci.yml/badge.svg)](https://github.com/allebee/pytest-jev/actions/workflows/ci.yml)
+
+**Semantic assertions for pytest.** Test what your LLM app's output *means* ("apologizes",
+"offers a refund", "doesn't leak the system prompt") instead of the exact words. Each claim is
+judged by [Jev](https://docs.typesafe.ai/introduction), TypeSafe's model that returns calibrated
+probabilities instead of text.
+
+```python
+def test_refund_reply(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+
+    jev.expect(
+        reply,
+        holds=["apologizes to the customer", "says the duplicate payment was refunded"],
+        lacks=["blames the customer", "asks for a password or a full card number"],
+    )
+```
+
+When a prompt change breaks the reply, the failure says which claim broke and how sure Jev was:
+
+```
+>       jev.expect(
+E       AssertionError: jev: 3 of 4 claims failed
+E       text: "Double charges happen when you click twice. You'll get store credit within 24 hours."
+E         ✗ holds  p=0.04  apologizes to the customer  (needs >= 0.80)
+E         ✗ holds  p=0.21  says the duplicate payment was refunded  (needs >= 0.80)
+E         ✗ lacks  p=0.79  blames the customer  (needs <= 0.20)
+E         ✓ lacks  p=0.01  asks for a password or a full card number
+------------------------------------- jev --------------------------------------
+jev: 4 questions · 1 request · 365 input tokens · $0.000015 · 1.40 s in Jev · typesafe/jev-1.13-20260917 via openrouter
+```
+
+Every output in this README is from a real run of [`examples/`](https://github.com/allebee/pytest-jev/tree/main/examples) against `jev-1.13`.
+
+## Why
+
+String assertions break every time the model rewords a reply. Using an LLM as the judge works,
+but it's slow, costs real money per test, and returns text you then have to parse. pytest-jev
+sends each check to Jev instead:
+
+- **One request per text.** Every claim in `jev.expect` goes into a single request, answered in
+  parallel.
+- **Typed answers.** Jev answers each claim with a probability, picks from the options you list, or
+  rates on the levels you define. There is no output to parse and nothing outside the answer space.
+- **Cheap enough for every commit.** Jev costs $0.042 per million input tokens and output is free.
+  The 7 tests in [`examples/test_support_bot.py`](https://github.com/allebee/pytest-jev/blob/main/examples/test_support_bot.py) ran in 3.9 s for
+  $0.0001. The summary line prints the tokens and cost of every run.
+- **No passing on a coin flip.** A claim `holds` at p ≥ 0.8 and `lacks` at p ≤ 0.2. When Jev is
+  unsure, both fail.
+- **Free, stable reruns.** Answers are cached in `.pytest_cache`, so rerunning unchanged tests
+  makes no requests and gives the same verdicts.
+
+## Install
+
+```bash
+pip install pytest-jev
+```
+
+It needs Python 3.10+ and pytest 7.4+.
+
+Set one API key:
+
+```bash
+export TYPESAFE_API_KEY=...     # https://console.typesafe.ai (early access)
+export OPENROUTER_API_KEY=...   # or https://openrouter.ai/settings/keys (no waitlist)
+```
+
+Without a key, tests that use `jev` are skipped (see [CI](#ci-and-running-without-a-key)).
+
+## Usage
+
+The `jev` fixture has five methods. Each sends one request and returns a result that works in a
+plain `assert`.
+
+### `holds` and `lacks`: one claim
+
+```python
+def test_reply_confirms_the_refund(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+    assert jev.holds(reply, "says the duplicate payment was refunded")
+    assert jev.lacks(reply, "asks for a password or a full card number")
+```
+
+With the broken reply from above:
+
+```
+E       assert <jev holds 'says the duplicate payment was refunded': p=0.16, needs >= 0.80>
+```
+
+The result also carries the probability: `jev.holds(reply, "...").p`.
+
+### `expect`: many claims, one request
+
+```python
+jev.expect(reply, holds=["apologizes", "offers a refund"], lacks=["blames the customer"])
+```
+
+It fails with a report of every claim, as shown at the top. It returns the claims when they pass.
+
+### `context`: check the text against something else
+
+Extra state goes in `context`, such as a policy or the documents a RAG app retrieved. A claim can
+name it in backticks:
+
+```python
+def test_reply_matches_the_policy(jev):
+    reply = support_bot("I was charged twice for order #1042.")
+    assert jev.lacks(reply, "contradicts the policy in `policy`", context={"policy": REFUND_POLICY})
+```
+
+The broken reply promises store credit in 24 hours; the policy says refunds to the card in 5
+business days:
+
+```
+E       assert <jev lacks 'contradicts the policy in `policy`': p=0.95, needs <= 0.20>
+```
+
+The text under test is always `text`, so `context` can't use that key.
+
+### `choice`: which option fits
+
+```python
+TEAMS = {
+    "billing": "Payments, charges, invoices and refunds",
+    "technical": "Bugs, errors, crashes and integrations",
+    "account": "Logins, passwords and account settings",
+    "other": "Anything that fits none of the teams above",
+}
+
+
+def test_checkout_errors_go_to_billing(jev):
+    ticket = "Your checkout page throws a 500 error when I enter my card."
+    assert jev.choice(ticket, "Which team should handle this ticket?", TEAMS) == "billing"
+```
+
+```
+E       AssertionError: assert jev chose 'technical', not 'billing'
+E         question: Which team should handle this ticket?
+E           technical  0.90  ██████████████████░░
+E           billing    0.10  ██░░░░░░░░░░░░░░░░░░
+E           account    0.00  ░░░░░░░░░░░░░░░░░░░░
+E           other      0.00  ░░░░░░░░░░░░░░░░░░░░
+E         confidence 0.87
+```
+
+Sometimes the failure means the test's expectation needs another look: a 500 error at checkout is
+arguably a bug first.
+
+Options can be a dict of label to description, or a plain list of labels. Comparing with a label
+that isn't an option (`team == "biling"`) raises an error instead of quietly failing.
+
+### `score`: rate on ordered levels
+
+```python
+POLITENESS = {
+    "rude": "Rude, dismissive or blaming the customer",
+    "neutral": "Neutral and matter-of-fact, no warmth",
+    "warm": "Warm and polite, acknowledges the customer's frustration",
+}
+
+
+def test_reply_is_warm(jev):
+    tone = jev.score(reply, "How polite is this support reply?", POLITENESS)
+    assert tone >= "warm"
+```
+
+Levels go lowest first. The comparison is probabilistic: `tone >= "warm"` passes when Jev puts at
+least 80% of its probability on "warm" or higher. `>`, `<=`, `<`, `==` and `!=` work the same way,
+with labels or level indices. The broken reply passes `tone >= "neutral"` (0.83) but not this:
+
+```
+E       AssertionError: assert jev gave P(level >= 'warm') = 0.00, needs >= 0.80
+E         question: How polite is this support reply?
+E           0 rude     0.17  ███░░░░░░░░░░░░░░░░░
+E           1 neutral  0.83  █████████████████░░░
+E           2 warm     0.00  ░░░░░░░░░░░░░░░░░░░░
+E         expected level 0.84, confidence 0.75
+```
+
+## Thresholds and models
+
+The threshold defaults to 0.8: `holds` needs p ≥ 0.8, `lacks` needs p ≤ 0.2. It must be between 0.5
+and 1. Set it per call, per test, or for the whole run:
+
+```python
+assert jev.holds(reply, "offers a refund", threshold=0.9)
+
+
+@pytest.mark.jev(threshold=0.9, model="jev-1.13")
+def test_strict(jev): ...
+```
+
+```ini
+# pytest.ini (or [tool.pytest.ini_options] in pyproject.toml)
+[pytest]
+jev_model = jev-1.13
+jev_threshold = 0.85
+```
+
+`jev-latest` changes when TypeSafe ships a new version, so pin `jev-1.13` when runs must be
+reproducible.
+
+| Option | ini | Default | |
+|---|---|---|---|
+| `--jev-model` | `jev_model` | `jev-latest` | Jev model to ask |
+| `--jev-threshold` | `jev_threshold` | `0.8` | Claim threshold |
+| `--jev-provider` | `jev_provider` | `auto` | `typesafe`, `openrouter`, or `auto` (OpenRouter if its key is set) |
+| `--jev-no-cache` | | off | Ask again instead of reusing cached answers |
+| `--jev-require` | `jev_require` | off | Fail instead of skip when no key is set |
+
+## CI and running without a key
+
+- Tests that use `jev` get the `jev` marker automatically. `pytest -m "not jev"` runs everything
+  else offline.
+- Without a key, `jev` tests are **skipped** and say why. In CI, pass `--jev-require` (or set
+  `jev_require = true`) so a missing secret fails the build instead.
+- Answers are cached by model, text, context and question. Pass `--jev-no-cache` to ask again.
+
+## Use another backend
+
+Requests go through the session-scoped `jev_client` fixture. Override it in `conftest.py` with
+anything that has the TypeSafe SDK's `system_one(state=, questions=, model=)` method, such as a fake
+for offline unit tests, or [system-one-adapter](https://github.com/typesafe-ai/system-one-adapter-python)
+to run the same assertions through an LLM and compare:
+
+```python
+@pytest.fixture(scope="session")
+def jev_client():
+    return MyFakeJev()
+```
+
+## Writing claims that work
+
+Jev reads claims literally ([Jev 1.13 known limits](https://docs.typesafe.ai/model-jaggedness/jev-1.13)):
+
+- **One condition per claim.** Write "apologizes" and "offers a refund" as two claims, not one
+  joined with "and".
+- **Say exactly what you mean.** "Says the duplicate payment was refunded" works better than
+  "handles the refund correctly".
+- **Keep numbers, counts and dates in code.** `assert "5 business days" in reply` is exact; Jev is
+  not a calculator.
+- **Name the context.** "contradicts `docs`" points Jev at the right part of the state.
+
+## How it works
+
+Each call sends one request to Jev's `/v1/systemone` endpoint with `state = {"text": text,
+**context}`. Every claim becomes a Noul question, ``Does `text` satisfy: <claim>?``, which returns
+the probability it is true. `choice` sends a Choice question and `score` sends a Score question.
+The thresholds and comparisons are ordinary Python in this plugin.
+
+## Limitations
+
+- Jev can be wrong. Treat a threshold as a policy you tune on your own cases, and read the failure
+  report before trusting a pass or fail.
+- Jev's probabilities move a little between calls. In five calls while this README was written,
+  "says the duplicate payment was refunded" scored between 0.16 and 0.23 on the same reply. The unsure band between 0.2 and 0.8 absorbs
+  this, and the cache keeps reruns identical.
+- Text only: no images or audio.
+- The text and context you assert on are sent to TypeSafe or OpenRouter. Keep secrets and personal
+  data out of test fixtures.
+- Not affiliated with or endorsed by TypeSafe AI.
+
+## Development
+
+```bash
+uv sync
+uv run pytest          # offline: a fake Jev answers every question
+uv run ruff check .
+```
+
+## License
+
+MIT