llm-output-guard 1.4.1 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/README.md +123 -141
  2. package/dist/{adapter-options-CQNPNjES.d.cts → adapter-options-B-fgYd--.d.cts} +35 -1
  3. package/dist/{adapter-options-BUOzVp3T.d.ts → adapter-options-DNkMNS92.d.ts} +35 -1
  4. package/dist/ai-sdk.cjs +213 -10
  5. package/dist/ai-sdk.cjs.map +1 -1
  6. package/dist/ai-sdk.d.cts +22 -5
  7. package/dist/ai-sdk.d.ts +22 -5
  8. package/dist/ai-sdk.js +28 -9
  9. package/dist/ai-sdk.js.map +1 -1
  10. package/dist/anthropic.cjs +232 -12
  11. package/dist/anthropic.cjs.map +1 -1
  12. package/dist/anthropic.d.cts +3 -3
  13. package/dist/anthropic.d.ts +3 -3
  14. package/dist/anthropic.js +17 -3
  15. package/dist/anthropic.js.map +1 -1
  16. package/dist/bin.cjs +684 -2
  17. package/dist/bin.cjs.map +1 -1
  18. package/dist/bin.js +684 -2
  19. package/dist/bin.js.map +1 -1
  20. package/dist/{chunk-R6QMUCVR.js → chunk-D3HQLFND.js} +112 -6
  21. package/dist/chunk-D3HQLFND.js.map +1 -0
  22. package/dist/{chunk-NHVIUA4Z.js → chunk-FCD42SUY.js} +28 -13
  23. package/dist/chunk-FCD42SUY.js.map +1 -0
  24. package/dist/chunk-WVML434P.js +105 -0
  25. package/dist/chunk-WVML434P.js.map +1 -0
  26. package/dist/index.cjs +111 -3
  27. package/dist/index.cjs.map +1 -1
  28. package/dist/index.d.cts +81 -3
  29. package/dist/index.d.ts +81 -3
  30. package/dist/index.js +1 -1
  31. package/dist/openai.cjs +241 -13
  32. package/dist/openai.cjs.map +1 -1
  33. package/dist/openai.d.cts +3 -3
  34. package/dist/openai.d.ts +3 -3
  35. package/dist/openai.js +26 -4
  36. package/dist/openai.js.map +1 -1
  37. package/dist/{stream-NO-TTGYs.d.cts → stream-BYy6vuGy.d.cts} +64 -3
  38. package/dist/{stream-NO-TTGYs.d.ts → stream-BYy6vuGy.d.ts} +64 -3
  39. package/package.json +3 -2
  40. package/dist/chunk-NHVIUA4Z.js.map +0 -1
  41. package/dist/chunk-OHF5SN7W.js +0 -17
  42. package/dist/chunk-OHF5SN7W.js.map +0 -1
  43. package/dist/chunk-R6QMUCVR.js.map +0 -1
package/README.md CHANGED
@@ -7,12 +7,26 @@
7
7
 
8
8
  **Detect LLM responses that failed while returning `200 OK`.**
9
9
 
10
+ <!--
11
+ Two rows on purpose: what you are installing, then whether it is looked after.
12
+
13
+ Every GitHub-backed badge carries an explicit cacheSeconds. Shields defaults
14
+ these to max-age=120, which has GitHub's camo proxy refetch 720 times a day
15
+ and gives 720 daily chances to catch the upstream API rate limited and cache
16
+ the error for the whole TTL. That is not hypothetical: it happened to the
17
+ downloads badge, which is why it carries one too.
18
+ -->
10
19
  [![npm](https://img.shields.io/npm/v/llm-output-guard?color=0b7285)](https://www.npmjs.com/package/llm-output-guard)
11
20
  [![downloads](https://img.shields.io/npm/dm/llm-output-guard?color=0b7285&cacheSeconds=86400)](https://www.npmjs.com/package/llm-output-guard)
12
21
  [![minzipped](https://img.shields.io/bundlejs/size/llm-output-guard?color=0b7285&label=min%2Bgzip)](https://bundlejs.com/?q=llm-output-guard)
13
22
  [![dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)](https://github.com/edwinsatya/llm-output-guard/blob/main/package.json)
23
+ [![types](https://img.shields.io/npm/types/llm-output-guard?color=0b7285&cacheSeconds=86400)](https://www.npmjs.com/package/llm-output-guard)
24
+ [![node](https://img.shields.io/node/v/llm-output-guard?color=0b7285&cacheSeconds=86400)](https://nodejs.org)
25
+
14
26
  [![CI](https://github.com/edwinsatya/llm-output-guard/actions/workflows/ci.yml/badge.svg)](https://github.com/edwinsatya/llm-output-guard/actions/workflows/ci.yml)
15
- [![license](https://img.shields.io/npm/l/llm-output-guard?color=0b7285)](./LICENSE)
27
+ [![last commit](https://img.shields.io/github/last-commit/edwinsatya/llm-output-guard?color=0b7285&cacheSeconds=21600)](https://github.com/edwinsatya/llm-output-guard/commits/main)
28
+ [![commit activity](https://img.shields.io/github/commit-activity/m/edwinsatya/llm-output-guard?color=0b7285&cacheSeconds=86400)](https://github.com/edwinsatya/llm-output-guard/graphs/commit-activity)
29
+ [![license](https://img.shields.io/npm/l/llm-output-guard?color=0b7285&cacheSeconds=86400)](./LICENSE)
16
30
 
17
31
  Your retry layer watches for `429`, `5xx` and timeouts. It cannot see a model that
18
32
  looped until `max_tokens`, returned `{}`, stopped mid-sentence, or answered in the
@@ -56,12 +70,16 @@ every detector, running on your own pasted output. No API key, no request.
56
70
  | `LOW_ENTROPY` | Character-level collapse, token artifacts | Compression ratio |
57
71
  | `TRUNCATED` | Cut off mid-thought | `finish_reason`, unbalanced fences |
58
72
  | `INVALID_JSON` | Prose around the payload, wrong types | Parse + key + schema contract |
59
- | `SCRIPT_MISMATCH` | Answered in the wrong alphabet | Share of letters outside the expected scripts (opt-in) |
60
- | `LANG_MISMATCH` | Answered in the wrong language, same alphabet | Function-word profile (opt-in) |
73
+ | `SCRIPT_MISMATCH` | Answered in the wrong alphabet | Share of letters outside expected scripts · opt-in |
74
+ | `LANG_MISMATCH` | Wrong language, same alphabet | Function-word profile · opt-in |
75
+ | `PROMPT_ECHO` | Returned your prompt instead of an answer | Share of output copied from the prompt · opt-in |
61
76
 
62
77
  Every detector runs even after one fails, so a verdict shows the whole picture
63
78
  rather than whichever check happened to be ordered first. Each returns **0–1, not
64
- a boolean** — you pick the line. Full reference: **[docs/detectors.md](docs/detectors.md)**.
79
+ a boolean** — you pick the line.
80
+
81
+ **Full reference, with the measurements behind every default:
82
+ [docs/detectors.md](docs/detectors.md)**
65
83
 
66
84
  ## Guard your provider in one wrap
67
85
 
@@ -77,8 +95,12 @@ Adapters for the **OpenAI SDK** (both `chat.completions` and `responses`),
77
95
  protocol: Groq, Together, OpenRouter, Fireworks, vLLM, Ollama.
78
96
 
79
97
  On a stream this **cancels the HTTP request** the moment a loop is detectable, so
80
- you stop paying for the rest of it. See **[docs/adapters.md](docs/adapters.md)**
81
- and **[docs/streaming.md](docs/streaming.md)**.
98
+ you stop paying for the rest of it. Two switches worth knowing:
99
+ `checkToolArguments: true` measures the arguments a model passes to a tool, and
100
+ `checkPromptEcho: true` reads the prompt out of each request so a model that
101
+ replays it instead of answering is caught.
102
+
103
+ **[docs/adapters.md](docs/adapters.md)** · **[docs/streaming.md](docs/streaming.md)**
82
104
 
83
105
  ## The hard part is not catching loops
84
106
 
@@ -86,89 +108,60 @@ A miss is annoying. **A false positive is worse** — a healthy response gets
86
108
  discarded and retried against a slower provider for nothing.
87
109
 
88
110
  So the corpus carries deliberate traps: markdown tables, repeated-prefix lists,
89
- code blocks, rhetorical refrains, a Chinese poem refrain. All repetitive, all
90
- fine, all flagged by a naive detector. Paste one into the
111
+ code blocks, rhetorical refrains, a Chinese poem refrain, a Chinese answer
112
+ wrapped around a TypeScript block. All of them look degenerate to a naive
113
+ detector, and all of them are fine. Paste one into the
91
114
  [playground](https://edwinsatya.github.io/llm-output-guard/) and watch it pass.
92
115
 
93
116
  ---
94
117
 
95
- ### Structured output
96
-
97
- ```ts
98
- const verdict = checkOutput(raw, {
99
- ...presets.strictJson,
100
- requiredKeys: ['score', 'notes', 'followUp'],
101
- });
102
-
103
- if (verdict.ok) use(verdict.json); // already parsed, fence stripped
104
- ```
118
+ ## Common setups
105
119
 
106
- `requiredKeys` only asks whether a name is present. A model returning
107
- `{ "score": "very good" }` where you wanted a number satisfies it and still
108
- breaks everything downstream that does arithmetic. Pass a **schema** to check
109
- the shape rather than the spelling:
120
+ **Structured output** parse, check keys, and validate against a schema you
121
+ already have:
110
122
 
111
123
  ```ts
112
124
  import { z } from 'zod';
113
125
 
114
- const Review = z.object({
115
- score: z.number().min(0).max(10),
116
- notes: z.string(),
117
- followUp: z.array(z.string()),
126
+ const verdict = checkOutput(raw, {
127
+ ...presets.strictJson,
128
+ schema: z.object({ score: z.number(), notes: z.string() }),
118
129
  });
119
130
 
120
- const verdict = checkOutput(raw, { ...presets.strictJson, schema: Review });
121
-
122
131
  if (verdict.ok) use(verdict.json); // parsed, validated, defaults applied
123
132
  ```
124
133
 
125
- Any [Standard Schema](https://standardschema.dev) validator works — **Zod 4,
126
- Valibot, ArkType**, or your own. The spec is types-only, so this costs an
127
- interface and **no dependency**: your validator is one you already have, and
128
- `llm-output-guard` still installs with nothing behind it.
129
-
130
- On success `verdict.json` is the schema's *output*, so defaults, coercions and
131
- transforms are applied and the value matches the type you declared. On failure
132
- you get `INVALID_JSON` with the failing path in the message —
133
- `score: Expected number, received string`. It is the same reason code as a
134
- missing key or an unparseable payload because it wants the same handling: retry,
135
- or fall through to another provider.
136
-
137
- The two compose, and keys are checked first, so a missing key is still reported
138
- as a missing key rather than as whatever the schema calls it.
139
-
140
- > **The schema must validate synchronously.** `checkOutput` is synchronous by
141
- > design — that is what makes it safe on a hot path — so a schema carrying an
142
- > async refinement throws a `TypeError` telling you so, rather than silently
143
- > passing. Everything Zod, Valibot and ArkType produce otherwise is synchronous.
144
- > This is the one thing in the package that throws about your configuration; it
145
- > still never throws about a response.
134
+ Any [Standard Schema](https://standardschema.dev) validator works — Zod 4,
135
+ Valibot, ArkType and the spec is types-only, so this still costs **no
136
+ dependency**. Details, and the one case that throws:
137
+ [docs/detectors.md](docs/detectors.md#structured-output)
146
138
 
147
- See **[docs/detectors.md](docs/detectors.md)** for arrays of repeated records
148
- a JSON array of identical rows reads as a loop under the default scope, and
149
- `redundancyScope: 'jsonValues'` is the fix.
150
-
151
- ### Answering in the wrong language
139
+ **Answered in the wrong language** a model that ignores "answer in English"
140
+ returns fluent Chinese, not broken English:
152
141
 
153
142
  ```ts
154
143
  checkOutput(raw, { ...presets.chat, expectScript: 'latin' });
155
144
  ```
156
145
 
157
- A model that ignores "answer in English" does not produce broken English, it
158
- produces fluent Chinese. That is a `200 OK` your retry layer cannot see, and it
159
- is detectable by counting characters no word list, no model, decisive from
160
- about a dozen letters. A response answered entirely in the wrong script scores
161
- **1.000**; a healthy response measured against its own script scores
162
- **0.000–0.028**.
146
+ Detectable by counting characters no word list, decisive from about a dozen
147
+ letters. A wrong-script answer scores **1.000**; a healthy one measured against
148
+ its own script scores **0.000–0.028**. Pass every script the answer may
149
+ legitimately contain (`['han', 'latin']` for Chinese). Code fences and URLs are
150
+ excluded, so a TypeScript block never counts as answering in English.
151
+ [More →](docs/detectors.md#answering-in-the-wrong-language)
152
+
153
+ **Returned your prompt instead of an answer** — invisible to every other
154
+ detector here, because such a response is fluent, well-formed and the right
155
+ length:
163
156
 
164
- Pass every script the answer may legitimately contain — `['han', 'latin']` for
165
- Chinese, `['han', 'kana', 'latin']` for Japanese. `'latin'` belongs in nearly all
166
- of them, because a Chinese answer about React still contains `useEffect`. Code
167
- fences, inline code and URLs are removed before measuring, so a TypeScript block
168
- never counts as answering in English.
157
+ ```ts
158
+ checkOutput(raw, { ...presets.chat, prompt });
159
+ // or, from an adapter, which reads the prompt out of the request itself:
160
+ withOutputGuard(new OpenAI(), { ...presets.chat, checkPromptEcho: true });
161
+ ```
169
162
 
170
- Same script means no signal: Spanish against English scores 0. That is what
171
- `expectLang` is for, and the two compose under separate codes.
163
+ Not for rewrite, translate or summarise endpoints, where copying the input is
164
+ the job. [More →](docs/detectors.md#returning-the-prompt-instead-of-an-answer)
172
165
 
173
166
  ## The verdict
174
167
 
@@ -185,71 +178,63 @@ Same script means no signal: Spanish against English scores 0. That is what
185
178
  }
186
179
  ```
187
180
 
188
- Every detector runs even after one fails, so `reasons` shows the whole picture instead of whichever check happened to be ordered first. `scores` includes passing detectors too — send them to your metrics and you will know your real degeneration rate within a day.
181
+ `reasons` carries every failing signal, not just the first. `scores` includes the
182
+ passing detectors too — send them to your metrics and you will know your real
183
+ degeneration rate within a day.
189
184
 
190
- `modes` says which tokenizer produced a score, for the detectors that have more
191
- than one. **Log it next to `scores`.** `TAIL_LOOP` measures words on spaced
192
- scripts and characters on Chinese, Japanese and Thai; those are two
193
- distributions with different base rates, and aggregating them into one histogram
194
- gives you a number that describes neither.
185
+ `modes` says which tokenizer produced a score. **Log it next to `scores`:**
186
+ `TAIL_LOOP` measures words on spaced scripts and characters on Chinese, Japanese
187
+ and Thai, and pooling two distributions into one histogram gives you a number
188
+ that describes neither.
195
189
 
196
190
  ## Presets
197
191
 
198
192
  `chat` · `strictJson` · `longForm` · `lenient`
199
193
 
200
- They are starting points calibrated against the fixture corpus in this repo — not universal truths. Log your scores for a week, then set your own thresholds.
194
+ Starting points calibrated against this repo's fixture corpus — not universal
195
+ truths.
201
196
 
202
- ---
203
-
204
- ## Calibrating against your own traffic
197
+ ## Calibrate against your own traffic
205
198
 
206
- The shipped presets are tuned on this repo's fixture corpus, which is **not your
207
- traffic**. Log your scores for a week, then derive thresholds you can defend:
199
+ The shipped presets are tuned on a corpus that is **not your traffic**. Score the
200
+ responses you already have, then derive thresholds you can defend:
208
201
 
209
202
  ```bash
210
- npx llm-output-guard calibrate scores.jsonl --fpr 0.001
203
+ npx llm-output-guard check logs/*.txt --json | npx llm-output-guard calibrate --fpr 0.001
211
204
  ```
212
205
 
206
+ `check` also works as a CI assertion — it exits 1 when anything is degenerate,
207
+ 2 when the input cannot be read.
208
+
213
209
  The report tells you when your sample is too small to support the rate you asked
214
210
  for, and distinguishes real separation in your data from a false-positive budget —
215
- because only one of those is evidence. Full guide:
216
- **[docs/calibration.md](docs/calibration.md)**.
217
-
218
- ## Script coverage
219
-
220
- Korean, Cyrillic, Greek, Arabic and Devanagari separate words and are handled like
221
- English. **Chinese, Japanese and Thai do not**, so `TAIL_LOOP` switches to
222
- character mode and reads its own threshold. `REPETITION` is blind on those scripts —
223
- a known, measured gap. `SCRIPT_MISMATCH` covers all ten scripts and is the one
224
- detector these are *not* the weak case for. Numbers behind both in
225
- **[docs/script-coverage.md](docs/script-coverage.md)**.
211
+ because only one of those is evidence.
212
+ **[docs/calibration.md](docs/calibration.md)**
226
213
 
227
214
  ## Design notes
228
215
 
229
- - **Zero runtime dependencies**, enforced in CI. Node ≥ 18, works on edge, browser, Deno, Bun.
230
- - **Hand-rolled LZ77** rather than `node:zlib`, so the package stays runtime-agnostic. It is not a real compressor; it only needs to move monotonically with redundancy.
216
+ - **Zero runtime dependencies**, enforced in CI. Node ≥ 18; works on edge, browser, Deno, Bun.
231
217
  - **Pure and synchronous.** No network, no clock, no randomness — safe on a hot path, trivial to test.
232
218
  - **Scores, not booleans.** Detectors report 0–1 and leave the threshold decision to you.
233
219
  - **Abstains rather than guesses.** Samples too short to judge score 0.
220
+ - **Hand-rolled LZ77** rather than `node:zlib`, so the package stays runtime-agnostic.
221
+ - **Sub-millisecond**, 0.383 ms at 500 B and 1.014 ms at 32 KB, with one detector accounting for most of it. `npm run bench` reproduces it — **[docs/performance.md](docs/performance.md)**
222
+ - **Chinese, Japanese and Thai** are handled where they differ: `TAIL_LOOP` switches to character mode, `REPETITION` is blind and says so — **[docs/script-coverage.md](docs/script-coverage.md)**
234
223
 
235
224
  ## Stability
236
225
 
237
- What semver means for this package specifically. These rules bind from **1.0.0**
238
- onward; under `0.x` they described an intent, and the surface was frozen — export
239
- by export — in the 1.0.0 release.
240
-
241
- **The public API is:** everything exported from `llm-output-guard`, plus
242
- `outputGuard` / `OutputGuardOptions` / `DegenerateAction` from `./ai-sdk`, and
243
- `withOutputGuard` / `OutputGuardOptions` / `DegenerateAction` from each of
244
- `./openai` and `./anthropic`. Each subpath is its own contract; the adapters
245
- share internal base types today and are free to diverge, so an option added to
246
- one is not a promise about the others. Anything not exported from those four
247
- entry points is internal, has no stability guarantee, and may move in any release
248
- `internal/proxy-guard.ts` and `internal/tool-calls.ts` included, however much
249
- behaviour they carry. The list is asserted in `test/surface.test.ts`, so an
250
- export cannot join it by accident.
251
-
252
- **Threshold and preset values are behaviour, not implementation.** This is the
226
+ What semver means here specifically. These rules bind from **1.0.0** onward, and
227
+ the public surface was frozen export by export in that release.
228
+
229
+ **The public API is** everything exported from `llm-output-guard`, plus
230
+ `outputGuard` / `OutputGuardOptions` / `DegenerateAction` from `./ai-sdk` and
231
+ `withOutputGuard` / `OutputGuardOptions` / `DegenerateAction` from `./openai` and
232
+ `./anthropic`. Each subpath is its own contract, so an option added to one is not
233
+ a promise about the others. Anything else is internal and may move in any
234
+ release. The list is asserted in `test/surface.test.ts`, so an export cannot join
235
+ it by accident.
236
+
237
+ **Threshold and preset values are behaviour, not implementation.** That is the
253
238
  interesting case, so it gets a rule of its own:
254
239
 
255
240
  | Change | Release type |
@@ -259,42 +244,39 @@ interesting case, so it gets a rule of its own:
259
244
  | Adding a new *option*, defaulted so nothing changes | minor |
260
245
  | Adding a new detector that is opt-in | minor |
261
246
  | Making an existing detector strictly more accurate on its own axis | minor |
262
- | Docs, internals, performance, fixing a detector that was returning a wrong score | patch |
263
-
264
- The reasoning: a threshold change does not break your build, it changes which of
265
- your production responses get discarded and retried. That is a larger event than
266
- a signature change, and it is invisible until your traffic hits it. A number in
267
- `presets.chat` is part of the contract in the same way a function name is.
268
-
269
- **Semver does not cover:** the exact scores a detector returns (only their
270
- direction and the thresholds that act on them), the contents of the fixture
271
- corpus, `message` strings in `Reason`, or the output format of the `calibrate`
272
- CLI's human-readable report. `--json` output *is* covered.
273
-
274
- **Peer ranges** are narrowed only in a major. They are verified rather than
275
- assumed `npm run check:peers` installs the packed tarball against each end of
276
- each declared range and both typechecks and runs the adapter.
277
-
278
- **Why this is written down.** Version 0.4.2 shipped the `ai` peer narrowing, the
279
- new `./openai` subpath, and character-mode `TAIL_LOOP` one breaking change, one
280
- feature, and one behaviour change under a **patch** number, which every default
281
- version range upgrades into automatically. It was withdrawn from npm within the
282
- 72-hour unpublish window and re-released as 0.5.0, where `^0.4.1` correctly
283
- resolves away from it. The rule it broke is the one in the table above: threshold
284
- and preset changes are behaviour changes, and behaviour changes are never
285
- patches.
247
+ | Docs, internals, performance, fixing a detector that returned a wrong score | patch |
248
+
249
+ A threshold change does not break your build. It changes which of your production
250
+ responses get discarded and retried, which is a larger event than a signature
251
+ change and invisible until your traffic hits it. A number in `presets.chat` is
252
+ part of the contract in the same way a function name is.
253
+
254
+ **Not covered:** the exact scores a detector returns (only their direction and
255
+ the thresholds acting on them), the fixture corpus, `message` strings in
256
+ `Reason`, or the human-readable `calibrate` report. `--json` output *is* covered.
257
+
258
+ **Peer ranges** narrow only in a major, and are verified rather than assumed —
259
+ `npm run check:peers` installs the packed tarball against each end of each
260
+ declared range, then typechecks and runs the adapter.
261
+
262
+ **Why this is written down:** 0.4.2 shipped a peer narrowing, a new subpath, and
263
+ a behaviour change under a **patch** number, which every default version range
264
+ upgrades into automatically. It was unpublished within the 72-hour window and
265
+ re-released as 0.5.0. The rule it broke is the one in the table above.
286
266
 
287
267
  ## Limitations
288
268
 
289
269
  - Not a hallucination detector. It measures *shape*, never truth.
290
- - Tool *arguments* are not checked, only the prose beside them. A model that loops inside a JSON argument string is invisible here — your provider validates those against the schema you gave it.
291
- - `openai`'s `responses.stream()` helper is not wrapped. See the note above; `create({ stream: true })` is.
292
- - `REPETITION` does not work on Chinese, Japanese or Thai. See above this is a known, measured gap, not an oversight.
293
- - Language detection is a function-word heuristic covering `id`/`en`/`es`. Opt-in, and unreliable under 25 words. `expectScript` is the stronger check where the languages differ in alphabet, and says nothing where they do not.
294
- - `SCRIPT_MISMATCH` does not run mid-stream. A mid-stream check reads a trailing window, and the language of a window is not the language of the response an English answer quoting a Chinese passage measures 0.114 whole and 0.500 over its last 400 characters.
295
- - Truncation from a missing full stop is weak evidence, scored 0.55 and left below the default thresholds on purpose. Lower `maxTruncation` to ~0.5 to catch it, and expect false positives.
296
- - A JSON array of repeated identical records reads as a loop under the default scope, and fails from three records up. Set `redundancyScope: 'jsonValues'` — see **Structured output**.
297
- - Thresholds calibrated on the bundled corpus. Yours will differ and the word and character thresholds need calibrating **separately**, because they are separate distributions.
270
+ - `REPETITION` does not work on Chinese, Japanese or Thai a known, measured gap, not an oversight.
271
+ - `LANG_MISMATCH` is a function-word heuristic covering `id`/`en`/`es`/`pt`/`it`/`fr`/`de`/`nl`, and is unreliable under 25 words. `expectScript` is the stronger check wherever the alphabets differ.
272
+ - `expectLang: 'es'` does not reliably catch Portuguese, Italian or French: its profile is built from function words all four share. Measured, and unfixable without a threshold change — see **[docs/detectors.md](docs/detectors.md#answering-in-the-wrong-language)**.
273
+ - `PROMPT_ECHO` cannot tell a degenerate echo from a rewrite or translation the difference is in what you asked for, not in the text.
274
+ - `SCRIPT_MISMATCH` and `PROMPT_ECHO` do not run mid-stream by default: both measure a property of the whole response, and a mid-stream check reads a trailing window. `earlyDocumentChecks: true` opts in, with a measured false-positive risk see **[docs/streaming.md](docs/streaming.md)**.
275
+ - Tool *arguments* are measured only with `checkToolArguments: true`, non-streaming responses only.
276
+ - `openai`'s `responses.stream()` helper is not wrapped; `create({ stream: true })` is.
277
+ - Truncation from a missing full stop is weak evidence, scored 0.55 and deliberately left below the defaults. Lower `maxTruncation` to ~0.5 to catch it, and expect false positives.
278
+ - A JSON array of repeated identical records reads as a loop and fails from three records up. Set `redundancyScope: 'jsonValues'`.
279
+ - Thresholds are calibrated on the bundled corpus. Yours will differ — and the word and character thresholds need calibrating **separately**.
298
280
 
299
281
  ## License
300
282
 
@@ -1,4 +1,4 @@
1
- import { V as Verdict } from './stream-NO-TTGYs.cjs';
1
+ import { V as Verdict } from './stream-BYy6vuGy.cjs';
2
2
 
3
3
  /**
4
4
  * The option surface every provider adapter shares.
@@ -62,6 +62,40 @@ interface AdapterGuardOptions {
62
62
  onVerdict?: (verdict: Verdict, context: {
63
63
  streaming: boolean;
64
64
  }) => void;
65
+ /**
66
+ * Also measure the arguments the model passed to a tool. Default `false`.
67
+ *
68
+ * A tool-calling turn is judged by its preamble, because the text beside a
69
+ * tool call is not the answer -- and that leaves the answer itself
70
+ * unmeasured. Your provider validates arguments against the schema you gave
71
+ * it, which covers types and not content: `{ "query": "site reliability site
72
+ * reliability site reliability ..." }` is a schema-valid string, and it is
73
+ * still a garbage query.
74
+ *
75
+ * Switching this on measures each string value of each call's arguments for
76
+ * redundancy, and folds the result into the same verdict the preamble
77
+ * produces. Reason codes are unchanged, so existing handling works; the
78
+ * `message` says the loop was found in an argument.
79
+ *
80
+ * **Non-streaming responses only.** Arguments arrive as JSON fragments that
81
+ * do not parse until the call is complete, so there is nothing meaningful to
82
+ * measure mid-stream.
83
+ */
84
+ checkToolArguments?: boolean;
85
+ /**
86
+ * Check the response against the prompt that produced it, so a model that
87
+ * replays your prompt instead of answering is caught. Default `false`.
88
+ *
89
+ * `PROMPT_ECHO` needs the prompt, and a guard is configured once when you
90
+ * wrap the client while the prompt changes on every call -- so this is a
91
+ * switch rather than a value. Turn it on and the adapter reads the prompt out
92
+ * of each request it is already forwarding.
93
+ *
94
+ * **Do not enable this on a rewrite, translate, summarise or extract
95
+ * endpoint.** Copying from the input is the job on those, so a correct answer
96
+ * scores high and the detector measures the task rather than a failure.
97
+ */
98
+ checkPromptEcho?: boolean;
65
99
  }
66
100
 
67
101
  export type { AdapterGuardOptions as A, DegenerateAction as D };
@@ -1,4 +1,4 @@
1
- import { V as Verdict } from './stream-NO-TTGYs.js';
1
+ import { V as Verdict } from './stream-BYy6vuGy.js';
2
2
 
3
3
  /**
4
4
  * The option surface every provider adapter shares.
@@ -62,6 +62,40 @@ interface AdapterGuardOptions {
62
62
  onVerdict?: (verdict: Verdict, context: {
63
63
  streaming: boolean;
64
64
  }) => void;
65
+ /**
66
+ * Also measure the arguments the model passed to a tool. Default `false`.
67
+ *
68
+ * A tool-calling turn is judged by its preamble, because the text beside a
69
+ * tool call is not the answer -- and that leaves the answer itself
70
+ * unmeasured. Your provider validates arguments against the schema you gave
71
+ * it, which covers types and not content: `{ "query": "site reliability site
72
+ * reliability site reliability ..." }` is a schema-valid string, and it is
73
+ * still a garbage query.
74
+ *
75
+ * Switching this on measures each string value of each call's arguments for
76
+ * redundancy, and folds the result into the same verdict the preamble
77
+ * produces. Reason codes are unchanged, so existing handling works; the
78
+ * `message` says the loop was found in an argument.
79
+ *
80
+ * **Non-streaming responses only.** Arguments arrive as JSON fragments that
81
+ * do not parse until the call is complete, so there is nothing meaningful to
82
+ * measure mid-stream.
83
+ */
84
+ checkToolArguments?: boolean;
85
+ /**
86
+ * Check the response against the prompt that produced it, so a model that
87
+ * replays your prompt instead of answering is caught. Default `false`.
88
+ *
89
+ * `PROMPT_ECHO` needs the prompt, and a guard is configured once when you
90
+ * wrap the client while the prompt changes on every call -- so this is a
91
+ * switch rather than a value. Turn it on and the adapter reads the prompt out
92
+ * of each request it is already forwarding.
93
+ *
94
+ * **Do not enable this on a rewrite, translate, summarise or extract
95
+ * endpoint.** Copying from the input is the job on those, so a correct answer
96
+ * scores high and the detector measures the task rather than a failure.
97
+ */
98
+ checkPromptEcho?: boolean;
65
99
  }
66
100
 
67
101
  export type { AdapterGuardOptions as A, DegenerateAction as D };