llm-output-guard 0.5.0 → 1.0.1

package/README.md

@@ -318,18 +318,34 @@ gives you a number that describes neither.
 
 ## Detectors
 
-| Code | Catches | Signal |
-|---|---|---|
-| `EMPTY` | Whitespace, lone punctuation, `{}`, empty fences | Content presence |
-| `TOO_SHORT` | Non-empty but useless | Length vs. minimum |
-| `REPETITION` | Loops and stutters | Duplicate word n-gram fraction |
-| `TAIL_LOOP` | Good start, then a stuck ending | Periodicity in the trailing window, over words or characters |
-| `LOW_ENTROPY` | Character-level collapse, token artifacts | Hand-rolled LZ77 compression ratio |
-| `TRUNCATED` | Cut off mid-thought | `finish_reason`, unbalanced fences/brackets |
-| `INVALID_JSON` | Prose around the payload, missing keys | Parse + key contract |
-| `LANG_MISMATCH` | Answered in the wrong language | Function-word profile (coarse, opt-in) |
+| Code | Catches | Signal | Exported as |
+|---|---|---|---|
+| `EMPTY` | Whitespace, lone punctuation, `{}`, empty fences | Content presence | `emptinessScore` |
+| `TOO_SHORT` | Non-empty but useless | Length vs. minimum | `shortnessScore` |
+| `REPETITION` | Loops and stutters | Duplicate word n-gram fraction | `repetitionScore` |
+| `TAIL_LOOP` | Good start, then a stuck ending | Periodicity in the trailing window, over words or characters | `tailLoopScore`, `tailLoopDetail` |
+| `LOW_ENTROPY` | Character-level collapse, token artifacts | Hand-rolled LZ77 compression ratio | `compressibilityScore`, `compressionRatio` |
+| `TRUNCATED` | Cut off mid-thought | `finish_reason`, unbalanced fences/brackets | `truncationScore` |
+| `INVALID_JSON` | Prose around the payload, missing keys | Parse + key contract | `jsonScore`, `stripFence` |
+| `LANG_MISMATCH` | Answered in the wrong language | Function-word profile (coarse, opt-in) | `languageMismatchScore`, `languageProfile`, `supportedLanguages` |
+
+Every detector is exported on its own if you only want one, and every name in
+that last column is covered by semver — see **Stability**.
+
+```ts
+import { repetitionScore, tailLoopDetail, stripFence } from 'llm-output-guard';
+
+repetitionScore(text);                  // 0..1, higher is worse
+repetitionScore(text, { n: 4 });        // n-gram size
+tailLoopDetail(text, { mode: 'char' }); // { score, mode } — which tokenizer ran
+stripFence('```json\n{"a":1}\n```');    // '{"a":1}'
+```
 
-Every detector is exported on its own if you only want one.
+Each takes `(text, options?)` and returns a `0..1` score, with three exceptions
+worth knowing: `shortnessScore(text, minChars)` takes its minimum positionally,
+`stripFence` returns a string, and `jsonScore` / `tailLoopDetail` return a detail
+object rather than a bare number. `supportedLanguages` is a value, not a
+function — the array `['id', 'en', 'es']`.
 
 ## Presets
 
@@ -390,6 +406,42 @@ the bulk and a cluster of outliers is real separation observed in your data
 rather than an assumption about rarity — and when that hole rests on one or
 two samples, the report says so.
 
+### The same thing, as a function
+
+The CLI is a wrapper. If your scores already live somewhere the shell cannot
+reach them — a metrics store, a warehouse query, a test — call `calibrate`
+directly. It takes the same flat objects the JSONL format describes:
+
+```ts
+import { calibrate } from 'llm-output-guard';
+
+const { n, summaries } = calibrate(
+  [
+    { REPETITION: 0.03, TAIL_LOOP: 0 },
+    { REPETITION: 0.91, TAIL_LOOP: 0.88, modes: { TAIL_LOOP: 'char' } },
+    // ...one entry per logged verdict
+  ],
+  { falsePositiveRate: 0.001 },
+);
+
+for (const s of summaries) {
+  s.code;               // 'REPETITION'
+  s.mode;               // 'word' | 'char', when the samples recorded one
+  s.suggested;          // threshold flagging falsePositiveRate of this sample
+  s.gap;                // { below, above, count, share } | null — stronger evidence
+  s.distribution;       // { n, nonZero, min, max, p50, p90, p99, p999 }
+  s.caveats;            // everything that makes `suggested` untrustworthy
+}
+```
+
+`modes` rides along in the same object and is not read as a score. Log it, and
+`summaries` comes back segmented — one entry per `code`+`mode` — for the reason
+in the paragraph above. `summarise(code, scores, options)` is exported too, for
+when you have one detector's numbers already grouped.
+
+**Read `caveats` before `suggested`.** It is where a sample too small for the
+requested rate says so, and a `suggested` number carries no warning of its own.
+
 ## On thresholds
 
 A miss is annoying. **A false positive is worse**: a healthy response gets discarded and retried against a slower provider for nothing.
@@ -526,7 +578,9 @@ output starts to reach.
 
 ## Stability
 
-What semver means for this package specifically.
+What semver means for this package specifically. These rules bind from **1.0.0**
+onward; under `0.x` they described an intent, and the surface was frozen — export
+by export — in the 1.0.0 release.
 
 **The public API is:** everything exported from `llm-output-guard`, plus
 `outputGuard` / `OutputGuardOptions` / `DegenerateAction` from `./ai-sdk` and
@@ -534,7 +588,8 @@ What semver means for this package specifically.
 Each subpath is its own contract; the two adapters share an internal base type
 today and are free to diverge, so an option added to one is not a promise about
 the other. Anything not exported from those three entry points is internal, has
-no stability guarantee, and may move in any release.
+no stability guarantee, and may move in any release. The list is asserted in
+`test/surface.test.ts`, so an export cannot join it by accident.
 
 **Threshold and preset values are behaviour, not implementation.** This is the
 interesting case, so it gets a rule of its own:

package/dist/index.cjs

@@ -579,12 +579,10 @@ exports.compressibilityScore = compressibilityScore;
 exports.compressionRatio = compressionRatio;
 exports.createStreamGuard = createStreamGuard;
 exports.emptinessScore = emptinessScore;
-exports.findGap = findGap;
 exports.guardStream = guardStream;
 exports.jsonScore = jsonScore;
 exports.languageMismatchScore = languageMismatchScore;
 exports.languageProfile = languageProfile;
-exports.percentile = percentile;
 exports.presets = presets;
 exports.repetitionScore = repetitionScore;
 exports.shortnessScore = shortnessScore;

package/dist/index.d.cts

@@ -102,19 +102,6 @@ interface Summary {
     /** Everything that would make the number above untrustworthy. */
     caveats: string[];
 }
-/** Linear-interpolated percentile. `sorted` must be ascending. */
-declare function percentile(sorted: number[], p: number): number;
-/**
- * The widest empty stretch in the upper tail, if there is one.
- *
- * A genuinely bimodal detector -- healthy output clustered near zero, a
- * handful of failures far above it -- leaves a visible hole between the two.
- * That hole is worth far more than a percentile, because it is evidence about
- * *this* detector on *your* traffic rather than an assumption about rarity.
- * Searching only above the median keeps the ordinary spread of healthy scores
- * from being mistaken for a separation.
- */
-declare function findGap(sorted: number[], minWidth?: number): Gap | null;
 /**
  * Summarise one detector's scores.
  *
@@ -412,4 +399,4 @@ declare function languageProfile(text: string): Record<string, number>;
 declare function languageMismatchScore(text: string, expected: string, options?: LanguageOptions): number;
 declare const supportedLanguages: string[];
 
-export { type Calibration, type CalibrationOptions, CheckOptions, type CompressibilityOptions, DegenerateOutputError, type Distribution, type Gap, type JsonOptions, type JsonResult, type LanguageOptions, ReasonCode, type RepetitionOptions, type ScoreSample, type Summary, type TailLoopOptions, type TailLoopResult, TokenMode, type TruncationOptions, Verdict, assertOutput, calibrate, checkOutput, compressibilityScore, compressionRatio, emptinessScore, findGap, jsonScore, languageMismatchScore, languageProfile, percentile, presets, repetitionScore, shortnessScore, stripFence, summarise, supportedLanguages, tailLoopDetail, tailLoopScore, truncationScore };
+export { type Calibration, type CalibrationOptions, CheckOptions, type CompressibilityOptions, DegenerateOutputError, type Distribution, type Gap, type JsonOptions, type JsonResult, type LanguageOptions, ReasonCode, type RepetitionOptions, type ScoreSample, type Summary, type TailLoopOptions, type TailLoopResult, TokenMode, type TruncationOptions, Verdict, assertOutput, calibrate, checkOutput, compressibilityScore, compressionRatio, emptinessScore, jsonScore, languageMismatchScore, languageProfile, presets, repetitionScore, shortnessScore, stripFence, summarise, supportedLanguages, tailLoopDetail, tailLoopScore, truncationScore };

package/dist/index.d.ts

@@ -102,19 +102,6 @@ interface Summary {
     /** Everything that would make the number above untrustworthy. */
     caveats: string[];
 }
-/** Linear-interpolated percentile. `sorted` must be ascending. */
-declare function percentile(sorted: number[], p: number): number;
-/**
- * The widest empty stretch in the upper tail, if there is one.
- *
- * A genuinely bimodal detector -- healthy output clustered near zero, a
- * handful of failures far above it -- leaves a visible hole between the two.
- * That hole is worth far more than a percentile, because it is evidence about
- * *this* detector on *your* traffic rather than an assumption about rarity.
- * Searching only above the median keeps the ordinary spread of healthy scores
- * from being mistaken for a separation.
- */
-declare function findGap(sorted: number[], minWidth?: number): Gap | null;
 /**
  * Summarise one detector's scores.
  *
@@ -412,4 +399,4 @@ declare function languageProfile(text: string): Record<string, number>;
 declare function languageMismatchScore(text: string, expected: string, options?: LanguageOptions): number;
 declare const supportedLanguages: string[];
 
-export { type Calibration, type CalibrationOptions, CheckOptions, type CompressibilityOptions, DegenerateOutputError, type Distribution, type Gap, type JsonOptions, type JsonResult, type LanguageOptions, ReasonCode, type RepetitionOptions, type ScoreSample, type Summary, type TailLoopOptions, type TailLoopResult, TokenMode, type TruncationOptions, Verdict, assertOutput, calibrate, checkOutput, compressibilityScore, compressionRatio, emptinessScore, findGap, jsonScore, languageMismatchScore, languageProfile, percentile, presets, repetitionScore, shortnessScore, stripFence, summarise, supportedLanguages, tailLoopDetail, tailLoopScore, truncationScore };
+export { type Calibration, type CalibrationOptions, CheckOptions, type CompressibilityOptions, DegenerateOutputError, type Distribution, type Gap, type JsonOptions, type JsonResult, type LanguageOptions, ReasonCode, type RepetitionOptions, type ScoreSample, type Summary, type TailLoopOptions, type TailLoopResult, TokenMode, type TruncationOptions, Verdict, assertOutput, calibrate, checkOutput, compressibilityScore, compressionRatio, emptinessScore, jsonScore, languageMismatchScore, languageProfile, presets, repetitionScore, shortnessScore, stripFence, summarise, supportedLanguages, tailLoopDetail, tailLoopScore, truncationScore };

package/dist/index.js

@@ -143,6 +143,6 @@ var presets = {
   }
 };
 
-export { calibrate, findGap, percentile, presets, summarise };
+export { calibrate, presets, summarise };
 //# sourceMappingURL=index.js.map
 //# sourceMappingURL=index.js.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "llm-output-guard",
-  "version": "0.5.0",
+  "version": "1.0.1",
   "description": "Detect degenerate LLM output that arrives with a 200 OK. Zero dependencies, deterministic, composes with any retry or fallback layer.",
   "keywords": [
     "llm",
@@ -38,7 +38,7 @@
     "./package.json": "./package.json"
   },
   "bin": {
-    "llm-output-guard": "./dist/bin.js"
+    "llm-output-guard": "dist/bin.js"
   },
   "files": [
     "dist"