open-classify 0.5.0 → 0.6.0

package/docs/resolver.md

@@ -1,12 +1,10 @@
 # Aggregation and model resolution
 
-The aggregator merges classifier outputs into an `Envelope`, picks a concrete model from the catalog, and returns a `PipelineResult`.
+The aggregator merges classifier outputs into a `PipelineResult` with a flat shape — no nested `audit` or `downstream` envelope.
 
-## Certainty threshold
+## Certainty labels
 
-Default: `0.65`. Configurable via `aggregator.certaintyThreshold` on `classifyOpenClassifyInput`. `aggregator.confidenceThreshold` remains as a deprecated compatibility alias.
-
-Per-classifier outputs carry `certainty` tags. The aggregator maps tags to scores:
+Classifier outputs carry a `certainty` label. The aggregator maps labels to numeric scores for comparison and reporting:
 
 ```ts
 {
@@ -21,23 +19,27 @@ Per-classifier outputs carry `certainty` tags. The aggregator maps tags to score
 }
 ```
 
-Reserved-field values from below-threshold classifiers are dropped from the named envelope slots. The full underlying output still appears in `audit.classifier_outputs[]` and `audit.meta.classifiers[name]`, so the caller can inspect or override.
+Labels stay in classifier prompts (the model understands them as semantic grades). Floats appear only in the final `PipelineResult` fields: `avg_certainty`, `min_certainty`, and `classifier_outputs[name].certainty`.
 
-Dropped routing axes are reported on `audit.model_recommendation.resolution.constraints_dropped` with `reason: "low_confidence"`.
+## Reserved-field merging
 
-Custom (non-reserved) outputs are surfaced regardless of certainty — callers can decide what to do with them — but the value still goes through schema validation.
+When multiple classifiers emit the same reserved field, the aggregator picks the highest-certainty contributor. Ties are broken by manifest `dispatch_order` ascending (first wins). Classifiers without `dispatch_order` sort last for tie-break purposes.
 
-## Reserved-field merging
+There is no certainty threshold gate — the highest-certainty value always wins, regardless of score. Values below any particular threshold are still reported in `classifier_outputs` for the caller to inspect.
 
-When multiple classifiers emit the same reserved field, the aggregator picks the highest-certainty contributor that meets the threshold. Ties are broken by manifest `dispatch_order` ascending (first wins). Classifiers without `dispatch_order` sort last for tie-break purposes.
+## Action
 
-The built-in classifiers each own a distinct reserved field, so the tie-break only matters if you add your own classifier that emits a field already covered by a built-in.
+Every result has `action: "route" | "block" | "reply"`.
 
-## Whole-run certainty summary
+**`"reply"`** — `preflight` emitted `final_reply`. The classifier determined it can answer the message immediately; no downstream model is needed. `result.reply` contains the text. All other classifiers still ran.
 
-Every run includes `audit.meta.certainty.{min, avg}`. These are the lowest and arithmetic-mean certainty scores across all classifiers, including failed classifiers that fell back to their manifest fallback (which use `no_signal` and score 0).
+**`"block"`** — something prevented routing. `result.block_reason` names the cause:
+- `"prompt_injection"` — `risk_level` is `"high_risk"` or `"unknown"`, regardless of certainty. This takes priority over other causes.
+- `"classification_error"` — one or more classifiers failed or timed out, or preflight provided no reply (which means the pipeline cannot fulfill its reply contract), or no model could be resolved.
 
-The pipeline does not block based on this summary — the worker pool always returns a `route` action. Callers inspect `audit.meta.certainty` and decide whether to trust the result or fall back to a safer behavior (e.g., force a frontier model, return an apology).
+**`"route"`** — all classifiers succeeded and `result.model_id` names the downstream model to call.
+
+Even on `"block"`, `model_id` and `reply` are populated when they can be (the caller may want to store them). `failed_classifiers` lists every classifier that errored or timed out.
 
 ## Model resolution
 
@@ -60,36 +62,10 @@ Within a pass, candidates are ranked:
 3. larger `context_window`
 4. earlier catalog order
 
-If every pass returns no candidates, the resolver returns `catalog.default` with `fell_back_to_default: true`. (In practice the no-constraints pass always finds at least one model unless the catalog is empty, so the default-fallback path is defensive.)
-
-## Resolution audit
+If every pass returns no candidates, the resolver uses `catalog.default`. In practice the no-constraints pass always finds at least one model unless the catalog is empty, so the default-fallback path is defensive.
 
-Every result carries a resolution report:
-
-```ts
-{
-  constraints_used: { model_specialization?: ..., model_tier?: ... },
-  constraints_dropped: Array<{
-    axis: "model_specialization" | "model_tier",
-    reason: "low_confidence" | "no_match_relaxed" | "default_fallback"
-  }>,
-  confidences: { routing?: number },
-  fell_back_to_default: boolean,
-}
-```
-
-Drop reasons:
-
-- `low_confidence` — a classifier emitted the axis but its certainty was below threshold.
-- `no_match_relaxed` — the axis was requested but no model matched, so the resolver relaxed it.
-- `default_fallback` — every pass failed; the resolver used `catalog.default`.
-
-## Audit envelope
+## Whole-run certainty summary
 
-The full `audit` envelope contains:
+Every run includes `avg_certainty` and `min_certainty` at the top level of `PipelineResult`. These are the arithmetic mean and minimum certainty scores across all classifiers, including failed classifiers that fell back to their manifest fallback (which use `no_signal` and score `0`).
 
-- Reserved-field slots that survived the certainty threshold: `final_reply`, `ack_reply`, `routing`, `tools`, `prompt_injection`
-- `classifier_outputs[]` — every classifier's full output, in registry order, including `reason`, `certainty`, all reserved fields, and all custom fields
-- `model_recommendation` with the resolution audit above
-- `meta.classifiers[name]` — per-classifier full output plus `status` and `version`
-- `meta.certainty.{min, avg}` — whole-run certainty summary
+The pipeline does not block based on these values — the caller inspects them and decides whether to trust the result or fall back to a safer behavior.

package/docs/signals.md

@@ -14,7 +14,7 @@ type Certainty =
   | "near_certain";
 ```
 
-The aggregator maps certainty tags to numeric scores. Reserved-field values from classifiers below the configured threshold (default `0.65`) are dropped from the envelope; the underlying output still appears in `audit.classifier_outputs` and `meta.classifiers` so the caller can decide whether to trust the run.
+The aggregator maps certainty tags to numeric scores. `classifier_outputs[name].certainty` is a float; `avg_certainty` and `min_certainty` on the top-level result are also floats. Certainty labels are internal to classifier prompts; floats are what callers see.
 
 ## Reserved fields
 
@@ -26,9 +26,9 @@ A manifest declares which reserved fields its classifier may emit via the `reser
 { text: string }  // 1–200 chars; must contain at least one non-whitespace character
 ```
 
-Use only for tiny terminal answers (greetings, thanks, spelling, simple arithmetic). The text IS the complete answer to the user — nothing else happens after this. Mutually exclusive with `ack_reply`.
+Use only for tiny terminal answers (greetings, thanks, spelling, simple arithmetic). The text IS the complete answer — nothing else happens after this. Mutually exclusive with `ack_reply`.
 
-When emitted with sufficient certainty, the highest-certainty value is surfaced in `audit.final_reply`. The pipeline does not short-circuit; the caller decides whether to return the reply or continue to the downstream model.
+When emitted, the pipeline sets `action: "reply"` and surfaces the text in `result.reply`. All other classifiers still run to completion.
 
 ### `ack_reply`
 
@@ -36,7 +36,9 @@ When emitted with sufficient certainty, the highest-certainty value is surfaced
 { text: string }  // 1–200 chars; must contain at least one non-whitespace character
 ```
 
-A brief acknowledgement to show while downstream work continues. Surfaced in `audit.ack_reply`. Mutually exclusive with `final_reply`.
+A brief, task-specific acknowledgement to show while downstream work continues. Mutually exclusive with `final_reply`.
+
+When emitted (and the action is `"route"`), the text is surfaced in `result.reply`. This is the immediate response your UI can show while the downstream model works.
 
 ### `model_tier`
 
@@ -62,22 +64,30 @@ Soft constraint for the catalog resolver. The resolver picks the cheapest catalo
 string[]   // each id must appear in the manifest's allowed_tools list
 ```
 
-Sets `downstream.tools.tools`. Any classifier emitting this reserved field must declare `allowed_tools` on its manifest — that menu of allowed ids becomes both the JSON Schema constraint and the prompt listing.
+Sets `result.tools`. Any classifier emitting this reserved field must declare `allowed_tools` on its manifest — that menu of allowed ids becomes both the JSON Schema constraint and the prompt listing.
 
 Common tool-id aliases (`browser`, `browsing`, `internet`, `web_browsing`, `web_search`) are normalized to `web` before validation, so the model can drift on phrasing without breaking.
 
+`result.tools` is always an array (empty if no classifier emitted it or no tools were selected).
+
 ### `risk_level`
 
 ```ts
 "normal" | "suspicious" | "high_risk" | "unknown"
 ```
 
-Prompt-injection posture for the target message. Surfaced in `audit.prompt_injection`. The pipeline does not short-circuit; the caller decides whether to block based on the risk level and certainty.
+Prompt-injection posture for the target message. Surfaced in `result.prompt_injection`.
+
+`"high_risk"` and `"unknown"` trigger `action: "block"` with `block_reason: "prompt_injection"`, regardless of certainty. `"suspicious"` is advisory — the pipeline routes normally and the caller decides whether to act on it.
+
+When the `prompt_injection` classifier fails (runtime error or timeout), it uses its fallback which does **not** include `risk_level`. The pipeline then blocks with `block_reason: "classification_error"`, not `"prompt_injection"` — a classifier failure is distinct from an assessed injection risk.
 
 ## Custom fields
 
-Anything not in the reserved list lives in your manifest's `output_schema.properties`. The runtime validates each output against the composed schema (custom properties + reserved sub-schemas + `reason` + `certainty`) at runtime, and surfaces the full output on `result.classifier_outputs[name]` with `reason` and `certainty` stripped for ergonomic access. The full output, including metadata, appears in `result.audit.classifier_outputs[]` and `result.audit.meta.classifiers[name]`.
+Anything not in the reserved list lives in your manifest's `output_schema.properties`. The runtime validates each output against the composed schema (custom properties + reserved sub-schemas + `reason` + `certainty`) at runtime, and surfaces the full output on `result.classifier_outputs[name]`.
+
+`classifier_outputs[name]` contains all payload fields plus `reason` (string) and `certainty` (float). The raw certainty label is not exposed; only the float score.
 
 ## Picking between reserved-field contributors
 
-When two classifiers declare the same reserved field, the aggregator picks the highest-certainty value above the threshold. Ties are broken by manifest `dispatch_order` ascending (first in registry order keeps the slot). Both classifiers' full outputs still appear in `audit.classifier_outputs` regardless of which one "won" the slot.
+When two classifiers declare the same reserved field, the aggregator picks the highest-certainty value. Ties are broken by manifest `dispatch_order` ascending (first in registry order keeps the slot). Both classifiers' full outputs still appear in `classifier_outputs` regardless of which one "won" the slot.

package/open-classify.config.example.json

@@ -11,15 +11,12 @@
     },
     "models": {
       "preflight": "gemma4:e4b-it-q4_K_M",
-      "routing": "gemma4:e4b-it-q4_K_M",
+      "model_tier": "gemma4:e4b-it-q4_K_M",
       "model_specialization": "gemma4:e4b-it-q4_K_M",
       "tools": "gemma4:e4b-it-q4_K_M",
       "prompt_injection": "gemma4:e4b-it-q4_K_M",
       "memory_retrieval_queries": "gemma4:e4b-it-q4_K_M"
     }
   },
-  "aggregator": {
-    "certaintyThreshold": 0.65
-  },
   "catalog": "downstream-models.json"
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-classify",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Manifest-driven classifier runtime for routing user messages to downstream AI models",
   "license": "MIT",
   "author": "Taylor Bayouth",