open-classify 0.2.0 → 0.4.0

package/README.md

@@ -83,7 +83,7 @@ Every call returns a `PipelineResult` with one of three `action` values:
 | `reply` | Preflight had a tiny terminal reply | `reply.text` |
 | `block` | Prompt injection flagged confident `high_risk` / `unknown`, or the certainty gate fired | `reason.kind` plus prompt-injection or low-certainty details |
 
-All three also carry `message_id`, `classifier_outputs` (custom classifier payloads, keyed by name), and an `audit` block. Route results include the downstream target message, not the caller's message history. Short-circuit results include the firing classifier's audit context.
+All three also carry `target_message_hash` (the stable 8-hex fingerprint of the target message), `classifier_outputs` (custom classifier payloads, keyed by name), and an `audit` block. Route results include the downstream target message, not the caller's message history. Short-circuit results include the firing classifier's audit context.
 
 For complex requests, look for `audit.ack_reply` on `route` results. It is the immediate acknowledgement your UI can show while the downstream model works. For trivial requests, `result.reply.text` is the complete response and no downstream model is needed.
 
@@ -92,7 +92,7 @@ Example `route` result:
 ```json
 {
   "action": "route",
-  "message_id": "b11d5268",
+  "target_message_hash": "b11d5268",
   "downstream": {
     "model_id": "gpt-5.5",
     "tools": { "tools": ["workspace"] },
@@ -102,7 +102,7 @@ Example `route` result:
     "memory_retrieval_queries": { "queries": ["user code review preferences"] }
   },
   "audit": {
-    "ack_reply": { "reply": "Let me check." },
+    "ack_reply": { "text": "Let me check." },
     "routing": { "model_tier": "frontier_strong" },
     "model_specialization": { "specialization": "coding" },
     "tools": { "tools": ["workspace"] },
@@ -224,14 +224,13 @@ The resolver picks the cheapest model matching `specialization` and `tier`, rela
 - Open Classify keeps whole messages only, drops oldest first to fit a 5,000-char budget, and caps history at 20 messages.
 - Unknown fields are rejected, not passed through.
 
-## Local workbench
+## Local setup
 
 ```sh
 npm run setup
-npm run start
 ```
 
-UI opens at `http://127.0.0.1:4317/`. Classifier cards use classifier names from the runtime, displayed with underscores as spaces; result rendering remains generic.
+Checks prerequisites (Node, npm, Ollama), confirms the base model is pulled, installs dependencies, and builds. Idempotent — safe to re-run.
 
 Optional Ollama runtime config:
 
@@ -262,7 +261,7 @@ cp open-classify.config.example.json open-classify.config.json
 }
 ```
 
-`runner.provider` currently supports `"ollama"` only. `runner.defaultModel` applies to any classifier without an explicit entry. `runner.models.stock` configures built-in classifiers; `runner.models.custom` configures custom classifiers by manifest name. `aggregator.certaintyGate` can be `"min_score"` (lowest score across all stock and custom classifiers), `"avg_score"`, or `"off"`. The setup and start scripts read `open-classify.config.json`, or `OPEN_CLASSIFY_CONFIG` when you want a different path.
+`runner.provider` currently supports `"ollama"` only. `runner.defaultModel` applies to any classifier without an explicit entry. `runner.models.stock` configures built-in classifiers; `runner.models.custom` configures custom classifiers by manifest name. `aggregator.certaintyGate` can be `"min_score"` (lowest score across all stock and custom classifiers), `"avg_score"`, or `"off"`. The setup script and `loadOpenClassifyConfig()` read `open-classify.config.json`, or `OPEN_CLASSIFY_CONFIG` when you want a different path.
 
 ## Bring your own backend
 
@@ -300,10 +299,5 @@ For the lowest-level entry point, `classifyOpenClassifyInput(input, { runClassif
 ## Development
 
 ```sh
-npm test      # build + run the Node test runner suite
-npm run ui    # build + serve the local workbench
+npm test    # build + run the Node test runner suite
 ```
-
-## Screenshot
-
-![Open Classify local workbench](https://raw.githubusercontent.com/taylorbayouth/open-classify/main/open-classify-screenshot.png)

package/dist/src/classifiers/stock/prompts/preflight-output.md

@@ -1,10 +1,10 @@
 Emit one of these optional fields when applicable:
 
-- final_reply: {"reply":"..."} only for tiny terminal answers that need no downstream work.
+- final_reply: {"text":"..."} only for tiny terminal answers that need no downstream work.
   Do not use final_reply for drafting, rewriting, analysis, coding, research, or any generated work.
-  reply must be 200 characters or fewer.
-- ack_reply: {"reply":"..."} when downstream work should continue and a brief acknowledgement would help.
-  reply must be 200 characters or fewer.
+  text must be 200 characters or fewer.
+- ack_reply: {"text":"..."} when downstream work should continue and a brief acknowledgement would help.
+  text must be 200 characters or fewer.
 
 Omit both when the request is ambiguous or no acknowledgement is useful.
-Do not answer the user except inside final_reply.reply or ack_reply.reply.
+Do not answer the user except inside final_reply.text or ack_reply.text.

package/dist/src/classifiers/stock/prompts/preflight.md

@@ -8,35 +8,35 @@ Decide whether the target user message can be answered immediately with a tiny t
 
 Emit **at most one** of these fields:
 
-- `final_reply: {"reply":"..."}` - the reply text **is the complete answer to the user**. Nothing else happens after this. Use for tiny terminal answers like greetings, thanks, spelling, simple arithmetic, and similarly trivial replies.
-- `ack_reply: {"reply":"..."}` - a brief acknowledgement shown while downstream work continues. Use when the request needs generated work (drafting, analysis, coding, research) and a courtesy line helps. The reply must not contain the answer.
+- `final_reply: {"text":"..."}` - the reply text **is the complete answer to the user**. Nothing else happens after this. Use for tiny terminal answers like greetings, thanks, spelling, simple arithmetic, and similarly trivial replies.
+- `ack_reply: {"text":"..."}` - a brief acknowledgement shown while downstream work continues. Use when the request needs generated work (drafting, analysis, coding, research) and a courtesy line helps. The text must not contain the answer.
 
 Omit both fields when the request is ambiguous or no acknowledgement is useful.
 
 Both replies must be 200 characters or fewer.
-Do not address the user anywhere except inside `final_reply.reply` or `ack_reply.reply`.
+Do not address the user anywhere except inside `final_reply.text` or `ack_reply.text`.
 
 ## Examples
 
 User: `hi`
--> `{"reason":"Greeting.","certainty":"near_certain","final_reply":{"reply":"Hi!"}}`
+-> `{"reason":"Greeting.","certainty":"near_certain","final_reply":{"text":"Hi!"}}`
 Why: greeting needs no downstream model - the reply IS the answer.
 
 User: `thanks!`
--> `{"reason":"Closing acknowledgement.","certainty":"near_certain","final_reply":{"reply":"Anytime."}}`
+-> `{"reason":"Closing acknowledgement.","certainty":"near_certain","final_reply":{"text":"Anytime."}}`
 
 User: `what's 2 + 2?`
--> `{"reason":"Trivial arithmetic.","certainty":"very_strong","final_reply":{"reply":"4"}}`
+-> `{"reason":"Trivial arithmetic.","certainty":"very_strong","final_reply":{"text":"4"}}`
 
 User: `how do you spell necessary?`
--> `{"reason":"Spelling lookup.","certainty":"very_strong","final_reply":{"reply":"necessary"}}`
+-> `{"reason":"Spelling lookup.","certainty":"very_strong","final_reply":{"text":"necessary"}}`
 
 User: `draft an email apologizing to the team for the missed deadline`
--> `{"reason":"Generated writing task.","certainty":"very_strong","ack_reply":{"reply":"On it."}}`
+-> `{"reason":"Generated writing task.","certainty":"very_strong","ack_reply":{"text":"On it."}}`
 Why: the request needs drafted prose. `final_reply` would skip the actual work.
 
 User: `review the routing code in this repo`
--> `{"reason":"Needs code analysis.","certainty":"very_strong","ack_reply":{"reply":"Let me check."}}`
+-> `{"reason":"Needs code analysis.","certainty":"very_strong","ack_reply":{"text":"Let me check."}}`
 
 User: `what should I do about the contract?`
 -> `{"reason":"Ambiguous; needs downstream model.","certainty":"strong"}`

package/dist/src/classifiers.js

@@ -62,17 +62,14 @@ function loadClassifierManifest(classifierDir, expectedKind) {
     return { ...manifest, systemPrompt };
 }
 function validateRegistry(manifests) {
+    // Duplicate orders are allowed: same-order classifiers schedule adjacent
+    // and run in parallel when concurrency permits, sequentially otherwise.
     const names = new Set();
-    const orders = new Set();
     for (const manifest of manifests) {
         if (names.has(manifest.name)) {
             throw new ClassifierManifestError(`duplicate classifier name: ${manifest.name}`);
         }
         names.add(manifest.name);
-        if (orders.has(manifest.order)) {
-            throw new ClassifierManifestError(`duplicate classifier order: ${manifest.order}`);
-        }
-        orders.add(manifest.order);
     }
 }
 export const REGISTRY = loadClassifierRegistry();

package/dist/src/classify.d.ts

@@ -17,6 +17,7 @@ export interface CreateClassifierOptions {
     fetch?: typeof fetch;
     classifierTimeoutMs?: number;
     classifierRetryCount?: number;
+    maxConcurrency?: number;
     aggregator?: AggregatorConfig;
 }
 export declare function createClassifier(options?: CreateClassifierOptions): Classifier;

package/dist/src/classify.js

@@ -43,6 +43,7 @@ export function createClassifier(options = {}) {
             catalog,
             classifierTimeoutMs: options.classifierTimeoutMs,
             classifierRetryCount: options.classifierRetryCount,
+            maxConcurrency: options.maxConcurrency,
             aggregator,
             signal: callOptions?.signal,
         });

package/dist/src/config.js

@@ -1,6 +1,5 @@
 import { existsSync, readFileSync } from "node:fs";
 import { REGISTRY } from "./classifiers.js";
-import { CERTAINTY_GATE_MODES, } from "./manifest.js";
 import { STOCK_CLASSIFIER_NAMES } from "./stock.js";
 import { isRecord } from "./validation.js";
 export const DEFAULT_OPEN_CLASSIFY_CONFIG_PATH = "open-classify.config.json";
@@ -49,7 +48,7 @@ function validateAggregator(value, path) {
     if (!isRecord(value)) {
         throwConfig(path, "aggregator must be an object");
     }
-    ensureAllowedKeys(value, ["certaintyThreshold", "confidenceThreshold", "certaintyGate"], path, "aggregator");
+    ensureAllowedKeys(value, ["certaintyThreshold", "confidenceThreshold"], path, "aggregator");
     return {
         ...(value.certaintyThreshold === undefined
             ? {}
@@ -57,9 +56,6 @@ function validateAggregator(value, path) {
         ...(value.confidenceThreshold === undefined
             ? {}
             : { confidenceThreshold: requireUnitFloat(value.confidenceThreshold, path, "aggregator.confidenceThreshold") }),
-        ...(value.certaintyGate === undefined
-            ? {}
-            : { certaintyGate: requireCertaintyGateMode(value.certaintyGate, path, "aggregator.certaintyGate") }),
     };
 }
 function validateRunner(value, path) {
@@ -157,12 +153,6 @@ function requireUnitFloat(value, path, field) {
     }
     return number;
 }
-function requireCertaintyGateMode(value, path, field) {
-    if (typeof value !== "string" || !CERTAINTY_GATE_MODES.includes(value)) {
-        throwConfig(path, `${field} must be one of ${CERTAINTY_GATE_MODES.join(", ")}`);
-    }
-    return value;
-}
 function ensureAllowedKeys(value, allowedKeys, path, field) {
     const allowed = new Set(allowedKeys);
     for (const key of Object.keys(value)) {

package/dist/src/manifest.d.ts

@@ -3,8 +3,6 @@ import type { ClassifierInput, ClassifierRunStatus } from "./types.js";
 import type { DownstreamModelTier, ModelSpecialization } from "./enums.js";
 export type ClassifierName = string;
 export type ClassifierResults = Record<ClassifierName, ClassifierOutput>;
-export declare const CERTAINTY_GATE_MODES: readonly ["min_score", "avg_score", "off"];
-export type CertaintyGateMode = (typeof CERTAINTY_GATE_MODES)[number];
 export type RunClassifier = (name: ClassifierName, input: ClassifierInput, signal: AbortSignal) => Promise<ClassifierOutput>;
 export interface CatalogEntry {
     readonly id: string;
@@ -67,57 +65,27 @@ export type ClassifierEntry = ClassifierOutput & {
     readonly status: ClassifierRunStatus;
     readonly version: string;
 };
+export interface CertaintySummary {
+    readonly min: number;
+    readonly avg: number;
+}
 export interface PipelineMeta {
     readonly classifiers: Record<string, ClassifierEntry>;
+    readonly certainty: CertaintySummary;
 }
 export interface PipelineAudit extends Envelope {
     readonly meta: PipelineMeta;
-    readonly fired_by?: string;
-    readonly certainty_gate?: LowCertaintyBlockReason;
-}
-export type BlockReason = PromptInjectionBlockReason | LowCertaintyBlockReason;
-export interface PromptInjectionBlockReason {
-    readonly kind: "prompt_injection";
-    readonly risk_level: PromptInjectionSignal["risk_level"];
-}
-export interface LowCertaintyBlockReason {
-    readonly kind: "low_certainty";
-    readonly mode: Exclude<CertaintyGateMode, "off">;
-    readonly threshold: number;
-    readonly score: number;
-    readonly classifier_scores: Readonly<Record<string, number>>;
-    readonly low_classifiers: ReadonlyArray<string>;
 }
-export type ReplyPipelineResult = {
-    readonly action: "reply";
-    readonly message_id: string;
-    readonly reply: {
-        readonly text: string;
-    };
-    readonly reason: "preflight_reply";
-    readonly classifier_outputs: ClassifierCustomOutputs;
-    readonly audit: Pick<PipelineAudit, "final_reply" | "meta" | "fired_by">;
-};
-export type BlockPipelineResult = {
-    readonly action: "block";
-    readonly message_id: string;
-    readonly fired_by?: string;
-    readonly reason: BlockReason;
-    readonly classifier_outputs: ClassifierCustomOutputs;
-    readonly audit: Pick<PipelineAudit, "prompt_injection" | "meta" | "fired_by" | "certainty_gate">;
-};
-export type RoutePipelineResult = {
+export interface PipelineResult {
     readonly action: "route";
-    readonly message_id: string;
+    readonly target_message_hash: string;
     readonly downstream: DownstreamPayload;
     readonly classifier_outputs: ClassifierCustomOutputs;
     readonly audit: PipelineAudit;
-};
-export type PipelineResult = ReplyPipelineResult | BlockPipelineResult | RoutePipelineResult;
+}
 export interface AggregatorConfig {
     readonly certaintyThreshold?: number;
     /** @deprecated Use certaintyThreshold. */
     readonly confidenceThreshold?: number;
-    readonly certaintyGate?: CertaintyGateMode;
 }
 export type ClassifierRegistry = ReadonlyArray<RuntimeClassifierManifest>;

package/dist/src/manifest.js

@@ -1,5 +1 @@
-export const CERTAINTY_GATE_MODES = [
-    "min_score",
-    "avg_score",
-    "off",
-];
+export {};

package/dist/src/pipeline.d.ts

@@ -3,7 +3,7 @@ import type { AggregatorConfig, Catalog, PipelineResult } from "./manifest.js";
 import type { OpenClassifyInput } from "./types.js";
 export declare const DEFAULT_CLASSIFIER_TIMEOUT_MS = 15000;
 export declare const DEFAULT_CLASSIFIER_RETRY_COUNT = 1;
-export declare const DEFAULT_CERTAINTY_GATE = "min_score";
+export declare const DEFAULT_MAX_CONCURRENCY = 7;
 export declare class OpenClassifyNormalizationError extends Error {
     constructor(cause: unknown);
 }
@@ -12,6 +12,7 @@ export interface ClassifyOptions {
     catalog: Catalog;
     classifierTimeoutMs?: number;
     classifierRetryCount?: number;
+    maxConcurrency?: number;
     aggregator?: AggregatorConfig;
     signal?: AbortSignal;
 }

package/dist/src/pipeline.js

@@ -1,21 +1,16 @@
-import { certaintyThreshold, composeEnvelope } from "./aggregator.js";
-import { CLASSIFIER_NAMES, MODULES_BY_NAME, REGISTRY, } from "./classifiers.js";
+import { composeEnvelope } from "./aggregator.js";
+import { MODULES_BY_NAME, REGISTRY, } from "./classifiers.js";
 import { normalizeOpenClassifyInput, toClassifierInput } from "./input.js";
 import { certaintyScore, isCustomManifest } from "./stock.js";
 export const DEFAULT_CLASSIFIER_TIMEOUT_MS = 15_000;
 export const DEFAULT_CLASSIFIER_RETRY_COUNT = 1;
-export const DEFAULT_CERTAINTY_GATE = "min_score";
+export const DEFAULT_MAX_CONCURRENCY = 7;
 export class OpenClassifyNormalizationError extends Error {
     constructor(cause) {
         super(errorMessage(cause), { cause });
         this.name = "OpenClassifyNormalizationError";
     }
 }
-// Short-circuit gates are intrinsic to specific stock signals — not configured
-// per-manifest. preflight.final_reply ⇒ reply; confident high_risk or unknown
-// prompt-injection risk ⇒ block. Order matters: preflight is
-// cheaper to evaluate, so we check it first.
-const SHORT_CIRCUIT_GATES = ["preflight", "prompt_injection"];
 export async function classifyOpenClassifyInput(input, options) {
     let request;
     try {
@@ -37,27 +32,13 @@ export async function classifyOpenClassifyInput(input, options) {
     const classifierInput = toClassifierInput(request);
     const classifierTimeoutMs = options.classifierTimeoutMs ?? DEFAULT_CLASSIFIER_TIMEOUT_MS;
     const classifierRetryCount = options.classifierRetryCount ?? DEFAULT_CLASSIFIER_RETRY_COUNT;
-    const threshold = certaintyThreshold(options.aggregator);
-    const runs = new Map(CLASSIFIER_NAMES.map((name) => [
-        name,
-        runClassifierWithRetry(name, classifierInput, options.runClassifier, controller.signal, classifierTimeoutMs, classifierRetryCount),
-    ]));
+    const maxConcurrency = resolveMaxConcurrency(options.maxConcurrency);
+    // REGISTRY is already sorted by `order` ascending (see classifiers.ts).
+    // The worker pool dispatches in array order, so classifiers with the same
+    // order are scheduled adjacent and run together when slots are free.
+    const queue = REGISTRY.map((m) => m.name);
     try {
-        for (const gate of SHORT_CIRCUIT_GATES) {
-            const gateRun = runs.get(gate);
-            if (gateRun === undefined)
-                continue;
-            const settled = await gateRun;
-            if (!settled.ok)
-                continue;
-            const verdict = shortCircuitVerdict(gate, settled.value, threshold);
-            if (!verdict)
-                continue;
-            controller.abort();
-            await settleClassifierRunsExcept(runs, [gate]);
-            return buildShortCircuitResult(gate, verdict, settled, request.target_message_hash);
-        }
-        const settled = await Promise.all([...runs.values()]);
+        const settled = await runWithConcurrency(queue, maxConcurrency, controller.signal, (name) => runClassifierWithRetry(name, classifierInput, options.runClassifier, controller.signal, classifierTimeoutMs, classifierRetryCount));
         const { results, meta } = collectFullEntries(settled);
         const envelope = composeEnvelope({
             registry: REGISTRY,
@@ -66,117 +47,48 @@ export async function classifyOpenClassifyInput(input, options) {
             input: classifierInput,
             config: options.aggregator,
         });
-        const certaintyGate = certaintyGateBlock(options.aggregator, results);
-        if (certaintyGate) {
-            return buildCertaintyGateBlockResult(request, envelope, results, meta, certaintyGate);
-        }
         return buildRouteResult(request, envelope, results, meta);
     }
     finally {
         options.signal?.removeEventListener("abort", abortFromOptions);
     }
 }
-function shortCircuitVerdict(gate, result, threshold) {
-    const score = scoreCertainty(result.certainty);
-    if (score < threshold)
-        return null;
-    if (gate === "preflight") {
-        const preflight = result;
-        if (preflight.final_reply !== undefined) {
-            return { kind: "reply", final_reply: preflight.final_reply };
-        }
-        return null;
+function resolveMaxConcurrency(value) {
+    if (value === undefined)
+        return DEFAULT_MAX_CONCURRENCY;
+    if (!Number.isFinite(value) || value < 1) {
+        throw new RangeError(`maxConcurrency must be a positive integer; received ${value}`);
     }
-    if (gate === "prompt_injection") {
-        const promptInjection = result;
-        if (promptInjection.risk_level === "high_risk" || promptInjection.risk_level === "unknown") {
-            const promptInjectionSignal = extractPromptInjection(promptInjection);
-            return {
-                kind: "block",
-                prompt_injection: promptInjectionSignal,
-                reason: {
-                    kind: "prompt_injection",
-                    risk_level: promptInjectionSignal.risk_level,
-                },
-            };
+    return Math.floor(value);
+}
+async function runWithConcurrency(names, maxConcurrency, signal, start) {
+    const results = new Array(names.length);
+    let next = 0;
+    const worker = async () => {
+        while (true) {
+            const i = next;
+            next += 1;
+            if (i >= names.length)
+                return;
+            const name = names[i];
+            if (signal.aborted) {
+                // Queued classifiers that never started are reported as not-run so
+                // the audit shows their fallback in `meta.classifiers`. In-flight
+                // classifiers receive the abort signal directly and resolve normally.
+                results[i] = {
+                    ok: false,
+                    name,
+                    error: signal.reason ?? new Error(`${name} classifier aborted before start`),
+                    reason: "error",
+                };
+                continue;
+            }
+            results[i] = await start(name);
         }
-    }
-    return null;
-}
-function certaintyGateBlock(config, results) {
-    const mode = config?.certaintyGate ?? DEFAULT_CERTAINTY_GATE;
-    if (mode === "off")
-        return undefined;
-    const threshold = certaintyThreshold(config);
-    const classifier_scores = classifierScores(results);
-    const scores = Object.values(classifier_scores);
-    const score = mode === "min_score"
-        ? Math.min(...scores)
-        : scores.reduce((sum, value) => sum + value, 0) / scores.length;
-    if (score >= threshold)
-        return undefined;
-    return {
-        kind: "low_certainty",
-        mode,
-        threshold,
-        score,
-        classifier_scores,
-        low_classifiers: Object.entries(classifier_scores)
-            .filter(([, value]) => value < threshold)
-            .map(([name]) => name),
-    };
-}
-function classifierScores(results) {
-    return Object.fromEntries(REGISTRY.map((manifest) => [
-        manifest.name,
-        scoreCertainty(results[manifest.name]?.certainty),
-    ]));
-}
-function scoreCertainty(certainty) {
-    return certainty === undefined ? 0 : certaintyScore[certainty];
-}
-function extractPromptInjection(value) {
-    return {
-        risk_level: value.risk_level,
-    };
-}
-function buildShortCircuitResult(name, verdict, settled, target_message_hash) {
-    const manifest = MODULES_BY_NAME[name];
-    const value = settled.ok ? settled.value : manifest.fallback;
-    const entry = {
-        ...value,
-        status: classifierRunStatus(settled),
-        version: manifest.version,
-    };
-    const meta = { classifiers: { [name]: entry } };
-    const classifier_outputs = classifierCustomOutputs({ [name]: value });
-    if (verdict.kind === "reply") {
-        const preflight = value;
-        return {
-            action: "reply",
-            message_id: target_message_hash,
-            reply: { text: verdict.final_reply.reply },
-            reason: "preflight_reply",
-            classifier_outputs,
-            audit: {
-                fired_by: name,
-                ...(preflight.final_reply === undefined ? {} : { final_reply: preflight.final_reply }),
-                meta,
-            },
-        };
-    }
-    return {
-        action: "block",
-        message_id: target_message_hash,
-        fired_by: name,
-        reason: verdict.reason,
-        classifier_outputs,
-        audit: {
-            fired_by: name,
-            prompt_injection: verdict.prompt_injection,
-            meta,
-        },
     };
+    const workerCount = Math.max(1, Math.min(maxConcurrency, names.length));
+    await Promise.all(Array.from({ length: workerCount }, () => worker()));
+    return results;
 }
 function collectFullEntries(settled) {
     const results = {};
@@ -191,7 +103,18 @@ function collectFullEntries(settled) {
             version: manifest.version,
         };
     }
-    return { results, meta: { classifiers } };
+    return { results, meta: { classifiers, certainty: certaintySummary(results) } };
+}
+function certaintySummary(results) {
+    const scores = REGISTRY.map((m) => scoreCertainty(results[m.name]?.certainty));
+    if (scores.length === 0)
+        return { min: 0, avg: 0 };
+    const min = Math.min(...scores);
+    const avg = scores.reduce((sum, v) => sum + v, 0) / scores.length;
+    return { min, avg };
+}
+function scoreCertainty(certainty) {
+    return certainty === undefined ? 0 : certaintyScore[certainty];
 }
 function buildRouteResult(request, envelope, results, meta) {
     const downstream = {
@@ -205,7 +128,7 @@ function buildRouteResult(request, envelope, results, meta) {
     };
     return {
         action: "route",
-        message_id: request.target_message_hash,
+        target_message_hash: request.target_message_hash,
         downstream,
         classifier_outputs: classifierCustomOutputs(results),
         audit: {
@@ -214,21 +137,6 @@ function buildRouteResult(request, envelope, results, meta) {
         },
     };
 }
-function buildCertaintyGateBlockResult(request, envelope, results, meta, certaintyGate) {
-    return {
-        action: "block",
-        message_id: request.target_message_hash,
-        fired_by: "certainty_gate",
-        reason: certaintyGate,
-        classifier_outputs: classifierCustomOutputs(results),
-        audit: {
-            ...envelope,
-            fired_by: "certainty_gate",
-            certainty_gate: certaintyGate,
-            meta,
-        },
-    };
-}
 function classifierCustomOutputs(results) {
     const out = {};
     for (const manifest of REGISTRY) {
@@ -285,10 +193,6 @@ async function runClassifierAttempt(name, input, runClassifier, rootSignal, time
             rootSignal.removeEventListener("abort", abortAttempt);
     }
 }
-async function settleClassifierRunsExcept(runs, keep) {
-    const keepSet = new Set(keep);
-    await Promise.all([...runs].filter(([name]) => !keepSet.has(name)).map(([, run]) => run.catch(() => undefined)));
-}
 function classifierRunStatus(settled) {
     if (settled.ok)
         return { ok: true, source: "model" };

package/dist/src/stock-validation.js

@@ -154,15 +154,15 @@ function validateReplySignal(value, classifier, model, field) {
     if (!isRecord(value)) {
         throwInvalid(classifier, model, `${field} must be an object`);
     }
-    ensureAllowedObjectKeys(value, ["reply"], classifier, model, field);
-    const reply = requireString(value.reply, classifier, model, `${field}.reply`);
-    if (reply.trim().length === 0) {
-        throwInvalid(classifier, model, `${field}.reply must not be empty`);
+    ensureAllowedObjectKeys(value, ["text"], classifier, model, field);
+    const text = requireString(value.text, classifier, model, `${field}.text`);
+    if (text.trim().length === 0) {
+        throwInvalid(classifier, model, `${field}.text must not be empty`);
     }
-    if (reply.length > STOCK_REPLY_MAX_CHARS) {
-        throwInvalid(classifier, model, `${field}.reply must be ${STOCK_REPLY_MAX_CHARS} characters or fewer`);
+    if (text.length > STOCK_REPLY_MAX_CHARS) {
+        throwInvalid(classifier, model, `${field}.text must be ${STOCK_REPLY_MAX_CHARS} characters or fewer`);
     }
-    return { reply };
+    return { text };
 }
 function validateTierRoutingOutput(value, model) {
     ensureAllowedObjectKeys(value, ["reason", "certainty", "model_tier"], "routing", model, "output");

package/dist/src/stock.d.ts

@@ -7,10 +7,10 @@ export interface StockClassifierInput {
     readonly messages: ReadonlyArray<StockClassifierMessageInput>;
 }
 export interface FinalReplySignal {
-    readonly reply: string;
+    readonly text: string;
 }
 export interface AckReplySignal {
-    readonly reply: string;
+    readonly text: string;
 }
 export interface RoutingSignal {
     readonly model_tier?: DownstreamModelTier;

package/open-classify.config.example.json

@@ -21,8 +21,7 @@
     }
   },
   "aggregator": {
-    "certaintyThreshold": 0.65,
-    "certaintyGate": "min_score"
+    "certaintyThreshold": 0.65
   },
   "catalog": "downstream-models.json"
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-classify",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "Manifest-driven classifier runtime for routing user messages to downstream AI models",
   "license": "MIT",
   "author": "Taylor Bayouth",
@@ -43,9 +43,7 @@
   "scripts": {
     "build": "node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\" && tsc && node scripts/copy-classifier-assets.mjs",
     "setup": "node scripts/setup.mjs",
-    "start": "node scripts/start.mjs",
     "test": "npm run build && node --test tests/*.test.mjs",
-    "ui": "npm run build && node dist/src/ui-server.js",
     "prepublishOnly": "npm run build && npm test"
   },
   "devDependencies": {

package/dist/src/ui-server.d.ts

@@ -1 +0,0 @@
-export {};

package/dist/src/ui-server.js

@@ -1,257 +0,0 @@
-// A tiny dev/demo HTTP server backing the bundled UI. Two responsibilities:
-//   1. Serve the static UI from `./ui` (HTML, CSS, JS).
-//   2. Run a classification over Server-Sent Events at /api/classify-stream.
-//
-// The SSE event vocabulary the UI listens for:
-//   pipeline_started        — pipeline boot, includes the classifier list
-//   pipeline_phase          — coarse phase ("normalizing" / "resource_check" /
-//                             "running"); useful for progress UI
-//   classifier_started      — a specific classifier is now running
-//   classifier_completed    — that classifier returned a model result
-//   classifier_failed       — that classifier threw without being aborted
-//   classifier_aborted      — early-exit short-circuit cancelled this classifier
-//   classifier_timed_out    — the per-classifier timeout fired
-//   pipeline_completed      — final PipelineResult payload
-//   pipeline_failed         — pipeline-level error (normalization, etc.)
-//
-// This server is intentionally minimal — no auth, no rate limiting, binds to
-// 127.0.0.1 by default. It is not meant for production.
-import { createReadStream, existsSync } from "node:fs";
-import { createServer } from "node:http";
-import { extname, join, normalize } from "node:path";
-import { loadCatalog } from "./catalog.js";
-import { CLASSIFIER_NAMES, REGISTRY } from "./classifiers.js";
-import { DEFAULT_CERTAINTY_THRESHOLD, certaintyThreshold, } from "./aggregator.js";
-import { classifierModelsFromConfig, loadOpenClassifyConfig, } from "./config.js";
-import { DEFAULT_CERTAINTY_GATE } from "./pipeline.js";
-import { DOWNSTREAM_MODEL_TIER_VALUES, MODEL_SPECIALIZATION_VALUES, PROMPT_INJECTION_RISK_LEVEL_VALUES, } from "./enums.js";
-import { createOllamaClassifierRunner, OLLAMA_CONTEXT_LENGTH, OLLAMA_DEFAULT_CATALOG_PATH, OLLAMA_MIN_AVAILABLE_MEMORY_BYTES, OLLAMA_MIN_TOTAL_MEMORY_BYTES, OLLAMA_REQUIRED_PARALLELISM, } from "./ollama.js";
-import { classifyOpenClassifyInput } from "./pipeline.js";
-// Served at GET /api/enums so the UI never needs to duplicate shared enum values.
-const CLASSIFIER_ENUMS = {
-    downstream_model_tier: [...DOWNSTREAM_MODEL_TIER_VALUES],
-    model_specialization: [...MODEL_SPECIALIZATION_VALUES],
-    prompt_injection_risk_level: [...PROMPT_INJECTION_RISK_LEVEL_VALUES],
-};
-const CLASSIFIER_METADATA = REGISTRY.map((classifier) => ({
-    name: classifier.name,
-    kind: classifier.kind,
-    version: classifier.version,
-    purpose: classifier.purpose,
-    order: classifier.order,
-    ...("tools" in classifier ? { tools: classifier.tools ?? [] } : {}),
-}));
-const PORT = Number(process.env.OPEN_CLASSIFY_UI_PORT ?? 4317);
-const HOST = process.env.OPEN_CLASSIFY_UI_HOST ?? "127.0.0.1";
-const UI_DIR = join(process.cwd(), "ui");
-const OPEN_CLASSIFY_CONFIG = loadOpenClassifyConfig(undefined, {
-    optional: process.env.OPEN_CLASSIFY_CONFIG === undefined,
-});
-const CATALOG_PATH = process.env.OPEN_CLASSIFY_CATALOG_PATH ??
-    OPEN_CLASSIFY_CONFIG?.catalog ??
-    OLLAMA_DEFAULT_CATALOG_PATH;
-const MIME_TYPES = {
-    ".html": "text/html; charset=utf-8",
-    ".css": "text/css; charset=utf-8",
-    ".js": "text/javascript; charset=utf-8",
-    ".json": "application/json; charset=utf-8",
-};
-const server = createServer((request, response) => {
-    void route(request, response);
-});
-server.listen(PORT, HOST, () => {
-    console.log(`Open Classify UI running at http://${HOST}:${PORT}/`);
-});
-async function route(request, response) {
-    const startedAt = Date.now();
-    console.log(`[req] ${request.method} ${request.url}`);
-    try {
-        const url = new URL(request.url ?? "/", `http://${request.headers.host ?? "localhost"}`);
-        if (request.method === "POST" && url.pathname === "/api/classify-stream") {
-            await classifyStream(request, response);
-            console.log(`[req] ${request.method} ${request.url} stream ended in ${Date.now() - startedAt}ms`);
-            return;
-        }
-        if (request.method === "GET" && url.pathname === "/api/enums") {
-            sendJson(response, CLASSIFIER_ENUMS);
-            return;
-        }
-        if (request.method === "GET" && url.pathname === "/api/classifiers") {
-            sendJson(response, {
-                classifiers: CLASSIFIER_METADATA,
-                aggregator: {
-                    certaintyGate: OPEN_CLASSIFY_CONFIG?.aggregator?.certaintyGate ?? DEFAULT_CERTAINTY_GATE,
-                    certaintyThreshold: certaintyThreshold(OPEN_CLASSIFY_CONFIG?.aggregator) ?? DEFAULT_CERTAINTY_THRESHOLD,
-                },
-            });
-            return;
-        }
-        if (request.method === "GET") {
-            serveStatic(url.pathname, response);
-            return;
-        }
-        sendJson(response, { error: "method not allowed" }, 405);
-    }
-    catch (error) {
-        console.error(`[req] ${request.method} ${request.url} failed:`, error);
-        sendJson(response, { error: errorMessage(error) }, 500);
-    }
-}
-async function classifyStream(request, response) {
-    response.writeHead(200, {
-        "content-type": "text/event-stream; charset=utf-8",
-        "cache-control": "no-cache, no-transform",
-        connection: "keep-alive",
-        "x-accel-buffering": "no",
-    });
-    response.flushHeaders();
-    // Disable Nagle so each event flushes immediately. SSE is interactive;
-    // batching kills the "live" feel.
-    request.socket.setNoDelay(true);
-    let closed = false;
-    const clientAbortController = new AbortController();
-    const abortForClientClose = () => {
-        closed = true;
-        clientAbortController.abort(new Error("SSE client disconnected"));
-    };
-    response.on("close", () => {
-        abortForClientClose();
-    });
-    response.on("error", () => {
-        abortForClientClose();
-    });
-    const send = (event, data) => {
-        if (closed || response.writableEnded || response.destroyed) {
-            console.warn(`[sse] dropped ${event} (closed=${closed} ended=${response.writableEnded})`);
-            return;
-        }
-        const ok = response.write(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
-        console.log(`[sse] -> ${event}${data?.name ? ` ${data.name}` : ""}${ok ? "" : " [backpressure]"}`);
-    };
-    // SSE comment heartbeat. Some intermediaries (proxies, load balancers)
-    // close idle connections; a tiny ping every 5s keeps the stream warm.
-    // The leading `:` makes browsers ignore the line as a comment.
-    const heartbeat = setInterval(() => {
-        if (closed || response.writableEnded || response.destroyed) {
-            return;
-        }
-        response.write(`: ping ${Date.now()}\n\n`);
-    }, 5000);
-    try {
-        const input = (await readJsonBody(request));
-        const baseRunner = createOllamaClassifierRunner({
-            host: OPEN_CLASSIFY_CONFIG?.runner?.host,
-            defaultModel: OPEN_CLASSIFY_CONFIG?.runner?.defaultModel,
-            models: classifierModelsFromConfig(OPEN_CLASSIFY_CONFIG),
-            options: OPEN_CLASSIFY_CONFIG?.runner?.options,
-        });
-        const runClassifier = async (name, classifierInput, signal) => {
-            send("classifier_started", { name, started_at: Date.now() });
-            try {
-                const result = await baseRunner(name, classifierInput, signal);
-                send("classifier_completed", { name, result, completed_at: Date.now() });
-                return result;
-            }
-            catch (error) {
-                console.error(`[classifier] ${name} threw:`, error);
-                if (signal.aborted) {
-                    send(isTimeoutAbort(name, signal) ? "classifier_timed_out" : "classifier_aborted", {
-                        name,
-                        reason: errorMessage(signal.reason ?? error),
-                        completed_at: Date.now(),
-                    });
-                }
-                else {
-                    send("classifier_failed", {
-                        name,
-                        error: errorMessage(error),
-                        completed_at: Date.now(),
-                    });
-                }
-                throw error;
-            }
-        };
-        send("pipeline_started", {
-            classifiers: CLASSIFIER_NAMES,
-            started_at: Date.now(),
-        });
-        send("pipeline_phase", { phase: "normalizing" });
-        send("pipeline_phase", {
-            phase: "resource_check",
-            required_parallelism: OLLAMA_REQUIRED_PARALLELISM,
-            context_length: OLLAMA_CONTEXT_LENGTH,
-            min_total_memory_bytes: OLLAMA_MIN_TOTAL_MEMORY_BYTES,
-            min_available_memory_bytes: OLLAMA_MIN_AVAILABLE_MEMORY_BYTES,
-        });
-        send("pipeline_phase", { phase: "running" });
-        const result = await classifyOpenClassifyInput(input, {
-            runClassifier,
-            catalog: loadCatalog(CATALOG_PATH),
-            aggregator: OPEN_CLASSIFY_CONFIG?.aggregator,
-            signal: clientAbortController.signal,
-        });
-        send("pipeline_completed", result);
-    }
-    catch (error) {
-        console.error("[pipeline] failed:", error);
-        send("pipeline_failed", { error: errorMessage(error) });
-    }
-    finally {
-        clearInterval(heartbeat);
-        closed = true;
-        if (!response.writableEnded && !response.destroyed) {
-            response.end();
-        }
-    }
-}
-// Distinguishes a timeout-driven abort from a pipeline early-exit abort, so
-// the UI can show the right state. We sniff the abort reason's message
-// because that's the only signal the pipeline gives us — it doesn't tag
-// reasons with a structured discriminator.
-function isTimeoutAbort(name, signal) {
-    return errorMessage(signal.reason).includes(`${name} classifier timed out`);
-}
-function serveStatic(pathname, response) {
-    const requestedPath = pathname === "/" ? "/index.html" : pathname;
-    // Two-layer path-traversal guard: strip leading `../` segments from the
-    // normalized path, then double-check the resolved file is still inside
-    // UI_DIR. The redundancy is intentional — defense in depth on a static
-    // file server is cheap.
-    const safePath = normalize(requestedPath).replace(/^(\.\.[/\\])+/, "");
-    const filePath = join(UI_DIR, safePath);
-    if (!filePath.startsWith(UI_DIR) || !existsSync(filePath)) {
-        sendJson(response, { error: "not found" }, 404);
-        return;
-    }
-    response.writeHead(200, {
-        "content-type": MIME_TYPES[extname(filePath)] ?? "application/octet-stream",
-        "cache-control": "no-store",
-    });
-    createReadStream(filePath).on("error", () => response.destroy()).pipe(response);
-}
-function sendJson(response, data, status = 200) {
-    response.writeHead(status, { "content-type": "application/json; charset=utf-8" });
-    response.end(JSON.stringify(data));
-}
-// 512 KiB cap matches the input contract (5,000-char message budget plus
-// generous slack for history). Big enough for any legitimate
-// classification request, small enough to not be a DoS vector.
-async function readJsonBody(request) {
-    const chunks = [];
-    let size = 0;
-    for await (const chunk of request) {
-        const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
-        size += buffer.byteLength;
-        if (size > 512 * 1024) {
-            throw new Error("request body is too large");
-        }
-        chunks.push(buffer);
-    }
-    return JSON.parse(Buffer.concat(chunks).toString("utf8"));
-}
-function errorMessage(error) {
-    if (error instanceof Error) {
-        return error.message;
-    }
-    return String(error);
-}