ai-trust-score 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,149 @@
+# ai-trust-score — deterministic, local validator for LLM outputs
+
+ai-trust-score helps teams detect and block common, deterministic problems in model-generated text and structured outputs. It's designed to run locally (no external APIs), in CI pipelines, or inside backend services that produce or relay LLM responses. The goal is to give you small, auditable reports that help automate gating, auditing, and monitoring of LLM outputs.
+
+This README is intentionally comprehensive: quickstart, programmatic examples (CommonJS and ESM), how to apply ai-trust-score to API responses, CLI/batch usage, an illustrative report, configuration options, recommended policies, and contribution notes.
+
+Why ai-trust-score
+
+- Deterministic and auditable — no external black-box calls.
+- Low operational cost — runs on your infrastructure.
+- Extensible — add domain-specific JSON rule packs or custom detectors.
+
+What ai-trust-score checks (examples)
+
+- Schema validation: verify JSON outputs conform to your schema.
+- Numeric consistency: flag mismatched percentages, impossible arithmetic, or inconsistent ranges.
+- Hallucination heuristics: detect claims that appear fabricated or unverifiable.
+- Overconfidence: detect absolute or sweeping claims presented without evidence.
+- Simple contradiction checks: find sentence-level contradictions within one output.
+
+Quick install
+
+```bash
+npm install ai-trust-score
+# or
+yarn add ai-trust-score
+```
+
+Programmatic usage — CommonJS (server-side)
+
+```js
+// Import the validator and run it on a single string output
+const { validateLLM } = require('ai-trust-score');
+
+const text = 'The product revenue grew 20% from 100 to 150.';
+const report = validateLLM(text, {
+  detectors: { numericConsistency: true, overconfidence: true, hallucination: true },
+  // customRules: { /* optional domain-specific rules */ }
+});
+
+console.log(report.score);   // 0..100
+console.log(report.issues);  // array of issues
+```
+
+Programmatic usage — ESM / TypeScript
+
+```ts
+import { validateLLM } from 'ai-trust-score';
+
+const report = validateLLM('The capital of France is Berlin.', { detectors: { hallucination: true } });
+console.log(report);
+```
+
+Applying ai-trust-score to API responses (recommended patterns)
+
+Inline validation (explicit)
+
+```js
+const modelOutput = await myLLM.generate(prompt);
+const report = validateLLM(modelOutput);
+if (report.score < 75) {
+  // return a safe fallback, request regeneration, or present a human review flag
+}
+```
+
+Middleware pattern (convenience)
+
+```js
+import express from 'express';
+import { llmGuardMiddleware } from 'ai-trust-score';
+
+const app = express();
+app.use(express.json());
+
+app.post('/generate', llmGuardMiddleware({ threshold: 80 }), (req, res) => {
+  const { allowed, report, output } = res.locals;
+  if (!allowed) return res.status(422).json({ error: 'Blocked by ai-trust-score', report });
+  res.json({ reply: output, report });
+});
+```
+
+Policy notes
+
+- Thresholds are organizational: 80 is a sensible starting point. Use higher thresholds in high-risk domains.
+- Instead of outright blocking, consider fallback behaviors: regenerate, lower-risk response, or human review.
+
+CLI & batch usage
+
+The package exposes a small CLI for ad-hoc checks and a batch mode for audits. Use the published package via `npx ai-trust-score` or install it locally.
+
+```bash
+# Human-friendly table
+npx ai-trust-score check --file path/to/output.txt
+
+# Machine-readable JSON
+npx ai-trust-score check --file path/to/output.txt --json
+
+# Batch audit (JSONL -> results + HTML summary)
+npx ai-trust-score batch --file samples.jsonl --out results.jsonl --parallel 8 --html report.html
+```
+
+Illustration — sample GuardReport
+
+Calling `validateLLM(text, config)` returns a `GuardReport` object. Example:
+
+```json
+{
+  "score": 92,
+  "issues": [
+    {
+      "detector": "numeric-consistency",
+      "severity": "medium",
+      "message": "20% inconsistent with increase from 100 to 150 (~50%).",
+      "meta": { "found": "20%", "expectedApprox": "50%", "evidence": "increase from 100 to 150" }
+    }
+  ],
+  "summary": "Detected 1 issue(s). Trust score 92/100."
+}
+```
+
+Fields explained
+
+- `score`: integer 0–100. Starts at 100 and subtracts penalties according to issue severities.
+- `issues`: array of objects { detector, severity, message, meta }.
+- `summary`: short, human-friendly summary.
+
+Configuration and extension points
+
+- `GuardConfig` (second parameter to `validateLLM`) accepts:
+  - `detectors`: enable/disable detector groups (e.g., `numericConsistency`, `hallucination`).
+  - `customRules`: JSON rule packs to add or override existing patterns.
+  - `threshold`: an app-level policy useful for middleware.
+
+Custom rules example
+
+```js
+const custom = {
+  hallucination: [
+    { id: 'hall-001', pattern: "\\bthe capital of mars\\b", severity: 'high', message: 'Fictional location' }
+  ]
+};
+const r = validateLLM('The capital of Mars is Olympus City.', { customRules: custom });
+```
+
+
+License, support, and ethos
+
+- License: see `LICENSE`.
+- Made with ❤️ by ahmadraza100. Open to expand — if you need help with curated rule packs or secure deployment, open an issue or reach out via the repository.

package/dist/batch.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/batch.js

@@ -0,0 +1,116 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const fs_1 = __importDefault(require("fs"));
+const readline_1 = __importDefault(require("readline"));
+const path_1 = __importDefault(require("path"));
+const validate_1 = require("./core/validate");
+function parseArgs() {
+    const argv = process.argv.slice(2);
+    const out = { parallel: 4 };
+    for (let i = 0; i < argv.length; i++) {
+        const a = argv[i];
+        if (a === '--file' && argv[i + 1]) {
+            out.file = argv[++i];
+        }
+        else if (a === '--out' && argv[i + 1]) {
+            out.out = argv[++i];
+        }
+        else if (a === '--parallel' && argv[i + 1]) {
+            out.parallel = parseInt(argv[++i], 10);
+        }
+        else if (a === '--html' && argv[i + 1]) {
+            out.html = argv[++i];
+        }
+    }
+    return out;
+}
+async function processFile(file, outPath, parallel = 4) {
+    const abs = path_1.default.resolve(process.cwd(), file);
+    if (!fs_1.default.existsSync(abs))
+        throw new Error('File not found: ' + abs);
+    const rl = readline_1.default.createInterface({ input: fs_1.default.createReadStream(abs), crlfDelay: Infinity });
+    const outStream = outPath ? fs_1.default.createWriteStream(path_1.default.resolve(process.cwd(), outPath), { encoding: 'utf-8' }) : null;
+    const htmlRows = [];
+    let total = 0;
+    let sumScore = 0;
+    const typeCounts = {};
+    const severityCounts = {};
+    const active = [];
+    for await (const line of rl) {
+        if (!line.trim())
+            continue;
+        const item = JSON.parse(line);
+        const p = (async () => {
+            try {
+                const report = (0, validate_1.validateLLM)(item.text ?? item, {});
+                total += 1;
+                sumScore += report.score;
+                for (const it of report.issues) {
+                    typeCounts[it.type] = (typeCounts[it.type] || 0) + 1;
+                    severityCounts[it.severity] = (severityCounts[it.severity] || 0) + 1;
+                }
+                if (outStream) {
+                    outStream.write(JSON.stringify({ id: item.id, report }) + '\n');
+                }
+                // collect html row (inside scope where report is defined)
+                if (typeof item.id !== 'undefined') {
+                    const safeText = (item.text || '').toString().replace(/</g, '&lt;').replace(/>/g, '&gt;');
+                    const issuesHtml = report.issues.map((it) => `<div><strong>${it.type}</strong> <em>${it.severity}</em>: ${it.message}</div>`).join('');
+                    htmlRows.push(`<tr><td>${item.id}</td><td>${report.score}</td><td>${safeText}</td><td>${issuesHtml}</td></tr>`);
+                }
+            }
+            catch (err) {
+                // log and continue
+                console.error('Item processing error:', err);
+            }
+        })();
+        active.push(p);
+        p.finally(() => {
+            const idx = active.indexOf(p);
+            if (idx >= 0)
+                active.splice(idx, 1);
+        });
+        if (active.length >= parallel) {
+            // wait for any to finish
+            try {
+                await Promise.race(active);
+            }
+            catch (e) { /* ignore single task errors */ }
+        }
+    }
+    // wait for remaining
+    await Promise.all(active.map(p => p.catch(() => { })));
+    if (outStream)
+        outStream.end();
+    const avg = total ? (sumScore / total) : 0;
+    return { total, avgScore: avg, typeCounts, severityCounts, htmlRows };
+}
+async function main() {
+    const args = parseArgs();
+    if (!args.file) {
+        console.error('Missing --file <path>');
+        process.exit(1);
+    }
+    try {
+        const res = await processFile(args.file, args.out, args.parallel ?? 4);
+        console.log('Batch run summary:');
+        console.log('Total items:', res.total);
+        console.log('Average score:', res.avgScore.toFixed(2));
+        console.log('Issue types:', res.typeCounts);
+        console.log('Severity counts:', res.severityCounts);
+        if (args.html) {
+            const htmlFile = path_1.default.resolve(process.cwd(), args.html);
+            const html = `<!doctype html><html><head><meta charset="utf-8"><title>ai-trust-score report</title><style>body{font-family:Arial,Helvetica,sans-serif}table{border-collapse:collapse;width:100%}td,th{border:1px solid #ddd;padding:8px}th{background:#f2f2f2}</style></head><body><h1>ai-trust-score batch report</h1><p>Total: ${res.total} — Average score: ${res.avgScore.toFixed(2)}</p><table><thead><tr><th>ID</th><th>Score</th><th>Text</th><th>Issues</th></tr></thead><tbody>${res.htmlRows.join('')}</tbody></table></body></html>`;
+            fs_1.default.writeFileSync(htmlFile, html, 'utf-8');
+            console.log('Wrote HTML report to', htmlFile);
+        }
+    }
+    catch (err) {
+        console.error('Batch run failed:', err.message || err);
+        process.exit(2);
+    }
+}
+main();

package/dist/cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/cli.js

@@ -0,0 +1,97 @@
+#!/usr/bin/env node
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const fs_1 = __importDefault(require("fs"));
+const path_1 = __importDefault(require("path"));
+const validate_1 = require("./core/validate");
+const chalk_1 = __importDefault(require("chalk"));
+const cli_table3_1 = __importDefault(require("cli-table3"));
+function printUsage() {
+    console.log(chalk_1.default.bold('ai-trust-score CLI'));
+    console.log('Usage: ai-trust-score check --file <path> [--threshold <number>]');
+    console.log('Or: cat output.txt | ai-trust-score check --stdin');
+}
+function colorForSeverity(s) {
+    if (s === 'high')
+        return chalk_1.default.red;
+    if (s === 'medium')
+        return chalk_1.default.yellow;
+    return chalk_1.default.gray;
+}
+async function main() {
+    const argv = process.argv.slice(2);
+    if (argv.length === 0) {
+        printUsage();
+        process.exit(1);
+    }
+    const cmd = argv[0];
+    const args = argv.slice(1);
+    if (cmd !== 'check') {
+        printUsage();
+        process.exit(1);
+    }
+    let file;
+    let useStdin = false;
+    let threshold = 0;
+    for (let i = 0; i < args.length; i++) {
+        const a = args[i];
+        if (a === '--file' && args[i + 1]) {
+            file = args[i + 1];
+            i++;
+        }
+        else if (a === '--stdin') {
+            useStdin = true;
+        }
+        else if (a === '--threshold' && args[i + 1]) {
+            threshold = parseInt(args[i + 1], 10);
+            i++;
+        }
+    }
+    let input = '';
+    if (useStdin) {
+        input = fs_1.default.readFileSync(0, 'utf-8');
+    }
+    else if (file) {
+        const p = path_1.default.resolve(process.cwd(), file);
+        if (!fs_1.default.existsSync(p)) {
+            console.error(chalk_1.default.red(`File not found: ${p}`));
+            process.exit(2);
+        }
+        input = fs_1.default.readFileSync(p, 'utf-8');
+    }
+    else {
+        console.error(chalk_1.default.red('No input: provide --file or --stdin'));
+        process.exit(2);
+    }
+    // Try to parse JSON, otherwise pass as text
+    let parsed = input;
+    try {
+        parsed = JSON.parse(input);
+    }
+    catch (e) { /* leave as string */ }
+    const report = (0, validate_1.validateLLM)(parsed, { numericConsistency: true, hallucinationCheck: true, overconfidenceCheck: true });
+    // Pretty output
+    console.log(chalk_1.default.bold(`Trust score: ${report.score}/100`));
+    console.log(chalk_1.default.dim(report.summary));
+    if (report.issues.length === 0) {
+        console.log(chalk_1.default.green('No issues detected.'));
+    }
+    else {
+        const table = new cli_table3_1.default({ head: ['Type', 'Severity', 'Message'] });
+        for (const it of report.issues) {
+            const color = colorForSeverity(it.severity);
+            table.push([it.type, color(it.severity), it.message]);
+        }
+        console.log(table.toString());
+    }
+    if (threshold && report.score < threshold) {
+        process.exit(3);
+    }
+}
+main().catch(err => {
+    console.error(err);
+    process.exit(10);
+});

package/dist/core/helpers.d.ts

@@ -0,0 +1,18 @@
+import { GuardConfig, GuardReport } from '../types';
+/**
+ * Helper to validate a single output and decide whether to allow it.
+ * Returns { allowed, report } where allowed is true when score >= threshold.
+ */
+export declare function validateAndDecide(output: string | object, config?: GuardConfig, threshold?: number): {
+    allowed: boolean;
+    report: GuardReport;
+};
+/**
+ * Express middleware factory example (lightweight):
+ * Use `app.post('/generate', llmGuardMiddleware({ threshold: 80 }), handler)`
+ * The middleware expects `req.body.output` (string) and will attach `req.llmGuardReport`.
+ */
+export declare function llmGuardMiddleware(opts?: {
+    threshold?: number;
+    config?: GuardConfig;
+}): (req: any, res: any, next: any) => any;

package/dist/core/helpers.js

@@ -0,0 +1,38 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.validateAndDecide = validateAndDecide;
+exports.llmGuardMiddleware = llmGuardMiddleware;
+const validate_1 = require("./validate");
+/**
+ * Helper to validate a single output and decide whether to allow it.
+ * Returns { allowed, report } where allowed is true when score >= threshold.
+ */
+function validateAndDecide(output, config = {}, threshold = 70) {
+    const report = (0, validate_1.validateLLM)(output, config);
+    return { allowed: report.score >= threshold, report };
+}
+/**
+ * Express middleware factory example (lightweight):
+ * Use `app.post('/generate', llmGuardMiddleware({ threshold: 80 }), handler)`
+ * The middleware expects `req.body.output` (string) and will attach `req.llmGuardReport`.
+ */
+function llmGuardMiddleware(opts = {}) {
+    const threshold = opts.threshold ?? 70;
+    const config = opts.config ?? {};
+    return (req, res, next) => {
+        try {
+            const text = req.body && req.body.output;
+            if (!text)
+                return next();
+            const { allowed, report } = validateAndDecide(text, config, threshold);
+            req.llmGuardReport = report;
+            if (!allowed) {
+                return res.status(422).json({ error: 'Output failed ai-trust-score validation', report });
+            }
+            next();
+        }
+        catch (e) {
+            next(e);
+        }
+    };
+}

package/dist/core/registry.d.ts

@@ -0,0 +1,5 @@
+import { Issue, GuardConfig } from '../types';
+export type DetectorFn = (text: string, config: GuardConfig) => Issue[];
+export declare function registerDetector(name: string, fn: DetectorFn): void;
+export declare function runDetectors(text: string, config: GuardConfig): Issue[];
+export declare function listDetectors(): string[];

package/dist/core/registry.js

@@ -0,0 +1,27 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.registerDetector = registerDetector;
+exports.runDetectors = runDetectors;
+exports.listDetectors = listDetectors;
+const detectors = [];
+function registerDetector(name, fn) {
+    detectors.push({ name, fn });
+}
+function runDetectors(text, config) {
+    const issues = [];
+    for (const d of detectors) {
+        try {
+            const res = d.fn(text, config);
+            if (Array.isArray(res) && res.length)
+                issues.push(...res);
+        }
+        catch (err) {
+            // detector error should not crash pipeline
+            issues.push({ type: 'schema', severity: 'low', message: `Detector ${d.name} failed: ${String(err)}` });
+        }
+    }
+    return issues;
+}
+function listDetectors() {
+    return detectors.map(d => d.name);
+}

package/dist/core/score.d.ts

@@ -0,0 +1,2 @@
+import { Issue } from "../types";
+export declare function computeScore(issues: Issue[]): number;

package/dist/core/score.js

@@ -0,0 +1,18 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.computeScore = computeScore;
+const penaltyBySeverity = {
+    high: 15,
+    medium: 8,
+    low: 3,
+};
+function computeScore(issues) {
+    let score = 100;
+    for (const it of issues) {
+        const p = penaltyBySeverity[it.severity] ?? 0;
+        score -= p;
+    }
+    if (score < 0)
+        score = 0;
+    return score;
+}

package/dist/core/validate.d.ts

@@ -0,0 +1,6 @@
+import { GuardConfig, GuardReport } from "../types";
+import '../detectors/hallucination';
+import '../detectors/overconfidence';
+import '../detectors/numeric';
+import '../detectors/inconsistency';
+export declare function validateLLM(output: string | object, config?: GuardConfig): GuardReport;

package/dist/core/validate.js

@@ -0,0 +1,31 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.validateLLM = validateLLM;
+const validator_1 = require("../schema/validator");
+const registry_1 = require("./registry");
+const score_1 = require("./score");
+// Import detectors for side-effects so they register themselves with the registry
+require("../detectors/hallucination");
+require("../detectors/overconfidence");
+require("../detectors/numeric");
+require("../detectors/inconsistency");
+function validateLLM(output, config = {}) {
+    const issues = [];
+    // If object and schema provided, run JSON Schema validation
+    if (config.schema && typeof output === 'object') {
+        const res = (0, validator_1.validateSchema)(output, config.schema);
+        if (!res.valid) {
+            for (const msg of res.errors) {
+                issues.push({ type: 'schema', severity: 'high', message: msg });
+            }
+        }
+    }
+    const text = typeof output === 'string' ? output : JSON.stringify(output);
+    // Run registered detectors (they decide internally whether to run given config)
+    issues.push(...(0, registry_1.runDetectors)(text, config));
+    // (for debugging) you can inspect available detectors via listDetectors()
+    // const available = listDetectors();
+    const score = (0, score_1.computeScore)(issues);
+    const summary = `Detected ${issues.length} issue(s). Trust score ${score}/100.`;
+    return { score, issues, summary };
+}

package/dist/demo.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/demo.js

@@ -0,0 +1,34 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const fs_1 = __importDefault(require("fs"));
+const path_1 = __importDefault(require("path"));
+const validate_1 = require("./core/validate");
+const chalk_1 = __importDefault(require("chalk"));
+const cli_table3_1 = __importDefault(require("cli-table3"));
+async function runDemo() {
+    const p = path_1.default.resolve(process.cwd(), 'examples', 'sample_output.txt');
+    if (!fs_1.default.existsSync(p)) {
+        console.error(chalk_1.default.red('demo example not found:'), p);
+        process.exit(2);
+    }
+    const input = fs_1.default.readFileSync(p, 'utf-8');
+    const report = (0, validate_1.validateLLM)(input, { numericConsistency: true, hallucinationCheck: true, overconfidenceCheck: true });
+    console.log(chalk_1.default.bold('--- ai-trust-score demo output ---'));
+    console.log(chalk_1.default.bold(`Trust score: ${report.score}/100`));
+    console.log(chalk_1.default.dim(report.summary));
+    if (report.issues.length === 0) {
+        console.log(chalk_1.default.green('No issues detected.'));
+    }
+    else {
+        const table = new cli_table3_1.default({ head: ['Type', 'Severity', 'Message'] });
+        for (const it of report.issues) {
+            const color = it.severity === 'high' ? chalk_1.default.red : it.severity === 'medium' ? chalk_1.default.yellow : chalk_1.default.gray;
+            table.push([it.type, color(it.severity), it.message]);
+        }
+        console.log(table.toString());
+    }
+}
+runDemo().catch(e => { console.error(e); process.exit(1); });

package/dist/detectors/hallucination.d.ts

@@ -0,0 +1,3 @@
+import { Issue } from "../types";
+import { GuardConfig } from '../types';
+export declare function detectHallucination(text: string, config?: GuardConfig): Issue[];

package/dist/detectors/hallucination.js

@@ -0,0 +1,53 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.detectHallucination = detectHallucination;
+const fs_1 = __importDefault(require("fs"));
+const path_1 = __importDefault(require("path"));
+const registry_1 = require("../core/registry");
+let rules = [];
+try {
+    const p = path_1.default.resolve(__dirname, 'patterns.json');
+    if (fs_1.default.existsSync(p)) {
+        const raw = fs_1.default.readFileSync(p, 'utf-8');
+        const json = JSON.parse(raw);
+        rules = json.hallucination || [];
+    }
+}
+catch (e) {
+    // ignore rule loading errors
+}
+function detectHallucination(text, config) {
+    const issues = [];
+    // combine built-in rules with custom rules from config
+    const merged = [...rules];
+    try {
+        if (config && config.customRules && Array.isArray(config.customRules['hallucination'])) {
+            merged.push(...config.customRules['hallucination']);
+        }
+    }
+    catch (e) { /* ignore */ }
+    for (const r of merged) {
+        try {
+            const re = new RegExp(r.pattern, r.flags || 'i');
+            if (re.test(text)) {
+                issues.push({ type: r.type || 'hallucination', severity: r.severity, message: r.message });
+            }
+        }
+        catch (e) {
+            // skip invalid pattern
+        }
+    }
+    // small fallback heuristic for institutions
+    const instRe = /([A-Z][a-z]+ (Institute|University|Center|Lab|Academy))/g;
+    let m;
+    while ((m = instRe.exec(text))) {
+        const name = m[1];
+        issues.push({ type: 'hallucination', severity: 'low', message: `Named institution detected: ${name}` });
+    }
+    return issues;
+}
+// register with registry so validateLLM picks it up automatically
+(0, registry_1.registerDetector)('hallucination', (text, config) => detectHallucination(text, config));

package/dist/detectors/inconsistency.d.ts

@@ -0,0 +1,3 @@
+import { Issue } from "../types";
+import { GuardConfig } from '../types';
+export declare function detectInconsistency(text: string, config?: GuardConfig): Issue[];

package/dist/detectors/inconsistency.js

@@ -0,0 +1,33 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.detectInconsistency = detectInconsistency;
+const opposites = {
+    'cost': ['increase', 'decrease', 'reduces', 'increases'],
+    'costs': ['increase', 'decrease', 'reduces', 'increases'],
+    'growth': ['increase', 'decrease', 'fell', 'rose'],
+    'reduce': ['reduce', 'increase', 'increases', 'reduces']
+};
+function detectInconsistency(text, config) {
+    const issues = [];
+    const sentences = text.split(/[\.\n\!\?]+/).map(s => s.trim()).filter(Boolean);
+    for (let i = 0; i < sentences.length; i++) {
+        for (let j = i + 1; j < Math.min(sentences.length, i + 6); j++) {
+            const a = sentences[i].toLowerCase();
+            const b = sentences[j].toLowerCase();
+            for (const key of Object.keys(opposites)) {
+                if (a.includes(key) && (a.includes('increase') || a.includes('decrease') || a.includes('reduce') || a.includes('reduce'))) {
+                    // check b for opposite word
+                    if ((b.includes('increase') || b.includes('increases') || b.includes('rose')) && (a.includes('decrease') || a.includes('fell') || a.includes('reduce'))) {
+                        issues.push({ type: 'inconsistency', severity: 'high', message: `Contradicting statements between sentences: "${sentences[i]}" vs "${sentences[j]}"` });
+                    }
+                    if ((b.includes('decrease') || b.includes('fell') || b.includes('reduce')) && (a.includes('increase') || a.includes('increases') || a.includes('rose'))) {
+                        issues.push({ type: 'inconsistency', severity: 'high', message: `Contradicting statements between sentences: "${sentences[i]}" vs "${sentences[j]}"` });
+                    }
+                }
+            }
+        }
+    }
+    return issues;
+}
+const registry_1 = require("../core/registry");
+(0, registry_1.registerDetector)('inconsistency', (text) => detectInconsistency(text));

package/dist/detectors/numeric.d.ts

@@ -0,0 +1,3 @@
+import { Issue } from "../types";
+import { GuardConfig } from '../types';
+export declare function detectNumeric(text: string, config?: GuardConfig): Issue[];

package/dist/detectors/numeric.js

@@ -0,0 +1,55 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.detectNumeric = detectNumeric;
+// Very small heuristic numeric checks: extracts simple number pairs and checks basic arithmetic
+const registry_1 = require("../core/registry");
+function detectNumeric(text, config) {
+    const issues = [];
+    // Detect patterns like "from X to Y" and check percent claims nearby
+    const rangeRe = /from\s+(\d+(?:\.\d+)?)\s*(million|billion|k|m)?\s*to\s+(\d+(?:\.\d+)?)\s*(million|billion|k|m)?/i;
+    const pctRe = /(\d{1,3}(?:\.\d+)?)%/g;
+    const r = rangeRe.exec(text);
+    if (r) {
+        const a = parseFloat(r[1]);
+        const b = parseFloat(r[3]);
+        if (!isNaN(a) && !isNaN(b) && a > 0) {
+            const implied = ((b - a) / a) * 100;
+            // See if a nearby percent is claimed that differs by >5 percentage points
+            const window = text.slice(Math.max(0, r.index - 100), Math.min(text.length, r.index + 200));
+            const pcts = [];
+            let m;
+            while ((m = pctRe.exec(window))) {
+                pcts.push(parseFloat(m[1]));
+            }
+            if (pcts.length > 0) {
+                const nearest = pcts[0];
+                if (Math.abs(nearest - implied) > 5) {
+                    issues.push({ type: 'numeric', severity: 'medium', message: `Percentage ${nearest}% inconsistent with increase from ${a} to ${b} (~${implied.toFixed(1)}%).` });
+                }
+            }
+        }
+    }
+    // Detect simple sum inconsistencies: look for "total" and list of numbers
+    const totalRe = /total(?:ed|s|:)\s*(\$?\d[\d,\.kmbMKB]*)/i;
+    const numsRe = /(\$?\d[\d,\.kmbMKB]*)/g;
+    const tot = totalRe.exec(text);
+    if (tot) {
+        // crude: sum first three numbers and compare
+        const window = text.slice(0, Math.min(text.length, tot.index));
+        const nums = [];
+        let m;
+        while ((m = numsRe.exec(window)) && nums.length < 5) {
+            const cleaned = m[1].replace(/[$,]/g, '');
+            const parsed = parseFloat(cleaned);
+            if (!isNaN(parsed))
+                nums.push(parsed);
+        }
+        const totalVal = parseFloat(tot[1].replace(/[$,]/g, ''));
+        const sum = nums.reduce((s, x) => s + x, 0);
+        if (nums.length >= 2 && Math.abs(sum - totalVal) / Math.max(1, totalVal) > 0.05) {
+            issues.push({ type: 'numeric', severity: 'medium', message: `Listed components sum (${sum}) differs from claimed total (${totalVal}).` });
+        }
+    }
+    return issues;
+}
+(0, registry_1.registerDetector)('numeric', (text) => detectNumeric(text));

package/dist/detectors/overconfidence.d.ts

@@ -0,0 +1,2 @@
+import { Issue, GuardConfig } from "../types";
+export declare function detectOverconfidence(text: string, config?: GuardConfig): Issue[];

package/dist/detectors/overconfidence.js

@@ -0,0 +1,50 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.detectOverconfidence = detectOverconfidence;
+const fs_1 = __importDefault(require("fs"));
+const path_1 = __importDefault(require("path"));
+const registry_1 = require("../core/registry");
+let rules = [];
+try {
+    const p = path_1.default.resolve(__dirname, 'patterns.json');
+    if (fs_1.default.existsSync(p)) {
+        const raw = fs_1.default.readFileSync(p, 'utf-8');
+        const json = JSON.parse(raw);
+        rules = json.overconfidence || [];
+    }
+}
+catch (e) {
+    // ignore
+}
+function detectOverconfidence(text, config) {
+    const issues = [];
+    let count = 0;
+    const merged = [...rules];
+    try {
+        if (config && config.customRules && Array.isArray(config.customRules['overconfidence'])) {
+            merged.push(...config.customRules['overconfidence']);
+        }
+    }
+    catch (e) { }
+    for (const r of merged) {
+        try {
+            const re = new RegExp(r.pattern, r.flags || 'i');
+            const m = text.match(re);
+            if (m) {
+                count += 1;
+                issues.push({ type: r.type || 'confidence', severity: r.severity, message: `${r.message} "${m[0]}"` });
+            }
+        }
+        catch (e) {
+            // ignore
+        }
+    }
+    if (count > 1) {
+        issues.push({ type: 'confidence', severity: 'medium', message: `${count} strong certainty markers found.` });
+    }
+    return issues;
+}
+(0, registry_1.registerDetector)('overconfidence', (text, config) => detectOverconfidence(text, config));

package/dist/generate_samples.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/generate_samples.js

@@ -0,0 +1,41 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const fs_1 = __importDefault(require("fs"));
+const path_1 = __importDefault(require("path"));
+// Improved generator: creates varied samples by altering numbers and certainty phrases
+function usage() { console.log('Usage: generate_samples <count> <out.jsonl>'); }
+function randChoice(arr) { return arr[Math.floor(Math.random() * arr.length)]; }
+function makeSample(i, base) {
+    // variations: replace percentage, numbers, and certainty phrases
+    const pct = Math.floor(Math.random() * 80) + 1; // 1-80%
+    const a = Math.floor(50 + Math.random() * 200);
+    const b = Math.floor(a + (Math.random() * a));
+    const cert = randChoice(['Definitely', 'Probably', 'It seems', 'Experts say', 'Without a doubt', 'Research shows']);
+    // build a sentence emulating the base
+    const text = `${cert} the product revenue grew ${pct}% from ${a} to ${b}. ${randChoice(['This is the best outcome.', 'This is an expected result.', 'This is surprising.'])}`;
+    return { id: i + 1, text };
+}
+function main() {
+    const argv = process.argv.slice(2);
+    if (argv.length < 2) {
+        usage();
+        process.exit(1);
+    }
+    const count = parseInt(argv[0], 10);
+    const out = path_1.default.resolve(process.cwd(), argv[1]);
+    if (isNaN(count) || count <= 0) {
+        console.error('count must be > 0');
+        process.exit(2);
+    }
+    const stream = fs_1.default.createWriteStream(out, { encoding: 'utf-8' });
+    for (let i = 0; i < count; i++) {
+        const obj = makeSample(i, '');
+        stream.write(JSON.stringify(obj) + '\n');
+    }
+    stream.end();
+    console.log(`Wrote ${count} samples to ${out}`);
+}
+main();

package/dist/index.d.ts

@@ -0,0 +1,4 @@
+export { validateLLM } from './core/validate';
+export * from './types';
+import { validateLLM as _validateLLM } from './core/validate';
+export default _validateLLM;

package/dist/index.js

@@ -0,0 +1,23 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.validateLLM = void 0;
+var validate_1 = require("./core/validate");
+Object.defineProperty(exports, "validateLLM", { enumerable: true, get: function () { return validate_1.validateLLM; } });
+__exportStar(require("./types"), exports);
+// Default export for convenience
+const validate_2 = require("./core/validate");
+exports.default = validate_2.validateLLM;

package/dist/schema/validator.d.ts

@@ -0,0 +1,4 @@
+export declare function validateSchema(data: object, schema: object): {
+    valid: boolean;
+    errors: string[];
+};

package/dist/schema/validator.js

@@ -0,0 +1,21 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.validateSchema = validateSchema;
+const ajv_1 = __importDefault(require("ajv"));
+const ajv = new ajv_1.default({ allErrors: true, strict: false });
+function validateSchema(data, schema) {
+    try {
+        const validate = ajv.compile(schema);
+        const valid = validate(data);
+        if (valid)
+            return { valid: true, errors: [] };
+        const errors = (validate.errors || []).map((e) => `${e.instancePath} ${e.message}`);
+        return { valid: false, errors };
+    }
+    catch (err) {
+        return { valid: false, errors: [String(err.message || err)] };
+    }
+}

package/dist/types/index.d.ts

@@ -0,0 +1,22 @@
+export type Severity = 'low' | 'medium' | 'high';
+export type IssueType = 'hallucination' | 'schema' | 'numeric' | 'confidence' | 'inconsistency';
+export interface Issue {
+    type: IssueType;
+    severity: Severity;
+    message: string;
+    location?: string;
+}
+export interface GuardReport {
+    score: number;
+    issues: Issue[];
+    summary: string;
+}
+export interface GuardConfig {
+    schema?: object;
+    requiredSections?: string[];
+    numericConsistency?: boolean;
+    hallucinationCheck?: boolean;
+    overconfidenceCheck?: boolean;
+    maxConfidenceWithoutEvidence?: number;
+    customRules?: Record<string, Array<Record<string, any>>>;
+}

package/dist/types/index.js

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/package.json

@@ -0,0 +1,45 @@
+{
+  "name": "ai-trust-score",
+  "version": "0.1.0",
+  "description": "ai-trust-score — deterministic validator for LLM outputs (schema, heuristics, consistency checks)",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "scripts": {
+  "build": "tsc -p tsconfig.json",
+  "test": "vitest",
+  "prepare": "echo 'skip build on pack'",
+    "cli": "node ./dist/cli.js",
+    "demo": "node ./dist/demo.js"
+  },
+  "keywords": ["llm","validator","schema","hallucination","offline","safety","audit"],
+  "author": "ai-trust-score Contributors <contributors@ai-trust-score.dev>",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ahmadraza/ai-trust-score.git"
+  },
+  "bugs": {
+    "url": "https://github.com/ahmadraza/ai-trust-score/issues"
+  },
+  "homepage": "https://github.com/ahmadraza/ai-trust-score#readme",
+  "devDependencies": {
+    "@types/node": "^20.4.2",
+    "ts-node": "^10.9.1",
+    "typescript": "^5.1.6",
+    "vitest": "^1.3.0"
+  },
+  "bin": {
+    "ai-trust-score": "dist/cli.js"
+  },
+  "files": ["dist","README.md"],
+  "engines": {
+    "node": ">=18"
+  },
+  "license": "MIT",
+  "dependencies": {
+    "ajv": "^8.12.0",
+    "chalk": "^4.1.2",
+    "cli-table3": "^0.6.0"
+  }
+}
+
+