@ericsanchezok/synergy-secret-detection 0.0.0-stage → 3.1.0

package/README.md

@@ -1,3 +1,57 @@
-# Temporary Holding Version
+# Secret detection
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Runtime-independent detection and evaluation for Synergy. The package exports a typed asynchronous detector, the `standalone-regex` version 2 baseline, shared redaction patterns and evaluation functions. It does not read credentials, access the filesystem, register secrets, load models or make network requests. Executable evaluation scripts own their input/output separately.
+
+## Detector contract
+
+```ts
+import { RegexDetector, SecretDetection } from "@ericsanchezok/synergy-secret-detection"
+
+const result = await SecretDetection.detect(
+  RegexDetector,
+  {
+    text: "original message or tool output",
+    source: "user_message",
+  },
+  { timeoutMs: 5000 },
+)
+```
+
+`Detector` has `id`, `version` and `detect({ text, source, signal })`. A successful scan returns `{ complete: true, findings }`; a truncated or unsupported scan returns `{ complete: false, reason, findings }`. Each finding has `start`, `end`, `kind` and optional `score`. Offsets are zero-based UTF-16 code units into the exact input, with an inclusive start and exclusive end. Findings must be sorted, non-overlapping, non-empty, within the input and outside surrogate pairs. Consumers extract `text.slice(start, end)`; adapters never regenerate the secret value. A score is adapter-specific and is not assumed calibrated across implementations.
+
+The runner validates results and exposes sanitized `SecretDetection.Error` codes for failure, invalid output, cancellation and timeout. Cancellation is cooperative: it forwards an `AbortSignal` and bounds the caller's wait, but cannot preempt synchronous JavaScript or forcibly stop a model process. A model adapter must honor cancellation and own loading/disposal outside individual detection calls; its version must identify weights, preprocessing, thresholds and other effective settings. No model adapter ships in this package.
+
+The regex baseline scans the whole input with the shared standalone prefix inventory and OAuth authorization-code URL spans, then merges overlapping hits. Authorization codes require an `auth`, `oauth`, `oauth2`, `authorize` or `callback` path segment; ordinary document URL `code` parameters remain ordinary text. It does not parse arbitrary credential fields, measure entropy, validate credentials or use the separate log/SmartAllow rules. Example keys can be false positives; unknown formats can be false negatives. These cases are retained in the evaluation corpus.
+
+## Harness integration
+
+The Harness `secrets/detector-source` entry accepts `SecretDetectorSource.register(detector)` during runtime composition; `register(undefined)` restores the regex default. Harness owns capture, Vault persistence, masking and execution-time resolution. An empty detection result does not disable matching already registered values. Incomplete scans, timeouts, invalid results and registration failures stop the capture operation instead of being treated as clean text. The production capture deadline is five seconds.
+
+Harness retains its eight-character minimum for global value replacement. An adapter returning a shorter candidate is rejected before registration rather than reporting success while leaving it unmasked. Offline evaluation supports shorter spans independently. Changing this host policy needs separate false-positive and masking evaluation.
+
+## Evaluation commands
+
+Run from this package:
+
+```bash
+bun test
+bun run typecheck
+bun run evaluate --output results/regex.json
+bun run evaluate --detector /absolute/path/to/adapter.ts --dataset fixtures/corpus.json --output results/candidate.json
+```
+
+An adapter module exports `detector: SecretDetection.Detector`. It is explicitly selected executable code; the runner does not scan for plugins or instantiate a model SDK. `--samples` defaults to 100, `--warmup` to 20, and `--timeout-ms` to 5000. `--help` lists all options.
+
+[The synthetic corpus](fixtures/corpus.json) has independent original-text span labels, recognized prefixes, unknown credentials, Unicode, JSON, URLs, multiple hits, ordinary code, hashes and benign example keys. These deliberately small fixtures validate the evaluator and expose baseline gaps; they are not a representative quality benchmark or training set. Future training and holdout splits must separate source/template and credential families, rather than only changing random key bytes.
+
+Reports include detector/version, dataset version/hash, evaluator source hash, optional adapter module hash, runtime/hardware, per-case outcomes and aggregate/group metrics. Gold labels are never passed to detectors. Exact precision/recall score span boundaries; full-coverage recall requires a finding to contain the whole labeled secret, so a prefix-only hit fails protection. Clean-message false-positive rate uses only completed negative cases; failed/incomplete negatives are reported separately. Failed positive cases remain in recall denominators. Null ratios mean no denominator, not perfect quality. Reports omit sample text, detected values and raw exception messages; use non-sensitive case identifiers and adapter metadata too.
+
+Timing scenarios use synthetic 1 KiB, 64 KiB and 1 MiB inputs with 0, 1 or 32 candidates. They measure the detector plus asynchronous result validation, first-call time per scenario after accuracy evaluation, warm p50/p95, throughput, process CPU and RSS snapshots. Adapter import and first-call times are not full process cold-start or peak-memory measurements. Failed/incomplete scenarios remain visible and produce a nonzero command exit. Misses and false positives are reported metrics, not infrastructure failures.
+
+The real masking pipeline has a separate isolated runner:
+
+```bash
+bun run --cwd packages/harness benchmark:secrets --samples 20
+```
+
+Run that command from the repository root. It varies Vault size, input length, hit count and first-registration versus already-registered paths. Each run owns and removes a temporary home. It includes detection, validation, Vault and replacement, excludes fixture resets from latency, and never starts a server or calls a model. Use [the task benchmark](../../benchmark/README.md) for session-level latency and task outcomes; detector microbenchmarks cannot establish those effects.

package/dist/detector.d.ts

@@ -0,0 +1,56 @@
+import { z } from "zod";
+export declare namespace SecretDetection {
+    const Source: z.ZodEnum<{
+        credential_file: "credential_file";
+        tool_output: "tool_output";
+        user_message: "user_message";
+    }>;
+    type Source = z.infer<typeof Source>;
+    const Finding: z.ZodObject<{
+        start: z.ZodNumber;
+        end: z.ZodNumber;
+        kind: z.ZodString;
+        score: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>;
+    type Finding = z.infer<typeof Finding>;
+    const Result: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        complete: z.ZodLiteral<true>;
+        findings: z.ZodArray<z.ZodObject<{
+            start: z.ZodNumber;
+            end: z.ZodNumber;
+            kind: z.ZodString;
+            score: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>>;
+    }, z.core.$strip>, z.ZodObject<{
+        complete: z.ZodLiteral<false>;
+        findings: z.ZodArray<z.ZodObject<{
+            start: z.ZodNumber;
+            end: z.ZodNumber;
+            kind: z.ZodString;
+            score: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>>;
+        reason: z.ZodEnum<{
+            input_limit: "input_limit";
+            unsupported: "unsupported";
+        }>;
+    }, z.core.$strip>], "complete">;
+    type Result = z.infer<typeof Result>;
+    interface Input {
+        text: string;
+        source: Source;
+        signal?: AbortSignal;
+    }
+    interface Detector {
+        readonly id: string;
+        readonly version: string;
+        detect(input: Input): Promise<Result>;
+    }
+    class Error extends globalThis.Error {
+        readonly code: "aborted" | "timeout" | "invalid_result" | "detector_failed" | "incomplete";
+        constructor(code: "aborted" | "timeout" | "invalid_result" | "detector_failed" | "incomplete");
+    }
+    function validate(text: string, result: unknown): Result;
+    function detect(detector: Detector, input: Input, options?: {
+        timeoutMs?: number;
+    }): Promise<Result>;
+}

package/dist/detector.js

@@ -0,0 +1,89 @@
+import { z } from "zod";
+export var SecretDetection;
+(function (SecretDetection) {
+    SecretDetection.Source = z.enum(["user_message", "tool_output", "credential_file"]);
+    SecretDetection.Finding = z.object({
+        start: z.number().int().nonnegative(),
+        end: z.number().int().positive(),
+        kind: z.string().min(1),
+        score: z.number().min(0).max(1).optional(),
+    });
+    SecretDetection.Result = z.discriminatedUnion("complete", [
+        z.object({ complete: z.literal(true), findings: z.array(SecretDetection.Finding) }),
+        z.object({
+            complete: z.literal(false),
+            findings: z.array(SecretDetection.Finding),
+            reason: z.enum(["input_limit", "unsupported"]),
+        }),
+    ]);
+    class Error extends globalThis.Error {
+        code;
+        constructor(code) {
+            super(`Secret detection failed: ${code}`);
+            this.code = code;
+            this.name = "SecretDetectionError";
+        }
+    }
+    SecretDetection.Error = Error;
+    function splitsSurrogate(text, offset) {
+        const left = text.charCodeAt(offset - 1);
+        const right = text.charCodeAt(offset);
+        return left >= 0xd800 && left <= 0xdbff && right >= 0xdc00 && right <= 0xdfff;
+    }
+    function validate(text, result) {
+        const parsed = SecretDetection.Result.safeParse(result);
+        if (!parsed.success)
+            throw new Error("invalid_result");
+        let end = 0;
+        for (const finding of parsed.data.findings) {
+            if (finding.start < end ||
+                finding.end <= finding.start ||
+                finding.end > text.length ||
+                splitsSurrogate(text, finding.start) ||
+                splitsSurrogate(text, finding.end))
+                throw new Error("invalid_result");
+            end = finding.end;
+        }
+        return parsed.data;
+    }
+    SecretDetection.validate = validate;
+    async function detect(detector, input, options = {}) {
+        if (input.signal?.aborted)
+            throw new Error("aborted");
+        if (options.timeoutMs !== undefined && (!Number.isFinite(options.timeoutMs) || options.timeoutMs <= 0)) {
+            throw new RangeError("timeoutMs must be a positive finite number");
+        }
+        const controller = new AbortController();
+        let rejectAbort;
+        const cancelled = new Promise((_, reject) => {
+            rejectAbort = reject;
+        });
+        const cancel = (code) => {
+            rejectAbort(new Error(code));
+            controller.abort();
+        };
+        const onAbort = () => cancel("aborted");
+        input.signal?.addEventListener("abort", onAbort, { once: true });
+        const timer = options.timeoutMs === undefined ? undefined : setTimeout(() => cancel("timeout"), options.timeoutMs);
+        try {
+            const result = await Promise.race([
+                Promise.resolve().then(() => detector.detect({ text: input.text, source: input.source, signal: controller.signal })),
+                cancelled,
+            ]);
+            if (input.signal?.aborted)
+                throw new Error("aborted");
+            return validate(input.text, result);
+        }
+        catch (error) {
+            if (error instanceof Error)
+                throw error;
+            throw new Error("detector_failed");
+        }
+        finally {
+            if (timer !== undefined)
+                clearTimeout(timer);
+            input.signal?.removeEventListener("abort", onAbort);
+        }
+    }
+    SecretDetection.detect = detect;
+})(SecretDetection || (SecretDetection = {}));

package/dist/evaluation.d.ts

@@ -0,0 +1,112 @@
+import { z } from "zod";
+import { SecretDetection } from "./detector.js";
+export declare namespace SecretEvaluation {
+    export const Case: z.ZodObject<{
+        id: z.ZodString;
+        group: z.ZodString;
+        text: z.ZodString;
+        source: z.ZodEnum<{
+            credential_file: "credential_file";
+            tool_output: "tool_output";
+            user_message: "user_message";
+        }>;
+        expected: z.ZodArray<z.ZodObject<{
+            start: z.ZodNumber;
+            end: z.ZodNumber;
+        }, z.core.$strip>>;
+    }, z.core.$strip>;
+    export type Case = z.infer<typeof Case>;
+    export const Dataset: z.ZodObject<{
+        version: z.ZodString;
+        cases: z.ZodArray<z.ZodObject<{
+            id: z.ZodString;
+            group: z.ZodString;
+            text: z.ZodString;
+            source: z.ZodEnum<{
+                credential_file: "credential_file";
+                tool_output: "tool_output";
+                user_message: "user_message";
+            }>;
+            expected: z.ZodArray<z.ZodObject<{
+                start: z.ZodNumber;
+                end: z.ZodNumber;
+            }, z.core.$strip>>;
+        }, z.core.$strip>>;
+    }, z.core.$strip>;
+    export type Dataset = z.infer<typeof Dataset>;
+    interface Counts {
+        expected: number;
+        predicted: number;
+        exactMatches: number;
+        fullyCovered: number;
+        falsePositives: number;
+        cleanCases: number;
+        completedCleanCases: number;
+        falsePositiveCases: number;
+        failures: number;
+        incomplete: number;
+    }
+    export function evaluate(detector: SecretDetection.Detector, cases: Case[], options?: {
+        timeoutMs?: number;
+    }): Promise<{
+        detector: {
+            id: string;
+            version: string;
+        };
+        metrics: {
+            expected: number;
+            predicted: number;
+            exactMatches: number;
+            fullyCovered: number;
+            falsePositives: number;
+            cleanCases: number;
+            completedCleanCases: number;
+            falsePositiveCases: number;
+            failures: number;
+            incomplete: number;
+            exactPrecision: number | null;
+            exactRecall: number | null;
+            fullCoverageRecall: number | null;
+            cleanMessageFalsePositiveRate: number | null;
+        };
+        groups: {
+            [k: string]: {
+                expected: number;
+                predicted: number;
+                exactMatches: number;
+                fullyCovered: number;
+                falsePositives: number;
+                cleanCases: number;
+                completedCleanCases: number;
+                falsePositiveCases: number;
+                failures: number;
+                incomplete: number;
+                exactPrecision: number | null;
+                exactRecall: number | null;
+                fullCoverageRecall: number | null;
+                cleanMessageFalsePositiveRate: number | null;
+            };
+        };
+        cases: ({
+            id: string;
+            group: string;
+            status: "complete" | "incomplete" | "failed";
+            error?: string;
+            durationMs: number;
+        } & Counts)[];
+    }>;
+    export function benchmark(detector: SecretDetection.Detector, input: SecretDetection.Input, options: {
+        samples: number;
+        warmup: number;
+        timeoutMs?: number;
+    }): Promise<{
+        samples: number;
+        warmup: number;
+        firstCallMs: number;
+        p50Ms: number;
+        p95Ms: number;
+        durationMs: number;
+        callsPerSecond: number | null;
+    }>;
+    export {};
+}

package/dist/evaluation.js

@@ -0,0 +1,132 @@
+import { z } from "zod";
+import { SecretDetection } from "./detector.js";
+export var SecretEvaluation;
+(function (SecretEvaluation) {
+    const Span = SecretDetection.Finding.pick({ start: true, end: true });
+    SecretEvaluation.Case = z.object({
+        id: z.string().min(1),
+        group: z.string().min(1),
+        text: z.string(),
+        source: SecretDetection.Source,
+        expected: z.array(Span),
+    });
+    SecretEvaluation.Dataset = z.object({ version: z.string().min(1), cases: z.array(SecretEvaluation.Case).min(1) });
+    function counts() {
+        return {
+            expected: 0,
+            predicted: 0,
+            exactMatches: 0,
+            fullyCovered: 0,
+            falsePositives: 0,
+            cleanCases: 0,
+            completedCleanCases: 0,
+            falsePositiveCases: 0,
+            failures: 0,
+            incomplete: 0,
+        };
+    }
+    function metrics(count) {
+        const ratio = (a, b) => (b ? a / b : null);
+        return {
+            ...count,
+            exactPrecision: ratio(count.exactMatches, count.predicted),
+            exactRecall: ratio(count.exactMatches, count.expected),
+            fullCoverageRecall: ratio(count.fullyCovered, count.expected),
+            cleanMessageFalsePositiveRate: ratio(count.falsePositiveCases, count.completedCleanCases),
+        };
+    }
+    async function evaluate(detector, cases, options = {}) {
+        const total = counts();
+        const groups = new Map();
+        const rows = [];
+        const ids = new Set();
+        for (const sample of cases) {
+            SecretEvaluation.Case.parse(sample);
+            if (ids.has(sample.id))
+                throw new Error("Duplicate evaluation case id");
+            ids.add(sample.id);
+            SecretDetection.validate(sample.text, {
+                complete: true,
+                findings: sample.expected.map((span) => ({ ...span, kind: "label" })),
+            });
+        }
+        for (const sample of cases) {
+            const count = counts();
+            count.expected = sample.expected.length;
+            count.cleanCases = sample.expected.length === 0 ? 1 : 0;
+            const start = performance.now();
+            let status = "complete";
+            let error;
+            try {
+                const result = await SecretDetection.detect(detector, sample, options);
+                if (!result.complete) {
+                    status = "incomplete";
+                    count.incomplete = 1;
+                }
+                count.predicted = result.findings.length;
+                count.exactMatches = sample.expected.filter((label) => result.findings.some((f) => f.start === label.start && f.end === label.end)).length;
+                count.fullyCovered = sample.expected.filter((label) => result.findings.some((f) => f.start <= label.start && f.end >= label.end)).length;
+                count.falsePositives = result.findings.filter((f) => !sample.expected.some((label) => f.start < label.end && f.end > label.start)).length;
+                count.completedCleanCases = result.complete ? count.cleanCases : 0;
+                count.falsePositiveCases = count.completedCleanCases && count.predicted > 0 ? 1 : 0;
+            }
+            catch (failure) {
+                status = "failed";
+                count.failures = 1;
+                error = failure instanceof SecretDetection.Error ? failure.code : "detector_failed";
+            }
+            const durationMs = performance.now() - start;
+            const group = groups.get(sample.group) ?? counts();
+            groups.set(sample.group, group);
+            for (const key of Object.keys(count)) {
+                total[key] += count[key];
+                group[key] += count[key];
+            }
+            rows.push({ id: sample.id, group: sample.group, status, ...(error ? { error } : {}), durationMs, ...count });
+        }
+        return {
+            detector: { id: detector.id, version: detector.version },
+            metrics: metrics(total),
+            groups: Object.fromEntries([...groups].map(([key, count]) => [key, metrics(count)])),
+            cases: rows,
+        };
+    }
+    SecretEvaluation.evaluate = evaluate;
+    async function benchmark(detector, input, options) {
+        if (!Number.isInteger(options.samples) ||
+            options.samples < 1 ||
+            !Number.isInteger(options.warmup) ||
+            options.warmup < 0)
+            throw new RangeError("Invalid benchmark sample count");
+        const start = performance.now();
+        const first = await SecretDetection.detect(detector, input, options);
+        const firstCallMs = performance.now() - start;
+        if (!first.complete)
+            throw new SecretDetection.Error("incomplete");
+        for (let i = 0; i < options.warmup; i++) {
+            const result = await SecretDetection.detect(detector, input, options);
+            if (!result.complete)
+                throw new SecretDetection.Error("incomplete");
+        }
+        const elapsed = [];
+        for (let i = 0; i < options.samples; i++) {
+            const begin = performance.now();
+            const result = await SecretDetection.detect(detector, input, options);
+            elapsed.push(performance.now() - begin);
+            if (!result.complete)
+                throw new SecretDetection.Error("incomplete");
+        }
+        elapsed.sort((a, b) => a - b);
+        const durationMs = elapsed.reduce((a, b) => a + b, 0);
+        return {
+            samples: options.samples,
+            warmup: options.warmup,
+            firstCallMs,
+            p50Ms: elapsed[Math.ceil(elapsed.length * 0.5) - 1],
+            p95Ms: elapsed[Math.ceil(elapsed.length * 0.95) - 1],
+            durationMs,
+            callsPerSecond: durationMs > 0 ? (options.samples * 1000) / durationMs : null,
+        };
+    }
+    SecretEvaluation.benchmark = benchmark;
+})(SecretEvaluation || (SecretEvaluation = {}));

package/dist/index.d.ts

@@ -0,0 +1,2 @@
+export { SecretDetection } from "./detector.js";
+export { RegexDetector } from "./regex.js";

package/dist/index.js

@@ -0,0 +1,2 @@
+export { SecretDetection } from "./detector.js";
+export { RegexDetector } from "./regex.js";

package/dist/patterns.d.ts

@@ -0,0 +1,25 @@
+export declare namespace SecretPatterns {
+    /**
+     * Single inventory of standalone secret token shapes. Consumers
+     * (observability redaction, heuristic capture) read from here so a new
+     * provider prefix updates one place instead of drifting across copies.
+     */
+    const standalone: readonly RegExp[];
+    /**
+     * Long-form token matcher used by SmartAllow evidence redaction: stricter
+     * minimum lengths plus a generic high-entropy catch-all.
+     */
+    function longForm(): RegExp;
+    const authSchemes: readonly [RegExp, string][];
+    const keyValue: RegExp;
+    const queryParam: RegExp;
+    function oauthCodes(text: string): Array<{
+        start: number;
+        end: number;
+    }>;
+    /** Replace every standalone token match; returns the text and match count. */
+    function replaceStandalone(text: string, replacement: string): {
+        text: string;
+        matches: number;
+    };
+}

package/dist/patterns.js

@@ -0,0 +1,70 @@
+export var SecretPatterns;
+(function (SecretPatterns) {
+    /**
+     * Single inventory of standalone secret token shapes. Consumers
+     * (observability redaction, heuristic capture) read from here so a new
+     * provider prefix updates one place instead of drifting across copies.
+     */
+    SecretPatterns.standalone = [
+        /\bsk-[A-Za-z0-9._-]{8,}\b/g,
+        /\bghp_[A-Za-z0-9_]{8,}\b/g,
+        /\bgithub_pat_[A-Za-z0-9_]{8,}\b/g,
+        /\bxox[baprs]-[A-Za-z0-9-]{8,}\b/g,
+        /\bhf_[A-Za-z0-9_]{8,}\b/g,
+        /\bglpat-[A-Za-z0-9-]{8,}\b/g,
+        /\bpk_live_[A-Za-z0-9_]{8,}\b/g,
+        /\brk_live_[A-Za-z0-9_]{8,}\b/g,
+        /\btok_[A-Za-z0-9._-]{8,}\b/g,
+        /\bkey_[A-Za-z0-9._-]{8,}\b/g,
+    ];
+    /**
+     * Long-form token matcher used by SmartAllow evidence redaction: stricter
+     * minimum lengths plus a generic high-entropy catch-all.
+     */
+    function longForm() {
+        return /\b(?:sk-[A-Za-z0-9_-]{16,}|sk-proj-[A-Za-z0-9_-]{16,}|github_pat_[A-Za-z0-9_]{16,}|gh[pousr]_[A-Za-z0-9_]{16,}|[A-Za-z0-9+/=_-]{48,})\b/g;
+    }
+    SecretPatterns.longForm = longForm;
+    SecretPatterns.authSchemes = [
+        [/(Bearer\s+)[A-Za-z0-9._~+/-]+=*/gi, "$1[redacted]"],
+        [/(Basic\s+)[A-Za-z0-9+/=]+/gi, "$1[redacted]"],
+        [/(Digest\s+)[A-Za-z0-9+/=]+/gi, "$1[redacted]"],
+    ];
+    SecretPatterns.keyValue = /((?:token|secret|password|authorization|api[_-]?key|cookie)["']?\s*[:=]\s*["']?)[^\s"'&]+/gi;
+    SecretPatterns.queryParam = /([?&](?:token|secret|password|authorization|api[_-]?key|cookie|key)=)[^&#\s"']+/gi;
+    function oauthCodes(text) {
+        const spans = [];
+        for (const url of text.matchAll(/\bhttps?:\/\/[^\s"'<>\\()[\]{}]+/gi)) {
+            let pathname;
+            try {
+                pathname = new URL(url[0]).pathname;
+            }
+            catch {
+                continue;
+            }
+            if (!/\/(?:auth|oauth2?|authorize|callback)(?:\/|$)/i.test(pathname))
+                continue;
+            for (const match of url[0].matchAll(/([?&]code=)([^&#]+)/gi)) {
+                if (match[2].startsWith("⟦sec:"))
+                    continue;
+                const start = url.index + match.index + match[1].length;
+                spans.push({ start, end: start + match[2].length });
+            }
+        }
+        return spans;
+    }
+    SecretPatterns.oauthCodes = oauthCodes;
+    /** Replace every standalone token match; returns the text and match count. */
+    function replaceStandalone(text, replacement) {
+        let matches = 0;
+        let result = text;
+        for (const pattern of SecretPatterns.standalone) {
+            result = result.replace(pattern, () => {
+                matches++;
+                return replacement;
+            });
+        }
+        return { text: result, matches };
+    }
+    SecretPatterns.replaceStandalone = replaceStandalone;
+})(SecretPatterns || (SecretPatterns = {}));

package/dist/regex.d.ts

@@ -0,0 +1,2 @@
+import type { SecretDetection } from "./detector.js";
+export declare const RegexDetector: SecretDetection.Detector;

package/dist/regex.js

@@ -0,0 +1,27 @@
+import { SecretPatterns } from "./patterns.js";
+export const RegexDetector = {
+    id: "standalone-regex",
+    version: "2",
+    async detect({ text, signal }) {
+        signal?.throwIfAborted();
+        const found = SecretPatterns.oauthCodes(text).map((span) => ({
+            ...span,
+            kind: "authorization_code",
+        }));
+        for (const pattern of SecretPatterns.standalone) {
+            for (const match of text.matchAll(new RegExp(pattern.source, pattern.flags))) {
+                found.push({ start: match.index, end: match.index + match[0].length, kind: "token" });
+            }
+        }
+        found.sort((a, b) => a.start - b.start || b.end - a.end);
+        const findings = [];
+        for (const finding of found) {
+            const previous = findings.at(-1);
+            if (previous && finding.start < previous.end)
+                previous.end = Math.max(previous.end, finding.end);
+            else
+                findings.push(finding);
+        }
+        return { findings, complete: true };
+    },
+};

package/package.json

@@ -1,6 +1,34 @@
 {
   "name": "@ericsanchezok/synergy-secret-detection",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "3.1.0",
+  "type": "module",
+  "license": "MIT",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    },
+    "./patterns": {
+      "types": "./dist/patterns.d.ts",
+      "import": "./dist/patterns.js"
+    },
+    "./evaluation": {
+      "types": "./dist/evaluation.d.ts",
+      "import": "./dist/evaluation.js"
+    }
+  },
+  "scripts": {
+    "typecheck": "tsc --noEmit",
+    "build": "bun ../../script/build-workspace.ts packages/secret-detection && tsc -p tsconfig.build.json",
+    "test": "bun test test/",
+    "evaluate": "bun script/evaluate.ts"
+  },
+  "files": [
+    "dist",
+    "src",
+    "README.md"
+  ],
+  "dependencies": {
+    "zod": "4.6.7"
+  }
+}

package/src/detector.ts

@@ -0,0 +1,101 @@
+import { z } from "zod"
+
+export namespace SecretDetection {
+  export const Source = z.enum(["user_message", "tool_output", "credential_file"])
+  export type Source = z.infer<typeof Source>
+  export const Finding = z.object({
+    start: z.number().int().nonnegative(),
+    end: z.number().int().positive(),
+    kind: z.string().min(1),
+    score: z.number().min(0).max(1).optional(),
+  })
+  export type Finding = z.infer<typeof Finding>
+  export const Result = z.discriminatedUnion("complete", [
+    z.object({ complete: z.literal(true), findings: z.array(Finding) }),
+    z.object({
+      complete: z.literal(false),
+      findings: z.array(Finding),
+      reason: z.enum(["input_limit", "unsupported"]),
+    }),
+  ])
+  export type Result = z.infer<typeof Result>
+  export interface Input {
+    text: string
+    source: Source
+    signal?: AbortSignal
+  }
+  export interface Detector {
+    readonly id: string
+    readonly version: string
+    detect(input: Input): Promise<Result>
+  }
+  export class Error extends globalThis.Error {
+    constructor(readonly code: "aborted" | "timeout" | "invalid_result" | "detector_failed" | "incomplete") {
+      super(`Secret detection failed: ${code}`)
+      this.name = "SecretDetectionError"
+    }
+  }
+
+  function splitsSurrogate(text: string, offset: number) {
+    const left = text.charCodeAt(offset - 1)
+    const right = text.charCodeAt(offset)
+    return left >= 0xd800 && left <= 0xdbff && right >= 0xdc00 && right <= 0xdfff
+  }
+
+  export function validate(text: string, result: unknown): Result {
+    const parsed = Result.safeParse(result)
+    if (!parsed.success) throw new Error("invalid_result")
+    let end = 0
+    for (const finding of parsed.data.findings) {
+      if (
+        finding.start < end ||
+        finding.end <= finding.start ||
+        finding.end > text.length ||
+        splitsSurrogate(text, finding.start) ||
+        splitsSurrogate(text, finding.end)
+      )
+        throw new Error("invalid_result")
+      end = finding.end
+    }
+    return parsed.data
+  }
+
+  export async function detect(
+    detector: Detector,
+    input: Input,
+    options: { timeoutMs?: number } = {},
+  ): Promise<Result> {
+    if (input.signal?.aborted) throw new Error("aborted")
+    if (options.timeoutMs !== undefined && (!Number.isFinite(options.timeoutMs) || options.timeoutMs <= 0)) {
+      throw new RangeError("timeoutMs must be a positive finite number")
+    }
+    const controller = new AbortController()
+    let rejectAbort!: (error: Error) => void
+    const cancelled = new Promise<never>((_, reject) => {
+      rejectAbort = reject
+    })
+    const cancel = (code: "aborted" | "timeout") => {
+      rejectAbort(new Error(code))
+      controller.abort()
+    }
+    const onAbort = () => cancel("aborted")
+    input.signal?.addEventListener("abort", onAbort, { once: true })
+    const timer = options.timeoutMs === undefined ? undefined : setTimeout(() => cancel("timeout"), options.timeoutMs)
+    try {
+      const result = await Promise.race([
+        Promise.resolve().then(() =>
+          detector.detect({ text: input.text, source: input.source, signal: controller.signal }),
+        ),
+        cancelled,
+      ])
+      if (input.signal?.aborted) throw new Error("aborted")
+      return validate(input.text, result)
+    } catch (error) {
+      if (error instanceof Error) throw error
+      throw new Error("detector_failed")
+    } finally {
+      if (timer !== undefined) clearTimeout(timer)
+      input.signal?.removeEventListener("abort", onAbort)
+    }
+  }
+}

package/src/evaluation.ts

@@ -0,0 +1,166 @@
+import { z } from "zod"
+import { SecretDetection } from "./detector.js"
+
+export namespace SecretEvaluation {
+  const Span = SecretDetection.Finding.pick({ start: true, end: true })
+  export const Case = z.object({
+    id: z.string().min(1),
+    group: z.string().min(1),
+    text: z.string(),
+    source: SecretDetection.Source,
+    expected: z.array(Span),
+  })
+  export type Case = z.infer<typeof Case>
+  export const Dataset = z.object({ version: z.string().min(1), cases: z.array(Case).min(1) })
+  export type Dataset = z.infer<typeof Dataset>
+
+  interface Counts {
+    expected: number
+    predicted: number
+    exactMatches: number
+    fullyCovered: number
+    falsePositives: number
+    cleanCases: number
+    completedCleanCases: number
+    falsePositiveCases: number
+    failures: number
+    incomplete: number
+  }
+  function counts(): Counts {
+    return {
+      expected: 0,
+      predicted: 0,
+      exactMatches: 0,
+      fullyCovered: 0,
+      falsePositives: 0,
+      cleanCases: 0,
+      completedCleanCases: 0,
+      falsePositiveCases: 0,
+      failures: 0,
+      incomplete: 0,
+    }
+  }
+  function metrics(count: Counts) {
+    const ratio = (a: number, b: number) => (b ? a / b : null)
+    return {
+      ...count,
+      exactPrecision: ratio(count.exactMatches, count.predicted),
+      exactRecall: ratio(count.exactMatches, count.expected),
+      fullCoverageRecall: ratio(count.fullyCovered, count.expected),
+      cleanMessageFalsePositiveRate: ratio(count.falsePositiveCases, count.completedCleanCases),
+    }
+  }
+
+  export async function evaluate(
+    detector: SecretDetection.Detector,
+    cases: Case[],
+    options: { timeoutMs?: number } = {},
+  ) {
+    const total = counts()
+    const groups = new Map<string, Counts>()
+    const rows: Array<
+      {
+        id: string
+        group: string
+        status: "complete" | "incomplete" | "failed"
+        error?: string
+        durationMs: number
+      } & Counts
+    > = []
+    const ids = new Set<string>()
+    for (const sample of cases) {
+      Case.parse(sample)
+      if (ids.has(sample.id)) throw new Error("Duplicate evaluation case id")
+      ids.add(sample.id)
+      SecretDetection.validate(sample.text, {
+        complete: true,
+        findings: sample.expected.map((span) => ({ ...span, kind: "label" })),
+      })
+    }
+    for (const sample of cases) {
+      const count = counts()
+      count.expected = sample.expected.length
+      count.cleanCases = sample.expected.length === 0 ? 1 : 0
+      const start = performance.now()
+      let status: "complete" | "incomplete" | "failed" = "complete"
+      let error: string | undefined
+      try {
+        const result = await SecretDetection.detect(detector, sample, options)
+        if (!result.complete) {
+          status = "incomplete"
+          count.incomplete = 1
+        }
+        count.predicted = result.findings.length
+        count.exactMatches = sample.expected.filter((label) =>
+          result.findings.some((f) => f.start === label.start && f.end === label.end),
+        ).length
+        count.fullyCovered = sample.expected.filter((label) =>
+          result.findings.some((f) => f.start <= label.start && f.end >= label.end),
+        ).length
+        count.falsePositives = result.findings.filter(
+          (f) => !sample.expected.some((label) => f.start < label.end && f.end > label.start),
+        ).length
+        count.completedCleanCases = result.complete ? count.cleanCases : 0
+        count.falsePositiveCases = count.completedCleanCases && count.predicted > 0 ? 1 : 0
+      } catch (failure) {
+        status = "failed"
+        count.failures = 1
+        error = failure instanceof SecretDetection.Error ? failure.code : "detector_failed"
+      }
+      const durationMs = performance.now() - start
+      const group = groups.get(sample.group) ?? counts()
+      groups.set(sample.group, group)
+      for (const key of Object.keys(count) as Array<keyof Counts>) {
+        total[key] += count[key]
+        group[key] += count[key]
+      }
+      rows.push({ id: sample.id, group: sample.group, status, ...(error ? { error } : {}), durationMs, ...count })
+    }
+    return {
+      detector: { id: detector.id, version: detector.version },
+      metrics: metrics(total),
+      groups: Object.fromEntries([...groups].map(([key, count]) => [key, metrics(count)])),
+      cases: rows,
+    }
+  }
+
+  export async function benchmark(
+    detector: SecretDetection.Detector,
+    input: SecretDetection.Input,
+    options: { samples: number; warmup: number; timeoutMs?: number },
+  ) {
+    if (
+      !Number.isInteger(options.samples) ||
+      options.samples < 1 ||
+      !Number.isInteger(options.warmup) ||
+      options.warmup < 0
+    )
+      throw new RangeError("Invalid benchmark sample count")
+    const start = performance.now()
+    const first = await SecretDetection.detect(detector, input, options)
+    const firstCallMs = performance.now() - start
+    if (!first.complete) throw new SecretDetection.Error("incomplete")
+    for (let i = 0; i < options.warmup; i++) {
+      const result = await SecretDetection.detect(detector, input, options)
+      if (!result.complete) throw new SecretDetection.Error("incomplete")
+    }
+    const elapsed: number[] = []
+    for (let i = 0; i < options.samples; i++) {
+      const begin = performance.now()
+      const result = await SecretDetection.detect(detector, input, options)
+      elapsed.push(performance.now() - begin)
+      if (!result.complete) throw new SecretDetection.Error("incomplete")
+    }
+    elapsed.sort((a, b) => a - b)
+    const durationMs = elapsed.reduce((a, b) => a + b, 0)
+    return {
+      samples: options.samples,
+      warmup: options.warmup,
+      firstCallMs,
+      p50Ms: elapsed[Math.ceil(elapsed.length * 0.5) - 1],
+      p95Ms: elapsed[Math.ceil(elapsed.length * 0.95) - 1],
+      durationMs,
+      callsPerSecond: durationMs > 0 ? (options.samples * 1000) / durationMs : null,
+    }
+  }
+}

package/src/index.ts

@@ -0,0 +1,2 @@
+export { SecretDetection } from "./detector.js"
+export { RegexDetector } from "./regex.js"

package/src/patterns.ts

@@ -0,0 +1,69 @@
+export namespace SecretPatterns {
+  /**
+   * Single inventory of standalone secret token shapes. Consumers
+   * (observability redaction, heuristic capture) read from here so a new
+   * provider prefix updates one place instead of drifting across copies.
+   */
+  export const standalone: readonly RegExp[] = [
+    /\bsk-[A-Za-z0-9._-]{8,}\b/g,
+    /\bghp_[A-Za-z0-9_]{8,}\b/g,
+    /\bgithub_pat_[A-Za-z0-9_]{8,}\b/g,
+    /\bxox[baprs]-[A-Za-z0-9-]{8,}\b/g,
+    /\bhf_[A-Za-z0-9_]{8,}\b/g,
+    /\bglpat-[A-Za-z0-9-]{8,}\b/g,
+    /\bpk_live_[A-Za-z0-9_]{8,}\b/g,
+    /\brk_live_[A-Za-z0-9_]{8,}\b/g,
+    /\btok_[A-Za-z0-9._-]{8,}\b/g,
+    /\bkey_[A-Za-z0-9._-]{8,}\b/g,
+  ]
+
+  /**
+   * Long-form token matcher used by SmartAllow evidence redaction: stricter
+   * minimum lengths plus a generic high-entropy catch-all.
+   */
+  export function longForm(): RegExp {
+    return /\b(?:sk-[A-Za-z0-9_-]{16,}|sk-proj-[A-Za-z0-9_-]{16,}|github_pat_[A-Za-z0-9_]{16,}|gh[pousr]_[A-Za-z0-9_]{16,}|[A-Za-z0-9+/=_-]{48,})\b/g
+  }
+
+  export const authSchemes: readonly [RegExp, string][] = [
+    [/(Bearer\s+)[A-Za-z0-9._~+/-]+=*/gi, "$1[redacted]"],
+    [/(Basic\s+)[A-Za-z0-9+/=]+/gi, "$1[redacted]"],
+    [/(Digest\s+)[A-Za-z0-9+/=]+/gi, "$1[redacted]"],
+  ]
+
+  export const keyValue = /((?:token|secret|password|authorization|api[_-]?key|cookie)["']?\s*[:=]\s*["']?)[^\s"'&]+/gi
+
+  export const queryParam = /([?&](?:token|secret|password|authorization|api[_-]?key|cookie|key)=)[^&#\s"']+/gi
+
+  export function oauthCodes(text: string): Array<{ start: number; end: number }> {
+    const spans: Array<{ start: number; end: number }> = []
+    for (const url of text.matchAll(/\bhttps?:\/\/[^\s"'<>\\()[\]{}]+/gi)) {
+      let pathname: string
+      try {
+        pathname = new URL(url[0]).pathname
+      } catch {
+        continue
+      }
+      if (!/\/(?:auth|oauth2?|authorize|callback)(?:\/|$)/i.test(pathname)) continue
+      for (const match of url[0].matchAll(/([?&]code=)([^&#]+)/gi)) {
+        if (match[2]!.startsWith("⟦sec:")) continue
+        const start = url.index + match.index + match[1]!.length
+        spans.push({ start, end: start + match[2]!.length })
+      }
+    }
+    return spans
+  }
+
+  /** Replace every standalone token match; returns the text and match count. */
+  export function replaceStandalone(text: string, replacement: string): { text: string; matches: number } {
+    let matches = 0
+    let result = text
+    for (const pattern of standalone) {
+      result = result.replace(pattern, () => {
+        matches++
+        return replacement
+      })
+    }
+    return { text: result, matches }
+  }
+}

package/src/regex.ts

@@ -0,0 +1,27 @@
+import type { SecretDetection } from "./detector.js"
+import { SecretPatterns } from "./patterns.js"
+
+export const RegexDetector: SecretDetection.Detector = {
+  id: "standalone-regex",
+  version: "2",
+  async detect({ text, signal }) {
+    signal?.throwIfAborted()
+    const found: SecretDetection.Finding[] = SecretPatterns.oauthCodes(text).map((span) => ({
+      ...span,
+      kind: "authorization_code",
+    }))
+    for (const pattern of SecretPatterns.standalone) {
+      for (const match of text.matchAll(new RegExp(pattern.source, pattern.flags))) {
+        found.push({ start: match.index, end: match.index + match[0].length, kind: "token" })
+      }
+    }
+    found.sort((a, b) => a.start - b.start || b.end - a.end)
+    const findings: SecretDetection.Finding[] = []
+    for (const finding of found) {
+      const previous = findings.at(-1)
+      if (previous && finding.start < previous.end) previous.end = Math.max(previous.end, finding.end)
+      else findings.push(finding)
+    }
+    return { findings, complete: true }
+  },
+}