@unblocklabs/unblock-memory 0.3.15 → 0.3.16

package/README.md

@@ -1,5 +1,203 @@
 # Unblock Memory
 
+## Response quality tracking (opt-in)
+
+`responseAudit` evaluates bounded human-agent exchanges in the background. It is
+separate from chunk-quality auditing and never changes memories or prompts. It
+creates private response-review tasks, not memory-curation tasks.
+Its primary purpose is tracking delivery quality over time: visible fulfillment,
+deliverable fit, clear underdelivery and its observable reason. Memory gaps are only
+an optional diagnostic lead, not a proxy for performance.
+Only approved Slack sender IDs with trusted `senderKind: human` or owner metadata
+qualify (older Slack records use unknown senderKind even for known owners).
+Explicit bots, unverified identities, internal messages, other senders and thread changes form
+hard boundaries. Synthetic delivery mirrors and gateway-injected answers are excluded.
+Assistant progress messages are grouped with the terminal answer.
+Recognized Slack envelopes are stripped even inside `upstreamUserText`; embedded
+history is not treated as current human text. Ambiguous envelopes are excluded.
+Removed history marks the context as limited; ordinary Markdown/JSON is preserved.
+Human feedback closes when the next assistant turn starts. Still-open feedback,
+no-response exchanges, incomplete/failed turns and oversized inputs are not graded.
+
+```json
+{
+  "responseAudit": {
+    "enabled": true,
+    "sentimentEnabled": true,
+    "senderIds": ["YOUR_SLACK_USER_ID"],
+    "chatTypes": ["direct"],
+    "historyMessages": 6,
+    "lookbackDays": 30,
+    "maxEpisodes": 20,
+    "intervalMinutes": 60,
+    "memoryCorpora": ["memory", "knowledge"]
+  }
+}
+```
+
+This is explicit approval to send those exchanges to TypeSafe. Sender IDs apply
+across the agent's Slack accounts; use only identities approved in all such accounts.
+`memoryCorpora` is optional and separately approves configured **file** corpora for
+memory-gap investigation. Leave it empty to send no indexed memory evidence.
+`typesafe.enabled: false` or missing credentials prevents evaluation. An interval
+of zero means manual-only. Defaults are disabled, no approved senders, direct chats,
+6 preceding visible messages, 30 days, 20 episodes per run and a 60-minute interval.
+`sentimentEnabled` defaults to **true within that opt-in audit**; it does not bypass
+approved senders or TypeSafe credentials. Set it false to omit polarity, annoyance,
+frustration and intensity questions while retaining quality/repair judgments.
+`intervalMinutes` controls their shared cadence; no second sentiment timer is needed.
+The Gateway checks a durable per-agent due time on startup and every minute (no
+agent-turn cron or separate launchd job). First enablement waits one interval;
+restarts preserve the due time and an overdue schedule gets one bounded catch-up,
+not one run per missed interval. Each attempt advances the due time before work,
+including missing-key skips, failures or interrupted runs, to prevent retry storms.
+Changing the interval recalculates the due time from the last scheduled attempt
+(or initial enablement). The Gateway must be running; manual audits do not change
+the automatic schedule. Missing/unreadable credentials skip all quality and sentiment
+inference without failing Gateway startup or normal memory functionality.
+Changing the interval does not invalidate cached results. Changing the sentiment
+toggle selects a separate reporting cohort, so older missing sentiment is not
+treated as neutral; unchanged quality/feedback stages are reused across the toggle.
+
+Operator commands (not agent tools):
+
+```sh
+openclaw memory-responses audit --agent main --dry-run
+openclaw memory-responses audit --agent main
+openclaw memory-responses report --agent main
+openclaw memory-responses report --agent main --episode EPISODE_ID
+openclaw memory-responses report --agent main --sender SLACK_USER_ID --account ACCOUNT_SCOPE --bucket day --since 2026-09-01 --until 2026-10-01
+openclaw memory-responses report --agent main --person PERSON_ID
+openclaw memory-responses tasks --agent main
+openclaw memory-responses review --agent main --id TASK_ID --status deferred --reviewer human --note "Review the linked exchanges before changing preferences"
+openclaw memory-responses annotate --agent main --date 2026-09-18 --kind prompt --note "Known prompt revision deployed"
+openclaw memory-responses retry-failed --agent main
+```
+
+Reports group by scoped human identity as well as task/model/time. Names are not
+identity keys. Existing active people-store links are resolved read-only at assessment
+time; missing links do not prevent analysis. Unknown account scopes stay isolated
+per session. No new identity fields are sent to TypeSafe. Date ranges are UTC with
+an inclusive start and exclusive end. `periodStart` identifies a day/week bucket;
+legacy `week`/`fromWeek`/`toWeek` fields remain aliases. `--task-type` and `--model`
+further narrow comparisons. Human-specific scores are not rankings of the humans:
+task difficulty, feedback habits and selection bias remain important.
+
+Session checkpoints hash bounded active source bytes; unchanged sessions skip
+extraction and all inference. Changed sessions are re-extracted within the existing
+budget, then stage hashes reuse unchanged quality, feedback, sentiment and later
+evidence judgments. Only hashes/counts are checkpointed, never a transcript copy.
+A persisted cursor rotates through discovery and tracked-session reconciliation;
+`deferredByLimit` includes known backlog and a lower-bound marker for unvisited
+sessions. `stages` exposes pending/failed/succeeded counts and exhausted retries
+for the cohort/date range, before person filters. `retry-failed` only resets failed
+work; successful stages remain cached. Source freshness is checked before activation.
+
+Review tasks distinguish concrete delivery shortfalls from high-intensity human
+experience complaints. Stable task keys include exchange, scoped human and issue
+family. Decisions survive rescoring; stale source evidence and superseded findings
+are labeled separately. Review status/provenance never changes the raw judgments.
+Tasks and change annotations are operator-only and stay out of memory/whisperer
+prompts. `--reviewer` records human/agent provenance, not authentication or a new
+permission grant. Task lists disclose their 1,000-item cap. There are no automatic
+dossier updates: review the evidence and approve any concrete preference separately.
+Old cohorts remain stored; the first staged-cohort run does not silently import
+unverified older rubric judgments. Audit-history retention is not automatic.
+
+Two separate TypeSafe requests prevent human feedback from influencing the original
+fulfillment/deliverable-fit grade. The feedback pass distinguishes acceptance,
+correction, continuation, unrelated replies, expressed sentiment, repeated constraints
+and avoidable rework. Current-index memory investigation runs only for a strong
+memory-gap signal: lexical retrieval selects up to three whole short documents from
+approved collections. This is an investigation lead, **not proof of historical
+availability, factual truth, or agent fault**. Tool-call counts do not establish what
+the model saw or whether it should have searched. Unseen artifacts are unassessable.
+
+Deliverable kind/format/scope has its own assessability gate, independent of whether
+execution or external facts can be verified. Feedback attribution distinguishes the
+current answer, earlier behavior, delivery, missing proactive action, external events,
+new work and mixed/unclear targets. A reported forgotten instruction does not prove
+searchable memory existed. A third, separate request examines the original exchange,
+human feedback and available next assistant block for specific reported shortfalls,
+acknowledgment, explicit factual corrections, delivery failures and regressions. These are
+retrospective signals, not independently verified facts and never inputs to the
+original grade. Clean text preceding a synthetic error/delivery notice can be assessed
+as **partial** evidence; the notice itself is excluded and no successful completion
+is inferred. Later evidence is capped at six messages/12K characters; incomplete,
+unsafe or oversized blocks stay explicitly pending/unavailable/oversized. New later
+evidence changes the input hash; only changed assessment stages are re-evaluated,
+within normal audit budgets. Successful stages survive failures in later stages.
+When the next block is unavailable, the third pass uses only the original exchange
+and feedback; it cannot infer a missing delivery from missing later evidence.
+
+Code combines narrow, confident evidence into an **observed outcome**, preserving
+its basis and reason. A concrete original-answer shortfall or later admission takes
+precedence over praise. Broad reported failures are used only when they do not
+depend on a newly introduced requirement. Accurate explanations of earlier mistakes,
+ordinary follow-ups, necessary clarification and unseen work are not automatically
+failures. Sentiment and earlier-workflow complaints remain separate review signals.
+Sentiment includes independent annoyance and frustration yes-probabilities (both
+can apply), plus an expressed-dissatisfaction intensity score from 0 to 3. Intensity
+means no expressed displeasure / restrained displeasure / pointed complaint /
+explicit rejection or loss of trust. It is **not confidence or failure severity**.
+External frustration, brevity and factual corrections alone do not establish
+annoyance or frustration; mixed praise and complaints can still carry both signals.
+Weekly reports show dissatisfaction, annoyance and frustration rates, intensity
+means, unknown counts and their own assessment denominators. Unassessed results
+are never counted as neutral. Sentiment deltas require 20 samples in both periods
+and matching assessment coverage; they remain descriptive, not causal evidence.
+Outcome, evidence basis and failure reasons remain distinct: a correction does not
+automatically mean `incorrect_claim`. Confident reason judgments and direct
+delivery/regression admissions supply reasons; otherwise `reasonStatus` is
+`uncertain`. `reasonDetails` retain each label's source and strength, distinguishing
+Choice confidence from Noul yes-probability. Multiple supported reasons can coexist.
+`reportVersion` identifies composition/reporting semantics independently of the
+judge rubric, allowing cached judgments to be re-reported without re-inference.
+
+Results live in the agent's private `unblock-memory/response-audit.sqlite`, outside
+the memory index. It stores judgments and source event references/hashes, not copies
+of conversations. Identical successful inputs are cached; source rewrites invalidate
+in-scope results on the next scan. Reports partition by fixed judge/rubric/context
+configuration, UTC week, task type and agent model. They expose eligible/assessed
+counts, excluded cases, confidence-qualified score means with per-dimension denominators, rework rates with Wilson
+intervals, and evidence IDs. Small groups (<20) are marked explicitly. Confidence
+thresholds are provisional, not calibrated guarantees. Human-reviewed evaluation
+data is still needed before drawing performance conclusions.
+Reports include dated clear-underdelivery examples and reason counts. Descriptive
+score deltas compare successive available UTC weeks within the same task type,
+agent model and rubric/configuration, with at least 20 confident scores per dimension
+in each period and unchanged scored coverage; changed coverage withholds the score
+delta. Outcome trends show acknowledgment, reported-shortfall and unknown rates
+against **all evaluated exchanges**, with at least 20 evaluated exchanges per period.
+Read the three rates together: fewer acknowledgments can mean more unknowns, not
+more failures. Every delta includes before/after values, sample counts, denominator
+and coverage-change flags. Unknown task types/models cannot produce deltas. These
+are not statistical change-point detections or proof of causality; model/version
+changes remain visible as separate groups rather than silently mixing cohorts.
+The legacy `observedSuccessRate` group field remains acknowledgment / known outcomes
+for compatibility, but is not used for trends. Unknowns are never successes.
+Coverage changes and threshold variability can move rates; acknowledgment is not
+factual verification. Week buckets
+may be partial, and several exchanges in one session are not independent. Wilson
+intervals are descriptive, not calibrated confidence about overall agent ability.
+
+Each run selects at most 100 recent sessions for inference, each at most 2,000 active events/2M
+characters; episodes must fit 24K characters and six feedback messages without
+truncating the answer. Coverage counts describe the scanned sessions; only episodes
+within `lookbackDays` are judged. Caps, failures and no-feedback cases remain visible.
+Saved sessions in the report window are also reconciled independently of that
+selection, so removing an entire active branch retires its scores. Oversized saved
+sessions defer reconciliation rather than being treated as deleted; the report
+exposes `reconciledSessions` and `reconciliationDeferred`. All reconciliation shares
+the run deadline. Freshness checks compare the assessed episode, not unrelated
+later session activity. Actual snapshot races do not exhaust provider retries.
+The whole run has a two-minute deadline, at most three provider attempts per input (ten-minute
+backoff), and a cross-process lease. Scheduling never starts inference on the agent
+turn path or boots a QMD manager. No model downloads or source re-indexing occur.
+The report is observational: different task mixes, selective human replies and judge
+changes can produce apparent trends. It does not automatically declare regressions,
+rewrite prompts, or treat silence as success.
+
 ## Review and diagnostics
 
 - `memory_diagnostics` reports credential **availability only**, per-agent process-local

package/dist/src/config.d.ts

@@ -1,3 +1,4 @@
+import { type ResponseAuditConfig } from "./response-config.js";
 export type FileCorpusConfig = {
     name: string;
     kind: "files";
@@ -40,6 +41,7 @@ export type UnblockMemoryConfig = {
         enabled: boolean;
         corpora: readonly string[];
     };
+    responseAudit: ResponseAuditConfig;
     people: {
         enabled: boolean;
         whisperer: {

package/dist/src/config.js

@@ -1,4 +1,5 @@
 import { isAbsolute } from "node:path";
+import { resolveResponseAudit } from "./response-config.js";
 const DEFAULT_PATHS = ["MEMORY.md", "USER.md", "memory/**/*.md"];
 const DEFAULT_SESSION_MAX_EXPANDED_TOKENS = 500;
 const MAX_SESSION_MAX_EXPANDED_TOKENS = 10_000;
@@ -262,6 +263,7 @@ export function resolveConfig(value) {
             typesafe: { ...DEFAULT_TYPESAFE_CONFIG },
             qualityAudit: { ...DEFAULT_QUALITY_AUDIT },
             evidenceReview: { enabled: false, corpora: [] },
+            responseAudit: resolveResponseAudit(undefined, DEFAULT_CORPORA),
             people: DEFAULT_PEOPLE_CONFIG,
             skillWhisperer: DEFAULT_SKILL_WHISPERER,
             memoryWhisperer: { ...DEFAULT_MEMORY_WHISPERER },
@@ -271,7 +273,7 @@ export function resolveConfig(value) {
         throw new Error("unblock-memory config must be an object");
     }
     const config = value;
-    assertOnlyKeys(config, ["corpora", "keepEmbeddingModelWarm", "analysis", "people", "skillWhisperer", "memoryWhisperer", "typesafe", "qualityAudit", "evidenceReview"], "config");
+    assertOnlyKeys(config, ["corpora", "keepEmbeddingModelWarm", "analysis", "people", "skillWhisperer", "memoryWhisperer", "typesafe", "qualityAudit", "evidenceReview", "responseAudit"], "config");
     const corpora = resolveCorpora(config.corpora);
     const people = resolvePeople(config.people);
     let evidenceReview = { enabled: false, corpora: [] };
@@ -347,5 +349,6 @@ export function resolveConfig(value) {
     return { corpora, keepEmbeddingModelWarm, analysis: analysisConfig, people, skillWhisperer,
         qualityAudit: resolveQualityAudit(config.qualityAudit, corpora),
         evidenceReview,
+        responseAudit: resolveResponseAudit(config.responseAudit, corpora),
         memoryWhisperer: resolveMemoryWhisperer(config.memoryWhisperer, corpora), typesafe: resolveTypeSafe(config.typesafe) };
 }

package/dist/src/plugin.js

@@ -12,6 +12,7 @@ import { registerMemoryWhisperer } from "./memory-whisperer.js";
 import { getContext } from "./tool-context.js";
 import { WhispererDiagnostics } from "./diagnostics.js";
 import { registerReviewTools } from "./review-tools.js";
+import { registerResponseAudit } from "./response-runtime.js";
 const searchParameters = Type.Object({
     query: Type.String({ pattern: "\\S" }),
     corpora: Type.Optional(Type.Array(Type.String({ pattern: "\\S" }), { minItems: 1 })),
@@ -413,6 +414,7 @@ export function resolveFlushPlan(params = {}) {
 }
 export function registerUnblockMemory(api) {
     const config = resolveConfig(api.pluginConfig);
+    registerResponseAudit(api, config);
     if (api.registrationMode === "cli-metadata")
         return;
     const runtime = new QmdMemoryRuntime(config.corpora, {

package/dist/src/response-audit.d.ts

@@ -0,0 +1,87 @@
+import type { UnblockMemoryConfig } from "./config.js";
+import type { ResolvedSource } from "./sources.js";
+export declare function responseCohort(config: UnblockMemoryConfig["responseAudit"]): string;
+export type ResponseAuditOptions = {
+    agentId: string;
+    databasePath: string;
+    storePath: string;
+    indexPath: string;
+    config: UnblockMemoryConfig;
+    sources: readonly ResolvedSource[];
+    signal?: AbortSignal;
+    dryRun?: boolean;
+    peoplePath?: string;
+};
+/** All inference is outside memory's mutation queue and outside transcript DB transactions. */
+export declare function auditResponses(options: ResponseAuditOptions): Promise<{
+    status: "disabled";
+    reason?: undefined;
+    cohort?: undefined;
+    coverage?: undefined;
+} | {
+    status: "unavailable";
+    reason: string;
+    cohort?: undefined;
+    coverage?: undefined;
+} | {
+    status: "already_running";
+    reason?: undefined;
+    cohort?: undefined;
+    coverage?: undefined;
+} | {
+    status: "ok" | "dry_run";
+    cohort: string;
+    coverage: {
+        sessions: number;
+        sessionLimitReached: boolean;
+        sessionsOverBudget: number;
+        completedResponses: number;
+        reconciledSessions: number;
+        reconciliationDeferred: number;
+        extractedSessions: number;
+        unchangedSessions: number;
+        stageAttempts: number;
+        stageCacheHits: number;
+        eligible: number;
+        noFeedback: number;
+        pendingFeedback: number;
+        oversized: number;
+        filteredEvents: number;
+        outsideLookback: number;
+        attempted: number;
+        evaluated: number;
+        cachedOrBackoff: number;
+        deferredByLimit: number;
+        failed: number;
+        stale: number;
+    };
+    reason?: undefined;
+} | {
+    status: "unavailable";
+    reason: string;
+    cohort: string;
+    coverage: {
+        sessions: number;
+        sessionLimitReached: boolean;
+        sessionsOverBudget: number;
+        completedResponses: number;
+        reconciledSessions: number;
+        reconciliationDeferred: number;
+        extractedSessions: number;
+        unchangedSessions: number;
+        stageAttempts: number;
+        stageCacheHits: number;
+        eligible: number;
+        noFeedback: number;
+        pendingFeedback: number;
+        oversized: number;
+        filteredEvents: number;
+        outsideLookback: number;
+        attempted: number;
+        evaluated: number;
+        cachedOrBackoff: number;
+        deferredByLimit: number;
+        failed: number;
+        stale: number;
+    };
+}>;

package/dist/src/response-audit.js

@@ -0,0 +1,193 @@
+import { createHash } from "node:crypto";
+import { abortable } from "./abortable.js";
+import { resolveTypeSafeApiKey } from "./typesafe.js";
+import { RESPONSE_EXTRACTOR_VERSION, ResponseTranscriptReader } from "./response-episodes.js";
+import { RESPONSE_RUBRIC_VERSION, RESPONSE_STAGE_VERSIONS, judgeMemoryOpportunity } from "./response-judge.js";
+import { ResponseAuditStore } from "./response-store.js";
+import { responseMemoryCandidates } from "./response-memory.js";
+import { assessResponseStages } from "./response-stages.js";
+import { ResponsePeople } from "./response-identity.js";
+import { TYPESAFE_REVIEW_MODEL } from "./typesafe-review.js";
+export function responseCohort(config) {
+    return RESPONSE_RUBRIC_VERSION + ":" + createHash("sha256").update(JSON.stringify({
+        extractor: RESPONSE_EXTRACTOR_VERSION, historyMessages: config.historyMessages,
+        stages: RESPONSE_STAGE_VERSIONS,
+        sentimentEnabled: config.sentimentEnabled,
+        senderIds: [...config.senderIds].sort(), chatTypes: [...config.chatTypes].sort(), memoryCorpora: [...config.memoryCorpora].sort(),
+    })).digest("hex").slice(0, 16);
+}
+/** All inference is outside memory's mutation queue and outside transcript DB transactions. */
+export async function auditResponses(options) {
+    const { config, agentId } = options;
+    if (!config.responseAudit.enabled || !config.typesafe.enabled)
+        return { status: "disabled" };
+    const signal = AbortSignal.any([AbortSignal.timeout(120_000), ...(options.signal ? [options.signal] : [])]);
+    const cohort = responseCohort(config.responseAudit);
+    let key;
+    if (!options.dryRun) {
+        try {
+            key = await abortable(resolveTypeSafeApiKey(config.typesafe), signal);
+        }
+        catch {
+            return { status: "unavailable", reason: "Credentials unavailable or audit cancelled" };
+        }
+        if (!key)
+            return { status: "unavailable", reason: "TypeSafe API key not configured" };
+    }
+    let reader, store, lease;
+    const people = new ResponsePeople(options.peoplePath);
+    const now = Date.now();
+    const coverage = { sessions: 0, sessionLimitReached: false, sessionsOverBudget: 0, completedResponses: 0,
+        reconciledSessions: 0, reconciliationDeferred: 0,
+        extractedSessions: 0, unchangedSessions: 0,
+        stageAttempts: 0, stageCacheHits: 0,
+        eligible: 0, noFeedback: 0, pendingFeedback: 0, oversized: 0, filteredEvents: 0, outsideLookback: 0,
+        attempted: 0, evaluated: 0, cachedOrBackoff: 0, deferredByLimit: 0, failed: 0, stale: 0 };
+    try {
+        signal.throwIfAborted();
+        if (!options.dryRun) {
+            store = new ResponseAuditStore(options.storePath);
+            lease = store.acquire(now);
+            if (!lease)
+                return { status: "already_running" };
+        }
+        reader = new ResponseTranscriptReader(options.databasePath, agentId);
+        const since = now - config.responseAudit.lookbackDays * 86400_000;
+        store?.reviews.refresh(cohort, since);
+        const cursor = store?.cursor(cohort) ?? "";
+        const tracked = new Set(store?.activeSessions(cohort, since) ?? []);
+        const sessions = [...new Set([...reader.sessions(config.responseAudit, now, cursor).map(s => s.sessionId),
+                ...[...tracked].filter(id => id > cursor)])].sort();
+        coverage.sessionLimitReached = sessions.length > 100;
+        let stoppedForBudget = false;
+        for (const session of sessions.slice(0, 100)) {
+            await new Promise(resolve => setImmediate(resolve));
+            signal.throwIfAborted();
+            coverage.sessions++;
+            const checkpoint = store?.checkpoint(cohort, session);
+            const work = store?.sessionWork(cohort, session, since, now);
+            const snapshot = reader.read(session, config.responseAudit, work?.due ? undefined : checkpoint?.revision);
+            if (snapshot === undefined) {
+                coverage.sessionsOverBudget++;
+                coverage.reconciliationDeferred++;
+                store?.advance(cohort, session);
+                continue;
+            }
+            if (snapshot === null) {
+                store?.observe(cohort, session, []);
+                store?.checkpointForget(cohort, session);
+                coverage.reconciledSessions++;
+                store?.advance(cohort, session);
+                continue;
+            }
+            if ("unchanged" in snapshot) {
+                coverage.unchangedSessions++;
+                coverage.cachedOrBackoff += work?.total ?? 0;
+                if (checkpoint)
+                    for (const key of Object.keys(checkpoint.coverage))
+                        coverage[key] += checkpoint.coverage[key];
+                store?.advance(cohort, session);
+                continue;
+            }
+            coverage.extractedSessions++;
+            if (tracked.has(session))
+                coverage.reconciledSessions++;
+            for (const key of Object.keys(snapshot.coverage))
+                coverage[key] += snapshot.coverage[key];
+            const episodes = snapshot.episodes.filter(e => e.timestamp >= now - config.responseAudit.lookbackDays * 86400_000);
+            coverage.outsideLookback += snapshot.episodes.length - episodes.length;
+            store?.observe(cohort, session, snapshot.episodes);
+            store?.checkpointSave(cohort, session, snapshot.revision, snapshot.coverage);
+            // Oldest first within a session; new data cannot forever starve its existing backlog.
+            for (const e of episodes) {
+                signal.throwIfAborted();
+                if (options.dryRun)
+                    continue;
+                if (!store.needsJudgment(cohort, e, Date.now())) {
+                    coverage.cachedOrBackoff++;
+                    continue;
+                }
+                if (coverage.attempted >= config.responseAudit.maxEpisodes) {
+                    coverage.deferredByLimit++;
+                    continue;
+                }
+                coverage.attempted++;
+                store.attempted(cohort, e, Date.now());
+                try {
+                    const params = { apiKey: key, timeoutMs: config.typesafe.timeoutMs, signal };
+                    const judgment = await abortable(assessResponseStages(store, cohort, e, params, config.responseAudit.sentimentEnabled, coverage), signal);
+                    let memory = { status: "not_requested", candidates: [] };
+                    if (judgment.feedback.memoryGap.noul >= 0.8 && config.responseAudit.memoryCorpora.length) {
+                        try {
+                            const approved = options.sources.filter(s => config.responseAudit.memoryCorpora.includes(s.corpus));
+                            const candidates = responseMemoryCandidates(options.indexPath, approved, e);
+                            const memoryKey = createHash("sha256").update(JSON.stringify([TYPESAFE_REVIEW_MODEL, RESPONSE_STAGE_VERSIONS.memory,
+                                e.id, e.session, e.senderId, e.request, e.answer, e.feedback, candidates,
+                                [...config.responseAudit.memoryCorpora].sort()])).digest("hex");
+                            let judged = store.stage(cohort, e, "memory", memoryKey);
+                            if (judged)
+                                coverage.stageCacheHits++;
+                            else {
+                                store.stageBegin([memoryKey], Date.now());
+                                coverage.stageAttempts++;
+                                judged = await abortable(judgeMemoryOpportunity(e, candidates, params), signal);
+                                store.stageSave(memoryKey, judged, Date.now());
+                            }
+                            memory = { status: "checked", candidates: judged };
+                        }
+                        catch {
+                            memory = { status: "unavailable", candidates: [] };
+                        }
+                    }
+                    signal.throwIfAborted();
+                    const fresh = reader.read(session, config.responseAudit);
+                    if (!fresh?.episodes.some(candidate => candidate.id === e.id && candidate.inputHash === e.inputHash)) {
+                        coverage.stale++;
+                        store.stale(cohort, e);
+                        if (fresh !== undefined) {
+                            store.observe(cohort, session, fresh?.episodes ?? []);
+                            store.checkpointForget(cohort, session);
+                        }
+                        break;
+                    }
+                    store.save(cohort, e, { ...judgment, human: people.resolve(e), references: references(e), agentModel: e.model,
+                        contextLimited: e.contextLimited, memorySearchCalls: e.memorySearchCalls, memory }, Date.now());
+                    coverage.evaluated++;
+                }
+                catch {
+                    coverage.failed++;
+                    if (signal.aborted)
+                        throw new Error("Audit cancelled");
+                }
+            }
+            // Let stop/cancellation handlers run between bounded synchronous session reads.
+            await new Promise(resolve => setImmediate(resolve));
+            store?.advance(cohort, session);
+            if (coverage.attempted >= config.responseAudit.maxEpisodes && session !== sessions.at(-1)) {
+                stoppedForBudget = true;
+                coverage.deferredByLimit++; // At least one unvisited session; precise episode count is not yet known.
+                break;
+            }
+        }
+        if (!coverage.sessionLimitReached && !stoppedForBudget)
+            store?.advance(cohort, "");
+        coverage.deferredByLimit = Math.max(coverage.deferredByLimit, store?.pendingWork(cohort, since, Date.now()) ?? 0);
+        store?.scan(cohort, coverage, Date.now());
+        return { status: options.dryRun ? "dry_run" : "ok", cohort, coverage };
+    }
+    catch {
+        store?.scan(cohort, { ...coverage, interrupted: true }, Date.now());
+        return { status: "unavailable", reason: "Response audit failed or was cancelled", cohort, coverage };
+    }
+    finally {
+        reader?.close();
+        if (lease)
+            store?.release(lease);
+        store?.close();
+        people.close();
+    }
+}
+function references(e) {
+    return { sessionId: e.session.sessionId, request: e.request.map(m => m.seq), answer: e.answer.map(m => m.seq),
+        feedback: e.feedback.map(m => m.seq), followup: e.followup.messages.map(m => m.seq), inputHash: e.inputHash };
+}

package/dist/src/response-config.d.ts

@@ -0,0 +1,13 @@
+import type { ChatType, CorpusConfig } from "./config.js";
+export type ResponseAuditConfig = {
+    enabled: boolean;
+    sentimentEnabled: boolean;
+    senderIds: string[];
+    chatTypes: ChatType[];
+    historyMessages: number;
+    lookbackDays: number;
+    maxEpisodes: number;
+    intervalMinutes: number;
+    memoryCorpora: string[];
+};
+export declare function resolveResponseAudit(value: unknown, corpora: readonly CorpusConfig[]): ResponseAuditConfig;

package/dist/src/response-config.js

@@ -0,0 +1,43 @@
+export function resolveResponseAudit(value, corpora) {
+    const defaults = { enabled: false, sentimentEnabled: true, senderIds: [], chatTypes: ["direct"],
+        historyMessages: 6, lookbackDays: 30, maxEpisodes: 20, intervalMinutes: 60, memoryCorpora: [] };
+    if (value === undefined)
+        return defaults;
+    if (!value || typeof value !== "object" || Array.isArray(value))
+        throw new Error("responseAudit must be an object");
+    const v = value;
+    for (const key of Object.keys(v))
+        if (!Object.hasOwn(defaults, key))
+            throw new Error(`unknown responseAudit property: ${key}`);
+    const enabled = v.enabled ?? false;
+    if (typeof enabled !== "boolean")
+        throw new Error("responseAudit.enabled must be boolean");
+    const sentimentEnabled = v.sentimentEnabled === undefined ? true : v.sentimentEnabled;
+    if (typeof sentimentEnabled !== "boolean")
+        throw new Error("responseAudit.sentimentEnabled must be boolean");
+    const strings = (key) => {
+        const raw = v[key] ?? defaults[key];
+        if (!Array.isArray(raw) || raw.length > 50 || !raw.every((x) => typeof x === "string" && !!x.trim())) {
+            throw new Error(`responseAudit.${key} must be a bounded string array`);
+        }
+        return [...new Set(raw.map(x => x.trim()))];
+    };
+    const senderIds = strings("senderIds"), chatTypes = strings("chatTypes"), memoryCorpora = strings("memoryCorpora");
+    if (!chatTypes.length || !chatTypes.every((x) => ["direct", "group", "channel"].includes(x))) {
+        throw new Error("responseAudit.chatTypes must specify direct, group or channel");
+    }
+    if (enabled && !senderIds.length)
+        throw new Error("responseAudit requires explicit approved human senderIds");
+    if (memoryCorpora.some(name => !corpora.some(c => c.name === name && c.kind === "files"))) {
+        throw new Error("responseAudit.memoryCorpora must name configured file corpora");
+    }
+    const integer = (key, min, max) => {
+        const n = v[key] ?? defaults[key];
+        if (typeof n !== "number" || !Number.isInteger(n) || n < min || n > max)
+            throw new Error(`invalid responseAudit.${key}`);
+        return n;
+    };
+    return { enabled, sentimentEnabled, senderIds, chatTypes, memoryCorpora, historyMessages: integer("historyMessages", 0, 20),
+        lookbackDays: integer("lookbackDays", 1, 90), maxEpisodes: integer("maxEpisodes", 1, 100),
+        intervalMinutes: integer("intervalMinutes", 0, 1440) };
+}