@tangle-network/agent-knowledge 7.0.0 → 7.0.3

package/CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## Unreleased
+
+### Added
+
+- Added `runAgentMemoryLearningExperiment` for matched stateful-versus-stateless memory measurement under one shared cost limit.
+- Added exact paired gain, explicitly labeled transfer probes, and repeated-probe forgetting reports.
+- Limited learning gain to post-first-step probes, averaged repetitions within independent sequences, and added per-candidate intervals.
+- Added recorded arm order for counterbalanced runs and exact sequence references for safe crash recovery.
+- Added abort and resume support, content-addressed comparison and probe evidence, and exact cell artifact hashes.
+
+### Changed
+
+- Memory experiment artifacts and cache identities now record `memoryMode` and a full `comparisonRef`; non-equivalent arms fail comparison.
+
+## 7.0.1
+
+### Changed
+
+- Updated `@tangle-network/agent-eval` to `0.135.4` and `@tangle-network/agent-interface` to `0.37.0` so one installed stack uses the same exact trace-ingestion, source-identity, and interface contracts without older nested copies.
+
 ## 7.0.0
 
 ### Breaking Changes
@@ -49,7 +69,7 @@
 
 - Load `proper-lockfile` with a dynamic import inside the functions that take a lock, instead of at module scope.
   It pulls in `graceful-fs`, which patches Node's `fs` at import time (`fs.close = ...`).
-  workerd exposes those as getter-only accessors, so the assignment threw while Cloudflare validated an uploaded Worker (`Cannot set property close of #<Object> which has only a getter [code: 10021]`), rejecting the whole Worker — including consumers that never take a lock.
+  workerd exposes those as getter-only accessors, so the assignment threw while Cloudflare validated an uploaded Worker (`Cannot set property close of #<Object> which has only a getter [code: 10021]`), rejecting the whole Worker, including consumers that never take a lock.
   `verify:package` now fails on any static import of a module that patches a Node builtin, because `wrangler deploy --dry-run` bundles without executing and cannot see this class of failure.
 
 ## 6.1.8

package/README.md

@@ -239,6 +239,36 @@ Positive external work without a receipt is recorded as incomplete cost accounti
 Use `0` only for a free local path.
 Paid memory improvement defaults to a zero-dollar total limit; set `maxTotalCostUsd` and `maximumEvaluationCostUsd` before enabling paid work.
 
+Use `runAgentMemoryLearningExperiment` to measure whether retained memory helps across ordered steps:
+
+```ts
+import { runAgentMemoryLearningExperiment } from '@tangle-network/agent-knowledge/memory'
+
+const result = await runAgentMemoryLearningExperiment({
+  experimentId: 'support-memory',
+  runDir: 'support-memory',
+  candidates: [memoryCandidate],
+  sequences,
+  seed: 42,
+  reps: 5,
+  armOrder: 'stateful-first',
+  costCeiling: 10,
+})
+
+console.log(result.comparison.gain)
+```
+
+The function runs matched stateful and stateless arms with the same immutable candidate, tasks, executor, policy, seed, and repetitions.
+The stateless arm clears declared scopes between steps; adapters must support scoped `clear`.
+Gain excludes first-step probes and averages candidates and repetitions within each independent sequence.
+Use `transferKey` on later probes for transfer and repeat one `retentionKey` across steps for forgetting.
+Unmarked probes are not assigned those meanings.
+
+Both arms share one cost limit and must have identical comparison references.
+Run independent experiments with opposite `armOrder` values when provider behavior may drift.
+Each saved probe includes the exact scoring input and content hash, so protect the run directory like the memory data itself.
+Pass `signal` to cancel; rerun the same options and directory to resume completed work and cost records.
+
 ## Run benchmarks
 
 `@tangle-network/agent-knowledge/benchmarks` provides common case and report types for:

package/dist/benchmarks/index.js

@@ -1,2 +1,2 @@
-import { A as INDUSTRY_RAG_BENCHMARKS, F as respondToIndustryRagBenchmarkSmokeCase, I as isKnowledgeMemoryBenchmarkCase, L as createInMemoryBenchmarkAdapter, M as buildIndustryMemoryBenchmarkSmokeCases, N as buildIndustryRagBenchmarkSmokeCases, P as respondToIndustryMemoryBenchmarkSmokeCase, R as createNoopMemoryBenchmarkAdapter, a as buildKnowledgeBenchmarkScenarios, c as runKnowledgeBenchmarkSuite, d as summarizeKnowledgeBenchmarkCampaign, i as runMemoryAdapterBenchmark, j as buildFirstPartyMemoryLifecycleBenchmarkCases, k as INDUSTRY_MEMORY_BENCHMARKS, l as scoreKnowledgeBenchmarkArtifact, n as parseKnowledgeBenchmarkJsonl, o as knowledgeBenchmarkJudge, r as parseKnowledgeBenchmarkQrels, s as renderKnowledgeBenchmarkReportMarkdown, t as buildRetrievalBenchmarkCasesFromQrels, u as scoreMemoryBenchmarkArtifact } from "../benchmarks-C8L7HJb4.js";
+import { B as createNoopMemoryBenchmarkAdapter, F as buildIndustryRagBenchmarkSmokeCases, I as respondToIndustryMemoryBenchmarkSmokeCase, L as respondToIndustryRagBenchmarkSmokeCase, M as INDUSTRY_RAG_BENCHMARKS, N as buildFirstPartyMemoryLifecycleBenchmarkCases, P as buildIndustryMemoryBenchmarkSmokeCases, R as isKnowledgeMemoryBenchmarkCase, a as buildKnowledgeBenchmarkScenarios, c as runKnowledgeBenchmarkSuite, d as summarizeKnowledgeBenchmarkCampaign, i as runMemoryAdapterBenchmark, j as INDUSTRY_MEMORY_BENCHMARKS, l as scoreKnowledgeBenchmarkArtifact, n as parseKnowledgeBenchmarkJsonl, o as knowledgeBenchmarkJudge, r as parseKnowledgeBenchmarkQrels, s as renderKnowledgeBenchmarkReportMarkdown, t as buildRetrievalBenchmarkCasesFromQrels, u as scoreMemoryBenchmarkArtifact, z as createInMemoryBenchmarkAdapter } from "../benchmarks-CPl4AO1k.js";
 export { INDUSTRY_MEMORY_BENCHMARKS, INDUSTRY_RAG_BENCHMARKS, buildFirstPartyMemoryLifecycleBenchmarkCases, buildIndustryMemoryBenchmarkSmokeCases, buildIndustryRagBenchmarkSmokeCases, buildKnowledgeBenchmarkScenarios, buildRetrievalBenchmarkCasesFromQrels, createInMemoryBenchmarkAdapter, createNoopMemoryBenchmarkAdapter, isKnowledgeMemoryBenchmarkCase, knowledgeBenchmarkJudge, parseKnowledgeBenchmarkJsonl, parseKnowledgeBenchmarkQrels, renderKnowledgeBenchmarkReportMarkdown, respondToIndustryMemoryBenchmarkSmokeCase, respondToIndustryRagBenchmarkSmokeCase, runKnowledgeBenchmarkSuite, runMemoryAdapterBenchmark, scoreKnowledgeBenchmarkArtifact, scoreMemoryBenchmarkArtifact, summarizeKnowledgeBenchmarkCampaign };

package/dist/{benchmarks-C8L7HJb4.js → benchmarks-CPl4AO1k.js}

@@ -1264,6 +1264,8 @@ function isMemoryProviderActor(actor) {
 //#endregion
 //#region src/memory/lifecycle.ts
 const DEFAULT_MEMORY_CLEANUP_TIMEOUT_MS = 18e4;
+const MEMORY_OPERATION_CANCELLATION_TIMEOUT_MS = 4e3;
+const MEMORY_CAMPAIGN_DISPATCH_SHUTDOWN_TIMEOUT_MS = 5e3;
 var AgentMemoryLifecycleTimeoutError = class extends Error {
 	operation;
 	timeoutMs;
@@ -1296,16 +1298,37 @@ function releaseMemoryAdapterCreatedAfterAbort(input) {
 		} catch {}
 	}, () => void 0);
 }
+async function createBoundedMemoryAdapter(input) {
+	input.signal?.throwIfAborted();
+	const abortController = new AbortController();
+	const creation = Promise.resolve().then(() => input.create(abortController.signal));
+	releaseMemoryAdapterCreatedAfterAbort({
+		creation,
+		signal: abortController.signal,
+		dispose: input.dispose
+	});
+	return runBoundedMemoryLifecycle({
+		operation: input.operation,
+		timeoutMs: input.timeoutMs,
+		signal: input.signal,
+		abortController,
+		run: () => creation
+	});
+}
 function resolveMemoryCleanupTimeoutMs(value, label) {
 	const timeoutMs = value ?? 18e4;
 	if (!Number.isSafeInteger(timeoutMs) || timeoutMs <= 0) throw new Error(`${label} cleanupTimeoutMs must be a positive safe integer`);
 	return timeoutMs;
 }
 async function runBoundedMemoryLifecycle(input) {
+	input.signal?.throwIfAborted();
+	if (input.cancellationTimeoutMs !== void 0 && (!Number.isSafeInteger(input.cancellationTimeoutMs) || input.cancellationTimeoutMs <= 0)) throw new Error(`${input.operation} cancellationTimeoutMs must be a positive safe integer`);
 	const priorTimeout = input.resource ? timedOutResources.get(input.resource) : void 0;
 	if (priorTimeout) throw new AgentMemoryLifecycleUnsafeError(input.operation, priorTimeout);
 	let timeout;
+	let cancellationTimeout;
 	let timeoutError;
+	let relayAbort;
 	const work = Promise.resolve().then(input.run);
 	if (input.resource) {
 		const resource = input.resource;
@@ -1323,12 +1346,41 @@ async function runBoundedMemoryLifecycle(input) {
 			reject(timeoutError);
 		}, input.timeoutMs);
 	});
+	const signal = input.signal;
+	const cancellation = signal ? new Promise((_, reject) => {
+		relayAbort = () => {
+			const error = memoryLifecycleAbortError(input.operation);
+			input.abortController?.abort(error);
+			if (input.cancellationTimeoutMs === void 0) {
+				reject(signal.reason ?? error);
+				return;
+			}
+			cancellationTimeout = setTimeout(() => {
+				timeoutError = new AgentMemoryLifecycleTimeoutError(`${input.operation} cancellation`, input.cancellationTimeoutMs);
+				if (input.resource) timedOutResources.set(input.resource, timeoutError);
+				reject(timeoutError);
+			}, input.cancellationTimeoutMs);
+		};
+		if (signal.aborted) relayAbort();
+		else signal.addEventListener("abort", relayAbort, { once: true });
+	}) : void 0;
 	try {
-		return await Promise.race([work, deadline]);
+		return await Promise.race([
+			work,
+			deadline,
+			...cancellation ? [cancellation] : []
+		]);
 	} finally {
 		if (timeout) clearTimeout(timeout);
+		if (cancellationTimeout) clearTimeout(cancellationTimeout);
+		if (relayAbort) signal?.removeEventListener("abort", relayAbort);
 	}
 }
+function memoryLifecycleAbortError(operation) {
+	const error = /* @__PURE__ */ new Error(`${operation} aborted`);
+	error.name = "AbortError";
+	return error;
+}
 function memoryRecoveryDelayMs(adapter) {
 	const isolation = adapter.branchIsolation;
 	if (isolation?.mode !== "scoped" || isolation.processExitSafe !== false) return 0;
@@ -1336,13 +1388,29 @@ function memoryRecoveryDelayMs(adapter) {
 	if (typeof delayMs !== "number" || !Number.isSafeInteger(delayMs) || delayMs <= 0) throw new Error(`${adapter.id}: processExitSafe=false requires a positive recoveryDelayMs for abandoned writes`);
 	return delayMs;
 }
-async function sleepForMemoryRecovery(delayMs, assertOwned, timeoutMs = delayMs, operation = "memory recovery visibility wait") {
+async function sleepForMemoryRecovery(delayMs, assertOwned, timeoutMs = delayMs, operation = "memory recovery visibility wait", signal) {
 	if (delayMs <= 0) return;
 	if (!Number.isSafeInteger(timeoutMs) || timeoutMs <= 0) throw new Error(`${operation} timeout must be a positive safe integer`);
-	await new Promise((resolve) => setTimeout(resolve, Math.min(delayMs, timeoutMs)));
+	signal?.throwIfAborted();
+	await waitForMemoryRecoveryDelay(Math.min(delayMs, timeoutMs), signal, operation);
 	await assertOwned();
 	if (delayMs > timeoutMs) throw new AgentMemoryLifecycleTimeoutError(operation, timeoutMs);
 }
+async function waitForMemoryRecoveryDelay(delayMs, signal, operation) {
+	await new Promise((resolve, reject) => {
+		let timeout;
+		const abort = () => {
+			if (timeout) clearTimeout(timeout);
+			reject(signal?.reason ?? memoryLifecycleAbortError(operation));
+		};
+		timeout = setTimeout(() => {
+			signal?.removeEventListener("abort", abort);
+			resolve();
+		}, delayMs);
+		if (signal?.aborted) abort();
+		else signal?.addEventListener("abort", abort, { once: true });
+	});
+}
 function createMemoryExecutionPool(limit) {
 	let active = 0;
 	const waiters = [];
@@ -2713,6 +2781,6 @@ function defaultDocumentTarget(documentId, targetKind) {
 	}
 }
 //#endregion
-export { INDUSTRY_RAG_BENCHMARKS as A, memoryWriteResultToSourceRecord as B, appendDurableJournalEvent as C, reconcileInterruptedMemoryPaidCalls as D, readActiveAttemptJournal as E, respondToIndustryRagBenchmarkSmokeCase as F, retrievalRecallJudge as G, partitionRetrievalScenarios as H, isKnowledgeMemoryBenchmarkCase as I, scoreRetrievalArtifact as K, createInMemoryBenchmarkAdapter as L, buildIndustryMemoryBenchmarkSmokeCases as M, buildIndustryRagBenchmarkSmokeCases as N, reserveRecoveryAttempts as O, respondToIndustryMemoryBenchmarkSmokeCase as P, createNoopMemoryBenchmarkAdapter as R, appendAttemptJournalEvent as S, hasSettledPaidCall as T, retrievalConfigFromSurface as U, buildRetrievalEvalDispatch as V, retrievalConfigSurface as W, memoryRecoveryDelayMs as _, buildKnowledgeBenchmarkScenarios as a, runBoundedMemoryLifecycle as b, runKnowledgeBenchmarkSuite as c, summarizeKnowledgeBenchmarkCampaign as d, acquireAgentMemoryRunLease as f, createMemoryExecutionPool as g, DEFAULT_MEMORY_CLEANUP_TIMEOUT_MS as h, runMemoryAdapterBenchmark as i, buildFirstPartyMemoryLifecycleBenchmarkCases as j, INDUSTRY_MEMORY_BENCHMARKS as k, scoreKnowledgeBenchmarkArtifact as l, AgentMemoryLifecycleUnsafeError as m, parseKnowledgeBenchmarkJsonl as n, knowledgeBenchmarkJudge as o, AgentMemoryLifecycleTimeoutError as p, parseKnowledgeBenchmarkQrels as r, renderKnowledgeBenchmarkReportMarkdown as s, buildRetrievalBenchmarkCasesFromQrels as t, scoreMemoryBenchmarkArtifact as u, releaseMemoryAdapterCreatedAfterAbort as v, assertNoInterruptedPaidCalls as w, sleepForMemoryRecovery as x, resolveMemoryCleanupTimeoutMs as y, memoryHitToSourceRecord as z };
+export { reserveRecoveryAttempts as A, createNoopMemoryBenchmarkAdapter as B, sleepForMemoryRecovery as C, hasSettledPaidCall as D, assertNoInterruptedPaidCalls as E, buildIndustryRagBenchmarkSmokeCases as F, retrievalConfigFromSurface as G, memoryWriteResultToSourceRecord as H, respondToIndustryMemoryBenchmarkSmokeCase as I, scoreRetrievalArtifact as J, retrievalConfigSurface as K, respondToIndustryRagBenchmarkSmokeCase as L, INDUSTRY_RAG_BENCHMARKS as M, buildFirstPartyMemoryLifecycleBenchmarkCases as N, readActiveAttemptJournal as O, buildIndustryMemoryBenchmarkSmokeCases as P, isKnowledgeMemoryBenchmarkCase as R, runBoundedMemoryLifecycle as S, appendDurableJournalEvent as T, buildRetrievalEvalDispatch as U, memoryHitToSourceRecord as V, partitionRetrievalScenarios as W, MEMORY_OPERATION_CANCELLATION_TIMEOUT_MS as _, buildKnowledgeBenchmarkScenarios as a, memoryRecoveryDelayMs as b, runKnowledgeBenchmarkSuite as c, summarizeKnowledgeBenchmarkCampaign as d, acquireAgentMemoryRunLease as f, MEMORY_CAMPAIGN_DISPATCH_SHUTDOWN_TIMEOUT_MS as g, DEFAULT_MEMORY_CLEANUP_TIMEOUT_MS as h, runMemoryAdapterBenchmark as i, INDUSTRY_MEMORY_BENCHMARKS as j, reconcileInterruptedMemoryPaidCalls as k, scoreKnowledgeBenchmarkArtifact as l, AgentMemoryLifecycleUnsafeError as m, parseKnowledgeBenchmarkJsonl as n, knowledgeBenchmarkJudge as o, AgentMemoryLifecycleTimeoutError as p, retrievalRecallJudge as q, parseKnowledgeBenchmarkQrels as r, renderKnowledgeBenchmarkReportMarkdown as s, buildRetrievalBenchmarkCasesFromQrels as t, scoreMemoryBenchmarkArtifact as u, createBoundedMemoryAdapter as v, appendAttemptJournalEvent as w, resolveMemoryCleanupTimeoutMs as x, createMemoryExecutionPool as y, createInMemoryBenchmarkAdapter as z };
 
-//# sourceMappingURL=benchmarks-C8L7HJb4.js.map
+//# sourceMappingURL=benchmarks-CPl4AO1k.js.map