ultravisor 1.3.2 → 1.3.4

package/docs/features/persistence-via-databeacon.md

@@ -1,10 +1,12 @@
 # Persistence via retold-databeacon
 
-**Status: planned + foundations in progress.** This doc is the cross-session plan
-for routing ultravisor's queue and manifest persistence through retold-databeacon
-instead of the specialized `ultravisor-queue-beacon` and `ultravisor-manifest-beacon`
-modules. It captures the architectural decision, the work breakdown, and the
-hand-off state so a fresh context can resume from here.
+**Status: shipped. Legacy modules removed 2026-05-03.** This doc is the
+cross-session plan for routing ultravisor's queue and manifest persistence
+through retold-databeacon instead of the specialized `ultravisor-queue-beacon`
+and `ultravisor-manifest-beacon` modules (now deleted from the monorepo —
+see [What changed and why](#what-changed-and-why) for the historical context).
+It captures the architectural decision, the work breakdown, and the hand-off
+state so a fresh context can resume from here.
 
 If you are picking this up cold, read this doc top-to-bottom before touching
 code — the decisions below are load-bearing and the cross-module dependencies
@@ -12,15 +14,18 @@ are not obvious from the source.
 
 ## What changed and why
 
-Before this redesign:
-- `ultravisor-queue-beacon` advertises capability `QueuePersistence` with
-  `QP_*` actions. Ships a `QueuePersistenceProviderBase` and a default
+Before this redesign (the modules described here have since been deleted from
+the monorepo on 2026-05-03 — kept here for historical context):
+- `ultravisor-queue-beacon` advertised capability `QueuePersistence` with
+  `QP_*` actions. Shipped a `QueuePersistenceProviderBase` and a default
   `MemoryQueuePersistenceProvider`.
-- `ultravisor-manifest-beacon` advertises `ManifestStore` with `MS_*` actions.
-  Ships a `ManifestStoreProviderBase` and `MemoryManifestStoreProvider`.
+- `ultravisor-manifest-beacon` advertised `ManifestStore` with `MS_*` actions.
+  Shipped a `ManifestStoreProviderBase` and `MemoryManifestStoreProvider`.
 - `Ultravisor-QueuePersistenceBridge.cjs` and
-  `Ultravisor-ManifestStoreBridge.cjs` dispatch into the corresponding
-  capability when a beacon is connected, fall back to local storage otherwise.
+  `Ultravisor-ManifestStoreBridge.cjs` dispatched into the corresponding
+  capability when a beacon was connected, fell back to local storage otherwise.
+  Both bridges still exist inside ultravisor — they now translate `QP_*` /
+  `MS_*` semantics into `MeadowProxy` calls against retold-databeacon.
 - `bootstrap-flush` (already shipped) replays locally-buffered writes into a
   newly-connected beacon via per-beacon HWM tracking persisted to
   `<DataPath>/persistence-bridge-hwm.json`.
@@ -36,10 +41,12 @@ The redesign:
   `Request` calls** (Method + Path + Body) instead of dispatching to a
   specialized capability. The schema (table names, column shapes) is owned
   by ultravisor; retold-databeacon is a generic gateway.
-- **The two specialized beacon modules are not deleted.** They stay as the
-  reference implementation of the Provider pattern, useful for embedded /
-  niche deployments that don't want retold-databeacon's REST surface. They
-  are no longer the lab's recommended path.
+- **The two specialized beacon modules have been deleted (2026-05-03).**
+  They were marked legacy throughout Session 4 and the migration path proved
+  out across all retold-databeacon-supported engines (mysql / mssql / postgres
+  / sqlite). Embedded deployments that prefer in-process Provider-pattern
+  persistence can still subclass the bridges' fallback path; the QP/MS
+  capability surfaces themselves are gone from the monorepo.
 - **The lab's UV detail view gains a "persistence databeacon" picker.** The
   operator picks a running retold-databeacon (which itself has an
   engine+database picker via `lab-engine-database-picker`); ultravisor's
@@ -1120,21 +1127,31 @@ engine-coverage work — same files get touched.
 
 #### Legacy beacon deprecation
 
-`ultravisor-queue-beacon` and `ultravisor-manifest-beacon` are still
-selectable in the lab's beacon-create form. They stay as-is in the
-codebase (they're the reference Provider implementations and useful for
-embedded deployments that don't want retold-databeacon's REST surface).
-Session 4 just makes the recommended path obvious to operators:
-
-- `Service-BeaconTypeRegistry.js` — append `(legacy)` to the
-  `DisplayName` of both types and add a `Deprecated: true` field on
-  the public descriptor.
-- `PictView-Lab-Beacons.js` — when the form's BeaconType dropdown
-  shows a deprecated type, render a tooltip / inline note: "Legacy
-  type. New deployments should use `retold-databeacon` + the lab's
-  Persistence assignment for queue / manifest persistence."
-- The seed-dataset and other paths that filter by `BeaconType ===
-  'retold-databeacon'` already do the right thing; no other changes.
+Originally Session 4 just labeled `ultravisor-queue-beacon` and
+`ultravisor-manifest-beacon` as `(legacy)` in the lab UI while keeping
+the modules selectable. As of 2026-05-03 the modules have been deleted
+outright — the migration path proved out across every retold-databeacon
+engine and there are no remaining users.
+
+What's gone:
+
+- The two module directories (`modules/apps/ultravisor-queue-beacon`,
+  `modules/apps/ultravisor-manifest-beacon`).
+- Manifest entries in `Retold-Modules-Manifest.json` and
+  `modules/Include-Retold-Module-List.sh`.
+- Lab registry entries — `Service-BeaconTypeRegistry.js`'s
+  `SCANNED_MODULES` no longer scans them and `DEPRECATED_BEACON_TYPES`
+  is now an empty Set (kept for future per-module deprecations).
+
+What stayed:
+
+- `Ultravisor-QueuePersistenceBridge.cjs` and
+  `Ultravisor-ManifestStoreBridge.cjs` inside ultravisor — they still
+  dispatch `QP_*` / `MS_*` semantics, just into MeadowProxy now.
+- The `LEGACY_TOOLTIP` constant + per-stanza
+  `retoldBeacon.deprecated: true` plumbing in
+  `Service-BeaconTypeRegistry.js` — useful infrastructure for whatever
+  the next deprecation needs.
 
 #### Test-fable cleanup (tangential hygiene)
 

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "ultravisor",
-    "version": "1.3.2",
+    "version": "1.3.4",
     "description": "Cyclic process execution with ai integration.",
     "main": "source/Ultravisor.cjs",
     "bin": {

package/source/services/Ultravisor-ExecutionEngine.cjs

@@ -1,5 +1,6 @@
 const libPictService = require('pict-serviceproviderbase');
 const libStatus = require('./Ultravisor-Status.cjs');
+const libOutputStore = require('./Ultravisor-OutputStore.cjs');
 
 /**
  * Event-driven graph executor for Ultravisor operations.
@@ -449,21 +450,17 @@ class UltravisorExecutionEngine extends libPictService
 			tmpStateManager.setAddress(tmpWaitingInfo.OutputAddress, pValue, tmpContext, pNodeHash);
 		}
 
-		// Store in task outputs
-		if (!tmpContext.TaskOutputs[pNodeHash])
-		{
-			tmpContext.TaskOutputs[pNodeHash] = {};
-		}
-
-		// If pValue is a structured object, merge all keys into TaskOutputs (beacon results)
-		// If pValue is a scalar, store as InputValue (backward compat for value-input)
+		// Store in task outputs. Routes through OutputStore so beacon
+		// results that arrive with a multi-megabyte field (typed-op
+		// `Result` etc.) get lifted to the run's staging dir instead of
+		// pinning the payload in the manifest.
 		if (typeof pValue === 'object' && pValue !== null && !Array.isArray(pValue))
 		{
-			Object.assign(tmpContext.TaskOutputs[pNodeHash], pValue);
+			libOutputStore.mergeAndLift(tmpContext, pNodeHash, pValue, { Fable: this.fable, Logger: this.log });
 		}
 		else
 		{
-			tmpContext.TaskOutputs[pNodeHash].InputValue = pValue;
+			libOutputStore.mergeAndLift(tmpContext, pNodeHash, { InputValue: pValue }, { Fable: this.fable, Logger: this.log });
 		}
 
 		// Update the task's ElapsedMs to include the time spent waiting for input
@@ -1060,12 +1057,8 @@ class UltravisorExecutionEngine extends libPictService
 					`Intermediate event [${pIntermediateEventName}] from [${pNodeHash}]`, 1);
 			}
 
-			// Store the intermediate outputs
-			if (!pContext.TaskOutputs[pNodeHash])
-			{
-				pContext.TaskOutputs[pNodeHash] = {};
-			}
-			Object.assign(pContext.TaskOutputs[pNodeHash], pIntermediateOutputs);
+			// Store the intermediate outputs (lift large fields per OutputStore policy)
+			libOutputStore.mergeAndLift(pContext, pNodeHash, pIntermediateOutputs, { Fable: this.fable, Logger: this.log });
 
 			// Find downstream nodes for this intermediate event
 			let tmpDownstreamEvents = this._getDownstreamEvents(pNodeHash, pIntermediateEventName, pContext);
@@ -1167,14 +1160,10 @@ class UltravisorExecutionEngine extends libPictService
 				return fCallback(null);
 			}
 
-			// Store outputs in TaskOutputs
+			// Store outputs in TaskOutputs (lift large fields per OutputStore policy)
 			if (pResult.Outputs && typeof(pResult.Outputs) === 'object')
 			{
-				if (!pContext.TaskOutputs[pNodeHash])
-				{
-					pContext.TaskOutputs[pNodeHash] = {};
-				}
-				Object.assign(pContext.TaskOutputs[pNodeHash], pResult.Outputs);
+				libOutputStore.mergeAndLift(pContext, pNodeHash, pResult.Outputs, { Fable: this.fable, Logger: this.log });
 			}
 
 			// Store any state writes from the result
@@ -1313,9 +1302,17 @@ class UltravisorExecutionEngine extends libPictService
 			let tmpSourcePortName = this._extractPortName(tmpConn.SourcePortHash, tmpPortLabelMap);
 			let tmpTargetPortName = this._extractPortName(tmpConn.TargetPortHash, tmpPortLabelMap);
 
-			// Read the source value from the source node's outputs
+			// Read the source value from the source node's outputs.
+			// Large outputs are stored as $$ref objects pointing into the
+			// run's staging dir — materialize before the value reaches a
+			// downstream task handler so the State edge contract stays
+			// transparent (handlers never see a ref).
 			let tmpSourceNodeOutputs = pContext.TaskOutputs[tmpConn.SourceNodeHash] || {};
 			let tmpSourceValue = tmpSourceNodeOutputs[tmpSourcePortName];
+			if (libOutputStore.isOutputRef(tmpSourceValue))
+			{
+				tmpSourceValue = libOutputStore.materializeRefValue(pContext.StagingPath, tmpSourceValue, this.log);
+			}
 
 			// Apply template if defined
 			if (tmpConn.Data && tmpConn.Data.Template && typeof(tmpConn.Data.Template) === 'string')

package/source/services/Ultravisor-OutputStore.cjs

@@ -0,0 +1,317 @@
+const libFS = require('fs');
+const libPath = require('path');
+
+/**
+ * Bound the in-memory size of TaskOutputs by lifting large per-port
+ * payloads to disk and replacing them in the manifest with a small
+ * reference object.
+ *
+ * The fix the lab needed: typed-op pipelines (Pull -> Intersect ->
+ * Comprehension -> Write at 100K rows) were emitting ~30 MB of JSON-
+ * stringified `Result` per node. UV held those strings in TaskOutputs
+ * and serialized the full manifest twice per snapshot (disk + sync
+ * /Trigger response), which saturated the V8 heap inside Builtin_
+ * JsonParse on sustained stress.
+ *
+ * Splitting the two jobs TaskOutputs was secretly doing — short-lived
+ * State-edge transport vs. durable manifest record — by lifting only
+ * the transport payload, breaks the link between manifest size and
+ * payload size while preserving the State-edge contract: handlers
+ * still see materialized values, never refs.
+ *
+ * Lift on write, materialize on read. Pure functions; the caller owns
+ * the staging path.
+ */
+
+const REF_KEY = '$$ref';
+const REF_DIR = 'outputs';
+const DEFAULT_THRESHOLD_BYTES = 1024 * 1024;
+const STREAM_CHUNK_BYTES = 64 * 1024;
+
+/**
+ * Resolve the byte threshold for lifting. Order: env var, program
+ * config, default. Env var wins so an operator can hot-tune in a
+ * stuck container without rebuilding. Returns the default for any
+ * non-positive or unparseable value.
+ */
+function getThresholdBytes(pFable)
+{
+	let tmpEnv = process.env.UV_OUTPUT_STORE_THRESHOLD_BYTES;
+	if (tmpEnv)
+	{
+		let tmpParsed = parseInt(tmpEnv, 10);
+		if (!isNaN(tmpParsed) && tmpParsed > 0)
+		{
+			return tmpParsed;
+		}
+	}
+	if (pFable && pFable.ProgramConfiguration
+		&& typeof(pFable.ProgramConfiguration.UltravisorOutputStoreThresholdBytes) === 'number'
+		&& pFable.ProgramConfiguration.UltravisorOutputStoreThresholdBytes > 0)
+	{
+		return pFable.ProgramConfiguration.UltravisorOutputStoreThresholdBytes;
+	}
+	return DEFAULT_THRESHOLD_BYTES;
+}
+
+/**
+ * True for the {$$ref, Bytes} shape produced by liftValue. Used by
+ * read-side materialization and by callers that want to special-case
+ * ref-bearing entries (e.g. the optional inline-follow on /Manifest).
+ */
+function isOutputRef(pValue)
+{
+	return pValue !== null
+		&& typeof(pValue) === 'object'
+		&& typeof(pValue[REF_KEY]) === 'string';
+}
+
+/**
+ * Decide whether a single value is large enough to lift. Strings and
+ * Buffers carry the bulk of typed-op payloads — they're the only
+ * shapes worth measuring without paying the JSON.stringify cost we're
+ * trying to avoid. Objects/Arrays under threshold stay inline; if a
+ * future workload produces a multi-megabyte object literal, lift the
+ * producer side first (cheaper) before reaching for a structural
+ * size walk here.
+ */
+function _shouldLift(pValue, pThresholdBytes)
+{
+	if (typeof(pValue) === 'string')
+	{
+		return Buffer.byteLength(pValue, 'utf8') >= pThresholdBytes;
+	}
+	if (Buffer.isBuffer(pValue))
+	{
+		return pValue.length >= pThresholdBytes;
+	}
+	return false;
+}
+
+/**
+ * Sanitize a path segment so node hashes / port names map to safe
+ * filesystem names. Anything outside [A-Za-z0-9_-] becomes '_'.
+ */
+function _safeSegment(pValue)
+{
+	let tmpStr = String(pValue == null ? '' : pValue);
+	if (tmpStr === '')
+	{
+		return '_';
+	}
+	return tmpStr.replace(/[^A-Za-z0-9_\-]/g, '_');
+}
+
+/**
+ * Stream a string or Buffer to disk in fixed-size chunks. Avoids the
+ * implicit Buffer.from copy that writeFileSync does for large strings
+ * (which is half the OOM blast radius — string + buffer alive at the
+ * same time). Peak overhead is one chunk regardless of payload size.
+ */
+function _streamWriteSync(pAbsolutePath, pValue)
+{
+	let tmpFd = libFS.openSync(pAbsolutePath, 'w');
+	try
+	{
+		if (typeof(pValue) === 'string')
+		{
+			let tmpOffset = 0;
+			let tmpLen = pValue.length;
+			while (tmpOffset < tmpLen)
+			{
+				let tmpEnd = Math.min(tmpOffset + STREAM_CHUNK_BYTES, tmpLen);
+				let tmpChunkBuf = Buffer.from(pValue.slice(tmpOffset, tmpEnd), 'utf8');
+				libFS.writeSync(tmpFd, tmpChunkBuf, 0, tmpChunkBuf.length);
+				tmpOffset = tmpEnd;
+			}
+		}
+		else if (Buffer.isBuffer(pValue))
+		{
+			let tmpOffset = 0;
+			while (tmpOffset < pValue.length)
+			{
+				let tmpEnd = Math.min(tmpOffset + STREAM_CHUNK_BYTES, pValue.length);
+				libFS.writeSync(tmpFd, pValue, tmpOffset, tmpEnd - tmpOffset);
+				tmpOffset = tmpEnd;
+			}
+		}
+		else
+		{
+			throw new Error('OutputStore: _streamWriteSync expects a string or Buffer');
+		}
+	}
+	finally
+	{
+		libFS.closeSync(tmpFd);
+	}
+}
+
+/**
+ * Lift one value to disk and return the ref object that should
+ * replace it in TaskOutputs. The ref's path is relative to the run's
+ * staging dir; readers join it back with the same staging dir so a
+ * staging folder can be moved without rewriting refs.
+ */
+function liftValue(pStagingPath, pNodeHash, pPort, pValue)
+{
+	let tmpRelDir = libPath.join(REF_DIR, _safeSegment(pNodeHash));
+	let tmpAbsDir = libPath.resolve(pStagingPath, tmpRelDir);
+	libFS.mkdirSync(tmpAbsDir, { recursive: true });
+
+	let tmpFileName = _safeSegment(pPort) + '.json';
+	let tmpAbsFile = libPath.resolve(tmpAbsDir, tmpFileName);
+	let tmpRelFile = libPath.join(tmpRelDir, tmpFileName);
+
+	_streamWriteSync(tmpAbsFile, pValue);
+
+	let tmpBytes;
+	if (typeof(pValue) === 'string')
+	{
+		tmpBytes = Buffer.byteLength(pValue, 'utf8');
+	}
+	else
+	{
+		tmpBytes = pValue.length;
+	}
+
+	let tmpRef = {};
+	tmpRef[REF_KEY] = tmpRelFile.split(libPath.sep).join('/');
+	tmpRef.Bytes = tmpBytes;
+	return tmpRef;
+}
+
+/**
+ * Read a ref's contents back from disk. Returns the original value
+ * (utf8 string — the only shape we lift today) or undefined when the
+ * staging file is gone (e.g. retention sweep ran, or a reloaded
+ * manifest references a folder that was cleaned).
+ *
+ * Non-ref values pass through unchanged so callers can wrap any read
+ * site without branching.
+ */
+function materializeRefValue(pStagingPath, pValue, pLogger)
+{
+	if (!isOutputRef(pValue))
+	{
+		return pValue;
+	}
+	if (!pStagingPath)
+	{
+		if (pLogger) pLogger.warn(`OutputStore: cannot materialize ref [${pValue[REF_KEY]}] without a staging path.`);
+		return undefined;
+	}
+	let tmpAbsFile = libPath.resolve(pStagingPath, pValue[REF_KEY]);
+	if (!libFS.existsSync(tmpAbsFile))
+	{
+		if (pLogger) pLogger.warn(`OutputStore: ref payload missing on disk [${tmpAbsFile}].`);
+		return undefined;
+	}
+	try
+	{
+		return libFS.readFileSync(tmpAbsFile, 'utf8');
+	}
+	catch (pError)
+	{
+		if (pLogger) pLogger.warn(`OutputStore: failed to read ref payload [${tmpAbsFile}]: ${pError.message}`);
+		return undefined;
+	}
+}
+
+/**
+ * Merge a set of new fields into TaskOutputs[<nodeHash>], lifting any
+ * field whose serialized size meets or exceeds the threshold. Falls
+ * back to inline storage on any IO failure so a flaky disk doesn't
+ * turn into lost outputs.
+ *
+ * This is the one entry point the engine uses for all three places
+ * that write to TaskOutputs (resume, intermediate event, task
+ * complete). Centralizing avoids the "I forgot to lift in this code
+ * path" bug the original conflated design encouraged.
+ */
+function mergeAndLift(pContext, pNodeHash, pNewFields, pOptions)
+{
+	if (!pNewFields || typeof(pNewFields) !== 'object')
+	{
+		return;
+	}
+	if (!pContext.TaskOutputs)
+	{
+		pContext.TaskOutputs = {};
+	}
+	if (!pContext.TaskOutputs[pNodeHash])
+	{
+		pContext.TaskOutputs[pNodeHash] = {};
+	}
+
+	let tmpTarget = pContext.TaskOutputs[pNodeHash];
+	let tmpFable = (pOptions && pOptions.Fable) || null;
+	let tmpLogger = (pOptions && pOptions.Logger) || null;
+	let tmpThreshold = getThresholdBytes(tmpFable);
+
+	let tmpKeys = Object.keys(pNewFields);
+	for (let i = 0; i < tmpKeys.length; i++)
+	{
+		let tmpKey = tmpKeys[i];
+		let tmpVal = pNewFields[tmpKey];
+
+		if (pContext.StagingPath && _shouldLift(tmpVal, tmpThreshold))
+		{
+			try
+			{
+				tmpTarget[tmpKey] = liftValue(pContext.StagingPath, pNodeHash, tmpKey, tmpVal);
+				continue;
+			}
+			catch (pError)
+			{
+				if (tmpLogger) tmpLogger.warn(`OutputStore: lift failed for [${pNodeHash}.${tmpKey}], falling back to inline: ${pError.message}`);
+			}
+		}
+		tmpTarget[tmpKey] = tmpVal;
+	}
+}
+
+/**
+ * Walk a TaskOutputs map and inline every ref into its materialized
+ * value. Used by the optional /Manifest?inline=outputs follow path —
+ * regular manifest reads return refs as-is.
+ */
+function inlineAllRefs(pStagingPath, pTaskOutputs, pLogger)
+{
+	if (!pTaskOutputs || typeof(pTaskOutputs) !== 'object')
+	{
+		return pTaskOutputs;
+	}
+	let tmpResult = {};
+	let tmpNodeKeys = Object.keys(pTaskOutputs);
+	for (let i = 0; i < tmpNodeKeys.length; i++)
+	{
+		let tmpNodeHash = tmpNodeKeys[i];
+		let tmpEntry = pTaskOutputs[tmpNodeHash];
+		if (!tmpEntry || typeof(tmpEntry) !== 'object')
+		{
+			tmpResult[tmpNodeHash] = tmpEntry;
+			continue;
+		}
+		let tmpInlined = {};
+		let tmpFieldKeys = Object.keys(tmpEntry);
+		for (let j = 0; j < tmpFieldKeys.length; j++)
+		{
+			let tmpField = tmpFieldKeys[j];
+			tmpInlined[tmpField] = materializeRefValue(pStagingPath, tmpEntry[tmpField], pLogger);
+		}
+		tmpResult[tmpNodeHash] = tmpInlined;
+	}
+	return tmpResult;
+}
+
+module.exports =
+{
+	REF_KEY: REF_KEY,
+	DEFAULT_THRESHOLD_BYTES: DEFAULT_THRESHOLD_BYTES,
+	getThresholdBytes: getThresholdBytes,
+	isOutputRef: isOutputRef,
+	liftValue: liftValue,
+	materializeRefValue: materializeRefValue,
+	mergeAndLift: mergeAndLift,
+	inlineAllRefs: inlineAllRefs
+};

package/source/services/Ultravisor-StateManager.cjs

@@ -1,4 +1,5 @@
 const libPictService = require('pict-serviceproviderbase');
+const libOutputStore = require('./Ultravisor-OutputStore.cjs');
 
 /**
  * Manages three-level state (Global, Operation, Task) and provides
@@ -67,9 +68,9 @@ class UltravisorStateManager extends libPictService
 					this.log.warn(`UltravisorStateManager: resolveAddress for Task.X requires a current node hash.`);
 					return undefined;
 				}
-				return this._resolveFromObject(
+				return this._materializeIfRef(pExecutionContext, this._resolveFromObject(
 					pExecutionContext.TaskOutputs[pCurrentNodeHash] || {},
-					tmpRemainder);
+					tmpRemainder));
 
 			case 'TaskOutput':
 			{
@@ -77,14 +78,19 @@ class UltravisorStateManager extends libPictService
 				let tmpSecondDot = tmpRemainder.indexOf('.');
 				if (tmpSecondDot < 0)
 				{
-					// Just TaskOutput.{NodeHash} -- return the whole output object
+					// Just TaskOutput.{NodeHash} -- return the whole output object.
+					// Whole-node return ships ref shapes through to the
+					// caller — single-port reads materialize, but a
+					// "give me the whole bag" caller has to handle refs
+					// itself (or use Task.X / TaskOutput.X.Y to get the
+					// materialized leaf).
 					return pExecutionContext.TaskOutputs[tmpRemainder] || {};
 				}
 				let tmpNodeHash = tmpRemainder.substring(0, tmpSecondDot);
 				let tmpPath = tmpRemainder.substring(tmpSecondDot + 1);
-				return this._resolveFromObject(
+				return this._materializeIfRef(pExecutionContext, this._resolveFromObject(
 					pExecutionContext.TaskOutputs[tmpNodeHash] || {},
-					tmpPath);
+					tmpPath));
 			}
 
 			case 'Output':
@@ -248,6 +254,20 @@ class UltravisorStateManager extends libPictService
 		this._Manyfest.setValueAtAddress(pObject, pPath, pValue);
 		return true;
 	}
+
+	// Resolve $$ref payloads to their materialized value when reading
+	// from TaskOutputs. Per-leaf reads (Task.X, TaskOutput.X.Y) go
+	// through this; whole-node reads (TaskOutput.X) deliberately do not
+	// — see the case-handler comment.
+	_materializeIfRef(pExecutionContext, pValue)
+	{
+		if (!libOutputStore.isOutputRef(pValue))
+		{
+			return pValue;
+		}
+		return libOutputStore.materializeRefValue(
+			pExecutionContext.StagingPath, pValue, this.log);
+	}
 }
 
 module.exports = UltravisorStateManager;

package/source/web_server/Ultravisor-API-Server.cjs

@@ -8,6 +8,7 @@ const libOratorServiceServerRestify = require(`orator-serviceserver-restify`);
 const libOratorAuthentication = require('orator-authentication');
 const libWebSocket = require('ws');
 const libStatus = require('../services/Ultravisor-Status.cjs');
+const libOutputStore = require('../services/Ultravisor-OutputStore.cjs');
 
 // Strip a manifest down to the JSON-serializable shape the wire
 // expects. The in-memory ExecutionContext can carry closures in
@@ -41,6 +42,18 @@ function _cleanManifestForWire(pManifest)
 	};
 }
 
+// Optional follow-the-refs pass for /Manifest/:RunHash?inline=outputs.
+// Default response ships ref shapes; this helper is invoked only when
+// the caller explicitly asks for materialized payloads. Reads files
+// synchronously off the staging dir — fine for the rare inspection
+// case but not something to call from a hot path.
+function _inlineIfRequested(pCleanManifest, pStagingPath, pInlineOutputs, pLogger)
+{
+	if (!pInlineOutputs || !pCleanManifest) return pCleanManifest;
+	pCleanManifest.TaskOutputs = libOutputStore.inlineAllRefs(pStagingPath, pCleanManifest.TaskOutputs, pLogger);
+	return pCleanManifest;
+}
+
 // Lightweight query-string parser. Restify's queryParser plugin is
 // not enabled on this server (mounting it would change the request
 // shape for every existing handler), so the few routes that need
@@ -1895,11 +1908,19 @@ class UltravisorAPIServer extends libPictService
 				function (pRequest, pResponse, fNext)
 				{
 					let tmpHash = pRequest.params.RunHash;
+					// Default ships TaskOutputs with ref shapes inline. The
+					// `?inline=outputs` flag follows refs to materialized
+					// values for UIs that need to see the full payload —
+					// this is the rare case (e.g. raw inspection in the
+					// manifest viewer) and bypasses the OOM guard, so
+					// callers should know what they're asking for.
+					let tmpQuery = _parseQueryString(pRequest.url);
+					let tmpInlineOutputs = (tmpQuery.inline === 'outputs');
 					let tmpManifest = this._getService('UltravisorExecutionManifest');
 					let tmpRun = tmpManifest ? tmpManifest.getRun(tmpHash) : null;
 					if (tmpRun)
 					{
-						pResponse.send(_cleanManifestForWire(tmpRun));
+						pResponse.send(_inlineIfRequested(_cleanManifestForWire(tmpRun), tmpRun.StagingPath, tmpInlineOutputs, this.log));
 						return fNext();
 					}
 					// Not in memory — try the bridge (beacon-backed history).
@@ -1913,7 +1934,8 @@ class UltravisorAPIServer extends libPictService
 					{
 						if (pResult && pResult.Success && pResult.Manifest)
 						{
-							pResponse.send(_cleanManifestForWire(pResult.Manifest));
+							let tmpClean = _cleanManifestForWire(pResult.Manifest);
+							pResponse.send(_inlineIfRequested(tmpClean, pResult.Manifest.StagingPath, tmpInlineOutputs, this.log));
 						}
 						else
 						{

package/webinterface/dist/docs/features/persistence-via-databeacon.md

@@ -1,10 +1,12 @@
 # Persistence via retold-databeacon
 
-**Status: planned + foundations in progress.** This doc is the cross-session plan
-for routing ultravisor's queue and manifest persistence through retold-databeacon
-instead of the specialized `ultravisor-queue-beacon` and `ultravisor-manifest-beacon`
-modules. It captures the architectural decision, the work breakdown, and the
-hand-off state so a fresh context can resume from here.
+**Status: shipped. Legacy modules removed 2026-05-03.** This doc is the
+cross-session plan for routing ultravisor's queue and manifest persistence
+through retold-databeacon instead of the specialized `ultravisor-queue-beacon`
+and `ultravisor-manifest-beacon` modules (now deleted from the monorepo —
+see [What changed and why](#what-changed-and-why) for the historical context).
+It captures the architectural decision, the work breakdown, and the hand-off
+state so a fresh context can resume from here.
 
 If you are picking this up cold, read this doc top-to-bottom before touching
 code — the decisions below are load-bearing and the cross-module dependencies
@@ -12,15 +14,18 @@ are not obvious from the source.
 
 ## What changed and why
 
-Before this redesign:
-- `ultravisor-queue-beacon` advertises capability `QueuePersistence` with
-  `QP_*` actions. Ships a `QueuePersistenceProviderBase` and a default
+Before this redesign (the modules described here have since been deleted from
+the monorepo on 2026-05-03 — kept here for historical context):
+- `ultravisor-queue-beacon` advertised capability `QueuePersistence` with
+  `QP_*` actions. Shipped a `QueuePersistenceProviderBase` and a default
   `MemoryQueuePersistenceProvider`.
-- `ultravisor-manifest-beacon` advertises `ManifestStore` with `MS_*` actions.
-  Ships a `ManifestStoreProviderBase` and `MemoryManifestStoreProvider`.
+- `ultravisor-manifest-beacon` advertised `ManifestStore` with `MS_*` actions.
+  Shipped a `ManifestStoreProviderBase` and `MemoryManifestStoreProvider`.
 - `Ultravisor-QueuePersistenceBridge.cjs` and
-  `Ultravisor-ManifestStoreBridge.cjs` dispatch into the corresponding
-  capability when a beacon is connected, fall back to local storage otherwise.
+  `Ultravisor-ManifestStoreBridge.cjs` dispatched into the corresponding
+  capability when a beacon was connected, fell back to local storage otherwise.
+  Both bridges still exist inside ultravisor — they now translate `QP_*` /
+  `MS_*` semantics into `MeadowProxy` calls against retold-databeacon.
 - `bootstrap-flush` (already shipped) replays locally-buffered writes into a
   newly-connected beacon via per-beacon HWM tracking persisted to
   `<DataPath>/persistence-bridge-hwm.json`.
@@ -36,10 +41,12 @@ The redesign:
   `Request` calls** (Method + Path + Body) instead of dispatching to a
   specialized capability. The schema (table names, column shapes) is owned
   by ultravisor; retold-databeacon is a generic gateway.
-- **The two specialized beacon modules are not deleted.** They stay as the
-  reference implementation of the Provider pattern, useful for embedded /
-  niche deployments that don't want retold-databeacon's REST surface. They
-  are no longer the lab's recommended path.
+- **The two specialized beacon modules have been deleted (2026-05-03).**
+  They were marked legacy throughout Session 4 and the migration path proved
+  out across all retold-databeacon-supported engines (mysql / mssql / postgres
+  / sqlite). Embedded deployments that prefer in-process Provider-pattern
+  persistence can still subclass the bridges' fallback path; the QP/MS
+  capability surfaces themselves are gone from the monorepo.
 - **The lab's UV detail view gains a "persistence databeacon" picker.** The
   operator picks a running retold-databeacon (which itself has an
   engine+database picker via `lab-engine-database-picker`); ultravisor's
@@ -1120,21 +1127,31 @@ engine-coverage work — same files get touched.
 
 #### Legacy beacon deprecation
 
-`ultravisor-queue-beacon` and `ultravisor-manifest-beacon` are still
-selectable in the lab's beacon-create form. They stay as-is in the
-codebase (they're the reference Provider implementations and useful for
-embedded deployments that don't want retold-databeacon's REST surface).
-Session 4 just makes the recommended path obvious to operators:
-
-- `Service-BeaconTypeRegistry.js` — append `(legacy)` to the
-  `DisplayName` of both types and add a `Deprecated: true` field on
-  the public descriptor.
-- `PictView-Lab-Beacons.js` — when the form's BeaconType dropdown
-  shows a deprecated type, render a tooltip / inline note: "Legacy
-  type. New deployments should use `retold-databeacon` + the lab's
-  Persistence assignment for queue / manifest persistence."
-- The seed-dataset and other paths that filter by `BeaconType ===
-  'retold-databeacon'` already do the right thing; no other changes.
+Originally Session 4 just labeled `ultravisor-queue-beacon` and
+`ultravisor-manifest-beacon` as `(legacy)` in the lab UI while keeping
+the modules selectable. As of 2026-05-03 the modules have been deleted
+outright — the migration path proved out across every retold-databeacon
+engine and there are no remaining users.
+
+What's gone:
+
+- The two module directories (`modules/apps/ultravisor-queue-beacon`,
+  `modules/apps/ultravisor-manifest-beacon`).
+- Manifest entries in `Retold-Modules-Manifest.json` and
+  `modules/Include-Retold-Module-List.sh`.
+- Lab registry entries — `Service-BeaconTypeRegistry.js`'s
+  `SCANNED_MODULES` no longer scans them and `DEPRECATED_BEACON_TYPES`
+  is now an empty Set (kept for future per-module deprecations).
+
+What stayed:
+
+- `Ultravisor-QueuePersistenceBridge.cjs` and
+  `Ultravisor-ManifestStoreBridge.cjs` inside ultravisor — they still
+  dispatch `QP_*` / `MS_*` semantics, just into MeadowProxy now.
+- The `LEGACY_TOOLTIP` constant + per-stanza
+  `retoldBeacon.deprecated: true` plumbing in
+  `Service-BeaconTypeRegistry.js` — useful infrastructure for whatever
+  the next deprecation needs.
 
 #### Test-fable cleanup (tangential hygiene)