@mastra/memory 1.16.0-alpha.4 → 1.17.0-alpha.0

package/CHANGELOG.md

@@ -1,5 +1,65 @@
 # @mastra/memory
 
+## 1.17.0-alpha.0
+
+### Minor Changes
+
+- Added opt-in temporal-gap markers for observational memory. When enabled via `observationalMemory.temporalMarkers: true`, the agent receives a `<system-reminder type="temporal-gap">` before any user message that arrives more than 10 minutes after the previous one, so it can anchor responses in real elapsed time. Markers are persisted, surfaced to the observer, and rendered by the MastraCode TUI on reload. ([#15605](https://github.com/mastra-ai/mastra/pull/15605))
+
+### Patch Changes
+
+- Fixed observer agent truncation that could cut UTF-16 surrogate pairs in half when formatting messages, tool results, or observation lines with emoji or other astral-plane characters. This produced lone surrogates that strict JSON parsers (including Anthropic's) reject with errors like `no low surrogate in string`, causing observer runs to fail. ([#15634](https://github.com/mastra-ai/mastra/pull/15634))
+
+- Updated dependencies [[`0a0aa94`](https://github.com/mastra-ai/mastra/commit/0a0aa94729592e99885af2efb90c56aaada62247), [`01a7d51`](https://github.com/mastra-ai/mastra/commit/01a7d513493d21562f677f98550f7ceb165ba78c)]:
+  - @mastra/core@1.27.0-alpha.1
+
+## 1.16.0
+
+### Minor Changes
+
+- Added activateAfterIdle setting for observational memory so buffered observations can activate after idle time before the next prompt. ([#15365](https://github.com/mastra-ai/mastra/pull/15365))
+
+  **Example**
+
+  Set `activateAfterIdle: 300_000` (or `"5m"`) on the `observationalMemory` config to activate buffered context after 5 minutes of inactivity.
+
+  This helps long-running threads reuse compressed context after prompt cache TTLs expire instead of sending a larger raw message window on the next request.
+
+- Added `activateOnProviderChange` so observational memory can activate buffered observations and reflections before switching to a different provider or model. ([#15420](https://github.com/mastra-ai/mastra/pull/15420))
+
+  ```ts
+  const memory = new Memory({
+    options: {
+      observationalMemory: {
+        model: 'google/gemini-2.5-flash',
+        activateOnProviderChange: true,
+      },
+    },
+  });
+  ```
+
+  This helps keep prompt-cache savings when the next step cannot reuse the previous provider's cache.
+
+### Patch Changes
+
+- Fixed early observational memory activations so buffered reflections are only activated when they still leave a healthy active observation set. ([#15462](https://github.com/mastra-ai/mastra/pull/15462))
+
+  Before this change, idle-timeout (`activateAfterIdle`) and model/provider-change (`activateOnProviderChange`) activations could swap in a buffered reflection too early. In bad cases, that replaced a large raw observation tail with a much smaller mostly-compressed result, which hurt reflection quality.
+
+  Early activations now stay buffered unless both of these checks pass:
+  - The unreflected observation tail is at least as large as the buffered reflection, so the activated result is not dominated by compressed content.
+  - The combined post-activation size is at least 75% of what a normal threshold activation would produce, so early activations do not cliff far below the regular target.
+
+  This update also fixes false `provider_change` activations when older persisted messages only contain a bare model id like `gpt-5.4` while newer turns use the fully qualified `provider/modelId` form.
+
+- Fixed a security issue where several parsing and tracing paths could slow down on malformed or attacker-crafted input. Normal behavior is unchanged, and these packages now handle pathological input in linear time. ([#15566](https://github.com/mastra-ai/mastra/pull/15566))
+
+- Fixed other-thread context filtering falling back to the observational memory record timestamp when thread metadata is missing. ([#15269](https://github.com/mastra-ai/mastra/pull/15269))
+
+- Updated dependencies [[`20f59b8`](https://github.com/mastra-ai/mastra/commit/20f59b876cf91199efbc49a0e36b391240708f08), [`aba393e`](https://github.com/mastra-ai/mastra/commit/aba393e2da7390c69b80e516a4f153cda6f09376), [`3d83d06`](https://github.com/mastra-ai/mastra/commit/3d83d06f776f00fb5f4163dddd32a030c5c20844), [`e2687a7`](https://github.com/mastra-ai/mastra/commit/e2687a7408790c384563816a9a28ed06735684c9), [`fdd54cf`](https://github.com/mastra-ai/mastra/commit/fdd54cf612a9af876e9fdd85e534454f6e7dd518), [`6315317`](https://github.com/mastra-ai/mastra/commit/63153175fe9a7b224e5be7c209bbebc01dd9b0d5), [`a371ac5`](https://github.com/mastra-ai/mastra/commit/a371ac534aa1bb368a1acf9d8b313378dfdc787e), [`7db42a9`](https://github.com/mastra-ai/mastra/commit/7db42a9cccd3b29c44fb0731f792c51575e8421c), [`0474c2b`](https://github.com/mastra-ai/mastra/commit/0474c2b2e7c7e1ad8691dca031284841391ff1ef), [`0a5fa1d`](https://github.com/mastra-ai/mastra/commit/0a5fa1d3cb0583889d06687155f26fd7d2edc76c), [`7e0e63e`](https://github.com/mastra-ai/mastra/commit/7e0e63e2e485e84442351f4c7a79a424c83539dc), [`ea43e64`](https://github.com/mastra-ai/mastra/commit/ea43e646dd95d507694b6112b0bf1df22ad552b2), [`f607106`](https://github.com/mastra-ai/mastra/commit/f607106854c6416c4a07d4082604b9f66d047221), [`30456b6`](https://github.com/mastra-ai/mastra/commit/30456b6b08c8fd17e109dd093b73d93b65e83bc5), [`9d11a8c`](https://github.com/mastra-ai/mastra/commit/9d11a8c1c8924eb975a245a5884d40ca1b7e0491), [`9d3b24b`](https://github.com/mastra-ai/mastra/commit/9d3b24b19407ae9c09586cf7766d38dc4dff4a69), [`00d1b16`](https://github.com/mastra-ai/mastra/commit/00d1b16b401199cb294fa23f43336547db4dca9b), [`47cee3e`](https://github.com/mastra-ai/mastra/commit/47cee3e137fe39109cf7fffd2a8cf47b76dc702e), [`62919a6`](https://github.com/mastra-ai/mastra/commit/62919a6ee0fbf3779ad21a97b1ec6696515d5104), [`d246696`](https://github.com/mastra-ai/mastra/commit/d246696139a3144a5b21b042d41c532688e957e1), [`354f9ce`](https://github.com/mastra-ai/mastra/commit/354f9ce1ca6af2074b6a196a23f8ec30012dccca), [`16e34ca`](https://github.com/mastra-ai/mastra/commit/16e34caa98b9a114b17a6125e4e3fd87f169d0d0), [`7020c06`](https://github.com/mastra-ai/mastra/commit/7020c0690b199d9da337f0e805f16948e557922e), [`8786a61`](https://github.com/mastra-ai/mastra/commit/8786a61fa54ba265f85eeff9985ca39863d18bb6), [`9467ea8`](https://github.com/mastra-ai/mastra/commit/9467ea87695749a53dfc041576410ebf9ee7bb67), [`7338d94`](https://github.com/mastra-ai/mastra/commit/7338d949380cf68b095342e8e42610dc51d557c1), [`c80dc16`](https://github.com/mastra-ai/mastra/commit/c80dc16e113e6cc159f510ffde501ad4711b2189), [`af8a57e`](https://github.com/mastra-ai/mastra/commit/af8a57ed9ba9685ad8601d5b71ae3706da6222f9), [`d63ffdb`](https://github.com/mastra-ai/mastra/commit/d63ffdbb2c11e76fe5ea45faab44bc15460f010c), [`47cee3e`](https://github.com/mastra-ai/mastra/commit/47cee3e137fe39109cf7fffd2a8cf47b76dc702e), [`1bd5104`](https://github.com/mastra-ai/mastra/commit/1bd51048b6da93507276d6623e3fd96a9e1a8944), [`e9837b5`](https://github.com/mastra-ai/mastra/commit/e9837b53699e18711b09e0ca010a4106376f2653), [`c65aec3`](https://github.com/mastra-ai/mastra/commit/c65aec356cc037ee7c4b30ccea946807d4c4f443), [`8f1b280`](https://github.com/mastra-ai/mastra/commit/8f1b280b7fe6999ec654f160cb69c1a8719e7a57), [`92dcf02`](https://github.com/mastra-ai/mastra/commit/92dcf029294210ac91b090900c1a0555a425c57a), [`0fd90a2`](https://github.com/mastra-ai/mastra/commit/0fd90a215caf5fca8099c15a67ca03e4427747a3), [`8fb2405`](https://github.com/mastra-ai/mastra/commit/8fb2405138f2d208b7962ad03f121ca25bcc28c5), [`12df98c`](https://github.com/mastra-ai/mastra/commit/12df98c4904643d9481f5c78f3bed443725b4c96)]:
+  - @mastra/core@1.26.0
+  - @mastra/schema-compat@1.2.9
+
 ## 1.16.0-alpha.4
 
 ### Patch Changes

package/dist/{chunk-HAUWGR76.cjs → chunk-5MTY2UYL.cjs}

@@ -385,6 +385,67 @@ ${gap}
   result += withInlineDates.slice(lastIndex);
   return result;
 }
+var MIN_TEMPORAL_GAP_MS = 10 * 60 * 1e3;
+function formatTemporalGap(diffMs) {
+  if (diffMs < MIN_TEMPORAL_GAP_MS) return null;
+  const minute = 60 * 1e3;
+  const hour = 60 * minute;
+  const day = 24 * hour;
+  const week = 7 * day;
+  const month = 30 * day;
+  const year = 365 * day;
+  const formatUnit = (value, unit) => `${value} ${unit}${value === 1 ? "" : "s"}`;
+  if (diffMs < hour) {
+    const minutes = Math.max(1, Math.round(diffMs / minute));
+    return `${formatUnit(minutes, "minute")} later`;
+  }
+  const formatTwoUnits = (primaryMs, primaryUnit, secondaryMs, secondaryUnit) => {
+    const primary = Math.floor(diffMs / primaryMs);
+    const remainder = diffMs - primary * primaryMs;
+    const secondary = Math.floor(remainder / secondaryMs);
+    const parts = [formatUnit(primary, primaryUnit)];
+    if (secondary > 0) {
+      parts.push(formatUnit(secondary, secondaryUnit));
+    }
+    return `${parts.join(" ")} later`;
+  };
+  if (diffMs < day) {
+    return formatTwoUnits(hour, "hour", minute, "minute");
+  }
+  if (diffMs < week) {
+    return formatTwoUnits(day, "day", hour, "hour");
+  }
+  if (diffMs < month) {
+    return formatTwoUnits(week, "week", day, "day");
+  }
+  if (diffMs < year) {
+    return formatTwoUnits(month, "month", week, "week");
+  }
+  return formatTwoUnits(year, "year", month, "month");
+}
+function formatTemporalTimestamp(date) {
+  return date.toLocaleString("en-US", {
+    year: "numeric",
+    month: "2-digit",
+    day: "2-digit",
+    hour: "numeric",
+    minute: "2-digit",
+    hour12: true,
+    timeZoneName: "short"
+  });
+}
+function getMessagePartTimestamp(msg, position) {
+  const timestamps = msg.content?.parts?.map((part) => "createdAt" in part ? part.createdAt : void 0).filter((timestamp) => typeof timestamp === "number");
+  if (timestamps && timestamps.length > 0) {
+    const index = position === "first" ? 0 : timestamps.length - 1;
+    const timestamp = timestamps[index];
+    if (timestamp !== void 0) return timestamp;
+  }
+  return new Date(msg.createdAt).getTime();
+}
+function isTemporalGapMarker(msg) {
+  return msg.id.startsWith("__temporal_");
+}
 
 // src/processors/observational-memory/markers.ts
 function createObservationStartMarker(params) {
@@ -2439,6 +2500,15 @@ function stripEphemeralAnchorIds(observations) {
   }
   return observations.replace(/(^|\n)([^\S\n]*)\[(O\d+(?:-N\d+)?)\][^\S\n]*/g, "$1$2");
 }
+
+// src/processors/observational-memory/string-utils.ts
+function safeSlice(str, end) {
+  if (end <= 0) return "";
+  if (end >= str.length) return str;
+  const code = str.charCodeAt(end - 1);
+  const safeEnd = code >= 55296 && code <= 56319 ? end - 1 : end;
+  return str.slice(0, safeEnd);
+}
 var ENCRYPTED_CONTENT_KEY = "encryptedContent";
 var ENCRYPTED_CONTENT_REDACTION_THRESHOLD = 256;
 var DEFAULT_OBSERVER_TOOL_RESULT_MAX_TOKENS = 1e4;
@@ -2504,7 +2574,7 @@ function truncateStringByTokens(text, maxTokens) {
     return text;
   }
   const buildCandidate = (sliceEnd) => {
-    const visible = text.slice(0, sliceEnd);
+    const visible = safeSlice(text, sliceEnd);
     return `${visible}
 ... [truncated ~${totalTokens - tokenx.estimateTokenCount(visible)} tokens]`;
   };
@@ -3058,8 +3128,11 @@ function formatObserverAttachmentPlaceholder(part, counter) {
   return label ? `[${attachmentType} #${attachmentId}: ${label}]` : `[${attachmentType} #${attachmentId}]`;
 }
 function formatObserverPartLine(title, body, time, previousTime) {
-  const timeLabel = time && time !== previousTime ? ` (${time})` : "";
-  return `${title}${timeLabel}: ${body}`;
+  const timeLabel = time && time !== previousTime ? `(${time})` : "";
+  if (!title) {
+    return timeLabel ? `${timeLabel}: ${body}` : body;
+  }
+  return `${title}${timeLabel ? ` ${timeLabel}` : ""}: ${body}`;
 }
 function normalizeObserverCreatedAt(createdAt) {
   if (createdAt instanceof Date) {
@@ -3095,6 +3168,16 @@ function formatObserverLines(lines, context = {}) {
     context: { previousDate, previousTime }
   };
 }
+function getTemporalGapMarkerText(msg) {
+  const metadata = typeof msg.content === "object" && msg.content && "metadata" in msg.content ? msg.content.metadata : void 0;
+  if (metadata?.reminderType === "temporal-gap" && typeof metadata.gapText === "string") {
+    return metadata.gapText;
+  }
+  if (typeof metadata?.systemReminder === "object" && metadata.systemReminder && "type" in metadata.systemReminder && metadata.systemReminder.type === "temporal-gap" && "gapText" in metadata.systemReminder && typeof metadata.systemReminder.gapText === "string") {
+    return metadata.systemReminder.gapText;
+  }
+  return void 0;
+}
 function formatObserverMessage(msg, counter, options) {
   const maxLen = options?.maxPartLength;
   const maxToolResultTokens = options?.maxToolResultTokens ?? DEFAULT_OBSERVER_TOOL_RESULT_MAX_TOKENS;
@@ -3102,6 +3185,7 @@ function formatObserverMessage(msg, counter, options) {
   const attachments = [];
   const messageCreatedAt = normalizeObserverCreatedAt(msg.createdAt);
   let lines = [];
+  const temporalGapText = isTemporalGapMarker(msg) ? getTemporalGapMarkerText(msg) : void 0;
   const pushLine = (title, body, createdAt) => {
     if (!body) {
       return;
@@ -3114,7 +3198,9 @@ function formatObserverMessage(msg, counter, options) {
       body
     });
   };
-  if (typeof msg.content === "string") {
+  if (temporalGapText) {
+    pushLine("", temporalGapText, messageCreatedAt);
+  } else if (typeof msg.content === "string") {
     pushLine(role, maybeTruncate(msg.content, maxLen), messageCreatedAt);
   } else if (msg.content?.parts && Array.isArray(msg.content.parts) && msg.content.parts.length > 0) {
     msg.content.parts.forEach((part) => {
@@ -3215,8 +3301,8 @@ function buildObserverHistoryMessage(messages, options) {
 }
 function maybeTruncate(str, maxLen) {
   if (!maxLen || str.length <= maxLen) return str;
-  const truncated = str.slice(0, maxLen);
-  const remaining = str.length - maxLen;
+  const truncated = safeSlice(str, maxLen);
+  const remaining = str.length - truncated.length;
   return `${truncated}
 ... [truncated ${remaining} characters]`;
 }
@@ -3520,7 +3606,7 @@ function sanitizeObservationLines(observations) {
   let changed = false;
   for (let i = 0; i < lines.length; i++) {
     if (lines[i].length > MAX_OBSERVATION_LINE_CHARS) {
-      lines[i] = lines[i].slice(0, MAX_OBSERVATION_LINE_CHARS) + " \u2026 [truncated]";
+      lines[i] = safeSlice(lines[i], MAX_OBSERVATION_LINE_CHARS) + " \u2026 [truncated]";
       changed = true;
     }
   }
@@ -8859,6 +8945,110 @@ ${grouped}` : grouped;
     });
   }
 };
+
+// src/processors/observational-memory/temporal-markers.ts
+var TEMPORAL_GAP_REMINDER_TYPE = "temporal-gap";
+function getTemporalGapReminderText(gapText, timestamp) {
+  return `${gapText} \u2014 ${formatTemporalTimestamp(new Date(timestamp))}`;
+}
+function getTemporalGapReminderMetadata(message, gapText, gapMs, timestamp) {
+  const formattedTimestamp = formatTemporalTimestamp(new Date(timestamp));
+  return {
+    reminderType: TEMPORAL_GAP_REMINDER_TYPE,
+    gapText,
+    gapMs,
+    timestamp: formattedTimestamp,
+    timestampMs: timestamp,
+    precedesMessageId: message.id,
+    systemReminder: {
+      type: TEMPORAL_GAP_REMINDER_TYPE,
+      message: getTemporalGapReminderText(gapText, timestamp),
+      gapText,
+      gapMs,
+      timestamp: formattedTimestamp,
+      timestampMs: timestamp,
+      precedesMessageId: message.id
+    }
+  };
+}
+function isTemporalGapMarkerForMessage(message, targetMessageId) {
+  if (!isTemporalGapMarker(message)) {
+    return false;
+  }
+  const metadata = message.content.metadata;
+  if (metadata?.precedesMessageId === targetMessageId) {
+    return true;
+  }
+  return metadata?.systemReminder?.type === TEMPORAL_GAP_REMINDER_TYPE && metadata.systemReminder.precedesMessageId === targetMessageId;
+}
+function createTemporalGapMarker(message, gapText, gapMs, timestamp) {
+  const metadata = getTemporalGapReminderMetadata(message, gapText, gapMs, timestamp);
+  return {
+    id: `__temporal_gap_${crypto.randomUUID()}`,
+    role: "user",
+    createdAt: new Date(timestamp - 1),
+    threadId: message.threadId,
+    resourceId: message.resourceId,
+    content: {
+      format: 2,
+      parts: [
+        {
+          type: "text",
+          text: `<system-reminder type="${TEMPORAL_GAP_REMINDER_TYPE}" precedesMessageId="${message.id}">${getTemporalGapReminderText(gapText, timestamp)}</system-reminder>`
+        }
+      ],
+      metadata
+    }
+  };
+}
+async function insertTemporalGapMarkers({
+  messageList,
+  writer
+}) {
+  const inputMessages = messageList.get.input.db().filter((message) => Boolean(message));
+  const latestInputMessage = inputMessages.at(-1);
+  if (!latestInputMessage || isTemporalGapMarker(latestInputMessage)) {
+    return;
+  }
+  const allMessages = messageList.get.all.db().filter((message) => Boolean(message));
+  const latestInputIndex = allMessages.findIndex((message) => message.id === latestInputMessage.id);
+  if (latestInputIndex <= 0) {
+    return;
+  }
+  if (allMessages.some((message) => isTemporalGapMarkerForMessage(message, latestInputMessage.id))) {
+    return;
+  }
+  let previousNonMarker;
+  for (let index = latestInputIndex - 1; index >= 0; index--) {
+    const candidate = allMessages[index];
+    if (candidate && !isTemporalGapMarker(candidate)) {
+      previousNonMarker = candidate;
+      break;
+    }
+  }
+  if (!previousNonMarker) {
+    return;
+  }
+  const timestamp = getMessagePartTimestamp(latestInputMessage, "first");
+  const gapMs = timestamp - getMessagePartTimestamp(previousNonMarker, "last");
+  const gapText = formatTemporalGap(gapMs);
+  if (!gapText) {
+    return;
+  }
+  const reminderMetadata = getTemporalGapReminderMetadata(latestInputMessage, gapText, gapMs, timestamp);
+  await writer?.custom({
+    type: "data-system-reminder",
+    data: {
+      message: getTemporalGapReminderText(gapText, timestamp),
+      ...reminderMetadata
+    },
+    transient: true
+  });
+  const marker = createTemporalGapMarker(latestInputMessage, gapText, gapMs, timestamp);
+  messageList.add(marker, "input");
+}
+
+// src/processors/observational-memory/processor.ts
 function getOmObservabilityContext(args) {
   if (!args.tracing || !args.tracingContext || !args.loggerVNext || !args.metrics) {
     return void 0;
@@ -8881,11 +9071,14 @@ var ObservationalMemoryProcessor = class {
   engine;
   /** Memory instance for loading context. */
   memory;
+  /** Whether temporal-gap reminder markers should be inserted. */
+  temporalMarkers;
   /** Active turn — created on first processInputStep, ended on processOutputResult. */
   turn;
-  constructor(engine, memory) {
+  constructor(engine, memory, options) {
     this.engine = engine;
     this.memory = memory;
+    this.temporalMarkers = options?.temporalMarkers ?? false;
   }
   // ─── Processor lifecycle hooks ──────────────────────────────────────────
   async processInputStep(args) {
@@ -8944,6 +9137,9 @@ var ObservationalMemoryProcessor = class {
         this.turn.writer = writer;
         this.turn.requestContext = requestContext;
         await this.turn.start(this.memory);
+        if (stepNumber === 0 && this.temporalMarkers) {
+          await insertTemporalGapMarkers({ messageList, writer });
+        }
         state.__omTurn = this.turn;
       }
       const observabilityContext = getOmObservabilityContext(args);
@@ -9108,5 +9304,5 @@ exports.stripEphemeralAnchorIds = stripEphemeralAnchorIds;
 exports.stripObservationGroups = stripObservationGroups;
 exports.truncateStringByTokens = truncateStringByTokens;
 exports.wrapInObservationGroup = wrapInObservationGroup;
-//# sourceMappingURL=chunk-HAUWGR76.cjs.map
-//# sourceMappingURL=chunk-HAUWGR76.cjs.map
+//# sourceMappingURL=chunk-5MTY2UYL.cjs.map
+//# sourceMappingURL=chunk-5MTY2UYL.cjs.map