@fluidframework/container-runtime 2.0.0-dev.6.4.0.191515 → 2.0.0-dev.7.2.0.203917

package/src/opLifecycle/README.md

@@ -1,5 +1,19 @@
 # Configs and feature gates for solving the 1MB limit.
 
+## Table of contents
+
+-   [Introduction](#introduction)
+    -   [How batching works](#how-batching-works)
+-   [Compression](#compression)
+-   [Grouped batching](#grouped-batching)
+    -   [Risks](#risks)
+-   [Chunking for compression](#chunking-for-compression)
+-   [Disabling in case of emergency](#disabling-in-case-of-emergency)
+-   [Example configs](#example-configs)
+-   [Note about performance and latency](#note-about-performance-and-latency)
+-   [How it works](#how-it-works)
+-   [How grouped batching works](#how-grouped-batching-works)
+
 ## Introduction
 
 There is a current limitation regarding the size of the payload a Fluid client can send and receive. [The limit is 1MB per payload](https://github.com/microsoft/FluidFramework/issues/9023) and it is currently enforced explicitly with the `BatchTooLarge` error which closes the container.
@@ -8,17 +22,35 @@ There are two features which can be used to work around this size limit, batch c
 
 By default, the runtime is configured with a max batch size of `716800` bytes, which is lower than the 1MB limit. The reason for the lower value is to account for possible overhead from the op envelope and metadata.
 
-## Table of contents
+### How batching works
 
--   [Introduction](#introduction)
-    -   [Compression](#compression)
-    -   [Grouped batching](#grouped-batching)
-        -   [Risks](#risks)
-    -   [Chunking for compression](#chunking-for-compression)
-    -   [Disabling in case of emergency](#disabling-in-case-of-emergency)
-    -   [Example configs](#example-configs)
-    -   [How it works](#how-it-works)
-    -   [How grouped batching works](#how-grouped-batching-works)
+Batching in the context of Fluid ops is a way in which the framework accumulates and applies ops. A batch is a group of ops accumulated within a single JS turn, which will be broadcasted in the same order to all the other connected clients and applied synchronously. Additional logic and validation ensure that batches are never interleaved, nested or interrupted and they are processed in isolation without interleaving of ops from other clients.
+
+The way batches are formed is governed by the `FlushMode` setting of the `ContainerRuntimeOptions` and it is immutable for the entire lifetime of the runtime and subsequently the container.
+
+```
+export enum FlushMode {
+    /**
+     * In Immediate flush mode the runtime will immediately send all operations to the driver layer.
+     */
+    Immediate,
+
+    /**
+     * When in TurnBased flush mode the runtime will buffer operations in the current turn and send them as a single
+     * batch at the end of the turn. The flush call on the runtime can be used to force send the current batch.
+     */
+    TurnBased,
+}
+```
+
+What this means is that `FlushMode.Immediate` will send each op in its own payload to the server, while `FlushMode.TurnBased` will accumulate all ops in a single JS turn and send them together in the same payload. Technically, `FlushMode.Immediate` can be simulated with `FlushMode.TurnBased` by interrupting the JS turn after producing only one op (for example by pausing the execution to wait on a promise). Therefore, for all intents and purposes, `FlushMode.Immediate` enables all batches to have only one op.
+
+**By default, Fluid uses `FlushMode.TurnBased`** as:
+
+-   it is more efficient from an I/O perspective (batching ops overall decrease the number of payloads sent to the server)
+-   reduces concurrency related bugs, as it ensures that all ops generated within the same JS turn are also applied by all other clients within a single JS turn. Clients using the same pattern can safely assume ops will be applied exactly as they are observed locally. The alternative would be for ops to be both produced and applied with interruptions (which may involve processing input or rendering), invalidating the state based off which the changes were produced.
+
+As `FlushMode.TurnBased` accumulates ops, it is the most vulnerable to run into the 1MB socket limit.
 
 ## Compression
 
@@ -29,6 +61,8 @@ By default, the runtime is configured with a max batch size of `716800` bytes, w
 -   `minimumBatchSizeInBytes` – the minimum size of the batch for which compression should kick in. If the payload is too small, compression may not yield too many benefits. To target the original 1MB issue, a good value here would be to match the default maxBatchSizeInBytes (972800), however, experimentally, a good lower value could be at around 614400 bytes. Setting this value to `Number.POSITIVE_INFINITY` will disable compression.
 -   `compressionAlgorithm` – currently, only `lz4` is supported.
 
+Compression is relevant for both `FlushMode.TurnBased` and `FlushMode.Immediate` as it only targets the contents of the ops and not the number of ops in a batch. Compression is opaque to the server and implementations of the Fluid protocol do not need to alter their behavior to support this client feature.
+
 ## Grouped batching
 
 **Note: This feature is currently considered experimental and is not ready for production usage.**
@@ -71,6 +105,8 @@ If all prerequisites in the previous section are met, enabling the feature can b
 
 In case of emergency grouped batching can be disabled at runtime, using feature gates. If `"Fluid.ContainerRuntime.DisableGroupedBatching"` is set to `true`, it will disable grouped batching if enabled from `IContainerRuntimeOptions` in the code.
 
+Grouped batching is only relevant for `FlushMode.TurnBased` as it only targets the number of ops in a batch. Grouped batching is opaque to the server and implementations of the Fluid protocol do not need to alter their behavior to support this client feature.
+
 ## Chunking for compression
 
 **Op chunking for compression targets payloads which exceed the max batch size after compression.** So, only payloads which are already compressed. By default, the feature is enabled.
@@ -79,6 +115,8 @@ The `IContainerRuntimeOptions.chunkSizeInBytes` property is the only configurati
 
 This config would govern chunking compressed batches only. We will not be enabling chunking across all types of ops/batches but **only when compression is enabled and when the batch is compressed**, and its payload size is more than `IContainerRuntimeOptions.chunkSizeInBytes`.
 
+Chunking is relevant for both `FlushMode.TurnBased` and `FlushMode.Immediate` as it only targets the contents of the ops and not the number of ops in a batch. Chunking is opaque to the server and implementations of the Fluid protocol do not need to alter their behavior to support this client feature.
+
 ## Disabling in case of emergency
 
 If the features are enabled using the configs, they can be disabled at runtime via feature gates as following:
@@ -102,32 +140,19 @@ By default, the runtime is configured with the following values related to compr
     }
 ```
 
-To use compression but disable chunking:
+To enable grouped batching, use the following property:
 
 ```
     const runtimeOptions: IContainerRuntimeOptions = {
-        chunkSizeInBytes: Number.POSITIVE_INFINITY,
+        enableGroupedBatching: true,
     }
 ```
 
-To disable compression (will also disable chunking, as chunking works only for compressed batches):
+## Note about performance and latency
 
-```
-    const runtimeOptions: IContainerRuntimeOptions = {
-        compressionOptions: {
-            minimumBatchSizeInBytes: Number.POSITIVE_INFINITY,
-            compressionAlgorithm: CompressionAlgorithms.lz4,
-        },
-    }
-```
+In terms of performance and impact on latency, the results greatly depend on payload size, payload structure, network speed and CPU speed. Therefore, customers must perform the required measurements and adjust the settings according to their scenarios.
 
-To enable grouped batching:
-
-```
-    const runtimeOptions: IContainerRuntimeOptions = {
-        enableGroupedBatching: true,
-    }
-```
+In general, compression offers a trade-off between higher compute costs, lower bandwidth consumption and lower storage requirements, while chunking slightly increases latency due to the overhead of splitting an op, sending the chunks and reconstructing them on each client. Grouped batching heavily decreases the number of ops observed by the server and slightly decreases the bandwidth requirements as it merges all the ops in a batch into a single op and also eliminates the op envelope overhead.
 
 ## How it works
 
@@ -212,8 +237,6 @@ On the receiving end, the client will accumulate chunks 1 and 2 and keep them in
 
 ## How grouped batching works
 
-**Note: There are plans to replace empty ops with something more efficient when doing grouped batching AB#4092**
-
 Given the following baseline batch:
 
 ```
@@ -223,68 +246,70 @@ Given the following baseline batch:
 +---------------+---------------+---------------+---------------+---------------+
 ```
 
-Compressed batch:
+Grouped batch:
 
 ```
-+--------------------+-----------------+-----------------+-----------------+-----------------+
-| Op 1               | Op 2            | Op 3            | Op 4            | Op 5            |
-| Contents: "abcde"  | Contents: empty | Contents: empty | Contents: empty | Contents: empty |
-| Compression: 'lz4' |                 |                 |                 |                 |
-+--------------------+-----------------+-----------------+-----------------+-----------------+
++---------------------------------------------------------------------------------------------------------------------+
+| Op 1                   Contents: +----------------+---------------+---------------+---------------+---------------+ |
+| Type: "groupedBatch"             | Op 1           | Op 2          | Op 3          | Op 4          | Op 5          | |
+|                                  | Contents: "a"  | Contents: "b" | Contents: "c" | Contents: "d" | Contents: "e" | |
+|                                  +----------------+---------------+---------------+---------------+---------------+ |
++---------------------------------------------------------------------------------------------------------------------+
 ```
 
-Grouped batch:
+Compressed batch:
 
 ```
-+---------------------------------------------------------------------------------------------------------------------------------+
-| Op 1                   Contents: +--------------------+-----------------+-----------------+-----------------+-----------------+ |
-| SeqNum: 1                        | Op 1               | Op 2            | Op 3            | Op 4            | Op 5            | |
-| Type: "groupedBatch"             | Contents: "abcde"  | Contents: empty | Contents: empty | Contents: empty | Contents: empty | |
-|                                  | Compression: 'lz4' |                 |                 |                 |                 | |
-|                                  +--------------------+-----------------+-----------------+-----------------+-----------------+ |
-+---------------------------------------------------------------------------------------------------------------------------------+
++-------------------------------------------------------------------------------------------------------------------------+
+| Op 1                   Contents: +------------------------------------------------------------------------------------+ |
+| Compression: 'lz4'               | Type: "groupedBatch"                                                               | |
+|                                  | +----------------+---------------+---------------+---------------+---------------+ | |
+|                                  | | Op 1           | Op 2          | Op 3          | Op 4          | Op 5          | | |
+|                                  | | Contents: "a"  | Contents: "b" | Contents: "c" | Contents: "d" | Contents: "e" | | |
+|                                  | +----------------+---------------+---------------+---------------+---------------+ | |
+|                                  +------------------------------------------------------------------------------------+ |
++-------------------------------------------------------------------------------------------------------------------------+
 ```
 
 Can produce the following chunks:
 
 ```
-+-------------------------------------------------+
-| Chunk 1/2    Contents: +----------------------+ |
-| SeqNum: 1              |  +-----------------+ | |
-|                        |  | Contents: "abc" | | |
-|                        |  +-----------------+ | |
-|                        +----------------------+ |
-+-------------------------------------------------+
++------------------------------------------------+
+| Chunk 1/2    Contents: +---------------------+ |
+|                        | +-----------------+ | |
+|                        | | Contents: "abc" | | |
+|                        | +-----------------+ | |
+|                        +---------------------+ |
++------------------------------------------------+
 ```
 
 ```
-+--------------------------------------------------------------------------------------------------------------------------+
-| Chunk 2/2    Contents: +---------------------------------------------------------------------------------------------+ | |
-| SeqNum: 2              |  +----------------+-----------------+-----------------+-----------------+-----------------+ | | |
-|                        |  | Contents: "de" | Contents: empty | Contents: empty | Contents: empty | Contents: empty | | | |
-|                        |  +----------------+-----------------+-----------------+-----------------+-----------------+ | | |
-|                        +---------------------------------------------------------------------------------------------+ | |
-+--------------------------------------------------------------------------------------------------------------------------+
++-----------------------------------------------+
+| Chunk 2/2    Contents: +--------------------+ |
+|                        | +----------------+ | |
+|                        | | Contents: "de" | | |
+|                        | +----------------+ | |
+|                        +--------------------+ |
++-----------------------------------------------+
 ```
 
 -   Send to service
 -   Service acks ops sent
 -   Receive chunks from service
--   Recompile to the grouped batch step
+-   Recompile to the compression step
 
-Ungrouped batch:
+Decompressed batch:
 
 ```
-+--------------------+-----------------+-----------------+-----------------+-----------------+
-| Op 1               | Op 2            | Op 3            | Op 4            | Op 5            |
-| Contents: "abcde"  | Contents: empty | Contents: empty | Contents: empty | Contents: empty |
-| SeqNum: 2          | SeqNum: 2       | SeqNum: 2       | SeqNum: 2       | SeqNum: 2       |
-| ClientSeqNum: 1    | ClientSeqNum: 2 | ClientSeqNum: 3 | ClientSeqNum: 4 | ClientSeqNum: 5 |
-| Compression: 'lz4' |                 |                 |                 |                 |
-+--------------------+-----------------+-----------------+-----------------+-----------------+
++---------------------------------------------------------------------------------------------------------------------+
+| Op 1                   Contents: +----------------+---------------+---------------+---------------+---------------+ |
+| SeqNum: 2                        | Op 1           | Op 2            | Op 3        | Op 4          | Op 5          | |
+| Type: "groupedBatch"             | Contents: "a"  | Contents: "b" | Contents: "c" | Contents: "d" | Contents: "e" | |
+|                                  +----------------+---------------+---------------+---------------+---------------+ |
++---------------------------------------------------------------------------------------------------------------------+
 ```
 
-Uncompressed batch:
+Ungrouped batch:
 
 ```
 +-----------------+-----------------+-----------------+-----------------+-----------------+

package/src/opLifecycle/definitions.ts

@@ -5,7 +5,8 @@
 
 import { IBatchMessage } from "@fluidframework/container-definitions";
 import { ISequencedDocumentMessage, MessageType } from "@fluidframework/protocol-definitions";
-import { CompressionAlgorithms, ContainerMessageType } from "..";
+import { CompressionAlgorithms } from "..";
+import { ContainerMessageType } from "../messageTypes";
 
 /**
  * Batch message type used internally by the runtime
@@ -51,6 +52,9 @@ export interface IBatchCheckpoint {
 	rollback: (action: (message: BatchMessage) => void) => void;
 }
 
+/**
+ * @public
+ */
 export interface IChunkedOp {
 	chunkId: number;
 	totalChunks: number;

package/src/opLifecycle/opDecompressor.ts

@@ -50,14 +50,6 @@ export class OpDecompressor {
 		) {
 			// Beginning of a compressed batch
 			assert(this.activeBatch === false, 0x4b8 /* shouldn't have multiple active batches */);
-			if (message.compression) {
-				// lz4 is the only supported compression algorithm for now
-				assert(
-					message.compression === CompressionAlgorithms.lz4,
-					0x4b9 /* lz4 is currently the only supported compression algorithm */,
-				);
-			}
-
 			this.activeBatch = true;
 
 			const contents = IsoBuffer.from(

package/src/opLifecycle/opGroupingManager.ts

@@ -5,7 +5,7 @@
 
 import { assert } from "@fluidframework/core-utils";
 import { ISequencedDocumentMessage } from "@fluidframework/protocol-definitions";
-import { ContainerMessageType } from "..";
+import { ContainerMessageType } from "../messageTypes";
 import { IBatch } from "./definitions";
 
 /**
@@ -22,9 +22,7 @@ interface IGroupedMessage {
 	compression?: string;
 }
 
-function isGroupContents(
-	opContents: IGroupedBatchMessageContents | { type?: unknown } | undefined,
-): opContents is IGroupedBatchMessageContents {
+function isGroupContents(opContents: any): opContents is IGroupedBatchMessageContents {
 	return opContents?.type === OpGroupingManager.groupedBatchOp;
 }
 

package/src/opLifecycle/opSplitter.ts

@@ -12,7 +12,7 @@ import { assert } from "@fluidframework/core-utils";
 import { IBatchMessage } from "@fluidframework/container-definitions";
 import { ISequencedDocumentMessage } from "@fluidframework/protocol-definitions";
 import { ITelemetryBaseLogger } from "@fluidframework/core-interfaces";
-import { ContainerMessageType, ContainerRuntimeMessage } from "../containerRuntime";
+import { ContainerMessageType, ContainerRuntimeChunkedOpMessage } from "../messageTypes";
 import { estimateSocketSize } from "./batchManager";
 import { BatchMessage, IBatch, IChunkedOp, IMessageProcessingResult } from "./definitions";
 
@@ -210,7 +210,7 @@ const chunkToBatchMessage = (
 	referenceSequenceNumber: number,
 	metadata: Record<string, unknown> | undefined = undefined,
 ): BatchMessage => {
-	const payload: ContainerRuntimeMessage = {
+	const payload: ContainerRuntimeChunkedOpMessage = {
 		type: ContainerMessageType.ChunkedOp,
 		contents: chunk,
 	};

package/src/opLifecycle/outbox.ts

@@ -65,10 +65,13 @@ export interface IOutboxParameters {
 export function getLongStack<T>(action: () => T, length: number = 50): T {
 	const errorObj = Error as any;
 	if (
+		/* eslint-disable @typescript-eslint/prefer-nullish-coalescing */
+		// ?? is not logically equivalent when the first clause returns false.
 		(
 			Object.getOwnPropertyDescriptor(errorObj, "stackTraceLimit") ||
 			Object.getOwnPropertyDescriptor(Object.getPrototypeOf(errorObj), "stackTraceLimit")
 		)?.writable !== true
+		/* eslint-enable @typescript-eslint/prefer-nullish-coalescing */
 	) {
 		return action();
 	}

package/src/opLifecycle/remoteMessageProcessor.ts

@@ -6,13 +6,21 @@
 import { ISequencedDocumentMessage, MessageType } from "@fluidframework/protocol-definitions";
 import {
 	ContainerMessageType,
-	ContainerRuntimeMessage,
-	SequencedContainerRuntimeMessage,
-} from "../containerRuntime";
+	type InboundContainerRuntimeMessage,
+	type InboundSequencedContainerRuntimeMessage,
+	type InboundSequencedContainerRuntimeMessageOrSystemMessage,
+	type InboundSequencedRecentlyAddedContainerRuntimeMessage,
+} from "../messageTypes";
 import { OpDecompressor } from "./opDecompressor";
 import { OpGroupingManager } from "./opGroupingManager";
 import { OpSplitter } from "./opSplitter";
 
+/**
+ * Stateful class for processing incoming remote messages as the virtualization measures are unwrapped,
+ * potentially across numerous inbound ops.
+ *
+ * @internal
+ */
 export class RemoteMessageProcessor {
 	constructor(
 		private readonly opSplitter: OpSplitter,
@@ -30,14 +38,25 @@ export class RemoteMessageProcessor {
 
 	/**
 	 * Ungroups and Unchunks the runtime ops encapsulated by the single remoteMessage received over the wire
-	 * @param remoteMessage - A message from another client, likely a chunked/grouped op
-	 * @returns the ungrouped, unchunked, unpacked SequencedContainerRuntimeMessage encapsulated in the remote message
+	 * @param remoteMessageCopy - A shallow copy of a message from another client, possibly virtualized
+	 * (grouped, compressed, and/or chunked).
+	 * Being a shallow copy, it's considered mutable, meaning no other Container or other parallel procedure
+	 * depends on this object instance.
+	 * Note remoteMessageCopy.contents (and other object props) MUST not be modified,
+	 * but may be overwritten (as is the case with contents).
+	 * @returns the unchunked, decompressed, ungrouped, unpacked SequencedContainerRuntimeMessages encapsulated in the remote message.
+	 * For ops that weren't virtualized (e.g. System ops that the ContainerRuntime will ultimately ignore),
+	 * a singleton array [remoteMessageCopy] is returned
 	 */
-	public process(remoteMessage: ISequencedDocumentMessage): ISequencedDocumentMessage[] {
-		const result: ISequencedDocumentMessage[] = [];
+	public process(
+		remoteMessageCopy: ISequencedDocumentMessage,
+	): InboundSequencedContainerRuntimeMessageOrSystemMessage[] {
+		const result: InboundSequencedContainerRuntimeMessageOrSystemMessage[] = [];
+
+		ensureContentsDeserialized(remoteMessageCopy);
 
 		// Ungroup before and after decompression for back-compat (cleanup tracked by AB#4371)
-		for (const ungroupedMessage of this.opGroupingManager.ungroupOp(copy(remoteMessage))) {
+		for (const ungroupedMessage of this.opGroupingManager.ungroupOp(remoteMessageCopy)) {
 			const message = this.opDecompressor.processMessage(ungroupedMessage).message;
 
 			for (let ungroupedMessage2 of this.opGroupingManager.ungroupOp(message)) {
@@ -50,7 +69,9 @@ export class RemoteMessageProcessor {
 				if (chunkProcessingResult.state !== "Processed") {
 					// If the message is not chunked or if the splitter is still rebuilding the original message,
 					// there is no need to continue processing
-					result.push(ungroupedMessage2);
+					result.push(
+						ungroupedMessage2 as InboundSequencedContainerRuntimeMessageOrSystemMessage,
+					);
 					continue;
 				}
 
@@ -68,7 +89,9 @@ export class RemoteMessageProcessor {
 						if (decompressionAfterChunking.state === "Skipped") {
 							// After chunking, if the original message was not compressed,
 							// there is no need to continue processing
-							result.push(ungroupedMessageAfterChunking2);
+							result.push(
+								ungroupedMessageAfterChunking2 as InboundSequencedContainerRuntimeMessageOrSystemMessage,
+							);
 							continue;
 						}
 
@@ -84,40 +107,38 @@ export class RemoteMessageProcessor {
 	}
 }
 
-const copy = (remoteMessage: ISequencedDocumentMessage): ISequencedDocumentMessage => {
-	// Do shallow copy of message, as the processing flow will modify it.
-	// There might be multiple container instances receiving same message
-	// We do not need to make deep copy, as each layer will just replace message.content itself,
-	// but would not modify contents details
-	const message = { ...remoteMessage };
-
+/** Takes an incoming message and if the contents is a string, JSON.parse's it in place */
+function ensureContentsDeserialized(mutableMessage: ISequencedDocumentMessage): void {
 	// back-compat: ADO #1385: eventually should become unconditional, but only for runtime messages!
 	// System message may have no contents, or in some cases (mostly for back-compat) they may have actual objects.
 	// Old ops may contain empty string (I assume noops).
-	if (typeof message.contents === "string" && message.contents !== "") {
-		message.contents = JSON.parse(message.contents);
+	if (typeof mutableMessage.contents === "string" && mutableMessage.contents !== "") {
+		mutableMessage.contents = JSON.parse(mutableMessage.contents);
 	}
-
-	return message;
-};
+}
 
 /**
- * For a given message, it moves the nested ContainerRuntimeMessage props one level up.
+ * For a given message, it moves the nested InboundContainerRuntimeMessage props one level up.
  *
  * The return type illustrates the assumption that the message param
- * becomes a ContainerRuntimeMessage by the time the function returns
- * (but there is no runtime validation of the 'type' or 'compatDetails' values)
+ * becomes a InboundSequencedContainerRuntimeMessage by the time the function returns
+ * (but there is no runtime validation of the 'type' or 'compatDetails' values).
  */
 function unpack(
 	message: ISequencedDocumentMessage,
-): asserts message is SequencedContainerRuntimeMessage {
-	const innerContents = message.contents as ContainerRuntimeMessage;
-
-	// We're going to turn message into a SequencedContainerRuntimeMessage in-place
-	const sequencedContainerRuntimeMessage = message as SequencedContainerRuntimeMessage;
-	sequencedContainerRuntimeMessage.type = innerContents.type;
-	sequencedContainerRuntimeMessage.contents = innerContents.contents;
-	sequencedContainerRuntimeMessage.compatDetails = innerContents.compatDetails;
+): asserts message is InboundSequencedContainerRuntimeMessage {
+	// We assume the contents is an InboundContainerRuntimeMessage (the message is "packed")
+	const contents = message.contents as InboundContainerRuntimeMessage;
+
+	// We're going to unpack message in-place (promoting those properties of contents up to message itself)
+	const messageUnpacked = message as InboundSequencedContainerRuntimeMessage;
+
+	messageUnpacked.type = contents.type;
+	messageUnpacked.contents = contents.contents;
+	if ("compatDetails" in contents) {
+		(messageUnpacked as InboundSequencedRecentlyAddedContainerRuntimeMessage).compatDetails =
+			contents.compatDetails;
+	}
 }
 
 /**

package/src/packageVersion.ts

@@ -6,4 +6,4 @@
  */
 
 export const pkgName = "@fluidframework/container-runtime";
-export const pkgVersion = "2.0.0-dev.6.4.0.191515";
+export const pkgVersion = "2.0.0-dev.7.2.0.203917";

package/src/pendingStateManager.ts

@@ -11,28 +11,15 @@ import { ICriticalContainerError } from "@fluidframework/container-definitions";
 import { ISequencedDocumentMessage } from "@fluidframework/protocol-definitions";
 import { DataProcessingError, ITelemetryLoggerExt } from "@fluidframework/telemetry-utils";
 
-import { ContainerMessageType, SequencedContainerRuntimeMessage } from "./containerRuntime";
+import { ContainerMessageType, InboundSequencedContainerRuntimeMessage } from "./messageTypes";
 import { pkgVersion } from "./packageVersion";
 import { IBatchMetadata } from "./metadata";
 
-/**
- * ! TODO: Remove this interface in "2.0.0-internal.7.0.0" once we only read IPendingMessageNew (AB#4763)
- */
-export interface IPendingMessageOld {
-	type: "message";
-	messageType: ContainerMessageType;
-	clientSequenceNumber: number;
-	referenceSequenceNumber: number;
-	content: any;
-	localOpMetadata: unknown;
-	opMetadata: Record<string, unknown> | undefined;
-}
-
 /**
  * This represents a message that has been submitted and is added to the pending queue when `submit` is called on the
  * ContainerRuntime. This message has either not been ack'd by the server or has not been submitted to the server yet.
  */
-export interface IPendingMessageNew {
+export interface IPendingMessage {
 	type: "message";
 	clientSequenceNumber: number;
 	referenceSequenceNumber: number;
@@ -41,16 +28,11 @@ export interface IPendingMessageNew {
 	opMetadata: Record<string, unknown> | undefined;
 }
 
-/**
- * ! TODO: Remove this type in "2.0.0-internal.7.0.0" (AB#4763)
- */
-export type IPendingState = IPendingMessageOld | IPendingMessageNew;
-
 export interface IPendingLocalState {
 	/**
 	 * list of pending states, including ops and batch information
 	 */
-	pendingStates: IPendingState[];
+	pendingStates: IPendingMessage[];
 }
 
 export interface IPendingBatchMessage {
@@ -69,6 +51,26 @@ export interface IRuntimeStateHandler {
 	isActiveConnection: () => boolean;
 }
 
+/** Union of keys of T */
+type KeysOfUnion<T extends object> = T extends T ? keyof T : never;
+/** *Partial* type all possible combinations of properties and values of union T.
+ * This loosens typing allowing access to all possible properties without
+ * narrowing.
+ */
+type AnyComboFromUnion<T extends object> = { [P in KeysOfUnion<T>]?: T[P] };
+
+function buildPendingMessageContent(
+	// AnyComboFromUnion is needed need to gain access to compatDetails that
+	// is only defined for some cases.
+	message: AnyComboFromUnion<InboundSequencedContainerRuntimeMessage>,
+): string {
+	// IMPORTANT: Order matters here, this must match the order of the properties used
+	// when submitting the message.
+	const { type, contents, compatDetails } = message;
+	// Any properties that are not defined, won't be emitted by stringify.
+	return JSON.stringify({ type, contents, compatDetails });
+}
+
 /**
  * PendingStateManager is responsible for maintaining the messages that have not been sent or have not yet been
  * acknowledged by the server. It also maintains the batch information for both automatically and manually flushed
@@ -79,13 +81,13 @@ export interface IRuntimeStateHandler {
  * It verifies that all the ops are acked, are received in the right order and batch information is correct.
  */
 export class PendingStateManager implements IDisposable {
-	private readonly pendingMessages = new Deque<IPendingMessageNew>();
-	private readonly initialMessages = new Deque<IPendingMessageNew>();
+	private readonly pendingMessages = new Deque<IPendingMessage>();
+	private readonly initialMessages = new Deque<IPendingMessage>();
 
 	/**
 	 * Sequenced local ops that are saved when stashing since pending ops may depend on them
 	 */
-	private savedOps: IPendingMessageNew[] = [];
+	private savedOps: IPendingMessage[] = [];
 
 	private readonly disposeOnce = new Lazy<void>(() => {
 		this.initialMessages.clear();
@@ -149,29 +151,8 @@ export class PendingStateManager implements IDisposable {
 		initialLocalState: IPendingLocalState | undefined,
 		private readonly logger: ITelemetryLoggerExt | undefined,
 	) {
-		/**
-		 * Convert old local state format to the new format (IPendingMessageOld to IPendingMessageNew)
-		 * ! TODO: Remove this conversion in "2.0.0-internal.7.0.0" (AB#4763)
-		 */
 		if (initialLocalState?.pendingStates) {
-			for (const initialState of initialLocalState.pendingStates) {
-				let messageContent = initialState.content;
-				if (
-					(initialState as IPendingMessageOld).messageType !== undefined &&
-					typeof initialState.content !== "string"
-				) {
-					// Convert IPendingMessageOld to IPendingMessageNew
-					messageContent = JSON.stringify({
-						type: (initialState as IPendingMessageOld).messageType,
-						contents: initialState.content,
-					});
-				}
-				// Note: this object may contain "messageType" prop, but it should not be easily accesible due to interface being used
-				this.initialMessages.push({
-					...initialState,
-					content: messageContent,
-				});
-			}
+			this.initialMessages.push(...initialLocalState.pendingStates);
 		}
 	}
 
@@ -193,7 +174,7 @@ export class PendingStateManager implements IDisposable {
 		localOpMetadata: unknown,
 		opMetadata: Record<string, unknown> | undefined,
 	) {
-		const pendingMessage: IPendingMessageNew = {
+		const pendingMessage: IPendingMessage = {
 			type: "message",
 			clientSequenceNumber: -1, // dummy value (not to be used anywhere)
 			referenceSequenceNumber,
@@ -242,7 +223,7 @@ export class PendingStateManager implements IDisposable {
 	 * the batch information was preserved for batch messages.
 	 * @param message - The message that got ack'd and needs to be processed.
 	 */
-	public processPendingLocalMessage(message: SequencedContainerRuntimeMessage): unknown {
+	public processPendingLocalMessage(message: InboundSequencedContainerRuntimeMessage): unknown {
 		// Pre-processing part - This may be the start of a batch.
 		this.maybeProcessBatchBegin(message);
 
@@ -256,10 +237,7 @@ export class PendingStateManager implements IDisposable {
 
 		this.pendingMessages.shift();
 
-		// IMPORTANT: Order matters here, this must match the order of the properties used
-		// when submitting the message.
-		const { type, contents, compatDetails } = message;
-		const messageContent = JSON.stringify({ type, contents, compatDetails });
+		const messageContent = buildPendingMessageContent(message);
 
 		// Stringified content should match
 		if (pendingMessage.content !== messageContent) {
@@ -340,6 +318,7 @@ export class PendingStateManager implements IDisposable {
 							{
 								runtimeVersion: pkgVersion,
 								batchClientId:
+									// eslint-disable-next-line @typescript-eslint/prefer-nullish-coalescing
 									this.pendingBatchBeginMessage.clientId === null
 										? "null"
 										: this.pendingBatchBeginMessage.clientId,