msgpackr 1.5.0 → 1.5.4

package/README.md

@@ -7,6 +7,7 @@
 [![module](https://img.shields.io/badge/module-ESM%2FCJS-blue)](README.md)
 [![license](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
 
+<img align="right" src="./assets/performance.png" width="380"/>
 
 The msgpackr package is an extremely fast MessagePack NodeJS/JavaScript implementation. Currently, it is significantly faster than any other known implementations, faster than Avro (for JS), and generally faster than native V8 JSON.stringify/parse, on NodeJS. It also includes an optional record extension (the `r` in msgpackr), for defining record structures that makes MessagePack even faster and more compact, often over twice as fast as even native JSON functions, several times faster than other JS implementations, and 15-50% more compact. See the performance section for more details. Structured cloning (with support for cyclical references) is also supported through optional extensions.
 
@@ -53,7 +54,7 @@ receivingStream.on('data', (data) => {
 The `PackrStream` and `UnpackrStream` instances  will have also the record structure extension enabled by default (see below).
 
 ## Deno Usage
-Msgpackr modules are standard ESM modules and can be loaded directly from github (https://raw.githubusercontent.com/kriszyp/msgpackr/master/index.js) or downloaded and used directly in Deno. The standard pack/encode and unpack/decode functionality is available on Deno, like other platforms.
+Msgpackr modules are standard ESM modules and can be loaded directly from the [deno.land registry for msgpackr](https://deno.land/x/msgpackr) for use in Deno. The standard pack/encode and unpack/decode functionality is available on Deno, like other platforms.
 
 ## Browser Usage
 Msgpackr works as standalone JavaScript as well, and runs on modern browsers. It includes a bundled script, at `dist/index.js` for ease of direct loading:
@@ -162,12 +163,14 @@ The following options properties can be provided to the Packr or Unpackr constru
 * `mapsAsObjects` - If `true`, this will decode MessagePack maps and JS `Object`s with the map entries decoded to object properties. If `false`, maps are decoded as JavaScript `Map`s. This is disabled by default if `useRecords` is enabled (which allows `Map`s to be preserved), and is enabled by default if `useRecords` is disabled.
 * `useFloat32` - This will enable msgpackr to encode non-integer numbers as `float32`. See next section for possible values.
 * `variableMapSize` - This will use varying map size definition (fixmap, map16, map32) based on the number of keys when encoding objects, which yields slightly more compact encodings (for small objects), but is typically 5-10% slower during encoding. This is necessary if you need to use objects with more than 65535 keys. This is only relevant when record extension is disabled.
+* `bundleStrings` - If `true` this uses a custom extension that bundles strings together, so that they can be decoded more quickly on browsers and Deno that do not have access to the NodeJS addon. This a custom extension, so both encoder and decoder need to support this. This can yield significant decoding performance increases on browsers (30%-50%).
 * `copyBuffers` - When decoding a MessagePack with binary data (Buffers are encoded as binary data), copy the buffer rather than providing a slice/view of the buffer. If you want your input data to be collected or modified while the decoded embedded buffer continues to live on, you can use this option (there is extra overhead to copying).
 * `useTimestamp32` - Encode JS `Date`s in 32-bit format when possible by dropping the milliseconds. This is a more efficient encoding of dates. You can also cause dates to use 32-bit format by manually setting the milliseconds to zero (`date.setMilliseconds(0)`).
 * `sequential` - Encode structures in serialized data, and reference previously encoded structures with expectation that decoder will read the encoded structures in the same order as encoded, with `unpackMultiple`.
 * `largeBigIntToFloat` - If a bigint needs to be encoded that is larger than will fit in 64-bit integers, it will be encoded as a float-64 (otherwise will throw a RangeError).
 * `encodeUndefinedAsNil` - Encodes a value of `undefined` as a MessagePack `nil`, the same as a `null`.
 * `int64AsNumber` - This will decode uint64 and int64 numbers as standard JS numbers rather than as bigint numbers.
+* `onInvalidDate` - This can be provided as function that will be called when an invalid date is provided. The function can throw an error, or return a value that will be encoded in place of the invalid date. If not provided, an invalid date will be encoded as an invalid timestamp (which decodes with msgpackr back to an invalid date).
 
 ### 32-bit Float Options
 By default all non-integer numbers are serialized as 64-bit float (double). This is fast, and ensures maximum precision. However, often real-world data doesn't not need 64-bits of precision, and using 32-bit encoding can be much more space efficient. There are several options that provide more efficient encodings. Using the decimal rounding options for encoding and decoding provides lossless storage of common decimal representations like 7.99, in more efficient 32-bit format (rather than 64-bit). The `useFloat32` property has several possible options, available from the module as constants:

package/dist/index.js

@@ -16,6 +16,7 @@
 	var srcString;
 	var srcStringStart = 0;
 	var srcStringEnd = 0;
+	var bundledStrings;
 	var referenceMap;
 	var currentExtensions = [];
 	var dataView;
@@ -54,6 +55,7 @@
 			position = 0;
 			srcStringEnd = 0;
 			srcString = null;
+			bundledStrings = null;
 			src = source;
 			// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
 			// technique for getting data from a database where it can be copied into an existing buffer instead of creating
@@ -150,6 +152,9 @@
 					currentStructures.length = sharedLength;
 			}
 			let result = read();
+			if (bundledStrings) // bundled strings to skip past
+				position = bundledStrings.postBundlePosition;
+
 			if (position == srcEnd) {
 				// finished reading this source, cleanup references
 				if (currentStructures.restoreStructures)
@@ -244,7 +249,15 @@
 			let value;
 			switch (token) {
 				case 0xc0: return null
-				case 0xc1: return C1; // "never-used", return special object to denote that
+				case 0xc1:
+					if (bundledStrings) {
+						value = read(); // followed by the length of the string in characters (not bytes!)
+						if (value > 0)
+							return bundledStrings[1].slice(bundledStrings.position1, bundledStrings.position1 += value)
+						else
+							return bundledStrings[0].slice(bundledStrings.position0, bundledStrings.position0 -= value)
+					}
+					return C1; // "never-used", return special object to denote that
 				case 0xc2: return false
 				case 0xc3: return true
 				case 0xc4:
@@ -705,6 +718,36 @@
 		}
 	}
 
+	function readOnlyJSString() {
+		let token = src[position++];
+		let length;
+		if (token < 0xc0) {
+			// fixstr
+			length = token - 0xa0;
+		} else {
+			switch(token) {
+				case 0xd9:
+				// str 8
+					length = src[position++];
+					break
+				case 0xda:
+				// str 16
+					length = dataView.getUint16(position);
+					position += 2;
+					break
+				case 0xdb:
+				// str 32
+					length = dataView.getUint32(position);
+					position += 4;
+					break
+				default:
+					throw new Error('Expected string')
+			}
+		}
+		return readStringJS(length)
+	}
+
+
 	function readBin(length) {
 		return currentUnpackr.copyBuffers ?
 			// specifically use the copying slice (not the node one)
@@ -856,6 +899,19 @@
 		let data = read();
 		return new RegExp(data[0], data[1])
 	};
+	const TEMP_BUNDLE = [];
+	currentExtensions[0x62] = (data) => {
+		let dataSize = (data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3];
+		let dataPosition = position;
+		position += dataSize - data.length;
+		bundledStrings = TEMP_BUNDLE;
+		bundledStrings = [readOnlyJSString(), readOnlyJSString()];
+		bundledStrings.position0 = 0;
+		bundledStrings.position1 = 0;
+		bundledStrings.postBundlePosition = position;
+		position = dataPosition;
+		return read()
+	};
 
 	currentExtensions[0xff] = (data) => {
 		// 32-bit date extension
@@ -870,7 +926,7 @@
 				((data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3]) / 1000000 +
 				(((data[4] & 0x80) ? -0x1000000000000 : 0) + data[6] * 0x10000000000 + data[7] * 0x100000000 + data[8] * 0x1000000 + (data[9] << 16) + (data[10] << 8) + data[11]) * 1000)
 		else
-			throw new Error('Invalid timestamp length')
+			return new Date('invalid')
 	}; // notepack defines extension 0 to mean undefined, so use that as the default here
 	// registration of bulk record definition?
 	// currentExtensions[0x52] = () =>
@@ -882,6 +938,7 @@
 		let savedSrcStringEnd = srcStringEnd;
 		let savedSrcString = srcString;
 		let savedReferenceMap = referenceMap;
+		let savedBundledStrings = bundledStrings;
 
 		// TODO: We may need to revisit this if we do more external calls to user code (since it could be slow)
 		let savedSrc = new Uint8Array(src.slice(0, srcEnd)); // we copy the data in case it changes while external data is processed
@@ -896,6 +953,7 @@
 		srcStringEnd = savedSrcStringEnd;
 		srcString = savedSrcString;
 		referenceMap = savedReferenceMap;
+		bundledStrings = savedBundledStrings;
 		src = savedSrc;
 		sequentialMode = savedSequentialMode;
 		currentStructures = savedStructures;
@@ -949,17 +1007,19 @@
 	const ByteArrayAllocate = hasNodeBuffer ? Buffer.allocUnsafeSlow : Uint8Array;
 	const ByteArray = hasNodeBuffer ? Buffer : Uint8Array;
 	const MAX_BUFFER_SIZE = hasNodeBuffer ? 0x100000000 : 0x7fd00000;
-	let target;
+	let target, keysTarget;
 	let targetView;
 	let position$1 = 0;
 	let safeEnd;
+	let bundledStrings$1 = null;
+	const MAX_BUNDLE_SIZE = 0xf000;
+	const hasNonLatin = /[\u0080-\uFFFF]/;
 	const RECORD_SYMBOL = Symbol('record-id');
 	class Packr extends Unpackr {
 		constructor(options) {
 			super(options);
 			this.offset = 0;
 			let start;
-			let sharedStructures;
 			let hasSharedUpdate;
 			let structures;
 			let referenceMap;
@@ -984,7 +1044,7 @@
 			let maxOwnStructures = options.maxOwnStructures;
 			if (maxOwnStructures == null)
 				maxOwnStructures = hasSharedStructures ? 32 : 64;
-			if (isSequential && !options.saveStructures)
+			if (!this.structures && options.useRecords != false)
 				this.structures = [];
 			// two byte record ids for shared structures
 			let useTwoByteRecords = maxSharedStructures > 32 || (maxOwnStructures + maxSharedStructures > 64);		
@@ -1014,23 +1074,28 @@
 					position$1 = (position$1 + 7) & 0x7ffffff8; // Word align to make any future copying of this buffer faster
 				start = position$1;
 				referenceMap = packr.structuredClone ? new Map() : null;
-				sharedStructures = packr.structures;
-				if (sharedStructures) {
-					if (sharedStructures.uninitialized)
-						sharedStructures = packr._mergeStructures(packr.getStructures());
-					let sharedLength = sharedStructures.sharedLength || 0;
+				if (packr.bundleStrings && typeof value !== 'string') {
+					bundledStrings$1 = [];
+					bundledStrings$1.size = Infinity; // force a new bundle start on first string
+				} else
+					bundledStrings$1 = null;
+				structures = packr.structures;
+				if (structures) {
+					if (structures.uninitialized)
+						structures = packr._mergeStructures(packr.getStructures());
+					let sharedLength = structures.sharedLength || 0;
 					if (sharedLength > maxSharedStructures) {
-						//if (maxSharedStructures <= 32 && sharedStructures.sharedLength > 32) // TODO: could support this, but would need to update the limit ids
-						throw new Error('Shared structures is larger than maximum shared structures, try increasing maxSharedStructures to ' + sharedStructures.sharedLength)
+						//if (maxSharedStructures <= 32 && structures.sharedLength > 32) // TODO: could support this, but would need to update the limit ids
+						throw new Error('Shared structures is larger than maximum shared structures, try increasing maxSharedStructures to ' + structures.sharedLength)
 					}
-					if (!sharedStructures.transitions) {
+					if (!structures.transitions) {
 						// rebuild our structure transitions
-						sharedStructures.transitions = Object.create(null);
+						structures.transitions = Object.create(null);
 						for (let i = 0; i < sharedLength; i++) {
-							let keys = sharedStructures[i];
+							let keys = structures[i];
 							if (!keys)
 								continue
-							let nextTransition, transition = sharedStructures.transitions;
+							let nextTransition, transition = structures.transitions;
 							for (let j = 0, l = keys.length; j < l; j++) {
 								let key = keys[j];
 								nextTransition = transition[key];
@@ -1044,14 +1109,16 @@
 						lastSharedStructuresLength = sharedLength;
 					}
 					if (!isSequential) {
-						sharedStructures.nextId = sharedLength + 0x40;
+						structures.nextId = sharedLength + 0x40;
 					}
 				}
 				if (hasSharedUpdate)
 					hasSharedUpdate = false;
-				structures = sharedStructures || [];
 				try {
 					pack(value);
+					if (bundledStrings$1) {
+						writeBundles(start, pack);
+					}
 					packr.offset = position$1; // update the offset so next serialization doesn't write over our buffer, but can continue writing to same buffer sequentially
 					if (referenceMap && referenceMap.idsToInsert) {
 						position$1 += referenceMap.idsToInsert.length * 6;
@@ -1062,19 +1129,22 @@
 						referenceMap = null;
 						return serialized
 					}
-					if (encodeOptions === REUSE_BUFFER_MODE) {
+					if (encodeOptions & REUSE_BUFFER_MODE) {
 						target.start = start;
 						target.end = position$1;
 						return target
 					}
 					return target.subarray(start, position$1) // position can change if we call pack again in saveStructures, so we get the buffer now
 				} finally {
-					if (sharedStructures) {
+					if (structures) {
 						if (serializationsSinceTransitionRebuild < 10)
 							serializationsSinceTransitionRebuild++;
+						let sharedLength = structures.sharedLength || maxSharedStructures;
+						if (structures.length > sharedLength)
+							structures.length = sharedLength;
 						if (transitionsCount > 10000) {
 							// force a rebuild occasionally after a lot of transitions so it can get cleaned up
-							sharedStructures.transitions = null;
+							structures.transitions = null;
 							serializationsSinceTransitionRebuild = 0;
 							transitionsCount = 0;
 							if (recordIdsToRemove.length > 0)
@@ -1086,13 +1156,9 @@
 							recordIdsToRemove = [];
 						}
 						if (hasSharedUpdate && packr.saveStructures) {
-							let sharedLength = sharedStructures.sharedLength || maxSharedStructures;
-							if (sharedStructures.length > sharedLength) {
-								sharedStructures = sharedStructures.slice(0, sharedLength);
-							}
 							// we can't rely on start/end with REUSE_BUFFER_MODE since they will (probably) change when we save
 							let returnBuffer = target.subarray(start, position$1);
-							if (packr.saveStructures(sharedStructures, lastSharedStructuresLength) === false) {
+							if (packr.saveStructures(structures, lastSharedStructuresLength) === false) {
 								// get updated structures and try again if the update failed
 								packr._mergeStructures(packr.getStructures());
 								return packr.pack(value)
@@ -1101,6 +1167,8 @@
 							return returnBuffer
 						}
 					}
+					if (encodeOptions & RESET_BUFFER_MODE)
+						position$1 = start;
 				}
 			};
 			const pack = (value) => {
@@ -1111,6 +1179,36 @@
 				var length;
 				if (type === 'string') {
 					let strLength = value.length;
+					if (bundledStrings$1 && strLength >= 4 && strLength < 0x1000) {
+						if ((bundledStrings$1.size += strLength) > MAX_BUNDLE_SIZE) {
+							let extStart;
+							let maxBytes = (bundledStrings$1[0] ? bundledStrings$1[0].length * 3 + bundledStrings$1[1].length : 0) + 10;
+							if (position$1 + maxBytes > safeEnd)
+								target = makeRoom(position$1 + maxBytes);
+							if (bundledStrings$1.position) { // here we use the 0x62 extension to write the last bundle and reserve sapce for the reference pointer to the next/current bundle
+								target[position$1] = 0xc8; // ext 16
+								position$1 += 3; // reserve for the writing bundle size
+								target[position$1++] = 0x62; // 'b'
+								extStart = position$1 - start;
+								position$1 += 4; // reserve for writing bundle reference
+								writeBundles(start, pack); // write the last bundles
+								targetView.setUint16(extStart + start - 3, position$1 - start - extStart);
+							} else { // here we use the 0x62 extension just to reserve the space for the reference pointer to the bundle (will be updated once the bundle is written)
+								target[position$1++] = 0xd6; // fixext 4
+								target[position$1++] = 0x62; // 'b'
+								extStart = position$1 - start;
+								position$1 += 4; // reserve for writing bundle reference
+							}
+							bundledStrings$1 = ['', '']; // create new ones
+							bundledStrings$1.size = 0;
+							bundledStrings$1.position = extStart;
+						}
+						let twoByte = hasNonLatin.test(value);
+						bundledStrings$1[twoByte ? 0 : 1] += value;
+						target[position$1++] = 0xc1;
+						pack(twoByte ? -strLength : strLength);
+						return
+					}
 					let headerSize;
 					// first we estimate the header size, so we can write to the correct location
 					if (strLength < 0x20) {
@@ -1409,53 +1507,55 @@
 				target[objectOffset++ + start] = size >> 8;
 				target[objectOffset + start] = size & 0xff;
 			} :
-
-		/*	sharedStructures ?  // For highly stable structures, using for-in can a little bit faster
+			(options.progressiveRecords && !useTwoByteRecords) ?  // this is about 2% faster for highly stable structures, since it only requires one for-in loop (but much more expensive when new structure needs to be written)
 			(object, safePrototype) => {
-				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null))
-				let objectOffset = position++ - start
-				let wroteKeys
+				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null));
+				let objectOffset = position$1++ - start;
+				let wroteKeys;
 				for (let key in object) {
 					if (safePrototype || object.hasOwnProperty(key)) {
-						nextTransition = transition[key]
-						if (!nextTransition) {
-							nextTransition = transition[key] = Object.create(null)
-							nextTransition.__keys__ = (transition.__keys__ || []).concat([key])
-							/*let keys = Object.keys(object)
-							if 
-							let size = 0
-							let startBranch = transition.__keys__ ? transition.__keys__.length : 0
-							for (let i = 0, l = keys.length; i++) {
-								let key = keys[i]
-								size += key.length << 2
-								if (i >= startBranch) {
-									nextTransition = nextTransition[key] = Object.create(null)
-									nextTransition.__keys__ = keys.slice(0, i + 1)
+						nextTransition = transition[key];
+						if (nextTransition)
+							transition = nextTransition;
+						else {
+							// record doesn't exist, create full new record and insert it
+							let keys = Object.keys(object);
+							let lastTransition = transition;
+							transition = structures.transitions;
+							let newTransitions = 0;
+							for (let i = 0, l = keys.length; i < l; i++) {
+								let key = keys[i];
+								nextTransition = transition[key];
+								if (!nextTransition) {
+									nextTransition = transition[key] = Object.create(null);
+									newTransitions++;
 								}
+								transition = nextTransition;
 							}
-							makeRoom(position + size)
-							nextTransition = transition[key]
-							target.copy(target, )
-							objectOffset
+							if (objectOffset + start + 1 == position$1) {
+								// first key, so we don't need to insert, we can just write record directly
+								position$1--;
+								newRecord(transition, keys, newTransitions);
+							} else // otherwise we need to insert the record, moving existing data after the record
+								insertNewRecord(transition, keys, objectOffset, newTransitions);
+							wroteKeys = true;
+							transition = lastTransition[key];
 						}
-						transition = nextTransition
-						pack(object[key])
+						pack(object[key]);
 					}
 				}
-				let id = transition.id
-				if (!id) {
-					id = transition.id = structures.push(transition.__keys__) + 63
-					if (sharedStructures.onUpdate)
-						sharedStructures.onUpdate(id, transition.__keys__)
+				if (!wroteKeys) {
+					let recordId = transition[RECORD_SYMBOL];
+					if (recordId)
+						target[objectOffset + start] = recordId;
+					else
+						insertNewRecord(transition, Object.keys(object), objectOffset, 0);
 				}
-				target[objectOffset + start] = id
-			}*/
-			(object) => {
-				let keys = Object.keys(object);
+			} :
+			(object, safePrototype) => {
 				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null));
 				let newTransitions = 0;
-				for (let i = 0, l = keys.length; i < l; i++) {
-					let key = keys[i];
+				for (let key in object) if (safePrototype || object.hasOwnProperty(key)) {
 					nextTransition = transition[key];
 					if (!nextTransition) {
 						nextTransition = transition[key] = Object.create(null);
@@ -1471,57 +1571,12 @@
 					} else
 						target[position$1++] = recordId;
 				} else {
-					recordId = structures.nextId;
-					if (!recordId)
-						recordId = 0x40;
-					if (recordId < sharedLimitId && this.shouldShareStructure && !this.shouldShareStructure(keys)) {
-						recordId = structures.nextOwnId;
-						if (!(recordId < maxStructureId))
-							recordId = sharedLimitId;
-						structures.nextOwnId = recordId + 1;
-					} else {
-						if (recordId >= maxStructureId)// cycle back around
-							recordId = sharedLimitId;
-						structures.nextId = recordId + 1;
-					}
-					let highByte = keys.highByte = recordId >= 0x60 && useTwoByteRecords ? (recordId - 0x60) >> 5 : -1;
-					transition[RECORD_SYMBOL] = recordId;
-					structures[recordId - 0x40] = keys;
-
-					if (recordId < sharedLimitId) {
-						keys.isShared = true;
-						structures.sharedLength = recordId - 0x3f;
-						hasSharedUpdate = true;
-						if (highByte >= 0) {
-							target[position$1++] = (recordId & 0x1f) + 0x60;
-							target[position$1++] = highByte;
-						} else {
-							target[position$1++] = recordId;
-						}
-					} else {
-						if (highByte >= 0) {
-							target[position$1++] = 0xd5; // fixext 2
-							target[position$1++] = 0x72; // "r" record defintion extension type
-							target[position$1++] = (recordId & 0x1f) + 0x60;
-							target[position$1++] = highByte;
-						} else {
-							target[position$1++] = 0xd4; // fixext 1
-							target[position$1++] = 0x72; // "r" record defintion extension type
-							target[position$1++] = recordId;
-						}
-
-						if (newTransitions)
-							transitionsCount += serializationsSinceTransitionRebuild * newTransitions;
-						// record the removal of the id, we can maintain our shared structure
-						if (recordIdsToRemove.length >= maxOwnStructures)
-							recordIdsToRemove.shift()[RECORD_SYMBOL] = 0; // we are cycling back through, and have to remove old ones
-						recordIdsToRemove.push(transition);
-						pack(keys);
-					}
+					newRecord(transition, transition.__keys__ || Object.keys(object), newTransitions);
 				}
 				// now write the values
-				for (let i = 0, l = keys.length; i < l; i++)
-					pack(object[keys[i]]);
+				for (let key in object)
+					if (safePrototype || object.hasOwnProperty(key))
+						pack(object[key]);
 			};
 			const makeRoom = (end) => {
 				let newSize;
@@ -1544,6 +1599,86 @@
 				safeEnd = newBuffer.length - 10;
 				return target = newBuffer
 			};
+			const newRecord = (transition, keys, newTransitions) => {
+				let recordId = structures.nextId;
+				if (!recordId)
+					recordId = 0x40;
+				if (recordId < sharedLimitId && this.shouldShareStructure && !this.shouldShareStructure(keys)) {
+					recordId = structures.nextOwnId;
+					if (!(recordId < maxStructureId))
+						recordId = sharedLimitId;
+					structures.nextOwnId = recordId + 1;
+				} else {
+					if (recordId >= maxStructureId)// cycle back around
+						recordId = sharedLimitId;
+					structures.nextId = recordId + 1;
+				}
+				let highByte = keys.highByte = recordId >= 0x60 && useTwoByteRecords ? (recordId - 0x60) >> 5 : -1;
+				transition[RECORD_SYMBOL] = recordId;
+				transition.__keys__ = keys;
+				structures[recordId - 0x40] = keys;
+
+				if (recordId < sharedLimitId) {
+					keys.isShared = true;
+					structures.sharedLength = recordId - 0x3f;
+					hasSharedUpdate = true;
+					if (highByte >= 0) {
+						target[position$1++] = (recordId & 0x1f) + 0x60;
+						target[position$1++] = highByte;
+					} else {
+						target[position$1++] = recordId;
+					}
+				} else {
+					if (highByte >= 0) {
+						target[position$1++] = 0xd5; // fixext 2
+						target[position$1++] = 0x72; // "r" record defintion extension type
+						target[position$1++] = (recordId & 0x1f) + 0x60;
+						target[position$1++] = highByte;
+					} else {
+						target[position$1++] = 0xd4; // fixext 1
+						target[position$1++] = 0x72; // "r" record defintion extension type
+						target[position$1++] = recordId;
+					}
+
+					if (newTransitions)
+						transitionsCount += serializationsSinceTransitionRebuild * newTransitions;
+					// record the removal of the id, we can maintain our shared structure
+					if (recordIdsToRemove.length >= maxOwnStructures)
+						recordIdsToRemove.shift()[RECORD_SYMBOL] = 0; // we are cycling back through, and have to remove old ones
+					recordIdsToRemove.push(transition);
+					pack(keys);
+				}
+			};
+			const insertNewRecord = (transition, keys, insertionOffset, newTransitions) => {
+				let mainTarget = target;
+				let mainPosition = position$1;
+				let mainSafeEnd = safeEnd;
+				let mainStart = start;
+				target = keysTarget;
+				position$1 = 0;
+				start = 0;
+				if (!target)
+					keysTarget = target = new ByteArrayAllocate(8192);
+				safeEnd = target.length - 10;
+				newRecord(transition, keys, newTransitions);
+				keysTarget = target;
+				let keysPosition = position$1;
+				target = mainTarget;
+				position$1 = mainPosition;
+				safeEnd = mainSafeEnd;
+				start = mainStart;
+				if (keysPosition > 1) {
+					let newEnd = position$1 + keysPosition - 1;
+					if (newEnd > safeEnd)
+						makeRoom(newEnd);
+					let insertionPosition = insertionOffset + start;
+					target.copyWithin(insertionPosition + keysPosition, insertionPosition + 1, position$1);
+					target.set(keysTarget.slice(0, keysPosition), insertionPosition);
+					position$1 = newEnd;
+				} else {
+					target[insertionOffset + start] = keysTarget[0];
+				}
+			};
 		}
 		useBuffer(buffer) {
 			// this means we are finished using our own buffer and we can write over it safely
@@ -1551,11 +1686,15 @@
 			targetView = new DataView(target.buffer, target.byteOffset, target.byteLength);
 			position$1 = 0;
 		}
+		clearSharedData() {
+			if (this.structures)
+				this.structures = [];
+		}
 	}
 
 	extensionClasses = [ Date, Set, Error, RegExp, ArrayBuffer, Object.getPrototypeOf(Uint8Array.prototype).constructor /*TypedArray*/, C1Type ];
 	extensions = [{
-		pack(date, allocateForWrite) {
+		pack(date, allocateForWrite, pack) {
 			let seconds = date.getTime() / 1000;
 			if ((this.useTimestamp32 || date.getMilliseconds() === 0) && seconds >= 0 && seconds < 0x100000000) {
 				// Timestamp 32
@@ -1570,6 +1709,16 @@
 				target[position++] = 0xff;
 				targetView.setUint32(position, date.getMilliseconds() * 4000000 + ((seconds / 1000 / 0x100000000) >> 0));
 				targetView.setUint32(position + 4, seconds);
+			} else if (isNaN(seconds)) {
+				if (this.onInvalidDate) {
+					allocateForWrite(0);
+					return pack(this.onInvalidDate())
+				}
+				// Intentionally invalid timestamp
+				let { target, targetView, position} = allocateForWrite(3);
+				target[position++] = 0xd4;
+				target[position++] = 0xff;
+				target[position++] = 0xff;
 			} else {
 				// Timestamp 96
 				let { target, targetView, position} = allocateForWrite(15);
@@ -1738,6 +1887,14 @@
 		return serialized
 	}
 
+	function writeBundles(start, pack) {
+		targetView.setUint32(bundledStrings$1.position + start, position$1 - bundledStrings$1.position - start);
+		let writeStrings = bundledStrings$1;
+		bundledStrings$1 = null;
+		pack(writeStrings[0]);
+		pack(writeStrings[1]);
+	}
+
 	function addExtension$1(extension) {
 		if (extension.Class) {
 			if (!extension.pack && !extension.write)
@@ -1755,7 +1912,8 @@
 	const encode = defaultPackr.pack;
 	const Encoder = Packr;
 	const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
-	const REUSE_BUFFER_MODE = 1000;
+	const REUSE_BUFFER_MODE = 512;
+	const RESET_BUFFER_MODE = 1024;
 
 	/**
 	 * Given an Iterable first argument, returns an Iterable where each value is packed as a Buffer