msgpackr 1.4.7 → 1.5.3

package/README.md

@@ -1,11 +1,13 @@
 # msgpackr
-[![license](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
 [![npm version](https://img.shields.io/npm/v/msgpackr.svg?style=flat-square)](https://www.npmjs.org/package/msgpackr)
+[![npm version](https://img.shields.io/npm/dw/msgpackr)](https://www.npmjs.org/package/msgpackr)
 [![encode](https://img.shields.io/badge/encode-1.5GB%2Fs-yellow)](benchmark.md)
 [![decode](https://img.shields.io/badge/decode-2GB%2Fs-yellow)](benchmark.md)
 [![types](https://img.shields.io/npm/types/msgpackr)](README.md)
 [![module](https://img.shields.io/badge/module-ESM%2FCJS-blue)](README.md)
+[![license](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
 
+<img align="right" src="./assets/performance.png" width="380"/>
 
 The msgpackr package is an extremely fast MessagePack NodeJS/JavaScript implementation. Currently, it is significantly faster than any other known implementations, faster than Avro (for JS), and generally faster than native V8 JSON.stringify/parse, on NodeJS. It also includes an optional record extension (the `r` in msgpackr), for defining record structures that makes MessagePack even faster and more compact, often over twice as fast as even native JSON functions, several times faster than other JS implementations, and 15-50% more compact. See the performance section for more details. Structured cloning (with support for cyclical references) is also supported through optional extensions.
 
@@ -52,7 +54,7 @@ receivingStream.on('data', (data) => {
 The `PackrStream` and `UnpackrStream` instances  will have also the record structure extension enabled by default (see below).
 
 ## Deno Usage
-Msgpackr modules are standard ESM modules and can be loaded directly from github (https://raw.githubusercontent.com/kriszyp/msgpackr/master/index.js) or downloaded and used directly in Deno. The standard pack/encode and unpack/decode functionality is available on Deno, like other platforms.
+Msgpackr modules are standard ESM modules and can be loaded directly from the [deno.land registry for msgpackr](https://deno.land/x/msgpackr) for use in Deno. The standard pack/encode and unpack/decode functionality is available on Deno, like other platforms.
 
 ## Browser Usage
 Msgpackr works as standalone JavaScript as well, and runs on modern browsers. It includes a bundled script, at `dist/index.js` for ease of direct loading:
@@ -161,11 +163,14 @@ The following options properties can be provided to the Packr or Unpackr constru
 * `mapsAsObjects` - If `true`, this will decode MessagePack maps and JS `Object`s with the map entries decoded to object properties. If `false`, maps are decoded as JavaScript `Map`s. This is disabled by default if `useRecords` is enabled (which allows `Map`s to be preserved), and is enabled by default if `useRecords` is disabled.
 * `useFloat32` - This will enable msgpackr to encode non-integer numbers as `float32`. See next section for possible values.
 * `variableMapSize` - This will use varying map size definition (fixmap, map16, map32) based on the number of keys when encoding objects, which yields slightly more compact encodings (for small objects), but is typically 5-10% slower during encoding. This is necessary if you need to use objects with more than 65535 keys. This is only relevant when record extension is disabled.
+* `bundleStrings` - If `true` this uses a custom extension that bundles strings together, so that they can be decoded more quickly on browsers and Deno that do not have access to the NodeJS addon. This a custom extension, so both encoder and decoder need to support this. This can yield significant decoding performance increases on browsers (30%-50%).
 * `copyBuffers` - When decoding a MessagePack with binary data (Buffers are encoded as binary data), copy the buffer rather than providing a slice/view of the buffer. If you want your input data to be collected or modified while the decoded embedded buffer continues to live on, you can use this option (there is extra overhead to copying).
 * `useTimestamp32` - Encode JS `Date`s in 32-bit format when possible by dropping the milliseconds. This is a more efficient encoding of dates. You can also cause dates to use 32-bit format by manually setting the milliseconds to zero (`date.setMilliseconds(0)`).
 * `sequential` - Encode structures in serialized data, and reference previously encoded structures with expectation that decoder will read the encoded structures in the same order as encoded, with `unpackMultiple`.
 * `largeBigIntToFloat` - If a bigint needs to be encoded that is larger than will fit in 64-bit integers, it will be encoded as a float-64 (otherwise will throw a RangeError).
 * `encodeUndefinedAsNil` - Encodes a value of `undefined` as a MessagePack `nil`, the same as a `null`.
+* `int64AsNumber` - This will decode uint64 and int64 numbers as standard JS numbers rather than as bigint numbers.
+* `onInvalidDate` - This can be provided as function that will be called when an invalid date is provided. The function can throw an error, or return a value that will be encoded in place of the invalid date. If not provided, an invalid date will be encoded as an invalid timestamp (which decodes with msgpackr back to an invalid date).
 
 ### 32-bit Float Options
 By default all non-integer numbers are serialized as 64-bit float (double). This is fast, and ensures maximum precision. However, often real-world data doesn't not need 64-bits of precision, and using 32-bit encoding can be much more space efficient. There are several options that provide more efficient encodings. Using the decimal rounding options for encoding and decoding provides lossless storage of common decimal representations like 7.99, in more efficient 32-bit format (rather than 64-bit). The `useFloat32` property has several possible options, available from the module as constants:
@@ -183,6 +188,15 @@ Note, that the performance is decreased with decimal rounding by about 20-25%, a
 In addition, msgpackr exports a `roundFloat32(number)` function that can be used to round floating point numbers to the maximum significant decimal digits that can be stored in 32-bit float, just as DECIMAL_ROUND does when decoding. This can be useful for determining how a number will be decoded prior to encoding it.
 
 ## Performance
+### Native Acceleration
+Msgpackr employs an optional native node-addon to accelerate the parsing of strings. This should be automatically installed and utilized on NodeJS. However, you can verify this by checking the `isNativeAccelerationEnabled` property that is exported from msgpackr. If this is `false`, the `msgpackr-extract` package may not have been properly installed, and you may want to verify that it is installed correctly:
+```js
+import { isNativeAccelerationEnabled } from 'msgpackr'
+if (!isNativeAccelerationEnabled)
+	console.warn('Native acceleration not enabled, verify that install finished properly')
+```
+
+### Benchmarks
 Msgpackr is fast. Really fast. Here is comparison with the next fastest JS projects using the benchmark tool from `msgpack-lite` (and the sample data is from some clinical research data we use that has a good mix of different value types and structures). It also includes comparison to V8 native JSON functionality, and JavaScript Avro (`avsc`, a very optimized Avro implementation):
 
 operation                                                  |   op   |   ms  |  op/s

package/dist/index.js

@@ -16,6 +16,7 @@
 	var srcString;
 	var srcStringStart = 0;
 	var srcStringEnd = 0;
+	var bundledStrings;
 	var referenceMap;
 	var currentExtensions = [];
 	var dataView;
@@ -54,12 +55,21 @@
 			position = 0;
 			srcStringEnd = 0;
 			srcString = null;
+			bundledStrings = null;
 			src = source;
 			// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
 			// technique for getting data from a database where it can be copied into an existing buffer instead of creating
 			// new ones
-			dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
-			if (this) {
+			try {
+				dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
+			} catch(error) {
+				// if it doesn't have a buffer, maybe it is the wrong type of object
+				src = null;
+				if (source instanceof Uint8Array)
+					throw error
+				throw new Error('Source must be a Uint8Array or Buffer but was a ' + ((source && typeof source == 'object') ? source.constructor.name : typeof source))
+			}
+			if (this instanceof Unpackr) {
 				currentUnpackr = this;
 				if (this.structures) {
 					currentStructures = this.structures;
@@ -142,6 +152,9 @@
 					currentStructures.length = sharedLength;
 			}
 			let result = read();
+			if (bundledStrings) // bundled strings to skip past
+				position = bundledStrings.postBundlePosition;
+
 			if (position == srcEnd) {
 				// finished reading this source, cleanup references
 				if (currentStructures.restoreStructures)
@@ -236,7 +249,15 @@
 			let value;
 			switch (token) {
 				case 0xc0: return null
-				case 0xc1: return C1; // "never-used", return special object to denote that
+				case 0xc1:
+					if (bundledStrings) {
+						value = read(); // followed by the length of the string in characters (not bytes!)
+						if (value > 0)
+							return bundledStrings[1].slice(bundledStrings.position1, bundledStrings.position1 += value)
+						else
+							return bundledStrings[0].slice(bundledStrings.position0, bundledStrings.position0 -= value)
+					}
+					return C1; // "never-used", return special object to denote that
 				case 0xc2: return false
 				case 0xc3: return true
 				case 0xc4:
@@ -291,10 +312,11 @@
 					position += 4;
 					return value
 				case 0xcf:
-					if (currentUnpackr.uint64AsNumber)
-						return src[position++] * 0x100000000000000 + src[position++] * 0x1000000000000 + src[position++] * 0x10000000000 + src[position++] * 0x100000000 +
-							src[position++] * 0x1000000 + (src[position++] << 16) + (src[position++] << 8) + src[position++]
-					value = dataView.getBigUint64(position);
+					if (currentUnpackr.int64AsNumber) {
+						value = dataView.getUint32(position) * 0x100000000;
+						value += dataView.getUint32(position + 4);
+					} else
+						value = dataView.getBigUint64(position);
 					position += 8;
 					return value
 
@@ -310,7 +332,11 @@
 					position += 4;
 					return value
 				case 0xd3:
-					value = dataView.getBigInt64(position);
+					if (currentUnpackr.int64AsNumber) {
+						value = dataView.getInt32(position) * 0x100000000;
+						value += dataView.getUint32(position + 4);
+					} else
+						value = dataView.getBigInt64(position);
 					position += 8;
 					return value
 
@@ -459,6 +485,7 @@
 	var readString8 = readStringJS;
 	var readString16 = readStringJS;
 	var readString32 = readStringJS;
+	let isNativeAccelerationEnabled = false;
 	function readStringJS(length) {
 		let result;
 		if (length < 16) {
@@ -691,6 +718,36 @@
 		}
 	}
 
+	function readOnlyJSString() {
+		let token = src[position++];
+		let length;
+		if (token < 0xc0) {
+			// fixstr
+			length = token - 0xa0;
+		} else {
+			switch(token) {
+				case 0xd9:
+				// str 8
+					length = src[position++];
+					break
+				case 0xda:
+				// str 16
+					length = dataView.getUint16(position);
+					position += 2;
+					break
+				case 0xdb:
+				// str 32
+					length = dataView.getUint32(position);
+					position += 4;
+					break
+				default:
+					throw new Error('Expected string')
+			}
+		}
+		return readStringJS(length)
+	}
+
+
 	function readBin(length) {
 		return currentUnpackr.copyBuffers ?
 			// specifically use the copying slice (not the node one)
@@ -842,6 +899,19 @@
 		let data = read();
 		return new RegExp(data[0], data[1])
 	};
+	const TEMP_BUNDLE = [];
+	currentExtensions[0x62] = (data) => {
+		let dataSize = (data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3];
+		let dataPosition = position;
+		position += dataSize - data.length;
+		bundledStrings = TEMP_BUNDLE;
+		bundledStrings = [readOnlyJSString(), readOnlyJSString()];
+		bundledStrings.position0 = 0;
+		bundledStrings.position1 = 0;
+		bundledStrings.postBundlePosition = position;
+		position = dataPosition;
+		return read()
+	};
 
 	currentExtensions[0xff] = (data) => {
 		// 32-bit date extension
@@ -856,7 +926,7 @@
 				((data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3]) / 1000000 +
 				(((data[4] & 0x80) ? -0x1000000000000 : 0) + data[6] * 0x10000000000 + data[7] * 0x100000000 + data[8] * 0x1000000 + (data[9] << 16) + (data[10] << 8) + data[11]) * 1000)
 		else
-			throw new Error('Invalid timestamp length')
+			return new Date('invalid')
 	}; // notepack defines extension 0 to mean undefined, so use that as the default here
 	// registration of bulk record definition?
 	// currentExtensions[0x52] = () =>
@@ -868,6 +938,7 @@
 		let savedSrcStringEnd = srcStringEnd;
 		let savedSrcString = srcString;
 		let savedReferenceMap = referenceMap;
+		let savedBundledStrings = bundledStrings;
 
 		// TODO: We may need to revisit this if we do more external calls to user code (since it could be slow)
 		let savedSrc = new Uint8Array(src.slice(0, srcEnd)); // we copy the data in case it changes while external data is processed
@@ -882,6 +953,7 @@
 		srcStringEnd = savedSrcStringEnd;
 		srcString = savedSrcString;
 		referenceMap = savedReferenceMap;
+		bundledStrings = savedBundledStrings;
 		src = savedSrc;
 		sequentialMode = savedSequentialMode;
 		currentStructures = savedStructures;
@@ -935,10 +1007,13 @@
 	const ByteArrayAllocate = hasNodeBuffer ? Buffer.allocUnsafeSlow : Uint8Array;
 	const ByteArray = hasNodeBuffer ? Buffer : Uint8Array;
 	const MAX_BUFFER_SIZE = hasNodeBuffer ? 0x100000000 : 0x7fd00000;
-	let target;
+	let target, keysTarget;
 	let targetView;
 	let position$1 = 0;
 	let safeEnd;
+	let bundledStrings$1 = null;
+	const MAX_BUNDLE_SIZE = 0xf000;
+	const hasNonLatin = /[\u0080-\uFFFF]/;
 	const RECORD_SYMBOL = Symbol('record-id');
 	class Packr extends Unpackr {
 		constructor(options) {
@@ -1000,6 +1075,11 @@
 					position$1 = (position$1 + 7) & 0x7ffffff8; // Word align to make any future copying of this buffer faster
 				start = position$1;
 				referenceMap = packr.structuredClone ? new Map() : null;
+				if (packr.bundleStrings && typeof value !== 'string') {
+					bundledStrings$1 = [];
+					bundledStrings$1.size = Infinity; // force a new bundle start on first string
+				} else
+					bundledStrings$1 = null;
 				sharedStructures = packr.structures;
 				if (sharedStructures) {
 					if (sharedStructures.uninitialized)
@@ -1035,9 +1115,12 @@
 				}
 				if (hasSharedUpdate)
 					hasSharedUpdate = false;
-				structures = sharedStructures || [];
+				structures = sharedStructures || (packr.structures = []);
 				try {
 					pack(value);
+					if (bundledStrings$1) {
+						writeBundles(start, pack);
+					}
 					packr.offset = position$1; // update the offset so next serialization doesn't write over our buffer, but can continue writing to same buffer sequentially
 					if (referenceMap && referenceMap.idsToInsert) {
 						position$1 += referenceMap.idsToInsert.length * 6;
@@ -1048,7 +1131,7 @@
 						referenceMap = null;
 						return serialized
 					}
-					if (encodeOptions === REUSE_BUFFER_MODE) {
+					if (encodeOptions & REUSE_BUFFER_MODE) {
 						target.start = start;
 						target.end = position$1;
 						return target
@@ -1058,6 +1141,8 @@
 					if (sharedStructures) {
 						if (serializationsSinceTransitionRebuild < 10)
 							serializationsSinceTransitionRebuild++;
+						if (sharedStructures.length > maxSharedStructures)
+							sharedStructures.length = maxSharedStructures;
 						if (transitionsCount > 10000) {
 							// force a rebuild occasionally after a lot of transitions so it can get cleaned up
 							sharedStructures.transitions = null;
@@ -1087,6 +1172,8 @@
 							return returnBuffer
 						}
 					}
+					if (encodeOptions & RESET_BUFFER_MODE)
+						position$1 = start;
 				}
 			};
 			const pack = (value) => {
@@ -1097,6 +1184,36 @@
 				var length;
 				if (type === 'string') {
 					let strLength = value.length;
+					if (bundledStrings$1 && strLength >= 4 && strLength < 0x1000) {
+						if ((bundledStrings$1.size += strLength) > MAX_BUNDLE_SIZE) {
+							let extStart;
+							let maxBytes = (bundledStrings$1[0] ? bundledStrings$1[0].length * 3 + bundledStrings$1[1].length : 0) + 10;
+							if (position$1 + maxBytes > safeEnd)
+								target = makeRoom(position$1 + maxBytes);
+							if (bundledStrings$1.position) { // here we use the 0x62 extension to write the last bundle and reserve sapce for the reference pointer to the next/current bundle
+								target[position$1] = 0xc8; // ext 16
+								position$1 += 3; // reserve for the writing bundle size
+								target[position$1++] = 0x62; // 'b'
+								extStart = position$1 - start;
+								position$1 += 4; // reserve for writing bundle reference
+								writeBundles(start, pack); // write the last bundles
+								targetView.setUint16(extStart + start - 3, position$1 - start - extStart);
+							} else { // here we use the 0x62 extension just to reserve the space for the reference pointer to the bundle (will be updated once the bundle is written)
+								target[position$1++] = 0xd6; // fixext 4
+								target[position$1++] = 0x62; // 'b'
+								extStart = position$1 - start;
+								position$1 += 4; // reserve for writing bundle reference
+							}
+							bundledStrings$1 = ['', '']; // create new ones
+							bundledStrings$1.size = 0;
+							bundledStrings$1.position = extStart;
+						}
+						let twoByte = hasNonLatin.test(value);
+						bundledStrings$1[twoByte ? 0 : 1] += value;
+						target[position$1++] = 0xc1;
+						pack(twoByte ? -strLength : strLength);
+						return
+					}
 					let headerSize;
 					// first we estimate the header size, so we can write to the correct location
 					if (strLength < 0x20) {
@@ -1395,53 +1512,55 @@
 				target[objectOffset++ + start] = size >> 8;
 				target[objectOffset + start] = size & 0xff;
 			} :
-
-		/*	sharedStructures ?  // For highly stable structures, using for-in can a little bit faster
+			(options.progressiveRecords && !useTwoByteRecords) ?  // this is about 2% faster for highly stable structures, since it only requires one for-in loop (but much more expensive when new structure needs to be written)
 			(object, safePrototype) => {
-				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null))
-				let objectOffset = position++ - start
-				let wroteKeys
+				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null));
+				let objectOffset = position$1++ - start;
+				let wroteKeys;
 				for (let key in object) {
 					if (safePrototype || object.hasOwnProperty(key)) {
-						nextTransition = transition[key]
-						if (!nextTransition) {
-							nextTransition = transition[key] = Object.create(null)
-							nextTransition.__keys__ = (transition.__keys__ || []).concat([key])
-							/*let keys = Object.keys(object)
-							if 
-							let size = 0
-							let startBranch = transition.__keys__ ? transition.__keys__.length : 0
-							for (let i = 0, l = keys.length; i++) {
-								let key = keys[i]
-								size += key.length << 2
-								if (i >= startBranch) {
-									nextTransition = nextTransition[key] = Object.create(null)
-									nextTransition.__keys__ = keys.slice(0, i + 1)
+						nextTransition = transition[key];
+						if (nextTransition)
+							transition = nextTransition;
+						else {
+							// record doesn't exist, create full new record and insert it
+							let keys = Object.keys(object);
+							let lastTransition = transition;
+							transition = structures.transitions;
+							let newTransitions = 0;
+							for (let i = 0, l = keys.length; i < l; i++) {
+								let key = keys[i];
+								nextTransition = transition[key];
+								if (!nextTransition) {
+									nextTransition = transition[key] = Object.create(null);
+									newTransitions++;
 								}
+								transition = nextTransition;
 							}
-							makeRoom(position + size)
-							nextTransition = transition[key]
-							target.copy(target, )
-							objectOffset
+							if (objectOffset + start + 1 == position$1) {
+								// first key, so we don't need to insert, we can just write record directly
+								position$1--;
+								newRecord(transition, keys, newTransitions);
+							} else // otherwise we need to insert the record, moving existing data after the record
+								insertNewRecord(transition, keys, objectOffset, newTransitions);
+							wroteKeys = true;
+							transition = lastTransition[key];
 						}
-						transition = nextTransition
-						pack(object[key])
+						pack(object[key]);
 					}
 				}
-				let id = transition.id
-				if (!id) {
-					id = transition.id = structures.push(transition.__keys__) + 63
-					if (sharedStructures.onUpdate)
-						sharedStructures.onUpdate(id, transition.__keys__)
+				if (!wroteKeys) {
+					let recordId = transition[RECORD_SYMBOL];
+					if (recordId)
+						target[objectOffset + start] = recordId;
+					else
+						insertNewRecord(transition, Object.keys(object), objectOffset, 0);
 				}
-				target[objectOffset + start] = id
-			}*/
-			(object) => {
-				let keys = Object.keys(object);
+			} :
+			(object, safePrototype) => {
 				let nextTransition, transition = structures.transitions || (structures.transitions = Object.create(null));
 				let newTransitions = 0;
-				for (let i = 0, l = keys.length; i < l; i++) {
-					let key = keys[i];
+				for (let key in object) if (safePrototype || object.hasOwnProperty(key)) {
 					nextTransition = transition[key];
 					if (!nextTransition) {
 						nextTransition = transition[key] = Object.create(null);
@@ -1457,57 +1576,12 @@
 					} else
 						target[position$1++] = recordId;
 				} else {
-					recordId = structures.nextId;
-					if (!recordId)
-						recordId = 0x40;
-					if (recordId < sharedLimitId && this.shouldShareStructure && !this.shouldShareStructure(keys)) {
-						recordId = structures.nextOwnId;
-						if (!(recordId < maxStructureId))
-							recordId = sharedLimitId;
-						structures.nextOwnId = recordId + 1;
-					} else {
-						if (recordId >= maxStructureId)// cycle back around
-							recordId = sharedLimitId;
-						structures.nextId = recordId + 1;
-					}
-					let highByte = keys.highByte = recordId >= 0x60 && useTwoByteRecords ? (recordId - 0x60) >> 5 : -1;
-					transition[RECORD_SYMBOL] = recordId;
-					structures[recordId - 0x40] = keys;
-
-					if (recordId < sharedLimitId) {
-						keys.isShared = true;
-						structures.sharedLength = recordId - 0x3f;
-						hasSharedUpdate = true;
-						if (highByte >= 0) {
-							target[position$1++] = (recordId & 0x1f) + 0x60;
-							target[position$1++] = highByte;
-						} else {
-							target[position$1++] = recordId;
-						}
-					} else {
-						if (highByte >= 0) {
-							target[position$1++] = 0xd5; // fixext 2
-							target[position$1++] = 0x72; // "r" record defintion extension type
-							target[position$1++] = (recordId & 0x1f) + 0x60;
-							target[position$1++] = highByte;
-						} else {
-							target[position$1++] = 0xd4; // fixext 1
-							target[position$1++] = 0x72; // "r" record defintion extension type
-							target[position$1++] = recordId;
-						}
-
-						if (newTransitions)
-							transitionsCount += serializationsSinceTransitionRebuild * newTransitions;
-						// record the removal of the id, we can maintain our shared structure
-						if (recordIdsToRemove.length >= maxOwnStructures)
-							recordIdsToRemove.shift()[RECORD_SYMBOL] = 0; // we are cycling back through, and have to remove old ones
-						recordIdsToRemove.push(transition);
-						pack(keys);
-					}
+					newRecord(transition, transition.__keys__ || Object.keys(object), newTransitions);
 				}
 				// now write the values
-				for (let i = 0, l = keys.length; i < l; i++)
-					pack(object[keys[i]]);
+				for (let key in object)
+					if (safePrototype || object.hasOwnProperty(key))
+						pack(object[key]);
 			};
 			const makeRoom = (end) => {
 				let newSize;
@@ -1516,7 +1590,7 @@
 					if ((end - start) > MAX_BUFFER_SIZE)
 						throw new Error('Packed buffer would be larger than maximum buffer size')
 					newSize = Math.min(MAX_BUFFER_SIZE,
-						Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x1000000) / 0x1000) * 0x1000);
+						Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x400000) / 0x1000) * 0x1000);
 				} else // faster handling for smaller buffers
 					newSize = ((Math.max((end - start) << 2, target.length - 1) >> 12) + 1) << 12;
 				let newBuffer = new ByteArrayAllocate(newSize);
@@ -1530,6 +1604,86 @@
 				safeEnd = newBuffer.length - 10;
 				return target = newBuffer
 			};
+			const newRecord = (transition, keys, newTransitions) => {
+				let recordId = structures.nextId;
+				if (!recordId)
+					recordId = 0x40;
+				if (recordId < sharedLimitId && this.shouldShareStructure && !this.shouldShareStructure(keys)) {
+					recordId = structures.nextOwnId;
+					if (!(recordId < maxStructureId))
+						recordId = sharedLimitId;
+					structures.nextOwnId = recordId + 1;
+				} else {
+					if (recordId >= maxStructureId)// cycle back around
+						recordId = sharedLimitId;
+					structures.nextId = recordId + 1;
+				}
+				let highByte = keys.highByte = recordId >= 0x60 && useTwoByteRecords ? (recordId - 0x60) >> 5 : -1;
+				transition[RECORD_SYMBOL] = recordId;
+				transition.__keys__ = keys;
+				structures[recordId - 0x40] = keys;
+
+				if (recordId < sharedLimitId) {
+					keys.isShared = true;
+					structures.sharedLength = recordId - 0x3f;
+					hasSharedUpdate = true;
+					if (highByte >= 0) {
+						target[position$1++] = (recordId & 0x1f) + 0x60;
+						target[position$1++] = highByte;
+					} else {
+						target[position$1++] = recordId;
+					}
+				} else {
+					if (highByte >= 0) {
+						target[position$1++] = 0xd5; // fixext 2
+						target[position$1++] = 0x72; // "r" record defintion extension type
+						target[position$1++] = (recordId & 0x1f) + 0x60;
+						target[position$1++] = highByte;
+					} else {
+						target[position$1++] = 0xd4; // fixext 1
+						target[position$1++] = 0x72; // "r" record defintion extension type
+						target[position$1++] = recordId;
+					}
+
+					if (newTransitions)
+						transitionsCount += serializationsSinceTransitionRebuild * newTransitions;
+					// record the removal of the id, we can maintain our shared structure
+					if (recordIdsToRemove.length >= maxOwnStructures)
+						recordIdsToRemove.shift()[RECORD_SYMBOL] = 0; // we are cycling back through, and have to remove old ones
+					recordIdsToRemove.push(transition);
+					pack(keys);
+				}
+			};
+			const insertNewRecord = (transition, keys, insertionOffset, newTransitions) => {
+				let mainTarget = target;
+				let mainPosition = position$1;
+				let mainSafeEnd = safeEnd;
+				let mainStart = start;
+				target = keysTarget;
+				position$1 = 0;
+				start = 0;
+				if (!target)
+					keysTarget = target = new ByteArrayAllocate(8192);
+				safeEnd = target.length - 10;
+				newRecord(transition, keys, newTransitions);
+				keysTarget = target;
+				let keysPosition = position$1;
+				target = mainTarget;
+				position$1 = mainPosition;
+				safeEnd = mainSafeEnd;
+				start = mainStart;
+				if (keysPosition > 1) {
+					let newEnd = position$1 + keysPosition - 1;
+					if (newEnd > safeEnd)
+						makeRoom(newEnd);
+					let insertionPosition = insertionOffset + start;
+					target.copyWithin(insertionPosition + keysPosition, insertionPosition + 1, position$1);
+					target.set(keysTarget.slice(0, keysPosition), insertionPosition);
+					position$1 = newEnd;
+				} else {
+					target[insertionOffset + start] = keysTarget[0];
+				}
+			};
 		}
 		useBuffer(buffer) {
 			// this means we are finished using our own buffer and we can write over it safely
@@ -1537,11 +1691,15 @@
 			targetView = new DataView(target.buffer, target.byteOffset, target.byteLength);
 			position$1 = 0;
 		}
+		clearSharedData() {
+			if (this.structures)
+				this.structures = [];
+		}
 	}
 
 	extensionClasses = [ Date, Set, Error, RegExp, ArrayBuffer, Object.getPrototypeOf(Uint8Array.prototype).constructor /*TypedArray*/, C1Type ];
 	extensions = [{
-		pack(date, allocateForWrite) {
+		pack(date, allocateForWrite, pack) {
 			let seconds = date.getTime() / 1000;
 			if ((this.useTimestamp32 || date.getMilliseconds() === 0) && seconds >= 0 && seconds < 0x100000000) {
 				// Timestamp 32
@@ -1556,6 +1714,16 @@
 				target[position++] = 0xff;
 				targetView.setUint32(position, date.getMilliseconds() * 4000000 + ((seconds / 1000 / 0x100000000) >> 0));
 				targetView.setUint32(position + 4, seconds);
+			} else if (isNaN(seconds)) {
+				if (this.onInvalidDate) {
+					allocateForWrite(0);
+					return pack(this.onInvalidDate())
+				}
+				// Intentionally invalid timestamp
+				let { target, targetView, position} = allocateForWrite(3);
+				target[position++] = 0xd4;
+				target[position++] = 0xff;
+				target[position++] = 0xff;
 			} else {
 				// Timestamp 96
 				let { target, targetView, position} = allocateForWrite(15);
@@ -1724,6 +1892,14 @@
 		return serialized
 	}
 
+	function writeBundles(start, pack) {
+		targetView.setUint32(bundledStrings$1.position + start, position$1 - bundledStrings$1.position - start);
+		let writeStrings = bundledStrings$1;
+		bundledStrings$1 = null;
+		pack(writeStrings[0]);
+		pack(writeStrings[1]);
+	}
+
 	function addExtension$1(extension) {
 		if (extension.Class) {
 			if (!extension.pack && !extension.write)
@@ -1741,7 +1917,8 @@
 	const encode = defaultPackr.pack;
 	const Encoder = Packr;
 	const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
-	const REUSE_BUFFER_MODE = 1000;
+	const REUSE_BUFFER_MODE = 512;
+	const RESET_BUFFER_MODE = 1024;
 
 	/**
 	 * Given an Iterable first argument, returns an Iterable where each value is packed as a Buffer
@@ -1848,6 +2025,7 @@
 	exports.decodeIter = decodeIter;
 	exports.encode = encode;
 	exports.encodeIter = encodeIter;
+	exports.isNativeAccelerationEnabled = isNativeAccelerationEnabled;
 	exports.mapsAsObjects = mapsAsObjects;
 	exports.pack = pack;
 	exports.roundFloat32 = roundFloat32;