msgpackr 1.4.4 → 1.5.0

package/README.md

@@ -1,10 +1,11 @@
 # msgpackr
-[![license](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
 [![npm version](https://img.shields.io/npm/v/msgpackr.svg?style=flat-square)](https://www.npmjs.org/package/msgpackr)
+[![npm version](https://img.shields.io/npm/dw/msgpackr)](https://www.npmjs.org/package/msgpackr)
 [![encode](https://img.shields.io/badge/encode-1.5GB%2Fs-yellow)](benchmark.md)
 [![decode](https://img.shields.io/badge/decode-2GB%2Fs-yellow)](benchmark.md)
 [![types](https://img.shields.io/npm/types/msgpackr)](README.md)
 [![module](https://img.shields.io/badge/module-ESM%2FCJS-blue)](README.md)
+[![license](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
 
 
 The msgpackr package is an extremely fast MessagePack NodeJS/JavaScript implementation. Currently, it is significantly faster than any other known implementations, faster than Avro (for JS), and generally faster than native V8 JSON.stringify/parse, on NodeJS. It also includes an optional record extension (the `r` in msgpackr), for defining record structures that makes MessagePack even faster and more compact, often over twice as fast as even native JSON functions, several times faster than other JS implementations, and 15-50% more compact. See the performance section for more details. Structured cloning (with support for cyclical references) is also supported through optional extensions.
@@ -166,6 +167,7 @@ The following options properties can be provided to the Packr or Unpackr constru
 * `sequential` - Encode structures in serialized data, and reference previously encoded structures with expectation that decoder will read the encoded structures in the same order as encoded, with `unpackMultiple`.
 * `largeBigIntToFloat` - If a bigint needs to be encoded that is larger than will fit in 64-bit integers, it will be encoded as a float-64 (otherwise will throw a RangeError).
 * `encodeUndefinedAsNil` - Encodes a value of `undefined` as a MessagePack `nil`, the same as a `null`.
+* `int64AsNumber` - This will decode uint64 and int64 numbers as standard JS numbers rather than as bigint numbers.
 
 ### 32-bit Float Options
 By default all non-integer numbers are serialized as 64-bit float (double). This is fast, and ensures maximum precision. However, often real-world data doesn't not need 64-bits of precision, and using 32-bit encoding can be much more space efficient. There are several options that provide more efficient encodings. Using the decimal rounding options for encoding and decoding provides lossless storage of common decimal representations like 7.99, in more efficient 32-bit format (rather than 64-bit). The `useFloat32` property has several possible options, available from the module as constants:
@@ -176,13 +178,22 @@ const { ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
 
 * `ALWAYS` (1) - Always will encode non-integers (absolute less than 2147483648) as 32-bit float.
 * `DECIMAL_ROUND` (3) - Always will encode non-integers as 32-bit float, and when decoding 32-bit float, round to the significant decimal digits (usually 7, but 6 or 8 digits for some ranges).
-* `DECIMAL_FIT` (4) - Only encode non-integers as 32-bit float if all significant digits (usually up to 7) can be unamiguously encoded as a 32-bit float, and decode/unpack with decimal rounding (same as above). This will ensure round-trip encoding/decoding without loss in precision and use 32-bit when possible.
+* `DECIMAL_FIT` (4) - Only encode non-integers as 32-bit float if all significant digits (usually up to 7) can be unambiguously encoded as a 32-bit float, and decode/unpack with decimal rounding (same as above). This will ensure round-trip encoding/decoding without loss in precision and uses 32-bit when possible.
 
 Note, that the performance is decreased with decimal rounding by about 20-25%, although if only 5% of your values are floating point, that will only have about a 1% impact overall.
 
-In addition, msgpackr exports a `roundFloat32(number)` function that can be used to round floating point numbers to the maximum significant decimal digits that can be stored in 32-bit float, just as DECIMAL_ROUND does when encoding. This can be useful for determine how a number will be stored prior to encoding it.
+In addition, msgpackr exports a `roundFloat32(number)` function that can be used to round floating point numbers to the maximum significant decimal digits that can be stored in 32-bit float, just as DECIMAL_ROUND does when decoding. This can be useful for determining how a number will be decoded prior to encoding it.
 
 ## Performance
+### Native Acceleration
+Msgpackr employs an optional native node-addon to accelerate the parsing of strings. This should be automatically installed and utilized on NodeJS. However, you can verify this by checking the `isNativeAccelerationEnabled` property that is exported from msgpackr. If this is `false`, the `msgpackr-extract` package may not have been properly installed, and you may want to verify that it is installed correctly:
+```js
+import { isNativeAccelerationEnabled } from 'msgpackr'
+if (!isNativeAccelerationEnabled)
+	console.warn('Native acceleration not enabled, verify that install finished properly')
+```
+
+### Benchmarks
 Msgpackr is fast. Really fast. Here is comparison with the next fastest JS projects using the benchmark tool from `msgpack-lite` (and the sample data is from some clinical research data we use that has a good mix of different value types and structures). It also includes comparison to V8 native JSON functionality, and JavaScript Avro (`avsc`, a very optimized Avro implementation):
 
 operation                                                  |   op   |   ms  |  op/s

package/SECURITY.md

@@ -0,0 +1,11 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 1.4.x   | :white_check_mark: |
+
+## Reporting a Vulnerability
+
+Please report security vulnerabilities to kriszyp@gmail.com.

package/dist/index.js

@@ -58,8 +58,16 @@
 			// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
 			// technique for getting data from a database where it can be copied into an existing buffer instead of creating
 			// new ones
-			dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
-			if (this) {
+			try {
+				dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
+			} catch(error) {
+				// if it doesn't have a buffer, maybe it is the wrong type of object
+				src = null;
+				if (source instanceof Uint8Array)
+					throw error
+				throw new Error('Source must be a Uint8Array or Buffer but was a ' + ((source && typeof source == 'object') ? source.constructor.name : typeof source))
+			}
+			if (this instanceof Unpackr) {
 				currentUnpackr = this;
 				if (this.structures) {
 					currentStructures = this.structures;
@@ -291,10 +299,11 @@
 					position += 4;
 					return value
 				case 0xcf:
-					if (currentUnpackr.uint64AsNumber)
-						return src[position++] * 0x100000000000000 + src[position++] * 0x1000000000000 + src[position++] * 0x10000000000 + src[position++] * 0x100000000 +
-							src[position++] * 0x1000000 + (src[position++] << 16) + (src[position++] << 8) + src[position++]
-					value = dataView.getBigUint64(position);
+					if (currentUnpackr.int64AsNumber) {
+						value = dataView.getUint32(position) * 0x100000000;
+						value += dataView.getUint32(position + 4);
+					} else
+						value = dataView.getBigUint64(position);
 					position += 8;
 					return value
 
@@ -310,7 +319,11 @@
 					position += 4;
 					return value
 				case 0xd3:
-					value = dataView.getBigInt64(position);
+					if (currentUnpackr.int64AsNumber) {
+						value = dataView.getInt32(position) * 0x100000000;
+						value += dataView.getUint32(position + 4);
+					} else
+						value = dataView.getBigInt64(position);
 					position += 8;
 					return value
 
@@ -459,6 +472,7 @@
 	var readString8 = readStringJS;
 	var readString16 = readStringJS;
 	var readString32 = readStringJS;
+	let isNativeAccelerationEnabled = false;
 	function readStringJS(length) {
 		let result;
 		if (length < 16) {
@@ -983,7 +997,7 @@
 			let transitionsCount = 0;
 			let serializationsSinceTransitionRebuild = 0;
 
-			this.pack = this.encode = function(value) {
+			this.pack = this.encode = function(value, encodeOptions) {
 				if (!target) {
 					target = new ByteArrayAllocate(8192);
 					targetView = new DataView(target.buffer, 0, 8192);
@@ -1048,6 +1062,11 @@
 						referenceMap = null;
 						return serialized
 					}
+					if (encodeOptions === REUSE_BUFFER_MODE) {
+						target.start = start;
+						target.end = position$1;
+						return target
+					}
 					return target.subarray(start, position$1) // position can change if we call pack again in saveStructures, so we get the buffer now
 				} finally {
 					if (sharedStructures) {
@@ -1071,13 +1090,15 @@
 							if (sharedStructures.length > sharedLength) {
 								sharedStructures = sharedStructures.slice(0, sharedLength);
 							}
-
+							// we can't rely on start/end with REUSE_BUFFER_MODE since they will (probably) change when we save
+							let returnBuffer = target.subarray(start, position$1);
 							if (packr.saveStructures(sharedStructures, lastSharedStructuresLength) === false) {
 								// get updated structures and try again if the update failed
 								packr._mergeStructures(packr.getStructures());
 								return packr.pack(value)
 							}
 							lastSharedStructuresLength = sharedLength;
+							return returnBuffer
 						}
 					}
 				}
@@ -1094,9 +1115,9 @@
 					// first we estimate the header size, so we can write to the correct location
 					if (strLength < 0x20) {
 						headerSize = 1;
-					} else if (strLength < 0xff) {
+					} else if (strLength < 0x100) {
 						headerSize = 2;
-					} else if (strLength < 0xff00) {
+					} else if (strLength < 0x10000) {
 						headerSize = 3;
 					} else {
 						headerSize = 5;
@@ -1509,7 +1530,7 @@
 					if ((end - start) > MAX_BUFFER_SIZE)
 						throw new Error('Packed buffer would be larger than maximum buffer size')
 					newSize = Math.min(MAX_BUFFER_SIZE,
-						Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x1000000) / 0x1000) * 0x1000);
+						Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x400000) / 0x1000) * 0x1000);
 				} else // faster handling for smaller buffers
 					newSize = ((Math.max((end - start) << 2, target.length - 1) >> 12) + 1) << 12;
 				let newBuffer = new ByteArrayAllocate(newSize);
@@ -1733,7 +1754,8 @@
 	const pack = defaultPackr.pack;
 	const encode = defaultPackr.pack;
 	const Encoder = Packr;
-	const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
+	const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
+	const REUSE_BUFFER_MODE = 1000;
 
 	/**
 	 * Given an Iterable first argument, returns an Iterable where each value is packed as a Buffer
@@ -1832,6 +1854,7 @@
 	exports.FLOAT32_OPTIONS = FLOAT32_OPTIONS;
 	exports.NEVER = NEVER;
 	exports.Packr = Packr;
+	exports.REUSE_BUFFER_MODE = REUSE_BUFFER_MODE;
 	exports.Unpackr = Unpackr;
 	exports.addExtension = addExtension$1;
 	exports.clearSource = clearSource;
@@ -1839,6 +1862,7 @@
 	exports.decodeIter = decodeIter;
 	exports.encode = encode;
 	exports.encodeIter = encodeIter;
+	exports.isNativeAccelerationEnabled = isNativeAccelerationEnabled;
 	exports.mapsAsObjects = mapsAsObjects;
 	exports.pack = pack;
 	exports.roundFloat32 = roundFloat32;

package/dist/index.min.js

@@ -7,8 +7,8 @@ case 194:return!1;case 195:return!0;case 196:// bin 8
 return m(A[G++]);case 197:return b=F.getUint16(G),G+=2,m(b);case 198:return b=F.getUint32(G),G+=4,m(b);case 199:// ext 8
 return n(A[G++]);case 200:return b=F.getUint16(G),G+=2,n(b);case 201:return b=F.getUint32(G),G+=4,n(b);case 202:if(b=F.getFloat32(G),2<H.useFloat32){// this does rounding of numbers that were encoded in 32-bit float to nearest significant decimal digit that could be preserved
 let a=_[(127&A[G])<<1|A[G+1]>>7];return G+=4,(a*b+(0<b?.5:-.5)>>0)/a}return G+=4,b;case 203:return b=F.getFloat64(G),G+=8,b;// uint handlers
-case 204:return A[G++];case 205:return b=F.getUint16(G),G+=2,b;case 206:return b=F.getUint32(G),G+=4,b;case 207:return H.uint64AsNumber?72057594037927940*A[G++]+281474976710656*A[G++]+1099511627776*A[G++]+4294967296*A[G++]+16777216*A[G++]+(A[G++]<<16)+(A[G++]<<8)+A[G++]:(b=F.getBigUint64(G),G+=8,b);// int handlers
-case 208:return F.getInt8(G++);case 209:return b=F.getInt16(G),G+=2,b;case 210:return b=F.getInt32(G),G+=4,b;case 211:return b=F.getBigInt64(G),G+=8,b;case 212:if(b=A[G++],114==b)return Y(63&A[G++]);else{let a=K[b];if(a)return a.read?(G++,a.read(e())):a.noBuffer?(G++,a()):a(A.subarray(G,++G));throw new Error("Unknown extension "+b)}case 213:return b=A[G],114==b?(G++,Y(63&A[G++],A[G++])):n(2);case 214:// fixext 4
+case 204:return A[G++];case 205:return b=F.getUint16(G),G+=2,b;case 206:return b=F.getUint32(G),G+=4,b;case 207:return H.int64AsNumber?(b=4294967296*F.getUint32(G),b+=F.getUint32(G+4)):b=F.getBigUint64(G),G+=8,b;// int handlers
+case 208:return F.getInt8(G++);case 209:return b=F.getInt16(G),G+=2,b;case 210:return b=F.getInt32(G),G+=4,b;case 211:return H.int64AsNumber?(b=4294967296*F.getInt32(G),b+=F.getUint32(G+4)):b=F.getBigInt64(G),G+=8,b;case 212:if(b=A[G++],114==b)return Y(63&A[G++]);else{let a=K[b];if(a)return a.read?(G++,a.read(e())):a.noBuffer?(G++,a()):a(A.subarray(G,++G));throw new Error("Unknown extension "+b)}case 213:return b=A[G],114==b?(G++,Y(63&A[G++],A[G++])):n(2);case 214:// fixext 4
 return n(4);case 215:// fixext 8
 return n(8);case 216:// fixext 16
 return n(16);case 217:return b=A[G++],J>=G?D.slice(G-I,(G+=b)-I):T(b);case 218:return b=F.getUint16(G),G+=2,J>=G?D.slice(G-I,(G+=b)-I):U(b);case 219:return b=F.getUint32(G),G+=4,J>=G?D.slice(G-I,(G+=b)-I):V(b);case 220:return b=F.getUint16(G),G+=2,i(b);case 221:return b=F.getUint32(G),G+=4,i(b);case 222:return b=F.getUint16(G),G+=2,j(b);case 223:return b=F.getUint32(G),G+=4,j(b);default:// negative int
@@ -35,7 +35,10 @@ a[g++]=214,a[g++]=105,a[g++]=f>>24,a[g++]=255&f>>16,a[g++]=255&f>>8,a[g++]=255&f
 	 * @param {object} [options] - unpackr options
 	 * @returns {IterableIterator|Promise.<AsyncIterableIterator}
 	 */var z;try{z=new TextDecoder}catch(a){}var A,B,C,D,E,F,G=0,H={},I=0,J=0,K=[],L={useRecords:!1,mapsAsObjects:!0};class M{}const N=new M;N.name="MessagePack 0xC1";var O=!1;class P{constructor(a){a&&(!1===a.useRecords&&a.mapsAsObjects===void 0&&(a.mapsAsObjects=!0),a.structures?a.structures.sharedLength=a.structures.length:a.getStructures&&((a.structures=[]).uninitialized=!0,a.structures.sharedLength=0)),Object.assign(this,a)}unpack(a,b){if(A)// re-entrant execution, save the state and restore it after we do this unpack
-return p(()=>(q(),this?this.unpack(a,b):P.prototype.unpack.call(L,a,b)));if(B=-1<b?b:a.length,G=0,J=0,D=null,A=a,F=a.dataView||(a.dataView=new DataView(a.buffer,a.byteOffset,a.byteLength)),this){if(H=this,this.structures)return C=this.structures,c();(!C||0<C.length)&&(C=[])}else H=L,(!C||0<C.length)&&(C=[]);return c()}unpackMultiple(a,b){let d,e=0;try{O=!0;let f=a.length,g=this?this.unpack(a,f):aa.unpack(a,f);if(b){for(b(g);G<f;)if(e=G,!1===b(c()))return;}else{for(d=[g];G<f;)e=G,d.push(c());return d}}catch(a){throw a.lastPosition=e,a.values=d,a}finally{O=!1,q()}}_mergeStructures(a,b){a=a||[];for(let c,d=0,e=a.length;d<e;d++)c=a[d],c&&(c.isShared=!0,32<=d&&(c.highByte=d-32>>5));for(let c in a.sharedLength=a.length,b||[])if(0<=c){let d=a[c],e=b[c];e&&(d&&((a.restoreStructures||(a.restoreStructures=[]))[c]=d),a[c]=e)}return this.structures=a}decode(a,b){return this.unpack(a,b)}}const Q=/^[a-zA-Z_$][a-zA-Z\d_$]*$/,R=(a,b)=>function(){let c=A[G++];if(0===c)return b();let d=32>a?-(a+(c<<5)):a+(c<<5),e=C[d]||g()[d];if(!e)throw new Error("Record id is not defined for "+d);return e.read||(e.read=f(e,a)),e.read()};var S=h,T=h,U=h,V=h,W=String.fromCharCode,X=Array(4096);const Y=(a,b)=>{var c=e();let d=a;void 0!==b&&(a=32>a?-((b<<5)+a):(b<<5)+a,c.highByte=b);let g=C[a];return g&&g.isShared&&((C.restoreStructures||(C.restoreStructures=[]))[a]=g),C[a]=c,c.read=f(c,d),c.read()};var Z="object"==typeof self?self:global;K[0]=()=>{},K[0].noBuffer=!0,K[101]=()=>{let a=e();return(Z[a[0]]||Error)(a[1])},K[105]=()=>{// id extension (for structured clones)
+return p(()=>(q(),this?this.unpack(a,b):P.prototype.unpack.call(L,a,b)));B=-1<b?b:a.length,G=0,J=0,D=null,A=a;// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
+// technique for getting data from a database where it can be copied into an existing buffer instead of creating
+// new ones
+try{F=a.dataView||(a.dataView=new DataView(a.buffer,a.byteOffset,a.byteLength))}catch(b){if(A=null,a instanceof Uint8Array)throw b;throw new Error("Source must be a Uint8Array or Buffer but was a "+(a&&"object"==typeof a?a.constructor.name:typeof a))}if(this instanceof P){if(H=this,this.structures)return C=this.structures,c();(!C||0<C.length)&&(C=[])}else H=L,(!C||0<C.length)&&(C=[]);return c()}unpackMultiple(a,b){let d,e=0;try{O=!0;let f=a.length,g=this?this.unpack(a,f):aa.unpack(a,f);if(b){for(b(g);G<f;)if(e=G,!1===b(c()))return;}else{for(d=[g];G<f;)e=G,d.push(c());return d}}catch(a){throw a.lastPosition=e,a.values=d,a}finally{O=!1,q()}}_mergeStructures(a,b){a=a||[];for(let c,d=0,e=a.length;d<e;d++)c=a[d],c&&(c.isShared=!0,32<=d&&(c.highByte=d-32>>5));for(let c in a.sharedLength=a.length,b||[])if(0<=c){let d=a[c],e=b[c];e&&(d&&((a.restoreStructures||(a.restoreStructures=[]))[c]=d),a[c]=e)}return this.structures=a}decode(a,b){return this.unpack(a,b)}}const Q=/^[a-zA-Z_$][a-zA-Z\d_$]*$/,R=(a,b)=>function(){let c=A[G++];if(0===c)return b();let d=32>a?-(a+(c<<5)):a+(c<<5),e=C[d]||g()[d];if(!e)throw new Error("Record id is not defined for "+d);return e.read||(e.read=f(e,a)),e.read()};var S=h,T=h,U=h,V=h;var W=String.fromCharCode,X=Array(4096);const Y=(a,b)=>{var c=e();let d=a;void 0!==b&&(a=32>a?-((b<<5)+a):(b<<5)+a,c.highByte=b);let g=C[a];return g&&g.isShared&&((C.restoreStructures||(C.restoreStructures=[]))[a]=g),C[a]=c,c.read=f(c,d),c.read()};var Z="object"==typeof self?self:global;K[0]=()=>{},K[0].noBuffer=!0,K[101]=()=>{let a=e();return(Z[a[0]]||Error)(a[1])},K[105]=()=>{// id extension (for structured clones)
 let a=F.getUint32(G-4);E||(E=new Map);let b,c=A[G];b=144<=c&&160>c||220==c||221==c?[]:{};let d={target:b};// a placeholder object
 E.set(a,d);let f=e();// read the next value as the target object to id
 return d.used?Object.assign(b,f):(d.target=f,f);// no cycle, can just use the returned read object
@@ -45,10 +48,11 @@ return new Z[c](Uint8Array.prototype.slice.call(a,1).buffer)},K[120]=()=>{let a=
 if(4==a.length)return new Date(1e3*(16777216*a[0]+(a[1]<<16)+(a[2]<<8)+a[3]));if(8==a.length)return new Date(((a[0]<<22)+(a[1]<<14)+(a[2]<<6)+(a[3]>>2))/1e6+1e3*(4294967296*(3&a[3])+16777216*a[4]+(a[5]<<16)+(a[6]<<8)+a[7]));if(12==a.length)// TODO: Implement support for negative
 return new Date(((a[0]<<24)+(a[1]<<16)+(a[2]<<8)+a[3])/1e6+1e3*((128&a[4]?-281474976710656:0)+1099511627776*a[6]+4294967296*a[7]+16777216*a[8]+(a[9]<<16)+(a[10]<<8)+a[11]));throw new Error("Invalid timestamp length")};const _=Array(147);// this is a table matching binary exponents to the multiplier to determine significant digit rounding
 for(let c=0;256>c;c++)_[c]=+("1e"+b(45.15-.30103*c));var aa=new P({useRecords:!1});const ba=aa.unpack,ca=aa.unpackMultiple,da=aa.unpack,ea={NEVER:0,ALWAYS:1,DECIMAL_ROUND:3,DECIMAL_FIT:4};let fa,ga=new Float32Array(1),ha=new Uint8Array(ga.buffer,0,4);try{fa=new TextEncoder}catch(a){}let ia,ja;const ka="undefined"!=typeof Buffer,la=ka?Buffer.allocUnsafeSlow:Uint8Array,ma=ka?Buffer:Uint8Array,na=ka?4294967296:2144337920;let oa,pa,qa,ra=0;const sa=Symbol("record-id");class ta extends P{constructor(a){super(a),this.offset=0;let b,c,d,e,f,g=0,h=ma.prototype.utf8Write?function(a,b,c){return oa.utf8Write(a,b,c)}:!!(fa&&fa.encodeInto)&&function(a,b){return fa.encodeInto(a,oa.subarray(b)).written},i=this;a||(a={});let j=a&&a.sequential,k=a.structures||a.saveStructures,l=a.maxSharedStructures;if(null==l&&(l=k?32:0),8160<l)throw new Error("Maximum maxSharedStructure is 8160");let m=a.maxOwnStructures;null==m&&(m=k?32:64),j&&!a.saveStructures&&(this.structures=[]);// two byte record ids for shared structures
-let n=32<l||64<m+l,o=l+64,p=l+m+64;if(8256<p)throw new Error("Maximum maxSharedStructure + maxOwnStructure is 8192");let q=[],r=0,s=0;this.pack=this.encode=function(a){if(oa||(oa=new la(8192),pa=new DataView(oa.buffer,0,8192),ra=0),qa=oa.length-10,2048>qa-ra?(oa=new la(oa.length),pa=new DataView(oa.buffer,0,oa.length),qa=oa.length-10,ra=0):ra=2147483640&ra+7,b=ra,f=i.structuredClone?new Map:null,c=i.structures,c){c.uninitialized&&(c=i._mergeStructures(i.getStructures()));let a=c.sharedLength||0;if(a>l)//if (maxSharedStructures <= 32 && sharedStructures.sharedLength > 32) // TODO: could support this, but would need to update the limit ids
+let n=32<l||64<m+l,o=l+64,p=l+m+64;if(8256<p)throw new Error("Maximum maxSharedStructure + maxOwnStructure is 8192");let q=[],r=0,s=0;this.pack=this.encode=function(a,h){if(oa||(oa=new la(8192),pa=new DataView(oa.buffer,0,8192),ra=0),qa=oa.length-10,2048>qa-ra?(oa=new la(oa.length),pa=new DataView(oa.buffer,0,oa.length),qa=oa.length-10,ra=0):ra=2147483640&ra+7,b=ra,f=i.structuredClone?new Map:null,c=i.structures,c){c.uninitialized&&(c=i._mergeStructures(i.getStructures()));let a=c.sharedLength||0;if(a>l)//if (maxSharedStructures <= 32 && sharedStructures.sharedLength > 32) // TODO: could support this, but would need to update the limit ids
 throw new Error("Shared structures is larger than maximum shared structures, try increasing maxSharedStructures to "+c.sharedLength);if(!c.transitions){c.transitions=Object.create(null);for(let b,d=0;d<a;d++){if(b=c[d],!b)continue;let a,e=c.transitions;for(let c,d=0,f=b.length;d<f;d++)c=b[d],a=e[c],a||(a=e[c]=Object.create(null)),e=a;e[sa]=d+64}g=a}j||(c.nextId=a+64)}d&&(d=!1),e=c||[];try{// update the offset so next serialization doesn't write over our buffer, but can continue writing to same buffer sequentially
-if(t(a),i.offset=ra,f&&f.idsToInsert){ra+=6*f.idsToInsert.length,ra>qa&&x(ra),i.offset=ra;let a=v(oa.subarray(b,ra),f.idsToInsert);return f=null,a}return oa.subarray(b,ra);// position can change if we call pack again in saveStructures, so we get the buffer now
-}finally{if(c){if(10>s&&s++,1e4<r)c.transitions=null,s=0,r=0,0<q.length&&(q=[]);else if(0<q.length&&!j){for(let a=0,b=q.length;a<b;a++)q[a][sa]=0;q=[]}if(d&&i.saveStructures){let b=c.sharedLength||l;if(c.length>b&&(c=c.slice(0,b)),!1===i.saveStructures(c,g))return i._mergeStructures(i.getStructures()),i.pack(a);g=b}}}};const t=a=>{ra>qa&&(oa=x(ra));var c,d=typeof a;if("string"==d){let b,d=a.length;b=32>d?1:255>d?2:65280>d?3:5;let e=3*d;if(ra+e>qa&&(oa=x(ra+e)),64>d||!h){let e,f,g,h=ra+b;for(e=0;e<d;e++)f=a.charCodeAt(e),128>f?oa[h++]=f:2048>f?(oa[h++]=192|f>>6,oa[h++]=128|63&f):55296==(64512&f)&&56320==(64512&(g=a.charCodeAt(e+1)))?(f=65536+((1023&f)<<10)+(1023&g),e++,oa[h++]=240|f>>18,oa[h++]=128|63&f>>12,oa[h++]=128|63&f>>6,oa[h++]=128|63&f):(oa[h++]=224|f>>12,oa[h++]=128|63&f>>6,oa[h++]=128|63&f);c=h-ra-b}else c=h(a,ra+b,e);32>c?oa[ra++]=160|c:256>c?(2>b&&oa.copyWithin(ra+2,ra+1,ra+1+c),oa[ra++]=217,oa[ra++]=c):65536>c?(3>b&&oa.copyWithin(ra+3,ra+2,ra+2+c),oa[ra++]=218,oa[ra++]=c>>8,oa[ra++]=255&c):(5>b&&oa.copyWithin(ra+5,ra+3,ra+3+c),oa[ra++]=219,pa.setUint32(ra,c),ra+=4),ra+=c}else if("number"===d){if(a>>>0===a)64>a?oa[ra++]=a:256>a?(oa[ra++]=204,oa[ra++]=a):65536>a?(oa[ra++]=205,oa[ra++]=a>>8,oa[ra++]=255&a):(oa[ra++]=206,pa.setUint32(ra,a),ra+=4);else if(a>>0===a)-32<=a?oa[ra++]=256+a:-128<=a?(oa[ra++]=208,oa[ra++]=a+256):-32768<=a?(oa[ra++]=209,pa.setInt16(ra,a),ra+=2):(oa[ra++]=210,pa.setInt32(ra,a),ra+=4);else{let b;if(0<(b=this.useFloat32)&&4294967296>a&&-2147483648<=a){oa[ra++]=202,pa.setFloat32(ra,a);let c;if(4>b||// this checks for rounding of numbers that were encoded in 32-bit float to nearest significant decimal digit that could be preserved
+if(t(a),i.offset=ra,f&&f.idsToInsert){ra+=6*f.idsToInsert.length,ra>qa&&x(ra),i.offset=ra;let a=v(oa.subarray(b,ra),f.idsToInsert);return f=null,a}return h===Ba?(oa.start=b,oa.end=ra,oa):oa.subarray(b,ra);// position can change if we call pack again in saveStructures, so we get the buffer now
+}finally{if(c){if(10>s&&s++,1e4<r)c.transitions=null,s=0,r=0,0<q.length&&(q=[]);else if(0<q.length&&!j){for(let a=0,b=q.length;a<b;a++)q[a][sa]=0;q=[]}if(d&&i.saveStructures){let d=c.sharedLength||l;c.length>d&&(c=c.slice(0,d));// we can't rely on start/end with REUSE_BUFFER_MODE since they will (probably) change when we save
+let e=oa.subarray(b,ra);return!1===i.saveStructures(c,g)?(i._mergeStructures(i.getStructures()),i.pack(a)):(g=d,e)}}}};const t=a=>{ra>qa&&(oa=x(ra));var c,d=typeof a;if("string"==d){let b,d=a.length;b=32>d?1:256>d?2:65536>d?3:5;let e=3*d;if(ra+e>qa&&(oa=x(ra+e)),64>d||!h){let e,f,g,h=ra+b;for(e=0;e<d;e++)f=a.charCodeAt(e),128>f?oa[h++]=f:2048>f?(oa[h++]=192|f>>6,oa[h++]=128|63&f):55296==(64512&f)&&56320==(64512&(g=a.charCodeAt(e+1)))?(f=65536+((1023&f)<<10)+(1023&g),e++,oa[h++]=240|f>>18,oa[h++]=128|63&f>>12,oa[h++]=128|63&f>>6,oa[h++]=128|63&f):(oa[h++]=224|f>>12,oa[h++]=128|63&f>>6,oa[h++]=128|63&f);c=h-ra-b}else c=h(a,ra+b,e);32>c?oa[ra++]=160|c:256>c?(2>b&&oa.copyWithin(ra+2,ra+1,ra+1+c),oa[ra++]=217,oa[ra++]=c):65536>c?(3>b&&oa.copyWithin(ra+3,ra+2,ra+2+c),oa[ra++]=218,oa[ra++]=c>>8,oa[ra++]=255&c):(5>b&&oa.copyWithin(ra+5,ra+3,ra+3+c),oa[ra++]=219,pa.setUint32(ra,c),ra+=4),ra+=c}else if("number"===d){if(a>>>0===a)64>a?oa[ra++]=a:256>a?(oa[ra++]=204,oa[ra++]=a):65536>a?(oa[ra++]=205,oa[ra++]=a>>8,oa[ra++]=255&a):(oa[ra++]=206,pa.setUint32(ra,a),ra+=4);else if(a>>0===a)-32<=a?oa[ra++]=256+a:-128<=a?(oa[ra++]=208,oa[ra++]=a+256):-32768<=a?(oa[ra++]=209,pa.setInt16(ra,a),ra+=2):(oa[ra++]=210,pa.setInt32(ra,a),ra+=4);else{let b;if(0<(b=this.useFloat32)&&4294967296>a&&-2147483648<=a){oa[ra++]=202,pa.setFloat32(ra,a);let c;if(4>b||// this checks for rounding of numbers that were encoded in 32-bit float to nearest significant decimal digit that could be preserved
 (c=a*_[(127&oa[ra])<<1|oa[ra+1]>>7])>>0===c)return void(ra+=4);// move back into position for writing a double
 ra--}oa[ra++]=203,pa.setFloat64(ra,a),ra+=8}}else if("object"===d){if(!a)oa[ra++]=192;else{if(f){let c=f.get(a);if(c){if(!c.id){let a=f.idsToInsert||(f.idsToInsert=[]);c.id=a.push(c)}return oa[ra++]=214,oa[ra++]=112,pa.setUint32(ra,c.id),void(ra+=4)}f.set(a,{offset:ra-b})}let d=a.constructor;if(d===Object)w(a,!0);else if(d===Array){c=a.length,16>c?oa[ra++]=144|c:65536>c?(oa[ra++]=220,oa[ra++]=c>>8,oa[ra++]=255&c):(oa[ra++]=221,pa.setUint32(ra,c),ra+=4);for(let b=0;b<c;b++)t(a[b])}else if(d===Map){c=a.size,16>c?oa[ra++]=128|c:65536>c?(oa[ra++]=222,oa[ra++]=c>>8,oa[ra++]=255&c):(oa[ra++]=223,pa.setUint32(ra,c),ra+=4);for(let[b,c]of a)t(b),t(c)}else{for(let b,c=0,d=ia.length;c<d;c++)if(b=ja[c],a instanceof b){let b=ia[c];if(b.write)return b.type&&(oa[ra++]=212,oa[ra++]=b.type,oa[ra++]=0),void t(b.write.call(this,a));let d=oa,e=pa,f=ra;oa=null;let g;try{g=b.pack.call(this,a,a=>(oa=d,d=null,ra+=a,ra>qa&&x(ra),{target:oa,targetView:pa,position:ra-a}),t)}finally{d&&(oa=d,pa=e,ra=f,qa=oa.length-10)}return void(g&&(g.length+ra>qa&&x(g.length+ra),ra=u(g,oa,ra,b.type)))}// no extension found, write as object
 w(a,!a.hasOwnProperty)}}}else if("boolean"===d)oa[ra++]=a?195:194;else if("bigint"===d){if(a<BigInt(1)<<BigInt(63)&&a>=-(BigInt(1)<<BigInt(63)))oa[ra++]=211,pa.setBigInt64(ra,a);else if(a<BigInt(1)<<BigInt(64)&&0<a)oa[ra++]=207,pa.setBigUint64(ra,a);else// overflow
@@ -96,17 +100,17 @@ let d=ra-b;ra+=2;let e=0;for(let b in a)(c||a.hasOwnProperty(b))&&(t(b),t(a[b]),
 			}*/a=>{let b,c=Object.keys(a),f=e.transitions||(e.transitions=Object.create(null)),g=0;for(let d,e=0,h=c.length;e<h;e++)d=c[e],b=f[d],b||(b=f[d]=Object.create(null),g++),f=b;let h=f[sa];if(h)96<=h&&n?(oa[ra++]=(31&(h-=96))+96,oa[ra++]=h>>5):oa[ra++]=h;else{h=e.nextId,h||(h=64),h<o&&this.shouldShareStructure&&!this.shouldShareStructure(c)?(h=e.nextOwnId,!(h<p)&&(h=o),e.nextOwnId=h+1):(h>=p&&(// cycle back around
 h=o),e.nextId=h+1);let a=c.highByte=96<=h&&n?h-96>>5:-1;f[sa]=h,e[h-64]=c,h<o?(c.isShared=!0,e.sharedLength=h-63,d=!0,0<=a?(oa[ra++]=(31&h)+96,oa[ra++]=a):oa[ra++]=h):(0<=a?(oa[ra++]=213,oa[ra++]=114,oa[ra++]=(31&h)+96,oa[ra++]=a):(oa[ra++]=212,oa[ra++]=114,oa[ra++]=h),g&&(r+=s*g),q.length>=m&&(q.shift()[sa]=0),q.push(f),t(c))}// now write the values
 for(let b=0,d=c.length;b<d;b++)t(a[c[b]])},x=a=>{var c=Math.min,d=Math.round,e=Math.max;let f;if(16777216<a){// special handling for really large buffers
-if(a-b>na)throw new Error("Packed buffer would be larger than maximum buffer size");f=c(na,4096*d(e((a-b)*(67108864<a?1.25:2),16777216)/4096))}else// faster handling for smaller buffers
+if(a-b>na)throw new Error("Packed buffer would be larger than maximum buffer size");f=c(na,4096*d(e((a-b)*(67108864<a?1.25:2),4194304)/4096))}else// faster handling for smaller buffers
 f=(e(a-b<<2,oa.length-1)>>12)+1<<12;let g=new la(f);return pa=new DataView(g.buffer,0,f),oa.copy?oa.copy(g,0,b,a):g.set(oa.slice(b,a)),ra-=b,b=0,qa=g.length-10,oa=g}}useBuffer(a){// this means we are finished using our own buffer and we can write over it safely
 oa=a,pa=new DataView(oa.buffer,oa.byteOffset,oa.byteLength),ra=0}}ja=[Date,Set,Error,RegExp,ArrayBuffer,Object.getPrototypeOf(Uint8Array.prototype).constructor/*TypedArray*/,M],ia=[{pack(a,c){let d=a.getTime()/1e3;if((this.useTimestamp32||0===a.getMilliseconds())&&0<=d&&4294967296>d){// Timestamp 32
 let{target:a,targetView:b,position:e}=c(6);a[e++]=214,a[e++]=255,b.setUint32(e,d)}else if(0<d&&17179869184>d){// Timestamp 64
 let{target:b,targetView:e,position:f}=c(10);b[f++]=215,b[f++]=255,e.setUint32(f,4e6*a.getMilliseconds()+(d/1e3/4294967296>>0)),e.setUint32(f+4,d)}else{// Timestamp 96
 let{target:e,targetView:f,position:g}=c(15);e[g++]=199,e[g++]=12,e[g++]=255,f.setUint32(g,1e6*a.getMilliseconds()),f.setBigInt64(g+4,BigInt(b(d)))}}},{pack(a,b,c){let d=Array.from(a),{target:e,position:f}=b(this.structuredClone?3:0);this.structuredClone&&(e[f++]=212,e[f++]=115,e[f++]=0),c(d)}},{pack(a,b,c){let{target:d,position:e}=b(this.structuredClone?3:0);this.structuredClone&&(d[e++]=212,d[e++]=101,d[e++]=0),c([a.name,a.message])}},{pack(a,b,c){let{target:d,position:e}=b(this.structuredClone?3:0);this.structuredClone&&(d[e++]=212,d[e++]=120,d[e++]=0),c([a.source,a.flags])}},{pack(a,b){this.structuredClone?s(a,16,b):t(ka?Buffer.from(a):new Uint8Array(a),b)}},{pack(a,b){let c=a.constructor;c!==ma&&this.structuredClone?s(a,$.indexOf(c.name),b):t(a,b)}},{pack(a,b){// specific 0xC1 object
-let{target:c,position:d}=b(1);c[d]=193}}];let ua=new ta({useRecords:!1});const va=ua.pack,wa=ua.pack,{NEVER:xa,ALWAYS:ya,DECIMAL_ROUND:za,DECIMAL_FIT:Aa}=ea;a.ALWAYS=ya,a.C1=N,a.DECIMAL_FIT=Aa,a.DECIMAL_ROUND=za,a.Decoder=P,a.Encoder=ta,a.FLOAT32_OPTIONS=ea,a.NEVER=xa,a.Packr=ta,a.Unpackr=P,a.addExtension=w,a.clearSource=q,a.decode=da,a.decodeIter=function(a,b={}){if(!a||"object"!=typeof a)throw new Error("first argument must be an Iterable, Async Iterable, Iterator, Async Iterator, or a promise");const c=new P(b);let d;const e=a=>{let b;// if there's incomplete data from previous chunk, concatinate and try again
+let{target:c,position:d}=b(1);c[d]=193}}];let ua=new ta({useRecords:!1});const va=ua.pack,wa=ua.pack,{NEVER:xa,ALWAYS:ya,DECIMAL_ROUND:za,DECIMAL_FIT:Aa}=ea,Ba=1e3;a.ALWAYS=ya,a.C1=N,a.DECIMAL_FIT=Aa,a.DECIMAL_ROUND=za,a.Decoder=P,a.Encoder=ta,a.FLOAT32_OPTIONS=ea,a.NEVER=xa,a.Packr=ta,a.REUSE_BUFFER_MODE=Ba,a.Unpackr=P,a.addExtension=w,a.clearSource=q,a.decode=da,a.decodeIter=function(a,b={}){if(!a||"object"!=typeof a)throw new Error("first argument must be an Iterable, Async Iterable, Iterator, Async Iterator, or a promise");const c=new P(b);let d;const e=a=>{let b;// if there's incomplete data from previous chunk, concatinate and try again
 d&&(a=Buffer.concat([d,a]),d=void 0);try{b=c.unpackMultiple(a)}catch(c){if(c.incomplete)d=a.slice(c.lastPosition),b=c.values;else throw c}return b};if("function"==typeof a[Symbol.iterator])return function*(){for(const b of a)yield*e(b)}();return"function"==typeof a[Symbol.asyncIterator]?async function*(){for await(const b of a)yield*e(b)}():void 0},a.encode=wa,a.encodeIter=/**
 	 * Given an Iterable first argument, returns an Iterable where each value is packed as a Buffer
 	 * If the argument is only Async Iterable, the return value will be an Async Iterable.
 	 * @param {Iterable|Iterator|AsyncIterable|AsyncIterator} objectIterator - iterable source, like a Readable object stream, an array, Set, or custom object
 	 * @param {options} [options] - msgpackr pack options
 	 * @returns {IterableIterator|Promise.<AsyncIterableIterator>}
-	 */function(a,b={}){if(!a||"object"!=typeof a)throw new Error("first argument must be an Iterable, Async Iterable, or a Promise for an Async Iterable");else{if("function"==typeof a[Symbol.iterator])return x(a,b);if("function"==typeof a.then||"function"==typeof a[Symbol.asyncIterator])return y(a,b);throw new Error("first argument must be an Iterable, Async Iterable, Iterator, Async Iterator, or a Promise")}},a.mapsAsObjects=!0,a.pack=va,a.roundFloat32=function(a){ga[0]=a;let b=_[(127&ha[3])<<1|ha[2]>>7];return(b*a+(0<a?.5:-.5)>>0)/b},a.unpack=ba,a.unpackMultiple=ca,a.useRecords=!1,Object.defineProperty(a,"__esModule",{value:!0})});
+	 */function(a,b={}){if(!a||"object"!=typeof a)throw new Error("first argument must be an Iterable, Async Iterable, or a Promise for an Async Iterable");else{if("function"==typeof a[Symbol.iterator])return x(a,b);if("function"==typeof a.then||"function"==typeof a[Symbol.asyncIterator])return y(a,b);throw new Error("first argument must be an Iterable, Async Iterable, Iterator, Async Iterator, or a Promise")}},a.isNativeAccelerationEnabled=!1,a.mapsAsObjects=!0,a.pack=va,a.roundFloat32=function(a){ga[0]=a;let b=_[(127&ha[3])<<1|ha[2]>>7];return(b*a+(0<a?.5:-.5)>>0)/b},a.unpack=ba,a.unpackMultiple=ca,a.useRecords=!1,Object.defineProperty(a,"__esModule",{value:!0})});

package/dist/node.cjs

@@ -64,8 +64,16 @@ class Unpackr {
 		// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
 		// technique for getting data from a database where it can be copied into an existing buffer instead of creating
 		// new ones
-		dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
-		if (this) {
+		try {
+			dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength));
+		} catch(error) {
+			// if it doesn't have a buffer, maybe it is the wrong type of object
+			src = null;
+			if (source instanceof Uint8Array)
+				throw error
+			throw new Error('Source must be a Uint8Array or Buffer but was a ' + ((source && typeof source == 'object') ? source.constructor.name : typeof source))
+		}
+		if (this instanceof Unpackr) {
 			currentUnpackr = this;
 			if (this.structures) {
 				currentStructures = this.structures;
@@ -297,10 +305,11 @@ function read() {
 				position += 4;
 				return value
 			case 0xcf:
-				if (currentUnpackr.uint64AsNumber)
-					return src[position++] * 0x100000000000000 + src[position++] * 0x1000000000000 + src[position++] * 0x10000000000 + src[position++] * 0x100000000 +
-						src[position++] * 0x1000000 + (src[position++] << 16) + (src[position++] << 8) + src[position++]
-				value = dataView.getBigUint64(position);
+				if (currentUnpackr.int64AsNumber) {
+					value = dataView.getUint32(position) * 0x100000000;
+					value += dataView.getUint32(position + 4);
+				} else
+					value = dataView.getBigUint64(position);
 				position += 8;
 				return value
 
@@ -316,7 +325,11 @@ function read() {
 				position += 4;
 				return value
 			case 0xd3:
-				value = dataView.getBigInt64(position);
+				if (currentUnpackr.int64AsNumber) {
+					value = dataView.getInt32(position) * 0x100000000;
+					value += dataView.getUint32(position + 4);
+				} else
+					value = dataView.getBigInt64(position);
 				position += 8;
 				return value
 
@@ -465,8 +478,10 @@ var readFixedString = readStringJS;
 var readString8 = readStringJS;
 var readString16 = readStringJS;
 var readString32 = readStringJS;
+exports.isNativeAccelerationEnabled = false;
 
 function setExtractor(extractStrings) {
+	exports.isNativeAccelerationEnabled = true;
 	readFixedString = readString(1);
 	readString8 = readString(2);
 	readString16 = readString(3);
@@ -1029,7 +1044,7 @@ class Packr extends Unpackr {
 		let transitionsCount = 0;
 		let serializationsSinceTransitionRebuild = 0;
 
-		this.pack = this.encode = function(value) {
+		this.pack = this.encode = function(value, encodeOptions) {
 			if (!target) {
 				target = new ByteArrayAllocate(8192);
 				targetView = new DataView(target.buffer, 0, 8192);
@@ -1094,6 +1109,11 @@ class Packr extends Unpackr {
 					referenceMap = null;
 					return serialized
 				}
+				if (encodeOptions === REUSE_BUFFER_MODE) {
+					target.start = start;
+					target.end = position$1;
+					return target
+				}
 				return target.subarray(start, position$1) // position can change if we call pack again in saveStructures, so we get the buffer now
 			} finally {
 				if (sharedStructures) {
@@ -1117,13 +1137,15 @@ class Packr extends Unpackr {
 						if (sharedStructures.length > sharedLength) {
 							sharedStructures = sharedStructures.slice(0, sharedLength);
 						}
-
+						// we can't rely on start/end with REUSE_BUFFER_MODE since they will (probably) change when we save
+						let returnBuffer = target.subarray(start, position$1);
 						if (packr.saveStructures(sharedStructures, lastSharedStructuresLength) === false) {
 							// get updated structures and try again if the update failed
 							packr._mergeStructures(packr.getStructures());
 							return packr.pack(value)
 						}
 						lastSharedStructuresLength = sharedLength;
+						return returnBuffer
 					}
 				}
 			}
@@ -1140,9 +1162,9 @@ class Packr extends Unpackr {
 				// first we estimate the header size, so we can write to the correct location
 				if (strLength < 0x20) {
 					headerSize = 1;
-				} else if (strLength < 0xff) {
+				} else if (strLength < 0x100) {
 					headerSize = 2;
-				} else if (strLength < 0xff00) {
+				} else if (strLength < 0x10000) {
 					headerSize = 3;
 				} else {
 					headerSize = 5;
@@ -1555,7 +1577,7 @@ class Packr extends Unpackr {
 				if ((end - start) > MAX_BUFFER_SIZE)
 					throw new Error('Packed buffer would be larger than maximum buffer size')
 				newSize = Math.min(MAX_BUFFER_SIZE,
-					Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x1000000) / 0x1000) * 0x1000);
+					Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x400000) / 0x1000) * 0x1000);
 			} else // faster handling for smaller buffers
 				newSize = ((Math.max((end - start) << 2, target.length - 1) >> 12) + 1) << 12;
 			let newBuffer = new ByteArrayAllocate(newSize);
@@ -1779,7 +1801,8 @@ let defaultPackr = new Packr({ useRecords: false });
 const pack = defaultPackr.pack;
 const encode = defaultPackr.pack;
 const Encoder = Packr;
-const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
+const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS;
+const REUSE_BUFFER_MODE = 1000;
 
 class PackrStream extends stream.Transform {
 	constructor(options) {
@@ -1931,9 +1954,7 @@ function tryRequire(moduleId) {
 		let require$1 = module$1.createRequire((typeof document === 'undefined' ? new (require('u' + 'rl').URL)('file:' + __filename).href : (document.currentScript && document.currentScript.src || new URL('node.cjs', document.baseURI).href)));
 		return require$1(moduleId)
 	} catch (error) {
-		if (typeof window == 'undefined')
-			console.warn('Native extraction module not loaded, msgpackr will still run, but with decreased performance. ' + error.message.split('\n')[0]);
-		else
+		if (typeof window != 'undefined')
 			console.warn('For browser usage, directly use msgpackr/unpack or msgpackr/pack modules. ' + error.message.split('\n')[0]);
 	}
 }

package/dist/test.js

@@ -660,6 +660,13 @@
 			assert.deepEqual(deserialized, data);
 		});
 
+		test('255 chars', function() {
+			const data = 'RRZG9A6I7xupPeOZhxcOcioFsuhszGOdyDUcbRf4Zef2kdPIfC9RaLO4jTM5JhuZvTsF09fbRHMGtqk7YAgu3vespeTe9l61ziZ6VrMnYu2CamK96wCkmz0VUXyqaiUoTPgzk414LS9yYrd5uh7w18ksJF5SlC2e91rukWvNqAZJjYN3jpkqHNOFchCwFrhbxq2Lrv1kSJPYCx9blRg2hGmYqTbElLTZHv20iNqwZeQbRMgSBPT6vnbCBPnOh1W';
+			var serialized = pack(data);
+			var deserialized = unpack(serialized);
+			assert.equal(deserialized, data);
+		});
+
 		test('pack/unpack sample data', function(){
 			var data = sampleData;
 			var serialized = pack(data);
@@ -1064,7 +1071,17 @@
 			var deserialized = packr.unpack(serialized);
 			assert.deepEqual(deserialized, data);
 		});
-
+		test('bigint to float', function() {
+			var data = {
+				a: 325283295382932843n
+			};
+			let packr = new Packr({
+				int64AsNumber: true
+			});
+			var serialized = packr.pack(data);
+			var deserialized = packr.unpack(serialized);
+			assert.deepEqual(deserialized.a, 325283295382932843);
+		});
 		test('numbers', function(){
 			var data = {
 				bigEncodable: 48978578104322,

package/index.d.ts

@@ -1,4 +1,4 @@
-export { Unpackr, Decoder, unpack, decode, addExtension, FLOAT32_OPTIONS, clearSource } from './unpack'
+export { Unpackr, Decoder, unpack, decode, addExtension, FLOAT32_OPTIONS, clearSource, roundFloat32, isNativeAccelerationEnabled } from './unpack'
 import { Options } from './unpack'
 export { Packr, Encoder, pack, encode } from './pack'
 import { Transform, Readable } from 'stream'

package/index.js

@@ -1,5 +1,5 @@
-export { Packr, Encoder, addExtension, pack, encode, NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } from './pack.js'
-export { Unpackr, Decoder, C1, unpack, unpackMultiple, decode, FLOAT32_OPTIONS, clearSource, roundFloat32 } from './unpack.js'
+export { Packr, Encoder, addExtension, pack, encode, NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT, REUSE_BUFFER_MODE } from './pack.js'
+export { Unpackr, Decoder, C1, unpack, unpackMultiple, decode, FLOAT32_OPTIONS, clearSource, roundFloat32, isNativeAccelerationEnabled } from './unpack.js'
 export { decodeIter, encodeIter } from './iterators.js'
 export const useRecords = false
 export const mapsAsObjects = true

package/node-index.js

@@ -1,5 +1,5 @@
 export { Packr, Encoder, addExtension, pack, encode, NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } from './pack.js'
-export { Unpackr, Decoder, C1, unpack, unpackMultiple, decode, FLOAT32_OPTIONS, clearSource, roundFloat32 } from './unpack.js'
+export { Unpackr, Decoder, C1, unpack, unpackMultiple, decode, FLOAT32_OPTIONS, clearSource, roundFloat32, isNativeAccelerationEnabled } from './unpack.js'
 export { PackrStream, UnpackrStream, PackrStream as EncoderStream, UnpackrStream as DecoderStream } from './stream.js'
 export { decodeIter, encodeIter } from './iterators.js'
 export const useRecords = false
@@ -16,9 +16,7 @@ function tryRequire(moduleId) {
 		let require = createRequire(import.meta.url)
 		return require(moduleId)
 	} catch (error) {
-		if (typeof window == 'undefined')
-			console.warn('Native extraction module not loaded, msgpackr will still run, but with decreased performance. ' + error.message.split('\n')[0])
-		else
+		if (typeof window != 'undefined')
 			console.warn('For browser usage, directly use msgpackr/unpack or msgpackr/pack modules. ' + error.message.split('\n')[0])
 	}
 }

package/pack.js

@@ -58,7 +58,7 @@ export class Packr extends Unpackr {
 		let transitionsCount = 0
 		let serializationsSinceTransitionRebuild = 0
 
-		this.pack = this.encode = function(value) {
+		this.pack = this.encode = function(value, encodeOptions) {
 			if (!target) {
 				target = new ByteArrayAllocate(8192)
 				targetView = new DataView(target.buffer, 0, 8192)
@@ -123,6 +123,11 @@ export class Packr extends Unpackr {
 					referenceMap = null
 					return serialized
 				}
+				if (encodeOptions === REUSE_BUFFER_MODE) {
+					target.start = start
+					target.end = position
+					return target
+				}
 				return target.subarray(start, position) // position can change if we call pack again in saveStructures, so we get the buffer now
 			} finally {
 				if (sharedStructures) {
@@ -146,13 +151,15 @@ export class Packr extends Unpackr {
 						if (sharedStructures.length > sharedLength) {
 							sharedStructures = sharedStructures.slice(0, sharedLength)
 						}
-
+						// we can't rely on start/end with REUSE_BUFFER_MODE since they will (probably) change when we save
+						let returnBuffer = target.subarray(start, position)
 						if (packr.saveStructures(sharedStructures, lastSharedStructuresLength) === false) {
 							// get updated structures and try again if the update failed
 							packr._mergeStructures(packr.getStructures())
 							return packr.pack(value)
 						}
 						lastSharedStructuresLength = sharedLength
+						return returnBuffer
 					}
 				}
 			}
@@ -169,9 +176,9 @@ export class Packr extends Unpackr {
 				// first we estimate the header size, so we can write to the correct location
 				if (strLength < 0x20) {
 					headerSize = 1
-				} else if (strLength < 0xff) {
+				} else if (strLength < 0x100) {
 					headerSize = 2
-				} else if (strLength < 0xff00) {
+				} else if (strLength < 0x10000) {
 					headerSize = 3
 				} else {
 					headerSize = 5
@@ -584,7 +591,7 @@ export class Packr extends Unpackr {
 				if ((end - start) > MAX_BUFFER_SIZE)
 					throw new Error('Packed buffer would be larger than maximum buffer size')
 				newSize = Math.min(MAX_BUFFER_SIZE,
-					Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x1000000) / 0x1000) * 0x1000)
+					Math.round(Math.max((end - start) * (end > 0x4000000 ? 1.25 : 2), 0x400000) / 0x1000) * 0x1000)
 			} else // faster handling for smaller buffers
 				newSize = ((Math.max((end - start) << 2, target.length - 1) >> 12) + 1) << 12
 			let newBuffer = new ByteArrayAllocate(newSize)
@@ -817,3 +824,4 @@ export const Encoder = Packr
 export { FLOAT32_OPTIONS } from './unpack.js'
 import { FLOAT32_OPTIONS } from './unpack.js'
 export const { NEVER, ALWAYS, DECIMAL_ROUND, DECIMAL_FIT } = FLOAT32_OPTIONS
+export const REUSE_BUFFER_MODE = 1000

package/package.json

@@ -1,10 +1,12 @@
 {
   "name": "msgpackr",
   "author": "Kris Zyp",
-  "version": "1.4.4",
+  "version": "1.5.0",
   "description": "Ultra-fast MessagePack implementation with extensions for records and structured cloning",
   "license": "MIT",
   "types": "./index.d.ts",
+  "main": "./dist/node.cjs",
+  "module": "./index.js",
   "keywords": [
     "MessagePack",
     "msgpack",
@@ -19,6 +21,7 @@
   "scripts": {
     "benchmark": "node ./tests/benchmark.cjs",
     "build": "rollup -c",
+    "dry-run": "npm publish --dry-run",
     "prepare": "npm run build",
     "test": "mocha tests/test**.*js -u tdd --experimental-json-modules"
   },

package/tests/test-compatibility.cjs

@@ -1,11 +1,12 @@
 const data = require('./example4.json');
-const { pack, unpack } = require('msgpackr/pack');
+const { pack, unpack, Packr } = require('msgpackr/pack');
 const chai = require('chai');
 
 function tryRequire(module) {
 	try {
 		return require(module)
 	} catch(error) {
+		console.log(error)
 	}
 }
 //if (typeof chai === 'undefined') { chai = require('chai') }
@@ -13,6 +14,7 @@ const assert = chai.assert
 //if (typeof msgpackr === 'undefined') { msgpackr = require('..') }
 var msgpack_msgpack = tryRequire('@msgpack/msgpack');
 var msgpack_lite = tryRequire('msgpack-lite');
+var msgpack = tryRequire('msgpack');
 
 const addCompatibilitySuite = (data) => () => {
 	if (msgpack_msgpack) {
@@ -41,8 +43,22 @@ const addCompatibilitySuite = (data) => () => {
 			assert.deepEqual(deserialized, data)
 		})
 	}
+	if (msgpack) {
+		test.skip('from msgpack', function(){
+			var serialized = msgpack.pack(data)
+			var deserialized = unpack(serialized)
+			assert.deepEqual(deserialized, data)
+		})
+
+		test('to msgpack', function(){
+			var serialized = pack(data)
+			var deserialized = msgpack.unpack(serialized)
+			assert.deepEqual(deserialized, data)
+		})
+	}
 }
 
 suite('msgpackr compatibility tests (example)', addCompatibilitySuite(require('./example.json')))
 suite('msgpackr compatibility tests (example4)', addCompatibilitySuite(require('./example4.json')))
-suite('msgpackr compatibility tests (example5)', addCompatibilitySuite(require('./example5.json')))
+suite('msgpackr compatibility tests (example5)', addCompatibilitySuite(require('./example5.json')))
+suite.skip('msgpackr compatibility tests with dates', addCompatibilitySuite({ date: new Date() }))

package/tests/test.js

@@ -84,6 +84,13 @@ suite('msgpackr basic tests', function(){
 		assert.deepEqual(deserialized, data)
 	})
 
+	test('255 chars', function() {
+		const data = 'RRZG9A6I7xupPeOZhxcOcioFsuhszGOdyDUcbRf4Zef2kdPIfC9RaLO4jTM5JhuZvTsF09fbRHMGtqk7YAgu3vespeTe9l61ziZ6VrMnYu2CamK96wCkmz0VUXyqaiUoTPgzk414LS9yYrd5uh7w18ksJF5SlC2e91rukWvNqAZJjYN3jpkqHNOFchCwFrhbxq2Lrv1kSJPYCx9blRg2hGmYqTbElLTZHv20iNqwZeQbRMgSBPT6vnbCBPnOh1W'
+		var serialized = pack(data)
+		var deserialized = unpack(serialized)
+		assert.equal(deserialized, data)
+	})
+
 	test('pack/unpack sample data', function(){
 		var data = sampleData
 		let structures = []
@@ -491,7 +498,17 @@ suite('msgpackr basic tests', function(){
 		var deserialized = packr.unpack(serialized)
 		assert.deepEqual(deserialized, data)
 	})
-
+	test('bigint to float', function() {
+		var data = {
+			a: 325283295382932843n
+		}
+		let packr = new Packr({
+			int64AsNumber: true
+		})
+		var serialized = packr.pack(data)
+		var deserialized = packr.unpack(serialized)
+		assert.deepEqual(deserialized.a, 325283295382932843)
+	})
 	test('numbers', function(){
 		var data = {
 			bigEncodable: 48978578104322,

package/unpack.d.ts

@@ -25,8 +25,10 @@ export interface Options {
 interface Extension {
 	Class: Function
 	type: number
-	pack(value: any): Buffer | Uint8Array
-	unpack(messagePack: Buffer | Uint8Array): any
+	pack?(value: any): Buffer | Uint8Array
+	unpack?(messagePack: Buffer | Uint8Array): any	
+	read?(datum: any): any
+	write?(instance: any): any
 }
 export class Unpackr {
 	constructor(options?: Options)
@@ -42,3 +44,4 @@ export function addExtension(extension: Extension): void
 export function clearSource(): void
 export function roundFloat32(float32Number: number): number
 export const C1: {}
+export let isNativeAccelerationEnabled: boolean

package/unpack.js

@@ -59,8 +59,16 @@ export class Unpackr {
 		// this provides cached access to the data view for a buffer if it is getting reused, which is a recommend
 		// technique for getting data from a database where it can be copied into an existing buffer instead of creating
 		// new ones
-		dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength))
-		if (this) {
+		try {
+			dataView = source.dataView || (source.dataView = new DataView(source.buffer, source.byteOffset, source.byteLength))
+		} catch(error) {
+			// if it doesn't have a buffer, maybe it is the wrong type of object
+			src = null
+			if (source instanceof Uint8Array)
+				throw error
+			throw new Error('Source must be a Uint8Array or Buffer but was a ' + ((source && typeof source == 'object') ? source.constructor.name : typeof source))
+		}
+		if (this instanceof Unpackr) {
 			currentUnpackr = this
 			if (this.structures) {
 				currentStructures = this.structures
@@ -295,10 +303,11 @@ export function read() {
 				position += 4
 				return value
 			case 0xcf:
-				if (currentUnpackr.uint64AsNumber)
-					return src[position++] * 0x100000000000000 + src[position++] * 0x1000000000000 + src[position++] * 0x10000000000 + src[position++] * 0x100000000 +
-						src[position++] * 0x1000000 + (src[position++] << 16) + (src[position++] << 8) + src[position++]
-				value = dataView.getBigUint64(position)
+				if (currentUnpackr.int64AsNumber) {
+					value = dataView.getUint32(position) * 0x100000000
+					value += dataView.getUint32(position + 4)
+				} else
+					value = dataView.getBigUint64(position)
 				position += 8
 				return value
 
@@ -314,7 +323,11 @@ export function read() {
 				position += 4
 				return value
 			case 0xd3:
-				value = dataView.getBigInt64(position)
+				if (currentUnpackr.int64AsNumber) {
+					value = dataView.getInt32(position) * 0x100000000
+					value += dataView.getUint32(position + 4)
+				} else
+					value = dataView.getBigInt64(position)
 				position += 8
 				return value
 
@@ -463,8 +476,10 @@ var readFixedString = readStringJS
 var readString8 = readStringJS
 var readString16 = readStringJS
 var readString32 = readStringJS
+export let isNativeAccelerationEnabled = false
 
 export function setExtractor(extractStrings) {
+	isNativeAccelerationEnabled = true
 	readFixedString = readString(1)
 	readString8 = readString(2)
 	readString16 = readString(3)
@@ -951,8 +966,6 @@ export const mult10 = new Array(147) // this is a table matching binary exponent
 for (let i = 0; i < 256; i++) {
 	mult10[i] = +('1e' + Math.floor(45.15 - i * 0.30103))
 }
-export const useRecords = false
-export const mapsAsObjects = true
 export const Decoder = Unpackr
 var defaultUnpackr = new Unpackr({ useRecords: false })
 export const unpack = defaultUnpackr.unpack