@travetto/runtime 7.1.4 → 8.0.0-alpha.0

package/README.md

@@ -19,6 +19,8 @@ Runtime is the foundation of all [Travetto](https://travetto.dev) applications.
    *  Standard Error Support
    *  Console Management
    *  Resource Access
+   *  Encoding and Decoding Utilities
+   *  Binary Utilities
    *  JSON Utilities
    *  Common Utilities
    *  Time Utilities
@@ -163,9 +165,9 @@ export class EnvProp<T> {
 ```
 
 ## Standard Error Support
-While the framework is 100 % compatible with standard `Error` instances, there are cases in which additional functionality is desired. Within the framework we use [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) (or its derivatives) to represent framework errors. This class is available for use in your own projects. Some of the additional benefits of using this class is enhanced error reporting, as well as better integration with other modules (e.g. the [Web API](https://github.com/travetto/travetto/tree/main/module/web#readme "Declarative support for creating Web Applications") module and HTTP status codes). 
+While the framework is 100 % compatible with standard `Error` instances, there are cases in which additional functionality is desired. Within the framework we use [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) (or its derivatives) to represent framework errors. This class is available for use in your own projects. Some of the additional benefits of using this class is enhanced error reporting, as well as better integration with other modules (e.g. the [Web API](https://github.com/travetto/travetto/tree/main/module/web#readme "Declarative support for creating Web Applications") module and HTTP status codes). 
 
-The [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) takes in a message, and an optional payload and / or error classification. The currently supported error classifications are:
+The [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) takes in a message, and an optional payload and / or error classification. The currently supported error classifications are:
    *  `general` - General purpose errors
    *  `system` - Synonym for `general`
    *  `data` - Data format, content, etc are incorrect. Generally correlated to bad input.
@@ -249,25 +251,26 @@ $ DEBUG=express:*,@travetto/web npx trv run web
 ## Resource Access
 The primary access patterns for resources, is to directly request a file, and to resolve that file either via file-system look up or leveraging the [Manifest](https://github.com/travetto/travetto/tree/main/module/manifest#readme "Support for project indexing, manifesting, along with file watching")'s data for what resources were found at manifesting time.
 
-The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L12) allows for accessing information about the resources, and subsequently reading the file as text/binary or to access the resource as a `Readable` stream.  If a file is not found, it will throw an [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) with a category of 'notfound'.  
+The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L11) allows for accessing information about the resources, and subsequently reading the file as text/binary or to access the resource as a `Readable` stream.  If a file is not found, it will throw an [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) with a category of 'notfound'.  
 
-The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L12) also supports tying itself to [Env](https://github.com/travetto/travetto/tree/main/module/runtime/src/env.ts#L114)'s `TRV_RESOURCES` information on where to attempt to find a requested resource.
+The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L11) also supports tying itself to [Env](https://github.com/travetto/travetto/tree/main/module/runtime/src/env.ts#L114)'s `TRV_RESOURCES` information on where to attempt to find a requested resource.
 
-## JSON Utilities
-The framework provides utilities for working with JSON data.  This module provides methods for reading and writing JSON files, as well as serializing and deserializing JSON data.  It also provides support for working with Base64 encoded data for web safe transfer.  The primary goal is ease of use, but also a centralized location for performance and security improvements over time.
-
-   *  `parseSafe(input: string | Buffer)` parses JSON safely from a string or Buffer.
-   *  `stringifyBase64(value: any)` encodes a JSON value as a base64 encoded string.
-   *  `parseBase64(input: string)` decodes a JSON value from a base64 encoded string.
-   *  `readFile(file: string)` reads a JSON file asynchronously.
-   *  `readFileSync(file: string, onMissing?: any)` reads a JSON file synchronously.
+## Encoding and Decoding Utilities
+The [CodecUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/codec.ts#L15) class provides a variety of static methods for encoding and decoding data. When working with JSON data, it also provide security checks to prevent prototype pollution. The utility supports the following formats:
+   *  Hex
+   *  Base64
+   *  UTF8
+   *  UTT8 Encoded JSON
+   *  Base64 Encoded JSON
+   *  New Line Delimited UTF8
 
 ## Common Utilities
-Common utilities used throughout the framework. Currently [Util](https://github.com/travetto/travetto/tree/main/module/runtime/src/util.ts#L11) includes:
+Common utilities used throughout the framework. Currently [Util](https://github.com/travetto/travetto/tree/main/module/runtime/src/util.ts#L14) includes:
    *  `uuid(len: number)` generates a simple uuid for use within the application.
    *  `allowDenyMatcher(rules[])` builds a matching function that leverages the rules as an allow/deny list, where order of the rules matters.  Negative rules are prefixed by '!'.
    *  `hash(text: string, size?: number)` produces a full sha512 hash.
    *  `resolvablePromise()` produces a `Promise` instance with the `resolve` and `reject` methods attached to the instance.  This is extremely useful for integrating promises into async iterations, or any other situation in which the promise creation and the execution flow don't always match up.
+   *  `bufferedFileWrite(file:string, content: string)` will write the file, using a temporary buffer file to ensure that the entire file is written before being moved to the final location.  This helps minimize file watch noise when writing files.
 
 **Code: Sample makeTemplate Usage**
 ```typescript
@@ -277,48 +280,46 @@ tpl`{{age:20}} {{name: 'bob'}}</>;
 '**age: 20** **name: bob**'
 ```
 
+## Binary Utilities
+The [BinaryUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/binary.ts#L59) class provides a unified interface for working with binary data across different formats, especially bridging the gap between Node.js specific types (`Buffer`, `Stream`) and Web Standard types (`Blob`, `ArrayBuffer`). The framework leverages this to allow for seamless handling of binary data, regardless of the source.
+
+## JSON Utilities
+The [JSONUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/json.ts#L31) class provides a comprehensive set of utilities for working with JSON data, including serialization, deserialization, encoding, and deep cloning capabilities. The utility handles special types like `Date`, `BigInt`, and `Error` objects seamlessly. Key features include:
+   *  `fromUTF8(input, config?)` - Parse JSON from a UTF-8 string
+   *  `toUTF8(value, config?)` - Serialize a value to JSON string
+   *  `toUTF8Pretty(value)` - Serialize with pretty formatting (2-space indent)
+   *  `fromBinaryArray(input)` - Parse JSON from binary array
+   *  `toBinaryArray(value, config?)` - Serialize to binary array (UTF-8 encoded)
+   *  `toBase64(value)` - Encode JSON as base64 string
+   *  `fromBase64(input)` - Decode JSON from base64 string
+   *  `clone(input, config?)` - Deep clone objects with optional transformations
+   *  `cloneForTransmit(input)` - Clone for transmission with error serialization
+   *  `cloneFromTransmit(input)` - Clone from transmission with type restoration
+
+The `TRANSMIT_REVIVER` automatically restores `Date` objects and `BigInt` values during deserialization, making it ideal for transmitting complex data structures across network boundaries.
+
 ## Time Utilities
-[TimeUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/time.ts#L20) contains general helper methods, created to assist with time-based inputs via environment variables, command line interfaces, and other string-heavy based input.
+[TimeUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/time.ts#L21) contains general helper methods, created to assist with time-based inputs via environment variables, command line interfaces, and other string-heavy based input.
 
 **Code: Time Utilities**
 ```typescript
 export class TimeUtil {
   /**
    * Test to see if a string is valid for relative time
-   * @param val
    */
   static isTimeSpan(value: string): value is TimeSpan;
   /**
-   * Returns time units convert to ms
-   * @param amount Number of units to extend
-   * @param unit Time unit to extend ('ms', 's', 'm', 'h', 'd', 'w', 'y')
-   */
-  static asMillis(amount: Date | number | TimeSpan, unit?: TimeUnit): number;
-  /**
-   * Returns the time converted to seconds
-   * @param date The date to convert
-   */
-  static asSeconds(date: Date | number | TimeSpan, unit?: TimeUnit): number;
-  /**
-   * Returns the time converted to a Date
-   * @param date The date to convert
-   */
-  static asDate(date: Date | number | TimeSpan, unit?: TimeUnit): Date;
-  /**
-   * Resolve time or span to possible time
+   * Exposes the ability to create a duration succinctly
    */
-  static fromValue(value: Date | number | string | undefined): number | undefined;
+  static duration(input: TimeSpan | number | string, outputUnit: TimeUnit): number;
   /**
    * Returns a new date with `amount` units into the future
-   * @param amount Number of units to extend
-   * @param unit Time unit to extend ('ms', 's', 'm', 'h', 'd', 'w', 'y')
    */
-  static fromNow(amount: number | TimeSpan, unit: TimeUnit = 'ms'): Date;
+  static fromNow(input: TimeSpan | number | string): Date;
   /**
    * Returns a pretty timestamp
-   * @param time Time in milliseconds
    */
-  static asClock(time: number): string;
+  static asClock(input: TimeSpan | number | string): string;
 }
 ```
 

package/__index__.ts

@@ -1,11 +1,13 @@
 import type { } from './src/global.d.ts';
 import type { } from './src/trv.d.ts';
 export * from './src/binary.ts';
+export * from './src/binary-metadata.ts';
 export * from './src/console.ts';
 export * from './src/context.ts';
 export * from './src/debug.ts';
 export * from './src/error.ts';
 export * from './src/exec.ts';
+export * from './src/codec.ts';
 export * from './src/env.ts';
 export * from './src/file-loader.ts';
 export * from './src/function.ts';
@@ -17,4 +19,4 @@ export * from './src/shutdown.ts';
 export * from './src/time.ts';
 export * from './src/types.ts';
 export * from './src/watch.ts';
-export * from './src/util.ts';
+export * from './src/util.ts';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@travetto/runtime",
-  "version": "7.1.4",
+  "version": "8.0.0-alpha.0",
   "type": "module",
   "description": "Runtime for travetto applications.",
   "keywords": [
@@ -26,12 +26,13 @@
     "directory": "module/runtime"
   },
   "dependencies": {
-    "@travetto/manifest": "^7.1.3",
+    "@travetto/manifest": "^8.0.0-alpha.0",
     "@types/debug": "^4.1.12",
-    "debug": "^4.4.3"
+    "debug": "^4.4.3",
+    "temporal-polyfill-lite": "^0.2.2"
   },
   "peerDependencies": {
-    "@travetto/transformer": "^7.1.3"
+    "@travetto/transformer": "^8.0.0-alpha.0"
   },
   "peerDependenciesMeta": {
     "@travetto/transformer": {

package/src/binary-metadata.ts

@@ -0,0 +1,174 @@
+import crypto from 'node:crypto';
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { createReadStream, ReadStream } from 'node:fs';
+
+import { BinaryUtil, type BinaryArray, type BinaryContainer, type BinaryStream, type BinaryType } from './binary.ts';
+import { RuntimeError } from './error.ts';
+import { CodecUtil } from './codec.ts';
+
+type BlobInput = BinaryType | (() => (BinaryType | Promise<BinaryType>));
+
+/** Range of bytes, inclusive */
+export type ByteRange = { start: number, end?: number };
+
+export interface BinaryMetadata {
+  /** Size of binary data */
+  size?: number;
+  /** Mime type of the content */
+  contentType?: string;
+  /** Hash of binary data contents */
+  hash?: string;
+  /** The original base filename of the file */
+  filename?: string;
+  /** Filenames title, optional for elements like images, audio, videos */
+  title?: string;
+  /** Content encoding */
+  contentEncoding?: string;
+  /** Content language */
+  contentLanguage?: string;
+  /** Cache control */
+  cacheControl?: string;
+  /** Byte range for binary data */
+  range?: Required<ByteRange>;
+}
+
+type HashConfig = {
+  length?: number;
+  hashAlgorithm?: 'sha1' | 'sha256' | 'sha512' | 'md5';
+  outputEncoding?: crypto.BinaryToTextEncoding;
+};
+
+const BinaryMetaSymbol = Symbol();
+
+export class BinaryMetadataUtil {
+  /** Set metadata for a binary type  */
+  static write(input: BinaryType, metadata: BinaryMetadata): BinaryMetadata {
+    const withMeta: BinaryType & { [BinaryMetaSymbol]?: BinaryMetadata } = input;
+    return withMeta[BinaryMetaSymbol] = metadata;
+  }
+
+  /** Read metadata for a binary type, if available  */
+  static read(input: BinaryType): BinaryMetadata {
+    const withMeta: BinaryType & { [BinaryMetaSymbol]?: BinaryMetadata } = input;
+    return withMeta[BinaryMetaSymbol] ?? {};
+  }
+
+  /** Generate a hash from an input value  * @param input The seed value to build the hash from
+   * @param length The optional length of the hash to generate
+   * @param hashAlgorithm The hash algorithm to use
+   * @param outputEncoding The output encoding format
+   */
+  static hash(input: string | BinaryArray, config?: HashConfig): string;
+  static hash(input: BinaryStream | BinaryContainer, config?: HashConfig): Promise<string>;
+  static hash(input: string | BinaryType, config?: HashConfig): string | Promise<string> {
+    const hashAlgorithm = config?.hashAlgorithm ?? 'sha512';
+    const outputEncoding = config?.outputEncoding ?? 'hex';
+    const length = config?.length;
+    const hash = crypto.createHash(hashAlgorithm).setEncoding(outputEncoding);
+
+    if (typeof input === 'string') {
+      input = CodecUtil.fromUTF8String(input);
+    }
+
+    if (BinaryUtil.isBinaryArray(input)) {
+      hash.update(BinaryUtil.binaryArrayToUint8Array(input));
+      return hash.digest(outputEncoding).substring(0, length);
+    } else {
+      return BinaryUtil.pipeline(input, hash).then(() =>
+        hash.digest(outputEncoding).substring(0, length)
+      );
+    }
+  }
+
+  /** Compute the length of the binary data to be returned */
+  static readLength(metadata: BinaryMetadata): number | undefined {
+    return metadata.range ? (metadata.range.end - metadata.range.start + 1) : metadata.size;
+  }
+
+  /** Compute metadata for a given binary input */
+  static async compute(input: BlobInput, base: BinaryMetadata = {}): Promise<BinaryMetadata> {
+    if (typeof input === 'function') {
+      input = await input();
+    }
+    const metadata = { ...BinaryMetadataUtil.read(input), ...base };
+
+    if (BinaryUtil.isBinaryContainer(input)) {
+      metadata.size ??= input.size;
+      metadata.contentType ??= input.type;
+      if (input instanceof File) {
+        metadata.filename ??= input.name;
+      }
+    } else if (BinaryUtil.isBinaryArray(input)) {
+      metadata.size ??= input.byteLength;
+      metadata.hash ??= this.hash(input, { hashAlgorithm: 'sha256' });
+    } else if (input instanceof ReadStream) {
+      const location = input.path.toString();
+      metadata.filename ??= path.basename(location);
+      metadata.contentEncoding ??= input.readableEncoding!;
+      metadata.size ??= (await fs.stat(location)).size;
+      metadata.hash ??= await this.hash(createReadStream(location), { hashAlgorithm: 'sha256' });
+    } else if (input && typeof input === 'object' && 'readableEncoding' in input && typeof input.readableEncoding === 'string') {
+      metadata.contentEncoding ??= input.readableEncoding!;
+    }
+
+    return metadata;
+  }
+
+  /**
+   * Rewrite a blob to support metadata, and provide a dynamic input source
+   */
+  static defineBlob<T extends Blob>(target: T, input: BlobInput, metadata: BinaryMetadata = {}): typeof target {
+    const inputFn = async (): Promise<BinaryType> => typeof input === 'function' ? await input() : input;
+    this.write(target, metadata);
+
+    Object.defineProperties(target, {
+      size: { get() { return BinaryMetadataUtil.readLength(metadata); } },
+      type: { get() { return metadata.contentType; } },
+      name: { get() { return metadata.filename; } },
+      arrayBuffer: { value: () => inputFn().then(BinaryUtil.toArrayBuffer) },
+      stream: { value: () => BinaryUtil.toReadableStream(BinaryUtil.toSynchronous(input)) },
+      bytes: { value: () => inputFn().then(BinaryUtil.toBuffer) },
+      text: { value: () => inputFn().then(BinaryUtil.toBinaryArray).then(CodecUtil.toUTF8String) },
+      slice: {
+        value: (start?: number, end?: number, _contentType?: string) => {
+          const result = target instanceof File ? new File([], '') : new Blob([]);
+          return BinaryMetadataUtil.defineBlob(result,
+            () => inputFn().then(BinaryUtil.toBinaryArray).then(data => BinaryUtil.sliceByteArray(data, start, end)),
+            {
+              ...metadata,
+              range: { start: start ?? 0, end: end ?? metadata.size! - 1 },
+            }
+          );
+        }
+      }
+    });
+    return target;
+  }
+
+  /**
+   * Make a blob that contains the appropriate metadata
+   */
+  static makeBlob(source: BlobInput, metadata?: BinaryMetadata): Blob {
+    return this.defineBlob(new Blob([]), source, metadata);
+  }
+
+  /**
+   * Enforce byte range for stream stream/file of a certain size
+   */
+  static enforceRange(range: ByteRange, metadata?: BinaryMetadata): Required<ByteRange> {
+    if (!metadata || metadata.size === undefined) {
+      throw new RuntimeError('Cannot enforce range on data with unknown size', { category: 'data' });
+    }
+    const size = metadata.size;
+
+    // End is inclusive
+    const [start, end] = [range.start, Math.min(range.end ?? (size - 1), size - 1)];
+
+    if (Number.isNaN(start) || Number.isNaN(end) || !Number.isFinite(start) || start >= size || start < 0 || start > end) {
+      throw new RuntimeError('Invalid position, out of range', { category: 'data', details: { start, end, size } });
+    }
+
+    return { start, end };
+  }
+}

package/src/binary.ts

@@ -1,132 +1,193 @@
-import path from 'node:path';
-import os from 'node:os';
-import crypto from 'node:crypto';
-import fs from 'node:fs/promises';
-import { PassThrough, Readable } from 'node:stream';
+import { PassThrough, Readable, type Writable } from 'node:stream';
 import { pipeline } from 'node:stream/promises';
 import { ReadableStream } from 'node:stream/web';
-import { text as toText, arrayBuffer as toArrayBuffer } from 'node:stream/consumers';
-import { isArrayBuffer } from 'node:util/types';
+import consumers from 'node:stream/consumers';
+import { isArrayBuffer, isPromise, isTypedArray, isUint16Array, isUint32Array, isUint8Array, isUint8ClampedArray } from 'node:util/types';
 
-import { type BinaryInput, type BlobMeta, hasFunction } from './types.ts';
-import { Util } from './util.ts';
+import { castTo, hasFunction, toConcrete } from './types.ts';
 
-const BlobMetaSymbol = Symbol();
+/**
+ * Binary Array
+ * @concrete
+ */
+export type BinaryArray = Uint32Array | Uint16Array | Uint8Array | Uint8ClampedArray | Buffer<ArrayBuffer> | ArrayBuffer;
+/**
+ * Binary Stream
+ * @concrete
+ */
+export type BinaryStream = Readable | ReadableStream | AsyncIterable<BinaryArray>;
+/**
+ * Binary Container
+ * @concrete
+ */
+export type BinaryContainer = Blob | File;
+/**
+ * Binary Type
+ * @concrete
+ */
+export type BinaryType = BinaryArray | BinaryStream | BinaryContainer;
+
+const BINARY_CONSTRUCTOR_SET = new Set<unknown>([
+  Readable, Buffer, Blob, ReadableStream, ArrayBuffer, Uint8Array,
+  Uint16Array, Uint32Array, Uint8ClampedArray, File
+]);
+
+let BINARY_REFS: Set<unknown> | undefined;
+const isBinaryTypeReference = (value: unknown): boolean =>
+  BINARY_CONSTRUCTOR_SET.has(value) ||
+  BINARY_CONSTRUCTOR_SET.has(Object.getPrototypeOf(value)) ||
+  (BINARY_REFS ||= new Set<unknown>([
+    toConcrete<BinaryType>(),
+    toConcrete<BinaryStream>(),
+    toConcrete<BinaryArray>(),
+    toConcrete<BinaryContainer>(),
+  ])).has(value);
+
+const isReadable = hasFunction<Readable>('pipe');
+const isReadableStream = hasFunction<ReadableStream>('pipeTo');
+const isAsyncIterable = (value: unknown): value is AsyncIterable<unknown> =>
+  !!value && (typeof value === 'object' || typeof value === 'function') && Symbol.asyncIterator in value;
+const isBinaryArray = (value: unknown): value is BinaryArray =>
+  isUint8Array(value) || isArrayBuffer(value) || isUint16Array(value) || isUint32Array(value) || isUint8ClampedArray(value);
+const isBinaryStream = (value: unknown): value is BinaryStream => isReadable(value) || isReadableStream(value) || isAsyncIterable(value);
+const isBinaryContainer = (value: unknown): value is BinaryContainer => value instanceof Blob;
+const isBinaryType = (value: unknown): value is BinaryType => !!value && (isBinaryArray(value) || isBinaryStream(value) || isBinaryContainer(value));
 
 /**
  * Common functions for dealing with binary data/streams
  */
 export class BinaryUtil {
-  /** Is Array Buffer */
-  static isArrayBuffer = isArrayBuffer;
-  /** Is Readable */
-  static isReadable = hasFunction<Readable>('pipe');
-  /** Is ReadableStream */
-  static isReadableStream = hasFunction<ReadableStream>('pipeTo');
-  /** Is Async Iterable */
-  static isAsyncIterable = (value: unknown): value is AsyncIterable<unknown> =>
-    !!value && (typeof value === 'object' || typeof value === 'function') && Symbol.asyncIterator in value;
 
-  /**
-   * Is value a binary type
-   */
-  static isBinaryType(value: unknown): boolean {
-    return value instanceof Blob || Buffer.isBuffer(value) || this.isReadable(value) ||
-      this.isArrayBuffer(value) || this.isReadableStream(value) || this.isAsyncIterable(value);
+  /** Is the input a byte array */
+  static isBinaryArray = isBinaryArray;
+  /** Is the input a byte stream */
+  static isBinaryStream = isBinaryStream;
+  /** Is the input a binary container */
+  static isBinaryContainer = isBinaryContainer;
+  /** Is value a binary type  */
+  static isBinaryType = isBinaryType;
+  /** Is a binary reference */
+  static isBinaryTypeReference = isBinaryTypeReference;
+
+  /** Convert binary array to an explicit buffer  */
+  static binaryArrayToBuffer(input: BinaryArray): Buffer<ArrayBuffer> {
+    if (Buffer.isBuffer(input)) {
+      return castTo(input);
+    } else if (isTypedArray(input)) {
+      return castTo(Buffer.from(input.buffer));
+    } else {
+      return Buffer.from(input);
+    }
   }
 
-  /**
-   * Generate a proper sha512 hash from an input value
-   * @param input The seed value to build the hash from
-   * @param length The optional length of the hash to generate
-   */
-  static hash(input: string, length: number = -1): string {
-    const hash = crypto.createHash('sha512');
-    hash.update(input);
-    const digest = hash.digest('hex');
-    return length > 0 ? digest.substring(0, length) : digest;
+  /** Convert binary array to an explicit uint8array  */
+  static binaryArrayToUint8Array(input: BinaryArray): Uint8Array {
+    if (isUint8Array(input)) {
+      return castTo(input);
+    } else if (isTypedArray(input)) {
+      return castTo(Buffer.from(input.buffer));
+    } else {
+      return Buffer.from(input);
+    }
   }
 
-  /**
-   * Compute hash from an input blob, buffer or readable stream.
-   */
-  static async hashInput(input: BinaryInput): Promise<string> {
-    const hash = crypto.createHash('sha256').setEncoding('hex');
-    if (Buffer.isBuffer(input)) {
-      hash.write(input);
-    } else if (input instanceof Blob) {
-      await pipeline(Readable.fromWeb(input.stream()), hash);
+  /** Convert input to a binary array  */
+  static async toBinaryArray(input: BinaryType): Promise<BinaryArray> {
+    if (isBinaryArray(input)) {
+      return input;
+    } else if (isBinaryStream(input)) {
+      return consumers.buffer(input);
     } else {
-      await pipeline(input, hash);
+      return input.arrayBuffer();
     }
-    return hash.digest('hex').toString();
   }
 
-  /**
-   * Write file and copy over when ready
-   */
-  static async bufferedFileWrite(file: string, content: string, checkHash = false): Promise<void> {
-    if (checkHash) {
-      const current = await fs.readFile(file, 'utf8').catch(() => '');
-      if (this.hash(current) === this.hash(content)) {
-        return;
-      }
+  /** Convert input to a buffer  */
+  static async toBuffer(input: BinaryType): Promise<Buffer<ArrayBuffer>> {
+    const bytes = await BinaryUtil.toBinaryArray(input);
+    return BinaryUtil.binaryArrayToBuffer(bytes);
+  }
+
+  /** Convert input to an ArrayBuffer  */
+  static async toArrayBuffer(input: BinaryType): Promise<ArrayBuffer> {
+    const data = await BinaryUtil.toBuffer(input);
+    return data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength);
+  }
+
+  /** Convert input to a readable stream  */
+  static toReadable(input: BinaryType): Readable {
+    if (isReadable(input)) {
+      return input;
+    } else if (isBinaryArray(input)) {
+      return Readable.from(BinaryUtil.binaryArrayToBuffer(input));
+    } else if (isBinaryContainer(input)) {
+      return Readable.fromWeb(input.stream());
+    } else if (isReadableStream(input)) {
+      return Readable.fromWeb(input);
+    } else {
+      return Readable.from(input);
     }
+  }
 
-    const temp = path.resolve(os.tmpdir(), `${process.hrtime()[1]}.${path.basename(file)}`);
-    await fs.writeFile(temp, content, 'utf8');
-    await fs.mkdir(path.dirname(file), { recursive: true });
-    await fs.copyFile(temp, file);
-    await fs.rm(temp, { force: true });
+  /** Convert input to a binary ReadableStream  */
+  static toReadableStream(input: BinaryType): ReadableStream {
+    if (isReadableStream(input)) {
+      return input;
+    } else if (isReadable(input)) {
+      return Readable.toWeb(input);
+    } else if (isBinaryContainer(input)) {
+      return input.stream();
+    } else {
+      return Readable.toWeb(BinaryUtil.toReadable(input));
+    }
   }
 
-  /**
-   * Make a blob, and assign metadata
-   */
-  static readableBlob(input: () => (Readable | Promise<Readable>), metadata: Omit<BlobMeta, 'filename'> & { filename: string }): File;
-  static readableBlob(input: () => (Readable | Promise<Readable>), metadata?: BlobMeta): Blob;
-  static readableBlob(input: () => (Readable | Promise<Readable>), metadata: BlobMeta = {}): Blob | File {
-    const go = (): Readable => {
-      const stream = new PassThrough();
-      Promise.resolve(input()).then(readable => readable.pipe(stream), (error) => stream.destroy(error));
-      return stream;
-    };
-
-    const size = metadata.range ? (metadata.range.end - metadata.range.start) + 1 : metadata.size;
-    const out: Blob = metadata.filename ?
-      new File([], path.basename(metadata.filename), { type: metadata.contentType }) :
-      new Blob([], { type: metadata.contentType });
-
-    return Object.defineProperties(out, {
-      size: { value: size },
-      stream: { value: () => ReadableStream.from(go()) },
-      arrayBuffer: { value: () => toArrayBuffer(go()) },
-      text: { value: () => toText(go()) },
-      bytes: { value: () => toArrayBuffer(go()).then(buffer => new Uint8Array(buffer)) },
-      [BlobMetaSymbol]: { value: metadata }
-    });
+  /** Convert input to a binary stream  */
+  static toBinaryStream(input: BinaryType): BinaryStream {
+    if (isBinaryStream(input)) {
+      return input;
+    } else {
+      return BinaryUtil.toReadableStream(input);
+    }
   }
 
-  /**
-   * Get blob metadata
-   */
-  static getBlobMeta(blob: Blob): BlobMeta | undefined {
-    const withMeta: Blob & { [BlobMetaSymbol]?: BlobMeta } = blob;
-    return withMeta[BlobMetaSymbol];
+  /** Combine binary arrays  */
+  static combineBinaryArrays(arrays: BinaryArray[]): BinaryArray {
+    return Buffer.concat(arrays.map(x => BinaryUtil.binaryArrayToBuffer(x)));
+  }
+
+  /** Agnostic slice of binary array  */
+  static sliceByteArray(input: BinaryArray, start?: number, end?: number): BinaryArray {
+    if (Buffer.isBuffer(input)) {
+      return input.subarray(start, end);
+    } else if (isArrayBuffer(input)) {
+      return input.slice(start, end);
+    } else {
+      return input.slice(start, end);
+    }
+  }
+
+  /** Consume input into output  */
+  static pipeline(input: BinaryType, output: Writable): Promise<void> {
+    return pipeline(BinaryUtil.toBinaryStream(input), output);
+  }
+
+  /** Create a binary array of specified size, optionally filled with a value */
+  static makeBinaryArray(size: number, fill?: string | number): BinaryArray {
+    return Buffer.alloc(size, fill);
   }
 
   /**
-   * Get a hashed location/path for a blob
+   * Convert an inbound binary type or factory into a synchronous binary type
    */
-  static hashedBlobLocation(meta: BlobMeta): string {
-    const hash = meta.hash ?? Util.uuid();
-
-    let parts = hash.match(/(.{1,4})/g)!.slice();
-    if (parts.length > 4) {
-      parts = [...parts.slice(0, 4), parts.slice(4).join('')];
+  static toSynchronous(input: BinaryType | (() => (BinaryType | Promise<BinaryType>))): BinaryType {
+    const value = (typeof input === 'function') ? input() : input;
+    if (isPromise(value)) {
+      const stream = new PassThrough();
+      value.then(result => BinaryUtil.pipeline(result, stream)).catch(error => stream.destroy(error));
+      return stream;
+    } else {
+      return value;
     }
-
-    const ext = path.extname(meta.filename ?? '') || '.bin';
-    return `${parts.join('/')}${ext}`;
   }
 }