@travetto/runtime 7.1.3 → 8.0.0-alpha.0

package/README.md

@@ -19,6 +19,8 @@ Runtime is the foundation of all [Travetto](https://travetto.dev) applications.
    *  Standard Error Support
    *  Console Management
    *  Resource Access
+   *  Encoding and Decoding Utilities
+   *  Binary Utilities
    *  JSON Utilities
    *  Common Utilities
    *  Time Utilities
@@ -67,6 +69,8 @@ class $Runtime {
   getImport(handle: Function): string;
   /** Import from a given path */
   async importFrom<T = unknown>(location?: string): Promise<T>;
+  /** Get an install command for a given npm module */
+  getInstallCommand(pkg: string, production = false): string;
 }
 ```
 
@@ -161,9 +165,9 @@ export class EnvProp<T> {
 ```
 
 ## Standard Error Support
-While the framework is 100 % compatible with standard `Error` instances, there are cases in which additional functionality is desired. Within the framework we use [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) (or its derivatives) to represent framework errors. This class is available for use in your own projects. Some of the additional benefits of using this class is enhanced error reporting, as well as better integration with other modules (e.g. the [Web API](https://github.com/travetto/travetto/tree/main/module/web#readme "Declarative support for creating Web Applications") module and HTTP status codes). 
+While the framework is 100 % compatible with standard `Error` instances, there are cases in which additional functionality is desired. Within the framework we use [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) (or its derivatives) to represent framework errors. This class is available for use in your own projects. Some of the additional benefits of using this class is enhanced error reporting, as well as better integration with other modules (e.g. the [Web API](https://github.com/travetto/travetto/tree/main/module/web#readme "Declarative support for creating Web Applications") module and HTTP status codes). 
 
-The [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) takes in a message, and an optional payload and / or error classification. The currently supported error classifications are:
+The [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) takes in a message, and an optional payload and / or error classification. The currently supported error classifications are:
    *  `general` - General purpose errors
    *  `system` - Synonym for `general`
    *  `data` - Data format, content, etc are incorrect. Generally correlated to bad input.
@@ -247,25 +251,26 @@ $ DEBUG=express:*,@travetto/web npx trv run web
 ## Resource Access
 The primary access patterns for resources, is to directly request a file, and to resolve that file either via file-system look up or leveraging the [Manifest](https://github.com/travetto/travetto/tree/main/module/manifest#readme "Support for project indexing, manifesting, along with file watching")'s data for what resources were found at manifesting time.
 
-The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L12) allows for accessing information about the resources, and subsequently reading the file as text/binary or to access the resource as a `Readable` stream.  If a file is not found, it will throw an [AppError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L26) with a category of 'notfound'.  
+The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L11) allows for accessing information about the resources, and subsequently reading the file as text/binary or to access the resource as a `Readable` stream.  If a file is not found, it will throw an [RuntimeError](https://github.com/travetto/travetto/tree/main/module/runtime/src/error.ts#L17) with a category of 'notfound'.  
 
-The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L12) also supports tying itself to [Env](https://github.com/travetto/travetto/tree/main/module/runtime/src/env.ts#L114)'s `TRV_RESOURCES` information on where to attempt to find a requested resource.
+The [FileLoader](https://github.com/travetto/travetto/tree/main/module/runtime/src/file-loader.ts#L11) also supports tying itself to [Env](https://github.com/travetto/travetto/tree/main/module/runtime/src/env.ts#L114)'s `TRV_RESOURCES` information on where to attempt to find a requested resource.
 
-## JSON Utilities
-The framework provides utilities for working with JSON data.  This module provides methods for reading and writing JSON files, as well as serializing and deserializing JSON data.  It also provides support for working with Base64 encoded data for web safe transfer.  The primary goal is ease of use, but also a centralized location for performance and security improvements over time.
-
-   *  `parseSafe(input: string | Buffer)` parses JSON safely from a string or Buffer.
-   *  `stringifyBase64(value: any)` encodes a JSON value as a base64 encoded string.
-   *  `parseBase64(input: string)` decodes a JSON value from a base64 encoded string.
-   *  `readFile(file: string)` reads a JSON file asynchronously.
-   *  `readFileSync(file: string, onMissing?: any)` reads a JSON file synchronously.
+## Encoding and Decoding Utilities
+The [CodecUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/codec.ts#L15) class provides a variety of static methods for encoding and decoding data. When working with JSON data, it also provide security checks to prevent prototype pollution. The utility supports the following formats:
+   *  Hex
+   *  Base64
+   *  UTF8
+   *  UTT8 Encoded JSON
+   *  Base64 Encoded JSON
+   *  New Line Delimited UTF8
 
 ## Common Utilities
-Common utilities used throughout the framework. Currently [Util](https://github.com/travetto/travetto/tree/main/module/runtime/src/util.ts#L11) includes:
+Common utilities used throughout the framework. Currently [Util](https://github.com/travetto/travetto/tree/main/module/runtime/src/util.ts#L14) includes:
    *  `uuid(len: number)` generates a simple uuid for use within the application.
    *  `allowDenyMatcher(rules[])` builds a matching function that leverages the rules as an allow/deny list, where order of the rules matters.  Negative rules are prefixed by '!'.
    *  `hash(text: string, size?: number)` produces a full sha512 hash.
    *  `resolvablePromise()` produces a `Promise` instance with the `resolve` and `reject` methods attached to the instance.  This is extremely useful for integrating promises into async iterations, or any other situation in which the promise creation and the execution flow don't always match up.
+   *  `bufferedFileWrite(file:string, content: string)` will write the file, using a temporary buffer file to ensure that the entire file is written before being moved to the final location.  This helps minimize file watch noise when writing files.
 
 **Code: Sample makeTemplate Usage**
 ```typescript
@@ -275,53 +280,51 @@ tpl`{{age:20}} {{name: 'bob'}}</>;
 '**age: 20** **name: bob**'
 ```
 
+## Binary Utilities
+The [BinaryUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/binary.ts#L59) class provides a unified interface for working with binary data across different formats, especially bridging the gap between Node.js specific types (`Buffer`, `Stream`) and Web Standard types (`Blob`, `ArrayBuffer`). The framework leverages this to allow for seamless handling of binary data, regardless of the source.
+
+## JSON Utilities
+The [JSONUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/json.ts#L31) class provides a comprehensive set of utilities for working with JSON data, including serialization, deserialization, encoding, and deep cloning capabilities. The utility handles special types like `Date`, `BigInt`, and `Error` objects seamlessly. Key features include:
+   *  `fromUTF8(input, config?)` - Parse JSON from a UTF-8 string
+   *  `toUTF8(value, config?)` - Serialize a value to JSON string
+   *  `toUTF8Pretty(value)` - Serialize with pretty formatting (2-space indent)
+   *  `fromBinaryArray(input)` - Parse JSON from binary array
+   *  `toBinaryArray(value, config?)` - Serialize to binary array (UTF-8 encoded)
+   *  `toBase64(value)` - Encode JSON as base64 string
+   *  `fromBase64(input)` - Decode JSON from base64 string
+   *  `clone(input, config?)` - Deep clone objects with optional transformations
+   *  `cloneForTransmit(input)` - Clone for transmission with error serialization
+   *  `cloneFromTransmit(input)` - Clone from transmission with type restoration
+
+The `TRANSMIT_REVIVER` automatically restores `Date` objects and `BigInt` values during deserialization, making it ideal for transmitting complex data structures across network boundaries.
+
 ## Time Utilities
-[TimeUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/time.ts#L20) contains general helper methods, created to assist with time-based inputs via environment variables, command line interfaces, and other string-heavy based input.
+[TimeUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/time.ts#L21) contains general helper methods, created to assist with time-based inputs via environment variables, command line interfaces, and other string-heavy based input.
 
 **Code: Time Utilities**
 ```typescript
 export class TimeUtil {
   /**
    * Test to see if a string is valid for relative time
-   * @param val
    */
   static isTimeSpan(value: string): value is TimeSpan;
   /**
-   * Returns time units convert to ms
-   * @param amount Number of units to extend
-   * @param unit Time unit to extend ('ms', 's', 'm', 'h', 'd', 'w', 'y')
-   */
-  static asMillis(amount: Date | number | TimeSpan, unit?: TimeUnit): number;
-  /**
-   * Returns the time converted to seconds
-   * @param date The date to convert
-   */
-  static asSeconds(date: Date | number | TimeSpan, unit?: TimeUnit): number;
-  /**
-   * Returns the time converted to a Date
-   * @param date The date to convert
-   */
-  static asDate(date: Date | number | TimeSpan, unit?: TimeUnit): Date;
-  /**
-   * Resolve time or span to possible time
+   * Exposes the ability to create a duration succinctly
    */
-  static fromValue(value: Date | number | string | undefined): number | undefined;
+  static duration(input: TimeSpan | number | string, outputUnit: TimeUnit): number;
   /**
    * Returns a new date with `amount` units into the future
-   * @param amount Number of units to extend
-   * @param unit Time unit to extend ('ms', 's', 'm', 'h', 'd', 'w', 'y')
    */
-  static fromNow(amount: number | TimeSpan, unit: TimeUnit = 'ms'): Date;
+  static fromNow(input: TimeSpan | number | string): Date;
   /**
    * Returns a pretty timestamp
-   * @param time Time in milliseconds
    */
-  static asClock(time: number): string;
+  static asClock(input: TimeSpan | number | string): string;
 }
 ```
 
 ## Process Execution
-[ExecUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/exec.ts#L42) exposes `getResult` as a means to wrap [child_process](https://nodejs.org/api/child_process.html)'s process object.  This wrapper allows for a promise-based resolution of the subprocess with the ability to capture the stderr/stdout.
+[ExecUtil](https://github.com/travetto/travetto/tree/main/module/runtime/src/exec.ts#L41) exposes `getResult` as a means to wrap [child_process](https://nodejs.org/api/child_process.html)'s process object.  This wrapper allows for a promise-based resolution of the subprocess with the ability to capture the stderr/stdout.
 
 A simple example would be:
 

package/__index__.ts

@@ -1,11 +1,13 @@
 import type { } from './src/global.d.ts';
 import type { } from './src/trv.d.ts';
 export * from './src/binary.ts';
+export * from './src/binary-metadata.ts';
 export * from './src/console.ts';
 export * from './src/context.ts';
 export * from './src/debug.ts';
 export * from './src/error.ts';
 export * from './src/exec.ts';
+export * from './src/codec.ts';
 export * from './src/env.ts';
 export * from './src/file-loader.ts';
 export * from './src/function.ts';
@@ -17,4 +19,4 @@ export * from './src/shutdown.ts';
 export * from './src/time.ts';
 export * from './src/types.ts';
 export * from './src/watch.ts';
-export * from './src/util.ts';
+export * from './src/util.ts';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@travetto/runtime",
-  "version": "7.1.3",
+  "version": "8.0.0-alpha.0",
   "type": "module",
   "description": "Runtime for travetto applications.",
   "keywords": [
@@ -26,12 +26,13 @@
     "directory": "module/runtime"
   },
   "dependencies": {
-    "@travetto/manifest": "^7.1.2",
+    "@travetto/manifest": "^8.0.0-alpha.0",
     "@types/debug": "^4.1.12",
-    "debug": "^4.4.3"
+    "debug": "^4.4.3",
+    "temporal-polyfill-lite": "^0.2.2"
   },
   "peerDependencies": {
-    "@travetto/transformer": "^7.1.2"
+    "@travetto/transformer": "^8.0.0-alpha.0"
   },
   "peerDependenciesMeta": {
     "@travetto/transformer": {

package/src/binary-metadata.ts

@@ -0,0 +1,174 @@
+import crypto from 'node:crypto';
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { createReadStream, ReadStream } from 'node:fs';
+
+import { BinaryUtil, type BinaryArray, type BinaryContainer, type BinaryStream, type BinaryType } from './binary.ts';
+import { RuntimeError } from './error.ts';
+import { CodecUtil } from './codec.ts';
+
+type BlobInput = BinaryType | (() => (BinaryType | Promise<BinaryType>));
+
+/** Range of bytes, inclusive */
+export type ByteRange = { start: number, end?: number };
+
+export interface BinaryMetadata {
+  /** Size of binary data */
+  size?: number;
+  /** Mime type of the content */
+  contentType?: string;
+  /** Hash of binary data contents */
+  hash?: string;
+  /** The original base filename of the file */
+  filename?: string;
+  /** Filenames title, optional for elements like images, audio, videos */
+  title?: string;
+  /** Content encoding */
+  contentEncoding?: string;
+  /** Content language */
+  contentLanguage?: string;
+  /** Cache control */
+  cacheControl?: string;
+  /** Byte range for binary data */
+  range?: Required<ByteRange>;
+}
+
+type HashConfig = {
+  length?: number;
+  hashAlgorithm?: 'sha1' | 'sha256' | 'sha512' | 'md5';
+  outputEncoding?: crypto.BinaryToTextEncoding;
+};
+
+const BinaryMetaSymbol = Symbol();
+
+export class BinaryMetadataUtil {
+  /** Set metadata for a binary type  */
+  static write(input: BinaryType, metadata: BinaryMetadata): BinaryMetadata {
+    const withMeta: BinaryType & { [BinaryMetaSymbol]?: BinaryMetadata } = input;
+    return withMeta[BinaryMetaSymbol] = metadata;
+  }
+
+  /** Read metadata for a binary type, if available  */
+  static read(input: BinaryType): BinaryMetadata {
+    const withMeta: BinaryType & { [BinaryMetaSymbol]?: BinaryMetadata } = input;
+    return withMeta[BinaryMetaSymbol] ?? {};
+  }
+
+  /** Generate a hash from an input value  * @param input The seed value to build the hash from
+   * @param length The optional length of the hash to generate
+   * @param hashAlgorithm The hash algorithm to use
+   * @param outputEncoding The output encoding format
+   */
+  static hash(input: string | BinaryArray, config?: HashConfig): string;
+  static hash(input: BinaryStream | BinaryContainer, config?: HashConfig): Promise<string>;
+  static hash(input: string | BinaryType, config?: HashConfig): string | Promise<string> {
+    const hashAlgorithm = config?.hashAlgorithm ?? 'sha512';
+    const outputEncoding = config?.outputEncoding ?? 'hex';
+    const length = config?.length;
+    const hash = crypto.createHash(hashAlgorithm).setEncoding(outputEncoding);
+
+    if (typeof input === 'string') {
+      input = CodecUtil.fromUTF8String(input);
+    }
+
+    if (BinaryUtil.isBinaryArray(input)) {
+      hash.update(BinaryUtil.binaryArrayToUint8Array(input));
+      return hash.digest(outputEncoding).substring(0, length);
+    } else {
+      return BinaryUtil.pipeline(input, hash).then(() =>
+        hash.digest(outputEncoding).substring(0, length)
+      );
+    }
+  }
+
+  /** Compute the length of the binary data to be returned */
+  static readLength(metadata: BinaryMetadata): number | undefined {
+    return metadata.range ? (metadata.range.end - metadata.range.start + 1) : metadata.size;
+  }
+
+  /** Compute metadata for a given binary input */
+  static async compute(input: BlobInput, base: BinaryMetadata = {}): Promise<BinaryMetadata> {
+    if (typeof input === 'function') {
+      input = await input();
+    }
+    const metadata = { ...BinaryMetadataUtil.read(input), ...base };
+
+    if (BinaryUtil.isBinaryContainer(input)) {
+      metadata.size ??= input.size;
+      metadata.contentType ??= input.type;
+      if (input instanceof File) {
+        metadata.filename ??= input.name;
+      }
+    } else if (BinaryUtil.isBinaryArray(input)) {
+      metadata.size ??= input.byteLength;
+      metadata.hash ??= this.hash(input, { hashAlgorithm: 'sha256' });
+    } else if (input instanceof ReadStream) {
+      const location = input.path.toString();
+      metadata.filename ??= path.basename(location);
+      metadata.contentEncoding ??= input.readableEncoding!;
+      metadata.size ??= (await fs.stat(location)).size;
+      metadata.hash ??= await this.hash(createReadStream(location), { hashAlgorithm: 'sha256' });
+    } else if (input && typeof input === 'object' && 'readableEncoding' in input && typeof input.readableEncoding === 'string') {
+      metadata.contentEncoding ??= input.readableEncoding!;
+    }
+
+    return metadata;
+  }
+
+  /**
+   * Rewrite a blob to support metadata, and provide a dynamic input source
+   */
+  static defineBlob<T extends Blob>(target: T, input: BlobInput, metadata: BinaryMetadata = {}): typeof target {
+    const inputFn = async (): Promise<BinaryType> => typeof input === 'function' ? await input() : input;
+    this.write(target, metadata);
+
+    Object.defineProperties(target, {
+      size: { get() { return BinaryMetadataUtil.readLength(metadata); } },
+      type: { get() { return metadata.contentType; } },
+      name: { get() { return metadata.filename; } },
+      arrayBuffer: { value: () => inputFn().then(BinaryUtil.toArrayBuffer) },
+      stream: { value: () => BinaryUtil.toReadableStream(BinaryUtil.toSynchronous(input)) },
+      bytes: { value: () => inputFn().then(BinaryUtil.toBuffer) },
+      text: { value: () => inputFn().then(BinaryUtil.toBinaryArray).then(CodecUtil.toUTF8String) },
+      slice: {
+        value: (start?: number, end?: number, _contentType?: string) => {
+          const result = target instanceof File ? new File([], '') : new Blob([]);
+          return BinaryMetadataUtil.defineBlob(result,
+            () => inputFn().then(BinaryUtil.toBinaryArray).then(data => BinaryUtil.sliceByteArray(data, start, end)),
+            {
+              ...metadata,
+              range: { start: start ?? 0, end: end ?? metadata.size! - 1 },
+            }
+          );
+        }
+      }
+    });
+    return target;
+  }
+
+  /**
+   * Make a blob that contains the appropriate metadata
+   */
+  static makeBlob(source: BlobInput, metadata?: BinaryMetadata): Blob {
+    return this.defineBlob(new Blob([]), source, metadata);
+  }
+
+  /**
+   * Enforce byte range for stream stream/file of a certain size
+   */
+  static enforceRange(range: ByteRange, metadata?: BinaryMetadata): Required<ByteRange> {
+    if (!metadata || metadata.size === undefined) {
+      throw new RuntimeError('Cannot enforce range on data with unknown size', { category: 'data' });
+    }
+    const size = metadata.size;
+
+    // End is inclusive
+    const [start, end] = [range.start, Math.min(range.end ?? (size - 1), size - 1)];
+
+    if (Number.isNaN(start) || Number.isNaN(end) || !Number.isFinite(start) || start >= size || start < 0 || start > end) {
+      throw new RuntimeError('Invalid position, out of range', { category: 'data', details: { start, end, size } });
+    }
+
+    return { start, end };
+  }
+}