@forwardimpact/libutil 0.1.60

package/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to the Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Dick Olsson
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/bin/fit-download-bundle.js

@@ -0,0 +1,24 @@
+#!/usr/bin/env node
+
+import { createScriptConfig } from "@forwardimpact/libconfig";
+import { createStorage } from "@forwardimpact/libstorage";
+import { createBundleDownloader, execLine } from "@forwardimpact/libutil";
+
+/**
+ * Downloads generated code bundle from remote storage.
+ * Used in containerized deployments to fetch pre-generated code.
+ * @returns {Promise<void>}
+ */
+async function main() {
+  await createScriptConfig("download-bundle");
+  const downloader = await createBundleDownloader(createStorage);
+  await downloader.download();
+
+  // If additional arguments provided, execute them after download
+  execLine();
+}
+
+main().catch((error) => {
+  console.error("Bundle download failed:", error);
+  process.exit(1);
+});

package/downloader.js

@@ -0,0 +1,108 @@
+/**
+ * BundleDownloader utility for retrieving and extracting bundle.tar.gz from remote storage.
+ * Used to download generated code bundles in containerized deployments.
+ */
+export class BundleDownloader {
+  #storageFactory;
+  #finder;
+  #logger;
+  #extractor;
+  #process;
+  #local;
+  #remote;
+  #initialized;
+
+  /**
+   * Creates a new download instance with dependency injection
+   * @param {Function} createStorageFn - Storage creation function
+   * @param {object} finder - Finder instance for symlink management
+   * @param {object} logger - Logger instance
+   * @param {object} extractor - TarExtractor instance for archive extraction
+   * @param {object} process - Process environment access (for testing)
+   */
+  constructor(
+    createStorageFn,
+    finder,
+    logger,
+    extractor,
+    process = global.process,
+  ) {
+    if (!createStorageFn) throw new Error("createStorageFn is required");
+    if (!finder) throw new Error("finder is required");
+    if (!logger) throw new Error("logger is required");
+    if (!extractor) throw new Error("extractor is required");
+    if (!process) throw new Error("process is required");
+
+    this.#storageFactory = createStorageFn;
+    this.#finder = finder;
+    this.#logger = logger;
+    this.#extractor = extractor;
+    this.#process = process;
+    this.#local = null;
+    this.#remote = null;
+    this.#initialized = false;
+  }
+
+  /**
+   * Initialize storage instances for download operations
+   * @returns {Promise<void>}
+   */
+  async initialize() {
+    // Initialize storage instances for the "generated" prefix
+    this.#local = this.#storageFactory("generated", "local", this.#process);
+    this.#remote = this.#storageFactory("generated", "s3", this.#process);
+
+    // Ensure directory and create symlinks for packages
+    await this.#local.ensureBucket();
+    const generatedPath = this.#local.path();
+    await this.#finder.createPackageSymlinks(generatedPath);
+
+    this.#initialized = true;
+  }
+
+  /**
+   * Download bundle.tar.gz from remote storage and extract to local storage
+   * Only downloads if STORAGE_TYPE is "s3"
+   * Automatically initializes storage instances if not already initialized
+   * @returns {Promise<void>}
+   */
+  async download() {
+    // Auto-initialize if not already initialized
+    if (!this.#initialized) await this.initialize();
+
+    const storageType = this.#process.env.STORAGE_TYPE || "local";
+
+    if (storageType === "local") {
+      this.#logger.debug(
+        "BundleDownloader",
+        "Download skipped, using local storage",
+      );
+      return;
+    }
+
+    const key = "bundle.tar.gz";
+
+    // Check if bundle exists in remote storage
+    const exists = await this.#remote.exists(key);
+    if (!exists) throw new Error(`Bundle not found`);
+
+    const data = await this.#remote.get(key);
+    await this.#local.put(key, data);
+    await this.#extractBundle(key);
+    await this.#local.delete(key);
+
+    this.#logger.debug("BundleDownloader", "Download completed", { key });
+  }
+
+  /**
+   * Extract bundle.tar.gz to local storage using TarExtractor
+   * @param {string} key - Bundle file key in local storage
+   * @returns {Promise<void>}
+   * @private
+   */
+  async #extractBundle(key) {
+    const path = this.#local.path(key);
+    const dir = this.#local.path();
+    await this.#extractor.extract(path, dir);
+  }
+}

package/extractor.js

@@ -0,0 +1,330 @@
+import { createReadStream } from "node:fs";
+import { createGunzip, createInflateRaw } from "node:zlib";
+
+/**
+ * Base extractor class with shared functionality
+ * @private
+ */
+class BaseExtractor {
+  #fs;
+  #path;
+
+  /**
+   * Creates a new extractor with dependency injection
+   * @param {object} fs - File system module (fs/promises)
+   * @param {object} path - Path module
+   */
+  constructor(fs, path) {
+    if (!fs) throw new Error("fs dependency is required");
+    if (!path) throw new Error("path dependency is required");
+
+    this.#fs = fs;
+    this.#path = path;
+  }
+
+  /**
+   * Check if file should be skipped (macOS resource forks, PAX headers)
+   * @param {string} name - File name
+   * @returns {boolean} True if file should be skipped
+   * @private
+   */
+  _shouldSkipFile(name) {
+    return name.includes("/._") || name.startsWith("._");
+  }
+
+  /**
+   * Create directory with proper mode
+   * @param {string} dirPath - Directory path
+   * @param {number} [mode] - Directory mode (permissions)
+   * @returns {Promise<void>}
+   * @private
+   */
+  async _createDirectory(dirPath, mode) {
+    await this.#fs.mkdir(dirPath, { recursive: true, mode });
+  }
+
+  /**
+   * Write file with proper mode
+   * @param {string} filePath - File path
+   * @param {Buffer} content - File content
+   * @param {number} [mode] - File mode (permissions)
+   * @returns {Promise<void>}
+   * @private
+   */
+  async _writeFile(filePath, content, mode) {
+    await this.#fs.mkdir(this.#path.dirname(filePath), { recursive: true });
+    await this.#fs.writeFile(filePath, content, { mode });
+  }
+
+  /**
+   * Join paths safely
+   * @param {...string} paths - Path segments to join
+   * @returns {string} Joined path
+   * @private
+   */
+  _joinPath(...paths) {
+    return this.#path.join(...paths);
+  }
+
+  /**
+   * Read 16-bit little-endian unsigned integer from buffer
+   * @param {Buffer} buffer - Source buffer
+   * @param {number} offset - Offset position
+   * @returns {number} Unsigned 16-bit integer
+   * @private
+   */
+  _readUInt16LE(buffer, offset) {
+    return buffer.readUInt16LE(offset);
+  }
+
+  /**
+   * Read 32-bit little-endian unsigned integer from buffer
+   * @param {Buffer} buffer - Source buffer
+   * @param {number} offset - Offset position
+   * @returns {number} Unsigned 32-bit integer
+   * @private
+   */
+  _readUInt32LE(buffer, offset) {
+    return buffer.readUInt32LE(offset);
+  }
+
+  /**
+   * Read null-terminated string from buffer
+   * @param {Buffer} buffer - Source buffer
+   * @param {number} start - Start position
+   * @param {number} length - Maximum length to read
+   * @returns {string} Extracted string
+   * @private
+   */
+  _readString(buffer, start, length) {
+    const slice = buffer.slice(start, start + length);
+    const nullIndex = slice.indexOf(0);
+    return slice
+      .slice(0, nullIndex > -1 ? nullIndex : length)
+      .toString("utf-8");
+  }
+}
+
+/**
+ * Simple TAR file extractor for .tar.gz files
+ * Implements native Node.js TAR format parsing without external dependencies
+ */
+export class TarExtractor extends BaseExtractor {
+  /**
+   * Extract a .tar.gz file to specified directory
+   * @param {string} tarGzPath - Path to the .tar.gz file
+   * @param {string} outputDir - Directory to extract files to
+   * @returns {Promise<void>}
+   */
+  async extract(tarGzPath, outputDir) {
+    // Decompress and collect chunks
+    const chunks = [];
+    const input = createReadStream(tarGzPath).pipe(createGunzip());
+    for await (const chunk of input) chunks.push(chunk);
+
+    // Parse TAR format
+    const buffer = Buffer.concat(chunks);
+    for (let offset = 0; offset < buffer.length; ) {
+      const header = buffer.slice(offset, offset + 512);
+
+      // Check for end of archive (zero block)
+      if (header.every((byte) => byte === 0)) break;
+
+      // Parse header fields
+      const name = this._readString(header, 0, 100);
+      const mode = parseInt(this._readString(header, 100, 8), 8);
+      const size = parseInt(this._readString(header, 124, 12), 8);
+      const typeFlag = header[156];
+
+      offset += 512;
+
+      if (size > 0) {
+        const blockSize = Math.ceil(size / 512) * 512;
+
+        // Skip unwanted files (PAX headers, macOS resource forks)
+        if (this._shouldSkipFile(name) || typeFlag === 120 || typeFlag === 88) {
+          offset += blockSize;
+          continue;
+        }
+
+        const content = buffer.slice(offset, offset + size);
+
+        if (typeFlag === 53) {
+          // Directory
+          await this._createDirectory(this._joinPath(outputDir, name), mode);
+        } else {
+          // Regular file
+          await this._writeFile(this._joinPath(outputDir, name), content, mode);
+        }
+
+        offset += blockSize;
+      }
+    }
+  }
+}
+
+/**
+ * Simple ZIP file extractor for .zip files
+ * Implements native Node.js ZIP format parsing without external dependencies
+ */
+export class ZipExtractor extends BaseExtractor {
+  /**
+   * Extract a .zip file to specified directory
+   * @param {string} zipPath - Path to the .zip file
+   * @param {string} outputDir - Directory to extract files to
+   * @returns {Promise<void>}
+   */
+  async extract(zipPath, outputDir) {
+    // Read entire ZIP file into buffer
+    const chunks = [];
+    const input = createReadStream(zipPath);
+    for await (const chunk of input) chunks.push(chunk);
+    const buffer = Buffer.concat(chunks);
+
+    // Find End of Central Directory Record (EOCD)
+    const eocdOffset = this.#findEOCD(buffer);
+    if (eocdOffset === -1) {
+      throw new Error("Invalid ZIP file: End of Central Directory not found");
+    }
+
+    // Parse EOCD to get central directory location
+    const centralDirOffset = this._readUInt32LE(buffer, eocdOffset + 16);
+    const entryCount = this._readUInt16LE(buffer, eocdOffset + 10);
+
+    // Parse central directory entries
+    let offset = centralDirOffset;
+    for (let i = 0; i < entryCount; i++) {
+      // Verify central directory file header signature (0x02014b50)
+      if (this._readUInt32LE(buffer, offset) !== 0x02014b50) {
+        throw new Error("Invalid central directory file header");
+      }
+
+      // Parse central directory header fields
+      const compressionMethod = this._readUInt16LE(buffer, offset + 10);
+      const compressedSize = this._readUInt32LE(buffer, offset + 20);
+      const fileNameLength = this._readUInt16LE(buffer, offset + 28);
+      const extraFieldLength = this._readUInt16LE(buffer, offset + 30);
+      const fileCommentLength = this._readUInt16LE(buffer, offset + 32);
+      const externalAttrs = this._readUInt32LE(buffer, offset + 38);
+      const localHeaderOffset = this._readUInt32LE(buffer, offset + 42);
+
+      // Extract file name
+      const fileName = buffer
+        .slice(offset + 46, offset + 46 + fileNameLength)
+        .toString("utf-8");
+
+      // Skip to next entry
+      offset += 46 + fileNameLength + extraFieldLength + fileCommentLength;
+
+      // Skip unwanted files
+      if (this._shouldSkipFile(fileName)) continue;
+
+      // Check if it's a directory (trailing slash or directory bit set)
+      const isDirectory =
+        fileName.endsWith("/") || (externalAttrs & 0x10) === 0x10;
+
+      if (isDirectory) {
+        // Extract Unix permissions from external attributes (high 16 bits)
+        const mode = (externalAttrs >>> 16) & 0o777;
+        await this._createDirectory(
+          this._joinPath(outputDir, fileName),
+          mode || 0o755,
+        );
+      } else {
+        // Extract file content from local file header
+        const content = await this.#extractFileContent(
+          buffer,
+          localHeaderOffset,
+          compressionMethod,
+          compressedSize,
+        );
+
+        // Extract Unix permissions from external attributes (high 16 bits)
+        const mode = (externalAttrs >>> 16) & 0o777;
+        await this._writeFile(
+          this._joinPath(outputDir, fileName),
+          content,
+          mode || 0o644,
+        );
+      }
+    }
+  }
+
+  /**
+   * Find End of Central Directory Record
+   * @param {Buffer} buffer - ZIP file buffer
+   * @returns {number} Offset of EOCD or -1 if not found
+   * @private
+   */
+  #findEOCD(buffer) {
+    // EOCD signature is 0x06054b50
+    // Search from end of file backwards (EOCD is at the end)
+    const signature = 0x06054b50;
+    for (let i = buffer.length - 22; i >= 0; i--) {
+      if (this._readUInt32LE(buffer, i) === signature) {
+        return i;
+      }
+    }
+    return -1;
+  }
+
+  /**
+   * Extract file content from local file header
+   * @param {Buffer} buffer - ZIP file buffer
+   * @param {number} offset - Local file header offset
+   * @param {number} compressionMethod - Compression method (0=store, 8=deflate)
+   * @param {number} compressedSize - Compressed size
+   * @returns {Promise<Buffer>} File content
+   * @private
+   */
+  async #extractFileContent(buffer, offset, compressionMethod, compressedSize) {
+    // Verify local file header signature (0x04034b50)
+    if (this._readUInt32LE(buffer, offset) !== 0x04034b50) {
+      throw new Error("Invalid local file header");
+    }
+
+    // Get file name and extra field lengths from local header
+    const fileNameLength = this._readUInt16LE(buffer, offset + 26);
+    const extraFieldLength = this._readUInt16LE(buffer, offset + 28);
+
+    // Calculate data offset (skip header, file name, and extra field)
+    const dataOffset = offset + 30 + fileNameLength + extraFieldLength;
+
+    // Extract compressed data
+    const compressedData = buffer.slice(
+      dataOffset,
+      dataOffset + compressedSize,
+    );
+
+    // Decompress based on compression method
+    if (compressionMethod === 0) {
+      // Stored (no compression)
+      return compressedData;
+    } else if (compressionMethod === 8) {
+      // DEFLATE compression
+      return await this.#inflateData(compressedData);
+    } else {
+      throw new Error(`Unsupported compression method: ${compressionMethod}`);
+    }
+  }
+
+  /**
+   * Inflate compressed data using DEFLATE
+   * @param {Buffer} compressedData - Compressed data
+   * @returns {Promise<Buffer>} Decompressed data
+   * @private
+   */
+  async #inflateData(compressedData) {
+    const chunks = [];
+    const inflate = createInflateRaw();
+
+    inflate.write(compressedData);
+    inflate.end();
+
+    for await (const chunk of inflate) {
+      chunks.push(chunk);
+    }
+
+    return Buffer.concat(chunks);
+  }
+}