@forzalabs/remora 2.2.0 → 2.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,14 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
6
6
 
7
7
  ## Unreleased
8
8
 
9
+ ## V 2.2.1 - 2026-10-15
10
+
11
+ ### Fixed
12
+
13
+ - **A compressed source file that could not be read failed the run with `unexpected end of file` and nothing else.** zlib reports WHAT went wrong and never WHICH file it was reading, and nothing between it and the CLI added the name back, so a producer staging hundreds of files through a `%` fileKey died with a six-word string that named no file, no producer and no consumer stage — the one thing needed to go and look at it. Worse, that one string covers three different mistakes: a truncated archive, a **zero-byte** file, and (as `incorrect header check`) a file that is not compressed at all. The first two are what a drop folder produces constantly, since a file being written into one exists at zero bytes before it holds anything. Every decompression failure is now re-thrown naming the file and the declared format — `Failed to decompress "/data/claims/claimline_2026_03.csv.gz" as GZ: unexpected end of file. The archive ends mid-stream, so it is truncated...` — and the two cases that are decidable before any decompressor runs are refused ahead of it by their own name: an empty file says it is empty and says that a writer should stage under a temporary name and rename into place, and a file whose first bytes are not the declared format's signature (`1f 8b` for GZ, `50 4b` for ZIP) says it is not that archive whatever its name says. TAR has no signature at offset 0, so it is left to the extractor. ZIP and TAR failures gain the same naming, since yauzl and tar are no more forthcoming than zlib. An unsupported `compressionType` is also now refused as a precondition, naming the three that exist, instead of from inside the switch
14
+
15
+ - **The file log sink could be ended while the records it had accepted were still in flight.** A winston logger is a Transform piped into its transports, and `Logger._final` ends every transport the moment the logger's writable side closes — while records already accepted sit in its readable buffer waiting for the transport to have room. The transport then drains, the pipe resumes on that drain, and the next record is written into a stream that has ended: `ERR_STREAM_WRITE_AFTER_END`, raised from a stream with no line of remora's in its stack. Re-pointing the sink and closing it now drain the in-flight records into the transports first, and a re-point releases the old logger only after the new one is serving, so nothing can be routed to a logger on its way out. `flush()` also writes ONE sentinel record rather than one per transport — every transport sees every record, so a sentinel each only wrote N blank lines into the same file
16
+
9
17
  ## V 2.2.0 - 2026-10-15
10
18
 
11
19
  **Upgrading from 2.1.** Three changes can alter what an existing project does, despite the minor version. `nullable` on a producer dimension is no longer documentation: it defaults to `true` and keeps a missing value as `null` as before, but a dimension that declares `"nullable": false` now gets the type's own empty value (`""`, `0`, `false`) where it used to get `null`. A `NULL` in a `number` column now stages as `null` rather than as `0`, so a consumer that was summing or filtering over those zeros sees different figures — the fix is in the staging of every CSV-staged source (SQL, Delta Share, local and S3 CSV). And `settings.sql` on a producer is gone from the schema, so a producer still carrying it fails `compile` — the property never did anything; delete it. Also worth a glance: `remora package` writes `./remora.zip` instead of `./remora-config.zip` when no `-o` is given, and the resource folder lists in `project.json` (`sources`, `producers`, `consumers`, `schemas`, `codeSets`) are ignored — the folders are fixed. They still validate, so nothing breaks, but a project pointing them somewhere else was already not being read that way.
package/index.js CHANGED
@@ -11114,8 +11114,7 @@ var init_FileLogService = __esm({
11114
11114
  return;
11115
11115
  if (!(0, import_fs.existsSync)(folder))
11116
11116
  (0, import_fs.mkdirSync)(folder, { recursive: true });
11117
- if (this._logger)
11118
- this._logger.end();
11117
+ const previous = this._logger;
11119
11118
  this._folder = folder;
11120
11119
  this._file = file;
11121
11120
  this._logger = import_winston.default.createLogger({
@@ -11143,6 +11142,7 @@ ${stack}` : base;
11143
11142
  });
11144
11143
  this._logger.on("error", (error) => console.error(`File logger error: ${error?.message ?? String(error)}`));
11145
11144
  this._enabled = true;
11145
+ if (previous) void this._endSafely(previous);
11146
11146
  };
11147
11147
  /**
11148
11148
  * Where the file sink writes, whether or not it is currently enabled.
@@ -11168,19 +11168,31 @@ ${stack}` : base;
11168
11168
  };
11169
11169
  this.flush = () => {
11170
11170
  if (!this._enabled || !this._logger) return Promise.resolve();
11171
+ return this.drain(this._logger);
11172
+ };
11173
+ /**
11174
+ * Resolves once every record already handed to `target` has reached its transports.
11175
+ *
11176
+ * A winston logger is a Transform piped into each transport, so a record accepted by `log()` is
11177
+ * not a record that has been written: it sits in the logger's readable buffer until the transport
11178
+ * has room. Writing one sentinel and waiting for every transport to report it `logged` is what
11179
+ * establishes that the queue ahead of it has gone — records travel the pipe in order, so the
11180
+ * sentinel landing means everything before it landed.
11181
+ */
11182
+ this.drain = (target) => {
11171
11183
  return new Promise((resolve) => {
11172
- let pending = this._logger.transports.length;
11184
+ let pending = target.transports.length;
11173
11185
  if (pending === 0) return resolve();
11174
- for (const transport of this._logger.transports) {
11175
- if (typeof transport.on === "function") {
11176
- transport.once("logged", () => {
11177
- if (--pending === 0) resolve();
11178
- });
11179
- this._logger.info("");
11180
- } else {
11186
+ for (const transport of target.transports) {
11187
+ if (typeof transport.on !== "function") {
11181
11188
  if (--pending === 0) resolve();
11189
+ continue;
11182
11190
  }
11191
+ transport.once("logged", () => {
11192
+ if (--pending === 0) resolve();
11193
+ });
11183
11194
  }
11195
+ target.info("");
11184
11196
  });
11185
11197
  };
11186
11198
  /**
@@ -11191,15 +11203,37 @@ ${stack}` : base;
11191
11203
  * listening to. Leaving `_enabled` true after a close is what turned "someone logged after
11192
11204
  * close" into "the process died". Use `flush()` when the sink is still needed afterwards.
11193
11205
  */
11194
- this.close = () => {
11195
- if (!this._enabled || !this._logger) return Promise.resolve();
11206
+ this.close = async () => {
11207
+ if (!this._enabled || !this._logger) return;
11196
11208
  const closing = this._logger;
11197
11209
  this._enabled = false;
11198
11210
  this._logger = null;
11199
- return new Promise((resolve) => {
11200
- closing.on("finish", resolve);
11201
- closing.end();
11202
- });
11211
+ await this._endSafely(closing);
11212
+ };
11213
+ /**
11214
+ * Ends a winston logger without the race its own `end()` carries.
11215
+ *
11216
+ * `Logger._final` ends EVERY TRANSPORT the moment the logger's writable side closes — while the
11217
+ * records it already accepted are still travelling down the pipe from its readable side. The
11218
+ * transport drains, the pipe resumes on that drain, and the next record is written into a stream
11219
+ * that has ended: ERR_STREAM_WRITE_AFTER_END, raised from a stream no line of ours is standing
11220
+ * next to. In an executor thread under a debug run, where the pipe is never empty, that is an
11221
+ * error per rotation and a stack with nothing of remora's in it. Draining the in-flight records
11222
+ * into the transports first is what removes the race rather than hiding it.
11223
+ *
11224
+ * A failure here is swallowed on purpose: this logger is on its way out, it carries its own
11225
+ * `error` handler, and a sink that cannot close must not take the caller's shutdown with it.
11226
+ */
11227
+ this._endSafely = async (target) => {
11228
+ try {
11229
+ await this.drain(target);
11230
+ await new Promise((resolve) => {
11231
+ target.once("finish", resolve);
11232
+ target.end();
11233
+ });
11234
+ } catch (error) {
11235
+ console.error(`File logger failed to close: ${error?.message ?? String(error)}`);
11236
+ }
11203
11237
  };
11204
11238
  }
11205
11239
  };
@@ -11727,7 +11761,7 @@ var CONSTANTS, Constants_default;
11727
11761
  var init_Constants = __esm({
11728
11762
  "../../packages/constants/src/Constants.ts"() {
11729
11763
  CONSTANTS = {
11730
- cliVersion: "2.2.0",
11764
+ cliVersion: "2.2.1",
11731
11765
  backendVersion: 1,
11732
11766
  backendPort: 5088,
11733
11767
  workerVersion: 2,
@@ -37732,7 +37766,7 @@ var init_index_min = __esm({
37732
37766
  });
37733
37767
 
37734
37768
  // ../../packages/parsing/src/ParseCompression.ts
37735
- var fs10, import_zlib2, import_path9, import_yauzl, import_promises6, ParseCompressionClass, ParseCompression, ParseCompression_default;
37769
+ var fs10, import_zlib2, import_path9, import_yauzl, import_promises6, SUPPORTED_FORMATS, FORMAT_SIGNATURES, TRUNCATED_INPUT_CODE, ParseCompressionClass, ParseCompression, ParseCompression_default;
37736
37770
  var init_ParseCompression = __esm({
37737
37771
  "../../packages/parsing/src/ParseCompression.ts"() {
37738
37772
  fs10 = __toESM(require("fs"), 1);
@@ -37744,6 +37778,12 @@ var init_ParseCompression = __esm({
37744
37778
  init_src();
37745
37779
  init_src2();
37746
37780
  init_src3();
37781
+ SUPPORTED_FORMATS = ["ZIP", "TAR", "GZ"];
37782
+ FORMAT_SIGNATURES = {
37783
+ GZ: { bytes: [31, 139], label: "1f 8b" },
37784
+ ZIP: { bytes: [80, 75], label: '50 4b ("PK")' }
37785
+ };
37786
+ TRUNCATED_INPUT_CODE = "Z_BUF_ERROR";
37747
37787
  ParseCompressionClass = class {
37748
37788
  constructor() {
37749
37789
  /**
@@ -37762,31 +37802,76 @@ var init_ParseCompression = __esm({
37762
37802
  Affirm_default(standardPath, "Missing required parameter standardPath");
37763
37803
  Affirm_default(executionPath, "Missing required parameter executionPath");
37764
37804
  const archivePath = import_path9.default.join(standardPath, fileKey);
37765
- switch (compressionType?.toUpperCase()) {
37766
- case "ZIP": {
37767
- this._warningPossibleConvertionError(compressionType, fileKey);
37768
- return this._extractZip(archivePath, this._prepareTargetDir(executionPath));
37769
- }
37770
- case "TAR": {
37771
- this._warningPossibleConvertionError(compressionType, fileKey);
37772
- return this._extractTar(archivePath, this._prepareTargetDir(executionPath));
37773
- }
37774
- case "GZ": {
37775
- this._warningPossibleConvertionError(compressionType, fileKey);
37776
- fs10.mkdirSync(import_path9.default.dirname(executionPath), { recursive: true });
37777
- await this._gunzipFile(archivePath, executionPath);
37778
- return [executionPath];
37779
- }
37780
- case null:
37781
- case void 0: {
37782
- if ([".zip", ".tar", ".gz"].some((extension) => fileKey.includes(extension)))
37783
- Logger_default.warn(`The file ${fileKey} looks compressed while its producer declares no compressionType`);
37784
- return [archivePath];
37785
- }
37786
- default: {
37787
- throw new Error(`this compression type it has not been implemented ${compressionType}`);
37805
+ const format4 = compressionType?.toUpperCase();
37806
+ if (!format4) {
37807
+ if ([".zip", ".tar", ".gz"].some((extension) => fileKey.includes(extension)))
37808
+ Logger_default.warn(`The file ${fileKey} looks compressed while its producer declares no compressionType`);
37809
+ return [archivePath];
37810
+ }
37811
+ Affirm_default(SUPPORTED_FORMATS.includes(format4), `Compression type "${compressionType}" has not been implemented. Declare one of ${SUPPORTED_FORMATS.join(", ")} on the producer, or no compressionType at all`);
37812
+ this._warningPossibleConvertionError(compressionType, fileKey);
37813
+ this._affirmArchive(archivePath, format4);
37814
+ try {
37815
+ switch (format4) {
37816
+ case "ZIP":
37817
+ return await this._extractZip(archivePath, this._prepareTargetDir(executionPath));
37818
+ case "TAR":
37819
+ return await this._extractTar(archivePath, this._prepareTargetDir(executionPath));
37820
+ // GZ, the only format the affirm above leaves
37821
+ default: {
37822
+ fs10.mkdirSync(import_path9.default.dirname(executionPath), { recursive: true });
37823
+ await this._gunzipFile(archivePath, executionPath);
37824
+ return [executionPath];
37825
+ }
37788
37826
  }
37827
+ } catch (error) {
37828
+ throw this._asNamedFailure(error, format4, archivePath);
37829
+ }
37830
+ };
37831
+ /**
37832
+ * What has to be true of a file before a decompressor is pointed at it.
37833
+ *
37834
+ * Both checks exist because of what zlib says when they fail: a zero-byte file and a half-written
37835
+ * one are each reported as the bare string "unexpected end of file", and a file that is not
37836
+ * compressed at all as "incorrect header check" — three different configuration mistakes, none of
37837
+ * them named, none of them naming the file. A drop folder produces the first two constantly,
37838
+ * since a file being written into one exists at zero bytes before it holds anything.
37839
+ */
37840
+ this._affirmArchive = (archivePath, format4) => {
37841
+ Affirm_default(fs10.existsSync(archivePath), `Source file does not exist: ${archivePath}`);
37842
+ const { size } = fs10.statSync(archivePath);
37843
+ Affirm_default(size > 0, `Source file "${archivePath}" is empty (0 bytes), so there is no ${format4} archive in it to read. A file still being written into the source folder looks exactly like this \u2014 have the writer stage it under a temporary name and rename it into place, which is atomic, so a run never sees a partial file`);
37844
+ const signature = FORMAT_SIGNATURES[format4];
37845
+ if (!signature) return;
37846
+ Affirm_default(this._startsWith(archivePath, signature.bytes), `Source file "${archivePath}" is declared as ${format4} but does not start with the ${format4} signature ${signature.label}, so it is not a ${format4} archive whatever its name says. Check the producer's compressionType against what actually lands in the folder`);
37847
+ };
37848
+ /** Whether a file's first bytes are exactly `expected`. Reads only those bytes, never the file. */
37849
+ this._startsWith = (filePath, expected) => {
37850
+ const head = Buffer.alloc(expected.length);
37851
+ const handle = fs10.openSync(filePath, "r");
37852
+ try {
37853
+ fs10.readSync(handle, head, 0, head.length, 0);
37854
+ } finally {
37855
+ fs10.closeSync(handle);
37789
37856
  }
37857
+ return head.equals(Buffer.from(expected));
37858
+ };
37859
+ /**
37860
+ * A decompression failure that names the file it happened on.
37861
+ *
37862
+ * Every decompression library reports WHAT went wrong and none of them report WHICH file they
37863
+ * were reading — zlib hands back "unexpected end of file" and nothing else. A producer whose
37864
+ * fileKey carries a `%` stages hundreds of files in one pass, so that bare string was the entire
37865
+ * cause of death of a run: no file, no producer, no way to tell which of the hundreds to go and
37866
+ * look at. Our own failures already name the archive and are passed through untouched.
37867
+ */
37868
+ this._asNamedFailure = (error, format4, archivePath) => {
37869
+ const reason = error instanceof Error ? error : new Error(String(error));
37870
+ if (reason.message.includes(archivePath)) return reason;
37871
+ const truncated = reason.code === TRUNCATED_INPUT_CODE;
37872
+ const hint = truncated ? " The archive ends mid-stream, so it is truncated: the transfer that wrote it did not finish, or it was read while it was still being written" : "";
37873
+ const message = reason.message.replace(/\s*\.\s*$/, "");
37874
+ return new Error(`Failed to decompress "${archivePath}" as ${format4}: ${message}.${hint}`, { cause: reason });
37790
37875
  };
37791
37876
  this._warningPossibleConvertionError = (compression, fileKey) => {
37792
37877
  if (!fileKey.split(".").includes(compression.toLowerCase()) && compression) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@forzalabs/remora",
3
- "version": "2.2.0",
3
+ "version": "2.2.1",
4
4
  "description": "A powerful CLI tool for seamless data translation.",
5
5
  "main": "index.js",
6
6
  "private": false,
@@ -11162,8 +11162,7 @@ var init_FileLogService = __esm({
11162
11162
  return;
11163
11163
  if (!(0, import_fs.existsSync)(folder))
11164
11164
  (0, import_fs.mkdirSync)(folder, { recursive: true });
11165
- if (this._logger)
11166
- this._logger.end();
11165
+ const previous = this._logger;
11167
11166
  this._folder = folder;
11168
11167
  this._file = file;
11169
11168
  this._logger = import_winston.default.createLogger({
@@ -11191,6 +11190,7 @@ ${stack}` : base;
11191
11190
  });
11192
11191
  this._logger.on("error", (error) => console.error(`File logger error: ${error?.message ?? String(error)}`));
11193
11192
  this._enabled = true;
11193
+ if (previous) void this._endSafely(previous);
11194
11194
  };
11195
11195
  /**
11196
11196
  * Where the file sink writes, whether or not it is currently enabled.
@@ -11216,19 +11216,31 @@ ${stack}` : base;
11216
11216
  };
11217
11217
  this.flush = () => {
11218
11218
  if (!this._enabled || !this._logger) return Promise.resolve();
11219
+ return this.drain(this._logger);
11220
+ };
11221
+ /**
11222
+ * Resolves once every record already handed to `target` has reached its transports.
11223
+ *
11224
+ * A winston logger is a Transform piped into each transport, so a record accepted by `log()` is
11225
+ * not a record that has been written: it sits in the logger's readable buffer until the transport
11226
+ * has room. Writing one sentinel and waiting for every transport to report it `logged` is what
11227
+ * establishes that the queue ahead of it has gone — records travel the pipe in order, so the
11228
+ * sentinel landing means everything before it landed.
11229
+ */
11230
+ this.drain = (target) => {
11219
11231
  return new Promise((resolve) => {
11220
- let pending = this._logger.transports.length;
11232
+ let pending = target.transports.length;
11221
11233
  if (pending === 0) return resolve();
11222
- for (const transport of this._logger.transports) {
11223
- if (typeof transport.on === "function") {
11224
- transport.once("logged", () => {
11225
- if (--pending === 0) resolve();
11226
- });
11227
- this._logger.info("");
11228
- } else {
11234
+ for (const transport of target.transports) {
11235
+ if (typeof transport.on !== "function") {
11229
11236
  if (--pending === 0) resolve();
11237
+ continue;
11230
11238
  }
11239
+ transport.once("logged", () => {
11240
+ if (--pending === 0) resolve();
11241
+ });
11231
11242
  }
11243
+ target.info("");
11232
11244
  });
11233
11245
  };
11234
11246
  /**
@@ -11239,15 +11251,37 @@ ${stack}` : base;
11239
11251
  * listening to. Leaving `_enabled` true after a close is what turned "someone logged after
11240
11252
  * close" into "the process died". Use `flush()` when the sink is still needed afterwards.
11241
11253
  */
11242
- this.close = () => {
11243
- if (!this._enabled || !this._logger) return Promise.resolve();
11254
+ this.close = async () => {
11255
+ if (!this._enabled || !this._logger) return;
11244
11256
  const closing = this._logger;
11245
11257
  this._enabled = false;
11246
11258
  this._logger = null;
11247
- return new Promise((resolve) => {
11248
- closing.on("finish", resolve);
11249
- closing.end();
11250
- });
11259
+ await this._endSafely(closing);
11260
+ };
11261
+ /**
11262
+ * Ends a winston logger without the race its own `end()` carries.
11263
+ *
11264
+ * `Logger._final` ends EVERY TRANSPORT the moment the logger's writable side closes — while the
11265
+ * records it already accepted are still travelling down the pipe from its readable side. The
11266
+ * transport drains, the pipe resumes on that drain, and the next record is written into a stream
11267
+ * that has ended: ERR_STREAM_WRITE_AFTER_END, raised from a stream no line of ours is standing
11268
+ * next to. In an executor thread under a debug run, where the pipe is never empty, that is an
11269
+ * error per rotation and a stack with nothing of remora's in it. Draining the in-flight records
11270
+ * into the transports first is what removes the race rather than hiding it.
11271
+ *
11272
+ * A failure here is swallowed on purpose: this logger is on its way out, it carries its own
11273
+ * `error` handler, and a sink that cannot close must not take the caller's shutdown with it.
11274
+ */
11275
+ this._endSafely = async (target) => {
11276
+ try {
11277
+ await this.drain(target);
11278
+ await new Promise((resolve) => {
11279
+ target.once("finish", resolve);
11280
+ target.end();
11281
+ });
11282
+ } catch (error) {
11283
+ console.error(`File logger failed to close: ${error?.message ?? String(error)}`);
11284
+ }
11251
11285
  };
11252
11286
  }
11253
11287
  };
@@ -11775,7 +11809,7 @@ var CONSTANTS, Constants_default;
11775
11809
  var init_Constants = __esm({
11776
11810
  "../../packages/constants/src/Constants.ts"() {
11777
11811
  CONSTANTS = {
11778
- cliVersion: "2.2.0",
11812
+ cliVersion: "2.2.1",
11779
11813
  backendVersion: 1,
11780
11814
  backendPort: 5088,
11781
11815
  workerVersion: 2,
@@ -37899,7 +37933,7 @@ var init_index_min = __esm({
37899
37933
  });
37900
37934
 
37901
37935
  // ../../packages/parsing/src/ParseCompression.ts
37902
- var fs8, import_zlib2, import_path8, import_yauzl, import_promises6, ParseCompressionClass, ParseCompression, ParseCompression_default;
37936
+ var fs8, import_zlib2, import_path8, import_yauzl, import_promises6, SUPPORTED_FORMATS, FORMAT_SIGNATURES, TRUNCATED_INPUT_CODE, ParseCompressionClass, ParseCompression, ParseCompression_default;
37903
37937
  var init_ParseCompression = __esm({
37904
37938
  "../../packages/parsing/src/ParseCompression.ts"() {
37905
37939
  fs8 = __toESM(require("fs"), 1);
@@ -37911,6 +37945,12 @@ var init_ParseCompression = __esm({
37911
37945
  init_src();
37912
37946
  init_src2();
37913
37947
  init_src3();
37948
+ SUPPORTED_FORMATS = ["ZIP", "TAR", "GZ"];
37949
+ FORMAT_SIGNATURES = {
37950
+ GZ: { bytes: [31, 139], label: "1f 8b" },
37951
+ ZIP: { bytes: [80, 75], label: '50 4b ("PK")' }
37952
+ };
37953
+ TRUNCATED_INPUT_CODE = "Z_BUF_ERROR";
37914
37954
  ParseCompressionClass = class {
37915
37955
  constructor() {
37916
37956
  /**
@@ -37929,31 +37969,76 @@ var init_ParseCompression = __esm({
37929
37969
  Affirm_default(standardPath, "Missing required parameter standardPath");
37930
37970
  Affirm_default(executionPath, "Missing required parameter executionPath");
37931
37971
  const archivePath = import_path8.default.join(standardPath, fileKey);
37932
- switch (compressionType?.toUpperCase()) {
37933
- case "ZIP": {
37934
- this._warningPossibleConvertionError(compressionType, fileKey);
37935
- return this._extractZip(archivePath, this._prepareTargetDir(executionPath));
37936
- }
37937
- case "TAR": {
37938
- this._warningPossibleConvertionError(compressionType, fileKey);
37939
- return this._extractTar(archivePath, this._prepareTargetDir(executionPath));
37940
- }
37941
- case "GZ": {
37942
- this._warningPossibleConvertionError(compressionType, fileKey);
37943
- fs8.mkdirSync(import_path8.default.dirname(executionPath), { recursive: true });
37944
- await this._gunzipFile(archivePath, executionPath);
37945
- return [executionPath];
37946
- }
37947
- case null:
37948
- case void 0: {
37949
- if ([".zip", ".tar", ".gz"].some((extension) => fileKey.includes(extension)))
37950
- Logger_default.warn(`The file ${fileKey} looks compressed while its producer declares no compressionType`);
37951
- return [archivePath];
37952
- }
37953
- default: {
37954
- throw new Error(`this compression type it has not been implemented ${compressionType}`);
37972
+ const format4 = compressionType?.toUpperCase();
37973
+ if (!format4) {
37974
+ if ([".zip", ".tar", ".gz"].some((extension) => fileKey.includes(extension)))
37975
+ Logger_default.warn(`The file ${fileKey} looks compressed while its producer declares no compressionType`);
37976
+ return [archivePath];
37977
+ }
37978
+ Affirm_default(SUPPORTED_FORMATS.includes(format4), `Compression type "${compressionType}" has not been implemented. Declare one of ${SUPPORTED_FORMATS.join(", ")} on the producer, or no compressionType at all`);
37979
+ this._warningPossibleConvertionError(compressionType, fileKey);
37980
+ this._affirmArchive(archivePath, format4);
37981
+ try {
37982
+ switch (format4) {
37983
+ case "ZIP":
37984
+ return await this._extractZip(archivePath, this._prepareTargetDir(executionPath));
37985
+ case "TAR":
37986
+ return await this._extractTar(archivePath, this._prepareTargetDir(executionPath));
37987
+ // GZ, the only format the affirm above leaves
37988
+ default: {
37989
+ fs8.mkdirSync(import_path8.default.dirname(executionPath), { recursive: true });
37990
+ await this._gunzipFile(archivePath, executionPath);
37991
+ return [executionPath];
37992
+ }
37955
37993
  }
37994
+ } catch (error) {
37995
+ throw this._asNamedFailure(error, format4, archivePath);
37996
+ }
37997
+ };
37998
+ /**
37999
+ * What has to be true of a file before a decompressor is pointed at it.
38000
+ *
38001
+ * Both checks exist because of what zlib says when they fail: a zero-byte file and a half-written
38002
+ * one are each reported as the bare string "unexpected end of file", and a file that is not
38003
+ * compressed at all as "incorrect header check" — three different configuration mistakes, none of
38004
+ * them named, none of them naming the file. A drop folder produces the first two constantly,
38005
+ * since a file being written into one exists at zero bytes before it holds anything.
38006
+ */
38007
+ this._affirmArchive = (archivePath, format4) => {
38008
+ Affirm_default(fs8.existsSync(archivePath), `Source file does not exist: ${archivePath}`);
38009
+ const { size } = fs8.statSync(archivePath);
38010
+ Affirm_default(size > 0, `Source file "${archivePath}" is empty (0 bytes), so there is no ${format4} archive in it to read. A file still being written into the source folder looks exactly like this \u2014 have the writer stage it under a temporary name and rename it into place, which is atomic, so a run never sees a partial file`);
38011
+ const signature = FORMAT_SIGNATURES[format4];
38012
+ if (!signature) return;
38013
+ Affirm_default(this._startsWith(archivePath, signature.bytes), `Source file "${archivePath}" is declared as ${format4} but does not start with the ${format4} signature ${signature.label}, so it is not a ${format4} archive whatever its name says. Check the producer's compressionType against what actually lands in the folder`);
38014
+ };
38015
+ /** Whether a file's first bytes are exactly `expected`. Reads only those bytes, never the file. */
38016
+ this._startsWith = (filePath, expected) => {
38017
+ const head = Buffer.alloc(expected.length);
38018
+ const handle = fs8.openSync(filePath, "r");
38019
+ try {
38020
+ fs8.readSync(handle, head, 0, head.length, 0);
38021
+ } finally {
38022
+ fs8.closeSync(handle);
37956
38023
  }
38024
+ return head.equals(Buffer.from(expected));
38025
+ };
38026
+ /**
38027
+ * A decompression failure that names the file it happened on.
38028
+ *
38029
+ * Every decompression library reports WHAT went wrong and none of them report WHICH file they
38030
+ * were reading — zlib hands back "unexpected end of file" and nothing else. A producer whose
38031
+ * fileKey carries a `%` stages hundreds of files in one pass, so that bare string was the entire
38032
+ * cause of death of a run: no file, no producer, no way to tell which of the hundreds to go and
38033
+ * look at. Our own failures already name the archive and are passed through untouched.
38034
+ */
38035
+ this._asNamedFailure = (error, format4, archivePath) => {
38036
+ const reason = error instanceof Error ? error : new Error(String(error));
38037
+ if (reason.message.includes(archivePath)) return reason;
38038
+ const truncated = reason.code === TRUNCATED_INPUT_CODE;
38039
+ const hint = truncated ? " The archive ends mid-stream, so it is truncated: the transfer that wrote it did not finish, or it was read while it was still being written" : "";
38040
+ const message = reason.message.replace(/\s*\.\s*$/, "");
38041
+ return new Error(`Failed to decompress "${archivePath}" as ${format4}: ${message}.${hint}`, { cause: reason });
37957
38042
  };
37958
38043
  this._warningPossibleConvertionError = (compression, fileKey) => {
37959
38044
  if (!fileKey.split(".").includes(compression.toLowerCase()) && compression) {