@xeplr/actions 1.4.3 → 1.4.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -123,8 +123,8 @@ Register the ones you want (`register(actions.builtins.dbFetch)`) or pass a modu
123
123
  | `emailRead` | `email-read` | List messages, newest first, with `unread_only`, `limit` (1–1000, default 50), `since`, `from_equals`, `from_contains`, `subject_contains`, `body: 'full' \| 'text' \| 'none'`, optional NDJSON spooling. |
124
124
  | `emailMove` | `email-move` | Move a message (`messageId`, `folder`) to `toFolder`. |
125
125
  | `emailDelete` | `email-delete` | Flag `\Deleted` **and expunge** — permanent, not a move to Trash. |
126
- | `emailDownloadEmail` | `email-download-email` | Stream the raw message to an `.eml` file. `{ count, saved[], path }`. |
127
- | `emailDownloadAttachments` | `email-download-attachments` | Save attachments, optionally filtered by `extension` / `filenameContains`. `{ count, saved[], path, skipped }`. |
126
+ | `emailDownloadEmail` | `email-download-email` | Stream the raw message to an `.eml` file, named by `fileName`. `{ count, saved[], path, existing[] }`. |
127
+ | `emailDownloadAttachments` | `email-download-attachments` | Save attachments, optionally filtered by `extension` / `filenameContains`, named by `fileName` (see [Saving files](#saving-files-names-and-a-name-already-taken)). `{ count, saved[], path, skipped, existing[] }`. |
128
128
  | `httpRequest`, `spawnProgram`, `dbQuery`, `fileMove` | — | **Placeholders.** Empty modules with no `name` or `execute`; `register()` rejects them. |
129
129
 
130
130
  ### Connections
@@ -231,9 +231,42 @@ should not take the other eight hundred and ninety-nine with it.
231
231
  **No aggregation.** `group by status, count` changes what a row *is*; that is
232
232
  a different shape and, when it is wanted, a different step.
233
233
 
234
+ ## Saving files: names, and a name already taken
235
+
236
+ Every action that saves a file follows one rule (`lib/fileNames.js`): `email-download-attachments`, `email-download-email` and `export-table`.
237
+
238
+ **`fileName`** (the email steps) or **`outputPath`** (`export-table`) is a template with two kinds of placeholder:
239
+
240
+ | | filled in by | examples |
241
+ |---|---|---|
242
+ | `{…}` | the workflow, before the step runs | `{trigger.window.from\|date:YYYY-MM}`, `{item.from}` |
243
+ | `[…]` | the action, from the file itself | attachments: `[original]` `[name]` `[ext]` `[n]` `[id]`; a whole email: `[subject]` `[id]` `[ext]` |
244
+
245
+ ```
246
+ Bank-{trigger.window.from|date:YYYY-MM:Asia/Kolkata}-[original] → Bank-2026-09-statement.csv
247
+ ```
248
+
249
+ - **Why two kinds:** the engine fills every `{…}` first, so a file's own placeholders in `{…}` would be blanked before the action saw them.
250
+ - **A placeholder the file does not have** is refused, listing the ones it has. A name that comes out empty is refused.
251
+ - **A name never leaves its folder:** `/ \ : * ? " < > |` become `_`, and `.` or `..` is refused. Spaces are kept.
252
+ - **No `fileName`:** a random prefix and the original name, as before.
253
+
254
+ **`ifExists`**, when the name is already taken:
255
+
256
+ | | |
257
+ |---|---|
258
+ | `fail` (default) | The step fails with `FILE_EXISTS`, naming the file, and **nothing is written**. A second run for the same period is caught, not repeated. |
259
+ | `skip` | Not written; listed in `existing[]` (`export-table`: `skipped: true`), so the next step can branch on "already downloaded". |
260
+ | `overwrite` | Replaced. Two files of one step with one name still get a suffix, so neither is lost. |
261
+ | `addSuffix` | `Bank-2026-09 (2).csv`, the next free number. |
262
+
263
+ Decided for the whole step before anything is written, so `fail` never leaves half a batch.
264
+
234
265
  ## Export
235
266
 
236
- `export-table` writes rows that arrive **already presented**: which columns, in what order, under what heading, and for each cell its raw value and the text a reader saw. The action knows nothing of reports or pivots; the caller decides all that (in xeplr-bi, `report-engine`'s `presentTable` does). Each format takes what it needs:
267
+ **From a workflow:** `rows` can be plain records, as a fetch or the data step gives them (`{ ref: 'A1', amount: 500 }`). Columns pick them by `key` (default: the header). With no columns, every field of the first row, in order. `outputPath` follows [Saving files](#saving-files-names-and-a-name-already-taken).
268
+
269
+ `export-table` also writes rows that arrive **already presented**: which columns, in what order, under what heading, and for each cell its raw value and the text a reader saw. The action knows nothing of reports or pivots; the caller decides all that (in xeplr-bi, `report-engine`'s `presentTable` does). Each format takes what it needs:
237
270
 
238
271
  | format | writes | from |
239
272
  |---|---|---|
@@ -484,20 +517,26 @@ await driver.query(pool, userSql, params) // reads only
484
517
 
485
518
  ## Tests
486
519
 
487
- There is no `npm test` script. The tests use `node:test`; run a file directly:
488
-
489
520
  ```sh
521
+ npm test # every folder of test/, one file at a time
522
+ PG_HOST=localhost PG_USER=$USER npm test # the same, with the Postgres suites running
490
523
  node --test test/uploader/reconcile.test.js
491
524
  ```
492
525
 
493
- | needs nothing | needs a database |
494
- |---|---|
495
- | `test/import-modes.test.js` (DuckDB in memory; its Postgres cases run when `PG_HOST`/`PG_PORT`/`PG_USER`/`PG_PASSWORD`/`PG_DATABASE` are set, its MySQL cases when `MYSQL_HOST`/`MYSQL_PORT`/`MYSQL_USER`/`MYSQL_PASSWORD` are), `test/formats/pasted-rows.test.js` (its last case loads into Postgres when `PG_HOST`/`PG_PORT`/`PG_USER`/`PG_PASSWORD`/`PG_DATABASE` are set, else skipped), `test/procedure-call.test.js`, `test/streaming/spool.test.js`, `test/uploader/reconcile.test.js`, `test/write-progress.test.js`, `test/stop-movement.test.js`, `test/formats/csv-gnarly.test.js`, `test/drivers/mysql.test.js`, `test/drivers/mssql.test.js`, `test/drivers/query-columns.test.js`, `test/builtins/email.test.js` (loads the package root, which needs `@xeplr/db`: a peer dependency, installed for development) | Postgres at `localhost:5435` (`PG_PASSWORD`, default `postgres`; database `xeplr_actions_test`): `test/drivers/postgres.test.js`, `test/builtins/db-push.test.js`, `test/builtins/file-upload.test.js` (also honours `PG_HOST`/`PG_PORT`/`PG_USER`), `test/uploader/upload.test.js`; plus `@xeplr/db` for `test/uploader/meta-store-knex.test.js` |
496
- | `test/list-columns-length.test.js` (`XEPLR_TEST_PG_URL`, `MYSQL_TEST_URL`, `MSSQL_TEST_URL`; each skipped when unset), `test/drivers/duckdb.test.js`, `test/drivers/duckdb-values.test.js`, `test/duckdb-index-guard.test.js`, `test/staged-load.test.js` (needs `@duckdb/node-api`) | MySQL `localhost:3306` (`MYSQL_HOST/PORT/USER/PASSWORD`): `test/drivers/mysql-integration.test.js` |
497
- | | SQL Server `localhost:1433` (`MSSQL_HOST/PORT/USER/PASSWORD`): `test/drivers/mssql-integration.test.js` |
498
- | | Postgres + MySQL + SQL Server (`PG_*`, `MYSQL_*`, `MSSQL_*`): `test/builtins/db-replication.test.js` |
526
+ **Databases come from the environment, never from code** (`test/helpers/databases.js`):
527
+
528
+ | database | settings | default port |
529
+ |---|---|---|
530
+ | Postgres | `PG_HOST` `PG_PORT` `PG_USER` `PG_PASSWORD` | 5432 |
531
+ | MySQL | `MYSQL_HOST` `MYSQL_PORT` `MYSQL_USER` `MYSQL_PASSWORD` | 3306 |
532
+ | SQL Server | `MSSQL_HOST` `MSSQL_PORT` `MSSQL_USER` `MSSQL_PASSWORD` | 1433 |
533
+
534
+ - **No `*_HOST`:** that database's tests are **skipped**, and each says which settings run it.
535
+ - **`*_HOST` set:** they run, and a login the database refuses **fails** them. A configured run never passes by not connecting.
536
+ - **One file at a time** (`--test-concurrency=1`): the database suites share the `xeplr_actions_test` database and race to create it when run side by side.
537
+ - **Why:** these files used to carry a fixed host, port and password each (Postgres on `:5435`). On any other machine they all failed, so the pre-commit gate and CI ran only `test/*.test.js` and never this package's sub-folders. That is how `data-operation` shipped returning no rows (fixed in 1.4.2).
499
538
 
500
- `test/db-move.live.js` runs against a developer's own `.env` path and is not a portable test. CI (`.github/workflows/ci.yml`) skips tests when there is no `test` script.
539
+ Other tests read their own settings: `test/list-columns-length.test.js` (`XEPLR_TEST_PG_URL`, `MYSQL_TEST_URL`, `MSSQL_TEST_URL`), `test/readonly-connections.test.js` (`PGHOST` …), and `test/dates-across-zones.test.js` skips a database it cannot reach. `test/db-move.live.js` runs against a developer's own `.env` path and is not part of `npm test`.
501
540
 
502
541
  ## License
503
542
 
@@ -13,11 +13,12 @@
13
13
 
14
14
  var emailDrivers = require('../../drivers/email');
15
15
  var emailConfig = require('../../drivers/email/config');
16
+ var fileNames = require('../../fileNames');
16
17
 
17
18
  module.exports = {
18
19
  name: 'email-download-attachments',
19
20
  description: 'Save a message\'s attachments to disk, optionally filtered by extension or ' +
20
- 'filename substring. Returns { count, saved[], path, skipped }.',
21
+ 'filename substring, named as you say. Returns { count, saved[], path, skipped, existing[] }.',
21
22
  requires: [],
22
23
 
23
24
  // Which mailbox first — it is the only answer that changes what the rest of
@@ -39,7 +40,7 @@ module.exports = {
39
40
  description: 'Only attachments with this extension, e.g. "pdf" or ".pdf".' },
40
41
  { name: 'filenameContains', type: 'string', order: 7, group: 'Filters',
41
42
  description: 'Only attachments whose filename contains this substring.' }
42
- ],
43
+ ].concat(fileNames.nameInputs(8, '[original], [name], [ext], [n] (1, 2, … per attachment), [id] (the message)')),
43
44
 
44
45
  execute: async function(ctx) {
45
46
  var input = ctx.input || {};
@@ -11,6 +11,7 @@
11
11
 
12
12
  var emailDrivers = require('../../drivers/email');
13
13
  var emailConfig = require('../../drivers/email/config');
14
+ var fileNames = require('../../fileNames');
14
15
 
15
16
  module.exports = {
16
17
  name: 'email-download-email',
@@ -32,7 +33,7 @@ module.exports = {
32
33
  description: 'Folder the message is currently in — where its id is valid.' },
33
34
  { name: 'outDir', type: 'string', order: 5,
34
35
  description: 'Directory to write into. Defaults to XEPLR_ACTIONS_TMP_DIR.' }
35
- ],
36
+ ].concat(fileNames.nameInputs(6, '[subject], [id] (the message), [ext] (eml)')),
36
37
 
37
38
  execute: async function(ctx) {
38
39
  var input = ctx.input || {};
@@ -22,6 +22,7 @@ var path = require('path');
22
22
  var readline = require('readline');
23
23
  var writers = require('../../formats/writers');
24
24
  var { checkFormatRequires } = require('../../formats');
25
+ var fileNames = require('../../fileNames');
25
26
 
26
27
  function tmpDir() {
27
28
  var dir = process.env.XEPLR_ACTIONS_TMP_DIR || path.join(os.tmpdir(), 'xeplr-actions');
@@ -35,10 +36,28 @@ function fileNameFor(title, extension) {
35
36
  return base + '.' + extension;
36
37
  }
37
38
 
38
- async function* rowsFrom(input) {
39
- if (Array.isArray(input.rows)) { for (var i = 0; i < input.rows.length; i++) yield input.rows[i]; return; }
39
+ /**
40
+ * A ROW AS THE WRITERS TAKE IT: { values: [...] }. A plain record — what a
41
+ * fetch or the data step gives, { ref: 'A1', amount: 500 } — is laid out by
42
+ * the columns' `key` (or header), so a workflow can hand its rows straight
43
+ * over instead of reshaping them first.
44
+ */
45
+ function asRow(row, columns) {
46
+ if (row && Array.isArray(row.values)) return row;
47
+ var r = row || {};
48
+ return { values: columns.map(function(c) { var k = c.key || c.header; return r[k] === undefined ? null : r[k]; }) };
49
+ }
50
+
51
+ async function* rowsFrom(input, columns) {
52
+ if (Array.isArray(input.rows)) { for (var i = 0; i < input.rows.length; i++) yield asRow(input.rows[i], columns); return; }
40
53
  var lines = readline.createInterface({ input: fs.createReadStream(input.rowsFile), crlfDelay: Infinity });
41
- for await (var line of lines) if (line.trim()) yield JSON.parse(line);
54
+ for await (var line of lines) if (line.trim()) yield asRow(JSON.parse(line), columns);
55
+ }
56
+
57
+ /** Columns for plain records when none are given: every key of the first one, as it is. */
58
+ function columnsOf(rows) {
59
+ var first = (rows || []).find(function(r) { return r && typeof r === 'object' && !Array.isArray(r.values); });
60
+ return first ? Object.keys(first).map(function(k) { return { header: k, key: k }; }) : [];
42
61
  }
43
62
 
44
63
  module.exports = {
@@ -50,22 +69,29 @@ module.exports = {
50
69
  inputSchema: [
51
70
  { name: 'format', type: 'string', required: true, order: 1, enum: writers.SUPPORTED,
52
71
  description: 'csv, xlsx or pdf.' },
53
- { name: 'document', type: 'object', required: true, order: 2,
54
- description: '{ title?, subtitle?, footer?, columns: [{ header, kind?: number|date|text, numFmt?, align?, width? }] }. ' +
55
- 'numFmt is an Excel number format ("#,##0.00", "0.0%", "mmm yyyy").' },
72
+ { name: 'document', type: 'object', order: 2,
73
+ description: '{ title?, subtitle?, footer?, columns: [{ header, key?, kind?: number|date|text, numFmt?, align?, width? }] }. ' +
74
+ 'numFmt is an Excel number format ("#,##0.00", "0.0%", "mmm yyyy"). key: which field of a plain record ' +
75
+ 'goes in the column (default: the header). No columns with plain records: every field of the first row.' },
56
76
  { name: 'rows', type: 'array', order: 3,
57
- description: 'Rows: { values: [], display?: [], kind?: detail|subtotal|total }. Or give rowsFile.' },
77
+ description: 'Plain records, as a fetch or the data step gives them ({ ref, amount }), or { values: [], display?: [], ' +
78
+ 'kind?: detail|subtotal|total }. Or give rowsFile.' },
58
79
  { name: 'rowsFile', type: 'string', order: 4,
59
80
  description: 'An NDJSON file with one row per line, instead of rows.' },
60
81
  { name: 'outputPath', type: 'string', order: 5,
61
- description: 'Where to write. Default: a new file in XEPLR_ACTIONS_TMP_DIR named after the title.' },
82
+ description: 'Where to write, e.g. /data/recon/Recon-{trigger.window.from|date:YYYY-MM}.xlsx. ' +
83
+ 'Default: a new file in XEPLR_ACTIONS_TMP_DIR named after the title.' },
84
+ { name: 'ifExists', type: 'string', order: 7, default: 'fail', options: fileNames.IF_EXISTS,
85
+ description: 'When outputPath is already there. fail: the step fails and nothing is written. skip: left alone ' +
86
+ '(skipped: true). overwrite: replaced. addSuffix: "name (2).xlsx".' },
62
87
  { name: 'maxRows', type: 'number', order: 6,
63
88
  description: 'PDF only: rows to draw before stopping with a note (default 5,000).' }
64
89
  ],
65
90
  outputSchema: [
66
91
  { name: 'filePath', type: 'string' }, { name: 'fileName', type: 'string' },
67
92
  { name: 'contentType', type: 'string' }, { name: 'bytes', type: 'number' },
68
- { name: 'rows', type: 'number' }, { name: 'truncated', type: 'boolean' }
93
+ { name: 'rows', type: 'number' }, { name: 'truncated', type: 'boolean' },
94
+ { name: 'skipped', type: 'boolean' }
69
95
  ],
70
96
 
71
97
  execute: async function(ctx) {
@@ -74,6 +100,7 @@ module.exports = {
74
100
  var writer = writers.getWriter(format);
75
101
  checkFormatRequires('export-table', format, writer);
76
102
  var doc = Object.assign({}, input.document || {});
103
+ if ((!Array.isArray(doc.columns) || !doc.columns.length) && Array.isArray(input.rows)) doc.columns = columnsOf(input.rows);
77
104
  if (!Array.isArray(doc.columns) || !doc.columns.length) {
78
105
  var bad = new Error('document.columns must list at least one column');
79
106
  bad.status = 400;
@@ -89,10 +116,22 @@ module.exports = {
89
116
  var info = writers.INFO[format];
90
117
  var fileName = fileNameFor(doc.title, info.extension);
91
118
  var filePath = input.outputPath || path.join(fs.mkdtempSync(path.join(tmpDir(), 'export-')), fileName);
119
+ if (input.outputPath) {
120
+ // The same rule as every action that saves a file (lib/fileNames.js):
121
+ // a name already taken is never silently replaced.
122
+ var dir = path.dirname(filePath);
123
+ fs.mkdirSync(dir, { recursive: true });
124
+ var place = fileNames.plan(dir, [path.basename(filePath)], input.ifExists)[0];
125
+ if (!place.write) {
126
+ return { filePath: place.path, fileName: place.name, contentType: info.contentType,
127
+ bytes: fs.statSync(place.path).size, rows: 0, truncated: false, skipped: true };
128
+ }
129
+ filePath = place.path;
130
+ }
92
131
  var open = writer.open(filePath, doc);
93
132
  var signal = ctx.signal;
94
133
  try {
95
- for await (var row of rowsFrom(input)) {
134
+ for await (var row of rowsFrom(input, doc.columns)) {
96
135
  if (signal && signal.aborted) throw new Error('Stopped by request');
97
136
  var more = await open.writeRow(row);
98
137
  // Excel's own limit: the rest are only counted. A PDF keeps counting
@@ -106,7 +145,8 @@ module.exports = {
106
145
  contentType: info.contentType,
107
146
  bytes: fs.statSync(filePath).size,
108
147
  rows: done.rows,
109
- truncated: done.truncated
148
+ truncated: done.truncated,
149
+ skipped: false
110
150
  };
111
151
  } catch (err) {
112
152
  try { fs.rmSync(filePath, { force: true }); } catch (e) { /* nothing to clean */ }
@@ -12,6 +12,7 @@ var path = require('path');
12
12
  var fs = require('fs');
13
13
  var fsp = require('fs/promises');
14
14
  var os = require('os');
15
+ var fileNames = require('../../fileNames');
15
16
 
16
17
  // Short random prefix. Attachment names collide constantly in mail —
17
18
  // "invoice.pdf" from fifty senders — and a download that silently overwrites
@@ -65,7 +66,74 @@ function emlFilename(subject, uid) {
65
66
  return shortId() + '_' + base + '.eml';
66
67
  }
67
68
 
69
+ /**
70
+ * Where each attachment goes. Named (input.fileName): every name decided
71
+ * before any file is written (fileNames.plan, with ifExists). Unnamed: a
72
+ * random prefix, as always — attachment names collide.
73
+ * @returns [{ name, path, write }] in the order of `attachments`
74
+ */
75
+ function attachmentPlaces(dir, attachments, uid, input) {
76
+ if (!input.fileName) {
77
+ return attachments.map(function(a) {
78
+ var n = shortId() + '_' + sanitizeFilename(a.filename);
79
+ return { name: n, path: path.join(dir, n), write: true };
80
+ });
81
+ }
82
+ return fileNames.plan(dir, attachments.map(function(a, i) {
83
+ var p = fileNames.partsOf(a.filename);
84
+ return fileNames.nameFrom(input.fileName, { original: p.original, name: p.name, ext: p.ext, n: i + 1, id: uid });
85
+ }), input.ifExists);
86
+ }
87
+
88
+ /** Where a whole message's .eml goes: named by input.fileName, else the subject. */
89
+ function emlPlace(dir, subject, uid, input) {
90
+ if (!input.fileName) {
91
+ var n = emlFilename(subject, uid);
92
+ return { name: n, path: path.join(dir, n), write: true };
93
+ }
94
+ var named = fileNames.nameFrom(input.fileName, { subject: fileNames.safeName(subject) || 'message', id: uid, ext: 'eml' });
95
+ return fileNames.plan(dir, [named], input.ifExists)[0];
96
+ }
97
+
98
+ /**
99
+ * Save a parsed message's attachments that pass the filters, named as asked.
100
+ * @returns { count, saved[], existing[], path, skipped }
101
+ * saved[] entries are { filename, path, size, contentType } — nodemailer's
102
+ * attachment shape, so email-send can forward them as they are.
103
+ */
104
+ function saveAttachments(dir, all, uid, input) {
105
+ var wanted = all.filter(function(a) { return attachmentMatches(a.filename, input.extension, input.filenameContains); });
106
+ var places = attachmentPlaces(dir, wanted, uid, input);
107
+ var saved = [];
108
+ var existing = [];
109
+ wanted.forEach(function(a, i) {
110
+ var place = places[i];
111
+ if (!place.write) { existing.push({ filename: place.name, path: place.path }); return; }
112
+ fs.writeFileSync(place.path, a.content);
113
+ saved.push({
114
+ filename: input.fileName ? place.name : (a.filename || place.name),
115
+ path: place.path,
116
+ size: a.size || (a.content && a.content.length) || null,
117
+ contentType: a.contentType || null
118
+ });
119
+ });
120
+ return {
121
+ count: saved.length,
122
+ saved: saved,
123
+ // Already there and left alone (ifExists: skip), so the next step can
124
+ // tell "nothing new" from "nothing attached".
125
+ existing: existing,
126
+ path: saved.length ? saved[0].path : null,
127
+ // Reported rather than silent: "0 attachments" and "6 attachments, none
128
+ // matched your filter" are different problems with the same count.
129
+ skipped: all.length - wanted.length
130
+ };
131
+ }
132
+
68
133
  module.exports = {
134
+ saveAttachments: saveAttachments,
135
+ attachmentPlaces: attachmentPlaces,
136
+ emlPlace: emlPlace,
69
137
  shortId: shortId,
70
138
  sanitizeFilename: sanitizeFilename,
71
139
  resolveOutDir: resolveOutDir,
@@ -317,12 +317,17 @@ async function downloadEmail(connection, input) {
317
317
  subject = (meta && meta.envelope && meta.envelope.subject) || null;
318
318
  } catch (_) { /* filename falls back to the uid — never fatal */ }
319
319
 
320
- var dl = await client.download(uid, undefined, { uid: true });
321
-
322
320
  var dir = await files.resolveOutDir(input.outDir);
323
- var filename = files.emlFilename(subject, uid);
324
- var filepath = files.path.join(dir, filename);
321
+ // Decided before the download: a name already taken under `fail` costs
322
+ // nothing, and under `skip` nothing is fetched at all.
323
+ var where = files.emlPlace(dir, subject, uid, input);
324
+ if (!where.write) {
325
+ return { count: 0, saved: [], path: null, existing: [{ filename: where.name, path: where.path }], subject: subject, messageId: uid };
326
+ }
327
+ var filename = where.name;
328
+ var filepath = where.path;
325
329
 
330
+ var dl = await client.download(uid, undefined, { uid: true });
326
331
  await pipeline(dl.content, files.fs.createWriteStream(filepath));
327
332
 
328
333
  var size = files.fs.statSync(filepath).size;
@@ -332,6 +337,7 @@ async function downloadEmail(connection, input) {
332
337
  // `path` / `saved[]` identically whichever produced the file.
333
338
  saved: [{ filename: filename, path: filepath, size: size, contentType: 'message/rfc822' }],
334
339
  path: filepath,
340
+ existing: [],
335
341
  subject: subject,
336
342
  messageId: uid
337
343
  };
@@ -353,35 +359,11 @@ async function downloadAttachments(connection, input) {
353
359
  return withMailbox('email-download-attachments', connection, input.folder || 'INBOX', async function(client) {
354
360
  var dl = await client.download(uid, undefined, { uid: true });
355
361
  var parsed = await simpleParser(dl.content);
356
- var all = parsed.attachments || [];
357
362
 
358
363
  var dir = await files.resolveOutDir(input.outDir);
359
- var saved = [];
360
- var skipped = 0;
361
-
362
- for (var i = 0; i < all.length; i++) {
363
- var a = all[i];
364
- if (!files.attachmentMatches(a.filename, input.extension, input.filenameContains)) { skipped++; continue; }
365
- var filename = files.shortId() + '_' + files.sanitizeFilename(a.filename);
366
- var filepath = files.path.join(dir, filename);
367
- files.fs.writeFileSync(filepath, a.content);
368
- saved.push({
369
- filename: a.filename || filename,
370
- path: filepath,
371
- size: a.size || (a.content && a.content.length) || null,
372
- contentType: a.contentType || null
373
- });
374
- }
375
-
376
- return {
377
- count: saved.length,
378
- saved: saved,
379
- path: saved.length ? saved[0].path : null,
380
- // Reported rather than silent: "0 attachments" and "6 attachments, none
381
- // matched your filter" are different problems with the same count.
382
- skipped: skipped,
383
- messageId: uid
384
- };
364
+ var out = files.saveAttachments(dir, parsed.attachments || [], uid, input);
365
+ out.messageId = uid;
366
+ return out;
385
367
  });
386
368
  }
387
369
 
@@ -0,0 +1,136 @@
1
+ // WHAT A SAVED FILE IS CALLED, AND WHAT HAPPENS WHEN ONE IS ALREADY THERE.
2
+ //
3
+ // One rule for every action that writes a file, so a workflow can say
4
+ // "Bank-2026-09.csv" instead of getting "3f9a1c02e4_statement.csv".
5
+ //
6
+ // A name is a template with two kinds of placeholder:
7
+ // {…} from the WORKFLOW, filled in by the engine before the step runs:
8
+ // Bank-{trigger.window.from|date:YYYY-MM}
9
+ // […] from the FILE ITSELF, filled in here: [original] [name] [ext] [n]
10
+ // and whatever the action adds (an email's [subject], [id]).
11
+ // Square brackets because the engine fills every {…} first, and would blank
12
+ // a file's own placeholders before the action ever saw them.
13
+ //
14
+ // ifExists, when the name is already taken:
15
+ // fail (default) the step fails, naming the file: nothing is written,
16
+ // so a second run for the same period is caught, not repeated
17
+ // skip nothing is written for it, and it is listed in `existing`, so
18
+ // the next step can branch on "already downloaded"
19
+ // overwrite replaced
20
+ // addSuffix "Bank-2026-09 (2).csv"
21
+ // Decided for the whole step before anything is written, so `fail` never
22
+ // leaves half a batch behind.
23
+
24
+ var fs = require('fs');
25
+ var path = require('path');
26
+
27
+ var IF_EXISTS = ['fail', 'skip', 'overwrite', 'addSuffix'];
28
+
29
+ function nameError(message, code) {
30
+ var err = new Error(message);
31
+ err.code = code || 'FILE_NAME_INVALID';
32
+ err.status = 400;
33
+ return err;
34
+ }
35
+
36
+ /**
37
+ * A name that is safe to join to a directory: no separators, no characters
38
+ * a filesystem refuses, not "." or "..". Spaces are kept — "Bank Sep 2026.csv"
39
+ * is a name a person wants.
40
+ */
41
+ function safeName(name) {
42
+ var out = String(name == null ? '' : name)
43
+ .replace(/[\/\\:*?"<>|\u0000-\u001f]+/g, '_')
44
+ .replace(/\s+/g, ' ')
45
+ .trim()
46
+ .slice(0, 200);
47
+ if (!out || /^\.+$/.test(out)) return '';
48
+ return out;
49
+ }
50
+
51
+ /** The parts of an original file name: [original], [name], [ext]. */
52
+ function partsOf(original) {
53
+ var o = safeName(original) || 'file';
54
+ var dot = o.lastIndexOf('.');
55
+ var hasExt = dot > 0 && dot < o.length - 1;
56
+ return { original: o, name: hasExt ? o.slice(0, dot) : o, ext: hasExt ? o.slice(dot + 1) : '' };
57
+ }
58
+
59
+ /**
60
+ * The name from `template` and the file's own `tokens`. A placeholder the
61
+ * file does not have is refused, listing the ones it does — a typo must not
62
+ * quietly drop part of a name.
63
+ */
64
+ function nameFrom(template, tokens) {
65
+ var unknown = [];
66
+ var out = String(template).replace(/\[([A-Za-z]+)\]/g, function(whole, key) {
67
+ if (!Object.prototype.hasOwnProperty.call(tokens, key)) { unknown.push(key); return whole; }
68
+ var v = tokens[key];
69
+ return v === null || v === undefined ? '' : String(v);
70
+ });
71
+ if (unknown.length) {
72
+ throw nameError('The file name "' + template + '" uses [' + unknown.join('], [') + '], which this file does not have. ' +
73
+ 'It has: [' + Object.keys(tokens).join('], [') + '].');
74
+ }
75
+ var safe = safeName(out);
76
+ if (!safe) throw nameError('The file name "' + template + '" comes out empty. Check the values it is made from.');
77
+ return safe;
78
+ }
79
+
80
+ function withSuffix(name, k) {
81
+ var dot = name.lastIndexOf('.');
82
+ return dot > 0 ? name.slice(0, dot) + ' (' + k + ')' + name.slice(dot) : name + ' (' + k + ')';
83
+ }
84
+
85
+ /**
86
+ * Where each file goes, decided for the whole step before anything is
87
+ * written. `names` in order; returns [{ name, path, write }] where `write`
88
+ * is false for one skipped. Throws FILE_EXISTS for `fail`.
89
+ */
90
+ function plan(dir, names, ifExists) {
91
+ var mode = ifExists || 'fail';
92
+ if (IF_EXISTS.indexOf(mode) === -1) {
93
+ throw nameError('ifExists is one of ' + IF_EXISTS.join(', ') + ', not "' + mode + '".');
94
+ }
95
+ var taken = {};
96
+ var exists = function(n) { return taken[n] || fs.existsSync(path.join(dir, n)); };
97
+ return names.map(function(name) {
98
+ var out = name;
99
+ if (exists(out)) {
100
+ if (mode === 'fail') {
101
+ throw nameError('"' + out + '" is already in ' + dir + '. Nothing was saved. ' +
102
+ 'Set ifExists to skip, overwrite or addSuffix to save anyway.', 'FILE_EXISTS');
103
+ }
104
+ if (mode === 'skip') return { name: out, path: path.join(dir, out), write: false };
105
+ if (mode === 'addSuffix') {
106
+ var k = 2;
107
+ while (exists(withSuffix(name, k))) k++;
108
+ out = withSuffix(name, k);
109
+ }
110
+ // overwrite: the same name, written again. Two files of THIS step with
111
+ // one name still get a suffix: overwriting its own first file would lose it.
112
+ if (mode === 'overwrite' && taken[out]) {
113
+ var j = 2;
114
+ while (taken[withSuffix(name, j)]) j++;
115
+ out = withSuffix(name, j);
116
+ }
117
+ }
118
+ taken[out] = true;
119
+ return { name: out, path: path.join(dir, out), write: true };
120
+ });
121
+ }
122
+
123
+ /** The inputs every file-writing action takes, to spread into its inputSchema. */
124
+ function nameInputs(order, tokensHelp) {
125
+ return [
126
+ { name: 'fileName', type: 'string', order: order, group: 'Saving',
127
+ description: 'What to call the file. {…} comes from the workflow, e.g. Bank-{trigger.window.from|date:YYYY-MM}; ' +
128
+ '[…] from the file: ' + tokensHelp + '. Empty: a random prefix and the original name.' },
129
+ { name: 'ifExists', type: 'string', order: order + 1, group: 'Saving', default: 'fail',
130
+ options: IF_EXISTS,
131
+ description: 'When the name is taken. fail: the step fails and nothing is saved. skip: listed in `existing`, not saved. ' +
132
+ 'overwrite: replaced. addSuffix: "name (2).ext". Only with a file name.' }
133
+ ];
134
+ }
135
+
136
+ module.exports = { safeName: safeName, partsOf: partsOf, nameFrom: nameFrom, plan: plan, nameInputs: nameInputs, IF_EXISTS: IF_EXISTS };
package/package.json CHANGED
@@ -1,8 +1,11 @@
1
1
  {
2
2
  "name": "@xeplr/actions",
3
- "version": "1.4.3",
3
+ "version": "1.4.4",
4
4
  "description": "Named-action registry with typed inputs and safe invocation — plus built-in actions for HTTP, DB, files, email, spawn",
5
5
  "main": "index.js",
6
+ "scripts": {
7
+ "test": "node --test --test-concurrency=1 \"test/**/*.test.js\""
8
+ },
6
9
  "files": [
7
10
  "index.js",
8
11
  "lib/"