@transcend-io/cli 4.18.0 → 4.19.0

package/README.md

@@ -25,10 +25,14 @@
     - [Authentication](#authentication-3)
     - [Arguments](#arguments-3)
   - [Usage](#usage-4)
-  - [tr-pull-cron-identifiers](#tr-pull-cron-identifiers)
+  - [tr-cron-pull-identifiers](#tr-cron-pull-identifiers)
     - [Authentication](#authentication-4)
     - [Arguments](#arguments-4)
   - [Usage](#usage-5)
+  - [tr-cron-mark-identifiers-completed](#tr-cron-mark-identifiers-completed)
+    - [Authentication](#authentication-5)
+    - [Arguments](#arguments-5)
+  - [Usage](#usage-6)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
@@ -51,7 +55,7 @@ yarn tr-pull --auth=xxx
 yarn tr-push --auth=xxx
 yarn tr-discover-silos --auth=xxx
 yarn tr-request-upload --auth=xxx
-yarn tr-pull-cron-identifiers --auth=xxx
+yarn tr-cron-pull-identifiers --auth=xxx
 ```
 
 or
@@ -65,7 +69,7 @@ tr-pull --auth=xxx
 tr-push --auth=xxx
 tr-discover-silos --auth=xxx
 tr-request-upload --auth=xxx
-tr-pull-cron-identifiers --auth=xxx
+tr-cron-pull-identifiers --auth=xxx
 ```
 
 alternatively, you can install the cli globally on your machine:
@@ -229,7 +233,7 @@ tr-pull --auth=$TRANSCEND_API_KEY --file=./custom/location.yml
 Or a specific data silo(s) can be pulled in:
 
 ```sh
-tr-pull --auth=$TRANSCEND_API_KEY --dataSiloIds=710fec3c-7bcc-4c9e-baff-bf39f9bec43e
+tr-pull --auth=$TRANSCEND_API_KEY ---dataSiloIds=710fec3c-7bcc-4c9e-baff-bf39f9bec43e
 ```
 
 Or a specific types of data silo(s) can be pulled in:
@@ -406,7 +410,7 @@ Then, you'll need to grab that `dataSiloId` and a Transcend API key and pass it
 
 ```sh
 # Scan a javascript project (package.json files) to look for new data silos
-yarn tr-discover-silos --scanPath=./myJavascriptProject --auth={{api_key}} --dataSiloId=abcdefg
+yarn tr-discover-silos --scanPath=./myJavascriptProject --auth={{api_key}} ---dataSiloId=abcdefg
 ```
 
 This call will look for all the package.json files that in the scan path `./myJavascriptProject`, parse each of the dependencies into their individual package names, and send it to our Transcend backend for classification. These classifications can then be viewed [here](https://app.transcend.io/data-map/data-inventory/silo-discovery/triage). The process is the same for scanning requirements.txt, podfiles and build.gradle files.
@@ -544,7 +548,7 @@ Include debug logs - warning, this logs out personal data.
 yarn tr-request-upload --auth=$TRANSCEND_API_KEY --file=/Users/transcend/Desktop/test.csv --debug=true
 ```
 
-### tr-pull-cron-identifiers
+### tr-cron-pull-identifiers
 
 If you are using the cron job integration, you can run this command to pull the outstanding identifiers
 for the data silo to a CSV.
@@ -574,34 +578,89 @@ The API key must be associated to the ID of the integration/data silo that is be
 ### Usage
 
 ```sh
-yarn tr-pull-cron-identifiers --auth=$TRANSCEND_API_KEY -dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
-   --requestType=ERASURE
+yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE
 ```
 
 Pull to a specific file location
 
 ```sh
-yarn tr-pull-cron-identifiers --auth=$TRANSCEND_API_KEY -dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE \
+yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE \
    --file=/Users/transcend/Desktop/test.csv
 ```
 
 For self-hosted sombras that use an internal key:
 
 ```sh
-yarn tr-pull-cron-identifiers --auth=$TRANSCEND_API_KEY -dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
-   --requestType=ERASURE --sombraAuth=$SOMBRA_INTERNAL_KEY
+yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE  \
+   --sombraAuth=$SOMBRA_INTERNAL_KEY
 ```
 
 Specifying the backend URL, needed for US hosted backend infrastructure.
 
 ```sh
-yarn tr-pull-cron-identifiers --auth=$TRANSCEND_API_KEY --sombraAuth=$SOMBRA_INTERNAL_KEY --file=/Users/transcend/Desktop/test.csv \
+yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE \
  --transcendUrl=https://api.us.transcend.io
 ```
 
 Specifying the page limit, defaults to 100.
 
 ```sh
-yarn tr-pull-cron-identifiers --auth=$TRANSCEND_API_KEY --sombraAuth=$SOMBRA_INTERNAL_KEY --file=/Users/transcend/Desktop/test.csv \
+yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE \
  --pageLimit=300
 ```
+
+### tr-cron-mark-identifiers-completed
+
+This command takes the output of `tr-cron-pull-identifiers` and notifies Transcend that all of the requests in the CSV have been processed.
+This is used in the workflow like:
+
+1. Pull identifiers to CSV:
+   `yarn tr-cron-pull-identifiers --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --requestType=ERASURE --file=./outstanding-requests.csv`
+2. Run your process to operate on that CSV of requests.
+3. Notify Transcend of completion
+   `yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f --file=./outstanding-requests.csv`
+
+Read more at https://docs.transcend.io/docs/integrations/cron-job-integration.
+
+#### Authentication
+
+In order to use this cli, you will first need to generate an API key on the Transcend Admin Dashboard (https://app.transcend.io/infrastructure/api-keys).
+
+The API key must be associated to the ID of the integration/data silo that is being operated on.
+
+#### Arguments
+
+| Argument     | Description                                                                          | Type               | Default                  | Required |
+| ------------ | ------------------------------------------------------------------------------------ | ------------------ | ------------------------ | -------- |
+| auth         | The Transcend API capable of pulling the cron identifiers.                           | string             | N/A                      | true     |
+| dataSiloId   | The ID of the data silo to pull in.                                                  | string - UUID      | N/A                      | true     |
+| file         | Path to the CSV file where identifiers will be written to.                           | string - file-path | ./cron-identifiers.csv   | false    |
+| transcendUrl | URL of the Transcend backend. Use https://api.us.transcend.io for US hosting.        | string - URL       | https://api.transcend.io | false    |
+| sombraAuth   | The sombra internal key, use for additional authentication when self-hosting sombra. | string             | N/A                      | false    |
+
+### Usage
+
+```sh
+yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f
+```
+
+Pull to a specific file location
+
+```sh
+yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
+   --file=/Users/transcend/Desktop/test.csv
+```
+
+For self-hosted sombras that use an internal key:
+
+```sh
+yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
+   --sombraAuth=$SOMBRA_INTERNAL_KEY
+```
+
+Specifying the backend URL, needed for US hosted backend infrastructure.
+
+```sh
+yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
+ --transcendUrl=https://api.us.transcend.io
+```

package/build/cli-cron-mark-identifiers-completed.d.ts

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+export {};
+//# sourceMappingURL=cli-cron-mark-identifiers-completed.d.ts.map

package/build/cli-cron-mark-identifiers-completed.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli-cron-mark-identifiers-completed.d.ts","sourceRoot":"","sources":["../src/cli-cron-mark-identifiers-completed.ts"],"names":[],"mappings":""}

package/build/cli-cron-mark-identifiers-completed.js

@@ -0,0 +1,44 @@
+#!/usr/bin/env node
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const yargs_parser_1 = __importDefault(require("yargs-parser"));
+const colors_1 = __importDefault(require("colors"));
+const logger_1 = require("./logger");
+const cron_1 = require("./cron");
+/**
+ * Mark all of the identifiers in a cron job CSV as completed.
+ *
+ * Requires an API key with scope for the cron integration being checked on.
+ *
+ * Dev Usage:
+ * yarn ts-node ./src/cli-cron-mark-identifiers-completed.ts --auth=asd123 \
+ *   --dataSiloId=92636cda-b7c6-48c6-b1b1-2df574596cbc \
+ *   --file=/Users/michaelfarrell/Desktop/test.csv
+ *
+ * Standard usage:
+ * yarn tr-pull-cron-mark-identifiers-completed --auth=asd123  \
+ *   --dataSiloId=92636cda-b7c6-48c6-b1b1-2df574596cbc \
+ *   --file=/Users/michaelfarrell/Desktop/test.csv
+ */
+async function main() {
+    // Parse command line arguments
+    const { file = './cron-identifiers.csv', transcendUrl = 'https://api.transcend.io', auth, sombraAuth, dataSiloId, } = (0, yargs_parser_1.default)(process.argv.slice(2));
+    // Ensure auth is passed
+    if (!auth) {
+        logger_1.logger.error(colors_1.default.red('A Transcend API key must be provided. You can specify using --auth=asd123'));
+        process.exit(1);
+    }
+    // Upload privacy requests
+    await (0, cron_1.pushCronIdentifiersFromCsv)({
+        file,
+        transcendUrl,
+        auth,
+        sombraAuth,
+        dataSiloId,
+    });
+}
+main();
+//# sourceMappingURL=cli-cron-mark-identifiers-completed.js.map

package/build/cli-cron-mark-identifiers-completed.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli-cron-mark-identifiers-completed.js","sourceRoot":"","sources":["../src/cli-cron-mark-identifiers-completed.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,oDAA4B;AAE5B,qCAAkC;AAClC,iCAAoD;AAEpD;;;;;;;;;;;;;;GAcG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,wBAAwB,EAC/B,YAAY,GAAG,0BAA0B,EACzC,IAAI,EACJ,UAAU,EACV,UAAU,GACX,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAA8B,CAAC;IAE9D,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,0BAA0B;IAC1B,MAAM,IAAA,iCAA0B,EAAC;QAC/B,IAAI;QACJ,YAAY;QACZ,IAAI;QACJ,UAAU;QACV,UAAU;KACX,CAAC,CAAC;AACL,CAAC;AAED,IAAI,EAAE,CAAC"}

package/build/cli-cron-pull-identifiers.d.ts

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+export {};
+//# sourceMappingURL=cli-cron-pull-identifiers.d.ts.map

package/build/cli-cron-pull-identifiers.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli-cron-pull-identifiers.d.ts","sourceRoot":"","sources":["../src/cli-cron-pull-identifiers.ts"],"names":[],"mappings":""}

package/build/{cli-pull-cron-identifiers.js → cli-cron-pull-identifiers.js}

@@ -15,13 +15,13 @@ const privacy_types_1 = require("@transcend-io/privacy-types");
  * Requires an API key with scope for the cron integration being checked on.
  *
  * Dev Usage:
- * yarn ts-node ./src/cli-pull-cron-identifiers.ts --auth=asd123 \
+ * yarn ts-node ./src/cli-cron-pull-identifiers.ts --auth=asd123 \
  *   --dataSiloId=92636cda-b7c6-48c6-b1b1-2df574596cbc \
  *   --requestType=ERASURE \
  *   --file=/Users/michaelfarrell/Desktop/test.csv
  *
  * Standard usage:
- * yarn tr-pull-cron-identifiers --auth=asd123  \
+ * yarn tr-cron-pull-identifiers --auth=asd123  \
  *   --dataSiloId=92636cda-b7c6-48c6-b1b1-2df574596cbc \
  *   --requestType=ERASURE \
  *   --file=/Users/michaelfarrell/Desktop/test.csv
@@ -53,4 +53,4 @@ async function main() {
     });
 }
 main();
-//# sourceMappingURL=cli-pull-cron-identifiers.js.map
+//# sourceMappingURL=cli-cron-pull-identifiers.js.map

package/build/{cli-pull-cron-identifiers.js.map → cli-cron-pull-identifiers.js.map}

@@ -1 +1 @@
-{"version":3,"file":"cli-pull-cron-identifiers.js","sourceRoot":"","sources":["../src/cli-pull-cron-identifiers.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,oDAA4B;AAE5B,qCAAkC;AAClC,iCAAkD;AAClD,+DAA4D;AAE5D;;;;;;;;;;;;;;;;GAgBG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,wBAAwB,EAC/B,YAAY,GAAG,0BAA0B,EACzC,IAAI,EACJ,UAAU,EACV,UAAU,EACV,WAAW,EACX,SAAS,GAAG,KAAK,GAClB,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAA8B,CAAC;IAE9D,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,wBAAwB;IACxB,8DAA8D;IAC9D,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,6BAAa,CAAC,CAAC,QAAQ,CAAC,WAAkB,CAAC,EAAE;QAC9D,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,gCAAgC,WAAW,MAAM;YAC/C,sBAAsB,MAAM,CAAC,MAAM,CAAC,6BAAa,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAClE,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,0BAA0B;IAC1B,MAAM,IAAA,+BAAwB,EAAC;QAC7B,IAAI;QACJ,YAAY;QACZ,SAAS,EAAE,QAAQ,CAAC,SAAS,EAAE,EAAE,CAAC;QAClC,WAAW,EAAE,WAA4B;QACzC,IAAI;QACJ,UAAU;QACV,UAAU;KACX,CAAC,CAAC;AACL,CAAC;AAED,IAAI,EAAE,CAAC"}
+{"version":3,"file":"cli-cron-pull-identifiers.js","sourceRoot":"","sources":["../src/cli-cron-pull-identifiers.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,oDAA4B;AAE5B,qCAAkC;AAClC,iCAAkD;AAClD,+DAA4D;AAE5D;;;;;;;;;;;;;;;;GAgBG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,wBAAwB,EAC/B,YAAY,GAAG,0BAA0B,EACzC,IAAI,EACJ,UAAU,EACV,UAAU,EACV,WAAW,EACX,SAAS,GAAG,KAAK,GAClB,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAA8B,CAAC;IAE9D,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,wBAAwB;IACxB,8DAA8D;IAC9D,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,6BAAa,CAAC,CAAC,QAAQ,CAAC,WAAkB,CAAC,EAAE;QAC9D,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,gCAAgC,WAAW,MAAM;YAC/C,sBAAsB,MAAM,CAAC,MAAM,CAAC,6BAAa,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAClE,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,0BAA0B;IAC1B,MAAM,IAAA,+BAAwB,EAAC;QAC7B,IAAI;QACJ,YAAY;QACZ,SAAS,EAAE,QAAQ,CAAC,SAAS,EAAE,EAAE,CAAC;QAClC,WAAW,EAAE,WAA4B;QACzC,IAAI;QACJ,UAAU;QACV,UAAU;KACX,CAAC,CAAC;AACL,CAAC;AAED,IAAI,EAAE,CAAC"}

package/build/cron/index.d.ts

@@ -1,4 +1,6 @@
 export * from './pullCronIdentifiersToCsv';
 export * from './pullCronPageOfIdentifiers';
 export * from './writeCsv';
+export * from './pushCronIdentifiersFromCsv';
+export * from './markCronIdentifierCompleted';
 //# sourceMappingURL=index.d.ts.map

package/build/cron/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/cron/index.ts"],"names":[],"mappings":"AAAA,cAAc,4BAA4B,CAAC;AAC3C,cAAc,6BAA6B,CAAC;AAC5C,cAAc,YAAY,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/cron/index.ts"],"names":[],"mappings":"AAAA,cAAc,4BAA4B,CAAC;AAC3C,cAAc,6BAA6B,CAAC;AAC5C,cAAc,YAAY,CAAC;AAC3B,cAAc,8BAA8B,CAAC;AAC7C,cAAc,+BAA+B,CAAC"}

package/build/cron/index.js

@@ -13,4 +13,6 @@ Object.defineProperty(exports, "__esModule", { value: true });
 __exportStar(require("./pullCronIdentifiersToCsv"), exports);
 __exportStar(require("./pullCronPageOfIdentifiers"), exports);
 __exportStar(require("./writeCsv"), exports);
+__exportStar(require("./pushCronIdentifiersFromCsv"), exports);
+__exportStar(require("./markCronIdentifierCompleted"), exports);
 //# sourceMappingURL=index.js.map

package/build/cron/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cron/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,6DAA2C;AAC3C,8DAA4C;AAC5C,6CAA2B"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cron/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,6DAA2C;AAC3C,8DAA4C;AAC5C,6CAA2B;AAC3B,+DAA6C;AAC7C,gEAA8C"}

package/build/cron/markCronIdentifierCompleted.d.ts

@@ -0,0 +1,21 @@
+import type { Got } from 'got';
+import * as t from 'io-ts';
+/**
+ * Minimal set required to mark as completed
+ */
+export declare const CronIdentifierPush: t.TypeC<{
+    nonce: t.StringC;
+    identifier: t.StringC;
+}>;
+/** Type override */
+export declare type CronIdentifierPush = t.TypeOf<typeof CronIdentifierPush>;
+/**
+ * Mark an identifier output by the cron job as completed.
+ *
+ * @see https://docs.transcend.io/docs/api-reference/PUT/v1/data-silo
+ * @param sombra - Sombra instance configured to make requests
+ * @param options - Additional options
+ * @returns Successfully submitted request
+ */
+export declare function markCronIdentifierCompleted(sombra: Got, { nonce, identifier }: CronIdentifierPush): Promise<void>;
+//# sourceMappingURL=markCronIdentifierCompleted.d.ts.map

package/build/cron/markCronIdentifierCompleted.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"markCronIdentifierCompleted.d.ts","sourceRoot":"","sources":["../../src/cron/markCronIdentifierCompleted.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,GAAG,EAAE,MAAM,KAAK,CAAC;AAC/B,OAAO,KAAK,CAAC,MAAM,OAAO,CAAC;AAE3B;;GAEG;AACH,eAAO,MAAM,kBAAkB;;;EAG7B,CAAC;AAEH,oBAAoB;AACpB,oBAAY,kBAAkB,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,kBAAkB,CAAC,CAAC;AAErE;;;;;;;GAOG;AACH,wBAAsB,2BAA2B,CAC/C,MAAM,EAAE,GAAG,EACX,EAAE,KAAK,EAAE,UAAU,EAAE,EAAE,kBAAkB,GACxC,OAAO,CAAC,IAAI,CAAC,CAcf"}

package/build/cron/markCronIdentifierCompleted.js

@@ -0,0 +1,55 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    Object.defineProperty(o, k2, { enumerable: true, get: function() { return m[k]; } });
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || function (mod) {
+    if (mod && mod.__esModule) return mod;
+    var result = {};
+    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
+    __setModuleDefault(result, mod);
+    return result;
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.markCronIdentifierCompleted = exports.CronIdentifierPush = void 0;
+const t = __importStar(require("io-ts"));
+/**
+ * Minimal set required to mark as completed
+ */
+exports.CronIdentifierPush = t.type({
+    nonce: t.string,
+    identifier: t.string,
+});
+/**
+ * Mark an identifier output by the cron job as completed.
+ *
+ * @see https://docs.transcend.io/docs/api-reference/PUT/v1/data-silo
+ * @param sombra - Sombra instance configured to make requests
+ * @param options - Additional options
+ * @returns Successfully submitted request
+ */
+async function markCronIdentifierCompleted(sombra, { nonce, identifier }) {
+    // Make the GraphQL request
+    await sombra.put('v1/data-silo', {
+        headers: {
+            'x-transcend-nonce': nonce,
+        },
+        json: {
+            profiles: [
+                {
+                    profileId: identifier,
+                },
+            ],
+        },
+    });
+}
+exports.markCronIdentifierCompleted = markCronIdentifierCompleted;
+//# sourceMappingURL=markCronIdentifierCompleted.js.map

package/build/cron/markCronIdentifierCompleted.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"markCronIdentifierCompleted.js","sourceRoot":"","sources":["../../src/cron/markCronIdentifierCompleted.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;AACA,yCAA2B;AAE3B;;GAEG;AACU,QAAA,kBAAkB,GAAG,CAAC,CAAC,IAAI,CAAC;IACvC,KAAK,EAAE,CAAC,CAAC,MAAM;IACf,UAAU,EAAE,CAAC,CAAC,MAAM;CACrB,CAAC,CAAC;AAKH;;;;;;;GAOG;AACI,KAAK,UAAU,2BAA2B,CAC/C,MAAW,EACX,EAAE,KAAK,EAAE,UAAU,EAAsB;IAEzC,2BAA2B;IAC3B,MAAM,MAAM,CAAC,GAAG,CAAC,cAAc,EAAE;QAC/B,OAAO,EAAE;YACP,mBAAmB,EAAE,KAAK;SAC3B;QACD,IAAI,EAAE;YACJ,QAAQ,EAAE;gBACR;oBACE,SAAS,EAAE,UAAU;iBACtB;aACF;SACF;KACF,CAAC,CAAC;AACL,CAAC;AAjBD,kEAiBC"}

package/build/cron/pullCronPageOfIdentifiers.d.ts

@@ -15,6 +15,7 @@ export declare const CronIdentifier: t.TypeC<{
     /** The request nonce */
     nonce: t.StringC;
     /** The time the request was created */
+    requestCreatedAt: t.StringC;
     /** The number of days until the request is overdue */
     daysUntilOverdue: t.NumberC;
     /** Request attributes */

package/build/cron/pullCronPageOfIdentifiers.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"pullCronPageOfIdentifiers.d.ts","sourceRoot":"","sources":["../../src/cron/pullCronPageOfIdentifiers.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,CAAC,MAAM,OAAO,CAAC;AAE3B,OAAO,EAAE,aAAa,EAAE,MAAM,6BAA6B,CAAC;AAC5D,OAAO,KAAK,EAAE,GAAG,EAAE,MAAM,KAAK,CAAC;AAE/B,eAAO,MAAM,cAAc;IACzB,2BAA2B;;IAE3B,6BAA6B;;IAE7B,yCAAyC;;IAEzC,yCAAyC;;IAEzC,uCAAuC;;IAEvC,wBAAwB;;IAExB,uCAAuC;IAEvC,sDAAsD;;IAEtD,yBAAyB;;;;;EAOzB,CAAC;AAEH,oBAAoB;AACpB,oBAAY,cAAc,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,cAAc,CAAC,CAAC;AAE7D;;;;;;;GAOG;AACH,wBAAsB,yBAAyB,CAC7C,MAAM,EAAE,GAAG,EACX,EACE,UAAU,EACV,KAAW,EACX,MAAU,EACV,WAAW,GACZ,EAAE;IACD,mBAAmB;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,sBAAsB;IACtB,WAAW,EAAE,aAAa,CAAC;IAC3B,uCAAuC;IACvC,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,sBAAsB;IACtB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB,GACA,OAAO,CAAC,cAAc,EAAE,CAAC,CAkB3B"}
+{"version":3,"file":"pullCronPageOfIdentifiers.d.ts","sourceRoot":"","sources":["../../src/cron/pullCronPageOfIdentifiers.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,CAAC,MAAM,OAAO,CAAC;AAE3B,OAAO,EAAE,aAAa,EAAE,MAAM,6BAA6B,CAAC;AAC5D,OAAO,KAAK,EAAE,GAAG,EAAE,MAAM,KAAK,CAAC;AAE/B,eAAO,MAAM,cAAc;IACzB,2BAA2B;;IAE3B,6BAA6B;;IAE7B,yCAAyC;;IAEzC,yCAAyC;;IAEzC,uCAAuC;;IAEvC,wBAAwB;;IAExB,uCAAuC;;IAEvC,sDAAsD;;IAEtD,yBAAyB;;;;;EAOzB,CAAC;AAEH,oBAAoB;AACpB,oBAAY,cAAc,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,cAAc,CAAC,CAAC;AAE7D;;;;;;;GAOG;AACH,wBAAsB,yBAAyB,CAC7C,MAAM,EAAE,GAAG,EACX,EACE,UAAU,EACV,KAAW,EACX,MAAU,EACV,WAAW,GACZ,EAAE;IACD,mBAAmB;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,sBAAsB;IACtB,WAAW,EAAE,aAAa,CAAC;IAC3B,uCAAuC;IACvC,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,sBAAsB;IACtB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB,GACA,OAAO,CAAC,cAAc,EAAE,CAAC,CAkB3B"}

package/build/cron/pullCronPageOfIdentifiers.js

@@ -36,7 +36,7 @@ exports.CronIdentifier = t.type({
     /** The request nonce */
     nonce: t.string,
     /** The time the request was created */
-    // requestCreatedAt: t.string, TODO
+    requestCreatedAt: t.string,
     /** The number of days until the request is overdue */
     daysUntilOverdue: t.number,
     /** Request attributes */

package/build/cron/pullCronPageOfIdentifiers.js.map

@@ -1 +1 @@
-{"version":3,"file":"pullCronPageOfIdentifiers.js","sourceRoot":"","sources":["../../src/cron/pullCronPageOfIdentifiers.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;AAAA,yCAA2B;AAC3B,yDAAuD;AAI1C,QAAA,cAAc,GAAG,CAAC,CAAC,IAAI,CAAC;IACnC,2BAA2B;IAC3B,UAAU,EAAE,CAAC,CAAC,MAAM;IACpB,6BAA6B;IAC7B,IAAI,EAAE,CAAC,CAAC,MAAM;IACd,yCAAyC;IACzC,cAAc,EAAE,CAAC,CAAC,MAAM;IACxB,yCAAyC;IACzC,UAAU,EAAE,CAAC,CAAC,MAAM;IACpB,uCAAuC;IACvC,SAAS,EAAE,CAAC,CAAC,MAAM;IACnB,wBAAwB;IACxB,KAAK,EAAE,CAAC,CAAC,MAAM;IACf,uCAAuC;IACvC,mCAAmC;IACnC,sDAAsD;IACtD,gBAAgB,EAAE,CAAC,CAAC,MAAM;IAC1B,yBAAyB;IACzB,UAAU,EAAE,CAAC,CAAC,KAAK,CACjB,CAAC,CAAC,IAAI,CAAC;QACL,GAAG,EAAE,CAAC,CAAC,MAAM;QACb,MAAM,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC;KAC1B,CAAC,CACH;CACF,CAAC,CAAC;AAKH;;;;;;;GAOG;AACI,KAAK,UAAU,yBAAyB,CAC7C,MAAW,EACX,EACE,UAAU,EACV,KAAK,GAAG,GAAG,EACX,MAAM,GAAG,CAAC,EACV,WAAW,GAUZ;IAED,2BAA2B;IAC3B,MAAM,QAAQ,GAAG,MAAM,MAAM;SAC1B,GAAG,CAAC,gBAAgB,UAAU,qBAAqB,WAAW,EAAE,EAAE;QACjE,YAAY,EAAE;YACZ,MAAM;YACN,KAAK;SACN;KACF,CAAC;SACD,IAAI,EAAE,CAAC;IAEV,MAAM,EAAE,KAAK,EAAE,GAAG,IAAA,wBAAW,EAC3B,CAAC,CAAC,IAAI,CAAC;QACL,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,sBAAc,CAAC;KAC/B,CAAC,EACF,QAAQ,CACT,CAAC;IACF,OAAO,KAAK,CAAC;AACf,CAAC;AAnCD,8DAmCC"}
+{"version":3,"file":"pullCronPageOfIdentifiers.js","sourceRoot":"","sources":["../../src/cron/pullCronPageOfIdentifiers.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;AAAA,yCAA2B;AAC3B,yDAAuD;AAI1C,QAAA,cAAc,GAAG,CAAC,CAAC,IAAI,CAAC;IACnC,2BAA2B;IAC3B,UAAU,EAAE,CAAC,CAAC,MAAM;IACpB,6BAA6B;IAC7B,IAAI,EAAE,CAAC,CAAC,MAAM;IACd,yCAAyC;IACzC,cAAc,EAAE,CAAC,CAAC,MAAM;IACxB,yCAAyC;IACzC,UAAU,EAAE,CAAC,CAAC,MAAM;IACpB,uCAAuC;IACvC,SAAS,EAAE,CAAC,CAAC,MAAM;IACnB,wBAAwB;IACxB,KAAK,EAAE,CAAC,CAAC,MAAM;IACf,uCAAuC;IACvC,gBAAgB,EAAE,CAAC,CAAC,MAAM;IAC1B,sDAAsD;IACtD,gBAAgB,EAAE,CAAC,CAAC,MAAM;IAC1B,yBAAyB;IACzB,UAAU,EAAE,CAAC,CAAC,KAAK,CACjB,CAAC,CAAC,IAAI,CAAC;QACL,GAAG,EAAE,CAAC,CAAC,MAAM;QACb,MAAM,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC;KAC1B,CAAC,CACH;CACF,CAAC,CAAC;AAKH;;;;;;;GAOG;AACI,KAAK,UAAU,yBAAyB,CAC7C,MAAW,EACX,EACE,UAAU,EACV,KAAK,GAAG,GAAG,EACX,MAAM,GAAG,CAAC,EACV,WAAW,GAUZ;IAED,2BAA2B;IAC3B,MAAM,QAAQ,GAAG,MAAM,MAAM;SAC1B,GAAG,CAAC,gBAAgB,UAAU,qBAAqB,WAAW,EAAE,EAAE;QACjE,YAAY,EAAE;YACZ,MAAM;YACN,KAAK;SACN;KACF,CAAC;SACD,IAAI,EAAE,CAAC;IAEV,MAAM,EAAE,KAAK,EAAE,GAAG,IAAA,wBAAW,EAC3B,CAAC,CAAC,IAAI,CAAC;QACL,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,sBAAc,CAAC;KAC/B,CAAC,EACF,QAAQ,CACT,CAAC;IACF,OAAO,KAAK,CAAC;AACf,CAAC;AAnCD,8DAmCC"}

package/build/cron/pushCronIdentifiersFromCsv.d.ts

@@ -0,0 +1,21 @@
+/**
+ * Given a CSV of cron job outputs, mark all requests as completed in Transcend
+ *
+ * @param options - Options
+ * @returns Number of items marked as completed
+ */
+export declare function pushCronIdentifiersFromCsv({ file, dataSiloId, auth, sombraAuth, concurrency, transcendUrl, }: {
+    /** CSV file path */
+    file: string;
+    /** Transcend API key authentication */
+    auth: string;
+    /** Data Silo ID to pull down jobs for */
+    dataSiloId: string;
+    /** Upload concurrency */
+    concurrency?: number;
+    /** API URL for Transcend backend */
+    transcendUrl?: string;
+    /** Sombra API key authentication */
+    sombraAuth?: string;
+}): Promise<number>;
+//# sourceMappingURL=pushCronIdentifiersFromCsv.d.ts.map

package/build/cron/pushCronIdentifiersFromCsv.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"pushCronIdentifiersFromCsv.d.ts","sourceRoot":"","sources":["../../src/cron/pushCronIdentifiersFromCsv.ts"],"names":[],"mappings":"AAUA;;;;;GAKG;AACH,wBAAsB,0BAA0B,CAAC,EAC/C,IAAI,EACJ,UAAU,EACV,IAAI,EACJ,UAAU,EACV,WAAiB,EACjB,YAAyC,GAC1C,EAAE;IACD,oBAAoB;IACpB,IAAI,EAAE,MAAM,CAAC;IACb,uCAAuC;IACvC,IAAI,EAAE,MAAM,CAAC;IACb,yCAAyC;IACzC,UAAU,EAAE,MAAM,CAAC;IACnB,yBAAyB;IACzB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,oCAAoC;IACpC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,oCAAoC;IACpC,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB,GAAG,OAAO,CAAC,MAAM,CAAC,CAwBlB"}

package/build/cron/pushCronIdentifiersFromCsv.js

@@ -0,0 +1,34 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.pushCronIdentifiersFromCsv = void 0;
+const bluebird_1 = require("bluebird");
+const graphql_1 = require("../graphql");
+const colors_1 = __importDefault(require("colors"));
+const markCronIdentifierCompleted_1 = require("./markCronIdentifierCompleted");
+const logger_1 = require("../logger");
+const requests_1 = require("../requests");
+/**
+ * Given a CSV of cron job outputs, mark all requests as completed in Transcend
+ *
+ * @param options - Options
+ * @returns Number of items marked as completed
+ */
+async function pushCronIdentifiersFromCsv({ file, dataSiloId, auth, sombraAuth, concurrency = 100, transcendUrl = 'https://api.transcend.io', }) {
+    // Create sombra instance to communicate with
+    const sombra = await (0, graphql_1.createSombraGotInstance)(transcendUrl, auth, sombraAuth);
+    // Read from CSV
+    logger_1.logger.info(colors_1.default.magenta(`Reading "${file}" from disk`));
+    const activeResults = (0, requests_1.readCsv)(file, markCronIdentifierCompleted_1.CronIdentifierPush);
+    // Notify Transcend
+    logger_1.logger.info(colors_1.default.magenta(`Notifying Transcend for data silo "${dataSiloId}" marking "${activeResults.length}" requests as completed.`));
+    await (0, bluebird_1.map)(activeResults, async (identifier) => {
+        await (0, markCronIdentifierCompleted_1.markCronIdentifierCompleted)(sombra, identifier);
+    }, { concurrency });
+    logger_1.logger.info(colors_1.default.green('Successfully notified Transcend!'));
+    return activeResults.length;
+}
+exports.pushCronIdentifiersFromCsv = pushCronIdentifiersFromCsv;
+//# sourceMappingURL=pushCronIdentifiersFromCsv.js.map

package/build/cron/pushCronIdentifiersFromCsv.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"pushCronIdentifiersFromCsv.js","sourceRoot":"","sources":["../../src/cron/pushCronIdentifiersFromCsv.ts"],"names":[],"mappings":";;;;;;AAAA,uCAA+B;AAC/B,wCAAqD;AACrD,oDAA4B;AAC5B,+EAGuC;AACvC,sCAAmC;AACnC,0CAAsC;AAEtC;;;;;GAKG;AACI,KAAK,UAAU,0BAA0B,CAAC,EAC/C,IAAI,EACJ,UAAU,EACV,IAAI,EACJ,UAAU,EACV,WAAW,GAAG,GAAG,EACjB,YAAY,GAAG,0BAA0B,GAc1C;IACC,6CAA6C;IAC7C,MAAM,MAAM,GAAG,MAAM,IAAA,iCAAuB,EAAC,YAAY,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC;IAE7E,gBAAgB;IAChB,eAAM,CAAC,IAAI,CAAC,gBAAM,CAAC,OAAO,CAAC,YAAY,IAAI,aAAa,CAAC,CAAC,CAAC;IAC3D,MAAM,aAAa,GAAG,IAAA,kBAAO,EAAC,IAAI,EAAE,gDAAkB,CAAC,CAAC;IAExD,mBAAmB;IACnB,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,sCAAsC,UAAU,cAAc,aAAa,CAAC,MAAM,0BAA0B,CAC7G,CACF,CAAC;IACF,MAAM,IAAA,cAAG,EACP,aAAa,EACb,KAAK,EAAE,UAAU,EAAE,EAAE;QACnB,MAAM,IAAA,yDAA2B,EAAC,MAAM,EAAE,UAAU,CAAC,CAAC;IACxD,CAAC,EACD,EAAE,WAAW,EAAE,CAChB,CAAC;IAEF,eAAM,CAAC,IAAI,CAAC,gBAAM,CAAC,KAAK,CAAC,kCAAkC,CAAC,CAAC,CAAC;IAC9D,OAAO,aAAa,CAAC,MAAM,CAAC;AAC9B,CAAC;AA5CD,gEA4CC"}