@transcend-io/cli 4.22.0 → 4.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +63 -0
- package/build/cli-manual-enrichment-pull-identifiers.d.ts +3 -0
- package/build/cli-manual-enrichment-pull-identifiers.d.ts.map +1 -0
- package/build/cli-manual-enrichment-pull-identifiers.js +54 -0
- package/build/cli-manual-enrichment-pull-identifiers.js.map +1 -0
- package/build/cron/pullCronIdentifiersToCsv.d.ts.map +1 -1
- package/build/cron/pullCronIdentifiersToCsv.js +5 -2
- package/build/cron/pullCronIdentifiersToCsv.js.map +1 -1
- package/build/cron/writeCsv.d.ts +2 -1
- package/build/cron/writeCsv.d.ts.map +1 -1
- package/build/cron/writeCsv.js +3 -2
- package/build/cron/writeCsv.js.map +1 -1
- package/build/graphql/fetchAllRequestEnrichers.d.ts +28 -0
- package/build/graphql/fetchAllRequestEnrichers.d.ts.map +1 -0
- package/build/graphql/fetchAllRequestEnrichers.js +34 -0
- package/build/graphql/fetchAllRequestEnrichers.js.map +1 -0
- package/build/graphql/fetchAllRequests.d.ts +16 -0
- package/build/graphql/fetchAllRequests.d.ts.map +1 -1
- package/build/graphql/fetchAllRequests.js.map +1 -1
- package/build/graphql/gqls/RequestEnricher.d.ts +2 -0
- package/build/graphql/gqls/RequestEnricher.d.ts.map +1 -0
- package/build/graphql/gqls/RequestEnricher.js +29 -0
- package/build/graphql/gqls/RequestEnricher.js.map +1 -0
- package/build/graphql/gqls/index.d.ts +1 -0
- package/build/graphql/gqls/index.d.ts.map +1 -1
- package/build/graphql/gqls/index.js +1 -0
- package/build/graphql/gqls/index.js.map +1 -1
- package/build/graphql/gqls/request.d.ts.map +1 -1
- package/build/graphql/gqls/request.js +9 -0
- package/build/graphql/gqls/request.js.map +1 -1
- package/build/graphql/index.d.ts +1 -0
- package/build/graphql/index.d.ts.map +1 -1
- package/build/graphql/index.js +1 -0
- package/build/graphql/index.js.map +1 -1
- package/build/manual-enrichment/index.d.ts +2 -0
- package/build/manual-enrichment/index.d.ts.map +1 -0
- package/build/manual-enrichment/index.js +14 -0
- package/build/manual-enrichment/index.js.map +1 -0
- package/build/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.d.ts +26 -0
- package/build/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.d.ts.map +1 -0
- package/build/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.js +72 -0
- package/build/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.js.map +1 -0
- package/build/tsbuildinfo +1 -1
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -37,6 +37,10 @@
|
|
|
37
37
|
- [Authentication](#authentication-6)
|
|
38
38
|
- [Arguments](#arguments-6)
|
|
39
39
|
- [Usage](#usage-7)
|
|
40
|
+
- [tr-manual-enrichment-pull-identifiers](#tr-manual-enrichment-pull-identifiers)
|
|
41
|
+
- [Authentication](#authentication-7)
|
|
42
|
+
- [Arguments](#arguments-7)
|
|
43
|
+
- [Usage](#usage-8)
|
|
40
44
|
|
|
41
45
|
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
|
|
42
46
|
|
|
@@ -774,3 +778,62 @@ Specifying the backend URL, needed for US hosted backend infrastructure.
|
|
|
774
778
|
yarn tr-cron-mark-identifiers-completed --auth=$TRANSCEND_API_KEY --dataSiloId=70810f2e-cf90-43f6-9776-901a5950599f \
|
|
775
779
|
--transcendUrl=https://api.us.transcend.io
|
|
776
780
|
```
|
|
781
|
+
|
|
782
|
+
### tr-manual-enrichment-pull-identifiers
|
|
783
|
+
|
|
784
|
+
This command pulls down the set of privacy requests that are currently pending manual enrichment. This is useful for the following workflow
|
|
785
|
+
|
|
786
|
+
1. Pull identifiers to CSV:
|
|
787
|
+
`yarn tr-manual-enrichment-pull-identifiers --file=./enrichment-requests.csv`
|
|
788
|
+
2. Fill out the CSV with additional identifiers
|
|
789
|
+
3. Push updated back to Transcend
|
|
790
|
+
`yarn tr-manual-enrichment-push-identifiers --file=./enrichment-requests.csv`
|
|
791
|
+
|
|
792
|
+
#### Authentication
|
|
793
|
+
|
|
794
|
+
In order to use this cli, you will first need to generate an API key on the Transcend Admin Dashboard (https://app.transcend.io/infrastructure/api-keys).
|
|
795
|
+
|
|
796
|
+
The API key must have the following scopes:
|
|
797
|
+
|
|
798
|
+
- View Incoming Requests
|
|
799
|
+
- View the Request Compilation
|
|
800
|
+
|
|
801
|
+
#### Arguments
|
|
802
|
+
|
|
803
|
+
| Argument | Description | Type | Default | Required |
|
|
804
|
+
| ------------ | ----------------------------------------------------------------------------- | ------------------ | ----------------------------------- | -------- |
|
|
805
|
+
| auth | The Transcend API capable of pull request information. | string | N/A | true |
|
|
806
|
+
| transcendUrl | URL of the Transcend backend. Use https://api.us.transcend.io for US hosting. | string - URL | https://api.transcend.io | false |
|
|
807
|
+
| file | Path to the CSV file where requests will be written to. | string - file-path | ./manual-enrichment-identifiers.csv | false |
|
|
808
|
+
| actions | The set of request actions to pull requests for. | RequestAction[] | [] | false |
|
|
809
|
+
| concurrency | The concurrency to use when uploading requests in parallel. | number | 100 | false |
|
|
810
|
+
|
|
811
|
+
### Usage
|
|
812
|
+
|
|
813
|
+
```sh
|
|
814
|
+
yarn tr-manual-enrichment-push-identifiers --auth=$TRANSCEND_API_KEY
|
|
815
|
+
```
|
|
816
|
+
|
|
817
|
+
Pull to a specific file location
|
|
818
|
+
|
|
819
|
+
```sh
|
|
820
|
+
yarn tr-manual-enrichment-push-identifiers --auth=$TRANSCEND_API_KEY --file=/Users/transcend/Desktop/test.csv
|
|
821
|
+
```
|
|
822
|
+
|
|
823
|
+
For specific types of requests
|
|
824
|
+
|
|
825
|
+
```sh
|
|
826
|
+
yarn tr-manual-enrichment-push-identifiers --auth=$TRANSCEND_API_KEY --actions=ACCESS,ERASURE
|
|
827
|
+
```
|
|
828
|
+
|
|
829
|
+
For US hosted infrastructure
|
|
830
|
+
|
|
831
|
+
```sh
|
|
832
|
+
yarn tr-manual-enrichment-push-identifiers --auth=$TRANSCEND_API_KEY --transcendUrl=https://api.us.transcend.io
|
|
833
|
+
```
|
|
834
|
+
|
|
835
|
+
With specific concurrency
|
|
836
|
+
|
|
837
|
+
```sh
|
|
838
|
+
yarn tr-manual-enrichment-push-identifiers --auth=$TRANSCEND_API_KEY --concurrency=200
|
|
839
|
+
```
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cli-manual-enrichment-pull-identifiers.d.ts","sourceRoot":"","sources":["../src/cli-manual-enrichment-pull-identifiers.ts"],"names":[],"mappings":""}
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
"use strict";
|
|
3
|
+
var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
4
|
+
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
5
|
+
};
|
|
6
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
7
|
+
const yargs_parser_1 = __importDefault(require("yargs-parser"));
|
|
8
|
+
const colors_1 = __importDefault(require("colors"));
|
|
9
|
+
const logger_1 = require("./logger");
|
|
10
|
+
const manual_enrichment_1 = require("./manual-enrichment");
|
|
11
|
+
const privacy_types_1 = require("@transcend-io/privacy-types");
|
|
12
|
+
/**
|
|
13
|
+
* Pull the the set of requests that actively require manual enrichment.
|
|
14
|
+
*
|
|
15
|
+
* Requires an API key with scope to View Incoming Requests
|
|
16
|
+
*
|
|
17
|
+
* Dev Usage:
|
|
18
|
+
* yarn ts-node ./src/cli-manual-enrichment-pull-identifiers.ts --auth=asd123 \
|
|
19
|
+
* --actions=ERASURE \
|
|
20
|
+
* --file=/Users/michaelfarrell/Desktop/test.csv
|
|
21
|
+
*
|
|
22
|
+
* Standard usage:
|
|
23
|
+
* yarn tr-manual-enricher-pull-identifiers --auth=asd123 \
|
|
24
|
+
* --actions=ERASURE \
|
|
25
|
+
* --file=/Users/michaelfarrell/Desktop/test.csv
|
|
26
|
+
*/
|
|
27
|
+
async function main() {
|
|
28
|
+
// Parse command line arguments
|
|
29
|
+
const { file = './manual-enrichment-identifiers.csv', transcendUrl = 'https://api.transcend.io', auth, actions = '', concurrency = '100', } = (0, yargs_parser_1.default)(process.argv.slice(2));
|
|
30
|
+
// Ensure auth is passed
|
|
31
|
+
if (!auth) {
|
|
32
|
+
logger_1.logger.error(colors_1.default.red('A Transcend API key must be provided. You can specify using --auth=asd123'));
|
|
33
|
+
process.exit(1);
|
|
34
|
+
}
|
|
35
|
+
// Validate actions
|
|
36
|
+
const requestActions = actions
|
|
37
|
+
.split(',')
|
|
38
|
+
.filter((x) => !!x);
|
|
39
|
+
const invalidActions = requestActions.filter((action) => !Object.values(privacy_types_1.RequestAction).includes(action));
|
|
40
|
+
if (invalidActions.length > 0) {
|
|
41
|
+
logger_1.logger.error(colors_1.default.red(`Received invalid action values: "${invalidActions.join(',')}"`));
|
|
42
|
+
process.exit(1);
|
|
43
|
+
}
|
|
44
|
+
// Pull manual enrichment identifiers
|
|
45
|
+
await (0, manual_enrichment_1.pullManualEnrichmentIdentifiersToCsv)({
|
|
46
|
+
file,
|
|
47
|
+
transcendUrl,
|
|
48
|
+
concurrency: parseInt(concurrency, 10),
|
|
49
|
+
requestActions,
|
|
50
|
+
auth,
|
|
51
|
+
});
|
|
52
|
+
}
|
|
53
|
+
main();
|
|
54
|
+
//# sourceMappingURL=cli-manual-enrichment-pull-identifiers.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cli-manual-enrichment-pull-identifiers.js","sourceRoot":"","sources":["../src/cli-manual-enrichment-pull-identifiers.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,oDAA4B;AAE5B,qCAAkC;AAClC,2DAA2E;AAC3E,+DAA4D;AAE5D;;;;;;;;;;;;;;GAcG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,qCAAqC,EAC5C,YAAY,GAAG,0BAA0B,EACzC,IAAI,EACJ,OAAO,GAAG,EAAE,EACZ,WAAW,GAAG,KAAK,GACpB,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAA8B,CAAC;IAE9D,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,mBAAmB;IACnB,MAAM,cAAc,GAAG,OAAO;SAC3B,KAAK,CAAC,GAAG,CAAC;SACV,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAoB,CAAC;IACzC,MAAM,cAAc,GAAG,cAAc,CAAC,MAAM,CAC1C,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,6BAAa,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAC3D,CAAC;IACF,IAAI,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE;QAC7B,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,oCAAoC,cAAc,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAChE,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,qCAAqC;IACrC,MAAM,IAAA,wDAAoC,EAAC;QACzC,IAAI;QACJ,YAAY;QACZ,WAAW,EAAE,QAAQ,CAAC,WAAW,EAAE,EAAE,CAAC;QACtC,cAAc;QACd,IAAI;KACL,CAAC,CAAC;AACL,CAAC;AAED,IAAI,EAAE,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"pullCronIdentifiersToCsv.d.ts","sourceRoot":"","sources":["../../src/cron/pullCronIdentifiersToCsv.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"pullCronIdentifiersToCsv.d.ts","sourceRoot":"","sources":["../../src/cron/pullCronIdentifiersToCsv.ts"],"names":[],"mappings":"AAGA,OAAO,EAEL,cAAc,EACf,MAAM,6BAA6B,CAAC;AACrC,OAAO,EAAE,aAAa,EAAE,MAAM,6BAA6B,CAAC;AAI5D;;;;GAIG;AACH,wBAAsB,wBAAwB,CAAC,EAC7C,IAAI,EACJ,UAAU,EACV,IAAI,EACJ,UAAU,EACV,WAAW,EACX,SAAe,EACf,YAAyC,GAC1C,EAAE;IACD,oBAAoB;IACpB,IAAI,EAAE,MAAM,CAAC;IACb,uCAAuC;IACvC,IAAI,EAAE,MAAM,CAAC;IACb,yCAAyC;IACzC,UAAU,EAAE,MAAM,CAAC;IACnB,kCAAkC;IAClC,WAAW,EAAE,aAAa,CAAC;IAC3B,2CAA2C;IAC3C,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,oCAAoC;IACpC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,oCAAoC;IACpC,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB,GAAG,OAAO,CAAC,cAAc,EAAE,CAAC,CAuD5B"}
|
|
@@ -6,6 +6,7 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
6
6
|
exports.pullCronIdentifiersToCsv = void 0;
|
|
7
7
|
const graphql_1 = require("../graphql");
|
|
8
8
|
const colors_1 = __importDefault(require("colors"));
|
|
9
|
+
const uniq_1 = __importDefault(require("lodash/uniq"));
|
|
9
10
|
const pullCronPageOfIdentifiers_1 = require("./pullCronPageOfIdentifiers");
|
|
10
11
|
const writeCsv_1 = require("./writeCsv");
|
|
11
12
|
const logger_1 = require("../logger");
|
|
@@ -36,12 +37,14 @@ async function pullCronIdentifiersToCsv({ file, dataSiloId, auth, sombraAuth, re
|
|
|
36
37
|
}
|
|
37
38
|
logger_1.logger.info(colors_1.default.magenta(`Found: ${identifiers.length} outstanding identifiers to parse`));
|
|
38
39
|
// Write out to CSV
|
|
39
|
-
|
|
40
|
+
const data = identifiers.map(({ attributes, ...identifier }) => ({
|
|
40
41
|
...identifier,
|
|
41
42
|
...attributes.reduce((acc, val) => Object.assign(acc, {
|
|
42
43
|
[val.key]: val.values.join(','),
|
|
43
44
|
}), {}),
|
|
44
|
-
}))
|
|
45
|
+
}));
|
|
46
|
+
const headers = (0, uniq_1.default)(data.map((d) => Object.keys(d)).flat());
|
|
47
|
+
(0, writeCsv_1.writeCsv)(file, data, headers);
|
|
45
48
|
logger_1.logger.info(colors_1.default.green(`Successfully wrote ${identifiers.length} identifiers to file "${file}"`));
|
|
46
49
|
return identifiers;
|
|
47
50
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"pullCronIdentifiersToCsv.js","sourceRoot":"","sources":["../../src/cron/pullCronIdentifiersToCsv.ts"],"names":[],"mappings":";;;;;;AAAA,wCAAqD;AACrD,oDAA4B;AAC5B,2EAGqC;AAErC,yCAAsC;AACtC,sCAAmC;AAEnC;;;;GAIG;AACI,KAAK,UAAU,wBAAwB,CAAC,EAC7C,IAAI,EACJ,UAAU,EACV,IAAI,EACJ,UAAU,EACV,WAAW,EACX,SAAS,GAAG,GAAG,EACf,YAAY,GAAG,0BAA0B,GAgB1C;IACC,6CAA6C;IAC7C,MAAM,MAAM,GAAG,MAAM,IAAA,iCAAuB,EAAC,YAAY,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC;IAE7E,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,qCAAqC,UAAU,YAAY,WAAW,EAAE,CACzE,CACF,CAAC;IAEF,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,MAAM,WAAW,GAAqB,EAAE,CAAC;IACzC,IAAI,cAAc,GAAG,IAAI,CAAC;IAE1B,8BAA8B;IAC9B,OAAO,cAAc,EAAE;QACrB,4CAA4C;QAC5C,MAAM,eAAe,GAAG,MAAM,IAAA,qDAAyB,EAAC,MAAM,EAAE;YAC9D,UAAU;YACV,KAAK,EAAE,SAAS;YAChB,MAAM;YACN,WAAW;SACZ,CAAC,CAAC;QACH,WAAW,CAAC,IAAI,CAAC,GAAG,eAAe,CAAC,CAAC;QACrC,cAAc,GAAG,eAAe,CAAC,MAAM,KAAK,SAAS,CAAC;QACtD,MAAM,IAAI,SAAS,CAAC;KACrB;IAED,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,UAAU,WAAW,CAAC,MAAM,mCAAmC,CAChE,CACF,CAAC;IAEF,mBAAmB;IACnB,
|
|
1
|
+
{"version":3,"file":"pullCronIdentifiersToCsv.js","sourceRoot":"","sources":["../../src/cron/pullCronIdentifiersToCsv.ts"],"names":[],"mappings":";;;;;;AAAA,wCAAqD;AACrD,oDAA4B;AAC5B,uDAA+B;AAC/B,2EAGqC;AAErC,yCAAsC;AACtC,sCAAmC;AAEnC;;;;GAIG;AACI,KAAK,UAAU,wBAAwB,CAAC,EAC7C,IAAI,EACJ,UAAU,EACV,IAAI,EACJ,UAAU,EACV,WAAW,EACX,SAAS,GAAG,GAAG,EACf,YAAY,GAAG,0BAA0B,GAgB1C;IACC,6CAA6C;IAC7C,MAAM,MAAM,GAAG,MAAM,IAAA,iCAAuB,EAAC,YAAY,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC;IAE7E,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,qCAAqC,UAAU,YAAY,WAAW,EAAE,CACzE,CACF,CAAC;IAEF,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,MAAM,WAAW,GAAqB,EAAE,CAAC;IACzC,IAAI,cAAc,GAAG,IAAI,CAAC;IAE1B,8BAA8B;IAC9B,OAAO,cAAc,EAAE;QACrB,4CAA4C;QAC5C,MAAM,eAAe,GAAG,MAAM,IAAA,qDAAyB,EAAC,MAAM,EAAE;YAC9D,UAAU;YACV,KAAK,EAAE,SAAS;YAChB,MAAM;YACN,WAAW;SACZ,CAAC,CAAC;QACH,WAAW,CAAC,IAAI,CAAC,GAAG,eAAe,CAAC,CAAC;QACrC,cAAc,GAAG,eAAe,CAAC,MAAM,KAAK,SAAS,CAAC;QACtD,MAAM,IAAI,SAAS,CAAC;KACrB;IAED,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,UAAU,WAAW,CAAC,MAAM,mCAAmC,CAChE,CACF,CAAC;IAEF,mBAAmB;IACnB,MAAM,IAAI,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC,EAAE,UAAU,EAAE,GAAG,UAAU,EAAE,EAAE,EAAE,CAAC,CAAC;QAC/D,GAAG,UAAU;QACb,GAAG,UAAU,CAAC,MAAM,CAClB,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CACX,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE;YACjB,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC;SAChC,CAAC,EACJ,EAAE,CACH;KACF,CAAC,CAAC,CAAC;IACJ,MAAM,OAAO,GAAG,IAAA,cAAI,EAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;IAC7D,IAAA,mBAAQ,EAAC,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;IAE9B,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,KAAK,CACV,sBAAsB,WAAW,CAAC,MAAM,yBAAyB,IAAI,GAAG,CACzE,CACF,CAAC;IAEF,OAAO,WAAW,CAAC;AACrB,CAAC;AA9ED,4DA8EC"}
|
package/build/cron/writeCsv.d.ts
CHANGED
|
@@ -4,6 +4,7 @@ import { ObjByString } from '@transcend-io/type-utils';
|
|
|
4
4
|
*
|
|
5
5
|
* @param filePath - File to write out to
|
|
6
6
|
* @param data - Data to write
|
|
7
|
+
* @param headers - Headers
|
|
7
8
|
*/
|
|
8
|
-
export declare function writeCsv(filePath: string, data: ObjByString[]): Promise<void>;
|
|
9
|
+
export declare function writeCsv(filePath: string, data: ObjByString[], headers?: boolean | string[]): Promise<void>;
|
|
9
10
|
//# sourceMappingURL=writeCsv.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"writeCsv.d.ts","sourceRoot":"","sources":["../../src/cron/writeCsv.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAEvD
|
|
1
|
+
{"version":3,"file":"writeCsv.d.ts","sourceRoot":"","sources":["../../src/cron/writeCsv.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAEvD;;;;;;GAMG;AACH,wBAAsB,QAAQ,CAC5B,QAAQ,EAAE,MAAM,EAChB,IAAI,EAAE,WAAW,EAAE,EACnB,OAAO,GAAE,OAAO,GAAG,MAAM,EAAS,GACjC,OAAO,CAAC,IAAI,CAAC,CAaf"}
|
package/build/cron/writeCsv.js
CHANGED
|
@@ -27,13 +27,14 @@ const fs_1 = require("fs");
|
|
|
27
27
|
*
|
|
28
28
|
* @param filePath - File to write out to
|
|
29
29
|
* @param data - Data to write
|
|
30
|
+
* @param headers - Headers
|
|
30
31
|
*/
|
|
31
|
-
async function writeCsv(filePath, data) {
|
|
32
|
+
async function writeCsv(filePath, data, headers = true) {
|
|
32
33
|
const ws = (0, fs_1.createWriteStream)(filePath);
|
|
33
34
|
await new Promise((resolve, reject) => {
|
|
34
35
|
try {
|
|
35
36
|
fastcsv
|
|
36
|
-
.write(data, { headers
|
|
37
|
+
.write(data, { headers })
|
|
37
38
|
.pipe(ws)
|
|
38
39
|
.on('error', reject)
|
|
39
40
|
.on('end', () => resolve(true));
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"writeCsv.js","sourceRoot":"","sources":["../../src/cron/writeCsv.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;AAAA,kDAAoC;AACpC,2BAAuC;AAIvC
|
|
1
|
+
{"version":3,"file":"writeCsv.js","sourceRoot":"","sources":["../../src/cron/writeCsv.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;AAAA,kDAAoC;AACpC,2BAAuC;AAIvC;;;;;;GAMG;AACI,KAAK,UAAU,QAAQ,CAC5B,QAAgB,EAChB,IAAmB,EACnB,UAA8B,IAAI;IAElC,MAAM,EAAE,GAAG,IAAA,sBAAiB,EAAC,QAAQ,CAAC,CAAC;IACvC,MAAM,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;QACpC,IAAI;YACF,OAAO;iBACJ,KAAK,CAAC,IAAI,EAAE,EAAE,OAAO,EAAE,CAAC;iBACxB,IAAI,CAAC,EAAE,CAAC;iBACR,EAAE,CAAC,OAAO,EAAE,MAAM,CAAC;iBACnB,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;SACnC;QAAC,OAAO,GAAG,EAAE;YACZ,MAAM,CAAC,GAAG,CAAC,CAAC;SACb;IACH,CAAC,CAAC,CAAC;AACL,CAAC;AAjBD,4BAiBC"}
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
import { GraphQLClient } from 'graphql-request';
|
|
2
|
+
export interface RequestEnricher {
|
|
3
|
+
/** ID of request */
|
|
4
|
+
id: string;
|
|
5
|
+
/** Name of identifier */
|
|
6
|
+
enricher: {
|
|
7
|
+
/** ID of enricher */
|
|
8
|
+
id: string;
|
|
9
|
+
/** Title of enricher */
|
|
10
|
+
title: string;
|
|
11
|
+
/** Typeof of enricher */
|
|
12
|
+
type: string;
|
|
13
|
+
};
|
|
14
|
+
/** The status of the enricher */
|
|
15
|
+
status: string;
|
|
16
|
+
}
|
|
17
|
+
/**
|
|
18
|
+
* Fetch all request enrichers for a particular request
|
|
19
|
+
*
|
|
20
|
+
* @param client - GraphQL client
|
|
21
|
+
* @param options - Filter options
|
|
22
|
+
* @returns List of request enrichers
|
|
23
|
+
*/
|
|
24
|
+
export declare function fetchAllRequestEnrichers(client: GraphQLClient, { requestId, }: {
|
|
25
|
+
/** ID of request to filter on */
|
|
26
|
+
requestId: string;
|
|
27
|
+
}): Promise<RequestEnricher[]>;
|
|
28
|
+
//# sourceMappingURL=fetchAllRequestEnrichers.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"fetchAllRequestEnrichers.d.ts","sourceRoot":"","sources":["../../src/graphql/fetchAllRequestEnrichers.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAIhD,MAAM,WAAW,eAAe;IAC9B,oBAAoB;IACpB,EAAE,EAAE,MAAM,CAAC;IACX,yBAAyB;IACzB,QAAQ,EAAE;QACR,qBAAqB;QACrB,EAAE,EAAE,MAAM,CAAC;QACX,wBAAwB;QACxB,KAAK,EAAE,MAAM,CAAC;QACd,yBAAyB;QACzB,IAAI,EAAE,MAAM,CAAC;KACd,CAAC;IACF,iCAAiC;IACjC,MAAM,EAAE,MAAM,CAAC;CAChB;AAID;;;;;;GAMG;AACH,wBAAsB,wBAAwB,CAC5C,MAAM,EAAE,aAAa,EACrB,EACE,SAAS,GACV,EAAE;IACD,iCAAiC;IACjC,SAAS,EAAE,MAAM,CAAC;CACnB,GACA,OAAO,CAAC,eAAe,EAAE,CAAC,CAqB5B"}
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.fetchAllRequestEnrichers = void 0;
|
|
4
|
+
const gqls_1 = require("./gqls");
|
|
5
|
+
const makeGraphQLRequest_1 = require("./makeGraphQLRequest");
|
|
6
|
+
const PAGE_SIZE = 50;
|
|
7
|
+
/**
|
|
8
|
+
* Fetch all request enrichers for a particular request
|
|
9
|
+
*
|
|
10
|
+
* @param client - GraphQL client
|
|
11
|
+
* @param options - Filter options
|
|
12
|
+
* @returns List of request enrichers
|
|
13
|
+
*/
|
|
14
|
+
async function fetchAllRequestEnrichers(client, { requestId, }) {
|
|
15
|
+
const requestEnrichers = [];
|
|
16
|
+
let offset = 0;
|
|
17
|
+
// Paginate
|
|
18
|
+
let shouldContinue = false;
|
|
19
|
+
do {
|
|
20
|
+
const { requestEnrichers: { nodes },
|
|
21
|
+
// eslint-disable-next-line no-await-in-loop
|
|
22
|
+
} = await (0, makeGraphQLRequest_1.makeGraphQLRequest)(client, gqls_1.REQUEST_ENRICHERS, {
|
|
23
|
+
first: PAGE_SIZE,
|
|
24
|
+
offset,
|
|
25
|
+
requestId,
|
|
26
|
+
});
|
|
27
|
+
requestEnrichers.push(...nodes);
|
|
28
|
+
offset += PAGE_SIZE;
|
|
29
|
+
shouldContinue = nodes.length === PAGE_SIZE;
|
|
30
|
+
} while (shouldContinue);
|
|
31
|
+
return requestEnrichers;
|
|
32
|
+
}
|
|
33
|
+
exports.fetchAllRequestEnrichers = fetchAllRequestEnrichers;
|
|
34
|
+
//# sourceMappingURL=fetchAllRequestEnrichers.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"fetchAllRequestEnrichers.js","sourceRoot":"","sources":["../../src/graphql/fetchAllRequestEnrichers.ts"],"names":[],"mappings":";;;AACA,iCAA2C;AAC3C,6DAA0D;AAkB1D,MAAM,SAAS,GAAG,EAAE,CAAC;AAErB;;;;;;GAMG;AACI,KAAK,UAAU,wBAAwB,CAC5C,MAAqB,EACrB,EACE,SAAS,GAIV;IAED,MAAM,gBAAgB,GAAsB,EAAE,CAAC;IAC/C,IAAI,MAAM,GAAG,CAAC,CAAC;IAEf,WAAW;IACX,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,GAAG;QACD,MAAM,EACJ,gBAAgB,EAAE,EAAE,KAAK,EAAE;QAC3B,4CAA4C;UAC7C,GAAG,MAAM,IAAA,uCAAkB,EAAC,MAAM,EAAE,wBAAiB,EAAE;YACtD,KAAK,EAAE,SAAS;YAChB,MAAM;YACN,SAAS;SACV,CAAC,CAAC;QACH,gBAAgB,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,CAAC;QAChC,MAAM,IAAI,SAAS,CAAC;QACpB,cAAc,GAAG,KAAK,CAAC,MAAM,KAAK,SAAS,CAAC;KAC7C,QAAQ,cAAc,EAAE;IAEzB,OAAO,gBAAgB,CAAC;AAC1B,CAAC;AA7BD,4DA6BC"}
|
|
@@ -22,12 +22,28 @@ export interface PrivacyRequest {
|
|
|
22
22
|
coreIdentifier: string;
|
|
23
23
|
/** Type of request action */
|
|
24
24
|
type: RequestAction;
|
|
25
|
+
/** STatus of request action */
|
|
26
|
+
status: RequestStatus;
|
|
25
27
|
/** Type of data subject */
|
|
26
28
|
subjectType: string;
|
|
27
29
|
/** Country of request */
|
|
28
30
|
country?: IsoCountryCode | null;
|
|
29
31
|
/** Sub division of request */
|
|
30
32
|
countrySubDivision?: IsoCountrySubdivisionCode | null;
|
|
33
|
+
/** Attribute values */
|
|
34
|
+
attributeValues: {
|
|
35
|
+
/** ID of value */
|
|
36
|
+
id: string;
|
|
37
|
+
/** Name of value */
|
|
38
|
+
name: string;
|
|
39
|
+
/** Attribute key */
|
|
40
|
+
attributeKey: {
|
|
41
|
+
/** ID of key */
|
|
42
|
+
id: string;
|
|
43
|
+
/** Name of key */
|
|
44
|
+
name: string;
|
|
45
|
+
};
|
|
46
|
+
}[];
|
|
31
47
|
}
|
|
32
48
|
/**
|
|
33
49
|
* Fetch all requests matching a set of filters
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"fetchAllRequests.d.ts","sourceRoot":"","sources":["../../src/graphql/fetchAllRequests.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAIhD,OAAO,EACL,aAAa,EACb,aAAa,EACb,cAAc,EACd,yBAAyB,EAC1B,MAAM,6BAA6B,CAAC;AAErC,OAAO,EAAE,WAAW,EAAE,MAAM,oCAAoC,CAAC;AAEjE,MAAM,WAAW,cAAc;IAC7B,oBAAoB;IACpB,EAAE,EAAE,MAAM,CAAC;IACX,4BAA4B;IAC5B,SAAS,EAAE,MAAM,CAAC;IAClB,uBAAuB;IACvB,KAAK,EAAE,MAAM,CAAC;IACd,uBAAuB;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,sCAAsC;IACtC,MAAM,EAAE,OAAO,CAAC;IAChB,sBAAsB;IACtB,OAAO,EAAE,MAAM,CAAC;IAChB,wBAAwB;IACxB,MAAM,EAAE,WAAW,CAAC;IACpB,wCAAwC;IACxC,QAAQ,EAAE,OAAO,CAAC;IAClB,iCAAiC;IACjC,cAAc,EAAE,MAAM,CAAC;IACvB,6BAA6B;IAC7B,IAAI,EAAE,aAAa,CAAC;IACpB,2BAA2B;IAC3B,WAAW,EAAE,MAAM,CAAC;IACpB,yBAAyB;IACzB,OAAO,CAAC,EAAE,cAAc,GAAG,IAAI,CAAC;IAChC,8BAA8B;IAC9B,kBAAkB,CAAC,EAAE,yBAAyB,GAAG,IAAI,CAAC;
|
|
1
|
+
{"version":3,"file":"fetchAllRequests.d.ts","sourceRoot":"","sources":["../../src/graphql/fetchAllRequests.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAIhD,OAAO,EACL,aAAa,EACb,aAAa,EACb,cAAc,EACd,yBAAyB,EAC1B,MAAM,6BAA6B,CAAC;AAErC,OAAO,EAAE,WAAW,EAAE,MAAM,oCAAoC,CAAC;AAEjE,MAAM,WAAW,cAAc;IAC7B,oBAAoB;IACpB,EAAE,EAAE,MAAM,CAAC;IACX,4BAA4B;IAC5B,SAAS,EAAE,MAAM,CAAC;IAClB,uBAAuB;IACvB,KAAK,EAAE,MAAM,CAAC;IACd,uBAAuB;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,sCAAsC;IACtC,MAAM,EAAE,OAAO,CAAC;IAChB,sBAAsB;IACtB,OAAO,EAAE,MAAM,CAAC;IAChB,wBAAwB;IACxB,MAAM,EAAE,WAAW,CAAC;IACpB,wCAAwC;IACxC,QAAQ,EAAE,OAAO,CAAC;IAClB,iCAAiC;IACjC,cAAc,EAAE,MAAM,CAAC;IACvB,6BAA6B;IAC7B,IAAI,EAAE,aAAa,CAAC;IACpB,+BAA+B;IAC/B,MAAM,EAAE,aAAa,CAAC;IACtB,2BAA2B;IAC3B,WAAW,EAAE,MAAM,CAAC;IACpB,yBAAyB;IACzB,OAAO,CAAC,EAAE,cAAc,GAAG,IAAI,CAAC;IAChC,8BAA8B;IAC9B,kBAAkB,CAAC,EAAE,yBAAyB,GAAG,IAAI,CAAC;IACtD,uBAAuB;IACvB,eAAe,EAAE;QACf,kBAAkB;QAClB,EAAE,EAAE,MAAM,CAAC;QACX,oBAAoB;QACpB,IAAI,EAAE,MAAM,CAAC;QACb,oBAAoB;QACpB,YAAY,EAAE;YACZ,gBAAgB;YAChB,EAAE,EAAE,MAAM,CAAC;YACX,kBAAkB;YAClB,IAAI,EAAE,MAAM,CAAC;SACd,CAAC;KACH,EAAE,CAAC;CACL;AAID;;;;;;GAMG;AACH,wBAAsB,gBAAgB,CACpC,MAAM,EAAE,aAAa,EACrB,EACE,OAAO,EACP,QAAQ,GACT,EAAE;IACD,2BAA2B;IAC3B,OAAO,EAAE,aAAa,EAAE,CAAC;IACzB,4BAA4B;IAC5B,QAAQ,EAAE,aAAa,EAAE,CAAC;CAC3B,GACA,OAAO,CAAC,cAAc,EAAE,CAAC,CA0B3B"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"fetchAllRequests.js","sourceRoot":"","sources":["../../src/graphql/fetchAllRequests.ts"],"names":[],"mappings":";;;;;;AACA,oDAA4B;AAC5B,iCAAkC;AAClC,6DAA0D;AAO1D,sCAAmC;
|
|
1
|
+
{"version":3,"file":"fetchAllRequests.js","sourceRoot":"","sources":["../../src/graphql/fetchAllRequests.ts"],"names":[],"mappings":";;;;;;AACA,oDAA4B;AAC5B,iCAAkC;AAClC,6DAA0D;AAO1D,sCAAmC;AAgDnC,MAAM,SAAS,GAAG,EAAE,CAAC;AAErB;;;;;;GAMG;AACI,KAAK,UAAU,gBAAgB,CACpC,MAAqB,EACrB,EACE,OAAO,EACP,QAAQ,GAMT;IAED,MAAM,QAAQ,GAAqB,EAAE,CAAC;IACtC,IAAI,MAAM,GAAG,CAAC,CAAC;IAEf,WAAW;IACX,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,GAAG;QACD,MAAM,EACJ,QAAQ,EAAE,EAAE,KAAK,EAAE,UAAU,EAAE;QAC/B,4CAA4C;UAC7C,GAAG,MAAM,IAAA,uCAAkB,EAAC,MAAM,EAAE,eAAQ,EAAE;YAC7C,KAAK,EAAE,SAAS;YAChB,MAAM;YACN,OAAO;YACP,QAAQ;SACT,CAAC,CAAC;QACH,IAAI,MAAM,KAAK,CAAC,IAAI,UAAU,GAAG,SAAS,EAAE;YAC1C,eAAM,CAAC,IAAI,CAAC,gBAAM,CAAC,OAAO,CAAC,YAAY,UAAU,WAAW,CAAC,CAAC,CAAC;SAChE;QAED,QAAQ,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,CAAC;QACxB,MAAM,IAAI,SAAS,CAAC;QACpB,cAAc,GAAG,KAAK,CAAC,MAAM,KAAK,SAAS,CAAC;KAC7C,QAAQ,cAAc,EAAE;IAEzB,OAAO,QAAQ,CAAC;AAClB,CAAC;AArCD,4CAqCC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"RequestEnricher.d.ts","sourceRoot":"","sources":["../../../src/graphql/gqls/RequestEnricher.ts"],"names":[],"mappings":"AAEA,eAAO,MAAM,iBAAiB,QAuB7B,CAAC"}
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.REQUEST_ENRICHERS = void 0;
|
|
4
|
+
const graphql_request_1 = require("graphql-request");
|
|
5
|
+
exports.REQUEST_ENRICHERS = (0, graphql_request_1.gql) `
|
|
6
|
+
query TranscendCliRequestEnrichers(
|
|
7
|
+
$first: Int!
|
|
8
|
+
$offset: Int!
|
|
9
|
+
$requestId: ID!
|
|
10
|
+
) {
|
|
11
|
+
requestEnrichers(
|
|
12
|
+
input: { requestId: $requestId }
|
|
13
|
+
first: $first
|
|
14
|
+
offset: $offset
|
|
15
|
+
) {
|
|
16
|
+
nodes {
|
|
17
|
+
id
|
|
18
|
+
status
|
|
19
|
+
enricher {
|
|
20
|
+
id
|
|
21
|
+
type
|
|
22
|
+
title
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
totalCount
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
`;
|
|
29
|
+
//# sourceMappingURL=RequestEnricher.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"RequestEnricher.js","sourceRoot":"","sources":["../../../src/graphql/gqls/RequestEnricher.ts"],"names":[],"mappings":";;;AAAA,qDAAsC;AAEzB,QAAA,iBAAiB,GAAG,IAAA,qBAAG,EAAA;;;;;;;;;;;;;;;;;;;;;;;CAuBnC,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/graphql/gqls/index.ts"],"names":[],"mappings":"AAAA,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,YAAY,CAAC;AAC3B,cAAc,cAAc,CAAC;AAC7B,cAAc,eAAe,CAAC;AAC9B,cAAc,UAAU,CAAC;AACzB,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,WAAW,CAAC;AAC1B,cAAc,qBAAqB,CAAC"}
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/graphql/gqls/index.ts"],"names":[],"mappings":"AAAA,cAAc,aAAa,CAAC;AAC5B,cAAc,YAAY,CAAC;AAC3B,cAAc,YAAY,CAAC;AAC3B,cAAc,cAAc,CAAC;AAC7B,cAAc,eAAe,CAAC;AAC9B,cAAc,UAAU,CAAC;AACzB,cAAc,iBAAiB,CAAC;AAChC,cAAc,YAAY,CAAC;AAC3B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,WAAW,CAAC;AAC1B,cAAc,qBAAqB,CAAC;AACpC,cAAc,mBAAmB,CAAC"}
|
|
@@ -22,4 +22,5 @@ __exportStar(require("./organization"), exports);
|
|
|
22
22
|
__exportStar(require("./attributeKey"), exports);
|
|
23
23
|
__exportStar(require("./request"), exports);
|
|
24
24
|
__exportStar(require("./RequestIdentifier"), exports);
|
|
25
|
+
__exportStar(require("./RequestEnricher"), exports);
|
|
25
26
|
//# sourceMappingURL=index.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/graphql/gqls/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,8CAA4B;AAC5B,6CAA2B;AAC3B,6CAA2B;AAC3B,+CAA6B;AAC7B,gDAA8B;AAC9B,2CAAyB;AACzB,kDAAgC;AAChC,6CAA2B;AAC3B,iDAA+B;AAC/B,iDAA+B;AAC/B,4CAA0B;AAC1B,sDAAoC"}
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/graphql/gqls/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,8CAA4B;AAC5B,6CAA2B;AAC3B,6CAA2B;AAC3B,+CAA6B;AAC7B,gDAA8B;AAC9B,2CAAyB;AACzB,kDAAgC;AAChC,6CAA2B;AAC3B,iDAA+B;AAC/B,iDAA+B;AAC/B,4CAA0B;AAC1B,sDAAoC;AACpC,oDAAkC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"request.d.ts","sourceRoot":"","sources":["../../../src/graphql/gqls/request.ts"],"names":[],"mappings":"AAEA,eAAO,MAAM,QAAQ,
|
|
1
|
+
{"version":3,"file":"request.d.ts","sourceRoot":"","sources":["../../../src/graphql/gqls/request.ts"],"names":[],"mappings":"AAEA,eAAO,MAAM,QAAQ,QAuCpB,CAAC"}
|
|
@@ -19,6 +19,7 @@ exports.REQUESTS = (0, graphql_request_1.gql) `
|
|
|
19
19
|
createdAt
|
|
20
20
|
email
|
|
21
21
|
link
|
|
22
|
+
status
|
|
22
23
|
details
|
|
23
24
|
isTest
|
|
24
25
|
locale
|
|
@@ -28,6 +29,14 @@ exports.REQUESTS = (0, graphql_request_1.gql) `
|
|
|
28
29
|
subjectType
|
|
29
30
|
country
|
|
30
31
|
countrySubDivision
|
|
32
|
+
attributeValues {
|
|
33
|
+
id
|
|
34
|
+
name
|
|
35
|
+
attributeKey {
|
|
36
|
+
id
|
|
37
|
+
name
|
|
38
|
+
}
|
|
39
|
+
}
|
|
31
40
|
}
|
|
32
41
|
totalCount
|
|
33
42
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"request.js","sourceRoot":"","sources":["../../../src/graphql/gqls/request.ts"],"names":[],"mappings":";;;AAAA,qDAAsC;AAEzB,QAAA,QAAQ,GAAG,IAAA,qBAAG,EAAA
|
|
1
|
+
{"version":3,"file":"request.js","sourceRoot":"","sources":["../../../src/graphql/gqls/request.ts"],"names":[],"mappings":";;;AAAA,qDAAsC;AAEzB,QAAA,QAAQ,GAAG,IAAA,qBAAG,EAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAuC1B,CAAC"}
|
package/build/graphql/index.d.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/graphql/index.ts"],"names":[],"mappings":"AAAA,cAAc,8BAA8B,CAAC;AAC7C,cAAc,gBAAgB,CAAC;AAC/B,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,gCAAgC,CAAC;AAC/C,cAAc,iBAAiB,CAAC;AAChC,cAAc,iBAAiB,CAAC;AAChC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,iBAAiB,CAAC;AAChC,cAAc,sBAAsB,CAAC;AACrC,cAAc,2BAA2B,CAAC;AAC1C,cAAc,+BAA+B,CAAC;AAC9C,cAAc,QAAQ,CAAC;AACvB,cAAc,yBAAyB,CAAC;AACxC,cAAc,oBAAoB,CAAC;AACnC,cAAc,8BAA8B,CAAC"}
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/graphql/index.ts"],"names":[],"mappings":"AAAA,cAAc,8BAA8B,CAAC;AAC7C,cAAc,gBAAgB,CAAC;AAC/B,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,gCAAgC,CAAC;AAC/C,cAAc,iBAAiB,CAAC;AAChC,cAAc,iBAAiB,CAAC;AAChC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,iBAAiB,CAAC;AAChC,cAAc,sBAAsB,CAAC;AACrC,cAAc,2BAA2B,CAAC;AAC1C,cAAc,+BAA+B,CAAC;AAC9C,cAAc,QAAQ,CAAC;AACvB,cAAc,yBAAyB,CAAC;AACxC,cAAc,oBAAoB,CAAC;AACnC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,4BAA4B,CAAC"}
|
package/build/graphql/index.js
CHANGED
|
@@ -27,4 +27,5 @@ __exportStar(require("./gqls"), exports);
|
|
|
27
27
|
__exportStar(require("./fetchAllAttributeKeys"), exports);
|
|
28
28
|
__exportStar(require("./fetchAllRequests"), exports);
|
|
29
29
|
__exportStar(require("./fetchAllRequestIdentifiers"), exports);
|
|
30
|
+
__exportStar(require("./fetchAllRequestEnrichers"), exports);
|
|
30
31
|
//# sourceMappingURL=index.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/graphql/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,+DAA6C;AAC7C,iDAA+B;AAC/B,sDAAoC;AACpC,qDAAmC;AACnC,+DAA6C;AAC7C,iEAA+C;AAC/C,kDAAgC;AAChC,kDAAgC;AAChC,+DAA6C;AAC7C,kDAAgC;AAChC,uDAAqC;AACrC,4DAA0C;AAC1C,gEAA8C;AAC9C,yCAAuB;AACvB,0DAAwC;AACxC,qDAAmC;AACnC,+DAA6C"}
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/graphql/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,+DAA6C;AAC7C,iDAA+B;AAC/B,sDAAoC;AACpC,qDAAmC;AACnC,+DAA6C;AAC7C,iEAA+C;AAC/C,kDAAgC;AAChC,kDAAgC;AAChC,+DAA6C;AAC7C,kDAAgC;AAChC,uDAAqC;AACrC,4DAA0C;AAC1C,gEAA8C;AAC9C,yCAAuB;AACvB,0DAAwC;AACxC,qDAAmC;AACnC,+DAA6C;AAC7C,6DAA2C"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/manual-enrichment/index.ts"],"names":[],"mappings":"AAAA,cAAc,wCAAwC,CAAC"}
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
Object.defineProperty(o, k2, { enumerable: true, get: function() { return m[k]; } });
|
|
5
|
+
}) : (function(o, m, k, k2) {
|
|
6
|
+
if (k2 === undefined) k2 = k;
|
|
7
|
+
o[k2] = m[k];
|
|
8
|
+
}));
|
|
9
|
+
var __exportStar = (this && this.__exportStar) || function(m, exports) {
|
|
10
|
+
for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
|
|
11
|
+
};
|
|
12
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
13
|
+
__exportStar(require("./pullManualEnrichmentIdentifiersToCsv"), exports);
|
|
14
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/manual-enrichment/index.ts"],"names":[],"mappings":";;;;;;;;;;;;AAAA,yEAAuD"}
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import { RequestEnricher, RequestIdentifier, PrivacyRequest } from '../graphql';
|
|
2
|
+
import { RequestAction } from '@transcend-io/privacy-types';
|
|
3
|
+
export interface PrivacyRequestWithIdentifiers extends PrivacyRequest {
|
|
4
|
+
/** Request Enrichers */
|
|
5
|
+
requestEnrichers: RequestEnricher[];
|
|
6
|
+
/** Request Identifiers */
|
|
7
|
+
requestIdentifiers: RequestIdentifier[];
|
|
8
|
+
}
|
|
9
|
+
/**
|
|
10
|
+
* Pull the set of manual enrichment jobs to CSV
|
|
11
|
+
*
|
|
12
|
+
* @param options - Options
|
|
13
|
+
*/
|
|
14
|
+
export declare function pullManualEnrichmentIdentifiersToCsv({ file, auth, requestActions, concurrency, transcendUrl, }: {
|
|
15
|
+
/** CSV file path */
|
|
16
|
+
file: string;
|
|
17
|
+
/** Transcend API key authentication */
|
|
18
|
+
auth: string;
|
|
19
|
+
/** Concurrency */
|
|
20
|
+
concurrency?: number;
|
|
21
|
+
/** The request actions to fetch */
|
|
22
|
+
requestActions: RequestAction[];
|
|
23
|
+
/** API URL for Transcend backend */
|
|
24
|
+
transcendUrl?: string;
|
|
25
|
+
}): Promise<PrivacyRequestWithIdentifiers[]>;
|
|
26
|
+
//# sourceMappingURL=pullManualEnrichmentIdentifiersToCsv.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"pullManualEnrichmentIdentifiersToCsv.d.ts","sourceRoot":"","sources":["../../src/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,eAAe,EAEf,iBAAiB,EACjB,cAAc,EAGf,MAAM,YAAY,CAAC;AAKpB,OAAO,EAAE,aAAa,EAAiB,MAAM,6BAA6B,CAAC;AAI3E,MAAM,WAAW,6BAA8B,SAAQ,cAAc;IACnE,wBAAwB;IACxB,gBAAgB,EAAE,eAAe,EAAE,CAAC;IACpC,0BAA0B;IAC1B,kBAAkB,EAAE,iBAAiB,EAAE,CAAC;CACzC;AAED;;;;GAIG;AACH,wBAAsB,oCAAoC,CAAC,EACzD,IAAI,EACJ,IAAI,EACJ,cAAc,EACd,WAAiB,EACjB,YAAyC,GAC1C,EAAE;IACD,oBAAoB;IACpB,IAAI,EAAE,MAAM,CAAC;IACb,uCAAuC;IACvC,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB;IAClB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,mCAAmC;IACnC,cAAc,EAAE,aAAa,EAAE,CAAC;IAChC,oCAAoC;IACpC,YAAY,CAAC,EAAE,MAAM,CAAC;CACvB,GAAG,OAAO,CAAC,6BAA6B,EAAE,CAAC,CAyF3C"}
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
3
|
+
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
4
|
+
};
|
|
5
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
6
|
+
exports.pullManualEnrichmentIdentifiersToCsv = void 0;
|
|
7
|
+
const graphql_1 = require("../graphql");
|
|
8
|
+
const colors_1 = __importDefault(require("colors"));
|
|
9
|
+
const groupBy_1 = __importDefault(require("lodash/groupBy"));
|
|
10
|
+
const uniq_1 = __importDefault(require("lodash/uniq"));
|
|
11
|
+
const bluebird_1 = require("bluebird");
|
|
12
|
+
const privacy_types_1 = require("@transcend-io/privacy-types");
|
|
13
|
+
const writeCsv_1 = require("../cron/writeCsv");
|
|
14
|
+
const logger_1 = require("../logger");
|
|
15
|
+
/**
|
|
16
|
+
* Pull the set of manual enrichment jobs to CSV
|
|
17
|
+
*
|
|
18
|
+
* @param options - Options
|
|
19
|
+
*/
|
|
20
|
+
async function pullManualEnrichmentIdentifiersToCsv({ file, auth, requestActions, concurrency = 100, transcendUrl = 'https://api.transcend.io', }) {
|
|
21
|
+
// Find all requests made before createdAt that are in a removing data state
|
|
22
|
+
const client = (0, graphql_1.buildTranscendGraphQLClient)(transcendUrl, auth);
|
|
23
|
+
logger_1.logger.info(colors_1.default.magenta(`Pulling manual enrichment requests, filtered for actions: ${requestActions.join(',')}`));
|
|
24
|
+
// Pull all privacy requests
|
|
25
|
+
const allRequests = await (0, graphql_1.fetchAllRequests)(client, {
|
|
26
|
+
actions: requestActions,
|
|
27
|
+
statuses: [privacy_types_1.RequestStatus.Enriching],
|
|
28
|
+
});
|
|
29
|
+
// Requests to save
|
|
30
|
+
const savedRequests = [];
|
|
31
|
+
// Filter down requests to what is needed
|
|
32
|
+
await (0, bluebird_1.map)(allRequests, async (request) => {
|
|
33
|
+
// Fetch enrichers
|
|
34
|
+
const requestEnrichers = await (0, graphql_1.fetchAllRequestEnrichers)(client, {
|
|
35
|
+
requestId: request.id,
|
|
36
|
+
});
|
|
37
|
+
// Check if manual enrichment exists for that request
|
|
38
|
+
const hasManualEnrichment = requestEnrichers.filter(({ status }) => status === 'ACTION_REQUIRED');
|
|
39
|
+
// Save request to queue
|
|
40
|
+
if (hasManualEnrichment) {
|
|
41
|
+
const requestIdentifiers = await (0, graphql_1.fetchAllRequestIdentifiers)(client, {
|
|
42
|
+
requestId: request.id,
|
|
43
|
+
});
|
|
44
|
+
savedRequests.push({
|
|
45
|
+
...request,
|
|
46
|
+
requestIdentifiers,
|
|
47
|
+
requestEnrichers,
|
|
48
|
+
});
|
|
49
|
+
}
|
|
50
|
+
}, {
|
|
51
|
+
concurrency,
|
|
52
|
+
});
|
|
53
|
+
const data = savedRequests.map(({ attributeValues, requestIdentifiers, requestEnrichers, // eslint-disable-line @typescript-eslint/no-unused-vars
|
|
54
|
+
...request }) => ({
|
|
55
|
+
...request,
|
|
56
|
+
// flatten identifiers
|
|
57
|
+
...Object.entries((0, groupBy_1.default)(requestIdentifiers, 'name')).reduce((acc, [key, values]) => Object.assign(acc, {
|
|
58
|
+
[key]: values.map(({ value }) => value).join(','),
|
|
59
|
+
}), {}),
|
|
60
|
+
// flatten attributes
|
|
61
|
+
...Object.entries((0, groupBy_1.default)(attributeValues, 'attributeKey.name')).reduce((acc, [key, values]) => Object.assign(acc, {
|
|
62
|
+
[key]: values.map(({ name }) => name).join(','),
|
|
63
|
+
}), {}),
|
|
64
|
+
}));
|
|
65
|
+
// Write out to CSV
|
|
66
|
+
const headers = (0, uniq_1.default)(data.map((d) => Object.keys(d)).flat());
|
|
67
|
+
(0, writeCsv_1.writeCsv)(file, data, headers);
|
|
68
|
+
logger_1.logger.info(colors_1.default.green(`Successfully wrote ${savedRequests.length} requests to file "${file}"`));
|
|
69
|
+
return savedRequests;
|
|
70
|
+
}
|
|
71
|
+
exports.pullManualEnrichmentIdentifiersToCsv = pullManualEnrichmentIdentifiersToCsv;
|
|
72
|
+
//# sourceMappingURL=pullManualEnrichmentIdentifiersToCsv.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"pullManualEnrichmentIdentifiersToCsv.js","sourceRoot":"","sources":["../../src/manual-enrichment/pullManualEnrichmentIdentifiersToCsv.ts"],"names":[],"mappings":";;;;;;AAAA,wCAQoB;AACpB,oDAA4B;AAC5B,6DAAqC;AACrC,uDAA+B;AAC/B,uCAA+B;AAC/B,+DAA2E;AAC3E,+CAA4C;AAC5C,sCAAmC;AASnC;;;;GAIG;AACI,KAAK,UAAU,oCAAoC,CAAC,EACzD,IAAI,EACJ,IAAI,EACJ,cAAc,EACd,WAAW,GAAG,GAAG,EACjB,YAAY,GAAG,0BAA0B,GAY1C;IACC,4EAA4E;IAC5E,MAAM,MAAM,GAAG,IAAA,qCAA2B,EAAC,YAAY,EAAE,IAAI,CAAC,CAAC;IAC/D,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,OAAO,CACZ,6DAA6D,cAAc,CAAC,IAAI,CAC9E,GAAG,CACJ,EAAE,CACJ,CACF,CAAC;IAEF,4BAA4B;IAC5B,MAAM,WAAW,GAAG,MAAM,IAAA,0BAAgB,EAAC,MAAM,EAAE;QACjD,OAAO,EAAE,cAAc;QACvB,QAAQ,EAAE,CAAC,6BAAa,CAAC,SAAS,CAAC;KACpC,CAAC,CAAC;IAEH,mBAAmB;IACnB,MAAM,aAAa,GAAoC,EAAE,CAAC;IAE1D,yCAAyC;IACzC,MAAM,IAAA,cAAG,EACP,WAAW,EACX,KAAK,EAAE,OAAO,EAAE,EAAE;QAChB,kBAAkB;QAClB,MAAM,gBAAgB,GAAG,MAAM,IAAA,kCAAwB,EAAC,MAAM,EAAE;YAC9D,SAAS,EAAE,OAAO,CAAC,EAAE;SACtB,CAAC,CAAC;QAEH,qDAAqD;QACrD,MAAM,mBAAmB,GAAG,gBAAgB,CAAC,MAAM,CACjD,CAAC,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,MAAM,KAAK,iBAAiB,CAC7C,CAAC;QAEF,wBAAwB;QACxB,IAAI,mBAAmB,EAAE;YACvB,MAAM,kBAAkB,GAAG,MAAM,IAAA,oCAA0B,EAAC,MAAM,EAAE;gBAClE,SAAS,EAAE,OAAO,CAAC,EAAE;aACtB,CAAC,CAAC;YACH,aAAa,CAAC,IAAI,CAAC;gBACjB,GAAG,OAAO;gBACV,kBAAkB;gBAClB,gBAAgB;aACjB,CAAC,CAAC;SACJ;IACH,CAAC,EACD;QACE,WAAW;KACZ,CACF,CAAC;IAEF,MAAM,IAAI,GAAG,aAAa,CAAC,GAAG,CAC5B,CAAC,EACC,eAAe,EACf,kBAAkB,EAClB,gBAAgB,EAAE,wDAAwD;IAC1E,GAAG,OAAO,EACX,EAAE,EAAE,CAAC,CAAC;QACL,GAAG,OAAO;QACV,sBAAsB;QACtB,GAAG,MAAM,CAAC,OAAO,CAAC,IAAA,iBAAO,EAAC,kBAAkB,EAAE,MAAM,CAAC,CAAC,CAAC,MAAM,CAC3D,CAAC,GAAG,EAAE,CAAC,GAAG,EAAE,MAAM,CAAC,EAAE,EAAE,CACrB,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE;YACjB,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC;SAClD,CAAC,EACJ,EAAE,CACH;QACD,qBAAqB;QACrB,GAAG,MAAM,CAAC,OAAO,CAAC,IAAA,iBAAO,EAAC,eAAe,EAAE,mBAAmB,CAAC,CAAC,CAAC,MAAM,CACrE,CAAC,GAAG,EAAE,CAAC,GAAG,EAAE,MAAM,CAAC,EAAE,EAAE,CACrB,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE;YACjB,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC;SAChD,CAAC,EACJ,EAAE,CACH;KACF,CAAC,CACH,CAAC;IAEF,mBAAmB;IACnB,MAAM,OAAO,GAAG,IAAA,cAAI,EAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;IAC7D,IAAA,mBAAQ,EAAC,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;IAE9B,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,KAAK,CACV,sBAAsB,aAAa,CAAC,MAAM,sBAAsB,IAAI,GAAG,CACxE,CACF,CAAC;IAEF,OAAO,aAAa,CAAC;AACvB,CAAC;AA1GD,oFA0GC"}
|