@devrev/ts-adaas 1.1.1 → 1.1.3

package/README.md

@@ -2,6 +2,18 @@
 
 ## Release Notes
 
+### v1.1.3
+
+- Exported `axios` and `axiosClient` with exponential backoff retry mechanism for HTTP requests and omitting Authorization headers from Axios errors.
+- Resolved issues with circular structure logging.
+- Fixed attachments metadata normalization bug.
+- Improved repository logging.
+
+#### v1.1.2
+
+- Unified incoming and outgoing event context.
+- Added `dev_oid` to logger tags.
+
 #### v1.1.1
 
 - Added default workers for loading deletion events.
@@ -54,14 +66,14 @@
 
 # Overview
 
-The ADaaS (Airdrop-as-a-Service) Library for TypeScript helps developers build Snap-ins that integrate with DevRev’s ADaaS platform. This library simplifies the workflow for handling data extraction, event-driven actions, state management, and artifact handling.
+The ADaaS (Airdrop-as-a-Service) Library for TypeScript helps developers build Snap-ins that integrate with DevRev’s ADaaS platform. This library simplifies the workflow for handling data extraction and loading, event-driven actions, state management, and artifact handling.
 
-## Features
+It provides features such as:
 
 - Type Definitions: Structured types for ADaaS control protocol
-- Event Management: Easily emit events for different extraction phases
+- Event Management: Easily emit events for different extraction or loading phases
 - State Handling: Update and access state in real-time within tasks
-- Artifact Management: Supports batched storage of artifacts (2000 items per batch)
+- Artifact Management: Supports batched storage of artifacts
 - Error & Timeout Support: Error handling and timeout management for long-running tasks
 
 # Installation
@@ -72,7 +84,22 @@ npm install @devrev/ts-adaas
 
 # Usage
 
-ADaaS Snap-ins are composed of several phases, each with unique requirements for initialization, data extraction, and error handling. The ADaaS library exports processTask to structure the work within each phase. The processTask function accepts task and onTimeout handlers, giving access to the adapter to streamline state updates, upload of extracted data, and event emission.
+ADaaS Snap-ins can import data in both directions: from external sources to DevRev and from DevRev to external sources. Both directions are composed of several phases.
+
+From external source to DevRev:
+
+- External Sync Units Extraction
+- Metadata Extraction
+- Data Extraction
+- Attachments Extraction
+
+From DevRev to external source:
+
+- Data Loading
+
+Each phase comes with unique requirements for processing task, and both timeout and error handling.
+
+The ADaaS library exports processTask to structure the work within each phase, and onTimeout function to handle timeouts.
 
 ### ADaaS Snap-in Invocation
 
@@ -127,10 +154,14 @@ const run = async (events: AirdropEvent[]) => {
 export default run;
 ```
 
-## Extraction Phases
+## Extraction
 
 The ADaaS snap-in extraction lifecycle consists of three main phases: External Sync Units Extraction, Metadata Extraction, and Data Extraction. Each phase is defined in a separate file and is responsible for fetching the respective data.
 
+The ADaaS library provides a repository management system to handle artifacts in batches. The `initializeRepos` function initializes the repositories, and the `push` function uploads the artifacts to the repositories. The `postState` function is used to post the state of the extraction task.
+
+State management is crucial for ADaaS Snap-ins to maintain the state of the extraction task. The `postState` function is used to post the state of the extraction task. The state is stored in the adapter and can be retrieved using the `adapter.state` property.
+
 ### 1. External Sync Units Extraction
 
 This phase is defined in `external-sync-units-extraction.ts` and is responsible for fetching the external sync units.
@@ -243,7 +274,7 @@ processTask({
 });
 ```
 
-## 4. Attachments Streaming
+### 4. Attachments Streaming
 
 The ADaaS library handles attachments streaming to improve efficiency and reduce complexity for developers. During the extraction phase, developers need only to provide metadata in a specific format for each attachment, and the library manages the streaming process.
 
@@ -259,12 +290,45 @@ export interface NormalizedAttachment {
 }
 ```
 
-## Artifact Uploading and State Management
+## Loading phases
 
-The ADaaS library provides a repository management system to handle artifacts in batches. The `initializeRepos` function initializes the repositories, and the `push` function uploads the artifacts to the repositories. The `postState` function is used to post the state of the extraction task.
+### 1. Data Loading
 
-State management is crucial for ADaaS Snap-ins to maintain the state of the extraction task. The `postState` function is used to post the state of the extraction task. The state is stored in the adapter and can be retrieved using the `adapter.state` property.
+This phase is defined in `data-loading.ts` and is responsible for loading the data to the external system.
+
+Loading is done by providing an ordered list of itemTypes to load and their respective create and update functions.
+
+```typescript
+  processTask({
+    task: async ({ adapter }) => {
+      const { reports, processed_files } = await adapter.loadItemTypes({
+        itemTypesToLoad: [
+          {
+            itemType: 'tickets',
+            create: createTicket,
+            update: updateTicket,
+          },
+          {
+            itemType: 'conversations',
+            create: createConversation,
+            update: updateConversation,
+          },
+        ],
+      });
+
+    await adapter.emit(LoaderEventType.DataLoadingDone, {
+      reports,
+      processed_files,
+    });
+  },
+  onTimeout: async ({ adapter }) => {
+    await adapter.emit(LoaderEventType.DataLoadingProgress, {
+      reports: adapter.reports,
+      processed_files: adapter.processedFiles,
+    });
+});
+```
 
-## Timeout Handling
+The loading functions `create` and `update` provide loading to the external system. They provide denormalization of the records to the schema of the external system and provide HTTP calls to the external system. Both loading functions must handle rate limiting for the external system and handle errors.
 
-The ADaaS library provides a timeout handler to handle timeouts in long-running tasks. The `onTimeout` handler is called when the task exceeds the timeout limit. The handler can be used to post the state of the extraction task and emit an event when a timeout occurs.
+Functions return an ID and modified date of the record in the external system, or specify rate-liming offset or errors, if the record could not be created or updated.

package/dist/common/control-protocol.js

@@ -1,23 +1,18 @@
 "use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.emit = void 0;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 const logger_1 = require("../logger/logger");
 const emit = async ({ event, eventType, data, }) => {
     const newEvent = {
         event_type: eventType,
-        event_context: Object.assign({ uuid: event.payload.event_context.uuid, sync_run: event.payload.event_context.sync_run_id }, (event.payload.event_context.sync_unit_id && {
-            sync_unit: event.payload.event_context.sync_unit_id,
-        })),
+        event_context: event.payload.event_context,
         event_data: Object.assign({}, data),
     };
     return new Promise(async (resolve, reject) => {
         console.info('Emitting event', JSON.stringify(newEvent));
         try {
-            await axios_1.default.post(event.payload.event_context.callback_url, Object.assign({}, newEvent), {
+            await axios_client_1.axiosClient.post(event.payload.event_context.callback_url, Object.assign({}, newEvent), {
                 headers: {
                     Accept: 'application/json, text/plain, */*',
                     Authorization: event.context.secrets.service_account_token,
@@ -27,7 +22,7 @@ const emit = async ({ event, eventType, data, }) => {
             resolve();
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error(`Failed to emit event with event type ${eventType}.`, (0, logger_1.formatAxiosError)(error));
             }
             else {

package/dist/common/helpers.d.ts

@@ -14,3 +14,4 @@ export declare function addReportToLoaderReport({ loaderReports, report, }: {
     loaderReports: LoaderReport[];
     report: LoaderReport;
 }): LoaderReport[];
+export declare function getCircularReplacer(): (key: any, value: any) => any;

package/dist/common/helpers.js

@@ -4,6 +4,7 @@ exports.getTimeoutErrorEventType = getTimeoutErrorEventType;
 exports.getSyncDirection = getSyncDirection;
 exports.getFilesToLoad = getFilesToLoad;
 exports.addReportToLoaderReport = addReportToLoaderReport;
+exports.getCircularReplacer = getCircularReplacer;
 const extraction_1 = require("../types/extraction");
 const loading_1 = require("../types/loading");
 function getTimeoutErrorEventType(eventType) {
@@ -101,3 +102,17 @@ function addReportToLoaderReport({ loaderReports, report, }) {
     }
     return loaderReports;
 }
+// https://stackoverflow.com/a/53731154
+function getCircularReplacer() {
+    const seen = new WeakSet();
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    return (key, value) => {
+        if (typeof value === 'object' && value !== null) {
+            if (seen.has(value)) {
+                return;
+            }
+            seen.add(value);
+        }
+        return value;
+    };
+}

package/dist/common/install-initial-domain-mapping.js

@@ -1,10 +1,7 @@
 "use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.installInitialDomainMapping = installInitialDomainMapping;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 const logger_1 = require("../logger/logger");
 async function installInitialDomainMapping(event, initialDomainMappingJson) {
     var _a, _b, _c, _d, _e, _f, _g;
@@ -16,7 +13,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
         return;
     }
     try {
-        const snapInResponse = await axios_1.default.get(devrevEndpoint + '/internal/snap-ins.get', {
+        const snapInResponse = await axios_client_1.axiosClient.get(devrevEndpoint + '/internal/snap-ins.get', {
             headers: {
                 Authorization: devrevToken,
             },
@@ -35,7 +32,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
         if (startingRecipeBlueprint &&
             Object.keys(startingRecipeBlueprint).length !== 0) {
             try {
-                const recipeBlueprintResponse = await axios_1.default.post(`${devrevEndpoint}/internal/airdrop.recipe.blueprints.create`, Object.assign({}, startingRecipeBlueprint), {
+                const recipeBlueprintResponse = await axios_client_1.axiosClient.post(`${devrevEndpoint}/internal/airdrop.recipe.blueprints.create`, Object.assign({}, startingRecipeBlueprint), {
                     headers: {
                         Authorization: devrevToken,
                     },
@@ -44,7 +41,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
                 console.log('Successfully created recipe blueprint with id: ' + recipeBlueprintId);
             }
             catch (error) {
-                if (axios_1.default.isAxiosError(error)) {
+                if (axios_client_1.axios.isAxiosError(error)) {
                     console.error('Error while creating recipe blueprint', (0, logger_1.formatAxiosError)(error));
                 }
                 else {
@@ -55,7 +52,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
         try {
             // 2. Install the initial domain mappings
             const additionalMappings = initialDomainMappingJson.additional_mappings || {};
-            const initialDomainMappingInstallResponse = await axios_1.default.post(`${devrevEndpoint}/internal/airdrop.recipe.initial-domain-mappings.install`, Object.assign(Object.assign({ external_system_type: 'ADaaS', import_slug: importSlug, snap_in_slug: snapInSlug }, (recipeBlueprintId && {
+            const initialDomainMappingInstallResponse = await axios_client_1.axiosClient.post(`${devrevEndpoint}/internal/airdrop.recipe.initial-domain-mappings.install`, Object.assign(Object.assign({ external_system_type: 'ADaaS', import_slug: importSlug, snap_in_slug: snapInSlug }, (recipeBlueprintId && {
                 starting_recipe_blueprint: recipeBlueprintId,
             })), additionalMappings), {
                 headers: {
@@ -66,7 +63,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
                 JSON.stringify(initialDomainMappingInstallResponse.data));
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while installing initial domain mapping', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -76,7 +73,7 @@ async function installInitialDomainMapping(event, initialDomainMappingJson) {
         }
     }
     catch (error) {
-        if (axios_1.default.isAxiosError(error)) {
+        if (axios_client_1.axios.isAxiosError(error)) {
             console.error('Error while fetching snap in', (0, logger_1.formatAxiosError)(error));
         }
         else {

package/dist/deprecated/adapter/index.js

@@ -93,9 +93,7 @@ class Adapter {
         }
         const newEvent = {
             event_type: newEventType,
-            event_context: Object.assign({ uuid: this.event.payload.event_context.uuid, sync_run: this.event.payload.event_context.sync_run_id }, (this.event.payload.event_context.sync_unit_id && {
-                sync_unit: this.event.payload.event_context.sync_unit_id,
-            })),
+            event_context: this.event.payload.event_context,
             event_data: Object.assign({}, data),
         };
         try {

package/dist/http/axios-client.d.ts

@@ -0,0 +1,3 @@
+import axios from 'axios';
+declare const axiosClient: import("axios").AxiosInstance;
+export { axios, axiosClient };

package/dist/http/axios-client.js

@@ -0,0 +1,41 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.axiosClient = exports.axios = void 0;
+const axios_1 = __importDefault(require("axios"));
+exports.axios = axios_1.default;
+const axios_retry_1 = __importDefault(require("axios-retry"));
+const axiosClient = axios_1.default.create();
+exports.axiosClient = axiosClient;
+// Exponential backoff algorithm: Retry 3 times and there will be a delay of more than 1 * no. of retries second + random number of milliseconds between each retry.
+(0, axios_retry_1.default)(axiosClient, {
+    retries: 3,
+    retryDelay: (retryCount, error) => {
+        var _a;
+        console.log(`Retry attempt: ${retryCount} of ${(_a = error.config) === null || _a === void 0 ? void 0 : _a.url}.`);
+        return axios_retry_1.default.exponentialDelay(retryCount, error, 1000);
+    },
+    retryCondition: (error) => {
+        var _a, _b, _c, _d;
+        if (((_a = error.response) === null || _a === void 0 ? void 0 : _a.status) &&
+            ((_b = error.response) === null || _b === void 0 ? void 0 : _b.status) >= 500 &&
+            ((_c = error.response) === null || _c === void 0 ? void 0 : _c.status) <= 599) {
+            return true;
+        }
+        else if (((_d = error.response) === null || _d === void 0 ? void 0 : _d.status) === 429) {
+            console.log('Rate limit exceeded. Delay: ' + error.response.headers['retry-after']);
+            return false;
+        }
+        else {
+            return false;
+        }
+    },
+    onMaxRetryTimesExceeded(error, retryCount) {
+        var _a;
+        console.log(`Max retries attempted: ${retryCount}`);
+        (_a = error.config) === null || _a === void 0 ? true : delete _a.headers.Authorization;
+        delete error.request._header;
+    },
+});

package/dist/http/client.d.ts

@@ -1,6 +1,10 @@
 import { RawAxiosRequestHeaders } from 'axios';
 import { HTTPResponse } from './types';
 export declare const defaultResponse: HTTPResponse;
+/**
+ * HTTPClient class to make HTTP requests
+ * @deprecated
+ */
 export declare class HTTPClient {
     private retryAfter;
     private retryAt;

package/dist/http/client.js

@@ -35,6 +35,10 @@ exports.defaultResponse = {
     message: '',
     success: false,
 };
+/**
+ * HTTPClient class to make HTTP requests
+ * @deprecated
+ */
 class HTTPClient {
     constructor() {
         this.retryAfter = 0;

package/dist/http/index.d.ts

@@ -1,2 +1,3 @@
 export * from './client';
 export * from './types';
+export * from './axios-client';

package/dist/http/index.js

@@ -16,3 +16,4 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
 Object.defineProperty(exports, "__esModule", { value: true });
 __exportStar(require("./client"), exports);
 __exportStar(require("./types"), exports);
+__exportStar(require("./axios-client"), exports);

package/dist/http/types.d.ts

@@ -1,3 +1,7 @@
+/**
+ * HTTP Response type
+ * @deprecated
+ */
 export type HTTPResponse = {
     success: boolean;
     message: string;

package/dist/logger/logger.d.ts

@@ -2,7 +2,6 @@ import { Console } from 'node:console';
 import { LoggerFactoryInterface, LogLevel, PrintableState } from './logger.interfaces';
 import { AxiosError } from 'axios';
 export declare class Logger extends Console {
-    private event;
     private options?;
     constructor({ event, options }: LoggerFactoryInterface);
     logFn(args: unknown[], level: LogLevel): void;

package/dist/logger/logger.js

@@ -11,15 +11,15 @@ const node_console_1 = require("node:console");
 const logger_interfaces_1 = require("./logger.interfaces");
 const node_worker_threads_1 = require("node:worker_threads");
 const workers_1 = require("../types/workers");
+const helpers_1 = require("../common/helpers");
 class Logger extends node_console_1.Console {
     constructor({ event, options }) {
         super(process.stdout, process.stderr);
-        this.event = event;
         this.options = options;
         lambda_log_1.default.options.levelKey = null;
         lambda_log_1.default.options.tagsKey = null;
         lambda_log_1.default.options.messageKey = 'message';
-        lambda_log_1.default.options.meta = Object.assign({}, event.payload.event_context);
+        lambda_log_1.default.options.meta = Object.assign(Object.assign({}, event.payload.event_context), { dev_oid: event.payload.event_context.dev_org });
     }
     logFn(args, level) {
         var _a;
@@ -35,7 +35,7 @@ class Logger extends node_console_1.Console {
             node_worker_threads_1.parentPort === null || node_worker_threads_1.parentPort === void 0 ? void 0 : node_worker_threads_1.parentPort.postMessage({
                 subject: workers_1.WorkerMessageSubject.WorkerMessageLog,
                 payload: {
-                    args: JSON.parse(JSON.stringify(args)),
+                    args: JSON.parse(JSON.stringify(args, (0, helpers_1.getCircularReplacer)())),
                     level,
                 },
             });

package/dist/mappers/mappers.js

@@ -1,10 +1,7 @@
 "use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.Mappers = void 0;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 class Mappers {
     constructor({ event }) {
         this.endpoint = event.execution_metadata.devrev_endpoint;
@@ -12,7 +9,7 @@ class Mappers {
     }
     async getByTargetId(params) {
         const { sync_unit, target } = params;
-        return axios_1.default.get(`${this.endpoint}/internal/airdrop.sync-mapper-record.get-by-target`, {
+        return axios_client_1.axiosClient.get(`${this.endpoint}/internal/airdrop.sync-mapper-record.get-by-target`, {
             headers: {
                 Authorization: this.token,
             },
@@ -20,14 +17,14 @@ class Mappers {
         });
     }
     async create(params) {
-        return axios_1.default.post(`${this.endpoint}/internal/airdrop.sync-mapper-record.create`, params, {
+        return axios_client_1.axiosClient.post(`${this.endpoint}/internal/airdrop.sync-mapper-record.create`, params, {
             headers: {
                 Authorization: this.token,
             },
         });
     }
     async update(params) {
-        return axios_1.default.post(`${this.endpoint}/internal/airdrop.sync-mapper-record.update`, params, {
+        return axios_client_1.axiosClient.post(`${this.endpoint}/internal/airdrop.sync-mapper-record.update`, params, {
             headers: {
                 Authorization: this.token,
             },

package/dist/repo/repo.js

@@ -17,7 +17,7 @@ class Repo {
     async upload(batch) {
         const itemsToUpload = batch || this.items;
         if (itemsToUpload.length > 0) {
-            console.log(`Uploading ${itemsToUpload.length} items of type ${this.itemType}.`);
+            console.log(`Uploading ${itemsToUpload.length} items of type ${this.itemType}. `);
             const { artifact, error } = await this.uploader.upload(this.itemType, itemsToUpload);
             if (error || !artifact) {
                 console.error('Error while uploading batch', error);
@@ -28,6 +28,7 @@ class Repo {
             if (!batch) {
                 this.items = [];
             }
+            console.log(`Uploaded ${itemsToUpload.length} items of type ${this.itemType}. Number of items left in repo: ${this.items.length}.`);
         }
         else {
             console.log(`No items to upload for type ${this.itemType}. Skipping upload.`);
@@ -38,7 +39,8 @@ class Repo {
             let recordsToPush;
             // Normalize items if needed
             if (this.normalize &&
-                !Object.values(constants_1.AIRDROP_DEFAULT_ITEM_TYPES).includes(this.itemType)) {
+                this.itemType != constants_1.AIRDROP_DEFAULT_ITEM_TYPES.EXTERNAL_DOMAIN_METADATA &&
+                this.itemType != constants_1.AIRDROP_DEFAULT_ITEM_TYPES.SSOR_ATTACHMENT) {
                 recordsToPush = items.map((item) => this.normalize(item));
             }
             else {
@@ -46,7 +48,7 @@ class Repo {
             }
             // Add the new records to the items array
             this.items.push(...recordsToPush);
-            console.log(`Extracted ${this.items.length} items of type ${this.itemType}.`);
+            console.log(`Extracted ${recordsToPush.length} new items of type ${this.itemType}. Total number of items in repo: ${this.items.length}.`);
             // Upload in batches while the number of items exceeds the batch size
             while (this.items.length >= constants_1.ARTIFACT_BATCH_SIZE) {
                 // Slice out a batch of ARTIFACT_BATCH_SIZE items to upload

package/dist/repo/repo.test.js

@@ -43,11 +43,23 @@ describe('Repo class push method', () => {
         await repo.push(items);
         expect(normalize).not.toHaveBeenCalled();
     });
-    describe('should not normalize items if they are airdrop default item types', () => {
-        it.each(Object.values(constants_1.AIRDROP_DEFAULT_ITEM_TYPES))('item type: %s', async (itemType) => {
+    describe('should not normalize items if type is "external_domain_metadata" or "ssor_attachment"', () => {
+        it('item type: external_domain_metadata', async () => {
             repo = new repo_1.Repo({
                 event: (0, test_helpers_1.createEvent)({ eventType: types_1.EventType.ExtractionDataStart }),
-                itemType,
+                itemType: constants_1.AIRDROP_DEFAULT_ITEM_TYPES.EXTERNAL_DOMAIN_METADATA,
+                normalize,
+                onUpload: jest.fn(),
+                options: {},
+            });
+            const items = (0, test_helpers_1.createItems)(10);
+            await repo.push(items);
+            expect(normalize).not.toHaveBeenCalled();
+        });
+        it('item type: ssor_attachment', async () => {
+            repo = new repo_1.Repo({
+                event: (0, test_helpers_1.createEvent)({ eventType: types_1.EventType.ExtractionDataStart }),
+                itemType: constants_1.AIRDROP_DEFAULT_ITEM_TYPES.SSOR_ATTACHMENT,
                 normalize,
                 onUpload: jest.fn(),
                 options: {},

package/dist/state/state.js

@@ -1,11 +1,8 @@
 "use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.State = void 0;
 exports.createAdapterState = createAdapterState;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 const extraction_1 = require("../types/extraction");
 const constants_1 = require("../common/constants");
 const logger_1 = require("../logger/logger");
@@ -59,7 +56,7 @@ class State {
      */
     async postState(state) {
         try {
-            await axios_1.default.post(this.workerUrl + '.update', {
+            await axios_client_1.axiosClient.post(this.workerUrl + '.update', {
                 state: JSON.stringify(state || this.state),
             }, {
                 headers: {
@@ -74,7 +71,7 @@ class State {
             console.log('State updated successfully to:', (0, logger_1.getPrintableState)(this.state));
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Failed to update state.', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -93,7 +90,7 @@ class State {
             this.event.payload.event_context.sync_unit_id +
             '.');
         try {
-            const response = await axios_1.default.post(this.workerUrl + '.get', {}, {
+            const response = await axios_client_1.axiosClient.post(this.workerUrl + '.get', {}, {
                 headers: {
                     Authorization: this.devrevToken,
                 },
@@ -107,7 +104,7 @@ class State {
             return this.state;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error) && ((_a = error.response) === null || _a === void 0 ? void 0 : _a.status) === 404) {
+            if (axios_client_1.axios.isAxiosError(error) && ((_a = error.response) === null || _a === void 0 ? void 0 : _a.status) === 404) {
                 const state = Object.assign(Object.assign({}, initialState), this.initialSdkState);
                 this.state = state;
                 console.log('State not found, returning initial state. Current state:', (0, logger_1.getPrintableState)(this.state));

package/dist/types/extraction.d.ts

@@ -74,6 +74,7 @@ export interface ExternalSyncUnit {
 }
 /**
  * EventContextIn is an interface that defines the structure of the input event context that is sent to the external extractor from ADaaS.
+ * @deprecated
  */
 export interface EventContextIn {
     callback_url: string;
@@ -100,12 +101,39 @@ export interface EventContextIn {
 }
 /**
  * EventContextOut is an interface that defines the structure of the output event context that is sent from the external extractor to ADaaS.
+ * @deprecated
  */
 export interface EventContextOut {
     uuid: string;
     sync_run: string;
     sync_unit?: string;
 }
+/**
+ * EventContext is an interface that defines the structure of the event context that is sent to and from the external connector.
+ */
+export interface EventContext {
+    callback_url: string;
+    dev_org: string;
+    dev_org_id: string;
+    dev_user: string;
+    dev_user_id: string;
+    external_sync_unit: string;
+    external_sync_unit_id: string;
+    external_sync_unit_name: string;
+    external_system: string;
+    external_system_type: string;
+    import_slug: string;
+    mode: string;
+    request_id: string;
+    snap_in_slug: string;
+    sync_run: string;
+    sync_run_id: string;
+    sync_tier: string;
+    sync_unit: DonV2;
+    sync_unit_id: string;
+    uuid: string;
+    worker_data_url: string;
+}
 /**
  * ConnectionData is an interface that defines the structure of the connection data that is sent to the external extractor from ADaaS.
  * It contains the organization ID, organization name, key, and key type.
@@ -169,7 +197,7 @@ export interface AirdropEvent {
  */
 export interface AirdropMessage {
     connection_data: ConnectionData;
-    event_context: EventContextIn;
+    event_context: EventContext;
     event_type: EventType;
     event_data?: EventData;
 }
@@ -179,7 +207,7 @@ export interface AirdropMessage {
  */
 export interface ExtractorEvent {
     event_type: string;
-    event_context: EventContextOut;
+    event_context: EventContext;
     event_data?: EventData;
 }
 /**
@@ -187,6 +215,6 @@ export interface ExtractorEvent {
  */
 export interface LoaderEvent {
     event_type: string;
-    event_context: EventContextOut;
+    event_context: EventContext;
     event_data?: EventData;
 }

package/dist/uploader/uploader.js

@@ -27,7 +27,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.Uploader = void 0;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 const fs_1 = __importStar(require("fs"));
 const zlib_1 = __importDefault(require("zlib"));
 const js_jsonl_1 = require("js-jsonl");
@@ -98,7 +98,7 @@ class Uploader {
             return response.data;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while preparing artifact.', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -115,13 +115,13 @@ class Uploader {
         }
         formData.append('file', file);
         try {
-            const response = await axios_1.default.post(preparedArtifact.url, formData, {
+            const response = await axios_client_1.axiosClient.post(preparedArtifact.url, formData, {
                 headers: Object.assign({}, formData.getHeaders()),
             });
             return response;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while uploading artifact.', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -138,7 +138,7 @@ class Uploader {
         }
         formData.append('file', fileStreamResponse.data);
         try {
-            const response = await axios_1.default.post(preparedArtifact.url, formData, {
+            const response = await axios_client_1.axiosClient.post(preparedArtifact.url, formData, {
                 headers: Object.assign(Object.assign({}, formData.getHeaders()), (!fileStreamResponse.headers['content-length'] && {
                     'Content-Length': constants_1.MAX_DEVREV_ARTIFACT_SIZE,
                 })),
@@ -146,7 +146,7 @@ class Uploader {
             return response;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while streaming artifact.', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -223,13 +223,13 @@ class Uploader {
     }
     async downloadArtifact(artifactUrl) {
         try {
-            const response = await axios_1.default.get(artifactUrl, {
+            const response = await axios_client_1.axiosClient.get(artifactUrl, {
                 responseType: 'arraybuffer',
             });
             return response.data;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while downloading artifact from URL.', (0, logger_1.formatAxiosError)(error));
             }
             else {
@@ -302,7 +302,7 @@ class Uploader {
     }
     async getFileStreamResponse(url) {
         try {
-            const fileStreamResponse = await axios_1.default.get(url, {
+            const fileStreamResponse = await axios_client_1.axiosClient.get(url, {
                 responseType: 'stream',
                 headers: {
                     Authorization: this.event.payload.connection_data.key,
@@ -311,7 +311,7 @@ class Uploader {
             return fileStreamResponse;
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 console.error('Error while fetching attachment from URL.', (0, logger_1.formatAxiosError)(error));
             }
             else {

package/dist/workers/worker-adapter.js

@@ -1,11 +1,8 @@
 "use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.WorkerAdapter = void 0;
 exports.createWorkerAdapter = createWorkerAdapter;
-const axios_1 = __importDefault(require("axios"));
+const axios_client_1 = require("../http/axios-client");
 const extraction_1 = require("../types/extraction");
 const loading_1 = require("../types/loading");
 const constants_1 = require("../common/constants");
@@ -77,7 +74,8 @@ class WorkerAdapter {
     }
     initializeRepos(repos) {
         this.repos = repos.map((repo) => {
-            const shouldNormalize = !Object.values(constants_1.AIRDROP_DEFAULT_ITEM_TYPES).includes(repo.itemType);
+            const shouldNormalize = repo.itemType !== constants_1.AIRDROP_DEFAULT_ITEM_TYPES.EXTERNAL_DOMAIN_METADATA &&
+                repo.itemType !== constants_1.AIRDROP_DEFAULT_ITEM_TYPES.SSOR_ATTACHMENT;
             return new repo_1.Repo(Object.assign(Object.assign({ event: this.event, itemType: repo.itemType }, (shouldNormalize && { normalize: repo.normalize })), { onUpload: (artifact) => {
                     var _a;
                     this.artifacts.push(artifact);
@@ -342,7 +340,7 @@ class WorkerAdapter {
                         console.log('Updated sync mapper record', JSON.stringify(updateSyncMapperRecordResponse.data));
                     }
                     catch (error) {
-                        if (axios_1.default.isAxiosError(error)) {
+                        if (axios_client_1.axios.isAxiosError(error)) {
                             console.error('Failed to update sync mapper record', (0, logger_1.formatAxiosError)(error));
                             return {
                                 error: {
@@ -387,7 +385,7 @@ class WorkerAdapter {
             // Update mapper (optional)
         }
         catch (error) {
-            if (axios_1.default.isAxiosError(error)) {
+            if (axios_client_1.axios.isAxiosError(error)) {
                 if (((_a = error.response) === null || _a === void 0 ? void 0 : _a.status) === 404) {
                     // Create item
                     const { id, delay, error } = await itemTypeToLoad.create({
@@ -413,7 +411,7 @@ class WorkerAdapter {
                             };
                         }
                         catch (error) {
-                            if (axios_1.default.isAxiosError(error)) {
+                            if (axios_client_1.axios.isAxiosError(error)) {
                                 console.error('Failed to create sync mapper record', (0, logger_1.formatAxiosError)(error));
                                 return {
                                     error: {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@devrev/ts-adaas",
-  "version": "1.1.1",
+  "version": "1.1.3",
   "description": "Typescript library containing the ADaaS(AirDrop as a Service) control protocol.",
   "type": "commonjs",
   "main": "./dist/index.js",
@@ -37,7 +37,8 @@
   },
   "dependencies": {
     "@devrev/typescript-sdk": "^1.1.27",
-    "axios": "^1.5.1",
+    "axios": "^1.7.9",
+    "axios-retry": "^4.5.0",
     "form-data": "^4.0.1",
     "js-jsonl": "^1.1.1",
     "lambda-log": "^3.1.0",