@devrev/ts-adaas 1.2.5 → 1.2.6

package/README.md

@@ -1,135 +1,297 @@
-# ADaaS Library
+# Airdrop SDK
 
-## Release Notes
+[![Coverage Status](https://coveralls.io/repos/github/devrev/adaas-sdk/badge.svg?branch=main&t=s4Otlm)](https://coveralls.io/github/devrev/adaas-sdk?branch=main)
 
-### v1.2.5
+## Overview
 
-- Add batch size option.
-- Replace DevRev Typescript SDK requests with Axios for uploading and downloading artifacts.
-- Remove unneccessary postState from default workers.
-- Fix bugs related to attachment streaming.
+The Airdrop SDK for TypeScript helps developers build snap-ins that integrate with DevRev’s Airdrop platform. 
+This SDK simplifies the workflow for handling data extraction and loading, event-driven actions, state management, and artifact handling.
 
-### v1.2.4
+It provides features such as:
 
-- Do not fail the extraction of attachments if streaming of single attachment fails.
+- Type Definitions: Structured types for Airdrop control protocol
+- Event Management: Easily emit events for different extraction or loading phases
+- State Handling: Update and access state in real-time within tasks
+- Artifact Management: Supports batched storage of artifacts
+- Error & Timeout Support: Error handling and timeout management for long-running tasks
 
-### v1.2.3
+## Installation
 
-- Add `local` flag to use for local development of the ADaaS snap-ins.
-- Send library version, snap-in version and snap-in slug in headers while emitting.
-- Make `actor_id` field optional for `SsorAttachment` interface.
-- Fix bugs related to event handling, error logging.
+```bash
+npm install @devrev/ts-adaas
+```
 
-### v1.2.2
+## Reference
 
-- Add library version as a part of control protocol.
-- Improve axios client and adapter logging.
-- Fix bugs related to state handling.
+### `spawn` function
 
-### v1.2.1
+This function initializes a new worker thread and oversees its lifecycle. 
+It should be invoked when the snap-in receives a message from the Airdrop platform.
+The worker script provided then handles the event accordingly.
 
-- Reduced the `delayFactor` to minimize unnecessary delays.
-- Correct the setting of the `lastSyncStarted` timestamp.
-- Improve logging for attachment extraction and loading.
-- Fix several bugs related to the control protocol.
+#### Usage
 
-### v1.2.0
+```typescript
+spawn({ event, initialState, workerPath, options })
+```
 
-- Add support for loading attachments from DevRev to external system.
+#### Parameters
 
-### v1.1.6
+* _event_
+  
+  Required. An object of type __AirdropEvent__ that is received from the Airdrop platform.
 
-- Add exponential retry and handle rate-limiting towards DevRev.
-- Gracefully handle failure to upload extracted attachments.
+* _initialState_
 
-### v1.1.5
+  Required. Object of __any__ type that represents the initial state of the snap-in.
 
-- Increase `delayFactor` and number of retries for the exponential backoff retry mechanism for HTTP requests.
-- Provide an inject function for streaming attachments.
-- Fix the attachments streaming bug.
+* _workerPath_
 
-### v1.1.4
+  Required. A __string__ that represents the path to the worker file.
 
-- Provide log lines and stack traces for runtime worker errors.
+* _options_
 
-### v1.1.3
+  Optional. An object of type **WorkerAdapterOptions**, which will be passed to the newly created worker. This worker will then initialize a `WorkerAdapter` by invoking the `processTask` function. The options include:
+  
+  * `isLocalDevelopment`
+  
+    A __boolean__ flag. If set to `true`, intermediary files containing extracted data will be stored on the local machine, which is useful during development. The default value is `false`.
 
-- Export `axios` and `axiosClient` with the exponential backoff retry mechanism for HTTP requests and omit Authorization headers from Axios errors.
-- Resolve circular structure logging issues.
-- Fix the attachments metadata normalization bug.
-- Improve repository logging.
+  * `timeout`
+  
+    A __number__ that specifies the timeout duration for the lambda function, in milliseconds. The default is 10 minutes (10 * 60 * 1000 milliseconds), with a maximum allowable duration of 13 minutes (13 * 60 * 1000 milliseconds).
+  
+  * `batchSize`
 
-### v1.1.2
+    A __number__ that determines the maximum number of items to be processed and saved to an intermediary file before being sent to the Airdrop platform. The default batch size is 2,000.
 
-- Unify incoming and outgoing event context.
-- Add `dev_oid` to logger tags.
+#### Return value
 
-### v1.1.1
+A __promise__ that resolves once the worker has completed processing.
 
-- Add default workers for loading deletion events.
+#### Example
 
-### v1.1.0
+```typescript
+const run = async (events: AirdropEvent[]) => {
+  for (const event of events) {
+    const file = getWorkerPerExtractionPhase(event);
+    await spawn<ExtractorState>({
+      event,
+      initialState,
+      workerPath: file,
+    });
+  }
+};
+```
 
-- Support sync from DevRev to the external system. (Known limitations: no support for loading attachments.)
+### `processTask` function
 
-### v1.0.4
+The `processTask` function retrieves the current state from the Airdrop platform and initializes a new `WorkerAdapter`.
+It executes the code specified in the `task` parameter, which contains the worker's functionality.
+If a timeout occurs, the function handles it by executing the `onTimeout` callback, ensuring the worker exits gracefully.
+Both functions receive an `adapter` parameter, representing the initialized `WorkerAdapter` object.
 
-- Fix logging from worker threads.
 
-### v1.0.3
+#### Usage
+```typescript
+processTask({ task, onTimeout })
+```
 
-- Add release notes.
+#### Parameters
 
-### v1.0.2
+* _task_
+  
+  Required. A __function__ that defines the logic associated with the given event type.
 
-- Fix bugs and improve local development.
-- Expose `formatAxiosError` function for error handling.
+* _onTimeout_
+  
+  Required. A __function__ managing the timeout of the lambda invocation, including saving any necessary progress at the time of timeout.
 
-### v1.0.1
+#### Example
 
-- Fix bugs and improve logging.
+````typescript
+// External sync units extraction
+processTask({
+  task: async ({ adapter }) => {
+    const httpClient = new HttpClient(adapter.event);
 
-### v1.0.0
+    const todoLists = await httpClient.getTodoLists();
 
-- Enable extractions to use the full lambda runtime and gracefully handle execution context timeout.
-- Simplify metadata and data normalization and uploading with the repo implementation.
-- Provide default handling of the attachment extraction phase in the ADaaS SDK library.
-- Reduce file size and streamline processes with gzip compression.
-- Fix bugs and improve error handling.
+    const externalSyncUnits: ExternalSyncUnit[] = todoLists.map((todoList) => normalizeTodoList(todoList));
 
-### v0.0.3
+    await adapter.emit(ExtractorEventType.ExtractionExternalSyncUnitsDone, {
+      external_sync_units: externalSyncUnits,
+    });
+  },
+  onTimeout: async ({ adapter }) => {
+    await adapter.emit(ExtractorEventType.ExtractionExternalSyncUnitsError, {
+      error: {
+        message: 'Failed to extract external sync units. Lambda timeout.',
+      },
+    });
+  },
+});
+````
 
-- Support new recipe management.
+### `WorkerAdapter` class
 
-### v0.0.2
+Used to interact with Airdrop platform. 
+Provides utilities to emit events to the Airdrop platform, update the state of the snap-in and upload artifacts (files with data) to the platform.
 
-- Support the State API.
-- Provide an HTTP client for API requests.
-- Create local artifact files in the local development environment.
-- Improve logging.
+### Usage
 
-### v0.0.1
+```typescript
+new WorkerAdapter({
+  event,
+  adapterState,
+  options,
+});
+```
 
-- Implement a demo of the ADaaS snap-in.
-- Add an adapter for the ADaaS control protocol with helper functions.
-- Provide an uploader for uploading artifacts.
+#### Parameters
 
-# Overview
+* _event_
+  
+  Required. An object of type __AirdropEvent__ that is received from the Airdrop platform.
 
-[![Coverage Status](https://coveralls.io/repos/github/devrev/adaas-sdk/badge.svg?branch=main&t=s4Otlm)](https://coveralls.io/github/devrev/adaas-sdk?branch=main)
+* _adapterState_
+  
+  Required. An object of type __State__, which represents the initial state of the adapter.
 
-The ADaaS (Airdrop-as-a-Service) Library for TypeScript helps developers build Snap-ins that integrate with DevRev’s ADaaS platform. This library simplifies the workflow for handling data extraction and loading, event-driven actions, state management, and artifact handling.
+* _options_
+  
+  Optional. An object of type __WorkerAdapterOptions__ that specifies additional configuration options for the `WorkerAdapter`. This object is passed via the `spawn` function.
 
-It provides features such as:
+#### Example
 
-- Type Definitions: Structured types for ADaaS control protocol
-- Event Management: Easily emit events for different extraction or loading phases
-- State Handling: Update and access state in real-time within tasks
-- Artifact Management: Supports batched storage of artifacts
-- Error & Timeout Support: Error handling and timeout management for long-running tasks
+```typescript
+const adapter = new WorkerAdapter<ConnectorState>({
+  event,
+  adapterState,
+  options,
+});
+```
 
-# Installation
+### `WorkerAdapter.state` property
 
-```bash
-npm install @devrev/ts-adaas
+Getter and setter methods for working with the adapter state.
+
+### Usage
+
+```typescript
+// get state
+const adapterState = adapter.state;
+
+// set state
+adapter.state = newAdapterState;
+```
+
+#### Example
+
+```typescript
+export const initialState: ExtractorState = {
+  users: { completed: false },
+  tasks: { completed: false },
+  attachments: { completed: false },
+};
+
+adapter.state = initialState;
+```
+
+### `WorkerAdapter.initializeRepos` method
+
+Initializes a `Repo` object for each item provided.
+
+### Usage
+
+```typescript
+adapter.initializeRepos(repos);
+```
+
+#### Parameters
+
+* _repos_
+  
+  Required. An array of objects of type `RepoInterface`. 
+
+#### Example
+
+This should typically be called within the function passed as a parameter to the `processTask` function in the data extraction phase.
+
+```typescript
+const repos = [
+  {
+    itemType: 'tasks',
+    normalize: normalizeTask,
+  }
+];
+
+adapter.initializeRepos(repos);
+```
+
+### `WorkerAdapter.getRepo` method
+
+Finds a Repo from the initialized repos.
+
+### Usage
+
+```typescript
+adapter.getRepo(itemType);
+```
+
+#### Parameters
+
+* _itemType_
+  
+  Required. A __string__ that represents the itemType property for the searched repo.
+
+#### Return value
+
+An object of type __Repo__ if the repo is found, otherwise __undefined__.
+
+#### Example
+
+This should typically be called within the function passed as a parameter to the `processTask` function.
+
+```typescript
+// Push users to the repository designated for 'users' data.
+await adapter.getRepo('users')?.push(users);
+```
+
+### `WorkerAdapter.emit` method
+
+Emits an event to the Airdrop platform.
+
+### Usage
+
+```typescript
+adapter.emit( newEventType, data ):
+```
+
+#### Parameters
+
+* _newEventType_
+  
+  Required. The event type to be emitted, of type __ExtractorEventType__ or __LoaderEventType__.
+
+* _data_
+  
+  Optional. An object of type __EventData__ which represents the data to be sent with the event.
+
+#### Return value
+
+A __promise__, which resolves to undefined after the emit function completes its execution or rejects with an error.
+
+#### Example
+
+This should typically be called within the function passed as a parameter to the `processTask` function.
+
+```typescript
+// Emitting successfully finished data extraction.
+await adapter.emit(ExtractorEventType.ExtractionDataDone);
+
+// Emitting a delay in attachments extraction phase.
+await adapter.emit(ExtractorEventType.ExtractionAttachmentsDelay, {
+  delay: 10,
+});
 ```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@devrev/ts-adaas",
-  "version": "1.2.5",
+  "version": "1.2.6",
   "description": "DevRev ADaaS (AirDrop-as-a-Service) Typescript SDK.",
   "type": "commonjs",
   "main": "./dist/index.js",