@trayio/cdk-dsl 0.1.0 → 0.3.0

package/README.md

@@ -0,0 +1,362 @@
+# Connector Development Kit (CDK) DSL
+
+The CDK Domain Specific Language (DSL) is the main component of Tray's CDK, it is used to define all the aspects of a connectors, including the behaviour of its operations.
+
+A CDK connector consists of code written only using the DSL, which is declarative, so it only describes the connector, that description is then interpreted by the runtime
+to execute a connector's operations.
+
+
+## Project Structure
+
+A CDK project is just a regular npm typescript project, preconfigured with all the dependencies, linter rules and compiler options that will be used to build the connector during deployment, so it is not recommended to change those.
+
+Other than the package.json, jest configuration and typescript configuration, a connector will have the following:
+
+- A `connector.json` file that includes metadata about the connector, such as the name, version, title, etc
+- A `src` directory that has:
+    - An Authentication typescript file that contains the type of the `auth` object that operations receive together with the input, this type is the same for all operation
+    - A `test.auth.json` json file that contains an auth value that can be used for tests.
+    **This file should not be committed to a repository as it will have sensitive information such as access tokens**
+    - One folder per operation
+
+## Authentications
+
+Authentication values are received by operations together with the input, they usually contain things like tokens to identify the user making the request to a third party service.
+
+Not all connectors need authentications, in that case, an empty type can be used (with an empty value in the authentication test file), which is what the init command generates by default.
+
+## Operations
+
+A connector will have one folder per operation under the `src` folder, this folder will contain the following files:
+
+- `input.ts` which contains the type of the input of the operation
+- `output.ts` which contains the type of the output of the operation
+- `handler.ts` which is where the logic of the operation is
+- `handler.test.ts` which contains the test cases for testing the operation's behaviour defined in the handler
+
+## Handler
+
+A handler at its core, describes a function, that takes an `auth` value described by the authentication type (which is the same for all operations) and it takes an `input` value described by the input type of the operation
+
+The output of the handler is described by the output type, in case of a success, or it could contain an error if something failed during the execution of the handler or if the third party returned an error response.
+
+This "successful value or failure error" result of running an operation is described by the `OperationHandlerResult<T>` type, which is a sum/union/or type of `OperationHandlerSuccess<T>` and `OperationHandlerFailure`
+
+So, the core of what an operation handler describes can be summarised as a function:
+
+```typescript
+(auth: AuthType, input: InputType) => OperationHandlerResult<OutputType>
+```
+
+However, when defining a handler, we can also specify things like validation or whether or not the handler is private, and this is where the DSL comes in.
+
+The `handler.ts` file needs to define a handler using the `OperationHandlerSetup.configureHandler()` function, which allows for configuring all aspects of the handler by chaining function calls together for all the components of the handler.
+
+The `OperationHandlerSetup.configureHandler()` function takes the operation name and a callback that is used to configure the handler, it looks like this:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        /* use the "handler" value to specify what implementation to use, validation, etc */
+    );
+```
+
+## Validation
+
+Handlers can have input validation (which runs before the handler implementation is executed to validate the input that will be used to run it) and output validation (which runs after the implementation to validate its output).
+
+To add validation just use the `handler` argument of the callback described in the previous section:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation((validation => 
+                    validation.condition((auth, input) => input.id > 0)
+                              .errorMessage((auth, input) => `Id ${input.id} is not positive`))
+                )
+                .addOutputValidation((validation => 
+                    validation.condition((auth, input, output) => output.id === input.id)
+                              .errorMessage((auth, input, output) => `Output and Input ids don't match`))
+                )
+    );
+```
+
+Note that validation is optional, the only thing that is necessary to define a handler is its implementation.
+
+## Implementation
+
+The main aspect of a handler is its implementation, which can be `HTTP` if the operation will make an HTTP call to a third party, or `Composite` if the operation will combine zero or more operations when it runs, more implementations for other protocols will be added in the future.
+
+A handler can only have one implementation, it describes what the handler does when it receives a request.
+
+## HTTP Implementation
+
+A very simple handler that makes an HTTP call can be configured in the following way:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation(...)
+               .addOutputValidation(...)
+               .usingHttp((http) =>
+                    http.get('https://someapi.com/someresource/:id')
+                        .handleRequest((auth, input, request) =>
+                            request.addPathParameter('id', input.id.toString())
+                        )
+                        .handleResponse((response) => response.withBodyAsJson())
+               )
+    );
+```
+
+The previous handler makes a `GET` http request, which is defined by the `http.get()` call, after which a `handleRequest()` function is chained, whose purpose is to take an auth value, an input value and a request configuration and add the necessary arguments to that request configuration based on what we want the http call to have, the supported methods on the request configuration are:
+
+- `addPathParameter(name, value)`: Will replace a parameter on the path specified as `:name` in the url as shown in the previous example, the value will be url encoded
+- `addHeader(name, value)`: Adds a header to the request
+- `withBearerToken(token)`: Adds an `Authorization` header with a `Bearer` token
+- `addQueryString(name, value)`: Adds a query string to the request, the value will be url encoded
+- `withBodyAsJson(body)`: Adds a body to the request that will be sent as json.
+
+
+A handler with an authenticated POST request would look like this:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation(...)
+               .addOutputValidation(...)
+               .usingHttp((http) =>
+                    http.post('https://someapi.com/someresource')
+                        .handleRequest((auth, input, request) =>
+                            request.withBearerToken(auth.access_token)
+                                   .withBodyAsJson(input)
+                        )
+                        .handleResponse((response) => response.withBodyAsJson())
+               )
+    );
+```
+
+The input does not have to match what is sent as the body, if for example the input has other flags that specify how the connector needs to behave and only part of it contains the body, the `withBodyAsJson` method can be called in the following way:
+
+```typescript
+    request.withBodyAsJson({name: input.name, title: input.title})
+```
+
+So the `handlerRequest` function can transform the input in any way it needs to before sending the HTTP request and the same is true for the `handleResponse`, in the previous examples, the `handleResponse` simply read the response body as json and returned it, but it can be more complex if necessary.
+
+The `withBodyAsJson<T>()` function on the `response` argument returns a value of type `OperationHandlerResult<T>`, which can be a successful response or a failure based on the status code or if something went wrong executing the call.
+
+Just like with the input type and the request, the type of the json body in the response can be different from the output type.
+
+This is an example of a handler that transforms the response body into the output type:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation(...)
+               .addOutputValidation(...)
+               .usingHttp((http) =>
+                    http.post('https://someapi.com/someresource')
+                        .handleRequest(...)
+                        .handleResponse((response) => { 
+                            const httpResponseBody = response.withBodyAsJson<{message: string}>()
+                            if (httpResponseBody.isSuccess) {
+                                const originalMessage = httpResponseBody.value.message
+                                const extendedMessage = originalMessage + ' Extension'
+                                return OperationHandlerResult.success({ message: extendedMessage })
+                            }
+                            return httpResponseBody
+                        })
+               )
+    );
+```
+
+Instead of using an `if` in the previous case, there is also a `OperationHandlerResult.map` function that can do the same with a callback.
+
+The handler can also return successful responses for some failure cases or viceversa, this is an example of a handler that "recovers" from errors to always return a successful response:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation(...)
+               .addOutputValidation(...)
+               .usingHttp((http) =>
+                    http.post('https://someapi.com/someresource')
+                        .handleRequest(...)
+                        .handleResponse((response) => { 
+                            const httpResponseBody = response.withBodyAsJson()
+                            if (httpResponseBody.isFailure) {
+                                return OperationHandlerResult.success({ completed: false })
+                            }
+                            return OperationHandlerResult.success({ completed: true })
+                        })
+               )
+    );
+```
+
+
+## Composite Implementation
+
+Composite handlers are used to define behaviours by invoking zero or more operations as part of their behaviour.
+
+They can be used to write "helper" connectors (such as those in Tray's builder), DDL operations or complex operations
+like an "upsert" that combines more granular "read, create and update" operations.
+
+A very simple composite handler, that just concatenates the `firstName` and `lastName` arguments it gets from the input into one string:
+
+```typescript
+export const myOperationHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, InputType, OutputType>('my_operation', (handler) =>
+        handler.addInputValidation(...)
+               .addOutputValidation(...)
+               .usingComposite(async (auth, input, invoke) => {
+                    const fullName = input.firstName + ' ' + input.lastName
+                    return OperationHandlerResult.success({fullName: fullName})
+               })
+    );
+```
+
+As an example of more complex behaviour, this handler reads a list of products using another operation and converts the result into a simple list of `{text: string, value: string}` pairs, this is known as a Dynamic Data List (DDL) operation used to help users select values as part of configuring workflows within the tray builder.
+
+
+To do this, the handler needs to invoke the regular `getProducts` operation, this is accomplished by using the `invoke` functions that composite handlers have access to, and passing it a **handler reference** and an input (no need to pass the auth value as it will be passed automatically), the handler reference passed to the `invoke` function is the result of the `OperationHandlerSetup.configureHandler()` function, which is why they are saved into a constant and exported like this:
+
+```typescript
+export const getProductsHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, GetProductsInput, GetProductsOutput>('get_products', (handler) =>
+        handler.usingHttp(...)
+    );
+```
+
+The `getProductsHandler` constant contains the handler reference, which also has the input and output type information as part of its type, to make sure that when invoked or tested, only valid input and output values can be used.
+
+
+With that in mind, this is what the DDL handler would look like:
+
+
+```typescript
+export const getProductsDDLHandler = 
+    OperationHandlerSetup.configureHandler<AuthType, GetProductsDDLInput, GetProductsDDLOutput>('get_products_ddl', (handler) =>
+        handler.usingComposite(async (auth, input, invoke) => {
+                    const productListResult: OperationHandlerResult<GetProductsOutput> = 
+                        await invoke(getProductsHandler)({ storeId: input.storeId })
+                    return OperationHandlerResult.map(
+                        productListResult, 
+                        productList => productList.map(product => { text: product.title, value: product.id })
+                    )
+               })
+    );
+```
+
+There are several things to note about the handler, both the DDL handler and the `getProductsHandler` expect a `storeId` in the input to return a list of products for that store. 
+
+The `getProductsHandler` is invoked using the `invoke` function that composite handlers receive as an argument, passing the handler reference, and then calling the result as a function passing the input that handler expects, in this case, just an object with a `storeId`
+
+The result of that invocation is of type `Promise<OperationHandlerResult<GetProductsOutput>>`, that is, a promise that has a value that is described by the invoked handler's output type, which is wrapped in the result object because it could be a successful invocation or a failure as described in previous sections.
+
+The `await` keyword unwraps the promise, and we are left with a `OperationHandlerResult<GetProductsOutput>`, which forces the handler to deal with both the failure case as well as the success case.
+
+There are multiple ways to do this
+- Using `if` or `switch` statements to narrow down the type
+- Using the `OperationHandlerResult.getSuccessfulValueOrFail()` function which unwraps the value if successful or terminates the function propagating the error if it is not
+- As shown in the example, using the `OperationHandlerResult.map()` function.
+
+Once the handler has access to the product list value, it just needs to convert each element to a `{text: string, value: string}` pair.
+
+## Testing
+
+The CDK DSL has declarative testing functions to test a handler's behaviour, the tests are in the `handler.test.ts` file within the operation folder, which can have zero, one or many test cases for that given operation.
+
+The `OperationHandlerTestSetup.configureHandlerTest()` function is used to describe a test, it takes a handler reference and a callback with an object used to configure the test, in a similar way handlers are configured.
+
+This is what a very basic test looks like:
+
+```typescript
+OperationHandlerTestSetup.configureHandlerTest(
+	myOperationHandler,
+	(handlerTest) =>
+		handlerTest
+			.usingAuth('test') //will use `test.auth.json` as the authentication value for all test cases
+			.nothingBeforeAll()
+			.testCase('should do something', (testCase) =>
+				testCase
+                    .usingAuth('another') //optionally, a test case can define its own auth instead of using the default one defined for all tests
+					.givenNoContext()
+					.when(() => /* return an input that matches the input type */)
+					.then(({ output }) => {
+                        /* output is OperationHandlerResult<T> where T is a value matching the output type */
+
+                        //This will contain a value of type T if the operation was successful or the test will fail if not
+                        const successValue = OperationHandlerResult.getSuccessfulValueOrFail(output)
+
+                        // jest-style matchers like "expect" are available here
+                        expect(successValue).toEqual(...)
+                    })
+					.finallyDoNothing()
+			)
+			.nothingAfterAll()
+);
+```
+
+The structure of a test is well defined, and the type safe declarative DSL will enforce that, in particular, there are a number of aspects that would apply to all test cases:
+
+- A default authentication to use for all test cases
+- Run one or more operations (doesn't have to be the one being tested) before all test cases using the `beforeAll()` function, or don't do anything before all test cases using the `nothingBeforeAll()` function, these functions can only be used before adding test cases (this is enforced by the type system)
+- Add a test case using the `testCase()` function.
+- Run one or more operations (doesn't have to be the one being tested) after all test cases using the `afterAll()` function, or don't do anything after all test cases using the `nothingAfterAll()` function, these functions can only be used after defining test cases, and no test cases can be added after this (this is enforced by the type system)
+
+
+As for the test cases, they use a BDD style Given/When/Then convention, in particular a test case has:
+
+- An optional `usingAuth()` function at the beginning of the test case to use a different auth than the default for all test cases
+- A `given()` function to run one or more operations at the beginning of the test case or `givenNoContext()` to go straight to running the operation under test 
+- A `when()` function to create an input value that will be used to run the operation under test, that value needs to match the input type of the operation
+- A `then()` function that gets the output, input, auth and optionally the result of `beforeAll()` and `given()` if present, which can be used to do the assertions of the test case using jest-style matchers
+- A `finally()` function to run one or more operations at the end of the test case, usually for cleanup, or `finallyDoNothing()` to don't do anything else after the assertions.
+
+
+Both the `beforeAll()` and `given()` functions allow to run multiple operations before all test cases or before a single test case, they receive the `invoke` function as an argument just like composite handlers and they return an object that can contain some of relevant information about the operations that ran if necessary (like ids of things created) in a value that can be accessed by the `when()`, `then()`, `finally()` and `afterAll()` functions, they receive them as arguments.
+
+As an example, the following test is for an `updateProduct` operation, it creates a store for all test cases, and a product for every test case to test the update on, using `beforeAll` and `given` respectively, and accessing the identities of the created objects from the `testContext` (the output of `beforeAll`) and the `testCaseContext` (the output of `given`):
+
+```typescript
+OperationHandlerTestSetup.configureHandlerTest(
+	updateProductOperationHandler,
+	(handlerTest) =>
+		handlerTest
+			.usingAuth('test')
+			.beforeAll<{storeId: string}>(async (auth, invoke) => {
+				//Creates an store that will be used by all tests
+				const createdStoreResult = invoke(createStoreHandler)({name: 'something'})
+				return OperationHandlerResult.map(
+					createdStoreResult, 
+					createdStoreOutput => { storeId: createdStoreOutput.id}
+				)
+			})
+			.testCase('should do something', (testCase) =>
+				testCase
+					.given<{ productId: string}>((auth, testContext, invoke) => {
+						//Creates an product in the store to be used by the test
+						const createdProductResult = invoke(createProductHandler)({name: 'some product', storeId: testContext.storeId})
+						return OperationHandlerResult.map(
+							createdProductResult, 
+							createdProductOutput => { productId: createdProductOutput.id}
+						)
+					})
+					.when((auth, testContext, testCaseContext) => ({ productId: testCaseContext.productId, name: 'updated name' }))
+					.then(({ output }) => {
+						const outputValue = OperationHandlerResult.getSuccessfulValueOrFail(output)
+						expect(outputValue.name).toEqual('updated name');
+					})
+					.finally(({ testCaseContext }) => {
+						//Deletes the product created for the test
+						return invoke(deleteProductHandler)({productId: testCaseContext.productId})
+					})
+			)
+			.afterAll(({ testContext }) => {
+				//Deletes the store created for all test cases
+				return invoke(deleteStoreHandler)({storeId: testContext.storeId})
+			})
+);
+```
+
+It is worth noting that while the DSL can be used to write complex functional tests, in practice, a connector test's focus is more about making sure that operations are properly communicating with the underlying implementation instead of testing its functionality, but ultimately it is up to the developer to decide how much and what type of coverage suits a given connector best.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trayio/cdk-dsl",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "A DSL for connector development",
   "exports": {
     "./*": "./dist/*.js"