@btc-embedded/cdk-extensions 0.14.12 → 0.14.13

package/docs/RFC002-developer-stacks.md

@@ -1,268 +0,0 @@
-# Support for Developer Stacks
-
-## Objective
-
-This document describes how to provide custom stacks for developers that allow to deactivate components in order to replace them with local services for testing and debugging.
-
-## Target Projects
-
-The current implementation is targeting the EP 3.x project, i.e. the suggestions are based on the projects' needs. However, the design is intended to be used for future projects and extended to meet their requirements.
-
-## Prerequisites
-
-The architecture of the stack to be used has to be designed in such a way that single components can be used locally. This can be achieved by providing loose coupling between the components, e.g. by communicating via SQS or databases. The design is not part of this document, but without it, it is not possible to deactivate components and replace them with local ones.
-
-## Problem Statement
-
-There are two problems that we want to solve with this proposal.
-
-### Separate Developer Stacks
-
-The developers do not want to use the default stack deployed to the development stage, as messages and other data may be consumed by components and services deployed within the stage or by other developers. This causes unpredictable behavior and thus makes testing and debugging components hard or even impossible.
-
-By allowing each developer to deploy a separate stage, this can be circumvented, as all data is used only by a single developer and the default development stage is not affected.
-
-### Partial Stack Deployments
-
-In shall be possible to test and debug single components of a stack. This is not easily possible, when the component is deployed to the cloud. Thus, a developer wants to execute and debug a single component locally, while still using the cloud environment and all other components.
-
-By allowing to deactivate components in the stack, we can enable the developers to perform the local tests.
-
-## Requirements
-
-- It should be prevented to have configuration files for each developer, as managing and extending the files, especially if the schema has changed, may cause additional effort.
-- It should be prevented that excessive changes within the existing stack implementations have to be performed. Instead, the `cdk-extensions` should be adjusted to allow generic configurability.
-- Storage space should not be used excessively and thus files in a bucket should be deleted after a short period.
-- The stack should be deletable without any remaining constructs or files.
-
-## Proposed Solution
-
-The `cdk-extensions` are currently already used to instantiate stacks based on the provided configuration files. The entry point is the `createApp` function in the file `configFileParsings.ts`. This is the point where we can extend the stack generation and instrument it to generate custom stages. Further, we can extend the constructs and extensions provided by the `cdk-extensions` to allow deactivating them.
-
-### Separate Developer Stack
-
-Currently, the stack is generated from the configuration files using the `createApp` function. The stage ID is set via the contents of a specified directory, i.e. all configuration files in the directory are used as stage, and can be controlled by the variable `CDK_STAGE`, i.e. if the variable is set to a stage id, only the given stage will be generated and used.
-
-We propose to adjust the function s.t. we use the given stage id, but extend it with a suffix that identifies it as the developer's custom stack and stage. This can be achieved by a new environment variable. If the `DEVELOPER_STAGE_SUFFIX` environment variable is set, the value is appended to the stage id. Otherwise, the original id is used. This enables us to re-use an existing stacks and stage configurations, e.g. the one used for the development environment, without having to create a configuration file for each developer.
-
-With these changes, the developer is now able to generate and deploy a custom stage e.g. based on an existing developer stage by just setting two variables. Assuming that the developer stage is called `dev-env` and the developer has the initials `AbCd`, setting `CDK_STAGE=dev-env` and `DEVELOPER_STAGE_SUFFIX=AbCd` before deploying the stage will deploy a custom developer stage with the ID `dev-env-AbCd` without any required adjustments to the stack or the stage configuration file.
-
-### Partial Stack Deployment
-
-Within the stack code, constructs are added with custom IDs. With these IDs, the constructs can unambiguously identified. We propose to use the IDs to declare the constructs to be omitted from the stack by providing them in an environment variable  This enables the developer to deactivate components while no modification to the stack's source code is required.
-
-We propose the introduction of a new environment variable `DEACTIVATED_CONSTRUCTS` that may contain a space-separated list of component IDs that shall be deactivated during stack synthetization.
-
-For the considered use-case, it is sufficient to deactivate single constructs, thus no mechanism to deactivate groups of constructs is required.
-
-To not create a Cloud Formation template with missing references, we propose not to remove the constructs, but rather deactivate them. What this means is based on the constructs.
-
-For ECS containers, deactivating means that the number of desired instances is set to 0, preventing containers to be started while still being able to reference them. The containers them will not be able to consume messages from the SQS queue.
-
-For the other use-case, batch executions, we can deactivate the Event Pipe that triggers the batch execution. This can be done by deploying the Event Pipe with the desired state `STOPPED` which will cause it to be present, but not consuming messages from the triggering SQS queue.
-
-### File Retainment and Clean Deletion
-
-Deletion of a stack deployment will remove all artifacts that have been created during deployment. However, there is a limitation for S3 Buckets that they, based on their configuration, will not be able to be deleted when they still contain files. Further, buckets have no data expiration time by default, thus all files will remain indefinitely.
-
-We propose to provide a custom construct for S3 buckets that automatically sets the required parameters for data expiration and bucket removal when a developer stack instance is deployed. Besides using a different construct, no changes in the stack's implementation are required.
-
-### Technical Implementation
-
-This section describes the required changes to the `cdk-extensions` to implement the proposed solutions.
-
-#### Implementation of Separate Developer Stacks
-
-As mentioned before, the `createApp` function is used to instantiate the stacks. The stage ID is set via the name of the configuration file and filtered via the variable `CDK_STAGE`.
-
-As mentioned in the proposal, we want to extend the ID with a suffix for the developer. We can achieve this by providing a function to generate the stage id based on the given id and a possible set environment variable.
-
-    const stageSuffixVariable = "DEVELOPER_STAGE_SUFFIX";
-
-    function generateId(id: string): string {
-      let stageSuffixValue = process.env[stageSuffixVariable];
-      if (stageSuffixValue !== undefined) {
-        if (!stageSuffixValue.match(/^[a-zA-Z0-9]+$/)) {
-          throw new Error(
-            `The value set in ${stageSuffixVariable} may not be empty and may only contain alphanumeric characters.`,
-          );
-        }
-        return `${id}-${stageSuffixValue}`;
-      }
-      return id;
-    }
-
-If the `DEVELOPER_STAGE_SUFFIX` environment variable is set, the function appends the value (if valid) to the stack ID. Otherwise, the original id is returned.
-
-To use the possibly adjusted id, we have to extend the `AppStage` class that is used within the `createApp` function to use the new id generation function.
-
-    class AppStage extends Stage {
-      constructor(
-        scope: Construct,
-        id: string,
-        props: StageProps,
-        stackConfig: T,
-      ) {
-        super(scope, generateId(id), props);
-        createStack(this, stackConfig);
-      }
-
-      // ...
-    }
-
-These changes enable to implement the proposed deployment process. No adjustments to the individual stack implementations are required and the deployment is only controlled by the environment variables.
-
-#### Implementation of Partial Stack Deployment
-
-As mentioned before, the implementation is dependent on the construct to be deactivated. In our proposal, we focus on deactivating containers and Event Pipes. The following sections describe the proposed extensions to the `cdk-extensions`.
-
-##### General Helper Function
-
-A general function that allows to parse the environment variable `DEACTIVATED_CONSTRUCTS` is proposed as follows.
-
-    const deactivatedConstructsVariable = "DEACTIVATED_CONSTRUCTS";
-
-    export function isDeactivatedConstruct(name: string): boolean {
-      let deactivatedConstructsValue = process.env[deactivatedConstructsVariable];
-      if (deactivatedConstructsValue) {
-        const deactivatedComponents = deactivatedConstructsValue
-          ? deactivatedConstructsValue.split(/\s+/)
-          : [];
-        if (deactivatedComponents.includes(name)) {
-          return true;
-        }
-      }
-      return false;
-    }
-
-Using this function, the following implementations can easily determine, if a construct is marked to be deactivated and no separate parsers have to be implemented.
-
-##### Containers
-
-Containers are currently created using a `Service` that is using an extensible `ServiceDescription`. The `cdk-extensions` provide a set of extensions for the `ServiceDescription`.
-
-We propose to provide a new extension `DeactivateableServiceExtension`. This extension can be used the same way as the currently existing service extension.
-
-By adding a single line of code to the CDK implementation, the new extension can be used and thus the service for the container can be deactivated.
-
-    const serviceDescription = new ServiceDescription();
-    // add further extensions
-    // ...
-    serviceDescription.add(new DeactivatableServiceExtension());
-
-With this extension in use, the developer can deactivate the construct by adding the ID of the service to the `DEACTIVATED_CONSTRUCTS` variable.
-
-The extension can be implemented as follows.
-
-    export class DeactivatableServiceExtension extends ServiceExtension {
-      constructor() {
-        super("deactivatable-service");
-      }
-
-      modifyServiceProps(props: ServiceBuild): ServiceBuild {
-        if (isDeactivatedConstruct(this.parentService.id)) {
-          Annotations.of(this.parentService).addInfo(
-            `${this.parentService.id} is set to be deactivated.`,
-          );
-          return {
-            ...props,
-            desiredCount: 0,
-          };
-        }
-        return props;
-      }
-    }
-
-The extension will set the `desiredCount` for the task in the service properties, effectively deactivating the task while still retaining the definition.
-
-##### Event Pipes
-
-There is currently no implementation for an Event Pipe in the `cdk-extensions`. Defining an Event Pipe in the CDK implementation requires the use of an L1 construct. Therefore, we propose to provide a new construct for implementing the currently required use-cases for Event Pipes in the `cdk-extensions` that also allows to deactivate the Event Pipe. Implementing an own construct supports the developers, as it helps them to define filters and templates, as those will otherwise only be validated on deployment. The currently required use-cases are to have an SQS queue as source and another queue or a StepFunction as target.
-
-The implementation of the construct goes beyond the scope of this document. However, the implementation has to set the attribute `desiredState` of the `CfnPipeProps` passed to the `CfnPipe` constructor to `STOPPED`, if the ID of the pipe is contained in the list of deactivated constructs. We try to illustrate this in the following code snippet.
-
-    export class EventPipe extends Construct {
-      constructor(scope: Construct, id: string, props: EventPipeProps) {
-        super(scope, id);
-
-        // ...
-
-        let desiredState = "RUNNING";
-        if (isDeactivatedConstruct(id)) {
-          Annotations.of(this).addInfo(`${id} is set do be deactivated.`);
-          desiredState = "STOPPED";
-        }
-
-        // ...
-
-        new CfnPipe(this, "EventPipe", {
-          desiredState: desiredState,
-          // further properties
-          // ...
-        });
-      }
-    }
-
-This implementation will deploy a deactivated Event Pipe as `STOPPED`, preventing it from being executed.
-
-#### Implementation of Bucket Behavior
-
-The correct behavior of the bucket can be enforced by providing a custom construct that detects if a developer stack instance is deployed and sets the correct parameters for the original bucket construct.
-
-    function adjustProps(props?: BucketProps): BucketProps {
-      if (isDeveloperStack()) {
-        return {
-          ...props,
-          removalPolicy: RemovalPolicy.DESTROY,
-          autoDeleteObjects: true,
-          lifecycleRules: [
-            {
-              expiration: Duration.days(3),
-              abortIncompleteMultipartUploadAfter: Duration.days(1),
-            },
-          ],
-        };
-      }
-      return { ...props };
-    }
-
-    export class S3Bucket extends Bucket {
-      constructor(scope: Construct, id: string, props?: BucketProps) {
-        super(scope, id, adjustProps(props));
-      }
-    }
-
-If a developer stack is deployed (detectable by checking for the presence of the environment variable `DEVELOPER_STAGE_SUFFIX`), the correct removal policy as well as the option to delete objects on removal are set. Further, all objects are set to expire after 3 days. If no developer stack is deployed, the original properties are used. As the original `Bucket` construct is extended, no adjustments besides changing the construct creation are required in the stack implementation.
-
-### Caveats
-
-There are some caveats that have to be considered when providing the developer stacks for each project.
-
-#### Available Constructs
-
-It is not possible for the developer to retrieve a list of constructs that are available and able to be deactivated without reading the source code of the stack. A list of available values should be provided. As only a small set of constructs is expected to be deactivated, the effort for this is not that high.
-
-#### Supported Constructs
-
-As the deactivation is performed on L2 by deactivating constructs using their respective mechanism, only supported constructs can be deactivated. If more that the currently provided constructs are required to be deactivated, support for this has to be added to the `cdk-extensions`.
-
-#### Deployment Support
-
-To support the developers in using the developer stage, we can provide a custom Projen/Yarn script to interact with the developer stage.
-
-This can be achieved by extending the `.projenrc.ts` file of the project or providing Projen templates with the setting already set.
-
-    project.addScripts({
-      "cdk-dev":
-        'export DEVELOPER_STAGE_SUFFIX="$(git config user.alias)" && export CDK_STAGE="btc-es-dev-eu" && cdk',
-    });
-
-In the shown example, the developer stage name is taken from the Git configuration. Another source for an unambiguous identifier could be chosen. The content of the variable is checked within the `cdk-extensions` code.
-
-#### Unique Names
-
-Some component, e.g. Batch queue names or job definitions, do not have unique names in the stacks, i.e. deploying multiple stacks or multiple stages of the stack in the same account could lead to an error due to conflicting names. This has to be considered when developing the stack and the stage name might be added to the names.
-
-#### Multiple Stack Deployments
-
-For some projects, the stack is split into multiple projects, i.e. it is not deployed as a single stack but a combination of multiple stacks. This can be an issue, if the stacks depend on each other and elements are linked e.g. by their ARN. For generic dependencies like VPCs or security groups, this is not an issue, as there should be no issue, if multiple developer stages use the same VPC. This potential limitation has to be taken into consideration when designing the stacks.

package/docs/RFC003-referencing-platform-components.md

@@ -1,125 +0,0 @@
-# Referencing platform components in application-level stacks
-
-The CDK extensions library provides components which are intended to be
-instantiated in a _base platform_ stack as well as an _application-level_ stack.
-Such resources (and their functionality) should then be accessible to services
-developed and maintained in independently deployed, _application-level_ CDK
-apps. This document describes the platform component interface to provide this
-functionality.
-
-## Platform component interface
-
-Technically, the application-level stack references base platform components
-with the help of stack output parameters. To encapsulate this aspect, every base
-platform component has a static method `fromBasePlatform(scope: Construct, id:
-string, name: string)` which, given the base platform stack name, returns an
-instance of the platform component where the required lookup has been performed.
-
-The resulting component instance then supports access to base platform component
-properties or can even provide operations to the application-level stack (i.e.
-registering a target group to an ALB). Example:
-
-```typescript
-export interface IPlatformComponent {
-  readonly property: IVpcLink;
-  readonly securityGroup: ISecurityGroup;
-}
-
-export class PlatformComponent extends Construct implements IPlatformComponent {
-  public static fromBasePlatform(
-    scope: Construct,
-    id: string,
-    name: string,
-  ): IPlatformComponent {
-    // ... code requird to look up the output parameters ...
-
-    class Import extends Construct implements IApiGatewayV2 {
-      public readonly securityGroup: ISecurityGroup = /* ... */;
-      public readonly vpcLink: IVpcLink = /* ... */;
-    }
-
-    // create an instance satisfying the interface defined above
-    return new Import(scope, id);
-  }
-
-  readonly property: IVpcLink;
-  readonly securityGroup: ISecurityGroup;
-
-  // ... further component code, constructor, ...
-}
-```
-
-In more complex scenarios where functionality should be provided to the
-application-level stack, we can use a base class in order to provide
-functionality in the interface, such that it is also available at the platform
-component instance.
-
-```typescript
-
-export interface IPlatformComponent {
-  readonly property: IVpcLink;
-  readonly securityGroup: ISecurityGroup;
-
-  doSomething(target: IConnectable): void;
-}
-
-export abstract class PlatformComponentBase extends Construct implements IPlatformComponent {
-  abstract readonly property: IVpcLink;
-  abstract readonly securityGroup: ISecurityGroup;
-
-  doSomething(target: IConnectable) {
-    securityGroup.allowFrom(target);
-  }
-}
-
-export class PlatformComponent extends PlatformComponentBase {
-  public static fromBasePlatform(
-    scope: Construct,
-    id: string,
-    name: string,
-  ): IPlatformComponent {
-    // ... code requird to look up the output parameters ...
-
-    class Import extends PlatformComponentBase {
-      readonly securityGroup: ISecurityGroup = /* ... */;
-      readonly vpcLink: IVpcLink = /* ... */;
-    }
-
-    // create an instance satisfying the interface defined above
-    return new Import(scope, id);
-  }
-
-  readonly property: IVpcLink;
-  readonly securityGroup: ISecurityGroup;
-
-  // ... further component code, constructor, ...
-}
-```
-
-This approach provides means to encapsulate component-specific handling of stack
-parameters for transporting the information necessary to instantiate the
-component interface in the application.
-
-## Usage in the application-level stack
-
-The example below shows how to use a base platform component in the
-application-level stack:
-
-```typescript
-// ...
-
-const component = PlatformComponent.fromBasePlatform(
-  this.scope,
-  "PlatformComponent",
-  this.props.basePlatformStackName, // must be specified in the stack config
-);
-
-component.doSomething(service);
-
-// ...
-```
-
-During CDK synthesis, this code will instantiate the necessary lookups to the
-parameters exposed by the base platform. The interface returned by the static
-lookup method allows us to explicitly state the base platform component
-capabilities in the context of the application-level CDK.

package/docs/adrs/0001-use-adrs.md

@@ -1,36 +0,0 @@
----
-status: Accepted
-date: 2025-04-07
-deciders: Cloud Architecture & Technololgy
----
-
-# AD: Use ADRs to Document Architecture decision
-
-## Context and Problem Statement
-
-Architecture decisions are not visibile from the code strucutre itself. These decisions are therefor prone to be reiteratied on multiple times.
-
-## Decision Drivers
-
-* Desire to have a comprehensive documentation of the system
-* Have a industrie standard for documentation of architecture decisions
-* 
-
-## Considered Options
-
-1. Ad hoc documents
-2. Architecture Decision Records (ADRs)
-
-## Decision Outcome
-
-We decided on using ADRs since it is the only of the two options satisfying all decision drivers.
-
-### Consequences
-
-* Good, architecture decisions are documented.
-* Bad, you have to write an ADR for every decision.
-
-## More Information
-
-* See the link to the definiton of Architecture Decision Records [https://adr.github.io/](https://adr.github.io/)
-* See the link to the MADR Desicion tempates for more information on the temaplate [https://github.com/adr/madr/tree/3.0.0](https://github.com/adr/madr/tree/3.0.0)

package/docs/index.md

@@ -1,20 +0,0 @@
-# Project Structure Documentation
-
-
-The project is organized into the following directories and files:
-
-```
-/src
-    ├── constructs               
-    ├── extensions            
-    ├── platform
-    ├── utils
-/docs
-/test
-```
-
-# To Install it run
-
-```bash
-    npm i @btc-embedded/cdk-extensions
-```

package/mkdocs.yml

@@ -1,8 +0,0 @@
-site_name: 'BTC-ES CDK Extensions Docs'
-
-nav:
-  - Home: index.md
-  - RFC01 - API Gateway and Extensions: RFC001-api-gateway.md
-  - RFC02 - Developer Stacks: RFC002-developer-stacks.md
-plugins:
-  - techdocs-core