cdk-nuxt 2.0.1 → 2.2.0

package/README.md

@@ -15,6 +15,7 @@ Easily deploy Nuxt 3 applications via CDK on AWS including the following feature
 - Automatic upload of the build files for CSR and static assets to [S3](https://aws.amazon.com/s3/) with optimized caching rules
 - Scheduled pings of the Nuxt app to keep the Lambda warm for fast responses via [EventBridge](https://aws.amazon.com/eventbridge/) rules
 - Automatic cleanup of outdated static assets and build files
+- Access logs analysis via [Athena](https://aws.amazon.com/athena/) for the Nuxt app's CloudFront distribution
 
 ## Prerequisites
 
@@ -110,9 +111,20 @@ Defaults to 1792MB (optimized for costs and performance for standard Nuxt apps).
 ### enableTracing?: boolean
 Whether to enable AWS X-Ray for the Nuxt Lambda function.
 
+### enableApi?: boolean
+Whether to enable (HTTPS only) API access to the Nuxt app via the `/api` path which support all HTTP methods. 
+See https://nuxt.com/docs/guide/directory-structure/server#recipes for details.
+
 ### enableSitemap?: boolean
 Whether to enable a global Sitemap bucket which is permanently accessible through multiple deployments.
 
+### enableAccessLogsAnalysis?: boolean
+Whether to enable access logs analysis for the Nuxt app's CloudFront distribution via Athena.
+
+### accessLogCookies?: string[]
+An array of cookies to include for reporting in the access logs analysis.
+Only has an effect when `enableAccessLogsAnalysis` is set to `true`.
+
 ### outdatedAssetsRetentionDays?: boolean
 The number of days to retain static assets of outdated deployments in the S3 bucket.
 Useful to allow users to still access old assets after a new deployment when they are still browsing on an old version.
@@ -172,7 +184,42 @@ Afterwards, the CDK stack will be deployed to AWS.
 node_modules/.bin/cdk-nuxt-deploy-server
 ```
 
-## Optional: Automatically deploy on every push (CD) via [GitHub Actions](https://github.com/features/actions)
+## Destroy the Stack
+
+If you want to destroy the stack and all its resources (including storage, e.g., access logs), run the following script:
+
+```bash
+node_modules/.bin/cdk-nuxt-destroy-server
+```
+
+## Reference: Created AWS Resources
+
+In the following, you can find an overview of the AWS resources that will be created by this package for reference.
+
+### NuxtServerAppStack
+
+This stack is responsible for deploying dynamic Nuxt 3 apps to AWS.
+The following AWS resources will be created by this stack:
+
+- [Lambda](https://aws.amazon.com/lambda/):
+    - A Lambda function to render the Nuxt app including a separated Lambda layer to provide the `node_modules` of the Nuxt app required for server-side rendering.
+    - A Lambda function that deletes the outdated static assets of the Nuxt app from S3.
+- [S3](https://aws.amazon.com/s3/):
+    - A bucket to store the client files and static assets of the Nuxt build (`.nuxt/dist/client`) with optimized cache settings.
+    - A bucket to store the CloudFront access logs for analysis via Athena. Only created if `enableAccessLogsAnalysis` is set to `true`.
+- [Route53](https://aws.amazon.com/route53/): Two DNS records (`A` for IPv4 and `AAAA` for IPv6) in the configured hosted zone to make the Nuxt app available on the internet via the configured custom domain.
+- [API Gateway](https://aws.amazon.com/api-gateway/): An HTTP API to make the Nuxt Lambda function publicly available.
+- [CloudFront](https://aws.amazon.com/cloudfront/): A distribution to route incoming requests to the Nuxt Lambda function (via the API Gateway) and the S3 bucket to serve the static assets for the Nuxt app.
+- [EventBridge](https://aws.amazon.com/eventbridge/):
+    - A scheduled rule to ping the Nuxt app's Lambda function every 5 minutes in order to keep it warm and to speed up initial SSR requests.
+    - A scheduled rule to trigger the cleanup Lambda function for deleting the outdated static assets of the Nuxt app from S3 every tuesday at 03:30 AM GMT.
+- [Athena](https://aws.amazon.com/athena/): A database and table to analyze the access logs of the Nuxt app's CloudFront distribution. Only created if `enableAccessLogsAnalysis` is set to `true`.
+
+## Guidelines
+
+In the following, you can find some guidelines for the deployment and usages of this package.
+
+### Automatically deploy on every push (CD) via [GitHub Actions](https://github.com/features/actions)
 
 Feel free to copy the following GitHub Actions YAML file content into a YAML file at `.github/workflows/deploy.yml` to automatically build and deploy the Nuxt app to AWS on every push to a specific branch.<br/>This only works if you're using GitHub for the project's VCS repository.
 
@@ -222,21 +269,3 @@ jobs:
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
 ```
-
-## Advanced: Used AWS Resources
-
-### NuxtServerAppStack
-
-This stack is responsible for deploying dynamic Nuxt 3 apps to AWS.
-The following AWS resources will be created by this stack:
-
-- [Lambda](https://aws.amazon.com/lambda/): 
-  - A Lambda function to render the Nuxt app including a separated Lambda layer to provide the `node_modules` of the Nuxt app required for server-side rendering.
-  - A Lambda function that deletes the outdated static assets of the Nuxt app from S3.
-- [S3](https://aws.amazon.com/s3/): A bucket to store the client files of the Nuxt build (`.nuxt/dist/client`) and the custom static files of the Nuxt app (`static`) with optimized cache settings.
-- [Route53](https://aws.amazon.com/route53/): Two DNS records (`A` for IPv4 and `AAAA` for IPv6) in the configured hosted zone to make the Nuxt app available on the internet via the configured custom domain.
-- [API Gateway](https://aws.amazon.com/api-gateway/): An HTTP API to make the Nuxt Lambda function publicly available.
-- [CloudFront](https://aws.amazon.com/cloudfront/): A distribution to route incoming requests to the Nuxt Lambda function (via the API Gateway) and the S3 bucket to serve the static assets for the Nuxt app.
-- [EventBridge](https://aws.amazon.com/eventbridge/): 
-  - A scheduled rule to ping the Nuxt app's Lambda function every 5 minutes in order to keep it warm and to speed up initial SSR requests.
-  - A scheduled rule to trigger the cleanup Lambda function for deleting the outdated static assets of the Nuxt app from S3 every tuesday at 03:30 AM GMT.

package/index.d.ts

@@ -1 +1,2 @@
-export { NuxtServerAppStack, NuxtServerAppStackProps } from "./lib/stack/server/nuxt-server-app-stack";
+export { NuxtServerAppStack } from "./lib/stack/server/NuxtServerAppStack";
+export { NuxtServerAppStackProps } from "./lib/stack/server/NuxtServerAppStackProps";

package/index.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.NuxtServerAppStack = void 0;
-var nuxt_server_app_stack_1 = require("./lib/stack/server/nuxt-server-app-stack");
-Object.defineProperty(exports, "NuxtServerAppStack", { enumerable: true, get: function () { return nuxt_server_app_stack_1.NuxtServerAppStack; } });
-//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaW5kZXguanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyJpbmRleC50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUFBQSxrRkFBb0c7QUFBNUYsMkhBQUEsa0JBQWtCLE9BQUEiLCJzb3VyY2VzQ29udGVudCI6WyJleHBvcnQge051eHRTZXJ2ZXJBcHBTdGFjaywgTnV4dFNlcnZlckFwcFN0YWNrUHJvcHN9IGZyb20gXCIuL2xpYi9zdGFjay9zZXJ2ZXIvbnV4dC1zZXJ2ZXItYXBwLXN0YWNrXCJcbiJdfQ==
+var NuxtServerAppStack_1 = require("./lib/stack/server/NuxtServerAppStack");
+Object.defineProperty(exports, "NuxtServerAppStack", { enumerable: true, get: function () { return NuxtServerAppStack_1.NuxtServerAppStack; } });
+//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaW5kZXguanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyJpbmRleC50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUFBQSw0RUFBd0U7QUFBaEUsd0hBQUEsa0JBQWtCLE9BQUEiLCJzb3VyY2VzQ29udGVudCI6WyJleHBvcnQge051eHRTZXJ2ZXJBcHBTdGFja30gZnJvbSBcIi4vbGliL3N0YWNrL3NlcnZlci9OdXh0U2VydmVyQXBwU3RhY2tcIlxuZXhwb3J0IHtOdXh0U2VydmVyQXBwU3RhY2tQcm9wc30gZnJvbSBcIi4vbGliL3N0YWNrL3NlcnZlci9OdXh0U2VydmVyQXBwU3RhY2tQcm9wc1wiXG4iXX0=

package/lib/cli/deploy-server.js

@@ -13,16 +13,44 @@ const deploymentSourceFolder = '.output/server';
 shell.echo(`${logPrefix}: Deleting outdated CDK files...`);
 shell.rm('-rf', 'cdk.out');
 
-// Preparing the assets cleanup lambda function
-shell.echo(`${logPrefix}: Installing the assets cleanup lambda function...`);
+// Preparing the assets cleanup Lambda function
+shell.echo(`${logPrefix}: Installing the assets cleanup Lambda function...`);
 shell.cd(path.join(__dirname, '../functions/assets-cleanup'));
 if (shell.exec('yarn install --frozen-lockfile && yarn install-layer').code !== 0) {
-    shell.echo(`${logPrefix} Error: Installation of assets cleanup lambda function failed.`);
+    shell.echo(`${logPrefix} Error: Installation of assets cleanup Lambda function failed.`);
     shell.exit(1);
 }
-shell.echo(`${logPrefix}: Building the assets cleanup lambda function...`);
+shell.echo(`${logPrefix}: Building the assets cleanup Lambda function...`);
 if (shell.exec('yarn build').code !== 0) {
-    shell.echo(`${logPrefix} Error: Build of assets cleanup lambda function failed.`);
+    shell.echo(`${logPrefix} Error: Build of assets cleanup Lambda function failed.`);
+    shell.exit(1);
+}
+shell.cd(rootFolder);
+
+// Preparing the access logs analysis group-by-date Lambda function
+shell.echo(`${logPrefix}: Installing the access logs analysis group-by-date Lambda function...`);
+shell.cd(path.join(__dirname, '../functions/access-logs-analysis/group-by-date'));
+if (shell.exec('yarn install --frozen-lockfile && yarn install-layer').code !== 0) {
+    shell.echo(`${logPrefix} Error: Installation of access logs analysis group-by-date Lambda function failed.`);
+    shell.exit(1);
+}
+shell.echo(`${logPrefix}: Building the access logs analysis group-by-date Lambda function...`);
+if (shell.exec('yarn build').code !== 0) {
+    shell.echo(`${logPrefix} Error: Build of access logs analysis group-by-date Lambda function failed.`);
+    shell.exit(1);
+}
+shell.cd(rootFolder);
+
+// Preparing the access logs analysis partitioning Lambda function
+shell.echo(`${logPrefix}: Installing the access logs analysis partitioning Lambda function...`);
+shell.cd(path.join(__dirname, '../functions/access-logs-analysis/partitioning'));
+if (shell.exec('yarn install --frozen-lockfile && yarn install-layer').code !== 0) {
+    shell.echo(`${logPrefix} Error: Installation of access logs analysis partitioning Lambda function failed.`);
+    shell.exit(1);
+}
+shell.echo(`${logPrefix}: Building the access logs analysis partitioning Lambda function...`);
+if (shell.exec('yarn build').code !== 0) {
+    shell.echo(`${logPrefix} Error: Build of access logs analysis partitioning Lambda function failed.`);
     shell.exit(1);
 }
 shell.cd(rootFolder);

package/lib/cli/destroy-server.js

@@ -0,0 +1,13 @@
+#!/usr/bin/env node
+
+const shell = require("shelljs");
+const logPrefix = 'CDK Nuxt Destroy';
+
+shell.echo(`${logPrefix}: Destroying stack on AWS via CDK...`);
+if (shell.exec('yarn cdk destroy --require-approval never --all --app="yarn ts-node stack/index.ts" ' + process.argv.slice(2)).code !== 0) {
+    shell.echo(`${logPrefix} Error: CDK destroy failed.`);
+    shell.exit(1);
+}
+
+shell.echo(`${logPrefix}: CDK destroy successful.`);
+

package/lib/functions/access-logs-analysis/group-by-date/.gitignore

@@ -0,0 +1,2 @@
+node_modules
+build

package/lib/functions/access-logs-analysis/group-by-date/build/app/index.d.ts

@@ -0,0 +1,2 @@
+import type { S3Event } from 'aws-lambda';
+export declare const handler: (event: S3Event) => Promise<void>;

package/lib/functions/access-logs-analysis/group-by-date/build/app/index.js

@@ -0,0 +1,57 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.handler = void 0;
+/**
+ * This script moves access logs into a target folder hierarchy, structured by year, month, day, and hour (UTC).
+ * That way, the logs can easier be processed with Athena.
+ * Taken and adjusted from https://github.com/aws-samples/amazon-cloudfront-access-logs-queries.
+ * This Lambda supports access logs from both: CloudFront and S3.
+ */
+const client_s3_1 = require("@aws-sdk/client-s3");
+// without leading and trailing slashes
+const targetFolder = process.env.TARGET_FOLDER;
+if (!targetFolder) {
+    throw new Error('Required environment variable TARGET_FOLDER missing!');
+}
+if (!process.env.RAW_ACCESS_LOG_FILE_PATTERN) {
+    throw new Error('Required environment variable RAW_ACCESS_LOG_FILE_PATTERN missing!');
+}
+// unfortunately, CloudFront logs and S3 logs are completely different. That's why the pattern to identify
+// unprocessed log files need to be configurable
+const rawAccessLogFilePattern = new RegExp(process.env.RAW_ACCESS_LOG_FILE_PATTERN);
+// matches for everything after the last slash
+const filenamePattern = /[^/]+$/;
+const s3Client = new client_s3_1.S3Client({ region: process.env.AWS_REGION });
+const internalHandler = async (event) => {
+    try {
+        const pendingMoves = event.Records.map(async (record) => {
+            const bucket = record.s3.bucket.name;
+            const sourceKey = record.s3.object.key;
+            const match = rawAccessLogFilePattern.exec(sourceKey);
+            // skip other files/folder
+            if (match?.groups) {
+                const { year, month, day, hour } = match.groups;
+                const filename = filenamePattern.exec(sourceKey)[0];
+                const targetKey = `${targetFolder}/year=${year}/month=${month}/day=${day}/hour=${hour}/${filename}`;
+                const copyCommand = new client_s3_1.CopyObjectCommand({
+                    Bucket: bucket,
+                    Key: targetKey,
+                    CopySource: bucket + '/' + sourceKey,
+                });
+                await s3Client.send(copyCommand);
+                const deleteSourceCommand = new client_s3_1.DeleteObjectCommand({
+                    Bucket: bucket,
+                    Key: sourceKey,
+                });
+                await s3Client.send(deleteSourceCommand);
+            }
+        });
+        const processed = await Promise.all(pendingMoves);
+        console.log(`successfully moved ${processed.length} files`);
+    }
+    catch (error) {
+        console.error('### unexpected runtime error ###', error);
+    }
+};
+exports.handler = internalHandler;
+//# sourceMappingURL=index.js.map

package/lib/functions/access-logs-analysis/group-by-date/build/app/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"/","sources":["index.ts"],"names":[],"mappings":";;;AAAA;;;;;GAKG;AACH,kDAAoF;AAGpF,uCAAuC;AACvC,MAAM,YAAY,GAAG,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;AAC/C,IAAI,CAAC,YAAY,EAAE;IACjB,MAAM,IAAI,KAAK,CAAC,sDAAsD,CAAC,CAAC;CACzE;AACD,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,2BAA2B,EAAE;IAC5C,MAAM,IAAI,KAAK,CAAC,oEAAoE,CAAC,CAAC;CACvF;AACD,0GAA0G;AAC1G,gDAAgD;AAChD,MAAM,uBAAuB,GAAG,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,2BAA2B,CAAC,CAAC;AAEpF,8CAA8C;AAC9C,MAAM,eAAe,GAAG,QAAQ,CAAC;AAEjC,MAAM,QAAQ,GAAG,IAAI,oBAAQ,CAAC,EAAC,MAAM,EAAE,OAAO,CAAC,GAAG,CAAC,UAAU,EAAC,CAAC,CAAC;AAEhE,MAAM,eAAe,GAAG,KAAK,EAAE,KAAc,EAAE,EAAE;IAC/C,IAAI;QACF,MAAM,YAAY,GAAG,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAC,MAAM,EAAC,EAAE;YACpD,MAAM,MAAM,GAAG,MAAM,CAAC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC;YACrC,MAAM,SAAS,GAAG,MAAM,CAAC,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC;YACvC,MAAM,KAAK,GAAG,uBAAuB,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YAEtD,0BAA0B;YAC1B,IAAI,KAAK,EAAE,MAAM,EAAE;gBACjB,MAAM,EAAC,IAAI,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,EAAC,GAAG,KAAK,CAAC,MAAM,CAAC;gBAC9C,MAAM,QAAQ,GAAG,eAAe,CAAC,IAAI,CAAC,SAAS,CAAE,CAAC,CAAC,CAAC,CAAC;gBACrD,MAAM,SAAS,GAAG,GAAG,YAAY,SAAS,IAAI,UAAU,KAAK,QAAQ,GAAG,SAAS,IAAI,IAAI,QAAQ,EAAE,CAAC;gBAEpG,MAAM,WAAW,GAAG,IAAI,6BAAiB,CAAC;oBACxC,MAAM,EAAE,MAAM;oBACd,GAAG,EAAE,SAAS;oBACd,UAAU,EAAE,MAAM,GAAG,GAAG,GAAG,SAAS;iBACrC,CAAC,CAAC;gBAEH,MAAM,QAAQ,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;gBAEjC,MAAM,mBAAmB,GAAG,IAAI,+BAAmB,CAAC;oBAClD,MAAM,EAAE,MAAM;oBACd,GAAG,EAAE,SAAS;iBACf,CAAC,CAAC;gBAEH,MAAM,QAAQ,CAAC,IAAI,CAAC,mBAAmB,CAAC,CAAC;aAC1C;QACH,CAAC,CAAC,CAAC;QAEH,MAAM,SAAS,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC;QAClD,OAAO,CAAC,GAAG,CAAC,sBAAsB,SAAS,CAAC,MAAM,QAAQ,CAAC,CAAC;KAC7D;IAAC,OAAO,KAAK,EAAE;QACd,OAAO,CAAC,KAAK,CAAC,kCAAkC,EAAE,KAAK,CAAC,CAAC;KAC1D;AACH,CAAC,CAAC;AAEW,QAAA,OAAO,GAAG,eAAe,CAAC","sourcesContent":["/**\n * This script moves access logs into a target folder hierarchy, structured by year, month, day, and hour (UTC).\n * That way, the logs can easier be processed with Athena.\n * Taken and adjusted from https://github.com/aws-samples/amazon-cloudfront-access-logs-queries.\n * This Lambda supports access logs from both: CloudFront and S3.\n */\nimport {S3Client, CopyObjectCommand, DeleteObjectCommand} from '@aws-sdk/client-s3';\nimport type {S3Event} from 'aws-lambda';\n\n// without leading and trailing slashes\nconst targetFolder = process.env.TARGET_FOLDER;\nif (!targetFolder) {\n  throw new Error('Required environment variable TARGET_FOLDER missing!');\n}\nif (!process.env.RAW_ACCESS_LOG_FILE_PATTERN) {\n  throw new Error('Required environment variable RAW_ACCESS_LOG_FILE_PATTERN missing!');\n}\n// unfortunately, CloudFront logs and S3 logs are completely different. That's why the pattern to identify\n// unprocessed log files need to be configurable\nconst rawAccessLogFilePattern = new RegExp(process.env.RAW_ACCESS_LOG_FILE_PATTERN);\n\n// matches for everything after the last slash\nconst filenamePattern = /[^/]+$/;\n\nconst s3Client = new S3Client({region: process.env.AWS_REGION});\n\nconst internalHandler = async (event: S3Event) => {\n  try {\n    const pendingMoves = event.Records.map(async record => {\n      const bucket = record.s3.bucket.name;\n      const sourceKey = record.s3.object.key;\n      const match = rawAccessLogFilePattern.exec(sourceKey);\n\n      // skip other files/folder\n      if (match?.groups) {\n        const {year, month, day, hour} = match.groups;\n        const filename = filenamePattern.exec(sourceKey)![0];\n        const targetKey = `${targetFolder}/year=${year}/month=${month}/day=${day}/hour=${hour}/${filename}`;\n\n        const copyCommand = new CopyObjectCommand({\n          Bucket: bucket,\n          Key: targetKey,\n          CopySource: bucket + '/' + sourceKey,\n        });\n\n        await s3Client.send(copyCommand);\n\n        const deleteSourceCommand = new DeleteObjectCommand({\n          Bucket: bucket,\n          Key: sourceKey,\n        });\n\n        await s3Client.send(deleteSourceCommand);\n      }\n    });\n\n    const processed = await Promise.all(pendingMoves);\n    console.log(`successfully moved ${processed.length} files`);\n  } catch (error) {\n    console.error('### unexpected runtime error ###', error);\n  }\n};\n\nexport const handler = internalHandler;\n"]}

package/lib/functions/access-logs-analysis/group-by-date/index.d.ts

@@ -0,0 +1,2 @@
+import type { S3Event } from 'aws-lambda';
+export declare const handler: (event: S3Event) => Promise<void>;

package/lib/functions/access-logs-analysis/group-by-date/index.js

@@ -0,0 +1,57 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.handler = void 0;
+/**
+ * This script moves access logs into a target folder hierarchy, structured by year, month, day, and hour (UTC).
+ * That way, the logs can easier be processed with Athena.
+ * Taken and adjusted from https://github.com/aws-samples/amazon-cloudfront-access-logs-queries.
+ * This Lambda supports access logs from both: CloudFront and S3.
+ */
+const client_s3_1 = require("@aws-sdk/client-s3");
+// without leading and trailing slashes
+const targetFolder = process.env.TARGET_FOLDER;
+if (!targetFolder) {
+    throw new Error('Required environment variable TARGET_FOLDER missing!');
+}
+if (!process.env.RAW_ACCESS_LOG_FILE_PATTERN) {
+    throw new Error('Required environment variable RAW_ACCESS_LOG_FILE_PATTERN missing!');
+}
+// unfortunately, CloudFront logs and S3 logs are completely different. That's why the pattern to identify
+// unprocessed log files need to be configurable
+const rawAccessLogFilePattern = new RegExp(process.env.RAW_ACCESS_LOG_FILE_PATTERN);
+// matches for everything after the last slash
+const filenamePattern = /[^/]+$/;
+const s3Client = new client_s3_1.S3Client({ region: process.env.AWS_REGION });
+const internalHandler = async (event) => {
+    try {
+        const pendingMoves = event.Records.map(async (record) => {
+            const bucket = record.s3.bucket.name;
+            const sourceKey = record.s3.object.key;
+            const match = rawAccessLogFilePattern.exec(sourceKey);
+            // skip other files/folder
+            if (match === null || match === void 0 ? void 0 : match.groups) {
+                const { year, month, day, hour } = match.groups;
+                const filename = filenamePattern.exec(sourceKey)[0];
+                const targetKey = `${targetFolder}/year=${year}/month=${month}/day=${day}/hour=${hour}/${filename}`;
+                const copyCommand = new client_s3_1.CopyObjectCommand({
+                    Bucket: bucket,
+                    Key: targetKey,
+                    CopySource: bucket + '/' + sourceKey,
+                });
+                await s3Client.send(copyCommand);
+                const deleteSourceCommand = new client_s3_1.DeleteObjectCommand({
+                    Bucket: bucket,
+                    Key: sourceKey,
+                });
+                await s3Client.send(deleteSourceCommand);
+            }
+        });
+        const processed = await Promise.all(pendingMoves);
+        console.log(`successfully moved ${processed.length} files`);
+    }
+    catch (error) {
+        console.error('### unexpected runtime error ###', error);
+    }
+};
+exports.handler = internalHandler;
+//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaW5kZXguanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyJpbmRleC50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUFBQTs7Ozs7R0FLRztBQUNILGtEQUFvRjtBQUdwRix1Q0FBdUM7QUFDdkMsTUFBTSxZQUFZLEdBQUcsT0FBTyxDQUFDLEdBQUcsQ0FBQyxhQUFhLENBQUM7QUFDL0MsSUFBSSxDQUFDLFlBQVksRUFBRTtJQUNqQixNQUFNLElBQUksS0FBSyxDQUFDLHNEQUFzRCxDQUFDLENBQUM7Q0FDekU7QUFDRCxJQUFJLENBQUMsT0FBTyxDQUFDLEdBQUcsQ0FBQywyQkFBMkIsRUFBRTtJQUM1QyxNQUFNLElBQUksS0FBSyxDQUFDLG9FQUFvRSxDQUFDLENBQUM7Q0FDdkY7QUFDRCwwR0FBMEc7QUFDMUcsZ0RBQWdEO0FBQ2hELE1BQU0sdUJBQXVCLEdBQUcsSUFBSSxNQUFNLENBQUMsT0FBTyxDQUFDLEdBQUcsQ0FBQywyQkFBMkIsQ0FBQyxDQUFDO0FBRXBGLDhDQUE4QztBQUM5QyxNQUFNLGVBQWUsR0FBRyxRQUFRLENBQUM7QUFFakMsTUFBTSxRQUFRLEdBQUcsSUFBSSxvQkFBUSxDQUFDLEVBQUMsTUFBTSxFQUFFLE9BQU8sQ0FBQyxHQUFHLENBQUMsVUFBVSxFQUFDLENBQUMsQ0FBQztBQUVoRSxNQUFNLGVBQWUsR0FBRyxLQUFLLEVBQUUsS0FBYyxFQUFFLEVBQUU7SUFDL0MsSUFBSTtRQUNGLE1BQU0sWUFBWSxHQUFHLEtBQUssQ0FBQyxPQUFPLENBQUMsR0FBRyxDQUFDLEtBQUssRUFBQyxNQUFNLEVBQUMsRUFBRTtZQUNwRCxNQUFNLE1BQU0sR0FBRyxNQUFNLENBQUMsRUFBRSxDQUFDLE1BQU0sQ0FBQyxJQUFJLENBQUM7WUFDckMsTUFBTSxTQUFTLEdBQUcsTUFBTSxDQUFDLEVBQUUsQ0FBQyxNQUFNLENBQUMsR0FBRyxDQUFDO1lBQ3ZDLE1BQU0sS0FBSyxHQUFHLHVCQUF1QixDQUFDLElBQUksQ0FBQyxTQUFTLENBQUMsQ0FBQztZQUV0RCwwQkFBMEI7WUFDMUIsSUFBSSxLQUFLLGFBQUwsS0FBSyx1QkFBTCxLQUFLLENBQUUsTUFBTSxFQUFFO2dCQUNqQixNQUFNLEVBQUMsSUFBSSxFQUFFLEtBQUssRUFBRSxHQUFHLEVBQUUsSUFBSSxFQUFDLEdBQUcsS0FBSyxDQUFDLE1BQU0sQ0FBQztnQkFDOUMsTUFBTSxRQUFRLEdBQUcsZUFBZSxDQUFDLElBQUksQ0FBQyxTQUFTLENBQUUsQ0FBQyxDQUFDLENBQUMsQ0FBQztnQkFDckQsTUFBTSxTQUFTLEdBQUcsR0FBRyxZQUFZLFNBQVMsSUFBSSxVQUFVLEtBQUssUUFBUSxHQUFHLFNBQVMsSUFBSSxJQUFJLFFBQVEsRUFBRSxDQUFDO2dCQUVwRyxNQUFNLFdBQVcsR0FBRyxJQUFJLDZCQUFpQixDQUFDO29CQUN4QyxNQUFNLEVBQUUsTUFBTTtvQkFDZCxHQUFHLEVBQUUsU0FBUztvQkFDZCxVQUFVLEVBQUUsTUFBTSxHQUFHLEdBQUcsR0FBRyxTQUFTO2lCQUNyQyxDQUFDLENBQUM7Z0JBRUgsTUFBTSxRQUFRLENBQUMsSUFBSSxDQUFDLFdBQVcsQ0FBQyxDQUFDO2dCQUVqQyxNQUFNLG1CQUFtQixHQUFHLElBQUksK0JBQW1CLENBQUM7b0JBQ2xELE1BQU0sRUFBRSxNQUFNO29CQUNkLEdBQUcsRUFBRSxTQUFTO2lCQUNmLENBQUMsQ0FBQztnQkFFSCxNQUFNLFFBQVEsQ0FBQyxJQUFJLENBQUMsbUJBQW1CLENBQUMsQ0FBQzthQUMxQztRQUNILENBQUMsQ0FBQyxDQUFDO1FBRUgsTUFBTSxTQUFTLEdBQUcsTUFBTSxPQUFPLENBQUMsR0FBRyxDQUFDLFlBQVksQ0FBQyxDQUFDO1FBQ2xELE9BQU8sQ0FBQyxHQUFHLENBQUMsc0JBQXNCLFNBQVMsQ0FBQyxNQUFNLFFBQVEsQ0FBQyxDQUFDO0tBQzdEO0lBQUMsT0FBTyxLQUFLLEVBQUU7UUFDZCxPQUFPLENBQUMsS0FBSyxDQUFDLGtDQUFrQyxFQUFFLEtBQUssQ0FBQyxDQUFDO0tBQzFEO0FBQ0gsQ0FBQyxDQUFDO0FBRVcsUUFBQSxPQUFPLEdBQUcsZUFBZSxDQUFDIiwic291cmNlc0NvbnRlbnQiOlsiLyoqXG4gKiBUaGlzIHNjcmlwdCBtb3ZlcyBhY2Nlc3MgbG9ncyBpbnRvIGEgdGFyZ2V0IGZvbGRlciBoaWVyYXJjaHksIHN0cnVjdHVyZWQgYnkgeWVhciwgbW9udGgsIGRheSwgYW5kIGhvdXIgKFVUQykuXG4gKiBUaGF0IHdheSwgdGhlIGxvZ3MgY2FuIGVhc2llciBiZSBwcm9jZXNzZWQgd2l0aCBBdGhlbmEuXG4gKiBUYWtlbiBhbmQgYWRqdXN0ZWQgZnJvbSBodHRwczovL2dpdGh1Yi5jb20vYXdzLXNhbXBsZXMvYW1hem9uLWNsb3VkZnJvbnQtYWNjZXNzLWxvZ3MtcXVlcmllcy5cbiAqIFRoaXMgTGFtYmRhIHN1cHBvcnRzIGFjY2VzcyBsb2dzIGZyb20gYm90aDogQ2xvdWRGcm9udCBhbmQgUzMuXG4gKi9cbmltcG9ydCB7UzNDbGllbnQsIENvcHlPYmplY3RDb21tYW5kLCBEZWxldGVPYmplY3RDb21tYW5kfSBmcm9tICdAYXdzLXNkay9jbGllbnQtczMnO1xuaW1wb3J0IHR5cGUge1MzRXZlbnR9IGZyb20gJ2F3cy1sYW1iZGEnO1xuXG4vLyB3aXRob3V0IGxlYWRpbmcgYW5kIHRyYWlsaW5nIHNsYXNoZXNcbmNvbnN0IHRhcmdldEZvbGRlciA9IHByb2Nlc3MuZW52LlRBUkdFVF9GT0xERVI7XG5pZiAoIXRhcmdldEZvbGRlcikge1xuICB0aHJvdyBuZXcgRXJyb3IoJ1JlcXVpcmVkIGVudmlyb25tZW50IHZhcmlhYmxlIFRBUkdFVF9GT0xERVIgbWlzc2luZyEnKTtcbn1cbmlmICghcHJvY2Vzcy5lbnYuUkFXX0FDQ0VTU19MT0dfRklMRV9QQVRURVJOKSB7XG4gIHRocm93IG5ldyBFcnJvcignUmVxdWlyZWQgZW52aXJvbm1lbnQgdmFyaWFibGUgUkFXX0FDQ0VTU19MT0dfRklMRV9QQVRURVJOIG1pc3NpbmchJyk7XG59XG4vLyB1bmZvcnR1bmF0ZWx5LCBDbG91ZEZyb250IGxvZ3MgYW5kIFMzIGxvZ3MgYXJlIGNvbXBsZXRlbHkgZGlmZmVyZW50LiBUaGF0J3Mgd2h5IHRoZSBwYXR0ZXJuIHRvIGlkZW50aWZ5XG4vLyB1bnByb2Nlc3NlZCBsb2cgZmlsZXMgbmVlZCB0byBiZSBjb25maWd1cmFibGVcbmNvbnN0IHJhd0FjY2Vzc0xvZ0ZpbGVQYXR0ZXJuID0gbmV3IFJlZ0V4cChwcm9jZXNzLmVudi5SQVdfQUNDRVNTX0xPR19GSUxFX1BBVFRFUk4pO1xuXG4vLyBtYXRjaGVzIGZvciBldmVyeXRoaW5nIGFmdGVyIHRoZSBsYXN0IHNsYXNoXG5jb25zdCBmaWxlbmFtZVBhdHRlcm4gPSAvW14vXSskLztcblxuY29uc3QgczNDbGllbnQgPSBuZXcgUzNDbGllbnQoe3JlZ2lvbjogcHJvY2Vzcy5lbnYuQVdTX1JFR0lPTn0pO1xuXG5jb25zdCBpbnRlcm5hbEhhbmRsZXIgPSBhc3luYyAoZXZlbnQ6IFMzRXZlbnQpID0+IHtcbiAgdHJ5IHtcbiAgICBjb25zdCBwZW5kaW5nTW92ZXMgPSBldmVudC5SZWNvcmRzLm1hcChhc3luYyByZWNvcmQgPT4ge1xuICAgICAgY29uc3QgYnVja2V0ID0gcmVjb3JkLnMzLmJ1Y2tldC5uYW1lO1xuICAgICAgY29uc3Qgc291cmNlS2V5ID0gcmVjb3JkLnMzLm9iamVjdC5rZXk7XG4gICAgICBjb25zdCBtYXRjaCA9IHJhd0FjY2Vzc0xvZ0ZpbGVQYXR0ZXJuLmV4ZWMoc291cmNlS2V5KTtcblxuICAgICAgLy8gc2tpcCBvdGhlciBmaWxlcy9mb2xkZXJcbiAgICAgIGlmIChtYXRjaD8uZ3JvdXBzKSB7XG4gICAgICAgIGNvbnN0IHt5ZWFyLCBtb250aCwgZGF5LCBob3VyfSA9IG1hdGNoLmdyb3VwcztcbiAgICAgICAgY29uc3QgZmlsZW5hbWUgPSBmaWxlbmFtZVBhdHRlcm4uZXhlYyhzb3VyY2VLZXkpIVswXTtcbiAgICAgICAgY29uc3QgdGFyZ2V0S2V5ID0gYCR7dGFyZ2V0Rm9sZGVyfS95ZWFyPSR7eWVhcn0vbW9udGg9JHttb250aH0vZGF5PSR7ZGF5fS9ob3VyPSR7aG91cn0vJHtmaWxlbmFtZX1gO1xuXG4gICAgICAgIGNvbnN0IGNvcHlDb21tYW5kID0gbmV3IENvcHlPYmplY3RDb21tYW5kKHtcbiAgICAgICAgICBCdWNrZXQ6IGJ1Y2tldCxcbiAgICAgICAgICBLZXk6IHRhcmdldEtleSxcbiAgICAgICAgICBDb3B5U291cmNlOiBidWNrZXQgKyAnLycgKyBzb3VyY2VLZXksXG4gICAgICAgIH0pO1xuXG4gICAgICAgIGF3YWl0IHMzQ2xpZW50LnNlbmQoY29weUNvbW1hbmQpO1xuXG4gICAgICAgIGNvbnN0IGRlbGV0ZVNvdXJjZUNvbW1hbmQgPSBuZXcgRGVsZXRlT2JqZWN0Q29tbWFuZCh7XG4gICAgICAgICAgQnVja2V0OiBidWNrZXQsXG4gICAgICAgICAgS2V5OiBzb3VyY2VLZXksXG4gICAgICAgIH0pO1xuXG4gICAgICAgIGF3YWl0IHMzQ2xpZW50LnNlbmQoZGVsZXRlU291cmNlQ29tbWFuZCk7XG4gICAgICB9XG4gICAgfSk7XG5cbiAgICBjb25zdCBwcm9jZXNzZWQgPSBhd2FpdCBQcm9taXNlLmFsbChwZW5kaW5nTW92ZXMpO1xuICAgIGNvbnNvbGUubG9nKGBzdWNjZXNzZnVsbHkgbW92ZWQgJHtwcm9jZXNzZWQubGVuZ3RofSBmaWxlc2ApO1xuICB9IGNhdGNoIChlcnJvcikge1xuICAgIGNvbnNvbGUuZXJyb3IoJyMjIyB1bmV4cGVjdGVkIHJ1bnRpbWUgZXJyb3IgIyMjJywgZXJyb3IpO1xuICB9XG59O1xuXG5leHBvcnQgY29uc3QgaGFuZGxlciA9IGludGVybmFsSGFuZGxlcjtcbiJdfQ==

package/lib/functions/access-logs-analysis/group-by-date/index.ts

@@ -0,0 +1,64 @@
+/**
+ * This script moves access logs into a target folder hierarchy, structured by year, month, day, and hour (UTC).
+ * That way, the logs can easier be processed with Athena.
+ * Taken and adjusted from https://github.com/aws-samples/amazon-cloudfront-access-logs-queries.
+ * This Lambda supports access logs from both: CloudFront and S3.
+ */
+import {S3Client, CopyObjectCommand, DeleteObjectCommand} from '@aws-sdk/client-s3';
+import type {S3Event} from 'aws-lambda';
+
+// without leading and trailing slashes
+const targetFolder = process.env.TARGET_FOLDER;
+if (!targetFolder) {
+  throw new Error('Required environment variable TARGET_FOLDER missing!');
+}
+if (!process.env.RAW_ACCESS_LOG_FILE_PATTERN) {
+  throw new Error('Required environment variable RAW_ACCESS_LOG_FILE_PATTERN missing!');
+}
+// unfortunately, CloudFront logs and S3 logs are completely different. That's why the pattern to identify
+// unprocessed log files need to be configurable
+const rawAccessLogFilePattern = new RegExp(process.env.RAW_ACCESS_LOG_FILE_PATTERN);
+
+// matches for everything after the last slash
+const filenamePattern = /[^/]+$/;
+
+const s3Client = new S3Client({region: process.env.AWS_REGION});
+
+const internalHandler = async (event: S3Event) => {
+  try {
+    const pendingMoves = event.Records.map(async record => {
+      const bucket = record.s3.bucket.name;
+      const sourceKey = record.s3.object.key;
+      const match = rawAccessLogFilePattern.exec(sourceKey);
+
+      // skip other files/folder
+      if (match?.groups) {
+        const {year, month, day, hour} = match.groups;
+        const filename = filenamePattern.exec(sourceKey)![0];
+        const targetKey = `${targetFolder}/year=${year}/month=${month}/day=${day}/hour=${hour}/${filename}`;
+
+        const copyCommand = new CopyObjectCommand({
+          Bucket: bucket,
+          Key: targetKey,
+          CopySource: bucket + '/' + sourceKey,
+        });
+
+        await s3Client.send(copyCommand);
+
+        const deleteSourceCommand = new DeleteObjectCommand({
+          Bucket: bucket,
+          Key: sourceKey,
+        });
+
+        await s3Client.send(deleteSourceCommand);
+      }
+    });
+
+    const processed = await Promise.all(pendingMoves);
+    console.log(`successfully moved ${processed.length} files`);
+  } catch (error) {
+    console.error('### unexpected runtime error ###', error);
+  }
+};
+
+export const handler = internalHandler;

package/lib/functions/access-logs-analysis/group-by-date/package.json

@@ -0,0 +1,20 @@
+{
+  "name": "cdk-nuxt-access-log-analysis-group-by-date",
+  "version": "1.0.0",
+  "main": "index.js",
+  "license": "UNLICENSED",
+  "private": true,
+  "scripts": {
+    "install-layer": "yarn install --frozen-lockfile --production --modules-folder build/layer/nodejs/node_modules",
+    "build": "tsc --project ./tsconfig.json"
+  },
+  "devDependencies": {
+    "@types/node": "^16.11.0",
+    "@types/aws-lambda": "^8.10.101",
+    "ts-node": "^10.9.1",
+    "typescript": "^4.7.4"
+  },
+  "dependencies": {
+    "@aws-sdk/client-s3": "^3.131.0"
+  }
+}

package/lib/functions/access-logs-analysis/group-by-date/tsconfig.json

@@ -0,0 +1,33 @@
+{
+  "compilerOptions": {
+    "outDir": "./build/app",
+    "target": "ES2021",
+    "module": "CommonJS",
+    "lib": ["es2021"],
+    "declaration": true,
+    "alwaysStrict": true,
+    "sourceMap": true,
+    "inlineSources": true,
+    "inlineSourceMap": false,
+    "sourceRoot": "/",
+    "noEmitHelpers": true,
+    "importHelpers": true,
+    "noImplicitAny": false,
+    "noUnusedLocals": false,
+    "noUnusedParameters": false,
+    "noImplicitReturns": true,
+    "noFallthroughCasesInSwitch": false,
+    "emitDecoratorMetadata": true,
+    "experimentalDecorators": true,
+    "strictPropertyInitialization": false,
+    "moduleResolution": "node",
+    "baseUrl": "./",
+    "paths": {
+      "*": ["./node_modules/*"]
+    },
+    "typeRoots": ["./node_modules/@types"],
+    "skipLibCheck": true
+  },
+  "include": ["./"],
+  "exclude": ["./build", "./node_modules"]
+}