@63klabs/cache-data 1.3.5 → 1.3.7

package/CHANGELOG.md

@@ -2,11 +2,58 @@
 
 All notable changes to this project will be documented in this file.
 
-Proposed and upcoming changes may be found on [63Klabs/cache-data Issues](https://github.com/63Klabs/cache-data/issues).
+> **Note:** This project is still in beta. While every effort is made to prevent breaking changes, they may still occur. If after upgrading to a new version you experience any issue, please report the issue and go back to a previous version until a fix is released.
+
+To report an issue, or to see proposed and upcoming enhancements, check out [63Klabs/cache-data Issues](https://github.com/63Klabs/cache-data/issues) page on GitHub.
 
 Report all vulnerabilities under the [Security menu](https://github.com/63Klabs/cache-data/security/advisories) in the Cache-Data GitHub repository.
 
-> Note: This project is still in beta. Even though changes are tested and breaking changes are avoided, things may break.
+## v1.3.8 (unreleased)
+
+- Nothing yet
+
+## v1.3.7 (2026-02-06)
+
+### Fixed
+- **Cache DAO Undefined Header Bug** [Spec: 1-3-7-cache-dao-fix](.kiro/specs/1-3-7-cache-dao-fix/) - Fixed production bug where undefined values were passed to HTTP headers, causing request failures with "Invalid value 'undefined' for header" errors
+  - Cache.getHeader() now normalizes undefined to null for consistent behavior
+  - Added defensive validation at header assignment points in CacheableDataAccess.getData()
+  - Added Cache._isValidHeaderValue() helper method for header validation
+  - **Most likely cause:** The move from over-use of JSON Stringify/parse cycles in favor of cloning in v1.3.6. JSON stringify/parse removed undefined values from objects, covering an underlying issue that is now fixed.
+
+### Added
+- **Jest Testing Framework** [Spec: 1-3-7-cache-dao-fix](.kiro/specs/1-3-7-cache-dao-fix/) - Set up Jest alongside Mocha for better AWS integration testing
+  - Configured Jest with ES module support
+  - Added npm scripts: `test:jest`, `test:all`, `test:cache:jest`
+  - Jest tests use `*.jest.mjs` pattern to avoid conflicts with Mocha tests
+
+## v1.3.6 (2025-02-02)
+
+### Enhancement
+
+- Added in-memory cache option for `Cache`. Even though concurrent functions will not be able to make use of another function's in memory cache, it will improve response time and lessen calls to DynamoDb.
+- Documentation has been enhanced and expanded
+- Added override for `"fast-xml-parser": ">=5.3.4"` to ensure no vulnerabilities during `npm install`.
+- Reduced internal use of JSON.parse() and JSON.stringify() to improve speed (1-2x improvement)
+- New `Config.getConnCacheProfile("myConnection", "myProfile")` method to get both `conn` and `cacheProfile` in one line rather than 3
+- New `Config.getConn("myConnection")` to get the connection object directly in one line rather than 2
+
+```js
+// OLD: Config.getConnection('name') object in 2 lines:
+const connection = Config.getConnection('myConnection');
+const conn = connection.toObject();
+
+// NEW: Config.getConn('name') in 1 line:
+const conn = Config.getConn('myConnection');
+
+// OLD: Get connection object and cache profile in 3 lines:
+const connection = Config.getConnection('myConnection');
+const conn = connection.toObject();
+const cacheProfile = connection.getCacheProfile();
+
+// NEW: Get connection object and cache profile in 1 line:
+const { conn, cacheProfile } = Config.getConnCacheProfile('myConnection', 'myProfile');
+```
 
 ## v1.3.5 (2025-01-13)
 

package/CONTRIBUTING.md

@@ -0,0 +1,167 @@
+# Contributing
+
+Contributions to this project are welcome! Whether you're fixing a bug, adding a new feature, or improving documentation, your help is appreciated.
+
+Building a reputation will require a multi-step approach.
+
+## Steps to Contributing
+
+Begin by submitting bugs to ensure the project works for everyone. If you can submit code suggestions or pseudo code that helps! You can also examine a bug reported by someone else and provide helpful solutions to the team.
+
+Submit feature requests. To keep this project simple and maintainable we accept features that are generally useful by an overwhelming majority of developers using the project. If you can submit code suggestions or pseudo code that helps! You can also examine a feature request from someone else and provide helpful solutions to the team.
+
+After you have successfully participated in the bug reporting and feature request process, fork the repository and make your changes in a separate branch. Once you're satisfied with your changes, submit a pull request for review. Please only submit small changes (a single feature) at first. Pull requests with major code updates or frequent pull requests will often get ignored. Changes should also have code and testing methods well documented.
+
+All code changes MUST start as an Issue (or security report) with a clear description of the problem or enhancement. No changes should be submitted to the repository without an attached, and approved, Issue.
+
+Code developed (by AI or Human) outside of Kiro (see below) must NOT be submitted directly to the repository. Instead submit a proof of concept for a new piece of code or method via the Issue tracker as an enhancement. Someone from the team will review, evaluate the usefulness, and then implement using the proper process.
+
+## Use of AI
+
+This project utilizes the Spec-Driven, AI-Assisted Engineering approach.
+
+Spec-Driven, AI-Assisted Engineering (SD-AI) is a software development methodology that prioritizes creating detailed, structured specifications before writing code. It priortizes context, requirements, and architectural constraints to generate accurate, non-hallucinated code. This approach shifts from ad-hoc, prompt-driven "vibe coding" to a structured, human-guided, AI-executed workflow, improving reliability in complex projects.
+
+> Contributors are responsible for every line of code--AI-generated or not.
+
+Code must be reviewed, understood, and tested by a human before being merged.
+
+Kiro is the required AI coding assistant for final integrations, documentation, and testing, as it is in the AWS Ecosystem and this project is deveoped to deploy on the AWS platform. Just like test suites, Kiro ensures the proper tests, documentation, and guardrails are in place. Kiro is as important as commit-hooks and tests as it is a tool that ensures quality checks and should not be bypassed.
+
+Ensure [AI Context](./AI_CONTEXT.md) and [Kiro steering documents](.kiro/steering/ai-context-reference.md) are reviewed, understood, and used by both humans and AI.
+
+## Development Setup
+
+Tests and documentation are critical to this project.
+
+Do not disable tests.
+Do not change tests.
+Do not break the build.
+
+Before contributing code, set up your development environment:
+
+### Quick Setup (Recommended)
+
+Run the automated setup script:
+
+```bash
+git clone https://github.com/63klabs/cache-data.git
+cd cache-data
+./scripts/setup-dev-environment.sh
+```
+
+This script will:
+- Install npm dependencies
+- Configure the pre-commit hook
+- Run validation tests
+- Display helpful next steps
+
+### Manual Setup
+
+If you prefer to set up manually:
+
+### 1. Clone and Install Dependencies
+
+```bash
+git clone https://github.com/63klabs/cache-data.git
+cd cache-data
+npm install
+```
+
+### 2. Install Pre-Commit Hook
+
+The project includes a pre-commit hook that validates documentation quality before allowing commits. This ensures all contributions maintain documentation standards.
+
+**Install the hook:**
+
+```bash
+# The hook is already in .git/hooks/pre-commit
+# Make sure it's executable
+chmod +x .git/hooks/pre-commit
+```
+
+**What the hook validates:**
+- ✅ JSDoc completeness (all required tags present)
+- ✅ JSDoc accuracy (parameters match function signatures)
+- ✅ Documentation links are valid
+- ✅ Code examples have valid syntax
+
+**The hook will block commits if:**
+- Incomplete JSDoc (missing @param, @returns, @example tags)
+- Inaccurate JSDoc (parameter names don't match code)
+- Broken links in documentation files
+- Invalid syntax in code examples
+
+**Note**: The hook reports "Missing JSDoc" items for informational purposes, but these don't block commits due to known parser limitations. Only the issues listed above will prevent commits.
+
+**Test the hook:**
+
+```bash
+# Run the validation manually
+.git/hooks/pre-commit
+
+# Expected output:
+# ✅ All validation checks passed
+# ✅ Documentation validation passed!
+```
+
+**If validation fails:**
+1. Review the error messages - they indicate specific files and issues
+2. Fix the documentation issues identified
+3. Run the hook again to verify fixes
+4. Only use `git commit --no-verify` if you're certain the errors are false positives
+
+### 3. Run Tests
+
+Ensure all tests pass before submitting changes:
+
+```bash
+# Run all tests
+npm test
+
+# Run specific test suites
+npm test -- test/documentation/
+npm test -- test/cache/
+```
+
+## Documentation Standards
+
+All public APIs must have complete JSDoc documentation. See [Documentation Standards](.kiro/steering/documentation-standards.md) for detailed requirements.
+
+**Required JSDoc tags:**
+- Description of what the function/class does
+- `@param` for each parameter with type and description
+- `@returns` with type and description (omit for void functions)
+- `@example` with at least one working code example
+- `@throws` for each error type that can be thrown
+
+**Example:**
+
+```javascript
+/**
+ * Retrieves cached data or fetches from source if not cached
+ * 
+ * @param {object} cacheProfile - Cache configuration profile
+ * @param {Function} fetchFunction - Function to fetch data if not cached
+ * @param {object} connection - Connection configuration
+ * @returns {Promise<{success: boolean, data: object, cached: boolean}>} Result object with data and cache status
+ * @throws {Error} If cache profile is invalid
+ * @example
+ * const result = await CacheableDataAccess.getData(
+ *   cacheProfile,
+ *   endpoint.get,
+ *   connection
+ * );
+ * console.log(result.data);
+ */
+```
+
+## Current Contributors
+
+Thank you to the following people who have contributed to this project:
+
+Chad Kluck\
+DevOps & Developer Experience Engineer\
+AWS Certified Cloud Practitioner | AWS Certified Developer - Associate | AWS
+Certified Solutions Architect - Associate\
+[Website](https://chadkluck.me)

package/README.md

@@ -5,7 +5,7 @@ A package for AWS Lambda Node.js applications to access and cache data from remo
 > Note: This repository and package has moved from chadkluck to 63Klabs but is still managed by the same developer.
 
 - [@63klabs/cache-data on npmjs.com](https://www.npmjs.com/package/@63klabs/cache-data)
-- [@63klabs/cache-data on GitHub](https://github.com/@63klabs/cache-data)
+- [@63klabs/cache-data on GitHub](https://github.com/63Klabs/cache-data)
 
 ## Description
 
@@ -17,19 +17,55 @@ It can be used in place of Express.js for simple web service applications as it
 
 This package has been used in production environments for web service applications receiving over 1 million requests per week with a 75% cache-hit rate lowering latency to less than 100ms in most cases. This is a considerable improvement when faced with resource intense processes, connection pools, API rate limits, and slow endpoints.
 
+## Features
+
+The @63klabs/cache-data package provides three main modules:
+
+### Cache Module
+- **Distributed Caching**: Share cached data across concurrent Lambda executions using DynamoDb and S3
+- **In-Memory Cache**: Optional in-memory caching layer for improved response times within a single execution
+- **Flexible Storage**: Automatic storage selection based on data size (DynamoDb for small items, S3 for large items)
+- **Data Encryption**: Secure your cached data with encryption keys stored in SSM Parameter Store
+- **Cache Profiles**: Configure multiple cache profiles with different expiration policies
+
+### Endpoint Module
+- **HTTP/HTTPS Requests**: Make requests to external APIs and endpoints with built-in retry logic
+- **Connection Management**: Define and manage multiple endpoint connections with authentication
+- **Request Caching**: Automatically cache endpoint responses to reduce API calls and improve performance
+- **Flexible Configuration**: Support for custom headers, query parameters, request bodies, and timeouts
+
+### Tools Module
+- **Logging and Debugging**: `DebugAndLog` class for structured logging with configurable log levels
+- **Performance Timing**: `Timer` class for measuring execution time and performance metrics
+- **Request Handling**: `ClientRequest` and `Response` classes for building web service applications
+- **AWS Integration**: Direct access to AWS SDK v3 for DynamoDb, S3, and SSM Parameter Store
+- **Parameter and Secret Caching**: `CachedParameterSecrets` classes for AWS Parameters and Secrets Lambda Extension
+- **Utility Functions**: Data sanitization, obfuscation, hashing, and immutable object creation
+- **Response Generators**: Built-in generators for JSON, HTML, XML, RSS, and text responses
+
 ## Getting Started
 
 ### Requirements
 
-- Node >22 runtime on Lambda
-- AWS Lambda, S3 bucket, DynamoDb table, and SSM Parameter Store
-- A basic understanding of CloudFormation, Lambda, S3, DynamoDb, and SSM Parameters
-- A basic understanding of IAM policies, especially the Lambda Execution Role, that will allow Lambda to access S3, DynamoDb, and SSM Parameter Store
-- Lambda function should have between 512MB and 2048MB of memory allocated. (>1024MB recommended). See [Lambda Optimization: Memory Allocation](./docs/lambda-optimization/README.md#lambda-memory-allocation).
+- Node.js >=20.0.0 runtime on Lambda
+- AWS Services:
+  - **AWS Lambda**: For running your serverless functions
+  - **Amazon S3**: For storing large cached objects
+  - **Amazon DynamoDB**: For storing cache metadata and small cached objects
+  - **AWS Systems Manager (SSM) Parameter Store**: For storing configuration and encryption keys
+- A basic understanding of CloudFormation, Lambda, S3, DynamoDB, and SSM Parameters
+- A basic understanding of IAM policies, especially the Lambda Execution Role, that will allow Lambda to access S3, DynamoDB, and SSM Parameter Store
+- Lambda function should have between 512MB and 2048MB of memory allocated (>1024MB recommended). See [Lambda Optimization: Memory Allocation](./docs/lambda-optimization/README.md#lambda-memory-allocation)
 
 ### Installing
 
-The simplest way to get started is to use the [63klabs Atlantis Templates and Script platform](hhttps://github.com/63Klabs/atlantis-cfn-configuration-repo-for-serverless-deployments) to deploy this and other ready-to-run solutions via CI/CD.
+Install the package using npm:
+
+```bash
+npm install @63klabs/cache-data
+```
+
+The simplest way to get started is to use the [63klabs Atlantis Templates and Script platform](https://github.com/63Klabs/atlantis-cfn-configuration-repo-for-serverless-deployments) to deploy this and other ready-to-run solutions via CI/CD.
 
 However, if you want to write your own templates and code, follow the following steps:
 
@@ -37,15 +73,17 @@ However, if you want to write your own templates and code, follow the following
    - Use the [key generation script](./docs/00-example-implementation/generate-put-ssm.py) during [the build](./docs/00-example-implementation/example-buildspec.yml) to establish a key to encrypt your data.
 2. Lambda CloudFormation Template:
    - See [Lambda template example](./docs/00-example-implementation/example-template-lambda-function.yml) 
-   - Node: AWS Lambda supported version of Node
-   - Memory: Allocate at least 256MB (512-1024MB recommended)
+   - Node: AWS Lambda supported version of Node (>=20.0.0)
+   - Memory: Allocate at least 512MB (1024MB+ recommended)
    - Environment Variables: Add the cache-data environment variables to your Lambda function.
-   - Execution Role: Include access to S3 and DynamoDb in your Lambda's execution role.
-3. S3 and DynamoDb CloudFormation Template to store your cache:
-   - See [S3 and DynamoDb Cache Store template example](./docs/00-example-implementation/example-template-s3-and-dynamodb-cache-store.yml)
+   - Execution Role: Include access to S3 and DynamoDB in your Lambda's execution role.
+3. S3 and DynamoDB CloudFormation Template to store your cache:
+   - See [S3 and DynamoDB Cache Store template example](./docs/00-example-implementation/example-template-s3-and-dynamodb-cache-store.yml)
    - Include in your application infrastructure template or as separate infrastructure.
 4. Install the @63klabs/cache-data package:
-   - `npm install @63klabs/cache-data`
+   ```bash
+   npm install @63klabs/cache-data
+   ```
 5. Add code to your Lambda function to utilize caching and other cache-data utilities:
    - See [example code for index and handler](./docs/00-example-implementation/example-handler.js)
    - See [example code for config initialization](./docs/00-example-implementation/example-config.js)
@@ -56,6 +94,68 @@ It is recommended that you use the quick-start method when implementing for the
 - [Advanced Implementation for Providing a Web Service](./docs/01-advanced-implementation-for-web-service/README.md)
 - [Additional Documentation](./docs/README.md)
 
+## Quick Start Examples
+
+### Basic Caching Example
+
+```javascript
+const { cache } = require("@63klabs/cache-data");
+
+// Initialize cache with your S3 bucket and DynamoDB table
+cache.Cache.init({
+  s3Bucket: process.env.CACHE_DATA_S3_BUCKET,
+  dynamoDbTable: process.env.CACHE_DATA_DYNAMODB_TABLE,
+  securityKey: process.env.CACHE_DATA_SECURITY_KEY
+});
+
+// Cache some data
+const cacheKey = "my-data-key";
+const dataToCache = { message: "Hello, World!", timestamp: Date.now() };
+
+await cache.Cache.put(cacheKey, dataToCache, 3600); // Cache for 1 hour
+
+// Retrieve cached data
+const cachedData = await cache.Cache.get(cacheKey);
+if (cachedData) {
+  console.log("Retrieved from cache:", cachedData);
+}
+```
+
+### Making Endpoint Requests
+
+```javascript
+const { endpoint } = require("@63klabs/cache-data");
+
+// Make a simple GET request to an API
+const response = await endpoint.get(
+  { host: "api.example.com", path: "/data" },
+  { parameters: { q: "search-term" } }
+);
+
+console.log("API Response:", response.body);
+console.log("Status Code:", response.statusCode);
+```
+
+### Using Utility Tools
+
+```javascript
+const { tools } = require("@63klabs/cache-data");
+
+// Create a timer to measure performance
+const timer = new tools.Timer("my-operation");
+timer.start();
+
+// Your code here...
+
+timer.stop();
+console.log(`Operation took ${timer.elapsed()}ms`);
+
+// Use the logger
+const logger = new tools.DebugAndLog("MyApp");
+logger.info("Application started");
+logger.error("An error occurred", { details: "error info" });
+```
+
 ## Help
 
 Make sure you have your S3 bucket, DynamoDb table, and SSM Parameter store set up. Also make sure that you have IAM policies to allow your Lambda function access to these, and CodeBuild to read and write to SSM Parameter store.
@@ -64,30 +164,42 @@ Review the [Documentation](./docs/README.md) which includes implementation guide
 
 A full implementation example and tutorial is provided as one of the Atlantis Application Starters available through the [Atlantis Tutorials repository](https://github.com/63klabs/atlantis-tutorials). (Atlantis is a collection of templates and deployment scripts to assist in starting and automating serverless deployments using AWS SAM and CloudFormation.)
 
-## Security
+## Tutorials
+
+Read the [Atlantis Tutorials introductory page](https://github.com/63Klabs/atlantis-tutorials) for overall usage of Atlantis Platform Templates and Scripts.
+
+## License
 
-See [SECURITY](./SECURITY.md) for information on reporting concerns.
+This project is licensed under the MIT License - see the [LICENSE](LICENSE.txt) file for details.
 
 ## Change Log
 
-See [Change Log](CHANGELOG.md) for version history and changes.
+See [Change Log](./CHANGELOG.md) for version history and changes.
 
 ## Issues, Features, and Enhancements
 
 Visit the [Issues section of the @63Klabs Cache-Data GitHub repository](https://github.com/63Klabs/cache-data/issues) for information on reported issues, upcoming fixes and enhancements, and to submit requests.
 
-## License
+## Security
+
+If you discover any security related issues, please see the [SECURITY](SECURITY.md) file for details.
+
+## Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](./CONTRIBUTING.md) for details.
+
+**For Contributors**: After cloning the repository, run the setup script to configure your development environment:
+
+```bash
+./scripts/setup-dev-environment.sh
+```
+
+This will install dependencies, configure the pre-commit hook for documentation validation, and run tests to ensure everything is working correctly.
 
-This project is licensed under the MIT License - see the LICENSE.txt file for details
+## AI Context
 
-## Author
+See [AI_CONTEXT.md](AI_CONTEXT.md) for important context and guidelines for AI-generated code in this repository.
 
-### Chad Kluck
+The context file is also helpful (and perhaps essential) for HUMANS developing within the application's structured platform as well.
 
-- Software, DevOps, and Developer Experience Engineer
-- [AWS Certified Developer - Associate](https://www.credly.com/users/chad-kluck/badges)
-- [Website: chadkluck.me](https://chadkluck.me/)
-- [GitHub: chadkluck](https://github.com/chadkluck)
-- [GitHub: 63Klabs](https://github.com/63klabs)
-- [Mastodon: @chadkluck@universeodon.com](https://universeodon.com/@chadkluck)
-- [LinkedIn](https://www.linkedin.com/in/chadkluck/)
+AI Assisted Engineering of this solution was provided by [Kiro](https://kiro.dev/). Steering documents are provided in the repository's [.kiro](.kiro/steering/ai-context-reference.md) directory. Because testing is tightly coupled with the implementation, it is suggested all documents, code, and tests are thoroughly reviewed before, and updated after, any changes.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@63klabs/cache-data",
-  "version": "1.3.5",
+  "version": "1.3.7",
   "description": "Cache data from an API endpoint or application process using AWS S3 and DynamoDb",
   "author": "Chad Leigh Kluck (https://chadkluck.me)",
   "license": "MIT",
@@ -25,14 +25,22 @@
     "@aws-sdk/client-s3": "3.x",
     "@aws-sdk/client-ssm": "3.x",
     "@aws-sdk/lib-dynamodb": "3.x",
-    "chai": "^6.0.1",
-    "chai-http": "^5.1.2",
-    "mocha": "^11.7.5",
-    "sinon": "^21.0.1"
+    "chai": "^6.x",
+    "chai-http": "^5.x",
+    "fast-check": "^4.x",
+    "jest": "^30.2.0",
+    "mocha": "^11.x",
+    "sinon": "^21.x"
+  },
+  "overrides": {
+    "fast-xml-parser": ">=5.3.4"
   },
   "scripts": {
     "test": "mocha 'test/**/*-tests.mjs'",
+    "test:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
+    "test:all": "npm test && npm run test:jest",
     "test:cache": "mocha 'test/cache/**/*-tests.mjs'",
+    "test:cache:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js test/cache",
     "test:config": "mocha 'test/config/**/*-tests.mjs'",
     "test:endpoint": "mocha 'test/endpoint/**/*-tests.mjs'",
     "test:logging": "mocha 'test/logging/**/*-tests.mjs'",