@63klabs/cache-data 1.3.4 → 1.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AI_CONTEXT.md +863 -0
- package/CHANGELOG.md +44 -1
- package/CONTRIBUTING.md +161 -0
- package/README.md +139 -27
- package/package.json +9 -5
- package/scripts/README.md +175 -0
- package/scripts/audit-documentation.mjs +856 -0
- package/scripts/review-documentation-files.mjs +406 -0
- package/scripts/setup-dev-environment.sh +59 -0
- package/scripts/verify-example-files.mjs +194 -0
- package/src/lib/dao-cache.js +1286 -288
- package/src/lib/dao-endpoint.js +181 -42
- package/src/lib/tools/AWS.classes.js +82 -0
- package/src/lib/tools/CachedParametersSecrets.classes.js +98 -7
- package/src/lib/tools/ClientRequest.class.js +43 -10
- package/src/lib/tools/Connections.classes.js +148 -13
- package/src/lib/tools/DebugAndLog.class.js +244 -75
- package/src/lib/tools/ImmutableObject.class.js +44 -2
- package/src/lib/tools/RequestInfo.class.js +38 -0
- package/src/lib/tools/Response.class.js +245 -81
- package/src/lib/tools/ResponseDataModel.class.js +123 -47
- package/src/lib/tools/Timer.class.js +138 -26
- package/src/lib/tools/index.js +89 -2
- package/src/lib/tools/utils.js +40 -4
- package/src/lib/utils/InMemoryCache.js +221 -0
package/CHANGELOG.md
CHANGED
|
@@ -6,7 +6,50 @@ Proposed and upcoming changes may be found on [63Klabs/cache-data Issues](https:
|
|
|
6
6
|
|
|
7
7
|
Report all vulnerabilities under the [Security menu](https://github.com/63Klabs/cache-data/security/advisories) in the Cache-Data GitHub repository.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
## v1.3.7 (unreleased)
|
|
10
|
+
|
|
11
|
+
- Next release
|
|
12
|
+
|
|
13
|
+
## v1.3.6 (2025-02-02)
|
|
14
|
+
|
|
15
|
+
### Enhancement
|
|
16
|
+
|
|
17
|
+
- Added in-memory cache option for `Cache`. Even though concurrent functions will not be able to make use of another function's in memory cache, it will improve response time and lessen calls to DynamoDb.
|
|
18
|
+
- Documentation has been enhanced and expanded
|
|
19
|
+
- Added override for `"fast-xml-parser": ">=5.3.4"` to ensure no vulnerabilities during `npm install`.
|
|
20
|
+
- Reduced internal use of JSON.parse() and JSON.stringify() to improve speed (1-2x improvement)
|
|
21
|
+
- New `Config.getConnCacheProfile("myConnection", "myProfile")` method to get both `conn` and `cacheProfile` in one line rather than 3
|
|
22
|
+
- New `Config.getConn("myConnection")` to get the connection object directly in one line rather than 2
|
|
23
|
+
|
|
24
|
+
```js
|
|
25
|
+
// OLD: Config.getConnection('name') object in 2 lines:
|
|
26
|
+
const connection = Config.getConnection('myConnection');
|
|
27
|
+
const conn = connection.toObject();
|
|
28
|
+
|
|
29
|
+
// NEW: Config.getConn('name') in 1 line:
|
|
30
|
+
const conn = Config.getConn('myConnection');
|
|
31
|
+
|
|
32
|
+
// OLD: Get connection object and cache profile in 3 lines:
|
|
33
|
+
const connection = Config.getConnection('myConnection');
|
|
34
|
+
const conn = connection.toObject();
|
|
35
|
+
const cacheProfile = connection.getCacheProfile();
|
|
36
|
+
|
|
37
|
+
// NEW: Get connection object and cache profile in 1 line:
|
|
38
|
+
const { conn, cacheProfile } = Config.getConnCacheProfile('myConnection', 'myProfile');
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## v1.3.5 (2025-01-13)
|
|
42
|
+
|
|
43
|
+
### Enhancements
|
|
44
|
+
|
|
45
|
+
- `endpoint.getDataDirectFromURI()` has been renamed to `endpoint.get()` and the old name is now a deprecated alias. This was just to simplify naming.
|
|
46
|
+
|
|
47
|
+
Example Use:
|
|
48
|
+
|
|
49
|
+
```javascript
|
|
50
|
+
const { endpoint } = require("@63klabs/cache-data");
|
|
51
|
+
const data = await endpoint.get({host: "api.example.com", path: "data"}, { parameters: {q: "Chicago" }});
|
|
52
|
+
```
|
|
10
53
|
|
|
11
54
|
## v1.3.4 (2025-01-12)
|
|
12
55
|
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Contributions to this project are welcome! Whether you're fixing a bug, adding a new feature, or improving documentation, your help is appreciated.
|
|
4
|
+
|
|
5
|
+
Building a reputation will require a multi-step approach.
|
|
6
|
+
|
|
7
|
+
## Steps to Contributing
|
|
8
|
+
|
|
9
|
+
Begin by submitting bugs to ensure the project works for everyone. If you can submit code suggestions or pseudo code that helps! You can also examine a bug reported by someone else and provide helpful solutions to the team.
|
|
10
|
+
|
|
11
|
+
Submit feature requests. To keep this project simple and maintainable we accept features that are generally useful by an overwhelming majority of developers using the project. If you can submit code suggestions or pseudo code that helps! You can also examine a feature request from someone else and provide helpful solutions to the team.
|
|
12
|
+
|
|
13
|
+
After you have successfully participated in the bug reporting and feature request process, fork the repository and make your changes in a separate branch. Once you're satisfied with your changes, submit a pull request for review. Please only submit small changes (a single feature) at first. Pull requests with major code updates or frequent pull requests will often get ignored. Changes should also have code and testing methods well documented.
|
|
14
|
+
|
|
15
|
+
## Development Setup
|
|
16
|
+
|
|
17
|
+
Tests and documentation are critical to this project.
|
|
18
|
+
|
|
19
|
+
Do not disable tests.
|
|
20
|
+
Do not change tests.
|
|
21
|
+
Do not break the build.
|
|
22
|
+
|
|
23
|
+
Before contributing code, set up your development environment:
|
|
24
|
+
|
|
25
|
+
### Quick Setup (Recommended)
|
|
26
|
+
|
|
27
|
+
Run the automated setup script:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
git clone https://github.com/63klabs/cache-data.git
|
|
31
|
+
cd cache-data
|
|
32
|
+
./scripts/setup-dev-environment.sh
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
This script will:
|
|
36
|
+
- Install npm dependencies
|
|
37
|
+
- Configure the pre-commit hook
|
|
38
|
+
- Run validation tests
|
|
39
|
+
- Display helpful next steps
|
|
40
|
+
|
|
41
|
+
### Manual Setup
|
|
42
|
+
|
|
43
|
+
If you prefer to set up manually:
|
|
44
|
+
|
|
45
|
+
### 1. Clone and Install Dependencies
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
git clone https://github.com/63klabs/cache-data.git
|
|
49
|
+
cd cache-data
|
|
50
|
+
npm install
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### 2. Install Pre-Commit Hook
|
|
54
|
+
|
|
55
|
+
The project includes a pre-commit hook that validates documentation quality before allowing commits. This ensures all contributions maintain documentation standards.
|
|
56
|
+
|
|
57
|
+
**Install the hook:**
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
# The hook is already in .git/hooks/pre-commit
|
|
61
|
+
# Make sure it's executable
|
|
62
|
+
chmod +x .git/hooks/pre-commit
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
**What the hook validates:**
|
|
66
|
+
- ✅ JSDoc completeness (all required tags present)
|
|
67
|
+
- ✅ JSDoc accuracy (parameters match function signatures)
|
|
68
|
+
- ✅ Documentation links are valid
|
|
69
|
+
- ✅ Code examples have valid syntax
|
|
70
|
+
|
|
71
|
+
**The hook will block commits if:**
|
|
72
|
+
- Incomplete JSDoc (missing @param, @returns, @example tags)
|
|
73
|
+
- Inaccurate JSDoc (parameter names don't match code)
|
|
74
|
+
- Broken links in documentation files
|
|
75
|
+
- Invalid syntax in code examples
|
|
76
|
+
|
|
77
|
+
**Note**: The hook reports "Missing JSDoc" items for informational purposes, but these don't block commits due to known parser limitations. Only the issues listed above will prevent commits.
|
|
78
|
+
|
|
79
|
+
**Test the hook:**
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
# Run the validation manually
|
|
83
|
+
.git/hooks/pre-commit
|
|
84
|
+
|
|
85
|
+
# Expected output:
|
|
86
|
+
# ✅ All validation checks passed
|
|
87
|
+
# ✅ Documentation validation passed!
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**If validation fails:**
|
|
91
|
+
1. Review the error messages - they indicate specific files and issues
|
|
92
|
+
2. Fix the documentation issues identified
|
|
93
|
+
3. Run the hook again to verify fixes
|
|
94
|
+
4. Only use `git commit --no-verify` if you're certain the errors are false positives
|
|
95
|
+
|
|
96
|
+
### 3. Run Tests
|
|
97
|
+
|
|
98
|
+
Ensure all tests pass before submitting changes:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
# Run all tests
|
|
102
|
+
npm test
|
|
103
|
+
|
|
104
|
+
# Run specific test suites
|
|
105
|
+
npm test -- test/documentation/
|
|
106
|
+
npm test -- test/cache/
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Documentation Standards
|
|
110
|
+
|
|
111
|
+
All public APIs must have complete JSDoc documentation. See [Documentation Standards](.kiro/steering/documentation-standards.md) for detailed requirements.
|
|
112
|
+
|
|
113
|
+
**Required JSDoc tags:**
|
|
114
|
+
- Description of what the function/class does
|
|
115
|
+
- `@param` for each parameter with type and description
|
|
116
|
+
- `@returns` with type and description (omit for void functions)
|
|
117
|
+
- `@example` with at least one working code example
|
|
118
|
+
- `@throws` for each error type that can be thrown
|
|
119
|
+
|
|
120
|
+
**Example:**
|
|
121
|
+
|
|
122
|
+
```javascript
|
|
123
|
+
/**
|
|
124
|
+
* Retrieves cached data or fetches from source if not cached
|
|
125
|
+
*
|
|
126
|
+
* @param {object} cacheProfile - Cache configuration profile
|
|
127
|
+
* @param {Function} fetchFunction - Function to fetch data if not cached
|
|
128
|
+
* @param {object} connection - Connection configuration
|
|
129
|
+
* @returns {Promise<{success: boolean, data: object, cached: boolean}>} Result object with data and cache status
|
|
130
|
+
* @throws {Error} If cache profile is invalid
|
|
131
|
+
* @example
|
|
132
|
+
* const result = await CacheableDataAccess.getData(
|
|
133
|
+
* cacheProfile,
|
|
134
|
+
* endpoint.get,
|
|
135
|
+
* connection
|
|
136
|
+
* );
|
|
137
|
+
* console.log(result.data);
|
|
138
|
+
*/
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Use of AI
|
|
142
|
+
|
|
143
|
+
This project utilizes the Spec-Driven, AI-Assisted Engineering approach.
|
|
144
|
+
|
|
145
|
+
Spec-Driven, AI-Assisted Engineering (SD-AI) is a software development methodology that prioritizes creating detailed, structured specifications before writing code. It priortizes context, requirements, and architectural constraints to generate accurate, non-hallucinated code. This approach shifts from ad-hoc, prompt-driven "vibe coding" to a structured, human-guided, AI-executed workflow, improving reliability in complex projects.
|
|
146
|
+
|
|
147
|
+
> Contributors are responsible for every line of code--AI-generated or not.
|
|
148
|
+
|
|
149
|
+
Code must be reviewed, understood, and tested by a human before being merged.
|
|
150
|
+
|
|
151
|
+
Kiro is the preferred AI coding assistant as it is in the AWS Ecosystem and this project is deveoped to deploy on the AWS platform. Ensure [AI Context](./AI_CONTEXT.md) and [Kiro steering documents](.kiro/steering/ai-context-reference.md) are reviewed, understood, and used by both humans and AI.
|
|
152
|
+
|
|
153
|
+
## Current Contributors
|
|
154
|
+
|
|
155
|
+
Thank you to the following people who have contributed to this project:
|
|
156
|
+
|
|
157
|
+
Chad Kluck\
|
|
158
|
+
DevOps & Developer Experience Engineer\
|
|
159
|
+
AWS Certified Cloud Practitioner | AWS Certified Developer - Associate | AWS
|
|
160
|
+
Certified Solutions Architect - Associate\
|
|
161
|
+
[Website](https://chadkluck.me)
|
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@ A package for AWS Lambda Node.js applications to access and cache data from remo
|
|
|
5
5
|
> Note: This repository and package has moved from chadkluck to 63Klabs but is still managed by the same developer.
|
|
6
6
|
|
|
7
7
|
- [@63klabs/cache-data on npmjs.com](https://www.npmjs.com/package/@63klabs/cache-data)
|
|
8
|
-
- [@63klabs/cache-data on GitHub](https://github.com
|
|
8
|
+
- [@63klabs/cache-data on GitHub](https://github.com/63Klabs/cache-data)
|
|
9
9
|
|
|
10
10
|
## Description
|
|
11
11
|
|
|
@@ -17,19 +17,55 @@ It can be used in place of Express.js for simple web service applications as it
|
|
|
17
17
|
|
|
18
18
|
This package has been used in production environments for web service applications receiving over 1 million requests per week with a 75% cache-hit rate lowering latency to less than 100ms in most cases. This is a considerable improvement when faced with resource intense processes, connection pools, API rate limits, and slow endpoints.
|
|
19
19
|
|
|
20
|
+
## Features
|
|
21
|
+
|
|
22
|
+
The @63klabs/cache-data package provides three main modules:
|
|
23
|
+
|
|
24
|
+
### Cache Module
|
|
25
|
+
- **Distributed Caching**: Share cached data across concurrent Lambda executions using DynamoDb and S3
|
|
26
|
+
- **In-Memory Cache**: Optional in-memory caching layer for improved response times within a single execution
|
|
27
|
+
- **Flexible Storage**: Automatic storage selection based on data size (DynamoDb for small items, S3 for large items)
|
|
28
|
+
- **Data Encryption**: Secure your cached data with encryption keys stored in SSM Parameter Store
|
|
29
|
+
- **Cache Profiles**: Configure multiple cache profiles with different expiration policies
|
|
30
|
+
|
|
31
|
+
### Endpoint Module
|
|
32
|
+
- **HTTP/HTTPS Requests**: Make requests to external APIs and endpoints with built-in retry logic
|
|
33
|
+
- **Connection Management**: Define and manage multiple endpoint connections with authentication
|
|
34
|
+
- **Request Caching**: Automatically cache endpoint responses to reduce API calls and improve performance
|
|
35
|
+
- **Flexible Configuration**: Support for custom headers, query parameters, request bodies, and timeouts
|
|
36
|
+
|
|
37
|
+
### Tools Module
|
|
38
|
+
- **Logging and Debugging**: `DebugAndLog` class for structured logging with configurable log levels
|
|
39
|
+
- **Performance Timing**: `Timer` class for measuring execution time and performance metrics
|
|
40
|
+
- **Request Handling**: `ClientRequest` and `Response` classes for building web service applications
|
|
41
|
+
- **AWS Integration**: Direct access to AWS SDK v3 for DynamoDb, S3, and SSM Parameter Store
|
|
42
|
+
- **Parameter and Secret Caching**: `CachedParameterSecrets` classes for AWS Parameters and Secrets Lambda Extension
|
|
43
|
+
- **Utility Functions**: Data sanitization, obfuscation, hashing, and immutable object creation
|
|
44
|
+
- **Response Generators**: Built-in generators for JSON, HTML, XML, RSS, and text responses
|
|
45
|
+
|
|
20
46
|
## Getting Started
|
|
21
47
|
|
|
22
48
|
### Requirements
|
|
23
49
|
|
|
24
|
-
- Node
|
|
25
|
-
- AWS
|
|
26
|
-
-
|
|
27
|
-
-
|
|
28
|
-
-
|
|
50
|
+
- Node.js >=20.0.0 runtime on Lambda
|
|
51
|
+
- AWS Services:
|
|
52
|
+
- **AWS Lambda**: For running your serverless functions
|
|
53
|
+
- **Amazon S3**: For storing large cached objects
|
|
54
|
+
- **Amazon DynamoDB**: For storing cache metadata and small cached objects
|
|
55
|
+
- **AWS Systems Manager (SSM) Parameter Store**: For storing configuration and encryption keys
|
|
56
|
+
- A basic understanding of CloudFormation, Lambda, S3, DynamoDB, and SSM Parameters
|
|
57
|
+
- A basic understanding of IAM policies, especially the Lambda Execution Role, that will allow Lambda to access S3, DynamoDB, and SSM Parameter Store
|
|
58
|
+
- Lambda function should have between 512MB and 2048MB of memory allocated (>1024MB recommended). See [Lambda Optimization: Memory Allocation](./docs/lambda-optimization/README.md#lambda-memory-allocation)
|
|
29
59
|
|
|
30
60
|
### Installing
|
|
31
61
|
|
|
32
|
-
|
|
62
|
+
Install the package using npm:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
npm install @63klabs/cache-data
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The simplest way to get started is to use the [63klabs Atlantis Templates and Script platform](https://github.com/63Klabs/atlantis-cfn-configuration-repo-for-serverless-deployments) to deploy this and other ready-to-run solutions via CI/CD.
|
|
33
69
|
|
|
34
70
|
However, if you want to write your own templates and code, follow the following steps:
|
|
35
71
|
|
|
@@ -37,15 +73,17 @@ However, if you want to write your own templates and code, follow the following
|
|
|
37
73
|
- Use the [key generation script](./docs/00-example-implementation/generate-put-ssm.py) during [the build](./docs/00-example-implementation/example-buildspec.yml) to establish a key to encrypt your data.
|
|
38
74
|
2. Lambda CloudFormation Template:
|
|
39
75
|
- See [Lambda template example](./docs/00-example-implementation/example-template-lambda-function.yml)
|
|
40
|
-
- Node: AWS Lambda supported version of Node
|
|
41
|
-
- Memory: Allocate at least
|
|
76
|
+
- Node: AWS Lambda supported version of Node (>=20.0.0)
|
|
77
|
+
- Memory: Allocate at least 512MB (1024MB+ recommended)
|
|
42
78
|
- Environment Variables: Add the cache-data environment variables to your Lambda function.
|
|
43
|
-
- Execution Role: Include access to S3 and
|
|
44
|
-
3. S3 and
|
|
45
|
-
- See [S3 and
|
|
79
|
+
- Execution Role: Include access to S3 and DynamoDB in your Lambda's execution role.
|
|
80
|
+
3. S3 and DynamoDB CloudFormation Template to store your cache:
|
|
81
|
+
- See [S3 and DynamoDB Cache Store template example](./docs/00-example-implementation/example-template-s3-and-dynamodb-cache-store.yml)
|
|
46
82
|
- Include in your application infrastructure template or as separate infrastructure.
|
|
47
83
|
4. Install the @63klabs/cache-data package:
|
|
48
|
-
|
|
84
|
+
```bash
|
|
85
|
+
npm install @63klabs/cache-data
|
|
86
|
+
```
|
|
49
87
|
5. Add code to your Lambda function to utilize caching and other cache-data utilities:
|
|
50
88
|
- See [example code for index and handler](./docs/00-example-implementation/example-handler.js)
|
|
51
89
|
- See [example code for config initialization](./docs/00-example-implementation/example-config.js)
|
|
@@ -56,6 +94,68 @@ It is recommended that you use the quick-start method when implementing for the
|
|
|
56
94
|
- [Advanced Implementation for Providing a Web Service](./docs/01-advanced-implementation-for-web-service/README.md)
|
|
57
95
|
- [Additional Documentation](./docs/README.md)
|
|
58
96
|
|
|
97
|
+
## Quick Start Examples
|
|
98
|
+
|
|
99
|
+
### Basic Caching Example
|
|
100
|
+
|
|
101
|
+
```javascript
|
|
102
|
+
const { cache } = require("@63klabs/cache-data");
|
|
103
|
+
|
|
104
|
+
// Initialize cache with your S3 bucket and DynamoDB table
|
|
105
|
+
cache.Cache.init({
|
|
106
|
+
s3Bucket: process.env.CACHE_DATA_S3_BUCKET,
|
|
107
|
+
dynamoDbTable: process.env.CACHE_DATA_DYNAMODB_TABLE,
|
|
108
|
+
securityKey: process.env.CACHE_DATA_SECURITY_KEY
|
|
109
|
+
});
|
|
110
|
+
|
|
111
|
+
// Cache some data
|
|
112
|
+
const cacheKey = "my-data-key";
|
|
113
|
+
const dataToCache = { message: "Hello, World!", timestamp: Date.now() };
|
|
114
|
+
|
|
115
|
+
await cache.Cache.put(cacheKey, dataToCache, 3600); // Cache for 1 hour
|
|
116
|
+
|
|
117
|
+
// Retrieve cached data
|
|
118
|
+
const cachedData = await cache.Cache.get(cacheKey);
|
|
119
|
+
if (cachedData) {
|
|
120
|
+
console.log("Retrieved from cache:", cachedData);
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Making Endpoint Requests
|
|
125
|
+
|
|
126
|
+
```javascript
|
|
127
|
+
const { endpoint } = require("@63klabs/cache-data");
|
|
128
|
+
|
|
129
|
+
// Make a simple GET request to an API
|
|
130
|
+
const response = await endpoint.get(
|
|
131
|
+
{ host: "api.example.com", path: "/data" },
|
|
132
|
+
{ parameters: { q: "search-term" } }
|
|
133
|
+
);
|
|
134
|
+
|
|
135
|
+
console.log("API Response:", response.body);
|
|
136
|
+
console.log("Status Code:", response.statusCode);
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### Using Utility Tools
|
|
140
|
+
|
|
141
|
+
```javascript
|
|
142
|
+
const { tools } = require("@63klabs/cache-data");
|
|
143
|
+
|
|
144
|
+
// Create a timer to measure performance
|
|
145
|
+
const timer = new tools.Timer("my-operation");
|
|
146
|
+
timer.start();
|
|
147
|
+
|
|
148
|
+
// Your code here...
|
|
149
|
+
|
|
150
|
+
timer.stop();
|
|
151
|
+
console.log(`Operation took ${timer.elapsed()}ms`);
|
|
152
|
+
|
|
153
|
+
// Use the logger
|
|
154
|
+
const logger = new tools.DebugAndLog("MyApp");
|
|
155
|
+
logger.info("Application started");
|
|
156
|
+
logger.error("An error occurred", { details: "error info" });
|
|
157
|
+
```
|
|
158
|
+
|
|
59
159
|
## Help
|
|
60
160
|
|
|
61
161
|
Make sure you have your S3 bucket, DynamoDb table, and SSM Parameter store set up. Also make sure that you have IAM policies to allow your Lambda function access to these, and CodeBuild to read and write to SSM Parameter store.
|
|
@@ -64,30 +164,42 @@ Review the [Documentation](./docs/README.md) which includes implementation guide
|
|
|
64
164
|
|
|
65
165
|
A full implementation example and tutorial is provided as one of the Atlantis Application Starters available through the [Atlantis Tutorials repository](https://github.com/63klabs/atlantis-tutorials). (Atlantis is a collection of templates and deployment scripts to assist in starting and automating serverless deployments using AWS SAM and CloudFormation.)
|
|
66
166
|
|
|
67
|
-
##
|
|
167
|
+
## Tutorials
|
|
168
|
+
|
|
169
|
+
Read the [Atlantis Tutorials introductory page](https://github.com/63Klabs/atlantis-tutorials) for overall usage of Atlantis Platform Templates and Scripts.
|
|
170
|
+
|
|
171
|
+
## License
|
|
68
172
|
|
|
69
|
-
|
|
173
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE.txt) file for details.
|
|
70
174
|
|
|
71
175
|
## Change Log
|
|
72
176
|
|
|
73
|
-
See [Change Log](CHANGELOG.md) for version history and changes.
|
|
177
|
+
See [Change Log](./CHANGELOG.md) for version history and changes.
|
|
74
178
|
|
|
75
179
|
## Issues, Features, and Enhancements
|
|
76
180
|
|
|
77
181
|
Visit the [Issues section of the @63Klabs Cache-Data GitHub repository](https://github.com/63Klabs/cache-data/issues) for information on reported issues, upcoming fixes and enhancements, and to submit requests.
|
|
78
182
|
|
|
79
|
-
##
|
|
183
|
+
## Security
|
|
184
|
+
|
|
185
|
+
If you discover any security related issues, please see the [SECURITY](SECURITY.md) file for details.
|
|
186
|
+
|
|
187
|
+
## Contributing
|
|
188
|
+
|
|
189
|
+
Contributions are welcome! Please see [CONTRIBUTING.md](./CONTRIBUTING.md) for details.
|
|
190
|
+
|
|
191
|
+
**For Contributors**: After cloning the repository, run the setup script to configure your development environment:
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
./scripts/setup-dev-environment.sh
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
This will install dependencies, configure the pre-commit hook for documentation validation, and run tests to ensure everything is working correctly.
|
|
80
198
|
|
|
81
|
-
|
|
199
|
+
## AI Context
|
|
82
200
|
|
|
83
|
-
|
|
201
|
+
See [AI_CONTEXT.md](AI_CONTEXT.md) for important context and guidelines for AI-generated code in this repository.
|
|
84
202
|
|
|
85
|
-
|
|
203
|
+
The context file is also helpful (and perhaps essential) for HUMANS developing within the application's structured platform as well.
|
|
86
204
|
|
|
87
|
-
-
|
|
88
|
-
- [AWS Certified Developer - Associate](https://www.credly.com/users/chad-kluck/badges)
|
|
89
|
-
- [Website: chadkluck.me](https://chadkluck.me/)
|
|
90
|
-
- [GitHub: chadkluck](https://github.com/chadkluck)
|
|
91
|
-
- [GitHub: 63Klabs](https://github.com/63klabs)
|
|
92
|
-
- [Mastodon: @chadkluck@universeodon.com](https://universeodon.com/@chadkluck)
|
|
93
|
-
- [LinkedIn](https://www.linkedin.com/in/chadkluck/)
|
|
205
|
+
AI Assisted Engineering of this solution was provided by [Kiro](https://kiro.dev/). Steering documents are provided in the repository's [.kiro](.kiro/steering/ai-context-reference.md) directory. Because testing is tightly coupled with the implementation, it is suggested all documents, code, and tests are thoroughly reviewed before, and updated after, any changes.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@63klabs/cache-data",
|
|
3
|
-
"version": "1.3.
|
|
3
|
+
"version": "1.3.6",
|
|
4
4
|
"description": "Cache data from an API endpoint or application process using AWS S3 and DynamoDb",
|
|
5
5
|
"author": "Chad Leigh Kluck (https://chadkluck.me)",
|
|
6
6
|
"license": "MIT",
|
|
@@ -25,10 +25,14 @@
|
|
|
25
25
|
"@aws-sdk/client-s3": "3.x",
|
|
26
26
|
"@aws-sdk/client-ssm": "3.x",
|
|
27
27
|
"@aws-sdk/lib-dynamodb": "3.x",
|
|
28
|
-
"chai": "^6.
|
|
29
|
-
"chai-http": "^5.
|
|
30
|
-
"
|
|
31
|
-
"
|
|
28
|
+
"chai": "^6.x",
|
|
29
|
+
"chai-http": "^5.x",
|
|
30
|
+
"fast-check": "^4.x",
|
|
31
|
+
"mocha": "^11.x",
|
|
32
|
+
"sinon": "^21.x"
|
|
33
|
+
},
|
|
34
|
+
"overrides": {
|
|
35
|
+
"fast-xml-parser": ">=5.3.4"
|
|
32
36
|
},
|
|
33
37
|
"scripts": {
|
|
34
38
|
"test": "mocha 'test/**/*-tests.mjs'",
|
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# Documentation Validation Scripts
|
|
2
|
+
|
|
3
|
+
This directory contains scripts for validating and auditing documentation in the @63klabs/cache-data package.
|
|
4
|
+
|
|
5
|
+
## Scripts
|
|
6
|
+
|
|
7
|
+
### audit-documentation.mjs
|
|
8
|
+
|
|
9
|
+
Comprehensive documentation audit and validation script that checks:
|
|
10
|
+
|
|
11
|
+
- **JSDoc Completeness**: Ensures all exported functions have complete JSDoc with required tags (@param, @returns, @example)
|
|
12
|
+
- **JSDoc Accuracy**: Verifies that documented parameters match actual function signatures
|
|
13
|
+
- **Link Validation**: Checks all links in documentation files to ensure they point to existing files
|
|
14
|
+
- **Example Code Validation**: Validates that JavaScript code examples have correct syntax
|
|
15
|
+
|
|
16
|
+
#### Usage
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
node scripts/audit-documentation.mjs
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
#### Output
|
|
23
|
+
|
|
24
|
+
The script generates a detailed report at:
|
|
25
|
+
```
|
|
26
|
+
.kiro/specs/1-3-6-documentation-enhancement/audit-report.json
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The report includes:
|
|
30
|
+
- Summary statistics (coverage percentages, error counts)
|
|
31
|
+
- List of functions missing JSDoc
|
|
32
|
+
- List of functions with incomplete JSDoc
|
|
33
|
+
- List of functions with inaccurate JSDoc (hallucinated parameters)
|
|
34
|
+
- List of broken links in documentation
|
|
35
|
+
- List of invalid code examples
|
|
36
|
+
|
|
37
|
+
#### Exit Codes
|
|
38
|
+
|
|
39
|
+
- `0`: All validation checks passed
|
|
40
|
+
- `1`: Critical errors found (missing JSDoc, inaccurate JSDoc, broken links, or invalid examples)
|
|
41
|
+
|
|
42
|
+
### Pre-Commit Hook
|
|
43
|
+
|
|
44
|
+
A Git pre-commit hook is installed at `.git/hooks/pre-commit` that automatically runs documentation validation before each commit.
|
|
45
|
+
|
|
46
|
+
#### How It Works
|
|
47
|
+
|
|
48
|
+
1. When you run `git commit`, the hook automatically executes
|
|
49
|
+
2. It runs `audit-documentation.mjs` to validate all documentation
|
|
50
|
+
3. If critical errors are found, the commit is blocked
|
|
51
|
+
4. You must fix the errors before committing
|
|
52
|
+
|
|
53
|
+
#### Bypassing the Hook
|
|
54
|
+
|
|
55
|
+
If you need to commit despite validation errors (not recommended):
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
git commit --no-verify
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
#### Installing the Hook
|
|
62
|
+
|
|
63
|
+
The hook is automatically created when task 16.3 is completed. If you need to reinstall it:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
chmod +x .git/hooks/pre-commit
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Validation Requirements
|
|
70
|
+
|
|
71
|
+
### JSDoc Requirements
|
|
72
|
+
|
|
73
|
+
All exported functions, methods, and classes must have:
|
|
74
|
+
|
|
75
|
+
1. **Description**: Clear explanation of what the function does
|
|
76
|
+
2. **@param tags**: For each parameter, with type and description
|
|
77
|
+
3. **@returns tag**: With detailed type information and description
|
|
78
|
+
4. **@example tag**: Demonstrating typical usage
|
|
79
|
+
5. **@throws tag**: If the function can throw errors (optional but recommended)
|
|
80
|
+
|
|
81
|
+
### Type Annotation Standards
|
|
82
|
+
|
|
83
|
+
- Promises: `{Promise<ResolvedType>}`
|
|
84
|
+
- Arrays: `{Array.<ElementType>}`
|
|
85
|
+
- Objects: `{Object.<string, Type>}` or detailed property list
|
|
86
|
+
- Complex returns: `{Promise<{prop1: Type1, prop2: Type2}>}`
|
|
87
|
+
- Optional parameters: `{Type} [paramName=default]`
|
|
88
|
+
- Union types: `{Type1|Type2}`
|
|
89
|
+
|
|
90
|
+
### Example Code Standards
|
|
91
|
+
|
|
92
|
+
All JavaScript code examples in documentation must:
|
|
93
|
+
|
|
94
|
+
- Have valid syntax (pass Node.js syntax check)
|
|
95
|
+
- Include necessary imports if using package functionality
|
|
96
|
+
- Not use deprecated APIs
|
|
97
|
+
- Be executable or clearly marked as snippets
|
|
98
|
+
|
|
99
|
+
### Link Standards
|
|
100
|
+
|
|
101
|
+
All links in documentation must:
|
|
102
|
+
|
|
103
|
+
- Point to existing files for internal links
|
|
104
|
+
- Be valid URLs for external links
|
|
105
|
+
- Use relative paths for internal documentation
|
|
106
|
+
|
|
107
|
+
## Common Issues and Fixes
|
|
108
|
+
|
|
109
|
+
### Missing JSDoc
|
|
110
|
+
|
|
111
|
+
**Issue**: Function has no JSDoc comment
|
|
112
|
+
|
|
113
|
+
**Fix**: Add a JSDoc comment block above the function:
|
|
114
|
+
|
|
115
|
+
```javascript
|
|
116
|
+
/**
|
|
117
|
+
* Description of what the function does
|
|
118
|
+
*
|
|
119
|
+
* @param {Type} paramName - Description of parameter
|
|
120
|
+
* @returns {ReturnType} Description of return value
|
|
121
|
+
* @example
|
|
122
|
+
* // Example usage
|
|
123
|
+
* const result = functionName(param);
|
|
124
|
+
*/
|
|
125
|
+
function functionName(paramName) {
|
|
126
|
+
// ...
|
|
127
|
+
}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Incomplete JSDoc
|
|
131
|
+
|
|
132
|
+
**Issue**: JSDoc is missing required tags
|
|
133
|
+
|
|
134
|
+
**Fix**: Add the missing tags (@param, @returns, @example)
|
|
135
|
+
|
|
136
|
+
### Inaccurate JSDoc
|
|
137
|
+
|
|
138
|
+
**Issue**: JSDoc documents parameters that don't exist in the function signature
|
|
139
|
+
|
|
140
|
+
**Fix**: Update JSDoc to match the actual function signature, or update the function signature if JSDoc is correct
|
|
141
|
+
|
|
142
|
+
### Broken Links
|
|
143
|
+
|
|
144
|
+
**Issue**: Documentation contains links to non-existent files
|
|
145
|
+
|
|
146
|
+
**Fix**: Update the link to point to the correct file, or create the missing file
|
|
147
|
+
|
|
148
|
+
### Invalid Code Examples
|
|
149
|
+
|
|
150
|
+
**Issue**: Code example has syntax errors
|
|
151
|
+
|
|
152
|
+
**Fix**: Correct the syntax error, or mark the code as a snippet if it's intentionally incomplete
|
|
153
|
+
|
|
154
|
+
## Integration with CI/CD
|
|
155
|
+
|
|
156
|
+
To integrate documentation validation into your CI/CD pipeline:
|
|
157
|
+
|
|
158
|
+
```yaml
|
|
159
|
+
# Example GitHub Actions workflow
|
|
160
|
+
- name: Validate Documentation
|
|
161
|
+
run: node scripts/audit-documentation.mjs
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
The script will exit with code 1 if validation fails, causing the CI/CD pipeline to fail.
|
|
165
|
+
|
|
166
|
+
## Maintenance
|
|
167
|
+
|
|
168
|
+
The validation script should be updated when:
|
|
169
|
+
|
|
170
|
+
- New JSDoc requirements are added
|
|
171
|
+
- New documentation standards are established
|
|
172
|
+
- New types of validation checks are needed
|
|
173
|
+
- Deprecated APIs change
|
|
174
|
+
|
|
175
|
+
See `.kiro/specs/1-3-6-documentation-enhancement/STEERING.md` for documentation standards and maintenance procedures.
|