@63klabs/cache-data 1.3.7 → 1.3.8

package/CHANGELOG.md

@@ -8,9 +8,65 @@ To report an issue, or to see proposed and upcoming enhancements, check out [63K
 
 Report all vulnerabilities under the [Security menu](https://github.com/63Klabs/cache-data/security/advisories) in the Cache-Data GitHub repository.
 
-## v1.3.8 (unreleased)
+## v1.3.8 (2026-03-02)
+
+### Security
+- **Updated the way dependencies are listed reduce npm Security Vulnerabilities** [Spec: 1-3-8-npm-security-vulnerabilities-fix](.kiro/specs/1-3-8-npm-security-vulnerabilities-fix/)
+  - Fixed 1 low, 20 high, and 1 critical severity vulnerabilities in development dependencies
+  - Updated devDependencies to specific secure versions:
+    - `@aws-sdk/client-dynamodb`, `@aws-sdk/client-s3`, `@aws-sdk/client-ssm`, `@aws-sdk/lib-dynamodb`: Updated from `3.x` to `^3.995.0`
+    - `mocha`: Updated from `^11.x` to `^11.7.5`
+    - `chai`: Updated from `^6.x` to `^6.2.2`
+    - `chai-http`: Updated from `^5.x` to `^5.1.2`
+    - `sinon`: Updated from `^21.x` to `^21.0.1`
+    - `fast-check`: Updated from `^4.x` to `^4.5.3`
+  - Added npm overrides for transitive dependencies:
+    - `diff`: `>=8.0.3` (fixes low severity DoS vulnerability)
+    - `minimatch`: `>=10.2.2` (fixes high severity ReDoS vulnerability)
+    - `glob`: `>=13.0.6` (fixes transitive vulnerabilities)
+  - No breaking changes to public APIs
+  - All existing tests pass
+  - Production dependencies remain secure (0 vulnerabilities)
 
-- Nothing yet
+### Added
+- **APIRequest Pagination, Retry, and X-Ray Enhancements** [Spec: 1-3-8-api-request-pagination-retries-xray](.kiro/specs/1-3-8-api-request-pagination-retries-xray/) addresses [#171](https://github.com/63Klabs/cache-data/issues/171), [#172](https://github.com/63Klabs/cache-data/issues/172), [#173](https://github.com/63Klabs/cache-data/issues/173)
+  - **Automatic Pagination**: APIRequest now supports automatic pagination for APIs that return paginated results
+    - Configurable pagination labels for different API response structures
+    - Batch processing for concurrent page requests
+    - Support for both offset-based and token-based pagination
+    - Automatic result combination into single response
+    - Pagination metadata in response (total pages, total items, incomplete status)
+  - **Automatic Retry Logic**: APIRequest now includes built-in retry functionality for transient failures
+    - Configurable retry attempts (default: 1 retry after initial attempt)
+    - Automatic retry on network errors, empty responses, parse errors, and 5xx status codes
+    - Optional retry on 4xx status codes (disabled by default)
+    - Retry metadata in response (attempts made, final attempt number)
+  - **Enhanced X-Ray Subsegments**: Improved AWS X-Ray tracing for better observability
+    - Unique subsegment names for each request using timestamps
+    - Retry and pagination metadata included in X-Ray traces
+    - Separate subsegments for each paginated request
+    - Detailed annotations and metadata for debugging
+  - **Response Metadata**: New optional metadata field in responses
+    - Retry information (occurred, attempts, final attempt)
+    - Pagination information (occurred, total pages, total items, incomplete status)
+    - Only present when retries or pagination occur
+  - **Backwards Compatibility Maintained**: All new features are opt-in via configuration
+    - Existing code continues to work without modification
+    - No breaking changes to public API
+    - Default behavior unchanged when new features not configured
+  - See documentation: [Pagination Guide](docs/features/tools/api-request-pagination.md), [Retry Guide](docs/features/tools/api-request-retry.md), [X-Ray Guide](docs/features/tools/api-request-xray.md)
+- **tools.AppConfig** replaces `tools._ConfigSuperClass` and receives an `.init()` to simplify initialization
+  - Updated documentation to reflect how `Config` can be implemented to extend `tools._ConfigSuperClass`
+  - Instead of separate `Response.init()`, `ClientRequest.init()` and `Connections.init()` a single `AppConfig.init()` can be used within the `Config.init()`
+
+### Enhancement
+
+- **Expanded Generic Responses** - Additional responses for status codes
+  - Added responses for 408, 418, and 427
+  - Ensured all generic responses have the same statuses
+- **Revised Documentation** - Clarified and revised documentation
+  - Reviewed all code examples
+  - Reviewed implementation instructions
 
 ## v1.3.7 (2026-02-06)
 

package/CONTRIBUTING.md

@@ -28,7 +28,7 @@ Code must be reviewed, understood, and tested by a human before being merged.
 
 Kiro is the required AI coding assistant for final integrations, documentation, and testing, as it is in the AWS Ecosystem and this project is deveoped to deploy on the AWS platform. Just like test suites, Kiro ensures the proper tests, documentation, and guardrails are in place. Kiro is as important as commit-hooks and tests as it is a tool that ensures quality checks and should not be bypassed.
 
-Ensure [AI Context](./AI_CONTEXT.md) and [Kiro steering documents](.kiro/steering/ai-context-reference.md) are reviewed, understood, and used by both humans and AI.
+Ensure [AGENTS](./AGENTS.md) and Kiro steering documents are reviewed, understood, and used by both humans and AI.
 
 ## Development Setup
 
@@ -162,6 +162,5 @@ Thank you to the following people who have contributed to this project:
 
 Chad Kluck\
 DevOps & Developer Experience Engineer\
-AWS Certified Cloud Practitioner | AWS Certified Developer - Associate | AWS
-Certified Solutions Architect - Associate\
+AWS Certified Developer and Solutions Architect\
 [Website](https://chadkluck.me)

package/README.md

@@ -29,7 +29,10 @@ The @63klabs/cache-data package provides three main modules:
 - **Cache Profiles**: Configure multiple cache profiles with different expiration policies
 
 ### Endpoint Module
-- **HTTP/HTTPS Requests**: Make requests to external APIs and endpoints with built-in retry logic
+- **HTTP/HTTPS Requests**: Make requests to external APIs and endpoints with automatic pagination, retry logic, and X-Ray tracing
+- **Automatic Pagination**: Fetch all pages from paginated APIs automatically with configurable batch processing
+- **Automatic Retry**: Retry failed requests with configurable conditions (network errors, server errors, empty responses)
+- **Enhanced X-Ray Tracing**: Detailed monitoring and debugging with AWS X-Ray subsegments
 - **Connection Management**: Define and manage multiple endpoint connections with authentication
 - **Request Caching**: Automatically cache endpoint responses to reduce API calls and improve performance
 - **Flexible Configuration**: Support for custom headers, query parameters, request bodies, and timeouts
@@ -136,6 +139,42 @@ console.log("API Response:", response.body);
 console.log("Status Code:", response.statusCode);
 ```
 
+### Using APIRequest with Pagination and Retry
+
+```javascript
+const { tools } = require("@63klabs/cache-data");
+
+// Make a request with automatic pagination and retry
+const request = new tools.APIRequest({
+  host: "api.example.com",
+  path: "/users",
+  parameters: {
+    limit: 100  // Items per page
+  },
+  pagination: {
+    enabled: true  // Automatically fetch all pages
+  },
+  retry: {
+    enabled: true,
+    maxRetries: 2  // Retry failed requests up to 2 times
+  }
+});
+
+const response = await request.send();
+
+// Response contains all users from all pages
+const allUsers = JSON.parse(response.body).items;
+console.log(`Retrieved ${allUsers.length} total users`);
+
+// Check metadata
+if (response.metadata?.pagination?.occurred) {
+  console.log(`Fetched ${response.metadata.pagination.totalPages} pages`);
+}
+if (response.metadata?.retries?.occurred) {
+  console.log(`Succeeded after ${response.metadata.retries.attempts} attempts`);
+}
+```
+
 ### Using Utility Tools
 
 ```javascript
@@ -198,8 +237,8 @@ This will install dependencies, configure the pre-commit hook for documentation
 
 ## AI Context
 
-See [AI_CONTEXT.md](AI_CONTEXT.md) for important context and guidelines for AI-generated code in this repository.
+See [AGENTS.md](AGENTS.md) for important context and guidelines for AI-generated code in this repository.
 
 The context file is also helpful (and perhaps essential) for HUMANS developing within the application's structured platform as well.
 
-AI Assisted Engineering of this solution was provided by [Kiro](https://kiro.dev/). Steering documents are provided in the repository's [.kiro](.kiro/steering/ai-context-reference.md) directory. Because testing is tightly coupled with the implementation, it is suggested all documents, code, and tests are thoroughly reviewed before, and updated after, any changes.
+AI Assisted Engineering of this solution was provided by [Kiro](https://kiro.dev/). Steering documents are provided in the repository's `.kiro/steering/` directory. Because testing is tightly coupled with the implementation, it is suggested all documents, code, and tests are thoroughly reviewed before, and updated after, any changes.

package/eslint.config.js

@@ -0,0 +1,53 @@
+import js from '@eslint/js';
+
+export default [
+	js.configs.recommended,
+	{
+		languageOptions: {
+			ecmaVersion: 'latest',
+			sourceType: 'module',
+			globals: {
+				// Node.js globals
+				console: 'readonly',
+				process: 'readonly',
+				Buffer: 'readonly',
+				__dirname: 'readonly',
+				__filename: 'readonly',
+				exports: 'writable',
+				module: 'writable',
+				require: 'readonly',
+				global: 'readonly',
+				// Mocha globals
+				describe: 'readonly',
+				it: 'readonly',
+				before: 'readonly',
+				after: 'readonly',
+				beforeEach: 'readonly',
+				afterEach: 'readonly',
+				// ES2021 globals
+				globalThis: 'readonly'
+			}
+		},
+		rules: {
+			'no-template-curly-in-string': 'error',
+			'no-eval': 'error',
+			'no-implied-eval': 'error',
+			'no-new-func': 'error'
+		}
+	},
+	{
+		files: ['test/**/*.mjs', 'test/**/*.js', 'scripts/**/*.mjs', 'scripts/**/*.js', 'src/**/*.js'],
+		rules: {
+			'no-restricted-imports': ['error', {
+				patterns: [{
+					group: ['child_process'],
+					importNames: ['exec', 'execSync'],
+					message: 'Use execFile or execFileSync instead of exec/execSync to prevent shell injection. See .kiro/steering/secure-coding-practices.md'
+				}]
+			}]
+		}
+	},
+	{
+		ignores: ['node_modules/**', 'coverage/**', 'coverage-jest/**', '.git/**']
+	}
+];

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@63klabs/cache-data",
-  "version": "1.3.7",
+  "version": "1.3.8",
   "description": "Cache data from an API endpoint or application process using AWS S3 and DynamoDb",
   "author": "Chad Leigh Kluck (https://chadkluck.me)",
   "license": "MIT",
@@ -21,32 +21,43 @@
     "object-hash": "^3.0.0"
   },
   "devDependencies": {
-    "@aws-sdk/client-dynamodb": "3.x",
-    "@aws-sdk/client-s3": "3.x",
-    "@aws-sdk/client-ssm": "3.x",
-    "@aws-sdk/lib-dynamodb": "3.x",
-    "chai": "^6.x",
-    "chai-http": "^5.x",
-    "fast-check": "^4.x",
+    "@aws-sdk/client-dynamodb": "^3.995.0",
+    "@aws-sdk/client-s3": "^3.995.0",
+    "@aws-sdk/client-ssm": "^3.995.0",
+    "@aws-sdk/lib-dynamodb": "^3.995.0",
+    "@eslint/js": "^10.0.1",
+    "chai": "^6.2.2",
+    "chai-http": "^5.1.2",
+    "eslint": "^10.0.1",
+    "fast-check": "^4.5.3",
     "jest": "^30.2.0",
-    "mocha": "^11.x",
-    "sinon": "^21.x"
+    "mocha": "^11.7.5",
+    "sinon": "^21.0.1"
   },
   "overrides": {
-    "fast-xml-parser": ">=5.3.4"
+    "fast-xml-parser": ">=5.3.4",
+    "diff": ">=8.0.3",
+    "minimatch": ">=10.2.2",
+    "glob": ">=13.0.6"
   },
   "scripts": {
-    "test": "mocha 'test/**/*-tests.mjs'",
+    "test": "mocha 'test/**/*-tests.mjs' --exclude 'test/migration/property/test-execution-equivalence-property-tests.mjs'",
     "test:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
     "test:all": "npm test && npm run test:jest",
+    "test:migration:validation": "mocha test/migration/property/test-execution-equivalence-property-tests.mjs",
     "test:cache": "mocha 'test/cache/**/*-tests.mjs'",
     "test:cache:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js test/cache",
     "test:config": "mocha 'test/config/**/*-tests.mjs'",
     "test:endpoint": "mocha 'test/endpoint/**/*-tests.mjs'",
+    "test:endpoint:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js test/endpoint",
+    "test:tools:jest": "node --experimental-vm-modules node_modules/jest/bin/jest.js test/tools",
     "test:logging": "mocha 'test/logging/**/*-tests.mjs'",
     "test:request": "mocha 'test/request/**/*-tests.mjs'",
     "test:response": "mocha 'test/response/**/*-tests.mjs'",
-    "test:utils": "mocha 'test/utils/**/*-tests.mjs'"
+    "test:utils": "mocha 'test/utils/**/*-tests.mjs'",
+    "lint": "eslint .",
+    "lint:fix": "eslint . --fix",
+    "lint:ci": "eslint . --max-warnings 0"
   },
   "keywords": [
     "api",

package/src/lib/dao-cache.js

@@ -3200,7 +3200,6 @@ class TestHarness {
 	 * WARNING: This method is for testing only and should never be used in production.
 	 * 
 	 * @returns {{CacheData: typeof CacheData, S3Cache: typeof S3Cache, DynamoDbCache: typeof DynamoDbCache}} Object containing internal classes
-	 * @private
 	 * @example
 	 * // In tests only - DO NOT use in production
 	 * const { CacheData, S3Cache, DynamoDbCache } = TestHarness.getInternals();