cat-a-logs 2.0.1 → 2.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +135 -62
- package/client/index.js +27 -74
- package/client/index.ts +26 -24
- package/package.json +2 -13
- package/snapshots/Catalog_art.png +0 -0
- package/snapshots/customNameSpace.png +0 -0
- package/snapshots/keyDefined.png +0 -0
- package/snapshots/metricName.png +0 -0
- package/snapshots/trackedVariable.png +0 -0
- package/tsconfig.json +1 -1
package/LICENSE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2025 Open Source Labs Beta
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
@@ -28,26 +28,147 @@ Your chosen Integated Development Environment (i.e. VS Code) must already be be
|
|
28
28
|
function catalog(
|
29
29
|
trackedVariable: number | Array<number>,
|
30
30
|
metricName: string,
|
31
|
-
metricNamespace: string,
|
31
|
+
metricNamespace: string = "CatALog-Default-Metrics",
|
32
32
|
metricUnitLabel: string = "None",
|
33
33
|
CustomerDefinedDimension: { [key: string]: string } = {},
|
34
34
|
resolution: 1 | 60 = 60,
|
35
35
|
deploy: boolean = false)
|
36
36
|
```
|
37
37
|
|
38
|
-
- **trackedVariable**: This variable represents a
|
39
|
-
- **metricName**: This is a unique label of the tracked variable that will be reflected inside AWS Lambda. Must be written as a `string`
|
40
|
-
- **metricNamespace**: This will be your metric namespace in AWS Cloudwatch the metric or metrics will appear in
|
41
|
-
- **metricUnitLabel**: Explict Unit that Cloudwatch uses for EMF Configuration. Please note - Can only be the following labels:
|
42
|
-
- Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None
|
38
|
+
- **trackedVariable**: This variable represents a the numerical value of the metric that will appear under the category "Custom namespace" in Cloudwatch Metrics. Custom metric category/namespace/AWS Namespace. This is AWS Cloudwatch>Metrics>All metrics>Custom namespaces(ex. CatALog)>Dimensions(ex. Server, functionVersion)
|
43
39
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
40
|
+
<p align="center">
|
41
|
+
<img src="./snapshots/trackedVariable.png" width="600" />
|
42
|
+
</p>
|
43
|
+
|
44
|
+
|
45
|
+
- **metricName**: This is a unique label of the tracked variable that will be reflected inside AWS Cloudwatch. Must be written as a `string`
|
46
|
+
In the below image this corresponds to `Latency` --> AWS Cloudwatch>Metrics>All metrics>Custom namespaces
|
48
47
|
|
48
|
+
<p align="center">
|
49
|
+
<img src="./snapshots/metricName.png" width="600"/>
|
50
|
+
</p>
|
49
51
|
|
52
|
+
- **metricNamespace**: This will be your "Custom namespace" in AWS Cloudwatch>Metrics>All metrics>Custom namespaces. In below image this is represented by CatALog
|
50
53
|
|
54
|
+
<p align="center">
|
55
|
+
<img src="./snapshots/customNameSpace.png" width="600"/>
|
56
|
+
</p>
|
57
|
+
|
58
|
+
- **metricUnitLabel**: Explict Unit that Cloudwatch uses for EMF Configuration. Please note - must be one of the following as a `string`:
|
59
|
+
- Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None
|
60
|
+
|
61
|
+
- To read more about Metric Datum see this <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html" target="_blank">link</a>
|
62
|
+
- **CustomerDefinedDimension**: This is an object - `{'functionVersion': '$LATEST', 'Server': 'Prod'}` functionVersion & Server is the dimension label/key - when you click on it see the value `$LATEST` and `Prod` is the value of the dimension
|
63
|
+
|
64
|
+
- The key will show up in AWS Cloudwatch as below:
|
65
|
+
|
66
|
+
<p align="center">
|
67
|
+
<img src="./snapshots/keyDefined.png" width="600"/>
|
68
|
+
</p>
|
69
|
+
|
70
|
+
- If the user clicks on the Server, functionVersion Dimension then you will see the value - in this example `$LATEST` & `Prod` reflected as below:
|
71
|
+
|
72
|
+
<p align="center">
|
73
|
+
<img src="./snapshots/metricName.png" width="600"/>
|
74
|
+
</p>
|
75
|
+
|
76
|
+
|
77
|
+
- **resolution**: This is automatically set to default value to 60. If you would like to learn more about High Resolution Metrics please follow the attached <a href= "https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics" target="_blank">link</a>
|
78
|
+
- **deploy**: automatically set to false. The final catalog call you make has to switch deploy flag to true. Failure to do so will cause the cache to grow without bound and use up memory
|
79
|
+
|
80
|
+
3. Start Building your Embedded Metric Formatted Logs. Call catalog as many times as needed.
|
81
|
+
<!-- You can also `console.log(cache)` at any time to see your EMF formatted logs being built in real time. -->
|
82
|
+
|
83
|
+
4. ON the very last function call - it is important to change the deploy parameter to `true`.
|
84
|
+
|
85
|
+
5. Deploy your code with AWS SAM. This will place the file in AWS Lambda waiting for invokation.
|
86
|
+
|
87
|
+
6. Invoke your AWS Lamda Function
|
88
|
+
|
89
|
+
7. See your metrics and structured in CloudWatch!
|
90
|
+
|
91
|
+
## Open Source Contributions:
|
92
|
+
We are actively looking for contributors to our project! In order to get started we ask that you follow the below guidelines:
|
93
|
+
|
94
|
+
- Clone our Repository from GitHub <a href="https://github.com/oslabs-beta/cat-a-log" target="_blank">here</a>
|
95
|
+
- Make a Feature Branch
|
96
|
+
- Make your contributions
|
97
|
+
- Push to your Feature Branch in GitHub
|
98
|
+
- Make a Pull Request to our Repository!
|
99
|
+
|
100
|
+
|
101
|
+
| AWS MicroService Support | Status |
|
102
|
+
|---------------------------------------------------------------------------------------|-----------|
|
103
|
+
| Lamda | ✅ |
|
104
|
+
| EC2 | ⏳ |
|
105
|
+
|
106
|
+
|
107
|
+
|
108
|
+
|Feature |Status |
|
109
|
+
|---------------------------------------------------------------------------------------|-----------|
|
110
|
+
| TypeScript | ✅ |
|
111
|
+
| Embedded Metric Format Object Caching | ✅ |
|
112
|
+
| Winson | ⏳ |
|
113
|
+
| Adding front end for Cat-A-Log | 🙏🏻 |
|
114
|
+
|
115
|
+
|
116
|
+
- ✅ = Ready to use
|
117
|
+
- ⏳ = In progress
|
118
|
+
- 🙏🏻 = Looking for contributors
|
119
|
+
|
120
|
+
## License Information:
|
121
|
+
Put License Information Here
|
122
|
+
|
123
|
+
|
124
|
+
## Contributor Information:
|
125
|
+
<table>
|
126
|
+
<tr>
|
127
|
+
<td align="center">
|
128
|
+
<img src="https://avatars.githubusercontent.com/u/161962009?v=4" width="140px;" alt=""/>
|
129
|
+
<br />
|
130
|
+
<sub><b>Clara Regula</b></sub>
|
131
|
+
<br />
|
132
|
+
<a href="http://www.linkedin.com/in/clara-regula">🖇️</a>
|
133
|
+
<a href="https://github.com/clararegula">🐙</a>
|
134
|
+
</td>
|
135
|
+
<td align="center">
|
136
|
+
<img src="https://avatars.githubusercontent.com/u/170294267?v=4" width="140px;" alt=""/>
|
137
|
+
<br />
|
138
|
+
<sub><b>Brian Anderson</b></sub>
|
139
|
+
<br />
|
140
|
+
<a href="https://www.linkedin.com/in/brian-anderson-24370630/">🖇️</a>
|
141
|
+
<a href="https://github.com/brianmichaelanderson">🐙</a>
|
142
|
+
</td>
|
143
|
+
<td align="center">
|
144
|
+
<img src="https://avatars.githubusercontent.com/u/167483334?v=4" width="140px;" alt=""/>
|
145
|
+
<br />
|
146
|
+
<sub><b>Harris Awan</b></sub>
|
147
|
+
<br />
|
148
|
+
<a href="http://www.linkedin.com/in/harrawan123/">🖇️</a>
|
149
|
+
<a href="https://github.com/HarrAwa">🐙</a>
|
150
|
+
</td>
|
151
|
+
<td align="center">
|
152
|
+
<img src="https://avatars.githubusercontent.com/u/26197909?v=4" width="140px;" alt=""/>
|
153
|
+
<br />
|
154
|
+
<sub><b>Curran Lee</b></sub>
|
155
|
+
<br />
|
156
|
+
<a href="https://www.linkedin.com/search/results/all/?fetchDeterministicClustersOnly=false&heroEntityKey=urn%3Ali%3Afsd_profile%3AACoAABxBUMYBYh3jl6z8XMVs4D1VjdqU-oastdc&keywords=natalie%20klein&origin=RICH_QUERY_SUGGESTION&position=0&searchId=7ca29d7e-56b5-4dce-a2a1-f9d9e5594052&sid=XY8">🖇️</a>
|
157
|
+
<a href="https://github.com/natalie-klein">🐙</a>
|
158
|
+
<td align="center">
|
159
|
+
<img src="https://avatars.githubusercontent.com/u/80185584?v=4" width="140px;" alt=""/>
|
160
|
+
<br />
|
161
|
+
<sub><b>Jacob Alexander</b></sub>
|
162
|
+
<br />
|
163
|
+
<a href="https://www.linkedin.com/search/results/all/?fetchDeterministicClustersOnly=false&heroEntityKey=urn%3Ali%3Afsd_profile%3AACoAAAnv9wwBJJ9SgtkuND-IT1hQIl6hVS50AJ4&keywords=mike%20masatsugu&origin=RICH_QUERY_SUGGESTION&position=0&searchId=51ea03d4-28fa-431c-b97c-df470d78d606&sid=~Ov">🖇️</a>
|
164
|
+
<a href="https://github.com/mikemasatsugu">🐙</a>
|
165
|
+
</td>
|
166
|
+
</table>
|
167
|
+
|
168
|
+
|
169
|
+
- 🖇️ = LinkedIn
|
170
|
+
- 🐙 = Github
|
171
|
+
|
51
172
|
## Notes to Self:
|
52
173
|
**Structure of the files:**
|
53
174
|
|
@@ -64,55 +185,7 @@ if you write name that it will overwrite the keys
|
|
64
185
|
**Tech Challenges**
|
65
186
|
Spent 3 days dealing with inconsistencies of ES6/CommonJS in our code before compiling .js in ES6
|
66
187
|
|
67
|
-
**
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
tailwind.config.js
|
72
|
-
input.css?
|
73
|
-
postcss.config.js
|
74
|
-
server(folder)
|
75
|
-
coverage (folder)
|
76
|
-
bin (folder)
|
77
|
-
template.html
|
78
|
-
testlogger.js
|
79
|
-
|
80
|
-
-->webpack.config.js? **Test renaming to see if it breaks.
|
81
|
-
|
82
|
-
Package json:
|
83
|
-
"@babel/core": "^7.25.7",
|
84
|
-
"@babel/plugin-transform-runtime": "^7.25.7",
|
85
|
-
"@babel/preset-env": "^7.25.8",
|
86
|
-
"@babel/preset-react": "^7.25.7",
|
87
|
-
"@babel/runtime": "^7.25.7",
|
88
|
-
"babel-eslint": "^10.1.0",
|
89
|
-
"babel-loader": "^9.2.1",
|
90
|
-
"cross-env": "^7.0.3",
|
91
|
-
"css-loader": "^7.1.2",
|
92
|
-
"html-webpack-plugin": "^5.6.0",
|
93
|
-
"node-mocks-http": "^1.16.1",
|
94
|
-
"nodemon": "^3.1.7",
|
95
|
-
"postcss": "^8.4.49",
|
96
|
-
"postcss-loader": "^8.1.1",
|
97
|
-
"postcss-preset-env": "^10.1.1",
|
98
|
-
"sass": "^1.80.5",
|
99
|
-
"sass-loader": "^16.0.2",
|
100
|
-
"style-loader": "^4.0.0",
|
101
|
-
"tailwindcss": "^3.4.15",
|
102
|
-
"webpack": "^5.95.0",
|
103
|
-
"webpack-cli": "^5.1.4",
|
104
|
-
"webpack-dev-server": "^5.1.0"
|
105
|
-
"bcrypt": "^5.1.1",
|
106
|
-
"cookie-parser": "^1.4.7",
|
107
|
-
"cors": "^2.8.5",
|
108
|
-
"express": "^4.21.1",
|
109
|
-
"express-session": "^1.18.1",
|
110
|
-
"react": "^18.3.1",
|
111
|
-
"react-dom": "^18.3.1",
|
112
|
-
"react-router-dom": "^6.27.0"
|
113
|
-
|
114
|
-
**npmignore**
|
115
|
-
lambda-nodejs22.x
|
116
|
-
|
117
|
-
**gitignore**
|
118
|
-
lambda-nodejs22.x
|
188
|
+
**To DO LIST ITEMS**
|
189
|
+
- How can the user visaulize the cache growing in real time?
|
190
|
+
- Creating more professional scrreenshots for the ReadMe - to replace the current ReadMe screenshots
|
191
|
+
- Add License Information
|
package/client/index.js
CHANGED
@@ -11,31 +11,39 @@ import { Logger } from "@aws-lambda-powertools/logger";
|
|
11
11
|
import { Ajv } from "ajv";
|
12
12
|
//cache entries are structured thusly: 'Namespace + Dimensions(Alphabetically)': EMFObject
|
13
13
|
const cache = {};
|
14
|
-
//
|
15
|
-
|
16
|
-
|
14
|
+
//let latency = 300; (Example metric to track)
|
15
|
+
//Example for in-line use of Cat-a-log w/maximum arguments: catalog(latency, "Latency" , "lambda-function-metrics", "Milliseconds", {'functionVersion': '$LATEST', 'Server': 'Prod'}, 60, deploy);
|
16
|
+
//Example for in-line use of Cat-a-log w/minimum arguments: catalog(latency, "Latency");
|
17
|
+
function catalog(trackedVariable_1, metricName_1) {
|
18
|
+
return __awaiter(this, arguments, void 0, function* (trackedVariable, metricName, metricNamespace = "CatALog-Default-Metrics", metricUnitLabel = "None", CustomerDefinedDimension = {}, resolution = 60, deploy = false) {
|
17
19
|
//Check for any errors & validate inputs based on documentations
|
18
20
|
if (!cache)
|
19
21
|
throw new Error("cache is not found, please import cache from cat-a-log");
|
20
|
-
|
22
|
+
//check if any provided dimension names or metric names conflict with native logger keys.
|
21
23
|
const badKeys = ["level", "message", "sampling_rate", "service", "timestamp", "xray_trace_id"];
|
22
24
|
const yourKeys = Object.keys(CustomerDefinedDimension).concat([metricName.toLowerCase()]);
|
23
25
|
for (let i = 0; i < yourKeys.length; i++) {
|
24
26
|
if (badKeys.includes(yourKeys[i])) {
|
27
|
+
//if a dimension name or metric name conflicts with native logger keys, throw error
|
25
28
|
throw new Error("metricName, or Dimension names cannot be the same as these native logger keys: level || message || sampling_rate || service || timestamp || xray_trace_id");
|
26
29
|
}
|
27
30
|
}
|
31
|
+
//EMF specification catch: if tracked variable is an array with a length greater than 100, throw error and do not log
|
28
32
|
if (Array.isArray(trackedVariable)) {
|
29
33
|
if (trackedVariable.length > 100)
|
30
34
|
throw new Error("metric value cannot have more than 100 elements");
|
31
35
|
}
|
32
|
-
|
33
|
-
|
34
|
-
|
36
|
+
//EMF specification catch: make sure provided dimension object does not have more than 30 entries
|
37
|
+
// if (Object.keys(CustomerDefinedDimension).length > 30) {
|
38
|
+
// throw new Error(
|
39
|
+
// "EMF has a limit of 30 user defined dimension keys per log"
|
40
|
+
// );
|
41
|
+
// }
|
42
|
+
//Create new instance of Logger to use in function
|
35
43
|
const logger = new Logger({ serviceName: "serverlessAirline" });
|
36
|
-
// Ajv instance
|
44
|
+
//Set up Ajv instance for JSON validation
|
37
45
|
const ajv = new Ajv();
|
38
|
-
// from AWS: EMF schema to test/validate against
|
46
|
+
// from AWS: EMF schema to test/validate against with Ajv
|
39
47
|
const emfSchema = {
|
40
48
|
type: "object",
|
41
49
|
title: "Root Node",
|
@@ -136,29 +144,29 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
136
144
|
},
|
137
145
|
},
|
138
146
|
};
|
147
|
+
//usable instance of the validation JSON
|
139
148
|
const validateEmf = ajv.compile(emfSchema);
|
140
|
-
//sort customerDimensions key values in alphabetical order
|
149
|
+
//sort customerDimensions key values in alphabetical order. We will use this to keep the keys in our cache consistant. Since the order of the dimensions do not change where the metrics are stored
|
141
150
|
const sortedDimensions = {};
|
142
151
|
for (let i = 0; i < Object.keys(CustomerDefinedDimension).sort().length; i++) {
|
143
152
|
sortedDimensions[Object.keys(CustomerDefinedDimension).sort()[i]] =
|
144
153
|
CustomerDefinedDimension[Object.keys(CustomerDefinedDimension).sort()[i]];
|
145
154
|
}
|
146
|
-
//if Object with Namespace and Dimensions already exists in
|
155
|
+
//Check if Object with Namespace and Dimensions already exists in cache
|
147
156
|
let check = cache[`${metricNamespace}${sortedDimensions}`];
|
148
157
|
if (check != undefined) {
|
149
|
-
//push the metrics object to Metrics array
|
158
|
+
//if the Namespace and Dimensions exist, push the metrics object to Metrics array
|
150
159
|
cache[`${metricNamespace}${sortedDimensions}`]["_aws"]["CloudWatchMetrics"][0]["Metrics"].push({
|
151
160
|
Name: metricName,
|
152
161
|
Unit: metricUnitLabel,
|
153
162
|
StorageResolution: resolution,
|
154
163
|
});
|
155
|
-
//add key value to Log
|
164
|
+
//add key value to root of existing structured Log
|
156
165
|
check[`${metricName}`] = trackedVariable;
|
157
166
|
}
|
158
167
|
else {
|
159
|
-
//
|
160
|
-
//
|
161
|
-
// NameSpace & Dimensions for EMF part don't exist yet. Initialize variable to capture EMF/aws key:value pair
|
168
|
+
//create new Structured Log and add it to cachedStructuredLogs
|
169
|
+
//If NameSpace & Dimensions for EMF part don't exist yet, initialize variable to capture EMF/aws key:value pair
|
162
170
|
const newEmfLog = Object.assign({
|
163
171
|
_aws: {
|
164
172
|
Timestamp: Date.now(),
|
@@ -182,16 +190,12 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
182
190
|
console.log("index.ts - Unit value before validation: ", newEmfLog._aws.CloudWatchMetrics[0].Metrics[0].Unit);
|
183
191
|
// validate the new EMF JSON schema against AWS EMF JSON schema before adding to cache object
|
184
192
|
const isValid = validateEmf(newEmfLog);
|
185
|
-
//
|
186
|
-
// console.log('index.ts - Validation result: ', isValid);
|
187
|
-
// troubleshooting console.error in emf test
|
188
|
-
// console.log('index.ts - Validation errors: ', validateEmf.errors);
|
189
|
-
// if it fails validation throw error
|
193
|
+
// if the new EMF object fails validation, throw error and do not cache flawed object
|
190
194
|
if (!isValid) {
|
191
195
|
console.error("An error occurred during EMF validation: ", validateEmf.errors);
|
192
196
|
throw new Error("Supplied/Proposed structured log does not comply with EMF schema");
|
193
197
|
}
|
194
|
-
// If
|
198
|
+
// If the new EMF object passes validation then add to cache object
|
195
199
|
cache[`${metricNamespace}${sortedDimensions}`] = newEmfLog;
|
196
200
|
}
|
197
201
|
if (deploy) {
|
@@ -199,7 +203,7 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
199
203
|
for (let i = 0; i < Object.keys(cache).length; i++) {
|
200
204
|
logger.info(`Your EMF compliant Structured Metrics Log ${i + 1}`, cache[Object.keys(cache)[i]]);
|
201
205
|
}
|
202
|
-
//clear cache
|
206
|
+
//clear cache after logging all cached objects to Lambda
|
203
207
|
console.log("BEFORE:", cache);
|
204
208
|
for (var member in cache)
|
205
209
|
delete cache[member];
|
@@ -208,54 +212,3 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
208
212
|
});
|
209
213
|
}
|
210
214
|
export { cache, catalog };
|
211
|
-
/*Current Working logger invocation
|
212
|
-
logger.info("Your EMF compliant Structured Metrics Log",
|
213
|
-
Object.assign({
|
214
|
-
_aws: {
|
215
|
-
Timestamp: Date.now(),
|
216
|
-
CloudWatchMetrics: [
|
217
|
-
{
|
218
|
-
Namespace: metricNamespace,
|
219
|
-
Dimensions: [Object.keys(CustomerDefinedDimension)],
|
220
|
-
Metrics: [
|
221
|
-
{
|
222
|
-
Name: metricName,
|
223
|
-
Unit: metricUnitLabel,
|
224
|
-
StorageResolution: resolution,
|
225
|
-
}
|
226
|
-
]
|
227
|
-
}
|
228
|
-
]
|
229
|
-
},
|
230
|
-
[`${metricName}`]: trackedVariable,
|
231
|
-
},
|
232
|
-
CustomerDefinedDimension
|
233
|
-
)
|
234
|
-
|
235
|
-
)
|
236
|
-
*/
|
237
|
-
//Old handler function
|
238
|
-
// export const handler = async (_event, _context): Promise<void> => {
|
239
|
-
// const testObj = {
|
240
|
-
// testguy: "hi",
|
241
|
-
// fool: 42,
|
242
|
-
// functionVersion: "$LATEST",
|
243
|
-
// _aws: {
|
244
|
-
// Timestamp: Date.now(),
|
245
|
-
// CloudWatchMetrics: [
|
246
|
-
// {
|
247
|
-
// Namespace: "lambda-function-metrics",
|
248
|
-
// Dimensions: [["functionVersion"]],
|
249
|
-
// Metrics: [
|
250
|
-
// {
|
251
|
-
// Name: "fool",
|
252
|
-
// Unit: "Milliseconds",
|
253
|
-
// StorageResolution: 60
|
254
|
-
// }
|
255
|
-
// ]
|
256
|
-
// }
|
257
|
-
// ]
|
258
|
-
// },
|
259
|
-
// }
|
260
|
-
// logger.info(JSON.stringify(testObj));
|
261
|
-
// };
|
package/client/index.ts
CHANGED
@@ -3,11 +3,13 @@ import { Ajv } from "ajv";
|
|
3
3
|
|
4
4
|
//cache entries are structured thusly: 'Namespace + Dimensions(Alphabetically)': EMFObject
|
5
5
|
const cache: { [key: string]: any } = {};
|
6
|
-
//
|
6
|
+
//let latency = 300; (Example metric to track)
|
7
|
+
//Example for in-line use of Cat-a-log w/maximum arguments: catalog(latency, "Latency" , "lambda-function-metrics", "Milliseconds", {'functionVersion': '$LATEST', 'Server': 'Prod'}, 60, deploy);
|
8
|
+
//Example for in-line use of Cat-a-log w/minimum arguments: catalog(latency, "Latency");
|
7
9
|
async function catalog(
|
8
10
|
trackedVariable: number | Array<number>,
|
9
11
|
metricName: string,
|
10
|
-
metricNamespace: string,
|
12
|
+
metricNamespace: string = "CatALog-Default-Metrics",
|
11
13
|
metricUnitLabel: string = "None",
|
12
14
|
CustomerDefinedDimension: { [key: string]: string } = {},
|
13
15
|
resolution: 1 | 60 = 60,
|
@@ -16,11 +18,12 @@ async function catalog(
|
|
16
18
|
//Check for any errors & validate inputs based on documentations
|
17
19
|
if (!cache)
|
18
20
|
throw new Error("cache is not found, please import cache from cat-a-log");
|
19
|
-
|
21
|
+
//check if any provided dimension names or metric names conflict with native logger keys.
|
20
22
|
const badKeys = ["level", "message", "sampling_rate", "service", "timestamp", "xray_trace_id"];
|
21
23
|
const yourKeys = Object.keys(CustomerDefinedDimension).concat([metricName.toLowerCase()]);
|
22
24
|
for(let i = 0; i < yourKeys.length; i++){
|
23
25
|
if(badKeys.includes(yourKeys[i])){
|
26
|
+
//if a dimension name or metric name conflicts with native logger keys, throw error
|
24
27
|
throw new Error(
|
25
28
|
"metricName, or Dimension names cannot be the same as these native logger keys: level || message || sampling_rate || service || timestamp || xray_trace_id"
|
26
29
|
);
|
@@ -28,20 +31,22 @@ async function catalog(
|
|
28
31
|
}
|
29
32
|
|
30
33
|
|
31
|
-
|
34
|
+
//EMF specification catch: if tracked variable is an array with a length greater than 100, throw error and do not log
|
32
35
|
if (Array.isArray(trackedVariable)) {
|
33
36
|
if (trackedVariable.length > 100)
|
34
37
|
throw new Error("metric value cannot have more than 100 elements");
|
35
38
|
}
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
39
|
+
//EMF specification catch: make sure provided dimension object does not have more than 30 entries
|
40
|
+
// if (Object.keys(CustomerDefinedDimension).length > 30) {
|
41
|
+
// throw new Error(
|
42
|
+
// "EMF has a limit of 30 user defined dimension keys per log"
|
43
|
+
// );
|
44
|
+
// }
|
45
|
+
//Create new instance of Logger to use in function
|
41
46
|
const logger = new Logger({ serviceName: "serverlessAirline" });
|
42
|
-
// Ajv instance
|
47
|
+
//Set up Ajv instance for JSON validation
|
43
48
|
const ajv = new Ajv();
|
44
|
-
// from AWS: EMF schema to test/validate against
|
49
|
+
// from AWS: EMF schema to test/validate against with Ajv
|
45
50
|
const emfSchema = {
|
46
51
|
type: "object",
|
47
52
|
title: "Root Node",
|
@@ -143,9 +148,9 @@ async function catalog(
|
|
143
148
|
},
|
144
149
|
},
|
145
150
|
};
|
146
|
-
|
151
|
+
//usable instance of the validation JSON
|
147
152
|
const validateEmf = ajv.compile(emfSchema);
|
148
|
-
//sort customerDimensions key values in alphabetical order
|
153
|
+
//sort customerDimensions key values in alphabetical order. We will use this to keep the keys in our cache consistant. Since the order of the dimensions do not change where the metrics are stored
|
149
154
|
const sortedDimensions: { [key: string]: string } = {};
|
150
155
|
for (
|
151
156
|
let i = 0;
|
@@ -155,10 +160,10 @@ async function catalog(
|
|
155
160
|
sortedDimensions[Object.keys(CustomerDefinedDimension).sort()[i]] =
|
156
161
|
CustomerDefinedDimension[Object.keys(CustomerDefinedDimension).sort()[i]];
|
157
162
|
}
|
158
|
-
//if Object with Namespace and Dimensions already exists in
|
163
|
+
//Check if Object with Namespace and Dimensions already exists in cache
|
159
164
|
let check = cache[`${metricNamespace}${sortedDimensions}`];
|
160
165
|
if (check != undefined) {
|
161
|
-
//push the metrics object to Metrics array
|
166
|
+
//if the Namespace and Dimensions exist, push the metrics object to Metrics array
|
162
167
|
cache[`${metricNamespace}${sortedDimensions}`]["_aws"][
|
163
168
|
"CloudWatchMetrics"
|
164
169
|
][0]["Metrics"].push({
|
@@ -166,12 +171,11 @@ async function catalog(
|
|
166
171
|
Unit: metricUnitLabel,
|
167
172
|
StorageResolution: resolution,
|
168
173
|
});
|
169
|
-
//add key value to Log
|
174
|
+
//add key value to root of existing structured Log
|
170
175
|
check[`${metricName}`] = trackedVariable;
|
171
176
|
} else {
|
172
|
-
//
|
173
|
-
//
|
174
|
-
// NameSpace & Dimensions for EMF part don't exist yet. Initialize variable to capture EMF/aws key:value pair
|
177
|
+
//create new Structured Log and add it to cachedStructuredLogs
|
178
|
+
//If NameSpace & Dimensions for EMF part don't exist yet, initialize variable to capture EMF/aws key:value pair
|
175
179
|
const newEmfLog = Object.assign(
|
176
180
|
{
|
177
181
|
_aws: {
|
@@ -203,9 +207,7 @@ async function catalog(
|
|
203
207
|
|
204
208
|
// validate the new EMF JSON schema against AWS EMF JSON schema before adding to cache object
|
205
209
|
const isValid = validateEmf(newEmfLog);
|
206
|
-
//
|
207
|
-
// troubleshooting console.error in emf test
|
208
|
-
// if it fails validation throw error
|
210
|
+
// if the new EMF object fails validation, throw error and do not cache flawed object
|
209
211
|
if (!isValid) {
|
210
212
|
console.error(
|
211
213
|
"An error occurred during EMF validation: ",
|
@@ -215,7 +217,7 @@ async function catalog(
|
|
215
217
|
"Supplied/Proposed structured log does not comply with EMF schema"
|
216
218
|
);
|
217
219
|
}
|
218
|
-
// If
|
220
|
+
// If the new EMF object passes validation then add to cache object
|
219
221
|
cache[`${metricNamespace}${sortedDimensions}`] = newEmfLog;
|
220
222
|
}
|
221
223
|
|
@@ -227,7 +229,7 @@ async function catalog(
|
|
227
229
|
cache[Object.keys(cache)[i]]
|
228
230
|
);
|
229
231
|
}
|
230
|
-
//clear cache
|
232
|
+
//clear cache after logging all cached objects to Lambda
|
231
233
|
console.log("BEFORE:", cache);
|
232
234
|
for (var member in cache) delete cache[member];
|
233
235
|
console.log("AFTER:", cache);
|
package/package.json
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
{
|
2
2
|
"name": "cat-a-logs",
|
3
|
-
"version": "2.0.
|
3
|
+
"version": "2.0.3",
|
4
4
|
"description": "Create & send structured logs to AWS Cloudwatch logs",
|
5
5
|
"main": "index.js",
|
6
6
|
"exports":{
|
@@ -23,18 +23,7 @@
|
|
23
23
|
"jest": "^29.7.0",
|
24
24
|
"@types/jest": "^29.5.14",
|
25
25
|
"concurrently": "^9.0.1",
|
26
|
-
"cross-env": "^7.0.3"
|
27
|
-
|
28
|
-
"@babel/core": "^7.25.7",
|
29
|
-
"@babel/plugin-transform-runtime": "^7.25.7",
|
30
|
-
"@babel/preset-env": "^7.25.8",
|
31
|
-
"@babel/preset-react": "^7.25.7",
|
32
|
-
"@babel/runtime": "^7.25.7",
|
33
|
-
"babel-eslint": "^10.1.0",
|
34
|
-
"babel-loader": "^9.2.1",
|
35
|
-
"webpack": "^5.95.0",
|
36
|
-
"webpack-cli": "^5.1.4",
|
37
|
-
"webpack-dev-server": "^5.1.0"
|
26
|
+
"cross-env": "^7.0.3"
|
38
27
|
},
|
39
28
|
"dependencies": {
|
40
29
|
"@aws-lambda-powertools/logger": "^2.11.0",
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
package/tsconfig.json
CHANGED
@@ -59,7 +59,7 @@
|
|
59
59
|
"noEmit": true, /* Disable emitting files from a compilation. */
|
60
60
|
// "outFile": "./", /* Specify a file that bundles all outputs into one JavaScript file. If 'declaration' is true, also designates a file that bundles all .d.ts output. */
|
61
61
|
"outDir": "./dist", /* Specify an output folder for all emitted files. */
|
62
|
-
|
62
|
+
"removeComments": true, /* Disable emitting comments. */
|
63
63
|
// "importHelpers": true, /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
|
64
64
|
// "downlevelIteration": true, /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
|
65
65
|
// "sourceRoot": "", /* Specify the root path for debuggers to find the reference source code. */
|