cat-a-logs 2.0.2 → 2.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +166 -89
- package/client/index.js +27 -74
- package/client/index.ts +26 -24
- package/package.json +1 -1
- package/snapshots/7.2.png +0 -0
- package/snapshots/Catalog_art.png +0 -0
- package/snapshots/customNameSpace.png +0 -0
- package/snapshots/keyDefined.png +0 -0
- package/snapshots/metricName.png +0 -0
- package/snapshots/step7.1.png +0 -0
- package/snapshots/trackedVariable.png +0 -0
- package/tsconfig.json +1 -1
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Open Source Labs Beta
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,118 +1,195 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="./snapshots/Catalog_art.png" width="200" />
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
# Welcome to Cat-A-Log!
|
|
2
|
-
This npm package
|
|
6
|
+
This npm package helps you integrate AWS CloudWatch with AWS Embedded Metric Format (EMF) Logs and publish them to Cloudwatch using AWS Lambda Powertools. EMF formatting will allow for chosen metrics to be automatically visualized in Cloudwatch metrics for centralized observability of your application KPIs. Read Our Medium article to learn more about the Cat-A-Log story:
|
|
7
|
+
<a href="https://medium.com/cat-a-log/adding-embedded-metric-formatting-to-aws-lambda-logs-for-simplified-debugging-ee388fdfd3db" target="_blank">Easily Automate Custom Metrics in CloudWatch with EMF in Lambda</a>
|
|
8
|
+
|
|
9
|
+
## Table of Contents
|
|
10
|
+
- [Cat-A-Log](#why-use-cat-a-log)
|
|
11
|
+
- [EMF](#about-embedded-metric-formatting-emf)
|
|
12
|
+
- [Instructions](#instructions)
|
|
13
|
+
- [How to Contribute](#open-source-contributions)
|
|
14
|
+
- [Contributors](#contributor-information)
|
|
3
15
|
|
|
4
|
-
|
|
16
|
+
|
|
17
|
+
<!-- <p align="center">
|
|
5
18
|
<img src="./snapshots/Catalog_art.png" width="200" />
|
|
6
|
-
</p>
|
|
19
|
+
</p> -->
|
|
7
20
|
|
|
21
|
+
## Why use Cat-A-Log?
|
|
22
|
+
Why use a washing machine when you can do them by hand? Because it saves you time and makes your job way easier! Leveraging AWS Lambda Powertools we can use the cat-a-log function to invoke and format logs into AWS Embedded Metric Format. By publishing these logs to AWS Cloudwatch, we are able to provide engineers with automatic metric visualization to make the process of debugging logs much more efficient. Cat-a-log utilizes a cache to make efficient work of sending logs to Cloudwatch.
|
|
8
23
|
|
|
9
24
|
## About Embedded Metric Formatting (EMF):
|
|
10
|
-
|
|
25
|
+
EMF is a JSON specification that enables CloudWatch Logs to automatically extract embedded metric values from structured log events. It simplifies real-time monitoring by reducing complexity and cost for applications needing custom metrics and structured logging. For more information please visit the following link:
|
|
11
26
|
<a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html" target="_blank">AWS Documentation on EMF Formatting</a>
|
|
12
27
|
|
|
13
|
-
## Why use Cat-A-Log?
|
|
14
|
-
Why use a washing machine when you can do them by hand? Because it saves you time and makes your job way easier! Leveraging AWS Lambda Powertools we can use the cat-a-log function to invoke and format logs into AWS Embedded Metric Format. By publishing these logs to AWS Cloudwatch, we are able to provide engineers with automatic metric visulaization to make the process of debugging logs much more efficient. Cat-a-log utilizies a cache to make effcient work of sending logs to Cloudwatch.
|
|
15
28
|
|
|
16
29
|
|
|
17
30
|
## Instructions
|
|
18
|
-
**
|
|
19
|
-
Your chosen
|
|
31
|
+
**Prerequisites:**
|
|
32
|
+
Your chosen Integrated Development Environment (i.e. VS Code) must already be connected to AWS Lambda. For more guidance on setting up AWS Lambda we recommend following this helpful tutorial from AWS: <a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html" target="_blank">Deploy Hello World Application with AWS SAM</a>
|
|
20
33
|
|
|
21
34
|
**Installation:**
|
|
22
|
-
1. Install our package using the command `npm install cat-a-logs` then import the function
|
|
35
|
+
1. Install our package using the command `npm install cat-a-logs` then import the function into your js file that connects to AWS Lambda `import { catalog } from "cat-a-logs/index.js";` Check out Cat-A-Log on npm using the attached link:
|
|
23
36
|
<a href="https://www.npmjs.com/package/cat-a-logs?activeTab=readme" target="_blank">Cat-A-Log</a>
|
|
24
37
|
|
|
25
|
-
2. Now enter your arguments into the catalog function!
|
|
38
|
+
2. Now enter your arguments into the catalog function! Let's go through each parameter one at a time and see what this looks like. First let's take a look at the function definition:
|
|
26
39
|
|
|
27
40
|
```
|
|
28
41
|
function catalog(
|
|
29
42
|
trackedVariable: number | Array<number>,
|
|
30
43
|
metricName: string,
|
|
31
|
-
metricNamespace: string,
|
|
44
|
+
metricNamespace: string = "CatALog-Default-Metrics",
|
|
32
45
|
metricUnitLabel: string = "None",
|
|
33
46
|
CustomerDefinedDimension: { [key: string]: string } = {},
|
|
34
47
|
resolution: 1 | 60 = 60,
|
|
35
48
|
deploy: boolean = false)
|
|
36
49
|
```
|
|
37
50
|
|
|
38
|
-
- **trackedVariable**: This variable represents a
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
51
|
+
- **trackedVariable**: This variable represents the numerical value (or an Array containing a maximum of 100 numerical values) of the metric that will appear under the category "Custom namespace" in Cloudwatch Metrics. This is AWS Cloudwatch>Metrics>All metrics>Custom namespaces(ex. CatALog)>Dimensions(ex. Server, functionVersion)
|
|
52
|
+
|
|
53
|
+
<p align="center">
|
|
54
|
+
<img src="./snapshots/trackedVariable.png" width="600" />
|
|
55
|
+
</p>
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
- **metricName**: This is a unique label of the tracked variable that will be reflected inside AWS Cloudwatch. Must be written as a `string`.
|
|
59
|
+
In the below image this corresponds to `Latency` --> AWS Cloudwatch>Metrics>All metrics>Custom namespaces
|
|
60
|
+
|
|
61
|
+
<p align="center">
|
|
62
|
+
<img src="./snapshots/metricName.png" width="600"/>
|
|
63
|
+
</p>
|
|
64
|
+
|
|
65
|
+
- **metricNamespace**: This will be your "Custom namespace" in AWS Cloudwatch>Metrics>All metrics>Custom namespaces. In the below image this is represented by CatALog
|
|
66
|
+
|
|
67
|
+
<p align="center">
|
|
68
|
+
<img src="./snapshots/customNameSpace.png" width="600"/>
|
|
69
|
+
</p>
|
|
70
|
+
|
|
71
|
+
- **metricUnitLabel**: The explicit unit that Cloudwatch uses for EMF Configuration. Please note - must be one of the following as a `string`:
|
|
42
72
|
- Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None
|
|
43
73
|
|
|
44
74
|
- To read more about Metric Datum see this <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html" target="_blank">link</a>
|
|
45
|
-
- **CustomerDefinedDimension**: This is an object - key
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
75
|
+
- **CustomerDefinedDimension**: This is an object - `{'functionVersion': '$LATEST', 'Server': 'Prod'}` functionVersion & Server is the dimension label/key - when you click on it see the value `$LATEST` and `Prod` is the value of the dimension
|
|
76
|
+
|
|
77
|
+
- The key will show up in AWS Cloudwatch as below:
|
|
78
|
+
|
|
79
|
+
<p align="center">
|
|
80
|
+
<img src="./snapshots/keyDefined.png" width="600"/>
|
|
81
|
+
</p>
|
|
82
|
+
|
|
83
|
+
- If the user clicks on the Server, functionVersion Dimension then you will see the value - in this example `$LATEST` & `Prod` reflected as below:
|
|
84
|
+
|
|
85
|
+
<p align="center">
|
|
86
|
+
<img src="./snapshots/metricName.png" width="600"/>
|
|
87
|
+
</p>
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
- **resolution**: This parameter can only be set to the numericalthe numerical val.uA of 1 OR 60 , theically set to is setdefault value to 60. If you would like to learn more about High Resolution Metrics please follow the attached <a href= "https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics" target="_blank">link</a>
|
|
91
|
+
- **deploy**: Automatically set to false. The final catalog call you make has to switch deploy flag to true. Failure to do so will cause the cache to grow without bound and use up memory
|
|
92
|
+
|
|
93
|
+
3. Start Building your Embedded Metric Formatted Logs. Call `catalog` as many times as needed.
|
|
94
|
+
<!-- You can also `console.log(cache)` at any time to see your EMF formatted logs being built in real time. -->
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
4. ON the very last function call - it is important to change the deploy parameter to `true`.
|
|
98
|
+
```
|
|
99
|
+
function catalog(
|
|
100
|
+
.
|
|
101
|
+
.
|
|
102
|
+
.
|
|
103
|
+
.
|
|
104
|
+
deploy: boolean = true)
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
5. Deploy your code with AWS SAM. This will place the file in AWS Lambda waiting for invocation. If you would like to learn more about deploying with SAM please follow the attached
|
|
108
|
+
<a href= "https://docs.aws.amazon.com/lambda/latest/dg/testing-functions.html" target="_blank">link</a>
|
|
109
|
+
6. Invoke your AWS Lambda Function
|
|
110
|
+
|
|
111
|
+
7. See your metrics and structured in CloudWatch!
|
|
112
|
+
<p align="center">
|
|
113
|
+
<img src="./snapshots/7.2.png" width="600"/>
|
|
114
|
+
</p>
|
|
115
|
+
|
|
116
|
+
## Open Source Contributions:
|
|
117
|
+
We are actively looking for contributors to our project! In order to get started we ask that you follow the below guidelines:
|
|
118
|
+
|
|
119
|
+
- Clone our Repository from GitHub <a href="https://github.com/oslabs-beta/cat-a-log" target="_blank">here</a>
|
|
120
|
+
- Make a Feature Branch
|
|
121
|
+
- Make your contributions
|
|
122
|
+
- Push to your Feature Branch in GitHub
|
|
123
|
+
- Make a Pull Request to our Repository!
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
| AWS MicroService Support | Status |
|
|
127
|
+
|---------------------------------------------------------------------------------------|-----------|
|
|
128
|
+
| Lambda | ✅ |
|
|
129
|
+
| EC2 | ⏳ |
|
|
130
|
+
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
|Feature |Status |
|
|
134
|
+
|---------------------------------------------------------------------------------------|-----------|
|
|
135
|
+
| TypeScript | ✅ |
|
|
136
|
+
| Embedded Metric Format Object Caching | ✅ |
|
|
137
|
+
| Winston | ⏳ |
|
|
138
|
+
| Adding front end for Cat-A-Log | 🙏🏻 |
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
- ✅ = Ready to use
|
|
142
|
+
- ⏳ = In progress
|
|
143
|
+
- 🙏🏻 = Looking for contributors
|
|
144
|
+
|
|
145
|
+
## License Information:
|
|
146
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
## Contributor Information:
|
|
150
|
+
<table>
|
|
151
|
+
<tr>
|
|
152
|
+
<td align="center">
|
|
153
|
+
<img src="https://avatars.githubusercontent.com/u/161962009?v=4" width="140px;" alt=""/>
|
|
154
|
+
<br />
|
|
155
|
+
<sub><b>Clara Regula</b></sub>
|
|
156
|
+
<br />
|
|
157
|
+
<a href="http://www.linkedin.com/in/clara-regula">🖇️</a>
|
|
158
|
+
<a href="https://github.com/clararegula">🐙</a>
|
|
159
|
+
</td>
|
|
160
|
+
<td align="center">
|
|
161
|
+
<img src="https://avatars.githubusercontent.com/u/170294267?v=4" width="140px;" alt=""/>
|
|
162
|
+
<br />
|
|
163
|
+
<sub><b>Brian Anderson</b></sub>
|
|
164
|
+
<br />
|
|
165
|
+
<a href="https://www.linkedin.com/in/brian-anderson-24370630/">🖇️</a>
|
|
166
|
+
<a href="https://github.com/brianmichaelanderson">🐙</a>
|
|
167
|
+
</td>
|
|
168
|
+
<td align="center">
|
|
169
|
+
<img src="https://avatars.githubusercontent.com/u/167483334?v=4" width="140px;" alt=""/>
|
|
170
|
+
<br />
|
|
171
|
+
<sub><b>Harris Awan</b></sub>
|
|
172
|
+
<br />
|
|
173
|
+
<a href="http://www.linkedin.com/in/harrawan123/">🖇️</a>
|
|
174
|
+
<a href="https://github.com/HarrAwa">🐙</a>
|
|
175
|
+
</td>
|
|
176
|
+
<td align="center">
|
|
177
|
+
<img src="https://avatars.githubusercontent.com/u/106503739?v=4" width="140px;" alt=""/>
|
|
178
|
+
<br />
|
|
179
|
+
<sub><b>Curran Lee</b></sub>
|
|
180
|
+
<br />
|
|
181
|
+
<a href="https://www.linkedin.com/in/curranjlee/">🖇️</a>
|
|
182
|
+
<a href="https://github.com/CJLee5">🐙</a>
|
|
183
|
+
<td align="center">
|
|
184
|
+
<img src="https://avatars.githubusercontent.com/u/142838412?v=4" width="140px;" alt=""/>
|
|
185
|
+
<br />
|
|
186
|
+
<sub><b>Jacob Alexander</b></sub>
|
|
187
|
+
<br />
|
|
188
|
+
<a href="https://www.linkedin.com/in/jacoblanealexander/">🖇️</a>
|
|
189
|
+
<a href="https://github.com/jacob-jpg1">🐙</a>
|
|
190
|
+
</td>
|
|
191
|
+
</table>
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
- 🖇️ = LinkedIn
|
|
195
|
+
- 🐙 = Github
|
package/client/index.js
CHANGED
|
@@ -11,31 +11,39 @@ import { Logger } from "@aws-lambda-powertools/logger";
|
|
|
11
11
|
import { Ajv } from "ajv";
|
|
12
12
|
//cache entries are structured thusly: 'Namespace + Dimensions(Alphabetically)': EMFObject
|
|
13
13
|
const cache = {};
|
|
14
|
-
//
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
//let latency = 300; (Example metric to track)
|
|
15
|
+
//Example for in-line use of Cat-a-log w/maximum arguments: catalog(latency, "Latency" , "lambda-function-metrics", "Milliseconds", {'functionVersion': '$LATEST', 'Server': 'Prod'}, 60, deploy);
|
|
16
|
+
//Example for in-line use of Cat-a-log w/minimum arguments: catalog(latency, "Latency");
|
|
17
|
+
function catalog(trackedVariable_1, metricName_1) {
|
|
18
|
+
return __awaiter(this, arguments, void 0, function* (trackedVariable, metricName, metricNamespace = "CatALog-Default-Metrics", metricUnitLabel = "None", CustomerDefinedDimension = {}, resolution = 60, deploy = false) {
|
|
17
19
|
//Check for any errors & validate inputs based on documentations
|
|
18
20
|
if (!cache)
|
|
19
21
|
throw new Error("cache is not found, please import cache from cat-a-log");
|
|
20
|
-
|
|
22
|
+
//check if any provided dimension names or metric names conflict with native logger keys.
|
|
21
23
|
const badKeys = ["level", "message", "sampling_rate", "service", "timestamp", "xray_trace_id"];
|
|
22
24
|
const yourKeys = Object.keys(CustomerDefinedDimension).concat([metricName.toLowerCase()]);
|
|
23
25
|
for (let i = 0; i < yourKeys.length; i++) {
|
|
24
26
|
if (badKeys.includes(yourKeys[i])) {
|
|
27
|
+
//if a dimension name or metric name conflicts with native logger keys, throw error
|
|
25
28
|
throw new Error("metricName, or Dimension names cannot be the same as these native logger keys: level || message || sampling_rate || service || timestamp || xray_trace_id");
|
|
26
29
|
}
|
|
27
30
|
}
|
|
31
|
+
//EMF specification catch: if tracked variable is an array with a length greater than 100, throw error and do not log
|
|
28
32
|
if (Array.isArray(trackedVariable)) {
|
|
29
33
|
if (trackedVariable.length > 100)
|
|
30
34
|
throw new Error("metric value cannot have more than 100 elements");
|
|
31
35
|
}
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
36
|
+
//EMF specification catch: make sure provided dimension object does not have more than 30 entries
|
|
37
|
+
// if (Object.keys(CustomerDefinedDimension).length > 30) {
|
|
38
|
+
// throw new Error(
|
|
39
|
+
// "EMF has a limit of 30 user defined dimension keys per log"
|
|
40
|
+
// );
|
|
41
|
+
// }
|
|
42
|
+
//Create new instance of Logger to use in function
|
|
35
43
|
const logger = new Logger({ serviceName: "serverlessAirline" });
|
|
36
|
-
// Ajv instance
|
|
44
|
+
//Set up Ajv instance for JSON validation
|
|
37
45
|
const ajv = new Ajv();
|
|
38
|
-
// from AWS: EMF schema to test/validate against
|
|
46
|
+
// from AWS: EMF schema to test/validate against with Ajv
|
|
39
47
|
const emfSchema = {
|
|
40
48
|
type: "object",
|
|
41
49
|
title: "Root Node",
|
|
@@ -136,29 +144,29 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
|
136
144
|
},
|
|
137
145
|
},
|
|
138
146
|
};
|
|
147
|
+
//usable instance of the validation JSON
|
|
139
148
|
const validateEmf = ajv.compile(emfSchema);
|
|
140
|
-
//sort customerDimensions key values in alphabetical order
|
|
149
|
+
//sort customerDimensions key values in alphabetical order. We will use this to keep the keys in our cache consistant. Since the order of the dimensions do not change where the metrics are stored
|
|
141
150
|
const sortedDimensions = {};
|
|
142
151
|
for (let i = 0; i < Object.keys(CustomerDefinedDimension).sort().length; i++) {
|
|
143
152
|
sortedDimensions[Object.keys(CustomerDefinedDimension).sort()[i]] =
|
|
144
153
|
CustomerDefinedDimension[Object.keys(CustomerDefinedDimension).sort()[i]];
|
|
145
154
|
}
|
|
146
|
-
//if Object with Namespace and Dimensions already exists in
|
|
155
|
+
//Check if Object with Namespace and Dimensions already exists in cache
|
|
147
156
|
let check = cache[`${metricNamespace}${sortedDimensions}`];
|
|
148
157
|
if (check != undefined) {
|
|
149
|
-
//push the metrics object to Metrics array
|
|
158
|
+
//if the Namespace and Dimensions exist, push the metrics object to Metrics array
|
|
150
159
|
cache[`${metricNamespace}${sortedDimensions}`]["_aws"]["CloudWatchMetrics"][0]["Metrics"].push({
|
|
151
160
|
Name: metricName,
|
|
152
161
|
Unit: metricUnitLabel,
|
|
153
162
|
StorageResolution: resolution,
|
|
154
163
|
});
|
|
155
|
-
//add key value to Log
|
|
164
|
+
//add key value to root of existing structured Log
|
|
156
165
|
check[`${metricName}`] = trackedVariable;
|
|
157
166
|
}
|
|
158
167
|
else {
|
|
159
|
-
//
|
|
160
|
-
//
|
|
161
|
-
// NameSpace & Dimensions for EMF part don't exist yet. Initialize variable to capture EMF/aws key:value pair
|
|
168
|
+
//create new Structured Log and add it to cachedStructuredLogs
|
|
169
|
+
//If NameSpace & Dimensions for EMF part don't exist yet, initialize variable to capture EMF/aws key:value pair
|
|
162
170
|
const newEmfLog = Object.assign({
|
|
163
171
|
_aws: {
|
|
164
172
|
Timestamp: Date.now(),
|
|
@@ -182,16 +190,12 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
|
182
190
|
console.log("index.ts - Unit value before validation: ", newEmfLog._aws.CloudWatchMetrics[0].Metrics[0].Unit);
|
|
183
191
|
// validate the new EMF JSON schema against AWS EMF JSON schema before adding to cache object
|
|
184
192
|
const isValid = validateEmf(newEmfLog);
|
|
185
|
-
//
|
|
186
|
-
// console.log('index.ts - Validation result: ', isValid);
|
|
187
|
-
// troubleshooting console.error in emf test
|
|
188
|
-
// console.log('index.ts - Validation errors: ', validateEmf.errors);
|
|
189
|
-
// if it fails validation throw error
|
|
193
|
+
// if the new EMF object fails validation, throw error and do not cache flawed object
|
|
190
194
|
if (!isValid) {
|
|
191
195
|
console.error("An error occurred during EMF validation: ", validateEmf.errors);
|
|
192
196
|
throw new Error("Supplied/Proposed structured log does not comply with EMF schema");
|
|
193
197
|
}
|
|
194
|
-
// If
|
|
198
|
+
// If the new EMF object passes validation then add to cache object
|
|
195
199
|
cache[`${metricNamespace}${sortedDimensions}`] = newEmfLog;
|
|
196
200
|
}
|
|
197
201
|
if (deploy) {
|
|
@@ -199,7 +203,7 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
|
199
203
|
for (let i = 0; i < Object.keys(cache).length; i++) {
|
|
200
204
|
logger.info(`Your EMF compliant Structured Metrics Log ${i + 1}`, cache[Object.keys(cache)[i]]);
|
|
201
205
|
}
|
|
202
|
-
//clear cache
|
|
206
|
+
//clear cache after logging all cached objects to Lambda
|
|
203
207
|
console.log("BEFORE:", cache);
|
|
204
208
|
for (var member in cache)
|
|
205
209
|
delete cache[member];
|
|
@@ -208,54 +212,3 @@ function catalog(trackedVariable_1, metricName_1, metricNamespace_1) {
|
|
|
208
212
|
});
|
|
209
213
|
}
|
|
210
214
|
export { cache, catalog };
|
|
211
|
-
/*Current Working logger invocation
|
|
212
|
-
logger.info("Your EMF compliant Structured Metrics Log",
|
|
213
|
-
Object.assign({
|
|
214
|
-
_aws: {
|
|
215
|
-
Timestamp: Date.now(),
|
|
216
|
-
CloudWatchMetrics: [
|
|
217
|
-
{
|
|
218
|
-
Namespace: metricNamespace,
|
|
219
|
-
Dimensions: [Object.keys(CustomerDefinedDimension)],
|
|
220
|
-
Metrics: [
|
|
221
|
-
{
|
|
222
|
-
Name: metricName,
|
|
223
|
-
Unit: metricUnitLabel,
|
|
224
|
-
StorageResolution: resolution,
|
|
225
|
-
}
|
|
226
|
-
]
|
|
227
|
-
}
|
|
228
|
-
]
|
|
229
|
-
},
|
|
230
|
-
[`${metricName}`]: trackedVariable,
|
|
231
|
-
},
|
|
232
|
-
CustomerDefinedDimension
|
|
233
|
-
)
|
|
234
|
-
|
|
235
|
-
)
|
|
236
|
-
*/
|
|
237
|
-
//Old handler function
|
|
238
|
-
// export const handler = async (_event, _context): Promise<void> => {
|
|
239
|
-
// const testObj = {
|
|
240
|
-
// testguy: "hi",
|
|
241
|
-
// fool: 42,
|
|
242
|
-
// functionVersion: "$LATEST",
|
|
243
|
-
// _aws: {
|
|
244
|
-
// Timestamp: Date.now(),
|
|
245
|
-
// CloudWatchMetrics: [
|
|
246
|
-
// {
|
|
247
|
-
// Namespace: "lambda-function-metrics",
|
|
248
|
-
// Dimensions: [["functionVersion"]],
|
|
249
|
-
// Metrics: [
|
|
250
|
-
// {
|
|
251
|
-
// Name: "fool",
|
|
252
|
-
// Unit: "Milliseconds",
|
|
253
|
-
// StorageResolution: 60
|
|
254
|
-
// }
|
|
255
|
-
// ]
|
|
256
|
-
// }
|
|
257
|
-
// ]
|
|
258
|
-
// },
|
|
259
|
-
// }
|
|
260
|
-
// logger.info(JSON.stringify(testObj));
|
|
261
|
-
// };
|
package/client/index.ts
CHANGED
|
@@ -3,11 +3,13 @@ import { Ajv } from "ajv";
|
|
|
3
3
|
|
|
4
4
|
//cache entries are structured thusly: 'Namespace + Dimensions(Alphabetically)': EMFObject
|
|
5
5
|
const cache: { [key: string]: any } = {};
|
|
6
|
-
//
|
|
6
|
+
//let latency = 300; (Example metric to track)
|
|
7
|
+
//Example for in-line use of Cat-a-log w/maximum arguments: catalog(latency, "Latency" , "lambda-function-metrics", "Milliseconds", {'functionVersion': '$LATEST', 'Server': 'Prod'}, 60, deploy);
|
|
8
|
+
//Example for in-line use of Cat-a-log w/minimum arguments: catalog(latency, "Latency");
|
|
7
9
|
async function catalog(
|
|
8
10
|
trackedVariable: number | Array<number>,
|
|
9
11
|
metricName: string,
|
|
10
|
-
metricNamespace: string,
|
|
12
|
+
metricNamespace: string = "CatALog-Default-Metrics",
|
|
11
13
|
metricUnitLabel: string = "None",
|
|
12
14
|
CustomerDefinedDimension: { [key: string]: string } = {},
|
|
13
15
|
resolution: 1 | 60 = 60,
|
|
@@ -16,11 +18,12 @@ async function catalog(
|
|
|
16
18
|
//Check for any errors & validate inputs based on documentations
|
|
17
19
|
if (!cache)
|
|
18
20
|
throw new Error("cache is not found, please import cache from cat-a-log");
|
|
19
|
-
|
|
21
|
+
//check if any provided dimension names or metric names conflict with native logger keys.
|
|
20
22
|
const badKeys = ["level", "message", "sampling_rate", "service", "timestamp", "xray_trace_id"];
|
|
21
23
|
const yourKeys = Object.keys(CustomerDefinedDimension).concat([metricName.toLowerCase()]);
|
|
22
24
|
for(let i = 0; i < yourKeys.length; i++){
|
|
23
25
|
if(badKeys.includes(yourKeys[i])){
|
|
26
|
+
//if a dimension name or metric name conflicts with native logger keys, throw error
|
|
24
27
|
throw new Error(
|
|
25
28
|
"metricName, or Dimension names cannot be the same as these native logger keys: level || message || sampling_rate || service || timestamp || xray_trace_id"
|
|
26
29
|
);
|
|
@@ -28,20 +31,22 @@ async function catalog(
|
|
|
28
31
|
}
|
|
29
32
|
|
|
30
33
|
|
|
31
|
-
|
|
34
|
+
//EMF specification catch: if tracked variable is an array with a length greater than 100, throw error and do not log
|
|
32
35
|
if (Array.isArray(trackedVariable)) {
|
|
33
36
|
if (trackedVariable.length > 100)
|
|
34
37
|
throw new Error("metric value cannot have more than 100 elements");
|
|
35
38
|
}
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
39
|
+
//EMF specification catch: make sure provided dimension object does not have more than 30 entries
|
|
40
|
+
// if (Object.keys(CustomerDefinedDimension).length > 30) {
|
|
41
|
+
// throw new Error(
|
|
42
|
+
// "EMF has a limit of 30 user defined dimension keys per log"
|
|
43
|
+
// );
|
|
44
|
+
// }
|
|
45
|
+
//Create new instance of Logger to use in function
|
|
41
46
|
const logger = new Logger({ serviceName: "serverlessAirline" });
|
|
42
|
-
// Ajv instance
|
|
47
|
+
//Set up Ajv instance for JSON validation
|
|
43
48
|
const ajv = new Ajv();
|
|
44
|
-
// from AWS: EMF schema to test/validate against
|
|
49
|
+
// from AWS: EMF schema to test/validate against with Ajv
|
|
45
50
|
const emfSchema = {
|
|
46
51
|
type: "object",
|
|
47
52
|
title: "Root Node",
|
|
@@ -143,9 +148,9 @@ async function catalog(
|
|
|
143
148
|
},
|
|
144
149
|
},
|
|
145
150
|
};
|
|
146
|
-
|
|
151
|
+
//usable instance of the validation JSON
|
|
147
152
|
const validateEmf = ajv.compile(emfSchema);
|
|
148
|
-
//sort customerDimensions key values in alphabetical order
|
|
153
|
+
//sort customerDimensions key values in alphabetical order. We will use this to keep the keys in our cache consistant. Since the order of the dimensions do not change where the metrics are stored
|
|
149
154
|
const sortedDimensions: { [key: string]: string } = {};
|
|
150
155
|
for (
|
|
151
156
|
let i = 0;
|
|
@@ -155,10 +160,10 @@ async function catalog(
|
|
|
155
160
|
sortedDimensions[Object.keys(CustomerDefinedDimension).sort()[i]] =
|
|
156
161
|
CustomerDefinedDimension[Object.keys(CustomerDefinedDimension).sort()[i]];
|
|
157
162
|
}
|
|
158
|
-
//if Object with Namespace and Dimensions already exists in
|
|
163
|
+
//Check if Object with Namespace and Dimensions already exists in cache
|
|
159
164
|
let check = cache[`${metricNamespace}${sortedDimensions}`];
|
|
160
165
|
if (check != undefined) {
|
|
161
|
-
//push the metrics object to Metrics array
|
|
166
|
+
//if the Namespace and Dimensions exist, push the metrics object to Metrics array
|
|
162
167
|
cache[`${metricNamespace}${sortedDimensions}`]["_aws"][
|
|
163
168
|
"CloudWatchMetrics"
|
|
164
169
|
][0]["Metrics"].push({
|
|
@@ -166,12 +171,11 @@ async function catalog(
|
|
|
166
171
|
Unit: metricUnitLabel,
|
|
167
172
|
StorageResolution: resolution,
|
|
168
173
|
});
|
|
169
|
-
//add key value to Log
|
|
174
|
+
//add key value to root of existing structured Log
|
|
170
175
|
check[`${metricName}`] = trackedVariable;
|
|
171
176
|
} else {
|
|
172
|
-
//
|
|
173
|
-
//
|
|
174
|
-
// NameSpace & Dimensions for EMF part don't exist yet. Initialize variable to capture EMF/aws key:value pair
|
|
177
|
+
//create new Structured Log and add it to cachedStructuredLogs
|
|
178
|
+
//If NameSpace & Dimensions for EMF part don't exist yet, initialize variable to capture EMF/aws key:value pair
|
|
175
179
|
const newEmfLog = Object.assign(
|
|
176
180
|
{
|
|
177
181
|
_aws: {
|
|
@@ -203,9 +207,7 @@ async function catalog(
|
|
|
203
207
|
|
|
204
208
|
// validate the new EMF JSON schema against AWS EMF JSON schema before adding to cache object
|
|
205
209
|
const isValid = validateEmf(newEmfLog);
|
|
206
|
-
//
|
|
207
|
-
// troubleshooting console.error in emf test
|
|
208
|
-
// if it fails validation throw error
|
|
210
|
+
// if the new EMF object fails validation, throw error and do not cache flawed object
|
|
209
211
|
if (!isValid) {
|
|
210
212
|
console.error(
|
|
211
213
|
"An error occurred during EMF validation: ",
|
|
@@ -215,7 +217,7 @@ async function catalog(
|
|
|
215
217
|
"Supplied/Proposed structured log does not comply with EMF schema"
|
|
216
218
|
);
|
|
217
219
|
}
|
|
218
|
-
// If
|
|
220
|
+
// If the new EMF object passes validation then add to cache object
|
|
219
221
|
cache[`${metricNamespace}${sortedDimensions}`] = newEmfLog;
|
|
220
222
|
}
|
|
221
223
|
|
|
@@ -227,7 +229,7 @@ async function catalog(
|
|
|
227
229
|
cache[Object.keys(cache)[i]]
|
|
228
230
|
);
|
|
229
231
|
}
|
|
230
|
-
//clear cache
|
|
232
|
+
//clear cache after logging all cached objects to Lambda
|
|
231
233
|
console.log("BEFORE:", cache);
|
|
232
234
|
for (var member in cache) delete cache[member];
|
|
233
235
|
console.log("AFTER:", cache);
|
package/package.json
CHANGED
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/tsconfig.json
CHANGED
|
@@ -59,7 +59,7 @@
|
|
|
59
59
|
"noEmit": true, /* Disable emitting files from a compilation. */
|
|
60
60
|
// "outFile": "./", /* Specify a file that bundles all outputs into one JavaScript file. If 'declaration' is true, also designates a file that bundles all .d.ts output. */
|
|
61
61
|
"outDir": "./dist", /* Specify an output folder for all emitted files. */
|
|
62
|
-
|
|
62
|
+
"removeComments": true, /* Disable emitting comments. */
|
|
63
63
|
// "importHelpers": true, /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
|
|
64
64
|
// "downlevelIteration": true, /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
|
|
65
65
|
// "sourceRoot": "", /* Specify the root path for debuggers to find the reference source code. */
|