cat-a-logs 2.0.3 → 2.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +56 -52
- package/package.json +1 -1
- package/snapshots/7.2.png +0 -0
- package/snapshots/step7.1.png +0 -0
package/README.md
CHANGED
|
@@ -1,28 +1,41 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="./snapshots/Catalog_art.png" width="200" />
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
# Welcome to Cat-A-Log!
|
|
2
|
-
This npm package
|
|
6
|
+
This npm package helps you integrate AWS CloudWatch with AWS Embedded Metric Format (EMF) Logs and publish them to Cloudwatch using AWS Lambda Powertools. EMF formatting will allow for chosen metrics to be automatically visualized in Cloudwatch metrics for centralized observability of your application KPIs. Read Our Medium article to learn more about the Cat-A-Log story:
|
|
7
|
+
<a href="https://medium.com/cat-a-log/adding-embedded-metric-formatting-to-aws-lambda-logs-for-simplified-debugging-ee388fdfd3db" target="_blank">Easily Automate Custom Metrics in CloudWatch with EMF in Lambda</a>
|
|
3
8
|
|
|
4
|
-
|
|
9
|
+
## Table of Contents
|
|
10
|
+
- [Cat-A-Log](#why-use-cat-a-log)
|
|
11
|
+
- [EMF](#about-embedded-metric-formatting-emf)
|
|
12
|
+
- [Instructions](#instructions)
|
|
13
|
+
- [How to Contribute](#open-source-contributions)
|
|
14
|
+
- [Contributors](#contributor-information)
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
<!-- <p align="center">
|
|
5
18
|
<img src="./snapshots/Catalog_art.png" width="200" />
|
|
6
|
-
</p>
|
|
19
|
+
</p> -->
|
|
7
20
|
|
|
21
|
+
## Why use Cat-A-Log?
|
|
22
|
+
Why use a washing machine when you can do them by hand? Because it saves you time and makes your job way easier! Leveraging AWS Lambda Powertools we can use the cat-a-log function to invoke and format logs into AWS Embedded Metric Format. By publishing these logs to AWS Cloudwatch, we are able to provide engineers with automatic metric visualization to make the process of debugging logs much more efficient. Cat-a-log utilizes a cache to make efficient work of sending logs to Cloudwatch.
|
|
8
23
|
|
|
9
24
|
## About Embedded Metric Formatting (EMF):
|
|
10
|
-
|
|
25
|
+
EMF is a JSON specification that enables CloudWatch Logs to automatically extract embedded metric values from structured log events. It simplifies real-time monitoring by reducing complexity and cost for applications needing custom metrics and structured logging. For more information please visit the following link:
|
|
11
26
|
<a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html" target="_blank">AWS Documentation on EMF Formatting</a>
|
|
12
27
|
|
|
13
|
-
## Why use Cat-A-Log?
|
|
14
|
-
Why use a washing machine when you can do them by hand? Because it saves you time and makes your job way easier! Leveraging AWS Lambda Powertools we can use the cat-a-log function to invoke and format logs into AWS Embedded Metric Format. By publishing these logs to AWS Cloudwatch, we are able to provide engineers with automatic metric visulaization to make the process of debugging logs much more efficient. Cat-a-log utilizies a cache to make effcient work of sending logs to Cloudwatch.
|
|
15
28
|
|
|
16
29
|
|
|
17
30
|
## Instructions
|
|
18
|
-
**
|
|
19
|
-
Your chosen
|
|
31
|
+
**Prerequisites:**
|
|
32
|
+
Your chosen Integrated Development Environment (i.e. VS Code) must already be connected to AWS Lambda. For more guidance on setting up AWS Lambda we recommend following this helpful tutorial from AWS: <a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html" target="_blank">Deploy Hello World Application with AWS SAM</a>
|
|
20
33
|
|
|
21
34
|
**Installation:**
|
|
22
|
-
1. Install our package using the command `npm install cat-a-logs` then import the function
|
|
35
|
+
1. Install our package using the command `npm install cat-a-logs` then import the function into your js file that connects to AWS Lambda `import { catalog } from "cat-a-logs/index.js";` Check out Cat-A-Log on npm using the attached link:
|
|
23
36
|
<a href="https://www.npmjs.com/package/cat-a-logs?activeTab=readme" target="_blank">Cat-A-Log</a>
|
|
24
37
|
|
|
25
|
-
2. Now enter your arguments into the catalog function!
|
|
38
|
+
2. Now enter your arguments into the catalog function! Let's go through each parameter one at a time and see what this looks like. First let's take a look at the function definition:
|
|
26
39
|
|
|
27
40
|
```
|
|
28
41
|
function catalog(
|
|
@@ -35,27 +48,27 @@ Your chosen Integated Development Environment (i.e. VS Code) must already be be
|
|
|
35
48
|
deploy: boolean = false)
|
|
36
49
|
```
|
|
37
50
|
|
|
38
|
-
- **trackedVariable**: This variable represents
|
|
51
|
+
- **trackedVariable**: This variable represents the numerical value (or an Array containing a maximum of 100 numerical values) of the metric that will appear under the category "Custom namespace" in Cloudwatch Metrics. This is AWS Cloudwatch>Metrics>All metrics>Custom namespaces(ex. CatALog)>Dimensions(ex. Server, functionVersion)
|
|
39
52
|
|
|
40
53
|
<p align="center">
|
|
41
54
|
<img src="./snapshots/trackedVariable.png" width="600" />
|
|
42
55
|
</p>
|
|
43
56
|
|
|
44
57
|
|
|
45
|
-
- **metricName**: This is a unique label of the tracked variable that will be reflected inside AWS Cloudwatch. Must be written as a `string
|
|
58
|
+
- **metricName**: This is a unique label of the tracked variable that will be reflected inside AWS Cloudwatch. Must be written as a `string`.
|
|
46
59
|
In the below image this corresponds to `Latency` --> AWS Cloudwatch>Metrics>All metrics>Custom namespaces
|
|
47
60
|
|
|
48
61
|
<p align="center">
|
|
49
62
|
<img src="./snapshots/metricName.png" width="600"/>
|
|
50
63
|
</p>
|
|
51
64
|
|
|
52
|
-
- **metricNamespace**: This will be your "Custom namespace" in AWS Cloudwatch>Metrics>All metrics>Custom namespaces. In below image this is represented by CatALog
|
|
65
|
+
- **metricNamespace**: This will be your "Custom namespace" in AWS Cloudwatch>Metrics>All metrics>Custom namespaces. In the below image this is represented by CatALog
|
|
53
66
|
|
|
54
67
|
<p align="center">
|
|
55
68
|
<img src="./snapshots/customNameSpace.png" width="600"/>
|
|
56
69
|
</p>
|
|
57
70
|
|
|
58
|
-
- **metricUnitLabel**:
|
|
71
|
+
- **metricUnitLabel**: The explicit unit that Cloudwatch uses for EMF Configuration. Please note - must be one of the following as a `string`:
|
|
59
72
|
- Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None
|
|
60
73
|
|
|
61
74
|
- To read more about Metric Datum see this <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html" target="_blank">link</a>
|
|
@@ -74,20 +87,32 @@ Your chosen Integated Development Environment (i.e. VS Code) must already be be
|
|
|
74
87
|
</p>
|
|
75
88
|
|
|
76
89
|
|
|
77
|
-
- **resolution**: This
|
|
78
|
-
- **deploy**:
|
|
90
|
+
- **resolution**: This parameter can only be set to the numericalthe numerical val.uA of 1 OR 60 , theically set to is setdefault value to 60. If you would like to learn more about High Resolution Metrics please follow the attached <a href= "https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics" target="_blank">link</a>
|
|
91
|
+
- **deploy**: Automatically set to false. The final catalog call you make has to switch deploy flag to true. Failure to do so will cause the cache to grow without bound and use up memory
|
|
79
92
|
|
|
80
|
-
3. Start Building your Embedded Metric Formatted Logs. Call catalog as many times as needed.
|
|
93
|
+
3. Start Building your Embedded Metric Formatted Logs. Call `catalog` as many times as needed.
|
|
81
94
|
<!-- You can also `console.log(cache)` at any time to see your EMF formatted logs being built in real time. -->
|
|
82
95
|
|
|
83
|
-
4. ON the very last function call - it is important to change the deploy parameter to `true`.
|
|
84
96
|
|
|
85
|
-
|
|
97
|
+
4. ON the very last function call - it is important to change the deploy parameter to `true`.
|
|
98
|
+
```
|
|
99
|
+
function catalog(
|
|
100
|
+
.
|
|
101
|
+
.
|
|
102
|
+
.
|
|
103
|
+
.
|
|
104
|
+
deploy: boolean = true)
|
|
105
|
+
```
|
|
86
106
|
|
|
87
|
-
|
|
107
|
+
5. Deploy your code with AWS SAM. This will place the file in AWS Lambda waiting for invocation. If you would like to learn more about deploying with SAM please follow the attached
|
|
108
|
+
<a href= "https://docs.aws.amazon.com/lambda/latest/dg/testing-functions.html" target="_blank">link</a>
|
|
109
|
+
6. Invoke your AWS Lambda Function
|
|
88
110
|
|
|
89
111
|
7. See your metrics and structured in CloudWatch!
|
|
90
|
-
|
|
112
|
+
<p align="center">
|
|
113
|
+
<img src="./snapshots/7.2.png" width="600"/>
|
|
114
|
+
</p>
|
|
115
|
+
|
|
91
116
|
## Open Source Contributions:
|
|
92
117
|
We are actively looking for contributors to our project! In order to get started we ask that you follow the below guidelines:
|
|
93
118
|
|
|
@@ -100,7 +125,7 @@ We are actively looking for contributors to our project! In order to get started
|
|
|
100
125
|
|
|
101
126
|
| AWS MicroService Support | Status |
|
|
102
127
|
|---------------------------------------------------------------------------------------|-----------|
|
|
103
|
-
|
|
|
128
|
+
| Lambda | ✅ |
|
|
104
129
|
| EC2 | ⏳ |
|
|
105
130
|
|
|
106
131
|
|
|
@@ -109,7 +134,7 @@ We are actively looking for contributors to our project! In order to get started
|
|
|
109
134
|
|---------------------------------------------------------------------------------------|-----------|
|
|
110
135
|
| TypeScript | ✅ |
|
|
111
136
|
| Embedded Metric Format Object Caching | ✅ |
|
|
112
|
-
|
|
|
137
|
+
| Winston | ⏳ |
|
|
113
138
|
| Adding front end for Cat-A-Log | 🙏🏻 |
|
|
114
139
|
|
|
115
140
|
|
|
@@ -118,7 +143,7 @@ We are actively looking for contributors to our project! In order to get started
|
|
|
118
143
|
- 🙏🏻 = Looking for contributors
|
|
119
144
|
|
|
120
145
|
## License Information:
|
|
121
|
-
|
|
146
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details
|
|
122
147
|
|
|
123
148
|
|
|
124
149
|
## Contributor Information:
|
|
@@ -149,43 +174,22 @@ Put License Information Here
|
|
|
149
174
|
<a href="https://github.com/HarrAwa">🐙</a>
|
|
150
175
|
</td>
|
|
151
176
|
<td align="center">
|
|
152
|
-
<img src="https://avatars.githubusercontent.com/u/
|
|
177
|
+
<img src="https://avatars.githubusercontent.com/u/106503739?v=4" width="140px;" alt=""/>
|
|
153
178
|
<br />
|
|
154
179
|
<sub><b>Curran Lee</b></sub>
|
|
155
180
|
<br />
|
|
156
|
-
<a href="https://www.linkedin.com/
|
|
157
|
-
<a href="https://github.com/
|
|
181
|
+
<a href="https://www.linkedin.com/in/curranjlee/">🖇️</a>
|
|
182
|
+
<a href="https://github.com/CJLee5">🐙</a>
|
|
158
183
|
<td align="center">
|
|
159
|
-
<img src="https://avatars.githubusercontent.com/u/
|
|
184
|
+
<img src="https://avatars.githubusercontent.com/u/142838412?v=4" width="140px;" alt=""/>
|
|
160
185
|
<br />
|
|
161
186
|
<sub><b>Jacob Alexander</b></sub>
|
|
162
187
|
<br />
|
|
163
|
-
<a href="https://www.linkedin.com/
|
|
164
|
-
<a href="https://github.com/
|
|
188
|
+
<a href="https://www.linkedin.com/in/jacoblanealexander/">🖇️</a>
|
|
189
|
+
<a href="https://github.com/jacob-jpg1">🐙</a>
|
|
165
190
|
</td>
|
|
166
191
|
</table>
|
|
167
192
|
|
|
168
193
|
|
|
169
194
|
- 🖇️ = LinkedIn
|
|
170
|
-
- 🐙 = Github
|
|
171
|
-
|
|
172
|
-
## Notes to Self:
|
|
173
|
-
**Structure of the files:**
|
|
174
|
-
|
|
175
|
-
`index.ts` is compiled to `index.js`. Important to compile `.ts` file to es6 js syntax using the `tsc —target es6 (filepath)` command
|
|
176
|
-
`app.mjs ` is a "pathway" to our lambda function. Here is where we will import catalog function and use it to involke our lambda function
|
|
177
|
-
|
|
178
|
-
index.ts lines 25-30 is checking to see if the value "level" || "message" || "sampling_rate" || "service" || "timestamp" ||"xray_trace_id"
|
|
179
|
-
|
|
180
|
-
logger.info gives you some information level is key and value is info
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
if you write name that it will overwrite the keys
|
|
184
|
-
|
|
185
|
-
**Tech Challenges**
|
|
186
|
-
Spent 3 days dealing with inconsistencies of ES6/CommonJS in our code before compiling .js in ES6
|
|
187
|
-
|
|
188
|
-
**To DO LIST ITEMS**
|
|
189
|
-
- How can the user visaulize the cache growing in real time?
|
|
190
|
-
- Creating more professional scrreenshots for the ReadMe - to replace the current ReadMe screenshots
|
|
191
|
-
- Add License Information
|
|
195
|
+
- 🐙 = Github
|
package/package.json
CHANGED
|
Binary file
|
|
Binary file
|