fastapi-gae-logging 0.0.2__tar.gz → 0.0.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,157 @@
1
+ Metadata-Version: 2.1
2
+ Name: fastapi-gae-logging
3
+ Version: 0.0.4
4
+ Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
+ Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
+ Author: Chris Karvouniaris
7
+ Author-email: christos.karvouniaris247@gmail.com
8
+ License: MIT
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Operating System :: OS Independent
11
+ Requires-Python: >=3.7
12
+ Description-Content-Type: text/markdown
13
+ License-File: LICENSE
14
+ Requires-Dist: fastapi>=0.112.2
15
+ Requires-Dist: google-cloud-logging>=3.11.2
16
+
17
+ # fastapi-gae-logging
18
+ Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
19
+
20
+ ## What problem does this package solve? Why do we need this?
21
+ When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
22
+
23
+ - **Scattered Logs**: Logs generated during a single request lifecycle were scattered across different log entries, making it difficult to trace the entire flow of a request, especially when troubleshooting issues.
24
+
25
+ - **Log Severity Mismatch**: The severity level of logs was not properly propagated throughout the request lifecycle. This meant that if an error occurred at any point in the request, the earlier logs did not reflect the severity of the final outcome, making it harder to identify problematic requests.
26
+
27
+ - **Payload Logging Issues**: Capturing and logging request payloads was cumbersome, requiring extra logging in the handlers and extra deployments. This led to incomplete logs, making it harder to reproduce issues or analyze request content.
28
+
29
+ - **Inconsistent Log Structures**: The default logging setup lacked a consistent structure, which made it challenging to filter, search, and analyze logs in the Google Cloud Log Explorer.
30
+
31
+ ## So what does it do?
32
+ The `fastapi-gae-logging` module addresses these problems by:
33
+
34
+ - **Grouping Logs by Request**: All logs generated during a request's lifecycle are grouped together, allowing for a complete view of the request flow in the Google Cloud Log Explorer. This makes it much easier to trace and troubleshoot issues.
35
+
36
+ - **Log Level Propagation**: The maximum log level observed during a request's lifecycle is propagated, ensuring that logs associated with a failed request reflect the appropriate severity. This improves the accuracy and utility of log searches based on severity.
37
+
38
+ - **Structured Payload Logging**: Request payloads are captured and logged in a structured format, even for non-dictionary JSON payloads. This ensures that all relevant request data is available for analysis, improving the ability to diagnose issues.
39
+
40
+
41
+ ## Install
42
+ `pip install fastapi-gae-logging`
43
+
44
+ ## Features:
45
+
46
+ - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
47
+ - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
48
+ - **Request Payload Logging**:
49
+ - Logs the payload of a request as part of the parent log in request-grouped logs.
50
+ - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
51
+ - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
52
+
53
+
54
+ ## API
55
+
56
+ - `FastAPIGAELoggingHandler` class: A custom Cloud Logging Handler for FastAPI applications to use with the official google-cloud-logging library.
57
+
58
+
59
+ ## Example of usage
60
+
61
+ ```python
62
+ from fastapi import FastAPI
63
+ from fastapi.responses import JSONResponse
64
+ from fastapi.exceptions import HTTPException
65
+ import logging
66
+ import os
67
+
68
+ app = FastAPI()
69
+
70
+ if os.getenv('GAE_ENV', '').startswith('standard'):
71
+ import google.cloud.logging
72
+ from google.cloud.logging_v2.handlers import setup_logging
73
+ from fastapi_gae_logging import FastAPIGAELoggingHandler
74
+
75
+ client = google.cloud.logging.Client()
76
+ gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
77
+ setup_logging(handler=gae_log_handler)
78
+
79
+ logging.getLogger().setLevel(logging.DEBUG)
80
+
81
+ @app.get("/info")
82
+ def info():
83
+ logging.debug("this is a debug")
84
+ logging.info("this is an info")
85
+ return JSONResponse(
86
+ content={"message": "info"}
87
+ )
88
+
89
+ @app.get("/warning")
90
+ async def warning():
91
+ logging.debug("this is a debug")
92
+ logging.info("this is an info")
93
+ logging.warning("this is a warning")
94
+ return JSONResponse(
95
+ content={"message": "warning"}
96
+ )
97
+
98
+ @app.get("/error")
99
+ def error():
100
+ logging.debug("this is a debug")
101
+ logging.info("this is an info")
102
+ logging.warning("this is a warning")
103
+ logging.error("this is an error")
104
+ return JSONResponse(
105
+ content={"message": "error"}
106
+ )
107
+
108
+ @app.get("/exception")
109
+ def exception():
110
+ logging.debug("this is a debug")
111
+ logging.info("this is an info")
112
+ logging.error("this is an error")
113
+ raise ValueError("This is a value error")
114
+
115
+ @app.get("/http_exception")
116
+ def http_exception():
117
+ logging.debug("this is a debug")
118
+ logging.info("this is an info")
119
+ raise HTTPException(
120
+ status_code=404,
121
+ detail={"error": "Resource not found"}
122
+ )
123
+
124
+ @app.post("/post_payload")
125
+ def post_payload(payload: Any = Body(None)):
126
+ logging.debug("this is an debug")
127
+ logging.info(payload)
128
+ return JSONResponse(content={"mirror_response": payload}, status_code=200)
129
+
130
+ ```
131
+
132
+ ## How it looks in Google Cloud Log Explorer
133
+
134
+ ### Logger selection
135
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/logger_selection.jpg)
136
+
137
+ ### Groupped logs with propagated log severity to the parent log
138
+
139
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/groupped_logs.jpg)
140
+
141
+ ### Grouped logs in request with payload
142
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/request_with_payload.jpg)
143
+
144
+ ## Dependencies
145
+ This tool is built upon the following packages:
146
+
147
+ - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
148
+ - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
149
+
150
+
151
+ ## Implementation Concept
152
+ - **Middleware Integration**: A custom middleware integrates into FastAPI to intercept requests and log data after processing.The custom middleware is added to the FastAPI application during the initialization of the FastAPIGAELoggingHandler.
153
+ - **Context Management**: Uses context variables to manage request-specific data and metadata such as request payload, Google Cloud trace ID, start time of the incoming request and the maximum log level observed during the request lifecycle.
154
+ - **Log Interception**: A logging filter intercepts log records, injecting trace information and adjusting the maximum log level based on observed log severity.
155
+ - **Cloud Logging**: Utilizes Google Cloud Logging to group logs by request and propagate the maximum log level, enhancing observability and troubleshooting.
156
+ - **Structured Logging**: Parent log of the request-response lifecycle is structured and sent to Google Cloud Logging with additional context, such as the request method, URL, and user agent after the request has been processed and served.
157
+
@@ -0,0 +1,141 @@
1
+ # fastapi-gae-logging
2
+ Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
3
+
4
+ ## What problem does this package solve? Why do we need this?
5
+ When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
6
+
7
+ - **Scattered Logs**: Logs generated during a single request lifecycle were scattered across different log entries, making it difficult to trace the entire flow of a request, especially when troubleshooting issues.
8
+
9
+ - **Log Severity Mismatch**: The severity level of logs was not properly propagated throughout the request lifecycle. This meant that if an error occurred at any point in the request, the earlier logs did not reflect the severity of the final outcome, making it harder to identify problematic requests.
10
+
11
+ - **Payload Logging Issues**: Capturing and logging request payloads was cumbersome, requiring extra logging in the handlers and extra deployments. This led to incomplete logs, making it harder to reproduce issues or analyze request content.
12
+
13
+ - **Inconsistent Log Structures**: The default logging setup lacked a consistent structure, which made it challenging to filter, search, and analyze logs in the Google Cloud Log Explorer.
14
+
15
+ ## So what does it do?
16
+ The `fastapi-gae-logging` module addresses these problems by:
17
+
18
+ - **Grouping Logs by Request**: All logs generated during a request's lifecycle are grouped together, allowing for a complete view of the request flow in the Google Cloud Log Explorer. This makes it much easier to trace and troubleshoot issues.
19
+
20
+ - **Log Level Propagation**: The maximum log level observed during a request's lifecycle is propagated, ensuring that logs associated with a failed request reflect the appropriate severity. This improves the accuracy and utility of log searches based on severity.
21
+
22
+ - **Structured Payload Logging**: Request payloads are captured and logged in a structured format, even for non-dictionary JSON payloads. This ensures that all relevant request data is available for analysis, improving the ability to diagnose issues.
23
+
24
+
25
+ ## Install
26
+ `pip install fastapi-gae-logging`
27
+
28
+ ## Features:
29
+
30
+ - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
31
+ - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
32
+ - **Request Payload Logging**:
33
+ - Logs the payload of a request as part of the parent log in request-grouped logs.
34
+ - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
35
+ - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
36
+
37
+
38
+ ## API
39
+
40
+ - `FastAPIGAELoggingHandler` class: A custom Cloud Logging Handler for FastAPI applications to use with the official google-cloud-logging library.
41
+
42
+
43
+ ## Example of usage
44
+
45
+ ```python
46
+ from fastapi import FastAPI
47
+ from fastapi.responses import JSONResponse
48
+ from fastapi.exceptions import HTTPException
49
+ import logging
50
+ import os
51
+
52
+ app = FastAPI()
53
+
54
+ if os.getenv('GAE_ENV', '').startswith('standard'):
55
+ import google.cloud.logging
56
+ from google.cloud.logging_v2.handlers import setup_logging
57
+ from fastapi_gae_logging import FastAPIGAELoggingHandler
58
+
59
+ client = google.cloud.logging.Client()
60
+ gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
61
+ setup_logging(handler=gae_log_handler)
62
+
63
+ logging.getLogger().setLevel(logging.DEBUG)
64
+
65
+ @app.get("/info")
66
+ def info():
67
+ logging.debug("this is a debug")
68
+ logging.info("this is an info")
69
+ return JSONResponse(
70
+ content={"message": "info"}
71
+ )
72
+
73
+ @app.get("/warning")
74
+ async def warning():
75
+ logging.debug("this is a debug")
76
+ logging.info("this is an info")
77
+ logging.warning("this is a warning")
78
+ return JSONResponse(
79
+ content={"message": "warning"}
80
+ )
81
+
82
+ @app.get("/error")
83
+ def error():
84
+ logging.debug("this is a debug")
85
+ logging.info("this is an info")
86
+ logging.warning("this is a warning")
87
+ logging.error("this is an error")
88
+ return JSONResponse(
89
+ content={"message": "error"}
90
+ )
91
+
92
+ @app.get("/exception")
93
+ def exception():
94
+ logging.debug("this is a debug")
95
+ logging.info("this is an info")
96
+ logging.error("this is an error")
97
+ raise ValueError("This is a value error")
98
+
99
+ @app.get("/http_exception")
100
+ def http_exception():
101
+ logging.debug("this is a debug")
102
+ logging.info("this is an info")
103
+ raise HTTPException(
104
+ status_code=404,
105
+ detail={"error": "Resource not found"}
106
+ )
107
+
108
+ @app.post("/post_payload")
109
+ def post_payload(payload: Any = Body(None)):
110
+ logging.debug("this is an debug")
111
+ logging.info(payload)
112
+ return JSONResponse(content={"mirror_response": payload}, status_code=200)
113
+
114
+ ```
115
+
116
+ ## How it looks in Google Cloud Log Explorer
117
+
118
+ ### Logger selection
119
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/logger_selection.jpg)
120
+
121
+ ### Groupped logs with propagated log severity to the parent log
122
+
123
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/groupped_logs.jpg)
124
+
125
+ ### Grouped logs in request with payload
126
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/request_with_payload.jpg)
127
+
128
+ ## Dependencies
129
+ This tool is built upon the following packages:
130
+
131
+ - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
132
+ - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
133
+
134
+
135
+ ## Implementation Concept
136
+ - **Middleware Integration**: A custom middleware integrates into FastAPI to intercept requests and log data after processing.The custom middleware is added to the FastAPI application during the initialization of the FastAPIGAELoggingHandler.
137
+ - **Context Management**: Uses context variables to manage request-specific data and metadata such as request payload, Google Cloud trace ID, start time of the incoming request and the maximum log level observed during the request lifecycle.
138
+ - **Log Interception**: A logging filter intercepts log records, injecting trace information and adjusting the maximum log level based on observed log severity.
139
+ - **Cloud Logging**: Utilizes Google Cloud Logging to group logs by request and propagate the maximum log level, enhancing observability and troubleshooting.
140
+ - **Structured Logging**: Parent log of the request-response lifecycle is structured and sent to Google Cloud Logging with additional context, such as the request method, URL, and user agent after the request has been processed and served.
141
+
@@ -65,7 +65,7 @@ class GAERequestLogger:
65
65
  logging.CRITICAL: 'CRITICAL',
66
66
  }
67
67
 
68
- def __init__(self, logger: Logger, resource: Resource) -> None:
68
+ def __init__(self, logger: Logger, resource: Resource, log_payload: bool = True) -> None:
69
69
  """
70
70
  Initialize the GAERequestLogger.
71
71
 
@@ -75,6 +75,7 @@ class GAERequestLogger:
75
75
  """
76
76
  self.logger = logger
77
77
  self.resource = resource
78
+ self.log_payload = log_payload
78
79
 
79
80
  def _log_level_to_severity(self, log_level: int) -> str:
80
81
  """
@@ -117,16 +118,19 @@ class GAERequestLogger:
117
118
  }
118
119
 
119
120
  payload = {}
120
- if request.method in ('POST', 'PUT', 'PATCH', 'DELETE'):
121
+
122
+ if self.log_payload and request.method in {'POST', 'PUT', 'PATCH', 'DELETE'}:
121
123
  try:
122
124
  payload = await request.json()
123
125
  except json.JSONDecodeError:
124
- pass
125
-
126
- if not isinstance(payload, dict):
127
- payload = {
128
- f"{type(payload).__name__}_payload_wrapper": payload
129
- }
126
+ logging.warning("Failed to decode request payload as JSON, skipping logging.")
127
+ except Exception as e:
128
+ logging.error(f"Unexpected error while logging payload: {e}")
129
+ else:
130
+ if not isinstance(payload, dict):
131
+ payload = {
132
+ f"{type(payload).__name__}_payload_wrapper": payload
133
+ }
130
134
 
131
135
  self.logger.log_struct(
132
136
  info=payload,
@@ -202,6 +206,7 @@ class FastAPIGAELoggingHandler(CloudLoggingHandler):
202
206
  self,
203
207
  app: FastAPI,
204
208
  request_logger_name: Optional[str] = None,
209
+ log_payload: bool = True,
205
210
  *args, **kwargs
206
211
  ) -> None:
207
212
  """
@@ -225,7 +230,8 @@ class FastAPIGAELoggingHandler(CloudLoggingHandler):
225
230
  name=request_logger_name or f"{os.getenv('GOOGLE_CLOUD_PROJECT')}{self.REQUEST_LOGGER_SUFFIX}",
226
231
  resource=self.resource
227
232
  ),
228
- resource=self.resource
233
+ resource=self.resource,
234
+ log_payload=log_payload
229
235
  )
230
236
  )
231
237
  self.addFilter(LogInterceptor())
@@ -0,0 +1,157 @@
1
+ Metadata-Version: 2.1
2
+ Name: fastapi-gae-logging
3
+ Version: 0.0.4
4
+ Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
+ Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
+ Author: Chris Karvouniaris
7
+ Author-email: christos.karvouniaris247@gmail.com
8
+ License: MIT
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Operating System :: OS Independent
11
+ Requires-Python: >=3.7
12
+ Description-Content-Type: text/markdown
13
+ License-File: LICENSE
14
+ Requires-Dist: fastapi>=0.112.2
15
+ Requires-Dist: google-cloud-logging>=3.11.2
16
+
17
+ # fastapi-gae-logging
18
+ Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
19
+
20
+ ## What problem does this package solve? Why do we need this?
21
+ When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
22
+
23
+ - **Scattered Logs**: Logs generated during a single request lifecycle were scattered across different log entries, making it difficult to trace the entire flow of a request, especially when troubleshooting issues.
24
+
25
+ - **Log Severity Mismatch**: The severity level of logs was not properly propagated throughout the request lifecycle. This meant that if an error occurred at any point in the request, the earlier logs did not reflect the severity of the final outcome, making it harder to identify problematic requests.
26
+
27
+ - **Payload Logging Issues**: Capturing and logging request payloads was cumbersome, requiring extra logging in the handlers and extra deployments. This led to incomplete logs, making it harder to reproduce issues or analyze request content.
28
+
29
+ - **Inconsistent Log Structures**: The default logging setup lacked a consistent structure, which made it challenging to filter, search, and analyze logs in the Google Cloud Log Explorer.
30
+
31
+ ## So what does it do?
32
+ The `fastapi-gae-logging` module addresses these problems by:
33
+
34
+ - **Grouping Logs by Request**: All logs generated during a request's lifecycle are grouped together, allowing for a complete view of the request flow in the Google Cloud Log Explorer. This makes it much easier to trace and troubleshoot issues.
35
+
36
+ - **Log Level Propagation**: The maximum log level observed during a request's lifecycle is propagated, ensuring that logs associated with a failed request reflect the appropriate severity. This improves the accuracy and utility of log searches based on severity.
37
+
38
+ - **Structured Payload Logging**: Request payloads are captured and logged in a structured format, even for non-dictionary JSON payloads. This ensures that all relevant request data is available for analysis, improving the ability to diagnose issues.
39
+
40
+
41
+ ## Install
42
+ `pip install fastapi-gae-logging`
43
+
44
+ ## Features:
45
+
46
+ - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
47
+ - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
48
+ - **Request Payload Logging**:
49
+ - Logs the payload of a request as part of the parent log in request-grouped logs.
50
+ - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
51
+ - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
52
+
53
+
54
+ ## API
55
+
56
+ - `FastAPIGAELoggingHandler` class: A custom Cloud Logging Handler for FastAPI applications to use with the official google-cloud-logging library.
57
+
58
+
59
+ ## Example of usage
60
+
61
+ ```python
62
+ from fastapi import FastAPI
63
+ from fastapi.responses import JSONResponse
64
+ from fastapi.exceptions import HTTPException
65
+ import logging
66
+ import os
67
+
68
+ app = FastAPI()
69
+
70
+ if os.getenv('GAE_ENV', '').startswith('standard'):
71
+ import google.cloud.logging
72
+ from google.cloud.logging_v2.handlers import setup_logging
73
+ from fastapi_gae_logging import FastAPIGAELoggingHandler
74
+
75
+ client = google.cloud.logging.Client()
76
+ gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
77
+ setup_logging(handler=gae_log_handler)
78
+
79
+ logging.getLogger().setLevel(logging.DEBUG)
80
+
81
+ @app.get("/info")
82
+ def info():
83
+ logging.debug("this is a debug")
84
+ logging.info("this is an info")
85
+ return JSONResponse(
86
+ content={"message": "info"}
87
+ )
88
+
89
+ @app.get("/warning")
90
+ async def warning():
91
+ logging.debug("this is a debug")
92
+ logging.info("this is an info")
93
+ logging.warning("this is a warning")
94
+ return JSONResponse(
95
+ content={"message": "warning"}
96
+ )
97
+
98
+ @app.get("/error")
99
+ def error():
100
+ logging.debug("this is a debug")
101
+ logging.info("this is an info")
102
+ logging.warning("this is a warning")
103
+ logging.error("this is an error")
104
+ return JSONResponse(
105
+ content={"message": "error"}
106
+ )
107
+
108
+ @app.get("/exception")
109
+ def exception():
110
+ logging.debug("this is a debug")
111
+ logging.info("this is an info")
112
+ logging.error("this is an error")
113
+ raise ValueError("This is a value error")
114
+
115
+ @app.get("/http_exception")
116
+ def http_exception():
117
+ logging.debug("this is a debug")
118
+ logging.info("this is an info")
119
+ raise HTTPException(
120
+ status_code=404,
121
+ detail={"error": "Resource not found"}
122
+ )
123
+
124
+ @app.post("/post_payload")
125
+ def post_payload(payload: Any = Body(None)):
126
+ logging.debug("this is an debug")
127
+ logging.info(payload)
128
+ return JSONResponse(content={"mirror_response": payload}, status_code=200)
129
+
130
+ ```
131
+
132
+ ## How it looks in Google Cloud Log Explorer
133
+
134
+ ### Logger selection
135
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/logger_selection.jpg)
136
+
137
+ ### Groupped logs with propagated log severity to the parent log
138
+
139
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/groupped_logs.jpg)
140
+
141
+ ### Grouped logs in request with payload
142
+ ![alt text](https://github.com/chrisK824/fastapi-gae-logging/raw/main/request_with_payload.jpg)
143
+
144
+ ## Dependencies
145
+ This tool is built upon the following packages:
146
+
147
+ - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
148
+ - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
149
+
150
+
151
+ ## Implementation Concept
152
+ - **Middleware Integration**: A custom middleware integrates into FastAPI to intercept requests and log data after processing.The custom middleware is added to the FastAPI application during the initialization of the FastAPIGAELoggingHandler.
153
+ - **Context Management**: Uses context variables to manage request-specific data and metadata such as request payload, Google Cloud trace ID, start time of the incoming request and the maximum log level observed during the request lifecycle.
154
+ - **Log Interception**: A logging filter intercepts log records, injecting trace information and adjusting the maximum log level based on observed log severity.
155
+ - **Cloud Logging**: Utilizes Google Cloud Logging to group logs by request and propagate the maximum log level, enhancing observability and troubleshooting.
156
+ - **Structured Logging**: Parent log of the request-response lifecycle is structured and sent to Google Cloud Logging with additional context, such as the request method, URL, and user agent after the request has been processed and served.
157
+
@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
2
2
 
3
3
  setup(
4
4
  name="fastapi-gae-logging",
5
- version="0.0.2",
5
+ version="0.0.4",
6
6
  description="Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. \
7
7
  Groups logs coming from the same request lifecycle and propagates the maximum log level \
8
8
  throughout the request lifecycle using middleware and context management.",
@@ -1,116 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: fastapi-gae-logging
3
- Version: 0.0.2
4
- Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
- Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
- Author: Chris Karvouniaris
7
- Author-email: christos.karvouniaris247@gmail.com
8
- License: MIT
9
- Classifier: Programming Language :: Python :: 3
10
- Classifier: Operating System :: OS Independent
11
- Requires-Python: >=3.7
12
- Description-Content-Type: text/markdown
13
- License-File: LICENSE
14
- Requires-Dist: fastapi>=0.112.2
15
- Requires-Dist: google-cloud-logging>=3.11.2
16
-
17
- # fastapi-gae-logging
18
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine.
19
- Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
20
-
21
- ## Install
22
- `pip install fastapi-gae-logging`
23
-
24
- ## Features:
25
-
26
- - **Request Logs Grouping**: Groups logs coming from the same request lifecycle to ease out log analysis using the Google Cloud Log Explorer. Grouping logger name can be customised and it defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
27
- - **Request Maximum Log Level propagation**: Propagates the maximum log level throughout the request lifecycle to ease out log searching based on severity of an issue.
28
- - **Request Payload Logging** - More options to be exposed on that behaviour:
29
- - In case the request contains a payload, then this is logged into the parent log of request grouped logs
30
- - For now content type of payload is assumed to be a valid JSON, otherwise it is discarded. More to be supported later on.
31
- - The payload is logged as a dictionary using the `google-cloud-logging`'s method for `log_struct`. Since that method is logging only dictionaries, any JSON valid payload that is not a dictionary is wrapped into a dummy keyword to constuct one. The dummy keyword looks like that: `<original_type_of_payload>__payload_wrapper` The original or virtual dictionary is logged with `log_struct` along with the parent log of request grouped logs and ends up in field `jsonPayload`.
32
-
33
- ## API
34
- - Custom Cloud Logging Handler to use with official library `google-cloud-logging`: `FastAPIGAELoggingHandler`
35
-
36
-
37
- ## Example
38
-
39
- ```python
40
- from fastapi import FastAPI
41
- from fastapi.responses import JSONResponse
42
- from fastapi.exceptions import HTTPException
43
- import logging
44
- import os
45
-
46
-
47
- app = FastAPI()
48
-
49
- if os.getenv('GAE_ENV', '').startswith('standard'):
50
- import google.cloud.logging
51
- from google.cloud.logging_v2.handlers import setup_logging
52
- from fastapi_gae_logging import FastAPIGAELoggingHandler
53
-
54
- client = google.cloud.logging.Client()
55
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
56
- setup_logging(handler=gae_log_handler)
57
-
58
-
59
- logging.getLogger().setLevel(logging.DEBUG)
60
-
61
-
62
- @app.get("/info")
63
- def info():
64
- logging.debug("this is a debug")
65
- logging.info("this is an info")
66
- return JSONResponse(
67
- content={
68
- "message": "info"
69
- }
70
- )
71
-
72
-
73
- @app.get("/warning")
74
- async def warning():
75
- logging.debug("this is a debug")
76
- logging.info("this is an info")
77
- logging.warning("this is a warning")
78
- return JSONResponse(
79
- content={
80
- "message": "warning"
81
- }
82
- )
83
-
84
-
85
- @app.get("/error")
86
- def error():
87
- logging.debug("this is a debug")
88
- logging.info("this is an info")
89
- logging.warning("this is a warning")
90
- logging.error("this is an error")
91
- return JSONResponse(
92
- content={
93
- "message": "error"
94
- }
95
- )
96
-
97
-
98
- @app.get("/exception")
99
- def exception():
100
- logging.debug("this is a debug")
101
- logging.info("this is an info")
102
- logging.error("this is an error")
103
- raise ValueError("This is a value error")
104
-
105
-
106
- @app.get("/http_exception")
107
- def http_exception():
108
- logging.debug("this is an debug")
109
- logging.info("this is an info")
110
- raise HTTPException(
111
- status_code=404,
112
- detail={
113
- "error": "Resource not found"
114
- }
115
- )
116
- ```
@@ -1,100 +0,0 @@
1
- # fastapi-gae-logging
2
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine.
3
- Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
4
-
5
- ## Install
6
- `pip install fastapi-gae-logging`
7
-
8
- ## Features:
9
-
10
- - **Request Logs Grouping**: Groups logs coming from the same request lifecycle to ease out log analysis using the Google Cloud Log Explorer. Grouping logger name can be customised and it defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
11
- - **Request Maximum Log Level propagation**: Propagates the maximum log level throughout the request lifecycle to ease out log searching based on severity of an issue.
12
- - **Request Payload Logging** - More options to be exposed on that behaviour:
13
- - In case the request contains a payload, then this is logged into the parent log of request grouped logs
14
- - For now content type of payload is assumed to be a valid JSON, otherwise it is discarded. More to be supported later on.
15
- - The payload is logged as a dictionary using the `google-cloud-logging`'s method for `log_struct`. Since that method is logging only dictionaries, any JSON valid payload that is not a dictionary is wrapped into a dummy keyword to constuct one. The dummy keyword looks like that: `<original_type_of_payload>__payload_wrapper` The original or virtual dictionary is logged with `log_struct` along with the parent log of request grouped logs and ends up in field `jsonPayload`.
16
-
17
- ## API
18
- - Custom Cloud Logging Handler to use with official library `google-cloud-logging`: `FastAPIGAELoggingHandler`
19
-
20
-
21
- ## Example
22
-
23
- ```python
24
- from fastapi import FastAPI
25
- from fastapi.responses import JSONResponse
26
- from fastapi.exceptions import HTTPException
27
- import logging
28
- import os
29
-
30
-
31
- app = FastAPI()
32
-
33
- if os.getenv('GAE_ENV', '').startswith('standard'):
34
- import google.cloud.logging
35
- from google.cloud.logging_v2.handlers import setup_logging
36
- from fastapi_gae_logging import FastAPIGAELoggingHandler
37
-
38
- client = google.cloud.logging.Client()
39
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
40
- setup_logging(handler=gae_log_handler)
41
-
42
-
43
- logging.getLogger().setLevel(logging.DEBUG)
44
-
45
-
46
- @app.get("/info")
47
- def info():
48
- logging.debug("this is a debug")
49
- logging.info("this is an info")
50
- return JSONResponse(
51
- content={
52
- "message": "info"
53
- }
54
- )
55
-
56
-
57
- @app.get("/warning")
58
- async def warning():
59
- logging.debug("this is a debug")
60
- logging.info("this is an info")
61
- logging.warning("this is a warning")
62
- return JSONResponse(
63
- content={
64
- "message": "warning"
65
- }
66
- )
67
-
68
-
69
- @app.get("/error")
70
- def error():
71
- logging.debug("this is a debug")
72
- logging.info("this is an info")
73
- logging.warning("this is a warning")
74
- logging.error("this is an error")
75
- return JSONResponse(
76
- content={
77
- "message": "error"
78
- }
79
- )
80
-
81
-
82
- @app.get("/exception")
83
- def exception():
84
- logging.debug("this is a debug")
85
- logging.info("this is an info")
86
- logging.error("this is an error")
87
- raise ValueError("This is a value error")
88
-
89
-
90
- @app.get("/http_exception")
91
- def http_exception():
92
- logging.debug("this is an debug")
93
- logging.info("this is an info")
94
- raise HTTPException(
95
- status_code=404,
96
- detail={
97
- "error": "Resource not found"
98
- }
99
- )
100
- ```
@@ -1,116 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: fastapi-gae-logging
3
- Version: 0.0.2
4
- Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
- Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
- Author: Chris Karvouniaris
7
- Author-email: christos.karvouniaris247@gmail.com
8
- License: MIT
9
- Classifier: Programming Language :: Python :: 3
10
- Classifier: Operating System :: OS Independent
11
- Requires-Python: >=3.7
12
- Description-Content-Type: text/markdown
13
- License-File: LICENSE
14
- Requires-Dist: fastapi>=0.112.2
15
- Requires-Dist: google-cloud-logging>=3.11.2
16
-
17
- # fastapi-gae-logging
18
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine.
19
- Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
20
-
21
- ## Install
22
- `pip install fastapi-gae-logging`
23
-
24
- ## Features:
25
-
26
- - **Request Logs Grouping**: Groups logs coming from the same request lifecycle to ease out log analysis using the Google Cloud Log Explorer. Grouping logger name can be customised and it defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
27
- - **Request Maximum Log Level propagation**: Propagates the maximum log level throughout the request lifecycle to ease out log searching based on severity of an issue.
28
- - **Request Payload Logging** - More options to be exposed on that behaviour:
29
- - In case the request contains a payload, then this is logged into the parent log of request grouped logs
30
- - For now content type of payload is assumed to be a valid JSON, otherwise it is discarded. More to be supported later on.
31
- - The payload is logged as a dictionary using the `google-cloud-logging`'s method for `log_struct`. Since that method is logging only dictionaries, any JSON valid payload that is not a dictionary is wrapped into a dummy keyword to constuct one. The dummy keyword looks like that: `<original_type_of_payload>__payload_wrapper` The original or virtual dictionary is logged with `log_struct` along with the parent log of request grouped logs and ends up in field `jsonPayload`.
32
-
33
- ## API
34
- - Custom Cloud Logging Handler to use with official library `google-cloud-logging`: `FastAPIGAELoggingHandler`
35
-
36
-
37
- ## Example
38
-
39
- ```python
40
- from fastapi import FastAPI
41
- from fastapi.responses import JSONResponse
42
- from fastapi.exceptions import HTTPException
43
- import logging
44
- import os
45
-
46
-
47
- app = FastAPI()
48
-
49
- if os.getenv('GAE_ENV', '').startswith('standard'):
50
- import google.cloud.logging
51
- from google.cloud.logging_v2.handlers import setup_logging
52
- from fastapi_gae_logging import FastAPIGAELoggingHandler
53
-
54
- client = google.cloud.logging.Client()
55
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
56
- setup_logging(handler=gae_log_handler)
57
-
58
-
59
- logging.getLogger().setLevel(logging.DEBUG)
60
-
61
-
62
- @app.get("/info")
63
- def info():
64
- logging.debug("this is a debug")
65
- logging.info("this is an info")
66
- return JSONResponse(
67
- content={
68
- "message": "info"
69
- }
70
- )
71
-
72
-
73
- @app.get("/warning")
74
- async def warning():
75
- logging.debug("this is a debug")
76
- logging.info("this is an info")
77
- logging.warning("this is a warning")
78
- return JSONResponse(
79
- content={
80
- "message": "warning"
81
- }
82
- )
83
-
84
-
85
- @app.get("/error")
86
- def error():
87
- logging.debug("this is a debug")
88
- logging.info("this is an info")
89
- logging.warning("this is a warning")
90
- logging.error("this is an error")
91
- return JSONResponse(
92
- content={
93
- "message": "error"
94
- }
95
- )
96
-
97
-
98
- @app.get("/exception")
99
- def exception():
100
- logging.debug("this is a debug")
101
- logging.info("this is an info")
102
- logging.error("this is an error")
103
- raise ValueError("This is a value error")
104
-
105
-
106
- @app.get("/http_exception")
107
- def http_exception():
108
- logging.debug("this is an debug")
109
- logging.info("this is an info")
110
- raise HTTPException(
111
- status_code=404,
112
- detail={
113
- "error": "Resource not found"
114
- }
115
- )
116
- ```