fastapi-gae-logging 0.0.7__tar.gz → 0.0.9__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: fastapi-gae-logging
3
- Version: 0.0.7
4
- Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
3
+ Version: 0.0.9
4
+ Summary: Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
5
  Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
6
  Author: Chris Karvouniaris
7
7
  Author-email: christos.karvouniaris247@gmail.com
@@ -11,11 +11,11 @@ Classifier: Operating System :: OS Independent
11
11
  Requires-Python: >=3.8
12
12
  Description-Content-Type: text/markdown
13
13
  License-File: LICENSE
14
- Requires-Dist: fastapi
14
+ Requires-Dist: starlette
15
15
  Requires-Dist: google-cloud-logging
16
16
 
17
17
  # fastapi-gae-logging
18
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
18
+ Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
19
19
 
20
20
  ## What problem does this package solve? Why do we need this?
21
21
  When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
@@ -45,11 +45,12 @@ The `fastapi-gae-logging` module addresses these problems by:
45
45
 
46
46
  - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
47
47
  - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
48
- - **Request Payload Logging**:
49
- - Logs the payload of a request as part of the parent log in request-grouped logs.
50
- - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
51
- - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
52
-
48
+ - **Optional incoming request logging**: Opt in/out to log headers and payload of incoming requests into the `jsonPayload` field of the parent log.
49
+ - **Optional request headers logging**: Defaults to True. Headers dict lands into field `request_headers` in the `jsonPayload` of parent log.
50
+ - **Request Payload Logging**: Defaults to True. Incoming payload parsed lands into field `request_payload` in the `jsonPayload` of parent log. Parsing is based on content type with capability to override. Currenty embedded parsers for:
51
+ - `application/json`
52
+ - `application/x-www-form-urlencoded`
53
+ - `text/plain`
53
54
 
54
55
  ## API
55
56
 
@@ -57,20 +58,25 @@ The `fastapi-gae-logging` module addresses these problems by:
57
58
 
58
59
  ```python
59
60
  FastAPIGAELoggingHandler(
60
- app: FastAPI,
61
+ app: Starlette,
61
62
  request_logger_name: Optional[str] = None,
62
63
  log_payload: bool = True,
64
+ log_headers: bool = True,
65
+ custom_payload_parsers: Dict[str, Callable] = None,
63
66
  *args, **kwargs
64
67
  )
65
68
  ```
66
69
 
67
70
  - Parameters
68
- - **app** (FastAPI): The FastAPI application instance.
71
+ - **app** (FastAPI | Starlette): The FastAPI or Starlette application instance.
69
72
  - **request_logger_name** (Optional[str], optional): The name of the Cloud Logging logger to use for request logs.
70
73
  Defaults to the Google Cloud Project ID with the suffix '-request-logger'.
71
74
 
72
75
  - **log_payload** (bool, optional): Whether to log the request payload. If True, the payload for POST, PUT, PATCH, and DELETE requests will be logged. Defaults to True.
73
76
 
77
+ - **log_headers** (bool, optional): Whether to log the request headers. Defaults to True.
78
+ - **custom_payload_parsers** (Dict[str, Callable], optional): A dictionary mapping content types to custom parser functions for logging request payloads. If provided, these will override default parsers. Defaults to None.
79
+
74
80
  - ***args**: Additional arguments to pass to the superclass constructor. Any argument you would pass to CloudLoggingHandler.
75
81
 
76
82
  - ****kwargs**: Additional keyword arguments to pass to the superclass constructor. Any keyword argument you would pass to CloudLoggingHandler.
@@ -87,13 +93,30 @@ import os
87
93
 
88
94
  app = FastAPI()
89
95
 
96
+ async def custom_payload_parser_plain_text(request: Request):
97
+ try:
98
+ body_bytes = await request.body()
99
+ incoming_payload = body_bytes.decode('utf-8')
100
+ return f"This was the original request payload: {incoming_payload}"
101
+ except Exception as e:
102
+ return f"Failed to read request payload as plain text: {e} | {traceback.format_exc()}"
103
+
104
+
105
+
90
106
  if os.getenv('GAE_ENV', '').startswith('standard'):
91
107
  import google.cloud.logging
92
108
  from google.cloud.logging_v2.handlers import setup_logging
93
109
  from fastapi_gae_logging import FastAPIGAELoggingHandler
94
110
 
95
111
  client = google.cloud.logging.Client()
96
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
112
+ # overriding default parsing for payload when content type is 'text/plain'
113
+ gae_log_handler = FastAPIGAELoggingHandler(
114
+ app=app,
115
+ client=client,
116
+ custom_payload_parsers={
117
+ "text/plain": custom_payload_parser_plain_text
118
+ }
119
+ )
97
120
  # use the log_payload parameter if you want to opt-out from payload logging
98
121
  # gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client, log_payload=False)
99
122
  setup_logging(handler=gae_log_handler)
@@ -166,7 +189,7 @@ def post_payload(payload: Any = Body(None)):
166
189
  ## Dependencies
167
190
  This tool is built upon the following packages:
168
191
 
169
- - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
192
+ - `starlette`: Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python. FastAPI is built on top of Starlette.
170
193
  - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
171
194
 
172
195
 
@@ -1,21 +1,5 @@
1
- Metadata-Version: 2.1
2
- Name: fastapi-gae-logging
3
- Version: 0.0.7
4
- Summary: Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
- Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
- Author: Chris Karvouniaris
7
- Author-email: christos.karvouniaris247@gmail.com
8
- License: MIT
9
- Classifier: Programming Language :: Python :: 3
10
- Classifier: Operating System :: OS Independent
11
- Requires-Python: >=3.8
12
- Description-Content-Type: text/markdown
13
- License-File: LICENSE
14
- Requires-Dist: fastapi
15
- Requires-Dist: google-cloud-logging
16
-
17
1
  # fastapi-gae-logging
18
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
2
+ Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
19
3
 
20
4
  ## What problem does this package solve? Why do we need this?
21
5
  When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
@@ -45,11 +29,12 @@ The `fastapi-gae-logging` module addresses these problems by:
45
29
 
46
30
  - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
47
31
  - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
48
- - **Request Payload Logging**:
49
- - Logs the payload of a request as part of the parent log in request-grouped logs.
50
- - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
51
- - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
52
-
32
+ - **Optional incoming request logging**: Opt in/out to log headers and payload of incoming requests into the `jsonPayload` field of the parent log.
33
+ - **Optional request headers logging**: Defaults to True. Headers dict lands into field `request_headers` in the `jsonPayload` of parent log.
34
+ - **Request Payload Logging**: Defaults to True. Incoming payload parsed lands into field `request_payload` in the `jsonPayload` of parent log. Parsing is based on content type with capability to override. Currenty embedded parsers for:
35
+ - `application/json`
36
+ - `application/x-www-form-urlencoded`
37
+ - `text/plain`
53
38
 
54
39
  ## API
55
40
 
@@ -57,20 +42,25 @@ The `fastapi-gae-logging` module addresses these problems by:
57
42
 
58
43
  ```python
59
44
  FastAPIGAELoggingHandler(
60
- app: FastAPI,
45
+ app: Starlette,
61
46
  request_logger_name: Optional[str] = None,
62
47
  log_payload: bool = True,
48
+ log_headers: bool = True,
49
+ custom_payload_parsers: Dict[str, Callable] = None,
63
50
  *args, **kwargs
64
51
  )
65
52
  ```
66
53
 
67
54
  - Parameters
68
- - **app** (FastAPI): The FastAPI application instance.
55
+ - **app** (FastAPI | Starlette): The FastAPI or Starlette application instance.
69
56
  - **request_logger_name** (Optional[str], optional): The name of the Cloud Logging logger to use for request logs.
70
57
  Defaults to the Google Cloud Project ID with the suffix '-request-logger'.
71
58
 
72
59
  - **log_payload** (bool, optional): Whether to log the request payload. If True, the payload for POST, PUT, PATCH, and DELETE requests will be logged. Defaults to True.
73
60
 
61
+ - **log_headers** (bool, optional): Whether to log the request headers. Defaults to True.
62
+ - **custom_payload_parsers** (Dict[str, Callable], optional): A dictionary mapping content types to custom parser functions for logging request payloads. If provided, these will override default parsers. Defaults to None.
63
+
74
64
  - ***args**: Additional arguments to pass to the superclass constructor. Any argument you would pass to CloudLoggingHandler.
75
65
 
76
66
  - ****kwargs**: Additional keyword arguments to pass to the superclass constructor. Any keyword argument you would pass to CloudLoggingHandler.
@@ -87,13 +77,30 @@ import os
87
77
 
88
78
  app = FastAPI()
89
79
 
80
+ async def custom_payload_parser_plain_text(request: Request):
81
+ try:
82
+ body_bytes = await request.body()
83
+ incoming_payload = body_bytes.decode('utf-8')
84
+ return f"This was the original request payload: {incoming_payload}"
85
+ except Exception as e:
86
+ return f"Failed to read request payload as plain text: {e} | {traceback.format_exc()}"
87
+
88
+
89
+
90
90
  if os.getenv('GAE_ENV', '').startswith('standard'):
91
91
  import google.cloud.logging
92
92
  from google.cloud.logging_v2.handlers import setup_logging
93
93
  from fastapi_gae_logging import FastAPIGAELoggingHandler
94
94
 
95
95
  client = google.cloud.logging.Client()
96
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
96
+ # overriding default parsing for payload when content type is 'text/plain'
97
+ gae_log_handler = FastAPIGAELoggingHandler(
98
+ app=app,
99
+ client=client,
100
+ custom_payload_parsers={
101
+ "text/plain": custom_payload_parser_plain_text
102
+ }
103
+ )
97
104
  # use the log_payload parameter if you want to opt-out from payload logging
98
105
  # gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client, log_payload=False)
99
106
  setup_logging(handler=gae_log_handler)
@@ -166,7 +173,7 @@ def post_payload(payload: Any = Body(None)):
166
173
  ## Dependencies
167
174
  This tool is built upon the following packages:
168
175
 
169
- - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
176
+ - `starlette`: Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python. FastAPI is built on top of Starlette.
170
177
  - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
171
178
 
172
179
 
@@ -3,15 +3,15 @@ import time
3
3
  import contextvars
4
4
  import os
5
5
  import re
6
- import json
7
- from typing import Optional, Dict, Any
8
- from fastapi import FastAPI
6
+ from typing import Optional, Dict, Any, Callable
7
+ from starlette.applications import Starlette
9
8
  from starlette.exceptions import HTTPException
10
9
  from starlette.types import ASGIApp, Receive, Scope, Send
11
10
  from starlette.responses import Response
12
11
  from starlette.requests import Request
13
12
  from google.cloud.logging_v2.handlers import CloudLoggingHandler
14
13
  from google.cloud.logging_v2 import Resource, Logger
14
+ import traceback
15
15
 
16
16
 
17
17
  gae_request_context = contextvars.ContextVar('gae_request_context', default={
@@ -55,6 +55,44 @@ class LogInterceptor(logging.Filter):
55
55
  return True
56
56
 
57
57
 
58
+ class RequestPayloadParser:
59
+ def __init__(self, custom_parsers: Dict[str, Callable] = None):
60
+ self.parsers = {
61
+ "application/json": self._parse_json,
62
+ "application/x-www-form-urlencoded": self._parse_form_urlencoded,
63
+ "text/plain": self._parse_plain_text,
64
+ }
65
+
66
+ if custom_parsers:
67
+ self.parsers.update(custom_parsers)
68
+
69
+ async def _parse_json(self, request: Request):
70
+ try:
71
+ return await request.json()
72
+ except Exception as e:
73
+ return f"Failed to decode request payload as JSON: {e} | {traceback.format_exc()}"
74
+
75
+ async def _parse_form_urlencoded(self, request: Request):
76
+ try:
77
+ form = await request.form()
78
+ return dict(form)
79
+ except Exception as e:
80
+ return f"Failed to parse request form data: {e} | {traceback.format_exc()}"
81
+
82
+ async def _parse_plain_text(self, request: Request):
83
+ try:
84
+ body_bytes = await request.body()
85
+ return body_bytes.decode('utf-8')
86
+ except Exception as e:
87
+ return f"Failed to read request payload as plain text: {e} | {traceback.format_exc()}"
88
+
89
+ def get_parser(self, content_type: str):
90
+ """
91
+ Returns the parser function for the given content type.
92
+ """
93
+ return self.parsers.get(content_type)
94
+
95
+
58
96
  class GAERequestLogger:
59
97
  """
60
98
  A logger for emitting structured request logs in Google App Engine.
@@ -69,6 +107,10 @@ class GAERequestLogger:
69
107
  logger (Logger): The Google Cloud Logger instance to log requests.
70
108
  resource (Resource): The Google Cloud resource associated with the logger.
71
109
  log_payload (bool): Whether to log the request payload for certain HTTP methods. Defaults to True.
110
+ log_headers (bool): Whether to log the request headers. Defaults to True.
111
+ custom_payload_parsers (Dict[str, Callable], optional): A dictionary mapping content types to custom
112
+ parser functions for logging request payloads. If provided, these will override default parsers.
113
+ Defaults to None.
72
114
  """
73
115
  LOG_LEVEL_TO_SEVERITY: Dict[int, str] = {
74
116
  logging.NOTSET: 'DEFAULT',
@@ -79,7 +121,8 @@ class GAERequestLogger:
79
121
  logging.CRITICAL: 'CRITICAL',
80
122
  }
81
123
 
82
- def __init__(self, logger: Logger, resource: Resource, log_payload: bool = True) -> None:
124
+ def __init__(self, logger: Logger, resource: Resource, log_payload: bool = True, log_headers: bool = True,
125
+ custom_payload_parsers: Dict[str, Callable] = None) -> None:
83
126
  """
84
127
  Initialize the GAERequestLogger.
85
128
 
@@ -87,10 +130,16 @@ class GAERequestLogger:
87
130
  logger (Logger): The Google Cloud Logger instance to log requests.
88
131
  resource (Resource): The resource associated with the logger.
89
132
  log_payload (bool): Whether to log the request payload for certain HTTP methods. Defaults to True.
133
+ log_headers (bool): Whether to log the request headers. Defaults to True.
134
+ custom_payload_parsers (Dict[str, Callable], optional): A dictionary mapping content types to custom
135
+ parser functions for logging request payloads. If provided, these will override default parsers.
136
+ Defaults to None.
90
137
  """
91
138
  self.logger = logger
92
139
  self.resource = resource
93
140
  self.log_payload = log_payload
141
+ self.log_headers = log_headers
142
+ self.payload_parsers = RequestPayloadParser(custom_payload_parsers)
94
143
 
95
144
  def _log_level_to_severity(self, log_level: int) -> str:
96
145
  """
@@ -132,23 +181,28 @@ class GAERequestLogger:
132
181
  'remoteIp': request.client.host
133
182
  }
134
183
 
135
- payload = {}
184
+ logging_payload = {}
185
+
186
+ if self.log_headers:
187
+ logging_payload['request_headers'] = dict(request.headers)
136
188
 
137
189
  if self.log_payload and request.method in {'POST', 'PUT', 'PATCH', 'DELETE'}:
138
- try:
139
- payload = await request.json()
140
- except json.JSONDecodeError:
141
- logging.warning("Failed to decode request payload as JSON, skipping logging.")
142
- except Exception as e:
143
- logging.error(f"Unexpected error while logging payload: {e}")
190
+ content_type = request.headers.get("content-type", "").split(";")[0].strip()
191
+ payload_parser = self.payload_parsers.get_parser(content_type)
192
+
193
+ if not payload_parser:
194
+ request_payload = f"Unsupported content type {content_type}. Skipping payload logging."
144
195
  else:
145
- if not isinstance(payload, dict):
146
- payload = {
147
- f"{type(payload).__name__}_payload_wrapper": payload
148
- }
196
+ try:
197
+ request_payload = await payload_parser(request)
198
+ except Exception as e:
199
+ request_payload = f"Parser of request payload for content type {content_type} failed: {e} | {traceback.format_exc()}"
200
+
201
+ if request_payload:
202
+ logging_payload['request_payload'] = request_payload
149
203
 
150
204
  self.logger.log_struct(
151
- info=payload,
205
+ info=logging_payload,
152
206
  resource=self.resource,
153
207
  trace=f"projects/{os.environ['GOOGLE_CLOUD_PROJECT']}/traces/{trace.split('/', 1)[0]}",
154
208
  http_request=http_request,
@@ -247,19 +301,25 @@ class FastAPIGAELoggingHandler(CloudLoggingHandler):
247
301
 
248
302
  def __init__(
249
303
  self,
250
- app: FastAPI,
304
+ app: Starlette,
251
305
  request_logger_name: Optional[str] = None,
252
306
  log_payload: bool = True,
307
+ log_headers: bool = True,
308
+ custom_payload_parsers: Dict[str, Callable] = None,
253
309
  *args, **kwargs
254
310
  ) -> None:
255
311
  """
256
312
  Initialize the handler.
257
313
 
258
314
  Args:
259
- app (FastAPI): The FastAPI application instance.
315
+ app (FastAPI | Starlette): The FastAPI or Starlette application instance.
260
316
  request_logger_name (Optional[str]): The name of the Cloud Logging logger to use for request logs.
261
317
  Defaults to the Google Cloud Project ID with '-request-logger' suffix.
262
318
  log_payload (bool): Whether to log the request payload for certain HTTP methods. Defaults to True.
319
+ log_headers (bool): Whether to log the request headers. Defaults to True.
320
+ custom_payload_parsers (Dict[str, Callable], optional): A dictionary mapping content types to custom
321
+ parser functions for logging request payloads. If provided, these will override default parsers.
322
+ Defaults to None.
263
323
  *args: Additional arguments to pass to the superclass constructor.
264
324
  Any argument you would pass to CloudLoggingHandler.
265
325
  **kwargs: Additional keyword arguments to pass to the superclass constructor.
@@ -275,7 +335,9 @@ class FastAPIGAELoggingHandler(CloudLoggingHandler):
275
335
  resource=self.resource
276
336
  ),
277
337
  resource=self.resource,
278
- log_payload=log_payload
338
+ log_payload=log_payload,
339
+ log_headers=log_headers,
340
+ custom_payload_parsers=custom_payload_parsers
279
341
  )
280
342
  )
281
343
  self.addFilter(LogInterceptor())
@@ -1,5 +1,21 @@
1
+ Metadata-Version: 2.1
2
+ Name: fastapi-gae-logging
3
+ Version: 0.0.9
4
+ Summary: Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine. Groups logs coming from the same request lifecycle and propagates the maximum log level throughout the request lifecycle using middleware and context management.
5
+ Home-page: https://github.com/chrisK824/fastapi-gae-logging
6
+ Author: Chris Karvouniaris
7
+ Author-email: christos.karvouniaris247@gmail.com
8
+ License: MIT
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Operating System :: OS Independent
11
+ Requires-Python: >=3.8
12
+ Description-Content-Type: text/markdown
13
+ License-File: LICENSE
14
+ Requires-Dist: starlette
15
+ Requires-Dist: google-cloud-logging
16
+
1
17
  # fastapi-gae-logging
2
- Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
18
+ Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine to ease out logs analysis and monitoring through Google Cloud Log Explorer.
3
19
 
4
20
  ## What problem does this package solve? Why do we need this?
5
21
  When deploying FastAPI applications on Google App Engine, I encountered several challenges with logging and observability even when using the official package for this purpose, `google-cloud-logging`.
@@ -29,11 +45,12 @@ The `fastapi-gae-logging` module addresses these problems by:
29
45
 
30
46
  - **Request Logs Grouping**: Groups logs from the same request lifecycle to simplify log analysis using Google Cloud Log Explorer. The logger name for grouping can be customized and defaults to the Google Cloud Project ID with '-request-logger' as a suffix.
31
47
  - **Request Maximum Log Level Propagation**: Propagates the maximum log level throughout the request lifecycle, making it easier to search logs based on the severity of an issue.
32
- - **Request Payload Logging**:
33
- - Logs the payload of a request as part of the parent log in request-grouped logs.
34
- - Assumes the payload is a valid JSON - basically what Python's `json.loads()` can parse - otherwise, it is discarded.
35
- - Logs the payload as a dictionary using the `google-cloud-logging`'s `log_struct` method. If the payload is not a dictionary, then it is wrapped in a dummy keyword to construct one. The dummy keyword follows the format: `<original_type_of_payload>__payload_wrapper`. For example, if the payload is a list, which still a valid JSON which Python's `json.loads()` will parse just fine, then the payload list will be wrapped in a virtual dictionary under the key `list_payload_wrapper`. The dictionary is logged alongside the parent log in request-grouped logs and ends up in the `jsonPayload` field.
36
-
48
+ - **Optional incoming request logging**: Opt in/out to log headers and payload of incoming requests into the `jsonPayload` field of the parent log.
49
+ - **Optional request headers logging**: Defaults to True. Headers dict lands into field `request_headers` in the `jsonPayload` of parent log.
50
+ - **Request Payload Logging**: Defaults to True. Incoming payload parsed lands into field `request_payload` in the `jsonPayload` of parent log. Parsing is based on content type with capability to override. Currenty embedded parsers for:
51
+ - `application/json`
52
+ - `application/x-www-form-urlencoded`
53
+ - `text/plain`
37
54
 
38
55
  ## API
39
56
 
@@ -41,20 +58,25 @@ The `fastapi-gae-logging` module addresses these problems by:
41
58
 
42
59
  ```python
43
60
  FastAPIGAELoggingHandler(
44
- app: FastAPI,
61
+ app: Starlette,
45
62
  request_logger_name: Optional[str] = None,
46
63
  log_payload: bool = True,
64
+ log_headers: bool = True,
65
+ custom_payload_parsers: Dict[str, Callable] = None,
47
66
  *args, **kwargs
48
67
  )
49
68
  ```
50
69
 
51
70
  - Parameters
52
- - **app** (FastAPI): The FastAPI application instance.
71
+ - **app** (FastAPI | Starlette): The FastAPI or Starlette application instance.
53
72
  - **request_logger_name** (Optional[str], optional): The name of the Cloud Logging logger to use for request logs.
54
73
  Defaults to the Google Cloud Project ID with the suffix '-request-logger'.
55
74
 
56
75
  - **log_payload** (bool, optional): Whether to log the request payload. If True, the payload for POST, PUT, PATCH, and DELETE requests will be logged. Defaults to True.
57
76
 
77
+ - **log_headers** (bool, optional): Whether to log the request headers. Defaults to True.
78
+ - **custom_payload_parsers** (Dict[str, Callable], optional): A dictionary mapping content types to custom parser functions for logging request payloads. If provided, these will override default parsers. Defaults to None.
79
+
58
80
  - ***args**: Additional arguments to pass to the superclass constructor. Any argument you would pass to CloudLoggingHandler.
59
81
 
60
82
  - ****kwargs**: Additional keyword arguments to pass to the superclass constructor. Any keyword argument you would pass to CloudLoggingHandler.
@@ -71,13 +93,30 @@ import os
71
93
 
72
94
  app = FastAPI()
73
95
 
96
+ async def custom_payload_parser_plain_text(request: Request):
97
+ try:
98
+ body_bytes = await request.body()
99
+ incoming_payload = body_bytes.decode('utf-8')
100
+ return f"This was the original request payload: {incoming_payload}"
101
+ except Exception as e:
102
+ return f"Failed to read request payload as plain text: {e} | {traceback.format_exc()}"
103
+
104
+
105
+
74
106
  if os.getenv('GAE_ENV', '').startswith('standard'):
75
107
  import google.cloud.logging
76
108
  from google.cloud.logging_v2.handlers import setup_logging
77
109
  from fastapi_gae_logging import FastAPIGAELoggingHandler
78
110
 
79
111
  client = google.cloud.logging.Client()
80
- gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client)
112
+ # overriding default parsing for payload when content type is 'text/plain'
113
+ gae_log_handler = FastAPIGAELoggingHandler(
114
+ app=app,
115
+ client=client,
116
+ custom_payload_parsers={
117
+ "text/plain": custom_payload_parser_plain_text
118
+ }
119
+ )
81
120
  # use the log_payload parameter if you want to opt-out from payload logging
82
121
  # gae_log_handler = FastAPIGAELoggingHandler(app=app, client=client, log_payload=False)
83
122
  setup_logging(handler=gae_log_handler)
@@ -150,7 +189,7 @@ def post_payload(payload: Any = Body(None)):
150
189
  ## Dependencies
151
190
  This tool is built upon the following packages:
152
191
 
153
- - `fastapi`: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
192
+ - `starlette`: Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python. FastAPI is built on top of Starlette.
154
193
  - `google-cloud-logging`: Google Cloud Logging API client library for logging and managing logs in Google Cloud Platform.
155
194
 
156
195
 
@@ -1,2 +1,2 @@
1
- fastapi
1
+ starlette
2
2
  google-cloud-logging
@@ -2,8 +2,8 @@ from setuptools import setup, find_packages
2
2
 
3
3
  setup(
4
4
  name="fastapi-gae-logging",
5
- version="0.0.7",
6
- description="Custom Cloud Logging handler for FastAPI applications deployed in Google App Engine. \
5
+ version="0.0.9",
6
+ description="Custom Cloud Logging handler for FastAPI (or any Starlette based) applications deployed in Google App Engine. \
7
7
  Groups logs coming from the same request lifecycle and propagates the maximum log level \
8
8
  throughout the request lifecycle using middleware and context management.",
9
9
  long_description=open("README.md").read(),
@@ -13,7 +13,7 @@ setup(
13
13
  author_email="christos.karvouniaris247@gmail.com",
14
14
  packages=find_packages(),
15
15
  install_requires=[
16
- "fastapi",
16
+ "starlette",
17
17
  "google-cloud-logging",
18
18
  ],
19
19
  python_requires=">=3.8",