aiohttp-request-id-logging 1.0.0__tar.gz

aiohttp_request_id_logging-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019 Petr Messner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

aiohttp_request_id_logging-1.0.0/PKG-INFO

@@ -0,0 +1,487 @@
+Metadata-Version: 2.4
+Name: aiohttp-request-id-logging
+Version: 1.0.0
+Summary: Setup proper request id logging for your Aiohttp app
+Maintainer-email: Petr Messner <petr.messner@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/messa/aiohttp-request-id-logging
+Project-URL: Bug Reports, https://github.com/messa/aiohttp-request-id-logging/issues
+Project-URL: Source, https://github.com/messa/aiohttp-request-id-logging
+Keywords: aiohttp,logging
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Framework :: AsyncIO
+Classifier: Framework :: aiohttp
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: aiohttp
+Dynamic: license-file
+
+aiohttp-request-id-logging
+==========================
+
+Motivation
+----------
+
+When logging from an async application (e.g. aiohttp web application),
+log messages from different async tasks, http requests etc. are intertwined and you cannot surely tell which line was generated by what request.
+For example:
+
+```
+2020-01-15 15:35:37,501  INFO: Processing transfer id 1234
+2020-01-15 15:35:37,976  INFO: Processing transfer id 5678
+2020-01-15 15:35:38,201 ERROR: Oh no, something bad has happened! Cannot finish the transfer.
+2020-01-15 15:35:38,504  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 200 165 "-" "curl/7.68.0"
+2020-01-15 15:35:38,982  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 500 165 "-" "curl/7.68.0"
+```
+
+So, which transfer has failed? The one with id 1234, or the one with id 5678?
+
+Of course, ideally the error message should contain also the transfer id.
+But it is not always this case, or you may want to inspect also other log messages that were logged by that specific task/request.
+
+When you start to use this library in your aiohttp web application, this is how your log messages will look like:
+
+```
+2020-01-15 15:58:47,238  INFO: [req:kbyLG8d] Processing GET / (__main__:hello)
+2020-01-15 15:58:47,950  INFO: [req:xtMacpA] Processing GET / (__main__:hello)
+2020-01-15 15:58:48,240  INFO: [req:kbyLG8d] Processing transfer id 1234
+2020-01-15 15:58:48,953  INFO: [req:xtMacpA] Processing transfer id 5678
+2020-01-15 15:58:49,182 ERROR: [req:xtMacpA] Oh no, something bad has happened! Cannot finish the transfer.
+2020-01-15 15:58:49,242  INFO: [req:kbyLG8d] 127.0.0.1 "GET / HTTP/1.1" 200 165 "-" "curl/7.68.0"
+2020-01-15 15:58:49,959  INFO: [req:xtMacpA] 127.0.0.1 "GET / HTTP/1.1" 500 165 "-" "curl/7.68.0"
+```
+
+
+Installation
+------------
+
+```shell
+$ uv add https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
+# or
+$ python3 -m pip install https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
+```
+
+Or add this line to `requirements.txt`:
+
+```
+aiohttp-request-id-logging @ https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
+```
+
+
+Usage
+-----
+
+```python
+from aiohttp.web import Application, Response, RouteTableDef, run_app
+from logging import DEBUG, basicConfig
+from aiohttp_request_id_logging import (
+    setup_logging_request_id_prefix,
+    RequestIdMiddleware,
+    RequestIdAccessLogger,
+)
+
+routes = RouteTableDef()
+
+@routes.get('/')
+async def hello(request):
+    return Response(text="Hello, world!\n")
+
+basicConfig(
+    level=DEBUG,
+    format='%(asctime)s [%(threadName)s] %(name)-37s %(levelname)5s: %(requestIdPrefix)s%(message)s')
+
+setup_logging_request_id_prefix()
+
+app = Application(middlewares=[RequestIdMiddleware()])
+app.router.add_routes(routes)
+
+run_app(app, access_log_class=RequestIdAccessLogger)
+```
+
+For more complete example see [examples/demo.py](examples/demo.py).
+
+Customization of the middleware behavior is demonstrated in
+[`examples/demo_customization_injection.py`](examples/demo_customization_injection.py)
+(constructor parameters) and
+[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py)
+(subclassing).
+
+
+How it works
+------------
+
+This library helps you to add request (correlation) id to the log messages in a few steps:
+
+1. **`RequestIdMiddleware`:** Generate random `request_id` for each aiohttp request and
+
+   - store it in a ContextVar `aiohttp_request_id_logging.request_id`
+   - store it also in the request storage: `request[aiohttp_request_id_logging.REQUEST_ID_KEY]`
+     (and, by default, also under the plain string key `request['request_id']` for backward compatibility)
+   - return it to the client in the `X-Request-Id` response header
+
+2. **`setup_logging_request_id_prefix()`:** Modify logging record factory so that the `request_id` is attached to every logging record created
+
+   - so you should modify your log format, for example `logging.basicConfig(format=... %(levelname)5s: %(requestIdPrefix)s%(message)s')`
+
+3. Because the aiohttp access logging happens out of the middleware scope, the request id ContextVar would be already resetted. So **`RequestIdAccessLogger`** is provided that adds the request_id to the access log message.
+
+4. Unhandled exceptions ("500 errors") are logged and converted to a 500 response directly by **`RequestIdMiddleware`** – if they were left to the standard aiohttp error handling, the traceback would be logged outside of the middleware scope, without the request id (see the [Reference](#requestidmiddleware) for details).
+
+5. If you use **[Sentry](https://docs.sentry.io/platforms/python/aiohttp/)**, a `request_id` [tag](https://docs.sentry.io/enriching-error-data/context/?platform=python#tagging-events) is added when the request is processed.
+
+Sentry integration will be active only if you have `sentry_sdk` installed.
+
+
+Reference
+---------
+
+All names are importable directly from the `aiohttp_request_id_logging` package.
+
+### `RequestIdMiddleware`
+
+The aiohttp middleware. Generates a request id for every request, stores it
+in the `request_id` ContextVar and in the request, adds an `X-Request-Id`
+response header, and (if Sentry is installed) creates a Sentry scope with
+a `request_id` tag.
+
+Exceptions raised from the handler are processed by the middleware too:
+
+- an `HTTPException` (the aiohttp way of returning error responses from
+  anywhere in the handler) is re-raised for aiohttp to process, with the
+  request id response header added to it first
+- any other unhandled exception is logged together with its traceback and
+  converted to a plain text `500 Internal Server Error` response (also
+  carrying the request id header) right in the middleware
+
+The latter is deliberate: if the exception were left to the standard aiohttp
+error handling, it would be logged outside of the middleware scope, where the
+`request_id` ContextVar is already reset (and the Sentry scope with the
+`request_id` tag closed) – so the traceback, usually the log message you need
+to pair with a request the most, would be the one line without the request id.
+Override `get_response_for_exception(request, exc)` to customize the error
+response (a nicer error page, JSON for API paths...).
+
+Constructor parameters (all keyword-only):
+
+- `request_id_factory` – a zero-argument callable returning the request id string,
+  used when `get_request_id` returns `None` (which the default implementation
+  always does); default: `random_request_id_factory`
+- `get_request_id` – a callable `(request)` returning the request id for the
+  given request, or `None` to have one generated with `request_id_factory`;
+  if you adopt an incoming header value here, validate it – it is controlled
+  by the client
+- `log_request_start` – a callable `(request, handler)` that logs the request
+  start message, replacing the default `Processing GET / (...)` one;
+  pass `noop` to disable the message
+- `log_function_name` – include the handler name in the default request start
+  message; default: `True`
+- `add_response_request_id_header` – a callable `(response, req_id)` that adds
+  the request id header to the response, replacing the default behavior
+  (which keeps a header already set by the handler); pass `noop` to disable
+  the header
+- `request_id_header_name` – name of the response header with the request id;
+  default: `X-Request-Id`
+- `no_fallback_request_id_key` – if `True`, the request id is stored in the request
+  only under `REQUEST_ID_KEY`, not also under the backward compatibility plain
+  string key `request['request_id']`; default: `False`
+
+The behavior can also be customized by subclassing – overriding the class
+attributes (`request_id_header_name`, `log_function_name`) or the methods:
+`get_request_id`, `before_request`, `after_request`,
+`get_response_for_exception`, `log_request_start`, `set_request_keys`,
+`setup_sentry_scope`, `add_response_request_id_header`, `get_function_name`.
+
+Both approaches are demonstrated in
+[`examples/demo_customization_injection.py`](examples/demo_customization_injection.py) and
+[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py).
+
+Functions stored in class attributes are tricky (Python would bind them
+as methods), that is why callables like `request_id_factory` are passed
+via the constructor parameters instead.
+
+The `get_request_id`, `log_request_start` and `add_response_request_id_header`
+parameters take precedence over the methods of the same name – when the
+parameter is passed, the method (even one overridden in a subclass) is not
+called.
+
+The middleware does not adopt a request id sent by the client – how (and
+whether) to trust such a value depends on the deployment. If you want that,
+pass (or override) `get_request_id(request)` and validate the incoming value
+there; see
+[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py).
+
+`request_id_middleware()` is a backward compatibility wrapper of this class;
+unlike the constructor, its `log_request_start` parameter is a bool –
+`log_request_start=False` translates to `log_request_start=noop`.
+
+### `setup_logging_request_id_prefix()`
+
+Wraps the logging record factory so that every log record gets two extra
+attributes usable in the log format:
+
+- `%(requestIdPrefix)s` – `"[req:kbyLG8d] "`, or an empty string outside of a request
+- `%(request_id)s` – the raw request id, or `None`
+
+The prefix can be customized with the `prefix_format` parameter
+(default: `"[req:{request_id}] "`).
+
+Safe to call multiple times (does the setup only once).
+
+### `RequestIdAccessLogger`
+
+Subclass of `aiohttp.web_log.AccessLogger` that sets the `request_id`
+ContextVar while writing the access log line. Needed because aiohttp writes
+the access log outside of the middleware scope. Pass it to
+`run_app(app, access_log_class=RequestIdAccessLogger)`.
+
+### `request_id`
+
+`ContextVar` holding the request id of the currently processed request.
+Read it with `request_id.get()` – it returns `None` outside of a request.
+
+### `REQUEST_ID_KEY`
+
+Key under which the request id is stored in the request:
+`request[REQUEST_ID_KEY]`. It is a `web.RequestKey` instance on aiohttp
+versions that support it, otherwise the plain string `'request_id'`.
+(The plain string key `request['request_id']` is set as well by default,
+for backward compatibility; pass `RequestIdMiddleware(no_fallback_request_id_key=True)`
+to store the id only under `REQUEST_ID_KEY`.)
+
+`FALLBACK_REQUEST_ID_KEY` is that backward compatibility plain string key –
+`'request_id'`, or `None` on aiohttp versions without `web.RequestKey`
+(there `REQUEST_ID_KEY` itself is the plain string, so there is no fallback).
+
+### Request id factories
+
+- `random_request_id_factory(length=7)` – the default; returns a random URL-safe
+  string of the given length, avoiding visually ambiguous characters
+  (`1`/`l`/`I`, `2`/`Z`, `O`/`0`)
+  (`generate_request_id` is its backward compatibility alias)
+- `sequential_request_id_factory` / `SequentialRequestIdFactory` – alternative
+  factory producing ids like `Wxyz0000`, `Wxyz0001`… – a random per-process
+  prefix followed by a sequential number
+
+Usage: `RequestIdMiddleware(request_id_factory=sequential_request_id_factory)`
+
+Caveat: if the request ids are ever exposed to clients (response header,
+error page…), sequential ids reveal how many requests the server processes
+and how many server processes there are. Note that the middleware sends the
+request id in the `X-Request-Id` response header by default – pass
+`add_response_request_id_header=noop` to disable that. If the exposure is
+a concern, stick with the default random factory.
+
+### `noop`
+
+A do-nothing function. Pass it as the `log_request_start` or
+`add_response_request_id_header` constructor parameter to disable
+the corresponding default behavior.
+
+### `RequestIdKeyAlreadySetError`
+
+Raised by the middleware when the request already contains a request id –
+most likely the middleware is applied twice, or something else sets it too.
+The existing id is available as the `existing_request_id` attribute.
+
+
+Development
+-----------
+
+The project uses [uv](https://docs.astral.sh/uv/) for dependency and environment
+management, [Ruff](https://docs.astral.sh/ruff/) for linting and formatting
+and [ty](https://docs.astral.sh/ty/) for type checking.
+
+```shell
+$ make check      # lint + ty check + tests (pytest)
+$ make lint       # ruff check + ruff format --check
+$ make lint-fix   # apply ruff fixes and formatting
+$ make typecheck  # ty check
+```
+
+The Makefile targets use `uv run`, which automatically creates the `.venv`
+virtualenv and installs the dependencies (including the `dev` dependency group)
+on the first run.
+
+
+Version changelog
+-----------------
+
+### 1.0.0 (2026-07-16)
+
+- Added `aiohttp_request_id_logging.__version__`
+- `RequestIdContextAccessLogger` was renamed to `RequestIdAccessLogger`;
+  the old name still works as a backward compatibility alias
+- The middleware was refactored into a class `RequestIdMiddleware` so that it can be
+  subclassed and individual parts of the behavior customized by overriding class
+  attributes and methods (`before_request`, `after_request`,
+  `log_request_start`, `set_request_keys`, `setup_sentry_scope`,
+  `add_response_request_id_header`...)
+  - `request_id_middleware()` is kept as a backward compatibility wrapper function
+  - the deprecated-in-aiohttp-4 `@web.middleware` decorator is no longer used
+  - see the new examples
+    [`examples/demo_customization_injection.py`](examples/demo_customization_injection.py) and
+    [`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py)
+- The middleware now returns the request id to the client in the `X-Request-Id`
+  response header (a header already set by the handler is not overwritten);
+  the header name can be changed with the `request_id_header_name`
+  parameter, or the header disabled completely with `add_response_request_id_header=noop`
+- New hook `get_request_id(request)` (a constructor parameter or an overridable
+  method) – returns the request id for the given request, e.g. adopted from
+  an incoming header, or `None` to have a new id generated with
+  `request_id_factory` (if you adopt an incoming value, validate it – see
+  [`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py))
+- The response request id header is silently skipped for responses that
+  were already prepared (streaming/WebSocket handlers)
+- The request start message ("Processing ...") can be disabled
+  (`request_id_middleware(log_request_start=False)`,
+  `RequestIdMiddleware(log_request_start=noop)`) or replaced with a custom callable
+  (`RequestIdMiddleware(log_request_start=my_log_function)`)
+- The middleware now raises `RequestIdKeyAlreadySetError` when the request
+  already contains a request id, for example when the middleware is applied twice
+  or something else also sets the request id
+- New parameter `RequestIdMiddleware(no_fallback_request_id_key=True)` stores the
+  request id only under `REQUEST_ID_KEY`, skipping the backward compatibility
+  plain string key `request['request_id']` (the plain string key is now also
+  exported as `FALLBACK_REQUEST_ID_KEY`)
+- Breaking: all `request_id_middleware()` parameters are now keyword-only; positional calls
+  like `request_id_middleware(my_factory)` raise `TypeError` – use
+  `request_id_middleware(request_id_factory=my_factory)` instead
+- `HTTPException` raised from a handler is re-raised as before, but the request id
+  response header is now added to it first (the exception is also the response
+  aiohttp sends to the client)
+- The 500 response for an unhandled exception from the handler can be customized
+  by overriding the `get_response_for_exception(request, exc)` method
+- New parameter `setup_logging_request_id_prefix(prefix_format=...)` customizes the log
+  record prefix (default: `"[req:{request_id}] "`)
+- The `request_id` ContextVar now has a default of `None`, so it can be read
+  with just `request_id.get()` – no more `LookupError` (or the `request_id.get(None)`
+  workaround) outside of a request
+- Breaking: `random_request_id_factory()` has a new `length` parameter, replacing the
+  removed module-level variables `request_id_default_length` and
+  `default_request_id_factory`; the module-level function `get_function_name`
+  was also removed (it is now a `RequestIdMiddleware` static method)
+- Generated request ids now avoid visually ambiguous characters
+  (`1`/`l`/`I`, `2`/`Z`, `O`/`0`)
+- The package was split into more modules (`middleware.py`, `errors.py`, `context.py`,
+  `logging_setup.py`, `request_id_factories.py`); everything is
+  still importable directly from `aiohttp_request_id_logging`, which now also defines `__all__`
+- Added type hints, together with the `py.typed` marker so that the types
+  are visible to type checkers (mypy, pyright) in projects using this library
+- Added example [examples/demo_legacy.py](examples/demo_legacy.py) demonstrating
+  the backward compatible `request_id_middleware()` usage
+- Tests: every example in [examples/](examples/) is now run and checked
+  (the request id in the response header, the log line prefixes, clean stderr);
+  added tests for the HTTPException and 500 error handling
+- Development: the project is now managed with [uv](https://docs.astral.sh/uv/)
+  (`uv.lock`, dev dependencies in the `[dependency-groups]` table, which replaced
+  the `test` extra); linting was migrated from flake8 to
+  [Ruff](https://docs.astral.sh/ruff/) and the codebase was reformatted
+  with `ruff format` (see the new `make lint` and `make lint-fix` targets)
+- Development: added type checking with [ty](https://docs.astral.sh/ty/)
+  (`make typecheck`, also run in CI); `sentry-sdk` was added to the dev
+  dependencies so that the Sentry integration is type checked too
+- Packaging: modernized the pyproject metadata – SPDX license expression
+  (`license = "MIT"`), added the Development Status, Framework and
+  Typing :: Typed classifiers
+- Github Actions: lint runs in a separate job; uv is used to install Python
+  and the dependencies; the test matrix also includes the latest aiohttp
+  release (unpinned) to catch regressions with new aiohttp versions early
+
+### 0.0.8 (2026-07-14)
+
+- Store the request id in the request under a `web.RequestKey` instance,
+  exported as `aiohttp_request_id_logging.REQUEST_ID_KEY` – fixes `NotAppKeyWarning`
+  emitted by aiohttp 3.13/3.14
+  - the plain string key `request['request_id']` is still set for backward compatibility
+- Moved `demo.py` to [examples/](examples/)
+- Fixed pyproject metadata
+- Tests: run the demo in Python dev mode and assert clean stderr
+  (aiohttp-internal warnings are whitelisted)
+- Github Actions: added Python 3.14, updated action versions
+
+### 0.0.7 (2025-05-12)
+
+- Support new scope handling (`isolation_scope`) in sentry_sdk 2.x
+  and removed an incorrect upgrade warning
+  ([#6](https://github.com/messa/aiohttp-request-id-logging/pull/6), thanks @martinhanzik)
+- Github Actions: added Python 3.13
+
+### 0.0.6 (2024-06-02)
+
+- Added `sequential_request_id_factory` as an alternative request id generator
+- Improved `demo.py` and added a test running it
+
+### 0.0.5 (2024-05-02)
+
+- `RequestIdContextAccessLogger` no longer fails when the request id is not set,
+  for example when an error occurs in a middleware
+- Github Actions: added Python 3.12
+
+### 0.0.4 (2023-05-11)
+
+- Fixed `request_id_default_length` handling: the generated request id is now
+  exactly that many characters long (and the default was changed from 5 to 7)
+
+### 0.0.3 (2023-05-11)
+
+- Added `req:` to the log record prefix: `[x12y3]` → `[req:x12y3]`
+
+### 0.0.2 (2023-05-11)
+
+- Migrated packaging from `setup.py` to `pyproject.toml`
+- Added flake8 lint configuration
+- Github Actions: added Python 3.10 and 3.11, added aiohttp version matrix
+
+### 0.0.1 (2021-05-18)
+
+- Initial release: `request_id_middleware()`, `setup_logging_request_id_prefix()`,
+  `RequestIdContextAccessLogger` and the Sentry `request_id` tag integration
+
+
+Alternatives
+------------
+
+- [belvaio-request-id](https://pypi.org/project/belvaio-request-id/) – another
+  request id middleware + access logger for aiohttp; it also adopts the id
+  from the incoming `X-Request-Id` header and attaches it to log records via
+  a logging filter (instead of the log record factory used here); last
+  released in 2021 and its source repository is no longer available
+- [asgi-correlation-id](https://github.com/snok/asgi-correlation-id) – the
+  same idea for the ASGI world (FastAPI, Starlette, Django...; aiohttp is not
+  ASGI, so it cannot be used here) – correlation id middleware, a logging
+  filter and a Sentry integration
+- [django-guid](https://github.com/snok/django-guid) – correlation id
+  middleware and logging integration for Django
+- [structlog contextvars](https://www.structlog.org/en/stable/contextvars.html) –
+  if you log via structlog, you can bind a request id to the context-local
+  context yourself (`structlog.contextvars.bind_contextvars(request_id=...)`)
+  in a small custom middleware
+- [aiotask-context](https://github.com/Skyscanner/aiotask-context) – stores
+  contextual data (e.g. a request id) directly in the asyncio task and
+  includes an aiohttp request id logging example; predates `contextvars`,
+  its last release was in 2020 and the repository is now archived
+- [OpenTelemetry Python](https://opentelemetry.io/docs/languages/python/) –
+  the heavyweight solution: full distributed tracing with an aiohttp server
+  instrumentation, and the logging instrumentation can optionally inject
+  `otelTraceID`/`otelSpanID` into log records (using the same log record
+  factory technique as this library)
+
+
+Links
+-----
+
+In general, you should be familiar with Aiohttp documentation related to production deployment:
+
+- https://docs.aiohttp.org/en/stable/web_advanced.html#deploying-behind-a-proxy
+- https://docs.aiohttp.org/en/stable/deployment.html
+
+This is where this started: https://stackoverflow.com/a/58801740/196206