cloudru-qa-grpc-client-structlog 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cloudru_qa_grpc_client_structlog-1.0.0/.agents/skills/grpc-client-structlog-usage/SKILL.md +191 -0
- cloudru_qa_grpc_client_structlog-1.0.0/.agents/skills/grpc-client-structlog-usage/references/fields.md +161 -0
- cloudru_qa_grpc_client_structlog-1.0.0/.agents/skills/grpc-client-structlog-usage/references/pitfalls.md +176 -0
- cloudru_qa_grpc_client_structlog-1.0.0/.gitignore +26 -0
- cloudru_qa_grpc_client_structlog-1.0.0/LICENSE +202 -0
- cloudru_qa_grpc_client_structlog-1.0.0/PKG-INFO +240 -0
- cloudru_qa_grpc_client_structlog-1.0.0/README.md +218 -0
- cloudru_qa_grpc_client_structlog-1.0.0/grpc_client_structlog/__init__.py +3 -0
- cloudru_qa_grpc_client_structlog-1.0.0/grpc_client_structlog/option.py +62 -0
- cloudru_qa_grpc_client_structlog-1.0.0/grpc_client_structlog/parser.py +156 -0
- cloudru_qa_grpc_client_structlog-1.0.0/grpc_client_structlog/py.typed +0 -0
- cloudru_qa_grpc_client_structlog-1.0.0/grpc_client_structlog/sink.py +95 -0
- cloudru_qa_grpc_client_structlog-1.0.0/pyproject.toml +69 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/Dockerfile +9 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/__init__.py +0 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/assertions.py +156 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/clients.py +163 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/conftest.py +130 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/docker-compose.yml +7 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/fakes.py +139 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/service.py +167 -0
- cloudru_qa_grpc_client_structlog-1.0.0/tests/test_logging.py +674 -0
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: grpc-client-structlog-usage
|
|
3
|
+
description: Structured request/response logging for the grpc_client library — the with_structlogger option that writes every gRPC call into structlog as separate fields instead of one text line, with secret masking on by default. Use when a project imports grpc_client_structlog, when wiring structlog into a gRPC client, or when the built-in options.with_logger crashes with "object has no attribute 'isEnabledFor'". Key names are with_structlogger, sanitize_keys, use_default_sanitize_keys, status_code, messages_count, trace_id.
|
|
4
|
+
license: Apache-2.0
|
|
5
|
+
metadata:
|
|
6
|
+
library-version: "1.0.0"
|
|
7
|
+
source: packages/grpc-client-structlog
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Structured logging for grpc_client
|
|
11
|
+
|
|
12
|
+
The library exports exactly one name: `with_structlogger`. It is a `ClientOption` that replaces
|
|
13
|
+
`grpc_client.options.with_logger`. Instead of one text line per call, every attribute — method, body,
|
|
14
|
+
metadata, status — becomes its own [structlog](https://www.structlog.org) field, so logs can be
|
|
15
|
+
filtered and searched by field rather than by substring.
|
|
16
|
+
|
|
17
|
+
Everything else — building the client, stubs, timeouts, auth, streaming — is plain `grpc_client`.
|
|
18
|
+
This library adds no client of its own and no way to log outside of the option.
|
|
19
|
+
|
|
20
|
+
## Hard rules
|
|
21
|
+
|
|
22
|
+
1. Install **one** logging option. `with_structlogger` and `options.with_logger` each add their own
|
|
23
|
+
interceptor; passing both logs every call twice, in two different formats.
|
|
24
|
+
2. Pass it as a constructor option, exactly like the built-in ones. There is no post-construction setup.
|
|
25
|
+
3. Put it **after** `options.with_auth(...)` so the log shows the request that actually goes out.
|
|
26
|
+
4. The library plugs into `options.with_log_sink` and the record types of `grpc_client`; the matching
|
|
27
|
+
`cloudru-qa-grpc-client` is installed as a dependency.
|
|
28
|
+
5. Do not name a structlog field `message` anywhere in the pipeline: this library calls the payload
|
|
29
|
+
`body` on purpose, because `EventRenamer("message")` would collide with it.
|
|
30
|
+
6. Do not turn off `use_default_sanitize_keys` to "clean up" the log. It unmasks passwords and tokens.
|
|
31
|
+
7. Levels are limited to `DEBUG` / `INFO` / `WARNING` / `ERROR` / `CRITICAL`. Anything else raises
|
|
32
|
+
`ValueError` when the client is built — structlog cannot write an unnamed level.
|
|
33
|
+
|
|
34
|
+
Before debugging odd output, read `references/pitfalls.md` — it covers the non-obvious behaviour.
|
|
35
|
+
|
|
36
|
+
## Why it exists
|
|
37
|
+
|
|
38
|
+
`options.with_logger` takes a stdlib logger and writes one text line per call. Feeding it a structlog
|
|
39
|
+
bound logger does not work: it checks the level with `logger.isEnabledFor(level)`, and a logger built
|
|
40
|
+
by `structlog.make_filtering_bound_logger(...)` spells that method `is_enabled_for`, so every call dies
|
|
41
|
+
with:
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
AttributeError: 'BoundLoggerFilteringAtDebug' object has no attribute 'isEnabledFor'.
|
|
45
|
+
Did you mean: 'is_enabled_for'?
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`with_structlogger` installs a `LogSink` instead — the public extension point of `grpc-client` 2.1.
|
|
49
|
+
The client builds a `RequestRecord` / `ResponseRecord` and picks the level; the sink turns it into
|
|
50
|
+
structlog fields. The level question goes through `is_enabled_for`, and a logger that cannot answer
|
|
51
|
+
it — the method is missing, or it raises, as `structlog.stdlib.BoundLogger` over a
|
|
52
|
+
`structlog.PrintLogger` does — counts as enabled. A question about the level must never take the
|
|
53
|
+
call down.
|
|
54
|
+
|
|
55
|
+
## Usage
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
import structlog
|
|
59
|
+
from api import orders_pb2, orders_pb2_grpc
|
|
60
|
+
from grpc_client import GRPCClient
|
|
61
|
+
from grpc_client_structlog import with_structlogger
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
class OrdersClient(GRPCClient[orders_pb2_grpc.OrdersStub]):
|
|
65
|
+
@property
|
|
66
|
+
def stub_class(self) -> type[orders_pb2_grpc.OrdersStub]:
|
|
67
|
+
return orders_pb2_grpc.OrdersStub
|
|
68
|
+
|
|
69
|
+
def get_order(self, order_id: str):
|
|
70
|
+
return self.send_unary_unary_request(orders_pb2.GetOrderRequest(id=order_id), self.stub.GetOrder)
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
with OrdersClient.from_addr(
|
|
74
|
+
"orders.example.com:443",
|
|
75
|
+
with_structlogger(structlog.get_logger("orders")),
|
|
76
|
+
) as client:
|
|
77
|
+
client.get_order("42")
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Combines with any other option:
|
|
81
|
+
|
|
82
|
+
```python
|
|
83
|
+
from grpc_client import options
|
|
84
|
+
|
|
85
|
+
client = OrdersClient.from_addr(
|
|
86
|
+
"orders.example.com:443",
|
|
87
|
+
options.with_auth(auth),
|
|
88
|
+
options.with_timeout(30),
|
|
89
|
+
with_structlogger(logger, json_indent=2),
|
|
90
|
+
)
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Async client
|
|
94
|
+
|
|
95
|
+
Same option, same arguments. The sync or async interceptor is picked from the client type inside the
|
|
96
|
+
option, so nothing extra is configured:
|
|
97
|
+
|
|
98
|
+
```python
|
|
99
|
+
from grpc_client import AsyncGRPCClient
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
class AsyncOrdersClient(AsyncGRPCClient[orders_pb2_grpc.OrdersStub]):
|
|
103
|
+
@property
|
|
104
|
+
def stub_class(self) -> type[orders_pb2_grpc.OrdersStub]:
|
|
105
|
+
return orders_pb2_grpc.OrdersStub
|
|
106
|
+
|
|
107
|
+
async def get_order(self, order_id: str):
|
|
108
|
+
return await self.send_unary_unary_request(orders_pb2.GetOrderRequest(id=order_id), self.stub.GetOrder)
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
async with AsyncOrdersClient.from_addr("orders.example.com:443", with_structlogger(logger)) as client:
|
|
112
|
+
await client.get_order("42")
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The final record of a response stream is written by a separate task, so an async client must be
|
|
116
|
+
closed — `close()` waits for those tasks. `async with` does it for you.
|
|
117
|
+
|
|
118
|
+
## What lands in the log
|
|
119
|
+
|
|
120
|
+
Up to two events per call, linked by `trace_id` — a level filtered out by structlog simply produces no
|
|
121
|
+
record, so a call can end up with one or none:
|
|
122
|
+
|
|
123
|
+
| Field | Event | Value |
|
|
124
|
+
|---|---|---|
|
|
125
|
+
| `event` | both | `grpc request sent` / `grpc response received` |
|
|
126
|
+
| `method` | both | `/package.Service/Method` |
|
|
127
|
+
| `body` | request always, response only on a successful unary answer | JSON string of the message; `<streamed content>` for a request stream |
|
|
128
|
+
| `metadata` | both | JSON string; **absent** when the call carried no metadata |
|
|
129
|
+
| `status_code` | response | status **name**: `OK`, `NOT_FOUND`, … |
|
|
130
|
+
| `details` | response | the service's error text; present only on a failed call |
|
|
131
|
+
| `messages_count` | response | number of messages read from a response stream |
|
|
132
|
+
| `trace_id` | both | UUID4 generated per call |
|
|
133
|
+
|
|
134
|
+
`status_code` holds the name, not the number: the name reads without a lookup table and filters
|
|
135
|
+
cleanly, while `5 (NOT_FOUND)` would force substring queries. `messages_count` is the only integer
|
|
136
|
+
field; other composite values are serialized to JSON strings. The full catalog is in
|
|
137
|
+
`references/fields.md`.
|
|
138
|
+
|
|
139
|
+
## Masking
|
|
140
|
+
|
|
141
|
+
Fields whose **name** is a common secret — `authorization`, `cookie`, `token`, `password`, `api_key`
|
|
142
|
+
and the rest of `grpc_client.utils.sanitizer.DEFAULT_SANITIZE_KEYS` — are replaced with `****` by
|
|
143
|
+
default, **at any depth**, and regardless of naming style (`accessToken` matches `access_token`).
|
|
144
|
+
|
|
145
|
+
Project-specific fields go into `sanitize_keys`. A key without separators is a **field name** masked
|
|
146
|
+
at any depth; a key with a dot, a star or brackets is a [glom](https://glom.readthedocs.io/en/latest/)
|
|
147
|
+
**path** applied as written:
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
with_structlogger(logger, sanitize_keys=["x-tenant-key", "user.internal_id", "items.*.card_number"])
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
| Key | What it masks |
|
|
154
|
+
|---|---|
|
|
155
|
+
| `x-tenant-key` | a metadata key (metadata is flat, whole-key match, case-insensitive) |
|
|
156
|
+
| `client_secret` | a body field with that name, at any depth |
|
|
157
|
+
| `user.internal_id` | one nested body field |
|
|
158
|
+
| `items.*.card_number` | a field in every array element |
|
|
159
|
+
|
|
160
|
+
Masking never touches the message the caller receives — only the copy that goes into the log.
|
|
161
|
+
|
|
162
|
+
## Parameters
|
|
163
|
+
|
|
164
|
+
| Parameter | Type | Meaning |
|
|
165
|
+
|---|---|---|
|
|
166
|
+
| `logger` | `BoundLogger \| FilteringBoundLogger` | where to write |
|
|
167
|
+
| `sanitize_keys` | `Sequence[str] \| None` | field names and glom paths replaced with `****` |
|
|
168
|
+
| `json_indent` | `int \| None` | indent used when serializing JSON fields |
|
|
169
|
+
| `log_level` | `int \| None` | level of a successful record, defaults to `INFO` |
|
|
170
|
+
| `err_log_level` | `int \| None` | level of a failed record, defaults to `ERROR` |
|
|
171
|
+
| `payload_maxlength` | `int \| None` | length limit for `body`, defaults to 4096 characters |
|
|
172
|
+
| `use_default_sanitize_keys` | `bool` | mask common secret names at any depth, default `True` |
|
|
173
|
+
|
|
174
|
+
Levels, indent and the body limit fall back to the global `grpc_client` settings when omitted — set
|
|
175
|
+
them once for the whole project with `grpc_client.configure_logging(...)`. An argument passed to
|
|
176
|
+
`with_structlogger` wins over the global value. The `GRPC_CLIENT_*` environment variables do nothing
|
|
177
|
+
on their own: `configure_logging(...)` reads them when it is called, so a project that never calls it
|
|
178
|
+
gets library defaults regardless of the environment.
|
|
179
|
+
|
|
180
|
+
`log_level` and `err_log_level` accept **only** `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. A
|
|
181
|
+
structlog bound logger picks its method by level name, and only those five have one. Anything else —
|
|
182
|
+
including a value coming from `configure_logging(...)` — raises `ValueError` while the client is being
|
|
183
|
+
built. The built-in `options.with_logger` accepts arbitrary integers; this is a deliberate divergence,
|
|
184
|
+
because the alternative is a `KeyError` from inside structlog that also stops the request from going
|
|
185
|
+
out at all.
|
|
186
|
+
|
|
187
|
+
## References
|
|
188
|
+
|
|
189
|
+
- `references/fields.md` — the field catalog, streaming behaviour, global defaults and env variables.
|
|
190
|
+
- `references/pitfalls.md` — JSON strings vs nested values, the `message` field, level filtering,
|
|
191
|
+
typing of `logger`, and how the option plugs into `grpc-client`.
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Field catalog and configuration
|
|
2
|
+
|
|
3
|
+
Two records per gRPC call. Field sets differ; `method`, `metadata` and `trace_id` are shared.
|
|
4
|
+
|
|
5
|
+
## Request — `grpc request sent`
|
|
6
|
+
|
|
7
|
+
| Field | Type | Value |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| `method` | `str` | full method name, `/package.Service/Method` |
|
|
10
|
+
| `body` | `str` | JSON object of the request message after masking |
|
|
11
|
+
| `metadata` | `str` | JSON object of the request metadata; **absent** when there was none |
|
|
12
|
+
| `trace_id` | `str` | UUID4 generated for this call |
|
|
13
|
+
|
|
14
|
+
The record is written **before** the call goes out, so it exists even when the call never reaches the
|
|
15
|
+
service.
|
|
16
|
+
|
|
17
|
+
## Response — `grpc response received`
|
|
18
|
+
|
|
19
|
+
The same event name covers a successful unary answer, a failed call, and the end of a response
|
|
20
|
+
stream. Which fields appear tells them apart:
|
|
21
|
+
|
|
22
|
+
| Field | Type | Present when |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| `method` | `str` | always |
|
|
25
|
+
| `status_code` | `str` | always — the status **name**, e.g. `OK`, `DEADLINE_EXCEEDED` |
|
|
26
|
+
| `body` | `str` | successful unary answer only |
|
|
27
|
+
| `details` | `str` | failed call only; may be an empty string if the service said nothing |
|
|
28
|
+
| `messages_count` | `int` | stream calls only, including a failed stream |
|
|
29
|
+
| `metadata` | `str` | trailing metadata was non-empty |
|
|
30
|
+
| `trace_id` | `str` | always — matches the request record |
|
|
31
|
+
|
|
32
|
+
`messages_count` is the only integer field. Everything composite (`body`, `metadata`) is serialized
|
|
33
|
+
into a JSON string, not passed as a nested dict.
|
|
34
|
+
|
|
35
|
+
A successful record is written at `log_level` (default `INFO`), a failed one at `err_log_level`
|
|
36
|
+
(default `ERROR`). "Failed" means any status other than `OK`, including `CANCELLED` after the caller
|
|
37
|
+
cancelled the stream itself.
|
|
38
|
+
|
|
39
|
+
## Streams
|
|
40
|
+
|
|
41
|
+
The body of a stream never lands in the log — reading it for the sake of logging would change the
|
|
42
|
+
behaviour of the call.
|
|
43
|
+
|
|
44
|
+
| Call type | Request `body` | Response fields |
|
|
45
|
+
|---|---|---|
|
|
46
|
+
| unary-unary | the message | `body` |
|
|
47
|
+
| unary-stream | the message | `messages_count` |
|
|
48
|
+
| stream-unary | `<streamed content>` | `body` |
|
|
49
|
+
| stream-stream | `<streamed content>` | `messages_count` |
|
|
50
|
+
|
|
51
|
+
`messages_count` counts messages actually read by the caller. Two consequences:
|
|
52
|
+
|
|
53
|
+
- a stream that is abandoned half-way logs the count read up to that point, with the status the call
|
|
54
|
+
ended on (usually `CANCELLED` or `DEADLINE_EXCEEDED`);
|
|
55
|
+
- on the async client, messages taken with `call.read()` instead of async iteration bypass the
|
|
56
|
+
counter — `grpc.aio` reads them past the interceptor's wrapper. `AsyncGRPCStreamResponse` iterates,
|
|
57
|
+
so `collect()` and `async for` count correctly.
|
|
58
|
+
|
|
59
|
+
The final stream record is written when the call finishes, not when the last message is read. On the
|
|
60
|
+
async client that happens in a separate task, which `AsyncGRPCClient.close()` waits for.
|
|
61
|
+
|
|
62
|
+
## Body rendering
|
|
63
|
+
|
|
64
|
+
The message is converted with `MessageToDict(always_print_fields_with_no_presence=True)`, masked, and
|
|
65
|
+
serialized with `ensure_ascii=False`. So field names appear in protobuf JSON style (`accessToken`, not
|
|
66
|
+
`access_token`), and unset scalar fields appear with their default values.
|
|
67
|
+
|
|
68
|
+
A protobuf `bytes` field is what `MessageToDict` makes of it — **Base64**, not text: `b"text"` is
|
|
69
|
+
logged as `"dGV4dA=="`. The `<двоичные данные>` marker never appears in a body; it belongs to binary
|
|
70
|
+
metadata values (`-bin` keys), which reach the JSON encoder as raw `bytes` and are decoded as UTF-8
|
|
71
|
+
when they can be.
|
|
72
|
+
|
|
73
|
+
Two failure modes replace the body:
|
|
74
|
+
|
|
75
|
+
| `body` holds | Meaning |
|
|
76
|
+
|---|---|
|
|
77
|
+
| `<не удалось подготовить тело сообщения>` | conversion, masking or serialization raised |
|
|
78
|
+
| `... <обрезано>` at the end | the JSON exceeded `payload_maxlength` (4096 characters by default) |
|
|
79
|
+
|
|
80
|
+
Both constants live in `grpc_client.utils.logutils` (`PAYLOAD_UNAVAILABLE`, `TRUNCATION_SUFFIX`).
|
|
81
|
+
A broken glom path in `sanitize_keys` is the usual cause of the first one — the request itself still
|
|
82
|
+
goes out, because the log is built defensively.
|
|
83
|
+
|
|
84
|
+
## Metadata rendering
|
|
85
|
+
|
|
86
|
+
Metadata is masked, then grouped into a JSON object. A repeated key keeps all of its values as a
|
|
87
|
+
list:
|
|
88
|
+
|
|
89
|
+
```json
|
|
90
|
+
{"x-region": ["ural", "siberia"], "authorization": "****"}
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
A dict would have kept only the last value, and the log would no longer match what was sent.
|
|
94
|
+
The field is omitted entirely when there is no metadata — an empty JSON object is never written.
|
|
95
|
+
|
|
96
|
+
## Masking mechanics
|
|
97
|
+
|
|
98
|
+
One pass over the payload, driven by the merged key list (`sanitize_keys` plus, unless disabled,
|
|
99
|
+
`DEFAULT_SANITIZE_KEYS`):
|
|
100
|
+
|
|
101
|
+
| Key shape | Applies to | Matches |
|
|
102
|
+
|---|---|---|
|
|
103
|
+
| no separators (`password`) | body | a field with that name at **any** depth |
|
|
104
|
+
| dot / star / brackets (`user.token`) | body | that glom path only |
|
|
105
|
+
| any | metadata | the whole key, case-insensitive — metadata is flat, paths are not parsed |
|
|
106
|
+
|
|
107
|
+
Name matching normalizes to snake_case, so `accessToken`, `AccessToken` and `access_token` all match
|
|
108
|
+
`access_token`, and a composite name ending in a secret (`client_secret`, `user_password`) matches
|
|
109
|
+
too. Substring matching is deliberately not used — it would mask a harmless `tokenizer`.
|
|
110
|
+
|
|
111
|
+
Default names: `authorization`, `proxy-authorization`, `cookie`, `set-cookie`, `x-api-key`, `api-key`,
|
|
112
|
+
`api_key`, `access_token`, `refresh_token`, `password`, `secret`, `token`.
|
|
113
|
+
|
|
114
|
+
Name-based masking replaces the whole value, so a nested message under a field named `token`
|
|
115
|
+
disappears entirely rather than being walked into. Traversal stops at 64 levels and replaces what is
|
|
116
|
+
left with `<превышена глубина вложенности>` — a recursive structure must not take the call down with it.
|
|
117
|
+
|
|
118
|
+
## Global defaults
|
|
119
|
+
|
|
120
|
+
`log_level`, `err_log_level`, `json_indent` and the payload length limit fall back to
|
|
121
|
+
`grpc_client.config.LOGGING_CONFIG` whenever they are not passed to `with_structlogger`. Fill that
|
|
122
|
+
singleton once at startup:
|
|
123
|
+
|
|
124
|
+
```python
|
|
125
|
+
import logging
|
|
126
|
+
|
|
127
|
+
from grpc_client import configure_logging
|
|
128
|
+
|
|
129
|
+
configure_logging(
|
|
130
|
+
log_level=logging.DEBUG,
|
|
131
|
+
err_log_level=logging.WARNING,
|
|
132
|
+
json_indent=2,
|
|
133
|
+
payload_log_maxlength=4096,
|
|
134
|
+
)
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Only `DEBUG`, `INFO`, `WARNING`, `ERROR` and `CRITICAL` are accepted for the two levels, here and in
|
|
138
|
+
`configure_logging(...)` alike — a level structlog has no name for raises `ValueError` when the client
|
|
139
|
+
is built.
|
|
140
|
+
|
|
141
|
+
Environment variables **override** the arguments of that function — but only while it runs. Nothing
|
|
142
|
+
reads them on import, so a project that never calls `configure_logging(...)` gets library defaults no
|
|
143
|
+
matter what the environment says:
|
|
144
|
+
|
|
145
|
+
| Variable | Field | Used by this library |
|
|
146
|
+
|---|---|---|
|
|
147
|
+
| `GRPC_CLIENT_LOG_LEVEL` | `log_level` | yes |
|
|
148
|
+
| `GRPC_CLIENT_ERR_LOG_LEVEL` | `err_log_level` | yes |
|
|
149
|
+
| `GRPC_CLIENT_JSON_INDENT` | `json_indent` | yes |
|
|
150
|
+
| `GRPC_CLIENT_PAYLOAD_LOG_MAXLENGTH` | `payload_log_maxlength` | yes — argument `payload_maxlength` |
|
|
151
|
+
| `GRPC_CLIENT_USE_LINE_BREAKS` | `use_line_breaks` | no — text-log only setting |
|
|
152
|
+
|
|
153
|
+
Precedence: argument of `with_structlogger` → env variable → `configure_logging` argument → library
|
|
154
|
+
default (the middle two only apply once `configure_logging(...)` has run). `use_line_breaks` is the only setting with no counterpart here — this library never joins
|
|
155
|
+
fields into one line.
|
|
156
|
+
|
|
157
|
+
## Event names
|
|
158
|
+
|
|
159
|
+
`grpc request sent` and `grpc response received`, exported as `sink.REQUEST_EVENT` and
|
|
160
|
+
`sink.RESPONSE_EVENT`. They are not part of the public API (`__init__` exports
|
|
161
|
+
`with_structlogger` only) — filter by them in log queries, but do not import them into production code.
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Pitfalls
|
|
2
|
+
|
|
3
|
+
Non-obvious behaviour that costs time when you hit it unprepared.
|
|
4
|
+
|
|
5
|
+
## The built-in option crashes on a filtering bound logger
|
|
6
|
+
|
|
7
|
+
This is the whole reason the library exists. `grpc_client.options.with_logger` calls
|
|
8
|
+
`logger.isEnabledFor(level)`. A logger from `structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(...))`
|
|
9
|
+
has `is_enabled_for` instead, so every call dies with:
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
AttributeError: 'BoundLoggerFilteringAtDebug' object has no attribute 'isEnabledFor'
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
`with_structlogger` asks through `is_enabled_for` and falls back to "enabled" when the logger cannot
|
|
16
|
+
answer. "Cannot answer" covers two cases: the method is absent, and the method **raises**. The second
|
|
17
|
+
one is real — `structlog.stdlib.BoundLogger` declares both spellings but forwards them to the logger
|
|
18
|
+
underneath, so
|
|
19
|
+
`structlog.configure(wrapper_class=structlog.stdlib.BoundLogger, logger_factory=structlog.PrintLoggerFactory())`
|
|
20
|
+
gives a logger that writes fine and explodes on the level question. Under the built-in option that
|
|
21
|
+
combination kills every call.
|
|
22
|
+
|
|
23
|
+
The check itself is not optional: building the body means `MessageToDict` plus masking plus
|
|
24
|
+
`json.dumps`, and doing that for a record nobody will keep is wasteful.
|
|
25
|
+
|
|
26
|
+
What is **not** guarded is the write itself. If the structlog processor chain raises, the exception
|
|
27
|
+
comes out of the call — unlike stdlib `logging`, structlog does not swallow errors of its own
|
|
28
|
+
pipeline, and hiding them would mean losing every record silently.
|
|
29
|
+
|
|
30
|
+
## The payload field is `body`, never `message`
|
|
31
|
+
|
|
32
|
+
Many pipelines end with `structlog.processors.EventRenamer("message")`, which moves `event` into
|
|
33
|
+
`message`. A field of the library's own named `message` would be overwritten by the event text and
|
|
34
|
+
disappear. So the payload is `body`, matching `http_client_structlog`. Do not rename it back.
|
|
35
|
+
|
|
36
|
+
## Composite fields are JSON strings, not nested values
|
|
37
|
+
|
|
38
|
+
`body` and `metadata` reach structlog already serialized. In a console renderer that reads fine; under
|
|
39
|
+
`JSONRenderer` the field is a string with escaped quotes inside the outer JSON:
|
|
40
|
+
|
|
41
|
+
```json
|
|
42
|
+
{"event": "grpc request sent", "body": "{\"login\": \"morgan\"}"}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
This is deliberate — nested structures would collide with field names of the surrounding application
|
|
46
|
+
context and would explode the index of a log store. If a collector must see the body as an object,
|
|
47
|
+
parse that string in a structlog processor of your own.
|
|
48
|
+
|
|
49
|
+
Consequence for `json_indent`: it puts real newlines inside those strings. Useful when a human reads
|
|
50
|
+
the console, harmful for a line-based collector — leave it unset in production.
|
|
51
|
+
|
|
52
|
+
## structlog does its own level filtering
|
|
53
|
+
|
|
54
|
+
`log_level` is just the level handed to `.log()`. If structlog (or the stdlib logging behind it) is
|
|
55
|
+
configured above that level, the record is dropped and the option looks broken. Lowering `log_level`
|
|
56
|
+
to `DEBUG` requires lowering the structlog threshold as well.
|
|
57
|
+
|
|
58
|
+
When **both** `log_level` and `err_log_level` are filtered out, the interceptor skips everything —
|
|
59
|
+
it does not even generate a `trace_id` or wrap the stream for counting.
|
|
60
|
+
|
|
61
|
+
## Only five log levels exist
|
|
62
|
+
|
|
63
|
+
`logging` lets you invent levels — `logger.log(15, ...)` is fine, and `options.with_logger` passes it
|
|
64
|
+
through. structlog does not: a bound logger maps the level to a method **name**, and only
|
|
65
|
+
`DEBUG` / `INFO` / `WARNING` / `ERROR` / `CRITICAL` have one. Both `structlog.stdlib.BoundLogger` and
|
|
66
|
+
the classes from `make_filtering_bound_logger` raise `KeyError: 15`.
|
|
67
|
+
|
|
68
|
+
So `with_structlogger` validates the two levels while the client is built and raises `ValueError`
|
|
69
|
+
instead. The check runs on the **resolved** values, so a bad level in
|
|
70
|
+
`grpc_client.configure_logging(...)` or in `GRPC_CLIENT_LOG_LEVEL` is caught too. Without it the
|
|
71
|
+
failure would land on the first RPC, during the request record — before the call goes out, so the
|
|
72
|
+
request would never be sent.
|
|
73
|
+
|
|
74
|
+
## `details` is the one field that is never masked
|
|
75
|
+
|
|
76
|
+
`body` and `metadata` go through the sanitizer. `details` — the service's error text — does not, and
|
|
77
|
+
`payload_maxlength` does not trim it either. It is a free-form string, and masking matches field names
|
|
78
|
+
and glom paths, so there is nothing to match on. `sanitize_keys=["details"]` does **not** help: that
|
|
79
|
+
list applies inside the payload, not to the record's own fields.
|
|
80
|
+
|
|
81
|
+
A service that does `context.abort(INVALID_ARGUMENT, f"bad token {token}")` leaks it. The built-in
|
|
82
|
+
`options.with_logger` behaves identically, so this is not a regression — but it is the one channel
|
|
83
|
+
where a secret reaches the log untouched.
|
|
84
|
+
|
|
85
|
+
## The final record of an unread sync stream comes from a grpcio thread
|
|
86
|
+
|
|
87
|
+
`_CountingCall` registers the finish log through grpcio's `add_done_callback`. When the caller is not
|
|
88
|
+
sitting inside `next()` — the stream was abandoned, hit its deadline, or the channel closed — grpcio
|
|
89
|
+
runs that callback on one of its **own** threads, and the record is written there. So the processor
|
|
90
|
+
chain and the sink get called from a thread the application never set up: they have to be thread-safe.
|
|
91
|
+
|
|
92
|
+
`structlog.contextvars` do survive that hop — verified on grpcio 1.83 for an abandoned stream, an
|
|
93
|
+
expired deadline, and a context bound in a worker thread: the finish record carried the same
|
|
94
|
+
`request_id` as the request record, even though a plain `threading.Thread` sees no contextvars at all.
|
|
95
|
+
Do not rely on it blindly across grpcio upgrades, but it is not a known hole today.
|
|
96
|
+
|
|
97
|
+
## Two logging options log everything twice
|
|
98
|
+
|
|
99
|
+
`with_structlogger` and `options.with_logger` each append their own interceptor and neither is aware
|
|
100
|
+
of the other. Both installed means two records per call in two different formats. Pick one.
|
|
101
|
+
|
|
102
|
+
## Order matters relative to auth
|
|
103
|
+
|
|
104
|
+
Interceptors run in the order they are added. Put `with_structlogger` **after**
|
|
105
|
+
`options.with_auth(...)`, otherwise the logged metadata is the request before the token was injected.
|
|
106
|
+
|
|
107
|
+
## `logger` accepts both bound logger flavours
|
|
108
|
+
|
|
109
|
+
The parameter type is `structlog.stdlib.BoundLogger | structlog.typing.FilteringBoundLogger`. The two
|
|
110
|
+
are not interchangeable for a type checker — the stdlib one spells the level check `getEffectiveLevel`
|
|
111
|
+
and the filtering protocol spells it `get_effective_level` — so neither alone would accept the other,
|
|
112
|
+
hence the union. At runtime the option only calls `.log(level, event, **fields)` and
|
|
113
|
+
`is_enabled_for(level)`, which any bound logger provides; a logger built by some other wrapper class
|
|
114
|
+
still works, it just needs a `cast` to type-check.
|
|
115
|
+
|
|
116
|
+
## The counter and the async client
|
|
117
|
+
|
|
118
|
+
`messages_count` counts what the caller actually read. On the async client, `call.read()` goes past
|
|
119
|
+
the interceptor's wrapper, so those messages are not counted; async iteration (`async for`,
|
|
120
|
+
`AsyncGRPCStreamResponse.collect()`) is counted correctly.
|
|
121
|
+
|
|
122
|
+
## The async stream record arrives after the call, not after the last message
|
|
123
|
+
|
|
124
|
+
It is written by a task scheduled from the call's completion callback. Assert on it only after
|
|
125
|
+
`await client.close()` (or after leaving `async with`), which drains those tasks. A test that checks
|
|
126
|
+
the stream record right after `collect()` is racy.
|
|
127
|
+
|
|
128
|
+
## Masking never touches the message the caller receives
|
|
129
|
+
|
|
130
|
+
Every masking pass builds a copy. The response message still holds the real values — `****` exists
|
|
131
|
+
only in the log. Do not use the option as a way to strip secrets out of a payload.
|
|
132
|
+
|
|
133
|
+
## `sanitize_keys` mixes two mechanisms
|
|
134
|
+
|
|
135
|
+
A key without separators is a **name** matched at any depth; a key with a dot, star or brackets is an
|
|
136
|
+
exact **glom path**. `password` masks the field everywhere, `user.password` masks exactly one place.
|
|
137
|
+
For metadata neither applies literally: metadata is flat, so the whole key is compared,
|
|
138
|
+
case-insensitively, and a glom-looking key is matched as a plain key name.
|
|
139
|
+
|
|
140
|
+
A path that matches nothing is skipped silently, so a typo shows up as an unmasked field, not as an
|
|
141
|
+
error.
|
|
142
|
+
|
|
143
|
+
## A broken glom path costs the whole body
|
|
144
|
+
|
|
145
|
+
Unlike `http_client_structlog`, where each field is protected separately, the gRPC body is one unit:
|
|
146
|
+
if masking or serialization raises, `body` becomes `<не удалось подготовить тело сообщения>` and no
|
|
147
|
+
part of the payload is logged. The call itself still goes out — the log is built defensively, so a
|
|
148
|
+
logging failure never breaks a request. `metadata` is unaffected and stays correct.
|
|
149
|
+
|
|
150
|
+
## How the option plugs into grpc-client
|
|
151
|
+
|
|
152
|
+
`with_structlogger` installs `StructlogSink` through `options.with_log_sink` — the public extension
|
|
153
|
+
point added in `grpc-client` 2.1. Nothing here subclasses an interceptor.
|
|
154
|
+
|
|
155
|
+
The split is: `grpc-client` decides **what happened** — it intercepts all four call types, wraps
|
|
156
|
+
response streams to count messages, reads the outcome, builds a `RequestRecord` / `ResponseRecord`
|
|
157
|
+
and picks the level. `StructlogSink` decides **how it is written** — masking, JSON serialization,
|
|
158
|
+
truncation, and the field names.
|
|
159
|
+
|
|
160
|
+
`ResponseRecord` covers three outcomes with one shape, and the sink tells them apart the same way
|
|
161
|
+
the built-in `TextLogSink` does:
|
|
162
|
+
|
|
163
|
+
| Outcome | `status_code` | `message` | `messages_count` | Parser used |
|
|
164
|
+
|---|---|---|---|---|
|
|
165
|
+
| unary success | `OK` | the response | `None` | `parse_response` |
|
|
166
|
+
| stream finished | `OK` | `None` | count | `parse_stream_response` |
|
|
167
|
+
| any failure | the error code | `None` | count for streams, `None` for unary | `parse_error` |
|
|
168
|
+
|
|
169
|
+
Everything the sink uses is public: `RequestRecord` / `ResponseRecord` / `resolve_log_levels` from
|
|
170
|
+
`grpc_client`, `sanitize` / `sanitize_metadata` from `grpc_client.utils.sanitizer`, and
|
|
171
|
+
`LogFormat` / `BytesEncoder` / the marker constants from `grpc_client.utils.logutils`.
|
|
172
|
+
|
|
173
|
+
## Only `with_structlogger` is public API
|
|
174
|
+
|
|
175
|
+
`__init__` exports that one name. `sink`, `parser` and the constants inside them are internal
|
|
176
|
+
and can move. Filter logs by the event strings if you must, but do not import them.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
/docs/plans/
|
|
2
|
+
.venv/
|
|
3
|
+
.venvs/
|
|
4
|
+
__pycache__/
|
|
5
|
+
*.py[cod]
|
|
6
|
+
.pytest_cache/
|
|
7
|
+
.mypy_cache/
|
|
8
|
+
.ruff_cache/
|
|
9
|
+
.hypothesis/
|
|
10
|
+
.coverage
|
|
11
|
+
coverage.xml
|
|
12
|
+
htmlcov/
|
|
13
|
+
report*.xml
|
|
14
|
+
allure-results/
|
|
15
|
+
allure-report/
|
|
16
|
+
.allure/
|
|
17
|
+
logs/
|
|
18
|
+
*.log
|
|
19
|
+
.env
|
|
20
|
+
.env.*
|
|
21
|
+
.DS_Store
|
|
22
|
+
dist/
|
|
23
|
+
build/
|
|
24
|
+
*.egg-info/
|
|
25
|
+
/packages/*/api/
|
|
26
|
+
.claude/settings.local.json
|