@adhd/apigen-python-env 0.1.5 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,954 @@
1
+ """
2
+ apigen_python.grpc_server — gRPC server for the Python apigen target.
3
+
4
+ Runs each exported Python function as a real gRPC unary method, generating
5
+ protobuf descriptors **in memory** (no .proto files written to disk) by
6
+ building per-method typed ``FileDescriptorProto`` objects from the operation's
7
+ JSON-Schema input descriptor.
8
+
9
+ Usage (apigen-serve-core py-grpc-serve-split — two-phase extract/serve split):
10
+ python3 -m apigen_python.grpc_server \\
11
+ --module <path.py> \\
12
+ --namespace <ns> \\
13
+ --port <p> \\
14
+ --plan-file <plan.json>
15
+
16
+ This server no longer self-extracts, and no longer derives its own
17
+ package/service/method names. `--plan-file` is REQUIRED and points to a
18
+ JSON file of the shape `{"operations": [...Operation dicts, exactly what
19
+ `apigen_python.extractor --emit-json` emits...], "grpc": {"<opId>":
20
+ {"package": "<dotted.package>", "service": "<PascalService>", "method":
21
+ "<PascalMethod>"}}}`, produced by the TS `py-grpc` plugin's two-phase
22
+ spawn: (1) spawn `apigen_python.extractor --emit-json` in a short-lived
23
+ process to get `Operation[]`, (2) call the REAL
24
+ `@adhd/apigen-engine-naming` `project(op).grpc` on each op — the SAME
25
+ canonical projector every other transport (fastify/express/mcp/cli) uses
26
+ — (3) spawn THIS server with the result. See
27
+ `packages/apigen/apigen-plugin-py-grpc/src/lib/plugin.ts`.
28
+
29
+ Service layout (apigen-engine-naming's `project(op).grpc` rule — computed
30
+ TS-side and injected via `--plan-file`; this module contains no Python port
31
+ of the formula):
32
+ package = dotted, snake_cased, all path segments except the method
33
+ service = Pascal-cased "file" segment (second-to-last of [namespace, ...path])
34
+ method = Pascal-cased "export" segment (last of [namespace, ...path])
35
+
36
+ Example: namespace='pkg', source file 'grpc_api.py', fn='add_decimal'
37
+ → package: 'pkg.grpc_api', service: 'GrpcApi', method: 'AddDecimal'
38
+ → full method path: pkg.grpc_api.GrpcApi/AddDecimal
39
+ → grpcurl call:
40
+ grpcurl -plaintext \\
41
+ -d '{"data":{"amount":"123.456"}}' \\
42
+ localhost:50051 pkg.grpc_api.GrpcApi/AddDecimal
43
+
44
+ A single server process serves exactly one Python source file, so every
45
+ operation it exposes shares the same "file" path segment — and therefore
46
+ the same projected package + service — by construction; `_build_state()`
47
+ asserts this invariant rather than silently picking one when the
48
+ injected plan ever disagreed (it never should for a single-module server).
49
+
50
+ Wire contract (canonical apigen logical-type tenet):
51
+ Request message per method (generated from JSON-Schema input descriptor):
52
+ message <Method>Request {
53
+ <Method>Request.Data data = 1;
54
+ message Data {
55
+ string <param1> = 1; // string for decimal/date-time/uuid/str
56
+ int64 <param2> = 2; // integer JSON type
57
+ double <param3> = 3; // number JSON type
58
+ bool <param4> = 4; // boolean JSON type
59
+ }
60
+ }
61
+ Response message:
62
+ message <Method>Response {
63
+ string data = 1; // JSON-encoded result (string, decimal, RFC3339, etc.)
64
+ }
65
+
66
+ CANONICAL WIRE TENET (must never change):
67
+ date-time → string field, RFC3339 value (NOT protobuf Timestamp)
68
+ decimal → string field, decimal string value
69
+ int64 → string field, decimal string value (preserve precision)
70
+ uuid → string field, lowercase hyphenated
71
+ bytes → string field, base64 encoded
72
+ integer → int64 proto field
73
+ number → double proto field
74
+ boolean → bool proto field
75
+
76
+ Streaming (deferred scope — [fix:pygrpc-streaming-deferral]):
77
+ gRPC natively supports streaming, but this transport does not implement
78
+ it yet. `_build_state()` rejects any injected operation with
79
+ `streaming: true` with a clear `ValueError` rather than silently
80
+ mishandling it as a unary call. Implementing real gRPC streaming is
81
+ tracked separately and out of scope for this split.
82
+
83
+ Reflection:
84
+ grpc_reflection v1alpha is enabled unconditionally so grpcurl can list,
85
+ describe, and call methods without a local .proto file.
86
+
87
+ Startup signal:
88
+ Emits ``{"ready": true}`` on stdout once the gRPC server is accepting
89
+ connections — same §13.1 readiness protocol as flask_server.py.
90
+
91
+ What serve.ts needs to mount a gRPC host:
92
+ - HTTP/2 front (gRPC is already HTTP/2 + length-prefixed framing)
93
+ - Route pattern: ``/<package>.<Service>/<Method>``
94
+ - Trailer-based error: ``grpc-status`` + ``grpc-message`` trailers
95
+ - Metadata passthrough: ``x-adhd-*`` request metadata → envelope/ctx dict
96
+ - No gatewayCode mapping — gRPC status codes are the canonical errors
97
+ - gRPC-Web (via sonora/grpclib) is a stretch goal — not included here;
98
+ see BACKLOG for the pure-Python gRPC-Web option
99
+ """
100
+
101
+ from __future__ import annotations
102
+
103
+ import argparse
104
+ import importlib.util
105
+ import json
106
+ import signal as _signal
107
+ import sys
108
+ import threading
109
+ from concurrent import futures
110
+ from pathlib import Path
111
+ from typing import Any
112
+
113
+ # ---------------------------------------------------------------------------
114
+ # Bootstrap: ensure the package is importable when run as __main__
115
+ # ---------------------------------------------------------------------------
116
+
117
+ _THIS_DIR = Path(__file__).resolve().parent.parent
118
+ if str(_THIS_DIR) not in sys.path:
119
+ sys.path.insert(0, str(_THIS_DIR))
120
+
121
+ from apigen_python.errors import ApiError, GRPC_CODE # noqa: E402
122
+ from apigen_python.runtime import HostRequest, Runtime # noqa: E402
123
+ from apigen_python.validator import validate, ValidationError # noqa: E402
124
+
125
+ # apigen_logical lives one level up from the package (repo layout)
126
+ try:
127
+ import apigen_logical # type: ignore[import]
128
+ _HAS_LOGICAL = True
129
+ except ImportError:
130
+ _HAS_LOGICAL = False
131
+
132
+ # ---------------------------------------------------------------------------
133
+ # grpcio / grpcio-reflection imports — fail clearly if absent
134
+ # ---------------------------------------------------------------------------
135
+
136
+ try:
137
+ import grpc # type: ignore[import]
138
+ from grpc_reflection.v1alpha import reflection # type: ignore[import]
139
+ _HAS_GRPC = True
140
+ _GRPC_IMPORT_ERR: str = ""
141
+ except ImportError as _err:
142
+ _HAS_GRPC = False
143
+ _GRPC_IMPORT_ERR = str(_err)
144
+
145
+ # ---------------------------------------------------------------------------
146
+ # Logical-type helpers (mirrors flask_server pattern)
147
+ # ---------------------------------------------------------------------------
148
+
149
+ def _decode_params(data: dict[str, Any], input_schema: dict[str, Any]) -> dict[str, Any]:
150
+ """Schema-driven decode of each wire parameter to its native Python type.
151
+
152
+ Only runs when apigen_logical is available and the schema carries ``format``
153
+ annotations. Plain string/int/bool params pass through unchanged.
154
+ """
155
+ if not _HAS_LOGICAL or not input_schema:
156
+ return data
157
+ props: dict[str, Any] = input_schema.get("properties", {})
158
+ if not props:
159
+ return data
160
+ decoded: dict[str, Any] = {}
161
+ for key, val in data.items():
162
+ prop_schema = props.get(key, {})
163
+ decoded[key] = apigen_logical.decode(val, prop_schema)
164
+ return decoded
165
+
166
+
167
+ def _encode_result(value: Any) -> Any:
168
+ """Encode a native Python return value to its canonical wire form.
169
+
170
+ Uses apigen_logical.encode_value when available. Falls back to handling
171
+ the most common types (datetime, Decimal, UUID, bytes) directly.
172
+ """
173
+ if _HAS_LOGICAL:
174
+ return apigen_logical.encode_value(value)
175
+ from datetime import datetime
176
+ from decimal import Decimal
177
+ from uuid import UUID
178
+ if isinstance(value, datetime):
179
+ from datetime import timezone
180
+ if value.tzinfo is not None:
181
+ value = value.astimezone(timezone.utc)
182
+ else:
183
+ value = value.replace(tzinfo=timezone.utc)
184
+ base = value.strftime("%Y-%m-%dT%H:%M:%S")
185
+ ms = value.microsecond // 1000
186
+ return f"{base}.{ms:03d}Z"
187
+ if isinstance(value, Decimal):
188
+ return str(value)
189
+ if isinstance(value, UUID):
190
+ return str(value).lower()
191
+ if isinstance(value, (bytes, bytearray)):
192
+ import base64
193
+ return base64.b64encode(bytes(value)).decode("ascii")
194
+ if isinstance(value, dict):
195
+ return {k: _encode_result(v) for k, v in value.items()}
196
+ if isinstance(value, list):
197
+ return [_encode_result(v) for v in value]
198
+ return value
199
+
200
+
201
+ def _json_dumps(obj: Any) -> str:
202
+ """Serialise to JSON using ApigenEncoder when available."""
203
+ if _HAS_LOGICAL:
204
+ return json.dumps(obj, cls=apigen_logical.ApigenEncoder)
205
+ return json.dumps(obj)
206
+
207
+
208
+ # ---------------------------------------------------------------------------
209
+ # gRPC ApiError → grpc.StatusCode mapping
210
+ # ---------------------------------------------------------------------------
211
+
212
+ def _grpc_status_for(code: str) -> Any:
213
+ """Map an apigen ApiErrorCode string to a grpc.StatusCode."""
214
+ _map = {
215
+ "invalid_argument": grpc.StatusCode.INVALID_ARGUMENT,
216
+ "unauthenticated": grpc.StatusCode.UNAUTHENTICATED,
217
+ "permission_denied": grpc.StatusCode.PERMISSION_DENIED,
218
+ "not_found": grpc.StatusCode.NOT_FOUND,
219
+ "internal": grpc.StatusCode.INTERNAL,
220
+ }
221
+ return _map.get(code, grpc.StatusCode.INTERNAL)
222
+
223
+
224
+ # ---------------------------------------------------------------------------
225
+ # JSON-Schema → proto3 field type mapping
226
+ # ---------------------------------------------------------------------------
227
+
228
+ def _json_type_to_proto_field_type(schema: dict[str, Any]) -> int:
229
+ """Map a JSON Schema type fragment to a proto3 FieldDescriptorProto TYPE_*.
230
+
231
+ Logical-type mappings (all map to string to preserve the canonical wire):
232
+ format:decimal → TYPE_STRING (decimal string "123.456")
233
+ format:date-time → TYPE_STRING (RFC3339 string)
234
+ format:uuid → TYPE_STRING (lowercase hyphenated string)
235
+ type:integer → TYPE_INT64
236
+ type:number → TYPE_DOUBLE
237
+ type:boolean → TYPE_BOOL
238
+ type:string → TYPE_STRING (default)
239
+ (anything else) → TYPE_STRING
240
+ """
241
+ from google.protobuf import descriptor_pb2 # type: ignore[import]
242
+
243
+ t = schema.get("type", "string")
244
+ # Note: integer maps to TYPE_INT64 for exact round-trips.
245
+ # All logical string formats (decimal, date-time, uuid, byte) stay TYPE_STRING.
246
+ if t == "integer":
247
+ return descriptor_pb2.FieldDescriptorProto.TYPE_INT64
248
+ if t == "number":
249
+ return descriptor_pb2.FieldDescriptorProto.TYPE_DOUBLE
250
+ if t == "boolean":
251
+ return descriptor_pb2.FieldDescriptorProto.TYPE_BOOL
252
+ return descriptor_pb2.FieldDescriptorProto.TYPE_STRING
253
+
254
+
255
+ # ---------------------------------------------------------------------------
256
+ # In-memory FileDescriptorProto construction
257
+ #
258
+ # Per-method typed messages for ergonomic grpcurl calls, named from the
259
+ # INJECTED plan's Pascal-cased `method` (never the raw Python function name):
260
+ #
261
+ # message AddDecimalRequest {
262
+ # AddDecimalRequest.Data data = 1;
263
+ # message Data { string amount = 1; }
264
+ # }
265
+ # message AddDecimalResponse { string data = 1; }
266
+ #
267
+ # grpcurl call: -d '{"data":{"amount":"123.456"}}'
268
+ # ---------------------------------------------------------------------------
269
+
270
+ def _build_file_descriptor_proto(
271
+ package: str,
272
+ service_name: str,
273
+ operations: list[dict[str, Any]],
274
+ grpc_map: dict[str, dict[str, str]],
275
+ ) -> Any:
276
+ """Build a FileDescriptorProto in memory for the given service.
277
+
278
+ Each operation's input JSON Schema properties become the fields of the
279
+ per-method ``Data`` sub-message. The response is always a
280
+ ``string data = 1`` field carrying the JSON-encoded result.
281
+
282
+ Args:
283
+ package: Proto package name — the plan's TS-computed
284
+ `project(op).grpc.package` (e.g. "pkg.grpc_api"),
285
+ shared by every op in a single-module server.
286
+ service_name: gRPC service name — the plan's TS-computed
287
+ `project(op).grpc.service` (e.g. "GrpcApi").
288
+ operations: List of operation dicts (from the injected plan).
289
+ grpc_map: `{opId: {"package", "service", "method"}}` from the
290
+ injected `--plan-file`, supplying each op's
291
+ Pascal-cased method name.
292
+
293
+ Returns:
294
+ A ``google.protobuf.descriptor_pb2.FileDescriptorProto``.
295
+ """
296
+ from google.protobuf import descriptor_pb2 # type: ignore[import]
297
+
298
+ file_proto = descriptor_pb2.FileDescriptorProto()
299
+ file_proto.name = f"{package}.proto"
300
+ file_proto.package = package
301
+ file_proto.syntax = "proto3"
302
+
303
+ for op in operations:
304
+ method_name: str = grpc_map[op["id"]]["method"]
305
+ input_schema: dict[str, Any] = op.get("input", {})
306
+ input_props: dict[str, Any] = input_schema.get("properties", {})
307
+
308
+ # --- Request message ---
309
+ req_msg = file_proto.message_type.add()
310
+ req_msg.name = f"{method_name}Request"
311
+
312
+ # Nested `Data` sub-message with typed per-param fields.
313
+ data_submsg = req_msg.nested_type.add()
314
+ data_submsg.name = "Data"
315
+ for field_num, (prop_name, prop_schema) in enumerate(
316
+ input_props.items(), start=1
317
+ ):
318
+ field = data_submsg.field.add()
319
+ field.name = prop_name
320
+ field.number = field_num
321
+ field.label = descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL
322
+ field.type = _json_type_to_proto_field_type(prop_schema)
323
+
324
+ # The outer `data` field pointing to the nested Data message.
325
+ data_field = req_msg.field.add()
326
+ data_field.name = "data"
327
+ data_field.number = 1
328
+ data_field.label = descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL
329
+ data_field.type = descriptor_pb2.FieldDescriptorProto.TYPE_MESSAGE
330
+ data_field.type_name = f".{package}.{method_name}Request.Data"
331
+
332
+ # --- Response message ---
333
+ resp_msg = file_proto.message_type.add()
334
+ resp_msg.name = f"{method_name}Response"
335
+ resp_field = resp_msg.field.add()
336
+ resp_field.name = "data"
337
+ resp_field.number = 1
338
+ resp_field.label = descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL
339
+ resp_field.type = descriptor_pb2.FieldDescriptorProto.TYPE_STRING
340
+
341
+ # --- Service method ---
342
+
343
+ # Service definition (must come after all message types).
344
+ service = file_proto.service.add()
345
+ service.name = service_name
346
+ for op in operations:
347
+ method_name = grpc_map[op["id"]]["method"]
348
+ method = service.method.add()
349
+ method.name = method_name
350
+ method.input_type = f".{package}.{method_name}Request"
351
+ method.output_type = f".{package}.{method_name}Response"
352
+
353
+ return file_proto
354
+
355
+
356
+ def _build_descriptor_pool(
357
+ package: str,
358
+ service_name: str,
359
+ operations: list[dict[str, Any]],
360
+ grpc_map: dict[str, dict[str, str]],
361
+ ) -> tuple[Any, dict[str, Any], dict[str, Any]]:
362
+ """Build a DescriptorPool + message classes for all operations.
363
+
364
+ Args:
365
+ package: Proto package name (plan-computed).
366
+ service_name: gRPC service name (plan-computed).
367
+ operations: Extracted operation descriptors (from the injected plan).
368
+ grpc_map: `{opId: {"package", "service", "method"}}` from the
369
+ injected `--plan-file`.
370
+
371
+ Returns:
372
+ (pool, req_classes, resp_classes) where:
373
+ pool — a DescriptorPool pre-populated with the generated file,
374
+ suitable for passing to enable_server_reflection.
375
+ req_classes — {method_name → Request message class}
376
+ resp_classes — {method_name → Response message class}
377
+ """
378
+ from google.protobuf import descriptor_pool, message_factory # type: ignore[import]
379
+
380
+ file_proto = _build_file_descriptor_proto(package, service_name, operations, grpc_map)
381
+ pool = descriptor_pool.DescriptorPool()
382
+ pool.Add(file_proto)
383
+
384
+ req_classes: dict[str, Any] = {}
385
+ resp_classes: dict[str, Any] = {}
386
+ for op in operations:
387
+ method_name = grpc_map[op["id"]]["method"]
388
+ req_desc = pool.FindMessageTypeByName(f"{package}.{method_name}Request")
389
+ resp_desc = pool.FindMessageTypeByName(f"{package}.{method_name}Response")
390
+ req_classes[method_name] = message_factory.GetMessageClass(req_desc)
391
+ resp_classes[method_name] = message_factory.GetMessageClass(resp_desc)
392
+
393
+ return pool, req_classes, resp_classes
394
+
395
+
396
+ # ---------------------------------------------------------------------------
397
+ # Proto3 message → Python dict conversion
398
+ # ---------------------------------------------------------------------------
399
+
400
+ def _msg_to_dict(msg: Any) -> dict[str, Any]:
401
+ """Recursively convert a protobuf message to a Python dict.
402
+
403
+ Only processes LABEL_OPTIONAL scalar and message fields (no repeated or
404
+ map fields — apigen schemas don't use them at the top data level).
405
+
406
+ Args:
407
+ msg: A protobuf Message instance.
408
+
409
+ Returns:
410
+ A plain Python dict with field names as keys and Python-native values.
411
+ """
412
+ result: dict[str, Any] = {}
413
+ for field in msg.DESCRIPTOR.fields:
414
+ val = getattr(msg, field.name)
415
+ if field.type == field.TYPE_MESSAGE:
416
+ result[field.name] = _msg_to_dict(val)
417
+ else:
418
+ result[field.name] = val
419
+ return result
420
+
421
+
422
+ # ---------------------------------------------------------------------------
423
+ # Server state
424
+ #
425
+ # apigen-serve-core py-grpc-serve-split: package/service are no longer
426
+ # derived here. The previous scheme built the service name by upper-casing
427
+ # just the namespace's first character and lower-casing the rest, then
428
+ # appending a fixed literal suffix, duplicated at (at least) two call sites,
429
+ # and used the raw un-cased Python function identifier as the wire method
430
+ # name everywhere else — an inline algorithm unrelated to
431
+ # `@adhd/apigen-engine-naming`'s real gRPC projection. That whole scheme is
432
+ # DELETED; `package`/`service_name` and each operation's wire `method` name
433
+ # are now supplied by the injected `--plan-file` (`grpc_map`, TS-computed via
434
+ # the real `project(op).grpc` — see the module docstring's "Service layout").
435
+ # ---------------------------------------------------------------------------
436
+
437
+ class _ServerState:
438
+ """Immutable server configuration shared across all gRPC threads."""
439
+
440
+ __slots__ = (
441
+ "package", "service_name", "runtime", "operations",
442
+ "op_map", "input_schema_map", "method_names",
443
+ "req_classes", "resp_classes",
444
+ )
445
+
446
+ def __init__(
447
+ self,
448
+ package: str,
449
+ service_name: str,
450
+ runtime: Runtime,
451
+ operations: list[dict[str, Any]],
452
+ grpc_map: dict[str, dict[str, str]],
453
+ req_classes: dict[str, Any],
454
+ resp_classes: dict[str, Any],
455
+ ) -> None:
456
+ self.package = package
457
+ self.service_name = service_name
458
+ self.runtime = runtime
459
+ self.operations = operations
460
+ self.req_classes = req_classes
461
+ self.resp_classes = resp_classes
462
+
463
+ self.op_map: dict[str, dict[str, Any]] = {}
464
+ self.input_schema_map: dict[str, dict[str, Any]] = {}
465
+ self.method_names: list[str] = []
466
+
467
+ for op in operations:
468
+ method_name = grpc_map[op["id"]]["method"]
469
+ self.op_map[method_name] = op
470
+ self.input_schema_map[method_name] = op.get("input", {})
471
+ self.method_names.append(method_name)
472
+
473
+
474
+ # ---------------------------------------------------------------------------
475
+ # GenericRpcHandler — dispatches all service methods using typed proto messages
476
+ # ---------------------------------------------------------------------------
477
+
478
+ class _ApigenGrpcHandler(grpc.GenericRpcHandler if _HAS_GRPC else object): # type: ignore[misc]
479
+ """Generic gRPC handler for the apigen service.
480
+
481
+ Each gRPC method has fully typed request/response messages (built from the
482
+ operation's input JSON schema) so grpcurl can call methods as:
483
+
484
+ grpcurl -plaintext \\
485
+ -d '{"data":{"amount":"123.456"}}' \\
486
+ localhost:50051 pkg.grpc_api.GrpcApi/AddDecimal
487
+
488
+ The ``data`` field of the request is a typed sub-message whose fields
489
+ correspond to the function's parameters (all logical-type params use
490
+ ``string`` proto fields, carrying their canonical wire values).
491
+
492
+ The ``data`` field of the response is a plain ``string`` carrying the
493
+ JSON-encoded return value.
494
+ """
495
+
496
+ def __init__(self, state: "_ServerState") -> None:
497
+ self._state = state
498
+ self._full_service = f"{state.package}.{state.service_name}"
499
+
500
+ def service_name(self) -> str:
501
+ return self._full_service
502
+
503
+ def service(self, handler_call_details: Any) -> Any: # type: ignore[override]
504
+ """Return an RpcMethodHandler for the requested method, or None."""
505
+ method: str = handler_call_details.method
506
+ prefix = f"/{self._full_service}/"
507
+ if not method.startswith(prefix):
508
+ return None
509
+ method_name = method[len(prefix):]
510
+ if method_name not in self._state.op_map:
511
+ return None
512
+
513
+ req_cls = self._state.req_classes[method_name]
514
+ resp_cls = self._state.resp_classes[method_name]
515
+ method_name_captured = method_name
516
+
517
+ def _handle(request: Any, context: Any) -> Any:
518
+ result_json = self._dispatch(method_name_captured, request, context)
519
+ resp = resp_cls()
520
+ resp.data = result_json # type: ignore[attr-defined]
521
+ return resp
522
+
523
+ return grpc.unary_unary_rpc_method_handler(
524
+ _handle,
525
+ request_deserializer=req_cls.FromString,
526
+ response_serializer=lambda r: r.SerializeToString(),
527
+ )
528
+
529
+ def _dispatch(self, method_name: str, request: Any, context: Any) -> str:
530
+ """Extract params from the typed request → validate → invoke → encode.
531
+
532
+ Args:
533
+ method_name: The wire gRPC method name (plan-computed, Pascal-cased).
534
+ request: The decoded request proto message instance.
535
+ context: The grpc.ServicerContext for abort/metadata.
536
+
537
+ Returns:
538
+ JSON-encoded result string (placed in response ``data`` field).
539
+ Returns "" and calls ``context.abort()`` on any error.
540
+ """
541
+ # Convert the typed ``data`` sub-message to a plain Python dict.
542
+ req_dict = _msg_to_dict(request)
543
+ data = req_dict.get("data", {})
544
+ if not isinstance(data, dict):
545
+ context.abort(
546
+ grpc.StatusCode.INVALID_ARGUMENT,
547
+ '"data" sub-message could not be decoded to a dict',
548
+ )
549
+ return ""
550
+
551
+ # Extract x-adhd-* from gRPC metadata → envelope (mirrors HTTP headers).
552
+ envelope: dict[str, Any] = {}
553
+ try:
554
+ for key, value in context.invocation_metadata():
555
+ if key.startswith("x-adhd-"):
556
+ field = key[len("x-adhd-"):]
557
+ if field:
558
+ envelope[field] = value
559
+ except Exception:
560
+ pass
561
+
562
+ # Pre-dispatch input validation (same gate as flask_server).
563
+ input_schema = self._state.input_schema_map.get(method_name, {})
564
+ if input_schema:
565
+ try:
566
+ validate(input_schema, data)
567
+ except ValidationError as exc:
568
+ context.abort(
569
+ grpc.StatusCode.INVALID_ARGUMENT,
570
+ f"input validation failed: {exc}",
571
+ )
572
+ return ""
573
+
574
+ # Schema-driven decode for logical types (Decimal, datetime, etc.).
575
+ decoded_data = _decode_params(data, input_schema)
576
+
577
+ host_req = HostRequest(
578
+ operation=self._state.op_map[method_name],
579
+ data=decoded_data,
580
+ envelope=envelope,
581
+ transport="grpc",
582
+ pre_validated=True,
583
+ )
584
+
585
+ try:
586
+ result = self._state.runtime.invoke_sync(host_req)
587
+ except ApiError as exc:
588
+ context.abort(_grpc_status_for(exc.code), exc.message)
589
+ return ""
590
+ except Exception as exc:
591
+ context.abort(grpc.StatusCode.INTERNAL, f"dispatch error: {exc}")
592
+ return ""
593
+
594
+ # Encode the result to its canonical wire form.
595
+ wire_result = _encode_result(result)
596
+ try:
597
+ return _json_dumps(wire_result)
598
+ except Exception as exc:
599
+ context.abort(grpc.StatusCode.INTERNAL, f"result encode error: {exc}")
600
+ return ""
601
+
602
+
603
+ # ---------------------------------------------------------------------------
604
+ # Server lifecycle
605
+ # ---------------------------------------------------------------------------
606
+
607
+ class ApigenGrpcServer:
608
+ """gRPC server that exposes a Python module as a typed gRPC service.
609
+
610
+ Each exported function becomes a unary gRPC method with fully typed
611
+ request/response messages generated from its JSON-Schema input descriptor.
612
+
613
+ Method path: ``/<package>.<Service>/<Method>`` (plan-computed — see the
614
+ module docstring's "Service layout").
615
+
616
+ grpcurl example::
617
+
618
+ grpcurl -plaintext \\
619
+ -d '{"data":{"amount":"123.456"}}' \\
620
+ localhost:50051 \\
621
+ pkg.grpc_api.GrpcApi/AddDecimal
622
+
623
+ Wire contract (canonical apigen tenet):
624
+ All logical types keep their string wire form in the proto ``string``
625
+ field — ``date-time`` stays RFC3339, ``decimal`` stays a decimal string.
626
+ No ``google.protobuf.Timestamp`` is used.
627
+ """
628
+
629
+ def __init__(
630
+ self,
631
+ module_path: str,
632
+ namespace: str,
633
+ plan: dict[str, Any],
634
+ host: str = "127.0.0.1",
635
+ port: int = 50051,
636
+ max_workers: int = 10,
637
+ ) -> None:
638
+ if not _HAS_GRPC:
639
+ raise ImportError(
640
+ f"grpcio / grpcio-reflection is required but could not be imported: "
641
+ f"{_GRPC_IMPORT_ERR}\n"
642
+ "Install with: pip install grpcio grpcio-tools grpcio-reflection"
643
+ )
644
+ self._module_path = module_path
645
+ self._namespace = namespace # informational only — package/service/method
646
+ # names are supplied by --plan-file (see _build_state()), never
647
+ # re-derived from `namespace` here.
648
+ self._plan = plan
649
+ self._host = host
650
+ self._port = port
651
+ self._max_workers = max_workers
652
+ self._server: Any = None
653
+
654
+ def _build_state(self) -> "_ServerState":
655
+ """Build descriptors + server state from the INJECTED `--plan-file` —
656
+ this server no longer self-extracts (apigen-serve-core
657
+ py-grpc-serve-split). `self._plan["operations"]` supplies the full
658
+ Operation[] (schema-bearing dicts, produced by
659
+ `apigen_python.extractor --emit-json` in a separate, short-lived
660
+ process — see plugin.ts's two-phase spawn) and
661
+ `self._plan["grpc"]` supplies each op's canonical
662
+ package/service/method, computed by the REAL
663
+ `@adhd/apigen-engine-naming` `project()` — never re-derived here.
664
+
665
+ The module is still loaded a SECOND time, independently, right here —
666
+ that load's only job is producing LIVE function references for the
667
+ runtime registry, which cannot cross a process boundary as data
668
+ (Python callables are not serialisable). This is unchanged from the
669
+ pre-split design and was confirmed safe by
670
+ `docs/apigen/proposals/py-extract-serve-split-findings.md` §1.2/§2.2.
671
+ """
672
+ ops: list[dict[str, Any]] = self._plan["operations"]
673
+ grpc_map: dict[str, dict[str, str]] = self._plan["grpc"]
674
+ if not ops:
675
+ raise ValueError(
676
+ f"--plan-file for {self._module_path!r} carries zero "
677
+ "operations. Ensure the module has public callable exports."
678
+ )
679
+
680
+ # gRPC-specific wrinkle (unchanged from pre-split design, confirmed
681
+ # by py-extract-serve-split-findings.md §2.2): only 'action'/
682
+ # 'constructor' kinds become gRPC methods; 'query' ops are dropped,
683
+ # falling back to all ops only if none qualify.
684
+ callable_ops = [op for op in ops if op.get("kind") in ("action", "constructor")]
685
+ if not callable_ops:
686
+ callable_ops = ops
687
+
688
+ # [fix:pygrpc-streaming-deferral] gRPC natively supports streaming,
689
+ # but this transport does not implement it. Reject explicitly rather
690
+ # than silently mishandling a streaming op as a plain unary call —
691
+ # this is a documented, already-tracked deferral, not a TODO here.
692
+ streaming_ops = [op for op in callable_ops if op.get("streaming")]
693
+ if streaming_ops:
694
+ ids = ", ".join(repr(op["id"]) for op in streaming_ops)
695
+ raise ValueError(
696
+ "apigen-py-grpc: streaming operations are not supported by "
697
+ f"this transport (deferred scope): {ids}. gRPC natively "
698
+ "supports streaming; implementing it here is out of scope "
699
+ "for the py-grpc-serve-split extract/serve split."
700
+ )
701
+
702
+ missing = [op["id"] for op in callable_ops if op["id"] not in grpc_map]
703
+ if missing:
704
+ raise ValueError(
705
+ f"apigen-py-grpc: --plan-file is missing a grpc entry for "
706
+ f"operation id(s) {missing!r} — the injected plan must "
707
+ f"cover every callable operation extract_module() (phase 1) "
708
+ f"produced, 1:1."
709
+ )
710
+
711
+ # A single server process serves exactly one Python source file, so
712
+ # every operation it exposes shares the same "file" path segment —
713
+ # and therefore the same projected package + service — by
714
+ # construction (see module docstring). Assert rather than silently
715
+ # pick one if that invariant were ever violated.
716
+ packages = {grpc_map[op["id"]]["package"] for op in callable_ops}
717
+ services = {grpc_map[op["id"]]["service"] for op in callable_ops}
718
+ if len(packages) != 1 or len(services) != 1:
719
+ raise ValueError(
720
+ "apigen-py-grpc: operations from a single module projected "
721
+ f"to multiple gRPC package/service pairs "
722
+ f"(packages={sorted(packages)!r}, services={sorted(services)!r}) "
723
+ "— a single server process can only host one file's worth "
724
+ "of gRPC package/service."
725
+ )
726
+ package = next(iter(packages))
727
+ service_name = next(iter(services))
728
+
729
+ # Load the module to get live function references.
730
+ # Register in sys.modules BEFORE exec_module (canonical importlib pattern)
731
+ # so dataclasses._is_type and typing.get_type_hints work correctly for
732
+ # modules that use `from __future__ import annotations` (PEP 563).
733
+ path = Path(self._module_path).resolve()
734
+ spec = importlib.util.spec_from_file_location("_apigen_grpc_module_", str(path))
735
+ if spec is None or spec.loader is None:
736
+ raise ImportError(f"Cannot load module from {path}")
737
+ mod = importlib.util.module_from_spec(spec)
738
+ import sys as _sys
739
+ _sys.modules[spec.name] = mod
740
+ try:
741
+ spec.loader.exec_module(mod) # type: ignore[union-attr]
742
+ except Exception:
743
+ _sys.modules.pop(spec.name, None)
744
+ raise
745
+
746
+ registry: dict[str, Any] = {}
747
+ for op in callable_ops:
748
+ fn_name = op["path"][-1]["raw"]
749
+ fn = getattr(mod, fn_name, None)
750
+ if callable(fn):
751
+ registry[op["id"]] = fn
752
+
753
+ runtime = Runtime(registry)
754
+
755
+ pool, req_classes, resp_classes = _build_descriptor_pool(
756
+ package, service_name, callable_ops, grpc_map
757
+ )
758
+
759
+ return _ServerState(
760
+ package=package,
761
+ service_name=service_name,
762
+ runtime=runtime,
763
+ operations=callable_ops,
764
+ grpc_map=grpc_map,
765
+ req_classes=req_classes,
766
+ resp_classes=resp_classes,
767
+ )
768
+
769
+ def start(self) -> None:
770
+ """Build state + descriptors, create the gRPC server, start serving."""
771
+ state = self._build_state()
772
+
773
+ # Re-build the pool for reflection (identical FileDescriptorProto).
774
+ pool, _, _ = _build_descriptor_pool(
775
+ state.package, state.service_name, state.operations, self._plan["grpc"]
776
+ )
777
+
778
+ handler = _ApigenGrpcHandler(state)
779
+ server = grpc.server(
780
+ futures.ThreadPoolExecutor(max_workers=self._max_workers)
781
+ )
782
+ server.add_generic_rpc_handlers([handler])
783
+
784
+ # Register reflection with our custom pool so grpcurl list/describe works.
785
+ service_names = [f"{state.package}.{state.service_name}"]
786
+ try:
787
+ reflection.enable_server_reflection(service_names, server, pool=pool)
788
+ except Exception as exc:
789
+ print(
790
+ f"apigen-py-grpc WARNING: reflection setup failed: {exc}",
791
+ file=sys.stderr,
792
+ )
793
+
794
+ address = f"{self._host}:{self._port}"
795
+ # BUG-APIGEN-TEST-FIXED-PORT-COLLISION-001: grpc's add_insecure_port
796
+ # returns the ACTUAL bound port — identical to the requested port
797
+ # unless 0 (OS-assigned ephemeral) was passed, in which case this is
798
+ # the only way to learn the real port. self._port is the single
799
+ # source of truth from here on (used by the log line below and by
800
+ # serve_forever()'s readiness signal).
801
+ self._port = server.add_insecure_port(address)
802
+ server.start()
803
+ self._server = server
804
+
805
+ # Print route summary to stderr (mirrors flask_server pattern).
806
+ pkg = state.package
807
+ svc = state.service_name
808
+ print(
809
+ f"apigen-py-grpc listening on grpc://{self._host}:{self._port}",
810
+ file=sys.stderr,
811
+ )
812
+ print(f" service: {pkg}.{svc}", file=sys.stderr)
813
+ for method_name in state.method_names:
814
+ print(f" method: /{pkg}.{svc}/{method_name}", file=sys.stderr)
815
+ sys.stderr.flush()
816
+
817
+ def stop(self, grace: float = 2.0) -> None:
818
+ """Shutdown the gRPC server gracefully."""
819
+ if self._server is not None:
820
+ self._server.stop(grace)
821
+ self._server = None
822
+
823
+ def serve_forever(self) -> None:
824
+ """Start the server and block until SIGTERM / SIGINT / Ctrl-C.
825
+
826
+ Emits ``{"ready": true, "port": <n>}`` on stdout immediately after
827
+ binding so the TS plugin subprocess launcher can detect readiness
828
+ (same §13.1 protocol as flask_server.py). ``port`` is the ACTUAL
829
+ bound port (BUG-APIGEN-TEST-FIXED-PORT-COLLISION-001) — identical to
830
+ the requested ``--port`` unless 0 (OS-assigned ephemeral) was passed,
831
+ in which case this is the only way the caller learns the real port.
832
+ """
833
+ self.start()
834
+ # §13.1 readiness signal — TS launcher polls stdout for this line.
835
+ print(json.dumps({"ready": True, "port": self._port}), flush=True)
836
+
837
+ stop_event = threading.Event()
838
+
839
+ def _sig_handler(signum: int, frame: Any) -> None:
840
+ stop_event.set()
841
+
842
+ _signal.signal(_signal.SIGTERM, _sig_handler)
843
+ _signal.signal(_signal.SIGINT, _sig_handler)
844
+
845
+ try:
846
+ stop_event.wait()
847
+ except KeyboardInterrupt:
848
+ pass
849
+ finally:
850
+ self.stop()
851
+
852
+
853
+ # ---------------------------------------------------------------------------
854
+ # Public convenience: build_server()
855
+ # ---------------------------------------------------------------------------
856
+
857
+ def build_server(
858
+ module_path: str,
859
+ namespace: str,
860
+ plan: dict[str, Any],
861
+ host: str = "127.0.0.1",
862
+ port: int = 50051,
863
+ ) -> "ApigenGrpcServer":
864
+ """Construct an :class:`ApigenGrpcServer` for the given module.
865
+
866
+ Args:
867
+ module_path: Path to the ``.py`` source file (absolute or relative).
868
+ namespace: The apigen namespace slug (informational only — the
869
+ served gRPC package/service/method names come from
870
+ `plan["grpc"]`, never re-derived from this value).
871
+ plan: The TS-computed serve plan (`{"operations": [...],
872
+ "grpc": {"<opId>": {"package": str, "service": str,
873
+ "method": str}}}`) — see the module docstring's
874
+ "Usage" section.
875
+ host: Bind address (default ``127.0.0.1``).
876
+ port: TCP port (default ``50051``).
877
+
878
+ Returns:
879
+ An :class:`ApigenGrpcServer` instance ready to call ``.start()`` or
880
+ ``.serve_forever()``.
881
+ """
882
+ return ApigenGrpcServer(module_path, namespace, plan, host=host, port=port)
883
+
884
+
885
+ # ---------------------------------------------------------------------------
886
+ # CLI entry point
887
+ # ---------------------------------------------------------------------------
888
+
889
+ def _main() -> None:
890
+ parser = argparse.ArgumentParser(
891
+ description=(
892
+ "apigen Python gRPC server — serves a .py module over gRPC.\n\n"
893
+ "Service layout (TS-computed via @adhd/apigen-engine-naming's\n"
894
+ "project().grpc and injected via --plan-file; apigen-serve-core\n"
895
+ "py-grpc-serve-split):\n"
896
+ " package: <dotted, all path segments but the method>\n"
897
+ " service: <Pascal file segment>\n"
898
+ " methods: /<package>.<Service>/<PascalMethod>\n\n"
899
+ "Wire encoding (canonical apigen tenet):\n"
900
+ " Request: typed sub-message per function\n"
901
+ " date-time/decimal/uuid → string fields\n"
902
+ " integer → int64, number → double\n"
903
+ " Response: message { string data = 1; } (JSON-encoded result)\n\n"
904
+ " date-time → RFC3339 string (NOT protobuf Timestamp)\n"
905
+ " decimal → decimal string\n\n"
906
+ "Startup: emits {ready: true} on stdout once the server is up.\n\n"
907
+ "grpcurl example:\n"
908
+ " grpcurl -plaintext \\\n"
909
+ " -d '{\"data\":{\"amount\":\"123.456\"}}' \\\n"
910
+ " localhost:50051 pkg.grpc_api.GrpcApi/AddDecimal\n"
911
+ ),
912
+ formatter_class=argparse.RawDescriptionHelpFormatter,
913
+ )
914
+ parser.add_argument("--module", required=True, help="Path to the Python source module")
915
+ parser.add_argument(
916
+ "--namespace", required=True,
917
+ help="Namespace slug (informational only; wire naming comes from --plan-file)",
918
+ )
919
+ parser.add_argument("--host", default="127.0.0.1", help="Bind host (default: 127.0.0.1)")
920
+ parser.add_argument(
921
+ "--port", type=int, default=50051,
922
+ help="TCP port (default: 50051; pass 0 to bind an OS-assigned "
923
+ "ephemeral port — the actual bound port is reported back in "
924
+ "the {ready: true, port: <n>} stdout line)",
925
+ )
926
+ parser.add_argument(
927
+ "--plan-file", required=True,
928
+ help=(
929
+ "Path to a JSON file: {\"operations\": [...Operation dicts, exactly "
930
+ "apigen_python.extractor --emit-json's output...], \"grpc\": "
931
+ "{\"<opId>\": {\"package\": str, \"service\": str, \"method\": str}}}. "
932
+ "REQUIRED — this server no longer self-extracts or re-derives "
933
+ "package/service/method (apigen-serve-core py-grpc-serve-split); "
934
+ "produced by the py-grpc TS plugin's two-phase spawn (extractor "
935
+ "--emit-json -> @adhd/apigen-engine-naming project() -> this flag)."
936
+ ),
937
+ )
938
+ args = parser.parse_args()
939
+
940
+ with open(args.plan_file, "r", encoding="utf-8") as plan_fh:
941
+ plan = json.load(plan_fh)
942
+
943
+ server = build_server(
944
+ module_path=args.module,
945
+ namespace=args.namespace,
946
+ plan=plan,
947
+ host=args.host,
948
+ port=args.port,
949
+ )
950
+ server.serve_forever()
951
+
952
+
953
+ if __name__ == "__main__":
954
+ _main()