leangeo 3.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. leangeo-3.0.0/.cargo/config.toml +7 -0
  2. leangeo-3.0.0/.gitignore +39 -0
  3. leangeo-3.0.0/CLIENT.md +398 -0
  4. leangeo-3.0.0/Cargo.lock +737 -0
  5. leangeo-3.0.0/Cargo.toml +35 -0
  6. leangeo-3.0.0/LICENSE +21 -0
  7. leangeo-3.0.0/MANIFEST.in +8 -0
  8. leangeo-3.0.0/PKG-INFO +446 -0
  9. leangeo-3.0.0/README.md +408 -0
  10. leangeo-3.0.0/client.py +21 -0
  11. leangeo-3.0.0/leangeo_engine/__init__.py +721 -0
  12. leangeo-3.0.0/leangeo_engine/_compat.py +177 -0
  13. leangeo-3.0.0/leangeo_engine/_guard.py +782 -0
  14. leangeo-3.0.0/leangeo_pb2.py +20 -0
  15. leangeo-3.0.0/leangeo_pb2_grpc.py +20 -0
  16. leangeo-3.0.0/leangeo_server/__init__.py +13 -0
  17. leangeo-3.0.0/leangeo_server/cli.py +81 -0
  18. leangeo-3.0.0/leangeo_server/client.py +308 -0
  19. leangeo-3.0.0/leangeo_server/leangeo.proto +109 -0
  20. leangeo-3.0.0/leangeo_server/leangeo_pb2.py +92 -0
  21. leangeo-3.0.0/leangeo_server/leangeo_pb2_grpc.py +700 -0
  22. leangeo-3.0.0/leangeo_server/py.typed +0 -0
  23. leangeo-3.0.0/leangeo_server/serve.py +693 -0
  24. leangeo-3.0.0/leangeo_server/serve_common.py +189 -0
  25. leangeo-3.0.0/leangeo_server/serve_grpc.py +517 -0
  26. leangeo-3.0.0/leangeo_server/serve_guardrails.py +714 -0
  27. leangeo-3.0.0/pyproject.toml +79 -0
  28. leangeo-3.0.0/serve.py +20 -0
  29. leangeo-3.0.0/serve_grpc.py +20 -0
  30. leangeo-3.0.0/serve_guardrails.py +20 -0
  31. leangeo-3.0.0/src/bitslice.rs +488 -0
  32. leangeo-3.0.0/src/grid_index.rs +310 -0
  33. leangeo-3.0.0/src/integrity.rs +192 -0
  34. leangeo-3.0.0/src/inverted_index.rs +239 -0
  35. leangeo-3.0.0/src/lib.rs +1979 -0
  36. leangeo-3.0.0/src/point.rs +187 -0
  37. leangeo-3.0.0/src/python.rs +545 -0
  38. leangeo-3.0.0/src/query.rs +488 -0
  39. leangeo-3.0.0/src/storage.rs +2163 -0
  40. leangeo-3.0.0/src/sysutil.rs +474 -0
  41. leangeo-3.0.0/src/tests.rs +424 -0
  42. leangeo-3.0.0/src/wal.rs +384 -0
@@ -0,0 +1,7 @@
1
+ # Align loop starts to 32 bytes on x86_64: a hot loop of up to 32 bytes then never
2
+ # straddles a 64-byte line. Without it, where each loop landed was luck of the build, and
3
+ # a straddling loop (UTF-8 validation, roaring intersections) ran 10-20% slower on Zen 2
4
+ # (measured with perf, HISTORY.md 2026-10-22). Cargo applies this to the release wheels
5
+ # too (maturin runs cargo in this directory).
6
+ [target.'cfg(target_arch = "x86_64")']
7
+ rustflags = ["-C", "llvm-args=-align-loops=32"]
@@ -0,0 +1,39 @@
1
+ /target
2
+ **/*.rs.bk
3
+ __pycache__/
4
+ *.py[cod]
5
+ *$py.class
6
+ *.so
7
+ .Python
8
+ build/
9
+ develop-eggs/
10
+ dist/
11
+ downloads/
12
+ eggs/
13
+ .eggs/
14
+ lib/
15
+ lib64/
16
+ parts/
17
+ sdist/
18
+ var/
19
+ wheels/
20
+ *.egg-info/
21
+ .installed.cfg
22
+ *.egg
23
+ *.mmap
24
+ *.db
25
+ .venv/
26
+ venv/
27
+
28
+ # test/debug artifacts
29
+ datetest/
30
+ *.commit
31
+ *.wal
32
+ *.lock
33
+ *.epoch
34
+ *.base
35
+ *.dels*.mmap
36
+ *.ids*.mmap
37
+ .pytest_cache/
38
+ !Cargo.lock
39
+ *.from2x
@@ -0,0 +1,398 @@
1
+ # LeanGeo API and Client Guide
2
+
3
+ LeanGeo can run as a network service over **gRPC** or **REST** (HTTP/JSON). Both servers
4
+ and a Python gRPC client ship in the wheel, in the `leangeo_server` package:
5
+
6
+ - gRPC server: `leangeo_server.serve_grpc` (command `leangeo-grpc`), service defined in
7
+ `leangeo_server/leangeo.proto`
8
+ - REST server: `leangeo_server.serve` (command `leangeo-serve`), FastAPI on uvicorn
9
+ - Python client: `leangeo_server.client.LeanGeoClient`
10
+
11
+ `import leangeo_engine` never imports any of this. In a source checkout, the root-level
12
+ `serve.py`, `serve_grpc.py`, `serve_guardrails.py`, `client.py`, `leangeo_pb2.py` and
13
+ `leangeo_pb2_grpc.py` are aliases of the packaged modules, so `uvicorn serve:app`,
14
+ `python serve_grpc.py` and `from client import LeanGeoClient` still work from the
15
+ repository root. The proto moved from `proto/leangeo.proto` to `leangeo_server/leangeo.proto`.
16
+
17
+ **Polygon vertex order changed meaning in 3.0 — read [Polygons](#polygons) if you used
18
+ the 2.x servers.**
19
+
20
+ ---
21
+
22
+ ## 1. Install
23
+
24
+ The engine has no server dependencies. Install the extras into the environment where
25
+ `leangeo` is installed:
26
+
27
+ ```bash
28
+ pip install "leangeo[server]" # REST server: fastapi, uvicorn[standard], pydantic>=2.8, orjson
29
+ pip install "leangeo[grpc]" # gRPC server and client: grpcio>=1.82.1, protobuf>=7.35,<8
30
+ ```
31
+
32
+ The generated stubs (`leangeo_pb2.py`, `leangeo_pb2_grpc.py`) ship in the wheel, so
33
+ `grpcio-tools` is only needed to regenerate them (section 9).
34
+
35
+ ---
36
+
37
+ ## 2. Run a server
38
+
39
+ ```bash
40
+ leangeo-serve --path /data/geo # REST, default 127.0.0.1:8003
41
+ leangeo-grpc --path /data/geo # gRPC, default 127.0.0.1:50053
42
+ ```
43
+
44
+ Equivalent forms: `LEANGEO_PATH=/data/geo python -m leangeo_server.serve_grpc`, and
45
+ `LEANGEO_PATH=/data/geo uvicorn leangeo_server.serve:app --workers 1` for REST (uvicorn
46
+ binds 127.0.0.1 unless you pass `--host`; see section 3 for what a non-loopback `--host`
47
+ requires).
48
+
49
+ Command-line flags only set the environment variables below, so flags and environment
50
+ configure the same thing. `leangeo-serve` takes `--path --capacity --host --port
51
+ --threads --allow-insecure`; `leangeo-grpc` takes `--path --capacity --host --port
52
+ --workers --tls-cert --tls-key --allow-insecure`. The bearer token is read from
53
+ `LEANGEO_API_TOKEN` only, so it never shows up in `ps`.
54
+
55
+ Each server opens one `LeanGeoEngine` for its lifetime and runs every request on a worker
56
+ thread (the gRPC `ThreadPoolExecutor`; FastAPI's threadpool for the REST endpoints, so no
57
+ engine call runs on the event loop). The native core releases the GIL, so reads run in
58
+ parallel; writes to one collection serialize. Run one process per data directory; the
59
+ engine is also safe across processes if you run several.
60
+
61
+ ### Configuration
62
+
63
+ | Variable | Default | Meaning |
64
+ |---|---|---|
65
+ | `LEANGEO_PATH` | `leangeo_data` | Data directory (one set of files per collection) |
66
+ | `LEANGEO_CAPACITY` | `100000` | Initial capacity hint for a new collection's files |
67
+ | `LEANGEO_HOST` / `LEANGEO_PORT` | `127.0.0.1` / `8003` | REST bind address (`HOST`/`PORT` are read when these are unset) |
68
+ | `LEANGEO_GRPC_HOST` / `LEANGEO_GRPC_PORT` | `127.0.0.1` / `50053` | gRPC bind address (`GRPC_HOST`/`GRPC_PORT` are read when these are unset) |
69
+ | `LEANGEO_THREADS` | `min(64, 2 x CPUs)` | REST worker threads |
70
+ | `LEANGEO_GRPC_WORKERS` | `min(64, 2 x CPUs)` | gRPC worker threads |
71
+ | `LEANGEO_API_TOKEN` | unset | Bearer token required on every endpoint/RPC except health |
72
+ | `LEANGEO_ALLOW_INSECURE` | unset | `1` permits non-loopback access without authentication |
73
+ | `LEANGEO_GRPC_TLS_CERT` / `LEANGEO_GRPC_TLS_KEY` | unset | PEM certificate chain and key; gRPC serves TLS when both are set |
74
+ | `LEANGEO_GRPC_MAX_CONCURRENT_RPCS` | unset (unbounded) | Running + queued RPCs; past it gRPC answers `RESOURCE_EXHAUSTED` immediately. Must be >= 1 |
75
+ | `LEANGEO_GRPC_KEEPALIVE_TIME_MS` / `..._TIMEOUT_MS` | `60000` / `20000` | gRPC keepalive pings; a peer that stops answering is dropped and its RPCs cancelled |
76
+ | `LEANGEO_WAL_SYNC_INTERVAL_MS` | unset | Unset: every acknowledged write is fsynced before the reply. N > 0: fsync within N ms (a crash may lose the last N ms) |
77
+
78
+ Guardrail variables are listed in section 7.
79
+
80
+ ### Shutdown
81
+
82
+ On SIGTERM or SIGINT both servers stop accepting work, let in-flight requests finish, and
83
+ close the engine, which checkpoints every collection (the WAL is folded into the data
84
+ files and emptied). The gRPC server waits up to 5 s for in-flight RPCs; REST waits up to
85
+ 30 s. From the moment shutdown begins, gRPC `Health` reports `ready: false` and other
86
+ RPCs answer `UNAVAILABLE`; REST `/health` reports `"ready": false` and other endpoints
87
+ answer 503.
88
+
89
+ ---
90
+
91
+ ## 3. Trust boundary, authentication, TLS
92
+
93
+ The API can read, delete and clear every collection and has no per-collection
94
+ authorization. Both servers therefore bind `127.0.0.1` by default, and the console
95
+ scripts refuse to start on any other address (exit with a message) unless one of these
96
+ holds:
97
+
98
+ - REST: `LEANGEO_API_TOKEN` is set. Every endpoint except `GET /health` then requires
99
+ `Authorization: Bearer <token>`; a missing or wrong token is 401. Tokens are compared
100
+ in constant time.
101
+ - gRPC: `LEANGEO_GRPC_TLS_CERT`, `LEANGEO_GRPC_TLS_KEY` **and** `LEANGEO_API_TOKEN` are all
102
+ set. Every RPC except `Health` then requires `authorization: Bearer <token>` metadata
103
+ (unary and streaming handlers alike); a missing or wrong token is `UNAUTHENTICATED`.
104
+ A token without TLS, or TLS without a token, is refused on a non-loopback address.
105
+ - `LEANGEO_ALLOW_INSECURE=1`: you state that the network is already trusted (for example,
106
+ an authenticating reverse proxy in front of a private listener).
107
+
108
+ The REST server also enforces this per request, on the address each connection actually
109
+ reached, because an ASGI app cannot see how it was bound: when started directly with
110
+ `uvicorn leangeo_server.serve:app --host 0.0.0.0` (or any non-loopback `--host`) and no
111
+ token, every request that arrives on a non-loopback address is refused with 403, except
112
+ `GET /health`; requests through `127.0.0.1` are served. Set `LEANGEO_API_TOKEN` (or
113
+ `LEANGEO_ALLOW_INSECURE=1`) to serve other addresses. A reverse proxy on the same host
114
+ that connects to `127.0.0.1` is served without a token.
115
+
116
+ The token also works on loopback: setting `LEANGEO_API_TOKEN` always enables the check.
117
+ IPv6 binds work for both servers (`LEANGEO_GRPC_HOST=::1`; the address is bracketed for
118
+ gRPC).
119
+
120
+ ---
121
+
122
+ ## 4. REST API
123
+
124
+ All bodies are JSON. Request models reject unknown fields with 422 (a misspelled
125
+ `filter` must not run a query unfiltered), and so are unknown query parameters
126
+ (`GET /count?colection=x` is 422, not a count of `default`). Every `collection` defaults
127
+ to `"default"`. Collection names are letters, digits, `-`, `_` and `.`, not starting with
128
+ `.` (the engine enforces this; anything else is 400).
129
+
130
+ | Method and path | Body / query | Response |
131
+ |---|---|---|
132
+ | `GET /health` | | `{"status": "ok", "ready": bool, "limits": {...}}` |
133
+ | `GET /count` | `?collection=` | `{"count": int}` |
134
+ | `GET /collections` | | `{"collections": [str, ...]}` |
135
+ | `POST /insert` | `{"latitude", "longitude", "metadata"?, "collection"?}` | `{"id": int}` |
136
+ | `POST /insert_batch` | `[{"latitude", "longitude", "metadata"?, "collection"?}, ...]` or `{"items": [...], "collection"?}` | `{"ids": [int, ...], "count": int}` |
137
+ | `POST /query` | `{"filter"?, "limit"?, "collection"?}` | `{"results": [...], "truncated": bool}` |
138
+ | `POST /query_radius` | `{"center_lat", "center_lon", "radius_meters", "filter"?, "limit"?, "collection"?}` | same |
139
+ | `POST /query_bbox` | `{"min_lat", "min_lon", "max_lat", "max_lon", "filter"?, "limit"?, "collection"?}` | same |
140
+ | `POST /query_polygon` | `{"coordinates": [[lon, lat], ...], "filter"?, "limit"?, "collection"?}` | same |
141
+ | `POST /exists` | `{"filter"?, "collection"?}` | `{"exists": bool}` |
142
+ | `POST /delete` | `{"filter", "collection"?}` (filter required) | `{"deleted": int}` |
143
+ | `POST /clear` | `{"collection", "confirm": true}` (both required) | `{"deleted": int}` |
144
+ | `POST /compact` | `{"collection"?}` | `{"reclaimed": int}` |
145
+ | `POST /persist` | none | `{"status": "ok"}` |
146
+ | `GET /verify` | `?collection=&flush=false` | `{"ok": bool, "errors": [str, ...]}` |
147
+
148
+ Result rows are `{"id": int, "latitude": float, "longitude": float, "metadata": {...}}`,
149
+ the same dicts the library returns, in the same order.
150
+
151
+ Notes:
152
+
153
+ - **Ids** are integers per collection, stable, and never reused.
154
+ - **`/insert_batch`** is atomic per collection: it makes one `engine.insert_batch` call
155
+ (one durable WAL record) per collection in the request, so one invalid point means none
156
+ of the batch is written. The 2.x server inserted point by point (one fsync each, and a
157
+ bad point left the earlier ones written). A batch that spans several collections is
158
+ validated in full (names, coordinates) before the first collection is written; after
159
+ that each collection commits on its own, so only a storage failure between two
160
+ collections can leave the earlier ones written. In the `{"items": ...}` form, an item
161
+ without its own `collection` uses the batch's.
162
+ - **`limit`** is optional on every read. Omitted means "everything, up to the server
163
+ cap"; `0` returns no rows (as in the library, after validating the filter); negative
164
+ is 400. See section 7 for `truncated`.
165
+ - **`collection`** defaults to `"default"` when omitted; an explicit `""` is refused with
166
+ 400, as the library refuses it.
167
+ - **`/query_bbox`** with `min_lon > max_lon` crosses the antimeridian.
168
+ - **`/delete`** refuses (400, nothing deleted) a filter that matches every point by its
169
+ shape (`{}`, `{"f": {"$nin": []}}`, ...) or because it tests a field no point has (a
170
+ misspelled name). `/clear` is the explicit wipe and needs `"confirm": true`.
171
+ - **`/compact`** reclaims the slots of deleted points now (it also runs in the background).
172
+ Ids do not change.
173
+ - **`/persist`** checkpoints every open collection; a failure is 500 with the detail.
174
+ - **`/verify`** recomputes the checksum of every committed region; `flush=true` checkpoints
175
+ first.
176
+
177
+ Example:
178
+
179
+ ```bash
180
+ curl -s localhost:8003/insert_batch -H 'content-type: application/json' -d '[
181
+ {"latitude": -23.55, "longitude": -46.63, "metadata": {"city": "SP"}},
182
+ {"latitude": -22.91, "longitude": -43.17, "metadata": {"city": "RJ"}}]'
183
+ curl -s localhost:8003/query_radius -H 'content-type: application/json' \
184
+ -d '{"center_lat": -23.0, "center_lon": -45.0, "radius_meters": 500000, "filter": {"city": "RJ"}}'
185
+ ```
186
+
187
+ ### Filters
188
+
189
+ Filters are the MongoDB-style contract shared by the Lean* engines (top-level metadata
190
+ fields; equality, `$eq $ne $gt $gte $lt $lte $in $nin $exists`, `$and $or $nor $not`, and
191
+ array membership). The engine validates them; an unknown operator, a
192
+ malformed shape, or a NaN/infinite number is 400 (`INVALID_ARGUMENT`), never silently
193
+ treated as "match all" or "match null". Over REST, `null` or an omitted `filter` means no
194
+ filter on reads.
195
+
196
+ ---
197
+
198
+ ## 5. Polygons
199
+
200
+ Polygon vertices are **(longitude, latitude)** pairs, GeoJSON order, everywhere in 3.0:
201
+
202
+ - library: `engine.query_polygon([(lon, lat), ...])`
203
+ - REST: `"coordinates": [[lon, lat], ...]`; objects `{"latitude": .., "longitude": ..}`
204
+ are accepted too and are unambiguous
205
+ - gRPC: `repeated LatLon coordinates`, a message with named `latitude` and `longitude`
206
+ fields; the server hands the engine `(longitude, latitude)`
207
+ - Python client: `query_polygon([(lon, lat), ...])`, or dicts with `"latitude"` and
208
+ `"longitude"` keys
209
+
210
+ **Migration from 2.x.** The 2.x `CLIENT.md` and `client.py` documented `(lat, lon)`
211
+ tuples, and the 2.x gRPC server built `(latitude, longitude)` tuples from `LatLon`
212
+ messages, which swapped the axes relative to the library. A 2.x Python caller that
213
+ passed `(lat, lon)` tuples to `LeanGeoClient.query_polygon` must swap them (or pass dicts).
214
+ A raw gRPC caller that filled `LatLon` by field name needs no change: those messages now
215
+ mean what their field names say. Containment is planar; split a polygon that crosses
216
+ the antimeridian yourself. Fewer than 3 vertices matches nothing. The vertex count is
217
+ capped (section 7).
218
+
219
+ ---
220
+
221
+ ## 6. gRPC API and the Python client
222
+
223
+ Service `leangeo.LeanGeo` (`leangeo_server/leangeo.proto`). Metadata, filters and result
224
+ rows travel as JSON strings (`metadata_json`, `filter_json`, `results_json`) so their JSON
225
+ types survive; coordinates are doubles. An empty `filter_json` means no filter on reads.
226
+ `collection` and `limit` are proto3 `optional` fields: unset `collection` means
227
+ `"default"` and an explicit `""` is `INVALID_ARGUMENT` (as in the library and REST);
228
+ unset `limit` means no limit of your own (the server cap still applies) and `limit = 0`
229
+ returns no rows after validating the request. All 2.x field numbers are unchanged and the
230
+ encoding is the same (a 2.x client never sends `""` or `0`, so its requests keep their
231
+ 2.x meaning); 3.0 only adds fields and RPCs.
232
+
233
+ | RPC | Request | Reply |
234
+ |---|---|---|
235
+ | `Health` | | `ready`, `limits_json` |
236
+ | `Count` | `collection` | `count` |
237
+ | `ListCollections` (3.0) | | `collections` |
238
+ | `Insert` | `point {latitude, longitude, metadata_json, collection}` | `id` |
239
+ | `InsertBatch` | `repeated Point points` | `repeated ids` (atomic per collection, as REST) |
240
+ | `Query` | `filter_json, limit, collection` | `results_json, truncated` |
241
+ | `QueryRadius` | `center_lat, center_lon, radius_meters, filter_json, collection, limit` | same |
242
+ | `QueryBbox` | `min_lat, min_lon, max_lat, max_lon, filter_json, collection, limit` | same |
243
+ | `QueryPolygon` | `repeated LatLon coordinates, filter_json, collection, limit` | same |
244
+ | `Exists` (3.0) | `filter_json, collection` | `exists` |
245
+ | `Delete` | `filter_json` (required), `collection` | `deleted` |
246
+ | `Clear` (3.0) | `collection, confirm` (must be true) | `deleted` |
247
+ | `Compact` (3.0) | `collection` | `reclaimed` |
248
+ | `Persist` | | `status` |
249
+ | `Verify` | `collection, flush` | `json` (`{"ok", "errors"}`) |
250
+
251
+ A `filter_json` or `metadata_json` that is not a JSON object (`[]`, `0`, `"x"`) is
252
+ `INVALID_ARGUMENT`.
253
+
254
+ ### LeanGeoClient
255
+
256
+ ```python
257
+ from leangeo_server.client import LeanGeoClient
258
+
259
+ with LeanGeoClient("localhost:50053", timeout=30) as c:
260
+ c.wait_ready()
261
+ pid = c.insert(-23.55, -46.63, {"city": "SP"})
262
+ ids = c.insert_batch([{"latitude": -22.91, "longitude": -43.17, "metadata": {"city": "RJ"}},
263
+ {"latitude": 40.71, "longitude": -74.0, "collection": "us"}])
264
+ near = c.query_radius(-23.0, -45.0, 500_000, filter={"city": "RJ"})
265
+ box = c.query_bbox(-40, -80, 10, -30, limit=100)
266
+ inside = c.query_polygon([(-50, -30), (-40, -30), (-40, -20), (-50, -20)]) # (lon, lat)
267
+ rows, truncated = c.query({"city": {"$in": ["SP", "RJ"]}}, with_truncated=True)
268
+ c.delete({"city": "SP"})
269
+ c.clear("us", confirm=True)
270
+ c.compact(); c.persist(); print(c.verify(), c.count(), c.list_collections(), c.limits())
271
+ ```
272
+
273
+ Constructor: `LeanGeoClient(target="localhost:50053", *, timeout=None, channel=None,
274
+ options=None, token=None, tls=False, allow_insecure_token=False)`.
275
+
276
+ - `token` defaults to `LEANGEO_API_TOKEN` from the environment and is sent as
277
+ `authorization: Bearer <token>`.
278
+ - `tls=True` uses the system roots; `tls=<PEM bytes>` trusts that root certificate (for a
279
+ self-signed server). The client refuses to send a token over a plaintext channel unless
280
+ `allow_insecure_token=True`.
281
+ - The channel is thread-safe and multiplexes concurrent RPCs: share one client across
282
+ threads.
283
+
284
+ Methods: `health()`, `limits()`, `count(collection)`, `list_collections()`,
285
+ `insert(lat, lon, metadata, collection)`, `insert_batch(points, collection)`,
286
+ `query(filter, limit, collection)`, `query_radius(center_lat, center_lon, radius_meters,
287
+ filter, collection, limit)`, `query_bbox(min_lat, min_lon, max_lat, max_lon, filter,
288
+ collection, limit)`, `query_polygon(coordinates, filter, collection, limit)`,
289
+ `exists(filter, collection)`, `delete(filter, collection)`, `clear(collection, *,
290
+ confirm)`, `compact(collection)`, `persist()`, `verify(collection, flush)`. The read
291
+ methods take `with_truncated=True` to return `(rows, truncated)`. Every method takes a
292
+ keyword `timeout` that overrides the default deadline.
293
+
294
+ Errors surface as `grpc.RpcError`:
295
+
296
+ ```python
297
+ import grpc
298
+ try:
299
+ c.delete({})
300
+ except grpc.RpcError as e:
301
+ print(e.code(), e.details()) # StatusCode.INVALID_ARGUMENT, "delete_by_metadata requires ..."
302
+ ```
303
+
304
+ ---
305
+
306
+ ## 7. Limits and guardrails
307
+
308
+ Static caps reject oversized requests before they reach the engine. They can be set
309
+ through the environment, but **only lowered**: a value above the default (or one that does
310
+ not parse) leaves the default in place.
311
+
312
+ | Variable | Default | Bounds |
313
+ |---|---|---|
314
+ | `LEANGEO_MAX_BODY_BYTES` | 128 MiB | One REST body (413 past it); the gRPC send/receive message size |
315
+ | `LEANGEO_MAX_BATCH` | 100000 | Points per `insert_batch` (429) |
316
+ | `LEANGEO_MAX_POLYGON_VERTICES` | 100000 | Vertices per polygon query (429) |
317
+ | `LEANGEO_HARD_MAX_K` | 1000000 | Absolute ceiling on rows per read reply |
318
+ | `LEANGEO_MIN_K` | 1000 | Floor of the dynamic cap while memory allows it |
319
+ | `LEANGEO_MEM_FRACTION` | 0.5 | Share of available RAM that read replies may use (0 < f <= 1) |
320
+ | `LEANGEO_BODY_MEM_FRACTION` | `LEANGEO_MEM_FRACTION` | Share of available RAM that request bodies being parsed may use |
321
+ | `LEANGEO_PER_RESULT_BYTES` | 4096 | Memory estimate per returned row |
322
+ | `LEANGEO_MEM_REFRESH_SECS` | 1.0 | How often the memory figures are re-read |
323
+ | `LEANGEO_MAX_OPEN_COLLECTIONS` | (soft fd limit - 256) / 8, at most 4096 | Collections held open at once; past it the least recently used idle ones are closed (checkpointed) and reopen on their next use |
324
+
325
+ **Dynamic result cap.** The rows one reply may hold are
326
+ `clamp(available_RAM * MEM_FRACTION / workers / PER_RESULT_BYTES)` between `MIN_K` and
327
+ `HARD_MAX_K`, where available RAM is the smaller of `MemAvailable` and the remaining
328
+ allowance of this process's cgroup (v1 or v2). Every read passes a limit to the engine,
329
+ so rows are bounded before they are materialized: when your `limit` is within the cap
330
+ it is used as is; otherwise the server asks the engine for `cap + 1` rows, returns
331
+ `cap`, and sets `truncated: true` if the extra row existed. The serialized rows must also
332
+ fit a byte budget (the same memory share, at most 8 x `MAX_BODY_BYTES`; for gRPC also at
333
+ most the message cap, `MAX_BODY_BYTES`); serialization stops at the first row that would
334
+ pass it and `truncated` is set. The byte budget is also passed to the engine, which stops
335
+ building rows once it is reached, so rows with large metadata never materialize the
336
+ whole row cap. `truncated: false` means you got every row
337
+ you asked for. To read more than the cap, narrow the query (a smaller box or radius, a
338
+ filter) or partition it. The current figures are in `GET /health` (`limits`) and gRPC
339
+ `Health.limits_json`.
340
+
341
+ **Body admission (REST).** A declared `Content-Length` past `MAX_BODY_BYTES` is refused
342
+ with 413 before any byte is read. A chunked body, or any body sent with
343
+ `Transfer-Encoding`, is counted as it arrives and refused once it passes the cap. Every
344
+ nonempty body also reserves its estimated parse memory (about 4x its size, more for
345
+ token-dense JSON such as `[0,0,0,...]`) against a budget the bodies in flight share; a
346
+ body that does not fit the memory free right now is refused with 429 and `Retry-After: 1`
347
+ before it is parsed. Refusals close the connection. Authentication runs before
348
+ admission.
349
+
350
+ **gRPC.** Messages larger than `MAX_BODY_BYTES` are refused by gRPC itself
351
+ (`RESOURCE_EXHAUSTED`). The JSON strings in a request (filters, metadata) reserve their
352
+ estimated parse memory from the same kind of budget while the RPC runs; one that does not
353
+ fit is `RESOURCE_EXHAUSTED`. `LEANGEO_GRPC_MAX_CONCURRENT_RPCS` optionally bounds running
354
+ plus queued RPCs.
355
+
356
+ **Open collections.** Each open collection holds about 6 to 8 file descriptors. At start
357
+ the servers raise the soft `RLIMIT_NOFILE` to the hard limit (at most 65536), and they keep
358
+ at most `LEANGEO_MAX_OPEN_COLLECTIONS` collections open, closing the least recently used
359
+ idle ones; a collection a request is using is never closed under it. Running out of
360
+ descriptors anyway is reported as 503 / `UNAVAILABLE`, not as corruption.
361
+
362
+ ---
363
+
364
+ ## 8. Errors
365
+
366
+ | Condition | REST | gRPC | Retry? |
367
+ |---|---|---|---|
368
+ | Bad input: coordinates out of range or non-finite, bad filter, bad collection name, `InvalidInputError`, `ValueError`, `TypeError`; guarded delete; `clear` without confirmation | 400 | `INVALID_ARGUMENT` | no |
369
+ | Missing or wrong bearer token | 401 | `UNAUTHENTICATED` | no |
370
+ | No token configured and the request reached a non-loopback address (REST, section 3) | 403 | n/a (the server refuses to start) | no |
371
+ | Unknown route, or the engine reports a missing item (`KeyError`) | 404 | `NOT_FOUND` | no |
372
+ | Body past `LEANGEO_MAX_BODY_BYTES` (also an invalid `Content-Length`: 400) | 413 | `RESOURCE_EXHAUSTED` | no |
373
+ | Unknown body field or query parameter, wrong JSON types | 422 | n/a | no |
374
+ | Over a guardrail (batch size, vertices, memory for parsing, `LEANGEO_GRPC_MAX_CONCURRENT_RPCS`) | 429 + `Retry-After: 1` | `RESOURCE_EXHAUSTED` | later |
375
+ | `StorageError` (I/O failure, corruption, a poisoned collection), or any unexpected error | 500 | `INTERNAL` | no, needs an operator |
376
+ | Engine not open, closed (`ClosedError`), or shutting down; out of file descriptors (EMFILE/ENFILE) | 503 + `Retry-After: 1` | `UNAVAILABLE` | yes |
377
+
378
+ Error details are bounded to 512 characters. REST 422 bodies list the location and
379
+ message of at most 16 validation errors and do not echo the submitted values.
380
+
381
+ ---
382
+
383
+ ## 9. Regenerating the gRPC stubs
384
+
385
+ Only needed after editing `leangeo_server/leangeo.proto` in a source checkout:
386
+
387
+ ```bash
388
+ pip install "grpcio-tools==1.82.1" # in a throwaway environment
389
+ python scripts/gen_grpc.py # rewrite leangeo_server/leangeo_pb2*.py
390
+ python scripts/gen_grpc.py --check # exit 1 if the checked-in stubs are stale
391
+ ```
392
+
393
+ The script refuses any other grpcio-tools version: newer tools stamp a newer minimum
394
+ runtime into the stubs. It compiles with the proto's own directory as the include root
395
+ (descriptor `leangeo.proto`, package `leangeo`), so the wire protocol does not depend on
396
+ where the package lives, and rewrites the generated `import leangeo_pb2` into a relative
397
+ import. `tests/test_server_packaging_lifecycle.py` runs the `--check` when grpcio-tools
398
+ 1.82.1 is importable or `LEANGEO_GRPC_TOOLS_PYTHON` points at a Python that has it.