periplus-python-sdk 0.6.0__py3-none-any.whl → 0.7.0__py3-none-any.whl

{periplus_python_sdk-0.6.0.dist-info → periplus_python_sdk-0.7.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: periplus-python-sdk
-Version: 0.6.0
+Version: 0.7.0
 Summary: Read-only Python client for the public Periplus query API
 License-Expression: Apache-2.0
 Project-URL: Repository, https://github.com/elei-io/periplus
@@ -43,7 +43,7 @@ The client reuses HTTP connections; close it with a context manager or `close()`
 Install the notebook integration from PyPI:
 
 ```sh
-uv add "periplus-python-sdk[notebook]>=0.6.0"
+uv add "periplus-python-sdk[notebook]>=0.7.0"
 ```
 
 In a Python setup cell, create a SQLAlchemy engine:
@@ -62,7 +62,7 @@ FROM public_v1.capture
 LIMIT 10
 ```
 
-Marimo displays the result as a table. Expand **pp → public_v1** in Data Sources
+Marimo displays the result as a table. Expand **pp → periplus → public_v1** in Data Sources
 to discover views and expand a view to load its columns for SQL completion.
 Discovery uses bounded `SHOW TABLES` and `DESCRIBE` through the same public API;
 no internal catalogue or storage credentials are used. Truncated discovery fails
@@ -82,7 +82,7 @@ captures = mo.sql(
 ```
 
 Set `mode="experimental"` for the experimental service. Omit the URL to use
-`PERIPLUS_PUBLIC_URL`. Optional `timeout=140` and `schema_version="public_v1"`
+`PERIPLUS_PUBLIC_URL`. Optional `timeout=620` and `schema_version="public_v1"`
 arguments configure the client deadline and public schema. Run `pp.dispose()` when finished. This is a read-only
 SQLAlchemy dialect for textual SQL and reflection, not a writable ORM backend.
 Each statement has its own server snapshot; SQLAlchemy transaction blocks do not
@@ -90,7 +90,7 @@ provide a shared snapshot or rollback. The adapter makes no transaction requests
 
 A complete notebook is in `examples/notebook.py`. The integration is tested with
 marimo 0.24.1 and SQLAlchemy 2.x. SQLAlchemy is included in the standard SDK install; the `notebook` extra adds
-marimo. Existing marimo environments only need `uv add "periplus-python-sdk>=0.6.0"`.
+marimo. Existing marimo environments only need `uv add "periplus-python-sdk>=0.7.0"`.
 The returned object is a standard SQLAlchemy Engine, also usable with pandas and
 ordinary Python scripts. Engine creation is lazy; the first query opens a connection.
 
@@ -112,14 +112,17 @@ with connect("https://periplus.dev", mode="stable") as connection:
 Connections expose `cursor`, `execute`, `close`, and context managers. Cursors
 support `execute`, `fetchone`, `fetchmany`, `fetchall`, iteration, and close.
 Use positional `?` parameters. Decimal and temporal parameters are sent as
-strings; use explicit SQL casts. Binary and nested parameters are not supported
-by this adapter. Fetching only consumes the bounded result already received;
-it never issues pagination or retries. Connections/cursors are not thread-shared.
+strings; use explicit SQL casts. One-dimensional lists/tuples of these scalar values are supported; use an explicit array cast
+for empty or typed lists. Binary, mappings, and nested collection parameters are unsupported. Fetching consumes incremental batches from one HTTP response and one source snapshot;
+it never issues pagination or retries. Close a cursor early to stop delivery. Connections/cursors are not thread-shared.
 `commit()` is a no-op; `rollback()` and `executemany()` are unsupported.
 
-`cursor.result` preserves the original query response. `connection.last_result`
+`cursor.result` preserves stream metadata and completion information without retaining consumed rows. `connection.last_result`
 also retains it after marimo closes a cursor; a new execution clears it first.
-Truncation emits `periplus_sdk.dbapi.TruncationWarning` and sets `rowcount` to -1.
+Truncation raises `OperationalError` with `code="result_limit"` when fetching reaches the terminal frame.
+`rowcount` stays -1 until completion, then reports the delivered count. `result.complete` means the
+terminal frame arrived; inspect `result.truncated` separately when opting into partial data.
+Use `allow_partial=True` on the engine or connection only when incomplete results are intentional.
 DB-API failures use the standard exception hierarchy in `periplus_sdk.dbapi`;
 HTTP errors retain `status_code`, `code`, and `retry_after_seconds`.
 
@@ -185,7 +188,7 @@ Use `aclose()` when managing an async client's lifetime explicitly.
   20-second server deadline. Always inspect `truncated`. The SDK does not silently fetch more rows or retry.
 - `ApiError` exposes `status_code`, safe `code`, and `retry_after_seconds` when supplied.
   `TransportError` means HTTP failed; `ResponseError` means a malformed successful response.
-  The client timeout defaults to 140 seconds and can be set with `timeout=`. A timeout or local
+  The client timeout defaults to 620 seconds and can be set with `timeout=`. A timeout or local
   cancellation does not guarantee server cancellation. Redirects are not followed automatically.
 - Preparation and execution are attributed to `sdk` in the existing private query history.
   Original SQL and parameters are retained for 30 days; result rows are not stored. Recording is
@@ -196,10 +199,10 @@ Use `aclose()` when managing an async client's lifetime explicitly.
 Install the public-v1 client from PyPI:
 
 ```sh
-python -m pip install "periplus-python-sdk>=0.6.0"
+python -m pip install "periplus-python-sdk>=0.7.0"
 ```
 
-Version 0.6.0 supports the current public-v1 contract. For production, configure
+Version 0.6.1 supports the current public-v1 contract. For production, configure
 `PERIPLUS_PUBLIC_URL=https://periplus.dev`; no API token is required.
 Run the installed package against an available public app:
 
@@ -211,11 +214,11 @@ PERIPLUS_PUBLIC_URL=http://localhost:8080 python packages/periplus-python-sdk/ex
 
 Repository CI publishes immutable releases from tags named
 `periplus-python-sdk-v<version>`. The tag must exactly match the static version
-in `pyproject.toml`; for example, version `0.6.0` is released with:
+in `pyproject.toml`; for example, version `0.7.0` is released with:
 
 ```sh
-git tag periplus-python-sdk-v0.6.0
-git push origin periplus-python-sdk-v0.6.0
+git tag periplus-python-sdk-v0.7.0
+git push origin periplus-python-sdk-v0.7.0
 ```
 
 PyPI publishing uses Trusted Publishing rather than a stored API token. The
@@ -225,10 +228,74 @@ that GitHub environment with required reviewers before the first release.
 
 ## Public v1
 
-Install the updated SDK from PyPI with `python -m pip install "periplus-python-sdk>=0.6.0"`. The previously published 0.2.0 release predates this contract. `prepare` and `execute` accept keyword-only `schema_version="public_v1"` (the default); responses preserve `schema_version` separately from `source_snapshot`. Unavailable versions are rejected by the server.
+Install the updated SDK from PyPI with `python -m pip install "periplus-python-sdk>=0.7.0"`. The previously published 0.2.0 release predates this contract. `prepare` and `execute` accept keyword-only `schema_version="public_v1"` (the default); responses preserve `schema_version` separately from `source_snapshot`. Unavailable versions are rejected by the server.
 
 ## License
 
 Copyright (c) 2026 Ekku Leivonen (elei.io). Licensed under [Apache-2.0](LICENSE);
 see [NOTICE](NOTICE). The server and other repository packages have separate
 licensing described in the root LICENSING.md.
+
+## Python filtering back into marimo SQL
+
+The existing engine streams by default. For SQL → Python filtering → SQL, bind the
+filtered IDs to another engine variable; it shares the original connection pool and
+creates no server-side table or upload. Marimo discovers it like any other SQLAlchemy engine.
+
+```python
+pp = sql_api.create_engine("https://periplus.dev")
+
+# In a Python cell, after filtering your initial SQL dataframe:
+selected = sql_api.bind(pp, content_ids=qualified_pages["content_id"].to_list())
+
+# In the next SQL cell (engine: selected):
+raw_elements = mo.sql(
+    """
+    SELECT content_id, node_index, parent_index, tag,
+           trim(text_direct) AS text, attributes['class'] AS elem_class
+    FROM public_v1.html_element
+    WHERE content_id IN (SELECT unnest(CAST(:content_ids AS VARCHAR[])))
+      AND text_direct IS NOT NULL AND trim(text_direct) <> ''
+    """,
+    engine=selected,
+)
+```
+
+Keep SQL strings plain: `:content_ids` is a bound parameter, not an f-string.
+An empty list selects no elements; duplicate IDs do not multiply rows. Re-running the
+binding cell snapshots the new Python list into its engine options. Bindings only apply
+to matching SQL placeholders, so catalogue discovery remains available. Explicit
+SQLAlchemy execution parameters take precedence over engine bindings.
+
+Marimo collects the stream into a local dataframe, which must fit notebook memory.
+Its dataframe conversion currently infers types and may lose empty/all-null column
+information; the DB-API cursor always exposes server column names and SQL types.
+Initial SQL results and subsequent element results both reject truncation by default.
+Each separate SQL execution has its own snapshot; Python filtering does not pin the
+first query's snapshot across the round trip.
+
+## Consume batches without a dataframe
+
+```python
+with Client("https://periplus.dev") as client:
+    with client.stream("SELECT content_id FROM public_v1.prose") as stream:
+        for rows in stream:
+            process_batch(rows)
+        print(stream.result.row_count, stream.result.source_snapshot)
+        print(stream.result.limits)
+```
+
+Batches retain JSON wire values with SQL types in `stream.result.types`. DB-API fetching
+performs scalar Python decoding. The synchronous `Client.stream()` raises on truncation
+unless `allow_partial=True`; missing completion, malformed frames, timeouts, and broken
+connections always fail. Do not treat already-consumed batches as a complete dataset
+until iteration finishes successfully. Use context managers when stopping early.
+
+Admin Query limits controls requests (default 4 MiB, ceiling 16 MiB), parameter values
+(default 100,000, ceiling 1,000,000), results (ceiling 10 million rows and 1 GiB), and
+execution duration (ceiling 600 seconds). Parameter counting includes collection containers
+and their values. Existing result/duration defaults remain 1,000 rows, 8 MiB, and 20 seconds.
+Operators must raise these settings for larger workloads. No SDK parameter grants a larger
+server budget. Stream metadata includes effective limits; input rejections name the budget.
+Deploy the API, query service, public gateway and SDK together after applying Alembic
+revision `20260911_0016`. Ingress must permit the configured request size and stream duration.

periplus_python_sdk-0.7.0.dist-info/RECORD

@@ -0,0 +1,16 @@
+periplus_python_sdk-0.7.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
+periplus_python_sdk-0.7.0.dist-info/licenses/NOTICE,sha256=bhbYSqcUB3U_P1-XzloiT81JGniqoYaRLxNkQ1Pm9MQ,52
+periplus_sdk/__init__.py,sha256=WimXYlPB6tCimBO4VSwhcp00dwSL87jMmMuQ4-kINfM,546
+periplus_sdk/client.py,sha256=bqIq2bmM_bnwf0ZnMN5kTXTctnEz7ITJUAfZohGsRIQ,7544
+periplus_sdk/dbapi.py,sha256=wyhtLfpgBZGxhC9rA53DNfQ7j8HegZkVp7mRq98bFSU,12256
+periplus_sdk/errors.py,sha256=rB1n-v8Hc2tu2dtHivz-MlqsCoRC5pTcWogTfM7SMLw,855
+periplus_sdk/py.typed,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
+periplus_sdk/sql_api.py,sha256=o5kLzwLim3VQ5oZfRyPUGCDr7RWmDidtktGLyBdkjQk,1884
+periplus_sdk/sqlalchemy.py,sha256=skyN2R5nIiWK55uBuxOHjeGRAPb8uPxm0f8nU0IIPqw,4836
+periplus_sdk/stream.py,sha256=pPXWikBWnOUZuHfBRXx08swBDMWJ4Gayx6azaz_wUCU,5197
+periplus_sdk/types.py,sha256=0eCwWev-Vrk0mRCIkot3Z_-dWWL3gpjljV4GyXEf8lk,1191
+periplus_python_sdk-0.7.0.dist-info/METADATA,sha256=6HTXHT4WnFKYbfB6MVc1Duxni8niJ3JQ7hP7w2GYkiA,14052
+periplus_python_sdk-0.7.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+periplus_python_sdk-0.7.0.dist-info/entry_points.txt,sha256=Pr14L_7AhLinq-4qDxB4awFVubrvR1BEfkaFVRpdCbU,73
+periplus_python_sdk-0.7.0.dist-info/top_level.txt,sha256=o41t5TzwgoxzSmbKoP6olWW1FyEAGWVYTjeoKadBK40,13
+periplus_python_sdk-0.7.0.dist-info/RECORD,,

periplus_sdk/client.py

@@ -79,7 +79,7 @@ def _payload(sql: str, parameters: Sequence[JsonValue] | None, schema_version: s
 class Client:
     """Reusable synchronous public query client. Close it or use a with block."""
 
-    def __init__(self, base_url: str | None = None, *, timeout: float = 140, mode: Literal["stable", "experimental"] = "stable"):
+    def __init__(self, base_url: str | None = None, *, timeout: float = 620, mode: Literal["stable", "experimental"] = "stable"):
         if mode not in {"stable", "experimental"}:
             raise ConfigurationError("mode must be stable or experimental.")
         self._query_path = "api/query/experimental/" if mode == "experimental" else "api/query/"
@@ -107,6 +107,25 @@ class Client:
     def execute(self, sql: str, parameters: Sequence[JsonValue] | None = None, *, schema_version: str = "public_v1") -> QueryResult:
         return self._request("POST", "exec", QueryResult, json=_payload(sql, parameters, schema_version))
 
+    def stream(self, sql: str, parameters: Sequence[JsonValue] | None = None, *,
+               schema_version: str = "public_v1", allow_partial: bool = False):
+        """Stream batches from one snapshot; use as a context manager for early exit."""
+        from .stream import MEDIA_TYPE, QueryStream
+        try:
+            request = self._http.build_request("POST", self._query_path + "exec",
+                headers={"accept": MEDIA_TYPE}, json=_payload(sql, parameters, schema_version))
+            response = self._http.send(request, stream=True)
+            try:
+                if not response.is_success:
+                    response.read()
+                    _decode(response, QueryResult)
+                return QueryStream(response, allow_partial=allow_partial)
+            except BaseException:
+                response.close()
+                raise
+        except httpx.RequestError:
+            raise TransportError("Could not open the public query stream.") from None
+
     def helpers(self) -> QueryHelpers:
         return self._request("GET", "helpers", QueryHelpers)
 
@@ -114,7 +133,7 @@ class Client:
 class AsyncClient:
     """Reusable asynchronous public query client. Use an async with block."""
 
-    def __init__(self, base_url: str | None = None, *, timeout: float = 140, mode: Literal["stable", "experimental"] = "stable"):
+    def __init__(self, base_url: str | None = None, *, timeout: float = 620, mode: Literal["stable", "experimental"] = "stable"):
         if mode not in {"stable", "experimental"}:
             raise ConfigurationError("mode must be stable or experimental.")
         self._query_path = "api/query/experimental/" if mode == "experimental" else "api/query/"

periplus_sdk/dbapi.py

@@ -1,7 +1,7 @@
 """Read-only DB-API 2.0 connection over the public query API.
 
-Each execute is an independent server snapshot. Fetching consumes a bounded local
-result, never a remote cursor. Connections and cursors must not be shared by threads.
+Each execute is an independent server snapshot. Fetching consumes a bounded stream.
+Connections and cursors must not be shared by threads.
 """
 from __future__ import annotations
 
@@ -11,11 +11,10 @@ from collections.abc import Sequence
 from datetime import date, datetime, time
 from decimal import Decimal
 from typing import Any, Literal
-import warnings
 
 from .client import Client
 from .errors import ApiError, ConfigurationError, PeriplusError, ResponseError, TransportError
-from .types import QueryResult
+from .stream import StreamResult
 
 apilevel = "2.0"
 threadsafety = 1
@@ -26,10 +25,6 @@ class Warning(builtins.Warning):
     """DB-API warning."""
 
 
-class TruncationWarning(Warning):
-    """The server returned only part of the query result."""
-
-
 class Error(PeriplusError):
     """Base DB-API error; API failures preserve their safe error attributes."""
 
@@ -147,7 +142,11 @@ def _parameter(value: Any) -> Any:
         return value
     if isinstance(value, (Decimal, date, time)):
         return str(value) if isinstance(value, Decimal) else value.isoformat()
-    raise ProgrammingError("Parameters must be scalar JSON values, Decimal, date, time or datetime; use explicit SQL casts for typed strings.")
+    if isinstance(value, (list, tuple)):
+        if any(isinstance(item, (list, tuple, dict)) for item in value):
+            raise ProgrammingError("Collection parameters must be one-dimensional lists of scalar values.")
+        return [_parameter(item) for item in value]
+    raise ProgrammingError("Parameters must be scalar values or scalar lists; use explicit SQL casts for typed values.")
 
 
 class Connection:
@@ -155,16 +154,18 @@ class Connection:
 
     dialect = "duckdb"
 
-    def __init__(self, base_url: str | None = None, *, timeout: float = 140,
+    def __init__(self, base_url: str | None = None, *, timeout: float = 620,
                  mode: Literal["stable", "experimental"] = "stable",
-                 schema_version: str = "public_v1"):
+                 schema_version: str = "public_v1", allow_partial: bool = False):
         try:
             self._client = Client(base_url, timeout=timeout, mode=mode)
         except ConfigurationError as exc:
             raise InterfaceError(str(exc)) from exc
         self.schema_version = schema_version
+        self.allow_partial = allow_partial
+        self._cursors = set()
         self.closed = False
-        self.last_result: QueryResult | None = None
+        self.last_result: StreamResult | None = None
 
     def _check(self) -> None:
         if self.closed:
@@ -172,7 +173,9 @@ class Connection:
 
     def cursor(self) -> Cursor:
         self._check()
-        return Cursor(self)
+        cursor = Cursor(self)
+        self._cursors.add(cursor)
+        return cursor
 
     def execute(self, operation: str, parameters: Sequence[Any] | None = None) -> Cursor:
         cursor = self.cursor()
@@ -192,6 +195,8 @@ class Connection:
 
     def close(self) -> None:
         if not self.closed:
+            for cursor in list(self._cursors):
+                cursor.close()
             self._client.close()
             self.closed = True
             self.last_result = None
@@ -204,25 +209,26 @@ class Connection:
         self.close()
 
 
-def connect(base_url: str | None = None, *, timeout: float = 140,
+def connect(base_url: str | None = None, *, timeout: float = 620,
             mode: Literal["stable", "experimental"] = "stable",
-            schema_version: str = "public_v1") -> Connection:
-    return Connection(base_url, timeout=timeout, mode=mode, schema_version=schema_version)
+            schema_version: str = "public_v1", allow_partial: bool = False) -> Connection:
+    return Connection(base_url, timeout=timeout, mode=mode, schema_version=schema_version, allow_partial=allow_partial)
 
 
 class Cursor:
-    """A buffered result. Metadata stays on result after rows are consumed."""
+    """Incremental cursor. Metadata stays available without retaining consumed rows."""
 
     arraysize = 1
 
     def __init__(self, connection: Connection):
         self.connection = connection
         self.closed = False
-        self.result: QueryResult | None = None
+        self.result: StreamResult | None = None
         self.description: list[tuple[Any, ...]] | None = None
         self.rowcount = -1
         self._rows: list[tuple[Any, ...]] = []
         self._position = 0
+        self._stream = None
 
     def _check(self, *, result: bool = False) -> None:
         self.connection._check()
@@ -233,6 +239,9 @@ class Cursor:
 
     def execute(self, operation: str, parameters: Sequence[Any] | None = None) -> Cursor:
         self._check()
+        if self._stream is not None:
+            self._stream.close()
+        self._stream = None
         self.result, self.description, self.rowcount = None, None, -1
         self._rows, self._position = [], 0
         self.connection.last_result = None
@@ -242,7 +251,9 @@ class Cursor:
             raise ProgrammingError("Use a positional parameter sequence with ? placeholders.")
         values = [_parameter(v) for v in parameters] if parameters is not None else []
         try:
-            result = self.connection._client.execute(operation, values, schema_version=self.connection.schema_version)
+            self._stream = self.connection._client.stream(operation, values, schema_version=self.connection.schema_version,
+                                                          allow_partial=self.connection.allow_partial)
+            result = self._stream.result
         except ApiError as exc:
             error = ProgrammingError if exc.code == "sql_invalid" else OperationalError
             raise error(str(exc), status_code=exc.status_code, code=exc.code,
@@ -251,26 +262,37 @@ class Cursor:
             raise OperationalError(str(exc)) from exc
         except ResponseError as exc:
             raise InterfaceError(str(exc)) from exc
-        if len(result.columns) != len(result.types) or any(len(r) != len(result.columns) for r in result.rows):
-            raise InterfaceError("Query columns, types and rows have inconsistent widths.")
-        try:
-            rows = [tuple(_value(v, t) for v, t in zip(row, result.types, strict=True)) for row in result.rows]
-        except (ValueError, TypeError, ArithmeticError) as exc:
-            raise DataError("Query value does not match its SQL type.") from exc
         self.result = self.connection.last_result = result
         self.description = [(name, kind, None, None, None, None, None)
                             for name, kind in zip(result.columns, result.types, strict=True)]
-        self._rows = rows
-        self.rowcount = -1 if result.truncated else len(rows)
-        if result.truncated:
-            warnings.warn(f"Periplus returned a truncated result ({len(rows)} rows); inspect connection.last_result or cursor.result. Fetching does not retrieve additional rows.",
-                          TruncationWarning, stacklevel=2)
         return self
 
+    def _batch(self):
+        try:
+            batch = next(self._stream)
+            self._rows = [tuple(_value(v, t) for v, t in zip(row, self.result.types, strict=True)) for row in batch]
+            self._position = 0
+            return True
+        except StopIteration:
+            self.rowcount = -1 if self.result.truncated else self.result.row_count
+            self._rows, self._position = [], 0
+            return False
+        except ApiError as exc:
+            raise OperationalError(str(exc), status_code=exc.status_code, code=exc.code,
+                                   retry_after_seconds=exc.retry_after_seconds) from exc
+        except TransportError as exc:
+            raise OperationalError(str(exc)) from exc
+        except ResponseError as exc:
+            raise InterfaceError(str(exc)) from exc
+        except (ValueError, TypeError, ArithmeticError) as exc:
+            self._stream.close()
+            raise DataError("Query value does not match its SQL type.") from exc
+
     def fetchone(self) -> tuple[Any, ...] | None:
         self._check(result=True)
-        if self._position == len(self._rows):
-            return None
+        while self._position == len(self._rows):
+            if not self._batch():
+                return None
         row = self._rows[self._position]
         self._position += 1
         return row
@@ -280,14 +302,17 @@ class Cursor:
         size = self.arraysize if size is None else size
         if not isinstance(size, int) or size < 0:
             raise ProgrammingError("Fetch size must be a non-negative integer.")
-        end = min(self._position + size, len(self._rows))
-        rows = self._rows[self._position:end]
-        self._position = end
+        rows = []
+        for _ in range(size):
+            row = self.fetchone()
+            if row is None:
+                break
+            rows.append(row)
         return rows
 
     def fetchall(self) -> list[tuple[Any, ...]]:
         self._check(result=True)
-        return self.fetchmany(len(self._rows) - self._position)
+        return list(self)
 
     def executemany(self, operation: str, seq_of_parameters: Any) -> None:
         self._check()
@@ -300,6 +325,9 @@ class Cursor:
         self._check()
 
     def close(self) -> None:
+        if self._stream is not None:
+            self._stream.close()
+        self.connection._cursors.discard(self)
         self.closed = True
         self._rows = []
         self.result = None

periplus_sdk/sql_api.py

@@ -3,7 +3,7 @@ from __future__ import annotations
 
 from typing import Literal
 
-from sqlalchemy import create_engine as _create_engine
+from sqlalchemy import create_engine as _create_engine, event
 from sqlalchemy.engine import Engine, URL
 
 
@@ -11,15 +11,37 @@ def create_engine(
     base_url: str | None = None,
     *,
     mode: Literal["stable", "experimental"] = "stable",
-    timeout: float = 140,
+    timeout: float = 620,
     schema_version: str = "public_v1",
+    allow_partial: bool = False,
 ) -> Engine:
     """Create a SQLAlchemy engine recognized by marimo and other SQL tools.
 
     The public URL defaults to PERIPLUS_PUBLIC_URL. Connections are opened lazily;
     dispose the engine when finished. Each query uses an independent server snapshot.
     """
-    return _create_engine(
-        URL.create("periplus", database=schema_version),
-        connect_args={"base_url": base_url, "mode": mode, "timeout": timeout},
+    engine = _create_engine(
+        URL.create("periplus", database="periplus"),
+        connect_args={"base_url": base_url, "mode": mode, "timeout": timeout, "schema_version": schema_version, "allow_partial": allow_partial},
     )
+
+    event.listen(engine, "before_execute", _parameters, retval=True)
+    return engine
+
+
+def _parameters(connection, clauseelement, multiparams, params, execution_options):
+    bindings = execution_options.get("periplus_parameters", {})
+    if bindings and not multiparams:
+        names = clauseelement.compile().params
+        params = {**{name: value for name, value in bindings.items() if name in names}, **params}
+    return clauseelement, multiparams, params
+
+
+def bind(engine: Engine, **parameters) -> Engine:
+    """Create a marimo-discoverable engine with named SQL parameters for this cell.
+
+    Uses the original engine's pool. No query, upload, or server state is created.
+    Write :name placeholders in SQL cells; lists can be CAST(:ids AS VARCHAR[]).
+    """
+    from .dbapi import _parameter
+    return engine.execution_options(periplus_parameters={name: _parameter(value) for name, value in parameters.items()})

periplus_sdk/sqlalchemy.py

@@ -60,10 +60,10 @@ class PeriplusDialect(default.DefaultDialect):
 
     def create_connect_args(self, url):
         if url.username or url.password or url.host or url.port:
-            raise exc.ArgumentError("Use periplus:///public_v1 with base_url and mode in connect_args.")
+            raise exc.ArgumentError("Use periplus:///periplus with base_url and mode in connect_args.")
         if url.query:
             raise exc.ArgumentError("Pass connection options in connect_args, not URL query parameters.")
-        return [], {"schema_version": url.database or "public_v1"}
+        return [], {}
 
     def initialize(self, connection):
         self.default_schema_name = connection.connection.dbapi_connection.schema_version
@@ -91,11 +91,11 @@ class PeriplusDialect(default.DefaultDialect):
 
     def _complete(self, result):
         raw = result.cursor.result
-        if raw.truncated:
-            result.close()
-            raise exc.InvalidRequestError("Catalogue discovery was truncated by public query limits; refusing an incomplete schema.")
         try:
-            return result.fetchall()
+            rows = result.fetchall()
+            if raw.truncated:
+                raise exc.InvalidRequestError("Catalogue discovery was truncated by public query limits; refusing an incomplete schema.")
+            return rows
         finally:
             result.close()
 

periplus_sdk/stream.py

@@ -0,0 +1,134 @@
+"""Incremental query frames. EOF is never a successful completion marker."""
+from __future__ import annotations
+
+import json
+from typing import Literal
+
+import httpx
+from pydantic import BaseModel, Field, JsonValue, ValidationError
+
+from .errors import ApiError, ResponseError, TransportError
+from .types import PreparedQuery
+
+MEDIA_TYPE = "application/x-ndjson"
+
+
+class StreamResult(PreparedQuery):
+    columns: list[str]
+    types: list[str]
+    source_snapshot: int = Field(ge=0)
+    limits: dict[str, int]
+    complete: bool = False
+    truncated: bool = False
+    truncation_reason: Literal["max_rows", "max_result_bytes"] | None = None
+    row_count: int = 0
+    result_bytes: int = 0
+    elapsed_ms: float = 0
+
+
+class Rows(BaseModel):
+    type: Literal["rows"]
+    rows: list[list[JsonValue]]
+
+
+class Completion(BaseModel):
+    type: Literal["complete"]
+    row_count: int = Field(ge=0)
+    result_bytes: int = Field(ge=0)
+    truncated: bool
+    truncation_reason: Literal["max_rows", "max_result_bytes"] | None
+    elapsed_ms: float = Field(ge=0)
+
+
+class QueryStream:
+    """One response, consumed a batch at a time. Close early to cancel delivery."""
+
+    def __init__(self, response: httpx.Response, *, allow_partial: bool = False):
+        self.response = response
+        self.allow_partial = allow_partial
+        self.closed = False
+        self._count = 0
+        self._lines = response.iter_lines()
+        try:
+            if response.headers.get("content-type", "").split(";")[0] != MEDIA_TYPE:
+                raise ResponseError("Expected a streaming query response.")
+            frame = self._frame()
+            if frame.get("type") != "metadata":
+                raise ResponseError("Query stream is missing metadata.")
+            self.result = StreamResult.model_validate(frame)
+            if len(self.result.columns) != len(self.result.types):
+                raise ResponseError("Query columns and types have inconsistent widths.")
+        except ValidationError:
+            self.close()
+            raise ResponseError("Invalid query stream metadata.") from None
+        except BaseException:
+            self.close()
+            raise
+
+    def _frame(self):
+        try:
+            for line in self._lines:
+                if not line.strip():
+                    continue
+                frame = json.loads(line)
+                if not isinstance(frame, dict):
+                    raise ResponseError("Invalid query stream frame.")
+                if frame.get("type") == "error":
+                    raise ApiError(frame.get("detail", "Query stream failed."),
+                                   status_code=frame.get("status", 500), code=frame.get("code"))
+                return frame
+        except httpx.RequestError:
+            raise TransportError("Query stream interrupted; the result is incomplete.") from None
+        except ValueError:
+            raise ResponseError("Invalid query stream frame.") from None
+        raise ResponseError("Query stream ended without completion; the result is incomplete.")
+
+    def __iter__(self):
+        return self
+
+    def __next__(self) -> list[list[JsonValue]]:
+        if self.closed:
+            if not self.result.complete:
+                raise ResponseError("Query stream closed before completion; the result is incomplete.")
+            if self.result.truncated and not self.allow_partial:
+                raise ResponseError("Query result exceeded its budget and is incomplete.")
+            raise StopIteration
+        try:
+            frame = self._frame()
+            if frame.get("type") == "rows":
+                rows = Rows.model_validate(frame).rows
+                if any(len(row) != len(self.result.columns) for row in rows):
+                    raise ResponseError("Query row has an inconsistent width.")
+                self._count += len(rows)
+                return rows
+            completion = Completion.model_validate(frame)
+            if completion.row_count != self._count:
+                raise ResponseError("Query completion row count does not match delivered rows.")
+            for name, value in completion.model_dump(exclude={"type"}).items():
+                setattr(self.result, name, value)
+            self.result.complete = True
+            self.close()
+            if completion.truncated and not self.allow_partial:
+                budget = completion.truncation_reason
+                limit = self.result.limits.get(budget, "unknown")
+                raise ApiError(f"Query result is incomplete: {budget} ({limit}) reached after {self._count} rows. "
+                               "Narrow the query or ask the administrator to increase this budget.",
+                               status_code=422, code="result_limit")
+            raise StopIteration
+        except ValidationError:
+            self.close()
+            raise ResponseError("Invalid query stream frame.") from None
+        except BaseException:
+            self.close()
+            raise
+
+    def close(self):
+        if not self.closed:
+            self.closed = True
+            self.response.close()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *args):
+        self.close()

periplus_sdk/types.py

@@ -29,6 +29,9 @@ class QueryResult(PreparedQuery):
     truncated: bool
     elapsed_ms: float
     source_snapshot: int = Field(ge=0)
+    row_count: int = Field(ge=0)
+    result_bytes: int = Field(ge=0)
+    truncation_reason: Literal["max_rows", "max_result_bytes"] | None = None
 
 
 class HelperField(BaseModel):

periplus_python_sdk-0.6.0.dist-info/RECORD

@@ -1,15 +0,0 @@
-periplus_python_sdk-0.6.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
-periplus_python_sdk-0.6.0.dist-info/licenses/NOTICE,sha256=bhbYSqcUB3U_P1-XzloiT81JGniqoYaRLxNkQ1Pm9MQ,52
-periplus_sdk/__init__.py,sha256=WimXYlPB6tCimBO4VSwhcp00dwSL87jMmMuQ4-kINfM,546
-periplus_sdk/client.py,sha256=ZwMmVJKNF-FrGh_qwiQ5myhu8XJodb_96ohkqK47yDA,6557
-periplus_sdk/dbapi.py,sha256=lK9ZROaMKXm26wvC7DCKywm3qwSvqeHa7zkktnJVP80,11244
-periplus_sdk/errors.py,sha256=rB1n-v8Hc2tu2dtHivz-MlqsCoRC5pTcWogTfM7SMLw,855
-periplus_sdk/py.typed,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
-periplus_sdk/sql_api.py,sha256=iBJt6hK_m84bDMBRZUDNwz20gi07gAxqWpm-NWWsqSo,850
-periplus_sdk/sqlalchemy.py,sha256=axFpEMnMP9uSxbsxzVNSUHNh41q2NruSX8W_5E8F59Q,4877
-periplus_sdk/types.py,sha256=PTdJO6BYTY97dBd3HMEZ52k6P9s0cMvidjpnc8zUpy4,1045
-periplus_python_sdk-0.6.0.dist-info/METADATA,sha256=UBcdaDmCCo0O0ISVlljy947LFFQIawX36y4dOioN4mo,10199
-periplus_python_sdk-0.6.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
-periplus_python_sdk-0.6.0.dist-info/entry_points.txt,sha256=Pr14L_7AhLinq-4qDxB4awFVubrvR1BEfkaFVRpdCbU,73
-periplus_python_sdk-0.6.0.dist-info/top_level.txt,sha256=o41t5TzwgoxzSmbKoP6olWW1FyEAGWVYTjeoKadBK40,13
-periplus_python_sdk-0.6.0.dist-info/RECORD,,