chalk-remote-call-python 1.10.3__tar.gz → 1.10.5__tar.gz

{chalk_remote_call_python-1.10.3 → chalk_remote_call_python-1.10.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: chalk-remote-call-python
-Version: 1.10.3
+Version: 1.10.5
 Summary: Chalk remote call Python runtime interface client
 Author: Chalk AI, Inc.
 Project-URL: Homepage, https://chalk.ai
@@ -12,6 +12,7 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
 Classifier: Programming Language :: Python
 Classifier: Typing :: Typed
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
@@ -19,7 +20,7 @@ Classifier: Topic :: Scientific/Engineering :: Information Analysis
 Classifier: Topic :: Software Development :: Code Generators
 Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Python: <3.15,>=3.10
+Requires-Python: <3.16,>=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: pyarrow>=14.0.0
 Provides-Extra: dev
@@ -45,7 +46,7 @@ The server receives Arrow IPC record batches over a bidirectional gRPC stream, t
 ## Requirements
 
 - Python >= 3.10
-- Rust toolchain (for building the native extension)
+- Rust 1.99.0 (for building the native extension)
 - `pyarrow >= 14.0.0`
 
 ## Usage
@@ -116,6 +117,12 @@ invalid environment configuration, and exceptions escaping startup or the server
 emit a single ERROR record with the full traceback and exit with status 1.
 Calls to `serve()` from Python retain the embedding application's logging configuration.
 
+On Linux, server shutdown drains buffered function-call metrics before the runtime exits,
+including when the server returns an error. Draining waits up to 10 seconds; a timeout or
+publisher failure is logged as a warning.
+Routine drain messages are logged at DEBUG; set `RUST_LOG=info,_native::metrics=debug`
+to include them.
+
 Missing optional StatsD endpoints or a metrics-bus topic do not emit warnings.
 Failures initializing configured metrics sinks still warn. Routine customer
 StatsD configuration messages are emitted at DEBUG by the shared metrics library;
@@ -126,6 +133,8 @@ set `RUST_LOG=info,chalk_metrics::metrics::config=debug` to include them.
 | Variable | Description |
 |----------|-------------|
 | `CHALK_INPUT_ARGS` | Comma-separated list of column names (e.g. `x,y,z`). Renames incoming RecordBatch columns by index. If unset, the original column names are used. Reserved columns (below) are held out of this mapping and keep their own name. |
+| `CHALK_FNQ_CONSUMER_ENDPOINT` | Opt into DP API `FnqConsumerService` consumption (HTTP or HTTPS gRPC endpoint). Unset preserves direct Redis consumption. API errors never fall back to Redis. |
+| `CHALK_FUNCTION_QUEUE_TOKEN_FILE` | Controller-mounted working JWT, reopened per RPC to observe rotation. Defaults to `/var/run/secrets/chalk-function-queue/identity-token`. The shim does not exchange credentials. |
 | `CHALK_FNQ_QUEUE_PROTOCOL` | Self-consumer work transport: `list_v1` (default/backward compatible) or `stream_v1`. Deployment tooling sets this from immutable scaling-group revision metadata. |
 
 #### Reserved columns
@@ -216,6 +225,15 @@ uv pip install -e ".[dev]"
 pytest tests/ -v
 ```
 
+The Linux metrics shutdown regression tests use an in-memory bus publisher:
+
+```bash
+PYO3_PYTHON=python3 cargo test --manifest-path chalk-remote-call-rs/chalk-remote-call-server/Cargo.toml --no-default-features --lib metrics::tests
+```
+
+`--no-default-features` links Python into the test executable; wheel builds retain
+the default `extension-module` feature.
+
 ### Rust code
 
 Rust source lives in `chalk-remote-call-rs/`:
@@ -230,7 +248,10 @@ PYO3_PYTHON=python3 cargo check --manifest-path chalk-remote-call-rs/chalk-remot
 
 Handler failures retain their original exception type, message, traceback,
 exception chains, and exception groups. Runtime dispatch frames are excluded
-from the user traceback; full diagnostics are retained in server logs.
+from the user traceback, along with callers above the handler dispatch boundary.
+Handler frames and their downstream calls remain visible (except for Chalk SDK
+and marked generated dispatch frames); full diagnostics are retained in server
+logs and `internal_stacktrace`.
 
 All call modes carry a JSON payload with `chalk_remote_error: 1` and an
 `exception` object containing the `ChalkException` fields `kind`, `message`,
@@ -245,3 +266,23 @@ Payloads are bounded for gRPC header limits. Truncation is marked explicitly,
 with complete tracebacks kept in server logs. Exception objects and locals are
 never pickled or sent. Chalk SDK-generated handlers identify their dispatch
 frames, and notebook source maps identify the deployed snapshot's lines.
+
+### Fnq integration test
+
+On Linux, build the native extension and the paired `chalk-private` test harness:
+
+```bash
+# chalk-private, inside the Nix dev shell
+bazel build //rust-crates/chalk/dataplane-api-server:fnq_test_server
+
+# chalk-remote-call-python
+uv pip install -e ".[dev]" redis PyJWT cryptography
+FNQ_TEST_SERVER=/path/to/chalk-private/bazel-bin/rust-crates/chalk/dataplane-api-server/fnq_test_server \
+REDIS_URL=redis://127.0.0.1:6379 .venv/bin/pytest tests/test_fnq_api_e2e.py -v
+```
+
+Use an isolated Redis instance. The tests create temporary signing keys and start
+the production JWT layer and Fnq service, then exercise real native-shim execution,
+lease renewal, result writes, token rotation, and environment isolation. These tests
+skip unless `FNQ_TEST_SERVER` and `REDIS_URL` are set. Deploy the service before
+setting `CHALK_FNQ_CONSUMER_ENDPOINT` on a workload.

{chalk_remote_call_python-1.10.3 → chalk_remote_call_python-1.10.5}/README.md

@@ -7,7 +7,7 @@ The server receives Arrow IPC record batches over a bidirectional gRPC stream, t
 ## Requirements
 
 - Python >= 3.10
-- Rust toolchain (for building the native extension)
+- Rust 1.99.0 (for building the native extension)
 - `pyarrow >= 14.0.0`
 
 ## Usage
@@ -78,6 +78,12 @@ invalid environment configuration, and exceptions escaping startup or the server
 emit a single ERROR record with the full traceback and exit with status 1.
 Calls to `serve()` from Python retain the embedding application's logging configuration.
 
+On Linux, server shutdown drains buffered function-call metrics before the runtime exits,
+including when the server returns an error. Draining waits up to 10 seconds; a timeout or
+publisher failure is logged as a warning.
+Routine drain messages are logged at DEBUG; set `RUST_LOG=info,_native::metrics=debug`
+to include them.
+
 Missing optional StatsD endpoints or a metrics-bus topic do not emit warnings.
 Failures initializing configured metrics sinks still warn. Routine customer
 StatsD configuration messages are emitted at DEBUG by the shared metrics library;
@@ -88,6 +94,8 @@ set `RUST_LOG=info,chalk_metrics::metrics::config=debug` to include them.
 | Variable | Description |
 |----------|-------------|
 | `CHALK_INPUT_ARGS` | Comma-separated list of column names (e.g. `x,y,z`). Renames incoming RecordBatch columns by index. If unset, the original column names are used. Reserved columns (below) are held out of this mapping and keep their own name. |
+| `CHALK_FNQ_CONSUMER_ENDPOINT` | Opt into DP API `FnqConsumerService` consumption (HTTP or HTTPS gRPC endpoint). Unset preserves direct Redis consumption. API errors never fall back to Redis. |
+| `CHALK_FUNCTION_QUEUE_TOKEN_FILE` | Controller-mounted working JWT, reopened per RPC to observe rotation. Defaults to `/var/run/secrets/chalk-function-queue/identity-token`. The shim does not exchange credentials. |
 | `CHALK_FNQ_QUEUE_PROTOCOL` | Self-consumer work transport: `list_v1` (default/backward compatible) or `stream_v1`. Deployment tooling sets this from immutable scaling-group revision metadata. |
 
 #### Reserved columns
@@ -178,6 +186,15 @@ uv pip install -e ".[dev]"
 pytest tests/ -v
 ```
 
+The Linux metrics shutdown regression tests use an in-memory bus publisher:
+
+```bash
+PYO3_PYTHON=python3 cargo test --manifest-path chalk-remote-call-rs/chalk-remote-call-server/Cargo.toml --no-default-features --lib metrics::tests
+```
+
+`--no-default-features` links Python into the test executable; wheel builds retain
+the default `extension-module` feature.
+
 ### Rust code
 
 Rust source lives in `chalk-remote-call-rs/`:
@@ -192,7 +209,10 @@ PYO3_PYTHON=python3 cargo check --manifest-path chalk-remote-call-rs/chalk-remot
 
 Handler failures retain their original exception type, message, traceback,
 exception chains, and exception groups. Runtime dispatch frames are excluded
-from the user traceback; full diagnostics are retained in server logs.
+from the user traceback, along with callers above the handler dispatch boundary.
+Handler frames and their downstream calls remain visible (except for Chalk SDK
+and marked generated dispatch frames); full diagnostics are retained in server
+logs and `internal_stacktrace`.
 
 All call modes carry a JSON payload with `chalk_remote_error: 1` and an
 `exception` object containing the `ChalkException` fields `kind`, `message`,
@@ -207,3 +227,23 @@ Payloads are bounded for gRPC header limits. Truncation is marked explicitly,
 with complete tracebacks kept in server logs. Exception objects and locals are
 never pickled or sent. Chalk SDK-generated handlers identify their dispatch
 frames, and notebook source maps identify the deployed snapshot's lines.
+
+### Fnq integration test
+
+On Linux, build the native extension and the paired `chalk-private` test harness:
+
+```bash
+# chalk-private, inside the Nix dev shell
+bazel build //rust-crates/chalk/dataplane-api-server:fnq_test_server
+
+# chalk-remote-call-python
+uv pip install -e ".[dev]" redis PyJWT cryptography
+FNQ_TEST_SERVER=/path/to/chalk-private/bazel-bin/rust-crates/chalk/dataplane-api-server/fnq_test_server \
+REDIS_URL=redis://127.0.0.1:6379 .venv/bin/pytest tests/test_fnq_api_e2e.py -v
+```
+
+Use an isolated Redis instance. The tests create temporary signing keys and start
+the production JWT layer and Fnq service, then exercise real native-shim execution,
+lease renewal, result writes, token rotation, and environment isolation. These tests
+skip unless `FNQ_TEST_SERVER` and `REDIS_URL` are set. Deploy the service before
+setting `CHALK_FNQ_CONSUMER_ENDPOINT` on a workload.

{chalk_remote_call_python-1.10.3 → chalk_remote_call_python-1.10.5}/chalk-remote-call-rs/Cargo.lock

@@ -537,6 +537,8 @@ name = "chalk-remote-call-server"
 version = "0.1.0"
 dependencies = [
  "arrow",
+ "async-trait",
+ "base64 0.22.1",
  "bytes",
  "chalk-queue",
  "chalk-remote-call-proto",
@@ -1634,15 +1636,6 @@ dependencies = [
  "serde_core",
 ]
 
-[[package]]
-name = "indoc"
-version = "2.0.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
-dependencies = [
- "rustversion",
-]
-
 [[package]]
 name = "ipnet"
 version = "2.12.0"
@@ -1856,15 +1849,6 @@ version = "2.8.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
 
-[[package]]
-name = "memoffset"
-version = "0.9.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
-dependencies = [
- "autocfg",
-]
-
 [[package]]
 name = "metrics-generator"
 version = "0.1.0"
@@ -2402,35 +2386,32 @@ dependencies = [
 
 [[package]]
 name = "pyo3"
-version = "0.27.1"
+version = "0.29.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "37a6df7eab65fc7bee654a421404947e10a0f7085b6951bf2ea395f4659fb0cf"
+checksum = "700d18fa267b73b9b521fd7e13580e2f446916f176cacee1ab63fcc8191f1655"
 dependencies = [
- "indoc",
  "libc",
- "memoffset",
  "once_cell",
  "portable-atomic",
  "pyo3-build-config",
  "pyo3-ffi",
  "pyo3-macros",
- "unindent",
 ]
 
 [[package]]
 name = "pyo3-build-config"
-version = "0.27.1"
+version = "0.29.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f77d387774f6f6eec64a004eac0ed525aab7fa1966d94b42f743797b3e395afb"
+checksum = "7b3fc0c4d08f6bb10e71fe39dfb9e2f59c6eb6854e22ec8092f50c69a4499adb"
 dependencies = [
  "target-lexicon",
 ]
 
 [[package]]
 name = "pyo3-ffi"
-version = "0.27.1"
+version = "0.29.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2dd13844a4242793e02df3e2ec093f540d948299a6a77ea9ce7afd8623f542be"
+checksum = "dfc0b8e19df29aad7086cf977bb0c2a2f143e30567eb113e9cf72b62ca698330"
 dependencies = [
  "libc",
  "pyo3-build-config",
@@ -2438,9 +2419,9 @@ dependencies = [
 
 [[package]]
 name = "pyo3-macros"
-version = "0.27.1"
+version = "0.29.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "eaf8f9f1108270b90d3676b8679586385430e5c0bb78bb5f043f95499c821a71"
+checksum = "6100e8a4b5eba53afaa5ed078364851a0b2499c553a44c31026b929049b49dc6"
 dependencies = [
  "proc-macro2",
  "pyo3-macros-backend",
@@ -2450,13 +2431,12 @@ dependencies = [
 
 [[package]]
 name = "pyo3-macros-backend"
-version = "0.27.1"
+version = "0.29.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "70a3b2274450ba5288bc9b8c1b69ff569d1d61189d4bff38f8d22e03d17f932b"
+checksum = "6143877a16e82b5a727b7127ff4cd86858a24a28d745d72f43e6f227c7b1bdb3"
 dependencies = [
  "heck 0.5.0",
  "proc-macro2",
- "pyo3-build-config",
  "quote",
  "syn 2.0.117",
 ]
@@ -2773,10 +2753,37 @@ checksum = "3f56a14d1f48b391359b22f731fd4bd7e43c97f3c50eee276f3aa09c94784d3e"
 dependencies = [
  "log",
  "ring",
- "rustls-webpki",
+ "rustls-webpki 0.101.7",
  "sct",
 ]
 
+[[package]]
+name = "rustls"
+version = "0.23.43"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0283386ce02abc0151e1761d08802dfe86c173b0b494af5cbc086574e453da06"
+dependencies = [
+ "log",
+ "once_cell",
+ "ring",
+ "rustls-pki-types",
+ "rustls-webpki 0.103.15",
+ "subtle",
+ "zeroize",
+]
+
+[[package]]
+name = "rustls-native-certs"
+version = "0.8.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dab5152771c58876a2146916e53e35057e1a4dfa2b9df0f0305b07f611fdea4d"
+dependencies = [
+ "openssl-probe",
+ "rustls-pki-types",
+ "schannel",
+ "security-framework",
+]
+
 [[package]]
 name = "rustls-pemfile"
 version = "1.0.4"
@@ -2786,6 +2793,15 @@ dependencies = [
  "base64 0.21.7",
 ]
 
+[[package]]
+name = "rustls-pemfile"
+version = "2.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50"
+dependencies = [
+ "rustls-pki-types",
+]
+
 [[package]]
 name = "rustls-pki-types"
 version = "1.14.1"
@@ -2805,6 +2821,17 @@ dependencies = [
  "untrusted",
 ]
 
+[[package]]
+name = "rustls-webpki"
+version = "0.103.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2"
+dependencies = [
+ "ring",
+ "rustls-pki-types",
+ "untrusted",
+]
+
 [[package]]
 name = "rustversion"
 version = "1.0.22"
@@ -3068,6 +3095,12 @@ version = "1.2.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
 
+[[package]]
+name = "subtle"
+version = "2.6.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
+
 [[package]]
 name = "syn"
 version = "1.0.109"
@@ -3312,7 +3345,17 @@ version = "0.24.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "c28327cf380ac148141087fbfb9de9d7bd4e84ab5d2c28fbc911d753de8a7081"
 dependencies = [
- "rustls",
+ "rustls 0.21.12",
+ "tokio",
+]
+
+[[package]]
+name = "tokio-rustls"
+version = "0.26.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1729aa945f29d91ba541258c8df89027d5792d85a8841fb65e8bf0f4ede4ef61"
+dependencies = [
+ "rustls 0.23.43",
  "tokio",
 ]
 
@@ -3418,10 +3461,10 @@ dependencies = [
  "percent-encoding",
  "pin-project",
  "prost 0.12.6",
- "rustls",
- "rustls-pemfile",
+ "rustls 0.21.12",
+ "rustls-pemfile 1.0.4",
  "tokio",
- "tokio-rustls",
+ "tokio-rustls 0.24.1",
  "tokio-stream",
  "tower 0.4.13",
  "tower-layer",
@@ -3451,8 +3494,11 @@ dependencies = [
  "percent-encoding",
  "pin-project",
  "prost 0.13.5",
+ "rustls-native-certs",
+ "rustls-pemfile 2.2.0",
  "socket2 0.5.10",
  "tokio",
+ "tokio-rustls 0.26.4",
  "tokio-stream",
  "tower 0.4.13",
  "tower-layer",
@@ -3663,12 +3709,6 @@ version = "0.2.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
 
-[[package]]
-name = "unindent"
-version = "0.2.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
-
 [[package]]
 name = "untrusted"
 version = "0.9.0"

{chalk_remote_call_python-1.10.3 → chalk_remote_call_python-1.10.5}/chalk-remote-call-rs/chalk-remote-call-proto/src/gen/chalk.auth.v1.rs

@@ -1,5 +1,33 @@
 // @generated
 // This file is @generated by prost-build.
+/// Principal types that a permission may be assigned to directly. User-role
+/// assignment is managed separately; this metadata controls non-user principals.
+#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, ::prost::Enumeration)]
+#[repr(i32)]
+pub enum PermissionPrincipal {
+    Unspecified = 0,
+    ServiceToken = 1,
+}
+impl PermissionPrincipal {
+    /// String value of the enum field names used in the ProtoBuf definition.
+    ///
+    /// The values are not transformed in any way and thus are considered stable
+    /// (if the ProtoBuf definition does not change) and safe for programmatic use.
+    pub fn as_str_name(&self) -> &'static str {
+        match self {
+            PermissionPrincipal::Unspecified => "PERMISSION_PRINCIPAL_UNSPECIFIED",
+            PermissionPrincipal::ServiceToken => "PERMISSION_PRINCIPAL_SERVICE_TOKEN",
+        }
+    }
+    /// Creates an enum from field names used in the ProtoBuf definition.
+    pub fn from_str_name(value: &str) -> ::core::option::Option<Self> {
+        match value {
+            "PERMISSION_PRINCIPAL_UNSPECIFIED" => Some(Self::Unspecified),
+            "PERMISSION_PRINCIPAL_SERVICE_TOKEN" => Some(Self::ServiceToken),
+            _ => None,
+        }
+    }
+}
 #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, ::prost::Enumeration)]
 #[repr(i32)]
 pub enum Permission {
@@ -42,6 +70,10 @@ pub enum Permission {
     InternalWorkingTokenExchange = 34,
     BillingWrite = 35,
     QueryOfflineRead = 36,
+    InternalResourceShareView = 37,
+    InternalExchangeTokenRenew = 38,
+    AgentsWrite = 39,
+    AgentsRead = 40,
 }
 impl Permission {
     /// String value of the enum field names used in the ProtoBuf definition.
@@ -87,6 +119,10 @@ impl Permission {
             Permission::InternalWorkingTokenExchange => "PERMISSION_INTERNAL_WORKING_TOKEN_EXCHANGE",
             Permission::BillingWrite => "PERMISSION_BILLING_WRITE",
             Permission::QueryOfflineRead => "PERMISSION_QUERY_OFFLINE_READ",
+            Permission::InternalResourceShareView => "PERMISSION_INTERNAL_RESOURCE_SHARE_VIEW",
+            Permission::InternalExchangeTokenRenew => "PERMISSION_INTERNAL_EXCHANGE_TOKEN_RENEW",
+            Permission::AgentsWrite => "PERMISSION_AGENTS_WRITE",
+            Permission::AgentsRead => "PERMISSION_AGENTS_READ",
         }
     }
     /// Creates an enum from field names used in the ProtoBuf definition.
@@ -129,6 +165,10 @@ impl Permission {
             "PERMISSION_INTERNAL_WORKING_TOKEN_EXCHANGE" => Some(Self::InternalWorkingTokenExchange),
             "PERMISSION_BILLING_WRITE" => Some(Self::BillingWrite),
             "PERMISSION_QUERY_OFFLINE_READ" => Some(Self::QueryOfflineRead),
+            "PERMISSION_INTERNAL_RESOURCE_SHARE_VIEW" => Some(Self::InternalResourceShareView),
+            "PERMISSION_INTERNAL_EXCHANGE_TOKEN_RENEW" => Some(Self::InternalExchangeTokenRenew),
+            "PERMISSION_AGENTS_WRITE" => Some(Self::AgentsWrite),
+            "PERMISSION_AGENTS_READ" => Some(Self::AgentsRead),
             _ => None,
         }
     }